2024-04-25T10:02:27.309 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:02:27.309 DEBUG:teuthology.run:Teuthology command: teuthology --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055 --verbose --owner scheduled_neesingh@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/orig.config.yaml 2024-04-25T10:02:27.354 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:02:27.447 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673055' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v2 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAGc3Klt6LAMmrSgcILM7JlDUR+7I9TzjabaLkKZ+p7si4+Q6RZ4su76s/U8y+/rRkDjRR+hQziHpTKBinxObcs= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPotNQ3sAkA2cyuLvk+qDvMYmy+4nXZwLHv3e422ePhSbZ/e6QWEjE42uWyP0eTRlxxaIR5AURDNCMY0RhIfjEI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:02:27.447 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T10:02:27.448 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T10:02:27.448 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:02:27.450 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:02:27.473 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T10:02:27.473 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:02:27.473 INFO:teuthology.packaging:ref: None 2024-04-25T10:02:27.473 INFO:teuthology.packaging:tag: None 2024-04-25T10:02:27.473 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T10:02:27.473 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:02:27.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T10:02:27.685 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:02:27.686 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:02:27.693 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:02:27.693 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:02:27.701 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:02:27.716 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:02:27.728 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:02:27.754 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:56:39.639315', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIILbvb3w6D8ilNMjpCqiu6b34hc/4I924cl5TSP/iVSS'} 2024-04-25T10:02:27.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:56:39.640774', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILt6WwfWHqj0BPFNrKRqp6OGPvlg9vgl1/oRfN4Qa68u'} 2024-04-25T10:02:27.777 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:02:27.788 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T10:02:27.788 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T10:02:27.788 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:02:27.864 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe7eaae50d0>, signals=[15]) 2024-04-25T10:02:27.865 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:02:27.877 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:02:27.878 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:02:27.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:02:27.963 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:02:27.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:02:28.043 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:02:28.052 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-25T10:02:28.057 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-25T10:02:28.057 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-25T10:02:28.103 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T10:02:28.103 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Ubuntu" 2024-04-25T10:02:28.103 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="22.04" 2024-04-25T10:02:28.103 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_CODENAME=jammy 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:ID=ubuntu 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE=debian 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T10:02:28.104 INFO:teuthology.orchestra.run.smithi045.stdout:UBUNTU_CODENAME=jammy 2024-04-25T10:02:28.105 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-25T10:02:28.129 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-04-25T10:02:28.135 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-04-25T10:02:28.135 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-04-25T10:02:28.182 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T10:02:28.182 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="22.04" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=jammy 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T10:02:28.183 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=jammy 2024-04-25T10:02:28.183 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-04-25T10:02:28.203 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:02:28.217 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:02:28.229 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:02:28.229 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:02:28.232 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:02:28.236 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:02:28.247 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:02:28.248 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:02:28.280 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:02:28.288 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:02:28.340 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:02:28.359 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T10:02:28.359 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T10:02:28.359 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T10:02:28.360 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:02:28.360 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:02:28.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:28.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T10:02:28.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:28.369 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T10:02:28.540 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:02:28.540 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T10:02:28.540 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-25T10:02:28.546 INFO:teuthology.orchestra.run.smithi045.stdout:5.15.0-56-generic 2024-04-25T10:02:28.546 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:02:28.547 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T10:02:28.547 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-04-25T10:02:28.547 DEBUG:teuthology.orchestra.run.smithi045:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T10:02:28.561 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:02:28.562 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T10:02:28.562 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-04-25T10:02:28.567 INFO:teuthology.orchestra.run.smithi117.stdout:5.15.0-56-generic 2024-04-25T10:02:28.567 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:02:28.568 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T10:02:28.568 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-04-25T10:02:28.568 DEBUG:teuthology.orchestra.run.smithi117:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T10:02:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T10:02:28.638 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi045.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: git config --global init.defaultBranch 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi045.stderr:hint: git branch -m 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi045.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T10:02:28.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi117.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi117.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: git config --global init.defaultBranch 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi117.stderr:hint: git branch -m 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi117.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T10:02:28.780 DEBUG:teuthology.orchestra.run.smithi117:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T10:02:28.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T10:02:28.846 DEBUG:teuthology.orchestra.run.smithi117:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T10:02:28.856 DEBUG:teuthology.orchestra.run.smithi045:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T10:02:28.903 DEBUG:teuthology.orchestra.run.smithi117:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T10:03:09.569 INFO:teuthology.orchestra.run.smithi117.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T10:03:09.569 INFO:teuthology.orchestra.run.smithi117.stderr: * [new branch] main -> origin/main 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T10:03:09.605 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T10:03:09.611 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T10:03:09.611 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T10:03:09.611 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi117.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T10:03:13.879 INFO:teuthology.orchestra.run.smithi117.stderr:Updating files: 45% (1528/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2009/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2811/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T10:03:13.884 INFO:teuthology.orchestra.run.smithi117.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T10:03:13.906 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T10:03:13.906 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:03:14.051 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T10:03:14.052 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T10:03:14.055 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi117.front.sepia.ceph.com to complete... 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi045.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T10:03:15.248 INFO:teuthology.orchestra.run.smithi045.stderr: * [new branch] main -> origin/main 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T10:03:15.284 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T10:03:15.290 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T10:03:15.290 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T10:03:15.290 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T10:03:15.292 INFO:teuthology.orchestra.run.smithi045.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T10:03:15.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25 10:03:16 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T10:03:15.981 INFO:teuthology.orchestra.run.smithi117.stderr:FINISHED --2024-04-25 10:03:16-- 2024-04-25T10:03:15.982 INFO:teuthology.orchestra.run.smithi117.stderr:Total wall clock time: 1.9s 2024-04-25T10:03:15.982 INFO:teuthology.orchestra.run.smithi117.stderr:Downloaded: 1 files, 43M in 1.8s (24.2 MB/s) 2024-04-25T10:03:15.986 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T10:03:15.986 DEBUG:teuthology.orchestra.run.smithi117:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T10:03:16.249 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T10:03:18.238 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T10:03:18.242 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T10:03:18.359 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T10:03:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:Updating files: 44% (1500/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 58% (1980/3381) Updating files: 59% (1995/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 68% (2330/3381) Updating files: 69% (2333/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 82% (2783/3381) Updating files: 83% (2807/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T10:03:19.793 INFO:teuthology.orchestra.run.smithi045.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T10:03:19.816 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T10:03:19.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:03:19.996 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T10:03:19.996 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T10:03:19.999 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi045.front.sepia.ceph.com to complete... 2024-04-25T10:03:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25 10:03:21 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T10:03:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:FINISHED --2024-04-25 10:03:21-- 2024-04-25T10:03:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:Total wall clock time: 1.7s 2024-04-25T10:03:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:Downloaded: 1 files, 43M in 1.6s (27.1 MB/s) 2024-04-25T10:03:21.756 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T10:03:21.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T10:03:22.015 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T10:03:22.465 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T10:03:22.579 INFO:teuthology.orchestra.run.smithi117.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:23.990 INFO:teuthology.orchestra.run.smithi045.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T10:03:23.993 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T10:03:24.157 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T10:03:28.551 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi045.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:44.993 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub' 2024-04-25T10:03:45.007 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T10:03:45.031 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-04-25T10:03:45.333 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:45.643 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T10:03:45.651 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T10:03:45.711 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T10:03:45.716 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T10:03:45.870 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T10:03:45.870 INFO:teuthology.orchestra.run.smithi117.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T10:03:45.870 INFO:teuthology.orchestra.run.smithi117.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T10:03:45.909 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-04-25T10:03:46.113 DEBUG:teuthology.orchestra.run.smithi117:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T10:03:46.120 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T10:03:46.120 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T10:03:46.121 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:46.139 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:46.139 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T10:03:46.161 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T10:03:46.162 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T10:03:46.162 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T10:03:46.165 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T10:03:46.166 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T10:03:46.167 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T10:03:46.167 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T10:03:46.167 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T10:03:46.167 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T10:03:46.167 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T10:03:46.223 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T10:03:46.224 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T10:03:46.225 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T10:03:46.226 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T10:03:46.227 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T10:03:46.228 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T10:03:46.229 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T10:03:46.230 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T10:03:46.231 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T10:03:46.232 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T10:03:46.233 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T10:03:46.234 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T10:03:46.235 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T10:03:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T10:03:46.237 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T10:03:46.238 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T10:03:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T10:03:46.240 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T10:03:46.241 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T10:03:46.242 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T10:03:46.243 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T10:03:46.244 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T10:03:46.245 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T10:03:46.246 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T10:03:46.247 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T10:03:46.248 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T10:03:46.249 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T10:03:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T10:03:46.251 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T10:03:46.252 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T10:03:46.253 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T10:03:46.254 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T10:03:46.255 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T10:03:46.256 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T10:03:46.257 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T10:03:46.258 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T10:03:46.259 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T10:03:46.260 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T10:03:46.261 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T10:03:46.262 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T10:03:46.263 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T10:03:46.264 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T10:03:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T10:03:46.325 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T10:03:46.326 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T10:03:46.327 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T10:03:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T10:03:46.329 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T10:03:46.330 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T10:03:46.331 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T10:03:46.332 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T10:03:46.333 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T10:03:46.334 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T10:03:46.336 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T10:03:46.337 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T10:03:46.338 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T10:03:46.343 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T10:03:46.344 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T10:03:46.345 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T10:03:46.346 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T10:03:46.347 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T10:03:46.348 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T10:03:46.349 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T10:03:46.350 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T10:03:46.351 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T10:03:46.352 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T10:03:46.353 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T10:03:46.354 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T10:03:46.355 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T10:03:46.356 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T10:03:46.357 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T10:03:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T10:03:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T10:03:46.360 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T10:03:46.361 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T10:03:46.362 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T10:03:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T10:03:46.364 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T10:03:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T10:03:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T10:03:46.367 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T10:03:46.368 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T10:03:46.369 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T10:03:46.370 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T10:03:46.371 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T10:03:46.372 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T10:03:46.373 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T10:03:46.374 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T10:03:46.375 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T10:03:46.375 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T10:03:46.420 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T10:03:46.421 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T10:03:46.422 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T10:03:46.423 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T10:03:46.424 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T10:03:46.425 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T10:03:46.426 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T10:03:46.427 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T10:03:46.428 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T10:03:46.429 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T10:03:46.430 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T10:03:46.431 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T10:03:46.432 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T10:03:46.433 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T10:03:46.434 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T10:03:46.435 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T10:03:46.436 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T10:03:46.437 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T10:03:46.438 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T10:03:46.439 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T10:03:46.440 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T10:03:46.441 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T10:03:46.442 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T10:03:46.443 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T10:03:46.444 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T10:03:46.445 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T10:03:46.446 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T10:03:46.447 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T10:03:46.448 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T10:03:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T10:03:46.450 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T10:03:46.451 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T10:03:46.452 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T10:03:46.453 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T10:03:46.454 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T10:03:46.455 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T10:03:46.456 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T10:03:46.457 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T10:03:46.458 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T10:03:46.459 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T10:03:46.460 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T10:03:46.460 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T10:03:46.460 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T10:03:46.502 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T10:03:46.503 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T10:03:46.504 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T10:03:46.505 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T10:03:46.506 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T10:03:46.507 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T10:03:46.508 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T10:03:46.509 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T10:03:46.510 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T10:03:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T10:03:46.512 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T10:03:46.513 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T10:03:46.514 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T10:03:46.515 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T10:03:46.516 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T10:03:46.517 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T10:03:46.518 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T10:03:46.519 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T10:03:46.520 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T10:03:46.521 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T10:03:46.522 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T10:03:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T10:03:46.524 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T10:03:46.525 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T10:03:46.526 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T10:03:46.527 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T10:03:46.528 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T10:03:46.529 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T10:03:46.530 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T10:03:46.531 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T10:03:46.532 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T10:03:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T10:03:46.534 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T10:03:46.535 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T10:03:46.536 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T10:03:46.537 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T10:03:46.538 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T10:03:46.539 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T10:03:46.540 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T10:03:46.545 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:46.545 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:46.545 DEBUG:teuthology.orchestra.run.smithi117:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi117.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi117.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi117.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T10:03:46.556 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T10:03:46.556 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:46.556 DEBUG:teuthology.orchestra.run.smithi117:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T10:03:46.600 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi117.front.sepia.ceph.com to complete... 2024-04-25T10:03:46.962 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub' 2024-04-25T10:03:46.963 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T10:03:46.992 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-04-25T10:03:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:47.205 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:47.431 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T10:03:47.437 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T10:03:47.480 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T10:03:47.615 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T10:03:47.615 INFO:teuthology.orchestra.run.smithi117.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T10:03:47.615 INFO:teuthology.orchestra.run.smithi117.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T10:03:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-04-25T10:03:48.766 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-04-25T10:03:51.362 INFO:teuthology.orchestra.run.smithi045.stderr:Sourcing file `/etc/default/grub' 2024-04-25T10:03:51.376 INFO:teuthology.orchestra.run.smithi045.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T10:03:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-04-25T10:03:51.743 INFO:teuthology.orchestra.run.smithi045.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:51.757 INFO:teuthology.orchestra.run.smithi045.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.051 INFO:teuthology.orchestra.run.smithi045.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T10:03:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T10:03:52.128 INFO:teuthology.orchestra.run.smithi045.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T10:03:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T10:03:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T10:03:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T10:03:52.300 INFO:teuthology.orchestra.run.smithi045.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T10:03:52.352 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-04-25T10:03:52.556 DEBUG:teuthology.orchestra.run.smithi045:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T10:03:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T10:03:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T10:03:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.579 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.579 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.600 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T10:03:52.601 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T10:03:52.602 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T10:03:52.603 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T10:03:52.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T10:03:52.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T10:03:52.604 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T10:03:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T10:03:52.665 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T10:03:52.666 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T10:03:52.667 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T10:03:52.668 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T10:03:52.669 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T10:03:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T10:03:52.671 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T10:03:52.672 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T10:03:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T10:03:52.674 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T10:03:52.675 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T10:03:52.676 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T10:03:52.677 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T10:03:52.678 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T10:03:52.679 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T10:03:52.680 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T10:03:52.681 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T10:03:52.682 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T10:03:52.683 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T10:03:52.684 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T10:03:52.685 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T10:03:52.686 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T10:03:52.687 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T10:03:52.688 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T10:03:52.689 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T10:03:52.690 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T10:03:52.691 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T10:03:52.692 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T10:03:52.693 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T10:03:52.694 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T10:03:52.695 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T10:03:52.696 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T10:03:52.697 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T10:03:52.698 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T10:03:52.699 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T10:03:52.700 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T10:03:52.701 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T10:03:52.702 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T10:03:52.703 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T10:03:52.704 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T10:03:52.705 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T10:03:52.706 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T10:03:52.707 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T10:03:52.708 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T10:03:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T10:03:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T10:03:52.711 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T10:03:52.712 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T10:03:52.713 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T10:03:52.714 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T10:03:52.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T10:03:52.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T10:03:52.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T10:03:52.715 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T10:03:52.762 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T10:03:52.763 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T10:03:52.764 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T10:03:52.765 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T10:03:52.766 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T10:03:52.767 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T10:03:52.768 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T10:03:52.769 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T10:03:52.770 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T10:03:52.771 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T10:03:52.772 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T10:03:52.773 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T10:03:52.774 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T10:03:52.775 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T10:03:52.776 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T10:03:52.777 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T10:03:52.778 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T10:03:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T10:03:52.780 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T10:03:52.781 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T10:03:52.782 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T10:03:52.783 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T10:03:52.784 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T10:03:52.785 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T10:03:52.786 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T10:03:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T10:03:52.788 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T10:03:52.789 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T10:03:52.790 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T10:03:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T10:03:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T10:03:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T10:03:52.794 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T10:03:52.795 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T10:03:52.796 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T10:03:52.797 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T10:03:52.798 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T10:03:52.799 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T10:03:52.800 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T10:03:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T10:03:52.802 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T10:03:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T10:03:52.804 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T10:03:52.805 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T10:03:52.806 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T10:03:52.844 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T10:03:52.844 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T10:03:52.844 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T10:03:52.844 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T10:03:52.844 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T10:03:52.845 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T10:03:52.846 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T10:03:52.847 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T10:03:52.849 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T10:03:52.850 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T10:03:52.852 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T10:03:52.858 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T10:03:52.865 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T10:03:52.866 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T10:03:52.868 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T10:03:52.869 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T10:03:52.870 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T10:03:52.871 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T10:03:52.872 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T10:03:52.873 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T10:03:52.874 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T10:03:52.875 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T10:03:52.876 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T10:03:52.877 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T10:03:52.878 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T10:03:52.879 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T10:03:52.880 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T10:03:52.881 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T10:03:52.882 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T10:03:52.883 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T10:03:52.884 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T10:03:52.918 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T10:03:52.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T10:03:52.918 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T10:03:52.918 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T10:03:52.919 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T10:03:52.920 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T10:03:52.921 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T10:03:52.922 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T10:03:52.923 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T10:03:52.924 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T10:03:52.925 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T10:03:52.926 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T10:03:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T10:03:52.928 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T10:03:52.929 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T10:03:52.930 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T10:03:52.931 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T10:03:52.932 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T10:03:52.933 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T10:03:52.934 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T10:03:52.935 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T10:03:52.936 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T10:03:52.937 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T10:03:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T10:03:52.939 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T10:03:52.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T10:03:52.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T10:03:52.940 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T10:03:52.943 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T10:03:52.944 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T10:03:52.945 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T10:03:52.946 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T10:03:52.947 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T10:03:52.948 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T10:03:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T10:03:52.950 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T10:03:52.951 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T10:03:52.952 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T10:03:52.953 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T10:03:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T10:03:52.955 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T10:03:52.956 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T10:03:52.957 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T10:03:52.957 INFO:teuthology.orchestra.run.smithi045.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T10:03:52.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T10:03:52.957 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T10:03:52.962 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.962 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.962 DEBUG:teuthology.orchestra.run.smithi045:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T10:03:52.969 INFO:teuthology.orchestra.run.smithi045.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T10:03:52.969 INFO:teuthology.orchestra.run.smithi045.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T10:03:52.969 INFO:teuthology.orchestra.run.smithi045.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T10:03:52.970 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T10:03:52.970 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:52.970 DEBUG:teuthology.orchestra.run.smithi045:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T10:03:53.012 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi045.front.sepia.ceph.com to complete... 2024-04-25T10:03:53.374 INFO:teuthology.orchestra.run.smithi045.stderr:Sourcing file `/etc/default/grub' 2024-04-25T10:03:53.375 INFO:teuthology.orchestra.run.smithi045.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T10:03:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-04-25T10:03:53.590 INFO:teuthology.orchestra.run.smithi045.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T10:03:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi045.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T10:03:53.876 INFO:teuthology.orchestra.run.smithi045.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T10:03:53.881 INFO:teuthology.orchestra.run.smithi045.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T10:03:54.004 INFO:teuthology.orchestra.run.smithi045.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T10:03:54.004 INFO:teuthology.orchestra.run.smithi045.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T10:03:54.004 INFO:teuthology.orchestra.run.smithi045.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T10:03:54.024 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-04-25T10:04:18.790 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T10:04:18.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:04:18.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:04:25.130 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T10:04:25.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T10:04:25.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:04:37.283 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-04-25T10:04:43.683 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-25T10:04:46.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:04:46.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:04:49.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-04-25T10:04:52.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T10:04:52.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:05:01.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:05:01.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:05:19.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-04-25T10:05:34.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:05:34.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:05:35.199 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-04-25T10:05:36.548 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:05:36.548 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T10:05:36.548 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-04-25T10:05:36.556 INFO:teuthology.orchestra.run.smithi117.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T10:05:36.557 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T10:05:36.557 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T10:05:36.557 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T10:05:37.558 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:52.690 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T10:06:04.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T10:06:04.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:06:04.952 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T10:06:05.588 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T10:06:05.588 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-25T10:06:05.636 INFO:teuthology.orchestra.run.smithi045.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T10:06:05.637 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T10:06:05.637 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T10:06:05.637 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T10:06:06.638 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:06.639 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T10:06:06.648 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T10:06:06.648 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:06:06.652 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T10:06:06.661 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T10:06:06.675 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T10:06:06.684 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T10:06:06.685 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:06:06.703 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T10:06:06.727 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T10:06:06.738 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T10:06:06.738 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:06:06.768 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T10:06:06.853 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:06:06.867 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:06:06.870 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:06:06.893 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T10:06:06.896 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T10:06:06.904 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T10:06:06.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:06:06.908 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T10:06:06.929 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T10:06:06.942 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T10:06:06.943 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:06:06.972 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T10:06:06.982 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:06:07.029 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:06:07.082 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:06:07.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:06:07.153 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T10:06:07.162 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T10:06:07.219 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:06:07.219 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T10:06:07.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-04-25T10:06:07.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-04-25T10:06:07.412 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T10:06:07.420 INFO:teuthology.task.internal:Starting timer... 2024-04-25T10:06:07.420 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T10:06:07.436 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T10:06:07.447 DEBUG:teuthology.task.selinux:Excluding smithi045: OS 'ubuntu' does not support SELinux 2024-04-25T10:06:07.447 DEBUG:teuthology.task.selinux:Excluding smithi117: OS 'ubuntu' does not support SELinux 2024-04-25T10:06:07.447 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T10:06:07.448 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T10:06:07.448 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T10:06:07.448 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T10:06:07.460 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T10:06:07.469 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T10:06:07.594 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T10:06:07.611 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T10:06:07.613 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T10:13:57.594 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-04-25T10:13:57.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T10:13:57.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:13:57.687 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-25T10:13:57.797 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T10:13:57.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:13:57.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:13:57.888 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-04-25T10:13:58.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-25T10:13:58.014 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T10:13:58.026 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T10:13:58.027 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:13:58.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:13:58.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T10:13:58.031 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Command line: ntpd -gq 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: ---------------------------------------------------- 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: corporation. Support and training for ntp-4 are 2024-04-25T10:13:58.067 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: available at https://www.nwtime.org/support 2024-04-25T10:13:58.068 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: ---------------------------------------------------- 2024-04-25T10:13:58.068 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: proto: precision = 0.043 usec (-24) 2024-04-25T10:13:58.068 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: basedate set to 2022-02-04 2024-04-25T10:13:58.069 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: gps base set to 2022-02-06 (week 2196) 2024-04-25T10:13:58.069 INFO:teuthology.orchestra.run.smithi045.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:13:58.069 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:13:58.070 INFO:teuthology.orchestra.run.smithi045.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:13:58.072 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:13:58.072 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T10:13:58.072 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T10:13:58.072 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T10:13:58.072 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listen normally on 3 ens1f0 172.21.15.45:123 2024-04-25T10:13:58.073 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listen normally on 4 lo [::1]:123 2024-04-25T10:13:58.073 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12be%4]:123 2024-04-25T10:13:58.073 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:13:58 ntpd[17112]: Listening on routing socket on fd #22 for interface updates 2024-04-25T10:13:58.094 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T10:13:58.095 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Command line: ntpd -gq 2024-04-25T10:13:58.095 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: ---------------------------------------------------- 2024-04-25T10:13:58.095 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T10:13:58.095 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T10:13:58.095 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: corporation. Support and training for ntp-4 are 2024-04-25T10:13:58.096 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: available at https://www.nwtime.org/support 2024-04-25T10:13:58.096 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: ---------------------------------------------------- 2024-04-25T10:13:58.097 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: proto: precision = 0.042 usec (-24) 2024-04-25T10:13:58.098 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: basedate set to 2022-02-04 2024-04-25T10:13:58.098 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: gps base set to 2022-02-06 (week 2196) 2024-04-25T10:13:58.098 INFO:teuthology.orchestra.run.smithi117.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:13:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T10:13:58.099 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:13:58.099 INFO:teuthology.orchestra.run.smithi117.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listen normally on 3 enp3s0f1 172.21.15.117:123 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listen normally on 4 lo [::1]:123 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4463%5]:123 2024-04-25T10:13:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:13:58 ntpd[17192]: Listening on routing socket on fd #22 for interface updates 2024-04-25T10:13:59.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:13:59 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:13:59 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:13:59 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:13:59 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:13:59 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:13:59.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:13:59 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:13:59.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:13:59.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:13:59 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:00.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:00.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:00.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:00.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:00.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:00.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:00.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:00.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:00 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:01.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:01 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:01 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:01 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:01 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:01 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:01.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:01 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:01.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:01.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:01 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:02 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:03 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:03 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:03 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:03 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:03 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:03 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:03.097 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:03 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:04 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:25 Apr 10:14:05 ntpd[17112]: ntpd: time slew -0.001809 s 2024-04-25T10:14:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:ntpd: time slew -0.001809s 2024-04-25T10:14:05.069 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T10:14:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:25 Apr 10:14:05 ntpd[17112]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T10:14:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:25 Apr 10:14:05 ntpd[17192]: ntpd: time slew -0.000220 s 2024-04-25T10:14:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:ntpd: time slew -0.000220s 2024-04-25T10:14:05.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T10:14:05.096 INFO:teuthology.orchestra.run.smithi117.stderr:25 Apr 10:14:05 ntpd[17192]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T10:14:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:14:05.167 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================== 2024-04-25T10:14:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.168 INFO:teuthology.orchestra.run.smithi045.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.210 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:14:05.210 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-04-25T10:14:05.210 INFO:teuthology.orchestra.run.smithi117.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.210 INFO:teuthology.orchestra.run.smithi117.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.210 INFO:teuthology.orchestra.run.smithi117.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.211 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T10:14:05.212 INFO:teuthology.run_tasks:Running task install... 2024-04-25T10:14:05.220 DEBUG:teuthology.task.install:project ceph 2024-04-25T10:14:05.220 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T10:14:05.220 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T10:14:05.221 INFO:teuthology.task.install:Using flavor: default 2024-04-25T10:14:05.226 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T10:14:05.227 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T10:14:05.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo apt-key list | grep Ceph 2024-04-25T10:14:05.228 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-key list | grep Ceph 2024-04-25T10:14:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T10:14:05.358 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T10:14:05.384 INFO:teuthology.orchestra.run.smithi045.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T10:14:05.384 INFO:teuthology.orchestra.run.smithi045.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T10:14:05.385 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T10:14:05.385 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T10:14:05.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:14:05.408 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T10:14:05.408 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T10:14:05.409 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T10:14:05.409 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T10:14:05.409 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:14:05.553 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T10:14:05.553 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:14:05.575 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T10:14:05.576 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:14:05.669 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:14:05.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T10:14:05.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo apt-get update 2024-04-25T10:14:05.704 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:14:05.704 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T10:14:05.724 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-04-25T10:14:05.880 INFO:teuthology.orchestra.run.smithi045.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T10:14:05.880 INFO:teuthology.orchestra.run.smithi045.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T10:14:05.891 INFO:teuthology.orchestra.run.smithi045.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T10:14:05.909 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T10:14:05.911 INFO:teuthology.orchestra.run.smithi045.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T10:14:05.930 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T10:14:05.945 INFO:teuthology.orchestra.run.smithi045.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T10:14:05.950 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T10:14:05.952 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi045.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T10:14:05.973 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T10:14:05.996 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T10:14:05.999 INFO:teuthology.orchestra.run.smithi045.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T10:14:06.016 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T10:14:06.025 INFO:teuthology.orchestra.run.smithi045.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T10:14:06.027 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T10:14:06.049 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T10:14:06.063 INFO:teuthology.orchestra.run.smithi045.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T10:14:06.488 INFO:teuthology.orchestra.run.smithi045.stdout:Fetched 28.9 kB in 1s (50.4 kB/s) 2024-04-25T10:14:06.495 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 28.9 kB in 1s (52.1 kB/s) 2024-04-25T10:14:07.479 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-04-25T10:14:07.494 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi045.stdout:Reading package lists... 2024-04-25T10:14:07.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:14:07.569 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-04-25T10:14:07.595 INFO:teuthology.orchestra.run.smithi045.stdout:Reading package lists... 2024-04-25T10:14:07.751 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-04-25T10:14:07.751 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi045.stdout:Building dependency tree... 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi045.stdout:Reading state information... 2024-04-25T10:14:07.933 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:14:07.933 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:14:07.933 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:14:07.933 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:14:07.933 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:14:07.934 INFO:teuthology.orchestra.run.smithi117.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:14:07.934 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:14:07.934 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-04-25T10:14:07.934 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T10:14:07.936 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T10:14:07.937 INFO:teuthology.orchestra.run.smithi117.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-04-25T10:14:07.938 INFO:teuthology.orchestra.run.smithi117.stdout: btrfs-tools 2024-04-25T10:14:07.957 INFO:teuthology.orchestra.run.smithi045.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:14:07.957 INFO:teuthology.orchestra.run.smithi045.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:14:07.957 INFO:teuthology.orchestra.run.smithi045.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:14:07.957 INFO:teuthology.orchestra.run.smithi045.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:14:07.957 INFO:teuthology.orchestra.run.smithi045.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:14:07.958 INFO:teuthology.orchestra.run.smithi045.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:14:07.958 INFO:teuthology.orchestra.run.smithi045.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:14:07.958 INFO:teuthology.orchestra.run.smithi045.stdout:The following additional packages will be installed: 2024-04-25T10:14:07.958 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T10:14:07.958 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T10:14:07.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T10:14:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:Suggested packages: 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout:Recommended packages: 2024-04-25T10:14:07.962 INFO:teuthology.orchestra.run.smithi045.stdout: btrfs-tools 2024-04-25T10:14:08.089 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-04-25T10:14:08.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T10:14:08.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T10:14:08.089 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T10:14:08.090 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T10:14:08.091 INFO:teuthology.orchestra.run.smithi117.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:14:08.091 INFO:teuthology.orchestra.run.smithi117.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T10:14:08.091 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi117.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi117.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi117.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T10:14:08.094 INFO:teuthology.orchestra.run.smithi117.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T10:14:08.094 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T10:14:08.094 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T10:14:08.094 INFO:teuthology.orchestra.run.smithi117.stdout: socat unzip xmlstarlet zip 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout:The following NEW packages will be installed: 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T10:14:08.120 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T10:14:08.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T10:14:08.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T10:14:08.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T10:14:08.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T10:14:08.121 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T10:14:08.121 INFO:teuthology.orchestra.run.smithi045.stdout: socat unzip xmlstarlet zip 2024-04-25T10:14:08.195 INFO:teuthology.orchestra.run.smithi045.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T10:14:08.196 INFO:teuthology.orchestra.run.smithi045.stdout:Need to get 176 MB of archives. 2024-04-25T10:14:08.196 INFO:teuthology.orchestra.run.smithi045.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T10:14:08.196 INFO:teuthology.orchestra.run.smithi045.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T10:14:08.217 INFO:teuthology.orchestra.run.smithi045.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T10:14:08.229 INFO:teuthology.orchestra.run.smithi045.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T10:14:08.269 INFO:teuthology.orchestra.run.smithi045.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T10:14:08.272 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T10:14:08.273 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 176 MB of archives. 2024-04-25T10:14:08.273 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T10:14:08.273 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T10:14:08.288 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T10:14:08.290 INFO:teuthology.orchestra.run.smithi045.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T10:14:08.326 INFO:teuthology.orchestra.run.smithi045.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T10:14:08.327 INFO:teuthology.orchestra.run.smithi045.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T10:14:08.340 INFO:teuthology.orchestra.run.smithi045.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T10:14:08.376 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T10:14:08.416 INFO:teuthology.orchestra.run.smithi045.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T10:14:08.420 INFO:teuthology.orchestra.run.smithi045.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T10:14:08.425 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T10:14:08.435 INFO:teuthology.orchestra.run.smithi045.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T10:14:08.440 INFO:teuthology.orchestra.run.smithi045.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T10:14:08.441 INFO:teuthology.orchestra.run.smithi045.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T10:14:08.452 INFO:teuthology.orchestra.run.smithi045.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T10:14:08.455 INFO:teuthology.orchestra.run.smithi045.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T10:14:08.457 INFO:teuthology.orchestra.run.smithi045.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T10:14:08.459 INFO:teuthology.orchestra.run.smithi045.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T10:14:08.460 INFO:teuthology.orchestra.run.smithi045.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T10:14:08.482 INFO:teuthology.orchestra.run.smithi045.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T10:14:08.484 INFO:teuthology.orchestra.run.smithi045.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T10:14:08.485 INFO:teuthology.orchestra.run.smithi045.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T10:14:08.486 INFO:teuthology.orchestra.run.smithi045.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T10:14:08.486 INFO:teuthology.orchestra.run.smithi045.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T10:14:08.487 INFO:teuthology.orchestra.run.smithi045.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T10:14:08.488 INFO:teuthology.orchestra.run.smithi045.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T10:14:08.504 INFO:teuthology.orchestra.run.smithi045.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T10:14:08.504 INFO:teuthology.orchestra.run.smithi045.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T10:14:08.504 INFO:teuthology.orchestra.run.smithi045.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T10:14:08.523 INFO:teuthology.orchestra.run.smithi045.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T10:14:08.523 INFO:teuthology.orchestra.run.smithi045.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T10:14:08.524 INFO:teuthology.orchestra.run.smithi045.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T10:14:08.527 INFO:teuthology.orchestra.run.smithi045.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T10:14:08.527 INFO:teuthology.orchestra.run.smithi045.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T10:14:08.528 INFO:teuthology.orchestra.run.smithi045.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T10:14:08.528 INFO:teuthology.orchestra.run.smithi045.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T10:14:08.529 INFO:teuthology.orchestra.run.smithi045.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T10:14:08.544 INFO:teuthology.orchestra.run.smithi045.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T10:14:08.545 INFO:teuthology.orchestra.run.smithi045.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T10:14:08.546 INFO:teuthology.orchestra.run.smithi045.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T10:14:08.565 INFO:teuthology.orchestra.run.smithi045.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T10:14:08.566 INFO:teuthology.orchestra.run.smithi045.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T10:14:08.571 INFO:teuthology.orchestra.run.smithi045.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T10:14:08.572 INFO:teuthology.orchestra.run.smithi045.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T10:14:08.572 INFO:teuthology.orchestra.run.smithi045.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T10:14:08.573 INFO:teuthology.orchestra.run.smithi045.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T10:14:08.574 INFO:teuthology.orchestra.run.smithi045.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T10:14:08.586 INFO:teuthology.orchestra.run.smithi045.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T10:14:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T10:14:08.588 INFO:teuthology.orchestra.run.smithi045.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T10:14:08.614 INFO:teuthology.orchestra.run.smithi045.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T10:14:08.617 INFO:teuthology.orchestra.run.smithi045.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T10:14:08.617 INFO:teuthology.orchestra.run.smithi045.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T10:14:08.617 INFO:teuthology.orchestra.run.smithi045.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T10:14:08.619 INFO:teuthology.orchestra.run.smithi045.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T10:14:08.625 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T10:14:08.634 INFO:teuthology.orchestra.run.smithi045.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T10:14:08.659 INFO:teuthology.orchestra.run.smithi045.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T10:14:08.672 INFO:teuthology.orchestra.run.smithi045.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T10:14:08.673 INFO:teuthology.orchestra.run.smithi045.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T10:14:08.673 INFO:teuthology.orchestra.run.smithi045.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T10:14:08.674 INFO:teuthology.orchestra.run.smithi045.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T10:14:08.676 INFO:teuthology.orchestra.run.smithi045.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T10:14:08.676 INFO:teuthology.orchestra.run.smithi045.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T10:14:08.677 INFO:teuthology.orchestra.run.smithi045.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T10:14:08.678 INFO:teuthology.orchestra.run.smithi045.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T10:14:08.681 INFO:teuthology.orchestra.run.smithi045.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T10:14:08.693 INFO:teuthology.orchestra.run.smithi045.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T10:14:08.702 INFO:teuthology.orchestra.run.smithi045.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T10:14:08.704 INFO:teuthology.orchestra.run.smithi045.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T10:14:08.705 INFO:teuthology.orchestra.run.smithi045.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T10:14:08.718 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T10:14:08.723 INFO:teuthology.orchestra.run.smithi045.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T10:14:08.724 INFO:teuthology.orchestra.run.smithi045.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T10:14:08.725 INFO:teuthology.orchestra.run.smithi045.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T10:14:08.726 INFO:teuthology.orchestra.run.smithi045.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T10:14:08.726 INFO:teuthology.orchestra.run.smithi045.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T10:14:08.727 INFO:teuthology.orchestra.run.smithi045.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T10:14:08.731 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T10:14:08.745 INFO:teuthology.orchestra.run.smithi045.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T10:14:08.746 INFO:teuthology.orchestra.run.smithi045.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T10:14:08.750 INFO:teuthology.orchestra.run.smithi045.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T10:14:08.752 INFO:teuthology.orchestra.run.smithi045.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T10:14:08.754 INFO:teuthology.orchestra.run.smithi045.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T10:14:08.767 INFO:teuthology.orchestra.run.smithi045.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T10:14:08.770 INFO:teuthology.orchestra.run.smithi045.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T10:14:08.773 INFO:teuthology.orchestra.run.smithi045.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T10:14:08.773 INFO:teuthology.orchestra.run.smithi045.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T10:14:08.774 INFO:teuthology.orchestra.run.smithi045.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T10:14:08.782 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T10:14:08.787 INFO:teuthology.orchestra.run.smithi045.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T10:14:08.792 INFO:teuthology.orchestra.run.smithi045.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T10:14:08.792 INFO:teuthology.orchestra.run.smithi045.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T10:14:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T10:14:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T10:14:08.807 INFO:teuthology.orchestra.run.smithi045.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T10:14:08.808 INFO:teuthology.orchestra.run.smithi045.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T10:14:08.810 INFO:teuthology.orchestra.run.smithi045.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T10:14:08.811 INFO:teuthology.orchestra.run.smithi045.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T10:14:08.818 INFO:teuthology.orchestra.run.smithi045.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T10:14:08.828 INFO:teuthology.orchestra.run.smithi045.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T10:14:08.828 INFO:teuthology.orchestra.run.smithi045.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T10:14:08.831 INFO:teuthology.orchestra.run.smithi045.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T10:14:08.831 INFO:teuthology.orchestra.run.smithi045.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T10:14:08.936 INFO:teuthology.orchestra.run.smithi045.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T10:14:08.939 INFO:teuthology.orchestra.run.smithi045.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T10:14:08.955 INFO:teuthology.orchestra.run.smithi045.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T10:14:08.962 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T10:14:08.974 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T10:14:09.002 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T10:14:09.010 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T10:14:09.175 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T10:14:09.438 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T10:14:09.835 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T10:14:09.854 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T10:14:09.874 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T10:14:09.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T10:14:09.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T10:14:09.909 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T10:14:09.915 INFO:teuthology.orchestra.run.smithi045.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T10:14:09.934 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T10:14:09.951 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T10:14:09.956 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T10:14:09.957 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T10:14:10.061 INFO:teuthology.orchestra.run.smithi045.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T10:14:10.067 INFO:teuthology.orchestra.run.smithi045.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T10:14:10.075 INFO:teuthology.orchestra.run.smithi045.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T10:14:10.099 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T10:14:10.129 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T10:14:10.142 INFO:teuthology.orchestra.run.smithi045.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T10:14:10.205 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T10:14:10.211 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T10:14:10.226 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T10:14:10.257 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T10:14:10.258 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T10:14:10.293 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T10:14:10.302 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T10:14:10.303 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T10:14:10.323 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T10:14:10.336 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T10:14:10.338 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T10:14:10.374 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T10:14:10.411 INFO:teuthology.orchestra.run.smithi045.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T10:14:10.546 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T10:14:10.561 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T10:14:10.568 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T10:14:10.573 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T10:14:10.759 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T10:14:10.858 INFO:teuthology.orchestra.run.smithi117.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T10:14:11.046 INFO:teuthology.orchestra.run.smithi117.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T10:14:11.085 INFO:teuthology.orchestra.run.smithi117.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T10:14:11.097 INFO:teuthology.orchestra.run.smithi117.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T10:14:11.098 INFO:teuthology.orchestra.run.smithi117.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi117.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T10:14:11.139 INFO:teuthology.orchestra.run.smithi117.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T10:14:11.289 INFO:teuthology.orchestra.run.smithi117.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T10:14:11.289 INFO:teuthology.orchestra.run.smithi117.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T10:14:11.294 INFO:teuthology.orchestra.run.smithi117.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T10:14:11.299 INFO:teuthology.orchestra.run.smithi117.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T10:14:11.325 INFO:teuthology.orchestra.run.smithi117.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T10:14:11.517 INFO:teuthology.orchestra.run.smithi117.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T10:14:11.566 INFO:teuthology.orchestra.run.smithi045.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T10:14:11.566 INFO:teuthology.orchestra.run.smithi045.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T10:14:11.574 INFO:teuthology.orchestra.run.smithi045.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T10:14:11.661 INFO:teuthology.orchestra.run.smithi045.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T10:14:11.704 INFO:teuthology.orchestra.run.smithi045.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T10:14:11.710 INFO:teuthology.orchestra.run.smithi045.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T10:14:11.771 INFO:teuthology.orchestra.run.smithi117.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T10:14:11.809 INFO:teuthology.orchestra.run.smithi045.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T10:14:11.910 INFO:teuthology.orchestra.run.smithi117.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T10:14:11.952 INFO:teuthology.orchestra.run.smithi117.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T10:14:12.044 INFO:teuthology.orchestra.run.smithi117.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T10:14:12.054 INFO:teuthology.orchestra.run.smithi117.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T10:14:12.067 INFO:teuthology.orchestra.run.smithi117.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T10:14:12.112 INFO:teuthology.orchestra.run.smithi117.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T10:14:12.268 INFO:teuthology.orchestra.run.smithi045.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T10:14:12.269 INFO:teuthology.orchestra.run.smithi045.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T10:14:12.377 INFO:teuthology.orchestra.run.smithi117.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T10:14:12.378 INFO:teuthology.orchestra.run.smithi117.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T10:14:12.380 INFO:teuthology.orchestra.run.smithi117.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T10:14:12.544 INFO:teuthology.orchestra.run.smithi117.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T10:14:12.893 INFO:teuthology.orchestra.run.smithi117.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T10:14:13.280 INFO:teuthology.orchestra.run.smithi117.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T10:14:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T10:14:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T10:14:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T10:14:13.384 INFO:teuthology.orchestra.run.smithi117.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T10:14:13.384 INFO:teuthology.orchestra.run.smithi117.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T10:14:13.385 INFO:teuthology.orchestra.run.smithi117.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T10:14:13.386 INFO:teuthology.orchestra.run.smithi117.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T10:14:13.386 INFO:teuthology.orchestra.run.smithi117.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T10:14:13.388 INFO:teuthology.orchestra.run.smithi117.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T10:14:13.694 INFO:teuthology.orchestra.run.smithi117.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T10:14:13.715 INFO:teuthology.orchestra.run.smithi117.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T10:14:13.722 INFO:teuthology.orchestra.run.smithi117.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T10:14:13.747 INFO:teuthology.orchestra.run.smithi117.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T10:14:13.850 INFO:teuthology.orchestra.run.smithi117.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T10:14:13.950 INFO:teuthology.orchestra.run.smithi117.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T10:14:14.273 INFO:teuthology.orchestra.run.smithi117.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T10:14:14.323 INFO:teuthology.orchestra.run.smithi117.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi117.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T10:14:14.357 INFO:teuthology.orchestra.run.smithi117.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T10:14:14.358 INFO:teuthology.orchestra.run.smithi117.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T10:14:14.404 INFO:teuthology.orchestra.run.smithi117.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T10:14:14.430 INFO:teuthology.orchestra.run.smithi117.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T10:14:14.432 INFO:teuthology.orchestra.run.smithi117.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T10:14:14.439 INFO:teuthology.orchestra.run.smithi117.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T10:14:14.442 INFO:teuthology.orchestra.run.smithi117.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T10:14:14.641 INFO:teuthology.orchestra.run.smithi117.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T10:14:14.903 INFO:teuthology.orchestra.run.smithi117.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T10:14:14.974 INFO:teuthology.orchestra.run.smithi045.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T10:14:14.975 INFO:teuthology.orchestra.run.smithi045.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T10:14:14.977 INFO:teuthology.orchestra.run.smithi045.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T10:14:14.978 INFO:teuthology.orchestra.run.smithi045.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T10:14:14.979 INFO:teuthology.orchestra.run.smithi045.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T10:14:14.981 INFO:teuthology.orchestra.run.smithi045.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T10:14:15.169 INFO:teuthology.orchestra.run.smithi117.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T10:14:15.172 INFO:teuthology.orchestra.run.smithi117.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T10:14:15.188 INFO:teuthology.orchestra.run.smithi117.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T10:14:15.194 INFO:teuthology.orchestra.run.smithi117.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T10:14:15.231 INFO:teuthology.orchestra.run.smithi117.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T10:14:15.246 INFO:teuthology.orchestra.run.smithi117.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T10:14:15.337 INFO:teuthology.orchestra.run.smithi117.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T10:14:15.338 INFO:teuthology.orchestra.run.smithi117.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T10:14:15.340 INFO:teuthology.orchestra.run.smithi117.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T10:14:15.528 INFO:teuthology.orchestra.run.smithi117.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T10:14:15.697 INFO:teuthology.orchestra.run.smithi045.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T10:14:15.796 INFO:teuthology.orchestra.run.smithi117.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T10:14:15.985 INFO:teuthology.orchestra.run.smithi117.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T10:14:15.986 INFO:teuthology.orchestra.run.smithi117.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T10:14:16.017 INFO:teuthology.orchestra.run.smithi117.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T10:14:16.119 INFO:teuthology.orchestra.run.smithi117.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T10:14:16.158 INFO:teuthology.orchestra.run.smithi117.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T10:14:16.165 INFO:teuthology.orchestra.run.smithi117.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T10:14:16.228 INFO:teuthology.orchestra.run.smithi117.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T10:14:16.373 INFO:teuthology.orchestra.run.smithi045.stdout:Fetched 176 MB in 8s (23.3 MB/s) 2024-04-25T10:14:16.512 INFO:teuthology.orchestra.run.smithi117.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T10:14:16.512 INFO:teuthology.orchestra.run.smithi117.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T10:14:16.551 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T10:14:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T10:14:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T10:14:16.675 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T10:14:17.044 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T10:14:17.059 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T10:14:17.111 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:14:17.464 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T10:14:17.478 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T10:14:17.522 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T10:14:17.875 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T10:14:17.891 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T10:14:17.975 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:14:18.163 INFO:teuthology.orchestra.run.smithi117.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T10:14:18.164 INFO:teuthology.orchestra.run.smithi117.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T10:14:18.166 INFO:teuthology.orchestra.run.smithi117.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi117.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi117.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T10:14:18.167 INFO:teuthology.orchestra.run.smithi117.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T10:14:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T10:14:18.327 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T10:14:18.370 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:14:18.592 INFO:teuthology.orchestra.run.smithi117.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T10:14:18.714 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T10:14:18.730 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T10:14:18.772 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:14:19.109 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T10:14:19.124 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:14:19.167 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:14:19.227 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 176 MB in 10s (16.8 MB/s) 2024-04-25T10:14:19.562 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T10:14:19.577 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T10:14:19.621 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:14:19.664 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T10:14:19.668 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T10:14:19.721 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T10:14:19.981 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T10:14:19.997 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:14:20.040 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:14:20.099 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T10:14:20.112 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T10:14:20.148 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:14:20.368 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T10:14:20.383 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T10:14:20.417 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:14:20.509 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T10:14:20.522 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T10:14:20.568 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T10:14:20.729 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package librados2. 2024-04-25T10:14:20.744 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:20.820 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:20.954 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T10:14:20.968 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T10:14:21.055 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:14:21.419 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T10:14:21.432 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T10:14:21.465 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package librbd1. 2024-04-25T10:14:21.480 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:21.483 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:14:21.522 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:21.769 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T10:14:21.783 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T10:14:21.819 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:14:21.959 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T10:14:21.975 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:22.017 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:22.244 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-rados. 2024-04-25T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:22.281 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:22.295 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:14:22.372 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:22.767 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T10:14:22.781 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:14:22.825 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T10:14:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:23.613 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T10:14:23.628 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T10:14:23.633 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T10:14:23.647 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T10:14:23.671 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T10:14:23.691 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:14:23.915 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T10:14:23.930 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:23.964 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:24.002 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librados2. 2024-04-25T10:14:24.017 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:24.061 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:24.293 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T10:14:24.308 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T10:14:24.351 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:14:24.620 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T10:14:24.635 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T10:14:24.664 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librbd1. 2024-04-25T10:14:24.678 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:14:24.678 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:24.864 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:24.973 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T10:14:24.988 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T10:14:25.031 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:14:25.308 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T10:14:25.322 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:25.359 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package lua5.1. 2024-04-25T10:14:25.366 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:25.374 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T10:14:25.409 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:14:25.644 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rados. 2024-04-25T10:14:25.658 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:25.695 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package lua-any. 2024-04-25T10:14:25.702 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:25.710 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T10:14:25.753 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T10:14:26.022 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T10:14:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:26.071 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:26.199 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package zip. 2024-04-25T10:14:26.214 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T10:14:26.274 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T10:14:26.288 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:26.289 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T10:14:26.332 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:26.634 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package unzip. 2024-04-25T10:14:26.650 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T10:14:26.692 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:14:26.693 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T10:14:26.707 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:26.751 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:27.079 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package luarocks. 2024-04-25T10:14:27.080 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T10:14:27.094 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T10:14:27.094 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T10:14:27.137 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:14:27.137 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T10:14:27.440 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T10:14:27.454 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:27.498 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:27.544 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package librgw2. 2024-04-25T10:14:27.559 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:27.602 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:27.801 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T10:14:27.816 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T10:14:27.851 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T10:14:28.079 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:28.121 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:28.137 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T10:14:28.151 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T10:14:28.195 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:14:28.458 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T10:14:28.473 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:14:28.529 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T10:14:28.573 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:14:28.819 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T10:14:28.838 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:28.868 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:28.934 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua5.1. 2024-04-25T10:14:28.949 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T10:14:28.993 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:14:29.271 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-common. 2024-04-25T10:14:29.287 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:29.295 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua-any. 2024-04-25T10:14:29.310 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T10:14:29.329 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:29.345 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T10:14:29.815 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package zip. 2024-04-25T10:14:29.830 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T10:14:29.873 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T10:14:30.226 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package unzip. 2024-04-25T10:14:30.241 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T10:14:30.284 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:14:30.376 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-base. 2024-04-25T10:14:30.392 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:30.444 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:30.713 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package luarocks. 2024-04-25T10:14:30.728 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T10:14:30.763 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:14:30.988 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T10:14:31.003 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T10:14:31.047 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:14:31.120 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librgw2. 2024-04-25T10:14:31.134 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:31.179 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:31.291 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T10:14:31.307 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T10:14:31.341 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:14:31.662 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T10:14:31.664 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T10:14:31.678 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T10:14:31.678 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:31.720 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:14:31.722 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:32.040 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T10:14:32.055 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T10:14:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T10:14:32.082 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T10:14:32.098 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:14:32.125 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:14:32.342 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T10:14:32.358 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T10:14:32.392 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:14:32.437 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T10:14:32.451 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:32.486 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T10:14:32.669 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T10:14:32.711 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T10:14:32.898 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-common. 2024-04-25T10:14:32.913 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:33.014 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-portend. 2024-04-25T10:14:33.030 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T10:14:33.072 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T10:14:33.400 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T10:14:33.416 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T10:14:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T10:14:33.719 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T10:14:33.735 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T10:14:33.769 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:14:34.111 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-base. 2024-04-25T10:14:34.114 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T10:14:34.126 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:34.130 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T10:14:34.164 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T10:14:34.179 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:34.459 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T10:14:34.474 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T10:14:34.509 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T10:14:34.757 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T10:14:34.771 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T10:14:34.787 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T10:14:34.802 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T10:14:34.815 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:14:34.845 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:14:35.093 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T10:14:35.107 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T10:14:35.130 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-mako. 2024-04-25T10:14:35.146 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T10:14:35.151 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:14:35.189 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:14:35.471 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T10:14:35.485 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T10:14:35.521 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:14:35.525 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T10:14:35.540 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T10:14:35.575 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:14:35.807 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T10:14:35.821 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T10:14:35.844 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T10:14:35.860 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T10:14:35.890 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:14:35.902 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:14:36.193 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T10:14:36.205 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-webob. 2024-04-25T10:14:36.208 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T10:14:36.221 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T10:14:36.251 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:14:36.263 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:14:36.529 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T10:14:36.544 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T10:14:36.604 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T10:14:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T10:14:36.849 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T10:14:36.873 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-portend. 2024-04-25T10:14:36.888 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T10:14:36.894 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:14:36.932 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T10:14:37.203 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T10:14:37.218 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T10:14:37.226 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T10:14:37.241 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T10:14:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:14:37.301 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T10:14:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-paste. 2024-04-25T10:14:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T10:14:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:14:37.671 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T10:14:37.686 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T10:14:37.737 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:14:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T10:14:37.973 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T10:14:38.008 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:14:38.141 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T10:14:38.156 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T10:14:38.190 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T10:14:38.269 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T10:14:38.284 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T10:14:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:14:38.494 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T10:14:38.509 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T10:14:38.552 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T10:14:38.630 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T10:14:38.645 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T10:14:38.688 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T10:14:38.913 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T10:14:38.928 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T10:14:38.972 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:14:39.033 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T10:14:39.048 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T10:14:39.091 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:14:39.258 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-mako. 2024-04-25T10:14:39.273 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T10:14:39.308 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:14:39.462 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T10:14:39.478 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T10:14:39.520 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T10:14:39.675 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T10:14:39.719 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:14:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-idna. 2024-04-25T10:14:39.838 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T10:14:39.890 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T10:14:40.117 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T10:14:40.132 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T10:14:40.167 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:14:40.255 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T10:14:40.270 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T10:14:40.313 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:14:40.454 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-requests. 2024-04-25T10:14:40.469 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T10:14:40.512 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:14:40.624 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webob. 2024-04-25T10:14:40.639 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T10:14:40.682 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:14:40.823 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T10:14:40.838 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T10:14:40.881 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:14:41.035 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T10:14:41.050 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T10:14:41.088 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:14:41.867 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T10:14:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:41.925 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:42.130 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T10:14:42.146 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T10:14:42.180 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:14:42.314 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T10:14:42.329 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:42.363 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:42.475 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-paste. 2024-04-25T10:14:42.490 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T10:14:42.533 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:14:42.616 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T10:14:42.632 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T10:14:42.674 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:14:42.853 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T10:14:42.867 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T10:14:42.911 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:14:42.977 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T10:14:42.992 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T10:14:43.035 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T10:14:43.214 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T10:14:43.229 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T10:14:43.272 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:14:43.388 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T10:14:43.404 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:43.446 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:43.587 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T10:14:43.598 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T10:14:43.633 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T10:14:43.782 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T10:14:43.798 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:43.894 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T10:14:43.909 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T10:14:43.952 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:14:44.307 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T10:14:44.321 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T10:14:44.361 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T10:14:44.365 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T10:14:44.376 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:44.418 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:44.701 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-idna. 2024-04-25T10:14:44.717 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T10:14:44.760 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T10:14:45.104 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T10:14:45.119 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T10:14:45.147 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph. 2024-04-25T10:14:45.162 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:45.162 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:14:45.205 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:45.490 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests. 2024-04-25T10:14:45.505 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T10:14:45.549 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T10:14:45.565 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:45.565 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:14:45.607 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:45.877 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T10:14:45.891 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T10:14:45.902 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T10:14:45.918 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:45.935 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:14:45.951 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:46.250 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T10:14:46.265 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:46.313 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:46.321 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package cephadm. 2024-04-25T10:14:46.337 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:46.379 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:46.659 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T10:14:46.673 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:46.709 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:46.732 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T10:14:46.748 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T10:14:46.790 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T10:14:47.012 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T10:14:47.027 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T10:14:47.070 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:14:47.142 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T10:14:47.158 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:14:47.200 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:14:47.390 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T10:14:47.405 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T10:14:47.447 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T10:14:47.817 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T10:14:47.832 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:47.875 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:48.090 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-babel. 2024-04-25T10:14:48.103 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:14:48.146 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:14:48.220 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T10:14:48.235 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:48.278 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:48.424 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T10:14:48.439 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T10:14:48.474 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:14:48.785 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T10:14:48.801 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:48.882 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T10:14:48.896 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:48.940 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:49.139 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T10:14:49.155 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T10:14:49.188 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T10:14:49.483 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-routes. 2024-04-25T10:14:49.499 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T10:14:49.541 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:14:49.743 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph. 2024-04-25T10:14:49.758 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:49.801 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:49.844 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T10:14:49.860 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:49.902 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:50.296 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T10:14:50.311 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:50.322 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T10:14:50.336 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T10:14:50.345 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:50.380 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:14:50.723 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T10:14:50.739 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:50.781 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:50.833 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T10:14:50.848 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T10:14:50.883 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:14:51.168 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephadm. 2024-04-25T10:14:51.181 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:51.226 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:51.269 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T10:14:51.285 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T10:14:51.414 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:14:51.571 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T10:14:51.585 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T10:14:51.620 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T10:14:51.666 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T10:14:51.682 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T10:14:51.716 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:14:51.956 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T10:14:51.965 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:14:52.031 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:14:52.532 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T10:14:52.547 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:52.582 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:53.019 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-babel. 2024-04-25T10:14:53.029 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T10:14:53.068 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:14:53.439 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T10:14:53.447 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T10:14:53.480 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:14:53.552 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T10:14:53.566 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T10:14:53.610 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T10:14:53.833 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T10:14:53.841 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:53.883 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:53.921 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T10:14:53.936 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T10:14:53.979 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:14:54.238 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T10:14:54.245 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T10:14:54.286 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T10:14:54.291 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T10:14:54.305 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T10:14:54.349 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:14:54.589 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-routes. 2024-04-25T10:14:54.598 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T10:14:54.638 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:14:54.693 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T10:14:54.710 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T10:14:54.752 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T10:14:55.009 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T10:14:55.018 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:55.030 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T10:14:55.045 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T10:14:55.058 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:55.080 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T10:14:55.375 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T10:14:55.390 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T10:14:55.433 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:14:55.453 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T10:14:55.461 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T10:14:55.503 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:14:55.769 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T10:14:55.784 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T10:14:55.827 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T10:14:56.023 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T10:14:56.032 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T10:14:56.073 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:14:56.130 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T10:14:56.148 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T10:14:56.189 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:14:56.451 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T10:14:56.462 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T10:14:56.587 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:14:56.898 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T10:14:56.908 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T10:14:56.952 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T10:14:56.967 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:56.981 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:14:57.018 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:57.380 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T10:14:57.395 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T10:14:57.438 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:14:57.698 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T10:14:57.708 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:14:57.748 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:57.849 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T10:14:57.865 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:14:57.916 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:14:58.252 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package jq. 2024-04-25T10:14:58.267 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:14:58.318 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T10:14:58.621 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package socat. 2024-04-25T10:14:58.637 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T10:14:58.687 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:14:58.760 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T10:14:58.770 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T10:14:58.809 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T10:14:59.024 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T10:14:59.040 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T10:14:59.082 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:14:59.112 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T10:14:59.123 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T10:14:59.166 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:14:59.427 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-test. 2024-04-25T10:14:59.442 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:14:59.485 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:14:59.524 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T10:14:59.534 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T10:14:59.574 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:14:59.935 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T10:14:59.945 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T10:14:59.984 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T10:15:00.272 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T10:15:00.284 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T10:15:00.505 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T10:15:00.866 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T10:15:00.877 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T10:15:00.916 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:15:01.202 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T10:15:01.213 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T10:15:01.252 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T10:15:01.357 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T10:15:01.373 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:01.418 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:01.531 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T10:15:01.541 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T10:15:01.581 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:15:01.794 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T10:15:01.809 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:01.852 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:02.146 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T10:15:02.162 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T10:15:02.204 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T10:15:02.336 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T10:15:02.343 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:02.474 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T10:15:02.489 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:02.523 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:02.528 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:02.784 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T10:15:02.800 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:02.815 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T10:15:02.823 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T10:15:02.842 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:02.857 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:15:03.086 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T10:15:03.106 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:03.144 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:03.193 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T10:15:03.202 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:15:03.243 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:15:03.389 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T10:15:03.403 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T10:15:03.447 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:15:03.529 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package jq. 2024-04-25T10:15:03.537 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T10:15:03.579 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T10:15:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T10:15:03.803 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T10:15:03.823 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package socat. 2024-04-25T10:15:03.831 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T10:15:03.842 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:15:03.865 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:15:04.193 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T10:15:04.203 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T10:15:04.213 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T10:15:04.224 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T10:15:04.234 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:15:04.263 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:15:04.563 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-test. 2024-04-25T10:15:04.571 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:04.613 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:04.692 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package pkg-config. 2024-04-25T10:15:04.702 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T10:15:04.742 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:15:04.970 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T10:15:04.980 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T10:15:05.020 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:15:05.448 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T10:15:05.462 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T10:15:05.506 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T10:15:05.792 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T10:15:05.804 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T10:15:05.833 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T10:15:06.095 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T10:15:06.106 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T10:15:06.136 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:15:06.431 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T10:15:06.443 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T10:15:06.481 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T10:15:06.684 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T10:15:06.693 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:06.734 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:06.783 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-py. 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T10:15:06.866 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T10:15:07.046 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T10:15:07.054 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:07.236 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T10:15:07.249 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T10:15:07.286 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:15:07.457 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T10:15:07.469 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T10:15:07.507 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T10:15:07.715 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T10:15:07.726 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T10:15:07.765 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:15:07.767 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T10:15:07.777 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:07.818 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:08.101 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-toml. 2024-04-25T10:15:08.111 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T10:15:08.113 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T10:15:08.122 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T10:15:08.150 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T10:15:08.161 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:08.406 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T10:15:08.416 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:08.420 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T10:15:08.432 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T10:15:08.455 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:08.461 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:15:08.808 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T10:15:08.809 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T10:15:08.821 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T10:15:08.821 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T10:15:08.858 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:15:08.866 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:15:09.144 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T10:15:09.157 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T10:15:09.179 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T10:15:09.190 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T10:15:09.195 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:15:09.220 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:15:09.616 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T10:15:09.627 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T10:15:09.666 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:15:09.890 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package radosgw. 2024-04-25T10:15:09.901 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:09.940 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:10.104 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package pkg-config. 2024-04-25T10:15:10.117 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T10:15:10.153 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:15:10.415 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T10:15:10.426 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T10:15:10.465 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:15:10.543 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T10:15:10.554 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:10.584 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:10.859 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T10:15:10.917 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T10:15:10.928 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T10:15:10.967 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T10:15:11.123 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T10:15:11.249 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T10:15:11.249 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:15:11.278 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T10:15:11.550 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T10:15:11.573 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T10:15:11.585 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T10:15:11.623 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:15:11.934 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T10:15:11.946 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T10:15:11.978 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T10:15:11.984 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T10:15:12.337 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-py. 2024-04-25T10:15:12.348 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T10:15:12.386 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T10:15:12.604 INFO:teuthology.orchestra.run.smithi045.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T10:15:12.638 INFO:teuthology.orchestra.run.smithi045.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T10:15:12.684 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:12.698 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T10:15:12.710 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T10:15:12.748 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:15:13.203 INFO:teuthology.orchestra.run.smithi045.stdout:Adding system user cephadm....done 2024-04-25T10:15:13.226 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T10:15:13.238 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T10:15:13.276 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:15:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:15:13.562 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-toml. 2024-04-25T10:15:13.573 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T10:15:13.612 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T10:15:13.649 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:15:13.874 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T10:15:13.885 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T10:15:13.910 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:15:13.923 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:15:14.026 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:15:14.251 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T10:15:14.262 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T10:15:14.269 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T10:15:14.301 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:15:14.521 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:15:14.579 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T10:15:14.590 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T10:15:14.629 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:15:14.646 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T10:15:14.990 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T10:15:15.408 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package radosgw. 2024-04-25T10:15:15.418 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:15.457 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:15.644 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T10:15:15.903 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T10:15:16.162 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:15:16.294 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T10:15:16.305 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T10:15:16.343 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:16.414 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:15:16.669 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T10:15:16.682 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:15:16.932 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T10:15:16.958 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:15:17.058 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T10:15:17.226 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:17.317 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T10:15:17.712 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T10:15:17.743 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:15:17.844 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:15:17.945 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T10:15:18.205 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:15:18.311 INFO:teuthology.orchestra.run.smithi117.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T10:15:18.339 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:15:18.365 INFO:teuthology.orchestra.run.smithi117.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T10:15:18.409 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:18.607 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:15:18.953 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user cephadm....done 2024-04-25T10:15:18.968 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T10:15:18.996 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T10:15:19.052 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:15:19.107 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T10:15:19.220 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T10:15:19.399 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T10:15:19.604 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:15:19.700 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T10:15:19.730 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T10:15:19.852 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T10:15:20.006 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:15:20.136 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T10:15:20.324 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:15:20.453 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T10:15:20.604 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T10:15:20.650 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T10:15:20.776 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:15:20.940 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T10:15:21.316 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T10:15:21.478 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:15:21.634 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T10:15:21.738 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:15:21.863 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:15:21.902 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T10:15:21.998 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:15:22.195 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T10:15:22.324 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T10:15:22.521 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T10:15:22.734 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T10:15:22.881 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T10:15:23.001 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:15:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T10:15:23.386 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:15:23.433 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:23.695 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T10:15:23.709 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T10:15:23.885 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T10:15:24.022 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:15:24.035 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T10:15:24.362 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T10:15:24.382 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:15:24.490 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T10:15:24.649 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:15:24.747 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T10:15:24.984 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:25.109 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T10:15:25.144 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T10:15:25.250 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T10:15:25.376 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T10:15:25.385 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T10:15:25.720 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:15:25.752 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T10:15:25.903 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T10:15:26.229 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T10:15:26.506 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T10:15:26.815 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T10:15:26.941 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T10:15:27.034 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:15:27.268 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:15:27.403 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T10:15:27.626 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T10:15:27.679 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:15:27.805 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:15:27.902 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T10:15:27.965 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:15:28.044 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T10:15:28.204 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T10:15:28.274 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:15:28.497 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T10:15:28.550 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T10:15:28.810 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:15:28.881 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T10:15:28.911 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:15:29.157 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T10:15:29.244 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:15:29.514 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T10:15:29.575 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T10:15:29.773 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T10:15:29.885 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T10:15:30.040 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:15:30.211 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T10:15:30.317 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T10:15:30.612 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T10:15:30.626 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:15:30.727 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T10:15:30.872 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T10:15:30.995 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:15:31.022 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:31.104 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:15:31.349 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T10:15:31.405 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:15:31.716 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T10:15:31.740 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:15:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:15:32.150 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:15:32.276 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:15:32.348 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T10:15:32.620 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:15:32.745 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:15:32.947 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T10:15:33.155 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:15:33.257 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:15:33.416 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T10:15:33.440 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T10:15:33.566 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T10:15:33.701 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T10:15:33.833 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:15:33.852 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T10:15:33.960 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:15:33.977 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T10:15:34.277 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:15:34.279 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T10:15:34.571 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T10:15:34.595 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:15:34.831 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:15:34.880 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T10:15:34.957 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T10:15:35.075 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T10:15:35.283 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T10:15:35.351 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T10:15:35.559 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T10:15:35.611 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:15:35.844 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T10:15:35.862 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:15:36.161 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T10:15:36.289 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:15:36.420 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T10:15:36.582 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:15:36.688 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T10:15:36.814 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T10:15:36.942 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:15:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:37.107 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T10:15:37.186 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:37.233 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T10:15:37.296 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:15:37.534 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T10:15:37.640 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:37.851 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T10:15:37.892 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:15:38.136 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T10:15:38.219 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:38.237 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T10:15:38.320 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:38.338 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T10:15:38.426 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T10:15:38.438 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:15:38.660 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:15:38.673 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T10:15:38.849 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T10:15:39.267 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T10:15:39.339 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:39.507 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:39.551 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T10:15:39.642 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:39.702 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T10:15:39.776 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:39.902 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:15:39.978 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T10:15:40.079 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T10:15:40.397 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T10:15:40.723 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T10:15:41.016 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T10:15:41.236 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T10:15:41.249 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:41.374 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:41.495 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T10:15:41.500 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:41.664 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:41.813 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T10:15:41.870 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:42.081 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:15:42.406 INFO:teuthology.orchestra.run.smithi045.stdout:Adding group ceph....done 2024-04-25T10:15:42.474 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:15:42.699 INFO:teuthology.orchestra.run.smithi045.stdout:Adding system user ceph....done 2024-04-25T10:15:42.792 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:15:42.869 INFO:teuthology.orchestra.run.smithi045.stdout:Setting system user ceph properties....done 2024-04-25T10:15:42.892 INFO:teuthology.orchestra.run.smithi045.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:15:43.046 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:15:43.153 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:15:43.254 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:43.380 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:43.467 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:15:43.531 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:15:43.908 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:44.034 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:44.113 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:44.169 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:15:44.312 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:15:44.312 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:15:44.471 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:44.613 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:44.764 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:44.989 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:15:44.989 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:15:45.015 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:45.166 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:45.715 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:45.750 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:15:45.750 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:15:45.900 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:46.051 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:46.202 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:46.328 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:15:46.424 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:46.919 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:15:47.598 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:47.612 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:47.724 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:47.825 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:15:47.825 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:15:47.850 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:48.001 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:48.127 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:48.236 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:48.646 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:48.772 INFO:teuthology.orchestra.run.smithi117.stdout:Adding group ceph....done 2024-04-25T10:15:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:15:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:15:49.165 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user ceph....done 2024-04-25T10:15:49.334 INFO:teuthology.orchestra.run.smithi117.stdout:Setting system user ceph properties....done 2024-04-25T10:15:49.350 INFO:teuthology.orchestra.run.smithi117.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:15:49.489 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:15:49.534 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:49.869 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:15:49.869 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:15:49.882 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:15:50.545 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:50.570 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:50.671 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:50.770 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:15:50.770 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:15:50.788 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:51.004 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:15:51.388 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:51.513 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:51.656 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:51.824 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:51.950 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:52.091 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:15:52.091 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:15:52.101 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:52.470 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:15:52.571 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:15:52.715 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:15:52.782 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:53.182 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:15:53.685 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:15:53.892 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:15:54.114 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:15:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:15:54.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:15:54.500 INFO:teuthology.orchestra.run.smithi045.stdout:Reading package lists... 2024-04-25T10:15:54.679 INFO:teuthology.orchestra.run.smithi045.stdout:Building dependency tree... 2024-04-25T10:15:54.679 INFO:teuthology.orchestra.run.smithi045.stdout:Reading state information... 2024-04-25T10:15:54.744 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:15:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:15:54.862 INFO:teuthology.orchestra.run.smithi045.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:15:54.863 INFO:teuthology.orchestra.run.smithi045.stdout:The following additional packages will be installed: 2024-04-25T10:15:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:15:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:15:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:15:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:15:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:15:54.864 INFO:teuthology.orchestra.run.smithi045.stdout:Suggested packages: 2024-04-25T10:15:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:15:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:15:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:15:54.920 INFO:teuthology.orchestra.run.smithi045.stdout:The following NEW packages will be installed: 2024-04-25T10:15:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:15:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:15:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:15:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:15:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:15:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:15:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:15:54.922 INFO:teuthology.orchestra.run.smithi045.stdout:The following packages will be upgraded: 2024-04-25T10:15:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:15:54.940 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:15:54.940 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:15:54.985 INFO:teuthology.orchestra.run.smithi045.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T10:15:54.985 INFO:teuthology.orchestra.run.smithi045.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:15:54.986 INFO:teuthology.orchestra.run.smithi045.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:15:54.986 INFO:teuthology.orchestra.run.smithi045.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:15:55.145 INFO:teuthology.orchestra.run.smithi045.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:15:55.149 INFO:teuthology.orchestra.run.smithi045.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:15:55.181 INFO:teuthology.orchestra.run.smithi045.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:15:55.182 INFO:teuthology.orchestra.run.smithi045.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:15:55.183 INFO:teuthology.orchestra.run.smithi045.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:15:55.191 INFO:teuthology.orchestra.run.smithi045.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:15:55.192 INFO:teuthology.orchestra.run.smithi045.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:15:55.193 INFO:teuthology.orchestra.run.smithi045.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:15:55.194 INFO:teuthology.orchestra.run.smithi045.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:15:55.195 INFO:teuthology.orchestra.run.smithi045.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:15:55.196 INFO:teuthology.orchestra.run.smithi045.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:15:55.205 INFO:teuthology.orchestra.run.smithi045.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:15:55.206 INFO:teuthology.orchestra.run.smithi045.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:15:55.226 INFO:teuthology.orchestra.run.smithi045.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:15:55.286 INFO:teuthology.orchestra.run.smithi045.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:15:55.537 INFO:teuthology.orchestra.run.smithi045.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:15:55.539 INFO:teuthology.orchestra.run.smithi045.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:15:55.539 INFO:teuthology.orchestra.run.smithi045.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:15:55.543 INFO:teuthology.orchestra.run.smithi045.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:15:55.543 INFO:teuthology.orchestra.run.smithi045.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:15:55.548 INFO:teuthology.orchestra.run.smithi045.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:15:55.550 INFO:teuthology.orchestra.run.smithi045.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:15:55.558 INFO:teuthology.orchestra.run.smithi045.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:15:55.558 INFO:teuthology.orchestra.run.smithi045.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:15:55.582 INFO:teuthology.orchestra.run.smithi045.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:15:55.582 INFO:teuthology.orchestra.run.smithi045.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:15:55.586 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:55.600 INFO:teuthology.orchestra.run.smithi045.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:15:55.623 INFO:teuthology.orchestra.run.smithi045.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:15:55.627 INFO:teuthology.orchestra.run.smithi045.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:15:55.628 INFO:teuthology.orchestra.run.smithi045.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:15:55.629 INFO:teuthology.orchestra.run.smithi045.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:15:55.629 INFO:teuthology.orchestra.run.smithi045.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:15:55.643 INFO:teuthology.orchestra.run.smithi045.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:15:55.645 INFO:teuthology.orchestra.run.smithi045.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:15:55.812 INFO:teuthology.orchestra.run.smithi045.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:15:55.812 INFO:teuthology.orchestra.run.smithi045.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:15:55.812 INFO:teuthology.orchestra.run.smithi045.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:15:55.812 INFO:teuthology.orchestra.run.smithi045.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:15:55.813 INFO:teuthology.orchestra.run.smithi045.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:15:55.813 INFO:teuthology.orchestra.run.smithi045.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:15:55.827 INFO:teuthology.orchestra.run.smithi045.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:15:55.827 INFO:teuthology.orchestra.run.smithi045.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:15:55.934 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:15:55.935 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:15:56.221 INFO:teuthology.orchestra.run.smithi045.stdout:Preconfiguring packages ... 2024-04-25T10:15:56.375 INFO:teuthology.orchestra.run.smithi045.stdout:Fetched 55.7 MB in 1s (62.2 MB/s) 2024-04-25T10:15:56.596 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:56.632 INFO:teuthology.orchestra.run.smithi045.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:15:56.636 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:15:56.764 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:56.774 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:15:56.957 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:57.183 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:15:57.183 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:15:57.404 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:15:57.878 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:58.087 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:58.246 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:58.389 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:15:58.511 INFO:teuthology.orchestra.run.smithi045.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:15:58.768 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:15:58.779 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package flex. 2024-04-25T10:15:58.853 INFO:teuthology.orchestra.run.smithi045.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:15:58.856 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:15:58.902 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:15:58.911 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:15:59.147 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:15:59.373 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:15:59.517 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:15:59.925 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:16:00.436 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:16:00.452 INFO:teuthology.orchestra.run.smithi045.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:16:00.455 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:16:00.551 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:16:00.956 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:16:01.075 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:16:01.141 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:16:01.146 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:16:01.220 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-04-25T10:16:01.358 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:16:01.403 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-04-25T10:16:01.404 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-04-25T10:16:01.582 INFO:teuthology.orchestra.run.smithi117.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:16:01.582 INFO:teuthology.orchestra.run.smithi117.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:16:01.582 INFO:teuthology.orchestra.run.smithi117.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:16:01.582 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:16:01.582 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:16:01.583 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:16:01.583 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:16:01.583 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:16:01.583 INFO:teuthology.orchestra.run.smithi117.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:16:01.583 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:16:01.584 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-04-25T10:16:01.584 INFO:teuthology.orchestra.run.smithi117.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:16:01.584 INFO:teuthology.orchestra.run.smithi117.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:16:01.585 INFO:teuthology.orchestra.run.smithi117.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:16:01.585 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:16:01.585 INFO:teuthology.orchestra.run.smithi117.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:16:01.586 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-04-25T10:16:01.586 INFO:teuthology.orchestra.run.smithi117.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:16:01.586 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:16:01.586 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:16:01.640 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-04-25T10:16:01.640 INFO:teuthology.orchestra.run.smithi117.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi117.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi117.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi117.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:16:01.641 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi117.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:16:01.642 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2024-04-25T10:16:01.643 INFO:teuthology.orchestra.run.smithi117.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:16:01.709 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package bison. 2024-04-25T10:16:01.789 INFO:teuthology.orchestra.run.smithi045.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:16:01.793 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:16:01.828 INFO:teuthology.orchestra.run.smithi117.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T10:16:01.828 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:16:01.829 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:16:01.829 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:16:01.862 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:16:02.271 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:16:02.287 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:16:02.329 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:16:02.502 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:16:02.523 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:16:02.616 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package dump. 2024-04-25T10:16:02.632 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:16:02.665 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:16:02.672 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:16:02.673 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:16:02.675 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:16:02.711 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:16:02.714 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:16:02.729 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:16:02.731 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:16:02.732 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:16:02.904 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:16:02.969 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package indent. 2024-04-25T10:16:02.985 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:03.028 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:16:03.198 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:16:03.208 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:16:03.213 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:16:03.289 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:16:03.301 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:16:03.306 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:16:03.347 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:16:03.685 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:16:03.702 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:16:03.743 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:16:03.963 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:16:03.979 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:16:04.013 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:16:04.375 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:16:04.391 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:16:04.395 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:16:04.397 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:16:04.421 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:16:04.421 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:16:04.425 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:16:04.433 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:04.591 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:16:04.686 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:16:04.702 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:16:04.744 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:04.980 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:16:04.996 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:16:05.030 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:16:05.209 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:16:05.342 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:16:05.358 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:16:05.396 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:16:05.396 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:16:05.408 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:16:05.439 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:16:05.499 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:16:05.504 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:16:05.505 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:16:05.505 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:16:05.506 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:16:05.719 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:16:06.146 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:16:06.737 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:16:06.753 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:16:06.795 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:06.935 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:16:06.936 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:16:06.936 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:16:06.936 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:16:06.936 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:16:06.936 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:16:06.980 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:16:06.981 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:16:07.039 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:16:07.056 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:16:07.098 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:07.363 INFO:teuthology.orchestra.run.smithi117.stdout:Preconfiguring packages ... 2024-04-25T10:16:07.426 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:16:07.442 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:16:07.484 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:07.519 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 55.7 MB in 5s (10.5 MB/s) 2024-04-25T10:16:07.801 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:16:07.805 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:16:07.829 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:16:07.845 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:16:07.887 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:16:07.977 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:16:08.198 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:16:08.215 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:16:08.256 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:16:08.568 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:16:08.585 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:08.618 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:08.644 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:16:08.938 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:16:08.954 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:08.996 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:09.290 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:16:09.307 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:09.349 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:09.735 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:16:09.751 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:16:09.784 INFO:teuthology.orchestra.run.smithi117.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:16:09.810 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:16:10.107 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package flex. 2024-04-25T10:16:10.177 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T10:16:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:16:10.264 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:16:10.280 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:16:10.322 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:16:10.372 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:16:10.659 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:16:10.676 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:16:10.717 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:10.933 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:16:11.103 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:16:11.366 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:16:11.382 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:16:11.424 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:16:11.517 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:16:11.957 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package ppp. 2024-04-25T10:16:11.974 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:16:12.065 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:16:12.068 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:16:12.279 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:16:12.351 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:16:12.637 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:16:12.652 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:16:12.672 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:16:12.687 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:16:12.792 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:16:12.940 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:16:12.955 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:16:12.998 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:16:13.156 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:16:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:16:13.343 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:16:13.385 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:16:13.526 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package bison. 2024-04-25T10:16:13.596 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T10:16:13.600 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:16:13.653 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:16:13.672 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:16:13.688 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:13.730 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:14.037 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:16:14.052 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:16:14.087 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:16:14.318 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:16:14.334 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:16:14.396 INFO:teuthology.orchestra.run.smithi045.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:16:14.426 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:16:14.465 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package dump. 2024-04-25T10:16:14.479 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:16:14.523 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:16:14.819 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package indent. 2024-04-25T10:16:14.834 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:14.877 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:16:14.892 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:16:14.909 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:14.954 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:15.138 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:16:15.153 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:16:15.188 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:16:15.509 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:16:15.524 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:16:15.567 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:16:15.887 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:16:15.902 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:16:15.945 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:16:15.988 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:16:16.003 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:16:16.037 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:16:16.248 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:16:16.264 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:16:16.265 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:16:16.280 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:16:16.298 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:16.323 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:16:16.559 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:16:16.575 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:16:16.617 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:16.625 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:16:16.641 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:16:16.684 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:16:16.879 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:16:16.894 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:16:16.936 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:16:16.995 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:16:17.011 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:16:17.045 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:16:17.282 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:16:17.289 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package tcl. 2024-04-25T10:16:17.297 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:16:17.305 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:16:17.340 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:16:17.352 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:16:17.717 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:16:17.734 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:16:17.775 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:16:18.130 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:16:18.147 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:16:18.188 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:16:18.566 INFO:teuthology.orchestra.run.smithi045.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:16:18.583 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:16:18.633 INFO:teuthology.orchestra.run.smithi045.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:16:18.736 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:16:18.751 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:16:18.794 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:19.063 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:16:19.079 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:16:19.146 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:19.321 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:16:19.441 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:16:19.456 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:16:19.491 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:19.506 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:16:19.607 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:16:19.733 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:19.844 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:16:19.859 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:16:19.860 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:16:19.902 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:16:20.011 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:16:20.137 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:20.221 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:16:20.237 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:16:20.280 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:16:20.666 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:16:20.682 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:20.724 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:21.094 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:16:21.101 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:16:21.110 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:21.152 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:21.201 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:16:21.471 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:16:21.487 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:21.521 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:21.588 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:16:21.588 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:16:21.749 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:16:21.765 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:16:21.807 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:16:22.304 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:16:22.369 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:16:22.384 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:16:22.427 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:16:22.430 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:16:22.531 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:16:22.632 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:16:22.733 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:16:22.859 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:22.994 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:16:23.120 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:16:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:16:23.246 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:16:23.289 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:23.758 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:16:23.884 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:16:23.904 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:16:23.919 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:16:23.961 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:16:23.980 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:16:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:24.171 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:16:24.314 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:16:24.473 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:16:24.574 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:16:24.656 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:16:24.669 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ppp. 2024-04-25T10:16:24.684 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:16:24.726 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:16:24.910 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:16:25.028 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:16:25.087 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:16:25.213 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:16:25.382 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:16:25.397 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:16:25.440 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:16:25.515 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:16:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:16:25.767 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:16:25.782 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:16:25.826 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout:- ens1f0 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:16:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:26.196 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:16:26.211 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:16:26.262 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:16:26.329 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:16:26.658 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:16:26.672 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:26.724 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:26.736 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:16:27.164 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:16:27.305 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:16:27.320 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:16:27.381 INFO:teuthology.orchestra.run.smithi117.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:16:27.411 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:16:27.845 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:16:27.859 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:16:27.877 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:27.922 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:28.130 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:16:29.106 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:16:29.120 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:16:29.227 INFO:teuthology.orchestra.run.smithi045.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:16:29.441 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:16:29.458 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:16:29.463 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:29.463 INFO:teuthology.orchestra.run.smithi045.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:16:29.499 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:16:29.718 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:16:29.733 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:16:29.739 INFO:teuthology.orchestra.run.smithi045.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:16:29.739 INFO:teuthology.orchestra.run.smithi045.stdout:Removing obsolete dictionary files: 2024-04-25T10:16:29.768 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:16:30.096 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:16:30.110 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:16:30.154 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:16:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:16:30.457 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package tcl. 2024-04-25T10:16:30.471 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:16:30.519 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:16:30.892 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:16:30.907 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:16:30.951 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:16:30.957 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:31.298 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:16:31.306 INFO:teuthology.orchestra.run.smithi045.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:16:31.312 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:16:31.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:16:31.355 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:This user must also own the server process. 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:The default text search configuration will be set to "english". 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:Data page checksums are disabled. 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:31.379 INFO:teuthology.orchestra.run.smithi045.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:16:31.380 INFO:teuthology.orchestra.run.smithi045.stdout:creating subdirectories ... ok 2024-04-25T10:16:31.380 INFO:teuthology.orchestra.run.smithi045.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:16:31.405 INFO:teuthology.orchestra.run.smithi045.stdout:selecting default max_connections ... 100 2024-04-25T10:16:31.440 INFO:teuthology.orchestra.run.smithi045.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:16:31.454 INFO:teuthology.orchestra.run.smithi045.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:16:31.455 INFO:teuthology.orchestra.run.smithi045.stdout:creating configuration files ... ok 2024-04-25T10:16:31.629 INFO:teuthology.orchestra.run.smithi045.stdout:running bootstrap script ... ok 2024-04-25T10:16:31.766 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:16:31.781 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:16:31.833 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:16:32.126 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:16:32.269 INFO:teuthology.orchestra.run.smithi045.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:16:32.355 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:16:32.456 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:16:32.574 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:32.725 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:16:32.774 INFO:teuthology.orchestra.run.smithi045.stdout:syncing data to disk ... ok 2024-04-25T10:16:32.876 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:16:33.036 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:33.304 INFO:teuthology.orchestra.run.smithi045.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:16:33.986 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:16:34.091 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:16:34.503 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:16:34.503 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:16:35.186 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:16:35.311 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:16:35.437 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:16:35.563 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:16:35.815 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:35.958 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:16:36.184 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:16:36.316 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:36.442 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:16:36.568 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:16:36.677 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:36.799 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:16:36.812 INFO:teuthology.orchestra.run.smithi045.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:36.925 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:16:37.020 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:16:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:16:37.080 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:16:37.085 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:37.211 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:16:37.379 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:16:37.488 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:16:37.589 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:16:37.662 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:16:37.732 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:16:37.858 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:16:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:16:37.990 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:16:38.054 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:16:38.185 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:38.300 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:16:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:16:38.444 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:16:38.496 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:16:38.596 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:16:39.182 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:39.183 INFO:teuthology.orchestra.run.smithi117.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:16:39.183 INFO:teuthology.orchestra.run.smithi117.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:16:39.183 INFO:teuthology.orchestra.run.smithi117.stdout:- enp3s0f1 2024-04-25T10:16:39.183 INFO:teuthology.orchestra.run.smithi117.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:16:39.183 INFO:teuthology.orchestra.run.smithi117.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:16:39.183 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:39.316 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:16:39.717 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:16:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:16:39.948 INFO:teuthology.orchestra.run.smithi045.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:16:40.123 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:16:40.523 INFO:teuthology.orchestra.run.smithi045.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:16:40.532 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:40.916 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:16:41.059 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:16:41.185 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:16:41.319 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:16:42.186 INFO:teuthology.orchestra.run.smithi117.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:16:42.463 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:42.464 INFO:teuthology.orchestra.run.smithi117.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:16:42.712 INFO:teuthology.orchestra.run.smithi117.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:16:42.712 INFO:teuthology.orchestra.run.smithi117.stdout:Removing obsolete dictionary files: 2024-04-25T10:16:43.457 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:16:43.953 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:16:44.458 INFO:teuthology.orchestra.run.smithi117.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:16:44.475 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:This user must also own the server process. 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:The default text search configuration will be set to "english". 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:Data page checksums are disabled. 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:44.533 INFO:teuthology.orchestra.run.smithi117.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:16:44.534 INFO:teuthology.orchestra.run.smithi117.stdout:creating subdirectories ... ok 2024-04-25T10:16:44.534 INFO:teuthology.orchestra.run.smithi117.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:16:44.558 INFO:teuthology.orchestra.run.smithi117.stdout:selecting default max_connections ... 100 2024-04-25T10:16:44.590 INFO:teuthology.orchestra.run.smithi117.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:16:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:16:44.605 INFO:teuthology.orchestra.run.smithi117.stdout:creating configuration files ... ok 2024-04-25T10:16:44.771 INFO:teuthology.orchestra.run.smithi117.stdout:running bootstrap script ... ok 2024-04-25T10:16:45.396 INFO:teuthology.orchestra.run.smithi117.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:16:45.991 INFO:teuthology.orchestra.run.smithi117.stdout:syncing data to disk ... ok 2024-04-25T10:16:46.511 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:16:49.543 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:16:49.845 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:16:49.996 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:16:50.164 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:16:50.448 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:16:50.448 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:16:51.191 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:16:51.292 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:16:51.644 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:16:51.812 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:16:53.348 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:16:53.448 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:16:54.021 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:16:54.029 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:54.029 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:16:54.178 DEBUG:teuthology.orchestra.run.smithi045:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:16:54.203 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:16:54.204 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:16:54.204 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:16:54.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:16:54.353 DEBUG:teuthology.orchestra.run.smithi117:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:16:54.375 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:16:54.375 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:16:54.375 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:16:54.377 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:16:54.377 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:54.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:16:54.400 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:54.400 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:16:54.438 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:16:54.438 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:54.438 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:16:54.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:16:54.528 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:54.528 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:16:54.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:16:54.608 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:16:54.608 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:54.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:16:54.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:16:54.696 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:54.696 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:16:54.715 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:16:54.776 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:16:54.777 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:54.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:16:54.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:16:54.859 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:54.860 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:16:54.878 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:16:54.940 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:16:55.032 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:16:55.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2024-04-25T10:16:55.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2024-04-25T10:16:55.055 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:16:55.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:16:55.102 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:16:55.121 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:16:55.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:16:55.170 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:16:55.199 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:16:55.199 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:16:55.199 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:16:55.199 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:55.199 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:16:55.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:55.337 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:55.337 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:16:55.346 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:55.448 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:16:55.448 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:16:55.449 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673055', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAGc3Klt6LAMmrSgcILM7JlDUR+7I9TzjabaLkKZ+p7si4+Q6RZ4su76s/U8y+/rRkDjRR+hQziHpTKBinxObcs=', 'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPotNQ3sAkA2cyuLvk+qDvMYmy+4nXZwLHv3e422ePhSbZ/e6QWEjE42uWyP0eTRlxxaIR5AURDNCMY0RhIfjEI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:16:55.450 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:16:55.453 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:16:55.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:16:55.510 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:16:55.530 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:55.530 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:16:55.579 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:16:55.580 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 898 Links: 1 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 10:10:23.680765631 +0000 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 10:10:23.412768021 +0000 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 10:10:23.412768021 +0000 2024-04-25T10:16:55.632 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 10:10:23.412768021 +0000 2024-04-25T10:16:55.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:16:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:16:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:16:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.00026503 s, 1.9 MB/s 2024-04-25T10:16:55.696 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:16:55.750 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 915 Links: 1 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 10:10:24.256760493 +0000 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 10:10:23.996762812 +0000 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 10:10:23.996762812 +0000 2024-04-25T10:16:55.800 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 10:10:23.996762812 +0000 2024-04-25T10:16:55.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:16:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:16:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:16:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000251338 s, 2.0 MB/s 2024-04-25T10:16:55.863 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:16:55.918 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 935 Links: 1 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 10:10:24.816755497 +0000 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 10:10:24.548757888 +0000 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 10:10:24.548757888 +0000 2024-04-25T10:16:55.968 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 10:10:24.548757888 +0000 2024-04-25T10:16:55.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:16:56.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:16:56.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:16:56.030 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000259711 s, 2.0 MB/s 2024-04-25T10:16:56.032 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:16:56.085 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-04-25T10:16:56.135 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 951 Links: 1 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 10:10:25.352750715 +0000 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 10:10:25.080753141 +0000 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 10:10:25.080753141 +0000 2024-04-25T10:16:56.136 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 10:10:25.080753141 +0000 2024-04-25T10:16:56.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:16:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:16:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:16:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000263063 s, 1.9 MB/s 2024-04-25T10:16:56.200 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:16:56.254 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:16:56.254 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:56.254 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:16:56.263 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:16:56.263 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 895 Links: 1 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-25 10:10:23.658840512 +0000 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-25 10:10:23.374852134 +0000 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-25 10:10:23.374852134 +0000 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-25 10:10:23.374852134 +0000 2024-04-25T10:16:56.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:16:56.376 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-25T10:16:56.377 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-25T10:16:56.377 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000388248 s, 1.3 MB/s 2024-04-25T10:16:56.378 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:16:56.432 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-04-25T10:16:56.482 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 910 Links: 1 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-25 10:10:24.242816614 +0000 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-25 10:10:23.958828236 +0000 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-25 10:10:23.958828236 +0000 2024-04-25T10:16:56.483 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-25 10:10:23.958828236 +0000 2024-04-25T10:16:56.483 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:16:56.544 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-25T10:16:56.544 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-25T10:16:56.544 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000382346 s, 1.3 MB/s 2024-04-25T10:16:56.546 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:16:56.600 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-04-25T10:16:56.650 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:16:56.650 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:56.650 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 927 Links: 1 2024-04-25T10:16:56.650 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:56.650 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-25 10:10:24.594802210 +0000 2024-04-25T10:16:56.651 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-25 10:10:24.590802374 +0000 2024-04-25T10:16:56.651 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-25 10:10:24.590802374 +0000 2024-04-25T10:16:56.651 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-25 10:10:24.590802374 +0000 2024-04-25T10:16:56.651 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:16:56.712 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-25T10:16:56.712 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-25T10:16:56.712 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000377698 s, 1.4 MB/s 2024-04-25T10:16:56.714 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:16:56.772 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-04-25T10:16:56.826 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 947 Links: 1 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-25 10:10:25.478766982 +0000 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-25 10:10:25.210777411 +0000 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-25 10:10:25.210777411 +0000 2024-04-25T10:16:56.827 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-25 10:10:25.210777411 +0000 2024-04-25T10:16:56.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:16:56.893 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-25T10:16:56.893 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-25T10:16:56.893 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000394291 s, 1.3 MB/s 2024-04-25T10:16:56.895 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:16:56.948 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:16:56.948 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:16:56.948 INFO:tasks.ceph:Generating config... 2024-04-25T10:16:56.949 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:16:56.949 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:16:56.950 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:16:56.950 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:16:56.950 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:16:56.950 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:16:56.950 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:16:56.950 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:16:56.951 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:16:56.951 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T10:16:56.951 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T10:16:56.951 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T10:16:56.951 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:16:56.951 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:16:56.952 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:16:56.952 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:16:56.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:16:56.999 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:16:57.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:16:57.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:16:57.089 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.45'), ('mon.b', '172.21.15.117'), ('mon.c', '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]')] 2024-04-25T10:16:57.090 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:16:57.090 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:57.091 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:16:57.143 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.45 --add b 172.21.15.117 --addv c '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:16:57.234 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:fsid d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2024-04-25T10:16:57.232131+0000 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-25T10:16:57.232131+0000 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 17 (quincy) 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 1 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:2: [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] mon.c 2024-04-25T10:16:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:16:57.238 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:16:57.246 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d7e12072-0435-4dda-862b-d1045153027e... 2024-04-25T10:16:57.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:57.290 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:16:57.331 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T10:16:57.331 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.332 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:57.333 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-04-25T10:16:57.334 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.336 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = bitmap 2024-04-25T10:16:57.337 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:16:57.338 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-04-25T10:16:57.339 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:57.340 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:57.341 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-04-25T10:16:57.342 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: mds cache max size = 1000 2024-04-25T10:16:57.343 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-04-25T10:16:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2024-04-25T10:16:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: mds log max segments = 1 2024-04-25T10:16:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-04-25T10:16:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2024-04-25T10:16:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:16:57.344 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:16:57.345 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:16:57.345 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-04-25T10:16:57.346 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.347 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-04-25T10:16:57.348 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:16:57.349 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-04-25T10:16:57.350 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.351 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-04-25T10:16:57.352 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.353 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore allocator = bitmap 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 1/20 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 1/20 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 4/10 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-04-25T10:16:57.356 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-04-25T10:16:57.357 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:16:57.358 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: mds cache max size = 1000 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-04-25T10:16:57.359 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2024-04-25T10:16:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: mds log max segments = 1 2024-04-25T10:16:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-04-25T10:16:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = True 2024-04-25T10:16:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:16:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:16:57.360 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:16:57.361 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:16:57.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:16:57.460 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:16:57.461 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:57.461 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:16:57.470 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:57.470 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:16:57.519 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:16:57.519 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:57.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:16:57.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:16:57.591 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:16:57.591 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:16:57.643 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:16:57.643 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:57.643 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:16:57.643 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:16:57.678 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:16:57.678 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:16:57.730 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:16:57.730 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:16:57.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:16:57.787 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:16:57.792 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:16:57.850 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:16:57.868 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:16:57.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:16:57.924 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:16:57.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:16:57.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:16:58.050 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:16:58.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:16:58.075 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:16:58.134 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:16:58.138 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:16:58.159 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:16:58.255 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:16:58.260 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:16:58.280 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:16:58.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:16:58.322 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:16:58.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:16:58.425 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:16:58.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:16:58.484 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:16:58.501 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:16:58.582 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:16:58.598 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:16:58.598 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:16:58.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:16:58.618 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:16:58.618 INFO:tasks.ceph:role: osd.0 2024-04-25T10:16:58.618 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:16:58.618 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:16:58.682 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:16:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-04-25T10:16:58.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-04-25T10:16:58.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:16:58.780 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:16:58.838 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:16:58.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:58.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:16:58.906 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:16:58.906 INFO:tasks.ceph:role: osd.1 2024-04-25T10:16:58.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:16:58.906 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:16:58.970 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:16:58.994 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-04-25T10:16:58.996 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-04-25T10:16:58.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:16:59.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:16:59.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:59.131 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:16:59.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:16:59.194 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:16:59.194 INFO:tasks.ceph:role: osd.2 2024-04-25T10:16:59.194 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:16:59.194 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:16:59.254 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:16:59.254 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:16:59.254 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:16:59.255 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:16:59.279 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-04-25T10:16:59.282 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-04-25T10:16:59.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:16:59.353 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:16:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:16:59.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:59.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:16:59.478 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:16:59.479 INFO:tasks.ceph:role: osd.3 2024-04-25T10:16:59.479 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:16:59.479 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:16:59.542 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:16:59.542 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:16:59.542 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:16:59.542 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:16:59.542 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:16:59.543 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:16:59.543 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:16:59.543 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:16:59.543 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:16:59.543 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:16:59.563 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-04-25T10:16:59.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-04-25T10:16:59.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:16:59.636 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:16:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:16:59.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:16:59.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:16:59.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:16:59.816+0000 7f036e7ad7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:16:59.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:16:59.816+0000 7f036e7ad7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:16:59.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:16:59.816+0000 7f036e7ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:16:59.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:16:59.816+0000 7f036e7ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:16:59.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:16:59.816+0000 7f036e7ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:16:59.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:16:59.820+0000 7f036e7ad7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:17:02.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:17:02.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:03.004+0000 7f454c5b67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:03.004+0000 7f454c5b67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:03.004+0000 7f454c5b67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:03.004+0000 7f454c5b67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:17:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:03.004+0000 7f454c5b67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:17:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:03.004+0000 7f454c5b67c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:17:06.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:17:06.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:06.184+0000 7fc21e2cb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:17:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:06.184+0000 7fc21e2cb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:17:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:06.184+0000 7fc21e2cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:17:06.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:06.184+0000 7fc21e2cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:17:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:06.184+0000 7fc21e2cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:17:06.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:06.188+0000 7fc21e2cb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:17:09.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:17:09.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:09.360+0000 7f41fb42b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:17:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:09.360+0000 7f41fb42b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:17:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:09.360+0000 7f41fb42b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:17:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:09.360+0000 7f41fb42b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:17:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:09.360+0000 7f41fb42b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:17:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:09.364+0000 7f41fb42b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:17:12.397 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:17:12.418 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:17:12.441 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:17:12.441 INFO:tasks.ceph:role: osd.4 2024-04-25T10:17:12.441 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:17:12.441 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:17:12.510 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:17:12.511 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:17:12.924 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-04-25T10:17:12.927 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-04-25T10:17:12.927 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:17:13.074 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:17:13.092 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:17:13.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:13.093 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:17:13.154 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:17:13.154 INFO:tasks.ceph:role: osd.5 2024-04-25T10:17:13.154 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:17:13.154 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:17:13.225 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:17:13.226 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:17:13.586 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-04-25T10:17:13.589 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-04-25T10:17:13.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:17:13.662 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:17:13.722 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:17:13.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:13.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:17:13.786 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:17:13.786 INFO:tasks.ceph:role: osd.6 2024-04-25T10:17:13.786 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:17:13.786 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:17:13.858 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:17:14.248 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-04-25T10:17:14.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-04-25T10:17:14.251 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:17:14.361 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:17:14.377 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:17:14.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:14.379 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:17:14.441 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:17:14.442 INFO:tasks.ceph:role: osd.7 2024-04-25T10:17:14.442 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:17:14.442 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:17:14.513 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:17:14.513 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:17:14.513 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:17:14.514 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:17:14.947 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-04-25T10:17:14.950 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-04-25T10:17:14.950 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:17:15.083 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:17:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:17:15.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:17:15.102 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:15.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:15.228+0000 7f4cfa2ea7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:17:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:15.232+0000 7f4cfa2ea7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:17:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:15.232+0000 7f4cfa2ea7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:17:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:15.232+0000 7f4cfa2ea7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:17:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:15.232+0000 7f4cfa2ea7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:17:15.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:15.244+0000 7f4cfa2ea7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:17:18.304 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:17:18.324 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:18.412+0000 7f28f24547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:17:18.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:18.412+0000 7f28f24547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:18.412+0000 7f28f24547c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:18.412+0000 7f28f24547c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:17:18.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:18.412+0000 7f28f24547c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:17:18.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:18.412+0000 7f28f24547c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:17:21.437 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:17:21.459 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:21.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:21.552+0000 7f08a27fb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:17:21.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:21.552+0000 7f08a27fb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:17:21.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:21.552+0000 7f08a27fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:17:21.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:21.552+0000 7f08a27fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:17:21.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:21.552+0000 7f08a27fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:17:21.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:21.556+0000 7f08a27fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:17:24.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:17:24.668 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:24.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:24.759+0000 7f9e2f81b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:17:24.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:24.763+0000 7f9e2f81b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:17:24.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:24.763+0000 7f9e2f81b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:17:24.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:24.763+0000 7f9e2f81b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:17:24.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:24.763+0000 7f9e2f81b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:17:24.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:24.763+0000 7f9e2f81b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:17:25.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:25.455 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:27.828 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:17:27.848 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:17:27.849 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:27.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:17:27.871 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:27.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:17:27.936 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:27.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:17:28.003 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:28.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:17:28.067 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:28.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:17:28.131 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:28.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:17:28.197 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:28.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:17:28.259 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.259 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:17:28.280 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.280 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:17:28.342 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.342 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:17:28.406 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.406 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:17:28.469 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.470 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:17:28.534 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.534 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:17:28.598 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:17:28.662 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:28.662 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:17:28.671 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:28.671 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:17:28.723 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.723 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:17:28.732 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:28.732 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:17:28.782 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:17:28.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:17:28.786 DEBUG:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:17:28.829 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2024-04-25T10:17:28.829 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ== 2024-04-25T10:17:28.829 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-04-25T10:17:28.829 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg== 2024-04-25T10:17:28.829 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg== 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w== 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA== 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug== 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ== 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA== 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-04-25T10:17:28.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout:[client.2] 2024-04-25T10:17:28.831 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg== 2024-04-25T10:17:28.832 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2024-04-25T10:17:28.832 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA== 2024-04-25T10:17:28.832 INFO:teuthology.orchestra.run.smithi045.stdout:[client.3] 2024-04-25T10:17:28.832 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.y] 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug== 2024-04-25T10:17:28.844 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.4] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.5] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.6] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg== 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.7] 2024-04-25T10:17:28.845 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ== 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout:[client.2] 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg== 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA== 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout:[client.3] 2024-04-25T10:17:28.846 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw== 2024-04-25T10:17:28.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:17:28.877 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:17:28.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:28.949 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.012 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.072 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.128 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.184 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.240 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:17:29.297 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:17:29.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.414 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:17:29.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.531 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.587 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.642 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:17:29.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.699 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.759 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.819 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.878 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:17:29.934 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:17:29.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:17:29.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:17:30.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:17:30.093 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:17:30.112 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:17:30.219 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:17:30.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:17:30.304 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:17:30.410 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:17:30.430 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:30.433 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:17:30.481 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:17:30.481 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:17:30.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:17:30.485 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:17:30.485 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:17:30.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:17:30.524 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:17:30.525 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:17:30.525 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:17:30.528 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:17:30.528 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:17:30.528 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:17:30.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:17:30.531 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:17:30.531 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:17:30.532 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:17:30.534 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:17:30.534 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:17:30.535 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.672+0000 7f1a1b200640 1 Processor -- start 2024-04-25T10:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.672+0000 7f1a1b200640 1 -- start start 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a1b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a1b200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140b1770 con 0x7f1a1415bc10 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a1b200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1415fec0 con 0x7f1a140cf980 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a1b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a141601a0 con 0x7f1a140d1830 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a13e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1a1415bc10 0x7f1a1415f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55998/0 (socket says 172.21.15.45:55998) 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 learned_addr learned my addr 172.21.15.45:0/1670994274 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:17:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:30.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.676+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:17:30.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.876+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.876+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:17:30.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.876+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:30.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.876+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:30.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:30.876+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:17:31.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:31.276+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:31.276+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:17:31.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:31.276+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:31.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:31.276+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:31.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:31.276+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:17:32.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:32.080+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:32.080+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:17:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:32.080+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:32.080+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:32.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:32.080+0000 7f1a19200640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:17:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 msgr2=0x7f1a140d1c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:17:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.117:3300/0 conn(0x7f1a140d1830 0x7f1a140d1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 >> v1:172.21.15.117:6789/0 conn(0x7f1a140cf980 legacy=0x7f1a140b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 >> v1:172.21.15.45:6789/0 conn(0x7f1a1415bc10 legacy=0x7f1a1415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a141601a0 con 0x7f19f00036d0 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0008d50 con 0x7f19f0006170 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.676+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:17:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:33.876+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:17:34.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:34.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:34.276+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a13e00640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:17:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:17:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:35.080+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T10:17:36.057 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:36.051+0000 7f66e6169180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:17:36.066 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:36.060+0000 7fd9d6119180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:17:36.553 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:36.547+0000 7f66e6169180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:17:36.569 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:36.564+0000 7fd9d6119180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 msgr2=0x7f19f0008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0006170 0x7f19f0008630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 msgr2=0x7f19f0005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f00036d0 0x7f19f0005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a19200640 1 -- 172.21.15.45:0/1670994274 reap_dead start 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f0003b40 0x7f19f0006620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140cc5b0 con 0x7f1a1415bc10 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f0008850 con 0x7f19f0006b60 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a12000640 1 -- 172.21.15.45:0/1670994274 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a141601a0 con 0x7f19f0003b40 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> v2:172.21.15.45:3300/0 conn(0x7f19f0003b40 0x7f19f0006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> v1:172.21.15.117:6789/0 conn(0x7f19f0006b60 legacy=0x7f19f0007010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 >> v1:172.21.15.45:6789/0 conn(0x7f1a1415bc10 legacy=0x7f19f000b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a0be00640 1 -- 172.21.15.45:0/1670994274 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00109b0 con 0x7f19f0003b40 2024-04-25T10:17:36.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.676+0000 7f1a13400640 1 -- 172.21.15.45:0/1670994274 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427422773 0 0) 0x7f1a140cc5b0 con 0x7f1a1415bc10 2024-04-25T10:17:36.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0003b40 0x7f19f0006620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f19fc007c40 tx=0x7f19fc01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d6bd815f0dcb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a13400640 1 -- 172.21.15.45:0/1670994274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0ab070 con 0x7f19f0003b40 2024-04-25T10:17:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a13400640 1 -- 172.21.15.45:0/1670994274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0a9070 con 0x7f19f0003b40 2024-04-25T10:17:36.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a13400640 1 -- 172.21.15.45:0/1670994274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0a5b20 con 0x7f19f0003b40 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 -- 172.21.15.45:0/1670994274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0003b40 msgr2=0x7f19f0006620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0003b40 0x7f19f0006620 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f19fc007c40 tx=0x7f19fc01cc90 comp rx=0 tx=0).stop 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 -- 172.21.15.45:0/1670994274 shutdown_connections 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 --2- 172.21.15.45:0/1670994274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0003b40 0x7f19f0006620 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 -- 172.21.15.45:0/1670994274 >> 172.21.15.45:0/1670994274 conn(0x7f1a1409a9d0 msgr2=0x7f1a1409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 -- 172.21.15.45:0/1670994274 shutdown_connections 2024-04-25T10:17:36.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 -- 172.21.15.45:0/1670994274 wait complete. 2024-04-25T10:17:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 Processor -- start 2024-04-25T10:17:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.680+0000 7f1a1b200640 1 -- start start 2024-04-25T10:17:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0003b40 0x7f1a140b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f0006b60 0x7f1a1406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 0x7f1a1406c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140d21a0 con 0x7f1a1415bc10 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140d2020 con 0x7f19f0006b60 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140ccd90 con 0x7f19f0003b40 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 0x7f1a1406c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 0x7f1a1406c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44428/0 (socket says 172.21.15.45:44428) 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 -- 172.21.15.45:0/1031150982 learned_addr learned my addr 172.21.15.45:0/1031150982 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a13e00640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0003b40 0x7f1a140b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 -- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0003b40 msgr2=0x7f1a140b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0003b40 0x7f1a140b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f0006b60 0x7f1a1406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 -- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f0006b60 msgr2=0x7f1a1406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f0006b60 0x7f1a1406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 -- 172.21.15.45:0/1031150982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140b7000 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a0be00640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f0006b60 0x7f1a1406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a19200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 0x7f1a1406c4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19f00060f0 tx=0x7f1a0c011600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c017460 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c01d070 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c017840 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a140b72e0 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a140c8e70 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1a0c0179e0 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1a0c022a80 con 0x7f1a1415bc10 2024-04-25T10:17:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.684+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1404ef30 con 0x7f1a1415bc10 2024-04-25T10:17:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.696+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1a0c010040 con 0x7f1a1415bc10 2024-04-25T10:17:36.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.756+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1a140b7900 con 0x7f1a1415bc10 2024-04-25T10:17:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a10c00640 1 -- 172.21.15.45:0/1031150982 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1a0c022460 con 0x7f1a1415bc10 2024-04-25T10:17:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 msgr2=0x7f1a1406c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 0x7f1a1406c4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f19f00060f0 tx=0x7f1a0c011600 comp rx=0 tx=0).stop 2024-04-25T10:17:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 shutdown_connections 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a1415bc10 0x7f1a1406c4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f0006b60 0x7f1a1406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 --2- 172.21.15.45:0/1031150982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f0003b40 0x7f1a140b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 >> 172.21.15.45:0/1031150982 conn(0x7f1a1409a9d0 msgr2=0x7f1a1406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 shutdown_connections 2024-04-25T10:17:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:36.760+0000 7f1a1b200640 1 -- 172.21.15.45:0/1031150982 wait complete. 2024-04-25T10:17:36.771 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:36.767+0000 7f66e6169180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:17:36.784 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:17:36.784 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:17:36.785 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:36.785 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:17:36.804 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:36.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:17:36.814 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:36.808+0000 7fd9d6119180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:17:36.833 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:36.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:17:36.894 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:17:36.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:17:36.953 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:36.954 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:17:36.973 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:36.973 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:17:37.038 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:37.038 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:17:37.101 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:17:37.101 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:17:37.161 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 1504397e-8735-47e0-a345-d9d96f7ea66c 0 2024-04-25T10:17:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.331+0000 7f3422400640 1 Processor -- start 2024-04-25T10:17:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.331+0000 7f3422400640 1 -- start start 2024-04-25T10:17:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f3422400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f341c15df30 0x7f341c1626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f3422400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f341c06df80 0x7f341c0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f3422400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c06f000 con 0x7f341c15e310 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f3422400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c162bf0 con 0x7f341c06df80 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f3422400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0afa20 con 0x7f341c15df30 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f341c15df30 0x7f341c1626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f341c15df30 0x7f341c1626b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57986/0 (socket says 172.21.15.117:57986) 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 -- 172.21.15.117:0/2581159335 learned_addr learned my addr 172.21.15.117:0/2581159335 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341aa00640 1 --2- 172.21.15.117:0/2581159335 >> v2:172.21.15.45:3300/0 conn(0x7f341c06df80 0x7f341c0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 -- 172.21.15.117:0/2581159335 >> v2:172.21.15.45:3300/0 conn(0x7f341c06df80 msgr2=0x7f341c0aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 --2- 172.21.15.117:0/2581159335 >> v2:172.21.15.45:3300/0 conn(0x7f341c06df80 0x7f341c0aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 -- 172.21.15.117:0/2581159335 >> v1:172.21.15.45:6789/0 conn(0x7f341c15e310 legacy=0x7f341c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341b400640 1 -- 172.21.15.117:0/2581159335 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c06e480 con 0x7f341c15df30 2024-04-25T10:17:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.335+0000 7f341a000640 1 -- 172.21.15.117:0/2581159335 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741995915 0 0) 0x7f341c06f000 con 0x7f341c15e310 2024-04-25T10:17:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f341b400640 1 --2- 172.21.15.117:0/2581159335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1626b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f341c073470 tx=0x7f340c01c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2013a16d517f78d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f341a000640 1 -- 172.21.15.117:0/2581159335 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0aa650 con 0x7f341c15df30 2024-04-25T10:17:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f341a000640 1 -- 172.21.15.117:0/2581159335 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c0af070 con 0x7f341c15df30 2024-04-25T10:17:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f341a000640 1 -- 172.21.15.117:0/2581159335 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0aaa10 con 0x7f341c15df30 2024-04-25T10:17:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- 172.21.15.117:0/2581159335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 msgr2=0x7f341c1626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 --2- 172.21.15.117:0/2581159335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1626b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f341c073470 tx=0x7f340c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- 172.21.15.117:0/2581159335 shutdown_connections 2024-04-25T10:17:37.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 --2- 172.21.15.117:0/2581159335 >> v2:172.21.15.45:3300/0 conn(0x7f341c06df80 0x7f341c0aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 --2- 172.21.15.117:0/2581159335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1626b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- 172.21.15.117:0/2581159335 >> 172.21.15.117:0/2581159335 conn(0x7f341c0b1ca0 msgr2=0x7f341c0b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- 172.21.15.117:0/2581159335 shutdown_connections 2024-04-25T10:17:37.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- 172.21.15.117:0/2581159335 wait complete. 2024-04-25T10:17:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 Processor -- start 2024-04-25T10:17:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- start start 2024-04-25T10:17:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f341c06df80 0x7f341c072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c15e310 0x7f341c1432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c16afe0 con 0x7f341c06df80 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0af410 con 0x7f341c15df30 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.339+0000 7f3422400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c16b160 con 0x7f341c15e310 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c15e310 0x7f341c1432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57990/0 (socket says 172.21.15.117:57990) 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341aa00640 1 -- 172.21.15.117:0/2965499254 learned_addr learned my addr 172.21.15.117:0/2965499254 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341b400640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f341c06df80 0x7f341c072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 -- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 msgr2=0x7f341c1531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 -- 172.21.15.117:0/2965499254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f341c06df80 msgr2=0x7f341c072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f341c06df80 0x7f341c072530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 -- 172.21.15.117:0/2965499254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c156fa0 con 0x7f341c15e310 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341b400640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f341c06df80 0x7f341c072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f341be00640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c15e310 0x7f341c1432b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f341400c780 tx=0x7f341400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414003b80 con 0x7f341c15e310 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414003d20 con 0x7f341c15e310 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c157280 con 0x7f341c15e310 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140052b0 con 0x7f341c15e310 2024-04-25T10:17:37.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c070b70 con 0x7f341c15e310 2024-04-25T10:17:37.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3414005450 con 0x7f341c15e310 2024-04-25T10:17:37.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f341401b780 con 0x7f341c15e310 2024-04-25T10:17:37.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.343+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c04ef30 con 0x7f341c15e310 2024-04-25T10:17:37.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.347+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f341400a640 con 0x7f341c15e310 2024-04-25T10:17:37.394 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.391+0000 7f66e6169180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:17:37.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.411+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1504397e-8735-47e0-a345-d9d96f7ea66c", "id": 0} v 0) v1 -- 0x7f341c068ef0 con 0x7f341c15e310 2024-04-25T10:17:37.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3413400640 1 -- 172.21.15.117:0/2965499254 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1504397e-8735-47e0-a345-d9d96f7ea66c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f341400a820 con 0x7f341c15e310 2024-04-25T10:17:37.425 INFO:teuthology.orchestra.run.smithi117.stdout:0 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c15e310 msgr2=0x7f341c1432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c15e310 0x7f341c1432b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f341400c780 tx=0x7f341400cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 shutdown_connections 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f341c15e310 0x7f341c1432b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f341c15df30 0x7f341c1531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 --2- 172.21.15.117:0/2965499254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f341c06df80 0x7f341c072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 >> 172.21.15.117:0/2965499254 conn(0x7f341c0b1ca0 msgr2=0x7f341c0d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 shutdown_connections 2024-04-25T10:17:37.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.419+0000 7f3422400640 1 -- 172.21.15.117:0/2965499254 wait complete. 2024-04-25T10:17:37.444 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 53fcceda-1557-4e7c-81ff-f2163ddc2f55 1 2024-04-25T10:17:37.459 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.455+0000 7f66e6169180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:17:37.467 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.464+0000 7fd9d6119180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:17:37.521 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.515+0000 7f66e6169180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:17:37.532 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.528+0000 7fd9d6119180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:17:37.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 Processor -- start 2024-04-25T10:17:37.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 -- start start 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0cf80ce8a0 0x7f0cf80d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf806ba80 0x7f0cf815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf80d2630 con 0x7f0cf806ac90 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf80b1620 con 0x7f0cf80ce8a0 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cffc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80d0d60 con 0x7f0cf806ba80 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfd200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0cf80ce8a0 0x7f0cf80d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfdc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf806ba80 0x7f0cf815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfd200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0cf80ce8a0 0x7f0cf80d06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57992/0 (socket says 172.21.15.117:57992) 2024-04-25T10:17:37.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfd200640 1 -- 172.21.15.117:0/3197318804 learned_addr learned my addr 172.21.15.117:0/3197318804 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:37.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfdc00640 1 -- 172.21.15.117:0/3197318804 >> v2:172.21.15.117:3300/0 conn(0x7f0cf80ce8a0 msgr2=0x7f0cf80d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfdc00640 1 --2- 172.21.15.117:0/3197318804 >> v2:172.21.15.117:3300/0 conn(0x7f0cf80ce8a0 0x7f0cf80d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfdc00640 1 -- 172.21.15.117:0/3197318804 >> v1:172.21.15.45:6789/0 conn(0x7f0cf806ac90 legacy=0x7f0cf806b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.575+0000 7f0cfdc00640 1 -- 172.21.15.117:0/3197318804 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf80b0fd0 con 0x7f0cf806ba80 2024-04-25T10:17:37.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cfdc00640 1 --2- 172.21.15.117:0/3197318804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf806ba80 0x7f0cf815d630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80b17a0 tx=0x7f0cf001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e871fa3403feaf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cf7400640 1 -- 172.21.15.117:0/3197318804 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00b2070 con 0x7f0cf806ba80 2024-04-25T10:17:37.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cf7400640 1 -- 172.21.15.117:0/3197318804 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf00a5920 con 0x7f0cf806ba80 2024-04-25T10:17:37.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cf7400640 1 -- 172.21.15.117:0/3197318804 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00a5cf0 con 0x7f0cf806ba80 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 -- 172.21.15.117:0/3197318804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf806ba80 msgr2=0x7f0cf815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 --2- 172.21.15.117:0/3197318804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf806ba80 0x7f0cf815d630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80b17a0 tx=0x7f0cf001c950 comp rx=0 tx=0).stop 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 -- 172.21.15.117:0/3197318804 shutdown_connections 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 --2- 172.21.15.117:0/3197318804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf806ba80 0x7f0cf815d630 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 --2- 172.21.15.117:0/3197318804 >> v2:172.21.15.117:3300/0 conn(0x7f0cf80ce8a0 0x7f0cf80d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 -- 172.21.15.117:0/3197318804 >> 172.21.15.117:0/3197318804 conn(0x7f0cf80d19c0 msgr2=0x7f0cf80d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 -- 172.21.15.117:0/3197318804 shutdown_connections 2024-04-25T10:17:37.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.579+0000 7f0cffc00640 1 -- 172.21.15.117:0/3197318804 wait complete. 2024-04-25T10:17:37.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 Processor -- start 2024-04-25T10:17:37.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 -- start start 2024-04-25T10:17:37.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf806ac90 0x7f0cf814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 0x7f0cf8158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80ce8a0 0x7f0cf814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b1e20 con 0x7f0cf806ac90 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b1ca0 con 0x7f0cf806ba80 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b1fa0 con 0x7f0cf80ce8a0 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 0x7f0cf8158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cfd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf806ac90 0x7f0cf814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cfdc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80ce8a0 0x7f0cf814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 0x7f0cf8158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57994/0 (socket says 172.21.15.117:57994) 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 -- 172.21.15.117:0/1599874791 learned_addr learned my addr 172.21.15.117:0/1599874791 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 -- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80ce8a0 msgr2=0x7f0cf814ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80ce8a0 0x7f0cf814ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 -- 172.21.15.117:0/1599874791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf806ac90 msgr2=0x7f0cf814e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf806ac90 0x7f0cf814e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 -- 172.21.15.117:0/1599874791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8158860 con 0x7f0cf806ba80 2024-04-25T10:17:37.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf7e00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 0x7f0cf8158320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c420 tx=0x7f0ce400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40052d0 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce4005470 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4007260 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf8158b00 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf81590c0 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0ce4007400 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.583+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0ce4003b20 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.587+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf804ef30 con 0x7f0cf806ba80 2024-04-25T10:17:37.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.587+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0ce4003600 con 0x7f0cf806ba80 2024-04-25T10:17:37.596 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.592+0000 7fd9d6119180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:17:37.613 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.607+0000 7f66e6169180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:17:37.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.655+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "53fcceda-1557-4e7c-81ff-f2163ddc2f55", "id": 1} v 0) v1 -- 0x7f0cf80b0d10 con 0x7f0cf806ba80 2024-04-25T10:17:37.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cf4c00640 1 -- 172.21.15.117:0/1599874791 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "53fcceda-1557-4e7c-81ff-f2163ddc2f55", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ce4003d20 con 0x7f0cf806ba80 2024-04-25T10:17:37.667 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-04-25T10:17:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 msgr2=0x7f0cf8158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 0x7f0cf8158320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400c420 tx=0x7f0ce400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 shutdown_connections 2024-04-25T10:17:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80ce8a0 0x7f0cf814ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf806ba80 0x7f0cf8158320 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 --2- 172.21.15.117:0/1599874791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cf806ac90 0x7f0cf814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 >> 172.21.15.117:0/1599874791 conn(0x7f0cf80d19c0 msgr2=0x7f0cf80d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 shutdown_connections 2024-04-25T10:17:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.663+0000 7f0cffc00640 1 -- 172.21.15.117:0/1599874791 wait complete. 2024-04-25T10:17:37.675 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.671+0000 7f66e6169180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:17:37.686 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new e6b66be6-3e3f-4880-9aec-e3caa98db3fe 2 2024-04-25T10:17:37.689 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.684+0000 7fd9d6119180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:17:37.753 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.748+0000 7fd9d6119180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:17:37.766 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.763+0000 7f66e6169180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:17:37.828 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.823+0000 7f66e6169180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:17:37.846 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.840+0000 7fd9d6119180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:17:37.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 Processor -- start 2024-04-25T10:17:37.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 -- start start 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f955015dcc0 0x7f9550162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 0x7f95500b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955015f0b0 con 0x7f95500b5480 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9550161800 con 0x7f955015dcc0 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f9556800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b3b20 con 0x7f95500670b0 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 0x7f95500b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 0x7f95500b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:48290/0 (socket says 172.21.15.117:48290) 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 -- 172.21.15.117:0/1729455665 learned_addr learned my addr 172.21.15.117:0/1729455665 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954f400640 1 --2- 172.21.15.117:0/1729455665 >> v2:172.21.15.45:3300/0 conn(0x7f955015dcc0 0x7f9550162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 -- 172.21.15.117:0/1729455665 >> v2:172.21.15.45:3300/0 conn(0x7f955015dcc0 msgr2=0x7f9550162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 --2- 172.21.15.117:0/1729455665 >> v2:172.21.15.45:3300/0 conn(0x7f955015dcc0 0x7f9550162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 -- 172.21.15.117:0/1729455665 >> v1:172.21.15.45:6789/0 conn(0x7f95500b5480 legacy=0x7f95500669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.863+0000 7f954fe00640 1 -- 172.21.15.117:0/1729455665 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500b3e00 con 0x7f95500670b0 2024-04-25T10:17:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f954fe00640 1 --2- 172.21.15.117:0/1729455665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 0x7f95500b58d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9548009e50 tx=0x7f954801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2899c2366b0426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f954ea00640 1 -- 172.21.15.117:0/1729455665 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480a58d0 con 0x7f95500670b0 2024-04-25T10:17:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f954ea00640 1 -- 172.21.15.117:0/1729455665 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95480a5a70 con 0x7f95500670b0 2024-04-25T10:17:37.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f954ea00640 1 -- 172.21.15.117:0/1729455665 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480a5e40 con 0x7f95500670b0 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 -- 172.21.15.117:0/1729455665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 msgr2=0x7f95500b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 --2- 172.21.15.117:0/1729455665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 0x7f95500b58d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9548009e50 tx=0x7f954801ca00 comp rx=0 tx=0).stop 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 -- 172.21.15.117:0/1729455665 shutdown_connections 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 --2- 172.21.15.117:0/1729455665 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500670b0 0x7f95500b58d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 --2- 172.21.15.117:0/1729455665 >> v2:172.21.15.45:3300/0 conn(0x7f955015dcc0 0x7f9550162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 -- 172.21.15.117:0/1729455665 >> 172.21.15.117:0/1729455665 conn(0x7f955006ac90 msgr2=0x7f955006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 -- 172.21.15.117:0/1729455665 shutdown_connections 2024-04-25T10:17:37.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.867+0000 7f9556800640 1 -- 172.21.15.117:0/1729455665 wait complete. 2024-04-25T10:17:37.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 Processor -- start 2024-04-25T10:17:37.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- start start 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95500670b0 0x7f9550155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500b5480 0x7f955014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 0x7f955014d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500d0ec0 con 0x7f95500670b0 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500d0d40 con 0x7f955015dcc0 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500d1040 con 0x7f95500b5480 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 0x7f955014d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9546a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500b5480 0x7f955014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 0x7f955014d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57998/0 (socket says 172.21.15.117:57998) 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954fe00640 1 -- 172.21.15.117:0/3618753043 learned_addr learned my addr 172.21.15.117:0/3618753043 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:37.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95500670b0 0x7f9550155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 -- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500b5480 msgr2=0x7f955014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500b5480 0x7f955014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 -- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 msgr2=0x7f955014d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 0x7f955014d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 -- 172.21.15.117:0/3618753043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500b32c0 con 0x7f95500670b0 2024-04-25T10:17:37.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954fe00640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 0x7f955014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9546a00640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500b5480 0x7f955014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f954f400640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95500670b0 0x7f9550155d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f954000e870 tx=0x7f954000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540017070 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000c940 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000ccf0 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9550156280 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500b2fe0 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f954001a070 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f9540013470 con 0x7f95500670b0 2024-04-25T10:17:37.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.871+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955004ef30 con 0x7f95500670b0 2024-04-25T10:17:37.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.875+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9540006020 con 0x7f95500670b0 2024-04-25T10:17:37.909 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:37.904+0000 7fd9d6119180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:17:37.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.943+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6b66be6-3e3f-4880-9aec-e3caa98db3fe", "id": 2} v 0) v1 -- 0x7f9550168970 con 0x7f95500670b0 2024-04-25T10:17:37.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.943+0000 7f9547e00640 1 -- 172.21.15.117:0/3618753043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6b66be6-3e3f-4880-9aec-e3caa98db3fe", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f95400065c0 con 0x7f95500670b0 2024-04-25T10:17:37.950 INFO:teuthology.orchestra.run.smithi117.stdout:2 2024-04-25T10:17:37.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95500670b0 msgr2=0x7f9550155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95500670b0 0x7f9550155d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f954000e870 tx=0x7f954000ed30 comp rx=0 tx=0).stop 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 shutdown_connections 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f955015dcc0 0x7f955014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95500b5480 0x7f955014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 --2- 172.21.15.117:0/3618753043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95500670b0 0x7f9550155d40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 >> 172.21.15.117:0/3618753043 conn(0x7f955006ac90 msgr2=0x7f9550159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 shutdown_connections 2024-04-25T10:17:37.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:37.947+0000 7f9556800640 1 -- 172.21.15.117:0/3618753043 wait complete. 2024-04-25T10:17:37.970 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4 3 2024-04-25T10:17:38.003 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:37.999+0000 7f66e6169180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:17:38.076 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:38.071+0000 7f66e6169180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:17:38.086 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:38.080+0000 7fd9d6119180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:17:38.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 Processor -- start 2024-04-25T10:17:38.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 -- start start 2024-04-25T10:17:38.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb4b815ab00 0x7fb4b806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb4b815f150 0x7fb4b80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b806e210 con 0x7fb4b80cf250 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b806f2b0 con 0x7fb4b815f150 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.131+0000 7fb4bd800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b806e4f0 con 0x7fb4b815ab00 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb4b815ab00 0x7fb4b806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb4b815ab00 0x7fb4b806d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58008/0 (socket says 172.21.15.117:58008) 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 -- 172.21.15.117:0/2362785673 learned_addr learned my addr 172.21.15.117:0/2362785673 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6000640 1 --2- 172.21.15.117:0/2362785673 >> v2:172.21.15.45:3300/0 conn(0x7fb4b815f150 0x7fb4b80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 -- 172.21.15.117:0/2362785673 >> v2:172.21.15.45:3300/0 conn(0x7fb4b815f150 msgr2=0x7fb4b80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 --2- 172.21.15.117:0/2362785673 >> v2:172.21.15.45:3300/0 conn(0x7fb4b815f150 0x7fb4b80ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 -- 172.21.15.117:0/2362785673 >> v1:172.21.15.117:6789/0 conn(0x7fb4b80cf250 legacy=0x7fb4b80713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 -- 172.21.15.117:0/2362785673 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b806e7d0 con 0x7fb4b815ab00 2024-04-25T10:17:38.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b6a00640 1 --2- 172.21.15.117:0/2362785673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b806d9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb4b806d080 tx=0x7fb4a801c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1599783646e0fb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b5600640 1 -- 172.21.15.117:0/2362785673 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80ab610 con 0x7fb4b815ab00 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b5600640 1 -- 172.21.15.117:0/2362785673 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a80b0070 con 0x7fb4b815ab00 2024-04-25T10:17:38.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.135+0000 7fb4b5600640 1 -- 172.21.15.117:0/2362785673 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80ab9d0 con 0x7fb4b815ab00 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- 172.21.15.117:0/2362785673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 msgr2=0x7fb4b806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 --2- 172.21.15.117:0/2362785673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b806d9e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb4b806d080 tx=0x7fb4a801c520 comp rx=0 tx=0).stop 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- 172.21.15.117:0/2362785673 shutdown_connections 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 --2- 172.21.15.117:0/2362785673 >> v2:172.21.15.45:3300/0 conn(0x7fb4b815f150 0x7fb4b80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 --2- 172.21.15.117:0/2362785673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b806d9e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- 172.21.15.117:0/2362785673 >> 172.21.15.117:0/2362785673 conn(0x7fb4b80b2380 msgr2=0x7fb4b80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- 172.21.15.117:0/2362785673 shutdown_connections 2024-04-25T10:17:38.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- 172.21.15.117:0/2362785673 wait complete. 2024-04-25T10:17:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 Processor -- start 2024-04-25T10:17:38.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- start start 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 0x7fb4b80b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b814fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b815f150 0x7fb4b81501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81678e0 con 0x7fb4b815f150 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80cf660 con 0x7fb4b815ab00 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4bd800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8167a60 con 0x7fb4b80cf250 2024-04-25T10:17:38.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 0x7fb4b80b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b814fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 0x7fb4b80b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:48302/0 (socket says 172.21.15.117:48302) 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 -- 172.21.15.117:0/459052076 learned_addr learned my addr 172.21.15.117:0/459052076 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b7400640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b815f150 0x7fb4b81501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 -- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 msgr2=0x7fb4b814fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b814fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 -- 172.21.15.117:0/459052076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b815f150 msgr2=0x7fb4b81501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b815f150 0x7fb4b81501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6a00640 1 -- 172.21.15.117:0/459052076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8150d70 con 0x7fb4b80cf250 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.139+0000 7fb4b6000640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b814fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:38.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4b6a00640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 0x7fb4b80b0450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4a801a150 tx=0x7fb4a80192c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80059a0 con 0x7fb4b80cf250 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8005b40 con 0x7fb4b80cf250 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8003930 con 0x7fb4b80cf250 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b8151050 con 0x7fb4b80cf250 2024-04-25T10:17:38.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b8151610 con 0x7fb4b80cf250 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb4a80bf020 con 0x7fb4b80cf250 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb4a80064b0 con 0x7fb4b80cf250 2024-04-25T10:17:38.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.143+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b804ef30 con 0x7fb4b80cf250 2024-04-25T10:17:38.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.147+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb4a80a8050 con 0x7fb4b80cf250 2024-04-25T10:17:38.160 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:38.156+0000 7fd9d6119180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:17:38.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.211+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4", "id": 3} v 0) v1 -- 0x7fb4b80af850 con 0x7fb4b80cf250 2024-04-25T10:17:38.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4aea00640 1 -- 172.21.15.117:0/459052076 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb4a8016110 con 0x7fb4b80cf250 2024-04-25T10:17:38.221 INFO:teuthology.orchestra.run.smithi117.stdout:3 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 msgr2=0x7fb4b80b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 0x7fb4b80b0450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb4a801a150 tx=0x7fb4a80192c0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 shutdown_connections 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4b815f150 0x7fb4b81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4b815ab00 0x7fb4b814fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 --2- 172.21.15.117:0/459052076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4b80cf250 0x7fb4b80b0450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 >> 172.21.15.117:0/459052076 conn(0x7fb4b80b2380 msgr2=0x7fb4b80ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 shutdown_connections 2024-04-25T10:17:38.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.215+0000 7fb4bd800640 1 -- 172.21.15.117:0/459052076 wait complete. 2024-04-25T10:17:38.239 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new cdeadede-062f-4fd2-9199-ab0c21659438 4 2024-04-25T10:17:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.375+0000 7f0f0f200640 1 Processor -- start 2024-04-25T10:17:38.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.375+0000 7f0f0f200640 1 -- start start 2024-04-25T10:17:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.375+0000 7f0f0f200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f1006c3a0 0x7f0f1006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.375+0000 7f0f0f200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0f10070190 0x7f0f1015a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.375+0000 7f0f0de00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f1006c3a0 0x7f0f1006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f10071780 0x7f0f100b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f1006c3a0 0x7f0f1006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58034/0 (socket says 172.21.15.117:58034) 2024-04-25T10:17:38.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 -- 172.21.15.117:0/297483857 learned_addr learned my addr 172.21.15.117:0/297483857 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0d400640 1 --2- 172.21.15.117:0/297483857 >> v2:172.21.15.45:3300/0 conn(0x7f0f10070190 0x7f0f1015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0e800640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f10071780 0x7f0f100b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f100cf330 con 0x7f0f10070190 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1006ccc0 con 0x7f0f1006c3a0 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100706e0 con 0x7f0f10071780 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 -- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f10071780 msgr2=0x7f0f100b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f10071780 0x7f0f100b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 -- 172.21.15.117:0/297483857 >> v2:172.21.15.45:3300/0 conn(0x7f0f10070190 msgr2=0x7f0f1015a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 --2- 172.21.15.117:0/297483857 >> v2:172.21.15.45:3300/0 conn(0x7f0f10070190 0x7f0f1015a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0de00640 1 -- 172.21.15.117:0/297483857 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100709c0 con 0x7f0f1006c3a0 2024-04-25T10:17:38.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.379+0000 7f0f0e800640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f10071780 0x7f0f100b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0de00640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1006c3a0 0x7f0f1006c780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009870 tx=0x7f0efc01c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6b621f8440b6810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0ca00640 1 -- 172.21.15.117:0/297483857 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0aa610 con 0x7f0f1006c3a0 2024-04-25T10:17:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0ca00640 1 -- 172.21.15.117:0/297483857 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc0af070 con 0x7f0f1006c3a0 2024-04-25T10:17:38.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0ca00640 1 -- 172.21.15.117:0/297483857 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0aa9d0 con 0x7f0f1006c3a0 2024-04-25T10:17:38.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1006c3a0 msgr2=0x7f0f1006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1006c3a0 0x7f0f1006c780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009870 tx=0x7f0efc01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 shutdown_connections 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f10071780 0x7f0f100b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 --2- 172.21.15.117:0/297483857 >> v2:172.21.15.45:3300/0 conn(0x7f0f10070190 0x7f0f1015a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 --2- 172.21.15.117:0/297483857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1006c3a0 0x7f0f1006c780 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 >> 172.21.15.117:0/297483857 conn(0x7f0f10066da0 msgr2=0x7f0f100671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 shutdown_connections 2024-04-25T10:17:38.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.391+0000 7f0f0f200640 1 -- 172.21.15.117:0/297483857 wait complete. 2024-04-25T10:17:38.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.395+0000 7f0f0f200640 1 Processor -- start 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.395+0000 7f0f0f200640 1 -- start start 2024-04-25T10:17:38.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10070190 0x7f0f1013c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f10071780 0x7f0f100b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1015b000 0x7f0f1015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100b2ee0 con 0x7f0f10070190 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1015ae80 con 0x7f0f10071780 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10070190 0x7f0f1013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10070190 0x7f0f1013c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.117:60580/0 (socket says 172.21.15.117:60580) 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0de00640 1 -- 172.21.15.117:0/1800191695 learned_addr learned my addr 172.21.15.117:0/1800191695 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1015b000 0x7f0f1015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0d400640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f10071780 0x7f0f100b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100b3060 con 0x7f0f1015b000 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 -- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f10071780 msgr2=0x7f0f100b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f10071780 0x7f0f100b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 -- 172.21.15.117:0/1800191695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10070190 msgr2=0x7f0f1013c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10070190 0x7f0f1013c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 -- 172.21.15.117:0/1800191695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10152ce0 con 0x7f0f1015b000 2024-04-25T10:17:38.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f0e800640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1015b000 0x7f0f1015b4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000bf30 tx=0x7f0f0000e730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00004220 con 0x7f0f1015b000 2024-04-25T10:17:38.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.399+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00010e90 con 0x7f0f1015b000 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.403+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00015860 con 0x7f0f1015b000 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.403+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f10152fc0 con 0x7f0f1015b000 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.403+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f10153580 con 0x7f0f1015b000 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.403+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0f00015a00 con 0x7f0f1015b000 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.403+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0f0001e400 con 0x7f0f1015b000 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.403+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f10075120 con 0x7f0f1015b000 2024-04-25T10:17:38.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.407+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0f0001a030 con 0x7f0f1015b000 2024-04-25T10:17:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.519+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cdeadede-062f-4fd2-9199-ab0c21659438", "id": 4} v 0) v1 -- 0x7f0f100af500 con 0x7f0f1015b000 2024-04-25T10:17:38.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f06000640 1 -- 172.21.15.117:0/1800191695 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cdeadede-062f-4fd2-9199-ab0c21659438", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0f00003620 con 0x7f0f1015b000 2024-04-25T10:17:38.533 INFO:teuthology.orchestra.run.smithi117.stdout:4 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1015b000 msgr2=0x7f0f1015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1015b000 0x7f0f1015b4b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000bf30 tx=0x7f0f0000e730 comp rx=0 tx=0).stop 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 shutdown_connections 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f1015b000 0x7f0f1015b4b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f10071780 0x7f0f100b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 --2- 172.21.15.117:0/1800191695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10070190 0x7f0f1013c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 >> 172.21.15.117:0/1800191695 conn(0x7f0f10066da0 msgr2=0x7f0f10067180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 shutdown_connections 2024-04-25T10:17:38.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.527+0000 7f0f0f200640 1 -- 172.21.15.117:0/1800191695 wait complete. 2024-04-25T10:17:38.554 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new f0e03339-ad7a-4105-a0c1-f9370002931c 5 2024-04-25T10:17:38.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 Processor -- start 2024-04-25T10:17:38.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 -- start start 2024-04-25T10:17:38.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f640cd9a0 0x7f6f640d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f6406bd10 0x7f6f64161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6406ac90 con 0x7f6f640d0ba0 2024-04-25T10:17:38.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6406b880 con 0x7f6f640cd9a0 2024-04-25T10:17:38.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f6be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6406af50 con 0x7f6f6406bd10 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f640cd9a0 0x7f6f640d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6f640d0ba0 0x7f6f6415d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:54178/0 (socket says 172.21.15.117:54178) 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f640cd9a0 0x7f6f640d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58054/0 (socket says 172.21.15.117:58054) 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 -- 172.21.15.117:0/1978755753 learned_addr learned my addr 172.21.15.117:0/1978755753 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f68a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f6406bd10 0x7f6f64161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 -- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f6406bd10 msgr2=0x7f6f64161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 --2- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f6406bd10 0x7f6f64161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 -- 172.21.15.117:0/1978755753 >> v1:172.21.15.117:6789/0 conn(0x7f6f640d0ba0 legacy=0x7f6f6415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.683+0000 7f6f69400640 1 -- 172.21.15.117:0/1978755753 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6406b230 con 0x7f6f640cd9a0 2024-04-25T10:17:38.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f69400640 1 --2- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f640cd9a0 0x7f6f640d0660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009870 tx=0x7f6f5801ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f65ef2b1cdb7b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f63e00640 1 -- 172.21.15.117:0/1978755753 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580b5070 con 0x7f6f640cd9a0 2024-04-25T10:17:38.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f63e00640 1 -- 172.21.15.117:0/1978755753 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f580a8a50 con 0x7f6f640cd9a0 2024-04-25T10:17:38.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f63e00640 1 -- 172.21.15.117:0/1978755753 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580a8e20 con 0x7f6f640cd9a0 2024-04-25T10:17:38.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 -- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f640cd9a0 msgr2=0x7f6f640d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f640cd9a0 0x7f6f640d0660 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009870 tx=0x7f6f5801ed50 comp rx=0 tx=0).stop 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 -- 172.21.15.117:0/1978755753 shutdown_connections 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f6406bd10 0x7f6f64161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1978755753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f640cd9a0 0x7f6f640d0660 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 -- 172.21.15.117:0/1978755753 >> 172.21.15.117:0/1978755753 conn(0x7f6f640d19a0 msgr2=0x7f6f640d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 -- 172.21.15.117:0/1978755753 shutdown_connections 2024-04-25T10:17:38.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 -- 172.21.15.117:0/1978755753 wait complete. 2024-04-25T10:17:38.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.687+0000 7f6f6be00640 1 Processor -- start 2024-04-25T10:17:38.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- start start 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 0x7f6f6409f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f640cd9a0 0x7f6f6414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f640d0ba0 0x7f6f641502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f64162660 con 0x7f6f640d0ba0 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640cdd80 con 0x7f6f6406bd10 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f64161e50 con 0x7f6f640cd9a0 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 0x7f6f6409f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 0x7f6f6409f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58060/0 (socket says 172.21.15.117:58060) 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 -- 172.21.15.117:0/1626093207 learned_addr learned my addr 172.21.15.117:0/1626093207 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f68a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f640cd9a0 0x7f6f6414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69e00640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f640d0ba0 0x7f6f641502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 -- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f640cd9a0 msgr2=0x7f6f6414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f640cd9a0 0x7f6f6414f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 -- 172.21.15.117:0/1626093207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f640d0ba0 msgr2=0x7f6f641502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f640d0ba0 0x7f6f641502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 -- 172.21.15.117:0/1626093207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64152f70 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69e00640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f640d0ba0 0x7f6f641502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f69400640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 0x7f6f6409f330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6f580b3040 tx=0x7f6f5801baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580c0070 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f580b0b70 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f641523f0 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580b5040 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f641529b0 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6f580c3070 con 0x7f6f6406bd10 2024-04-25T10:17:38.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f6f580bb9b0 con 0x7f6f6406bd10 2024-04-25T10:17:38.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.691+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6404ef30 con 0x7f6f6406bd10 2024-04-25T10:17:38.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.695+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6f580dcdd0 con 0x7f6f6406bd10 2024-04-25T10:17:38.722 INFO:tasks.ceph.mgr.x.smithi117.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:17:38.722 INFO:tasks.ceph.mgr.x.smithi117.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:17:38.722 INFO:tasks.ceph.mgr.x.smithi117.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:17:38.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.759+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0e03339-ad7a-4105-a0c1-f9370002931c", "id": 5} v 0) v1 -- 0x7f6f640b7920 con 0x7f6f6406bd10 2024-04-25T10:17:38.767 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:38.763+0000 7f66e6169180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:17:38.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.763+0000 7f6f61600640 1 -- 172.21.15.117:0/1626093207 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0e03339-ad7a-4105-a0c1-f9370002931c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6f580cb020 con 0x7f6f6406bd10 2024-04-25T10:17:38.770 INFO:teuthology.orchestra.run.smithi117.stdout:5 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 msgr2=0x7f6f6409f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 0x7f6f6409f330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6f580b3040 tx=0x7f6f5801baf0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 shutdown_connections 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f640d0ba0 0x7f6f641502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f640cd9a0 0x7f6f6414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 --2- 172.21.15.117:0/1626093207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6406bd10 0x7f6f6409f330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 >> 172.21.15.117:0/1626093207 conn(0x7f6f640d19a0 msgr2=0x7f6f640cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 shutdown_connections 2024-04-25T10:17:38.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.767+0000 7f6f6be00640 1 -- 172.21.15.117:0/1626093207 wait complete. 2024-04-25T10:17:38.789 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new adb80701-c6c3-4d37-9711-1b803f66a674 6 2024-04-25T10:17:38.835 INFO:tasks.ceph.mgr.y.smithi045.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:17:38.835 INFO:tasks.ceph.mgr.y.smithi045.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:17:38.835 INFO:tasks.ceph.mgr.y.smithi045.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:17:38.889 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:38.884+0000 7fd9d6119180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:17:38.922 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:38.915+0000 7f66e6169180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:17:38.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.919+0000 7f933da00640 1 Processor -- start 2024-04-25T10:17:38.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.919+0000 7f933da00640 1 -- start start 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f933da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f933815dcc0 0x7f9338162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f933da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f93380b5480 0x7f93380b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f933da00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933815f0b0 con 0x7f93380669a0 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f933da00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9338161800 con 0x7f933815dcc0 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f933da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93380b3b20 con 0x7f93380b5480 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f932e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f93380b5480 0x7f93380b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f9337400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f93380669a0 0x7f93380d0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:54190/0 (socket says 172.21.15.117:54190) 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f9337400640 1 -- 172.21.15.117:0/3200494102 learned_addr learned my addr 172.21.15.117:0/3200494102 (peer_addr_for_me v1:172.21.15.117:0/0) 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f9336a00640 1 --2- 172.21.15.117:0/3200494102 >> v2:172.21.15.45:3300/0 conn(0x7f933815dcc0 0x7f9338162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f932e000640 1 -- 172.21.15.117:0/3200494102 >> v2:172.21.15.45:3300/0 conn(0x7f933815dcc0 msgr2=0x7f9338162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f932e000640 1 --2- 172.21.15.117:0/3200494102 >> v2:172.21.15.45:3300/0 conn(0x7f933815dcc0 0x7f9338162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f932e000640 1 -- 172.21.15.117:0/3200494102 >> v1:172.21.15.117:6789/0 conn(0x7f93380669a0 legacy=0x7f93380d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f932e000640 1 -- 172.21.15.117:0/3200494102 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380b3e00 con 0x7f93380b5480 2024-04-25T10:17:38.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.923+0000 7f9336000640 1 -- 172.21.15.117:0/3200494102 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449771513 0 0) 0x7f933815f0b0 con 0x7f93380669a0 2024-04-25T10:17:38.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f932e000640 1 --2- 172.21.15.117:0/3200494102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93380b5480 0x7f93380b58d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f933806bce0 tx=0x7f932001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98870bef3251fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f9336000640 1 -- 172.21.15.117:0/3200494102 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200ab650 con 0x7f93380b5480 2024-04-25T10:17:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f9336000640 1 -- 172.21.15.117:0/3200494102 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93200b0070 con 0x7f93380b5480 2024-04-25T10:17:38.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f9336000640 1 -- 172.21.15.117:0/3200494102 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200aba10 con 0x7f93380b5480 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 -- 172.21.15.117:0/3200494102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93380b5480 msgr2=0x7f93380b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 --2- 172.21.15.117:0/3200494102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93380b5480 0x7f93380b58d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f933806bce0 tx=0x7f932001c450 comp rx=0 tx=0).stop 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 -- 172.21.15.117:0/3200494102 shutdown_connections 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 --2- 172.21.15.117:0/3200494102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93380b5480 0x7f93380b58d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 --2- 172.21.15.117:0/3200494102 >> v2:172.21.15.45:3300/0 conn(0x7f933815dcc0 0x7f9338162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 -- 172.21.15.117:0/3200494102 >> 172.21.15.117:0/3200494102 conn(0x7f933806ac90 msgr2=0x7f933806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 -- 172.21.15.117:0/3200494102 shutdown_connections 2024-04-25T10:17:38.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 -- 172.21.15.117:0/3200494102 wait complete. 2024-04-25T10:17:38.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 Processor -- start 2024-04-25T10:17:38.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.927+0000 7f933da00640 1 -- start start 2024-04-25T10:17:38.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 0x7f933814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93380b5480 0x7f9338155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f933815dcc0 0x7f9338156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93380b3900 con 0x7f93380b5480 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93380b3780 con 0x7f933815dcc0 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93380d1050 con 0x7f93380669a0 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 0x7f933814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 0x7f933814fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:48340/0 (socket says 172.21.15.117:48340) 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 -- 172.21.15.117:0/3894150198 learned_addr learned my addr 172.21.15.117:0/3894150198 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9337400640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f933815dcc0 0x7f9338156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f932e000640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93380b5480 0x7f9338155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 -- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f933815dcc0 msgr2=0x7f9338156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f933815dcc0 0x7f9338156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 -- 172.21.15.117:0/3894150198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93380b5480 msgr2=0x7f9338155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93380b5480 0x7f9338155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 -- 172.21.15.117:0/3894150198 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933815c680 con 0x7f93380669a0 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9337400640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f933815dcc0 0x7f9338156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:38.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f9336a00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 0x7f933814fa50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f932800e930 tx=0x7f932800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328017070 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932800ca00 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932800cdb0 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933815c960 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933815cf20 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f932801a070 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f932801cb80 con 0x7f93380669a0 2024-04-25T10:17:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.931+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933804ef30 con 0x7f93380669a0 2024-04-25T10:17:38.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:38.935+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f932801c590 con 0x7f93380669a0 2024-04-25T10:17:39.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.019+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "adb80701-c6c3-4d37-9711-1b803f66a674", "id": 6} v 0) v1 -- 0x7f9338168970 con 0x7f93380669a0 2024-04-25T10:17:39.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f932f400640 1 -- 172.21.15.117:0/3894150198 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adb80701-c6c3-4d37-9711-1b803f66a674", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f932801c770 con 0x7f93380669a0 2024-04-25T10:17:39.031 INFO:teuthology.orchestra.run.smithi117.stdout:6 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 msgr2=0x7f933814fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 0x7f933814fa50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f932800e930 tx=0x7f932800edf0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 shutdown_connections 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f933815dcc0 0x7f9338156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93380b5480 0x7f9338155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 --2- 172.21.15.117:0/3894150198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93380669a0 0x7f933814fa50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 >> 172.21.15.117:0/3894150198 conn(0x7f933806ac90 msgr2=0x7f93381575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 shutdown_connections 2024-04-25T10:17:39.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.027+0000 7f933da00640 1 -- 172.21.15.117:0/3894150198 wait complete. 2024-04-25T10:17:39.044 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:39.040+0000 7fd9d6119180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:17:39.050 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 445fb14c-32f4-4101-9d09-e7bcd2e969de 7 2024-04-25T10:17:39.121 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:39.115+0000 7f66e6169180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:17:39.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.183+0000 7fd613200640 1 Processor -- start 2024-04-25T10:17:39.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.183+0000 7fd613200640 1 -- start start 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd613200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd60c15df30 0x7fd60c1626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd613200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd613200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd60c15e310 0x7fd60c0af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd613200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60c162bf0 con 0x7fd60c15e310 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd613200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60c0af540 con 0x7fd60c15df30 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd613200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c06f000 con 0x7fd60c06df80 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd60c15df30 0x7fd60c1626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c0aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:48348/0 (socket says 172.21.15.117:48348) 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60b400640 1 -- 172.21.15.117:0/669825409 learned_addr learned my addr 172.21.15.117:0/669825409 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd611200640 1 --2- 172.21.15.117:0/669825409 >> v2:172.21.15.45:3300/0 conn(0x7fd60c15e310 0x7fd60c0af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60be00640 1 -- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 msgr2=0x7fd60c0aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60be00640 1 --2- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c0aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60be00640 1 -- 172.21.15.117:0/669825409 >> v2:172.21.15.45:3300/0 conn(0x7fd60c15e310 msgr2=0x7fd60c0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60be00640 1 --2- 172.21.15.117:0/669825409 >> v2:172.21.15.45:3300/0 conn(0x7fd60c15e310 0x7fd60c0af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60be00640 1 -- 172.21.15.117:0/669825409 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c0afa20 con 0x7fd60c15df30 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd60b400640 1 --2- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c0aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:39.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.187+0000 7fd611200640 1 --2- 172.21.15.117:0/669825409 >> v2:172.21.15.45:3300/0 conn(0x7fd60c15e310 0x7fd60c0af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:39.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd60be00640 1 --2- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15df30 0x7fd60c1626b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd60c073470 tx=0x7fd5fc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f66c133e892395d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd60aa00640 1 -- 172.21.15.117:0/669825409 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0ab610 con 0x7fd60c15df30 2024-04-25T10:17:39.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd60aa00640 1 -- 172.21.15.117:0/669825409 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc0b0070 con 0x7fd60c15df30 2024-04-25T10:17:39.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd60aa00640 1 -- 172.21.15.117:0/669825409 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0ab9d0 con 0x7fd60c15df30 2024-04-25T10:17:39.196 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:39.191+0000 7f66e6169180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:17:39.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 -- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15df30 msgr2=0x7fd60c1626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 --2- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15df30 0x7fd60c1626b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd60c073470 tx=0x7fd5fc01c450 comp rx=0 tx=0).stop 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 -- 172.21.15.117:0/669825409 shutdown_connections 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 --2- 172.21.15.117:0/669825409 >> v2:172.21.15.45:3300/0 conn(0x7fd60c15e310 0x7fd60c0af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 --2- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c0aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 --2- 172.21.15.117:0/669825409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15df30 0x7fd60c1626b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 -- 172.21.15.117:0/669825409 >> 172.21.15.117:0/669825409 conn(0x7fd60c0b1ca0 msgr2=0x7fd60c0b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 -- 172.21.15.117:0/669825409 shutdown_connections 2024-04-25T10:17:39.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 -- 172.21.15.117:0/669825409 wait complete. 2024-04-25T10:17:39.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 Processor -- start 2024-04-25T10:17:39.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.191+0000 7fd613200640 1 -- start start 2024-04-25T10:17:39.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60c15df30 0x7fd60c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15e310 0x7fd60c1458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c0cd2b0 con 0x7fd60c15df30 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c06ed20 con 0x7fd60c15e310 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60c0cd430 con 0x7fd60c06df80 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd611200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15e310 0x7fd60c1458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd611200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15e310 0x7fd60c1458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58094/0 (socket says 172.21.15.117:58094) 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd611200640 1 -- 172.21.15.117:0/2585223071 learned_addr learned my addr 172.21.15.117:0/2585223071 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:48360/0 (socket says 172.21.15.117:48360) 2024-04-25T10:17:39.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60b400640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60c15df30 0x7fd60c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 -- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15e310 msgr2=0x7fd60c1458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15e310 0x7fd60c1458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 -- 172.21.15.117:0/2585223071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60c15df30 msgr2=0x7fd60c15f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60c15df30 0x7fd60c15f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 -- 172.21.15.117:0/2585223071 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c072520 con 0x7fd60c06df80 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd60be00640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c071fe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0af6c0 tx=0x7fd5fc01ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0bf610 con 0x7fd60c06df80 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc0bf7d0 con 0x7fd60c06df80 2024-04-25T10:17:39.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0bfbc0 con 0x7fd60c06df80 2024-04-25T10:17:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60c0727f0 con 0x7fd60c06df80 2024-04-25T10:17:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60c072db0 con 0x7fd60c06df80 2024-04-25T10:17:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd5fc0bd020 con 0x7fd60c06df80 2024-04-25T10:17:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fd5fc0b4a90 con 0x7fd60c06df80 2024-04-25T10:17:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.195+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd60c150de0 con 0x7fd60c06df80 2024-04-25T10:17:39.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.199+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd5fc0c6070 con 0x7fd60c06df80 2024-04-25T10:17:39.231 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:39.228+0000 7fd9d6119180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:17:39.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.263+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "445fb14c-32f4-4101-9d09-e7bcd2e969de", "id": 7} v 0) v1 -- 0x7fd60c04ef30 con 0x7fd60c06df80 2024-04-25T10:17:39.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd603e00640 1 -- 172.21.15.117:0/2585223071 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "445fb14c-32f4-4101-9d09-e7bcd2e969de", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd5fc0a8050 con 0x7fd60c06df80 2024-04-25T10:17:39.276 INFO:teuthology.orchestra.run.smithi117.stdout:7 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 msgr2=0x7fd60c071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c071fe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0af6c0 tx=0x7fd5fc01ac90 comp rx=0 tx=0).stop 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 shutdown_connections 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd60c15e310 0x7fd60c1458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60c15df30 0x7fd60c15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 --2- 172.21.15.117:0/2585223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd60c06df80 0x7fd60c071fe0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 >> 172.21.15.117:0/2585223071 conn(0x7fd60c0b1ca0 msgr2=0x7fd60c0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 shutdown_connections 2024-04-25T10:17:39.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-25T10:17:39.271+0000 7fd613200640 1 -- 172.21.15.117:0/2585223071 wait complete. 2024-04-25T10:17:39.295 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:17:39.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:17:39.298 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:17:39.298 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:17:39.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:17:39.304 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:17:39.304 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:17:39.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:17:39.307 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:17:39.307 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:17:39.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:17:39.312 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:17:39.312 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:17:39.312 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:17:39.312 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:39.304+0000 7fd9d6119180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:17:39.315 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:17:39.315 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:17:39.315 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:17:39.319 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:17:39.319 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:17:39.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:17:39.322 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:17:39.322 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:17:39.322 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:17:39.327 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:17:39.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:39.366 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:39.363+0000 7f66e6169180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:17:39.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0db200640 1 Processor -- start 2024-04-25T10:17:39.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0db200640 1 -- start start 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0db200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff0d40b15b0 0x7ff0d4168990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0db200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d40b1af0 con 0x7ff0d40d1820 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0db200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d40b1dd0 con 0x7ff0d415e050 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0db200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40b20b0 con 0x7ff0d40b15b0 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0d40d1820 0x7ff0d40d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37932/0 (socket says 172.21.15.45:37932) 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d3e00640 1 -- 172.21.15.45:0/761832569 learned_addr learned my addr 172.21.15.45:0/761832569 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:39.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d9200640 1 --2- 172.21.15.45:0/761832569 >> v2:172.21.15.117:3300/0 conn(0x7ff0d40b15b0 0x7ff0d4168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d3400640 1 -- 172.21.15.45:0/761832569 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378896535 0 0) 0x7ff0d40b1af0 con 0x7ff0d40d1820 2024-04-25T10:17:39.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d3400640 1 -- 172.21.15.45:0/761832569 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b8003530 con 0x7ff0d40d1820 2024-04-25T10:17:39.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d9200640 1 -- 172.21.15.45:0/761832569 >> v1:172.21.15.117:6789/0 conn(0x7ff0d415e050 legacy=0x7ff0d40b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d3400640 1 -- 172.21.15.45:0/761832569 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874292417 0 0) 0x7ff0d40b1dd0 con 0x7ff0d415e050 2024-04-25T10:17:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d9200640 1 -- 172.21.15.45:0/761832569 >> v1:172.21.15.45:6789/0 conn(0x7ff0d40d1820 legacy=0x7ff0d40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.500+0000 7ff0d9200640 1 -- 172.21.15.45:0/761832569 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40b2390 con 0x7ff0d40b15b0 2024-04-25T10:17:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0d9200640 1 --2- 172.21.15.45:0/761832569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d40b15b0 0x7ff0d4168990 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097c0 tx=0x7ff0cc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb6404be3995160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0d3400640 1 -- 172.21.15.45:0/761832569 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0b0070 con 0x7ff0d40b15b0 2024-04-25T10:17:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0d3400640 1 -- 172.21.15.45:0/761832569 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc0ab600 con 0x7ff0d40b15b0 2024-04-25T10:17:39.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0d3400640 1 -- 172.21.15.45:0/761832569 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0ab9d0 con 0x7ff0d40b15b0 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 -- 172.21.15.45:0/761832569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d40b15b0 msgr2=0x7ff0d4168990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 --2- 172.21.15.45:0/761832569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d40b15b0 0x7ff0d4168990 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097c0 tx=0x7ff0cc01c410 comp rx=0 tx=0).stop 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 -- 172.21.15.45:0/761832569 shutdown_connections 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 --2- 172.21.15.45:0/761832569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d40b15b0 0x7ff0d4168990 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 -- 172.21.15.45:0/761832569 >> 172.21.15.45:0/761832569 conn(0x7ff0d409a9d0 msgr2=0x7ff0d409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 -- 172.21.15.45:0/761832569 shutdown_connections 2024-04-25T10:17:39.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.504+0000 7ff0db200640 1 -- 172.21.15.45:0/761832569 wait complete. 2024-04-25T10:17:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 Processor -- start 2024-04-25T10:17:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 -- start start 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 0x7ff0d415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d40d1820 0x7ff0d415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d415e050 0x7ff0d41554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d416bda0 con 0x7ff0d40b15b0 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d416bc20 con 0x7ff0d415e050 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d416bf20 con 0x7ff0d40d1820 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 0x7ff0d415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 0x7ff0d415a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44464/0 (socket says 172.21.15.45:44464) 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 -- 172.21.15.45:0/1498953188 learned_addr learned my addr 172.21.15.45:0/1498953188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0cb400640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d40d1820 0x7ff0d415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d9200640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d415e050 0x7ff0d41554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 -- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d40d1820 msgr2=0x7ff0d415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d40d1820 0x7ff0d415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 -- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d415e050 msgr2=0x7ff0d41554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d415e050 0x7ff0d41554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 -- 172.21.15.45:0/1498953188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d415f880 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d3e00640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 0x7ff0d415a770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c450 tx=0x7ff0c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4017070 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4005cf0 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007290 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d415fb60 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d415b850 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.508+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff0c401a070 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.512+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff0c4007430 con 0x7ff0d40b15b0 2024-04-25T10:17:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.512+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d404ef30 con 0x7ff0d40b15b0 2024-04-25T10:17:39.517 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:39.512+0000 7fd9d6119180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:17:39.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.512+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff0c4021080 con 0x7ff0d40b15b0 2024-04-25T10:17:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0d406b480 con 0x7ff0d40b15b0 2024-04-25T10:17:39.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0d0c00640 1 -- 172.21.15.45:0/1498953188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7ff0c401c440 con 0x7ff0d40b15b0 2024-04-25T10:17:39.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:17:39.578 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:17:39.269192+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 msgr2=0x7ff0d415a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 0x7ff0d415a770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c450 tx=0x7ff0c400c910 comp rx=0 tx=0).stop 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 shutdown_connections 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0d415e050 0x7ff0d41554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0d40d1820 0x7ff0d415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 --2- 172.21.15.45:0/1498953188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d40b15b0 0x7ff0d415a770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 >> 172.21.15.45:0/1498953188 conn(0x7ff0d409a9d0 msgr2=0x7ff0d406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 shutdown_connections 2024-04-25T10:17:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:39.572+0000 7ff0db200640 1 -- 172.21.15.45:0/1498953188 wait complete. 2024-04-25T10:17:39.597 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:17:39.597 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:17:39.976 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-04-25T10:17:39.972+0000 7fcd8c1037c0 -1 Falling back to public interface 2024-04-25T10:17:39.988 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-04-25T10:17:39.983+0000 7f89511bd7c0 -1 Falling back to public interface 2024-04-25T10:17:39.988 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-04-25T10:17:39.983+0000 7fcbd882b7c0 -1 Falling back to public interface 2024-04-25T10:17:39.995 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-04-25T10:17:39.992+0000 7f030782c7c0 -1 Falling back to public interface 2024-04-25T10:17:39.999 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-04-25T10:17:39.996+0000 7efe90b8f7c0 -1 Falling back to public interface 2024-04-25T10:17:40.000 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-04-25T10:17:39.996+0000 7f700bad47c0 -1 Falling back to public interface 2024-04-25T10:17:40.008 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-04-25T10:17:40.003+0000 7fde886d67c0 -1 Falling back to public interface 2024-04-25T10:17:40.008 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-04-25T10:17:40.003+0000 7f3688eab7c0 -1 Falling back to public interface 2024-04-25T10:17:40.090 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:40.083+0000 7f66e6169180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:17:40.175 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:40.171+0000 7f66e6169180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:17:40.261 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:40.256+0000 7fd9d6119180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:17:40.352 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:40.348+0000 7fd9d6119180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:17:40.375 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:40.371+0000 7f66e6169180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:17:40.402 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:40.458 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:40.451+0000 7f66e6169180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:17:40.565 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:40.560+0000 7fd9d6119180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:17:40.573 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.568+0000 7fdad6600640 1 Processor -- start 2024-04-25T10:17:40.573 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.568+0000 7fdad6600640 1 -- start start 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdad6600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdad00b4a30 0x7fdad00d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdad6600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad0147520 con 0x7fdad00d0a80 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdad6600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad006c6a0 con 0x7fdad0147140 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdad6600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad0161f60 con 0x7fdad00b4a30 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdac7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdad00d0a80 0x7fdad006cb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37934/0 (socket says 172.21.15.45:37934) 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdac7e00640 1 -- 172.21.15.45:0/2614870396 learned_addr learned my addr 172.21.15.45:0/2614870396 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:40.575 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacfe00640 1 --2- 172.21.15.45:0/2614870396 >> v2:172.21.15.117:3300/0 conn(0x7fdad00b4a30 0x7fdad00d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259363639 0 0) 0x7fdab8001120 con 0x7fdad00d0a80 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab4003530 con 0x7fdad00d0a80 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897893996 0 0) 0x7fdad006c6a0 con 0x7fdad0147140 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab8001120 con 0x7fdad0147140 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacfe00640 1 -- 172.21.15.45:0/2614870396 >> v1:172.21.15.117:6789/0 conn(0x7fdad0147140 legacy=0x7fdad00d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacfe00640 1 -- 172.21.15.45:0/2614870396 >> v1:172.21.15.45:6789/0 conn(0x7fdad00d0a80 legacy=0x7fdad006cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.576 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.572+0000 7fdacfe00640 1 -- 172.21.15.45:0/2614870396 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0162240 con 0x7fdad00b4a30 2024-04-25T10:17:40.579 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdacfe00640 1 --2- 172.21.15.45:0/2614870396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00b4a30 0x7fdad00d2390 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009d90 tx=0x7fdac801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82686ed8a2eb2564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.579 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80b2070 con 0x7fdad00b4a30 2024-04-25T10:17:40.579 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac80a5a00 con 0x7fdad00b4a30 2024-04-25T10:17:40.579 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdacea00640 1 -- 172.21.15.45:0/2614870396 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80a5dd0 con 0x7fdad00b4a30 2024-04-25T10:17:40.579 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 -- 172.21.15.45:0/2614870396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00b4a30 msgr2=0x7fdad00d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.580 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 --2- 172.21.15.45:0/2614870396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00b4a30 0x7fdad00d2390 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009d90 tx=0x7fdac801c940 comp rx=0 tx=0).stop 2024-04-25T10:17:40.580 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 -- 172.21.15.45:0/2614870396 shutdown_connections 2024-04-25T10:17:40.580 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 --2- 172.21.15.45:0/2614870396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00b4a30 0x7fdad00d2390 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.580 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 -- 172.21.15.45:0/2614870396 >> 172.21.15.45:0/2614870396 conn(0x7fdad006ef40 msgr2=0x7fdad006f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.580 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 -- 172.21.15.45:0/2614870396 shutdown_connections 2024-04-25T10:17:40.580 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 -- 172.21.15.45:0/2614870396 wait complete. 2024-04-25T10:17:40.581 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 Processor -- start 2024-04-25T10:17:40.581 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 -- start start 2024-04-25T10:17:40.583 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad00b4a30 0x7fdad0140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.583 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.576+0000 7fdad6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00d0a80 0x7fdad013e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.583 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 0x7fdad0143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.583 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad016bfb0 con 0x7fdad0147140 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad016be30 con 0x7fdad00d0a80 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad016c130 con 0x7fdad00b4a30 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 0x7fdad0143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 0x7fdad0143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44480/0 (socket says 172.21.15.45:44480) 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 -- 172.21.15.45:0/1224624 learned_addr learned my addr 172.21.15.45:0/1224624 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdac7e00640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00d0a80 0x7fdad013e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacf400640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad00b4a30 0x7fdad0140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 -- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad00b4a30 msgr2=0x7fdad0140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad00b4a30 0x7fdad0140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 -- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00d0a80 msgr2=0x7fdad013e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00d0a80 0x7fdad013e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 -- 172.21.15.45:0/1224624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad00ced20 con 0x7fdad0147140 2024-04-25T10:17:40.584 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdacfe00640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 0x7fdad0143680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdac801a570 tx=0x7fdac80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.585 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80b2050 con 0x7fdad0147140 2024-04-25T10:17:40.585 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac80be070 con 0x7fdad0147140 2024-04-25T10:17:40.585 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad00cefe0 con 0x7fdad0147140 2024-04-25T10:17:40.586 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80ada50 con 0x7fdad0147140 2024-04-25T10:17:40.586 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad015aa20 con 0x7fdad0147140 2024-04-25T10:17:40.586 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdac80adbf0 con 0x7fdad0147140 2024-04-25T10:17:40.586 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fdac80a5980 con 0x7fdad0147140 2024-04-25T10:17:40.586 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.580+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad004ef30 con 0x7fdad0147140 2024-04-25T10:17:40.587 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.584+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdac8016560 con 0x7fdad0147140 2024-04-25T10:17:40.646 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdad006fba0 con 0x7fdad0147140 2024-04-25T10:17:40.647 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdac7400640 1 -- 172.21.15.45:0/1224624 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fdac80c3d20 con 0x7fdad0147140 2024-04-25T10:17:40.647 INFO:teuthology.misc.health.smithi045.stdout: 2024-04-25T10:17:40.647 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":10,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:17:39.269192+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:40.648 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 msgr2=0x7fdad0143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.648 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 0x7fdad0143680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdac801a570 tx=0x7fdac80196e0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 shutdown_connections 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdad0147140 0x7fdad0143680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdad00d0a80 0x7fdad013e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 --2- 172.21.15.45:0/1224624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdad00b4a30 0x7fdad0140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 >> 172.21.15.45:0/1224624 conn(0x7fdad006ef40 msgr2=0x7fdad01596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 shutdown_connections 2024-04-25T10:17:40.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:40.644+0000 7fdad6600640 1 -- 172.21.15.45:0/1224624 wait complete. 2024-04-25T10:17:40.659 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:40.656+0000 7fd9d6119180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:17:40.665 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:17:40.785 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:40.779+0000 7f66e6169180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:17:40.856 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:40.851+0000 7f66e6169180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:17:41.007 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:41.004+0000 7fd9d6119180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:17:41.081 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-04-25T10:17:41.076+0000 7fd9d6119180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:17:41.879 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:41.875+0000 7f668c200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:17:41.879 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-04-25T10:17:41.875+0000 7f668c200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:17:43.867 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-04-25T10:17:43.863+0000 7f89511bd7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T10:17:43.921 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-04-25T10:17:43.916+0000 7fcd8c1037c0 -1 osd.3 0 log_to_monitors true 2024-04-25T10:17:43.932 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-04-25T10:17:43.928+0000 7f700bad47c0 -1 osd.2 0 log_to_monitors true 2024-04-25T10:17:43.977 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-04-25T10:17:43.971+0000 7fcbd882b7c0 -1 osd.5 0 log_to_monitors true 2024-04-25T10:17:43.980 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-04-25T10:17:43.976+0000 7f030782c7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T10:17:43.994 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-04-25T10:17:43.987+0000 7fde886d67c0 -1 osd.6 0 log_to_monitors true 2024-04-25T10:17:44.107 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-04-25T10:17:44.103+0000 7f3688eab7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T10:17:44.107 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-04-25T10:17:44.104+0000 7efe90b8f7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T10:17:45.969 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-04-25T10:17:45.962+0000 7f894b600640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:17:46.058 INFO:tasks.ceph.osd.4.smithi117.stderr:2024-04-25T10:17:46.050+0000 7f8942000640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.471 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:47.576 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-04-25T10:17:47.572+0000 7efe8b000640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:17:47.577 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-04-25T10:17:47.572+0000 7f7006000640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:17:47.593 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-04-25T10:17:47.588+0000 7fcd86600640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:17:47.606 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-04-25T10:17:47.600+0000 7f0301200640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:17:47.612 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-04-25T10:17:47.608+0000 7efe81a00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.612 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-04-25T10:17:47.608+0000 7f6ffca00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.627 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-04-25T10:17:47.624+0000 7fcd7d000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.643 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-04-25T10:17:47.640+0000 7f02f7c00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.644+0000 7fde8b200640 1 Processor -- start 2024-04-25T10:17:47.649 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.644+0000 7fde8b200640 1 -- start start 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde8c15b800 0x7fde8c0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c06b2e0 0x7fde8c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8b200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c15f520 con 0x7fde8c0af950 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c1600f0 con 0x7fde8c15b800 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c15f7c0 con 0x7fde8c06b2e0 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde89400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fde8c0af950 0x7fde8c06abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:58800/0 (socket says 172.21.15.45:58800) 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde89400640 1 -- 172.21.15.45:0/2995647677 learned_addr learned my addr 172.21.15.45:0/2995647677 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8a800640 1 --2- 172.21.15.45:0/2995647677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c06b2e0 0x7fde8c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.651 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde89e00640 1 --2- 172.21.15.45:0/2995647677 >> v2:172.21.15.117:3300/0 conn(0x7fde8c15b800 0x7fde8c0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.652 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde88a00640 1 -- 172.21.15.45:0/2995647677 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838856658 0 0) 0x7fde8c15f520 con 0x7fde8c0af950 2024-04-25T10:17:47.652 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde88a00640 1 -- 172.21.15.45:0/2995647677 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde70003530 con 0x7fde8c0af950 2024-04-25T10:17:47.652 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8a800640 1 -- 172.21.15.45:0/2995647677 >> v2:172.21.15.117:3300/0 conn(0x7fde8c15b800 msgr2=0x7fde8c0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.652 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8a800640 1 --2- 172.21.15.45:0/2995647677 >> v2:172.21.15.117:3300/0 conn(0x7fde8c15b800 0x7fde8c0afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.652 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8a800640 1 -- 172.21.15.45:0/2995647677 >> v1:172.21.15.117:6789/0 conn(0x7fde8c0af950 legacy=0x7fde8c06abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.652 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde8a800640 1 -- 172.21.15.45:0/2995647677 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c15faa0 con 0x7fde8c06b2e0 2024-04-25T10:17:47.653 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.648+0000 7fde89e00640 1 --2- 172.21.15.45:0/2995647677 >> v2:172.21.15.117:3300/0 conn(0x7fde8c15b800 0x7fde8c0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:47.655 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8a800640 1 --2- 172.21.15.45:0/2995647677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c06b2e0 0x7fde8c0ce8c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde8c160270 tx=0x7fde8401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed1660fdfc1dc8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.655 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde88a00640 1 -- 172.21.15.45:0/2995647677 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde840b2070 con 0x7fde8c06b2e0 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde88a00640 1 -- 172.21.15.45:0/2995647677 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde840a5870 con 0x7fde8c06b2e0 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde88a00640 1 -- 172.21.15.45:0/2995647677 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde840a5c20 con 0x7fde8c06b2e0 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- 172.21.15.45:0/2995647677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c06b2e0 msgr2=0x7fde8c0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 --2- 172.21.15.45:0/2995647677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c06b2e0 0x7fde8c0ce8c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde8c160270 tx=0x7fde8401c920 comp rx=0 tx=0).stop 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- 172.21.15.45:0/2995647677 shutdown_connections 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 --2- 172.21.15.45:0/2995647677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c06b2e0 0x7fde8c0ce8c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 --2- 172.21.15.45:0/2995647677 >> v2:172.21.15.117:3300/0 conn(0x7fde8c15b800 0x7fde8c0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- 172.21.15.45:0/2995647677 >> 172.21.15.45:0/2995647677 conn(0x7fde8c06bce0 msgr2=0x7fde8c06c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- 172.21.15.45:0/2995647677 shutdown_connections 2024-04-25T10:17:47.656 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- 172.21.15.45:0/2995647677 wait complete. 2024-04-25T10:17:47.657 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 Processor -- start 2024-04-25T10:17:47.657 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- start start 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c06b2e0 0x7fde8c157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c0af950 0x7fde8c143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 0x7fde8c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cf790 con 0x7fde8c15b800 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cf610 con 0x7fde8c06b2e0 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.652+0000 7fde8b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cf910 con 0x7fde8c0af950 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 0x7fde8c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 0x7fde8c151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52922/0 (socket says 172.21.15.45:52922) 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 -- 172.21.15.45:0/3401427951 learned_addr learned my addr 172.21.15.45:0/3401427951 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde89e00640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c06b2e0 0x7fde8c157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde89400640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c0af950 0x7fde8c143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 -- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c0af950 msgr2=0x7fde8c143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.659 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c0af950 0x7fde8c143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.660 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 -- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c06b2e0 msgr2=0x7fde8c157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.660 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c06b2e0 0x7fde8c157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.660 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 -- 172.21.15.45:0/3401427951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c15c6c0 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8a800640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 0x7fde8c151eb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fde840b0040 tx=0x7fde840196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde840b2050 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde840be070 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde840ad890 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c15c9a0 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c15cf60 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c073400 con 0x7fde8c15b800 2024-04-25T10:17:47.661 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.656+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7fde840ada30 con 0x7fde8c15b800 2024-04-25T10:17:47.662 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.660+0000 7fde82000640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fde6c066f00 0x7fde6c0693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.662 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.660+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7fde8411eb90 con 0x7fde8c15b800 2024-04-25T10:17:47.662 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.660+0000 7fde89e00640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fde6c066f00 0x7fde6c0693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.663 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.660+0000 7fde89e00640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fde6c066f00 0x7fde6c0693c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fde8c153780 tx=0x7fde7c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.665 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-04-25T10:17:47.658+0000 7fcbd2200640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:17:47.667 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.664+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde840c7020 con 0x7fde8c15b800 2024-04-25T10:17:47.696 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-04-25T10:17:47.690+0000 7f3683400640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:17:47.701 INFO:tasks.ceph.osd.5.smithi117.stderr:2024-04-25T10:17:47.694+0000 7fcbc8c00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.728 INFO:tasks.ceph.osd.7.smithi117.stderr:2024-04-25T10:17:47.722+0000 7f3679e00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.732 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-04-25T10:17:47.726+0000 7fde82c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:17:47.759 INFO:tasks.ceph.osd.6.smithi117.stderr:2024-04-25T10:17:47.754+0000 7fde79600640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:17:47.837 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.832+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde8c0721c0 con 0x7fde8c15b800 2024-04-25T10:17:47.837 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.832+0000 7fde82000640 1 -- 172.21.15.45:0/3401427951 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7555 (secure 0 0 0) 0x7fde84016540 con 0x7fde8c15b800 2024-04-25T10:17:47.838 INFO:teuthology.misc.health.smithi045.stdout: 2024-04-25T10:17:47.838 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":14,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:17:46.978479+0000","last_up_change":"2024-04-25T10:17:46.978479+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fde6c066f00 msgr2=0x7fde6c0693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fde6c066f00 0x7fde6c0693c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fde8c153780 tx=0x7fde7c009190 comp rx=0 tx=0).stop 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 msgr2=0x7fde8c151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 0x7fde8c151eb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fde840b0040 tx=0x7fde840196c0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 shutdown_connections 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fde6c066f00 0x7fde6c0693c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.840 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15b800 0x7fde8c151eb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.841 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde8c0af950 0x7fde8c143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.841 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 --2- 172.21.15.45:0/3401427951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde8c06b2e0 0x7fde8c157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.841 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 >> 172.21.15.45:0/3401427951 conn(0x7fde8c06bce0 msgr2=0x7fde8c156db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.841 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 shutdown_connections 2024-04-25T10:17:47.841 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:47.836+0000 7fde8b200640 1 -- 172.21.15.45:0/3401427951 wait complete. 2024-04-25T10:17:47.857 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-25T10:17:54.663 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:54.802 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.797+0000 7fd622800640 1 Processor -- start 2024-04-25T10:17:54.802 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.797+0000 7fd622800640 1 -- start start 2024-04-25T10:17:54.803 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd622800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0b46e0 0x7fd61c06d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.803 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd622800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd61c0cf890 0x7fd61c0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.803 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd622800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0b4bf0 con 0x7fd61c159640 2024-04-25T10:17:54.803 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd622800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0b4ed0 con 0x7fd61c0cf890 2024-04-25T10:17:54.804 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd622800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c06d4f0 con 0x7fd61c0b46e0 2024-04-25T10:17:54.804 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd61c159640 0x7fd61c15cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:39974/0 (socket says 172.21.15.45:39974) 2024-04-25T10:17:54.804 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61b400640 1 -- 172.21.15.45:0/1160308247 learned_addr learned my addr 172.21.15.45:0/1160308247 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:54.804 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd613e00640 1 --2- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0b46e0 0x7fd61c06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.804 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61be00640 1 --2- 172.21.15.45:0/1160308247 >> v2:172.21.15.117:3300/0 conn(0x7fd61c0cf890 0x7fd61c0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.805 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61aa00640 1 -- 172.21.15.45:0/1160308247 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878259668 0 0) 0x7fd61c0b4bf0 con 0x7fd61c159640 2024-04-25T10:17:54.805 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61aa00640 1 -- 172.21.15.45:0/1160308247 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd600003530 con 0x7fd61c159640 2024-04-25T10:17:54.805 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61be00640 1 -- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0b46e0 msgr2=0x7fd61c06d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.805 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61be00640 1 --2- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0b46e0 0x7fd61c06d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.805 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61be00640 1 -- 172.21.15.45:0/1160308247 >> v1:172.21.15.45:6789/0 conn(0x7fd61c159640 legacy=0x7fd61c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.805 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.801+0000 7fd61be00640 1 -- 172.21.15.45:0/1160308247 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c06cea0 con 0x7fd61c0cf890 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd61be00640 1 --2- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c0cf890 0x7fd61c0cfd40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd614009d90 tx=0x7fd61401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74834018d7c160e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd61aa00640 1 -- 172.21.15.45:0/1160308247 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140b2070 con 0x7fd61c0cf890 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd61aa00640 1 -- 172.21.15.45:0/1160308247 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6140a58b0 con 0x7fd61c0cf890 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd61aa00640 1 -- 172.21.15.45:0/1160308247 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140a5c80 con 0x7fd61c0cf890 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 -- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c0cf890 msgr2=0x7fd61c0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 --2- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c0cf890 0x7fd61c0cfd40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd614009d90 tx=0x7fd61401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 -- 172.21.15.45:0/1160308247 shutdown_connections 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 --2- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c0cf890 0x7fd61c0cfd40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 --2- 172.21.15.45:0/1160308247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0b46e0 0x7fd61c06d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 -- 172.21.15.45:0/1160308247 >> 172.21.15.45:0/1160308247 conn(0x7fd61c06abd0 msgr2=0x7fd61c06afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 -- 172.21.15.45:0/1160308247 shutdown_connections 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 -- 172.21.15.45:0/1160308247 wait complete. 2024-04-25T10:17:54.811 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.805+0000 7fd622800640 1 Processor -- start 2024-04-25T10:17:54.812 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- start start 2024-04-25T10:17:54.813 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 0x7fd61c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.813 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0cf890 0x7fd61c155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c159640 0x7fd61c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0b3550 con 0x7fd61c0b46e0 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0b33d0 con 0x7fd61c159640 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0b36d0 con 0x7fd61c0cf890 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 0x7fd61c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 0x7fd61c1531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52926/0 (socket says 172.21.15.45:52926) 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 -- 172.21.15.45:0/4237145254 learned_addr learned my addr 172.21.15.45:0/4237145254 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd613e00640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0cf890 0x7fd61c155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61be00640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c159640 0x7fd61c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 -- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0cf890 msgr2=0x7fd61c155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0cf890 0x7fd61c155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 -- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c159640 msgr2=0x7fd61c155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c159640 0x7fd61c155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.814 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 -- 172.21.15.45:0/4237145254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c1536f0 con 0x7fd61c0b46e0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd61b400640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 0x7fd61c1531b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c450 tx=0x7fd60c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017070 con 0x7fd61c0b46e0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c005cf0 con 0x7fd61c0b46e0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c007290 con 0x7fd61c0b46e0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c152110 con 0x7fd61c0b46e0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c1526d0 con 0x7fd61c0b46e0 2024-04-25T10:17:54.815 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.809+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c04ef30 con 0x7fd61c0b46e0 2024-04-25T10:17:54.816 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.813+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd60c01a070 con 0x7fd61c0b46e0 2024-04-25T10:17:54.816 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.813+0000 7fd613400640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5f8066f60 0x7fd5f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.816 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.813+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd60c084050 con 0x7fd61c0b46e0 2024-04-25T10:17:54.816 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.813+0000 7fd613e00640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5f8066f60 0x7fd5f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.816 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.813+0000 7fd613e00640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5f8066f60 0x7fd5f8069420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd61c158900 tx=0x7fd604009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.820 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.817+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd60c054d50 con 0x7fd61c0b46e0 2024-04-25T10:17:54.981 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.977+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd61c06e3b0 con 0x7fd61c0b46e0 2024-04-25T10:17:54.982 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd613400640 1 -- 172.21.15.45:0/4237145254 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fd60c058df0 con 0x7fd61c0b46e0 2024-04-25T10:17:54.982 INFO:teuthology.misc.health.smithi045.stdout: 2024-04-25T10:17:54.982 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":18,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:17:51.625493+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5f8066f60 msgr2=0x7fd5f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5f8066f60 0x7fd5f8069420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd61c158900 tx=0x7fd604009190 comp rx=0 tx=0).stop 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 msgr2=0x7fd61c1531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 0x7fd61c1531b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c450 tx=0x7fd60c00c910 comp rx=0 tx=0).stop 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 shutdown_connections 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5f8066f60 0x7fd5f8069420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.984 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd61c159640 0x7fd61c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.985 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd61c0cf890 0x7fd61c155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.985 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 --2- 172.21.15.45:0/4237145254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd61c0b46e0 0x7fd61c1531b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.985 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 >> 172.21.15.45:0/4237145254 conn(0x7fd61c06abd0 msgr2=0x7fd61c15d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.985 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 shutdown_connections 2024-04-25T10:17:54.985 INFO:teuthology.misc.health.smithi045.stderr:2024-04-25T10:17:54.981+0000 7fd622800640 1 -- 172.21.15.45:0/4237145254 wait complete. 2024-04-25T10:17:55.001 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:17:55.002 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:17:55.002 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:17:55.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:17:55.005 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:17:55.005 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:17:55.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:17:55.008 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:17:55.008 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:17:55.008 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:17:55.014 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:17:55.015 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:17:55.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:17:55.018 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:17:55.018 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:17:55.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:55.106 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:17:55.107 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:17:55.122 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:17:55.123 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:17:55.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.161+0000 7f46d2c00640 1 Processor -- start 2024-04-25T10:17:55.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.161+0000 7f46d2c00640 1 -- start start 2024-04-25T10:17:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d2c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d2c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc0ae610 con 0x7f46cc15f180 2024-04-25T10:17:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d2c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc0d27b0 con 0x7f46cc0cea00 2024-04-25T10:17:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d2c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0cf570 con 0x7f46cc0afde0 2024-04-25T10:17:55.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46cbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f46cc15f180 0x7f46cc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43450/0 (socket says 172.21.15.45:43450) 2024-04-25T10:17:55.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46cbe00640 1 -- 172.21.15.45:0/2505224643 learned_addr learned my addr 172.21.15.45:0/2505224643 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:55.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d0c00640 1 --2- 172.21.15.45:0/2505224643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125159585 0 0) 0x7f46cc0ae610 con 0x7f46cc15f180 2024-04-25T10:17:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b0003530 con 0x7f46cc15f180 2024-04-25T10:17:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949252474 0 0) 0x7f46cc0d27b0 con 0x7f46cc0cea00 2024-04-25T10:17:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc0ae610 con 0x7f46cc0cea00 2024-04-25T10:17:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d0c00640 1 -- 172.21.15.45:0/2505224643 >> v1:172.21.15.117:6789/0 conn(0x7f46cc0cea00 legacy=0x7f46cc0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d0c00640 1 -- 172.21.15.45:0/2505224643 >> v1:172.21.15.45:6789/0 conn(0x7f46cc15f180 legacy=0x7f46cc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.165+0000 7f46d0c00640 1 -- 172.21.15.45:0/2505224643 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0cf850 con 0x7f46cc0afde0 2024-04-25T10:17:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d0c00640 1 --2- 172.21.15.45:0/2505224643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc0ccc20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009dc0 tx=0x7f46c401c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8c62bd411c97c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40b2070 con 0x7f46cc0afde0 2024-04-25T10:17:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c40a58a0 con 0x7f46cc0afde0 2024-04-25T10:17:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46cb400640 1 -- 172.21.15.45:0/2505224643 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40a5c70 con 0x7f46cc0afde0 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 -- 172.21.15.45:0/2505224643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 msgr2=0x7f46cc0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 --2- 172.21.15.45:0/2505224643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc0ccc20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009dc0 tx=0x7f46c401c8d0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 -- 172.21.15.45:0/2505224643 shutdown_connections 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 --2- 172.21.15.45:0/2505224643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc0ccc20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 -- 172.21.15.45:0/2505224643 >> 172.21.15.45:0/2505224643 conn(0x7f46cc0d1900 msgr2=0x7f46cc0d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 -- 172.21.15.45:0/2505224643 shutdown_connections 2024-04-25T10:17:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 -- 172.21.15.45:0/2505224643 wait complete. 2024-04-25T10:17:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 Processor -- start 2024-04-25T10:17:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.169+0000 7f46d2c00640 1 -- start start 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc166390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46cc0cea00 0x7f46cc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46cc15f180 0x7f46cc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc169a30 con 0x7f46cc15f180 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1698b0 con 0x7f46cc0cea00 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc169bb0 con 0x7f46cc0afde0 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46cbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc166390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46cc15f180 0x7f46cc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46cbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc166390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57932/0 (socket says 172.21.15.45:57932) 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46cbe00640 1 -- 172.21.15.45:0/24309863 learned_addr learned my addr 172.21.15.45:0/24309863 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46c3e00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46cc0cea00 0x7f46cc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 -- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 msgr2=0x7f46cc166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc166390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 -- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46cc0cea00 msgr2=0x7f46cc15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46cc0cea00 0x7f46cc15df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 -- 172.21.15.45:0/24309863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0c6360 con 0x7f46cc15f180 2024-04-25T10:17:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d0c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46cc15f180 0x7f46cc15e490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46c40ac000 tx=0x7f46c4019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40bd020 con 0x7f46cc15f180 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c40b8470 con 0x7f46cc15f180 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc0c6640 con 0x7f46cc15f180 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40b2040 con 0x7f46cc15f180 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc0c6c00 con 0x7f46cc15f180 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.173+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc04ef30 con 0x7f46cc15f180 2024-04-25T10:17:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.177+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46c40c0020 con 0x7f46cc15f180 2024-04-25T10:17:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.177+0000 7f46c8c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46a8066f60 0x7f46a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.177+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f46c411f9c0 con 0x7f46cc15f180 2024-04-25T10:17:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.177+0000 7f46cbe00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46a8066f60 0x7f46a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.177+0000 7f46cbe00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46a8066f60 0x7f46a8069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000f70 tx=0x7f46bc007e20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.181+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46c40c7030 con 0x7f46cc15f180 2024-04-25T10:17:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.341+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46cc070270 con 0x7f46cc15f180 2024-04-25T10:17:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46c8c00640 1 -- 172.21.15.45:0/24309863 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f46c4016580 con 0x7f46cc15f180 2024-04-25T10:17:55.347 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:17:55.347 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:17:51.625493+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46a8066f60 msgr2=0x7f46a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46a8066f60 0x7f46a8069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000f70 tx=0x7f46bc007e20 comp rx=0 tx=0).stop 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46cc15f180 msgr2=0x7f46cc15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46cc15f180 0x7f46cc15e490 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46c40ac000 tx=0x7f46c4019670 comp rx=0 tx=0).stop 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 shutdown_connections 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46a8066f60 0x7f46a8069420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46cc15f180 0x7f46cc15e490 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46cc0cea00 0x7f46cc15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 --2- 172.21.15.45:0/24309863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46cc0afde0 0x7f46cc166390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 >> 172.21.15.45:0/24309863 conn(0x7f46cc0d1900 msgr2=0x7f46cc15cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 shutdown_connections 2024-04-25T10:17:55.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.345+0000 7f46d2c00640 1 -- 172.21.15.45:0/24309863 wait complete. 2024-04-25T10:17:55.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:55.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:55.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 Processor -- start 2024-04-25T10:17:55.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 -- start start 2024-04-25T10:17:55.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c159540 0x7f9b9c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9b9c0721f0 0x7f9b9c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c159920 con 0x7f9b9c0aebd0 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c0af030 con 0x7f9b9c0721f0 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba4800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c15c2f0 con 0x7f9b9c159540 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9b9c0aebd0 0x7f9b9c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43466/0 (socket says 172.21.15.45:43466) 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1400640 1 -- 172.21.15.45:0/29028117 learned_addr learned my addr 172.21.15.45:0/29028117 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/29028117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c159540 0x7f9b9c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba2800640 1 --2- 172.21.15.45:0/29028117 >> v2:172.21.15.117:3300/0 conn(0x7f9b9c0721f0 0x7f9b9c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba0a00640 1 -- 172.21.15.45:0/29028117 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371875137 0 0) 0x7f9b9c159920 con 0x7f9b9c0aebd0 2024-04-25T10:17:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba0a00640 1 -- 172.21.15.45:0/29028117 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b84003530 con 0x7f9b9c0aebd0 2024-04-25T10:17:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1e00640 1 -- 172.21.15.45:0/29028117 >> v2:172.21.15.117:3300/0 conn(0x7f9b9c0721f0 msgr2=0x7f9b9c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/29028117 >> v2:172.21.15.117:3300/0 conn(0x7f9b9c0721f0 0x7f9b9c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1e00640 1 -- 172.21.15.45:0/29028117 >> v1:172.21.15.45:6789/0 conn(0x7f9b9c0aebd0 legacy=0x7f9b9c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.509+0000 7f9ba1e00640 1 -- 172.21.15.45:0/29028117 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c15c5d0 con 0x7f9b9c159540 2024-04-25T10:17:55.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/29028117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c159540 0x7f9b9c0ae690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b900097c0 tx=0x7f9b9001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990a76c497cf4ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba0a00640 1 -- 172.21.15.45:0/29028117 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900a7070 con 0x7f9b9c159540 2024-04-25T10:17:55.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba0a00640 1 -- 172.21.15.45:0/29028117 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b900b0070 con 0x7f9b9c159540 2024-04-25T10:17:55.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba0a00640 1 -- 172.21.15.45:0/29028117 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900ab890 con 0x7f9b9c159540 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 -- 172.21.15.45:0/29028117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c159540 msgr2=0x7f9b9c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 --2- 172.21.15.45:0/29028117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c159540 0x7f9b9c0ae690 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b900097c0 tx=0x7f9b9001c470 comp rx=0 tx=0).stop 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 -- 172.21.15.45:0/29028117 shutdown_connections 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 --2- 172.21.15.45:0/29028117 >> v2:172.21.15.117:3300/0 conn(0x7f9b9c0721f0 0x7f9b9c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 --2- 172.21.15.45:0/29028117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c159540 0x7f9b9c0ae690 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 -- 172.21.15.45:0/29028117 >> 172.21.15.45:0/29028117 conn(0x7f9b9c0669a0 msgr2=0x7f9b9c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 -- 172.21.15.45:0/29028117 shutdown_connections 2024-04-25T10:17:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.513+0000 7f9ba4800640 1 -- 172.21.15.45:0/29028117 wait complete. 2024-04-25T10:17:55.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 Processor -- start 2024-04-25T10:17:55.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- start start 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c0721f0 0x7f9b9c154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b9c0aebd0 0x7f9b9c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 0x7f9b9c157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0710a0 con 0x7f9b9c159540 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c06c0d0 con 0x7f9b9c0aebd0 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c071220 con 0x7f9b9c0721f0 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 0x7f9b9c157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 0x7f9b9c157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43028/0 (socket says 172.21.15.45:43028) 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 -- 172.21.15.45:0/2405236686 learned_addr learned my addr 172.21.15.45:0/2405236686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:55.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba1400640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b9c0aebd0 0x7f9b9c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c0721f0 0x7f9b9c154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 -- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c0721f0 msgr2=0x7f9b9c154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c0721f0 0x7f9b9c154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 -- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b9c0aebd0 msgr2=0x7f9b9c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b9c0aebd0 0x7f9b9c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 -- 172.21.15.45:0/2405236686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0aea00 con 0x7f9b9c159540 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c0721f0 0x7f9b9c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba1400640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b9c0aebd0 0x7f9b9c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba2800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 0x7f9b9c157d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800efc0 tx=0x7f9b9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98018020 con 0x7f9b9c159540 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b98014480 con 0x7f9b9c159540 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98014870 con 0x7f9b9c159540 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c0b1640 con 0x7f9b9c159540 2024-04-25T10:17:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c0b1c00 con 0x7f9b9c159540 2024-04-25T10:17:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.517+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c0b22c0 con 0x7f9b9c159540 2024-04-25T10:17:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.521+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b9801b020 con 0x7f9b9c159540 2024-04-25T10:17:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.521+0000 7f9b96000640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b80066f60 0x7f9b80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.521+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b98085bf0 con 0x7f9b9c159540 2024-04-25T10:17:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.521+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b80066f60 0x7f9b80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.521+0000 7f9ba1e00640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b80066f60 0x7f9b80069420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b900af000 tx=0x7f9b90008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.525+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b9c0b22c0 con 0x7f9b9c159540 2024-04-25T10:17:55.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:55.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b9c166700 con 0x7f9b9c159540 2024-04-25T10:17:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9b96000640 1 -- 172.21.15.45:0/2405236686 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b98022030 con 0x7f9b9c159540 2024-04-25T10:17:55.691 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b80066f60 msgr2=0x7f9b80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b80066f60 0x7f9b80069420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9b900af000 tx=0x7f9b90008040 comp rx=0 tx=0).stop 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 msgr2=0x7f9b9c157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 0x7f9b9c157d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800efc0 tx=0x7f9b9800c490 comp rx=0 tx=0).stop 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 shutdown_connections 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b80066f60 0x7f9b80069420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b9c159540 0x7f9b9c157d40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b9c0aebd0 0x7f9b9c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 --2- 172.21.15.45:0/2405236686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b9c0721f0 0x7f9b9c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 >> 172.21.15.45:0/2405236686 conn(0x7f9b9c0669a0 msgr2=0x7f9b9c0717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 shutdown_connections 2024-04-25T10:17:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.689+0000 7f9ba4800640 1 -- 172.21.15.45:0/2405236686 wait complete. 2024-04-25T10:17:55.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:55.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.889+0000 7f9eb7200640 1 Processor -- start 2024-04-25T10:17:55.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.889+0000 7f9eb7200640 1 -- start start 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb7200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9eb806b9d0 0x7f9eb80b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb7200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb80b5780 con 0x7f9eb815db40 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb7200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb80b5a60 con 0x7f9eb806ac90 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb7200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80b5550 con 0x7f9eb806b9d0 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9eb815db40 0x7f9eb815df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43468/0 (socket says 172.21.15.45:43468) 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb5e00640 1 -- 172.21.15.45:0/95475632 learned_addr learned my addr 172.21.15.45:0/95475632 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb6800640 1 --2- 172.21.15.45:0/95475632 >> v2:172.21.15.45:3300/0 conn(0x7f9eb806b9d0 0x7f9eb80b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb5400640 1 -- 172.21.15.45:0/95475632 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660521869 0 0) 0x7f9eb80b5780 con 0x7f9eb815db40 2024-04-25T10:17:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb5400640 1 -- 172.21.15.45:0/95475632 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e9c003530 con 0x7f9eb815db40 2024-04-25T10:17:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb6800640 1 -- 172.21.15.45:0/95475632 >> v1:172.21.15.117:6789/0 conn(0x7f9eb806ac90 legacy=0x7f9eb806b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb5400640 1 -- 172.21.15.45:0/95475632 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503546555 0 0) 0x7f9eb80b5a60 con 0x7f9eb806ac90 2024-04-25T10:17:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb6800640 1 -- 172.21.15.45:0/95475632 >> v1:172.21.15.45:6789/0 conn(0x7f9eb815db40 legacy=0x7f9eb815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.893+0000 7f9eb6800640 1 -- 172.21.15.45:0/95475632 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80b5d40 con 0x7f9eb806b9d0 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb6800640 1 --2- 172.21.15.45:0/95475632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb80b5010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009d90 tx=0x7f9eb001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f1c6516861694e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb5400640 1 -- 172.21.15.45:0/95475632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00b4070 con 0x7f9eb806b9d0 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb5400640 1 -- 172.21.15.45:0/95475632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb00af600 con 0x7f9eb806b9d0 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb5400640 1 -- 172.21.15.45:0/95475632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00af9d0 con 0x7f9eb806b9d0 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 -- 172.21.15.45:0/95475632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 msgr2=0x7f9eb80b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 --2- 172.21.15.45:0/95475632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb80b5010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009d90 tx=0x7f9eb001f150 comp rx=0 tx=0).stop 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 -- 172.21.15.45:0/95475632 shutdown_connections 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 --2- 172.21.15.45:0/95475632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb80b5010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 -- 172.21.15.45:0/95475632 >> 172.21.15.45:0/95475632 conn(0x7f9eb80d17b0 msgr2=0x7f9eb80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 -- 172.21.15.45:0/95475632 shutdown_connections 2024-04-25T10:17:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 -- 172.21.15.45:0/95475632 wait complete. 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 Processor -- start 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.897+0000 7f9eb7200640 1 -- start start 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9eb806ac90 0x7f9eb81f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb8149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eb815db40 0x7f9eb8146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80b6820 con 0x7f9eb806b9d0 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80b6300 con 0x7f9eb815db40 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80b69a0 con 0x7f9eb806ac90 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb8149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb8149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43054/0 (socket says 172.21.15.45:43054) 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 -- 172.21.15.45:0/2411798519 learned_addr learned my addr 172.21.15.45:0/2411798519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb5e00640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9eb806ac90 0x7f9eb81f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb6800640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eb815db40 0x7f9eb8146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 -- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9eb806ac90 msgr2=0x7f9eb81f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9eb806ac90 0x7f9eb81f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 -- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eb815db40 msgr2=0x7f9eb8146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eb815db40 0x7f9eb8146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 -- 172.21.15.45:0/2411798519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb814a080 con 0x7f9eb806b9d0 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eafe00640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb8149b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c4d0 tx=0x7f9ea000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0017070 con 0x7f9eb806b9d0 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0005480 con 0x7f9eb806b9d0 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb81f38d0 con 0x7f9eb806b9d0 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0003840 con 0x7f9eb806b9d0 2024-04-25T10:17:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb81f4110 con 0x7f9eb806b9d0 2024-04-25T10:17:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb8075120 con 0x7f9eb806b9d0 2024-04-25T10:17:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.901+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ea001a070 con 0x7f9eb806b9d0 2024-04-25T10:17:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.905+0000 7f9eae000640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.905+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9ea00840c0 con 0x7f9eb806b9d0 2024-04-25T10:17:55.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.905+0000 7f9eb5e00640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.905+0000 7f9eb5e00640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e94066f60 0x7f9e94069420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007fd0 tx=0x7f9ea8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:55.909+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9eb81f38d0 con 0x7f9eb806b9d0 2024-04-25T10:17:56.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.105+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb80d21f0 con 0x7f9eb806b9d0 2024-04-25T10:17:56.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.105+0000 7f9eae000640 1 -- 172.21.15.45:0/2411798519 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f9eb814a080 con 0x7f9eb806b9d0 2024-04-25T10:17:56.108 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:17:56.108 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:17:56.108 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-04-25T10:17:56.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.105+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e94066f60 msgr2=0x7f9e94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.105+0000 7f9eb7200640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e94066f60 0x7f9e94069420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8007fd0 tx=0x7f9ea8009040 comp rx=0 tx=0).stop 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.105+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 msgr2=0x7f9eb8149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.105+0000 7f9eb7200640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb8149b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c4d0 tx=0x7f9ea000c990 comp rx=0 tx=0).stop 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 shutdown_connections 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9eb815db40 0x7f9eb8146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9eb806b9d0 0x7f9eb8149b40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 --2- 172.21.15.45:0/2411798519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9eb806ac90 0x7f9eb81f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 >> 172.21.15.45:0/2411798519 conn(0x7f9eb80d17b0 msgr2=0x7f9eb815df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 shutdown_connections 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:56.109+0000 7f9eb7200640 1 -- 172.21.15.45:0/2411798519 wait complete. 2024-04-25T10:17:58.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.281+0000 7fd737600640 1 Processor -- start 2024-04-25T10:17:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.281+0000 7fd737600640 1 -- start start 2024-04-25T10:17:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd737600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7300cf930 0x7fd7300cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd737600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 0x7fd7300ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd737600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7300d0250 con 0x7fd730070c20 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd737600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73015bb90 con 0x7fd7300cf930 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd737600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300ad840 con 0x7fd7300ae020 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd734c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7300cf930 0x7fd7300cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd727e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd730070c20 0x7fd73015e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43472/0 (socket says 172.21.15.45:43472) 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd727e00640 1 -- 172.21.15.45:0/2205552236 learned_addr learned my addr 172.21.15.45:0/2205552236 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd734c00640 1 -- 172.21.15.45:0/2205552236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 msgr2=0x7fd7300ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd734c00640 1 --2- 172.21.15.45:0/2205552236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 0x7fd7300ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd734c00640 1 -- 172.21.15.45:0/2205552236 >> v1:172.21.15.45:6789/0 conn(0x7fd730070c20 legacy=0x7fd73015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd734c00640 1 -- 172.21.15.45:0/2205552236 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73015e9f0 con 0x7fd7300cf930 2024-04-25T10:17:58.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.285+0000 7fd734c00640 1 --2- 172.21.15.45:0/2205552236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7300cf930 0x7fd7300cfd10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097c0 tx=0x7fd72001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49435999a452c49e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd72fe00640 1 -- 172.21.15.45:0/2205552236 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200a7070 con 0x7fd7300cf930 2024-04-25T10:17:58.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd72fe00640 1 -- 172.21.15.45:0/2205552236 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200b0070 con 0x7fd7300cf930 2024-04-25T10:17:58.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd72fe00640 1 -- 172.21.15.45:0/2205552236 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200ab890 con 0x7fd7300cf930 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- 172.21.15.45:0/2205552236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7300cf930 msgr2=0x7fd7300cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 --2- 172.21.15.45:0/2205552236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7300cf930 0x7fd7300cfd10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097c0 tx=0x7fd72001c470 comp rx=0 tx=0).stop 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- 172.21.15.45:0/2205552236 shutdown_connections 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 --2- 172.21.15.45:0/2205552236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 0x7fd7300ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 --2- 172.21.15.45:0/2205552236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7300cf930 0x7fd7300cfd10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- 172.21.15.45:0/2205552236 >> 172.21.15.45:0/2205552236 conn(0x7fd730066b70 msgr2=0x7fd730066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- 172.21.15.45:0/2205552236 shutdown_connections 2024-04-25T10:17:58.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- 172.21.15.45:0/2205552236 wait complete. 2024-04-25T10:17:58.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 Processor -- start 2024-04-25T10:17:58.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- start start 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 0x7fd7300ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 0x7fd7301542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7300cf930 0x7fd7300cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73016a8e0 con 0x7fd730070c20 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73006e130 con 0x7fd7300cf930 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd737600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73016aa60 con 0x7fd7300ae020 2024-04-25T10:17:58.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 0x7fd7300ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 0x7fd7300ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43072/0 (socket says 172.21.15.45:43072) 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 -- 172.21.15.45:0/3460710077 learned_addr learned my addr 172.21.15.45:0/3460710077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 -- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 msgr2=0x7fd7301542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 0x7fd7301542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 -- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7300cf930 msgr2=0x7fd7300cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.289+0000 7fd734c00640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7300cf930 0x7fd7300cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd734c00640 1 -- 172.21.15.45:0/3460710077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7300ccdd0 con 0x7fd730070c20 2024-04-25T10:17:58.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd734c00640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 0x7fd7300ced30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7200ad040 tx=0x7fd72001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200abe40 con 0x7fd730070c20 2024-04-25T10:17:58.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200bf410 con 0x7fd730070c20 2024-04-25T10:17:58.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200bf800 con 0x7fd730070c20 2024-04-25T10:17:58.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7300cf440 con 0x7fd730070c20 2024-04-25T10:17:58.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7301557a0 con 0x7fd730070c20 2024-04-25T10:17:58.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73004ef30 con 0x7fd730070c20 2024-04-25T10:17:58.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7200bd020 con 0x7fd730070c20 2024-04-25T10:17:58.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd72d600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd70c066f60 0x7fd70c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd72011f3d0 con 0x7fd730070c20 2024-04-25T10:17:58.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.293+0000 7fd727e00640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd70c066f60 0x7fd70c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.297+0000 7fd727e00640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd70c066f60 0x7fd70c069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd718000f70 tx=0x7fd718007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.297+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd720016090 con 0x7fd730070c20 2024-04-25T10:17:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7301621e0 con 0x7fd730070c20 2024-04-25T10:17:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd72d600640 1 -- 172.21.15.45:0/3460710077 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fd7200f6030 con 0x7fd730070c20 2024-04-25T10:17:58.496 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:17:58.496 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:17:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd70c066f60 msgr2=0x7fd70c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd70c066f60 0x7fd70c069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd718000f70 tx=0x7fd718007a90 comp rx=0 tx=0).stop 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 msgr2=0x7fd7300ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 0x7fd7300ced30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7200ad040 tx=0x7fd72001acb0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 shutdown_connections 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd70c066f60 0x7fd70c069420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7300cf930 0x7fd7300cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7300ae020 0x7fd7301542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 --2- 172.21.15.45:0/3460710077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd730070c20 0x7fd7300ced30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 >> 172.21.15.45:0/3460710077 conn(0x7fd730066b70 msgr2=0x7fd7300b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 shutdown_connections 2024-04-25T10:17:58.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.493+0000 7fd737600640 1 -- 172.21.15.45:0/3460710077 wait complete. 2024-04-25T10:17:58.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 Processor -- start 2024-04-25T10:17:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 -- start start 2024-04-25T10:17:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef3015ad40 0x7fef30073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef300cc600 con 0x7fef3015e760 2024-04-25T10:17:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef300cc8e0 con 0x7fef3015ad40 2024-04-25T10:17:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef35200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300b3030 con 0x7fef300b4b20 2024-04-25T10:17:58.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef25400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef3015e760 0x7fef30070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43478/0 (socket says 172.21.15.45:43478) 2024-04-25T10:17:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef25400640 1 -- 172.21.15.45:0/577971479 learned_addr learned my addr 172.21.15.45:0/577971479 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2e800640 1 --2- 172.21.15.45:0/577971479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2de00640 1 --2- 172.21.15.45:0/577971479 >> v2:172.21.15.117:3300/0 conn(0x7fef3015ad40 0x7fef30073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2d400640 1 -- 172.21.15.45:0/577971479 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934327029 0 0) 0x7fef300cc600 con 0x7fef3015e760 2024-04-25T10:17:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2d400640 1 -- 172.21.15.45:0/577971479 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef14003530 con 0x7fef3015e760 2024-04-25T10:17:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2e800640 1 -- 172.21.15.45:0/577971479 >> v2:172.21.15.117:3300/0 conn(0x7fef3015ad40 msgr2=0x7fef30073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2e800640 1 --2- 172.21.15.45:0/577971479 >> v2:172.21.15.117:3300/0 conn(0x7fef3015ad40 0x7fef30073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2e800640 1 -- 172.21.15.45:0/577971479 >> v1:172.21.15.45:6789/0 conn(0x7fef3015e760 legacy=0x7fef30070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.657+0000 7fef2e800640 1 -- 172.21.15.45:0/577971479 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300b3310 con 0x7fef300b4b20 2024-04-25T10:17:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef2e800640 1 --2- 172.21.15.45:0/577971479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3015ec10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef28009d90 tx=0x7fef2801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcd2ef40857d34b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef2d400640 1 -- 172.21.15.45:0/577971479 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280a66d0 con 0x7fef300b4b20 2024-04-25T10:17:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef2d400640 1 -- 172.21.15.45:0/577971479 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef280a6870 con 0x7fef300b4b20 2024-04-25T10:17:58.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef2d400640 1 -- 172.21.15.45:0/577971479 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280a6c40 con 0x7fef300b4b20 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 -- 172.21.15.45:0/577971479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 msgr2=0x7fef3015ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 --2- 172.21.15.45:0/577971479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3015ec10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef28009d90 tx=0x7fef2801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 -- 172.21.15.45:0/577971479 shutdown_connections 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 --2- 172.21.15.45:0/577971479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3015ec10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 --2- 172.21.15.45:0/577971479 >> v2:172.21.15.117:3300/0 conn(0x7fef3015ad40 0x7fef30073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 -- 172.21.15.45:0/577971479 >> 172.21.15.45:0/577971479 conn(0x7fef300d00a0 msgr2=0x7fef300d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 -- 172.21.15.45:0/577971479 shutdown_connections 2024-04-25T10:17:58.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.661+0000 7fef35200640 1 -- 172.21.15.45:0/577971479 wait complete. 2024-04-25T10:17:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 Processor -- start 2024-04-25T10:17:58.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- start start 2024-04-25T10:17:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3015ad40 0x7fef3015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 0x7fef300b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30167c20 con 0x7fef3015e760 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30167aa0 con 0x7fef3015ad40 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef30167da0 con 0x7fef300b4b20 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 0x7fef300b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 0x7fef300b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43074/0 (socket says 172.21.15.45:43074) 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 -- 172.21.15.45:0/3234152019 learned_addr learned my addr 172.21.15.45:0/3234152019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef25400640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3015ad40 0x7fef3015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2de00640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 -- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 msgr2=0x7fef3014b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3014b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 -- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3015ad40 msgr2=0x7fef3015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3015ad40 0x7fef3015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 -- 172.21.15.45:0/3234152019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30151db0 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef2e800640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 0x7fef300b0da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef280b0040 tx=0x7fef28019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280be070 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef280b8630 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef30152090 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280b8a40 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3014bc70 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.665+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3004ef30 con 0x7fef3015e760 2024-04-25T10:17:58.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.669+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef280c1070 con 0x7fef3015e760 2024-04-25T10:17:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.669+0000 7fef26a00640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef0c066f90 0x7fef0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.669+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef2811fce0 con 0x7fef3015e760 2024-04-25T10:17:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.669+0000 7fef2de00640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef0c066f90 0x7fef0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.669+0000 7fef2de00640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef0c066f90 0x7fef0c069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef20009b60 tx=0x7fef20009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.673+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef28016560 con 0x7fef3015e760 2024-04-25T10:17:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.833+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef3006f3f0 con 0x7fef3015e760 2024-04-25T10:17:58.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef26a00640 1 -- 172.21.15.45:0/3234152019 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fef280f0a80 con 0x7fef3015e760 2024-04-25T10:17:58.838 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:17:58.839 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:17:51.625493+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef0c066f90 msgr2=0x7fef0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef0c066f90 0x7fef0c069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef20009b60 tx=0x7fef20009190 comp rx=0 tx=0).stop 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 msgr2=0x7fef300b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 0x7fef300b0da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef280b0040 tx=0x7fef28019680 comp rx=0 tx=0).stop 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 shutdown_connections 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef0c066f90 0x7fef0c069450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015e760 0x7fef300b0da0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef3015ad40 0x7fef3015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 --2- 172.21.15.45:0/3234152019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef300b4b20 0x7fef3014b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 >> 172.21.15.45:0/3234152019 conn(0x7fef300d00a0 msgr2=0x7fef30070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 shutdown_connections 2024-04-25T10:17:58.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:58.837+0000 7fef35200640 1 -- 172.21.15.45:0/3234152019 wait complete. 2024-04-25T10:17:58.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:58.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 Processor -- start 2024-04-25T10:17:59.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 -- start start 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f381815b760 0x7f381806c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381806d380 con 0x7f381806c880 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381806cdc0 con 0x7f381815b760 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381806d0a0 con 0x7f381815bb40 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f380e000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f381806c880 0x7f38180cdad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43494/0 (socket says 172.21.15.45:43494) 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f380e000640 1 -- 172.21.15.45:0/2859008634 learned_addr learned my addr 172.21.15.45:0/2859008634 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817400640 1 --2- 172.21.15.45:0/2859008634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3816a00640 1 --2- 172.21.15.45:0/2859008634 >> v2:172.21.15.117:3300/0 conn(0x7f381815b760 0x7f381806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3816000640 1 -- 172.21.15.45:0/2859008634 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828999765 0 0) 0x7f381806d380 con 0x7f381806c880 2024-04-25T10:17:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3816000640 1 -- 172.21.15.45:0/2859008634 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37fc003530 con 0x7f381806c880 2024-04-25T10:17:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817400640 1 -- 172.21.15.45:0/2859008634 >> v2:172.21.15.117:3300/0 conn(0x7f381815b760 msgr2=0x7f381806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817400640 1 --2- 172.21.15.45:0/2859008634 >> v2:172.21.15.117:3300/0 conn(0x7f381815b760 0x7f381806c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817400640 1 -- 172.21.15.45:0/2859008634 >> v1:172.21.15.45:6789/0 conn(0x7f381806c880 legacy=0x7f38180cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3817400640 1 -- 172.21.15.45:0/2859008634 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180b2ad0 con 0x7f381815bb40 2024-04-25T10:17:59.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.001+0000 7f3816a00640 1 --2- 172.21.15.45:0/2859008634 >> v2:172.21.15.117:3300/0 conn(0x7f381815b760 0x7f381806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817400640 1 --2- 172.21.15.45:0/2859008634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b32d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3810009d90 tx=0x7f381001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281686cc1d7e0763 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3816000640 1 -- 172.21.15.45:0/2859008634 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100b2070 con 0x7f381815bb40 2024-04-25T10:17:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3816000640 1 -- 172.21.15.45:0/2859008634 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38100a5880 con 0x7f381815bb40 2024-04-25T10:17:59.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3816000640 1 -- 172.21.15.45:0/2859008634 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100a5c50 con 0x7f381815bb40 2024-04-25T10:17:59.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- 172.21.15.45:0/2859008634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 msgr2=0x7f38180b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 --2- 172.21.15.45:0/2859008634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b32d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3810009d90 tx=0x7f381001c8b0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- 172.21.15.45:0/2859008634 shutdown_connections 2024-04-25T10:17:59.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 --2- 172.21.15.45:0/2859008634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b32d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 --2- 172.21.15.45:0/2859008634 >> v2:172.21.15.117:3300/0 conn(0x7f381815b760 0x7f381806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- 172.21.15.45:0/2859008634 >> 172.21.15.45:0/2859008634 conn(0x7f38180669a0 msgr2=0x7f3818066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- 172.21.15.45:0/2859008634 shutdown_connections 2024-04-25T10:17:59.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- 172.21.15.45:0/2859008634 wait complete. 2024-04-25T10:17:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 Processor -- start 2024-04-25T10:17:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- start start 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381806c880 0x7f381814a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 0x7f3818071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.009+0000 7f3817e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181689b0 con 0x7f381815b760 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3817e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818168830 con 0x7f381806c880 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3817e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3818168b30 con 0x7f381815bb40 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 0x7f3818071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3816a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381806c880 0x7f381814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 0x7f3818071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43084/0 (socket says 172.21.15.45:43084) 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 -- 172.21.15.45:0/1589767732 learned_addr learned my addr 172.21.15.45:0/1589767732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3817400640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 -- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 msgr2=0x7f38180b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 -- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381806c880 msgr2=0x7f381814a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381806c880 0x7f381814a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 -- 172.21.15.45:0/1589767732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180af510 con 0x7f381815b760 2024-04-25T10:17:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380e000640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 0x7f3818071d20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f380000c450 tx=0x7f380000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800017070 con 0x7f381815b760 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800005cf0 con 0x7f381815b760 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180af7f0 con 0x7f381815b760 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800007290 con 0x7f381815b760 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180afdb0 con 0x7f381815b760 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818075120 con 0x7f381815b760 2024-04-25T10:17:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.013+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f380001a070 con 0x7f381815b760 2024-04-25T10:17:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.017+0000 7f380f400640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f37f4066f60 0x7f37f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.017+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3800083fe0 con 0x7f381815b760 2024-04-25T10:17:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.017+0000 7f3816a00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f37f4066f60 0x7f37f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.017+0000 7f3816a00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f37f4066f60 0x7f37f4069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38180b1e30 tx=0x7f3808009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.025+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3800054cf0 con 0x7f381815b760 2024-04-25T10:17:59.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.185+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f381806da40 con 0x7f381815b760 2024-04-25T10:17:59.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.185+0000 7f380f400640 1 -- 172.21.15.45:0/1589767732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3800058d90 con 0x7f381815b760 2024-04-25T10:17:59.189 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:17:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f37f4066f60 msgr2=0x7f37f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f37f4066f60 0x7f37f4069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38180b1e30 tx=0x7f3808009190 comp rx=0 tx=0).stop 2024-04-25T10:17:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 msgr2=0x7f3818071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 0x7f3818071d20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f380000c450 tx=0x7f380000c910 comp rx=0 tx=0).stop 2024-04-25T10:17:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 shutdown_connections 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f37f4066f60 0x7f37f4069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f381815bb40 0x7f38180b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381815b760 0x7f3818071d20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 --2- 172.21.15.45:0/1589767732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381806c880 0x7f381814a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 >> 172.21.15.45:0/1589767732 conn(0x7f38180669a0 msgr2=0x7f38180cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 shutdown_connections 2024-04-25T10:17:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.189+0000 7f3817e00640 1 -- 172.21.15.45:0/1589767732 wait complete. 2024-04-25T10:17:59.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:17:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb218400640 1 Processor -- start 2024-04-25T10:17:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb218400640 1 -- start start 2024-04-25T10:17:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb218400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb21006e0a0 0x7fb21015c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb218400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21006f450 con 0x7fb210072260 2024-04-25T10:17:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb218400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2100cccc0 con 0x7fb210073d10 2024-04-25T10:17:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb218400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2100cca70 con 0x7fb21006e0a0 2024-04-25T10:17:59.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb215000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb210072260 0x7fb21006e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43506/0 (socket says 172.21.15.45:43506) 2024-04-25T10:17:59.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb215000640 1 -- 172.21.15.45:0/238603012 learned_addr learned my addr 172.21.15.45:0/238603012 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb215a00640 1 --2- 172.21.15.45:0/238603012 >> v2:172.21.15.117:3300/0 conn(0x7fb21006e0a0 0x7fb21015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480695906 0 0) 0x7fb21006f450 con 0x7fb210072260 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f8003530 con 0x7fb210072260 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068648456 0 0) 0x7fb2100cccc0 con 0x7fb210073d10 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21006f450 con 0x7fb210073d10 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb215a00640 1 -- 172.21.15.45:0/238603012 >> v1:172.21.15.117:6789/0 conn(0x7fb210073d10 legacy=0x7fb2100cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047724167 0 0) 0x7fb1f8003530 con 0x7fb210072260 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb215a00640 1 -- 172.21.15.45:0/238603012 >> v1:172.21.15.45:6789/0 conn(0x7fb210072260 legacy=0x7fb21006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.349+0000 7fb215a00640 1 -- 172.21.15.45:0/238603012 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100ccfa0 con 0x7fb21006e0a0 2024-04-25T10:17:59.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb215a00640 1 --2- 172.21.15.45:0/238603012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21006e0a0 0x7fb21015c070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2040097c0 tx=0x7fb20401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce24d048ed3cf40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040b0070 con 0x7fb21006e0a0 2024-04-25T10:17:59.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2040a4e50 con 0x7fb21006e0a0 2024-04-25T10:17:59.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb20fe00640 1 -- 172.21.15.45:0/238603012 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc001680 con 0x7fb21006e0a0 2024-04-25T10:17:59.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 -- 172.21.15.45:0/238603012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21006e0a0 msgr2=0x7fb21015c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 --2- 172.21.15.45:0/238603012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21006e0a0 0x7fb21015c070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2040097c0 tx=0x7fb20401c330 comp rx=0 tx=0).stop 2024-04-25T10:17:59.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 -- 172.21.15.45:0/238603012 shutdown_connections 2024-04-25T10:17:59.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 --2- 172.21.15.45:0/238603012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21006e0a0 0x7fb21015c070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 -- 172.21.15.45:0/238603012 >> 172.21.15.45:0/238603012 conn(0x7fb2100d00a0 msgr2=0x7fb2100d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 -- 172.21.15.45:0/238603012 shutdown_connections 2024-04-25T10:17:59.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.353+0000 7fb218400640 1 -- 172.21.15.45:0/238603012 wait complete. 2024-04-25T10:17:59.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 Processor -- start 2024-04-25T10:17:59.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 -- start start 2024-04-25T10:17:59.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb21006e0a0 0x7fb2100b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 0x7fb2101488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb210073d10 0x7fb2100b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210169b60 con 0x7fb210072260 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101699e0 con 0x7fb210073d10 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb210169ce0 con 0x7fb21006e0a0 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 0x7fb2101488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 0x7fb2101488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43100/0 (socket says 172.21.15.45:43100) 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 -- 172.21.15.45:0/2154699322 learned_addr learned my addr 172.21.15.45:0/2154699322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215a00640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb21006e0a0 0x7fb2100b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb216400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb210073d10 0x7fb2100b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 -- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb21006e0a0 msgr2=0x7fb2100b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb21006e0a0 0x7fb2100b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 -- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb210073d10 msgr2=0x7fb2100b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb210073d10 0x7fb2100b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 -- 172.21.15.45:0/2154699322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100b4db0 con 0x7fb210072260 2024-04-25T10:17:59.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215a00640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb21006e0a0 0x7fb2100b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb215000640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 0x7fb2101488c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c780 tx=0x7fb1fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc005ae0 con 0x7fb210072260 2024-04-25T10:17:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc005c80 con 0x7fb210072260 2024-04-25T10:17:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc007270 con 0x7fb210072260 2024-04-25T10:17:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2100b5090 con 0x7fb210072260 2024-04-25T10:17:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.357+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2100b34b0 con 0x7fb210072260 2024-04-25T10:17:59.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.361+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21004ef30 con 0x7fb210072260 2024-04-25T10:17:59.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.361+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1fc007410 con 0x7fb210072260 2024-04-25T10:17:59.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.361+0000 7fb20d600640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f0066f60 0x7fb1f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.361+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb1fc084c90 con 0x7fb210072260 2024-04-25T10:17:59.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.361+0000 7fb215a00640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f0066f60 0x7fb1f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.361+0000 7fb215a00640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f0066f60 0x7fb1f0069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2040ad000 tx=0x7fb2040a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.365+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1fc055f50 con 0x7fb210072260 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb2101594e0 con 0x7fb210072260 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb20d600640 1 -- 172.21.15.45:0/2154699322 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb1fc055850 con 0x7fb210072260 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:17:59.531 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:17:59.532 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f0066f60 msgr2=0x7fb1f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f0066f60 0x7fb1f0069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2040ad000 tx=0x7fb2040a6000 comp rx=0 tx=0).stop 2024-04-25T10:17:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 msgr2=0x7fb2101488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 0x7fb2101488c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c780 tx=0x7fb1fc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 shutdown_connections 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f0066f60 0x7fb1f0069420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb210073d10 0x7fb2100b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210072260 0x7fb2101488c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 --2- 172.21.15.45:0/2154699322 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb21006e0a0 0x7fb2100b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 >> 172.21.15.45:0/2154699322 conn(0x7fb2100d00a0 msgr2=0x7fb2100b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 shutdown_connections 2024-04-25T10:17:59.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.529+0000 7fb218400640 1 -- 172.21.15.45:0/2154699322 wait complete. 2024-04-25T10:17:59.553 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:17:59.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:17:59.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 Processor -- start 2024-04-25T10:17:59.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 -- start start 2024-04-25T10:17:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc45415dd20 0x7fc4541622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540b2b00 0x7fc45406dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540cb240 con 0x7fc4540cac80 2024-04-25T10:17:59.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45406d760 con 0x7fc45415dd20 2024-04-25T10:17:59.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc458a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4540cb520 con 0x7fc4540b2b00 2024-04-25T10:17:59.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc44be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4540cac80 0x7fc4540b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43516/0 (socket says 172.21.15.45:43516) 2024-04-25T10:17:59.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc44be00640 1 -- 172.21.15.45:0/977869081 learned_addr learned my addr 172.21.15.45:0/977869081 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc453e00640 1 --2- 172.21.15.45:0/977869081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540b2b00 0x7fc45406dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc453400640 1 --2- 172.21.15.45:0/977869081 >> v2:172.21.15.117:3300/0 conn(0x7fc45415dd20 0x7fc4541622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc452a00640 1 -- 172.21.15.45:0/977869081 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878576530 0 0) 0x7fc4540cb240 con 0x7fc4540cac80 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc452a00640 1 -- 172.21.15.45:0/977869081 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc438003530 con 0x7fc4540cac80 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc452a00640 1 -- 172.21.15.45:0/977869081 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367482804 0 0) 0x7fc438003530 con 0x7fc4540cac80 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc453e00640 1 -- 172.21.15.45:0/977869081 >> v2:172.21.15.117:3300/0 conn(0x7fc45415dd20 msgr2=0x7fc4541622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc453e00640 1 --2- 172.21.15.45:0/977869081 >> v2:172.21.15.117:3300/0 conn(0x7fc45415dd20 0x7fc4541622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc453e00640 1 -- 172.21.15.45:0/977869081 >> v1:172.21.15.45:6789/0 conn(0x7fc4540cac80 legacy=0x7fc4540b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.697+0000 7fc453e00640 1 -- 172.21.15.45:0/977869081 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45406d110 con 0x7fc4540b2b00 2024-04-25T10:17:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc453e00640 1 --2- 172.21.15.45:0/977869081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540b2b00 0x7fc45406dbf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009d90 tx=0x7fc44c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e560a313c165db8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc452a00640 1 -- 172.21.15.45:0/977869081 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0b2070 con 0x7fc4540b2b00 2024-04-25T10:17:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc452a00640 1 -- 172.21.15.45:0/977869081 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c0a5510 con 0x7fc4540b2b00 2024-04-25T10:17:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc452a00640 1 -- 172.21.15.45:0/977869081 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c001680 con 0x7fc4540b2b00 2024-04-25T10:17:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 -- 172.21.15.45:0/977869081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540b2b00 msgr2=0x7fc45406dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 --2- 172.21.15.45:0/977869081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540b2b00 0x7fc45406dbf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009d90 tx=0x7fc44c01c890 comp rx=0 tx=0).stop 2024-04-25T10:17:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 -- 172.21.15.45:0/977869081 shutdown_connections 2024-04-25T10:17:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 --2- 172.21.15.45:0/977869081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540b2b00 0x7fc45406dbf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 --2- 172.21.15.45:0/977869081 >> v2:172.21.15.117:3300/0 conn(0x7fc45415dd20 0x7fc4541622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 -- 172.21.15.45:0/977869081 >> 172.21.15.45:0/977869081 conn(0x7fc4540cd4b0 msgr2=0x7fc4540cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 -- 172.21.15.45:0/977869081 shutdown_connections 2024-04-25T10:17:59.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.701+0000 7fc458a00640 1 -- 172.21.15.45:0/977869081 wait complete. 2024-04-25T10:17:59.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 Processor -- start 2024-04-25T10:17:59.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- start start 2024-04-25T10:17:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4540b2b00 0x7fc454152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540cac80 0x7fc454155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 0x7fc4540b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406ed10 con 0x7fc45415dd20 2024-04-25T10:17:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406eb90 con 0x7fc4540b2b00 2024-04-25T10:17:59.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406ee90 con 0x7fc4540cac80 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 0x7fc4540b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 0x7fc4540b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43108/0 (socket says 172.21.15.45:43108) 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 -- 172.21.15.45:0/367886603 learned_addr learned my addr 172.21.15.45:0/367886603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 -- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540cac80 msgr2=0x7fc454155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540cac80 0x7fc454155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 -- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4540b2b00 msgr2=0x7fc454152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4540b2b00 0x7fc454152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 -- 172.21.15.45:0/367886603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454155cc0 con 0x7fc45415dd20 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc453e00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 0x7fc4540b1130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc44c01d130 tx=0x7fc44c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0b2050 con 0x7fc45415dd20 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c0be070 con 0x7fc45415dd20 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c009450 con 0x7fc45415dd20 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc454155fa0 con 0x7fc45415dd20 2024-04-25T10:17:59.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4540adbd0 con 0x7fc45415dd20 2024-04-25T10:17:59.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.705+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc454075120 con 0x7fc45415dd20 2024-04-25T10:17:59.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.709+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc44c0b9850 con 0x7fc45415dd20 2024-04-25T10:17:59.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.709+0000 7fc44b400640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc430066f60 0x7fc430069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:59.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.709+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc44c11fce0 con 0x7fc45415dd20 2024-04-25T10:17:59.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.709+0000 7fc453400640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc430066f60 0x7fc430069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:59.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.709+0000 7fc453400640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc430066f60 0x7fc430069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc444006aa0 tx=0x7fc444006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:59.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.713+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc44c016480 con 0x7fc45415dd20 2024-04-25T10:17:59.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:17:59.877+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc4540b65b0 con 0x7fc45415dd20 2024-04-25T10:18:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc44b400640 1 -- 172.21.15.45:0/367886603 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc44c0f10c0 con 0x7fc45415dd20 2024-04-25T10:18:00.662 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc430066f60 msgr2=0x7fc430069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc430066f60 0x7fc430069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc444006aa0 tx=0x7fc444006a30 comp rx=0 tx=0).stop 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 msgr2=0x7fc4540b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 0x7fc4540b1130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc44c01d130 tx=0x7fc44c019630 comp rx=0 tx=0).stop 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 shutdown_connections 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc430066f60 0x7fc430069420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc45415dd20 0x7fc4540b1130 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4540cac80 0x7fc454155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 --2- 172.21.15.45:0/367886603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4540b2b00 0x7fc454152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 >> 172.21.15.45:0/367886603 conn(0x7fc4540cd4b0 msgr2=0x7fc454159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 shutdown_connections 2024-04-25T10:18:00.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.661+0000 7fc458a00640 1 -- 172.21.15.45:0/367886603 wait complete. 2024-04-25T10:18:00.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:18:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 Processor -- start 2024-04-25T10:18:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 -- start start 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff2540ccba0 0x7ff2540cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2540b5220 0x7ff2540b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2540d0970 con 0x7ff25415bc80 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540b43a0 con 0x7ff2540ccba0 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff25b000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540d03b0 con 0x7ff2540b5220 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff253e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff25415bc80 0x7ff2540cc490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43522/0 (socket says 172.21.15.45:43522) 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff253e00640 1 -- 172.21.15.45:0/537529782 learned_addr learned my addr 172.21.15.45:0/537529782 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff259000640 1 --2- 172.21.15.45:0/537529782 >> v2:172.21.15.117:3300/0 conn(0x7ff2540b5220 0x7ff2540b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.841+0000 7ff24be00640 1 --2- 172.21.15.45:0/537529782 >> v2:172.21.15.45:3300/0 conn(0x7ff2540ccba0 0x7ff2540cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff253400640 1 -- 172.21.15.45:0/537529782 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228111108 0 0) 0x7ff244001120 con 0x7ff25415bc80 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff253400640 1 -- 172.21.15.45:0/537529782 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff238003530 con 0x7ff25415bc80 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff24be00640 1 -- 172.21.15.45:0/537529782 >> v2:172.21.15.117:3300/0 conn(0x7ff2540b5220 msgr2=0x7ff2540b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff24be00640 1 --2- 172.21.15.45:0/537529782 >> v2:172.21.15.117:3300/0 conn(0x7ff2540b5220 0x7ff2540b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff24be00640 1 -- 172.21.15.45:0/537529782 >> v1:172.21.15.45:6789/0 conn(0x7ff25415bc80 legacy=0x7ff2540cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff24be00640 1 -- 172.21.15.45:0/537529782 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540d0690 con 0x7ff2540ccba0 2024-04-25T10:18:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff259000640 1 --2- 172.21.15.45:0/537529782 >> v2:172.21.15.117:3300/0 conn(0x7ff2540b5220 0x7ff2540b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff24be00640 1 --2- 172.21.15.45:0/537529782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540ccba0 0x7ff2540cd050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2540cde20 tx=0x7ff23c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e92c31a9043a83b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff253400640 1 -- 172.21.15.45:0/537529782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0ab650 con 0x7ff2540ccba0 2024-04-25T10:18:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff253400640 1 -- 172.21.15.45:0/537529782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c0b0070 con 0x7ff2540ccba0 2024-04-25T10:18:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.845+0000 7ff253400640 1 -- 172.21.15.45:0/537529782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0aba10 con 0x7ff2540ccba0 2024-04-25T10:18:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- 172.21.15.45:0/537529782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540ccba0 msgr2=0x7ff2540cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 --2- 172.21.15.45:0/537529782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540ccba0 0x7ff2540cd050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2540cde20 tx=0x7ff23c01c450 comp rx=0 tx=0).stop 2024-04-25T10:18:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- 172.21.15.45:0/537529782 shutdown_connections 2024-04-25T10:18:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 --2- 172.21.15.45:0/537529782 >> v2:172.21.15.117:3300/0 conn(0x7ff2540b5220 0x7ff2540b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 --2- 172.21.15.45:0/537529782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540ccba0 0x7ff2540cd050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- 172.21.15.45:0/537529782 >> 172.21.15.45:0/537529782 conn(0x7ff2540b3580 msgr2=0x7ff2540b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- 172.21.15.45:0/537529782 shutdown_connections 2024-04-25T10:18:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- 172.21.15.45:0/537529782 wait complete. 2024-04-25T10:18:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 Processor -- start 2024-04-25T10:18:00.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- start start 2024-04-25T10:18:00.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 0x7ff25414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2540ccba0 0x7ff2541535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25415bc80 0x7ff254153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540d0120 con 0x7ff2540b5220 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540cffa0 con 0x7ff25415bc80 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff25b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540cf140 con 0x7ff2540ccba0 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 0x7ff25414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 0x7ff25414e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43124/0 (socket says 172.21.15.45:43124) 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 -- 172.21.15.45:0/729090802 learned_addr learned my addr 172.21.15.45:0/729090802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff259000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25415bc80 0x7ff254153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff24be00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2540ccba0 0x7ff2541535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 -- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2540ccba0 msgr2=0x7ff2541535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2540ccba0 0x7ff2541535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 -- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25415bc80 msgr2=0x7ff254153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25415bc80 0x7ff254153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:00.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 -- 172.21.15.45:0/729090802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2541f7cd0 con 0x7ff2540b5220 2024-04-25T10:18:00.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff259000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25415bc80 0x7ff254153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:00.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.849+0000 7ff253e00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 0x7ff25414e620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff24400c3f0 tx=0x7ff24400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:00.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244017070 con 0x7ff2540b5220 2024-04-25T10:18:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244005cf0 con 0x7ff2540b5220 2024-04-25T10:18:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244007310 con 0x7ff2540b5220 2024-04-25T10:18:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2541f7fb0 con 0x7ff2540b5220 2024-04-25T10:18:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2541f8570 con 0x7ff2540b5220 2024-04-25T10:18:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25404ef30 con 0x7ff2540b5220 2024-04-25T10:18:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff24401a070 con 0x7ff2540b5220 2024-04-25T10:18:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff250c00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff230066f60 0x7ff230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff244084280 con 0x7ff2540b5220 2024-04-25T10:18:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.853+0000 7ff24be00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff230066f60 0x7ff230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:00.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.857+0000 7ff24be00640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff230066f60 0x7ff230069420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0af000 tx=0x7ff23c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:00.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:00.857+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2540d0970 con 0x7ff2540b5220 2024-04-25T10:18:01.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.025+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff25406eb50 con 0x7ff2540b5220 2024-04-25T10:18:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.665+0000 7ff250c00640 1 -- 172.21.15.45:0/729090802 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff244054d80 con 0x7ff2540b5220 2024-04-25T10:18:01.669 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff230066f60 msgr2=0x7ff230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff230066f60 0x7ff230069420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0af000 tx=0x7ff23c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 msgr2=0x7ff25414e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 0x7ff25414e620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff24400c3f0 tx=0x7ff24400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 shutdown_connections 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff230066f60 0x7ff230069420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff25415bc80 0x7ff254153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2540ccba0 0x7ff2541535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 --2- 172.21.15.45:0/729090802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2540b5220 0x7ff25414e620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 >> 172.21.15.45:0/729090802 conn(0x7ff2540b3580 msgr2=0x7ff2540ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 shutdown_connections 2024-04-25T10:18:01.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.669+0000 7ff25b000640 1 -- 172.21.15.45:0/729090802 wait complete. 2024-04-25T10:18:01.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:18:01.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.857+0000 7f869b600640 1 Processor -- start 2024-04-25T10:18:01.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.857+0000 7f869b600640 1 -- start start 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f869b600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f86940669a0 0x7f8694066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f869b600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940ccec0 0x7f869415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f869b600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940672c0 con 0x7f86940cd340 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f869b600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8694071d60 con 0x7f86940669a0 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f869b600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694072040 con 0x7f86940ccec0 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f86940669a0 0x7f8694066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f868be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940ccec0 0x7f869415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f86940669a0 0x7f8694066d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38178/0 (socket says 172.21.15.45:38178) 2024-04-25T10:18:01.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 -- 172.21.15.45:0/289043662 learned_addr learned my addr 172.21.15.45:0/289043662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 -- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940ccec0 msgr2=0x7f869415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 --2- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940ccec0 0x7f869415c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 -- 172.21.15.45:0/289043662 >> v1:172.21.15.117:6789/0 conn(0x7f86940cd340 legacy=0x7f86940cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.861+0000 7f8698c00640 1 -- 172.21.15.45:0/289043662 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694072320 con 0x7f86940669a0 2024-04-25T10:18:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f8698c00640 1 --2- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940669a0 0x7f8694066d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8684009870 tx=0x7f868401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91dce11db10d8c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f8693e00640 1 -- 172.21.15.45:0/289043662 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840b2070 con 0x7f86940669a0 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f8693e00640 1 -- 172.21.15.45:0/289043662 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86840a9070 con 0x7f86940669a0 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f8693e00640 1 -- 172.21.15.45:0/289043662 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840ad880 con 0x7f86940669a0 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940669a0 msgr2=0x7f8694066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 --2- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940669a0 0x7f8694066d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8684009870 tx=0x7f868401c560 comp rx=0 tx=0).stop 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- 172.21.15.45:0/289043662 shutdown_connections 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 --2- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940ccec0 0x7f869415c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 --2- 172.21.15.45:0/289043662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940669a0 0x7f8694066d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- 172.21.15.45:0/289043662 >> 172.21.15.45:0/289043662 conn(0x7f86940b24f0 msgr2=0x7f86940b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- 172.21.15.45:0/289043662 shutdown_connections 2024-04-25T10:18:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- 172.21.15.45:0/289043662 wait complete. 2024-04-25T10:18:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 Processor -- start 2024-04-25T10:18:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- start start 2024-04-25T10:18:01.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 0x7f86940b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940ccec0 0x7f869414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940cd340 0x7f869414d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694168ec0 con 0x7f86940cd340 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694159af0 con 0x7f86940ccec0 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.865+0000 7f869b600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694169040 con 0x7f86940669a0 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 0x7f86940b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8699600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940cd340 0x7f869414d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 0x7f86940b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:58114/0 (socket says 172.21.15.45:58114) 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 -- 172.21.15.45:0/2077828781 learned_addr learned my addr 172.21.15.45:0/2077828781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f868be00640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940ccec0 0x7f869414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 -- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940ccec0 msgr2=0x7f869414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940ccec0 0x7f869414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 -- 172.21.15.45:0/2077828781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940cd340 msgr2=0x7f869414d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940cd340 0x7f869414d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 -- 172.21.15.45:0/2077828781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940b43a0 con 0x7f86940669a0 2024-04-25T10:18:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8698c00640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 0x7f86940b53d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f86840b0040 tx=0x7f868401ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840bc020 con 0x7f86940669a0 2024-04-25T10:18:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86840addc0 con 0x7f86940669a0 2024-04-25T10:18:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86840b2040 con 0x7f86940669a0 2024-04-25T10:18:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86940b4610 con 0x7f86940669a0 2024-04-25T10:18:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940b4bd0 con 0x7f86940669a0 2024-04-25T10:18:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869404ef30 con 0x7f86940669a0 2024-04-25T10:18:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.869+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86840bf020 con 0x7f86940669a0 2024-04-25T10:18:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.873+0000 7f8691600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8670066f60 0x7f8670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.873+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f868411eff0 con 0x7f86940669a0 2024-04-25T10:18:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.873+0000 7f868be00640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8670066f60 0x7f8670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.873+0000 7f868be00640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8670066f60 0x7f8670069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f869414eb50 tx=0x7f867c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:01.877+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86840c6030 con 0x7f86940669a0 2024-04-25T10:18:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.073+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86940b0140 con 0x7f86940669a0 2024-04-25T10:18:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.673+0000 7f8691600640 1 -- 172.21.15.45:0/2077828781 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8684016150 con 0x7f86940669a0 2024-04-25T10:18:02.676 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:18:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8670066f60 msgr2=0x7f8670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8670066f60 0x7f8670069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f869414eb50 tx=0x7f867c009190 comp rx=0 tx=0).stop 2024-04-25T10:18:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 msgr2=0x7f86940b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 0x7f86940b53d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f86840b0040 tx=0x7f868401ada0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 shutdown_connections 2024-04-25T10:18:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8670066f60 0x7f8670069420 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f869414eb50 tx=0x7f867c009190 comp rx=0 tx=0).stop 2024-04-25T10:18:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86940cd340 0x7f869414d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86940ccec0 0x7f869414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 --2- 172.21.15.45:0/2077828781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86940669a0 0x7f86940b53d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 >> 172.21.15.45:0/2077828781 conn(0x7f86940b24f0 msgr2=0x7f86940ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.681+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 shutdown_connections 2024-04-25T10:18:02.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.685+0000 7f869b600640 1 -- 172.21.15.45:0/2077828781 wait complete. 2024-04-25T10:18:02.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:02.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 Processor -- start 2024-04-25T10:18:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 -- start start 2024-04-25T10:18:02.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f832c0ccb20 0x7f832c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15bdc0 0x7f832c0d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0ccfb0 con 0x7f832c15b9e0 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0cd290 con 0x7f832c0ccb20 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8332e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0cd570 con 0x7f832c15bdc0 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8323400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f832c0ccb20 0x7f832c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f832be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f832c15b9e0 0x7f832c0cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43530/0 (socket says 172.21.15.45:43530) 2024-04-25T10:18:02.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f832be00640 1 -- 172.21.15.45:0/3770976341 learned_addr learned my addr 172.21.15.45:0/3770976341 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:02.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8323400640 1 -- 172.21.15.45:0/3770976341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15bdc0 msgr2=0x7f832c0d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8323400640 1 --2- 172.21.15.45:0/3770976341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15bdc0 0x7f832c0d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8323400640 1 -- 172.21.15.45:0/3770976341 >> v1:172.21.15.45:6789/0 conn(0x7f832c15b9e0 legacy=0x7f832c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f8323400640 1 -- 172.21.15.45:0/3770976341 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0cd850 con 0x7f832c0ccb20 2024-04-25T10:18:02.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.873+0000 7f832b400640 1 -- 172.21.15.45:0/3770976341 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075758498 0 0) 0x7f832c0ccfb0 con 0x7f832c15b9e0 2024-04-25T10:18:02.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8323400640 1 --2- 172.21.15.45:0/3770976341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c0ccb20 0x7f832c0d1520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83140097c0 tx=0x7f831401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28883321698c6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f832b400640 1 -- 172.21.15.45:0/3770976341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140b0070 con 0x7f832c0ccb20 2024-04-25T10:18:02.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f832b400640 1 -- 172.21.15.45:0/3770976341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83140ab600 con 0x7f832c0ccb20 2024-04-25T10:18:02.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f832b400640 1 -- 172.21.15.45:0/3770976341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140ab9d0 con 0x7f832c0ccb20 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 -- 172.21.15.45:0/3770976341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c0ccb20 msgr2=0x7f832c0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 --2- 172.21.15.45:0/3770976341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c0ccb20 0x7f832c0d1520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83140097c0 tx=0x7f831401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 -- 172.21.15.45:0/3770976341 shutdown_connections 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 --2- 172.21.15.45:0/3770976341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15bdc0 0x7f832c0d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 --2- 172.21.15.45:0/3770976341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c0ccb20 0x7f832c0d1520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 -- 172.21.15.45:0/3770976341 >> 172.21.15.45:0/3770976341 conn(0x7f832c09a9d0 msgr2=0x7f832c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 -- 172.21.15.45:0/3770976341 shutdown_connections 2024-04-25T10:18:02.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.877+0000 7f8332e00640 1 -- 172.21.15.45:0/3770976341 wait complete. 2024-04-25T10:18:02.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 Processor -- start 2024-04-25T10:18:02.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 -- start start 2024-04-25T10:18:02.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15b9e0 0x7f832c151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c15bdc0 0x7f832c15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 0x7f832c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c65f0 con 0x7f832c14ca10 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c6470 con 0x7f832c15bdc0 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c6770 con 0x7f832c15b9e0 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 0x7f832c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 0x7f832c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43138/0 (socket says 172.21.15.45:43138) 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 -- 172.21.15.45:0/1235016282 learned_addr learned my addr 172.21.15.45:0/1235016282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8323400640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c15bdc0 0x7f832c15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f832be00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15b9e0 0x7f832c151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 -- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15b9e0 msgr2=0x7f832c151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15b9e0 0x7f832c151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 -- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c15bdc0 msgr2=0x7f832c15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c15bdc0 0x7f832c15e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 -- 172.21.15.45:0/1235016282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c1523a0 con 0x7f832c14ca10 2024-04-25T10:18:02.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8330e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 0x7f832c06bce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f832400b970 tx=0x7f832400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324019070 con 0x7f832c14ca10 2024-04-25T10:18:02.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832400cb60 con 0x7f832c14ca10 2024-04-25T10:18:02.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c152680 con 0x7f832c14ca10 2024-04-25T10:18:02.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324014440 con 0x7f832c14ca10 2024-04-25T10:18:02.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.881+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c152c40 con 0x7f832c14ca10 2024-04-25T10:18:02.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.885+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c04ef30 con 0x7f832c14ca10 2024-04-25T10:18:02.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.885+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f832401c070 con 0x7f832c14ca10 2024-04-25T10:18:02.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.885+0000 7f8328c00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8308066f60 0x7f8308069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.885+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8324084cf0 con 0x7f832c14ca10 2024-04-25T10:18:02.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.885+0000 7f832be00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8308066f60 0x7f8308069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.885+0000 7f832be00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8308066f60 0x7f8308069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f831c007fd0 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:02.889+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83240556a0 con 0x7f832c14ca10 2024-04-25T10:18:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.053+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f832c0c6d80 con 0x7f832c14ca10 2024-04-25T10:18:03.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8328c00640 1 -- 172.21.15.45:0/1235016282 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f8324059740 con 0x7f832c14ca10 2024-04-25T10:18:03.058 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:03.059 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:02.076819+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8308066f60 msgr2=0x7f8308069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8308066f60 0x7f8308069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f831c007fd0 tx=0x7f831c009040 comp rx=0 tx=0).stop 2024-04-25T10:18:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 msgr2=0x7f832c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 0x7f832c06bce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f832400b970 tx=0x7f832400be30 comp rx=0 tx=0).stop 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 shutdown_connections 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8308066f60 0x7f8308069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f832c14ca10 0x7f832c06bce0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c15bdc0 0x7f832c15e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 --2- 172.21.15.45:0/1235016282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c15b9e0 0x7f832c151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 >> 172.21.15.45:0/1235016282 conn(0x7f832c09a9d0 msgr2=0x7f832c1565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 shutdown_connections 2024-04-25T10:18:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.057+0000 7f8332e00640 1 -- 172.21.15.45:0/1235016282 wait complete. 2024-04-25T10:18:03.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:03.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e92400640 1 Processor -- start 2024-04-25T10:18:03.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e92400640 1 -- start start 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e92400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e8c0b15b0 0x7f1e8c168990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e92400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c0b1af0 con 0x7f1e8c0d1820 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e92400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c0b1dd0 con 0x7f1e8c15e050 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e92400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8c0b20b0 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1e8c0d1820 0x7f1e8c0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43542/0 (socket says 172.21.15.45:43542) 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8b400640 1 -- 172.21.15.45:0/465339795 learned_addr learned my addr 172.21.15.45:0/465339795 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:03.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8be00640 1 --2- 172.21.15.45:0/465339795 >> v2:172.21.15.117:3300/0 conn(0x7f1e8c0b15b0 0x7f1e8c168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491322602 0 0) 0x7f1e8c0b1af0 con 0x7f1e8c0d1820 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e70003530 con 0x7f1e8c0d1820 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990702335 0 0) 0x7f1e8c0b1dd0 con 0x7f1e8c15e050 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e8c0b1af0 con 0x7f1e8c15e050 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8be00640 1 -- 172.21.15.45:0/465339795 >> v1:172.21.15.117:6789/0 conn(0x7f1e8c15e050 legacy=0x7f1e8c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8be00640 1 -- 172.21.15.45:0/465339795 >> v1:172.21.15.45:6789/0 conn(0x7f1e8c0d1820 legacy=0x7f1e8c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.221+0000 7f1e8be00640 1 -- 172.21.15.45:0/465339795 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0b2390 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e8be00640 1 --2- 172.21.15.45:0/465339795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c168990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009d90 tx=0x7f1e8401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117c3a031f98a174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840b2070 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e840a58b0 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e8aa00640 1 -- 172.21.15.45:0/465339795 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840a5c80 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 -- 172.21.15.45:0/465339795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0b15b0 msgr2=0x7f1e8c168990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 --2- 172.21.15.45:0/465339795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c168990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009d90 tx=0x7f1e8401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 -- 172.21.15.45:0/465339795 shutdown_connections 2024-04-25T10:18:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 --2- 172.21.15.45:0/465339795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c168990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 -- 172.21.15.45:0/465339795 >> 172.21.15.45:0/465339795 conn(0x7f1e8c09a9d0 msgr2=0x7f1e8c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 -- 172.21.15.45:0/465339795 shutdown_connections 2024-04-25T10:18:03.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.225+0000 7f1e92400640 1 -- 172.21.15.45:0/465339795 wait complete. 2024-04-25T10:18:03.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 Processor -- start 2024-04-25T10:18:03.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 -- start start 2024-04-25T10:18:03.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c1516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0d1820 0x7f1e8c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e8c15e050 0x7f1e8c0b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c16bda0 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c16bc20 con 0x7f1e8c0d1820 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c16bf20 con 0x7f1e8c15e050 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c1516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c1516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43142/0 (socket says 172.21.15.45:43142) 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 -- 172.21.15.45:0/376008273 learned_addr learned my addr 172.21.15.45:0/376008273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 -- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e8c15e050 msgr2=0x7f1e8c0b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e8c15e050 0x7f1e8c0b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 -- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0d1820 msgr2=0x7f1e8c153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0d1820 0x7f1e8c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 -- 172.21.15.45:0/376008273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c1558b0 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e8b400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c1516b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00c4f0 tx=0x7f1e7c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c017070 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7c005cf0 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c007270 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c155b90 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.229+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c156150 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.233+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c04ef30 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.233+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e7c01a070 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.233+0000 7f1e83e00640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e68066f60 0x7f1e68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.233+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e7c084380 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.233+0000 7f1e82a00640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e68066f60 0x7f1e68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.233+0000 7f1e82a00640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e68066f60 0x7f1e68069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e74007fd0 tx=0x7f1e74007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.237+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e7c054d30 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.401+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e8c0d1330 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.401+0000 7f1e83e00640 1 -- 172.21.15.45:0/376008273 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f1e7c058dd0 con 0x7f1e8c0b15b0 2024-04-25T10:18:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:03.405 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:02.076819+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e68066f60 msgr2=0x7f1e68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e68066f60 0x7f1e68069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e74007fd0 tx=0x7f1e74007900 comp rx=0 tx=0).stop 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 msgr2=0x7f1e8c1516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c1516b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00c4f0 tx=0x7f1e7c00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 shutdown_connections 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e68066f60 0x7f1e68069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e8c15e050 0x7f1e8c0b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e8c0d1820 0x7f1e8c153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 --2- 172.21.15.45:0/376008273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e8c0b15b0 0x7f1e8c1516b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 >> 172.21.15.45:0/376008273 conn(0x7f1e8c09a9d0 msgr2=0x7f1e8c06fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 shutdown_connections 2024-04-25T10:18:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.405+0000 7f1e92400640 1 -- 172.21.15.45:0/376008273 wait complete. 2024-04-25T10:18:03.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:18:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 Processor -- start 2024-04-25T10:18:03.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 -- start start 2024-04-25T10:18:03.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff690070350 0x7ff69015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff6900b31c0 0x7ff6900b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69015b080 con 0x7ff6900aef80 2024-04-25T10:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69015adb0 con 0x7ff6900b31c0 2024-04-25T10:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff696800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015eb60 con 0x7ff690070350 2024-04-25T10:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff68f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff6900aef80 0x7ff69015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43546/0 (socket says 172.21.15.45:43546) 2024-04-25T10:18:03.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.565+0000 7ff68f400640 1 -- 172.21.15.45:0/383251323 learned_addr learned my addr 172.21.15.45:0/383251323 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff686a00640 1 --2- 172.21.15.45:0/383251323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff690070350 0x7ff69015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff68fe00640 1 --2- 172.21.15.45:0/383251323 >> v2:172.21.15.117:3300/0 conn(0x7ff6900b31c0 0x7ff6900b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff68ea00640 1 -- 172.21.15.45:0/383251323 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253735652 0 0) 0x7ff69015b080 con 0x7ff6900aef80 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff68ea00640 1 -- 172.21.15.45:0/383251323 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff674005740 con 0x7ff6900aef80 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff686a00640 1 -- 172.21.15.45:0/383251323 >> v2:172.21.15.117:3300/0 conn(0x7ff6900b31c0 msgr2=0x7ff6900b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff686a00640 1 --2- 172.21.15.45:0/383251323 >> v2:172.21.15.117:3300/0 conn(0x7ff6900b31c0 0x7ff6900b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff686a00640 1 -- 172.21.15.45:0/383251323 >> v1:172.21.15.45:6789/0 conn(0x7ff6900aef80 legacy=0x7ff69015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff686a00640 1 -- 172.21.15.45:0/383251323 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69015ee40 con 0x7ff690070350 2024-04-25T10:18:03.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff686a00640 1 --2- 172.21.15.45:0/383251323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff690070350 0x7ff69015e480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff6780097c0 tx=0x7ff67801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b2a0fab606174a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff68ea00640 1 -- 172.21.15.45:0/383251323 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780b1070 con 0x7ff690070350 2024-04-25T10:18:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff68ea00640 1 -- 172.21.15.45:0/383251323 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6780ac410 con 0x7ff690070350 2024-04-25T10:18:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.569+0000 7ff68ea00640 1 -- 172.21.15.45:0/383251323 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780ac7e0 con 0x7ff690070350 2024-04-25T10:18:03.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- 172.21.15.45:0/383251323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff690070350 msgr2=0x7ff69015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 --2- 172.21.15.45:0/383251323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff690070350 0x7ff69015e480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff6780097c0 tx=0x7ff67801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- 172.21.15.45:0/383251323 shutdown_connections 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 --2- 172.21.15.45:0/383251323 >> v2:172.21.15.117:3300/0 conn(0x7ff6900b31c0 0x7ff6900b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 --2- 172.21.15.45:0/383251323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff690070350 0x7ff69015e480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- 172.21.15.45:0/383251323 >> 172.21.15.45:0/383251323 conn(0x7ff6900669a0 msgr2=0x7ff690066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- 172.21.15.45:0/383251323 shutdown_connections 2024-04-25T10:18:03.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- 172.21.15.45:0/383251323 wait complete. 2024-04-25T10:18:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 Processor -- start 2024-04-25T10:18:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- start start 2024-04-25T10:18:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff690070350 0x7ff69006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 0x7ff690144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6900b31c0 0x7ff690145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69006c250 con 0x7ff6900aef80 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69006c0d0 con 0x7ff690070350 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff696800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69006c3d0 con 0x7ff6900b31c0 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 0x7ff690144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 0x7ff690144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43150/0 (socket says 172.21.15.45:43150) 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 -- 172.21.15.45:0/3905442196 learned_addr learned my addr 172.21.15.45:0/3905442196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff68f400640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff690070350 0x7ff69006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 -- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6900b31c0 msgr2=0x7ff690145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6900b31c0 0x7ff690145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 -- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff690070350 msgr2=0x7ff69006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.573+0000 7ff686a00640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff690070350 0x7ff69006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff686a00640 1 -- 172.21.15.45:0/3905442196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69014fc60 con 0x7ff6900aef80 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff68f400640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff690070350 0x7ff69006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff686a00640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 0x7ff690144d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff67801a0d0 tx=0x7ff678019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780b1040 con 0x7ff6900aef80 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6780ba070 con 0x7ff6900aef80 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780b5650 con 0x7ff6900aef80 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69014ff40 con 0x7ff6900aef80 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff690150500 con 0x7ff6900aef80 2024-04-25T10:18:03.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69004ef30 con 0x7ff6900aef80 2024-04-25T10:18:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6780b57f0 con 0x7ff6900aef80 2024-04-25T10:18:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff687e00640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff66c066f60 0x7ff66c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff67811ec80 con 0x7ff6900aef80 2024-04-25T10:18:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.577+0000 7ff68f400640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff66c066f60 0x7ff66c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.581+0000 7ff68f400640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff66c066f60 0x7ff66c069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff680005890 tx=0x7ff680009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.581+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff678016090 con 0x7ff6900aef80 2024-04-25T10:18:03.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:03.781+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff690070f70 con 0x7ff6900aef80 2024-04-25T10:18:04.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff687e00640 1 -- 172.21.15.45:0/3905442196 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff6780efc90 con 0x7ff6900aef80 2024-04-25T10:18:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff66c066f60 msgr2=0x7ff66c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff66c066f60 0x7ff66c069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff680005890 tx=0x7ff680009040 comp rx=0 tx=0).stop 2024-04-25T10:18:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 msgr2=0x7ff690144d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 0x7ff690144d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff67801a0d0 tx=0x7ff678019240 comp rx=0 tx=0).stop 2024-04-25T10:18:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 shutdown_connections 2024-04-25T10:18:04.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff66c066f60 0x7ff66c069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6900b31c0 0x7ff690145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6900aef80 0x7ff690144d50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 --2- 172.21.15.45:0/3905442196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff690070350 0x7ff69006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 >> 172.21.15.45:0/3905442196 conn(0x7ff6900669a0 msgr2=0x7ff69015d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 shutdown_connections 2024-04-25T10:18:04.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.697+0000 7ff696800640 1 -- 172.21.15.45:0/3905442196 wait complete. 2024-04-25T10:18:04.724 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:18:04.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:18:04.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 Processor -- start 2024-04-25T10:18:04.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 -- start start 2024-04-25T10:18:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc06aea0 0x7f8fcc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8fcc0cd7f0 0x7f8fcc15d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc0cc080 con 0x7f8fcc0b2e70 2024-04-25T10:18:04.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cc320 con 0x7f8fcc0cd7f0 2024-04-25T10:18:04.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fd1a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cc600 con 0x7f8fcc06aea0 2024-04-25T10:18:04.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8fcc0cd7f0 0x7f8fcc15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fc3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc06aea0 0x7f8fcc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8fcc0cd7f0 0x7f8fcc15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38250/0 (socket says 172.21.15.45:38250) 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 -- 172.21.15.45:0/179541981 learned_addr learned my addr 172.21.15.45:0/179541981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 -- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc06aea0 msgr2=0x7f8fcc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 --2- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc06aea0 0x7f8fcc0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 -- 172.21.15.45:0/179541981 >> v1:172.21.15.117:6789/0 conn(0x7f8fcc0b2e70 legacy=0x7f8fcc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.873+0000 7f8fcb400640 1 -- 172.21.15.45:0/179541981 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0cc8e0 con 0x7f8fcc0cd7f0 2024-04-25T10:18:04.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fcb400640 1 --2- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0cd7f0 0x7f8fcc15d600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0cc250 tx=0x7f8fc401f200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aafd8b902bb7c435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:04.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fca000640 1 -- 172.21.15.45:0/179541981 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40b4070 con 0x7f8fcc0cd7f0 2024-04-25T10:18:04.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fca000640 1 -- 172.21.15.45:0/179541981 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc40af400 con 0x7f8fcc0cd7f0 2024-04-25T10:18:04.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fca000640 1 -- 172.21.15.45:0/179541981 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40af7d0 con 0x7f8fcc0cd7f0 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 -- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0cd7f0 msgr2=0x7f8fcc15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0cd7f0 0x7f8fcc15d600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0cc250 tx=0x7f8fc401f200 comp rx=0 tx=0).stop 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 -- 172.21.15.45:0/179541981 shutdown_connections 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0cd7f0 0x7f8fcc15d600 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/179541981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc06aea0 0x7f8fcc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 -- 172.21.15.45:0/179541981 >> 172.21.15.45:0/179541981 conn(0x7f8fcc0b34a0 msgr2=0x7f8fcc0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 -- 172.21.15.45:0/179541981 shutdown_connections 2024-04-25T10:18:04.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.877+0000 7f8fd1a00640 1 -- 172.21.15.45:0/179541981 wait complete. 2024-04-25T10:18:04.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 Processor -- start 2024-04-25T10:18:04.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- start start 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 0x7f8fcc155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0b2e70 0x7f8fcc15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc0cd7f0 0x7f8fcc0b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cdc80 con 0x7f8fcc06aea0 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0b3250 con 0x7f8fcc0b2e70 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cde00 con 0x7f8fcc0cd7f0 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 0x7f8fcc155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 0x7f8fcc155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57020/0 (socket says 172.21.15.45:57020) 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 -- 172.21.15.45:0/3533543072 learned_addr learned my addr 172.21.15.45:0/3533543072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcb400640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc0cd7f0 0x7f8fcc0b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fc3e00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0b2e70 0x7f8fcc15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 -- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc0cd7f0 msgr2=0x7f8fcc0b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc0cd7f0 0x7f8fcc0b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 -- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0b2e70 msgr2=0x7f8fcc15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0b2e70 0x7f8fcc15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 -- 172.21.15.45:0/3533543072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0c5530 con 0x7f8fcc06aea0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fcaa00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 0x7f8fcc155d30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c450 tx=0x7f8fbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc016020 con 0x7f8fcc06aea0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0052d0 con 0x7f8fcc06aea0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc0c57d0 con 0x7f8fcc06aea0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc007260 con 0x7f8fcc06aea0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0b67c0 con 0x7f8fcc06aea0 2024-04-25T10:18:04.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.881+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc04ef30 con 0x7f8fcc06aea0 2024-04-25T10:18:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.885+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fbc007400 con 0x7f8fcc06aea0 2024-04-25T10:18:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.885+0000 7f8fc2a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fa8066f90 0x7f8fa8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.885+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fbc084880 con 0x7f8fcc06aea0 2024-04-25T10:18:04.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.885+0000 7f8fc3e00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fa8066f90 0x7f8fa8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.885+0000 7f8fc3e00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fa8066f90 0x7f8fa8069450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc156a90 tx=0x7f8fb4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:04.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:04.889+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fbc020020 con 0x7f8fcc06aea0 2024-04-25T10:18:05.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.085+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8fcc0d1390 con 0x7f8fcc06aea0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.701+0000 7f8fc2a00640 1 -- 172.21.15.45:0/3533543072 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8fbc005480 con 0x7f8fcc06aea0 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fa8066f90 msgr2=0x7f8fa8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fa8066f90 0x7f8fa8069450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc156a90 tx=0x7f8fb4009190 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 msgr2=0x7f8fcc155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 0x7f8fcc155d30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c450 tx=0x7f8fbc00c910 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 shutdown_connections 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fa8066f90 0x7f8fa8069450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fcc0cd7f0 0x7f8fcc0b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fcc0b2e70 0x7f8fcc15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 --2- 172.21.15.45:0/3533543072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fcc06aea0 0x7f8fcc155d30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 >> 172.21.15.45:0/3533543072 conn(0x7f8fcc0b34a0 msgr2=0x7f8fcc15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 shutdown_connections 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.705+0000 7f8fd1a00640 1 -- 172.21.15.45:0/3533543072 wait complete. 2024-04-25T10:18:05.730 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:18:05.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 Processor -- start 2024-04-25T10:18:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 -- start start 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d8415f180 0x7f8d840ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d840d0630 0x7f8d840ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840ae610 con 0x7f8d840aedd0 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840d27b0 con 0x7f8d840d0630 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d89000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cf570 con 0x7f8d8415f180 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d840d0630 0x7f8d840ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d840d0630 0x7f8d840ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57024/0 (socket says 172.21.15.45:57024) 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 -- 172.21.15.45:0/3640225554 learned_addr learned my addr 172.21.15.45:0/3640225554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d82800640 1 --1- 172.21.15.45:0/3640225554 >> v1:172.21.15.45:6789/0 conn(0x7f8d840aedd0 0x7f8d840af260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59106/0 (socket says 172.21.15.45:59106) 2024-04-25T10:18:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d81e00640 1 --2- 172.21.15.45:0/3640225554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d8415f180 0x7f8d840ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 -- 172.21.15.45:0/3640225554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d8415f180 msgr2=0x7f8d840ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 --2- 172.21.15.45:0/3640225554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d8415f180 0x7f8d840ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 -- 172.21.15.45:0/3640225554 >> v1:172.21.15.45:6789/0 conn(0x7f8d840aedd0 legacy=0x7f8d840af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.873+0000 7f8d79400640 1 -- 172.21.15.45:0/3640225554 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840cf850 con 0x7f8d840d0630 2024-04-25T10:18:05.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d79400640 1 --2- 172.21.15.45:0/3640225554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d840d0630 0x7f8d840ce830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009870 tx=0x7f8d6c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e978763ea157eb2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d81400640 1 -- 172.21.15.45:0/3640225554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0a9610 con 0x7f8d840d0630 2024-04-25T10:18:05.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d81400640 1 -- 172.21.15.45:0/3640225554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0a97b0 con 0x7f8d840d0630 2024-04-25T10:18:05.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d81400640 1 -- 172.21.15.45:0/3640225554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0a9b80 con 0x7f8d840d0630 2024-04-25T10:18:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 -- 172.21.15.45:0/3640225554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d840d0630 msgr2=0x7f8d840ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 --2- 172.21.15.45:0/3640225554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d840d0630 0x7f8d840ce830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009870 tx=0x7f8d6c01c530 comp rx=0 tx=0).stop 2024-04-25T10:18:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 -- 172.21.15.45:0/3640225554 shutdown_connections 2024-04-25T10:18:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 --2- 172.21.15.45:0/3640225554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d840d0630 0x7f8d840ce830 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 --2- 172.21.15.45:0/3640225554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d8415f180 0x7f8d840ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 -- 172.21.15.45:0/3640225554 >> 172.21.15.45:0/3640225554 conn(0x7f8d840d1900 msgr2=0x7f8d840d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 -- 172.21.15.45:0/3640225554 shutdown_connections 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.877+0000 7f8d89000640 1 -- 172.21.15.45:0/3640225554 wait complete. 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 Processor -- start 2024-04-25T10:18:05.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 -- start start 2024-04-25T10:18:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d840aedd0 0x7f8d8414be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d840d0630 0x7f8d84164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 0x7f8d840c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840d2570 con 0x7f8d8415f180 2024-04-25T10:18:05.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840d23f0 con 0x7f8d840d0630 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841688a0 con 0x7f8d840aedd0 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 0x7f8d840c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 0x7f8d840c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57030/0 (socket says 172.21.15.45:57030) 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 -- 172.21.15.45:0/1176650857 learned_addr learned my addr 172.21.15.45:0/1176650857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d81e00640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d840aedd0 0x7f8d8414be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d79400640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d840d0630 0x7f8d84164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 -- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d840aedd0 msgr2=0x7f8d8414be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d840aedd0 0x7f8d8414be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 -- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d840d0630 msgr2=0x7f8d84164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d840d0630 0x7f8d84164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 -- 172.21.15.45:0/1176650857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d841655b0 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d82800640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 0x7f8d840c7840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c3c0 tx=0x7f8d7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c0074e0 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c007680 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004a10 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d84165890 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.881+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8406b150 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.885+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8406ac90 con 0x7f8d8415f180 2024-04-25T10:18:05.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.885+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d7c01a020 con 0x7f8d8415f180 2024-04-25T10:18:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.885+0000 7f8d7aa00640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d60066f90 0x7f8d60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.885+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d7c084680 con 0x7f8d8415f180 2024-04-25T10:18:05.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.885+0000 7f8d81e00640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d60066f90 0x7f8d60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.885+0000 7f8d81e00640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d60066f90 0x7f8d60069450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d740113e0 tx=0x7f8d7400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:05.889+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d8406ac90 con 0x7f8d8415f180 2024-04-25T10:18:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d8404ef30 con 0x7f8d8415f180 2024-04-25T10:18:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d7aa00640 1 -- 172.21.15.45:0/1176650857 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f8d7c054fa0 con 0x7f8d8415f180 2024-04-25T10:18:06.087 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:06.088 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:06.088 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d60066f90 msgr2=0x7f8d60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d60066f90 0x7f8d60069450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d740113e0 tx=0x7f8d7400b040 comp rx=0 tx=0).stop 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 msgr2=0x7f8d840c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 0x7f8d840c7840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c3c0 tx=0x7f8d7c00c880 comp rx=0 tx=0).stop 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 shutdown_connections 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d60066f90 0x7f8d60069450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d8415f180 0x7f8d840c7840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d840d0630 0x7f8d84164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 --2- 172.21.15.45:0/1176650857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d840aedd0 0x7f8d8414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 >> 172.21.15.45:0/1176650857 conn(0x7f8d840d1900 msgr2=0x7f8d8415df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 shutdown_connections 2024-04-25T10:18:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.085+0000 7f8d89000640 1 -- 172.21.15.45:0/1176650857 wait complete. 2024-04-25T10:18:06.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:06.128 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:18:06.126+0000 7fab2bc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:06.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932f200640 1 Processor -- start 2024-04-25T10:18:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932f200640 1 -- start start 2024-04-25T10:18:06.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f93281669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932f200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280b1040 con 0x7f932815bce0 2024-04-25T10:18:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280b1320 con 0x7f932815c0c0 2024-04-25T10:18:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b1600 con 0x7f93280d06d0 2024-04-25T10:18:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f932815bce0 0x7f93280cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59114/0 (socket says 172.21.15.45:59114) 2024-04-25T10:18:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327e00640 1 -- 172.21.15.45:0/2523936743 learned_addr learned my addr 172.21.15.45:0/2523936743 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657223852 0 0) 0x7f93280b1040 con 0x7f932815bce0 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f930c003530 con 0x7f932815bce0 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932d200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f93281669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128748616 0 0) 0x7f930c003530 con 0x7f932815bce0 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 msgr2=0x7f93281669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f93281669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 >> v1:172.21.15.117:6789/0 conn(0x7f932815c0c0 legacy=0x7f93280cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b18e0 con 0x7f932815bce0 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f9318001680 con 0x7f932815bce0 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f932d200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f93281669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 >> v1:172.21.15.45:6789/0 conn(0x7f932815bce0 legacy=0x7f93280cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f930c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f930c003fe0 0x7f930c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f930c004d50 0x7f930c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b1600 con 0x7f930c004d50 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c00bb30 con 0x7f930c003fe0 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.249+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930c00bd30 con 0x7f93280d06d0 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f930c004d50 0x7f930c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f9327e00640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f930c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f931fe00640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f930c003fe0 0x7f930c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 -- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 msgr2=0x7f930c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93280d06d0 0x7f930c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 -- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f930c003fe0 msgr2=0x7f930c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f930c003fe0 0x7f930c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 -- 172.21.15.45:0/2523936743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c003530 con 0x7f930c004d50 2024-04-25T10:18:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f932d200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f930c004d50 0x7f930c0051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93200073e0 tx=0x7f932001cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f933b1f37f6ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200b2070 con 0x7f930c004d50 2024-04-25T10:18:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93200a5dc0 con 0x7f930c004d50 2024-04-25T10:18:06.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.253+0000 7f9327400640 1 -- 172.21.15.45:0/2523936743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320009c80 con 0x7f930c004d50 2024-04-25T10:18:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- 172.21.15.45:0/2523936743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f930c004d50 msgr2=0x7f930c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 --2- 172.21.15.45:0/2523936743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f930c004d50 0x7f930c0051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93200073e0 tx=0x7f932001cec0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932d200640 1 -- 172.21.15.45:0/2523936743 reap_dead start 2024-04-25T10:18:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- 172.21.15.45:0/2523936743 shutdown_connections 2024-04-25T10:18:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- 172.21.15.45:0/2523936743 >> 172.21.15.45:0/2523936743 conn(0x7f93280d19e0 msgr2=0x7f93280d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- 172.21.15.45:0/2523936743 shutdown_connections 2024-04-25T10:18:06.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- 172.21.15.45:0/2523936743 wait complete. 2024-04-25T10:18:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 Processor -- start 2024-04-25T10:18:06.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- start start 2024-04-25T10:18:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f930c004d50 0x7f932814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 0x7f932815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f932814f190 0x7f9328151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cfb00 con 0x7f932814ed10 2024-04-25T10:18:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cf980 con 0x7f932814f190 2024-04-25T10:18:06.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b2e10 con 0x7f930c004d50 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 0x7f932815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 0x7f932815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57046/0 (socket says 172.21.15.45:57046) 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 -- 172.21.15.45:0/2825923701 learned_addr learned my addr 172.21.15.45:0/2825923701 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f9327e00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f930c004d50 0x7f932814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f932d200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f932814f190 0x7f9328151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 -- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f930c004d50 msgr2=0x7f932814f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f930c004d50 0x7f932814f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 -- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f932814f190 msgr2=0x7f9328151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f932814f190 0x7f9328151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 -- 172.21.15.45:0/2825923701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280c6a10 con 0x7f932814ed10 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.257+0000 7f931fe00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 0x7f932815d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f931000e550 tx=0x7f931000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100052e0 con 0x7f932814ed10 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310018070 con 0x7f932814ed10 2024-04-25T10:18:06.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310003820 con 0x7f932814ed10 2024-04-25T10:18:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280c6cb0 con 0x7f932814ed10 2024-04-25T10:18:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93280c7230 con 0x7f932814ed10 2024-04-25T10:18:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932804ef30 con 0x7f932814ed10 2024-04-25T10:18:06.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93100039c0 con 0x7f932814ed10 2024-04-25T10:18:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9324c00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9304066f60 0x7f9304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9310084ce0 con 0x7f932814ed10 2024-04-25T10:18:06.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.261+0000 7f9327e00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9304066f60 0x7f9304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.265+0000 7f9327e00640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9304066f60 0x7f9304069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9318005c00 tx=0x7f9318007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.265+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9310005480 con 0x7f932814ed10 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.429+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f932814fde0 con 0x7f932814ed10 2024-04-25T10:18:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f9324c00640 1 -- 172.21.15.45:0/2825923701 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f9310055600 con 0x7f932814ed10 2024-04-25T10:18:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:06.435 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9304066f60 msgr2=0x7f9304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9304066f60 0x7f9304069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9318005c00 tx=0x7f9318007040 comp rx=0 tx=0).stop 2024-04-25T10:18:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 msgr2=0x7f932815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 0x7f932815d3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f931000e550 tx=0x7f931000ea10 comp rx=0 tx=0).stop 2024-04-25T10:18:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 shutdown_connections 2024-04-25T10:18:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9304066f60 0x7f9304069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f932814f190 0x7f9328151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932814ed10 0x7f932815d3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 --2- 172.21.15.45:0/2825923701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f930c004d50 0x7f932814f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 >> 172.21.15.45:0/2825923701 conn(0x7f93280d19e0 msgr2=0x7f932815ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 shutdown_connections 2024-04-25T10:18:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.433+0000 7f932f200640 1 -- 172.21.15.45:0/2825923701 wait complete. 2024-04-25T10:18:06.456 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:18:06.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 Processor -- start 2024-04-25T10:18:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 -- start start 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28c80b37f0 0x7f28c806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28c806b1d0 0x7f28c815e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80aff80 con 0x7f28c815a9a0 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c806e230 con 0x7f28c806b1d0 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cf000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c806b5b0 con 0x7f28c80b37f0 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28c7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28c815a9a0 0x7f28c80af6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59128/0 (socket says 172.21.15.45:59128) 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28c7e00640 1 -- 172.21.15.45:0/3421147956 learned_addr learned my addr 172.21.15.45:0/3421147956 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:06.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cd000640 1 --2- 172.21.15.45:0/3421147956 >> v2:172.21.15.45:3300/0 conn(0x7f28c806b1d0 0x7f28c815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28bfe00640 1 --2- 172.21.15.45:0/3421147956 >> v2:172.21.15.117:3300/0 conn(0x7f28c80b37f0 0x7f28c806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cd000640 1 -- 172.21.15.45:0/3421147956 >> v2:172.21.15.117:3300/0 conn(0x7f28c80b37f0 msgr2=0x7f28c806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cd000640 1 --2- 172.21.15.45:0/3421147956 >> v2:172.21.15.117:3300/0 conn(0x7f28c80b37f0 0x7f28c806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cd000640 1 -- 172.21.15.45:0/3421147956 >> v1:172.21.15.45:6789/0 conn(0x7f28c815a9a0 legacy=0x7f28c80af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28c7400640 1 -- 172.21.15.45:0/3421147956 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016486053 0 0) 0x7f28c80aff80 con 0x7f28c815a9a0 2024-04-25T10:18:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.593+0000 7f28cd000640 1 -- 172.21.15.45:0/3421147956 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c806efb0 con 0x7f28c806b1d0 2024-04-25T10:18:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cd000640 1 --2- 172.21.15.45:0/3421147956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c806b1d0 0x7f28c815e630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28c806bda0 tx=0x7f28c001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a89a48b5afeb8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28c7400640 1 -- 172.21.15.45:0/3421147956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00b2070 con 0x7f28c806b1d0 2024-04-25T10:18:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28c7400640 1 -- 172.21.15.45:0/3421147956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00a5850 con 0x7f28c806b1d0 2024-04-25T10:18:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28c7400640 1 -- 172.21.15.45:0/3421147956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00a5c20 con 0x7f28c806b1d0 2024-04-25T10:18:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 -- 172.21.15.45:0/3421147956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c806b1d0 msgr2=0x7f28c815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 --2- 172.21.15.45:0/3421147956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c806b1d0 0x7f28c815e630 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28c806bda0 tx=0x7f28c001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 -- 172.21.15.45:0/3421147956 shutdown_connections 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 --2- 172.21.15.45:0/3421147956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c806b1d0 0x7f28c815e630 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 --2- 172.21.15.45:0/3421147956 >> v2:172.21.15.117:3300/0 conn(0x7f28c80b37f0 0x7f28c806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 -- 172.21.15.45:0/3421147956 >> 172.21.15.45:0/3421147956 conn(0x7f28c80b24f0 msgr2=0x7f28c80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 -- 172.21.15.45:0/3421147956 shutdown_connections 2024-04-25T10:18:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.597+0000 7f28cf000640 1 -- 172.21.15.45:0/3421147956 wait complete. 2024-04-25T10:18:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 Processor -- start 2024-04-25T10:18:06.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 -- start start 2024-04-25T10:18:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c806b1d0 0x7f28c813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 0x7f28c815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c815a9a0 0x7f28c815b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c815ad80 con 0x7f28c80b37f0 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80cdac0 con 0x7f28c815a9a0 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c815af00 con 0x7f28c806b1d0 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 0x7f28c815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 0x7f28c815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57064/0 (socket says 172.21.15.45:57064) 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 -- 172.21.15.45:0/1032025244 learned_addr learned my addr 172.21.15.45:0/1032025244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28c7e00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c806b1d0 0x7f28c813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cd000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c815a9a0 0x7f28c815b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 -- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c806b1d0 msgr2=0x7f28c813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c806b1d0 0x7f28c813d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 -- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c815a9a0 msgr2=0x7f28c815b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c815a9a0 0x7f28c815b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 -- 172.21.15.45:0/1032025244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c81675f0 con 0x7f28c80b37f0 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28bfe00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 0x7f28c815b0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28b000d800 tx=0x7f28b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b000b6c0 con 0x7f28c80b37f0 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0019070 con 0x7f28c80b37f0 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c81678d0 con 0x7f28c80b37f0 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b000ba60 con 0x7f28c80b37f0 2024-04-25T10:18:06.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.601+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c8167e90 con 0x7f28c80b37f0 2024-04-25T10:18:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.605+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c804ef30 con 0x7f28c80b37f0 2024-04-25T10:18:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.605+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28b000bc00 con 0x7f28c80b37f0 2024-04-25T10:18:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.605+0000 7f28c4c00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28a4066f90 0x7f28a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.605+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28b00854a0 con 0x7f28c80b37f0 2024-04-25T10:18:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.605+0000 7f28c7e00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28a4066f90 0x7f28a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.605+0000 7f28c7e00640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28a4066f90 0x7f28a4069450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000f70 tx=0x7f28b8007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.609+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28b00564a0 con 0x7f28c80b37f0 2024-04-25T10:18:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.769+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28c81673a0 con 0x7f28a4066f90 2024-04-25T10:18:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.777+0000 7f28c4c00640 1 -- 172.21.15.45:0/1032025244 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251934 (secure 0 0 0) 0x7f28c81673a0 con 0x7f28a4066f90 2024-04-25T10:18:06.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:18:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28a4066f90 msgr2=0x7f28a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28a4066f90 0x7f28a4069450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000f70 tx=0x7f28b8007b60 comp rx=0 tx=0).stop 2024-04-25T10:18:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 msgr2=0x7f28c815b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 0x7f28c815b0e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f28b000d800 tx=0x7f28b000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 shutdown_connections 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28a4066f90 0x7f28a4069450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28c815a9a0 0x7f28c815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28c80b37f0 0x7f28c815b0e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 --2- 172.21.15.45:0/1032025244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28c806b1d0 0x7f28c813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 >> 172.21.15.45:0/1032025244 conn(0x7f28c80b24f0 msgr2=0x7f28c8067230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 shutdown_connections 2024-04-25T10:18:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.781+0000 7f28cf000640 1 -- 172.21.15.45:0/1032025244 wait complete. 2024-04-25T10:18:06.803 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T10:18:04.873027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235680,"kb_used_data":2868,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739040,"statfs":{"total":773094113280,"available":772852776960,"internally_reserved":0,"allocated":2936832,"data_stored":1751728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.205525"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704515+0000","last_change":"2024-04-25T10:18:02.719059+0000","last_active":"2024-04-25T10:18:03.704515+0000","last_peered":"2024-04-25T10:18:03.704515+0000","last_clean":"2024-04-25T10:18:03.704515+0000","last_became_active":"2024-04-25T10:18:02.718765+0000","last_became_peered":"2024-04-25T10:18:02.718765+0000","last_unstale":"2024-04-25T10:18:03.704515+0000","last_undegraded":"2024-04-25T10:18:03.704515+0000","last_fullsized":"2024-04-25T10:18:03.704515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694866+0000","last_change":"2024-04-25T10:18:02.693052+0000","last_active":"2024-04-25T10:18:03.694866+0000","last_peered":"2024-04-25T10:18:03.694866+0000","last_clean":"2024-04-25T10:18:03.694866+0000","last_became_active":"2024-04-25T10:18:02.692751+0000","last_became_peered":"2024-04-25T10:18:02.692751+0000","last_unstale":"2024-04-25T10:18:03.694866+0000","last_undegraded":"2024-04-25T10:18:03.694866+0000","last_fullsized":"2024-04-25T10:18:03.694866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694673+0000","last_change":"2024-04-25T10:18:02.694847+0000","last_active":"2024-04-25T10:18:03.694673+0000","last_peered":"2024-04-25T10:18:03.694673+0000","last_clean":"2024-04-25T10:18:03.694673+0000","last_became_active":"2024-04-25T10:18:02.694498+0000","last_became_peered":"2024-04-25T10:18:02.694498+0000","last_unstale":"2024-04-25T10:18:03.694673+0000","last_undegraded":"2024-04-25T10:18:03.694673+0000","last_fullsized":"2024-04-25T10:18:03.694673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703302+0000","last_change":"2024-04-25T10:18:02.693910+0000","last_active":"2024-04-25T10:18:03.703302+0000","last_peered":"2024-04-25T10:18:03.703302+0000","last_clean":"2024-04-25T10:18:03.703302+0000","last_became_active":"2024-04-25T10:18:02.692960+0000","last_became_peered":"2024-04-25T10:18:02.692960+0000","last_unstale":"2024-04-25T10:18:03.703302+0000","last_undegraded":"2024-04-25T10:18:03.703302+0000","last_fullsized":"2024-04-25T10:18:03.703302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703719+0000","last_change":"2024-04-25T10:18:02.709103+0000","last_active":"2024-04-25T10:18:03.703719+0000","last_peered":"2024-04-25T10:18:03.703719+0000","last_clean":"2024-04-25T10:18:03.703719+0000","last_became_active":"2024-04-25T10:18:02.708360+0000","last_became_peered":"2024-04-25T10:18:02.708360+0000","last_unstale":"2024-04-25T10:18:03.703719+0000","last_undegraded":"2024-04-25T10:18:03.703719+0000","last_fullsized":"2024-04-25T10:18:03.703719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702789+0000","last_change":"2024-04-25T10:18:02.686452+0000","last_active":"2024-04-25T10:18:03.702789+0000","last_peered":"2024-04-25T10:18:03.702789+0000","last_clean":"2024-04-25T10:18:03.702789+0000","last_became_active":"2024-04-25T10:18:02.685301+0000","last_became_peered":"2024-04-25T10:18:02.685301+0000","last_unstale":"2024-04-25T10:18:03.702789+0000","last_undegraded":"2024-04-25T10:18:03.702789+0000","last_fullsized":"2024-04-25T10:18:03.702789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.708545+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.707896+0000","last_became_peered":"2024-04-25T10:18:02.707896+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693563+0000","last_change":"2024-04-25T10:18:02.696678+0000","last_active":"2024-04-25T10:18:03.693563+0000","last_peered":"2024-04-25T10:18:03.693563+0000","last_clean":"2024-04-25T10:18:03.693563+0000","last_became_active":"2024-04-25T10:18:02.696428+0000","last_became_peered":"2024-04-25T10:18:02.696428+0000","last_unstale":"2024-04-25T10:18:03.693563+0000","last_undegraded":"2024-04-25T10:18:03.693563+0000","last_fullsized":"2024-04-25T10:18:03.693563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:19.470065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691719+0000","last_change":"2024-04-25T10:18:02.702250+0000","last_active":"2024-04-25T10:18:03.691719+0000","last_peered":"2024-04-25T10:18:03.691719+0000","last_clean":"2024-04-25T10:18:03.691719+0000","last_became_active":"2024-04-25T10:18:02.701992+0000","last_became_peered":"2024-04-25T10:18:02.701992+0000","last_unstale":"2024-04-25T10:18:03.691719+0000","last_undegraded":"2024-04-25T10:18:03.691719+0000","last_fullsized":"2024-04-25T10:18:03.691719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705200+0000","last_change":"2024-04-25T10:18:02.715021+0000","last_active":"2024-04-25T10:18:03.705200+0000","last_peered":"2024-04-25T10:18:03.705200+0000","last_clean":"2024-04-25T10:18:03.705200+0000","last_became_active":"2024-04-25T10:18:02.714746+0000","last_became_peered":"2024-04-25T10:18:02.714746+0000","last_unstale":"2024-04-25T10:18:03.705200+0000","last_undegraded":"2024-04-25T10:18:03.705200+0000","last_fullsized":"2024-04-25T10:18:03.705200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704831+0000","last_change":"2024-04-25T10:18:02.718912+0000","last_active":"2024-04-25T10:18:03.704831+0000","last_peered":"2024-04-25T10:18:03.704831+0000","last_clean":"2024-04-25T10:18:03.704831+0000","last_became_active":"2024-04-25T10:18:02.718611+0000","last_became_peered":"2024-04-25T10:18:02.718611+0000","last_unstale":"2024-04-25T10:18:03.704831+0000","last_undegraded":"2024-04-25T10:18:03.704831+0000","last_fullsized":"2024-04-25T10:18:03.704831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704014+0000","last_change":"2024-04-25T10:18:02.706863+0000","last_active":"2024-04-25T10:18:03.704014+0000","last_peered":"2024-04-25T10:18:03.704014+0000","last_clean":"2024-04-25T10:18:03.704014+0000","last_became_active":"2024-04-25T10:18:02.706447+0000","last_became_peered":"2024-04-25T10:18:02.706447+0000","last_unstale":"2024-04-25T10:18:03.704014+0000","last_undegraded":"2024-04-25T10:18:03.704014+0000","last_fullsized":"2024-04-25T10:18:03.704014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693746+0000","last_change":"2024-04-25T10:18:02.714632+0000","last_active":"2024-04-25T10:18:03.693746+0000","last_peered":"2024-04-25T10:18:03.693746+0000","last_clean":"2024-04-25T10:18:03.693746+0000","last_became_active":"2024-04-25T10:18:02.714302+0000","last_became_peered":"2024-04-25T10:18:02.714302+0000","last_unstale":"2024-04-25T10:18:03.693746+0000","last_undegraded":"2024-04-25T10:18:03.693746+0000","last_fullsized":"2024-04-25T10:18:03.693746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694231+0000","last_change":"2024-04-25T10:18:02.707614+0000","last_active":"2024-04-25T10:18:03.694231+0000","last_peered":"2024-04-25T10:18:03.694231+0000","last_clean":"2024-04-25T10:18:03.694231+0000","last_became_active":"2024-04-25T10:18:02.703697+0000","last_became_peered":"2024-04-25T10:18:02.703697+0000","last_unstale":"2024-04-25T10:18:03.694231+0000","last_undegraded":"2024-04-25T10:18:03.694231+0000","last_fullsized":"2024-04-25T10:18:03.694231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:45.419224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693574+0000","last_change":"2024-04-25T10:18:02.711384+0000","last_active":"2024-04-25T10:18:03.693574+0000","last_peered":"2024-04-25T10:18:03.693574+0000","last_clean":"2024-04-25T10:18:03.693574+0000","last_became_active":"2024-04-25T10:18:02.711036+0000","last_became_peered":"2024-04-25T10:18:02.711036+0000","last_unstale":"2024-04-25T10:18:03.693574+0000","last_undegraded":"2024-04-25T10:18:03.693574+0000","last_fullsized":"2024-04-25T10:18:03.693574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.002103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702415+0000","last_change":"2024-04-25T10:18:02.697670+0000","last_active":"2024-04-25T10:18:03.702415+0000","last_peered":"2024-04-25T10:18:03.702415+0000","last_clean":"2024-04-25T10:18:03.702415+0000","last_became_active":"2024-04-25T10:18:02.695341+0000","last_became_peered":"2024-04-25T10:18:02.695341+0000","last_unstale":"2024-04-25T10:18:03.702415+0000","last_undegraded":"2024-04-25T10:18:03.702415+0000","last_fullsized":"2024-04-25T10:18:03.702415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705262+0000","last_change":"2024-04-25T10:18:02.709655+0000","last_active":"2024-04-25T10:18:03.705262+0000","last_peered":"2024-04-25T10:18:03.705262+0000","last_clean":"2024-04-25T10:18:03.705262+0000","last_became_active":"2024-04-25T10:18:02.709349+0000","last_became_peered":"2024-04-25T10:18:02.709349+0000","last_unstale":"2024-04-25T10:18:03.705262+0000","last_undegraded":"2024-04-25T10:18:03.705262+0000","last_fullsized":"2024-04-25T10:18:03.705262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:57.065003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703843+0000","last_change":"2024-04-25T10:18:02.712551+0000","last_active":"2024-04-25T10:18:03.703843+0000","last_peered":"2024-04-25T10:18:03.703843+0000","last_clean":"2024-04-25T10:18:03.703843+0000","last_became_active":"2024-04-25T10:18:02.712257+0000","last_became_peered":"2024-04-25T10:18:02.712257+0000","last_unstale":"2024-04-25T10:18:03.703843+0000","last_undegraded":"2024-04-25T10:18:03.703843+0000","last_fullsized":"2024-04-25T10:18:03.703843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703878+0000","last_change":"2024-04-25T10:18:02.703048+0000","last_active":"2024-04-25T10:18:03.703878+0000","last_peered":"2024-04-25T10:18:03.703878+0000","last_clean":"2024-04-25T10:18:03.703878+0000","last_became_active":"2024-04-25T10:18:02.702433+0000","last_became_peered":"2024-04-25T10:18:02.702433+0000","last_unstale":"2024-04-25T10:18:03.703878+0000","last_undegraded":"2024-04-25T10:18:03.703878+0000","last_fullsized":"2024-04-25T10:18:03.703878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:25.741442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704147+0000","last_change":"2024-04-25T10:18:02.706776+0000","last_active":"2024-04-25T10:18:03.704147+0000","last_peered":"2024-04-25T10:18:03.704147+0000","last_clean":"2024-04-25T10:18:03.704147+0000","last_became_active":"2024-04-25T10:18:02.706327+0000","last_became_peered":"2024-04-25T10:18:02.706327+0000","last_unstale":"2024-04-25T10:18:03.704147+0000","last_undegraded":"2024-04-25T10:18:03.704147+0000","last_fullsized":"2024-04-25T10:18:03.704147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.700841+0000","last_change":"2024-04-25T10:18:02.685113+0000","last_active":"2024-04-25T10:18:02.700841+0000","last_peered":"2024-04-25T10:18:02.700841+0000","last_clean":"2024-04-25T10:18:02.700841+0000","last_became_active":"2024-04-25T10:18:02.684839+0000","last_became_peered":"2024-04-25T10:18:02.684839+0000","last_unstale":"2024-04-25T10:18:02.700841+0000","last_undegraded":"2024-04-25T10:18:02.700841+0000","last_fullsized":"2024-04-25T10:18:02.700841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:44.467176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691179+0000","last_change":"2024-04-25T10:18:02.713609+0000","last_active":"2024-04-25T10:18:03.691179+0000","last_peered":"2024-04-25T10:18:03.691179+0000","last_clean":"2024-04-25T10:18:03.691179+0000","last_became_active":"2024-04-25T10:18:02.713344+0000","last_became_peered":"2024-04-25T10:18:02.713344+0000","last_unstale":"2024-04-25T10:18:03.691179+0000","last_undegraded":"2024-04-25T10:18:03.691179+0000","last_fullsized":"2024-04-25T10:18:03.691179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:08.145654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693768+0000","last_change":"2024-04-25T10:18:02.688007+0000","last_active":"2024-04-25T10:18:03.693768+0000","last_peered":"2024-04-25T10:18:03.693768+0000","last_clean":"2024-04-25T10:18:03.693768+0000","last_became_active":"2024-04-25T10:18:02.687708+0000","last_became_peered":"2024-04-25T10:18:02.687708+0000","last_unstale":"2024-04-25T10:18:03.693768+0000","last_undegraded":"2024-04-25T10:18:03.693768+0000","last_fullsized":"2024-04-25T10:18:03.693768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.697693+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.696374+0000","last_became_peered":"2024-04-25T10:18:02.696374+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695340+0000","last_change":"2024-04-25T10:18:02.690318+0000","last_active":"2024-04-25T10:18:03.695340+0000","last_peered":"2024-04-25T10:18:03.695340+0000","last_clean":"2024-04-25T10:18:03.695340+0000","last_became_active":"2024-04-25T10:18:02.688844+0000","last_became_peered":"2024-04-25T10:18:02.688844+0000","last_unstale":"2024-04-25T10:18:03.695340+0000","last_undegraded":"2024-04-25T10:18:03.695340+0000","last_fullsized":"2024-04-25T10:18:03.695340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703210+0000","last_change":"2024-04-25T10:18:02.699592+0000","last_active":"2024-04-25T10:18:03.703210+0000","last_peered":"2024-04-25T10:18:03.703210+0000","last_clean":"2024-04-25T10:18:03.703210+0000","last_became_active":"2024-04-25T10:18:02.685005+0000","last_became_peered":"2024-04-25T10:18:02.685005+0000","last_unstale":"2024-04-25T10:18:03.703210+0000","last_undegraded":"2024-04-25T10:18:03.703210+0000","last_fullsized":"2024-04-25T10:18:03.703210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694078+0000","last_change":"2024-04-25T10:18:02.707729+0000","last_active":"2024-04-25T10:18:03.694078+0000","last_peered":"2024-04-25T10:18:03.694078+0000","last_clean":"2024-04-25T10:18:03.694078+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.694078+0000","last_undegraded":"2024-04-25T10:18:03.694078+0000","last_fullsized":"2024-04-25T10:18:03.694078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693598+0000","last_change":"2024-04-25T10:18:02.708638+0000","last_active":"2024-04-25T10:18:03.693598+0000","last_peered":"2024-04-25T10:18:03.693598+0000","last_clean":"2024-04-25T10:18:03.693598+0000","last_became_active":"2024-04-25T10:18:02.708316+0000","last_became_peered":"2024-04-25T10:18:02.708316+0000","last_unstale":"2024-04-25T10:18:03.693598+0000","last_undegraded":"2024-04-25T10:18:03.693598+0000","last_fullsized":"2024-04-25T10:18:03.693598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694317+0000","last_change":"2024-04-25T10:18:02.703831+0000","last_active":"2024-04-25T10:18:03.694317+0000","last_peered":"2024-04-25T10:18:03.694317+0000","last_clean":"2024-04-25T10:18:03.694317+0000","last_became_active":"2024-04-25T10:18:02.703544+0000","last_became_peered":"2024-04-25T10:18:02.703544+0000","last_unstale":"2024-04-25T10:18:03.694317+0000","last_undegraded":"2024-04-25T10:18:03.694317+0000","last_fullsized":"2024-04-25T10:18:03.694317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703007+0000","last_change":"2024-04-25T10:18:02.704805+0000","last_active":"2024-04-25T10:18:03.703007+0000","last_peered":"2024-04-25T10:18:03.703007+0000","last_clean":"2024-04-25T10:18:03.703007+0000","last_became_active":"2024-04-25T10:18:02.704430+0000","last_became_peered":"2024-04-25T10:18:02.704430+0000","last_unstale":"2024-04-25T10:18:03.703007+0000","last_undegraded":"2024-04-25T10:18:03.703007+0000","last_fullsized":"2024-04-25T10:18:03.703007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703417+0000","last_change":"2024-04-25T10:18:02.700147+0000","last_active":"2024-04-25T10:18:03.703417+0000","last_peered":"2024-04-25T10:18:03.703417+0000","last_clean":"2024-04-25T10:18:03.703417+0000","last_became_active":"2024-04-25T10:18:02.699857+0000","last_became_peered":"2024-04-25T10:18:02.699857+0000","last_unstale":"2024-04-25T10:18:03.703417+0000","last_undegraded":"2024-04-25T10:18:03.703417+0000","last_fullsized":"2024-04-25T10:18:03.703417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702527+0000","last_change":"2024-04-25T10:18:02.692533+0000","last_active":"2024-04-25T10:18:03.702527+0000","last_peered":"2024-04-25T10:18:03.702527+0000","last_clean":"2024-04-25T10:18:03.702527+0000","last_became_active":"2024-04-25T10:18:02.692239+0000","last_became_peered":"2024-04-25T10:18:02.692239+0000","last_unstale":"2024-04-25T10:18:03.702527+0000","last_undegraded":"2024-04-25T10:18:03.702527+0000","last_fullsized":"2024-04-25T10:18:03.702527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691316+0000","last_change":"2024-04-25T10:18:02.691725+0000","last_active":"2024-04-25T10:18:03.691316+0000","last_peered":"2024-04-25T10:18:03.691316+0000","last_clean":"2024-04-25T10:18:03.691316+0000","last_became_active":"2024-04-25T10:18:02.691415+0000","last_became_peered":"2024-04-25T10:18:02.691415+0000","last_unstale":"2024-04-25T10:18:03.691316+0000","last_undegraded":"2024-04-25T10:18:03.691316+0000","last_fullsized":"2024-04-25T10:18:03.691316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694378+0000","last_change":"2024-04-25T10:18:01.697822+0000","last_active":"2024-04-25T10:18:03.694378+0000","last_peered":"2024-04-25T10:18:03.694378+0000","last_clean":"2024-04-25T10:18:03.694378+0000","last_became_active":"2024-04-25T10:18:01.696666+0000","last_became_peered":"2024-04-25T10:18:01.696666+0000","last_unstale":"2024-04-25T10:18:03.694378+0000","last_undegraded":"2024-04-25T10:18:03.694378+0000","last_fullsized":"2024-04-25T10:18:03.694378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.711203+0000","last_change":"2024-04-25T10:18:02.709407+0000","last_active":"2024-04-25T10:18:02.711203+0000","last_peered":"2024-04-25T10:18:02.711203+0000","last_clean":"2024-04-25T10:18:02.711203+0000","last_became_active":"2024-04-25T10:18:02.708477+0000","last_became_peered":"2024-04-25T10:18:02.708477+0000","last_unstale":"2024-04-25T10:18:02.711203+0000","last_undegraded":"2024-04-25T10:18:02.711203+0000","last_fullsized":"2024-04-25T10:18:02.711203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:10.536062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703290+0000","last_change":"2024-04-25T10:18:01.706968+0000","last_active":"2024-04-25T10:18:03.703290+0000","last_peered":"2024-04-25T10:18:03.703290+0000","last_clean":"2024-04-25T10:18:03.703290+0000","last_became_active":"2024-04-25T10:18:01.706660+0000","last_became_peered":"2024-04-25T10:18:01.706660+0000","last_unstale":"2024-04-25T10:18:03.703290+0000","last_undegraded":"2024-04-25T10:18:03.703290+0000","last_fullsized":"2024-04-25T10:18:03.703290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694922+0000","last_change":"2024-04-25T10:18:01.705160+0000","last_active":"2024-04-25T10:18:03.694922+0000","last_peered":"2024-04-25T10:18:03.694922+0000","last_clean":"2024-04-25T10:18:03.694922+0000","last_became_active":"2024-04-25T10:18:01.704852+0000","last_became_peered":"2024-04-25T10:18:01.704852+0000","last_unstale":"2024-04-25T10:18:03.694922+0000","last_undegraded":"2024-04-25T10:18:03.694922+0000","last_fullsized":"2024-04-25T10:18:03.694922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:33.140157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693846+0000","last_change":"2024-04-25T10:18:02.691734+0000","last_active":"2024-04-25T10:18:03.693846+0000","last_peered":"2024-04-25T10:18:03.693846+0000","last_clean":"2024-04-25T10:18:03.693846+0000","last_became_active":"2024-04-25T10:18:02.691454+0000","last_became_peered":"2024-04-25T10:18:02.691454+0000","last_unstale":"2024-04-25T10:18:03.693846+0000","last_undegraded":"2024-04-25T10:18:03.693846+0000","last_fullsized":"2024-04-25T10:18:03.693846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702652+0000","last_change":"2024-04-25T10:18:01.684950+0000","last_active":"2024-04-25T10:18:03.702652+0000","last_peered":"2024-04-25T10:18:03.702652+0000","last_clean":"2024-04-25T10:18:03.702652+0000","last_became_active":"2024-04-25T10:18:01.682962+0000","last_became_peered":"2024-04-25T10:18:01.682962+0000","last_unstale":"2024-04-25T10:18:03.702652+0000","last_undegraded":"2024-04-25T10:18:03.702652+0000","last_fullsized":"2024-04-25T10:18:03.702652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:45.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.704948+0000","last_change":"2024-04-25T10:18:02.695250+0000","last_active":"2024-04-25T10:18:02.704948+0000","last_peered":"2024-04-25T10:18:02.704948+0000","last_clean":"2024-04-25T10:18:02.704948+0000","last_became_active":"2024-04-25T10:18:02.694926+0000","last_became_peered":"2024-04-25T10:18:02.694926+0000","last_unstale":"2024-04-25T10:18:02.704948+0000","last_undegraded":"2024-04-25T10:18:02.704948+0000","last_fullsized":"2024-04-25T10:18:02.704948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:47.209619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691706+0000","last_change":"2024-04-25T10:18:01.691624+0000","last_active":"2024-04-25T10:18:03.691706+0000","last_peered":"2024-04-25T10:18:03.691706+0000","last_clean":"2024-04-25T10:18:03.691706+0000","last_became_active":"2024-04-25T10:18:01.687161+0000","last_became_peered":"2024-04-25T10:18:01.687161+0000","last_unstale":"2024-04-25T10:18:03.691706+0000","last_undegraded":"2024-04-25T10:18:03.691706+0000","last_fullsized":"2024-04-25T10:18:03.691706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703606+0000","last_change":"2024-04-25T10:18:02.685373+0000","last_active":"2024-04-25T10:18:03.703606+0000","last_peered":"2024-04-25T10:18:03.703606+0000","last_clean":"2024-04-25T10:18:03.703606+0000","last_became_active":"2024-04-25T10:18:02.684600+0000","last_became_peered":"2024-04-25T10:18:02.684600+0000","last_unstale":"2024-04-25T10:18:03.703606+0000","last_undegraded":"2024-04-25T10:18:03.703606+0000","last_fullsized":"2024-04-25T10:18:03.703606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705362+0000","last_change":"2024-04-25T10:18:01.703304+0000","last_active":"2024-04-25T10:18:03.705362+0000","last_peered":"2024-04-25T10:18:03.705362+0000","last_clean":"2024-04-25T10:18:03.705362+0000","last_became_active":"2024-04-25T10:18:01.701629+0000","last_became_peered":"2024-04-25T10:18:01.701629+0000","last_unstale":"2024-04-25T10:18:03.705362+0000","last_undegraded":"2024-04-25T10:18:03.705362+0000","last_fullsized":"2024-04-25T10:18:03.705362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695152+0000","last_change":"2024-04-25T10:18:02.696745+0000","last_active":"2024-04-25T10:18:03.695152+0000","last_peered":"2024-04-25T10:18:03.695152+0000","last_clean":"2024-04-25T10:18:03.695152+0000","last_became_active":"2024-04-25T10:18:02.696514+0000","last_became_peered":"2024-04-25T10:18:02.696514+0000","last_unstale":"2024-04-25T10:18:03.695152+0000","last_undegraded":"2024-04-25T10:18:03.695152+0000","last_fullsized":"2024-04-25T10:18:03.695152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703782+0000","last_change":"2024-04-25T10:18:01.692901+0000","last_active":"2024-04-25T10:18:03.703782+0000","last_peered":"2024-04-25T10:18:03.703782+0000","last_clean":"2024-04-25T10:18:03.703782+0000","last_became_active":"2024-04-25T10:18:01.692604+0000","last_became_peered":"2024-04-25T10:18:01.692604+0000","last_unstale":"2024-04-25T10:18:03.703782+0000","last_undegraded":"2024-04-25T10:18:03.703782+0000","last_fullsized":"2024-04-25T10:18:03.703782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703310+0000","last_change":"2024-04-25T10:18:02.696653+0000","last_active":"2024-04-25T10:18:03.703310+0000","last_peered":"2024-04-25T10:18:03.703310+0000","last_clean":"2024-04-25T10:18:03.703310+0000","last_became_active":"2024-04-25T10:18:02.696271+0000","last_became_peered":"2024-04-25T10:18:02.696271+0000","last_unstale":"2024-04-25T10:18:03.703310+0000","last_undegraded":"2024-04-25T10:18:03.703310+0000","last_fullsized":"2024-04-25T10:18:03.703310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:35.561629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703399+0000","last_change":"2024-04-25T10:18:01.696254+0000","last_active":"2024-04-25T10:18:03.703399+0000","last_peered":"2024-04-25T10:18:03.703399+0000","last_clean":"2024-04-25T10:18:03.703399+0000","last_became_active":"2024-04-25T10:18:01.695894+0000","last_became_peered":"2024-04-25T10:18:01.695894+0000","last_unstale":"2024-04-25T10:18:03.703399+0000","last_undegraded":"2024-04-25T10:18:03.703399+0000","last_fullsized":"2024-04-25T10:18:03.703399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704898+0000","last_change":"2024-04-25T10:18:02.707539+0000","last_active":"2024-04-25T10:18:03.704898+0000","last_peered":"2024-04-25T10:18:03.704898+0000","last_clean":"2024-04-25T10:18:03.704898+0000","last_became_active":"2024-04-25T10:18:02.707294+0000","last_became_peered":"2024-04-25T10:18:02.707294+0000","last_unstale":"2024-04-25T10:18:03.704898+0000","last_undegraded":"2024-04-25T10:18:03.704898+0000","last_fullsized":"2024-04-25T10:18:03.704898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703612+0000","last_change":"2024-04-25T10:18:01.703958+0000","last_active":"2024-04-25T10:18:03.703612+0000","last_peered":"2024-04-25T10:18:03.703612+0000","last_clean":"2024-04-25T10:18:03.703612+0000","last_became_active":"2024-04-25T10:18:01.703524+0000","last_became_peered":"2024-04-25T10:18:01.703524+0000","last_unstale":"2024-04-25T10:18:03.703612+0000","last_undegraded":"2024-04-25T10:18:03.703612+0000","last_fullsized":"2024-04-25T10:18:03.703612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694368+0000","last_change":"2024-04-25T10:18:02.715822+0000","last_active":"2024-04-25T10:18:03.694368+0000","last_peered":"2024-04-25T10:18:03.694368+0000","last_clean":"2024-04-25T10:18:03.694368+0000","last_became_active":"2024-04-25T10:18:02.715507+0000","last_became_peered":"2024-04-25T10:18:02.715507+0000","last_unstale":"2024-04-25T10:18:03.694368+0000","last_undegraded":"2024-04-25T10:18:03.694368+0000","last_fullsized":"2024-04-25T10:18:03.694368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:16.553948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703351+0000","last_change":"2024-04-25T10:18:01.702739+0000","last_active":"2024-04-25T10:18:03.703351+0000","last_peered":"2024-04-25T10:18:03.703351+0000","last_clean":"2024-04-25T10:18:03.703351+0000","last_became_active":"2024-04-25T10:18:01.700517+0000","last_became_peered":"2024-04-25T10:18:01.700517+0000","last_unstale":"2024-04-25T10:18:03.703351+0000","last_undegraded":"2024-04-25T10:18:03.703351+0000","last_fullsized":"2024-04-25T10:18:03.703351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:31.803881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703257+0000","last_change":"2024-04-25T10:18:02.709746+0000","last_active":"2024-04-25T10:18:03.703257+0000","last_peered":"2024-04-25T10:18:03.703257+0000","last_clean":"2024-04-25T10:18:03.703257+0000","last_became_active":"2024-04-25T10:18:02.709446+0000","last_became_peered":"2024-04-25T10:18:02.709446+0000","last_unstale":"2024-04-25T10:18:03.703257+0000","last_undegraded":"2024-04-25T10:18:03.703257+0000","last_fullsized":"2024-04-25T10:18:03.703257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:58.435069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702550+0000","last_change":"2024-04-25T10:18:01.710729+0000","last_active":"2024-04-25T10:18:03.702550+0000","last_peered":"2024-04-25T10:18:03.702550+0000","last_clean":"2024-04-25T10:18:03.702550+0000","last_became_active":"2024-04-25T10:18:01.710485+0000","last_became_peered":"2024-04-25T10:18:01.710485+0000","last_unstale":"2024-04-25T10:18:03.702550+0000","last_undegraded":"2024-04-25T10:18:03.702550+0000","last_fullsized":"2024-04-25T10:18:03.702550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691145+0000","last_change":"2024-04-25T10:18:02.688051+0000","last_active":"2024-04-25T10:18:03.691145+0000","last_peered":"2024-04-25T10:18:03.691145+0000","last_clean":"2024-04-25T10:18:03.691145+0000","last_became_active":"2024-04-25T10:18:02.687770+0000","last_became_peered":"2024-04-25T10:18:02.687770+0000","last_unstale":"2024-04-25T10:18:03.691145+0000","last_undegraded":"2024-04-25T10:18:03.691145+0000","last_fullsized":"2024-04-25T10:18:03.691145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:51.841941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.702729+0000","last_change":"2024-04-25T10:18:01.693928+0000","last_active":"2024-04-25T10:18:02.702729+0000","last_peered":"2024-04-25T10:18:02.702729+0000","last_clean":"2024-04-25T10:18:02.702729+0000","last_became_active":"2024-04-25T10:18:01.693589+0000","last_became_peered":"2024-04-25T10:18:01.693589+0000","last_unstale":"2024-04-25T10:18:02.702729+0000","last_undegraded":"2024-04-25T10:18:02.702729+0000","last_fullsized":"2024-04-25T10:18:02.702729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:31.631333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702833+0000","last_change":"2024-04-25T10:18:02.685007+0000","last_active":"2024-04-25T10:18:03.702833+0000","last_peered":"2024-04-25T10:18:03.702833+0000","last_clean":"2024-04-25T10:18:03.702833+0000","last_became_active":"2024-04-25T10:18:02.684369+0000","last_became_peered":"2024-04-25T10:18:02.684369+0000","last_unstale":"2024-04-25T10:18:03.702833+0000","last_undegraded":"2024-04-25T10:18:03.702833+0000","last_fullsized":"2024-04-25T10:18:03.702833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704706+0000","last_change":"2024-04-25T10:18:01.702018+0000","last_active":"2024-04-25T10:18:03.704706+0000","last_peered":"2024-04-25T10:18:03.704706+0000","last_clean":"2024-04-25T10:18:03.704706+0000","last_became_active":"2024-04-25T10:18:01.701680+0000","last_became_peered":"2024-04-25T10:18:01.701680+0000","last_unstale":"2024-04-25T10:18:03.704706+0000","last_undegraded":"2024-04-25T10:18:03.704706+0000","last_fullsized":"2024-04-25T10:18:03.704706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703739+0000","last_change":"2024-04-25T10:18:02.709479+0000","last_active":"2024-04-25T10:18:03.703739+0000","last_peered":"2024-04-25T10:18:03.703739+0000","last_clean":"2024-04-25T10:18:03.703739+0000","last_became_active":"2024-04-25T10:18:02.709098+0000","last_became_peered":"2024-04-25T10:18:02.709098+0000","last_unstale":"2024-04-25T10:18:03.703739+0000","last_undegraded":"2024-04-25T10:18:03.703739+0000","last_fullsized":"2024-04-25T10:18:03.703739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.704320+0000","last_change":"2024-04-25T10:18:01.691170+0000","last_active":"2024-04-25T10:18:02.704320+0000","last_peered":"2024-04-25T10:18:02.704320+0000","last_clean":"2024-04-25T10:18:02.704320+0000","last_became_active":"2024-04-25T10:18:01.690897+0000","last_became_peered":"2024-04-25T10:18:01.690897+0000","last_unstale":"2024-04-25T10:18:02.704320+0000","last_undegraded":"2024-04-25T10:18:02.704320+0000","last_fullsized":"2024-04-25T10:18:02.704320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:19.034996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704586+0000","last_change":"2024-04-25T10:18:02.708709+0000","last_active":"2024-04-25T10:18:03.704586+0000","last_peered":"2024-04-25T10:18:03.704586+0000","last_clean":"2024-04-25T10:18:03.704586+0000","last_became_active":"2024-04-25T10:18:02.703952+0000","last_became_peered":"2024-04-25T10:18:02.703952+0000","last_unstale":"2024-04-25T10:18:03.704586+0000","last_undegraded":"2024-04-25T10:18:03.704586+0000","last_fullsized":"2024-04-25T10:18:03.704586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694644+0000","last_change":"2024-04-25T10:18:01.690031+0000","last_active":"2024-04-25T10:18:03.694644+0000","last_peered":"2024-04-25T10:18:03.694644+0000","last_clean":"2024-04-25T10:18:03.694644+0000","last_became_active":"2024-04-25T10:18:01.689608+0000","last_became_peered":"2024-04-25T10:18:01.689608+0000","last_unstale":"2024-04-25T10:18:03.694644+0000","last_undegraded":"2024-04-25T10:18:03.694644+0000","last_fullsized":"2024-04-25T10:18:03.694644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704278+0000","last_change":"2024-04-25T10:18:02.706632+0000","last_active":"2024-04-25T10:18:03.704278+0000","last_peered":"2024-04-25T10:18:03.704278+0000","last_clean":"2024-04-25T10:18:03.704278+0000","last_became_active":"2024-04-25T10:18:02.705983+0000","last_became_peered":"2024-04-25T10:18:02.705983+0000","last_unstale":"2024-04-25T10:18:03.704278+0000","last_undegraded":"2024-04-25T10:18:03.704278+0000","last_fullsized":"2024-04-25T10:18:03.704278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693832+0000","last_change":"2024-04-25T10:18:01.705153+0000","last_active":"2024-04-25T10:18:03.693832+0000","last_peered":"2024-04-25T10:18:03.693832+0000","last_clean":"2024-04-25T10:18:03.693832+0000","last_became_active":"2024-04-25T10:18:01.704844+0000","last_became_peered":"2024-04-25T10:18:01.704844+0000","last_unstale":"2024-04-25T10:18:03.693832+0000","last_undegraded":"2024-04-25T10:18:03.693832+0000","last_fullsized":"2024-04-25T10:18:03.693832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703740+0000","last_change":"2024-04-25T10:18:02.703183+0000","last_active":"2024-04-25T10:18:03.703740+0000","last_peered":"2024-04-25T10:18:03.703740+0000","last_clean":"2024-04-25T10:18:03.703740+0000","last_became_active":"2024-04-25T10:18:02.702789+0000","last_became_peered":"2024-04-25T10:18:02.702789+0000","last_unstale":"2024-04-25T10:18:03.703740+0000","last_undegraded":"2024-04-25T10:18:03.703740+0000","last_fullsized":"2024-04-25T10:18:03.703740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691080+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:03.691080+0000","last_peered":"2024-04-25T10:18:03.691080+0000","last_clean":"2024-04-25T10:18:03.691080+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:03.691080+0000","last_undegraded":"2024-04-25T10:18:03.691080+0000","last_fullsized":"2024-04-25T10:18:03.691080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703473+0000","last_change":"2024-04-25T10:18:01.687105+0000","last_active":"2024-04-25T10:18:03.703473+0000","last_peered":"2024-04-25T10:18:03.703473+0000","last_clean":"2024-04-25T10:18:03.703473+0000","last_became_active":"2024-04-25T10:18:01.686184+0000","last_became_peered":"2024-04-25T10:18:01.686184+0000","last_unstale":"2024-04-25T10:18:03.703473+0000","last_undegraded":"2024-04-25T10:18:03.703473+0000","last_fullsized":"2024-04-25T10:18:03.703473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.700867+0000","last_change":"2024-04-25T10:18:02.685204+0000","last_active":"2024-04-25T10:18:02.700867+0000","last_peered":"2024-04-25T10:18:02.700867+0000","last_clean":"2024-04-25T10:18:02.700867+0000","last_became_active":"2024-04-25T10:18:02.684900+0000","last_became_peered":"2024-04-25T10:18:02.684900+0000","last_unstale":"2024-04-25T10:18:02.700867+0000","last_undegraded":"2024-04-25T10:18:02.700867+0000","last_fullsized":"2024-04-25T10:18:02.700867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:11.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702748+0000","last_change":"2024-04-25T10:18:01.700977+0000","last_active":"2024-04-25T10:18:03.702748+0000","last_peered":"2024-04-25T10:18:03.702748+0000","last_clean":"2024-04-25T10:18:03.702748+0000","last_became_active":"2024-04-25T10:18:01.682598+0000","last_became_peered":"2024-04-25T10:18:01.682598+0000","last_unstale":"2024-04-25T10:18:03.702748+0000","last_undegraded":"2024-04-25T10:18:03.702748+0000","last_fullsized":"2024-04-25T10:18:03.702748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:17.642626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693408+0000","last_change":"2024-04-25T10:18:02.693588+0000","last_active":"2024-04-25T10:18:03.693408+0000","last_peered":"2024-04-25T10:18:03.693408+0000","last_clean":"2024-04-25T10:18:03.693408+0000","last_became_active":"2024-04-25T10:18:02.693296+0000","last_became_peered":"2024-04-25T10:18:02.693296+0000","last_unstale":"2024-04-25T10:18:03.693408+0000","last_undegraded":"2024-04-25T10:18:03.693408+0000","last_fullsized":"2024-04-25T10:18:03.693408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703521+0000","last_change":"2024-04-25T10:18:01.693364+0000","last_active":"2024-04-25T10:18:03.703521+0000","last_peered":"2024-04-25T10:18:03.703521+0000","last_clean":"2024-04-25T10:18:03.703521+0000","last_became_active":"2024-04-25T10:18:01.692980+0000","last_became_peered":"2024-04-25T10:18:01.692980+0000","last_unstale":"2024-04-25T10:18:03.703521+0000","last_undegraded":"2024-04-25T10:18:03.703521+0000","last_fullsized":"2024-04-25T10:18:03.703521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693704+0000","last_change":"2024-04-25T10:18:02.704788+0000","last_active":"2024-04-25T10:18:03.693704+0000","last_peered":"2024-04-25T10:18:03.693704+0000","last_clean":"2024-04-25T10:18:03.693704+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.693704+0000","last_undegraded":"2024-04-25T10:18:03.693704+0000","last_fullsized":"2024-04-25T10:18:03.693704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:18.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703433+0000","last_change":"2024-04-25T10:18:01.702159+0000","last_active":"2024-04-25T10:18:03.703433+0000","last_peered":"2024-04-25T10:18:03.703433+0000","last_clean":"2024-04-25T10:18:03.703433+0000","last_became_active":"2024-04-25T10:18:01.701049+0000","last_became_peered":"2024-04-25T10:18:01.701049+0000","last_unstale":"2024-04-25T10:18:03.703433+0000","last_undegraded":"2024-04-25T10:18:03.703433+0000","last_fullsized":"2024-04-25T10:18:03.703433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:31.455848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703720+0000","last_change":"2024-04-25T10:18:02.710983+0000","last_active":"2024-04-25T10:18:03.703720+0000","last_peered":"2024-04-25T10:18:03.703720+0000","last_clean":"2024-04-25T10:18:03.703720+0000","last_became_active":"2024-04-25T10:18:02.710479+0000","last_became_peered":"2024-04-25T10:18:02.710479+0000","last_unstale":"2024-04-25T10:18:03.703720+0000","last_undegraded":"2024-04-25T10:18:03.703720+0000","last_fullsized":"2024-04-25T10:18:03.703720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694576+0000","last_change":"2024-04-25T10:18:01.697707+0000","last_active":"2024-04-25T10:18:03.694576+0000","last_peered":"2024-04-25T10:18:03.694576+0000","last_clean":"2024-04-25T10:18:03.694576+0000","last_became_active":"2024-04-25T10:18:01.697322+0000","last_became_peered":"2024-04-25T10:18:01.697322+0000","last_unstale":"2024-04-25T10:18:03.694576+0000","last_undegraded":"2024-04-25T10:18:03.694576+0000","last_fullsized":"2024-04-25T10:18:03.694576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702991+0000","last_change":"2024-04-25T10:18:02.697739+0000","last_active":"2024-04-25T10:18:03.702991+0000","last_peered":"2024-04-25T10:18:03.702991+0000","last_clean":"2024-04-25T10:18:03.702991+0000","last_became_active":"2024-04-25T10:18:02.697420+0000","last_became_peered":"2024-04-25T10:18:02.697420+0000","last_unstale":"2024-04-25T10:18:03.702991+0000","last_undegraded":"2024-04-25T10:18:03.702991+0000","last_fullsized":"2024-04-25T10:18:03.702991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703868+0000","last_change":"2024-04-25T10:18:01.706705+0000","last_active":"2024-04-25T10:18:03.703868+0000","last_peered":"2024-04-25T10:18:03.703868+0000","last_clean":"2024-04-25T10:18:03.703868+0000","last_became_active":"2024-04-25T10:18:01.705850+0000","last_became_peered":"2024-04-25T10:18:01.705850+0000","last_unstale":"2024-04-25T10:18:03.703868+0000","last_undegraded":"2024-04-25T10:18:03.703868+0000","last_fullsized":"2024-04-25T10:18:03.703868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.828188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693716+0000","last_change":"2024-04-25T10:18:02.704532+0000","last_active":"2024-04-25T10:18:03.693716+0000","last_peered":"2024-04-25T10:18:03.693716+0000","last_clean":"2024-04-25T10:18:03.693716+0000","last_became_active":"2024-04-25T10:18:02.701511+0000","last_became_peered":"2024-04-25T10:18:02.701511+0000","last_unstale":"2024-04-25T10:18:03.693716+0000","last_undegraded":"2024-04-25T10:18:03.693716+0000","last_fullsized":"2024-04-25T10:18:03.693716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.681630+0000","last_change":"2024-04-25T10:18:01.698971+0000","last_active":"2024-04-25T10:18:02.681630+0000","last_peered":"2024-04-25T10:18:02.681630+0000","last_clean":"2024-04-25T10:18:02.681630+0000","last_became_active":"2024-04-25T10:18:01.698719+0000","last_became_peered":"2024-04-25T10:18:01.698719+0000","last_unstale":"2024-04-25T10:18:02.681630+0000","last_undegraded":"2024-04-25T10:18:02.681630+0000","last_fullsized":"2024-04-25T10:18:02.681630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:55.382109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.706307+0000","last_change":"2024-04-25T10:18:02.700120+0000","last_active":"2024-04-25T10:18:02.706307+0000","last_peered":"2024-04-25T10:18:02.706307+0000","last_clean":"2024-04-25T10:18:02.706307+0000","last_became_active":"2024-04-25T10:18:02.699859+0000","last_became_peered":"2024-04-25T10:18:02.699859+0000","last_unstale":"2024-04-25T10:18:02.706307+0000","last_undegraded":"2024-04-25T10:18:02.706307+0000","last_fullsized":"2024-04-25T10:18:02.706307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:56.757626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705253+0000","last_change":"2024-04-25T10:18:01.709433+0000","last_active":"2024-04-25T10:18:03.705253+0000","last_peered":"2024-04-25T10:18:03.705253+0000","last_clean":"2024-04-25T10:18:03.705253+0000","last_became_active":"2024-04-25T10:18:01.709103+0000","last_became_peered":"2024-04-25T10:18:01.709103+0000","last_unstale":"2024-04-25T10:18:03.705253+0000","last_undegraded":"2024-04-25T10:18:03.705253+0000","last_fullsized":"2024-04-25T10:18:03.705253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:59.703487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.710452+0000","last_change":"2024-04-25T10:18:02.708776+0000","last_active":"2024-04-25T10:18:02.710452+0000","last_peered":"2024-04-25T10:18:02.710452+0000","last_clean":"2024-04-25T10:18:02.710452+0000","last_became_active":"2024-04-25T10:18:02.708166+0000","last_became_peered":"2024-04-25T10:18:02.708166+0000","last_unstale":"2024-04-25T10:18:02.710452+0000","last_undegraded":"2024-04-25T10:18:02.710452+0000","last_fullsized":"2024-04-25T10:18:02.710452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:09.396400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702969+0000","last_change":"2024-04-25T10:18:01.698359+0000","last_active":"2024-04-25T10:18:03.702969+0000","last_peered":"2024-04-25T10:18:03.702969+0000","last_clean":"2024-04-25T10:18:03.702969+0000","last_became_active":"2024-04-25T10:18:01.697573+0000","last_became_peered":"2024-04-25T10:18:01.697573+0000","last_unstale":"2024-04-25T10:18:03.702969+0000","last_undegraded":"2024-04-25T10:18:03.702969+0000","last_fullsized":"2024-04-25T10:18:03.702969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702843+0000","last_change":"2024-04-25T10:18:02.703801+0000","last_active":"2024-04-25T10:18:03.702843+0000","last_peered":"2024-04-25T10:18:03.702843+0000","last_clean":"2024-04-25T10:18:03.702843+0000","last_became_active":"2024-04-25T10:18:02.703508+0000","last_became_peered":"2024-04-25T10:18:02.703508+0000","last_unstale":"2024-04-25T10:18:03.702843+0000","last_undegraded":"2024-04-25T10:18:03.702843+0000","last_fullsized":"2024-04-25T10:18:03.702843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.680166+0000","last_change":"2024-04-25T10:18:01.704197+0000","last_active":"2024-04-25T10:18:02.680166+0000","last_peered":"2024-04-25T10:18:02.680166+0000","last_clean":"2024-04-25T10:18:02.680166+0000","last_became_active":"2024-04-25T10:18:01.703932+0000","last_became_peered":"2024-04-25T10:18:01.703932+0000","last_unstale":"2024-04-25T10:18:02.680166+0000","last_undegraded":"2024-04-25T10:18:02.680166+0000","last_fullsized":"2024-04-25T10:18:02.680166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:40.452243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.706741+0000","last_change":"2024-04-25T10:18:02.702635+0000","last_active":"2024-04-25T10:18:02.706741+0000","last_peered":"2024-04-25T10:18:02.706741+0000","last_clean":"2024-04-25T10:18:02.706741+0000","last_became_active":"2024-04-25T10:18:02.702335+0000","last_became_peered":"2024-04-25T10:18:02.702335+0000","last_unstale":"2024-04-25T10:18:02.706741+0000","last_undegraded":"2024-04-25T10:18:02.706741+0000","last_fullsized":"2024-04-25T10:18:02.706741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:56.743148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694476+0000","last_change":"2024-04-25T10:18:01.692670+0000","last_active":"2024-04-25T10:18:03.694476+0000","last_peered":"2024-04-25T10:18:03.694476+0000","last_clean":"2024-04-25T10:18:03.694476+0000","last_became_active":"2024-04-25T10:18:01.691338+0000","last_became_peered":"2024-04-25T10:18:01.691338+0000","last_unstale":"2024-04-25T10:18:03.694476+0000","last_undegraded":"2024-04-25T10:18:03.694476+0000","last_fullsized":"2024-04-25T10:18:03.694476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703461+0000","last_change":"2024-04-25T10:18:02.702741+0000","last_active":"2024-04-25T10:18:03.703461+0000","last_peered":"2024-04-25T10:18:03.703461+0000","last_clean":"2024-04-25T10:18:03.703461+0000","last_became_active":"2024-04-25T10:18:02.702299+0000","last_became_peered":"2024-04-25T10:18:02.702299+0000","last_unstale":"2024-04-25T10:18:03.703461+0000","last_undegraded":"2024-04-25T10:18:03.703461+0000","last_fullsized":"2024-04-25T10:18:03.703461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703382+0000","last_change":"2024-04-25T10:18:01.686488+0000","last_active":"2024-04-25T10:18:03.703382+0000","last_peered":"2024-04-25T10:18:03.703382+0000","last_clean":"2024-04-25T10:18:03.703382+0000","last_became_active":"2024-04-25T10:18:01.685599+0000","last_became_peered":"2024-04-25T10:18:01.685599+0000","last_unstale":"2024-04-25T10:18:03.703382+0000","last_undegraded":"2024-04-25T10:18:03.703382+0000","last_fullsized":"2024-04-25T10:18:03.703382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703213+0000","last_change":"2024-04-25T10:18:02.705187+0000","last_active":"2024-04-25T10:18:03.703213+0000","last_peered":"2024-04-25T10:18:03.703213+0000","last_clean":"2024-04-25T10:18:03.703213+0000","last_became_active":"2024-04-25T10:18:02.704486+0000","last_became_peered":"2024-04-25T10:18:02.704486+0000","last_unstale":"2024-04-25T10:18:03.703213+0000","last_undegraded":"2024-04-25T10:18:03.703213+0000","last_fullsized":"2024-04-25T10:18:03.703213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703616+0000","last_change":"2024-04-25T10:18:02.716723+0000","last_active":"2024-04-25T10:18:03.703616+0000","last_peered":"2024-04-25T10:18:03.703616+0000","last_clean":"2024-04-25T10:18:03.703616+0000","last_became_active":"2024-04-25T10:18:02.716264+0000","last_became_peered":"2024-04-25T10:18:02.716264+0000","last_unstale":"2024-04-25T10:18:03.703616+0000","last_undegraded":"2024-04-25T10:18:03.703616+0000","last_fullsized":"2024-04-25T10:18:03.703616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.581498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704370+0000","last_change":"2024-04-25T10:18:01.698545+0000","last_active":"2024-04-25T10:18:03.704370+0000","last_peered":"2024-04-25T10:18:03.704370+0000","last_clean":"2024-04-25T10:18:03.704370+0000","last_became_active":"2024-04-25T10:18:01.696867+0000","last_became_peered":"2024-04-25T10:18:01.696867+0000","last_unstale":"2024-04-25T10:18:03.704370+0000","last_undegraded":"2024-04-25T10:18:03.704370+0000","last_fullsized":"2024-04-25T10:18:03.704370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691426+0000","last_change":"2024-04-25T10:18:02.712672+0000","last_active":"2024-04-25T10:18:03.691426+0000","last_peered":"2024-04-25T10:18:03.691426+0000","last_clean":"2024-04-25T10:18:03.691426+0000","last_became_active":"2024-04-25T10:18:02.712103+0000","last_became_peered":"2024-04-25T10:18:02.712103+0000","last_unstale":"2024-04-25T10:18:03.691426+0000","last_undegraded":"2024-04-25T10:18:03.691426+0000","last_fullsized":"2024-04-25T10:18:03.691426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704949+0000","last_change":"2024-04-25T10:18:01.704511+0000","last_active":"2024-04-25T10:18:03.704949+0000","last_peered":"2024-04-25T10:18:03.704949+0000","last_clean":"2024-04-25T10:18:03.704949+0000","last_became_active":"2024-04-25T10:18:01.703001+0000","last_became_peered":"2024-04-25T10:18:01.703001+0000","last_unstale":"2024-04-25T10:18:03.704949+0000","last_undegraded":"2024-04-25T10:18:03.704949+0000","last_fullsized":"2024-04-25T10:18:03.704949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694997+0000","last_change":"2024-04-25T10:18:02.690317+0000","last_active":"2024-04-25T10:18:03.694997+0000","last_peered":"2024-04-25T10:18:03.694997+0000","last_clean":"2024-04-25T10:18:03.694997+0000","last_became_active":"2024-04-25T10:18:02.687277+0000","last_became_peered":"2024-04-25T10:18:02.687277+0000","last_unstale":"2024-04-25T10:18:03.694997+0000","last_undegraded":"2024-04-25T10:18:03.694997+0000","last_fullsized":"2024-04-25T10:18:03.694997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694980+0000","last_change":"2024-04-25T10:18:01.688261+0000","last_active":"2024-04-25T10:18:03.694980+0000","last_peered":"2024-04-25T10:18:03.694980+0000","last_clean":"2024-04-25T10:18:03.694980+0000","last_became_active":"2024-04-25T10:18:01.687516+0000","last_became_peered":"2024-04-25T10:18:01.687516+0000","last_unstale":"2024-04-25T10:18:03.694980+0000","last_undegraded":"2024-04-25T10:18:03.694980+0000","last_fullsized":"2024-04-25T10:18:03.694980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694903+0000","last_change":"2024-04-25T10:18:02.687787+0000","last_active":"2024-04-25T10:18:03.694903+0000","last_peered":"2024-04-25T10:18:03.694903+0000","last_clean":"2024-04-25T10:18:03.694903+0000","last_became_active":"2024-04-25T10:18:02.687351+0000","last_became_peered":"2024-04-25T10:18:02.687351+0000","last_unstale":"2024-04-25T10:18:03.694903+0000","last_undegraded":"2024-04-25T10:18:03.694903+0000","last_fullsized":"2024-04-25T10:18:03.694903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.713027+0000","last_change":"2024-04-25T10:18:01.710038+0000","last_active":"2024-04-25T10:18:02.713027+0000","last_peered":"2024-04-25T10:18:02.713027+0000","last_clean":"2024-04-25T10:18:02.713027+0000","last_became_active":"2024-04-25T10:18:01.709789+0000","last_became_peered":"2024-04-25T10:18:01.709789+0000","last_unstale":"2024-04-25T10:18:02.713027+0000","last_undegraded":"2024-04-25T10:18:02.713027+0000","last_fullsized":"2024-04-25T10:18:02.713027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:49.219702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":660444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":806912,"data_stored":660422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":72268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:06.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 Processor -- start 2024-04-25T10:18:06.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 -- start start 2024-04-25T10:18:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b280669a0 0x7f6b28066de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b2815c210 0x7f6b2815c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b28071440 con 0x7f6b28145310 2024-04-25T10:18:06.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b28067320 con 0x7f6b2815c210 2024-04-25T10:18:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b2cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28071720 con 0x7f6b280669a0 2024-04-25T10:18:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b2815c210 0x7f6b2815c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6b28145310 0x7f6b280cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59144/0 (socket says 172.21.15.45:59144) 2024-04-25T10:18:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27400640 1 -- 172.21.15.45:0/3246141670 learned_addr learned my addr 172.21.15.45:0/3246141670 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b1ea00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b280669a0 0x7f6b28066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27e00640 1 -- 172.21.15.45:0/3246141670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b280669a0 msgr2=0x7f6b28066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b26a00640 1 -- 172.21.15.45:0/3246141670 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275138578 0 0) 0x7f6b28071440 con 0x7f6b28145310 2024-04-25T10:18:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27e00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b280669a0 0x7f6b28066de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27e00640 1 -- 172.21.15.45:0/3246141670 >> v1:172.21.15.45:6789/0 conn(0x7f6b28145310 legacy=0x7f6b280cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b27e00640 1 -- 172.21.15.45:0/3246141670 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28071a00 con 0x7f6b2815c210 2024-04-25T10:18:06.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.985+0000 7f6b1ea00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b280669a0 0x7f6b28066de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b27e00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b2815c6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009d90 tx=0x7f6b2001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3101283bc1ed0ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b26a00640 1 -- 172.21.15.45:0/3246141670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200b2070 con 0x7f6b2815c210 2024-04-25T10:18:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b26a00640 1 -- 172.21.15.45:0/3246141670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b200a5830 con 0x7f6b2815c210 2024-04-25T10:18:06.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b26a00640 1 -- 172.21.15.45:0/3246141670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200a5c00 con 0x7f6b2815c210 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 -- 172.21.15.45:0/3246141670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 msgr2=0x7f6b2815c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b2815c6a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009d90 tx=0x7f6b2001c970 comp rx=0 tx=0).stop 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 -- 172.21.15.45:0/3246141670 shutdown_connections 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b2815c6a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/3246141670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b280669a0 0x7f6b28066de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 -- 172.21.15.45:0/3246141670 >> 172.21.15.45:0/3246141670 conn(0x7f6b280d0690 msgr2=0x7f6b280d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 -- 172.21.15.45:0/3246141670 shutdown_connections 2024-04-25T10:18:06.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.989+0000 7f6b2cc00640 1 -- 172.21.15.45:0/3246141670 wait complete. 2024-04-25T10:18:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 Processor -- start 2024-04-25T10:18:06.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- start start 2024-04-25T10:18:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b280669a0 0x7f6b2815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b28145310 0x7f6b280c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b280b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280d0380 con 0x7f6b2815c210 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28168500 con 0x7f6b280669a0 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280cdb10 con 0x7f6b28145310 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b280b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b280b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57088/0 (socket says 172.21.15.45:57088) 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 -- 172.21.15.45:0/251241407 learned_addr learned my addr 172.21.15.45:0/251241407 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b1ea00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b28145310 0x7f6b280c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27400640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b280669a0 0x7f6b2815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 -- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b28145310 msgr2=0x7f6b280c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b28145310 0x7f6b280c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 -- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b280669a0 msgr2=0x7f6b2815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b280669a0 0x7f6b2815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 -- 172.21.15.45:0/251241407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280cddf0 con 0x7f6b2815c210 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27400640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b280669a0 0x7f6b2815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b27e00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b280b55e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009d90 tx=0x7f6b20019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200b2050 con 0x7f6b2815c210 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b200be070 con 0x7f6b2815c210 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b280ce0d0 con 0x7f6b2815c210 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200ada90 con 0x7f6b2815c210 2024-04-25T10:18:06.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b280ca500 con 0x7f6b2815c210 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.993+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b28075120 con 0x7f6b2815c210 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.997+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b2815d3f0 con 0x7f6b2815c210 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.997+0000 7f6b1fe00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b04066f60 0x7f6b04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.997+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b2011fec0 con 0x7f6b2815c210 2024-04-25T10:18:06.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.997+0000 7f6b27400640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b04066f60 0x7f6b04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:06.997+0000 7f6b27400640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b04066f60 0x7f6b04069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b10000f70 tx=0x7f6b10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.001+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b20016560 con 0x7f6b2815c210 2024-04-25T10:18:07.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.161+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b2813a230 con 0x7f6b04066f60 2024-04-25T10:18:07.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.169+0000 7f6b1fe00640 1 -- 172.21.15.45:0/251241407 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251934 (secure 0 0 0) 0x7f6b2813a230 con 0x7f6b04066f60 2024-04-25T10:18:07.174 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:07.176 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b04066f60 msgr2=0x7f6b04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b04066f60 0x7f6b04069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b10000f70 tx=0x7f6b10009040 comp rx=0 tx=0).stop 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 msgr2=0x7f6b280b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b280b55e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009d90 tx=0x7f6b20019710 comp rx=0 tx=0).stop 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 shutdown_connections 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b04066f60 0x7f6b04069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2815c210 0x7f6b280b55e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b28145310 0x7f6b280c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 --2- 172.21.15.45:0/251241407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b280669a0 0x7f6b2815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 >> 172.21.15.45:0/251241407 conn(0x7f6b280d0690 msgr2=0x7f6b280b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 shutdown_connections 2024-04-25T10:18:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.173+0000 7f6b2cc00640 1 -- 172.21.15.45:0/251241407 wait complete. 2024-04-25T10:18:07.195 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-25T10:18:06.873763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235680,"kb_used_data":2868,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739040,"statfs":{"total":773094113280,"available":772852776960,"internally_reserved":0,"allocated":2936832,"data_stored":1751728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001629"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704515+0000","last_change":"2024-04-25T10:18:02.719059+0000","last_active":"2024-04-25T10:18:03.704515+0000","last_peered":"2024-04-25T10:18:03.704515+0000","last_clean":"2024-04-25T10:18:03.704515+0000","last_became_active":"2024-04-25T10:18:02.718765+0000","last_became_peered":"2024-04-25T10:18:02.718765+0000","last_unstale":"2024-04-25T10:18:03.704515+0000","last_undegraded":"2024-04-25T10:18:03.704515+0000","last_fullsized":"2024-04-25T10:18:03.704515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694866+0000","last_change":"2024-04-25T10:18:02.693052+0000","last_active":"2024-04-25T10:18:03.694866+0000","last_peered":"2024-04-25T10:18:03.694866+0000","last_clean":"2024-04-25T10:18:03.694866+0000","last_became_active":"2024-04-25T10:18:02.692751+0000","last_became_peered":"2024-04-25T10:18:02.692751+0000","last_unstale":"2024-04-25T10:18:03.694866+0000","last_undegraded":"2024-04-25T10:18:03.694866+0000","last_fullsized":"2024-04-25T10:18:03.694866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694673+0000","last_change":"2024-04-25T10:18:02.694847+0000","last_active":"2024-04-25T10:18:03.694673+0000","last_peered":"2024-04-25T10:18:03.694673+0000","last_clean":"2024-04-25T10:18:03.694673+0000","last_became_active":"2024-04-25T10:18:02.694498+0000","last_became_peered":"2024-04-25T10:18:02.694498+0000","last_unstale":"2024-04-25T10:18:03.694673+0000","last_undegraded":"2024-04-25T10:18:03.694673+0000","last_fullsized":"2024-04-25T10:18:03.694673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703302+0000","last_change":"2024-04-25T10:18:02.693910+0000","last_active":"2024-04-25T10:18:03.703302+0000","last_peered":"2024-04-25T10:18:03.703302+0000","last_clean":"2024-04-25T10:18:03.703302+0000","last_became_active":"2024-04-25T10:18:02.692960+0000","last_became_peered":"2024-04-25T10:18:02.692960+0000","last_unstale":"2024-04-25T10:18:03.703302+0000","last_undegraded":"2024-04-25T10:18:03.703302+0000","last_fullsized":"2024-04-25T10:18:03.703302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703719+0000","last_change":"2024-04-25T10:18:02.709103+0000","last_active":"2024-04-25T10:18:03.703719+0000","last_peered":"2024-04-25T10:18:03.703719+0000","last_clean":"2024-04-25T10:18:03.703719+0000","last_became_active":"2024-04-25T10:18:02.708360+0000","last_became_peered":"2024-04-25T10:18:02.708360+0000","last_unstale":"2024-04-25T10:18:03.703719+0000","last_undegraded":"2024-04-25T10:18:03.703719+0000","last_fullsized":"2024-04-25T10:18:03.703719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702789+0000","last_change":"2024-04-25T10:18:02.686452+0000","last_active":"2024-04-25T10:18:03.702789+0000","last_peered":"2024-04-25T10:18:03.702789+0000","last_clean":"2024-04-25T10:18:03.702789+0000","last_became_active":"2024-04-25T10:18:02.685301+0000","last_became_peered":"2024-04-25T10:18:02.685301+0000","last_unstale":"2024-04-25T10:18:03.702789+0000","last_undegraded":"2024-04-25T10:18:03.702789+0000","last_fullsized":"2024-04-25T10:18:03.702789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.708545+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.707896+0000","last_became_peered":"2024-04-25T10:18:02.707896+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693563+0000","last_change":"2024-04-25T10:18:02.696678+0000","last_active":"2024-04-25T10:18:03.693563+0000","last_peered":"2024-04-25T10:18:03.693563+0000","last_clean":"2024-04-25T10:18:03.693563+0000","last_became_active":"2024-04-25T10:18:02.696428+0000","last_became_peered":"2024-04-25T10:18:02.696428+0000","last_unstale":"2024-04-25T10:18:03.693563+0000","last_undegraded":"2024-04-25T10:18:03.693563+0000","last_fullsized":"2024-04-25T10:18:03.693563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:19.470065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691719+0000","last_change":"2024-04-25T10:18:02.702250+0000","last_active":"2024-04-25T10:18:03.691719+0000","last_peered":"2024-04-25T10:18:03.691719+0000","last_clean":"2024-04-25T10:18:03.691719+0000","last_became_active":"2024-04-25T10:18:02.701992+0000","last_became_peered":"2024-04-25T10:18:02.701992+0000","last_unstale":"2024-04-25T10:18:03.691719+0000","last_undegraded":"2024-04-25T10:18:03.691719+0000","last_fullsized":"2024-04-25T10:18:03.691719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705200+0000","last_change":"2024-04-25T10:18:02.715021+0000","last_active":"2024-04-25T10:18:03.705200+0000","last_peered":"2024-04-25T10:18:03.705200+0000","last_clean":"2024-04-25T10:18:03.705200+0000","last_became_active":"2024-04-25T10:18:02.714746+0000","last_became_peered":"2024-04-25T10:18:02.714746+0000","last_unstale":"2024-04-25T10:18:03.705200+0000","last_undegraded":"2024-04-25T10:18:03.705200+0000","last_fullsized":"2024-04-25T10:18:03.705200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704831+0000","last_change":"2024-04-25T10:18:02.718912+0000","last_active":"2024-04-25T10:18:03.704831+0000","last_peered":"2024-04-25T10:18:03.704831+0000","last_clean":"2024-04-25T10:18:03.704831+0000","last_became_active":"2024-04-25T10:18:02.718611+0000","last_became_peered":"2024-04-25T10:18:02.718611+0000","last_unstale":"2024-04-25T10:18:03.704831+0000","last_undegraded":"2024-04-25T10:18:03.704831+0000","last_fullsized":"2024-04-25T10:18:03.704831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704014+0000","last_change":"2024-04-25T10:18:02.706863+0000","last_active":"2024-04-25T10:18:03.704014+0000","last_peered":"2024-04-25T10:18:03.704014+0000","last_clean":"2024-04-25T10:18:03.704014+0000","last_became_active":"2024-04-25T10:18:02.706447+0000","last_became_peered":"2024-04-25T10:18:02.706447+0000","last_unstale":"2024-04-25T10:18:03.704014+0000","last_undegraded":"2024-04-25T10:18:03.704014+0000","last_fullsized":"2024-04-25T10:18:03.704014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693746+0000","last_change":"2024-04-25T10:18:02.714632+0000","last_active":"2024-04-25T10:18:03.693746+0000","last_peered":"2024-04-25T10:18:03.693746+0000","last_clean":"2024-04-25T10:18:03.693746+0000","last_became_active":"2024-04-25T10:18:02.714302+0000","last_became_peered":"2024-04-25T10:18:02.714302+0000","last_unstale":"2024-04-25T10:18:03.693746+0000","last_undegraded":"2024-04-25T10:18:03.693746+0000","last_fullsized":"2024-04-25T10:18:03.693746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694231+0000","last_change":"2024-04-25T10:18:02.707614+0000","last_active":"2024-04-25T10:18:03.694231+0000","last_peered":"2024-04-25T10:18:03.694231+0000","last_clean":"2024-04-25T10:18:03.694231+0000","last_became_active":"2024-04-25T10:18:02.703697+0000","last_became_peered":"2024-04-25T10:18:02.703697+0000","last_unstale":"2024-04-25T10:18:03.694231+0000","last_undegraded":"2024-04-25T10:18:03.694231+0000","last_fullsized":"2024-04-25T10:18:03.694231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:45.419224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693574+0000","last_change":"2024-04-25T10:18:02.711384+0000","last_active":"2024-04-25T10:18:03.693574+0000","last_peered":"2024-04-25T10:18:03.693574+0000","last_clean":"2024-04-25T10:18:03.693574+0000","last_became_active":"2024-04-25T10:18:02.711036+0000","last_became_peered":"2024-04-25T10:18:02.711036+0000","last_unstale":"2024-04-25T10:18:03.693574+0000","last_undegraded":"2024-04-25T10:18:03.693574+0000","last_fullsized":"2024-04-25T10:18:03.693574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.002103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702415+0000","last_change":"2024-04-25T10:18:02.697670+0000","last_active":"2024-04-25T10:18:03.702415+0000","last_peered":"2024-04-25T10:18:03.702415+0000","last_clean":"2024-04-25T10:18:03.702415+0000","last_became_active":"2024-04-25T10:18:02.695341+0000","last_became_peered":"2024-04-25T10:18:02.695341+0000","last_unstale":"2024-04-25T10:18:03.702415+0000","last_undegraded":"2024-04-25T10:18:03.702415+0000","last_fullsized":"2024-04-25T10:18:03.702415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705262+0000","last_change":"2024-04-25T10:18:02.709655+0000","last_active":"2024-04-25T10:18:03.705262+0000","last_peered":"2024-04-25T10:18:03.705262+0000","last_clean":"2024-04-25T10:18:03.705262+0000","last_became_active":"2024-04-25T10:18:02.709349+0000","last_became_peered":"2024-04-25T10:18:02.709349+0000","last_unstale":"2024-04-25T10:18:03.705262+0000","last_undegraded":"2024-04-25T10:18:03.705262+0000","last_fullsized":"2024-04-25T10:18:03.705262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:57.065003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703843+0000","last_change":"2024-04-25T10:18:02.712551+0000","last_active":"2024-04-25T10:18:03.703843+0000","last_peered":"2024-04-25T10:18:03.703843+0000","last_clean":"2024-04-25T10:18:03.703843+0000","last_became_active":"2024-04-25T10:18:02.712257+0000","last_became_peered":"2024-04-25T10:18:02.712257+0000","last_unstale":"2024-04-25T10:18:03.703843+0000","last_undegraded":"2024-04-25T10:18:03.703843+0000","last_fullsized":"2024-04-25T10:18:03.703843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703878+0000","last_change":"2024-04-25T10:18:02.703048+0000","last_active":"2024-04-25T10:18:03.703878+0000","last_peered":"2024-04-25T10:18:03.703878+0000","last_clean":"2024-04-25T10:18:03.703878+0000","last_became_active":"2024-04-25T10:18:02.702433+0000","last_became_peered":"2024-04-25T10:18:02.702433+0000","last_unstale":"2024-04-25T10:18:03.703878+0000","last_undegraded":"2024-04-25T10:18:03.703878+0000","last_fullsized":"2024-04-25T10:18:03.703878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:25.741442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704147+0000","last_change":"2024-04-25T10:18:02.706776+0000","last_active":"2024-04-25T10:18:03.704147+0000","last_peered":"2024-04-25T10:18:03.704147+0000","last_clean":"2024-04-25T10:18:03.704147+0000","last_became_active":"2024-04-25T10:18:02.706327+0000","last_became_peered":"2024-04-25T10:18:02.706327+0000","last_unstale":"2024-04-25T10:18:03.704147+0000","last_undegraded":"2024-04-25T10:18:03.704147+0000","last_fullsized":"2024-04-25T10:18:03.704147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.700841+0000","last_change":"2024-04-25T10:18:02.685113+0000","last_active":"2024-04-25T10:18:02.700841+0000","last_peered":"2024-04-25T10:18:02.700841+0000","last_clean":"2024-04-25T10:18:02.700841+0000","last_became_active":"2024-04-25T10:18:02.684839+0000","last_became_peered":"2024-04-25T10:18:02.684839+0000","last_unstale":"2024-04-25T10:18:02.700841+0000","last_undegraded":"2024-04-25T10:18:02.700841+0000","last_fullsized":"2024-04-25T10:18:02.700841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:44.467176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691179+0000","last_change":"2024-04-25T10:18:02.713609+0000","last_active":"2024-04-25T10:18:03.691179+0000","last_peered":"2024-04-25T10:18:03.691179+0000","last_clean":"2024-04-25T10:18:03.691179+0000","last_became_active":"2024-04-25T10:18:02.713344+0000","last_became_peered":"2024-04-25T10:18:02.713344+0000","last_unstale":"2024-04-25T10:18:03.691179+0000","last_undegraded":"2024-04-25T10:18:03.691179+0000","last_fullsized":"2024-04-25T10:18:03.691179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:08.145654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693768+0000","last_change":"2024-04-25T10:18:02.688007+0000","last_active":"2024-04-25T10:18:03.693768+0000","last_peered":"2024-04-25T10:18:03.693768+0000","last_clean":"2024-04-25T10:18:03.693768+0000","last_became_active":"2024-04-25T10:18:02.687708+0000","last_became_peered":"2024-04-25T10:18:02.687708+0000","last_unstale":"2024-04-25T10:18:03.693768+0000","last_undegraded":"2024-04-25T10:18:03.693768+0000","last_fullsized":"2024-04-25T10:18:03.693768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.697693+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.696374+0000","last_became_peered":"2024-04-25T10:18:02.696374+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695340+0000","last_change":"2024-04-25T10:18:02.690318+0000","last_active":"2024-04-25T10:18:03.695340+0000","last_peered":"2024-04-25T10:18:03.695340+0000","last_clean":"2024-04-25T10:18:03.695340+0000","last_became_active":"2024-04-25T10:18:02.688844+0000","last_became_peered":"2024-04-25T10:18:02.688844+0000","last_unstale":"2024-04-25T10:18:03.695340+0000","last_undegraded":"2024-04-25T10:18:03.695340+0000","last_fullsized":"2024-04-25T10:18:03.695340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703210+0000","last_change":"2024-04-25T10:18:02.699592+0000","last_active":"2024-04-25T10:18:03.703210+0000","last_peered":"2024-04-25T10:18:03.703210+0000","last_clean":"2024-04-25T10:18:03.703210+0000","last_became_active":"2024-04-25T10:18:02.685005+0000","last_became_peered":"2024-04-25T10:18:02.685005+0000","last_unstale":"2024-04-25T10:18:03.703210+0000","last_undegraded":"2024-04-25T10:18:03.703210+0000","last_fullsized":"2024-04-25T10:18:03.703210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694078+0000","last_change":"2024-04-25T10:18:02.707729+0000","last_active":"2024-04-25T10:18:03.694078+0000","last_peered":"2024-04-25T10:18:03.694078+0000","last_clean":"2024-04-25T10:18:03.694078+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.694078+0000","last_undegraded":"2024-04-25T10:18:03.694078+0000","last_fullsized":"2024-04-25T10:18:03.694078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693598+0000","last_change":"2024-04-25T10:18:02.708638+0000","last_active":"2024-04-25T10:18:03.693598+0000","last_peered":"2024-04-25T10:18:03.693598+0000","last_clean":"2024-04-25T10:18:03.693598+0000","last_became_active":"2024-04-25T10:18:02.708316+0000","last_became_peered":"2024-04-25T10:18:02.708316+0000","last_unstale":"2024-04-25T10:18:03.693598+0000","last_undegraded":"2024-04-25T10:18:03.693598+0000","last_fullsized":"2024-04-25T10:18:03.693598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694317+0000","last_change":"2024-04-25T10:18:02.703831+0000","last_active":"2024-04-25T10:18:03.694317+0000","last_peered":"2024-04-25T10:18:03.694317+0000","last_clean":"2024-04-25T10:18:03.694317+0000","last_became_active":"2024-04-25T10:18:02.703544+0000","last_became_peered":"2024-04-25T10:18:02.703544+0000","last_unstale":"2024-04-25T10:18:03.694317+0000","last_undegraded":"2024-04-25T10:18:03.694317+0000","last_fullsized":"2024-04-25T10:18:03.694317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703007+0000","last_change":"2024-04-25T10:18:02.704805+0000","last_active":"2024-04-25T10:18:03.703007+0000","last_peered":"2024-04-25T10:18:03.703007+0000","last_clean":"2024-04-25T10:18:03.703007+0000","last_became_active":"2024-04-25T10:18:02.704430+0000","last_became_peered":"2024-04-25T10:18:02.704430+0000","last_unstale":"2024-04-25T10:18:03.703007+0000","last_undegraded":"2024-04-25T10:18:03.703007+0000","last_fullsized":"2024-04-25T10:18:03.703007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703417+0000","last_change":"2024-04-25T10:18:02.700147+0000","last_active":"2024-04-25T10:18:03.703417+0000","last_peered":"2024-04-25T10:18:03.703417+0000","last_clean":"2024-04-25T10:18:03.703417+0000","last_became_active":"2024-04-25T10:18:02.699857+0000","last_became_peered":"2024-04-25T10:18:02.699857+0000","last_unstale":"2024-04-25T10:18:03.703417+0000","last_undegraded":"2024-04-25T10:18:03.703417+0000","last_fullsized":"2024-04-25T10:18:03.703417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702527+0000","last_change":"2024-04-25T10:18:02.692533+0000","last_active":"2024-04-25T10:18:03.702527+0000","last_peered":"2024-04-25T10:18:03.702527+0000","last_clean":"2024-04-25T10:18:03.702527+0000","last_became_active":"2024-04-25T10:18:02.692239+0000","last_became_peered":"2024-04-25T10:18:02.692239+0000","last_unstale":"2024-04-25T10:18:03.702527+0000","last_undegraded":"2024-04-25T10:18:03.702527+0000","last_fullsized":"2024-04-25T10:18:03.702527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691316+0000","last_change":"2024-04-25T10:18:02.691725+0000","last_active":"2024-04-25T10:18:03.691316+0000","last_peered":"2024-04-25T10:18:03.691316+0000","last_clean":"2024-04-25T10:18:03.691316+0000","last_became_active":"2024-04-25T10:18:02.691415+0000","last_became_peered":"2024-04-25T10:18:02.691415+0000","last_unstale":"2024-04-25T10:18:03.691316+0000","last_undegraded":"2024-04-25T10:18:03.691316+0000","last_fullsized":"2024-04-25T10:18:03.691316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694378+0000","last_change":"2024-04-25T10:18:01.697822+0000","last_active":"2024-04-25T10:18:03.694378+0000","last_peered":"2024-04-25T10:18:03.694378+0000","last_clean":"2024-04-25T10:18:03.694378+0000","last_became_active":"2024-04-25T10:18:01.696666+0000","last_became_peered":"2024-04-25T10:18:01.696666+0000","last_unstale":"2024-04-25T10:18:03.694378+0000","last_undegraded":"2024-04-25T10:18:03.694378+0000","last_fullsized":"2024-04-25T10:18:03.694378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.711203+0000","last_change":"2024-04-25T10:18:02.709407+0000","last_active":"2024-04-25T10:18:02.711203+0000","last_peered":"2024-04-25T10:18:02.711203+0000","last_clean":"2024-04-25T10:18:02.711203+0000","last_became_active":"2024-04-25T10:18:02.708477+0000","last_became_peered":"2024-04-25T10:18:02.708477+0000","last_unstale":"2024-04-25T10:18:02.711203+0000","last_undegraded":"2024-04-25T10:18:02.711203+0000","last_fullsized":"2024-04-25T10:18:02.711203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:10.536062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703290+0000","last_change":"2024-04-25T10:18:01.706968+0000","last_active":"2024-04-25T10:18:03.703290+0000","last_peered":"2024-04-25T10:18:03.703290+0000","last_clean":"2024-04-25T10:18:03.703290+0000","last_became_active":"2024-04-25T10:18:01.706660+0000","last_became_peered":"2024-04-25T10:18:01.706660+0000","last_unstale":"2024-04-25T10:18:03.703290+0000","last_undegraded":"2024-04-25T10:18:03.703290+0000","last_fullsized":"2024-04-25T10:18:03.703290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694922+0000","last_change":"2024-04-25T10:18:01.705160+0000","last_active":"2024-04-25T10:18:03.694922+0000","last_peered":"2024-04-25T10:18:03.694922+0000","last_clean":"2024-04-25T10:18:03.694922+0000","last_became_active":"2024-04-25T10:18:01.704852+0000","last_became_peered":"2024-04-25T10:18:01.704852+0000","last_unstale":"2024-04-25T10:18:03.694922+0000","last_undegraded":"2024-04-25T10:18:03.694922+0000","last_fullsized":"2024-04-25T10:18:03.694922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:33.140157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693846+0000","last_change":"2024-04-25T10:18:02.691734+0000","last_active":"2024-04-25T10:18:03.693846+0000","last_peered":"2024-04-25T10:18:03.693846+0000","last_clean":"2024-04-25T10:18:03.693846+0000","last_became_active":"2024-04-25T10:18:02.691454+0000","last_became_peered":"2024-04-25T10:18:02.691454+0000","last_unstale":"2024-04-25T10:18:03.693846+0000","last_undegraded":"2024-04-25T10:18:03.693846+0000","last_fullsized":"2024-04-25T10:18:03.693846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702652+0000","last_change":"2024-04-25T10:18:01.684950+0000","last_active":"2024-04-25T10:18:03.702652+0000","last_peered":"2024-04-25T10:18:03.702652+0000","last_clean":"2024-04-25T10:18:03.702652+0000","last_became_active":"2024-04-25T10:18:01.682962+0000","last_became_peered":"2024-04-25T10:18:01.682962+0000","last_unstale":"2024-04-25T10:18:03.702652+0000","last_undegraded":"2024-04-25T10:18:03.702652+0000","last_fullsized":"2024-04-25T10:18:03.702652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:45.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.704948+0000","last_change":"2024-04-25T10:18:02.695250+0000","last_active":"2024-04-25T10:18:02.704948+0000","last_peered":"2024-04-25T10:18:02.704948+0000","last_clean":"2024-04-25T10:18:02.704948+0000","last_became_active":"2024-04-25T10:18:02.694926+0000","last_became_peered":"2024-04-25T10:18:02.694926+0000","last_unstale":"2024-04-25T10:18:02.704948+0000","last_undegraded":"2024-04-25T10:18:02.704948+0000","last_fullsized":"2024-04-25T10:18:02.704948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:47.209619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691706+0000","last_change":"2024-04-25T10:18:01.691624+0000","last_active":"2024-04-25T10:18:03.691706+0000","last_peered":"2024-04-25T10:18:03.691706+0000","last_clean":"2024-04-25T10:18:03.691706+0000","last_became_active":"2024-04-25T10:18:01.687161+0000","last_became_peered":"2024-04-25T10:18:01.687161+0000","last_unstale":"2024-04-25T10:18:03.691706+0000","last_undegraded":"2024-04-25T10:18:03.691706+0000","last_fullsized":"2024-04-25T10:18:03.691706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703606+0000","last_change":"2024-04-25T10:18:02.685373+0000","last_active":"2024-04-25T10:18:03.703606+0000","last_peered":"2024-04-25T10:18:03.703606+0000","last_clean":"2024-04-25T10:18:03.703606+0000","last_became_active":"2024-04-25T10:18:02.684600+0000","last_became_peered":"2024-04-25T10:18:02.684600+0000","last_unstale":"2024-04-25T10:18:03.703606+0000","last_undegraded":"2024-04-25T10:18:03.703606+0000","last_fullsized":"2024-04-25T10:18:03.703606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705362+0000","last_change":"2024-04-25T10:18:01.703304+0000","last_active":"2024-04-25T10:18:03.705362+0000","last_peered":"2024-04-25T10:18:03.705362+0000","last_clean":"2024-04-25T10:18:03.705362+0000","last_became_active":"2024-04-25T10:18:01.701629+0000","last_became_peered":"2024-04-25T10:18:01.701629+0000","last_unstale":"2024-04-25T10:18:03.705362+0000","last_undegraded":"2024-04-25T10:18:03.705362+0000","last_fullsized":"2024-04-25T10:18:03.705362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695152+0000","last_change":"2024-04-25T10:18:02.696745+0000","last_active":"2024-04-25T10:18:03.695152+0000","last_peered":"2024-04-25T10:18:03.695152+0000","last_clean":"2024-04-25T10:18:03.695152+0000","last_became_active":"2024-04-25T10:18:02.696514+0000","last_became_peered":"2024-04-25T10:18:02.696514+0000","last_unstale":"2024-04-25T10:18:03.695152+0000","last_undegraded":"2024-04-25T10:18:03.695152+0000","last_fullsized":"2024-04-25T10:18:03.695152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703782+0000","last_change":"2024-04-25T10:18:01.692901+0000","last_active":"2024-04-25T10:18:03.703782+0000","last_peered":"2024-04-25T10:18:03.703782+0000","last_clean":"2024-04-25T10:18:03.703782+0000","last_became_active":"2024-04-25T10:18:01.692604+0000","last_became_peered":"2024-04-25T10:18:01.692604+0000","last_unstale":"2024-04-25T10:18:03.703782+0000","last_undegraded":"2024-04-25T10:18:03.703782+0000","last_fullsized":"2024-04-25T10:18:03.703782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703310+0000","last_change":"2024-04-25T10:18:02.696653+0000","last_active":"2024-04-25T10:18:03.703310+0000","last_peered":"2024-04-25T10:18:03.703310+0000","last_clean":"2024-04-25T10:18:03.703310+0000","last_became_active":"2024-04-25T10:18:02.696271+0000","last_became_peered":"2024-04-25T10:18:02.696271+0000","last_unstale":"2024-04-25T10:18:03.703310+0000","last_undegraded":"2024-04-25T10:18:03.703310+0000","last_fullsized":"2024-04-25T10:18:03.703310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:35.561629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703399+0000","last_change":"2024-04-25T10:18:01.696254+0000","last_active":"2024-04-25T10:18:03.703399+0000","last_peered":"2024-04-25T10:18:03.703399+0000","last_clean":"2024-04-25T10:18:03.703399+0000","last_became_active":"2024-04-25T10:18:01.695894+0000","last_became_peered":"2024-04-25T10:18:01.695894+0000","last_unstale":"2024-04-25T10:18:03.703399+0000","last_undegraded":"2024-04-25T10:18:03.703399+0000","last_fullsized":"2024-04-25T10:18:03.703399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704898+0000","last_change":"2024-04-25T10:18:02.707539+0000","last_active":"2024-04-25T10:18:03.704898+0000","last_peered":"2024-04-25T10:18:03.704898+0000","last_clean":"2024-04-25T10:18:03.704898+0000","last_became_active":"2024-04-25T10:18:02.707294+0000","last_became_peered":"2024-04-25T10:18:02.707294+0000","last_unstale":"2024-04-25T10:18:03.704898+0000","last_undegraded":"2024-04-25T10:18:03.704898+0000","last_fullsized":"2024-04-25T10:18:03.704898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703612+0000","last_change":"2024-04-25T10:18:01.703958+0000","last_active":"2024-04-25T10:18:03.703612+0000","last_peered":"2024-04-25T10:18:03.703612+0000","last_clean":"2024-04-25T10:18:03.703612+0000","last_became_active":"2024-04-25T10:18:01.703524+0000","last_became_peered":"2024-04-25T10:18:01.703524+0000","last_unstale":"2024-04-25T10:18:03.703612+0000","last_undegraded":"2024-04-25T10:18:03.703612+0000","last_fullsized":"2024-04-25T10:18:03.703612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694368+0000","last_change":"2024-04-25T10:18:02.715822+0000","last_active":"2024-04-25T10:18:03.694368+0000","last_peered":"2024-04-25T10:18:03.694368+0000","last_clean":"2024-04-25T10:18:03.694368+0000","last_became_active":"2024-04-25T10:18:02.715507+0000","last_became_peered":"2024-04-25T10:18:02.715507+0000","last_unstale":"2024-04-25T10:18:03.694368+0000","last_undegraded":"2024-04-25T10:18:03.694368+0000","last_fullsized":"2024-04-25T10:18:03.694368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:16.553948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703351+0000","last_change":"2024-04-25T10:18:01.702739+0000","last_active":"2024-04-25T10:18:03.703351+0000","last_peered":"2024-04-25T10:18:03.703351+0000","last_clean":"2024-04-25T10:18:03.703351+0000","last_became_active":"2024-04-25T10:18:01.700517+0000","last_became_peered":"2024-04-25T10:18:01.700517+0000","last_unstale":"2024-04-25T10:18:03.703351+0000","last_undegraded":"2024-04-25T10:18:03.703351+0000","last_fullsized":"2024-04-25T10:18:03.703351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:31.803881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703257+0000","last_change":"2024-04-25T10:18:02.709746+0000","last_active":"2024-04-25T10:18:03.703257+0000","last_peered":"2024-04-25T10:18:03.703257+0000","last_clean":"2024-04-25T10:18:03.703257+0000","last_became_active":"2024-04-25T10:18:02.709446+0000","last_became_peered":"2024-04-25T10:18:02.709446+0000","last_unstale":"2024-04-25T10:18:03.703257+0000","last_undegraded":"2024-04-25T10:18:03.703257+0000","last_fullsized":"2024-04-25T10:18:03.703257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:58.435069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702550+0000","last_change":"2024-04-25T10:18:01.710729+0000","last_active":"2024-04-25T10:18:03.702550+0000","last_peered":"2024-04-25T10:18:03.702550+0000","last_clean":"2024-04-25T10:18:03.702550+0000","last_became_active":"2024-04-25T10:18:01.710485+0000","last_became_peered":"2024-04-25T10:18:01.710485+0000","last_unstale":"2024-04-25T10:18:03.702550+0000","last_undegraded":"2024-04-25T10:18:03.702550+0000","last_fullsized":"2024-04-25T10:18:03.702550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691145+0000","last_change":"2024-04-25T10:18:02.688051+0000","last_active":"2024-04-25T10:18:03.691145+0000","last_peered":"2024-04-25T10:18:03.691145+0000","last_clean":"2024-04-25T10:18:03.691145+0000","last_became_active":"2024-04-25T10:18:02.687770+0000","last_became_peered":"2024-04-25T10:18:02.687770+0000","last_unstale":"2024-04-25T10:18:03.691145+0000","last_undegraded":"2024-04-25T10:18:03.691145+0000","last_fullsized":"2024-04-25T10:18:03.691145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:51.841941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.702729+0000","last_change":"2024-04-25T10:18:01.693928+0000","last_active":"2024-04-25T10:18:02.702729+0000","last_peered":"2024-04-25T10:18:02.702729+0000","last_clean":"2024-04-25T10:18:02.702729+0000","last_became_active":"2024-04-25T10:18:01.693589+0000","last_became_peered":"2024-04-25T10:18:01.693589+0000","last_unstale":"2024-04-25T10:18:02.702729+0000","last_undegraded":"2024-04-25T10:18:02.702729+0000","last_fullsized":"2024-04-25T10:18:02.702729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:31.631333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702833+0000","last_change":"2024-04-25T10:18:02.685007+0000","last_active":"2024-04-25T10:18:03.702833+0000","last_peered":"2024-04-25T10:18:03.702833+0000","last_clean":"2024-04-25T10:18:03.702833+0000","last_became_active":"2024-04-25T10:18:02.684369+0000","last_became_peered":"2024-04-25T10:18:02.684369+0000","last_unstale":"2024-04-25T10:18:03.702833+0000","last_undegraded":"2024-04-25T10:18:03.702833+0000","last_fullsized":"2024-04-25T10:18:03.702833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704706+0000","last_change":"2024-04-25T10:18:01.702018+0000","last_active":"2024-04-25T10:18:03.704706+0000","last_peered":"2024-04-25T10:18:03.704706+0000","last_clean":"2024-04-25T10:18:03.704706+0000","last_became_active":"2024-04-25T10:18:01.701680+0000","last_became_peered":"2024-04-25T10:18:01.701680+0000","last_unstale":"2024-04-25T10:18:03.704706+0000","last_undegraded":"2024-04-25T10:18:03.704706+0000","last_fullsized":"2024-04-25T10:18:03.704706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703739+0000","last_change":"2024-04-25T10:18:02.709479+0000","last_active":"2024-04-25T10:18:03.703739+0000","last_peered":"2024-04-25T10:18:03.703739+0000","last_clean":"2024-04-25T10:18:03.703739+0000","last_became_active":"2024-04-25T10:18:02.709098+0000","last_became_peered":"2024-04-25T10:18:02.709098+0000","last_unstale":"2024-04-25T10:18:03.703739+0000","last_undegraded":"2024-04-25T10:18:03.703739+0000","last_fullsized":"2024-04-25T10:18:03.703739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.704320+0000","last_change":"2024-04-25T10:18:01.691170+0000","last_active":"2024-04-25T10:18:02.704320+0000","last_peered":"2024-04-25T10:18:02.704320+0000","last_clean":"2024-04-25T10:18:02.704320+0000","last_became_active":"2024-04-25T10:18:01.690897+0000","last_became_peered":"2024-04-25T10:18:01.690897+0000","last_unstale":"2024-04-25T10:18:02.704320+0000","last_undegraded":"2024-04-25T10:18:02.704320+0000","last_fullsized":"2024-04-25T10:18:02.704320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:19.034996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704586+0000","last_change":"2024-04-25T10:18:02.708709+0000","last_active":"2024-04-25T10:18:03.704586+0000","last_peered":"2024-04-25T10:18:03.704586+0000","last_clean":"2024-04-25T10:18:03.704586+0000","last_became_active":"2024-04-25T10:18:02.703952+0000","last_became_peered":"2024-04-25T10:18:02.703952+0000","last_unstale":"2024-04-25T10:18:03.704586+0000","last_undegraded":"2024-04-25T10:18:03.704586+0000","last_fullsized":"2024-04-25T10:18:03.704586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694644+0000","last_change":"2024-04-25T10:18:01.690031+0000","last_active":"2024-04-25T10:18:03.694644+0000","last_peered":"2024-04-25T10:18:03.694644+0000","last_clean":"2024-04-25T10:18:03.694644+0000","last_became_active":"2024-04-25T10:18:01.689608+0000","last_became_peered":"2024-04-25T10:18:01.689608+0000","last_unstale":"2024-04-25T10:18:03.694644+0000","last_undegraded":"2024-04-25T10:18:03.694644+0000","last_fullsized":"2024-04-25T10:18:03.694644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704278+0000","last_change":"2024-04-25T10:18:02.706632+0000","last_active":"2024-04-25T10:18:03.704278+0000","last_peered":"2024-04-25T10:18:03.704278+0000","last_clean":"2024-04-25T10:18:03.704278+0000","last_became_active":"2024-04-25T10:18:02.705983+0000","last_became_peered":"2024-04-25T10:18:02.705983+0000","last_unstale":"2024-04-25T10:18:03.704278+0000","last_undegraded":"2024-04-25T10:18:03.704278+0000","last_fullsized":"2024-04-25T10:18:03.704278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693832+0000","last_change":"2024-04-25T10:18:01.705153+0000","last_active":"2024-04-25T10:18:03.693832+0000","last_peered":"2024-04-25T10:18:03.693832+0000","last_clean":"2024-04-25T10:18:03.693832+0000","last_became_active":"2024-04-25T10:18:01.704844+0000","last_became_peered":"2024-04-25T10:18:01.704844+0000","last_unstale":"2024-04-25T10:18:03.693832+0000","last_undegraded":"2024-04-25T10:18:03.693832+0000","last_fullsized":"2024-04-25T10:18:03.693832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703740+0000","last_change":"2024-04-25T10:18:02.703183+0000","last_active":"2024-04-25T10:18:03.703740+0000","last_peered":"2024-04-25T10:18:03.703740+0000","last_clean":"2024-04-25T10:18:03.703740+0000","last_became_active":"2024-04-25T10:18:02.702789+0000","last_became_peered":"2024-04-25T10:18:02.702789+0000","last_unstale":"2024-04-25T10:18:03.703740+0000","last_undegraded":"2024-04-25T10:18:03.703740+0000","last_fullsized":"2024-04-25T10:18:03.703740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691080+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:03.691080+0000","last_peered":"2024-04-25T10:18:03.691080+0000","last_clean":"2024-04-25T10:18:03.691080+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:03.691080+0000","last_undegraded":"2024-04-25T10:18:03.691080+0000","last_fullsized":"2024-04-25T10:18:03.691080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703473+0000","last_change":"2024-04-25T10:18:01.687105+0000","last_active":"2024-04-25T10:18:03.703473+0000","last_peered":"2024-04-25T10:18:03.703473+0000","last_clean":"2024-04-25T10:18:03.703473+0000","last_became_active":"2024-04-25T10:18:01.686184+0000","last_became_peered":"2024-04-25T10:18:01.686184+0000","last_unstale":"2024-04-25T10:18:03.703473+0000","last_undegraded":"2024-04-25T10:18:03.703473+0000","last_fullsized":"2024-04-25T10:18:03.703473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.700867+0000","last_change":"2024-04-25T10:18:02.685204+0000","last_active":"2024-04-25T10:18:02.700867+0000","last_peered":"2024-04-25T10:18:02.700867+0000","last_clean":"2024-04-25T10:18:02.700867+0000","last_became_active":"2024-04-25T10:18:02.684900+0000","last_became_peered":"2024-04-25T10:18:02.684900+0000","last_unstale":"2024-04-25T10:18:02.700867+0000","last_undegraded":"2024-04-25T10:18:02.700867+0000","last_fullsized":"2024-04-25T10:18:02.700867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:11.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702748+0000","last_change":"2024-04-25T10:18:01.700977+0000","last_active":"2024-04-25T10:18:03.702748+0000","last_peered":"2024-04-25T10:18:03.702748+0000","last_clean":"2024-04-25T10:18:03.702748+0000","last_became_active":"2024-04-25T10:18:01.682598+0000","last_became_peered":"2024-04-25T10:18:01.682598+0000","last_unstale":"2024-04-25T10:18:03.702748+0000","last_undegraded":"2024-04-25T10:18:03.702748+0000","last_fullsized":"2024-04-25T10:18:03.702748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:17.642626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693408+0000","last_change":"2024-04-25T10:18:02.693588+0000","last_active":"2024-04-25T10:18:03.693408+0000","last_peered":"2024-04-25T10:18:03.693408+0000","last_clean":"2024-04-25T10:18:03.693408+0000","last_became_active":"2024-04-25T10:18:02.693296+0000","last_became_peered":"2024-04-25T10:18:02.693296+0000","last_unstale":"2024-04-25T10:18:03.693408+0000","last_undegraded":"2024-04-25T10:18:03.693408+0000","last_fullsized":"2024-04-25T10:18:03.693408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703521+0000","last_change":"2024-04-25T10:18:01.693364+0000","last_active":"2024-04-25T10:18:03.703521+0000","last_peered":"2024-04-25T10:18:03.703521+0000","last_clean":"2024-04-25T10:18:03.703521+0000","last_became_active":"2024-04-25T10:18:01.692980+0000","last_became_peered":"2024-04-25T10:18:01.692980+0000","last_unstale":"2024-04-25T10:18:03.703521+0000","last_undegraded":"2024-04-25T10:18:03.703521+0000","last_fullsized":"2024-04-25T10:18:03.703521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693704+0000","last_change":"2024-04-25T10:18:02.704788+0000","last_active":"2024-04-25T10:18:03.693704+0000","last_peered":"2024-04-25T10:18:03.693704+0000","last_clean":"2024-04-25T10:18:03.693704+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.693704+0000","last_undegraded":"2024-04-25T10:18:03.693704+0000","last_fullsized":"2024-04-25T10:18:03.693704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:18.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703433+0000","last_change":"2024-04-25T10:18:01.702159+0000","last_active":"2024-04-25T10:18:03.703433+0000","last_peered":"2024-04-25T10:18:03.703433+0000","last_clean":"2024-04-25T10:18:03.703433+0000","last_became_active":"2024-04-25T10:18:01.701049+0000","last_became_peered":"2024-04-25T10:18:01.701049+0000","last_unstale":"2024-04-25T10:18:03.703433+0000","last_undegraded":"2024-04-25T10:18:03.703433+0000","last_fullsized":"2024-04-25T10:18:03.703433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:31.455848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703720+0000","last_change":"2024-04-25T10:18:02.710983+0000","last_active":"2024-04-25T10:18:03.703720+0000","last_peered":"2024-04-25T10:18:03.703720+0000","last_clean":"2024-04-25T10:18:03.703720+0000","last_became_active":"2024-04-25T10:18:02.710479+0000","last_became_peered":"2024-04-25T10:18:02.710479+0000","last_unstale":"2024-04-25T10:18:03.703720+0000","last_undegraded":"2024-04-25T10:18:03.703720+0000","last_fullsized":"2024-04-25T10:18:03.703720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694576+0000","last_change":"2024-04-25T10:18:01.697707+0000","last_active":"2024-04-25T10:18:03.694576+0000","last_peered":"2024-04-25T10:18:03.694576+0000","last_clean":"2024-04-25T10:18:03.694576+0000","last_became_active":"2024-04-25T10:18:01.697322+0000","last_became_peered":"2024-04-25T10:18:01.697322+0000","last_unstale":"2024-04-25T10:18:03.694576+0000","last_undegraded":"2024-04-25T10:18:03.694576+0000","last_fullsized":"2024-04-25T10:18:03.694576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702991+0000","last_change":"2024-04-25T10:18:02.697739+0000","last_active":"2024-04-25T10:18:03.702991+0000","last_peered":"2024-04-25T10:18:03.702991+0000","last_clean":"2024-04-25T10:18:03.702991+0000","last_became_active":"2024-04-25T10:18:02.697420+0000","last_became_peered":"2024-04-25T10:18:02.697420+0000","last_unstale":"2024-04-25T10:18:03.702991+0000","last_undegraded":"2024-04-25T10:18:03.702991+0000","last_fullsized":"2024-04-25T10:18:03.702991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703868+0000","last_change":"2024-04-25T10:18:01.706705+0000","last_active":"2024-04-25T10:18:03.703868+0000","last_peered":"2024-04-25T10:18:03.703868+0000","last_clean":"2024-04-25T10:18:03.703868+0000","last_became_active":"2024-04-25T10:18:01.705850+0000","last_became_peered":"2024-04-25T10:18:01.705850+0000","last_unstale":"2024-04-25T10:18:03.703868+0000","last_undegraded":"2024-04-25T10:18:03.703868+0000","last_fullsized":"2024-04-25T10:18:03.703868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.828188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693716+0000","last_change":"2024-04-25T10:18:02.704532+0000","last_active":"2024-04-25T10:18:03.693716+0000","last_peered":"2024-04-25T10:18:03.693716+0000","last_clean":"2024-04-25T10:18:03.693716+0000","last_became_active":"2024-04-25T10:18:02.701511+0000","last_became_peered":"2024-04-25T10:18:02.701511+0000","last_unstale":"2024-04-25T10:18:03.693716+0000","last_undegraded":"2024-04-25T10:18:03.693716+0000","last_fullsized":"2024-04-25T10:18:03.693716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.681630+0000","last_change":"2024-04-25T10:18:01.698971+0000","last_active":"2024-04-25T10:18:02.681630+0000","last_peered":"2024-04-25T10:18:02.681630+0000","last_clean":"2024-04-25T10:18:02.681630+0000","last_became_active":"2024-04-25T10:18:01.698719+0000","last_became_peered":"2024-04-25T10:18:01.698719+0000","last_unstale":"2024-04-25T10:18:02.681630+0000","last_undegraded":"2024-04-25T10:18:02.681630+0000","last_fullsized":"2024-04-25T10:18:02.681630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:55.382109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.706307+0000","last_change":"2024-04-25T10:18:02.700120+0000","last_active":"2024-04-25T10:18:02.706307+0000","last_peered":"2024-04-25T10:18:02.706307+0000","last_clean":"2024-04-25T10:18:02.706307+0000","last_became_active":"2024-04-25T10:18:02.699859+0000","last_became_peered":"2024-04-25T10:18:02.699859+0000","last_unstale":"2024-04-25T10:18:02.706307+0000","last_undegraded":"2024-04-25T10:18:02.706307+0000","last_fullsized":"2024-04-25T10:18:02.706307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:56.757626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705253+0000","last_change":"2024-04-25T10:18:01.709433+0000","last_active":"2024-04-25T10:18:03.705253+0000","last_peered":"2024-04-25T10:18:03.705253+0000","last_clean":"2024-04-25T10:18:03.705253+0000","last_became_active":"2024-04-25T10:18:01.709103+0000","last_became_peered":"2024-04-25T10:18:01.709103+0000","last_unstale":"2024-04-25T10:18:03.705253+0000","last_undegraded":"2024-04-25T10:18:03.705253+0000","last_fullsized":"2024-04-25T10:18:03.705253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:59.703487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.710452+0000","last_change":"2024-04-25T10:18:02.708776+0000","last_active":"2024-04-25T10:18:02.710452+0000","last_peered":"2024-04-25T10:18:02.710452+0000","last_clean":"2024-04-25T10:18:02.710452+0000","last_became_active":"2024-04-25T10:18:02.708166+0000","last_became_peered":"2024-04-25T10:18:02.708166+0000","last_unstale":"2024-04-25T10:18:02.710452+0000","last_undegraded":"2024-04-25T10:18:02.710452+0000","last_fullsized":"2024-04-25T10:18:02.710452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:09.396400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702969+0000","last_change":"2024-04-25T10:18:01.698359+0000","last_active":"2024-04-25T10:18:03.702969+0000","last_peered":"2024-04-25T10:18:03.702969+0000","last_clean":"2024-04-25T10:18:03.702969+0000","last_became_active":"2024-04-25T10:18:01.697573+0000","last_became_peered":"2024-04-25T10:18:01.697573+0000","last_unstale":"2024-04-25T10:18:03.702969+0000","last_undegraded":"2024-04-25T10:18:03.702969+0000","last_fullsized":"2024-04-25T10:18:03.702969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702843+0000","last_change":"2024-04-25T10:18:02.703801+0000","last_active":"2024-04-25T10:18:03.702843+0000","last_peered":"2024-04-25T10:18:03.702843+0000","last_clean":"2024-04-25T10:18:03.702843+0000","last_became_active":"2024-04-25T10:18:02.703508+0000","last_became_peered":"2024-04-25T10:18:02.703508+0000","last_unstale":"2024-04-25T10:18:03.702843+0000","last_undegraded":"2024-04-25T10:18:03.702843+0000","last_fullsized":"2024-04-25T10:18:03.702843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.680166+0000","last_change":"2024-04-25T10:18:01.704197+0000","last_active":"2024-04-25T10:18:02.680166+0000","last_peered":"2024-04-25T10:18:02.680166+0000","last_clean":"2024-04-25T10:18:02.680166+0000","last_became_active":"2024-04-25T10:18:01.703932+0000","last_became_peered":"2024-04-25T10:18:01.703932+0000","last_unstale":"2024-04-25T10:18:02.680166+0000","last_undegraded":"2024-04-25T10:18:02.680166+0000","last_fullsized":"2024-04-25T10:18:02.680166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:40.452243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.706741+0000","last_change":"2024-04-25T10:18:02.702635+0000","last_active":"2024-04-25T10:18:02.706741+0000","last_peered":"2024-04-25T10:18:02.706741+0000","last_clean":"2024-04-25T10:18:02.706741+0000","last_became_active":"2024-04-25T10:18:02.702335+0000","last_became_peered":"2024-04-25T10:18:02.702335+0000","last_unstale":"2024-04-25T10:18:02.706741+0000","last_undegraded":"2024-04-25T10:18:02.706741+0000","last_fullsized":"2024-04-25T10:18:02.706741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:56.743148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694476+0000","last_change":"2024-04-25T10:18:01.692670+0000","last_active":"2024-04-25T10:18:03.694476+0000","last_peered":"2024-04-25T10:18:03.694476+0000","last_clean":"2024-04-25T10:18:03.694476+0000","last_became_active":"2024-04-25T10:18:01.691338+0000","last_became_peered":"2024-04-25T10:18:01.691338+0000","last_unstale":"2024-04-25T10:18:03.694476+0000","last_undegraded":"2024-04-25T10:18:03.694476+0000","last_fullsized":"2024-04-25T10:18:03.694476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703461+0000","last_change":"2024-04-25T10:18:02.702741+0000","last_active":"2024-04-25T10:18:03.703461+0000","last_peered":"2024-04-25T10:18:03.703461+0000","last_clean":"2024-04-25T10:18:03.703461+0000","last_became_active":"2024-04-25T10:18:02.702299+0000","last_became_peered":"2024-04-25T10:18:02.702299+0000","last_unstale":"2024-04-25T10:18:03.703461+0000","last_undegraded":"2024-04-25T10:18:03.703461+0000","last_fullsized":"2024-04-25T10:18:03.703461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703382+0000","last_change":"2024-04-25T10:18:01.686488+0000","last_active":"2024-04-25T10:18:03.703382+0000","last_peered":"2024-04-25T10:18:03.703382+0000","last_clean":"2024-04-25T10:18:03.703382+0000","last_became_active":"2024-04-25T10:18:01.685599+0000","last_became_peered":"2024-04-25T10:18:01.685599+0000","last_unstale":"2024-04-25T10:18:03.703382+0000","last_undegraded":"2024-04-25T10:18:03.703382+0000","last_fullsized":"2024-04-25T10:18:03.703382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703213+0000","last_change":"2024-04-25T10:18:02.705187+0000","last_active":"2024-04-25T10:18:03.703213+0000","last_peered":"2024-04-25T10:18:03.703213+0000","last_clean":"2024-04-25T10:18:03.703213+0000","last_became_active":"2024-04-25T10:18:02.704486+0000","last_became_peered":"2024-04-25T10:18:02.704486+0000","last_unstale":"2024-04-25T10:18:03.703213+0000","last_undegraded":"2024-04-25T10:18:03.703213+0000","last_fullsized":"2024-04-25T10:18:03.703213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703616+0000","last_change":"2024-04-25T10:18:02.716723+0000","last_active":"2024-04-25T10:18:03.703616+0000","last_peered":"2024-04-25T10:18:03.703616+0000","last_clean":"2024-04-25T10:18:03.703616+0000","last_became_active":"2024-04-25T10:18:02.716264+0000","last_became_peered":"2024-04-25T10:18:02.716264+0000","last_unstale":"2024-04-25T10:18:03.703616+0000","last_undegraded":"2024-04-25T10:18:03.703616+0000","last_fullsized":"2024-04-25T10:18:03.703616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.581498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704370+0000","last_change":"2024-04-25T10:18:01.698545+0000","last_active":"2024-04-25T10:18:03.704370+0000","last_peered":"2024-04-25T10:18:03.704370+0000","last_clean":"2024-04-25T10:18:03.704370+0000","last_became_active":"2024-04-25T10:18:01.696867+0000","last_became_peered":"2024-04-25T10:18:01.696867+0000","last_unstale":"2024-04-25T10:18:03.704370+0000","last_undegraded":"2024-04-25T10:18:03.704370+0000","last_fullsized":"2024-04-25T10:18:03.704370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691426+0000","last_change":"2024-04-25T10:18:02.712672+0000","last_active":"2024-04-25T10:18:03.691426+0000","last_peered":"2024-04-25T10:18:03.691426+0000","last_clean":"2024-04-25T10:18:03.691426+0000","last_became_active":"2024-04-25T10:18:02.712103+0000","last_became_peered":"2024-04-25T10:18:02.712103+0000","last_unstale":"2024-04-25T10:18:03.691426+0000","last_undegraded":"2024-04-25T10:18:03.691426+0000","last_fullsized":"2024-04-25T10:18:03.691426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704949+0000","last_change":"2024-04-25T10:18:01.704511+0000","last_active":"2024-04-25T10:18:03.704949+0000","last_peered":"2024-04-25T10:18:03.704949+0000","last_clean":"2024-04-25T10:18:03.704949+0000","last_became_active":"2024-04-25T10:18:01.703001+0000","last_became_peered":"2024-04-25T10:18:01.703001+0000","last_unstale":"2024-04-25T10:18:03.704949+0000","last_undegraded":"2024-04-25T10:18:03.704949+0000","last_fullsized":"2024-04-25T10:18:03.704949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694997+0000","last_change":"2024-04-25T10:18:02.690317+0000","last_active":"2024-04-25T10:18:03.694997+0000","last_peered":"2024-04-25T10:18:03.694997+0000","last_clean":"2024-04-25T10:18:03.694997+0000","last_became_active":"2024-04-25T10:18:02.687277+0000","last_became_peered":"2024-04-25T10:18:02.687277+0000","last_unstale":"2024-04-25T10:18:03.694997+0000","last_undegraded":"2024-04-25T10:18:03.694997+0000","last_fullsized":"2024-04-25T10:18:03.694997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694980+0000","last_change":"2024-04-25T10:18:01.688261+0000","last_active":"2024-04-25T10:18:03.694980+0000","last_peered":"2024-04-25T10:18:03.694980+0000","last_clean":"2024-04-25T10:18:03.694980+0000","last_became_active":"2024-04-25T10:18:01.687516+0000","last_became_peered":"2024-04-25T10:18:01.687516+0000","last_unstale":"2024-04-25T10:18:03.694980+0000","last_undegraded":"2024-04-25T10:18:03.694980+0000","last_fullsized":"2024-04-25T10:18:03.694980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694903+0000","last_change":"2024-04-25T10:18:02.687787+0000","last_active":"2024-04-25T10:18:03.694903+0000","last_peered":"2024-04-25T10:18:03.694903+0000","last_clean":"2024-04-25T10:18:03.694903+0000","last_became_active":"2024-04-25T10:18:02.687351+0000","last_became_peered":"2024-04-25T10:18:02.687351+0000","last_unstale":"2024-04-25T10:18:03.694903+0000","last_undegraded":"2024-04-25T10:18:03.694903+0000","last_fullsized":"2024-04-25T10:18:03.694903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T10:18:02.713027+0000","last_change":"2024-04-25T10:18:01.710038+0000","last_active":"2024-04-25T10:18:02.713027+0000","last_peered":"2024-04-25T10:18:02.713027+0000","last_clean":"2024-04-25T10:18:02.713027+0000","last_became_active":"2024-04-25T10:18:01.709789+0000","last_became_peered":"2024-04-25T10:18:01.709789+0000","last_unstale":"2024-04-25T10:18:02.713027+0000","last_undegraded":"2024-04-25T10:18:02.713027+0000","last_fullsized":"2024-04-25T10:18:02.713027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:49.219702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":660444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":71596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":806912,"data_stored":660422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":70040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":72268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:07.199 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:18:07.199 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:18:07.199 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:18:07.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:18:07.200 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:18:07.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.377+0000 7fd17cc00640 1 Processor -- start 2024-04-25T10:18:07.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.377+0000 7fd17cc00640 1 -- start start 2024-04-25T10:18:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd17cc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1780ce8a0 0x7fd17806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd17cc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1780d0950 con 0x7fd17815de60 2024-04-25T10:18:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd17cc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1780d0c30 con 0x7fd1780b2af0 2024-04-25T10:18:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd17cc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1780d0f10 con 0x7fd1780ce8a0 2024-04-25T10:18:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1780ce8a0 0x7fd17806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd1780ce8a0 0x7fd17806aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57094/0 (socket says 172.21.15.45:57094) 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 -- 172.21.15.45:0/2649367245 learned_addr learned my addr 172.21.15.45:0/2649367245 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd175e00640 1 --1- 172.21.15.45:0/2649367245 >> v1:172.21.15.45:6789/0 conn(0x7fd17815de60 0x7fd1780ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59148/0 (socket says 172.21.15.45:59148) 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 -- 172.21.15.45:0/2649367245 >> v1:172.21.15.117:6789/0 conn(0x7fd1780b2af0 legacy=0x7fd17806b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd175400640 1 -- 172.21.15.45:0/2649367245 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471748633 0 0) 0x7fd1780d0950 con 0x7fd17815de60 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 -- 172.21.15.45:0/2649367245 >> v1:172.21.15.45:6789/0 conn(0x7fd17815de60 legacy=0x7fd1780ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 -- 172.21.15.45:0/2649367245 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780d11f0 con 0x7fd1780ce8a0 2024-04-25T10:18:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.381+0000 7fd16fe00640 1 --2- 172.21.15.45:0/2649367245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1780ce8a0 0x7fd17806aea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd160009870 tx=0x7fd16001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5349a6c843a42b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd175400640 1 -- 172.21.15.45:0/2649367245 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600b1070 con 0x7fd1780ce8a0 2024-04-25T10:18:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd175400640 1 -- 172.21.15.45:0/2649367245 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1600ac600 con 0x7fd1780ce8a0 2024-04-25T10:18:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd175400640 1 -- 172.21.15.45:0/2649367245 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600ac9d0 con 0x7fd1780ce8a0 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- 172.21.15.45:0/2649367245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1780ce8a0 msgr2=0x7fd17806aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 --2- 172.21.15.45:0/2649367245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1780ce8a0 0x7fd17806aea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd160009870 tx=0x7fd16001c490 comp rx=0 tx=0).stop 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- 172.21.15.45:0/2649367245 shutdown_connections 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 --2- 172.21.15.45:0/2649367245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd1780ce8a0 0x7fd17806aea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- 172.21.15.45:0/2649367245 >> 172.21.15.45:0/2649367245 conn(0x7fd1780d1910 msgr2=0x7fd1780d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- 172.21.15.45:0/2649367245 shutdown_connections 2024-04-25T10:18:07.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- 172.21.15.45:0/2649367245 wait complete. 2024-04-25T10:18:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 Processor -- start 2024-04-25T10:18:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- start start 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1780b2af0 0x7fd1781542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1780ce8a0 0x7fd17815c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 0x7fd1780b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17816bff0 con 0x7fd17815de60 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17816be70 con 0x7fd1780ce8a0 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd17cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17816c170 con 0x7fd1780b2af0 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 0x7fd1780b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 0x7fd1780b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57100/0 (socket says 172.21.15.45:57100) 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 -- 172.21.15.45:0/413858808 learned_addr learned my addr 172.21.15.45:0/413858808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 -- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1780b2af0 msgr2=0x7fd1781542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1780b2af0 0x7fd1781542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 -- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1780ce8a0 msgr2=0x7fd17815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1780ce8a0 0x7fd17815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.385+0000 7fd176800640 1 -- 172.21.15.45:0/413858808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780c93e0 con 0x7fd17815de60 2024-04-25T10:18:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd176800640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 0x7fd1780b6480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b970 tx=0x7fd17000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170019070 con 0x7fd17815de60 2024-04-25T10:18:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17000cb40 con 0x7fd17815de60 2024-04-25T10:18:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170014440 con 0x7fd17815de60 2024-04-25T10:18:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1780c96c0 con 0x7fd17815de60 2024-04-25T10:18:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1780c9c80 con 0x7fd17815de60 2024-04-25T10:18:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17804ef30 con 0x7fd17815de60 2024-04-25T10:18:07.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd17001c070 con 0x7fd17815de60 2024-04-25T10:18:07.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd16e000640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd154066f60 0x7fd154069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.389+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd170084d60 con 0x7fd17815de60 2024-04-25T10:18:07.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.393+0000 7fd175e00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd154066f60 0x7fd154069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.393+0000 7fd175e00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd154066f60 0x7fd154069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd1680011f0 tx=0x7fd168007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.397+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd170055cd0 con 0x7fd17815de60 2024-04-25T10:18:07.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.593+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd1780cdf90 con 0x7fd17815de60 2024-04-25T10:18:07.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.597+0000 7fd16e000640 1 -- 172.21.15.45:0/413858808 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165739 (secure 0 0 0) 0x7fd1700555d0 con 0x7fd17815de60 2024-04-25T10:18:07.602 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:07.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd154066f60 msgr2=0x7fd154069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd154066f60 0x7fd154069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd1680011f0 tx=0x7fd168007930 comp rx=0 tx=0).stop 2024-04-25T10:18:07.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 msgr2=0x7fd1780b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 0x7fd1780b6480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b970 tx=0x7fd17000be30 comp rx=0 tx=0).stop 2024-04-25T10:18:07.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 shutdown_connections 2024-04-25T10:18:07.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd154066f60 0x7fd154069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd17815de60 0x7fd1780b6480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1780ce8a0 0x7fd17815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 --2- 172.21.15.45:0/413858808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1780b2af0 0x7fd1781542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 >> 172.21.15.45:0/413858808 conn(0x7fd1780d1910 msgr2=0x7fd1780b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 shutdown_connections 2024-04-25T10:18:07.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.601+0000 7fd17cc00640 1 -- 172.21.15.45:0/413858808 wait complete. 2024-04-25T10:18:07.624 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1944228396},{"type":"v1","addr":"172.21.15.117:6833","nonce":1944228396}]},"active_addr":"172.21.15.117:6833/1944228396","active_change":"2024-04-25T10:17:40.857425+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2814431615}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3309898826}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":4099328193}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":477879244}]}]} 2024-04-25T10:18:07.627 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:18:07.627 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:18:07.627 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 Processor -- start 2024-04-25T10:18:07.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 -- start start 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c0d0040 0x7f505c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f505c15b740 0x7f505c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c071050 con 0x7f505c1550b0 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15d040 con 0x7f505c15b740 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5064400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0712f0 con 0x7f505c0d0040 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5061a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f505c1550b0 0x7f505c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59160/0 (socket says 172.21.15.45:59160) 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5061a00640 1 -- 172.21.15.45:0/1194682692 learned_addr learned my addr 172.21.15.45:0/1194682692 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.765+0000 7f5062400640 1 --2- 172.21.15.45:0/1194682692 >> v2:172.21.15.45:3300/0 conn(0x7f505c15b740 0x7f505c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f5061000640 1 --2- 172.21.15.45:0/1194682692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c0d0040 0x7f505c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f505be00640 1 -- 172.21.15.45:0/1194682692 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377775917 0 0) 0x7f505c071050 con 0x7f505c1550b0 2024-04-25T10:18:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f505be00640 1 -- 172.21.15.45:0/1194682692 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5044005740 con 0x7f505c1550b0 2024-04-25T10:18:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f5062400640 1 -- 172.21.15.45:0/1194682692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c0d0040 msgr2=0x7f505c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f5062400640 1 --2- 172.21.15.45:0/1194682692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c0d0040 0x7f505c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f5062400640 1 -- 172.21.15.45:0/1194682692 >> v1:172.21.15.45:6789/0 conn(0x7f505c1550b0 legacy=0x7f505c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f5062400640 1 -- 172.21.15.45:0/1194682692 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0715d0 con 0x7f505c15b740 2024-04-25T10:18:07.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f5062400640 1 --2- 172.21.15.45:0/1194682692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c15b740 0x7f505c0b3260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5054009d90 tx=0x7f505401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594e4e77e2cb78e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f505be00640 1 -- 172.21.15.45:0/1194682692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540b2070 con 0x7f505c15b740 2024-04-25T10:18:07.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f505be00640 1 -- 172.21.15.45:0/1194682692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50540a5a00 con 0x7f505c15b740 2024-04-25T10:18:07.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.769+0000 7f505be00640 1 -- 172.21.15.45:0/1194682692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540a5db0 con 0x7f505c15b740 2024-04-25T10:18:07.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- 172.21.15.45:0/1194682692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c15b740 msgr2=0x7f505c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 --2- 172.21.15.45:0/1194682692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c15b740 0x7f505c0b3260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5054009d90 tx=0x7f505401c940 comp rx=0 tx=0).stop 2024-04-25T10:18:07.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- 172.21.15.45:0/1194682692 shutdown_connections 2024-04-25T10:18:07.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 --2- 172.21.15.45:0/1194682692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c15b740 0x7f505c0b3260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 --2- 172.21.15.45:0/1194682692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c0d0040 0x7f505c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- 172.21.15.45:0/1194682692 >> 172.21.15.45:0/1194682692 conn(0x7f505c066b70 msgr2=0x7f505c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- 172.21.15.45:0/1194682692 shutdown_connections 2024-04-25T10:18:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- 172.21.15.45:0/1194682692 wait complete. 2024-04-25T10:18:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 Processor -- start 2024-04-25T10:18:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- start start 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 0x7f505c154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c1550b0 0x7f505c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f505c15b740 0x7f505c0ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15bd70 con 0x7f505c0d0040 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15bbf0 con 0x7f505c15b740 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c15c030 con 0x7f505c1550b0 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 0x7f505c154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 0x7f505c154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57126/0 (socket says 172.21.15.45:57126) 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 -- 172.21.15.45:0/1784591421 learned_addr learned my addr 172.21.15.45:0/1784591421 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 -- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c1550b0 msgr2=0x7f505c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c1550b0 0x7f505c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 -- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f505c15b740 msgr2=0x7f505c0ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f505c15b740 0x7f505c0ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 -- 172.21.15.45:0/1784591421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c14f030 con 0x7f505c0d0040 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5061a00640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 0x7f505c154100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f505000c450 tx=0x7f505000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050017070 con 0x7f505c0d0040 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050005cf0 con 0x7f505c0d0040 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500072f0 con 0x7f505c0d0040 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.773+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c1546e0 con 0x7f505c0d0040 2024-04-25T10:18:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.777+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c14f5b0 con 0x7f505c0d0040 2024-04-25T10:18:07.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.777+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c04ef30 con 0x7f505c0d0040 2024-04-25T10:18:07.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.777+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f505001a070 con 0x7f505c0d0040 2024-04-25T10:18:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.777+0000 7f5059600640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f503c066f60 0x7f503c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.777+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5050084560 con 0x7f505c0d0040 2024-04-25T10:18:07.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.777+0000 7f5061000640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f503c066f60 0x7f503c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:07.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.781+0000 7f5061000640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f503c066f60 0x7f503c069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5048006aa0 tx=0x7f5048006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:07.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.781+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50500554d0 con 0x7f505c0d0040 2024-04-25T10:18:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.945+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f505c0cf000 con 0x7f505c0d0040 2024-04-25T10:18:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5059600640 1 -- 172.21.15.45:0/1784591421 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f5050054dd0 con 0x7f505c0d0040 2024-04-25T10:18:07.951 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:07.951 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f503c066f60 msgr2=0x7f503c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f503c066f60 0x7f503c069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5048006aa0 tx=0x7f5048006a30 comp rx=0 tx=0).stop 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 msgr2=0x7f505c154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 0x7f505c154100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f505000c450 tx=0x7f505000c910 comp rx=0 tx=0).stop 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 shutdown_connections 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f503c066f60 0x7f503c069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f505c15b740 0x7f505c0ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f505c1550b0 0x7f505c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 --2- 172.21.15.45:0/1784591421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f505c0d0040 0x7f505c154100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 >> 172.21.15.45:0/1784591421 conn(0x7f505c066b70 msgr2=0x7f505c0b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 shutdown_connections 2024-04-25T10:18:07.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:07.949+0000 7f5064400640 1 -- 172.21.15.45:0/1784591421 wait complete. 2024-04-25T10:18:07.972 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:18:07.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.109+0000 7f741ce00640 1 Processor -- start 2024-04-25T10:18:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.109+0000 7f741ce00640 1 -- start start 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f741ce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f741815f180 0x7f74180ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f741ce00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74180ce830 0x7f74180cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f741ce00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74180ae610 con 0x7f74180d0630 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f741ce00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74180d27b0 con 0x7f74180ce830 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f741ce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74180cf570 con 0x7f741815f180 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74180ce830 0x7f74180cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74180ce830 0x7f74180cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57140/0 (socket says 172.21.15.45:57140) 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 -- 172.21.15.45:0/3518876765 learned_addr learned my addr 172.21.15.45:0/3518876765 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7415e00640 1 --2- 172.21.15.45:0/3518876765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f741815f180 0x7f74180ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 -- 172.21.15.45:0/3518876765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f741815f180 msgr2=0x7f74180ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 --2- 172.21.15.45:0/3518876765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f741815f180 0x7f74180ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 -- 172.21.15.45:0/3518876765 >> v1:172.21.15.117:6789/0 conn(0x7f74180d0630 legacy=0x7f74180aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.113+0000 7f7416800640 1 -- 172.21.15.45:0/3518876765 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74180cf850 con 0x7f74180ce830 2024-04-25T10:18:08.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7416800640 1 --2- 172.21.15.45:0/3518876765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74180ce830 0x7f74180cd520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7410009e50 tx=0x7f741001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12573b6af09f4265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7414a00640 1 -- 172.21.15.45:0/3518876765 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74100a5700 con 0x7f74180ce830 2024-04-25T10:18:08.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7414a00640 1 -- 172.21.15.45:0/3518876765 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74100a58a0 con 0x7f74180ce830 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- 172.21.15.45:0/3518876765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74180ce830 msgr2=0x7f74180cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 --2- 172.21.15.45:0/3518876765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74180ce830 0x7f74180cd520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7410009e50 tx=0x7f741001ca30 comp rx=0 tx=0).stop 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- 172.21.15.45:0/3518876765 shutdown_connections 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 --2- 172.21.15.45:0/3518876765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74180ce830 0x7f74180cd520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 --2- 172.21.15.45:0/3518876765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f741815f180 0x7f74180ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- 172.21.15.45:0/3518876765 >> 172.21.15.45:0/3518876765 conn(0x7f74180d1900 msgr2=0x7f74180d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- 172.21.15.45:0/3518876765 shutdown_connections 2024-04-25T10:18:08.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- 172.21.15.45:0/3518876765 wait complete. 2024-04-25T10:18:08.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 Processor -- start 2024-04-25T10:18:08.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- start start 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74180ce830 0x7f741814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74180d0630 0x7f741806eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 0x7f741806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74180afe70 con 0x7f741815f180 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74180afcf0 con 0x7f74180d0630 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f741ce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74180afff0 con 0x7f74180ce830 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7416800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 0x7f741806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7416800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 0x7f741806f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57150/0 (socket says 172.21.15.45:57150) 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7416800640 1 -- 172.21.15.45:0/2681797536 learned_addr learned my addr 172.21.15.45:0/2681797536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7415e00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74180ce830 0x7f741814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.117+0000 7f7415400640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74180d0630 0x7f741806eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7416800640 1 -- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74180ce830 msgr2=0x7f741814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7416800640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74180ce830 0x7f741814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7416800640 1 -- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74180d0630 msgr2=0x7f741806eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7416800640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74180d0630 0x7f741806eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7416800640 1 -- 172.21.15.45:0/2681797536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741806baa0 con 0x7f741815f180 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7415e00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74180ce830 0x7f741814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f7416800640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 0x7f741806f090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74100af040 tx=0x7f74100197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74100a5700 con 0x7f741815f180 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74100be070 con 0x7f741815f180 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741806bd80 con 0x7f741815f180 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410009510 con 0x7f741815f180 2024-04-25T10:18:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741806c340 con 0x7f741815f180 2024-04-25T10:18:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741804ef30 con 0x7f741815f180 2024-04-25T10:18:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74100a58a0 con 0x7f741815f180 2024-04-25T10:18:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.121+0000 7f740e000640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73f8066f60 0x7f73f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.125+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74101203b0 con 0x7f741815f180 2024-04-25T10:18:08.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.125+0000 7f7415e00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73f8066f60 0x7f73f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.125+0000 7f7415e00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73f8066f60 0x7f73f8069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f740800b9f0 tx=0x7f7408008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.129+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7410016620 con 0x7f741815f180 2024-04-25T10:18:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.293+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74180cd840 con 0x7f741815f180 2024-04-25T10:18:08.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f740e000640 1 -- 172.21.15.45:0/2681797536 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f74100f1320 con 0x7f741815f180 2024-04-25T10:18:08.299 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:08.299 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73f8066f60 msgr2=0x7f73f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73f8066f60 0x7f73f8069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f740800b9f0 tx=0x7f7408008040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 msgr2=0x7f741806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 0x7f741806f090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74100af040 tx=0x7f74100197d0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 shutdown_connections 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73f8066f60 0x7f73f8069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f741815f180 0x7f741806f090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74180d0630 0x7f741806eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 --2- 172.21.15.45:0/2681797536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74180ce830 0x7f741814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 >> 172.21.15.45:0/2681797536 conn(0x7f74180d1900 msgr2=0x7f74180d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 shutdown_connections 2024-04-25T10:18:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.297+0000 7f741ce00640 1 -- 172.21.15.45:0/2681797536 wait complete. 2024-04-25T10:18:08.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:18:08.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:18:08.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:18:08.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:18:08.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:18:08.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:18:08.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:18:08.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:18:08.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.565+0000 7f4488a00640 1 Processor -- start 2024-04-25T10:18:08.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.565+0000 7f4488a00640 1 -- start start 2024-04-25T10:18:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.565+0000 7fc952200640 1 Processor -- start 2024-04-25T10:18:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.565+0000 7fc952200640 1 -- start start 2024-04-25T10:18:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.565+0000 7fc537e00640 1 Processor -- start 2024-04-25T10:18:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4488a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4488a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44840cd5a0 0x7f4484165b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4488a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840cb600 con 0x7f4484161d90 2024-04-25T10:18:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4488a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4484071c50 con 0x7f44840cd5a0 2024-04-25T10:18:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4488a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4484071690 con 0x7f4484162170 2024-04-25T10:18:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc952200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 0x7fc94404faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc952200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc944054400 con 0x7fc94404ffe0 2024-04-25T10:18:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc952200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440546e0 con 0x7fc94400aae0 2024-04-25T10:18:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc952200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440e6030 con 0x7fc944033d00 2024-04-25T10:18:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53724/0 (socket says 172.21.15.45:53724) 2024-04-25T10:18:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 -- 172.21.15.45:0/615448593 learned_addr learned my addr 172.21.15.45:0/615448593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc94be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 0x7fc94404faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc94be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 0x7fc94404faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53730/0 (socket says 172.21.15.45:53730) 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc94be00640 1 -- 172.21.15.45:0/3263329390 learned_addr learned my addr 172.21.15.45:0/3263329390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 -- 172.21.15.45:0/615448593 >> v2:172.21.15.45:3300/0 conn(0x7f44840cd5a0 msgr2=0x7f4484165b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 --2- 172.21.15.45:0/615448593 >> v2:172.21.15.45:3300/0 conn(0x7f44840cd5a0 0x7f4484165b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 -- 172.21.15.45:0/615448593 >> v1:172.21.15.45:6789/0 conn(0x7f4484161d90 legacy=0x7f448406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7f4482a00640 1 -- 172.21.15.45:0/615448593 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484071970 con 0x7f4484162170 2024-04-25T10:18:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc94be00640 1 -- 172.21.15.45:0/3263329390 >> v1:172.21.15.117:6789/0 conn(0x7fc94400aae0 legacy=0x7fc9440df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc94be00640 1 -- 172.21.15.45:0/3263329390 >> v1:172.21.15.45:6789/0 conn(0x7fc94404ffe0 legacy=0x7fc944006540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc94be00640 1 -- 172.21.15.45:0/3263329390 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944006020 con 0x7fc944033d00 2024-04-25T10:18:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc537e00640 1 -- start start 2024-04-25T10:18:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc537e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53815a330 0x7fc5380cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc537e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5380ccf50 0x7fc5380cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc536a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53815a330 0x7fc5380cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc536a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53815a330 0x7fc5380cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53732/0 (socket says 172.21.15.45:53732) 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc536a00640 1 -- 172.21.15.45:0/154854169 learned_addr learned my addr 172.21.15.45:0/154854169 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.569+0000 7fc537400640 1 --2- 172.21.15.45:0/154854169 >> v2:172.21.15.117:3300/0 conn(0x7fc5380ccf50 0x7fc5380cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53815e0f0 con 0x7fc5380cf820 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53815e3d0 con 0x7fc5380ccf50 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53815e6b0 con 0x7fc53815a330 2024-04-25T10:18:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 Processor -- start 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 -- start start 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537400640 1 -- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53815a330 msgr2=0x7fc5380cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537400640 1 --2- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53815a330 0x7fc5380cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537400640 1 -- 172.21.15.45:0/154854169 >> v1:172.21.15.45:6789/0 conn(0x7fc5380cf820 legacy=0x7fc5380cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc537400640 1 -- 172.21.15.45:0/154854169 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53815e990 con 0x7fc5380ccf50 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc535600640 1 -- 172.21.15.45:0/154854169 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285999508 0 0) 0x7fc53815e0f0 con 0x7fc5380cf820 2024-04-25T10:18:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f4482a00640 1 --2- 172.21.15.45:0/615448593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484071000 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4470009870 tx=0x7f447001c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31ec0f9c3a71f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7fc94be00640 1 --2- 172.21.15.45:0/3263329390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 0x7fc94404faa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc9380098a0 tx=0x7fc93801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c5d67249ec93573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c06ca50 0x7f810c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f810c070310 0x7f810c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810c0b4470 con 0x7f810c0cf930 2024-04-25T10:18:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810c0cf4d0 con 0x7f810c070310 2024-04-25T10:18:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f8111e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0b4cd0 con 0x7f810c06ca50 2024-04-25T10:18:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f810aa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f810c0cf930 0x7f810c0cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59196/0 (socket says 172.21.15.45:59196) 2024-04-25T10:18:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.573+0000 7f810aa00640 1 -- 172.21.15.45:0/2148460644 learned_addr learned my addr 172.21.15.45:0/2148460644 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f810b400640 1 --2- 172.21.15.45:0/2148460644 >> v2:172.21.15.117:3300/0 conn(0x7f810c070310 0x7f810c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc94b400640 1 -- 172.21.15.45:0/3263329390 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b0070 con 0x7fc944033d00 2024-04-25T10:18:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc94b400640 1 -- 172.21.15.45:0/3263329390 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380ab600 con 0x7fc944033d00 2024-04-25T10:18:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc94b400640 1 -- 172.21.15.45:0/3263329390 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380ab9d0 con 0x7fc944033d00 2024-04-25T10:18:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 Processor -- start 2024-04-25T10:18:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 -- 172.21.15.45:0/3263329390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 msgr2=0x7fc94404faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 --2- 172.21.15.45:0/3263329390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 0x7fc94404faa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc9380098a0 tx=0x7fc93801c470 comp rx=0 tx=0).stop 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 -- 172.21.15.45:0/3263329390 shutdown_connections 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 --2- 172.21.15.45:0/3263329390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc944033d00 0x7fc94404faa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 -- 172.21.15.45:0/3263329390 >> 172.21.15.45:0/3263329390 conn(0x7fc944009fc0 msgr2=0x7fc94400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f8109600640 1 -- 172.21.15.45:0/2148460644 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852164946 0 0) 0x7f810c0b4470 con 0x7f810c0cf930 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f8109600640 1 -- 172.21.15.45:0/2148460644 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f0003530 con 0x7f810c0cf930 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f8109600640 1 -- 172.21.15.45:0/2148460644 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969774520 0 0) 0x7f80f0003530 con 0x7f810c0cf930 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f810b400640 1 -- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c06ca50 msgr2=0x7f810c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f810b400640 1 --2- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c06ca50 0x7f810c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f810b400640 1 -- 172.21.15.45:0/2148460644 >> v1:172.21.15.45:6789/0 conn(0x7f810c0cf930 legacy=0x7f810c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f810b400640 1 -- 172.21.15.45:0/2148460644 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0b4710 con 0x7f810c070310 2024-04-25T10:18:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f1357200640 1 Processor -- start 2024-04-25T10:18:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc537400640 1 --2- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5380ccf50 0x7fc5380cdd30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc530009d90 tx=0x7fc53001c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab16a3620a511e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4482000640 1 -- 172.21.15.45:0/615448593 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700b0070 con 0x7f4484162170 2024-04-25T10:18:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4482000640 1 -- 172.21.15.45:0/615448593 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700ab600 con 0x7f4484162170 2024-04-25T10:18:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4482000640 1 -- 172.21.15.45:0/615448593 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700ab9d0 con 0x7f4484162170 2024-04-25T10:18:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 -- 172.21.15.45:0/3263329390 shutdown_connections 2024-04-25T10:18:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fc952200640 1 -- 172.21.15.45:0/3263329390 wait complete. 2024-04-25T10:18:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 -- start start 2024-04-25T10:18:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4488a00640 1 -- 172.21.15.45:0/615448593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 msgr2=0x7f4484071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4488a00640 1 --2- 172.21.15.45:0/615448593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484071000 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4470009870 tx=0x7f447001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3180b48b0 0x7fd3180cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4488a00640 1 -- 172.21.15.45:0/615448593 shutdown_connections 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4488a00640 1 --2- 172.21.15.45:0/615448593 >> v2:172.21.15.45:3300/0 conn(0x7f44840cd5a0 0x7f4484165b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4488a00640 1 --2- 172.21.15.45:0/615448593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484071000 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f4488a00640 1 -- 172.21.15.45:0/615448593 >> 172.21.15.45:0/615448593 conn(0x7f44840b16a0 msgr2=0x7f44840b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f4488a00640 1 -- 172.21.15.45:0/615448593 shutdown_connections 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f4488a00640 1 -- 172.21.15.45:0/615448593 wait complete. 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806c4b0 0x7fd3180d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd318165b90 con 0x7fd31806d270 2024-04-25T10:18:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd31806d650 con 0x7fd3180b48b0 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7fd31e200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd318165e70 con 0x7fd31806c4b0 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.577+0000 7f1357200640 1 -- start start 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1357200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f135815bd40 0x7f13581601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1357200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13580d00f0 0x7f135815f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1357200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13580cd560 con 0x7f135815b960 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1357200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13580cf330 con 0x7f13580d00f0 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1357200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13580cf610 con 0x7f135815bd40 2024-04-25T10:18:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3180b48b0 0x7fd3180cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3180b48b0 0x7fd3180cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51692/0 (socket says 172.21.15.45:51692) 2024-04-25T10:18:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 -- 172.21.15.45:0/59630330 learned_addr learned my addr 172.21.15.45:0/59630330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 Processor -- start 2024-04-25T10:18:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 -- start start 2024-04-25T10:18:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f134fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f135815bd40 0x7f13581601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f134fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f135815bd40 0x7f13581601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51698/0 (socket says 172.21.15.45:51698) 2024-04-25T10:18:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f134fe00640 1 -- 172.21.15.45:0/3359571964 learned_addr learned my addr 172.21.15.45:0/3359571964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fb0efe00640 1 Processor -- start 2024-04-25T10:18:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 -- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806c4b0 msgr2=0x7fd3180d0030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:18:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 --2- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806c4b0 0x7fd3180d0030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 -- 172.21.15.45:0/59630330 >> v1:172.21.15.45:6789/0 conn(0x7fd31806d270 legacy=0x7fd3180b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd317e00640 1 -- 172.21.15.45:0/59630330 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232444126 0 0) 0x7fd318165b90 con 0x7fd31806d270 2024-04-25T10:18:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fd30fe00640 1 -- 172.21.15.45:0/59630330 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318166150 con 0x7fd3180b48b0 2024-04-25T10:18:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fb0efe00640 1 -- start start 2024-04-25T10:18:08.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1356800640 1 --2- 172.21.15.45:0/3359571964 >> v2:172.21.15.45:3300/0 conn(0x7f13580d00f0 0x7f135815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc535600640 1 -- 172.21.15.45:0/154854169 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5300ac070 con 0x7fc5380ccf50 2024-04-25T10:18:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc535600640 1 -- 172.21.15.45:0/154854169 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5300a6620 con 0x7fc5380ccf50 2024-04-25T10:18:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc535600640 1 -- 172.21.15.45:0/154854169 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5300a69f0 con 0x7fc5380ccf50 2024-04-25T10:18:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f810b400640 1 --2- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c070310 0x7f810c0707a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8104009dc0 tx=0x7f810401c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3547a7311170361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f4488a00640 1 Processor -- start 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc94400aae0 0x7fc9440d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc944033d00 0x7fc9440e05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94404ffe0 0x7fc944039380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440075e0 con 0x7fc944033d00 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944007460 con 0x7fc94404ffe0 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc952200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc944007760 con 0x7fc94400aae0 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f8109600640 1 -- 172.21.15.45:0/2148460644 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81040ac070 con 0x7f810c070310 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f8109600640 1 -- 172.21.15.45:0/2148460644 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81040a6430 con 0x7f810c070310 2024-04-25T10:18:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f8109600640 1 -- 172.21.15.45:0/2148460644 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81040ac070 con 0x7f810c070310 2024-04-25T10:18:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc94400aae0 0x7fc9440d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc94400aae0 0x7fc9440d39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53766/0 (socket says 172.21.15.45:53766) 2024-04-25T10:18:08.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f4488a00640 1 -- start start 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 -- 172.21.15.45:0/896570209 learned_addr learned my addr 172.21.15.45:0/896570209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc951800640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94404ffe0 0x7fc944039380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 -- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94404ffe0 msgr2=0x7fc944039380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94404ffe0 0x7fc944039380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 -- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc944033d00 msgr2=0x7fc9440e05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc944033d00 0x7fc9440e05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc950e00640 1 -- 172.21.15.45:0/896570209 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9440e1050 con 0x7fc94400aae0 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5380ccf50 msgr2=0x7fc5380cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7fc537e00640 1 --2- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5380ccf50 0x7fc5380cdd30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc530009d90 tx=0x7fc53001c900 comp rx=0 tx=0).stop 2024-04-25T10:18:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f134fe00640 1 -- 172.21.15.45:0/3359571964 >> v2:172.21.15.45:3300/0 conn(0x7f13580d00f0 msgr2=0x7f135815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f134fe00640 1 --2- 172.21.15.45:0/3359571964 >> v2:172.21.15.45:3300/0 conn(0x7f13580d00f0 0x7f135815f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.581+0000 7f1355400640 1 -- 172.21.15.45:0/3359571964 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383839794 0 0) 0x7f13580cd560 con 0x7f135815b960 2024-04-25T10:18:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f134fe00640 1 -- 172.21.15.45:0/3359571964 >> v1:172.21.15.117:6789/0 conn(0x7f135815b960 legacy=0x7f13580ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f134fe00640 1 -- 172.21.15.45:0/3359571964 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580cf8f0 con 0x7f135815bd40 2024-04-25T10:18:08.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc950e00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc94400aae0 0x7fc9440d39e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b3e0 tx=0x7fc94000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 -- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c070310 msgr2=0x7f810c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 shutdown_connections 2024-04-25T10:18:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc537e00640 1 --2- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5380ccf50 0x7fc5380cdd30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc537e00640 1 --2- 172.21.15.45:0/154854169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc53815a330 0x7fc5380cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 >> 172.21.15.45:0/154854169 conn(0x7fc5380b1a20 msgr2=0x7fc5380b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 --2- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c070310 0x7f810c0707a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8104009dc0 tx=0x7f810401c7e0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 -- 172.21.15.45:0/2148460644 shutdown_connections 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 --2- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c070310 0x7f810c0707a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 --2- 172.21.15.45:0/2148460644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c06ca50 0x7f810c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 -- 172.21.15.45:0/2148460644 >> 172.21.15.45:0/2148460644 conn(0x7f810c066b70 msgr2=0x7f810c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940015020 con 0x7fc94400aae0 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940011850 con 0x7fc94400aae0 2024-04-25T10:18:08.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440e1330 con 0x7fc94400aae0 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9440e18f0 con 0x7fc94400aae0 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 -- 172.21.15.45:0/2148460644 shutdown_connections 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4488a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840cd5a0 0x7f44841522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4488a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484161d90 0x7f4484071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4488a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4488a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840ccc80 con 0x7f4484161d90 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4488a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840ccb00 con 0x7f44840cd5a0 2024-04-25T10:18:08.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4488a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840cce00 con 0x7f4484162170 2024-04-25T10:18:08.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940011c40 con 0x7fc94400aae0 2024-04-25T10:18:08.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4482a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484161d90 0x7f4484071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f8111e00640 1 -- 172.21.15.45:0/2148460644 wait complete. 2024-04-25T10:18:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00b5900 0x7fb0f00b8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0efe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f015deb0 con 0x7fb0f015dad0 2024-04-25T10:18:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0efe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00b5d90 con 0x7fb0f00b5470 2024-04-25T10:18:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0efe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b9940 con 0x7fb0f00b5900 2024-04-25T10:18:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 shutdown_connections 2024-04-25T10:18:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7feea2800640 1 Processor -- start 2024-04-25T10:18:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fc537e00640 1 -- 172.21.15.45:0/154854169 wait complete. 2024-04-25T10:18:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4482a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484161d90 0x7f4484071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57194/0 (socket says 172.21.15.45:57194) 2024-04-25T10:18:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4482a00640 1 -- 172.21.15.45:0/4025700633 learned_addr learned my addr 172.21.15.45:0/4025700633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4483400640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840cd5a0 0x7f44841522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7f4483e00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fd30fe00640 1 --2- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd3180cfaf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd300009870 tx=0x7fd30001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd9624d4a2784e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fd317e00640 1 -- 172.21.15.45:0/59630330 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000a7070 con 0x7fd3180b48b0 2024-04-25T10:18:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fd317e00640 1 -- 172.21.15.45:0/59630330 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000b0070 con 0x7fd3180b48b0 2024-04-25T10:18:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fd317e00640 1 -- 172.21.15.45:0/59630330 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000ab890 con 0x7fd3180b48b0 2024-04-25T10:18:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fd31e200640 1 -- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 msgr2=0x7fd3180cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fd31e200640 1 --2- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd3180cfaf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd300009870 tx=0x7fd30001c540 comp rx=0 tx=0).stop 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0e7e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb0f00b5470 0x7fb0f00b7d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:60960/0 (socket says 172.21.15.45:60960) 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0e7e00640 1 -- 172.21.15.45:0/1308417313 learned_addr learned my addr 172.21.15.45:0/1308417313 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0ef400640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00b5900 0x7fb0f00b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673629588 0 0) 0x7fb0f00b5d90 con 0x7fb0f00b5470 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d4003530 con 0x7fb0f00b5470 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982607894 0 0) 0x7fb0f015deb0 con 0x7fb0f015dad0 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f00b5d90 con 0x7fb0f015dad0 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2681714518 0 0) 0x7fb0d4003530 con 0x7fb0f00b5470 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00b5900 msgr2=0x7fb0f00b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00b5900 0x7fb0f00b8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 >> v1:172.21.15.45:6789/0 conn(0x7fb0f015dad0 legacy=0x7fb0f00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00d1cf0 con 0x7fb0f00b5470 2024-04-25T10:18:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fb0d8001680 con 0x7fb0f00b5470 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9440053a0 con 0x7fc94400aae0 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 >> v1:172.21.15.117:6789/0 conn(0x7fb0f00b5470 legacy=0x7fb0f00b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d4003bf0 0x7fb0d4003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d40052f0 0x7fb0d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b9940 con 0x7fb0d40052f0 2024-04-25T10:18:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c120 con 0x7fb0d4004810 2024-04-25T10:18:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d400c320 con 0x7fb0d4003bf0 2024-04-25T10:18:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 Processor -- start 2024-04-25T10:18:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.585+0000 7feea2800640 1 -- start start 2024-04-25T10:18:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4482a00640 1 -- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 msgr2=0x7f4484072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4482a00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4484162170 0x7f4484072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 -- start start 2024-04-25T10:18:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fc537e00640 1 Processor -- start 2024-04-25T10:18:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f134fe00640 1 --2- 172.21.15.45:0/3359571964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13581601e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13580b76b0 tx=0x7f134001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf521d8cf2df16e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4482a00640 1 -- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840cd5a0 msgr2=0x7f44841522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4482a00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840cd5a0 0x7f44841522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4482a00640 1 -- 172.21.15.45:0/4025700633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840afd10 con 0x7f4484161d90 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4483400640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44840cd5a0 0x7f44841522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7feea2800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c0cf5c0 0x7fee9c06c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7feea2800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fee9c06cc50 0x7fee9c0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7feea2800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9c0ca1c0 con 0x7fee9c162090 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7feea2800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c0ca4a0 con 0x7fee9c06cc50 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7feea2800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0ca780 con 0x7fee9c0cf5c0 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fd31e200640 1 -- 172.21.15.45:0/59630330 shutdown_connections 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fd31e200640 1 --2- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806c4b0 0x7fd3180d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fd31e200640 1 --2- 172.21.15.45:0/59630330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd3180cfaf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fd31e200640 1 -- 172.21.15.45:0/59630330 >> 172.21.15.45:0/59630330 conn(0x7fd3180669a0 msgr2=0x7fd318066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0e7e00640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fc537e00640 1 -- start start 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ef400640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d40052f0 0x7fb0d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0eea00640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d4003bf0 0x7fb0d4003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f4482a00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484161d90 0x7f4484071df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44700ae040 tx=0x7f447001ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1355400640 1 -- 172.21.15.45:0/3359571964 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400aa670 con 0x7f135815bd40 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1355400640 1 -- 172.21.15.45:0/3359571964 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13400af070 con 0x7f135815bd40 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1355400640 1 -- 172.21.15.45:0/3359571964 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400aaa30 con 0x7f135815bd40 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9b400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fee9c162090 0x7fee9c0ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:60976/0 (socket says 172.21.15.45:60976) 2024-04-25T10:18:08.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc940018020 con 0x7fc94400aae0 2024-04-25T10:18:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ef400640 1 -- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d4003bf0 msgr2=0x7fb0d4003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ef400640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d4003bf0 0x7fb0d4003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ef400640 1 -- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d4004810 msgr2=0x7fb0d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ef400640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d4004810 0x7fb0d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0ef400640 1 -- 172.21.15.45:0/1308417313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00d1cf0 con 0x7fb0d40052f0 2024-04-25T10:18:08.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c06ca50 0x7f810c14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c070310 0x7f810c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f810c0cf930 0x7f810c163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fb0eea00640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d4003bf0 0x7fb0d4003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0b2e30 con 0x7f810c0cf930 2024-04-25T10:18:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c072050 con 0x7f810c06ca50 2024-04-25T10:18:08.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f8111e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c0b2fb0 con 0x7f810c070310 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9b400640 1 -- 172.21.15.45:0/371862305 learned_addr learned my addr 172.21.15.45:0/371862305 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9be00640 1 --2- 172.21.15.45:0/371862305 >> v2:172.21.15.117:3300/0 conn(0x7fee9c06cc50 0x7fee9c0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9aa00640 1 --2- 172.21.15.45:0/371862305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c0cf5c0 0x7fee9c06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c070310 0x7f810c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c070310 0x7f810c15c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53804/0 (socket says 172.21.15.45:53804) 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 -- 172.21.15.45:0/1763749884 learned_addr learned my addr 172.21.15.45:0/1763749884 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810aa00640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c06ca50 0x7f810c14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1357200640 1 -- 172.21.15.45:0/3359571964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 msgr2=0x7f13581601e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1357200640 1 --2- 172.21.15.45:0/3359571964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13581601e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13580b76b0 tx=0x7f134001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810b400640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f810c0cf930 0x7f810c163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1357200640 1 -- 172.21.15.45:0/3359571964 shutdown_connections 2024-04-25T10:18:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9a000640 1 -- 172.21.15.45:0/371862305 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958010877 0 0) 0x7fee9c0ca1c0 con 0x7fee9c162090 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1357200640 1 --2- 172.21.15.45:0/3359571964 >> v2:172.21.15.45:3300/0 conn(0x7f13580d00f0 0x7f135815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1357200640 1 --2- 172.21.15.45:0/3359571964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13581601e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9a000640 1 -- 172.21.15.45:0/371862305 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee80003530 con 0x7fee9c162090 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 -- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c06ca50 msgr2=0x7f810c14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f810c06ca50 0x7f810c14ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 -- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f810c0cf930 msgr2=0x7f810c163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f810c0cf930 0x7f810c163050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 -- 172.21.15.45:0/1763749884 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c15b360 con 0x7f810c070310 2024-04-25T10:18:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f1357200640 1 -- 172.21.15.45:0/3359571964 >> 172.21.15.45:0/3359571964 conn(0x7f13580b32b0 msgr2=0x7f13580b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9aa00640 1 -- 172.21.15.45:0/371862305 >> v2:172.21.15.117:3300/0 conn(0x7fee9c06cc50 msgr2=0x7fee9c0cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700ba020 con 0x7f4484161d90 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9aa00640 1 --2- 172.21.15.45:0/371862305 >> v2:172.21.15.117:3300/0 conn(0x7fee9c06cc50 0x7fee9c0cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7fee9aa00640 1 -- 172.21.15.45:0/371862305 >> v1:172.21.15.117:6789/0 conn(0x7fee9c162090 legacy=0x7fee9c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700b6640 con 0x7f4484161d90 2024-04-25T10:18:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700b0040 con 0x7f4484161d90 2024-04-25T10:18:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.589+0000 7f810a000640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c070310 0x7f810c15c400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80f800b580 tx=0x7f80f800ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fee9aa00640 1 -- 172.21.15.45:0/371862305 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0caa60 con 0x7fee9c0cf5c0 2024-04-25T10:18:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8014070 con 0x7f810c070310 2024-04-25T10:18:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f8010890 con 0x7f810c070310 2024-04-25T10:18:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8010c80 con 0x7f810c070310 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5380ccf50 0x7fc53809d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5380cf820 0x7fc538072510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5381525f0 0x7fc53815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c15b640 con 0x7f810c070310 2024-04-25T10:18:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c15bc00 con 0x7f810c070310 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4484152820 con 0x7f4484161d90 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484154360 con 0x7f4484161d90 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53806d650 con 0x7fc5380cf820 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53806d4d0 con 0x7fc5381525f0 2024-04-25T10:18:08.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5381671c0 con 0x7fc5380ccf50 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5380ccf50 0x7fc53809d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5380ccf50 0x7fc53809d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53814/0 (socket says 172.21.15.45:53814) 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 -- 172.21.15.45:0/1609063290 learned_addr learned my addr 172.21.15.45:0/1609063290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536000640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5380cf820 0x7fc538072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc537400640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5381525f0 0x7fc53815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 -- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5381525f0 msgr2=0x7fc53815a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5381525f0 0x7fc53815a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 -- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5380cf820 msgr2=0x7fc538072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5380cf820 0x7fc538072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 -- 172.21.15.45:0/1609063290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5380ad890 con 0x7fc5380ccf50 2024-04-25T10:18:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4484075120 con 0x7f4484161d90 2024-04-25T10:18:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc948c00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc92c066fb0 0x7fc92c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc940084f10 con 0x7fc94400aae0 2024-04-25T10:18:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f810c15bfe0 con 0x7f810c070310 2024-04-25T10:18:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fd31e200640 1 -- 172.21.15.45:0/59630330 shutdown_connections 2024-04-25T10:18:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fd31e200640 1 -- 172.21.15.45:0/59630330 wait complete. 2024-04-25T10:18:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f1357200640 1 -- 172.21.15.45:0/3359571964 shutdown_connections 2024-04-25T10:18:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc948c00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7fc92c06f1b0 0x7fc92c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f1357200640 1 -- 172.21.15.45:0/3359571964 wait complete. 2024-04-25T10:18:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 --> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc940004460 con 0x7fc92c06f1b0 2024-04-25T10:18:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc940087530 con 0x7fc94400aae0 2024-04-25T10:18:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc951800640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7fc92c06f1b0 0x7fc92c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80f8017070 con 0x7f810c070310 2024-04-25T10:18:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc94be00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc92c066fb0 0x7fc92c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fee9be00640 1 --2- 172.21.15.45:0/371862305 >> v2:172.21.15.117:3300/0 conn(0x7fee9c06cc50 0x7fee9c0cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44700bd020 con 0x7f4484161d90 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.593+0000 7fc536a00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5380ccf50 0x7fc53809d110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e890 tx=0x7fc52800ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528017070 con 0x7fc5380ccf50 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52800c940 con 0x7fc5380ccf50 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800ccf0 con 0x7fc5380ccf50 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5380adb50 con 0x7fc5380ccf50 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc94be00640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc92c066fb0 0x7fc92c069470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ad000 tx=0x7fc938008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc951800640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7fc92c06f1b0 0x7fc92c0715d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f8103400640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f80ec066f60 0x7f80ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80f80850f0 con 0x7f810c070310 2024-04-25T10:18:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5380ae110 con 0x7fc5380ccf50 2024-04-25T10:18:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 Processor -- start 2024-04-25T10:18:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f1357200640 1 Processor -- start 2024-04-25T10:18:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fee9aa00640 1 --2- 172.21.15.45:0/371862305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c0cf5c0 0x7fee9c06c710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee880097c0 tx=0x7fee8801c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c735c7d81adb615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f8103400640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f80ec06f100 0x7f80ec071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 --> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80f8004460 con 0x7f80ec06f100 2024-04-25T10:18:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80f80854a0 con 0x7f810c070310 2024-04-25T10:18:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f810aa00640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f80ec066f60 0x7f80ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc538075120 con 0x7fc5380ccf50 2024-04-25T10:18:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 -- start start 2024-04-25T10:18:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f810b400640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f80ec06f100 0x7f80ec071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fee9a000640 1 -- 172.21.15.45:0/371862305 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880b0070 con 0x7fee9c0cf5c0 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f447b400640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4464066f60 0x7f4464069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f447011f370 con 0x7f4484161d90 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f447b400640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7f446406f100 0x7f4464071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 --> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44700b3e50 con 0x7f446406f100 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4470126050 con 0x7f4484161d90 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f4483e00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7f446406f100 0x7f4464071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f4483400640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4464066f60 0x7f4464069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f4483e00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7f446406f100 0x7f4464071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f4483400640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4464066f60 0x7f4464069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f447c00a7f0 tx=0x7f447c00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== osd.1 v2:172.21.15.45:6816/3290058656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc940004460 con 0x7fc92c06f1b0 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f810b400640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f80ec06f100 0x7f80ec071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f810aa00640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f80ec066f60 0x7f80ec069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f80fc001650 tx=0x7f80fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== osd.3 v2:172.21.15.45:6800/3296322295 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f80f8004460 con 0x7f80ec06f100 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31806c4b0 0x7fd318155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806d270 0x7fd318160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd31815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318163170 con 0x7fd31806c4b0 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd318162ff0 con 0x7fd3180b48b0 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fd31e200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3181632f0 con 0x7fd31806d270 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd31d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd31815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd31d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd31815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51748/0 (socket says 172.21.15.45:51748) 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd31d800640 1 -- 172.21.15.45:0/335421786 learned_addr learned my addr 172.21.15.45:0/335421786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806d270 0x7fd318160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd31ce00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31806c4b0 0x7fd318155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 -- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 msgr2=0x7fd31815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3180b48b0 0x7fd31815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 -- 172.21.15.45:0/335421786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31806c4b0 msgr2=0x7fd318155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31806c4b0 0x7fd318155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 -- 172.21.15.45:0/335421786 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3181562c0 con 0x7fd31806d270 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7f1357200640 1 -- start start 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1357200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13580d00f0 0x7f135814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1357200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f135815b960 0x7f13580c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1357200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13580c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1357200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580b4fd0 con 0x7f13580d00f0 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1357200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1358168980 con 0x7f135815bd40 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1357200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135814ae00 con 0x7f135815b960 2024-04-25T10:18:08.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fb0ef400640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d40052f0 0x7fb0d400b9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00b5ce0 tx=0x7fb0e801cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6b882a33c6efd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fee9a000640 1 -- 172.21.15.45:0/371862305 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee880ab400 con 0x7fee9c0cf5c0 2024-04-25T10:18:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.597+0000 7fee9a000640 1 -- 172.21.15.45:0/371862305 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c001680 con 0x7fee9c0cf5c0 2024-04-25T10:18:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 -- 172.21.15.45:0/371862305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c0cf5c0 msgr2=0x7fee9c06c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 --2- 172.21.15.45:0/371862305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c0cf5c0 0x7fee9c06c710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee880097c0 tx=0x7fee8801c240 comp rx=0 tx=0).stop 2024-04-25T10:18:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 -- 172.21.15.45:0/371862305 shutdown_connections 2024-04-25T10:18:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 --2- 172.21.15.45:0/371862305 >> v2:172.21.15.117:3300/0 conn(0x7fee9c06cc50 0x7fee9c0cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 --2- 172.21.15.45:0/371862305 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c0cf5c0 0x7fee9c06c710 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee880097c0 tx=0x7fee8801c240 comp rx=0 tx=0).stop 2024-04-25T10:18:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 -- 172.21.15.45:0/371862305 >> 172.21.15.45:0/371862305 conn(0x7fee9c0cfb70 msgr2=0x7fee9c0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd30fe00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806d270 0x7fd318160670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3000ac000 tx=0x7fd30001ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 -- 172.21.15.45:0/371862305 shutdown_connections 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7feea2800640 1 -- 172.21.15.45:0/371862305 wait complete. 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== osd.0 v2:172.21.15.45:6808/3877436580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f44700b3e50 con 0x7f446406f100 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b4800 con 0x7fd31806d270 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31815f5d0 con 0x7fd31806d270 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31815fb90 con 0x7fd31806d270 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3000abe40 con 0x7fd31806d270 2024-04-25T10:18:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b4800 con 0x7fd31806d270 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f134fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f135815b960 0x7f13580c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f134fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f135815b960 0x7f13580c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53832/0 (socket says 172.21.15.45:53832) 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f134fe00640 1 -- 172.21.15.45:0/1684507502 learned_addr learned my addr 172.21.15.45:0/1684507502 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1355e00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13580d00f0 0x7f135814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13580c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 -- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f135815b960 msgr2=0x7f13580c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f135815b960 0x7f13580c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 -- 172.21.15.45:0/1684507502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13580d00f0 msgr2=0x7f135814a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13580d00f0 0x7f135814a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 -- 172.21.15.45:0/1684507502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135814b7f0 con 0x7f135815bd40 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f134fe00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f135815b960 0x7f13580c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:08.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1355e00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13580d00f0 0x7f135814a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7f1356800640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13580c58a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f135000df50 tx=0x7f135000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80b2070 con 0x7fb0d40052f0 2024-04-25T10:18:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e80a5d60 con 0x7fb0d40052f0 2024-04-25T10:18:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fb0ee000640 1 -- 172.21.15.45:0/1308417313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8007a90 con 0x7fb0d40052f0 2024-04-25T10:18:08.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.601+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc52801a070 con 0x7fc5380ccf50 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0efe00640 1 -- 172.21.15.45:0/1308417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d40052f0 msgr2=0x7fb0d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0efe00640 1 --2- 172.21.15.45:0/1308417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d40052f0 0x7fb0d400b9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00b5ce0 tx=0x7fb0e801cd50 comp rx=0 tx=0).stop 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0ef400640 1 -- 172.21.15.45:0/1308417313 reap_dead start 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0efe00640 1 -- 172.21.15.45:0/1308417313 shutdown_connections 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0efe00640 1 -- 172.21.15.45:0/1308417313 >> 172.21.15.45:0/1308417313 conn(0x7fb0f006aec0 msgr2=0x7fb0f006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0efe00640 1 -- 172.21.15.45:0/1308417313 shutdown_connections 2024-04-25T10:18:08.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7feea2800640 1 Processor -- start 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc52ea00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc518066fb0 0x7fc518069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc528084df0 con 0x7fc5380ccf50 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc536000640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc518066fb0 0x7fc518069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc52ea00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7fc51806f230 0x7fc518071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 --> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc528008eb0 con 0x7fc51806f230 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc528087540 con 0x7fc5380ccf50 2024-04-25T10:18:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fc536000640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc518066fb0 0x7fc518069470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc538156400 tx=0x7fc524009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd2f0001040 con 0x7fd31806d270 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135002b400 con 0x7f135815bd40 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135002b5c0 con 0x7f135815bd40 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135002b9b0 con 0x7f135815bd40 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135814bad0 con 0x7f135815bd40 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135814c090 con 0x7f135815bd40 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1358075120 con 0x7f135815bd40 2024-04-25T10:18:08.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f135002bb50 con 0x7f135815bd40 2024-04-25T10:18:08.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fb0efe00640 1 -- 172.21.15.45:0/1308417313 wait complete. 2024-04-25T10:18:08.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fb0efe00640 1 Processor -- start 2024-04-25T10:18:08.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 -- start start 2024-04-25T10:18:08.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fb0efe00640 1 -- start start 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.605+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3000b49a0 con 0x7fd31806d270 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd315600640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2f4067360 0x7fd2f4069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3000b4600 con 0x7fd31806d270 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd315600640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7fd2f406f500 0x7fd2f4071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 --> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3000b7ea0 con 0x7fd2f406f500 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd300125050 con 0x7fd31806d270 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd31d800640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7fd2f406f500 0x7fd2f4071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd31ce00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2f4067360 0x7fd2f4069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee9c06cc50 0x7fee9c159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c162090 0x7fee9c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c1611f0 0x7fee9c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c16f270 con 0x7fee9c162090 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c162680 con 0x7fee9c06cc50 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c16f3f0 con 0x7fee9c1611f0 2024-04-25T10:18:08.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134e000640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1334066fe0 0x7f13340694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c1611f0 0x7fee9c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c1611f0 0x7fee9c0b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53836/0 (socket says 172.21.15.45:53836) 2024-04-25T10:18:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 -- 172.21.15.45:0/858573087 learned_addr learned my addr 172.21.15.45:0/858573087 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fc537400640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7fc51806f230 0x7fc518071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9b400640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee9c06cc50 0x7fee9c159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1350096d90 con 0x7f135815bd40 2024-04-25T10:18:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f1355e00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1334066fe0 0x7f13340694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd31d800640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7fd2f406f500 0x7fd2f4071920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 -- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee9c06cc50 msgr2=0x7fee9c159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee9c06cc50 0x7fee9c159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 -- 172.21.15.45:0/858573087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c162090 msgr2=0x7fee9c160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee9c162090 0x7fee9c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 -- 172.21.15.45:0/858573087 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c161770 con 0x7fee9c1611f0 2024-04-25T10:18:08.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fd31ce00640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2f4067360 0x7fd2f4069820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd308000f70 tx=0x7fd308009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134e000640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f133406f200 0x7f1334071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 --> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1350004820 con 0x7f133406f200 2024-04-25T10:18:08.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13500970c0 con 0x7f135815bd40 2024-04-25T10:18:08.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f134fe00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f133406f200 0x7f1334071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee9be00640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c1611f0 0x7fee9c0b2eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee9400cf20 tx=0x7fee9400a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7f1355e00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1334066fe0 0x7f13340694a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1348007fd0 tx=0x7f1348009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9401a070 con 0x7fee9c1611f0 2024-04-25T10:18:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9400d840 con 0x7fee9c1611f0 2024-04-25T10:18:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400dc30 con 0x7fee9c1611f0 2024-04-25T10:18:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c0c36f0 con 0x7fee9c1611f0 2024-04-25T10:18:08.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c0c3cb0 con 0x7fee9c1611f0 2024-04-25T10:18:08.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.609+0000 7fc537400640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7fc51806f230 0x7fc518071650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7f134fe00640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f133406f200 0x7f1334071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee9c04ef30 con 0x7fee9c1611f0 2024-04-25T10:18:08.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== osd.4 v2:172.21.15.117:6800/728189779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd3000b7ea0 con 0x7fd2f406f500 2024-04-25T10:18:08.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d40052f0 0x7fb0f01ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== osd.2 v2:172.21.15.45:6817/962547941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc528008eb0 con 0x7fc51806f230 2024-04-25T10:18:08.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158b50 0x7fb0f0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0efe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d2f60 0x7fb0f00d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0efe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b6c70 con 0x7fb0f0158b50 2024-04-25T10:18:08.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee9401d070 con 0x7fee9c1611f0 2024-04-25T10:18:08.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== osd.5 v2:172.21.15.117:6802/2444936005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1350004820 con 0x7f133406f200 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0efe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b7880 con 0x7fb0d40052f0 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0efe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b6f50 con 0x7fb0f00d2f60 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0ef400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d2f60 0x7fb0f00d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0ef400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d2f60 0x7fb0f00d33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53850/0 (socket says 172.21.15.45:53850) 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0ef400640 1 -- 172.21.15.45:0/92913240 learned_addr learned my addr 172.21.15.45:0/92913240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0eea00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d40052f0 0x7fb0f01ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.613+0000 7fb0e7e00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158b50 0x7fb0f0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee93400640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee7c066fb0 0x7fee7c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee940858a0 con 0x7fee9c1611f0 2024-04-25T10:18:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee9b400640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee7c066fb0 0x7fee7c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee93400640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7fee7c06f150 0x7fee7c071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 --> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee7c071c20 con 0x7fee7c06f150 2024-04-25T10:18:08.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee9aa00640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7fee7c06f150 0x7fee7c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee9aa00640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7fee7c06f150 0x7fee7c071570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee9b400640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee7c066fb0 0x7fee7c069470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000f70 tx=0x7fee8c0018e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fb0eea00640 1 -- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d2f60 msgr2=0x7fb0f00d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fb0eea00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d2f60 0x7fb0f00d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fb0eea00640 1 -- 172.21.15.45:0/92913240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158b50 msgr2=0x7fb0f0156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== osd.7 v2:172.21.15.117:6817/2004387070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fee7c071c20 con 0x7fee7c06f150 2024-04-25T10:18:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fb0eea00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158b50 0x7fb0f0156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.617+0000 7fb0eea00640 1 -- 172.21.15.45:0/92913240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00c8db0 con 0x7fb0d40052f0 2024-04-25T10:18:08.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.621+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee9408c050 con 0x7fee9c1611f0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.621+0000 7fb0e7e00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f0158b50 0x7fb0f0156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.625+0000 7fb0eea00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d40052f0 0x7fb0f01ffe10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e530 tx=0x7fb0e000e9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.625+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00052e0 con 0x7fb0d40052f0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.625+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0005480 con 0x7fb0d40052f0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.625+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0007230 con 0x7fb0d40052f0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.625+0000 7fb0efe00640 1 -- 172.21.15.45:0/92913240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f015f680 con 0x7fb0d40052f0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.625+0000 7fb0efe00640 1 -- 172.21.15.45:0/92913240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f015fc40 con 0x7fb0d40052f0 2024-04-25T10:18:08.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.629+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0f0075120 con 0x7fb0d40052f0 2024-04-25T10:18:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.629+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 --> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f810c04ef30 con 0x7f80ec06f100 2024-04-25T10:18:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.629+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0e00073d0 con 0x7fb0d40052f0 2024-04-25T10:18:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e6a00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0cc067060 0x7fb0cc069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0e00849e0 con 0x7fb0d40052f0 2024-04-25T10:18:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e6a00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7fb0cc06f260 0x7fb0cc071680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8103400640 1 -- 172.21.15.45:0/1763749884 <== osd.3 v2:172.21.15.45:6800/3296322295 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f810c04ef30 con 0x7f80ec06f100 2024-04-25T10:18:08.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 --> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0cc071d30 con 0x7fb0cc06f260 2024-04-25T10:18:08.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f80ec06f100 msgr2=0x7f80ec071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f80ec06f100 0x7f80ec071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f80ec066f60 msgr2=0x7f80ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f80ec066f60 0x7f80ec069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f80fc001650 tx=0x7f80fc009040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e7e00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0cc067060 0x7fb0cc069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0ef400640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7fb0cc06f260 0x7fb0cc071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c070310 msgr2=0x7f810c15c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0ef400640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7fb0cc06f260 0x7fb0cc071680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 --2- 172.21.15.45:0/1763749884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f810c070310 0x7f810c15c400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f80f800b580 tx=0x7f80f800ba40 comp rx=0 tx=0).stop 2024-04-25T10:18:08.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f810b400640 1 -- 172.21.15.45:0/1763749884 reap_dead start 2024-04-25T10:18:08.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 --> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc944052cf0 con 0x7fc92c06f1b0 2024-04-25T10:18:08.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e7e00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0cc067060 0x7fb0cc069520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00ce0a0 tx=0x7fb0d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 --> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44841658b0 con 0x7f446406f100 2024-04-25T10:18:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 shutdown_connections 2024-04-25T10:18:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 >> 172.21.15.45:0/1763749884 conn(0x7f810c066b70 msgr2=0x7f810c0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e0051600 con 0x7fb0d40052f0 2024-04-25T10:18:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 shutdown_connections 2024-04-25T10:18:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7f8111e00640 1 -- 172.21.15.45:0/1763749884 wait complete. 2024-04-25T10:18:08.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.633+0000 7fc948c00640 1 -- 172.21.15.45:0/896570209 <== osd.1 v2:172.21.15.45:6816/3290058656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc944052cf0 con 0x7fc92c06f1b0 2024-04-25T10:18:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 --> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc5380ceff0 con 0x7fc51806f230 2024-04-25T10:18:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== osd.6 v2:172.21.15.117:6816/2565393264 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0cc071d30 con 0x7fb0cc06f260 2024-04-25T10:18:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc52ea00640 1 -- 172.21.15.45:0/1609063290 <== osd.2 v2:172.21.15.45:6817/962547941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc5380ceff0 con 0x7fc51806f230 2024-04-25T10:18:08.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7fc92c06f1b0 msgr2=0x7fc92c0715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7fc92c06f1b0 0x7fc92c0715d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc92c066fb0 msgr2=0x7fc92c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc92c066fb0 0x7fc92c069470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ad000 tx=0x7fc938008040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc94400aae0 msgr2=0x7fc9440d39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 --2- 172.21.15.45:0/896570209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc94400aae0 0x7fc9440d39e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b3e0 tx=0x7fc94000b8a0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc951800640 1 -- 172.21.15.45:0/896570209 reap_dead start 2024-04-25T10:18:08.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 shutdown_connections 2024-04-25T10:18:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 >> 172.21.15.45:0/896570209 conn(0x7fc944009fc0 msgr2=0x7fc94400a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 shutdown_connections 2024-04-25T10:18:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc952200640 1 -- 172.21.15.45:0/896570209 wait complete. 2024-04-25T10:18:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f447b400640 1 -- 172.21.15.45:0/4025700633 <== osd.0 v2:172.21.15.45:6808/3877436580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f44841658b0 con 0x7f446406f100 2024-04-25T10:18:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7fc51806f230 msgr2=0x7fc518071650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7fc51806f230 0x7fc518071650 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc518066fb0 msgr2=0x7fc518069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc518066fb0 0x7fc518069470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc538156400 tx=0x7fc524009040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5380ccf50 msgr2=0x7fc53809d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 --2- 172.21.15.45:0/1609063290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5380ccf50 0x7fc53809d110 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e890 tx=0x7fc52800ed50 comp rx=0 tx=0).stop 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537400640 1 -- 172.21.15.45:0/1609063290 reap_dead start 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 shutdown_connections 2024-04-25T10:18:08.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 >> 172.21.15.45:0/1609063290 conn(0x7fc5380b1a20 msgr2=0x7fc5380b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7f446406f100 msgr2=0x7f4464071520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7f446406f100 0x7f4464071520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4464066f60 msgr2=0x7f4464069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4464066f60 0x7f4464069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f447c00a7f0 tx=0x7f447c00d040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484161d90 msgr2=0x7f4484071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 --2- 172.21.15.45:0/4025700633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4484161d90 0x7f4484071df0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44700ae040 tx=0x7f447001ac10 comp rx=0 tx=0).stop 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4483e00640 1 -- 172.21.15.45:0/4025700633 reap_dead start 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 shutdown_connections 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.637+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 >> 172.21.15.45:0/4025700633 conn(0x7f44840b16a0 msgr2=0x7f448415e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 shutdown_connections 2024-04-25T10:18:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fc537e00640 1 -- 172.21.15.45:0/1609063290 wait complete. 2024-04-25T10:18:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 --> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2f0002de0 con 0x7fd2f406f500 2024-04-25T10:18:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 shutdown_connections 2024-04-25T10:18:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7f4488a00640 1 -- 172.21.15.45:0/4025700633 wait complete. 2024-04-25T10:18:08.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd315600640 1 -- 172.21.15.45:0/335421786 <== osd.4 v2:172.21.15.117:6800/728189779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd2f0002de0 con 0x7fd2f406f500 2024-04-25T10:18:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7fd2f406f500 msgr2=0x7fd2f4071920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7fd2f406f500 0x7fd2f4071920 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2f4067360 msgr2=0x7fd2f4069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2f4067360 0x7fd2f4069820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd308000f70 tx=0x7fd308009040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806d270 msgr2=0x7fd318160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 --2- 172.21.15.45:0/335421786 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd31806d270 0x7fd318160670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3000ac000 tx=0x7fd30001ad80 comp rx=0 tx=0).stop 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31d800640 1 -- 172.21.15.45:0/335421786 reap_dead start 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 shutdown_connections 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 >> 172.21.15.45:0/335421786 conn(0x7fd3180669a0 msgr2=0x7fd3180af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 shutdown_connections 2024-04-25T10:18:08.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.641+0000 7fd31e200640 1 -- 172.21.15.45:0/335421786 wait complete. 2024-04-25T10:18:08.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.645+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 --> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee9c0c65f0 con 0x7fee7c06f150 2024-04-25T10:18:08.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.645+0000 7fee93400640 1 -- 172.21.15.45:0/858573087 <== osd.7 v2:172.21.15.117:6817/2004387070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee9c0c65f0 con 0x7fee7c06f150 2024-04-25T10:18:08.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 --> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f13580c80c0 con 0x7f133406f200 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7fee7c06f150 msgr2=0x7fee7c071570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7fee7c06f150 0x7fee7c071570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee7c066fb0 msgr2=0x7fee7c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee7c066fb0 0x7fee7c069470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000f70 tx=0x7fee8c0018e0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c1611f0 msgr2=0x7fee9c0b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 --2- 172.21.15.45:0/858573087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee9c1611f0 0x7fee9c0b2eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fee9400cf20 tx=0x7fee9400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7fee9be00640 1 -- 172.21.15.45:0/858573087 reap_dead start 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 shutdown_connections 2024-04-25T10:18:08.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.649+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 >> 172.21.15.45:0/858573087 conn(0x7fee9c0cfb70 msgr2=0x7fee9c15e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f134e000640 1 -- 172.21.15.45:0/1684507502 <== osd.5 v2:172.21.15.117:6802/2444936005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f13580c80c0 con 0x7f133406f200 2024-04-25T10:18:08.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 shutdown_connections 2024-04-25T10:18:08.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7feea2800640 1 -- 172.21.15.45:0/858573087 wait complete. 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f133406f200 msgr2=0x7f1334071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f133406f200 0x7f1334071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1334066fe0 msgr2=0x7f13340694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1334066fe0 0x7f13340694a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1348007fd0 tx=0x7f1348009040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 msgr2=0x7f13580c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 --2- 172.21.15.45:0/1684507502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f135815bd40 0x7f13580c58a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f135000df50 tx=0x7f135000b480 comp rx=0 tx=0).stop 2024-04-25T10:18:08.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1356800640 1 -- 172.21.15.45:0/1684507502 reap_dead start 2024-04-25T10:18:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 shutdown_connections 2024-04-25T10:18:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 >> 172.21.15.45:0/1684507502 conn(0x7f13580b32b0 msgr2=0x7f135806e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 shutdown_connections 2024-04-25T10:18:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.653+0000 7f1357200640 1 -- 172.21.15.45:0/1684507502 wait complete. 2024-04-25T10:18:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.669+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 --> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0f00d15c0 con 0x7fb0cc06f260 2024-04-25T10:18:08.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.669+0000 7fb0e6a00640 1 -- 172.21.15.45:0/92913240 <== osd.6 v2:172.21.15.117:6816/2565393264 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb0f00d15c0 con 0x7fb0cc06f260 2024-04-25T10:18:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7fb0cc06f260 msgr2=0x7fb0cc071680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7fb0cc06f260 0x7fb0cc071680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0cc067060 msgr2=0x7fb0cc069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0cc067060 0x7fb0cc069520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00ce0a0 tx=0x7fb0d8009040 comp rx=0 tx=0).stop 2024-04-25T10:18:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d40052f0 msgr2=0x7fb0f01ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 --2- 172.21.15.45:0/92913240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d40052f0 0x7fb0f01ffe10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e530 tx=0x7fb0e000e9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0ef400640 1 -- 172.21.15.45:0/92913240 reap_dead start 2024-04-25T10:18:08.680 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:08.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:18:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 shutdown_connections 2024-04-25T10:18:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 >> 172.21.15.45:0/92913240 conn(0x7fb0f006aec0 msgr2=0x7fb0f00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 shutdown_connections 2024-04-25T10:18:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.673+0000 7fb0dfe00640 1 -- 172.21.15.45:0/92913240 wait complete. 2024-04-25T10:18:08.687 INFO:teuthology.orchestra.run.smithi045.stdout:64424509447 2024-04-25T10:18:08.688 INFO:teuthology.orchestra.run.smithi045.stdout:60129542150 2024-04-25T10:18:08.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:18:08.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:18:08.690 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:08.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:18:08.695 INFO:teuthology.orchestra.run.smithi045.stdout:64424509447 2024-04-25T10:18:08.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:18:08.701 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:08.701 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:08.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:18:08.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:18:08.704 INFO:teuthology.orchestra.run.smithi045.stdout:64424509447 2024-04-25T10:18:08.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:18:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f7925600640 1 Processor -- start 2024-04-25T10:18:08.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f7925600640 1 -- start start 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f7925600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7920070e00 0x7f79200cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f7925600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200caa50 con 0x7f7920159e70 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f7925600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200cad30 con 0x7f79200cca50 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f7925600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79200ccec0 con 0x7f7920070e00 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f791ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7920159e70 0x7f79200ca150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59224/0 (socket says 172.21.15.45:59224) 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.917+0000 7f791ea00640 1 -- 172.21.15.45:0/3828507911 learned_addr learned my addr 172.21.15.45:0/3828507911 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.921+0000 7f791e000640 1 --2- 172.21.15.45:0/3828507911 >> v2:172.21.15.117:3300/0 conn(0x7f7920070e00 0x7f79200cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.921+0000 7f791d600640 1 -- 172.21.15.45:0/3828507911 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420119149 0 0) 0x7f79200cad30 con 0x7f79200cca50 2024-04-25T10:18:08.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.921+0000 7f791d600640 1 -- 172.21.15.45:0/3828507911 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7904005740 con 0x7f79200cca50 2024-04-25T10:18:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.921+0000 7f791e000640 1 -- 172.21.15.45:0/3828507911 >> v1:172.21.15.117:6789/0 conn(0x7f79200cca50 legacy=0x7f792006c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.921+0000 7f791e000640 1 -- 172.21.15.45:0/3828507911 >> v1:172.21.15.45:6789/0 conn(0x7f7920159e70 legacy=0x7f79200ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.921+0000 7f791e000640 1 -- 172.21.15.45:0/3828507911 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200cb010 con 0x7f7920070e00 2024-04-25T10:18:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.925+0000 7f791e000640 1 --2- 172.21.15.45:0/3828507911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7920070e00 0x7f79200cc510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f791000a960 tx=0x7f791001c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d706f238b5fec42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.925+0000 7f791d600640 1 -- 172.21.15.45:0/3828507911 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79100b2070 con 0x7f7920070e00 2024-04-25T10:18:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.925+0000 7f791d600640 1 -- 172.21.15.45:0/3828507911 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79100a5720 con 0x7f7920070e00 2024-04-25T10:18:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.925+0000 7f791d600640 1 -- 172.21.15.45:0/3828507911 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791000b040 con 0x7f7920070e00 2024-04-25T10:18:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 -- 172.21.15.45:0/3828507911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7920070e00 msgr2=0x7f79200cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 --2- 172.21.15.45:0/3828507911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7920070e00 0x7f79200cc510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f791000a960 tx=0x7f791001c790 comp rx=0 tx=0).stop 2024-04-25T10:18:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 -- 172.21.15.45:0/3828507911 shutdown_connections 2024-04-25T10:18:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 --2- 172.21.15.45:0/3828507911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7920070e00 0x7f79200cc510 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f791000a960 tx=0x7f791001c790 comp rx=0 tx=0).stop 2024-04-25T10:18:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 -- 172.21.15.45:0/3828507911 >> 172.21.15.45:0/3828507911 conn(0x7f79200b1330 msgr2=0x7f79200b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 -- 172.21.15.45:0/3828507911 shutdown_connections 2024-04-25T10:18:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.929+0000 7f7925600640 1 -- 172.21.15.45:0/3828507911 wait complete. 2024-04-25T10:18:08.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.933+0000 7f7925600640 1 Processor -- start 2024-04-25T10:18:08.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.933+0000 7f7925600640 1 -- start start 2024-04-25T10:18:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.933+0000 7f7925600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 0x7f7920203350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.933+0000 7f7925600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7920159e70 0x7f7920152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.933+0000 7f7925600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7920152e50 0x7f79200c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.933+0000 7f791ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 0x7f7920203350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 0x7f7920203350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51778/0 (socket says 172.21.15.45:51778) 2024-04-25T10:18:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f7925600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200cbc70 con 0x7f7920159e70 2024-04-25T10:18:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f7925600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200cbaf0 con 0x7f79200cca50 2024-04-25T10:18:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f7925600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920167a10 con 0x7f7920152e50 2024-04-25T10:18:08.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 Processor -- start 2024-04-25T10:18:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 -- 172.21.15.45:0/2900332636 learned_addr learned my addr 172.21.15.45:0/2900332636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791f400640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7920152e50 0x7f79200c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 -- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7920152e50 msgr2=0x7f79200c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7920152e50 0x7f79200c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 -- 172.21.15.45:0/2900332636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7920159e70 msgr2=0x7f7920152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7920159e70 0x7f7920152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 -- 172.21.15.45:0/2900332636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200c4f20 con 0x7f79200cca50 2024-04-25T10:18:08.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791f400640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7920152e50 0x7f79200c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 -- start start 2024-04-25T10:18:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f791ea00640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 0x7f7920203350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7918001a60 tx=0x7f7918001f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4e8806d3d0 0x7f4e880b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e880b2fc0 0x7f4e880708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e880b33d0 con 0x7f4e88071e30 2024-04-25T10:18:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8806cf40 con 0x7f4e8806d3d0 2024-04-25T10:18:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e8cc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8806c090 con 0x7f4e880b2fc0 2024-04-25T10:18:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918018070 con 0x7f79200cca50 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e86800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e880b2fc0 0x7f4e880708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e86800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e880b2fc0 0x7f4e880708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51790/0 (socket says 172.21.15.45:51790) 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.937+0000 7f4e86800640 1 -- 172.21.15.45:0/3173523747 learned_addr learned my addr 172.21.15.45:0/3173523747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f4e85400640 1 --2- 172.21.15.45:0/3173523747 >> v2:172.21.15.45:3300/0 conn(0x7f4e8806d3d0 0x7f4e880b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79180082d0 con 0x7f79200cca50 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79200c5200 con 0x7f79200cca50 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79200c57c0 con 0x7f79200cca50 2024-04-25T10:18:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800d900 con 0x7f79200cca50 2024-04-25T10:18:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f4e86800640 1 -- 172.21.15.45:0/3173523747 >> v2:172.21.15.45:3300/0 conn(0x7f4e8806d3d0 msgr2=0x7f4e880b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f4e86800640 1 --2- 172.21.15.45:0/3173523747 >> v2:172.21.15.45:3300/0 conn(0x7f4e8806d3d0 0x7f4e880b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f4e86800640 1 -- 172.21.15.45:0/3173523747 >> v1:172.21.15.45:6789/0 conn(0x7f4e88071e30 legacy=0x7f4e8815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f4e86800640 1 -- 172.21.15.45:0/3173523747 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8806c330 con 0x7f4e880b2fc0 2024-04-25T10:18:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.941+0000 7f4e85400640 1 --2- 172.21.15.45:0/3173523747 >> v2:172.21.15.45:3300/0 conn(0x7f4e8806d3d0 0x7f4e880b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79200c97b0 con 0x7f79200cca50 2024-04-25T10:18:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f791801b070 con 0x7f79200cca50 2024-04-25T10:18:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f7916a00640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7900066fb0 0x7f7900069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7918084660 con 0x7f79200cca50 2024-04-25T10:18:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f8231a00640 1 Processor -- start 2024-04-25T10:18:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f8231a00640 1 -- start start 2024-04-25T10:18:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.945+0000 7f791e000640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7900066fb0 0x7f7900069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f4e86800640 1 --2- 172.21.15.45:0/3173523747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e880708a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e88158250 tx=0x7f4e8001eb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b086ea9a11365b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f791e000640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7900066fb0 0x7f7900069470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79100af000 tx=0x7f7910004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f8231a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82240df2b0 0x7f822404faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f822be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82240df2b0 0x7f822404faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f822be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82240df2b0 0x7f822404faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57302/0 (socket says 172.21.15.45:57302) 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f8231a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82240e3b30 con 0x7f8224007550 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f8231a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8224054120 con 0x7f8224033d00 2024-04-25T10:18:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f8231a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82240df690 con 0x7f82240df2b0 2024-04-25T10:18:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f822be00640 1 -- 172.21.15.45:0/1708461188 learned_addr learned my addr 172.21.15.45:0/1708461188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f822be00640 1 -- 172.21.15.45:0/1708461188 >> v1:172.21.15.117:6789/0 conn(0x7f8224033d00 legacy=0x7f82240e2d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:18:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f822be00640 1 -- 172.21.15.45:0/1708461188 >> v1:172.21.15.45:6789/0 conn(0x7f8224007550 legacy=0x7f82240e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.949+0000 7f822be00640 1 -- 172.21.15.45:0/1708461188 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224054400 con 0x7f82240df2b0 2024-04-25T10:18:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e84a00640 1 -- 172.21.15.45:0/3173523747 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800ad070 con 0x7f4e880b2fc0 2024-04-25T10:18:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e84a00640 1 -- 172.21.15.45:0/3173523747 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e800a7c00 con 0x7f4e880b2fc0 2024-04-25T10:18:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e84a00640 1 -- 172.21.15.45:0/3173523747 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800b1400 con 0x7f4e880b2fc0 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 Processor -- start 2024-04-25T10:18:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 -- start start 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7918055100 con 0x7f79200cca50 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 -- 172.21.15.45:0/3173523747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 msgr2=0x7f4e880708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/3173523747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e880708a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e88158250 tx=0x7f4e8001eb30 comp rx=0 tx=0).stop 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 -- 172.21.15.45:0/3173523747 shutdown_connections 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/3173523747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e880708a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/3173523747 >> v2:172.21.15.45:3300/0 conn(0x7f4e8806d3d0 0x7f4e880b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 -- 172.21.15.45:0/3173523747 >> 172.21.15.45:0/3173523747 conn(0x7f4e88066b70 msgr2=0x7f4e88066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f2d19200640 1 Processor -- start 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 -- 172.21.15.45:0/3173523747 shutdown_connections 2024-04-25T10:18:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f822be00640 1 --2- 172.21.15.45:0/1708461188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82240df2b0 0x7f822404faa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8220008870 tx=0x7f822001e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacf56c17956d4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f2d19200640 1 -- start start 2024-04-25T10:18:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f4e8cc00640 1 -- 172.21.15.45:0/3173523747 wait complete. 2024-04-25T10:18:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f822aa00640 1 -- 172.21.15.45:0/1708461188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200ad070 con 0x7f82240df2b0 2024-04-25T10:18:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46e415a850 0x7f46e415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46e40709b0 0x7f46e4070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40cbe80 con 0x7f46e415d900 2024-04-25T10:18:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e4071350 con 0x7f46e40709b0 2024-04-25T10:18:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f46eae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40cb220 con 0x7f46e415a850 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.953+0000 7f822aa00640 1 -- 172.21.15.45:0/1708461188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200a78f0 con 0x7f82240df2b0 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 -- 172.21.15.45:0/1708461188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82240df2b0 msgr2=0x7f822404faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f822aa00640 1 -- 172.21.15.45:0/1708461188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200ad070 con 0x7f82240df2b0 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 --2- 172.21.15.45:0/1708461188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82240df2b0 0x7f822404faa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8220008870 tx=0x7f822001e930 comp rx=0 tx=0).stop 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d19200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2d140d0920 0x7f2d140d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e8e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f46e415d900 0x7f46e415dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32780/0 (socket says 172.21.15.45:32780) 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d19200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 0x7f2d140b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e8e00640 1 -- 172.21.15.45:0/1057550151 learned_addr learned my addr 172.21.15.45:0/1057550151 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f4e8cc00640 1 Processor -- start 2024-04-25T10:18:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e3400640 1 --2- 172.21.15.45:0/1057550151 >> v2:172.21.15.45:3300/0 conn(0x7f46e40709b0 0x7f46e4070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d19200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d140d2490 con 0x7f2d1415df00 2024-04-25T10:18:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f4e8cc00640 1 -- start start 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 0x7f2d140b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 0x7f2d140b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53884/0 (socket says 172.21.15.45:53884) 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 -- 172.21.15.45:0/2873470895 learned_addr learned my addr 172.21.15.45:0/2873470895 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d140d2770 con 0x7f2d140d0920 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140d2a50 con 0x7f2d140d1ce0 2024-04-25T10:18:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e3e00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46e415a850 0x7f46e415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510552176 0 0) 0x7f46e40cbe80 con 0x7f46e415d900 2024-04-25T10:18:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c8003530 con 0x7f46e415d900 2024-04-25T10:18:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 -- 172.21.15.45:0/1708461188 shutdown_connections 2024-04-25T10:18:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 --2- 172.21.15.45:0/1708461188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82240df2b0 0x7f822404faa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d11400640 1 --2- 172.21.15.45:0/2873470895 >> v2:172.21.15.117:3300/0 conn(0x7f2d140d0920 0x7f2d140d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 -- 172.21.15.45:0/1708461188 >> 172.21.15.45:0/1708461188 conn(0x7f8224009fc0 msgr2=0x7f822400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2090293930 0 0) 0x7f46c8003530 con 0x7f46e415d900 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46e415a850 msgr2=0x7f46e415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46e415a850 0x7f46e415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 >> v2:172.21.15.45:3300/0 conn(0x7f46e40709b0 msgr2=0x7f46e4070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 --2- 172.21.15.45:0/1057550151 >> v2:172.21.15.45:3300/0 conn(0x7f46e40709b0 0x7f46e4070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40ca610 con 0x7f46e415d900 2024-04-25T10:18:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f46dc005060 con 0x7f46e415d900 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 >> v1:172.21.15.117:6789/0 conn(0x7f46e415d900 legacy=0x7f46e415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80039c0 0x7f46c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c80051d0 0x7f46c800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40cb220 con 0x7f46c80051d0 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c800e1d0 con 0x7f46c8004810 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c800e3a0 con 0x7f46c80039c0 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 -- 172.21.15.45:0/2873470895 >> v2:172.21.15.117:3300/0 conn(0x7f2d140d0920 msgr2=0x7f2d140d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 --2- 172.21.15.45:0/2873470895 >> v2:172.21.15.117:3300/0 conn(0x7f2d140d0920 0x7f2d140d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 -- 172.21.15.45:0/2873470895 >> v1:172.21.15.45:6789/0 conn(0x7f2d1415df00 legacy=0x7f2d140d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f2d12800640 1 -- 172.21.15.45:0/2873470895 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140d2d30 con 0x7f2d140d1ce0 2024-04-25T10:18:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f46e3e00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80039c0 0x7f46c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e3400640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 -- 172.21.15.45:0/1708461188 shutdown_connections 2024-04-25T10:18:08.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.957+0000 7f8231a00640 1 -- 172.21.15.45:0/1708461188 wait complete. 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e8cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8806d3d0 0x7f4e880ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e8e00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c80051d0 0x7f46c800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e3400640 1 -- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80039c0 msgr2=0x7f46c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e3400640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c80039c0 0x7f46c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e3400640 1 -- 172.21.15.45:0/1057550151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c80051d0 msgr2=0x7f46c800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e3400640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c80051d0 0x7f46c800dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f46e3400640 1 -- 172.21.15.45:0/1057550151 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40ca610 con 0x7f46c8004810 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e8cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e88071e30 0x7f4e88152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e8cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e88153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e8cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880711f0 con 0x7f4e8806d3d0 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e8cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88072210 con 0x7f4e880b2fc0 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e8cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88071370 con 0x7f4e88071e30 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e86800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e88153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e86800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e88153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51820/0 (socket says 172.21.15.45:51820) 2024-04-25T10:18:08.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e86800640 1 -- 172.21.15.45:0/620553955 learned_addr learned my addr 172.21.15.45:0/620553955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8806d3d0 0x7f4e880ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85400640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e88071e30 0x7f4e88152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 -- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e88071e30 msgr2=0x7f4e88152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e88071e30 0x7f4e88152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 -- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 msgr2=0x7f4e88153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e88153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 -- 172.21.15.45:0/620553955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8814ae80 con 0x7f4e8806d3d0 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f8231a00640 1 Processor -- start 2024-04-25T10:18:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f596b800640 1 Processor -- start 2024-04-25T10:18:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f596b800640 1 -- start start 2024-04-25T10:18:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f8231a00640 1 -- start start 2024-04-25T10:18:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.961+0000 7f4e85e00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8806d3d0 0x7f4e880ccef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e78000f70 tx=0x7f4e7800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f596b800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f596406e400 0x7f5964159e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f596b800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f596415d7d0 0x7f596415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f596b800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640724b0 con 0x7f59640b3180 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f596b800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596415b180 con 0x7f596406e400 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f596b800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596406c090 con 0x7f596415d7d0 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f2d12800640 1 --2- 172.21.15.45:0/2873470895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 0x7f2d140b71e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009870 tx=0x7f2d0401cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359c87b898b35f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78017070 con 0x7f4e8806d3d0 2024-04-25T10:18:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f8231a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8224007550 0x7f82241808c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5963e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59640b3180 0x7f5964071cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59270/0 (socket says 172.21.15.45:59270) 2024-04-25T10:18:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5963e00640 1 -- 172.21.15.45:0/1242896823 learned_addr learned my addr 172.21.15.45:0/1242896823 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e880cee30 con 0x7f4e8806d3d0 2024-04-25T10:18:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e88146e40 con 0x7f4e8806d3d0 2024-04-25T10:18:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f8231a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 0x7f8224036ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f8231a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82240df2b0 0x7f822404a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f8231a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224053dd0 con 0x7f8224033d00 2024-04-25T10:18:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f8231a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240503f0 con 0x7f8224007550 2024-04-25T10:18:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f8231a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822404f660 con 0x7f82240df2b0 2024-04-25T10:18:08.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5969800640 1 --2- 172.21.15.45:0/1242896823 >> v2:172.21.15.117:3300/0 conn(0x7f596415d7d0 0x7f596415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e780052d0 con 0x7f4e8806d3d0 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e78017070 con 0x7f4e8806d3d0 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5963400640 1 -- 172.21.15.45:0/1242896823 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072172243 0 0) 0x7f59640724b0 con 0x7f59640b3180 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5969800640 1 -- 172.21.15.45:0/1242896823 >> v2:172.21.15.45:3300/0 conn(0x7f596406e400 msgr2=0x7f5964159e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5969800640 1 --2- 172.21.15.45:0/1242896823 >> v2:172.21.15.45:3300/0 conn(0x7f596406e400 0x7f5964159e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5969800640 1 -- 172.21.15.45:0/1242896823 >> v1:172.21.15.45:6789/0 conn(0x7f59640b3180 legacy=0x7f5964071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f5969800640 1 -- 172.21.15.45:0/1242896823 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596406c370 con 0x7f596415d7d0 2024-04-25T10:18:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 0x7f8224036ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 0x7f8224036ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57342/0 (socket says 172.21.15.45:57342) 2024-04-25T10:18:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 -- 172.21.15.45:0/2420669299 learned_addr learned my addr 172.21.15.45:0/2420669299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822be00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8224007550 0x7f82241808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f6580800640 1 Processor -- start 2024-04-25T10:18:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f2d10a00640 1 -- 172.21.15.45:0/2873470895 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040aa430 con 0x7f2d140d1ce0 2024-04-25T10:18:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f6580800640 1 -- start start 2024-04-25T10:18:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f2d10a00640 1 -- 172.21.15.45:0/2873470895 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040af070 con 0x7f2d140d1ce0 2024-04-25T10:18:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f2d10a00640 1 -- 172.21.15.45:0/2873470895 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040aa7f0 con 0x7f2d140d1ce0 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8804ef30 con 0x7f4e8806d3d0 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46e3400640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46d00073d0 tx=0x7f46d001c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff7336bdcc6dc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0007050 con 0x7f46c8004810 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f6580800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65780cda10 0x7f657806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f6580800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cf930 0x7f65780cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f6580800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65780b2770 con 0x7f657815bc40 2024-04-25T10:18:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f6580800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657806c670 con 0x7f65780cda10 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f6580800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657806c950 con 0x7f65780cf930 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657de00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f657815bc40 0x7f65780b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32782/0 (socket says 172.21.15.45:32782) 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657de00640 1 -- 172.21.15.45:0/3295044550 learned_addr learned my addr 172.21.15.45:0/3295044550 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e7801a070 con 0x7f4e8806d3d0 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f4e7e000640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4e68067360 0x7f4e68069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e780050d0 con 0x7f4e8806d3d0 2024-04-25T10:18:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 -- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82240df2b0 msgr2=0x7f822404a510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82240df2b0 0x7f822404a510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 -- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8224007550 msgr2=0x7f82241808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8224007550 0x7f82241808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 -- 172.21.15.45:0/2420669299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822404d7a0 con 0x7f8224033d00 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f822b400640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 0x7f8224036ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f821800b410 tx=0x7f821800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218015020 con 0x7f8224033d00 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.965+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8218011830 con 0x7f8224033d00 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218011c20 con 0x7f8224033d00 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822404da80 con 0x7f8224033d00 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822404e040 con 0x7f8224033d00 2024-04-25T10:18:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82240089e0 con 0x7f8224033d00 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 msgr2=0x7f2d140b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 --2- 172.21.15.45:0/2873470895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 0x7f2d140b71e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009870 tx=0x7f2d0401cf50 comp rx=0 tx=0).stop 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 shutdown_connections 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 --2- 172.21.15.45:0/2873470895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d140d1ce0 0x7f2d140b71e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 --2- 172.21.15.45:0/2873470895 >> v2:172.21.15.117:3300/0 conn(0x7f2d140d0920 0x7f2d140d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 >> 172.21.15.45:0/2873470895 conn(0x7f2d1406b290 msgr2=0x7f2d140669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657e800640 1 --2- 172.21.15.45:0/3295044550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cf930 0x7f65780cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657d400640 1 --2- 172.21.15.45:0/3295044550 >> v2:172.21.15.45:3300/0 conn(0x7f65780cda10 0x7f657806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d00a8660 con 0x7f46c8004810 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46e2a00640 1 -- 172.21.15.45:0/1057550151 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00a8a10 con 0x7f46c8004810 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46eae00640 1 -- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8004810 msgr2=0x7f46c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46eae00640 1 --2- 172.21.15.45:0/1057550151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46d00073d0 tx=0x7f46d001c6f0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46e8e00640 1 -- 172.21.15.45:0/1057550151 reap_dead start 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657ca00640 1 -- 172.21.15.45:0/3295044550 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914966194 0 0) 0x7f65780b2770 con 0x7f657815bc40 2024-04-25T10:18:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657ca00640 1 -- 172.21.15.45:0/3295044550 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6560003530 con 0x7f657815bc40 2024-04-25T10:18:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46eae00640 1 -- 172.21.15.45:0/1057550151 shutdown_connections 2024-04-25T10:18:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46eae00640 1 -- 172.21.15.45:0/1057550151 >> 172.21.15.45:0/1057550151 conn(0x7f46e40cfb70 msgr2=0x7f46e40cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f5969800640 1 --2- 172.21.15.45:0/1242896823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596415d7d0 0x7f596415dc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f596406e7e0 tx=0x7f595401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a3441af70edd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f5963400640 1 -- 172.21.15.45:0/1242896823 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540b1070 con 0x7f596415d7d0 2024-04-25T10:18:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f5963400640 1 -- 172.21.15.45:0/1242896823 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540ac5f0 con 0x7f596415d7d0 2024-04-25T10:18:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f5963400640 1 -- 172.21.15.45:0/1242896823 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540ac9c0 con 0x7f596415d7d0 2024-04-25T10:18:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657e800640 1 -- 172.21.15.45:0/3295044550 >> v2:172.21.15.45:3300/0 conn(0x7f65780cda10 msgr2=0x7f657806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f4e85400640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4e68067360 0x7f4e68069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657ca00640 1 -- 172.21.15.45:0/3295044550 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929719369 0 0) 0x7f6560003530 con 0x7f657815bc40 2024-04-25T10:18:08.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f46eae00640 1 -- 172.21.15.45:0/1057550151 shutdown_connections 2024-04-25T10:18:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f596b800640 1 -- 172.21.15.45:0/1242896823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596415d7d0 msgr2=0x7f596415dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f596b800640 1 --2- 172.21.15.45:0/1242896823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596415d7d0 0x7f596415dc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f596406e7e0 tx=0x7f595401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f596b800640 1 -- 172.21.15.45:0/1242896823 shutdown_connections 2024-04-25T10:18:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f596b800640 1 --2- 172.21.15.45:0/1242896823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596415d7d0 0x7f596415dc60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f596b800640 1 --2- 172.21.15.45:0/1242896823 >> v2:172.21.15.45:3300/0 conn(0x7f596406e400 0x7f5964159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f596b800640 1 -- 172.21.15.45:0/1242896823 >> 172.21.15.45:0/1242896823 conn(0x7f5964066b70 msgr2=0x7f5964066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f4e85400640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4e68067360 0x7f4e68069820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e880cec40 tx=0x7f4e74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657e800640 1 --2- 172.21.15.45:0/3295044550 >> v2:172.21.15.45:3300/0 conn(0x7f65780cda10 0x7f657806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657e800640 1 -- 172.21.15.45:0/3295044550 >> v1:172.21.15.117:6789/0 conn(0x7f657815bc40 legacy=0x7f65780b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f657e800640 1 -- 172.21.15.45:0/3295044550 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657806cc30 con 0x7f65780cf930 2024-04-25T10:18:08.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f46eae00640 1 -- 172.21.15.45:0/1057550151 wait complete. 2024-04-25T10:18:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 shutdown_connections 2024-04-25T10:18:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.969+0000 7f2d19200640 1 -- 172.21.15.45:0/2873470895 wait complete. 2024-04-25T10:18:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f2d19200640 1 Processor -- start 2024-04-25T10:18:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f46eae00640 1 Processor -- start 2024-04-25T10:18:08.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f46eae00640 1 -- start start 2024-04-25T10:18:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.973+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e880b33d0 con 0x7f4e8806d3d0 2024-04-25T10:18:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f596b800640 1 -- 172.21.15.45:0/1242896823 shutdown_connections 2024-04-25T10:18:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f596b800640 1 -- 172.21.15.45:0/1242896823 wait complete. 2024-04-25T10:18:08.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 -- start start 2024-04-25T10:18:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f657e800640 1 --2- 172.21.15.45:0/3295044550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cf930 0x7f65780cfde0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6574009d90 tx=0x7f657401c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e4e3868c9a5c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d140d0920 0x7f2d14150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d140d1ce0 0x7f2d140c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 0x7f2d141fa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140b80b0 con 0x7f2d140d0920 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140b7f30 con 0x7f2d140d1ce0 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d19200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140b8230 con 0x7f2d1415df00 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 0x7f2d141fa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 0x7f2d141fa850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53926/0 (socket says 172.21.15.45:53926) 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 -- 172.21.15.45:0/2459487773 learned_addr learned my addr 172.21.15.45:0/2459487773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d11400640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d140d1ce0 0x7f2d140c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d11e00640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d140d0920 0x7f2d14150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 -- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d140d1ce0 msgr2=0x7f2d140c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d140d1ce0 0x7f2d140c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 -- 172.21.15.45:0/2459487773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d140d0920 msgr2=0x7f2d14150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d140d0920 0x7f2d14150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 -- 172.21.15.45:0/2459487773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140c7430 con 0x7f2d1415df00 2024-04-25T10:18:08.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d11e00640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d140d0920 0x7f2d14150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d12800640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 0x7f2d141fa850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d040ac040 tx=0x7f2d0401ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f596b800640 1 Processor -- start 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f596b800640 1 -- start start 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406e400 0x7f59641392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59640b3180 0x7f5964158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 0x7f5964166ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964166960 con 0x7f59640b3180 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596415e5b0 con 0x7f596406e400 2024-04-25T10:18:08.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5964166ae0 con 0x7f596415d7d0 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f46eae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 0x7f46e40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f46eae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e40b5440 0x7f46e414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46eae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e414fad0 0x7f46e4150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46eae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40cabd0 con 0x7f46e414fad0 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46eae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40ce750 con 0x7f46e40b5440 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46eae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e40caeb0 con 0x7f46c8004810 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 0x7f46e40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 0x7f46e40b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53940/0 (socket says 172.21.15.45:53940) 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 -- 172.21.15.45:0/2807959290 learned_addr learned my addr 172.21.15.45:0/2807959290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3400640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e40b5440 0x7f46e414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040bd5f0 con 0x7f2d1415df00 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040bd7b0 con 0x7f2d1415df00 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040bdba0 con 0x7f2d1415df00 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d141509f0 con 0x7f2d1415df00 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14159b40 con 0x7f2d1415df00 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f657ca00640 1 -- 172.21.15.45:0/3295044550 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740ac070 con 0x7f65780cf930 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f657ca00640 1 -- 172.21.15.45:0/3295044550 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65740a6410 con 0x7f65780cf930 2024-04-25T10:18:08.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.977+0000 7f657ca00640 1 -- 172.21.15.45:0/3295044550 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c001680 con 0x7f65780cf930 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 -- 172.21.15.45:0/3295044550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cf930 msgr2=0x7f65780cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 --2- 172.21.15.45:0/3295044550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cf930 0x7f65780cfde0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6574009d90 tx=0x7f657401c800 comp rx=0 tx=0).stop 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 -- 172.21.15.45:0/3295044550 shutdown_connections 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 --2- 172.21.15.45:0/3295044550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cf930 0x7f65780cfde0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 --2- 172.21.15.45:0/3295044550 >> v2:172.21.15.45:3300/0 conn(0x7f65780cda10 0x7f657806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 -- 172.21.15.45:0/3295044550 >> 172.21.15.45:0/3295044550 conn(0x7f6578066b70 msgr2=0x7f6578066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e19000640 1 Processor -- start 2024-04-25T10:18:08.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e19000640 1 -- start start 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 0x7f5964166ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5968e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406e400 0x7f59641392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 0x7f5964166ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53948/0 (socket says 172.21.15.45:53948) 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 -- 172.21.15.45:0/1331593566 learned_addr learned my addr 172.21.15.45:0/1331593566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5963e00640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59640b3180 0x7f5964158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 -- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406e400 msgr2=0x7f59641392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406e400 0x7f59641392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 -- 172.21.15.45:0/1331593566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59640b3180 msgr2=0x7f5964158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59640b3180 0x7f5964158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 -- 172.21.15.45:0/1331593566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596413a890 con 0x7f596415d7d0 2024-04-25T10:18:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e8e00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e414fad0 0x7f46e4150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8218018020 con 0x7f8224033d00 2024-04-25T10:18:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f821fe00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f820c067030 0x7f820c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82180855c0 con 0x7f8224033d00 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5969800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 0x7f5964166ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f595401c980 tx=0x7f5954019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 -- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e40b5440 msgr2=0x7f46e414f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e40b5440 0x7f46e414f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 -- 172.21.15.45:0/2807959290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e414fad0 msgr2=0x7f46e4150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e414fad0 0x7f46e4150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3e00640 1 -- 172.21.15.45:0/2807959290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40b4130 con 0x7f46c8004810 2024-04-25T10:18:08.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 -- 172.21.15.45:0/3295044550 shutdown_connections 2024-04-25T10:18:08.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f6580800640 1 -- 172.21.15.45:0/3295044550 wait complete. 2024-04-25T10:18:08.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e19000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e14162290 0x7f3e140ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:08.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f822be00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f820c067030 0x7f820c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e11400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3e140d27b0 0x7f3e140d02b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32798/0 (socket says 172.21.15.45:32798) 2024-04-25T10:18:08.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e11e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e14162290 0x7f3e140ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:08.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e11e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e14162290 0x7f3e140ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51866/0 (socket says 172.21.15.45:51866) 2024-04-25T10:18:08.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f46e3400640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e40b5440 0x7f46e414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:08.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540b1040 con 0x7f596415d7d0 2024-04-25T10:18:08.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540bb070 con 0x7f596415d7d0 2024-04-25T10:18:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f46e3e00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 0x7f46e40b4eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46d400b6c0 tx=0x7f46d400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4018070 con 0x7f46c8004810 2024-04-25T10:18:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4010830 con 0x7f46c8004810 2024-04-25T10:18:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4010c20 con 0x7f46c8004810 2024-04-25T10:18:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1404ef30 con 0x7f2d1415df00 2024-04-25T10:18:09.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 Processor -- start 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e19000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140ce8e0 con 0x7f3e140d09c0 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e19000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e140cebc0 con 0x7f3e140d27b0 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e19000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140ceea0 con 0x7f3e14162290 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f3e11e00640 1 -- 172.21.15.45:0/4180194095 learned_addr learned my addr 172.21.15.45:0/4180194095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f3e11e00640 1 -- 172.21.15.45:0/4180194095 >> v1:172.21.15.117:6789/0 conn(0x7f3e140d27b0 legacy=0x7f3e140d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f3e11e00640 1 -- 172.21.15.45:0/4180194095 >> v1:172.21.15.45:6789/0 conn(0x7f3e140d09c0 legacy=0x7f3e140b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f3e11e00640 1 -- 172.21.15.45:0/4180194095 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140b6c30 con 0x7f3e14162290 2024-04-25T10:18:09.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540b5610 con 0x7f596415d7d0 2024-04-25T10:18:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596413ab70 con 0x7f596415d7d0 2024-04-25T10:18:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596413b130 con 0x7f596415d7d0 2024-04-25T10:18:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 -- start start 2024-04-25T10:18:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59540b57b0 con 0x7f596415d7d0 2024-04-25T10:18:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593c0052a0 con 0x7f596415d7d0 2024-04-25T10:18:09.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f5960c00640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5944066fe0 0x7f59440694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e414ff50 con 0x7f46c8004810 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e40b43d0 con 0x7f46c8004810 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e404ef30 con 0x7f46c8004810 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cda10 0x7f65781341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65780cf930 0x7f65781498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 0x7f6578149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65780b3890 con 0x7f65780cf930 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65780b3710 con 0x7f657815bc40 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f6580800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780b3a10 con 0x7f65780cda10 2024-04-25T10:18:09.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f657e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 0x7f6578149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f657e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 0x7f6578149e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51874/0 (socket says 172.21.15.45:51874) 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f657e800640 1 -- 172.21.15.45:0/1251816134 learned_addr learned my addr 172.21.15.45:0/1251816134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f657e800640 1 -- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cda10 msgr2=0x7f65781341d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f657e800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cda10 0x7f65781341d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f657e800640 1 -- 172.21.15.45:0/1251816134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65780cf930 msgr2=0x7f65781498f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f657e800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65780cf930 0x7f65781498f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f657e800640 1 -- 172.21.15.45:0/1251816134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657814a420 con 0x7f657815bc40 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.981+0000 7f822be00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f820c067030 0x7f820c0694f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82200b2000 tx=0x7f82200a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.985+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d040c6430 con 0x7f2d1415df00 2024-04-25T10:18:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f2d0a000640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cf4067030 0x7f2cf40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d0411fbc0 con 0x7f2d1415df00 2024-04-25T10:18:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e11e00640 1 --2- 172.21.15.45:0/4180194095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e140ce1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e50 tx=0x7f3e0c01f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391ade514188b5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f595411ec00 con 0x7f596415d7d0 2024-04-25T10:18:09.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f5968e00640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5944066fe0 0x7f59440694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f657e800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 0x7f6578149e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f657401aeb0 tx=0x7f65740195a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e10a00640 1 -- 172.21.15.45:0/4180194095 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b4070 con 0x7f3e14162290 2024-04-25T10:18:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f2d11e00640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cf4067030 0x7f2cf40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740ac070 con 0x7f657815bc40 2024-04-25T10:18:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65740b1880 con 0x7f657815bc40 2024-04-25T10:18:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e10a00640 1 -- 172.21.15.45:0/4180194095 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0af400 con 0x7f3e14162290 2024-04-25T10:18:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e10a00640 1 -- 172.21.15.45:0/4180194095 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0af7d0 con 0x7f3e14162290 2024-04-25T10:18:09.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6574009450 con 0x7f657815bc40 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657814a700 con 0x7f657815bc40 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657814acc0 con 0x7f657815bc40 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 -- 172.21.15.45:0/4180194095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 msgr2=0x7f3e140ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 --2- 172.21.15.45:0/4180194095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e140ce1e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009e50 tx=0x7f3e0c01f1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 -- 172.21.15.45:0/4180194095 shutdown_connections 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 --2- 172.21.15.45:0/4180194095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e140ce1e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 -- 172.21.15.45:0/4180194095 >> 172.21.15.45:0/4180194095 conn(0x7f3e1406b290 msgr2=0x7f3e140669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f5968e00640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5944066fe0 0x7f59440694a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595c0073a0 tx=0x7f595c015040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f2d11e00640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cf4067030 0x7f2cf40694f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d1414c050 tx=0x7f2d0c009590 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f821801f020 con 0x7f8224033d00 2024-04-25T10:18:09.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d04016110 con 0x7f2d1415df00 2024-04-25T10:18:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657804ef30 con 0x7f657815bc40 2024-04-25T10:18:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 -- 172.21.15.45:0/4180194095 shutdown_connections 2024-04-25T10:18:09.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65740af070 con 0x7f657815bc40 2024-04-25T10:18:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.989+0000 7f3e19000640 1 -- 172.21.15.45:0/4180194095 wait complete. 2024-04-25T10:18:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f6572000640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f655c066fb0 0x7f655c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f657411f860 con 0x7f657815bc40 2024-04-25T10:18:09.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f3e19000640 1 Processor -- start 2024-04-25T10:18:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f657de00640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f655c066fb0 0x7f655c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f3e19000640 1 -- start start 2024-04-25T10:18:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f657de00640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f655c066fb0 0x7f655c069470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f656c001390 tx=0x7f656c000ff0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.993+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59540a9070 con 0x7f596415d7d0 2024-04-25T10:18:09.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e19000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e140d09c0 0x7f3e140b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e19000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e140d27b0 0x7f3e14070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e19000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e14070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e19000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140d2bf0 con 0x7f3e140d27b0 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e19000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b69e0 con 0x7f3e14162290 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e19000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14162670 con 0x7f3e140d09c0 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e140d27b0 0x7f3e14070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e140d27b0 0x7f3e14070570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57390/0 (socket says 172.21.15.45:57390) 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11400640 1 -- 172.21.15.45:0/3652897779 learned_addr learned my addr 172.21.15.45:0/3652897779 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e140d09c0 0x7f3e140b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e12800640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e14070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 -- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 msgr2=0x7f3e14070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e14070ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 -- 172.21.15.45:0/3652897779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e140d27b0 msgr2=0x7f3e14070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e140d27b0 0x7f3e14070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 -- 172.21.15.45:0/3652897779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140cbde0 con 0x7f3e140d09c0 2024-04-25T10:18:09.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f3e11e00640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e140d09c0 0x7f3e140b8810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ae000 tx=0x7f3e0c01bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46d401b070 con 0x7f46c8004810 2024-04-25T10:18:09.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65740c1060 con 0x7f657815bc40 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:08.997+0000 7f46dbe00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46c4067030 0x7f46c40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46d4085780 con 0x7f46c8004810 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f46e3400640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46c4067030 0x7f46c40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46d4056770 con 0x7f46c8004810 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b4040 con 0x7f3e140d09c0 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0090d0 con 0x7f3e140d09c0 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b4040 con 0x7f3e140d09c0 2024-04-25T10:18:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f46e3400640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46c4067030 0x7f46c40694f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46d00af000 tx=0x7f46d00046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.001+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e140cc0c0 con 0x7f3e140d09c0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.005+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e140cc680 con 0x7f3e140d09c0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.005+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e0c0c2020 con 0x7f3e140d09c0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.009+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e140b64f0 con 0x7f3e140d09c0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.009+0000 7f3e0a000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3df4067160 0x7f3df4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.009+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e0c0f13b0 con 0x7f3e140d09c0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.009+0000 7f3e11400640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3df4067160 0x7f3df4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.009+0000 7f3e11400640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3df4067160 0x7f3df4069620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e04007fd0 tx=0x7f3e04007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:09.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.013+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e0c018d40 con 0x7f3e140d09c0 2024-04-25T10:18:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.289+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7920075190 con 0x7f79200cca50 2024-04-25T10:18:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.289+0000 7f7916a00640 1 -- 172.21.15.45:0/2900332636 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79180591a0 con 0x7f79200cca50 2024-04-25T10:18:09.292 INFO:teuthology.orchestra.run.smithi045.stdout:64424509445 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7900066fb0 msgr2=0x7f7900069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7900066fb0 0x7f7900069470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f79100af000 tx=0x7f7910004740 comp rx=0 tx=0).stop 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 msgr2=0x7f7920203350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 0x7f7920203350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7918001a60 tx=0x7f7918001f20 comp rx=0 tx=0).stop 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 shutdown_connections 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7900066fb0 0x7f7900069470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7920152e50 0x7f79200c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7920159e70 0x7f7920152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 --2- 172.21.15.45:0/2900332636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79200cca50 0x7f7920203350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 >> 172.21.15.45:0/2900332636 conn(0x7f79200b1330 msgr2=0x7f792006d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.293+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 shutdown_connections 2024-04-25T10:18:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.301+0000 7f7925600640 1 -- 172.21.15.45:0/2900332636 wait complete. 2024-04-25T10:18:09.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.325+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e8815c000 con 0x7f4e8806d3d0 2024-04-25T10:18:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.325+0000 7f4e7e000640 1 -- 172.21.15.45:0/620553955 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e78005480 con 0x7f4e8806d3d0 2024-04-25T10:18:09.332 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4e68067360 msgr2=0x7f4e68069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4e68067360 0x7f4e68069820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e880cec40 tx=0x7f4e74008040 comp rx=0 tx=0).stop 2024-04-25T10:18:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8806d3d0 msgr2=0x7f4e880ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8806d3d0 0x7f4e880ccef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e78000f70 tx=0x7f4e7800c890 comp rx=0 tx=0).stop 2024-04-25T10:18:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 shutdown_connections 2024-04-25T10:18:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4e68067360 0x7f4e68069820 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e880b2fc0 0x7f4e88153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e88071e30 0x7f4e88152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 --2- 172.21.15.45:0/620553955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e8806d3d0 0x7f4e880ccef0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.329+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 >> 172.21.15.45:0/620553955 conn(0x7f4e88066b70 msgr2=0x7f4e8814d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.333+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 shutdown_connections 2024-04-25T10:18:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.333+0000 7f4e8cc00640 1 -- 172.21.15.45:0/620553955 wait complete. 2024-04-25T10:18:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.333+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f46e40b5af0 con 0x7f46c8004810 2024-04-25T10:18:09.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-04-25T10:18:09.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.333+0000 7f46dbe00640 1 -- 172.21.15.45:0/2807959290 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f46d4056070 con 0x7f46c8004810 2024-04-25T10:18:09.337 INFO:teuthology.orchestra.run.smithi045.stdout:64424509445 2024-04-25T10:18:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f822400a6c0 con 0x7f8224033d00 2024-04-25T10:18:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46c4067030 msgr2=0x7f46c40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46c4067030 0x7f46c40694f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46d00af000 tx=0x7f46d00046d0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 msgr2=0x7f46e40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 0x7f46e40b4eb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46d400b6c0 tx=0x7f46d400bb80 comp rx=0 tx=0).stop 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 shutdown_connections 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f46c4067030 0x7f46c40694f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e414fad0 0x7f46e4150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e40b5440 0x7f46e414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 --2- 172.21.15.45:0/2807959290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46c8004810 0x7f46e40b4eb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 >> 172.21.15.45:0/2807959290 conn(0x7f46e40cfb70 msgr2=0x7f46e406cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 shutdown_connections 2024-04-25T10:18:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f821fe00640 1 -- 172.21.15.45:0/2420669299 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f821800f2e0 con 0x7f8224033d00 2024-04-25T10:18:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.337+0000 7f46eae00640 1 -- 172.21.15.45:0/2807959290 wait complete. 2024-04-25T10:18:09.341 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-04-25T10:18:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2d1414ea30 con 0x7f2d1415df00 2024-04-25T10:18:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f820c067030 msgr2=0x7f820c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f820c067030 0x7f820c0694f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82200b2000 tx=0x7f82200a9070 comp rx=0 tx=0).stop 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 msgr2=0x7f8224036ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 0x7f8224036ca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f821800b410 tx=0x7f821800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 shutdown_connections 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f820c067030 0x7f820c0694f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f2d0a000640 1 -- 172.21.15.45:0/2459487773 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d040f05f0 con 0x7f2d1415df00 2024-04-25T10:18:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82240df2b0 0x7f822404a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8224033d00 0x7f8224036ca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 --2- 172.21.15.45:0/2420669299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8224007550 0x7f82241808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 >> 172.21.15.45:0/2420669299 conn(0x7f8224009fc0 msgr2=0x7f82240e2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 shutdown_connections 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.341+0000 7f8231a00640 1 -- 172.21.15.45:0/2420669299 wait complete. 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:09.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f593c005590 con 0x7f596415d7d0 2024-04-25T10:18:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f5960c00640 1 -- 172.21.15.45:0/1331593566 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5954016090 con 0x7f596415d7d0 2024-04-25T10:18:09.348 INFO:teuthology.orchestra.run.smithi045.stdout:64424509445 2024-04-25T10:18:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cf4067030 msgr2=0x7f2cf40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cf4067030 0x7f2cf40694f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d1414c050 tx=0x7f2d0c009590 comp rx=0 tx=0).stop 2024-04-25T10:18:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 msgr2=0x7f2d141fa850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 0x7f2d141fa850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d040ac040 tx=0x7f2d0401ad30 comp rx=0 tx=0).stop 2024-04-25T10:18:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f657806d960 con 0x7f657815bc40 2024-04-25T10:18:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f6572000640 1 -- 172.21.15.45:0/1251816134 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65740164a0 con 0x7f657815bc40 2024-04-25T10:18:09.350 INFO:teuthology.orchestra.run.smithi045.stdout:64424509445 2024-04-25T10:18:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 shutdown_connections 2024-04-25T10:18:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cf4067030 0x7f2cf40694f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d1415df00 0x7f2d141fa850 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d140d1ce0 0x7f2d140c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 --2- 172.21.15.45:0/2459487773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d140d0920 0x7f2d14150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.345+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 >> 172.21.15.45:0/2459487773 conn(0x7f2d1406b290 msgr2=0x7f2d14158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 shutdown_connections 2024-04-25T10:18:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f2d19200640 1 -- 172.21.15.45:0/2459487773 wait complete. 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5944066fe0 msgr2=0x7f59440694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5944066fe0 0x7f59440694a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f595c0073a0 tx=0x7f595c015040 comp rx=0 tx=0).stop 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 msgr2=0x7f5964166ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 0x7f5964166ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f595401c980 tx=0x7f5954019240 comp rx=0 tx=0).stop 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 shutdown_connections 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5944066fe0 0x7f59440694a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596415d7d0 0x7f5964166ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59640b3180 0x7f5964158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 --2- 172.21.15.45:0/1331593566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406e400 0x7f59641392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 >> 172.21.15.45:0/1331593566 conn(0x7f5964066b70 msgr2=0x7f59641683f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 shutdown_connections 2024-04-25T10:18:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f596b800640 1 -- 172.21.15.45:0/1331593566 wait complete. 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f655c066fb0 msgr2=0x7f655c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f655c066fb0 0x7f655c069470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f656c001390 tx=0x7f656c000ff0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 msgr2=0x7f6578149e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 0x7f6578149e30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f657401aeb0 tx=0x7f65740195a0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 shutdown_connections 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f655c066fb0 0x7f655c069470 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f656c001390 tx=0x7f656c000ff0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f657815bc40 0x7f6578149e30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65780cf930 0x7f65781498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 --2- 172.21.15.45:0/1251816134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65780cda10 0x7f65781341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.349+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 >> 172.21.15.45:0/1251816134 conn(0x7f6578066b70 msgr2=0x7f6578159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.353+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 shutdown_connections 2024-04-25T10:18:09.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.353+0000 7f6580800640 1 -- 172.21.15.45:0/1251816134 wait complete. 2024-04-25T10:18:09.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.353+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3e141601d0 con 0x7f3e140d09c0 2024-04-25T10:18:09.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e0a000640 1 -- 172.21.15.45:0/3652897779 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e0c0f3a10 con 0x7f3e140d09c0 2024-04-25T10:18:09.358 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3df4067160 msgr2=0x7f3df4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3df4067160 0x7f3df4069620 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e04007fd0 tx=0x7f3e04007d60 comp rx=0 tx=0).stop 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e140d09c0 msgr2=0x7f3e140b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e140d09c0 0x7f3e140b8810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ae000 tx=0x7f3e0c01bf80 comp rx=0 tx=0).stop 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 shutdown_connections 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3df4067160 0x7f3df4069620 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e14162290 0x7f3e14070ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e140d27b0 0x7f3e14070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 --2- 172.21.15.45:0/3652897779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e140d09c0 0x7f3e140b8810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 >> 172.21.15.45:0/3652897779 conn(0x7f3e1406b290 msgr2=0x7f3e140b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 shutdown_connections 2024-04-25T10:18:09.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:09.357+0000 7f3e19000640 1 -- 172.21.15.45:0/3652897779 wait complete. 2024-04-25T10:18:09.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-04-25T10:18:09.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-04-25T10:18:09.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.4 2024-04-25T10:18:09.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-04-25T10:18:09.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-04-25T10:18:09.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.0 2024-04-25T10:18:09.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-04-25T10:18:10.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:18:10.375 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:18:10.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:18:10.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:18:10.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:18:10.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:18:10.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:18:10.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:18:10.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.533+0000 7f6e57e00640 1 Processor -- start 2024-04-25T10:18:10.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.533+0000 7f6e57e00640 1 -- start start 2024-04-25T10:18:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.533+0000 7f6e57e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e50159e70 0x7f6e500ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.533+0000 7f6e57e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e500ccc20 0x7f6e5006c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.533+0000 7f6e54a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6e50070e00 0x7f6e500cc510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32812/0 (socket says 172.21.15.45:32812) 2024-04-25T10:18:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.533+0000 7f6e54a00640 1 -- 172.21.15.45:0/4114143396 learned_addr learned my addr 172.21.15.45:0/4114143396 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e500caa50 con 0x7f6e50070e00 2024-04-25T10:18:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e500cad30 con 0x7f6e500ccc20 2024-04-25T10:18:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500cb010 con 0x7f6e50159e70 2024-04-25T10:18:10.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e55e00640 1 --2- 172.21.15.45:0/4114143396 >> v2:172.21.15.45:3300/0 conn(0x7f6e500ccc20 0x7f6e5006c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e55e00640 1 -- 172.21.15.45:0/4114143396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e50159e70 msgr2=0x7f6e500ca150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:18:10.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e55e00640 1 --2- 172.21.15.45:0/4114143396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e50159e70 0x7f6e500ca150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e55e00640 1 -- 172.21.15.45:0/4114143396 >> v1:172.21.15.117:6789/0 conn(0x7f6e50070e00 legacy=0x7f6e500cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.537+0000 7f6e55e00640 1 -- 172.21.15.45:0/4114143396 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500cb2f0 con 0x7f6e500ccc20 2024-04-25T10:18:10.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.541+0000 7f6e55e00640 1 --2- 172.21.15.45:0/4114143396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e500ccc20 0x7f6e5006c180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800bfa0 tx=0x7f6e4801eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ba4becfeb4f0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e4fe00640 1 -- 172.21.15.45:0/4114143396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480ad070 con 0x7f6e500ccc20 2024-04-25T10:18:10.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e4fe00640 1 -- 172.21.15.45:0/4114143396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e480a7c50 con 0x7f6e500ccc20 2024-04-25T10:18:10.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e4fe00640 1 -- 172.21.15.45:0/4114143396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e480b1400 con 0x7f6e500ccc20 2024-04-25T10:18:10.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e500ccc20 msgr2=0x7f6e5006c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4114143396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e500ccc20 0x7f6e5006c180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800bfa0 tx=0x7f6e4801eb80 comp rx=0 tx=0).stop 2024-04-25T10:18:10.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 shutdown_connections 2024-04-25T10:18:10.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4114143396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e500ccc20 0x7f6e5006c180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4114143396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e50159e70 0x7f6e500ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 >> 172.21.15.45:0/4114143396 conn(0x7f6e500b1330 msgr2=0x7f6e500b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 shutdown_connections 2024-04-25T10:18:10.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.545+0000 7f6e57e00640 1 -- 172.21.15.45:0/4114143396 wait complete. 2024-04-25T10:18:10.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.549+0000 7f6e57e00640 1 Processor -- start 2024-04-25T10:18:10.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.549+0000 7f6e57e00640 1 -- start start 2024-04-25T10:18:10.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 0x7f6e500c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e500ccc20 0x7f6e500c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50159e70 0x7f6e5014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500cbc70 con 0x7f6e50159e70 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 0x7f6e500c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e54a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e500ccc20 0x7f6e500c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 0x7f6e500c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51888/0 (socket says 172.21.15.45:51888) 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 -- 172.21.15.45:0/4103675 learned_addr learned my addr 172.21.15.45:0/4103675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50159e70 0x7f6e5014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e500cbaf0 con 0x7f6e50070e00 2024-04-25T10:18:10.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e50166a00 con 0x7f6e500ccc20 2024-04-25T10:18:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 -- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e500ccc20 msgr2=0x7f6e500c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e500ccc20 0x7f6e500c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 -- 172.21.15.45:0/4103675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50159e70 msgr2=0x7f6e5014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50159e70 0x7f6e5014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500c6000 con 0x7f6e50070e00 2024-04-25T10:18:10.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e54a00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e500ccc20 0x7f6e500c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e55400640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 0x7f6e500c8e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400b3e0 tx=0x7f6e4400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015020 con 0x7f6e50070e00 2024-04-25T10:18:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e50157190 con 0x7f6e50070e00 2024-04-25T10:18:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e50157750 con 0x7f6e50070e00 2024-04-25T10:18:10.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.553+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44011830 con 0x7f6e50070e00 2024-04-25T10:18:10.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.557+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44015020 con 0x7f6e50070e00 2024-04-25T10:18:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.561+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e44018020 con 0x7f6e50070e00 2024-04-25T10:18:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.561+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c0052a0 con 0x7f6e50070e00 2024-04-25T10:18:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.561+0000 7f6e4d600640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e30067360 0x7f6e30069820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.561+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e44011630 con 0x7f6e50070e00 2024-04-25T10:18:10.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.561+0000 7f6e54a00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e30067360 0x7f6e30069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.561+0000 7f6e54a00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e30067360 0x7f6e30069820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e500c3780 tx=0x7f6e3c005f20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.569+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e4401f020 con 0x7f6e50070e00 2024-04-25T10:18:10.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.605+0000 7f2db6400640 1 Processor -- start 2024-04-25T10:18:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.605+0000 7f2db6400640 1 -- start start 2024-04-25T10:18:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.605+0000 7f2db6400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db00cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.605+0000 7f2db6400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00b4470 con 0x7f2db0070310 2024-04-25T10:18:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.605+0000 7f2db6400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00b4710 con 0x7f2db00cf930 2024-04-25T10:18:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.605+0000 7f2db6400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00cf4d0 con 0x7f2db006ca50 2024-04-25T10:18:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2daf400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2db00cf930 0x7f2db00cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32816/0 (socket says 172.21.15.45:32816) 2024-04-25T10:18:10.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2daf400640 1 -- 172.21.15.45:0/1375920102 learned_addr learned my addr 172.21.15.45:0/1375920102 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:10.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2daea00640 1 --2- 172.21.15.45:0/1375920102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db00cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2dae000640 1 -- 172.21.15.45:0/1375920102 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174021602 0 0) 0x7f2db00b4710 con 0x7f2db00cf930 2024-04-25T10:18:10.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2dae000640 1 -- 172.21.15.45:0/1375920102 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d94003530 con 0x7f2db00cf930 2024-04-25T10:18:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2daea00640 1 -- 172.21.15.45:0/1375920102 >> v1:172.21.15.117:6789/0 conn(0x7f2db00cf930 legacy=0x7f2db00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2daea00640 1 -- 172.21.15.45:0/1375920102 >> v1:172.21.15.45:6789/0 conn(0x7f2db0070310 legacy=0x7f2db00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.609+0000 7f2daea00640 1 -- 172.21.15.45:0/1375920102 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00b49f0 con 0x7f2db006ca50 2024-04-25T10:18:10.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.613+0000 7f3173600640 1 Processor -- start 2024-04-25T10:18:10.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.613+0000 7f3173600640 1 -- start start 2024-04-25T10:18:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f2daea00640 1 --2- 172.21.15.45:0/1375920102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db00cef90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009820 tx=0x7f2da001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a03b97ecd2723e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3173600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f316c0cf930 0x7f316c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 Processor -- start 2024-04-25T10:18:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3173600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0b4470 con 0x7f316c0cef90 2024-04-25T10:18:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3173600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0b4710 con 0x7f316c070310 2024-04-25T10:18:10.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3173600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c0b4cd0 con 0x7f316c0cf930 2024-04-25T10:18:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f2dae000640 1 -- 172.21.15.45:0/1375920102 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da00b0070 con 0x7f2db006ca50 2024-04-25T10:18:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f2dae000640 1 -- 172.21.15.45:0/1375920102 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da00ab600 con 0x7f2db006ca50 2024-04-25T10:18:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f2dae000640 1 -- 172.21.15.45:0/1375920102 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da00ab9d0 con 0x7f2db006ca50 2024-04-25T10:18:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3170c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f316c0cf930 0x7f316c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3170c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f316c0cf930 0x7f316c06ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51896/0 (socket says 172.21.15.45:51896) 2024-04-25T10:18:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3170c00640 1 -- 172.21.15.45:0/3790314038 learned_addr learned my addr 172.21.15.45:0/3790314038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 -- start start 2024-04-25T10:18:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4528162450 0x7f452806c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f2db6400640 1 -- 172.21.15.45:0/1375920102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 msgr2=0x7f2db00cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f2db6400640 1 --2- 172.21.15.45:0/1375920102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db00cef90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009820 tx=0x7f2da001c410 comp rx=0 tx=0).stop 2024-04-25T10:18:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4528162f40 0x7f4528070110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280721b0 con 0x7f45280b3050 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45280b34e0 con 0x7f4528162450 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4527200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45280b2570 con 0x7f4528162f40 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4525400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f45280b3050 0x7f4528162830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32826/0 (socket says 172.21.15.45:32826) 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f4525400640 1 -- 172.21.15.45:0/2961771577 learned_addr learned my addr 172.21.15.45:0/2961771577 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f4525e00640 1 --2- 172.21.15.45:0/2961771577 >> v2:172.21.15.45:3300/0 conn(0x7f4528162450 0x7f452806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f4526800640 1 --2- 172.21.15.45:0/2961771577 >> v2:172.21.15.117:3300/0 conn(0x7f4528162f40 0x7f4528070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 -- 172.21.15.45:0/1375920102 shutdown_connections 2024-04-25T10:18:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 --2- 172.21.15.45:0/1375920102 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db00cef90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 -- 172.21.15.45:0/1375920102 >> 172.21.15.45:0/1375920102 conn(0x7f2db0066b70 msgr2=0x7f2db0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 -- 172.21.15.45:0/1375920102 shutdown_connections 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 -- 172.21.15.45:0/1375920102 wait complete. 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 Processor -- start 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f2db6400640 1 -- start start 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3170c00640 1 -- 172.21.15.45:0/3790314038 >> v1:172.21.15.117:6789/0 conn(0x7f316c070310 legacy=0x7f316c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3170c00640 1 -- 172.21.15.45:0/3790314038 >> v1:172.21.15.45:6789/0 conn(0x7f316c0cef90 legacy=0x7f316c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.617+0000 7f3170c00640 1 -- 172.21.15.45:0/3790314038 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0b49f0 con 0x7f316c0cf930 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f4525e00640 1 -- 172.21.15.45:0/2961771577 >> v2:172.21.15.117:3300/0 conn(0x7f4528162f40 msgr2=0x7f4528070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f4525e00640 1 --2- 172.21.15.45:0/2961771577 >> v2:172.21.15.117:3300/0 conn(0x7f4528162f40 0x7f4528070110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f4525e00640 1 -- 172.21.15.45:0/2961771577 >> v1:172.21.15.117:6789/0 conn(0x7f45280b3050 legacy=0x7f4528162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.621+0000 7f4525e00640 1 -- 172.21.15.45:0/2961771577 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280b2850 con 0x7f4528162450 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2db6400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db013d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f3170c00640 1 --2- 172.21.15.45:0/3790314038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c06ca50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3164009e50 tx=0x7f316401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f706e3506e53b64b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2db6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 0x7f2db015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2db6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db00cf930 0x7f2db013f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2daf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db013d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f316b400640 1 -- 172.21.15.45:0/3790314038 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640a5770 con 0x7f316c0cf930 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f316b400640 1 -- 172.21.15.45:0/3790314038 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31640a5910 con 0x7f316c0cf930 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2daf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db013d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54012/0 (socket says 172.21.15.45:54012) 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2daea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 0x7f2db015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2daea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 0x7f2db015edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51916/0 (socket says 172.21.15.45:51916) 2024-04-25T10:18:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f4525e00640 1 --2- 172.21.15.45:0/2961771577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162450 0x7f452806c140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f452815ff70 tx=0x7f451401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff13163effea82fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2daf400640 1 -- 172.21.15.45:0/3023353047 learned_addr learned my addr 172.21.15.45:0/3023353047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db0072050 con 0x7f2db00cf930 2024-04-25T10:18:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f3173600640 1 -- 172.21.15.45:0/3790314038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 msgr2=0x7f316c06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f316b400640 1 -- 172.21.15.45:0/3790314038 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640a5770 con 0x7f316c0cf930 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f3173600640 1 --2- 172.21.15.45:0/3790314038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c06ca50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3164009e50 tx=0x7f316401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db006ced0 con 0x7f2db0070310 2024-04-25T10:18:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b2e30 con 0x7f2db006ca50 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f3173600640 1 -- 172.21.15.45:0/3790314038 shutdown_connections 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f3173600640 1 --2- 172.21.15.45:0/3790314038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c06ca50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f3173600640 1 -- 172.21.15.45:0/3790314038 >> 172.21.15.45:0/3790314038 conn(0x7f316c066b70 msgr2=0x7f316c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f3173600640 1 -- 172.21.15.45:0/3790314038 shutdown_connections 2024-04-25T10:18:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f3173600640 1 -- 172.21.15.45:0/3790314038 wait complete. 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.625+0000 7f2dafe00640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db00cf930 0x7f2db013f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daf400640 1 -- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 msgr2=0x7f2db015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daf400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 0x7f2db015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daf400640 1 -- 172.21.15.45:0/3023353047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db00cf930 msgr2=0x7f2db013f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daf400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db00cf930 0x7f2db013f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f4524a00640 1 -- 172.21.15.45:0/2961771577 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45140b0070 con 0x7f4528162450 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f4524a00640 1 -- 172.21.15.45:0/2961771577 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45140ab600 con 0x7f4528162450 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f4524a00640 1 -- 172.21.15.45:0/2961771577 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45140ab9d0 con 0x7f4528162450 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daf400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db013daf0 con 0x7f2db006ca50 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daea00640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 0x7f2db015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2daf400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db013d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c4d0 tx=0x7f2d9800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98017070 con 0x7f2db006ca50 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db013ffd0 con 0x7f2db006ca50 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db0160fc0 con 0x7f2db006ca50 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98005cf0 con 0x7f2db006ca50 2024-04-25T10:18:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98007270 con 0x7f2db006ca50 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f4527200640 1 -- 172.21.15.45:0/2961771577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162450 msgr2=0x7f452806c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.629+0000 7f4527200640 1 --2- 172.21.15.45:0/2961771577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162450 0x7f452806c140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f452815ff70 tx=0x7f451401c460 comp rx=0 tx=0).stop 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 Processor -- start 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 -- start start 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f4527200640 1 -- 172.21.15.45:0/2961771577 shutdown_connections 2024-04-25T10:18:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d9801a070 con 0x7f2db006ca50 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f4527200640 1 --2- 172.21.15.45:0/2961771577 >> v2:172.21.15.117:3300/0 conn(0x7f4528162f40 0x7f4528070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f4527200640 1 --2- 172.21.15.45:0/2961771577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162450 0x7f452806c140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f4527200640 1 -- 172.21.15.45:0/2961771577 >> 172.21.15.45:0/2961771577 conn(0x7f45280669a0 msgr2=0x7f4528066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 0x7f316c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f316c0cef90 0x7f316c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c071fc0 con 0x7f316c070310 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c071e40 con 0x7f316c0cf930 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3173600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0b2e30 con 0x7f316c0cef90 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3170c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 0x7f316c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3170c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 0x7f316c0af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57436/0 (socket says 172.21.15.45:57436) 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f2da7400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d8c066f60 0x7f2d8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d98084300 con 0x7f2db006ca50 2024-04-25T10:18:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3170c00640 1 -- 172.21.15.45:0/3418125829 learned_addr learned my addr 172.21.15.45:0/3418125829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f3171600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f2daea00640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d8c066f60 0x7f2d8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.633+0000 7f316be00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f316c0cef90 0x7f316c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3170c00640 1 -- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f316c0cef90 msgr2=0x7f316c14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f4527200640 1 -- 172.21.15.45:0/2961771577 shutdown_connections 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f2daea00640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d8c066f60 0x7f2d8c069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2da00ad000 tx=0x7f2da0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3170c00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f316c0cef90 0x7f316c14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d880052a0 con 0x7f2db006ca50 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3170c00640 1 -- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 msgr2=0x7f316c14edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3170c00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c14edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3170c00640 1 -- 172.21.15.45:0/3418125829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c157a20 con 0x7f316c070310 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f316be00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f316c0cef90 0x7f316c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3171600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c14edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3170c00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 0x7f316c0af060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f31640195b0 tx=0x7f3164019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640bc020 con 0x7f316c070310 2024-04-25T10:18:10.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c157d00 con 0x7f316c070310 2024-04-25T10:18:10.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3164004040 con 0x7f316c070310 2024-04-25T10:18:10.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f4527200640 1 -- 172.21.15.45:0/2961771577 wait complete. 2024-04-25T10:18:10.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f4527200640 1 Processor -- start 2024-04-25T10:18:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31640043f0 con 0x7f316c070310 2024-04-25T10:18:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.637+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c1582c0 con 0x7f316c070310 2024-04-25T10:18:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.641+0000 7f4527200640 1 -- start start 2024-04-25T10:18:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.641+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c04ef30 con 0x7f316c070310 2024-04-25T10:18:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.641+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31640bf020 con 0x7f316c070310 2024-04-25T10:18:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.641+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d98054cb0 con 0x7f2db006ca50 2024-04-25T10:18:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.641+0000 7f3168c00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f314c066f60 0x7f314c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.641+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3164120420 con 0x7f316c070310 2024-04-25T10:18:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45280b3050 0x7f45281549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 0x7f4528204e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162f40 0x7f45282053d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f316be00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f314c066f60 0x7f314c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 0x7f4528204e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 0x7f4528204e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54024/0 (socket says 172.21.15.45:54024) 2024-04-25T10:18:10.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 -- 172.21.15.45:0/897518053 learned_addr learned my addr 172.21.15.45:0/897518053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452816ec00 con 0x7f4528162f40 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452816ea80 con 0x7f45280b3050 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452816ed80 con 0x7f4528162450 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4526800640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162f40 0x7f45282053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 -- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45280b3050 msgr2=0x7f45281549c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45280b3050 0x7f45281549c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 -- 172.21.15.45:0/897518053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162f40 msgr2=0x7f45282053d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162f40 0x7f45282053d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280b4480 con 0x7f4528162450 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4525400640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 0x7f4528204e90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451800c4a0 tx=0x7f451800c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518017070 con 0x7f4528162450 2024-04-25T10:18:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518005d10 con 0x7f4528162450 2024-04-25T10:18:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45180072b0 con 0x7f4528162450 2024-04-25T10:18:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45280cc070 con 0x7f4528162450 2024-04-25T10:18:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.645+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45280cc330 con 0x7f4528162450 2024-04-25T10:18:10.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.649+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4528075120 con 0x7f4528162450 2024-04-25T10:18:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.649+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f451801a070 con 0x7f4528162450 2024-04-25T10:18:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.653+0000 7f316be00640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f314c066f60 0x7f314c069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f316c15dc30 tx=0x7f315c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.653+0000 7f451e000640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4508067030 0x7f45080694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.653+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31640c6030 con 0x7f316c070310 2024-04-25T10:18:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.653+0000 7f4525e00640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4508067030 0x7f45080694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.657+0000 7f4525e00640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4508067030 0x7f45080694f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f45140ad000 tx=0x7f45140a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.657+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45180844a0 con 0x7f4528162450 2024-04-25T10:18:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.657+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45180848c0 con 0x7f4528162450 2024-04-25T10:18:10.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.681+0000 7fb0a9e00640 1 Processor -- start 2024-04-25T10:18:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.681+0000 7fb0a9e00640 1 -- start start 2024-04-25T10:18:10.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.681+0000 7fb0a9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0a4161dc0 0x7fb0a40b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a9e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a40b59f0 con 0x7fb0a4166780 2024-04-25T10:18:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a9e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a4165ca0 con 0x7fb0a41621a0 2024-04-25T10:18:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a9e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a40b5cd0 con 0x7fb0a4161dc0 2024-04-25T10:18:10.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a2000640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb0a41621a0 0x7fb0a4162f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32842/0 (socket says 172.21.15.45:32842) 2024-04-25T10:18:10.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a2000640 1 -- 172.21.15.45:0/3486084480 learned_addr learned my addr 172.21.15.45:0/3486084480 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:10.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a3400640 1 --1- 172.21.15.45:0/3486084480 >> v1:172.21.15.45:6789/0 conn(0x7fb0a4166780 0x7fb0a40b3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59308/0 (socket says 172.21.15.45:59308) 2024-04-25T10:18:10.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a2a00640 1 --2- 172.21.15.45:0/3486084480 >> v2:172.21.15.45:3300/0 conn(0x7fb0a4161dc0 0x7fb0a40b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.685+0000 7fb0a2a00640 1 -- 172.21.15.45:0/3486084480 >> v1:172.21.15.117:6789/0 conn(0x7fb0a41621a0 legacy=0x7fb0a4162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.689+0000 7fb0a2a00640 1 -- 172.21.15.45:0/3486084480 >> v1:172.21.15.45:6789/0 conn(0x7fb0a4166780 legacy=0x7fb0a40b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.689+0000 7fb0a2a00640 1 -- 172.21.15.45:0/3486084480 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4165f80 con 0x7fb0a4161dc0 2024-04-25T10:18:10.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.693+0000 7fb0a2a00640 1 --2- 172.21.15.45:0/3486084480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4161dc0 0x7fb0a40b5350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097c0 tx=0x7fb09001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cdd4154f09c5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1be00640 1 Processor -- start 2024-04-25T10:18:10.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1be00640 1 -- start start 2024-04-25T10:18:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a1600640 1 -- 172.21.15.45:0/3486084480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900aa5f0 con 0x7fb0a4161dc0 2024-04-25T10:18:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a1600640 1 -- 172.21.15.45:0/3486084480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0900af070 con 0x7fb0a4161dc0 2024-04-25T10:18:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a1600640 1 -- 172.21.15.45:0/3486084480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900aa9b0 con 0x7fb0a4161dc0 2024-04-25T10:18:10.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb634000640 1 Processor -- start 2024-04-25T10:18:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 -- 172.21.15.45:0/3486084480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4161dc0 msgr2=0x7fb0a40b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/3486084480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4161dc0 0x7fb0a40b5350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097c0 tx=0x7fb09001c490 comp rx=0 tx=0).stop 2024-04-25T10:18:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06ac90 0x7f7a1c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1be00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1c0b5780 con 0x7f7a1c15db40 2024-04-25T10:18:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1c0b5a60 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c06b0d0 con 0x7f7a1c06ac90 2024-04-25T10:18:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 -- 172.21.15.45:0/3486084480 shutdown_connections 2024-04-25T10:18:10.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/3486084480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4161dc0 0x7fb0a40b5350 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 -- 172.21.15.45:0/3486084480 >> 172.21.15.45:0/3486084480 conn(0x7fb0a40cfab0 msgr2=0x7fb0a40cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 -- 172.21.15.45:0/3486084480 shutdown_connections 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06ac90 0x7f7a1c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1b400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f7a1c06b2c0 0x7f7a1c0b6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32856/0 (socket says 172.21.15.45:32856) 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1b400640 1 -- 172.21.15.45:0/1061359529 learned_addr learned my addr 172.21.15.45:0/1061359529 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7f7a1aa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7a1c15db40 0x7f7a1c15df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59312/0 (socket says 172.21.15.45:59312) 2024-04-25T10:18:10.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7f7a19600640 1 -- 172.21.15.45:0/1061359529 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062422969 0 0) 0x7f7a1c0b5a60 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7f7a19600640 1 -- 172.21.15.45:0/1061359529 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a00003530 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb634000640 1 -- start start 2024-04-25T10:18:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7f7a1a000640 1 -- 172.21.15.45:0/1061359529 >> v1:172.21.15.117:6789/0 conn(0x7f7a1c06b2c0 legacy=0x7f7a1c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7f7a1a000640 1 -- 172.21.15.45:0/1061359529 >> v1:172.21.15.45:6789/0 conn(0x7f7a1c15db40 legacy=0x7f7a1c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7f7a1a000640 1 -- 172.21.15.45:0/1061359529 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0b5d40 con 0x7f7a1c06ac90 2024-04-25T10:18:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.697+0000 7fb0a9e00640 1 -- 172.21.15.45:0/3486084480 wait complete. 2024-04-25T10:18:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb0a9e00640 1 Processor -- start 2024-04-25T10:18:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb0a9e00640 1 -- start start 2024-04-25T10:18:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb634000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb62c0cf930 0x7fb62c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb634000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c070310 0x7fb62c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb631600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb62c0cf930 0x7fb62c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb634000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0b4470 con 0x7fb62c0cef90 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb634000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c0b4cd0 con 0x7fb62c0cf930 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb634000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b4710 con 0x7fb62c070310 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb631600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb62c0cf930 0x7fb62c06ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51956/0 (socket says 172.21.15.45:51956) 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb631600640 1 -- 172.21.15.45:0/1667986213 learned_addr learned my addr 172.21.15.45:0/1667986213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb632000640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c070310 0x7fb62c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.701+0000 7fb62be00640 1 -- 172.21.15.45:0/1667986213 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491100208 0 0) 0x7fb62c0b4470 con 0x7fb62c0cef90 2024-04-25T10:18:10.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb62be00640 1 -- 172.21.15.45:0/1667986213 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb614003530 con 0x7fb62c0cef90 2024-04-25T10:18:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 0x7fb0a415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 0x7fb0a42036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7f7a1a000640 1 --2- 172.21.15.45:0/1061359529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06ac90 0x7f7a1c0b5010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a080097f0 tx=0x7f7a0801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8efaa1571504865a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 0x7fb0a4206510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 0x7fb0a415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 0x7fb0a415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54044/0 (socket says 172.21.15.45:54044) 2024-04-25T10:18:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 0x7fb0a42036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7f7a19600640 1 -- 172.21.15.45:0/1061359529 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080b5070 con 0x7f7a1c06ac90 2024-04-25T10:18:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7f7a19600640 1 -- 172.21.15.45:0/1061359529 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a080a8990 con 0x7f7a1c06ac90 2024-04-25T10:18:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7f7a19600640 1 -- 172.21.15.45:0/1061359529 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080a8d60 con 0x7f7a1c06ac90 2024-04-25T10:18:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 0x7fb0a42036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51960/0 (socket says 172.21.15.45:51960) 2024-04-25T10:18:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 0x7fb0a4206510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 0x7fb0a4206510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57464/0 (socket says 172.21.15.45:57464) 2024-04-25T10:18:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a2a00640 1 -- 172.21.15.45:0/2953702774 learned_addr learned my addr 172.21.15.45:0/2953702774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a406d210 con 0x7fb0a4166780 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb631600640 1 -- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c070310 msgr2=0x7fb62c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb631600640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c070310 0x7fb62c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb631600640 1 -- 172.21.15.45:0/1667986213 >> v1:172.21.15.117:6789/0 conn(0x7fb62c0cef90 legacy=0x7fb62c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb631600640 1 -- 172.21.15.45:0/1667986213 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0b49f0 con 0x7fb62c0cf930 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb632000640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c070310 0x7fb62c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a406d090 con 0x7fb0a41621a0 2024-04-25T10:18:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.705+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a406d390 con 0x7fb0a4161dc0 2024-04-25T10:18:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 -- 172.21.15.45:0/1061359529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06ac90 msgr2=0x7f7a1c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 --2- 172.21.15.45:0/1061359529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06ac90 0x7f7a1c0b5010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a080097f0 tx=0x7f7a0801ecb0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 Processor -- start 2024-04-25T10:18:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a3400640 1 -- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 msgr2=0x7fb0a415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a3400640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 0x7fb0a415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 -- start start 2024-04-25T10:18:10.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a3400640 1 -- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 msgr2=0x7fb0a42036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a3400640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 0x7fb0a42036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a3400640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a415c790 con 0x7fb0a4166780 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a2000640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 0x7fb0a42036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a3400640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 0x7fb0a4206510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c730 tx=0x7fb09c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 -- 172.21.15.45:0/1061359529 shutdown_connections 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 --2- 172.21.15.45:0/1061359529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06ac90 0x7f7a1c0b5010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 -- 172.21.15.45:0/1061359529 >> 172.21.15.45:0/1061359529 conn(0x7f7a1c0d17b0 msgr2=0x7f7a1c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 -- 172.21.15.45:0/1061359529 shutdown_connections 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c018070 con 0x7fb0a4166780 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c004d70 con 0x7fb0a4166780 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00a860 con 0x7fb0a4166780 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a2a00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 0x7fb0a415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a4159b70 con 0x7fb0a4166780 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a415a130 con 0x7fb0a4166780 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f7a1be00640 1 -- 172.21.15.45:0/1061359529 wait complete. 2024-04-25T10:18:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f7a1be00640 1 Processor -- start 2024-04-25T10:18:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cf930 0x7f50200cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50200ae0e0 0x7f50200b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50200d0250 con 0x7f5020070c20 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502015bb90 con 0x7f50200ae0e0 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5027200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502015f060 con 0x7f50200cf930 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f501f400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5020070c20 0x7f502015e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32864/0 (socket says 172.21.15.45:32864) 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f501f400640 1 -- 172.21.15.45:0/222656272 learned_addr learned my addr 172.21.15.45:0/222656272 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.709+0000 7f5025200640 1 --2- 172.21.15.45:0/222656272 >> v2:172.21.15.117:3300/0 conn(0x7f50200ae0e0 0x7f50200b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cf930 0x7f50200cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740551165 0 0) 0x7f50200d0250 con 0x7f5020070c20 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5004003530 con 0x7f5020070c20 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3912247312 0 0) 0x7f5004003530 con 0x7f5020070c20 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cf930 msgr2=0x7f50200cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cf930 0x7f50200cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 >> v2:172.21.15.117:3300/0 conn(0x7f50200ae0e0 msgr2=0x7f50200b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 --2- 172.21.15.45:0/222656272 >> v2:172.21.15.117:3300/0 conn(0x7f50200ae0e0 0x7f50200b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502015eaa0 con 0x7f5020070c20 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 <== mon.1 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f500c001680 con 0x7f5020070c20 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 >> v1:172.21.15.117:6789/0 conn(0x7f5020070c20 legacy=0x7f502015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50040039c0 0x7f5004003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5004004810 0x7f5004004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50040051d0 0x7f500400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502015bb90 con 0x7f50040051d0 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502015f060 con 0x7f50040039c0 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500400bf10 con 0x7f5004004810 2024-04-25T10:18:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f7a1be00640 1 -- start start 2024-04-25T10:18:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501f400640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5004004810 0x7f5004004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50040039c0 0x7f5004003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7fb631600640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c0cf930 0x7fb62c06ca50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb618009870 tx=0x7fb61801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc37c3cdb15475ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb09c01b070 con 0x7fb0a4166780 2024-04-25T10:18:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c06ac90 0x7f7a1c1f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 0x7f7a1c09d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a4073040 con 0x7fb0a4166780 2024-04-25T10:18:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 -- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5004004810 msgr2=0x7f5004004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5004004810 0x7f5004004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 -- 172.21.15.45:0/222656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50040051d0 msgr2=0x7f500400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50040051d0 0x7f500400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.713+0000 7f501fe00640 1 -- 172.21.15.45:0/222656272 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502015eaa0 con 0x7f50040039c0 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 0x7f7a1c09d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 0x7f7a1c09d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54068/0 (socket says 172.21.15.45:54068) 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c15db40 0x7f7a1c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c0b6300 con 0x7f7a1c15db40 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c06b750 con 0x7f7a1c06ac90 2024-04-25T10:18:10.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c06b8d0 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb62be00640 1 -- 172.21.15.45:0/1667986213 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6180b0070 con 0x7fb62c0cf930 2024-04-25T10:18:10.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c06ac90 0x7f7a1c1f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c06ac90 0x7f7a1c1f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51986/0 (socket says 172.21.15.45:51986) 2024-04-25T10:18:10.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1aa00640 1 -- 172.21.15.45:0/2403481726 learned_addr learned my addr 172.21.15.45:0/2403481726 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb62be00640 1 -- 172.21.15.45:0/1667986213 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180ab400 con 0x7fb62c0cf930 2024-04-25T10:18:10.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb62be00640 1 -- 172.21.15.45:0/1667986213 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620001680 con 0x7fb62c0cf930 2024-04-25T10:18:10.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 -- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c06ac90 msgr2=0x7f7a1c1f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c06ac90 0x7f7a1c1f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb634000640 1 -- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c0cf930 msgr2=0x7fb62c06ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb634000640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c0cf930 0x7fb62c06ca50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb618009870 tx=0x7fb61801c330 comp rx=0 tx=0).stop 2024-04-25T10:18:10.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb09b400640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb084066fb0 0x7fb084069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 -- 172.21.15.45:0/2403481726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c15db40 msgr2=0x7f7a1c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb09c084680 con 0x7fb0a4166780 2024-04-25T10:18:10.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c15db40 0x7f7a1c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 -- 172.21.15.45:0/2403481726 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0c9720 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a1a000640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 0x7f7a1c09d520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c071ea0 tx=0x7f7a0801ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb634000640 1 -- 172.21.15.45:0/1667986213 shutdown_connections 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb634000640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c070310 0x7fb62c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb634000640 1 --2- 172.21.15.45:0/1667986213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c0cf930 0x7fb62c06ca50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb634000640 1 -- 172.21.15.45:0/1667986213 >> 172.21.15.45:0/1667986213 conn(0x7fb62c066b70 msgr2=0x7fb62c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7fb0a2a00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb084066fb0 0x7fb084069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080bf070 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.717+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a080b0700 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080bf070 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1c0c9a00 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c0c9fc0 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7fb0a2a00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb084066fb0 0x7fb084069470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb090000c10 tx=0x7fb090008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f501fe00640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50040039c0 0x7f5004003de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50100099a0 tx=0x7f501001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54914af54b5ca8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7fb634000640 1 -- 172.21.15.45:0/1667986213 shutdown_connections 2024-04-25T10:18:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50100b2070 con 0x7f50040039c0 2024-04-25T10:18:10.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100a5520 con 0x7f50040039c0 2024-04-25T10:18:10.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f501ea00640 1 -- 172.21.15.45:0/222656272 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010009670 con 0x7f50040039c0 2024-04-25T10:18:10.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1c075120 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7fb634000640 1 -- 172.21.15.45:0/1667986213 wait complete. 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5027200640 1 -- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50040039c0 msgr2=0x7f5004003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5027200640 1 --2- 172.21.15.45:0/222656272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50040039c0 0x7f5004003de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50100099a0 tx=0x7f501001c950 comp rx=0 tx=0).stop 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5025200640 1 -- 172.21.15.45:0/222656272 reap_dead start 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5027200640 1 -- 172.21.15.45:0/222656272 shutdown_connections 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5027200640 1 -- 172.21.15.45:0/222656272 >> 172.21.15.45:0/222656272 conn(0x7f5020066b70 msgr2=0x7f5020066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5027200640 1 -- 172.21.15.45:0/222656272 shutdown_connections 2024-04-25T10:18:10.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f5027200640 1 -- 172.21.15.45:0/222656272 wait complete. 2024-04-25T10:18:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.721+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a080b08b0 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f7a12a00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79fc067160 0x7f79fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a081217a0 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f7a1aa00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79fc067160 0x7f79fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f5027200640 1 Processor -- start 2024-04-25T10:18:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7fb634000640 1 Processor -- start 2024-04-25T10:18:10.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f7a1aa00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79fc067160 0x7f79fc069620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400e250 tx=0x7f7a1400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f5027200640 1 -- start start 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f5027200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50201f7c80 0x7f50200ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.725+0000 7f5027200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 0x7f5020158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f5027200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201592d0 0x7f50200cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 0x7f5020158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 0x7f5020158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54070/0 (socket says 172.21.15.45:54070) 2024-04-25T10:18:10.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 -- 172.21.15.45:0/2835614227 learned_addr learned my addr 172.21.15.45:0/2835614227 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f5025200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201592d0 0x7f50200cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200adbc0 con 0x7f50201f7c80 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200ad8e0 con 0x7f50201592d0 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502006e0a0 con 0x7f50200cd030 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 -- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201592d0 msgr2=0x7f50200cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201592d0 0x7f50200cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 -- 172.21.15.45:0/2835614227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50201f7c80 msgr2=0x7f50200ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50201f7c80 0x7f50200ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020157cf0 con 0x7f50200cd030 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 -- start start 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 0x7fb62c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c0cef90 0x7fb62c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb62c0cf930 0x7fb62c162640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b2e30 con 0x7fb62c0cf930 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c072050 con 0x7fb62c070310 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb634000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b2fb0 con 0x7fb62c0cef90 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 0x7fb62c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 0x7fb62c157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51996/0 (socket says 172.21.15.45:51996) 2024-04-25T10:18:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 -- 172.21.15.45:0/2866214433 learned_addr learned my addr 172.21.15.45:0/2866214433 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f501f400640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 0x7f5020158d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f500c0017f0 tx=0x7f500c004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb632000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb62c0cf930 0x7fb62c162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 -- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c0cef90 msgr2=0x7fb62c15a440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:18:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c0cef90 0x7fb62c15a440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 -- 172.21.15.45:0/2866214433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb62c0cf930 msgr2=0x7fb62c162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb62c0cf930 0x7fb62c162640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a08018810 con 0x7f7a1c06b2c0 2024-04-25T10:18:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7fb631600640 1 -- 172.21.15.45:0/2866214433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c1580b0 con 0x7fb62c070310 2024-04-25T10:18:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.729+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c01a070 con 0x7f50200cd030 2024-04-25T10:18:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7fb631600640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 0x7fb62c157b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb618019f60 tx=0x7fb6180190d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb09c055820 con 0x7fb0a4166780 2024-04-25T10:18:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c00fa20 con 0x7f50200cd030 2024-04-25T10:18:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5020157fd0 con 0x7f50200cd030 2024-04-25T10:18:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5020158590 con 0x7f50200cd030 2024-04-25T10:18:10.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c00fdf0 con 0x7f50200cd030 2024-04-25T10:18:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.733+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff80052a0 con 0x7f50200cd030 2024-04-25T10:18:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.737+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6180bb070 con 0x7fb62c070310 2024-04-25T10:18:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.737+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180b1040 con 0x7fb62c070310 2024-04-25T10:18:10.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.737+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6180bb070 con 0x7fb62c070310 2024-04-25T10:18:10.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.737+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c15c7b0 con 0x7fb62c070310 2024-04-25T10:18:10.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.737+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c15cd70 con 0x7fb62c070310 2024-04-25T10:18:10.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.741+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f500c01d070 con 0x7f50200cd030 2024-04-25T10:18:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.741+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6180b5600 con 0x7fb62c070310 2024-04-25T10:18:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c04ef30 con 0x7fb62c070310 2024-04-25T10:18:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7fb629600640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb60c067160 0x7fb60c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb61811e4b0 con 0x7fb62c070310 2024-04-25T10:18:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.741+0000 7f5017e00640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5000067030 0x7f50000694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:10.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f500c085010 con 0x7f50200cd030 2024-04-25T10:18:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f500c024020 con 0x7f50200cd030 2024-04-25T10:18:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7f501fe00640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5000067030 0x7f50000694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7f501fe00640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5000067030 0x7f50000694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50100af000 tx=0x7f5010008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7fb630c00640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb60c067160 0x7fb60c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:10.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.745+0000 7fb630c00640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb60c067160 0x7fb60c069620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb620001ec0 tx=0x7fb620001500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:10.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.749+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb618016070 con 0x7fb62c070310 2024-04-25T10:18:10.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.925+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6e2c005590 con 0x7f6e50070e00 2024-04-25T10:18:10.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.925+0000 7f6e4d600640 1 -- 172.21.15.45:0/4103675 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e440550e0 con 0x7f6e50070e00 2024-04-25T10:18:10.927 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e30067360 msgr2=0x7f6e30069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e30067360 0x7f6e30069820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e500c3780 tx=0x7f6e3c005f20 comp rx=0 tx=0).stop 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 msgr2=0x7f6e500c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 0x7f6e500c8e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400b3e0 tx=0x7f6e4400b8a0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 shutdown_connections 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e30067360 0x7f6e30069820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e50159e70 0x7f6e5014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e500ccc20 0x7f6e500c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 --2- 172.21.15.45:0/4103675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e50070e00 0x7f6e500c8e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 >> 172.21.15.45:0/4103675 conn(0x7f6e500b1330 msgr2=0x7f6e500b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 shutdown_connections 2024-04-25T10:18:10.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.929+0000 7f6e57e00640 1 -- 172.21.15.45:0/4103675 wait complete. 2024-04-25T10:18:10.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-04-25T10:18:10.965 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.989+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2d88005590 con 0x7f2db006ca50 2024-04-25T10:18:10.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.993+0000 7f2da7400640 1 -- 172.21.15.45:0/3023353047 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d98058d50 con 0x7f2db006ca50 2024-04-25T10:18:10.995 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:10.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.993+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f45281616e0 con 0x7f4528162450 2024-04-25T10:18:10.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.993+0000 7f451e000640 1 -- 172.21.15.45:0/897518053 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4518055550 con 0x7f4528162450 2024-04-25T10:18:10.997 INFO:teuthology.orchestra.run.smithi045.stdout:64424509447 2024-04-25T10:18:10.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d8c066f60 msgr2=0x7f2d8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d8c066f60 0x7f2d8c069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2da00ad000 tx=0x7f2da0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 msgr2=0x7f2db013d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db013d480 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c4d0 tx=0x7f2d9800c990 comp rx=0 tx=0).stop 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 shutdown_connections 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d8c066f60 0x7f2d8c069420 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2da00ad000 tx=0x7f2da0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db00cf930 0x7f2db013f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0070310 0x7f2db015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 --2- 172.21.15.45:0/3023353047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2db006ca50 0x7f2db013d480 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 >> 172.21.15.45:0/3023353047 conn(0x7f2db0066b70 msgr2=0x7f2db00cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 shutdown_connections 2024-04-25T10:18:10.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f2db6400640 1 -- 172.21.15.45:0/3023353047 wait complete. 2024-04-25T10:18:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4508067030 msgr2=0x7f45080694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f4527200640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4508067030 0x7f45080694f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f45140ad000 tx=0x7f45140a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 msgr2=0x7f4528204e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:10.997+0000 7f4527200640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 0x7f4528204e90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451800c4a0 tx=0x7f451800c960 comp rx=0 tx=0).stop 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 shutdown_connections 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4508067030 0x7f45080694f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4528162f40 0x7f45282053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4528162450 0x7f4528204e90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 --2- 172.21.15.45:0/897518053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45280b3050 0x7f45281549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 >> 172.21.15.45:0/897518053 conn(0x7f45280669a0 msgr2=0x7f4528155200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 shutdown_connections 2024-04-25T10:18:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.001+0000 7f4527200640 1 -- 172.21.15.45:0/897518053 wait complete. 2024-04-25T10:18:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.005+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f316c15de00 con 0x7f316c070310 2024-04-25T10:18:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.005+0000 7f3168c00640 1 -- 172.21.15.45:0/3418125829 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f31640c6030 con 0x7f316c070310 2024-04-25T10:18:11.009 INFO:teuthology.orchestra.run.smithi045.stdout:60129542150 2024-04-25T10:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f314c066f60 msgr2=0x7f314c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f314c066f60 0x7f314c069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f316c15dc30 tx=0x7f315c009040 comp rx=0 tx=0).stop 2024-04-25T10:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 msgr2=0x7f316c0af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 0x7f316c0af060 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f31640195b0 tx=0x7f3164019740 comp rx=0 tx=0).stop 2024-04-25T10:18:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 shutdown_connections 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f314c066f60 0x7f314c069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f316c0cf930 0x7f316c14edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f316c0cef90 0x7f316c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 --2- 172.21.15.45:0/3418125829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f316c070310 0x7f316c0af060 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 >> 172.21.15.45:0/3418125829 conn(0x7f316c066b70 msgr2=0x7f316c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 shutdown_connections 2024-04-25T10:18:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.009+0000 7f3173600640 1 -- 172.21.15.45:0/3418125829 wait complete. 2024-04-25T10:18:11.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-04-25T10:18:11.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-04-25T10:18:11.037 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.037 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-04-25T10:18:11.042 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.045+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb62c06b390 con 0x7fb62c070310 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.045+0000 7fb629600640 1 -- 172.21.15.45:0/2866214433 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6180ef390 con 0x7fb62c070310 2024-04-25T10:18:11.049 INFO:teuthology.orchestra.run.smithi045.stdout:64424509447 2024-04-25T10:18:11.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb60c067160 msgr2=0x7fb60c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb60c067160 0x7fb60c069620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb620001ec0 tx=0x7fb620001500 comp rx=0 tx=0).stop 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 msgr2=0x7fb62c157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 0x7fb62c157b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb618019f60 tx=0x7fb6180190d0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 shutdown_connections 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb60c067160 0x7fb60c069620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb62c0cf930 0x7fb62c162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb62c0cef90 0x7fb62c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 --2- 172.21.15.45:0/2866214433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c070310 0x7fb62c157b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 >> 172.21.15.45:0/2866214433 conn(0x7fb62c066b70 msgr2=0x7fb62c0cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 shutdown_connections 2024-04-25T10:18:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.049+0000 7fb634000640 1 -- 172.21.15.45:0/2866214433 wait complete. 2024-04-25T10:18:11.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb0a415a300 con 0x7fb0a4166780 2024-04-25T10:18:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb09b400640 1 -- 172.21.15.45:0/2953702774 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb09c055120 con 0x7fb0a4166780 2024-04-25T10:18:11.059 INFO:teuthology.orchestra.run.smithi045.stdout:64424509447 2024-04-25T10:18:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb084066fb0 msgr2=0x7fb084069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb084066fb0 0x7fb084069470 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb090000c10 tx=0x7fb090008040 comp rx=0 tx=0).stop 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 msgr2=0x7fb0a4206510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 0x7fb0a4206510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c730 tx=0x7fb09c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 shutdown_connections 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb084066fb0 0x7fb084069470 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0a4166780 0x7fb0a4206510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0a41621a0 0x7fb0a42036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 --2- 172.21.15.45:0/2953702774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0a4161dc0 0x7fb0a415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 >> 172.21.15.45:0/2953702774 conn(0x7fb0a40cfab0 msgr2=0x7fb0a40b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 shutdown_connections 2024-04-25T10:18:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.057+0000 7fb0a9e00640 1 -- 172.21.15.45:0/2953702774 wait complete. 2024-04-25T10:18:11.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-04-25T10:18:11.073 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-04-25T10:18:11.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.077+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7a1c06ec70 con 0x7f7a1c06b2c0 2024-04-25T10:18:11.082 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:11.082 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.077+0000 7f7a12a00640 1 -- 172.21.15.45:0/2403481726 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a080f20d0 con 0x7f7a1c06b2c0 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79fc067160 msgr2=0x7f79fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79fc067160 0x7f79fc069620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400e250 tx=0x7f7a1400b040 comp rx=0 tx=0).stop 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 msgr2=0x7f7a1c09d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 0x7f7a1c09d520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c071ea0 tx=0x7f7a0801ba50 comp rx=0 tx=0).stop 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 shutdown_connections 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79fc067160 0x7f79fc069620 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c15db40 0x7f7a1c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1c06b2c0 0x7f7a1c09d520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 --2- 172.21.15.45:0/2403481726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c06ac90 0x7f7a1c1f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 >> 172.21.15.45:0/2403481726 conn(0x7f7a1c0d17b0 msgr2=0x7f7a1c0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 shutdown_connections 2024-04-25T10:18:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.081+0000 7f7a1be00640 1 -- 172.21.15.45:0/2403481726 wait complete. 2024-04-25T10:18:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.089+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4ff8005860 con 0x7f50200cd030 2024-04-25T10:18:11.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.089+0000 7f5017e00640 1 -- 172.21.15.45:0/2835614227 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f500c055a40 con 0x7f50200cd030 2024-04-25T10:18:11.091 INFO:teuthology.orchestra.run.smithi045.stdout:64424509446 2024-04-25T10:18:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5000067030 msgr2=0x7f50000694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5000067030 0x7f50000694f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50100af000 tx=0x7f5010008040 comp rx=0 tx=0).stop 2024-04-25T10:18:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 msgr2=0x7f5020158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 0x7f5020158d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f500c0017f0 tx=0x7f500c004f90 comp rx=0 tx=0).stop 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 shutdown_connections 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5000067030 0x7f50000694f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50201592d0 0x7f50200cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50200cd030 0x7f5020158d90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 --2- 172.21.15.45:0/2835614227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50201f7c80 0x7f50200ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 >> 172.21.15.45:0/2835614227 conn(0x7f5020066b70 msgr2=0x7f50200b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 shutdown_connections 2024-04-25T10:18:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.093+0000 7f5027200640 1 -- 172.21.15.45:0/2835614227 wait complete. 2024-04-25T10:18:11.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-04-25T10:18:11.437 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-04-25T10:18:11.439 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:11.439 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:18:11.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:11.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.629+0000 7fdf31200640 1 Processor -- start 2024-04-25T10:18:11.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.629+0000 7fdf31200640 1 -- start start 2024-04-25T10:18:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf31200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf2c0cf930 0x7fdf2c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf31200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf2c070c20 0x7fdf2c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf31200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c0d0250 con 0x7fdf2c0ae020 2024-04-25T10:18:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf31200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c15bb90 con 0x7fdf2c0cf930 2024-04-25T10:18:11.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf31200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0ad840 con 0x7fdf2c070c20 2024-04-25T10:18:11.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf2c0cf930 0x7fdf2c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf2c0cf930 0x7fdf2c0cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57514/0 (socket says 172.21.15.45:57514) 2024-04-25T10:18:11.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 -- 172.21.15.45:0/570897104 learned_addr learned my addr 172.21.15.45:0/570897104 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:11.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf23e00640 1 --2- 172.21.15.45:0/570897104 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c070c20 0x7fdf2c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 -- 172.21.15.45:0/570897104 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c070c20 msgr2=0x7fdf2c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:11.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 --2- 172.21.15.45:0/570897104 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c070c20 0x7fdf2c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 -- 172.21.15.45:0/570897104 >> v1:172.21.15.117:6789/0 conn(0x7fdf2c0ae020 legacy=0x7fdf2c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf29e00640 1 -- 172.21.15.45:0/570897104 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c15e9f0 con 0x7fdf2c0cf930 2024-04-25T10:18:11.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.633+0000 7fdf23e00640 1 --2- 172.21.15.45:0/570897104 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c070c20 0x7fdf2c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:11.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf29e00640 1 --2- 172.21.15.45:0/570897104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c0cfd10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0d0630 tx=0x7fdf1c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdaced2ad172aea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf29400640 1 -- 172.21.15.45:0/570897104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0aa650 con 0x7fdf2c0cf930 2024-04-25T10:18:11.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf29400640 1 -- 172.21.15.45:0/570897104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0af070 con 0x7fdf2c0cf930 2024-04-25T10:18:11.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf29400640 1 -- 172.21.15.45:0/570897104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0aaa10 con 0x7fdf2c0cf930 2024-04-25T10:18:11.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 -- 172.21.15.45:0/570897104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c0cf930 msgr2=0x7fdf2c0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 --2- 172.21.15.45:0/570897104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c0cfd10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0d0630 tx=0x7fdf1c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 -- 172.21.15.45:0/570897104 shutdown_connections 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 --2- 172.21.15.45:0/570897104 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c070c20 0x7fdf2c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 --2- 172.21.15.45:0/570897104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c0cfd10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 -- 172.21.15.45:0/570897104 >> 172.21.15.45:0/570897104 conn(0x7fdf2c066b70 msgr2=0x7fdf2c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 -- 172.21.15.45:0/570897104 shutdown_connections 2024-04-25T10:18:11.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 -- 172.21.15.45:0/570897104 wait complete. 2024-04-25T10:18:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 Processor -- start 2024-04-25T10:18:11.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.637+0000 7fdf31200640 1 -- start start 2024-04-25T10:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 0x7fdf2c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0ae020 0x7fdf2c0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c06e130 con 0x7fdf2c070c20 2024-04-25T10:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0ae840 con 0x7fdf2c0cf930 2024-04-25T10:18:11.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15e6b0 con 0x7fdf2c0ae020 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 0x7fdf2c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 0x7fdf2c0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57520/0 (socket says 172.21.15.45:57520) 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 -- 172.21.15.45:0/2469139603 learned_addr learned my addr 172.21.15.45:0/2469139603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf2a800640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf23e00640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0ae020 0x7fdf2c0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 -- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0ae020 msgr2=0x7fdf2c0af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0ae020 0x7fdf2c0af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 -- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0cf930 msgr2=0x7fdf2c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 -- 172.21.15.45:0/2469139603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0af820 con 0x7fdf2c070c20 2024-04-25T10:18:11.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf29e00640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 0x7fdf2c0cc3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0ac040 tx=0x7fdf1c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0bd5f0 con 0x7fdf2c070c20 2024-04-25T10:18:11.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0a6040 con 0x7fdf2c070c20 2024-04-25T10:18:11.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0afb50 con 0x7fdf2c070c20 2024-04-25T10:18:11.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0bd9b0 con 0x7fdf2c070c20 2024-04-25T10:18:11.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0cdda0 con 0x7fdf2c070c20 2024-04-25T10:18:11.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c04ef30 con 0x7fdf2c070c20 2024-04-25T10:18:11.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.641+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf1c0b33f0 con 0x7fdf2c070c20 2024-04-25T10:18:11.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.645+0000 7fdf22000640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.645+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf1c11f510 con 0x7fdf2c070c20 2024-04-25T10:18:11.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.645+0000 7fdf23e00640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.645+0000 7fdf23e00640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c161b70 tx=0x7fdf14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.649+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf1c0c3030 con 0x7fdf2c070c20 2024-04-25T10:18:11.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.809+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf2c161500 con 0x7fdf08066f60 2024-04-25T10:18:11.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.817+0000 7fdf22000640 1 -- 172.21.15.45:0/2469139603 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251832 (secure 0 0 0) 0x7fdf2c161500 con 0x7fdf08066f60 2024-04-25T10:18:11.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:11.845 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 msgr2=0x7fdf08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c161b70 tx=0x7fdf14008040 comp rx=0 tx=0).stop 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 msgr2=0x7fdf2c0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 0x7fdf2c0cc3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0ac040 tx=0x7fdf1c01ad30 comp rx=0 tx=0).stop 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 shutdown_connections 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0cf930 0x7fdf2c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0ae020 0x7fdf2c0af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 --2- 172.21.15.45:0/2469139603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c070c20 0x7fdf2c0cc3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 >> 172.21.15.45:0/2469139603 conn(0x7fdf2c066b70 msgr2=0x7fdf2c168060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 shutdown_connections 2024-04-25T10:18:11.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.821+0000 7fdf31200640 1 -- 172.21.15.45:0/2469139603 wait complete. 2024-04-25T10:18:11.848 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T10:18:10.875038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235792,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738928,"statfs":{"total":773094113280,"available":772852662272,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.201583"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704515+0000","last_change":"2024-04-25T10:18:02.719059+0000","last_active":"2024-04-25T10:18:03.704515+0000","last_peered":"2024-04-25T10:18:03.704515+0000","last_clean":"2024-04-25T10:18:03.704515+0000","last_became_active":"2024-04-25T10:18:02.718765+0000","last_became_peered":"2024-04-25T10:18:02.718765+0000","last_unstale":"2024-04-25T10:18:03.704515+0000","last_undegraded":"2024-04-25T10:18:03.704515+0000","last_fullsized":"2024-04-25T10:18:03.704515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694866+0000","last_change":"2024-04-25T10:18:02.693052+0000","last_active":"2024-04-25T10:18:03.694866+0000","last_peered":"2024-04-25T10:18:03.694866+0000","last_clean":"2024-04-25T10:18:03.694866+0000","last_became_active":"2024-04-25T10:18:02.692751+0000","last_became_peered":"2024-04-25T10:18:02.692751+0000","last_unstale":"2024-04-25T10:18:03.694866+0000","last_undegraded":"2024-04-25T10:18:03.694866+0000","last_fullsized":"2024-04-25T10:18:03.694866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694673+0000","last_change":"2024-04-25T10:18:02.694847+0000","last_active":"2024-04-25T10:18:03.694673+0000","last_peered":"2024-04-25T10:18:03.694673+0000","last_clean":"2024-04-25T10:18:03.694673+0000","last_became_active":"2024-04-25T10:18:02.694498+0000","last_became_peered":"2024-04-25T10:18:02.694498+0000","last_unstale":"2024-04-25T10:18:03.694673+0000","last_undegraded":"2024-04-25T10:18:03.694673+0000","last_fullsized":"2024-04-25T10:18:03.694673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703302+0000","last_change":"2024-04-25T10:18:02.693910+0000","last_active":"2024-04-25T10:18:03.703302+0000","last_peered":"2024-04-25T10:18:03.703302+0000","last_clean":"2024-04-25T10:18:03.703302+0000","last_became_active":"2024-04-25T10:18:02.692960+0000","last_became_peered":"2024-04-25T10:18:02.692960+0000","last_unstale":"2024-04-25T10:18:03.703302+0000","last_undegraded":"2024-04-25T10:18:03.703302+0000","last_fullsized":"2024-04-25T10:18:03.703302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703719+0000","last_change":"2024-04-25T10:18:02.709103+0000","last_active":"2024-04-25T10:18:03.703719+0000","last_peered":"2024-04-25T10:18:03.703719+0000","last_clean":"2024-04-25T10:18:03.703719+0000","last_became_active":"2024-04-25T10:18:02.708360+0000","last_became_peered":"2024-04-25T10:18:02.708360+0000","last_unstale":"2024-04-25T10:18:03.703719+0000","last_undegraded":"2024-04-25T10:18:03.703719+0000","last_fullsized":"2024-04-25T10:18:03.703719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702789+0000","last_change":"2024-04-25T10:18:02.686452+0000","last_active":"2024-04-25T10:18:03.702789+0000","last_peered":"2024-04-25T10:18:03.702789+0000","last_clean":"2024-04-25T10:18:03.702789+0000","last_became_active":"2024-04-25T10:18:02.685301+0000","last_became_peered":"2024-04-25T10:18:02.685301+0000","last_unstale":"2024-04-25T10:18:03.702789+0000","last_undegraded":"2024-04-25T10:18:03.702789+0000","last_fullsized":"2024-04-25T10:18:03.702789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.708545+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.707896+0000","last_became_peered":"2024-04-25T10:18:02.707896+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693563+0000","last_change":"2024-04-25T10:18:02.696678+0000","last_active":"2024-04-25T10:18:03.693563+0000","last_peered":"2024-04-25T10:18:03.693563+0000","last_clean":"2024-04-25T10:18:03.693563+0000","last_became_active":"2024-04-25T10:18:02.696428+0000","last_became_peered":"2024-04-25T10:18:02.696428+0000","last_unstale":"2024-04-25T10:18:03.693563+0000","last_undegraded":"2024-04-25T10:18:03.693563+0000","last_fullsized":"2024-04-25T10:18:03.693563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:19.470065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691719+0000","last_change":"2024-04-25T10:18:02.702250+0000","last_active":"2024-04-25T10:18:03.691719+0000","last_peered":"2024-04-25T10:18:03.691719+0000","last_clean":"2024-04-25T10:18:03.691719+0000","last_became_active":"2024-04-25T10:18:02.701992+0000","last_became_peered":"2024-04-25T10:18:02.701992+0000","last_unstale":"2024-04-25T10:18:03.691719+0000","last_undegraded":"2024-04-25T10:18:03.691719+0000","last_fullsized":"2024-04-25T10:18:03.691719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705200+0000","last_change":"2024-04-25T10:18:02.715021+0000","last_active":"2024-04-25T10:18:03.705200+0000","last_peered":"2024-04-25T10:18:03.705200+0000","last_clean":"2024-04-25T10:18:03.705200+0000","last_became_active":"2024-04-25T10:18:02.714746+0000","last_became_peered":"2024-04-25T10:18:02.714746+0000","last_unstale":"2024-04-25T10:18:03.705200+0000","last_undegraded":"2024-04-25T10:18:03.705200+0000","last_fullsized":"2024-04-25T10:18:03.705200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704831+0000","last_change":"2024-04-25T10:18:02.718912+0000","last_active":"2024-04-25T10:18:03.704831+0000","last_peered":"2024-04-25T10:18:03.704831+0000","last_clean":"2024-04-25T10:18:03.704831+0000","last_became_active":"2024-04-25T10:18:02.718611+0000","last_became_peered":"2024-04-25T10:18:02.718611+0000","last_unstale":"2024-04-25T10:18:03.704831+0000","last_undegraded":"2024-04-25T10:18:03.704831+0000","last_fullsized":"2024-04-25T10:18:03.704831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704014+0000","last_change":"2024-04-25T10:18:02.706863+0000","last_active":"2024-04-25T10:18:03.704014+0000","last_peered":"2024-04-25T10:18:03.704014+0000","last_clean":"2024-04-25T10:18:03.704014+0000","last_became_active":"2024-04-25T10:18:02.706447+0000","last_became_peered":"2024-04-25T10:18:02.706447+0000","last_unstale":"2024-04-25T10:18:03.704014+0000","last_undegraded":"2024-04-25T10:18:03.704014+0000","last_fullsized":"2024-04-25T10:18:03.704014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693746+0000","last_change":"2024-04-25T10:18:02.714632+0000","last_active":"2024-04-25T10:18:03.693746+0000","last_peered":"2024-04-25T10:18:03.693746+0000","last_clean":"2024-04-25T10:18:03.693746+0000","last_became_active":"2024-04-25T10:18:02.714302+0000","last_became_peered":"2024-04-25T10:18:02.714302+0000","last_unstale":"2024-04-25T10:18:03.693746+0000","last_undegraded":"2024-04-25T10:18:03.693746+0000","last_fullsized":"2024-04-25T10:18:03.693746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694231+0000","last_change":"2024-04-25T10:18:02.707614+0000","last_active":"2024-04-25T10:18:03.694231+0000","last_peered":"2024-04-25T10:18:03.694231+0000","last_clean":"2024-04-25T10:18:03.694231+0000","last_became_active":"2024-04-25T10:18:02.703697+0000","last_became_peered":"2024-04-25T10:18:02.703697+0000","last_unstale":"2024-04-25T10:18:03.694231+0000","last_undegraded":"2024-04-25T10:18:03.694231+0000","last_fullsized":"2024-04-25T10:18:03.694231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:45.419224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693574+0000","last_change":"2024-04-25T10:18:02.711384+0000","last_active":"2024-04-25T10:18:03.693574+0000","last_peered":"2024-04-25T10:18:03.693574+0000","last_clean":"2024-04-25T10:18:03.693574+0000","last_became_active":"2024-04-25T10:18:02.711036+0000","last_became_peered":"2024-04-25T10:18:02.711036+0000","last_unstale":"2024-04-25T10:18:03.693574+0000","last_undegraded":"2024-04-25T10:18:03.693574+0000","last_fullsized":"2024-04-25T10:18:03.693574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.002103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702415+0000","last_change":"2024-04-25T10:18:02.697670+0000","last_active":"2024-04-25T10:18:03.702415+0000","last_peered":"2024-04-25T10:18:03.702415+0000","last_clean":"2024-04-25T10:18:03.702415+0000","last_became_active":"2024-04-25T10:18:02.695341+0000","last_became_peered":"2024-04-25T10:18:02.695341+0000","last_unstale":"2024-04-25T10:18:03.702415+0000","last_undegraded":"2024-04-25T10:18:03.702415+0000","last_fullsized":"2024-04-25T10:18:03.702415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705262+0000","last_change":"2024-04-25T10:18:02.709655+0000","last_active":"2024-04-25T10:18:03.705262+0000","last_peered":"2024-04-25T10:18:03.705262+0000","last_clean":"2024-04-25T10:18:03.705262+0000","last_became_active":"2024-04-25T10:18:02.709349+0000","last_became_peered":"2024-04-25T10:18:02.709349+0000","last_unstale":"2024-04-25T10:18:03.705262+0000","last_undegraded":"2024-04-25T10:18:03.705262+0000","last_fullsized":"2024-04-25T10:18:03.705262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:57.065003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703843+0000","last_change":"2024-04-25T10:18:02.712551+0000","last_active":"2024-04-25T10:18:03.703843+0000","last_peered":"2024-04-25T10:18:03.703843+0000","last_clean":"2024-04-25T10:18:03.703843+0000","last_became_active":"2024-04-25T10:18:02.712257+0000","last_became_peered":"2024-04-25T10:18:02.712257+0000","last_unstale":"2024-04-25T10:18:03.703843+0000","last_undegraded":"2024-04-25T10:18:03.703843+0000","last_fullsized":"2024-04-25T10:18:03.703843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703878+0000","last_change":"2024-04-25T10:18:02.703048+0000","last_active":"2024-04-25T10:18:03.703878+0000","last_peered":"2024-04-25T10:18:03.703878+0000","last_clean":"2024-04-25T10:18:03.703878+0000","last_became_active":"2024-04-25T10:18:02.702433+0000","last_became_peered":"2024-04-25T10:18:02.702433+0000","last_unstale":"2024-04-25T10:18:03.703878+0000","last_undegraded":"2024-04-25T10:18:03.703878+0000","last_fullsized":"2024-04-25T10:18:03.703878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:25.741442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704147+0000","last_change":"2024-04-25T10:18:02.706776+0000","last_active":"2024-04-25T10:18:03.704147+0000","last_peered":"2024-04-25T10:18:03.704147+0000","last_clean":"2024-04-25T10:18:03.704147+0000","last_became_active":"2024-04-25T10:18:02.706327+0000","last_became_peered":"2024-04-25T10:18:02.706327+0000","last_unstale":"2024-04-25T10:18:03.704147+0000","last_undegraded":"2024-04-25T10:18:03.704147+0000","last_fullsized":"2024-04-25T10:18:03.704147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703147+0000","last_change":"2024-04-25T10:18:02.685113+0000","last_active":"2024-04-25T10:18:03.703147+0000","last_peered":"2024-04-25T10:18:03.703147+0000","last_clean":"2024-04-25T10:18:03.703147+0000","last_became_active":"2024-04-25T10:18:02.684839+0000","last_became_peered":"2024-04-25T10:18:02.684839+0000","last_unstale":"2024-04-25T10:18:03.703147+0000","last_undegraded":"2024-04-25T10:18:03.703147+0000","last_fullsized":"2024-04-25T10:18:03.703147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.141759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691179+0000","last_change":"2024-04-25T10:18:02.713609+0000","last_active":"2024-04-25T10:18:03.691179+0000","last_peered":"2024-04-25T10:18:03.691179+0000","last_clean":"2024-04-25T10:18:03.691179+0000","last_became_active":"2024-04-25T10:18:02.713344+0000","last_became_peered":"2024-04-25T10:18:02.713344+0000","last_unstale":"2024-04-25T10:18:03.691179+0000","last_undegraded":"2024-04-25T10:18:03.691179+0000","last_fullsized":"2024-04-25T10:18:03.691179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:08.145654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693768+0000","last_change":"2024-04-25T10:18:02.688007+0000","last_active":"2024-04-25T10:18:03.693768+0000","last_peered":"2024-04-25T10:18:03.693768+0000","last_clean":"2024-04-25T10:18:03.693768+0000","last_became_active":"2024-04-25T10:18:02.687708+0000","last_became_peered":"2024-04-25T10:18:02.687708+0000","last_unstale":"2024-04-25T10:18:03.693768+0000","last_undegraded":"2024-04-25T10:18:03.693768+0000","last_fullsized":"2024-04-25T10:18:03.693768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.697693+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.696374+0000","last_became_peered":"2024-04-25T10:18:02.696374+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695340+0000","last_change":"2024-04-25T10:18:02.690318+0000","last_active":"2024-04-25T10:18:03.695340+0000","last_peered":"2024-04-25T10:18:03.695340+0000","last_clean":"2024-04-25T10:18:03.695340+0000","last_became_active":"2024-04-25T10:18:02.688844+0000","last_became_peered":"2024-04-25T10:18:02.688844+0000","last_unstale":"2024-04-25T10:18:03.695340+0000","last_undegraded":"2024-04-25T10:18:03.695340+0000","last_fullsized":"2024-04-25T10:18:03.695340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703210+0000","last_change":"2024-04-25T10:18:02.699592+0000","last_active":"2024-04-25T10:18:03.703210+0000","last_peered":"2024-04-25T10:18:03.703210+0000","last_clean":"2024-04-25T10:18:03.703210+0000","last_became_active":"2024-04-25T10:18:02.685005+0000","last_became_peered":"2024-04-25T10:18:02.685005+0000","last_unstale":"2024-04-25T10:18:03.703210+0000","last_undegraded":"2024-04-25T10:18:03.703210+0000","last_fullsized":"2024-04-25T10:18:03.703210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694078+0000","last_change":"2024-04-25T10:18:02.707729+0000","last_active":"2024-04-25T10:18:03.694078+0000","last_peered":"2024-04-25T10:18:03.694078+0000","last_clean":"2024-04-25T10:18:03.694078+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.694078+0000","last_undegraded":"2024-04-25T10:18:03.694078+0000","last_fullsized":"2024-04-25T10:18:03.694078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693598+0000","last_change":"2024-04-25T10:18:02.708638+0000","last_active":"2024-04-25T10:18:03.693598+0000","last_peered":"2024-04-25T10:18:03.693598+0000","last_clean":"2024-04-25T10:18:03.693598+0000","last_became_active":"2024-04-25T10:18:02.708316+0000","last_became_peered":"2024-04-25T10:18:02.708316+0000","last_unstale":"2024-04-25T10:18:03.693598+0000","last_undegraded":"2024-04-25T10:18:03.693598+0000","last_fullsized":"2024-04-25T10:18:03.693598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694317+0000","last_change":"2024-04-25T10:18:02.703831+0000","last_active":"2024-04-25T10:18:03.694317+0000","last_peered":"2024-04-25T10:18:03.694317+0000","last_clean":"2024-04-25T10:18:03.694317+0000","last_became_active":"2024-04-25T10:18:02.703544+0000","last_became_peered":"2024-04-25T10:18:02.703544+0000","last_unstale":"2024-04-25T10:18:03.694317+0000","last_undegraded":"2024-04-25T10:18:03.694317+0000","last_fullsized":"2024-04-25T10:18:03.694317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703007+0000","last_change":"2024-04-25T10:18:02.704805+0000","last_active":"2024-04-25T10:18:03.703007+0000","last_peered":"2024-04-25T10:18:03.703007+0000","last_clean":"2024-04-25T10:18:03.703007+0000","last_became_active":"2024-04-25T10:18:02.704430+0000","last_became_peered":"2024-04-25T10:18:02.704430+0000","last_unstale":"2024-04-25T10:18:03.703007+0000","last_undegraded":"2024-04-25T10:18:03.703007+0000","last_fullsized":"2024-04-25T10:18:03.703007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703417+0000","last_change":"2024-04-25T10:18:02.700147+0000","last_active":"2024-04-25T10:18:03.703417+0000","last_peered":"2024-04-25T10:18:03.703417+0000","last_clean":"2024-04-25T10:18:03.703417+0000","last_became_active":"2024-04-25T10:18:02.699857+0000","last_became_peered":"2024-04-25T10:18:02.699857+0000","last_unstale":"2024-04-25T10:18:03.703417+0000","last_undegraded":"2024-04-25T10:18:03.703417+0000","last_fullsized":"2024-04-25T10:18:03.703417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702527+0000","last_change":"2024-04-25T10:18:02.692533+0000","last_active":"2024-04-25T10:18:03.702527+0000","last_peered":"2024-04-25T10:18:03.702527+0000","last_clean":"2024-04-25T10:18:03.702527+0000","last_became_active":"2024-04-25T10:18:02.692239+0000","last_became_peered":"2024-04-25T10:18:02.692239+0000","last_unstale":"2024-04-25T10:18:03.702527+0000","last_undegraded":"2024-04-25T10:18:03.702527+0000","last_fullsized":"2024-04-25T10:18:03.702527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691316+0000","last_change":"2024-04-25T10:18:02.691725+0000","last_active":"2024-04-25T10:18:03.691316+0000","last_peered":"2024-04-25T10:18:03.691316+0000","last_clean":"2024-04-25T10:18:03.691316+0000","last_became_active":"2024-04-25T10:18:02.691415+0000","last_became_peered":"2024-04-25T10:18:02.691415+0000","last_unstale":"2024-04-25T10:18:03.691316+0000","last_undegraded":"2024-04-25T10:18:03.691316+0000","last_fullsized":"2024-04-25T10:18:03.691316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694378+0000","last_change":"2024-04-25T10:18:01.697822+0000","last_active":"2024-04-25T10:18:03.694378+0000","last_peered":"2024-04-25T10:18:03.694378+0000","last_clean":"2024-04-25T10:18:03.694378+0000","last_became_active":"2024-04-25T10:18:01.696666+0000","last_became_peered":"2024-04-25T10:18:01.696666+0000","last_unstale":"2024-04-25T10:18:03.694378+0000","last_undegraded":"2024-04-25T10:18:03.694378+0000","last_fullsized":"2024-04-25T10:18:03.694378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703321+0000","last_change":"2024-04-25T10:18:02.709407+0000","last_active":"2024-04-25T10:18:03.703321+0000","last_peered":"2024-04-25T10:18:03.703321+0000","last_clean":"2024-04-25T10:18:03.703321+0000","last_became_active":"2024-04-25T10:18:02.708477+0000","last_became_peered":"2024-04-25T10:18:02.708477+0000","last_unstale":"2024-04-25T10:18:03.703321+0000","last_undegraded":"2024-04-25T10:18:03.703321+0000","last_fullsized":"2024-04-25T10:18:03.703321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:03.389578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703290+0000","last_change":"2024-04-25T10:18:01.706968+0000","last_active":"2024-04-25T10:18:03.703290+0000","last_peered":"2024-04-25T10:18:03.703290+0000","last_clean":"2024-04-25T10:18:03.703290+0000","last_became_active":"2024-04-25T10:18:01.706660+0000","last_became_peered":"2024-04-25T10:18:01.706660+0000","last_unstale":"2024-04-25T10:18:03.703290+0000","last_undegraded":"2024-04-25T10:18:03.703290+0000","last_fullsized":"2024-04-25T10:18:03.703290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694922+0000","last_change":"2024-04-25T10:18:01.705160+0000","last_active":"2024-04-25T10:18:03.694922+0000","last_peered":"2024-04-25T10:18:03.694922+0000","last_clean":"2024-04-25T10:18:03.694922+0000","last_became_active":"2024-04-25T10:18:01.704852+0000","last_became_peered":"2024-04-25T10:18:01.704852+0000","last_unstale":"2024-04-25T10:18:03.694922+0000","last_undegraded":"2024-04-25T10:18:03.694922+0000","last_fullsized":"2024-04-25T10:18:03.694922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:33.140157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693846+0000","last_change":"2024-04-25T10:18:02.691734+0000","last_active":"2024-04-25T10:18:03.693846+0000","last_peered":"2024-04-25T10:18:03.693846+0000","last_clean":"2024-04-25T10:18:03.693846+0000","last_became_active":"2024-04-25T10:18:02.691454+0000","last_became_peered":"2024-04-25T10:18:02.691454+0000","last_unstale":"2024-04-25T10:18:03.693846+0000","last_undegraded":"2024-04-25T10:18:03.693846+0000","last_fullsized":"2024-04-25T10:18:03.693846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702652+0000","last_change":"2024-04-25T10:18:01.684950+0000","last_active":"2024-04-25T10:18:03.702652+0000","last_peered":"2024-04-25T10:18:03.702652+0000","last_clean":"2024-04-25T10:18:03.702652+0000","last_became_active":"2024-04-25T10:18:01.682962+0000","last_became_peered":"2024-04-25T10:18:01.682962+0000","last_unstale":"2024-04-25T10:18:03.702652+0000","last_undegraded":"2024-04-25T10:18:03.702652+0000","last_fullsized":"2024-04-25T10:18:03.702652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:45.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702988+0000","last_change":"2024-04-25T10:18:02.695250+0000","last_active":"2024-04-25T10:18:03.702988+0000","last_peered":"2024-04-25T10:18:03.702988+0000","last_clean":"2024-04-25T10:18:03.702988+0000","last_became_active":"2024-04-25T10:18:02.694926+0000","last_became_peered":"2024-04-25T10:18:02.694926+0000","last_unstale":"2024-04-25T10:18:03.702988+0000","last_undegraded":"2024-04-25T10:18:03.702988+0000","last_fullsized":"2024-04-25T10:18:03.702988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:12.876579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691706+0000","last_change":"2024-04-25T10:18:01.691624+0000","last_active":"2024-04-25T10:18:03.691706+0000","last_peered":"2024-04-25T10:18:03.691706+0000","last_clean":"2024-04-25T10:18:03.691706+0000","last_became_active":"2024-04-25T10:18:01.687161+0000","last_became_peered":"2024-04-25T10:18:01.687161+0000","last_unstale":"2024-04-25T10:18:03.691706+0000","last_undegraded":"2024-04-25T10:18:03.691706+0000","last_fullsized":"2024-04-25T10:18:03.691706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703606+0000","last_change":"2024-04-25T10:18:02.685373+0000","last_active":"2024-04-25T10:18:03.703606+0000","last_peered":"2024-04-25T10:18:03.703606+0000","last_clean":"2024-04-25T10:18:03.703606+0000","last_became_active":"2024-04-25T10:18:02.684600+0000","last_became_peered":"2024-04-25T10:18:02.684600+0000","last_unstale":"2024-04-25T10:18:03.703606+0000","last_undegraded":"2024-04-25T10:18:03.703606+0000","last_fullsized":"2024-04-25T10:18:03.703606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705362+0000","last_change":"2024-04-25T10:18:01.703304+0000","last_active":"2024-04-25T10:18:03.705362+0000","last_peered":"2024-04-25T10:18:03.705362+0000","last_clean":"2024-04-25T10:18:03.705362+0000","last_became_active":"2024-04-25T10:18:01.701629+0000","last_became_peered":"2024-04-25T10:18:01.701629+0000","last_unstale":"2024-04-25T10:18:03.705362+0000","last_undegraded":"2024-04-25T10:18:03.705362+0000","last_fullsized":"2024-04-25T10:18:03.705362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695152+0000","last_change":"2024-04-25T10:18:02.696745+0000","last_active":"2024-04-25T10:18:03.695152+0000","last_peered":"2024-04-25T10:18:03.695152+0000","last_clean":"2024-04-25T10:18:03.695152+0000","last_became_active":"2024-04-25T10:18:02.696514+0000","last_became_peered":"2024-04-25T10:18:02.696514+0000","last_unstale":"2024-04-25T10:18:03.695152+0000","last_undegraded":"2024-04-25T10:18:03.695152+0000","last_fullsized":"2024-04-25T10:18:03.695152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703782+0000","last_change":"2024-04-25T10:18:01.692901+0000","last_active":"2024-04-25T10:18:03.703782+0000","last_peered":"2024-04-25T10:18:03.703782+0000","last_clean":"2024-04-25T10:18:03.703782+0000","last_became_active":"2024-04-25T10:18:01.692604+0000","last_became_peered":"2024-04-25T10:18:01.692604+0000","last_unstale":"2024-04-25T10:18:03.703782+0000","last_undegraded":"2024-04-25T10:18:03.703782+0000","last_fullsized":"2024-04-25T10:18:03.703782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703310+0000","last_change":"2024-04-25T10:18:02.696653+0000","last_active":"2024-04-25T10:18:03.703310+0000","last_peered":"2024-04-25T10:18:03.703310+0000","last_clean":"2024-04-25T10:18:03.703310+0000","last_became_active":"2024-04-25T10:18:02.696271+0000","last_became_peered":"2024-04-25T10:18:02.696271+0000","last_unstale":"2024-04-25T10:18:03.703310+0000","last_undegraded":"2024-04-25T10:18:03.703310+0000","last_fullsized":"2024-04-25T10:18:03.703310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:35.561629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703399+0000","last_change":"2024-04-25T10:18:01.696254+0000","last_active":"2024-04-25T10:18:03.703399+0000","last_peered":"2024-04-25T10:18:03.703399+0000","last_clean":"2024-04-25T10:18:03.703399+0000","last_became_active":"2024-04-25T10:18:01.695894+0000","last_became_peered":"2024-04-25T10:18:01.695894+0000","last_unstale":"2024-04-25T10:18:03.703399+0000","last_undegraded":"2024-04-25T10:18:03.703399+0000","last_fullsized":"2024-04-25T10:18:03.703399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704898+0000","last_change":"2024-04-25T10:18:02.707539+0000","last_active":"2024-04-25T10:18:03.704898+0000","last_peered":"2024-04-25T10:18:03.704898+0000","last_clean":"2024-04-25T10:18:03.704898+0000","last_became_active":"2024-04-25T10:18:02.707294+0000","last_became_peered":"2024-04-25T10:18:02.707294+0000","last_unstale":"2024-04-25T10:18:03.704898+0000","last_undegraded":"2024-04-25T10:18:03.704898+0000","last_fullsized":"2024-04-25T10:18:03.704898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703612+0000","last_change":"2024-04-25T10:18:01.703958+0000","last_active":"2024-04-25T10:18:03.703612+0000","last_peered":"2024-04-25T10:18:03.703612+0000","last_clean":"2024-04-25T10:18:03.703612+0000","last_became_active":"2024-04-25T10:18:01.703524+0000","last_became_peered":"2024-04-25T10:18:01.703524+0000","last_unstale":"2024-04-25T10:18:03.703612+0000","last_undegraded":"2024-04-25T10:18:03.703612+0000","last_fullsized":"2024-04-25T10:18:03.703612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694368+0000","last_change":"2024-04-25T10:18:02.715822+0000","last_active":"2024-04-25T10:18:03.694368+0000","last_peered":"2024-04-25T10:18:03.694368+0000","last_clean":"2024-04-25T10:18:03.694368+0000","last_became_active":"2024-04-25T10:18:02.715507+0000","last_became_peered":"2024-04-25T10:18:02.715507+0000","last_unstale":"2024-04-25T10:18:03.694368+0000","last_undegraded":"2024-04-25T10:18:03.694368+0000","last_fullsized":"2024-04-25T10:18:03.694368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:16.553948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703351+0000","last_change":"2024-04-25T10:18:01.702739+0000","last_active":"2024-04-25T10:18:03.703351+0000","last_peered":"2024-04-25T10:18:03.703351+0000","last_clean":"2024-04-25T10:18:03.703351+0000","last_became_active":"2024-04-25T10:18:01.700517+0000","last_became_peered":"2024-04-25T10:18:01.700517+0000","last_unstale":"2024-04-25T10:18:03.703351+0000","last_undegraded":"2024-04-25T10:18:03.703351+0000","last_fullsized":"2024-04-25T10:18:03.703351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:31.803881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703257+0000","last_change":"2024-04-25T10:18:02.709746+0000","last_active":"2024-04-25T10:18:03.703257+0000","last_peered":"2024-04-25T10:18:03.703257+0000","last_clean":"2024-04-25T10:18:03.703257+0000","last_became_active":"2024-04-25T10:18:02.709446+0000","last_became_peered":"2024-04-25T10:18:02.709446+0000","last_unstale":"2024-04-25T10:18:03.703257+0000","last_undegraded":"2024-04-25T10:18:03.703257+0000","last_fullsized":"2024-04-25T10:18:03.703257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:58.435069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702550+0000","last_change":"2024-04-25T10:18:01.710729+0000","last_active":"2024-04-25T10:18:03.702550+0000","last_peered":"2024-04-25T10:18:03.702550+0000","last_clean":"2024-04-25T10:18:03.702550+0000","last_became_active":"2024-04-25T10:18:01.710485+0000","last_became_peered":"2024-04-25T10:18:01.710485+0000","last_unstale":"2024-04-25T10:18:03.702550+0000","last_undegraded":"2024-04-25T10:18:03.702550+0000","last_fullsized":"2024-04-25T10:18:03.702550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691145+0000","last_change":"2024-04-25T10:18:02.688051+0000","last_active":"2024-04-25T10:18:03.691145+0000","last_peered":"2024-04-25T10:18:03.691145+0000","last_clean":"2024-04-25T10:18:03.691145+0000","last_became_active":"2024-04-25T10:18:02.687770+0000","last_became_peered":"2024-04-25T10:18:02.687770+0000","last_unstale":"2024-04-25T10:18:03.691145+0000","last_undegraded":"2024-04-25T10:18:03.691145+0000","last_fullsized":"2024-04-25T10:18:03.691145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:51.841941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702747+0000","last_change":"2024-04-25T10:18:01.693928+0000","last_active":"2024-04-25T10:18:03.702747+0000","last_peered":"2024-04-25T10:18:03.702747+0000","last_clean":"2024-04-25T10:18:03.702747+0000","last_became_active":"2024-04-25T10:18:01.693589+0000","last_became_peered":"2024-04-25T10:18:01.693589+0000","last_unstale":"2024-04-25T10:18:03.702747+0000","last_undegraded":"2024-04-25T10:18:03.702747+0000","last_fullsized":"2024-04-25T10:18:03.702747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:31.631333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702833+0000","last_change":"2024-04-25T10:18:02.685007+0000","last_active":"2024-04-25T10:18:03.702833+0000","last_peered":"2024-04-25T10:18:03.702833+0000","last_clean":"2024-04-25T10:18:03.702833+0000","last_became_active":"2024-04-25T10:18:02.684369+0000","last_became_peered":"2024-04-25T10:18:02.684369+0000","last_unstale":"2024-04-25T10:18:03.702833+0000","last_undegraded":"2024-04-25T10:18:03.702833+0000","last_fullsized":"2024-04-25T10:18:03.702833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704706+0000","last_change":"2024-04-25T10:18:01.702018+0000","last_active":"2024-04-25T10:18:03.704706+0000","last_peered":"2024-04-25T10:18:03.704706+0000","last_clean":"2024-04-25T10:18:03.704706+0000","last_became_active":"2024-04-25T10:18:01.701680+0000","last_became_peered":"2024-04-25T10:18:01.701680+0000","last_unstale":"2024-04-25T10:18:03.704706+0000","last_undegraded":"2024-04-25T10:18:03.704706+0000","last_fullsized":"2024-04-25T10:18:03.704706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703739+0000","last_change":"2024-04-25T10:18:02.709479+0000","last_active":"2024-04-25T10:18:03.703739+0000","last_peered":"2024-04-25T10:18:03.703739+0000","last_clean":"2024-04-25T10:18:03.703739+0000","last_became_active":"2024-04-25T10:18:02.709098+0000","last_became_peered":"2024-04-25T10:18:02.709098+0000","last_unstale":"2024-04-25T10:18:03.703739+0000","last_undegraded":"2024-04-25T10:18:03.703739+0000","last_fullsized":"2024-04-25T10:18:03.703739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703210+0000","last_change":"2024-04-25T10:18:01.691170+0000","last_active":"2024-04-25T10:18:03.703210+0000","last_peered":"2024-04-25T10:18:03.703210+0000","last_clean":"2024-04-25T10:18:03.703210+0000","last_became_active":"2024-04-25T10:18:01.690897+0000","last_became_peered":"2024-04-25T10:18:01.690897+0000","last_unstale":"2024-04-25T10:18:03.703210+0000","last_undegraded":"2024-04-25T10:18:03.703210+0000","last_fullsized":"2024-04-25T10:18:03.703210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:19.034996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704586+0000","last_change":"2024-04-25T10:18:02.708709+0000","last_active":"2024-04-25T10:18:03.704586+0000","last_peered":"2024-04-25T10:18:03.704586+0000","last_clean":"2024-04-25T10:18:03.704586+0000","last_became_active":"2024-04-25T10:18:02.703952+0000","last_became_peered":"2024-04-25T10:18:02.703952+0000","last_unstale":"2024-04-25T10:18:03.704586+0000","last_undegraded":"2024-04-25T10:18:03.704586+0000","last_fullsized":"2024-04-25T10:18:03.704586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694644+0000","last_change":"2024-04-25T10:18:01.690031+0000","last_active":"2024-04-25T10:18:03.694644+0000","last_peered":"2024-04-25T10:18:03.694644+0000","last_clean":"2024-04-25T10:18:03.694644+0000","last_became_active":"2024-04-25T10:18:01.689608+0000","last_became_peered":"2024-04-25T10:18:01.689608+0000","last_unstale":"2024-04-25T10:18:03.694644+0000","last_undegraded":"2024-04-25T10:18:03.694644+0000","last_fullsized":"2024-04-25T10:18:03.694644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704278+0000","last_change":"2024-04-25T10:18:02.706632+0000","last_active":"2024-04-25T10:18:03.704278+0000","last_peered":"2024-04-25T10:18:03.704278+0000","last_clean":"2024-04-25T10:18:03.704278+0000","last_became_active":"2024-04-25T10:18:02.705983+0000","last_became_peered":"2024-04-25T10:18:02.705983+0000","last_unstale":"2024-04-25T10:18:03.704278+0000","last_undegraded":"2024-04-25T10:18:03.704278+0000","last_fullsized":"2024-04-25T10:18:03.704278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693832+0000","last_change":"2024-04-25T10:18:01.705153+0000","last_active":"2024-04-25T10:18:03.693832+0000","last_peered":"2024-04-25T10:18:03.693832+0000","last_clean":"2024-04-25T10:18:03.693832+0000","last_became_active":"2024-04-25T10:18:01.704844+0000","last_became_peered":"2024-04-25T10:18:01.704844+0000","last_unstale":"2024-04-25T10:18:03.693832+0000","last_undegraded":"2024-04-25T10:18:03.693832+0000","last_fullsized":"2024-04-25T10:18:03.693832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703740+0000","last_change":"2024-04-25T10:18:02.703183+0000","last_active":"2024-04-25T10:18:03.703740+0000","last_peered":"2024-04-25T10:18:03.703740+0000","last_clean":"2024-04-25T10:18:03.703740+0000","last_became_active":"2024-04-25T10:18:02.702789+0000","last_became_peered":"2024-04-25T10:18:02.702789+0000","last_unstale":"2024-04-25T10:18:03.703740+0000","last_undegraded":"2024-04-25T10:18:03.703740+0000","last_fullsized":"2024-04-25T10:18:03.703740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691080+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:03.691080+0000","last_peered":"2024-04-25T10:18:03.691080+0000","last_clean":"2024-04-25T10:18:03.691080+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:03.691080+0000","last_undegraded":"2024-04-25T10:18:03.691080+0000","last_fullsized":"2024-04-25T10:18:03.691080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703473+0000","last_change":"2024-04-25T10:18:01.687105+0000","last_active":"2024-04-25T10:18:03.703473+0000","last_peered":"2024-04-25T10:18:03.703473+0000","last_clean":"2024-04-25T10:18:03.703473+0000","last_became_active":"2024-04-25T10:18:01.686184+0000","last_became_peered":"2024-04-25T10:18:01.686184+0000","last_unstale":"2024-04-25T10:18:03.703473+0000","last_undegraded":"2024-04-25T10:18:03.703473+0000","last_fullsized":"2024-04-25T10:18:03.703473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702986+0000","last_change":"2024-04-25T10:18:02.685204+0000","last_active":"2024-04-25T10:18:03.702986+0000","last_peered":"2024-04-25T10:18:03.702986+0000","last_clean":"2024-04-25T10:18:03.702986+0000","last_became_active":"2024-04-25T10:18:02.684900+0000","last_became_peered":"2024-04-25T10:18:02.684900+0000","last_unstale":"2024-04-25T10:18:03.702986+0000","last_undegraded":"2024-04-25T10:18:03.702986+0000","last_fullsized":"2024-04-25T10:18:03.702986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:06.719729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702748+0000","last_change":"2024-04-25T10:18:01.700977+0000","last_active":"2024-04-25T10:18:03.702748+0000","last_peered":"2024-04-25T10:18:03.702748+0000","last_clean":"2024-04-25T10:18:03.702748+0000","last_became_active":"2024-04-25T10:18:01.682598+0000","last_became_peered":"2024-04-25T10:18:01.682598+0000","last_unstale":"2024-04-25T10:18:03.702748+0000","last_undegraded":"2024-04-25T10:18:03.702748+0000","last_fullsized":"2024-04-25T10:18:03.702748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:17.642626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693408+0000","last_change":"2024-04-25T10:18:02.693588+0000","last_active":"2024-04-25T10:18:03.693408+0000","last_peered":"2024-04-25T10:18:03.693408+0000","last_clean":"2024-04-25T10:18:03.693408+0000","last_became_active":"2024-04-25T10:18:02.693296+0000","last_became_peered":"2024-04-25T10:18:02.693296+0000","last_unstale":"2024-04-25T10:18:03.693408+0000","last_undegraded":"2024-04-25T10:18:03.693408+0000","last_fullsized":"2024-04-25T10:18:03.693408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703521+0000","last_change":"2024-04-25T10:18:01.693364+0000","last_active":"2024-04-25T10:18:03.703521+0000","last_peered":"2024-04-25T10:18:03.703521+0000","last_clean":"2024-04-25T10:18:03.703521+0000","last_became_active":"2024-04-25T10:18:01.692980+0000","last_became_peered":"2024-04-25T10:18:01.692980+0000","last_unstale":"2024-04-25T10:18:03.703521+0000","last_undegraded":"2024-04-25T10:18:03.703521+0000","last_fullsized":"2024-04-25T10:18:03.703521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693704+0000","last_change":"2024-04-25T10:18:02.704788+0000","last_active":"2024-04-25T10:18:03.693704+0000","last_peered":"2024-04-25T10:18:03.693704+0000","last_clean":"2024-04-25T10:18:03.693704+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.693704+0000","last_undegraded":"2024-04-25T10:18:03.693704+0000","last_fullsized":"2024-04-25T10:18:03.693704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:18.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703433+0000","last_change":"2024-04-25T10:18:01.702159+0000","last_active":"2024-04-25T10:18:03.703433+0000","last_peered":"2024-04-25T10:18:03.703433+0000","last_clean":"2024-04-25T10:18:03.703433+0000","last_became_active":"2024-04-25T10:18:01.701049+0000","last_became_peered":"2024-04-25T10:18:01.701049+0000","last_unstale":"2024-04-25T10:18:03.703433+0000","last_undegraded":"2024-04-25T10:18:03.703433+0000","last_fullsized":"2024-04-25T10:18:03.703433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:31.455848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703720+0000","last_change":"2024-04-25T10:18:02.710983+0000","last_active":"2024-04-25T10:18:03.703720+0000","last_peered":"2024-04-25T10:18:03.703720+0000","last_clean":"2024-04-25T10:18:03.703720+0000","last_became_active":"2024-04-25T10:18:02.710479+0000","last_became_peered":"2024-04-25T10:18:02.710479+0000","last_unstale":"2024-04-25T10:18:03.703720+0000","last_undegraded":"2024-04-25T10:18:03.703720+0000","last_fullsized":"2024-04-25T10:18:03.703720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694576+0000","last_change":"2024-04-25T10:18:01.697707+0000","last_active":"2024-04-25T10:18:03.694576+0000","last_peered":"2024-04-25T10:18:03.694576+0000","last_clean":"2024-04-25T10:18:03.694576+0000","last_became_active":"2024-04-25T10:18:01.697322+0000","last_became_peered":"2024-04-25T10:18:01.697322+0000","last_unstale":"2024-04-25T10:18:03.694576+0000","last_undegraded":"2024-04-25T10:18:03.694576+0000","last_fullsized":"2024-04-25T10:18:03.694576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702991+0000","last_change":"2024-04-25T10:18:02.697739+0000","last_active":"2024-04-25T10:18:03.702991+0000","last_peered":"2024-04-25T10:18:03.702991+0000","last_clean":"2024-04-25T10:18:03.702991+0000","last_became_active":"2024-04-25T10:18:02.697420+0000","last_became_peered":"2024-04-25T10:18:02.697420+0000","last_unstale":"2024-04-25T10:18:03.702991+0000","last_undegraded":"2024-04-25T10:18:03.702991+0000","last_fullsized":"2024-04-25T10:18:03.702991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703868+0000","last_change":"2024-04-25T10:18:01.706705+0000","last_active":"2024-04-25T10:18:03.703868+0000","last_peered":"2024-04-25T10:18:03.703868+0000","last_clean":"2024-04-25T10:18:03.703868+0000","last_became_active":"2024-04-25T10:18:01.705850+0000","last_became_peered":"2024-04-25T10:18:01.705850+0000","last_unstale":"2024-04-25T10:18:03.703868+0000","last_undegraded":"2024-04-25T10:18:03.703868+0000","last_fullsized":"2024-04-25T10:18:03.703868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.828188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693716+0000","last_change":"2024-04-25T10:18:02.704532+0000","last_active":"2024-04-25T10:18:03.693716+0000","last_peered":"2024-04-25T10:18:03.693716+0000","last_clean":"2024-04-25T10:18:03.693716+0000","last_became_active":"2024-04-25T10:18:02.701511+0000","last_became_peered":"2024-04-25T10:18:02.701511+0000","last_unstale":"2024-04-25T10:18:03.693716+0000","last_undegraded":"2024-04-25T10:18:03.693716+0000","last_fullsized":"2024-04-25T10:18:03.693716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703374+0000","last_change":"2024-04-25T10:18:01.698971+0000","last_active":"2024-04-25T10:18:03.703374+0000","last_peered":"2024-04-25T10:18:03.703374+0000","last_clean":"2024-04-25T10:18:03.703374+0000","last_became_active":"2024-04-25T10:18:01.698719+0000","last_became_peered":"2024-04-25T10:18:01.698719+0000","last_unstale":"2024-04-25T10:18:03.703374+0000","last_undegraded":"2024-04-25T10:18:03.703374+0000","last_fullsized":"2024-04-25T10:18:03.703374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:55.382109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703430+0000","last_change":"2024-04-25T10:18:02.700120+0000","last_active":"2024-04-25T10:18:03.703430+0000","last_peered":"2024-04-25T10:18:03.703430+0000","last_clean":"2024-04-25T10:18:03.703430+0000","last_became_active":"2024-04-25T10:18:02.699859+0000","last_became_peered":"2024-04-25T10:18:02.699859+0000","last_unstale":"2024-04-25T10:18:03.703430+0000","last_undegraded":"2024-04-25T10:18:03.703430+0000","last_fullsized":"2024-04-25T10:18:03.703430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:17.794954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705253+0000","last_change":"2024-04-25T10:18:01.709433+0000","last_active":"2024-04-25T10:18:03.705253+0000","last_peered":"2024-04-25T10:18:03.705253+0000","last_clean":"2024-04-25T10:18:03.705253+0000","last_became_active":"2024-04-25T10:18:01.709103+0000","last_became_peered":"2024-04-25T10:18:01.709103+0000","last_unstale":"2024-04-25T10:18:03.705253+0000","last_undegraded":"2024-04-25T10:18:03.705253+0000","last_fullsized":"2024-04-25T10:18:03.705253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:59.703487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703245+0000","last_change":"2024-04-25T10:18:02.708776+0000","last_active":"2024-04-25T10:18:03.703245+0000","last_peered":"2024-04-25T10:18:03.703245+0000","last_clean":"2024-04-25T10:18:03.703245+0000","last_became_active":"2024-04-25T10:18:02.708166+0000","last_became_peered":"2024-04-25T10:18:02.708166+0000","last_unstale":"2024-04-25T10:18:03.703245+0000","last_undegraded":"2024-04-25T10:18:03.703245+0000","last_fullsized":"2024-04-25T10:18:03.703245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:03.973278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702969+0000","last_change":"2024-04-25T10:18:01.698359+0000","last_active":"2024-04-25T10:18:03.702969+0000","last_peered":"2024-04-25T10:18:03.702969+0000","last_clean":"2024-04-25T10:18:03.702969+0000","last_became_active":"2024-04-25T10:18:01.697573+0000","last_became_peered":"2024-04-25T10:18:01.697573+0000","last_unstale":"2024-04-25T10:18:03.702969+0000","last_undegraded":"2024-04-25T10:18:03.702969+0000","last_fullsized":"2024-04-25T10:18:03.702969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702843+0000","last_change":"2024-04-25T10:18:02.703801+0000","last_active":"2024-04-25T10:18:03.702843+0000","last_peered":"2024-04-25T10:18:03.702843+0000","last_clean":"2024-04-25T10:18:03.702843+0000","last_became_active":"2024-04-25T10:18:02.703508+0000","last_became_peered":"2024-04-25T10:18:02.703508+0000","last_unstale":"2024-04-25T10:18:03.702843+0000","last_undegraded":"2024-04-25T10:18:03.702843+0000","last_fullsized":"2024-04-25T10:18:03.702843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702956+0000","last_change":"2024-04-25T10:18:01.704197+0000","last_active":"2024-04-25T10:18:03.702956+0000","last_peered":"2024-04-25T10:18:03.702956+0000","last_clean":"2024-04-25T10:18:03.702956+0000","last_became_active":"2024-04-25T10:18:01.703932+0000","last_became_peered":"2024-04-25T10:18:01.703932+0000","last_unstale":"2024-04-25T10:18:03.702956+0000","last_undegraded":"2024-04-25T10:18:03.702956+0000","last_fullsized":"2024-04-25T10:18:03.702956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:40.452243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702798+0000","last_change":"2024-04-25T10:18:02.702635+0000","last_active":"2024-04-25T10:18:03.702798+0000","last_peered":"2024-04-25T10:18:03.702798+0000","last_clean":"2024-04-25T10:18:03.702798+0000","last_became_active":"2024-04-25T10:18:02.702335+0000","last_became_peered":"2024-04-25T10:18:02.702335+0000","last_unstale":"2024-04-25T10:18:03.702798+0000","last_undegraded":"2024-04-25T10:18:03.702798+0000","last_fullsized":"2024-04-25T10:18:03.702798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:20.552276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694476+0000","last_change":"2024-04-25T10:18:01.692670+0000","last_active":"2024-04-25T10:18:03.694476+0000","last_peered":"2024-04-25T10:18:03.694476+0000","last_clean":"2024-04-25T10:18:03.694476+0000","last_became_active":"2024-04-25T10:18:01.691338+0000","last_became_peered":"2024-04-25T10:18:01.691338+0000","last_unstale":"2024-04-25T10:18:03.694476+0000","last_undegraded":"2024-04-25T10:18:03.694476+0000","last_fullsized":"2024-04-25T10:18:03.694476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703461+0000","last_change":"2024-04-25T10:18:02.702741+0000","last_active":"2024-04-25T10:18:03.703461+0000","last_peered":"2024-04-25T10:18:03.703461+0000","last_clean":"2024-04-25T10:18:03.703461+0000","last_became_active":"2024-04-25T10:18:02.702299+0000","last_became_peered":"2024-04-25T10:18:02.702299+0000","last_unstale":"2024-04-25T10:18:03.703461+0000","last_undegraded":"2024-04-25T10:18:03.703461+0000","last_fullsized":"2024-04-25T10:18:03.703461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703382+0000","last_change":"2024-04-25T10:18:01.686488+0000","last_active":"2024-04-25T10:18:03.703382+0000","last_peered":"2024-04-25T10:18:03.703382+0000","last_clean":"2024-04-25T10:18:03.703382+0000","last_became_active":"2024-04-25T10:18:01.685599+0000","last_became_peered":"2024-04-25T10:18:01.685599+0000","last_unstale":"2024-04-25T10:18:03.703382+0000","last_undegraded":"2024-04-25T10:18:03.703382+0000","last_fullsized":"2024-04-25T10:18:03.703382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703213+0000","last_change":"2024-04-25T10:18:02.705187+0000","last_active":"2024-04-25T10:18:03.703213+0000","last_peered":"2024-04-25T10:18:03.703213+0000","last_clean":"2024-04-25T10:18:03.703213+0000","last_became_active":"2024-04-25T10:18:02.704486+0000","last_became_peered":"2024-04-25T10:18:02.704486+0000","last_unstale":"2024-04-25T10:18:03.703213+0000","last_undegraded":"2024-04-25T10:18:03.703213+0000","last_fullsized":"2024-04-25T10:18:03.703213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703616+0000","last_change":"2024-04-25T10:18:02.716723+0000","last_active":"2024-04-25T10:18:03.703616+0000","last_peered":"2024-04-25T10:18:03.703616+0000","last_clean":"2024-04-25T10:18:03.703616+0000","last_became_active":"2024-04-25T10:18:02.716264+0000","last_became_peered":"2024-04-25T10:18:02.716264+0000","last_unstale":"2024-04-25T10:18:03.703616+0000","last_undegraded":"2024-04-25T10:18:03.703616+0000","last_fullsized":"2024-04-25T10:18:03.703616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.581498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704370+0000","last_change":"2024-04-25T10:18:01.698545+0000","last_active":"2024-04-25T10:18:03.704370+0000","last_peered":"2024-04-25T10:18:03.704370+0000","last_clean":"2024-04-25T10:18:03.704370+0000","last_became_active":"2024-04-25T10:18:01.696867+0000","last_became_peered":"2024-04-25T10:18:01.696867+0000","last_unstale":"2024-04-25T10:18:03.704370+0000","last_undegraded":"2024-04-25T10:18:03.704370+0000","last_fullsized":"2024-04-25T10:18:03.704370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691426+0000","last_change":"2024-04-25T10:18:02.712672+0000","last_active":"2024-04-25T10:18:03.691426+0000","last_peered":"2024-04-25T10:18:03.691426+0000","last_clean":"2024-04-25T10:18:03.691426+0000","last_became_active":"2024-04-25T10:18:02.712103+0000","last_became_peered":"2024-04-25T10:18:02.712103+0000","last_unstale":"2024-04-25T10:18:03.691426+0000","last_undegraded":"2024-04-25T10:18:03.691426+0000","last_fullsized":"2024-04-25T10:18:03.691426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704949+0000","last_change":"2024-04-25T10:18:01.704511+0000","last_active":"2024-04-25T10:18:03.704949+0000","last_peered":"2024-04-25T10:18:03.704949+0000","last_clean":"2024-04-25T10:18:03.704949+0000","last_became_active":"2024-04-25T10:18:01.703001+0000","last_became_peered":"2024-04-25T10:18:01.703001+0000","last_unstale":"2024-04-25T10:18:03.704949+0000","last_undegraded":"2024-04-25T10:18:03.704949+0000","last_fullsized":"2024-04-25T10:18:03.704949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694997+0000","last_change":"2024-04-25T10:18:02.690317+0000","last_active":"2024-04-25T10:18:03.694997+0000","last_peered":"2024-04-25T10:18:03.694997+0000","last_clean":"2024-04-25T10:18:03.694997+0000","last_became_active":"2024-04-25T10:18:02.687277+0000","last_became_peered":"2024-04-25T10:18:02.687277+0000","last_unstale":"2024-04-25T10:18:03.694997+0000","last_undegraded":"2024-04-25T10:18:03.694997+0000","last_fullsized":"2024-04-25T10:18:03.694997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694980+0000","last_change":"2024-04-25T10:18:01.688261+0000","last_active":"2024-04-25T10:18:03.694980+0000","last_peered":"2024-04-25T10:18:03.694980+0000","last_clean":"2024-04-25T10:18:03.694980+0000","last_became_active":"2024-04-25T10:18:01.687516+0000","last_became_peered":"2024-04-25T10:18:01.687516+0000","last_unstale":"2024-04-25T10:18:03.694980+0000","last_undegraded":"2024-04-25T10:18:03.694980+0000","last_fullsized":"2024-04-25T10:18:03.694980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694903+0000","last_change":"2024-04-25T10:18:02.687787+0000","last_active":"2024-04-25T10:18:03.694903+0000","last_peered":"2024-04-25T10:18:03.694903+0000","last_clean":"2024-04-25T10:18:03.694903+0000","last_became_active":"2024-04-25T10:18:02.687351+0000","last_became_peered":"2024-04-25T10:18:02.687351+0000","last_unstale":"2024-04-25T10:18:03.694903+0000","last_undegraded":"2024-04-25T10:18:03.694903+0000","last_fullsized":"2024-04-25T10:18:03.694903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703546+0000","last_change":"2024-04-25T10:18:01.710038+0000","last_active":"2024-04-25T10:18:03.703546+0000","last_peered":"2024-04-25T10:18:03.703546+0000","last_clean":"2024-04-25T10:18:03.703546+0000","last_became_active":"2024-04-25T10:18:01.709789+0000","last_became_peered":"2024-04-25T10:18:01.709789+0000","last_unstale":"2024-04-25T10:18:03.703546+0000","last_undegraded":"2024-04-25T10:18:03.703546+0000","last_fullsized":"2024-04-25T10:18:03.703546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:49.219702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:11.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:11.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.989+0000 7f8a64200640 1 Processor -- start 2024-04-25T10:18:11.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.989+0000 7f8a64200640 1 -- start start 2024-04-25T10:18:11.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a64200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a5c1550b0 0x7f8a5c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a64200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c071050 con 0x7f8a5c0cfe70 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a64200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c15d040 con 0x7f8a5c15b910 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a64200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c15bd80 con 0x7f8a5c1550b0 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a61800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a5c1550b0 0x7f8a5c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a60e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8a5c0cfe70 0x7f8a5c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59314/0 (socket says 172.21.15.45:59314) 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a60e00640 1 -- 172.21.15.45:0/1789368088 learned_addr learned my addr 172.21.15.45:0/1789368088 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:11.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a61800640 1 -- 172.21.15.45:0/1789368088 >> v1:172.21.15.117:6789/0 conn(0x7f8a5c15b910 legacy=0x7f8a5c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:11.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a61800640 1 -- 172.21.15.45:0/1789368088 >> v1:172.21.15.45:6789/0 conn(0x7f8a5c0cfe70 legacy=0x7f8a5c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.993+0000 7f8a61800640 1 -- 172.21.15.45:0/1789368088 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0712f0 con 0x7f8a5c1550b0 2024-04-25T10:18:11.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a61800640 1 --2- 172.21.15.45:0/1789368088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c0cf930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a500097f0 tx=0x7f8a5001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad056bfe49d6f016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:11.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a5be00640 1 -- 172.21.15.45:0/1789368088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500b0070 con 0x7f8a5c1550b0 2024-04-25T10:18:11.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a5be00640 1 -- 172.21.15.45:0/1789368088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500ab600 con 0x7f8a5c1550b0 2024-04-25T10:18:11.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a5be00640 1 -- 172.21.15.45:0/1789368088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500ab9d0 con 0x7f8a5c1550b0 2024-04-25T10:18:11.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- 172.21.15.45:0/1789368088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c1550b0 msgr2=0x7f8a5c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 --2- 172.21.15.45:0/1789368088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c0cf930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a500097f0 tx=0x7f8a5001c460 comp rx=0 tx=0).stop 2024-04-25T10:18:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- 172.21.15.45:0/1789368088 shutdown_connections 2024-04-25T10:18:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 --2- 172.21.15.45:0/1789368088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c0cf930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- 172.21.15.45:0/1789368088 >> 172.21.15.45:0/1789368088 conn(0x7f8a5c066b70 msgr2=0x7f8a5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- 172.21.15.45:0/1789368088 shutdown_connections 2024-04-25T10:18:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- 172.21.15.45:0/1789368088 wait complete. 2024-04-25T10:18:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 Processor -- start 2024-04-25T10:18:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- start start 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a5c0cfe70 0x7f8a5c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 0x7f8a5c14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c06c090 con 0x7f8a5c15b910 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0720b0 con 0x7f8a5c1550b0 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a64200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c06c210 con 0x7f8a5c0cfe70 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:11.997+0000 7f8a62200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 0x7f8a5c14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 0x7f8a5c14b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57548/0 (socket says 172.21.15.45:57548) 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 -- 172.21.15.45:0/2319875180 learned_addr learned my addr 172.21.15.45:0/2319875180 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a60e00640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a61800640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a5c0cfe70 0x7f8a5c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 -- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a5c0cfe70 msgr2=0x7f8a5c149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a5c0cfe70 0x7f8a5c149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 -- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a5c1550b0 msgr2=0x7f8a5c14aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c14aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 -- 172.21.15.45:0/2319875180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c159ab0 con 0x7f8a5c15b910 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a61800640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a5c0cfe70 0x7f8a5c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a62200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 0x7f8a5c14b000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400b990 tx=0x7f8a5400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54019070 con 0x7f8a5c15b910 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400cba0 con 0x7f8a5c15b910 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54014440 con 0x7f8a5c15b910 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5c159d90 con 0x7f8a5c15b910 2024-04-25T10:18:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c15a350 con 0x7f8a5c15b910 2024-04-25T10:18:12.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5c04ef30 con 0x7f8a5c15b910 2024-04-25T10:18:12.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a5401c070 con 0x7f8a5c15b910 2024-04-25T10:18:12.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a59600640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a3c066f60 0x7f8a3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.001+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a54084d70 con 0x7f8a5c15b910 2024-04-25T10:18:12.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.005+0000 7f8a61800640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a3c066f60 0x7f8a3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.005+0000 7f8a61800640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a3c066f60 0x7f8a3c069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a500ad000 tx=0x7f8a500a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.005+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a54055d70 con 0x7f8a5c15b910 2024-04-25T10:18:12.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.165+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a5c15a870 con 0x7f8a3c066f60 2024-04-25T10:18:12.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a59600640 1 -- 172.21.15.45:0/2319875180 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251832 (secure 0 0 0) 0x7f8a5c15a870 con 0x7f8a3c066f60 2024-04-25T10:18:12.176 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:12.178 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a3c066f60 msgr2=0x7f8a3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a3c066f60 0x7f8a3c069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a500ad000 tx=0x7f8a500a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 msgr2=0x7f8a5c14b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 0x7f8a5c14b000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400b990 tx=0x7f8a5400be50 comp rx=0 tx=0).stop 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 shutdown_connections 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a3c066f60 0x7f8a3c069420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a5c15b910 0x7f8a5c14b000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a5c1550b0 0x7f8a5c14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 --2- 172.21.15.45:0/2319875180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a5c0cfe70 0x7f8a5c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 >> 172.21.15.45:0/2319875180 conn(0x7f8a5c066b70 msgr2=0x7f8a5c06d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 shutdown_connections 2024-04-25T10:18:12.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.173+0000 7f8a64200640 1 -- 172.21.15.45:0/2319875180 wait complete. 2024-04-25T10:18:12.197 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-25T10:18:10.875038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235792,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738928,"statfs":{"total":773094113280,"available":772852662272,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.201583"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704515+0000","last_change":"2024-04-25T10:18:02.719059+0000","last_active":"2024-04-25T10:18:03.704515+0000","last_peered":"2024-04-25T10:18:03.704515+0000","last_clean":"2024-04-25T10:18:03.704515+0000","last_became_active":"2024-04-25T10:18:02.718765+0000","last_became_peered":"2024-04-25T10:18:02.718765+0000","last_unstale":"2024-04-25T10:18:03.704515+0000","last_undegraded":"2024-04-25T10:18:03.704515+0000","last_fullsized":"2024-04-25T10:18:03.704515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694866+0000","last_change":"2024-04-25T10:18:02.693052+0000","last_active":"2024-04-25T10:18:03.694866+0000","last_peered":"2024-04-25T10:18:03.694866+0000","last_clean":"2024-04-25T10:18:03.694866+0000","last_became_active":"2024-04-25T10:18:02.692751+0000","last_became_peered":"2024-04-25T10:18:02.692751+0000","last_unstale":"2024-04-25T10:18:03.694866+0000","last_undegraded":"2024-04-25T10:18:03.694866+0000","last_fullsized":"2024-04-25T10:18:03.694866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694673+0000","last_change":"2024-04-25T10:18:02.694847+0000","last_active":"2024-04-25T10:18:03.694673+0000","last_peered":"2024-04-25T10:18:03.694673+0000","last_clean":"2024-04-25T10:18:03.694673+0000","last_became_active":"2024-04-25T10:18:02.694498+0000","last_became_peered":"2024-04-25T10:18:02.694498+0000","last_unstale":"2024-04-25T10:18:03.694673+0000","last_undegraded":"2024-04-25T10:18:03.694673+0000","last_fullsized":"2024-04-25T10:18:03.694673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703302+0000","last_change":"2024-04-25T10:18:02.693910+0000","last_active":"2024-04-25T10:18:03.703302+0000","last_peered":"2024-04-25T10:18:03.703302+0000","last_clean":"2024-04-25T10:18:03.703302+0000","last_became_active":"2024-04-25T10:18:02.692960+0000","last_became_peered":"2024-04-25T10:18:02.692960+0000","last_unstale":"2024-04-25T10:18:03.703302+0000","last_undegraded":"2024-04-25T10:18:03.703302+0000","last_fullsized":"2024-04-25T10:18:03.703302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703719+0000","last_change":"2024-04-25T10:18:02.709103+0000","last_active":"2024-04-25T10:18:03.703719+0000","last_peered":"2024-04-25T10:18:03.703719+0000","last_clean":"2024-04-25T10:18:03.703719+0000","last_became_active":"2024-04-25T10:18:02.708360+0000","last_became_peered":"2024-04-25T10:18:02.708360+0000","last_unstale":"2024-04-25T10:18:03.703719+0000","last_undegraded":"2024-04-25T10:18:03.703719+0000","last_fullsized":"2024-04-25T10:18:03.703719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702789+0000","last_change":"2024-04-25T10:18:02.686452+0000","last_active":"2024-04-25T10:18:03.702789+0000","last_peered":"2024-04-25T10:18:03.702789+0000","last_clean":"2024-04-25T10:18:03.702789+0000","last_became_active":"2024-04-25T10:18:02.685301+0000","last_became_peered":"2024-04-25T10:18:02.685301+0000","last_unstale":"2024-04-25T10:18:03.702789+0000","last_undegraded":"2024-04-25T10:18:03.702789+0000","last_fullsized":"2024-04-25T10:18:03.702789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.708545+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.707896+0000","last_became_peered":"2024-04-25T10:18:02.707896+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693563+0000","last_change":"2024-04-25T10:18:02.696678+0000","last_active":"2024-04-25T10:18:03.693563+0000","last_peered":"2024-04-25T10:18:03.693563+0000","last_clean":"2024-04-25T10:18:03.693563+0000","last_became_active":"2024-04-25T10:18:02.696428+0000","last_became_peered":"2024-04-25T10:18:02.696428+0000","last_unstale":"2024-04-25T10:18:03.693563+0000","last_undegraded":"2024-04-25T10:18:03.693563+0000","last_fullsized":"2024-04-25T10:18:03.693563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:19.470065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691719+0000","last_change":"2024-04-25T10:18:02.702250+0000","last_active":"2024-04-25T10:18:03.691719+0000","last_peered":"2024-04-25T10:18:03.691719+0000","last_clean":"2024-04-25T10:18:03.691719+0000","last_became_active":"2024-04-25T10:18:02.701992+0000","last_became_peered":"2024-04-25T10:18:02.701992+0000","last_unstale":"2024-04-25T10:18:03.691719+0000","last_undegraded":"2024-04-25T10:18:03.691719+0000","last_fullsized":"2024-04-25T10:18:03.691719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705200+0000","last_change":"2024-04-25T10:18:02.715021+0000","last_active":"2024-04-25T10:18:03.705200+0000","last_peered":"2024-04-25T10:18:03.705200+0000","last_clean":"2024-04-25T10:18:03.705200+0000","last_became_active":"2024-04-25T10:18:02.714746+0000","last_became_peered":"2024-04-25T10:18:02.714746+0000","last_unstale":"2024-04-25T10:18:03.705200+0000","last_undegraded":"2024-04-25T10:18:03.705200+0000","last_fullsized":"2024-04-25T10:18:03.705200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704831+0000","last_change":"2024-04-25T10:18:02.718912+0000","last_active":"2024-04-25T10:18:03.704831+0000","last_peered":"2024-04-25T10:18:03.704831+0000","last_clean":"2024-04-25T10:18:03.704831+0000","last_became_active":"2024-04-25T10:18:02.718611+0000","last_became_peered":"2024-04-25T10:18:02.718611+0000","last_unstale":"2024-04-25T10:18:03.704831+0000","last_undegraded":"2024-04-25T10:18:03.704831+0000","last_fullsized":"2024-04-25T10:18:03.704831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704014+0000","last_change":"2024-04-25T10:18:02.706863+0000","last_active":"2024-04-25T10:18:03.704014+0000","last_peered":"2024-04-25T10:18:03.704014+0000","last_clean":"2024-04-25T10:18:03.704014+0000","last_became_active":"2024-04-25T10:18:02.706447+0000","last_became_peered":"2024-04-25T10:18:02.706447+0000","last_unstale":"2024-04-25T10:18:03.704014+0000","last_undegraded":"2024-04-25T10:18:03.704014+0000","last_fullsized":"2024-04-25T10:18:03.704014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:38.874432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693746+0000","last_change":"2024-04-25T10:18:02.714632+0000","last_active":"2024-04-25T10:18:03.693746+0000","last_peered":"2024-04-25T10:18:03.693746+0000","last_clean":"2024-04-25T10:18:03.693746+0000","last_became_active":"2024-04-25T10:18:02.714302+0000","last_became_peered":"2024-04-25T10:18:02.714302+0000","last_unstale":"2024-04-25T10:18:03.693746+0000","last_undegraded":"2024-04-25T10:18:03.693746+0000","last_fullsized":"2024-04-25T10:18:03.693746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694231+0000","last_change":"2024-04-25T10:18:02.707614+0000","last_active":"2024-04-25T10:18:03.694231+0000","last_peered":"2024-04-25T10:18:03.694231+0000","last_clean":"2024-04-25T10:18:03.694231+0000","last_became_active":"2024-04-25T10:18:02.703697+0000","last_became_peered":"2024-04-25T10:18:02.703697+0000","last_unstale":"2024-04-25T10:18:03.694231+0000","last_undegraded":"2024-04-25T10:18:03.694231+0000","last_fullsized":"2024-04-25T10:18:03.694231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:45.419224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693574+0000","last_change":"2024-04-25T10:18:02.711384+0000","last_active":"2024-04-25T10:18:03.693574+0000","last_peered":"2024-04-25T10:18:03.693574+0000","last_clean":"2024-04-25T10:18:03.693574+0000","last_became_active":"2024-04-25T10:18:02.711036+0000","last_became_peered":"2024-04-25T10:18:02.711036+0000","last_unstale":"2024-04-25T10:18:03.693574+0000","last_undegraded":"2024-04-25T10:18:03.693574+0000","last_fullsized":"2024-04-25T10:18:03.693574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.002103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702415+0000","last_change":"2024-04-25T10:18:02.697670+0000","last_active":"2024-04-25T10:18:03.702415+0000","last_peered":"2024-04-25T10:18:03.702415+0000","last_clean":"2024-04-25T10:18:03.702415+0000","last_became_active":"2024-04-25T10:18:02.695341+0000","last_became_peered":"2024-04-25T10:18:02.695341+0000","last_unstale":"2024-04-25T10:18:03.702415+0000","last_undegraded":"2024-04-25T10:18:03.702415+0000","last_fullsized":"2024-04-25T10:18:03.702415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705262+0000","last_change":"2024-04-25T10:18:02.709655+0000","last_active":"2024-04-25T10:18:03.705262+0000","last_peered":"2024-04-25T10:18:03.705262+0000","last_clean":"2024-04-25T10:18:03.705262+0000","last_became_active":"2024-04-25T10:18:02.709349+0000","last_became_peered":"2024-04-25T10:18:02.709349+0000","last_unstale":"2024-04-25T10:18:03.705262+0000","last_undegraded":"2024-04-25T10:18:03.705262+0000","last_fullsized":"2024-04-25T10:18:03.705262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:57.065003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703843+0000","last_change":"2024-04-25T10:18:02.712551+0000","last_active":"2024-04-25T10:18:03.703843+0000","last_peered":"2024-04-25T10:18:03.703843+0000","last_clean":"2024-04-25T10:18:03.703843+0000","last_became_active":"2024-04-25T10:18:02.712257+0000","last_became_peered":"2024-04-25T10:18:02.712257+0000","last_unstale":"2024-04-25T10:18:03.703843+0000","last_undegraded":"2024-04-25T10:18:03.703843+0000","last_fullsized":"2024-04-25T10:18:03.703843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:00.710098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703878+0000","last_change":"2024-04-25T10:18:02.703048+0000","last_active":"2024-04-25T10:18:03.703878+0000","last_peered":"2024-04-25T10:18:03.703878+0000","last_clean":"2024-04-25T10:18:03.703878+0000","last_became_active":"2024-04-25T10:18:02.702433+0000","last_became_peered":"2024-04-25T10:18:02.702433+0000","last_unstale":"2024-04-25T10:18:03.703878+0000","last_undegraded":"2024-04-25T10:18:03.703878+0000","last_fullsized":"2024-04-25T10:18:03.703878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:25.741442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704147+0000","last_change":"2024-04-25T10:18:02.706776+0000","last_active":"2024-04-25T10:18:03.704147+0000","last_peered":"2024-04-25T10:18:03.704147+0000","last_clean":"2024-04-25T10:18:03.704147+0000","last_became_active":"2024-04-25T10:18:02.706327+0000","last_became_peered":"2024-04-25T10:18:02.706327+0000","last_unstale":"2024-04-25T10:18:03.704147+0000","last_undegraded":"2024-04-25T10:18:03.704147+0000","last_fullsized":"2024-04-25T10:18:03.704147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.885173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703147+0000","last_change":"2024-04-25T10:18:02.685113+0000","last_active":"2024-04-25T10:18:03.703147+0000","last_peered":"2024-04-25T10:18:03.703147+0000","last_clean":"2024-04-25T10:18:03.703147+0000","last_became_active":"2024-04-25T10:18:02.684839+0000","last_became_peered":"2024-04-25T10:18:02.684839+0000","last_unstale":"2024-04-25T10:18:03.703147+0000","last_undegraded":"2024-04-25T10:18:03.703147+0000","last_fullsized":"2024-04-25T10:18:03.703147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.141759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691179+0000","last_change":"2024-04-25T10:18:02.713609+0000","last_active":"2024-04-25T10:18:03.691179+0000","last_peered":"2024-04-25T10:18:03.691179+0000","last_clean":"2024-04-25T10:18:03.691179+0000","last_became_active":"2024-04-25T10:18:02.713344+0000","last_became_peered":"2024-04-25T10:18:02.713344+0000","last_unstale":"2024-04-25T10:18:03.691179+0000","last_undegraded":"2024-04-25T10:18:03.691179+0000","last_fullsized":"2024-04-25T10:18:03.691179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:08.145654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693768+0000","last_change":"2024-04-25T10:18:02.688007+0000","last_active":"2024-04-25T10:18:03.693768+0000","last_peered":"2024-04-25T10:18:03.693768+0000","last_clean":"2024-04-25T10:18:03.693768+0000","last_became_active":"2024-04-25T10:18:02.687708+0000","last_became_peered":"2024-04-25T10:18:02.687708+0000","last_unstale":"2024-04-25T10:18:03.693768+0000","last_undegraded":"2024-04-25T10:18:03.693768+0000","last_fullsized":"2024-04-25T10:18:03.693768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:07.777969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702555+0000","last_change":"2024-04-25T10:18:02.697693+0000","last_active":"2024-04-25T10:18:03.702555+0000","last_peered":"2024-04-25T10:18:03.702555+0000","last_clean":"2024-04-25T10:18:03.702555+0000","last_became_active":"2024-04-25T10:18:02.696374+0000","last_became_peered":"2024-04-25T10:18:02.696374+0000","last_unstale":"2024-04-25T10:18:03.702555+0000","last_undegraded":"2024-04-25T10:18:03.702555+0000","last_fullsized":"2024-04-25T10:18:03.702555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695340+0000","last_change":"2024-04-25T10:18:02.690318+0000","last_active":"2024-04-25T10:18:03.695340+0000","last_peered":"2024-04-25T10:18:03.695340+0000","last_clean":"2024-04-25T10:18:03.695340+0000","last_became_active":"2024-04-25T10:18:02.688844+0000","last_became_peered":"2024-04-25T10:18:02.688844+0000","last_unstale":"2024-04-25T10:18:03.695340+0000","last_undegraded":"2024-04-25T10:18:03.695340+0000","last_fullsized":"2024-04-25T10:18:03.695340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703210+0000","last_change":"2024-04-25T10:18:02.699592+0000","last_active":"2024-04-25T10:18:03.703210+0000","last_peered":"2024-04-25T10:18:03.703210+0000","last_clean":"2024-04-25T10:18:03.703210+0000","last_became_active":"2024-04-25T10:18:02.685005+0000","last_became_peered":"2024-04-25T10:18:02.685005+0000","last_unstale":"2024-04-25T10:18:03.703210+0000","last_undegraded":"2024-04-25T10:18:03.703210+0000","last_fullsized":"2024-04-25T10:18:03.703210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694078+0000","last_change":"2024-04-25T10:18:02.707729+0000","last_active":"2024-04-25T10:18:03.694078+0000","last_peered":"2024-04-25T10:18:03.694078+0000","last_clean":"2024-04-25T10:18:03.694078+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.694078+0000","last_undegraded":"2024-04-25T10:18:03.694078+0000","last_fullsized":"2024-04-25T10:18:03.694078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693598+0000","last_change":"2024-04-25T10:18:02.708638+0000","last_active":"2024-04-25T10:18:03.693598+0000","last_peered":"2024-04-25T10:18:03.693598+0000","last_clean":"2024-04-25T10:18:03.693598+0000","last_became_active":"2024-04-25T10:18:02.708316+0000","last_became_peered":"2024-04-25T10:18:02.708316+0000","last_unstale":"2024-04-25T10:18:03.693598+0000","last_undegraded":"2024-04-25T10:18:03.693598+0000","last_fullsized":"2024-04-25T10:18:03.693598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:32.810492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694317+0000","last_change":"2024-04-25T10:18:02.703831+0000","last_active":"2024-04-25T10:18:03.694317+0000","last_peered":"2024-04-25T10:18:03.694317+0000","last_clean":"2024-04-25T10:18:03.694317+0000","last_became_active":"2024-04-25T10:18:02.703544+0000","last_became_peered":"2024-04-25T10:18:02.703544+0000","last_unstale":"2024-04-25T10:18:03.694317+0000","last_undegraded":"2024-04-25T10:18:03.694317+0000","last_fullsized":"2024-04-25T10:18:03.694317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:11.127541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703007+0000","last_change":"2024-04-25T10:18:02.704805+0000","last_active":"2024-04-25T10:18:03.703007+0000","last_peered":"2024-04-25T10:18:03.703007+0000","last_clean":"2024-04-25T10:18:03.703007+0000","last_became_active":"2024-04-25T10:18:02.704430+0000","last_became_peered":"2024-04-25T10:18:02.704430+0000","last_unstale":"2024-04-25T10:18:03.703007+0000","last_undegraded":"2024-04-25T10:18:03.703007+0000","last_fullsized":"2024-04-25T10:18:03.703007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703417+0000","last_change":"2024-04-25T10:18:02.700147+0000","last_active":"2024-04-25T10:18:03.703417+0000","last_peered":"2024-04-25T10:18:03.703417+0000","last_clean":"2024-04-25T10:18:03.703417+0000","last_became_active":"2024-04-25T10:18:02.699857+0000","last_became_peered":"2024-04-25T10:18:02.699857+0000","last_unstale":"2024-04-25T10:18:03.703417+0000","last_undegraded":"2024-04-25T10:18:03.703417+0000","last_fullsized":"2024-04-25T10:18:03.703417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702527+0000","last_change":"2024-04-25T10:18:02.692533+0000","last_active":"2024-04-25T10:18:03.702527+0000","last_peered":"2024-04-25T10:18:03.702527+0000","last_clean":"2024-04-25T10:18:03.702527+0000","last_became_active":"2024-04-25T10:18:02.692239+0000","last_became_peered":"2024-04-25T10:18:02.692239+0000","last_unstale":"2024-04-25T10:18:03.702527+0000","last_undegraded":"2024-04-25T10:18:03.702527+0000","last_fullsized":"2024-04-25T10:18:03.702527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691316+0000","last_change":"2024-04-25T10:18:02.691725+0000","last_active":"2024-04-25T10:18:03.691316+0000","last_peered":"2024-04-25T10:18:03.691316+0000","last_clean":"2024-04-25T10:18:03.691316+0000","last_became_active":"2024-04-25T10:18:02.691415+0000","last_became_peered":"2024-04-25T10:18:02.691415+0000","last_unstale":"2024-04-25T10:18:03.691316+0000","last_undegraded":"2024-04-25T10:18:03.691316+0000","last_fullsized":"2024-04-25T10:18:03.691316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:46.227003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694378+0000","last_change":"2024-04-25T10:18:01.697822+0000","last_active":"2024-04-25T10:18:03.694378+0000","last_peered":"2024-04-25T10:18:03.694378+0000","last_clean":"2024-04-25T10:18:03.694378+0000","last_became_active":"2024-04-25T10:18:01.696666+0000","last_became_peered":"2024-04-25T10:18:01.696666+0000","last_unstale":"2024-04-25T10:18:03.694378+0000","last_undegraded":"2024-04-25T10:18:03.694378+0000","last_fullsized":"2024-04-25T10:18:03.694378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703321+0000","last_change":"2024-04-25T10:18:02.709407+0000","last_active":"2024-04-25T10:18:03.703321+0000","last_peered":"2024-04-25T10:18:03.703321+0000","last_clean":"2024-04-25T10:18:03.703321+0000","last_became_active":"2024-04-25T10:18:02.708477+0000","last_became_peered":"2024-04-25T10:18:02.708477+0000","last_unstale":"2024-04-25T10:18:03.703321+0000","last_undegraded":"2024-04-25T10:18:03.703321+0000","last_fullsized":"2024-04-25T10:18:03.703321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:03.389578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703290+0000","last_change":"2024-04-25T10:18:01.706968+0000","last_active":"2024-04-25T10:18:03.703290+0000","last_peered":"2024-04-25T10:18:03.703290+0000","last_clean":"2024-04-25T10:18:03.703290+0000","last_became_active":"2024-04-25T10:18:01.706660+0000","last_became_peered":"2024-04-25T10:18:01.706660+0000","last_unstale":"2024-04-25T10:18:03.703290+0000","last_undegraded":"2024-04-25T10:18:03.703290+0000","last_fullsized":"2024-04-25T10:18:03.703290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694922+0000","last_change":"2024-04-25T10:18:01.705160+0000","last_active":"2024-04-25T10:18:03.694922+0000","last_peered":"2024-04-25T10:18:03.694922+0000","last_clean":"2024-04-25T10:18:03.694922+0000","last_became_active":"2024-04-25T10:18:01.704852+0000","last_became_peered":"2024-04-25T10:18:01.704852+0000","last_unstale":"2024-04-25T10:18:03.694922+0000","last_undegraded":"2024-04-25T10:18:03.694922+0000","last_fullsized":"2024-04-25T10:18:03.694922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:33.140157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693846+0000","last_change":"2024-04-25T10:18:02.691734+0000","last_active":"2024-04-25T10:18:03.693846+0000","last_peered":"2024-04-25T10:18:03.693846+0000","last_clean":"2024-04-25T10:18:03.693846+0000","last_became_active":"2024-04-25T10:18:02.691454+0000","last_became_peered":"2024-04-25T10:18:02.691454+0000","last_unstale":"2024-04-25T10:18:03.693846+0000","last_undegraded":"2024-04-25T10:18:03.693846+0000","last_fullsized":"2024-04-25T10:18:03.693846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702652+0000","last_change":"2024-04-25T10:18:01.684950+0000","last_active":"2024-04-25T10:18:03.702652+0000","last_peered":"2024-04-25T10:18:03.702652+0000","last_clean":"2024-04-25T10:18:03.702652+0000","last_became_active":"2024-04-25T10:18:01.682962+0000","last_became_peered":"2024-04-25T10:18:01.682962+0000","last_unstale":"2024-04-25T10:18:03.702652+0000","last_undegraded":"2024-04-25T10:18:03.702652+0000","last_fullsized":"2024-04-25T10:18:03.702652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:45.220393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702988+0000","last_change":"2024-04-25T10:18:02.695250+0000","last_active":"2024-04-25T10:18:03.702988+0000","last_peered":"2024-04-25T10:18:03.702988+0000","last_clean":"2024-04-25T10:18:03.702988+0000","last_became_active":"2024-04-25T10:18:02.694926+0000","last_became_peered":"2024-04-25T10:18:02.694926+0000","last_unstale":"2024-04-25T10:18:03.702988+0000","last_undegraded":"2024-04-25T10:18:03.702988+0000","last_fullsized":"2024-04-25T10:18:03.702988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:12.876579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691706+0000","last_change":"2024-04-25T10:18:01.691624+0000","last_active":"2024-04-25T10:18:03.691706+0000","last_peered":"2024-04-25T10:18:03.691706+0000","last_clean":"2024-04-25T10:18:03.691706+0000","last_became_active":"2024-04-25T10:18:01.687161+0000","last_became_peered":"2024-04-25T10:18:01.687161+0000","last_unstale":"2024-04-25T10:18:03.691706+0000","last_undegraded":"2024-04-25T10:18:03.691706+0000","last_fullsized":"2024-04-25T10:18:03.691706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703606+0000","last_change":"2024-04-25T10:18:02.685373+0000","last_active":"2024-04-25T10:18:03.703606+0000","last_peered":"2024-04-25T10:18:03.703606+0000","last_clean":"2024-04-25T10:18:03.703606+0000","last_became_active":"2024-04-25T10:18:02.684600+0000","last_became_peered":"2024-04-25T10:18:02.684600+0000","last_unstale":"2024-04-25T10:18:03.703606+0000","last_undegraded":"2024-04-25T10:18:03.703606+0000","last_fullsized":"2024-04-25T10:18:03.703606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:00.834799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705362+0000","last_change":"2024-04-25T10:18:01.703304+0000","last_active":"2024-04-25T10:18:03.705362+0000","last_peered":"2024-04-25T10:18:03.705362+0000","last_clean":"2024-04-25T10:18:03.705362+0000","last_became_active":"2024-04-25T10:18:01.701629+0000","last_became_peered":"2024-04-25T10:18:01.701629+0000","last_unstale":"2024-04-25T10:18:03.705362+0000","last_undegraded":"2024-04-25T10:18:03.705362+0000","last_fullsized":"2024-04-25T10:18:03.705362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.695152+0000","last_change":"2024-04-25T10:18:02.696745+0000","last_active":"2024-04-25T10:18:03.695152+0000","last_peered":"2024-04-25T10:18:03.695152+0000","last_clean":"2024-04-25T10:18:03.695152+0000","last_became_active":"2024-04-25T10:18:02.696514+0000","last_became_peered":"2024-04-25T10:18:02.696514+0000","last_unstale":"2024-04-25T10:18:03.695152+0000","last_undegraded":"2024-04-25T10:18:03.695152+0000","last_fullsized":"2024-04-25T10:18:03.695152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703782+0000","last_change":"2024-04-25T10:18:01.692901+0000","last_active":"2024-04-25T10:18:03.703782+0000","last_peered":"2024-04-25T10:18:03.703782+0000","last_clean":"2024-04-25T10:18:03.703782+0000","last_became_active":"2024-04-25T10:18:01.692604+0000","last_became_peered":"2024-04-25T10:18:01.692604+0000","last_unstale":"2024-04-25T10:18:03.703782+0000","last_undegraded":"2024-04-25T10:18:03.703782+0000","last_fullsized":"2024-04-25T10:18:03.703782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703310+0000","last_change":"2024-04-25T10:18:02.696653+0000","last_active":"2024-04-25T10:18:03.703310+0000","last_peered":"2024-04-25T10:18:03.703310+0000","last_clean":"2024-04-25T10:18:03.703310+0000","last_became_active":"2024-04-25T10:18:02.696271+0000","last_became_peered":"2024-04-25T10:18:02.696271+0000","last_unstale":"2024-04-25T10:18:03.703310+0000","last_undegraded":"2024-04-25T10:18:03.703310+0000","last_fullsized":"2024-04-25T10:18:03.703310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:35.561629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703399+0000","last_change":"2024-04-25T10:18:01.696254+0000","last_active":"2024-04-25T10:18:03.703399+0000","last_peered":"2024-04-25T10:18:03.703399+0000","last_clean":"2024-04-25T10:18:03.703399+0000","last_became_active":"2024-04-25T10:18:01.695894+0000","last_became_peered":"2024-04-25T10:18:01.695894+0000","last_unstale":"2024-04-25T10:18:03.703399+0000","last_undegraded":"2024-04-25T10:18:03.703399+0000","last_fullsized":"2024-04-25T10:18:03.703399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:48.049944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704898+0000","last_change":"2024-04-25T10:18:02.707539+0000","last_active":"2024-04-25T10:18:03.704898+0000","last_peered":"2024-04-25T10:18:03.704898+0000","last_clean":"2024-04-25T10:18:03.704898+0000","last_became_active":"2024-04-25T10:18:02.707294+0000","last_became_peered":"2024-04-25T10:18:02.707294+0000","last_unstale":"2024-04-25T10:18:03.704898+0000","last_undegraded":"2024-04-25T10:18:03.704898+0000","last_fullsized":"2024-04-25T10:18:03.704898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:05.833080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703612+0000","last_change":"2024-04-25T10:18:01.703958+0000","last_active":"2024-04-25T10:18:03.703612+0000","last_peered":"2024-04-25T10:18:03.703612+0000","last_clean":"2024-04-25T10:18:03.703612+0000","last_became_active":"2024-04-25T10:18:01.703524+0000","last_became_peered":"2024-04-25T10:18:01.703524+0000","last_unstale":"2024-04-25T10:18:03.703612+0000","last_undegraded":"2024-04-25T10:18:03.703612+0000","last_fullsized":"2024-04-25T10:18:03.703612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694368+0000","last_change":"2024-04-25T10:18:02.715822+0000","last_active":"2024-04-25T10:18:03.694368+0000","last_peered":"2024-04-25T10:18:03.694368+0000","last_clean":"2024-04-25T10:18:03.694368+0000","last_became_active":"2024-04-25T10:18:02.715507+0000","last_became_peered":"2024-04-25T10:18:02.715507+0000","last_unstale":"2024-04-25T10:18:03.694368+0000","last_undegraded":"2024-04-25T10:18:03.694368+0000","last_fullsized":"2024-04-25T10:18:03.694368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:16.553948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703351+0000","last_change":"2024-04-25T10:18:01.702739+0000","last_active":"2024-04-25T10:18:03.703351+0000","last_peered":"2024-04-25T10:18:03.703351+0000","last_clean":"2024-04-25T10:18:03.703351+0000","last_became_active":"2024-04-25T10:18:01.700517+0000","last_became_peered":"2024-04-25T10:18:01.700517+0000","last_unstale":"2024-04-25T10:18:03.703351+0000","last_undegraded":"2024-04-25T10:18:03.703351+0000","last_fullsized":"2024-04-25T10:18:03.703351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:31.803881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703257+0000","last_change":"2024-04-25T10:18:02.709746+0000","last_active":"2024-04-25T10:18:03.703257+0000","last_peered":"2024-04-25T10:18:03.703257+0000","last_clean":"2024-04-25T10:18:03.703257+0000","last_became_active":"2024-04-25T10:18:02.709446+0000","last_became_peered":"2024-04-25T10:18:02.709446+0000","last_unstale":"2024-04-25T10:18:03.703257+0000","last_undegraded":"2024-04-25T10:18:03.703257+0000","last_fullsized":"2024-04-25T10:18:03.703257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:58.435069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702550+0000","last_change":"2024-04-25T10:18:01.710729+0000","last_active":"2024-04-25T10:18:03.702550+0000","last_peered":"2024-04-25T10:18:03.702550+0000","last_clean":"2024-04-25T10:18:03.702550+0000","last_became_active":"2024-04-25T10:18:01.710485+0000","last_became_peered":"2024-04-25T10:18:01.710485+0000","last_unstale":"2024-04-25T10:18:03.702550+0000","last_undegraded":"2024-04-25T10:18:03.702550+0000","last_fullsized":"2024-04-25T10:18:03.702550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691145+0000","last_change":"2024-04-25T10:18:02.688051+0000","last_active":"2024-04-25T10:18:03.691145+0000","last_peered":"2024-04-25T10:18:03.691145+0000","last_clean":"2024-04-25T10:18:03.691145+0000","last_became_active":"2024-04-25T10:18:02.687770+0000","last_became_peered":"2024-04-25T10:18:02.687770+0000","last_unstale":"2024-04-25T10:18:03.691145+0000","last_undegraded":"2024-04-25T10:18:03.691145+0000","last_fullsized":"2024-04-25T10:18:03.691145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:51.841941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702747+0000","last_change":"2024-04-25T10:18:01.693928+0000","last_active":"2024-04-25T10:18:03.702747+0000","last_peered":"2024-04-25T10:18:03.702747+0000","last_clean":"2024-04-25T10:18:03.702747+0000","last_became_active":"2024-04-25T10:18:01.693589+0000","last_became_peered":"2024-04-25T10:18:01.693589+0000","last_unstale":"2024-04-25T10:18:03.702747+0000","last_undegraded":"2024-04-25T10:18:03.702747+0000","last_fullsized":"2024-04-25T10:18:03.702747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:31.631333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702833+0000","last_change":"2024-04-25T10:18:02.685007+0000","last_active":"2024-04-25T10:18:03.702833+0000","last_peered":"2024-04-25T10:18:03.702833+0000","last_clean":"2024-04-25T10:18:03.702833+0000","last_became_active":"2024-04-25T10:18:02.684369+0000","last_became_peered":"2024-04-25T10:18:02.684369+0000","last_unstale":"2024-04-25T10:18:03.702833+0000","last_undegraded":"2024-04-25T10:18:03.702833+0000","last_fullsized":"2024-04-25T10:18:03.702833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704706+0000","last_change":"2024-04-25T10:18:01.702018+0000","last_active":"2024-04-25T10:18:03.704706+0000","last_peered":"2024-04-25T10:18:03.704706+0000","last_clean":"2024-04-25T10:18:03.704706+0000","last_became_active":"2024-04-25T10:18:01.701680+0000","last_became_peered":"2024-04-25T10:18:01.701680+0000","last_unstale":"2024-04-25T10:18:03.704706+0000","last_undegraded":"2024-04-25T10:18:03.704706+0000","last_fullsized":"2024-04-25T10:18:03.704706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703739+0000","last_change":"2024-04-25T10:18:02.709479+0000","last_active":"2024-04-25T10:18:03.703739+0000","last_peered":"2024-04-25T10:18:03.703739+0000","last_clean":"2024-04-25T10:18:03.703739+0000","last_became_active":"2024-04-25T10:18:02.709098+0000","last_became_peered":"2024-04-25T10:18:02.709098+0000","last_unstale":"2024-04-25T10:18:03.703739+0000","last_undegraded":"2024-04-25T10:18:03.703739+0000","last_fullsized":"2024-04-25T10:18:03.703739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:33.242631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703210+0000","last_change":"2024-04-25T10:18:01.691170+0000","last_active":"2024-04-25T10:18:03.703210+0000","last_peered":"2024-04-25T10:18:03.703210+0000","last_clean":"2024-04-25T10:18:03.703210+0000","last_became_active":"2024-04-25T10:18:01.690897+0000","last_became_peered":"2024-04-25T10:18:01.690897+0000","last_unstale":"2024-04-25T10:18:03.703210+0000","last_undegraded":"2024-04-25T10:18:03.703210+0000","last_fullsized":"2024-04-25T10:18:03.703210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:19.034996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704586+0000","last_change":"2024-04-25T10:18:02.708709+0000","last_active":"2024-04-25T10:18:03.704586+0000","last_peered":"2024-04-25T10:18:03.704586+0000","last_clean":"2024-04-25T10:18:03.704586+0000","last_became_active":"2024-04-25T10:18:02.703952+0000","last_became_peered":"2024-04-25T10:18:02.703952+0000","last_unstale":"2024-04-25T10:18:03.704586+0000","last_undegraded":"2024-04-25T10:18:03.704586+0000","last_fullsized":"2024-04-25T10:18:03.704586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:42.075427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694644+0000","last_change":"2024-04-25T10:18:01.690031+0000","last_active":"2024-04-25T10:18:03.694644+0000","last_peered":"2024-04-25T10:18:03.694644+0000","last_clean":"2024-04-25T10:18:03.694644+0000","last_became_active":"2024-04-25T10:18:01.689608+0000","last_became_peered":"2024-04-25T10:18:01.689608+0000","last_unstale":"2024-04-25T10:18:03.694644+0000","last_undegraded":"2024-04-25T10:18:03.694644+0000","last_fullsized":"2024-04-25T10:18:03.694644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704278+0000","last_change":"2024-04-25T10:18:02.706632+0000","last_active":"2024-04-25T10:18:03.704278+0000","last_peered":"2024-04-25T10:18:03.704278+0000","last_clean":"2024-04-25T10:18:03.704278+0000","last_became_active":"2024-04-25T10:18:02.705983+0000","last_became_peered":"2024-04-25T10:18:02.705983+0000","last_unstale":"2024-04-25T10:18:03.704278+0000","last_undegraded":"2024-04-25T10:18:03.704278+0000","last_fullsized":"2024-04-25T10:18:03.704278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693832+0000","last_change":"2024-04-25T10:18:01.705153+0000","last_active":"2024-04-25T10:18:03.693832+0000","last_peered":"2024-04-25T10:18:03.693832+0000","last_clean":"2024-04-25T10:18:03.693832+0000","last_became_active":"2024-04-25T10:18:01.704844+0000","last_became_peered":"2024-04-25T10:18:01.704844+0000","last_unstale":"2024-04-25T10:18:03.693832+0000","last_undegraded":"2024-04-25T10:18:03.693832+0000","last_fullsized":"2024-04-25T10:18:03.693832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703740+0000","last_change":"2024-04-25T10:18:02.703183+0000","last_active":"2024-04-25T10:18:03.703740+0000","last_peered":"2024-04-25T10:18:03.703740+0000","last_clean":"2024-04-25T10:18:03.703740+0000","last_became_active":"2024-04-25T10:18:02.702789+0000","last_became_peered":"2024-04-25T10:18:02.702789+0000","last_unstale":"2024-04-25T10:18:03.703740+0000","last_undegraded":"2024-04-25T10:18:03.703740+0000","last_fullsized":"2024-04-25T10:18:03.703740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:02.862404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691080+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:03.691080+0000","last_peered":"2024-04-25T10:18:03.691080+0000","last_clean":"2024-04-25T10:18:03.691080+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:03.691080+0000","last_undegraded":"2024-04-25T10:18:03.691080+0000","last_fullsized":"2024-04-25T10:18:03.691080+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703473+0000","last_change":"2024-04-25T10:18:01.687105+0000","last_active":"2024-04-25T10:18:03.703473+0000","last_peered":"2024-04-25T10:18:03.703473+0000","last_clean":"2024-04-25T10:18:03.703473+0000","last_became_active":"2024-04-25T10:18:01.686184+0000","last_became_peered":"2024-04-25T10:18:01.686184+0000","last_unstale":"2024-04-25T10:18:03.703473+0000","last_undegraded":"2024-04-25T10:18:03.703473+0000","last_fullsized":"2024-04-25T10:18:03.703473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.139043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702986+0000","last_change":"2024-04-25T10:18:02.685204+0000","last_active":"2024-04-25T10:18:03.702986+0000","last_peered":"2024-04-25T10:18:03.702986+0000","last_clean":"2024-04-25T10:18:03.702986+0000","last_became_active":"2024-04-25T10:18:02.684900+0000","last_became_peered":"2024-04-25T10:18:02.684900+0000","last_unstale":"2024-04-25T10:18:03.702986+0000","last_undegraded":"2024-04-25T10:18:03.702986+0000","last_fullsized":"2024-04-25T10:18:03.702986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:06.719729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702748+0000","last_change":"2024-04-25T10:18:01.700977+0000","last_active":"2024-04-25T10:18:03.702748+0000","last_peered":"2024-04-25T10:18:03.702748+0000","last_clean":"2024-04-25T10:18:03.702748+0000","last_became_active":"2024-04-25T10:18:01.682598+0000","last_became_peered":"2024-04-25T10:18:01.682598+0000","last_unstale":"2024-04-25T10:18:03.702748+0000","last_undegraded":"2024-04-25T10:18:03.702748+0000","last_fullsized":"2024-04-25T10:18:03.702748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:17.642626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693408+0000","last_change":"2024-04-25T10:18:02.693588+0000","last_active":"2024-04-25T10:18:03.693408+0000","last_peered":"2024-04-25T10:18:03.693408+0000","last_clean":"2024-04-25T10:18:03.693408+0000","last_became_active":"2024-04-25T10:18:02.693296+0000","last_became_peered":"2024-04-25T10:18:02.693296+0000","last_unstale":"2024-04-25T10:18:03.693408+0000","last_undegraded":"2024-04-25T10:18:03.693408+0000","last_fullsized":"2024-04-25T10:18:03.693408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703521+0000","last_change":"2024-04-25T10:18:01.693364+0000","last_active":"2024-04-25T10:18:03.703521+0000","last_peered":"2024-04-25T10:18:03.703521+0000","last_clean":"2024-04-25T10:18:03.703521+0000","last_became_active":"2024-04-25T10:18:01.692980+0000","last_became_peered":"2024-04-25T10:18:01.692980+0000","last_unstale":"2024-04-25T10:18:03.703521+0000","last_undegraded":"2024-04-25T10:18:03.703521+0000","last_fullsized":"2024-04-25T10:18:03.703521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:36.676949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693704+0000","last_change":"2024-04-25T10:18:02.704788+0000","last_active":"2024-04-25T10:18:03.693704+0000","last_peered":"2024-04-25T10:18:03.693704+0000","last_clean":"2024-04-25T10:18:03.693704+0000","last_became_active":"2024-04-25T10:18:02.703457+0000","last_became_peered":"2024-04-25T10:18:02.703457+0000","last_unstale":"2024-04-25T10:18:03.693704+0000","last_undegraded":"2024-04-25T10:18:03.693704+0000","last_fullsized":"2024-04-25T10:18:03.693704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:18.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703433+0000","last_change":"2024-04-25T10:18:01.702159+0000","last_active":"2024-04-25T10:18:03.703433+0000","last_peered":"2024-04-25T10:18:03.703433+0000","last_clean":"2024-04-25T10:18:03.703433+0000","last_became_active":"2024-04-25T10:18:01.701049+0000","last_became_peered":"2024-04-25T10:18:01.701049+0000","last_unstale":"2024-04-25T10:18:03.703433+0000","last_undegraded":"2024-04-25T10:18:03.703433+0000","last_fullsized":"2024-04-25T10:18:03.703433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:31.455848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703720+0000","last_change":"2024-04-25T10:18:02.710983+0000","last_active":"2024-04-25T10:18:03.703720+0000","last_peered":"2024-04-25T10:18:03.703720+0000","last_clean":"2024-04-25T10:18:03.703720+0000","last_became_active":"2024-04-25T10:18:02.710479+0000","last_became_peered":"2024-04-25T10:18:02.710479+0000","last_unstale":"2024-04-25T10:18:03.703720+0000","last_undegraded":"2024-04-25T10:18:03.703720+0000","last_fullsized":"2024-04-25T10:18:03.703720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:33.161558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694576+0000","last_change":"2024-04-25T10:18:01.697707+0000","last_active":"2024-04-25T10:18:03.694576+0000","last_peered":"2024-04-25T10:18:03.694576+0000","last_clean":"2024-04-25T10:18:03.694576+0000","last_became_active":"2024-04-25T10:18:01.697322+0000","last_became_peered":"2024-04-25T10:18:01.697322+0000","last_unstale":"2024-04-25T10:18:03.694576+0000","last_undegraded":"2024-04-25T10:18:03.694576+0000","last_fullsized":"2024-04-25T10:18:03.694576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702991+0000","last_change":"2024-04-25T10:18:02.697739+0000","last_active":"2024-04-25T10:18:03.702991+0000","last_peered":"2024-04-25T10:18:03.702991+0000","last_clean":"2024-04-25T10:18:03.702991+0000","last_became_active":"2024-04-25T10:18:02.697420+0000","last_became_peered":"2024-04-25T10:18:02.697420+0000","last_unstale":"2024-04-25T10:18:03.702991+0000","last_undegraded":"2024-04-25T10:18:03.702991+0000","last_fullsized":"2024-04-25T10:18:03.702991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.234532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703868+0000","last_change":"2024-04-25T10:18:01.706705+0000","last_active":"2024-04-25T10:18:03.703868+0000","last_peered":"2024-04-25T10:18:03.703868+0000","last_clean":"2024-04-25T10:18:03.703868+0000","last_became_active":"2024-04-25T10:18:01.705850+0000","last_became_peered":"2024-04-25T10:18:01.705850+0000","last_unstale":"2024-04-25T10:18:03.703868+0000","last_undegraded":"2024-04-25T10:18:03.703868+0000","last_fullsized":"2024-04-25T10:18:03.703868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.828188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.693716+0000","last_change":"2024-04-25T10:18:02.704532+0000","last_active":"2024-04-25T10:18:03.693716+0000","last_peered":"2024-04-25T10:18:03.693716+0000","last_clean":"2024-04-25T10:18:03.693716+0000","last_became_active":"2024-04-25T10:18:02.701511+0000","last_became_peered":"2024-04-25T10:18:02.701511+0000","last_unstale":"2024-04-25T10:18:03.693716+0000","last_undegraded":"2024-04-25T10:18:03.693716+0000","last_fullsized":"2024-04-25T10:18:03.693716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:32.462459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703374+0000","last_change":"2024-04-25T10:18:01.698971+0000","last_active":"2024-04-25T10:18:03.703374+0000","last_peered":"2024-04-25T10:18:03.703374+0000","last_clean":"2024-04-25T10:18:03.703374+0000","last_became_active":"2024-04-25T10:18:01.698719+0000","last_became_peered":"2024-04-25T10:18:01.698719+0000","last_unstale":"2024-04-25T10:18:03.703374+0000","last_undegraded":"2024-04-25T10:18:03.703374+0000","last_fullsized":"2024-04-25T10:18:03.703374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:55.382109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703430+0000","last_change":"2024-04-25T10:18:02.700120+0000","last_active":"2024-04-25T10:18:03.703430+0000","last_peered":"2024-04-25T10:18:03.703430+0000","last_clean":"2024-04-25T10:18:03.703430+0000","last_became_active":"2024-04-25T10:18:02.699859+0000","last_became_peered":"2024-04-25T10:18:02.699859+0000","last_unstale":"2024-04-25T10:18:03.703430+0000","last_undegraded":"2024-04-25T10:18:03.703430+0000","last_fullsized":"2024-04-25T10:18:03.703430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:17.794954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.705253+0000","last_change":"2024-04-25T10:18:01.709433+0000","last_active":"2024-04-25T10:18:03.705253+0000","last_peered":"2024-04-25T10:18:03.705253+0000","last_clean":"2024-04-25T10:18:03.705253+0000","last_became_active":"2024-04-25T10:18:01.709103+0000","last_became_peered":"2024-04-25T10:18:01.709103+0000","last_unstale":"2024-04-25T10:18:03.705253+0000","last_undegraded":"2024-04-25T10:18:03.705253+0000","last_fullsized":"2024-04-25T10:18:03.705253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:59.703487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703245+0000","last_change":"2024-04-25T10:18:02.708776+0000","last_active":"2024-04-25T10:18:03.703245+0000","last_peered":"2024-04-25T10:18:03.703245+0000","last_clean":"2024-04-25T10:18:03.703245+0000","last_became_active":"2024-04-25T10:18:02.708166+0000","last_became_peered":"2024-04-25T10:18:02.708166+0000","last_unstale":"2024-04-25T10:18:03.703245+0000","last_undegraded":"2024-04-25T10:18:03.703245+0000","last_fullsized":"2024-04-25T10:18:03.703245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:03.973278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702969+0000","last_change":"2024-04-25T10:18:01.698359+0000","last_active":"2024-04-25T10:18:03.702969+0000","last_peered":"2024-04-25T10:18:03.702969+0000","last_clean":"2024-04-25T10:18:03.702969+0000","last_became_active":"2024-04-25T10:18:01.697573+0000","last_became_peered":"2024-04-25T10:18:01.697573+0000","last_unstale":"2024-04-25T10:18:03.702969+0000","last_undegraded":"2024-04-25T10:18:03.702969+0000","last_fullsized":"2024-04-25T10:18:03.702969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:30.384066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702843+0000","last_change":"2024-04-25T10:18:02.703801+0000","last_active":"2024-04-25T10:18:03.702843+0000","last_peered":"2024-04-25T10:18:03.702843+0000","last_clean":"2024-04-25T10:18:03.702843+0000","last_became_active":"2024-04-25T10:18:02.703508+0000","last_became_peered":"2024-04-25T10:18:02.703508+0000","last_unstale":"2024-04-25T10:18:03.702843+0000","last_undegraded":"2024-04-25T10:18:03.702843+0000","last_fullsized":"2024-04-25T10:18:03.702843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:35.285726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702956+0000","last_change":"2024-04-25T10:18:01.704197+0000","last_active":"2024-04-25T10:18:03.702956+0000","last_peered":"2024-04-25T10:18:03.702956+0000","last_clean":"2024-04-25T10:18:03.702956+0000","last_became_active":"2024-04-25T10:18:01.703932+0000","last_became_peered":"2024-04-25T10:18:01.703932+0000","last_unstale":"2024-04-25T10:18:03.702956+0000","last_undegraded":"2024-04-25T10:18:03.702956+0000","last_fullsized":"2024-04-25T10:18:03.702956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:40.452243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.702798+0000","last_change":"2024-04-25T10:18:02.702635+0000","last_active":"2024-04-25T10:18:03.702798+0000","last_peered":"2024-04-25T10:18:03.702798+0000","last_clean":"2024-04-25T10:18:03.702798+0000","last_became_active":"2024-04-25T10:18:02.702335+0000","last_became_peered":"2024-04-25T10:18:02.702335+0000","last_unstale":"2024-04-25T10:18:03.702798+0000","last_undegraded":"2024-04-25T10:18:03.702798+0000","last_fullsized":"2024-04-25T10:18:03.702798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:20.552276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694476+0000","last_change":"2024-04-25T10:18:01.692670+0000","last_active":"2024-04-25T10:18:03.694476+0000","last_peered":"2024-04-25T10:18:03.694476+0000","last_clean":"2024-04-25T10:18:03.694476+0000","last_became_active":"2024-04-25T10:18:01.691338+0000","last_became_peered":"2024-04-25T10:18:01.691338+0000","last_unstale":"2024-04-25T10:18:03.694476+0000","last_undegraded":"2024-04-25T10:18:03.694476+0000","last_fullsized":"2024-04-25T10:18:03.694476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:23.804861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703461+0000","last_change":"2024-04-25T10:18:02.702741+0000","last_active":"2024-04-25T10:18:03.703461+0000","last_peered":"2024-04-25T10:18:03.703461+0000","last_clean":"2024-04-25T10:18:03.703461+0000","last_became_active":"2024-04-25T10:18:02.702299+0000","last_became_peered":"2024-04-25T10:18:02.702299+0000","last_unstale":"2024-04-25T10:18:03.703461+0000","last_undegraded":"2024-04-25T10:18:03.703461+0000","last_fullsized":"2024-04-25T10:18:03.703461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703382+0000","last_change":"2024-04-25T10:18:01.686488+0000","last_active":"2024-04-25T10:18:03.703382+0000","last_peered":"2024-04-25T10:18:03.703382+0000","last_clean":"2024-04-25T10:18:03.703382+0000","last_became_active":"2024-04-25T10:18:01.685599+0000","last_became_peered":"2024-04-25T10:18:01.685599+0000","last_unstale":"2024-04-25T10:18:03.703382+0000","last_undegraded":"2024-04-25T10:18:03.703382+0000","last_fullsized":"2024-04-25T10:18:03.703382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:50.835330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703213+0000","last_change":"2024-04-25T10:18:02.705187+0000","last_active":"2024-04-25T10:18:03.703213+0000","last_peered":"2024-04-25T10:18:03.703213+0000","last_clean":"2024-04-25T10:18:03.703213+0000","last_became_active":"2024-04-25T10:18:02.704486+0000","last_became_peered":"2024-04-25T10:18:02.704486+0000","last_unstale":"2024-04-25T10:18:03.703213+0000","last_undegraded":"2024-04-25T10:18:03.703213+0000","last_fullsized":"2024-04-25T10:18:03.703213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:14.560066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703616+0000","last_change":"2024-04-25T10:18:02.716723+0000","last_active":"2024-04-25T10:18:03.703616+0000","last_peered":"2024-04-25T10:18:03.703616+0000","last_clean":"2024-04-25T10:18:03.703616+0000","last_became_active":"2024-04-25T10:18:02.716264+0000","last_became_peered":"2024-04-25T10:18:02.716264+0000","last_unstale":"2024-04-25T10:18:03.703616+0000","last_undegraded":"2024-04-25T10:18:03.703616+0000","last_fullsized":"2024-04-25T10:18:03.703616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.581498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704370+0000","last_change":"2024-04-25T10:18:01.698545+0000","last_active":"2024-04-25T10:18:03.704370+0000","last_peered":"2024-04-25T10:18:03.704370+0000","last_clean":"2024-04-25T10:18:03.704370+0000","last_became_active":"2024-04-25T10:18:01.696867+0000","last_became_peered":"2024-04-25T10:18:01.696867+0000","last_unstale":"2024-04-25T10:18:03.704370+0000","last_undegraded":"2024-04-25T10:18:03.704370+0000","last_fullsized":"2024-04-25T10:18:03.704370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.691426+0000","last_change":"2024-04-25T10:18:02.712672+0000","last_active":"2024-04-25T10:18:03.691426+0000","last_peered":"2024-04-25T10:18:03.691426+0000","last_clean":"2024-04-25T10:18:03.691426+0000","last_became_active":"2024-04-25T10:18:02.712103+0000","last_became_peered":"2024-04-25T10:18:02.712103+0000","last_unstale":"2024-04-25T10:18:03.691426+0000","last_undegraded":"2024-04-25T10:18:03.691426+0000","last_fullsized":"2024-04-25T10:18:03.691426+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:18.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.704949+0000","last_change":"2024-04-25T10:18:01.704511+0000","last_active":"2024-04-25T10:18:03.704949+0000","last_peered":"2024-04-25T10:18:03.704949+0000","last_clean":"2024-04-25T10:18:03.704949+0000","last_became_active":"2024-04-25T10:18:01.703001+0000","last_became_peered":"2024-04-25T10:18:01.703001+0000","last_unstale":"2024-04-25T10:18:03.704949+0000","last_undegraded":"2024-04-25T10:18:03.704949+0000","last_fullsized":"2024-04-25T10:18:03.704949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.227921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694997+0000","last_change":"2024-04-25T10:18:02.690317+0000","last_active":"2024-04-25T10:18:03.694997+0000","last_peered":"2024-04-25T10:18:03.694997+0000","last_clean":"2024-04-25T10:18:03.694997+0000","last_became_active":"2024-04-25T10:18:02.687277+0000","last_became_peered":"2024-04-25T10:18:02.687277+0000","last_unstale":"2024-04-25T10:18:03.694997+0000","last_undegraded":"2024-04-25T10:18:03.694997+0000","last_fullsized":"2024-04-25T10:18:03.694997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.438653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694980+0000","last_change":"2024-04-25T10:18:01.688261+0000","last_active":"2024-04-25T10:18:03.694980+0000","last_peered":"2024-04-25T10:18:03.694980+0000","last_clean":"2024-04-25T10:18:03.694980+0000","last_became_active":"2024-04-25T10:18:01.687516+0000","last_became_peered":"2024-04-25T10:18:01.687516+0000","last_unstale":"2024-04-25T10:18:03.694980+0000","last_undegraded":"2024-04-25T10:18:03.694980+0000","last_fullsized":"2024-04-25T10:18:03.694980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:05.082872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.694903+0000","last_change":"2024-04-25T10:18:02.687787+0000","last_active":"2024-04-25T10:18:03.694903+0000","last_peered":"2024-04-25T10:18:03.694903+0000","last_clean":"2024-04-25T10:18:03.694903+0000","last_became_active":"2024-04-25T10:18:02.687351+0000","last_became_peered":"2024-04-25T10:18:02.687351+0000","last_unstale":"2024-04-25T10:18:03.694903+0000","last_undegraded":"2024-04-25T10:18:03.694903+0000","last_fullsized":"2024-04-25T10:18:03.694903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:01.662128+0000","last_clean_scrub_stamp":"2024-04-25T10:18:01.662128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.305229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-25T10:18:03.703546+0000","last_change":"2024-04-25T10:18:01.710038+0000","last_active":"2024-04-25T10:18:03.703546+0000","last_peered":"2024-04-25T10:18:03.703546+0000","last_clean":"2024-04-25T10:18:03.703546+0000","last_became_active":"2024-04-25T10:18:01.709789+0000","last_became_peered":"2024-04-25T10:18:01.709789+0000","last_unstale":"2024-04-25T10:18:03.703546+0000","last_undegraded":"2024-04-25T10:18:03.703546+0000","last_fullsized":"2024-04-25T10:18:03.703546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:00.655517+0000","last_clean_scrub_stamp":"2024-04-25T10:18:00.655517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:49.219702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:12.205 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:18:12.205 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:18:12.205 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:18:12.205 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:18:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 Processor -- start 2024-04-25T10:18:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 -- start start 2024-04-25T10:18:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcc4c15df60 0x7fcc4c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c0d0180 0x7fcc4c0d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc4c0cdb40 0x7fcc4c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c15e340 con 0x7fcc4c0cdb40 2024-04-25T10:18:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0d0b20 con 0x7fcc4c15df60 2024-04-25T10:18:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.377+0000 7fcc50e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0ccaa0 con 0x7fcc4c0d0180 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc4c0cdb40 0x7fcc4c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcc4c0cdb40 0x7fcc4c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57554/0 (socket says 172.21.15.45:57554) 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 -- 172.21.15.45:0/3984061451 learned_addr learned my addr 172.21.15.45:0/3984061451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc49e00640 1 --2- 172.21.15.45:0/3984061451 >> v2:172.21.15.117:3300/0 conn(0x7fcc4c15df60 0x7fcc4c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc43e00640 1 --2- 172.21.15.45:0/3984061451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c0d0180 0x7fcc4c0d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 -- 172.21.15.45:0/3984061451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c0d0180 msgr2=0x7fcc4c0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 --2- 172.21.15.45:0/3984061451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c0d0180 0x7fcc4c0d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 -- 172.21.15.45:0/3984061451 >> v2:172.21.15.117:3300/0 conn(0x7fcc4c15df60 msgr2=0x7fcc4c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 --2- 172.21.15.45:0/3984061451 >> v2:172.21.15.117:3300/0 conn(0x7fcc4c15df60 0x7fcc4c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 -- 172.21.15.45:0/3984061451 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0ccd80 con 0x7fcc4c0cdb40 2024-04-25T10:18:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc4a800640 1 --2- 172.21.15.45:0/3984061451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c0cdfb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c067740 tx=0x7fcc4401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cad020ad20e973f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc49400640 1 -- 172.21.15.45:0/3984061451 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440a56d0 con 0x7fcc4c0cdb40 2024-04-25T10:18:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc49400640 1 -- 172.21.15.45:0/3984061451 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc440a5870 con 0x7fcc4c0cdb40 2024-04-25T10:18:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.381+0000 7fcc49400640 1 -- 172.21.15.45:0/3984061451 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440a5c40 con 0x7fcc4c0cdb40 2024-04-25T10:18:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- 172.21.15.45:0/3984061451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0cdb40 msgr2=0x7fcc4c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- 172.21.15.45:0/3984061451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c0cdfb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c067740 tx=0x7fcc4401c960 comp rx=0 tx=0).stop 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- 172.21.15.45:0/3984061451 shutdown_connections 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- 172.21.15.45:0/3984061451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c0cdfb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- 172.21.15.45:0/3984061451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c0d0180 0x7fcc4c0d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- 172.21.15.45:0/3984061451 >> v2:172.21.15.117:3300/0 conn(0x7fcc4c15df60 0x7fcc4c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- 172.21.15.45:0/3984061451 >> 172.21.15.45:0/3984061451 conn(0x7fcc4c0d1910 msgr2=0x7fcc4c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- 172.21.15.45:0/3984061451 shutdown_connections 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- 172.21.15.45:0/3984061451 wait complete. 2024-04-25T10:18:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 Processor -- start 2024-04-25T10:18:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- start start 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 0x7fcc4c0b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c15df60 0x7fcc4c0b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0d0ff0 con 0x7fcc4c0d0180 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0d0e70 con 0x7fcc4c0cdb40 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc50e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c06aea0 con 0x7fcc4c15df60 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 0x7fcc4c0b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 0x7fcc4c0b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57556/0 (socket says 172.21.15.45:57556) 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 -- 172.21.15.45:0/2802992943 learned_addr learned my addr 172.21.15.45:0/2802992943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc49e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc4a800640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c15df60 0x7fcc4c0b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 -- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c15df60 msgr2=0x7fcc4c0b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c15df60 0x7fcc4c0b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 -- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc4c0cdb40 msgr2=0x7fcc4c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 -- 172.21.15.45:0/2802992943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0b74f0 con 0x7fcc4c0d0180 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc49e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc4a800640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c15df60 0x7fcc4c0b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.385+0000 7fcc43e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 0x7fcc4c0b6a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e960 tx=0x7fcc3400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34017070 con 0x7fcc4c0d0180 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3400ca00 con 0x7fcc4c0d0180 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400cdb0 con 0x7fcc4c0d0180 2024-04-25T10:18:12.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c154420 con 0x7fcc4c0d0180 2024-04-25T10:18:12.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c154cc0 con 0x7fcc4c0d0180 2024-04-25T10:18:12.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c04ef30 con 0x7fcc4c0d0180 2024-04-25T10:18:12.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc3401a070 con 0x7fcc4c0d0180 2024-04-25T10:18:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc42000640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc28066f60 0x7fcc28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc34085860 con 0x7fcc4c0d0180 2024-04-25T10:18:12.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.389+0000 7fcc49e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc28066f60 0x7fcc28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.393+0000 7fcc49e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc28066f60 0x7fcc28069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0099c0 tx=0x7fcc3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.393+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc34056850 con 0x7fcc4c0d0180 2024-04-25T10:18:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.589+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcc4c0cff60 con 0x7fcc4c0d0180 2024-04-25T10:18:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc42000640 1 -- 172.21.15.45:0/2802992943 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcc34056150 con 0x7fcc4c0d0180 2024-04-25T10:18:12.594 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:12.594 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc28066f60 msgr2=0x7fcc28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc28066f60 0x7fcc28069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0099c0 tx=0x7fcc3c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 msgr2=0x7fcc4c0b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 0x7fcc4c0b6a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e960 tx=0x7fcc3400ee20 comp rx=0 tx=0).stop 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 shutdown_connections 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc28066f60 0x7fcc28069420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc4c15df60 0x7fcc4c0b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc4c0d0180 0x7fcc4c0b6a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 --2- 172.21.15.45:0/2802992943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc4c0cdb40 0x7fcc4c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 >> 172.21.15.45:0/2802992943 conn(0x7fcc4c0d1910 msgr2=0x7fcc4c06ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 shutdown_connections 2024-04-25T10:18:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.593+0000 7fcc50e00640 1 -- 172.21.15.45:0/2802992943 wait complete. 2024-04-25T10:18:12.615 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:18:12.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.753+0000 7f4ad4000640 1 Processor -- start 2024-04-25T10:18:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.753+0000 7f4ad4000640 1 -- start start 2024-04-25T10:18:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad4000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4acc0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad4000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc0d15e0 con 0x7f4acc15f570 2024-04-25T10:18:12.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad4000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc0d0140 con 0x7f4acc15f190 2024-04-25T10:18:12.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad4000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0cfef0 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad2000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4acc15f570 0x7f4acc0ccb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59326/0 (socket says 172.21.15.45:59326) 2024-04-25T10:18:12.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad2000640 1 -- 172.21.15.45:0/381624001 learned_addr learned my addr 172.21.15.45:0/381624001 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad0c00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4acc0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654292403 0 0) 0x7f4ac40014a0 con 0x7f4acc15f570 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab4003530 con 0x7f4acc15f570 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785324973 0 0) 0x7f4acc0d0140 con 0x7f4acc15f190 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac40014a0 con 0x7f4acc15f190 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078316083 0 0) 0x7f4ab4003530 con 0x7f4acc15f570 2024-04-25T10:18:12.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 msgr2=0x7f4acc0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4acc0cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 >> v1:172.21.15.117:6789/0 conn(0x7f4acc15f190 legacy=0x7f4acc0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc0d0420 con 0x7f4acc15f570 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f4ac4001bd0 con 0x7f4acc15f570 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 >> v1:172.21.15.45:6789/0 conn(0x7f4acc15f570 legacy=0x7f4acc0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4ab4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ab40040e0 0x7f4ab4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab4004d50 0x7f4ab40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0cfef0 con 0x7f4ab40040e0 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab400bbe0 con 0x7f4ab4004d50 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40101f0 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad1600640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4ab4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad0c00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ab40040e0 0x7f4ab4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad2000640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab4004d50 0x7f4ab40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad1600640 1 -- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab4004d50 msgr2=0x7f4ab40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad1600640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ab4004d50 0x7f4ab40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad1600640 1 -- 172.21.15.45:0/381624001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ab40040e0 msgr2=0x7f4ab4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad1600640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ab40040e0 0x7f4ab4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad1600640 1 -- 172.21.15.45:0/381624001 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4003530 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.757+0000 7f4ad0c00640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ab40040e0 0x7f4ab4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad1600640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4ab4003b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00018d0 tx=0x7f4ac001c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f826e935a4c220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00af070 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac00052f0 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4acbe00640 1 -- 172.21.15.45:0/381624001 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0001520 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad4000640 1 -- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 msgr2=0x7f4ab4003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad4000640 1 --2- 172.21.15.45:0/381624001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0ce9a0 0x7f4ab4003b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00018d0 tx=0x7f4ac001c550 comp rx=0 tx=0).stop 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad2000640 1 -- 172.21.15.45:0/381624001 reap_dead start 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad4000640 1 -- 172.21.15.45:0/381624001 shutdown_connections 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad4000640 1 -- 172.21.15.45:0/381624001 >> 172.21.15.45:0/381624001 conn(0x7f4acc0d1910 msgr2=0x7f4acc0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad4000640 1 -- 172.21.15.45:0/381624001 shutdown_connections 2024-04-25T10:18:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.761+0000 7f4ad4000640 1 -- 172.21.15.45:0/381624001 wait complete. 2024-04-25T10:18:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 Processor -- start 2024-04-25T10:18:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- start start 2024-04-25T10:18:12.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc0ce9a0 0x7f4acc0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 0x7f4acc157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0b7220 0x7f4acc158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0d25b0 con 0x7f4acc0b6da0 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc0d2430 con 0x7f4acc0ce9a0 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc06ac90 con 0x7f4acc0b7220 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 0x7f4acc157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 0x7f4acc157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57564/0 (socket says 172.21.15.45:57564) 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 -- 172.21.15.45:0/3273082536 learned_addr learned my addr 172.21.15.45:0/3273082536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad2000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0b7220 0x7f4acc158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad1600640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc0ce9a0 0x7f4acc0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 -- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0b7220 msgr2=0x7f4acc158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0b7220 0x7f4acc158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 -- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc0ce9a0 msgr2=0x7f4acc0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc0ce9a0 0x7f4acc0b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 -- 172.21.15.45:0/3273082536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc0b50c0 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad0c00640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 0x7f4acc157b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8011d00 tx=0x7f4ab800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800fd20 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8004020 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80043f0 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc0b53a0 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc160ac0 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.765+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc04ef30 con 0x7f4acc0b6da0 2024-04-25T10:18:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.769+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ab8012040 con 0x7f4acc0b6da0 2024-04-25T10:18:12.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.769+0000 7f4ac9600640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4aac066f60 0x7f4aac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:12.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.769+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ab8085560 con 0x7f4acc0b6da0 2024-04-25T10:18:12.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.769+0000 7f4ad1600640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4aac066f60 0x7f4aac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:12.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.769+0000 7f4ad1600640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4aac066f60 0x7f4aac069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00ac000 tx=0x7f4ac00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:12.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.773+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ab8019030 con 0x7f4acc0b6da0 2024-04-25T10:18:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.937+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4acc0730b0 con 0x7f4acc0b6da0 2024-04-25T10:18:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.937+0000 7f4ac9600640 1 -- 172.21.15.45:0/3273082536 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f4ab8055f20 con 0x7f4acc0b6da0 2024-04-25T10:18:12.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:12.940 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:12.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.937+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4aac066f60 msgr2=0x7f4aac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.937+0000 7f4ad4000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4aac066f60 0x7f4aac069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00ac000 tx=0x7f4ac00a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:12.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.937+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 msgr2=0x7f4acc157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:12.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.937+0000 7f4ad4000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 0x7f4acc157b10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8011d00 tx=0x7f4ab800f610 comp rx=0 tx=0).stop 2024-04-25T10:18:12.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 shutdown_connections 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4aac066f60 0x7f4aac069420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4acc0b7220 0x7f4acc158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4acc0b6da0 0x7f4acc157b10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 --2- 172.21.15.45:0/3273082536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc0ce9a0 0x7f4acc0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 >> 172.21.15.45:0/3273082536 conn(0x7f4acc0d1910 msgr2=0x7f4acc06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 shutdown_connections 2024-04-25T10:18:12.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:12.941+0000 7f4ad4000640 1 -- 172.21.15.45:0/3273082536 wait complete. 2024-04-25T10:18:12.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:12.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:13.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.101+0000 7f9624e00640 1 Processor -- start 2024-04-25T10:18:13.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.101+0000 7f9624e00640 1 -- start start 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9624e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f96200aef80 0x7f962015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9624e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96200b31c0 0x7f96200b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9624e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962015b080 con 0x7f9620070350 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9624e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962015adb0 con 0x7f96200b31c0 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9624e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962015eb60 con 0x7f96200aef80 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96200b31c0 0x7f96200b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9617e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9620070350 0x7f962015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59332/0 (socket says 172.21.15.45:59332) 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f9617e00640 1 -- 172.21.15.45:0/3232014624 learned_addr learned my addr 172.21.15.45:0/3232014624 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96200b31c0 0x7f96200b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57574/0 (socket says 172.21.15.45:57574) 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961de00640 1 --2- 172.21.15.45:0/3232014624 >> v2:172.21.15.117:3300/0 conn(0x7f96200aef80 0x7f962015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 -- 172.21.15.45:0/3232014624 >> v2:172.21.15.117:3300/0 conn(0x7f96200aef80 msgr2=0x7f962015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 --2- 172.21.15.45:0/3232014624 >> v2:172.21.15.117:3300/0 conn(0x7f96200aef80 0x7f962015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 -- 172.21.15.45:0/3232014624 >> v1:172.21.15.45:6789/0 conn(0x7f9620070350 legacy=0x7f962015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961d400640 1 -- 172.21.15.45:0/3232014624 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834351803 0 0) 0x7f962015b080 con 0x7f9620070350 2024-04-25T10:18:13.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 -- 172.21.15.45:0/3232014624 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962015ee40 con 0x7f96200b31c0 2024-04-25T10:18:13.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961e800640 1 --2- 172.21.15.45:0/3232014624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96200b31c0 0x7f96200b3610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f962006e220 tx=0x7f961801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47326eff636223fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.105+0000 7f961d400640 1 -- 172.21.15.45:0/3232014624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96180b2070 con 0x7f96200b31c0 2024-04-25T10:18:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f961d400640 1 -- 172.21.15.45:0/3232014624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96180a5900 con 0x7f96200b31c0 2024-04-25T10:18:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f961d400640 1 -- 172.21.15.45:0/3232014624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96180a5cd0 con 0x7f96200b31c0 2024-04-25T10:18:13.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- 172.21.15.45:0/3232014624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96200b31c0 msgr2=0x7f96200b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 --2- 172.21.15.45:0/3232014624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96200b31c0 0x7f96200b3610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f962006e220 tx=0x7f961801c930 comp rx=0 tx=0).stop 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- 172.21.15.45:0/3232014624 shutdown_connections 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 --2- 172.21.15.45:0/3232014624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96200b31c0 0x7f96200b3610 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 --2- 172.21.15.45:0/3232014624 >> v2:172.21.15.117:3300/0 conn(0x7f96200aef80 0x7f962015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- 172.21.15.45:0/3232014624 >> 172.21.15.45:0/3232014624 conn(0x7f96200669a0 msgr2=0x7f9620066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- 172.21.15.45:0/3232014624 shutdown_connections 2024-04-25T10:18:13.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- 172.21.15.45:0/3232014624 wait complete. 2024-04-25T10:18:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 Processor -- start 2024-04-25T10:18:13.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- start start 2024-04-25T10:18:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 0x7f9620158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96200aef80 0x7f96201f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96200b31c0 0x7f96200adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962006c250 con 0x7f9620070350 2024-04-25T10:18:13.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962006c0d0 con 0x7f96200aef80 2024-04-25T10:18:13.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9624e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962006c3d0 con 0x7f96200b31c0 2024-04-25T10:18:13.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f961de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 0x7f9620158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f961de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 0x7f9620158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57582/0 (socket says 172.21.15.45:57582) 2024-04-25T10:18:13.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f961de00640 1 -- 172.21.15.45:0/3759225854 learned_addr learned my addr 172.21.15.45:0/3759225854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f961e800640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96200b31c0 0x7f96200adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.109+0000 7f9617e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96200aef80 0x7f96201f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f961de00640 1 -- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96200b31c0 msgr2=0x7f96200adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f961de00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96200b31c0 0x7f96200adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f961de00640 1 -- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96200aef80 msgr2=0x7f96201f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f961de00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96200aef80 0x7f96201f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f961de00640 1 -- 172.21.15.45:0/3759225854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9620159420 con 0x7f9620070350 2024-04-25T10:18:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f961de00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 0x7f9620158bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f961000c7f0 tx=0x7f961000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610015070 con 0x7f9620070350 2024-04-25T10:18:13.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610004240 con 0x7f9620070350 2024-04-25T10:18:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610003620 con 0x7f9620070350 2024-04-25T10:18:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96201572c0 con 0x7f9620070350 2024-04-25T10:18:13.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9620157880 con 0x7f9620070350 2024-04-25T10:18:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9620157a80 con 0x7f9620070350 2024-04-25T10:18:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96100039d0 con 0x7f9620070350 2024-04-25T10:18:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9616000640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95fc066f60 0x7f95fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96100850d0 con 0x7f9620070350 2024-04-25T10:18:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.113+0000 7f9617e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95fc066f60 0x7f95fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.117+0000 7f9617e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95fc066f60 0x7f95fc069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9608000f70 tx=0x7f9608007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.117+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9620157a80 con 0x7f9620070350 2024-04-25T10:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96200b2fa0 con 0x7f9620070350 2024-04-25T10:18:13.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9616000640 1 -- 172.21.15.45:0/3759225854 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96100560d0 con 0x7f9620070350 2024-04-25T10:18:13.283 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:13.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95fc066f60 msgr2=0x7f95fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95fc066f60 0x7f95fc069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9608000f70 tx=0x7f9608007b60 comp rx=0 tx=0).stop 2024-04-25T10:18:13.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 msgr2=0x7f9620158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 0x7f9620158bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f961000c7f0 tx=0x7f961000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 shutdown_connections 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95fc066f60 0x7f95fc069420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96200b31c0 0x7f96200adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96200aef80 0x7f96201f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 --2- 172.21.15.45:0/3759225854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9620070350 0x7f9620158bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 >> 172.21.15.45:0/3759225854 conn(0x7f96200669a0 msgr2=0x7f962015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 shutdown_connections 2024-04-25T10:18:13.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.281+0000 7f9624e00640 1 -- 172.21.15.45:0/3759225854 wait complete. 2024-04-25T10:18:13.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:13.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 Processor -- start 2024-04-25T10:18:13.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 -- start start 2024-04-25T10:18:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1adc0aef80 0x7f1adc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc15b080 con 0x7f1adc15bac0 2024-04-25T10:18:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc15adb0 con 0x7f1adc0aef80 2024-04-25T10:18:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ae2400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc15eb60 con 0x7f1adc070350 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adb400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1adc0aef80 0x7f1adc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1ad2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adbe00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1adc15bac0 0x7f1adc0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32904/0 (socket says 172.21.15.45:32904) 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adbe00640 1 -- 172.21.15.45:0/1630373917 learned_addr learned my addr 172.21.15.45:0/1630373917 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adaa00640 1 -- 172.21.15.45:0/1630373917 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234649364 0 0) 0x7f1adc15b080 con 0x7f1adc15bac0 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adb400640 1 -- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 msgr2=0x7f1adc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adb400640 1 --2- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adb400640 1 -- 172.21.15.45:0/1630373917 >> v1:172.21.15.117:6789/0 conn(0x7f1adc15bac0 legacy=0x7f1adc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.445+0000 7f1adb400640 1 -- 172.21.15.45:0/1630373917 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc15ee40 con 0x7f1adc0aef80 2024-04-25T10:18:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1adb400640 1 --2- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0aef80 0x7f1adc15e480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0097c0 tx=0x7f1acc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237451f2c58f6f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1adaa00640 1 -- 172.21.15.45:0/1630373917 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0ab680 con 0x7f1adc0aef80 2024-04-25T10:18:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1adaa00640 1 -- 172.21.15.45:0/1630373917 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc0b0070 con 0x7f1adc0aef80 2024-04-25T10:18:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1adaa00640 1 -- 172.21.15.45:0/1630373917 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0aba40 con 0x7f1adc0aef80 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 -- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0aef80 msgr2=0x7f1adc15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 --2- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0aef80 0x7f1adc15e480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0097c0 tx=0x7f1acc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 -- 172.21.15.45:0/1630373917 shutdown_connections 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 --2- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 --2- 172.21.15.45:0/1630373917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc0aef80 0x7f1adc15e480 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 -- 172.21.15.45:0/1630373917 >> 172.21.15.45:0/1630373917 conn(0x7f1adc0669a0 msgr2=0x7f1adc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 -- 172.21.15.45:0/1630373917 shutdown_connections 2024-04-25T10:18:13.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.449+0000 7f1ae2400640 1 -- 172.21.15.45:0/1630373917 wait complete. 2024-04-25T10:18:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 Processor -- start 2024-04-25T10:18:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- start start 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 0x7f1adc15fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc15bac0 0x7f1adc0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc167820 con 0x7f1adc0aef80 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc0b3b20 con 0x7f1adc15bac0 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc1679a0 con 0x7f1adc070350 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 0x7f1adc15fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 0x7f1adc15fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57592/0 (socket says 172.21.15.45:57592) 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 -- 172.21.15.45:0/2737373462 learned_addr learned my addr 172.21.15.45:0/2737373462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1adb400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1adbe00640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc15bac0 0x7f1adc0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 -- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 msgr2=0x7f1adc15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 -- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc15bac0 msgr2=0x7f1adc0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc15bac0 0x7f1adc0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 -- 172.21.15.45:0/2737373462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc1613a0 con 0x7f1adc0aef80 2024-04-25T10:18:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad2a00640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 0x7f1adc15fa60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400e930 tx=0x7f1ac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4017070 con 0x7f1adc0aef80 2024-04-25T10:18:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac400ca00 con 0x7f1adc0aef80 2024-04-25T10:18:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc161680 con 0x7f1adc0aef80 2024-04-25T10:18:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400cdb0 con 0x7f1adc0aef80 2024-04-25T10:18:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc156380 con 0x7f1adc0aef80 2024-04-25T10:18:13.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.453+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc04ef30 con 0x7f1adc0aef80 2024-04-25T10:18:13.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.457+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ac401a070 con 0x7f1adc0aef80 2024-04-25T10:18:13.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.457+0000 7f1ad3e00640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ab8066f90 0x7f1ab8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.457+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ac4085370 con 0x7f1adc0aef80 2024-04-25T10:18:13.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.457+0000 7f1adb400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ab8066f90 0x7f1ab8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.457+0000 7f1adb400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ab8066f90 0x7f1ab8069450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0af000 tx=0x7f1acc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.461+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ac4055de0 con 0x7f1adc0aef80 2024-04-25T10:18:13.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.625+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1adc067030 con 0x7f1adc0aef80 2024-04-25T10:18:13.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.625+0000 7f1ad3e00640 1 -- 172.21.15.45:0/2737373462 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ac4059e80 con 0x7f1adc0aef80 2024-04-25T10:18:13.629 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:13.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ab8066f90 msgr2=0x7f1ab8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ab8066f90 0x7f1ab8069450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0af000 tx=0x7f1acc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:13.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 msgr2=0x7f1adc15fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 0x7f1adc15fa60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400e930 tx=0x7f1ac400edf0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 shutdown_connections 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ab8066f90 0x7f1ab8069450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1adc15bac0 0x7f1adc0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1adc0aef80 0x7f1adc15fa60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 --2- 172.21.15.45:0/2737373462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1adc070350 0x7f1adc15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 >> 172.21.15.45:0/2737373462 conn(0x7f1adc0669a0 msgr2=0x7f1adc15d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 shutdown_connections 2024-04-25T10:18:13.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.629+0000 7f1ae2400640 1 -- 172.21.15.45:0/2737373462 wait complete. 2024-04-25T10:18:13.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61da200640 1 Processor -- start 2024-04-25T10:18:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61da200640 1 -- start start 2024-04-25T10:18:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61da200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61d4159540 0x7f61d40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61da200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d4159920 con 0x7f61d40aebd0 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61da200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d415c2f0 con 0x7f61d40721f0 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61da200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d415c5d0 con 0x7f61d4159540 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f61d40aebd0 0x7f61d4071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59338/0 (socket says 172.21.15.45:59338) 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d2a00640 1 -- 172.21.15.45:0/455011360 learned_addr learned my addr 172.21.15.45:0/455011360 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d3400640 1 --2- 172.21.15.45:0/455011360 >> v2:172.21.15.117:3300/0 conn(0x7f61d4159540 0x7f61d40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d2000640 1 -- 172.21.15.45:0/455011360 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96070676 0 0) 0x7f61d4159920 con 0x7f61d40aebd0 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d2000640 1 -- 172.21.15.45:0/455011360 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8003530 con 0x7f61d40aebd0 2024-04-25T10:18:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d3400640 1 -- 172.21.15.45:0/455011360 >> v1:172.21.15.117:6789/0 conn(0x7f61d40721f0 legacy=0x7f61d4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d3400640 1 -- 172.21.15.45:0/455011360 >> v1:172.21.15.45:6789/0 conn(0x7f61d40aebd0 legacy=0x7f61d4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.793+0000 7f61d3400640 1 -- 172.21.15.45:0/455011360 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d415c8b0 con 0x7f61d4159540 2024-04-25T10:18:13.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61d3400640 1 --2- 172.21.15.45:0/455011360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4159540 0x7f61d40ae690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61c40097c0 tx=0x7f61c401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c80280851acf6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61d2000640 1 -- 172.21.15.45:0/455011360 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40b1070 con 0x7f61d4159540 2024-04-25T10:18:13.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61d2000640 1 -- 172.21.15.45:0/455011360 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c40ac410 con 0x7f61d4159540 2024-04-25T10:18:13.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61d2000640 1 -- 172.21.15.45:0/455011360 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40ac7e0 con 0x7f61d4159540 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 -- 172.21.15.45:0/455011360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4159540 msgr2=0x7f61d40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 --2- 172.21.15.45:0/455011360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4159540 0x7f61d40ae690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61c40097c0 tx=0x7f61c401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 -- 172.21.15.45:0/455011360 shutdown_connections 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 --2- 172.21.15.45:0/455011360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d4159540 0x7f61d40ae690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 -- 172.21.15.45:0/455011360 >> 172.21.15.45:0/455011360 conn(0x7f61d40669a0 msgr2=0x7f61d4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 -- 172.21.15.45:0/455011360 shutdown_connections 2024-04-25T10:18:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 -- 172.21.15.45:0/455011360 wait complete. 2024-04-25T10:18:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.797+0000 7f61da200640 1 Processor -- start 2024-04-25T10:18:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- start start 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d40721f0 0x7f61d414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 0x7f61d4151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d4159540 0x7f61d4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d4169ac0 con 0x7f61d40aebd0 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d4169940 con 0x7f61d40721f0 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d4169c40 con 0x7f61d4159540 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 0x7f61d4151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d40721f0 0x7f61d414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 0x7f61d4151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57604/0 (socket says 172.21.15.45:57604) 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 -- 172.21.15.45:0/3215155923 learned_addr learned my addr 172.21.15.45:0/3215155923 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d3e00640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d4159540 0x7f61d4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 -- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d4159540 msgr2=0x7f61d4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d4159540 0x7f61d4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 -- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d40721f0 msgr2=0x7f61d414cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d40721f0 0x7f61d414cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 -- 172.21.15.45:0/3215155923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4150e00 con 0x7f61d40aebd0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d3400640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d40721f0 0x7f61d414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61d2a00640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 0x7f61d4151d00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c4d0 tx=0x7f61c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0017070 con 0x7f61d40aebd0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0005cf0 con 0x7f61d40aebd0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d41510e0 con 0x7f61d40aebd0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0007270 con 0x7f61d40aebd0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d4152240 con 0x7f61d40aebd0 2024-04-25T10:18:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.801+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d404ef30 con 0x7f61d40aebd0 2024-04-25T10:18:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.805+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61c001a070 con 0x7f61d40aebd0 2024-04-25T10:18:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.805+0000 7f61cb400640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61b4066f60 0x7f61b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.805+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61c0084830 con 0x7f61d40aebd0 2024-04-25T10:18:13.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.805+0000 7f61d3400640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61b4066f60 0x7f61b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:13.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.805+0000 7f61d3400640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61b4066f60 0x7f61b4069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ae000 tx=0x7f61c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.809+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61c00551d0 con 0x7f61d40aebd0 2024-04-25T10:18:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.969+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61d406d7f0 con 0x7f61d40aebd0 2024-04-25T10:18:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.969+0000 7f61cb400640 1 -- 172.21.15.45:0/3215155923 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61c0059270 con 0x7f61d40aebd0 2024-04-25T10:18:13.972 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61b4066f60 msgr2=0x7f61b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61b4066f60 0x7f61b4069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ae000 tx=0x7f61c4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 msgr2=0x7f61d4151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 0x7f61d4151d00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c4d0 tx=0x7f61c000c990 comp rx=0 tx=0).stop 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 shutdown_connections 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61b4066f60 0x7f61b4069420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d4159540 0x7f61d4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d40aebd0 0x7f61d4151d00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 --2- 172.21.15.45:0/3215155923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d40721f0 0x7f61d414cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 >> 172.21.15.45:0/3215155923 conn(0x7f61d40669a0 msgr2=0x7f61d40717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 shutdown_connections 2024-04-25T10:18:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:13.973+0000 7f61da200640 1 -- 172.21.15.45:0/3215155923 wait complete. 2024-04-25T10:18:13.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.129+0000 7f9597200640 1 Processor -- start 2024-04-25T10:18:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9597200640 1 -- start start 2024-04-25T10:18:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9597200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95900d0910 0x7f9590070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9597200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9590071aa0 0x7f9590071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9597200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95900723f0 con 0x7f9590156010 2024-04-25T10:18:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9597200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959015c350 con 0x7f95900d0910 2024-04-25T10:18:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9597200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959015c000 con 0x7f9590071aa0 2024-04-25T10:18:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95900d0910 0x7f9590070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95900d0910 0x7f9590070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57618/0 (socket says 172.21.15.45:57618) 2024-04-25T10:18:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 -- 172.21.15.45:0/3114666722 learned_addr learned my addr 172.21.15.45:0/3114666722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9596800640 1 --2- 172.21.15.45:0/3114666722 >> v2:172.21.15.117:3300/0 conn(0x7f9590071aa0 0x7f9590071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 -- 172.21.15.45:0/3114666722 >> v2:172.21.15.117:3300/0 conn(0x7f9590071aa0 msgr2=0x7f9590071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 --2- 172.21.15.45:0/3114666722 >> v2:172.21.15.117:3300/0 conn(0x7f9590071aa0 0x7f9590071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 -- 172.21.15.45:0/3114666722 >> v1:172.21.15.117:6789/0 conn(0x7f9590156010 legacy=0x7f959015c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9595400640 1 -- 172.21.15.45:0/3114666722 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900713a0 con 0x7f95900d0910 2024-04-25T10:18:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.133+0000 7f9596800640 1 --2- 172.21.15.45:0/3114666722 >> v2:172.21.15.117:3300/0 conn(0x7f9590071aa0 0x7f9590071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:14.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9595400640 1 --2- 172.21.15.45:0/3114666722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95900d0910 0x7f9590070a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f959015f340 tx=0x7f958001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df62feca3b02bbaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9594a00640 1 -- 172.21.15.45:0/3114666722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95800b0070 con 0x7f95900d0910 2024-04-25T10:18:14.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9594a00640 1 -- 172.21.15.45:0/3114666722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95800ab600 con 0x7f95900d0910 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9594a00640 1 -- 172.21.15.45:0/3114666722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95800ab9d0 con 0x7f95900d0910 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 -- 172.21.15.45:0/3114666722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95900d0910 msgr2=0x7f9590070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 --2- 172.21.15.45:0/3114666722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95900d0910 0x7f9590070a30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f959015f340 tx=0x7f958001c270 comp rx=0 tx=0).stop 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 -- 172.21.15.45:0/3114666722 shutdown_connections 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 --2- 172.21.15.45:0/3114666722 >> v2:172.21.15.117:3300/0 conn(0x7f9590071aa0 0x7f9590071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 --2- 172.21.15.45:0/3114666722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95900d0910 0x7f9590070a30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 -- 172.21.15.45:0/3114666722 >> 172.21.15.45:0/3114666722 conn(0x7f95900b1c30 msgr2=0x7f95900b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 -- 172.21.15.45:0/3114666722 shutdown_connections 2024-04-25T10:18:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 -- 172.21.15.45:0/3114666722 wait complete. 2024-04-25T10:18:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 Processor -- start 2024-04-25T10:18:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.137+0000 7f9597200640 1 -- start start 2024-04-25T10:18:14.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9590071aa0 0x7f95900cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95900d0910 0x7f9590145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 0x7f9590144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b48d0 con 0x7f9590156010 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95900b4750 con 0x7f95900d0910 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9590168450 con 0x7f9590071aa0 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 0x7f9590144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 0x7f9590144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57628/0 (socket says 172.21.15.45:57628) 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 -- 172.21.15.45:0/3736926533 learned_addr learned my addr 172.21.15.45:0/3736926533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9595400640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95900d0910 0x7f9590145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9595e00640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9590071aa0 0x7f95900cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 -- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9590071aa0 msgr2=0x7f95900cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9590071aa0 0x7f95900cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 -- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95900d0910 msgr2=0x7f9590145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95900d0910 0x7f9590145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 -- 172.21.15.45:0/3736926533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900cf740 con 0x7f9590156010 2024-04-25T10:18:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9595400640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95900d0910 0x7f9590145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9596800640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 0x7f9590144b60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f958c00cef0 tx=0x7f958c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c01a070 con 0x7f9590156010 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c00d860 con 0x7f9590156010 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00dc50 con 0x7f9590156010 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95901484d0 con 0x7f9590156010 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9590146100 con 0x7f9590156010 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9590075120 con 0x7f9590156010 2024-04-25T10:18:14.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.141+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f958c01d070 con 0x7f9590156010 2024-04-25T10:18:14.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.145+0000 7f958a000640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9574066f60 0x7f9574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.145+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f958c0858b0 con 0x7f9590156010 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.145+0000 7f9595e00640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9574066f60 0x7f9574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.145+0000 7f9595e00640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9574066f60 0x7f9574069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f958400b810 tx=0x7f9584008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.149+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f958c056a10 con 0x7f9590156010 2024-04-25T10:18:14.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9590152d00 con 0x7f9590156010 2024-04-25T10:18:14.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f958a000640 1 -- 172.21.15.45:0/3736926533 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f958c056310 con 0x7f9590156010 2024-04-25T10:18:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:14.344 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:14.344 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:14.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9574066f60 msgr2=0x7f9574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9574066f60 0x7f9574069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f958400b810 tx=0x7f9584008040 comp rx=0 tx=0).stop 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 msgr2=0x7f9590144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 0x7f9590144b60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f958c00cef0 tx=0x7f958c00a710 comp rx=0 tx=0).stop 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 shutdown_connections 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9574066f60 0x7f9574069420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9590156010 0x7f9590144b60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95900d0910 0x7f9590145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 --2- 172.21.15.45:0/3736926533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9590071aa0 0x7f95900cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 >> 172.21.15.45:0/3736926533 conn(0x7f95900b1c30 msgr2=0x7f959006d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 shutdown_connections 2024-04-25T10:18:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.341+0000 7f9597200640 1 -- 172.21.15.45:0/3736926533 wait complete. 2024-04-25T10:18:14.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.501+0000 7fa8d9000640 1 Processor -- start 2024-04-25T10:18:14.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.501+0000 7fa8d9000640 1 -- start start 2024-04-25T10:18:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 0x7fa8d4071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d9000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8d40cdba0 0x7fa8d40b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d9000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa8d40ce000 0x7fa8d40b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d9000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d4144650 con 0x7fa8d40cdba0 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d9000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d40cd930 con 0x7fa8d40ce000 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4071240 con 0x7fa8d4144270 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8d40cdba0 0x7fa8d40b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d3400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 0x7fa8d4071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8d40cdba0 0x7fa8d40b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57636/0 (socket says 172.21.15.45:57636) 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d3400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 0x7fa8d4071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54196/0 (socket says 172.21.15.45:54196) 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 -- 172.21.15.45:0/2212975935 learned_addr learned my addr 172.21.15.45:0/2212975935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d3e00640 1 --2- 172.21.15.45:0/2212975935 >> v2:172.21.15.117:3300/0 conn(0x7fa8d40ce000 0x7fa8d40b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 -- 172.21.15.45:0/2212975935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 msgr2=0x7fa8d4071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 --2- 172.21.15.45:0/2212975935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 0x7fa8d4071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 -- 172.21.15.45:0/2212975935 >> v2:172.21.15.117:3300/0 conn(0x7fa8d40ce000 msgr2=0x7fa8d40b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 --2- 172.21.15.45:0/2212975935 >> v2:172.21.15.117:3300/0 conn(0x7fa8d40ce000 0x7fa8d40b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8caa00640 1 -- 172.21.15.45:0/2212975935 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40cd370 con 0x7fa8d40cdba0 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d3400640 1 --2- 172.21.15.45:0/2212975935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 0x7fa8d4071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.505+0000 7fa8d3e00640 1 --2- 172.21.15.45:0/2212975935 >> v2:172.21.15.117:3300/0 conn(0x7fa8d40ce000 0x7fa8d40b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8caa00640 1 --2- 172.21.15.45:0/2212975935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40cdba0 0x7fa8d40b5850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40b37d0 tx=0x7fa8bc01eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6affc2a1d1d8eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d2a00640 1 -- 172.21.15.45:0/2212975935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00b2d0 con 0x7fa8d40cdba0 2024-04-25T10:18:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d2a00640 1 -- 172.21.15.45:0/2212975935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00b470 con 0x7fa8d40cdba0 2024-04-25T10:18:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d2a00640 1 -- 172.21.15.45:0/2212975935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0a7b50 con 0x7fa8d40cdba0 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- 172.21.15.45:0/2212975935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40cdba0 msgr2=0x7fa8d40b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- 172.21.15.45:0/2212975935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40cdba0 0x7fa8d40b5850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40b37d0 tx=0x7fa8bc01eaa0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- 172.21.15.45:0/2212975935 shutdown_connections 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- 172.21.15.45:0/2212975935 >> v2:172.21.15.117:3300/0 conn(0x7fa8d40ce000 0x7fa8d40b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- 172.21.15.45:0/2212975935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40cdba0 0x7fa8d40b5850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- 172.21.15.45:0/2212975935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d4144270 0x7fa8d4071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- 172.21.15.45:0/2212975935 >> 172.21.15.45:0/2212975935 conn(0x7fa8d40d0690 msgr2=0x7fa8d40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- 172.21.15.45:0/2212975935 shutdown_connections 2024-04-25T10:18:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- 172.21.15.45:0/2212975935 wait complete. 2024-04-25T10:18:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 Processor -- start 2024-04-25T10:18:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- start start 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d40cdba0 0x7fa8d415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d40ce000 0x7fa8d4163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 0x7fa8d41641f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4071f40 con 0x7fa8d4144270 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d406eec0 con 0x7fa8d40cdba0 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d40720c0 con 0x7fa8d40ce000 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 0x7fa8d41641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 0x7fa8d41641f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57640/0 (socket says 172.21.15.45:57640) 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.509+0000 7fa8d3e00640 1 -- 172.21.15.45:0/1807962140 learned_addr learned my addr 172.21.15.45:0/1807962140 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3400640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d40cdba0 0x7fa8d415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8caa00640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d40ce000 0x7fa8d4163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3e00640 1 -- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d40ce000 msgr2=0x7fa8d4163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3e00640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d40ce000 0x7fa8d4163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3e00640 1 -- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d40cdba0 msgr2=0x7fa8d415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3e00640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d40cdba0 0x7fa8d415a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3e00640 1 -- 172.21.15.45:0/1807962140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40c5850 con 0x7fa8d4144270 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d3e00640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 0x7fa8d41641f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e820 tx=0x7fa8cc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc017070 con 0x7fa8d4144270 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc004020 con 0x7fa8d4144270 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0043f0 con 0x7fa8d4144270 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d40c5b30 con 0x7fa8d4144270 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d40c60f0 con 0x7fa8d4144270 2024-04-25T10:18:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d4075120 con 0x7fa8d4144270 2024-04-25T10:18:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8cc00c8c0 con 0x7fa8d4144270 2024-04-25T10:18:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8cbe00640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa8b0066f60 0x7fa8b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.513+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8cc085080 con 0x7fa8d4144270 2024-04-25T10:18:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.517+0000 7fa8d3400640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa8b0066f60 0x7fa8b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.517+0000 7fa8d3400640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa8b0066f60 0x7fa8b0069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa8d41fb010 tx=0x7fa8c4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.521+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8cc0559a0 con 0x7fa8d4144270 2024-04-25T10:18:14.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d40b65c0 con 0x7fa8d4144270 2024-04-25T10:18:14.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8cbe00640 1 -- 172.21.15.45:0/1807962140 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fa8cc059a40 con 0x7fa8d4144270 2024-04-25T10:18:14.687 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:14.688 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa8b0066f60 msgr2=0x7fa8b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa8b0066f60 0x7fa8b0069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa8d41fb010 tx=0x7fa8c4009190 comp rx=0 tx=0).stop 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 msgr2=0x7fa8d41641f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 0x7fa8d41641f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00e820 tx=0x7fa8cc00ece0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 shutdown_connections 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa8b0066f60 0x7fa8b0069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d4144270 0x7fa8d41641f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8d40ce000 0x7fa8d4163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 --2- 172.21.15.45:0/1807962140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8d40cdba0 0x7fa8d415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 >> 172.21.15.45:0/1807962140 conn(0x7fa8d40d0690 msgr2=0x7fa8d4068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.685+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 shutdown_connections 2024-04-25T10:18:14.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.689+0000 7fa8d9000640 1 -- 172.21.15.45:0/1807962140 wait complete. 2024-04-25T10:18:14.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:14.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 Processor -- start 2024-04-25T10:18:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 -- start start 2024-04-25T10:18:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd80cf690 0x7f6fd815c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6fd806e480 0x7f6fd815f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd806ec40 con 0x7f6fd815f9b0 2024-04-25T10:18:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd80b3060 con 0x7f6fd806e480 2024-04-25T10:18:14.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fddc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd815fde0 con 0x7f6fd80cf690 2024-04-25T10:18:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd80cf690 0x7f6fd815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6fd806e480 0x7f6fd815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fdd200640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6fd815f9b0 0x7f6fd8152020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:32930/0 (socket says 172.21.15.45:32930) 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fdd200640 1 -- 172.21.15.45:0/2213705707 learned_addr learned my addr 172.21.15.45:0/2213705707 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd6a00640 1 -- 172.21.15.45:0/2213705707 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583726913 0 0) 0x7f6fd806ec40 con 0x7f6fd815f9b0 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd6a00640 1 -- 172.21.15.45:0/2213705707 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc003530 con 0x7f6fd815f9b0 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7400640 1 -- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd80cf690 msgr2=0x7f6fd815c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7400640 1 --2- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd80cf690 0x7f6fd815c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7400640 1 -- 172.21.15.45:0/2213705707 >> v1:172.21.15.117:6789/0 conn(0x7f6fd815f9b0 legacy=0x7f6fd8152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7400640 1 -- 172.21.15.45:0/2213705707 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd806ef20 con 0x7f6fd806e480 2024-04-25T10:18:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.853+0000 7f6fd7e00640 1 --2- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd80cf690 0x7f6fd815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fd7400640 1 --2- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 0x7f6fd815f470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b31e0 tx=0x7f6fc401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78ee6ce676a7326b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fd6a00640 1 -- 172.21.15.45:0/2213705707 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40b1070 con 0x7f6fd806e480 2024-04-25T10:18:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fd6a00640 1 -- 172.21.15.45:0/2213705707 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc40ac410 con 0x7f6fd806e480 2024-04-25T10:18:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fd6a00640 1 -- 172.21.15.45:0/2213705707 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40ac7e0 con 0x7f6fd806e480 2024-04-25T10:18:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 -- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 msgr2=0x7f6fd815f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 --2- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 0x7f6fd815f470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80b31e0 tx=0x7f6fc401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 -- 172.21.15.45:0/2213705707 shutdown_connections 2024-04-25T10:18:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 --2- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 0x7f6fd815f470 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 --2- 172.21.15.45:0/2213705707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd80cf690 0x7f6fd815c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 -- 172.21.15.45:0/2213705707 >> 172.21.15.45:0/2213705707 conn(0x7f6fd806b780 msgr2=0x7f6fd80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 -- 172.21.15.45:0/2213705707 shutdown_connections 2024-04-25T10:18:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.857+0000 7f6fddc00640 1 -- 172.21.15.45:0/2213705707 wait complete. 2024-04-25T10:18:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 Processor -- start 2024-04-25T10:18:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- start start 2024-04-25T10:18:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 0x7f6fd80b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 0x7f6fd814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd815f9b0 0x7f6fd8144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8152cc0 con 0x7f6fd80cf690 2024-04-25T10:18:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8152b40 con 0x7f6fd806e480 2024-04-25T10:18:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd80b3ce0 con 0x7f6fd815f9b0 2024-04-25T10:18:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 0x7f6fd814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 0x7f6fd814ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57656/0 (socket says 172.21.15.45:57656) 2024-04-25T10:18:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 -- 172.21.15.45:0/3111890405 learned_addr learned my addr 172.21.15.45:0/3111890405 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 -- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd815f9b0 msgr2=0x7f6fd8144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd815f9b0 0x7f6fd8144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 -- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 msgr2=0x7f6fd80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 0x7f6fd80b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 -- 172.21.15.45:0/3111890405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd80b0af0 con 0x7f6fd80cf690 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fd7400640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 0x7f6fd814ec60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40af040 tx=0x7f6fc4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40b1040 con 0x7f6fd80cf690 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc40bb070 con 0x7f6fd80cf690 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd80b0dd0 con 0x7f6fd80cf690 2024-04-25T10:18:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40b5410 con 0x7f6fd80cf690 2024-04-25T10:18:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd80b1e00 con 0x7f6fd80cf690 2024-04-25T10:18:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.861+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd8075120 con 0x7f6fd80cf690 2024-04-25T10:18:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.865+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fc40b55b0 con 0x7f6fd80cf690 2024-04-25T10:18:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.865+0000 7f6fcfe00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6fb8066f60 0x7f6fb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.865+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fc411eb10 con 0x7f6fd80cf690 2024-04-25T10:18:14.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.865+0000 7f6fd7e00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6fb8066f60 0x7f6fb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:14.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.865+0000 7f6fd7e00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6fb8066f60 0x7f6fb8069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006e90 tx=0x7f6fc8006de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:14.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:14.869+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fc4016090 con 0x7f6fd80cf690 2024-04-25T10:18:15.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fd815b600 con 0x7f6fd80cf690 2024-04-25T10:18:15.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fcfe00640 1 -- 172.21.15.45:0/3111890405 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fc40efba0 con 0x7f6fd80cf690 2024-04-25T10:18:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:15.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6fb8066f60 msgr2=0x7f6fb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6fb8066f60 0x7f6fb8069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8006e90 tx=0x7f6fc8006de0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 msgr2=0x7f6fd814ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 0x7f6fd814ec60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40af040 tx=0x7f6fc4019240 comp rx=0 tx=0).stop 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 shutdown_connections 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6fb8066f60 0x7f6fb8069420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6fd815f9b0 0x7f6fd8144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fd80cf690 0x7f6fd814ec60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 --2- 172.21.15.45:0/3111890405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fd806e480 0x7f6fd80b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 >> 172.21.15.45:0/3111890405 conn(0x7f6fd806b780 msgr2=0x7f6fd8151c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 shutdown_connections 2024-04-25T10:18:15.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.033+0000 7f6fddc00640 1 -- 172.21.15.45:0/3111890405 wait complete. 2024-04-25T10:18:15.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda71000640 1 Processor -- start 2024-04-25T10:18:15.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda71000640 1 -- start start 2024-04-25T10:18:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda71000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda6c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda71000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0b2fd0 con 0x7fda6c15b150 2024-04-25T10:18:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda71000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c0b32b0 con 0x7fda6c15a0e0 2024-04-25T10:18:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda71000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c15b5c0 con 0x7fda6c0aea50 2024-04-25T10:18:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda61400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda6c15b150 0x7fda6c06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51296/0 (socket says 172.21.15.45:51296) 2024-04-25T10:18:15.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda61400640 1 -- 172.21.15.45:0/2193674244 learned_addr learned my addr 172.21.15.45:0/2193674244 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.197+0000 7fda6a800640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda6c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170606954 0 0) 0x7fda6c0b2fd0 con 0x7fda6c15b150 2024-04-25T10:18:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda50003530 con 0x7fda6c15b150 2024-04-25T10:18:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440478861 0 0) 0x7fda6c0b32b0 con 0x7fda6c15a0e0 2024-04-25T10:18:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6c0b2fd0 con 0x7fda6c15a0e0 2024-04-25T10:18:15.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977236306 0 0) 0x7fda50003530 con 0x7fda6c15b150 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 msgr2=0x7fda6c0705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda6c0705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 >> v1:172.21.15.117:6789/0 conn(0x7fda6c15a0e0 legacy=0x7fda6c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0b3590 con 0x7fda6c15b150 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fda54001680 con 0x7fda6c15b150 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda6c0705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 >> v1:172.21.15.45:6789/0 conn(0x7fda6c15b150 legacy=0x7fda6c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda50003b80 0x7fda50003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50005aa0 con 0x7fda50004d50 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50005c50 con 0x7fda50003b80 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda50005e50 con 0x7fda6c0aea50 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda61400640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda69e00640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda50003b80 0x7fda50003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 -- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 msgr2=0x7fda50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c0aea50 0x7fda50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 -- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda50003b80 msgr2=0x7fda50003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda50003b80 0x7fda50003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 -- 172.21.15.45:0/2193674244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda50003530 con 0x7fda50004d50 2024-04-25T10:18:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.201+0000 7fda6a800640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda500051d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda6400e350 tx=0x7fda6401ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daf44a184ca4459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640b2070 con 0x7fda50004d50 2024-04-25T10:18:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640a5e10 con 0x7fda50004d50 2024-04-25T10:18:15.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda69400640 1 -- 172.21.15.45:0/2193674244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64009de0 con 0x7fda50004d50 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- 172.21.15.45:0/2193674244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 msgr2=0x7fda500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 --2- 172.21.15.45:0/2193674244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda500051d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fda6400e350 tx=0x7fda6401ce00 comp rx=0 tx=0).stop 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda6a800640 1 -- 172.21.15.45:0/2193674244 reap_dead start 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- 172.21.15.45:0/2193674244 shutdown_connections 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- 172.21.15.45:0/2193674244 >> 172.21.15.45:0/2193674244 conn(0x7fda6c0669a0 msgr2=0x7fda6c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- 172.21.15.45:0/2193674244 shutdown_connections 2024-04-25T10:18:15.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- 172.21.15.45:0/2193674244 wait complete. 2024-04-25T10:18:15.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 Processor -- start 2024-04-25T10:18:15.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- start start 2024-04-25T10:18:15.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda6c14e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c156520 0x7fda6c1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda6c157d60 0x7fda6c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0aee80 con 0x7fda50004d50 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0af1d0 con 0x7fda6c157d60 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda71000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c0b44a0 con 0x7fda6c156520 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda69e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda6c14e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda69e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda6c14e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48786/0 (socket says 172.21.15.45:48786) 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.205+0000 7fda69e00640 1 -- 172.21.15.45:0/3793745195 learned_addr learned my addr 172.21.15.45:0/3793745195 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda61400640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c156520 0x7fda6c1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda6a800640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda6c157d60 0x7fda6c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda69e00640 1 -- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c156520 msgr2=0x7fda6c1569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda69e00640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c156520 0x7fda6c1569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda69e00640 1 -- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda6c157d60 msgr2=0x7fda6c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda69e00640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda6c157d60 0x7fda6c1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda69e00640 1 -- 172.21.15.45:0/3793745195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c14ea60 con 0x7fda50004d50 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda6a800640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda6c157d60 0x7fda6c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda61400640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c156520 0x7fda6c1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda69e00640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda6c14e520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda5c011ee0 tx=0x7fda5c00f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c010040 con 0x7fda50004d50 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c01a070 con 0x7fda50004d50 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c015850 con 0x7fda50004d50 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c157650 con 0x7fda50004d50 2024-04-25T10:18:15.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c157090 con 0x7fda50004d50 2024-04-25T10:18:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c04ef30 con 0x7fda50004d50 2024-04-25T10:18:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda5c0159f0 con 0x7fda50004d50 2024-04-25T10:18:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda62a00640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda48066f90 0x7fda48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda5c088990 con 0x7fda50004d50 2024-04-25T10:18:15.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.209+0000 7fda61400640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda48066f90 0x7fda48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.213+0000 7fda61400640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda48066f90 0x7fda48069450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda54000c10 tx=0x7fda54001d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.213+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda5c059900 con 0x7fda50004d50 2024-04-25T10:18:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.373+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda6c06e1f0 con 0x7fda50004d50 2024-04-25T10:18:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda62a00640 1 -- 172.21.15.45:0/3793745195 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda5c059200 con 0x7fda50004d50 2024-04-25T10:18:15.378 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda48066f90 msgr2=0x7fda48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda48066f90 0x7fda48069450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda54000c10 tx=0x7fda54001d40 comp rx=0 tx=0).stop 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 msgr2=0x7fda6c14e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda6c14e520 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda5c011ee0 tx=0x7fda5c00f540 comp rx=0 tx=0).stop 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 shutdown_connections 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda48066f90 0x7fda48069450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda6c157d60 0x7fda6c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda6c156520 0x7fda6c1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 --2- 172.21.15.45:0/3793745195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda50004d50 0x7fda6c14e520 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 >> 172.21.15.45:0/3793745195 conn(0x7fda6c0669a0 msgr2=0x7fda6c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 shutdown_connections 2024-04-25T10:18:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.377+0000 7fda71000640 1 -- 172.21.15.45:0/3793745195 wait complete. 2024-04-25T10:18:15.399 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:15.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 Processor -- start 2024-04-25T10:18:15.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 -- start start 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37bc072020 0x7f37bc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc159920 con 0x7f37bc159540 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc15c2f0 con 0x7f37bc072020 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37c1000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc15c5d0 con 0x7f37bc0aeda0 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b9e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f37bc159540 0x7f37bc0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51302/0 (socket says 172.21.15.45:51302) 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b9e00640 1 -- 172.21.15.45:0/1793752395 learned_addr learned my addr 172.21.15.45:0/1793752395 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b1400640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37ba800640 1 --2- 172.21.15.45:0/1793752395 >> v2:172.21.15.117:3300/0 conn(0x7f37bc072020 0x7f37bc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b9400640 1 -- 172.21.15.45:0/1793752395 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756758009 0 0) 0x7f37bc159920 con 0x7f37bc159540 2024-04-25T10:18:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b9400640 1 -- 172.21.15.45:0/1793752395 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a0003530 con 0x7f37bc159540 2024-04-25T10:18:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37ba800640 1 -- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 msgr2=0x7f37bc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37ba800640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b9400640 1 -- 172.21.15.45:0/1793752395 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 788901715 0 0) 0x7f37a0003530 con 0x7f37bc159540 2024-04-25T10:18:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37ba800640 1 -- 172.21.15.45:0/1793752395 >> v1:172.21.15.45:6789/0 conn(0x7f37bc159540 legacy=0x7f37bc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37ba800640 1 -- 172.21.15.45:0/1793752395 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc15c8b0 con 0x7f37bc072020 2024-04-25T10:18:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.541+0000 7f37b1400640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37ba800640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc072020 0x7f37bc070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009d90 tx=0x7f37b401c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83f6553f330b496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37b9400640 1 -- 172.21.15.45:0/1793752395 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40b2070 con 0x7f37bc072020 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37b9400640 1 -- 172.21.15.45:0/1793752395 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b40a5680 con 0x7f37bc072020 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37b9400640 1 -- 172.21.15.45:0/1793752395 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac001680 con 0x7f37bc072020 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 -- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc072020 msgr2=0x7f37bc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc072020 0x7f37bc070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009d90 tx=0x7f37b401c860 comp rx=0 tx=0).stop 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 -- 172.21.15.45:0/1793752395 shutdown_connections 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc072020 0x7f37bc070350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 --2- 172.21.15.45:0/1793752395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 -- 172.21.15.45:0/1793752395 >> 172.21.15.45:0/1793752395 conn(0x7f37bc0669a0 msgr2=0x7f37bc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 -- 172.21.15.45:0/1793752395 shutdown_connections 2024-04-25T10:18:15.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.545+0000 7f37c1000640 1 -- 172.21.15.45:0/1793752395 wait complete. 2024-04-25T10:18:15.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 Processor -- start 2024-04-25T10:18:15.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- start start 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 0x7f37bc1421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc159540 0x7f37bc1581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0710a0 con 0x7f37bc072020 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc06c0d0 con 0x7f37bc159540 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc071220 con 0x7f37bc0aeda0 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 0x7f37bc1421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 0x7f37bc1421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48788/0 (socket says 172.21.15.45:48788) 2024-04-25T10:18:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 -- 172.21.15.45:0/60146027 learned_addr learned my addr 172.21.15.45:0/60146027 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 -- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 msgr2=0x7f37bc157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 -- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc159540 msgr2=0x7f37bc1581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc159540 0x7f37bc1581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 -- 172.21.15.45:0/60146027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc158a40 con 0x7f37bc072020 2024-04-25T10:18:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b9e00640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 0x7f37bc1421d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00c700 tx=0x7f37ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac017070 con 0x7f37bc072020 2024-04-25T10:18:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac005ae0 con 0x7f37bc072020 2024-04-25T10:18:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc0b1350 con 0x7f37bc072020 2024-04-25T10:18:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac007070 con 0x7f37bc072020 2024-04-25T10:18:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc0b1910 con 0x7f37bc072020 2024-04-25T10:18:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.549+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc04ef30 con 0x7f37bc072020 2024-04-25T10:18:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.553+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37ac01a070 con 0x7f37bc072020 2024-04-25T10:18:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.553+0000 7f37b2a00640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3798066f60 0x7f3798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.553+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37ac084320 con 0x7f37bc072020 2024-04-25T10:18:15.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.553+0000 7f37b1400640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3798066f60 0x7f3798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.553+0000 7f37b1400640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3798066f60 0x7f3798069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37a4006e90 tx=0x7f37a4006de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.557+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37ac054d70 con 0x7f37bc072020 2024-04-25T10:18:15.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37bc166700 con 0x7f37bc072020 2024-04-25T10:18:15.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37b2a00640 1 -- 172.21.15.45:0/60146027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37ac058e10 con 0x7f37bc072020 2024-04-25T10:18:15.723 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:15.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3798066f60 msgr2=0x7f3798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3798066f60 0x7f3798069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f37a4006e90 tx=0x7f37a4006de0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 msgr2=0x7f37bc1421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 0x7f37bc1421d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00c700 tx=0x7f37ac00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 shutdown_connections 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3798066f60 0x7f3798069420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37bc159540 0x7f37bc1581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37bc0aeda0 0x7f37bc157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 --2- 172.21.15.45:0/60146027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37bc072020 0x7f37bc1421d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 >> 172.21.15.45:0/60146027 conn(0x7f37bc0669a0 msgr2=0x7f37bc15d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 shutdown_connections 2024-04-25T10:18:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.721+0000 7f37c1000640 1 -- 172.21.15.45:0/60146027 wait complete. 2024-04-25T10:18:15.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:15.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 Processor -- start 2024-04-25T10:18:15.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 -- start start 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5780d1ce0 0x7ff5780b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5780d2490 con 0x7ff57815df00 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5780d2770 con 0x7ff5780d1ce0 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff57cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780d2a50 con 0x7ff5780d0920 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5780d1ce0 0x7ff5780b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5780d1ce0 0x7ff5780b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48804/0 (socket says 172.21.15.45:48804) 2024-04-25T10:18:15.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 -- 172.21.15.45:0/3016582129 learned_addr learned my addr 172.21.15.45:0/3016582129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:15.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff56d400640 1 --2- 172.21.15.45:0/3016582129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 -- 172.21.15.45:0/3016582129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 msgr2=0x7ff5780d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 --2- 172.21.15.45:0/3016582129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 -- 172.21.15.45:0/3016582129 >> v1:172.21.15.117:6789/0 conn(0x7ff57815df00 legacy=0x7ff5780d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:15.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.885+0000 7ff576800640 1 -- 172.21.15.45:0/3016582129 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780d2d30 con 0x7ff5780d1ce0 2024-04-25T10:18:15.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff576800640 1 --2- 172.21.15.45:0/3016582129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780b71e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff570009870 tx=0x7ff57001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9bc4ee58cd5a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff575400640 1 -- 172.21.15.45:0/3016582129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700aa430 con 0x7ff5780d1ce0 2024-04-25T10:18:15.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff575400640 1 -- 172.21.15.45:0/3016582129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5700af070 con 0x7ff5780d1ce0 2024-04-25T10:18:15.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff575400640 1 -- 172.21.15.45:0/3016582129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700aa7f0 con 0x7ff5780d1ce0 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 -- 172.21.15.45:0/3016582129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 msgr2=0x7ff5780b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 --2- 172.21.15.45:0/3016582129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780b71e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff570009870 tx=0x7ff57001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 -- 172.21.15.45:0/3016582129 shutdown_connections 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 --2- 172.21.15.45:0/3016582129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780b71e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 --2- 172.21.15.45:0/3016582129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 -- 172.21.15.45:0/3016582129 >> 172.21.15.45:0/3016582129 conn(0x7ff57806b290 msgr2=0x7ff5780669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 -- 172.21.15.45:0/3016582129 shutdown_connections 2024-04-25T10:18:15.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.889+0000 7ff57cc00640 1 -- 172.21.15.45:0/3016582129 wait complete. 2024-04-25T10:18:15.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 Processor -- start 2024-04-25T10:18:15.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- start start 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57815df00 0x7ff578154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780b80b0 con 0x7ff5780d1ce0 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780b7f30 con 0x7ff57815df00 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780b8230 con 0x7ff5780d0920 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48808/0 (socket says 172.21.15.45:48808) 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 -- 172.21.15.45:0/2050355257 learned_addr learned my addr 172.21.15.45:0/2050355257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff576800640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57815df00 0x7ff578154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff575e00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 -- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 msgr2=0x7ff5780b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 -- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57815df00 msgr2=0x7ff578154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57815df00 0x7ff578154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 -- 172.21.15.45:0/2050355257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780b4440 con 0x7ff5780d1ce0 2024-04-25T10:18:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff575e00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56d400640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780cccf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff56000d830 tx=0x7ff56000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000b6c0 con 0x7ff5780d1ce0 2024-04-25T10:18:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff560019070 con 0x7ff5780d1ce0 2024-04-25T10:18:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5780cd520 con 0x7ff5780d1ce0 2024-04-25T10:18:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000ba60 con 0x7ff5780d1ce0 2024-04-25T10:18:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5780b3720 con 0x7ff5780d1ce0 2024-04-25T10:18:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.893+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57804ef30 con 0x7ff5780d1ce0 2024-04-25T10:18:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.897+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff56001e460 con 0x7ff5780d1ce0 2024-04-25T10:18:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.897+0000 7ff56ea00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f60 0x7ff554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.897+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5600855f0 con 0x7ff5780d1ce0 2024-04-25T10:18:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.897+0000 7ff575e00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f60 0x7ff554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.897+0000 7ff575e00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f60 0x7ff554069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff568009fc0 tx=0x7ff568009550 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:15.901+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff560056690 con 0x7ff5780d1ce0 2024-04-25T10:18:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.097+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5780b98c0 con 0x7ff5780d1ce0 2024-04-25T10:18:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.097+0000 7ff56ea00640 1 -- 172.21.15.45:0/2050355257 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7ff560055f90 con 0x7ff5780d1ce0 2024-04-25T10:18:16.101 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:16.101 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f60 msgr2=0x7ff554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f60 0x7ff554069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff568009fc0 tx=0x7ff568009550 comp rx=0 tx=0).stop 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 msgr2=0x7ff5780cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780cccf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff56000d830 tx=0x7ff56000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 shutdown_connections 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f60 0x7ff554069420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff57815df00 0x7ff578154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5780d1ce0 0x7ff5780cccf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 --2- 172.21.15.45:0/2050355257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5780d0920 0x7ff5780b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 >> 172.21.15.45:0/2050355257 conn(0x7ff57806b290 msgr2=0x7ff578158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 shutdown_connections 2024-04-25T10:18:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.101+0000 7ff57cc00640 1 -- 172.21.15.45:0/2050355257 wait complete. 2024-04-25T10:18:16.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:16.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 Processor -- start 2024-04-25T10:18:16.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 -- start start 2024-04-25T10:18:16.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f236815a9f0 0x7f23680cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f23680cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680cc2c0 con 0x7f23680cbab0 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680cf770 con 0x7f236815a9f0 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f236d200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cc5a0 con 0x7f23680cf310 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f236815a9f0 0x7f23680cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f236815a9f0 0x7f23680cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48816/0 (socket says 172.21.15.45:48816) 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 -- 172.21.15.45:0/70977924 learned_addr learned my addr 172.21.15.45:0/70977924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f235fe00640 1 --2- 172.21.15.45:0/70977924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f23680cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 -- 172.21.15.45:0/70977924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 msgr2=0x7f23680cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 --2- 172.21.15.45:0/70977924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f23680cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 -- 172.21.15.45:0/70977924 >> v1:172.21.15.117:6789/0 conn(0x7f23680cbab0 legacy=0x7f2368071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.261+0000 7f2365e00640 1 -- 172.21.15.45:0/70977924 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680cc880 con 0x7f236815a9f0 2024-04-25T10:18:16.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f2365e00640 1 --2- 172.21.15.45:0/70977924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236815a9f0 0x7f23680cedd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2358009870 tx=0x7f235801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b6c07737532c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f2365400640 1 -- 172.21.15.45:0/70977924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580a95f0 con 0x7f236815a9f0 2024-04-25T10:18:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f2365400640 1 -- 172.21.15.45:0/70977924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580a9790 con 0x7f236815a9f0 2024-04-25T10:18:16.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f2365400640 1 -- 172.21.15.45:0/70977924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580a9b60 con 0x7f236815a9f0 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 -- 172.21.15.45:0/70977924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236815a9f0 msgr2=0x7f23680cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 --2- 172.21.15.45:0/70977924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236815a9f0 0x7f23680cedd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2358009870 tx=0x7f235801c530 comp rx=0 tx=0).stop 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 -- 172.21.15.45:0/70977924 shutdown_connections 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 --2- 172.21.15.45:0/70977924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f23680cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 --2- 172.21.15.45:0/70977924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f236815a9f0 0x7f23680cedd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 -- 172.21.15.45:0/70977924 >> 172.21.15.45:0/70977924 conn(0x7f23680b2410 msgr2=0x7f23680b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 -- 172.21.15.45:0/70977924 shutdown_connections 2024-04-25T10:18:16.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.265+0000 7f236d200640 1 -- 172.21.15.45:0/70977924 wait complete. 2024-04-25T10:18:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 Processor -- start 2024-04-25T10:18:16.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- start start 2024-04-25T10:18:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 0x7f23680b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f2368146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236815a9f0 0x7f2368146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cc040 con 0x7f23680cbab0 2024-04-25T10:18:16.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680cbec0 con 0x7f236815a9f0 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368167640 con 0x7f23680cf310 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 0x7f23680b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 0x7f23680b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48826/0 (socket says 172.21.15.45:48826) 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 -- 172.21.15.45:0/761772530 learned_addr learned my addr 172.21.15.45:0/761772530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f235fe00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f2368146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2366800640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236815a9f0 0x7f2368146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 -- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 msgr2=0x7f2368146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f2368146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 -- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236815a9f0 msgr2=0x7f2368146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236815a9f0 0x7f2368146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 -- 172.21.15.45:0/761772530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680c8360 con 0x7f23680cbab0 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f235fe00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f2368146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f2365e00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 0x7f23680b4e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f235801abe0 tx=0x7f23580192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580b1800 con 0x7f23680cbab0 2024-04-25T10:18:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580b19c0 con 0x7f23680cbab0 2024-04-25T10:18:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580b1db0 con 0x7f23680cbab0 2024-04-25T10:18:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23680c8640 con 0x7f23680cbab0 2024-04-25T10:18:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23680c8c00 con 0x7f23680cbab0 2024-04-25T10:18:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.269+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236804ef30 con 0x7f23680cbab0 2024-04-25T10:18:16.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.273+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23580c43f0 con 0x7f23680cbab0 2024-04-25T10:18:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.273+0000 7f235e000640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2344066f60 0x7f2344069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.273+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f235811f7b0 con 0x7f23680cbab0 2024-04-25T10:18:16.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.273+0000 7f235fe00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2344066f60 0x7f2344069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.273+0000 7f235fe00640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2344066f60 0x7f2344069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23681f7ef0 tx=0x7f2350008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.277+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23580ad020 con 0x7f23680cbab0 2024-04-25T10:18:16.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.437+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236806df30 con 0x7f23680cbab0 2024-04-25T10:18:16.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.437+0000 7f235e000640 1 -- 172.21.15.45:0/761772530 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f2358016150 con 0x7f23680cbab0 2024-04-25T10:18:16.442 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:16.442 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2344066f60 msgr2=0x7f2344069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2344066f60 0x7f2344069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23681f7ef0 tx=0x7f2350008040 comp rx=0 tx=0).stop 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 msgr2=0x7f23680b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 0x7f23680b4e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f235801abe0 tx=0x7f23580192d0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 shutdown_connections 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2344066f60 0x7f2344069420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236815a9f0 0x7f2368146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23680cf310 0x7f2368146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 --2- 172.21.15.45:0/761772530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23680cbab0 0x7f23680b4e00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 >> 172.21.15.45:0/761772530 conn(0x7f23680b2410 msgr2=0x7f23680b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 shutdown_connections 2024-04-25T10:18:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.441+0000 7f236d200640 1 -- 172.21.15.45:0/761772530 wait complete. 2024-04-25T10:18:16.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:16.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 Processor -- start 2024-04-25T10:18:16.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 -- start start 2024-04-25T10:18:16.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f983c15bc10 0x7f983c0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c0d0440 0x7f983c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15bff0 con 0x7f983c0ccd40 2024-04-25T10:18:16.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c0cd550 con 0x7f983c15bc10 2024-04-25T10:18:16.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f9841000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b0da0 con 0x7f983c0d0440 2024-04-25T10:18:16.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983aa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f983c0ccd40 0x7f983c0cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51310/0 (socket says 172.21.15.45:51310) 2024-04-25T10:18:16.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983aa00640 1 -- 172.21.15.45:0/3812422052 learned_addr learned my addr 172.21.15.45:0/3812422052 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983be00640 1 --2- 172.21.15.45:0/3812422052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c0d0440 0x7f983c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983b400640 1 --2- 172.21.15.45:0/3812422052 >> v2:172.21.15.117:3300/0 conn(0x7f983c15bc10 0x7f983c0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983a000640 1 -- 172.21.15.45:0/3812422052 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836338495 0 0) 0x7f983c15bff0 con 0x7f983c0ccd40 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983a000640 1 -- 172.21.15.45:0/3812422052 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9820003530 con 0x7f983c0ccd40 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983be00640 1 -- 172.21.15.45:0/3812422052 >> v2:172.21.15.117:3300/0 conn(0x7f983c15bc10 msgr2=0x7f983c0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983be00640 1 --2- 172.21.15.45:0/3812422052 >> v2:172.21.15.117:3300/0 conn(0x7f983c15bc10 0x7f983c0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983be00640 1 -- 172.21.15.45:0/3812422052 >> v1:172.21.15.45:6789/0 conn(0x7f983c0ccd40 legacy=0x7f983c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983be00640 1 -- 172.21.15.45:0/3812422052 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0b1080 con 0x7f983c0d0440 2024-04-25T10:18:16.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.601+0000 7f983b400640 1 --2- 172.21.15.45:0/3812422052 >> v2:172.21.15.117:3300/0 conn(0x7f983c15bc10 0x7f983c0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:16.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f983be00640 1 --2- 172.21.15.45:0/3812422052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c0d0440 0x7f983c0b1880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9834009dc0 tx=0x7f983401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=508685949d3ef39f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f983a000640 1 -- 172.21.15.45:0/3812422052 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340b2070 con 0x7f983c0d0440 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f983a000640 1 -- 172.21.15.45:0/3812422052 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340a5a30 con 0x7f983c0d0440 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f983a000640 1 -- 172.21.15.45:0/3812422052 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340a5de0 con 0x7f983c0d0440 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 -- 172.21.15.45:0/3812422052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c0d0440 msgr2=0x7f983c0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 --2- 172.21.15.45:0/3812422052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c0d0440 0x7f983c0b1880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9834009dc0 tx=0x7f983401c950 comp rx=0 tx=0).stop 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 -- 172.21.15.45:0/3812422052 shutdown_connections 2024-04-25T10:18:16.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 --2- 172.21.15.45:0/3812422052 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c0d0440 0x7f983c0b1880 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 --2- 172.21.15.45:0/3812422052 >> v2:172.21.15.117:3300/0 conn(0x7f983c15bc10 0x7f983c0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 -- 172.21.15.45:0/3812422052 >> 172.21.15.45:0/3812422052 conn(0x7f983c0d1750 msgr2=0x7f983c0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 -- 172.21.15.45:0/3812422052 shutdown_connections 2024-04-25T10:18:16.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.605+0000 7f9841000640 1 -- 172.21.15.45:0/3812422052 wait complete. 2024-04-25T10:18:16.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 Processor -- start 2024-04-25T10:18:16.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- start start 2024-04-25T10:18:16.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 0x7f983c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0d0440 0x7f983c06eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c15bc10 0x7f983c06afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b29a0 con 0x7f983c0ccd40 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b2820 con 0x7f983c0d0440 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b2b20 con 0x7f983c15bc10 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 0x7f983c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 0x7f983c06e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48840/0 (socket says 172.21.15.45:48840) 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 -- 172.21.15.45:0/1408045898 learned_addr learned my addr 172.21.15.45:0/1408045898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983be00640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c15bc10 0x7f983c06afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983aa00640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0d0440 0x7f983c06eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 -- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c15bc10 msgr2=0x7f983c06afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c15bc10 0x7f983c06afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 -- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0d0440 msgr2=0x7f983c06eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0d0440 0x7f983c06eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 -- 172.21.15.45:0/1408045898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c06bc30 con 0x7f983c0ccd40 2024-04-25T10:18:16.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f983b400640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 0x7f983c06e980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f982c00ebe0 tx=0x7f982c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c017070 con 0x7f983c0ccd40 2024-04-25T10:18:16.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c00cc30 con 0x7f983c0ccd40 2024-04-25T10:18:16.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c013470 con 0x7f983c0ccd40 2024-04-25T10:18:16.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c06bf10 con 0x7f983c0ccd40 2024-04-25T10:18:16.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c06c4d0 con 0x7f983c0ccd40 2024-04-25T10:18:16.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.609+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c075120 con 0x7f983c0ccd40 2024-04-25T10:18:16.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.613+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f982c01a070 con 0x7f983c0ccd40 2024-04-25T10:18:16.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.613+0000 7f9833400640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f981c066f60 0x7f981c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:16.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.613+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f982c085330 con 0x7f983c0ccd40 2024-04-25T10:18:16.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.613+0000 7f983aa00640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f981c066f60 0x7f981c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:16.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.613+0000 7f983aa00640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f981c066f60 0x7f981c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f983c06af40 tx=0x7f9828009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.617+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f982c056340 con 0x7f983c0ccd40 2024-04-25T10:18:16.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.813+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f983c0700d0 con 0x7f983c0ccd40 2024-04-25T10:18:16.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9833400640 1 -- 172.21.15.45:0/1408045898 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f982c055c40 con 0x7f983c0ccd40 2024-04-25T10:18:16.818 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:16.818 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:16.818 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f981c066f60 msgr2=0x7f981c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f981c066f60 0x7f981c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f983c06af40 tx=0x7f9828009040 comp rx=0 tx=0).stop 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 msgr2=0x7f983c06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 0x7f983c06e980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f982c00ebe0 tx=0x7f982c00c520 comp rx=0 tx=0).stop 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 shutdown_connections 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f981c066f60 0x7f981c069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f983c15bc10 0x7f983c06afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f983c0d0440 0x7f983c06eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 --2- 172.21.15.45:0/1408045898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f983c0ccd40 0x7f983c06e980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 >> 172.21.15.45:0/1408045898 conn(0x7f983c0d1750 msgr2=0x7f983c0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 shutdown_connections 2024-04-25T10:18:16.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:16.817+0000 7f9841000640 1 -- 172.21.15.45:0/1408045898 wait complete. 2024-04-25T10:18:17.241 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:18:02.077189+0000', 'modified': '2024-04-25T10:18:05.087771+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/1341306304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1341306304}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1341306304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:17.241 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:18:17.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:18:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 Processor -- start 2024-04-25T10:18:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 -- start start 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4800d0040 0x7fd48015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd48015b740 0x7fd4800b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd480071050 con 0x7fd4801550b0 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48015d040 con 0x7fd4800d0040 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.381+0000 7fd488000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4800712f0 con 0x7fd48015b740 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd485600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd4801550b0 0x7fd4800cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51326/0 (socket says 172.21.15.45:51326) 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd484c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd4800d0040 0x7fd48015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd485600640 1 -- 172.21.15.45:0/4052125124 learned_addr learned my addr 172.21.15.45:0/4052125124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd486000640 1 --2- 172.21.15.45:0/4052125124 >> v2:172.21.15.117:3300/0 conn(0x7fd48015b740 0x7fd4800b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd484c00640 1 -- 172.21.15.45:0/4052125124 >> v2:172.21.15.117:3300/0 conn(0x7fd48015b740 msgr2=0x7fd4800b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd484c00640 1 --2- 172.21.15.45:0/4052125124 >> v2:172.21.15.117:3300/0 conn(0x7fd48015b740 0x7fd4800b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd484c00640 1 -- 172.21.15.45:0/4052125124 >> v1:172.21.15.45:6789/0 conn(0x7fd4801550b0 legacy=0x7fd4800cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd47fe00640 1 -- 172.21.15.45:0/4052125124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901367741 0 0) 0x7fd480071050 con 0x7fd4801550b0 2024-04-25T10:18:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd484c00640 1 -- 172.21.15.45:0/4052125124 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800715d0 con 0x7fd4800d0040 2024-04-25T10:18:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd484c00640 1 --2- 172.21.15.45:0/4052125124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0040 0x7fd48015b200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd4800d04d0 tx=0x7fd46c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d9f0d1393b8885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd47fe00640 1 -- 172.21.15.45:0/4052125124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0ab650 con 0x7fd4800d0040 2024-04-25T10:18:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd47fe00640 1 -- 172.21.15.45:0/4052125124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c0b0070 con 0x7fd4800d0040 2024-04-25T10:18:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.385+0000 7fd47fe00640 1 -- 172.21.15.45:0/4052125124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0aba10 con 0x7fd4800d0040 2024-04-25T10:18:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- 172.21.15.45:0/4052125124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0040 msgr2=0x7fd48015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 --2- 172.21.15.45:0/4052125124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0040 0x7fd48015b200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd4800d04d0 tx=0x7fd46c01c450 comp rx=0 tx=0).stop 2024-04-25T10:18:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- 172.21.15.45:0/4052125124 shutdown_connections 2024-04-25T10:18:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 --2- 172.21.15.45:0/4052125124 >> v2:172.21.15.117:3300/0 conn(0x7fd48015b740 0x7fd4800b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 --2- 172.21.15.45:0/4052125124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0040 0x7fd48015b200 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- 172.21.15.45:0/4052125124 >> 172.21.15.45:0/4052125124 conn(0x7fd480066b70 msgr2=0x7fd480066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- 172.21.15.45:0/4052125124 shutdown_connections 2024-04-25T10:18:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- 172.21.15.45:0/4052125124 wait complete. 2024-04-25T10:18:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 Processor -- start 2024-04-25T10:18:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- start start 2024-04-25T10:18:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4800d0040 0x7fd4800afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 0x7fd48014f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd48015b740 0x7fd48014d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48015bcd0 con 0x7fd4801550b0 2024-04-25T10:18:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48015bb50 con 0x7fd4800d0040 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd488000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48015c030 con 0x7fd48015b740 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd484c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 0x7fd48014f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd484c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 0x7fd48014f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48862/0 (socket says 172.21.15.45:48862) 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd484c00640 1 -- 172.21.15.45:0/3495658862 learned_addr learned my addr 172.21.15.45:0/3495658862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd485600640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4800d0040 0x7fd4800afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd486000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd48015b740 0x7fd48014d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.389+0000 7fd484c00640 1 -- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd48015b740 msgr2=0x7fd48014d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd484c00640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd48015b740 0x7fd48014d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd484c00640 1 -- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4800d0040 msgr2=0x7fd4800afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd484c00640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4800d0040 0x7fd4800afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd484c00640 1 -- 172.21.15.45:0/3495658862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48014fad0 con 0x7fd4801550b0 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd485600640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4800d0040 0x7fd4800afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd484c00640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 0x7fd48014f080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0ad040 tx=0x7fd46c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd47d600640 1 -- 172.21.15.45:0/3495658862 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0bf610 con 0x7fd4801550b0 2024-04-25T10:18:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd47d600640 1 -- 172.21.15.45:0/3495658862 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c0bf7d0 con 0x7fd4801550b0 2024-04-25T10:18:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd47d600640 1 -- 172.21.15.45:0/3495658862 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0bfbc0 con 0x7fd4801550b0 2024-04-25T10:18:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48014fdb0 con 0x7fd4801550b0 2024-04-25T10:18:17.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4800b0070 con 0x7fd4801550b0 2024-04-25T10:18:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.393+0000 7fd47d600640 1 -- 172.21.15.45:0/3495658862 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd46c0bd020 con 0x7fd4801550b0 2024-04-25T10:18:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd47d600640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd460066f60 0x7fd460069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd47d600640 1 -- 172.21.15.45:0/3495658862 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd46c11fc00 con 0x7fd4801550b0 2024-04-25T10:18:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd460066f60 0x7fd460069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 Processor -- start 2024-04-25T10:18:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- start start 2024-04-25T10:18:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd460066f60 0x7fd460069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd474007fd0 tx=0x7fd474007c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f1240 0x55e37a9a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9a7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37a88e5f0 con 0x55e37a9f0920 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37a7f5da0 con 0x55e37a9a6f30 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e37a77e150 con 0x55e37a9f1240 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48870/0 (socket says 172.21.15.45:48870) 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 -- 172.21.15.45:0/2608762650 learned_addr learned my addr 172.21.15.45:0/2608762650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd486000640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd484c00640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f1240 0x55e37a9a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 -- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f1240 msgr2=0x55e37a9a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f1240 0x55e37a9a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 -- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 msgr2=0x55e37a9a7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9a7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 -- 172.21.15.45:0/2608762650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37a797ca0 con 0x55e37a9f0920 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd486000640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd484c00640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f1240 0x55e37a9a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd485600640 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f0d00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd474005dc0 tx=0x7fd474012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620a28c509ce44c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd472000640 1 -- 172.21.15.45:0/2608762650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474015110 con 0x55e37a9f0920 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd472000640 1 -- 172.21.15.45:0/2608762650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740159c0 con 0x55e37a9f0920 2024-04-25T10:18:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd472000640 1 -- 172.21.15.45:0/2608762650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474015d70 con 0x55e37a9f0920 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- 172.21.15.45:0/2608762650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 msgr2=0x55e37a9f0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f0d00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd474005dc0 tx=0x7fd474012000 comp rx=0 tx=0).stop 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- 172.21.15.45:0/2608762650 shutdown_connections 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f1240 0x55e37a9a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/2608762650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f0d00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- 172.21.15.45:0/2608762650 >> 172.21.15.45:0/2608762650 conn(0x55e37a9ad2a0 msgr2=0x55e37a9a52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- 172.21.15.45:0/2608762650 shutdown_connections 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.397+0000 7fd48b4b7000 1 -- 172.21.15.45:0/2608762650 wait complete. 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- start start 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 Processor -- start 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37a9f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f1240 0x55e37a9aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37a7b0990 con 0x55e37a9f0920 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37a7f1e40 con 0x55e37a9f1240 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e37a781900 con 0x55e37a9a6f30 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 10 client.0 _ll_register_callbacks cb 0x55e37a8a91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48876/0 (socket says 172.21.15.45:48876) 2024-04-25T10:18:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 -- 172.21.15.45:0/750629004 learned_addr learned my addr 172.21.15.45:0/750629004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd485600640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37a9f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd486000640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f1240 0x55e37a9aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 -- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 msgr2=0x55e37a9f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37a9f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 -- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f1240 msgr2=0x55e37a9aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f1240 0x55e37a9aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 -- 172.21.15.45:0/750629004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e37a7ae630 con 0x55e37a9f0920 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd486000640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f1240 0x55e37a9aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd484c00640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f8330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0f8780 tx=0x7fd46c0effc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287b6140566e8952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd466a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd466a00640 1 -- 172.21.15.45:0/750629004 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0c4040 con 0x55e37a9f0920 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd466a00640 1 -- 172.21.15.45:0/750629004 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd46c0f8a20 con 0x55e37a9f0920 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4740159c0 con 0x55e37a9f0920 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd466a00640 1 -- 172.21.15.45:0/750629004 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd46c10d0a0 con 0x55e37a9f0920 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 10 client.4626 fetch_fsmap learned FSMap version 8 2024-04-25T10:18:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e37a77e3a0 con 0x55e37a9f0920 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd466a00640 1 -- 172.21.15.45:0/750629004 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740159c0 con 0x55e37a9f0920 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd466a00640 1 -- 172.21.15.45:0/750629004 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd46c0ef9f0 con 0x55e37a9f0920 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 10 client.4626 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 10 client.4626 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 20 client.4626 populate_metadata read hostname 'smithi045' 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37a9ac160 0x55e37aa01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 4 client.4626 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 --> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e37a791a50 con 0x55e37a9ac160 2024-04-25T10:18:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.401+0000 7fd485600640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37a9ac160 0x55e37aa01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd485600640 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37a9ac160 0x55e37aa01e50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63873aa8781cdc8b server_cookie=4cd3e15d21931149 in_seq=0 out_seq=0 2024-04-25T10:18:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd466a00640 10 client.4626 ms_handle_connect on v2:172.21.15.117:6835/1341306304 2024-04-25T10:18:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd466a00640 1 -- 172.21.15.45:0/750629004 <== mds.0 v2:172.21.15.117:6835/1341306304 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e37a791a50 con 0x55e37a9ac160 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd466a00640 10 client.4626 handle_command_reply: tid=0 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 client.4626 shutdown 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 msgr2=0x55e37a9f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f8330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0f8780 tx=0x7fd46c0effc0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 shutdown_connections 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f1240 0x55e37a9aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f0920 0x55e37a9f8330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37a9ac160 0x55e37aa01e50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/750629004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37a9f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 >> 172.21.15.45:0/750629004 conn(0x55e37a9ad2a0 msgr2=0x55e37a9a59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd466a00640 0 client.4626 ms_handle_reset on v2:172.21.15.117:6835/1341306304 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 shutdown_connections 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 1 -- 172.21.15.45:0/750629004 wait complete. 2024-04-25T10:18:17.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.405+0000 7fd48b4b7000 20 client.4626 trim_cache size 0 max 16384 2024-04-25T10:18:17.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 Processor -- start 2024-04-25T10:18:17.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 -- start start 2024-04-25T10:18:17.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f0920 0x55e37a9fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa0c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801ffc50 con 0x55e37a9f1240 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48006ce00 con 0x55e37a9a6f30 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd480145740 con 0x55e37a9f0920 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa0c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa0c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48880/0 (socket says 172.21.15.45:48880) 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 -- 172.21.15.45:0/1895637414 learned_addr learned my addr 172.21.15.45:0/1895637414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd484c00640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f0920 0x55e37a9fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd485600640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 -- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f0920 msgr2=0x55e37a9fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f0920 0x55e37a9fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 -- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 msgr2=0x55e37a9fce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9fce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 -- 172.21.15.45:0/1895637414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37a9f1070 con 0x55e37a9f1240 2024-04-25T10:18:17.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd484c00640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f0920 0x55e37a9fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:17.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd485600640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:17.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd486000640 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa0c8e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd478007680 tx=0x7fd478009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0f1af5cd2b7395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd471600640 1 -- 172.21.15.45:0/1895637414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47801c070 con 0x55e37a9f1240 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd471600640 1 -- 172.21.15.45:0/1895637414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478015880 con 0x55e37a9f1240 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd471600640 1 -- 172.21.15.45:0/1895637414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478015c70 con 0x55e37a9f1240 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 -- 172.21.15.45:0/1895637414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 msgr2=0x55e37aa0c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.421+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa0c8e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd478007680 tx=0x7fd478009750 comp rx=0 tx=0).stop 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/1895637414 shutdown_connections 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa0c8e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9f0920 0x55e37a9fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/1895637414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9a6f30 0x55e37a9fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/1895637414 >> 172.21.15.45:0/1895637414 conn(0x55e37a9ad2a0 msgr2=0x55e37a9aa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/1895637414 shutdown_connections 2024-04-25T10:18:17.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/1895637414 wait complete. 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- start start 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 Processor -- start 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37aa39b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f0920 0x55e37aa3a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37a7fa3d0 con 0x55e37a9f1240 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37a8b1d50 con 0x55e37a9f0920 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e37a790980 con 0x55e37a9a6f30 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 10 client.0 _ll_register_callbacks cb 0x7fd474007270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd484c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f0920 0x55e37aa3a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa3a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48894/0 (socket says 172.21.15.45:48894) 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 -- 172.21.15.45:0/3514034351 learned_addr learned my addr 172.21.15.45:0/3514034351 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:17.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd485600640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37aa39b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 -- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 msgr2=0x55e37aa39b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37aa39b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 -- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f0920 msgr2=0x55e37aa3a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f0920 0x55e37aa3a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 -- 172.21.15.45:0/3514034351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e37a877600 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd484c00640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f0920 0x55e37aa3a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd486000640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa3a5f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd478009790 tx=0x7fd478004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc457cd87b0e252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd466a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd466a00640 1 -- 172.21.15.45:0/3514034351 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47801c070 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd466a00640 1 -- 172.21.15.45:0/3514034351 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd478021860 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd478015880 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd466a00640 1 -- 172.21.15.45:0/3514034351 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd478032050 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 10 client.4632 fetch_fsmap learned FSMap version 8 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e37a78a9b0 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd466a00640 1 -- 172.21.15.45:0/3514034351 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478015880 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd466a00640 1 -- 172.21.15.45:0/3514034351 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd478035020 con 0x55e37a9f1240 2024-04-25T10:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 10 client.4632 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 20 client.4632 populate_metadata read hostname 'smithi045' 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37aa3ac50 0x55e37aa3b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 4 client.4632 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.425+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 --> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e37a8032e0 con 0x55e37aa3ac50 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd485600640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37aa3ac50 0x55e37aa3b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd485600640 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37aa3ac50 0x55e37aa3b030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53685cdf83b29342 server_cookie=1d0c239d9bbd4a2 in_seq=0 out_seq=0 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd466a00640 10 client.4632 ms_handle_connect on v2:172.21.15.117:6835/1341306304 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd466a00640 1 -- 172.21.15.45:0/3514034351 <== mds.0 v2:172.21.15.117:6835/1341306304 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e37a8032e0 con 0x55e37aa3ac50 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd466a00640 10 client.4632 handle_command_reply: tid=0 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 client.4632 shutdown 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 msgr2=0x55e37aa3a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa3a5f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd478009790 tx=0x7fd478004ac0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 shutdown_connections 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e37a9f1240 0x55e37aa3a5f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e37a9f0920 0x55e37aa3a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:6835/1341306304,v1:172.21.15.117:6837/1341306304] conn(0x55e37aa3ac50 0x55e37aa3b030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 --2- 172.21.15.45:0/3514034351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e37a9a6f30 0x55e37aa39b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd466a00640 0 client.4632 ms_handle_reset on v2:172.21.15.117:6835/1341306304 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 >> 172.21.15.45:0/3514034351 conn(0x55e37a9ad2a0 msgr2=0x55e37a9aa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 shutdown_connections 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 1 -- 172.21.15.45:0/3514034351 wait complete. 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd48b4b7000 20 client.4632 trim_cache size 0 max 16384 2024-04-25T10:18:17.435 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4285, 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 14.757348299, 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 8, 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 22, 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 22, 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 22.303604277000002 2024-04-25T10:18:17.436 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:18:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd460066f60 msgr2=0x7fd460069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd460066f60 0x7fd460069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd474007fd0 tx=0x7fd474007c20 comp rx=0 tx=0).stop 2024-04-25T10:18:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 msgr2=0x7fd48014f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 0x7fd48014f080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0ad040 tx=0x7fd46c01ac90 comp rx=0 tx=0).stop 2024-04-25T10:18:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 shutdown_connections 2024-04-25T10:18:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd460066f60 0x7fd460069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd48015b740 0x7fd48014d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4801550b0 0x7fd48014f080 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 --2- 172.21.15.45:0/3495658862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4800d0040 0x7fd4800afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 >> 172.21.15.45:0/3495658862 conn(0x7fd480066b70 msgr2=0x7fd4800cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 shutdown_connections 2024-04-25T10:18:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:17.429+0000 7fd488000640 1 -- 172.21.15.45:0/3495658862 wait complete. 2024-04-25T10:18:17.450 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:18:17.475 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:18:17.475 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:18:17.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.475 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:18:17.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T10:18:17.547 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-04-25T10:18:17.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:17.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.550 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:18:17.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T10:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-04-25T10:18:17.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:17.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.669 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-25T10:18:17.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.680 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-04-25T10:18:17.746 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-04-25T10:18:17.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:17.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.749 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-25T10:18:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:17.803 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-04-25T10:18:17.873 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-04-25T10:18:17.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:18:17.875 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T10:18:17.875 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:18:17.876 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:17.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:18.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 Processor -- start 2024-04-25T10:18:18.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 -- start start 2024-04-25T10:18:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13880cfd90 0x7f13880ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880cd110 0x7f13880cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138815f3b0 con 0x7f13880b0f70 2024-04-25T10:18:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138815ef20 con 0x7f13880cfd90 2024-04-25T10:18:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f138ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880ca810 con 0x7f13880cd110 2024-04-25T10:18:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f1385e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f13880b0f70 0x7f13880b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51334/0 (socket says 172.21.15.45:51334) 2024-04-25T10:18:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f1385e00640 1 -- 172.21.15.45:0/1082292799 learned_addr learned my addr 172.21.15.45:0/1082292799 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.013+0000 7f137d400640 1 --2- 172.21.15.45:0/1082292799 >> v2:172.21.15.45:3300/0 conn(0x7f13880cfd90 0x7f13880ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f1386800640 1 --2- 172.21.15.45:0/1082292799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880cd110 0x7f13880cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f1385400640 1 -- 172.21.15.45:0/1082292799 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204428133 0 0) 0x7f138815f3b0 con 0x7f13880b0f70 2024-04-25T10:18:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f137d400640 1 -- 172.21.15.45:0/1082292799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880cd110 msgr2=0x7f13880cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f137d400640 1 --2- 172.21.15.45:0/1082292799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880cd110 0x7f13880cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f137d400640 1 -- 172.21.15.45:0/1082292799 >> v1:172.21.15.45:6789/0 conn(0x7f13880b0f70 legacy=0x7f13880b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f137d400640 1 -- 172.21.15.45:0/1082292799 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138815e5f0 con 0x7f13880cfd90 2024-04-25T10:18:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f137d400640 1 --2- 172.21.15.45:0/1082292799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cfd90 0x7f13880ccbd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13700097c0 tx=0x7f137001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb766a5468cd90c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f1385400640 1 -- 172.21.15.45:0/1082292799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700b0070 con 0x7f13880cfd90 2024-04-25T10:18:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f1385400640 1 -- 172.21.15.45:0/1082292799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13700ab600 con 0x7f13880cfd90 2024-04-25T10:18:18.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f1385400640 1 -- 172.21.15.45:0/1082292799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700ab9d0 con 0x7f13880cfd90 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f138ca00640 1 -- 172.21.15.45:0/1082292799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cfd90 msgr2=0x7f13880ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.017+0000 7f138ca00640 1 --2- 172.21.15.45:0/1082292799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cfd90 0x7f13880ccbd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13700097c0 tx=0x7f137001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- 172.21.15.45:0/1082292799 shutdown_connections 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 --2- 172.21.15.45:0/1082292799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880cd110 0x7f13880cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 --2- 172.21.15.45:0/1082292799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cfd90 0x7f13880ccbd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- 172.21.15.45:0/1082292799 >> 172.21.15.45:0/1082292799 conn(0x7f13880b1d50 msgr2=0x7f13880b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- 172.21.15.45:0/1082292799 shutdown_connections 2024-04-25T10:18:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- 172.21.15.45:0/1082292799 wait complete. 2024-04-25T10:18:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 Processor -- start 2024-04-25T10:18:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- start start 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880b0f70 0x7f1388157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 0x7f13880aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13880cfd90 0x7f13880af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388161b50 con 0x7f13880cd110 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881619d0 con 0x7f13880cfd90 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f138ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1388161cd0 con 0x7f13880b0f70 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 0x7f13880aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 0x7f13880aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48916/0 (socket says 172.21.15.45:48916) 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 -- 172.21.15.45:0/3262639331 learned_addr learned my addr 172.21.15.45:0/3262639331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f1386800640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13880cfd90 0x7f13880af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f1385e00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880b0f70 0x7f1388157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 -- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880b0f70 msgr2=0x7f1388157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880b0f70 0x7f1388157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 -- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13880cfd90 msgr2=0x7f13880af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13880cfd90 0x7f13880af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 -- 172.21.15.45:0/3262639331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880b4830 con 0x7f13880cd110 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f1386800640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13880cfd90 0x7f13880af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f1385e00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880b0f70 0x7f1388157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.021+0000 7f137d400640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 0x7f13880aedc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13700ae040 tx=0x7f1370019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700b0040 con 0x7f13880cd110 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13700ab400 con 0x7f13880cd110 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880b4af0 con 0x7f13880cd110 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700b4650 con 0x7f13880cd110 2024-04-25T10:18:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880b50b0 con 0x7f13880cd110 2024-04-25T10:18:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880b5770 con 0x7f13880cd110 2024-04-25T10:18:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13700bc020 con 0x7f13880cd110 2024-04-25T10:18:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f137ea00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1364066f90 0x7f1364069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f137011e460 con 0x7f13880cd110 2024-04-25T10:18:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f1385e00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1364066f90 0x7f1364069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.025+0000 7f1385e00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1364066f90 0x7f1364069450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1378007fd0 tx=0x7f1378007c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.029+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13880b5770 con 0x7f13880cd110 2024-04-25T10:18:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f138804ef30 con 0x7f13880cd110 2024-04-25T10:18:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f137ea00640 1 -- 172.21.15.45:0/3262639331 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f13700c3020 con 0x7f13880cd110 2024-04-25T10:18:18.231 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1364066f90 msgr2=0x7f1364069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1364066f90 0x7f1364069450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1378007fd0 tx=0x7f1378007c20 comp rx=0 tx=0).stop 2024-04-25T10:18:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 msgr2=0x7f13880aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:18.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 0x7f13880aedc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13700ae040 tx=0x7f1370019240 comp rx=0 tx=0).stop 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 shutdown_connections 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1364066f90 0x7f1364069450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13880cfd90 0x7f13880af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13880cd110 0x7f13880aedc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 --2- 172.21.15.45:0/3262639331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13880b0f70 0x7f1388157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 >> 172.21.15.45:0/3262639331 conn(0x7f13880b1d50 msgr2=0x7f13880c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 shutdown_connections 2024-04-25T10:18:18.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:18.229+0000 7f138ca00640 1 -- 172.21.15.45:0/3262639331 wait complete. 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:18:18.251 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:18:18.252 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:18.252 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:18.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.252 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:18:18.261 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:18:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:18.264 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:18:18.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:18:18.264 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:18.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.374 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:18.392 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:18:18.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.394 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:18.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.463 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:18.464 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:18.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:18:18.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:18:18.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:18:18.464 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:18.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.554 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:18:18.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.564 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-25T10:18:18.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.618 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:18.618 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:18.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:18.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:18:18.618 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:18.700 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:18:18.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:18:18.701 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:18.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:18.868 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:18.868 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:18.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:18:18.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:18:18.868 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:18.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:18.909 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:18.962 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:18.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:18.962 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:19.013 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:18:19.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:19.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:19.107 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:19.107 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:19.700 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:19.700 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:19.700 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:18:19.700 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:18:19.701 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:18:19.705 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:19.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:19.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:19.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:19.727 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:19.800 INFO:teuthology.orchestra.run.smithi045.stdout:client4644 2024-04-25T10:18:19.801 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:19.801 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:19.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:19.849 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:19.913 INFO:teuthology.orchestra.run.smithi045.stdout:client4644 2024-04-25T10:18:19.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:19.913 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:19.976 INFO:teuthology.orchestra.run.smithi045.stdout:client4644 2024-04-25T10:18:19.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:19.976 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7e12072-0435-4dda-862b-d1045153027e.client4644/status') 2024-04-25T10:18:20.047 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4644 (3)192.168.0.1:0/3060215460 2024-04-25T10:18:20.047 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-04-25T10:18:20.047 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-04-25T10:18:20.048 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-04-25T10:18:20.048 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 5.6227e-05 s, 1.2 MB/s 2024-04-25T10:18:20.049 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T10:18:20.049 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:18:20.050 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:18:20.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:20.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.228+0000 7feeeac00640 1 Processor -- start 2024-04-25T10:18:20.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.228+0000 7feeeac00640 1 -- start start 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feeeac00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feee40cda10 0x7feee406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feeeac00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feee40cf930 0x7feee40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feeeac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40b2770 con 0x7feee415bc40 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feeeac00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee406c670 con 0x7feee40cda10 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feeeac00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee406c950 con 0x7feee40cf930 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feee3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feee415bc40 0x7feee40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51338/0 (socket says 172.21.15.45:51338) 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feee3e00640 1 -- 172.21.15.45:0/4105704598 learned_addr learned my addr 172.21.15.45:0/4105704598 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feedb400640 1 --2- 172.21.15.45:0/4105704598 >> v2:172.21.15.45:3300/0 conn(0x7feee40cda10 0x7feee406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feee8c00640 1 --2- 172.21.15.45:0/4105704598 >> v2:172.21.15.117:3300/0 conn(0x7feee40cf930 0x7feee40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feee3400640 1 -- 172.21.15.45:0/4105704598 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016887092 0 0) 0x7feee40b2770 con 0x7feee415bc40 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feee3400640 1 -- 172.21.15.45:0/4105704598 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec8003530 con 0x7feee415bc40 2024-04-25T10:18:20.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feedb400640 1 -- 172.21.15.45:0/4105704598 >> v2:172.21.15.117:3300/0 conn(0x7feee40cf930 msgr2=0x7feee40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:20.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feedb400640 1 --2- 172.21.15.45:0/4105704598 >> v2:172.21.15.117:3300/0 conn(0x7feee40cf930 0x7feee40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feedb400640 1 -- 172.21.15.45:0/4105704598 >> v1:172.21.15.45:6789/0 conn(0x7feee415bc40 legacy=0x7feee40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.232+0000 7feedb400640 1 -- 172.21.15.45:0/4105704598 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee406cc30 con 0x7feee40cda10 2024-04-25T10:18:20.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feedb400640 1 --2- 172.21.15.45:0/4105704598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee406c090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feee40ae4c0 tx=0x7feecc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f97139790b3eb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feee3400640 1 -- 172.21.15.45:0/4105704598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0b0070 con 0x7feee40cda10 2024-04-25T10:18:20.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feee3400640 1 -- 172.21.15.45:0/4105704598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0ab600 con 0x7feee40cda10 2024-04-25T10:18:20.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feee3400640 1 -- 172.21.15.45:0/4105704598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc0ab9d0 con 0x7feee40cda10 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- 172.21.15.45:0/4105704598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 msgr2=0x7feee406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 --2- 172.21.15.45:0/4105704598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee406c090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feee40ae4c0 tx=0x7feecc01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- 172.21.15.45:0/4105704598 shutdown_connections 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 --2- 172.21.15.45:0/4105704598 >> v2:172.21.15.117:3300/0 conn(0x7feee40cf930 0x7feee40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 --2- 172.21.15.45:0/4105704598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee406c090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- 172.21.15.45:0/4105704598 >> 172.21.15.45:0/4105704598 conn(0x7feee4066b70 msgr2=0x7feee4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- 172.21.15.45:0/4105704598 shutdown_connections 2024-04-25T10:18:20.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- 172.21.15.45:0/4105704598 wait complete. 2024-04-25T10:18:20.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 Processor -- start 2024-04-25T10:18:20.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- start start 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feee40cf930 0x7feee415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee415bc40 0x7feee40cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40b3850 con 0x7feee40cda10 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40b36d0 con 0x7feee415bc40 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feeeac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee40b39d0 con 0x7feee40cf930 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.236+0000 7feee3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48934/0 (socket says 172.21.15.45:48934) 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 -- 172.21.15.45:0/3404771479 learned_addr learned my addr 172.21.15.45:0/3404771479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee8c00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee415bc40 0x7feee40cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feedb400640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feee40cf930 0x7feee415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 -- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feee40cf930 msgr2=0x7feee415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feee40cf930 0x7feee415fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 -- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee415bc40 msgr2=0x7feee40cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee415bc40 0x7feee40cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 -- 172.21.15.45:0/3404771479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee415d870 con 0x7feee40cda10 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee3e00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee415d200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feed400c3c0 tx=0x7feed400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4017070 con 0x7feee40cda10 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4005cf0 con 0x7feee40cda10 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee415db50 con 0x7feee40cda10 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40072f0 con 0x7feee40cda10 2024-04-25T10:18:20.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee415e3f0 con 0x7feee40cda10 2024-04-25T10:18:20.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee415df40 con 0x7feee40cda10 2024-04-25T10:18:20.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feed401a070 con 0x7feee40cda10 2024-04-25T10:18:20.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee0c00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feec0066f60 0x7feec0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:20.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.240+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feed4084480 con 0x7feee40cda10 2024-04-25T10:18:20.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.244+0000 7feedb400640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feec0066f60 0x7feec0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:20.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.244+0000 7feedb400640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feec0066f60 0x7feec0069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feecc0ad000 tx=0x7feecc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:20.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.244+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feed4055480 con 0x7feee40cda10 2024-04-25T10:18:20.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feee404ef30 con 0x7feee40cda10 2024-04-25T10:18:20.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feee0c00640 1 -- 172.21.15.45:0/3404771479 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7feed4054d80 con 0x7feee40cda10 2024-04-25T10:18:20.447 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feec0066f60 msgr2=0x7feec0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feec0066f60 0x7feec0069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feecc0ad000 tx=0x7feecc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 msgr2=0x7feee415d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee415d200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feed400c3c0 tx=0x7feed400c880 comp rx=0 tx=0).stop 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 shutdown_connections 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feec0066f60 0x7feec0069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee415bc40 0x7feee40cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feee40cf930 0x7feee415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 --2- 172.21.15.45:0/3404771479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee40cda10 0x7feee415d200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 >> 172.21.15.45:0/3404771479 conn(0x7feee4066b70 msgr2=0x7feee4158a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 shutdown_connections 2024-04-25T10:18:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:20.444+0000 7feeeac00640 1 -- 172.21.15.45:0/3404771479 wait complete. 2024-04-25T10:18:20.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:18:20.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:20.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:18:20.469 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:18:20.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:18:20.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:18:20.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:18:20.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:20.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:18:20.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.470 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:18:20.479 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:20.479 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:20.480 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42440sec preferred_lft 42440sec 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:18:20.481 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.482 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:20.539 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:18:20.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.541 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:18:20.595 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:18:20.595 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:20.595 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:18:20.595 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:18:20.595 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:18:20.595 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:20.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.596 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:20.596 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:20.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:18:20.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:18:20.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:18:20.596 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:20.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.687 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:18:20.696 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:18:20.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.697 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-25T10:18:20.749 INFO:teuthology.orchestra.run.smithi045.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:18:20.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.750 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:20.750 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:20.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:18:20.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:18:20.751 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:20.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:20.893 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 3e:31:8e:79:ce:66 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::3c31:8eff:fe79:ce66/64 scope link 2024-04-25T10:18:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:18:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:20.961 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 3e:31:8e:79:ce:66 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::3c31:8eff:fe79:ce66/64 scope link 2024-04-25T10:18:20.962 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:20.963 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:18:20.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:18:20.964 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:21.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.099 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:18:21.099 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:18:21.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.1 up 2024-04-25T10:18:21.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:18:21.099 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:18:21.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:21.138 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:18:21.193 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:18:21.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:21.194 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:18:21.249 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T10:18:21.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:21.342 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:21.356 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:21.356 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:21.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:18:21.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 2@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:18:21.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:18:21.360 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:21.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:18:21.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.380 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:21.437 INFO:teuthology.orchestra.run.smithi045.stdout:client4656 2024-04-25T10:18:21.437 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:21.437 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:21.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.490 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:21.544 INFO:teuthology.orchestra.run.smithi045.stdout:client4656 2024-04-25T10:18:21.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.544 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:21.596 INFO:teuthology.orchestra.run.smithi045.stdout:client4656 2024-04-25T10:18:21.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:21.597 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7e12072-0435-4dda-862b-d1045153027e.client4656/status') 2024-04-25T10:18:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4656 (3)192.168.0.2:0/1767847528 2024-04-25T10:18:21.657 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-04-25T10:18:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-04-25T10:18:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-04-25T10:18:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:69 bytes copied, 5.3752e-05 s, 1.3 MB/s 2024-04-25T10:18:21.659 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T10:18:21.659 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:18:21.659 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:18:21.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 Processor -- start 2024-04-25T10:18:21.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 -- start start 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 0x7f4d6006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d600cda10 0x7f4d600b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d600b2770 con 0x7f4d600cf930 2024-04-25T10:18:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6006c670 con 0x7f4d600cda10 2024-04-25T10:18:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d66600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6006c950 con 0x7f4d6015bc40 2024-04-25T10:18:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4d600cf930 0x7f4d6015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51344/0 (socket says 172.21.15.45:51344) 2024-04-25T10:18:21.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5fe00640 1 -- 172.21.15.45:0/1905470251 learned_addr learned my addr 172.21.15.45:0/1905470251 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5f400640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 0x7f4d6006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5ea00640 1 --2- 172.21.15.45:0/1905470251 >> v2:172.21.15.117:3300/0 conn(0x7f4d600cda10 0x7f4d600b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032932596 0 0) 0x7f4d600b2770 con 0x7f4d600cf930 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d44003530 con 0x7f4d600cf930 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367314483 0 0) 0x7f4d44003530 con 0x7f4d600cf930 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 msgr2=0x7f4d6006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 0x7f4d6006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 >> v2:172.21.15.117:3300/0 conn(0x7f4d600cda10 msgr2=0x7f4d600b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 --2- 172.21.15.45:0/1905470251 >> v2:172.21.15.117:3300/0 conn(0x7f4d600cda10 0x7f4d600b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6006cc30 con 0x7f4d600cf930 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f4d58001680 con 0x7f4d600cf930 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 >> v1:172.21.15.45:6789/0 conn(0x7f4d600cf930 legacy=0x7f4d6015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 0x7f4d440039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d44004d50 0x7f4d440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6006c670 con 0x7f4d44004d50 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6006c950 con 0x7f4d44003f40 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4400b9d0 con 0x7f4d6015bc40 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5ea00640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5f400640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 0x7f4d440039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.840+0000 7f4d5fe00640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d44004d50 0x7f4d440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5ea00640 1 -- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 msgr2=0x7f4d440039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5ea00640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015bc40 0x7f4d440039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5ea00640 1 -- 172.21.15.45:0/1905470251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d44004d50 msgr2=0x7f4d440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5ea00640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d44004d50 0x7f4d440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5ea00640 1 -- 172.21.15.45:0/1905470251 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6006cc30 con 0x7f4d44003f40 2024-04-25T10:18:21.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5fe00640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d44004d50 0x7f4d440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5ea00640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d44004810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009ac0 tx=0x7f4d4c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36713015a88ac45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0b2070 con 0x7f4d44003f40 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c0a5680 con 0x7f4d44003f40 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5e000640 1 -- 172.21.15.45:0/1905470251 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c009690 con 0x7f4d44003f40 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d66600640 1 -- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 msgr2=0x7f4d44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d66600640 1 --2- 172.21.15.45:0/1905470251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d44004810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009ac0 tx=0x7f4d4c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d5fe00640 1 -- 172.21.15.45:0/1905470251 reap_dead start 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d66600640 1 -- 172.21.15.45:0/1905470251 shutdown_connections 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d66600640 1 -- 172.21.15.45:0/1905470251 >> 172.21.15.45:0/1905470251 conn(0x7f4d60066b70 msgr2=0x7f4d60066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d66600640 1 -- 172.21.15.45:0/1905470251 shutdown_connections 2024-04-25T10:18:21.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.844+0000 7f4d66600640 1 -- 172.21.15.45:0/1905470251 wait complete. 2024-04-25T10:18:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 Processor -- start 2024-04-25T10:18:21.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 -- start start 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d601f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015db40 0x7f4d60145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 0x7f4d60145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6006c090 con 0x7f4d6015dfc0 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d600b4b50 con 0x7f4d44003f40 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6006c210 con 0x7f4d6015db40 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 0x7f4d60145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 0x7f4d60145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48966/0 (socket says 172.21.15.45:48966) 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 -- 172.21.15.45:0/1330088961 learned_addr learned my addr 172.21.15.45:0/1330088961 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:21.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5f400640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d601f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5ea00640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015db40 0x7f4d60145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 -- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015db40 msgr2=0x7f4d60145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015db40 0x7f4d60145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 -- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 msgr2=0x7f4d601f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d601f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 -- 172.21.15.45:0/1330088961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60148c50 con 0x7f4d6015dfc0 2024-04-25T10:18:21.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d5fe00640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 0x7f4d60145550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d600b44f0 tx=0x7f4d58007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58007d80 con 0x7f4d6015dfc0 2024-04-25T10:18:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58018070 con 0x7f4d6015dfc0 2024-04-25T10:18:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800fd50 con 0x7f4d6015dfc0 2024-04-25T10:18:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.848+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d60148970 con 0x7f4d6015dfc0 2024-04-25T10:18:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.852+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d601f5720 con 0x7f4d6015dfc0 2024-04-25T10:18:21.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.852+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6004ef30 con 0x7f4d6015dfc0 2024-04-25T10:18:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.852+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d5801d460 con 0x7f4d6015dfc0 2024-04-25T10:18:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.852+0000 7f4d57400640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d40066f60 0x7f4d40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.852+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d58084ff0 con 0x7f4d6015dfc0 2024-04-25T10:18:21.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.852+0000 7f4d5f400640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d40066f60 0x7f4d40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:21.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.856+0000 7f4d5f400640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d40066f60 0x7f4d40069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d50014430 tx=0x7f4d5000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:21.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:21.856+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d58055910 con 0x7f4d6015dfc0 2024-04-25T10:18:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.052+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d600af1c0 con 0x7f4d6015dfc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.052+0000 7f4d57400640 1 -- 172.21.15.45:0/1330088961 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d580599b0 con 0x7f4d6015dfc0 2024-04-25T10:18:22.057 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d40066f60 msgr2=0x7f4d40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d40066f60 0x7f4d40069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d50014430 tx=0x7f4d5000b040 comp rx=0 tx=0).stop 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 msgr2=0x7f4d60145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 0x7f4d60145550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d600b44f0 tx=0x7f4d58007860 comp rx=0 tx=0).stop 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 shutdown_connections 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d40066f60 0x7f4d40069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d6015dfc0 0x7f4d60145550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d6015db40 0x7f4d60145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 --2- 172.21.15.45:0/1330088961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d44003f40 0x7f4d601f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 >> 172.21.15.45:0/1330088961 conn(0x7f4d60066b70 msgr2=0x7f4d600b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 shutdown_connections 2024-04-25T10:18:22.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:22.056+0000 7f4d66600640 1 -- 172.21.15.45:0/1330088961 wait complete. 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:18:22.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:18:22.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:22.079 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:18:22.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.079 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-04-25T10:18:22.088 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:22.089 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42430sec preferred_lft 42430sec 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 02:42:0e:f3:64:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:18:22.090 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:22.091 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:18:22.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.091 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:22.092 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:22.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:18:22.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-04-25T10:18:22.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-04-25T10:18:22.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:18:22.092 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:22.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.212 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:22.231 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-04-25T10:18:22.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.233 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:22.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.305 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:22.305 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:22.305 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:18:22.305 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:18:22.305 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:18:22.305 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:22.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.400 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-25T10:18:22.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.409 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-04-25T10:18:22.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.462 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:22.462 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:22.463 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:18:22.463 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:18:22.463 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:22.546 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:18:22.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:18:22.547 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:22.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.730 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:22.730 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:22.730 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-04-25T10:18:22.731 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:18:22.731 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:22.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:22.770 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:18:22.823 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:18:22.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:22.823 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:18:22.874 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T10:18:22.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:22.874 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:23.519 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:23.519 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:23.519 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:18:23.519 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with new device syntax: 1@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:18:23.519 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:18:23.523 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:23.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:23.524 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:18:23.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:23.543 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:23.608 INFO:teuthology.orchestra.run.smithi117.stdout:client4636 2024-04-25T10:18:23.609 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:23.609 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:23.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:23.659 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:23.712 INFO:teuthology.orchestra.run.smithi117.stdout:client4636 2024-04-25T10:18:23.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:23.713 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:23.764 INFO:teuthology.orchestra.run.smithi117.stdout:client4636 2024-04-25T10:18:23.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:23.765 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7e12072-0435-4dda-862b-d1045153027e.client4636/status') 2024-04-25T10:18:23.825 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.4636 (3)192.168.0.1:0/3839939997 2024-04-25T10:18:23.826 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-04-25T10:18:23.826 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-04-25T10:18:23.826 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-04-25T10:18:23.826 INFO:teuthology.orchestra.run.smithi117.stderr:69 bytes copied, 6.1502e-05 s, 1.1 MB/s 2024-04-25T10:18:23.828 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T10:18:23.828 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:18:23.828 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:18:23.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:23.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.976+0000 7f209b000640 1 Processor -- start 2024-04-25T10:18:23.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.976+0000 7f209b000640 1 -- start start 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f209b000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2094158260 0x7f209415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f209b000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20940b5c20 con 0x7f2094066c00 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f209b000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20940af6e0 con 0x7f2094066fe0 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f209b000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20940af9c0 con 0x7f2094158260 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2093e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2094066c00 0x7f209406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51358/0 (socket says 172.21.15.45:51358) 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2093e00640 1 -- 172.21.15.45:0/2921143245 learned_addr learned my addr 172.21.15.45:0/2921143245 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:23.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2099000640 1 --2- 172.21.15.45:0/2921143245 >> v2:172.21.15.117:3300/0 conn(0x7f2094158260 0x7f209415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732476404 0 0) 0x7f20940b5c20 con 0x7f2094066c00 2024-04-25T10:18:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2078003530 con 0x7f2094066c00 2024-04-25T10:18:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559635681 0 0) 0x7f20940af6e0 con 0x7f2094066fe0 2024-04-25T10:18:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20940b5c20 con 0x7f2094066fe0 2024-04-25T10:18:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2099000640 1 -- 172.21.15.45:0/2921143245 >> v1:172.21.15.117:6789/0 conn(0x7f2094066fe0 legacy=0x7f2094157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2099000640 1 -- 172.21.15.45:0/2921143245 >> v1:172.21.15.45:6789/0 conn(0x7f2094066c00 legacy=0x7f209406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2099000640 1 -- 172.21.15.45:0/2921143245 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940afca0 con 0x7f2094158260 2024-04-25T10:18:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.980+0000 7f2099000640 1 --2- 172.21.15.45:0/2921143245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094158260 0x7f209415e110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f208c0097c0 tx=0x7f208c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed02fc590a3a6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0b1070 con 0x7f2094158260 2024-04-25T10:18:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c0ac410 con 0x7f2094158260 2024-04-25T10:18:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2093400640 1 -- 172.21.15.45:0/2921143245 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0ac7e0 con 0x7f2094158260 2024-04-25T10:18:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- 172.21.15.45:0/2921143245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094158260 msgr2=0x7f209415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 --2- 172.21.15.45:0/2921143245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094158260 0x7f209415e110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f208c0097c0 tx=0x7f208c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- 172.21.15.45:0/2921143245 shutdown_connections 2024-04-25T10:18:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 --2- 172.21.15.45:0/2921143245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094158260 0x7f209415e110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- 172.21.15.45:0/2921143245 >> 172.21.15.45:0/2921143245 conn(0x7f20940b24f0 msgr2=0x7f20940b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- 172.21.15.45:0/2921143245 shutdown_connections 2024-04-25T10:18:23.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- 172.21.15.45:0/2921143245 wait complete. 2024-04-25T10:18:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 Processor -- start 2024-04-25T10:18:23.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- start start 2024-04-25T10:18:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 0x7f2094160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094066fe0 0x7f209415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2094158260 0x7f209414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209416bb00 con 0x7f2094066c00 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20940af270 con 0x7f2094066fe0 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f209b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209416bc80 con 0x7f2094158260 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2093e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 0x7f2094160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2093e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 0x7f2094160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48968/0 (socket says 172.21.15.45:48968) 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f208b400640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094066fe0 0x7f209415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2093e00640 1 -- 172.21.15.45:0/1559899329 learned_addr learned my addr 172.21.15.45:0/1559899329 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.984+0000 7f2099000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2094158260 0x7f209414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f208b400640 1 -- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2094158260 msgr2=0x7f209414eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f208b400640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2094158260 0x7f209414eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f208b400640 1 -- 172.21.15.45:0/1559899329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 msgr2=0x7f2094160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f208b400640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 0x7f2094160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f208b400640 1 -- 172.21.15.45:0/1559899329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209415fb10 con 0x7f2094066fe0 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2093e00640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 0x7f2094160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2099000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2094158260 0x7f209414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f208b400640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094066fe0 0x7f209415f250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c3c0 tx=0x7f207c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c017070 con 0x7f2094066fe0 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c0052f0 con 0x7f2094066fe0 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c003840 con 0x7f2094066fe0 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209415fdf0 con 0x7f2094066fe0 2024-04-25T10:18:23.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20941603b0 con 0x7f2094066fe0 2024-04-25T10:18:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209404ef30 con 0x7f2094066fe0 2024-04-25T10:18:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f207c01a070 con 0x7f2094066fe0 2024-04-25T10:18:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2090c00640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2070066f60 0x7f2070069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:23.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.988+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f207c084390 con 0x7f2094066fe0 2024-04-25T10:18:23.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.992+0000 7f2093e00640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2070066f60 0x7f2070069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:23.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.992+0000 7f2093e00640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2070066f60 0x7f2070069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2084002240 tx=0x7f2084009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:23.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:23.996+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f207c054cb0 con 0x7f2094066fe0 2024-04-25T10:18:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2094071c70 con 0x7f2094066fe0 2024-04-25T10:18:24.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f2090c00640 1 -- 172.21.15.45:0/1559899329 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f207c058d50 con 0x7f2094066fe0 2024-04-25T10:18:24.199 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2070066f60 msgr2=0x7f2070069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2070066f60 0x7f2070069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2084002240 tx=0x7f2084009040 comp rx=0 tx=0).stop 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094066fe0 msgr2=0x7f209415f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094066fe0 0x7f209415f250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c3c0 tx=0x7f207c00c880 comp rx=0 tx=0).stop 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 shutdown_connections 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2070066f60 0x7f2070069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2094158260 0x7f209414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2094066fe0 0x7f209415f250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 --2- 172.21.15.45:0/1559899329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2094066c00 0x7f2094160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:24.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 >> 172.21.15.45:0/1559899329 conn(0x7f20940b24f0 msgr2=0x7f20940b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:24.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 shutdown_connections 2024-04-25T10:18:24.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:24.196+0000 7f209b000640 1 -- 172.21.15.45:0/1559899329 wait complete. 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:18:24.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:18:24.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:24.221 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:18:24.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.221 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-04-25T10:18:24.230 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:24.230 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:18:24.231 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 02:42:0e:f3:64:f1 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:24.232 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:18:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:18:24.233 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.234 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:24.295 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-04-25T10:18:24.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.298 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:24.353 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:24.354 INFO:teuthology.orchestra.run.smithi117.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:18:24.354 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:24.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.354 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:24.354 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:24.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:18:24.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:18:24.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:18:24.354 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:24.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.447 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-25T10:18:24.454 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:18:24.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.456 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-04-25T10:18:24.509 INFO:teuthology.orchestra.run.smithi117.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:18:24.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.510 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:24.510 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:24.511 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:18:24.511 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:18:24.511 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:24.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.592 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:18:24.654 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:24.654 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:24.654 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:24.654 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.654 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:24.654 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.655 INFO:teuthology.orchestra.run.smithi117.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:24.655 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 7a:34:b9:91:41:fc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:18:24.655 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:18:24.655 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.655 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::7834:b9ff:fe91:41fc/64 scope link 2024-04-25T10:18:24.655 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.656 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:18:24.717 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:24.717 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:24.717 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:24.717 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.717 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 7a:34:b9:91:41:fc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::7834:b9ff:fe91:41fc/64 scope link 2024-04-25T10:18:24.718 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:24.719 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:18:24.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.719 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:24.719 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:24.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:18:24.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:18:24.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:18:24.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:18:24.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:18:24.720 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:24.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.855 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec bash -c ' 2024-04-25T10:18:24.855 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-04-25T10:18:24.855 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.1 up 2024-04-25T10:18:24.856 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:18:24.856 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-04-25T10:18:24.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:24.893 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:18:24.942 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:18:24.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:24.942 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:18:24.993 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T10:18:24.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:24.994 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:25.081 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:25.081 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:25.094 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:18:25.094 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:18:25.094 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:18:25.094 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with new device syntax: 3@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:18:25.094 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:18:25.098 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:25.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:25.098 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:18:25.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:25.156 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:25.208 INFO:teuthology.orchestra.run.smithi117.stdout:client4648 2024-04-25T10:18:25.209 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:25.209 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:25.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:25.258 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:25.309 INFO:teuthology.orchestra.run.smithi117.stdout:client4648 2024-04-25T10:18:25.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:25.309 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:25.360 INFO:teuthology.orchestra.run.smithi117.stdout:client4648 2024-04-25T10:18:25.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:25.361 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d7e12072-0435-4dda-862b-d1045153027e.client4648/status') 2024-04-25T10:18:25.421 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.4648 (3)192.168.0.2:0/913662047 2024-04-25T10:18:25.421 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-04-25T10:18:25.421 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-04-25T10:18:25.421 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-04-25T10:18:25.421 INFO:teuthology.orchestra.run.smithi117.stderr:68 bytes copied, 5.011e-05 s, 1.4 MB/s 2024-04-25T10:18:25.423 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:18:25.438 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:25.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 Processor -- start 2024-04-25T10:18:25.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 -- start start 2024-04-25T10:18:25.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec6415a0e0 0x7fec6415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec640b3190 0x7fec6406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6415b6e0 con 0x7fec64071f70 2024-04-25T10:18:25.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64070ed0 con 0x7fec640b3190 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec6be00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640711b0 con 0x7fec6415a0e0 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec68a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec64071f70 0x7fec640b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50044/0 (socket says 172.21.15.45:50044) 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec68a00640 1 -- 172.21.15.45:0/3696767500 learned_addr learned my addr 172.21.15.45:0/3696767500 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec69e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec640b3190 0x7fec6406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec69400640 1 --2- 172.21.15.45:0/3696767500 >> v2:172.21.15.117:3300/0 conn(0x7fec6415a0e0 0x7fec6415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec69e00640 1 -- 172.21.15.45:0/3696767500 >> v2:172.21.15.117:3300/0 conn(0x7fec6415a0e0 msgr2=0x7fec6415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec69e00640 1 --2- 172.21.15.45:0/3696767500 >> v2:172.21.15.117:3300/0 conn(0x7fec6415a0e0 0x7fec6415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec69e00640 1 -- 172.21.15.45:0/3696767500 >> v1:172.21.15.45:6789/0 conn(0x7fec64071f70 legacy=0x7fec640b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec69e00640 1 -- 172.21.15.45:0/3696767500 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64071490 con 0x7fec640b3190 2024-04-25T10:18:25.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.588+0000 7fec63e00640 1 -- 172.21.15.45:0/3696767500 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452524230 0 0) 0x7fec50001120 con 0x7fec64071f70 2024-04-25T10:18:25.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec69e00640 1 --2- 172.21.15.45:0/3696767500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec640b3190 0x7fec6406c090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec640d0800 tx=0x7fec5c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec47470a45f86b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec63e00640 1 -- 172.21.15.45:0/3696767500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0b2070 con 0x7fec640b3190 2024-04-25T10:18:25.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec63e00640 1 -- 172.21.15.45:0/3696767500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0a58b0 con 0x7fec640b3190 2024-04-25T10:18:25.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec63e00640 1 -- 172.21.15.45:0/3696767500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0a5c80 con 0x7fec640b3190 2024-04-25T10:18:25.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 -- 172.21.15.45:0/3696767500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec640b3190 msgr2=0x7fec6406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 --2- 172.21.15.45:0/3696767500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec640b3190 0x7fec6406c090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec640d0800 tx=0x7fec5c01c870 comp rx=0 tx=0).stop 2024-04-25T10:18:25.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 -- 172.21.15.45:0/3696767500 shutdown_connections 2024-04-25T10:18:25.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 --2- 172.21.15.45:0/3696767500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec640b3190 0x7fec6406c090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 --2- 172.21.15.45:0/3696767500 >> v2:172.21.15.117:3300/0 conn(0x7fec6415a0e0 0x7fec6415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 -- 172.21.15.45:0/3696767500 >> 172.21.15.45:0/3696767500 conn(0x7fec64066b70 msgr2=0x7fec64066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 -- 172.21.15.45:0/3696767500 shutdown_connections 2024-04-25T10:18:25.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.592+0000 7fec6be00640 1 -- 172.21.15.45:0/3696767500 wait complete. 2024-04-25T10:18:25.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 Processor -- start 2024-04-25T10:18:25.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 -- start start 2024-04-25T10:18:25.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec64071f70 0x7fec6414ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec640b3190 0x7fec640afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 0x7fec64147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64070b50 con 0x7fec6415a0e0 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6415a4c0 con 0x7fec640b3190 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec640723b0 con 0x7fec64071f70 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 0x7fec64147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 0x7fec64147cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42054/0 (socket says 172.21.15.45:42054) 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 -- 172.21.15.45:0/1826142003 learned_addr learned my addr 172.21.15.45:0/1826142003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec68a00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec640b3190 0x7fec640afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69400640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec64071f70 0x7fec6414ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 -- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec64071f70 msgr2=0x7fec6414ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec64071f70 0x7fec6414ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 -- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec640b3190 msgr2=0x7fec640afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec640b3190 0x7fec640afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 -- 172.21.15.45:0/1826142003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64151f90 con 0x7fec6415a0e0 2024-04-25T10:18:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec69e00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 0x7fec64147cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec5c019480 tx=0x7fec5c019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0b2050 con 0x7fec6415a0e0 2024-04-25T10:18:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0b3040 con 0x7fec6415a0e0 2024-04-25T10:18:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0ad850 con 0x7fec6415a0e0 2024-04-25T10:18:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec64157c80 con 0x7fec6415a0e0 2024-04-25T10:18:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.596+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64158240 con 0x7fec6415a0e0 2024-04-25T10:18:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.600+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6404ef30 con 0x7fec6415a0e0 2024-04-25T10:18:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.600+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec5c0c0020 con 0x7fec6415a0e0 2024-04-25T10:18:25.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.600+0000 7fec61600640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec44066f60 0x7fec44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.600+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec5c11f9d0 con 0x7fec6415a0e0 2024-04-25T10:18:25.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.600+0000 7fec69400640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec44066f60 0x7fec44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.600+0000 7fec69400640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec44066f60 0x7fec44069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec58007a60 tx=0x7fec58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.604+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec5c016520 con 0x7fec6415a0e0 2024-04-25T10:18:25.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:25.696 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:25.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.772+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec640703e0 con 0x7fec6415a0e0 2024-04-25T10:18:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.772+0000 7fec61600640 1 -- 172.21.15.45:0/1826142003 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fec5c0f04a0 con 0x7fec6415a0e0 2024-04-25T10:18:25.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:25.776 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec44066f60 msgr2=0x7fec44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec44066f60 0x7fec44069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec58007a60 tx=0x7fec58008040 comp rx=0 tx=0).stop 2024-04-25T10:18:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 msgr2=0x7fec64147cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 0x7fec64147cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec5c019480 tx=0x7fec5c019610 comp rx=0 tx=0).stop 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 shutdown_connections 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec44066f60 0x7fec44069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec6415a0e0 0x7fec64147cf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec640b3190 0x7fec640afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 --2- 172.21.15.45:0/1826142003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec64071f70 0x7fec6414ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 >> 172.21.15.45:0/1826142003 conn(0x7fec64066b70 msgr2=0x7fec64150460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 shutdown_connections 2024-04-25T10:18:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.776+0000 7fec6be00640 1 -- 172.21.15.45:0/1826142003 wait complete. 2024-04-25T10:18:25.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:25.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:25.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.984+0000 7fe3a2600640 1 Processor -- start 2024-04-25T10:18:25.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.984+0000 7fe3a2600640 1 -- start start 2024-04-25T10:18:25.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe39c15d810 0x7fe39c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a2600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39c0b59f0 con 0x7fe39c0cff90 2024-04-25T10:18:25.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a2600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe39c0b5cd0 con 0x7fe39c066ee0 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a2600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39c067350 con 0x7fe39c15d810 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe39c15d810 0x7fe39c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe39c15d810 0x7fe39c0669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42064/0 (socket says 172.21.15.45:42064) 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1200640 1 -- 172.21.15.45:0/2594972406 learned_addr learned my addr 172.21.15.45:0/2594972406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1c00640 1 --1- 172.21.15.45:0/2594972406 >> v1:172.21.15.45:6789/0 conn(0x7fe39c0cff90 0x7fe39c15dbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50060/0 (socket says 172.21.15.45:50060) 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1200640 1 -- 172.21.15.45:0/2594972406 >> v1:172.21.15.117:6789/0 conn(0x7fe39c066ee0 legacy=0x7fe39c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1200640 1 -- 172.21.15.45:0/2594972406 >> v1:172.21.15.45:6789/0 conn(0x7fe39c0cff90 legacy=0x7fe39c15dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.988+0000 7fe3a1200640 1 -- 172.21.15.45:0/2594972406 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0b5fb0 con 0x7fe39c15d810 2024-04-25T10:18:25.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a1200640 1 --2- 172.21.15.45:0/2594972406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c15d810 0x7fe39c0669a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe390009870 tx=0x7fe39001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72cc3533ae6a7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:25.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe39b400640 1 -- 172.21.15.45:0/2594972406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900b0070 con 0x7fe39c15d810 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe39b400640 1 -- 172.21.15.45:0/2594972406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390004230 con 0x7fe39c15d810 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe39b400640 1 -- 172.21.15.45:0/2594972406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390003620 con 0x7fe39c15d810 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- 172.21.15.45:0/2594972406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c15d810 msgr2=0x7fe39c0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 --2- 172.21.15.45:0/2594972406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c15d810 0x7fe39c0669a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe390009870 tx=0x7fe39001c440 comp rx=0 tx=0).stop 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- 172.21.15.45:0/2594972406 shutdown_connections 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 --2- 172.21.15.45:0/2594972406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c15d810 0x7fe39c0669a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- 172.21.15.45:0/2594972406 >> 172.21.15.45:0/2594972406 conn(0x7fe39c06ac90 msgr2=0x7fe39c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- 172.21.15.45:0/2594972406 shutdown_connections 2024-04-25T10:18:25.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- 172.21.15.45:0/2594972406 wait complete. 2024-04-25T10:18:25.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 Processor -- start 2024-04-25T10:18:25.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- start start 2024-04-25T10:18:25.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe39c066ee0 0x7fe39c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 0x7fe39c151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe39c15d810 0x7fe39c15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c15eea0 con 0x7fe39c0cff90 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c15ed20 con 0x7fe39c066ee0 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe3a2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c16b740 con 0x7fe39c15d810 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.992+0000 7fe39be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 0x7fe39c151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 0x7fe39c151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42072/0 (socket says 172.21.15.45:42072) 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 -- 172.21.15.45:0/1240725008 learned_addr learned my addr 172.21.15.45:0/1240725008 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe3a1200640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe39c066ee0 0x7fe39c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe3a1c00640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe39c15d810 0x7fe39c15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 -- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe39c15d810 msgr2=0x7fe39c15fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe39c15d810 0x7fe39c15fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 -- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe39c066ee0 msgr2=0x7fe39c153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe39c066ee0 0x7fe39c153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 -- 172.21.15.45:0/1240725008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c14d740 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe39be00640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 0x7fe39c151660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c370 tx=0x7fe38800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880052d0 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388005470 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c139a00 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388007260 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c153910 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c075120 con 0x7fe39c0cff90 2024-04-25T10:18:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe388007400 con 0x7fe39c0cff90 2024-04-25T10:18:26.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe398c00640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe37c066f60 0x7fe37c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:25.996+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe388084480 con 0x7fe39c0cff90 2024-04-25T10:18:26.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.000+0000 7fe3a1200640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe37c066f60 0x7fe37c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.000+0000 7fe3a1200640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe37c066f60 0x7fe37c069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ad000 tx=0x7fe3900a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.000+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe39c14d740 con 0x7fe39c0cff90 2024-04-25T10:18:26.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.164+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe39c071260 con 0x7fe39c0cff90 2024-04-25T10:18:26.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.164+0000 7fe398c00640 1 -- 172.21.15.45:0/1240725008 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe388054da0 con 0x7fe39c0cff90 2024-04-25T10:18:26.169 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe37c066f60 msgr2=0x7fe37c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe37c066f60 0x7fe37c069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ad000 tx=0x7fe3900a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 msgr2=0x7fe39c151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 0x7fe39c151660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c370 tx=0x7fe38800c830 comp rx=0 tx=0).stop 2024-04-25T10:18:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 shutdown_connections 2024-04-25T10:18:26.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe37c066f60 0x7fe37c069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe39c15d810 0x7fe39c15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe39c0cff90 0x7fe39c151660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 --2- 172.21.15.45:0/1240725008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe39c066ee0 0x7fe39c153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 >> 172.21.15.45:0/1240725008 conn(0x7fe39c06ac90 msgr2=0x7fe39c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 shutdown_connections 2024-04-25T10:18:26.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.168+0000 7fe3a2600640 1 -- 172.21.15.45:0/1240725008 wait complete. 2024-04-25T10:18:26.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:26.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.328+0000 7effc6200640 1 Processor -- start 2024-04-25T10:18:26.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.328+0000 7effc6200640 1 -- start start 2024-04-25T10:18:26.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effc6200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effc015dd30 0x7effc006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effc6200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effc00d1fb0 0x7effc00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effc6200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc006b1d0 con 0x7effc00cfb30 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effc6200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc015e110 con 0x7effc015dd30 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effc6200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc006b470 con 0x7effc00d1fb0 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effc015dd30 0x7effc006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effc015dd30 0x7effc006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42074/0 (socket says 172.21.15.45:42074) 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 -- 172.21.15.45:0/3542592872 learned_addr learned my addr 172.21.15.45:0/3542592872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbfe00640 1 --2- 172.21.15.45:0/3542592872 >> v2:172.21.15.117:3300/0 conn(0x7effc00d1fb0 0x7effc00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 -- 172.21.15.45:0/3542592872 >> v2:172.21.15.117:3300/0 conn(0x7effc00d1fb0 msgr2=0x7effc00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:26.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 --2- 172.21.15.45:0/3542592872 >> v2:172.21.15.117:3300/0 conn(0x7effc00d1fb0 0x7effc00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 -- 172.21.15.45:0/3542592872 >> v1:172.21.15.117:6789/0 conn(0x7effc00cfb30 legacy=0x7effc00d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.332+0000 7effbf400640 1 -- 172.21.15.45:0/3542592872 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc006b750 con 0x7effc015dd30 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effbf400640 1 --2- 172.21.15.45:0/3542592872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc015dd30 0x7effc006ac90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effc00c89f0 tx=0x7effb001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90038151ec32b6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effbea00640 1 -- 172.21.15.45:0/3542592872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00ab610 con 0x7effc015dd30 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effbea00640 1 -- 172.21.15.45:0/3542592872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00b0070 con 0x7effc015dd30 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effbea00640 1 -- 172.21.15.45:0/3542592872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00ab9d0 con 0x7effc015dd30 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- 172.21.15.45:0/3542592872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc015dd30 msgr2=0x7effc006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 --2- 172.21.15.45:0/3542592872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc015dd30 0x7effc006ac90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7effc00c89f0 tx=0x7effb001c520 comp rx=0 tx=0).stop 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- 172.21.15.45:0/3542592872 shutdown_connections 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 --2- 172.21.15.45:0/3542592872 >> v2:172.21.15.117:3300/0 conn(0x7effc00d1fb0 0x7effc00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 --2- 172.21.15.45:0/3542592872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc015dd30 0x7effc006ac90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- 172.21.15.45:0/3542592872 >> 172.21.15.45:0/3542592872 conn(0x7effc009a9d0 msgr2=0x7effc009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- 172.21.15.45:0/3542592872 shutdown_connections 2024-04-25T10:18:26.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- 172.21.15.45:0/3542592872 wait complete. 2024-04-25T10:18:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 Processor -- start 2024-04-25T10:18:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- start start 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc00cfb30 0x7effc00af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 0x7effc01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc015dd30 0x7effc0159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc016ab60 con 0x7effc00d1fb0 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc00b2890 con 0x7effc015dd30 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.336+0000 7effc6200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc016ace0 con 0x7effc00cfb30 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 0x7effc01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 0x7effc01539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42084/0 (socket says 172.21.15.45:42084) 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 -- 172.21.15.45:0/3650666050 learned_addr learned my addr 172.21.15.45:0/3650666050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effbf400640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc00cfb30 0x7effc00af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effbfe00640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc015dd30 0x7effc0159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 -- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc00cfb30 msgr2=0x7effc00af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc00cfb30 0x7effc00af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 -- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc015dd30 msgr2=0x7effc0159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc015dd30 0x7effc0159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 -- 172.21.15.45:0/3650666050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00af950 con 0x7effc00d1fb0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb6a00640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 0x7effc01539d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7effa8007fd0 tx=0x7effa800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8017070 con 0x7effc00d1fb0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa80052d0 con 0x7effc00d1fb0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8003840 con 0x7effc00d1fb0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc00afc30 con 0x7effc00d1fb0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc00b01f0 con 0x7effc00d1fb0 2024-04-25T10:18:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc004ef30 con 0x7effc00d1fb0 2024-04-25T10:18:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effa801a070 con 0x7effc00d1fb0 2024-04-25T10:18:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb7e00640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff9c066f60 0x7eff9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.340+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effa8084330 con 0x7effc00d1fb0 2024-04-25T10:18:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.344+0000 7effbf400640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff9c066f60 0x7eff9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.344+0000 7effbf400640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff9c066f60 0x7eff9c069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effb00af000 tx=0x7effb00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.344+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effa8005480 con 0x7effc00d1fb0 2024-04-25T10:18:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effc00b7920 con 0x7effc00d1fb0 2024-04-25T10:18:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effb7e00640 1 -- 172.21.15.45:0/3650666050 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7effa8054ce0 con 0x7effc00d1fb0 2024-04-25T10:18:26.511 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff9c066f60 msgr2=0x7eff9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff9c066f60 0x7eff9c069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7effb00af000 tx=0x7effb00a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 msgr2=0x7effc01539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 0x7effc01539d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7effa8007fd0 tx=0x7effa800c890 comp rx=0 tx=0).stop 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 shutdown_connections 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff9c066f60 0x7eff9c069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc015dd30 0x7effc0159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc00d1fb0 0x7effc01539d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 --2- 172.21.15.45:0/3650666050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc00cfb30 0x7effc00af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 >> 172.21.15.45:0/3650666050 conn(0x7effc009a9d0 msgr2=0x7effc00cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 shutdown_connections 2024-04-25T10:18:26.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.508+0000 7effc6200640 1 -- 172.21.15.45:0/3650666050 wait complete. 2024-04-25T10:18:26.533 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.672+0000 7f235da00640 1 Processor -- start 2024-04-25T10:18:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.672+0000 7f235da00640 1 -- start start 2024-04-25T10:18:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f235da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f2358161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f235da00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235806ac90 con 0x7f23580cd9a0 2024-04-25T10:18:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f235da00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235806af50 con 0x7f2358161e50 2024-04-25T10:18:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f235da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235806b880 con 0x7f235806bd10 2024-04-25T10:18:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23580cd9a0 0x7f23580d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50062/0 (socket says 172.21.15.45:50062) 2024-04-25T10:18:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356a00640 1 -- 172.21.15.45:0/1674034383 learned_addr learned my addr 172.21.15.45:0/1674034383 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:26.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f234e000640 1 --2- 172.21.15.45:0/1674034383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f2358161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114420288 0 0) 0x7f235806ac90 con 0x7f23580cd9a0 2024-04-25T10:18:26.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2338003530 con 0x7f23580cd9a0 2024-04-25T10:18:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914173551 0 0) 0x7f235806af50 con 0x7f2358161e50 2024-04-25T10:18:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235806ac90 con 0x7f2358161e50 2024-04-25T10:18:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f234e000640 1 -- 172.21.15.45:0/1674034383 >> v1:172.21.15.117:6789/0 conn(0x7f2358161e50 legacy=0x7f235815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f234e000640 1 -- 172.21.15.45:0/1674034383 >> v1:172.21.15.45:6789/0 conn(0x7f23580cd9a0 legacy=0x7f23580d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952651656 0 0) 0x7f2338003530 con 0x7f23580cd9a0 2024-04-25T10:18:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.676+0000 7f234e000640 1 -- 172.21.15.45:0/1674034383 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235806b230 con 0x7f235806bd10 2024-04-25T10:18:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f234e000640 1 --2- 172.21.15.45:0/1674034383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f2358161910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f23400097f0 tx=0x7f234001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662c4a55b2d8ee43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23400b5070 con 0x7f235806bd10 2024-04-25T10:18:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23400a89d0 con 0x7f235806bd10 2024-04-25T10:18:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f2356000640 1 -- 172.21.15.45:0/1674034383 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23400a8da0 con 0x7f235806bd10 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- 172.21.15.45:0/1674034383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 msgr2=0x7f2358161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 --2- 172.21.15.45:0/1674034383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f2358161910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f23400097f0 tx=0x7f234001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- 172.21.15.45:0/1674034383 shutdown_connections 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 --2- 172.21.15.45:0/1674034383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f2358161910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- 172.21.15.45:0/1674034383 >> 172.21.15.45:0/1674034383 conn(0x7f23580d19a0 msgr2=0x7f23580d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- 172.21.15.45:0/1674034383 shutdown_connections 2024-04-25T10:18:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- 172.21.15.45:0/1674034383 wait complete. 2024-04-25T10:18:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 Processor -- start 2024-04-25T10:18:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- start start 2024-04-25T10:18:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f235809f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23580cd9a0 0x7f2358155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 0x7f2358155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358162660 con 0x7f2358161e50 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580cdd80 con 0x7f23580cd9a0 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f235da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580d0d70 con 0x7f235806bd10 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f2357400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 0x7f2358155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f2357400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 0x7f2358155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42094/0 (socket says 172.21.15.45:42094) 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.680+0000 7f2357400640 1 -- 172.21.15.45:0/3862564428 learned_addr learned my addr 172.21.15.45:0/3862564428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f234e000640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23580cd9a0 0x7f2358155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2356a00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f235809f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2357400640 1 -- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 msgr2=0x7f235809f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2357400640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f235809f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2357400640 1 -- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23580cd9a0 msgr2=0x7f2358155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2357400640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23580cd9a0 0x7f2358155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2357400640 1 -- 172.21.15.45:0/3862564428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23581566f0 con 0x7f2358161e50 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f2357400640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 0x7f2358155fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f235000c790 tx=0x7f235000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350003b80 con 0x7f2358161e50 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2350003d20 con 0x7f2358161e50 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000aa50 con 0x7f2358161e50 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23581f7090 con 0x7f2358161e50 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235809f5a0 con 0x7f2358161e50 2024-04-25T10:18:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235804ef30 con 0x7f2358161e50 2024-04-25T10:18:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.684+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f235000abf0 con 0x7f2358161e50 2024-04-25T10:18:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.688+0000 7f234f400640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2334066f60 0x7f2334069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.688+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2350084f80 con 0x7f2358161e50 2024-04-25T10:18:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.688+0000 7f2356a00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2334066f60 0x7f2334069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.688+0000 7f2356a00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2334066f60 0x7f2334069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2348005890 tx=0x7f2348009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:26.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.692+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23500558a0 con 0x7f2358161e50 2024-04-25T10:18:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23580b7920 con 0x7f2358161e50 2024-04-25T10:18:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f234f400640 1 -- 172.21.15.45:0/3862564428 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2350059940 con 0x7f2358161e50 2024-04-25T10:18:26.861 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2334066f60 msgr2=0x7f2334069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2334066f60 0x7f2334069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2348005890 tx=0x7f2348009040 comp rx=0 tx=0).stop 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 msgr2=0x7f2358155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 0x7f2358155fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f235000c790 tx=0x7f235000cc50 comp rx=0 tx=0).stop 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 shutdown_connections 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2334066f60 0x7f2334069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2358161e50 0x7f2358155fb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23580cd9a0 0x7f2358155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 --2- 172.21.15.45:0/3862564428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f235806bd10 0x7f235809f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 >> 172.21.15.45:0/3862564428 conn(0x7f23580d19a0 msgr2=0x7f23580ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 shutdown_connections 2024-04-25T10:18:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:26.856+0000 7f235da00640 1 -- 172.21.15.45:0/3862564428 wait complete. 2024-04-25T10:18:26.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.024+0000 7ff523400640 1 Processor -- start 2024-04-25T10:18:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.024+0000 7ff523400640 1 -- start start 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.024+0000 7ff523400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff51c0d1820 0x7ff51c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.024+0000 7ff523400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c0b1af0 con 0x7ff51c0b15b0 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.024+0000 7ff523400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c0b1dd0 con 0x7ff51c15e050 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.024+0000 7ff523400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51c0b20b0 con 0x7ff51c0d1820 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff51c0d1820 0x7ff51c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff51c0d1820 0x7ff51c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42102/0 (socket says 172.21.15.45:42102) 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 -- 172.21.15.45:0/813913705 learned_addr learned my addr 172.21.15.45:0/813913705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff521400640 1 --1- 172.21.15.45:0/813913705 >> v1:172.21.15.45:6789/0 conn(0x7ff51c0b15b0 0x7ff51c0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50078/0 (socket says 172.21.15.45:50078) 2024-04-25T10:18:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 -- 172.21.15.45:0/813913705 >> v1:172.21.15.117:6789/0 conn(0x7ff51c15e050 legacy=0x7ff51c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 -- 172.21.15.45:0/813913705 >> v1:172.21.15.45:6789/0 conn(0x7ff51c0b15b0 legacy=0x7ff51c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 -- 172.21.15.45:0/813913705 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0b2390 con 0x7ff51c0d1820 2024-04-25T10:18:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff520a00640 1 --2- 172.21.15.45:0/813913705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0d1820 0x7ff51c0d1c00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009870 tx=0x7ff50c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678c7f74d3d727a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff51be00640 1 -- 172.21.15.45:0/813913705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0a9640 con 0x7ff51c0d1820 2024-04-25T10:18:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff51be00640 1 -- 172.21.15.45:0/813913705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c0a97e0 con 0x7ff51c0d1820 2024-04-25T10:18:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.028+0000 7ff51be00640 1 -- 172.21.15.45:0/813913705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0a9bb0 con 0x7ff51c0d1820 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- 172.21.15.45:0/813913705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0d1820 msgr2=0x7ff51c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 --2- 172.21.15.45:0/813913705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0d1820 0x7ff51c0d1c00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009870 tx=0x7ff50c01c560 comp rx=0 tx=0).stop 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- 172.21.15.45:0/813913705 shutdown_connections 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 --2- 172.21.15.45:0/813913705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0d1820 0x7ff51c0d1c00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- 172.21.15.45:0/813913705 >> 172.21.15.45:0/813913705 conn(0x7ff51c09a9d0 msgr2=0x7ff51c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- 172.21.15.45:0/813913705 shutdown_connections 2024-04-25T10:18:27.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- 172.21.15.45:0/813913705 wait complete. 2024-04-25T10:18:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 Processor -- start 2024-04-25T10:18:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- start start 2024-04-25T10:18:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 0x7ff51c14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c0d1820 0x7ff51c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff51c15e050 0x7ff51c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0d0f20 con 0x7ff51c0b15b0 2024-04-25T10:18:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0d0da0 con 0x7ff51c0d1820 2024-04-25T10:18:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff523400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c16b9b0 con 0x7ff51c15e050 2024-04-25T10:18:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 0x7ff51c14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 0x7ff51c14d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42108/0 (socket says 172.21.15.45:42108) 2024-04-25T10:18:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 -- 172.21.15.45:0/1387300087 learned_addr learned my addr 172.21.15.45:0/1387300087 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff513e00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c0d1820 0x7ff51c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff521400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff51c15e050 0x7ff51c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 -- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff51c15e050 msgr2=0x7ff51c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff51c15e050 0x7ff51c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 -- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c0d1820 msgr2=0x7ff51c14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c0d1820 0x7ff51c14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 -- 172.21.15.45:0/1387300087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c158db0 con 0x7ff51c0b15b0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff513e00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c0d1820 0x7ff51c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff521400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff51c15e050 0x7ff51c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.032+0000 7ff520a00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 0x7ff51c14d470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff50c019170 tx=0x7ff50c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0b1880 con 0x7ff51c0b15b0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c0b1a20 con 0x7ff51c0b15b0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0b1e10 con 0x7ff51c0b15b0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c0ca070 con 0x7ff51c0b15b0 2024-04-25T10:18:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c0ca630 con 0x7ff51c0b15b0 2024-04-25T10:18:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c04ef30 con 0x7ff51c0b15b0 2024-04-25T10:18:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff50c0bc450 con 0x7ff51c0b15b0 2024-04-25T10:18:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff519600640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4f8066f60 0x7ff4f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff50c11f500 con 0x7ff51c0b15b0 2024-04-25T10:18:27.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff513e00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4f8066f60 0x7ff4f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.036+0000 7ff513e00640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4f8066f60 0x7ff4f8069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff51c06f280 tx=0x7ff504009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.040+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff50c0c3070 con 0x7ff51c0b15b0 2024-04-25T10:18:27.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.204+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff51c06b480 con 0x7ff51c0b15b0 2024-04-25T10:18:27.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.204+0000 7ff519600640 1 -- 172.21.15.45:0/1387300087 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7ff50c0ad020 con 0x7ff51c0b15b0 2024-04-25T10:18:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:27.209 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:27.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4f8066f60 msgr2=0x7ff4f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4f8066f60 0x7ff4f8069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff51c06f280 tx=0x7ff504009040 comp rx=0 tx=0).stop 2024-04-25T10:18:27.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 msgr2=0x7ff51c14d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 0x7ff51c14d470 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff50c019170 tx=0x7ff50c01ada0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 shutdown_connections 2024-04-25T10:18:27.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4f8066f60 0x7ff4f8069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff51c15e050 0x7ff51c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c0d1820 0x7ff51c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 --2- 172.21.15.45:0/1387300087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c0b15b0 0x7ff51c14d470 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 >> 172.21.15.45:0/1387300087 conn(0x7ff51c09a9d0 msgr2=0x7ff51c0af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 shutdown_connections 2024-04-25T10:18:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.208+0000 7ff523400640 1 -- 172.21.15.45:0/1387300087 wait complete. 2024-04-25T10:18:27.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:27.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:27.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 Processor -- start 2024-04-25T10:18:27.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 -- start start 2024-04-25T10:18:27.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 0x7f13f015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13f00b31c0 0x7f13f00b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f015b080 con 0x7f13f00aef80 2024-04-25T10:18:27.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f015adb0 con 0x7f13f00b31c0 2024-04-25T10:18:27.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f7c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015eb60 con 0x7f13f0070350 2024-04-25T10:18:27.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f5200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f13f00aef80 0x7f13f015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50094/0 (socket says 172.21.15.45:50094) 2024-04-25T10:18:27.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f5200640 1 -- 172.21.15.45:0/274911289 learned_addr learned my addr 172.21.15.45:0/274911289 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f5c00640 1 --2- 172.21.15.45:0/274911289 >> v2:172.21.15.117:3300/0 conn(0x7f13f00b31c0 0x7f13f00b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 0x7f13f015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80394619 0 0) 0x7f13f015b080 con 0x7f13f00aef80 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d8003530 con 0x7f13f00aef80 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186406963 0 0) 0x7f13d8003530 con 0x7f13f00aef80 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 msgr2=0x7f13f015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 0x7f13f015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 >> v2:172.21.15.117:3300/0 conn(0x7f13f00b31c0 msgr2=0x7f13f00b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 --2- 172.21.15.45:0/274911289 >> v2:172.21.15.117:3300/0 conn(0x7f13f00b31c0 0x7f13f00b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f015ee40 con 0x7f13f00aef80 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f13e4001680 con 0x7f13f00aef80 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 >> v1:172.21.15.45:6789/0 conn(0x7f13f00aef80 legacy=0x7f13f015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d80039c0 0x7f13d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d8004810 0x7f13d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 0x7f13d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015adb0 con 0x7f13d8004810 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015eb60 con 0x7f13d80039c0 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800b9d0 con 0x7f13f0070350 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d8004810 0x7f13d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f5200640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d80039c0 0x7f13d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13f5c00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 0x7f13d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 -- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 msgr2=0x7f13d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13f0070350 0x7f13d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 -- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d80039c0 msgr2=0x7f13d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d80039c0 0x7f13d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.372+0000 7f13efe00640 1 -- 172.21.15.45:0/274911289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8003530 con 0x7f13d8004810 2024-04-25T10:18:27.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13efe00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d8004810 0x7f13d8004c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009ae0 tx=0x7f13dc01c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a1decdffd19041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0b1070 con 0x7f13d8004810 2024-04-25T10:18:27.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc0a4ab0 con 0x7f13d8004810 2024-04-25T10:18:27.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13ef400640 1 -- 172.21.15.45:0/274911289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0096b0 con 0x7f13d8004810 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f7c00640 1 -- 172.21.15.45:0/274911289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d8004810 msgr2=0x7f13d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f7c00640 1 --2- 172.21.15.45:0/274911289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d8004810 0x7f13d8004c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009ae0 tx=0x7f13dc01c750 comp rx=0 tx=0).stop 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f5c00640 1 -- 172.21.15.45:0/274911289 reap_dead start 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f7c00640 1 -- 172.21.15.45:0/274911289 shutdown_connections 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f7c00640 1 -- 172.21.15.45:0/274911289 >> 172.21.15.45:0/274911289 conn(0x7f13f00669a0 msgr2=0x7f13f0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f7c00640 1 -- 172.21.15.45:0/274911289 shutdown_connections 2024-04-25T10:18:27.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.376+0000 7f13f7c00640 1 -- 172.21.15.45:0/274911289 wait complete. 2024-04-25T10:18:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 Processor -- start 2024-04-25T10:18:27.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 -- start start 2024-04-25T10:18:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d8004810 0x7f13f014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f00b1350 0x7f13f00b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 0x7f13f01570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015bc90 con 0x7f13f0156c40 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0071a00 con 0x7f13f00b1350 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015e480 con 0x7f13d8004810 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 0x7f13f01570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 0x7f13f01570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42118/0 (socket says 172.21.15.45:42118) 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 -- 172.21.15.45:0/1264194100 learned_addr learned my addr 172.21.15.45:0/1264194100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5200640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d8004810 0x7f13f014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13efe00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f00b1350 0x7f13f00b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 -- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d8004810 msgr2=0x7f13f014b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d8004810 0x7f13f014b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 -- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f00b1350 msgr2=0x7f13f00b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f00b1350 0x7f13f00b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 -- 172.21.15.45:0/1264194100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0157600 con 0x7f13f0156c40 2024-04-25T10:18:27.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f5c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 0x7f13f01570c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f13e8011bf0 tx=0x7f13e800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800fd10 con 0x7f13f0156c40 2024-04-25T10:18:27.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e801d070 con 0x7f13f0156c40 2024-04-25T10:18:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f0153ec0 con 0x7f13f0156c40 2024-04-25T10:18:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8015670 con 0x7f13f0156c40 2024-04-25T10:18:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.380+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f0154440 con 0x7f13f0156c40 2024-04-25T10:18:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.384+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f004ef30 con 0x7f13f0156c40 2024-04-25T10:18:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.384+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13e8015810 con 0x7f13f0156c40 2024-04-25T10:18:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.384+0000 7f13ecc00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13d0066f60 0x7f13d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.384+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13e8085930 con 0x7f13f0156c40 2024-04-25T10:18:27.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.384+0000 7f13f5200640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13d0066f60 0x7f13d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.384+0000 7f13f5200640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13d0066f60 0x7f13d0069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f13f015a1f0 tx=0x7f13e4007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.388+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13e8010040 con 0x7f13f0156c40 2024-04-25T10:18:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.548+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13f00b2fa0 con 0x7f13f0156c40 2024-04-25T10:18:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13ecc00640 1 -- 172.21.15.45:0/1264194100 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13e80568a0 con 0x7f13f0156c40 2024-04-25T10:18:27.554 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13d0066f60 msgr2=0x7f13d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13d0066f60 0x7f13d0069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f13f015a1f0 tx=0x7f13e4007980 comp rx=0 tx=0).stop 2024-04-25T10:18:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 msgr2=0x7f13f01570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 0x7f13f01570c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f13e8011bf0 tx=0x7f13e800f1b0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 shutdown_connections 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13d0066f60 0x7f13d0069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f0156c40 0x7f13f01570c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f00b1350 0x7f13f00b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 --2- 172.21.15.45:0/1264194100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d8004810 0x7f13f014b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 >> 172.21.15.45:0/1264194100 conn(0x7f13f00669a0 msgr2=0x7f13f00cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 shutdown_connections 2024-04-25T10:18:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.552+0000 7f13f7c00640 1 -- 172.21.15.45:0/1264194100 wait complete. 2024-04-25T10:18:27.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:27.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.716+0000 7fdd89400640 1 Processor -- start 2024-04-25T10:18:27.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.716+0000 7fdd89400640 1 -- start start 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd89400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd840b2e70 0x7fdd840b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd89400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd840cd7f0 0x7fdd8415d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd89400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd840cc080 con 0x7fdd8406aea0 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd89400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd840cc320 con 0x7fdd840b2e70 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd89400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cc600 con 0x7fdd840cd7f0 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd79e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd8406aea0 0x7fdd840cd0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50102/0 (socket says 172.21.15.45:50102) 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd79e00640 1 -- 172.21.15.45:0/3509258789 learned_addr learned my addr 172.21.15.45:0/3509258789 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd82800640 1 --2- 172.21.15.45:0/3509258789 >> v2:172.21.15.117:3300/0 conn(0x7fdd840b2e70 0x7fdd840b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd83200640 1 --2- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd840cd7f0 0x7fdd8415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd81e00640 1 -- 172.21.15.45:0/3509258789 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133026744 0 0) 0x7fdd840cc080 con 0x7fdd8406aea0 2024-04-25T10:18:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd81e00640 1 -- 172.21.15.45:0/3509258789 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd68003530 con 0x7fdd8406aea0 2024-04-25T10:18:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd82800640 1 -- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd840cd7f0 msgr2=0x7fdd8415d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd82800640 1 --2- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd840cd7f0 0x7fdd8415d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd82800640 1 -- 172.21.15.45:0/3509258789 >> v1:172.21.15.45:6789/0 conn(0x7fdd8406aea0 legacy=0x7fdd840cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.720+0000 7fdd82800640 1 -- 172.21.15.45:0/3509258789 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840cc8e0 con 0x7fdd840b2e70 2024-04-25T10:18:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd82800640 1 --2- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd840b47e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0097c0 tx=0x7fdd6c01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1b4727e94ccc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd81e00640 1 -- 172.21.15.45:0/3509258789 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0b5070 con 0x7fdd840b2e70 2024-04-25T10:18:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd81e00640 1 -- 172.21.15.45:0/3509258789 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0a8990 con 0x7fdd840b2e70 2024-04-25T10:18:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd81e00640 1 -- 172.21.15.45:0/3509258789 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a8d60 con 0x7fdd840b2e70 2024-04-25T10:18:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 -- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 msgr2=0x7fdd840b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 --2- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd840b47e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0097c0 tx=0x7fdd6c01ec90 comp rx=0 tx=0).stop 2024-04-25T10:18:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 -- 172.21.15.45:0/3509258789 shutdown_connections 2024-04-25T10:18:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 --2- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd840cd7f0 0x7fdd8415d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 --2- 172.21.15.45:0/3509258789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd840b47e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 -- 172.21.15.45:0/3509258789 >> 172.21.15.45:0/3509258789 conn(0x7fdd840b34a0 msgr2=0x7fdd840b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 -- 172.21.15.45:0/3509258789 shutdown_connections 2024-04-25T10:18:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 -- 172.21.15.45:0/3509258789 wait complete. 2024-04-25T10:18:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 Processor -- start 2024-04-25T10:18:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.724+0000 7fdd89400640 1 -- start start 2024-04-25T10:18:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd8406aea0 0x7fdd840c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd841f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 0x7fdd8406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cdc60 con 0x7fdd840cd7f0 2024-04-25T10:18:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840b3250 con 0x7fdd840b2e70 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cdde0 con 0x7fdd8406aea0 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 0x7fdd8406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 0x7fdd8406ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42130/0 (socket says 172.21.15.45:42130) 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 -- 172.21.15.45:0/3825962761 learned_addr learned my addr 172.21.15.45:0/3825962761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd79e00640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd841f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd82800640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd8406aea0 0x7fdd840c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 -- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd8406aea0 msgr2=0x7fdd840c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd8406aea0 0x7fdd840c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 -- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 msgr2=0x7fdd841f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd841f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 -- 172.21.15.45:0/3825962761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840c80e0 con 0x7fdd840cd7f0 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd79e00640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd841f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd83200640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 0x7fdd8406ec80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009d30 tx=0x7fdd7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c018020 con 0x7fdd840cd7f0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c014480 con 0x7fdd840cd7f0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c014870 con 0x7fdd840cd7f0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd841f7740 con 0x7fdd840cd7f0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd840c55f0 con 0x7fdd840cd7f0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8404ef30 con 0x7fdd840cd7f0 2024-04-25T10:18:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.728+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd7c01b020 con 0x7fdd840cd7f0 2024-04-25T10:18:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.732+0000 7fdd7b400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdd60066f60 0x7fdd60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.732+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd7c0859c0 con 0x7fdd840cd7f0 2024-04-25T10:18:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.732+0000 7fdd82800640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdd60066f60 0x7fdd60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.732+0000 7fdd82800640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdd60066f60 0x7fdd60069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0b2000 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.736+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd7c022070 con 0x7fdd840cd7f0 2024-04-25T10:18:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.896+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd8406e910 con 0x7fdd840cd7f0 2024-04-25T10:18:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.896+0000 7fdd7b400640 1 -- 172.21.15.45:0/3825962761 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd7c010040 con 0x7fdd840cd7f0 2024-04-25T10:18:27.901 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdd60066f60 msgr2=0x7fdd60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdd60066f60 0x7fdd60069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0b2000 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 msgr2=0x7fdd8406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 0x7fdd8406ec80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009d30 tx=0x7fdd7c00c490 comp rx=0 tx=0).stop 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 shutdown_connections 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdd60066f60 0x7fdd60069420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd840cd7f0 0x7fdd8406ec80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd840b2e70 0x7fdd841f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 --2- 172.21.15.45:0/3825962761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd8406aea0 0x7fdd840c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 >> 172.21.15.45:0/3825962761 conn(0x7fdd840b34a0 msgr2=0x7fdd840b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 shutdown_connections 2024-04-25T10:18:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:27.900+0000 7fdd89400640 1 -- 172.21.15.45:0/3825962761 wait complete. 2024-04-25T10:18:27.923 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 Processor -- start 2024-04-25T10:18:28.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 -- start start 2024-04-25T10:18:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280cc200 0x7f85280ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f852806cf40 0x7f852806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85280cdab0 con 0x7f85280d0d00 2024-04-25T10:18:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85280cd210 con 0x7f852806cf40 2024-04-25T10:18:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f852d400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85280cd4f0 con 0x7f85280cc200 2024-04-25T10:18:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f852806cf40 0x7f852806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f851de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280cc200 0x7f85280ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f852806cf40 0x7f852806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43290/0 (socket says 172.21.15.45:43290) 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8526800640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f85280d0d00 0x7f8528162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:51222/0 (socket says 172.21.15.45:51222) 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 -- 172.21.15.45:0/1149271371 learned_addr learned my addr 172.21.15.45:0/1149271371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 -- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280cc200 msgr2=0x7f85280ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 --2- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280cc200 0x7f85280ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 -- 172.21.15.45:0/1149271371 >> v1:172.21.15.117:6789/0 conn(0x7f85280d0d00 legacy=0x7f8528162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f8527200640 1 -- 172.21.15.45:0/1149271371 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85280cd7d0 con 0x7f852806cf40 2024-04-25T10:18:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.064+0000 7f851de00640 1 --2- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280cc200 0x7f85280ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f8527200640 1 --2- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f852806d3d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8520009e50 tx=0x7f852001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2054ffc0d9ffbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f8525e00640 1 -- 172.21.15.45:0/1149271371 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200a5660 con 0x7f852806cf40 2024-04-25T10:18:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f8525e00640 1 -- 172.21.15.45:0/1149271371 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85200a5800 con 0x7f852806cf40 2024-04-25T10:18:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f8525e00640 1 -- 172.21.15.45:0/1149271371 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200a5bd0 con 0x7f852806cf40 2024-04-25T10:18:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 -- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 msgr2=0x7f852806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 --2- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f852806d3d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8520009e50 tx=0x7f852001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 -- 172.21.15.45:0/1149271371 shutdown_connections 2024-04-25T10:18:28.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 --2- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f852806d3d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 --2- 172.21.15.45:0/1149271371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280cc200 0x7f85280ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 -- 172.21.15.45:0/1149271371 >> 172.21.15.45:0/1149271371 conn(0x7f8528072290 msgr2=0x7f85280726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 -- 172.21.15.45:0/1149271371 shutdown_connections 2024-04-25T10:18:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.068+0000 7f852d400640 1 -- 172.21.15.45:0/1149271371 wait complete. 2024-04-25T10:18:28.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 Processor -- start 2024-04-25T10:18:28.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- start start 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f85280c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 0x7f85281fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280d0d00 0x7f8528160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852815e040 con 0x7f85280cc200 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528162aa0 con 0x7f852806cf40 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852815e1c0 con 0x7f85280d0d00 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 0x7f85281fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 0x7f85281fadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42142/0 (socket says 172.21.15.45:42142) 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 -- 172.21.15.45:0/2185986627 learned_addr learned my addr 172.21.15.45:0/2185986627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f8526800640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f85280c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f8527200640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280d0d00 0x7f8528160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 -- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280d0d00 msgr2=0x7f8528160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280d0d00 0x7f8528160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 -- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 msgr2=0x7f85280c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f85280c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 -- 172.21.15.45:0/2185986627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85280c9420 con 0x7f85280cc200 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851de00640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 0x7f85281fadb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f851000ebe0 tx=0x7f851000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510017070 con 0x7f85280cc200 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851000cc30 con 0x7f85280cc200 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510013470 con 0x7f85280cc200 2024-04-25T10:18:28.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85280c9700 con 0x7f85280cc200 2024-04-25T10:18:28.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85280c9cc0 con 0x7f85280cc200 2024-04-25T10:18:28.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.072+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852804ef30 con 0x7f85280cc200 2024-04-25T10:18:28.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.076+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f851001a070 con 0x7f85280cc200 2024-04-25T10:18:28.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.076+0000 7f851f400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.076+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8510085720 con 0x7f85280cc200 2024-04-25T10:18:28.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.076+0000 7f8526800640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.076+0000 7f8526800640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8518000f70 tx=0x7f8518007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.080+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85100560c0 con 0x7f85280cc200 2024-04-25T10:18:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.240+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8528072ff0 con 0x7f85280cc200 2024-04-25T10:18:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.240+0000 7f851f400640 1 -- 172.21.15.45:0/2185986627 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f851005a160 con 0x7f85280cc200 2024-04-25T10:18:28.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 msgr2=0x7f8504069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8518000f70 tx=0x7f8518007e90 comp rx=0 tx=0).stop 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 msgr2=0x7f85281fadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 0x7f85281fadb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f851000ebe0 tx=0x7f851000c520 comp rx=0 tx=0).stop 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 shutdown_connections 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85280d0d00 0x7f8528160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cc200 0x7f85281fadb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 --2- 172.21.15.45:0/2185986627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f852806cf40 0x7f85280c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 >> 172.21.15.45:0/2185986627 conn(0x7f8528072290 msgr2=0x7f85281596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 shutdown_connections 2024-04-25T10:18:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.244+0000 7f852d400640 1 -- 172.21.15.45:0/2185986627 wait complete. 2024-04-25T10:18:28.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 Processor -- start 2024-04-25T10:18:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 -- start start 2024-04-25T10:18:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a900cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4a900cfe70 0x7f4a9015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a90071050 con 0x7f4a9015b740 2024-04-25T10:18:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9015d040 con 0x7f4a900cfe70 2024-04-25T10:18:28.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a96600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900712f0 con 0x7f4a901550b0 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a8f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a900cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a86a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4a900cfe70 0x7f4a9015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a8f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a900cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:34290/0 (socket says 172.21.15.45:34290) 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.404+0000 7f4a8f400640 1 -- 172.21.15.45:0/1195700308 learned_addr learned my addr 172.21.15.45:0/1195700308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8f400640 1 -- 172.21.15.45:0/1195700308 >> v2:172.21.15.117:3300/0 conn(0x7f4a900cfe70 msgr2=0x7f4a9015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8f400640 1 --2- 172.21.15.45:0/1195700308 >> v2:172.21.15.117:3300/0 conn(0x7f4a900cfe70 0x7f4a9015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8f400640 1 -- 172.21.15.45:0/1195700308 >> v1:172.21.15.117:6789/0 conn(0x7f4a9015b740 legacy=0x7f4a9006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8f400640 1 -- 172.21.15.45:0/1195700308 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a900715d0 con 0x7f4a901550b0 2024-04-25T10:18:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8f400640 1 --2- 172.21.15.45:0/1195700308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a900cf930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009870 tx=0x7f4a8001e770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64818517548cecfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8ea00640 1 -- 172.21.15.45:0/1195700308 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800ab650 con 0x7f4a901550b0 2024-04-25T10:18:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8ea00640 1 -- 172.21.15.45:0/1195700308 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a800ab7f0 con 0x7f4a901550b0 2024-04-25T10:18:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.408+0000 7f4a8ea00640 1 -- 172.21.15.45:0/1195700308 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800abbc0 con 0x7f4a901550b0 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- 172.21.15.45:0/1195700308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 msgr2=0x7f4a900cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 --2- 172.21.15.45:0/1195700308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a900cf930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009870 tx=0x7f4a8001e770 comp rx=0 tx=0).stop 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- 172.21.15.45:0/1195700308 shutdown_connections 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 --2- 172.21.15.45:0/1195700308 >> v2:172.21.15.117:3300/0 conn(0x7f4a900cfe70 0x7f4a9015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 --2- 172.21.15.45:0/1195700308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a900cf930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- 172.21.15.45:0/1195700308 >> 172.21.15.45:0/1195700308 conn(0x7f4a90066b70 msgr2=0x7f4a90066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- 172.21.15.45:0/1195700308 shutdown_connections 2024-04-25T10:18:28.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- 172.21.15.45:0/1195700308 wait complete. 2024-04-25T10:18:28.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 Processor -- start 2024-04-25T10:18:28.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- start start 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a900cfe70 0x7f4a900cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a9013e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 0x7f4a901540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9015bcd0 con 0x7f4a9015b740 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9015bb50 con 0x7f4a900cfe70 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a96600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9015c030 con 0x7f4a901550b0 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 0x7f4a901540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 0x7f4a901540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42148/0 (socket says 172.21.15.45:42148) 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 -- 172.21.15.45:0/2498343655 learned_addr learned my addr 172.21.15.45:0/2498343655 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a86a00640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a9013e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8f400640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a900cfe70 0x7f4a900cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 -- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 msgr2=0x7f4a9013e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a9013e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 -- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a900cfe70 msgr2=0x7f4a900cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a900cfe70 0x7f4a900cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 -- 172.21.15.45:0/2498343655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a900b42f0 con 0x7f4a9015b740 2024-04-25T10:18:28.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a8fe00640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 0x7f4a901540b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c730 tx=0x7f4a8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.412+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88003b80 con 0x7f4a9015b740 2024-04-25T10:18:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88003d20 con 0x7f4a9015b740 2024-04-25T10:18:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a901545f0 con 0x7f4a9015b740 2024-04-25T10:18:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800aa30 con 0x7f4a9015b740 2024-04-25T10:18:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9014f820 con 0x7f4a9015b740 2024-04-25T10:18:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9004ef30 con 0x7f4a9015b740 2024-04-25T10:18:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a8800abd0 con 0x7f4a9015b740 2024-04-25T10:18:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a87e00640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a6c066f60 0x7f4a6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a88084e50 con 0x7f4a9015b740 2024-04-25T10:18:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a8f400640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a6c066f60 0x7f4a6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.416+0000 7f4a8f400640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a6c066f60 0x7f4a6c069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4a800b0000 tx=0x7f4a800b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.420+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a88055770 con 0x7f4a9015b740 2024-04-25T10:18:28.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a900cef40 con 0x7f4a9015b740 2024-04-25T10:18:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a87e00640 1 -- 172.21.15.45:0/2498343655 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f4a88059810 con 0x7f4a9015b740 2024-04-25T10:18:28.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:28.587 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:28.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a6c066f60 msgr2=0x7f4a6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a6c066f60 0x7f4a6c069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4a800b0000 tx=0x7f4a800b5040 comp rx=0 tx=0).stop 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 msgr2=0x7f4a901540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 0x7f4a901540b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c730 tx=0x7f4a8800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 shutdown_connections 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a6c066f60 0x7f4a6c069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a9015b740 0x7f4a901540b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a901550b0 0x7f4a9013e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 --2- 172.21.15.45:0/2498343655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a900cfe70 0x7f4a900cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 >> 172.21.15.45:0/2498343655 conn(0x7f4a90066b70 msgr2=0x7f4a9015c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 shutdown_connections 2024-04-25T10:18:28.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.584+0000 7f4a96600640 1 -- 172.21.15.45:0/2498343655 wait complete. 2024-04-25T10:18:28.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:28.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:28.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 Processor -- start 2024-04-25T10:18:28.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 -- start start 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f41340aef80 0x7f413415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4134070350 0x7f413415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413415b080 con 0x7f413415bac0 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413415adb0 con 0x7f41340aef80 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4138a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413415eb60 con 0x7f4134070350 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f41340aef80 0x7f413415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f41340aef80 0x7f413415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42154/0 (socket says 172.21.15.45:42154) 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 -- 172.21.15.45:0/1245360692 learned_addr learned my addr 172.21.15.45:0/1245360692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 -- 172.21.15.45:0/1245360692 >> v2:172.21.15.117:3300/0 conn(0x7f4134070350 msgr2=0x7f413415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f412be00640 1 --2- 172.21.15.45:0/1245360692 >> v2:172.21.15.117:3300/0 conn(0x7f4134070350 0x7f413415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 --2- 172.21.15.45:0/1245360692 >> v2:172.21.15.117:3300/0 conn(0x7f4134070350 0x7f413415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 -- 172.21.15.45:0/1245360692 >> v1:172.21.15.117:6789/0 conn(0x7f413415bac0 legacy=0x7f41340b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f4131e00640 1 -- 172.21.15.45:0/1245360692 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413415ee40 con 0x7f41340aef80 2024-04-25T10:18:28.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.752+0000 7f412be00640 1 --2- 172.21.15.45:0/1245360692 >> v2:172.21.15.117:3300/0 conn(0x7f4134070350 0x7f413415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:28.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4131e00640 1 --2- 172.21.15.45:0/1245360692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f413415e480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f413406e390 tx=0x7f412401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef86d8093d8ad56e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4131400640 1 -- 172.21.15.45:0/1245360692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240aa620 con 0x7f41340aef80 2024-04-25T10:18:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4131400640 1 -- 172.21.15.45:0/1245360692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41240af070 con 0x7f41340aef80 2024-04-25T10:18:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4131400640 1 -- 172.21.15.45:0/1245360692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41240aa9e0 con 0x7f41340aef80 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 -- 172.21.15.45:0/1245360692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 msgr2=0x7f413415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 --2- 172.21.15.45:0/1245360692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f413415e480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f413406e390 tx=0x7f412401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 -- 172.21.15.45:0/1245360692 shutdown_connections 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 --2- 172.21.15.45:0/1245360692 >> v2:172.21.15.117:3300/0 conn(0x7f4134070350 0x7f413415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 --2- 172.21.15.45:0/1245360692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f413415e480 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 -- 172.21.15.45:0/1245360692 >> 172.21.15.45:0/1245360692 conn(0x7f41340669a0 msgr2=0x7f4134066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 -- 172.21.15.45:0/1245360692 shutdown_connections 2024-04-25T10:18:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 -- 172.21.15.45:0/1245360692 wait complete. 2024-04-25T10:18:28.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.756+0000 7f4138a00640 1 Processor -- start 2024-04-25T10:18:28.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- start start 2024-04-25T10:18:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4134070350 0x7f413414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f4134154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f413415bac0 0x7f4134151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41341679a0 con 0x7f41340aef80 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4134167820 con 0x7f4134070350 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4134167b20 con 0x7f413415bac0 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f4134154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f4134154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42162/0 (socket says 172.21.15.45:42162) 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 -- 172.21.15.45:0/894381620 learned_addr learned my addr 172.21.15.45:0/894381620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4131e00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4134070350 0x7f413414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4132800640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f413415bac0 0x7f4134151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 -- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f413415bac0 msgr2=0x7f4134151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f413415bac0 0x7f4134151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 -- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4134070350 msgr2=0x7f413414cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4134070350 0x7f413414cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 -- 172.21.15.45:0/894381620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4134155270 con 0x7f41340aef80 2024-04-25T10:18:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412be00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f4134154900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c7c0 tx=0x7f411c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c007a30 con 0x7f41340aef80 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c007bd0 con 0x7f41340aef80 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0113f0 con 0x7f41340aef80 2024-04-25T10:18:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4134155550 con 0x7f41340aef80 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4134155830 con 0x7f41340aef80 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.760+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413404ef30 con 0x7f41340aef80 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.764+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f411c011590 con 0x7f41340aef80 2024-04-25T10:18:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.764+0000 7f412a000640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4110066f90 0x7f4110069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.764+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f411c085220 con 0x7f41340aef80 2024-04-25T10:18:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.764+0000 7f4131e00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4110066f90 0x7f4110069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.764+0000 7f4131e00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4110066f90 0x7f4110069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41240ae000 tx=0x7f41240a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:28.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.768+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f411c015020 con 0x7f41340aef80 2024-04-25T10:18:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41340b2fa0 con 0x7f41340aef80 2024-04-25T10:18:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f412a000640 1 -- 172.21.15.45:0/894381620 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f411c055b40 con 0x7f41340aef80 2024-04-25T10:18:28.935 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4110066f90 msgr2=0x7f4110069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4110066f90 0x7f4110069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41240ae000 tx=0x7f41240a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 msgr2=0x7f4134154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f4134154900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c7c0 tx=0x7f411c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 shutdown_connections 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4110066f90 0x7f4110069450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f413415bac0 0x7f4134151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41340aef80 0x7f4134154900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 --2- 172.21.15.45:0/894381620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4134070350 0x7f413414cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 >> 172.21.15.45:0/894381620 conn(0x7f41340669a0 msgr2=0x7f41340b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 shutdown_connections 2024-04-25T10:18:28.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:28.932+0000 7f4138a00640 1 -- 172.21.15.45:0/894381620 wait complete. 2024-04-25T10:18:28.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 Processor -- start 2024-04-25T10:18:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 -- start start 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c6815e130 0x7f3c680cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c68071bc0 0x7f3c68072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c680ad840 con 0x7f3c680b3480 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c68072570 con 0x7f3c6815e130 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6f000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c680d0090 con 0x7f3c68071bc0 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c6815e130 0x7f3c680cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c6815e130 0x7f3c680cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42172/0 (socket says 172.21.15.45:42172) 2024-04-25T10:18:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 -- 172.21.15.45:0/2057477785 learned_addr learned my addr 172.21.15.45:0/2057477785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c6d000640 1 --2- 172.21.15.45:0/2057477785 >> v2:172.21.15.117:3300/0 conn(0x7f3c68071bc0 0x7f3c68072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 -- 172.21.15.45:0/2057477785 >> v2:172.21.15.117:3300/0 conn(0x7f3c68071bc0 msgr2=0x7f3c68072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 --2- 172.21.15.45:0/2057477785 >> v2:172.21.15.117:3300/0 conn(0x7f3c68071bc0 0x7f3c68072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 -- 172.21.15.45:0/2057477785 >> v1:172.21.15.117:6789/0 conn(0x7f3c680b3480 legacy=0x7f3c68070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.096+0000 7f3c5fe00640 1 -- 172.21.15.45:0/2057477785 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6815e710 con 0x7f3c6815e130 2024-04-25T10:18:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c5fe00640 1 --2- 172.21.15.45:0/2057477785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c6815e130 0x7f3c680cfb50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c681627c0 tx=0x7f3c5001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e48a7cbaee073c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c67400640 1 -- 172.21.15.45:0/2057477785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500b0070 con 0x7f3c6815e130 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c67400640 1 -- 172.21.15.45:0/2057477785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c500ab600 con 0x7f3c6815e130 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c67400640 1 -- 172.21.15.45:0/2057477785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500ab9d0 con 0x7f3c6815e130 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 -- 172.21.15.45:0/2057477785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c6815e130 msgr2=0x7f3c680cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 --2- 172.21.15.45:0/2057477785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c6815e130 0x7f3c680cfb50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c681627c0 tx=0x7f3c5001c520 comp rx=0 tx=0).stop 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 -- 172.21.15.45:0/2057477785 shutdown_connections 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 --2- 172.21.15.45:0/2057477785 >> v2:172.21.15.117:3300/0 conn(0x7f3c68071bc0 0x7f3c68072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 --2- 172.21.15.45:0/2057477785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c6815e130 0x7f3c680cfb50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 -- 172.21.15.45:0/2057477785 >> 172.21.15.45:0/2057477785 conn(0x7f3c680669a0 msgr2=0x7f3c68066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 -- 172.21.15.45:0/2057477785 shutdown_connections 2024-04-25T10:18:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.100+0000 7f3c6f000640 1 -- 172.21.15.45:0/2057477785 wait complete. 2024-04-25T10:18:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 Processor -- start 2024-04-25T10:18:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- start start 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 0x7f3c68159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c680b3480 0x7f3c68155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6815e130 0x7f3c68156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680ae530 con 0x7f3c68071bc0 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6815bb90 con 0x7f3c680b3480 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680ae6b0 con 0x7f3c6815e130 2024-04-25T10:18:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 0x7f3c68159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 0x7f3c68159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42174/0 (socket says 172.21.15.45:42174) 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 -- 172.21.15.45:0/188967998 learned_addr learned my addr 172.21.15.45:0/188967998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c5fe00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c680b3480 0x7f3c68155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6d000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6815e130 0x7f3c68156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 -- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6815e130 msgr2=0x7f3c68156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6815e130 0x7f3c68156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 -- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c680b3480 msgr2=0x7f3c68155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c680b3480 0x7f3c68155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 -- 172.21.15.45:0/188967998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6815a640 con 0x7f3c68071bc0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c67e00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 0x7f3c68159bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c450 tx=0x7f3c5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580052d0 con 0x7f3c68071bc0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58005490 con 0x7f3c68071bc0 2024-04-25T10:18:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58012850 con 0x7f3c68071bc0 2024-04-25T10:18:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6815a920 con 0x7f3c68071bc0 2024-04-25T10:18:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c68152cc0 con 0x7f3c68071bc0 2024-04-25T10:18:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.104+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6804ef30 con 0x7f3c68071bc0 2024-04-25T10:18:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.108+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c580129f0 con 0x7f3c68071bc0 2024-04-25T10:18:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.108+0000 7f3c64c00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c44066f60 0x7f3c44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.108+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c58084b90 con 0x7f3c68071bc0 2024-04-25T10:18:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.108+0000 7f3c5fe00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c44066f60 0x7f3c44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.108+0000 7f3c5fe00640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c44066f60 0x7f3c44069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c500ad000 tx=0x7f3c500a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.112+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c58055b90 con 0x7f3c68071bc0 2024-04-25T10:18:29.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.272+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c680cf0a0 con 0x7f3c68071bc0 2024-04-25T10:18:29.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c64c00640 1 -- 172.21.15.45:0/188967998 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c58055490 con 0x7f3c68071bc0 2024-04-25T10:18:29.278 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c44066f60 msgr2=0x7f3c44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c44066f60 0x7f3c44069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c500ad000 tx=0x7f3c500a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 msgr2=0x7f3c68159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 0x7f3c68159bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c450 tx=0x7f3c5800c910 comp rx=0 tx=0).stop 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 shutdown_connections 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c44066f60 0x7f3c44069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6815e130 0x7f3c68156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c680b3480 0x7f3c68155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 --2- 172.21.15.45:0/188967998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c68071bc0 0x7f3c68159bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 >> 172.21.15.45:0/188967998 conn(0x7f3c680669a0 msgr2=0x7f3c680b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 shutdown_connections 2024-04-25T10:18:29.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.276+0000 7f3c6f000640 1 -- 172.21.15.45:0/188967998 wait complete. 2024-04-25T10:18:29.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:29.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f8400640 1 Processor -- start 2024-04-25T10:18:29.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f8400640 1 -- start start 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f8400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06f00aeda0 0x7f06f0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f8400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f0159920 con 0x7f06f0159540 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f8400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f015c2f0 con 0x7f06f0072020 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f8400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f015c5d0 con 0x7f06f00aeda0 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f5000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06f00aeda0 0x7f06f0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f5a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06f0159540 0x7f06f00ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50114/0 (socket says 172.21.15.45:50114) 2024-04-25T10:18:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f5a00640 1 -- 172.21.15.45:0/2917908014 learned_addr learned my addr 172.21.15.45:0/2917908014 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f5000640 1 -- 172.21.15.45:0/2917908014 >> v1:172.21.15.117:6789/0 conn(0x7f06f0072020 legacy=0x7f06f0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f5000640 1 -- 172.21.15.45:0/2917908014 >> v1:172.21.15.45:6789/0 conn(0x7f06f0159540 legacy=0x7f06f00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.440+0000 7f06f5000640 1 -- 172.21.15.45:0/2917908014 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f015c8b0 con 0x7f06f00aeda0 2024-04-25T10:18:29.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f5000640 1 --2- 172.21.15.45:0/2917908014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f00aeda0 0x7f06f0071ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097c0 tx=0x7f06dc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8f9359efea1120 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06efe00640 1 -- 172.21.15.45:0/2917908014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0ab610 con 0x7f06f00aeda0 2024-04-25T10:18:29.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06efe00640 1 -- 172.21.15.45:0/2917908014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc0b0070 con 0x7f06f00aeda0 2024-04-25T10:18:29.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06efe00640 1 -- 172.21.15.45:0/2917908014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0ab9d0 con 0x7f06f00aeda0 2024-04-25T10:18:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 -- 172.21.15.45:0/2917908014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f00aeda0 msgr2=0x7f06f0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 --2- 172.21.15.45:0/2917908014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f00aeda0 0x7f06f0071ae0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097c0 tx=0x7f06dc01c430 comp rx=0 tx=0).stop 2024-04-25T10:18:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 -- 172.21.15.45:0/2917908014 shutdown_connections 2024-04-25T10:18:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 --2- 172.21.15.45:0/2917908014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f00aeda0 0x7f06f0071ae0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 -- 172.21.15.45:0/2917908014 >> 172.21.15.45:0/2917908014 conn(0x7f06f00669a0 msgr2=0x7f06f0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 -- 172.21.15.45:0/2917908014 shutdown_connections 2024-04-25T10:18:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.444+0000 7f06f8400640 1 -- 172.21.15.45:0/2917908014 wait complete. 2024-04-25T10:18:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 Processor -- start 2024-04-25T10:18:29.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- start start 2024-04-25T10:18:29.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06f0072020 0x7f06f013c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06f00aeda0 0x7f06f013c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 0x7f06f00af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f0169940 con 0x7f06f0159540 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f006c0d0 con 0x7f06f0072020 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f0169ac0 con 0x7f06f00aeda0 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 0x7f06f00af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 0x7f06f00af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42188/0 (socket says 172.21.15.45:42188) 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 -- 172.21.15.45:0/1594036277 learned_addr learned my addr 172.21.15.45:0/1594036277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f5a00640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06f0072020 0x7f06f013c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f5000640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06f00aeda0 0x7f06f013c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 -- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06f00aeda0 msgr2=0x7f06f013c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06f00aeda0 0x7f06f013c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 -- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06f0072020 msgr2=0x7f06f013c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06f0072020 0x7f06f013c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 -- 172.21.15.45:0/1594036277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f00cbef0 con 0x7f06f0159540 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f6400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 0x7f06f00af2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e800c780 tx=0x7f06e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004b70 con 0x7f06f0159540 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8004d10 con 0x7f06f0159540 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e800aa30 con 0x7f06f0159540 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f00afda0 con 0x7f06f0159540 2024-04-25T10:18:29.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f00b0360 con 0x7f06f0159540 2024-04-25T10:18:29.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.448+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f004ef30 con 0x7f06f0159540 2024-04-25T10:18:29.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.452+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06e800abd0 con 0x7f06f0159540 2024-04-25T10:18:29.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.452+0000 7f06ed600640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06d0066f60 0x7f06d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.452+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06e80850b0 con 0x7f06f0159540 2024-04-25T10:18:29.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.452+0000 7f06f5a00640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06d0066f60 0x7f06d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.452+0000 7f06f5a00640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06d0066f60 0x7f06d0069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06e4000f70 tx=0x7f06e4007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.456+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06e8020070 con 0x7f06f0159540 2024-04-25T10:18:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06f013ce50 con 0x7f06f0159540 2024-04-25T10:18:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06ed600640 1 -- 172.21.15.45:0/1594036277 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06e8055a60 con 0x7f06f0159540 2024-04-25T10:18:29.618 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:29.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06d0066f60 msgr2=0x7f06d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06d0066f60 0x7f06d0069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06e4000f70 tx=0x7f06e4007e90 comp rx=0 tx=0).stop 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 msgr2=0x7f06f00af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 0x7f06f00af2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06e800c780 tx=0x7f06e800cc40 comp rx=0 tx=0).stop 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 shutdown_connections 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06d0066f60 0x7f06d0069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06f0159540 0x7f06f00af2a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06f00aeda0 0x7f06f013c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 --2- 172.21.15.45:0/1594036277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06f0072020 0x7f06f013c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 >> 172.21.15.45:0/1594036277 conn(0x7f06f00669a0 msgr2=0x7f06f0154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 shutdown_connections 2024-04-25T10:18:29.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.616+0000 7f06f8400640 1 -- 172.21.15.45:0/1594036277 wait complete. 2024-04-25T10:18:29.640 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.776+0000 7f9105a00640 1 Processor -- start 2024-04-25T10:18:29.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.776+0000 7f9105a00640 1 -- start start 2024-04-25T10:18:29.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f9105a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 0x7f910015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f9105a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f91000b5560 0x7f9100157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f9105a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000b5c20 con 0x7f9100066c00 2024-04-25T10:18:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f9105a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000b59f0 con 0x7f91000b5560 2024-04-25T10:18:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f9105a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000af6e0 con 0x7f9100066fe0 2024-04-25T10:18:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9100066c00 0x7f910006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50130/0 (socket says 172.21.15.45:50130) 2024-04-25T10:18:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fea00640 1 -- 172.21.15.45:0/1311754048 learned_addr learned my addr 172.21.15.45:0/1311754048 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809888738 0 0) 0x7f91000b5c20 con 0x7f9100066c00 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90f7e00640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 0x7f910015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 --2- 172.21.15.45:0/1311754048 >> v2:172.21.15.117:3300/0 conn(0x7f91000b5560 0x7f9100157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e4003530 con 0x7f9100066c00 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4022814831 0 0) 0x7f90e4003530 con 0x7f9100066c00 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 msgr2=0x7f910015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 0x7f910015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 >> v2:172.21.15.117:3300/0 conn(0x7f91000b5560 msgr2=0x7f9100157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 --2- 172.21.15.45:0/1311754048 >> v2:172.21.15.117:3300/0 conn(0x7f91000b5560 0x7f9100157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000af9c0 con 0x7f9100066c00 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f90f0001680 con 0x7f9100066c00 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 >> v1:172.21.15.45:6789/0 conn(0x7f9100066c00 legacy=0x7f910006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e40039c0 0x7f90e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 0x7f90e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e4004d50 0x7f90e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000af6e0 con 0x7f90e4004d50 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e400b9d0 con 0x7f90e40039c0 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e400bb50 con 0x7f9100066fe0 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e4004d50 0x7f90e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90f7e00640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 0x7f90e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90fea00640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e40039c0 0x7f90e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 -- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 msgr2=0x7f90e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9100066fe0 0x7f90e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 -- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e40039c0 msgr2=0x7f90e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e40039c0 0x7f90e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.780+0000 7f90ff400640 1 -- 172.21.15.45:0/1311754048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e4003530 con 0x7f90e4004d50 2024-04-25T10:18:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f90ff400640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e4004d50 0x7f90e40051d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e170 tx=0x7f90f801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2763d2037754f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80b2070 con 0x7f90e4004d50 2024-04-25T10:18:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f80a5cb0 con 0x7f90e4004d50 2024-04-25T10:18:29.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f90fe000640 1 -- 172.21.15.45:0/1311754048 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8009c10 con 0x7f90e4004d50 2024-04-25T10:18:29.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f9105a00640 1 -- 172.21.15.45:0/1311754048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e4004d50 msgr2=0x7f90e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f9105a00640 1 --2- 172.21.15.45:0/1311754048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90e4004d50 0x7f90e40051d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e170 tx=0x7f90f801cca0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f90ff400640 1 -- 172.21.15.45:0/1311754048 reap_dead start 2024-04-25T10:18:29.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f9105a00640 1 -- 172.21.15.45:0/1311754048 shutdown_connections 2024-04-25T10:18:29.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f9105a00640 1 -- 172.21.15.45:0/1311754048 >> 172.21.15.45:0/1311754048 conn(0x7f91000b24f0 msgr2=0x7f91000b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f9105a00640 1 -- 172.21.15.45:0/1311754048 shutdown_connections 2024-04-25T10:18:29.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.784+0000 7f9105a00640 1 -- 172.21.15.45:0/1311754048 wait complete. 2024-04-25T10:18:29.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 Processor -- start 2024-04-25T10:18:29.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- start start 2024-04-25T10:18:29.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e4004d50 0x7f91000cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 0x7f91000c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f910014f0a0 0x7f910014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910015e110 con 0x7f91000c7380 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000aff80 con 0x7f90e4004d50 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910015e290 con 0x7f910014f0a0 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 0x7f91000c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 0x7f91000c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42196/0 (socket says 172.21.15.45:42196) 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 -- 172.21.15.45:0/2747128045 learned_addr learned my addr 172.21.15.45:0/2747128045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90ff400640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f910014f0a0 0x7f910014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90fea00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e4004d50 0x7f91000cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 -- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f910014f0a0 msgr2=0x7f910014f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f910014f0a0 0x7f910014f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 -- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e4004d50 msgr2=0x7f91000cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e4004d50 0x7f91000cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 -- 172.21.15.45:0/2747128045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000c80b0 con 0x7f91000c7380 2024-04-25T10:18:29.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f7e00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 0x7f91000c77e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90e80119d0 tx=0x7f90e8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800fb50 con 0x7f91000c7380 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e800fcf0 con 0x7f91000c7380 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800fea0 con 0x7f91000c7380 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910014d1b0 con 0x7f91000c7380 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91000cd620 con 0x7f91000c7380 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.788+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910004ef30 con 0x7f91000c7380 2024-04-25T10:18:29.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.792+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90e8012040 con 0x7f91000c7380 2024-04-25T10:18:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.792+0000 7f90f6a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f90dc066f90 0x7f90dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.792+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90e8085710 con 0x7f91000c7380 2024-04-25T10:18:29.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.792+0000 7f90fea00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f90dc066f90 0x7f90dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.792+0000 7f90fea00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f90dc066f90 0x7f90dc069450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c10 tx=0x7f90f0007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:29.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.796+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90e8019030 con 0x7f91000c7380 2024-04-25T10:18:29.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9100071c70 con 0x7f91000c7380 2024-04-25T10:18:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f90f6a00640 1 -- 172.21.15.45:0/2747128045 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f90e8056060 con 0x7f91000c7380 2024-04-25T10:18:29.967 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:29.967 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:29.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f90dc066f90 msgr2=0x7f90dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f90dc066f90 0x7f90dc069450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c10 tx=0x7f90f0007900 comp rx=0 tx=0).stop 2024-04-25T10:18:29.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 msgr2=0x7f91000c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:29.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 0x7f91000c77e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90e80119d0 tx=0x7f90e8011e90 comp rx=0 tx=0).stop 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 shutdown_connections 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f90dc066f90 0x7f90dc069450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f910014f0a0 0x7f910014f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91000c7380 0x7f91000c77e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 --2- 172.21.15.45:0/2747128045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90e4004d50 0x7f91000cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 >> 172.21.15.45:0/2747128045 conn(0x7f91000b24f0 msgr2=0x7f910006d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 shutdown_connections 2024-04-25T10:18:29.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:29.964+0000 7f9105a00640 1 -- 172.21.15.45:0/2747128045 wait complete. 2024-04-25T10:18:29.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:29.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa99b800640 1 Processor -- start 2024-04-25T10:18:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa99b800640 1 -- start start 2024-04-25T10:18:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa99b800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa9940b38d0 0x7fa99415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa99b800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99415b080 con 0x7fa994070350 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa99b800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99415adb0 con 0x7fa9940aef80 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa99b800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99415eb60 con 0x7fa9940b38d0 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa98be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa994070350 0x7fa9940b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50132/0 (socket says 172.21.15.45:50132) 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa98be00640 1 -- 172.21.15.45:0/76450828 learned_addr learned my addr 172.21.15.45:0/76450828 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa999800640 1 --2- 172.21.15.45:0/76450828 >> v2:172.21.15.117:3300/0 conn(0x7fa9940b38d0 0x7fa99415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964359338 0 0) 0x7fa99415b080 con 0x7fa994070350 2024-04-25T10:18:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa978003530 con 0x7fa994070350 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3344573092 0 0) 0x7fa978003530 con 0x7fa994070350 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 >> v2:172.21.15.117:3300/0 conn(0x7fa9940b38d0 msgr2=0x7fa99415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 --2- 172.21.15.45:0/76450828 >> v2:172.21.15.117:3300/0 conn(0x7fa9940b38d0 0x7fa99415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 >> v1:172.21.15.117:6789/0 conn(0x7fa9940aef80 legacy=0x7fa99415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99415ee40 con 0x7fa994070350 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fa97c001680 con 0x7fa994070350 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 >> v1:172.21.15.45:6789/0 conn(0x7fa994070350 legacy=0x7fa9940b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9780039c0 0x7fa978003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978004810 0x7fa978004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa97800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415eb60 con 0x7fa9780051d0 2024-04-25T10:18:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97800c070 con 0x7fa978004810 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.132+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97800c270 con 0x7fa9780039c0 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa97800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa98be00640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978004810 0x7fa978004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa998e00640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9780039c0 0x7fa978003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 -- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9780039c0 msgr2=0x7fa978003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9780039c0 0x7fa978003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 -- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978004810 msgr2=0x7fa978004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa978004810 0x7fa978004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 -- 172.21.15.45:0/76450828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa978003530 con 0x7fa9780051d0 2024-04-25T10:18:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 -- 172.21.15.45:0/76450828 reap_dead start 2024-04-25T10:18:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa999800640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa97800b9d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009f90 tx=0x7fa98c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d045a26fd666077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c009c10 con 0x7fa9780051d0 2024-04-25T10:18:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c0a5c90 con 0x7fa9780051d0 2024-04-25T10:18:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.136+0000 7fa993e00640 1 -- 172.21.15.45:0/76450828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c003be0 con 0x7fa9780051d0 2024-04-25T10:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- 172.21.15.45:0/76450828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 msgr2=0x7fa97800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa97800b9d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009f90 tx=0x7fa98c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- 172.21.15.45:0/76450828 shutdown_connections 2024-04-25T10:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 --2- 172.21.15.45:0/76450828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa97800b9d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- 172.21.15.45:0/76450828 >> 172.21.15.45:0/76450828 conn(0x7fa9940669a0 msgr2=0x7fa994066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- 172.21.15.45:0/76450828 shutdown_connections 2024-04-25T10:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- 172.21.15.45:0/76450828 wait complete. 2024-04-25T10:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 Processor -- start 2024-04-25T10:18:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- start start 2024-04-25T10:18:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa9941440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9940aac60 0x7fa9940ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99414ba20 0x7fa9940ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415e480 con 0x7fa9780051d0 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa994071a00 con 0x7fa99414ba20 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa99b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415e600 con 0x7fa9940aac60 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa9941440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa9941440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42208/0 (socket says 172.21.15.45:42208) 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 -- 172.21.15.45:0/4272172059 learned_addr learned my addr 172.21.15.45:0/4272172059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa98be00640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9940aac60 0x7fa9940ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa999800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99414ba20 0x7fa9940ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 -- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9940aac60 msgr2=0x7fa9940ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9940aac60 0x7fa9940ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 -- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99414ba20 msgr2=0x7fa9940ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99414ba20 0x7fa9940ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 -- 172.21.15.45:0/4272172059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940a9ee0 con 0x7fa9780051d0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa998e00640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa9941440f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa98400f620 tx=0x7fa98400fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9840052e0 con 0x7fa9780051d0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.140+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984018070 con 0x7fa9780051d0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984003820 con 0x7fa9780051d0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9940aa180 con 0x7fa9780051d0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940aa740 con 0x7fa9780051d0 2024-04-25T10:18:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99404ef30 con 0x7fa9780051d0 2024-04-25T10:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9840039c0 con 0x7fa9780051d0 2024-04-25T10:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa991600640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa970066f90 0x7fa970069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa984084b70 con 0x7fa9780051d0 2024-04-25T10:18:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.144+0000 7fa98be00640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa970066f90 0x7fa970069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.148+0000 7fa98be00640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa970066f90 0x7fa970069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa97c001060 tx=0x7fa97c0011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.148+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa984055b70 con 0x7fa9780051d0 2024-04-25T10:18:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.308+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9940723b0 con 0x7fa9780051d0 2024-04-25T10:18:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa991600640 1 -- 172.21.15.45:0/4272172059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa984005480 con 0x7fa9780051d0 2024-04-25T10:18:30.314 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa970066f90 msgr2=0x7fa970069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa970066f90 0x7fa970069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa97c001060 tx=0x7fa97c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 msgr2=0x7fa9941440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa9941440f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa98400f620 tx=0x7fa98400fa20 comp rx=0 tx=0).stop 2024-04-25T10:18:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 shutdown_connections 2024-04-25T10:18:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa970066f90 0x7fa970069450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa99414ba20 0x7fa9940ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9940aac60 0x7fa9940ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 --2- 172.21.15.45:0/4272172059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780051d0 0x7fa9941440f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 >> 172.21.15.45:0/4272172059 conn(0x7fa9940669a0 msgr2=0x7fa9940cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 shutdown_connections 2024-04-25T10:18:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.312+0000 7fa99b800640 1 -- 172.21.15.45:0/4272172059 wait complete. 2024-04-25T10:18:30.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 Processor -- start 2024-04-25T10:18:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 -- start start 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c0d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd63c15bef0 0x7fd63c0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0d0960 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0cc470 con 0x7fd63c15bef0 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd644600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0d0680 con 0x7fd63c15bb10 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd63c15bef0 0x7fd63c0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd63c15bef0 0x7fd63c0cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42222/0 (socket says 172.21.15.45:42222) 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 -- 172.21.15.45:0/2629233392 learned_addr learned my addr 172.21.15.45:0/2629233392 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641c00640 1 --2- 172.21.15.45:0/2629233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c0d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 -- 172.21.15.45:0/2629233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 msgr2=0x7fd63c0d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 --2- 172.21.15.45:0/2629233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c0d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 -- 172.21.15.45:0/2629233392 >> v1:172.21.15.117:6789/0 conn(0x7fd63c0cd1f0 legacy=0x7fd63c15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.480+0000 7fd641200640 1 -- 172.21.15.45:0/2629233392 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0cc710 con 0x7fd63c15bef0 2024-04-25T10:18:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd641200640 1 --2- 172.21.15.45:0/2629233392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c15bef0 0x7fd63c0cff50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd62801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0cf0e76704d843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd63be00640 1 -- 172.21.15.45:0/2629233392 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280ab650 con 0x7fd63c15bef0 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd63be00640 1 -- 172.21.15.45:0/2629233392 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6280b0070 con 0x7fd63c15bef0 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd63be00640 1 -- 172.21.15.45:0/2629233392 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280aba10 con 0x7fd63c15bef0 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 -- 172.21.15.45:0/2629233392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c15bef0 msgr2=0x7fd63c0cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 --2- 172.21.15.45:0/2629233392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c15bef0 0x7fd63c0cff50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd628009870 tx=0x7fd62801c560 comp rx=0 tx=0).stop 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 -- 172.21.15.45:0/2629233392 shutdown_connections 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 --2- 172.21.15.45:0/2629233392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c15bef0 0x7fd63c0cff50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 --2- 172.21.15.45:0/2629233392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c0d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 -- 172.21.15.45:0/2629233392 >> 172.21.15.45:0/2629233392 conn(0x7fd63c0d19a0 msgr2=0x7fd63c0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 -- 172.21.15.45:0/2629233392 shutdown_connections 2024-04-25T10:18:30.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.484+0000 7fd644600640 1 -- 172.21.15.45:0/2629233392 wait complete. 2024-04-25T10:18:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 Processor -- start 2024-04-25T10:18:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- start start 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 0x7fd63c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd63c15bef0 0x7fd63c1f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1600a0 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15ff20 con 0x7fd63c15bef0 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0cd600 con 0x7fd63c15bb10 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 0x7fd63c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 0x7fd63c0c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42230/0 (socket says 172.21.15.45:42230) 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 -- 172.21.15.45:0/3373833308 learned_addr learned my addr 172.21.15.45:0/3373833308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd642600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd63c15bef0 0x7fd63c1f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641200640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 -- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 msgr2=0x7fd63c156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 -- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd63c15bef0 msgr2=0x7fd63c1f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd63c15bef0 0x7fd63c1f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 -- 172.21.15.45:0/3373833308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0ca6a0 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd641c00640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 0x7fd63c0c94f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e930 tx=0x7fd63000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630017070 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63000ca00 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c0ca980 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63000cdb0 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c0caf40 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.488+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c04ef30 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.492+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd63001a070 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.492+0000 7fd639600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd61c066f60 0x7fd61c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.492+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd630085440 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.492+0000 7fd641200640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd61c066f60 0x7fd61c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.492+0000 7fd641200640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd61c066f60 0x7fd61c069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6280af000 tx=0x7fd6280a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.496+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd630056500 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd63c0c5030 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd639600640 1 -- 172.21.15.45:0/3373833308 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd630055e00 con 0x7fd63c0cd1f0 2024-04-25T10:18:30.663 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:30.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd61c066f60 msgr2=0x7fd61c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd61c066f60 0x7fd61c069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd6280af000 tx=0x7fd6280a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 msgr2=0x7fd63c0c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 0x7fd63c0c94f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e930 tx=0x7fd63000edf0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 shutdown_connections 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd61c066f60 0x7fd61c069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd63c15bef0 0x7fd63c1f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd63c15bb10 0x7fd63c156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 --2- 172.21.15.45:0/3373833308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd63c0cd1f0 0x7fd63c0c94f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 >> 172.21.15.45:0/3373833308 conn(0x7fd63c0d19a0 msgr2=0x7fd63c165f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 shutdown_connections 2024-04-25T10:18:30.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.660+0000 7fd644600640 1 -- 172.21.15.45:0/3373833308 wait complete. 2024-04-25T10:18:30.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc93200640 1 Processor -- start 2024-04-25T10:18:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc93200640 1 -- start start 2024-04-25T10:18:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc93200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc9415f420 0x7efc94166900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc93200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940cfde0 con 0x7efc940b3dd0 2024-04-25T10:18:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc93200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940d0080 con 0x7efc940ce290 2024-04-25T10:18:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc93200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc940d0360 con 0x7efc9415f420 2024-04-25T10:18:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc89400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efc940b3dd0 0x7efc9415ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50144/0 (socket says 172.21.15.45:50144) 2024-04-25T10:18:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.824+0000 7efc89400640 1 -- 172.21.15.45:0/180866017 learned_addr learned my addr 172.21.15.45:0/180866017 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc92800640 1 --2- 172.21.15.45:0/180866017 >> v2:172.21.15.117:3300/0 conn(0x7efc9415f420 0x7efc94166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc91400640 1 -- 172.21.15.45:0/180866017 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255414756 0 0) 0x7efc940cfde0 con 0x7efc940b3dd0 2024-04-25T10:18:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc91400640 1 -- 172.21.15.45:0/180866017 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc78003530 con 0x7efc940b3dd0 2024-04-25T10:18:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc92800640 1 -- 172.21.15.45:0/180866017 >> v1:172.21.15.117:6789/0 conn(0x7efc940ce290 legacy=0x7efc940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc92800640 1 -- 172.21.15.45:0/180866017 >> v1:172.21.15.45:6789/0 conn(0x7efc940b3dd0 legacy=0x7efc9415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc92800640 1 -- 172.21.15.45:0/180866017 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940d0640 con 0x7efc9415f420 2024-04-25T10:18:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc92800640 1 --2- 172.21.15.45:0/180866017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc9415f420 0x7efc94166900 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0097f0 tx=0x7efc8c01cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdbeb0916d93212 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc91400640 1 -- 172.21.15.45:0/180866017 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0aa620 con 0x7efc9415f420 2024-04-25T10:18:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc91400640 1 -- 172.21.15.45:0/180866017 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c0af070 con 0x7efc9415f420 2024-04-25T10:18:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.828+0000 7efc91400640 1 -- 172.21.15.45:0/180866017 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0aa9c0 con 0x7efc9415f420 2024-04-25T10:18:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- 172.21.15.45:0/180866017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc9415f420 msgr2=0x7efc94166900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 --2- 172.21.15.45:0/180866017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc9415f420 0x7efc94166900 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0097f0 tx=0x7efc8c01cf00 comp rx=0 tx=0).stop 2024-04-25T10:18:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- 172.21.15.45:0/180866017 shutdown_connections 2024-04-25T10:18:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 --2- 172.21.15.45:0/180866017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc9415f420 0x7efc94166900 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- 172.21.15.45:0/180866017 >> 172.21.15.45:0/180866017 conn(0x7efc940b32b0 msgr2=0x7efc940b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- 172.21.15.45:0/180866017 shutdown_connections 2024-04-25T10:18:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- 172.21.15.45:0/180866017 wait complete. 2024-04-25T10:18:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 Processor -- start 2024-04-25T10:18:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- start start 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc940b3dd0 0x7efc9414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc940ce290 0x7efc94153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 0x7efc940c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc941697e0 con 0x7efc9415f420 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc94169660 con 0x7efc940ce290 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc93200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc94169960 con 0x7efc940b3dd0 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 0x7efc940c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 0x7efc940c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42240/0 (socket says 172.21.15.45:42240) 2024-04-25T10:18:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 -- 172.21.15.45:0/1901728977 learned_addr learned my addr 172.21.15.45:0/1901728977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 -- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc940b3dd0 msgr2=0x7efc9414fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc89400640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc940ce290 0x7efc94153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc91e00640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc940b3dd0 0x7efc9414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc940b3dd0 0x7efc9414fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 -- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc940ce290 msgr2=0x7efc94153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc940ce290 0x7efc94153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc92800640 1 -- 172.21.15.45:0/1901728977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94156880 con 0x7efc9415f420 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc89400640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc940ce290 0x7efc94153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.832+0000 7efc91e00640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc940b3dd0 0x7efc9414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc92800640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 0x7efc940c6800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0097f0 tx=0x7efc8c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0aac00 con 0x7efc9415f420 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c0aada0 con 0x7efc9415f420 2024-04-25T10:18:30.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc94150210 con 0x7efc9415f420 2024-04-25T10:18:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c004250 con 0x7efc9415f420 2024-04-25T10:18:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc941562c0 con 0x7efc9415f420 2024-04-25T10:18:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc94075120 con 0x7efc9415f420 2024-04-25T10:18:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc8c003620 con 0x7efc9415f420 2024-04-25T10:18:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc8aa00640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc70066f90 0x7efc70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc8c11f640 con 0x7efc9415f420 2024-04-25T10:18:30.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.836+0000 7efc91e00640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc70066f90 0x7efc70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:30.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.840+0000 7efc91e00640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc70066f90 0x7efc70069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc84007fd0 tx=0x7efc84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:30.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:30.840+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc94156880 con 0x7efc9415f420 2024-04-25T10:18:31.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc940b3970 con 0x7efc9415f420 2024-04-25T10:18:31.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc8aa00640 1 -- 172.21.15.45:0/1901728977 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc8c016090 con 0x7efc9415f420 2024-04-25T10:18:31.011 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:31.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc70066f90 msgr2=0x7efc70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc70066f90 0x7efc70069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc84007fd0 tx=0x7efc84009040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 msgr2=0x7efc940c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 0x7efc940c6800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0097f0 tx=0x7efc8c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 shutdown_connections 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc70066f90 0x7efc70069450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc9415f420 0x7efc940c6800 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc940ce290 0x7efc94153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 --2- 172.21.15.45:0/1901728977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc940b3dd0 0x7efc9414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 >> 172.21.15.45:0/1901728977 conn(0x7efc940b32b0 msgr2=0x7efc940d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 shutdown_connections 2024-04-25T10:18:31.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.008+0000 7efc93200640 1 -- 172.21.15.45:0/1901728977 wait complete. 2024-04-25T10:18:31.042 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T10:18:31.042 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:18:31.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 Processor -- start 2024-04-25T10:18:31.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 -- start start 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff89415de60 0x7ff8940ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff89415e240 0x7ff89406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940d0950 con 0x7ff8940b2af0 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8940d0c30 con 0x7ff89415de60 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff89a200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8940d0f10 con 0x7ff89415e240 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff893400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff89415de60 0x7ff8940ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff893400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff89415de60 0x7ff8940ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42246/0 (socket says 172.21.15.45:42246) 2024-04-25T10:18:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff88aa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff8940b2af0 0x7ff8940cc340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50152/0 (socket says 172.21.15.45:50152) 2024-04-25T10:18:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff893400640 1 -- 172.21.15.45:0/1634651959 learned_addr learned my addr 172.21.15.45:0/1634651959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:31.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.180+0000 7ff893e00640 1 --2- 172.21.15.45:0/1634651959 >> v2:172.21.15.117:3300/0 conn(0x7ff89415e240 0x7ff89406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff893400640 1 -- 172.21.15.45:0/1634651959 >> v2:172.21.15.117:3300/0 conn(0x7ff89415e240 msgr2=0x7ff89406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff893400640 1 --2- 172.21.15.45:0/1634651959 >> v2:172.21.15.117:3300/0 conn(0x7ff89415e240 0x7ff89406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff893400640 1 -- 172.21.15.45:0/1634651959 >> v1:172.21.15.45:6789/0 conn(0x7ff8940b2af0 legacy=0x7ff8940cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff893400640 1 -- 172.21.15.45:0/1634651959 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940d11f0 con 0x7ff89415de60 2024-04-25T10:18:31.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff892a00640 1 -- 172.21.15.45:0/1634651959 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148930610 0 0) 0x7ff8940d0950 con 0x7ff8940b2af0 2024-04-25T10:18:31.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff893400640 1 --2- 172.21.15.45:0/1634651959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff89415de60 0x7ff8940ce190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff884009870 tx=0x7ff88401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574340e823a91cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff892a00640 1 -- 172.21.15.45:0/1634651959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b0070 con 0x7ff89415de60 2024-04-25T10:18:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff892a00640 1 -- 172.21.15.45:0/1634651959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840ab600 con 0x7ff89415de60 2024-04-25T10:18:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff892a00640 1 -- 172.21.15.45:0/1634651959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ab9d0 con 0x7ff89415de60 2024-04-25T10:18:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff89a200640 1 -- 172.21.15.45:0/1634651959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff89415de60 msgr2=0x7ff8940ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff89a200640 1 --2- 172.21.15.45:0/1634651959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff89415de60 0x7ff8940ce190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff884009870 tx=0x7ff88401c490 comp rx=0 tx=0).stop 2024-04-25T10:18:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff89a200640 1 -- 172.21.15.45:0/1634651959 shutdown_connections 2024-04-25T10:18:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff89a200640 1 --2- 172.21.15.45:0/1634651959 >> v2:172.21.15.117:3300/0 conn(0x7ff89415e240 0x7ff89406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff89a200640 1 --2- 172.21.15.45:0/1634651959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff89415de60 0x7ff8940ce190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.184+0000 7ff89a200640 1 -- 172.21.15.45:0/1634651959 >> 172.21.15.45:0/1634651959 conn(0x7ff8940d1910 msgr2=0x7ff8940d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- 172.21.15.45:0/1634651959 shutdown_connections 2024-04-25T10:18:31.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- 172.21.15.45:0/1634651959 wait complete. 2024-04-25T10:18:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 Processor -- start 2024-04-25T10:18:31.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- start start 2024-04-25T10:18:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 0x7ff89406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff89415de60 0x7ff894156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff89415e240 0x7ff89406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940cd630 con 0x7ff8940b2af0 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940cd4b0 con 0x7ff89415e240 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89416ac50 con 0x7ff89415de60 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 0x7ff89406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 0x7ff89406fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42254/0 (socket says 172.21.15.45:42254) 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 -- 172.21.15.45:0/3486021627 learned_addr learned my addr 172.21.15.45:0/3486021627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 -- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff89415de60 msgr2=0x7ff894156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff89415de60 0x7ff894156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 -- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff89415e240 msgr2=0x7ff89406ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff89415e240 0x7ff89406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 -- 172.21.15.45:0/3486021627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894155c80 con 0x7ff8940b2af0 2024-04-25T10:18:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff893400640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 0x7ff89406fe40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8840ae040 tx=0x7ff884019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b0040 con 0x7ff8940b2af0 2024-04-25T10:18:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840b9070 con 0x7ff8940b2af0 2024-04-25T10:18:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894155f60 con 0x7ff8940b2af0 2024-04-25T10:18:31.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840aa970 con 0x7ff8940b2af0 2024-04-25T10:18:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.188+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff894151d10 con 0x7ff8940b2af0 2024-04-25T10:18:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.192+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89404ef30 con 0x7ff8940b2af0 2024-04-25T10:18:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.192+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8840aab10 con 0x7ff8940b2af0 2024-04-25T10:18:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.192+0000 7ff88be00640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.192+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff88411eec0 con 0x7ff8940b2af0 2024-04-25T10:18:31.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.192+0000 7ff88aa00640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.192+0000 7ff88aa00640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8940c87e0 tx=0x7ff87c007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.196+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff884016110 con 0x7ff8940b2af0 2024-04-25T10:18:31.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8940d15e0 con 0x7ff8940b2af0 2024-04-25T10:18:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff88be00640 1 -- 172.21.15.45:0/3486021627 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7ff8840efec0 con 0x7ff8940b2af0 2024-04-25T10:18:31.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:31.363 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff870066f60 msgr2=0x7ff870069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff870066f60 0x7ff870069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8940c87e0 tx=0x7ff87c007970 comp rx=0 tx=0).stop 2024-04-25T10:18:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 msgr2=0x7ff89406fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 0x7ff89406fe40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8840ae040 tx=0x7ff884019230 comp rx=0 tx=0).stop 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 shutdown_connections 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff870066f60 0x7ff870069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff89415e240 0x7ff89406ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff89415de60 0x7ff894156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 --2- 172.21.15.45:0/3486021627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8940b2af0 0x7ff89406fe40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 >> 172.21.15.45:0/3486021627 conn(0x7ff8940d1910 msgr2=0x7ff8940b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 shutdown_connections 2024-04-25T10:18:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.360+0000 7ff89a200640 1 -- 172.21.15.45:0/3486021627 wait complete. 2024-04-25T10:18:31.385 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:31.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:31.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.524+0000 7f40f4e00640 1 Processor -- start 2024-04-25T10:18:31.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.524+0000 7f40f4e00640 1 -- start start 2024-04-25T10:18:31.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40f4e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40f0159540 0x7f40f00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40f4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40f00aebd0 0x7f40f0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40f4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f0072020 0x7f40f015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40f4e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f0159920 con 0x7f40f00aebd0 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40f4e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f015c2f0 con 0x7f40f0159540 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40f4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f015c5d0 con 0x7f40f0072020 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40f00aebd0 0x7f40f0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40f00aebd0 0x7f40f0071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42268/0 (socket says 172.21.15.45:42268) 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 -- 172.21.15.45:0/1012461366 learned_addr learned my addr 172.21.15.45:0/1012461366 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40ee800640 1 --2- 172.21.15.45:0/1012461366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f0072020 0x7f40f015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40ede00640 1 --2- 172.21.15.45:0/1012461366 >> v2:172.21.15.117:3300/0 conn(0x7f40f0159540 0x7f40f00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 -- 172.21.15.45:0/1012461366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f0072020 msgr2=0x7f40f015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 --2- 172.21.15.45:0/1012461366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f0072020 0x7f40f015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 -- 172.21.15.45:0/1012461366 >> v2:172.21.15.117:3300/0 conn(0x7f40f0159540 msgr2=0x7f40f00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 --2- 172.21.15.45:0/1012461366 >> v2:172.21.15.117:3300/0 conn(0x7f40f0159540 0x7f40f00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 -- 172.21.15.45:0/1012461366 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f015c8b0 con 0x7f40f00aebd0 2024-04-25T10:18:31.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.528+0000 7f40e7e00640 1 --2- 172.21.15.45:0/1012461366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f00aebd0 0x7f40f0071ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40f00b4c20 tx=0x7f40d801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f295755a9a4506a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40ed400640 1 -- 172.21.15.45:0/1012461366 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80b0070 con 0x7f40f00aebd0 2024-04-25T10:18:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40ed400640 1 -- 172.21.15.45:0/1012461366 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d80ab600 con 0x7f40f00aebd0 2024-04-25T10:18:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40ed400640 1 -- 172.21.15.45:0/1012461366 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80ab9d0 con 0x7f40f00aebd0 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- 172.21.15.45:0/1012461366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f00aebd0 msgr2=0x7f40f0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- 172.21.15.45:0/1012461366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f00aebd0 0x7f40f0071ae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40f00b4c20 tx=0x7f40d801c260 comp rx=0 tx=0).stop 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- 172.21.15.45:0/1012461366 shutdown_connections 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- 172.21.15.45:0/1012461366 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f0072020 0x7f40f015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- 172.21.15.45:0/1012461366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f00aebd0 0x7f40f0071ae0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- 172.21.15.45:0/1012461366 >> v2:172.21.15.117:3300/0 conn(0x7f40f0159540 0x7f40f00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- 172.21.15.45:0/1012461366 >> 172.21.15.45:0/1012461366 conn(0x7f40f00669a0 msgr2=0x7f40f0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- 172.21.15.45:0/1012461366 shutdown_connections 2024-04-25T10:18:31.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- 172.21.15.45:0/1012461366 wait complete. 2024-04-25T10:18:31.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 Processor -- start 2024-04-25T10:18:31.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- start start 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f0072020 0x7f40f00ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f00aebd0 0x7f40f00b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 0x7f40f00cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0070350 con 0x7f40f0159540 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0072430 con 0x7f40f0072020 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40f4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00704d0 con 0x7f40f00aebd0 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 0x7f40f00cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 0x7f40f00cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42276/0 (socket says 172.21.15.45:42276) 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.532+0000 7f40ee800640 1 -- 172.21.15.45:0/3959067089 learned_addr learned my addr 172.21.15.45:0/3959067089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e7e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f00aebd0 0x7f40f00b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ede00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f0072020 0x7f40f00ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ee800640 1 -- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f00aebd0 msgr2=0x7f40f00b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ee800640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f00aebd0 0x7f40f00b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ee800640 1 -- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f0072020 msgr2=0x7f40f00ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ee800640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f0072020 0x7f40f00ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ee800640 1 -- 172.21.15.45:0/3959067089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00b0400 con 0x7f40f0159540 2024-04-25T10:18:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40ee800640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 0x7f40f00cf8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40e800efc0 tx=0x7f40e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8018020 con 0x7f40f0159540 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e8014480 con 0x7f40f0159540 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8014870 con 0x7f40f0159540 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f00b06a0 con 0x7f40f0159540 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f00b0c60 con 0x7f40f0159540 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f004ef30 con 0x7f40f0159540 2024-04-25T10:18:31.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40e801b020 con 0x7f40f0159540 2024-04-25T10:18:31.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e6000640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f40cc066f60 0x7f40cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.536+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40e8085a50 con 0x7f40f0159540 2024-04-25T10:18:31.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.540+0000 7f40ede00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f40cc066f60 0x7f40cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.540+0000 7f40ede00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f40cc066f60 0x7f40cc069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40e00099d0 tx=0x7f40e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.540+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40e8022070 con 0x7f40f0159540 2024-04-25T10:18:31.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40f00cf470 con 0x7f40f0159540 2024-04-25T10:18:31.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40e6000640 1 -- 172.21.15.45:0/3959067089 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40e8010040 con 0x7f40f0159540 2024-04-25T10:18:31.706 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f40cc066f60 msgr2=0x7f40cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f40cc066f60 0x7f40cc069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40e00099d0 tx=0x7f40e0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 msgr2=0x7f40f00cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 0x7f40f00cf8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40e800efc0 tx=0x7f40e800c490 comp rx=0 tx=0).stop 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 shutdown_connections 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f40cc066f60 0x7f40cc069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40f0159540 0x7f40f00cf8f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40f00aebd0 0x7f40f00b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 --2- 172.21.15.45:0/3959067089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f0072020 0x7f40f00ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 >> 172.21.15.45:0/3959067089 conn(0x7f40f00669a0 msgr2=0x7f40f015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 shutdown_connections 2024-04-25T10:18:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.704+0000 7f40f4e00640 1 -- 172.21.15.45:0/3959067089 wait complete. 2024-04-25T10:18:31.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:31.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 Processor -- start 2024-04-25T10:18:31.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 -- start start 2024-04-25T10:18:31.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c159540 0x7ff62c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff62c0721f0 0x7ff62c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62c159920 con 0x7ff62c0aebd0 2024-04-25T10:18:31.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c0af030 con 0x7ff62c0721f0 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff631e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c15c2f0 con 0x7ff62c159540 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff62b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff62c0721f0 0x7ff62c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff62b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff62c0721f0 0x7ff62c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42286/0 (socket says 172.21.15.45:42286) 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff62b400640 1 -- 172.21.15.45:0/3861428298 learned_addr learned my addr 172.21.15.45:0/3861428298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff623e00640 1 --1- 172.21.15.45:0/3861428298 >> v1:172.21.15.45:6789/0 conn(0x7ff62c0aebd0 0x7ff62c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50168/0 (socket says 172.21.15.45:50168) 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.868+0000 7ff62aa00640 1 --2- 172.21.15.45:0/3861428298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c159540 0x7ff62c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62b400640 1 -- 172.21.15.45:0/3861428298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c159540 msgr2=0x7ff62c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62b400640 1 --2- 172.21.15.45:0/3861428298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c159540 0x7ff62c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62b400640 1 -- 172.21.15.45:0/3861428298 >> v1:172.21.15.45:6789/0 conn(0x7ff62c0aebd0 legacy=0x7ff62c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62b400640 1 -- 172.21.15.45:0/3861428298 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c15c5d0 con 0x7ff62c0721f0 2024-04-25T10:18:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62b400640 1 --2- 172.21.15.45:0/3861428298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0721f0 0x7ff62c070350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff624009e50 tx=0x7ff62401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19269e90084de113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62a000640 1 -- 172.21.15.45:0/3861428298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240a58d0 con 0x7ff62c0721f0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62a000640 1 -- 172.21.15.45:0/3861428298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240a5a70 con 0x7ff62c0721f0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff62a000640 1 -- 172.21.15.45:0/3861428298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240a5e40 con 0x7ff62c0721f0 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 -- 172.21.15.45:0/3861428298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0721f0 msgr2=0x7ff62c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 --2- 172.21.15.45:0/3861428298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0721f0 0x7ff62c070350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff624009e50 tx=0x7ff62401ca00 comp rx=0 tx=0).stop 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 -- 172.21.15.45:0/3861428298 shutdown_connections 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 --2- 172.21.15.45:0/3861428298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0721f0 0x7ff62c070350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 --2- 172.21.15.45:0/3861428298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c159540 0x7ff62c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 -- 172.21.15.45:0/3861428298 >> 172.21.15.45:0/3861428298 conn(0x7ff62c0669a0 msgr2=0x7ff62c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 -- 172.21.15.45:0/3861428298 shutdown_connections 2024-04-25T10:18:31.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.872+0000 7ff631e00640 1 -- 172.21.15.45:0/3861428298 wait complete. 2024-04-25T10:18:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 Processor -- start 2024-04-25T10:18:31.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 -- start start 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c0721f0 0x7ff62c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 0x7ff62c141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff62c159540 0x7ff62c142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c0710a0 con 0x7ff62c0aebd0 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c06c0d0 con 0x7ff62c159540 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c071220 con 0x7ff62c0721f0 2024-04-25T10:18:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 0x7ff62c141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 0x7ff62c141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42288/0 (socket says 172.21.15.45:42288) 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 -- 172.21.15.45:0/474352277 learned_addr learned my addr 172.21.15.45:0/474352277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff62aa00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c0721f0 0x7ff62c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff62b400640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff62c159540 0x7ff62c142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 -- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c0721f0 msgr2=0x7ff62c0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c0721f0 0x7ff62c0b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 -- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff62c159540 msgr2=0x7ff62c142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff62c159540 0x7ff62c142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 -- 172.21.15.45:0/474352277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c153ee0 con 0x7ff62c0aebd0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 0x7ff62c141d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c450 tx=0x7ff61400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614018070 con 0x7ff62c0aebd0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614007480 con 0x7ff62c0aebd0 2024-04-25T10:18:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140047f0 con 0x7ff62c0aebd0 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c154150 con 0x7ff62c0aebd0 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.876+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c152da0 con 0x7ff62c0aebd0 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.880+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c04ef30 con 0x7ff62c0aebd0 2024-04-25T10:18:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.880+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff61401b070 con 0x7ff62c0aebd0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.880+0000 7ff623400640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff608066f60 0x7ff608069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.880+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff614084250 con 0x7ff62c0aebd0 2024-04-25T10:18:31.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.880+0000 7ff62aa00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff608066f60 0x7ff608069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:31.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.880+0000 7ff62aa00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff608066f60 0x7ff608069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0cba50 tx=0x7ff61c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:31.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:31.884+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff614054b70 con 0x7ff62c0aebd0 2024-04-25T10:18:32.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff62c0cfc30 con 0x7ff62c0aebd0 2024-04-25T10:18:32.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff623400640 1 -- 172.21.15.45:0/474352277 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff614058c10 con 0x7ff62c0aebd0 2024-04-25T10:18:32.051 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:32.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff608066f60 msgr2=0x7ff608069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff608066f60 0x7ff608069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0cba50 tx=0x7ff61c009190 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 msgr2=0x7ff62c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 0x7ff62c141d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c450 tx=0x7ff61400c910 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 shutdown_connections 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff608066f60 0x7ff608069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff62c159540 0x7ff62c142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff62c0aebd0 0x7ff62c141d20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 --2- 172.21.15.45:0/474352277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff62c0721f0 0x7ff62c0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 >> 172.21.15.45:0/474352277 conn(0x7ff62c0669a0 msgr2=0x7ff62c0ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 shutdown_connections 2024-04-25T10:18:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.048+0000 7ff631e00640 1 -- 172.21.15.45:0/474352277 wait complete. 2024-04-25T10:18:32.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:32.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.212+0000 7f9e6de00640 1 Processor -- start 2024-04-25T10:18:32.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.212+0000 7f9e6de00640 1 -- start start 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e6de00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e680aef80 0x7f9e6815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e68070350 0x7f9e6815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e6de00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6815b080 con 0x7f9e6815bac0 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e6de00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6815adb0 con 0x7f9e680aef80 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e6de00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6815eb60 con 0x7f9e68070350 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e680aef80 0x7f9e6815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e68070350 0x7f9e6815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e680aef80 0x7f9e6815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43488/0 (socket says 172.21.15.45:43488) 2024-04-25T10:18:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 -- 172.21.15.45:0/3068108141 learned_addr learned my addr 172.21.15.45:0/3068108141 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 -- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e68070350 msgr2=0x7f9e6815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 --2- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e68070350 0x7f9e6815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 -- 172.21.15.45:0/3068108141 >> v1:172.21.15.117:6789/0 conn(0x7f9e6815bac0 legacy=0x7f9e680b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.216+0000 7f9e66a00640 1 -- 172.21.15.45:0/3068108141 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6815ee40 con 0x7f9e680aef80 2024-04-25T10:18:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e66a00640 1 --2- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e680aef80 0x7f9e6815e480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009870 tx=0x7f9e5801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3772c385c177db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e66000640 1 -- 172.21.15.45:0/3068108141 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580a9400 con 0x7f9e680aef80 2024-04-25T10:18:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e66000640 1 -- 172.21.15.45:0/3068108141 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580a95a0 con 0x7f9e680aef80 2024-04-25T10:18:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e66000640 1 -- 172.21.15.45:0/3068108141 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580a9970 con 0x7f9e680aef80 2024-04-25T10:18:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 -- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e680aef80 msgr2=0x7f9e6815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 --2- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e680aef80 0x7f9e6815e480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009870 tx=0x7f9e5801c560 comp rx=0 tx=0).stop 2024-04-25T10:18:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 -- 172.21.15.45:0/3068108141 shutdown_connections 2024-04-25T10:18:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 --2- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e68070350 0x7f9e6815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 --2- 172.21.15.45:0/3068108141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e680aef80 0x7f9e6815e480 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 -- 172.21.15.45:0/3068108141 >> 172.21.15.45:0/3068108141 conn(0x7f9e680669a0 msgr2=0x7f9e68066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 -- 172.21.15.45:0/3068108141 shutdown_connections 2024-04-25T10:18:32.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 -- 172.21.15.45:0/3068108141 wait complete. 2024-04-25T10:18:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 Processor -- start 2024-04-25T10:18:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.220+0000 7f9e6de00640 1 -- start start 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e68070350 0x7f9e6814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e680aef80 0x7f9e6813f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 0x7f9e68152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e681679a0 con 0x7f9e6815bac0 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68167820 con 0x7f9e68070350 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68167b20 con 0x7f9e680aef80 2024-04-25T10:18:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 0x7f9e68152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 0x7f9e68152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42290/0 (socket says 172.21.15.45:42290) 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 -- 172.21.15.45:0/258282547 learned_addr learned my addr 172.21.15.45:0/258282547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e66a00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e68070350 0x7f9e6814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e5fe00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e680aef80 0x7f9e6813f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 -- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e680aef80 msgr2=0x7f9e6813f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e680aef80 0x7f9e6813f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 -- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e68070350 msgr2=0x7f9e6814de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e68070350 0x7f9e6814de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 -- 172.21.15.45:0/258282547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6814e340 con 0x7f9e6815bac0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e66a00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e68070350 0x7f9e6814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e67400640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 0x7f9e68152fd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c730 tx=0x7f9e6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60003b80 con 0x7f9e6815bac0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e60003d20 con 0x7f9e6815bac0 2024-04-25T10:18:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000aa30 con 0x7f9e6815bac0 2024-04-25T10:18:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e68151c50 con 0x7f9e6815bac0 2024-04-25T10:18:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68152210 con 0x7f9e6815bac0 2024-04-25T10:18:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6804ef30 con 0x7f9e6815bac0 2024-04-25T10:18:32.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.224+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e6000abd0 con 0x7f9e6815bac0 2024-04-25T10:18:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.228+0000 7f9e5f400640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.228+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e60084ff0 con 0x7f9e6815bac0 2024-04-25T10:18:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.228+0000 7f9e66a00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.228+0000 7f9e66a00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e580ae000 tx=0x7f9e580b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.232+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e60055f10 con 0x7f9e6815bac0 2024-04-25T10:18:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e680b3620 con 0x7f9e6815bac0 2024-04-25T10:18:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e5f400640 1 -- 172.21.15.45:0/258282547 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e60055810 con 0x7f9e6815bac0 2024-04-25T10:18:32.395 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 msgr2=0x7f9e44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e580ae000 tx=0x7f9e580b3040 comp rx=0 tx=0).stop 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 msgr2=0x7f9e68152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 0x7f9e68152fd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c730 tx=0x7f9e6000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 shutdown_connections 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6815bac0 0x7f9e68152fd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e680aef80 0x7f9e6813f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 --2- 172.21.15.45:0/258282547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e68070350 0x7f9e6814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 >> 172.21.15.45:0/258282547 conn(0x7f9e680669a0 msgr2=0x7f9e68157dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 shutdown_connections 2024-04-25T10:18:32.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.392+0000 7f9e6de00640 1 -- 172.21.15.45:0/258282547 wait complete. 2024-04-25T10:18:32.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:18:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 Processor -- start 2024-04-25T10:18:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 -- start start 2024-04-25T10:18:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d140cf930 0x7f0d140cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d140ae020 0x7f0d140ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140d0250 con 0x7f0d14070c20 2024-04-25T10:18:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1415bb90 con 0x7f0d140cf930 2024-04-25T10:18:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d18e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140ad840 con 0x7f0d140ae020 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d11e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d140cf930 0x7f0d140cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.552+0000 7f0d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d140ae020 0x7f0d140ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d140cf930 0x7f0d140cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43508/0 (socket says 172.21.15.45:43508) 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d09400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0d14070c20 0x7f0d1415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:51294/0 (socket says 172.21.15.45:51294) 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 -- 172.21.15.45:0/1807654808 learned_addr learned my addr 172.21.15.45:0/1807654808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 -- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d140ae020 msgr2=0x7f0d140ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 --2- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d140ae020 0x7f0d140ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 -- 172.21.15.45:0/1807654808 >> v1:172.21.15.117:6789/0 conn(0x7f0d14070c20 legacy=0x7f0d1415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 -- 172.21.15.45:0/1807654808 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1415e9f0 con 0x7f0d140cf930 2024-04-25T10:18:32.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d12800640 1 --2- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d140ae020 0x7f0d140ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11e00640 1 --2- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140cf930 0x7f0d140cfd10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009e50 tx=0x7f0d0401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f59dd736c650d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.556+0000 7f0d11400640 1 -- 172.21.15.45:0/1807654808 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040a58d0 con 0x7f0d140cf930 2024-04-25T10:18:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d11400640 1 -- 172.21.15.45:0/1807654808 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d040a5a70 con 0x7f0d140cf930 2024-04-25T10:18:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d11400640 1 -- 172.21.15.45:0/1807654808 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040a5e40 con 0x7f0d140cf930 2024-04-25T10:18:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140cf930 msgr2=0x7f0d140cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 --2- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140cf930 0x7f0d140cfd10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009e50 tx=0x7f0d0401ca00 comp rx=0 tx=0).stop 2024-04-25T10:18:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- 172.21.15.45:0/1807654808 shutdown_connections 2024-04-25T10:18:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 --2- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d140ae020 0x7f0d140ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 --2- 172.21.15.45:0/1807654808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140cf930 0x7f0d140cfd10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- 172.21.15.45:0/1807654808 >> 172.21.15.45:0/1807654808 conn(0x7f0d14066b70 msgr2=0x7f0d14066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- 172.21.15.45:0/1807654808 shutdown_connections 2024-04-25T10:18:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- 172.21.15.45:0/1807654808 wait complete. 2024-04-25T10:18:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 Processor -- start 2024-04-25T10:18:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- start start 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d14070c20 0x7f0d14152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140ae020 0x7f0d14157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 0x7f0d14155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1416a8e0 con 0x7f0d140cf930 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1406e130 con 0x7f0d140ae020 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d18e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1416aa60 con 0x7f0d14070c20 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d12800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 0x7f0d14155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d12800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 0x7f0d14155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42294/0 (socket says 172.21.15.45:42294) 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.560+0000 7f0d12800640 1 -- 172.21.15.45:0/2897868047 learned_addr learned my addr 172.21.15.45:0/2897868047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d11e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d14070c20 0x7f0d14152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d09400640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140ae020 0x7f0d14157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d12800640 1 -- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d14070c20 msgr2=0x7f0d14152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d12800640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d14070c20 0x7f0d14152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d12800640 1 -- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140ae020 msgr2=0x7f0d14157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d12800640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140ae020 0x7f0d14157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d12800640 1 -- 172.21.15.45:0/2897868047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d14153df0 con 0x7f0d140cf930 2024-04-25T10:18:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d09400640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140ae020 0x7f0d14157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d12800640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 0x7f0d14155700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e950 tx=0x7f0d0c00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c017070 con 0x7f0d140cf930 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00ca00 con 0x7f0d140cf930 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d141540d0 con 0x7f0d140cf930 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00cdd0 con 0x7f0d140cf930 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14157400 con 0x7f0d140cf930 2024-04-25T10:18:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1404ef30 con 0x7f0d140cf930 2024-04-25T10:18:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d0c013470 con 0x7f0d140cf930 2024-04-25T10:18:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d0aa00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0cf0066f60 0x7f0cf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.564+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d0c084b00 con 0x7f0d140cf930 2024-04-25T10:18:32.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.568+0000 7f0d11e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0cf0066f60 0x7f0cf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.568+0000 7f0d11e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0cf0066f60 0x7f0cf0069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d040ae000 tx=0x7f0d040a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:32.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.568+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d0c0554d0 con 0x7f0d140cf930 2024-04-25T10:18:32.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.788+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f0d14158e00 con 0x7f0d140cf930 2024-04-25T10:18:32.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d0aa00640 1 -- 172.21.15.45:0/2897868047 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v53) v1 ==== 151+0+0 (secure 0 0 0) 0x7f0d0c059570 con 0x7f0d140cf930 2024-04-25T10:18:32.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0cf0066f60 msgr2=0x7f0cf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0cf0066f60 0x7f0cf0069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d040ae000 tx=0x7f0d040a7070 comp rx=0 tx=0).stop 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 msgr2=0x7f0d14155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 0x7f0d14155700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e950 tx=0x7f0d0c00ee10 comp rx=0 tx=0).stop 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 shutdown_connections 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0cf0066f60 0x7f0cf0069420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d140cf930 0x7f0d14155700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d140ae020 0x7f0d14157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 --2- 172.21.15.45:0/2897868047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d14070c20 0x7f0d14152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 >> 172.21.15.45:0/2897868047 conn(0x7f0d14066b70 msgr2=0x7f0d14161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 shutdown_connections 2024-04-25T10:18:32.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:32.852+0000 7f0d18e00640 1 -- 172.21.15.45:0/2897868047 wait complete. 2024-04-25T10:18:32.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:18:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.016+0000 7fdfdf600640 1 Processor -- start 2024-04-25T10:18:33.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.016+0000 7fdfdf600640 1 -- start start 2024-04-25T10:18:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.016+0000 7fdfdf600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfd806bd10 0x7fdfd8161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.016+0000 7fdfdf600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd8161e50 0x7fdfd815d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfdf600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd806ac90 con 0x7fdfd80cd9a0 2024-04-25T10:18:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfdf600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd806b880 con 0x7fdfd806bd10 2024-04-25T10:18:33.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfdf600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd806af50 con 0x7fdfd8161e50 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfd806bd10 0x7fdfd8161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdfd806bd10 0x7fdfd8161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42300/0 (socket says 172.21.15.45:42300) 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 -- 172.21.15.45:0/1862233323 learned_addr learned my addr 172.21.15.45:0/1862233323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfdd600640 1 --2- 172.21.15.45:0/1862233323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd8161e50 0x7fdfd815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 -- 172.21.15.45:0/1862233323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd8161e50 msgr2=0x7fdfd815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 --2- 172.21.15.45:0/1862233323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd8161e50 0x7fdfd815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 -- 172.21.15.45:0/1862233323 >> v1:172.21.15.117:6789/0 conn(0x7fdfd80cd9a0 legacy=0x7fdfd80d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 -- 172.21.15.45:0/1862233323 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd806b230 con 0x7fdfd806bd10 2024-04-25T10:18:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7e00640 1 --2- 172.21.15.45:0/1862233323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd8161910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009870 tx=0x7fdfc401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cc633443dba469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7400640 1 -- 172.21.15.45:0/1862233323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40b5070 con 0x7fdfd806bd10 2024-04-25T10:18:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7400640 1 -- 172.21.15.45:0/1862233323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc40a8a50 con 0x7fdfd806bd10 2024-04-25T10:18:33.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.020+0000 7fdfd7400640 1 -- 172.21.15.45:0/1862233323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40a8e20 con 0x7fdfd806bd10 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- 172.21.15.45:0/1862233323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 msgr2=0x7fdfd8161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 --2- 172.21.15.45:0/1862233323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd8161910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009870 tx=0x7fdfc401ed50 comp rx=0 tx=0).stop 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- 172.21.15.45:0/1862233323 shutdown_connections 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 --2- 172.21.15.45:0/1862233323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd8161e50 0x7fdfd815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 --2- 172.21.15.45:0/1862233323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd8161910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- 172.21.15.45:0/1862233323 >> 172.21.15.45:0/1862233323 conn(0x7fdfd80d19a0 msgr2=0x7fdfd80d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- 172.21.15.45:0/1862233323 shutdown_connections 2024-04-25T10:18:33.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- 172.21.15.45:0/1862233323 wait complete. 2024-04-25T10:18:33.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 Processor -- start 2024-04-25T10:18:33.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- start start 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd815e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd80cd9a0 0x7fdfd81579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd8161e50 0x7fdfd80c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd8162660 con 0x7fdfd806bd10 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80cdd80 con 0x7fdfd8161e50 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdf600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80d0d70 con 0x7fdfd80cd9a0 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd815e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd815e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42310/0 (socket says 172.21.15.45:42310) 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 -- 172.21.15.45:0/3256727107 learned_addr learned my addr 172.21.15.45:0/3256727107 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdd600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd8161e50 0x7fdfd80c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfd7e00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd80cd9a0 0x7fdfd81579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 -- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd80cd9a0 msgr2=0x7fdfd81579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd80cd9a0 0x7fdfd81579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 -- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd8161e50 msgr2=0x7fdfd80c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd8161e50 0x7fdfd80c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.024+0000 7fdfdcc00640 1 -- 172.21.15.45:0/3256727107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd81f48a0 con 0x7fdfd806bd10 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfdcc00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd815e310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c420 tx=0x7fdfcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0052d0 con 0x7fdfd806bd10 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc005470 con 0x7fdfd806bd10 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc007260 con 0x7fdfd806bd10 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd81f3ae0 con 0x7fdfd806bd10 2024-04-25T10:18:33.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd81f40a0 con 0x7fdfd806bd10 2024-04-25T10:18:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd804ef30 con 0x7fdfd806bd10 2024-04-25T10:18:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfcc007400 con 0x7fdfd806bd10 2024-04-25T10:18:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd4c00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfb8066f60 0x7fdfb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfcc084600 con 0x7fdfd806bd10 2024-04-25T10:18:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.028+0000 7fdfd7e00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfb8066f60 0x7fdfb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.032+0000 7fdfd7e00640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfb8066f60 0x7fdfb8069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b2000 tx=0x7fdfc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.032+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfcc055570 con 0x7fdfd806bd10 2024-04-25T10:18:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.196+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdfd80b7920 con 0x7fdfd806bd10 2024-04-25T10:18:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfd4c00640 1 -- 172.21.15.45:0/3256727107 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdfcc054e70 con 0x7fdfd806bd10 2024-04-25T10:18:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfb8066f60 msgr2=0x7fdfb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfb8066f60 0x7fdfb8069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b2000 tx=0x7fdfc4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 msgr2=0x7fdfd815e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd815e310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c420 tx=0x7fdfcc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 shutdown_connections 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfb8066f60 0x7fdfb8069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfd8161e50 0x7fdfd80c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfd80cd9a0 0x7fdfd81579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 --2- 172.21.15.45:0/3256727107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd806bd10 0x7fdfd815e310 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 >> 172.21.15.45:0/3256727107 conn(0x7fdfd80d19a0 msgr2=0x7fdfd80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 shutdown_connections 2024-04-25T10:18:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.200+0000 7fdfdf600640 1 -- 172.21.15.45:0/3256727107 wait complete. 2024-04-25T10:18:33.224 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:18:33.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:33.308 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:33.393 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:18:33.394 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:33.394 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.403 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:33.403 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.457 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:33.457 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:33.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:33.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f0205800640 1 Processor -- start 2024-04-25T10:18:33.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f0205800640 1 -- start start 2024-04-25T10:18:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f0205800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02000670b0 0x7f020006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f0205800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020006f200 con 0x7f02000af9c0 2024-04-25T10:18:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f0205800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02000ccc10 con 0x7f02000afda0 2024-04-25T10:18:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f0205800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020006d380 con 0x7f02000670b0 2024-04-25T10:18:33.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01fea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f02000af9c0 0x7f02000cc3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50182/0 (socket says 172.21.15.45:50182) 2024-04-25T10:18:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01fea00640 1 -- 172.21.15.45:0/2031621091 learned_addr learned my addr 172.21.15.45:0/2031621091 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01ff400640 1 --2- 172.21.15.45:0/2031621091 >> v2:172.21.15.117:3300/0 conn(0x7f02000670b0 0x7f020006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01fd600640 1 -- 172.21.15.45:0/2031621091 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490130905 0 0) 0x7f01f0001120 con 0x7f02000af9c0 2024-04-25T10:18:33.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01fd600640 1 -- 172.21.15.45:0/2031621091 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e4003530 con 0x7f02000af9c0 2024-04-25T10:18:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01ff400640 1 -- 172.21.15.45:0/2031621091 >> v1:172.21.15.117:6789/0 conn(0x7f02000afda0 legacy=0x7f02000669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01ff400640 1 -- 172.21.15.45:0/2031621091 >> v1:172.21.15.45:6789/0 conn(0x7f02000af9c0 legacy=0x7f02000cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.692+0000 7f01ff400640 1 -- 172.21.15.45:0/2031621091 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000ccef0 con 0x7f02000670b0 2024-04-25T10:18:33.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f01ff400640 1 --2- 172.21.15.45:0/2031621091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f020006ddd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009d90 tx=0x7f01f801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71cc007ec93adeaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f01fd600640 1 -- 172.21.15.45:0/2031621091 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80b2070 con 0x7f02000670b0 2024-04-25T10:18:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f01fd600640 1 -- 172.21.15.45:0/2031621091 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f80a5970 con 0x7f02000670b0 2024-04-25T10:18:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f01fd600640 1 -- 172.21.15.45:0/2031621091 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80a5d40 con 0x7f02000670b0 2024-04-25T10:18:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 -- 172.21.15.45:0/2031621091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 msgr2=0x7f020006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 --2- 172.21.15.45:0/2031621091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f020006ddd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009d90 tx=0x7f01f801c8b0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 -- 172.21.15.45:0/2031621091 shutdown_connections 2024-04-25T10:18:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 --2- 172.21.15.45:0/2031621091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f020006ddd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 -- 172.21.15.45:0/2031621091 >> 172.21.15.45:0/2031621091 conn(0x7f02000b2410 msgr2=0x7f02000b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 -- 172.21.15.45:0/2031621091 shutdown_connections 2024-04-25T10:18:33.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.696+0000 7f0205800640 1 -- 172.21.15.45:0/2031621091 wait complete. 2024-04-25T10:18:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 Processor -- start 2024-04-25T10:18:33.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 -- start start 2024-04-25T10:18:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f02001538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 0x7f02000c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02000afda0 0x7f0200151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0200160460 con 0x7f02000af9c0 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02001602e0 con 0x7f02000670b0 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c5f10 con 0x7f02000afda0 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 0x7f02000c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f02001538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 0x7f02000c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42312/0 (socket says 172.21.15.45:42312) 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f02001538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43552/0 (socket says 172.21.15.45:43552) 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 -- 172.21.15.45:0/3290093731 learned_addr learned my addr 172.21.15.45:0/3290093731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01ff400640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02000afda0 0x7f0200151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 -- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02000afda0 msgr2=0x7f0200151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02000afda0 0x7f0200151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 -- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 msgr2=0x7f02001538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f02001538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 -- 172.21.15.45:0/3290093731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000c4e70 con 0x7f02000af9c0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01fe000640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 0x7f02000c43b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c370 tx=0x7f01ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0052d0 con 0x7f02000af9c0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec005470 con 0x7f02000af9c0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007260 con 0x7f02000af9c0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000c5150 con 0x7f02000af9c0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.700+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02000c5710 con 0x7f02000af9c0 2024-04-25T10:18:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.704+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020004ef30 con 0x7f02000af9c0 2024-04-25T10:18:33.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.704+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01ec007400 con 0x7f02000af9c0 2024-04-25T10:18:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.704+0000 7f01f6a00640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01e0066f60 0x7f01e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.704+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01ec0843d0 con 0x7f02000af9c0 2024-04-25T10:18:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.704+0000 7f01fea00640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01e0066f60 0x7f01e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.708+0000 7f01fea00640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01e0066f60 0x7f01e0069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005890 tx=0x7f01f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:33.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.708+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01ec054d80 con 0x7f02000af9c0 2024-04-25T10:18:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0200071420 con 0x7f02000af9c0 2024-04-25T10:18:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f01f6a00640 1 -- 172.21.15.45:0/3290093731 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f01ec058e20 con 0x7f02000af9c0 2024-04-25T10:18:33.879 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:33.880 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01e0066f60 msgr2=0x7f01e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01e0066f60 0x7f01e0069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005890 tx=0x7f01f0009040 comp rx=0 tx=0).stop 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 msgr2=0x7f02000c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 0x7f02000c43b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c370 tx=0x7f01ec00c830 comp rx=0 tx=0).stop 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 shutdown_connections 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01e0066f60 0x7f01e0069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02000afda0 0x7f0200151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02000af9c0 0x7f02000c43b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 --2- 172.21.15.45:0/3290093731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02000670b0 0x7f02001538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 >> 172.21.15.45:0/3290093731 conn(0x7f02000b2410 msgr2=0x7f02000669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.876+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 shutdown_connections 2024-04-25T10:18:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:33.880+0000 7f0205800640 1 -- 172.21.15.45:0/3290093731 wait complete. 2024-04-25T10:18:33.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:33.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.040+0000 7f8b00000640 1 Processor -- start 2024-04-25T10:18:34.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.040+0000 7f8b00000640 1 -- start start 2024-04-25T10:18:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.040+0000 7f8b00000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8af8159540 0x7f8af80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8b00000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80721f0 0x7f8af8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8b00000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af8159920 con 0x7f8af80aebd0 2024-04-25T10:18:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8b00000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af815c2f0 con 0x7f8af8159540 2024-04-25T10:18:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8b00000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af815c5d0 con 0x7f8af80721f0 2024-04-25T10:18:34.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8af8159540 0x7f8af80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8af8159540 0x7f8af80ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42324/0 (socket says 172.21.15.45:42324) 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 -- 172.21.15.45:0/456665619 learned_addr learned my addr 172.21.15.45:0/456665619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afcc00640 1 --1- 172.21.15.45:0/456665619 >> v1:172.21.15.45:6789/0 conn(0x7f8af80aebd0 0x7f8af8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50184/0 (socket says 172.21.15.45:50184) 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afe000640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80721f0 0x7f8af8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 -- 172.21.15.45:0/456665619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80721f0 msgr2=0x7f8af8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80721f0 0x7f8af8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 -- 172.21.15.45:0/456665619 >> v1:172.21.15.45:6789/0 conn(0x7f8af80aebd0 legacy=0x7f8af8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 -- 172.21.15.45:0/456665619 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af815c8b0 con 0x7f8af8159540 2024-04-25T10:18:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afe000640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80721f0 0x7f8af8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8afd600640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af80ae690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009870 tx=0x7f8aec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ba579998b3a571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8af7e00640 1 -- 172.21.15.45:0/456665619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0a7070 con 0x7f8af8159540 2024-04-25T10:18:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8af7e00640 1 -- 172.21.15.45:0/456665619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec0b0070 con 0x7f8af8159540 2024-04-25T10:18:34.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.044+0000 7f8af7e00640 1 -- 172.21.15.45:0/456665619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0ab890 con 0x7f8af8159540 2024-04-25T10:18:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- 172.21.15.45:0/456665619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 msgr2=0x7f8af80ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af80ae690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009870 tx=0x7f8aec01c560 comp rx=0 tx=0).stop 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- 172.21.15.45:0/456665619 shutdown_connections 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80721f0 0x7f8af8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 --2- 172.21.15.45:0/456665619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af80ae690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- 172.21.15.45:0/456665619 >> 172.21.15.45:0/456665619 conn(0x7f8af80669a0 msgr2=0x7f8af8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- 172.21.15.45:0/456665619 shutdown_connections 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- 172.21.15.45:0/456665619 wait complete. 2024-04-25T10:18:34.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 Processor -- start 2024-04-25T10:18:34.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- start start 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80721f0 0x7f8af80a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80aebd0 0x7f8af8150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af81526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8071220 con 0x7f8af8159540 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80710a0 con 0x7f8af80721f0 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8b00000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80713a0 con 0x7f8af80aebd0 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af81526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af81526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42336/0 (socket says 172.21.15.45:42336) 2024-04-25T10:18:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 -- 172.21.15.45:0/1482013925 learned_addr learned my addr 172.21.15.45:0/1482013925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afcc00640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80aebd0 0x7f8af8150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afd600640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80721f0 0x7f8af80a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 -- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80aebd0 msgr2=0x7f8af8150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80aebd0 0x7f8af8150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 -- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80721f0 msgr2=0x7f8af80a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80721f0 0x7f8af80a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afe000640 1 -- 172.21.15.45:0/1482013925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af8152f00 con 0x7f8af8159540 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.048+0000 7f8afd600640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80721f0 0x7f8af80a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8afe000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af81526d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8af000cef0 tx=0x7f8af000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af001a070 con 0x7f8af8159540 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af000d860 con 0x7f8af8159540 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000dc50 con 0x7f8af8159540 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af81531e0 con 0x7f8af8159540 2024-04-25T10:18:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af8152c10 con 0x7f8af8159540 2024-04-25T10:18:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80af2a0 con 0x7f8af8159540 2024-04-25T10:18:34.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8af001d070 con 0x7f8af8159540 2024-04-25T10:18:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8af5600640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad8066f60 0x7f8ad8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8af0085950 con 0x7f8af8159540 2024-04-25T10:18:34.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.052+0000 7f8afd600640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad8066f60 0x7f8ad8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.056+0000 7f8afd600640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad8066f60 0x7f8ad8069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0af000 tx=0x7f8aec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.056+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8af0056a10 con 0x7f8af8159540 2024-04-25T10:18:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.220+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8af804ef30 con 0x7f8af8159540 2024-04-25T10:18:34.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.220+0000 7f8af5600640 1 -- 172.21.15.45:0/1482013925 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8af0056310 con 0x7f8af8159540 2024-04-25T10:18:34.224 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad8066f60 msgr2=0x7f8ad8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad8066f60 0x7f8ad8069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0af000 tx=0x7f8aec008040 comp rx=0 tx=0).stop 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 msgr2=0x7f8af81526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af81526d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8af000cef0 tx=0x7f8af000a710 comp rx=0 tx=0).stop 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 shutdown_connections 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad8066f60 0x7f8ad8069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8159540 0x7f8af81526d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af80aebd0 0x7f8af8150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 --2- 172.21.15.45:0/1482013925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80721f0 0x7f8af80a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 >> 172.21.15.45:0/1482013925 conn(0x7f8af80669a0 msgr2=0x7f8af815a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 shutdown_connections 2024-04-25T10:18:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.224+0000 7f8b00000640 1 -- 172.21.15.45:0/1482013925 wait complete. 2024-04-25T10:18:34.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:34.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 Processor -- start 2024-04-25T10:18:34.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 -- start start 2024-04-25T10:18:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5e4159540 0x7fd5e40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40721f0 0x7fd5e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e4159920 con 0x7fd5e40aebd0 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e415c2f0 con 0x7fd5e4159540 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5eb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e415c5d0 con 0x7fd5e40721f0 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5e4159540 0x7fd5e40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5e4159540 0x7fd5e40ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42350/0 (socket says 172.21.15.45:42350) 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 -- 172.21.15.45:0/587120669 learned_addr learned my addr 172.21.15.45:0/587120669 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e9200640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40721f0 0x7fd5e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 -- 172.21.15.45:0/587120669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40721f0 msgr2=0x7fd5e4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40721f0 0x7fd5e4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 -- 172.21.15.45:0/587120669 >> v1:172.21.15.117:6789/0 conn(0x7fd5e40aebd0 legacy=0x7fd5e4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e3e00640 1 -- 172.21.15.45:0/587120669 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e415c8b0 con 0x7fd5e4159540 2024-04-25T10:18:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.388+0000 7fd5e9200640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40721f0 0x7fd5e4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:34.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5e3e00640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e40ae690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009870 tx=0x7fd5d401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6378694f608479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5e3400640 1 -- 172.21.15.45:0/587120669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40a7070 con 0x7fd5e4159540 2024-04-25T10:18:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5e3400640 1 -- 172.21.15.45:0/587120669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d40b0070 con 0x7fd5e4159540 2024-04-25T10:18:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5e3400640 1 -- 172.21.15.45:0/587120669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40ab890 con 0x7fd5e4159540 2024-04-25T10:18:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 -- 172.21.15.45:0/587120669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 msgr2=0x7fd5e40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e40ae690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009870 tx=0x7fd5d401c560 comp rx=0 tx=0).stop 2024-04-25T10:18:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 -- 172.21.15.45:0/587120669 shutdown_connections 2024-04-25T10:18:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40721f0 0x7fd5e4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 --2- 172.21.15.45:0/587120669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e40ae690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 -- 172.21.15.45:0/587120669 >> 172.21.15.45:0/587120669 conn(0x7fd5e40669a0 msgr2=0x7fd5e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 -- 172.21.15.45:0/587120669 shutdown_connections 2024-04-25T10:18:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.392+0000 7fd5eb200640 1 -- 172.21.15.45:0/587120669 wait complete. 2024-04-25T10:18:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 Processor -- start 2024-04-25T10:18:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 -- start start 2024-04-25T10:18:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40721f0 0x7fd5e4155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40aebd0 0x7fd5e4154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e4071220 con 0x7fd5e4159540 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40710a0 con 0x7fd5e40721f0 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40713a0 con 0x7fd5e40aebd0 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e415b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42366/0 (socket says 172.21.15.45:42366) 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 -- 172.21.15.45:0/1002145926 learned_addr learned my addr 172.21.15.45:0/1002145926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5db400640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40aebd0 0x7fd5e4154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e3e00640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40721f0 0x7fd5e4155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 -- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40aebd0 msgr2=0x7fd5e4154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40aebd0 0x7fd5e4154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 -- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40721f0 msgr2=0x7fd5e4155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40721f0 0x7fd5e4155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 -- 172.21.15.45:0/1002145926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e41f8c10 con 0x7fd5e4159540 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e9200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e415b270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009fa0 tx=0x7fd5dc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc01a070 con 0x7fd5e4159540 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc003bc0 con 0x7fd5e4159540 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e4155100 con 0x7fd5e4159540 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004030 con 0x7fd5e4159540 2024-04-25T10:18:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.396+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e41556c0 con 0x7fd5e4159540 2024-04-25T10:18:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.400+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e404ef30 con 0x7fd5e4159540 2024-04-25T10:18:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.400+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5dc01d070 con 0x7fd5e4159540 2024-04-25T10:18:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.400+0000 7fd5e0c00640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5c0066f60 0x7fd5c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.400+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5dc0853e0 con 0x7fd5e4159540 2024-04-25T10:18:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.400+0000 7fd5e3e00640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5c0066f60 0x7fd5c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.400+0000 7fd5e3e00640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5c0066f60 0x7fd5c0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40af000 tx=0x7fd5d40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.404+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5dc010070 con 0x7fd5e4159540 2024-04-25T10:18:34.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.568+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5e40cf470 con 0x7fd5e4159540 2024-04-25T10:18:34.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.568+0000 7fd5e0c00640 1 -- 172.21.15.45:0/1002145926 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5dc056350 con 0x7fd5e4159540 2024-04-25T10:18:34.572 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5c0066f60 msgr2=0x7fd5c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5c0066f60 0x7fd5c0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40af000 tx=0x7fd5d40a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 msgr2=0x7fd5e415b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e415b270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009fa0 tx=0x7fd5dc00d560 comp rx=0 tx=0).stop 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 shutdown_connections 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd5c0066f60 0x7fd5c0069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5e4159540 0x7fd5e415b270 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e40aebd0 0x7fd5e4154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 --2- 172.21.15.45:0/1002145926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40721f0 0x7fd5e4155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 >> 172.21.15.45:0/1002145926 conn(0x7fd5e40669a0 msgr2=0x7fd5e415f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 shutdown_connections 2024-04-25T10:18:34.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.572+0000 7fd5eb200640 1 -- 172.21.15.45:0/1002145926 wait complete. 2024-04-25T10:18:34.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:34.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 Processor -- start 2024-04-25T10:18:34.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 -- start start 2024-04-25T10:18:34.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97500aeda0 0x7f9750071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9750072020 0x7f9750070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9750159920 con 0x7f9750159540 2024-04-25T10:18:34.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975015c2f0 con 0x7f9750072020 2024-04-25T10:18:34.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9758000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975015c5d0 con 0x7f97500aeda0 2024-04-25T10:18:34.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9755600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9750159540 0x7f97500ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50198/0 (socket says 172.21.15.45:50198) 2024-04-25T10:18:34.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9755600640 1 -- 172.21.15.45:0/3420213841 learned_addr learned my addr 172.21.15.45:0/3420213841 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:34.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.736+0000 7f9756000640 1 --2- 172.21.15.45:0/3420213841 >> v2:172.21.15.45:3300/0 conn(0x7f9750072020 0x7f9750070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9754c00640 1 --2- 172.21.15.45:0/3420213841 >> v2:172.21.15.117:3300/0 conn(0x7f97500aeda0 0x7f9750071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f974fe00640 1 -- 172.21.15.45:0/3420213841 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708900810 0 0) 0x7f9750159920 con 0x7f9750159540 2024-04-25T10:18:34.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9756000640 1 -- 172.21.15.45:0/3420213841 >> v2:172.21.15.117:3300/0 conn(0x7f97500aeda0 msgr2=0x7f9750071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:34.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9756000640 1 --2- 172.21.15.45:0/3420213841 >> v2:172.21.15.117:3300/0 conn(0x7f97500aeda0 0x7f9750071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9756000640 1 -- 172.21.15.45:0/3420213841 >> v1:172.21.15.45:6789/0 conn(0x7f9750159540 legacy=0x7f97500ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9756000640 1 -- 172.21.15.45:0/3420213841 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975015c8b0 con 0x7f9750072020 2024-04-25T10:18:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9756000640 1 --2- 172.21.15.45:0/3420213841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f9750070350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f97500cd510 tx=0x7f974801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfd31964791ce5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f974fe00640 1 -- 172.21.15.45:0/3420213841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480b2070 con 0x7f9750072020 2024-04-25T10:18:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f974fe00640 1 -- 172.21.15.45:0/3420213841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97480a59a0 con 0x7f9750072020 2024-04-25T10:18:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f974fe00640 1 -- 172.21.15.45:0/3420213841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480a5d70 con 0x7f9750072020 2024-04-25T10:18:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9758000640 1 -- 172.21.15.45:0/3420213841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 msgr2=0x7f9750070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.740+0000 7f9758000640 1 --2- 172.21.15.45:0/3420213841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f9750070350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f97500cd510 tx=0x7f974801c900 comp rx=0 tx=0).stop 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- 172.21.15.45:0/3420213841 shutdown_connections 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 --2- 172.21.15.45:0/3420213841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f9750070350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 --2- 172.21.15.45:0/3420213841 >> v2:172.21.15.117:3300/0 conn(0x7f97500aeda0 0x7f9750071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- 172.21.15.45:0/3420213841 >> 172.21.15.45:0/3420213841 conn(0x7f97500669a0 msgr2=0x7f9750066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- 172.21.15.45:0/3420213841 shutdown_connections 2024-04-25T10:18:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- 172.21.15.45:0/3420213841 wait complete. 2024-04-25T10:18:34.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 Processor -- start 2024-04-25T10:18:34.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- start start 2024-04-25T10:18:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f975014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97500aeda0 0x7f9750155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9750159540 0x7f975015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500710a0 con 0x7f9750072020 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975006c0d0 con 0x7f97500aeda0 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9758000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9750071220 con 0x7f9750159540 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f975014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f975014fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42390/0 (socket says 172.21.15.45:42390) 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 -- 172.21.15.45:0/606225737 learned_addr learned my addr 172.21.15.45:0/606225737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9754c00640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97500aeda0 0x7f9750155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9756000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9750159540 0x7f975015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 -- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9750159540 msgr2=0x7f975015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9750159540 0x7f975015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 -- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97500aeda0 msgr2=0x7f9750155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97500aeda0 0x7f9750155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 -- 172.21.15.45:0/606225737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750150310 con 0x7f9750072020 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.744+0000 7f9755600640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f975014fdd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97440011f0 tx=0x7f974400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744018070 con 0x7f9750072020 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97440074e0 con 0x7f9750072020 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97501525a0 con 0x7f9750072020 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744004a10 con 0x7f9750072020 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9750151640 con 0x7f9750072020 2024-04-25T10:18:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975004ef30 con 0x7f9750072020 2024-04-25T10:18:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f974401b070 con 0x7f9750072020 2024-04-25T10:18:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f974d600640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9730066f60 0x7f9730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:34.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97440842c0 con 0x7f9750072020 2024-04-25T10:18:34.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.748+0000 7f9754c00640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9730066f60 0x7f9730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:34.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.752+0000 7f9754c00640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9730066f60 0x7f9730069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9750153ed0 tx=0x7f973c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:34.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.752+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9750150310 con 0x7f9750072020 2024-04-25T10:18:34.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.916+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97500cb980 con 0x7f9750072020 2024-04-25T10:18:34.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.916+0000 7f974d600640 1 -- 172.21.15.45:0/606225737 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9744054c80 con 0x7f9750072020 2024-04-25T10:18:34.921 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9730066f60 msgr2=0x7f9730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9730066f60 0x7f9730069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9750153ed0 tx=0x7f973c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 msgr2=0x7f975014fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f975014fdd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97440011f0 tx=0x7f974400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 shutdown_connections 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9730066f60 0x7f9730069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9750159540 0x7f975015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97500aeda0 0x7f9750155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 --2- 172.21.15.45:0/606225737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9750072020 0x7f975014fdd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 >> 172.21.15.45:0/606225737 conn(0x7f97500669a0 msgr2=0x7f97500cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 shutdown_connections 2024-04-25T10:18:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:34.920+0000 7f9758000640 1 -- 172.21.15.45:0/606225737 wait complete. 2024-04-25T10:18:34.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.080+0000 7f6bb5c00640 1 Processor -- start 2024-04-25T10:18:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.080+0000 7f6bb5c00640 1 -- start start 2024-04-25T10:18:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bb5c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6bb0066c00 0x7f6bb015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bb5c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb00b5970 con 0x7f6bb0066fe0 2024-04-25T10:18:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bb5c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb00b5c20 con 0x7f6bb00b5560 2024-04-25T10:18:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bb5c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb00af6e0 con 0x7f6bb0066c00 2024-04-25T10:18:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6ba6000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6bb0066fe0 0x7f6bb006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38270/0 (socket says 172.21.15.45:38270) 2024-04-25T10:18:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6ba6000640 1 -- 172.21.15.45:0/1045022613 learned_addr learned my addr 172.21.15.45:0/1045022613 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baea00640 1 --2- 172.21.15.45:0/1045022613 >> v2:172.21.15.117:3300/0 conn(0x7f6bb0066c00 0x7f6bb015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422966982 0 0) 0x7f6bb00b5970 con 0x7f6bb0066fe0 2024-04-25T10:18:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b94003530 con 0x7f6bb0066fe0 2024-04-25T10:18:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1814496662 0 0) 0x7f6b94003530 con 0x7f6bb0066fe0 2024-04-25T10:18:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 >> v2:172.21.15.117:3300/0 conn(0x7f6bb0066c00 msgr2=0x7f6bb015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 --2- 172.21.15.45:0/1045022613 >> v2:172.21.15.117:3300/0 conn(0x7f6bb0066c00 0x7f6bb015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 >> v1:172.21.15.117:6789/0 conn(0x7f6bb00b5560 legacy=0x7f6bb0157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00af9c0 con 0x7f6bb0066fe0 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6b98001680 con 0x7f6bb0066fe0 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 >> v1:172.21.15.45:6789/0 conn(0x7f6bb0066fe0 legacy=0x7f6bb006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b940039c0 0x7f6b94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b94004810 0x7f6b94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00af6e0 con 0x7f6b940051d0 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9400c070 con 0x7f6b940039c0 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9400c270 con 0x7f6b94004810 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baf400640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baea00640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b940039c0 0x7f6b94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6ba6000640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b94004810 0x7f6b94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baf400640 1 -- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b94004810 msgr2=0x7f6b94004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baf400640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b94004810 0x7f6b94004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baf400640 1 -- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b940039c0 msgr2=0x7f6b94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baf400640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b940039c0 0x7f6b94003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.084+0000 7f6baf400640 1 -- 172.21.15.45:0/1045022613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b94003530 con 0x7f6b940051d0 2024-04-25T10:18:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6baf400640 1 -- 172.21.15.45:0/1045022613 reap_dead start 2024-04-25T10:18:35.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6baf400640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8001e70 tx=0x7f6ba801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e204f4b25ea98b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8001a90 con 0x7f6b940051d0 2024-04-25T10:18:35.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba8003ba0 con 0x7f6b940051d0 2024-04-25T10:18:35.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bae000640 1 -- 172.21.15.45:0/1045022613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8005020 con 0x7f6b940051d0 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 -- 172.21.15.45:0/1045022613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b940051d0 msgr2=0x7f6b9400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8001e70 tx=0x7f6ba801c780 comp rx=0 tx=0).stop 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 -- 172.21.15.45:0/1045022613 shutdown_connections 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/1045022613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 -- 172.21.15.45:0/1045022613 >> 172.21.15.45:0/1045022613 conn(0x7f6bb00b24f0 msgr2=0x7f6bb00b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 -- 172.21.15.45:0/1045022613 shutdown_connections 2024-04-25T10:18:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 -- 172.21.15.45:0/1045022613 wait complete. 2024-04-25T10:18:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 Processor -- start 2024-04-25T10:18:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.088+0000 7f6bb5c00640 1 -- start start 2024-04-25T10:18:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b940051d0 0x7f6bb0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb0154430 0x7f6bb01474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 0x7f6bb015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb015e110 con 0x7f6bb0159d80 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00aff80 con 0x7f6bb0154430 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb015e290 con 0x7f6b940051d0 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 0x7f6bb015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b940051d0 0x7f6bb0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 0x7f6bb015cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52308/0 (socket says 172.21.15.45:52308) 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 -- 172.21.15.45:0/3620120300 learned_addr learned my addr 172.21.15.45:0/3620120300 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6ba6000640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb0154430 0x7f6bb01474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 -- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b940051d0 msgr2=0x7f6bb0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b940051d0 0x7f6bb0155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 -- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb0154430 msgr2=0x7f6bb01474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb0154430 0x7f6bb01474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 -- 172.21.15.45:0/3620120300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0154ee0 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6baf400640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 0x7f6bb015cd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8098190 tx=0x7f6b94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8007dc0 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba8001260 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8006d80 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb01551c0 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb0155780 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.092+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb004ef30 con 0x7f6bb0159d80 2024-04-25T10:18:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.096+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ba8005070 con 0x7f6bb0159d80 2024-04-25T10:18:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.096+0000 7f6ba7400640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b8c066f90 0x7f6b8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.096+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ba80ad020 con 0x7f6bb0159d80 2024-04-25T10:18:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.096+0000 7f6baea00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b8c066f90 0x7f6b8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.096+0000 7f6baea00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b8c066f90 0x7f6b8c069450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009ac0 tx=0x7f6ba0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.100+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ba8016490 con 0x7f6bb0159d80 2024-04-25T10:18:35.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6bb0071c70 con 0x7f6bb0159d80 2024-04-25T10:18:35.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6ba7400640 1 -- 172.21.15.45:0/3620120300 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6ba80eedf0 con 0x7f6bb0159d80 2024-04-25T10:18:35.279 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:18:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b8c066f90 msgr2=0x7f6b8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b8c066f90 0x7f6b8c069450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009ac0 tx=0x7f6ba0009190 comp rx=0 tx=0).stop 2024-04-25T10:18:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 msgr2=0x7f6bb015cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 0x7f6bb015cd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8098190 tx=0x7f6b94005840 comp rx=0 tx=0).stop 2024-04-25T10:18:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 shutdown_connections 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b8c066f90 0x7f6b8c069450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bb0159d80 0x7f6bb015cd20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bb0154430 0x7f6bb01474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 --2- 172.21.15.45:0/3620120300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b940051d0 0x7f6bb0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 >> 172.21.15.45:0/3620120300 conn(0x7f6bb00b24f0 msgr2=0x7f6bb006ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 shutdown_connections 2024-04-25T10:18:35.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.276+0000 7f6bb5c00640 1 -- 172.21.15.45:0/3620120300 wait complete. 2024-04-25T10:18:35.300 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:18:35.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:35.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:18:35.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:18:35.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:35.336 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:18:35.394 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:35.394 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:35.403 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:35.403 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:35.455 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:35.455 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:35.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 Processor -- start 2024-04-25T10:18:35.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 -- start start 2024-04-25T10:18:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5f40071f70 0x7f5f400b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f400b2fc0 0x7f5f4006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f40070ed0 con 0x7f5f4015a0e0 2024-04-25T10:18:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f400711b0 con 0x7f5f40071f70 2024-04-25T10:18:35.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f46600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40071490 con 0x7f5f400b2fc0 2024-04-25T10:18:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5f4015a0e0 0x7f5f4015b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38278/0 (socket says 172.21.15.45:38278) 2024-04-25T10:18:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3f400640 1 -- 172.21.15.45:0/957290577 learned_addr learned my addr 172.21.15.45:0/957290577 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3ea00640 1 --2- 172.21.15.45:0/957290577 >> v2:172.21.15.117:3300/0 conn(0x7f5f40071f70 0x7f5f400b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3fe00640 1 --2- 172.21.15.45:0/957290577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f400b2fc0 0x7f5f4006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3e000640 1 -- 172.21.15.45:0/957290577 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020453330 0 0) 0x7f5f40070ed0 con 0x7f5f4015a0e0 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3e000640 1 -- 172.21.15.45:0/957290577 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f24003450 con 0x7f5f4015a0e0 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3e000640 1 -- 172.21.15.45:0/957290577 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1527055511 0 0) 0x7f5f24003450 con 0x7f5f4015a0e0 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3fe00640 1 -- 172.21.15.45:0/957290577 >> v2:172.21.15.117:3300/0 conn(0x7f5f40071f70 msgr2=0x7f5f400b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3fe00640 1 --2- 172.21.15.45:0/957290577 >> v2:172.21.15.117:3300/0 conn(0x7f5f40071f70 0x7f5f400b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3fe00640 1 -- 172.21.15.45:0/957290577 >> v1:172.21.15.45:6789/0 conn(0x7f5f4015a0e0 legacy=0x7f5f4015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.648+0000 7f5f3fe00640 1 -- 172.21.15.45:0/957290577 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40071770 con 0x7f5f400b2fc0 2024-04-25T10:18:35.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f3fe00640 1 --2- 172.21.15.45:0/957290577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f400b2fc0 0x7f5f4006c090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009d90 tx=0x7f5f3801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52cdb6af33f3b71e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f3e000640 1 -- 172.21.15.45:0/957290577 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b2070 con 0x7f5f400b2fc0 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f3e000640 1 -- 172.21.15.45:0/957290577 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380a5520 con 0x7f5f400b2fc0 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f3e000640 1 -- 172.21.15.45:0/957290577 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30001680 con 0x7f5f400b2fc0 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 -- 172.21.15.45:0/957290577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f400b2fc0 msgr2=0x7f5f4006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 --2- 172.21.15.45:0/957290577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f400b2fc0 0x7f5f4006c090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009d90 tx=0x7f5f3801c890 comp rx=0 tx=0).stop 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 -- 172.21.15.45:0/957290577 shutdown_connections 2024-04-25T10:18:35.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 --2- 172.21.15.45:0/957290577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f400b2fc0 0x7f5f4006c090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 --2- 172.21.15.45:0/957290577 >> v2:172.21.15.117:3300/0 conn(0x7f5f40071f70 0x7f5f400b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 -- 172.21.15.45:0/957290577 >> 172.21.15.45:0/957290577 conn(0x7f5f40066b70 msgr2=0x7f5f40066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 -- 172.21.15.45:0/957290577 shutdown_connections 2024-04-25T10:18:35.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.652+0000 7f5f46600640 1 -- 172.21.15.45:0/957290577 wait complete. 2024-04-25T10:18:35.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 Processor -- start 2024-04-25T10:18:35.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- start start 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f40071f70 0x7f5f401f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 0x7f5f401580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f4015a0e0 0x7f5f401551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40072350 con 0x7f5f400b2fc0 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40070b50 con 0x7f5f40071f70 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400724d0 con 0x7f5f4015a0e0 2024-04-25T10:18:35.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 0x7f5f401580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 0x7f5f401580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52322/0 (socket says 172.21.15.45:52322) 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 -- 172.21.15.45:0/1195494969 learned_addr learned my addr 172.21.15.45:0/1195494969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3f400640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f40071f70 0x7f5f401f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3fe00640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f4015a0e0 0x7f5f401551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 -- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f4015a0e0 msgr2=0x7f5f401551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f4015a0e0 0x7f5f401551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 -- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f40071f70 msgr2=0x7f5f401f6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f40071f70 0x7f5f401f6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 -- 172.21.15.45:0/1195494969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f401f7360 con 0x7f5f400b2fc0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3fe00640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f4015a0e0 0x7f5f401551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f3ea00640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 0x7f5f401580c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00e960 tx=0x7f5f2c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c017070 con 0x7f5f400b2fc0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c00ca00 con 0x7f5f400b2fc0 2024-04-25T10:18:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f401f7640 con 0x7f5f400b2fc0 2024-04-25T10:18:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00cdb0 con 0x7f5f400b2fc0 2024-04-25T10:18:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f401556f0 con 0x7f5f400b2fc0 2024-04-25T10:18:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.656+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4004ef30 con 0x7f5f400b2fc0 2024-04-25T10:18:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.660+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f2c01a070 con 0x7f5f400b2fc0 2024-04-25T10:18:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.660+0000 7f5f37400640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.660+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f2c0853a0 con 0x7f5f400b2fc0 2024-04-25T10:18:35.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.660+0000 7f5f3f400640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.660+0000 7f5f3f400640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001ec0 tx=0x7f5f300016e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.664+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f2c055d50 con 0x7f5f400b2fc0 2024-04-25T10:18:35.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.824+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f400703e0 con 0x7f5f400b2fc0 2024-04-25T10:18:35.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f37400640 1 -- 172.21.15.45:0/1195494969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f5f2c059df0 con 0x7f5f400b2fc0 2024-04-25T10:18:35.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:35.831 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 msgr2=0x7f5f20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001ec0 tx=0x7f5f300016e0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 msgr2=0x7f5f401580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 0x7f5f401580c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00e960 tx=0x7f5f2c00ee20 comp rx=0 tx=0).stop 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 shutdown_connections 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f4015a0e0 0x7f5f401551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f400b2fc0 0x7f5f401580c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 --2- 172.21.15.45:0/1195494969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f40071f70 0x7f5f401f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 >> 172.21.15.45:0/1195494969 conn(0x7f5f40066b70 msgr2=0x7f5f40156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 shutdown_connections 2024-04-25T10:18:35.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.828+0000 7f5f46600640 1 -- 172.21.15.45:0/1195494969 wait complete. 2024-04-25T10:18:35.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:35.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:35.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 Processor -- start 2024-04-25T10:18:35.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 -- start start 2024-04-25T10:18:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97ac15dcc0 0x7f97ac162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 0x7f97ac0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac15f0b0 con 0x7f97ac0b5480 2024-04-25T10:18:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac161800 con 0x7f97ac15dcc0 2024-04-25T10:18:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97b0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b3b20 con 0x7f97ac0670b0 2024-04-25T10:18:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a9400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f97ac0b5480 0x7f97ac0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38284/0 (socket says 172.21.15.45:38284) 2024-04-25T10:18:36.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a9400640 1 -- 172.21.15.45:0/1777247439 learned_addr learned my addr 172.21.15.45:0/1777247439 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627313861 0 0) 0x7f97ac15f0b0 con 0x7f97ac0b5480 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a9e00640 1 --2- 172.21.15.45:0/1777247439 >> v2:172.21.15.117:3300/0 conn(0x7f97ac15dcc0 0x7f97ac162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9790003530 con 0x7f97ac0b5480 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 0x7f97ac0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272916392 0 0) 0x7f9790003530 con 0x7f97ac0b5480 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 msgr2=0x7f97ac0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 0x7f97ac0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 >> v2:172.21.15.117:3300/0 conn(0x7f97ac15dcc0 msgr2=0x7f97ac162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 --2- 172.21.15.45:0/1777247439 >> v2:172.21.15.117:3300/0 conn(0x7f97ac15dcc0 0x7f97ac162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0b3e00 con 0x7f97ac0b5480 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f9798001680 con 0x7f97ac0b5480 2024-04-25T10:18:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 >> v1:172.21.15.45:6789/0 conn(0x7f97ac0b5480 legacy=0x7f97ac0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97900039c0 0x7f9790003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 0x7f9790004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9790004d50 0x7f97900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac161800 con 0x7f9790004d50 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b3b20 con 0x7f97900039c0 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979000b9d0 con 0x7f97ac0670b0 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9790004d50 0x7f97900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a9e00640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97900039c0 0x7f9790003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97a9400640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 0x7f9790004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 -- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 msgr2=0x7f9790004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac0670b0 0x7f9790004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 -- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97900039c0 msgr2=0x7f9790003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97900039c0 0x7f9790003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:35.996+0000 7f97aa800640 1 -- 172.21.15.45:0/1777247439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9790003530 con 0x7f9790004d50 2024-04-25T10:18:36.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97aa800640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9790004d50 0x7f97900051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009fc0 tx=0x7f97a401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befa4126c3b09b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40b2070 con 0x7f9790004d50 2024-04-25T10:18:36.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a40a5c50 con 0x7f9790004d50 2024-04-25T10:18:36.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97a8a00640 1 -- 172.21.15.45:0/1777247439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4009c10 con 0x7f9790004d50 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97b0a00640 1 -- 172.21.15.45:0/1777247439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9790004d50 msgr2=0x7f97900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97b0a00640 1 --2- 172.21.15.45:0/1777247439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9790004d50 0x7f97900051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009fc0 tx=0x7f97a401cc40 comp rx=0 tx=0).stop 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97aa800640 1 -- 172.21.15.45:0/1777247439 reap_dead start 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97b0a00640 1 -- 172.21.15.45:0/1777247439 shutdown_connections 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97b0a00640 1 -- 172.21.15.45:0/1777247439 >> 172.21.15.45:0/1777247439 conn(0x7f97ac06ac90 msgr2=0x7f97ac06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97b0a00640 1 -- 172.21.15.45:0/1777247439 shutdown_connections 2024-04-25T10:18:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.000+0000 7f97b0a00640 1 -- 172.21.15.45:0/1777247439 wait complete. 2024-04-25T10:18:36.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 Processor -- start 2024-04-25T10:18:36.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 -- start start 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac140c10 0x7f97ac1414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 0x7f97ac0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac142b60 0x7f97ac06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b3780 con 0x7f97ac140ff0 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac161d30 con 0x7f97ac140c10 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b3900 con 0x7f97ac142b60 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 0x7f97ac0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 0x7f97ac0b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52338/0 (socket says 172.21.15.45:52338) 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 -- 172.21.15.45:0/927310999 learned_addr learned my addr 172.21.15.45:0/927310999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9e00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac140c10 0x7f97ac1414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97aa800640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac142b60 0x7f97ac06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 -- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac142b60 msgr2=0x7f97ac06e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac142b60 0x7f97ac06e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 -- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac140c10 msgr2=0x7f97ac1414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac140c10 0x7f97ac1414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 -- 172.21.15.45:0/927310999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac141ac0 con 0x7f97ac140ff0 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9e00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac140c10 0x7f97ac1414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97aa800640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac142b60 0x7f97ac06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a9400640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 0x7f97ac0b1ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9798007340 tx=0x7f9798004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979800fa80 con 0x7f97ac140ff0 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9798018070 con 0x7f97ac140ff0 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979800fe60 con 0x7f97ac140ff0 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac141da0 con 0x7f97ac140ff0 2024-04-25T10:18:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.004+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac142360 con 0x7f97ac140ff0 2024-04-25T10:18:36.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.008+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac04ef30 con 0x7f97ac140ff0 2024-04-25T10:18:36.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.008+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f979801d460 con 0x7f97ac140ff0 2024-04-25T10:18:36.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.008+0000 7f97a2000640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f978c066f60 0x7f978c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.008+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9798085400 con 0x7f97ac140ff0 2024-04-25T10:18:36.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.008+0000 7f97a9e00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f978c066f60 0x7f978c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.008+0000 7f97a9e00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f978c066f60 0x7f978c069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f979c009ad0 tx=0x7f979c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.012+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97980562e0 con 0x7f97ac140ff0 2024-04-25T10:18:36.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97ac066c80 con 0x7f97ac140ff0 2024-04-25T10:18:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97a2000640 1 -- 172.21.15.45:0/927310999 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9798055be0 con 0x7f97ac140ff0 2024-04-25T10:18:36.179 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:36.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f978c066f60 msgr2=0x7f978c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f978c066f60 0x7f978c069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f979c009ad0 tx=0x7f979c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 msgr2=0x7f97ac0b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 0x7f97ac0b1ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9798007340 tx=0x7f9798004f90 comp rx=0 tx=0).stop 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 shutdown_connections 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f978c066f60 0x7f978c069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97ac142b60 0x7f97ac06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97ac140ff0 0x7f97ac0b1ee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 --2- 172.21.15.45:0/927310999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97ac140c10 0x7f97ac1414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 >> 172.21.15.45:0/927310999 conn(0x7f97ac06ac90 msgr2=0x7f97ac06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 shutdown_connections 2024-04-25T10:18:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.176+0000 7f97b0a00640 1 -- 172.21.15.45:0/927310999 wait complete. 2024-04-25T10:18:36.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:36.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 Processor -- start 2024-04-25T10:18:36.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 -- start start 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3280cd9a0 0x7fc3280d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32806bd10 0x7fc328161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3280d0ba0 0x7fc328161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32806b880 con 0x7fc3280d0ba0 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32806ac90 con 0x7fc3280cd9a0 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc32ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32806af50 con 0x7fc32806bd10 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc326800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3280d0ba0 0x7fc328161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc326800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3280d0ba0 0x7fc328161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52346/0 (socket says 172.21.15.45:52346) 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc326800640 1 -- 172.21.15.45:0/1538874515 learned_addr learned my addr 172.21.15.45:0/1538874515 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc325e00640 1 --2- 172.21.15.45:0/1538874515 >> v2:172.21.15.117:3300/0 conn(0x7fc3280cd9a0 0x7fc3280d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.340+0000 7fc31d400640 1 --2- 172.21.15.45:0/1538874515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32806bd10 0x7fc328161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc326800640 1 -- 172.21.15.45:0/1538874515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32806bd10 msgr2=0x7fc328161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc326800640 1 --2- 172.21.15.45:0/1538874515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32806bd10 0x7fc328161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc326800640 1 -- 172.21.15.45:0/1538874515 >> v2:172.21.15.117:3300/0 conn(0x7fc3280cd9a0 msgr2=0x7fc3280d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc326800640 1 --2- 172.21.15.45:0/1538874515 >> v2:172.21.15.117:3300/0 conn(0x7fc3280cd9a0 0x7fc3280d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc326800640 1 -- 172.21.15.45:0/1538874515 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32806b230 con 0x7fc3280d0ba0 2024-04-25T10:18:36.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc326800640 1 --2- 172.21.15.45:0/1538874515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280d0ba0 0x7fc328161e50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cf4a0 tx=0x7fc32001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81499f070baadd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc325400640 1 -- 172.21.15.45:0/1538874515 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200b4070 con 0x7fc3280d0ba0 2024-04-25T10:18:36.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc325400640 1 -- 172.21.15.45:0/1538874515 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3200af400 con 0x7fc3280d0ba0 2024-04-25T10:18:36.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc325400640 1 -- 172.21.15.45:0/1538874515 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200af7d0 con 0x7fc3280d0ba0 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc32ca00640 1 -- 172.21.15.45:0/1538874515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280d0ba0 msgr2=0x7fc328161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.344+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1538874515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280d0ba0 0x7fc328161e50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cf4a0 tx=0x7fc32001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- 172.21.15.45:0/1538874515 shutdown_connections 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1538874515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280d0ba0 0x7fc328161e50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1538874515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32806bd10 0x7fc328161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1538874515 >> v2:172.21.15.117:3300/0 conn(0x7fc3280cd9a0 0x7fc3280d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- 172.21.15.45:0/1538874515 >> 172.21.15.45:0/1538874515 conn(0x7fc3280d19a0 msgr2=0x7fc3280d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- 172.21.15.45:0/1538874515 shutdown_connections 2024-04-25T10:18:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- 172.21.15.45:0/1538874515 wait complete. 2024-04-25T10:18:36.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 Processor -- start 2024-04-25T10:18:36.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- start start 2024-04-25T10:18:36.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32806bd10 0x7fc3280c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 0x7fc32806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3280d0ba0 0x7fc3280b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc328162660 con 0x7fc3280cd9a0 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280cdd80 con 0x7fc32806bd10 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc32ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32815d610 con 0x7fc3280d0ba0 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 0x7fc32806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 0x7fc32806ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52358/0 (socket says 172.21.15.45:52358) 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 -- 172.21.15.45:0/1035602628 learned_addr learned my addr 172.21.15.45:0/1035602628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc326800640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3280d0ba0 0x7fc3280b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc325e00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32806bd10 0x7fc3280c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 -- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3280d0ba0 msgr2=0x7fc3280b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3280d0ba0 0x7fc3280b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 -- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32806bd10 msgr2=0x7fc3280c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32806bd10 0x7fc3280c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 -- 172.21.15.45:0/1035602628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280b58a0 con 0x7fc3280cd9a0 2024-04-25T10:18:36.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc326800640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3280d0ba0 0x7fc3280b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc325e00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32806bd10 0x7fc3280c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31d400640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 0x7fc32806ef20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc31000e8b0 tx=0x7fc31000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310017070 con 0x7fc3280cd9a0 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.348+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31000c940 con 0x7fc3280cd9a0 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000cd10 con 0x7fc3280cd9a0 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32815a770 con 0x7fc3280cd9a0 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3280b5df0 con 0x7fc3280cd9a0 2024-04-25T10:18:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32804ef30 con 0x7fc3280cd9a0 2024-04-25T10:18:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc31001a070 con 0x7fc3280cd9a0 2024-04-25T10:18:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc31ea00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc304066f60 0x7fc304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3100850a0 con 0x7fc3280cd9a0 2024-04-25T10:18:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc325e00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc304066f60 0x7fc304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.352+0000 7fc325e00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc304066f60 0x7fc304069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc318009ae0 tx=0x7fc318008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.356+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc310056010 con 0x7fc3280cd9a0 2024-04-25T10:18:36.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc328070070 con 0x7fc3280cd9a0 2024-04-25T10:18:36.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc31ea00640 1 -- 172.21.15.45:0/1035602628 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc310055910 con 0x7fc3280cd9a0 2024-04-25T10:18:36.523 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc304066f60 msgr2=0x7fc304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc304066f60 0x7fc304069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc318009ae0 tx=0x7fc318008040 comp rx=0 tx=0).stop 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 msgr2=0x7fc32806ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 0x7fc32806ef20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc31000e8b0 tx=0x7fc31000ed70 comp rx=0 tx=0).stop 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 shutdown_connections 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc304066f60 0x7fc304069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3280d0ba0 0x7fc3280b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3280cd9a0 0x7fc32806ef20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 --2- 172.21.15.45:0/1035602628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32806bd10 0x7fc3280c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 >> 172.21.15.45:0/1035602628 conn(0x7fc3280d19a0 msgr2=0x7fc3280ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 shutdown_connections 2024-04-25T10:18:36.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.520+0000 7fc32ca00640 1 -- 172.21.15.45:0/1035602628 wait complete. 2024-04-25T10:18:36.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682c000640 1 Processor -- start 2024-04-25T10:18:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682c000640 1 -- start start 2024-04-25T10:18:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682c000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6824159540 0x7f68240ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682c000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6824159920 con 0x7f68240721f0 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682c000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682415c2f0 con 0x7f68240aebd0 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682c000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68240af030 con 0x7f6824159540 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682a000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68240721f0 0x7f6824070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38300/0 (socket says 172.21.15.45:38300) 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f682a000640 1 -- 172.21.15.45:0/3600628903 learned_addr learned my addr 172.21.15.45:0/3600628903 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f6829600640 1 --2- 172.21.15.45:0/3600628903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6824159540 0x7f68240ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f6823e00640 1 -- 172.21.15.45:0/3600628903 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833818366 0 0) 0x7f6824159920 con 0x7f68240721f0 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f6823e00640 1 -- 172.21.15.45:0/3600628903 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f680c003530 con 0x7f68240721f0 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f6829600640 1 -- 172.21.15.45:0/3600628903 >> v1:172.21.15.117:6789/0 conn(0x7f68240aebd0 legacy=0x7f6824071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f6829600640 1 -- 172.21.15.45:0/3600628903 >> v1:172.21.15.45:6789/0 conn(0x7f68240721f0 legacy=0x7f6824070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.684+0000 7f6829600640 1 -- 172.21.15.45:0/3600628903 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682415c5d0 con 0x7f6824159540 2024-04-25T10:18:36.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f6829600640 1 --2- 172.21.15.45:0/3600628903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6824159540 0x7f68240ae690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68180097f0 tx=0x7f681801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51107127d8a98750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f6823e00640 1 -- 172.21.15.45:0/3600628903 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68180aa610 con 0x7f6824159540 2024-04-25T10:18:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f6823e00640 1 -- 172.21.15.45:0/3600628903 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68180af070 con 0x7f6824159540 2024-04-25T10:18:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f6823e00640 1 -- 172.21.15.45:0/3600628903 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68180aa9b0 con 0x7f6824159540 2024-04-25T10:18:36.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 -- 172.21.15.45:0/3600628903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6824159540 msgr2=0x7f68240ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 --2- 172.21.15.45:0/3600628903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6824159540 0x7f68240ae690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68180097f0 tx=0x7f681801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 -- 172.21.15.45:0/3600628903 shutdown_connections 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 --2- 172.21.15.45:0/3600628903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6824159540 0x7f68240ae690 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 -- 172.21.15.45:0/3600628903 >> 172.21.15.45:0/3600628903 conn(0x7f68240669a0 msgr2=0x7f6824066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 -- 172.21.15.45:0/3600628903 shutdown_connections 2024-04-25T10:18:36.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.688+0000 7f682c000640 1 -- 172.21.15.45:0/3600628903 wait complete. 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 Processor -- start 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 -- start start 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68240721f0 0x7f68240b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68240aebd0 0x7f68240b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 0x7f682414c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824169940 con 0x7f6824159540 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682406c0d0 con 0x7f68240aebd0 2024-04-25T10:18:36.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824169ac0 con 0x7f68240721f0 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 0x7f682414c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 0x7f682414c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52364/0 (socket says 172.21.15.45:52364) 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 -- 172.21.15.45:0/2194694376 learned_addr learned my addr 172.21.15.45:0/2194694376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f6829600640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68240721f0 0x7f68240b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f6828c00640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68240aebd0 0x7f68240b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 -- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68240721f0 msgr2=0x7f68240b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68240721f0 0x7f68240b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 -- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68240aebd0 msgr2=0x7f68240b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68240aebd0 0x7f68240b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 -- 172.21.15.45:0/2194694376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68240b0420 con 0x7f6824159540 2024-04-25T10:18:36.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682a000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 0x7f682414c830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f681c00cb70 tx=0x7f681c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c017070 con 0x7f6824159540 2024-04-25T10:18:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00ad40 con 0x7f6824159540 2024-04-25T10:18:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c012620 con 0x7f6824159540 2024-04-25T10:18:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68240afc40 con 0x7f6824159540 2024-04-25T10:18:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.692+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68241585b0 con 0x7f6824159540 2024-04-25T10:18:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.696+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68240b2e50 con 0x7f6824159540 2024-04-25T10:18:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.696+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f681c01a070 con 0x7f6824159540 2024-04-25T10:18:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.696+0000 7f6821600640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6804066f90 0x7f6804069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.696+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f681c084e30 con 0x7f6824159540 2024-04-25T10:18:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.696+0000 7f6829600640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6804066f90 0x7f6804069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.696+0000 7f6829600640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6804066f90 0x7f6804069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68180ae000 tx=0x7f68180a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.700+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68240b2e50 con 0x7f6824159540 2024-04-25T10:18:36.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.860+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f682404ef30 con 0x7f6824159540 2024-04-25T10:18:36.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.860+0000 7f6821600640 1 -- 172.21.15.45:0/2194694376 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f681c055750 con 0x7f6824159540 2024-04-25T10:18:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:36.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6804066f90 msgr2=0x7f6804069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6804066f90 0x7f6804069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68180ae000 tx=0x7f68180a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:36.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 msgr2=0x7f682414c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 0x7f682414c830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f681c00cb70 tx=0x7f681c00a430 comp rx=0 tx=0).stop 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 shutdown_connections 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6804066f90 0x7f6804069450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6824159540 0x7f682414c830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68240aebd0 0x7f68240b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 --2- 172.21.15.45:0/2194694376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68240721f0 0x7f68240b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 >> 172.21.15.45:0/2194694376 conn(0x7f68240669a0 msgr2=0x7f68240cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 shutdown_connections 2024-04-25T10:18:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:36.864+0000 7f682c000640 1 -- 172.21.15.45:0/2194694376 wait complete. 2024-04-25T10:18:36.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 Processor -- start 2024-04-25T10:18:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 -- start start 2024-04-25T10:18:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7b00cda10 0x7fe7b006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b00cf930 0x7fe7b00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b00b2770 con 0x7fe7b015bc40 2024-04-25T10:18:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b006c670 con 0x7fe7b00cda10 2024-04-25T10:18:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7b5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b006c950 con 0x7fe7b00cf930 2024-04-25T10:18:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7aea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe7b015bc40 0x7fe7b00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38314/0 (socket says 172.21.15.45:38314) 2024-04-25T10:18:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7aea00640 1 -- 172.21.15.45:0/1804877197 learned_addr learned my addr 172.21.15.45:0/1804877197 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7a7e00640 1 --2- 172.21.15.45:0/1804877197 >> v2:172.21.15.117:3300/0 conn(0x7fe7b00cda10 0x7fe7b006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7af400640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b00cf930 0x7fe7b00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7ae000640 1 -- 172.21.15.45:0/1804877197 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134957215 0 0) 0x7fe7b00b2770 con 0x7fe7b015bc40 2024-04-25T10:18:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7ae000640 1 -- 172.21.15.45:0/1804877197 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe794003530 con 0x7fe7b015bc40 2024-04-25T10:18:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7a7e00640 1 -- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b00cf930 msgr2=0x7fe7b00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7a7e00640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b00cf930 0x7fe7b00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7a7e00640 1 -- 172.21.15.45:0/1804877197 >> v1:172.21.15.45:6789/0 conn(0x7fe7b015bc40 legacy=0x7fe7b00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7a7e00640 1 -- 172.21.15.45:0/1804877197 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b006cc30 con 0x7fe7b00cda10 2024-04-25T10:18:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.028+0000 7fe7af400640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b00cf930 0x7fe7b00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:37.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7a7e00640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cda10 0x7fe7b006c090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7980097c0 tx=0x7fe79801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=945a44d693540d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7ae000640 1 -- 172.21.15.45:0/1804877197 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980b0070 con 0x7fe7b00cda10 2024-04-25T10:18:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7ae000640 1 -- 172.21.15.45:0/1804877197 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7980ab600 con 0x7fe7b00cda10 2024-04-25T10:18:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7ae000640 1 -- 172.21.15.45:0/1804877197 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980ab9d0 con 0x7fe7b00cda10 2024-04-25T10:18:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 -- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cda10 msgr2=0x7fe7b006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cda10 0x7fe7b006c090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7980097c0 tx=0x7fe79801c410 comp rx=0 tx=0).stop 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 -- 172.21.15.45:0/1804877197 shutdown_connections 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b00cf930 0x7fe7b00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 --2- 172.21.15.45:0/1804877197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cda10 0x7fe7b006c090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 -- 172.21.15.45:0/1804877197 >> 172.21.15.45:0/1804877197 conn(0x7fe7b0066b70 msgr2=0x7fe7b0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 -- 172.21.15.45:0/1804877197 shutdown_connections 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 -- 172.21.15.45:0/1804877197 wait complete. 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.032+0000 7fe7b5800640 1 Processor -- start 2024-04-25T10:18:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- start start 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 0x7fe7b015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cf930 0x7fe7b01f9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b015bc40 0x7fe7b015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00b3890 con 0x7fe7b00cda10 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00b3710 con 0x7fe7b00cf930 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00b3a10 con 0x7fe7b015bc40 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 0x7fe7b015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 0x7fe7b015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52378/0 (socket says 172.21.15.45:52378) 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 -- 172.21.15.45:0/3018503524 learned_addr learned my addr 172.21.15.45:0/3018503524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7a7e00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cf930 0x7fe7b01f9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7af400640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b015bc40 0x7fe7b015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 -- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b015bc40 msgr2=0x7fe7b015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b015bc40 0x7fe7b015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 -- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cf930 msgr2=0x7fe7b01f9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cf930 0x7fe7b01f9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 -- 172.21.15.45:0/3018503524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b015ee70 con 0x7fe7b00cda10 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7aea00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 0x7fe7b015e930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c3f0 tx=0x7fe7a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017070 con 0x7fe7b00cda10 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a0005cf0 con 0x7fe7b00cda10 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00072f0 con 0x7fe7b00cda10 2024-04-25T10:18:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b01fa1a0 con 0x7fe7b00cda10 2024-04-25T10:18:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b015da60 con 0x7fe7b00cda10 2024-04-25T10:18:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.036+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b004ef30 con 0x7fe7b00cda10 2024-04-25T10:18:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.040+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7a001a070 con 0x7fe7b00cda10 2024-04-25T10:18:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.040+0000 7fe7a6a00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe78c066f90 0x7fe78c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.040+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7a0084240 con 0x7fe7b00cda10 2024-04-25T10:18:37.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.040+0000 7fe7a7e00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe78c066f90 0x7fe78c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.040+0000 7fe7a7e00640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe78c066f90 0x7fe78c069450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe7980ad000 tx=0x7fe798008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.044+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7a0054c90 con 0x7fe7b00cda10 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe7b006d960 con 0x7fe7b00cda10 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7a6a00640 1 -- 172.21.15.45:0/3018503524 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe7a0058d30 con 0x7fe7b00cda10 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe78c066f90 msgr2=0x7fe78c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe78c066f90 0x7fe78c069450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe7980ad000 tx=0x7fe798008040 comp rx=0 tx=0).stop 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 msgr2=0x7fe7b015e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 0x7fe7b015e930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c3f0 tx=0x7fe7a000c7f0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 shutdown_connections 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe78c066f90 0x7fe78c069450 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7b015bc40 0x7fe7b015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7b00cf930 0x7fe7b01f9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 --2- 172.21.15.45:0/3018503524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7b00cda10 0x7fe7b015e930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 >> 172.21.15.45:0/3018503524 conn(0x7fe7b0066b70 msgr2=0x7fe7b0155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 shutdown_connections 2024-04-25T10:18:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.220+0000 7fe7b5800640 1 -- 172.21.15.45:0/3018503524 wait complete. 2024-04-25T10:18:37.244 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:37.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:18:37.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:18:37.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:37.281 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:18:37.338 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:37.338 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:37.347 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:37.347 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:37.398 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:18:37.398 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:37.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.632+0000 7f5f24e00640 1 Processor -- start 2024-04-25T10:18:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.632+0000 7f5f24e00640 1 -- start start 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f24e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f200cf930 0x7f5f200cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f24e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f20070c20 0x7f5f2015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f24e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200d0250 con 0x7f5f200ae020 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f24e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2015bb90 con 0x7f5f200cf930 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f24e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200ad840 con 0x7f5f20070c20 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f200cf930 0x7f5f200cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f200cf930 0x7f5f200cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52386/0 (socket says 172.21.15.45:52386) 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 -- 172.21.15.45:0/3062635405 learned_addr learned my addr 172.21.15.45:0/3062635405 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f17e00640 1 --2- 172.21.15.45:0/3062635405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f20070c20 0x7f5f2015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 -- 172.21.15.45:0/3062635405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f20070c20 msgr2=0x7f5f2015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 --2- 172.21.15.45:0/3062635405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f20070c20 0x7f5f2015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 -- 172.21.15.45:0/3062635405 >> v1:172.21.15.117:6789/0 conn(0x7f5f200ae020 legacy=0x7f5f200b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.636+0000 7f5f1de00640 1 -- 172.21.15.45:0/3062635405 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2015e9f0 con 0x7f5f200cf930 2024-04-25T10:18:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f1de00640 1 --2- 172.21.15.45:0/3062635405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f200cfd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf4a47764e26806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f1d400640 1 -- 172.21.15.45:0/3062635405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100a95f0 con 0x7f5f200cf930 2024-04-25T10:18:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f1d400640 1 -- 172.21.15.45:0/3062635405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f100a9790 con 0x7f5f200cf930 2024-04-25T10:18:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f1d400640 1 -- 172.21.15.45:0/3062635405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100a9b60 con 0x7f5f200cf930 2024-04-25T10:18:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 -- 172.21.15.45:0/3062635405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 msgr2=0x7f5f200cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 --2- 172.21.15.45:0/3062635405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f200cfd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1001c530 comp rx=0 tx=0).stop 2024-04-25T10:18:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 -- 172.21.15.45:0/3062635405 shutdown_connections 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 --2- 172.21.15.45:0/3062635405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f20070c20 0x7f5f2015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 --2- 172.21.15.45:0/3062635405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f200cfd10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 -- 172.21.15.45:0/3062635405 >> 172.21.15.45:0/3062635405 conn(0x7f5f20066b70 msgr2=0x7f5f20066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 -- 172.21.15.45:0/3062635405 shutdown_connections 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 -- 172.21.15.45:0/3062635405 wait complete. 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 Processor -- start 2024-04-25T10:18:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.640+0000 7f5f24e00640 1 -- start start 2024-04-25T10:18:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f20070c20 0x7f5f20159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f200ae020 0x7f5f200b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f20147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2006e130 con 0x7f5f200cf930 2024-04-25T10:18:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200ae840 con 0x7f5f20070c20 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2015e6b0 con 0x7f5f200ae020 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f20147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f20070c20 0x7f5f20159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f20147d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52394/0 (socket says 172.21.15.45:52394) 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 -- 172.21.15.45:0/1442472978 learned_addr learned my addr 172.21.15.45:0/1442472978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f17e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f200ae020 0x7f5f200b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 -- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f200ae020 msgr2=0x7f5f200b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f200ae020 0x7f5f200b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 -- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f20070c20 msgr2=0x7f5f20159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f20070c20 0x7f5f20159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 -- 172.21.15.45:0/1442472978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200cc2b0 con 0x7f5f200cf930 2024-04-25T10:18:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f1e800640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f20147d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c770 tx=0x7f5f1800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003b80 con 0x7f5f200cf930 2024-04-25T10:18:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18003d20 con 0x7f5f200cf930 2024-04-25T10:18:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800aa90 con 0x7f5f200cf930 2024-04-25T10:18:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200cc590 con 0x7f5f200cf930 2024-04-25T10:18:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200ccb50 con 0x7f5f200cf930 2024-04-25T10:18:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2004ef30 con 0x7f5f200cf930 2024-04-25T10:18:37.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.644+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f1800ac30 con 0x7f5f200cf930 2024-04-25T10:18:37.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.648+0000 7f5f16000640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5efc066f60 0x7f5efc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.648+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f18084ec0 con 0x7f5f200cf930 2024-04-25T10:18:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.648+0000 7f5f1de00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5efc066f60 0x7f5efc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.648+0000 7f5f1de00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5efc066f60 0x7f5efc069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f100ae000 tx=0x7f5f100b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.652+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f180557e0 con 0x7f5f200cf930 2024-04-25T10:18:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.812+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f20068cc0 con 0x7f5f200cf930 2024-04-25T10:18:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.812+0000 7f5f16000640 1 -- 172.21.15.45:0/1442472978 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f5f18059880 con 0x7f5f200cf930 2024-04-25T10:18:37.816 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5efc066f60 msgr2=0x7f5efc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5efc066f60 0x7f5efc069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5f100ae000 tx=0x7f5f100b3040 comp rx=0 tx=0).stop 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 msgr2=0x7f5f20147d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f20147d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c770 tx=0x7f5f1800cc30 comp rx=0 tx=0).stop 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 shutdown_connections 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5efc066f60 0x7f5efc069420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f200cf930 0x7f5f20147d80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f200ae020 0x7f5f200b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 --2- 172.21.15.45:0/1442472978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f20070c20 0x7f5f20159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 >> 172.21.15.45:0/1442472978 conn(0x7f5f20066b70 msgr2=0x7f5f20162010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 shutdown_connections 2024-04-25T10:18:37.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.816+0000 7f5f24e00640 1 -- 172.21.15.45:0/1442472978 wait complete. 2024-04-25T10:18:37.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:37.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.976+0000 7f09d4e00640 1 Processor -- start 2024-04-25T10:18:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.976+0000 7f09d4e00640 1 -- start start 2024-04-25T10:18:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09d4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09d00cd9a0 0x7f09d00d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09d4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d006bd10 0x7f09d0161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09d4e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d006ac90 con 0x7f09d00d0ba0 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09d4e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d006b880 con 0x7f09d00cd9a0 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09d4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d006af50 con 0x7f09d006bd10 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09cde00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09d00cd9a0 0x7f09d00d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09ce800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f09d00d0ba0 0x7f09d015d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38318/0 (socket says 172.21.15.45:38318) 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09ce800640 1 -- 172.21.15.45:0/1760126404 learned_addr learned my addr 172.21.15.45:0/1760126404 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09c7e00640 1 --2- 172.21.15.45:0/1760126404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d006bd10 0x7f09d0161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09cde00640 1 -- 172.21.15.45:0/1760126404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d006bd10 msgr2=0x7f09d0161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09cde00640 1 --2- 172.21.15.45:0/1760126404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d006bd10 0x7f09d0161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09cde00640 1 -- 172.21.15.45:0/1760126404 >> v1:172.21.15.45:6789/0 conn(0x7f09d00d0ba0 legacy=0x7f09d015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.980+0000 7f09cde00640 1 -- 172.21.15.45:0/1760126404 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d006b230 con 0x7f09d00cd9a0 2024-04-25T10:18:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09cde00640 1 --2- 172.21.15.45:0/1760126404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d00cd9a0 0x7f09d00d0660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09c00097c0 tx=0x7f09c001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb4a458ba517f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09cd400640 1 -- 172.21.15.45:0/1760126404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00b5070 con 0x7f09d00cd9a0 2024-04-25T10:18:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09cd400640 1 -- 172.21.15.45:0/1760126404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c00a8990 con 0x7f09d00cd9a0 2024-04-25T10:18:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09cd400640 1 -- 172.21.15.45:0/1760126404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00a8d60 con 0x7f09d00cd9a0 2024-04-25T10:18:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 -- 172.21.15.45:0/1760126404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d00cd9a0 msgr2=0x7f09d00d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 --2- 172.21.15.45:0/1760126404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d00cd9a0 0x7f09d00d0660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f09c00097c0 tx=0x7f09c001ec90 comp rx=0 tx=0).stop 2024-04-25T10:18:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 -- 172.21.15.45:0/1760126404 shutdown_connections 2024-04-25T10:18:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 --2- 172.21.15.45:0/1760126404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d006bd10 0x7f09d0161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 --2- 172.21.15.45:0/1760126404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d00cd9a0 0x7f09d00d0660 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 -- 172.21.15.45:0/1760126404 >> 172.21.15.45:0/1760126404 conn(0x7f09d00d19a0 msgr2=0x7f09d00d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 -- 172.21.15.45:0/1760126404 shutdown_connections 2024-04-25T10:18:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 -- 172.21.15.45:0/1760126404 wait complete. 2024-04-25T10:18:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 Processor -- start 2024-04-25T10:18:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 -- start start 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 0x7f09d00c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.984+0000 7f09d4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d00cd9a0 0x7f09d01f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d00d0ba0 0x7f09d014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0162660 con 0x7f09d006bd10 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00cdd80 con 0x7f09d00d0ba0 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0161e50 con 0x7f09d00cd9a0 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 0x7f09d00c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 0x7f09d00c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52406/0 (socket says 172.21.15.45:52406) 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 -- 172.21.15.45:0/2763267452 learned_addr learned my addr 172.21.15.45:0/2763267452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09ce800640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d00d0ba0 0x7f09d014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09c7e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d00cd9a0 0x7f09d01f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 -- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d00cd9a0 msgr2=0x7f09d01f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d00cd9a0 0x7f09d01f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 -- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d00d0ba0 msgr2=0x7f09d014dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d00d0ba0 0x7f09d014dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 -- 172.21.15.45:0/2763267452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00680c0 con 0x7f09d006bd10 2024-04-25T10:18:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09ce800640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d00d0ba0 0x7f09d014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09cde00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 0x7f09d00c7ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f09c001c8c0 tx=0x7f09c001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00b5040 con 0x7f09d006bd10 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c00c0070 con 0x7f09d006bd10 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00b09a0 con 0x7f09d006bd10 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d00c56b0 con 0x7f09d006bd10 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d01f4460 con 0x7f09d006bd10 2024-04-25T10:18:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d004ef30 con 0x7f09d006bd10 2024-04-25T10:18:37.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.988+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09c00b0b40 con 0x7f09d006bd10 2024-04-25T10:18:37.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.992+0000 7f09c6000640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f09ac066f60 0x7f09ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.992+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c0122560 con 0x7f09d006bd10 2024-04-25T10:18:37.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.992+0000 7f09c7e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f09ac066f60 0x7f09ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.992+0000 7f09c7e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f09ac066f60 0x7f09ac069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09d00c5ae0 tx=0x7f09b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:37.996+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09c00187f0 con 0x7f09d006bd10 2024-04-25T10:18:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.156+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09d00b7920 con 0x7f09d006bd10 2024-04-25T10:18:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.156+0000 7f09c6000640 1 -- 172.21.15.45:0/2763267452 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09c00f3560 con 0x7f09d006bd10 2024-04-25T10:18:38.161 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f09ac066f60 msgr2=0x7f09ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f09ac066f60 0x7f09ac069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09d00c5ae0 tx=0x7f09b8008040 comp rx=0 tx=0).stop 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 msgr2=0x7f09d00c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 0x7f09d00c7ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f09c001c8c0 tx=0x7f09c001ba30 comp rx=0 tx=0).stop 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 shutdown_connections 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f09ac066f60 0x7f09ac069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09d00d0ba0 0x7f09d014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09d00cd9a0 0x7f09d01f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 --2- 172.21.15.45:0/2763267452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09d006bd10 0x7f09d00c7ce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 >> 172.21.15.45:0/2763267452 conn(0x7f09d00d19a0 msgr2=0x7f09d00cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 shutdown_connections 2024-04-25T10:18:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.160+0000 7f09d4e00640 1 -- 172.21.15.45:0/2763267452 wait complete. 2024-04-25T10:18:38.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.320+0000 7f84bd200640 1 Processor -- start 2024-04-25T10:18:38.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.320+0000 7f84bd200640 1 -- start start 2024-04-25T10:18:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84bd200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b80aeda0 0x7f84b8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84bd200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84b8072020 0x7f84b8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84bd200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b8159920 con 0x7f84b8159540 2024-04-25T10:18:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84bd200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b815c2f0 con 0x7f84b8072020 2024-04-25T10:18:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84bd200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b815c5d0 con 0x7f84b80aeda0 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84b8072020 0x7f84b8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84afe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b80aeda0 0x7f84b8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84b8072020 0x7f84b8070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33612/0 (socket says 172.21.15.45:33612) 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 -- 172.21.15.45:0/1117341333 learned_addr learned my addr 172.21.15.45:0/1117341333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 -- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b80aeda0 msgr2=0x7f84b8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 --2- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b80aeda0 0x7f84b8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 -- 172.21.15.45:0/1117341333 >> v1:172.21.15.117:6789/0 conn(0x7f84b8159540 legacy=0x7f84b80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.324+0000 7f84b6800640 1 -- 172.21.15.45:0/1117341333 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b815c8b0 con 0x7f84b8072020 2024-04-25T10:18:38.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84b6800640 1 --2- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8070350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009e50 tx=0x7f84b001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d44f1aa7d98dcee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84b5400640 1 -- 172.21.15.45:0/1117341333 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b00a58f0 con 0x7f84b8072020 2024-04-25T10:18:38.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84b5400640 1 -- 172.21.15.45:0/1117341333 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b00a5a90 con 0x7f84b8072020 2024-04-25T10:18:38.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84b5400640 1 -- 172.21.15.45:0/1117341333 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b00a5e60 con 0x7f84b8072020 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 -- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 msgr2=0x7f84b8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 --2- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8070350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009e50 tx=0x7f84b001ca00 comp rx=0 tx=0).stop 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 -- 172.21.15.45:0/1117341333 shutdown_connections 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 --2- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8070350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 --2- 172.21.15.45:0/1117341333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b80aeda0 0x7f84b8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 -- 172.21.15.45:0/1117341333 >> 172.21.15.45:0/1117341333 conn(0x7f84b80669a0 msgr2=0x7f84b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 -- 172.21.15.45:0/1117341333 shutdown_connections 2024-04-25T10:18:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 -- 172.21.15.45:0/1117341333 wait complete. 2024-04-25T10:18:38.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 Processor -- start 2024-04-25T10:18:38.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.328+0000 7f84bd200640 1 -- start start 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 0x7f84b813fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b8159540 0x7f84b81544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80710a0 con 0x7f84b80aeda0 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b806c0d0 con 0x7f84b8072020 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8071220 con 0x7f84b8159540 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 0x7f84b813fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 0x7f84b813fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52416/0 (socket says 172.21.15.45:52416) 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 -- 172.21.15.45:0/2657150385 learned_addr learned my addr 172.21.15.45:0/2657150385 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84b6800640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b8159540 0x7f84b81544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84b5e00640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 -- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b8159540 msgr2=0x7f84b81544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b8159540 0x7f84b81544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 -- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 msgr2=0x7f84b8152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 -- 172.21.15.45:0/2657150385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b814e610 con 0x7f84b80aeda0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84afe00640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 0x7f84b813fdd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f84a000e930 tx=0x7f84a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0017070 con 0x7f84b80aeda0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a000ca00 con 0x7f84b80aeda0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a000cdb0 con 0x7f84b80aeda0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8151060 con 0x7f84b80aeda0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b81549f0 con 0x7f84b80aeda0 2024-04-25T10:18:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.332+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b804ef30 con 0x7f84b80aeda0 2024-04-25T10:18:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.336+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84a001a070 con 0x7f84b80aeda0 2024-04-25T10:18:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.336+0000 7f84ae000640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8494066f60 0x7f8494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.336+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a0085370 con 0x7f84b80aeda0 2024-04-25T10:18:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.336+0000 7f84b5e00640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8494066f60 0x7f8494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.336+0000 7f84b5e00640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8494066f60 0x7f8494069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001810 tx=0x7f84a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.340+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84a0055de0 con 0x7f84b80aeda0 2024-04-25T10:18:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.500+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84b8166700 con 0x7f84b80aeda0 2024-04-25T10:18:38.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.500+0000 7f84ae000640 1 -- 172.21.15.45:0/2657150385 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84a0059e80 con 0x7f84b80aeda0 2024-04-25T10:18:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8494066f60 msgr2=0x7f8494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8494066f60 0x7f8494069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001810 tx=0x7f84a8009040 comp rx=0 tx=0).stop 2024-04-25T10:18:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 msgr2=0x7f84b813fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 0x7f84b813fdd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f84a000e930 tx=0x7f84a000edf0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 shutdown_connections 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8494066f60 0x7f8494069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84b8159540 0x7f84b81544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84b80aeda0 0x7f84b813fdd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 --2- 172.21.15.45:0/2657150385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84b8072020 0x7f84b8152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 >> 172.21.15.45:0/2657150385 conn(0x7f84b80669a0 msgr2=0x7f84b8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 shutdown_connections 2024-04-25T10:18:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.504+0000 7f84bd200640 1 -- 172.21.15.45:0/2657150385 wait complete. 2024-04-25T10:18:38.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:38.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.664+0000 7ff229600640 1 Processor -- start 2024-04-25T10:18:38.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.664+0000 7ff229600640 1 -- start start 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff229600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff22406bd10 0x7ff224161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff229600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff224161e50 0x7ff22415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff229600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22406ac90 con 0x7ff2240cd9a0 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff229600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2241622a0 con 0x7ff22406bd10 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff229600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22406b880 con 0x7ff224161e50 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff22406bd10 0x7ff224161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff22406bd10 0x7ff224161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52422/0 (socket says 172.21.15.45:52422) 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff222a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2240cd9a0 0x7ff2240d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38322/0 (socket says 172.21.15.45:38322) 2024-04-25T10:18:38.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 -- 172.21.15.45:0/3424170381 learned_addr learned my addr 172.21.15.45:0/3424170381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:38.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 -- 172.21.15.45:0/3424170381 >> v2:172.21.15.117:3300/0 conn(0x7ff224161e50 msgr2=0x7ff22415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff223400640 1 --2- 172.21.15.45:0/3424170381 >> v2:172.21.15.117:3300/0 conn(0x7ff224161e50 0x7ff22415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 --2- 172.21.15.45:0/3424170381 >> v2:172.21.15.117:3300/0 conn(0x7ff224161e50 0x7ff22415d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 -- 172.21.15.45:0/3424170381 >> v1:172.21.15.45:6789/0 conn(0x7ff2240cd9a0 legacy=0x7ff2240d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 -- 172.21.15.45:0/3424170381 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22406af50 con 0x7ff22406bd10 2024-04-25T10:18:38.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff223400640 1 --2- 172.21.15.45:0/3424170381 >> v2:172.21.15.117:3300/0 conn(0x7ff224161e50 0x7ff22415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:38.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.668+0000 7ff21be00640 1 --2- 172.21.15.45:0/3424170381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff224161910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff2241629b0 tx=0x7ff20c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28ed03751d7acf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222000640 1 -- 172.21.15.45:0/3424170381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0b5070 con 0x7ff22406bd10 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222000640 1 -- 172.21.15.45:0/3424170381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c0a8a10 con 0x7ff22406bd10 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222000640 1 -- 172.21.15.45:0/3424170381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0a8de0 con 0x7ff22406bd10 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- 172.21.15.45:0/3424170381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 msgr2=0x7ff224161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 --2- 172.21.15.45:0/3424170381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff224161910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff2241629b0 tx=0x7ff20c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- 172.21.15.45:0/3424170381 shutdown_connections 2024-04-25T10:18:38.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 --2- 172.21.15.45:0/3424170381 >> v2:172.21.15.117:3300/0 conn(0x7ff224161e50 0x7ff22415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 --2- 172.21.15.45:0/3424170381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff224161910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- 172.21.15.45:0/3424170381 >> 172.21.15.45:0/3424170381 conn(0x7ff2240d19a0 msgr2=0x7ff2240d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- 172.21.15.45:0/3424170381 shutdown_connections 2024-04-25T10:18:38.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- 172.21.15.45:0/3424170381 wait complete. 2024-04-25T10:18:38.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 Processor -- start 2024-04-25T10:18:38.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- start start 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff22415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2240cd9a0 0x7ff22415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224161e50 0x7ff2240c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240cdd80 con 0x7ff22406bd10 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22406b510 con 0x7ff224161e50 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff229600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff224162660 con 0x7ff2240cd9a0 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff22415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff22415c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52424/0 (socket says 172.21.15.45:52424) 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 -- 172.21.15.45:0/3119354764 learned_addr learned my addr 172.21.15.45:0/3119354764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 -- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2240cd9a0 msgr2=0x7ff22415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2240cd9a0 0x7ff22415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 -- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224161e50 msgr2=0x7ff2240c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224161e50 0x7ff2240c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.672+0000 7ff222a00640 1 -- 172.21.15.45:0/3119354764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22415d730 con 0x7ff22406bd10 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff222a00640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff22415c970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c420 tx=0x7ff21400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140074e0 con 0x7ff22406bd10 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff214007680 con 0x7ff22406bd10 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22415da10 con 0x7ff22406bd10 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214004a30 con 0x7ff22406bd10 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22415dfd0 con 0x7ff22406bd10 2024-04-25T10:18:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22404ef30 con 0x7ff22406bd10 2024-04-25T10:18:38.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff21401a020 con 0x7ff22406bd10 2024-04-25T10:18:38.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21aa00640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff200066f60 0x7ff200069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff214084900 con 0x7ff22406bd10 2024-04-25T10:18:38.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.676+0000 7ff21be00640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff200066f60 0x7ff200069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.680+0000 7ff21be00640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff200066f60 0x7ff200069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0b2000 tx=0x7ff20c004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.680+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff214055870 con 0x7ff22406bd10 2024-04-25T10:18:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2240b6e20 con 0x7ff22406bd10 2024-04-25T10:18:38.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff21aa00640 1 -- 172.21.15.45:0/3119354764 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff214084070 con 0x7ff22406bd10 2024-04-25T10:18:38.847 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff200066f60 msgr2=0x7ff200069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff200066f60 0x7ff200069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0b2000 tx=0x7ff20c004740 comp rx=0 tx=0).stop 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 msgr2=0x7ff22415c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff22415c970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c420 tx=0x7ff21400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 shutdown_connections 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff200066f60 0x7ff200069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224161e50 0x7ff2240c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2240cd9a0 0x7ff22415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 --2- 172.21.15.45:0/3119354764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff22406bd10 0x7ff22415c970 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 >> 172.21.15.45:0/3119354764 conn(0x7ff2240d19a0 msgr2=0x7ff2240cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 shutdown_connections 2024-04-25T10:18:38.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:38.844+0000 7ff229600640 1 -- 172.21.15.45:0/3119354764 wait complete. 2024-04-25T10:18:38.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:39.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.008+0000 7ff5cba00640 1 Processor -- start 2024-04-25T10:18:39.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.008+0000 7ff5cba00640 1 -- start start 2024-04-25T10:18:39.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5cba00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff5c4066c00 0x7ff5c415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5cba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 0x7ff5c40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5cba00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c40b5c20 con 0x7ff5c406ed30 2024-04-25T10:18:39.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5cba00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c40af6e0 con 0x7ff5c4066c00 2024-04-25T10:18:39.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5cba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40af9c0 con 0x7ff5c4066fe0 2024-04-25T10:18:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff5c4066c00 0x7ff5c415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 0x7ff5c40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff5c4066c00 0x7ff5c415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33640/0 (socket says 172.21.15.45:33640) 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 0x7ff5c40b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:51242/0 (socket says 172.21.15.45:51242) 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 -- 172.21.15.45:0/3622974494 learned_addr learned my addr 172.21.15.45:0/3622974494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 -- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 msgr2=0x7ff5c40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 --2- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 0x7ff5c40b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 -- 172.21.15.45:0/3622974494 >> v1:172.21.15.117:6789/0 conn(0x7ff5c406ed30 legacy=0x7ff5c4157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5c9000640 1 -- 172.21.15.45:0/3622974494 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c40afca0 con 0x7ff5c4066c00 2024-04-25T10:18:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.012+0000 7ff5bbe00640 1 --2- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 0x7ff5c40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:39.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5c9000640 1 --2- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c4066c00 0x7ff5c415e110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009870 tx=0x7ff5b401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149bb6aa237c21ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5c3e00640 1 -- 172.21.15.45:0/3622974494 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40a9610 con 0x7ff5c4066c00 2024-04-25T10:18:39.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5c3e00640 1 -- 172.21.15.45:0/3622974494 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40a97b0 con 0x7ff5c4066c00 2024-04-25T10:18:39.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5c3e00640 1 -- 172.21.15.45:0/3622974494 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40a9b80 con 0x7ff5c4066c00 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 -- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c4066c00 msgr2=0x7ff5c415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c4066c00 0x7ff5c415e110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009870 tx=0x7ff5b401c530 comp rx=0 tx=0).stop 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 -- 172.21.15.45:0/3622974494 shutdown_connections 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066fe0 0x7ff5c40b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3622974494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c4066c00 0x7ff5c415e110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 -- 172.21.15.45:0/3622974494 >> 172.21.15.45:0/3622974494 conn(0x7ff5c40b24f0 msgr2=0x7ff5c40b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 -- 172.21.15.45:0/3622974494 shutdown_connections 2024-04-25T10:18:39.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 -- 172.21.15.45:0/3622974494 wait complete. 2024-04-25T10:18:39.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 Processor -- start 2024-04-25T10:18:39.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.016+0000 7ff5cba00640 1 -- start start 2024-04-25T10:18:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066c00 0x7ff5c41471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 0x7ff5c4152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c406ed30 0x7ff5c414ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c406ac90 con 0x7ff5c4066fe0 2024-04-25T10:18:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40af270 con 0x7ff5c406ed30 2024-04-25T10:18:39.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c406ae10 con 0x7ff5c4066c00 2024-04-25T10:18:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 0x7ff5c4152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066c00 0x7ff5c41471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 0x7ff5c4152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52430/0 (socket says 172.21.15.45:52430) 2024-04-25T10:18:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c406ed30 0x7ff5c414ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5bbe00640 1 -- 172.21.15.45:0/3240934319 learned_addr learned my addr 172.21.15.45:0/3240934319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 -- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c406ed30 msgr2=0x7ff5c414ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c406ed30 0x7ff5c414ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 -- 172.21.15.45:0/3240934319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 msgr2=0x7ff5c4152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 0x7ff5c4152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 -- 172.21.15.45:0/3240934319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c41531e0 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5bbe00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 0x7ff5c4152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c9000640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066c00 0x7ff5c41471c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40aa000 tx=0x7ff5b401ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40bc650 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b40bc830 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40bcc20 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c4151c00 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c41521c0 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.020+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c404ef30 con 0x7ff5c4066c00 2024-04-25T10:18:39.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.024+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5b40b13f0 con 0x7ff5c4066c00 2024-04-25T10:18:39.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.024+0000 7ff5c1600640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a0066f60 0x7ff5a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.024+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5b411f330 con 0x7ff5c4066c00 2024-04-25T10:18:39.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.024+0000 7ff5bbe00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a0066f60 0x7ff5a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.024+0000 7ff5bbe00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a0066f60 0x7ff5a0069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5c41519d0 tx=0x7ff5ac009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.028+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5b40c3030 con 0x7ff5c4066c00 2024-04-25T10:18:39.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.200+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff5c4071c70 con 0x7ff5c4066c00 2024-04-25T10:18:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5c1600640 1 -- 172.21.15.45:0/3240934319 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff5b40ad020 con 0x7ff5c4066c00 2024-04-25T10:18:39.206 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:18:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a0066f60 msgr2=0x7ff5a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a0066f60 0x7ff5a0069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5c41519d0 tx=0x7ff5ac009190 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066c00 msgr2=0x7ff5c41471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066c00 0x7ff5c41471c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40aa000 tx=0x7ff5b401ad70 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 shutdown_connections 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a0066f60 0x7ff5a0069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c406ed30 0x7ff5c414ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5c4066fe0 0x7ff5c4152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 --2- 172.21.15.45:0/3240934319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c4066c00 0x7ff5c41471c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 >> 172.21.15.45:0/3240934319 conn(0x7ff5c40b24f0 msgr2=0x7ff5c4154470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 shutdown_connections 2024-04-25T10:18:39.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.204+0000 7ff5cba00640 1 -- 172.21.15.45:0/3240934319 wait complete. 2024-04-25T10:18:39.227 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:18:39.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:39.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:18:39.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:18:39.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:39.252 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:18:39.302 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:39.302 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:39.312 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:39.312 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:39.367 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:18:39.367 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:39.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.556+0000 7fef6b000640 1 Processor -- start 2024-04-25T10:18:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.556+0000 7fef6b000640 1 -- start start 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef6b000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6415bc40 0x7fef6406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef6b000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef640cda10 0x7fef640b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef6b000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640b2770 con 0x7fef640cf930 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef6b000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6406c670 con 0x7fef6415bc40 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef6b000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6406c950 con 0x7fef640cda10 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6415bc40 0x7fef6406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef6415bc40 0x7fef6406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52434/0 (socket says 172.21.15.45:52434) 2024-04-25T10:18:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 -- 172.21.15.45:0/1300737544 learned_addr learned my addr 172.21.15.45:0/1300737544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 -- 172.21.15.45:0/1300737544 >> v2:172.21.15.117:3300/0 conn(0x7fef640cda10 msgr2=0x7fef640b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 --2- 172.21.15.45:0/1300737544 >> v2:172.21.15.117:3300/0 conn(0x7fef640cda10 0x7fef640b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 -- 172.21.15.45:0/1300737544 >> v1:172.21.15.117:6789/0 conn(0x7fef640cf930 legacy=0x7fef6415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 -- 172.21.15.45:0/1300737544 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6406cc30 con 0x7fef6415bc40 2024-04-25T10:18:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.560+0000 7fef63e00640 1 --2- 172.21.15.45:0/1300737544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6415bc40 0x7fef6406c090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef640726f0 tx=0x7fef5401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffb7a3aae4099e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef63400640 1 -- 172.21.15.45:0/1300737544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540ab5f0 con 0x7fef6415bc40 2024-04-25T10:18:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef63400640 1 -- 172.21.15.45:0/1300737544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef540b0070 con 0x7fef6415bc40 2024-04-25T10:18:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef63400640 1 -- 172.21.15.45:0/1300737544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540ab9b0 con 0x7fef6415bc40 2024-04-25T10:18:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- 172.21.15.45:0/1300737544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6415bc40 msgr2=0x7fef6406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 --2- 172.21.15.45:0/1300737544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6415bc40 0x7fef6406c090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef640726f0 tx=0x7fef5401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- 172.21.15.45:0/1300737544 shutdown_connections 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 --2- 172.21.15.45:0/1300737544 >> v2:172.21.15.117:3300/0 conn(0x7fef640cda10 0x7fef640b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 --2- 172.21.15.45:0/1300737544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6415bc40 0x7fef6406c090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- 172.21.15.45:0/1300737544 >> 172.21.15.45:0/1300737544 conn(0x7fef64066b70 msgr2=0x7fef64066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- 172.21.15.45:0/1300737544 shutdown_connections 2024-04-25T10:18:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- 172.21.15.45:0/1300737544 wait complete. 2024-04-25T10:18:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 Processor -- start 2024-04-25T10:18:39.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- start start 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef640cda10 0x7fef6415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 0x7fef640b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef6415bc40 0x7fef640b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef64168b50 con 0x7fef640cf930 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef641689d0 con 0x7fef640cda10 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef6b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef64168cd0 con 0x7fef6415bc40 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 0x7fef640b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 0x7fef640b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52440/0 (socket says 172.21.15.45:52440) 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef5be00640 1 -- 172.21.15.45:0/2265016073 learned_addr learned my addr 172.21.15.45:0/2265016073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef63e00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef640cda10 0x7fef6415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.564+0000 7fef69000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef6415bc40 0x7fef640b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef5be00640 1 -- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef6415bc40 msgr2=0x7fef640b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef5be00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef6415bc40 0x7fef640b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef5be00640 1 -- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef640cda10 msgr2=0x7fef6415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef5be00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef640cda10 0x7fef6415ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef5be00640 1 -- 172.21.15.45:0/2265016073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6415c190 con 0x7fef640cf930 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef63e00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef640cda10 0x7fef6415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef5be00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 0x7fef640b1720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00b440 tx=0x7fef4c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c015020 con 0x7fef640cf930 2024-04-25T10:18:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c0118b0 con 0x7fef640cf930 2024-04-25T10:18:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011ca0 con 0x7fef640cf930 2024-04-25T10:18:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef641439f0 con 0x7fef640cf930 2024-04-25T10:18:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef64143fb0 con 0x7fef640cf930 2024-04-25T10:18:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6404ef30 con 0x7fef640cf930 2024-04-25T10:18:39.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef4c018020 con 0x7fef640cf930 2024-04-25T10:18:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef60c00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef40066f60 0x7fef40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.568+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef4c085170 con 0x7fef640cf930 2024-04-25T10:18:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.572+0000 7fef63e00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef40066f60 0x7fef40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.572+0000 7fef63e00640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef40066f60 0x7fef40069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef540af000 tx=0x7fef540a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.572+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef4c01f020 con 0x7fef640cf930 2024-04-25T10:18:39.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.736+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef640b40b0 con 0x7fef640cf930 2024-04-25T10:18:39.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.736+0000 7fef60c00640 1 -- 172.21.15.45:0/2265016073 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fef4c00d040 con 0x7fef640cf930 2024-04-25T10:18:39.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:39.741 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:03.683965+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:39.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef40066f60 msgr2=0x7fef40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef40066f60 0x7fef40069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef540af000 tx=0x7fef540a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:39.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 msgr2=0x7fef640b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 0x7fef640b1720 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00b440 tx=0x7fef4c00b900 comp rx=0 tx=0).stop 2024-04-25T10:18:39.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 shutdown_connections 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fef40066f60 0x7fef40069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef6415bc40 0x7fef640b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef640cf930 0x7fef640b1720 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 --2- 172.21.15.45:0/2265016073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef640cda10 0x7fef6415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 >> 172.21.15.45:0/2265016073 conn(0x7fef64066b70 msgr2=0x7fef640ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 shutdown_connections 2024-04-25T10:18:39.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.740+0000 7fef6b000640 1 -- 172.21.15.45:0/2265016073 wait complete. 2024-04-25T10:18:39.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:39.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 Processor -- start 2024-04-25T10:18:39.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 -- start start 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7940d0d00 0x7fb7940ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb7940cc200 0x7fb794162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7940cdab0 con 0x7fb7941626c0 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7940cd210 con 0x7fb7940d0d00 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb79b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7940cd4f0 con 0x7fb7940cc200 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb793e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7940d0d00 0x7fb7940ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb799200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb7941626c0 0x7fb79406cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38336/0 (socket says 172.21.15.45:38336) 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb799200640 1 -- 172.21.15.45:0/3401720406 learned_addr learned my addr 172.21.15.45:0/3401720406 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:39.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb78be00640 1 --2- 172.21.15.45:0/3401720406 >> v2:172.21.15.117:3300/0 conn(0x7fb7940cc200 0x7fb794162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb793e00640 1 -- 172.21.15.45:0/3401720406 >> v2:172.21.15.117:3300/0 conn(0x7fb7940cc200 msgr2=0x7fb794162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb793e00640 1 --2- 172.21.15.45:0/3401720406 >> v2:172.21.15.117:3300/0 conn(0x7fb7940cc200 0x7fb794162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb793e00640 1 -- 172.21.15.45:0/3401720406 >> v1:172.21.15.45:6789/0 conn(0x7fb7941626c0 legacy=0x7fb79406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.904+0000 7fb793e00640 1 -- 172.21.15.45:0/3401720406 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7940cd7d0 con 0x7fb7940d0d00 2024-04-25T10:18:39.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb793e00640 1 --2- 172.21.15.45:0/3401720406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb7940ccc30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb794071ca0 tx=0x7fb78401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f2293d511dcb18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb793400640 1 -- 172.21.15.45:0/3401720406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7840b0070 con 0x7fb7940d0d00 2024-04-25T10:18:39.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb793400640 1 -- 172.21.15.45:0/3401720406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7840ab600 con 0x7fb7940d0d00 2024-04-25T10:18:39.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb793400640 1 -- 172.21.15.45:0/3401720406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7840ab9d0 con 0x7fb7940d0d00 2024-04-25T10:18:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 -- 172.21.15.45:0/3401720406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 msgr2=0x7fb7940ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 --2- 172.21.15.45:0/3401720406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb7940ccc30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb794071ca0 tx=0x7fb78401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 -- 172.21.15.45:0/3401720406 shutdown_connections 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 --2- 172.21.15.45:0/3401720406 >> v2:172.21.15.117:3300/0 conn(0x7fb7940cc200 0x7fb794162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 --2- 172.21.15.45:0/3401720406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb7940ccc30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 -- 172.21.15.45:0/3401720406 >> 172.21.15.45:0/3401720406 conn(0x7fb794072290 msgr2=0x7fb7940726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 -- 172.21.15.45:0/3401720406 shutdown_connections 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.908+0000 7fb79b200640 1 -- 172.21.15.45:0/3401720406 wait complete. 2024-04-25T10:18:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 Processor -- start 2024-04-25T10:18:39.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 -- start start 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7940cc200 0x7fb794099380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb794097460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7941626c0 0x7fb794157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79415e040 con 0x7fb7940d0d00 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79406d860 con 0x7fb7940cc200 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79415e1c0 con 0x7fb7941626c0 2024-04-25T10:18:39.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb794097460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb794097460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52468/0 (socket says 172.21.15.45:52468) 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 -- 172.21.15.45:0/126635853 learned_addr learned my addr 172.21.15.45:0/126635853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb793e00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7940cc200 0x7fb794099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb799200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7941626c0 0x7fb794157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 -- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7941626c0 msgr2=0x7fb794157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7941626c0 0x7fb794157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 -- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7940cc200 msgr2=0x7fb794099380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7940cc200 0x7fb794099380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 -- 172.21.15.45:0/126635853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794156b30 con 0x7fb7940d0d00 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb78be00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb794097460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00d800 tx=0x7fb77c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00b6c0 con 0x7fb7940d0d00 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c019070 con 0x7fb7940d0d00 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00ba60 con 0x7fb7940d0d00 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb794156e10 con 0x7fb7940d0d00 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.912+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7941573d0 con 0x7fb7940d0d00 2024-04-25T10:18:39.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.916+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7941588d0 con 0x7fb7940d0d00 2024-04-25T10:18:39.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.916+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb77c00bc00 con 0x7fb7940d0d00 2024-04-25T10:18:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.916+0000 7fb790c00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb770066f60 0x7fb770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.916+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb77c085400 con 0x7fb7940d0d00 2024-04-25T10:18:39.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.916+0000 7fb793e00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb770066f60 0x7fb770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.916+0000 7fb793e00640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb770066f60 0x7fb770069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb7840ad000 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:39.920+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7941588d0 con 0x7fb7940d0d00 2024-04-25T10:18:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb79404ef30 con 0x7fb7940d0d00 2024-04-25T10:18:40.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb790c00640 1 -- 172.21.15.45:0/126635853 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb77c056400 con 0x7fb7940d0d00 2024-04-25T10:18:40.087 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb770066f60 msgr2=0x7fb770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb770066f60 0x7fb770069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb7840ad000 tx=0x7fb784008040 comp rx=0 tx=0).stop 2024-04-25T10:18:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 msgr2=0x7fb794097460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb794097460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00d800 tx=0x7fb77c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 shutdown_connections 2024-04-25T10:18:40.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb770066f60 0x7fb770069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7941626c0 0x7fb794157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7940d0d00 0x7fb794097460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 --2- 172.21.15.45:0/126635853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7940cc200 0x7fb794099380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 >> 172.21.15.45:0/126635853 conn(0x7fb794072290 msgr2=0x7fb7940ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 shutdown_connections 2024-04-25T10:18:40.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.084+0000 7fb79b200640 1 -- 172.21.15.45:0/126635853 wait complete. 2024-04-25T10:18:40.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:40.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.248+0000 7f881aa00640 1 Processor -- start 2024-04-25T10:18:40.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.248+0000 7f881aa00640 1 -- start start 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f881aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8814066c00 0x7f8814161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f881aa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88140b40c0 con 0x7f88140b5560 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f881aa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88140b5b00 con 0x7f8814066fe0 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f881aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8814161d60 con 0x7f8814066c00 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8818a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f88140b5560 0x7f88140ae850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38352/0 (socket says 172.21.15.45:38352) 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8818a00640 1 -- 172.21.15.45:0/747819533 learned_addr learned my addr 172.21.15.45:0/747819533 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:40.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813e00640 1 --2- 172.21.15.45:0/747819533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8814066c00 0x7f8814161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813400640 1 -- 172.21.15.45:0/747819533 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009820596 0 0) 0x7f88140b40c0 con 0x7f88140b5560 2024-04-25T10:18:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813400640 1 -- 172.21.15.45:0/747819533 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f8003530 con 0x7f88140b5560 2024-04-25T10:18:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813e00640 1 -- 172.21.15.45:0/747819533 >> v1:172.21.15.117:6789/0 conn(0x7f8814066fe0 legacy=0x7f8814072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813400640 1 -- 172.21.15.45:0/747819533 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273690427 0 0) 0x7f88140b5b00 con 0x7f8814066fe0 2024-04-25T10:18:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813e00640 1 -- 172.21.15.45:0/747819533 >> v1:172.21.15.45:6789/0 conn(0x7f88140b5560 legacy=0x7f88140ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.252+0000 7f8813e00640 1 -- 172.21.15.45:0/747819533 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88140b3a00 con 0x7f8814066c00 2024-04-25T10:18:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f8813e00640 1 --2- 172.21.15.45:0/747819533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8814066c00 0x7f8814161820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88040097f0 tx=0x7f880401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80793b6cac760230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f8813400640 1 -- 172.21.15.45:0/747819533 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88040b0070 con 0x7f8814066c00 2024-04-25T10:18:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f8813400640 1 -- 172.21.15.45:0/747819533 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88040ab600 con 0x7f8814066c00 2024-04-25T10:18:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f8813400640 1 -- 172.21.15.45:0/747819533 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88040ab9d0 con 0x7f8814066c00 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 -- 172.21.15.45:0/747819533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8814066c00 msgr2=0x7f8814161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 --2- 172.21.15.45:0/747819533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8814066c00 0x7f8814161820 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88040097f0 tx=0x7f880401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 -- 172.21.15.45:0/747819533 shutdown_connections 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 --2- 172.21.15.45:0/747819533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8814066c00 0x7f8814161820 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 -- 172.21.15.45:0/747819533 >> 172.21.15.45:0/747819533 conn(0x7f88140b2560 msgr2=0x7f88140b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 -- 172.21.15.45:0/747819533 shutdown_connections 2024-04-25T10:18:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 -- 172.21.15.45:0/747819533 wait complete. 2024-04-25T10:18:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 Processor -- start 2024-04-25T10:18:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.256+0000 7f881aa00640 1 -- start start 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 0x7f88141f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8814066fe0 0x7f881415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88140b5560 0x7f88140b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881415e540 con 0x7f8814066c00 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8814158620 con 0x7f8814066fe0 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881415e6c0 con 0x7f88140b5560 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 0x7f88141f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 0x7f88141f6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52482/0 (socket says 172.21.15.45:52482) 2024-04-25T10:18:40.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 -- 172.21.15.45:0/3575546619 learned_addr learned my addr 172.21.15.45:0/3575546619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 -- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88140b5560 msgr2=0x7f88140b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88140b5560 0x7f88140b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 -- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8814066fe0 msgr2=0x7f881415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8814066fe0 0x7f881415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 -- 172.21.15.45:0/3575546619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8814153640 con 0x7f8814066c00 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8813e00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 0x7f88141f6af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f88040ae040 tx=0x7f880401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88040bb020 con 0x7f8814066c00 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88040b1040 con 0x7f8814066c00 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88040b0050 con 0x7f8814066c00 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8814153920 con 0x7f8814066c00 2024-04-25T10:18:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8814153ee0 con 0x7f8814066c00 2024-04-25T10:18:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881404ef30 con 0x7f8814066c00 2024-04-25T10:18:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.260+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88040be070 con 0x7f8814066c00 2024-04-25T10:18:40.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.264+0000 7f8810c00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f87f0066f60 0x7f87f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.264+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f880411e8e0 con 0x7f8814066c00 2024-04-25T10:18:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.264+0000 7f880be00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f87f0066f60 0x7f87f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.264+0000 7f880be00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f87f0066f60 0x7f87f0069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f87fc007fd0 tx=0x7f87fc007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.268+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88040160d0 con 0x7f8814066c00 2024-04-25T10:18:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.428+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8814161160 con 0x7f8814066c00 2024-04-25T10:18:40.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.428+0000 7f8810c00640 1 -- 172.21.15.45:0/3575546619 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88040b3300 con 0x7f8814066c00 2024-04-25T10:18:40.433 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f87f0066f60 msgr2=0x7f87f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f87f0066f60 0x7f87f0069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f87fc007fd0 tx=0x7f87fc007920 comp rx=0 tx=0).stop 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 msgr2=0x7f88141f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 0x7f88141f6af0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f88040ae040 tx=0x7f880401ac00 comp rx=0 tx=0).stop 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 shutdown_connections 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f87f0066f60 0x7f87f0069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88140b5560 0x7f88140b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8814066fe0 0x7f881415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 --2- 172.21.15.45:0/3575546619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8814066c00 0x7f88141f6af0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 >> 172.21.15.45:0/3575546619 conn(0x7f88140b2560 msgr2=0x7f88140ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 shutdown_connections 2024-04-25T10:18:40.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.432+0000 7f881aa00640 1 -- 172.21.15.45:0/3575546619 wait complete. 2024-04-25T10:18:40.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:40.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.592+0000 7f796b000640 1 Processor -- start 2024-04-25T10:18:40.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.592+0000 7f796b000640 1 -- start start 2024-04-25T10:18:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f796b000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7964159540 0x7f79640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f796b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79640721f0 0x7f7964070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f796b000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7964159920 con 0x7f79640aebd0 2024-04-25T10:18:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f796b000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796415c2f0 con 0x7f7964159540 2024-04-25T10:18:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f796b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796415c5d0 con 0x7f79640721f0 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f795be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f79640aebd0 0x7f7964071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38356/0 (socket says 172.21.15.45:38356) 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f795be00640 1 -- 172.21.15.45:0/421886507 learned_addr learned my addr 172.21.15.45:0/421886507 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7969000640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79640721f0 0x7f7964070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963e00640 1 --2- 172.21.15.45:0/421886507 >> v2:172.21.15.117:3300/0 conn(0x7f7964159540 0x7f79640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963400640 1 -- 172.21.15.45:0/421886507 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596341709 0 0) 0x7f7964159920 con 0x7f79640aebd0 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963400640 1 -- 172.21.15.45:0/421886507 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7948003530 con 0x7f79640aebd0 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963e00640 1 -- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79640721f0 msgr2=0x7f7964070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963e00640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79640721f0 0x7f7964070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963e00640 1 -- 172.21.15.45:0/421886507 >> v1:172.21.15.45:6789/0 conn(0x7f79640aebd0 legacy=0x7f7964071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7963e00640 1 -- 172.21.15.45:0/421886507 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796415c8b0 con 0x7f7964159540 2024-04-25T10:18:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.596+0000 7f7969000640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79640721f0 0x7f7964070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:40.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f7963e00640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964159540 0x7f79640ae690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79540097c0 tx=0x7f795401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1215dd526afa9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f7963400640 1 -- 172.21.15.45:0/421886507 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540b1070 con 0x7f7964159540 2024-04-25T10:18:40.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f7963400640 1 -- 172.21.15.45:0/421886507 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79540ac410 con 0x7f7964159540 2024-04-25T10:18:40.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f7963400640 1 -- 172.21.15.45:0/421886507 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79540ac7e0 con 0x7f7964159540 2024-04-25T10:18:40.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 -- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964159540 msgr2=0x7f79640ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964159540 0x7f79640ae690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79540097c0 tx=0x7f795401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 -- 172.21.15.45:0/421886507 shutdown_connections 2024-04-25T10:18:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79640721f0 0x7f7964070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 --2- 172.21.15.45:0/421886507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7964159540 0x7f79640ae690 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 -- 172.21.15.45:0/421886507 >> 172.21.15.45:0/421886507 conn(0x7f79640669a0 msgr2=0x7f7964066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 -- 172.21.15.45:0/421886507 shutdown_connections 2024-04-25T10:18:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 -- 172.21.15.45:0/421886507 wait complete. 2024-04-25T10:18:40.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 Processor -- start 2024-04-25T10:18:40.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.600+0000 7f796b000640 1 -- start start 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79640721f0 0x7f796415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 0x7f796414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7964159540 0x7f796414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964071220 con 0x7f79640aebd0 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79640710a0 con 0x7f79640721f0 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79640713a0 con 0x7f7964159540 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 0x7f796414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 0x7f796414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52490/0 (socket says 172.21.15.45:52490) 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 -- 172.21.15.45:0/1160202743 learned_addr learned my addr 172.21.15.45:0/1160202743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f7963e00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79640721f0 0x7f796415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f7969000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7964159540 0x7f796414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 -- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7964159540 msgr2=0x7f796414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7964159540 0x7f796414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 -- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79640721f0 msgr2=0x7f796415b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79640721f0 0x7f796415b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 -- 172.21.15.45:0/1160202743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796414be30 con 0x7f79640aebd0 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f795be00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 0x7f796414b6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c480 tx=0x7f794c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c017070 con 0x7f79640aebd0 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c005cf0 con 0x7f79640aebd0 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c0072f0 con 0x7f79640aebd0 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796414c110 con 0x7f79640aebd0 2024-04-25T10:18:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796414c6d0 con 0x7f79640aebd0 2024-04-25T10:18:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.604+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796404ef30 con 0x7f79640aebd0 2024-04-25T10:18:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.608+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f794c01a070 con 0x7f79640aebd0 2024-04-25T10:18:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.608+0000 7f7960c00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7940066f90 0x7f7940069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.608+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f794c0842c0 con 0x7f79640aebd0 2024-04-25T10:18:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.608+0000 7f7963e00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7940066f90 0x7f7940069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.608+0000 7f7963e00640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7940066f90 0x7f7940069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79540ae000 tx=0x7f7954008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.612+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f794c054c80 con 0x7f79640aebd0 2024-04-25T10:18:40.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.772+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f796406d7f0 con 0x7f79640aebd0 2024-04-25T10:18:40.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.772+0000 7f7960c00640 1 -- 172.21.15.45:0/1160202743 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f794c058d20 con 0x7f79640aebd0 2024-04-25T10:18:40.776 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7940066f90 msgr2=0x7f7940069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7940066f90 0x7f7940069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79540ae000 tx=0x7f7954008040 comp rx=0 tx=0).stop 2024-04-25T10:18:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 msgr2=0x7f796414b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 0x7f796414b6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c480 tx=0x7f794c00c880 comp rx=0 tx=0).stop 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 shutdown_connections 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7940066f90 0x7f7940069450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7964159540 0x7f796414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79640aebd0 0x7f796414b6b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 --2- 172.21.15.45:0/1160202743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79640721f0 0x7f796415b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 >> 172.21.15.45:0/1160202743 conn(0x7f79640669a0 msgr2=0x7f796415d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 shutdown_connections 2024-04-25T10:18:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.776+0000 7f796b000640 1 -- 172.21.15.45:0/1160202743 wait complete. 2024-04-25T10:18:40.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.936+0000 7f4fd1600640 1 Processor -- start 2024-04-25T10:18:40.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.936+0000 7f4fd1600640 1 -- start start 2024-04-25T10:18:40.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fd1600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fcc0b26c0 0x7f4fcc0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fd1600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fd1600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc15f5e0 con 0x7f4fcc1451d0 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fd1600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc0b3040 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fd1600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc15f8c0 con 0x7f4fcc06d2a0 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fc3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fcc0b26c0 0x7f4fcc0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fcbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4fcc1451d0 0x7f4fcc0cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38358/0 (socket says 172.21.15.45:38358) 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fcbe00640 1 -- 172.21.15.45:0/75312285 learned_addr learned my addr 172.21.15.45:0/75312285 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fc3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fcc0b26c0 0x7f4fcc0b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52500/0 (socket says 172.21.15.45:52500) 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fd0c00640 1 --2- 172.21.15.45:0/75312285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fc3400640 1 -- 172.21.15.45:0/75312285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 msgr2=0x7f4fcc06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fc3400640 1 --2- 172.21.15.45:0/75312285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc06d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fc3400640 1 -- 172.21.15.45:0/75312285 >> v1:172.21.15.45:6789/0 conn(0x7f4fcc1451d0 legacy=0x7f4fcc0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.940+0000 7f4fc3400640 1 -- 172.21.15.45:0/75312285 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc15fba0 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fc3400640 1 --2- 172.21.15.45:0/75312285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc0b2b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009d90 tx=0x7f4fb401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce15f3dc4e25b71f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fcb400640 1 -- 172.21.15.45:0/75312285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40b2070 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fcb400640 1 -- 172.21.15.45:0/75312285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40a5870 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fcb400640 1 -- 172.21.15.45:0/75312285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40a5c40 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- 172.21.15.45:0/75312285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 msgr2=0x7f4fcc0b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 --2- 172.21.15.45:0/75312285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc0b2b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009d90 tx=0x7f4fb401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- 172.21.15.45:0/75312285 shutdown_connections 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 --2- 172.21.15.45:0/75312285 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc06d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 --2- 172.21.15.45:0/75312285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc0b2b00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- 172.21.15.45:0/75312285 >> 172.21.15.45:0/75312285 conn(0x7f4fcc074310 msgr2=0x7f4fcc06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- 172.21.15.45:0/75312285 shutdown_connections 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- 172.21.15.45:0/75312285 wait complete. 2024-04-25T10:18:40.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 Processor -- start 2024-04-25T10:18:40.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- start start 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc13a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fcc1451d0 0x7f4fcc13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc06ece0 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0ce240 con 0x7f4fcc1451d0 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd1600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc06ee60 con 0x7f4fcc06d2a0 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc13a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc13a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52510/0 (socket says 172.21.15.45:52510) 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fc3400640 1 -- 172.21.15.45:0/3853003693 learned_addr learned my addr 172.21.15.45:0/3853003693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fcbe00640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.944+0000 7f4fd0c00640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fcc1451d0 0x7f4fcc13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc3400640 1 -- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 msgr2=0x7f4fcc158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc3400640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc3400640 1 -- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fcc1451d0 msgr2=0x7f4fcc13ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc3400640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fcc1451d0 0x7f4fcc13ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc3400640 1 -- 172.21.15.45:0/3853003693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc153fb0 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc3400640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc13a830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4fb401a510 tx=0x7f4fb4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40b2050 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40be070 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc154290 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40ada70 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc154850 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc075120 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fb40adc10 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc8c00640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fa8066f60 0x7f4fa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.948+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fb41201f0 con 0x7f4fcc0b26c0 2024-04-25T10:18:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.952+0000 7f4fcbe00640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fa8066f60 0x7f4fa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.952+0000 7f4fcbe00640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fa8066f60 0x7f4fa8069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc000f70 tx=0x7f4fbc007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:40.956+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fb4016560 con 0x7f4fcc0b26c0 2024-04-25T10:18:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.128+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4fcc0ccd40 con 0x7f4fcc0b26c0 2024-04-25T10:18:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.128+0000 7f4fc8c00640 1 -- 172.21.15.45:0/3853003693 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4fb40f0b10 con 0x7f4fcc0b26c0 2024-04-25T10:18:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:18:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fa8066f60 msgr2=0x7f4fa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fa8066f60 0x7f4fa8069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc000f70 tx=0x7f4fbc007e90 comp rx=0 tx=0).stop 2024-04-25T10:18:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 msgr2=0x7f4fcc13a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc13a830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4fb401a510 tx=0x7f4fb4019680 comp rx=0 tx=0).stop 2024-04-25T10:18:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 shutdown_connections 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fa8066f60 0x7f4fa8069420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fcc1451d0 0x7f4fcc13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fcc0b26c0 0x7f4fcc13a830 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 --2- 172.21.15.45:0/3853003693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fcc06d2a0 0x7f4fcc158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 >> 172.21.15.45:0/3853003693 conn(0x7f4fcc074310 msgr2=0x7f4fcc143cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 shutdown_connections 2024-04-25T10:18:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.132+0000 7f4fd1600640 1 -- 172.21.15.45:0/3853003693 wait complete. 2024-04-25T10:18:41.154 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:18:41.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:41.154 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:18:41.181 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:18:41.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:41.181 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:18:41.232 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:18:41.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:41.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.368+0000 7f9d00c00640 1 Processor -- start 2024-04-25T10:18:41.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.368+0000 7f9d00c00640 1 -- start start 2024-04-25T10:18:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9d00c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cfc1474d0 0x7f9cfc06e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9d00c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9cfc066ee0 0x7f9cfc15f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9d00c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc067350 con 0x7f9cfc1474d0 2024-04-25T10:18:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9d00c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc06f610 con 0x7f9cfc066ee0 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9d00c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc15f060 con 0x7f9cfc15ec00 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cfc1474d0 0x7f9cfc06e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cfc1474d0 0x7f9cfc06e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52514/0 (socket says 172.21.15.45:52514) 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 -- 172.21.15.45:0/2823852865 learned_addr learned my addr 172.21.15.45:0/2823852865 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 -- 172.21.15.45:0/2823852865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 msgr2=0x7f9cfc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 --2- 172.21.15.45:0/2823852865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 -- 172.21.15.45:0/2823852865 >> v2:172.21.15.117:3300/0 conn(0x7f9cfc066ee0 msgr2=0x7f9cfc15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 --2- 172.21.15.45:0/2823852865 >> v2:172.21.15.117:3300/0 conn(0x7f9cfc066ee0 0x7f9cfc15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 -- 172.21.15.45:0/2823852865 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc06ea90 con 0x7f9cfc1474d0 2024-04-25T10:18:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.372+0000 7f9cfb400640 1 --2- 172.21.15.45:0/2823852865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc06e360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc06d170 tx=0x7f9cec01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915e5d194ef5080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cfaa00640 1 -- 172.21.15.45:0/2823852865 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0a55e0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cfaa00640 1 -- 172.21.15.45:0/2823852865 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0a5780 con 0x7f9cfc1474d0 2024-04-25T10:18:41.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cfaa00640 1 -- 172.21.15.45:0/2823852865 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0a5b50 con 0x7f9cfc1474d0 2024-04-25T10:18:41.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- 172.21.15.45:0/2823852865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 msgr2=0x7f9cfc06e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- 172.21.15.45:0/2823852865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc06e360 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc06d170 tx=0x7f9cec01c960 comp rx=0 tx=0).stop 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- 172.21.15.45:0/2823852865 shutdown_connections 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- 172.21.15.45:0/2823852865 >> v2:172.21.15.117:3300/0 conn(0x7f9cfc066ee0 0x7f9cfc15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- 172.21.15.45:0/2823852865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- 172.21.15.45:0/2823852865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc06e360 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- 172.21.15.45:0/2823852865 >> 172.21.15.45:0/2823852865 conn(0x7f9cfc071280 msgr2=0x7f9cfc071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- 172.21.15.45:0/2823852865 shutdown_connections 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- 172.21.15.45:0/2823852865 wait complete. 2024-04-25T10:18:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 Processor -- start 2024-04-25T10:18:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- start start 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cfc066ee0 0x7f9cfc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc1604f0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc160370 con 0x7f9cfc066ee0 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9d00c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc160670 con 0x7f9cfc15ec00 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cf2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cf2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc0b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52524/0 (socket says 172.21.15.45:52524) 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cf2a00640 1 -- 172.21.15.45:0/1317310051 learned_addr learned my addr 172.21.15.45:0/1317310051 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cfb400640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cfc066ee0 0x7f9cfc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.376+0000 7f9cfbe00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf2a00640 1 -- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 msgr2=0x7f9cfc0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf2a00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf2a00640 1 -- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cfc066ee0 msgr2=0x7f9cfc0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf2a00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cfc066ee0 0x7f9cfc0ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf2a00640 1 -- 172.21.15.45:0/1317310051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0ce5b0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cfb400640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cfc066ee0 0x7f9cfc0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf2a00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc0b55b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b440 tx=0x7f9ce400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4015020 con 0x7f9cfc1474d0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce40118b0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4011ca0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc0cfcf0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc0cef30 con 0x7f9cfc1474d0 2024-04-25T10:18:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc075120 con 0x7f9cfc1474d0 2024-04-25T10:18:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ce4018020 con 0x7f9cfc1474d0 2024-04-25T10:18:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.380+0000 7f9cf3e00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd8066f60 0x7f9cd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.384+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ce40851d0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.384+0000 7f9cfb400640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd8066f60 0x7f9cd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.384+0000 7f9cfb400640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd8066f60 0x7f9cd8069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0ae000 tx=0x7f9cec0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.388+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ce401f020 con 0x7f9cfc1474d0 2024-04-25T10:18:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cfc0b2180 con 0x7f9cfc1474d0 2024-04-25T10:18:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9cf3e00640 1 -- 172.21.15.45:0/1317310051 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f9cfc0ce5b0 con 0x7f9cfc1474d0 2024-04-25T10:18:41.584 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:41.584 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd8066f60 msgr2=0x7f9cd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd8066f60 0x7f9cd8069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0ae000 tx=0x7f9cec0a7070 comp rx=0 tx=0).stop 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 msgr2=0x7f9cfc0b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc0b55b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b440 tx=0x7f9ce400b900 comp rx=0 tx=0).stop 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 shutdown_connections 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd8066f60 0x7f9cd8069420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cfc15ec00 0x7f9cfc0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cfc1474d0 0x7f9cfc0b55b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 --2- 172.21.15.45:0/1317310051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cfc066ee0 0x7f9cfc0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.580+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 >> 172.21.15.45:0/1317310051 conn(0x7f9cfc071280 msgr2=0x7f9cfc0b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.584+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 shutdown_connections 2024-04-25T10:18:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.584+0000 7f9d00c00640 1 -- 172.21.15.45:0/1317310051 wait complete. 2024-04-25T10:18:41.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:41.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 Processor -- start 2024-04-25T10:18:41.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 -- start start 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42a815b0b0 0x7f42a80713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81586c0 0x7f42a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a806e4e0 con 0x7f42a81582e0 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a806e2a0 con 0x7f42a815b0b0 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42b0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a806e7c0 con 0x7f42a81586c0 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42acc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42a815b0b0 0x7f42a80713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42ad600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f42a81582e0 0x7f42a806db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38362/0 (socket says 172.21.15.45:38362) 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42ad600640 1 -- 172.21.15.45:0/1619011914 learned_addr learned my addr 172.21.15.45:0/1619011914 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42ae000640 1 --2- 172.21.15.45:0/1619011914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81586c0 0x7f42a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42acc00640 1 -- 172.21.15.45:0/1619011914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81586c0 msgr2=0x7f42a8071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42acc00640 1 --2- 172.21.15.45:0/1619011914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81586c0 0x7f42a8071910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42acc00640 1 -- 172.21.15.45:0/1619011914 >> v1:172.21.15.45:6789/0 conn(0x7f42a81582e0 legacy=0x7f42a806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42acc00640 1 -- 172.21.15.45:0/1619011914 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a806eaa0 con 0x7f42a815b0b0 2024-04-25T10:18:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.744+0000 7f42a7e00640 1 -- 172.21.15.45:0/1619011914 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971307710 0 0) 0x7f42a806e4e0 con 0x7f42a81582e0 2024-04-25T10:18:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42acc00640 1 --2- 172.21.15.45:0/1619011914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a815b0b0 0x7f42a80713d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f42940097f0 tx=0x7f429401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91b586da02af6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42a7e00640 1 -- 172.21.15.45:0/1619011914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940b1070 con 0x7f42a815b0b0 2024-04-25T10:18:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42a7e00640 1 -- 172.21.15.45:0/1619011914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940ac410 con 0x7f42a815b0b0 2024-04-25T10:18:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42a7e00640 1 -- 172.21.15.45:0/1619011914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a815c220 con 0x7f42a815b0b0 2024-04-25T10:18:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 -- 172.21.15.45:0/1619011914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a815b0b0 msgr2=0x7f42a80713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 --2- 172.21.15.45:0/1619011914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a815b0b0 0x7f42a80713d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f42940097f0 tx=0x7f429401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 -- 172.21.15.45:0/1619011914 shutdown_connections 2024-04-25T10:18:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 --2- 172.21.15.45:0/1619011914 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81586c0 0x7f42a8071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 --2- 172.21.15.45:0/1619011914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a815b0b0 0x7f42a80713d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 -- 172.21.15.45:0/1619011914 >> 172.21.15.45:0/1619011914 conn(0x7f42a806ac90 msgr2=0x7f42a806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 -- 172.21.15.45:0/1619011914 shutdown_connections 2024-04-25T10:18:41.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.748+0000 7f42b0000640 1 -- 172.21.15.45:0/1619011914 wait complete. 2024-04-25T10:18:41.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 Processor -- start 2024-04-25T10:18:41.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- start start 2024-04-25T10:18:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81582e0 0x7f42a8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 0x7f42a815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42a815b0b0 0x7f42a815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b4a90 con 0x7f42a81586c0 2024-04-25T10:18:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b4910 con 0x7f42a815b0b0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b4c10 con 0x7f42a81582e0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 0x7f42a815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 0x7f42a815e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52548/0 (socket says 172.21.15.45:52548) 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 -- 172.21.15.45:0/868349586 learned_addr learned my addr 172.21.15.45:0/868349586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42ad600640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81582e0 0x7f42a8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42ae000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42a815b0b0 0x7f42a815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 -- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81582e0 msgr2=0x7f42a8152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81582e0 0x7f42a8152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 -- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42a815b0b0 msgr2=0x7f42a815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42a815b0b0 0x7f42a815eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 -- 172.21.15.45:0/868349586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8140610 con 0x7f42a81586c0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42ae000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42a815b0b0 0x7f42a815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42acc00640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 0x7f42a815e970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f42940af040 tx=0x7f4294019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940b1040 con 0x7f42a81586c0 2024-04-25T10:18:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940bb070 con 0x7f42a81586c0 2024-04-25T10:18:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940b5410 con 0x7f42a81586c0 2024-04-25T10:18:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a81408f0 con 0x7f42a81586c0 2024-04-25T10:18:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80cdd10 con 0x7f42a81586c0 2024-04-25T10:18:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.752+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a804ef30 con 0x7f42a81586c0 2024-04-25T10:18:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.756+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42940b55b0 con 0x7f42a81586c0 2024-04-25T10:18:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.756+0000 7f42a5600640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4288066f60 0x7f4288069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.756+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f429411ecb0 con 0x7f42a81586c0 2024-04-25T10:18:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.756+0000 7f42ad600640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4288066f60 0x7f4288069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.756+0000 7f42ad600640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4288066f60 0x7f4288069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f429c000f70 tx=0x7f429c007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.760+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4294016090 con 0x7f42a81586c0 2024-04-25T10:18:41.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.924+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42a806d740 con 0x7f42a81586c0 2024-04-25T10:18:41.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42a5600640 1 -- 172.21.15.45:0/868349586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f42940efb90 con 0x7f42a81586c0 2024-04-25T10:18:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:41.931 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":23,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:41.883603+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4288066f60 msgr2=0x7f4288069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4288066f60 0x7f4288069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f429c000f70 tx=0x7f429c007b60 comp rx=0 tx=0).stop 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 msgr2=0x7f42a815e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 0x7f42a815e970 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f42940af040 tx=0x7f4294019240 comp rx=0 tx=0).stop 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 shutdown_connections 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4288066f60 0x7f4288069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42a815b0b0 0x7f42a815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a81586c0 0x7f42a815e970 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 --2- 172.21.15.45:0/868349586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42a81582e0 0x7f42a8152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 >> 172.21.15.45:0/868349586 conn(0x7f42a806ac90 msgr2=0x7f42a8164f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 shutdown_connections 2024-04-25T10:18:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:41.928+0000 7f42b0000640 1 -- 172.21.15.45:0/868349586 wait complete. 2024-04-25T10:18:41.952 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:59.922144+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:01.069700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:41.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:42.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 Processor -- start 2024-04-25T10:18:42.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 -- start start 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f28e40aef80 0x7f28e415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28e4070350 0x7f28e415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e415bac0 0x7f28e40b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e415b080 con 0x7f28e4070350 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e415adb0 con 0x7f28e40aef80 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e415eb60 con 0x7f28e415bac0 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28dbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28e4070350 0x7f28e415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28dbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f28e4070350 0x7f28e415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52554/0 (socket says 172.21.15.45:52554) 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28dbe00640 1 -- 172.21.15.45:0/3403446107 learned_addr learned my addr 172.21.15.45:0/3403446107 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e2800640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e415bac0 0x7f28e40b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.092+0000 7f28e1e00640 1 --2- 172.21.15.45:0/3403446107 >> v2:172.21.15.117:3300/0 conn(0x7f28e40aef80 0x7f28e415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28dbe00640 1 -- 172.21.15.45:0/3403446107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e415bac0 msgr2=0x7f28e40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28dbe00640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e415bac0 0x7f28e40b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28dbe00640 1 -- 172.21.15.45:0/3403446107 >> v2:172.21.15.117:3300/0 conn(0x7f28e40aef80 msgr2=0x7f28e415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28dbe00640 1 --2- 172.21.15.45:0/3403446107 >> v2:172.21.15.117:3300/0 conn(0x7f28e40aef80 0x7f28e415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28dbe00640 1 -- 172.21.15.45:0/3403446107 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e415ee40 con 0x7f28e4070350 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e2800640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e415bac0 0x7f28e40b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e1e00640 1 --2- 172.21.15.45:0/3403446107 >> v2:172.21.15.117:3300/0 conn(0x7f28e40aef80 0x7f28e415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28dbe00640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e4070350 0x7f28e415b580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f28e406e220 tx=0x7f28cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91bb6993077a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e1400640 1 -- 172.21.15.45:0/3403446107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0ab5f0 con 0x7f28e4070350 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e1400640 1 -- 172.21.15.45:0/3403446107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc0b0070 con 0x7f28e4070350 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e1400640 1 -- 172.21.15.45:0/3403446107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0ab9b0 con 0x7f28e4070350 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 -- 172.21.15.45:0/3403446107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e4070350 msgr2=0x7f28e415b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e4070350 0x7f28e415b580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f28e406e220 tx=0x7f28cc01c520 comp rx=0 tx=0).stop 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 -- 172.21.15.45:0/3403446107 shutdown_connections 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e415bac0 0x7f28e40b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 --2- 172.21.15.45:0/3403446107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e4070350 0x7f28e415b580 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 --2- 172.21.15.45:0/3403446107 >> v2:172.21.15.117:3300/0 conn(0x7f28e40aef80 0x7f28e415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 -- 172.21.15.45:0/3403446107 >> 172.21.15.45:0/3403446107 conn(0x7f28e40669a0 msgr2=0x7f28e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 -- 172.21.15.45:0/3403446107 shutdown_connections 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.096+0000 7f28e9200640 1 -- 172.21.15.45:0/3403446107 wait complete. 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 Processor -- start 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 -- start start 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e4070350 0x7f28e414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 0x7f28e40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e415bac0 0x7f28e40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e406d570 con 0x7f28e40aef80 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e406d3f0 con 0x7f28e415bac0 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e406d6f0 con 0x7f28e4070350 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 0x7f28e40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 0x7f28e40b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52558/0 (socket says 172.21.15.45:52558) 2024-04-25T10:18:42.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 -- 172.21.15.45:0/1448890084 learned_addr learned my addr 172.21.15.45:0/1448890084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e1e00640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e4070350 0x7f28e414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e2800640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e415bac0 0x7f28e40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 -- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e4070350 msgr2=0x7f28e414c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e4070350 0x7f28e414c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 -- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e415bac0 msgr2=0x7f28e40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e415bac0 0x7f28e40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 -- 172.21.15.45:0/1448890084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4150e80 con 0x7f28e40aef80 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e2800640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e415bac0 0x7f28e40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28dbe00640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 0x7f28e40b0220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f28cc01abd0 tx=0x7f28cc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0be5f0 con 0x7f28e40aef80 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc0be790 con 0x7f28e40aef80 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0beb80 con 0x7f28e40aef80 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e4151160 con 0x7f28e40aef80 2024-04-25T10:18:42.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.100+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e4151720 con 0x7f28e40aef80 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.104+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e404ef30 con 0x7f28e40aef80 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.104+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28cc0b43f0 con 0x7f28e40aef80 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.104+0000 7f28da000640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28c0066f60 0x7f28c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.104+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f28cc11f6d0 con 0x7f28e40aef80 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.104+0000 7f28e1e00640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28c0066f60 0x7f28c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.104+0000 7f28e1e00640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28c0066f60 0x7f28c0069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f28d40098d0 tx=0x7f28d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.108+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28cc016110 con 0x7f28e40aef80 2024-04-25T10:18:42.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28e40cec70 con 0x7f28e40aef80 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28da000640 1 -- 172.21.15.45:0/1448890084 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28cc0f00f0 con 0x7f28e40aef80 2024-04-25T10:18:42.275 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28c0066f60 msgr2=0x7f28c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28c0066f60 0x7f28c0069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f28d40098d0 tx=0x7f28d4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 msgr2=0x7f28e40b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 0x7f28e40b0220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f28cc01abd0 tx=0x7f28cc01ad60 comp rx=0 tx=0).stop 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 shutdown_connections 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28c0066f60 0x7f28c0069420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28e415bac0 0x7f28e40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e40aef80 0x7f28e40b0220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 --2- 172.21.15.45:0/1448890084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28e4070350 0x7f28e414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 >> 172.21.15.45:0/1448890084 conn(0x7f28e40669a0 msgr2=0x7f28e40b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 shutdown_connections 2024-04-25T10:18:42.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.272+0000 7f28e9200640 1 -- 172.21.15.45:0/1448890084 wait complete. 2024-04-25T10:18:42.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 Processor -- start 2024-04-25T10:18:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 -- start start 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac0aef80 0x7f89ac15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89ac070350 0x7f89ac15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89ac15bac0 0x7f89ac0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac15b080 con 0x7f89ac070350 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac15adb0 con 0x7f89ac15bac0 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89b1000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac15eb60 con 0x7f89ac0aef80 2024-04-25T10:18:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89ac070350 0x7f89ac15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89ac070350 0x7f89ac15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52560/0 (socket says 172.21.15.45:52560) 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 -- 172.21.15.45:0/1127532187 learned_addr learned my addr 172.21.15.45:0/1127532187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a9e00640 1 --2- 172.21.15.45:0/1127532187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac0aef80 0x7f89ac15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89aa800640 1 --2- 172.21.15.45:0/1127532187 >> v2:172.21.15.117:3300/0 conn(0x7f89ac15bac0 0x7f89ac0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 -- 172.21.15.45:0/1127532187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac0aef80 msgr2=0x7f89ac15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 --2- 172.21.15.45:0/1127532187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac0aef80 0x7f89ac15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 -- 172.21.15.45:0/1127532187 >> v2:172.21.15.117:3300/0 conn(0x7f89ac15bac0 msgr2=0x7f89ac0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 --2- 172.21.15.45:0/1127532187 >> v2:172.21.15.117:3300/0 conn(0x7f89ac15bac0 0x7f89ac0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.436+0000 7f89a1400640 1 -- 172.21.15.45:0/1127532187 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac15ee40 con 0x7f89ac070350 2024-04-25T10:18:42.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89a1400640 1 --2- 172.21.15.45:0/1127532187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac070350 0x7f89ac15b580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0cc330 tx=0x7f899401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4020792f2800b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89a9400640 1 -- 172.21.15.45:0/1127532187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940ab5f0 con 0x7f89ac070350 2024-04-25T10:18:42.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89a9400640 1 -- 172.21.15.45:0/1127532187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89940b0070 con 0x7f89ac070350 2024-04-25T10:18:42.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89a9400640 1 -- 172.21.15.45:0/1127532187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940ab9b0 con 0x7f89ac070350 2024-04-25T10:18:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 -- 172.21.15.45:0/1127532187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac070350 msgr2=0x7f89ac15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 --2- 172.21.15.45:0/1127532187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac070350 0x7f89ac15b580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0cc330 tx=0x7f899401c520 comp rx=0 tx=0).stop 2024-04-25T10:18:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 -- 172.21.15.45:0/1127532187 shutdown_connections 2024-04-25T10:18:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 --2- 172.21.15.45:0/1127532187 >> v2:172.21.15.117:3300/0 conn(0x7f89ac15bac0 0x7f89ac0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 --2- 172.21.15.45:0/1127532187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac070350 0x7f89ac15b580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 --2- 172.21.15.45:0/1127532187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac0aef80 0x7f89ac15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 -- 172.21.15.45:0/1127532187 >> 172.21.15.45:0/1127532187 conn(0x7f89ac0669a0 msgr2=0x7f89ac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 -- 172.21.15.45:0/1127532187 shutdown_connections 2024-04-25T10:18:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.440+0000 7f89b1000640 1 -- 172.21.15.45:0/1127532187 wait complete. 2024-04-25T10:18:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 Processor -- start 2024-04-25T10:18:42.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 -- start start 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ac070350 0x7f89ac147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 0x7f89ac155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac15bac0 0x7f89ac156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac06d570 con 0x7f89ac0aef80 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac06d3f0 con 0x7f89ac070350 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac06d6f0 con 0x7f89ac15bac0 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 0x7f89ac155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 0x7f89ac155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52564/0 (socket says 172.21.15.45:52564) 2024-04-25T10:18:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 -- 172.21.15.45:0/934836775 learned_addr learned my addr 172.21.15.45:0/934836775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89aa800640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac15bac0 0x7f89ac156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a9e00640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ac070350 0x7f89ac147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 -- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac15bac0 msgr2=0x7f89ac156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac15bac0 0x7f89ac156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 -- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ac070350 msgr2=0x7f89ac147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ac070350 0x7f89ac147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 -- 172.21.15.45:0/934836775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac14bcc0 con 0x7f89ac0aef80 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a9e00640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ac070350 0x7f89ac147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a1400640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 0x7f89ac155f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f899401abd0 tx=0x7f899401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940be5f0 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89940be790 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940beb40 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac0b00b0 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.444+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac0b0bb0 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.448+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac04ef30 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.448+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89940b43f0 con 0x7f89ac0aef80 2024-04-25T10:18:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.448+0000 7f89a2a00640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f898806b370 0x7f898806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.448+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f899411f7f0 con 0x7f89ac0aef80 2024-04-25T10:18:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.448+0000 7f89a9e00640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f898806b370 0x7f898806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.452+0000 7f89a9e00640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f898806b370 0x7f898806d830 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f899c004660 tx=0x7f899c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.452+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8994016110 con 0x7f89ac0aef80 2024-04-25T10:18:42.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89ac0b2780 con 0x7f89ac0aef80 2024-04-25T10:18:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89a2a00640 1 -- 172.21.15.45:0/934836775 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89940f02a0 con 0x7f89ac0aef80 2024-04-25T10:18:42.619 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:18:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f898806b370 msgr2=0x7f898806d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f898806b370 0x7f898806d830 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f899c004660 tx=0x7f899c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:42.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 msgr2=0x7f89ac155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 0x7f89ac155f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f899401abd0 tx=0x7f899401ad60 comp rx=0 tx=0).stop 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 shutdown_connections 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f898806b370 0x7f898806d830 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89ac15bac0 0x7f89ac156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89ac0aef80 0x7f89ac155f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 --2- 172.21.15.45:0/934836775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89ac070350 0x7f89ac147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 >> 172.21.15.45:0/934836775 conn(0x7f89ac0669a0 msgr2=0x7f89ac0cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 shutdown_connections 2024-04-25T10:18:42.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.616+0000 7f89b1000640 1 -- 172.21.15.45:0/934836775 wait complete. 2024-04-25T10:18:42.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:42.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 Processor -- start 2024-04-25T10:18:42.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 -- start start 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f55cc15dcc0 0x7f55cc162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0670b0 0x7f55cc0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc15f0b0 con 0x7f55cc0b5480 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc161800 con 0x7f55cc15dcc0 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0b3b20 con 0x7f55cc0670b0 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f55cc15dcc0 0x7f55cc162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55c3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f55cc0b5480 0x7f55cc0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38372/0 (socket says 172.21.15.45:38372) 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55c3e00640 1 -- 172.21.15.45:0/913050202 learned_addr learned my addr 172.21.15.45:0/913050202 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d1c00640 1 --2- 172.21.15.45:0/913050202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0670b0 0x7f55cc0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d1200640 1 -- 172.21.15.45:0/913050202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0670b0 msgr2=0x7f55cc0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d1200640 1 --2- 172.21.15.45:0/913050202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0670b0 0x7f55cc0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d1200640 1 -- 172.21.15.45:0/913050202 >> v1:172.21.15.45:6789/0 conn(0x7f55cc0b5480 legacy=0x7f55cc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55d1200640 1 -- 172.21.15.45:0/913050202 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0b3e00 con 0x7f55cc15dcc0 2024-04-25T10:18:42.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.784+0000 7f55cbe00640 1 -- 172.21.15.45:0/913050202 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826596955 0 0) 0x7f55cc15f0b0 con 0x7f55cc0b5480 2024-04-25T10:18:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d1200640 1 --2- 172.21.15.45:0/913050202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc15dcc0 0x7f55cc162250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0097c0 tx=0x7f55bc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc088db5baffe6ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55cbe00640 1 -- 172.21.15.45:0/913050202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0a7070 con 0x7f55cc15dcc0 2024-04-25T10:18:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55cbe00640 1 -- 172.21.15.45:0/913050202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc0b0070 con 0x7f55cc15dcc0 2024-04-25T10:18:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55cbe00640 1 -- 172.21.15.45:0/913050202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0ab890 con 0x7f55cc15dcc0 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 -- 172.21.15.45:0/913050202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc15dcc0 msgr2=0x7f55cc162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 --2- 172.21.15.45:0/913050202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc15dcc0 0x7f55cc162250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0097c0 tx=0x7f55bc01c470 comp rx=0 tx=0).stop 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 -- 172.21.15.45:0/913050202 shutdown_connections 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 --2- 172.21.15.45:0/913050202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0670b0 0x7f55cc0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 --2- 172.21.15.45:0/913050202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc15dcc0 0x7f55cc162250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 -- 172.21.15.45:0/913050202 >> 172.21.15.45:0/913050202 conn(0x7f55cc06ac90 msgr2=0x7f55cc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 -- 172.21.15.45:0/913050202 shutdown_connections 2024-04-25T10:18:42.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.788+0000 7f55d3c00640 1 -- 172.21.15.45:0/913050202 wait complete. 2024-04-25T10:18:42.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 Processor -- start 2024-04-25T10:18:42.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 -- start start 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 0x7f55cc14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0b5480 0x7f55cc15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc15dcc0 0x7f55cc15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d0ec0 con 0x7f55cc0670b0 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d0d40 con 0x7f55cc15dcc0 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d1040 con 0x7f55cc0b5480 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 0x7f55cc14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 0x7f55cc14e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52588/0 (socket says 172.21.15.45:52588) 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 -- 172.21.15.45:0/2661979897 learned_addr learned my addr 172.21.15.45:0/2661979897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55c3e00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0b5480 0x7f55cc15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc15dcc0 0x7f55cc15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 -- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0b5480 msgr2=0x7f55cc15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0b5480 0x7f55cc15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 -- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc15dcc0 msgr2=0x7f55cc15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc15dcc0 0x7f55cc15f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 -- 172.21.15.45:0/2661979897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc150160 con 0x7f55cc0670b0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d1200640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 0x7f55cc14e4d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f55bc01a0a0 tx=0x7f55bc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc006e20 con 0x7f55cc0670b0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc004220 con 0x7f55cc0670b0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc14ea10 con 0x7f55cc0670b0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc003620 con 0x7f55cc0670b0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.792+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc157170 con 0x7f55cc0670b0 2024-04-25T10:18:42.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.796+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc04ef30 con 0x7f55cc0670b0 2024-04-25T10:18:42.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.796+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55bc0037c0 con 0x7f55cc0670b0 2024-04-25T10:18:42.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.796+0000 7f55c9600640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.796+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f55bc11f7a0 con 0x7f55cc0670b0 2024-04-25T10:18:42.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.796+0000 7f55c3e00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.796+0000 7f55c3e00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f55a8066f60 0x7f55a8069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0b16d0 tx=0x7f55b4007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.800+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55bc016090 con 0x7f55cc0670b0 2024-04-25T10:18:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.964+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55cc071bb0 con 0x7f55cc0670b0 2024-04-25T10:18:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.964+0000 7f55c9600640 1 -- 172.21.15.45:0/2661979897 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55bc0f0780 con 0x7f55cc0670b0 2024-04-25T10:18:42.968 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.964+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f55a8066f60 msgr2=0x7f55a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.964+0000 7f55d3c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f55a8066f60 0x7f55a8069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0b16d0 tx=0x7f55b4007c10 comp rx=0 tx=0).stop 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 msgr2=0x7f55cc14e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 0x7f55cc14e4d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f55bc01a0a0 tx=0x7f55bc01acb0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 shutdown_connections 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f55a8066f60 0x7f55a8069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55cc15dcc0 0x7f55cc15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55cc0b5480 0x7f55cc15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 --2- 172.21.15.45:0/2661979897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55cc0670b0 0x7f55cc14e4d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 >> 172.21.15.45:0/2661979897 conn(0x7f55cc06ac90 msgr2=0x7f55cc153020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 shutdown_connections 2024-04-25T10:18:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:42.968+0000 7f55d3c00640 1 -- 172.21.15.45:0/2661979897 wait complete. 2024-04-25T10:18:42.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.124+0000 7fabbf200640 1 Processor -- start 2024-04-25T10:18:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.124+0000 7fabbf200640 1 -- start start 2024-04-25T10:18:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabbf200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80669a0 0x7fabb80cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabbf200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb80ced90 con 0x7fabb80afbf0 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabbf200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb80cf070 con 0x7fabb806d480 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabbf200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80cf350 con 0x7fabb80669a0 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabafe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fabb80afbf0 0x7fabb8066d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38374/0 (socket says 172.21.15.45:38374) 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabafe00640 1 -- 172.21.15.45:0/3051000116 learned_addr learned my addr 172.21.15.45:0/3051000116 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7e00640 1 --2- 172.21.15.45:0/3051000116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80669a0 0x7fabb80cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263794366 0 0) 0x7fabb80ced90 con 0x7fabb80afbf0 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9c003530 con 0x7fabb80afbf0 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563188591 0 0) 0x7fabb80cf070 con 0x7fabb806d480 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb80ced90 con 0x7fabb806d480 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7e00640 1 -- 172.21.15.45:0/3051000116 >> v1:172.21.15.117:6789/0 conn(0x7fabb806d480 legacy=0x7fabb806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7e00640 1 -- 172.21.15.45:0/3051000116 >> v1:172.21.15.45:6789/0 conn(0x7fabb80afbf0 legacy=0x7fabb8066d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4137292459 0 0) 0x7fab9c003530 con 0x7fabb80afbf0 2024-04-25T10:18:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.128+0000 7fabb7e00640 1 -- 172.21.15.45:0/3051000116 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80cf630 con 0x7fabb80669a0 2024-04-25T10:18:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabb7e00640 1 --2- 172.21.15.45:0/3051000116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80669a0 0x7fabb80cd2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faba80097f0 tx=0x7faba801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592ca8632114f955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80b0070 con 0x7fabb80669a0 2024-04-25T10:18:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba80ab400 con 0x7fabb80669a0 2024-04-25T10:18:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabb7400640 1 -- 172.21.15.45:0/3051000116 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0001680 con 0x7fabb80669a0 2024-04-25T10:18:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 -- 172.21.15.45:0/3051000116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80669a0 msgr2=0x7fabb80cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 --2- 172.21.15.45:0/3051000116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80669a0 0x7fabb80cd2b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faba80097f0 tx=0x7faba801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 -- 172.21.15.45:0/3051000116 shutdown_connections 2024-04-25T10:18:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 --2- 172.21.15.45:0/3051000116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80669a0 0x7fabb80cd2b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 -- 172.21.15.45:0/3051000116 >> 172.21.15.45:0/3051000116 conn(0x7fabb80b24f0 msgr2=0x7fabb80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 -- 172.21.15.45:0/3051000116 shutdown_connections 2024-04-25T10:18:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 -- 172.21.15.45:0/3051000116 wait complete. 2024-04-25T10:18:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 Processor -- start 2024-04-25T10:18:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.132+0000 7fabbf200640 1 -- start start 2024-04-25T10:18:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabb80669a0 0x7fabb815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 0x7fabb815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80afbf0 0x7fabb80b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80cfe30 con 0x7fabb806d480 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80af950 con 0x7fabb80669a0 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb80b39a0 con 0x7fabb80afbf0 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 0x7fabb815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 0x7fabb815c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52590/0 (socket says 172.21.15.45:52590) 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 -- 172.21.15.45:0/3007438718 learned_addr learned my addr 172.21.15.45:0/3007438718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabb7e00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabb80669a0 0x7fabb815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbd200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80afbf0 0x7fabb80b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 -- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80afbf0 msgr2=0x7fabb80b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80afbf0 0x7fabb80b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 -- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabb80669a0 msgr2=0x7fabb815edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabb80669a0 0x7fabb815edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 -- 172.21.15.45:0/3007438718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb815f330 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabafe00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 0x7fabb815c500 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faba000c730 tx=0x7faba000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0017070 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0005b00 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba0007070 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb815ca40 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb80b3ed0 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb804ef30 con 0x7fabb806d480 2024-04-25T10:18:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.136+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faba001a070 con 0x7fabb806d480 2024-04-25T10:18:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.140+0000 7fabb4c00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab94066f60 0x7fab94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.140+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7faba00844c0 con 0x7fabb806d480 2024-04-25T10:18:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.140+0000 7fabb7e00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab94066f60 0x7fab94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.140+0000 7fabb7e00640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab94066f60 0x7fab94069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faba80ad000 tx=0x7faba80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.144+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faba0055540 con 0x7fabb806d480 2024-04-25T10:18:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.340+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabb80b0140 con 0x7fabb806d480 2024-04-25T10:18:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabb4c00640 1 -- 172.21.15.45:0/3007438718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7faba0054e40 con 0x7fabb806d480 2024-04-25T10:18:43.346 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:43.346 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab94066f60 msgr2=0x7fab94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab94066f60 0x7fab94069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faba80ad000 tx=0x7faba80a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 msgr2=0x7fabb815c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 0x7fabb815c500 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faba000c730 tx=0x7faba000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 shutdown_connections 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab94066f60 0x7fab94069420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabb80afbf0 0x7fabb80b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabb806d480 0x7fabb815c500 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 --2- 172.21.15.45:0/3007438718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabb80669a0 0x7fabb815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 >> 172.21.15.45:0/3007438718 conn(0x7fabb80b24f0 msgr2=0x7fabb81593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 shutdown_connections 2024-04-25T10:18:43.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.344+0000 7fabbf200640 1 -- 172.21.15.45:0/3007438718 wait complete. 2024-04-25T10:18:43.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.504+0000 7fbf2d800640 1 Processor -- start 2024-04-25T10:18:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.504+0000 7fbf2d800640 1 -- start start 2024-04-25T10:18:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.504+0000 7fbf2d800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf2815b9e0 0x7fbf280cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.504+0000 7fbf2d800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280ccfb0 con 0x7fbf280d1c30 2024-04-25T10:18:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.504+0000 7fbf2d800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280cd290 con 0x7fbf2815bdc0 2024-04-25T10:18:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.504+0000 7fbf2d800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf280cd570 con 0x7fbf2815b9e0 2024-04-25T10:18:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf27400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf280d1c30 0x7fbf280c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38386/0 (socket says 172.21.15.45:38386) 2024-04-25T10:18:43.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf27400640 1 -- 172.21.15.45:0/566128815 learned_addr learned my addr 172.21.15.45:0/566128815 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26a00640 1 --2- 172.21.15.45:0/566128815 >> v2:172.21.15.117:3300/0 conn(0x7fbf2815b9e0 0x7fbf280cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26000640 1 -- 172.21.15.45:0/566128815 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115968541 0 0) 0x7fbf280ccfb0 con 0x7fbf280d1c30 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26000640 1 -- 172.21.15.45:0/566128815 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c003530 con 0x7fbf280d1c30 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26a00640 1 -- 172.21.15.45:0/566128815 >> v1:172.21.15.117:6789/0 conn(0x7fbf2815bdc0 legacy=0x7fbf280d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26000640 1 -- 172.21.15.45:0/566128815 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951447726 0 0) 0x7fbf280cd290 con 0x7fbf2815bdc0 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26a00640 1 -- 172.21.15.45:0/566128815 >> v1:172.21.15.45:6789/0 conn(0x7fbf280d1c30 legacy=0x7fbf280c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26a00640 1 -- 172.21.15.45:0/566128815 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf280cd850 con 0x7fbf2815b9e0 2024-04-25T10:18:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.508+0000 7fbf26a00640 1 --2- 172.21.15.45:0/566128815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815b9e0 0x7fbf280cc410 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf100097c0 tx=0x7fbf1001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7949188bf6b2811f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf26000640 1 -- 172.21.15.45:0/566128815 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf100b0070 con 0x7fbf2815b9e0 2024-04-25T10:18:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf26000640 1 -- 172.21.15.45:0/566128815 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf100ab600 con 0x7fbf2815b9e0 2024-04-25T10:18:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf26000640 1 -- 172.21.15.45:0/566128815 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf100ab9b0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- 172.21.15.45:0/566128815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815b9e0 msgr2=0x7fbf280cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 --2- 172.21.15.45:0/566128815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815b9e0 0x7fbf280cc410 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf100097c0 tx=0x7fbf1001c410 comp rx=0 tx=0).stop 2024-04-25T10:18:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- 172.21.15.45:0/566128815 shutdown_connections 2024-04-25T10:18:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 --2- 172.21.15.45:0/566128815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815b9e0 0x7fbf280cc410 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- 172.21.15.45:0/566128815 >> 172.21.15.45:0/566128815 conn(0x7fbf2809a9d0 msgr2=0x7fbf2809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- 172.21.15.45:0/566128815 shutdown_connections 2024-04-25T10:18:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- 172.21.15.45:0/566128815 wait complete. 2024-04-25T10:18:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 Processor -- start 2024-04-25T10:18:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- start start 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf280d1c30 0x7fbf2814b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 0x7fbf2815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815bdc0 0x7fbf28153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280c6d40 con 0x7fbf2815b9e0 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280c6bc0 con 0x7fbf2815bdc0 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf2d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28169840 con 0x7fbf280d1c30 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf1e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 0x7fbf2815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf1e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 0x7fbf2815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52606/0 (socket says 172.21.15.45:52606) 2024-04-25T10:18:43.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf1e000640 1 -- 172.21.15.45:0/4045731966 learned_addr learned my addr 172.21.15.45:0/4045731966 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf26a00640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf280d1c30 0x7fbf2814b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.512+0000 7fbf27400640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815bdc0 0x7fbf28153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1e000640 1 -- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf280d1c30 msgr2=0x7fbf2814b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1e000640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf280d1c30 0x7fbf2814b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1e000640 1 -- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815bdc0 msgr2=0x7fbf28153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1e000640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815bdc0 0x7fbf28153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1e000640 1 -- 172.21.15.45:0/4045731966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2815dd00 con 0x7fbf2815b9e0 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1e000640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 0x7fbf2815d320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c500 tx=0x7fbf1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18017070 con 0x7fbf2815b9e0 2024-04-25T10:18:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf18003600 con 0x7fbf2815b9e0 2024-04-25T10:18:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180039f0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2815dfe0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2815e5a0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2804ef30 con 0x7fbf2815b9e0 2024-04-25T10:18:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf1801a070 con 0x7fbf2815b9e0 2024-04-25T10:18:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1f400640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf04066f60 0x7fbf04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbf18084510 con 0x7fbf2815b9e0 2024-04-25T10:18:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.516+0000 7fbf26a00640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf04066f60 0x7fbf04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.520+0000 7fbf26a00640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf04066f60 0x7fbf04069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf100ad000 tx=0x7fbf10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.520+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf180555b0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.684+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf280c4fc0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.684+0000 7fbf1f400640 1 -- 172.21.15.45:0/4045731966 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fbf18054eb0 con 0x7fbf2815b9e0 2024-04-25T10:18:43.689 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:43.689 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:42.891612+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:43.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf04066f60 msgr2=0x7fbf04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf04066f60 0x7fbf04069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf100ad000 tx=0x7fbf10008040 comp rx=0 tx=0).stop 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 msgr2=0x7fbf2815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 0x7fbf2815d320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf1800c500 tx=0x7fbf1800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 shutdown_connections 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf04066f60 0x7fbf04069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf2815bdc0 0x7fbf28153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf2815b9e0 0x7fbf2815d320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 --2- 172.21.15.45:0/4045731966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf280d1c30 0x7fbf2814b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 >> 172.21.15.45:0/4045731966 conn(0x7fbf2809a9d0 msgr2=0x7fbf280c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 shutdown_connections 2024-04-25T10:18:43.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.688+0000 7fbf2d800640 1 -- 172.21.15.45:0/4045731966 wait complete. 2024-04-25T10:18:43.711 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:18:43.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.848+0000 7f43c1000640 1 Processor -- start 2024-04-25T10:18:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.848+0000 7f43c1000640 1 -- start start 2024-04-25T10:18:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43c1000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f43bc0c6a20 0x7f43bc166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43c1000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0ccfb0 con 0x7f43bc0ccb20 2024-04-25T10:18:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43c1000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0cd290 con 0x7f43bc15b9e0 2024-04-25T10:18:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43c1000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc0cd570 con 0x7f43bc0c6a20 2024-04-25T10:18:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f43bc0ccb20 0x7f43bc0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38396/0 (socket says 172.21.15.45:38396) 2024-04-25T10:18:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 -- 172.21.15.45:0/2488175556 learned_addr learned my addr 172.21.15.45:0/2488175556 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43ba800640 1 --2- 172.21.15.45:0/2488175556 >> v2:172.21.15.117:3300/0 conn(0x7f43bc0c6a20 0x7f43bc166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234990277 0 0) 0x7f43bc0ccfb0 con 0x7f43bc0ccb20 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a0003530 con 0x7f43bc0ccb20 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895838796 0 0) 0x7f43a0003530 con 0x7f43bc0ccb20 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 >> v2:172.21.15.117:3300/0 conn(0x7f43bc0c6a20 msgr2=0x7f43bc166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 --2- 172.21.15.45:0/2488175556 >> v2:172.21.15.117:3300/0 conn(0x7f43bc0c6a20 0x7f43bc166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 >> v1:172.21.15.117:6789/0 conn(0x7f43bc15b9e0 legacy=0x7f43bc0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0cd850 con 0x7f43bc0ccb20 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f43a4001680 con 0x7f43bc0ccb20 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 >> v1:172.21.15.45:6789/0 conn(0x7f43bc0ccb20 legacy=0x7f43bc0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00039c0 0x7f43a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43a00043c0 0x7f43a0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43a0005090 0x7f43a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0cd570 con 0x7f43a00043c0 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a000bff0 con 0x7f43a00039c0 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a000c1f0 con 0x7f43a0005090 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43a00043c0 0x7f43a0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b9e00640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00039c0 0x7f43a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43ba800640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43a0005090 0x7f43a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 -- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43a0005090 msgr2=0x7f43a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43a0005090 0x7f43a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 -- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00039c0 msgr2=0x7f43a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00039c0 0x7f43a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43b1400640 1 -- 172.21.15.45:0/2488175556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0003530 con 0x7f43a00043c0 2024-04-25T10:18:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43ba800640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43a0005090 0x7f43a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.852+0000 7f43ba800640 1 -- 172.21.15.45:0/2488175556 reap_dead start 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43b1400640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43a00043c0 0x7f43a0004890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0d2a00 tx=0x7f43a401cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a6bed953dd86d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4007cb0 con 0x7f43a00043c0 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a40a9020 con 0x7f43a00043c0 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43b9400640 1 -- 172.21.15.45:0/2488175556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4005ae0 con 0x7f43a00043c0 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 -- 172.21.15.45:0/2488175556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43a00043c0 msgr2=0x7f43a0004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43a00043c0 0x7f43a0004890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0d2a00 tx=0x7f43a401cb40 comp rx=0 tx=0).stop 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 -- 172.21.15.45:0/2488175556 shutdown_connections 2024-04-25T10:18:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 --2- 172.21.15.45:0/2488175556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43a00043c0 0x7f43a0004890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 -- 172.21.15.45:0/2488175556 >> 172.21.15.45:0/2488175556 conn(0x7f43bc09a9d0 msgr2=0x7f43bc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 -- 172.21.15.45:0/2488175556 shutdown_connections 2024-04-25T10:18:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 -- 172.21.15.45:0/2488175556 wait complete. 2024-04-25T10:18:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 Processor -- start 2024-04-25T10:18:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.856+0000 7f43c1000640 1 -- start start 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00043c0 0x7f43bc0b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43bc152270 0x7f43bc0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 0x7f43bc15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc15bdc0 con 0x7f43bc15cfc0 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0cf740 con 0x7f43a00043c0 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc15bf40 con 0x7f43bc152270 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 0x7f43bc15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 0x7f43bc15d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52626/0 (socket says 172.21.15.45:52626) 2024-04-25T10:18:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 -- 172.21.15.45:0/242950676 learned_addr learned my addr 172.21.15.45:0/242950676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43b9e00640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00043c0 0x7f43bc0b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 -- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43bc152270 msgr2=0x7f43bc0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43bc152270 0x7f43bc0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 -- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00043c0 msgr2=0x7f43bc0b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00043c0 0x7f43bc0b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 -- 172.21.15.45:0/242950676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0b4c50 con 0x7f43bc15cfc0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43b9e00640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00043c0 0x7f43bc0b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43ba800640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 0x7f43bc15d470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f43bc070e60 tx=0x7f43b4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4010e60 con 0x7f43bc15cfc0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40039a0 con 0x7f43bc15cfc0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4003d70 con 0x7f43bc15cfc0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc0c8d90 con 0x7f43bc15cfc0 2024-04-25T10:18:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc0cba50 con 0x7f43bc15cfc0 2024-04-25T10:18:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc04ef30 con 0x7f43bc15cfc0 2024-04-25T10:18:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.860+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43b401e020 con 0x7f43bc15cfc0 2024-04-25T10:18:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.864+0000 7f43b2a00640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4398066f90 0x7f4398069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.864+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f43b4085790 con 0x7f43bc15cfc0 2024-04-25T10:18:43.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.864+0000 7f43b9e00640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4398066f90 0x7f4398069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.864+0000 7f43b9e00640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4398066f90 0x7f4398069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005630 tx=0x7f43ac007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:43.868+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43b4025020 con 0x7f43bc15cfc0 2024-04-25T10:18:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43bc0d2750 con 0x7f43bc15cfc0 2024-04-25T10:18:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43b2a00640 1 -- 172.21.15.45:0/242950676 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f43b4056810 con 0x7f43bc15cfc0 2024-04-25T10:18:44.066 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:44.066 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4398066f90 msgr2=0x7f4398069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4398066f90 0x7f4398069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005630 tx=0x7f43ac007cd0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 msgr2=0x7f43bc15d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 0x7f43bc15d470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f43bc070e60 tx=0x7f43b4010590 comp rx=0 tx=0).stop 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 shutdown_connections 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4398066f90 0x7f4398069450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43bc15cfc0 0x7f43bc15d470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43bc152270 0x7f43bc0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 --2- 172.21.15.45:0/242950676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a00043c0 0x7f43bc0b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 >> 172.21.15.45:0/242950676 conn(0x7f43bc09a9d0 msgr2=0x7f43bc06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 shutdown_connections 2024-04-25T10:18:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.064+0000 7f43c1000640 1 -- 172.21.15.45:0/242950676 wait complete. 2024-04-25T10:18:44.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.224+0000 7fab10000640 1 Processor -- start 2024-04-25T10:18:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.224+0000 7fab10000640 1 -- start start 2024-04-25T10:18:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab10000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab080d07b0 0x7fab0806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab10000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab080ccaa0 con 0x7fab0815df60 2024-04-25T10:18:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab10000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab080ccd80 con 0x7fab080d0350 2024-04-25T10:18:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab10000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0815e340 con 0x7fab080d07b0 2024-04-25T10:18:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0d600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fab0815df60 0x7fab080cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38410/0 (socket says 172.21.15.45:38410) 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0d600640 1 -- 172.21.15.45:0/1113671054 learned_addr learned my addr 172.21.15.45:0/1113671054 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 --2- 172.21.15.45:0/1113671054 >> v2:172.21.15.117:3300/0 conn(0x7fab080d07b0 0x7fab0806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868258285 0 0) 0x7fab080ccaa0 con 0x7fab0815df60 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaf0003530 con 0x7fab0815df60 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145721109 0 0) 0x7fab080ccd80 con 0x7fab080d0350 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab080ccaa0 con 0x7fab080d0350 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3907053155 0 0) 0x7faaf0003530 con 0x7fab0815df60 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 >> v2:172.21.15.117:3300/0 conn(0x7fab080d07b0 msgr2=0x7fab0806b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 --2- 172.21.15.45:0/1113671054 >> v2:172.21.15.117:3300/0 conn(0x7fab080d07b0 0x7fab0806b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 >> v1:172.21.15.117:6789/0 conn(0x7fab080d0350 legacy=0x7fab0806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080cd060 con 0x7fab0815df60 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7faafc001680 con 0x7fab0815df60 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 --2- 172.21.15.45:0/1113671054 >> v2:172.21.15.117:3300/0 conn(0x7fab080d07b0 0x7fab0806b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 >> v1:172.21.15.45:6789/0 conn(0x7fab0815df60 legacy=0x7fab080cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0003ba0 0x7faaf0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaf0004810 0x7faaf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0005220 0x7faaf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf000c120 con 0x7faaf0005220 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf000c2d0 con 0x7faaf0003ba0 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf000c4d0 con 0x7faaf0004810 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0005220 0x7faaf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0d600640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0003ba0 0x7faaf0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0cc00640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaf0004810 0x7faaf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 -- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaf0004810 msgr2=0x7faaf0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaf0004810 0x7faaf0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 -- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0003ba0 msgr2=0x7faaf0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0003ba0 0x7faaf0003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0e000640 1 -- 172.21.15.45:0/1113671054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf0003530 con 0x7faaf0005220 2024-04-25T10:18:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.228+0000 7fab0cc00640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaf0004810 0x7faaf0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab0e000640 1 -- 172.21.15.45:0/1113671054 reap_dead start 2024-04-25T10:18:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab0e000640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0005220 0x7faaf000b9d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab0000e350 tx=0x7fab0001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3550e01d4c109b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00009de0 con 0x7faaf0005220 2024-04-25T10:18:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab000039a0 con 0x7faaf0005220 2024-04-25T10:18:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab07e00640 1 -- 172.21.15.45:0/1113671054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00003d90 con 0x7faaf0005220 2024-04-25T10:18:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 -- 172.21.15.45:0/1113671054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0005220 msgr2=0x7faaf000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0005220 0x7faaf000b9d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab0000e350 tx=0x7fab0001ce80 comp rx=0 tx=0).stop 2024-04-25T10:18:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 -- 172.21.15.45:0/1113671054 shutdown_connections 2024-04-25T10:18:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 --2- 172.21.15.45:0/1113671054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0005220 0x7faaf000b9d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 -- 172.21.15.45:0/1113671054 >> 172.21.15.45:0/1113671054 conn(0x7fab080d1910 msgr2=0x7fab080d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 -- 172.21.15.45:0/1113671054 shutdown_connections 2024-04-25T10:18:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.232+0000 7fab10000640 1 -- 172.21.15.45:0/1113671054 wait complete. 2024-04-25T10:18:44.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 Processor -- start 2024-04-25T10:18:44.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- start start 2024-04-25T10:18:44.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0005220 0x7fab080c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab0814fa50 0x7fab080c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 0x7fab08160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab080cd620 con 0x7fab0815fcf0 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab080d0e70 con 0x7faaf0005220 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab080d0ff0 con 0x7fab0814fa50 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 0x7fab08160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 0x7fab08160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52644/0 (socket says 172.21.15.45:52644) 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 -- 172.21.15.45:0/141179032 learned_addr learned my addr 172.21.15.45:0/141179032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0d600640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0005220 0x7fab080c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0cc00640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab0814fa50 0x7fab080c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 -- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab0814fa50 msgr2=0x7fab080c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab0814fa50 0x7fab080c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 -- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0005220 msgr2=0x7fab080c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0005220 0x7fab080c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 -- 172.21.15.45:0/141179032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080c5420 con 0x7fab0815fcf0 2024-04-25T10:18:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab0e000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 0x7fab08160170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab0001d720 tx=0x7fab00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00009de0 con 0x7fab0815fcf0 2024-04-25T10:18:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab000b6440 con 0x7fab0815fcf0 2024-04-25T10:18:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab000b6810 con 0x7fab0815fcf0 2024-04-25T10:18:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab080c5700 con 0x7fab0815fcf0 2024-04-25T10:18:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab080c5cc0 con 0x7fab0815fcf0 2024-04-25T10:18:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.236+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0804ef30 con 0x7fab0815fcf0 2024-04-25T10:18:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.240+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab000bf060 con 0x7fab0815fcf0 2024-04-25T10:18:44.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.240+0000 7fab05600640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faae8066f60 0x7faae8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.240+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fab0011f530 con 0x7fab0815fcf0 2024-04-25T10:18:44.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.240+0000 7fab0d600640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faae8066f60 0x7faae8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.240+0000 7fab0d600640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faae8066f60 0x7faae8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faafc001130 tx=0x7faafc000ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.244+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab00016b90 con 0x7fab0815fcf0 2024-04-25T10:18:44.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.436+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab080d1e90 con 0x7fab0815fcf0 2024-04-25T10:18:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.436+0000 7fab05600640 1 -- 172.21.15.45:0/141179032 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fab000f0510 con 0x7fab0815fcf0 2024-04-25T10:18:44.441 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:44.441 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3030029059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3030029059},{"type":"v1","addr":"172.21.15.45:6834","nonce":3030029059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2111808227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2111808227},{"type":"v1","addr":"172.21.15.45:6835","nonce":2111808227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2512615869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2512615869},{"type":"v1","addr":"172.21.15.117:6836","nonce":2512615869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:02.077189+0000","modified":"2024-04-25T10:18:05.087771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1341306304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1341306304},{"type":"v1","addr":"172.21.15.117:6837","nonce":1341306304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:18:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faae8066f60 msgr2=0x7faae8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faae8066f60 0x7faae8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faafc001130 tx=0x7faafc000ee0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 msgr2=0x7fab08160170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 0x7fab08160170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab0001d720 tx=0x7fab00009750 comp rx=0 tx=0).stop 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 shutdown_connections 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faae8066f60 0x7faae8069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab0815fcf0 0x7fab08160170 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab0814fa50 0x7fab080c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 --2- 172.21.15.45:0/141179032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaf0005220 0x7fab080c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 >> 172.21.15.45:0/141179032 conn(0x7fab080d1910 msgr2=0x7fab0815c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 shutdown_connections 2024-04-25T10:18:44.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.440+0000 7fab10000640 1 -- 172.21.15.45:0/141179032 wait complete. 2024-04-25T10:18:44.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.600+0000 7f723f800640 1 Processor -- start 2024-04-25T10:18:44.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.600+0000 7f723f800640 1 -- start start 2024-04-25T10:18:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.600+0000 7f723f800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f723815df60 0x7f72380cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.600+0000 7f723f800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72380ccaa0 con 0x7f72380d0180 2024-04-25T10:18:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723f800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72380ccd80 con 0x7f723806aea0 2024-04-25T10:18:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723f800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723815e340 con 0x7f723815df60 2024-04-25T10:18:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f722fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f72380d0180 0x7f72380d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38424/0 (socket says 172.21.15.45:38424) 2024-04-25T10:18:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f722fe00640 1 -- 172.21.15.45:0/3617031088 learned_addr learned my addr 172.21.15.45:0/3617031088 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:44.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723ce00640 1 --2- 172.21.15.45:0/3617031088 >> v2:172.21.15.117:3300/0 conn(0x7f723815df60 0x7f72380cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f7237e00640 1 -- 172.21.15.45:0/3617031088 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506896917 0 0) 0x7f72380ccaa0 con 0x7f72380d0180 2024-04-25T10:18:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f7237e00640 1 -- 172.21.15.45:0/3617031088 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f721c003530 con 0x7f72380d0180 2024-04-25T10:18:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723ce00640 1 -- 172.21.15.45:0/3617031088 >> v1:172.21.15.117:6789/0 conn(0x7f723806aea0 legacy=0x7f723806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723ce00640 1 -- 172.21.15.45:0/3617031088 >> v1:172.21.15.45:6789/0 conn(0x7f72380d0180 legacy=0x7f72380d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f7237e00640 1 -- 172.21.15.45:0/3617031088 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 198481617 0 0) 0x7f721c003530 con 0x7f72380d0180 2024-04-25T10:18:44.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723ce00640 1 -- 172.21.15.45:0/3617031088 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380cd060 con 0x7f723815df60 2024-04-25T10:18:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.604+0000 7f723ce00640 1 --2- 172.21.15.45:0/3617031088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f72380cfc40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72280097c0 tx=0x7f722801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512f2feb9d2ab672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f7237e00640 1 -- 172.21.15.45:0/3617031088 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280b0070 con 0x7f723815df60 2024-04-25T10:18:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f7237e00640 1 -- 172.21.15.45:0/3617031088 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723806c1d0 con 0x7f723815df60 2024-04-25T10:18:44.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f7237e00640 1 -- 172.21.15.45:0/3617031088 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220001680 con 0x7f723815df60 2024-04-25T10:18:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- 172.21.15.45:0/3617031088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 msgr2=0x7f72380cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 --2- 172.21.15.45:0/3617031088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f72380cfc40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72280097c0 tx=0x7f722801c330 comp rx=0 tx=0).stop 2024-04-25T10:18:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- 172.21.15.45:0/3617031088 shutdown_connections 2024-04-25T10:18:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 --2- 172.21.15.45:0/3617031088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f72380cfc40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- 172.21.15.45:0/3617031088 >> 172.21.15.45:0/3617031088 conn(0x7f72380d1910 msgr2=0x7f72380d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- 172.21.15.45:0/3617031088 shutdown_connections 2024-04-25T10:18:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- 172.21.15.45:0/3617031088 wait complete. 2024-04-25T10:18:44.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 Processor -- start 2024-04-25T10:18:44.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- start start 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 0x7f723814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72380d0180 0x7f7238156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f7238156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380d0ff0 con 0x7f723806aea0 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380d0e70 con 0x7f723815df60 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723f800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723816ba30 con 0x7f72380d0180 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 0x7f723814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 0x7f723814ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52650/0 (socket says 172.21.15.45:52650) 2024-04-25T10:18:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.608+0000 7f723ce00640 1 -- 172.21.15.45:0/3468766547 learned_addr learned my addr 172.21.15.45:0/3468766547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f722fe00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72380d0180 0x7f7238156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723d800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f7238156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723ce00640 1 -- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72380d0180 msgr2=0x7f7238156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723ce00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72380d0180 0x7f7238156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723ce00640 1 -- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 msgr2=0x7f7238156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723ce00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f7238156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723ce00640 1 -- 172.21.15.45:0/3468766547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723814f440 con 0x7f723806aea0 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f722fe00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72380d0180 0x7f7238156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723d800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f7238156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723ce00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 0x7f723814ef00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f72280ae040 tx=0x7f72280190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280b9020 con 0x7f723806aea0 2024-04-25T10:18:44.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72280038a0 con 0x7f723806aea0 2024-04-25T10:18:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280b0040 con 0x7f723806aea0 2024-04-25T10:18:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7238155320 con 0x7f723806aea0 2024-04-25T10:18:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380c5830 con 0x7f723806aea0 2024-04-25T10:18:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723804ef30 con 0x7f723806aea0 2024-04-25T10:18:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72280bc020 con 0x7f723806aea0 2024-04-25T10:18:44.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f7235600640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7214066f60 0x7f7214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.612+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f722811ef80 con 0x7f723806aea0 2024-04-25T10:18:44.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.616+0000 7f722fe00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7214066f60 0x7f7214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.616+0000 7f722fe00640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7214066f60 0x7f7214069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7220001390 tx=0x7f7220009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.616+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72280c3030 con 0x7f723806aea0 2024-04-25T10:18:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.780+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7238159af0 con 0x7f723806aea0 2024-04-25T10:18:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f7235600640 1 -- 172.21.15.45:0/3468766547 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f7228015fb0 con 0x7f723806aea0 2024-04-25T10:18:44.786 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:42.891612+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:59.922144+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:01.069700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7214066f60 msgr2=0x7f7214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7214066f60 0x7f7214069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7220001390 tx=0x7f7220009040 comp rx=0 tx=0).stop 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 msgr2=0x7f723814ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 0x7f723814ef00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f72280ae040 tx=0x7f72280190d0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 shutdown_connections 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7214066f60 0x7f7214069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f723815df60 0x7f7238156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72380d0180 0x7f7238156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 --2- 172.21.15.45:0/3468766547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f723806aea0 0x7f723814ef00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 >> 172.21.15.45:0/3468766547 conn(0x7f72380d1910 msgr2=0x7f723806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 shutdown_connections 2024-04-25T10:18:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.784+0000 7f723f800640 1 -- 172.21.15.45:0/3468766547 wait complete. 2024-04-25T10:18:44.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:18:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 Processor -- start 2024-04-25T10:18:44.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 -- start start 2024-04-25T10:18:44.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49e015bc40 0x7f49e006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49e00cf930 0x7f49e00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00b2770 con 0x7f49e00cda10 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e006c670 con 0x7f49e00cf930 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49e5a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e006c950 con 0x7f49e015bc40 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49df400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49e00cf930 0x7f49e00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49d7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f49e00cda10 0x7f49e00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40118/0 (socket says 172.21.15.45:40118) 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49d7e00640 1 -- 172.21.15.45:0/2554361864 learned_addr learned my addr 172.21.15.45:0/2554361864 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49df400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f49e00cf930 0x7f49e00cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42524/0 (socket says 172.21.15.45:42524) 2024-04-25T10:18:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49dea00640 1 --2- 172.21.15.45:0/2554361864 >> v2:172.21.15.117:3300/0 conn(0x7f49e015bc40 0x7f49e006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49df400640 1 -- 172.21.15.45:0/2554361864 >> v2:172.21.15.117:3300/0 conn(0x7f49e015bc40 msgr2=0x7f49e006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49df400640 1 --2- 172.21.15.45:0/2554361864 >> v2:172.21.15.117:3300/0 conn(0x7f49e015bc40 0x7f49e006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49df400640 1 -- 172.21.15.45:0/2554361864 >> v1:172.21.15.45:6789/0 conn(0x7f49e00cda10 legacy=0x7f49e00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49df400640 1 -- 172.21.15.45:0/2554361864 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e006cc30 con 0x7f49e00cf930 2024-04-25T10:18:44.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.948+0000 7f49dea00640 1 --2- 172.21.15.45:0/2554361864 >> v2:172.21.15.117:3300/0 conn(0x7f49e015bc40 0x7f49e006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49df400640 1 --2- 172.21.15.45:0/2554361864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00cfdc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f49e00d0800 tx=0x7f49d801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399bb25053417b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49de000640 1 -- 172.21.15.45:0/2554361864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80b2070 con 0x7f49e00cf930 2024-04-25T10:18:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49de000640 1 -- 172.21.15.45:0/2554361864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d80a5760 con 0x7f49e00cf930 2024-04-25T10:18:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49de000640 1 -- 172.21.15.45:0/2554361864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80a5b30 con 0x7f49e00cf930 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 -- 172.21.15.45:0/2554361864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 msgr2=0x7f49e00cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 --2- 172.21.15.45:0/2554361864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00cfdc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f49e00d0800 tx=0x7f49d801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 -- 172.21.15.45:0/2554361864 shutdown_connections 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 --2- 172.21.15.45:0/2554361864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00cfdc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 --2- 172.21.15.45:0/2554361864 >> v2:172.21.15.117:3300/0 conn(0x7f49e015bc40 0x7f49e006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 -- 172.21.15.45:0/2554361864 >> 172.21.15.45:0/2554361864 conn(0x7f49e0066b70 msgr2=0x7f49e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 -- 172.21.15.45:0/2554361864 shutdown_connections 2024-04-25T10:18:44.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.952+0000 7f49e5a00640 1 -- 172.21.15.45:0/2554361864 wait complete. 2024-04-25T10:18:44.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 Processor -- start 2024-04-25T10:18:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 -- start start 2024-04-25T10:18:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49e00cda10 0x7f49e015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49e015bc40 0x7f49e015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00b3850 con 0x7f49e00cf930 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00b36d0 con 0x7f49e00cda10 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00b39d0 con 0x7f49e015bc40 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42538/0 (socket says 172.21.15.45:42538) 2024-04-25T10:18:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 -- 172.21.15.45:0/202622059 learned_addr learned my addr 172.21.15.45:0/202622059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49dea00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49e00cda10 0x7f49e015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49df400640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49e015bc40 0x7f49e015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 -- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49e015bc40 msgr2=0x7f49e015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49e015bc40 0x7f49e015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 -- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49e00cda10 msgr2=0x7f49e015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49e00cda10 0x7f49e015de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 -- 172.21.15.45:0/202622059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e015cde0 con 0x7f49e00cf930 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49dea00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49e00cda10 0x7f49e015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49df400640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49e015bc40 0x7f49e015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d7e00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00adaa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c370 tx=0x7f49c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8017070 con 0x7f49e00cf930 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c8005cf0 con 0x7f49e00cf930 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80072f0 con 0x7f49e00cf930 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e015d0c0 con 0x7f49e00cf930 2024-04-25T10:18:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.956+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e015d680 con 0x7f49e00cf930 2024-04-25T10:18:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.960+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e004ef30 con 0x7f49e00cf930 2024-04-25T10:18:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.960+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49c801a070 con 0x7f49e00cf930 2024-04-25T10:18:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.960+0000 7f49d6a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f49bc066f60 0x7f49bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.960+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f49c8084490 con 0x7f49e00cf930 2024-04-25T10:18:44.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.960+0000 7f49dea00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f49bc066f60 0x7f49bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.960+0000 7f49dea00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f49bc066f60 0x7f49bc069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f49e00ae400 tx=0x7f49d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:44.964+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49c80553e0 con 0x7f49e00cf930 2024-04-25T10:18:45.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.160+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49e00af1c0 con 0x7f49e00cf930 2024-04-25T10:18:45.162 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:18:45.160+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:18:45.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.160+0000 7f49d6a00640 1 -- 172.21.15.45:0/202622059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f49c8054ce0 con 0x7f49e00cf930 2024-04-25T10:18:45.165 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f49bc066f60 msgr2=0x7f49bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f49bc066f60 0x7f49bc069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f49e00ae400 tx=0x7f49d0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 msgr2=0x7f49e00adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00adaa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c370 tx=0x7f49c800c830 comp rx=0 tx=0).stop 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 shutdown_connections 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f49bc066f60 0x7f49bc069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49e015bc40 0x7f49e015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cf930 0x7f49e00adaa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 --2- 172.21.15.45:0/202622059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49e00cda10 0x7f49e015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 >> 172.21.15.45:0/202622059 conn(0x7f49e0066b70 msgr2=0x7f49e0156340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 shutdown_connections 2024-04-25T10:18:45.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.164+0000 7f49e5a00640 1 -- 172.21.15.45:0/202622059 wait complete. 2024-04-25T10:18:45.181 INFO:tasks.ceph.mds.d.smithi117.stderr: -7001> 2024-04-25T10:18:06.126+0000 7fab2bc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:45.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:18:45.233 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:18:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 Processor -- start 2024-04-25T10:18:45.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 -- start start 2024-04-25T10:18:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c15e000 0x7f6b2c06b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b2c0d0dd0 0x7f6b2c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c06aea0 con 0x7f6b2c0b25d0 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c06b460 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b33400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c06b180 con 0x7f6b2c15e000 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b2c0d0dd0 0x7f6b2c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b2c0d0dd0 0x7f6b2c06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42542/0 (socket says 172.21.15.45:42542) 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 -- 172.21.15.45:0/2013167344 learned_addr learned my addr 172.21.15.45:0/2013167344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b30a00640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c15e000 0x7f6b2c06b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 -- 172.21.15.45:0/2013167344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c15e000 msgr2=0x7f6b2c06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c15e000 0x7f6b2c06b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 -- 172.21.15.45:0/2013167344 >> v1:172.21.15.117:6789/0 conn(0x7f6b2c0b25d0 legacy=0x7f6b2c161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b31400640 1 -- 172.21.15.45:0/2013167344 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0b1de0 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.328+0000 7f6b30a00640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c15e000 0x7f6b2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:45.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b31400640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c06bbe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009e50 tx=0x7f6b2401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4096e413e27d73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b2be00640 1 -- 172.21.15.45:0/2013167344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240a5760 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b2be00640 1 -- 172.21.15.45:0/2013167344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b240a5900 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b2be00640 1 -- 172.21.15.45:0/2013167344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b240a5cd0 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 -- 172.21.15.45:0/2013167344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 msgr2=0x7f6b2c06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c06bbe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009e50 tx=0x7f6b2401c970 comp rx=0 tx=0).stop 2024-04-25T10:18:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 -- 172.21.15.45:0/2013167344 shutdown_connections 2024-04-25T10:18:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c06bbe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 --2- 172.21.15.45:0/2013167344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c15e000 0x7f6b2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 -- 172.21.15.45:0/2013167344 >> 172.21.15.45:0/2013167344 conn(0x7f6b2c0b3430 msgr2=0x7f6b2c0b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 -- 172.21.15.45:0/2013167344 shutdown_connections 2024-04-25T10:18:45.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.332+0000 7f6b33400640 1 -- 172.21.15.45:0/2013167344 wait complete. 2024-04-25T10:18:45.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 Processor -- start 2024-04-25T10:18:45.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- start start 2024-04-25T10:18:45.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0b25d0 0x7f6b2c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c0b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c15e000 0x7f6b2c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c16b0d0 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c16af50 con 0x7f6b2c15e000 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c16b250 con 0x7f6b2c0b25d0 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c0b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c0b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42554/0 (socket says 172.21.15.45:42554) 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 -- 172.21.15.45:0/1803959228 learned_addr learned my addr 172.21.15.45:0/1803959228 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b30a00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0b25d0 0x7f6b2c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b31400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c15e000 0x7f6b2c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 -- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0b25d0 msgr2=0x7f6b2c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0b25d0 0x7f6b2c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 -- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c15e000 msgr2=0x7f6b2c0b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c15e000 0x7f6b2c0b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 -- 172.21.15.45:0/1803959228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c0b75a0 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b30a00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0b25d0 0x7f6b2c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b23e00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c0b7060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c340 tx=0x7f6b1400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140052d0 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b14005470 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c0b6070 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14007260 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0b6630 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.336+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c04ef30 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.340+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b14007400 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.340+0000 7f6b29600640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b08066f60 0x7f6b08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.340+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f6b140844c0 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.340+0000 7f6b30a00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b08066f60 0x7f6b08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.340+0000 7f6b30a00640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b08066f60 0x7f6b08069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0b45f0 tx=0x7f6b1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.344+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b14054d90 con 0x7f6b2c0d0dd0 2024-04-25T10:18:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:45.544+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6b2c0c5340 con 0x7f6b2c0d0dd0 2024-04-25T10:18:46.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.164+0000 7f6b29600640 1 -- 172.21.15.45:0/1803959228 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b14058e30 con 0x7f6b2c0d0dd0 2024-04-25T10:18:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b08066f60 msgr2=0x7f6b08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b08066f60 0x7f6b08069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0b45f0 tx=0x7f6b1c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 msgr2=0x7f6b2c0b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c0b7060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c340 tx=0x7f6b1400c800 comp rx=0 tx=0).stop 2024-04-25T10:18:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 shutdown_connections 2024-04-25T10:18:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b08066f60 0x7f6b08069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c15e000 0x7f6b2c0b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b2c0d0dd0 0x7f6b2c0b7060 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 --2- 172.21.15.45:0/1803959228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0b25d0 0x7f6b2c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 >> 172.21.15.45:0/1803959228 conn(0x7f6b2c0b3430 msgr2=0x7f6b2c0b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 shutdown_connections 2024-04-25T10:18:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.168+0000 7f6b33400640 1 -- 172.21.15.45:0/1803959228 wait complete. 2024-04-25T10:18:46.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:18:46.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 Processor -- start 2024-04-25T10:18:46.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 -- start start 2024-04-25T10:18:46.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3f4162310 0x7fc3f415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f40b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f415e9b0 con 0x7fc3f41618a0 2024-04-25T10:18:46.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f415f250 con 0x7fc3f4162310 2024-04-25T10:18:46.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40afce0 con 0x7fc3f40b4f50 2024-04-25T10:18:46.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3e9400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc3f41618a0 0x7fc3f40b4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40134/0 (socket says 172.21.15.45:40134) 2024-04-25T10:18:46.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.340+0000 7fc3e9400640 1 -- 172.21.15.45:0/1811614838 learned_addr learned my addr 172.21.15.45:0/1811614838 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:46.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f2800640 1 --2- 172.21.15.45:0/1811614838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f40b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1e00640 1 --2- 172.21.15.45:0/1811614838 >> v2:172.21.15.117:3300/0 conn(0x7fc3f4162310 0x7fc3f415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1400640 1 -- 172.21.15.45:0/1811614838 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225811539 0 0) 0x7fc3f415e9b0 con 0x7fc3f41618a0 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1400640 1 -- 172.21.15.45:0/1811614838 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d8003530 con 0x7fc3f41618a0 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f2800640 1 -- 172.21.15.45:0/1811614838 >> v2:172.21.15.117:3300/0 conn(0x7fc3f4162310 msgr2=0x7fc3f415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f2800640 1 --2- 172.21.15.45:0/1811614838 >> v2:172.21.15.117:3300/0 conn(0x7fc3f4162310 0x7fc3f415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f2800640 1 -- 172.21.15.45:0/1811614838 >> v1:172.21.15.45:6789/0 conn(0x7fc3f41618a0 legacy=0x7fc3f40b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f2800640 1 -- 172.21.15.45:0/1811614838 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f415ec90 con 0x7fc3f40b4f50 2024-04-25T10:18:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1e00640 1 --2- 172.21.15.45:0/1811614838 >> v2:172.21.15.117:3300/0 conn(0x7fc3f4162310 0x7fc3f415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:46.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f2800640 1 --2- 172.21.15.45:0/1811614838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f40b0170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009d90 tx=0x7fc3ec01c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b1285c48dcef831 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:46.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1400640 1 -- 172.21.15.45:0/1811614838 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0b2070 con 0x7fc3f40b4f50 2024-04-25T10:18:46.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1400640 1 -- 172.21.15.45:0/1811614838 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0a58f0 con 0x7fc3f40b4f50 2024-04-25T10:18:46.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.344+0000 7fc3f1400640 1 -- 172.21.15.45:0/1811614838 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0a5cc0 con 0x7fc3f40b4f50 2024-04-25T10:18:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- 172.21.15.45:0/1811614838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 msgr2=0x7fc3f40b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/1811614838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f40b0170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009d90 tx=0x7fc3ec01c850 comp rx=0 tx=0).stop 2024-04-25T10:18:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- 172.21.15.45:0/1811614838 shutdown_connections 2024-04-25T10:18:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/1811614838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f40b0170 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/1811614838 >> v2:172.21.15.117:3300/0 conn(0x7fc3f4162310 0x7fc3f415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- 172.21.15.45:0/1811614838 >> 172.21.15.45:0/1811614838 conn(0x7fc3f40b5610 msgr2=0x7fc3f40b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- 172.21.15.45:0/1811614838 shutdown_connections 2024-04-25T10:18:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- 172.21.15.45:0/1811614838 wait complete. 2024-04-25T10:18:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 Processor -- start 2024-04-25T10:18:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- start start 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f4150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3f41618a0 0x7fc3f40ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 0x7fc3f40ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b1040 con 0x7fc3f4162310 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b0ec0 con 0x7fc3f41618a0 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f8a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40b11c0 con 0x7fc3f40b4f50 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 0x7fc3f40ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 0x7fc3f40ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42564/0 (socket says 172.21.15.45:42564) 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 -- 172.21.15.45:0/643281003 learned_addr learned my addr 172.21.15.45:0/643281003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:46.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f1e00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3e9400640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3f41618a0 0x7fc3f40ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 -- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 msgr2=0x7fc3f4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 -- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3f41618a0 msgr2=0x7fc3f40ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3f41618a0 0x7fc3f40ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.348+0000 7fc3f2800640 1 -- 172.21.15.45:0/643281003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4157770 con 0x7fc3f4162310 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3f2800640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 0x7fc3f40ad0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec01af00 tx=0x7fc3ec0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0b2050 con 0x7fc3f4162310 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0be070 con 0x7fc3f4162310 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0ad850 con 0x7fc3f4162310 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f4157a50 con 0x7fc3f4162310 2024-04-25T10:18:46.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f415beb0 con 0x7fc3f4162310 2024-04-25T10:18:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f404ef30 con 0x7fc3f4162310 2024-04-25T10:18:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3ec0ad9f0 con 0x7fc3f4162310 2024-04-25T10:18:46.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3eaa00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc3d0066f90 0x7fc3d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc3ec120000 con 0x7fc3f4162310 2024-04-25T10:18:46.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.352+0000 7fc3f1e00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc3d0066f90 0x7fc3d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.356+0000 7fc3f1e00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc3d0066f90 0x7fc3d0069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009580 tx=0x7fc3e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:46.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.356+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3ec016560 con 0x7fc3f4162310 2024-04-25T10:18:46.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:46.524+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3f406bea0 con 0x7fc3f4162310 2024-04-25T10:18:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.168+0000 7fc3eaa00640 1 -- 172.21.15.45:0/643281003 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc3ec0f0fb0 con 0x7fc3f4162310 2024-04-25T10:18:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:18:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc3d0066f90 msgr2=0x7fc3d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc3d0066f90 0x7fc3d0069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009580 tx=0x7fc3e4009190 comp rx=0 tx=0).stop 2024-04-25T10:18:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 msgr2=0x7fc3f40ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 0x7fc3f40ad0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec01af00 tx=0x7fc3ec0195f0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 shutdown_connections 2024-04-25T10:18:47.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc3d0066f90 0x7fc3d0069450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3f4162310 0x7fc3f40ad0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3f41618a0 0x7fc3f40ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 --2- 172.21.15.45:0/643281003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3f40b4f50 0x7fc3f4150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.172+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 >> 172.21.15.45:0/643281003 conn(0x7fc3f40b5610 msgr2=0x7fc3f4151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.176+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 shutdown_connections 2024-04-25T10:18:47.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.176+0000 7fc3f8a00640 1 -- 172.21.15.45:0/643281003 wait complete. 2024-04-25T10:18:47.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:18:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.348+0000 7f717f000640 1 Processor -- start 2024-04-25T10:18:47.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.348+0000 7f717f000640 1 -- start start 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717f000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71780669a0 0x7f7178066df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717f000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71781478a0 con 0x7f71781474c0 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717f000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7178067330 con 0x7f71780669a0 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7178161960 con 0x7f717815e1f0 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71780669a0 0x7f7178066df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71780669a0 0x7f7178066df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42576/0 (socket says 172.21.15.45:42576) 2024-04-25T10:18:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 -- 172.21.15.45:0/233292101 learned_addr learned my addr 172.21.15.45:0/233292101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 -- 172.21.15.45:0/233292101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 msgr2=0x7f717815e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 --2- 172.21.15.45:0/233292101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 -- 172.21.15.45:0/233292101 >> v1:172.21.15.117:6789/0 conn(0x7f71781474c0 legacy=0x7f7178162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 -- 172.21.15.45:0/233292101 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717815ecb0 con 0x7f71780669a0 2024-04-25T10:18:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f717d000640 1 --2- 172.21.15.45:0/233292101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71780669a0 0x7f7178066df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7170009e50 tx=0x7f717001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb19811d62a8eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f7177400640 1 -- 172.21.15.45:0/233292101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700a5600 con 0x7f71780669a0 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f7177400640 1 -- 172.21.15.45:0/233292101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71700a57a0 con 0x7f71780669a0 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.352+0000 7f7177400640 1 -- 172.21.15.45:0/233292101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700a5b70 con 0x7f71780669a0 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- 172.21.15.45:0/233292101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71780669a0 msgr2=0x7f7178066df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 --2- 172.21.15.45:0/233292101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71780669a0 0x7f7178066df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7170009e50 tx=0x7f717001c780 comp rx=0 tx=0).stop 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- 172.21.15.45:0/233292101 shutdown_connections 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 --2- 172.21.15.45:0/233292101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71780669a0 0x7f7178066df0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 --2- 172.21.15.45:0/233292101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- 172.21.15.45:0/233292101 >> 172.21.15.45:0/233292101 conn(0x7f71780b2560 msgr2=0x7f71780b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- 172.21.15.45:0/233292101 shutdown_connections 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- 172.21.15.45:0/233292101 wait complete. 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 Processor -- start 2024-04-25T10:18:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- start start 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780669a0 0x7f71781298a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 0x7f717815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717806de80 con 0x7f71781474c0 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717806dd00 con 0x7f71780669a0 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717806e000 con 0x7f717815e1f0 2024-04-25T10:18:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f716fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 0x7f717815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f716fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 0x7f717815afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42586/0 (socket says 172.21.15.45:42586) 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f716fe00640 1 -- 172.21.15.45:0/2928383060 learned_addr learned my addr 172.21.15.45:0/2928383060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f7177e00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780669a0 0x7f71781298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.356+0000 7f717d000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f716fe00640 1 -- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 msgr2=0x7f717815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f716fe00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f716fe00640 1 -- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780669a0 msgr2=0x7f71781298a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f716fe00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780669a0 0x7f71781298a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f716fe00640 1 -- 172.21.15.45:0/2928383060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717815a210 con 0x7f71781474c0 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7177e00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780669a0 0x7f71781298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f717d000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f716fe00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 0x7f717815afd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f716000b410 tx=0x7f716000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160015020 con 0x7f71781474c0 2024-04-25T10:18:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160011830 con 0x7f71781474c0 2024-04-25T10:18:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717815a4f0 con 0x7f71781474c0 2024-04-25T10:18:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160011c20 con 0x7f71781474c0 2024-04-25T10:18:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717815d9a0 con 0x7f71781474c0 2024-04-25T10:18:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717804ef30 con 0x7f71781474c0 2024-04-25T10:18:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7160018020 con 0x7f71781474c0 2024-04-25T10:18:47.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7174c00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7154066f60 0x7f7154069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f7160084e80 con 0x7f71781474c0 2024-04-25T10:18:47.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.360+0000 7f7177e00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7154066f60 0x7f7154069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.364+0000 7f7177e00640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7154066f60 0x7f7154069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71680016e0 tx=0x7f7168009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:47.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.364+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f716001f020 con 0x7f71781474c0 2024-04-25T10:18:47.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:47.528+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71780ae240 con 0x7f71781474c0 2024-04-25T10:18:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.176+0000 7f7174c00640 1 -- 172.21.15.45:0/2928383060 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f71600172c0 con 0x7f71781474c0 2024-04-25T10:18:48.179 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:18:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.176+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7154066f60 msgr2=0x7f7154069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.176+0000 7f717f000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7154066f60 0x7f7154069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71680016e0 tx=0x7f7168009040 comp rx=0 tx=0).stop 2024-04-25T10:18:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.176+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 msgr2=0x7f717815afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.176+0000 7f717f000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 0x7f717815afd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f716000b410 tx=0x7f716000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 shutdown_connections 2024-04-25T10:18:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7154066f60 0x7f7154069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717815e1f0 0x7f717815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71781474c0 0x7f717815afd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 --2- 172.21.15.45:0/2928383060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71780669a0 0x7f71781298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 >> 172.21.15.45:0/2928383060 conn(0x7f71780b2560 msgr2=0x7f71780d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 shutdown_connections 2024-04-25T10:18:48.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.180+0000 7f717f000640 1 -- 172.21.15.45:0/2928383060 wait complete. 2024-04-25T10:18:48.220 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:18:48.221 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:18:48.221 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:18:48.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:48.221 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:18:48.221 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:18:48.221 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:18:48.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:48.222 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:18:48.222 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:18:48.222 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:18:48.222 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:48.222 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:18:48.222 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:18:48.222 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:18:48.222 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:48.233 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:18:48.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:18:48.236 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:18:48.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:18:48.238 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:18:48.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:18:48.239 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:18:48.239 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:18:48.279 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:18:48.279 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:48.281 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:18:48.282 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:48.285 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:18:48.285 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:48.287 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:18:48.287 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:48.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:18:48.370 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:18:48.370 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:18:48.372 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:18:48.372 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:18:48.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.432+0000 7f24f8600640 1 Processor -- start 2024-04-25T10:18:48.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.432+0000 7f24f8600640 1 -- start start 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f8600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24f00d0040 0x7f24f015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f8600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f8600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f0071050 con 0x7f24f01550b0 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f8600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f015d040 con 0x7f24f00d0040 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f8600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f00712f0 con 0x7f24f015b740 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f5c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f24f01550b0 0x7f24f00cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40168/0 (socket says 172.21.15.45:40168) 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f5c00640 1 -- 172.21.15.45:0/4194180390 learned_addr learned my addr 172.21.15.45:0/4194180390 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f6600640 1 --2- 172.21.15.45:0/4194180390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f5200640 1 --2- 172.21.15.45:0/4194180390 >> v2:172.21.15.117:3300/0 conn(0x7f24f00d0040 0x7f24f015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24efe00640 1 -- 172.21.15.45:0/4194180390 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78533534 0 0) 0x7f24f0071050 con 0x7f24f01550b0 2024-04-25T10:18:48.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24efe00640 1 -- 172.21.15.45:0/4194180390 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d8003530 con 0x7f24f01550b0 2024-04-25T10:18:48.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f6600640 1 -- 172.21.15.45:0/4194180390 >> v2:172.21.15.117:3300/0 conn(0x7f24f00d0040 msgr2=0x7f24f015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:48.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f6600640 1 --2- 172.21.15.45:0/4194180390 >> v2:172.21.15.117:3300/0 conn(0x7f24f00d0040 0x7f24f015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f6600640 1 -- 172.21.15.45:0/4194180390 >> v1:172.21.15.45:6789/0 conn(0x7f24f01550b0 legacy=0x7f24f00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.436+0000 7f24f6600640 1 -- 172.21.15.45:0/4194180390 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f00715d0 con 0x7f24f015b740 2024-04-25T10:18:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f6600640 1 --2- 172.21.15.45:0/4194180390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f00b3260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009d90 tx=0x7f24e801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190664930150a6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24efe00640 1 -- 172.21.15.45:0/4194180390 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e80b2070 con 0x7f24f015b740 2024-04-25T10:18:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24efe00640 1 -- 172.21.15.45:0/4194180390 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e80a5a00 con 0x7f24f015b740 2024-04-25T10:18:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24efe00640 1 -- 172.21.15.45:0/4194180390 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e80a5db0 con 0x7f24f015b740 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 -- 172.21.15.45:0/4194180390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 msgr2=0x7f24f00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 --2- 172.21.15.45:0/4194180390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f00b3260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009d90 tx=0x7f24e801c940 comp rx=0 tx=0).stop 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 -- 172.21.15.45:0/4194180390 shutdown_connections 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 --2- 172.21.15.45:0/4194180390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f00b3260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 --2- 172.21.15.45:0/4194180390 >> v2:172.21.15.117:3300/0 conn(0x7f24f00d0040 0x7f24f015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 -- 172.21.15.45:0/4194180390 >> 172.21.15.45:0/4194180390 conn(0x7f24f0066b70 msgr2=0x7f24f0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 -- 172.21.15.45:0/4194180390 shutdown_connections 2024-04-25T10:18:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 -- 172.21.15.45:0/4194180390 wait complete. 2024-04-25T10:18:48.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 Processor -- start 2024-04-25T10:18:48.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.440+0000 7f24f8600640 1 -- start start 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24f00d0040 0x7f24f01511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 0x7f24f013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015bd50 con 0x7f24f01550b0 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015bbd0 con 0x7f24f00d0040 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f015c030 con 0x7f24f015b740 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 0x7f24f013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 0x7f24f013b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42624/0 (socket says 172.21.15.45:42624) 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 -- 172.21.15.45:0/4048430894 learned_addr learned my addr 172.21.15.45:0/4048430894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:48.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5c00640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24f00d0040 0x7f24f01511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f6600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 -- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 msgr2=0x7f24f0153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f0153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 -- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24f00d0040 msgr2=0x7f24f01511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24f00d0040 0x7f24f01511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 -- 172.21.15.45:0/4048430894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f0150140 con 0x7f24f01550b0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f6600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f5200640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 0x7f24f013b040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00e930 tx=0x7f24dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc017070 con 0x7f24f01550b0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc00ca00 con 0x7f24f01550b0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00cdb0 con 0x7f24f01550b0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f0150420 con 0x7f24f01550b0 2024-04-25T10:18:48.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f01509e0 con 0x7f24f01550b0 2024-04-25T10:18:48.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f004ef30 con 0x7f24f01550b0 2024-04-25T10:18:48.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.444+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24dc01a070 con 0x7f24f01550b0 2024-04-25T10:18:48.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.448+0000 7f24ed600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.448+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f24dc084fe0 con 0x7f24f01550b0 2024-04-25T10:18:48.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.448+0000 7f24f5c00640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.448+0000 7f24f5c00640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24d0066f90 0x7f24d0069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f24e4007fd0 tx=0x7f24e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:48.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.452+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24dc055d70 con 0x7f24f01550b0 2024-04-25T10:18:48.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:48.624+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24f00cc170 con 0x7f24f01550b0 2024-04-25T10:18:49.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24ed600640 1 -- 172.21.15.45:0/4048430894 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24dc05c0c0 con 0x7f24f01550b0 2024-04-25T10:18:49.190 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:18:49.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24d0066f90 msgr2=0x7f24d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24d0066f90 0x7f24d0069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f24e4007fd0 tx=0x7f24e4009040 comp rx=0 tx=0).stop 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 msgr2=0x7f24f013b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 0x7f24f013b040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00e930 tx=0x7f24dc00edf0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 shutdown_connections 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24d0066f90 0x7f24d0069450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24f015b740 0x7f24f0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f01550b0 0x7f24f013b040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 --2- 172.21.15.45:0/4048430894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24f00d0040 0x7f24f01511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.188+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 >> 172.21.15.45:0/4048430894 conn(0x7f24f0066b70 msgr2=0x7f24f006ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.192+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 shutdown_connections 2024-04-25T10:18:49.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.192+0000 7f24f8600640 1 -- 172.21.15.45:0/4048430894 wait complete. 2024-04-25T10:18:49.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:18:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.356+0000 7fa12ee00640 1 Processor -- start 2024-04-25T10:18:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.356+0000 7fa12ee00640 1 -- start start 2024-04-25T10:18:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1281550b0 0x7fa1280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1280cfe70 0x7fa12815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ee00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa128071050 con 0x7fa12815b740 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12815d040 con 0x7fa1281550b0 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280712f0 con 0x7fa1280cfe70 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa127e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1281550b0 0x7fa1280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ce00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa12815b740 0x7fa12806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40170/0 (socket says 172.21.15.45:40170) 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa127e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1281550b0 0x7fa1280cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42628/0 (socket says 172.21.15.45:42628) 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa12ce00640 1 -- 172.21.15.45:0/1629411772 learned_addr learned my addr 172.21.15.45:0/1629411772 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa11f400640 1 --2- 172.21.15.45:0/1629411772 >> v2:172.21.15.117:3300/0 conn(0x7fa1280cfe70 0x7fa12815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa127e00640 1 -- 172.21.15.45:0/1629411772 >> v2:172.21.15.117:3300/0 conn(0x7fa1280cfe70 msgr2=0x7fa12815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa127e00640 1 --2- 172.21.15.45:0/1629411772 >> v2:172.21.15.117:3300/0 conn(0x7fa1280cfe70 0x7fa12815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa127e00640 1 -- 172.21.15.45:0/1629411772 >> v1:172.21.15.45:6789/0 conn(0x7fa12815b740 legacy=0x7fa12806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.360+0000 7fa127e00640 1 -- 172.21.15.45:0/1629411772 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280715d0 con 0x7fa1281550b0 2024-04-25T10:18:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127e00640 1 --2- 172.21.15.45:0/1629411772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1281550b0 0x7fa1280cf930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa118009d90 tx=0x7fa11801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dc8d118ff486cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127400640 1 -- 172.21.15.45:0/1629411772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b2070 con 0x7fa1281550b0 2024-04-25T10:18:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127400640 1 -- 172.21.15.45:0/1629411772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1180a5800 con 0x7fa1281550b0 2024-04-25T10:18:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127400640 1 -- 172.21.15.45:0/1629411772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180a5bd0 con 0x7fa1281550b0 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- 172.21.15.45:0/1629411772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1281550b0 msgr2=0x7fa1280cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 --2- 172.21.15.45:0/1629411772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1281550b0 0x7fa1280cf930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa118009d90 tx=0x7fa11801c930 comp rx=0 tx=0).stop 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- 172.21.15.45:0/1629411772 shutdown_connections 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 --2- 172.21.15.45:0/1629411772 >> v2:172.21.15.117:3300/0 conn(0x7fa1280cfe70 0x7fa12815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 --2- 172.21.15.45:0/1629411772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1281550b0 0x7fa1280cf930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- 172.21.15.45:0/1629411772 >> 172.21.15.45:0/1629411772 conn(0x7fa128066b70 msgr2=0x7fa128066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- 172.21.15.45:0/1629411772 shutdown_connections 2024-04-25T10:18:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- 172.21.15.45:0/1629411772 wait complete. 2024-04-25T10:18:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 Processor -- start 2024-04-25T10:18:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- start start 2024-04-25T10:18:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 0x7fa1280adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1281550b0 0x7fa12813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa12815b740 0x7fa128140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815bcd0 con 0x7fa1280cfe70 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815bb50 con 0x7fa1281550b0 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa12ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815c030 con 0x7fa12815b740 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 0x7fa1280adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 0x7fa1280adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42644/0 (socket says 172.21.15.45:42644) 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.364+0000 7fa127e00640 1 -- 172.21.15.45:0/3658504806 learned_addr learned my addr 172.21.15.45:0/3658504806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa12ce00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa12815b740 0x7fa128140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa11f400640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1281550b0 0x7fa12813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa127e00640 1 -- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa12815b740 msgr2=0x7fa128140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa127e00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa12815b740 0x7fa128140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa127e00640 1 -- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1281550b0 msgr2=0x7fa12813fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa127e00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1281550b0 0x7fa12813fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa127e00640 1 -- 172.21.15.45:0/3658504806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280b0a20 con 0x7fa1280cfe70 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa12ce00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa12815b740 0x7fa128140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa11f400640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1281550b0 0x7fa12813fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa127e00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 0x7fa1280adaa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa118019540 tx=0x7fa1180196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b2040 con 0x7fa1280cfe70 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1180bf070 con 0x7fa1280cfe70 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1280b0d00 con 0x7fa1280cfe70 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b8890 con 0x7fa1280cfe70 2024-04-25T10:18:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1280b0680 con 0x7fa1280cfe70 2024-04-25T10:18:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12804ef30 con 0x7fa1280cfe70 2024-04-25T10:18:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1180a5680 con 0x7fa1280cfe70 2024-04-25T10:18:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa124c00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa104066f60 0x7fa104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa11811fd70 con 0x7fa1280cfe70 2024-04-25T10:18:49.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.368+0000 7fa11f400640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa104066f60 0x7fa104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.372+0000 7fa11f400640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa104066f60 0x7fa104069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1281593c0 tx=0x7fa110008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.372+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa118016520 con 0x7fa1280cfe70 2024-04-25T10:18:49.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.592+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa12815ad90 con 0x7fa1280cfe70 2024-04-25T10:18:49.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa124c00640 1 -- 172.21.15.45:0/3658504806 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa1180f1150 con 0x7fa1280cfe70 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:18:49.598 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.599 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:18:49.600 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:18:49.601 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.602 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.603 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.605 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:18:49.606 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:49.607 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:18:49.608 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:18:49.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.612 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC/LSpmsrzgMBAA+VjLg7N2tBDVJ9XbSjFdrQ==", 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC/LSpmrsjgMBAAO9gJqkk2qLyTNqt640hQyA==", 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:18:49.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC/LSpmH9LgMBAAEw9rBzyrJHRwSrE08ZAa7Q==", 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC/LSpmndvgMBAAgipx+muEitqxdwLrS6KfoA==", 2024-04-25T10:18:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC/LSpmjeTgMBAAuVvVZxqoNlYpkZcABVDB7g==", 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.615 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC/LSpmre3gMBAArFRmZeYAxoFb7ZP6ZDYDgw==", 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:18:49.617 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:18:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa104066f60 msgr2=0x7fa104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa104066f60 0x7fa104069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1281593c0 tx=0x7fa110008040 comp rx=0 tx=0).stop 2024-04-25T10:18:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 msgr2=0x7fa1280adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 0x7fa1280adaa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa118019540 tx=0x7fa1180196d0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 shutdown_connections 2024-04-25T10:18:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa104066f60 0x7fa104069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa12815b740 0x7fa128140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1281550b0 0x7fa12813fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 --2- 172.21.15.45:0/3658504806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1280cfe70 0x7fa1280adaa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 >> 172.21.15.45:0/3658504806 conn(0x7fa128066b70 msgr2=0x7fa12814e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 shutdown_connections 2024-04-25T10:18:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.596+0000 7fa12ee00640 1 -- 172.21.15.45:0/3658504806 wait complete. 2024-04-25T10:18:49.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:18:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.760+0000 7f25f9e00640 1 Processor -- start 2024-04-25T10:18:49.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.760+0000 7f25f9e00640 1 -- start start 2024-04-25T10:18:49.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25f41550b0 0x7f25f40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f9e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f25f415b910 0x7f25f40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f9e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f4071050 con 0x7f25f40cfe70 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f9e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f415bd80 con 0x7f25f41550b0 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f9e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f415d040 con 0x7f25f415b910 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25f41550b0 0x7f25f40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25f41550b0 0x7f25f40cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42654/0 (socket says 172.21.15.45:42654) 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 -- 172.21.15.45:0/1138593630 learned_addr learned my addr 172.21.15.45:0/1138593630 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2000640 1 --1- 172.21.15.45:0/1138593630 >> v1:172.21.15.45:6789/0 conn(0x7f25f40cfe70 0x7f25f415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40176/0 (socket says 172.21.15.45:40176) 2024-04-25T10:18:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f3400640 1 --2- 172.21.15.45:0/1138593630 >> v2:172.21.15.117:3300/0 conn(0x7f25f415b910 0x7f25f40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 -- 172.21.15.45:0/1138593630 >> v2:172.21.15.117:3300/0 conn(0x7f25f415b910 msgr2=0x7f25f40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 --2- 172.21.15.45:0/1138593630 >> v2:172.21.15.117:3300/0 conn(0x7f25f415b910 0x7f25f40b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 -- 172.21.15.45:0/1138593630 >> v1:172.21.15.45:6789/0 conn(0x7f25f40cfe70 legacy=0x7f25f415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 -- 172.21.15.45:0/1138593630 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40712f0 con 0x7f25f41550b0 2024-04-25T10:18:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f2a00640 1 --2- 172.21.15.45:0/1138593630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f41550b0 0x7f25f40cf930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009e50 tx=0x7f25e401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d4d6d78d7823f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.764+0000 7f25f1600640 1 -- 172.21.15.45:0/1138593630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40a57a0 con 0x7f25f41550b0 2024-04-25T10:18:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f1600640 1 -- 172.21.15.45:0/1138593630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e40a5940 con 0x7f25f41550b0 2024-04-25T10:18:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f1600640 1 -- 172.21.15.45:0/1138593630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40a5d10 con 0x7f25f41550b0 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- 172.21.15.45:0/1138593630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f41550b0 msgr2=0x7f25f40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1138593630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f41550b0 0x7f25f40cf930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009e50 tx=0x7f25e401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- 172.21.15.45:0/1138593630 shutdown_connections 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1138593630 >> v2:172.21.15.117:3300/0 conn(0x7f25f415b910 0x7f25f40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1138593630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f41550b0 0x7f25f40cf930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- 172.21.15.45:0/1138593630 >> 172.21.15.45:0/1138593630 conn(0x7f25f4066b70 msgr2=0x7f25f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- 172.21.15.45:0/1138593630 shutdown_connections 2024-04-25T10:18:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- 172.21.15.45:0/1138593630 wait complete. 2024-04-25T10:18:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 Processor -- start 2024-04-25T10:18:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- start start 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 0x7f25f4146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f41550b0 0x7f25f414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f415b910 0x7f25f415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f415c030 con 0x7f25f40cfe70 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f40720b0 con 0x7f25f41550b0 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f9e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f415c1b0 con 0x7f25f415b910 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 0x7f25f4146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 0x7f25f4146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42660/0 (socket says 172.21.15.45:42660) 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 -- 172.21.15.45:0/1910738897 learned_addr learned my addr 172.21.15.45:0/1910738897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2000640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f41550b0 0x7f25f414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f3400640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f415b910 0x7f25f415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 -- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f415b910 msgr2=0x7f25f415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f415b910 0x7f25f415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 -- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f41550b0 msgr2=0x7f25f414f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f41550b0 0x7f25f414f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2a00640 1 -- 172.21.15.45:0/1910738897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f414afd0 con 0x7f25f40cfe70 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f3400640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f415b910 0x7f25f415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.768+0000 7f25f2000640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f41550b0 0x7f25f414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25f2a00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 0x7f25f4146bb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f25e4019600 tx=0x7f25e4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40b8470 con 0x7f25f40cfe70 2024-04-25T10:18:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e40b8610 con 0x7f25f40cfe70 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40b8a00 con 0x7f25f40cfe70 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f414b830 con 0x7f25f40cfe70 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f414b550 con 0x7f25f40cfe70 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f404ef30 con 0x7f25f40cfe70 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25e40c0030 con 0x7f25f40cfe70 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25eb400640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d0066f90 0x7f25d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25e411f440 con 0x7f25f40cfe70 2024-04-25T10:18:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.772+0000 7f25f2000640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d0066f90 0x7f25d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.776+0000 7f25f2000640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d0066f90 0x7f25d0069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f25f414df00 tx=0x7f25dc007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:49.776+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25e40165e0 con 0x7f25f40cfe70 2024-04-25T10:18:50.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.012+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f25f40cc100 con 0x7f25f40cfe70 2024-04-25T10:18:50.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25eb400640 1 -- 172.21.15.45:0/1910738897 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f25e40f0790 con 0x7f25f40cfe70 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d0066f90 msgr2=0x7f25d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d0066f90 0x7f25d0069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f25f414df00 tx=0x7f25dc007d70 comp rx=0 tx=0).stop 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 msgr2=0x7f25f4146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 0x7f25f4146bb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f25e4019600 tx=0x7f25e4019790 comp rx=0 tx=0).stop 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 shutdown_connections 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d0066f90 0x7f25d0069450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f415b910 0x7f25f415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f41550b0 0x7f25f414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 --2- 172.21.15.45:0/1910738897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f40cfe70 0x7f25f4146bb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 >> 172.21.15.45:0/1910738897 conn(0x7f25f4066b70 msgr2=0x7f25f415abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 shutdown_connections 2024-04-25T10:18:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.016+0000 7f25f9e00640 1 -- 172.21.15.45:0/1910738897 wait complete. 2024-04-25T10:18:50.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:18:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 Processor -- start 2024-04-25T10:18:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 -- start start 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66f40d0040 0x7f66f415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f66f415b740 0x7f66f40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f4071050 con 0x7f66f41550b0 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f415d040 con 0x7f66f40d0040 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66fb800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40712f0 con 0x7f66f415b740 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66f3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66f40d0040 0x7f66f415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66f3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66f40d0040 0x7f66f415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42672/0 (socket says 172.21.15.45:42672) 2024-04-25T10:18:50.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66f3e00640 1 -- 172.21.15.45:0/912370743 learned_addr learned my addr 172.21.15.45:0/912370743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.180+0000 7f66f9800640 1 --2- 172.21.15.45:0/912370743 >> v2:172.21.15.117:3300/0 conn(0x7f66f415b740 0x7f66f40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.184+0000 7f66f3e00640 1 -- 172.21.15.45:0/912370743 >> v2:172.21.15.117:3300/0 conn(0x7f66f415b740 msgr2=0x7f66f40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.184+0000 7f66f3e00640 1 --2- 172.21.15.45:0/912370743 >> v2:172.21.15.117:3300/0 conn(0x7f66f415b740 0x7f66f40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.184+0000 7f66f3e00640 1 -- 172.21.15.45:0/912370743 >> v1:172.21.15.117:6789/0 conn(0x7f66f41550b0 legacy=0x7f66f40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.184+0000 7f66f3e00640 1 -- 172.21.15.45:0/912370743 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40715d0 con 0x7f66f40d0040 2024-04-25T10:18:50.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.184+0000 7f66f3e00640 1 --2- 172.21.15.45:0/912370743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f40d0040 0x7f66f415b200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f66f40d04d0 tx=0x7f66e001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c25c28df57b52c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66f3400640 1 -- 172.21.15.45:0/912370743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00aa650 con 0x7f66f40d0040 2024-04-25T10:18:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66f3400640 1 -- 172.21.15.45:0/912370743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e00af070 con 0x7f66f40d0040 2024-04-25T10:18:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66f3400640 1 -- 172.21.15.45:0/912370743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00aaa10 con 0x7f66f40d0040 2024-04-25T10:18:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 -- 172.21.15.45:0/912370743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f40d0040 msgr2=0x7f66f415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 --2- 172.21.15.45:0/912370743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f40d0040 0x7f66f415b200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f66f40d04d0 tx=0x7f66e001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 -- 172.21.15.45:0/912370743 shutdown_connections 2024-04-25T10:18:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 --2- 172.21.15.45:0/912370743 >> v2:172.21.15.117:3300/0 conn(0x7f66f415b740 0x7f66f40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 --2- 172.21.15.45:0/912370743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f40d0040 0x7f66f415b200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 -- 172.21.15.45:0/912370743 >> 172.21.15.45:0/912370743 conn(0x7f66f4066b70 msgr2=0x7f66f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 -- 172.21.15.45:0/912370743 shutdown_connections 2024-04-25T10:18:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 -- 172.21.15.45:0/912370743 wait complete. 2024-04-25T10:18:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.188+0000 7f66fb800640 1 Processor -- start 2024-04-25T10:18:50.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 -- start start 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f40d0040 0x7f66f41f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 0x7f66f4155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f415b740 0x7f66f415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415bcd0 con 0x7f66f41550b0 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415bb50 con 0x7f66f415b740 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415c030 con 0x7f66f40d0040 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 0x7f66f4155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 0x7f66f4155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42686/0 (socket says 172.21.15.45:42686) 2024-04-25T10:18:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 -- 172.21.15.45:0/1438349412 learned_addr learned my addr 172.21.15.45:0/1438349412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f9800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f415b740 0x7f66f415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f8e00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f40d0040 0x7f66f41f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 -- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f40d0040 msgr2=0x7f66f41f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f40d0040 0x7f66f41f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 -- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f415b740 msgr2=0x7f66f415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f415b740 0x7f66f415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 -- 172.21.15.45:0/1438349412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f41fa510 con 0x7f66f41550b0 2024-04-25T10:18:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f3e00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 0x7f66f4155f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f66e0019100 tx=0x7f66e001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00bd5f0 con 0x7f66f41550b0 2024-04-25T10:18:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e00a6040 con 0x7f66f41550b0 2024-04-25T10:18:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00bd9d0 con 0x7f66f41550b0 2024-04-25T10:18:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f41fa7f0 con 0x7f66f41550b0 2024-04-25T10:18:50.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.192+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4151940 con 0x7f66f41550b0 2024-04-25T10:18:50.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.196+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f404ef30 con 0x7f66f41550b0 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.196+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66e00b33f0 con 0x7f66f41550b0 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.196+0000 7f66f0c00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66d4066f60 0x7f66d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.196+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66e011ef90 con 0x7f66f41550b0 2024-04-25T10:18:50.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.196+0000 7f66f8e00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66d4066f60 0x7f66d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.200+0000 7f66f8e00640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66d4066f60 0x7f66d4069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66f4158580 tx=0x7f66e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.204+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66e0016110 con 0x7f66f41550b0 2024-04-25T10:18:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.424+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f66f4151b40 con 0x7f66f41550b0 2024-04-25T10:18:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66f0c00640 1 -- 172.21.15.45:0/1438349412 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f66e00efc00 con 0x7f66f41550b0 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66d4066f60 msgr2=0x7f66d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66d4066f60 0x7f66d4069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66f4158580 tx=0x7f66e8009040 comp rx=0 tx=0).stop 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 msgr2=0x7f66f4155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 0x7f66f4155f20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f66e0019100 tx=0x7f66e001ad30 comp rx=0 tx=0).stop 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 shutdown_connections 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66d4066f60 0x7f66d4069420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66f415b740 0x7f66f415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66f41550b0 0x7f66f4155f20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 --2- 172.21.15.45:0/1438349412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66f40d0040 0x7f66f41f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 >> 172.21.15.45:0/1438349412 conn(0x7f66f4066b70 msgr2=0x7f66f40cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 shutdown_connections 2024-04-25T10:18:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.428+0000 7f66fb800640 1 -- 172.21.15.45:0/1438349412 wait complete. 2024-04-25T10:18:50.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 Processor -- start 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 -- start start 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2a6015dc10 0x7f2a600d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a6015dff0 0x7f2a6006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600cbff0 con 0x7f2a600d09e0 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600b45a0 con 0x7f2a6015dff0 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5f200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600cbda0 con 0x7f2a6015dc10 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a6015dff0 0x7f2a6006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a6015dff0 0x7f2a6006b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42702/0 (socket says 172.21.15.45:42702) 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 -- 172.21.15.45:0/2889564773 learned_addr learned my addr 172.21.15.45:0/2889564773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5de00640 1 --2- 172.21.15.45:0/2889564773 >> v2:172.21.15.117:3300/0 conn(0x7f2a6015dc10 0x7f2a600d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 -- 172.21.15.45:0/2889564773 >> v2:172.21.15.117:3300/0 conn(0x7f2a6015dc10 msgr2=0x7f2a600d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 --2- 172.21.15.45:0/2889564773 >> v2:172.21.15.117:3300/0 conn(0x7f2a6015dc10 0x7f2a600d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 -- 172.21.15.45:0/2889564773 >> v1:172.21.15.117:6789/0 conn(0x7f2a600d09e0 legacy=0x7f2a600ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a57e00640 1 -- 172.21.15.45:0/2889564773 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600cc2d0 con 0x7f2a6015dff0 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.592+0000 7f2a5de00640 1 --2- 172.21.15.45:0/2889564773 >> v2:172.21.15.117:3300/0 conn(0x7f2a6015dc10 0x7f2a600d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:50.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a57e00640 1 --2- 172.21.15.45:0/2889564773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a6006b6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b4270 tx=0x7f2a4801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b713d5ac1566e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5d400640 1 -- 172.21.15.45:0/2889564773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480ab600 con 0x7f2a6015dff0 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5d400640 1 -- 172.21.15.45:0/2889564773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a480b0070 con 0x7f2a6015dff0 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5d400640 1 -- 172.21.15.45:0/2889564773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480ab9c0 con 0x7f2a6015dff0 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 -- 172.21.15.45:0/2889564773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 msgr2=0x7f2a6006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2889564773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a6006b6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b4270 tx=0x7f2a4801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 -- 172.21.15.45:0/2889564773 shutdown_connections 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2889564773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a6006b6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2889564773 >> v2:172.21.15.117:3300/0 conn(0x7f2a6015dc10 0x7f2a600d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 -- 172.21.15.45:0/2889564773 >> 172.21.15.45:0/2889564773 conn(0x7f2a600d1750 msgr2=0x7f2a600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 -- 172.21.15.45:0/2889564773 shutdown_connections 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.596+0000 7f2a5f200640 1 -- 172.21.15.45:0/2889564773 wait complete. 2024-04-25T10:18:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 Processor -- start 2024-04-25T10:18:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 -- start start 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600d09e0 0x7f2a60153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a6015dc10 0x7f2a600b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a600c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6006b1e0 con 0x7f2a6015dff0 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6006b060 con 0x7f2a6015dc10 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6016a960 con 0x7f2a600d09e0 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a600c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a600c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42704/0 (socket says 172.21.15.45:42704) 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 -- 172.21.15.45:0/2087361018 learned_addr learned my addr 172.21.15.45:0/2087361018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5de00640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600d09e0 0x7f2a60153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a57e00640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a6015dc10 0x7f2a600b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 -- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600d09e0 msgr2=0x7f2a60153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600d09e0 0x7f2a60153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 -- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a6015dc10 msgr2=0x7f2a600b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a6015dc10 0x7f2a600b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 -- 172.21.15.45:0/2087361018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600b6740 con 0x7f2a6015dff0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5e800640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a600c76b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c730 tx=0x7f2a5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58003b80 con 0x7f2a6015dff0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a58003d20 con 0x7f2a6015dff0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800aa30 con 0x7f2a6015dff0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a600b6a20 con 0x7f2a6015dff0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.600+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600b6fe0 con 0x7f2a6015dff0 2024-04-25T10:18:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.604+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60075120 con 0x7f2a6015dff0 2024-04-25T10:18:50.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.604+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a5800abd0 con 0x7f2a6015dff0 2024-04-25T10:18:50.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.604+0000 7f2a56000640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066fe0 0x7f2a3c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.604+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a580849e0 con 0x7f2a6015dff0 2024-04-25T10:18:50.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.604+0000 7f2a5de00640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066fe0 0x7f2a3c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.604+0000 7f2a5de00640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066fe0 0x7f2a3c0694a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006e90 tx=0x7f2a50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.608+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a580556d0 con 0x7f2a6015dff0 2024-04-25T10:18:50.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.832+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f2a6006d380 con 0x7f2a6015dff0 2024-04-25T10:18:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a56000640 1 -- 172.21.15.45:0/2087361018 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2a58059770 con 0x7f2a6015dff0 2024-04-25T10:18:50.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066fe0 msgr2=0x7f2a3c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066fe0 0x7f2a3c0694a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2a50006e90 tx=0x7f2a50008040 comp rx=0 tx=0).stop 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 msgr2=0x7f2a600c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a600c76b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c730 tx=0x7f2a5800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 shutdown_connections 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066fe0 0x7f2a3c0694a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015dff0 0x7f2a600c76b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a6015dc10 0x7f2a600b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 --2- 172.21.15.45:0/2087361018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600d09e0 0x7f2a60153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 >> 172.21.15.45:0/2087361018 conn(0x7f2a600d1750 msgr2=0x7f2a600d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 shutdown_connections 2024-04-25T10:18:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.836+0000 7f2a5f200640 1 -- 172.21.15.45:0/2087361018 wait complete. 2024-04-25T10:18:50.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:18:51.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.996+0000 7f2995200640 1 Processor -- start 2024-04-25T10:18:51.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:50.996+0000 7f2995200640 1 -- start start 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2995200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f299015bc40 0x7f299006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2995200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29900cda10 0x7f29900b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2995200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29900b2770 con 0x7f29900cf930 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2995200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299006c670 con 0x7f29900cda10 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2995200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299006c950 con 0x7f299015bc40 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2985400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29900cda10 0x7f29900b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f298e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f29900cf930 0x7f299015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40186/0 (socket says 172.21.15.45:40186) 2024-04-25T10:18:51.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f298e800640 1 -- 172.21.15.45:0/2735653773 learned_addr learned my addr 172.21.15.45:0/2735653773 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f298de00640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f299015bc40 0x7f299006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2985400640 1 -- 172.21.15.45:0/2735653773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f299015bc40 msgr2=0x7f299006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2985400640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f299015bc40 0x7f299006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2985400640 1 -- 172.21.15.45:0/2735653773 >> v1:172.21.15.45:6789/0 conn(0x7f29900cf930 legacy=0x7f299015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f2985400640 1 -- 172.21.15.45:0/2735653773 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299006cc30 con 0x7f29900cda10 2024-04-25T10:18:51.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.000+0000 7f298de00640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f299015bc40 0x7f299006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:51.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2985400640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cda10 0x7f29900b4610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f29780097c0 tx=0x7f297801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690744df1459b970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f298d400640 1 -- 172.21.15.45:0/2735653773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29780b1070 con 0x7f29900cda10 2024-04-25T10:18:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f298d400640 1 -- 172.21.15.45:0/2735653773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29780ac410 con 0x7f29900cda10 2024-04-25T10:18:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f298d400640 1 -- 172.21.15.45:0/2735653773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29780ac7e0 con 0x7f29900cda10 2024-04-25T10:18:51.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- 172.21.15.45:0/2735653773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cda10 msgr2=0x7f29900b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cda10 0x7f29900b4610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f29780097c0 tx=0x7f297801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- 172.21.15.45:0/2735653773 shutdown_connections 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cda10 0x7f29900b4610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 --2- 172.21.15.45:0/2735653773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f299015bc40 0x7f299006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- 172.21.15.45:0/2735653773 >> 172.21.15.45:0/2735653773 conn(0x7f2990066b70 msgr2=0x7f2990066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- 172.21.15.45:0/2735653773 shutdown_connections 2024-04-25T10:18:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- 172.21.15.45:0/2735653773 wait complete. 2024-04-25T10:18:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 Processor -- start 2024-04-25T10:18:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- start start 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29900cda10 0x7f2990144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 0x7f2990071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f299015bc40 0x7f29900ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2990168b50 con 0x7f29900cf930 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29901689d0 con 0x7f299015bc40 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2995200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2990168cd0 con 0x7f29900cda10 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2985400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 0x7f2990071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2985400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 0x7f2990071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42720/0 (socket says 172.21.15.45:42720) 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.004+0000 7f2985400640 1 -- 172.21.15.45:0/2636344259 learned_addr learned my addr 172.21.15.45:0/2636344259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2985400640 1 -- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29900cda10 msgr2=0x7f2990144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2985400640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29900cda10 0x7f2990144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2985400640 1 -- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f299015bc40 msgr2=0x7f29900ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2985400640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f299015bc40 0x7f29900ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2985400640 1 -- 172.21.15.45:0/2636344259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2990072310 con 0x7f29900cf930 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2985400640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 0x7f2990071dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f29780af040 tx=0x7f2978019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29780b1040 con 0x7f29900cf930 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29780bb070 con 0x7f29900cf930 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2990071010 con 0x7f29900cf930 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978006e20 con 0x7f29900cf930 2024-04-25T10:18:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29900715d0 con 0x7f29900cf930 2024-04-25T10:18:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299004ef30 con 0x7f29900cf930 2024-04-25T10:18:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29780b5670 con 0x7f29900cf930 2024-04-25T10:18:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2986a00640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f296c066f60 0x7f296c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.008+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f297811e790 con 0x7f29900cf930 2024-04-25T10:18:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.012+0000 7f298de00640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f296c066f60 0x7f296c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.012+0000 7f298de00640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f296c066f60 0x7f296c069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f298000a8d0 tx=0x7f2980008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.012+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2978016090 con 0x7f29900cf930 2024-04-25T10:18:51.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.236+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2990068400 con 0x7f29900cf930 2024-04-25T10:18:51.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.236+0000 7f2986a00640 1 -- 172.21.15.45:0/2636344259 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f29780ace20 con 0x7f29900cf930 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f296c066f60 msgr2=0x7f296c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f296c066f60 0x7f296c069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f298000a8d0 tx=0x7f2980008040 comp rx=0 tx=0).stop 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 msgr2=0x7f2990071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 0x7f2990071dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f29780af040 tx=0x7f2978019240 comp rx=0 tx=0).stop 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 shutdown_connections 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f296c066f60 0x7f296c069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f299015bc40 0x7f29900ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29900cf930 0x7f2990071dd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 --2- 172.21.15.45:0/2636344259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29900cda10 0x7f2990144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 >> 172.21.15.45:0/2636344259 conn(0x7f2990066b70 msgr2=0x7f2990066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 shutdown_connections 2024-04-25T10:18:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.240+0000 7f2995200640 1 -- 172.21.15.45:0/2636344259 wait complete. 2024-04-25T10:18:51.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:18:51.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.400+0000 7f01c2200640 1 Processor -- start 2024-04-25T10:18:51.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.400+0000 7f01c2200640 1 -- start start 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01c2200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01bc0d0040 0x7f01bc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01c2200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 0x7f01bc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01c2200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc071050 con 0x7f01bc1550b0 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01c2200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc15d040 con 0x7f01bc0d0040 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01c2200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0712f0 con 0x7f01bc15b740 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01bc1550b0 0x7f01bc0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40188/0 (socket says 172.21.15.45:40188) 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 -- 172.21.15.45:0/4185406883 learned_addr learned my addr 172.21.15.45:0/4185406883 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01b3e00640 1 --2- 172.21.15.45:0/4185406883 >> v2:172.21.15.117:3300/0 conn(0x7f01bc0d0040 0x7f01bc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bbe00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 0x7f01bc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097621066 0 0) 0x7f01bc071050 con 0x7f01bc1550b0 2024-04-25T10:18:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a0003530 con 0x7f01bc1550b0 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3928709827 0 0) 0x7f01a0003530 con 0x7f01bc1550b0 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 msgr2=0x7f01bc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 0x7f01bc0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 >> v2:172.21.15.117:3300/0 conn(0x7f01bc0d0040 msgr2=0x7f01bc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 --2- 172.21.15.45:0/4185406883 >> v2:172.21.15.117:3300/0 conn(0x7f01bc0d0040 0x7f01bc15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc0715d0 con 0x7f01bc1550b0 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f01ac001680 con 0x7f01bc1550b0 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 >> v1:172.21.15.45:6789/0 conn(0x7f01bc1550b0 legacy=0x7f01bc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 0x7f01a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a0004d50 0x7f01a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc15d040 con 0x7f01a00039c0 2024-04-25T10:18:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0712f0 con 0x7f01a0004d50 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a000b9d0 con 0x7f01bc15b740 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bbe00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a0004d50 0x7f01a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01b3e00640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 0x7f01a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 -- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 msgr2=0x7f01a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc15b740 0x7f01a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 -- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a0004d50 msgr2=0x7f01a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a0004d50 0x7f01a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.404+0000 7f01bb400640 1 -- 172.21.15.45:0/4185406883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0003530 con 0x7f01a00039c0 2024-04-25T10:18:51.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01bb400640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01a0003de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c190 tx=0x7f01ac01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f5b21fa92ce8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac007c90 con 0x7f01a00039c0 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac0a9020 con 0x7f01a00039c0 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01baa00640 1 -- 172.21.15.45:0/4185406883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac005ae0 con 0x7f01a00039c0 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 -- 172.21.15.45:0/4185406883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 msgr2=0x7f01a0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 --2- 172.21.15.45:0/4185406883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01a0003de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c190 tx=0x7f01ac01cbd0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01bbe00640 1 -- 172.21.15.45:0/4185406883 reap_dead start 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 -- 172.21.15.45:0/4185406883 shutdown_connections 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 -- 172.21.15.45:0/4185406883 >> 172.21.15.45:0/4185406883 conn(0x7f01bc066b70 msgr2=0x7f01bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 -- 172.21.15.45:0/4185406883 shutdown_connections 2024-04-25T10:18:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 -- 172.21.15.45:0/4185406883 wait complete. 2024-04-25T10:18:51.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 Processor -- start 2024-04-25T10:18:51.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.408+0000 7f01c2200640 1 -- start start 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01bc143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc155490 0x7f01bc143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc144210 0x7f01bc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc15bcd0 con 0x7f01a00039c0 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc15bb50 con 0x7f01bc144210 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc15c840 con 0x7f01bc155490 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01bc143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01bc143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42738/0 (socket says 172.21.15.45:42738) 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 -- 172.21.15.45:0/26725242 learned_addr learned my addr 172.21.15.45:0/26725242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01b3e00640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc155490 0x7f01bc143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bbe00640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc144210 0x7f01bc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 -- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc155490 msgr2=0x7f01bc143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc155490 0x7f01bc143cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 -- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc144210 msgr2=0x7f01bc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc144210 0x7f01bc0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 -- 172.21.15.45:0/26725242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc143790 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01bb400640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01bc143250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f01ac007040 tx=0x7f01a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac0a5ab0 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac0a5c50 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac0047f0 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc0b0860 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc0b0e20 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.412+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc04ef30 con 0x7f01a00039c0 2024-04-25T10:18:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.416+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01ac004990 con 0x7f01a00039c0 2024-04-25T10:18:51.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.416+0000 7f01b3400640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0198066f00 0x7f01980693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.416+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f01ac11ea60 con 0x7f01a00039c0 2024-04-25T10:18:51.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.416+0000 7f01b3e00640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0198066f00 0x7f01980693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.416+0000 7f01b3e00640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0198066f00 0x7f01980693c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f01bc158500 tx=0x7f01a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.420+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01ac016910 con 0x7f01a00039c0 2024-04-25T10:18:51.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.636+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f01bc15cc10 con 0x7f01a00039c0 2024-04-25T10:18:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.640+0000 7f01b3400640 1 -- 172.21.15.45:0/26725242 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f01ac0ef6d0 con 0x7f01a00039c0 2024-04-25T10:18:51.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.640+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0198066f00 msgr2=0x7f01980693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.640+0000 7f01c2200640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0198066f00 0x7f01980693c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f01bc158500 tx=0x7f01a4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.640+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 msgr2=0x7f01bc143250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.640+0000 7f01c2200640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01bc143250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f01ac007040 tx=0x7f01a0005840 comp rx=0 tx=0).stop 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 shutdown_connections 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0198066f00 0x7f01980693c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01bc144210 0x7f01bc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01bc155490 0x7f01bc143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 --2- 172.21.15.45:0/26725242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01a00039c0 0x7f01bc143250 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 >> 172.21.15.45:0/26725242 conn(0x7f01bc066b70 msgr2=0x7f01bc159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 shutdown_connections 2024-04-25T10:18:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.644+0000 7f01c2200640 1 -- 172.21.15.45:0/26725242 wait complete. 2024-04-25T10:18:51.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:18:51.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 Processor -- start 2024-04-25T10:18:51.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 -- start start 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5d415b750 0x7fa5d40cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5d40b4690 0x7fa5d40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40b2460 con 0x7fa5d40cd590 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40b2d70 con 0x7fa5d40b4690 2024-04-25T10:18:51.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5da200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4070fb0 con 0x7fa5d415b750 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5d40b4690 0x7fa5d40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5d40b4690 0x7fa5d40cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42744/0 (socket says 172.21.15.45:42744) 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 -- 172.21.15.45:0/1661162617 learned_addr learned my addr 172.21.15.45:0/1661162617 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d8e00640 1 --2- 172.21.15.45:0/1661162617 >> v2:172.21.15.117:3300/0 conn(0x7fa5d415b750 0x7fa5d40cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 -- 172.21.15.45:0/1661162617 >> v2:172.21.15.117:3300/0 conn(0x7fa5d415b750 msgr2=0x7fa5d40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 --2- 172.21.15.45:0/1661162617 >> v2:172.21.15.117:3300/0 conn(0x7fa5d415b750 0x7fa5d40cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 -- 172.21.15.45:0/1661162617 >> v1:172.21.15.117:6789/0 conn(0x7fa5d40cd590 legacy=0x7fa5d40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.804+0000 7fa5d9800640 1 -- 172.21.15.45:0/1661162617 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40b2720 con 0x7fa5d40b4690 2024-04-25T10:18:51.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5d9800640 1 --2- 172.21.15.45:0/1661162617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d40cd9e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40b2ef0 tx=0x7fa5cc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575b61508337b2f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5d3400640 1 -- 172.21.15.45:0/1661162617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a5840 con 0x7fa5d40b4690 2024-04-25T10:18:51.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5d3400640 1 -- 172.21.15.45:0/1661162617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0a59e0 con 0x7fa5d40b4690 2024-04-25T10:18:51.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5d3400640 1 -- 172.21.15.45:0/1661162617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a5db0 con 0x7fa5d40b4690 2024-04-25T10:18:51.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 -- 172.21.15.45:0/1661162617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 msgr2=0x7fa5d40cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 --2- 172.21.15.45:0/1661162617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d40cd9e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40b2ef0 tx=0x7fa5cc01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 -- 172.21.15.45:0/1661162617 shutdown_connections 2024-04-25T10:18:51.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 --2- 172.21.15.45:0/1661162617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d40cd9e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 --2- 172.21.15.45:0/1661162617 >> v2:172.21.15.117:3300/0 conn(0x7fa5d415b750 0x7fa5d40cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 -- 172.21.15.45:0/1661162617 >> 172.21.15.45:0/1661162617 conn(0x7fa5d40743f0 msgr2=0x7fa5d40669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 -- 172.21.15.45:0/1661162617 shutdown_connections 2024-04-25T10:18:51.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.808+0000 7fa5da200640 1 -- 172.21.15.45:0/1661162617 wait complete. 2024-04-25T10:18:51.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 Processor -- start 2024-04-25T10:18:51.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- start start 2024-04-25T10:18:51.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40cd590 0x7fa5d41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d415b750 0x7fa5d4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4070ce0 con 0x7fa5d40b4690 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4070b60 con 0x7fa5d415b750 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40b39f0 con 0x7fa5d40cd590 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d4150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42756/0 (socket says 172.21.15.45:42756) 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 -- 172.21.15.45:0/3465524062 learned_addr learned my addr 172.21.15.45:0/3465524062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d9800640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d415b750 0x7fa5d4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d3e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40cd590 0x7fa5d41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 -- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40cd590 msgr2=0x7fa5d41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40cd590 0x7fa5d41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 -- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d415b750 msgr2=0x7fa5d4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d415b750 0x7fa5d4159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 -- 172.21.15.45:0/3465524062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40aef90 con 0x7fa5d40b4690 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d3e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40cd590 0x7fa5d41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:51.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d8e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d4150210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800d620 tx=0x7fa5c800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80125f0 con 0x7fa5d40b4690 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8019070 con 0x7fa5d40b4690 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80129b0 con 0x7fa5d40b4690 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d415a490 con 0x7fa5d40b4690 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d40af250 con 0x7fa5d40b4690 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.812+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4075120 con 0x7fa5d40b4690 2024-04-25T10:18:51.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.816+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5c8012b50 con 0x7fa5d40b4690 2024-04-25T10:18:51.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.816+0000 7fa5d0c00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b4066f60 0x7fa5b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.816+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5c8085090 con 0x7fa5d40b4690 2024-04-25T10:18:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.816+0000 7fa5d3e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b4066f60 0x7fa5b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.816+0000 7fa5d3e00640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b4066f60 0x7fa5b4069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40cf410 tx=0x7fa5c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:51.820+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5c8056350 con 0x7fa5d40b4690 2024-04-25T10:18:52.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.040+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa5d4067f20 con 0x7fa5d40b4690 2024-04-25T10:18:52.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.040+0000 7fa5d0c00640 1 -- 172.21.15.45:0/3465524062 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa5c8055c50 con 0x7fa5d40b4690 2024-04-25T10:18:52.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b4066f60 msgr2=0x7fa5b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b4066f60 0x7fa5b4069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40cf410 tx=0x7fa5c0009040 comp rx=0 tx=0).stop 2024-04-25T10:18:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 msgr2=0x7fa5d4150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d4150210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800d620 tx=0x7fa5c800dae0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 shutdown_connections 2024-04-25T10:18:52.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b4066f60 0x7fa5b4069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d415b750 0x7fa5d4159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40cd590 0x7fa5d41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 --2- 172.21.15.45:0/3465524062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40b4690 0x7fa5d4150210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 >> 172.21.15.45:0/3465524062 conn(0x7fa5d40743f0 msgr2=0x7fa5d4152f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 shutdown_connections 2024-04-25T10:18:52.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.044+0000 7fa5da200640 1 -- 172.21.15.45:0/3465524062 wait complete. 2024-04-25T10:18:52.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7ea0600640 1 Processor -- start 2024-04-25T10:18:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7ea0600640 1 -- start start 2024-04-25T10:18:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7ea0600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e980d0040 0x7f7e9815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7ea0600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98071050 con 0x7f7e9815b740 2024-04-25T10:18:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7ea0600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9815d040 con 0x7f7e981550b0 2024-04-25T10:18:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7ea0600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980712f0 con 0x7f7e980d0040 2024-04-25T10:18:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e9e600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7e9815b740 0x7f7e980b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40194/0 (socket says 172.21.15.45:40194) 2024-04-25T10:18:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e9e600640 1 -- 172.21.15.45:0/2597710878 learned_addr learned my addr 172.21.15.45:0/2597710878 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e9d200640 1 --2- 172.21.15.45:0/2597710878 >> v2:172.21.15.117:3300/0 conn(0x7f7e980d0040 0x7f7e9815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250763803 0 0) 0x7f7e98071050 con 0x7f7e9815b740 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e80003530 con 0x7f7e9815b740 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121412325 0 0) 0x7f7e9815d040 con 0x7f7e981550b0 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e98071050 con 0x7f7e981550b0 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e9d200640 1 -- 172.21.15.45:0/2597710878 >> v1:172.21.15.117:6789/0 conn(0x7f7e981550b0 legacy=0x7f7e980cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e9d200640 1 -- 172.21.15.45:0/2597710878 >> v1:172.21.15.45:6789/0 conn(0x7f7e9815b740 legacy=0x7f7e980b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.204+0000 7f7e9d200640 1 -- 172.21.15.45:0/2597710878 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980715d0 con 0x7f7e980d0040 2024-04-25T10:18:52.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7e9d200640 1 --2- 172.21.15.45:0/2597710878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e980d0040 0x7f7e9815b200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e840097c0 tx=0x7f7e8401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9820f288d807b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e840b0070 con 0x7f7e980d0040 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e840ab600 con 0x7f7e980d0040 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7e97e00640 1 -- 172.21.15.45:0/2597710878 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e840ab9d0 con 0x7f7e980d0040 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 -- 172.21.15.45:0/2597710878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e980d0040 msgr2=0x7f7e9815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 --2- 172.21.15.45:0/2597710878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e980d0040 0x7f7e9815b200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e840097c0 tx=0x7f7e8401c470 comp rx=0 tx=0).stop 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 -- 172.21.15.45:0/2597710878 shutdown_connections 2024-04-25T10:18:52.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 --2- 172.21.15.45:0/2597710878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e980d0040 0x7f7e9815b200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 -- 172.21.15.45:0/2597710878 >> 172.21.15.45:0/2597710878 conn(0x7f7e98066b70 msgr2=0x7f7e98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 -- 172.21.15.45:0/2597710878 shutdown_connections 2024-04-25T10:18:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.208+0000 7f7ea0600640 1 -- 172.21.15.45:0/2597710878 wait complete. 2024-04-25T10:18:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 Processor -- start 2024-04-25T10:18:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 -- start start 2024-04-25T10:18:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e980d0040 0x7f7e9814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e981550b0 0x7f7e9814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 0x7f7e9814b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9815bcd0 con 0x7f7e9815b740 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9815bb50 con 0x7f7e981550b0 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e981684c0 con 0x7f7e980d0040 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 0x7f7e9814b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 0x7f7e9814b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42762/0 (socket says 172.21.15.45:42762) 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 -- 172.21.15.45:0/1827610596 learned_addr learned my addr 172.21.15.45:0/1827610596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9dc00640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e980d0040 0x7f7e9814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9d200640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e981550b0 0x7f7e9814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 -- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e980d0040 msgr2=0x7f7e9814ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e980d0040 0x7f7e9814ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 -- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e981550b0 msgr2=0x7f7e9814b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e981550b0 0x7f7e9814b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 -- 172.21.15.45:0/1827610596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980722c0 con 0x7f7e9815b740 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9dc00640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e980d0040 0x7f7e9814ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e9e600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 0x7f7e9814b840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000caf0 tx=0x7f7e9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90017070 con 0x7f7e9815b740 2024-04-25T10:18:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9000aca0 con 0x7f7e9815b740 2024-04-25T10:18:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e98152300 con 0x7f7e9815b740 2024-04-25T10:18:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90012620 con 0x7f7e9815b740 2024-04-25T10:18:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.212+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e98152880 con 0x7f7e9815b740 2024-04-25T10:18:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.216+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9804ef30 con 0x7f7e9815b740 2024-04-25T10:18:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.216+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e9001a070 con 0x7f7e9815b740 2024-04-25T10:18:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.216+0000 7f7e95600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e78066f60 0x7f7e78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.216+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e900849d0 con 0x7f7e9815b740 2024-04-25T10:18:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.216+0000 7f7e9dc00640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e78066f60 0x7f7e78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.216+0000 7f7e9dc00640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e78066f60 0x7f7e78069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0011f0 tx=0x7f7e8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.220+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e900556d0 con 0x7f7e9815b740 2024-04-25T10:18:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e9806cb70 con 0x7f7e9815b740 2024-04-25T10:18:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7e95600640 1 -- 172.21.15.45:0/1827610596 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f7e90059770 con 0x7f7e9815b740 2024-04-25T10:18:52.387 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:52.387 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:49.196203+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e78066f60 msgr2=0x7f7e78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e78066f60 0x7f7e78069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0011f0 tx=0x7f7e8c009040 comp rx=0 tx=0).stop 2024-04-25T10:18:52.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 msgr2=0x7f7e9814b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 0x7f7e9814b840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000caf0 tx=0x7f7e9000a060 comp rx=0 tx=0).stop 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 shutdown_connections 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e78066f60 0x7f7e78069420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e9815b740 0x7f7e9814b840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e981550b0 0x7f7e9814b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 --2- 172.21.15.45:0/1827610596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e980d0040 0x7f7e9814ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 >> 172.21.15.45:0/1827610596 conn(0x7f7e98066b70 msgr2=0x7f7e980b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 shutdown_connections 2024-04-25T10:18:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.384+0000 7f7ea0600640 1 -- 172.21.15.45:0/1827610596 wait complete. 2024-04-25T10:18:52.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:52.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:52.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f979ea00640 1 Processor -- start 2024-04-25T10:18:52.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f979ea00640 1 -- start start 2024-04-25T10:18:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f979ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97980b3480 0x7f979815e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f979ea00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97980ad840 con 0x7f97980cfb50 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f979ea00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979815e710 con 0x7f9798071bc0 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f979ea00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979815bb90 con 0x7f97980b3480 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f9797e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97980b3480 0x7f979815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f978f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f97980cfb50 0x7f9798070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40196/0 (socket says 172.21.15.45:40196) 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f978f400640 1 -- 172.21.15.45:0/3751122948 learned_addr learned my addr 172.21.15.45:0/3751122948 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f9797e00640 1 -- 172.21.15.45:0/3751122948 >> v1:172.21.15.117:6789/0 conn(0x7f9798071bc0 legacy=0x7f97980add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:52.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f9797400640 1 -- 172.21.15.45:0/3751122948 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71578955 0 0) 0x7f97980ad840 con 0x7f97980cfb50 2024-04-25T10:18:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f9797e00640 1 -- 172.21.15.45:0/3751122948 >> v1:172.21.15.45:6789/0 conn(0x7f97980cfb50 legacy=0x7f9798070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.556+0000 7f9797e00640 1 -- 172.21.15.45:0/3751122948 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979815e9f0 con 0x7f97980b3480 2024-04-25T10:18:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f9797e00640 1 --2- 172.21.15.45:0/3751122948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980b3480 0x7f979815e130 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f97880097c0 tx=0x7f978801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda6fab5a4931edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f9797400640 1 -- 172.21.15.45:0/3751122948 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880b0070 con 0x7f97980b3480 2024-04-25T10:18:52.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f9797400640 1 -- 172.21.15.45:0/3751122948 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97880ab600 con 0x7f97980b3480 2024-04-25T10:18:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f9797400640 1 -- 172.21.15.45:0/3751122948 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880ab9d0 con 0x7f97980b3480 2024-04-25T10:18:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 -- 172.21.15.45:0/3751122948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980b3480 msgr2=0x7f979815e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 --2- 172.21.15.45:0/3751122948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980b3480 0x7f979815e130 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f97880097c0 tx=0x7f978801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 -- 172.21.15.45:0/3751122948 shutdown_connections 2024-04-25T10:18:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 --2- 172.21.15.45:0/3751122948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97980b3480 0x7f979815e130 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 -- 172.21.15.45:0/3751122948 >> 172.21.15.45:0/3751122948 conn(0x7f97980669a0 msgr2=0x7f9798066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 -- 172.21.15.45:0/3751122948 shutdown_connections 2024-04-25T10:18:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.560+0000 7f979ea00640 1 -- 172.21.15.45:0/3751122948 wait complete. 2024-04-25T10:18:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 Processor -- start 2024-04-25T10:18:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- start start 2024-04-25T10:18:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 0x7f9798161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980b3480 0x7f97981602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980cfb50 0x7f9798158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980723e0 con 0x7f9798071bc0 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980ae840 con 0x7f97980b3480 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9798160960 con 0x7f97980cfb50 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 0x7f9798161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 0x7f9798161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42778/0 (socket says 172.21.15.45:42778) 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 -- 172.21.15.45:0/146636958 learned_addr learned my addr 172.21.15.45:0/146636958 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f978f400640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980b3480 0x7f97981602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ca00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980cfb50 0x7f9798158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 -- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980cfb50 msgr2=0x7f9798158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980cfb50 0x7f9798158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 -- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980b3480 msgr2=0x7f97981602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980b3480 0x7f97981602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 -- 172.21.15.45:0/146636958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798160c40 con 0x7f9798071bc0 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f978f400640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980b3480 0x7f97981602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ca00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980cfb50 0x7f9798158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9797e00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 0x7f9798161a00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97880190b0 tx=0x7f978801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880b0050 con 0x7f9798071bc0 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97880bc070 con 0x7f9798071bc0 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880b5a50 con 0x7f9798071bc0 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9798160f20 con 0x7f9798071bc0 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97981614e0 con 0x7f9798071bc0 2024-04-25T10:18:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.564+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979804ef30 con 0x7f9798071bc0 2024-04-25T10:18:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.568+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97880ab4a0 con 0x7f9798071bc0 2024-04-25T10:18:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.568+0000 7f9794c00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066f60 0x7f9774069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.568+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f978811f680 con 0x7f9798071bc0 2024-04-25T10:18:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.568+0000 7f978f400640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066f60 0x7f9774069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.568+0000 7f978f400640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066f60 0x7f9774069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97980ceb70 tx=0x7f9780009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.572+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9788016090 con 0x7f9798071bc0 2024-04-25T10:18:52.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.732+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97980705a0 con 0x7f9798071bc0 2024-04-25T10:18:52.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f9794c00640 1 -- 172.21.15.45:0/146636958 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97880f09d0 con 0x7f9798071bc0 2024-04-25T10:18:52.739 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066f60 msgr2=0x7f9774069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066f60 0x7f9774069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f97980ceb70 tx=0x7f9780009040 comp rx=0 tx=0).stop 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 msgr2=0x7f9798161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 0x7f9798161a00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97880190b0 tx=0x7f978801ace0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 shutdown_connections 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066f60 0x7f9774069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980cfb50 0x7f9798158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980b3480 0x7f97981602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 --2- 172.21.15.45:0/146636958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798071bc0 0x7f9798161a00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 >> 172.21.15.45:0/146636958 conn(0x7f97980669a0 msgr2=0x7f9798070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 shutdown_connections 2024-04-25T10:18:52.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.736+0000 7f979ea00640 1 -- 172.21.15.45:0/146636958 wait complete. 2024-04-25T10:18:52.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:18:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.896+0000 7f958dc00640 1 Processor -- start 2024-04-25T10:18:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.896+0000 7f958dc00640 1 -- start start 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f958dc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9588066c00 0x7f958815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f958dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95880b5560 0x7f9588157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f958dc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880b59b0 con 0x7f9588066fe0 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f958dc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880b5c20 con 0x7f9588066c00 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f958dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880af6e0 con 0x7f95880b5560 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9586a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9588066c00 0x7f958815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9587400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95880b5560 0x7f9588157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f957e000640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9588066fe0 0x7f958806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:35552/0 (socket says 172.21.15.45:35552) 2024-04-25T10:18:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f957e000640 1 -- 172.21.15.45:0/934309124 learned_addr learned my addr 172.21.15.45:0/934309124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9586a00640 1 -- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95880b5560 msgr2=0x7f9588157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9586a00640 1 --2- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95880b5560 0x7f9588157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9586a00640 1 -- 172.21.15.45:0/934309124 >> v1:172.21.15.117:6789/0 conn(0x7f9588066fe0 legacy=0x7f958806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9586a00640 1 -- 172.21.15.45:0/934309124 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880af9c0 con 0x7f9588066c00 2024-04-25T10:18:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9586000640 1 -- 172.21.15.45:0/934309124 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312788444 0 0) 0x7f95880b59b0 con 0x7f9588066fe0 2024-04-25T10:18:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.900+0000 7f9587400640 1 --2- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95880b5560 0x7f9588157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f9586a00640 1 --2- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f958815e110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95780097c0 tx=0x7f957801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf83cdc94e6a6a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f9586000640 1 -- 172.21.15.45:0/934309124 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780a7070 con 0x7f9588066c00 2024-04-25T10:18:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f9586000640 1 -- 172.21.15.45:0/934309124 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95780b0070 con 0x7f9588066c00 2024-04-25T10:18:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f9586000640 1 -- 172.21.15.45:0/934309124 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780ab890 con 0x7f9588066c00 2024-04-25T10:18:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 -- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 msgr2=0x7f958815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 --2- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f958815e110 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95780097c0 tx=0x7f957801c470 comp rx=0 tx=0).stop 2024-04-25T10:18:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 -- 172.21.15.45:0/934309124 shutdown_connections 2024-04-25T10:18:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 --2- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95880b5560 0x7f9588157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 --2- 172.21.15.45:0/934309124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f958815e110 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 -- 172.21.15.45:0/934309124 >> 172.21.15.45:0/934309124 conn(0x7f95880b24f0 msgr2=0x7f95880b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 -- 172.21.15.45:0/934309124 shutdown_connections 2024-04-25T10:18:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 -- 172.21.15.45:0/934309124 wait complete. 2024-04-25T10:18:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 Processor -- start 2024-04-25T10:18:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.904+0000 7f958dc00640 1 -- start start 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f95881409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9588066fe0 0x7f9588140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 0x7f9588150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880af1e0 con 0x7f95880b5560 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880af060 con 0x7f9588066c00 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958806ac90 con 0x7f9588066fe0 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 0x7f9588150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 0x7f9588150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42780/0 (socket says 172.21.15.45:42780) 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 -- 172.21.15.45:0/3338889287 learned_addr learned my addr 172.21.15.45:0/3338889287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9586a00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f95881409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f957e000640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9588066fe0 0x7f9588140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 -- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9588066fe0 msgr2=0x7f9588140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9588066fe0 0x7f9588140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 -- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 msgr2=0x7f95881409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f95881409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 -- 172.21.15.45:0/3338889287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95881591c0 con 0x7f95880b5560 2024-04-25T10:18:52.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9586a00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f95881409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:18:52.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f9587400640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 0x7f9588150310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9580009e60 tx=0x7f958000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580018030 con 0x7f95880b5560 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000cdf0 con 0x7f95880b5560 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580003bc0 con 0x7f95880b5560 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9588160450 con 0x7f95880b5560 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9588160a10 con 0x7f95880b5560 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958804ef30 con 0x7f95880b5560 2024-04-25T10:18:52.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.908+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f958001b070 con 0x7f95880b5560 2024-04-25T10:18:52.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.912+0000 7f957f400640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9564066f90 0x7f9564069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.912+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95800855a0 con 0x7f95880b5560 2024-04-25T10:18:52.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.912+0000 7f9586a00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9564066f90 0x7f9564069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.912+0000 7f9586a00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9564066f90 0x7f9564069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95780af000 tx=0x7f95780a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:52.916+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9580010040 con 0x7f95880b5560 2024-04-25T10:18:53.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.080+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f958806b9b0 con 0x7f95880b5560 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.080+0000 7f957f400640 1 -- 172.21.15.45:0/3338889287 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9580056210 con 0x7f95880b5560 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:18:53.085 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:18:53.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9564066f90 msgr2=0x7f9564069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9564066f90 0x7f9564069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95780af000 tx=0x7f95780a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:53.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 msgr2=0x7f9588150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 0x7f9588150310 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9580009e60 tx=0x7f958000c490 comp rx=0 tx=0).stop 2024-04-25T10:18:53.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 shutdown_connections 2024-04-25T10:18:53.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9564066f90 0x7f9564069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880b5560 0x7f9588150310 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9588066fe0 0x7f9588140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 --2- 172.21.15.45:0/3338889287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588066c00 0x7f95881409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 >> 172.21.15.45:0/3338889287 conn(0x7f95880b24f0 msgr2=0x7f958806ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 shutdown_connections 2024-04-25T10:18:53.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.084+0000 7f958dc00640 1 -- 172.21.15.45:0/3338889287 wait complete. 2024-04-25T10:18:53.106 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:18:53.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:18:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.248+0000 7fbb81c00640 1 Processor -- start 2024-04-25T10:18:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.248+0000 7fbb81c00640 1 -- start start 2024-04-25T10:18:53.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb81c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb7c0b3fc0 0x7fbb7c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb81c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c0b2f50 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb81c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c06b040 con 0x7fbb7c06b6a0 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb81c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c15f360 con 0x7fbb7c0b3fc0 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb7b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb7c0b3fc0 0x7fbb7c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb7b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb7c0b3fc0 0x7fbb7c0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42790/0 (socket says 172.21.15.45:42790) 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb7b400640 1 -- 172.21.15.45:0/1619083609 learned_addr learned my addr 172.21.15.45:0/1619083609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb72000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbb7c0b26b0 0x7fbb7c0cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40200/0 (socket says 172.21.15.45:40200) 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb7b400640 1 -- 172.21.15.45:0/1619083609 >> v1:172.21.15.117:6789/0 conn(0x7fbb7c06b6a0 legacy=0x7fbb7c06ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:53.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb7b400640 1 -- 172.21.15.45:0/1619083609 >> v1:172.21.15.45:6789/0 conn(0x7fbb7c0b26b0 legacy=0x7fbb7c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.252+0000 7fbb7b400640 1 -- 172.21.15.45:0/1619083609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c15f640 con 0x7fbb7c0b3fc0 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb7b400640 1 --2- 172.21.15.45:0/1619083609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c0b1890 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbb740098a0 tx=0x7fbb7401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3357cd8ec5c8add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb7a000640 1 -- 172.21.15.45:0/1619083609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740b0070 con 0x7fbb7c0b3fc0 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb7a000640 1 -- 172.21.15.45:0/1619083609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb740ab600 con 0x7fbb7c0b3fc0 2024-04-25T10:18:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb7a000640 1 -- 172.21.15.45:0/1619083609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740ab9d0 con 0x7fbb7c0b3fc0 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- 172.21.15.45:0/1619083609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b3fc0 msgr2=0x7fbb7c0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1619083609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c0b1890 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbb740098a0 tx=0x7fbb7401c470 comp rx=0 tx=0).stop 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- 172.21.15.45:0/1619083609 shutdown_connections 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1619083609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c0b1890 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- 172.21.15.45:0/1619083609 >> 172.21.15.45:0/1619083609 conn(0x7fbb7c0b34a0 msgr2=0x7fbb7c0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- 172.21.15.45:0/1619083609 shutdown_connections 2024-04-25T10:18:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- 172.21.15.45:0/1619083609 wait complete. 2024-04-25T10:18:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 Processor -- start 2024-04-25T10:18:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- start start 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7c06b6a0 0x7fbb7c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 0x7fbb7c0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c16cba0 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c16ca20 con 0x7fbb7c0b3fc0 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.256+0000 7fbb81c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c16cd20 con 0x7fbb7c06b6a0 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 0x7fbb7c0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 0x7fbb7c0b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42800/0 (socket says 172.21.15.45:42800) 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 -- 172.21.15.45:0/1085575041 learned_addr learned my addr 172.21.15.45:0/1085575041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb7aa00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7c06b6a0 0x7fbb7c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb7b400640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 -- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7c06b6a0 msgr2=0x7fbb7c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7c06b6a0 0x7fbb7c0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 -- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb7c0b3fc0 msgr2=0x7fbb7c14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 -- 172.21.15.45:0/1085575041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0b66a0 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb72000640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 0x7fbb7c0b4ad0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c420 tx=0x7fbb6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640074e0 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64007680 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64004a30 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c15e580 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0b6c00 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c0b73e0 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.260+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb6401a020 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.264+0000 7fbb73400640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.264+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb64084300 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.264+0000 7fbb7aa00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.264+0000 7fbb7aa00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c15ecb0 tx=0x7fbb6c00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.264+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb7c0b73e0 con 0x7fbb7c0b26b0 2024-04-25T10:18:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:53.428+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb7c04ef30 con 0x7fbb7c0b26b0 2024-04-25T10:18:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.204+0000 7fbb73400640 1 -- 172.21.15.45:0/1085575041 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb7c0b66a0 con 0x7fbb7c0b26b0 2024-04-25T10:18:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:18:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 msgr2=0x7fbb58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c15ecb0 tx=0x7fbb6c00d040 comp rx=0 tx=0).stop 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 msgr2=0x7fbb7c0b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 0x7fbb7c0b4ad0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c420 tx=0x7fbb6400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 shutdown_connections 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb7c0b3fc0 0x7fbb7c14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb7c0b26b0 0x7fbb7c0b4ad0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 --2- 172.21.15.45:0/1085575041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7c06b6a0 0x7fbb7c0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 >> 172.21.15.45:0/1085575041 conn(0x7fbb7c0b34a0 msgr2=0x7fbb7c15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 shutdown_connections 2024-04-25T10:18:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.208+0000 7fbb81c00640 1 -- 172.21.15.45:0/1085575041 wait complete. 2024-04-25T10:18:54.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:18:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.380+0000 7fb21da00640 1 Processor -- start 2024-04-25T10:18:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.380+0000 7fb21da00640 1 -- start start 2024-04-25T10:18:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.380+0000 7fb21da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb21815bbb0 0x7fb2180cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.380+0000 7fb21da00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21806ca90 con 0x7fb2180b4660 2024-04-25T10:18:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.380+0000 7fb21da00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21806cd70 con 0x7fb2180cf960 2024-04-25T10:18:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.380+0000 7fb21da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21806d050 con 0x7fb21815bbb0 2024-04-25T10:18:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb20fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb2180b4660 0x7fb21806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40214/0 (socket says 172.21.15.45:40214) 2024-04-25T10:18:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb20fe00640 1 -- 172.21.15.45:0/2497337103 learned_addr learned my addr 172.21.15.45:0/2497337103 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216a00640 1 --2- 172.21.15.45:0/2497337103 >> v2:172.21.15.117:3300/0 conn(0x7fb21815bbb0 0x7fb2180cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216000640 1 -- 172.21.15.45:0/2497337103 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745065197 0 0) 0x7fb21806ca90 con 0x7fb2180b4660 2024-04-25T10:18:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216000640 1 -- 172.21.15.45:0/2497337103 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc003530 con 0x7fb2180b4660 2024-04-25T10:18:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216a00640 1 -- 172.21.15.45:0/2497337103 >> v1:172.21.15.117:6789/0 conn(0x7fb2180cf960 legacy=0x7fb2180b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216a00640 1 -- 172.21.15.45:0/2497337103 >> v1:172.21.15.45:6789/0 conn(0x7fb2180b4660 legacy=0x7fb21806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216a00640 1 -- 172.21.15.45:0/2497337103 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180b2880 con 0x7fb21815bbb0 2024-04-25T10:18:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.384+0000 7fb216a00640 1 --2- 172.21.15.45:0/2497337103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb2180cda80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2080097c0 tx=0x7fb20801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41fb494629234a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb216000640 1 -- 172.21.15.45:0/2497337103 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080b0070 con 0x7fb21815bbb0 2024-04-25T10:18:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb216000640 1 -- 172.21.15.45:0/2497337103 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2080ab600 con 0x7fb21815bbb0 2024-04-25T10:18:54.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb216000640 1 -- 172.21.15.45:0/2497337103 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2080ab9b0 con 0x7fb21815bbb0 2024-04-25T10:18:54.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- 172.21.15.45:0/2497337103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 msgr2=0x7fb2180cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 --2- 172.21.15.45:0/2497337103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb2180cda80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2080097c0 tx=0x7fb20801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- 172.21.15.45:0/2497337103 shutdown_connections 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 --2- 172.21.15.45:0/2497337103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb2180cda80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- 172.21.15.45:0/2497337103 >> 172.21.15.45:0/2497337103 conn(0x7fb218066b70 msgr2=0x7fb218066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- 172.21.15.45:0/2497337103 shutdown_connections 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- 172.21.15.45:0/2497337103 wait complete. 2024-04-25T10:18:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 Processor -- start 2024-04-25T10:18:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- start start 2024-04-25T10:18:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180b4660 0x7fb21814a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 0x7fb21815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb218153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180ce140 con 0x7fb2180cf960 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180cdfc0 con 0x7fb21815bbb0 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb21da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21816a180 con 0x7fb2180b4660 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb20fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 0x7fb21815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb20fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 0x7fb21815ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42804/0 (socket says 172.21.15.45:42804) 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb20fe00640 1 -- 172.21.15.45:0/113147971 learned_addr learned my addr 172.21.15.45:0/113147971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb217400640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb218153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.388+0000 7fb216a00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180b4660 0x7fb21814a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20fe00640 1 -- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180b4660 msgr2=0x7fb21814a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20fe00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180b4660 0x7fb21814a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20fe00640 1 -- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 msgr2=0x7fb218153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20fe00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb218153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20fe00640 1 -- 172.21.15.45:0/113147971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21814aac0 con 0x7fb2180cf960 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb217400640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb218153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20fe00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 0x7fb21815ad30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb200001ec0 tx=0x7fb20000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200017070 con 0x7fb2180cf960 2024-04-25T10:18:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb200005d50 con 0x7fb2180cf960 2024-04-25T10:18:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218159e30 con 0x7fb2180cf960 2024-04-25T10:18:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200007270 con 0x7fb2180cf960 2024-04-25T10:18:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218153390 con 0x7fb2180cf960 2024-04-25T10:18:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21804ef30 con 0x7fb2180cf960 2024-04-25T10:18:54.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.392+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb20001a070 con 0x7fb2180cf960 2024-04-25T10:18:54.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.396+0000 7fb20ea00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f4066f60 0x7fb1f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.396+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb2000845f0 con 0x7fb2180cf960 2024-04-25T10:18:54.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.396+0000 7fb216a00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f4066f60 0x7fb1f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.396+0000 7fb216a00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f4066f60 0x7fb1f4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ad000 tx=0x7fb2080a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:54.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.400+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb200055230 con 0x7fb2180cf960 2024-04-25T10:18:54.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:54.564+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb21806b400 con 0x7fb2180cf960 2024-04-25T10:18:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.212+0000 7fb20ea00640 1 -- 172.21.15.45:0/113147971 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb2000592d0 con 0x7fb2180cf960 2024-04-25T10:18:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:18:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f4066f60 msgr2=0x7fb1f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f4066f60 0x7fb1f4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2080ad000 tx=0x7fb2080a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 msgr2=0x7fb21815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 0x7fb21815ad30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb200001ec0 tx=0x7fb20000c960 comp rx=0 tx=0).stop 2024-04-25T10:18:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 shutdown_connections 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1f4066f60 0x7fb1f4069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb21815bbb0 0x7fb218153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2180cf960 0x7fb21815ad30 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb200001ec0 tx=0x7fb20000c960 comp rx=0 tx=0).stop 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 --2- 172.21.15.45:0/113147971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180b4660 0x7fb21814a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 >> 172.21.15.45:0/113147971 conn(0x7fb218066b70 msgr2=0x7fb218066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 shutdown_connections 2024-04-25T10:18:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.216+0000 7fb21da00640 1 -- 172.21.15.45:0/113147971 wait complete. 2024-04-25T10:18:55.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:18:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.396+0000 7f6e61200640 1 Processor -- start 2024-04-25T10:18:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.396+0000 7f6e61200640 1 -- start start 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e61200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e5c0b1f50 0x7f6e5c0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e61200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6e5c0cd780 0x7f6e5c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e61200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5c06be90 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e61200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c0d0b50 con 0x7f6e5c0b1f50 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e61200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c06b680 con 0x7f6e5c0cd780 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e5c0b1f50 0x7f6e5c0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e5c0b1f50 0x7f6e5c0b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60268/0 (socket says 172.21.15.45:60268) 2024-04-25T10:18:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 -- 172.21.15.45:0/2551069535 learned_addr learned my addr 172.21.15.45:0/2551069535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5be00640 1 --2- 172.21.15.45:0/2551069535 >> v2:172.21.15.117:3300/0 conn(0x7f6e5c0cd780 0x7f6e5c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 -- 172.21.15.45:0/2551069535 >> v2:172.21.15.117:3300/0 conn(0x7f6e5c0cd780 msgr2=0x7f6e5c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 --2- 172.21.15.45:0/2551069535 >> v2:172.21.15.117:3300/0 conn(0x7f6e5c0cd780 0x7f6e5c0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 -- 172.21.15.45:0/2551069535 >> v1:172.21.15.117:6789/0 conn(0x7f6e5c0b3dd0 legacy=0x7f6e5c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.400+0000 7f6e5b400640 1 -- 172.21.15.45:0/2551069535 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c06b8d0 con 0x7f6e5c0b1f50 2024-04-25T10:18:55.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e5b400640 1 --2- 172.21.15.45:0/2551069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b1f50 0x7f6e5c0b23c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0722f0 tx=0x7f6e4c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529b6a25683f7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e5aa00640 1 -- 172.21.15.45:0/2551069535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0b1070 con 0x7f6e5c0b1f50 2024-04-25T10:18:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e5aa00640 1 -- 172.21.15.45:0/2551069535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c0ac640 con 0x7f6e5c0b1f50 2024-04-25T10:18:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e5aa00640 1 -- 172.21.15.45:0/2551069535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0aca10 con 0x7f6e5c0b1f50 2024-04-25T10:18:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- 172.21.15.45:0/2551069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b1f50 msgr2=0x7f6e5c0b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 --2- 172.21.15.45:0/2551069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b1f50 0x7f6e5c0b23c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0722f0 tx=0x7f6e4c01c400 comp rx=0 tx=0).stop 2024-04-25T10:18:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- 172.21.15.45:0/2551069535 shutdown_connections 2024-04-25T10:18:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 --2- 172.21.15.45:0/2551069535 >> v2:172.21.15.117:3300/0 conn(0x7f6e5c0cd780 0x7f6e5c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 --2- 172.21.15.45:0/2551069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b1f50 0x7f6e5c0b23c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- 172.21.15.45:0/2551069535 >> 172.21.15.45:0/2551069535 conn(0x7f6e5c0b32b0 msgr2=0x7f6e5c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- 172.21.15.45:0/2551069535 shutdown_connections 2024-04-25T10:18:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- 172.21.15.45:0/2551069535 wait complete. 2024-04-25T10:18:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 Processor -- start 2024-04-25T10:18:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- start start 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e5c0b1f50 0x7f6e5c1fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 0x7f6e5c158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e5c0cd780 0x7f6e5c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0d03c0 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0d0240 con 0x7f6e5c0cd780 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e61200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0d0540 con 0x7f6e5c0b1f50 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e52a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 0x7f6e5c158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e5b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e5c0b1f50 0x7f6e5c1fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.404+0000 7f6e52a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 0x7f6e5c158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60282/0 (socket says 172.21.15.45:60282) 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 -- 172.21.15.45:0/1388071894 learned_addr learned my addr 172.21.15.45:0/1388071894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e5be00640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e5c0cd780 0x7f6e5c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 -- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e5c0b1f50 msgr2=0x7f6e5c1fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e5c0b1f50 0x7f6e5c1fae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 -- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e5c0cd780 msgr2=0x7f6e5c0b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e5c0cd780 0x7f6e5c0b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 -- 172.21.15.45:0/1388071894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c158d50 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e52a00640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 0x7f6e5c158810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c3a0 tx=0x7f6e4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e440052d0 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44005470 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44007260 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c0c5830 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c0c5df0 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c158f90 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.408+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e44007400 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.412+0000 7f6e53e00640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e38066f90 0x7f6e38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.412+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6e440844c0 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.412+0000 7f6e5b400640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e38066f90 0x7f6e38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.412+0000 7f6e5b400640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e38066f90 0x7f6e38069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0ae000 tx=0x7f6e4c0a6070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.416+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e5c06be90 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:55.612+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6e5c075120 con 0x7f6e5c0b3dd0 2024-04-25T10:18:55.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:55.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.212+0000 7f6e53e00640 1 -- 172.21.15.45:0/1388071894 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6e44054dc0 con 0x7f6e5c0b3dd0 2024-04-25T10:18:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T10:18:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e38066f90 msgr2=0x7f6e38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e38066f90 0x7f6e38069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0ae000 tx=0x7f6e4c0a6070 comp rx=0 tx=0).stop 2024-04-25T10:18:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 msgr2=0x7f6e5c158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 0x7f6e5c158810 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c3a0 tx=0x7f6e4400c860 comp rx=0 tx=0).stop 2024-04-25T10:18:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 shutdown_connections 2024-04-25T10:18:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e38066f90 0x7f6e38069450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e5c0cd780 0x7f6e5c0b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e5c0b3dd0 0x7f6e5c158810 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 --2- 172.21.15.45:0/1388071894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e5c0b1f50 0x7f6e5c1fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.220+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 >> 172.21.15.45:0/1388071894 conn(0x7f6e5c0b32b0 msgr2=0x7f6e5c0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.224+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 shutdown_connections 2024-04-25T10:18:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.224+0000 7f6e61200640 1 -- 172.21.15.45:0/1388071894 wait complete. 2024-04-25T10:18:56.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 Processor -- start 2024-04-25T10:18:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 -- start start 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704815e050 0x7f70480b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70480b13e0 0x7f70480cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480b1870 con 0x7f70480d1820 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480b1af0 con 0x7f70480b13e0 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f704dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70480b1dd0 con 0x7f704815e050 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70480b13e0 0x7f70480cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70480b13e0 0x7f70480cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60298/0 (socket says 172.21.15.45:60298) 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 -- 172.21.15.45:0/2468408776 learned_addr learned my addr 172.21.15.45:0/2468408776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7046000640 1 --2- 172.21.15.45:0/2468408776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704815e050 0x7f70480b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 -- 172.21.15.45:0/2468408776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704815e050 msgr2=0x7f70480b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 --2- 172.21.15.45:0/2468408776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704815e050 0x7f70480b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 -- 172.21.15.45:0/2468408776 >> v1:172.21.15.117:6789/0 conn(0x7f70480d1820 legacy=0x7f70480d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.412+0000 7f7047400640 1 -- 172.21.15.45:0/2468408776 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480b20b0 con 0x7f70480b13e0 2024-04-25T10:18:56.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f7047400640 1 --2- 172.21.15.45:0/2468408776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480b13e0 0x7f70480cfc70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7040009e50 tx=0x7f704001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c236f7c9c4b31473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f7045600640 1 -- 172.21.15.45:0/2468408776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400a5860 con 0x7f70480b13e0 2024-04-25T10:18:56.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f7045600640 1 -- 172.21.15.45:0/2468408776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70400a5a00 con 0x7f70480b13e0 2024-04-25T10:18:56.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f7045600640 1 -- 172.21.15.45:0/2468408776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400a5dd0 con 0x7f70480b13e0 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 -- 172.21.15.45:0/2468408776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480b13e0 msgr2=0x7f70480cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 --2- 172.21.15.45:0/2468408776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480b13e0 0x7f70480cfc70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7040009e50 tx=0x7f704001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 -- 172.21.15.45:0/2468408776 shutdown_connections 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 --2- 172.21.15.45:0/2468408776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70480b13e0 0x7f70480cfc70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 --2- 172.21.15.45:0/2468408776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704815e050 0x7f70480b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 -- 172.21.15.45:0/2468408776 >> 172.21.15.45:0/2468408776 conn(0x7f704809a9d0 msgr2=0x7f704809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 -- 172.21.15.45:0/2468408776 shutdown_connections 2024-04-25T10:18:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 -- 172.21.15.45:0/2468408776 wait complete. 2024-04-25T10:18:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 Processor -- start 2024-04-25T10:18:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.416+0000 7f704dc00640 1 -- start start 2024-04-25T10:18:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70480b13e0 0x7f704815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70480d1820 0x7f70480b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 0x7f7048145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70480d0d90 con 0x7f704815e050 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70480d0c10 con 0x7f70480d1820 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70480d0f10 con 0x7f70480b13e0 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 0x7f7048145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 0x7f7048145c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60300/0 (socket says 172.21.15.45:60300) 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 -- 172.21.15.45:0/2072097675 learned_addr learned my addr 172.21.15.45:0/2072097675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7046a00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70480b13e0 0x7f704815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7046000640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70480d1820 0x7f70480b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 -- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70480b13e0 msgr2=0x7f704815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70480b13e0 0x7f704815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 -- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70480d1820 msgr2=0x7f70480b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70480d1820 0x7f70480b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 -- 172.21.15.45:0/2072097675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704815b5b0 con 0x7f704815e050 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7046a00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70480b13e0 0x7f704815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7046000640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70480d1820 0x7f70480b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7047400640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 0x7f7048145c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f704001a5d0 tx=0x7f7040019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400bc030 con 0x7f704815e050 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70400b8470 con 0x7f704815e050 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400b88a0 con 0x7f704815e050 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 -- 172.21.15.45:0/2072097675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704815b890 con 0x7f704815e050 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f704dc00640 1 -- 172.21.15.45:0/2072097675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704815be50 con 0x7f704815e050 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.420+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c0052a0 con 0x7f704815e050 2024-04-25T10:18:56.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.424+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70400bf070 con 0x7f704815e050 2024-04-25T10:18:56.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.424+0000 7f703ea00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7028066f60 0x7f7028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.424+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f704011fb50 con 0x7f704815e050 2024-04-25T10:18:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.424+0000 7f7046a00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7028066f60 0x7f7028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.424+0000 7f7046a00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7028066f60 0x7f7028069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7038001710 tx=0x7f7038009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.428+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7040016620 con 0x7f704815e050 2024-04-25T10:18:56.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.588+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f701c005590 con 0x7f704815e050 2024-04-25T10:18:56.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f703ea00640 1 -- 172.21.15.45:0/2072097675 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f70400c0e50 con 0x7f704815e050 2024-04-25T10:18:56.595 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:56.595 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:55.617637+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:56.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7028066f60 msgr2=0x7f7028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7028066f60 0x7f7028069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7038001710 tx=0x7f7038009040 comp rx=0 tx=0).stop 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 msgr2=0x7f7048145c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 0x7f7048145c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f704001a5d0 tx=0x7f7040019740 comp rx=0 tx=0).stop 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 shutdown_connections 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7028066f60 0x7f7028069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f704815e050 0x7f7048145c00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70480d1820 0x7f70480b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 --2- 172.21.15.45:0/2072097675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70480b13e0 0x7f704815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 >> 172.21.15.45:0/2072097675 conn(0x7f704809a9d0 msgr2=0x7f704806ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 shutdown_connections 2024-04-25T10:18:56.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.592+0000 7f7033e00640 1 -- 172.21.15.45:0/2072097675 wait complete. 2024-04-25T10:18:56.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.752+0000 7f88d4800640 1 Processor -- start 2024-04-25T10:18:56.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.752+0000 7f88d4800640 1 -- start start 2024-04-25T10:18:56.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d4800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88cc0d0040 0x7f88cc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d4800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f88cc15b740 0x7f88cc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d4800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc071050 con 0x7f88cc1550b0 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d4800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc15d040 con 0x7f88cc0d0040 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d4800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc0712f0 con 0x7f88cc15b740 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88cc0d0040 0x7f88cc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88cc0d0040 0x7f88cc15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60302/0 (socket says 172.21.15.45:60302) 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 -- 172.21.15.45:0/1334349490 learned_addr learned my addr 172.21.15.45:0/1334349490 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d2800640 1 --2- 172.21.15.45:0/1334349490 >> v2:172.21.15.117:3300/0 conn(0x7f88cc15b740 0x7f88cc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 -- 172.21.15.45:0/1334349490 >> v2:172.21.15.117:3300/0 conn(0x7f88cc15b740 msgr2=0x7f88cc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 --2- 172.21.15.45:0/1334349490 >> v2:172.21.15.117:3300/0 conn(0x7f88cc15b740 0x7f88cc0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 -- 172.21.15.45:0/1334349490 >> v1:172.21.15.117:6789/0 conn(0x7f88cc1550b0 legacy=0x7f88cc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.756+0000 7f88d1400640 1 -- 172.21.15.45:0/1334349490 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0715d0 con 0x7f88cc0d0040 2024-04-25T10:18:56.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d1400640 1 --2- 172.21.15.45:0/1334349490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc15b200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0d04d0 tx=0x7f88bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d7851e4d6e949e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d0a00640 1 -- 172.21.15.45:0/1334349490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0aa650 con 0x7f88cc0d0040 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d0a00640 1 -- 172.21.15.45:0/1334349490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc0af070 con 0x7f88cc0d0040 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d0a00640 1 -- 172.21.15.45:0/1334349490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0aaa10 con 0x7f88cc0d0040 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 -- 172.21.15.45:0/1334349490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 msgr2=0x7f88cc15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 --2- 172.21.15.45:0/1334349490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc15b200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0d04d0 tx=0x7f88bc01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 -- 172.21.15.45:0/1334349490 shutdown_connections 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 --2- 172.21.15.45:0/1334349490 >> v2:172.21.15.117:3300/0 conn(0x7f88cc15b740 0x7f88cc0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 --2- 172.21.15.45:0/1334349490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc15b200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 -- 172.21.15.45:0/1334349490 >> 172.21.15.45:0/1334349490 conn(0x7f88cc066b70 msgr2=0x7f88cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 -- 172.21.15.45:0/1334349490 shutdown_connections 2024-04-25T10:18:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 -- 172.21.15.45:0/1334349490 wait complete. 2024-04-25T10:18:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 Processor -- start 2024-04-25T10:18:56.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.760+0000 7f88d4800640 1 -- start start 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88cc1550b0 0x7f88cc1493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc15b740 0x7f88cc155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc15bcd0 con 0x7f88cc0d0040 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc15bb50 con 0x7f88cc15b740 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc15c030 con 0x7f88cc1550b0 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60304/0 (socket says 172.21.15.45:60304) 2024-04-25T10:18:56.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 -- 172.21.15.45:0/3819506557 learned_addr learned my addr 172.21.15.45:0/3819506557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d2800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc15b740 0x7f88cc155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1400640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88cc1550b0 0x7f88cc1493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 -- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88cc1550b0 msgr2=0x7f88cc1493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88cc1550b0 0x7f88cc1493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 -- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc15b740 msgr2=0x7f88cc155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc15b740 0x7f88cc155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 -- 172.21.15.45:0/3819506557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc1563f0 con 0x7f88cc0d0040 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1400640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88cc1550b0 0x7f88cc1493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d1e00640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc155490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c450 tx=0x7f88c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0016020 con 0x7f88cc0d0040 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c00052d0 con 0x7f88cc0d0040 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0007260 con 0x7f88cc0d0040 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc1f5e50 con 0x7f88cc0d0040 2024-04-25T10:18:56.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc1f6410 con 0x7f88cc0d0040 2024-04-25T10:18:56.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc04ef30 con 0x7f88cc0d0040 2024-04-25T10:18:56.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.764+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88c0007400 con 0x7f88cc0d0040 2024-04-25T10:18:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.768+0000 7f88c6000640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f88b0066f60 0x7f88b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.768+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88c0084890 con 0x7f88cc0d0040 2024-04-25T10:18:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.768+0000 7f88d1400640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f88b0066f60 0x7f88b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.768+0000 7f88d1400640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f88b0066f60 0x7f88b0069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ae000 tx=0x7f88bc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.772+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88c0020020 con 0x7f88cc0d0040 2024-04-25T10:18:56.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.932+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88cc06c130 con 0x7f88cc0d0040 2024-04-25T10:18:56.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88c6000640 1 -- 172.21.15.45:0/3819506557 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f88cc1563f0 con 0x7f88cc0d0040 2024-04-25T10:18:56.938 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:56.938 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:55.617637+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f88b0066f60 msgr2=0x7f88b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f88b0066f60 0x7f88b0069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ae000 tx=0x7f88bc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 msgr2=0x7f88cc155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc155490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c450 tx=0x7f88c000c910 comp rx=0 tx=0).stop 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 shutdown_connections 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f88b0066f60 0x7f88b0069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88cc15b740 0x7f88cc155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88cc1550b0 0x7f88cc1493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 --2- 172.21.15.45:0/3819506557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88cc0d0040 0x7f88cc155490 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 >> 172.21.15.45:0/3819506557 conn(0x7f88cc066b70 msgr2=0x7f88cc15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 shutdown_connections 2024-04-25T10:18:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:56.936+0000 7f88d4800640 1 -- 172.21.15.45:0/3819506557 wait complete. 2024-04-25T10:18:56.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:18:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.100+0000 7fde15400640 1 Processor -- start 2024-04-25T10:18:57.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde15400640 1 -- start start 2024-04-25T10:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde15400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde10159540 0x7fde100ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde15400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde10159920 con 0x7fde100721f0 2024-04-25T10:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde15400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1015c2f0 con 0x7fde100aebd0 2024-04-25T10:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde15400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1015c5d0 con 0x7fde10159540 2024-04-25T10:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0f200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fde100721f0 0x7fde10070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44450/0 (socket says 172.21.15.45:44450) 2024-04-25T10:18:57.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0f200640 1 -- 172.21.15.45:0/2036198531 learned_addr learned my addr 172.21.15.45:0/2036198531 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0e800640 1 --2- 172.21.15.45:0/2036198531 >> v2:172.21.15.117:3300/0 conn(0x7fde10159540 0x7fde100ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226525727 0 0) 0x7fde10159920 con 0x7fde100721f0 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddf4003530 con 0x7fde100721f0 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242348407 0 0) 0x7fde1015c2f0 con 0x7fde100aebd0 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde10159920 con 0x7fde100aebd0 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0e800640 1 -- 172.21.15.45:0/2036198531 >> v1:172.21.15.117:6789/0 conn(0x7fde100aebd0 legacy=0x7fde10071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0e800640 1 -- 172.21.15.45:0/2036198531 >> v1:172.21.15.45:6789/0 conn(0x7fde100721f0 legacy=0x7fde10070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679278473 0 0) 0x7fddf4003530 con 0x7fde100721f0 2024-04-25T10:18:57.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.104+0000 7fde0e800640 1 -- 172.21.15.45:0/2036198531 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1015c8b0 con 0x7fde10159540 2024-04-25T10:18:57.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde0e800640 1 --2- 172.21.15.45:0/2036198531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde100ae690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fde000097c0 tx=0x7fde0001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616d3efb10d051bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:57.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000b0070 con 0x7fde10159540 2024-04-25T10:18:57.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde000ab400 con 0x7fde10159540 2024-04-25T10:18:57.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde0de00640 1 -- 172.21.15.45:0/2036198531 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08001a00 con 0x7fde10159540 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 -- 172.21.15.45:0/2036198531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 msgr2=0x7fde100ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 --2- 172.21.15.45:0/2036198531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde100ae690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fde000097c0 tx=0x7fde0001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 -- 172.21.15.45:0/2036198531 shutdown_connections 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 --2- 172.21.15.45:0/2036198531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde100ae690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 -- 172.21.15.45:0/2036198531 >> 172.21.15.45:0/2036198531 conn(0x7fde100669a0 msgr2=0x7fde10066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 -- 172.21.15.45:0/2036198531 shutdown_connections 2024-04-25T10:18:57.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 -- 172.21.15.45:0/2036198531 wait complete. 2024-04-25T10:18:57.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.108+0000 7fde15400640 1 Processor -- start 2024-04-25T10:18:57.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- start start 2024-04-25T10:18:57.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 0x7fde1015df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde100aebd0 0x7fde1015e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde1014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde10169ac0 con 0x7fde100721f0 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde10169940 con 0x7fde10159540 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde10169c40 con 0x7fde100aebd0 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 0x7fde1015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 0x7fde1015df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60320/0 (socket says 172.21.15.45:60320) 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 -- 172.21.15.45:0/1578891233 learned_addr learned my addr 172.21.15.45:0/1578891233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0f200640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde1014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde05e00640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde100aebd0 0x7fde1015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 -- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde100aebd0 msgr2=0x7fde1015e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde100aebd0 0x7fde1015e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 -- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 msgr2=0x7fde1014edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde1014edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 -- 172.21.15.45:0/1578891233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1014fb40 con 0x7fde100721f0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde05e00640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde100aebd0 0x7fde1015e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde0e800640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 0x7fde1015df30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fde000097c0 tx=0x7fde0001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000b0050 con 0x7fde100721f0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde000b1040 con 0x7fde100721f0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00004260 con 0x7fde100721f0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1014fe20 con 0x7fde100721f0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde101503e0 con 0x7fde100721f0 2024-04-25T10:18:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.112+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1004ef30 con 0x7fde100721f0 2024-04-25T10:18:57.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.116+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde000be020 con 0x7fde100721f0 2024-04-25T10:18:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.116+0000 7fde07400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddec066f60 0x7fddec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.116+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde0011ee30 con 0x7fde100721f0 2024-04-25T10:18:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.116+0000 7fde05e00640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddec066f60 0x7fddec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.116+0000 7fde05e00640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddec066f60 0x7fddec069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde10151cf0 tx=0x7fddf8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:57.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.120+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde00015fb0 con 0x7fde100721f0 2024-04-25T10:18:57.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:57.316+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fde100ceff0 con 0x7fde100721f0 2024-04-25T10:18:58.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde07400640 1 -- 172.21.15.45:0/1578891233 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde000efda0 con 0x7fde100721f0 2024-04-25T10:18:58.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddec066f60 msgr2=0x7fddec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddec066f60 0x7fddec069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde10151cf0 tx=0x7fddf8009040 comp rx=0 tx=0).stop 2024-04-25T10:18:58.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 msgr2=0x7fde1015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 0x7fde1015df30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fde000097c0 tx=0x7fde0001ac00 comp rx=0 tx=0).stop 2024-04-25T10:18:58.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 shutdown_connections 2024-04-25T10:18:58.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddec066f60 0x7fddec069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde10159540 0x7fde1014edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde100aebd0 0x7fde1015e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 --2- 172.21.15.45:0/1578891233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde100721f0 0x7fde1015df30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 >> 172.21.15.45:0/1578891233 conn(0x7fde100669a0 msgr2=0x7fde1015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:58.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 shutdown_connections 2024-04-25T10:18:58.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.240+0000 7fde15400640 1 -- 172.21.15.45:0/1578891233 wait complete. 2024-04-25T10:18:58.267 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:18:58.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:18:58.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 Processor -- start 2024-04-25T10:18:58.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 -- start start 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36a4070350 0x7f36a415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f36a40b31c0 0x7f36a40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a415b080 con 0x7f36a40aef80 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a415adb0 con 0x7f36a4070350 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a8c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a415eb60 con 0x7f36a40b31c0 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f369be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36a4070350 0x7f36a415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a1e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f36a40aef80 0x7f36a415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44462/0 (socket says 172.21.15.45:44462) 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a1e00640 1 -- 172.21.15.45:0/2336753389 learned_addr learned my addr 172.21.15.45:0/2336753389 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a2800640 1 --2- 172.21.15.45:0/2336753389 >> v2:172.21.15.117:3300/0 conn(0x7f36a40b31c0 0x7f36a40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f369be00640 1 -- 172.21.15.45:0/2336753389 >> v2:172.21.15.117:3300/0 conn(0x7f36a40b31c0 msgr2=0x7f36a40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f369be00640 1 --2- 172.21.15.45:0/2336753389 >> v2:172.21.15.117:3300/0 conn(0x7f36a40b31c0 0x7f36a40b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f369be00640 1 -- 172.21.15.45:0/2336753389 >> v1:172.21.15.45:6789/0 conn(0x7f36a40aef80 legacy=0x7f36a415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f369be00640 1 -- 172.21.15.45:0/2336753389 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a415ee40 con 0x7f36a4070350 2024-04-25T10:18:58.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.412+0000 7f36a1400640 1 -- 172.21.15.45:0/2336753389 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371388006 0 0) 0x7f36a415b080 con 0x7f36a40aef80 2024-04-25T10:18:58.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f369be00640 1 --2- 172.21.15.45:0/2336753389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a4070350 0x7f36a415e480 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f36a40d0750 tx=0x7f368c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d932579b8b65132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:58.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a1400640 1 -- 172.21.15.45:0/2336753389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0b1070 con 0x7f36a4070350 2024-04-25T10:18:58.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a1400640 1 -- 172.21.15.45:0/2336753389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c0ac5f0 con 0x7f36a4070350 2024-04-25T10:18:58.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a1400640 1 -- 172.21.15.45:0/2336753389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0ac9c0 con 0x7f36a4070350 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 -- 172.21.15.45:0/2336753389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a4070350 msgr2=0x7f36a415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 --2- 172.21.15.45:0/2336753389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a4070350 0x7f36a415e480 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f36a40d0750 tx=0x7f368c01c480 comp rx=0 tx=0).stop 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 -- 172.21.15.45:0/2336753389 shutdown_connections 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 --2- 172.21.15.45:0/2336753389 >> v2:172.21.15.117:3300/0 conn(0x7f36a40b31c0 0x7f36a40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 --2- 172.21.15.45:0/2336753389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a4070350 0x7f36a415e480 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 -- 172.21.15.45:0/2336753389 >> 172.21.15.45:0/2336753389 conn(0x7f36a40669a0 msgr2=0x7f36a4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 -- 172.21.15.45:0/2336753389 shutdown_connections 2024-04-25T10:18:58.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.416+0000 7f36a8c00640 1 -- 172.21.15.45:0/2336753389 wait complete. 2024-04-25T10:18:58.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 Processor -- start 2024-04-25T10:18:58.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- start start 2024-04-25T10:18:58.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36a4070350 0x7f36a4158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a40aef80 0x7f36a4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 0x7f36a4155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a41679a0 con 0x7f36a40b31c0 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4167820 con 0x7f36a40aef80 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4167b20 con 0x7f36a4070350 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 0x7f36a4155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 0x7f36a4155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60328/0 (socket says 172.21.15.45:60328) 2024-04-25T10:18:58.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 -- 172.21.15.45:0/4025394040 learned_addr learned my addr 172.21.15.45:0/4025394040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:58.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 -- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36a4070350 msgr2=0x7f36a4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36a4070350 0x7f36a4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 -- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a40aef80 msgr2=0x7f36a4155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a40aef80 0x7f36a4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 -- 172.21.15.45:0/4025394040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a41546d0 con 0x7f36a40b31c0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a2800640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 0x7f36a4155f90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f369c007e60 tx=0x7f369c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c014420 con 0x7f36a40b31c0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c019070 con 0x7f36a40b31c0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c014800 con 0x7f36a40b31c0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a41549b0 con 0x7f36a40b31c0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a4154f70 con 0x7f36a40b31c0 2024-04-25T10:18:58.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.420+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a404ef30 con 0x7f36a40b31c0 2024-04-25T10:18:58.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.424+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f369c0149a0 con 0x7f36a40b31c0 2024-04-25T10:18:58.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.424+0000 7f369a000640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3680066f60 0x7f3680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.424+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f369c085960 con 0x7f36a40b31c0 2024-04-25T10:18:58.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.424+0000 7f36a1e00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3680066f60 0x7f3680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.424+0000 7f36a1e00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3680066f60 0x7f3680069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36940011f0 tx=0x7f3694007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:58.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.428+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f369c0568d0 con 0x7f36a40b31c0 2024-04-25T10:18:58.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:58.624+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f36a40b2fa0 con 0x7f36a40b31c0 2024-04-25T10:18:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.244+0000 7f369a000640 1 -- 172.21.15.45:0/4025394040 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f369c0561d0 con 0x7f36a40b31c0 2024-04-25T10:18:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3680066f60 msgr2=0x7f3680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3680066f60 0x7f3680069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36940011f0 tx=0x7f3694007930 comp rx=0 tx=0).stop 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 msgr2=0x7f36a4155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 0x7f36a4155f90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f369c007e60 tx=0x7f369c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 shutdown_connections 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3680066f60 0x7f3680069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36a40b31c0 0x7f36a4155f90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36a40aef80 0x7f36a4155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 --2- 172.21.15.45:0/4025394040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f36a4070350 0x7f36a4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 >> 172.21.15.45:0/4025394040 conn(0x7f36a40669a0 msgr2=0x7f36a415d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 shutdown_connections 2024-04-25T10:18:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.248+0000 7f36a8c00640 1 -- 172.21.15.45:0/4025394040 wait complete. 2024-04-25T10:18:59.271 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:18:59.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.408+0000 7f54cc800640 1 Processor -- start 2024-04-25T10:18:59.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.408+0000 7f54cc800640 1 -- start start 2024-04-25T10:18:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54cc800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54c415bc40 0x7f54c406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54cc800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cf930 0x7f54c40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54cc800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c40b2770 con 0x7f54c40cda10 2024-04-25T10:18:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54cc800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c406c670 con 0x7f54c415bc40 2024-04-25T10:18:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54cc800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c406c950 con 0x7f54c40cf930 2024-04-25T10:18:59.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54c415bc40 0x7f54c406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54c415bc40 0x7f54c406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60332/0 (socket says 172.21.15.45:60332) 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 -- 172.21.15.45:0/3361091756 learned_addr learned my addr 172.21.15.45:0/3361091756 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54ca800640 1 --2- 172.21.15.45:0/3361091756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cf930 0x7f54c40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 -- 172.21.15.45:0/3361091756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cf930 msgr2=0x7f54c40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 --2- 172.21.15.45:0/3361091756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cf930 0x7f54c40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 -- 172.21.15.45:0/3361091756 >> v1:172.21.15.117:6789/0 conn(0x7f54c40cda10 legacy=0x7f54c40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 -- 172.21.15.45:0/3361091756 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c406cc30 con 0x7f54c415bc40 2024-04-25T10:18:59.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.412+0000 7f54c9e00640 1 --2- 172.21.15.45:0/3361091756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c406c090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009870 tx=0x7f54b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826ad557de0fc31c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54c8a00640 1 -- 172.21.15.45:0/3361091756 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80b0070 con 0x7f54c415bc40 2024-04-25T10:18:59.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54c8a00640 1 -- 172.21.15.45:0/3361091756 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b80ab600 con 0x7f54c415bc40 2024-04-25T10:18:59.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54c8a00640 1 -- 172.21.15.45:0/3361091756 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80ab9d0 con 0x7f54c415bc40 2024-04-25T10:18:59.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- 172.21.15.45:0/3361091756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 msgr2=0x7f54c406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 --2- 172.21.15.45:0/3361091756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c406c090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009870 tx=0x7f54b801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- 172.21.15.45:0/3361091756 shutdown_connections 2024-04-25T10:18:59.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 --2- 172.21.15.45:0/3361091756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cf930 0x7f54c40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 --2- 172.21.15.45:0/3361091756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c406c090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- 172.21.15.45:0/3361091756 >> 172.21.15.45:0/3361091756 conn(0x7f54c4066b70 msgr2=0x7f54c4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- 172.21.15.45:0/3361091756 shutdown_connections 2024-04-25T10:18:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- 172.21.15.45:0/3361091756 wait complete. 2024-04-25T10:18:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 Processor -- start 2024-04-25T10:18:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- start start 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cda10 0x7f54c415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c40cf930 0x7f54c4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c4155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b3890 con 0x7f54c415bc40 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b3710 con 0x7f54c40cf930 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54cc800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b3a10 con 0x7f54c40cda10 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c4155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c4155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60348/0 (socket says 172.21.15.45:60348) 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 -- 172.21.15.45:0/278667905 learned_addr learned my addr 172.21.15.45:0/278667905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54c9e00640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cda10 0x7f54c415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54c9400640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c40cf930 0x7f54c4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 -- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cda10 msgr2=0x7f54c415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cda10 0x7f54c415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 -- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c40cf930 msgr2=0x7f54c4154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.416+0000 7f54ca800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c40cf930 0x7f54c4154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54ca800640 1 -- 172.21.15.45:0/278667905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4154120 con 0x7f54c415bc40 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54c9e00640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cda10 0x7f54c415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54c9400640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c40cf930 0x7f54c4154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54ca800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c4155280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009d30 tx=0x7f54c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0018020 con 0x7f54c415bc40 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c0014480 con 0x7f54c415bc40 2024-04-25T10:18:59.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0014870 con 0x7f54c415bc40 2024-04-25T10:18:59.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c414bb60 con 0x7f54c415bc40 2024-04-25T10:18:59.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c414ffb0 con 0x7f54c415bc40 2024-04-25T10:18:59.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c404ef30 con 0x7f54c415bc40 2024-04-25T10:18:59.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54c001b020 con 0x7f54c415bc40 2024-04-25T10:18:59.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54be000640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54a8066f60 0x7f54a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.420+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54c0085e40 con 0x7f54c415bc40 2024-04-25T10:18:59.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.424+0000 7f54c9e00640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54a8066f60 0x7f54a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.424+0000 7f54c9e00640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54a8066f60 0x7f54a8069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54b80ad000 tx=0x7f54b80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.424+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54c0022030 con 0x7f54c415bc40 2024-04-25T10:18:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.620+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54c40af1c0 con 0x7f54c415bc40 2024-04-25T10:18:59.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54be000640 1 -- 172.21.15.45:0/278667905 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f54c0010040 con 0x7f54c415bc40 2024-04-25T10:18:59.627 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:59.627 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:55.618016+0000","modified":"2024-04-25T10:18:58.627157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3482819589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3482819589},{"type":"v1","addr":"172.21.15.45:6835","nonce":3482819589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:18:59.627 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54a8066f60 msgr2=0x7f54a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54a8066f60 0x7f54a8069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54b80ad000 tx=0x7f54b80a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 msgr2=0x7f54c4155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c4155280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009d30 tx=0x7f54c000c490 comp rx=0 tx=0).stop 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 shutdown_connections 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54a8066f60 0x7f54a8069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54c415bc40 0x7f54c4155280 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54c40cf930 0x7f54c4154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 --2- 172.21.15.45:0/278667905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54c40cda10 0x7f54c415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 >> 172.21.15.45:0/278667905 conn(0x7f54c4066b70 msgr2=0x7f54c4159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 shutdown_connections 2024-04-25T10:18:59.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.624+0000 7f54cc800640 1 -- 172.21.15.45:0/278667905 wait complete. 2024-04-25T10:18:59.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:59.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.784+0000 7f450e000640 1 Processor -- start 2024-04-25T10:18:59.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.784+0000 7f450e000640 1 -- start start 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f450e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45081550b0 0x7f45080cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f450e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450815b910 0x7f45080b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f450e000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4508071050 con 0x7f45080cfe70 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f450e000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450815bda0 con 0x7f45081550b0 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f450e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450815d040 con 0x7f450815b910 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f44ffe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45080cfe70 0x7f450815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44478/0 (socket says 172.21.15.45:44478) 2024-04-25T10:18:59.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f44ffe00640 1 -- 172.21.15.45:0/3520821536 learned_addr learned my addr 172.21.15.45:0/3520821536 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:18:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4507400640 1 --2- 172.21.15.45:0/3520821536 >> v2:172.21.15.117:3300/0 conn(0x7f45081550b0 0x7f45080cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4507e00640 1 --2- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450815b910 0x7f45080b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4506a00640 1 -- 172.21.15.45:0/3520821536 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984779504 0 0) 0x7f4508071050 con 0x7f45080cfe70 2024-04-25T10:18:59.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4506a00640 1 -- 172.21.15.45:0/3520821536 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44ec003530 con 0x7f45080cfe70 2024-04-25T10:18:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4507400640 1 -- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450815b910 msgr2=0x7f45080b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4507400640 1 --2- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450815b910 0x7f45080b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4506a00640 1 -- 172.21.15.45:0/3520821536 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976434447 0 0) 0x7f44ec003530 con 0x7f45080cfe70 2024-04-25T10:18:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4507400640 1 -- 172.21.15.45:0/3520821536 >> v1:172.21.15.45:6789/0 conn(0x7f45080cfe70 legacy=0x7f450815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.788+0000 7f4507400640 1 -- 172.21.15.45:0/3520821536 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080712f0 con 0x7f45081550b0 2024-04-25T10:18:59.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f4507400640 1 --2- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f45080cf930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097f0 tx=0x7f44f801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9520d1c2e6d694e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f4506a00640 1 -- 172.21.15.45:0/3520821536 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80b0070 con 0x7f45081550b0 2024-04-25T10:18:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f4506a00640 1 -- 172.21.15.45:0/3520821536 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f80ab400 con 0x7f45081550b0 2024-04-25T10:18:59.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f4506a00640 1 -- 172.21.15.45:0/3520821536 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0001680 con 0x7f45081550b0 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 -- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 msgr2=0x7f45080cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 --2- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f45080cf930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44f80097f0 tx=0x7f44f801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 -- 172.21.15.45:0/3520821536 shutdown_connections 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 --2- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450815b910 0x7f45080b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 --2- 172.21.15.45:0/3520821536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f45080cf930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 -- 172.21.15.45:0/3520821536 >> 172.21.15.45:0/3520821536 conn(0x7f4508066b70 msgr2=0x7f4508066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 -- 172.21.15.45:0/3520821536 shutdown_connections 2024-04-25T10:18:59.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 -- 172.21.15.45:0/3520821536 wait complete. 2024-04-25T10:18:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 Processor -- start 2024-04-25T10:18:59.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 -- start start 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45080cfe70 0x7f450813f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.792+0000 7f450e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f450813c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 0x7f450813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450815c030 con 0x7f450815b910 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080720b0 con 0x7f45081550b0 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450815c1b0 con 0x7f45080cfe70 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 0x7f450813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45080cfe70 0x7f450813f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 0x7f450813df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60362/0 (socket says 172.21.15.45:60362) 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 -- 172.21.15.45:0/1497330788 learned_addr learned my addr 172.21.15.45:0/1497330788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:18:59.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f44ffe00640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f450813c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 -- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45080cfe70 msgr2=0x7f450813f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45080cfe70 0x7f450813f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 -- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 msgr2=0x7f450813c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f450813c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 -- 172.21.15.45:0/1497330788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450813cbc0 con 0x7f450815b910 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f44ffe00640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f450813c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f4507e00640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 0x7f450813df00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f450000efc0 tx=0x7f450000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500018020 con 0x7f450815b910 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500014480 con 0x7f450815b910 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500014870 con 0x7f450815b910 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450813ce60 con 0x7f450815b910 2024-04-25T10:18:59.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450813d420 con 0x7f450815b910 2024-04-25T10:18:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450804ef30 con 0x7f450815b910 2024-04-25T10:18:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.796+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f450001b020 con 0x7f450815b910 2024-04-25T10:18:59.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.800+0000 7f44ff400640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e4066f60 0x7f44e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.800+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45000859f0 con 0x7f450815b910 2024-04-25T10:18:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.800+0000 7f4507400640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e4066f60 0x7f44e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.800+0000 7f4507400640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e4066f60 0x7f44e4069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44f80ad000 tx=0x7f44f80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.804+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4500022070 con 0x7f450815b910 2024-04-25T10:18:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45081546d0 con 0x7f450815b910 2024-04-25T10:18:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f44ff400640 1 -- 172.21.15.45:0/1497330788 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f4500010040 con 0x7f450815b910 2024-04-25T10:18:59.971 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:18:59.971 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":35,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:18:57.229178+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e4066f60 msgr2=0x7f44e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e4066f60 0x7f44e4069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44f80ad000 tx=0x7f44f80a6000 comp rx=0 tx=0).stop 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 msgr2=0x7f450813df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 0x7f450813df00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f450000efc0 tx=0x7f450000c490 comp rx=0 tx=0).stop 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 shutdown_connections 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e4066f60 0x7f44e4069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450815b910 0x7f450813df00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45081550b0 0x7f450813c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 --2- 172.21.15.45:0/1497330788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45080cfe70 0x7f450813f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 >> 172.21.15.45:0/1497330788 conn(0x7f4508066b70 msgr2=0x7f450806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 shutdown_connections 2024-04-25T10:18:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:18:59.968+0000 7f450e000640 1 -- 172.21.15.45:0/1497330788 wait complete. 2024-04-25T10:18:59.992 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:18:59.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.128+0000 7ff3b4a00640 1 Processor -- start 2024-04-25T10:19:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.128+0000 7ff3b4a00640 1 -- start start 2024-04-25T10:19:00.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3b4a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3b015dc70 0x7ff3b006b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3b4a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff3b015e050 0x7ff3b006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3b4a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b00d2650 con 0x7ff3b00b2970 2024-04-25T10:19:00.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3b4a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b006c010 con 0x7ff3b015dc70 2024-04-25T10:19:00.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3b4a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00d2370 con 0x7ff3b015e050 2024-04-25T10:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3b015dc70 0x7ff3b006b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff3b015dc70 0x7ff3b006b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60372/0 (socket says 172.21.15.45:60372) 2024-04-25T10:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 -- 172.21.15.45:0/3816219836 learned_addr learned my addr 172.21.15.45:0/3816219836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3a6a00640 1 --2- 172.21.15.45:0/3816219836 >> v2:172.21.15.117:3300/0 conn(0x7ff3b015e050 0x7ff3b006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 -- 172.21.15.45:0/3816219836 >> v2:172.21.15.117:3300/0 conn(0x7ff3b015e050 msgr2=0x7ff3b006bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 --2- 172.21.15.45:0/3816219836 >> v2:172.21.15.117:3300/0 conn(0x7ff3b015e050 0x7ff3b006bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 -- 172.21.15.45:0/3816219836 >> v1:172.21.15.117:6789/0 conn(0x7ff3b00b2970 legacy=0x7ff3b00cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3af400640 1 -- 172.21.15.45:0/3816219836 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b006afd0 con 0x7ff3b015dc70 2024-04-25T10:19:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.132+0000 7ff3aea00640 1 -- 172.21.15.45:0/3816219836 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931737947 0 0) 0x7ff3b00d2650 con 0x7ff3b00b2970 2024-04-25T10:19:00.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3af400640 1 --2- 172.21.15.45:0/3816219836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b015dc70 0x7ff3b006b590 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006d000 tx=0x7ff3a001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8677a1bb1659c687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3aea00640 1 -- 172.21.15.45:0/3816219836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00a5670 con 0x7ff3b015dc70 2024-04-25T10:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3aea00640 1 -- 172.21.15.45:0/3816219836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a00a5810 con 0x7ff3b015dc70 2024-04-25T10:19:00.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3aea00640 1 -- 172.21.15.45:0/3816219836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00a5be0 con 0x7ff3b015dc70 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3816219836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b015dc70 msgr2=0x7ff3b006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3816219836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b015dc70 0x7ff3b006b590 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006d000 tx=0x7ff3a001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3816219836 shutdown_connections 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3816219836 >> v2:172.21.15.117:3300/0 conn(0x7ff3b015e050 0x7ff3b006bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3816219836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b015dc70 0x7ff3b006b590 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3816219836 >> 172.21.15.45:0/3816219836 conn(0x7ff3b00d1720 msgr2=0x7ff3b00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3816219836 shutdown_connections 2024-04-25T10:19:00.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3816219836 wait complete. 2024-04-25T10:19:00.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 Processor -- start 2024-04-25T10:19:00.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- start start 2024-04-25T10:19:00.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 0x7ff3b015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3b015dc70 0x7ff3b0159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3b015e050 0x7ff3b0152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00cd400 con 0x7ff3b00b2970 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00cd280 con 0x7ff3b015dc70 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.136+0000 7ff3b4a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b016aa60 con 0x7ff3b015e050 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 0x7ff3b015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 0x7ff3b015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60386/0 (socket says 172.21.15.45:60386) 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3af400640 1 -- 172.21.15.45:0/3036063058 learned_addr learned my addr 172.21.15.45:0/3036063058 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3b015dc70 0x7ff3b0159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3afe00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3b015e050 0x7ff3b0152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 -- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3b015e050 msgr2=0x7ff3b0152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3b015e050 0x7ff3b0152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 -- 172.21.15.45:0/3036063058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 msgr2=0x7ff3b015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 0x7ff3b015b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 -- 172.21.15.45:0/3036063058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b015c2a0 con 0x7ff3b015dc70 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3af400640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 0x7ff3b015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a6a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3b015dc70 0x7ff3b0159470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff39800e840 tx=0x7ff39800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398017070 con 0x7ff3b015dc70 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39800c730 con 0x7ff3b015dc70 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39800cb00 con 0x7ff3b015dc70 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b015c580 con 0x7ff3b015dc70 2024-04-25T10:19:00.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b015cb40 con 0x7ff3b015dc70 2024-04-25T10:19:00.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0075120 con 0x7ff3b015dc70 2024-04-25T10:19:00.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.140+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff39801a070 con 0x7ff3b015dc70 2024-04-25T10:19:00.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.144+0000 7ff3a7e00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff38c066f60 0x7ff38c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.144+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff398085250 con 0x7ff3b015dc70 2024-04-25T10:19:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.144+0000 7ff3af400640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff38c066f60 0x7ff38c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.144+0000 7ff3af400640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff38c066f60 0x7ff38c069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000f90 tx=0x7ff3a00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.148+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff398055af0 con 0x7ff3b015dc70 2024-04-25T10:19:00.202 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-04-25T10:19:00.200+0000 7fd1fb200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:19:00.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.304+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3b00b3ca0 con 0x7ff38c066f60 2024-04-25T10:19:00.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.320+0000 7ff3a7e00640 1 -- 172.21.15.45:0/3036063058 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239619 (secure 0 0 0) 0x7ff3b00b3ca0 con 0x7ff38c066f60 2024-04-25T10:19:00.325 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:00.326 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:19:00.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff38c066f60 msgr2=0x7ff38c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff38c066f60 0x7ff38c069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000f90 tx=0x7ff3a00a7070 comp rx=0 tx=0).stop 2024-04-25T10:19:00.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3b015dc70 msgr2=0x7ff3b0159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3b015dc70 0x7ff3b0159470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff39800e840 tx=0x7ff39800ed00 comp rx=0 tx=0).stop 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 shutdown_connections 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff38c066f60 0x7ff38c069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3b015e050 0x7ff3b0152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3b015dc70 0x7ff3b0159470 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 --2- 172.21.15.45:0/3036063058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b00b2970 0x7ff3b015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 >> 172.21.15.45:0/3036063058 conn(0x7ff3b00d1720 msgr2=0x7ff3b006f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 shutdown_connections 2024-04-25T10:19:00.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.324+0000 7ff3b4a00640 1 -- 172.21.15.45:0/3036063058 wait complete. 2024-04-25T10:19:00.345 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-25T10:18:58.886973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":36,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":240904,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733816,"statfs":{"total":773094113280,"available":772847427584,"internally_reserved":0,"allocated":3960832,"data_stored":2211677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.678356"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260210+0000","last_change":"2024-04-25T10:18:56.258089+0000","last_active":"2024-04-25T10:18:56.260210+0000","last_peered":"2024-04-25T10:18:56.260210+0000","last_clean":"2024-04-25T10:18:56.260210+0000","last_became_active":"2024-04-25T10:18:56.257834+0000","last_became_peered":"2024-04-25T10:18:56.257834+0000","last_unstale":"2024-04-25T10:18:56.260210+0000","last_undegraded":"2024-04-25T10:18:56.260210+0000","last_fullsized":"2024-04-25T10:18:56.260210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:48.224888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.253696+0000","last_change":"2024-04-25T10:18:56.250051+0000","last_active":"2024-04-25T10:18:56.253696+0000","last_peered":"2024-04-25T10:18:56.253696+0000","last_clean":"2024-04-25T10:18:56.253696+0000","last_became_active":"2024-04-25T10:18:56.247576+0000","last_became_peered":"2024-04-25T10:18:56.247576+0000","last_unstale":"2024-04-25T10:18:56.253696+0000","last_undegraded":"2024-04-25T10:18:56.253696+0000","last_fullsized":"2024-04-25T10:18:56.253696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:39.926018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250984+0000","last_change":"2024-04-25T10:18:56.247202+0000","last_active":"2024-04-25T10:18:56.250984+0000","last_peered":"2024-04-25T10:18:56.250984+0000","last_clean":"2024-04-25T10:18:56.250984+0000","last_became_active":"2024-04-25T10:18:56.240273+0000","last_became_peered":"2024-04-25T10:18:56.240273+0000","last_unstale":"2024-04-25T10:18:56.250984+0000","last_undegraded":"2024-04-25T10:18:56.250984+0000","last_fullsized":"2024-04-25T10:18:56.250984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:18.802983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252157+0000","last_change":"2024-04-25T10:18:56.247980+0000","last_active":"2024-04-25T10:18:56.252157+0000","last_peered":"2024-04-25T10:18:56.252157+0000","last_clean":"2024-04-25T10:18:56.252157+0000","last_became_active":"2024-04-25T10:18:56.247334+0000","last_became_peered":"2024-04-25T10:18:56.247334+0000","last_unstale":"2024-04-25T10:18:56.252157+0000","last_undegraded":"2024-04-25T10:18:56.252157+0000","last_fullsized":"2024-04-25T10:18:56.252157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:51.612725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252020+0000","last_change":"2024-04-25T10:18:56.247318+0000","last_active":"2024-04-25T10:18:56.252020+0000","last_peered":"2024-04-25T10:18:56.252020+0000","last_clean":"2024-04-25T10:18:56.252020+0000","last_became_active":"2024-04-25T10:18:56.246972+0000","last_became_peered":"2024-04-25T10:18:56.246972+0000","last_unstale":"2024-04-25T10:18:56.252020+0000","last_undegraded":"2024-04-25T10:18:56.252020+0000","last_fullsized":"2024-04-25T10:18:56.252020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:04.302027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252501+0000","last_change":"2024-04-25T10:18:55.232915+0000","last_active":"2024-04-25T10:18:56.252501+0000","last_peered":"2024-04-25T10:18:56.252501+0000","last_clean":"2024-04-25T10:18:56.252501+0000","last_became_active":"2024-04-25T10:18:55.232635+0000","last_became_peered":"2024-04-25T10:18:55.232635+0000","last_unstale":"2024-04-25T10:18:56.252501+0000","last_undegraded":"2024-04-25T10:18:56.252501+0000","last_fullsized":"2024-04-25T10:18:56.252501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:16.184267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250485+0000","last_change":"2024-04-25T10:18:56.238845+0000","last_active":"2024-04-25T10:18:56.250485+0000","last_peered":"2024-04-25T10:18:56.250485+0000","last_clean":"2024-04-25T10:18:56.250485+0000","last_became_active":"2024-04-25T10:18:56.233835+0000","last_became_peered":"2024-04-25T10:18:56.233835+0000","last_unstale":"2024-04-25T10:18:56.250485+0000","last_undegraded":"2024-04-25T10:18:56.250485+0000","last_fullsized":"2024-04-25T10:18:56.250485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:35.710530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.259903+0000","last_change":"2024-04-25T10:18:55.239850+0000","last_active":"2024-04-25T10:18:56.259903+0000","last_peered":"2024-04-25T10:18:56.259903+0000","last_clean":"2024-04-25T10:18:56.259903+0000","last_became_active":"2024-04-25T10:18:55.236286+0000","last_became_peered":"2024-04-25T10:18:55.236286+0000","last_unstale":"2024-04-25T10:18:56.259903+0000","last_undegraded":"2024-04-25T10:18:56.259903+0000","last_fullsized":"2024-04-25T10:18:56.259903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.659174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250985+0000","last_change":"2024-04-25T10:18:56.245450+0000","last_active":"2024-04-25T10:18:56.250985+0000","last_peered":"2024-04-25T10:18:56.250985+0000","last_clean":"2024-04-25T10:18:56.250985+0000","last_became_active":"2024-04-25T10:18:56.245230+0000","last_became_peered":"2024-04-25T10:18:56.245230+0000","last_unstale":"2024-04-25T10:18:56.250985+0000","last_undegraded":"2024-04-25T10:18:56.250985+0000","last_fullsized":"2024-04-25T10:18:56.250985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:02.976563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252183+0000","last_change":"2024-04-25T10:18:56.248260+0000","last_active":"2024-04-25T10:18:56.252183+0000","last_peered":"2024-04-25T10:18:56.252183+0000","last_clean":"2024-04-25T10:18:56.252183+0000","last_became_active":"2024-04-25T10:18:56.247668+0000","last_became_peered":"2024-04-25T10:18:56.247668+0000","last_unstale":"2024-04-25T10:18:56.252183+0000","last_undegraded":"2024-04-25T10:18:56.252183+0000","last_fullsized":"2024-04-25T10:18:56.252183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:26.759261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260231+0000","last_change":"2024-04-25T10:18:56.258154+0000","last_active":"2024-04-25T10:18:56.260231+0000","last_peered":"2024-04-25T10:18:56.260231+0000","last_clean":"2024-04-25T10:18:56.260231+0000","last_became_active":"2024-04-25T10:18:56.257886+0000","last_became_peered":"2024-04-25T10:18:56.257886+0000","last_unstale":"2024-04-25T10:18:56.260231+0000","last_undegraded":"2024-04-25T10:18:56.260231+0000","last_fullsized":"2024-04-25T10:18:56.260231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.213066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.251009+0000","last_change":"2024-04-25T10:18:56.247101+0000","last_active":"2024-04-25T10:18:56.251009+0000","last_peered":"2024-04-25T10:18:56.251009+0000","last_clean":"2024-04-25T10:18:56.251009+0000","last_became_active":"2024-04-25T10:18:56.239778+0000","last_became_peered":"2024-04-25T10:18:56.239778+0000","last_unstale":"2024-04-25T10:18:56.251009+0000","last_undegraded":"2024-04-25T10:18:56.251009+0000","last_fullsized":"2024-04-25T10:18:56.251009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.437960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.229069+0000","last_change":"2024-04-25T10:18:55.239619+0000","last_active":"2024-04-25T10:18:56.229069+0000","last_peered":"2024-04-25T10:18:56.229069+0000","last_clean":"2024-04-25T10:18:56.229069+0000","last_became_active":"2024-04-25T10:18:55.239342+0000","last_became_peered":"2024-04-25T10:18:55.239342+0000","last_unstale":"2024-04-25T10:18:56.229069+0000","last_undegraded":"2024-04-25T10:18:56.229069+0000","last_fullsized":"2024-04-25T10:18:56.229069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:54.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.228693+0000","last_change":"2024-04-25T10:18:55.240382+0000","last_active":"2024-04-25T10:18:56.228693+0000","last_peered":"2024-04-25T10:18:56.228693+0000","last_clean":"2024-04-25T10:18:56.228693+0000","last_became_active":"2024-04-25T10:18:55.235717+0000","last_became_peered":"2024-04-25T10:18:55.235717+0000","last_unstale":"2024-04-25T10:18:56.228693+0000","last_undegraded":"2024-04-25T10:18:56.228693+0000","last_fullsized":"2024-04-25T10:18:56.228693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:57.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250487+0000","last_change":"2024-04-25T10:18:56.239706+0000","last_active":"2024-04-25T10:18:56.250487+0000","last_peered":"2024-04-25T10:18:56.250487+0000","last_clean":"2024-04-25T10:18:56.250487+0000","last_became_active":"2024-04-25T10:18:56.239021+0000","last_became_peered":"2024-04-25T10:18:56.239021+0000","last_unstale":"2024-04-25T10:18:56.250487+0000","last_undegraded":"2024-04-25T10:18:56.250487+0000","last_fullsized":"2024-04-25T10:18:56.250487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:02.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.238269+0000","last_change":"2024-04-25T10:18:56.234182+0000","last_active":"2024-04-25T10:18:56.238269+0000","last_peered":"2024-04-25T10:18:56.238269+0000","last_clean":"2024-04-25T10:18:56.238269+0000","last_became_active":"2024-04-25T10:18:56.233910+0000","last_became_peered":"2024-04-25T10:18:56.233910+0000","last_unstale":"2024-04-25T10:18:56.238269+0000","last_undegraded":"2024-04-25T10:18:56.238269+0000","last_fullsized":"2024-04-25T10:18:56.238269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:24.496059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.225752+0000","last_change":"2024-04-25T10:18:55.231867+0000","last_active":"2024-04-25T10:18:56.225752+0000","last_peered":"2024-04-25T10:18:56.225752+0000","last_clean":"2024-04-25T10:18:56.225752+0000","last_became_active":"2024-04-25T10:18:55.231431+0000","last_became_peered":"2024-04-25T10:18:55.231431+0000","last_unstale":"2024-04-25T10:18:56.225752+0000","last_undegraded":"2024-04-25T10:18:56.225752+0000","last_fullsized":"2024-04-25T10:18:56.225752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.615886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:18:49.201591+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:49.201591+0000","last_peered":"2024-04-25T10:18:49.201591+0000","last_clean":"2024-04-25T10:18:49.201591+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:49.201591+0000","last_undegraded":"2024-04-25T10:18:49.201591+0000","last_fullsized":"2024-04-25T10:18:49.201591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":10,"acting":10,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":933888,"data_stored":716673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31528,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":933888,"data_stored":716673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31524,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":146806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:00.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.520+0000 7fb61b800640 1 Processor -- start 2024-04-25T10:19:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.520+0000 7fb61b800640 1 -- start start 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb61b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb6140b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb61b800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 0x7fb6140cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb61b800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6140b1870 con 0x7fb6140d1820 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb61b800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6140b1af0 con 0x7fb6140b13e0 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb61b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140b1dd0 con 0x7fb61415e050 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb619800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 0x7fb6140cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb60be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb6140b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb619800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 0x7fb6140cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33398/0 (socket says 172.21.15.45:33398) 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb619800640 1 -- 172.21.15.45:0/4240881901 learned_addr learned my addr 172.21.15.45:0/4240881901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb618e00640 1 --1- 172.21.15.45:0/4240881901 >> v1:172.21.15.117:6789/0 conn(0x7fb6140d1820 0x7fb6140d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:42694/0 (socket says 172.21.15.45:42694) 2024-04-25T10:19:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb60be00640 1 -- 172.21.15.45:0/4240881901 >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 msgr2=0x7fb6140cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb60be00640 1 --2- 172.21.15.45:0/4240881901 >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 0x7fb6140cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb60be00640 1 -- 172.21.15.45:0/4240881901 >> v1:172.21.15.117:6789/0 conn(0x7fb6140d1820 legacy=0x7fb6140d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb60be00640 1 -- 172.21.15.45:0/4240881901 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140b20b0 con 0x7fb61415e050 2024-04-25T10:19:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.524+0000 7fb619800640 1 --2- 172.21.15.45:0/4240881901 >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 0x7fb6140cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb60be00640 1 --2- 172.21.15.45:0/4240881901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb6140b0ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0097c0 tx=0x7fb5fc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ebbdde444fcf08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb613e00640 1 -- 172.21.15.45:0/4240881901 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ab610 con 0x7fb61415e050 2024-04-25T10:19:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb613e00640 1 -- 172.21.15.45:0/4240881901 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0b0070 con 0x7fb61415e050 2024-04-25T10:19:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb613e00640 1 -- 172.21.15.45:0/4240881901 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ab9d0 con 0x7fb61415e050 2024-04-25T10:19:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 -- 172.21.15.45:0/4240881901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 msgr2=0x7fb6140b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 --2- 172.21.15.45:0/4240881901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb6140b0ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0097c0 tx=0x7fb5fc01c410 comp rx=0 tx=0).stop 2024-04-25T10:19:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 -- 172.21.15.45:0/4240881901 shutdown_connections 2024-04-25T10:19:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 --2- 172.21.15.45:0/4240881901 >> v2:172.21.15.117:3300/0 conn(0x7fb6140b13e0 0x7fb6140cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 --2- 172.21.15.45:0/4240881901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb6140b0ea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 -- 172.21.15.45:0/4240881901 >> 172.21.15.45:0/4240881901 conn(0x7fb61409a9d0 msgr2=0x7fb61409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 -- 172.21.15.45:0/4240881901 shutdown_connections 2024-04-25T10:19:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 -- 172.21.15.45:0/4240881901 wait complete. 2024-04-25T10:19:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 Processor -- start 2024-04-25T10:19:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.528+0000 7fb61b800640 1 -- start start 2024-04-25T10:19:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 0x7fb614143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6140d1820 0x7fb614152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb614158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140d0dd0 con 0x7fb6140b13e0 2024-04-25T10:19:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140d0c50 con 0x7fb6140d1820 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140d0f50 con 0x7fb61415e050 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 0x7fb614143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 0x7fb614143c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60388/0 (socket says 172.21.15.45:60388) 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 -- 172.21.15.45:0/4047000511 learned_addr learned my addr 172.21.15.45:0/4047000511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb60be00640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6140d1820 0x7fb614152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb619800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb614158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 -- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 msgr2=0x7fb614158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb614158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 -- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6140d1820 msgr2=0x7fb614152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6140d1820 0x7fb614152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 -- 172.21.15.45:0/4047000511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61415f0b0 con 0x7fb6140b13e0 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb618e00640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 0x7fb614143c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb60400c420 tx=0x7fb60400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040074e0 con 0x7fb6140b13e0 2024-04-25T10:19:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604007680 con 0x7fb6140b13e0 2024-04-25T10:19:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604004a30 con 0x7fb6140b13e0 2024-04-25T10:19:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61415f390 con 0x7fb6140b13e0 2024-04-25T10:19:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61415f950 con 0x7fb6140b13e0 2024-04-25T10:19:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.532+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61404ef30 con 0x7fb6140b13e0 2024-04-25T10:19:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.536+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb60401a020 con 0x7fb6140b13e0 2024-04-25T10:19:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.536+0000 7fb611600640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5f0066f60 0x7fb5f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.536+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb604084850 con 0x7fb6140b13e0 2024-04-25T10:19:00.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.536+0000 7fb60be00640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5f0066f60 0x7fb5f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.536+0000 7fb60be00640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5f0066f60 0x7fb5f0069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0af000 tx=0x7fb5fc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.540+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6140b1870 con 0x7fb6140b13e0 2024-04-25T10:19:00.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.696+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6140b0e50 con 0x7fb5f0066f60 2024-04-25T10:19:00.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.704+0000 7fb611600640 1 -- 172.21.15.45:0/4047000511 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239619 (secure 0 0 0) 0x7fb6140b0e50 con 0x7fb5f0066f60 2024-04-25T10:19:00.709 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:00.710 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.704+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5f0066f60 msgr2=0x7fb5f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5f0066f60 0x7fb5f0069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0af000 tx=0x7fb5fc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 msgr2=0x7fb614143c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 0x7fb614143c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb60400c420 tx=0x7fb60400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 shutdown_connections 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5f0066f60 0x7fb5f0069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb61415e050 0x7fb614158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6140d1820 0x7fb614152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 --2- 172.21.15.45:0/4047000511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6140b13e0 0x7fb614143c80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 >> 172.21.15.45:0/4047000511 conn(0x7fb61409a9d0 msgr2=0x7fb61415cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 shutdown_connections 2024-04-25T10:19:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.708+0000 7fb61b800640 1 -- 172.21.15.45:0/4047000511 wait complete. 2024-04-25T10:19:00.730 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-25T10:18:58.886973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":36,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":240904,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733816,"statfs":{"total":773094113280,"available":772847427584,"internally_reserved":0,"allocated":3960832,"data_stored":2211677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.678356"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260210+0000","last_change":"2024-04-25T10:18:56.258089+0000","last_active":"2024-04-25T10:18:56.260210+0000","last_peered":"2024-04-25T10:18:56.260210+0000","last_clean":"2024-04-25T10:18:56.260210+0000","last_became_active":"2024-04-25T10:18:56.257834+0000","last_became_peered":"2024-04-25T10:18:56.257834+0000","last_unstale":"2024-04-25T10:18:56.260210+0000","last_undegraded":"2024-04-25T10:18:56.260210+0000","last_fullsized":"2024-04-25T10:18:56.260210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:48.224888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.253696+0000","last_change":"2024-04-25T10:18:56.250051+0000","last_active":"2024-04-25T10:18:56.253696+0000","last_peered":"2024-04-25T10:18:56.253696+0000","last_clean":"2024-04-25T10:18:56.253696+0000","last_became_active":"2024-04-25T10:18:56.247576+0000","last_became_peered":"2024-04-25T10:18:56.247576+0000","last_unstale":"2024-04-25T10:18:56.253696+0000","last_undegraded":"2024-04-25T10:18:56.253696+0000","last_fullsized":"2024-04-25T10:18:56.253696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:39.926018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250984+0000","last_change":"2024-04-25T10:18:56.247202+0000","last_active":"2024-04-25T10:18:56.250984+0000","last_peered":"2024-04-25T10:18:56.250984+0000","last_clean":"2024-04-25T10:18:56.250984+0000","last_became_active":"2024-04-25T10:18:56.240273+0000","last_became_peered":"2024-04-25T10:18:56.240273+0000","last_unstale":"2024-04-25T10:18:56.250984+0000","last_undegraded":"2024-04-25T10:18:56.250984+0000","last_fullsized":"2024-04-25T10:18:56.250984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:18.802983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252157+0000","last_change":"2024-04-25T10:18:56.247980+0000","last_active":"2024-04-25T10:18:56.252157+0000","last_peered":"2024-04-25T10:18:56.252157+0000","last_clean":"2024-04-25T10:18:56.252157+0000","last_became_active":"2024-04-25T10:18:56.247334+0000","last_became_peered":"2024-04-25T10:18:56.247334+0000","last_unstale":"2024-04-25T10:18:56.252157+0000","last_undegraded":"2024-04-25T10:18:56.252157+0000","last_fullsized":"2024-04-25T10:18:56.252157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:51.612725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252020+0000","last_change":"2024-04-25T10:18:56.247318+0000","last_active":"2024-04-25T10:18:56.252020+0000","last_peered":"2024-04-25T10:18:56.252020+0000","last_clean":"2024-04-25T10:18:56.252020+0000","last_became_active":"2024-04-25T10:18:56.246972+0000","last_became_peered":"2024-04-25T10:18:56.246972+0000","last_unstale":"2024-04-25T10:18:56.252020+0000","last_undegraded":"2024-04-25T10:18:56.252020+0000","last_fullsized":"2024-04-25T10:18:56.252020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:04.302027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252501+0000","last_change":"2024-04-25T10:18:55.232915+0000","last_active":"2024-04-25T10:18:56.252501+0000","last_peered":"2024-04-25T10:18:56.252501+0000","last_clean":"2024-04-25T10:18:56.252501+0000","last_became_active":"2024-04-25T10:18:55.232635+0000","last_became_peered":"2024-04-25T10:18:55.232635+0000","last_unstale":"2024-04-25T10:18:56.252501+0000","last_undegraded":"2024-04-25T10:18:56.252501+0000","last_fullsized":"2024-04-25T10:18:56.252501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:16.184267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250485+0000","last_change":"2024-04-25T10:18:56.238845+0000","last_active":"2024-04-25T10:18:56.250485+0000","last_peered":"2024-04-25T10:18:56.250485+0000","last_clean":"2024-04-25T10:18:56.250485+0000","last_became_active":"2024-04-25T10:18:56.233835+0000","last_became_peered":"2024-04-25T10:18:56.233835+0000","last_unstale":"2024-04-25T10:18:56.250485+0000","last_undegraded":"2024-04-25T10:18:56.250485+0000","last_fullsized":"2024-04-25T10:18:56.250485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:35.710530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.259903+0000","last_change":"2024-04-25T10:18:55.239850+0000","last_active":"2024-04-25T10:18:56.259903+0000","last_peered":"2024-04-25T10:18:56.259903+0000","last_clean":"2024-04-25T10:18:56.259903+0000","last_became_active":"2024-04-25T10:18:55.236286+0000","last_became_peered":"2024-04-25T10:18:55.236286+0000","last_unstale":"2024-04-25T10:18:56.259903+0000","last_undegraded":"2024-04-25T10:18:56.259903+0000","last_fullsized":"2024-04-25T10:18:56.259903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.659174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250985+0000","last_change":"2024-04-25T10:18:56.245450+0000","last_active":"2024-04-25T10:18:56.250985+0000","last_peered":"2024-04-25T10:18:56.250985+0000","last_clean":"2024-04-25T10:18:56.250985+0000","last_became_active":"2024-04-25T10:18:56.245230+0000","last_became_peered":"2024-04-25T10:18:56.245230+0000","last_unstale":"2024-04-25T10:18:56.250985+0000","last_undegraded":"2024-04-25T10:18:56.250985+0000","last_fullsized":"2024-04-25T10:18:56.250985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:02.976563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252183+0000","last_change":"2024-04-25T10:18:56.248260+0000","last_active":"2024-04-25T10:18:56.252183+0000","last_peered":"2024-04-25T10:18:56.252183+0000","last_clean":"2024-04-25T10:18:56.252183+0000","last_became_active":"2024-04-25T10:18:56.247668+0000","last_became_peered":"2024-04-25T10:18:56.247668+0000","last_unstale":"2024-04-25T10:18:56.252183+0000","last_undegraded":"2024-04-25T10:18:56.252183+0000","last_fullsized":"2024-04-25T10:18:56.252183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:26.759261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260231+0000","last_change":"2024-04-25T10:18:56.258154+0000","last_active":"2024-04-25T10:18:56.260231+0000","last_peered":"2024-04-25T10:18:56.260231+0000","last_clean":"2024-04-25T10:18:56.260231+0000","last_became_active":"2024-04-25T10:18:56.257886+0000","last_became_peered":"2024-04-25T10:18:56.257886+0000","last_unstale":"2024-04-25T10:18:56.260231+0000","last_undegraded":"2024-04-25T10:18:56.260231+0000","last_fullsized":"2024-04-25T10:18:56.260231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.213066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.251009+0000","last_change":"2024-04-25T10:18:56.247101+0000","last_active":"2024-04-25T10:18:56.251009+0000","last_peered":"2024-04-25T10:18:56.251009+0000","last_clean":"2024-04-25T10:18:56.251009+0000","last_became_active":"2024-04-25T10:18:56.239778+0000","last_became_peered":"2024-04-25T10:18:56.239778+0000","last_unstale":"2024-04-25T10:18:56.251009+0000","last_undegraded":"2024-04-25T10:18:56.251009+0000","last_fullsized":"2024-04-25T10:18:56.251009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.437960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.229069+0000","last_change":"2024-04-25T10:18:55.239619+0000","last_active":"2024-04-25T10:18:56.229069+0000","last_peered":"2024-04-25T10:18:56.229069+0000","last_clean":"2024-04-25T10:18:56.229069+0000","last_became_active":"2024-04-25T10:18:55.239342+0000","last_became_peered":"2024-04-25T10:18:55.239342+0000","last_unstale":"2024-04-25T10:18:56.229069+0000","last_undegraded":"2024-04-25T10:18:56.229069+0000","last_fullsized":"2024-04-25T10:18:56.229069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:54.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.228693+0000","last_change":"2024-04-25T10:18:55.240382+0000","last_active":"2024-04-25T10:18:56.228693+0000","last_peered":"2024-04-25T10:18:56.228693+0000","last_clean":"2024-04-25T10:18:56.228693+0000","last_became_active":"2024-04-25T10:18:55.235717+0000","last_became_peered":"2024-04-25T10:18:55.235717+0000","last_unstale":"2024-04-25T10:18:56.228693+0000","last_undegraded":"2024-04-25T10:18:56.228693+0000","last_fullsized":"2024-04-25T10:18:56.228693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:57.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250487+0000","last_change":"2024-04-25T10:18:56.239706+0000","last_active":"2024-04-25T10:18:56.250487+0000","last_peered":"2024-04-25T10:18:56.250487+0000","last_clean":"2024-04-25T10:18:56.250487+0000","last_became_active":"2024-04-25T10:18:56.239021+0000","last_became_peered":"2024-04-25T10:18:56.239021+0000","last_unstale":"2024-04-25T10:18:56.250487+0000","last_undegraded":"2024-04-25T10:18:56.250487+0000","last_fullsized":"2024-04-25T10:18:56.250487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:02.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.238269+0000","last_change":"2024-04-25T10:18:56.234182+0000","last_active":"2024-04-25T10:18:56.238269+0000","last_peered":"2024-04-25T10:18:56.238269+0000","last_clean":"2024-04-25T10:18:56.238269+0000","last_became_active":"2024-04-25T10:18:56.233910+0000","last_became_peered":"2024-04-25T10:18:56.233910+0000","last_unstale":"2024-04-25T10:18:56.238269+0000","last_undegraded":"2024-04-25T10:18:56.238269+0000","last_fullsized":"2024-04-25T10:18:56.238269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:24.496059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.225752+0000","last_change":"2024-04-25T10:18:55.231867+0000","last_active":"2024-04-25T10:18:56.225752+0000","last_peered":"2024-04-25T10:18:56.225752+0000","last_clean":"2024-04-25T10:18:56.225752+0000","last_became_active":"2024-04-25T10:18:55.231431+0000","last_became_peered":"2024-04-25T10:18:55.231431+0000","last_unstale":"2024-04-25T10:18:56.225752+0000","last_undegraded":"2024-04-25T10:18:56.225752+0000","last_fullsized":"2024-04-25T10:18:56.225752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.615886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:55.208844+0000","last_change":"2024-04-25T10:18:55.208844+0000","last_active":"2024-04-25T10:18:55.208844+0000","last_peered":"2024-04-25T10:18:55.208844+0000","last_clean":"2024-04-25T10:18:55.208844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:55.208844+0000","last_undegraded":"2024-04-25T10:18:55.208844+0000","last_fullsized":"2024-04-25T10:18:55.208844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:18:54.202585+0000","last_change":"2024-04-25T10:18:54.202585+0000","last_active":"2024-04-25T10:18:54.202585+0000","last_peered":"2024-04-25T10:18:54.202585+0000","last_clean":"2024-04-25T10:18:54.202585+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:18:54.202585+0000","last_undegraded":"2024-04-25T10:18:54.202585+0000","last_fullsized":"2024-04-25T10:18:54.202585+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":186,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T10:18:49.201591+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:49.201591+0000","last_peered":"2024-04-25T10:18:49.201591+0000","last_clean":"2024-04-25T10:18:49.201591+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:49.201591+0000","last_undegraded":"2024-04-25T10:18:49.201591+0000","last_fullsized":"2024-04-25T10:18:49.201591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":10,"acting":10,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":933888,"data_stored":716673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31528,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":933888,"data_stored":716673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31524,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":339968,"data_stored":126305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":146806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:00.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:00.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b3400640 1 Processor -- start 2024-04-25T10:19:00.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b3400640 1 -- start start 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b3400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20ac0b3190 0x7f20ac06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b3400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac070ed0 con 0x7f20ac071f70 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b3400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0711b0 con 0x7f20ac15a0e0 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b3400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac071490 con 0x7f20ac0b3190 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20a3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f20ac071f70 0x7f20ac0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44488/0 (socket says 172.21.15.45:44488) 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20a3e00640 1 -- 172.21.15.45:0/1895725404 learned_addr learned my addr 172.21.15.45:0/1895725404 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:19:00.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b1400640 1 --2- 172.21.15.45:0/1895725404 >> v2:172.21.15.117:3300/0 conn(0x7f20ac0b3190 0x7f20ac06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20abe00640 1 -- 172.21.15.45:0/1895725404 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255233015 0 0) 0x7f20ac070ed0 con 0x7f20ac071f70 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20abe00640 1 -- 172.21.15.45:0/1895725404 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2090003450 con 0x7f20ac071f70 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20abe00640 1 -- 172.21.15.45:0/1895725404 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009715231 0 0) 0x7f20ac0711b0 con 0x7f20ac15a0e0 2024-04-25T10:19:00.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b1400640 1 -- 172.21.15.45:0/1895725404 >> v1:172.21.15.117:6789/0 conn(0x7f20ac15a0e0 legacy=0x7f20ac15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b1400640 1 -- 172.21.15.45:0/1895725404 >> v1:172.21.15.45:6789/0 conn(0x7f20ac071f70 legacy=0x7f20ac0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.912+0000 7f20b1400640 1 -- 172.21.15.45:0/1895725404 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac071770 con 0x7f20ac0b3190 2024-04-25T10:19:00.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b1400640 1 --2- 172.21.15.45:0/1895725404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac06c090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009d90 tx=0x7f20a401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1caa125f50c7f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20abe00640 1 -- 172.21.15.45:0/1895725404 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40b2070 con 0x7f20ac0b3190 2024-04-25T10:19:00.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20abe00640 1 -- 172.21.15.45:0/1895725404 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a40a5910 con 0x7f20ac0b3190 2024-04-25T10:19:00.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20abe00640 1 -- 172.21.15.45:0/1895725404 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40a5ce0 con 0x7f20ac0b3190 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 -- 172.21.15.45:0/1895725404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 msgr2=0x7f20ac06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 --2- 172.21.15.45:0/1895725404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac06c090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009d90 tx=0x7f20a401c8b0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 -- 172.21.15.45:0/1895725404 shutdown_connections 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 --2- 172.21.15.45:0/1895725404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac06c090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 -- 172.21.15.45:0/1895725404 >> 172.21.15.45:0/1895725404 conn(0x7f20ac066b70 msgr2=0x7f20ac066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 -- 172.21.15.45:0/1895725404 shutdown_connections 2024-04-25T10:19:00.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.916+0000 7f20b3400640 1 -- 172.21.15.45:0/1895725404 wait complete. 2024-04-25T10:19:00.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 Processor -- start 2024-04-25T10:19:00.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 -- start start 2024-04-25T10:19:00.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 0x7f20ac152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20ac15a0e0 0x7f20ac0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac072370 con 0x7f20ac071f70 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac070b50 con 0x7f20ac0b3190 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0724f0 con 0x7f20ac15a0e0 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 0x7f20ac152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 0x7f20ac152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60396/0 (socket says 172.21.15.45:60396) 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 -- 172.21.15.45:0/893167593 learned_addr learned my addr 172.21.15.45:0/893167593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20a3e00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b1400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20ac15a0e0 0x7f20ac0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 -- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20ac15a0e0 msgr2=0x7f20ac0b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20ac15a0e0 0x7f20ac0b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 -- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 msgr2=0x7f20ac15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac15cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 -- 172.21.15.45:0/893167593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac152c90 con 0x7f20ac071f70 2024-04-25T10:19:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20a3e00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac15cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b0a00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 0x7f20ac152750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f209c001ec0 tx=0x7f209c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c017070 con 0x7f20ac071f70 2024-04-25T10:19:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c003600 con 0x7f20ac071f70 2024-04-25T10:19:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c0039f0 con 0x7f20ac071f70 2024-04-25T10:19:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac139cd0 con 0x7f20ac071f70 2024-04-25T10:19:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.920+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac13a250 con 0x7f20ac071f70 2024-04-25T10:19:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.924+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac04ef30 con 0x7f20ac071f70 2024-04-25T10:19:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.924+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f209c01a070 con 0x7f20ac071f70 2024-04-25T10:19:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.924+0000 7f20a9600640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2088066f60 0x7f2088069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.924+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f209c084370 con 0x7f20ac071f70 2024-04-25T10:19:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.924+0000 7f20a3e00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2088066f60 0x7f2088069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.924+0000 7f20a3e00640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2088066f60 0x7f2088069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20ac14cde0 tx=0x7f2094009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:00.928+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20ac152c90 con 0x7f20ac071f70 2024-04-25T10:19:01.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.088+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20ac06d630 con 0x7f2088066f60 2024-04-25T10:19:01.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.092+0000 7f20a9600640 1 -- 172.21.15.45:0/893167593 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251940 (secure 0 0 0) 0x7f20ac06d630 con 0x7f2088066f60 2024-04-25T10:19:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:01.100 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:19:01.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2088066f60 msgr2=0x7f2088069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2088066f60 0x7f2088069420 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20ac14cde0 tx=0x7f2094009040 comp rx=0 tx=0).stop 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 msgr2=0x7f20ac152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 0x7f20ac152750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f209c001ec0 tx=0x7f209c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 shutdown_connections 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2088066f60 0x7f2088069420 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20ac15a0e0 0x7f20ac0b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20ac0b3190 0x7f20ac15cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 --2- 172.21.15.45:0/893167593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20ac071f70 0x7f20ac152750 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 >> 172.21.15.45:0/893167593 conn(0x7f20ac066b70 msgr2=0x7f20ac14f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 shutdown_connections 2024-04-25T10:19:01.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.096+0000 7f20b3400640 1 -- 172.21.15.45:0/893167593 wait complete. 2024-04-25T10:19:01.120 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-25T10:19:00.887417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241224,"kb_used_data":4240,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733496,"statfs":{"total":773094113280,"available":772847099904,"internally_reserved":0,"allocated":4341760,"data_stored":2384732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001448"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098495+0000","last_change":"2024-04-25T10:18:56.253253+0000","last_active":"2024-04-25T10:18:58.098495+0000","last_peered":"2024-04-25T10:18:58.098495+0000","last_clean":"2024-04-25T10:18:58.098495+0000","last_became_active":"2024-04-25T10:18:56.252862+0000","last_became_peered":"2024-04-25T10:18:56.252862+0000","last_unstale":"2024-04-25T10:18:58.098495+0000","last_undegraded":"2024-04-25T10:18:58.098495+0000","last_fullsized":"2024-04-25T10:18:58.098495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:11.669773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237424+0000","last_change":"2024-04-25T10:18:56.249803+0000","last_active":"2024-04-25T10:18:57.237424+0000","last_peered":"2024-04-25T10:18:57.237424+0000","last_clean":"2024-04-25T10:18:57.237424+0000","last_became_active":"2024-04-25T10:18:56.249526+0000","last_became_peered":"2024-04-25T10:18:56.249526+0000","last_unstale":"2024-04-25T10:18:57.237424+0000","last_undegraded":"2024-04-25T10:18:57.237424+0000","last_fullsized":"2024-04-25T10:18:57.237424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532682+0000","last_change":"2024-04-25T10:18:56.251908+0000","last_active":"2024-04-25T10:18:57.532682+0000","last_peered":"2024-04-25T10:18:57.532682+0000","last_clean":"2024-04-25T10:18:57.532682+0000","last_became_active":"2024-04-25T10:18:56.251585+0000","last_became_peered":"2024-04-25T10:18:56.251585+0000","last_unstale":"2024-04-25T10:18:57.532682+0000","last_undegraded":"2024-04-25T10:18:57.532682+0000","last_fullsized":"2024-04-25T10:18:57.532682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:37.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260210+0000","last_change":"2024-04-25T10:18:56.258089+0000","last_active":"2024-04-25T10:18:56.260210+0000","last_peered":"2024-04-25T10:18:56.260210+0000","last_clean":"2024-04-25T10:18:56.260210+0000","last_became_active":"2024-04-25T10:18:56.257834+0000","last_became_peered":"2024-04-25T10:18:56.257834+0000","last_unstale":"2024-04-25T10:18:56.260210+0000","last_undegraded":"2024-04-25T10:18:56.260210+0000","last_fullsized":"2024-04-25T10:18:56.260210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:48.224888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237046+0000","last_change":"2024-04-25T10:18:56.234170+0000","last_active":"2024-04-25T10:18:57.237046+0000","last_peered":"2024-04-25T10:18:57.237046+0000","last_clean":"2024-04-25T10:18:57.237046+0000","last_became_active":"2024-04-25T10:18:56.233918+0000","last_became_peered":"2024-04-25T10:18:56.233918+0000","last_unstale":"2024-04-25T10:18:57.237046+0000","last_undegraded":"2024-04-25T10:18:57.237046+0000","last_fullsized":"2024-04-25T10:18:57.237046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238362+0000","last_change":"2024-04-25T10:18:56.254897+0000","last_active":"2024-04-25T10:18:57.238362+0000","last_peered":"2024-04-25T10:18:57.238362+0000","last_clean":"2024-04-25T10:18:57.238362+0000","last_became_active":"2024-04-25T10:18:56.254641+0000","last_became_peered":"2024-04-25T10:18:56.254641+0000","last_unstale":"2024-04-25T10:18:57.238362+0000","last_undegraded":"2024-04-25T10:18:57.238362+0000","last_fullsized":"2024-04-25T10:18:57.238362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:37.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098293+0000","last_change":"2024-04-25T10:18:56.239390+0000","last_active":"2024-04-25T10:18:58.098293+0000","last_peered":"2024-04-25T10:18:58.098293+0000","last_clean":"2024-04-25T10:18:58.098293+0000","last_became_active":"2024-04-25T10:18:56.239075+0000","last_became_peered":"2024-04-25T10:18:56.239075+0000","last_unstale":"2024-04-25T10:18:58.098293+0000","last_undegraded":"2024-04-25T10:18:58.098293+0000","last_fullsized":"2024-04-25T10:18:58.098293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098140+0000","last_change":"2024-04-25T10:18:56.240095+0000","last_active":"2024-04-25T10:18:58.098140+0000","last_peered":"2024-04-25T10:18:58.098140+0000","last_clean":"2024-04-25T10:18:58.098140+0000","last_became_active":"2024-04-25T10:18:56.239416+0000","last_became_peered":"2024-04-25T10:18:56.239416+0000","last_unstale":"2024-04-25T10:18:58.098140+0000","last_undegraded":"2024-04-25T10:18:58.098140+0000","last_fullsized":"2024-04-25T10:18:58.098140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532815+0000","last_change":"2024-04-25T10:18:56.237743+0000","last_active":"2024-04-25T10:18:57.532815+0000","last_peered":"2024-04-25T10:18:57.532815+0000","last_clean":"2024-04-25T10:18:57.532815+0000","last_became_active":"2024-04-25T10:18:56.237529+0000","last_became_peered":"2024-04-25T10:18:56.237529+0000","last_unstale":"2024-04-25T10:18:57.532815+0000","last_undegraded":"2024-04-25T10:18:57.532815+0000","last_fullsized":"2024-04-25T10:18:57.532815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.140873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.253696+0000","last_change":"2024-04-25T10:18:56.250051+0000","last_active":"2024-04-25T10:18:56.253696+0000","last_peered":"2024-04-25T10:18:56.253696+0000","last_clean":"2024-04-25T10:18:56.253696+0000","last_became_active":"2024-04-25T10:18:56.247576+0000","last_became_peered":"2024-04-25T10:18:56.247576+0000","last_unstale":"2024-04-25T10:18:56.253696+0000","last_undegraded":"2024-04-25T10:18:56.253696+0000","last_fullsized":"2024-04-25T10:18:56.253696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:39.926018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236922+0000","last_change":"2024-04-25T10:18:56.254679+0000","last_active":"2024-04-25T10:18:57.236922+0000","last_peered":"2024-04-25T10:18:57.236922+0000","last_clean":"2024-04-25T10:18:57.236922+0000","last_became_active":"2024-04-25T10:18:56.254334+0000","last_became_peered":"2024-04-25T10:18:56.254334+0000","last_unstale":"2024-04-25T10:18:57.236922+0000","last_undegraded":"2024-04-25T10:18:57.236922+0000","last_fullsized":"2024-04-25T10:18:57.236922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:47.661457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532655+0000","last_change":"2024-04-25T10:18:56.234697+0000","last_active":"2024-04-25T10:18:57.532655+0000","last_peered":"2024-04-25T10:18:57.532655+0000","last_clean":"2024-04-25T10:18:57.532655+0000","last_became_active":"2024-04-25T10:18:56.234433+0000","last_became_peered":"2024-04-25T10:18:56.234433+0000","last_unstale":"2024-04-25T10:18:57.532655+0000","last_undegraded":"2024-04-25T10:18:57.532655+0000","last_fullsized":"2024-04-25T10:18:57.532655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:34.950803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814702+0000","last_change":"2024-04-25T10:18:56.243767+0000","last_active":"2024-04-25T10:18:57.814702+0000","last_peered":"2024-04-25T10:18:57.814702+0000","last_clean":"2024-04-25T10:18:57.814702+0000","last_became_active":"2024-04-25T10:18:56.243469+0000","last_became_peered":"2024-04-25T10:18:56.243469+0000","last_unstale":"2024-04-25T10:18:57.814702+0000","last_undegraded":"2024-04-25T10:18:57.814702+0000","last_fullsized":"2024-04-25T10:18:57.814702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250984+0000","last_change":"2024-04-25T10:18:56.247202+0000","last_active":"2024-04-25T10:18:56.250984+0000","last_peered":"2024-04-25T10:18:56.250984+0000","last_clean":"2024-04-25T10:18:56.250984+0000","last_became_active":"2024-04-25T10:18:56.240273+0000","last_became_peered":"2024-04-25T10:18:56.240273+0000","last_unstale":"2024-04-25T10:18:56.250984+0000","last_undegraded":"2024-04-25T10:18:56.250984+0000","last_fullsized":"2024-04-25T10:18:56.250984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:18.802983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098827+0000","last_change":"2024-04-25T10:18:56.243167+0000","last_active":"2024-04-25T10:18:58.098827+0000","last_peered":"2024-04-25T10:18:58.098827+0000","last_clean":"2024-04-25T10:18:58.098827+0000","last_became_active":"2024-04-25T10:18:56.242874+0000","last_became_peered":"2024-04-25T10:18:56.242874+0000","last_unstale":"2024-04-25T10:18:58.098827+0000","last_undegraded":"2024-04-25T10:18:58.098827+0000","last_fullsized":"2024-04-25T10:18:58.098827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:27.931466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098186+0000","last_change":"2024-04-25T10:18:56.254154+0000","last_active":"2024-04-25T10:18:58.098186+0000","last_peered":"2024-04-25T10:18:58.098186+0000","last_clean":"2024-04-25T10:18:58.098186+0000","last_became_active":"2024-04-25T10:18:56.251652+0000","last_became_peered":"2024-04-25T10:18:56.251652+0000","last_unstale":"2024-04-25T10:18:58.098186+0000","last_undegraded":"2024-04-25T10:18:58.098186+0000","last_fullsized":"2024-04-25T10:18:58.098186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.086557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236907+0000","last_change":"2024-04-25T10:18:56.261187+0000","last_active":"2024-04-25T10:18:57.236907+0000","last_peered":"2024-04-25T10:18:57.236907+0000","last_clean":"2024-04-25T10:18:57.236907+0000","last_became_active":"2024-04-25T10:18:56.259913+0000","last_became_peered":"2024-04-25T10:18:56.259913+0000","last_unstale":"2024-04-25T10:18:57.236907+0000","last_undegraded":"2024-04-25T10:18:57.236907+0000","last_fullsized":"2024-04-25T10:18:57.236907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:31.833160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238694+0000","last_change":"2024-04-25T10:18:56.254191+0000","last_active":"2024-04-25T10:18:57.238694+0000","last_peered":"2024-04-25T10:18:57.238694+0000","last_clean":"2024-04-25T10:18:57.238694+0000","last_became_active":"2024-04-25T10:18:56.253840+0000","last_became_peered":"2024-04-25T10:18:56.253840+0000","last_unstale":"2024-04-25T10:18:57.238694+0000","last_undegraded":"2024-04-25T10:18:57.238694+0000","last_fullsized":"2024-04-25T10:18:57.238694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:46.664788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238638+0000","last_change":"2024-04-25T10:18:56.237511+0000","last_active":"2024-04-25T10:18:57.238638+0000","last_peered":"2024-04-25T10:18:57.238638+0000","last_clean":"2024-04-25T10:18:57.238638+0000","last_became_active":"2024-04-25T10:18:56.237233+0000","last_became_peered":"2024-04-25T10:18:56.237233+0000","last_unstale":"2024-04-25T10:18:57.238638+0000","last_undegraded":"2024-04-25T10:18:57.238638+0000","last_fullsized":"2024-04-25T10:18:57.238638+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.450233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814870+0000","last_change":"2024-04-25T10:18:56.245428+0000","last_active":"2024-04-25T10:18:57.814870+0000","last_peered":"2024-04-25T10:18:57.814870+0000","last_clean":"2024-04-25T10:18:57.814870+0000","last_became_active":"2024-04-25T10:18:56.245143+0000","last_became_peered":"2024-04-25T10:18:56.245143+0000","last_unstale":"2024-04-25T10:18:57.814870+0000","last_undegraded":"2024-04-25T10:18:57.814870+0000","last_fullsized":"2024-04-25T10:18:57.814870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099156+0000","last_change":"2024-04-25T10:18:56.246924+0000","last_active":"2024-04-25T10:18:58.099156+0000","last_peered":"2024-04-25T10:18:58.099156+0000","last_clean":"2024-04-25T10:18:58.099156+0000","last_became_active":"2024-04-25T10:18:56.246327+0000","last_became_peered":"2024-04-25T10:18:56.246327+0000","last_unstale":"2024-04-25T10:18:58.099156+0000","last_undegraded":"2024-04-25T10:18:58.099156+0000","last_fullsized":"2024-04-25T10:18:58.099156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098486+0000","last_change":"2024-04-25T10:18:56.252008+0000","last_active":"2024-04-25T10:18:58.098486+0000","last_peered":"2024-04-25T10:18:58.098486+0000","last_clean":"2024-04-25T10:18:58.098486+0000","last_became_active":"2024-04-25T10:18:56.251724+0000","last_became_peered":"2024-04-25T10:18:56.251724+0000","last_unstale":"2024-04-25T10:18:58.098486+0000","last_undegraded":"2024-04-25T10:18:58.098486+0000","last_fullsized":"2024-04-25T10:18:58.098486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.306720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814498+0000","last_change":"2024-04-25T10:18:56.238540+0000","last_active":"2024-04-25T10:18:57.814498+0000","last_peered":"2024-04-25T10:18:57.814498+0000","last_clean":"2024-04-25T10:18:57.814498+0000","last_became_active":"2024-04-25T10:18:56.238216+0000","last_became_peered":"2024-04-25T10:18:56.238216+0000","last_unstale":"2024-04-25T10:18:57.814498+0000","last_undegraded":"2024-04-25T10:18:57.814498+0000","last_fullsized":"2024-04-25T10:18:57.814498+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:27.931466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098348+0000","last_change":"2024-04-25T10:18:56.240657+0000","last_active":"2024-04-25T10:18:58.098348+0000","last_peered":"2024-04-25T10:18:58.098348+0000","last_clean":"2024-04-25T10:18:58.098348+0000","last_became_active":"2024-04-25T10:18:56.240396+0000","last_became_peered":"2024-04-25T10:18:56.240396+0000","last_unstale":"2024-04-25T10:18:58.098348+0000","last_undegraded":"2024-04-25T10:18:58.098348+0000","last_fullsized":"2024-04-25T10:18:58.098348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252157+0000","last_change":"2024-04-25T10:18:56.247980+0000","last_active":"2024-04-25T10:18:56.252157+0000","last_peered":"2024-04-25T10:18:56.252157+0000","last_clean":"2024-04-25T10:18:56.252157+0000","last_became_active":"2024-04-25T10:18:56.247334+0000","last_became_peered":"2024-04-25T10:18:56.247334+0000","last_unstale":"2024-04-25T10:18:56.252157+0000","last_undegraded":"2024-04-25T10:18:56.252157+0000","last_fullsized":"2024-04-25T10:18:56.252157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:51.612725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252020+0000","last_change":"2024-04-25T10:18:56.247318+0000","last_active":"2024-04-25T10:18:56.252020+0000","last_peered":"2024-04-25T10:18:56.252020+0000","last_clean":"2024-04-25T10:18:56.252020+0000","last_became_active":"2024-04-25T10:18:56.246972+0000","last_became_peered":"2024-04-25T10:18:56.246972+0000","last_unstale":"2024-04-25T10:18:56.252020+0000","last_undegraded":"2024-04-25T10:18:56.252020+0000","last_fullsized":"2024-04-25T10:18:56.252020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:04.302027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237571+0000","last_change":"2024-04-25T10:18:56.239544+0000","last_active":"2024-04-25T10:18:57.237571+0000","last_peered":"2024-04-25T10:18:57.237571+0000","last_clean":"2024-04-25T10:18:57.237571+0000","last_became_active":"2024-04-25T10:18:56.236085+0000","last_became_peered":"2024-04-25T10:18:56.236085+0000","last_unstale":"2024-04-25T10:18:57.237571+0000","last_undegraded":"2024-04-25T10:18:57.237571+0000","last_fullsized":"2024-04-25T10:18:57.237571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:25.904643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098644+0000","last_change":"2024-04-25T10:18:56.257431+0000","last_active":"2024-04-25T10:18:58.098644+0000","last_peered":"2024-04-25T10:18:58.098644+0000","last_clean":"2024-04-25T10:18:58.098644+0000","last_became_active":"2024-04-25T10:18:56.257033+0000","last_became_peered":"2024-04-25T10:18:56.257033+0000","last_unstale":"2024-04-25T10:18:58.098644+0000","last_undegraded":"2024-04-25T10:18:58.098644+0000","last_fullsized":"2024-04-25T10:18:58.098644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:30.757565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099042+0000","last_change":"2024-04-25T10:18:56.247122+0000","last_active":"2024-04-25T10:18:58.099042+0000","last_peered":"2024-04-25T10:18:58.099042+0000","last_clean":"2024-04-25T10:18:58.099042+0000","last_became_active":"2024-04-25T10:18:56.246671+0000","last_became_peered":"2024-04-25T10:18:56.246671+0000","last_unstale":"2024-04-25T10:18:58.099042+0000","last_undegraded":"2024-04-25T10:18:58.099042+0000","last_fullsized":"2024-04-25T10:18:58.099042+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099002+0000","last_change":"2024-04-25T10:18:56.244715+0000","last_active":"2024-04-25T10:18:58.099002+0000","last_peered":"2024-04-25T10:18:58.099002+0000","last_clean":"2024-04-25T10:18:58.099002+0000","last_became_active":"2024-04-25T10:18:56.244456+0000","last_became_peered":"2024-04-25T10:18:56.244456+0000","last_unstale":"2024-04-25T10:18:58.099002+0000","last_undegraded":"2024-04-25T10:18:58.099002+0000","last_fullsized":"2024-04-25T10:18:58.099002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:26.300653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098514+0000","last_change":"2024-04-25T10:18:56.243348+0000","last_active":"2024-04-25T10:18:58.098514+0000","last_peered":"2024-04-25T10:18:58.098514+0000","last_clean":"2024-04-25T10:18:58.098514+0000","last_became_active":"2024-04-25T10:18:56.243050+0000","last_became_peered":"2024-04-25T10:18:56.243050+0000","last_unstale":"2024-04-25T10:18:58.098514+0000","last_undegraded":"2024-04-25T10:18:58.098514+0000","last_fullsized":"2024-04-25T10:18:58.098514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:05.345870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237064+0000","last_change":"2024-04-25T10:18:56.261132+0000","last_active":"2024-04-25T10:18:57.237064+0000","last_peered":"2024-04-25T10:18:57.237064+0000","last_clean":"2024-04-25T10:18:57.237064+0000","last_became_active":"2024-04-25T10:18:56.259485+0000","last_became_peered":"2024-04-25T10:18:56.259485+0000","last_unstale":"2024-04-25T10:18:57.237064+0000","last_undegraded":"2024-04-25T10:18:57.237064+0000","last_fullsized":"2024-04-25T10:18:57.237064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:04.765169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.815019+0000","last_change":"2024-04-25T10:18:56.245528+0000","last_active":"2024-04-25T10:18:57.815019+0000","last_peered":"2024-04-25T10:18:57.815019+0000","last_clean":"2024-04-25T10:18:57.815019+0000","last_became_active":"2024-04-25T10:18:56.245224+0000","last_became_peered":"2024-04-25T10:18:56.245224+0000","last_unstale":"2024-04-25T10:18:57.815019+0000","last_undegraded":"2024-04-25T10:18:57.815019+0000","last_fullsized":"2024-04-25T10:18:57.815019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236935+0000","last_change":"2024-04-25T10:18:55.228870+0000","last_active":"2024-04-25T10:18:57.236935+0000","last_peered":"2024-04-25T10:18:57.236935+0000","last_clean":"2024-04-25T10:18:57.236935+0000","last_became_active":"2024-04-25T10:18:55.228570+0000","last_became_peered":"2024-04-25T10:18:55.228570+0000","last_unstale":"2024-04-25T10:18:57.236935+0000","last_undegraded":"2024-04-25T10:18:57.236935+0000","last_fullsized":"2024-04-25T10:18:57.236935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.897606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098657+0000","last_change":"2024-04-25T10:18:56.251166+0000","last_active":"2024-04-25T10:18:58.098657+0000","last_peered":"2024-04-25T10:18:58.098657+0000","last_clean":"2024-04-25T10:18:58.098657+0000","last_became_active":"2024-04-25T10:18:56.250808+0000","last_became_peered":"2024-04-25T10:18:56.250808+0000","last_unstale":"2024-04-25T10:18:58.098657+0000","last_undegraded":"2024-04-25T10:18:58.098657+0000","last_fullsized":"2024-04-25T10:18:58.098657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:06.007308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252501+0000","last_change":"2024-04-25T10:18:55.232915+0000","last_active":"2024-04-25T10:18:56.252501+0000","last_peered":"2024-04-25T10:18:56.252501+0000","last_clean":"2024-04-25T10:18:56.252501+0000","last_became_active":"2024-04-25T10:18:55.232635+0000","last_became_peered":"2024-04-25T10:18:55.232635+0000","last_unstale":"2024-04-25T10:18:56.252501+0000","last_undegraded":"2024-04-25T10:18:56.252501+0000","last_fullsized":"2024-04-25T10:18:56.252501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:16.184267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098660+0000","last_change":"2024-04-25T10:18:56.259417+0000","last_active":"2024-04-25T10:18:58.098660+0000","last_peered":"2024-04-25T10:18:58.098660+0000","last_clean":"2024-04-25T10:18:58.098660+0000","last_became_active":"2024-04-25T10:18:56.256746+0000","last_became_peered":"2024-04-25T10:18:56.256746+0000","last_unstale":"2024-04-25T10:18:58.098660+0000","last_undegraded":"2024-04-25T10:18:58.098660+0000","last_fullsized":"2024-04-25T10:18:58.098660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.560656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238766+0000","last_change":"2024-04-25T10:18:55.241402+0000","last_active":"2024-04-25T10:18:57.238766+0000","last_peered":"2024-04-25T10:18:57.238766+0000","last_clean":"2024-04-25T10:18:57.238766+0000","last_became_active":"2024-04-25T10:18:55.241057+0000","last_became_peered":"2024-04-25T10:18:55.241057+0000","last_unstale":"2024-04-25T10:18:57.238766+0000","last_undegraded":"2024-04-25T10:18:57.238766+0000","last_fullsized":"2024-04-25T10:18:57.238766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:48.540119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250485+0000","last_change":"2024-04-25T10:18:56.238845+0000","last_active":"2024-04-25T10:18:56.250485+0000","last_peered":"2024-04-25T10:18:56.250485+0000","last_clean":"2024-04-25T10:18:56.250485+0000","last_became_active":"2024-04-25T10:18:56.233835+0000","last_became_peered":"2024-04-25T10:18:56.233835+0000","last_unstale":"2024-04-25T10:18:56.250485+0000","last_undegraded":"2024-04-25T10:18:56.250485+0000","last_fullsized":"2024-04-25T10:18:56.250485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:35.710530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.259903+0000","last_change":"2024-04-25T10:18:55.239850+0000","last_active":"2024-04-25T10:18:56.259903+0000","last_peered":"2024-04-25T10:18:56.259903+0000","last_clean":"2024-04-25T10:18:56.259903+0000","last_became_active":"2024-04-25T10:18:55.236286+0000","last_became_peered":"2024-04-25T10:18:55.236286+0000","last_unstale":"2024-04-25T10:18:56.259903+0000","last_undegraded":"2024-04-25T10:18:56.259903+0000","last_fullsized":"2024-04-25T10:18:56.259903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.659174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532529+0000","last_change":"2024-04-25T10:18:56.229987+0000","last_active":"2024-04-25T10:18:57.532529+0000","last_peered":"2024-04-25T10:18:57.532529+0000","last_clean":"2024-04-25T10:18:57.532529+0000","last_became_active":"2024-04-25T10:18:56.229484+0000","last_became_peered":"2024-04-25T10:18:56.229484+0000","last_unstale":"2024-04-25T10:18:57.532529+0000","last_undegraded":"2024-04-25T10:18:57.532529+0000","last_fullsized":"2024-04-25T10:18:57.532529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:20.518835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098686+0000","last_change":"2024-04-25T10:18:55.233523+0000","last_active":"2024-04-25T10:18:58.098686+0000","last_peered":"2024-04-25T10:18:58.098686+0000","last_clean":"2024-04-25T10:18:58.098686+0000","last_became_active":"2024-04-25T10:18:55.233010+0000","last_became_peered":"2024-04-25T10:18:55.233010+0000","last_unstale":"2024-04-25T10:18:58.098686+0000","last_undegraded":"2024-04-25T10:18:58.098686+0000","last_fullsized":"2024-04-25T10:18:58.098686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:21.018773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098430+0000","last_change":"2024-04-25T10:18:56.244147+0000","last_active":"2024-04-25T10:18:58.098430+0000","last_peered":"2024-04-25T10:18:58.098430+0000","last_clean":"2024-04-25T10:18:58.098430+0000","last_became_active":"2024-04-25T10:18:56.243866+0000","last_became_peered":"2024-04-25T10:18:56.243866+0000","last_unstale":"2024-04-25T10:18:58.098430+0000","last_undegraded":"2024-04-25T10:18:58.098430+0000","last_fullsized":"2024-04-25T10:18:58.098430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:25.904643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098678+0000","last_change":"2024-04-25T10:18:55.233989+0000","last_active":"2024-04-25T10:18:58.098678+0000","last_peered":"2024-04-25T10:18:58.098678+0000","last_clean":"2024-04-25T10:18:58.098678+0000","last_became_active":"2024-04-25T10:18:55.233717+0000","last_became_peered":"2024-04-25T10:18:55.233717+0000","last_unstale":"2024-04-25T10:18:58.098678+0000","last_undegraded":"2024-04-25T10:18:58.098678+0000","last_fullsized":"2024-04-25T10:18:58.098678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:02.604590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532416+0000","last_change":"2024-04-25T10:18:56.235738+0000","last_active":"2024-04-25T10:18:57.532416+0000","last_peered":"2024-04-25T10:18:57.532416+0000","last_clean":"2024-04-25T10:18:57.532416+0000","last_became_active":"2024-04-25T10:18:56.235450+0000","last_became_peered":"2024-04-25T10:18:56.235450+0000","last_unstale":"2024-04-25T10:18:57.532416+0000","last_undegraded":"2024-04-25T10:18:57.532416+0000","last_fullsized":"2024-04-25T10:18:57.532416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:03.610849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238301+0000","last_change":"2024-04-25T10:18:55.244101+0000","last_active":"2024-04-25T10:18:57.238301+0000","last_peered":"2024-04-25T10:18:57.238301+0000","last_clean":"2024-04-25T10:18:57.238301+0000","last_became_active":"2024-04-25T10:18:55.243832+0000","last_became_peered":"2024-04-25T10:18:55.243832+0000","last_unstale":"2024-04-25T10:18:57.238301+0000","last_undegraded":"2024-04-25T10:18:57.238301+0000","last_fullsized":"2024-04-25T10:18:57.238301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:19.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250985+0000","last_change":"2024-04-25T10:18:56.245450+0000","last_active":"2024-04-25T10:18:56.250985+0000","last_peered":"2024-04-25T10:18:56.250985+0000","last_clean":"2024-04-25T10:18:56.250985+0000","last_became_active":"2024-04-25T10:18:56.245230+0000","last_became_peered":"2024-04-25T10:18:56.245230+0000","last_unstale":"2024-04-25T10:18:56.250985+0000","last_undegraded":"2024-04-25T10:18:56.250985+0000","last_fullsized":"2024-04-25T10:18:56.250985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:02.976563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098626+0000","last_change":"2024-04-25T10:18:55.241837+0000","last_active":"2024-04-25T10:18:58.098626+0000","last_peered":"2024-04-25T10:18:58.098626+0000","last_clean":"2024-04-25T10:18:58.098626+0000","last_became_active":"2024-04-25T10:18:55.241552+0000","last_became_peered":"2024-04-25T10:18:55.241552+0000","last_unstale":"2024-04-25T10:18:58.098626+0000","last_undegraded":"2024-04-25T10:18:58.098626+0000","last_fullsized":"2024-04-25T10:18:58.098626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:37.997874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236783+0000","last_change":"2024-04-25T10:18:56.245632+0000","last_active":"2024-04-25T10:18:57.236783+0000","last_peered":"2024-04-25T10:18:57.236783+0000","last_clean":"2024-04-25T10:18:57.236783+0000","last_became_active":"2024-04-25T10:18:56.244499+0000","last_became_peered":"2024-04-25T10:18:56.244499+0000","last_unstale":"2024-04-25T10:18:57.236783+0000","last_undegraded":"2024-04-25T10:18:57.236783+0000","last_fullsized":"2024-04-25T10:18:57.236783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:05.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237522+0000","last_change":"2024-04-25T10:18:55.237962+0000","last_active":"2024-04-25T10:18:57.237522+0000","last_peered":"2024-04-25T10:18:57.237522+0000","last_clean":"2024-04-25T10:18:57.237522+0000","last_became_active":"2024-04-25T10:18:55.234540+0000","last_became_peered":"2024-04-25T10:18:55.234540+0000","last_unstale":"2024-04-25T10:18:57.237522+0000","last_undegraded":"2024-04-25T10:18:57.237522+0000","last_fullsized":"2024-04-25T10:18:57.237522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.443974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252183+0000","last_change":"2024-04-25T10:18:56.248260+0000","last_active":"2024-04-25T10:18:56.252183+0000","last_peered":"2024-04-25T10:18:56.252183+0000","last_clean":"2024-04-25T10:18:56.252183+0000","last_became_active":"2024-04-25T10:18:56.247668+0000","last_became_peered":"2024-04-25T10:18:56.247668+0000","last_unstale":"2024-04-25T10:18:56.252183+0000","last_undegraded":"2024-04-25T10:18:56.252183+0000","last_fullsized":"2024-04-25T10:18:56.252183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:26.759261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237444+0000","last_change":"2024-04-25T10:18:55.234932+0000","last_active":"2024-04-25T10:18:57.237444+0000","last_peered":"2024-04-25T10:18:57.237444+0000","last_clean":"2024-04-25T10:18:57.237444+0000","last_became_active":"2024-04-25T10:18:55.234587+0000","last_became_peered":"2024-04-25T10:18:55.234587+0000","last_unstale":"2024-04-25T10:18:57.237444+0000","last_undegraded":"2024-04-25T10:18:57.237444+0000","last_fullsized":"2024-04-25T10:18:57.237444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:04.346918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260231+0000","last_change":"2024-04-25T10:18:56.258154+0000","last_active":"2024-04-25T10:18:56.260231+0000","last_peered":"2024-04-25T10:18:56.260231+0000","last_clean":"2024-04-25T10:18:56.260231+0000","last_became_active":"2024-04-25T10:18:56.257886+0000","last_became_peered":"2024-04-25T10:18:56.257886+0000","last_unstale":"2024-04-25T10:18:56.260231+0000","last_undegraded":"2024-04-25T10:18:56.260231+0000","last_fullsized":"2024-04-25T10:18:56.260231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.213066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237047+0000","last_change":"2024-04-25T10:18:55.240359+0000","last_active":"2024-04-25T10:18:57.237047+0000","last_peered":"2024-04-25T10:18:57.237047+0000","last_clean":"2024-04-25T10:18:57.237047+0000","last_became_active":"2024-04-25T10:18:55.238481+0000","last_became_peered":"2024-04-25T10:18:55.238481+0000","last_unstale":"2024-04-25T10:18:57.237047+0000","last_undegraded":"2024-04-25T10:18:57.237047+0000","last_fullsized":"2024-04-25T10:18:57.237047+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:23.698128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236771+0000","last_change":"2024-04-25T10:18:56.254595+0000","last_active":"2024-04-25T10:18:57.236771+0000","last_peered":"2024-04-25T10:18:57.236771+0000","last_clean":"2024-04-25T10:18:57.236771+0000","last_became_active":"2024-04-25T10:18:56.253982+0000","last_became_peered":"2024-04-25T10:18:56.253982+0000","last_unstale":"2024-04-25T10:18:57.236771+0000","last_undegraded":"2024-04-25T10:18:57.236771+0000","last_fullsized":"2024-04-25T10:18:57.236771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:01.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814959+0000","last_change":"2024-04-25T10:18:55.242961+0000","last_active":"2024-04-25T10:18:57.814959+0000","last_peered":"2024-04-25T10:18:57.814959+0000","last_clean":"2024-04-25T10:18:57.814959+0000","last_became_active":"2024-04-25T10:18:55.242653+0000","last_became_peered":"2024-04-25T10:18:55.242653+0000","last_unstale":"2024-04-25T10:18:57.814959+0000","last_undegraded":"2024-04-25T10:18:57.814959+0000","last_fullsized":"2024-04-25T10:18:57.814959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:19.512576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814465+0000","last_change":"2024-04-25T10:18:56.235468+0000","last_active":"2024-04-25T10:18:57.814465+0000","last_peered":"2024-04-25T10:18:57.814465+0000","last_clean":"2024-04-25T10:18:57.814465+0000","last_became_active":"2024-04-25T10:18:56.235134+0000","last_became_peered":"2024-04-25T10:18:56.235134+0000","last_unstale":"2024-04-25T10:18:57.814465+0000","last_undegraded":"2024-04-25T10:18:57.814465+0000","last_fullsized":"2024-04-25T10:18:57.814465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:22.129702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098585+0000","last_change":"2024-04-25T10:18:55.241199+0000","last_active":"2024-04-25T10:18:58.098585+0000","last_peered":"2024-04-25T10:18:58.098585+0000","last_clean":"2024-04-25T10:18:58.098585+0000","last_became_active":"2024-04-25T10:18:55.240168+0000","last_became_peered":"2024-04-25T10:18:55.240168+0000","last_unstale":"2024-04-25T10:18:58.098585+0000","last_undegraded":"2024-04-25T10:18:58.098585+0000","last_fullsized":"2024-04-25T10:18:58.098585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:33.944544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.251009+0000","last_change":"2024-04-25T10:18:56.247101+0000","last_active":"2024-04-25T10:18:56.251009+0000","last_peered":"2024-04-25T10:18:56.251009+0000","last_clean":"2024-04-25T10:18:56.251009+0000","last_became_active":"2024-04-25T10:18:56.239778+0000","last_became_peered":"2024-04-25T10:18:56.239778+0000","last_unstale":"2024-04-25T10:18:56.251009+0000","last_undegraded":"2024-04-25T10:18:56.251009+0000","last_fullsized":"2024-04-25T10:18:56.251009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.437960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.229069+0000","last_change":"2024-04-25T10:18:55.239619+0000","last_active":"2024-04-25T10:18:56.229069+0000","last_peered":"2024-04-25T10:18:56.229069+0000","last_clean":"2024-04-25T10:18:56.229069+0000","last_became_active":"2024-04-25T10:18:55.239342+0000","last_became_peered":"2024-04-25T10:18:55.239342+0000","last_unstale":"2024-04-25T10:18:56.229069+0000","last_undegraded":"2024-04-25T10:18:56.229069+0000","last_fullsized":"2024-04-25T10:18:56.229069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:54.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098479+0000","last_change":"2024-04-25T10:18:56.239731+0000","last_active":"2024-04-25T10:18:58.098479+0000","last_peered":"2024-04-25T10:18:58.098479+0000","last_clean":"2024-04-25T10:18:58.098479+0000","last_became_active":"2024-04-25T10:18:56.239151+0000","last_became_peered":"2024-04-25T10:18:56.239151+0000","last_unstale":"2024-04-25T10:18:58.098479+0000","last_undegraded":"2024-04-25T10:18:58.098479+0000","last_fullsized":"2024-04-25T10:18:58.098479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:24.279467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.228693+0000","last_change":"2024-04-25T10:18:55.240382+0000","last_active":"2024-04-25T10:18:56.228693+0000","last_peered":"2024-04-25T10:18:56.228693+0000","last_clean":"2024-04-25T10:18:56.228693+0000","last_became_active":"2024-04-25T10:18:55.235717+0000","last_became_peered":"2024-04-25T10:18:55.235717+0000","last_unstale":"2024-04-25T10:18:56.228693+0000","last_undegraded":"2024-04-25T10:18:56.228693+0000","last_fullsized":"2024-04-25T10:18:56.228693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:57.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236616+0000","last_change":"2024-04-25T10:18:56.260154+0000","last_active":"2024-04-25T10:18:57.236616+0000","last_peered":"2024-04-25T10:18:57.236616+0000","last_clean":"2024-04-25T10:18:57.236616+0000","last_became_active":"2024-04-25T10:18:56.259124+0000","last_became_peered":"2024-04-25T10:18:56.259124+0000","last_unstale":"2024-04-25T10:18:57.236616+0000","last_undegraded":"2024-04-25T10:18:57.236616+0000","last_fullsized":"2024-04-25T10:18:57.236616+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:59.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237057+0000","last_change":"2024-04-25T10:18:55.241451+0000","last_active":"2024-04-25T10:18:57.237057+0000","last_peered":"2024-04-25T10:18:57.237057+0000","last_clean":"2024-04-25T10:18:57.237057+0000","last_became_active":"2024-04-25T10:18:55.241168+0000","last_became_peered":"2024-04-25T10:18:55.241168+0000","last_unstale":"2024-04-25T10:18:57.237057+0000","last_undegraded":"2024-04-25T10:18:57.237057+0000","last_fullsized":"2024-04-25T10:18:57.237057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:48.653430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236854+0000","last_change":"2024-04-25T10:18:56.253037+0000","last_active":"2024-04-25T10:18:57.236854+0000","last_peered":"2024-04-25T10:18:57.236854+0000","last_clean":"2024-04-25T10:18:57.236854+0000","last_became_active":"2024-04-25T10:18:56.252771+0000","last_became_peered":"2024-04-25T10:18:56.252771+0000","last_unstale":"2024-04-25T10:18:57.236854+0000","last_undegraded":"2024-04-25T10:18:57.236854+0000","last_fullsized":"2024-04-25T10:18:57.236854+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:07.919477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238867+0000","last_change":"2024-04-25T10:18:55.244388+0000","last_active":"2024-04-25T10:18:57.238867+0000","last_peered":"2024-04-25T10:18:57.238867+0000","last_clean":"2024-04-25T10:18:57.238867+0000","last_became_active":"2024-04-25T10:18:55.244013+0000","last_became_peered":"2024-04-25T10:18:55.244013+0000","last_unstale":"2024-04-25T10:18:57.238867+0000","last_undegraded":"2024-04-25T10:18:57.238867+0000","last_fullsized":"2024-04-25T10:18:57.238867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:06.417351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532826+0000","last_change":"2024-04-25T10:18:56.252562+0000","last_active":"2024-04-25T10:18:57.532826+0000","last_peered":"2024-04-25T10:18:57.532826+0000","last_clean":"2024-04-25T10:18:57.532826+0000","last_became_active":"2024-04-25T10:18:56.252305+0000","last_became_peered":"2024-04-25T10:18:56.252305+0000","last_unstale":"2024-04-25T10:18:57.532826+0000","last_undegraded":"2024-04-25T10:18:57.532826+0000","last_fullsized":"2024-04-25T10:18:57.532826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.450233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098217+0000","last_change":"2024-04-25T10:18:55.242568+0000","last_active":"2024-04-25T10:18:58.098217+0000","last_peered":"2024-04-25T10:18:58.098217+0000","last_clean":"2024-04-25T10:18:58.098217+0000","last_became_active":"2024-04-25T10:18:55.242249+0000","last_became_peered":"2024-04-25T10:18:55.242249+0000","last_unstale":"2024-04-25T10:18:58.098217+0000","last_undegraded":"2024-04-25T10:18:58.098217+0000","last_fullsized":"2024-04-25T10:18:58.098217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237336+0000","last_change":"2024-04-25T10:18:56.230062+0000","last_active":"2024-04-25T10:18:57.237336+0000","last_peered":"2024-04-25T10:18:57.237336+0000","last_clean":"2024-04-25T10:18:57.237336+0000","last_became_active":"2024-04-25T10:18:56.229753+0000","last_became_peered":"2024-04-25T10:18:56.229753+0000","last_unstale":"2024-04-25T10:18:57.237336+0000","last_undegraded":"2024-04-25T10:18:57.237336+0000","last_fullsized":"2024-04-25T10:18:57.237336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237044+0000","last_change":"2024-04-25T10:18:55.226724+0000","last_active":"2024-04-25T10:18:57.237044+0000","last_peered":"2024-04-25T10:18:57.237044+0000","last_clean":"2024-04-25T10:18:57.237044+0000","last_became_active":"2024-04-25T10:18:55.226402+0000","last_became_peered":"2024-04-25T10:18:55.226402+0000","last_unstale":"2024-04-25T10:18:57.237044+0000","last_undegraded":"2024-04-25T10:18:57.237044+0000","last_fullsized":"2024-04-25T10:18:57.237044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:50.978256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238622+0000","last_change":"2024-04-25T10:18:56.243829+0000","last_active":"2024-04-25T10:18:57.238622+0000","last_peered":"2024-04-25T10:18:57.238622+0000","last_clean":"2024-04-25T10:18:57.238622+0000","last_became_active":"2024-04-25T10:18:56.243579+0000","last_became_peered":"2024-04-25T10:18:56.243579+0000","last_unstale":"2024-04-25T10:18:57.238622+0000","last_undegraded":"2024-04-25T10:18:57.238622+0000","last_fullsized":"2024-04-25T10:18:57.238622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.140873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099156+0000","last_change":"2024-04-25T10:18:55.237736+0000","last_active":"2024-04-25T10:18:58.099156+0000","last_peered":"2024-04-25T10:18:58.099156+0000","last_clean":"2024-04-25T10:18:58.099156+0000","last_became_active":"2024-04-25T10:18:55.237487+0000","last_became_peered":"2024-04-25T10:18:55.237487+0000","last_unstale":"2024-04-25T10:18:58.099156+0000","last_undegraded":"2024-04-25T10:18:58.099156+0000","last_fullsized":"2024-04-25T10:18:58.099156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814574+0000","last_change":"2024-04-25T10:18:56.248910+0000","last_active":"2024-04-25T10:18:57.814574+0000","last_peered":"2024-04-25T10:18:57.814574+0000","last_clean":"2024-04-25T10:18:57.814574+0000","last_became_active":"2024-04-25T10:18:56.246574+0000","last_became_peered":"2024-04-25T10:18:56.246574+0000","last_unstale":"2024-04-25T10:18:57.814574+0000","last_undegraded":"2024-04-25T10:18:57.814574+0000","last_fullsized":"2024-04-25T10:18:57.814574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:26.300653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237128+0000","last_change":"2024-04-25T10:18:55.236067+0000","last_active":"2024-04-25T10:18:57.237128+0000","last_peered":"2024-04-25T10:18:57.237128+0000","last_clean":"2024-04-25T10:18:57.237128+0000","last_became_active":"2024-04-25T10:18:55.235652+0000","last_became_peered":"2024-04-25T10:18:55.235652+0000","last_unstale":"2024-04-25T10:18:57.237128+0000","last_undegraded":"2024-04-25T10:18:57.237128+0000","last_fullsized":"2024-04-25T10:18:57.237128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:45.658529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237089+0000","last_change":"2024-04-25T10:18:56.241808+0000","last_active":"2024-04-25T10:18:57.237089+0000","last_peered":"2024-04-25T10:18:57.237089+0000","last_clean":"2024-04-25T10:18:57.237089+0000","last_became_active":"2024-04-25T10:18:56.241407+0000","last_became_peered":"2024-04-25T10:18:56.241407+0000","last_unstale":"2024-04-25T10:18:57.237089+0000","last_undegraded":"2024-04-25T10:18:57.237089+0000","last_fullsized":"2024-04-25T10:18:57.237089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098721+0000","last_change":"2024-04-25T10:18:55.240208+0000","last_active":"2024-04-25T10:18:58.098721+0000","last_peered":"2024-04-25T10:18:58.098721+0000","last_clean":"2024-04-25T10:18:58.098721+0000","last_became_active":"2024-04-25T10:18:55.239948+0000","last_became_peered":"2024-04-25T10:18:55.239948+0000","last_unstale":"2024-04-25T10:18:58.098721+0000","last_undegraded":"2024-04-25T10:18:58.098721+0000","last_fullsized":"2024-04-25T10:18:58.098721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:19.512576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814473+0000","last_change":"2024-04-25T10:18:56.243705+0000","last_active":"2024-04-25T10:18:57.814473+0000","last_peered":"2024-04-25T10:18:57.814473+0000","last_clean":"2024-04-25T10:18:57.814473+0000","last_became_active":"2024-04-25T10:18:56.241679+0000","last_became_peered":"2024-04-25T10:18:56.241679+0000","last_unstale":"2024-04-25T10:18:57.814473+0000","last_undegraded":"2024-04-25T10:18:57.814473+0000","last_fullsized":"2024-04-25T10:18:57.814473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.086557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532490+0000","last_change":"2024-04-25T10:18:55.233741+0000","last_active":"2024-04-25T10:18:57.532490+0000","last_peered":"2024-04-25T10:18:57.532490+0000","last_clean":"2024-04-25T10:18:57.532490+0000","last_became_active":"2024-04-25T10:18:55.228199+0000","last_became_peered":"2024-04-25T10:18:55.228199+0000","last_unstale":"2024-04-25T10:18:57.532490+0000","last_undegraded":"2024-04-25T10:18:57.532490+0000","last_fullsized":"2024-04-25T10:18:57.532490+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.605460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532541+0000","last_change":"2024-04-25T10:18:56.242696+0000","last_active":"2024-04-25T10:18:57.532541+0000","last_peered":"2024-04-25T10:18:57.532541+0000","last_clean":"2024-04-25T10:18:57.532541+0000","last_became_active":"2024-04-25T10:18:56.242407+0000","last_became_peered":"2024-04-25T10:18:56.242407+0000","last_unstale":"2024-04-25T10:18:57.532541+0000","last_undegraded":"2024-04-25T10:18:57.532541+0000","last_fullsized":"2024-04-25T10:18:57.532541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.025032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236550+0000","last_change":"2024-04-25T10:18:55.231050+0000","last_active":"2024-04-25T10:18:57.236550+0000","last_peered":"2024-04-25T10:18:57.236550+0000","last_clean":"2024-04-25T10:18:57.236550+0000","last_became_active":"2024-04-25T10:18:55.230645+0000","last_became_peered":"2024-04-25T10:18:55.230645+0000","last_unstale":"2024-04-25T10:18:57.236550+0000","last_undegraded":"2024-04-25T10:18:57.236550+0000","last_fullsized":"2024-04-25T10:18:57.236550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:25.294394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238881+0000","last_change":"2024-04-25T10:18:56.242781+0000","last_active":"2024-04-25T10:18:57.238881+0000","last_peered":"2024-04-25T10:18:57.238881+0000","last_clean":"2024-04-25T10:18:57.238881+0000","last_became_active":"2024-04-25T10:18:56.242495+0000","last_became_peered":"2024-04-25T10:18:56.242495+0000","last_unstale":"2024-04-25T10:18:57.238881+0000","last_undegraded":"2024-04-25T10:18:57.238881+0000","last_fullsized":"2024-04-25T10:18:57.238881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:05.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814954+0000","last_change":"2024-04-25T10:18:55.238477+0000","last_active":"2024-04-25T10:18:57.814954+0000","last_peered":"2024-04-25T10:18:57.814954+0000","last_clean":"2024-04-25T10:18:57.814954+0000","last_became_active":"2024-04-25T10:18:55.238126+0000","last_became_peered":"2024-04-25T10:18:55.238126+0000","last_unstale":"2024-04-25T10:18:57.814954+0000","last_undegraded":"2024-04-25T10:18:57.814954+0000","last_fullsized":"2024-04-25T10:18:57.814954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:37.997874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236882+0000","last_change":"2024-04-25T10:18:56.249055+0000","last_active":"2024-04-25T10:18:57.236882+0000","last_peered":"2024-04-25T10:18:57.236882+0000","last_clean":"2024-04-25T10:18:57.236882+0000","last_became_active":"2024-04-25T10:18:56.248708+0000","last_became_peered":"2024-04-25T10:18:56.248708+0000","last_unstale":"2024-04-25T10:18:57.236882+0000","last_undegraded":"2024-04-25T10:18:57.236882+0000","last_fullsized":"2024-04-25T10:18:57.236882+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:26.665895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.226720+0000","last_change":"2024-04-25T10:18:55.236135+0000","last_active":"2024-04-25T10:18:58.226720+0000","last_peered":"2024-04-25T10:18:58.226720+0000","last_clean":"2024-04-25T10:18:58.226720+0000","last_became_active":"2024-04-25T10:18:55.235788+0000","last_became_peered":"2024-04-25T10:18:55.235788+0000","last_unstale":"2024-04-25T10:18:58.226720+0000","last_undegraded":"2024-04-25T10:18:58.226720+0000","last_fullsized":"2024-04-25T10:18:58.226720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:57.667019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238907+0000","last_change":"2024-04-25T10:18:56.251890+0000","last_active":"2024-04-25T10:18:57.238907+0000","last_peered":"2024-04-25T10:18:57.238907+0000","last_clean":"2024-04-25T10:18:57.238907+0000","last_became_active":"2024-04-25T10:18:56.251588+0000","last_became_peered":"2024-04-25T10:18:56.251588+0000","last_unstale":"2024-04-25T10:18:57.238907+0000","last_undegraded":"2024-04-25T10:18:57.238907+0000","last_fullsized":"2024-04-25T10:18:57.238907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:58.673279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.815004+0000","last_change":"2024-04-25T10:18:55.236899+0000","last_active":"2024-04-25T10:18:57.815004+0000","last_peered":"2024-04-25T10:18:57.815004+0000","last_clean":"2024-04-25T10:18:57.815004+0000","last_became_active":"2024-04-25T10:18:55.236580+0000","last_became_peered":"2024-04-25T10:18:55.236580+0000","last_unstale":"2024-04-25T10:18:57.815004+0000","last_undegraded":"2024-04-25T10:18:57.815004+0000","last_fullsized":"2024-04-25T10:18:57.815004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:21.018773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099216+0000","last_change":"2024-04-25T10:18:56.235360+0000","last_active":"2024-04-25T10:18:58.099216+0000","last_peered":"2024-04-25T10:18:58.099216+0000","last_clean":"2024-04-25T10:18:58.099216+0000","last_became_active":"2024-04-25T10:18:56.235089+0000","last_became_peered":"2024-04-25T10:18:56.235089+0000","last_unstale":"2024-04-25T10:18:58.099216+0000","last_undegraded":"2024-04-25T10:18:58.099216+0000","last_fullsized":"2024-04-25T10:18:58.099216+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814753+0000","last_change":"2024-04-25T10:18:55.238591+0000","last_active":"2024-04-25T10:18:57.814753+0000","last_peered":"2024-04-25T10:18:57.814753+0000","last_clean":"2024-04-25T10:18:57.814753+0000","last_became_active":"2024-04-25T10:18:55.238176+0000","last_became_peered":"2024-04-25T10:18:55.238176+0000","last_unstale":"2024-04-25T10:18:57.814753+0000","last_undegraded":"2024-04-25T10:18:57.814753+0000","last_fullsized":"2024-04-25T10:18:57.814753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:02.604590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250487+0000","last_change":"2024-04-25T10:18:56.239706+0000","last_active":"2024-04-25T10:18:56.250487+0000","last_peered":"2024-04-25T10:18:56.250487+0000","last_clean":"2024-04-25T10:18:56.250487+0000","last_became_active":"2024-04-25T10:18:56.239021+0000","last_became_peered":"2024-04-25T10:18:56.239021+0000","last_unstale":"2024-04-25T10:18:56.250487+0000","last_undegraded":"2024-04-25T10:18:56.250487+0000","last_fullsized":"2024-04-25T10:18:56.250487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:02.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814556+0000","last_change":"2024-04-25T10:18:55.242723+0000","last_active":"2024-04-25T10:18:57.814556+0000","last_peered":"2024-04-25T10:18:57.814556+0000","last_clean":"2024-04-25T10:18:57.814556+0000","last_became_active":"2024-04-25T10:18:55.242338+0000","last_became_peered":"2024-04-25T10:18:55.242338+0000","last_unstale":"2024-04-25T10:18:57.814556+0000","last_undegraded":"2024-04-25T10:18:57.814556+0000","last_fullsized":"2024-04-25T10:18:57.814556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:03.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.238269+0000","last_change":"2024-04-25T10:18:56.234182+0000","last_active":"2024-04-25T10:18:56.238269+0000","last_peered":"2024-04-25T10:18:56.238269+0000","last_clean":"2024-04-25T10:18:56.238269+0000","last_became_active":"2024-04-25T10:18:56.233910+0000","last_became_peered":"2024-04-25T10:18:56.233910+0000","last_unstale":"2024-04-25T10:18:56.238269+0000","last_undegraded":"2024-04-25T10:18:56.238269+0000","last_fullsized":"2024-04-25T10:18:56.238269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:24.496059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098781+0000","last_change":"2024-04-25T10:18:55.241653+0000","last_active":"2024-04-25T10:18:58.098781+0000","last_peered":"2024-04-25T10:18:58.098781+0000","last_clean":"2024-04-25T10:18:58.098781+0000","last_became_active":"2024-04-25T10:18:55.239465+0000","last_became_peered":"2024-04-25T10:18:55.239465+0000","last_unstale":"2024-04-25T10:18:58.098781+0000","last_undegraded":"2024-04-25T10:18:58.098781+0000","last_fullsized":"2024-04-25T10:18:58.098781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:03.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237037+0000","last_change":"2024-04-25T10:18:56.250780+0000","last_active":"2024-04-25T10:18:57.237037+0000","last_peered":"2024-04-25T10:18:57.237037+0000","last_clean":"2024-04-25T10:18:57.237037+0000","last_became_active":"2024-04-25T10:18:56.250540+0000","last_became_peered":"2024-04-25T10:18:56.250540+0000","last_unstale":"2024-04-25T10:18:57.237037+0000","last_undegraded":"2024-04-25T10:18:57.237037+0000","last_fullsized":"2024-04-25T10:18:57.237037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.656466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.225752+0000","last_change":"2024-04-25T10:18:55.231867+0000","last_active":"2024-04-25T10:18:56.225752+0000","last_peered":"2024-04-25T10:18:56.225752+0000","last_clean":"2024-04-25T10:18:56.225752+0000","last_became_active":"2024-04-25T10:18:55.231431+0000","last_became_peered":"2024-04-25T10:18:55.231431+0000","last_unstale":"2024-04-25T10:18:56.225752+0000","last_undegraded":"2024-04-25T10:18:56.225752+0000","last_fullsized":"2024-04-25T10:18:56.225752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.615886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099151+0000","last_change":"2024-04-25T10:18:56.234895+0000","last_active":"2024-04-25T10:18:58.099151+0000","last_peered":"2024-04-25T10:18:58.099151+0000","last_clean":"2024-04-25T10:18:58.099151+0000","last_became_active":"2024-04-25T10:18:56.234511+0000","last_became_peered":"2024-04-25T10:18:56.234511+0000","last_unstale":"2024-04-25T10:18:58.099151+0000","last_undegraded":"2024-04-25T10:18:58.099151+0000","last_fullsized":"2024-04-25T10:18:58.099151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237143+0000","last_change":"2024-04-25T10:18:55.225543+0000","last_active":"2024-04-25T10:18:57.237143+0000","last_peered":"2024-04-25T10:18:57.237143+0000","last_clean":"2024-04-25T10:18:57.237143+0000","last_became_active":"2024-04-25T10:18:55.225225+0000","last_became_peered":"2024-04-25T10:18:55.225225+0000","last_unstale":"2024-04-25T10:18:57.237143+0000","last_undegraded":"2024-04-25T10:18:57.237143+0000","last_fullsized":"2024-04-25T10:18:57.237143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:59.193894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532479+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:57.532479+0000","last_peered":"2024-04-25T10:18:57.532479+0000","last_clean":"2024-04-25T10:18:57.532479+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:57.532479+0000","last_undegraded":"2024-04-25T10:18:57.532479+0000","last_fullsized":"2024-04-25T10:18:57.532479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":742422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":152248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":145206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":741910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":146806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:01.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:19:01.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 Processor -- start 2024-04-25T10:19:01.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 -- start start 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0678066fe0 0x7f067815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f06780b5560 0x7f0678157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780b5970 con 0x7f0678066c00 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780b5c20 con 0x7f0678066fe0 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f067ca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780af6e0 con 0x7f06780b5560 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f066fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0678066fe0 0x7f067815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f0675e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0678066c00 0x7f067806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44490/0 (socket says 172.21.15.45:44490) 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f0675e00640 1 -- 172.21.15.45:0/2610196061 learned_addr learned my addr 172.21.15.45:0/2610196061 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f0676800640 1 --2- 172.21.15.45:0/2610196061 >> v2:172.21.15.117:3300/0 conn(0x7f06780b5560 0x7f0678157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f066fe00640 1 -- 172.21.15.45:0/2610196061 >> v2:172.21.15.117:3300/0 conn(0x7f06780b5560 msgr2=0x7f0678157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f066fe00640 1 --2- 172.21.15.45:0/2610196061 >> v2:172.21.15.117:3300/0 conn(0x7f06780b5560 0x7f0678157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f066fe00640 1 -- 172.21.15.45:0/2610196061 >> v1:172.21.15.45:6789/0 conn(0x7f0678066c00 legacy=0x7f067806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.300+0000 7f066fe00640 1 -- 172.21.15.45:0/2610196061 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780af9c0 con 0x7f0678066fe0 2024-04-25T10:19:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f066fe00640 1 --2- 172.21.15.45:0/2610196061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0678066fe0 0x7f067815e110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06780b3360 tx=0x7f066001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd1c8f5787d2b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f0675400640 1 -- 172.21.15.45:0/2610196061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06600ab650 con 0x7f0678066fe0 2024-04-25T10:19:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f0675400640 1 -- 172.21.15.45:0/2610196061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06600b0070 con 0x7f0678066fe0 2024-04-25T10:19:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f0675400640 1 -- 172.21.15.45:0/2610196061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06600aba10 con 0x7f0678066fe0 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 -- 172.21.15.45:0/2610196061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0678066fe0 msgr2=0x7f067815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 --2- 172.21.15.45:0/2610196061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0678066fe0 0x7f067815e110 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06780b3360 tx=0x7f066001c450 comp rx=0 tx=0).stop 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 -- 172.21.15.45:0/2610196061 shutdown_connections 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 --2- 172.21.15.45:0/2610196061 >> v2:172.21.15.117:3300/0 conn(0x7f06780b5560 0x7f0678157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 --2- 172.21.15.45:0/2610196061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0678066fe0 0x7f067815e110 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 -- 172.21.15.45:0/2610196061 >> 172.21.15.45:0/2610196061 conn(0x7f06780b24f0 msgr2=0x7f06780b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 -- 172.21.15.45:0/2610196061 shutdown_connections 2024-04-25T10:19:01.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.304+0000 7f067ca00640 1 -- 172.21.15.45:0/2610196061 wait complete. 2024-04-25T10:19:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 Processor -- start 2024-04-25T10:19:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 -- start start 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0678066c00 0x7f0678150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0678066fe0 0x7f0678157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 0x7f067813a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06780af1e0 con 0x7f06780b5560 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06780af060 con 0x7f0678066fe0 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067806ac90 con 0x7f0678066c00 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 0x7f067813a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 0x7f067813a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60420/0 (socket says 172.21.15.45:60420) 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 -- 172.21.15.45:0/183904270 learned_addr learned my addr 172.21.15.45:0/183904270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f066fe00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0678066fe0 0x7f0678157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0675e00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0678066c00 0x7f0678150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 -- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0678066c00 msgr2=0x7f0678150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0678066c00 0x7f0678150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 -- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0678066fe0 msgr2=0x7f0678157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0678066fe0 0x7f0678157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 -- 172.21.15.45:0/183904270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06781486a0 con 0x7f06780b5560 2024-04-25T10:19:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f0676800640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 0x7f067813a2c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0670007e60 tx=0x7f067000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670014420 con 0x7f06780b5560 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0670019070 con 0x7f06780b5560 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670014820 con 0x7f06780b5560 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0678148980 con 0x7f06780b5560 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.308+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0678148f40 con 0x7f06780b5560 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.312+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067804ef30 con 0x7f06780b5560 2024-04-25T10:19:01.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.312+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06700149c0 con 0x7f06780b5560 2024-04-25T10:19:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.312+0000 7f066e000640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0654066f60 0x7f0654069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.312+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0670085810 con 0x7f06780b5560 2024-04-25T10:19:01.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.312+0000 7f0675e00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0654066f60 0x7f0654069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.312+0000 7f0675e00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0654066f60 0x7f0654069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0668000f70 tx=0x7f0668007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.316+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0670056130 con 0x7f06780b5560 2024-04-25T10:19:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0678071c70 con 0x7f06780b5560 2024-04-25T10:19:01.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f066e000640 1 -- 172.21.15.45:0/183904270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1560 (secure 0 0 0) 0x7f067005a1d0 con 0x7f06780b5560 2024-04-25T10:19:01.539 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:01.539 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":85,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":247013376,"bytes_avail":772847099904,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5055}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-25T10:18:50.885349+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:19:01.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0654066f60 msgr2=0x7f0654069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0654066f60 0x7f0654069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0668000f70 tx=0x7f0668007e90 comp rx=0 tx=0).stop 2024-04-25T10:19:01.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 msgr2=0x7f067813a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 0x7f067813a2c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0670007e60 tx=0x7f067000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 shutdown_connections 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0654066f60 0x7f0654069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06780b5560 0x7f067813a2c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0678066fe0 0x7f0678157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 --2- 172.21.15.45:0/183904270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0678066c00 0x7f0678150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 >> 172.21.15.45:0/183904270 conn(0x7f06780b24f0 msgr2=0x7f067806b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.536+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 shutdown_connections 2024-04-25T10:19:01.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.540+0000 7f067ca00640 1 -- 172.21.15.45:0/183904270 wait complete. 2024-04-25T10:19:01.561 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:19:01.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:01.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf75000640 1 Processor -- start 2024-04-25T10:19:01.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf75000640 1 -- start start 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf75000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 0x7fbf700b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf75000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf700b1870 con 0x7fbf700d1820 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf75000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf700b1af0 con 0x7fbf700b13e0 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf75000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700b1dd0 con 0x7fbf7015e050 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6de00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf700d1820 0x7fbf700d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44500/0 (socket says 172.21.15.45:44500) 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6de00640 1 -- 172.21.15.45:0/2854424260 learned_addr learned my addr 172.21.15.45:0/2854424260 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436009322 0 0) 0x7fbf60001120 con 0x7fbf700d1820 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf54003530 con 0x7fbf700d1820 2024-04-25T10:19:01.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 0x7fbf700b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2282674305 0 0) 0x7fbf54003530 con 0x7fbf700d1820 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 msgr2=0x7fbf700b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 0x7fbf700b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 >> v1:172.21.15.117:6789/0 conn(0x7fbf700b13e0 legacy=0x7fbf700cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700b20b0 con 0x7fbf700d1820 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fbf60001850 con 0x7fbf700d1820 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 >> v1:172.21.15.45:6789/0 conn(0x7fbf700d1820 legacy=0x7fbf700d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 0x7fbf54005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700b1dd0 con 0x7fbf540039f0 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5400bb60 con 0x7fbf54004810 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5400bd60 con 0x7fbf7015e050 2024-04-25T10:19:01.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6de00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6e800640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 0x7fbf54005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 -- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 msgr2=0x7fbf54005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7015e050 0x7fbf54005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 -- 172.21.15.45:0/2854424260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf540039f0 msgr2=0x7fbf54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf67e00640 1 -- 172.21.15.45:0/2854424260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf54003530 con 0x7fbf54004810 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.700+0000 7fbf6de00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:01.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf67e00640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009ae0 tx=0x7fbf5801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20df4fb7c658968f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580b2070 con 0x7fbf54004810 2024-04-25T10:19:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf580a5510 con 0x7fbf54004810 2024-04-25T10:19:01.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf6d400640 1 -- 172.21.15.45:0/2854424260 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580096b0 con 0x7fbf54004810 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf75000640 1 -- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 msgr2=0x7fbf54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf75000640 1 --2- 172.21.15.45:0/2854424260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009ae0 tx=0x7fbf5801ca30 comp rx=0 tx=0).stop 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf6e800640 1 -- 172.21.15.45:0/2854424260 reap_dead start 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf75000640 1 -- 172.21.15.45:0/2854424260 shutdown_connections 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf75000640 1 -- 172.21.15.45:0/2854424260 >> 172.21.15.45:0/2854424260 conn(0x7fbf7009a9d0 msgr2=0x7fbf7009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf75000640 1 -- 172.21.15.45:0/2854424260 shutdown_connections 2024-04-25T10:19:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.704+0000 7fbf75000640 1 -- 172.21.15.45:0/2854424260 wait complete. 2024-04-25T10:19:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 Processor -- start 2024-04-25T10:19:01.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 -- start start 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf700b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7014e480 0x7fbf700c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 0x7fbf70166e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700d0fa0 con 0x7fbf70166980 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700d0e20 con 0x7fbf54004810 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf701699a0 con 0x7fbf7014e480 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 0x7fbf70166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 0x7fbf70166e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60436/0 (socket says 172.21.15.45:60436) 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 -- 172.21.15.45:0/3513708469 learned_addr learned my addr 172.21.15.45:0/3513708469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6de00640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf700b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf67e00640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7014e480 0x7fbf700c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 -- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7014e480 msgr2=0x7fbf700c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7014e480 0x7fbf700c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 -- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 msgr2=0x7fbf700b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf700b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 -- 172.21.15.45:0/3513708469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7014e900 con 0x7fbf70166980 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf6e800640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 0x7fbf70166e00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000f90 tx=0x7fbf6800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6801a070 con 0x7fbf70166980 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68006d70 con 0x7fbf70166980 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60007e10 con 0x7fbf70166980 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf70168740 con 0x7fbf70166980 2024-04-25T10:19:01.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.708+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf700c8b50 con 0x7fbf70166980 2024-04-25T10:19:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.712+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7004ef30 con 0x7fbf70166980 2024-04-25T10:19:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.712+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf68003ba0 con 0x7fbf70166980 2024-04-25T10:19:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.712+0000 7fbf66000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf4c066f60 0x7fbf4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.712+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf680843d0 con 0x7fbf70166980 2024-04-25T10:19:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.712+0000 7fbf6de00640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf4c066f60 0x7fbf4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.712+0000 7fbf6de00640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf4c066f60 0x7fbf4c069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf60001300 tx=0x7fbf600079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.716+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf68054d80 con 0x7fbf70166980 2024-04-25T10:19:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.876+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf7006b480 con 0x7fbf4c066f60 2024-04-25T10:19:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.880+0000 7fbf66000640 1 -- 172.21.15.45:0/3513708469 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251940 (secure 0 0 0) 0x7fbf7006b480 con 0x7fbf4c066f60 2024-04-25T10:19:01.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:19:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf4c066f60 msgr2=0x7fbf4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf4c066f60 0x7fbf4c069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf60001300 tx=0x7fbf600079b0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 msgr2=0x7fbf70166e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 0x7fbf70166e00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000f90 tx=0x7fbf6800fc90 comp rx=0 tx=0).stop 2024-04-25T10:19:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 shutdown_connections 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf4c066f60 0x7fbf4c069420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf70166980 0x7fbf70166e00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf7014e480 0x7fbf700c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 --2- 172.21.15.45:0/3513708469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf54004810 0x7fbf700b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 >> 172.21.15.45:0/3513708469 conn(0x7fbf7009a9d0 msgr2=0x7fbf700d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 shutdown_connections 2024-04-25T10:19:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:01.884+0000 7fbf75000640 1 -- 172.21.15.45:0/3513708469 wait complete. 2024-04-25T10:19:01.908 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-25T10:19:00.887417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241224,"kb_used_data":4240,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733496,"statfs":{"total":773094113280,"available":772847099904,"internally_reserved":0,"allocated":4341760,"data_stored":2384732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001448"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098495+0000","last_change":"2024-04-25T10:18:56.253253+0000","last_active":"2024-04-25T10:18:58.098495+0000","last_peered":"2024-04-25T10:18:58.098495+0000","last_clean":"2024-04-25T10:18:58.098495+0000","last_became_active":"2024-04-25T10:18:56.252862+0000","last_became_peered":"2024-04-25T10:18:56.252862+0000","last_unstale":"2024-04-25T10:18:58.098495+0000","last_undegraded":"2024-04-25T10:18:58.098495+0000","last_fullsized":"2024-04-25T10:18:58.098495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:11.669773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237424+0000","last_change":"2024-04-25T10:18:56.249803+0000","last_active":"2024-04-25T10:18:57.237424+0000","last_peered":"2024-04-25T10:18:57.237424+0000","last_clean":"2024-04-25T10:18:57.237424+0000","last_became_active":"2024-04-25T10:18:56.249526+0000","last_became_peered":"2024-04-25T10:18:56.249526+0000","last_unstale":"2024-04-25T10:18:57.237424+0000","last_undegraded":"2024-04-25T10:18:57.237424+0000","last_fullsized":"2024-04-25T10:18:57.237424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532682+0000","last_change":"2024-04-25T10:18:56.251908+0000","last_active":"2024-04-25T10:18:57.532682+0000","last_peered":"2024-04-25T10:18:57.532682+0000","last_clean":"2024-04-25T10:18:57.532682+0000","last_became_active":"2024-04-25T10:18:56.251585+0000","last_became_peered":"2024-04-25T10:18:56.251585+0000","last_unstale":"2024-04-25T10:18:57.532682+0000","last_undegraded":"2024-04-25T10:18:57.532682+0000","last_fullsized":"2024-04-25T10:18:57.532682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:37.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260210+0000","last_change":"2024-04-25T10:18:56.258089+0000","last_active":"2024-04-25T10:18:56.260210+0000","last_peered":"2024-04-25T10:18:56.260210+0000","last_clean":"2024-04-25T10:18:56.260210+0000","last_became_active":"2024-04-25T10:18:56.257834+0000","last_became_peered":"2024-04-25T10:18:56.257834+0000","last_unstale":"2024-04-25T10:18:56.260210+0000","last_undegraded":"2024-04-25T10:18:56.260210+0000","last_fullsized":"2024-04-25T10:18:56.260210+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:48.224888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237046+0000","last_change":"2024-04-25T10:18:56.234170+0000","last_active":"2024-04-25T10:18:57.237046+0000","last_peered":"2024-04-25T10:18:57.237046+0000","last_clean":"2024-04-25T10:18:57.237046+0000","last_became_active":"2024-04-25T10:18:56.233918+0000","last_became_peered":"2024-04-25T10:18:56.233918+0000","last_unstale":"2024-04-25T10:18:57.237046+0000","last_undegraded":"2024-04-25T10:18:57.237046+0000","last_fullsized":"2024-04-25T10:18:57.237046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238362+0000","last_change":"2024-04-25T10:18:56.254897+0000","last_active":"2024-04-25T10:18:57.238362+0000","last_peered":"2024-04-25T10:18:57.238362+0000","last_clean":"2024-04-25T10:18:57.238362+0000","last_became_active":"2024-04-25T10:18:56.254641+0000","last_became_peered":"2024-04-25T10:18:56.254641+0000","last_unstale":"2024-04-25T10:18:57.238362+0000","last_undegraded":"2024-04-25T10:18:57.238362+0000","last_fullsized":"2024-04-25T10:18:57.238362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:37.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098293+0000","last_change":"2024-04-25T10:18:56.239390+0000","last_active":"2024-04-25T10:18:58.098293+0000","last_peered":"2024-04-25T10:18:58.098293+0000","last_clean":"2024-04-25T10:18:58.098293+0000","last_became_active":"2024-04-25T10:18:56.239075+0000","last_became_peered":"2024-04-25T10:18:56.239075+0000","last_unstale":"2024-04-25T10:18:58.098293+0000","last_undegraded":"2024-04-25T10:18:58.098293+0000","last_fullsized":"2024-04-25T10:18:58.098293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098140+0000","last_change":"2024-04-25T10:18:56.240095+0000","last_active":"2024-04-25T10:18:58.098140+0000","last_peered":"2024-04-25T10:18:58.098140+0000","last_clean":"2024-04-25T10:18:58.098140+0000","last_became_active":"2024-04-25T10:18:56.239416+0000","last_became_peered":"2024-04-25T10:18:56.239416+0000","last_unstale":"2024-04-25T10:18:58.098140+0000","last_undegraded":"2024-04-25T10:18:58.098140+0000","last_fullsized":"2024-04-25T10:18:58.098140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532815+0000","last_change":"2024-04-25T10:18:56.237743+0000","last_active":"2024-04-25T10:18:57.532815+0000","last_peered":"2024-04-25T10:18:57.532815+0000","last_clean":"2024-04-25T10:18:57.532815+0000","last_became_active":"2024-04-25T10:18:56.237529+0000","last_became_peered":"2024-04-25T10:18:56.237529+0000","last_unstale":"2024-04-25T10:18:57.532815+0000","last_undegraded":"2024-04-25T10:18:57.532815+0000","last_fullsized":"2024-04-25T10:18:57.532815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.140873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.253696+0000","last_change":"2024-04-25T10:18:56.250051+0000","last_active":"2024-04-25T10:18:56.253696+0000","last_peered":"2024-04-25T10:18:56.253696+0000","last_clean":"2024-04-25T10:18:56.253696+0000","last_became_active":"2024-04-25T10:18:56.247576+0000","last_became_peered":"2024-04-25T10:18:56.247576+0000","last_unstale":"2024-04-25T10:18:56.253696+0000","last_undegraded":"2024-04-25T10:18:56.253696+0000","last_fullsized":"2024-04-25T10:18:56.253696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:39.926018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236922+0000","last_change":"2024-04-25T10:18:56.254679+0000","last_active":"2024-04-25T10:18:57.236922+0000","last_peered":"2024-04-25T10:18:57.236922+0000","last_clean":"2024-04-25T10:18:57.236922+0000","last_became_active":"2024-04-25T10:18:56.254334+0000","last_became_peered":"2024-04-25T10:18:56.254334+0000","last_unstale":"2024-04-25T10:18:57.236922+0000","last_undegraded":"2024-04-25T10:18:57.236922+0000","last_fullsized":"2024-04-25T10:18:57.236922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:47.661457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532655+0000","last_change":"2024-04-25T10:18:56.234697+0000","last_active":"2024-04-25T10:18:57.532655+0000","last_peered":"2024-04-25T10:18:57.532655+0000","last_clean":"2024-04-25T10:18:57.532655+0000","last_became_active":"2024-04-25T10:18:56.234433+0000","last_became_peered":"2024-04-25T10:18:56.234433+0000","last_unstale":"2024-04-25T10:18:57.532655+0000","last_undegraded":"2024-04-25T10:18:57.532655+0000","last_fullsized":"2024-04-25T10:18:57.532655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:34.950803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814702+0000","last_change":"2024-04-25T10:18:56.243767+0000","last_active":"2024-04-25T10:18:57.814702+0000","last_peered":"2024-04-25T10:18:57.814702+0000","last_clean":"2024-04-25T10:18:57.814702+0000","last_became_active":"2024-04-25T10:18:56.243469+0000","last_became_peered":"2024-04-25T10:18:56.243469+0000","last_unstale":"2024-04-25T10:18:57.814702+0000","last_undegraded":"2024-04-25T10:18:57.814702+0000","last_fullsized":"2024-04-25T10:18:57.814702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250984+0000","last_change":"2024-04-25T10:18:56.247202+0000","last_active":"2024-04-25T10:18:56.250984+0000","last_peered":"2024-04-25T10:18:56.250984+0000","last_clean":"2024-04-25T10:18:56.250984+0000","last_became_active":"2024-04-25T10:18:56.240273+0000","last_became_peered":"2024-04-25T10:18:56.240273+0000","last_unstale":"2024-04-25T10:18:56.250984+0000","last_undegraded":"2024-04-25T10:18:56.250984+0000","last_fullsized":"2024-04-25T10:18:56.250984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:18.802983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098827+0000","last_change":"2024-04-25T10:18:56.243167+0000","last_active":"2024-04-25T10:18:58.098827+0000","last_peered":"2024-04-25T10:18:58.098827+0000","last_clean":"2024-04-25T10:18:58.098827+0000","last_became_active":"2024-04-25T10:18:56.242874+0000","last_became_peered":"2024-04-25T10:18:56.242874+0000","last_unstale":"2024-04-25T10:18:58.098827+0000","last_undegraded":"2024-04-25T10:18:58.098827+0000","last_fullsized":"2024-04-25T10:18:58.098827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:27.931466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098186+0000","last_change":"2024-04-25T10:18:56.254154+0000","last_active":"2024-04-25T10:18:58.098186+0000","last_peered":"2024-04-25T10:18:58.098186+0000","last_clean":"2024-04-25T10:18:58.098186+0000","last_became_active":"2024-04-25T10:18:56.251652+0000","last_became_peered":"2024-04-25T10:18:56.251652+0000","last_unstale":"2024-04-25T10:18:58.098186+0000","last_undegraded":"2024-04-25T10:18:58.098186+0000","last_fullsized":"2024-04-25T10:18:58.098186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.086557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236907+0000","last_change":"2024-04-25T10:18:56.261187+0000","last_active":"2024-04-25T10:18:57.236907+0000","last_peered":"2024-04-25T10:18:57.236907+0000","last_clean":"2024-04-25T10:18:57.236907+0000","last_became_active":"2024-04-25T10:18:56.259913+0000","last_became_peered":"2024-04-25T10:18:56.259913+0000","last_unstale":"2024-04-25T10:18:57.236907+0000","last_undegraded":"2024-04-25T10:18:57.236907+0000","last_fullsized":"2024-04-25T10:18:57.236907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:31.833160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238694+0000","last_change":"2024-04-25T10:18:56.254191+0000","last_active":"2024-04-25T10:18:57.238694+0000","last_peered":"2024-04-25T10:18:57.238694+0000","last_clean":"2024-04-25T10:18:57.238694+0000","last_became_active":"2024-04-25T10:18:56.253840+0000","last_became_peered":"2024-04-25T10:18:56.253840+0000","last_unstale":"2024-04-25T10:18:57.238694+0000","last_undegraded":"2024-04-25T10:18:57.238694+0000","last_fullsized":"2024-04-25T10:18:57.238694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:46.664788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238638+0000","last_change":"2024-04-25T10:18:56.237511+0000","last_active":"2024-04-25T10:18:57.238638+0000","last_peered":"2024-04-25T10:18:57.238638+0000","last_clean":"2024-04-25T10:18:57.238638+0000","last_became_active":"2024-04-25T10:18:56.237233+0000","last_became_peered":"2024-04-25T10:18:56.237233+0000","last_unstale":"2024-04-25T10:18:57.238638+0000","last_undegraded":"2024-04-25T10:18:57.238638+0000","last_fullsized":"2024-04-25T10:18:57.238638+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.450233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814870+0000","last_change":"2024-04-25T10:18:56.245428+0000","last_active":"2024-04-25T10:18:57.814870+0000","last_peered":"2024-04-25T10:18:57.814870+0000","last_clean":"2024-04-25T10:18:57.814870+0000","last_became_active":"2024-04-25T10:18:56.245143+0000","last_became_peered":"2024-04-25T10:18:56.245143+0000","last_unstale":"2024-04-25T10:18:57.814870+0000","last_undegraded":"2024-04-25T10:18:57.814870+0000","last_fullsized":"2024-04-25T10:18:57.814870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099156+0000","last_change":"2024-04-25T10:18:56.246924+0000","last_active":"2024-04-25T10:18:58.099156+0000","last_peered":"2024-04-25T10:18:58.099156+0000","last_clean":"2024-04-25T10:18:58.099156+0000","last_became_active":"2024-04-25T10:18:56.246327+0000","last_became_peered":"2024-04-25T10:18:56.246327+0000","last_unstale":"2024-04-25T10:18:58.099156+0000","last_undegraded":"2024-04-25T10:18:58.099156+0000","last_fullsized":"2024-04-25T10:18:58.099156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098486+0000","last_change":"2024-04-25T10:18:56.252008+0000","last_active":"2024-04-25T10:18:58.098486+0000","last_peered":"2024-04-25T10:18:58.098486+0000","last_clean":"2024-04-25T10:18:58.098486+0000","last_became_active":"2024-04-25T10:18:56.251724+0000","last_became_peered":"2024-04-25T10:18:56.251724+0000","last_unstale":"2024-04-25T10:18:58.098486+0000","last_undegraded":"2024-04-25T10:18:58.098486+0000","last_fullsized":"2024-04-25T10:18:58.098486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.306720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814498+0000","last_change":"2024-04-25T10:18:56.238540+0000","last_active":"2024-04-25T10:18:57.814498+0000","last_peered":"2024-04-25T10:18:57.814498+0000","last_clean":"2024-04-25T10:18:57.814498+0000","last_became_active":"2024-04-25T10:18:56.238216+0000","last_became_peered":"2024-04-25T10:18:56.238216+0000","last_unstale":"2024-04-25T10:18:57.814498+0000","last_undegraded":"2024-04-25T10:18:57.814498+0000","last_fullsized":"2024-04-25T10:18:57.814498+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:27.931466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098348+0000","last_change":"2024-04-25T10:18:56.240657+0000","last_active":"2024-04-25T10:18:58.098348+0000","last_peered":"2024-04-25T10:18:58.098348+0000","last_clean":"2024-04-25T10:18:58.098348+0000","last_became_active":"2024-04-25T10:18:56.240396+0000","last_became_peered":"2024-04-25T10:18:56.240396+0000","last_unstale":"2024-04-25T10:18:58.098348+0000","last_undegraded":"2024-04-25T10:18:58.098348+0000","last_fullsized":"2024-04-25T10:18:58.098348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252157+0000","last_change":"2024-04-25T10:18:56.247980+0000","last_active":"2024-04-25T10:18:56.252157+0000","last_peered":"2024-04-25T10:18:56.252157+0000","last_clean":"2024-04-25T10:18:56.252157+0000","last_became_active":"2024-04-25T10:18:56.247334+0000","last_became_peered":"2024-04-25T10:18:56.247334+0000","last_unstale":"2024-04-25T10:18:56.252157+0000","last_undegraded":"2024-04-25T10:18:56.252157+0000","last_fullsized":"2024-04-25T10:18:56.252157+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:51.612725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252020+0000","last_change":"2024-04-25T10:18:56.247318+0000","last_active":"2024-04-25T10:18:56.252020+0000","last_peered":"2024-04-25T10:18:56.252020+0000","last_clean":"2024-04-25T10:18:56.252020+0000","last_became_active":"2024-04-25T10:18:56.246972+0000","last_became_peered":"2024-04-25T10:18:56.246972+0000","last_unstale":"2024-04-25T10:18:56.252020+0000","last_undegraded":"2024-04-25T10:18:56.252020+0000","last_fullsized":"2024-04-25T10:18:56.252020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:04.302027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237571+0000","last_change":"2024-04-25T10:18:56.239544+0000","last_active":"2024-04-25T10:18:57.237571+0000","last_peered":"2024-04-25T10:18:57.237571+0000","last_clean":"2024-04-25T10:18:57.237571+0000","last_became_active":"2024-04-25T10:18:56.236085+0000","last_became_peered":"2024-04-25T10:18:56.236085+0000","last_unstale":"2024-04-25T10:18:57.237571+0000","last_undegraded":"2024-04-25T10:18:57.237571+0000","last_fullsized":"2024-04-25T10:18:57.237571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:25.904643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098644+0000","last_change":"2024-04-25T10:18:56.257431+0000","last_active":"2024-04-25T10:18:58.098644+0000","last_peered":"2024-04-25T10:18:58.098644+0000","last_clean":"2024-04-25T10:18:58.098644+0000","last_became_active":"2024-04-25T10:18:56.257033+0000","last_became_peered":"2024-04-25T10:18:56.257033+0000","last_unstale":"2024-04-25T10:18:58.098644+0000","last_undegraded":"2024-04-25T10:18:58.098644+0000","last_fullsized":"2024-04-25T10:18:58.098644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:30.757565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099042+0000","last_change":"2024-04-25T10:18:56.247122+0000","last_active":"2024-04-25T10:18:58.099042+0000","last_peered":"2024-04-25T10:18:58.099042+0000","last_clean":"2024-04-25T10:18:58.099042+0000","last_became_active":"2024-04-25T10:18:56.246671+0000","last_became_peered":"2024-04-25T10:18:56.246671+0000","last_unstale":"2024-04-25T10:18:58.099042+0000","last_undegraded":"2024-04-25T10:18:58.099042+0000","last_fullsized":"2024-04-25T10:18:58.099042+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099002+0000","last_change":"2024-04-25T10:18:56.244715+0000","last_active":"2024-04-25T10:18:58.099002+0000","last_peered":"2024-04-25T10:18:58.099002+0000","last_clean":"2024-04-25T10:18:58.099002+0000","last_became_active":"2024-04-25T10:18:56.244456+0000","last_became_peered":"2024-04-25T10:18:56.244456+0000","last_unstale":"2024-04-25T10:18:58.099002+0000","last_undegraded":"2024-04-25T10:18:58.099002+0000","last_fullsized":"2024-04-25T10:18:58.099002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:26.300653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098514+0000","last_change":"2024-04-25T10:18:56.243348+0000","last_active":"2024-04-25T10:18:58.098514+0000","last_peered":"2024-04-25T10:18:58.098514+0000","last_clean":"2024-04-25T10:18:58.098514+0000","last_became_active":"2024-04-25T10:18:56.243050+0000","last_became_peered":"2024-04-25T10:18:56.243050+0000","last_unstale":"2024-04-25T10:18:58.098514+0000","last_undegraded":"2024-04-25T10:18:58.098514+0000","last_fullsized":"2024-04-25T10:18:58.098514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:05.345870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237064+0000","last_change":"2024-04-25T10:18:56.261132+0000","last_active":"2024-04-25T10:18:57.237064+0000","last_peered":"2024-04-25T10:18:57.237064+0000","last_clean":"2024-04-25T10:18:57.237064+0000","last_became_active":"2024-04-25T10:18:56.259485+0000","last_became_peered":"2024-04-25T10:18:56.259485+0000","last_unstale":"2024-04-25T10:18:57.237064+0000","last_undegraded":"2024-04-25T10:18:57.237064+0000","last_fullsized":"2024-04-25T10:18:57.237064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:04.765169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.815019+0000","last_change":"2024-04-25T10:18:56.245528+0000","last_active":"2024-04-25T10:18:57.815019+0000","last_peered":"2024-04-25T10:18:57.815019+0000","last_clean":"2024-04-25T10:18:57.815019+0000","last_became_active":"2024-04-25T10:18:56.245224+0000","last_became_peered":"2024-04-25T10:18:56.245224+0000","last_unstale":"2024-04-25T10:18:57.815019+0000","last_undegraded":"2024-04-25T10:18:57.815019+0000","last_fullsized":"2024-04-25T10:18:57.815019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236935+0000","last_change":"2024-04-25T10:18:55.228870+0000","last_active":"2024-04-25T10:18:57.236935+0000","last_peered":"2024-04-25T10:18:57.236935+0000","last_clean":"2024-04-25T10:18:57.236935+0000","last_became_active":"2024-04-25T10:18:55.228570+0000","last_became_peered":"2024-04-25T10:18:55.228570+0000","last_unstale":"2024-04-25T10:18:57.236935+0000","last_undegraded":"2024-04-25T10:18:57.236935+0000","last_fullsized":"2024-04-25T10:18:57.236935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.897606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098657+0000","last_change":"2024-04-25T10:18:56.251166+0000","last_active":"2024-04-25T10:18:58.098657+0000","last_peered":"2024-04-25T10:18:58.098657+0000","last_clean":"2024-04-25T10:18:58.098657+0000","last_became_active":"2024-04-25T10:18:56.250808+0000","last_became_peered":"2024-04-25T10:18:56.250808+0000","last_unstale":"2024-04-25T10:18:58.098657+0000","last_undegraded":"2024-04-25T10:18:58.098657+0000","last_fullsized":"2024-04-25T10:18:58.098657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:06.007308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252501+0000","last_change":"2024-04-25T10:18:55.232915+0000","last_active":"2024-04-25T10:18:56.252501+0000","last_peered":"2024-04-25T10:18:56.252501+0000","last_clean":"2024-04-25T10:18:56.252501+0000","last_became_active":"2024-04-25T10:18:55.232635+0000","last_became_peered":"2024-04-25T10:18:55.232635+0000","last_unstale":"2024-04-25T10:18:56.252501+0000","last_undegraded":"2024-04-25T10:18:56.252501+0000","last_fullsized":"2024-04-25T10:18:56.252501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:16.184267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098660+0000","last_change":"2024-04-25T10:18:56.259417+0000","last_active":"2024-04-25T10:18:58.098660+0000","last_peered":"2024-04-25T10:18:58.098660+0000","last_clean":"2024-04-25T10:18:58.098660+0000","last_became_active":"2024-04-25T10:18:56.256746+0000","last_became_peered":"2024-04-25T10:18:56.256746+0000","last_unstale":"2024-04-25T10:18:58.098660+0000","last_undegraded":"2024-04-25T10:18:58.098660+0000","last_fullsized":"2024-04-25T10:18:58.098660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.560656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238766+0000","last_change":"2024-04-25T10:18:55.241402+0000","last_active":"2024-04-25T10:18:57.238766+0000","last_peered":"2024-04-25T10:18:57.238766+0000","last_clean":"2024-04-25T10:18:57.238766+0000","last_became_active":"2024-04-25T10:18:55.241057+0000","last_became_peered":"2024-04-25T10:18:55.241057+0000","last_unstale":"2024-04-25T10:18:57.238766+0000","last_undegraded":"2024-04-25T10:18:57.238766+0000","last_fullsized":"2024-04-25T10:18:57.238766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:48.540119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250485+0000","last_change":"2024-04-25T10:18:56.238845+0000","last_active":"2024-04-25T10:18:56.250485+0000","last_peered":"2024-04-25T10:18:56.250485+0000","last_clean":"2024-04-25T10:18:56.250485+0000","last_became_active":"2024-04-25T10:18:56.233835+0000","last_became_peered":"2024-04-25T10:18:56.233835+0000","last_unstale":"2024-04-25T10:18:56.250485+0000","last_undegraded":"2024-04-25T10:18:56.250485+0000","last_fullsized":"2024-04-25T10:18:56.250485+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:35.710530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.259903+0000","last_change":"2024-04-25T10:18:55.239850+0000","last_active":"2024-04-25T10:18:56.259903+0000","last_peered":"2024-04-25T10:18:56.259903+0000","last_clean":"2024-04-25T10:18:56.259903+0000","last_became_active":"2024-04-25T10:18:55.236286+0000","last_became_peered":"2024-04-25T10:18:55.236286+0000","last_unstale":"2024-04-25T10:18:56.259903+0000","last_undegraded":"2024-04-25T10:18:56.259903+0000","last_fullsized":"2024-04-25T10:18:56.259903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.659174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532529+0000","last_change":"2024-04-25T10:18:56.229987+0000","last_active":"2024-04-25T10:18:57.532529+0000","last_peered":"2024-04-25T10:18:57.532529+0000","last_clean":"2024-04-25T10:18:57.532529+0000","last_became_active":"2024-04-25T10:18:56.229484+0000","last_became_peered":"2024-04-25T10:18:56.229484+0000","last_unstale":"2024-04-25T10:18:57.532529+0000","last_undegraded":"2024-04-25T10:18:57.532529+0000","last_fullsized":"2024-04-25T10:18:57.532529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:20.518835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098686+0000","last_change":"2024-04-25T10:18:55.233523+0000","last_active":"2024-04-25T10:18:58.098686+0000","last_peered":"2024-04-25T10:18:58.098686+0000","last_clean":"2024-04-25T10:18:58.098686+0000","last_became_active":"2024-04-25T10:18:55.233010+0000","last_became_peered":"2024-04-25T10:18:55.233010+0000","last_unstale":"2024-04-25T10:18:58.098686+0000","last_undegraded":"2024-04-25T10:18:58.098686+0000","last_fullsized":"2024-04-25T10:18:58.098686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:21.018773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098430+0000","last_change":"2024-04-25T10:18:56.244147+0000","last_active":"2024-04-25T10:18:58.098430+0000","last_peered":"2024-04-25T10:18:58.098430+0000","last_clean":"2024-04-25T10:18:58.098430+0000","last_became_active":"2024-04-25T10:18:56.243866+0000","last_became_peered":"2024-04-25T10:18:56.243866+0000","last_unstale":"2024-04-25T10:18:58.098430+0000","last_undegraded":"2024-04-25T10:18:58.098430+0000","last_fullsized":"2024-04-25T10:18:58.098430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:25.904643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098678+0000","last_change":"2024-04-25T10:18:55.233989+0000","last_active":"2024-04-25T10:18:58.098678+0000","last_peered":"2024-04-25T10:18:58.098678+0000","last_clean":"2024-04-25T10:18:58.098678+0000","last_became_active":"2024-04-25T10:18:55.233717+0000","last_became_peered":"2024-04-25T10:18:55.233717+0000","last_unstale":"2024-04-25T10:18:58.098678+0000","last_undegraded":"2024-04-25T10:18:58.098678+0000","last_fullsized":"2024-04-25T10:18:58.098678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:02.604590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532416+0000","last_change":"2024-04-25T10:18:56.235738+0000","last_active":"2024-04-25T10:18:57.532416+0000","last_peered":"2024-04-25T10:18:57.532416+0000","last_clean":"2024-04-25T10:18:57.532416+0000","last_became_active":"2024-04-25T10:18:56.235450+0000","last_became_peered":"2024-04-25T10:18:56.235450+0000","last_unstale":"2024-04-25T10:18:57.532416+0000","last_undegraded":"2024-04-25T10:18:57.532416+0000","last_fullsized":"2024-04-25T10:18:57.532416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:03.610849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238301+0000","last_change":"2024-04-25T10:18:55.244101+0000","last_active":"2024-04-25T10:18:57.238301+0000","last_peered":"2024-04-25T10:18:57.238301+0000","last_clean":"2024-04-25T10:18:57.238301+0000","last_became_active":"2024-04-25T10:18:55.243832+0000","last_became_peered":"2024-04-25T10:18:55.243832+0000","last_unstale":"2024-04-25T10:18:57.238301+0000","last_undegraded":"2024-04-25T10:18:57.238301+0000","last_fullsized":"2024-04-25T10:18:57.238301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:19.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250985+0000","last_change":"2024-04-25T10:18:56.245450+0000","last_active":"2024-04-25T10:18:56.250985+0000","last_peered":"2024-04-25T10:18:56.250985+0000","last_clean":"2024-04-25T10:18:56.250985+0000","last_became_active":"2024-04-25T10:18:56.245230+0000","last_became_peered":"2024-04-25T10:18:56.245230+0000","last_unstale":"2024-04-25T10:18:56.250985+0000","last_undegraded":"2024-04-25T10:18:56.250985+0000","last_fullsized":"2024-04-25T10:18:56.250985+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:02.976563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098626+0000","last_change":"2024-04-25T10:18:55.241837+0000","last_active":"2024-04-25T10:18:58.098626+0000","last_peered":"2024-04-25T10:18:58.098626+0000","last_clean":"2024-04-25T10:18:58.098626+0000","last_became_active":"2024-04-25T10:18:55.241552+0000","last_became_peered":"2024-04-25T10:18:55.241552+0000","last_unstale":"2024-04-25T10:18:58.098626+0000","last_undegraded":"2024-04-25T10:18:58.098626+0000","last_fullsized":"2024-04-25T10:18:58.098626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:37.997874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236783+0000","last_change":"2024-04-25T10:18:56.245632+0000","last_active":"2024-04-25T10:18:57.236783+0000","last_peered":"2024-04-25T10:18:57.236783+0000","last_clean":"2024-04-25T10:18:57.236783+0000","last_became_active":"2024-04-25T10:18:56.244499+0000","last_became_peered":"2024-04-25T10:18:56.244499+0000","last_unstale":"2024-04-25T10:18:57.236783+0000","last_undegraded":"2024-04-25T10:18:57.236783+0000","last_fullsized":"2024-04-25T10:18:57.236783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:05.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237522+0000","last_change":"2024-04-25T10:18:55.237962+0000","last_active":"2024-04-25T10:18:57.237522+0000","last_peered":"2024-04-25T10:18:57.237522+0000","last_clean":"2024-04-25T10:18:57.237522+0000","last_became_active":"2024-04-25T10:18:55.234540+0000","last_became_peered":"2024-04-25T10:18:55.234540+0000","last_unstale":"2024-04-25T10:18:57.237522+0000","last_undegraded":"2024-04-25T10:18:57.237522+0000","last_fullsized":"2024-04-25T10:18:57.237522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.443974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.252183+0000","last_change":"2024-04-25T10:18:56.248260+0000","last_active":"2024-04-25T10:18:56.252183+0000","last_peered":"2024-04-25T10:18:56.252183+0000","last_clean":"2024-04-25T10:18:56.252183+0000","last_became_active":"2024-04-25T10:18:56.247668+0000","last_became_peered":"2024-04-25T10:18:56.247668+0000","last_unstale":"2024-04-25T10:18:56.252183+0000","last_undegraded":"2024-04-25T10:18:56.252183+0000","last_fullsized":"2024-04-25T10:18:56.252183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:26.759261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237444+0000","last_change":"2024-04-25T10:18:55.234932+0000","last_active":"2024-04-25T10:18:57.237444+0000","last_peered":"2024-04-25T10:18:57.237444+0000","last_clean":"2024-04-25T10:18:57.237444+0000","last_became_active":"2024-04-25T10:18:55.234587+0000","last_became_peered":"2024-04-25T10:18:55.234587+0000","last_unstale":"2024-04-25T10:18:57.237444+0000","last_undegraded":"2024-04-25T10:18:57.237444+0000","last_fullsized":"2024-04-25T10:18:57.237444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:04.346918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.260231+0000","last_change":"2024-04-25T10:18:56.258154+0000","last_active":"2024-04-25T10:18:56.260231+0000","last_peered":"2024-04-25T10:18:56.260231+0000","last_clean":"2024-04-25T10:18:56.260231+0000","last_became_active":"2024-04-25T10:18:56.257886+0000","last_became_peered":"2024-04-25T10:18:56.257886+0000","last_unstale":"2024-04-25T10:18:56.260231+0000","last_undegraded":"2024-04-25T10:18:56.260231+0000","last_fullsized":"2024-04-25T10:18:56.260231+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:41.213066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237047+0000","last_change":"2024-04-25T10:18:55.240359+0000","last_active":"2024-04-25T10:18:57.237047+0000","last_peered":"2024-04-25T10:18:57.237047+0000","last_clean":"2024-04-25T10:18:57.237047+0000","last_became_active":"2024-04-25T10:18:55.238481+0000","last_became_peered":"2024-04-25T10:18:55.238481+0000","last_unstale":"2024-04-25T10:18:57.237047+0000","last_undegraded":"2024-04-25T10:18:57.237047+0000","last_fullsized":"2024-04-25T10:18:57.237047+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:23.698128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236771+0000","last_change":"2024-04-25T10:18:56.254595+0000","last_active":"2024-04-25T10:18:57.236771+0000","last_peered":"2024-04-25T10:18:57.236771+0000","last_clean":"2024-04-25T10:18:57.236771+0000","last_became_active":"2024-04-25T10:18:56.253982+0000","last_became_peered":"2024-04-25T10:18:56.253982+0000","last_unstale":"2024-04-25T10:18:57.236771+0000","last_undegraded":"2024-04-25T10:18:57.236771+0000","last_fullsized":"2024-04-25T10:18:57.236771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:01.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814959+0000","last_change":"2024-04-25T10:18:55.242961+0000","last_active":"2024-04-25T10:18:57.814959+0000","last_peered":"2024-04-25T10:18:57.814959+0000","last_clean":"2024-04-25T10:18:57.814959+0000","last_became_active":"2024-04-25T10:18:55.242653+0000","last_became_peered":"2024-04-25T10:18:55.242653+0000","last_unstale":"2024-04-25T10:18:57.814959+0000","last_undegraded":"2024-04-25T10:18:57.814959+0000","last_fullsized":"2024-04-25T10:18:57.814959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:19.512576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814465+0000","last_change":"2024-04-25T10:18:56.235468+0000","last_active":"2024-04-25T10:18:57.814465+0000","last_peered":"2024-04-25T10:18:57.814465+0000","last_clean":"2024-04-25T10:18:57.814465+0000","last_became_active":"2024-04-25T10:18:56.235134+0000","last_became_peered":"2024-04-25T10:18:56.235134+0000","last_unstale":"2024-04-25T10:18:57.814465+0000","last_undegraded":"2024-04-25T10:18:57.814465+0000","last_fullsized":"2024-04-25T10:18:57.814465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:22.129702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098585+0000","last_change":"2024-04-25T10:18:55.241199+0000","last_active":"2024-04-25T10:18:58.098585+0000","last_peered":"2024-04-25T10:18:58.098585+0000","last_clean":"2024-04-25T10:18:58.098585+0000","last_became_active":"2024-04-25T10:18:55.240168+0000","last_became_peered":"2024-04-25T10:18:55.240168+0000","last_unstale":"2024-04-25T10:18:58.098585+0000","last_undegraded":"2024-04-25T10:18:58.098585+0000","last_fullsized":"2024-04-25T10:18:58.098585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:33.944544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.251009+0000","last_change":"2024-04-25T10:18:56.247101+0000","last_active":"2024-04-25T10:18:56.251009+0000","last_peered":"2024-04-25T10:18:56.251009+0000","last_clean":"2024-04-25T10:18:56.251009+0000","last_became_active":"2024-04-25T10:18:56.239778+0000","last_became_peered":"2024-04-25T10:18:56.239778+0000","last_unstale":"2024-04-25T10:18:56.251009+0000","last_undegraded":"2024-04-25T10:18:56.251009+0000","last_fullsized":"2024-04-25T10:18:56.251009+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.437960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.229069+0000","last_change":"2024-04-25T10:18:55.239619+0000","last_active":"2024-04-25T10:18:56.229069+0000","last_peered":"2024-04-25T10:18:56.229069+0000","last_clean":"2024-04-25T10:18:56.229069+0000","last_became_active":"2024-04-25T10:18:55.239342+0000","last_became_peered":"2024-04-25T10:18:55.239342+0000","last_unstale":"2024-04-25T10:18:56.229069+0000","last_undegraded":"2024-04-25T10:18:56.229069+0000","last_fullsized":"2024-04-25T10:18:56.229069+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:54.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098479+0000","last_change":"2024-04-25T10:18:56.239731+0000","last_active":"2024-04-25T10:18:58.098479+0000","last_peered":"2024-04-25T10:18:58.098479+0000","last_clean":"2024-04-25T10:18:58.098479+0000","last_became_active":"2024-04-25T10:18:56.239151+0000","last_became_peered":"2024-04-25T10:18:56.239151+0000","last_unstale":"2024-04-25T10:18:58.098479+0000","last_undegraded":"2024-04-25T10:18:58.098479+0000","last_fullsized":"2024-04-25T10:18:58.098479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:24.279467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.228693+0000","last_change":"2024-04-25T10:18:55.240382+0000","last_active":"2024-04-25T10:18:56.228693+0000","last_peered":"2024-04-25T10:18:56.228693+0000","last_clean":"2024-04-25T10:18:56.228693+0000","last_became_active":"2024-04-25T10:18:55.235717+0000","last_became_peered":"2024-04-25T10:18:55.235717+0000","last_unstale":"2024-04-25T10:18:56.228693+0000","last_undegraded":"2024-04-25T10:18:56.228693+0000","last_fullsized":"2024-04-25T10:18:56.228693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:57.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236616+0000","last_change":"2024-04-25T10:18:56.260154+0000","last_active":"2024-04-25T10:18:57.236616+0000","last_peered":"2024-04-25T10:18:57.236616+0000","last_clean":"2024-04-25T10:18:57.236616+0000","last_became_active":"2024-04-25T10:18:56.259124+0000","last_became_peered":"2024-04-25T10:18:56.259124+0000","last_unstale":"2024-04-25T10:18:57.236616+0000","last_undegraded":"2024-04-25T10:18:57.236616+0000","last_fullsized":"2024-04-25T10:18:57.236616+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:59.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237057+0000","last_change":"2024-04-25T10:18:55.241451+0000","last_active":"2024-04-25T10:18:57.237057+0000","last_peered":"2024-04-25T10:18:57.237057+0000","last_clean":"2024-04-25T10:18:57.237057+0000","last_became_active":"2024-04-25T10:18:55.241168+0000","last_became_peered":"2024-04-25T10:18:55.241168+0000","last_unstale":"2024-04-25T10:18:57.237057+0000","last_undegraded":"2024-04-25T10:18:57.237057+0000","last_fullsized":"2024-04-25T10:18:57.237057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:48.653430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236854+0000","last_change":"2024-04-25T10:18:56.253037+0000","last_active":"2024-04-25T10:18:57.236854+0000","last_peered":"2024-04-25T10:18:57.236854+0000","last_clean":"2024-04-25T10:18:57.236854+0000","last_became_active":"2024-04-25T10:18:56.252771+0000","last_became_peered":"2024-04-25T10:18:56.252771+0000","last_unstale":"2024-04-25T10:18:57.236854+0000","last_undegraded":"2024-04-25T10:18:57.236854+0000","last_fullsized":"2024-04-25T10:18:57.236854+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:07.919477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238867+0000","last_change":"2024-04-25T10:18:55.244388+0000","last_active":"2024-04-25T10:18:57.238867+0000","last_peered":"2024-04-25T10:18:57.238867+0000","last_clean":"2024-04-25T10:18:57.238867+0000","last_became_active":"2024-04-25T10:18:55.244013+0000","last_became_peered":"2024-04-25T10:18:55.244013+0000","last_unstale":"2024-04-25T10:18:57.238867+0000","last_undegraded":"2024-04-25T10:18:57.238867+0000","last_fullsized":"2024-04-25T10:18:57.238867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:06.417351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532826+0000","last_change":"2024-04-25T10:18:56.252562+0000","last_active":"2024-04-25T10:18:57.532826+0000","last_peered":"2024-04-25T10:18:57.532826+0000","last_clean":"2024-04-25T10:18:57.532826+0000","last_became_active":"2024-04-25T10:18:56.252305+0000","last_became_peered":"2024-04-25T10:18:56.252305+0000","last_unstale":"2024-04-25T10:18:57.532826+0000","last_undegraded":"2024-04-25T10:18:57.532826+0000","last_fullsized":"2024-04-25T10:18:57.532826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.450233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098217+0000","last_change":"2024-04-25T10:18:55.242568+0000","last_active":"2024-04-25T10:18:58.098217+0000","last_peered":"2024-04-25T10:18:58.098217+0000","last_clean":"2024-04-25T10:18:58.098217+0000","last_became_active":"2024-04-25T10:18:55.242249+0000","last_became_peered":"2024-04-25T10:18:55.242249+0000","last_unstale":"2024-04-25T10:18:58.098217+0000","last_undegraded":"2024-04-25T10:18:58.098217+0000","last_fullsized":"2024-04-25T10:18:58.098217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237336+0000","last_change":"2024-04-25T10:18:56.230062+0000","last_active":"2024-04-25T10:18:57.237336+0000","last_peered":"2024-04-25T10:18:57.237336+0000","last_clean":"2024-04-25T10:18:57.237336+0000","last_became_active":"2024-04-25T10:18:56.229753+0000","last_became_peered":"2024-04-25T10:18:56.229753+0000","last_unstale":"2024-04-25T10:18:57.237336+0000","last_undegraded":"2024-04-25T10:18:57.237336+0000","last_fullsized":"2024-04-25T10:18:57.237336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237044+0000","last_change":"2024-04-25T10:18:55.226724+0000","last_active":"2024-04-25T10:18:57.237044+0000","last_peered":"2024-04-25T10:18:57.237044+0000","last_clean":"2024-04-25T10:18:57.237044+0000","last_became_active":"2024-04-25T10:18:55.226402+0000","last_became_peered":"2024-04-25T10:18:55.226402+0000","last_unstale":"2024-04-25T10:18:57.237044+0000","last_undegraded":"2024-04-25T10:18:57.237044+0000","last_fullsized":"2024-04-25T10:18:57.237044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:50.978256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238622+0000","last_change":"2024-04-25T10:18:56.243829+0000","last_active":"2024-04-25T10:18:57.238622+0000","last_peered":"2024-04-25T10:18:57.238622+0000","last_clean":"2024-04-25T10:18:57.238622+0000","last_became_active":"2024-04-25T10:18:56.243579+0000","last_became_peered":"2024-04-25T10:18:56.243579+0000","last_unstale":"2024-04-25T10:18:57.238622+0000","last_undegraded":"2024-04-25T10:18:57.238622+0000","last_fullsized":"2024-04-25T10:18:57.238622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.140873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099156+0000","last_change":"2024-04-25T10:18:55.237736+0000","last_active":"2024-04-25T10:18:58.099156+0000","last_peered":"2024-04-25T10:18:58.099156+0000","last_clean":"2024-04-25T10:18:58.099156+0000","last_became_active":"2024-04-25T10:18:55.237487+0000","last_became_peered":"2024-04-25T10:18:55.237487+0000","last_unstale":"2024-04-25T10:18:58.099156+0000","last_undegraded":"2024-04-25T10:18:58.099156+0000","last_fullsized":"2024-04-25T10:18:58.099156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814574+0000","last_change":"2024-04-25T10:18:56.248910+0000","last_active":"2024-04-25T10:18:57.814574+0000","last_peered":"2024-04-25T10:18:57.814574+0000","last_clean":"2024-04-25T10:18:57.814574+0000","last_became_active":"2024-04-25T10:18:56.246574+0000","last_became_peered":"2024-04-25T10:18:56.246574+0000","last_unstale":"2024-04-25T10:18:57.814574+0000","last_undegraded":"2024-04-25T10:18:57.814574+0000","last_fullsized":"2024-04-25T10:18:57.814574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:26.300653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237128+0000","last_change":"2024-04-25T10:18:55.236067+0000","last_active":"2024-04-25T10:18:57.237128+0000","last_peered":"2024-04-25T10:18:57.237128+0000","last_clean":"2024-04-25T10:18:57.237128+0000","last_became_active":"2024-04-25T10:18:55.235652+0000","last_became_peered":"2024-04-25T10:18:55.235652+0000","last_unstale":"2024-04-25T10:18:57.237128+0000","last_undegraded":"2024-04-25T10:18:57.237128+0000","last_fullsized":"2024-04-25T10:18:57.237128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:45.658529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237089+0000","last_change":"2024-04-25T10:18:56.241808+0000","last_active":"2024-04-25T10:18:57.237089+0000","last_peered":"2024-04-25T10:18:57.237089+0000","last_clean":"2024-04-25T10:18:57.237089+0000","last_became_active":"2024-04-25T10:18:56.241407+0000","last_became_peered":"2024-04-25T10:18:56.241407+0000","last_unstale":"2024-04-25T10:18:57.237089+0000","last_undegraded":"2024-04-25T10:18:57.237089+0000","last_fullsized":"2024-04-25T10:18:57.237089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098721+0000","last_change":"2024-04-25T10:18:55.240208+0000","last_active":"2024-04-25T10:18:58.098721+0000","last_peered":"2024-04-25T10:18:58.098721+0000","last_clean":"2024-04-25T10:18:58.098721+0000","last_became_active":"2024-04-25T10:18:55.239948+0000","last_became_peered":"2024-04-25T10:18:55.239948+0000","last_unstale":"2024-04-25T10:18:58.098721+0000","last_undegraded":"2024-04-25T10:18:58.098721+0000","last_fullsized":"2024-04-25T10:18:58.098721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:19.512576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814473+0000","last_change":"2024-04-25T10:18:56.243705+0000","last_active":"2024-04-25T10:18:57.814473+0000","last_peered":"2024-04-25T10:18:57.814473+0000","last_clean":"2024-04-25T10:18:57.814473+0000","last_became_active":"2024-04-25T10:18:56.241679+0000","last_became_peered":"2024-04-25T10:18:56.241679+0000","last_unstale":"2024-04-25T10:18:57.814473+0000","last_undegraded":"2024-04-25T10:18:57.814473+0000","last_fullsized":"2024-04-25T10:18:57.814473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.086557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532490+0000","last_change":"2024-04-25T10:18:55.233741+0000","last_active":"2024-04-25T10:18:57.532490+0000","last_peered":"2024-04-25T10:18:57.532490+0000","last_clean":"2024-04-25T10:18:57.532490+0000","last_became_active":"2024-04-25T10:18:55.228199+0000","last_became_peered":"2024-04-25T10:18:55.228199+0000","last_unstale":"2024-04-25T10:18:57.532490+0000","last_undegraded":"2024-04-25T10:18:57.532490+0000","last_fullsized":"2024-04-25T10:18:57.532490+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.605460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532541+0000","last_change":"2024-04-25T10:18:56.242696+0000","last_active":"2024-04-25T10:18:57.532541+0000","last_peered":"2024-04-25T10:18:57.532541+0000","last_clean":"2024-04-25T10:18:57.532541+0000","last_became_active":"2024-04-25T10:18:56.242407+0000","last_became_peered":"2024-04-25T10:18:56.242407+0000","last_unstale":"2024-04-25T10:18:57.532541+0000","last_undegraded":"2024-04-25T10:18:57.532541+0000","last_fullsized":"2024-04-25T10:18:57.532541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.025032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236550+0000","last_change":"2024-04-25T10:18:55.231050+0000","last_active":"2024-04-25T10:18:57.236550+0000","last_peered":"2024-04-25T10:18:57.236550+0000","last_clean":"2024-04-25T10:18:57.236550+0000","last_became_active":"2024-04-25T10:18:55.230645+0000","last_became_peered":"2024-04-25T10:18:55.230645+0000","last_unstale":"2024-04-25T10:18:57.236550+0000","last_undegraded":"2024-04-25T10:18:57.236550+0000","last_fullsized":"2024-04-25T10:18:57.236550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:25.294394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238881+0000","last_change":"2024-04-25T10:18:56.242781+0000","last_active":"2024-04-25T10:18:57.238881+0000","last_peered":"2024-04-25T10:18:57.238881+0000","last_clean":"2024-04-25T10:18:57.238881+0000","last_became_active":"2024-04-25T10:18:56.242495+0000","last_became_peered":"2024-04-25T10:18:56.242495+0000","last_unstale":"2024-04-25T10:18:57.238881+0000","last_undegraded":"2024-04-25T10:18:57.238881+0000","last_fullsized":"2024-04-25T10:18:57.238881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:05.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814954+0000","last_change":"2024-04-25T10:18:55.238477+0000","last_active":"2024-04-25T10:18:57.814954+0000","last_peered":"2024-04-25T10:18:57.814954+0000","last_clean":"2024-04-25T10:18:57.814954+0000","last_became_active":"2024-04-25T10:18:55.238126+0000","last_became_peered":"2024-04-25T10:18:55.238126+0000","last_unstale":"2024-04-25T10:18:57.814954+0000","last_undegraded":"2024-04-25T10:18:57.814954+0000","last_fullsized":"2024-04-25T10:18:57.814954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:37.997874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236882+0000","last_change":"2024-04-25T10:18:56.249055+0000","last_active":"2024-04-25T10:18:57.236882+0000","last_peered":"2024-04-25T10:18:57.236882+0000","last_clean":"2024-04-25T10:18:57.236882+0000","last_became_active":"2024-04-25T10:18:56.248708+0000","last_became_peered":"2024-04-25T10:18:56.248708+0000","last_unstale":"2024-04-25T10:18:57.236882+0000","last_undegraded":"2024-04-25T10:18:57.236882+0000","last_fullsized":"2024-04-25T10:18:57.236882+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:26.665895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.226720+0000","last_change":"2024-04-25T10:18:55.236135+0000","last_active":"2024-04-25T10:18:58.226720+0000","last_peered":"2024-04-25T10:18:58.226720+0000","last_clean":"2024-04-25T10:18:58.226720+0000","last_became_active":"2024-04-25T10:18:55.235788+0000","last_became_peered":"2024-04-25T10:18:55.235788+0000","last_unstale":"2024-04-25T10:18:58.226720+0000","last_undegraded":"2024-04-25T10:18:58.226720+0000","last_fullsized":"2024-04-25T10:18:58.226720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:57.667019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238907+0000","last_change":"2024-04-25T10:18:56.251890+0000","last_active":"2024-04-25T10:18:57.238907+0000","last_peered":"2024-04-25T10:18:57.238907+0000","last_clean":"2024-04-25T10:18:57.238907+0000","last_became_active":"2024-04-25T10:18:56.251588+0000","last_became_peered":"2024-04-25T10:18:56.251588+0000","last_unstale":"2024-04-25T10:18:57.238907+0000","last_undegraded":"2024-04-25T10:18:57.238907+0000","last_fullsized":"2024-04-25T10:18:57.238907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:58.673279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.815004+0000","last_change":"2024-04-25T10:18:55.236899+0000","last_active":"2024-04-25T10:18:57.815004+0000","last_peered":"2024-04-25T10:18:57.815004+0000","last_clean":"2024-04-25T10:18:57.815004+0000","last_became_active":"2024-04-25T10:18:55.236580+0000","last_became_peered":"2024-04-25T10:18:55.236580+0000","last_unstale":"2024-04-25T10:18:57.815004+0000","last_undegraded":"2024-04-25T10:18:57.815004+0000","last_fullsized":"2024-04-25T10:18:57.815004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:21.018773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099216+0000","last_change":"2024-04-25T10:18:56.235360+0000","last_active":"2024-04-25T10:18:58.099216+0000","last_peered":"2024-04-25T10:18:58.099216+0000","last_clean":"2024-04-25T10:18:58.099216+0000","last_became_active":"2024-04-25T10:18:56.235089+0000","last_became_peered":"2024-04-25T10:18:56.235089+0000","last_unstale":"2024-04-25T10:18:58.099216+0000","last_undegraded":"2024-04-25T10:18:58.099216+0000","last_fullsized":"2024-04-25T10:18:58.099216+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814753+0000","last_change":"2024-04-25T10:18:55.238591+0000","last_active":"2024-04-25T10:18:57.814753+0000","last_peered":"2024-04-25T10:18:57.814753+0000","last_clean":"2024-04-25T10:18:57.814753+0000","last_became_active":"2024-04-25T10:18:55.238176+0000","last_became_peered":"2024-04-25T10:18:55.238176+0000","last_unstale":"2024-04-25T10:18:57.814753+0000","last_undegraded":"2024-04-25T10:18:57.814753+0000","last_fullsized":"2024-04-25T10:18:57.814753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:02.604590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.250487+0000","last_change":"2024-04-25T10:18:56.239706+0000","last_active":"2024-04-25T10:18:56.250487+0000","last_peered":"2024-04-25T10:18:56.250487+0000","last_clean":"2024-04-25T10:18:56.250487+0000","last_became_active":"2024-04-25T10:18:56.239021+0000","last_became_peered":"2024-04-25T10:18:56.239021+0000","last_unstale":"2024-04-25T10:18:56.250487+0000","last_undegraded":"2024-04-25T10:18:56.250487+0000","last_fullsized":"2024-04-25T10:18:56.250487+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:02.670157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814556+0000","last_change":"2024-04-25T10:18:55.242723+0000","last_active":"2024-04-25T10:18:57.814556+0000","last_peered":"2024-04-25T10:18:57.814556+0000","last_clean":"2024-04-25T10:18:57.814556+0000","last_became_active":"2024-04-25T10:18:55.242338+0000","last_became_peered":"2024-04-25T10:18:55.242338+0000","last_unstale":"2024-04-25T10:18:57.814556+0000","last_undegraded":"2024-04-25T10:18:57.814556+0000","last_fullsized":"2024-04-25T10:18:57.814556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:03.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.238269+0000","last_change":"2024-04-25T10:18:56.234182+0000","last_active":"2024-04-25T10:18:56.238269+0000","last_peered":"2024-04-25T10:18:56.238269+0000","last_clean":"2024-04-25T10:18:56.238269+0000","last_became_active":"2024-04-25T10:18:56.233910+0000","last_became_peered":"2024-04-25T10:18:56.233910+0000","last_unstale":"2024-04-25T10:18:56.238269+0000","last_undegraded":"2024-04-25T10:18:56.238269+0000","last_fullsized":"2024-04-25T10:18:56.238269+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:24.496059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098781+0000","last_change":"2024-04-25T10:18:55.241653+0000","last_active":"2024-04-25T10:18:58.098781+0000","last_peered":"2024-04-25T10:18:58.098781+0000","last_clean":"2024-04-25T10:18:58.098781+0000","last_became_active":"2024-04-25T10:18:55.239465+0000","last_became_peered":"2024-04-25T10:18:55.239465+0000","last_unstale":"2024-04-25T10:18:58.098781+0000","last_undegraded":"2024-04-25T10:18:58.098781+0000","last_fullsized":"2024-04-25T10:18:58.098781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:03.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237037+0000","last_change":"2024-04-25T10:18:56.250780+0000","last_active":"2024-04-25T10:18:57.237037+0000","last_peered":"2024-04-25T10:18:57.237037+0000","last_clean":"2024-04-25T10:18:57.237037+0000","last_became_active":"2024-04-25T10:18:56.250540+0000","last_became_peered":"2024-04-25T10:18:56.250540+0000","last_unstale":"2024-04-25T10:18:57.237037+0000","last_undegraded":"2024-04-25T10:18:57.237037+0000","last_fullsized":"2024-04-25T10:18:57.237037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.656466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-25T10:18:56.225752+0000","last_change":"2024-04-25T10:18:55.231867+0000","last_active":"2024-04-25T10:18:56.225752+0000","last_peered":"2024-04-25T10:18:56.225752+0000","last_clean":"2024-04-25T10:18:56.225752+0000","last_became_active":"2024-04-25T10:18:55.231431+0000","last_became_peered":"2024-04-25T10:18:55.231431+0000","last_unstale":"2024-04-25T10:18:56.225752+0000","last_undegraded":"2024-04-25T10:18:56.225752+0000","last_fullsized":"2024-04-25T10:18:56.225752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.615886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099151+0000","last_change":"2024-04-25T10:18:56.234895+0000","last_active":"2024-04-25T10:18:58.099151+0000","last_peered":"2024-04-25T10:18:58.099151+0000","last_clean":"2024-04-25T10:18:58.099151+0000","last_became_active":"2024-04-25T10:18:56.234511+0000","last_became_peered":"2024-04-25T10:18:56.234511+0000","last_unstale":"2024-04-25T10:18:58.099151+0000","last_undegraded":"2024-04-25T10:18:58.099151+0000","last_fullsized":"2024-04-25T10:18:58.099151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237143+0000","last_change":"2024-04-25T10:18:55.225543+0000","last_active":"2024-04-25T10:18:57.237143+0000","last_peered":"2024-04-25T10:18:57.237143+0000","last_clean":"2024-04-25T10:18:57.237143+0000","last_became_active":"2024-04-25T10:18:55.225225+0000","last_became_peered":"2024-04-25T10:18:55.225225+0000","last_unstale":"2024-04-25T10:18:57.237143+0000","last_undegraded":"2024-04-25T10:18:57.237143+0000","last_fullsized":"2024-04-25T10:18:57.237143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:59.193894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532479+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:57.532479+0000","last_peered":"2024-04-25T10:18:57.532479+0000","last_clean":"2024-04-25T10:18:57.532479+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:57.532479+0000","last_undegraded":"2024-04-25T10:18:57.532479+0000","last_fullsized":"2024-04-25T10:18:57.532479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":742422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":152248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":145206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":741910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":146806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:04.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:05.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb7a00640 1 Processor -- start 2024-04-25T10:19:05.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb7a00640 1 -- start start 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb7a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb00c6a20 0x7f8eb0166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb7a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb00ccfb0 con 0x7f8eb015b9e0 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb7a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb00cd290 con 0x7f8eb00ccb20 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb7a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb00cd570 con 0x7f8eb00c6a20 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb00c6a20 0x7f8eb0166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8eb015b9e0 0x7f8eb00cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48132/0 (socket says 172.21.15.45:48132) 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5000640 1 -- 172.21.15.45:0/1639820576 learned_addr learned my addr 172.21.15.45:0/1639820576 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8eb00c6a20 0x7f8eb0166820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50062/0 (socket says 172.21.15.45:50062) 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5a00640 1 -- 172.21.15.45:0/1639820576 >> v1:172.21.15.117:6789/0 conn(0x7f8eb00ccb20 legacy=0x7f8eb00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5a00640 1 -- 172.21.15.45:0/1639820576 >> v1:172.21.15.45:6789/0 conn(0x7f8eb015b9e0 legacy=0x7f8eb00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.064+0000 7f8eb5a00640 1 -- 172.21.15.45:0/1639820576 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00cd850 con 0x7f8eb00c6a20 2024-04-25T10:19:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb5a00640 1 --2- 172.21.15.45:0/1639820576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb0166820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009d90 tx=0x7f8ea801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36b2959732629b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eaf400640 1 -- 172.21.15.45:0/1639820576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80b2070 con 0x7f8eb00c6a20 2024-04-25T10:19:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eaf400640 1 -- 172.21.15.45:0/1639820576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80a58e0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eaf400640 1 -- 172.21.15.45:0/1639820576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80a5cb0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 -- 172.21.15.45:0/1639820576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 msgr2=0x7f8eb0166820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/1639820576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb0166820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009d90 tx=0x7f8ea801c970 comp rx=0 tx=0).stop 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 -- 172.21.15.45:0/1639820576 shutdown_connections 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/1639820576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb0166820 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 -- 172.21.15.45:0/1639820576 >> 172.21.15.45:0/1639820576 conn(0x7f8eb009a9d0 msgr2=0x7f8eb009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 -- 172.21.15.45:0/1639820576 shutdown_connections 2024-04-25T10:19:05.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 -- 172.21.15.45:0/1639820576 wait complete. 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 Processor -- start 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.068+0000 7f8eb7a00640 1 -- start start 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb00cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eb015b9e0 0x7f8eb01f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb0159e30 0x7f8eb00ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0169910 con 0x7f8eb00c6a20 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0169790 con 0x7f8eb0159e30 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0169a90 con 0x7f8eb015b9e0 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb00cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb00cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50072/0 (socket says 172.21.15.45:50072) 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 -- 172.21.15.45:0/866958491 learned_addr learned my addr 172.21.15.45:0/866958491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eafe00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eb015b9e0 0x7f8eb01f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb0159e30 0x7f8eb00ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 -- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eb015b9e0 msgr2=0x7f8eb01f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eb015b9e0 0x7f8eb01f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 -- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb0159e30 msgr2=0x7f8eb00ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb0159e30 0x7f8eb00ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 -- 172.21.15.45:0/866958491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb015d5e0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb0159e30 0x7f8eb00ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eafe00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eb015b9e0 0x7f8eb01f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb5000640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb00cb520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400c400 tx=0x7f8ea400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4017070 con 0x7f8eb00c6a20 2024-04-25T10:19:05.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea4005cf0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40072d0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb015d8c0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb015de80 con 0x7f8eb00c6a20 2024-04-25T10:19:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.072+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb004ef30 con 0x7f8eb00c6a20 2024-04-25T10:19:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.076+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ea401a070 con 0x7f8eb00c6a20 2024-04-25T10:19:05.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.076+0000 7f8eacc00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8e90066f60 0x7f8e90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.076+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ea4084320 con 0x7f8eb00c6a20 2024-04-25T10:19:05.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.076+0000 7f8eafe00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8e90066f60 0x7f8e90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.076+0000 7f8eafe00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8e90066f60 0x7f8e90069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8eb015a330 tx=0x7f8e9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.080+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ea4054cd0 con 0x7f8eb00c6a20 2024-04-25T10:19:05.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.236+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8eb01f7280 con 0x7f8e90066f60 2024-04-25T10:19:05.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.244+0000 7f8eacc00640 1 -- 172.21.15.45:0/866958491 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251867 (secure 0 0 0) 0x7f8eb01f7280 con 0x7f8e90066f60 2024-04-25T10:19:05.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:19:05.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8e90066f60 msgr2=0x7f8e90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8e90066f60 0x7f8e90069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8eb015a330 tx=0x7f8e9c009040 comp rx=0 tx=0).stop 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 msgr2=0x7f8eb00cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb00cb520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400c400 tx=0x7f8ea400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 shutdown_connections 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8e90066f60 0x7f8e90069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb0159e30 0x7f8eb00ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8eb015b9e0 0x7f8eb01f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 --2- 172.21.15.45:0/866958491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb00c6a20 0x7f8eb00cb520 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 >> 172.21.15.45:0/866958491 conn(0x7f8eb009a9d0 msgr2=0x7f8eb006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 shutdown_connections 2024-04-25T10:19:05.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.248+0000 7f8eb7a00640 1 -- 172.21.15.45:0/866958491 wait complete. 2024-04-25T10:19:05.270 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T10:19:04.888337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241248,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733472,"statfs":{"total":773094113280,"available":772847075328,"internally_reserved":0,"allocated":4366336,"data_stored":2397332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.670767"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098495+0000","last_change":"2024-04-25T10:18:56.253253+0000","last_active":"2024-04-25T10:18:58.098495+0000","last_peered":"2024-04-25T10:18:58.098495+0000","last_clean":"2024-04-25T10:18:58.098495+0000","last_became_active":"2024-04-25T10:18:56.252862+0000","last_became_peered":"2024-04-25T10:18:56.252862+0000","last_unstale":"2024-04-25T10:18:58.098495+0000","last_undegraded":"2024-04-25T10:18:58.098495+0000","last_fullsized":"2024-04-25T10:18:58.098495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:11.669773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237424+0000","last_change":"2024-04-25T10:18:56.249803+0000","last_active":"2024-04-25T10:18:57.237424+0000","last_peered":"2024-04-25T10:18:57.237424+0000","last_clean":"2024-04-25T10:18:57.237424+0000","last_became_active":"2024-04-25T10:18:56.249526+0000","last_became_peered":"2024-04-25T10:18:56.249526+0000","last_unstale":"2024-04-25T10:18:57.237424+0000","last_undegraded":"2024-04-25T10:18:57.237424+0000","last_fullsized":"2024-04-25T10:18:57.237424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532682+0000","last_change":"2024-04-25T10:18:56.251908+0000","last_active":"2024-04-25T10:18:57.532682+0000","last_peered":"2024-04-25T10:18:57.532682+0000","last_clean":"2024-04-25T10:18:57.532682+0000","last_became_active":"2024-04-25T10:18:56.251585+0000","last_became_peered":"2024-04-25T10:18:56.251585+0000","last_unstale":"2024-04-25T10:18:57.532682+0000","last_undegraded":"2024-04-25T10:18:57.532682+0000","last_fullsized":"2024-04-25T10:18:57.532682+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:37.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097756+0000","last_change":"2024-04-25T10:18:56.258089+0000","last_active":"2024-04-25T10:18:58.097756+0000","last_peered":"2024-04-25T10:18:58.097756+0000","last_clean":"2024-04-25T10:18:58.097756+0000","last_became_active":"2024-04-25T10:18:56.257834+0000","last_became_peered":"2024-04-25T10:18:56.257834+0000","last_unstale":"2024-04-25T10:18:58.097756+0000","last_undegraded":"2024-04-25T10:18:58.097756+0000","last_fullsized":"2024-04-25T10:18:58.097756+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:58.138865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237046+0000","last_change":"2024-04-25T10:18:56.234170+0000","last_active":"2024-04-25T10:18:57.237046+0000","last_peered":"2024-04-25T10:18:57.237046+0000","last_clean":"2024-04-25T10:18:57.237046+0000","last_became_active":"2024-04-25T10:18:56.233918+0000","last_became_peered":"2024-04-25T10:18:56.233918+0000","last_unstale":"2024-04-25T10:18:57.237046+0000","last_undegraded":"2024-04-25T10:18:57.237046+0000","last_fullsized":"2024-04-25T10:18:57.237046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238362+0000","last_change":"2024-04-25T10:18:56.254897+0000","last_active":"2024-04-25T10:18:57.238362+0000","last_peered":"2024-04-25T10:18:57.238362+0000","last_clean":"2024-04-25T10:18:57.238362+0000","last_became_active":"2024-04-25T10:18:56.254641+0000","last_became_peered":"2024-04-25T10:18:56.254641+0000","last_unstale":"2024-04-25T10:18:57.238362+0000","last_undegraded":"2024-04-25T10:18:57.238362+0000","last_fullsized":"2024-04-25T10:18:57.238362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:37.199395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098293+0000","last_change":"2024-04-25T10:18:56.239390+0000","last_active":"2024-04-25T10:18:58.098293+0000","last_peered":"2024-04-25T10:18:58.098293+0000","last_clean":"2024-04-25T10:18:58.098293+0000","last_became_active":"2024-04-25T10:18:56.239075+0000","last_became_peered":"2024-04-25T10:18:56.239075+0000","last_unstale":"2024-04-25T10:18:58.098293+0000","last_undegraded":"2024-04-25T10:18:58.098293+0000","last_fullsized":"2024-04-25T10:18:58.098293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098140+0000","last_change":"2024-04-25T10:18:56.240095+0000","last_active":"2024-04-25T10:18:58.098140+0000","last_peered":"2024-04-25T10:18:58.098140+0000","last_clean":"2024-04-25T10:18:58.098140+0000","last_became_active":"2024-04-25T10:18:56.239416+0000","last_became_peered":"2024-04-25T10:18:56.239416+0000","last_unstale":"2024-04-25T10:18:58.098140+0000","last_undegraded":"2024-04-25T10:18:58.098140+0000","last_fullsized":"2024-04-25T10:18:58.098140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532815+0000","last_change":"2024-04-25T10:18:56.237743+0000","last_active":"2024-04-25T10:18:57.532815+0000","last_peered":"2024-04-25T10:18:57.532815+0000","last_clean":"2024-04-25T10:18:57.532815+0000","last_became_active":"2024-04-25T10:18:56.237529+0000","last_became_peered":"2024-04-25T10:18:56.237529+0000","last_unstale":"2024-04-25T10:18:57.532815+0000","last_undegraded":"2024-04-25T10:18:57.532815+0000","last_fullsized":"2024-04-25T10:18:57.532815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.140873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097987+0000","last_change":"2024-04-25T10:18:56.250051+0000","last_active":"2024-04-25T10:18:58.097987+0000","last_peered":"2024-04-25T10:18:58.097987+0000","last_clean":"2024-04-25T10:18:58.097987+0000","last_became_active":"2024-04-25T10:18:56.247576+0000","last_became_peered":"2024-04-25T10:18:56.247576+0000","last_unstale":"2024-04-25T10:18:58.097987+0000","last_undegraded":"2024-04-25T10:18:58.097987+0000","last_fullsized":"2024-04-25T10:18:58.097987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:55.912866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236922+0000","last_change":"2024-04-25T10:18:56.254679+0000","last_active":"2024-04-25T10:18:57.236922+0000","last_peered":"2024-04-25T10:18:57.236922+0000","last_clean":"2024-04-25T10:18:57.236922+0000","last_became_active":"2024-04-25T10:18:56.254334+0000","last_became_peered":"2024-04-25T10:18:56.254334+0000","last_unstale":"2024-04-25T10:18:57.236922+0000","last_undegraded":"2024-04-25T10:18:57.236922+0000","last_fullsized":"2024-04-25T10:18:57.236922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:47.661457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532655+0000","last_change":"2024-04-25T10:18:56.234697+0000","last_active":"2024-04-25T10:18:57.532655+0000","last_peered":"2024-04-25T10:18:57.532655+0000","last_clean":"2024-04-25T10:18:57.532655+0000","last_became_active":"2024-04-25T10:18:56.234433+0000","last_became_peered":"2024-04-25T10:18:56.234433+0000","last_unstale":"2024-04-25T10:18:57.532655+0000","last_undegraded":"2024-04-25T10:18:57.532655+0000","last_fullsized":"2024-04-25T10:18:57.532655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:34.950803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814702+0000","last_change":"2024-04-25T10:18:56.243767+0000","last_active":"2024-04-25T10:18:57.814702+0000","last_peered":"2024-04-25T10:18:57.814702+0000","last_clean":"2024-04-25T10:18:57.814702+0000","last_became_active":"2024-04-25T10:18:56.243469+0000","last_became_peered":"2024-04-25T10:18:56.243469+0000","last_unstale":"2024-04-25T10:18:57.814702+0000","last_undegraded":"2024-04-25T10:18:57.814702+0000","last_fullsized":"2024-04-25T10:18:57.814702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097367+0000","last_change":"2024-04-25T10:18:56.247202+0000","last_active":"2024-04-25T10:18:58.097367+0000","last_peered":"2024-04-25T10:18:58.097367+0000","last_clean":"2024-04-25T10:18:58.097367+0000","last_became_active":"2024-04-25T10:18:56.240273+0000","last_became_peered":"2024-04-25T10:18:56.240273+0000","last_unstale":"2024-04-25T10:18:58.097367+0000","last_undegraded":"2024-04-25T10:18:58.097367+0000","last_fullsized":"2024-04-25T10:18:58.097367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:55.091475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098827+0000","last_change":"2024-04-25T10:18:56.243167+0000","last_active":"2024-04-25T10:18:58.098827+0000","last_peered":"2024-04-25T10:18:58.098827+0000","last_clean":"2024-04-25T10:18:58.098827+0000","last_became_active":"2024-04-25T10:18:56.242874+0000","last_became_peered":"2024-04-25T10:18:56.242874+0000","last_unstale":"2024-04-25T10:18:58.098827+0000","last_undegraded":"2024-04-25T10:18:58.098827+0000","last_fullsized":"2024-04-25T10:18:58.098827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:27.931466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098186+0000","last_change":"2024-04-25T10:18:56.254154+0000","last_active":"2024-04-25T10:18:58.098186+0000","last_peered":"2024-04-25T10:18:58.098186+0000","last_clean":"2024-04-25T10:18:58.098186+0000","last_became_active":"2024-04-25T10:18:56.251652+0000","last_became_peered":"2024-04-25T10:18:56.251652+0000","last_unstale":"2024-04-25T10:18:58.098186+0000","last_undegraded":"2024-04-25T10:18:58.098186+0000","last_fullsized":"2024-04-25T10:18:58.098186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.086557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236907+0000","last_change":"2024-04-25T10:18:56.261187+0000","last_active":"2024-04-25T10:18:57.236907+0000","last_peered":"2024-04-25T10:18:57.236907+0000","last_clean":"2024-04-25T10:18:57.236907+0000","last_became_active":"2024-04-25T10:18:56.259913+0000","last_became_peered":"2024-04-25T10:18:56.259913+0000","last_unstale":"2024-04-25T10:18:57.236907+0000","last_undegraded":"2024-04-25T10:18:57.236907+0000","last_fullsized":"2024-04-25T10:18:57.236907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:31.833160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238694+0000","last_change":"2024-04-25T10:18:56.254191+0000","last_active":"2024-04-25T10:18:57.238694+0000","last_peered":"2024-04-25T10:18:57.238694+0000","last_clean":"2024-04-25T10:18:57.238694+0000","last_became_active":"2024-04-25T10:18:56.253840+0000","last_became_peered":"2024-04-25T10:18:56.253840+0000","last_unstale":"2024-04-25T10:18:57.238694+0000","last_undegraded":"2024-04-25T10:18:57.238694+0000","last_fullsized":"2024-04-25T10:18:57.238694+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:46.664788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238638+0000","last_change":"2024-04-25T10:18:56.237511+0000","last_active":"2024-04-25T10:18:57.238638+0000","last_peered":"2024-04-25T10:18:57.238638+0000","last_clean":"2024-04-25T10:18:57.238638+0000","last_became_active":"2024-04-25T10:18:56.237233+0000","last_became_peered":"2024-04-25T10:18:56.237233+0000","last_unstale":"2024-04-25T10:18:57.238638+0000","last_undegraded":"2024-04-25T10:18:57.238638+0000","last_fullsized":"2024-04-25T10:18:57.238638+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.450233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814870+0000","last_change":"2024-04-25T10:18:56.245428+0000","last_active":"2024-04-25T10:18:57.814870+0000","last_peered":"2024-04-25T10:18:57.814870+0000","last_clean":"2024-04-25T10:18:57.814870+0000","last_became_active":"2024-04-25T10:18:56.245143+0000","last_became_peered":"2024-04-25T10:18:56.245143+0000","last_unstale":"2024-04-25T10:18:57.814870+0000","last_undegraded":"2024-04-25T10:18:57.814870+0000","last_fullsized":"2024-04-25T10:18:57.814870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099156+0000","last_change":"2024-04-25T10:18:56.246924+0000","last_active":"2024-04-25T10:18:58.099156+0000","last_peered":"2024-04-25T10:18:58.099156+0000","last_clean":"2024-04-25T10:18:58.099156+0000","last_became_active":"2024-04-25T10:18:56.246327+0000","last_became_peered":"2024-04-25T10:18:56.246327+0000","last_unstale":"2024-04-25T10:18:58.099156+0000","last_undegraded":"2024-04-25T10:18:58.099156+0000","last_fullsized":"2024-04-25T10:18:58.099156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098486+0000","last_change":"2024-04-25T10:18:56.252008+0000","last_active":"2024-04-25T10:18:58.098486+0000","last_peered":"2024-04-25T10:18:58.098486+0000","last_clean":"2024-04-25T10:18:58.098486+0000","last_became_active":"2024-04-25T10:18:56.251724+0000","last_became_peered":"2024-04-25T10:18:56.251724+0000","last_unstale":"2024-04-25T10:18:58.098486+0000","last_undegraded":"2024-04-25T10:18:58.098486+0000","last_fullsized":"2024-04-25T10:18:58.098486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.306720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814498+0000","last_change":"2024-04-25T10:18:56.238540+0000","last_active":"2024-04-25T10:18:57.814498+0000","last_peered":"2024-04-25T10:18:57.814498+0000","last_clean":"2024-04-25T10:18:57.814498+0000","last_became_active":"2024-04-25T10:18:56.238216+0000","last_became_peered":"2024-04-25T10:18:56.238216+0000","last_unstale":"2024-04-25T10:18:57.814498+0000","last_undegraded":"2024-04-25T10:18:57.814498+0000","last_fullsized":"2024-04-25T10:18:57.814498+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:27.931466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098348+0000","last_change":"2024-04-25T10:18:56.240657+0000","last_active":"2024-04-25T10:18:58.098348+0000","last_peered":"2024-04-25T10:18:58.098348+0000","last_clean":"2024-04-25T10:18:58.098348+0000","last_became_active":"2024-04-25T10:18:56.240396+0000","last_became_peered":"2024-04-25T10:18:56.240396+0000","last_unstale":"2024-04-25T10:18:58.098348+0000","last_undegraded":"2024-04-25T10:18:58.098348+0000","last_fullsized":"2024-04-25T10:18:58.098348+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098134+0000","last_change":"2024-04-25T10:18:56.247980+0000","last_active":"2024-04-25T10:18:58.098134+0000","last_peered":"2024-04-25T10:18:58.098134+0000","last_clean":"2024-04-25T10:18:58.098134+0000","last_became_active":"2024-04-25T10:18:56.247334+0000","last_became_peered":"2024-04-25T10:18:56.247334+0000","last_unstale":"2024-04-25T10:18:58.098134+0000","last_undegraded":"2024-04-25T10:18:58.098134+0000","last_fullsized":"2024-04-25T10:18:58.098134+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:40.744260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098062+0000","last_change":"2024-04-25T10:18:56.247318+0000","last_active":"2024-04-25T10:18:58.098062+0000","last_peered":"2024-04-25T10:18:58.098062+0000","last_clean":"2024-04-25T10:18:58.098062+0000","last_became_active":"2024-04-25T10:18:56.246972+0000","last_became_peered":"2024-04-25T10:18:56.246972+0000","last_unstale":"2024-04-25T10:18:58.098062+0000","last_undegraded":"2024-04-25T10:18:58.098062+0000","last_fullsized":"2024-04-25T10:18:58.098062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:03.220997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237571+0000","last_change":"2024-04-25T10:18:56.239544+0000","last_active":"2024-04-25T10:18:57.237571+0000","last_peered":"2024-04-25T10:18:57.237571+0000","last_clean":"2024-04-25T10:18:57.237571+0000","last_became_active":"2024-04-25T10:18:56.236085+0000","last_became_peered":"2024-04-25T10:18:56.236085+0000","last_unstale":"2024-04-25T10:18:57.237571+0000","last_undegraded":"2024-04-25T10:18:57.237571+0000","last_fullsized":"2024-04-25T10:18:57.237571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:25.904643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098644+0000","last_change":"2024-04-25T10:18:56.257431+0000","last_active":"2024-04-25T10:18:58.098644+0000","last_peered":"2024-04-25T10:18:58.098644+0000","last_clean":"2024-04-25T10:18:58.098644+0000","last_became_active":"2024-04-25T10:18:56.257033+0000","last_became_peered":"2024-04-25T10:18:56.257033+0000","last_unstale":"2024-04-25T10:18:58.098644+0000","last_undegraded":"2024-04-25T10:18:58.098644+0000","last_fullsized":"2024-04-25T10:18:58.098644+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:30.757565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099042+0000","last_change":"2024-04-25T10:18:56.247122+0000","last_active":"2024-04-25T10:18:58.099042+0000","last_peered":"2024-04-25T10:18:58.099042+0000","last_clean":"2024-04-25T10:18:58.099042+0000","last_became_active":"2024-04-25T10:18:56.246671+0000","last_became_peered":"2024-04-25T10:18:56.246671+0000","last_unstale":"2024-04-25T10:18:58.099042+0000","last_undegraded":"2024-04-25T10:18:58.099042+0000","last_fullsized":"2024-04-25T10:18:58.099042+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:51.984516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099002+0000","last_change":"2024-04-25T10:18:56.244715+0000","last_active":"2024-04-25T10:18:58.099002+0000","last_peered":"2024-04-25T10:18:58.099002+0000","last_clean":"2024-04-25T10:18:58.099002+0000","last_became_active":"2024-04-25T10:18:56.244456+0000","last_became_peered":"2024-04-25T10:18:56.244456+0000","last_unstale":"2024-04-25T10:18:58.099002+0000","last_undegraded":"2024-04-25T10:18:58.099002+0000","last_fullsized":"2024-04-25T10:18:58.099002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:26.300653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098514+0000","last_change":"2024-04-25T10:18:56.243348+0000","last_active":"2024-04-25T10:18:58.098514+0000","last_peered":"2024-04-25T10:18:58.098514+0000","last_clean":"2024-04-25T10:18:58.098514+0000","last_became_active":"2024-04-25T10:18:56.243050+0000","last_became_peered":"2024-04-25T10:18:56.243050+0000","last_unstale":"2024-04-25T10:18:58.098514+0000","last_undegraded":"2024-04-25T10:18:58.098514+0000","last_fullsized":"2024-04-25T10:18:58.098514+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:05.345870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237064+0000","last_change":"2024-04-25T10:18:56.261132+0000","last_active":"2024-04-25T10:18:57.237064+0000","last_peered":"2024-04-25T10:18:57.237064+0000","last_clean":"2024-04-25T10:18:57.237064+0000","last_became_active":"2024-04-25T10:18:56.259485+0000","last_became_peered":"2024-04-25T10:18:56.259485+0000","last_unstale":"2024-04-25T10:18:57.237064+0000","last_undegraded":"2024-04-25T10:18:57.237064+0000","last_fullsized":"2024-04-25T10:18:57.237064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:04.765169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.815019+0000","last_change":"2024-04-25T10:18:56.245528+0000","last_active":"2024-04-25T10:18:57.815019+0000","last_peered":"2024-04-25T10:18:57.815019+0000","last_clean":"2024-04-25T10:18:57.815019+0000","last_became_active":"2024-04-25T10:18:56.245224+0000","last_became_peered":"2024-04-25T10:18:56.245224+0000","last_unstale":"2024-04-25T10:18:57.815019+0000","last_undegraded":"2024-04-25T10:18:57.815019+0000","last_fullsized":"2024-04-25T10:18:57.815019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236935+0000","last_change":"2024-04-25T10:18:55.228870+0000","last_active":"2024-04-25T10:18:57.236935+0000","last_peered":"2024-04-25T10:18:57.236935+0000","last_clean":"2024-04-25T10:18:57.236935+0000","last_became_active":"2024-04-25T10:18:55.228570+0000","last_became_peered":"2024-04-25T10:18:55.228570+0000","last_unstale":"2024-04-25T10:18:57.236935+0000","last_undegraded":"2024-04-25T10:18:57.236935+0000","last_fullsized":"2024-04-25T10:18:57.236935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:39.897606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098657+0000","last_change":"2024-04-25T10:18:56.251166+0000","last_active":"2024-04-25T10:18:58.098657+0000","last_peered":"2024-04-25T10:18:58.098657+0000","last_clean":"2024-04-25T10:18:58.098657+0000","last_became_active":"2024-04-25T10:18:56.250808+0000","last_became_peered":"2024-04-25T10:18:56.250808+0000","last_unstale":"2024-04-25T10:18:58.098657+0000","last_undegraded":"2024-04-25T10:18:58.098657+0000","last_fullsized":"2024-04-25T10:18:58.098657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:06.007308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098084+0000","last_change":"2024-04-25T10:18:55.232915+0000","last_active":"2024-04-25T10:18:58.098084+0000","last_peered":"2024-04-25T10:18:58.098084+0000","last_clean":"2024-04-25T10:18:58.098084+0000","last_became_active":"2024-04-25T10:18:55.232635+0000","last_became_peered":"2024-04-25T10:18:55.232635+0000","last_unstale":"2024-04-25T10:18:58.098084+0000","last_undegraded":"2024-04-25T10:18:58.098084+0000","last_fullsized":"2024-04-25T10:18:58.098084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:16.184267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098660+0000","last_change":"2024-04-25T10:18:56.259417+0000","last_active":"2024-04-25T10:18:58.098660+0000","last_peered":"2024-04-25T10:18:58.098660+0000","last_clean":"2024-04-25T10:18:58.098660+0000","last_became_active":"2024-04-25T10:18:56.256746+0000","last_became_peered":"2024-04-25T10:18:56.256746+0000","last_unstale":"2024-04-25T10:18:58.098660+0000","last_undegraded":"2024-04-25T10:18:58.098660+0000","last_fullsized":"2024-04-25T10:18:58.098660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.560656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238766+0000","last_change":"2024-04-25T10:18:55.241402+0000","last_active":"2024-04-25T10:18:57.238766+0000","last_peered":"2024-04-25T10:18:57.238766+0000","last_clean":"2024-04-25T10:18:57.238766+0000","last_became_active":"2024-04-25T10:18:55.241057+0000","last_became_peered":"2024-04-25T10:18:55.241057+0000","last_unstale":"2024-04-25T10:18:57.238766+0000","last_undegraded":"2024-04-25T10:18:57.238766+0000","last_fullsized":"2024-04-25T10:18:57.238766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:48.540119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097873+0000","last_change":"2024-04-25T10:18:56.238845+0000","last_active":"2024-04-25T10:18:58.097873+0000","last_peered":"2024-04-25T10:18:58.097873+0000","last_clean":"2024-04-25T10:18:58.097873+0000","last_became_active":"2024-04-25T10:18:56.233835+0000","last_became_peered":"2024-04-25T10:18:56.233835+0000","last_unstale":"2024-04-25T10:18:58.097873+0000","last_undegraded":"2024-04-25T10:18:58.097873+0000","last_fullsized":"2024-04-25T10:18:58.097873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:15.426455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097862+0000","last_change":"2024-04-25T10:18:55.239850+0000","last_active":"2024-04-25T10:18:58.097862+0000","last_peered":"2024-04-25T10:18:58.097862+0000","last_clean":"2024-04-25T10:18:58.097862+0000","last_became_active":"2024-04-25T10:18:55.236286+0000","last_became_peered":"2024-04-25T10:18:55.236286+0000","last_unstale":"2024-04-25T10:18:58.097862+0000","last_undegraded":"2024-04-25T10:18:58.097862+0000","last_fullsized":"2024-04-25T10:18:58.097862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:59.659174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532529+0000","last_change":"2024-04-25T10:18:56.229987+0000","last_active":"2024-04-25T10:18:57.532529+0000","last_peered":"2024-04-25T10:18:57.532529+0000","last_clean":"2024-04-25T10:18:57.532529+0000","last_became_active":"2024-04-25T10:18:56.229484+0000","last_became_peered":"2024-04-25T10:18:56.229484+0000","last_unstale":"2024-04-25T10:18:57.532529+0000","last_undegraded":"2024-04-25T10:18:57.532529+0000","last_fullsized":"2024-04-25T10:18:57.532529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:20.518835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098686+0000","last_change":"2024-04-25T10:18:55.233523+0000","last_active":"2024-04-25T10:18:58.098686+0000","last_peered":"2024-04-25T10:18:58.098686+0000","last_clean":"2024-04-25T10:18:58.098686+0000","last_became_active":"2024-04-25T10:18:55.233010+0000","last_became_peered":"2024-04-25T10:18:55.233010+0000","last_unstale":"2024-04-25T10:18:58.098686+0000","last_undegraded":"2024-04-25T10:18:58.098686+0000","last_fullsized":"2024-04-25T10:18:58.098686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:21.018773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098430+0000","last_change":"2024-04-25T10:18:56.244147+0000","last_active":"2024-04-25T10:18:58.098430+0000","last_peered":"2024-04-25T10:18:58.098430+0000","last_clean":"2024-04-25T10:18:58.098430+0000","last_became_active":"2024-04-25T10:18:56.243866+0000","last_became_peered":"2024-04-25T10:18:56.243866+0000","last_unstale":"2024-04-25T10:18:58.098430+0000","last_undegraded":"2024-04-25T10:18:58.098430+0000","last_fullsized":"2024-04-25T10:18:58.098430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:25.904643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098678+0000","last_change":"2024-04-25T10:18:55.233989+0000","last_active":"2024-04-25T10:18:58.098678+0000","last_peered":"2024-04-25T10:18:58.098678+0000","last_clean":"2024-04-25T10:18:58.098678+0000","last_became_active":"2024-04-25T10:18:55.233717+0000","last_became_peered":"2024-04-25T10:18:55.233717+0000","last_unstale":"2024-04-25T10:18:58.098678+0000","last_undegraded":"2024-04-25T10:18:58.098678+0000","last_fullsized":"2024-04-25T10:18:58.098678+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:02.604590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532416+0000","last_change":"2024-04-25T10:18:56.235738+0000","last_active":"2024-04-25T10:18:57.532416+0000","last_peered":"2024-04-25T10:18:57.532416+0000","last_clean":"2024-04-25T10:18:57.532416+0000","last_became_active":"2024-04-25T10:18:56.235450+0000","last_became_peered":"2024-04-25T10:18:56.235450+0000","last_unstale":"2024-04-25T10:18:57.532416+0000","last_undegraded":"2024-04-25T10:18:57.532416+0000","last_fullsized":"2024-04-25T10:18:57.532416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:03.610849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238301+0000","last_change":"2024-04-25T10:18:55.244101+0000","last_active":"2024-04-25T10:18:57.238301+0000","last_peered":"2024-04-25T10:18:57.238301+0000","last_clean":"2024-04-25T10:18:57.238301+0000","last_became_active":"2024-04-25T10:18:55.243832+0000","last_became_peered":"2024-04-25T10:18:55.243832+0000","last_unstale":"2024-04-25T10:18:57.238301+0000","last_undegraded":"2024-04-25T10:18:57.238301+0000","last_fullsized":"2024-04-25T10:18:57.238301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:19.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097284+0000","last_change":"2024-04-25T10:18:56.245450+0000","last_active":"2024-04-25T10:18:58.097284+0000","last_peered":"2024-04-25T10:18:58.097284+0000","last_clean":"2024-04-25T10:18:58.097284+0000","last_became_active":"2024-04-25T10:18:56.245230+0000","last_became_peered":"2024-04-25T10:18:56.245230+0000","last_unstale":"2024-04-25T10:18:58.097284+0000","last_undegraded":"2024-04-25T10:18:58.097284+0000","last_fullsized":"2024-04-25T10:18:58.097284+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:49.973526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098626+0000","last_change":"2024-04-25T10:18:55.241837+0000","last_active":"2024-04-25T10:18:58.098626+0000","last_peered":"2024-04-25T10:18:58.098626+0000","last_clean":"2024-04-25T10:18:58.098626+0000","last_became_active":"2024-04-25T10:18:55.241552+0000","last_became_peered":"2024-04-25T10:18:55.241552+0000","last_unstale":"2024-04-25T10:18:58.098626+0000","last_undegraded":"2024-04-25T10:18:58.098626+0000","last_fullsized":"2024-04-25T10:18:58.098626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:37.997874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236783+0000","last_change":"2024-04-25T10:18:56.245632+0000","last_active":"2024-04-25T10:18:57.236783+0000","last_peered":"2024-04-25T10:18:57.236783+0000","last_clean":"2024-04-25T10:18:57.236783+0000","last_became_active":"2024-04-25T10:18:56.244499+0000","last_became_peered":"2024-04-25T10:18:56.244499+0000","last_unstale":"2024-04-25T10:18:57.236783+0000","last_undegraded":"2024-04-25T10:18:57.236783+0000","last_fullsized":"2024-04-25T10:18:57.236783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:05.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237522+0000","last_change":"2024-04-25T10:18:55.237962+0000","last_active":"2024-04-25T10:18:57.237522+0000","last_peered":"2024-04-25T10:18:57.237522+0000","last_clean":"2024-04-25T10:18:57.237522+0000","last_became_active":"2024-04-25T10:18:55.234540+0000","last_became_peered":"2024-04-25T10:18:55.234540+0000","last_unstale":"2024-04-25T10:18:57.237522+0000","last_undegraded":"2024-04-25T10:18:57.237522+0000","last_fullsized":"2024-04-25T10:18:57.237522+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:05.443974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098160+0000","last_change":"2024-04-25T10:18:56.248260+0000","last_active":"2024-04-25T10:18:58.098160+0000","last_peered":"2024-04-25T10:18:58.098160+0000","last_clean":"2024-04-25T10:18:58.098160+0000","last_became_active":"2024-04-25T10:18:56.247668+0000","last_became_peered":"2024-04-25T10:18:56.247668+0000","last_unstale":"2024-04-25T10:18:58.098160+0000","last_undegraded":"2024-04-25T10:18:58.098160+0000","last_fullsized":"2024-04-25T10:18:58.098160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:03.923402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237444+0000","last_change":"2024-04-25T10:18:55.234932+0000","last_active":"2024-04-25T10:18:57.237444+0000","last_peered":"2024-04-25T10:18:57.237444+0000","last_clean":"2024-04-25T10:18:57.237444+0000","last_became_active":"2024-04-25T10:18:55.234587+0000","last_became_peered":"2024-04-25T10:18:55.234587+0000","last_unstale":"2024-04-25T10:18:57.237444+0000","last_undegraded":"2024-04-25T10:18:57.237444+0000","last_fullsized":"2024-04-25T10:18:57.237444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:04.346918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097825+0000","last_change":"2024-04-25T10:18:56.258154+0000","last_active":"2024-04-25T10:18:58.097825+0000","last_peered":"2024-04-25T10:18:58.097825+0000","last_clean":"2024-04-25T10:18:58.097825+0000","last_became_active":"2024-04-25T10:18:56.257886+0000","last_became_peered":"2024-04-25T10:18:56.257886+0000","last_unstale":"2024-04-25T10:18:58.097825+0000","last_undegraded":"2024-04-25T10:18:58.097825+0000","last_fullsized":"2024-04-25T10:18:58.097825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:56.103772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237047+0000","last_change":"2024-04-25T10:18:55.240359+0000","last_active":"2024-04-25T10:18:57.237047+0000","last_peered":"2024-04-25T10:18:57.237047+0000","last_clean":"2024-04-25T10:18:57.237047+0000","last_became_active":"2024-04-25T10:18:55.238481+0000","last_became_peered":"2024-04-25T10:18:55.238481+0000","last_unstale":"2024-04-25T10:18:57.237047+0000","last_undegraded":"2024-04-25T10:18:57.237047+0000","last_fullsized":"2024-04-25T10:18:57.237047+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:23.698128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236771+0000","last_change":"2024-04-25T10:18:56.254595+0000","last_active":"2024-04-25T10:18:57.236771+0000","last_peered":"2024-04-25T10:18:57.236771+0000","last_clean":"2024-04-25T10:18:57.236771+0000","last_became_active":"2024-04-25T10:18:56.253982+0000","last_became_peered":"2024-04-25T10:18:56.253982+0000","last_unstale":"2024-04-25T10:18:57.236771+0000","last_undegraded":"2024-04-25T10:18:57.236771+0000","last_fullsized":"2024-04-25T10:18:57.236771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:01.355924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814959+0000","last_change":"2024-04-25T10:18:55.242961+0000","last_active":"2024-04-25T10:18:57.814959+0000","last_peered":"2024-04-25T10:18:57.814959+0000","last_clean":"2024-04-25T10:18:57.814959+0000","last_became_active":"2024-04-25T10:18:55.242653+0000","last_became_peered":"2024-04-25T10:18:55.242653+0000","last_unstale":"2024-04-25T10:18:57.814959+0000","last_undegraded":"2024-04-25T10:18:57.814959+0000","last_fullsized":"2024-04-25T10:18:57.814959+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:19.512576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814465+0000","last_change":"2024-04-25T10:18:56.235468+0000","last_active":"2024-04-25T10:18:57.814465+0000","last_peered":"2024-04-25T10:18:57.814465+0000","last_clean":"2024-04-25T10:18:57.814465+0000","last_became_active":"2024-04-25T10:18:56.235134+0000","last_became_peered":"2024-04-25T10:18:56.235134+0000","last_unstale":"2024-04-25T10:18:57.814465+0000","last_undegraded":"2024-04-25T10:18:57.814465+0000","last_fullsized":"2024-04-25T10:18:57.814465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:22.129702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098585+0000","last_change":"2024-04-25T10:18:55.241199+0000","last_active":"2024-04-25T10:18:58.098585+0000","last_peered":"2024-04-25T10:18:58.098585+0000","last_clean":"2024-04-25T10:18:58.098585+0000","last_became_active":"2024-04-25T10:18:55.240168+0000","last_became_peered":"2024-04-25T10:18:55.240168+0000","last_unstale":"2024-04-25T10:18:58.098585+0000","last_undegraded":"2024-04-25T10:18:58.098585+0000","last_fullsized":"2024-04-25T10:18:58.098585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:33.944544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097621+0000","last_change":"2024-04-25T10:18:56.247101+0000","last_active":"2024-04-25T10:18:58.097621+0000","last_peered":"2024-04-25T10:18:58.097621+0000","last_clean":"2024-04-25T10:18:58.097621+0000","last_became_active":"2024-04-25T10:18:56.239778+0000","last_became_peered":"2024-04-25T10:18:56.239778+0000","last_unstale":"2024-04-25T10:18:58.097621+0000","last_undegraded":"2024-04-25T10:18:58.097621+0000","last_fullsized":"2024-04-25T10:18:58.097621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:37.550731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097345+0000","last_change":"2024-04-25T10:18:55.239619+0000","last_active":"2024-04-25T10:18:58.097345+0000","last_peered":"2024-04-25T10:18:58.097345+0000","last_clean":"2024-04-25T10:18:58.097345+0000","last_became_active":"2024-04-25T10:18:55.239342+0000","last_became_peered":"2024-04-25T10:18:55.239342+0000","last_unstale":"2024-04-25T10:18:58.097345+0000","last_undegraded":"2024-04-25T10:18:58.097345+0000","last_fullsized":"2024-04-25T10:18:58.097345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:54.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098479+0000","last_change":"2024-04-25T10:18:56.239731+0000","last_active":"2024-04-25T10:18:58.098479+0000","last_peered":"2024-04-25T10:18:58.098479+0000","last_clean":"2024-04-25T10:18:58.098479+0000","last_became_active":"2024-04-25T10:18:56.239151+0000","last_became_peered":"2024-04-25T10:18:56.239151+0000","last_unstale":"2024-04-25T10:18:58.098479+0000","last_undegraded":"2024-04-25T10:18:58.098479+0000","last_fullsized":"2024-04-25T10:18:58.098479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:24.279467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097192+0000","last_change":"2024-04-25T10:18:55.240382+0000","last_active":"2024-04-25T10:18:58.097192+0000","last_peered":"2024-04-25T10:18:58.097192+0000","last_clean":"2024-04-25T10:18:58.097192+0000","last_became_active":"2024-04-25T10:18:55.235717+0000","last_became_peered":"2024-04-25T10:18:55.235717+0000","last_unstale":"2024-04-25T10:18:58.097192+0000","last_undegraded":"2024-04-25T10:18:58.097192+0000","last_fullsized":"2024-04-25T10:18:58.097192+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:57.294140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236616+0000","last_change":"2024-04-25T10:18:56.260154+0000","last_active":"2024-04-25T10:18:57.236616+0000","last_peered":"2024-04-25T10:18:57.236616+0000","last_clean":"2024-04-25T10:18:57.236616+0000","last_became_active":"2024-04-25T10:18:56.259124+0000","last_became_peered":"2024-04-25T10:18:56.259124+0000","last_unstale":"2024-04-25T10:18:57.236616+0000","last_undegraded":"2024-04-25T10:18:57.236616+0000","last_fullsized":"2024-04-25T10:18:57.236616+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:59.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237057+0000","last_change":"2024-04-25T10:18:55.241451+0000","last_active":"2024-04-25T10:18:57.237057+0000","last_peered":"2024-04-25T10:18:57.237057+0000","last_clean":"2024-04-25T10:18:57.237057+0000","last_became_active":"2024-04-25T10:18:55.241168+0000","last_became_peered":"2024-04-25T10:18:55.241168+0000","last_unstale":"2024-04-25T10:18:57.237057+0000","last_undegraded":"2024-04-25T10:18:57.237057+0000","last_fullsized":"2024-04-25T10:18:57.237057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:48.653430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236854+0000","last_change":"2024-04-25T10:18:56.253037+0000","last_active":"2024-04-25T10:18:57.236854+0000","last_peered":"2024-04-25T10:18:57.236854+0000","last_clean":"2024-04-25T10:18:57.236854+0000","last_became_active":"2024-04-25T10:18:56.252771+0000","last_became_peered":"2024-04-25T10:18:56.252771+0000","last_unstale":"2024-04-25T10:18:57.236854+0000","last_undegraded":"2024-04-25T10:18:57.236854+0000","last_fullsized":"2024-04-25T10:18:57.236854+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:07.919477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238867+0000","last_change":"2024-04-25T10:18:55.244388+0000","last_active":"2024-04-25T10:18:57.238867+0000","last_peered":"2024-04-25T10:18:57.238867+0000","last_clean":"2024-04-25T10:18:57.238867+0000","last_became_active":"2024-04-25T10:18:55.244013+0000","last_became_peered":"2024-04-25T10:18:55.244013+0000","last_unstale":"2024-04-25T10:18:57.238867+0000","last_undegraded":"2024-04-25T10:18:57.238867+0000","last_fullsized":"2024-04-25T10:18:57.238867+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:06.417351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532826+0000","last_change":"2024-04-25T10:18:56.252562+0000","last_active":"2024-04-25T10:18:57.532826+0000","last_peered":"2024-04-25T10:18:57.532826+0000","last_clean":"2024-04-25T10:18:57.532826+0000","last_became_active":"2024-04-25T10:18:56.252305+0000","last_became_peered":"2024-04-25T10:18:56.252305+0000","last_unstale":"2024-04-25T10:18:57.532826+0000","last_undegraded":"2024-04-25T10:18:57.532826+0000","last_fullsized":"2024-04-25T10:18:57.532826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:06.450233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098217+0000","last_change":"2024-04-25T10:18:55.242568+0000","last_active":"2024-04-25T10:18:58.098217+0000","last_peered":"2024-04-25T10:18:58.098217+0000","last_clean":"2024-04-25T10:18:58.098217+0000","last_became_active":"2024-04-25T10:18:55.242249+0000","last_became_peered":"2024-04-25T10:18:55.242249+0000","last_unstale":"2024-04-25T10:18:58.098217+0000","last_undegraded":"2024-04-25T10:18:58.098217+0000","last_fullsized":"2024-04-25T10:18:58.098217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237336+0000","last_change":"2024-04-25T10:18:56.230062+0000","last_active":"2024-04-25T10:18:57.237336+0000","last_peered":"2024-04-25T10:18:57.237336+0000","last_clean":"2024-04-25T10:18:57.237336+0000","last_became_active":"2024-04-25T10:18:56.229753+0000","last_became_peered":"2024-04-25T10:18:56.229753+0000","last_unstale":"2024-04-25T10:18:57.237336+0000","last_undegraded":"2024-04-25T10:18:57.237336+0000","last_fullsized":"2024-04-25T10:18:57.237336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:00.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237044+0000","last_change":"2024-04-25T10:18:55.226724+0000","last_active":"2024-04-25T10:18:57.237044+0000","last_peered":"2024-04-25T10:18:57.237044+0000","last_clean":"2024-04-25T10:18:57.237044+0000","last_became_active":"2024-04-25T10:18:55.226402+0000","last_became_peered":"2024-04-25T10:18:55.226402+0000","last_unstale":"2024-04-25T10:18:57.237044+0000","last_undegraded":"2024-04-25T10:18:57.237044+0000","last_fullsized":"2024-04-25T10:18:57.237044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:50.978256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238622+0000","last_change":"2024-04-25T10:18:56.243829+0000","last_active":"2024-04-25T10:18:57.238622+0000","last_peered":"2024-04-25T10:18:57.238622+0000","last_clean":"2024-04-25T10:18:57.238622+0000","last_became_active":"2024-04-25T10:18:56.243579+0000","last_became_peered":"2024-04-25T10:18:56.243579+0000","last_unstale":"2024-04-25T10:18:57.238622+0000","last_undegraded":"2024-04-25T10:18:57.238622+0000","last_fullsized":"2024-04-25T10:18:57.238622+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:28.140873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099156+0000","last_change":"2024-04-25T10:18:55.237736+0000","last_active":"2024-04-25T10:18:58.099156+0000","last_peered":"2024-04-25T10:18:58.099156+0000","last_clean":"2024-04-25T10:18:58.099156+0000","last_became_active":"2024-04-25T10:18:55.237487+0000","last_became_peered":"2024-04-25T10:18:55.237487+0000","last_unstale":"2024-04-25T10:18:58.099156+0000","last_undegraded":"2024-04-25T10:18:58.099156+0000","last_fullsized":"2024-04-25T10:18:58.099156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814574+0000","last_change":"2024-04-25T10:18:56.248910+0000","last_active":"2024-04-25T10:18:57.814574+0000","last_peered":"2024-04-25T10:18:57.814574+0000","last_clean":"2024-04-25T10:18:57.814574+0000","last_became_active":"2024-04-25T10:18:56.246574+0000","last_became_peered":"2024-04-25T10:18:56.246574+0000","last_unstale":"2024-04-25T10:18:57.814574+0000","last_undegraded":"2024-04-25T10:18:57.814574+0000","last_fullsized":"2024-04-25T10:18:57.814574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:26.300653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237128+0000","last_change":"2024-04-25T10:18:55.236067+0000","last_active":"2024-04-25T10:18:57.237128+0000","last_peered":"2024-04-25T10:18:57.237128+0000","last_clean":"2024-04-25T10:18:57.237128+0000","last_became_active":"2024-04-25T10:18:55.235652+0000","last_became_peered":"2024-04-25T10:18:55.235652+0000","last_unstale":"2024-04-25T10:18:57.237128+0000","last_undegraded":"2024-04-25T10:18:57.237128+0000","last_fullsized":"2024-04-25T10:18:57.237128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:45.658529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237089+0000","last_change":"2024-04-25T10:18:56.241808+0000","last_active":"2024-04-25T10:18:57.237089+0000","last_peered":"2024-04-25T10:18:57.237089+0000","last_clean":"2024-04-25T10:18:57.237089+0000","last_became_active":"2024-04-25T10:18:56.241407+0000","last_became_peered":"2024-04-25T10:18:56.241407+0000","last_unstale":"2024-04-25T10:18:57.237089+0000","last_undegraded":"2024-04-25T10:18:57.237089+0000","last_fullsized":"2024-04-25T10:18:57.237089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098721+0000","last_change":"2024-04-25T10:18:55.240208+0000","last_active":"2024-04-25T10:18:58.098721+0000","last_peered":"2024-04-25T10:18:58.098721+0000","last_clean":"2024-04-25T10:18:58.098721+0000","last_became_active":"2024-04-25T10:18:55.239948+0000","last_became_peered":"2024-04-25T10:18:55.239948+0000","last_unstale":"2024-04-25T10:18:58.098721+0000","last_undegraded":"2024-04-25T10:18:58.098721+0000","last_fullsized":"2024-04-25T10:18:58.098721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:19.512576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814473+0000","last_change":"2024-04-25T10:18:56.243705+0000","last_active":"2024-04-25T10:18:57.814473+0000","last_peered":"2024-04-25T10:18:57.814473+0000","last_clean":"2024-04-25T10:18:57.814473+0000","last_became_active":"2024-04-25T10:18:56.241679+0000","last_became_peered":"2024-04-25T10:18:56.241679+0000","last_unstale":"2024-04-25T10:18:57.814473+0000","last_undegraded":"2024-04-25T10:18:57.814473+0000","last_fullsized":"2024-04-25T10:18:57.814473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.086557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532490+0000","last_change":"2024-04-25T10:18:55.233741+0000","last_active":"2024-04-25T10:18:57.532490+0000","last_peered":"2024-04-25T10:18:57.532490+0000","last_clean":"2024-04-25T10:18:57.532490+0000","last_became_active":"2024-04-25T10:18:55.228199+0000","last_became_peered":"2024-04-25T10:18:55.228199+0000","last_unstale":"2024-04-25T10:18:57.532490+0000","last_undegraded":"2024-04-25T10:18:57.532490+0000","last_fullsized":"2024-04-25T10:18:57.532490+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:49.605460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532541+0000","last_change":"2024-04-25T10:18:56.242696+0000","last_active":"2024-04-25T10:18:57.532541+0000","last_peered":"2024-04-25T10:18:57.532541+0000","last_clean":"2024-04-25T10:18:57.532541+0000","last_became_active":"2024-04-25T10:18:56.242407+0000","last_became_peered":"2024-04-25T10:18:56.242407+0000","last_unstale":"2024-04-25T10:18:57.532541+0000","last_undegraded":"2024-04-25T10:18:57.532541+0000","last_fullsized":"2024-04-25T10:18:57.532541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.025032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236550+0000","last_change":"2024-04-25T10:18:55.231050+0000","last_active":"2024-04-25T10:18:57.236550+0000","last_peered":"2024-04-25T10:18:57.236550+0000","last_clean":"2024-04-25T10:18:57.236550+0000","last_became_active":"2024-04-25T10:18:55.230645+0000","last_became_peered":"2024-04-25T10:18:55.230645+0000","last_unstale":"2024-04-25T10:18:57.236550+0000","last_undegraded":"2024-04-25T10:18:57.236550+0000","last_fullsized":"2024-04-25T10:18:57.236550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:25.294394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238881+0000","last_change":"2024-04-25T10:18:56.242781+0000","last_active":"2024-04-25T10:18:57.238881+0000","last_peered":"2024-04-25T10:18:57.238881+0000","last_clean":"2024-04-25T10:18:57.238881+0000","last_became_active":"2024-04-25T10:18:56.242495+0000","last_became_peered":"2024-04-25T10:18:56.242495+0000","last_unstale":"2024-04-25T10:18:57.238881+0000","last_undegraded":"2024-04-25T10:18:57.238881+0000","last_fullsized":"2024-04-25T10:18:57.238881+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:05.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814954+0000","last_change":"2024-04-25T10:18:55.238477+0000","last_active":"2024-04-25T10:18:57.814954+0000","last_peered":"2024-04-25T10:18:57.814954+0000","last_clean":"2024-04-25T10:18:57.814954+0000","last_became_active":"2024-04-25T10:18:55.238126+0000","last_became_peered":"2024-04-25T10:18:55.238126+0000","last_unstale":"2024-04-25T10:18:57.814954+0000","last_undegraded":"2024-04-25T10:18:57.814954+0000","last_fullsized":"2024-04-25T10:18:57.814954+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:37.997874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.236882+0000","last_change":"2024-04-25T10:18:56.249055+0000","last_active":"2024-04-25T10:18:57.236882+0000","last_peered":"2024-04-25T10:18:57.236882+0000","last_clean":"2024-04-25T10:18:57.236882+0000","last_became_active":"2024-04-25T10:18:56.248708+0000","last_became_peered":"2024-04-25T10:18:56.248708+0000","last_unstale":"2024-04-25T10:18:57.236882+0000","last_undegraded":"2024-04-25T10:18:57.236882+0000","last_fullsized":"2024-04-25T10:18:57.236882+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:26.665895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.226720+0000","last_change":"2024-04-25T10:18:55.236135+0000","last_active":"2024-04-25T10:18:58.226720+0000","last_peered":"2024-04-25T10:18:58.226720+0000","last_clean":"2024-04-25T10:18:58.226720+0000","last_became_active":"2024-04-25T10:18:55.235788+0000","last_became_peered":"2024-04-25T10:18:55.235788+0000","last_unstale":"2024-04-25T10:18:58.226720+0000","last_undegraded":"2024-04-25T10:18:58.226720+0000","last_fullsized":"2024-04-25T10:18:58.226720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:57.667019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.238907+0000","last_change":"2024-04-25T10:18:56.251890+0000","last_active":"2024-04-25T10:18:57.238907+0000","last_peered":"2024-04-25T10:18:57.238907+0000","last_clean":"2024-04-25T10:18:57.238907+0000","last_became_active":"2024-04-25T10:18:56.251588+0000","last_became_peered":"2024-04-25T10:18:56.251588+0000","last_unstale":"2024-04-25T10:18:57.238907+0000","last_undegraded":"2024-04-25T10:18:57.238907+0000","last_fullsized":"2024-04-25T10:18:57.238907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:58.673279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.815004+0000","last_change":"2024-04-25T10:18:55.236899+0000","last_active":"2024-04-25T10:18:57.815004+0000","last_peered":"2024-04-25T10:18:57.815004+0000","last_clean":"2024-04-25T10:18:57.815004+0000","last_became_active":"2024-04-25T10:18:55.236580+0000","last_became_peered":"2024-04-25T10:18:55.236580+0000","last_unstale":"2024-04-25T10:18:57.815004+0000","last_undegraded":"2024-04-25T10:18:57.815004+0000","last_fullsized":"2024-04-25T10:18:57.815004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:21.018773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099216+0000","last_change":"2024-04-25T10:18:56.235360+0000","last_active":"2024-04-25T10:18:58.099216+0000","last_peered":"2024-04-25T10:18:58.099216+0000","last_clean":"2024-04-25T10:18:58.099216+0000","last_became_active":"2024-04-25T10:18:56.235089+0000","last_became_peered":"2024-04-25T10:18:56.235089+0000","last_unstale":"2024-04-25T10:18:58.099216+0000","last_undegraded":"2024-04-25T10:18:58.099216+0000","last_fullsized":"2024-04-25T10:18:58.099216+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.903865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814753+0000","last_change":"2024-04-25T10:18:55.238591+0000","last_active":"2024-04-25T10:18:57.814753+0000","last_peered":"2024-04-25T10:18:57.814753+0000","last_clean":"2024-04-25T10:18:57.814753+0000","last_became_active":"2024-04-25T10:18:55.238176+0000","last_became_peered":"2024-04-25T10:18:55.238176+0000","last_unstale":"2024-04-25T10:18:57.814753+0000","last_undegraded":"2024-04-25T10:18:57.814753+0000","last_fullsized":"2024-04-25T10:18:57.814753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:02.604590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097709+0000","last_change":"2024-04-25T10:18:56.239706+0000","last_active":"2024-04-25T10:18:58.097709+0000","last_peered":"2024-04-25T10:18:58.097709+0000","last_clean":"2024-04-25T10:18:58.097709+0000","last_became_active":"2024-04-25T10:18:56.239021+0000","last_became_peered":"2024-04-25T10:18:56.239021+0000","last_unstale":"2024-04-25T10:18:58.097709+0000","last_undegraded":"2024-04-25T10:18:58.097709+0000","last_fullsized":"2024-04-25T10:18:58.097709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:32.913719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.814556+0000","last_change":"2024-04-25T10:18:55.242723+0000","last_active":"2024-04-25T10:18:57.814556+0000","last_peered":"2024-04-25T10:18:57.814556+0000","last_clean":"2024-04-25T10:18:57.814556+0000","last_became_active":"2024-04-25T10:18:55.242338+0000","last_became_peered":"2024-04-25T10:18:55.242338+0000","last_unstale":"2024-04-25T10:18:57.814556+0000","last_undegraded":"2024-04-25T10:18:57.814556+0000","last_fullsized":"2024-04-25T10:18:57.814556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:03.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097338+0000","last_change":"2024-04-25T10:18:56.234182+0000","last_active":"2024-04-25T10:18:58.097338+0000","last_peered":"2024-04-25T10:18:58.097338+0000","last_clean":"2024-04-25T10:18:58.097338+0000","last_became_active":"2024-04-25T10:18:56.233910+0000","last_became_peered":"2024-04-25T10:18:56.233910+0000","last_unstale":"2024-04-25T10:18:58.097338+0000","last_undegraded":"2024-04-25T10:18:58.097338+0000","last_fullsized":"2024-04-25T10:18:58.097338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:08.780462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.098781+0000","last_change":"2024-04-25T10:18:55.241653+0000","last_active":"2024-04-25T10:18:58.098781+0000","last_peered":"2024-04-25T10:18:58.098781+0000","last_clean":"2024-04-25T10:18:58.098781+0000","last_became_active":"2024-04-25T10:18:55.239465+0000","last_became_peered":"2024-04-25T10:18:55.239465+0000","last_unstale":"2024-04-25T10:18:58.098781+0000","last_undegraded":"2024-04-25T10:18:58.098781+0000","last_fullsized":"2024-04-25T10:18:58.098781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:03.210836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237037+0000","last_change":"2024-04-25T10:18:56.250780+0000","last_active":"2024-04-25T10:18:57.237037+0000","last_peered":"2024-04-25T10:18:57.237037+0000","last_clean":"2024-04-25T10:18:57.237037+0000","last_became_active":"2024-04-25T10:18:56.250540+0000","last_became_peered":"2024-04-25T10:18:56.250540+0000","last_unstale":"2024-04-25T10:18:57.237037+0000","last_undegraded":"2024-04-25T10:18:57.237037+0000","last_fullsized":"2024-04-25T10:18:57.237037+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:08.656466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.097273+0000","last_change":"2024-04-25T10:18:55.231867+0000","last_active":"2024-04-25T10:18:58.097273+0000","last_peered":"2024-04-25T10:18:58.097273+0000","last_clean":"2024-04-25T10:18:58.097273+0000","last_became_active":"2024-04-25T10:18:55.231431+0000","last_became_peered":"2024-04-25T10:18:55.231431+0000","last_unstale":"2024-04-25T10:18:58.097273+0000","last_undegraded":"2024-04-25T10:18:58.097273+0000","last_fullsized":"2024-04-25T10:18:58.097273+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:14.615886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:58.099151+0000","last_change":"2024-04-25T10:18:56.234895+0000","last_active":"2024-04-25T10:18:58.099151+0000","last_peered":"2024-04-25T10:18:58.099151+0000","last_clean":"2024-04-25T10:18:58.099151+0000","last_became_active":"2024-04-25T10:18:56.234511+0000","last_became_peered":"2024-04-25T10:18:56.234511+0000","last_unstale":"2024-04-25T10:18:58.099151+0000","last_undegraded":"2024-04-25T10:18:58.099151+0000","last_fullsized":"2024-04-25T10:18:58.099151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:55.208844+0000","last_clean_scrub_stamp":"2024-04-25T10:18:55.208844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.704387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.237143+0000","last_change":"2024-04-25T10:18:55.225543+0000","last_active":"2024-04-25T10:18:57.237143+0000","last_peered":"2024-04-25T10:18:57.237143+0000","last_clean":"2024-04-25T10:18:57.237143+0000","last_became_active":"2024-04-25T10:18:55.225225+0000","last_became_peered":"2024-04-25T10:18:55.225225+0000","last_unstale":"2024-04-25T10:18:57.237143+0000","last_undegraded":"2024-04-25T10:18:57.237143+0000","last_fullsized":"2024-04-25T10:18:57.237143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:54.202585+0000","last_clean_scrub_stamp":"2024-04-25T10:18:54.202585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:59.193894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":194,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-25T10:18:57.532479+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:18:57.532479+0000","last_peered":"2024-04-25T10:18:57.532479+0000","last_clean":"2024-04-25T10:18:57.532479+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:18:57.532479+0000","last_undegraded":"2024-04-25T10:18:57.532479+0000","last_fullsized":"2024-04-25T10:18:57.532479+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":393216,"data_stored":151494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":153084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":742422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":405504,"data_stored":152248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":741910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":393216,"data_stored":151562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":153106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:05.274 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:19:05.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:19:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.456+0000 7f1ac7e00640 1 Processor -- start 2024-04-25T10:19:05.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.456+0000 7f1ac7e00640 1 -- start start 2024-04-25T10:19:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ac80d1510 0x7f1ac80b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ac80cfbb0 0x7f1ac80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d18f0 0x7f1ac806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80d0170 con 0x7f1ac80d1510 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80d03e0 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80d06c0 con 0x7f1ac80d18f0 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ac80d1510 0x7f1ac80b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ac80d1510 0x7f1ac80b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50078/0 (socket says 172.21.15.45:50078) 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 -- 172.21.15.45:0/2605163031 learned_addr learned my addr 172.21.15.45:0/2605163031 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac7400640 1 --2- 172.21.15.45:0/2605163031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d18f0 0x7f1ac806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1abe000640 1 --2- 172.21.15.45:0/2605163031 >> v2:172.21.15.117:3300/0 conn(0x7f1ac80cfbb0 0x7f1ac80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 -- 172.21.15.45:0/2605163031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d18f0 msgr2=0x7f1ac806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/2605163031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d18f0 0x7f1ac806bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 -- 172.21.15.45:0/2605163031 >> v2:172.21.15.117:3300/0 conn(0x7f1ac80cfbb0 msgr2=0x7f1ac80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/2605163031 >> v2:172.21.15.117:3300/0 conn(0x7f1ac80cfbb0 0x7f1ac80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.460+0000 7f1ac6a00640 1 -- 172.21.15.45:0/2605163031 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac806b5f0 con 0x7f1ac80d1510 2024-04-25T10:19:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/2605163031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80d1510 0x7f1ac80b1450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80d11c0 tx=0x7f1ab801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560bdd3271a026f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac6000640 1 -- 172.21.15.45:0/2605163031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80a5860 con 0x7f1ac80d1510 2024-04-25T10:19:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac6000640 1 -- 172.21.15.45:0/2605163031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab80a5a00 con 0x7f1ac80d1510 2024-04-25T10:19:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac6000640 1 -- 172.21.15.45:0/2605163031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80a5dd0 con 0x7f1ac80d1510 2024-04-25T10:19:05.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- 172.21.15.45:0/2605163031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80d1510 msgr2=0x7f1ac80b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/2605163031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80d1510 0x7f1ac80b1450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80d11c0 tx=0x7f1ab801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- 172.21.15.45:0/2605163031 shutdown_connections 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/2605163031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d18f0 0x7f1ac806bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/2605163031 >> v2:172.21.15.117:3300/0 conn(0x7f1ac80cfbb0 0x7f1ac80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/2605163031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80d1510 0x7f1ac80b1450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- 172.21.15.45:0/2605163031 >> 172.21.15.45:0/2605163031 conn(0x7f1ac8072820 msgr2=0x7f1ac809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- 172.21.15.45:0/2605163031 shutdown_connections 2024-04-25T10:19:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- 172.21.15.45:0/2605163031 wait complete. 2024-04-25T10:19:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 Processor -- start 2024-04-25T10:19:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- start start 2024-04-25T10:19:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 0x7f1ac8160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d1510 0x7f1ac814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ac80d18f0 0x7f1ac80c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80b28c0 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80b2740 con 0x7f1ac80d18f0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.464+0000 7f1ac7e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80d1d00 con 0x7f1ac80d1510 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 0x7f1ac8160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 0x7f1ac8160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50094/0 (socket says 172.21.15.45:50094) 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 -- 172.21.15.45:0/306077346 learned_addr learned my addr 172.21.15.45:0/306077346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abe000640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d1510 0x7f1ac814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac7400640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ac80d18f0 0x7f1ac80c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 -- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d1510 msgr2=0x7f1ac814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d1510 0x7f1ac814f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 -- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ac80d18f0 msgr2=0x7f1ac80c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ac80d18f0 0x7f1ac80c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 -- 172.21.15.45:0/306077346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80c60c0 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 0x7f1ac8160870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80af040 tx=0x7f1ab8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80bd020 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab80b86b0 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80b8aa0 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac815eb90 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac815f150 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8075120 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ab80c0070 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abf400640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aa4066f60 0x7f1aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.468+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ab811f890 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.472+0000 7f1abe000640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aa4066f60 0x7f1aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.472+0000 7f1abe000640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aa4066f60 0x7f1aa4069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ac815dff0 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.472+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ab80165e0 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.692+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1ac80d2690 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.696+0000 7f1abf400640 1 -- 172.21.15.45:0/306077346 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1ab80f0800 con 0x7f1ac80cfbb0 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:19:05.700 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aa4066f60 msgr2=0x7f1aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aa4066f60 0x7f1aa4069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1ac815dff0 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 msgr2=0x7f1ac8160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 0x7f1ac8160870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80af040 tx=0x7f1ab8019760 comp rx=0 tx=0).stop 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 shutdown_connections 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aa4066f60 0x7f1aa4069420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ac80d18f0 0x7f1ac80c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ac80d1510 0x7f1ac814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/306077346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ac80cfbb0 0x7f1ac8160870 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 >> 172.21.15.45:0/306077346 conn(0x7f1ac8072820 msgr2=0x7f1ac80b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 shutdown_connections 2024-04-25T10:19:05.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.700+0000 7f1ac7e00640 1 -- 172.21.15.45:0/306077346 wait complete. 2024-04-25T10:19:05.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 Processor -- start 2024-04-25T10:19:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 -- start start 2024-04-25T10:19:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c15bdd0 0x7f5c6c0d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c6c0cf6f0 0x7f5c6c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5c6c0d0430 0x7f5c6c0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0ae5a0 con 0x7f5c6c0cf6f0 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c15c1b0 con 0x7f5c6c0d0430 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.860+0000 7f5c73800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b1c40 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c6c0cf6f0 0x7f5c6c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c6c0cf6f0 0x7f5c6c0cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50104/0 (socket says 172.21.15.45:50104) 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 -- 172.21.15.45:0/745541477 learned_addr learned my addr 172.21.15.45:0/745541477 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c70e00640 1 --2- 172.21.15.45:0/745541477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c15bdd0 0x7f5c6c0d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c71800640 1 --2- 172.21.15.45:0/745541477 >> v2:172.21.15.117:3300/0 conn(0x7f5c6c0d0430 0x7f5c6c0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 -- 172.21.15.45:0/745541477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c15bdd0 msgr2=0x7f5c6c0d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 --2- 172.21.15.45:0/745541477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c15bdd0 0x7f5c6c0d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 -- 172.21.15.45:0/745541477 >> v2:172.21.15.117:3300/0 conn(0x7f5c6c0d0430 msgr2=0x7f5c6c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 --2- 172.21.15.45:0/745541477 >> v2:172.21.15.117:3300/0 conn(0x7f5c6c0d0430 0x7f5c6c0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 -- 172.21.15.45:0/745541477 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0b1310 con 0x7f5c6c0cf6f0 2024-04-25T10:19:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c63e00640 1 --2- 172.21.15.45:0/745541477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c0cf6f0 0x7f5c6c0cfef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c072910 tx=0x7f5c5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e277fc64dced9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.864+0000 7f5c6be00640 1 -- 172.21.15.45:0/745541477 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c540b1070 con 0x7f5c6c0cf6f0 2024-04-25T10:19:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c6be00640 1 -- 172.21.15.45:0/745541477 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c540ac630 con 0x7f5c6c0cf6f0 2024-04-25T10:19:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c6be00640 1 -- 172.21.15.45:0/745541477 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c540a8050 con 0x7f5c6c0cf6f0 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- 172.21.15.45:0/745541477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c0cf6f0 msgr2=0x7f5c6c0cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- 172.21.15.45:0/745541477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c0cf6f0 0x7f5c6c0cfef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c072910 tx=0x7f5c5401c520 comp rx=0 tx=0).stop 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- 172.21.15.45:0/745541477 shutdown_connections 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- 172.21.15.45:0/745541477 >> v2:172.21.15.117:3300/0 conn(0x7f5c6c0d0430 0x7f5c6c0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- 172.21.15.45:0/745541477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c0cf6f0 0x7f5c6c0cfef0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- 172.21.15.45:0/745541477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c15bdd0 0x7f5c6c0d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- 172.21.15.45:0/745541477 >> 172.21.15.45:0/745541477 conn(0x7f5c6c0d1910 msgr2=0x7f5c6c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- 172.21.15.45:0/745541477 shutdown_connections 2024-04-25T10:19:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- 172.21.15.45:0/745541477 wait complete. 2024-04-25T10:19:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 Processor -- start 2024-04-25T10:19:05.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- start start 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c0cf6f0 0x7f5c6c150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c6c0d0430 0x7f5c6c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 0x7f5c6c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b20b0 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0cd860 con 0x7f5c6c0d0430 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c73800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b2230 con 0x7f5c6c0cf6f0 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c71800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 0x7f5c6c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c71800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 0x7f5c6c15d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50110/0 (socket says 172.21.15.45:50110) 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c71800640 1 -- 172.21.15.45:0/2920301689 learned_addr learned my addr 172.21.15.45:0/2920301689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c70e00640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c0cf6f0 0x7f5c6c150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c63e00640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c6c0d0430 0x7f5c6c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c71800640 1 -- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c0cf6f0 msgr2=0x7f5c6c150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.868+0000 7f5c71800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c0cf6f0 0x7f5c6c150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c71800640 1 -- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c6c0d0430 msgr2=0x7f5c6c0c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c71800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c6c0d0430 0x7f5c6c0c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c71800640 1 -- 172.21.15.45:0/2920301689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c150c50 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c71800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 0x7f5c6c15d380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e850 tx=0x7f5c6400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64017070 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400c8e0 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400ccb0 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c151210 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c1517d0 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c04ef30 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c6401a070 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.872+0000 7f5c69600640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c48066f60 0x7f5c48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.876+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c64084fd0 con 0x7f5c6c15bdd0 2024-04-25T10:19:05.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.876+0000 7f5c70e00640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c48066f60 0x7f5c48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.876+0000 7f5c70e00640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c48066f60 0x7f5c48069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c1524d0 tx=0x7f5c5c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:05.880+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c6c150c50 con 0x7f5c6c15bdd0 2024-04-25T10:19:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c6c158a40 con 0x7f5c6c15bdd0 2024-04-25T10:19:06.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c69600640 1 -- 172.21.15.45:0/2920301689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f5c64055f40 con 0x7f5c6c15bdd0 2024-04-25T10:19:06.079 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:19:06.079 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:55.618016+0000","modified":"2024-04-25T10:18:58.627157+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3482819589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3482819589},{"type":"v1","addr":"172.21.15.45:6835","nonce":3482819589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:19:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-04-25T10:19:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c48066f60 msgr2=0x7f5c48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c48066f60 0x7f5c48069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c1524d0 tx=0x7f5c5c00b040 comp rx=0 tx=0).stop 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 msgr2=0x7f5c6c15d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 0x7f5c6c15d380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400e850 tx=0x7f5c6400ed10 comp rx=0 tx=0).stop 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 shutdown_connections 2024-04-25T10:19:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c48066f60 0x7f5c48069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c6c15bdd0 0x7f5c6c15d380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c6c0d0430 0x7f5c6c0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 --2- 172.21.15.45:0/2920301689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c6c0cf6f0 0x7f5c6c150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 >> 172.21.15.45:0/2920301689 conn(0x7f5c6c0d1910 msgr2=0x7f5c6c06fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 shutdown_connections 2024-04-25T10:19:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.076+0000 7f5c73800640 1 -- 172.21.15.45:0/2920301689 wait complete. 2024-04-25T10:19:06.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:18:55.618016+0000', 'modified': '2024-04-25T10:18:58.627157+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5055}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5055': {'gid': 5055, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/3482819589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 3482819589}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3482819589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:19:06.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:19:06.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:19:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.640+0000 7f3de5c00640 1 Processor -- start 2024-04-25T10:19:06.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3de5c00640 1 -- start start 2024-04-25T10:19:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3de5c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3de00d0040 0x7f3de006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3de5c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00b2770 con 0x7f3de015bc40 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3de5c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de006c670 con 0x7f3de00cda10 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3de5c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de006c950 con 0x7f3de00d0040 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3ddf400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3de00d0040 0x7f3de006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3ddea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3de015bc40 0x7f3de00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48142/0 (socket says 172.21.15.45:48142) 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3ddea00640 1 -- 172.21.15.45:0/3989192177 learned_addr learned my addr 172.21.15.45:0/3989192177 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3ddf400640 1 -- 172.21.15.45:0/3989192177 >> v1:172.21.15.117:6789/0 conn(0x7f3de00cda10 legacy=0x7f3de00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3ddf400640 1 -- 172.21.15.45:0/3989192177 >> v1:172.21.15.45:6789/0 conn(0x7f3de015bc40 legacy=0x7f3de00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.644+0000 7f3ddf400640 1 -- 172.21.15.45:0/3989192177 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de006cc30 con 0x7f3de00d0040 2024-04-25T10:19:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3ddf400640 1 --2- 172.21.15.45:0/3989192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00d0040 0x7f3de006c090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80097c0 tx=0x7f3dd801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d189314545e23603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3dde000640 1 -- 172.21.15.45:0/3989192177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80b0070 con 0x7f3de00d0040 2024-04-25T10:19:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3dde000640 1 -- 172.21.15.45:0/3989192177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd80ab600 con 0x7f3de00d0040 2024-04-25T10:19:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3dde000640 1 -- 172.21.15.45:0/3989192177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80ab9d0 con 0x7f3de00d0040 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 -- 172.21.15.45:0/3989192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00d0040 msgr2=0x7f3de006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 --2- 172.21.15.45:0/3989192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00d0040 0x7f3de006c090 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80097c0 tx=0x7f3dd801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 -- 172.21.15.45:0/3989192177 shutdown_connections 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 --2- 172.21.15.45:0/3989192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00d0040 0x7f3de006c090 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 -- 172.21.15.45:0/3989192177 >> 172.21.15.45:0/3989192177 conn(0x7f3de0066b70 msgr2=0x7f3de0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 -- 172.21.15.45:0/3989192177 shutdown_connections 2024-04-25T10:19:06.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 -- 172.21.15.45:0/3989192177 wait complete. 2024-04-25T10:19:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 Processor -- start 2024-04-25T10:19:06.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.648+0000 7f3de5c00640 1 -- start start 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 0x7f3de0144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de00d0040 0x7f3de015e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de015bc40 0x7f3de0070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0169980 con 0x7f3de00cda10 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0169800 con 0x7f3de015bc40 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0169b00 con 0x7f3de00d0040 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 0x7f3de0144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 0x7f3de0144280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50130/0 (socket says 172.21.15.45:50130) 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 -- 172.21.15.45:0/1859530120 learned_addr learned my addr 172.21.15.45:0/1859530120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de015bc40 0x7f3de0070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3dd6000640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de00d0040 0x7f3de015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 -- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de00d0040 msgr2=0x7f3de015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de00d0040 0x7f3de015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 -- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de015bc40 msgr2=0x7f3de0070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de015bc40 0x7f3de0070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 -- 172.21.15.45:0/1859530120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0071470 con 0x7f3de00cda10 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de015bc40 0x7f3de0070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3ddea00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 0x7f3de0144280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c3a0 tx=0x7f3dd000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3dd7400640 1 -- 172.21.15.45:0/1859530120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0017070 con 0x7f3de00cda10 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3dd7400640 1 -- 172.21.15.45:0/1859530120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0005cf0 con 0x7f3de00cda10 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3dd7400640 1 -- 172.21.15.45:0/1859530120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00072f0 con 0x7f3de00cda10 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de0071750 con 0x7f3de00cda10 2024-04-25T10:19:06.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de0071d10 con 0x7f3de00cda10 2024-04-25T10:19:06.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.652+0000 7f3dd7400640 1 -- 172.21.15.45:0/1859530120 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3dd001a070 con 0x7f3de00cda10 2024-04-25T10:19:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dd7400640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3dbc066f60 0x7f3dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dd7400640 1 -- 172.21.15.45:0/1859530120 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dd0084500 con 0x7f3de00cda10 2024-04-25T10:19:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 Processor -- start 2024-04-25T10:19:06.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 -- start start 2024-04-25T10:19:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dd6000640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3dbc066f60 0x7f3dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dd6000640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3dbc066f60 0x7f3dbc069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3de00b13f0 tx=0x7f3dc8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 0x55b7745b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b77456b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b77456bf30 0x55b77456c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7744535f0 con 0x55b7745b5920 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7743bada0 con 0x55b7745b6240 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b774343150 con 0x55b77456bf30 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 0x55b7745b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 0x55b7745b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50146/0 (socket says 172.21.15.45:50146) 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 -- 172.21.15.45:0/1443185483 learned_addr learned my addr 172.21.15.45:0/1443185483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dd6000640 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b77456b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b77456bf30 0x55b77456c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 -- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b77456bf30 msgr2=0x55b77456c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b77456bf30 0x55b77456c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 -- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 msgr2=0x55b77456b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b77456b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 -- 172.21.15.45:0/1443185483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b77435cca0 con 0x55b7745b5920 2024-04-25T10:19:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b77456bf30 0x55b77456c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3ddea00640 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 0x55b7745b5d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0051140 tx=0x7f3dd0054be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5959c396c3452c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dcf400640 1 -- 172.21.15.45:0/1443185483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd005d480 con 0x55b7745b5920 2024-04-25T10:19:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dcf400640 1 -- 172.21.15.45:0/1443185483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0052d80 con 0x55b7745b5920 2024-04-25T10:19:06.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3dcf400640 1 -- 172.21.15.45:0/1443185483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0051350 con 0x55b7745b5920 2024-04-25T10:19:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 -- 172.21.15.45:0/1443185483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 msgr2=0x55b7745b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.656+0000 7f3de90db000 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 0x55b7745b5d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0051140 tx=0x7f3dd0054be0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- 172.21.15.45:0/1443185483 shutdown_connections 2024-04-25T10:19:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b77456bf30 0x55b77456c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b77456b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 --2- 172.21.15.45:0/1443185483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b5920 0x55b7745b5d00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- 172.21.15.45:0/1443185483 >> 172.21.15.45:0/1443185483 conn(0x55b7745722a0 msgr2=0x55b77456a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- 172.21.15.45:0/1443185483 shutdown_connections 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- 172.21.15.45:0/1443185483 wait complete. 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- start start 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 Processor -- start 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b7745bce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b77456faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b774375990 con 0x55b7745b6240 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7743b6e40 con 0x55b77456bf30 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b774346900 con 0x55b7745b5920 2024-04-25T10:19:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 10 client.0 _ll_register_callbacks cb 0x55b77446e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b77456faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b77456faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50154/0 (socket says 172.21.15.45:50154) 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b7745bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 -- 172.21.15.45:0/769226504 learned_addr learned my addr 172.21.15.45:0/769226504 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dd6000640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 -- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 msgr2=0x55b7745bd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745bd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 -- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 msgr2=0x55b7745bce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b7745bce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 -- 172.21.15.45:0/769226504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b774373630 con 0x55b7745b6240 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddea00640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b7745bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3ddf400640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b77456faa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80097c0 tx=0x7f3dd801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c82e8bbdb12884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:19:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dc3e00640 1 -- 172.21.15.45:0/769226504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80bb070 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd0052d80 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dc3e00640 1 -- 172.21.15.45:0/769226504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dd80abc40 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dc3e00640 1 -- 172.21.15.45:0/769226504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd80c6050 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 10 client.5262 fetch_fsmap learned FSMap version 17 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b7743433a0 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dc3e00640 1 -- 172.21.15.45:0/769226504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0052d80 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.660+0000 7f3dc3e00640 1 -- 172.21.15.45:0/769226504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3dd80c9020 con 0x55b7745b6240 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 10 client.5262 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 10 client.5262 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 20 client.5262 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745704b0 0x55b774570890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 4 client.5262 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b774356a50 con 0x55b7745704b0 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3ddea00640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745704b0 0x55b774570890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3ddea00640 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745704b0 0x55b774570890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3aebccc7b2291d3 server_cookie=4eb603226438e6cf in_seq=0 out_seq=0 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3dc3e00640 10 client.5262 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3dc3e00640 1 -- 172.21.15.45:0/769226504 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b774356a50 con 0x55b7745704b0 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3dc3e00640 10 client.5262 handle_command_reply: tid=0 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 client.5262 shutdown 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 msgr2=0x55b77456faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b77456faa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80097c0 tx=0x7f3dd801ac10 comp rx=0 tx=0).stop 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 shutdown_connections 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745704b0 0x55b774570890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b77456faa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745bd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3dc3e00640 0 client.5262 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 --2- 172.21.15.45:0/769226504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b7745bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 >> 172.21.15.45:0/769226504 conn(0x55b7745722a0 msgr2=0x55b77456a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 shutdown_connections 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 1 -- 172.21.15.45:0/769226504 wait complete. 2024-04-25T10:19:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.664+0000 7f3de90db000 20 client.5262 trim_cache size 0 max 16384 2024-04-25T10:19:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 Processor -- start 2024-04-25T10:19:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 -- start start 2024-04-25T10:19:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b77456f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b77456f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b7745c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de0156eb0 con 0x55b7745b6240 2024-04-25T10:19:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015dec0 con 0x55b77456bf30 2024-04-25T10:19:06.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01568b0 con 0x55b7745b5920 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b7745c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b7745c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50166/0 (socket says 172.21.15.45:50166) 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 -- 172.21.15.45:0/1374977296 learned_addr learned my addr 172.21.15.45:0/1374977296 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 -- 172.21.15.45:0/1374977296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 msgr2=0x55b77456f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b77456f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 -- 172.21.15.45:0/1374977296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 msgr2=0x55b77456f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b77456f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 -- 172.21.15.45:0/1374977296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7745b6070 con 0x55b7745b6240 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3ddf400640 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b7745c8040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3dd801cc70 tx=0x7f3dd80b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc06f0cb673d91fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3dcea00640 1 -- 172.21.15.45:0/1374977296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80bb070 con 0x55b7745b6240 2024-04-25T10:19:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3dcea00640 1 -- 172.21.15.45:0/1374977296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd8000e50 con 0x55b7745b6240 2024-04-25T10:19:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.680+0000 7f3dcea00640 1 -- 172.21.15.45:0/1374977296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd80b0040 con 0x55b7745b6240 2024-04-25T10:19:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/1374977296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 msgr2=0x55b7745c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b7745c8040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3dd801cc70 tx=0x7f3dd80b5000 comp rx=0 tx=0).stop 2024-04-25T10:19:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/1374977296 shutdown_connections 2024-04-25T10:19:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7745b6240 0x55b7745c8040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b77456f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- 172.21.15.45:0/1374977296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b77456bf30 0x55b77456f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/1374977296 >> 172.21.15.45:0/1374977296 conn(0x55b7745722a0 msgr2=0x55b7745a2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/1374977296 shutdown_connections 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/1374977296 wait complete. 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- start start 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 Processor -- start 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 0x55b774615b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b7745cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7743bf3d0 con 0x55b77456bf30 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b774476d50 con 0x55b7745b6240 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b774355980 con 0x55b7745b5920 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 10 client.0 _ll_register_callbacks cb 0x7f3dd0005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 0x55b774615b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 0x55b774615b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50176/0 (socket says 172.21.15.45:50176) 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 -- 172.21.15.45:0/357990040 learned_addr learned my addr 172.21.15.45:0/357990040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddf400640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b7745cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3dd6000640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 -- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 msgr2=0x55b7745cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 -- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 msgr2=0x55b7745cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b7745cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 -- 172.21.15.45:0/357990040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b77443c600 con 0x55b77456bf30 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3dd6000640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3ddea00640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 0x55b774615b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0058ff0 tx=0x7f3dd0082860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d941c218735813ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3dc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3dc3e00640 1 -- 172.21.15.45:0/357990040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0082a10 con 0x55b77456bf30 2024-04-25T10:19:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3dc3e00640 1 -- 172.21.15.45:0/357990040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dd0084070 con 0x55b77456bf30 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd8000e50 con 0x55b77456bf30 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3dc3e00640 1 -- 172.21.15.45:0/357990040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd0084270 con 0x55b77456bf30 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 10 client.5268 fetch_fsmap learned FSMap version 17 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.684+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55b77434f9b0 con 0x55b77456bf30 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dc3e00640 1 -- 172.21.15.45:0/357990040 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd8000e50 con 0x55b77456bf30 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dc3e00640 1 -- 172.21.15.45:0/357990040 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3dd007d070 con 0x55b77456bf30 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 10 client.5268 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 10 client.5268 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 20 client.5268 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745d1580 0x55b7745d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 4 client.5268 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7743c82e0 con 0x55b7745d1580 2024-04-25T10:19:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dd6000640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745d1580 0x55b7745d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dd6000640 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745d1580 0x55b7745d1960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5cb8a7c34f30a3 server_cookie=2cf624cd73fc7d52 in_seq=0 out_seq=0 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dc3e00640 10 client.5268 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dc3e00640 1 -- 172.21.15.45:0/357990040 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b7743c82e0 con 0x55b7745d1580 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dc3e00640 10 client.5268 handle_command_reply: tid=0 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 client.5268 shutdown 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 msgr2=0x55b774615b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 0x55b774615b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0058ff0 tx=0x7f3dd0082860 comp rx=0 tx=0).stop 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 shutdown_connections 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55b7745d1580 0x55b7745d1960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b7745b6240 0x55b7745cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b7745b5920 0x55b7745cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3dc3e00640 0 client.5268 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 --2- 172.21.15.45:0/357990040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b77456bf30 0x55b774615b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 >> 172.21.15.45:0/357990040 conn(0x55b7745722a0 msgr2=0x55b77456e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 shutdown_connections 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 1 -- 172.21.15.45:0/357990040 wait complete. 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de90db000 20 client.5268 trim_cache size 0 max 16384 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:19:06.696 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5055, 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.472876152, 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 35, 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 35, 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 18.316072613999999 2024-04-25T10:19:06.697 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3dbc066f60 msgr2=0x7f3dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3dbc066f60 0x7f3dbc069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3de00b13f0 tx=0x7f3dc8009040 comp rx=0 tx=0).stop 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 msgr2=0x7f3de0144280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 0x7f3de0144280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c3a0 tx=0x7f3dd000c860 comp rx=0 tx=0).stop 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 shutdown_connections 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3dbc066f60 0x7f3dbc069420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de015bc40 0x7f3de0070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de00d0040 0x7f3de015e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 --2- 172.21.15.45:0/1859530120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3de00cda10 0x7f3de0144280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 >> 172.21.15.45:0/1859530120 conn(0x7f3de0066b70 msgr2=0x7f3de00cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 shutdown_connections 2024-04-25T10:19:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.688+0000 7f3de5c00640 1 -- 172.21.15.45:0/1859530120 wait complete. 2024-04-25T10:19:06.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:19:06.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.848+0000 7fd324a00640 1 Processor -- start 2024-04-25T10:19:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.848+0000 7fd324a00640 1 -- start start 2024-04-25T10:19:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd324a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 0x7fd3200cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd324a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3200ae020 0x7fd3200ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd324a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3200d0250 con 0x7fd320070c20 2024-04-25T10:19:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd324a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32015bb90 con 0x7fd3200ae020 2024-04-25T10:19:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd324a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200ad840 con 0x7fd3200cf930 2024-04-25T10:19:06.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31e800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3200ae020 0x7fd3200ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 0x7fd3200cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31e800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3200ae020 0x7fd3200ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52870/0 (socket says 172.21.15.45:52870) 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd317e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd320070c20 0x7fd32015e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:56328/0 (socket says 172.21.15.45:56328) 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 0x7fd3200cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:49704/0 (socket says 172.21.15.45:49704) 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31e800640 1 -- 172.21.15.45:0/1688651440 learned_addr learned my addr 172.21.15.45:0/1688651440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31de00640 1 -- 172.21.15.45:0/1688651440 >> v2:172.21.15.117:3300/0 conn(0x7fd3200ae020 msgr2=0x7fd3200ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31de00640 1 --2- 172.21.15.45:0/1688651440 >> v2:172.21.15.117:3300/0 conn(0x7fd3200ae020 0x7fd3200ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31de00640 1 -- 172.21.15.45:0/1688651440 >> v1:172.21.15.117:6789/0 conn(0x7fd320070c20 legacy=0x7fd32015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.852+0000 7fd31de00640 1 -- 172.21.15.45:0/1688651440 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32015e9f0 con 0x7fd3200cf930 2024-04-25T10:19:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd31de00640 1 --2- 172.21.15.45:0/1688651440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 0x7fd3200cfd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3100097c0 tx=0x7fd31001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b2a69598e36632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd31d400640 1 -- 172.21.15.45:0/1688651440 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3100a7070 con 0x7fd3200cf930 2024-04-25T10:19:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd31d400640 1 -- 172.21.15.45:0/1688651440 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3100b0070 con 0x7fd3200cf930 2024-04-25T10:19:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd31d400640 1 -- 172.21.15.45:0/1688651440 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3100ab890 con 0x7fd3200cf930 2024-04-25T10:19:06.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- 172.21.15.45:0/1688651440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 msgr2=0x7fd3200cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 --2- 172.21.15.45:0/1688651440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 0x7fd3200cfd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3100097c0 tx=0x7fd31001c470 comp rx=0 tx=0).stop 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- 172.21.15.45:0/1688651440 shutdown_connections 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 --2- 172.21.15.45:0/1688651440 >> v2:172.21.15.117:3300/0 conn(0x7fd3200ae020 0x7fd3200ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 --2- 172.21.15.45:0/1688651440 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200cf930 0x7fd3200cfd10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- 172.21.15.45:0/1688651440 >> 172.21.15.45:0/1688651440 conn(0x7fd320066b70 msgr2=0x7fd320066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- 172.21.15.45:0/1688651440 shutdown_connections 2024-04-25T10:19:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- 172.21.15.45:0/1688651440 wait complete. 2024-04-25T10:19:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 Processor -- start 2024-04-25T10:19:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- start start 2024-04-25T10:19:06.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 0x7fd320158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200ae020 0x7fd3201529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3200cf930 0x7fd3200cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32016a8e0 con 0x7fd320070c20 2024-04-25T10:19:06.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32006e130 con 0x7fd3200cf930 2024-04-25T10:19:06.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd324a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32016aa60 con 0x7fd3200ae020 2024-04-25T10:19:06.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.856+0000 7fd31de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 0x7fd320158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 0x7fd320158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50186/0 (socket says 172.21.15.45:50186) 2024-04-25T10:19:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 -- 172.21.15.45:0/3682735484 learned_addr learned my addr 172.21.15.45:0/3682735484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31e800640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3200cf930 0x7fd3200cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd317e00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200ae020 0x7fd3201529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 -- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200ae020 msgr2=0x7fd3201529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200ae020 0x7fd3201529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 -- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3200cf930 msgr2=0x7fd3200cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3200cf930 0x7fd3200cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 -- 172.21.15.45:0/3682735484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320154140 con 0x7fd320070c20 2024-04-25T10:19:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd31de00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 0x7fd320158f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd31001ab20 tx=0x7fd310019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3100abd20 con 0x7fd320070c20 2024-04-25T10:19:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3100bf450 con 0x7fd320070c20 2024-04-25T10:19:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32015c7d0 con 0x7fd320070c20 2024-04-25T10:19:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3100bf840 con 0x7fd320070c20 2024-04-25T10:19:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32015cd90 con 0x7fd320070c20 2024-04-25T10:19:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32004ef30 con 0x7fd320070c20 2024-04-25T10:19:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.860+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3100bf9e0 con 0x7fd320070c20 2024-04-25T10:19:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.864+0000 7fd316000640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2fc066f60 0x7fd2fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.864+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd31011f9a0 con 0x7fd320070c20 2024-04-25T10:19:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.864+0000 7fd317e00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2fc066f60 0x7fd2fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.864+0000 7fd317e00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2fc066f60 0x7fd2fc069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3200cf180 tx=0x7fd308007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:06.868+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd310016090 con 0x7fd320070c20 2024-04-25T10:19:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.064+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd3200b4020 con 0x7fd320070c20 2024-04-25T10:19:07.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd316000640 1 -- 172.21.15.45:0/3682735484 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd3100f0ab0 con 0x7fd320070c20 2024-04-25T10:19:07.070 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:19:07.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2fc066f60 msgr2=0x7fd2fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2fc066f60 0x7fd2fc069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd3200cf180 tx=0x7fd308007cd0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 msgr2=0x7fd320158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 0x7fd320158f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd31001ab20 tx=0x7fd310019210 comp rx=0 tx=0).stop 2024-04-25T10:19:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 shutdown_connections 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2fc066f60 0x7fd2fc069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3200cf930 0x7fd3200cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3200ae020 0x7fd3201529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 --2- 172.21.15.45:0/3682735484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd320070c20 0x7fd320158f30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 >> 172.21.15.45:0/3682735484 conn(0x7fd320066b70 msgr2=0x7fd3200ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 shutdown_connections 2024-04-25T10:19:07.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.068+0000 7fd324a00640 1 -- 172.21.15.45:0/3682735484 wait complete. 2024-04-25T10:19:07.092 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:19:07.092 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:19:07.092 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:19:07.093 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:19:07.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.093 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:19:07.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:19:07.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:19:07.093 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:07.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.094 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:19:07.104 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:19:07.105 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:19:07.105 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:19:07.105 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.105 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:19:07.105 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.106 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:19:07.106 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:07.106 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:19:07.106 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:19:07.106 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:07.106 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:19:07.107 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:19:07.107 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:07.107 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:19:07.107 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:19:07.107 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42393sec preferred_lft 42393sec 2024-04-25T10:19:07.108 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:19:07.108 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.108 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:19:07.108 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:07.108 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:19:07.108 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:19:07.109 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:07.109 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:19:07.109 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.109 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:19:07.109 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:07.109 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:19:07.110 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.110 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:19:07.110 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.110 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:19:07.110 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:19:07.111 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:19:07.111 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.111 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:19:07.111 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:19:07.111 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:19:07.111 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:07.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.112 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:19:07.162 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:19:07.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.165 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:19:07.222 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:19:07.222 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:19:07.223 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:19:07.223 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:19:07.223 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:19:07.223 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:19:07.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.223 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:19:07.223 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:19:07.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:19:07.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:19:07.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:19:07.224 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:19:07.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.320 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:19:07.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:19:07.329 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:19:07.330 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:07.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:19:07.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:19:07.397 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.453 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:07.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:19:07.453 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.505 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:19:07.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:19:07.597 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:19:07.598 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:19:07.611 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:19:07.611 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:19:07.611 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:19:07.611 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:19:07.611 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:19:07.615 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:19:07.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.635 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:19:07.695 INFO:teuthology.orchestra.run.smithi045.stdout:client4883 2024-04-25T10:19:07.696 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:19:07.696 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:07.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:19:07.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 Processor -- start 2024-04-25T10:19:07.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 -- start start 2024-04-25T10:19:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e9c0b3fc0 0x7f5e9c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b24b0 0x7f5e9c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9c1608c0 con 0x7f5e9c15cf90 2024-04-25T10:19:07.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c0cd6c0 con 0x7f5e9c0b3fc0 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea3600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0cd070 con 0x7f5e9c0b24b0 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e9c0b3fc0 0x7f5e9c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e9c0b3fc0 0x7f5e9c1601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50188/0 (socket says 172.21.15.45:50188) 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 -- 172.21.15.45:0/2424663199 learned_addr learned my addr 172.21.15.45:0/2424663199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5ea1600640 1 --2- 172.21.15.45:0/2424663199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b24b0 0x7f5e9c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 -- 172.21.15.45:0/2424663199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b24b0 msgr2=0x7f5e9c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 --2- 172.21.15.45:0/2424663199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b24b0 0x7f5e9c0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 -- 172.21.15.45:0/2424663199 >> v1:172.21.15.117:6789/0 conn(0x7f5e9c15cf90 legacy=0x7f5e9c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.932+0000 7f5e9be00640 1 -- 172.21.15.45:0/2424663199 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0cd350 con 0x7f5e9c0b3fc0 2024-04-25T10:19:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5e9be00640 1 --2- 172.21.15.45:0/2424663199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c0b3fc0 0x7f5e9c1601c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009870 tx=0x7f5e8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545f80a1426f3054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5e9b400640 1 -- 172.21.15.45:0/2424663199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e880aa650 con 0x7f5e9c0b3fc0 2024-04-25T10:19:07.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5e9b400640 1 -- 172.21.15.45:0/2424663199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e880af070 con 0x7f5e9c0b3fc0 2024-04-25T10:19:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5e9b400640 1 -- 172.21.15.45:0/2424663199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e880aaa10 con 0x7f5e9c0b3fc0 2024-04-25T10:19:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 -- 172.21.15.45:0/2424663199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c0b3fc0 msgr2=0x7f5e9c1601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 --2- 172.21.15.45:0/2424663199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c0b3fc0 0x7f5e9c1601c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009870 tx=0x7f5e8801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 -- 172.21.15.45:0/2424663199 shutdown_connections 2024-04-25T10:19:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 --2- 172.21.15.45:0/2424663199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b24b0 0x7f5e9c0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 --2- 172.21.15.45:0/2424663199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c0b3fc0 0x7f5e9c1601c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 -- 172.21.15.45:0/2424663199 >> 172.21.15.45:0/2424663199 conn(0x7f5e9c0b34a0 msgr2=0x7f5e9c0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 -- 172.21.15.45:0/2424663199 shutdown_connections 2024-04-25T10:19:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 -- 172.21.15.45:0/2424663199 wait complete. 2024-04-25T10:19:07.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 Processor -- start 2024-04-25T10:19:07.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.936+0000 7f5ea3600640 1 -- start start 2024-04-25T10:19:07.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e9c0b24b0 0x7f5e9c0c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b3fc0 0x7f5e9c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 0x7f5e9c0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0d0680 con 0x7f5e9c15cf90 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0d0500 con 0x7f5e9c0b24b0 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0d0800 con 0x7f5e9c0b3fc0 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 0x7f5e9c0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 0x7f5e9c0b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50196/0 (socket says 172.21.15.45:50196) 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 -- 172.21.15.45:0/774715759 learned_addr learned my addr 172.21.15.45:0/774715759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea0c00640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e9c0b24b0 0x7f5e9c0c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5e9be00640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b3fc0 0x7f5e9c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 -- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b3fc0 msgr2=0x7f5e9c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b3fc0 0x7f5e9c0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 -- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e9c0b24b0 msgr2=0x7f5e9c0c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e9c0b24b0 0x7f5e9c0c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 -- 172.21.15.45:0/774715759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0c5a60 con 0x7f5e9c15cf90 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea1600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 0x7f5e9c0b4ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5e94007e60 tx=0x7f5e9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94014420 con 0x7f5e9c15cf90 2024-04-25T10:19:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94019070 con 0x7f5e9c15cf90 2024-04-25T10:19:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94014820 con 0x7f5e9c15cf90 2024-04-25T10:19:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9c0c5d20 con 0x7f5e9c15cf90 2024-04-25T10:19:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c0c62e0 con 0x7f5e9c15cf90 2024-04-25T10:19:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.940+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c04ef30 con 0x7f5e9c15cf90 2024-04-25T10:19:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.944+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e940149c0 con 0x7f5e9c15cf90 2024-04-25T10:19:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.944+0000 7f5e98c00640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e7c066f60 0x7f5e7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.944+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e940856e0 con 0x7f5e9c15cf90 2024-04-25T10:19:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.944+0000 7f5ea0c00640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e7c066f60 0x7f5e7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:07.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.944+0000 7f5ea0c00640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e7c066f60 0x7f5e7c069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e90001810 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:07.948+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e94056090 con 0x7f5e9c15cf90 2024-04-25T10:19:08.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.144+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5e9c0b78d0 con 0x7f5e9c15cf90 2024-04-25T10:19:08.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5e98c00640 1 -- 172.21.15.45:0/774715759 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v18) v1 ==== 167+0+0 (secure 0 0 0) 0x7f5e9405a130 con 0x7f5e9c15cf90 2024-04-25T10:19:08.274 INFO:teuthology.orchestra.run.smithi045.stderr:enabled new snapshots 2024-04-25T10:19:08.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e7c066f60 msgr2=0x7f5e7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e7c066f60 0x7f5e7c069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e90001810 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2024-04-25T10:19:08.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 msgr2=0x7f5e9c0b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 0x7f5e9c0b4ad0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5e94007e60 tx=0x7f5e9400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 shutdown_connections 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e7c066f60 0x7f5e7c069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e9c15cf90 0x7f5e9c0b4ad0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e9c0b3fc0 0x7f5e9c0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 --2- 172.21.15.45:0/774715759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e9c0b24b0 0x7f5e9c0c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 >> 172.21.15.45:0/774715759 conn(0x7f5e9c0b34a0 msgr2=0x7f5e9c166600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 shutdown_connections 2024-04-25T10:19:08.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:08.272+0000 7f5ea3600640 1 -- 172.21.15.45:0/774715759 wait complete. 2024-04-25T10:19:08.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:08.298 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:19:08.314 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:19:08.314 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:08.367 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:19:08.367 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:19:08.367 DEBUG:teuthology.orchestra.run.smithi045:> import uuid 2024-04-25T10:19:08.367 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:19:08.367 DEBUG:teuthology.orchestra.run.smithi045:> n = 2000 2024-04-25T10:19:08.368 DEBUG:teuthology.orchestra.run.smithi045:> create_hard_links = False 2024-04-25T10:19:08.368 DEBUG:teuthology.orchestra.run.smithi045:> if 3 > 0: 2024-04-25T10:19:08.368 DEBUG:teuthology.orchestra.run.smithi045:> create_hard_links = True 2024-04-25T10:19:08.368 DEBUG:teuthology.orchestra.run.smithi045:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:19:08.368 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:19:08.368 DEBUG:teuthology.orchestra.run.smithi045:> dpath = os.path.dirname(path) 2024-04-25T10:19:08.369 DEBUG:teuthology.orchestra.run.smithi045:> fnameprefix = os.path.basename(path) 2024-04-25T10:19:08.369 DEBUG:teuthology.orchestra.run.smithi045:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:19:08.369 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:19:08.369 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:19:08.369 DEBUG:teuthology.orchestra.run.smithi045:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:19:08.370 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:19:08.370 DEBUG:teuthology.orchestra.run.smithi045:> for i in range(n): 2024-04-25T10:19:08.370 DEBUG:teuthology.orchestra.run.smithi045:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:19:08.370 DEBUG:teuthology.orchestra.run.smithi045:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:19:08.370 DEBUG:teuthology.orchestra.run.smithi045:> f.write(f"{i}") 2024-04-25T10:19:08.370 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:19:08.371 DEBUG:teuthology.orchestra.run.smithi045:> f.flush() 2024-04-25T10:19:08.371 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(f.fileno()) 2024-04-25T10:19:08.371 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:19:08.371 DEBUG:teuthology.orchestra.run.smithi045:> os.unlink(fpath) 2024-04-25T10:19:08.371 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:19:08.371 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(dirfd) 2024-04-25T10:19:08.372 DEBUG:teuthology.orchestra.run.smithi045:> if create_hard_links: 2024-04-25T10:19:08.372 DEBUG:teuthology.orchestra.run.smithi045:> for j in range(3): 2024-04-25T10:19:08.372 DEBUG:teuthology.orchestra.run.smithi045:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:19:08.372 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:19:08.372 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(dirfd) 2024-04-25T10:19:08.372 DEBUG:teuthology.orchestra.run.smithi045:> finally: 2024-04-25T10:19:08.373 DEBUG:teuthology.orchestra.run.smithi045:> os.close(dirfd) 2024-04-25T10:19:08.373 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:19:25.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:25.817 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:19:30.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:30.275 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:19:30.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:30.297 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:19:56.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:56.526 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:58.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:58.337 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:19:58.391 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:19:58.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:19:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 Processor -- start 2024-04-25T10:19:58.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 -- start start 2024-04-25T10:19:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f849015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8490070350 0x7f849015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849015b080 con 0x7f849015bac0 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849015adb0 con 0x7f8490070350 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8497400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849015eb60 con 0x7f84900aef80 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f848fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8490070350 0x7f849015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8495400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f849015bac0 0x7f84900b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37678/0 (socket says 172.21.15.45:37678) 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8495400640 1 -- 172.21.15.45:0/4172184587 learned_addr learned my addr 172.21.15.45:0/4172184587 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f8494a00640 1 --2- 172.21.15.45:0/4172184587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f849015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f848fe00640 1 -- 172.21.15.45:0/4172184587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 msgr2=0x7f849015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f848fe00640 1 --2- 172.21.15.45:0/4172184587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f849015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f848fe00640 1 -- 172.21.15.45:0/4172184587 >> v1:172.21.15.45:6789/0 conn(0x7f849015bac0 legacy=0x7f84900b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f848fe00640 1 -- 172.21.15.45:0/4172184587 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849015ee40 con 0x7f8490070350 2024-04-25T10:19:58.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.540+0000 7f848f400640 1 -- 172.21.15.45:0/4172184587 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151638381 0 0) 0x7f849015b080 con 0x7f849015bac0 2024-04-25T10:19:58.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f848fe00640 1 --2- 172.21.15.45:0/4172184587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f849015b580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097c0 tx=0x7f847c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6b690461cc68e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f848f400640 1 -- 172.21.15.45:0/4172184587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c0ab680 con 0x7f8490070350 2024-04-25T10:19:58.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f848f400640 1 -- 172.21.15.45:0/4172184587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c0b0070 con 0x7f8490070350 2024-04-25T10:19:58.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f848f400640 1 -- 172.21.15.45:0/4172184587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c0aba40 con 0x7f8490070350 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 -- 172.21.15.45:0/4172184587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 msgr2=0x7f849015b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 --2- 172.21.15.45:0/4172184587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f849015b580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097c0 tx=0x7f847c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 -- 172.21.15.45:0/4172184587 shutdown_connections 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 --2- 172.21.15.45:0/4172184587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f849015b580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 --2- 172.21.15.45:0/4172184587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f849015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 -- 172.21.15.45:0/4172184587 >> 172.21.15.45:0/4172184587 conn(0x7f84900669a0 msgr2=0x7f8490066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 -- 172.21.15.45:0/4172184587 shutdown_connections 2024-04-25T10:19:58.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.544+0000 7f8497400640 1 -- 172.21.15.45:0/4172184587 wait complete. 2024-04-25T10:19:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 Processor -- start 2024-04-25T10:19:58.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 -- start start 2024-04-25T10:19:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f8490147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f84900af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849015bac0 0x7f8490156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8490167820 con 0x7f8490070350 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900b3ae0 con 0x7f849015bac0 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84901679a0 con 0x7f84900aef80 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f8490147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f8490147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42912/0 (socket says 172.21.15.45:42912) 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 -- 172.21.15.45:0/1681923809 learned_addr learned my addr 172.21.15.45:0/1681923809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8495400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849015bac0 0x7f8490156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f848fe00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f84900af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 -- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 msgr2=0x7f84900af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f84900af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 -- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849015bac0 msgr2=0x7f8490156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849015bac0 0x7f8490156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 -- 172.21.15.45:0/1681923809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900b0090 con 0x7f8490070350 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8494a00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f8490147fe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f848400e960 tx=0x7f848400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f848cc00640 1 -- 172.21.15.45:0/1681923809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484017070 con 0x7f8490070350 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f848cc00640 1 -- 172.21.15.45:0/1681923809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400ca00 con 0x7f8490070350 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84900b0370 con 0x7f8490070350 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f848cc00640 1 -- 172.21.15.45:0/1681923809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400cdb0 con 0x7f8490070350 2024-04-25T10:19:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.548+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84900b0930 con 0x7f8490070350 2024-04-25T10:19:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f848cc00640 1 -- 172.21.15.45:0/1681923809 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f848401a070 con 0x7f8490070350 2024-04-25T10:19:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f848cc00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8470066f90 0x7f8470069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f848cc00640 1 -- 172.21.15.45:0/1681923809 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8484085510 con 0x7f8490070350 2024-04-25T10:19:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f848fe00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8470066f90 0x7f8470069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 Processor -- start 2024-04-25T10:19:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 -- start start 2024-04-25T10:19:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f848fe00640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8470066f90 0x7f8470069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f847c0af000 tx=0x7f847c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b6a40 0x5587b46b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b7360 0x5587b4671850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b4672170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b45541f0 con 0x5587b4671d90 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b44bae90 con 0x5587b46b7360 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587b4442f50 con 0x5587b46b6a40 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f8495400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b4672170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f8495400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b4672170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42914/0 (socket says 172.21.15.45:42914) 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.552+0000 7f8495400640 1 -- 172.21.15.45:0/1882184097 learned_addr learned my addr 172.21.15.45:0/1882184097 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f848fe00640 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b7360 0x5587b4671850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8494a00640 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b6a40 0x5587b46b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 -- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b6a40 msgr2=0x5587b46b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b6a40 0x5587b46b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 -- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b7360 msgr2=0x5587b4671850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b7360 0x5587b4671850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 -- 172.21.15.45:0/1882184097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b44928b0 con 0x5587b4671d90 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b4672170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8488001be0 tx=0x7f84880020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0021d4555a43b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8481600640 1 -- 172.21.15.45:0/1882184097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488007850 con 0x5587b4671d90 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8481600640 1 -- 172.21.15.45:0/1882184097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488018070 con 0x5587b4671d90 2024-04-25T10:19:58.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8481600640 1 -- 172.21.15.45:0/1882184097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488013630 con 0x5587b4671d90 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- 172.21.15.45:0/1882184097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 msgr2=0x5587b4672170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b4672170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8488001be0 tx=0x7f84880020a0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- 172.21.15.45:0/1882184097 shutdown_connections 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b4672170 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b7360 0x5587b4671850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- 172.21.15.45:0/1882184097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b6a40 0x5587b46b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- 172.21.15.45:0/1882184097 >> 172.21.15.45:0/1882184097 conn(0x5587b466dd40 msgr2=0x5587b4670130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- 172.21.15.45:0/1882184097 shutdown_connections 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- 172.21.15.45:0/1882184097 wait complete. 2024-04-25T10:19:58.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- start start 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 Processor -- start 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b46be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b46be700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4677e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4475540 con 0x5587b4671d90 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b44b6f30 con 0x5587b46b6a40 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587b44b9e60 con 0x5587b46b7360 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f849a926000 10 client.0 _ll_register_callbacks cb 0x5587b456f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b46be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b46be1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42924/0 (socket says 172.21.15.45:42924) 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8494a00640 1 -- 172.21.15.45:0/2279568137 learned_addr learned my addr 172.21.15.45:0/2279568137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f848fe00640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b46be700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.556+0000 7f8495400640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4677e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8494a00640 1 -- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 msgr2=0x5587b4677e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8494a00640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4677e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8494a00640 1 -- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 msgr2=0x5587b46be700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8494a00640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b46be700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8494a00640 1 -- 172.21.15.45:0/2279568137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587b44431a0 con 0x5587b4671d90 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8495400640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4677e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8494a00640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b46be1c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f848400c520 tx=0x7f8484055de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518bde94a555aeab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 1 -- 172.21.15.45:0/2279568137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848406d070 con 0x5587b4671d90 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 1 -- 172.21.15.45:0/2279568137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f848405e180 con 0x5587b4671d90 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8488018070 con 0x5587b4671d90 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 1 -- 172.21.15.45:0/2279568137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8484072050 con 0x5587b4671d90 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 10 client.5292 fetch_fsmap learned FSMap version 18 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5587b453d840 con 0x5587b4671d90 2024-04-25T10:19:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 1 -- 172.21.15.45:0/2279568137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488018070 con 0x5587b4671d90 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 1 -- 172.21.15.45:0/2279568137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8484055170 con 0x5587b4671d90 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 10 client.5292 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 10 client.5292 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 20 client.5292 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b4678bb0 0x5587b4678f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 4 client.5292 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587b4456be0 con 0x5587b4678bb0 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f848fe00640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b4678bb0 0x5587b4678f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f848fe00640 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b4678bb0 0x5587b4678f90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54ab082dfed7a24 server_cookie=3853dac2b12fc272 in_seq=0 out_seq=0 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 10 client.5292 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 1 -- 172.21.15.45:0/2279568137 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587b4456be0 con 0x5587b4678bb0 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 10 client.5292 handle_command_reply: tid=0 2024-04-25T10:19:58.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 client.5292 shutdown 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 msgr2=0x5587b46be1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b46be1c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f848400c520 tx=0x7f8484055de0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 shutdown_connections 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b4678bb0 0x5587b4678f90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4677e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b46be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f8476000640 0 client.5292 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 --2- 172.21.15.45:0/2279568137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b4671d90 0x5587b46be1c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 >> 172.21.15.45:0/2279568137 conn(0x5587b466dd40 msgr2=0x5587b46709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 shutdown_connections 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 1 -- 172.21.15.45:0/2279568137 wait complete. 2024-04-25T10:19:58.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.560+0000 7f849a926000 20 client.5292 trim_cache size 0 max 16384 2024-04-25T10:19:58.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 Processor -- start 2024-04-25T10:19:58.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 -- start start 2024-04-25T10:19:58.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b4671d90 0x5587b4713290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 0x5587b47137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4713d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8490203080 con 0x5587b46b6a40 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901f7bb0 con 0x5587b4671d90 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84901f6fb0 con 0x5587b46b7360 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 0x5587b47137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 0x5587b47137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42936/0 (socket says 172.21.15.45:42936) 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 -- 172.21.15.45:0/144661850 learned_addr learned my addr 172.21.15.45:0/144661850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 -- 172.21.15.45:0/144661850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 msgr2=0x5587b4713d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4713d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 -- 172.21.15.45:0/144661850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b4671d90 msgr2=0x5587b4713290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b4671d90 0x5587b4713290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 -- 172.21.15.45:0/144661850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b46b7190 con 0x5587b46b6a40 2024-04-25T10:19:58.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f848fe00640 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 0x5587b47137d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f847c01ab50 tx=0x7f847c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e917ceb68fc583c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f8480c00640 1 -- 172.21.15.45:0/144661850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c0bc070 con 0x5587b46b6a40 2024-04-25T10:19:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f8480c00640 1 -- 172.21.15.45:0/144661850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c0b0040 con 0x5587b46b6a40 2024-04-25T10:19:58.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f8480c00640 1 -- 172.21.15.45:0/144661850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c0cf250 con 0x5587b46b6a40 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 -- 172.21.15.45:0/144661850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 msgr2=0x5587b47137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.576+0000 7f849a926000 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 0x5587b47137d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f847c01ab50 tx=0x7f847c009190 comp rx=0 tx=0).stop 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/144661850 shutdown_connections 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b46b7360 0x5587b4713d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b6a40 0x5587b47137d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- 172.21.15.45:0/144661850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b4671d90 0x5587b4713290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/144661850 >> 172.21.15.45:0/144661850 conn(0x5587b466dd40 msgr2=0x5587b46b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/144661850 shutdown_connections 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/144661850 wait complete. 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- start start 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 Processor -- start 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b4671d90 0x5587b47172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b47177e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 0x5587b46bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b46585b0 con 0x5587b46b7360 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901f75b0 con 0x5587b46b6a40 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587b44b6f30 con 0x5587b4671d90 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 10 client.0 _ll_register_callbacks cb 0x5587b447da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8494a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b4671d90 0x5587b47172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 0x5587b46bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8494a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b4671d90 0x5587b47172a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46948/0 (socket says 172.21.15.45:46948) 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8494a00640 1 -- 172.21.15.45:0/574170397 learned_addr learned my addr 172.21.15.45:0/574170397 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 0x5587b46bbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42946/0 (socket says 172.21.15.45:42946) 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f848fe00640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b47177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 -- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b4671d90 msgr2=0x5587b47172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b4671d90 0x5587b47172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 -- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 msgr2=0x5587b47177e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b47177e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 -- 172.21.15.45:0/574170397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587b4514a10 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8495400640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 0x5587b46bbed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8488017000 tx=0x7f8488001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9e605265147d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8476000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8476000640 1 -- 172.21.15.45:0/574170397 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848801e820 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8476000640 1 -- 172.21.15.45:0/574170397 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f848801e9c0 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f847c0b0040 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8476000640 1 -- 172.21.15.45:0/574170397 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f848801ebc0 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 10 client.5298 fetch_fsmap learned FSMap version 18 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5587b444fb00 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8476000640 1 -- 172.21.15.45:0/574170397 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c0b0040 con 0x5587b46b7360 2024-04-25T10:19:58.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8476000640 1 -- 172.21.15.45:0/574170397 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f848801c450 con 0x5587b46b7360 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 10 client.5298 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 10 client.5298 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 20 client.5298 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b46bc8f0 0x5587b46bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 4 client.5298 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5587b44c83d0 con 0x5587b46bc8f0 2024-04-25T10:19:58.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.580+0000 7f8494a00640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b46bc8f0 0x5587b46bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8494a00640 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b46bc8f0 0x5587b46bcd00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369687f8cf187445 server_cookie=e2bbd82329ea1dd3 in_seq=0 out_seq=0 2024-04-25T10:19:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8476000640 10 client.5298 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8476000640 1 -- 172.21.15.45:0/574170397 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17920 (crc 0 0 0) 0x5587b44c83d0 con 0x5587b46bc8f0 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8476000640 10 client.5298 handle_command_reply: tid=0 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 client.5298 shutdown 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 msgr2=0x5587b46bbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 0x5587b46bbed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8488017000 tx=0x7f8488001260 comp rx=0 tx=0).stop 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 shutdown_connections 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5587b46bc8f0 0x5587b46bcd00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5587b46b7360 0x5587b46bbed0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587b46b6a40 0x5587b47177e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 --2- 172.21.15.45:0/574170397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5587b4671d90 0x5587b47172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8476000640 0 client.5298 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 >> 172.21.15.45:0/574170397 conn(0x5587b466dd40 msgr2=0x5587b46b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 shutdown_connections 2024-04-25T10:19:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 1 -- 172.21.15.45:0/574170397 wait complete. 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f849a926000 20 client.5298 trim_cache size 0 max 16384 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 29976, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 50914, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 10870945, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 44868605, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 10, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 5.031577010, 2024-04-25T10:19:58.591 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 2.811136449, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 2.369364675, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.367089701, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 50914, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.553783382, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000030517 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 72319, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.016587122, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000229 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 115008, 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 19472 2024-04-25T10:19:58.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2514, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2527, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 802809, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 44020045, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.681686437, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.160224997, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.060279319, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.442053198, 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:19:58.593 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2526, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.380731910, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000150725 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 77744 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:19:58.594 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2272, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2273, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 568345, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 26164581, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.259927244, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.128430018, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.048078780, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.067467178, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2272, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.342331004, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000150673 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.595 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4363, 2024-04-25T10:19:58.596 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 14.081113493, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003227392 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1539, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.612117118, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000397736 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 26248, 2024-04-25T10:19:58.597 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 26248, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 26248, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 270.370894641, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.010300628 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 12, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T10:19:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 19711, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 19711, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 19711, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 49, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 49, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 2, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 49233, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 48717, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 147, 2024-04-25T10:19:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 1, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:19:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 1520, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 4058, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 4557, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 31, 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 14075 2024-04-25T10:19:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 2000, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 317, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 4124, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 1636, 2024-04-25T10:19:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 2222, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T10:19:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 28181, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 28181, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 44, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 44, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 4, 2024-04-25T10:19:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 94977785, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4106, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 79.353899178, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.019326327 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 2014, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 2014, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 0, 2024-04-25T10:19:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 13, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 13, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 19711, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 19712, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 1, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 49, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 3716, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 3667, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 108920, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215340 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 29922, 2024-04-25T10:19:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 26248, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 4, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2000, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.293985227, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000646992 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 28, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.005549609, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000198200 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6000, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 7.249193994, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001208198 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7985, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 7.114268350, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000890954 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000072030, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000072030 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003365621, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003365621 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.007541183, 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.007541183 2024-04-25T10:19:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.091971947, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.011496493 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2222, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 237.779287555, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.107011380 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.043346003, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.043346003 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:19:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000377494, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000377494 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T10:19:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8000, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 16.781935628, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002097741 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-04-25T10:19:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 18689, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 18689, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 50, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:19:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:19:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:19:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 16291101, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 4116, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 5696, 2024-04-25T10:19:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 83, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 20901723, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 815173, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T10:19:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 5963, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 91181287, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 37, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 5963, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5963, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 26.783730289, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.004491653 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5963, 2024-04-25T10:19:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 6057 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 5963, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 43, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 5920, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 11, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 43, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 4149, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 28, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T10:19:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 1636, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 33, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 22, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T10:19:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T10:19:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 37, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 3, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 8, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 8, 2024-04-25T10:19:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 106, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 29 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-25T10:19:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 32, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 1636, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 1636, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:19:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 34756, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 9108811, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 34756, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 34756, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9108811, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 5963, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 91359611, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 5963, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 91359611, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 5963, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 5963, 2024-04-25T10:19:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 5963, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 5963, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 1636, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 178324, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:19:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 1636, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 43, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 178324, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x55ba2f044960": { 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 84, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 28181, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 90783481, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 28181, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4106, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 90783397, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:58.636 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8470066f90 msgr2=0x7f8470069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8470066f90 0x7f8470069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f847c0af000 tx=0x7f847c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 msgr2=0x7f8490147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f8490147fe0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f848400e960 tx=0x7f848400ee20 comp rx=0 tx=0).stop 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 shutdown_connections 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8470066f90 0x7f8470069450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849015bac0 0x7f8490156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84900aef80 0x7f84900af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 --2- 172.21.15.45:0/1681923809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490070350 0x7f8490147fe0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 >> 172.21.15.45:0/1681923809 conn(0x7f84900669a0 msgr2=0x7f8490158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 shutdown_connections 2024-04-25T10:19:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.584+0000 7f8497400640 1 -- 172.21.15.45:0/1681923809 wait complete. 2024-04-25T10:19:58.638 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T10:19:58.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:58.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T10:19:58.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 Processor -- start 2024-04-25T10:19:58.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 -- start start 2024-04-25T10:19:58.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f00000cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00000af020 0x7f000006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f000006de00 0x7f00000ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0000162080 con 0x7f00000af020 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000015df10 con 0x7f000006de00 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0006a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0000162510 con 0x7f000015db30 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0004c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00000af020 0x7f000006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0004c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00000af020 0x7f000006d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42956/0 (socket says 172.21.15.45:42956) 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.808+0000 7f0004c00640 1 -- 172.21.15.45:0/3193124666 learned_addr learned my addr 172.21.15.45:0/3193124666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0005600640 1 --2- 172.21.15.45:0/3193124666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f00000cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0006000640 1 --2- 172.21.15.45:0/3193124666 >> v2:172.21.15.117:3300/0 conn(0x7f000006de00 0x7f00000ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0004c00640 1 -- 172.21.15.45:0/3193124666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 msgr2=0x7f00000cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0004c00640 1 --2- 172.21.15.45:0/3193124666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f00000cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0004c00640 1 -- 172.21.15.45:0/3193124666 >> v2:172.21.15.117:3300/0 conn(0x7f000006de00 msgr2=0x7f00000ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0004c00640 1 --2- 172.21.15.45:0/3193124666 >> v2:172.21.15.117:3300/0 conn(0x7f000006de00 0x7f00000ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0004c00640 1 -- 172.21.15.45:0/3193124666 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0000161750 con 0x7f00000af020 2024-04-25T10:19:58.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0004c00640 1 --2- 172.21.15.45:0/3193124666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000af020 0x7f000006d8c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f00000ae900 tx=0x7effec01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3a117bd3bddf47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7effffe00640 1 -- 172.21.15.45:0/3193124666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0b0070 con 0x7f00000af020 2024-04-25T10:19:58.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7effffe00640 1 -- 172.21.15.45:0/3193124666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec0ab600 con 0x7f00000af020 2024-04-25T10:19:58.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7effffe00640 1 -- 172.21.15.45:0/3193124666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0ab9d0 con 0x7f00000af020 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0006a00640 1 -- 172.21.15.45:0/3193124666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000af020 msgr2=0x7f000006d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.812+0000 7f0006a00640 1 --2- 172.21.15.45:0/3193124666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000af020 0x7f000006d8c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f00000ae900 tx=0x7effec01c1d0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- 172.21.15.45:0/3193124666 shutdown_connections 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 --2- 172.21.15.45:0/3193124666 >> v2:172.21.15.117:3300/0 conn(0x7f000006de00 0x7f00000ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 --2- 172.21.15.45:0/3193124666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00000af020 0x7f000006d8c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 --2- 172.21.15.45:0/3193124666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f00000cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- 172.21.15.45:0/3193124666 >> 172.21.15.45:0/3193124666 conn(0x7f00000b15c0 msgr2=0x7f00000b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- 172.21.15.45:0/3193124666 shutdown_connections 2024-04-25T10:19:58.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- 172.21.15.45:0/3193124666 wait complete. 2024-04-25T10:19:58.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 Processor -- start 2024-04-25T10:19:58.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- start start 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 0x7f00000c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000af020 0x7f00000c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f0000156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000006c510 con 0x7f000006de00 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00000cdb20 con 0x7f00000af020 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000006c690 con 0x7f000015db30 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 0x7f00000c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 0x7f00000c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42962/0 (socket says 172.21.15.45:42962) 2024-04-25T10:19:58.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 -- 172.21.15.45:0/1659147167 learned_addr learned my addr 172.21.15.45:0/1659147167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0004c00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000af020 0x7f00000c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0006000640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f0000156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 -- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 msgr2=0x7f0000156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f0000156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 -- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000af020 msgr2=0x7f00000c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000af020 0x7f00000c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 -- 172.21.15.45:0/1659147167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000015e1c0 con 0x7f000006de00 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0004c00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000af020 0x7f00000c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.816+0000 7f0005600640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 0x7f00000c77e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efff400e870 tx=0x7efff400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7efffd600640 1 -- 172.21.15.45:0/1659147167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4017070 con 0x7f000006de00 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7efffd600640 1 -- 172.21.15.45:0/1659147167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400c940 con 0x7f000006de00 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7efffd600640 1 -- 172.21.15.45:0/1659147167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400ccf0 con 0x7f000006de00 2024-04-25T10:19:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00000ca140 con 0x7f000006de00 2024-04-25T10:19:58.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00000ca6a0 con 0x7f000006de00 2024-04-25T10:19:58.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7efffd600640 1 -- 172.21.15.45:0/1659147167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efff401a070 con 0x7f000006de00 2024-04-25T10:19:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7efffd600640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effe0066f60 0x7effe0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7efffd600640 1 -- 172.21.15.45:0/1659147167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efff4084ee0 con 0x7f000006de00 2024-04-25T10:19:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7f0004c00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effe0066f60 0x7effe0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7f000b553000 1 Processor -- start 2024-04-25T10:19:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.820+0000 7f000b553000 1 -- start start 2024-04-25T10:19:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0004c00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effe0066f60 0x7effe0069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7effec0ad000 tx=0x7effec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae708900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae708e40 0x5611ae6bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae6bf970 0x5611ae6bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ae50cd00 con 0x5611ae708520 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ae494df0 con 0x5611ae6bf970 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ae508da0 con 0x5611ae708e40 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae708900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae708900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42966/0 (socket says 172.21.15.45:42966) 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 -- 172.21.15.45:0/3798950264 learned_addr learned my addr 172.21.15.45:0/3798950264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0004c00640 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae708e40 0x5611ae6bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0006000640 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae6bf970 0x5611ae6bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 -- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae708e40 msgr2=0x5611ae6bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae708e40 0x5611ae6bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 -- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae6bf970 msgr2=0x5611ae6bfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae6bf970 0x5611ae6bfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 -- 172.21.15.45:0/3798950264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ae4c7580 con 0x5611ae708520 2024-04-25T10:19:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f0005600640 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae708900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efff405d8b0 tx=0x7efff40553f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ea19d040e39d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7efff2000640 1 -- 172.21.15.45:0/3798950264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff406d020 con 0x5611ae708520 2024-04-25T10:19:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7efff2000640 1 -- 172.21.15.45:0/3798950264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff405dde0 con 0x5611ae708520 2024-04-25T10:19:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7efff2000640 1 -- 172.21.15.45:0/3798950264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4021070 con 0x5611ae708520 2024-04-25T10:19:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- 172.21.15.45:0/3798950264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 msgr2=0x5611ae708900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae708900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efff405d8b0 tx=0x7efff40553f0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- 172.21.15.45:0/3798950264 shutdown_connections 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae6bf970 0x5611ae6bfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae708e40 0x5611ae6bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 --2- 172.21.15.45:0/3798950264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae708900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- 172.21.15.45:0/3798950264 >> 172.21.15.45:0/3798950264 conn(0x5611ae5a6c10 msgr2=0x5611ae6bdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- 172.21.15.45:0/3798950264 shutdown_connections 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- 172.21.15.45:0/3798950264 wait complete. 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 -- start start 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.824+0000 7f000b553000 1 Processor -- start 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae70ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae70f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae6c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ae4a56d0 con 0x5611ae708e40 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ae4c7950 con 0x5611ae708520 2024-04-25T10:19:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ae50c010 con 0x5611ae6bf970 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 10 client.0 _ll_register_callbacks cb 0x5611ae6ab030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0006000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae6c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae70ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0004c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae70f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0006000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae6c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42978/0 (socket says 172.21.15.45:42978) 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae70ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46988/0 (socket says 172.21.15.45:46988) 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0004c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae70f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36812/0 (socket says 172.21.15.45:36812) 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0006000640 1 -- 172.21.15.45:0/2502597283 learned_addr learned my addr 172.21.15.45:0/2502597283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 -- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 msgr2=0x5611ae70f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae70f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 -- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 msgr2=0x5611ae6c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae6c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 -- 172.21.15.45:0/2502597283 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611ae4a9d40 con 0x5611ae6bf970 2024-04-25T10:19:58.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0006000640 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae6c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0005600640 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae70ec60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efff400c520 tx=0x7efff401e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c743f7636156130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 1 -- 172.21.15.45:0/2502597283 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff4071030 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 1 -- 172.21.15.45:0/2502597283 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efff406d020 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efff405dde0 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 1 -- 172.21.15.45:0/2502597283 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff406d220 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 10 client.4916 fetch_fsmap learned FSMap version 18 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5611ae507cf0 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 1 -- 172.21.15.45:0/2502597283 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff405dde0 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 1 -- 172.21.15.45:0/2502597283 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efff4024060 con 0x5611ae6bf970 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 10 client.4916 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 10 client.4916 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 20 client.4916 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae6c4c20 0x5611ae6c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 4 client.4916 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611ae4a89d0 con 0x5611ae6c4c20 2024-04-25T10:19:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0004c00640 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae6c4c20 0x5611ae6c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7f0004c00640 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae6c4c20 0x5611ae6c5000 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae6d39504ff1760 server_cookie=46bb2d2dba3aa6cd in_seq=0 out_seq=0 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.828+0000 7effe6a00640 10 client.4916 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7effe6a00640 1 -- 172.21.15.45:0/2502597283 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611ae4a89d0 con 0x5611ae6c4c20 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7effe6a00640 10 client.4916 handle_command_reply: tid=0 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 client.4916 shutdown 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 msgr2=0x5611ae70ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae70ec60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efff400c520 tx=0x7efff401e040 comp rx=0 tx=0).stop 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 shutdown_connections 2024-04-25T10:19:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae6c4c20 0x5611ae6c5000 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae6c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae70f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7effe6a00640 0 client.4916 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 --2- 172.21.15.45:0/2502597283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae70ec60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 >> 172.21.15.45:0/2502597283 conn(0x5611ae5a6c10 msgr2=0x5611ae6be420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 shutdown_connections 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 1 -- 172.21.15.45:0/2502597283 wait complete. 2024-04-25T10:19:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.832+0000 7f000b553000 20 client.4916 trim_cache size 0 max 16384 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 Processor -- start 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- start start 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae763990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae763ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae764410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0000203000 con 0x5611ae708e40 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00002021c0 con 0x5611ae708520 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000015f3b0 con 0x5611ae6bf970 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae764410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae764410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42992/0 (socket says 172.21.15.45:42992) 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 -- 172.21.15.45:0/474183582 learned_addr learned my addr 172.21.15.45:0/474183582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0005600640 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae763990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0004c00640 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae763ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 -- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 msgr2=0x5611ae763990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae763990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 -- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 msgr2=0x5611ae763ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae763ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 -- 172.21.15.45:0/474183582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ae708c70 con 0x5611ae708e40 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f0006000640 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae764410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efff8003cc0 tx=0x7efff8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b73143e72cce2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7efff1600640 1 -- 172.21.15.45:0/474183582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8014a80 con 0x5611ae708e40 2024-04-25T10:19:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7efff1600640 1 -- 172.21.15.45:0/474183582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff8014c20 con 0x5611ae708e40 2024-04-25T10:19:58.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7efff1600640 1 -- 172.21.15.45:0/474183582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8020440 con 0x5611ae708e40 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- 172.21.15.45:0/474183582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 msgr2=0x5611ae764410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae764410 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efff8003cc0 tx=0x7efff8009750 comp rx=0 tx=0).stop 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- 172.21.15.45:0/474183582 shutdown_connections 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708e40 0x5611ae764410 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708520 0x5611ae763ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 --2- 172.21.15.45:0/474183582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae763990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- 172.21.15.45:0/474183582 >> 172.21.15.45:0/474183582 conn(0x5611ae5a6c10 msgr2=0x5611ae707820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- 172.21.15.45:0/474183582 shutdown_connections 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.848+0000 7f000b553000 1 -- 172.21.15.45:0/474183582 wait complete. 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- start start 2024-04-25T10:19:58.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 Processor -- start 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae7689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae70d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708e40 0x5611ae70de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ae511330 con 0x5611ae708520 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ae508da0 con 0x5611ae708e40 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ae5c9a90 con 0x5611ae6bf970 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 10 client.0 _ll_register_callbacks cb 0x7efff8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae70d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae70d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43008/0 (socket says 172.21.15.45:43008) 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 -- 172.21.15.45:0/4128806023 learned_addr learned my addr 172.21.15.45:0/4128806023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0005600640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae7689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0006000640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708e40 0x5611ae70de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 -- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 msgr2=0x5611ae7689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae7689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 -- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708e40 msgr2=0x5611ae70de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708e40 0x5611ae70de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 -- 172.21.15.45:0/4128806023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611ae58fd30 con 0x5611ae708520 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0006000640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708e40 0x5611ae70de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0004c00640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae70d910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7effec01a880 tx=0x7effec0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc06a0d93e3da006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7effe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7effe6a00640 1 -- 172.21.15.45:0/4128806023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0b7ac0 con 0x5611ae708520 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7effe6a00640 1 -- 172.21.15.45:0/4128806023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7effec0b7c60 con 0x5611ae708520 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efff8014c20 con 0x5611ae708520 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7effe6a00640 1 -- 172.21.15.45:0/4128806023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7effec0d8050 con 0x5611ae708520 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 10 client.5316 fetch_fsmap learned FSMap version 18 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5611ae4d95a0 con 0x5611ae708520 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7effe6a00640 1 -- 172.21.15.45:0/4128806023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff8014c20 con 0x5611ae708520 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7effe6a00640 1 -- 172.21.15.45:0/4128806023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effec0db020 con 0x5611ae708520 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 10 client.5316 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 10 client.5316 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 20 client.5316 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae724c50 0x5611ae7721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 4 client.5316 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x5611ae4e9610 con 0x5611ae724c50 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.852+0000 7f0005600640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae724c50 0x5611ae7721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.856+0000 7f0005600640 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae724c50 0x5611ae7721c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ef947a443e64c6 server_cookie=96b984548059e42 in_seq=0 out_seq=0 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.856+0000 7effe6a00640 10 client.5316 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7effe6a00640 1 -- 172.21.15.45:0/4128806023 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5611ae4e9610 con 0x5611ae724c50 2024-04-25T10:19:58.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7effe6a00640 10 client.5316 handle_command_reply: tid=0 2024-04-25T10:19:58.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 client.5316 shutdown 2024-04-25T10:19:58.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 msgr2=0x5611ae70d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae70d910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7effec01a880 tx=0x7effec0046d0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 shutdown_connections 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x5611ae724c50 0x5611ae7721c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ae708e40 0x5611ae70de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ae708520 0x5611ae70d910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 --2- 172.21.15.45:0/4128806023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5611ae6bf970 0x5611ae7689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 >> 172.21.15.45:0/4128806023 conn(0x5611ae5a6c10 msgr2=0x5611ae709220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7effe6a00640 0 client.5316 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 shutdown_connections 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 1 -- 172.21.15.45:0/4128806023 wait complete. 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f000b553000 20 client.5316 trim_cache size 0 max 16384 2024-04-25T10:19:58.863 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:19:58.864 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:19:58.864 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "dedc3508-4af4-40b8-bd43-6d3fb6c5d402", 2024-04-25T10:19:58.864 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:19:58.864 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:19:58.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effe0066f60 msgr2=0x7effe0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effe0066f60 0x7effe0069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7effec0ad000 tx=0x7effec008040 comp rx=0 tx=0).stop 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 msgr2=0x7f00000c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 0x7f00000c77e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efff400e870 tx=0x7efff400ec70 comp rx=0 tx=0).stop 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 shutdown_connections 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effe0066f60 0x7effe0069420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f000015db30 0x7f0000156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00000af020 0x7f00000c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 --2- 172.21.15.45:0/1659147167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f000006de00 0x7f00000c77e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 >> 172.21.15.45:0/1659147167 conn(0x7f00000b15c0 msgr2=0x7f000015a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 shutdown_connections 2024-04-25T10:19:58.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:58.860+0000 7f0006a00640 1 -- 172.21.15.45:0/1659147167 wait complete. 2024-04-25T10:19:58.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:58.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:19:59.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.072+0000 7f44d7400640 1 Processor -- start 2024-04-25T10:19:59.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.072+0000 7f44d7400640 1 -- start start 2024-04-25T10:19:59.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44d00b3360 0x7f44d00cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 0x7f44d00ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d7400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d015e920 con 0x7f44d015e1b0 2024-04-25T10:19:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d7400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d015f1c0 con 0x7f44d00b3360 2024-04-25T10:19:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d7400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00ada70 con 0x7f44d00adf00 2024-04-25T10:19:59.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 0x7f44d00ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 0x7f44d00ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:47020/0 (socket says 172.21.15.45:47020) 2024-04-25T10:19:59.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 -- 172.21.15.45:0/1594268672 learned_addr learned my addr 172.21.15.45:0/1594268672 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:59.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44cfe00640 1 --1- 172.21.15.45:0/1594268672 >> v1:172.21.15.117:6789/0 conn(0x7f44d015e1b0 0x7f44d0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:49558/0 (socket says 172.21.15.45:49558) 2024-04-25T10:19:59.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 -- 172.21.15.45:0/1594268672 >> v2:172.21.15.117:3300/0 conn(0x7f44d00b3360 msgr2=0x7f44d00cfc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:19:59.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 --2- 172.21.15.45:0/1594268672 >> v2:172.21.15.117:3300/0 conn(0x7f44d00b3360 0x7f44d00cfc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 -- 172.21.15.45:0/1594268672 >> v1:172.21.15.117:6789/0 conn(0x7f44d015e1b0 legacy=0x7f44d0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44cf400640 1 -- 172.21.15.45:0/1594268672 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759373046 0 0) 0x7f44d015e920 con 0x7f44d015e1b0 2024-04-25T10:19:59.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.076+0000 7f44d5400640 1 -- 172.21.15.45:0/1594268672 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d015ec00 con 0x7f44d00adf00 2024-04-25T10:19:59.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d5400640 1 --2- 172.21.15.45:0/1594268672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 0x7f44d00ae390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c001c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbe30dd7ffbb5cf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44cf400640 1 -- 172.21.15.45:0/1594268672 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00b2070 con 0x7f44d00adf00 2024-04-25T10:19:59.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44cf400640 1 -- 172.21.15.45:0/1594268672 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c00ad680 con 0x7f44d00adf00 2024-04-25T10:19:59.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44cf400640 1 -- 172.21.15.45:0/1594268672 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00ada50 con 0x7f44d00adf00 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 -- 172.21.15.45:0/1594268672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 msgr2=0x7f44d00ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 --2- 172.21.15.45:0/1594268672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 0x7f44d00ae390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c001c580 comp rx=0 tx=0).stop 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 -- 172.21.15.45:0/1594268672 shutdown_connections 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 --2- 172.21.15.45:0/1594268672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d00adf00 0x7f44d00ae390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 --2- 172.21.15.45:0/1594268672 >> v2:172.21.15.117:3300/0 conn(0x7f44d00b3360 0x7f44d00cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 -- 172.21.15.45:0/1594268672 >> 172.21.15.45:0/1594268672 conn(0x7f44d00b5480 msgr2=0x7f44d00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 -- 172.21.15.45:0/1594268672 shutdown_connections 2024-04-25T10:19:59.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.080+0000 7f44d7400640 1 -- 172.21.15.45:0/1594268672 wait complete. 2024-04-25T10:19:59.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 Processor -- start 2024-04-25T10:19:59.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 -- start start 2024-04-25T10:19:59.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d00adf00 0x7f44d0156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 0x7f44d0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d015e1b0 0x7f44d01439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0070be0 con 0x7f44d00b3360 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0070a60 con 0x7f44d00adf00 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d7400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d01579c0 con 0x7f44d015e1b0 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 0x7f44d0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 0x7f44d0152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43018/0 (socket says 172.21.15.45:43018) 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d00adf00 0x7f44d0156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 -- 172.21.15.45:0/130560788 learned_addr learned my addr 172.21.15.45:0/130560788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44d5400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d015e1b0 0x7f44d01439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 -- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d015e1b0 msgr2=0x7f44d01439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d015e1b0 0x7f44d01439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 -- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d00adf00 msgr2=0x7f44d0156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.084+0000 7f44cfe00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d00adf00 0x7f44d0156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44cfe00640 1 -- 172.21.15.45:0/130560788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d01513a0 con 0x7f44d00b3360 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44d4a00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d00adf00 0x7f44d0156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:59.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44cfe00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 0x7f44d0152440 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c480 tx=0x7f44bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44ccc00640 1 -- 172.21.15.45:0/130560788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc018070 con 0x7f44d00b3360 2024-04-25T10:19:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44ccc00640 1 -- 172.21.15.45:0/130560788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc007500 con 0x7f44d00b3360 2024-04-25T10:19:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d0151680 con 0x7f44d00b3360 2024-04-25T10:19:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44ccc00640 1 -- 172.21.15.45:0/130560788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc004a30 con 0x7f44d00b3360 2024-04-25T10:19:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.088+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d0151c40 con 0x7f44d00b3360 2024-04-25T10:19:59.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44ccc00640 1 -- 172.21.15.45:0/130560788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44bc01b070 con 0x7f44d00b3360 2024-04-25T10:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44ccc00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44b0066f90 0x7f44b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44ccc00640 1 -- 172.21.15.45:0/130560788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f44bc0843a0 con 0x7f44d00b3360 2024-04-25T10:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 Processor -- start 2024-04-25T10:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44d4a00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44b0066f90 0x7f44b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 -- start start 2024-04-25T10:19:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44d4a00640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44b0066f90 0x7f44b0069450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44d0153b20 tx=0x7f44c800d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbc1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc22f0 0x55793bb77b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b9c4ff0 con 0x55793bbc22f0 2024-04-25T10:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b94d150 con 0x55793bbc19d0 2024-04-25T10:19:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.092+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55793b9c1090 con 0x55793bb780a0 2024-04-25T10:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb78480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:47038/0 (socket says 172.21.15.45:47038) 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 -- 172.21.15.45:0/4070186958 learned_addr learned my addr 172.21.15.45:0/4070186958 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44cfe00640 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc22f0 0x55793bb77b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d4a00640 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbc1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 -- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 msgr2=0x55793bbc1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbc1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 -- 172.21.15.45:0/4070186958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc22f0 msgr2=0x55793bb77b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc22f0 0x55793bb77b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 -- 172.21.15.45:0/4070186958 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793bb5b0d0 con 0x55793bb780a0 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44d5400640 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb78480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c0004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53587be2defa54b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44c5600640 1 -- 172.21.15.45:0/4070186958 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00b2040 con 0x55793bb780a0 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44c5600640 1 -- 172.21.15.45:0/4070186958 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c00ad860 con 0x55793bb780a0 2024-04-25T10:19:59.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.096+0000 7f44c5600640 1 -- 172.21.15.45:0/4070186958 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00adc10 con 0x55793bb780a0 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 -- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 msgr2=0x55793bb78480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb78480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c0004740 comp rx=0 tx=0).stop 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 -- 172.21.15.45:0/4070186958 shutdown_connections 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb78480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc22f0 0x55793bb77b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 --2- 172.21.15.45:0/4070186958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbc1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 -- 172.21.15.45:0/4070186958 >> 172.21.15.45:0/4070186958 conn(0x55793bb74020 msgr2=0x55793bb76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 -- 172.21.15.45:0/4070186958 shutdown_connections 2024-04-25T10:19:59.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 -- 172.21.15.45:0/4070186958 wait complete. 2024-04-25T10:19:59.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 -- start start 2024-04-25T10:19:59.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.100+0000 7f44da8dd000 1 Processor -- start 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bb780a0 0x55793bbaf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbaf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbafa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b989250 con 0x55793bb780a0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b97f690 con 0x55793bbc19d0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55793b94da40 con 0x55793bbc22f0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 10 client.0 _ll_register_callbacks cb 0x55793bb62e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbafa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bb780a0 0x55793bbaf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbafa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:47040/0 (socket says 172.21.15.45:47040) 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 -- 172.21.15.45:0/1511671764 learned_addr learned my addr 172.21.15.45:0/1511671764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44cfe00640 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbaf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 -- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 msgr2=0x55793bbaf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbaf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 -- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bb780a0 msgr2=0x55793bbaf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bb780a0 0x55793bbaf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 -- 172.21.15.45:0/1511671764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793ba721c0 con 0x55793bbc22f0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d5400640 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbafa80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c001adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c3f9e9b788e2c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44ba000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44ba000640 1 -- 172.21.15.45:0/1511671764 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00c5070 con 0x55793bbc22f0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44ba000640 1 -- 172.21.15.45:0/1511671764 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f44c00adc10 con 0x55793bbc22f0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44c00ad860 con 0x55793bbc22f0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44ba000640 1 -- 172.21.15.45:0/1511671764 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44c00d0050 con 0x55793bbc22f0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 10 client.4934 fetch_fsmap learned FSMap version 18 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55793ba6e170 con 0x55793bbc22f0 2024-04-25T10:19:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44ba000640 1 -- 172.21.15.45:0/1511671764 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c00ad860 con 0x55793bbc22f0 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44ba000640 1 -- 172.21.15.45:0/1511671764 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f44c00d3020 con 0x55793bbc22f0 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 10 client.4934 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 10 client.4934 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 20 client.4934 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbd9d00 0x55793bbdc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 4 client.4934 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55793b960c60 con 0x55793bbd9d00 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.104+0000 7f44d4a00640 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbd9d00 0x55793bbdc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44d4a00640 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbd9d00 0x55793bbdc0f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427c36f8480a167b server_cookie=b74f7ba42941b446 in_seq=0 out_seq=0 2024-04-25T10:19:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44ba000640 10 client.4934 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:59.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44ba000640 1 -- 172.21.15.45:0/1511671764 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55793b960c60 con 0x55793bbd9d00 2024-04-25T10:19:59.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44ba000640 10 client.4934 handle_command_reply: tid=0 2024-04-25T10:19:59.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 client.4934 shutdown 2024-04-25T10:19:59.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 msgr2=0x55793bbafa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbafa80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c001adc0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 shutdown_connections 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbd9d00 0x55793bbdc0f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbafa80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc19d0 0x55793bbaf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1511671764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bb780a0 0x55793bbaf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44ba000640 0 client.4934 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 >> 172.21.15.45:0/1511671764 conn(0x55793bb74020 msgr2=0x55793bb76ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 shutdown_connections 2024-04-25T10:19:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 1 -- 172.21.15.45:0/1511671764 wait complete. 2024-04-25T10:19:59.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.108+0000 7f44da8dd000 20 client.4934 trim_cache size 0 max 16384 2024-04-25T10:19:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.136+0000 7f44da8dd000 1 Processor -- start 2024-04-25T10:19:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.136+0000 7f44da8dd000 1 -- start start 2024-04-25T10:19:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bbd6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc22f0 0x55793bbd73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00ab820 con 0x55793bbc19d0 2024-04-25T10:19:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00aa9a0 con 0x55793bbc22f0 2024-04-25T10:19:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0148410 con 0x55793bb780a0 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44d4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44d4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb77840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:47050/0 (socket says 172.21.15.45:47050) 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44d4a00640 1 -- 172.21.15.45:0/50548039 learned_addr learned my addr 172.21.15.45:0/50548039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bbd6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44d5400640 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc22f0 0x55793bbd73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 -- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 msgr2=0x55793bb77840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb77840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 -- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc22f0 msgr2=0x55793bbd73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc22f0 0x55793bbd73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 -- 172.21.15.45:0/50548039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793ba0bde0 con 0x55793bbc19d0 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44d5400640 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc22f0 0x55793bbd73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44cfe00640 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bbd6e80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0549d0 tx=0x7f44bc084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bd8658c28bdff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44c4c00640 1 -- 172.21.15.45:0/50548039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0828c0 con 0x55793bbc19d0 2024-04-25T10:19:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44c4c00640 1 -- 172.21.15.45:0/50548039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc052fd0 con 0x55793bbc19d0 2024-04-25T10:19:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44c4c00640 1 -- 172.21.15.45:0/50548039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc051340 con 0x55793bbc19d0 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- 172.21.15.45:0/50548039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 msgr2=0x55793bbd6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bbd6e80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0549d0 tx=0x7f44bc084040 comp rx=0 tx=0).stop 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- 172.21.15.45:0/50548039 shutdown_connections 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bbc22f0 0x55793bbd73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bbd6e80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 --2- 172.21.15.45:0/50548039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bb780a0 0x55793bb77840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- 172.21.15.45:0/50548039 >> 172.21.15.45:0/50548039 conn(0x55793bb74020 msgr2=0x55793bbd6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- 172.21.15.45:0/50548039 shutdown_connections 2024-04-25T10:19:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.140+0000 7f44da8dd000 1 -- 172.21.15.45:0/50548039 wait complete. 2024-04-25T10:19:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 -- start start 2024-04-25T10:19:59.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 Processor -- start 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bb780a0 0x55793bc2a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bc2af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbdcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b95fe00 con 0x55793bbc19d0 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0154f50 con 0x55793bb780a0 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55793b9c4300 con 0x55793bbc22f0 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44da8dd000 10 client.0 _ll_register_callbacks cb 0x55793bb43040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbdcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbdcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:47064/0 (socket says 172.21.15.45:47064) 2024-04-25T10:19:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 -- 172.21.15.45:0/1683007555 learned_addr learned my addr 172.21.15.45:0/1683007555 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:19:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44cfe00640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bc2af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 -- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bb780a0 msgr2=0x55793bc2a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bb780a0 0x55793bc2a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 -- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 msgr2=0x55793bc2af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bc2af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.144+0000 7f44d5400640 1 -- 172.21.15.45:0/1683007555 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793b9668e0 con 0x55793bbc22f0 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44cfe00640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bc2af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44d5400640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbdcd70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44c001ce20 tx=0x7f44c00a7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd3ff4076e5f2ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44ba000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44ba000640 1 -- 172.21.15.45:0/1683007555 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c00c5040 con 0x55793bbc22f0 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44ba000640 1 -- 172.21.15.45:0/1683007555 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f44c00b8b60 con 0x55793bbc22f0 2024-04-25T10:19:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44bc052fd0 con 0x55793bbc22f0 2024-04-25T10:19:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.148+0000 7f44ba000640 1 -- 172.21.15.45:0/1683007555 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44c00b2040 con 0x55793bbc22f0 2024-04-25T10:19:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 10 client.4940 fetch_fsmap learned FSMap version 18 2024-04-25T10:19:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55793ba1e8f0 con 0x55793bbc22f0 2024-04-25T10:19:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44ba000640 1 -- 172.21.15.45:0/1683007555 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc052fd0 con 0x55793bbc22f0 2024-04-25T10:19:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44ba000640 1 -- 172.21.15.45:0/1683007555 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f44c00d3040 con 0x55793bbc22f0 2024-04-25T10:19:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 10 client.4940 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:19:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 10 client.4940 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:19:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 20 client.4940 populate_metadata read hostname 'smithi045' 2024-04-25T10:19:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbdd760 0x55793bbddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 4 client.4940 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:19:59.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55793b9d2530 con 0x55793bbdd760 2024-04-25T10:19:59.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44d4a00640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbdd760 0x55793bbddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44d4a00640 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbdd760 0x55793bbddb70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84692c7c1853df7 server_cookie=696b2fbd526b3802 in_seq=0 out_seq=0 2024-04-25T10:19:59.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44ba000640 10 client.4940 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:59.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44ba000640 1 -- 172.21.15.45:0/1683007555 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+273 (crc 0 0 0) 0x55793b9d2530 con 0x55793bbdd760 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44ba000640 10 client.4940 handle_command_reply: tid=0 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 client.4940 shutdown 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 msgr2=0x55793bbdcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbdcd70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44c001ce20 tx=0x7f44c00a7000 comp rx=0 tx=0).stop 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 shutdown_connections 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55793bbdd760 0x55793bbddb70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55793bbc22f0 0x55793bbdcd70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55793bbc19d0 0x55793bc2af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 --2- 172.21.15.45:0/1683007555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55793bb780a0 0x55793bc2a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 >> 172.21.15.45:0/1683007555 conn(0x55793bb74020 msgr2=0x55793bbd7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.152+0000 7f44ba000640 0 client.4940 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 shutdown_connections 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44da8dd000 1 -- 172.21.15.45:0/1683007555 wait complete. 2024-04-25T10:19:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44da8dd000 20 client.4940 trim_cache size 0 max 16384 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (86 inodes in the stack)", 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: "dedc3508-4af4-40b8-bd43-6d3fb6c5d402": { 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "dedc3508-4af4-40b8-bd43-6d3fb6c5d402", 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:19:59.189 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:19:59.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44b0066f90 msgr2=0x7f44b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44b0066f90 0x7f44b0069450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44d0153b20 tx=0x7f44c800d040 comp rx=0 tx=0).stop 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 msgr2=0x7f44d0152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 0x7f44d0152440 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c480 tx=0x7f44bc00c940 comp rx=0 tx=0).stop 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 shutdown_connections 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44b0066f90 0x7f44b0069450 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d015e1b0 0x7f44d01439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44d00b3360 0x7f44d0152440 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 --2- 172.21.15.45:0/130560788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d00adf00 0x7f44d0156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 >> 172.21.15.45:0/130560788 conn(0x7f44d00b5480 msgr2=0x7f44d00b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 shutdown_connections 2024-04-25T10:19:59.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:19:59.156+0000 7f44d7400640 1 -- 172.21.15.45:0/130560788 wait complete. 2024-04-25T10:19:59.191 INFO:tasks.cephfs.filesystem:scrub status for tag:dedc3508-4af4-40b8-bd43-6d3fb6c5d402 - {'path': '/', 'tag': 'dedc3508-4af4-40b8-bd43-6d3fb6c5d402', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T10:20:27.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:27.565 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:29.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:29.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:20:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.343+0000 7ff50fa00640 1 Processor -- start 2024-04-25T10:20:29.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.343+0000 7ff50fa00640 1 -- start start 2024-04-25T10:20:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50fa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff508159540 0x7ff5080ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50fa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff508159920 con 0x7ff5080aebd0 2024-04-25T10:20:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50fa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50815c2f0 con 0x7ff5080721f0 2024-04-25T10:20:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50fa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50815c5d0 con 0x7ff508159540 2024-04-25T10:20:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff4ffe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5080aebd0 0x7ff508071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40488/0 (socket says 172.21.15.45:40488) 2024-04-25T10:20:29.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff4ffe00640 1 -- 172.21.15.45:0/3678204908 learned_addr learned my addr 172.21.15.45:0/3678204908 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 --2- 172.21.15.45:0/3678204908 >> v2:172.21.15.117:3300/0 conn(0x7ff508159540 0x7ff5080ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307102105 0 0) 0x7ff508159920 con 0x7ff5080aebd0 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ec003530 con 0x7ff5080aebd0 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1412864103 0 0) 0x7ff4ec003530 con 0x7ff5080aebd0 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 >> v2:172.21.15.117:3300/0 conn(0x7ff508159540 msgr2=0x7ff5080ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 --2- 172.21.15.45:0/3678204908 >> v2:172.21.15.117:3300/0 conn(0x7ff508159540 0x7ff5080ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 >> v1:172.21.15.117:6789/0 conn(0x7ff5080721f0 legacy=0x7ff508070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50815c8b0 con 0x7ff5080aebd0 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7ff4f0001680 con 0x7ff5080aebd0 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 >> v1:172.21.15.45:6789/0 conn(0x7ff5080aebd0 legacy=0x7ff508071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4ec0039c0 0x7ff4ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50815c5d0 con 0x7ff4ec0039c0 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec00c070 con 0x7ff4ec004810 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec00c270 con 0x7ff4ec0051d0 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4ec0039c0 0x7ff4ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50da00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 -- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0051d0 msgr2=0x7ff4ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0051d0 0x7ff4ec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 -- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ec004810 msgr2=0x7ff4ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50d000640 1 -- 172.21.15.45:0/3678204908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec003530 con 0x7ff4ec0039c0 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff50da00640 1 -- 172.21.15.45:0/3678204908 reap_dead start 2024-04-25T10:20:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.347+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4ec004810 0x7ff4ec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50d000640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4ec0039c0 0x7ff4ec003de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009ae0 tx=0x7ff4f801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dea7edec4eb147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80096b0 con 0x7ff4ec0039c0 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f80a6500 con 0x7ff4ec0039c0 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff507e00640 1 -- 172.21.15.45:0/3678204908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80a68f0 con 0x7ff4ec0039c0 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 -- 172.21.15.45:0/3678204908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4ec0039c0 msgr2=0x7ff4ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4ec0039c0 0x7ff4ec003de0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009ae0 tx=0x7ff4f801ca90 comp rx=0 tx=0).stop 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 -- 172.21.15.45:0/3678204908 shutdown_connections 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 --2- 172.21.15.45:0/3678204908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4ec0039c0 0x7ff4ec003de0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 -- 172.21.15.45:0/3678204908 >> 172.21.15.45:0/3678204908 conn(0x7ff5080669a0 msgr2=0x7ff508066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 -- 172.21.15.45:0/3678204908 shutdown_connections 2024-04-25T10:20:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 -- 172.21.15.45:0/3678204908 wait complete. 2024-04-25T10:20:29.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.351+0000 7ff50fa00640 1 Processor -- start 2024-04-25T10:20:29.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 -- start start 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0039c0 0x7ff5081fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 0x7ff508154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50815b010 0x7ff50815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508072070 con 0x7ff5081f77a0 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508071ef0 con 0x7ff50815b010 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50815fda0 con 0x7ff4ec0039c0 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 0x7ff508154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 0x7ff508154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53118/0 (socket says 172.21.15.45:53118) 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1162709245 learned_addr learned my addr 172.21.15.45:0/1162709245 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50d000640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0039c0 0x7ff5081fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50da00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50815b010 0x7ff50815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0039c0 msgr2=0x7ff5081fadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0039c0 0x7ff5081fadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50815b010 msgr2=0x7ff50815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50815b010 0x7ff50815b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1162709245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5081fbc40 con 0x7ff5081f77a0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 0x7ff508154240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000e7b0 tx=0x7ff4f000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff505600640 1 -- 172.21.15.45:0/1162709245 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000c660 con 0x7ff5081f77a0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff505600640 1 -- 172.21.15.45:0/1162709245 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0018070 con 0x7ff5081f77a0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5081fb3a0 con 0x7ff5081f77a0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff505600640 1 -- 172.21.15.45:0/1162709245 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000ca00 con 0x7ff5081f77a0 2024-04-25T10:20:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.355+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5081fb960 con 0x7ff5081f77a0 2024-04-25T10:20:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff505600640 1 -- 172.21.15.45:0/1162709245 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4f000cba0 con 0x7ff5081f77a0 2024-04-25T10:20:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff505600640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4e4066f90 0x7ff4e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff505600640 1 -- 172.21.15.45:0/1162709245 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff4f0084cd0 con 0x7ff5081f77a0 2024-04-25T10:20:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff50d000640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4e4066f90 0x7ff4e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 Processor -- start 2024-04-25T10:20:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 -- start start 2024-04-25T10:20:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff50d000640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4e4066f90 0x7ff4e4069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80af000 tx=0x7ff4f80046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637ad93db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ad49b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637ad4a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637ab96ff0 con 0x55637ad942f0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637ab1f150 con 0x55637ad4a0a0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637ab93090 con 0x55637ad939d0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ad49b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ad49b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53130/0 (socket says 172.21.15.45:53130) 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 -- 172.21.15.45:0/335617267 learned_addr learned my addr 172.21.15.45:0/335617267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff50d000640 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637ad93db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff50da00640 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637ad4a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 -- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 msgr2=0x55637ad93db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637ad93db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 -- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 msgr2=0x55637ad4a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637ad4a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 -- 172.21.15.45:0/335617267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55637ad2d0d0 con 0x55637ad942f0 2024-04-25T10:20:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ad49b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00519c0 tx=0x7ff4f0055560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b774f227dfcebe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4fd600640 1 -- 172.21.15.45:0/335617267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f005dbc0 con 0x55637ad942f0 2024-04-25T10:20:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4fd600640 1 -- 172.21.15.45:0/335617267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0053860 con 0x55637ad942f0 2024-04-25T10:20:29.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.359+0000 7ff4fd600640 1 -- 172.21.15.45:0/335617267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0059170 con 0x55637ad942f0 2024-04-25T10:20:29.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/335617267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 msgr2=0x55637ad49b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ad49b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00519c0 tx=0x7ff4f0055560 comp rx=0 tx=0).stop 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/335617267 shutdown_connections 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637ad4a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ad49b60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- 172.21.15.45:0/335617267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637ad93db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/335617267 >> 172.21.15.45:0/335617267 conn(0x55637ad46020 msgr2=0x55637ad48440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/335617267 shutdown_connections 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/335617267 wait complete. 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- start start 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 Processor -- start 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad78d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637ab5b250 con 0x55637ad4a0a0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637ab51690 con 0x55637ad939d0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637ab1fa40 con 0x55637ad942f0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 10 client.0 _ll_register_callbacks cb 0x55637ad34e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad787f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53140/0 (socket says 172.21.15.45:53140) 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 -- 172.21.15.45:0/2806702675 learned_addr learned my addr 172.21.15.45:0/2806702675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad78d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50da00640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 -- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 msgr2=0x55637ad79270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad79270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 -- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 msgr2=0x55637ad78d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad78d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 -- 172.21.15.45:0/2806702675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55637ac441c0 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff50d000640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad787f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8000c10 tx=0x7ff4f8019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbfe919cdc4e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff4f6000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff4f6000640 1 -- 172.21.15.45:0/2806702675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80bd040 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff4f6000640 1 -- 172.21.15.45:0/2806702675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff4f8004460 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4f0053860 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff4f6000640 1 -- 172.21.15.45:0/2806702675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4f80d1050 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 10 client.5343 fetch_fsmap learned FSMap version 18 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.363+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55637ac40170 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4f6000640 1 -- 172.21.15.45:0/2806702675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0053860 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4f6000640 1 -- 172.21.15.45:0/2806702675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff4f80d4020 con 0x55637ad4a0a0 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 10 client.5343 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 10 client.5343 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 20 client.5343 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada3c80 0x55637ada6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 4 client.5343 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55637ab32c60 con 0x55637ada3c80 2024-04-25T10:20:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada3c80 0x55637ada6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4f6000640 10 client.5343 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada3c80 0x55637ada6070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7ce8c01d368977 server_cookie=e16187408e6ad377 in_seq=0 out_seq=0 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4f6000640 1 -- 172.21.15.45:0/2806702675 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55637ab32c60 con 0x55637ada3c80 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4f6000640 10 client.5343 handle_command_reply: tid=0 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 client.5343 shutdown 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 msgr2=0x55637ad787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad787f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8000c10 tx=0x7ff4f8019830 comp rx=0 tx=0).stop 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 shutdown_connections 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada3c80 0x55637ada6070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad79270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad78d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 --2- 172.21.15.45:0/2806702675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad787f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff4f6000640 0 client.5343 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 >> 172.21.15.45:0/2806702675 conn(0x55637ad46020 msgr2=0x55637ad48ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 shutdown_connections 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 1 -- 172.21.15.45:0/2806702675 wait complete. 2024-04-25T10:20:29.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.367+0000 7ff512f59000 20 client.5343 trim_cache size 0 max 16384 2024-04-25T10:20:29.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 Processor -- start 2024-04-25T10:20:29.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- start start 2024-04-25T10:20:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508203e40 con 0x55637ad4a0a0 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff508203150 con 0x55637ad939d0 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080b2130 con 0x55637ad942f0 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad92e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53144/0 (socket says 172.21.15.45:53144) 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 -- 172.21.15.45:0/1010524471 learned_addr learned my addr 172.21.15.45:0/1010524471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 -- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 msgr2=0x55637ad49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 -- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 msgr2=0x55637ad49420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad49420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 -- 172.21.15.45:0/1010524471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55637abddde0 con 0x55637ad4a0a0 2024-04-25T10:20:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad49420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff50d000640 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad92e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff4f801d330 tx=0x7ff4f80c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746ac2804865ba4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff4fcc00640 1 -- 172.21.15.45:0/1010524471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80bd040 con 0x55637ad4a0a0 2024-04-25T10:20:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff4fcc00640 1 -- 172.21.15.45:0/1010524471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f80c9070 con 0x55637ad4a0a0 2024-04-25T10:20:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff4fcc00640 1 -- 172.21.15.45:0/1010524471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8003c70 con 0x55637ad4a0a0 2024-04-25T10:20:29.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- 172.21.15.45:0/1010524471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 msgr2=0x55637ad92e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad92e10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff4f801d330 tx=0x7ff4f80c2000 comp rx=0 tx=0).stop 2024-04-25T10:20:29.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- 172.21.15.45:0/1010524471 shutdown_connections 2024-04-25T10:20:29.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad942f0 0x55637ad49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad939d0 0x55637ad49420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 --2- 172.21.15.45:0/1010524471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad4a0a0 0x55637ad92e10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- 172.21.15.45:0/1010524471 >> 172.21.15.45:0/1010524471 conn(0x55637ad46020 msgr2=0x55637ad920d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.383+0000 7ff512f59000 1 -- 172.21.15.45:0/1010524471 shutdown_connections 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- 172.21.15.45:0/1010524471 wait complete. 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- start start 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 Processor -- start 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637adf4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637adf4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ada6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55637ab31e00 con 0x55637ad942f0 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080b3ba0 con 0x55637ad4a0a0 2024-04-25T10:20:29.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55637ab96300 con 0x55637ad939d0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 10 client.0 _ll_register_callbacks cb 0x55637ad15040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ada6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ada6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53160/0 (socket says 172.21.15.45:53160) 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 -- 172.21.15.45:0/629905036 learned_addr learned my addr 172.21.15.45:0/629905036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50d000640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637adf4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4ffe00640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637adf4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 -- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 msgr2=0x55637adf4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637adf4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 -- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 msgr2=0x55637adf4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637adf4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 -- 172.21.15.45:0/629905036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55637ab388e0 con 0x55637ad942f0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50da00640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ada6380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff500005730 tx=0x7ff500001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5bbb0b233d5fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4f6000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4f6000640 1 -- 172.21.15.45:0/629905036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50001a070 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4f6000640 1 -- 172.21.15.45:0/629905036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff500016630 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4f80c9070 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4f6000640 1 -- 172.21.15.45:0/629905036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff500025050 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 10 client.5349 fetch_fsmap learned FSMap version 18 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55637abf08f0 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4f6000640 1 -- 172.21.15.45:0/629905036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f80c9070 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff4f6000640 1 -- 172.21.15.45:0/629905036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff500028020 con 0x55637ad942f0 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 10 client.5349 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 10 client.5349 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 20 client.5349 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:29.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada7060 0x55637ada7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 4 client.5349 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:20:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55637aba4530 con 0x55637ada7060 2024-04-25T10:20:29.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.387+0000 7ff50d000640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada7060 0x55637ada7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50d000640 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada7060 0x55637ada7470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52faa3be3e3f2c48 server_cookie=6fe5419128d3e1ef in_seq=0 out_seq=0 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff4f6000640 10 client.5349 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff4f6000640 1 -- 172.21.15.45:0/629905036 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55637aba4530 con 0x55637ada7060 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff4f6000640 10 client.5349 handle_command_reply: tid=0 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 client.5349 shutdown 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 msgr2=0x55637ada6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ada6380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff500005730 tx=0x7ff500001b30 comp rx=0 tx=0).stop 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 shutdown_connections 2024-04-25T10:20:29.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55637ada7060 0x55637ada7470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55637ad942f0 0x55637ada6380 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55637ad939d0 0x55637adf4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 --2- 172.21.15.45:0/629905036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55637ad4a0a0 0x55637adf4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 >> 172.21.15.45:0/629905036 conn(0x55637ad46020 msgr2=0x55637ad92960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff4f6000640 0 client.5349 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 shutdown_connections 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 1 -- 172.21.15.45:0/629905036 wait complete. 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff512f59000 20 client.5349 trim_cache size 0 max 16384 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:20:29.398 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:20:29.399 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:20:29.399 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:20:29.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4e4066f90 msgr2=0x7ff4e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4e4066f90 0x7ff4e4069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80af000 tx=0x7ff4f80046d0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 msgr2=0x7ff508154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 0x7ff508154240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000e7b0 tx=0x7ff4f000ec70 comp rx=0 tx=0).stop 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 shutdown_connections 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4e4066f90 0x7ff4e4069450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff50815b010 0x7ff50815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5081f77a0 0x7ff508154240 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 --2- 172.21.15.45:0/1162709245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4ec0039c0 0x7ff5081fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 >> 172.21.15.45:0/1162709245 conn(0x7ff5080669a0 msgr2=0x7ff508156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 shutdown_connections 2024-04-25T10:20:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.391+0000 7ff50fa00640 1 -- 172.21.15.45:0/1162709245 wait complete. 2024-04-25T10:20:29.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:20:29.413 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:20:29.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:20:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.551+0000 7f1a62600640 1 Processor -- start 2024-04-25T10:20:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.551+0000 7f1a62600640 1 -- start start 2024-04-25T10:20:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a62600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a5c159540 0x7f1a5c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a62600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a5c0aebd0 0x7f1a5c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a62600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c071360 con 0x7f1a5c072020 2024-04-25T10:20:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a62600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c159920 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a62600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c15c2f0 con 0x7f1a5c159540 2024-04-25T10:20:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a53e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a5c0aebd0 0x7f1a5c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a5be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1a5c072020 0x7f1a5c15f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40490/0 (socket says 172.21.15.45:40490) 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a5be00640 1 -- 172.21.15.45:0/2783200246 learned_addr learned my addr 172.21.15.45:0/2783200246 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a5b400640 1 --2- 172.21.15.45:0/2783200246 >> v2:172.21.15.117:3300/0 conn(0x7f1a5c159540 0x7f1a5c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a53e00640 1 -- 172.21.15.45:0/2783200246 >> v2:172.21.15.117:3300/0 conn(0x7f1a5c159540 msgr2=0x7f1a5c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a53e00640 1 --2- 172.21.15.45:0/2783200246 >> v2:172.21.15.117:3300/0 conn(0x7f1a5c159540 0x7f1a5c0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a53e00640 1 -- 172.21.15.45:0/2783200246 >> v1:172.21.15.45:6789/0 conn(0x7f1a5c072020 legacy=0x7f1a5c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a53e00640 1 -- 172.21.15.45:0/2783200246 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c15c5d0 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.555+0000 7f1a5b400640 1 --2- 172.21.15.45:0/2783200246 >> v2:172.21.15.117:3300/0 conn(0x7f1a5c159540 0x7f1a5c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a53e00640 1 --2- 172.21.15.45:0/2783200246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c071ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0b4c20 tx=0x7f1a4401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da73261011c6a130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a5aa00640 1 -- 172.21.15.45:0/2783200246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440b0070 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a5aa00640 1 -- 172.21.15.45:0/2783200246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a440ab600 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a5aa00640 1 -- 172.21.15.45:0/2783200246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440ab9d0 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 -- 172.21.15.45:0/2783200246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 msgr2=0x7f1a5c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 --2- 172.21.15.45:0/2783200246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c071ae0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0b4c20 tx=0x7f1a4401c480 comp rx=0 tx=0).stop 2024-04-25T10:20:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 -- 172.21.15.45:0/2783200246 shutdown_connections 2024-04-25T10:20:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 --2- 172.21.15.45:0/2783200246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c071ae0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 --2- 172.21.15.45:0/2783200246 >> v2:172.21.15.117:3300/0 conn(0x7f1a5c159540 0x7f1a5c0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 -- 172.21.15.45:0/2783200246 >> 172.21.15.45:0/2783200246 conn(0x7f1a5c0669a0 msgr2=0x7f1a5c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 -- 172.21.15.45:0/2783200246 shutdown_connections 2024-04-25T10:20:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 -- 172.21.15.45:0/2783200246 wait complete. 2024-04-25T10:20:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 Processor -- start 2024-04-25T10:20:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.559+0000 7f1a62600640 1 -- start start 2024-04-25T10:20:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a5c072020 0x7f1a5c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a5c159540 0x7f1a5c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c072430 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c06c0d0 con 0x7f1a5c072020 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c168930 con 0x7f1a5c159540 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c14faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53172/0 (socket says 172.21.15.45:53172) 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 -- 172.21.15.45:0/2297137742 learned_addr learned my addr 172.21.15.45:0/2297137742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a5b400640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a5c072020 0x7f1a5c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a5be00640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a5c159540 0x7f1a5c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 -- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a5c159540 msgr2=0x7f1a5c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a5c159540 0x7f1a5c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 -- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a5c072020 msgr2=0x7f1a5c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a5c072020 0x7f1a5c15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 -- 172.21.15.45:0/2297137742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c1f6e20 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a5be00640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a5c159540 0x7f1a5c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53e00640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c14faf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1a440ae040 tx=0x7f1a44019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53400640 1 -- 172.21.15.45:0/2297137742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440b0040 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53400640 1 -- 172.21.15.45:0/2297137742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a440b9070 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c1f7100 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53400640 1 -- 172.21.15.45:0/2297137742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440b4690 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c1f76c0 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.563+0000 7f1a53400640 1 -- 172.21.15.45:0/2297137742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a440b4830 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a53400640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a38066f60 0x7f1a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a53400640 1 -- 172.21.15.45:0/2297137742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1a4411ee10 con 0x7f1a5c0aebd0 2024-04-25T10:20:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a38066f60 0x7f1a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 Processor -- start 2024-04-25T10:20:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 -- start start 2024-04-25T10:20:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a38066f60 0x7f1a38069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c15b6e0 tx=0x7f1a4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892227e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8921e2d90 0x55f8921e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8920c51f0 con 0x55f892227a40 2024-04-25T10:20:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89202be90 con 0x55f8921e2d90 2024-04-25T10:20:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f891fb3f50 con 0x55f892228360 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892227e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892227e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53176/0 (socket says 172.21.15.45:53176) 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 -- 172.21.15.45:0/1129475223 learned_addr learned my addr 172.21.15.45:0/1129475223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5be00640 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8921e2d90 0x55f8921e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a53e00640 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 -- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 msgr2=0x55f8921e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 -- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8921e2d90 msgr2=0x55f8921e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8921e2d90 0x55f8921e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 -- 172.21.15.45:0/1129475223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8920038b0 con 0x55f892227a40 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a5b400640 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892227e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c010ab0 tx=0x7f1a4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2348f722898ebef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a4be00640 1 -- 172.21.15.45:0/1129475223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c01c050 con 0x55f892227a40 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a4be00640 1 -- 172.21.15.45:0/1129475223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c026070 con 0x55f892227a40 2024-04-25T10:20:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.567+0000 7f1a4be00640 1 -- 172.21.15.45:0/1129475223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c016e10 con 0x55f892227a40 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/1129475223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 msgr2=0x55f892227e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892227e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c010ab0 tx=0x7f1a4c013000 comp rx=0 tx=0).stop 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/1129475223 shutdown_connections 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f8921e2d90 0x55f8921e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- 172.21.15.45:0/1129475223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892227e20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/1129475223 >> 172.21.15.45:0/1129475223 conn(0x55f8921ded40 msgr2=0x55f8921e1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/1129475223 shutdown_connections 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/1129475223 wait complete. 2024-04-25T10:20:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- start start 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 Processor -- start 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f89222f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892227a40 0x55f89222f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f891fe6540 con 0x55f8921e2d90 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f892027f30 con 0x55f892227a40 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f891fe7ff0 con 0x55f892228360 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 10 client.0 _ll_register_callbacks cb 0x55f8920e0180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f89222f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f89222f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53180/0 (socket says 172.21.15.45:53180) 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 -- 172.21.15.45:0/65748825 learned_addr learned my addr 172.21.15.45:0/65748825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a53e00640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892227a40 0x55f89222f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5be00640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 -- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 msgr2=0x55f8921e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 -- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892227a40 msgr2=0x55f89222f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892227a40 0x55f89222f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 -- 172.21.15.45:0/65748825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f891fb41a0 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a53e00640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892227a40 0x55f89222f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5be00640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a5b400640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f89222f1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55f8921b1a00 tx=0x7f1a4c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8042881b1f0b377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a48c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a48c00640 1 -- 172.21.15.45:0/65748825 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0217f0 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a48c00640 1 -- 172.21.15.45:0/65748825 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1a4c036b70 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a4c026070 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a48c00640 1 -- 172.21.15.45:0/65748825 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4c01c020 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 10 client.5364 fetch_fsmap learned FSMap version 18 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.571+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55f8920ae840 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a48c00640 1 -- 172.21.15.45:0/65748825 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c026070 con 0x55f8921e2d90 2024-04-25T10:20:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a48c00640 1 -- 172.21.15.45:0/65748825 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a4c021990 con 0x55f8921e2d90 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 10 client.5364 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 10 client.5364 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 20 client.5364 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f8921e9c80 0x55f8921ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 4 client.5364 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f891fc7be0 con 0x55f8921e9c80 2024-04-25T10:20:29.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a53e00640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f8921e9c80 0x55f8921ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a53e00640 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f8921e9c80 0x55f8921ea060 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ccd8054dc13ecf3 server_cookie=a660b20f1313dd30 in_seq=0 out_seq=0 2024-04-25T10:20:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a48c00640 10 client.5364 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a48c00640 1 -- 172.21.15.45:0/65748825 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f891fc7be0 con 0x55f8921e9c80 2024-04-25T10:20:29.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a48c00640 10 client.5364 handle_command_reply: tid=0 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 client.5364 shutdown 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 msgr2=0x55f89222f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f89222f1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x55f8921b1a00 tx=0x7f1a4c004740 comp rx=0 tx=0).stop 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 shutdown_connections 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f8921e9c80 0x55f8921ea060 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892228360 0x55f8921e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892227a40 0x55f89222f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 --2- 172.21.15.45:0/65748825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f89222f1c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 >> 172.21.15.45:0/65748825 conn(0x55f8921ded40 msgr2=0x55f8921e19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a48c00640 0 client.5364 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 shutdown_connections 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 1 -- 172.21.15.45:0/65748825 wait complete. 2024-04-25T10:20:29.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.575+0000 7f1a659db000 20 client.5364 trim_cache size 0 max 16384 2024-04-25T10:20:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 Processor -- start 2024-04-25T10:20:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- start start 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f8921e2d90 0x55f8922848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892284e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f892285340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c202f80 con 0x55f892227a40 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c15a810 con 0x55f892228360 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c159c10 con 0x55f8921e2d90 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892284e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892284e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53194/0 (socket says 172.21.15.45:53194) 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 -- 172.21.15.45:0/1260270565 learned_addr learned my addr 172.21.15.45:0/1260270565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a5b400640 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f8921e2d90 0x55f8922848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a5be00640 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f892285340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 -- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f8921e2d90 msgr2=0x55f8922848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f8921e2d90 0x55f8922848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 -- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 msgr2=0x55f892285340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f892285340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 -- 172.21.15.45:0/1260270565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f892085a10 con 0x55f892227a40 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a5b400640 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f8921e2d90 0x55f8922848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a53e00640 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892284e00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1a440ef610 tx=0x7f1a4411d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95ee52e04b78438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a4b400640 1 -- 172.21.15.45:0/1260270565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440f37d0 con 0x55f892227a40 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a4b400640 1 -- 172.21.15.45:0/1260270565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a440f7d90 con 0x55f892227a40 2024-04-25T10:20:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a4b400640 1 -- 172.21.15.45:0/1260270565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4411d320 con 0x55f892227a40 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- 172.21.15.45:0/1260270565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 msgr2=0x55f892284e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892284e00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1a440ef610 tx=0x7f1a4411d170 comp rx=0 tx=0).stop 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- 172.21.15.45:0/1260270565 shutdown_connections 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f892285340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f892227a40 0x55f892284e00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 --2- 172.21.15.45:0/1260270565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f8921e2d90 0x55f8922848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- 172.21.15.45:0/1260270565 >> 172.21.15.45:0/1260270565 conn(0x55f8921ded40 msgr2=0x55f892226240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- 172.21.15.45:0/1260270565 shutdown_connections 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- 172.21.15.45:0/1260270565 wait complete. 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.591+0000 7f1a659db000 1 -- start start 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 Processor -- start 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f8922884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892227a40 0x55f8922889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f89222ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8921debc0 con 0x55f8921e2d90 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c15a210 con 0x55f892228360 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f891ff87c0 con 0x55f892227a40 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 10 client.0 _ll_register_callbacks cb 0x55f891feea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f8922884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f8922884b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53196/0 (socket says 172.21.15.45:53196) 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 -- 172.21.15.45:0/236910864 learned_addr learned my addr 172.21.15.45:0/236910864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a53e00640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892227a40 0x55f8922889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5be00640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f89222ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 -- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892227a40 msgr2=0x55f8922889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892227a40 0x55f8922889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 -- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 msgr2=0x55f89222ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f89222ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 -- 172.21.15.45:0/236910864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f891fc0b00 con 0x55f8921e2d90 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a5b400640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f8922884b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0047c0 tx=0x7f1a4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfac3f830c4d7850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 1 -- 172.21.15.45:0/236910864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c035640 con 0x55f8921e2d90 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a440f7d90 con 0x55f8921e2d90 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 1 -- 172.21.15.45:0/236910864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1a4c0357e0 con 0x55f8921e2d90 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 1 -- 172.21.15.45:0/236910864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4c04e050 con 0x55f8921e2d90 2024-04-25T10:20:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 10 client.5370 fetch_fsmap learned FSMap version 18 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55f891fd2f80 con 0x55f8921e2d90 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 1 -- 172.21.15.45:0/236910864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a440f7d90 con 0x55f8921e2d90 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 1 -- 172.21.15.45:0/236910864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a4c051020 con 0x55f8921e2d90 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 10 client.5370 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 10 client.5370 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 20 client.5370 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f89222d890 0x55f89222dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 4 client.5370 mds_command: new command op to 5055 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 --> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55f8920393d0 con 0x55f89222d890 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a53e00640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f89222d890 0x55f89222dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a53e00640 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f89222d890 0x55f89222dca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d9177217c03fac2 server_cookie=e8e7fa8d88489671 in_seq=0 out_seq=0 2024-04-25T10:20:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.595+0000 7f1a48c00640 10 client.5370 ms_handle_connect on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a48c00640 1 -- 172.21.15.45:0/236910864 <== mds.0 v2:172.21.15.45:6833/3482819589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17929 (crc 0 0 0) 0x55f8920393d0 con 0x55f89222d890 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a48c00640 10 client.5370 handle_command_reply: tid=0 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 client.5370 shutdown 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 msgr2=0x55f8922884b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f8922884b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0047c0 tx=0x7f1a4c016000 comp rx=0 tx=0).stop 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 shutdown_connections 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:6833/3482819589,v1:172.21.15.45:6835/3482819589] conn(0x55f89222d890 0x55f89222dca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f892228360 0x55f89222ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f892227a40 0x55f8922889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 --2- 172.21.15.45:0/236910864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8921e2d90 0x55f8922884b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a48c00640 0 client.5370 ms_handle_reset on v2:172.21.15.45:6833/3482819589 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 >> 172.21.15.45:0/236910864 conn(0x55f8921ded40 msgr2=0x55f892226ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 shutdown_connections 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 1 -- 172.21.15.45:0/236910864 wait complete. 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a659db000 20 client.5370 trim_cache size 0 max 16384 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 30688, 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 51594, 2024-04-25T10:20:29.606 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 11057694, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 45187005, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 18, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 6, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 5.102819443, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 2.833463935, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 2.388469255, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.389681650, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 51594, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.680120575, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000032564 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.607 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 72361, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.016598575, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000229 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 119152, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 26320 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3902, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 3921, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1357501, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 45890716, 2024-04-25T10:20:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.811793833, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.218610095, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.087997763, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.477446523, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3920, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.702279028, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000179152 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 88496 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3956, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 3957, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 997489, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 26718807, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.402686630, 2024-04-25T10:20:29.610 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.185650218, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.084215846, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.106356021, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3956, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.715326737, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000180820 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:20:29.611 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5595, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 14.440493114, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002580963 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.612 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3200, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.834461485, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000260769 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 26248, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 26248, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 26248, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 270.370894641, 2024-04-25T10:20:29.613 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.010300628 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 12, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 17711, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 17711, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-25T10:20:29.614 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 17711, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 2, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 2, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 2, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 49234, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 48718, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 147, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-25T10:20:29.615 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 1696, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 1696, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 13, 2024-04-25T10:20:29.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 1683, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 1520, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 4058, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 4557, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 32, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 14075 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.617 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 4124, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 3636, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 2222, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T10:20:29.618 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 30187, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T10:20:29.619 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 30187, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 46, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 46, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 4, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 96179579, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-04-25T10:20:29.620 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4109, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 79.427655671, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.019330166 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 14, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 2016, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 2002, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 13, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 13, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 17711, 2024-04-25T10:20:29.621 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 19712, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 2001, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 2, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 3716, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 3714, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 98060, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215340 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 29922, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 26248, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 5, 2024-04-25T10:20:29.622 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2000, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.293985227, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000646992 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 28, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.005549609, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000198200 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.623 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6000, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 7.249193994, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001208198 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7985, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 7.114268350, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000890954 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.624 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.625 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000072030, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000072030 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003365621, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003365621 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.007541183, 2024-04-25T10:20:29.627 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.007541183 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.091971947, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.011496493 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2222, 2024-04-25T10:20:29.628 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 237.779287555, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.107011380 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.043346003, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.043346003 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000377494, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000377494 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.631 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.632 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8000, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 16.781935628, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002097741 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.633 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 18689, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 18689, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 80, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T10:20:29.634 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T10:20:29.635 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:20:29.636 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 250468, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 86, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 5800, 2024-04-25T10:20:29.637 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 85, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 13338260, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 659958, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T10:20:29.638 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 9692, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 92595994, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 37, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 9692, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 9692, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 41.103587029, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.004240980 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 9692, 2024-04-25T10:20:29.639 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 9789 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 9692, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 1752, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 7940, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 11, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 54, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 4161, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 32, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T10:20:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 1636, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 2033, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1698, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 22, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T10:20:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T10:20:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 39, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 5, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 8, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 8, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 111, 2024-04-25T10:20:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 31 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T10:20:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 32, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 3636, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 3636, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 38540, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 9938163, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:20:29.645 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 38540, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 38540, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9938163, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:20:29.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 9692, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 92976318, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 9692, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 92976318, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 9692, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 9692, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 9692, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9692, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.648 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 3636, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 380324, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 3636, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 52, 2024-04-25T10:20:29.649 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 380324, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x55ba2f044960": { 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 30187, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 91985275, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:20:29.650 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 30187, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4109, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 91985275, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:29.651 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:20:29.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a38066f60 msgr2=0x7f1a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a38066f60 0x7f1a38069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c15b6e0 tx=0x7f1a4c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 msgr2=0x7f1a5c14faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c14faf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1a440ae040 tx=0x7f1a44019220 comp rx=0 tx=0).stop 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 shutdown_connections 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a38066f60 0x7f1a38069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a5c159540 0x7f1a5c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5c0aebd0 0x7f1a5c14faf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 --2- 172.21.15.45:0/2297137742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a5c072020 0x7f1a5c15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 >> 172.21.15.45:0/2297137742 conn(0x7f1a5c0669a0 msgr2=0x7f1a5c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 shutdown_connections 2024-04-25T10:20:29.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:29.599+0000 7f1a62600640 1 -- 172.21.15.45:0/2297137742 wait complete. 2024-04-25T10:20:29.654 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:20:29.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:29.715 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:29.821 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:20:29.822 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:20:29.822 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:29.831 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:20:29.831 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:29.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:30.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 Processor -- start 2024-04-25T10:20:30.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 -- start start 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff54c0d0910 0x7ff54c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff54c071aa0 0x7ff54c071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c0723f0 con 0x7ff54c156010 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c15c350 con 0x7ff54c071aa0 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff551200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c15c000 con 0x7ff54c0d0910 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff54c071aa0 0x7ff54c071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff54c071aa0 0x7ff54c071eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53208/0 (socket says 172.21.15.45:53208) 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 -- 172.21.15.45:0/2053159946 learned_addr learned my addr 172.21.15.45:0/2053159946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54aa00640 1 --2- 172.21.15.45:0/2053159946 >> v2:172.21.15.117:3300/0 conn(0x7ff54c0d0910 0x7ff54c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 -- 172.21.15.45:0/2053159946 >> v2:172.21.15.117:3300/0 conn(0x7ff54c0d0910 msgr2=0x7ff54c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 --2- 172.21.15.45:0/2053159946 >> v2:172.21.15.117:3300/0 conn(0x7ff54c0d0910 0x7ff54c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 -- 172.21.15.45:0/2053159946 >> v1:172.21.15.117:6789/0 conn(0x7ff54c156010 legacy=0x7ff54c15c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.067+0000 7ff54be00640 1 -- 172.21.15.45:0/2053159946 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0713a0 con 0x7ff54c071aa0 2024-04-25T10:20:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff54be00640 1 --2- 172.21.15.45:0/2053159946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c071eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0744f0 tx=0x7ff54401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897a85296f929d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff54a000640 1 -- 172.21.15.45:0/2053159946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440a5620 con 0x7ff54c071aa0 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff54a000640 1 -- 172.21.15.45:0/2053159946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5440a57c0 con 0x7ff54c071aa0 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff54a000640 1 -- 172.21.15.45:0/2053159946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5440a5b90 con 0x7ff54c071aa0 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 -- 172.21.15.45:0/2053159946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 msgr2=0x7ff54c071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 --2- 172.21.15.45:0/2053159946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c071eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0744f0 tx=0x7ff54401c740 comp rx=0 tx=0).stop 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 -- 172.21.15.45:0/2053159946 shutdown_connections 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 --2- 172.21.15.45:0/2053159946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c071eb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 --2- 172.21.15.45:0/2053159946 >> v2:172.21.15.117:3300/0 conn(0x7ff54c0d0910 0x7ff54c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 -- 172.21.15.45:0/2053159946 >> 172.21.15.45:0/2053159946 conn(0x7ff54c0b1c30 msgr2=0x7ff54c0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 -- 172.21.15.45:0/2053159946 shutdown_connections 2024-04-25T10:20:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 -- 172.21.15.45:0/2053159946 wait complete. 2024-04-25T10:20:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 Processor -- start 2024-04-25T10:20:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.071+0000 7ff551200640 1 -- start start 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c0d0910 0x7ff54c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff54c156010 0x7ff54c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0b4890 con 0x7ff54c071aa0 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0b4710 con 0x7ff54c0d0910 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c168450 con 0x7ff54c156010 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53214/0 (socket says 172.21.15.45:53214) 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 -- 172.21.15.45:0/3859966198 learned_addr learned my addr 172.21.15.45:0/3859966198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54aa00640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c0d0910 0x7ff54c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54be00640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff54c156010 0x7ff54c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 -- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff54c156010 msgr2=0x7ff54c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff54c156010 0x7ff54c141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 -- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c0d0910 msgr2=0x7ff54c151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c0d0910 0x7ff54c151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 -- 172.21.15.45:0/3859966198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c15d100 con 0x7ff54c071aa0 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54aa00640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c0d0910 0x7ff54c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff54b400640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c15e1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b410 tx=0x7ff53c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c015030 con 0x7ff54c071aa0 2024-04-25T10:20:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c004020 con 0x7ff54c071aa0 2024-04-25T10:20:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c004450 con 0x7ff54c071aa0 2024-04-25T10:20:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c15d3e0 con 0x7ff54c071aa0 2024-04-25T10:20:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c15d9a0 con 0x7ff54c071aa0 2024-04-25T10:20:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.075+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c075120 con 0x7ff54c071aa0 2024-04-25T10:20:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.079+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff53c018070 con 0x7ff54c071aa0 2024-04-25T10:20:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.079+0000 7ff543400640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff52c066f60 0x7ff52c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.079+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff53c084be0 con 0x7ff54c071aa0 2024-04-25T10:20:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.079+0000 7ff54aa00640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff52c066f60 0x7ff52c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.079+0000 7ff54aa00640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff52c066f60 0x7ff52c069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0ce2f0 tx=0x7ff538008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.083+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff53c055bf0 con 0x7ff54c071aa0 2024-04-25T10:20:30.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.243+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff54c06dd30 con 0x7ff54c071aa0 2024-04-25T10:20:30.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.243+0000 7ff543400640 1 -- 172.21.15.45:0/3859966198 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16912 (secure 0 0 0) 0x7ff53c0554f0 con 0x7ff54c071aa0 2024-04-25T10:20:30.248 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:30.249 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:15.602282+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":38,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.14","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff52c066f60 msgr2=0x7ff52c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff52c066f60 0x7ff52c069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0ce2f0 tx=0x7ff538008040 comp rx=0 tx=0).stop 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 msgr2=0x7ff54c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c15e1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b410 tx=0x7ff53c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 shutdown_connections 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff52c066f60 0x7ff52c069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff54c156010 0x7ff54c141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c0d0910 0x7ff54c151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 --2- 172.21.15.45:0/3859966198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff54c071aa0 0x7ff54c15e1a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 >> 172.21.15.45:0/3859966198 conn(0x7ff54c0b1c30 msgr2=0x7ff54c153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 shutdown_connections 2024-04-25T10:20:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.247+0000 7ff551200640 1 -- 172.21.15.45:0/3859966198 wait complete. 2024-04-25T10:20:30.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:18:53.475873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:54.611281+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 38, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:30.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 Processor -- start 2024-04-25T10:20:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 -- start start 2024-04-25T10:20:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2a3815df20 0x7f2a380b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380b1970 0x7f2a380ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a380b1e10 con 0x7f2a380aeb50 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a380b20f0 con 0x7f2a3815df20 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a3d200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380b23d0 con 0x7f2a380b1970 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a36800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a380aeb50 0x7f2a380cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40492/0 (socket says 172.21.15.45:40492) 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a36800640 1 -- 172.21.15.45:0/3785653479 learned_addr learned my addr 172.21.15.45:0/3785653479 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35e00640 1 --2- 172.21.15.45:0/3785653479 >> v2:172.21.15.117:3300/0 conn(0x7f2a3815df20 0x7f2a380b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a2d400640 1 --2- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380b1970 0x7f2a380ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35400640 1 -- 172.21.15.45:0/3785653479 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031308263 0 0) 0x7f2a380b1e10 con 0x7f2a380aeb50 2024-04-25T10:20:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35400640 1 -- 172.21.15.45:0/3785653479 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a1c003530 con 0x7f2a380aeb50 2024-04-25T10:20:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35e00640 1 -- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380b1970 msgr2=0x7f2a380ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35e00640 1 --2- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380b1970 0x7f2a380ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35e00640 1 -- 172.21.15.45:0/3785653479 >> v1:172.21.15.45:6789/0 conn(0x7f2a380aeb50 legacy=0x7f2a380cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.411+0000 7f2a35e00640 1 -- 172.21.15.45:0/3785653479 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380b26b0 con 0x7f2a3815df20 2024-04-25T10:20:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a35e00640 1 --2- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a380b2eb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a280097c0 tx=0x7f2a2801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68f3d63b1d5056c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a35400640 1 -- 172.21.15.45:0/3785653479 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280b5070 con 0x7f2a3815df20 2024-04-25T10:20:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a35400640 1 -- 172.21.15.45:0/3785653479 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a280a8990 con 0x7f2a3815df20 2024-04-25T10:20:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a35400640 1 -- 172.21.15.45:0/3785653479 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a280a8d60 con 0x7f2a3815df20 2024-04-25T10:20:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 -- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 msgr2=0x7f2a380b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 --2- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a380b2eb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2a280097c0 tx=0x7f2a2801ec90 comp rx=0 tx=0).stop 2024-04-25T10:20:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 -- 172.21.15.45:0/3785653479 shutdown_connections 2024-04-25T10:20:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 --2- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380b1970 0x7f2a380ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 --2- 172.21.15.45:0/3785653479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a380b2eb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 -- 172.21.15.45:0/3785653479 >> 172.21.15.45:0/3785653479 conn(0x7f2a380d1960 msgr2=0x7f2a380d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 -- 172.21.15.45:0/3785653479 shutdown_connections 2024-04-25T10:20:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.415+0000 7f2a3d200640 1 -- 172.21.15.45:0/3785653479 wait complete. 2024-04-25T10:20:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 Processor -- start 2024-04-25T10:20:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 -- start start 2024-04-25T10:20:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380aeb50 0x7f2a3814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 0x7f2a380b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a38160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3806beb0 con 0x7f2a380b1970 2024-04-25T10:20:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3815e300 con 0x7f2a3815df20 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3806c030 con 0x7f2a380aeb50 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 0x7f2a380b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 0x7f2a380b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53224/0 (socket says 172.21.15.45:53224) 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 -- 172.21.15.45:0/348945128 learned_addr learned my addr 172.21.15.45:0/348945128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a35e00640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380aeb50 0x7f2a3814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a36800640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a38160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 -- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380aeb50 msgr2=0x7f2a3814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380aeb50 0x7f2a3814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 -- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 msgr2=0x7f2a38160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a38160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 -- 172.21.15.45:0/348945128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3814f300 con 0x7f2a380b1970 2024-04-25T10:20:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2d400640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 0x7f2a380b4530 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000e930 tx=0x7f2a2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20017070 con 0x7f2a380b1970 2024-04-25T10:20:30.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2000ca00 con 0x7f2a380b1970 2024-04-25T10:20:30.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000cdb0 con 0x7f2a380b1970 2024-04-25T10:20:30.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a380b4f40 con 0x7f2a380b1970 2024-04-25T10:20:30.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.419+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a380b5500 con 0x7f2a380b1970 2024-04-25T10:20:30.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.423+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3804ef30 con 0x7f2a380b1970 2024-04-25T10:20:30.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.423+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a2001a070 con 0x7f2a380b1970 2024-04-25T10:20:30.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.423+0000 7f2a2ea00640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a14066f60 0x7f2a14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.423+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2a200850d0 con 0x7f2a380b1970 2024-04-25T10:20:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.423+0000 7f2a35e00640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a14066f60 0x7f2a14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.423+0000 7f2a35e00640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a14066f60 0x7f2a14069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2a280b2000 tx=0x7f2a28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.427+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a20055a20 con 0x7f2a380b1970 2024-04-25T10:20:30.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.591+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a380b1500 con 0x7f2a380b1970 2024-04-25T10:20:30.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.591+0000 7f2a2ea00640 1 -- 172.21.15.45:0/348945128 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a20059ac0 con 0x7f2a380b1970 2024-04-25T10:20:30.596 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a14066f60 msgr2=0x7f2a14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a14066f60 0x7f2a14069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2a280b2000 tx=0x7f2a28008040 comp rx=0 tx=0).stop 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 msgr2=0x7f2a380b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 0x7f2a380b4530 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000e930 tx=0x7f2a2000edf0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 shutdown_connections 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a14066f60 0x7f2a14069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a3815df20 0x7f2a38160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a380b1970 0x7f2a380b4530 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 --2- 172.21.15.45:0/348945128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a380aeb50 0x7f2a3814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 >> 172.21.15.45:0/348945128 conn(0x7f2a380d1960 msgr2=0x7f2a380b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 shutdown_connections 2024-04-25T10:20:30.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.595+0000 7f2a3d200640 1 -- 172.21.15.45:0/348945128 wait complete. 2024-04-25T10:20:30.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:30.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 Processor -- start 2024-04-25T10:20:30.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 -- start start 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8159540 0x7f6bc80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bc80aebd0 0x7f6bc8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc8071360 con 0x7f6bc8072020 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc80af030 con 0x7f6bc80aebd0 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bcda00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc8159920 con 0x7f6bc8159540 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bc80aebd0 0x7f6bc8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6bc80aebd0 0x7f6bc8071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53238/0 (socket says 172.21.15.45:53238) 2024-04-25T10:20:30.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 -- 172.21.15.45:0/1584287936 learned_addr learned my addr 172.21.15.45:0/1584287936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:30.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6a00640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8159540 0x7f6bc80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 -- 172.21.15.45:0/1584287936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8159540 msgr2=0x7f6bc80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8159540 0x7f6bc80ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 -- 172.21.15.45:0/1584287936 >> v1:172.21.15.117:6789/0 conn(0x7f6bc8072020 legacy=0x7f6bc815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6000640 1 -- 172.21.15.45:0/1584287936 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc815c2f0 con 0x7f6bc80aebd0 2024-04-25T10:20:30.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.759+0000 7f6bc6a00640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8159540 0x7f6bc80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:30.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bc6000640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8071ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009870 tx=0x7f6bb401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aead340f1fb779b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bc5600640 1 -- 172.21.15.45:0/1584287936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40b1070 con 0x7f6bc80aebd0 2024-04-25T10:20:30.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bc5600640 1 -- 172.21.15.45:0/1584287936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb40ac630 con 0x7f6bc80aebd0 2024-04-25T10:20:30.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bc5600640 1 -- 172.21.15.45:0/1584287936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40a8050 con 0x7f6bc80aebd0 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 -- 172.21.15.45:0/1584287936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc80aebd0 msgr2=0x7f6bc8071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8071ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009870 tx=0x7f6bb401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 -- 172.21.15.45:0/1584287936 shutdown_connections 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8071ae0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 --2- 172.21.15.45:0/1584287936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8159540 0x7f6bc80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 -- 172.21.15.45:0/1584287936 >> 172.21.15.45:0/1584287936 conn(0x7f6bc80669a0 msgr2=0x7f6bc8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 -- 172.21.15.45:0/1584287936 shutdown_connections 2024-04-25T10:20:30.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.763+0000 7f6bcda00640 1 -- 172.21.15.45:0/1584287936 wait complete. 2024-04-25T10:20:30.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 Processor -- start 2024-04-25T10:20:30.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- start start 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8072020 0x7f6bc80ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8139500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 0x7f6bc8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc8072450 con 0x7f6bc8159540 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc815bfa0 con 0x7f6bc80aebd0 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc806c0d0 con 0x7f6bc8072020 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 0x7f6bc8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 0x7f6bc8155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53246/0 (socket says 172.21.15.45:53246) 2024-04-25T10:20:30.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 -- 172.21.15.45:0/2237134494 learned_addr learned my addr 172.21.15.45:0/2237134494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc6a00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8072020 0x7f6bc80ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc6000640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 -- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8072020 msgr2=0x7f6bc80ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8072020 0x7f6bc80ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 -- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc80aebd0 msgr2=0x7f6bc8139500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8139500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 -- 172.21.15.45:0/2237134494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8139d20 con 0x7f6bc8159540 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bc7400640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 0x7f6bc8155aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000c780 tx=0x7f6bc000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0004b70 con 0x7f6bc8159540 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc0004d10 con 0x7f6bc8159540 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc000aa90 con 0x7f6bc8159540 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc813a000 con 0x7f6bc8159540 2024-04-25T10:20:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc8139a40 con 0x7f6bc8159540 2024-04-25T10:20:30.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.767+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc813aac0 con 0x7f6bc8159540 2024-04-25T10:20:30.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.771+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bc000ac30 con 0x7f6bc8159540 2024-04-25T10:20:30.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.771+0000 7f6bbea00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ba8066f60 0x7f6ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.771+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6bc0085220 con 0x7f6bc8159540 2024-04-25T10:20:30.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.771+0000 7f6bc6a00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ba8066f60 0x7f6ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.771+0000 7f6bc6a00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ba8066f60 0x7f6ba8069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8006fd0 tx=0x7f6bb8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.775+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6bc0020070 con 0x7f6bc8159540 2024-04-25T10:20:30.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.935+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6bc804ef30 con 0x7f6bc8159540 2024-04-25T10:20:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.935+0000 7f6bbea00640 1 -- 172.21.15.45:0/2237134494 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6bc0055b40 con 0x7f6bc8159540 2024-04-25T10:20:30.939 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.935+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ba8066f60 msgr2=0x7f6ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ba8066f60 0x7f6ba8069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8006fd0 tx=0x7f6bb8008040 comp rx=0 tx=0).stop 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 msgr2=0x7f6bc8155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 0x7f6bc8155aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000c780 tx=0x7f6bc000cc40 comp rx=0 tx=0).stop 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 shutdown_connections 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ba8066f60 0x7f6ba8069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bc8159540 0x7f6bc8155aa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bc80aebd0 0x7f6bc8139500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 --2- 172.21.15.45:0/2237134494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bc8072020 0x7f6bc80ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 >> 172.21.15.45:0/2237134494 conn(0x7f6bc80669a0 msgr2=0x7f6bc80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 shutdown_connections 2024-04-25T10:20:30.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:30.939+0000 7f6bcda00640 1 -- 172.21.15.45:0/2237134494 wait complete. 2024-04-25T10:20:30.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.099+0000 7f47eb200640 1 Processor -- start 2024-04-25T10:20:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.099+0000 7f47eb200640 1 -- start start 2024-04-25T10:20:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47eb200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47ec06afc0 0x7f47ec06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47eb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec0d0510 0x7f47ec15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47eb200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ec06b8e0 con 0x7f47ec072290 2024-04-25T10:20:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47eb200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ec158800 con 0x7f47ec06afc0 2024-04-25T10:20:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47eb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec06d930 con 0x7f47ec0d0510 2024-04-25T10:20:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47ea800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec0d0510 0x7f47ec15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47ec06afc0 0x7f47ec06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47ec06afc0 0x7f47ec06b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34310/0 (socket says 172.21.15.45:34310) 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 -- 172.21.15.45:0/1586386905 learned_addr learned my addr 172.21.15.45:0/1586386905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 -- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec0d0510 msgr2=0x7f47ec15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 --2- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec0d0510 0x7f47ec15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 -- 172.21.15.45:0/1586386905 >> v1:172.21.15.117:6789/0 conn(0x7f47ec072290 legacy=0x7f47ec06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.103+0000 7f47e9e00640 1 -- 172.21.15.45:0/1586386905 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec15afa0 con 0x7f47ec06afc0 2024-04-25T10:20:31.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47e9e00640 1 --2- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec06b3a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009870 tx=0x7f47dc01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d610fe68931db81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47e9400640 1 -- 172.21.15.45:0/1586386905 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0ab650 con 0x7f47ec06afc0 2024-04-25T10:20:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47e9400640 1 -- 172.21.15.45:0/1586386905 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc0b0070 con 0x7f47ec06afc0 2024-04-25T10:20:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47e9400640 1 -- 172.21.15.45:0/1586386905 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0aba10 con 0x7f47ec06afc0 2024-04-25T10:20:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 -- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 msgr2=0x7f47ec06b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 --2- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec06b3a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009870 tx=0x7f47dc01c560 comp rx=0 tx=0).stop 2024-04-25T10:20:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 -- 172.21.15.45:0/1586386905 shutdown_connections 2024-04-25T10:20:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 --2- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec0d0510 0x7f47ec15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 --2- 172.21.15.45:0/1586386905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec06b3a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 -- 172.21.15.45:0/1586386905 >> 172.21.15.45:0/1586386905 conn(0x7f47ec073d10 msgr2=0x7f47ec074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 -- 172.21.15.45:0/1586386905 shutdown_connections 2024-04-25T10:20:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 -- 172.21.15.45:0/1586386905 wait complete. 2024-04-25T10:20:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 Processor -- start 2024-04-25T10:20:31.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.107+0000 7f47eb200640 1 -- start start 2024-04-25T10:20:31.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec072290 0x7f47ec13e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 0x7f47ec154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec167720 con 0x7f47ec0d0510 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec15bb20 con 0x7f47ec06afc0 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec1678a0 con 0x7f47ec072290 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 0x7f47ec154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 0x7f47ec154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53248/0 (socket says 172.21.15.45:53248) 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 -- 172.21.15.45:0/4176586480 learned_addr learned my addr 172.21.15.45:0/4176586480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47e9e00640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47e1400640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec072290 0x7f47ec13e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 -- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec072290 msgr2=0x7f47ec13e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec072290 0x7f47ec13e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 -- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 msgr2=0x7f47ec152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 -- 172.21.15.45:0/4176586480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec157280 con 0x7f47ec0d0510 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47ea800640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 0x7f47ec154e40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009d30 tx=0x7f47e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4018020 con 0x7f47ec0d0510 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e4014480 con 0x7f47ec0d0510 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec153ac0 con 0x7f47ec0d0510 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4014870 con 0x7f47ec0d0510 2024-04-25T10:20:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec154080 con 0x7f47ec0d0510 2024-04-25T10:20:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ec075120 con 0x7f47ec0d0510 2024-04-25T10:20:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.111+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47e401b020 con 0x7f47ec0d0510 2024-04-25T10:20:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.115+0000 7f47e2a00640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f47c8066f60 0x7f47c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.115+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f47e4085eb0 con 0x7f47ec0d0510 2024-04-25T10:20:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.115+0000 7f47e9e00640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f47c8066f60 0x7f47c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.115+0000 7f47e9e00640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f47c8066f60 0x7f47c8069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0af000 tx=0x7f47dc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.119+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47e4022020 con 0x7f47ec0d0510 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47ec0d02f0 con 0x7f47ec0d0510 2024-04-25T10:20:31.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47e2a00640 1 -- 172.21.15.45:0/4176586480 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47e4010040 con 0x7f47ec0d0510 2024-04-25T10:20:31.286 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f47c8066f60 msgr2=0x7f47c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f47c8066f60 0x7f47c8069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0af000 tx=0x7f47dc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 msgr2=0x7f47ec154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 0x7f47ec154e40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009d30 tx=0x7f47e400c490 comp rx=0 tx=0).stop 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 shutdown_connections 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f47c8066f60 0x7f47c8069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47ec0d0510 0x7f47ec154e40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47ec072290 0x7f47ec13e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 --2- 172.21.15.45:0/4176586480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47ec06afc0 0x7f47ec152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 >> 172.21.15.45:0/4176586480 conn(0x7f47ec073d10 msgr2=0x7f47ec0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 shutdown_connections 2024-04-25T10:20:31.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.283+0000 7f47eb200640 1 -- 172.21.15.45:0/4176586480 wait complete. 2024-04-25T10:20:31.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:20:31.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 Processor -- start 2024-04-25T10:20:31.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 -- start start 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a0c071f70 0x7f8a0c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c0b2a80 0x7f8a0c0b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c070ed0 con 0x7f8a0c15a0e0 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0711b0 con 0x7f8a0c071f70 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a12400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c071490 con 0x7f8a0c0b2a80 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8a0c15a0e0 0x7f8a0c15b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40502/0 (socket says 172.21.15.45:40502) 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0b400640 1 -- 172.21.15.45:0/2802006149 learned_addr learned my addr 172.21.15.45:0/2802006149 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0be00640 1 --2- 172.21.15.45:0/2802006149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c0b2a80 0x7f8a0c0b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0aa00640 1 --2- 172.21.15.45:0/2802006149 >> v2:172.21.15.117:3300/0 conn(0x7f8a0c071f70 0x7f8a0c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0a000640 1 -- 172.21.15.45:0/2802006149 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589073374 0 0) 0x7f8a0c070ed0 con 0x7f8a0c15a0e0 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0a000640 1 -- 172.21.15.45:0/2802006149 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f0003530 con 0x7f8a0c15a0e0 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0be00640 1 -- 172.21.15.45:0/2802006149 >> v2:172.21.15.117:3300/0 conn(0x7f8a0c071f70 msgr2=0x7f8a0c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0be00640 1 --2- 172.21.15.45:0/2802006149 >> v2:172.21.15.117:3300/0 conn(0x7f8a0c071f70 0x7f8a0c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0be00640 1 -- 172.21.15.45:0/2802006149 >> v1:172.21.15.45:6789/0 conn(0x7f8a0c15a0e0 legacy=0x7f8a0c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0a000640 1 -- 172.21.15.45:0/2802006149 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179741015 0 0) 0x7f89f0003530 con 0x7f8a0c15a0e0 2024-04-25T10:20:31.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.447+0000 7f8a0be00640 1 -- 172.21.15.45:0/2802006149 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c071770 con 0x7f8a0c0b2a80 2024-04-25T10:20:31.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a0be00640 1 --2- 172.21.15.45:0/2802006149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c0b2a80 0x7f8a0c0b2f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00bb450c0ab08b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a0a000640 1 -- 172.21.15.45:0/2802006149 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b2070 con 0x7f8a0c0b2a80 2024-04-25T10:20:31.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a0a000640 1 -- 172.21.15.45:0/2802006149 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a040a55a0 con 0x7f8a0c0b2a80 2024-04-25T10:20:31.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a0a000640 1 -- 172.21.15.45:0/2802006149 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc001680 con 0x7f8a0c0b2a80 2024-04-25T10:20:31.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 -- 172.21.15.45:0/2802006149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c0b2a80 msgr2=0x7f8a0c0b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 --2- 172.21.15.45:0/2802006149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c0b2a80 0x7f8a0c0b2f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0401c890 comp rx=0 tx=0).stop 2024-04-25T10:20:31.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 -- 172.21.15.45:0/2802006149 shutdown_connections 2024-04-25T10:20:31.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 --2- 172.21.15.45:0/2802006149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c0b2a80 0x7f8a0c0b2f10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 --2- 172.21.15.45:0/2802006149 >> v2:172.21.15.117:3300/0 conn(0x7f8a0c071f70 0x7f8a0c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 -- 172.21.15.45:0/2802006149 >> 172.21.15.45:0/2802006149 conn(0x7f8a0c066b70 msgr2=0x7f8a0c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 -- 172.21.15.45:0/2802006149 shutdown_connections 2024-04-25T10:20:31.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.451+0000 7f8a12400640 1 -- 172.21.15.45:0/2802006149 wait complete. 2024-04-25T10:20:31.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 Processor -- start 2024-04-25T10:20:31.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- start start 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 0x7f8a0c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0c0b2a80 0x7f8a0c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c15a0e0 0x7f8a0c142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cfe70 con 0x7f8a0c071f70 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c070b50 con 0x7f8a0c0b2a80 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cfff0 con 0x7f8a0c15a0e0 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 0x7f8a0c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 0x7f8a0c154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53260/0 (socket says 172.21.15.45:53260) 2024-04-25T10:20:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 -- 172.21.15.45:0/3770765864 learned_addr learned my addr 172.21.15.45:0/3770765864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 -- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c15a0e0 msgr2=0x7f8a0c142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c15a0e0 0x7f8a0c142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 -- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0c0b2a80 msgr2=0x7f8a0c14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0c0b2a80 0x7f8a0c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 -- 172.21.15.45:0/3770765864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c15a6d0 con 0x7f8a0c071f70 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a0b400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 0x7f8a0c154a50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c750 tx=0x7f89fc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc005ae0 con 0x7f8a0c071f70 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc005ca0 con 0x7f8a0c071f70 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0072b0 con 0x7f8a0c071f70 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c14b610 con 0x7f8a0c071f70 2024-04-25T10:20:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c14bbd0 con 0x7f8a0c071f70 2024-04-25T10:20:31.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.455+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c04ef30 con 0x7f8a0c071f70 2024-04-25T10:20:31.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.459+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89fc007450 con 0x7f8a0c071f70 2024-04-25T10:20:31.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.459+0000 7f8a03400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f89ec066f60 0x7f89ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.459+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f89fc084eb0 con 0x7f8a0c071f70 2024-04-25T10:20:31.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.459+0000 7f8a0aa00640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f89ec066f60 0x7f89ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.459+0000 7f8a0aa00640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f89ec066f60 0x7f89ec069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006aa0 tx=0x7f89f8006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.463+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a0c15a6d0 con 0x7f8a0c071f70 2024-04-25T10:20:31.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a0c0703e0 con 0x7f8a0c071f70 2024-04-25T10:20:31.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a03400640 1 -- 172.21.15.45:0/3770765864 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f89fc010300 con 0x7f8a0c071f70 2024-04-25T10:20:31.641 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f89ec066f60 msgr2=0x7f89ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f89ec066f60 0x7f89ec069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006aa0 tx=0x7f89f8006a30 comp rx=0 tx=0).stop 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 msgr2=0x7f8a0c154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 0x7f8a0c154a50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c750 tx=0x7f89fc00cc10 comp rx=0 tx=0).stop 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 shutdown_connections 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f89ec066f60 0x7f89ec069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a0c15a0e0 0x7f8a0c142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0c0b2a80 0x7f8a0c14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 --2- 172.21.15.45:0/3770765864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a0c071f70 0x7f8a0c154a50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 >> 172.21.15.45:0/3770765864 conn(0x7f8a0c066b70 msgr2=0x7f8a0c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 shutdown_connections 2024-04-25T10:20:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.639+0000 7f8a12400640 1 -- 172.21.15.45:0/3770765864 wait complete. 2024-04-25T10:20:31.663 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:20:31.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:31.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:31.709 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:20:31.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:31.709 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:20:31.761 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:20:31.761 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:31.771 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:20:31.771 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:31.813 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:20:31.813 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:31.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:31.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.991+0000 7f4036000640 1 Processor -- start 2024-04-25T10:20:31.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.991+0000 7f4036000640 1 -- start start 2024-04-25T10:20:31.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f4036000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4030066fe0 0x7f403015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f4036000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40300b5560 0x7f4030157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f4036000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40300b5970 con 0x7f4030066c00 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f4036000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40300b5c20 con 0x7f40300b5560 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f4036000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40300af6e0 con 0x7f4030066fe0 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40300b5560 0x7f4030157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4030066c00 0x7f403006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40516/0 (socket says 172.21.15.45:40516) 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402f400640 1 -- 172.21.15.45:0/238842854 learned_addr learned my addr 172.21.15.45:0/238842854 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f4027e00640 1 --2- 172.21.15.45:0/238842854 >> v2:172.21.15.117:3300/0 conn(0x7f4030066fe0 0x7f403015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402fe00640 1 -- 172.21.15.45:0/238842854 >> v2:172.21.15.117:3300/0 conn(0x7f4030066fe0 msgr2=0x7f403015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402fe00640 1 --2- 172.21.15.45:0/238842854 >> v2:172.21.15.117:3300/0 conn(0x7f4030066fe0 0x7f403015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402fe00640 1 -- 172.21.15.45:0/238842854 >> v1:172.21.15.45:6789/0 conn(0x7f4030066c00 legacy=0x7f403006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402fe00640 1 -- 172.21.15.45:0/238842854 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40300af9c0 con 0x7f40300b5560 2024-04-25T10:20:31.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402ea00640 1 -- 172.21.15.45:0/238842854 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627776957 0 0) 0x7f40300b5970 con 0x7f4030066c00 2024-04-25T10:20:32.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.995+0000 7f402fe00640 1 --2- 172.21.15.45:0/238842854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40300b5560 0x7f4030157b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f403006e850 tx=0x7f402801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31c2a130faf9d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402ea00640 1 -- 172.21.15.45:0/238842854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280b2070 con 0x7f40300b5560 2024-04-25T10:20:32.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402ea00640 1 -- 172.21.15.45:0/238842854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280a59c0 con 0x7f40300b5560 2024-04-25T10:20:32.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402ea00640 1 -- 172.21.15.45:0/238842854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280a5d90 con 0x7f40300b5560 2024-04-25T10:20:32.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- 172.21.15.45:0/238842854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40300b5560 msgr2=0x7f4030157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 --2- 172.21.15.45:0/238842854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40300b5560 0x7f4030157b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f403006e850 tx=0x7f402801c900 comp rx=0 tx=0).stop 2024-04-25T10:20:32.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- 172.21.15.45:0/238842854 shutdown_connections 2024-04-25T10:20:32.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 --2- 172.21.15.45:0/238842854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40300b5560 0x7f4030157b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 --2- 172.21.15.45:0/238842854 >> v2:172.21.15.117:3300/0 conn(0x7f4030066fe0 0x7f403015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- 172.21.15.45:0/238842854 >> 172.21.15.45:0/238842854 conn(0x7f40300b24f0 msgr2=0x7f40300b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- 172.21.15.45:0/238842854 shutdown_connections 2024-04-25T10:20:32.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- 172.21.15.45:0/238842854 wait complete. 2024-04-25T10:20:32.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 Processor -- start 2024-04-25T10:20:32.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- start start 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 0x7f403015f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4030066fe0 0x7f403014a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40300b5560 0x7f403014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300af1e0 con 0x7f4030066c00 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40300af060 con 0x7f40300b5560 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4036000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403006ac90 con 0x7f4030066fe0 2024-04-25T10:20:32.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 0x7f403015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 0x7f403015f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53276/0 (socket says 172.21.15.45:53276) 2024-04-25T10:20:32.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402f400640 1 -- 172.21.15.45:0/1777648508 learned_addr learned my addr 172.21.15.45:0/1777648508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:32.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f4027e00640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4030066fe0 0x7f403014a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:31.999+0000 7f402fe00640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40300b5560 0x7f403014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f402f400640 1 -- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4030066fe0 msgr2=0x7f403014a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f402f400640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4030066fe0 0x7f403014a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f402f400640 1 -- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40300b5560 msgr2=0x7f403014cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f402f400640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40300b5560 0x7f403014cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f402f400640 1 -- 172.21.15.45:0/1777648508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403015f800 con 0x7f4030066c00 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f402f400640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 0x7f403015f220 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40200011f0 tx=0x7f402000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020018070 con 0x7f4030066c00 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40200074e0 con 0x7f4030066c00 2024-04-25T10:20:32.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403015fae0 con 0x7f4030066c00 2024-04-25T10:20:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020004a10 con 0x7f4030066c00 2024-04-25T10:20:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40301600a0 con 0x7f4030066c00 2024-04-25T10:20:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403004ef30 con 0x7f4030066c00 2024-04-25T10:20:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f402001b070 con 0x7f4030066c00 2024-04-25T10:20:32.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4027400640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f400c066f90 0x7f400c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.003+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4020084310 con 0x7f4030066c00 2024-04-25T10:20:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.007+0000 7f4027e00640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f400c066f90 0x7f400c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.007+0000 7f4027e00640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f400c066f90 0x7f400c069450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f403014f500 tx=0x7f4018008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.007+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4020054c30 con 0x7f4030066c00 2024-04-25T10:20:32.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f403006d800 con 0x7f4030066c00 2024-04-25T10:20:32.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4027400640 1 -- 172.21.15.45:0/1777648508 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f4020058cd0 con 0x7f4030066c00 2024-04-25T10:20:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:32.209 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:18:55.618016+0000","modified":"2024-04-25T10:19:08.145993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3482819589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3482819589},{"type":"v1","addr":"172.21.15.45:6835","nonce":3482819589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:20:32.210 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 18 2024-04-25T10:20:32.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f400c066f90 msgr2=0x7f400c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f400c066f90 0x7f400c069450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f403014f500 tx=0x7f4018008040 comp rx=0 tx=0).stop 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 msgr2=0x7f403015f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 0x7f403015f220 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40200011f0 tx=0x7f402000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 shutdown_connections 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f400c066f90 0x7f400c069450 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40300b5560 0x7f403014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4030066fe0 0x7f403014a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 --2- 172.21.15.45:0/1777648508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4030066c00 0x7f403015f220 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 >> 172.21.15.45:0/1777648508 conn(0x7f40300b24f0 msgr2=0x7f40300b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 shutdown_connections 2024-04-25T10:20:32.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.207+0000 7f4036000640 1 -- 172.21.15.45:0/1777648508 wait complete. 2024-04-25T10:20:32.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:32.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 Processor -- start 2024-04-25T10:20:32.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 -- start start 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb8415b9e0 0x7fcb840cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb840d1c30 0x7fcb840cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb840ccfb0 con 0x7fcb8415bdc0 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb840cd290 con 0x7fcb8415b9e0 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb88e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840cd570 con 0x7fcb840d1c30 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb8415b9e0 0x7fcb840cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcb8415b9e0 0x7fcb840cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53290/0 (socket says 172.21.15.45:53290) 2024-04-25T10:20:32.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 -- 172.21.15.45:0/1007090565 learned_addr learned my addr 172.21.15.45:0/1007090565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 -- 172.21.15.45:0/1007090565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb840d1c30 msgr2=0x7fcb840cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb82800640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb840d1c30 0x7fcb840cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb840d1c30 0x7fcb840cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 -- 172.21.15.45:0/1007090565 >> v1:172.21.15.45:6789/0 conn(0x7fcb8415bdc0 legacy=0x7fcb840d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb81e00640 1 -- 172.21.15.45:0/1007090565 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840cd850 con 0x7fcb8415b9e0 2024-04-25T10:20:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.371+0000 7fcb82800640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb840d1c30 0x7fcb840cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb81e00640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415b9e0 0x7fcb840cc410 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009870 tx=0x7fcb7401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf1b86ba541b342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb81400640 1 -- 172.21.15.45:0/1007090565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb740b2070 con 0x7fcb8415b9e0 2024-04-25T10:20:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb81400640 1 -- 172.21.15.45:0/1007090565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb740ad680 con 0x7fcb8415b9e0 2024-04-25T10:20:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb81400640 1 -- 172.21.15.45:0/1007090565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb740ada50 con 0x7fcb8415b9e0 2024-04-25T10:20:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 -- 172.21.15.45:0/1007090565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415b9e0 msgr2=0x7fcb840cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415b9e0 0x7fcb840cc410 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009870 tx=0x7fcb7401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 -- 172.21.15.45:0/1007090565 shutdown_connections 2024-04-25T10:20:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb840d1c30 0x7fcb840cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 --2- 172.21.15.45:0/1007090565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415b9e0 0x7fcb840cc410 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 -- 172.21.15.45:0/1007090565 >> 172.21.15.45:0/1007090565 conn(0x7fcb8409a9d0 msgr2=0x7fcb8409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 -- 172.21.15.45:0/1007090565 shutdown_connections 2024-04-25T10:20:32.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.375+0000 7fcb88e00640 1 -- 172.21.15.45:0/1007090565 wait complete. 2024-04-25T10:20:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 Processor -- start 2024-04-25T10:20:32.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 -- start start 2024-04-25T10:20:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb840d1c30 0x7fcb8415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8415b9e0 0x7fcb84153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 0x7fcb840ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840c65f0 con 0x7fcb8415bdc0 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840c6470 con 0x7fcb840d1c30 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840c6770 con 0x7fcb8415b9e0 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 0x7fcb840ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 0x7fcb840ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53302/0 (socket says 172.21.15.45:53302) 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 -- 172.21.15.45:0/575372172 learned_addr learned my addr 172.21.15.45:0/575372172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb81e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb840d1c30 0x7fcb8415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb7be00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8415b9e0 0x7fcb84153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 -- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8415b9e0 msgr2=0x7fcb84153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8415b9e0 0x7fcb84153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 -- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb840d1c30 msgr2=0x7fcb8415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb840d1c30 0x7fcb8415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 -- 172.21.15.45:0/575372172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8415d2c0 con 0x7fcb8415bdc0 2024-04-25T10:20:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb82800640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 0x7fcb840ca2d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00cf20 tx=0x7fcb7c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c01a070 con 0x7fcb8415bdc0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c00d840 con 0x7fcb8415bdc0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00dc30 con 0x7fcb8415bdc0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8415c2e0 con 0x7fcb8415bdc0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.379+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb840cab90 con 0x7fcb8415bdc0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.383+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8404ef30 con 0x7fcb8415bdc0 2024-04-25T10:20:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.383+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb7c01d070 con 0x7fcb8415bdc0 2024-04-25T10:20:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.383+0000 7fcb7a000640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcb60066f60 0x7fcb60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.383+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcb7c085940 con 0x7fcb8415bdc0 2024-04-25T10:20:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.383+0000 7fcb81e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcb60066f60 0x7fcb60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.383+0000 7fcb81e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcb60066f60 0x7fcb60069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb740af000 tx=0x7fcb740a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.387+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb7c0561d0 con 0x7fcb8415bdc0 2024-04-25T10:20:32.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.551+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb840d16b0 con 0x7fcb8415bdc0 2024-04-25T10:20:32.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.551+0000 7fcb7a000640 1 -- 172.21.15.45:0/575372172 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16912 (secure 0 0 0) 0x7fcb7c05a270 con 0x7fcb8415bdc0 2024-04-25T10:20:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:32.555 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:15.602282+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":38,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.14","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcb60066f60 msgr2=0x7fcb60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcb60066f60 0x7fcb60069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb740af000 tx=0x7fcb740a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 msgr2=0x7fcb840ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 0x7fcb840ca2d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00cf20 tx=0x7fcb7c00a7d0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 shutdown_connections 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcb60066f60 0x7fcb60069420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb8415bdc0 0x7fcb840ca2d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb8415b9e0 0x7fcb84153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 --2- 172.21.15.45:0/575372172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb840d1c30 0x7fcb8415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 >> 172.21.15.45:0/575372172 conn(0x7fcb8409a9d0 msgr2=0x7fcb8406f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 shutdown_connections 2024-04-25T10:20:32.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.555+0000 7fcb88e00640 1 -- 172.21.15.45:0/575372172 wait complete. 2024-04-25T10:20:32.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:18:53.475873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:54.611281+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 38, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:32.576 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:32.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e35c00640 1 Processor -- start 2024-04-25T10:20:32.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e35c00640 1 -- start start 2024-04-25T10:20:32.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e35c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e30070350 0x7f2e3015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e35c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3015b080 con 0x7f2e300b31c0 2024-04-25T10:20:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e35c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3015adb0 con 0x7f2e300aef80 2024-04-25T10:20:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e35c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3015eb60 con 0x7f2e30070350 2024-04-25T10:20:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2e300b31c0 0x7f2e300cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40546/0 (socket says 172.21.15.45:40546) 2024-04-25T10:20:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2f400640 1 -- 172.21.15.45:0/1240007236 learned_addr learned my addr 172.21.15.45:0/1240007236 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2e000640 1 --2- 172.21.15.45:0/1240007236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e30070350 0x7f2e3015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732583395 0 0) 0x7f2e3015b080 con 0x7f2e300b31c0 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e14003530 con 0x7f2e300b31c0 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381422095 0 0) 0x7f2e3015adb0 con 0x7f2e300aef80 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e3015b080 con 0x7f2e300aef80 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2e000640 1 -- 172.21.15.45:0/1240007236 >> v1:172.21.15.117:6789/0 conn(0x7f2e300aef80 legacy=0x7f2e3015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2e000640 1 -- 172.21.15.45:0/1240007236 >> v1:172.21.15.45:6789/0 conn(0x7f2e300b31c0 legacy=0x7f2e300cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.719+0000 7f2e2e000640 1 -- 172.21.15.45:0/1240007236 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3015ee40 con 0x7f2e30070350 2024-04-25T10:20:32.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e2e000640 1 --2- 172.21.15.45:0/1240007236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e30070350 0x7f2e3015e480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097f0 tx=0x7f2e1c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4f134eefc07555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0aa620 con 0x7f2e30070350 2024-04-25T10:20:32.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0af070 con 0x7f2e30070350 2024-04-25T10:20:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e2d600640 1 -- 172.21.15.45:0/1240007236 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0aa9e0 con 0x7f2e30070350 2024-04-25T10:20:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 -- 172.21.15.45:0/1240007236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e30070350 msgr2=0x7f2e3015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1240007236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e30070350 0x7f2e3015e480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097f0 tx=0x7f2e1c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 -- 172.21.15.45:0/1240007236 shutdown_connections 2024-04-25T10:20:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1240007236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e30070350 0x7f2e3015e480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 -- 172.21.15.45:0/1240007236 >> 172.21.15.45:0/1240007236 conn(0x7f2e300669a0 msgr2=0x7f2e30066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 -- 172.21.15.45:0/1240007236 shutdown_connections 2024-04-25T10:20:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.723+0000 7f2e35c00640 1 -- 172.21.15.45:0/1240007236 wait complete. 2024-04-25T10:20:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 Processor -- start 2024-04-25T10:20:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- start start 2024-04-25T10:20:32.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 0x7f2e3014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e300aef80 0x7f2e301503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e300b31c0 0x7f2e300cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300cfef0 con 0x7f2e30070350 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300cfd70 con 0x7f2e300aef80 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300b35d0 con 0x7f2e300b31c0 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 0x7f2e3014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 0x7f2e3014e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53310/0 (socket says 172.21.15.45:53310) 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 -- 172.21.15.45:0/1177509678 learned_addr learned my addr 172.21.15.45:0/1177509678 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2f400640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e300b31c0 0x7f2e300cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2e000640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e300aef80 0x7f2e301503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 -- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e300b31c0 msgr2=0x7f2e300cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e300b31c0 0x7f2e300cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 -- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e300aef80 msgr2=0x7f2e301503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e300aef80 0x7f2e301503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 -- 172.21.15.45:0/1177509678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300af4f0 con 0x7f2e30070350 2024-04-25T10:20:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e2ea00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 0x7f2e3014e930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c490 tx=0x7f2e2000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200052d0 con 0x7f2e30070350 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e20005470 con 0x7f2e30070350 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20012870 con 0x7f2e30070350 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e300af7d0 con 0x7f2e30070350 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e300afd90 con 0x7f2e30070350 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.727+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3004ef30 con 0x7f2e30070350 2024-04-25T10:20:32.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.731+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e20012a10 con 0x7f2e30070350 2024-04-25T10:20:32.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.731+0000 7f2e26a00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e10066f60 0x7f2e10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.731+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2e20084bf0 con 0x7f2e30070350 2024-04-25T10:20:32.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.731+0000 7f2e2e000640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e10066f60 0x7f2e10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.731+0000 7f2e2e000640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e10066f60 0x7f2e10069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0ae000 tx=0x7f2e1c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.735+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e20055510 con 0x7f2e30070350 2024-04-25T10:20:32.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.895+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e3014f3d0 con 0x7f2e30070350 2024-04-25T10:20:32.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.895+0000 7f2e26a00640 1 -- 172.21.15.45:0/1177509678 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e200595b0 con 0x7f2e30070350 2024-04-25T10:20:32.899 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e10066f60 msgr2=0x7f2e10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e10066f60 0x7f2e10069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0ae000 tx=0x7f2e1c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 msgr2=0x7f2e3014e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 0x7f2e3014e930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c490 tx=0x7f2e2000c950 comp rx=0 tx=0).stop 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 shutdown_connections 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e10066f60 0x7f2e10069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e300b31c0 0x7f2e300cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e300aef80 0x7f2e301503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 --2- 172.21.15.45:0/1177509678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e30070350 0x7f2e3014e930 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 >> 172.21.15.45:0/1177509678 conn(0x7f2e300669a0 msgr2=0x7f2e300cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 shutdown_connections 2024-04-25T10:20:32.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:32.899+0000 7f2e35c00640 1 -- 172.21.15.45:0/1177509678 wait complete. 2024-04-25T10:20:32.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 Processor -- start 2024-04-25T10:20:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 -- start start 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35f015a0e0 0x7f35f015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35f00aea50 0x7f35f00b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00b42d0 0x7f35f006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00af1d0 con 0x7f35f015a0e0 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00b2fd0 con 0x7f35f00aea50 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35f5200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00b32b0 con 0x7f35f00b42d0 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35ede00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35f015a0e0 0x7f35f015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35ede00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35f015a0e0 0x7f35f015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53314/0 (socket says 172.21.15.45:53314) 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35ede00640 1 -- 172.21.15.45:0/3633049632 learned_addr learned my addr 172.21.15.45:0/3633049632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35e7e00640 1 --2- 172.21.15.45:0/3633049632 >> v2:172.21.15.117:3300/0 conn(0x7f35f00aea50 0x7f35f00b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.059+0000 7f35ee800640 1 --2- 172.21.15.45:0/3633049632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00b42d0 0x7f35f006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ede00640 1 -- 172.21.15.45:0/3633049632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00b42d0 msgr2=0x7f35f006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ede00640 1 --2- 172.21.15.45:0/3633049632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00b42d0 0x7f35f006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ede00640 1 -- 172.21.15.45:0/3633049632 >> v2:172.21.15.117:3300/0 conn(0x7f35f00aea50 msgr2=0x7f35f00b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ede00640 1 --2- 172.21.15.45:0/3633049632 >> v2:172.21.15.117:3300/0 conn(0x7f35f00aea50 0x7f35f00b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ede00640 1 -- 172.21.15.45:0/3633049632 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00b3590 con 0x7f35f015a0e0 2024-04-25T10:20:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ede00640 1 --2- 172.21.15.45:0/3633049632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f015a0e0 0x7f35f015b150 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f35f015dae0 tx=0x7f35e001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98027e50b75e5de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ed400640 1 -- 172.21.15.45:0/3633049632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00b4070 con 0x7f35f015a0e0 2024-04-25T10:20:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ed400640 1 -- 172.21.15.45:0/3633049632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e00a7970 con 0x7f35f015a0e0 2024-04-25T10:20:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35ed400640 1 -- 172.21.15.45:0/3633049632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00a7d40 con 0x7f35f015a0e0 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 -- 172.21.15.45:0/3633049632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f015a0e0 msgr2=0x7f35f015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 --2- 172.21.15.45:0/3633049632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f015a0e0 0x7f35f015b150 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f35f015dae0 tx=0x7f35e001eb60 comp rx=0 tx=0).stop 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 -- 172.21.15.45:0/3633049632 shutdown_connections 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 --2- 172.21.15.45:0/3633049632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00b42d0 0x7f35f006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 --2- 172.21.15.45:0/3633049632 >> v2:172.21.15.117:3300/0 conn(0x7f35f00aea50 0x7f35f00b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 --2- 172.21.15.45:0/3633049632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f015a0e0 0x7f35f015b150 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.063+0000 7f35f5200640 1 -- 172.21.15.45:0/3633049632 >> 172.21.15.45:0/3633049632 conn(0x7f35f00669a0 msgr2=0x7f35f0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 -- 172.21.15.45:0/3633049632 shutdown_connections 2024-04-25T10:20:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 -- 172.21.15.45:0/3633049632 wait complete. 2024-04-25T10:20:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 Processor -- start 2024-04-25T10:20:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 -- start start 2024-04-25T10:20:33.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00aea50 0x7f35f0155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 0x7f35f0143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f015a0e0 0x7f35f0157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0070980 con 0x7f35f00b42d0 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0070800 con 0x7f35f015a0e0 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35f5200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0070b00 con 0x7f35f00aea50 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 0x7f35f0143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 0x7f35f0143b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53330/0 (socket says 172.21.15.45:53330) 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 -- 172.21.15.45:0/3635827957 learned_addr learned my addr 172.21.15.45:0/3635827957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35ede00640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00aea50 0x7f35f0155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35ee800640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f015a0e0 0x7f35f0157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 -- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00aea50 msgr2=0x7f35f0155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00aea50 0x7f35f0155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 -- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f015a0e0 msgr2=0x7f35f0157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f015a0e0 0x7f35f0157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 -- 172.21.15.45:0/3635827957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0158f60 con 0x7f35f00b42d0 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e7e00640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 0x7f35f0143b60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d800 tx=0x7f35d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.067+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800b6c0 con 0x7f35f00b42d0 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d8019070 con 0x7f35f00b42d0 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f01586c0 con 0x7f35f00b42d0 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d800ba60 con 0x7f35f00b42d0 2024-04-25T10:20:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f0158c80 con 0x7f35f00b42d0 2024-04-25T10:20:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f015ad70 con 0x7f35f00b42d0 2024-04-25T10:20:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35d800bc00 con 0x7f35f00b42d0 2024-04-25T10:20:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35e6000640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35cc066f60 0x7f35cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f35d80853e0 con 0x7f35f00b42d0 2024-04-25T10:20:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.071+0000 7f35ede00640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35cc066f60 0x7f35cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.075+0000 7f35ede00640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35cc066f60 0x7f35cc069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f35e00b1000 tx=0x7f35e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.075+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35f015ad70 con 0x7f35f00b42d0 2024-04-25T10:20:33.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35f006e1f0 con 0x7f35f00b42d0 2024-04-25T10:20:33.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35e6000640 1 -- 172.21.15.45:0/3635827957 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35d8055d00 con 0x7f35f00b42d0 2024-04-25T10:20:33.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:20:33.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35cc066f60 msgr2=0x7f35cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35cc066f60 0x7f35cc069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f35e00b1000 tx=0x7f35e0008040 comp rx=0 tx=0).stop 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 msgr2=0x7f35f0143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 0x7f35f0143b60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d800 tx=0x7f35d800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 shutdown_connections 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35cc066f60 0x7f35cc069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35f015a0e0 0x7f35f0157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35f00b42d0 0x7f35f0143b60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 --2- 172.21.15.45:0/3635827957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35f00aea50 0x7f35f0155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 >> 172.21.15.45:0/3635827957 conn(0x7f35f00669a0 msgr2=0x7f35f00af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 shutdown_connections 2024-04-25T10:20:33.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.243+0000 7f35f5200640 1 -- 172.21.15.45:0/3635827957 wait complete. 2024-04-25T10:20:33.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a52a00640 1 Processor -- start 2024-04-25T10:20:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a52a00640 1 -- start start 2024-04-25T10:20:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a52a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9a4c0b38d0 0x7f9a4c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a52a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c15b080 con 0x7f9a4c0aef80 2024-04-25T10:20:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a52a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c15adb0 con 0x7f9a4c070350 2024-04-25T10:20:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a52a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c15eb60 con 0x7f9a4c0b38d0 2024-04-25T10:20:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9a4c0aef80 0x7f9a4c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40554/0 (socket says 172.21.15.45:40554) 2024-04-25T10:20:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 -- 172.21.15.45:0/3327567501 learned_addr learned my addr 172.21.15.45:0/3327567501 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a50a00640 1 --2- 172.21.15.45:0/3327567501 >> v2:172.21.15.117:3300/0 conn(0x7f9a4c0b38d0 0x7f9a4c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332602721 0 0) 0x7f9a4c15b080 con 0x7f9a4c0aef80 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a30003530 con 0x7f9a4c0aef80 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2690122888 0 0) 0x7f9a30003530 con 0x7f9a4c0aef80 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 >> v2:172.21.15.117:3300/0 conn(0x7f9a4c0b38d0 msgr2=0x7f9a4c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 --2- 172.21.15.45:0/3327567501 >> v2:172.21.15.117:3300/0 conn(0x7f9a4c0b38d0 0x7f9a4c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 >> v1:172.21.15.117:6789/0 conn(0x7f9a4c070350 legacy=0x7f9a4c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c15ee40 con 0x7f9a4c0aef80 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f9a3c001680 con 0x7f9a4c0aef80 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 >> v1:172.21.15.45:6789/0 conn(0x7f9a4c0aef80 legacy=0x7f9a4c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a30003a10 0x7f9a30003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a30004810 0x7f9a30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a300051d0 0x7f9a3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c15eb60 con 0x7f9a30003a10 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3000c0d0 con 0x7f9a300051d0 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3000c2d0 con 0x7f9a30004810 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a30003a10 0x7f9a30003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a50a00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a300051d0 0x7f9a3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a43e00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a30004810 0x7f9a30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 -- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a30004810 msgr2=0x7f9a30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a30004810 0x7f9a30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 -- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a300051d0 msgr2=0x7f9a3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a300051d0 0x7f9a3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a4be00640 1 -- 172.21.15.45:0/3327567501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30003530 con 0x7f9a30003a10 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.407+0000 7f9a50a00640 1 -- 172.21.15.45:0/3327567501 reap_dead start 2024-04-25T10:20:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a43e00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a30004810 0x7f9a30004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a4be00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a30003a10 0x7f9a30003e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007c30 tx=0x7f9a3c01ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117ac5d260b839ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c007c70 con 0x7f9a30003a10 2024-04-25T10:20:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0ab070 con 0x7f9a30003a10 2024-04-25T10:20:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a4b400640 1 -- 172.21.15.45:0/3327567501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0058e0 con 0x7f9a30003a10 2024-04-25T10:20:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 -- 172.21.15.45:0/3327567501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a30003a10 msgr2=0x7f9a30003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a30003a10 0x7f9a30003e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007c30 tx=0x7f9a3c01ed70 comp rx=0 tx=0).stop 2024-04-25T10:20:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 -- 172.21.15.45:0/3327567501 shutdown_connections 2024-04-25T10:20:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 --2- 172.21.15.45:0/3327567501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a30003a10 0x7f9a30003e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 -- 172.21.15.45:0/3327567501 >> 172.21.15.45:0/3327567501 conn(0x7f9a4c0669a0 msgr2=0x7f9a4c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 -- 172.21.15.45:0/3327567501 shutdown_connections 2024-04-25T10:20:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.411+0000 7f9a52a00640 1 -- 172.21.15.45:0/3327567501 wait complete. 2024-04-25T10:20:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 Processor -- start 2024-04-25T10:20:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 -- start start 2024-04-25T10:20:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a30003a10 0x7f9a4c14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a4c0ad840 0x7f9a4c0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 0x7f9a4c1e6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0b35d0 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c15bd90 con 0x7f9a30003a10 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0b3750 con 0x7f9a4c0ad840 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 0x7f9a4c1e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 0x7f9a4c1e6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53350/0 (socket says 172.21.15.45:53350) 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 -- 172.21.15.45:0/349553063 learned_addr learned my addr 172.21.15.45:0/349553063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a4be00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a30003a10 0x7f9a4c14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a43e00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a4c0ad840 0x7f9a4c0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 -- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a4c0ad840 msgr2=0x7f9a4c0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a4c0ad840 0x7f9a4c0adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 -- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a30003a10 msgr2=0x7f9a4c14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a30003a10 0x7f9a4c14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 -- 172.21.15.45:0/349553063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c157120 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a50a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 0x7f9a4c1e6420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0b2f70 tx=0x7f9a4400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400fdb0 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a44015430 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c157400 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44015820 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.415+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c1e6960 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.419+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c0ae200 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.419+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a440159c0 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.419+0000 7f9a48c00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a28066f60 0x7f9a28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.419+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9a44085ed0 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.419+0000 7f9a4be00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a28066f60 0x7f9a28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.423+0000 7f9a4be00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a28066f60 0x7f9a28069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00c160 tx=0x7f9a3c0a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.423+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a44019030 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.583+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a4c04ef30 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.583+0000 7f9a48c00640 1 -- 172.21.15.45:0/349553063 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a44010040 con 0x7f9a4c1e5fa0 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a28066f60 msgr2=0x7f9a28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a28066f60 0x7f9a28069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00c160 tx=0x7f9a3c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 msgr2=0x7f9a4c1e6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 0x7f9a4c1e6420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0b2f70 tx=0x7f9a4400f580 comp rx=0 tx=0).stop 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 shutdown_connections 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a28066f60 0x7f9a28069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4c1e5fa0 0x7f9a4c1e6420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a4c0ad840 0x7f9a4c0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 --2- 172.21.15.45:0/349553063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a30003a10 0x7f9a4c14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 >> 172.21.15.45:0/349553063 conn(0x7f9a4c0669a0 msgr2=0x7f9a4c1555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 shutdown_connections 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.587+0000 7f9a52a00640 1 -- 172.21.15.45:0/349553063 wait complete. 2024-04-25T10:20:33.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:33.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.747+0000 7f92d5400640 1 Processor -- start 2024-04-25T10:20:33.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 -- start start 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92d00d1820 0x7f92d00d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d015e050 0x7f92d00b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f92d00b13e0 0x7f92d00ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d00b1870 con 0x7f92d00d1820 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d00b1af0 con 0x7f92d00b13e0 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92d5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00b1dd0 con 0x7f92d015e050 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92d00d1820 0x7f92d00d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92d00d1820 0x7f92d00d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53356/0 (socket says 172.21.15.45:53356) 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 -- 172.21.15.45:0/2792177143 learned_addr learned my addr 172.21.15.45:0/2792177143 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:33.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 -- 172.21.15.45:0/2792177143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d015e050 msgr2=0x7f92d00b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92cde00640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d015e050 0x7f92d00b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92cf200640 1 --2- 172.21.15.45:0/2792177143 >> v2:172.21.15.117:3300/0 conn(0x7f92d00b13e0 0x7f92d00ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d015e050 0x7f92d00b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 -- 172.21.15.45:0/2792177143 >> v2:172.21.15.117:3300/0 conn(0x7f92d00b13e0 msgr2=0x7f92d00ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 --2- 172.21.15.45:0/2792177143 >> v2:172.21.15.117:3300/0 conn(0x7f92d00b13e0 0x7f92d00ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92ce800640 1 -- 172.21.15.45:0/2792177143 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00b20b0 con 0x7f92d00d1820 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92cde00640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d015e050 0x7f92d00b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.751+0000 7f92cf200640 1 --2- 172.21.15.45:0/2792177143 >> v2:172.21.15.117:3300/0 conn(0x7f92d00b13e0 0x7f92d00ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92ce800640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d00d1820 0x7f92d00d1c00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92d006ce10 tx=0x7f92c001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5996bd1a085a674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92cd400640 1 -- 172.21.15.45:0/2792177143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c00b0070 con 0x7f92d00d1820 2024-04-25T10:20:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92cd400640 1 -- 172.21.15.45:0/2792177143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c00ab600 con 0x7f92d00d1820 2024-04-25T10:20:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92cd400640 1 -- 172.21.15.45:0/2792177143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c00ab9d0 con 0x7f92d00d1820 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 -- 172.21.15.45:0/2792177143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d00d1820 msgr2=0x7f92d00d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d00d1820 0x7f92d00d1c00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92d006ce10 tx=0x7f92c001c260 comp rx=0 tx=0).stop 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 -- 172.21.15.45:0/2792177143 shutdown_connections 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 --2- 172.21.15.45:0/2792177143 >> v2:172.21.15.117:3300/0 conn(0x7f92d00b13e0 0x7f92d00ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d015e050 0x7f92d00b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 --2- 172.21.15.45:0/2792177143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d00d1820 0x7f92d00d1c00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 -- 172.21.15.45:0/2792177143 >> 172.21.15.45:0/2792177143 conn(0x7f92d009a9d0 msgr2=0x7f92d009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 -- 172.21.15.45:0/2792177143 shutdown_connections 2024-04-25T10:20:33.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 -- 172.21.15.45:0/2792177143 wait complete. 2024-04-25T10:20:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 Processor -- start 2024-04-25T10:20:33.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.755+0000 7f92d5400640 1 -- start start 2024-04-25T10:20:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d00b13e0 0x7f92d014f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92d00d1820 0x7f92d00b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 0x7f92d006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00cfdf0 con 0x7f92d015e050 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00cfc70 con 0x7f92d00d1820 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00cff70 con 0x7f92d00b13e0 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 0x7f92d006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 0x7f92d006fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53364/0 (socket says 172.21.15.45:53364) 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 -- 172.21.15.45:0/3211146048 learned_addr learned my addr 172.21.15.45:0/3211146048 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cde00640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92d00d1820 0x7f92d00b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92ce800640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d00b13e0 0x7f92d014f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 -- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d00b13e0 msgr2=0x7f92d014f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d00b13e0 0x7f92d014f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 -- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92d00d1820 msgr2=0x7f92d00b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92d00d1820 0x7f92d00b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 -- 172.21.15.45:0/3211146048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00cc610 con 0x7f92d015e050 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92cf200640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 0x7f92d006fa30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f92c800cef0 tx=0x7f92c800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c801a070 con 0x7f92d015e050 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c800d860 con 0x7f92d015e050 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800dc50 con 0x7f92d015e050 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d015b7f0 con 0x7f92d015e050 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d015bdb0 con 0x7f92d015e050 2024-04-25T10:20:33.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.759+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d004ef30 con 0x7f92d015e050 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.763+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92c801d070 con 0x7f92d015e050 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.763+0000 7f92c6a00640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92b0066f90 0x7f92b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.763+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f92c8085890 con 0x7f92d015e050 2024-04-25T10:20:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.763+0000 7f92ce800640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92b0066f90 0x7f92b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.763+0000 7f92ce800640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92b0066f90 0x7f92b0069450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f92c00ad000 tx=0x7f92c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.767+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92c8056270 con 0x7f92d015e050 2024-04-25T10:20:33.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.963+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92d00ce310 con 0x7f92d015e050 2024-04-25T10:20:33.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.963+0000 7f92c6a00640 1 -- 172.21.15.45:0/3211146048 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f92c805a310 con 0x7f92d015e050 2024-04-25T10:20:33.968 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:33.968 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:18:55.618016+0000","modified":"2024-04-25T10:19:08.145993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3482819589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3482819589},{"type":"v1","addr":"172.21.15.45:6835","nonce":3482819589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:20:33.968 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 18 2024-04-25T10:20:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92b0066f90 msgr2=0x7f92b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92b0066f90 0x7f92b0069450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f92c00ad000 tx=0x7f92c0008040 comp rx=0 tx=0).stop 2024-04-25T10:20:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 msgr2=0x7f92d006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 0x7f92d006fa30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f92c800cef0 tx=0x7f92c800a710 comp rx=0 tx=0).stop 2024-04-25T10:20:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 shutdown_connections 2024-04-25T10:20:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92b0066f90 0x7f92b0069450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92d015e050 0x7f92d006fa30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92d00d1820 0x7f92d00b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 --2- 172.21.15.45:0/3211146048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92d00b13e0 0x7f92d014f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 >> 172.21.15.45:0/3211146048 conn(0x7f92d009a9d0 msgr2=0x7f92d00d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 shutdown_connections 2024-04-25T10:20:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:33.967+0000 7f92d5400640 1 -- 172.21.15.45:0/3211146048 wait complete. 2024-04-25T10:20:33.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21df000640 1 Processor -- start 2024-04-25T10:20:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21df000640 1 -- start start 2024-04-25T10:20:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21df000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21d815bc40 0x7f21d806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21df000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80b2770 con 0x7f21d80cda10 2024-04-25T10:20:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21df000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d806c670 con 0x7f21d80cf930 2024-04-25T10:20:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21df000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d806c950 con 0x7f21d815bc40 2024-04-25T10:20:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21d815bc40 0x7f21d806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f21d815bc40 0x7f21d806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53368/0 (socket says 172.21.15.45:53368) 2024-04-25T10:20:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7e00640 1 -- 172.21.15.45:0/529838757 learned_addr learned my addr 172.21.15.45:0/529838757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7e00640 1 -- 172.21.15.45:0/529838757 >> v1:172.21.15.117:6789/0 conn(0x7f21d80cf930 legacy=0x7f21d80b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7e00640 1 -- 172.21.15.45:0/529838757 >> v1:172.21.15.45:6789/0 conn(0x7f21d80cda10 legacy=0x7f21d80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7400640 1 -- 172.21.15.45:0/529838757 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899318378 0 0) 0x7f21d80b2770 con 0x7f21d80cda10 2024-04-25T10:20:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.127+0000 7f21d7e00640 1 -- 172.21.15.45:0/529838757 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d806cc30 con 0x7f21d815bc40 2024-04-25T10:20:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21d7e00640 1 --2- 172.21.15.45:0/529838757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d815bc40 0x7f21d806c090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009870 tx=0x7f21c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb374e754b501cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21d7400640 1 -- 172.21.15.45:0/529838757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80b0070 con 0x7f21d815bc40 2024-04-25T10:20:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21d7400640 1 -- 172.21.15.45:0/529838757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c80ab600 con 0x7f21d815bc40 2024-04-25T10:20:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21d7400640 1 -- 172.21.15.45:0/529838757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80ab9d0 con 0x7f21d815bc40 2024-04-25T10:20:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 -- 172.21.15.45:0/529838757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d815bc40 msgr2=0x7f21d806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 --2- 172.21.15.45:0/529838757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d815bc40 0x7f21d806c090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009870 tx=0x7f21c801c490 comp rx=0 tx=0).stop 2024-04-25T10:20:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 -- 172.21.15.45:0/529838757 shutdown_connections 2024-04-25T10:20:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 --2- 172.21.15.45:0/529838757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d815bc40 0x7f21d806c090 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 -- 172.21.15.45:0/529838757 >> 172.21.15.45:0/529838757 conn(0x7f21d8066b70 msgr2=0x7f21d8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 -- 172.21.15.45:0/529838757 shutdown_connections 2024-04-25T10:20:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.131+0000 7f21df000640 1 -- 172.21.15.45:0/529838757 wait complete. 2024-04-25T10:20:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 Processor -- start 2024-04-25T10:20:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- start start 2024-04-25T10:20:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d80cda10 0x7f21d8157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 0x7f21d815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d815bc40 0x7f21d80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8169d70 con 0x7f21d80cf930 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8169bf0 con 0x7f21d815bc40 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8169ef0 con 0x7f21d80cda10 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 0x7f21d815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 0x7f21d815ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53374/0 (socket says 172.21.15.45:53374) 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 -- 172.21.15.45:0/393398115 learned_addr learned my addr 172.21.15.45:0/393398115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21d7e00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d80cda10 0x7f21d8157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21dd000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d815bc40 0x7f21d80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 -- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d80cda10 msgr2=0x7f21d8157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d80cda10 0x7f21d8157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 -- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d815bc40 msgr2=0x7f21d80af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d815bc40 0x7f21d80af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 -- 172.21.15.45:0/393398115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80b03d0 con 0x7f21d80cf930 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21cfe00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 0x7f21d815ec50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21c0001ec0 tx=0x7f21c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0017070 con 0x7f21d80cf930 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c0003600 con 0x7f21d80cf930 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c00039f0 con 0x7f21d80cf930 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80b1d90 con 0x7f21d80cf930 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d80b1a40 con 0x7f21d80cf930 2024-04-25T10:20:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.135+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d804ef30 con 0x7f21d80cf930 2024-04-25T10:20:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.139+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21c001a070 con 0x7f21d80cf930 2024-04-25T10:20:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.139+0000 7f21d4c00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f21b4066f90 0x7f21b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.139+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f21c0084460 con 0x7f21d80cf930 2024-04-25T10:20:34.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.139+0000 7f21d7e00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f21b4066f90 0x7f21b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.139+0000 7f21d7e00640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f21b4066f90 0x7f21b4069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f21c80ad000 tx=0x7f21c80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.143+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21c00553d0 con 0x7f21d80cf930 2024-04-25T10:20:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.303+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21d8072300 con 0x7f21d80cf930 2024-04-25T10:20:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.303+0000 7f21d4c00640 1 -- 172.21.15.45:0/393398115 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16912 (secure 0 0 0) 0x7f21c0054cd0 con 0x7f21d80cf930 2024-04-25T10:20:34.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:34.308 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:15.602282+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":38,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.14","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f21b4066f90 msgr2=0x7f21b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f21b4066f90 0x7f21b4069450 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f21c80ad000 tx=0x7f21c80a6000 comp rx=0 tx=0).stop 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 msgr2=0x7f21d815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 0x7f21d815ec50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21c0001ec0 tx=0x7f21c000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 shutdown_connections 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f21b4066f90 0x7f21b4069450 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d815bc40 0x7f21d80af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21d80cf930 0x7f21d815ec50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 --2- 172.21.15.45:0/393398115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21d80cda10 0x7f21d8157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 >> 172.21.15.45:0/393398115 conn(0x7f21d8066b70 msgr2=0x7f21d8159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 shutdown_connections 2024-04-25T10:20:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.307+0000 7f21df000640 1 -- 172.21.15.45:0/393398115 wait complete. 2024-04-25T10:20:34.332 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:20:34.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:34.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.467+0000 7f653a800640 1 Processor -- start 2024-04-25T10:20:34.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.467+0000 7f653a800640 1 -- start start 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f653a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6534066c00 0x7f653415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f653a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066fe0 0x7f65340b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f653a800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65340b5c20 con 0x7f653406ed30 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f653a800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65340af6e0 con 0x7f6534066c00 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f653a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65340af9c0 con 0x7f6534066fe0 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6534066c00 0x7f653415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6534066c00 0x7f653415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53378/0 (socket says 172.21.15.45:53378) 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f653406ed30 0x7f6534157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40574/0 (socket says 172.21.15.45:40574) 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 -- 172.21.15.45:0/2754040367 learned_addr learned my addr 172.21.15.45:0/2754040367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f652aa00640 1 --2- 172.21.15.45:0/2754040367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066fe0 0x7f65340b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 -- 172.21.15.45:0/2754040367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066fe0 msgr2=0x7f65340b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 --2- 172.21.15.45:0/2754040367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066fe0 0x7f65340b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 -- 172.21.15.45:0/2754040367 >> v1:172.21.15.45:6789/0 conn(0x7f653406ed30 legacy=0x7f6534157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6533400640 1 -- 172.21.15.45:0/2754040367 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65340afca0 con 0x7f6534066c00 2024-04-25T10:20:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.471+0000 7f6532a00640 1 -- 172.21.15.45:0/2754040367 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251667917 0 0) 0x7f65340b5c20 con 0x7f653406ed30 2024-04-25T10:20:34.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f6533400640 1 --2- 172.21.15.45:0/2754040367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066c00 0x7f653415e110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6524009870 tx=0x7f652401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1731a84721077b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f6532a00640 1 -- 172.21.15.45:0/2754040367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240a9610 con 0x7f6534066c00 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f6532a00640 1 -- 172.21.15.45:0/2754040367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65240a97b0 con 0x7f6534066c00 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f6532a00640 1 -- 172.21.15.45:0/2754040367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240a9b80 con 0x7f6534066c00 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- 172.21.15.45:0/2754040367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066c00 msgr2=0x7f653415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 --2- 172.21.15.45:0/2754040367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066c00 0x7f653415e110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6524009870 tx=0x7f652401c530 comp rx=0 tx=0).stop 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- 172.21.15.45:0/2754040367 shutdown_connections 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 --2- 172.21.15.45:0/2754040367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066fe0 0x7f65340b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 --2- 172.21.15.45:0/2754040367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066c00 0x7f653415e110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- 172.21.15.45:0/2754040367 >> 172.21.15.45:0/2754040367 conn(0x7f65340b24f0 msgr2=0x7f65340b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- 172.21.15.45:0/2754040367 shutdown_connections 2024-04-25T10:20:34.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- 172.21.15.45:0/2754040367 wait complete. 2024-04-25T10:20:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 Processor -- start 2024-04-25T10:20:34.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- start start 2024-04-25T10:20:34.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066c00 0x7f6534151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 0x7f653414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f653406ed30 0x7f653415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653406ac90 con 0x7f6534066fe0 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65340af270 con 0x7f653406ed30 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.475+0000 7f653a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653406ae10 con 0x7f6534066c00 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 0x7f653414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 0x7f653414d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53386/0 (socket says 172.21.15.45:53386) 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 -- 172.21.15.45:0/22824090 learned_addr learned my addr 172.21.15.45:0/22824090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f6533400640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066c00 0x7f6534151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 -- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066c00 msgr2=0x7f6534151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f6533e00640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f653406ed30 0x7f653415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066c00 0x7f6534151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 -- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f653406ed30 msgr2=0x7f653415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f653406ed30 0x7f653415d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 -- 172.21.15.45:0/22824090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534151c80 con 0x7f6534066fe0 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f6533400640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066c00 0x7f6534151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652aa00640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 0x7f653414d170 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f651c00d830 tx=0x7f651c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00b6c0 con 0x7f6534066fe0 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c019070 con 0x7f6534066fe0 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00ba60 con 0x7f6534066fe0 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6534151f60 con 0x7f6534066fe0 2024-04-25T10:20:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6534152520 con 0x7f6534066fe0 2024-04-25T10:20:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65341453f0 con 0x7f6534066fe0 2024-04-25T10:20:34.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.479+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f651c01e460 con 0x7f6534066fe0 2024-04-25T10:20:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.483+0000 7f652be00640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6510066f60 0x7f6510069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.483+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f651c0857b0 con 0x7f6534066fe0 2024-04-25T10:20:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.483+0000 7f6533400640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6510066f60 0x7f6510069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.483+0000 7f6533400640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6510066f60 0x7f6510069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65240ae000 tx=0x7f65240b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.487+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f651c056720 con 0x7f6534066fe0 2024-04-25T10:20:34.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f653404ef30 con 0x7f6534066fe0 2024-04-25T10:20:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f652be00640 1 -- 172.21.15.45:0/22824090 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f651c056020 con 0x7f6534066fe0 2024-04-25T10:20:34.686 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:34.687 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:18:55.618016+0000","modified":"2024-04-25T10:19:08.145993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3482819589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3482819589},{"type":"v1","addr":"172.21.15.45:6835","nonce":3482819589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:20:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 18 2024-04-25T10:20:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6510066f60 msgr2=0x7f6510069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6510066f60 0x7f6510069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65240ae000 tx=0x7f65240b3040 comp rx=0 tx=0).stop 2024-04-25T10:20:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 msgr2=0x7f653414d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 0x7f653414d170 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f651c00d830 tx=0x7f651c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 shutdown_connections 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6510066f60 0x7f6510069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f653406ed30 0x7f653415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534066fe0 0x7f653414d170 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 --2- 172.21.15.45:0/22824090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6534066c00 0x7f6534151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 >> 172.21.15.45:0/22824090 conn(0x7f65340b24f0 msgr2=0x7f65340ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 shutdown_connections 2024-04-25T10:20:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.683+0000 7f653a800640 1 -- 172.21.15.45:0/22824090 wait complete. 2024-04-25T10:20:34.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.843+0000 7f9e14c00640 1 Processor -- start 2024-04-25T10:20:34.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.843+0000 7f9e14c00640 1 -- start start 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e14c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101550b0 0x7f9e100cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e14c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e1015b910 0x7f9e100b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e14c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e10071050 con 0x7f9e100cfe70 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e14c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1015bda0 con 0x7f9e1015b910 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e14c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1015d040 con 0x7f9e101550b0 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e1015b910 0x7f9e100b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e1015b910 0x7f9e100b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53396/0 (socket says 172.21.15.45:53396) 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 -- 172.21.15.45:0/204143924 learned_addr learned my addr 172.21.15.45:0/204143924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:34.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0de00640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101550b0 0x7f9e100cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 -- 172.21.15.45:0/204143924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101550b0 msgr2=0x7f9e100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101550b0 0x7f9e100cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 -- 172.21.15.45:0/204143924 >> v1:172.21.15.117:6789/0 conn(0x7f9e100cfe70 legacy=0x7f9e1015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0e800640 1 -- 172.21.15.45:0/204143924 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100712f0 con 0x7f9e1015b910 2024-04-25T10:20:34.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.847+0000 7f9e0de00640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101550b0 0x7f9e100cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e0e800640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e1015b910 0x7f9e100b3260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009e50 tx=0x7f9e0801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3a37727f36ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e0d400640 1 -- 172.21.15.45:0/204143924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a5760 con 0x7f9e1015b910 2024-04-25T10:20:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e0d400640 1 -- 172.21.15.45:0/204143924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e080a5900 con 0x7f9e1015b910 2024-04-25T10:20:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e0d400640 1 -- 172.21.15.45:0/204143924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a5cd0 con 0x7f9e1015b910 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- 172.21.15.45:0/204143924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e1015b910 msgr2=0x7f9e100b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e1015b910 0x7f9e100b3260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009e50 tx=0x7f9e0801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- 172.21.15.45:0/204143924 shutdown_connections 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e1015b910 0x7f9e100b3260 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 --2- 172.21.15.45:0/204143924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e101550b0 0x7f9e100cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- 172.21.15.45:0/204143924 >> 172.21.15.45:0/204143924 conn(0x7f9e10066b70 msgr2=0x7f9e10066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- 172.21.15.45:0/204143924 shutdown_connections 2024-04-25T10:20:34.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- 172.21.15.45:0/204143924 wait complete. 2024-04-25T10:20:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 Processor -- start 2024-04-25T10:20:34.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- start start 2024-04-25T10:20:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 0x7f9e10145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e101550b0 0x7f9e10152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e1015b910 0x7f9e100cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1015c030 con 0x7f9e100cfe70 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100720b0 con 0x7f9e101550b0 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e14c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1015c1b0 con 0x7f9e1015b910 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.851+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 0x7f9e10145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 0x7f9e10145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53404/0 (socket says 172.21.15.45:53404) 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 -- 172.21.15.45:0/2006923207 learned_addr learned my addr 172.21.15.45:0/2006923207 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0e800640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e1015b910 0x7f9e100cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e07e00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e101550b0 0x7f9e10152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 -- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e1015b910 msgr2=0x7f9e100cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e1015b910 0x7f9e100cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 -- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e101550b0 msgr2=0x7f9e10152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e101550b0 0x7f9e10152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 -- 172.21.15.45:0/2006923207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10158aa0 con 0x7f9e100cfe70 2024-04-25T10:20:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e07e00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e101550b0 0x7f9e10152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e0de00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 0x7f9e10145300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c7c0 tx=0x7f9e0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00007a30 con 0x7f9e100cfe70 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e00007bd0 con 0x7f9e100cfe70 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e000113f0 con 0x7f9e100cfe70 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e10152f30 con 0x7f9e100cfe70 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e10153210 con 0x7f9e100cfe70 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1004ef30 con 0x7f9e100cfe70 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e00011590 con 0x7f9e100cfe70 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e06000640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dec066f90 0x7f9dec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.855+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9e00085270 con 0x7f9e100cfe70 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.859+0000 7f9e07e00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dec066f90 0x7f9dec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.859+0000 7f9e07e00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dec066f90 0x7f9dec069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e101fa010 tx=0x7f9df8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:34.859+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e00015020 con 0x7f9e100cfe70 2024-04-25T10:20:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.055+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e1014da80 con 0x7f9e100cfe70 2024-04-25T10:20:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e06000640 1 -- 172.21.15.45:0/2006923207 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f9e00055b00 con 0x7f9e100cfe70 2024-04-25T10:20:35.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:35.061 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:18:55.618016+0000","modified":"2024-04-25T10:19:08.145993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5055":{"gid":5055,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3482819589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3482819589},{"type":"v1","addr":"172.21.15.45:6835","nonce":3482819589}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:20:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 18 2024-04-25T10:20:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dec066f90 msgr2=0x7f9dec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dec066f90 0x7f9dec069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e101fa010 tx=0x7f9df8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 msgr2=0x7f9e10145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 0x7f9e10145300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c7c0 tx=0x7f9e0000cc80 comp rx=0 tx=0).stop 2024-04-25T10:20:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 shutdown_connections 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dec066f90 0x7f9dec069450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e1015b910 0x7f9e100cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e101550b0 0x7f9e10152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 --2- 172.21.15.45:0/2006923207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e100cfe70 0x7f9e10145300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 >> 172.21.15.45:0/2006923207 conn(0x7f9e10066b70 msgr2=0x7f9e100b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 shutdown_connections 2024-04-25T10:20:35.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.059+0000 7f9e14c00640 1 -- 172.21.15.45:0/2006923207 wait complete. 2024-04-25T10:20:35.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 Processor -- start 2024-04-25T10:20:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 -- start start 2024-04-25T10:20:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa1640cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1640cfe70 0x7fa16415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa164071050 con 0x7fa16415b740 2024-04-25T10:20:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16415d040 con 0x7fa1640cfe70 2024-04-25T10:20:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa16ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640712f0 con 0x7fa1641550b0 2024-04-25T10:20:35.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa163e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1640cfe70 0x7fa16415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa1640cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa16415b740 0x7fa16406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:60468/0 (socket says 172.21.15.45:60468) 2024-04-25T10:20:35.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa1640cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:38940/0 (socket says 172.21.15.45:38940) 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169a00640 1 -- 172.21.15.45:0/2458839495 learned_addr learned my addr 172.21.15.45:0/2458839495 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa163e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1640cfe70 0x7fa16415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49466/0 (socket says 172.21.15.45:49466) 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa163400640 1 -- 172.21.15.45:0/2458839495 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738648307 0 0) 0x7fa164071050 con 0x7fa16415b740 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169000640 1 -- 172.21.15.45:0/2458839495 >> v2:172.21.15.117:3300/0 conn(0x7fa1640cfe70 msgr2=0x7fa16415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169000640 1 --2- 172.21.15.45:0/2458839495 >> v2:172.21.15.117:3300/0 conn(0x7fa1640cfe70 0x7fa16415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169000640 1 -- 172.21.15.45:0/2458839495 >> v1:172.21.15.117:6789/0 conn(0x7fa16415b740 legacy=0x7fa16406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.219+0000 7fa169000640 1 -- 172.21.15.45:0/2458839495 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640715d0 con 0x7fa1641550b0 2024-04-25T10:20:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa169000640 1 --2- 172.21.15.45:0/2458839495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa1640cf930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1580097c0 tx=0x7fa15801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb3ce0c386d796d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa163400640 1 -- 172.21.15.45:0/2458839495 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580b0070 con 0x7fa1641550b0 2024-04-25T10:20:35.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa163400640 1 -- 172.21.15.45:0/2458839495 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1580ab600 con 0x7fa1641550b0 2024-04-25T10:20:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa163400640 1 -- 172.21.15.45:0/2458839495 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580ab9d0 con 0x7fa1641550b0 2024-04-25T10:20:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 -- 172.21.15.45:0/2458839495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 msgr2=0x7fa1640cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 --2- 172.21.15.45:0/2458839495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa1640cf930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1580097c0 tx=0x7fa15801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 -- 172.21.15.45:0/2458839495 shutdown_connections 2024-04-25T10:20:35.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 --2- 172.21.15.45:0/2458839495 >> v2:172.21.15.117:3300/0 conn(0x7fa1640cfe70 0x7fa16415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 --2- 172.21.15.45:0/2458839495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa1640cf930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 -- 172.21.15.45:0/2458839495 >> 172.21.15.45:0/2458839495 conn(0x7fa164066b70 msgr2=0x7fa164066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 -- 172.21.15.45:0/2458839495 shutdown_connections 2024-04-25T10:20:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.223+0000 7fa16ba00640 1 -- 172.21.15.45:0/2458839495 wait complete. 2024-04-25T10:20:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 Processor -- start 2024-04-25T10:20:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 -- start start 2024-04-25T10:20:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1640cfe70 0x7fa1640cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa164155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 0x7fa164135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415bd10 con 0x7fa16415b740 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415bb90 con 0x7fa1640cfe70 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa16ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415c030 con 0x7fa1641550b0 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 0x7fa164135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1640cfe70 0x7fa1640cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 0x7fa164135e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39556/0 (socket says 172.21.15.45:39556) 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169a00640 1 -- 172.21.15.45:0/1706132485 learned_addr learned my addr 172.21.15.45:0/1706132485 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa163e00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa164155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 -- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 msgr2=0x7fa164155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa164155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 -- 172.21.15.45:0/1706132485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 msgr2=0x7fa164135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 0x7fa164135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 -- 172.21.15.45:0/1706132485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640cd940 con 0x7fa1640cfe70 2024-04-25T10:20:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169a00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 0x7fa164135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa169000640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1640cfe70 0x7fa1640cd0b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1580ab000 tx=0x7fa15801ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.227+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580bd070 con 0x7fa1640cfe70 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158004030 con 0x7fa1640cfe70 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1580b0040 con 0x7fa1640cfe70 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1640cdc20 con 0x7fa1640cfe70 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1640ce1e0 con 0x7fa1640cfe70 2024-04-25T10:20:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16404ef30 con 0x7fa1640cfe70 2024-04-25T10:20:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa158003620 con 0x7fa1640cfe70 2024-04-25T10:20:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa160c00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa15811ef60 con 0x7fa1640cfe70 2024-04-25T10:20:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.231+0000 7fa163e00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.235+0000 7fa163e00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa144066f60 0x7fa144069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1640ce850 tx=0x7fa150009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.235+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1580c6020 con 0x7fa1640cfe70 2024-04-25T10:20:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.403+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1640cfc50 con 0x7fa1640cfe70 2024-04-25T10:20:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.403+0000 7fa160c00640 1 -- 172.21.15.45:0/1706132485 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16912 (secure 0 0 0) 0x7fa158016090 con 0x7fa1640cfe70 2024-04-25T10:20:35.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:35.408 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:15.602282+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:53.475873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:54.611281+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":38,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.14","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.3c","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa144066f60 msgr2=0x7fa144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa144066f60 0x7fa144069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1640ce850 tx=0x7fa150009190 comp rx=0 tx=0).stop 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1640cfe70 msgr2=0x7fa1640cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1640cfe70 0x7fa1640cd0b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1580ab000 tx=0x7fa15801ace0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 shutdown_connections 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa16415b740 0x7fa164135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1641550b0 0x7fa164155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 --2- 172.21.15.45:0/1706132485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1640cfe70 0x7fa1640cd0b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 >> 172.21.15.45:0/1706132485 conn(0x7fa164066b70 msgr2=0x7fa16414c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 shutdown_connections 2024-04-25T10:20:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.407+0000 7fa16ba00640 1 -- 172.21.15.45:0/1706132485 wait complete. 2024-04-25T10:20:35.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:20:35.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 Processor -- start 2024-04-25T10:20:35.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 -- start start 2024-04-25T10:20:35.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc300b0f70 0x7fbc300b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cfd90 0x7fbc300ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbc300cd110 0x7fbc300cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3015ef20 con 0x7fbc300b0f70 2024-04-25T10:20:35.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3015f3b0 con 0x7fbc300cd110 2024-04-25T10:20:35.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc35e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300ca810 con 0x7fbc300cfd90 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc300b0f70 0x7fbc300b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc300b0f70 0x7fbc300b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39562/0 (socket says 172.21.15.45:39562) 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 -- 172.21.15.45:0/847671588 learned_addr learned my addr 172.21.15.45:0/847671588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc27e00640 1 --2- 172.21.15.45:0/847671588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cfd90 0x7fbc300ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2f400640 1 --2- 172.21.15.45:0/847671588 >> v2:172.21.15.117:3300/0 conn(0x7fbc300cd110 0x7fbc300cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 -- 172.21.15.45:0/847671588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cfd90 msgr2=0x7fbc300ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 --2- 172.21.15.45:0/847671588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cfd90 0x7fbc300ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 -- 172.21.15.45:0/847671588 >> v2:172.21.15.117:3300/0 conn(0x7fbc300cd110 msgr2=0x7fbc300cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 --2- 172.21.15.45:0/847671588 >> v2:172.21.15.117:3300/0 conn(0x7fbc300cd110 0x7fbc300cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.567+0000 7fbc2ea00640 1 -- 172.21.15.45:0/847671588 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300b1890 con 0x7fbc300b0f70 2024-04-25T10:20:35.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc2ea00640 1 --2- 172.21.15.45:0/847671588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300b1350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbc300b2d50 tx=0x7fbc2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d9f09e4b22be90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc2e000640 1 -- 172.21.15.45:0/847671588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200b0070 con 0x7fbc300b0f70 2024-04-25T10:20:35.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc2e000640 1 -- 172.21.15.45:0/847671588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc200ab600 con 0x7fbc300b0f70 2024-04-25T10:20:35.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc2e000640 1 -- 172.21.15.45:0/847671588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200ab9d0 con 0x7fbc300b0f70 2024-04-25T10:20:35.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 -- 172.21.15.45:0/847671588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 msgr2=0x7fbc300b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 --2- 172.21.15.45:0/847671588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300b1350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbc300b2d50 tx=0x7fbc2001c520 comp rx=0 tx=0).stop 2024-04-25T10:20:35.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 -- 172.21.15.45:0/847671588 shutdown_connections 2024-04-25T10:20:35.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 --2- 172.21.15.45:0/847671588 >> v2:172.21.15.117:3300/0 conn(0x7fbc300cd110 0x7fbc300cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 --2- 172.21.15.45:0/847671588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cfd90 0x7fbc300ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 --2- 172.21.15.45:0/847671588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300b1350 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 -- 172.21.15.45:0/847671588 >> 172.21.15.45:0/847671588 conn(0x7fbc300b1d50 msgr2=0x7fbc300b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 -- 172.21.15.45:0/847671588 shutdown_connections 2024-04-25T10:20:35.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.571+0000 7fbc35e00640 1 -- 172.21.15.45:0/847671588 wait complete. 2024-04-25T10:20:35.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 Processor -- start 2024-04-25T10:20:35.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 -- start start 2024-04-25T10:20:35.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cd110 0x7fbc30155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc300cfd90 0x7fbc300c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30072200 con 0x7fbc300b0f70 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300cd520 con 0x7fbc300cfd90 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30072380 con 0x7fbc300cd110 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39564/0 (socket says 172.21.15.45:39564) 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 -- 172.21.15.45:0/1231033809 learned_addr learned my addr 172.21.15.45:0/1231033809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc27e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cd110 0x7fbc30155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2f400640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc300cfd90 0x7fbc300c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 -- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cd110 msgr2=0x7fbc30155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cd110 0x7fbc30155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 -- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc300cfd90 msgr2=0x7fbc300c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc300cfd90 0x7fbc300c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 -- 172.21.15.45:0/1231033809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300afe20 con 0x7fbc300b0f70 2024-04-25T10:20:35.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2f400640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc300cfd90 0x7fbc300c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:35.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc2ea00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300ae570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbc2001cdc0 tx=0x7fbc2001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200ba020 con 0x7fbc300b0f70 2024-04-25T10:20:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc200b6420 con 0x7fbc300b0f70 2024-04-25T10:20:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc301581b0 con 0x7fbc300b0f70 2024-04-25T10:20:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200b0040 con 0x7fbc300b0f70 2024-04-25T10:20:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.575+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc300b0090 con 0x7fbc300b0f70 2024-04-25T10:20:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.579+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3004ef30 con 0x7fbc300b0f70 2024-04-25T10:20:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.579+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc200bd020 con 0x7fbc300b0f70 2024-04-25T10:20:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.579+0000 7fbc27400640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc0c066f90 0x7fbc0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.579+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbc2011f2a0 con 0x7fbc300b0f70 2024-04-25T10:20:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.579+0000 7fbc27e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc0c066f90 0x7fbc0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.579+0000 7fbc27e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc0c066f90 0x7fbc0c069450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc300d0200 tx=0x7fbc18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.583+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc200c4030 con 0x7fbc300b0f70 2024-04-25T10:20:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.783+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc300cc9b0 con 0x7fbc300b0f70 2024-04-25T10:20:35.787 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:20:35.783+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:20:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc27400640 1 -- 172.21.15.45:0/1231033809 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc20016110 con 0x7fbc300b0f70 2024-04-25T10:20:35.789 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:20:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc0c066f90 msgr2=0x7fbc0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc0c066f90 0x7fbc0c069450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc300d0200 tx=0x7fbc18008040 comp rx=0 tx=0).stop 2024-04-25T10:20:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 msgr2=0x7fbc300ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300ae570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbc2001cdc0 tx=0x7fbc2001ad60 comp rx=0 tx=0).stop 2024-04-25T10:20:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 shutdown_connections 2024-04-25T10:20:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc0c066f90 0x7fbc0c069450 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc300cfd90 0x7fbc300c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc300cd110 0x7fbc30155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 --2- 172.21.15.45:0/1231033809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc300b0f70 0x7fbc300ae570 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 >> 172.21.15.45:0/1231033809 conn(0x7fbc300b1d50 msgr2=0x7fbc30072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 shutdown_connections 2024-04-25T10:20:35.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.787+0000 7fbc35e00640 1 -- 172.21.15.45:0/1231033809 wait complete. 2024-04-25T10:20:35.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:20:35.868 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:20:35.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.955+0000 7f4cad600640 1 Processor -- start 2024-04-25T10:20:35.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.955+0000 7f4cad600640 1 -- start start 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4cad600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ca8066c00 0x7f4ca8161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4cad600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066fe0 0x7f4ca8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4cad600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ca80b5560 0x7f4ca80ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4cad600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca8161d60 con 0x7f4ca80b5560 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4cad600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca80b40c0 con 0x7f4ca8066c00 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4cad600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80b5b00 con 0x7f4ca8066fe0 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ca80b5560 0x7f4ca80ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ca80b5560 0x7f4ca80ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39590/0 (socket says 172.21.15.45:39590) 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 -- 172.21.15.45:0/4098129634 learned_addr learned my addr 172.21.15.45:0/4098129634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca6a00640 1 --2- 172.21.15.45:0/4098129634 >> v2:172.21.15.117:3300/0 conn(0x7f4ca8066c00 0x7f4ca8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4c9fe00640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066fe0 0x7f4ca8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 -- 172.21.15.45:0/4098129634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066fe0 msgr2=0x7f4ca8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066fe0 0x7f4ca8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 -- 172.21.15.45:0/4098129634 >> v2:172.21.15.117:3300/0 conn(0x7f4ca8066c00 msgr2=0x7f4ca8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 --2- 172.21.15.45:0/4098129634 >> v2:172.21.15.117:3300/0 conn(0x7f4ca8066c00 0x7f4ca8161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca7400640 1 -- 172.21.15.45:0/4098129634 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80b3a00 con 0x7f4ca80b5560 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4ca6a00640 1 --2- 172.21.15.45:0/4098129634 >> v2:172.21.15.117:3300/0 conn(0x7f4ca8066c00 0x7f4ca8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.959+0000 7f4c9fe00640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066fe0 0x7f4ca8072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:35.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4ca7400640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca80b5560 0x7f4ca80ae850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80d1930 tx=0x7f4ca001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bcc5668395de98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4ca6000640 1 -- 172.21.15.45:0/4098129634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00a5550 con 0x7f4ca80b5560 2024-04-25T10:20:35.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4ca6000640 1 -- 172.21.15.45:0/4098129634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca00a56f0 con 0x7f4ca80b5560 2024-04-25T10:20:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4ca6000640 1 -- 172.21.15.45:0/4098129634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00a5ac0 con 0x7f4ca80b5560 2024-04-25T10:20:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 -- 172.21.15.45:0/4098129634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca80b5560 msgr2=0x7f4ca80ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca80b5560 0x7f4ca80ae850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80d1930 tx=0x7f4ca001c8d0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 -- 172.21.15.45:0/4098129634 shutdown_connections 2024-04-25T10:20:35.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca80b5560 0x7f4ca80ae850 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 --2- 172.21.15.45:0/4098129634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066fe0 0x7f4ca8072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 --2- 172.21.15.45:0/4098129634 >> v2:172.21.15.117:3300/0 conn(0x7f4ca8066c00 0x7f4ca8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 -- 172.21.15.45:0/4098129634 >> 172.21.15.45:0/4098129634 conn(0x7f4ca80b2560 msgr2=0x7f4ca80b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 -- 172.21.15.45:0/4098129634 shutdown_connections 2024-04-25T10:20:35.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 -- 172.21.15.45:0/4098129634 wait complete. 2024-04-25T10:20:35.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 Processor -- start 2024-04-25T10:20:35.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.963+0000 7f4cad600640 1 -- start start 2024-04-25T10:20:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066c00 0x7f4ca813d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 0x7f4ca8152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca80b5560 0x7f4ca8150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81584d0 con 0x7f4ca8066fe0 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8158350 con 0x7f4ca80b5560 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8158650 con 0x7f4ca8066c00 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 0x7f4ca8152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 0x7f4ca8152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39600/0 (socket says 172.21.15.45:39600) 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 -- 172.21.15.45:0/971490608 learned_addr learned my addr 172.21.15.45:0/971490608 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4ca6a00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066c00 0x7f4ca813d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4ca7400640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca80b5560 0x7f4ca8150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 -- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066c00 msgr2=0x7f4ca813d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066c00 0x7f4ca813d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 -- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca80b5560 msgr2=0x7f4ca8150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca80b5560 0x7f4ca8150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 -- 172.21.15.45:0/971490608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8151af0 con 0x7f4ca8066fe0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9fe00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 0x7f4ca8152b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c820 tx=0x7f4c9000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90015070 con 0x7f4ca8066fe0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90007a30 con 0x7f4ca8066fe0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca8151dd0 con 0x7f4ca8066fe0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90007e40 con 0x7f4ca8066fe0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca8152390 con 0x7f4ca8066fe0 2024-04-25T10:20:35.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca804ef30 con 0x7f4ca8066fe0 2024-04-25T10:20:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.967+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c90018070 con 0x7f4ca8066fe0 2024-04-25T10:20:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.971+0000 7f4c9ea00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c84066f60 0x7f4c84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.971+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f4c90084ff0 con 0x7f4ca8066fe0 2024-04-25T10:20:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.971+0000 7f4ca6a00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c84066f60 0x7f4c84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.971+0000 7f4ca6a00640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c84066f60 0x7f4c84069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800aaf0 tx=0x7f4c98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:35.975+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c90055860 con 0x7f4ca8066fe0 2024-04-25T10:20:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.171+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ca81699a0 con 0x7f4ca8066fe0 2024-04-25T10:20:36.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.787+0000 7f4c9ea00640 1 -- 172.21.15.45:0/971490608 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4c90059900 con 0x7f4ca8066fe0 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c84066f60 msgr2=0x7f4c84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c84066f60 0x7f4c84069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800aaf0 tx=0x7f4c98008040 comp rx=0 tx=0).stop 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 msgr2=0x7f4ca8152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 0x7f4ca8152b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c820 tx=0x7f4c9000cce0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 shutdown_connections 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c84066f60 0x7f4c84069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca80b5560 0x7f4ca8150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ca8066fe0 0x7f4ca8152b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 --2- 172.21.15.45:0/971490608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ca8066c00 0x7f4ca813d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 >> 172.21.15.45:0/971490608 conn(0x7f4ca80b2560 msgr2=0x7f4ca806d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 shutdown_connections 2024-04-25T10:20:36.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.791+0000 7f4cad600640 1 -- 172.21.15.45:0/971490608 wait complete. 2024-04-25T10:20:36.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:20:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 Processor -- start 2024-04-25T10:20:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 -- start start 2024-04-25T10:20:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b5015f0c0 0x7f1b500d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b500b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b500b0cf0 con 0x7f1b500b05d0 2024-04-25T10:20:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b500b11a0 con 0x7f1b5015f0c0 2024-04-25T10:20:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b56800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500ccfe0 con 0x7f1b500ce8b0 2024-04-25T10:20:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b5015f0c0 0x7f1b500d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b54a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1b500b05d0 0x7f1b500ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:60482/0 (socket says 172.21.15.45:60482) 2024-04-25T10:20:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b500b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b54a00640 1 -- 172.21.15.45:0/3319073591 learned_addr learned my addr 172.21.15.45:0/3319073591 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55400640 1 -- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 msgr2=0x7f1b500b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55400640 1 --2- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b500b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55400640 1 -- 172.21.15.45:0/3319073591 >> v1:172.21.15.117:6789/0 conn(0x7f1b500b05d0 legacy=0x7f1b500ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55400640 1 -- 172.21.15.45:0/3319073591 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b500cd2c0 con 0x7f1b5015f0c0 2024-04-25T10:20:36.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b4fe00640 1 -- 172.21.15.45:0/3319073591 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416143443 0 0) 0x7f1b3c001120 con 0x7f1b500b05d0 2024-04-25T10:20:36.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.959+0000 7f1b55e00640 1 --2- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b500b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b55400640 1 --2- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5015f0c0 0x7f1b500d22f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b440097a0 tx=0x7f1b4401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b365f37507ca31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b4fe00640 1 -- 172.21.15.45:0/3319073591 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440ab650 con 0x7f1b5015f0c0 2024-04-25T10:20:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b4fe00640 1 -- 172.21.15.45:0/3319073591 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440b0070 con 0x7f1b5015f0c0 2024-04-25T10:20:36.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b4fe00640 1 -- 172.21.15.45:0/3319073591 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440aba10 con 0x7f1b5015f0c0 2024-04-25T10:20:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 -- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5015f0c0 msgr2=0x7f1b500d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 --2- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5015f0c0 0x7f1b500d22f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b440097a0 tx=0x7f1b4401c450 comp rx=0 tx=0).stop 2024-04-25T10:20:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 -- 172.21.15.45:0/3319073591 shutdown_connections 2024-04-25T10:20:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 --2- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b500b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 --2- 172.21.15.45:0/3319073591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5015f0c0 0x7f1b500d22f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 -- 172.21.15.45:0/3319073591 >> 172.21.15.45:0/3319073591 conn(0x7f1b5009a9d0 msgr2=0x7f1b500d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 -- 172.21.15.45:0/3319073591 shutdown_connections 2024-04-25T10:20:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.963+0000 7f1b56800640 1 -- 172.21.15.45:0/3319073591 wait complete. 2024-04-25T10:20:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 Processor -- start 2024-04-25T10:20:36.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- start start 2024-04-25T10:20:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b500b05d0 0x7f1b500cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b501643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 0x7f1b501551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500b2500 con 0x7f1b5015f0c0 2024-04-25T10:20:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500b2380 con 0x7f1b500b05d0 2024-04-25T10:20:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b500b2680 con 0x7f1b500ce8b0 2024-04-25T10:20:36.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 0x7f1b501551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 0x7f1b501551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39610/0 (socket says 172.21.15.45:39610) 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 -- 172.21.15.45:0/2205241902 learned_addr learned my addr 172.21.15.45:0/2205241902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55400640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b500b05d0 0x7f1b500cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b54a00640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b501643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 -- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 msgr2=0x7f1b501643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b501643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 -- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b500b05d0 msgr2=0x7f1b500cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b500b05d0 0x7f1b500cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 -- 172.21.15.45:0/2205241902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50164dc0 con 0x7f1b5015f0c0 2024-04-25T10:20:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b55e00640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 0x7f1b501551f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800efb0 tx=0x7f1b4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48018020 con 0x7f1b5015f0c0 2024-04-25T10:20:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b48014480 con 0x7f1b5015f0c0 2024-04-25T10:20:36.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5015ccf0 con 0x7f1b5015f0c0 2024-04-25T10:20:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48014850 con 0x7f1b5015f0c0 2024-04-25T10:20:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5015d2b0 con 0x7f1b5015f0c0 2024-04-25T10:20:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.967+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5015cb00 con 0x7f1b5015f0c0 2024-04-25T10:20:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.971+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b4801b020 con 0x7f1b5015f0c0 2024-04-25T10:20:36.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.971+0000 7f1b4d600640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b30066f60 0x7f1b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.971+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f1b48085e60 con 0x7f1b5015f0c0 2024-04-25T10:20:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.971+0000 7f1b55400640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b30066f60 0x7f1b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.971+0000 7f1b55400640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b30066f60 0x7f1b30069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b440a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:36.975+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b48022030 con 0x7f1b5015f0c0 2024-04-25T10:20:37.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.139+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b500d1950 con 0x7f1b5015f0c0 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.795+0000 7f1b4d600640 1 -- 172.21.15.45:0/2205241902 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1b48010040 con 0x7f1b5015f0c0 2024-04-25T10:20:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.795+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b30066f60 msgr2=0x7f1b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.795+0000 7f1b56800640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b30066f60 0x7f1b30069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b440a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.795+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 msgr2=0x7f1b501551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.795+0000 7f1b56800640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 0x7f1b501551f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800efb0 tx=0x7f1b4800c490 comp rx=0 tx=0).stop 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 shutdown_connections 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b30066f60 0x7f1b30069420 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b440af000 tx=0x7f1b440a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5015f0c0 0x7f1b501551f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b500ce8b0 0x7f1b501643e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 --2- 172.21.15.45:0/2205241902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b500b05d0 0x7f1b500cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 >> 172.21.15.45:0/2205241902 conn(0x7f1b5009a9d0 msgr2=0x7f1b5006fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 shutdown_connections 2024-04-25T10:20:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.799+0000 7f1b56800640 1 -- 172.21.15.45:0/2205241902 wait complete. 2024-04-25T10:20:37.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:20:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 Processor -- start 2024-04-25T10:20:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 -- start start 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674162310 0x7f467415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f46741618a0 0x7f46740b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467415e9b0 con 0x7f46740b4d80 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467415f250 con 0x7f46741618a0 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467c800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46740afce0 con 0x7f4674162310 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467a800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f46740b4d80 0x7f4674072250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37010/0 (socket says 172.21.15.45:37010) 2024-04-25T10:20:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467a800640 1 -- 172.21.15.45:0/3766533136 learned_addr learned my addr 172.21.15.45:0/3766533136 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4679e00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674162310 0x7f467415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4679400640 1 --2- 172.21.15.45:0/3766533136 >> v2:172.21.15.117:3300/0 conn(0x7f46741618a0 0x7f46740b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269878171 0 0) 0x7f467415e9b0 con 0x7f46740b4d80 2024-04-25T10:20:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f465c003530 con 0x7f46740b4d80 2024-04-25T10:20:37.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315053248 0 0) 0x7f465c003530 con 0x7f46740b4d80 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674162310 msgr2=0x7f467415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674162310 0x7f467415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 >> v2:172.21.15.117:3300/0 conn(0x7f46741618a0 msgr2=0x7f46740b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 --2- 172.21.15.45:0/3766533136 >> v2:172.21.15.117:3300/0 conn(0x7f46741618a0 0x7f46740b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467415ec90 con 0x7f46740b4d80 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f4670001a20 con 0x7f46740b4d80 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 >> v1:172.21.15.45:6789/0 conn(0x7f46740b4d80 legacy=0x7f4674072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0039c0 0x7f465c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c004810 0x7f465c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0051d0 0x7f465c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467415f250 con 0x7f465c0051d0 2024-04-25T10:20:37.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46740afce0 con 0x7f465c004810 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c00c040 con 0x7f465c0039c0 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4679e00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674162310 0x7f467415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f4679400640 1 --2- 172.21.15.45:0/3766533136 >> v2:172.21.15.117:3300/0 conn(0x7f46741618a0 0x7f46740b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.975+0000 7f467a800640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0051d0 0x7f465c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f4679e00640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0039c0 0x7f465c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f4679400640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c004810 0x7f465c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f467a800640 1 -- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0039c0 msgr2=0x7f465c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f467a800640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0039c0 0x7f465c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f467a800640 1 -- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c004810 msgr2=0x7f465c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f467a800640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c004810 0x7f465c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f467a800640 1 -- 172.21.15.45:0/3766533136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c003530 con 0x7f465c0051d0 2024-04-25T10:20:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f4679400640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c004810 0x7f465c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f467a800640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0051d0 0x7f465c00b9d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f467000a7a0 tx=0x7f467001d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18ef9f11d3de9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467000a3f0 con 0x7f465c0051d0 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46700ad630 con 0x7f465c0051d0 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.979+0000 7f4678a00640 1 -- 172.21.15.45:0/3766533136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46700ada20 con 0x7f465c0051d0 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- 172.21.15.45:0/3766533136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0051d0 msgr2=0x7f465c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 --2- 172.21.15.45:0/3766533136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0051d0 0x7f465c00b9d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f467000a7a0 tx=0x7f467001d270 comp rx=0 tx=0).stop 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467a800640 1 -- 172.21.15.45:0/3766533136 reap_dead start 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- 172.21.15.45:0/3766533136 shutdown_connections 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- 172.21.15.45:0/3766533136 >> 172.21.15.45:0/3766533136 conn(0x7f46740b5610 msgr2=0x7f46740b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- 172.21.15.45:0/3766533136 shutdown_connections 2024-04-25T10:20:37.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- 172.21.15.45:0/3766533136 wait complete. 2024-04-25T10:20:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 Processor -- start 2024-04-25T10:20:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- start start 2024-04-25T10:20:37.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 0x7f46741605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4674160b10 0x7f4674166790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674166cd0 0x7f467406cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46741626f0 con 0x7f4674099980 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46740b51b0 con 0x7f4674160b10 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467c800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4674168a20 con 0x7f4674166cd0 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 0x7f46741605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 0x7f46741605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39618/0 (socket says 172.21.15.45:39618) 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 -- 172.21.15.45:0/3689989029 learned_addr learned my addr 172.21.15.45:0/3689989029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679400640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4674160b10 0x7f4674166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f467a800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674166cd0 0x7f467406cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 -- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674166cd0 msgr2=0x7f467406cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674166cd0 0x7f467406cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 -- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4674160b10 msgr2=0x7f4674166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4674160b10 0x7f4674166790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 -- 172.21.15.45:0/3689989029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467406e920 con 0x7f4674099980 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.983+0000 7f4679e00640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 0x7f46741605d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f46680119d0 tx=0x7f4668011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800fb50 con 0x7f4674099980 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466800fcf0 con 0x7f4674099980 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800fea0 con 0x7f4674099980 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4674167470 con 0x7f4674099980 2024-04-25T10:20:37.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4674156890 con 0x7f4674099980 2024-04-25T10:20:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467404ef30 con 0x7f4674099980 2024-04-25T10:20:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4668012040 con 0x7f4674099980 2024-04-25T10:20:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f466e000640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4658066f90 0x7f4658069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f4668085620 con 0x7f4674099980 2024-04-25T10:20:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.987+0000 7f4679400640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4658066f90 0x7f4658069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.991+0000 7f4679400640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4658066f90 0x7f4658069450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4674161190 tx=0x7f4664008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:37.991+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4668019030 con 0x7f4674099980 2024-04-25T10:20:38.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.163+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f467415ce50 con 0x7f4674099980 2024-04-25T10:20:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.795+0000 7f466e000640 1 -- 172.21.15.45:0/3689989029 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4668056020 con 0x7f4674099980 2024-04-25T10:20:38.799 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:20:38.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4658066f90 msgr2=0x7f4658069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4658066f90 0x7f4658069450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4674161190 tx=0x7f4664008040 comp rx=0 tx=0).stop 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 msgr2=0x7f46741605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 0x7f46741605d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f46680119d0 tx=0x7f4668011e90 comp rx=0 tx=0).stop 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 shutdown_connections 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4658066f90 0x7f4658069450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4674166cd0 0x7f467406cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4674160b10 0x7f4674166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 --2- 172.21.15.45:0/3689989029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4674099980 0x7f46741605d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 >> 172.21.15.45:0/3689989029 conn(0x7f46740b5610 msgr2=0x7f467415b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 shutdown_connections 2024-04-25T10:20:38.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.799+0000 7f467c800640 1 -- 172.21.15.45:0/3689989029 wait complete. 2024-04-25T10:20:38.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:20:38.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0213400640 1 Processor -- start 2024-04-25T10:20:38.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0213400640 1 -- start start 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0213400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f020c0721f0 0x7f020c166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0213400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c071360 con 0x7f020c0aeda0 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0213400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c159920 con 0x7f020c159540 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0213400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c15c2f0 con 0x7f020c0721f0 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0211400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f020c0721f0 0x7f020c166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0211400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f020c0721f0 0x7f020c166920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39622/0 (socket says 172.21.15.45:39622) 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0211400640 1 -- 172.21.15.45:0/3111916678 learned_addr learned my addr 172.21.15.45:0/3111916678 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:38.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f020be00640 1 --1- 172.21.15.45:0/3111916678 >> v1:172.21.15.45:6789/0 conn(0x7f020c0aeda0 0x7f020c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37014/0 (socket says 172.21.15.45:37014) 2024-04-25T10:20:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0211400640 1 -- 172.21.15.45:0/3111916678 >> v1:172.21.15.117:6789/0 conn(0x7f020c159540 legacy=0x7f020c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0211400640 1 -- 172.21.15.45:0/3111916678 >> v1:172.21.15.45:6789/0 conn(0x7f020c0aeda0 legacy=0x7f020c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.975+0000 7f0211400640 1 -- 172.21.15.45:0/3111916678 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c15c5d0 con 0x7f020c0721f0 2024-04-25T10:20:38.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0211400640 1 --2- 172.21.15.45:0/3111916678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c0721f0 0x7f020c166920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e50 tx=0x7f020401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e441be7fa2eb88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f020b400640 1 -- 172.21.15.45:0/3111916678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040a58d0 con 0x7f020c0721f0 2024-04-25T10:20:38.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f020b400640 1 -- 172.21.15.45:0/3111916678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02040a5a70 con 0x7f020c0721f0 2024-04-25T10:20:38.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f020b400640 1 -- 172.21.15.45:0/3111916678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040a5e40 con 0x7f020c0721f0 2024-04-25T10:20:38.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 -- 172.21.15.45:0/3111916678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c0721f0 msgr2=0x7f020c166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 --2- 172.21.15.45:0/3111916678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c0721f0 0x7f020c166920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e50 tx=0x7f020401ca00 comp rx=0 tx=0).stop 2024-04-25T10:20:38.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 -- 172.21.15.45:0/3111916678 shutdown_connections 2024-04-25T10:20:38.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 --2- 172.21.15.45:0/3111916678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c0721f0 0x7f020c166920 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 -- 172.21.15.45:0/3111916678 >> 172.21.15.45:0/3111916678 conn(0x7f020c0669a0 msgr2=0x7f020c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 -- 172.21.15.45:0/3111916678 shutdown_connections 2024-04-25T10:20:38.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.979+0000 7f0213400640 1 -- 172.21.15.45:0/3111916678 wait complete. 2024-04-25T10:20:38.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 Processor -- start 2024-04-25T10:20:38.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 -- start start 2024-04-25T10:20:38.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f020c0721f0 0x7f020c149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f020c0aeda0 0x7f020c151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 0x7f020c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c169680 con 0x7f020c159540 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c06c0d0 con 0x7f020c0721f0 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c169800 con 0x7f020c0aeda0 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 0x7f020c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 0x7f020c0b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39626/0 (socket says 172.21.15.45:39626) 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 -- 172.21.15.45:0/3367757402 learned_addr learned my addr 172.21.15.45:0/3367757402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0210a00640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f020c0721f0 0x7f020c149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f020be00640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f020c0aeda0 0x7f020c151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 -- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f020c0aeda0 msgr2=0x7f020c151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f020c0aeda0 0x7f020c151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 -- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f020c0721f0 msgr2=0x7f020c149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f020c0721f0 0x7f020c149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 -- 172.21.15.45:0/3367757402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0b1a10 con 0x7f020c159540 2024-04-25T10:20:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0211400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 0x7f020c0b14d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e50 tx=0x7f02040197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040a58d0 con 0x7f020c159540 2024-04-25T10:20:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02040be070 con 0x7f020c159540 2024-04-25T10:20:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c152270 con 0x7f020c159540 2024-04-25T10:20:38.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204009510 con 0x7f020c159540 2024-04-25T10:20:38.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.983+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c152830 con 0x7f020c159540 2024-04-25T10:20:38.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.987+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c04ef30 con 0x7f020c159540 2024-04-25T10:20:38.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.987+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02040b86b0 con 0x7f020c159540 2024-04-25T10:20:38.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.987+0000 7f0208c00640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.987+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f02041205a0 con 0x7f020c159540 2024-04-25T10:20:38.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.987+0000 7f0210a00640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.987+0000 7f0210a00640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02000015b0 tx=0x7f0200009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:38.991+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0204016620 con 0x7f020c159540 2024-04-25T10:20:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.155+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f020c071ae0 con 0x7f020c159540 2024-04-25T10:20:39.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0208c00640 1 -- 172.21.15.45:0/3367757402 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02040f1810 con 0x7f020c159540 2024-04-25T10:20:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ec066f60 msgr2=0x7f01ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02000015b0 tx=0x7f0200009040 comp rx=0 tx=0).stop 2024-04-25T10:20:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 msgr2=0x7f020c0b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 0x7f020c0b14d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e50 tx=0x7f02040197a0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 shutdown_connections 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f020c159540 0x7f020c0b14d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f020c0aeda0 0x7f020c151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 --2- 172.21.15.45:0/3367757402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f020c0721f0 0x7f020c149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 >> 172.21.15.45:0/3367757402 conn(0x7f020c0669a0 msgr2=0x7f020c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 shutdown_connections 2024-04-25T10:20:39.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.159+0000 7f0213400640 1 -- 172.21.15.45:0/3367757402 wait complete. 2024-04-25T10:20:39.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:20:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5989600640 1 Processor -- start 2024-04-25T10:20:39.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5989600640 1 -- start start 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5989600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5984158170 0x7f598415e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5989600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59840b1e30 con 0x7f598406e380 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5989600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59840b2110 con 0x7f59840b2ed0 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5989600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5984158550 con 0x7f5984158170 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5983e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5984158170 0x7f598415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5983e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5984158170 0x7f598415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39628/0 (socket says 172.21.15.45:39628) 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5983e00640 1 -- 172.21.15.45:0/1613054374 learned_addr learned my addr 172.21.15.45:0/1613054374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5988c00640 1 --1- 172.21.15.45:0/1613054374 >> v1:172.21.15.45:6789/0 conn(0x7f598406e380 0x7f598415f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37020/0 (socket says 172.21.15.45:37020) 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5983e00640 1 -- 172.21.15.45:0/1613054374 >> v1:172.21.15.117:6789/0 conn(0x7f59840b2ed0 legacy=0x7f598406dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5983e00640 1 -- 172.21.15.45:0/1613054374 >> v1:172.21.15.45:6789/0 conn(0x7f598406e380 legacy=0x7f598415f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.323+0000 7f5983e00640 1 -- 172.21.15.45:0/1613054374 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59840b23f0 con 0x7f5984158170 2024-04-25T10:20:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5983e00640 1 --2- 172.21.15.45:0/1613054374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f598415e050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5974009870 tx=0x7f597401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773bc3780c4eab3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5983400640 1 -- 172.21.15.45:0/1613054374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740ab640 con 0x7f5984158170 2024-04-25T10:20:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5983400640 1 -- 172.21.15.45:0/1613054374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740b0070 con 0x7f5984158170 2024-04-25T10:20:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5983400640 1 -- 172.21.15.45:0/1613054374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740aba00 con 0x7f5984158170 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 -- 172.21.15.45:0/1613054374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 msgr2=0x7f598415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 --2- 172.21.15.45:0/1613054374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f598415e050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5974009870 tx=0x7f597401c560 comp rx=0 tx=0).stop 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 -- 172.21.15.45:0/1613054374 shutdown_connections 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 --2- 172.21.15.45:0/1613054374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f598415e050 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 -- 172.21.15.45:0/1613054374 >> 172.21.15.45:0/1613054374 conn(0x7f5984073d10 msgr2=0x7f59840740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 -- 172.21.15.45:0/1613054374 shutdown_connections 2024-04-25T10:20:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 -- 172.21.15.45:0/1613054374 wait complete. 2024-04-25T10:20:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.327+0000 7f5989600640 1 Processor -- start 2024-04-25T10:20:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- start start 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f598406e380 0x7f5984149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59840b2ed0 0x7f59840ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f5984151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598416b820 con 0x7f5984158170 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598416b6a0 con 0x7f598406e380 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598416b9a0 con 0x7f59840b2ed0 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f5984151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f5984151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39636/0 (socket says 172.21.15.45:39636) 2024-04-25T10:20:39.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 -- 172.21.15.45:0/3803529396 learned_addr learned my addr 172.21.15.45:0/3803529396 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5983e00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f598406e380 0x7f5984149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f597be00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59840b2ed0 0x7f59840ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 -- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59840b2ed0 msgr2=0x7f59840ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59840b2ed0 0x7f59840ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 -- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f598406e380 msgr2=0x7f5984149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f598406e380 0x7f5984149410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 -- 172.21.15.45:0/3803529396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59840cc400 con 0x7f5984158170 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5988c00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f5984151f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c760 tx=0x7f597c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c017070 con 0x7f5984158170 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c00a640 con 0x7f5984158170 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c00a9f0 con 0x7f5984158170 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59840cc6e0 con 0x7f5984158170 2024-04-25T10:20:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59840c9f20 con 0x7f5984158170 2024-04-25T10:20:39.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.331+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598406bca0 con 0x7f5984158170 2024-04-25T10:20:39.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.335+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f597c01a070 con 0x7f5984158170 2024-04-25T10:20:39.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.335+0000 7f5980c00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5960066f60 0x7f5960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.335+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f597c0841a0 con 0x7f5984158170 2024-04-25T10:20:39.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.335+0000 7f5983e00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5960066f60 0x7f5960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.335+0000 7f5983e00640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5960066f60 0x7f5960069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f59740af000 tx=0x7f59740a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.339+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f597c054e50 con 0x7f5984158170 2024-04-25T10:20:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.559+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f59840cc8b0 con 0x7f5984158170 2024-04-25T10:20:39.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5980c00640 1 -- 172.21.15.45:0/3803529396 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v140) v1 ==== 148+0+0 (secure 0 0 0) 0x7f597c058ef0 con 0x7f5984158170 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5960066f60 msgr2=0x7f5960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5960066f60 0x7f5960069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f59740af000 tx=0x7f59740a7000 comp rx=0 tx=0).stop 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 msgr2=0x7f5984151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f5984151f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f597c00c760 tx=0x7f597c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 shutdown_connections 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5960066f60 0x7f5960069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5984158170 0x7f5984151f20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59840b2ed0 0x7f59840ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 --2- 172.21.15.45:0/3803529396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f598406e380 0x7f5984149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 >> 172.21.15.45:0/3803529396 conn(0x7f5984073d10 msgr2=0x7f59841502e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 shutdown_connections 2024-04-25T10:20:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.811+0000 7f5989600640 1 -- 172.21.15.45:0/3803529396 wait complete. 2024-04-25T10:20:39.836 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T10:20:39.836 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:20:39.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:39.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a73200640 1 Processor -- start 2024-04-25T10:20:39.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a73200640 1 -- start start 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a73200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a6c1550b0 0x7f5a6c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a73200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c071050 con 0x7f5a6c15b910 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a73200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c15d040 con 0x7f5a6c0cfe70 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a73200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c15bd80 con 0x7f5a6c1550b0 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a6be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a6c1550b0 0x7f5a6c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a6be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5a6c1550b0 0x7f5a6c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39650/0 (socket says 172.21.15.45:39650) 2024-04-25T10:20:39.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a6be00640 1 -- 172.21.15.45:0/3595675779 learned_addr learned my addr 172.21.15.45:0/3595675779 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:39.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.975+0000 7f5a71200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5a6c15b910 0x7f5a6c0b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37034/0 (socket says 172.21.15.45:37034) 2024-04-25T10:20:39.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6be00640 1 -- 172.21.15.45:0/3595675779 >> v1:172.21.15.117:6789/0 conn(0x7f5a6c0cfe70 legacy=0x7f5a6c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6be00640 1 -- 172.21.15.45:0/3595675779 >> v1:172.21.15.45:6789/0 conn(0x7f5a6c15b910 legacy=0x7f5a6c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6be00640 1 -- 172.21.15.45:0/3595675779 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0712f0 con 0x7f5a6c1550b0 2024-04-25T10:20:39.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6b400640 1 -- 172.21.15.45:0/3595675779 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862563825 0 0) 0x7f5a6c071050 con 0x7f5a6c15b910 2024-04-25T10:20:39.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6be00640 1 --2- 172.21.15.45:0/3595675779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c0cf930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098a0 tx=0x7f5a5c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1832802ad663f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6b400640 1 -- 172.21.15.45:0/3595675779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0b0070 con 0x7f5a6c1550b0 2024-04-25T10:20:39.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6b400640 1 -- 172.21.15.45:0/3595675779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0ab600 con 0x7f5a6c1550b0 2024-04-25T10:20:39.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a6b400640 1 -- 172.21.15.45:0/3595675779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0ab9d0 con 0x7f5a6c1550b0 2024-04-25T10:20:39.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a73200640 1 -- 172.21.15.45:0/3595675779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 msgr2=0x7f5a6c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a73200640 1 --2- 172.21.15.45:0/3595675779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c0cf930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098a0 tx=0x7f5a5c01c540 comp rx=0 tx=0).stop 2024-04-25T10:20:39.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a73200640 1 -- 172.21.15.45:0/3595675779 shutdown_connections 2024-04-25T10:20:39.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a73200640 1 --2- 172.21.15.45:0/3595675779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c0cf930 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.979+0000 7f5a73200640 1 -- 172.21.15.45:0/3595675779 >> 172.21.15.45:0/3595675779 conn(0x7f5a6c066b70 msgr2=0x7f5a6c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- 172.21.15.45:0/3595675779 shutdown_connections 2024-04-25T10:20:39.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- 172.21.15.45:0/3595675779 wait complete. 2024-04-25T10:20:39.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 Processor -- start 2024-04-25T10:20:39.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- start start 2024-04-25T10:20:39.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a6c0cfe70 0x7f5a6c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a6c15b910 0x7f5a6c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c1684c0 con 0x7f5a6c1550b0 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0720b0 con 0x7f5a6c0cfe70 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c168640 con 0x7f5a6c15b910 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c14ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39652/0 (socket says 172.21.15.45:39652) 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 -- 172.21.15.45:0/929663331 learned_addr learned my addr 172.21.15.45:0/929663331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a6be00640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a6c0cfe70 0x7f5a6c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a71200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a6c15b910 0x7f5a6c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 -- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a6c15b910 msgr2=0x7f5a6c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a6c15b910 0x7f5a6c156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 -- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a6c0cfe70 msgr2=0x7f5a6c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a6c0cfe70 0x7f5a6c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 -- 172.21.15.45:0/929663331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c152640 con 0x7f5a6c1550b0 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a63400640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c14ba40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c3c0 tx=0x7f5a5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54017070 con 0x7f5a6c1550b0 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a540052d0 con 0x7f5a6c1550b0 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c152920 con 0x7f5a6c1550b0 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.983+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54003840 con 0x7f5a6c1550b0 2024-04-25T10:20:39.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c1531c0 con 0x7f5a6c1550b0 2024-04-25T10:20:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c1533c0 con 0x7f5a6c1550b0 2024-04-25T10:20:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a5401a070 con 0x7f5a6c1550b0 2024-04-25T10:20:39.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a68c00640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a48066f60 0x7f5a48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a54084070 con 0x7f5a6c1550b0 2024-04-25T10:20:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a6be00640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a48066f60 0x7f5a48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.987+0000 7f5a6be00640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a48066f60 0x7f5a48069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0ad000 tx=0x7f5a5c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:39.991+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a6c1533c0 con 0x7f5a6c1550b0 2024-04-25T10:20:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a6c04ef30 con 0x7f5a6c1550b0 2024-04-25T10:20:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a68c00640 1 -- 172.21.15.45:0/929663331 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+13008 (secure 0 0 0) 0x7f5a54005480 con 0x7f5a6c1550b0 2024-04-25T10:20:40.159 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:40.159 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":42,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:38.797342+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/3482819589":"2024-04-26T10:20:35.787145+0000","172.21.15.45:6833/3482819589":"2024-04-26T10:20:35.787145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:40.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a48066f60 msgr2=0x7f5a48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a48066f60 0x7f5a48069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0ad000 tx=0x7f5a5c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 msgr2=0x7f5a6c14ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c14ba40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c3c0 tx=0x7f5a5400c880 comp rx=0 tx=0).stop 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 shutdown_connections 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a48066f60 0x7f5a48069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a6c15b910 0x7f5a6c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a6c1550b0 0x7f5a6c14ba40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 --2- 172.21.15.45:0/929663331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a6c0cfe70 0x7f5a6c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.155+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 >> 172.21.15.45:0/929663331 conn(0x7f5a6c066b70 msgr2=0x7f5a6c15ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.159+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 shutdown_connections 2024-04-25T10:20:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.159+0000 7f5a73200640 1 -- 172.21.15.45:0/929663331 wait complete. 2024-04-25T10:20:40.181 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:40.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a4200640 1 Processor -- start 2024-04-25T10:20:40.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a4200640 1 -- start start 2024-04-25T10:20:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a4200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f379c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a4200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c15b080 con 0x7f379c0aef80 2024-04-25T10:20:40.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a4200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c15adb0 con 0x7f379c0b31c0 2024-04-25T10:20:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a4200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c15eb60 con 0x7f379c070350 2024-04-25T10:20:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a1800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f379c0aef80 0x7f379c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37042/0 (socket says 172.21.15.45:37042) 2024-04-25T10:20:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a1800640 1 -- 172.21.15.45:0/3093969273 learned_addr learned my addr 172.21.15.45:0/3093969273 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a0e00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f379c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356719447 0 0) 0x7f379c15b080 con 0x7f379c0aef80 2024-04-25T10:20:40.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3784003530 con 0x7f379c0aef80 2024-04-25T10:20:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185662064 0 0) 0x7f3784003530 con 0x7f379c0aef80 2024-04-25T10:20:40.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 msgr2=0x7f379c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f379c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 >> v1:172.21.15.117:6789/0 conn(0x7f379c0b31c0 legacy=0x7f379c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c15ee40 con 0x7f379c0aef80 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f3790001680 con 0x7f379c0aef80 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 >> v1:172.21.15.45:6789/0 conn(0x7f379c0aef80 legacy=0x7f379c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37840039f0 0x7f3784003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f3784004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3784004d50 0x7f37840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c15eb60 con 0x7f3784004d50 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378400b9d0 con 0x7f37840039f0 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378400bbd0 con 0x7f379c070350 2024-04-25T10:20:40.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a2200640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3784004d50 0x7f37840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a1800640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37840039f0 0x7f3784003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.323+0000 7f37a0e00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f3784004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a2200640 1 -- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 msgr2=0x7f3784004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a2200640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f3784004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a2200640 1 -- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37840039f0 msgr2=0x7f3784003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a2200640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37840039f0 0x7f3784003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a2200640 1 -- 172.21.15.45:0/3093969273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784003530 con 0x7f3784004d50 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a1800640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37840039f0 0x7f3784003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a0e00640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c070350 0x7f3784004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a2200640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3784004d50 0x7f37840051d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3794002010 tx=0x7f379401eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4477ac8587e17beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37940b4070 con 0x7f3784004d50 2024-04-25T10:20:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37940a7900 con 0x7f3784004d50 2024-04-25T10:20:40.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f379be00640 1 -- 172.21.15.45:0/3093969273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794001c10 con 0x7f3784004d50 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a4200640 1 -- 172.21.15.45:0/3093969273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3784004d50 msgr2=0x7f37840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.327+0000 7f37a4200640 1 --2- 172.21.15.45:0/3093969273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3784004d50 0x7f37840051d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3794002010 tx=0x7f379401eaf0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 -- 172.21.15.45:0/3093969273 reap_dead start 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- 172.21.15.45:0/3093969273 shutdown_connections 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- 172.21.15.45:0/3093969273 >> 172.21.15.45:0/3093969273 conn(0x7f379c0669a0 msgr2=0x7f379c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- 172.21.15.45:0/3093969273 shutdown_connections 2024-04-25T10:20:40.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- 172.21.15.45:0/3093969273 wait complete. 2024-04-25T10:20:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 Processor -- start 2024-04-25T10:20:40.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- start start 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784004d50 0x7f379c148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c158b90 0x7f379c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 0x7f379c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c15bc90 con 0x7f379c154d00 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c071a00 con 0x7f3784004d50 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a4200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0cfd70 con 0x7f379c158b90 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 0x7f379c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 0x7f379c155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39660/0 (socket says 172.21.15.45:39660) 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 -- 172.21.15.45:0/2356778400 learned_addr learned my addr 172.21.15.45:0/2356778400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a1800640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784004d50 0x7f379c148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a0e00640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c158b90 0x7f379c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 -- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c158b90 msgr2=0x7f379c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c158b90 0x7f379c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 -- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784004d50 msgr2=0x7f379c148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784004d50 0x7f379c148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 -- 172.21.15.45:0/2356778400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0b0190 con 0x7f379c154d00 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.331+0000 7f37a2200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 0x7f379c155180 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37940b2040 tx=0x7f3784005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37940b4070 con 0x7f379c154d00 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794003d90 con 0x7f379c154d00 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790007cf0 con 0x7f379c154d00 2024-04-25T10:20:40.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c155a30 con 0x7f379c154d00 2024-04-25T10:20:40.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c159550 con 0x7f379c154d00 2024-04-25T10:20:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c04ef30 con 0x7f379c154d00 2024-04-25T10:20:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3794005070 con 0x7f379c154d00 2024-04-25T10:20:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f3799600640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f377c066f60 0x7f377c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f379411e5e0 con 0x7f379c154d00 2024-04-25T10:20:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.335+0000 7f37a1800640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f377c066f60 0x7f377c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.339+0000 7f37a1800640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f377c066f60 0x7f377c069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f379c0af6e0 tx=0x7f3790009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.339+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37940be060 con 0x7f379c154d00 2024-04-25T10:20:40.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.503+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f379c0b2fa0 con 0x7f379c154d00 2024-04-25T10:20:40.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.503+0000 7f3799600640 1 -- 172.21.15.45:0/2356778400 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3794018800 con 0x7f379c154d00 2024-04-25T10:20:40.509 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:20:40.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f377c066f60 msgr2=0x7f377c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f377c066f60 0x7f377c069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f379c0af6e0 tx=0x7f3790009040 comp rx=0 tx=0).stop 2024-04-25T10:20:40.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 msgr2=0x7f379c155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 0x7f379c155180 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f37940b2040 tx=0x7f3784005840 comp rx=0 tx=0).stop 2024-04-25T10:20:40.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 shutdown_connections 2024-04-25T10:20:40.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f377c066f60 0x7f377c069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f379c154d00 0x7f379c155180 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379c158b90 0x7f379c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 --2- 172.21.15.45:0/2356778400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3784004d50 0x7f379c148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 >> 172.21.15.45:0/2356778400 conn(0x7f379c0669a0 msgr2=0x7f379c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 shutdown_connections 2024-04-25T10:20:40.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.507+0000 7f37a4200640 1 -- 172.21.15.45:0/2356778400 wait complete. 2024-04-25T10:20:40.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:20:40.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 Processor -- start 2024-04-25T10:20:40.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 -- start start 2024-04-25T10:20:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68d00cda10 0x7f68d006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 0x7f68d00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d00b2770 con 0x7f68d015bc40 2024-04-25T10:20:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d006c670 con 0x7f68d00cda10 2024-04-25T10:20:40.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.667+0000 7f68d5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d006c950 con 0x7f68d00cf930 2024-04-25T10:20:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68ce000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68d00cda10 0x7f68d006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 0x7f68d00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 0x7f68d00cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:39102/0 (socket says 172.21.15.45:39102) 2024-04-25T10:20:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 -- 172.21.15.45:0/2658583223 learned_addr learned my addr 172.21.15.45:0/2658583223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:40.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cea00640 1 --1- 172.21.15.45:0/2658583223 >> v1:172.21.15.117:6789/0 conn(0x7f68d015bc40 0x7f68d00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:60538/0 (socket says 172.21.15.45:60538) 2024-04-25T10:20:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 -- 172.21.15.45:0/2658583223 >> v2:172.21.15.117:3300/0 conn(0x7f68d00cda10 msgr2=0x7f68d006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 --2- 172.21.15.45:0/2658583223 >> v2:172.21.15.117:3300/0 conn(0x7f68d00cda10 0x7f68d006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 -- 172.21.15.45:0/2658583223 >> v1:172.21.15.117:6789/0 conn(0x7f68d015bc40 legacy=0x7f68d00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 -- 172.21.15.45:0/2658583223 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d006cc30 con 0x7f68d00cf930 2024-04-25T10:20:40.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cf400640 1 --2- 172.21.15.45:0/2658583223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 0x7f68d00cfde0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009e50 tx=0x7f68c801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e19b266b9ba626e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cd600640 1 -- 172.21.15.45:0/2658583223 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80a5660 con 0x7f68d00cf930 2024-04-25T10:20:40.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cd600640 1 -- 172.21.15.45:0/2658583223 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c80a5800 con 0x7f68d00cf930 2024-04-25T10:20:40.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.671+0000 7f68cd600640 1 -- 172.21.15.45:0/2658583223 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80a5bd0 con 0x7f68d00cf930 2024-04-25T10:20:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- 172.21.15.45:0/2658583223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 msgr2=0x7f68d00cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 --2- 172.21.15.45:0/2658583223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 0x7f68d00cfde0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009e50 tx=0x7f68c801c9a0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- 172.21.15.45:0/2658583223 shutdown_connections 2024-04-25T10:20:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 --2- 172.21.15.45:0/2658583223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d00cf930 0x7f68d00cfde0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 --2- 172.21.15.45:0/2658583223 >> v2:172.21.15.117:3300/0 conn(0x7f68d00cda10 0x7f68d006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- 172.21.15.45:0/2658583223 >> 172.21.15.45:0/2658583223 conn(0x7f68d0066b70 msgr2=0x7f68d0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- 172.21.15.45:0/2658583223 shutdown_connections 2024-04-25T10:20:40.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- 172.21.15.45:0/2658583223 wait complete. 2024-04-25T10:20:40.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 Processor -- start 2024-04-25T10:20:40.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- start start 2024-04-25T10:20:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d00cda10 0x7f68d00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 0x7f68d014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d015bc40 0x7f68d00afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00b3890 con 0x7f68d00cf930 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00b3710 con 0x7f68d00cda10 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68d5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00b3a10 con 0x7f68d015bc40 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 0x7f68d014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 0x7f68d014c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39662/0 (socket says 172.21.15.45:39662) 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 -- 172.21.15.45:0/3627764972 learned_addr learned my addr 172.21.15.45:0/3627764972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68cea00640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d00cda10 0x7f68d00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68cf400640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d015bc40 0x7f68d00afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 -- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d015bc40 msgr2=0x7f68d00afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d015bc40 0x7f68d00afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 -- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d00cda10 msgr2=0x7f68d00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.675+0000 7f68ce000640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d00cda10 0x7f68d00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68ce000640 1 -- 172.21.15.45:0/3627764972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d015e990 con 0x7f68d00cf930 2024-04-25T10:20:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68ce000640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 0x7f68d014c270 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00e870 tx=0x7f68bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc017070 con 0x7f68d00cf930 2024-04-25T10:20:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc00c940 con 0x7f68d00cf930 2024-04-25T10:20:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00ccf0 con 0x7f68d00cf930 2024-04-25T10:20:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d00afae0 con 0x7f68d00cf930 2024-04-25T10:20:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d0154a30 con 0x7f68d00cf930 2024-04-25T10:20:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d004ef30 con 0x7f68d00cf930 2024-04-25T10:20:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68bc01a070 con 0x7f68d00cf930 2024-04-25T10:20:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68c6a00640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f68b0066f90 0x7f68b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f68bc084b20 con 0x7f68d00cf930 2024-04-25T10:20:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.679+0000 7f68cea00640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f68b0066f90 0x7f68b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.683+0000 7f68cea00640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f68b0066f90 0x7f68b0069450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68c0000f70 tx=0x7f68c0007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.683+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68bc0557d0 con 0x7f68d00cf930 2024-04-25T10:20:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:40.919+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f68d006d960 con 0x7f68d00cf930 2024-04-25T10:20:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.823+0000 7f68c6a00640 1 -- 172.21.15.45:0/3627764972 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v142) v1 ==== 146+0+0 (secure 0 0 0) 0x7f68bc059870 con 0x7f68d00cf930 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f68b0066f90 msgr2=0x7f68b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f68b0066f90 0x7f68b0069450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68c0000f70 tx=0x7f68c0007e90 comp rx=0 tx=0).stop 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 msgr2=0x7f68d014c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 0x7f68d014c270 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00e870 tx=0x7f68bc00ec70 comp rx=0 tx=0).stop 2024-04-25T10:20:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 shutdown_connections 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f68b0066f90 0x7f68b0069450 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68d015bc40 0x7f68d00afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68d00cf930 0x7f68d014c270 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 --2- 172.21.15.45:0/3627764972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d00cda10 0x7f68d00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 >> 172.21.15.45:0/3627764972 conn(0x7f68d0066b70 msgr2=0x7f68d0155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 shutdown_connections 2024-04-25T10:20:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.827+0000 7f68d5800640 1 -- 172.21.15.45:0/3627764972 wait complete. 2024-04-25T10:20:41.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:20:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 Processor -- start 2024-04-25T10:20:42.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 -- start start 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8d40ce290 0x7fd8d40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8d40ce670 0x7fd8d415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ceb20 0x7fd8d415f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d40cfde0 con 0x7fd8d40ce290 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d40d0080 con 0x7fd8d40ce670 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40d0360 con 0x7fd8d40ceb20 2024-04-25T10:20:42.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8d40ce290 0x7fd8d40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8d40ce290 0x7fd8d40b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39668/0 (socket says 172.21.15.45:39668) 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 -- 172.21.15.45:0/1623180475 learned_addr learned my addr 172.21.15.45:0/1623180475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d9000640 1 --2- 172.21.15.45:0/1623180475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ceb20 0x7fd8d415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8cbe00640 1 --2- 172.21.15.45:0/1623180475 >> v2:172.21.15.117:3300/0 conn(0x7fd8d40ce670 0x7fd8d415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 -- 172.21.15.45:0/1623180475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ceb20 msgr2=0x7fd8d415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 --2- 172.21.15.45:0/1623180475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ceb20 0x7fd8d415f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 -- 172.21.15.45:0/1623180475 >> v2:172.21.15.117:3300/0 conn(0x7fd8d40ce670 msgr2=0x7fd8d415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 --2- 172.21.15.45:0/1623180475 >> v2:172.21.15.117:3300/0 conn(0x7fd8d40ce670 0x7fd8d415ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:41.999+0000 7fd8d3e00640 1 -- 172.21.15.45:0/1623180475 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d40d0640 con 0x7fd8d40ce290 2024-04-25T10:20:42.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d3e00640 1 --2- 172.21.15.45:0/1623180475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce290 0x7fd8d40b3dd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163cc6b5074067c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d3400640 1 -- 172.21.15.45:0/1623180475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40b0070 con 0x7fd8d40ce290 2024-04-25T10:20:42.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d3400640 1 -- 172.21.15.45:0/1623180475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c40ab600 con 0x7fd8d40ce290 2024-04-25T10:20:42.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d3400640 1 -- 172.21.15.45:0/1623180475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40ab9d0 con 0x7fd8d40ce290 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 -- 172.21.15.45:0/1623180475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce290 msgr2=0x7fd8d40b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/1623180475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce290 0x7fd8d40b3dd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c401c490 comp rx=0 tx=0).stop 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 -- 172.21.15.45:0/1623180475 shutdown_connections 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/1623180475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ceb20 0x7fd8d415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/1623180475 >> v2:172.21.15.117:3300/0 conn(0x7fd8d40ce670 0x7fd8d415ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/1623180475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce290 0x7fd8d40b3dd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 -- 172.21.15.45:0/1623180475 >> 172.21.15.45:0/1623180475 conn(0x7fd8d40b32b0 msgr2=0x7fd8d40b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.003+0000 7fd8d9a00640 1 -- 172.21.15.45:0/1623180475 shutdown_connections 2024-04-25T10:20:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 -- 172.21.15.45:0/1623180475 wait complete. 2024-04-25T10:20:42.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 Processor -- start 2024-04-25T10:20:42.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 -- start start 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ce290 0x7fd8d414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 0x7fd8d40cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8d40ceb20 0x7fd8d40c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40ccea0 con 0x7fd8d40ce670 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40ccd20 con 0x7fd8d40ceb20 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40cd020 con 0x7fd8d40ce290 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 0x7fd8d40cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 0x7fd8d40cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39684/0 (socket says 172.21.15.45:39684) 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 -- 172.21.15.45:0/3939768487 learned_addr learned my addr 172.21.15.45:0/3939768487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:42.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d9000640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8d40ceb20 0x7fd8d40c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 -- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ce290 msgr2=0x7fd8d414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ce290 0x7fd8d414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 -- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8d40ceb20 msgr2=0x7fd8d40c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8d40ceb20 0x7fd8d40c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 -- 172.21.15.45:0/3939768487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d40c5c50 con 0x7fd8d40ce670 2024-04-25T10:20:42.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8cbe00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 0x7fd8d40cac90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00d830 tx=0x7fd8bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.007+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc018030 con 0x7fd8d40ce670 2024-04-25T10:20:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc00be10 con 0x7fd8d40ce670 2024-04-25T10:20:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d40c5f30 con 0x7fd8d40ce670 2024-04-25T10:20:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0098b0 con 0x7fd8d40ce670 2024-04-25T10:20:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d40c64f0 con 0x7fd8d40ce670 2024-04-25T10:20:42.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d4075120 con 0x7fd8d40ce670 2024-04-25T10:20:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8bc01b070 con 0x7fd8d40ce670 2024-04-25T10:20:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d0c00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd8b0066f60 0x7fd8b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8bc085070 con 0x7fd8d40ce670 2024-04-25T10:20:42.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.011+0000 7fd8d3e00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd8b0066f60 0x7fd8b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.015+0000 7fd8d3e00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd8b0066f60 0x7fd8b0069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40ad000 tx=0x7fd8c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.015+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8bc056250 con 0x7fd8d40ce670 2024-04-25T10:20:42.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.179+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd8d415e7a0 con 0x7fd8d40ce670 2024-04-25T10:20:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d0c00640 1 -- 172.21.15.45:0/3939768487 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8bc055b50 con 0x7fd8d40ce670 2024-04-25T10:20:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd8b0066f60 msgr2=0x7fd8b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd8b0066f60 0x7fd8b0069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd8c40ad000 tx=0x7fd8c4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 msgr2=0x7fd8d40cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 0x7fd8d40cac90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00d830 tx=0x7fd8bc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 shutdown_connections 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd8b0066f60 0x7fd8b0069420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8d40ceb20 0x7fd8d40c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8d40ce670 0x7fd8d40cac90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 --2- 172.21.15.45:0/3939768487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8d40ce290 0x7fd8d414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 >> 172.21.15.45:0/3939768487 conn(0x7fd8d40b32b0 msgr2=0x7fd8d406f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 shutdown_connections 2024-04-25T10:20:42.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.183+0000 7fd8d9a00640 1 -- 172.21.15.45:0/3939768487 wait complete. 2024-04-25T10:20:42.207 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:20:42.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:42.263 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:42.342 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:20:42.342 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:20:42.342 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:42.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:42.351 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:20:42.351 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:42.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:42.360 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:20:42.360 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:42.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:42.400 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:20:42.400 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:42.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:42.411 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:20:42.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 Processor -- start 2024-04-25T10:20:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 -- start start 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f52e80d1820 0x7f52e80d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52e815e050 0x7f52e80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e80b1af0 con 0x7f52e80b13e0 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80b1dd0 con 0x7f52e815e050 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ef600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80b20b0 con 0x7f52e80d1820 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52e815e050 0x7f52e80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52e815e050 0x7f52e80b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39692/0 (socket says 172.21.15.45:39692) 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ed600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f52e80b13e0 0x7f52e80ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37046/0 (socket says 172.21.15.45:37046) 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 -- 172.21.15.45:0/3889750398 learned_addr learned my addr 172.21.15.45:0/3889750398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52ecc00640 1 --2- 172.21.15.45:0/3889750398 >> v2:172.21.15.117:3300/0 conn(0x7f52e80d1820 0x7f52e80d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 -- 172.21.15.45:0/3889750398 >> v2:172.21.15.117:3300/0 conn(0x7f52e80d1820 msgr2=0x7f52e80d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 --2- 172.21.15.45:0/3889750398 >> v2:172.21.15.117:3300/0 conn(0x7f52e80d1820 0x7f52e80d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 -- 172.21.15.45:0/3889750398 >> v1:172.21.15.45:6789/0 conn(0x7f52e80b13e0 legacy=0x7f52e80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7e00640 1 -- 172.21.15.45:0/3889750398 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80b2390 con 0x7f52e815e050 2024-04-25T10:20:42.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.583+0000 7f52e7400640 1 -- 172.21.15.45:0/3889750398 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108637021 0 0) 0x7f52e80b1af0 con 0x7f52e80b13e0 2024-04-25T10:20:42.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52e7e00640 1 --2- 172.21.15.45:0/3889750398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e815e050 0x7f52e80b0ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52e8070e90 tx=0x7f52d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0aecea9c652cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52e7400640 1 -- 172.21.15.45:0/3889750398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40a9400 con 0x7f52e815e050 2024-04-25T10:20:42.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52e7400640 1 -- 172.21.15.45:0/3889750398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d40a95a0 con 0x7f52e815e050 2024-04-25T10:20:42.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52e7400640 1 -- 172.21.15.45:0/3889750398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40a9970 con 0x7f52e815e050 2024-04-25T10:20:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 -- 172.21.15.45:0/3889750398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e815e050 msgr2=0x7f52e80b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 --2- 172.21.15.45:0/3889750398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e815e050 0x7f52e80b0ea0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f52e8070e90 tx=0x7f52d401c520 comp rx=0 tx=0).stop 2024-04-25T10:20:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 -- 172.21.15.45:0/3889750398 shutdown_connections 2024-04-25T10:20:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 --2- 172.21.15.45:0/3889750398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e815e050 0x7f52e80b0ea0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 --2- 172.21.15.45:0/3889750398 >> v2:172.21.15.117:3300/0 conn(0x7f52e80d1820 0x7f52e80d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 -- 172.21.15.45:0/3889750398 >> 172.21.15.45:0/3889750398 conn(0x7f52e809a9d0 msgr2=0x7f52e809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 -- 172.21.15.45:0/3889750398 shutdown_connections 2024-04-25T10:20:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.587+0000 7f52ef600640 1 -- 172.21.15.45:0/3889750398 wait complete. 2024-04-25T10:20:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 Processor -- start 2024-04-25T10:20:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- start start 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 0x7f52e815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52e80d1820 0x7f52e8154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52e815e050 0x7f52e809b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e816ab20 con 0x7f52e80b13e0 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e816a9a0 con 0x7f52e80d1820 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e816aca0 con 0x7f52e815e050 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 0x7f52e815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 0x7f52e815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39700/0 (socket says 172.21.15.45:39700) 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 -- 172.21.15.45:0/392979864 learned_addr learned my addr 172.21.15.45:0/392979864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 -- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52e815e050 msgr2=0x7f52e809b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52e815e050 0x7f52e809b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 -- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52e80d1820 msgr2=0x7f52e8154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52e80d1820 0x7f52e8154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 -- 172.21.15.45:0/392979864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e809d8c0 con 0x7f52e80b13e0 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ecc00640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 0x7f52e815e450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00d800 tx=0x7f52dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00b660 con 0x7f52e80b13e0 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc019070 con 0x7f52e80b13e0 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e809dba0 con 0x7f52e80b13e0 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00ba20 con 0x7f52e80b13e0 2024-04-25T10:20:42.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e809e440 con 0x7f52e80b13e0 2024-04-25T10:20:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.591+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e809df90 con 0x7f52e80b13e0 2024-04-25T10:20:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.595+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52dc00bbc0 con 0x7f52e80b13e0 2024-04-25T10:20:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.595+0000 7f52e4c00640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52c8066f60 0x7f52c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.595+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f52dc0851a0 con 0x7f52e80b13e0 2024-04-25T10:20:42.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.595+0000 7f52e7e00640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52c8066f60 0x7f52c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.595+0000 7f52e7e00640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52c8066f60 0x7f52c8069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f52d40ae000 tx=0x7f52d4004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.599+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52dc0564a0 con 0x7f52e80b13e0 2024-04-25T10:20:42.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.795+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52e804ef30 con 0x7f52e80b13e0 2024-04-25T10:20:42.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.795+0000 7f52e4c00640 1 -- 172.21.15.45:0/392979864 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f52dc055da0 con 0x7f52e80b13e0 2024-04-25T10:20:42.800 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:42.800 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3560218580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3560218580},{"type":"v1","addr":"172.21.15.45:6834","nonce":3560218580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1700278738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1700278738},{"type":"v1","addr":"172.21.15.117:6836","nonce":1700278738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1331840791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1331840791},{"type":"v1","addr":"172.21.15.117:6837","nonce":1331840791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/241691971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":241691971},{"type":"v1","addr":"172.21.15.45:6835","nonce":241691971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-04-25T10:20:42.800 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 20 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52c8066f60 msgr2=0x7f52c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52c8066f60 0x7f52c8069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f52d40ae000 tx=0x7f52d4004740 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 msgr2=0x7f52e815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 0x7f52e815e450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00d800 tx=0x7f52dc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 shutdown_connections 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52c8066f60 0x7f52c8069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52e815e050 0x7f52e809b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52e80d1820 0x7f52e8154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 --2- 172.21.15.45:0/392979864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52e80b13e0 0x7f52e815e450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 >> 172.21.15.45:0/392979864 conn(0x7f52e809a9d0 msgr2=0x7f52e80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 shutdown_connections 2024-04-25T10:20:42.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:42.799+0000 7f52ef600640 1 -- 172.21.15.45:0/392979864 wait complete. 2024-04-25T10:20:42.820 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:20:42.821 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:20:42.821 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:20:42.821 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:42.821 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:20:42.821 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:20:42.821 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:20:42.821 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:42.822 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:20:42.822 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:20:42.822 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:20:42.822 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:42.822 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:20:42.822 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:20:42.823 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:20:42.823 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:42.834 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:20:42.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:20:42.836 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:20:42.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:20:42.838 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:20:42.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:20:42.839 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:20:42.839 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:20:42.883 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:20:42.883 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:42.884 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:20:42.884 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:42.886 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:20:42.886 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:42.887 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:20:42.887 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:42.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:20:42.971 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:20:42.971 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:20:42.976 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:20:42.976 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:20:43.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 Processor -- start 2024-04-25T10:20:43.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 -- start start 2024-04-25T10:20:43.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febfc15bc40 0x7febfc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7febfc0cf930 0x7febfc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febfc0b2770 con 0x7febfc0cda10 2024-04-25T10:20:43.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febfc06c670 con 0x7febfc15bc40 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec03800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febfc06c950 con 0x7febfc0cf930 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febfc15bc40 0x7febfc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febfc15bc40 0x7febfc06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39740/0 (socket says 172.21.15.45:39740) 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 -- 172.21.15.45:0/1335700287 learned_addr learned my addr 172.21.15.45:0/1335700287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec01800640 1 --2- 172.21.15.45:0/1335700287 >> v2:172.21.15.117:3300/0 conn(0x7febfc0cf930 0x7febfc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 -- 172.21.15.45:0/1335700287 >> v2:172.21.15.117:3300/0 conn(0x7febfc0cf930 msgr2=0x7febfc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 --2- 172.21.15.45:0/1335700287 >> v2:172.21.15.117:3300/0 conn(0x7febfc0cf930 0x7febfc0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 -- 172.21.15.45:0/1335700287 >> v1:172.21.15.117:6789/0 conn(0x7febfc0cda10 legacy=0x7febfc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec00e00640 1 -- 172.21.15.45:0/1335700287 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc06cc30 con 0x7febfc15bc40 2024-04-25T10:20:43.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.031+0000 7fec01800640 1 --2- 172.21.15.45:0/1335700287 >> v2:172.21.15.117:3300/0 conn(0x7febfc0cf930 0x7febfc0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec00e00640 1 --2- 172.21.15.45:0/1335700287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc06c090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7febfc0ae4c0 tx=0x7febec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a1b8db2e5a45c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7febfbe00640 1 -- 172.21.15.45:0/1335700287 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0b0070 con 0x7febfc15bc40 2024-04-25T10:20:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7febfbe00640 1 -- 172.21.15.45:0/1335700287 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec0ab600 con 0x7febfc15bc40 2024-04-25T10:20:43.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7febfbe00640 1 -- 172.21.15.45:0/1335700287 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0ab9d0 con 0x7febfc15bc40 2024-04-25T10:20:43.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 -- 172.21.15.45:0/1335700287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 msgr2=0x7febfc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 --2- 172.21.15.45:0/1335700287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc06c090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7febfc0ae4c0 tx=0x7febec01c490 comp rx=0 tx=0).stop 2024-04-25T10:20:43.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 -- 172.21.15.45:0/1335700287 shutdown_connections 2024-04-25T10:20:43.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 --2- 172.21.15.45:0/1335700287 >> v2:172.21.15.117:3300/0 conn(0x7febfc0cf930 0x7febfc0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 --2- 172.21.15.45:0/1335700287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc06c090 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 -- 172.21.15.45:0/1335700287 >> 172.21.15.45:0/1335700287 conn(0x7febfc066b70 msgr2=0x7febfc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 -- 172.21.15.45:0/1335700287 shutdown_connections 2024-04-25T10:20:43.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.035+0000 7fec03800640 1 -- 172.21.15.45:0/1335700287 wait complete. 2024-04-25T10:20:43.040 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:20:43.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:20:43.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:20:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 Processor -- start 2024-04-25T10:20:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- start start 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 0x7febfc14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febfc0cf930 0x7febfc150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc13a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc0b3850 con 0x7febfc15bc40 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc0b36d0 con 0x7febfc0cda10 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc0b39d0 con 0x7febfc0cf930 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec01800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc13a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 0x7febfc14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7febf3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febfc0cf930 0x7febfc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 0x7febfc14fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49718/0 (socket says 172.21.15.45:49718) 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 -- 172.21.15.45:0/292398889 learned_addr learned my addr 172.21.15.45:0/292398889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 -- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febfc0cf930 msgr2=0x7febfc150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febfc0cf930 0x7febfc150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 -- 172.21.15.45:0/292398889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 msgr2=0x7febfc13a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc13a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 -- 172.21.15.45:0/292398889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc15b420 con 0x7febfc0cda10 2024-04-25T10:20:43.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec01800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc13a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:43.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec00e00640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 0x7febfc14fc40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7febec0ae040 tx=0x7febec019230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0b0040 con 0x7febfc0cda10 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec0b9070 con 0x7febfc0cda10 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0b48a0 con 0x7febfc0cda10 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc1569d0 con 0x7febfc0cda10 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc1387e0 con 0x7febfc0cda10 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.039+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febfc04ef30 con 0x7febfc0cda10 2024-04-25T10:20:43.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.043+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febec0b4a40 con 0x7febfc0cda10 2024-04-25T10:20:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.043+0000 7febf9600640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7febd8066f60 0x7febd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.043+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7febec11f3c0 con 0x7febfc0cda10 2024-04-25T10:20:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.043+0000 7febf3e00640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7febd8066f60 0x7febd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.043+0000 7febf3e00640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7febd8066f60 0x7febd8069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7febfc159b00 tx=0x7febe4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.047+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7febfc1569d0 con 0x7febfc0cda10 2024-04-25T10:20:43.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.219+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7febfc06d960 con 0x7febfc0cda10 2024-04-25T10:20:43.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7febf9600640 1 -- 172.21.15.45:0/292398889 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7febec016110 con 0x7febfc0cda10 2024-04-25T10:20:43.841 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:20:43.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7febd8066f60 msgr2=0x7febd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7febd8066f60 0x7febd8069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7febfc159b00 tx=0x7febe4009040 comp rx=0 tx=0).stop 2024-04-25T10:20:43.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 msgr2=0x7febfc14fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 0x7febfc14fc40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7febec0ae040 tx=0x7febec019230 comp rx=0 tx=0).stop 2024-04-25T10:20:43.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 shutdown_connections 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7febd8066f60 0x7febd8069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febfc15bc40 0x7febfc13a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febfc0cf930 0x7febfc150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 --2- 172.21.15.45:0/292398889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febfc0cda10 0x7febfc14fc40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 >> 172.21.15.45:0/292398889 conn(0x7febfc066b70 msgr2=0x7febfc0ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 shutdown_connections 2024-04-25T10:20:43.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:43.839+0000 7fec03800640 1 -- 172.21.15.45:0/292398889 wait complete. 2024-04-25T10:20:43.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:20:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 Processor -- start 2024-04-25T10:20:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 -- start start 2024-04-25T10:20:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 0x7f12b815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12b815b740 0x7f12b80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b8071050 con 0x7f12b81550b0 2024-04-25T10:20:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b815d040 con 0x7f12b815b740 2024-04-25T10:20:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80712f0 con 0x7f12b80d0040 2024-04-25T10:20:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f12b81550b0 0x7f12b80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37054/0 (socket says 172.21.15.45:37054) 2024-04-25T10:20:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7e00640 1 -- 172.21.15.45:0/3622791073 learned_addr learned my addr 172.21.15.45:0/3622791073 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:44.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bce00640 1 --2- 172.21.15.45:0/3622791073 >> v2:172.21.15.117:3300/0 conn(0x7f12b815b740 0x7f12b80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 0x7f12b815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921595363 0 0) 0x7f12b8071050 con 0x7f12b81550b0 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f129c003530 con 0x7f12b81550b0 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177320793 0 0) 0x7f129c003530 con 0x7f12b81550b0 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 msgr2=0x7f12b815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 0x7f12b815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 >> v2:172.21.15.117:3300/0 conn(0x7f12b815b740 msgr2=0x7f12b80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 --2- 172.21.15.45:0/3622791073 >> v2:172.21.15.117:3300/0 conn(0x7f12b815b740 0x7f12b80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80715d0 con 0x7f12b81550b0 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f12a8001680 con 0x7f12b81550b0 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 >> v1:172.21.15.45:6789/0 conn(0x7f12b81550b0 legacy=0x7f12b80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129c0039c0 0x7f129c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129c004810 0x7f129c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 0x7f129c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b815d040 con 0x7f129c004810 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80712f0 con 0x7f129c0039c0 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c00b9d0 con 0x7f12b80d0040 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129c004810 0x7f129c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7e00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129c0039c0 0x7f129c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12bce00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 0x7f129c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 -- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 msgr2=0x7f129c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12b80d0040 0x7f129c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 -- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129c0039c0 msgr2=0x7f129c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129c0039c0 0x7f129c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7400640 1 -- 172.21.15.45:0/3622791073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c003530 con 0x7f129c004810 2024-04-25T10:20:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.011+0000 7f12b7e00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f129c0039c0 0x7f129c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12b7400640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129c004810 0x7f129c004c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009ae0 tx=0x7f12a401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56e4a896209b622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40096b0 con 0x7f129c004810 2024-04-25T10:20:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a40a64a0 con 0x7f129c004810 2024-04-25T10:20:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12b6a00640 1 -- 172.21.15.45:0/3622791073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40a6890 con 0x7f129c004810 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bee00640 1 -- 172.21.15.45:0/3622791073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129c004810 msgr2=0x7f129c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bee00640 1 --2- 172.21.15.45:0/3622791073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129c004810 0x7f129c004c90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009ae0 tx=0x7f12a401ca90 comp rx=0 tx=0).stop 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bce00640 1 -- 172.21.15.45:0/3622791073 reap_dead start 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bee00640 1 -- 172.21.15.45:0/3622791073 shutdown_connections 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bee00640 1 -- 172.21.15.45:0/3622791073 >> 172.21.15.45:0/3622791073 conn(0x7f12b8066b70 msgr2=0x7f12b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bee00640 1 -- 172.21.15.45:0/3622791073 shutdown_connections 2024-04-25T10:20:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.015+0000 7f12bee00640 1 -- 172.21.15.45:0/3622791073 wait complete. 2024-04-25T10:20:44.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 Processor -- start 2024-04-25T10:20:44.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 -- start start 2024-04-25T10:20:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129c004810 0x7f12b81570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b8151fc0 0x7f12b81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 0x7f12b8151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80720b0 con 0x7f12b81513f0 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8071b90 con 0x7f12b8151fc0 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b815bb50 con 0x7f129c004810 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 0x7f12b8151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 0x7f12b8151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39772/0 (socket says 172.21.15.45:39772) 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 -- 172.21.15.45:0/2012264880 learned_addr learned my addr 172.21.15.45:0/2012264880 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12b7e00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129c004810 0x7f12b81570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12b7400640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b8151fc0 0x7f12b81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 -- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129c004810 msgr2=0x7f12b81570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129c004810 0x7f12b81570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 -- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b8151fc0 msgr2=0x7f12b81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b8151fc0 0x7f12b81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 -- 172.21.15.45:0/2012264880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8151da0 con 0x7f12b81513f0 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12b7400640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b8151fc0 0x7f12b81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bce00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 0x7f12b8151860 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12b80d04d0 tx=0x7f12b00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011e10 con 0x7f12b81513f0 2024-04-25T10:20:44.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b001b070 con 0x7f12b81513f0 2024-04-25T10:20:44.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00039a0 con 0x7f12b81513f0 2024-04-25T10:20:44.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b8156020 con 0x7f12b81513f0 2024-04-25T10:20:44.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.019+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b81565e0 con 0x7f12b81513f0 2024-04-25T10:20:44.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.023+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b804ef30 con 0x7f12b81513f0 2024-04-25T10:20:44.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.023+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12b0003b60 con 0x7f12b81513f0 2024-04-25T10:20:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.023+0000 7f12afe00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1298066ed0 0x7f1298069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.023+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12b00854f0 con 0x7f12b81513f0 2024-04-25T10:20:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.023+0000 7f12b7e00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1298066ed0 0x7f1298069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.023+0000 7f12b7e00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1298066ed0 0x7f1298069390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12a8001040 tx=0x7f12a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.027+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12b0056840 con 0x7f12b81513f0 2024-04-25T10:20:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.247+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f12b80b4280 con 0x7f12b81513f0 2024-04-25T10:20:44.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.247+0000 7f12afe00640 1 -- 172.21.15.45:0/2012264880 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f12b005c070 con 0x7f12b81513f0 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:20:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:20:44.255 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:20:44.256 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:20:44.258 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:44.260 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.261 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.262 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:20:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:20:44.265 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:20:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1298066ed0 msgr2=0x7f1298069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1298066ed0 0x7f1298069390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f12a8001040 tx=0x7f12a8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 msgr2=0x7f12b8151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 0x7f12b8151860 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f12b80d04d0 tx=0x7f12b00115a0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 shutdown_connections 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1298066ed0 0x7f1298069390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12b81513f0 0x7f12b8151860 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12b8151fc0 0x7f12b81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 --2- 172.21.15.45:0/2012264880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f129c004810 0x7f12b81570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 >> 172.21.15.45:0/2012264880 conn(0x7f12b8066b70 msgr2=0x7f12b80afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 shutdown_connections 2024-04-25T10:20:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.251+0000 7f12bee00640 1 -- 172.21.15.45:0/2012264880 wait complete. 2024-04-25T10:20:44.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.411+0000 7fe670000640 1 Processor -- start 2024-04-25T10:20:44.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.411+0000 7fe670000640 1 -- start start 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe670000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe66815bc40 0x7fe66806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe670000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6680cf930 0x7fe6680cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe670000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6680b2770 con 0x7fe6680cda10 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe670000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66806c670 con 0x7fe66815bc40 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe670000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66806c950 con 0x7fe6680cf930 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66cc00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe6680cda10 0x7fe6680b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37066/0 (socket says 172.21.15.45:37066) 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66cc00640 1 -- 172.21.15.45:0/3025959087 learned_addr learned my addr 172.21.15.45:0/3025959087 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66d600640 1 --2- 172.21.15.45:0/3025959087 >> v2:172.21.15.45:3300/0 conn(0x7fe66815bc40 0x7fe66806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66e000640 1 --2- 172.21.15.45:0/3025959087 >> v2:172.21.15.117:3300/0 conn(0x7fe6680cf930 0x7fe6680cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66d600640 1 -- 172.21.15.45:0/3025959087 >> v2:172.21.15.117:3300/0 conn(0x7fe6680cf930 msgr2=0x7fe6680cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66d600640 1 --2- 172.21.15.45:0/3025959087 >> v2:172.21.15.117:3300/0 conn(0x7fe6680cf930 0x7fe6680cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66d600640 1 -- 172.21.15.45:0/3025959087 >> v1:172.21.15.45:6789/0 conn(0x7fe6680cda10 legacy=0x7fe6680b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66d600640 1 -- 172.21.15.45:0/3025959087 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66806cc30 con 0x7fe66815bc40 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe66e000640 1 --2- 172.21.15.45:0/3025959087 >> v2:172.21.15.117:3300/0 conn(0x7fe6680cf930 0x7fe6680cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:44.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.415+0000 7fe667e00640 1 -- 172.21.15.45:0/3025959087 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472723884 0 0) 0x7fe6680b2770 con 0x7fe6680cda10 2024-04-25T10:20:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe66d600640 1 --2- 172.21.15.45:0/3025959087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66815bc40 0x7fe66806c090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6680ae4c0 tx=0x7fe65c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d10c01dd938a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe667e00640 1 -- 172.21.15.45:0/3025959087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0b0070 con 0x7fe66815bc40 2024-04-25T10:20:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe667e00640 1 -- 172.21.15.45:0/3025959087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c0ab600 con 0x7fe66815bc40 2024-04-25T10:20:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe667e00640 1 -- 172.21.15.45:0/3025959087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0ab9d0 con 0x7fe66815bc40 2024-04-25T10:20:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- 172.21.15.45:0/3025959087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66815bc40 msgr2=0x7fe66806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 --2- 172.21.15.45:0/3025959087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66815bc40 0x7fe66806c090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6680ae4c0 tx=0x7fe65c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- 172.21.15.45:0/3025959087 shutdown_connections 2024-04-25T10:20:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 --2- 172.21.15.45:0/3025959087 >> v2:172.21.15.117:3300/0 conn(0x7fe6680cf930 0x7fe6680cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 --2- 172.21.15.45:0/3025959087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe66815bc40 0x7fe66806c090 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- 172.21.15.45:0/3025959087 >> 172.21.15.45:0/3025959087 conn(0x7fe668066b70 msgr2=0x7fe668066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- 172.21.15.45:0/3025959087 shutdown_connections 2024-04-25T10:20:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- 172.21.15.45:0/3025959087 wait complete. 2024-04-25T10:20:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 Processor -- start 2024-04-25T10:20:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- start start 2024-04-25T10:20:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 0x7fe668148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6680cf930 0x7fe668071840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe66815bc40 0x7fe6680b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b3850 con 0x7fe6680cda10 2024-04-25T10:20:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b36d0 con 0x7fe6680cf930 2024-04-25T10:20:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe670000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b39d0 con 0x7fe66815bc40 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe66d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 0x7fe668148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe66d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 0x7fe668148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39788/0 (socket says 172.21.15.45:39788) 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.419+0000 7fe66d600640 1 -- 172.21.15.45:0/2929417750 learned_addr learned my addr 172.21.15.45:0/2929417750 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe66d600640 1 -- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe66815bc40 msgr2=0x7fe6680b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe66d600640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe66815bc40 0x7fe6680b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe66d600640 1 -- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6680cf930 msgr2=0x7fe668071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe66d600640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6680cf930 0x7fe668071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe66d600640 1 -- 172.21.15.45:0/2929417750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6680707a0 con 0x7fe6680cda10 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe66d600640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 0x7fe668148d60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe65c01a020 tx=0x7fe65c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0b8020 con 0x7fe6680cda10 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c0b4630 con 0x7fe6680cda10 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0b0040 con 0x7fe6680cda10 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe668070a80 con 0x7fe6680cda10 2024-04-25T10:20:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe668071040 con 0x7fe6680cda10 2024-04-25T10:20:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66804ef30 con 0x7fe6680cda10 2024-04-25T10:20:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe65c0bb020 con 0x7fe6680cda10 2024-04-25T10:20:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe665600640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe648066f90 0x7fe648069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.423+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe65c11ea10 con 0x7fe6680cda10 2024-04-25T10:20:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.427+0000 7fe66cc00640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe648066f90 0x7fe648069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.427+0000 7fe66cc00640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe648066f90 0x7fe648069450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe66814efe0 tx=0x7fe654007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.427+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe65c0c2030 con 0x7fe6680cda10 2024-04-25T10:20:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.591+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe668159a30 con 0x7fe6680cda10 2024-04-25T10:20:44.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.591+0000 7fe665600640 1 -- 172.21.15.45:0/2929417750 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12879 (secure 0 0 0) 0x7fe65c016070 con 0x7fe6680cda10 2024-04-25T10:20:44.595 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:44.595 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:43.845932+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe648066f90 msgr2=0x7fe648069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe648066f90 0x7fe648069450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe66814efe0 tx=0x7fe654007970 comp rx=0 tx=0).stop 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 msgr2=0x7fe668148d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 0x7fe668148d60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe65c01a020 tx=0x7fe65c019190 comp rx=0 tx=0).stop 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 shutdown_connections 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe648066f90 0x7fe648069450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe66815bc40 0x7fe6680b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6680cf930 0x7fe668071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 --2- 172.21.15.45:0/2929417750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6680cda10 0x7fe668148d60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 >> 172.21.15.45:0/2929417750 conn(0x7fe668066b70 msgr2=0x7fe668151960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 shutdown_connections 2024-04-25T10:20:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.595+0000 7fe670000640 1 -- 172.21.15.45:0/2929417750 wait complete. 2024-04-25T10:20:44.617 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:44.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:44.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 Processor -- start 2024-04-25T10:20:44.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 -- start start 2024-04-25T10:20:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2a840aef80 0x7f2a8415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a84070350 0x7f2a8415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8415b080 con 0x7f2a8415bac0 2024-04-25T10:20:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8415adb0 con 0x7f2a840aef80 2024-04-25T10:20:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a89600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8415eb60 con 0x7f2a84070350 2024-04-25T10:20:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a83400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a8415bac0 0x7f2a840b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37068/0 (socket says 172.21.15.45:37068) 2024-04-25T10:20:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a83400640 1 -- 172.21.15.45:0/1731195835 learned_addr learned my addr 172.21.15.45:0/1731195835 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a82a00640 1 --2- 172.21.15.45:0/1731195835 >> v2:172.21.15.117:3300/0 conn(0x7f2a840aef80 0x7f2a8415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a82000640 1 --2- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a84070350 0x7f2a8415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a81600640 1 -- 172.21.15.45:0/1731195835 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010307848 0 0) 0x7f2a8415b080 con 0x7f2a8415bac0 2024-04-25T10:20:44.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a81600640 1 -- 172.21.15.45:0/1731195835 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a68003530 con 0x7f2a8415bac0 2024-04-25T10:20:44.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a82a00640 1 -- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a84070350 msgr2=0x7f2a8415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a82a00640 1 --2- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a84070350 0x7f2a8415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a82a00640 1 -- 172.21.15.45:0/1731195835 >> v1:172.21.15.45:6789/0 conn(0x7f2a8415bac0 legacy=0x7f2a840b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.759+0000 7f2a82a00640 1 -- 172.21.15.45:0/1731195835 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8415ee40 con 0x7f2a840aef80 2024-04-25T10:20:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a82a00640 1 --2- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a8415e480 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a740097c0 tx=0x7f2a7401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59aa3c388807b467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a81600640 1 -- 172.21.15.45:0/1731195835 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740ab680 con 0x7f2a840aef80 2024-04-25T10:20:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a81600640 1 -- 172.21.15.45:0/1731195835 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a740b0070 con 0x7f2a840aef80 2024-04-25T10:20:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a81600640 1 -- 172.21.15.45:0/1731195835 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740aba40 con 0x7f2a840aef80 2024-04-25T10:20:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 -- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 msgr2=0x7f2a8415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 --2- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a8415e480 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a740097c0 tx=0x7f2a7401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 -- 172.21.15.45:0/1731195835 shutdown_connections 2024-04-25T10:20:44.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 --2- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a84070350 0x7f2a8415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 --2- 172.21.15.45:0/1731195835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a8415e480 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 -- 172.21.15.45:0/1731195835 >> 172.21.15.45:0/1731195835 conn(0x7f2a840669a0 msgr2=0x7f2a84066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 -- 172.21.15.45:0/1731195835 shutdown_connections 2024-04-25T10:20:44.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.763+0000 7f2a89600640 1 -- 172.21.15.45:0/1731195835 wait complete. 2024-04-25T10:20:44.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 Processor -- start 2024-04-25T10:20:44.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- start start 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 0x7f2a840b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a840b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a8415bac0 0x7f2a840b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8406c0d0 con 0x7f2a84070350 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840b3b20 con 0x7f2a840aef80 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8406c250 con 0x7f2a8415bac0 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 0x7f2a840b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 0x7f2a840b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39794/0 (socket says 172.21.15.45:39794) 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 -- 172.21.15.45:0/234934870 learned_addr learned my addr 172.21.15.45:0/234934870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82000640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a840b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a83400640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a8415bac0 0x7f2a840b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 -- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a8415bac0 msgr2=0x7f2a840b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a8415bac0 0x7f2a840b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 -- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 msgr2=0x7f2a840b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a840b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 -- 172.21.15.45:0/234934870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840cedc0 con 0x7f2a84070350 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a82a00640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 0x7f2a840b1670 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2a7401a0d0 tx=0x7f2a7401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740abe60 con 0x7f2a84070350 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a740a7070 con 0x7f2a84070350 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740bf870 con 0x7f2a84070350 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a84157a40 con 0x7f2a84070350 2024-04-25T10:20:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841423f0 con 0x7f2a84070350 2024-04-25T10:20:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.767+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8404ef30 con 0x7f2a84070350 2024-04-25T10:20:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.771+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a740bd020 con 0x7f2a84070350 2024-04-25T10:20:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.771+0000 7f2a7aa00640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.771+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a7411ebd0 con 0x7f2a84070350 2024-04-25T10:20:44.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.771+0000 7f2a82000640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.771+0000 7f2a82000640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a64066f60 0x7f2a64069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a84067220 tx=0x7f2a7000b750 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.775+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a740c6070 con 0x7f2a84070350 2024-04-25T10:20:44.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a840b2fa0 con 0x7f2a84070350 2024-04-25T10:20:44.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a7aa00640 1 -- 172.21.15.45:0/234934870 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a74016090 con 0x7f2a84070350 2024-04-25T10:20:44.946 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a64066f60 msgr2=0x7f2a64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a64066f60 0x7f2a64069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a84067220 tx=0x7f2a7000b750 comp rx=0 tx=0).stop 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 msgr2=0x7f2a840b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 0x7f2a840b1670 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2a7401a0d0 tx=0x7f2a7401ace0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 shutdown_connections 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a8415bac0 0x7f2a840b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a840aef80 0x7f2a840b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 --2- 172.21.15.45:0/234934870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a84070350 0x7f2a840b1670 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 >> 172.21.15.45:0/234934870 conn(0x7f2a840669a0 msgr2=0x7f2a840b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 shutdown_connections 2024-04-25T10:20:44.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:44.943+0000 7f2a89600640 1 -- 172.21.15.45:0/234934870 wait complete. 2024-04-25T10:20:44.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:20:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc74a400640 1 Processor -- start 2024-04-25T10:20:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc74a400640 1 -- start start 2024-04-25T10:20:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc74a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 0x7fc74406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc74a400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440b2770 con 0x7fc74415bc40 2024-04-25T10:20:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc74a400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74406c670 con 0x7fc7440cda10 2024-04-25T10:20:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc74a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406c950 con 0x7fc7440d0040 2024-04-25T10:20:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc743400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc74415bc40 0x7fc7440b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46174/0 (socket says 172.21.15.45:46174) 2024-04-25T10:20:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.103+0000 7fc743400640 1 -- 172.21.15.45:0/294106268 learned_addr learned my addr 172.21.15.45:0/294106268 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451880601 0 0) 0x7fc7440b2770 con 0x7fc74415bc40 2024-04-25T10:20:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc728003530 con 0x7fc74415bc40 2024-04-25T10:20:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc743e00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 0x7fc74406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213909346 0 0) 0x7fc728003530 con 0x7fc74415bc40 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 msgr2=0x7fc74406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 0x7fc74406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 >> v1:172.21.15.117:6789/0 conn(0x7fc7440cda10 legacy=0x7fc7440cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74406cc30 con 0x7fc74415bc40 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fc734001680 con 0x7fc74415bc40 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 >> v1:172.21.15.45:6789/0 conn(0x7fc74415bc40 legacy=0x7fc7440b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 0x7fc7280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc728003f30 0x7fc728004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728004d50 0x7fc7280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74406c950 con 0x7fc728003f30 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72800bae0 con 0x7fc728004d50 2024-04-25T10:20:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72800bce0 con 0x7fc7440d0040 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc73be00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc728003f30 0x7fc728004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc743400640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 0x7fc7280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc743e00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728004d50 0x7fc7280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc73be00640 1 -- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 msgr2=0x7fc7280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc73be00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7440d0040 0x7fc7280039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc73be00640 1 -- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728004d50 msgr2=0x7fc7280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc73be00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728004d50 0x7fc7280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.107+0000 7fc73be00640 1 -- 172.21.15.45:0/294106268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728003530 con 0x7fc728003f30 2024-04-25T10:20:45.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc73be00640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc728003f30 0x7fc728004810 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001860 tx=0x7fc72c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6ecd477f155b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0b0070 con 0x7fc728003f30 2024-04-25T10:20:45.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c0ab600 con 0x7fc728003f30 2024-04-25T10:20:45.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc742a00640 1 -- 172.21.15.45:0/294106268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c001460 con 0x7fc728003f30 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 -- 172.21.15.45:0/294106268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc728003f30 msgr2=0x7fc728004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 --2- 172.21.15.45:0/294106268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc728003f30 0x7fc728004810 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001860 tx=0x7fc72c01c500 comp rx=0 tx=0).stop 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc743e00640 1 -- 172.21.15.45:0/294106268 reap_dead start 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 -- 172.21.15.45:0/294106268 shutdown_connections 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 -- 172.21.15.45:0/294106268 >> 172.21.15.45:0/294106268 conn(0x7fc744066b70 msgr2=0x7fc744066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 -- 172.21.15.45:0/294106268 shutdown_connections 2024-04-25T10:20:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 -- 172.21.15.45:0/294106268 wait complete. 2024-04-25T10:20:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 Processor -- start 2024-04-25T10:20:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.111+0000 7fc74a400640 1 -- start start 2024-04-25T10:20:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728003f30 0x7fc74415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7441586d0 0x7fc7440716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 0x7fc7441f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc744167560 con 0x7fc7441f7440 2024-04-25T10:20:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441673e0 con 0x7fc728003f30 2024-04-25T10:20:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441676e0 con 0x7fc7441586d0 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 0x7fc7441f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 0x7fc7441f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54102/0 (socket says 172.21.15.45:54102) 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 -- 172.21.15.45:0/3465377578 learned_addr learned my addr 172.21.15.45:0/3465377578 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc73be00640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7441586d0 0x7fc7440716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728003f30 0x7fc74415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 -- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7441586d0 msgr2=0x7fc7440716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7441586d0 0x7fc7440716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 -- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728003f30 msgr2=0x7fc74415ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728003f30 0x7fc74415ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 -- 172.21.15.45:0/3465377578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440b06d0 con 0x7fc7441f7440 2024-04-25T10:20:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc743e00640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 0x7fc7441f78d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc7440ae4e0 tx=0x7fc73c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c019070 con 0x7fc7441f7440 2024-04-25T10:20:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c011e90 con 0x7fc7441f7440 2024-04-25T10:20:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc744158ed0 con 0x7fc7441f7440 2024-04-25T10:20:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c003c00 con 0x7fc7441f7440 2024-04-25T10:20:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7440b0c50 con 0x7fc7441f7440 2024-04-25T10:20:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74404ef30 con 0x7fc7441f7440 2024-04-25T10:20:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.115+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc73c01c070 con 0x7fc7441f7440 2024-04-25T10:20:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.119+0000 7fc73b400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc720066f60 0x7fc720069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.119+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc73c085650 con 0x7fc7441f7440 2024-04-25T10:20:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.119+0000 7fc743400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc720066f60 0x7fc720069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.119+0000 7fc743400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc720066f60 0x7fc720069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc74406e2d0 tx=0x7fc734000f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.123+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc73c010040 con 0x7fc7441f7440 2024-04-25T10:20:45.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc74406d960 con 0x7fc7441f7440 2024-04-25T10:20:45.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc73b400640 1 -- 172.21.15.45:0/3465377578 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc73c056350 con 0x7fc7441f7440 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:20:45.371 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:20:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc720066f60 msgr2=0x7fc720069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc720066f60 0x7fc720069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc74406e2d0 tx=0x7fc734000f00 comp rx=0 tx=0).stop 2024-04-25T10:20:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 msgr2=0x7fc7441f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 0x7fc7441f78d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc7440ae4e0 tx=0x7fc73c00f580 comp rx=0 tx=0).stop 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 shutdown_connections 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc720066f60 0x7fc720069420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7441f7440 0x7fc7441f78d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7441586d0 0x7fc7440716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 --2- 172.21.15.45:0/3465377578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc728003f30 0x7fc74415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 >> 172.21.15.45:0/3465377578 conn(0x7fc744066b70 msgr2=0x7fc7440cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 shutdown_connections 2024-04-25T10:20:45.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.287+0000 7fc74a400640 1 -- 172.21.15.45:0/3465377578 wait complete. 2024-04-25T10:20:45.374 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:20:45.375 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:20:45.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.515+0000 7f4de1400640 1 Processor -- start 2024-04-25T10:20:45.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.515+0000 7f4de1400640 1 -- start start 2024-04-25T10:20:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4de1400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ddc157cb0 0x7f4ddc0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4de1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc070ba0 0x7f4ddc15b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4de1400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc15b7a0 con 0x7f4ddc0b2af0 2024-04-25T10:20:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4de1400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc0af150 con 0x7f4ddc157cb0 2024-04-25T10:20:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4de1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc071040 con 0x7f4ddc070ba0 2024-04-25T10:20:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4ddb200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ddc0b2af0 0x7f4ddc0b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46182/0 (socket says 172.21.15.45:46182) 2024-04-25T10:20:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4ddb200640 1 -- 172.21.15.45:0/4144985564 learned_addr learned my addr 172.21.15.45:0/4144985564 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dda800640 1 --2- 172.21.15.45:0/4144985564 >> v2:172.21.15.117:3300/0 conn(0x7f4ddc157cb0 0x7f4ddc0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dd3e00640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc070ba0 0x7f4ddc15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dd9e00640 1 -- 172.21.15.45:0/4144985564 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647825260 0 0) 0x7f4dd40014a0 con 0x7f4ddc0b2af0 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dd9e00640 1 -- 172.21.15.45:0/4144985564 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc0003530 con 0x7f4ddc0b2af0 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dda800640 1 -- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc070ba0 msgr2=0x7f4ddc15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dda800640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc070ba0 0x7f4ddc15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dda800640 1 -- 172.21.15.45:0/4144985564 >> v1:172.21.15.45:6789/0 conn(0x7f4ddc0b2af0 legacy=0x7f4ddc0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dda800640 1 -- 172.21.15.45:0/4144985564 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc071320 con 0x7f4ddc157cb0 2024-04-25T10:20:45.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.519+0000 7f4dd3e00640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc070ba0 0x7f4ddc15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:45.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4dda800640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc0720e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0097c0 tx=0x7f4dcc01e6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ede9d8f36fc166d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4dd9e00640 1 -- 172.21.15.45:0/4144985564 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0ab5f0 con 0x7f4ddc157cb0 2024-04-25T10:20:45.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4dd9e00640 1 -- 172.21.15.45:0/4144985564 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0ab790 con 0x7f4ddc157cb0 2024-04-25T10:20:45.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4dd9e00640 1 -- 172.21.15.45:0/4144985564 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0abb60 con 0x7f4ddc157cb0 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 -- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc157cb0 msgr2=0x7f4ddc0720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc0720e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0097c0 tx=0x7f4dcc01e6b0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 -- 172.21.15.45:0/4144985564 shutdown_connections 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc070ba0 0x7f4ddc15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 --2- 172.21.15.45:0/4144985564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc0720e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 -- 172.21.15.45:0/4144985564 >> 172.21.15.45:0/4144985564 conn(0x7f4ddc066b70 msgr2=0x7f4ddc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 -- 172.21.15.45:0/4144985564 shutdown_connections 2024-04-25T10:20:45.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 -- 172.21.15.45:0/4144985564 wait complete. 2024-04-25T10:20:45.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 Processor -- start 2024-04-25T10:20:45.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.523+0000 7f4de1400640 1 -- start start 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc070ba0 0x7f4ddc153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc0b2af0 0x7f4ddc0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc158210 con 0x7f4ddc157cb0 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc158090 con 0x7f4ddc070ba0 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc06c0e0 con 0x7f4ddc0b2af0 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc15a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54118/0 (socket says 172.21.15.45:54118) 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 -- 172.21.15.45:0/1197351479 learned_addr learned my addr 172.21.15.45:0/1197351479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dda800640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc070ba0 0x7f4ddc153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dd3e00640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc0b2af0 0x7f4ddc0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 -- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc0b2af0 msgr2=0x7f4ddc0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc0b2af0 0x7f4ddc0cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 -- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc070ba0 msgr2=0x7f4ddc153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc070ba0 0x7f4ddc153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 -- 172.21.15.45:0/1197351479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc154d40 con 0x7f4ddc157cb0 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dda800640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc070ba0 0x7f4ddc153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dd3e00640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc0b2af0 0x7f4ddc0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4ddb200640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc15a7f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400cb50 tx=0x7f4dd400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4017070 con 0x7f4ddc157cb0 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd400ad40 con 0x7f4ddc157cb0 2024-04-25T10:20:45.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4012620 con 0x7f4ddc157cb0 2024-04-25T10:20:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc155020 con 0x7f4ddc157cb0 2024-04-25T10:20:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc1555e0 con 0x7f4ddc157cb0 2024-04-25T10:20:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc04ef30 con 0x7f4ddc157cb0 2024-04-25T10:20:45.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.527+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4dd401a070 con 0x7f4ddc157cb0 2024-04-25T10:20:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.531+0000 7f4dd2a00640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4db8066f60 0x7f4db8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.531+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4dd4084c00 con 0x7f4ddc157cb0 2024-04-25T10:20:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.531+0000 7f4dda800640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4db8066f60 0x7f4db8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.531+0000 7f4dda800640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4db8066f60 0x7f4db8069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0b0000 tx=0x7f4dcc0b5040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:45.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.535+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ddc15b7a0 con 0x7f4ddc157cb0 2024-04-25T10:20:45.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.695+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4ddc06d920 con 0x7f4ddc157cb0 2024-04-25T10:20:45.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4dd2a00640 1 -- 172.21.15.45:0/1197351479 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4dd4055ec0 con 0x7f4ddc157cb0 2024-04-25T10:20:45.846 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:20:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4db8066f60 msgr2=0x7f4db8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4db8066f60 0x7f4db8069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0b0000 tx=0x7f4dcc0b5040 comp rx=0 tx=0).stop 2024-04-25T10:20:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 msgr2=0x7f4ddc15a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc15a7f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400cb50 tx=0x7f4dd400a430 comp rx=0 tx=0).stop 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 shutdown_connections 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4db8066f60 0x7f4db8069420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc157cb0 0x7f4ddc15a7f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc0b2af0 0x7f4ddc0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 --2- 172.21.15.45:0/1197351479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc070ba0 0x7f4ddc153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 >> 172.21.15.45:0/1197351479 conn(0x7f4ddc066b70 msgr2=0x7f4ddc152870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 shutdown_connections 2024-04-25T10:20:45.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:45.843+0000 7f4de1400640 1 -- 172.21.15.45:0/1197351479 wait complete. 2024-04-25T10:20:45.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 Processor -- start 2024-04-25T10:20:46.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 -- start start 2024-04-25T10:20:46.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45400b06b0 0x7f45400b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f454015bdc0 0x7f454015c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454015ca90 con 0x7f4540144f80 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400cea10 con 0x7f454015bdc0 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400ccf60 con 0x7f45400b06b0 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f454015bdc0 0x7f454015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453de00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4540144f80 0x7f454006c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46190/0 (socket says 172.21.15.45:46190) 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453de00640 1 -- 172.21.15.45:0/231190334 learned_addr learned my addr 172.21.15.45:0/231190334 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:46.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f4535400640 1 --2- 172.21.15.45:0/231190334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45400b06b0 0x7f45400b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453e800640 1 -- 172.21.15.45:0/231190334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45400b06b0 msgr2=0x7f45400b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453e800640 1 --2- 172.21.15.45:0/231190334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45400b06b0 0x7f45400b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453e800640 1 -- 172.21.15.45:0/231190334 >> v1:172.21.15.45:6789/0 conn(0x7f4540144f80 legacy=0x7f454006c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:46.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.019+0000 7f453e800640 1 -- 172.21.15.45:0/231190334 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454015cd70 con 0x7f454015bdc0 2024-04-25T10:20:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453e800640 1 --2- 172.21.15.45:0/231190334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f454015bdc0 0x7f454015c230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4538009d90 tx=0x7f453801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c4d995283e78f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453d400640 1 -- 172.21.15.45:0/231190334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380b2070 con 0x7f454015bdc0 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453d400640 1 -- 172.21.15.45:0/231190334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45380a5920 con 0x7f454015bdc0 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453d400640 1 -- 172.21.15.45:0/231190334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45380a5cf0 con 0x7f454015bdc0 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 -- 172.21.15.45:0/231190334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f454015bdc0 msgr2=0x7f454015c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 --2- 172.21.15.45:0/231190334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f454015bdc0 0x7f454015c230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4538009d90 tx=0x7f453801c970 comp rx=0 tx=0).stop 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 -- 172.21.15.45:0/231190334 shutdown_connections 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 --2- 172.21.15.45:0/231190334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f454015bdc0 0x7f454015c230 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 --2- 172.21.15.45:0/231190334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45400b06b0 0x7f45400b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 -- 172.21.15.45:0/231190334 >> 172.21.15.45:0/231190334 conn(0x7f45400743a0 msgr2=0x7f45400669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 -- 172.21.15.45:0/231190334 shutdown_connections 2024-04-25T10:20:46.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.023+0000 7f453f200640 1 -- 172.21.15.45:0/231190334 wait complete. 2024-04-25T10:20:46.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 Processor -- start 2024-04-25T10:20:46.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 -- start start 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 0x7f454013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4540144f80 0x7f4540139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f454015bdc0 0x7f454013d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401454e0 con 0x7f45400b06b0 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540145360 con 0x7f454015bdc0 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400cdbe0 con 0x7f4540144f80 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 0x7f454013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 0x7f454013e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54136/0 (socket says 172.21.15.45:54136) 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 -- 172.21.15.45:0/2693790868 learned_addr learned my addr 172.21.15.45:0/2693790868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:46.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453e800640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f454015bdc0 0x7f454013d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f4535400640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4540144f80 0x7f4540139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:46.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 -- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4540144f80 msgr2=0x7f4540139cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4540144f80 0x7f4540139cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 -- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f454015bdc0 msgr2=0x7f454013d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f454015bdc0 0x7f454013d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 -- 172.21.15.45:0/2693790868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540157f90 con 0x7f45400b06b0 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453de00640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 0x7f454013e2c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f453000c450 tx=0x7f453000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530017070 con 0x7f45400b06b0 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530005cf0 con 0x7f45400b06b0 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4540158270 con 0x7f45400b06b0 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530007290 con 0x7f45400b06b0 2024-04-25T10:20:46.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.027+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454013b940 con 0x7f45400b06b0 2024-04-25T10:20:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.031+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4540075120 con 0x7f45400b06b0 2024-04-25T10:20:46.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.031+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f453001a070 con 0x7f45400b06b0 2024-04-25T10:20:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.031+0000 7f4536a00640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f451c066f60 0x7f451c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.031+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4530084410 con 0x7f45400b06b0 2024-04-25T10:20:46.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.035+0000 7f4535400640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f451c066f60 0x7f451c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:46.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.035+0000 7f4535400640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f451c066f60 0x7f451c069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4540158880 tx=0x7f4528009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:46.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.035+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4530021080 con 0x7f45400b06b0 2024-04-25T10:20:46.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.203+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f454006dbd0 con 0x7f45400b06b0 2024-04-25T10:20:46.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.851+0000 7f4536a00640 1 -- 172.21.15.45:0/2693790868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4530054fa0 con 0x7f45400b06b0 2024-04-25T10:20:46.855 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:20:46.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f451c066f60 msgr2=0x7f451c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:46.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f451c066f60 0x7f451c069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4540158880 tx=0x7f4528009190 comp rx=0 tx=0).stop 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 msgr2=0x7f454013e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 0x7f454013e2c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f453000c450 tx=0x7f453000c910 comp rx=0 tx=0).stop 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 shutdown_connections 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f451c066f60 0x7f451c069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f454015bdc0 0x7f454013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4540144f80 0x7f4540139cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 --2- 172.21.15.45:0/2693790868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45400b06b0 0x7f454013e2c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:46.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 >> 172.21.15.45:0/2693790868 conn(0x7f45400743a0 msgr2=0x7f4540070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 shutdown_connections 2024-04-25T10:20:46.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:46.855+0000 7f453f200640 1 -- 172.21.15.45:0/2693790868 wait complete. 2024-04-25T10:20:46.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:20:47.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.043+0000 7fe6b9c00640 1 Processor -- start 2024-04-25T10:20:47.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.043+0000 7fe6b9c00640 1 -- start start 2024-04-25T10:20:47.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b9c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6b40aef80 0x7fe6b415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b9c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b415b080 con 0x7fe6b415bac0 2024-04-25T10:20:47.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b9c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b415adb0 con 0x7fe6b40aef80 2024-04-25T10:20:47.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b415eb60 con 0x7fe6b4070350 2024-04-25T10:20:47.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b3400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe6b415bac0 0x7fe6b40b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36842/0 (socket says 172.21.15.45:36842) 2024-04-25T10:20:47.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b415b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46274/0 (socket says 172.21.15.45:46274) 2024-04-25T10:20:47.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b3400640 1 -- 172.21.15.45:0/962244124 learned_addr learned my addr 172.21.15.45:0/962244124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:47.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6b2a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6b40aef80 0x7fe6b415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6aa000640 1 -- 172.21.15.45:0/962244124 >> v2:172.21.15.117:3300/0 conn(0x7fe6b40aef80 msgr2=0x7fe6b415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:47.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6aa000640 1 --2- 172.21.15.45:0/962244124 >> v2:172.21.15.117:3300/0 conn(0x7fe6b40aef80 0x7fe6b415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6aa000640 1 -- 172.21.15.45:0/962244124 >> v1:172.21.15.117:6789/0 conn(0x7fe6b415bac0 legacy=0x7fe6b40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.047+0000 7fe6aa000640 1 -- 172.21.15.45:0/962244124 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b415ee40 con 0x7fe6b4070350 2024-04-25T10:20:47.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6aa000640 1 --2- 172.21.15.45:0/962244124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b415b580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097c0 tx=0x7fe69c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573942a083a6c3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b2000640 1 -- 172.21.15.45:0/962244124 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0ab680 con 0x7fe6b4070350 2024-04-25T10:20:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b2000640 1 -- 172.21.15.45:0/962244124 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0b0070 con 0x7fe6b4070350 2024-04-25T10:20:47.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b2000640 1 -- 172.21.15.45:0/962244124 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0aba40 con 0x7fe6b4070350 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- 172.21.15.45:0/962244124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 msgr2=0x7fe6b415b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/962244124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b415b580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097c0 tx=0x7fe69c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- 172.21.15.45:0/962244124 shutdown_connections 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/962244124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b415b580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/962244124 >> v2:172.21.15.117:3300/0 conn(0x7fe6b40aef80 0x7fe6b415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- 172.21.15.45:0/962244124 >> 172.21.15.45:0/962244124 conn(0x7fe6b40669a0 msgr2=0x7fe6b4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- 172.21.15.45:0/962244124 shutdown_connections 2024-04-25T10:20:47.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- 172.21.15.45:0/962244124 wait complete. 2024-04-25T10:20:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 Processor -- start 2024-04-25T10:20:47.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- start start 2024-04-25T10:20:47.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b414d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 0x7fe6b4159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b415bac0 0x7fe6b4159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4165810 con 0x7fe6b415bac0 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40b3b20 con 0x7fe6b40aef80 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.051+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4165990 con 0x7fe6b4070350 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b415bac0 0x7fe6b4159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b414d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 0x7fe6b4159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b414d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46286/0 (socket says 172.21.15.45:46286) 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 0x7fe6b4159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50620/0 (socket says 172.21.15.45:50620) 2024-04-25T10:20:47.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b2a00640 1 -- 172.21.15.45:0/1266354321 learned_addr learned my addr 172.21.15.45:0/1266354321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:47.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 -- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 msgr2=0x7fe6b414d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b414d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 -- 172.21.15.45:0/1266354321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b415bac0 msgr2=0x7fe6b4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b415bac0 0x7fe6b4159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 -- 172.21.15.45:0/1266354321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4158240 con 0x7fe6b40aef80 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6aa000640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 0x7fe6b4159ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe6b406e620 tx=0x7fe69c01ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c006e20 con 0x7fe6b40aef80 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0a7070 con 0x7fe6b40aef80 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0b5860 con 0x7fe6b40aef80 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b4158520 con 0x7fe6b40aef80 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b40af740 con 0x7fe6b40aef80 2024-04-25T10:20:47.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.055+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b404ef30 con 0x7fe6b40aef80 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.059+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe69c0bd030 con 0x7fe6b40aef80 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.059+0000 7fe6ab400640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe690066f60 0x7fe690069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.059+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe69c11f8e0 con 0x7fe6b40aef80 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.059+0000 7fe6b2a00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe690066f60 0x7fe690069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.059+0000 7fe6b2a00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe690066f60 0x7fe690069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009c20 tx=0x7fe6a4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.063+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe69c016090 con 0x7fe6b40aef80 2024-04-25T10:20:47.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.259+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe6b40b2fa0 con 0x7fe6b40aef80 2024-04-25T10:20:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.883+0000 7fe6ab400640 1 -- 172.21.15.45:0/1266354321 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe69c0f02e0 con 0x7fe6b40aef80 2024-04-25T10:20:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T10:20:47.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe690066f60 msgr2=0x7fe690069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe690066f60 0x7fe690069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009c20 tx=0x7fe6a4009190 comp rx=0 tx=0).stop 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 msgr2=0x7fe6b4159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 0x7fe6b4159ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe6b406e620 tx=0x7fe69c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 shutdown_connections 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe690066f60 0x7fe690069420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6b415bac0 0x7fe6b4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6b40aef80 0x7fe6b4159ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 --2- 172.21.15.45:0/1266354321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6b4070350 0x7fe6b414d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 >> 172.21.15.45:0/1266354321 conn(0x7fe6b40669a0 msgr2=0x7fe6b40b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 shutdown_connections 2024-04-25T10:20:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:47.887+0000 7fe6b9c00640 1 -- 172.21.15.45:0/1266354321 wait complete. 2024-04-25T10:20:47.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.055+0000 7ff700800640 1 Processor -- start 2024-04-25T10:20:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.055+0000 7ff700800640 1 -- start start 2024-04-25T10:20:48.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff700800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff700800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff6f806aea0 0x7ff6f806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff700800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f80ccaa0 con 0x7ff6f815df60 2024-04-25T10:20:48.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff700800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f815e340 con 0x7ff6f806aea0 2024-04-25T10:20:48.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff700800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80ccd80 con 0x7ff6f80d0350 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fd400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fe800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff6f806aea0 0x7ff6f806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fde00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff6f815df60 0x7ff6f80cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36854/0 (socket says 172.21.15.45:36854) 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fd400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f80d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46294/0 (socket says 172.21.15.45:46294) 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fde00640 1 -- 172.21.15.45:0/3034250172 learned_addr learned my addr 172.21.15.45:0/3034250172 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fca00640 1 -- 172.21.15.45:0/3034250172 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267063107 0 0) 0x7ff6f80ccaa0 con 0x7ff6f815df60 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fca00640 1 -- 172.21.15.45:0/3034250172 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e0003530 con 0x7ff6f815df60 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fd400640 1 -- 172.21.15.45:0/3034250172 >> v2:172.21.15.117:3300/0 conn(0x7ff6f806aea0 msgr2=0x7ff6f806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fd400640 1 --2- 172.21.15.45:0/3034250172 >> v2:172.21.15.117:3300/0 conn(0x7ff6f806aea0 0x7ff6f806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fd400640 1 -- 172.21.15.45:0/3034250172 >> v1:172.21.15.117:6789/0 conn(0x7ff6f815df60 legacy=0x7ff6f80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fd400640 1 -- 172.21.15.45:0/3034250172 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f80cd060 con 0x7ff6f80d0350 2024-04-25T10:20:48.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.059+0000 7ff6fe800640 1 --2- 172.21.15.45:0/3034250172 >> v2:172.21.15.117:3300/0 conn(0x7ff6f806aea0 0x7ff6f806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:48.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff6fd400640 1 --2- 172.21.15.45:0/3034250172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f80d07b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80097c0 tx=0x7ff6e801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bedf3229633f1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff6fca00640 1 -- 172.21.15.45:0/3034250172 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80b0070 con 0x7ff6f80d0350 2024-04-25T10:20:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff6fca00640 1 -- 172.21.15.45:0/3034250172 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e80ab600 con 0x7ff6f80d0350 2024-04-25T10:20:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff6fca00640 1 -- 172.21.15.45:0/3034250172 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80ab9b0 con 0x7ff6f80d0350 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 -- 172.21.15.45:0/3034250172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 msgr2=0x7ff6f80d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 --2- 172.21.15.45:0/3034250172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f80d07b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80097c0 tx=0x7ff6e801c410 comp rx=0 tx=0).stop 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 -- 172.21.15.45:0/3034250172 shutdown_connections 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 --2- 172.21.15.45:0/3034250172 >> v2:172.21.15.117:3300/0 conn(0x7ff6f806aea0 0x7ff6f806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 --2- 172.21.15.45:0/3034250172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f80d07b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 -- 172.21.15.45:0/3034250172 >> 172.21.15.45:0/3034250172 conn(0x7ff6f80d1910 msgr2=0x7ff6f80d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 -- 172.21.15.45:0/3034250172 shutdown_connections 2024-04-25T10:20:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 -- 172.21.15.45:0/3034250172 wait complete. 2024-04-25T10:20:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 Processor -- start 2024-04-25T10:20:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.063+0000 7ff700800640 1 -- start start 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f806aea0 0x7ff6f8152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f81559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 0x7ff6f8155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80d0ff0 con 0x7ff6f815df60 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80d0e70 con 0x7ff6f806aea0 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f816aa20 con 0x7ff6f80d0350 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 0x7ff6f8155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 0x7ff6f8155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54152/0 (socket says 172.21.15.45:54152) 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 -- 172.21.15.45:0/597734390 learned_addr learned my addr 172.21.15.45:0/597734390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fde00640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f806aea0 0x7ff6f8152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fd400640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f81559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 -- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 msgr2=0x7ff6f81559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f81559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 -- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f806aea0 msgr2=0x7ff6f8152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f806aea0 0x7ff6f8152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 -- 172.21.15.45:0/597734390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f80b5e30 con 0x7ff6f815df60 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fd400640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f81559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:48.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6fe800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 0x7ff6f8155ee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009f00 tx=0x7ff6f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4018030 con 0x7ff6f815df60 2024-04-25T10:20:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f400ce50 con 0x7ff6f815df60 2024-04-25T10:20:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4003bc0 con 0x7ff6f815df60 2024-04-25T10:20:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f80b60d0 con 0x7ff6f815df60 2024-04-25T10:20:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f80b6690 con 0x7ff6f815df60 2024-04-25T10:20:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.067+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f804ef30 con 0x7ff6f815df60 2024-04-25T10:20:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.071+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6f401b070 con 0x7ff6f815df60 2024-04-25T10:20:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.071+0000 7ff6f2000640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6dc066f60 0x7ff6dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.071+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6f4085a30 con 0x7ff6f815df60 2024-04-25T10:20:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.071+0000 7ff6fde00640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6dc066f60 0x7ff6dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.071+0000 7ff6fde00640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6dc066f60 0x7ff6dc069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec007fd0 tx=0x7ff6ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.075+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6f4010040 con 0x7ff6f815df60 2024-04-25T10:20:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.235+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6f80c8780 con 0x7ff6f815df60 2024-04-25T10:20:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff6f2000640 1 -- 172.21.15.45:0/597734390 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16610 (secure 0 0 0) 0x7ff6f4056a30 con 0x7ff6f815df60 2024-04-25T10:20:48.241 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:48.242 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:47.262414+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6dc066f60 msgr2=0x7ff6dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6dc066f60 0x7ff6dc069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec007fd0 tx=0x7ff6ec009040 comp rx=0 tx=0).stop 2024-04-25T10:20:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 msgr2=0x7ff6f8155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 0x7ff6f8155ee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009f00 tx=0x7ff6f400c490 comp rx=0 tx=0).stop 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 shutdown_connections 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6dc066f60 0x7ff6dc069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6f815df60 0x7ff6f8155ee0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f80d0350 0x7ff6f81559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 --2- 172.21.15.45:0/597734390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f806aea0 0x7ff6f8152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 >> 172.21.15.45:0/597734390 conn(0x7ff6f80d1910 msgr2=0x7ff6f8068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 shutdown_connections 2024-04-25T10:20:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.239+0000 7ff700800640 1 -- 172.21.15.45:0/597734390 wait complete. 2024-04-25T10:20:48.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.399+0000 7f3dbce00640 1 Processor -- start 2024-04-25T10:20:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.399+0000 7f3dbce00640 1 -- start start 2024-04-25T10:20:48.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3dbce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3dbce00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3db815e050 0x7f3db80cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3dbce00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80d2650 con 0x7f3db815dc70 2024-04-25T10:20:48.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3dbce00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80d2370 con 0x7f3db815e050 2024-04-25T10:20:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3dbce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db806afd0 con 0x7f3db80cd160 2024-04-25T10:20:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3db815e050 0x7f3db80cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3db815e050 0x7f3db80cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54160/0 (socket says 172.21.15.45:54160) 2024-04-25T10:20:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 -- 172.21.15.45:0/2373170194 learned_addr learned my addr 172.21.15.45:0/2373170194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db6a00640 1 --2- 172.21.15.45:0/2373170194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 -- 172.21.15.45:0/2373170194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 msgr2=0x7f3db80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 --2- 172.21.15.45:0/2373170194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db80cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 -- 172.21.15.45:0/2373170194 >> v1:172.21.15.117:6789/0 conn(0x7f3db815dc70 legacy=0x7f3db806b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.403+0000 7f3db7e00640 1 -- 172.21.15.45:0/2373170194 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db806b2b0 con 0x7f3db815e050 2024-04-25T10:20:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db7e00640 1 --2- 172.21.15.45:0/2373170194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db80cfd80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009e50 tx=0x7f3db001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804a61aea57ed714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db6000640 1 -- 172.21.15.45:0/2373170194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db00a57f0 con 0x7f3db815e050 2024-04-25T10:20:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db6000640 1 -- 172.21.15.45:0/2373170194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db00a5990 con 0x7f3db815e050 2024-04-25T10:20:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db6000640 1 -- 172.21.15.45:0/2373170194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db00a5d60 con 0x7f3db815e050 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- 172.21.15.45:0/2373170194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 msgr2=0x7f3db80cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 --2- 172.21.15.45:0/2373170194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db80cfd80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009e50 tx=0x7f3db001c910 comp rx=0 tx=0).stop 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- 172.21.15.45:0/2373170194 shutdown_connections 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 --2- 172.21.15.45:0/2373170194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db80cfd80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 --2- 172.21.15.45:0/2373170194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- 172.21.15.45:0/2373170194 >> 172.21.15.45:0/2373170194 conn(0x7f3db80d1720 msgr2=0x7f3db80d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- 172.21.15.45:0/2373170194 shutdown_connections 2024-04-25T10:20:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- 172.21.15.45:0/2373170194 wait complete. 2024-04-25T10:20:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 Processor -- start 2024-04-25T10:20:48.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- start start 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db8150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db815dc70 0x7f3db80b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db8150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db816adf0 con 0x7f3db815e050 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db816ac70 con 0x7f3db815dc70 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3dbce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db816af70 con 0x7f3db80cd160 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db8150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db8150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54162/0 (socket says 172.21.15.45:54162) 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.407+0000 7f3db7e00640 1 -- 172.21.15.45:0/1609240425 learned_addr learned my addr 172.21.15.45:0/1609240425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db6a00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db815dc70 0x7f3db80b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7400640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db8150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7e00640 1 -- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 msgr2=0x7f3db8150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7e00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db8150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7e00640 1 -- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db815dc70 msgr2=0x7f3db80b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7e00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db815dc70 0x7f3db80b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7e00640 1 -- 172.21.15.45:0/1609240425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8150a60 con 0x7f3db815e050 2024-04-25T10:20:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db6a00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db815dc70 0x7f3db80b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3db7e00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db8150c60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3db001a540 tx=0x7f3db00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db00bc020 con 0x7f3db815e050 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db00b8670 con 0x7f3db815e050 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80b59c0 con 0x7f3db815e050 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db00b8a60 con 0x7f3db815e050 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db815c390 con 0x7f3db815e050 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db8075120 con 0x7f3db815e050 2024-04-25T10:20:48.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3db00bf020 con 0x7f3db815e050 2024-04-25T10:20:48.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3daf400640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d98066f60 0x7f3d98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.411+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db0120190 con 0x7f3db815e050 2024-04-25T10:20:48.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.415+0000 7f3db7400640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d98066f60 0x7f3d98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.415+0000 7f3db7400640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d98066f60 0x7f3d98069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3da80015b0 tx=0x7f3da8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.419+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3db00c6030 con 0x7f3db815e050 2024-04-25T10:20:48.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.579+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3db80d1ca0 con 0x7f3db815e050 2024-04-25T10:20:48.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3daf400640 1 -- 172.21.15.45:0/1609240425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f3db8150a60 con 0x7f3db815e050 2024-04-25T10:20:48.585 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:48.585 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:47.262414+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d98066f60 msgr2=0x7f3d98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d98066f60 0x7f3d98069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3da80015b0 tx=0x7f3da8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 msgr2=0x7f3db8150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db8150c60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3db001a540 tx=0x7f3db00196b0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 shutdown_connections 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d98066f60 0x7f3d98069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3db815e050 0x7f3db8150c60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db815dc70 0x7f3db80b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 --2- 172.21.15.45:0/1609240425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db80cd160 0x7f3db8150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 >> 172.21.15.45:0/1609240425 conn(0x7f3db80d1720 msgr2=0x7f3db806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 shutdown_connections 2024-04-25T10:20:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.583+0000 7f3dbce00640 1 -- 172.21.15.45:0/1609240425 wait complete. 2024-04-25T10:20:48.606 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:20:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 Processor -- start 2024-04-25T10:20:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 -- start start 2024-04-25T10:20:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f717c159540 0x7f717c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c0aebd0 0x7f717c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c071360 con 0x7f717c072020 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c159920 con 0x7f717c159540 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7184400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c15c2f0 con 0x7f717c0aebd0 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7181a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f717c159540 0x7f717c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7182400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f717c072020 0x7f717c15f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46198/0 (socket says 172.21.15.45:46198) 2024-04-25T10:20:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7182400640 1 -- 172.21.15.45:0/1791890218 learned_addr learned my addr 172.21.15.45:0/1791890218 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f7181000640 1 --2- 172.21.15.45:0/1791890218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c0aebd0 0x7f717c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.747+0000 7f717be00640 1 -- 172.21.15.45:0/1791890218 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190911142 0 0) 0x7f717c071360 con 0x7f717c072020 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f717be00640 1 -- 172.21.15.45:0/1791890218 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7164003530 con 0x7f717c072020 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7181a00640 1 -- 172.21.15.45:0/1791890218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c0aebd0 msgr2=0x7f717c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7181a00640 1 --2- 172.21.15.45:0/1791890218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c0aebd0 0x7f717c071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7181a00640 1 -- 172.21.15.45:0/1791890218 >> v1:172.21.15.45:6789/0 conn(0x7f717c072020 legacy=0x7f717c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7181a00640 1 -- 172.21.15.45:0/1791890218 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c15c5d0 con 0x7f717c159540 2024-04-25T10:20:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7181a00640 1 --2- 172.21.15.45:0/1791890218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c159540 0x7f717c0ae690 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71700097c0 tx=0x7f717001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27ee8ffa9ed7059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f717be00640 1 -- 172.21.15.45:0/1791890218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700b0070 con 0x7f717c159540 2024-04-25T10:20:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f717be00640 1 -- 172.21.15.45:0/1791890218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71700ab600 con 0x7f717c159540 2024-04-25T10:20:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f717be00640 1 -- 172.21.15.45:0/1791890218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700ab9d0 con 0x7f717c159540 2024-04-25T10:20:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 -- 172.21.15.45:0/1791890218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c159540 msgr2=0x7f717c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 --2- 172.21.15.45:0/1791890218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c159540 0x7f717c0ae690 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71700097c0 tx=0x7f717001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 -- 172.21.15.45:0/1791890218 shutdown_connections 2024-04-25T10:20:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 --2- 172.21.15.45:0/1791890218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c0aebd0 0x7f717c071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 --2- 172.21.15.45:0/1791890218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c159540 0x7f717c0ae690 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 -- 172.21.15.45:0/1791890218 >> 172.21.15.45:0/1791890218 conn(0x7f717c0669a0 msgr2=0x7f717c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 -- 172.21.15.45:0/1791890218 shutdown_connections 2024-04-25T10:20:48.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.751+0000 7f7184400640 1 -- 172.21.15.45:0/1791890218 wait complete. 2024-04-25T10:20:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 Processor -- start 2024-04-25T10:20:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 -- start start 2024-04-25T10:20:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717c072020 0x7f717c155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 0x7f717c142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c159540 0x7f717c1fa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c072430 con 0x7f717c0aebd0 2024-04-25T10:20:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c06c0d0 con 0x7f717c072020 2024-04-25T10:20:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7184400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c168930 con 0x7f717c159540 2024-04-25T10:20:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 0x7f717c142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 0x7f717c142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54174/0 (socket says 172.21.15.45:54174) 2024-04-25T10:20:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 -- 172.21.15.45:0/2525308699 learned_addr learned my addr 172.21.15.45:0/2525308699 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181a00640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717c072020 0x7f717c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 -- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c159540 msgr2=0x7f717c1fa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c159540 0x7f717c1fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 -- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717c072020 msgr2=0x7f717c155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717c072020 0x7f717c155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 -- 172.21.15.45:0/2525308699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c1429a0 con 0x7f717c0aebd0 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.755+0000 7f7181000640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 0x7f717c142460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f716800e8a0 tx=0x7f716800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168017070 con 0x7f717c0aebd0 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800c940 con 0x7f717c0aebd0 2024-04-25T10:20:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800ccf0 con 0x7f717c0aebd0 2024-04-25T10:20:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c1fac50 con 0x7f717c0aebd0 2024-04-25T10:20:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c1fb210 con 0x7f717c0aebd0 2024-04-25T10:20:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c04ef30 con 0x7f717c0aebd0 2024-04-25T10:20:48.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f716801a070 con 0x7f717c0aebd0 2024-04-25T10:20:48.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7179600640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f715c066f60 0x7f715c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7168084f40 con 0x7f717c0aebd0 2024-04-25T10:20:48.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.759+0000 7f7181a00640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f715c066f60 0x7f715c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.763+0000 7f7181a00640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f715c066f60 0x7f715c069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f71700ad000 tx=0x7f7170008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.763+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7168055f40 con 0x7f717c0aebd0 2024-04-25T10:20:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:48.967+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f717c166700 con 0x7f717c0aebd0 2024-04-25T10:20:49.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:20:49.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:20:49.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:20:49.847 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:20:49.842+0000 7faef2000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:20:49.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7179600640 1 -- 172.21.15.45:0/2525308699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7168055840 con 0x7f717c0aebd0 2024-04-25T10:20:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f715c066f60 msgr2=0x7f715c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f715c066f60 0x7f715c069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f71700ad000 tx=0x7f7170008040 comp rx=0 tx=0).stop 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 msgr2=0x7f717c142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 0x7f717c142460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f716800e8a0 tx=0x7f716800eca0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 shutdown_connections 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f715c066f60 0x7f715c069420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f717c159540 0x7f717c1fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f717c0aebd0 0x7f717c142460 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 --2- 172.21.15.45:0/2525308699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f717c072020 0x7f717c155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 >> 172.21.15.45:0/2525308699 conn(0x7f717c0669a0 msgr2=0x7f717c15e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 shutdown_connections 2024-04-25T10:20:49.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:49.879+0000 7f7184400640 1 -- 172.21.15.45:0/2525308699 wait complete. 2024-04-25T10:20:49.907 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:20:49.907 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:20:50.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 Processor -- start 2024-04-25T10:20:50.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 -- start start 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9be80cfb30 0x7f9be80d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80ce220 0x7f9be80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be806b470 con 0x7f9be815dd30 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be815e110 con 0x7f9be80cfb30 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9befe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80b2940 con 0x7f9be80ce220 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9beca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9be80cfb30 0x7f9be80d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9bed400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9be815dd30 0x7f9be806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46200/0 (socket says 172.21.15.45:46200) 2024-04-25T10:20:50.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.067+0000 7f9bed400640 1 -- 172.21.15.45:0/150823418 learned_addr learned my addr 172.21.15.45:0/150823418 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9bede00640 1 --2- 172.21.15.45:0/150823418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80ce220 0x7f9be80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9beca00640 1 -- 172.21.15.45:0/150823418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80ce220 msgr2=0x7f9be80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9beca00640 1 --2- 172.21.15.45:0/150823418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80ce220 0x7f9be80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9beca00640 1 -- 172.21.15.45:0/150823418 >> v1:172.21.15.45:6789/0 conn(0x7f9be815dd30 legacy=0x7f9be806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9beca00640 1 -- 172.21.15.45:0/150823418 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be806b750 con 0x7f9be80cfb30 2024-04-25T10:20:50.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9be7e00640 1 -- 172.21.15.45:0/150823418 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267118126 0 0) 0x7f9be806b470 con 0x7f9be815dd30 2024-04-25T10:20:50.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9beca00640 1 --2- 172.21.15.45:0/150823418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80cfb30 0x7f9be80d18a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097c0 tx=0x7f9bd401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a2519ea6b26961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9be7e00640 1 -- 172.21.15.45:0/150823418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40b2070 con 0x7f9be80cfb30 2024-04-25T10:20:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9be7e00640 1 -- 172.21.15.45:0/150823418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40ad640 con 0x7f9be80cfb30 2024-04-25T10:20:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9be7e00640 1 -- 172.21.15.45:0/150823418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40ada10 con 0x7f9be80cfb30 2024-04-25T10:20:50.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9befe00640 1 -- 172.21.15.45:0/150823418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80cfb30 msgr2=0x7f9be80d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.071+0000 7f9befe00640 1 --2- 172.21.15.45:0/150823418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80cfb30 0x7f9be80d18a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097c0 tx=0x7f9bd401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- 172.21.15.45:0/150823418 shutdown_connections 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 --2- 172.21.15.45:0/150823418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80ce220 0x7f9be80d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 --2- 172.21.15.45:0/150823418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80cfb30 0x7f9be80d18a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- 172.21.15.45:0/150823418 >> 172.21.15.45:0/150823418 conn(0x7f9be809a9d0 msgr2=0x7f9be809adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- 172.21.15.45:0/150823418 shutdown_connections 2024-04-25T10:20:50.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- 172.21.15.45:0/150823418 wait complete. 2024-04-25T10:20:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 Processor -- start 2024-04-25T10:20:50.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- start start 2024-04-25T10:20:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 0x7f9be814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80cfb30 0x7f9be815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be815dd30 0x7f9be806f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80b2510 con 0x7f9be80ce220 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80b2390 con 0x7f9be815dd30 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9befe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80b2690 con 0x7f9be80cfb30 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 0x7f9be814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 0x7f9be814ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54202/0 (socket says 172.21.15.45:54202) 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 -- 172.21.15.45:0/2331373127 learned_addr learned my addr 172.21.15.45:0/2331373127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bede00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be815dd30 0x7f9be806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9beca00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80cfb30 0x7f9be815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 -- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80cfb30 msgr2=0x7f9be815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80cfb30 0x7f9be815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 -- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be815dd30 msgr2=0x7f9be806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be815dd30 0x7f9be806f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 -- 172.21.15.45:0/2331373127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80af350 con 0x7f9be80ce220 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9bed400640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 0x7f9be814ca20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0011f0 tx=0x7f9bdc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.075+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc018070 con 0x7f9be80ce220 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc0074e0 con 0x7f9be80ce220 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be80af630 con 0x7f9be80ce220 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc004a10 con 0x7f9be80ce220 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be80afbf0 con 0x7f9be80ce220 2024-04-25T10:20:50.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be804ef30 con 0x7f9be80ce220 2024-04-25T10:20:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bdc01b070 con 0x7f9be80ce220 2024-04-25T10:20:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9be5600640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc8066f60 0x7f9bc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bdc084490 con 0x7f9be80ce220 2024-04-25T10:20:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.079+0000 7f9beca00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc8066f60 0x7f9bc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.083+0000 7f9beca00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc8066f60 0x7f9bc8069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40af000 tx=0x7f9bd4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.087+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bdc055400 con 0x7f9be80ce220 2024-04-25T10:20:50.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.283+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9be80b7920 con 0x7f9be80ce220 2024-04-25T10:20:50.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.879+0000 7f9be5600640 1 -- 172.21.15.45:0/2331373127 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9bdc054d00 con 0x7f9be80ce220 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc8066f60 msgr2=0x7f9bc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc8066f60 0x7f9bc8069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40af000 tx=0x7f9bd4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 msgr2=0x7f9be814ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 0x7f9be814ca20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0011f0 tx=0x7f9bdc00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 shutdown_connections 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc8066f60 0x7f9bc8069420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be815dd30 0x7f9be806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be80cfb30 0x7f9be815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 --2- 172.21.15.45:0/2331373127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be80ce220 0x7f9be814ca20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 >> 172.21.15.45:0/2331373127 conn(0x7f9be809a9d0 msgr2=0x7f9be80cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 shutdown_connections 2024-04-25T10:20:50.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:50.883+0000 7f9befe00640 1 -- 172.21.15.45:0/2331373127 wait complete. 2024-04-25T10:20:50.908 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:20:50.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.047+0000 7feb0b600640 1 Processor -- start 2024-04-25T10:20:51.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.047+0000 7feb0b600640 1 -- start start 2024-04-25T10:20:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb0b600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb0415a0e0 0x7feb0415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb0b600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb040b3190 0x7feb0406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb0b600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0415b6e0 con 0x7feb04071f70 2024-04-25T10:20:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb0b600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb04070ed0 con 0x7feb0415a0e0 2024-04-25T10:20:51.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb0b600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb040711b0 con 0x7feb040b3190 2024-04-25T10:20:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb03e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feb04071f70 0x7feb040b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46206/0 (socket says 172.21.15.45:46206) 2024-04-25T10:20:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb03e00640 1 -- 172.21.15.45:0/845092207 learned_addr learned my addr 172.21.15.45:0/845092207 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb08c00640 1 --2- 172.21.15.45:0/845092207 >> v2:172.21.15.45:3300/0 conn(0x7feb0415a0e0 0x7feb0415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb09600640 1 --2- 172.21.15.45:0/845092207 >> v2:172.21.15.117:3300/0 conn(0x7feb040b3190 0x7feb0406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb03400640 1 -- 172.21.15.45:0/845092207 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293736113 0 0) 0x7feaf0001120 con 0x7feb04071f70 2024-04-25T10:20:51.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb03400640 1 -- 172.21.15.45:0/845092207 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaec005740 con 0x7feb04071f70 2024-04-25T10:20:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb08c00640 1 -- 172.21.15.45:0/845092207 >> v2:172.21.15.117:3300/0 conn(0x7feb040b3190 msgr2=0x7feb0406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb08c00640 1 --2- 172.21.15.45:0/845092207 >> v2:172.21.15.117:3300/0 conn(0x7feb040b3190 0x7feb0406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb08c00640 1 -- 172.21.15.45:0/845092207 >> v1:172.21.15.45:6789/0 conn(0x7feb04071f70 legacy=0x7feb040b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb08c00640 1 -- 172.21.15.45:0/845092207 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04071490 con 0x7feb0415a0e0 2024-04-25T10:20:51.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.051+0000 7feb09600640 1 --2- 172.21.15.45:0/845092207 >> v2:172.21.15.117:3300/0 conn(0x7feb040b3190 0x7feb0406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb08c00640 1 --2- 172.21.15.45:0/845092207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0415a0e0 0x7feb0415b1a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feb040d0670 tx=0x7feaf801c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c417261707322147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb03400640 1 -- 172.21.15.45:0/845092207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf800b040 con 0x7feb0415a0e0 2024-04-25T10:20:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb03400640 1 -- 172.21.15.45:0/845092207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80a66e0 con 0x7feb0415a0e0 2024-04-25T10:20:51.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb03400640 1 -- 172.21.15.45:0/845092207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80a6a90 con 0x7feb0415a0e0 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- 172.21.15.45:0/845092207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0415a0e0 msgr2=0x7feb0415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 --2- 172.21.15.45:0/845092207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0415a0e0 0x7feb0415b1a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feb040d0670 tx=0x7feaf801c690 comp rx=0 tx=0).stop 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- 172.21.15.45:0/845092207 shutdown_connections 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 --2- 172.21.15.45:0/845092207 >> v2:172.21.15.117:3300/0 conn(0x7feb040b3190 0x7feb0406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 --2- 172.21.15.45:0/845092207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0415a0e0 0x7feb0415b1a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- 172.21.15.45:0/845092207 >> 172.21.15.45:0/845092207 conn(0x7feb04066b70 msgr2=0x7feb04066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- 172.21.15.45:0/845092207 shutdown_connections 2024-04-25T10:20:51.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- 172.21.15.45:0/845092207 wait complete. 2024-04-25T10:20:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 Processor -- start 2024-04-25T10:20:51.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- start start 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 0x7feb04157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040b3190 0x7feb0414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415a0e0 0x7feb0414bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb04070b50 con 0x7feb04071f70 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415a4c0 con 0x7feb0415a0e0 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.055+0000 7feb0b600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb04072370 con 0x7feb040b3190 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 0x7feb04157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 0x7feb04157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54212/0 (socket says 172.21.15.45:54212) 2024-04-25T10:20:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 -- 172.21.15.45:0/3958963574 learned_addr learned my addr 172.21.15.45:0/3958963574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 -- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040b3190 msgr2=0x7feb0414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040b3190 0x7feb0414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 -- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415a0e0 msgr2=0x7feb0414bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415a0e0 0x7feb0414bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 -- 172.21.15.45:0/3958963574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04158220 con 0x7feb04071f70 2024-04-25T10:20:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb08c00640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 0x7feb04157ce0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feaf80b0040 tx=0x7feaf8019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80ba680 con 0x7feb04071f70 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80c0070 con 0x7feb04071f70 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb041584f0 con 0x7feb04071f70 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80baa20 con 0x7feb04071f70 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04158ab0 con 0x7feb04071f70 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0404ef30 con 0x7feb04071f70 2024-04-25T10:20:51.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.059+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaf80babc0 con 0x7feb04071f70 2024-04-25T10:20:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.063+0000 7feb00c00640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.063+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaf81203c0 con 0x7feb04071f70 2024-04-25T10:20:51.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.063+0000 7feb03e00640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.063+0000 7feb03e00640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000f70 tx=0x7feaf0001120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.067+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf8016340 con 0x7feb04071f70 2024-04-25T10:20:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.259+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb040703e0 con 0x7feb04071f70 2024-04-25T10:20:51.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb00c00640 1 -- 172.21.15.45:0/3958963574 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7feaf80f1330 con 0x7feb04071f70 2024-04-25T10:20:51.265 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:51.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:20:51.266 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:20:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 msgr2=0x7feae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000f70 tx=0x7feaf0001120 comp rx=0 tx=0).stop 2024-04-25T10:20:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 msgr2=0x7feb04157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 0x7feb04157ce0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feaf80b0040 tx=0x7feaf8019430 comp rx=0 tx=0).stop 2024-04-25T10:20:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 shutdown_connections 2024-04-25T10:20:51.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415a0e0 0x7feb0414bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040b3190 0x7feb0414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 --2- 172.21.15.45:0/3958963574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb04071f70 0x7feb04157ce0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 >> 172.21.15.45:0/3958963574 conn(0x7feb04066b70 msgr2=0x7feb040ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 shutdown_connections 2024-04-25T10:20:51.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.263+0000 7feb0b600640 1 -- 172.21.15.45:0/3958963574 wait complete. 2024-04-25T10:20:51.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:51.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.423+0000 7f83ea400640 1 Processor -- start 2024-04-25T10:20:51.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.423+0000 7f83ea400640 1 -- start start 2024-04-25T10:20:51.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83ea400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83e40cfd90 0x7f83e40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83ea400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e415f3b0 con 0x7f83e40cd110 2024-04-25T10:20:51.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83ea400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e40ca810 con 0x7f83e40b0f70 2024-04-25T10:20:51.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83ea400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e415ef20 con 0x7f83e40cfd90 2024-04-25T10:20:51.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f83e40cd110 0x7f83e40cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46212/0 (socket says 172.21.15.45:46212) 2024-04-25T10:20:51.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e3e00640 1 -- 172.21.15.45:0/1470216937 learned_addr learned my addr 172.21.15.45:0/1470216937 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 --2- 172.21.15.45:0/1470216937 >> v2:172.21.15.117:3300/0 conn(0x7f83e40cfd90 0x7f83e40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603847478 0 0) 0x7f83e415f3b0 con 0x7f83e40cd110 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c8003530 con 0x7f83e40cd110 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225312526 0 0) 0x7f83e40ca810 con 0x7f83e40b0f70 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e415f3b0 con 0x7f83e40b0f70 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 379351600 0 0) 0x7f83c8003530 con 0x7f83e40cd110 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 >> v2:172.21.15.117:3300/0 conn(0x7f83e40cfd90 msgr2=0x7f83e40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 --2- 172.21.15.45:0/1470216937 >> v2:172.21.15.117:3300/0 conn(0x7f83e40cfd90 0x7f83e40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 >> v1:172.21.15.117:6789/0 conn(0x7f83e40b0f70 legacy=0x7f83e40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e415e5f0 con 0x7f83e40cd110 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f83dc001680 con 0x7f83e40cd110 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 --2- 172.21.15.45:0/1470216937 >> v2:172.21.15.117:3300/0 conn(0x7f83e40cfd90 0x7f83e40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:51.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 >> v1:172.21.15.45:6789/0 conn(0x7f83e40cd110 legacy=0x7f83e40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8003ba0 0x7f83c8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e415ef20 con 0x7f83c8004810 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c800c0d0 con 0x7f83c8003ba0 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c800c2d0 con 0x7f83c80051d0 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e3400640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8003ba0 0x7f83c8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e3e00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 -- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c80051d0 msgr2=0x7f83c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 -- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8003ba0 msgr2=0x7f83c8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83c8003ba0 0x7f83c8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83daa00640 1 -- 172.21.15.45:0/1470216937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8003530 con 0x7f83c8004810 2024-04-25T10:20:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.427+0000 7f83e3e00640 1 -- 172.21.15.45:0/1470216937 reap_dead start 2024-04-25T10:20:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83daa00640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e350 tx=0x7f83cc01ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e86b7d30326c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0b2070 con 0x7f83c8004810 2024-04-25T10:20:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc0a5e30 con 0x7f83c8004810 2024-04-25T10:20:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83e2a00640 1 -- 172.21.15.45:0/1470216937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc009de0 con 0x7f83c8004810 2024-04-25T10:20:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 -- 172.21.15.45:0/1470216937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8004810 msgr2=0x7f83c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e350 tx=0x7f83cc01ce20 comp rx=0 tx=0).stop 2024-04-25T10:20:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 -- 172.21.15.45:0/1470216937 shutdown_connections 2024-04-25T10:20:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 --2- 172.21.15.45:0/1470216937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 -- 172.21.15.45:0/1470216937 >> 172.21.15.45:0/1470216937 conn(0x7f83e40b1d50 msgr2=0x7f83e40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 -- 172.21.15.45:0/1470216937 shutdown_connections 2024-04-25T10:20:51.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.431+0000 7f83ea400640 1 -- 172.21.15.45:0/1470216937 wait complete. 2024-04-25T10:20:51.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 Processor -- start 2024-04-25T10:20:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- start start 2024-04-25T10:20:51.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c8004810 0x7f83e40afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 0x7f83e41f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e40727a0 0x7f83e4072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40cd520 con 0x7f83e4099980 2024-04-25T10:20:51.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e415ebb0 con 0x7f83e40727a0 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40ccbd0 con 0x7f83c8004810 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 0x7f83e41f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 0x7f83e41f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54234/0 (socket says 172.21.15.45:54234) 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 -- 172.21.15.45:0/2290981358 learned_addr learned my addr 172.21.15.45:0/2290981358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83e3e00640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e40727a0 0x7f83e4072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83e3400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c8004810 0x7f83e40afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 -- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c8004810 msgr2=0x7f83e40afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c8004810 0x7f83e40afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 -- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e40727a0 msgr2=0x7f83e4072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e40727a0 0x7f83e4072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 -- 172.21.15.45:0/2290981358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e41f7d80 con 0x7f83e4099980 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83e3e00640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e40727a0 0x7f83e4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83daa00640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 0x7f83e41f7610 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e350 tx=0x7f83cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0b2040 con 0x7f83e4099980 2024-04-25T10:20:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc0a5c30 con 0x7f83e4099980 2024-04-25T10:20:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc009de0 con 0x7f83e4099980 2024-04-25T10:20:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e41f8060 con 0x7f83e4099980 2024-04-25T10:20:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e4099e00 con 0x7f83e4099980 2024-04-25T10:20:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.435+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e41f8580 con 0x7f83e4099980 2024-04-25T10:20:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.439+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83cc0bd020 con 0x7f83e4099980 2024-04-25T10:20:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.439+0000 7f83dbe00640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83c0066f60 0x7f83c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.439+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83cc11fc20 con 0x7f83e4099980 2024-04-25T10:20:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.439+0000 7f83e3400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83c0066f60 0x7f83c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.439+0000 7f83e3400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83c0066f60 0x7f83c0069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f83d40079b0 tx=0x7f83d4007940 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.443+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83e41f8580 con 0x7f83e4099980 2024-04-25T10:20:51.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.603+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83e404ef30 con 0x7f83e4099980 2024-04-25T10:20:51.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83dbe00640 1 -- 172.21.15.45:0/2290981358 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f83cc016b30 con 0x7f83e4099980 2024-04-25T10:20:51.610 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:51.610 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":50,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:48.857409+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83c0066f60 msgr2=0x7f83c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83c0066f60 0x7f83c0069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f83d40079b0 tx=0x7f83d4007940 comp rx=0 tx=0).stop 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 msgr2=0x7f83e41f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 0x7f83e41f7610 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e350 tx=0x7f83cc009750 comp rx=0 tx=0).stop 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 shutdown_connections 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83c0066f60 0x7f83c0069420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e40727a0 0x7f83e4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83e4099980 0x7f83e41f7610 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 --2- 172.21.15.45:0/2290981358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83c8004810 0x7f83e40afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 >> 172.21.15.45:0/2290981358 conn(0x7f83e40b1d50 msgr2=0x7f83e40caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 shutdown_connections 2024-04-25T10:20:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.607+0000 7f83ea400640 1 -- 172.21.15.45:0/2290981358 wait complete. 2024-04-25T10:20:51.631 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:20:51.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:51.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 Processor -- start 2024-04-25T10:20:51.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 -- start start 2024-04-25T10:20:51.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efdc80d0350 0x7efdc80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc806aea0 0x7efdc806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc80ccaa0 con 0x7efdc815df60 2024-04-25T10:20:51.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc815e340 con 0x7efdc80d0350 2024-04-25T10:20:51.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80ccd80 con 0x7efdc806aea0 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdc7e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efdc80d0350 0x7efdc80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcd800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc806aea0 0x7efdc806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcce00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7efdc815df60 0x7efdc80cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36892/0 (socket says 172.21.15.45:36892) 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.767+0000 7efdcce00640 1 -- 172.21.15.45:0/1820100560 learned_addr learned my addr 172.21.15.45:0/1820100560 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7e00640 1 -- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc806aea0 msgr2=0x7efdc806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7e00640 1 --2- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc806aea0 0x7efdc806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7e00640 1 -- 172.21.15.45:0/1820100560 >> v1:172.21.15.117:6789/0 conn(0x7efdc815df60 legacy=0x7efdc80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7e00640 1 -- 172.21.15.45:0/1820100560 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc80cd060 con 0x7efdc80d0350 2024-04-25T10:20:51.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7e00640 1 --2- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc80d07b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efdb40097c0 tx=0x7efdb401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae75e6b5c326511b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7400640 1 -- 172.21.15.45:0/1820100560 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40b0070 con 0x7efdc80d0350 2024-04-25T10:20:51.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7400640 1 -- 172.21.15.45:0/1820100560 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb40ab600 con 0x7efdc80d0350 2024-04-25T10:20:51.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.771+0000 7efdc7400640 1 -- 172.21.15.45:0/1820100560 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40ab9d0 con 0x7efdc80d0350 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 msgr2=0x7efdc80d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 --2- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc80d07b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efdb40097c0 tx=0x7efdb401c410 comp rx=0 tx=0).stop 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- 172.21.15.45:0/1820100560 shutdown_connections 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 --2- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc806aea0 0x7efdc806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 --2- 172.21.15.45:0/1820100560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc80d07b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- 172.21.15.45:0/1820100560 >> 172.21.15.45:0/1820100560 conn(0x7efdc80d1910 msgr2=0x7efdc80d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- 172.21.15.45:0/1820100560 shutdown_connections 2024-04-25T10:20:51.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- 172.21.15.45:0/1820100560 wait complete. 2024-04-25T10:20:51.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 Processor -- start 2024-04-25T10:20:51.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- start start 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 0x7efdc80b6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc8153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc815df60 0x7efdc8151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80d0ff0 con 0x7efdc806aea0 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc80d0e70 con 0x7efdc80d0350 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc816aa20 con 0x7efdc815df60 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 0x7efdc80b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 0x7efdc80b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54236/0 (socket says 172.21.15.45:54236) 2024-04-25T10:20:51.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 -- 172.21.15.45:0/3719614713 learned_addr learned my addr 172.21.15.45:0/3719614713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdc7e00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc8153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcd800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc815df60 0x7efdc8151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 -- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc815df60 msgr2=0x7efdc8151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc815df60 0x7efdc8151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 -- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 msgr2=0x7efdc8153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc8153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 -- 172.21.15.45:0/3719614713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc8158150 con 0x7efdc806aea0 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdc7e00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc8153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:51.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.775+0000 7efdcce00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 0x7efdc80b6930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c420 tx=0x7efdbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc017070 con 0x7efdc806aea0 2024-04-25T10:20:51.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc005cf0 con 0x7efdc806aea0 2024-04-25T10:20:51.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0072f0 con 0x7efdc806aea0 2024-04-25T10:20:51.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc8158430 con 0x7efdc806aea0 2024-04-25T10:20:51.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc81589f0 con 0x7efdc806aea0 2024-04-25T10:20:51.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc804ef30 con 0x7efdc806aea0 2024-04-25T10:20:51.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efdbc01a070 con 0x7efdc806aea0 2024-04-25T10:20:51.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc4c00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efda8066f60 0x7efda8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdbc084490 con 0x7efdc806aea0 2024-04-25T10:20:51.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.779+0000 7efdc7e00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efda8066f60 0x7efda8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.783+0000 7efdc7e00640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efda8066f60 0x7efda8069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efdb40ad000 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.783+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efdbc055400 con 0x7efdc806aea0 2024-04-25T10:20:51.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.947+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdc8158dc0 con 0x7efda8066f60 2024-04-25T10:20:51.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.955+0000 7efdc4c00640 1 -- 172.21.15.45:0/3719614713 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251926 (secure 0 0 0) 0x7efdc8158dc0 con 0x7efda8066f60 2024-04-25T10:20:51.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:51.962 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efda8066f60 msgr2=0x7efda8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efda8066f60 0x7efda8069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efdb40ad000 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 msgr2=0x7efdc80b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 0x7efdc80b6930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00c420 tx=0x7efdbc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 shutdown_connections 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efda8066f60 0x7efda8069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdc815df60 0x7efdc8151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdc80d0350 0x7efdc8153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 --2- 172.21.15.45:0/3719614713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc806aea0 0x7efdc80b6930 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 >> 172.21.15.45:0/3719614713 conn(0x7efdc80d1910 msgr2=0x7efdc8156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 shutdown_connections 2024-04-25T10:20:51.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:51.959+0000 7efdcf800640 1 -- 172.21.15.45:0/3719614713 wait complete. 2024-04-25T10:20:51.980 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-04-25T10:20:50.910837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":309192,"kb_used_data":5652,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754665528,"statfs":{"total":773094113280,"available":772777500672,"internally_reserved":0,"allocated":5787648,"data_stored":3051332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.059486"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.864148+0000","last_change":"2024-04-25T10:20:47.877059+0000","last_active":"2024-04-25T10:20:48.864148+0000","last_peered":"2024-04-25T10:20:48.864148+0000","last_clean":"2024-04-25T10:20:48.864148+0000","last_became_active":"2024-04-25T10:20:47.876332+0000","last_became_peered":"2024-04-25T10:20:47.876332+0000","last_unstale":"2024-04-25T10:20:48.864148+0000","last_undegraded":"2024-04-25T10:20:48.864148+0000","last_fullsized":"2024-04-25T10:20:48.864148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.194709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.875210+0000","last_change":"2024-04-25T10:20:47.871724+0000","last_active":"2024-04-25T10:20:47.875210+0000","last_peered":"2024-04-25T10:20:47.875210+0000","last_clean":"2024-04-25T10:20:47.875210+0000","last_became_active":"2024-04-25T10:20:47.871378+0000","last_became_peered":"2024-04-25T10:20:47.871378+0000","last_unstale":"2024-04-25T10:20:47.875210+0000","last_undegraded":"2024-04-25T10:20:47.875210+0000","last_fullsized":"2024-04-25T10:20:47.875210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:53.489221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862289+0000","last_change":"2024-04-25T10:20:47.874404+0000","last_active":"2024-04-25T10:20:48.862289+0000","last_peered":"2024-04-25T10:20:48.862289+0000","last_clean":"2024-04-25T10:20:48.862289+0000","last_became_active":"2024-04-25T10:20:47.874025+0000","last_became_peered":"2024-04-25T10:20:47.874025+0000","last_unstale":"2024-04-25T10:20:48.862289+0000","last_undegraded":"2024-04-25T10:20:48.862289+0000","last_fullsized":"2024-04-25T10:20:48.862289+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:17.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862692+0000","last_change":"2024-04-25T10:20:47.885775+0000","last_active":"2024-04-25T10:20:48.862692+0000","last_peered":"2024-04-25T10:20:48.862692+0000","last_clean":"2024-04-25T10:20:48.862692+0000","last_became_active":"2024-04-25T10:20:47.885485+0000","last_became_peered":"2024-04-25T10:20:47.885485+0000","last_unstale":"2024-04-25T10:20:48.862692+0000","last_undegraded":"2024-04-25T10:20:48.862692+0000","last_fullsized":"2024-04-25T10:20:48.862692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:28.459527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863362+0000","last_change":"2024-04-25T10:20:47.884354+0000","last_active":"2024-04-25T10:20:48.863362+0000","last_peered":"2024-04-25T10:20:48.863362+0000","last_clean":"2024-04-25T10:20:48.863362+0000","last_became_active":"2024-04-25T10:20:47.884081+0000","last_became_peered":"2024-04-25T10:20:47.884081+0000","last_unstale":"2024-04-25T10:20:48.863362+0000","last_undegraded":"2024-04-25T10:20:48.863362+0000","last_fullsized":"2024-04-25T10:20:48.863362+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.891569+0000","last_change":"2024-04-25T10:20:47.888754+0000","last_active":"2024-04-25T10:20:47.891569+0000","last_peered":"2024-04-25T10:20:47.891569+0000","last_clean":"2024-04-25T10:20:47.891569+0000","last_became_active":"2024-04-25T10:20:47.888503+0000","last_became_peered":"2024-04-25T10:20:47.888503+0000","last_unstale":"2024-04-25T10:20:47.891569+0000","last_undegraded":"2024-04-25T10:20:47.891569+0000","last_fullsized":"2024-04-25T10:20:47.891569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:53.489221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883054+0000","last_change":"2024-04-25T10:20:47.882480+0000","last_active":"2024-04-25T10:20:48.883054+0000","last_peered":"2024-04-25T10:20:48.883054+0000","last_clean":"2024-04-25T10:20:48.883054+0000","last_became_active":"2024-04-25T10:20:47.882213+0000","last_became_peered":"2024-04-25T10:20:47.882213+0000","last_unstale":"2024-04-25T10:20:48.883054+0000","last_undegraded":"2024-04-25T10:20:48.883054+0000","last_fullsized":"2024-04-25T10:20:48.883054+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:29.153863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862160+0000","last_change":"2024-04-25T10:20:47.875265+0000","last_active":"2024-04-25T10:20:48.862160+0000","last_peered":"2024-04-25T10:20:48.862160+0000","last_clean":"2024-04-25T10:20:48.862160+0000","last_became_active":"2024-04-25T10:20:47.874978+0000","last_became_peered":"2024-04-25T10:20:47.874978+0000","last_unstale":"2024-04-25T10:20:48.862160+0000","last_undegraded":"2024-04-25T10:20:48.862160+0000","last_fullsized":"2024-04-25T10:20:48.862160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:54.014411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.891283+0000","last_change":"2024-04-25T10:20:47.887995+0000","last_active":"2024-04-25T10:20:47.891283+0000","last_peered":"2024-04-25T10:20:47.891283+0000","last_clean":"2024-04-25T10:20:47.891283+0000","last_became_active":"2024-04-25T10:20:47.887742+0000","last_became_peered":"2024-04-25T10:20:47.887742+0000","last_unstale":"2024-04-25T10:20:47.891283+0000","last_undegraded":"2024-04-25T10:20:47.891283+0000","last_fullsized":"2024-04-25T10:20:47.891283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:07.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.872605+0000","last_change":"2024-04-25T10:20:47.868855+0000","last_active":"2024-04-25T10:20:47.872605+0000","last_peered":"2024-04-25T10:20:47.872605+0000","last_clean":"2024-04-25T10:20:47.872605+0000","last_became_active":"2024-04-25T10:20:47.868484+0000","last_became_peered":"2024-04-25T10:20:47.868484+0000","last_unstale":"2024-04-25T10:20:47.872605+0000","last_undegraded":"2024-04-25T10:20:47.872605+0000","last_fullsized":"2024-04-25T10:20:47.872605+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:16.491024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863053+0000","last_change":"2024-04-25T10:20:47.891178+0000","last_active":"2024-04-25T10:20:48.863053+0000","last_peered":"2024-04-25T10:20:48.863053+0000","last_clean":"2024-04-25T10:20:48.863053+0000","last_became_active":"2024-04-25T10:20:47.890900+0000","last_became_peered":"2024-04-25T10:20:47.890900+0000","last_unstale":"2024-04-25T10:20:48.863053+0000","last_undegraded":"2024-04-25T10:20:48.863053+0000","last_fullsized":"2024-04-25T10:20:48.863053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.412783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888595+0000","last_change":"2024-04-25T10:20:47.879551+0000","last_active":"2024-04-25T10:20:47.888595+0000","last_peered":"2024-04-25T10:20:47.888595+0000","last_clean":"2024-04-25T10:20:47.888595+0000","last_became_active":"2024-04-25T10:20:47.879282+0000","last_became_peered":"2024-04-25T10:20:47.879282+0000","last_unstale":"2024-04-25T10:20:47.888595+0000","last_undegraded":"2024-04-25T10:20:47.888595+0000","last_fullsized":"2024-04-25T10:20:47.888595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:57.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.891128+0000","last_change":"2024-04-25T10:20:47.886687+0000","last_active":"2024-04-25T10:20:47.891128+0000","last_peered":"2024-04-25T10:20:47.891128+0000","last_clean":"2024-04-25T10:20:47.891128+0000","last_became_active":"2024-04-25T10:20:47.884821+0000","last_became_peered":"2024-04-25T10:20:47.884821+0000","last_unstale":"2024-04-25T10:20:47.891128+0000","last_undegraded":"2024-04-25T10:20:47.891128+0000","last_fullsized":"2024-04-25T10:20:47.891128+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:04.188422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862690+0000","last_change":"2024-04-25T10:20:47.875438+0000","last_active":"2024-04-25T10:20:48.862690+0000","last_peered":"2024-04-25T10:20:48.862690+0000","last_clean":"2024-04-25T10:20:48.862690+0000","last_became_active":"2024-04-25T10:20:47.874535+0000","last_became_peered":"2024-04-25T10:20:47.874535+0000","last_unstale":"2024-04-25T10:20:48.862690+0000","last_undegraded":"2024-04-25T10:20:48.862690+0000","last_fullsized":"2024-04-25T10:20:48.862690+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:24.308428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862180+0000","last_change":"2024-04-25T10:20:47.871234+0000","last_active":"2024-04-25T10:20:48.862180+0000","last_peered":"2024-04-25T10:20:48.862180+0000","last_clean":"2024-04-25T10:20:48.862180+0000","last_became_active":"2024-04-25T10:20:47.870909+0000","last_became_peered":"2024-04-25T10:20:47.870909+0000","last_unstale":"2024-04-25T10:20:48.862180+0000","last_undegraded":"2024-04-25T10:20:48.862180+0000","last_fullsized":"2024-04-25T10:20:48.862180+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:20.528597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863258+0000","last_change":"2024-04-25T10:20:47.886290+0000","last_active":"2024-04-25T10:20:48.863258+0000","last_peered":"2024-04-25T10:20:48.863258+0000","last_clean":"2024-04-25T10:20:48.863258+0000","last_became_active":"2024-04-25T10:20:47.885778+0000","last_became_peered":"2024-04-25T10:20:47.885778+0000","last_unstale":"2024-04-25T10:20:48.863258+0000","last_undegraded":"2024-04-25T10:20:48.863258+0000","last_fullsized":"2024-04-25T10:20:48.863258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:40.110717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888566+0000","last_change":"2024-04-25T10:20:47.878579+0000","last_active":"2024-04-25T10:20:47.888566+0000","last_peered":"2024-04-25T10:20:47.888566+0000","last_clean":"2024-04-25T10:20:47.888566+0000","last_became_active":"2024-04-25T10:20:47.877877+0000","last_became_peered":"2024-04-25T10:20:47.877877+0000","last_unstale":"2024-04-25T10:20:47.888566+0000","last_undegraded":"2024-04-25T10:20:47.888566+0000","last_fullsized":"2024-04-25T10:20:47.888566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:47.793676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862572+0000","last_change":"2024-04-25T10:20:47.879970+0000","last_active":"2024-04-25T10:20:48.862572+0000","last_peered":"2024-04-25T10:20:48.862572+0000","last_clean":"2024-04-25T10:20:48.862572+0000","last_became_active":"2024-04-25T10:20:47.879448+0000","last_became_peered":"2024-04-25T10:20:47.879448+0000","last_unstale":"2024-04-25T10:20:48.862572+0000","last_undegraded":"2024-04-25T10:20:48.862572+0000","last_fullsized":"2024-04-25T10:20:48.862572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:48.714745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863965+0000","last_change":"2024-04-25T10:20:47.883187+0000","last_active":"2024-04-25T10:20:48.863965+0000","last_peered":"2024-04-25T10:20:48.863965+0000","last_clean":"2024-04-25T10:20:48.863965+0000","last_became_active":"2024-04-25T10:20:47.882871+0000","last_became_peered":"2024-04-25T10:20:47.882871+0000","last_unstale":"2024-04-25T10:20:48.863965+0000","last_undegraded":"2024-04-25T10:20:48.863965+0000","last_fullsized":"2024-04-25T10:20:48.863965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:57.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.882899+0000","last_change":"2024-04-25T10:20:47.878182+0000","last_active":"2024-04-25T10:20:47.882899+0000","last_peered":"2024-04-25T10:20:47.882899+0000","last_clean":"2024-04-25T10:20:47.882899+0000","last_became_active":"2024-04-25T10:20:47.877904+0000","last_became_peered":"2024-04-25T10:20:47.877904+0000","last_unstale":"2024-04-25T10:20:47.882899+0000","last_undegraded":"2024-04-25T10:20:47.882899+0000","last_fullsized":"2024-04-25T10:20:47.882899+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.637178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862533+0000","last_change":"2024-04-25T10:20:47.873400+0000","last_active":"2024-04-25T10:20:48.862533+0000","last_peered":"2024-04-25T10:20:48.862533+0000","last_clean":"2024-04-25T10:20:48.862533+0000","last_became_active":"2024-04-25T10:20:47.873018+0000","last_became_peered":"2024-04-25T10:20:47.873018+0000","last_unstale":"2024-04-25T10:20:48.862533+0000","last_undegraded":"2024-04-25T10:20:48.862533+0000","last_fullsized":"2024-04-25T10:20:48.862533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:38.306149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.895590+0000","last_change":"2024-04-25T10:20:47.889885+0000","last_active":"2024-04-25T10:20:47.895590+0000","last_peered":"2024-04-25T10:20:47.895590+0000","last_clean":"2024-04-25T10:20:47.895590+0000","last_became_active":"2024-04-25T10:20:47.889597+0000","last_became_peered":"2024-04-25T10:20:47.889597+0000","last_unstale":"2024-04-25T10:20:47.895590+0000","last_undegraded":"2024-04-25T10:20:47.895590+0000","last_fullsized":"2024-04-25T10:20:47.895590+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:55.331215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883129+0000","last_change":"2024-04-25T10:20:47.881233+0000","last_active":"2024-04-25T10:20:48.883129+0000","last_peered":"2024-04-25T10:20:48.883129+0000","last_clean":"2024-04-25T10:20:48.883129+0000","last_became_active":"2024-04-25T10:20:47.880961+0000","last_became_peered":"2024-04-25T10:20:47.880961+0000","last_unstale":"2024-04-25T10:20:48.883129+0000","last_undegraded":"2024-04-25T10:20:48.883129+0000","last_fullsized":"2024-04-25T10:20:48.883129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:46.294315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.880756+0000","last_change":"2024-04-25T10:20:47.874865+0000","last_active":"2024-04-25T10:20:47.880756+0000","last_peered":"2024-04-25T10:20:47.880756+0000","last_clean":"2024-04-25T10:20:47.880756+0000","last_became_active":"2024-04-25T10:20:47.874572+0000","last_became_peered":"2024-04-25T10:20:47.874572+0000","last_unstale":"2024-04-25T10:20:47.880756+0000","last_undegraded":"2024-04-25T10:20:47.880756+0000","last_fullsized":"2024-04-25T10:20:47.880756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:40.110717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.864157+0000","last_change":"2024-04-25T10:20:47.877594+0000","last_active":"2024-04-25T10:20:48.864157+0000","last_peered":"2024-04-25T10:20:48.864157+0000","last_clean":"2024-04-25T10:20:48.864157+0000","last_became_active":"2024-04-25T10:20:47.877256+0000","last_became_peered":"2024-04-25T10:20:47.877256+0000","last_unstale":"2024-04-25T10:20:48.864157+0000","last_undegraded":"2024-04-25T10:20:48.864157+0000","last_fullsized":"2024-04-25T10:20:48.864157+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:54.014411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863022+0000","last_change":"2024-04-25T10:20:47.890874+0000","last_active":"2024-04-25T10:20:48.863022+0000","last_peered":"2024-04-25T10:20:48.863022+0000","last_clean":"2024-04-25T10:20:48.863022+0000","last_became_active":"2024-04-25T10:20:47.890573+0000","last_became_peered":"2024-04-25T10:20:47.890573+0000","last_unstale":"2024-04-25T10:20:48.863022+0000","last_undegraded":"2024-04-25T10:20:48.863022+0000","last_fullsized":"2024-04-25T10:20:48.863022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:38.480032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883359+0000","last_change":"2024-04-25T10:20:47.881157+0000","last_active":"2024-04-25T10:20:48.883359+0000","last_peered":"2024-04-25T10:20:48.883359+0000","last_clean":"2024-04-25T10:20:48.883359+0000","last_became_active":"2024-04-25T10:20:47.879832+0000","last_became_peered":"2024-04-25T10:20:47.879832+0000","last_unstale":"2024-04-25T10:20:48.883359+0000","last_undegraded":"2024-04-25T10:20:48.883359+0000","last_fullsized":"2024-04-25T10:20:48.883359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:10.516200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883369+0000","last_change":"2024-04-25T10:20:47.873379+0000","last_active":"2024-04-25T10:20:48.883369+0000","last_peered":"2024-04-25T10:20:48.883369+0000","last_clean":"2024-04-25T10:20:48.883369+0000","last_became_active":"2024-04-25T10:20:47.873118+0000","last_became_peered":"2024-04-25T10:20:47.873118+0000","last_unstale":"2024-04-25T10:20:48.883369+0000","last_undegraded":"2024-04-25T10:20:48.883369+0000","last_fullsized":"2024-04-25T10:20:48.883369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:44.408570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863392+0000","last_change":"2024-04-25T10:20:47.880827+0000","last_active":"2024-04-25T10:20:48.863392+0000","last_peered":"2024-04-25T10:20:48.863392+0000","last_clean":"2024-04-25T10:20:48.863392+0000","last_became_active":"2024-04-25T10:20:47.880475+0000","last_became_peered":"2024-04-25T10:20:47.880475+0000","last_unstale":"2024-04-25T10:20:48.863392+0000","last_undegraded":"2024-04-25T10:20:48.863392+0000","last_fullsized":"2024-04-25T10:20:48.863392+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:47.793676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.875863+0000","last_change":"2024-04-25T10:20:47.873710+0000","last_active":"2024-04-25T10:20:47.875863+0000","last_peered":"2024-04-25T10:20:47.875863+0000","last_clean":"2024-04-25T10:20:47.875863+0000","last_became_active":"2024-04-25T10:20:47.873159+0000","last_became_peered":"2024-04-25T10:20:47.873159+0000","last_unstale":"2024-04-25T10:20:47.875863+0000","last_undegraded":"2024-04-25T10:20:47.875863+0000","last_fullsized":"2024-04-25T10:20:47.875863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:11.068484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888936+0000","last_change":"2024-04-25T10:20:47.880898+0000","last_active":"2024-04-25T10:20:47.888936+0000","last_peered":"2024-04-25T10:20:47.888936+0000","last_clean":"2024-04-25T10:20:47.888936+0000","last_became_active":"2024-04-25T10:20:47.880621+0000","last_became_peered":"2024-04-25T10:20:47.880621+0000","last_unstale":"2024-04-25T10:20:47.888936+0000","last_undegraded":"2024-04-25T10:20:47.888936+0000","last_fullsized":"2024-04-25T10:20:47.888936+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:33.788604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862211+0000","last_change":"2024-04-25T10:20:47.879404+0000","last_active":"2024-04-25T10:20:48.862211+0000","last_peered":"2024-04-25T10:20:48.862211+0000","last_clean":"2024-04-25T10:20:48.862211+0000","last_became_active":"2024-04-25T10:20:47.878888+0000","last_became_peered":"2024-04-25T10:20:47.878888+0000","last_unstale":"2024-04-25T10:20:48.862211+0000","last_undegraded":"2024-04-25T10:20:48.862211+0000","last_fullsized":"2024-04-25T10:20:48.862211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:23.701491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.885762+0000","last_change":"2024-04-25T10:20:47.879704+0000","last_active":"2024-04-25T10:20:47.885762+0000","last_peered":"2024-04-25T10:20:47.885762+0000","last_clean":"2024-04-25T10:20:47.885762+0000","last_became_active":"2024-04-25T10:20:47.879409+0000","last_became_peered":"2024-04-25T10:20:47.879409+0000","last_unstale":"2024-04-25T10:20:47.885762+0000","last_undegraded":"2024-04-25T10:20:47.885762+0000","last_fullsized":"2024-04-25T10:20:47.885762+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:36.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.899431+0000","last_change":"2024-04-25T10:20:46.884683+0000","last_active":"2024-04-25T10:20:47.899431+0000","last_peered":"2024-04-25T10:20:47.899431+0000","last_clean":"2024-04-25T10:20:47.899431+0000","last_became_active":"2024-04-25T10:20:46.884265+0000","last_became_peered":"2024-04-25T10:20:46.884265+0000","last_unstale":"2024-04-25T10:20:47.899431+0000","last_undegraded":"2024-04-25T10:20:47.899431+0000","last_fullsized":"2024-04-25T10:20:47.899431+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:23.225945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.864229+0000","last_change":"2024-04-25T10:20:47.890023+0000","last_active":"2024-04-25T10:20:48.864229+0000","last_peered":"2024-04-25T10:20:48.864229+0000","last_clean":"2024-04-25T10:20:48.864229+0000","last_became_active":"2024-04-25T10:20:47.889737+0000","last_became_peered":"2024-04-25T10:20:47.889737+0000","last_unstale":"2024-04-25T10:20:48.864229+0000","last_undegraded":"2024-04-25T10:20:48.864229+0000","last_fullsized":"2024-04-25T10:20:48.864229+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:20.528597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883415+0000","last_change":"2024-04-25T10:20:46.868419+0000","last_active":"2024-04-25T10:20:48.883415+0000","last_peered":"2024-04-25T10:20:48.883415+0000","last_clean":"2024-04-25T10:20:48.883415+0000","last_became_active":"2024-04-25T10:20:46.867442+0000","last_became_peered":"2024-04-25T10:20:46.867442+0000","last_unstale":"2024-04-25T10:20:48.883415+0000","last_undegraded":"2024-04-25T10:20:48.883415+0000","last_fullsized":"2024-04-25T10:20:48.883415+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:27.478041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.878450+0000","last_change":"2024-04-25T10:20:47.873631+0000","last_active":"2024-04-25T10:20:47.878450+0000","last_peered":"2024-04-25T10:20:47.878450+0000","last_clean":"2024-04-25T10:20:47.878450+0000","last_became_active":"2024-04-25T10:20:47.872869+0000","last_became_peered":"2024-04-25T10:20:47.872869+0000","last_unstale":"2024-04-25T10:20:47.878450+0000","last_undegraded":"2024-04-25T10:20:47.878450+0000","last_fullsized":"2024-04-25T10:20:47.878450+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.346819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.899399+0000","last_change":"2024-04-25T10:20:46.884610+0000","last_active":"2024-04-25T10:20:47.899399+0000","last_peered":"2024-04-25T10:20:47.899399+0000","last_clean":"2024-04-25T10:20:47.899399+0000","last_became_active":"2024-04-25T10:20:46.884283+0000","last_became_peered":"2024-04-25T10:20:46.884283+0000","last_unstale":"2024-04-25T10:20:47.899399+0000","last_undegraded":"2024-04-25T10:20:47.899399+0000","last_fullsized":"2024-04-25T10:20:47.899399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:46.601649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863112+0000","last_change":"2024-04-25T10:20:47.881733+0000","last_active":"2024-04-25T10:20:48.863112+0000","last_peered":"2024-04-25T10:20:48.863112+0000","last_clean":"2024-04-25T10:20:48.863112+0000","last_became_active":"2024-04-25T10:20:47.881437+0000","last_became_peered":"2024-04-25T10:20:47.881437+0000","last_unstale":"2024-04-25T10:20:48.863112+0000","last_undegraded":"2024-04-25T10:20:48.863112+0000","last_fullsized":"2024-04-25T10:20:48.863112+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:08.891127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883322+0000","last_change":"2024-04-25T10:20:46.868261+0000","last_active":"2024-04-25T10:20:48.883322+0000","last_peered":"2024-04-25T10:20:48.883322+0000","last_clean":"2024-04-25T10:20:48.883322+0000","last_became_active":"2024-04-25T10:20:46.866759+0000","last_became_peered":"2024-04-25T10:20:46.866759+0000","last_unstale":"2024-04-25T10:20:48.883322+0000","last_undegraded":"2024-04-25T10:20:48.883322+0000","last_fullsized":"2024-04-25T10:20:48.883322+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.406449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883163+0000","last_change":"2024-04-25T10:20:47.871713+0000","last_active":"2024-04-25T10:20:48.883163+0000","last_peered":"2024-04-25T10:20:48.883163+0000","last_clean":"2024-04-25T10:20:48.883163+0000","last_became_active":"2024-04-25T10:20:47.871386+0000","last_became_peered":"2024-04-25T10:20:47.871386+0000","last_unstale":"2024-04-25T10:20:48.883163+0000","last_undegraded":"2024-04-25T10:20:48.883163+0000","last_fullsized":"2024-04-25T10:20:48.883163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:51.144452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.896865+0000","last_change":"2024-04-25T10:20:46.874018+0000","last_active":"2024-04-25T10:20:47.896865+0000","last_peered":"2024-04-25T10:20:47.896865+0000","last_clean":"2024-04-25T10:20:47.896865+0000","last_became_active":"2024-04-25T10:20:46.873655+0000","last_became_peered":"2024-04-25T10:20:46.873655+0000","last_unstale":"2024-04-25T10:20:47.896865+0000","last_undegraded":"2024-04-25T10:20:47.896865+0000","last_fullsized":"2024-04-25T10:20:47.896865+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.708411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888233+0000","last_change":"2024-04-25T10:20:47.884040+0000","last_active":"2024-04-25T10:20:47.888233+0000","last_peered":"2024-04-25T10:20:47.888233+0000","last_clean":"2024-04-25T10:20:47.888233+0000","last_became_active":"2024-04-25T10:20:47.883661+0000","last_became_peered":"2024-04-25T10:20:47.883661+0000","last_unstale":"2024-04-25T10:20:47.888233+0000","last_undegraded":"2024-04-25T10:20:47.888233+0000","last_fullsized":"2024-04-25T10:20:47.888233+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:57.582900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883135+0000","last_change":"2024-04-25T10:20:46.878545+0000","last_active":"2024-04-25T10:20:48.883135+0000","last_peered":"2024-04-25T10:20:48.883135+0000","last_clean":"2024-04-25T10:20:48.883135+0000","last_became_active":"2024-04-25T10:20:46.878258+0000","last_became_peered":"2024-04-25T10:20:46.878258+0000","last_unstale":"2024-04-25T10:20:48.883135+0000","last_undegraded":"2024-04-25T10:20:48.883135+0000","last_fullsized":"2024-04-25T10:20:48.883135+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.209684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.884098+0000","last_change":"2024-04-25T10:20:47.873612+0000","last_active":"2024-04-25T10:20:47.884098+0000","last_peered":"2024-04-25T10:20:47.884098+0000","last_clean":"2024-04-25T10:20:47.884098+0000","last_became_active":"2024-04-25T10:20:47.872765+0000","last_became_peered":"2024-04-25T10:20:47.872765+0000","last_unstale":"2024-04-25T10:20:47.884098+0000","last_undegraded":"2024-04-25T10:20:47.884098+0000","last_fullsized":"2024-04-25T10:20:47.884098+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:22.429987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863480+0000","last_change":"2024-04-25T10:20:46.882647+0000","last_active":"2024-04-25T10:20:48.863480+0000","last_peered":"2024-04-25T10:20:48.863480+0000","last_clean":"2024-04-25T10:20:48.863480+0000","last_became_active":"2024-04-25T10:20:46.882369+0000","last_became_peered":"2024-04-25T10:20:46.882369+0000","last_unstale":"2024-04-25T10:20:48.863480+0000","last_undegraded":"2024-04-25T10:20:48.863480+0000","last_fullsized":"2024-04-25T10:20:48.863480+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:46.601649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883220+0000","last_change":"2024-04-25T10:20:47.883202+0000","last_active":"2024-04-25T10:20:48.883220+0000","last_peered":"2024-04-25T10:20:48.883220+0000","last_clean":"2024-04-25T10:20:48.883220+0000","last_became_active":"2024-04-25T10:20:47.882924+0000","last_became_peered":"2024-04-25T10:20:47.882924+0000","last_unstale":"2024-04-25T10:20:48.883220+0000","last_undegraded":"2024-04-25T10:20:48.883220+0000","last_fullsized":"2024-04-25T10:20:48.883220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:53.654062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.862702+0000","last_change":"2024-04-25T10:20:46.877454+0000","last_active":"2024-04-25T10:20:47.862702+0000","last_peered":"2024-04-25T10:20:47.862702+0000","last_clean":"2024-04-25T10:20:47.862702+0000","last_became_active":"2024-04-25T10:20:46.877180+0000","last_became_peered":"2024-04-25T10:20:46.877180+0000","last_unstale":"2024-04-25T10:20:47.862702+0000","last_undegraded":"2024-04-25T10:20:47.862702+0000","last_fullsized":"2024-04-25T10:20:47.862702+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:30.251948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862710+0000","last_change":"2024-04-25T10:20:47.879713+0000","last_active":"2024-04-25T10:20:48.862710+0000","last_peered":"2024-04-25T10:20:48.862710+0000","last_clean":"2024-04-25T10:20:48.862710+0000","last_became_active":"2024-04-25T10:20:47.879053+0000","last_became_peered":"2024-04-25T10:20:47.879053+0000","last_unstale":"2024-04-25T10:20:48.862710+0000","last_undegraded":"2024-04-25T10:20:48.862710+0000","last_fullsized":"2024-04-25T10:20:48.862710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:24.232279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862566+0000","last_change":"2024-04-25T10:20:46.872253+0000","last_active":"2024-04-25T10:20:48.862566+0000","last_peered":"2024-04-25T10:20:48.862566+0000","last_clean":"2024-04-25T10:20:48.862566+0000","last_became_active":"2024-04-25T10:20:46.871913+0000","last_became_peered":"2024-04-25T10:20:46.871913+0000","last_unstale":"2024-04-25T10:20:48.862566+0000","last_undegraded":"2024-04-25T10:20:48.862566+0000","last_fullsized":"2024-04-25T10:20:48.862566+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:08.544860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862557+0000","last_change":"2024-04-25T10:20:47.877536+0000","last_active":"2024-04-25T10:20:48.862557+0000","last_peered":"2024-04-25T10:20:48.862557+0000","last_clean":"2024-04-25T10:20:48.862557+0000","last_became_active":"2024-04-25T10:20:47.877259+0000","last_became_peered":"2024-04-25T10:20:47.877259+0000","last_unstale":"2024-04-25T10:20:48.862557+0000","last_undegraded":"2024-04-25T10:20:48.862557+0000","last_fullsized":"2024-04-25T10:20:48.862557+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:08.041828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.865386+0000","last_change":"2024-04-25T10:20:46.880727+0000","last_active":"2024-04-25T10:20:47.865386+0000","last_peered":"2024-04-25T10:20:47.865386+0000","last_clean":"2024-04-25T10:20:47.865386+0000","last_became_active":"2024-04-25T10:20:46.880295+0000","last_became_peered":"2024-04-25T10:20:46.880295+0000","last_unstale":"2024-04-25T10:20:47.865386+0000","last_undegraded":"2024-04-25T10:20:47.865386+0000","last_fullsized":"2024-04-25T10:20:47.865386+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:18.511050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862854+0000","last_change":"2024-04-25T10:20:47.876024+0000","last_active":"2024-04-25T10:20:48.862854+0000","last_peered":"2024-04-25T10:20:48.862854+0000","last_clean":"2024-04-25T10:20:48.862854+0000","last_became_active":"2024-04-25T10:20:47.875775+0000","last_became_peered":"2024-04-25T10:20:47.875775+0000","last_unstale":"2024-04-25T10:20:48.862854+0000","last_undegraded":"2024-04-25T10:20:48.862854+0000","last_fullsized":"2024-04-25T10:20:48.862854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:28.484374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.900398+0000","last_change":"2024-04-25T10:20:46.882814+0000","last_active":"2024-04-25T10:20:47.900398+0000","last_peered":"2024-04-25T10:20:47.900398+0000","last_clean":"2024-04-25T10:20:47.900398+0000","last_became_active":"2024-04-25T10:20:46.881112+0000","last_became_peered":"2024-04-25T10:20:46.881112+0000","last_unstale":"2024-04-25T10:20:47.900398+0000","last_undegraded":"2024-04-25T10:20:47.900398+0000","last_fullsized":"2024-04-25T10:20:47.900398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:03.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883450+0000","last_change":"2024-04-25T10:20:47.885057+0000","last_active":"2024-04-25T10:20:48.883450+0000","last_peered":"2024-04-25T10:20:48.883450+0000","last_clean":"2024-04-25T10:20:48.883450+0000","last_became_active":"2024-04-25T10:20:47.884002+0000","last_became_peered":"2024-04-25T10:20:47.884002+0000","last_unstale":"2024-04-25T10:20:48.883450+0000","last_undegraded":"2024-04-25T10:20:48.883450+0000","last_fullsized":"2024-04-25T10:20:48.883450+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:11.463257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.900929+0000","last_change":"2024-04-25T10:20:46.879675+0000","last_active":"2024-04-25T10:20:47.900929+0000","last_peered":"2024-04-25T10:20:47.900929+0000","last_clean":"2024-04-25T10:20:47.900929+0000","last_became_active":"2024-04-25T10:20:46.878328+0000","last_became_peered":"2024-04-25T10:20:46.878328+0000","last_unstale":"2024-04-25T10:20:47.900929+0000","last_undegraded":"2024-04-25T10:20:47.900929+0000","last_fullsized":"2024-04-25T10:20:47.900929+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:47.346556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862276+0000","last_change":"2024-04-25T10:20:47.875959+0000","last_active":"2024-04-25T10:20:48.862276+0000","last_peered":"2024-04-25T10:20:48.862276+0000","last_clean":"2024-04-25T10:20:48.862276+0000","last_became_active":"2024-04-25T10:20:47.875677+0000","last_became_peered":"2024-04-25T10:20:47.875677+0000","last_unstale":"2024-04-25T10:20:48.862276+0000","last_undegraded":"2024-04-25T10:20:48.862276+0000","last_fullsized":"2024-04-25T10:20:48.862276+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:55.331215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.898861+0000","last_change":"2024-04-25T10:20:46.875351+0000","last_active":"2024-04-25T10:20:47.898861+0000","last_peered":"2024-04-25T10:20:47.898861+0000","last_clean":"2024-04-25T10:20:47.898861+0000","last_became_active":"2024-04-25T10:20:46.875082+0000","last_became_peered":"2024-04-25T10:20:46.875082+0000","last_unstale":"2024-04-25T10:20:47.898861+0000","last_undegraded":"2024-04-25T10:20:47.898861+0000","last_fullsized":"2024-04-25T10:20:47.898861+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:39.217749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.886784+0000","last_change":"2024-04-25T10:20:47.878163+0000","last_active":"2024-04-25T10:20:47.886784+0000","last_peered":"2024-04-25T10:20:47.886784+0000","last_clean":"2024-04-25T10:20:47.886784+0000","last_became_active":"2024-04-25T10:20:47.877858+0000","last_became_peered":"2024-04-25T10:20:47.877858+0000","last_unstale":"2024-04-25T10:20:47.886784+0000","last_undegraded":"2024-04-25T10:20:47.886784+0000","last_fullsized":"2024-04-25T10:20:47.886784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:48.758822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863307+0000","last_change":"2024-04-25T10:20:46.885774+0000","last_active":"2024-04-25T10:20:48.863307+0000","last_peered":"2024-04-25T10:20:48.863307+0000","last_clean":"2024-04-25T10:20:48.863307+0000","last_became_active":"2024-04-25T10:20:46.885362+0000","last_became_peered":"2024-04-25T10:20:46.885362+0000","last_unstale":"2024-04-25T10:20:48.863307+0000","last_undegraded":"2024-04-25T10:20:48.863307+0000","last_fullsized":"2024-04-25T10:20:48.863307+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:07.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.890509+0000","last_change":"2024-04-25T10:20:47.887088+0000","last_active":"2024-04-25T10:20:47.890509+0000","last_peered":"2024-04-25T10:20:47.890509+0000","last_clean":"2024-04-25T10:20:47.890509+0000","last_became_active":"2024-04-25T10:20:47.886801+0000","last_became_peered":"2024-04-25T10:20:47.886801+0000","last_unstale":"2024-04-25T10:20:47.890509+0000","last_undegraded":"2024-04-25T10:20:47.890509+0000","last_fullsized":"2024-04-25T10:20:47.890509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.799875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.876678+0000","last_change":"2024-04-25T10:20:46.875036+0000","last_active":"2024-04-25T10:20:47.876678+0000","last_peered":"2024-04-25T10:20:47.876678+0000","last_clean":"2024-04-25T10:20:47.876678+0000","last_became_active":"2024-04-25T10:20:46.874710+0000","last_became_peered":"2024-04-25T10:20:46.874710+0000","last_unstale":"2024-04-25T10:20:47.876678+0000","last_undegraded":"2024-04-25T10:20:47.876678+0000","last_fullsized":"2024-04-25T10:20:47.876678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:59.488107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.886120+0000","last_change":"2024-04-25T10:20:47.878068+0000","last_active":"2024-04-25T10:20:47.886120+0000","last_peered":"2024-04-25T10:20:47.886120+0000","last_clean":"2024-04-25T10:20:47.886120+0000","last_became_active":"2024-04-25T10:20:47.877786+0000","last_became_peered":"2024-04-25T10:20:47.877786+0000","last_unstale":"2024-04-25T10:20:47.886120+0000","last_undegraded":"2024-04-25T10:20:47.886120+0000","last_fullsized":"2024-04-25T10:20:47.886120+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.898888+0000","last_change":"2024-04-25T10:20:46.878325+0000","last_active":"2024-04-25T10:20:47.898888+0000","last_peered":"2024-04-25T10:20:47.898888+0000","last_clean":"2024-04-25T10:20:47.898888+0000","last_became_active":"2024-04-25T10:20:46.877816+0000","last_became_peered":"2024-04-25T10:20:46.877816+0000","last_unstale":"2024-04-25T10:20:47.898888+0000","last_undegraded":"2024-04-25T10:20:47.898888+0000","last_fullsized":"2024-04-25T10:20:47.898888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:13.946309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.889560+0000","last_change":"2024-04-25T10:20:47.882566+0000","last_active":"2024-04-25T10:20:47.889560+0000","last_peered":"2024-04-25T10:20:47.889560+0000","last_clean":"2024-04-25T10:20:47.889560+0000","last_became_active":"2024-04-25T10:20:47.882204+0000","last_became_peered":"2024-04-25T10:20:47.882204+0000","last_unstale":"2024-04-25T10:20:47.889560+0000","last_undegraded":"2024-04-25T10:20:47.889560+0000","last_fullsized":"2024-04-25T10:20:47.889560+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.022546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863056+0000","last_change":"2024-04-25T10:20:46.878072+0000","last_active":"2024-04-25T10:20:48.863056+0000","last_peered":"2024-04-25T10:20:48.863056+0000","last_clean":"2024-04-25T10:20:48.863056+0000","last_became_active":"2024-04-25T10:20:46.877827+0000","last_became_peered":"2024-04-25T10:20:46.877827+0000","last_unstale":"2024-04-25T10:20:48.863056+0000","last_undegraded":"2024-04-25T10:20:48.863056+0000","last_fullsized":"2024-04-25T10:20:48.863056+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:07.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888997+0000","last_change":"2024-04-25T10:20:47.883311+0000","last_active":"2024-04-25T10:20:47.888997+0000","last_peered":"2024-04-25T10:20:47.888997+0000","last_clean":"2024-04-25T10:20:47.888997+0000","last_became_active":"2024-04-25T10:20:47.883065+0000","last_became_peered":"2024-04-25T10:20:47.883065+0000","last_unstale":"2024-04-25T10:20:47.888997+0000","last_undegraded":"2024-04-25T10:20:47.888997+0000","last_fullsized":"2024-04-25T10:20:47.888997+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:11.110634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862796+0000","last_change":"2024-04-25T10:20:46.875942+0000","last_active":"2024-04-25T10:20:48.862796+0000","last_peered":"2024-04-25T10:20:48.862796+0000","last_clean":"2024-04-25T10:20:48.862796+0000","last_became_active":"2024-04-25T10:20:46.875665+0000","last_became_peered":"2024-04-25T10:20:46.875665+0000","last_unstale":"2024-04-25T10:20:48.862796+0000","last_undegraded":"2024-04-25T10:20:48.862796+0000","last_fullsized":"2024-04-25T10:20:48.862796+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:19.347047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.889005+0000","last_change":"2024-04-25T10:20:47.884126+0000","last_active":"2024-04-25T10:20:47.889005+0000","last_peered":"2024-04-25T10:20:47.889005+0000","last_clean":"2024-04-25T10:20:47.889005+0000","last_became_active":"2024-04-25T10:20:47.881206+0000","last_became_peered":"2024-04-25T10:20:47.881206+0000","last_unstale":"2024-04-25T10:20:47.889005+0000","last_undegraded":"2024-04-25T10:20:47.889005+0000","last_fullsized":"2024-04-25T10:20:47.889005+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.194709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862442+0000","last_change":"2024-04-25T10:20:46.882965+0000","last_active":"2024-04-25T10:20:48.862442+0000","last_peered":"2024-04-25T10:20:48.862442+0000","last_clean":"2024-04-25T10:20:48.862442+0000","last_became_active":"2024-04-25T10:20:46.882648+0000","last_became_peered":"2024-04-25T10:20:46.882648+0000","last_unstale":"2024-04-25T10:20:48.862442+0000","last_undegraded":"2024-04-25T10:20:48.862442+0000","last_fullsized":"2024-04-25T10:20:48.862442+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.756799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862467+0000","last_change":"2024-04-25T10:20:47.874222+0000","last_active":"2024-04-25T10:20:48.862467+0000","last_peered":"2024-04-25T10:20:48.862467+0000","last_clean":"2024-04-25T10:20:48.862467+0000","last_became_active":"2024-04-25T10:20:47.873906+0000","last_became_peered":"2024-04-25T10:20:47.873906+0000","last_unstale":"2024-04-25T10:20:48.862467+0000","last_undegraded":"2024-04-25T10:20:48.862467+0000","last_fullsized":"2024-04-25T10:20:48.862467+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:47.607983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.876083+0000","last_change":"2024-04-25T10:20:46.875643+0000","last_active":"2024-04-25T10:20:47.876083+0000","last_peered":"2024-04-25T10:20:47.876083+0000","last_clean":"2024-04-25T10:20:47.876083+0000","last_became_active":"2024-04-25T10:20:46.874700+0000","last_became_peered":"2024-04-25T10:20:46.874700+0000","last_unstale":"2024-04-25T10:20:47.876083+0000","last_undegraded":"2024-04-25T10:20:47.876083+0000","last_fullsized":"2024-04-25T10:20:47.876083+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:07.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.896069+0000","last_change":"2024-04-25T10:20:47.888777+0000","last_active":"2024-04-25T10:20:47.896069+0000","last_peered":"2024-04-25T10:20:47.896069+0000","last_clean":"2024-04-25T10:20:47.896069+0000","last_became_active":"2024-04-25T10:20:47.887910+0000","last_became_peered":"2024-04-25T10:20:47.887910+0000","last_unstale":"2024-04-25T10:20:47.896069+0000","last_undegraded":"2024-04-25T10:20:47.896069+0000","last_fullsized":"2024-04-25T10:20:47.896069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:07.838343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.896944+0000","last_change":"2024-04-25T10:20:46.880209+0000","last_active":"2024-04-25T10:20:47.896944+0000","last_peered":"2024-04-25T10:20:47.896944+0000","last_clean":"2024-04-25T10:20:47.896944+0000","last_became_active":"2024-04-25T10:20:46.879794+0000","last_became_peered":"2024-04-25T10:20:46.879794+0000","last_unstale":"2024-04-25T10:20:47.896944+0000","last_undegraded":"2024-04-25T10:20:47.896944+0000","last_fullsized":"2024-04-25T10:20:47.896944+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:56.254801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888968+0000","last_change":"2024-04-25T10:20:47.880626+0000","last_active":"2024-04-25T10:20:47.888968+0000","last_peered":"2024-04-25T10:20:47.888968+0000","last_clean":"2024-04-25T10:20:47.888968+0000","last_became_active":"2024-04-25T10:20:47.880291+0000","last_became_peered":"2024-04-25T10:20:47.880291+0000","last_unstale":"2024-04-25T10:20:47.888968+0000","last_undegraded":"2024-04-25T10:20:47.888968+0000","last_fullsized":"2024-04-25T10:20:47.888968+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:02.162348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863332+0000","last_change":"2024-04-25T10:20:46.881826+0000","last_active":"2024-04-25T10:20:48.863332+0000","last_peered":"2024-04-25T10:20:48.863332+0000","last_clean":"2024-04-25T10:20:48.863332+0000","last_became_active":"2024-04-25T10:20:46.881427+0000","last_became_peered":"2024-04-25T10:20:46.881427+0000","last_unstale":"2024-04-25T10:20:48.863332+0000","last_undegraded":"2024-04-25T10:20:48.863332+0000","last_fullsized":"2024-04-25T10:20:48.863332+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:16.251360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862436+0000","last_change":"2024-04-25T10:20:47.872684+0000","last_active":"2024-04-25T10:20:48.862436+0000","last_peered":"2024-04-25T10:20:48.862436+0000","last_clean":"2024-04-25T10:20:48.862436+0000","last_became_active":"2024-04-25T10:20:47.872389+0000","last_became_peered":"2024-04-25T10:20:47.872389+0000","last_unstale":"2024-04-25T10:20:48.862436+0000","last_undegraded":"2024-04-25T10:20:48.862436+0000","last_fullsized":"2024-04-25T10:20:48.862436+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:12.303794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.865070+0000","last_change":"2024-04-25T10:20:46.877213+0000","last_active":"2024-04-25T10:20:47.865070+0000","last_peered":"2024-04-25T10:20:47.865070+0000","last_clean":"2024-04-25T10:20:47.865070+0000","last_became_active":"2024-04-25T10:20:46.876923+0000","last_became_peered":"2024-04-25T10:20:46.876923+0000","last_unstale":"2024-04-25T10:20:47.865070+0000","last_undegraded":"2024-04-25T10:20:47.865070+0000","last_fullsized":"2024-04-25T10:20:47.865070+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:36.827454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862689+0000","last_change":"2024-04-25T10:20:47.875310+0000","last_active":"2024-04-25T10:20:48.862689+0000","last_peered":"2024-04-25T10:20:48.862689+0000","last_clean":"2024-04-25T10:20:48.862689+0000","last_became_active":"2024-04-25T10:20:47.875049+0000","last_became_peered":"2024-04-25T10:20:47.875049+0000","last_unstale":"2024-04-25T10:20:48.862689+0000","last_undegraded":"2024-04-25T10:20:48.862689+0000","last_fullsized":"2024-04-25T10:20:48.862689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:14.216017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.882972+0000","last_change":"2024-04-25T10:20:46.871304+0000","last_active":"2024-04-25T10:20:48.882972+0000","last_peered":"2024-04-25T10:20:48.882972+0000","last_clean":"2024-04-25T10:20:48.882972+0000","last_became_active":"2024-04-25T10:20:46.871031+0000","last_became_peered":"2024-04-25T10:20:46.871031+0000","last_unstale":"2024-04-25T10:20:48.882972+0000","last_undegraded":"2024-04-25T10:20:48.882972+0000","last_fullsized":"2024-04-25T10:20:48.882972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:23.302094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883417+0000","last_change":"2024-04-25T10:20:47.882033+0000","last_active":"2024-04-25T10:20:48.883417+0000","last_peered":"2024-04-25T10:20:48.883417+0000","last_clean":"2024-04-25T10:20:48.883417+0000","last_became_active":"2024-04-25T10:20:47.881763+0000","last_became_peered":"2024-04-25T10:20:47.881763+0000","last_unstale":"2024-04-25T10:20:48.883417+0000","last_undegraded":"2024-04-25T10:20:48.883417+0000","last_fullsized":"2024-04-25T10:20:48.883417+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:09.190412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.881312+0000","last_change":"2024-04-25T10:20:46.870518+0000","last_active":"2024-04-25T10:20:48.881312+0000","last_peered":"2024-04-25T10:20:48.881312+0000","last_clean":"2024-04-25T10:20:48.881312+0000","last_became_active":"2024-04-25T10:20:46.870240+0000","last_became_peered":"2024-04-25T10:20:46.870240+0000","last_unstale":"2024-04-25T10:20:48.881312+0000","last_undegraded":"2024-04-25T10:20:48.881312+0000","last_fullsized":"2024-04-25T10:20:48.881312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.708411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.882921+0000","last_change":"2024-04-25T10:20:47.879506+0000","last_active":"2024-04-25T10:20:47.882921+0000","last_peered":"2024-04-25T10:20:47.882921+0000","last_clean":"2024-04-25T10:20:47.882921+0000","last_became_active":"2024-04-25T10:20:47.879244+0000","last_became_peered":"2024-04-25T10:20:47.879244+0000","last_unstale":"2024-04-25T10:20:47.882921+0000","last_undegraded":"2024-04-25T10:20:47.882921+0000","last_fullsized":"2024-04-25T10:20:47.882921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:25.270796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863934+0000","last_change":"2024-04-25T10:20:46.881532+0000","last_active":"2024-04-25T10:20:48.863934+0000","last_peered":"2024-04-25T10:20:48.863934+0000","last_clean":"2024-04-25T10:20:48.863934+0000","last_became_active":"2024-04-25T10:20:46.881181+0000","last_became_peered":"2024-04-25T10:20:46.881181+0000","last_unstale":"2024-04-25T10:20:48.863934+0000","last_undegraded":"2024-04-25T10:20:48.863934+0000","last_fullsized":"2024-04-25T10:20:48.863934+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.708411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883513+0000","last_change":"2024-04-25T10:20:47.884301+0000","last_active":"2024-04-25T10:20:48.883513+0000","last_peered":"2024-04-25T10:20:48.883513+0000","last_clean":"2024-04-25T10:20:48.883513+0000","last_became_active":"2024-04-25T10:20:47.883654+0000","last_became_peered":"2024-04-25T10:20:47.883654+0000","last_unstale":"2024-04-25T10:20:48.883513+0000","last_undegraded":"2024-04-25T10:20:48.883513+0000","last_fullsized":"2024-04-25T10:20:48.883513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:55.385973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.871544+0000","last_change":"2024-04-25T10:20:46.871719+0000","last_active":"2024-04-25T10:20:47.871544+0000","last_peered":"2024-04-25T10:20:47.871544+0000","last_clean":"2024-04-25T10:20:47.871544+0000","last_became_active":"2024-04-25T10:20:46.871441+0000","last_became_peered":"2024-04-25T10:20:46.871441+0000","last_unstale":"2024-04-25T10:20:47.871544+0000","last_undegraded":"2024-04-25T10:20:47.871544+0000","last_fullsized":"2024-04-25T10:20:47.871544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:00.444923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.889535+0000","last_change":"2024-04-25T10:20:47.884506+0000","last_active":"2024-04-25T10:20:47.889535+0000","last_peered":"2024-04-25T10:20:47.889535+0000","last_clean":"2024-04-25T10:20:47.889535+0000","last_became_active":"2024-04-25T10:20:47.882427+0000","last_became_peered":"2024-04-25T10:20:47.882427+0000","last_unstale":"2024-04-25T10:20:47.889535+0000","last_undegraded":"2024-04-25T10:20:47.889535+0000","last_fullsized":"2024-04-25T10:20:47.889535+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:23.249153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863435+0000","last_change":"2024-04-25T10:20:46.883106+0000","last_active":"2024-04-25T10:20:48.863435+0000","last_peered":"2024-04-25T10:20:48.863435+0000","last_clean":"2024-04-25T10:20:48.863435+0000","last_became_active":"2024-04-25T10:20:46.882853+0000","last_became_peered":"2024-04-25T10:20:46.882853+0000","last_unstale":"2024-04-25T10:20:48.863435+0000","last_undegraded":"2024-04-25T10:20:48.863435+0000","last_fullsized":"2024-04-25T10:20:48.863435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:23.225945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862270+0000","last_change":"2024-04-25T10:20:47.873403+0000","last_active":"2024-04-25T10:20:48.862270+0000","last_peered":"2024-04-25T10:20:48.862270+0000","last_clean":"2024-04-25T10:20:48.862270+0000","last_became_active":"2024-04-25T10:20:47.873045+0000","last_became_peered":"2024-04-25T10:20:47.873045+0000","last_unstale":"2024-04-25T10:20:48.862270+0000","last_undegraded":"2024-04-25T10:20:48.862270+0000","last_fullsized":"2024-04-25T10:20:48.862270+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:30.930965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883247+0000","last_change":"2024-04-25T10:20:46.880970+0000","last_active":"2024-04-25T10:20:48.883247+0000","last_peered":"2024-04-25T10:20:48.883247+0000","last_clean":"2024-04-25T10:20:48.883247+0000","last_became_active":"2024-04-25T10:20:46.880637+0000","last_became_peered":"2024-04-25T10:20:46.880637+0000","last_unstale":"2024-04-25T10:20:48.883247+0000","last_undegraded":"2024-04-25T10:20:48.883247+0000","last_fullsized":"2024-04-25T10:20:48.883247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:27.453194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.861927+0000","last_change":"2024-04-25T10:20:47.876723+0000","last_active":"2024-04-25T10:20:48.861927+0000","last_peered":"2024-04-25T10:20:48.861927+0000","last_clean":"2024-04-25T10:20:48.861927+0000","last_became_active":"2024-04-25T10:20:47.876394+0000","last_became_peered":"2024-04-25T10:20:47.876394+0000","last_unstale":"2024-04-25T10:20:48.861927+0000","last_undegraded":"2024-04-25T10:20:48.861927+0000","last_fullsized":"2024-04-25T10:20:48.861927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.799875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.875912+0000","last_change":"2024-04-25T10:20:46.874300+0000","last_active":"2024-04-25T10:20:47.875912+0000","last_peered":"2024-04-25T10:20:47.875912+0000","last_clean":"2024-04-25T10:20:47.875912+0000","last_became_active":"2024-04-25T10:20:46.873743+0000","last_became_peered":"2024-04-25T10:20:46.873743+0000","last_unstale":"2024-04-25T10:20:47.875912+0000","last_undegraded":"2024-04-25T10:20:47.875912+0000","last_fullsized":"2024-04-25T10:20:47.875912+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:16.251360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.867092+0000","last_change":"2024-04-25T10:20:47.865801+0000","last_active":"2024-04-25T10:20:47.867092+0000","last_peered":"2024-04-25T10:20:47.867092+0000","last_clean":"2024-04-25T10:20:47.867092+0000","last_became_active":"2024-04-25T10:20:47.865504+0000","last_became_peered":"2024-04-25T10:20:47.865504+0000","last_unstale":"2024-04-25T10:20:47.867092+0000","last_undegraded":"2024-04-25T10:20:47.867092+0000","last_fullsized":"2024-04-25T10:20:47.867092+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:07.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.871050+0000","last_change":"2024-04-25T10:20:46.881160+0000","last_active":"2024-04-25T10:20:47.871050+0000","last_peered":"2024-04-25T10:20:47.871050+0000","last_clean":"2024-04-25T10:20:47.871050+0000","last_became_active":"2024-04-25T10:20:46.880707+0000","last_became_peered":"2024-04-25T10:20:46.880707+0000","last_unstale":"2024-04-25T10:20:47.871050+0000","last_undegraded":"2024-04-25T10:20:47.871050+0000","last_fullsized":"2024-04-25T10:20:47.871050+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:37.615463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863064+0000","last_change":"2024-04-25T10:20:47.876620+0000","last_active":"2024-04-25T10:20:48.863064+0000","last_peered":"2024-04-25T10:20:48.863064+0000","last_clean":"2024-04-25T10:20:48.863064+0000","last_became_active":"2024-04-25T10:20:47.876311+0000","last_became_peered":"2024-04-25T10:20:47.876311+0000","last_unstale":"2024-04-25T10:20:48.863064+0000","last_undegraded":"2024-04-25T10:20:48.863064+0000","last_fullsized":"2024-04-25T10:20:48.863064+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:36.968970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883428+0000","last_change":"2024-04-25T10:20:46.869232+0000","last_active":"2024-04-25T10:20:48.883428+0000","last_peered":"2024-04-25T10:20:48.883428+0000","last_clean":"2024-04-25T10:20:48.883428+0000","last_became_active":"2024-04-25T10:20:46.868946+0000","last_became_peered":"2024-04-25T10:20:46.868946+0000","last_unstale":"2024-04-25T10:20:48.883428+0000","last_undegraded":"2024-04-25T10:20:48.883428+0000","last_fullsized":"2024-04-25T10:20:48.883428+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:37.473698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":224,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.861892+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:20:48.861892+0000","last_peered":"2024-04-25T10:20:48.861892+0000","last_clean":"2024-04-25T10:20:48.861892+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:20:48.861892+0000","last_undegraded":"2024-04-25T10:20:48.861892+0000","last_fullsized":"2024-04-25T10:20:48.861892+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":573440,"data_stored":233469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35912,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335928,"statfs":{"total":96636764160,"available":96599990272,"internally_reserved":0,"allocated":577536,"data_stored":234051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40648,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331192,"statfs":{"total":96636764160,"available":96595140608,"internally_reserved":0,"allocated":1167360,"data_stored":823825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40068,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331772,"statfs":{"total":96636764160,"available":96595734528,"internally_reserved":0,"allocated":581632,"data_stored":235047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40060,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331780,"statfs":{"total":96636764160,"available":96595742720,"internally_reserved":0,"allocated":573440,"data_stored":233525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1167360,"data_stored":823873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":577536,"data_stored":234107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40056,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331784,"statfs":{"total":96636764160,"available":96595746816,"internally_reserved":0,"allocated":569344,"data_stored":233435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:51.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:52.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.159+0000 7f3c41200640 1 Processor -- start 2024-04-25T10:20:52.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c41200640 1 -- start start 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c41200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c3c0d0350 0x7f3c3c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c41200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c41200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c0ccaa0 con 0x7f3c3c15df60 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c41200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c15e340 con 0x7f3c3c0d0350 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c41200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0ccd80 con 0x7f3c3c06aea0 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c3c15df60 0x7f3c3c0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46222/0 (socket says 172.21.15.45:46222) 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39e00640 1 -- 172.21.15.45:0/3183752072 learned_addr learned my addr 172.21.15.45:0/3183752072 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:52.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c3a800640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39400640 1 --2- 172.21.15.45:0/3183752072 >> v2:172.21.15.117:3300/0 conn(0x7f3c3c0d0350 0x7f3c3c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c38a00640 1 -- 172.21.15.45:0/3183752072 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369227416 0 0) 0x7f3c3c0ccaa0 con 0x7f3c3c15df60 2024-04-25T10:20:52.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c38a00640 1 -- 172.21.15.45:0/3183752072 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c20003530 con 0x7f3c3c15df60 2024-04-25T10:20:52.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39400640 1 -- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 msgr2=0x7f3c3c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:52.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39400640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39400640 1 -- 172.21.15.45:0/3183752072 >> v1:172.21.15.45:6789/0 conn(0x7f3c3c15df60 legacy=0x7f3c3c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c39400640 1 -- 172.21.15.45:0/3183752072 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c0cd060 con 0x7f3c3c0d0350 2024-04-25T10:20:52.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.163+0000 7f3c3a800640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:52.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c39400640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c0d07b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3c280097c0 tx=0x7f3c2801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523b9a3dc529aadf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c38a00640 1 -- 172.21.15.45:0/3183752072 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280b0070 con 0x7f3c3c0d0350 2024-04-25T10:20:52.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c38a00640 1 -- 172.21.15.45:0/3183752072 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c280ab600 con 0x7f3c3c0d0350 2024-04-25T10:20:52.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c38a00640 1 -- 172.21.15.45:0/3183752072 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280ab9d0 con 0x7f3c3c0d0350 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 -- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c0d0350 msgr2=0x7f3c3c0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c0d07b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3c280097c0 tx=0x7f3c2801c410 comp rx=0 tx=0).stop 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 -- 172.21.15.45:0/3183752072 shutdown_connections 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 --2- 172.21.15.45:0/3183752072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c0d07b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 -- 172.21.15.45:0/3183752072 >> 172.21.15.45:0/3183752072 conn(0x7f3c3c0d1910 msgr2=0x7f3c3c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 -- 172.21.15.45:0/3183752072 shutdown_connections 2024-04-25T10:20:52.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.167+0000 7f3c41200640 1 -- 172.21.15.45:0/3183752072 wait complete. 2024-04-25T10:20:52.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 Processor -- start 2024-04-25T10:20:52.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- start start 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c15df60 0x7f3c3c14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0d0ff0 con 0x7f3c3c0d0350 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0d0e70 con 0x7f3c3c15df60 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c16aa20 con 0x7f3c3c06aea0 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c14efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54246/0 (socket says 172.21.15.45:54246) 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 -- 172.21.15.45:0/183396739 learned_addr learned my addr 172.21.15.45:0/183396739 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:52.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39e00640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c3a800640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c15df60 0x7f3c3c14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 -- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 msgr2=0x7f3c3c148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 -- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c15df60 msgr2=0x7f3c3c14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c15df60 0x7f3c3c14f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 -- 172.21.15.45:0/183396739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c0cf050 con 0x7f3c3c0d0350 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c39400640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c14efd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801aac0 tx=0x7f3c280191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280b8020 con 0x7f3c3c0d0350 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c280b4600 con 0x7f3c3c0d0350 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c149470 con 0x7f3c3c0d0350 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280b0040 con 0x7f3c3c0d0350 2024-04-25T10:20:52.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c14e7d0 con 0x7f3c3c0d0350 2024-04-25T10:20:52.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.171+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c04ef30 con 0x7f3c3c0d0350 2024-04-25T10:20:52.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.175+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c280bb020 con 0x7f3c3c0d0350 2024-04-25T10:20:52.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.175+0000 7f3c32000640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c1c066f60 0x7f3c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.175+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c2811e9a0 con 0x7f3c3c0d0350 2024-04-25T10:20:52.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.175+0000 7f3c39e00640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c1c066f60 0x7f3c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.175+0000 7f3c39e00640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c1c066f60 0x7f3c1c069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005890 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.179+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c28016090 con 0x7f3c3c0d0350 2024-04-25T10:20:52.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.335+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c3c0d08f0 con 0x7f3c1c066f60 2024-04-25T10:20:52.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.343+0000 7f3c32000640 1 -- 172.21.15.45:0/183396739 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251926 (secure 0 0 0) 0x7f3c3c0d08f0 con 0x7f3c1c066f60 2024-04-25T10:20:52.348 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:20:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c1c066f60 msgr2=0x7f3c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c1c066f60 0x7f3c1c069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005890 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 msgr2=0x7f3c3c14efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c14efd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3c2801aac0 tx=0x7f3c280191b0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 shutdown_connections 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c1c066f60 0x7f3c1c069420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c3c15df60 0x7f3c3c14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c3c0d0350 0x7f3c3c14efd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 --2- 172.21.15.45:0/183396739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c3c06aea0 0x7f3c3c148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 >> 172.21.15.45:0/183396739 conn(0x7f3c3c0d1910 msgr2=0x7f3c3c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 shutdown_connections 2024-04-25T10:20:52.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.347+0000 7f3c41200640 1 -- 172.21.15.45:0/183396739 wait complete. 2024-04-25T10:20:52.368 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-04-25T10:20:50.910837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":309192,"kb_used_data":5652,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754665528,"statfs":{"total":773094113280,"available":772777500672,"internally_reserved":0,"allocated":5787648,"data_stored":3051332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.059486"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.864148+0000","last_change":"2024-04-25T10:20:47.877059+0000","last_active":"2024-04-25T10:20:48.864148+0000","last_peered":"2024-04-25T10:20:48.864148+0000","last_clean":"2024-04-25T10:20:48.864148+0000","last_became_active":"2024-04-25T10:20:47.876332+0000","last_became_peered":"2024-04-25T10:20:47.876332+0000","last_unstale":"2024-04-25T10:20:48.864148+0000","last_undegraded":"2024-04-25T10:20:48.864148+0000","last_fullsized":"2024-04-25T10:20:48.864148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.194709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.875210+0000","last_change":"2024-04-25T10:20:47.871724+0000","last_active":"2024-04-25T10:20:47.875210+0000","last_peered":"2024-04-25T10:20:47.875210+0000","last_clean":"2024-04-25T10:20:47.875210+0000","last_became_active":"2024-04-25T10:20:47.871378+0000","last_became_peered":"2024-04-25T10:20:47.871378+0000","last_unstale":"2024-04-25T10:20:47.875210+0000","last_undegraded":"2024-04-25T10:20:47.875210+0000","last_fullsized":"2024-04-25T10:20:47.875210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:53.489221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862289+0000","last_change":"2024-04-25T10:20:47.874404+0000","last_active":"2024-04-25T10:20:48.862289+0000","last_peered":"2024-04-25T10:20:48.862289+0000","last_clean":"2024-04-25T10:20:48.862289+0000","last_became_active":"2024-04-25T10:20:47.874025+0000","last_became_peered":"2024-04-25T10:20:47.874025+0000","last_unstale":"2024-04-25T10:20:48.862289+0000","last_undegraded":"2024-04-25T10:20:48.862289+0000","last_fullsized":"2024-04-25T10:20:48.862289+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:17.257694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862692+0000","last_change":"2024-04-25T10:20:47.885775+0000","last_active":"2024-04-25T10:20:48.862692+0000","last_peered":"2024-04-25T10:20:48.862692+0000","last_clean":"2024-04-25T10:20:48.862692+0000","last_became_active":"2024-04-25T10:20:47.885485+0000","last_became_peered":"2024-04-25T10:20:47.885485+0000","last_unstale":"2024-04-25T10:20:48.862692+0000","last_undegraded":"2024-04-25T10:20:48.862692+0000","last_fullsized":"2024-04-25T10:20:48.862692+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:28.459527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863362+0000","last_change":"2024-04-25T10:20:47.884354+0000","last_active":"2024-04-25T10:20:48.863362+0000","last_peered":"2024-04-25T10:20:48.863362+0000","last_clean":"2024-04-25T10:20:48.863362+0000","last_became_active":"2024-04-25T10:20:47.884081+0000","last_became_peered":"2024-04-25T10:20:47.884081+0000","last_unstale":"2024-04-25T10:20:48.863362+0000","last_undegraded":"2024-04-25T10:20:48.863362+0000","last_fullsized":"2024-04-25T10:20:48.863362+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.891569+0000","last_change":"2024-04-25T10:20:47.888754+0000","last_active":"2024-04-25T10:20:47.891569+0000","last_peered":"2024-04-25T10:20:47.891569+0000","last_clean":"2024-04-25T10:20:47.891569+0000","last_became_active":"2024-04-25T10:20:47.888503+0000","last_became_peered":"2024-04-25T10:20:47.888503+0000","last_unstale":"2024-04-25T10:20:47.891569+0000","last_undegraded":"2024-04-25T10:20:47.891569+0000","last_fullsized":"2024-04-25T10:20:47.891569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:53.489221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883054+0000","last_change":"2024-04-25T10:20:47.882480+0000","last_active":"2024-04-25T10:20:48.883054+0000","last_peered":"2024-04-25T10:20:48.883054+0000","last_clean":"2024-04-25T10:20:48.883054+0000","last_became_active":"2024-04-25T10:20:47.882213+0000","last_became_peered":"2024-04-25T10:20:47.882213+0000","last_unstale":"2024-04-25T10:20:48.883054+0000","last_undegraded":"2024-04-25T10:20:48.883054+0000","last_fullsized":"2024-04-25T10:20:48.883054+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:29.153863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862160+0000","last_change":"2024-04-25T10:20:47.875265+0000","last_active":"2024-04-25T10:20:48.862160+0000","last_peered":"2024-04-25T10:20:48.862160+0000","last_clean":"2024-04-25T10:20:48.862160+0000","last_became_active":"2024-04-25T10:20:47.874978+0000","last_became_peered":"2024-04-25T10:20:47.874978+0000","last_unstale":"2024-04-25T10:20:48.862160+0000","last_undegraded":"2024-04-25T10:20:48.862160+0000","last_fullsized":"2024-04-25T10:20:48.862160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:54.014411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.891283+0000","last_change":"2024-04-25T10:20:47.887995+0000","last_active":"2024-04-25T10:20:47.891283+0000","last_peered":"2024-04-25T10:20:47.891283+0000","last_clean":"2024-04-25T10:20:47.891283+0000","last_became_active":"2024-04-25T10:20:47.887742+0000","last_became_peered":"2024-04-25T10:20:47.887742+0000","last_unstale":"2024-04-25T10:20:47.891283+0000","last_undegraded":"2024-04-25T10:20:47.891283+0000","last_fullsized":"2024-04-25T10:20:47.891283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:07.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.872605+0000","last_change":"2024-04-25T10:20:47.868855+0000","last_active":"2024-04-25T10:20:47.872605+0000","last_peered":"2024-04-25T10:20:47.872605+0000","last_clean":"2024-04-25T10:20:47.872605+0000","last_became_active":"2024-04-25T10:20:47.868484+0000","last_became_peered":"2024-04-25T10:20:47.868484+0000","last_unstale":"2024-04-25T10:20:47.872605+0000","last_undegraded":"2024-04-25T10:20:47.872605+0000","last_fullsized":"2024-04-25T10:20:47.872605+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:16.491024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863053+0000","last_change":"2024-04-25T10:20:47.891178+0000","last_active":"2024-04-25T10:20:48.863053+0000","last_peered":"2024-04-25T10:20:48.863053+0000","last_clean":"2024-04-25T10:20:48.863053+0000","last_became_active":"2024-04-25T10:20:47.890900+0000","last_became_peered":"2024-04-25T10:20:47.890900+0000","last_unstale":"2024-04-25T10:20:48.863053+0000","last_undegraded":"2024-04-25T10:20:48.863053+0000","last_fullsized":"2024-04-25T10:20:48.863053+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.412783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888595+0000","last_change":"2024-04-25T10:20:47.879551+0000","last_active":"2024-04-25T10:20:47.888595+0000","last_peered":"2024-04-25T10:20:47.888595+0000","last_clean":"2024-04-25T10:20:47.888595+0000","last_became_active":"2024-04-25T10:20:47.879282+0000","last_became_peered":"2024-04-25T10:20:47.879282+0000","last_unstale":"2024-04-25T10:20:47.888595+0000","last_undegraded":"2024-04-25T10:20:47.888595+0000","last_fullsized":"2024-04-25T10:20:47.888595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:57.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.891128+0000","last_change":"2024-04-25T10:20:47.886687+0000","last_active":"2024-04-25T10:20:47.891128+0000","last_peered":"2024-04-25T10:20:47.891128+0000","last_clean":"2024-04-25T10:20:47.891128+0000","last_became_active":"2024-04-25T10:20:47.884821+0000","last_became_peered":"2024-04-25T10:20:47.884821+0000","last_unstale":"2024-04-25T10:20:47.891128+0000","last_undegraded":"2024-04-25T10:20:47.891128+0000","last_fullsized":"2024-04-25T10:20:47.891128+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:04.188422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862690+0000","last_change":"2024-04-25T10:20:47.875438+0000","last_active":"2024-04-25T10:20:48.862690+0000","last_peered":"2024-04-25T10:20:48.862690+0000","last_clean":"2024-04-25T10:20:48.862690+0000","last_became_active":"2024-04-25T10:20:47.874535+0000","last_became_peered":"2024-04-25T10:20:47.874535+0000","last_unstale":"2024-04-25T10:20:48.862690+0000","last_undegraded":"2024-04-25T10:20:48.862690+0000","last_fullsized":"2024-04-25T10:20:48.862690+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:24.308428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862180+0000","last_change":"2024-04-25T10:20:47.871234+0000","last_active":"2024-04-25T10:20:48.862180+0000","last_peered":"2024-04-25T10:20:48.862180+0000","last_clean":"2024-04-25T10:20:48.862180+0000","last_became_active":"2024-04-25T10:20:47.870909+0000","last_became_peered":"2024-04-25T10:20:47.870909+0000","last_unstale":"2024-04-25T10:20:48.862180+0000","last_undegraded":"2024-04-25T10:20:48.862180+0000","last_fullsized":"2024-04-25T10:20:48.862180+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:20.528597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863258+0000","last_change":"2024-04-25T10:20:47.886290+0000","last_active":"2024-04-25T10:20:48.863258+0000","last_peered":"2024-04-25T10:20:48.863258+0000","last_clean":"2024-04-25T10:20:48.863258+0000","last_became_active":"2024-04-25T10:20:47.885778+0000","last_became_peered":"2024-04-25T10:20:47.885778+0000","last_unstale":"2024-04-25T10:20:48.863258+0000","last_undegraded":"2024-04-25T10:20:48.863258+0000","last_fullsized":"2024-04-25T10:20:48.863258+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:40.110717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888566+0000","last_change":"2024-04-25T10:20:47.878579+0000","last_active":"2024-04-25T10:20:47.888566+0000","last_peered":"2024-04-25T10:20:47.888566+0000","last_clean":"2024-04-25T10:20:47.888566+0000","last_became_active":"2024-04-25T10:20:47.877877+0000","last_became_peered":"2024-04-25T10:20:47.877877+0000","last_unstale":"2024-04-25T10:20:47.888566+0000","last_undegraded":"2024-04-25T10:20:47.888566+0000","last_fullsized":"2024-04-25T10:20:47.888566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:47.793676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862572+0000","last_change":"2024-04-25T10:20:47.879970+0000","last_active":"2024-04-25T10:20:48.862572+0000","last_peered":"2024-04-25T10:20:48.862572+0000","last_clean":"2024-04-25T10:20:48.862572+0000","last_became_active":"2024-04-25T10:20:47.879448+0000","last_became_peered":"2024-04-25T10:20:47.879448+0000","last_unstale":"2024-04-25T10:20:48.862572+0000","last_undegraded":"2024-04-25T10:20:48.862572+0000","last_fullsized":"2024-04-25T10:20:48.862572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:48.714745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863965+0000","last_change":"2024-04-25T10:20:47.883187+0000","last_active":"2024-04-25T10:20:48.863965+0000","last_peered":"2024-04-25T10:20:48.863965+0000","last_clean":"2024-04-25T10:20:48.863965+0000","last_became_active":"2024-04-25T10:20:47.882871+0000","last_became_peered":"2024-04-25T10:20:47.882871+0000","last_unstale":"2024-04-25T10:20:48.863965+0000","last_undegraded":"2024-04-25T10:20:48.863965+0000","last_fullsized":"2024-04-25T10:20:48.863965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:57.824002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.882899+0000","last_change":"2024-04-25T10:20:47.878182+0000","last_active":"2024-04-25T10:20:47.882899+0000","last_peered":"2024-04-25T10:20:47.882899+0000","last_clean":"2024-04-25T10:20:47.882899+0000","last_became_active":"2024-04-25T10:20:47.877904+0000","last_became_peered":"2024-04-25T10:20:47.877904+0000","last_unstale":"2024-04-25T10:20:47.882899+0000","last_undegraded":"2024-04-25T10:20:47.882899+0000","last_fullsized":"2024-04-25T10:20:47.882899+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:22.637178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862533+0000","last_change":"2024-04-25T10:20:47.873400+0000","last_active":"2024-04-25T10:20:48.862533+0000","last_peered":"2024-04-25T10:20:48.862533+0000","last_clean":"2024-04-25T10:20:48.862533+0000","last_became_active":"2024-04-25T10:20:47.873018+0000","last_became_peered":"2024-04-25T10:20:47.873018+0000","last_unstale":"2024-04-25T10:20:48.862533+0000","last_undegraded":"2024-04-25T10:20:48.862533+0000","last_fullsized":"2024-04-25T10:20:48.862533+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:38.306149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.895590+0000","last_change":"2024-04-25T10:20:47.889885+0000","last_active":"2024-04-25T10:20:47.895590+0000","last_peered":"2024-04-25T10:20:47.895590+0000","last_clean":"2024-04-25T10:20:47.895590+0000","last_became_active":"2024-04-25T10:20:47.889597+0000","last_became_peered":"2024-04-25T10:20:47.889597+0000","last_unstale":"2024-04-25T10:20:47.895590+0000","last_undegraded":"2024-04-25T10:20:47.895590+0000","last_fullsized":"2024-04-25T10:20:47.895590+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:55.331215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883129+0000","last_change":"2024-04-25T10:20:47.881233+0000","last_active":"2024-04-25T10:20:48.883129+0000","last_peered":"2024-04-25T10:20:48.883129+0000","last_clean":"2024-04-25T10:20:48.883129+0000","last_became_active":"2024-04-25T10:20:47.880961+0000","last_became_peered":"2024-04-25T10:20:47.880961+0000","last_unstale":"2024-04-25T10:20:48.883129+0000","last_undegraded":"2024-04-25T10:20:48.883129+0000","last_fullsized":"2024-04-25T10:20:48.883129+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:46.294315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.880756+0000","last_change":"2024-04-25T10:20:47.874865+0000","last_active":"2024-04-25T10:20:47.880756+0000","last_peered":"2024-04-25T10:20:47.880756+0000","last_clean":"2024-04-25T10:20:47.880756+0000","last_became_active":"2024-04-25T10:20:47.874572+0000","last_became_peered":"2024-04-25T10:20:47.874572+0000","last_unstale":"2024-04-25T10:20:47.880756+0000","last_undegraded":"2024-04-25T10:20:47.880756+0000","last_fullsized":"2024-04-25T10:20:47.880756+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:40.110717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.864157+0000","last_change":"2024-04-25T10:20:47.877594+0000","last_active":"2024-04-25T10:20:48.864157+0000","last_peered":"2024-04-25T10:20:48.864157+0000","last_clean":"2024-04-25T10:20:48.864157+0000","last_became_active":"2024-04-25T10:20:47.877256+0000","last_became_peered":"2024-04-25T10:20:47.877256+0000","last_unstale":"2024-04-25T10:20:48.864157+0000","last_undegraded":"2024-04-25T10:20:48.864157+0000","last_fullsized":"2024-04-25T10:20:48.864157+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:54.014411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863022+0000","last_change":"2024-04-25T10:20:47.890874+0000","last_active":"2024-04-25T10:20:48.863022+0000","last_peered":"2024-04-25T10:20:48.863022+0000","last_clean":"2024-04-25T10:20:48.863022+0000","last_became_active":"2024-04-25T10:20:47.890573+0000","last_became_peered":"2024-04-25T10:20:47.890573+0000","last_unstale":"2024-04-25T10:20:48.863022+0000","last_undegraded":"2024-04-25T10:20:48.863022+0000","last_fullsized":"2024-04-25T10:20:48.863022+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:38.480032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883359+0000","last_change":"2024-04-25T10:20:47.881157+0000","last_active":"2024-04-25T10:20:48.883359+0000","last_peered":"2024-04-25T10:20:48.883359+0000","last_clean":"2024-04-25T10:20:48.883359+0000","last_became_active":"2024-04-25T10:20:47.879832+0000","last_became_peered":"2024-04-25T10:20:47.879832+0000","last_unstale":"2024-04-25T10:20:48.883359+0000","last_undegraded":"2024-04-25T10:20:48.883359+0000","last_fullsized":"2024-04-25T10:20:48.883359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:10.516200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883369+0000","last_change":"2024-04-25T10:20:47.873379+0000","last_active":"2024-04-25T10:20:48.883369+0000","last_peered":"2024-04-25T10:20:48.883369+0000","last_clean":"2024-04-25T10:20:48.883369+0000","last_became_active":"2024-04-25T10:20:47.873118+0000","last_became_peered":"2024-04-25T10:20:47.873118+0000","last_unstale":"2024-04-25T10:20:48.883369+0000","last_undegraded":"2024-04-25T10:20:48.883369+0000","last_fullsized":"2024-04-25T10:20:48.883369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:44.408570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863392+0000","last_change":"2024-04-25T10:20:47.880827+0000","last_active":"2024-04-25T10:20:48.863392+0000","last_peered":"2024-04-25T10:20:48.863392+0000","last_clean":"2024-04-25T10:20:48.863392+0000","last_became_active":"2024-04-25T10:20:47.880475+0000","last_became_peered":"2024-04-25T10:20:47.880475+0000","last_unstale":"2024-04-25T10:20:48.863392+0000","last_undegraded":"2024-04-25T10:20:48.863392+0000","last_fullsized":"2024-04-25T10:20:48.863392+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:47.793676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.875863+0000","last_change":"2024-04-25T10:20:47.873710+0000","last_active":"2024-04-25T10:20:47.875863+0000","last_peered":"2024-04-25T10:20:47.875863+0000","last_clean":"2024-04-25T10:20:47.875863+0000","last_became_active":"2024-04-25T10:20:47.873159+0000","last_became_peered":"2024-04-25T10:20:47.873159+0000","last_unstale":"2024-04-25T10:20:47.875863+0000","last_undegraded":"2024-04-25T10:20:47.875863+0000","last_fullsized":"2024-04-25T10:20:47.875863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:11.068484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888936+0000","last_change":"2024-04-25T10:20:47.880898+0000","last_active":"2024-04-25T10:20:47.888936+0000","last_peered":"2024-04-25T10:20:47.888936+0000","last_clean":"2024-04-25T10:20:47.888936+0000","last_became_active":"2024-04-25T10:20:47.880621+0000","last_became_peered":"2024-04-25T10:20:47.880621+0000","last_unstale":"2024-04-25T10:20:47.888936+0000","last_undegraded":"2024-04-25T10:20:47.888936+0000","last_fullsized":"2024-04-25T10:20:47.888936+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:33.788604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862211+0000","last_change":"2024-04-25T10:20:47.879404+0000","last_active":"2024-04-25T10:20:48.862211+0000","last_peered":"2024-04-25T10:20:48.862211+0000","last_clean":"2024-04-25T10:20:48.862211+0000","last_became_active":"2024-04-25T10:20:47.878888+0000","last_became_peered":"2024-04-25T10:20:47.878888+0000","last_unstale":"2024-04-25T10:20:48.862211+0000","last_undegraded":"2024-04-25T10:20:48.862211+0000","last_fullsized":"2024-04-25T10:20:48.862211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:23.701491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.885762+0000","last_change":"2024-04-25T10:20:47.879704+0000","last_active":"2024-04-25T10:20:47.885762+0000","last_peered":"2024-04-25T10:20:47.885762+0000","last_clean":"2024-04-25T10:20:47.885762+0000","last_became_active":"2024-04-25T10:20:47.879409+0000","last_became_peered":"2024-04-25T10:20:47.879409+0000","last_unstale":"2024-04-25T10:20:47.885762+0000","last_undegraded":"2024-04-25T10:20:47.885762+0000","last_fullsized":"2024-04-25T10:20:47.885762+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:36.999978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.899431+0000","last_change":"2024-04-25T10:20:46.884683+0000","last_active":"2024-04-25T10:20:47.899431+0000","last_peered":"2024-04-25T10:20:47.899431+0000","last_clean":"2024-04-25T10:20:47.899431+0000","last_became_active":"2024-04-25T10:20:46.884265+0000","last_became_peered":"2024-04-25T10:20:46.884265+0000","last_unstale":"2024-04-25T10:20:47.899431+0000","last_undegraded":"2024-04-25T10:20:47.899431+0000","last_fullsized":"2024-04-25T10:20:47.899431+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:23.225945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.864229+0000","last_change":"2024-04-25T10:20:47.890023+0000","last_active":"2024-04-25T10:20:48.864229+0000","last_peered":"2024-04-25T10:20:48.864229+0000","last_clean":"2024-04-25T10:20:48.864229+0000","last_became_active":"2024-04-25T10:20:47.889737+0000","last_became_peered":"2024-04-25T10:20:47.889737+0000","last_unstale":"2024-04-25T10:20:48.864229+0000","last_undegraded":"2024-04-25T10:20:48.864229+0000","last_fullsized":"2024-04-25T10:20:48.864229+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:20.528597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883415+0000","last_change":"2024-04-25T10:20:46.868419+0000","last_active":"2024-04-25T10:20:48.883415+0000","last_peered":"2024-04-25T10:20:48.883415+0000","last_clean":"2024-04-25T10:20:48.883415+0000","last_became_active":"2024-04-25T10:20:46.867442+0000","last_became_peered":"2024-04-25T10:20:46.867442+0000","last_unstale":"2024-04-25T10:20:48.883415+0000","last_undegraded":"2024-04-25T10:20:48.883415+0000","last_fullsized":"2024-04-25T10:20:48.883415+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:27.478041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.878450+0000","last_change":"2024-04-25T10:20:47.873631+0000","last_active":"2024-04-25T10:20:47.878450+0000","last_peered":"2024-04-25T10:20:47.878450+0000","last_clean":"2024-04-25T10:20:47.878450+0000","last_became_active":"2024-04-25T10:20:47.872869+0000","last_became_peered":"2024-04-25T10:20:47.872869+0000","last_unstale":"2024-04-25T10:20:47.878450+0000","last_undegraded":"2024-04-25T10:20:47.878450+0000","last_fullsized":"2024-04-25T10:20:47.878450+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:11.346819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.899399+0000","last_change":"2024-04-25T10:20:46.884610+0000","last_active":"2024-04-25T10:20:47.899399+0000","last_peered":"2024-04-25T10:20:47.899399+0000","last_clean":"2024-04-25T10:20:47.899399+0000","last_became_active":"2024-04-25T10:20:46.884283+0000","last_became_peered":"2024-04-25T10:20:46.884283+0000","last_unstale":"2024-04-25T10:20:47.899399+0000","last_undegraded":"2024-04-25T10:20:47.899399+0000","last_fullsized":"2024-04-25T10:20:47.899399+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:46.601649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863112+0000","last_change":"2024-04-25T10:20:47.881733+0000","last_active":"2024-04-25T10:20:48.863112+0000","last_peered":"2024-04-25T10:20:48.863112+0000","last_clean":"2024-04-25T10:20:48.863112+0000","last_became_active":"2024-04-25T10:20:47.881437+0000","last_became_peered":"2024-04-25T10:20:47.881437+0000","last_unstale":"2024-04-25T10:20:48.863112+0000","last_undegraded":"2024-04-25T10:20:48.863112+0000","last_fullsized":"2024-04-25T10:20:48.863112+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:08.891127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883322+0000","last_change":"2024-04-25T10:20:46.868261+0000","last_active":"2024-04-25T10:20:48.883322+0000","last_peered":"2024-04-25T10:20:48.883322+0000","last_clean":"2024-04-25T10:20:48.883322+0000","last_became_active":"2024-04-25T10:20:46.866759+0000","last_became_peered":"2024-04-25T10:20:46.866759+0000","last_unstale":"2024-04-25T10:20:48.883322+0000","last_undegraded":"2024-04-25T10:20:48.883322+0000","last_fullsized":"2024-04-25T10:20:48.883322+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.406449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883163+0000","last_change":"2024-04-25T10:20:47.871713+0000","last_active":"2024-04-25T10:20:48.883163+0000","last_peered":"2024-04-25T10:20:48.883163+0000","last_clean":"2024-04-25T10:20:48.883163+0000","last_became_active":"2024-04-25T10:20:47.871386+0000","last_became_peered":"2024-04-25T10:20:47.871386+0000","last_unstale":"2024-04-25T10:20:48.883163+0000","last_undegraded":"2024-04-25T10:20:48.883163+0000","last_fullsized":"2024-04-25T10:20:48.883163+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:51.144452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.896865+0000","last_change":"2024-04-25T10:20:46.874018+0000","last_active":"2024-04-25T10:20:47.896865+0000","last_peered":"2024-04-25T10:20:47.896865+0000","last_clean":"2024-04-25T10:20:47.896865+0000","last_became_active":"2024-04-25T10:20:46.873655+0000","last_became_peered":"2024-04-25T10:20:46.873655+0000","last_unstale":"2024-04-25T10:20:47.896865+0000","last_undegraded":"2024-04-25T10:20:47.896865+0000","last_fullsized":"2024-04-25T10:20:47.896865+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.708411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888233+0000","last_change":"2024-04-25T10:20:47.884040+0000","last_active":"2024-04-25T10:20:47.888233+0000","last_peered":"2024-04-25T10:20:47.888233+0000","last_clean":"2024-04-25T10:20:47.888233+0000","last_became_active":"2024-04-25T10:20:47.883661+0000","last_became_peered":"2024-04-25T10:20:47.883661+0000","last_unstale":"2024-04-25T10:20:47.888233+0000","last_undegraded":"2024-04-25T10:20:47.888233+0000","last_fullsized":"2024-04-25T10:20:47.888233+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:57.582900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883135+0000","last_change":"2024-04-25T10:20:46.878545+0000","last_active":"2024-04-25T10:20:48.883135+0000","last_peered":"2024-04-25T10:20:48.883135+0000","last_clean":"2024-04-25T10:20:48.883135+0000","last_became_active":"2024-04-25T10:20:46.878258+0000","last_became_peered":"2024-04-25T10:20:46.878258+0000","last_unstale":"2024-04-25T10:20:48.883135+0000","last_undegraded":"2024-04-25T10:20:48.883135+0000","last_fullsized":"2024-04-25T10:20:48.883135+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.209684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.884098+0000","last_change":"2024-04-25T10:20:47.873612+0000","last_active":"2024-04-25T10:20:47.884098+0000","last_peered":"2024-04-25T10:20:47.884098+0000","last_clean":"2024-04-25T10:20:47.884098+0000","last_became_active":"2024-04-25T10:20:47.872765+0000","last_became_peered":"2024-04-25T10:20:47.872765+0000","last_unstale":"2024-04-25T10:20:47.884098+0000","last_undegraded":"2024-04-25T10:20:47.884098+0000","last_fullsized":"2024-04-25T10:20:47.884098+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:22.429987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863480+0000","last_change":"2024-04-25T10:20:46.882647+0000","last_active":"2024-04-25T10:20:48.863480+0000","last_peered":"2024-04-25T10:20:48.863480+0000","last_clean":"2024-04-25T10:20:48.863480+0000","last_became_active":"2024-04-25T10:20:46.882369+0000","last_became_peered":"2024-04-25T10:20:46.882369+0000","last_unstale":"2024-04-25T10:20:48.863480+0000","last_undegraded":"2024-04-25T10:20:48.863480+0000","last_fullsized":"2024-04-25T10:20:48.863480+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:46.601649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883220+0000","last_change":"2024-04-25T10:20:47.883202+0000","last_active":"2024-04-25T10:20:48.883220+0000","last_peered":"2024-04-25T10:20:48.883220+0000","last_clean":"2024-04-25T10:20:48.883220+0000","last_became_active":"2024-04-25T10:20:47.882924+0000","last_became_peered":"2024-04-25T10:20:47.882924+0000","last_unstale":"2024-04-25T10:20:48.883220+0000","last_undegraded":"2024-04-25T10:20:48.883220+0000","last_fullsized":"2024-04-25T10:20:48.883220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:53.654062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.862702+0000","last_change":"2024-04-25T10:20:46.877454+0000","last_active":"2024-04-25T10:20:47.862702+0000","last_peered":"2024-04-25T10:20:47.862702+0000","last_clean":"2024-04-25T10:20:47.862702+0000","last_became_active":"2024-04-25T10:20:46.877180+0000","last_became_peered":"2024-04-25T10:20:46.877180+0000","last_unstale":"2024-04-25T10:20:47.862702+0000","last_undegraded":"2024-04-25T10:20:47.862702+0000","last_fullsized":"2024-04-25T10:20:47.862702+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:30.251948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862710+0000","last_change":"2024-04-25T10:20:47.879713+0000","last_active":"2024-04-25T10:20:48.862710+0000","last_peered":"2024-04-25T10:20:48.862710+0000","last_clean":"2024-04-25T10:20:48.862710+0000","last_became_active":"2024-04-25T10:20:47.879053+0000","last_became_peered":"2024-04-25T10:20:47.879053+0000","last_unstale":"2024-04-25T10:20:48.862710+0000","last_undegraded":"2024-04-25T10:20:48.862710+0000","last_fullsized":"2024-04-25T10:20:48.862710+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:24.232279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862566+0000","last_change":"2024-04-25T10:20:46.872253+0000","last_active":"2024-04-25T10:20:48.862566+0000","last_peered":"2024-04-25T10:20:48.862566+0000","last_clean":"2024-04-25T10:20:48.862566+0000","last_became_active":"2024-04-25T10:20:46.871913+0000","last_became_peered":"2024-04-25T10:20:46.871913+0000","last_unstale":"2024-04-25T10:20:48.862566+0000","last_undegraded":"2024-04-25T10:20:48.862566+0000","last_fullsized":"2024-04-25T10:20:48.862566+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:08.544860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862557+0000","last_change":"2024-04-25T10:20:47.877536+0000","last_active":"2024-04-25T10:20:48.862557+0000","last_peered":"2024-04-25T10:20:48.862557+0000","last_clean":"2024-04-25T10:20:48.862557+0000","last_became_active":"2024-04-25T10:20:47.877259+0000","last_became_peered":"2024-04-25T10:20:47.877259+0000","last_unstale":"2024-04-25T10:20:48.862557+0000","last_undegraded":"2024-04-25T10:20:48.862557+0000","last_fullsized":"2024-04-25T10:20:48.862557+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:08.041828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.865386+0000","last_change":"2024-04-25T10:20:46.880727+0000","last_active":"2024-04-25T10:20:47.865386+0000","last_peered":"2024-04-25T10:20:47.865386+0000","last_clean":"2024-04-25T10:20:47.865386+0000","last_became_active":"2024-04-25T10:20:46.880295+0000","last_became_peered":"2024-04-25T10:20:46.880295+0000","last_unstale":"2024-04-25T10:20:47.865386+0000","last_undegraded":"2024-04-25T10:20:47.865386+0000","last_fullsized":"2024-04-25T10:20:47.865386+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:18.511050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862854+0000","last_change":"2024-04-25T10:20:47.876024+0000","last_active":"2024-04-25T10:20:48.862854+0000","last_peered":"2024-04-25T10:20:48.862854+0000","last_clean":"2024-04-25T10:20:48.862854+0000","last_became_active":"2024-04-25T10:20:47.875775+0000","last_became_peered":"2024-04-25T10:20:47.875775+0000","last_unstale":"2024-04-25T10:20:48.862854+0000","last_undegraded":"2024-04-25T10:20:48.862854+0000","last_fullsized":"2024-04-25T10:20:48.862854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:28.484374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.900398+0000","last_change":"2024-04-25T10:20:46.882814+0000","last_active":"2024-04-25T10:20:47.900398+0000","last_peered":"2024-04-25T10:20:47.900398+0000","last_clean":"2024-04-25T10:20:47.900398+0000","last_became_active":"2024-04-25T10:20:46.881112+0000","last_became_peered":"2024-04-25T10:20:46.881112+0000","last_unstale":"2024-04-25T10:20:47.900398+0000","last_undegraded":"2024-04-25T10:20:47.900398+0000","last_fullsized":"2024-04-25T10:20:47.900398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:03.182089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883450+0000","last_change":"2024-04-25T10:20:47.885057+0000","last_active":"2024-04-25T10:20:48.883450+0000","last_peered":"2024-04-25T10:20:48.883450+0000","last_clean":"2024-04-25T10:20:48.883450+0000","last_became_active":"2024-04-25T10:20:47.884002+0000","last_became_peered":"2024-04-25T10:20:47.884002+0000","last_unstale":"2024-04-25T10:20:48.883450+0000","last_undegraded":"2024-04-25T10:20:48.883450+0000","last_fullsized":"2024-04-25T10:20:48.883450+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:11.463257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.900929+0000","last_change":"2024-04-25T10:20:46.879675+0000","last_active":"2024-04-25T10:20:47.900929+0000","last_peered":"2024-04-25T10:20:47.900929+0000","last_clean":"2024-04-25T10:20:47.900929+0000","last_became_active":"2024-04-25T10:20:46.878328+0000","last_became_peered":"2024-04-25T10:20:46.878328+0000","last_unstale":"2024-04-25T10:20:47.900929+0000","last_undegraded":"2024-04-25T10:20:47.900929+0000","last_fullsized":"2024-04-25T10:20:47.900929+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:47.346556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862276+0000","last_change":"2024-04-25T10:20:47.875959+0000","last_active":"2024-04-25T10:20:48.862276+0000","last_peered":"2024-04-25T10:20:48.862276+0000","last_clean":"2024-04-25T10:20:48.862276+0000","last_became_active":"2024-04-25T10:20:47.875677+0000","last_became_peered":"2024-04-25T10:20:47.875677+0000","last_unstale":"2024-04-25T10:20:48.862276+0000","last_undegraded":"2024-04-25T10:20:48.862276+0000","last_fullsized":"2024-04-25T10:20:48.862276+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:55.331215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.898861+0000","last_change":"2024-04-25T10:20:46.875351+0000","last_active":"2024-04-25T10:20:47.898861+0000","last_peered":"2024-04-25T10:20:47.898861+0000","last_clean":"2024-04-25T10:20:47.898861+0000","last_became_active":"2024-04-25T10:20:46.875082+0000","last_became_peered":"2024-04-25T10:20:46.875082+0000","last_unstale":"2024-04-25T10:20:47.898861+0000","last_undegraded":"2024-04-25T10:20:47.898861+0000","last_fullsized":"2024-04-25T10:20:47.898861+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:39.217749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.886784+0000","last_change":"2024-04-25T10:20:47.878163+0000","last_active":"2024-04-25T10:20:47.886784+0000","last_peered":"2024-04-25T10:20:47.886784+0000","last_clean":"2024-04-25T10:20:47.886784+0000","last_became_active":"2024-04-25T10:20:47.877858+0000","last_became_peered":"2024-04-25T10:20:47.877858+0000","last_unstale":"2024-04-25T10:20:47.886784+0000","last_undegraded":"2024-04-25T10:20:47.886784+0000","last_fullsized":"2024-04-25T10:20:47.886784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:48.758822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863307+0000","last_change":"2024-04-25T10:20:46.885774+0000","last_active":"2024-04-25T10:20:48.863307+0000","last_peered":"2024-04-25T10:20:48.863307+0000","last_clean":"2024-04-25T10:20:48.863307+0000","last_became_active":"2024-04-25T10:20:46.885362+0000","last_became_peered":"2024-04-25T10:20:46.885362+0000","last_unstale":"2024-04-25T10:20:48.863307+0000","last_undegraded":"2024-04-25T10:20:48.863307+0000","last_fullsized":"2024-04-25T10:20:48.863307+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:07.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.890509+0000","last_change":"2024-04-25T10:20:47.887088+0000","last_active":"2024-04-25T10:20:47.890509+0000","last_peered":"2024-04-25T10:20:47.890509+0000","last_clean":"2024-04-25T10:20:47.890509+0000","last_became_active":"2024-04-25T10:20:47.886801+0000","last_became_peered":"2024-04-25T10:20:47.886801+0000","last_unstale":"2024-04-25T10:20:47.890509+0000","last_undegraded":"2024-04-25T10:20:47.890509+0000","last_fullsized":"2024-04-25T10:20:47.890509+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.799875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.876678+0000","last_change":"2024-04-25T10:20:46.875036+0000","last_active":"2024-04-25T10:20:47.876678+0000","last_peered":"2024-04-25T10:20:47.876678+0000","last_clean":"2024-04-25T10:20:47.876678+0000","last_became_active":"2024-04-25T10:20:46.874710+0000","last_became_peered":"2024-04-25T10:20:46.874710+0000","last_unstale":"2024-04-25T10:20:47.876678+0000","last_undegraded":"2024-04-25T10:20:47.876678+0000","last_fullsized":"2024-04-25T10:20:47.876678+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:59.488107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.886120+0000","last_change":"2024-04-25T10:20:47.878068+0000","last_active":"2024-04-25T10:20:47.886120+0000","last_peered":"2024-04-25T10:20:47.886120+0000","last_clean":"2024-04-25T10:20:47.886120+0000","last_became_active":"2024-04-25T10:20:47.877786+0000","last_became_peered":"2024-04-25T10:20:47.877786+0000","last_unstale":"2024-04-25T10:20:47.886120+0000","last_undegraded":"2024-04-25T10:20:47.886120+0000","last_fullsized":"2024-04-25T10:20:47.886120+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.898888+0000","last_change":"2024-04-25T10:20:46.878325+0000","last_active":"2024-04-25T10:20:47.898888+0000","last_peered":"2024-04-25T10:20:47.898888+0000","last_clean":"2024-04-25T10:20:47.898888+0000","last_became_active":"2024-04-25T10:20:46.877816+0000","last_became_peered":"2024-04-25T10:20:46.877816+0000","last_unstale":"2024-04-25T10:20:47.898888+0000","last_undegraded":"2024-04-25T10:20:47.898888+0000","last_fullsized":"2024-04-25T10:20:47.898888+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:13.946309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.889560+0000","last_change":"2024-04-25T10:20:47.882566+0000","last_active":"2024-04-25T10:20:47.889560+0000","last_peered":"2024-04-25T10:20:47.889560+0000","last_clean":"2024-04-25T10:20:47.889560+0000","last_became_active":"2024-04-25T10:20:47.882204+0000","last_became_peered":"2024-04-25T10:20:47.882204+0000","last_unstale":"2024-04-25T10:20:47.889560+0000","last_undegraded":"2024-04-25T10:20:47.889560+0000","last_fullsized":"2024-04-25T10:20:47.889560+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:51.022546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863056+0000","last_change":"2024-04-25T10:20:46.878072+0000","last_active":"2024-04-25T10:20:48.863056+0000","last_peered":"2024-04-25T10:20:48.863056+0000","last_clean":"2024-04-25T10:20:48.863056+0000","last_became_active":"2024-04-25T10:20:46.877827+0000","last_became_peered":"2024-04-25T10:20:46.877827+0000","last_unstale":"2024-04-25T10:20:48.863056+0000","last_undegraded":"2024-04-25T10:20:48.863056+0000","last_fullsized":"2024-04-25T10:20:48.863056+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:07.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888997+0000","last_change":"2024-04-25T10:20:47.883311+0000","last_active":"2024-04-25T10:20:47.888997+0000","last_peered":"2024-04-25T10:20:47.888997+0000","last_clean":"2024-04-25T10:20:47.888997+0000","last_became_active":"2024-04-25T10:20:47.883065+0000","last_became_peered":"2024-04-25T10:20:47.883065+0000","last_unstale":"2024-04-25T10:20:47.888997+0000","last_undegraded":"2024-04-25T10:20:47.888997+0000","last_fullsized":"2024-04-25T10:20:47.888997+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:11.110634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862796+0000","last_change":"2024-04-25T10:20:46.875942+0000","last_active":"2024-04-25T10:20:48.862796+0000","last_peered":"2024-04-25T10:20:48.862796+0000","last_clean":"2024-04-25T10:20:48.862796+0000","last_became_active":"2024-04-25T10:20:46.875665+0000","last_became_peered":"2024-04-25T10:20:46.875665+0000","last_unstale":"2024-04-25T10:20:48.862796+0000","last_undegraded":"2024-04-25T10:20:48.862796+0000","last_fullsized":"2024-04-25T10:20:48.862796+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:19.347047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.889005+0000","last_change":"2024-04-25T10:20:47.884126+0000","last_active":"2024-04-25T10:20:47.889005+0000","last_peered":"2024-04-25T10:20:47.889005+0000","last_clean":"2024-04-25T10:20:47.889005+0000","last_became_active":"2024-04-25T10:20:47.881206+0000","last_became_peered":"2024-04-25T10:20:47.881206+0000","last_unstale":"2024-04-25T10:20:47.889005+0000","last_undegraded":"2024-04-25T10:20:47.889005+0000","last_fullsized":"2024-04-25T10:20:47.889005+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:50.194709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862442+0000","last_change":"2024-04-25T10:20:46.882965+0000","last_active":"2024-04-25T10:20:48.862442+0000","last_peered":"2024-04-25T10:20:48.862442+0000","last_clean":"2024-04-25T10:20:48.862442+0000","last_became_active":"2024-04-25T10:20:46.882648+0000","last_became_peered":"2024-04-25T10:20:46.882648+0000","last_unstale":"2024-04-25T10:20:48.862442+0000","last_undegraded":"2024-04-25T10:20:48.862442+0000","last_fullsized":"2024-04-25T10:20:48.862442+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:22.756799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862467+0000","last_change":"2024-04-25T10:20:47.874222+0000","last_active":"2024-04-25T10:20:48.862467+0000","last_peered":"2024-04-25T10:20:48.862467+0000","last_clean":"2024-04-25T10:20:48.862467+0000","last_became_active":"2024-04-25T10:20:47.873906+0000","last_became_peered":"2024-04-25T10:20:47.873906+0000","last_unstale":"2024-04-25T10:20:48.862467+0000","last_undegraded":"2024-04-25T10:20:48.862467+0000","last_fullsized":"2024-04-25T10:20:48.862467+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:47.607983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.876083+0000","last_change":"2024-04-25T10:20:46.875643+0000","last_active":"2024-04-25T10:20:47.876083+0000","last_peered":"2024-04-25T10:20:47.876083+0000","last_clean":"2024-04-25T10:20:47.876083+0000","last_became_active":"2024-04-25T10:20:46.874700+0000","last_became_peered":"2024-04-25T10:20:46.874700+0000","last_unstale":"2024-04-25T10:20:47.876083+0000","last_undegraded":"2024-04-25T10:20:47.876083+0000","last_fullsized":"2024-04-25T10:20:47.876083+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:07.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.896069+0000","last_change":"2024-04-25T10:20:47.888777+0000","last_active":"2024-04-25T10:20:47.896069+0000","last_peered":"2024-04-25T10:20:47.896069+0000","last_clean":"2024-04-25T10:20:47.896069+0000","last_became_active":"2024-04-25T10:20:47.887910+0000","last_became_peered":"2024-04-25T10:20:47.887910+0000","last_unstale":"2024-04-25T10:20:47.896069+0000","last_undegraded":"2024-04-25T10:20:47.896069+0000","last_fullsized":"2024-04-25T10:20:47.896069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:07.838343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.896944+0000","last_change":"2024-04-25T10:20:46.880209+0000","last_active":"2024-04-25T10:20:47.896944+0000","last_peered":"2024-04-25T10:20:47.896944+0000","last_clean":"2024-04-25T10:20:47.896944+0000","last_became_active":"2024-04-25T10:20:46.879794+0000","last_became_peered":"2024-04-25T10:20:46.879794+0000","last_unstale":"2024-04-25T10:20:47.896944+0000","last_undegraded":"2024-04-25T10:20:47.896944+0000","last_fullsized":"2024-04-25T10:20:47.896944+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:56.254801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.888968+0000","last_change":"2024-04-25T10:20:47.880626+0000","last_active":"2024-04-25T10:20:47.888968+0000","last_peered":"2024-04-25T10:20:47.888968+0000","last_clean":"2024-04-25T10:20:47.888968+0000","last_became_active":"2024-04-25T10:20:47.880291+0000","last_became_peered":"2024-04-25T10:20:47.880291+0000","last_unstale":"2024-04-25T10:20:47.888968+0000","last_undegraded":"2024-04-25T10:20:47.888968+0000","last_fullsized":"2024-04-25T10:20:47.888968+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:02.162348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863332+0000","last_change":"2024-04-25T10:20:46.881826+0000","last_active":"2024-04-25T10:20:48.863332+0000","last_peered":"2024-04-25T10:20:48.863332+0000","last_clean":"2024-04-25T10:20:48.863332+0000","last_became_active":"2024-04-25T10:20:46.881427+0000","last_became_peered":"2024-04-25T10:20:46.881427+0000","last_unstale":"2024-04-25T10:20:48.863332+0000","last_undegraded":"2024-04-25T10:20:48.863332+0000","last_fullsized":"2024-04-25T10:20:48.863332+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:16.251360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862436+0000","last_change":"2024-04-25T10:20:47.872684+0000","last_active":"2024-04-25T10:20:48.862436+0000","last_peered":"2024-04-25T10:20:48.862436+0000","last_clean":"2024-04-25T10:20:48.862436+0000","last_became_active":"2024-04-25T10:20:47.872389+0000","last_became_peered":"2024-04-25T10:20:47.872389+0000","last_unstale":"2024-04-25T10:20:48.862436+0000","last_undegraded":"2024-04-25T10:20:48.862436+0000","last_fullsized":"2024-04-25T10:20:48.862436+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:12.303794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.865070+0000","last_change":"2024-04-25T10:20:46.877213+0000","last_active":"2024-04-25T10:20:47.865070+0000","last_peered":"2024-04-25T10:20:47.865070+0000","last_clean":"2024-04-25T10:20:47.865070+0000","last_became_active":"2024-04-25T10:20:46.876923+0000","last_became_peered":"2024-04-25T10:20:46.876923+0000","last_unstale":"2024-04-25T10:20:47.865070+0000","last_undegraded":"2024-04-25T10:20:47.865070+0000","last_fullsized":"2024-04-25T10:20:47.865070+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:36.827454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862689+0000","last_change":"2024-04-25T10:20:47.875310+0000","last_active":"2024-04-25T10:20:48.862689+0000","last_peered":"2024-04-25T10:20:48.862689+0000","last_clean":"2024-04-25T10:20:48.862689+0000","last_became_active":"2024-04-25T10:20:47.875049+0000","last_became_peered":"2024-04-25T10:20:47.875049+0000","last_unstale":"2024-04-25T10:20:48.862689+0000","last_undegraded":"2024-04-25T10:20:48.862689+0000","last_fullsized":"2024-04-25T10:20:48.862689+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:14.216017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.882972+0000","last_change":"2024-04-25T10:20:46.871304+0000","last_active":"2024-04-25T10:20:48.882972+0000","last_peered":"2024-04-25T10:20:48.882972+0000","last_clean":"2024-04-25T10:20:48.882972+0000","last_became_active":"2024-04-25T10:20:46.871031+0000","last_became_peered":"2024-04-25T10:20:46.871031+0000","last_unstale":"2024-04-25T10:20:48.882972+0000","last_undegraded":"2024-04-25T10:20:48.882972+0000","last_fullsized":"2024-04-25T10:20:48.882972+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:23.302094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883417+0000","last_change":"2024-04-25T10:20:47.882033+0000","last_active":"2024-04-25T10:20:48.883417+0000","last_peered":"2024-04-25T10:20:48.883417+0000","last_clean":"2024-04-25T10:20:48.883417+0000","last_became_active":"2024-04-25T10:20:47.881763+0000","last_became_peered":"2024-04-25T10:20:47.881763+0000","last_unstale":"2024-04-25T10:20:48.883417+0000","last_undegraded":"2024-04-25T10:20:48.883417+0000","last_fullsized":"2024-04-25T10:20:48.883417+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:09.190412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.881312+0000","last_change":"2024-04-25T10:20:46.870518+0000","last_active":"2024-04-25T10:20:48.881312+0000","last_peered":"2024-04-25T10:20:48.881312+0000","last_clean":"2024-04-25T10:20:48.881312+0000","last_became_active":"2024-04-25T10:20:46.870240+0000","last_became_peered":"2024-04-25T10:20:46.870240+0000","last_unstale":"2024-04-25T10:20:48.881312+0000","last_undegraded":"2024-04-25T10:20:48.881312+0000","last_fullsized":"2024-04-25T10:20:48.881312+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.708411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.882921+0000","last_change":"2024-04-25T10:20:47.879506+0000","last_active":"2024-04-25T10:20:47.882921+0000","last_peered":"2024-04-25T10:20:47.882921+0000","last_clean":"2024-04-25T10:20:47.882921+0000","last_became_active":"2024-04-25T10:20:47.879244+0000","last_became_peered":"2024-04-25T10:20:47.879244+0000","last_unstale":"2024-04-25T10:20:47.882921+0000","last_undegraded":"2024-04-25T10:20:47.882921+0000","last_fullsized":"2024-04-25T10:20:47.882921+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:25.270796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863934+0000","last_change":"2024-04-25T10:20:46.881532+0000","last_active":"2024-04-25T10:20:48.863934+0000","last_peered":"2024-04-25T10:20:48.863934+0000","last_clean":"2024-04-25T10:20:48.863934+0000","last_became_active":"2024-04-25T10:20:46.881181+0000","last_became_peered":"2024-04-25T10:20:46.881181+0000","last_unstale":"2024-04-25T10:20:48.863934+0000","last_undegraded":"2024-04-25T10:20:48.863934+0000","last_fullsized":"2024-04-25T10:20:48.863934+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.708411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883513+0000","last_change":"2024-04-25T10:20:47.884301+0000","last_active":"2024-04-25T10:20:48.883513+0000","last_peered":"2024-04-25T10:20:48.883513+0000","last_clean":"2024-04-25T10:20:48.883513+0000","last_became_active":"2024-04-25T10:20:47.883654+0000","last_became_peered":"2024-04-25T10:20:47.883654+0000","last_unstale":"2024-04-25T10:20:48.883513+0000","last_undegraded":"2024-04-25T10:20:48.883513+0000","last_fullsized":"2024-04-25T10:20:48.883513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:55.385973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.871544+0000","last_change":"2024-04-25T10:20:46.871719+0000","last_active":"2024-04-25T10:20:47.871544+0000","last_peered":"2024-04-25T10:20:47.871544+0000","last_clean":"2024-04-25T10:20:47.871544+0000","last_became_active":"2024-04-25T10:20:46.871441+0000","last_became_peered":"2024-04-25T10:20:46.871441+0000","last_unstale":"2024-04-25T10:20:47.871544+0000","last_undegraded":"2024-04-25T10:20:47.871544+0000","last_fullsized":"2024-04-25T10:20:47.871544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:00.444923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.889535+0000","last_change":"2024-04-25T10:20:47.884506+0000","last_active":"2024-04-25T10:20:47.889535+0000","last_peered":"2024-04-25T10:20:47.889535+0000","last_clean":"2024-04-25T10:20:47.889535+0000","last_became_active":"2024-04-25T10:20:47.882427+0000","last_became_peered":"2024-04-25T10:20:47.882427+0000","last_unstale":"2024-04-25T10:20:47.889535+0000","last_undegraded":"2024-04-25T10:20:47.889535+0000","last_fullsized":"2024-04-25T10:20:47.889535+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:23.249153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863435+0000","last_change":"2024-04-25T10:20:46.883106+0000","last_active":"2024-04-25T10:20:48.863435+0000","last_peered":"2024-04-25T10:20:48.863435+0000","last_clean":"2024-04-25T10:20:48.863435+0000","last_became_active":"2024-04-25T10:20:46.882853+0000","last_became_peered":"2024-04-25T10:20:46.882853+0000","last_unstale":"2024-04-25T10:20:48.863435+0000","last_undegraded":"2024-04-25T10:20:48.863435+0000","last_fullsized":"2024-04-25T10:20:48.863435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:23.225945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.862270+0000","last_change":"2024-04-25T10:20:47.873403+0000","last_active":"2024-04-25T10:20:48.862270+0000","last_peered":"2024-04-25T10:20:48.862270+0000","last_clean":"2024-04-25T10:20:48.862270+0000","last_became_active":"2024-04-25T10:20:47.873045+0000","last_became_peered":"2024-04-25T10:20:47.873045+0000","last_unstale":"2024-04-25T10:20:48.862270+0000","last_undegraded":"2024-04-25T10:20:48.862270+0000","last_fullsized":"2024-04-25T10:20:48.862270+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:30.930965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883247+0000","last_change":"2024-04-25T10:20:46.880970+0000","last_active":"2024-04-25T10:20:48.883247+0000","last_peered":"2024-04-25T10:20:48.883247+0000","last_clean":"2024-04-25T10:20:48.883247+0000","last_became_active":"2024-04-25T10:20:46.880637+0000","last_became_peered":"2024-04-25T10:20:46.880637+0000","last_unstale":"2024-04-25T10:20:48.883247+0000","last_undegraded":"2024-04-25T10:20:48.883247+0000","last_fullsized":"2024-04-25T10:20:48.883247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:27.453194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.861927+0000","last_change":"2024-04-25T10:20:47.876723+0000","last_active":"2024-04-25T10:20:48.861927+0000","last_peered":"2024-04-25T10:20:48.861927+0000","last_clean":"2024-04-25T10:20:48.861927+0000","last_became_active":"2024-04-25T10:20:47.876394+0000","last_became_peered":"2024-04-25T10:20:47.876394+0000","last_unstale":"2024-04-25T10:20:48.861927+0000","last_undegraded":"2024-04-25T10:20:48.861927+0000","last_fullsized":"2024-04-25T10:20:48.861927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:37.799875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.875912+0000","last_change":"2024-04-25T10:20:46.874300+0000","last_active":"2024-04-25T10:20:47.875912+0000","last_peered":"2024-04-25T10:20:47.875912+0000","last_clean":"2024-04-25T10:20:47.875912+0000","last_became_active":"2024-04-25T10:20:46.873743+0000","last_became_peered":"2024-04-25T10:20:46.873743+0000","last_unstale":"2024-04-25T10:20:47.875912+0000","last_undegraded":"2024-04-25T10:20:47.875912+0000","last_fullsized":"2024-04-25T10:20:47.875912+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:16.251360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.867092+0000","last_change":"2024-04-25T10:20:47.865801+0000","last_active":"2024-04-25T10:20:47.867092+0000","last_peered":"2024-04-25T10:20:47.867092+0000","last_clean":"2024-04-25T10:20:47.867092+0000","last_became_active":"2024-04-25T10:20:47.865504+0000","last_became_peered":"2024-04-25T10:20:47.865504+0000","last_unstale":"2024-04-25T10:20:47.867092+0000","last_undegraded":"2024-04-25T10:20:47.867092+0000","last_fullsized":"2024-04-25T10:20:47.867092+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:07.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:20:47.871050+0000","last_change":"2024-04-25T10:20:46.881160+0000","last_active":"2024-04-25T10:20:47.871050+0000","last_peered":"2024-04-25T10:20:47.871050+0000","last_clean":"2024-04-25T10:20:47.871050+0000","last_became_active":"2024-04-25T10:20:46.880707+0000","last_became_peered":"2024-04-25T10:20:46.880707+0000","last_unstale":"2024-04-25T10:20:47.871050+0000","last_undegraded":"2024-04-25T10:20:47.871050+0000","last_fullsized":"2024-04-25T10:20:47.871050+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:37.615463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.863064+0000","last_change":"2024-04-25T10:20:47.876620+0000","last_active":"2024-04-25T10:20:48.863064+0000","last_peered":"2024-04-25T10:20:48.863064+0000","last_clean":"2024-04-25T10:20:48.863064+0000","last_became_active":"2024-04-25T10:20:47.876311+0000","last_became_peered":"2024-04-25T10:20:47.876311+0000","last_unstale":"2024-04-25T10:20:48.863064+0000","last_undegraded":"2024-04-25T10:20:48.863064+0000","last_fullsized":"2024-04-25T10:20:48.863064+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:46.846599+0000","last_clean_scrub_stamp":"2024-04-25T10:20:46.846599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:36.968970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.883428+0000","last_change":"2024-04-25T10:20:46.869232+0000","last_active":"2024-04-25T10:20:48.883428+0000","last_peered":"2024-04-25T10:20:48.883428+0000","last_clean":"2024-04-25T10:20:48.883428+0000","last_became_active":"2024-04-25T10:20:46.868946+0000","last_became_peered":"2024-04-25T10:20:46.868946+0000","last_unstale":"2024-04-25T10:20:48.883428+0000","last_undegraded":"2024-04-25T10:20:48.883428+0000","last_fullsized":"2024-04-25T10:20:48.883428+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:45.840266+0000","last_clean_scrub_stamp":"2024-04-25T10:20:45.840266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:37.473698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":224,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:20:48.861892+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:20:48.861892+0000","last_peered":"2024-04-25T10:20:48.861892+0000","last_clean":"2024-04-25T10:20:48.861892+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:20:48.861892+0000","last_undegraded":"2024-04-25T10:20:48.861892+0000","last_fullsized":"2024-04-25T10:20:48.861892+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":573440,"data_stored":233469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35912,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335928,"statfs":{"total":96636764160,"available":96599990272,"internally_reserved":0,"allocated":577536,"data_stored":234051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40648,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331192,"statfs":{"total":96636764160,"available":96595140608,"internally_reserved":0,"allocated":1167360,"data_stored":823825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40068,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331772,"statfs":{"total":96636764160,"available":96595734528,"internally_reserved":0,"allocated":581632,"data_stored":235047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40060,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331780,"statfs":{"total":96636764160,"available":96595742720,"internally_reserved":0,"allocated":573440,"data_stored":233525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1167360,"data_stored":823873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":577536,"data_stored":234107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40056,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331784,"statfs":{"total":96636764160,"available":96595746816,"internally_reserved":0,"allocated":569344,"data_stored":233435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:52.371 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:20:52.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:20:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.555+0000 7f3796800640 1 Processor -- start 2024-04-25T10:20:52.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.555+0000 7f3796800640 1 -- start start 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3796800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37900d1510 0x7f37900b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3796800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37900cfbb0 0x7f37900cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3796800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37900d0170 con 0x7f37900d18f0 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3796800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900d03e0 con 0x7f37900cfbb0 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3796800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900d06c0 con 0x7f37900d1510 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3794a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37900cfbb0 0x7f37900cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3795e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f37900d18f0 0x7f379006bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46230/0 (socket says 172.21.15.45:46230) 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3795e00640 1 -- 172.21.15.45:0/891642895 learned_addr learned my addr 172.21.15.45:0/891642895 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3795400640 1 --2- 172.21.15.45:0/891642895 >> v2:172.21.15.117:3300/0 conn(0x7f37900d1510 0x7f37900b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3794a00640 1 -- 172.21.15.45:0/891642895 >> v2:172.21.15.117:3300/0 conn(0x7f37900d1510 msgr2=0x7f37900b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3794a00640 1 --2- 172.21.15.45:0/891642895 >> v2:172.21.15.117:3300/0 conn(0x7f37900d1510 0x7f37900b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3794a00640 1 -- 172.21.15.45:0/891642895 >> v1:172.21.15.45:6789/0 conn(0x7f37900d18f0 legacy=0x7f379006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3794a00640 1 -- 172.21.15.45:0/891642895 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379006b5f0 con 0x7f37900cfbb0 2024-04-25T10:20:52.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.559+0000 7f3795400640 1 --2- 172.21.15.45:0/891642895 >> v2:172.21.15.117:3300/0 conn(0x7f37900d1510 0x7f37900b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:52.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3794a00640 1 --2- 172.21.15.45:0/891642895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900cfbb0 0x7f37900cea70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3790072fd0 tx=0x7f377c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8a6c65297cad6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f378fe00640 1 -- 172.21.15.45:0/891642895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0a7070 con 0x7f37900cfbb0 2024-04-25T10:20:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f378fe00640 1 -- 172.21.15.45:0/891642895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c0b0070 con 0x7f37900cfbb0 2024-04-25T10:20:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f378fe00640 1 -- 172.21.15.45:0/891642895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0ab890 con 0x7f37900cfbb0 2024-04-25T10:20:52.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- 172.21.15.45:0/891642895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900cfbb0 msgr2=0x7f37900cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 --2- 172.21.15.45:0/891642895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900cfbb0 0x7f37900cea70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3790072fd0 tx=0x7f377c01c450 comp rx=0 tx=0).stop 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- 172.21.15.45:0/891642895 shutdown_connections 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 --2- 172.21.15.45:0/891642895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900cfbb0 0x7f37900cea70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 --2- 172.21.15.45:0/891642895 >> v2:172.21.15.117:3300/0 conn(0x7f37900d1510 0x7f37900b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- 172.21.15.45:0/891642895 >> 172.21.15.45:0/891642895 conn(0x7f3790072820 msgr2=0x7f379009a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- 172.21.15.45:0/891642895 shutdown_connections 2024-04-25T10:20:52.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- 172.21.15.45:0/891642895 wait complete. 2024-04-25T10:20:52.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 Processor -- start 2024-04-25T10:20:52.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- start start 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37900cfbb0 0x7f379015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37900d1510 0x7f3790157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 0x7f37900b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900b28c0 con 0x7f37900d18f0 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900b2740 con 0x7f37900d1510 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3796800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37900d1d00 con 0x7f37900cfbb0 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.563+0000 7f3795e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 0x7f37900b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 0x7f37900b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54272/0 (socket says 172.21.15.45:54272) 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 -- 172.21.15.45:0/389468869 learned_addr learned my addr 172.21.15.45:0/389468869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3794a00640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37900d1510 0x7f3790157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795400640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37900cfbb0 0x7f379015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 -- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37900cfbb0 msgr2=0x7f379015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37900cfbb0 0x7f379015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 -- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37900d1510 msgr2=0x7f3790157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37900d1510 0x7f3790157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 -- 172.21.15.45:0/389468869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379015d100 con 0x7f37900d18f0 2024-04-25T10:20:52.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3795e00640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 0x7f37900b43d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f378800ca80 tx=0x7f378800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788017070 con 0x7f37900d18f0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378800ac30 con 0x7f37900d18f0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37900b7150 con 0x7f37900d18f0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788012600 con 0x7f37900d18f0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379015de00 con 0x7f37900d18f0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37900d2690 con 0x7f37900d18f0 2024-04-25T10:20:52.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f378801a070 con 0x7f37900d18f0 2024-04-25T10:20:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.567+0000 7f378d600640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3770066f60 0x7f3770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.571+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3788084cb0 con 0x7f37900d18f0 2024-04-25T10:20:52.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.571+0000 7f3795400640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3770066f60 0x7f3770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.571+0000 7f3795400640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3770066f60 0x7f3770069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3784009950 tx=0x7f3784008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.571+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f378800ae20 con 0x7f37900d18f0 2024-04-25T10:20:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.795+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f37900687c0 con 0x7f37900d18f0 2024-04-25T10:20:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f378d600640 1 -- 172.21.15.45:0/389468869 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7f37880555d0 con 0x7f37900d18f0 2024-04-25T10:20:52.801 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:20:52.801 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:20:52.801 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:20:52.801 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:20:52.801 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:52.802 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:20:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3770066f60 msgr2=0x7f3770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3770066f60 0x7f3770069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3784009950 tx=0x7f3784008040 comp rx=0 tx=0).stop 2024-04-25T10:20:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 msgr2=0x7f37900b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 0x7f37900b43d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f378800ca80 tx=0x7f378800cf40 comp rx=0 tx=0).stop 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 shutdown_connections 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3770066f60 0x7f3770069420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37900d18f0 0x7f37900b43d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37900d1510 0x7f3790157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 --2- 172.21.15.45:0/389468869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37900cfbb0 0x7f379015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 >> 172.21.15.45:0/389468869 conn(0x7f3790072820 msgr2=0x7f37900ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 shutdown_connections 2024-04-25T10:20:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.799+0000 7f3796800640 1 -- 172.21.15.45:0/389468869 wait complete. 2024-04-25T10:20:52.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 Processor -- start 2024-04-25T10:20:52.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 -- start start 2024-04-25T10:20:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 0x7f33dc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33dc15eea0 0x7f33dc0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc15e400 con 0x7f33dc15d9b0 2024-04-25T10:20:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc0b02e0 con 0x7f33dc15eea0 2024-04-25T10:20:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33e0e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0afda0 con 0x7f33dc0b38d0 2024-04-25T10:20:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f33dc15d9b0 0x7f33dc0af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46242/0 (socket says 172.21.15.45:46242) 2024-04-25T10:20:52.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9e00640 1 -- 172.21.15.45:0/362282386 learned_addr learned my addr 172.21.15.45:0/362282386 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 --2- 172.21.15.45:0/362282386 >> v2:172.21.15.117:3300/0 conn(0x7f33dc15eea0 0x7f33dc0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307508979 0 0) 0x7f33dc15e400 con 0x7f33dc15d9b0 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d3e00640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 0x7f33dc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c0003530 con 0x7f33dc15d9b0 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038095184 0 0) 0x7f33c0003530 con 0x7f33dc15d9b0 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 msgr2=0x7f33dc15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 0x7f33dc15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 >> v2:172.21.15.117:3300/0 conn(0x7f33dc15eea0 msgr2=0x7f33dc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 --2- 172.21.15.45:0/362282386 >> v2:172.21.15.117:3300/0 conn(0x7f33dc15eea0 0x7f33dc0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0b0000 con 0x7f33dc15d9b0 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f33cc0016a0 con 0x7f33dc15d9b0 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 >> v1:172.21.15.45:6789/0 conn(0x7f33dc15d9b0 legacy=0x7f33dc0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c00039c0 0x7f33c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 0x7f33c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0b02e0 con 0x7f33c0004d50 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0afda0 con 0x7f33c00039c0 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c000ba00 con 0x7f33dc0b38d0 2024-04-25T10:20:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d9e00640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c00039c0 0x7f33c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33d3e00640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 0x7f33c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 -- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 msgr2=0x7f33c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc0b38d0 0x7f33c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 -- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c00039c0 msgr2=0x7f33c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c00039c0 0x7f33c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.963+0000 7f33da800640 1 -- 172.21.15.45:0/362282386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c0003530 con 0x7f33c0004d50 2024-04-25T10:20:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33da800640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33c00051d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009fc0 tx=0x7f33d401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353829939f833906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40b2070 con 0x7f33c0004d50 2024-04-25T10:20:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d40a5cd0 con 0x7f33c0004d50 2024-04-25T10:20:52.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33d9400640 1 -- 172.21.15.45:0/362282386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4009c10 con 0x7f33c0004d50 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33e0e00640 1 -- 172.21.15.45:0/362282386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 msgr2=0x7f33c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33e0e00640 1 --2- 172.21.15.45:0/362282386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33c00051d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009fc0 tx=0x7f33d401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33da800640 1 -- 172.21.15.45:0/362282386 reap_dead start 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33e0e00640 1 -- 172.21.15.45:0/362282386 shutdown_connections 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33e0e00640 1 -- 172.21.15.45:0/362282386 >> 172.21.15.45:0/362282386 conn(0x7f33dc06ac90 msgr2=0x7f33dc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33e0e00640 1 -- 172.21.15.45:0/362282386 shutdown_connections 2024-04-25T10:20:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.967+0000 7f33e0e00640 1 -- 172.21.15.45:0/362282386 wait complete. 2024-04-25T10:20:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 Processor -- start 2024-04-25T10:20:52.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 -- start start 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33dc156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc147290 0x7f33dc1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33dc160570 0x7f33dc160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc15dd90 con 0x7f33c0004d50 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc15e980 con 0x7f33dc160570 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc15df10 con 0x7f33dc147290 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33dc156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33dc156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54290/0 (socket says 172.21.15.45:54290) 2024-04-25T10:20:52.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 -- 172.21.15.45:0/474784554 learned_addr learned my addr 172.21.15.45:0/474784554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d3e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc147290 0x7f33dc1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33da800640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33dc160570 0x7f33dc160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 -- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc147290 msgr2=0x7f33dc1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc147290 0x7f33dc1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 -- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33dc160570 msgr2=0x7f33dc160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33dc160570 0x7f33dc160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 -- 172.21.15.45:0/474784554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc1f3c80 con 0x7f33c0004d50 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33da800640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33dc160570 0x7f33dc160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d9e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33dc156f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33cc007c80 tx=0x7f33cc0126b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc01c070 con 0x7f33c0004d50 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc014770 con 0x7f33c0004d50 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc014b60 con 0x7f33c0004d50 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc1f3f60 con 0x7f33c0004d50 2024-04-25T10:20:52.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.971+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc1f4520 con 0x7f33c0004d50 2024-04-25T10:20:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.975+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc04ef30 con 0x7f33c0004d50 2024-04-25T10:20:52.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.975+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33cc014d00 con 0x7f33c0004d50 2024-04-25T10:20:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.975+0000 7f33d2000640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b8066f90 0x7f33b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.975+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33cc0878b0 con 0x7f33c0004d50 2024-04-25T10:20:52.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.975+0000 7f33d3e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b8066f90 0x7f33b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:52.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.979+0000 7f33d3e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b8066f90 0x7f33b8069450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33dc158be0 tx=0x7f33c4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:52.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:52.979+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33cc0588b0 con 0x7f33c0004d50 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.175+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33dc1f6e40 con 0x7f33c0004d50 2024-04-25T10:20:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33d2000640 1 -- 172.21.15.45:0/474784554 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f33cc0581b0 con 0x7f33c0004d50 2024-04-25T10:20:53.181 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:53.181 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:20:53.181 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:20:53.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b8066f90 msgr2=0x7f33b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b8066f90 0x7f33b8069450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33dc158be0 tx=0x7f33c4009190 comp rx=0 tx=0).stop 2024-04-25T10:20:53.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 msgr2=0x7f33dc156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33dc156f20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33cc007c80 tx=0x7f33cc0126b0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 shutdown_connections 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b8066f90 0x7f33b8069450 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33dc160570 0x7f33dc160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33dc147290 0x7f33dc1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 --2- 172.21.15.45:0/474784554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33c0004d50 0x7f33dc156f20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 >> 172.21.15.45:0/474784554 conn(0x7f33dc06ac90 msgr2=0x7f33dc15fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 shutdown_connections 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.179+0000 7f33e0e00640 1 -- 172.21.15.45:0/474784554 wait complete. 2024-04-25T10:20:53.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:47.262741+0000', 'modified': '2024-04-25T10:20:50.288582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5574}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5574': {'gid': 5574, 'name': 'd', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6836/3308184634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 3308184634}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 3308184634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:53.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:20:53.606 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:20:53.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 Processor -- start 2024-04-25T10:20:53.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 -- start start 2024-04-25T10:20:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b0c0b0f70 0x7f1b0c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cfd90 0x7f1b0c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b0c0cd110 0x7f1b0c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c15ef20 con 0x7f1b0c0b0f70 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c15f3b0 con 0x7f1b0c0cd110 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b13c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0ca810 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b0c0b0f70 0x7f1b0c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b0c0b0f70 0x7f1b0c0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54304/0 (socket says 172.21.15.45:54304) 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 -- 172.21.15.45:0/1805233526 learned_addr learned my addr 172.21.15.45:0/1805233526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b03e00640 1 --2- 172.21.15.45:0/1805233526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cfd90 0x7f1b0c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11c00640 1 --2- 172.21.15.45:0/1805233526 >> v2:172.21.15.117:3300/0 conn(0x7f1b0c0cd110 0x7f1b0c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 -- 172.21.15.45:0/1805233526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cfd90 msgr2=0x7f1b0c0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 --2- 172.21.15.45:0/1805233526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cfd90 0x7f1b0c0ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 -- 172.21.15.45:0/1805233526 >> v2:172.21.15.117:3300/0 conn(0x7f1b0c0cd110 msgr2=0x7f1b0c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 --2- 172.21.15.45:0/1805233526 >> v2:172.21.15.117:3300/0 conn(0x7f1b0c0cd110 0x7f1b0c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.747+0000 7f1b11200640 1 -- 172.21.15.45:0/1805233526 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0b1890 con 0x7f1b0c0b0f70 2024-04-25T10:20:53.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b11200640 1 --2- 172.21.15.45:0/1805233526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0b1350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b2d50 tx=0x7f1afc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6869ac15419c306a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b0be00640 1 -- 172.21.15.45:0/1805233526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0b0070 con 0x7f1b0c0b0f70 2024-04-25T10:20:53.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b0be00640 1 -- 172.21.15.45:0/1805233526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc0ab600 con 0x7f1b0c0b0f70 2024-04-25T10:20:53.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b0be00640 1 -- 172.21.15.45:0/1805233526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0ab9d0 con 0x7f1b0c0b0f70 2024-04-25T10:20:53.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 -- 172.21.15.45:0/1805233526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0b0f70 msgr2=0x7f1b0c0b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1805233526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0b1350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b2d50 tx=0x7f1afc01c520 comp rx=0 tx=0).stop 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 -- 172.21.15.45:0/1805233526 shutdown_connections 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1805233526 >> v2:172.21.15.117:3300/0 conn(0x7f1b0c0cd110 0x7f1b0c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1805233526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cfd90 0x7f1b0c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1805233526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0b1350 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 -- 172.21.15.45:0/1805233526 >> 172.21.15.45:0/1805233526 conn(0x7f1b0c0b1d50 msgr2=0x7f1b0c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 -- 172.21.15.45:0/1805233526 shutdown_connections 2024-04-25T10:20:53.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 -- 172.21.15.45:0/1805233526 wait complete. 2024-04-25T10:20:53.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 Processor -- start 2024-04-25T10:20:53.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.751+0000 7f1b13c00640 1 -- start start 2024-04-25T10:20:53.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cd110 0x7f1b0c155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 0x7f1b0c0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c072200 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cd520 con 0x7f1b0c0b0f70 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c072380 con 0x7f1b0c0cd110 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 0x7f1b0c0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 0x7f1b0c0c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54306/0 (socket says 172.21.15.45:54306) 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 -- 172.21.15.45:0/1158127740 learned_addr learned my addr 172.21.15.45:0/1158127740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11200640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b03e00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cd110 0x7f1b0c155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 -- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cd110 msgr2=0x7f1b0c155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cd110 0x7f1b0c155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 -- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b0c0b0f70 msgr2=0x7f1b0c0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 -- 172.21.15.45:0/1158127740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0afe20 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b11c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 0x7f1b0c0c3c20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1b04007e90 tx=0x7f1b0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b09600640 1 -- 172.21.15.45:0/1158127740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04014420 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b09600640 1 -- 172.21.15.45:0/1158127740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04019070 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c1581b0 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b09600640 1 -- 172.21.15.45:0/1158127740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04014820 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.755+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0b0090 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b09600640 1 -- 172.21.15.45:0/1158127740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b040149c0 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b09600640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ae8066f90 0x7f1ae8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b09600640 1 -- 172.21.15.45:0/1158127740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b040856e0 con 0x7f1b0c0cfd90 2024-04-25T10:20:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b11200640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ae8066f90 0x7f1ae8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 Processor -- start 2024-04-25T10:20:53.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 -- start start 2024-04-25T10:20:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b11200640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ae8066f90 0x7f1ae8069450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0ad000 tx=0x7f1afc0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afbdad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 0x5586afb909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afb91310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586afa785f0 con 0x5586afbdb240 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586af9dfda0 con 0x5586afb90f30 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586af968150 con 0x5586afbda920 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 0x5586afb909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 0x5586afb909f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54322/0 (socket says 172.21.15.45:54322) 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 -- 172.21.15.45:0/3128189662 learned_addr learned my addr 172.21.15.45:0/3128189662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b11200640 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afbdad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b11c00640 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afb91310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 -- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 msgr2=0x5586afbdad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afbdad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 -- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 msgr2=0x5586afb91310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afb91310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 -- 172.21.15.45:0/3128189662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586af981ca0 con 0x5586afbdb240 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b03e00640 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 0x5586afb909f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1af4007880 tx=0x7f1af4007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9f70425089d1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.759+0000 7f1b01600640 1 -- 172.21.15.45:0/3128189662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4010b00 con 0x5586afbdb240 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b01600640 1 -- 172.21.15.45:0/3128189662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4010ca0 con 0x5586afbdb240 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b01600640 1 -- 172.21.15.45:0/3128189662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4004270 con 0x5586afbdb240 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- 172.21.15.45:0/3128189662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 msgr2=0x5586afb909f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 0x5586afb909f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1af4007880 tx=0x7f1af4007d40 comp rx=0 tx=0).stop 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- 172.21.15.45:0/3128189662 shutdown_connections 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afb91310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbdb240 0x5586afb909f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- 172.21.15.45:0/3128189662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afbdad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- 172.21.15.45:0/3128189662 >> 172.21.15.45:0/3128189662 conn(0x5586afb972a0 msgr2=0x5586afb8f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- 172.21.15.45:0/3128189662 shutdown_connections 2024-04-25T10:20:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- 172.21.15.45:0/3128189662 wait complete. 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- start start 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 Processor -- start 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afbe1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afbe2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afb94aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586af99a990 con 0x5586afbda920 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586af9dbe40 con 0x5586afb90f30 2024-04-25T10:20:53.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586af96b900 con 0x5586afbdb240 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b171a4000 10 client.0 _ll_register_callbacks cb 0x5586afa931c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afbe2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afbe2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54332/0 (socket says 172.21.15.45:54332) 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 -- 172.21.15.45:0/4051504642 learned_addr learned my addr 172.21.15.45:0/4051504642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b11200640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afbe1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b11c00640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afb94aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 -- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 msgr2=0x5586afb94aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afb94aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 -- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 msgr2=0x5586afbe1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afbe1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 -- 172.21.15.45:0/4051504642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586af998630 con 0x5586afbda920 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1b03e00640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afbe2230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5586af983200 tx=0x7f1af4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ca1b453a617c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1afa000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.763+0000 7f1afa000640 1 -- 172.21.15.45:0/4051504642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4020030 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1af4010ca0 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 1 -- 172.21.15.45:0/4051504642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1af400f050 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 1 -- 172.21.15.45:0/4051504642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af402f050 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 10 client.5706 fetch_fsmap learned FSMap version 27 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5586af9683a0 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 1 -- 172.21.15.45:0/4051504642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4010ca0 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 1 -- 172.21.15.45:0/4051504642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1af4032020 con 0x5586afbda920 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 10 client.5706 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 10 client.5706 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 20 client.5706 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96160 0x5586afbebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 4 client.5706 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:53.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586af97ba50 con 0x5586afb96160 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b11200640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96160 0x5586afbebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b11200640 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96160 0x5586afbebd50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f08212181501671 server_cookie=b1fdb4b5c6d22260 in_seq=0 out_seq=0 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 10 client.5706 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 1 -- 172.21.15.45:0/4051504642 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586af97ba50 con 0x5586afb96160 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 10 client.5706 handle_command_reply: tid=0 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 client.5706 shutdown 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 msgr2=0x5586afbe2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afbe2230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5586af983200 tx=0x7f1af4004740 comp rx=0 tx=0).stop 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 shutdown_connections 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96160 0x5586afbebd50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afb94aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afbe2230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 --2- 172.21.15.45:0/4051504642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afbe1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1afa000640 0 client.5706 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 >> 172.21.15.45:0/4051504642 conn(0x5586afb972a0 msgr2=0x5586afb8f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 shutdown_connections 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 1 -- 172.21.15.45:0/4051504642 wait complete. 2024-04-25T10:20:53.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.767+0000 7f1b171a4000 20 client.5706 trim_cache size 0 max 16384 2024-04-25T10:20:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.783+0000 7f1b171a4000 1 Processor -- start 2024-04-25T10:20:53.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.783+0000 7f1b171a4000 1 -- start start 2024-04-25T10:20:53.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afc36200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afc36740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afc36c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c202280 con 0x5586afbda920 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0ad3b0 con 0x5586afb90f30 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0ad9b0 con 0x5586afbdb240 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afc36740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afc36740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54342/0 (socket says 172.21.15.45:54342) 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 -- 172.21.15.45:0/1893805934 learned_addr learned my addr 172.21.15.45:0/1893805934 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b11200640 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afc36200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b11c00640 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afc36c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 -- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 msgr2=0x5586afc36c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afc36c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 -- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 msgr2=0x5586afc36200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afc36200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 -- 172.21.15.45:0/1893805934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586afbdb070 con 0x5586afbda920 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b11c00640 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afc36c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b03e00640 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afc36740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1af40043c0 tx=0x7f1af4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf698f0bd8b94f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b00c00640 1 -- 172.21.15.45:0/1893805934 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4038070 con 0x5586afbda920 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b00c00640 1 -- 172.21.15.45:0/1893805934 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4010620 con 0x5586afbda920 2024-04-25T10:20:53.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b00c00640 1 -- 172.21.15.45:0/1893805934 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4020040 con 0x5586afbda920 2024-04-25T10:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- 172.21.15.45:0/1893805934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 msgr2=0x5586afc36740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afc36740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1af40043c0 tx=0x7f1af4023000 comp rx=0 tx=0).stop 2024-04-25T10:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- 172.21.15.45:0/1893805934 shutdown_connections 2024-04-25T10:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbdb240 0x5586afc36c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afbda920 0x5586afc36740 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- 172.21.15.45:0/1893805934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afb90f30 0x5586afc36200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- 172.21.15.45:0/1893805934 >> 172.21.15.45:0/1893805934 conn(0x5586afb972a0 msgr2=0x5586afb94220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- 172.21.15.45:0/1893805934 shutdown_connections 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- 172.21.15.45:0/1893805934 wait complete. 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- start start 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 Processor -- start 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 0x5586afbe71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afc3b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afbdb240 0x5586afc3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586af9e43d0 con 0x5586afb90f30 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586afa9bd50 con 0x5586afbdb240 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586af97a980 con 0x5586afbda920 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.787+0000 7f1b171a4000 10 client.0 _ll_register_callbacks cb 0x7f1af4007550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 0x5586afbe71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 0x5586afbe71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54354/0 (socket says 172.21.15.45:54354) 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 -- 172.21.15.45:0/813644222 learned_addr learned my addr 172.21.15.45:0/813644222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b03e00640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afc3b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11c00640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afbdb240 0x5586afc3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 -- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 msgr2=0x5586afc3b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afc3b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 -- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afbdb240 msgr2=0x5586afc3bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afbdb240 0x5586afc3bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 -- 172.21.15.45:0/813644222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586afa61600 con 0x5586afb90f30 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11c00640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afbdb240 0x5586afc3bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b11200640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 0x5586afbe71c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1afc01cdc0 tx=0x7f1afc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27eac2beabc1398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 1 -- 172.21.15.45:0/813644222 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0b0040 con 0x5586afb90f30 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 1 -- 172.21.15.45:0/813644222 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1afc0abdd0 con 0x5586afb90f30 2024-04-25T10:20:53.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1af4010620 con 0x5586afb90f30 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 1 -- 172.21.15.45:0/813644222 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1afc0d9050 con 0x5586afb90f30 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 10 client.5712 fetch_fsmap learned FSMap version 27 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5586af9749b0 con 0x5586afb90f30 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 1 -- 172.21.15.45:0/813644222 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af4010620 con 0x5586afb90f30 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 1 -- 172.21.15.45:0/813644222 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1afc0dc020 con 0x5586afb90f30 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 10 client.5712 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 10 client.5712 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 20 client.5712 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96e80 0x5586afc3c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 4 client.5712 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5586af9ed2e0 con 0x5586afb96e80 2024-04-25T10:20:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b03e00640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96e80 0x5586afc3c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b03e00640 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96e80 0x5586afc3c000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41aadb3fe99628cf server_cookie=4ceeda46f6bec309 in_seq=0 out_seq=0 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 10 client.5712 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 1 -- 172.21.15.45:0/813644222 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5586af9ed2e0 con 0x5586afb96e80 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 10 client.5712 handle_command_reply: tid=0 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 client.5712 shutdown 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 msgr2=0x5586afbe71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 0x5586afbe71c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1afc01cdc0 tx=0x7f1afc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 shutdown_connections 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5586afb96e80 0x5586afc3c000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586afbdb240 0x5586afc3bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5586afbda920 0x5586afc3b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 --2- 172.21.15.45:0/813644222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5586afb90f30 0x5586afbe71c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1afa000640 0 client.5712 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 >> 172.21.15.45:0/813644222 conn(0x5586afb972a0 msgr2=0x5586afbe28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 shutdown_connections 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 1 -- 172.21.15.45:0/813644222 wait complete. 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.791+0000 7f1b171a4000 20 client.5712 trim_cache size 0 max 16384 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5574, 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:20:53.800 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:20:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9207943969999999, 2024-04-25T10:20:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 27, 2024-04-25T10:20:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 50, 2024-04-25T10:20:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 50, 2024-04-25T10:20:53.801 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.815214644999999 2024-04-25T10:20:53.801 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ae8066f90 msgr2=0x7f1ae8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ae8066f90 0x7f1ae8069450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0ad000 tx=0x7f1afc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 msgr2=0x7f1b0c0c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 0x7f1b0c0c3c20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1b04007e90 tx=0x7f1b0400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 shutdown_connections 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ae8066f90 0x7f1ae8069450 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b0c0cfd90 0x7f1b0c0c3c20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b0c0cd110 0x7f1b0c155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 --2- 172.21.15.45:0/1158127740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b0c0b0f70 0x7f1b0c0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 >> 172.21.15.45:0/1158127740 conn(0x7f1b0c0b1d50 msgr2=0x7f1b0c072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 shutdown_connections 2024-04-25T10:20:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.795+0000 7f1b13c00640 1 -- 172.21.15.45:0/1158127740 wait complete. 2024-04-25T10:20:53.816 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:20:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 Processor -- start 2024-04-25T10:20:53.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 -- start start 2024-04-25T10:20:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 0x7f5a5c06b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5a5c15e1a0 0x7f5a5c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c0cc060 con 0x7f5a5c0d2460 2024-04-25T10:20:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cf2f0 con 0x7f5a5c15e1a0 2024-04-25T10:20:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a62e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c06af30 con 0x7f5a5c15ddc0 2024-04-25T10:20:53.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a60e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5a5c0d2460 0x7f5a5c0cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46258/0 (socket says 172.21.15.45:46258) 2024-04-25T10:20:53.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a60e00640 1 -- 172.21.15.45:0/1995161893 learned_addr learned my addr 172.21.15.45:0/1995161893 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:53.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a53e00640 1 --2- 172.21.15.45:0/1995161893 >> v2:172.21.15.117:3300/0 conn(0x7f5a5c15e1a0 0x7f5a5c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 0x7f5a5c06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107685743 0 0) 0x7f5a5c0cc060 con 0x7f5a5c0d2460 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a40003530 con 0x7f5a5c0d2460 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2371445041 0 0) 0x7f5a40003530 con 0x7f5a5c0d2460 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 msgr2=0x7f5a5c06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 0x7f5a5c06b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 >> v2:172.21.15.117:3300/0 conn(0x7f5a5c15e1a0 msgr2=0x7f5a5c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 --2- 172.21.15.45:0/1995161893 >> v2:172.21.15.117:3300/0 conn(0x7f5a5c15e1a0 0x7f5a5c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0cc340 con 0x7f5a5c0d2460 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f5a54001a00 con 0x7f5a5c0d2460 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 >> v1:172.21.15.45:6789/0 conn(0x7f5a5c0d2460 legacy=0x7f5a5c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 0x7f5a40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a40004d50 0x7f5a400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cf2f0 con 0x7f5a400039c0 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c06af30 con 0x7f5a40004d50 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4000b9d0 con 0x7f5a5c15ddc0 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a53e00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 0x7f5a40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a60e00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a40004d50 0x7f5a400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 -- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 msgr2=0x7f5a40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c15ddc0 0x7f5a40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 -- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a40004d50 msgr2=0x7f5a400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a40004d50 0x7f5a400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a5be00640 1 -- 172.21.15.45:0/1995161893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40003530 con 0x7f5a400039c0 2024-04-25T10:20:53.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.955+0000 7f5a60e00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a40004d50 0x7f5a400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:53.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a40003de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009ae0 tx=0x7f5a4c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a97a5552988b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0096b0 con 0x7f5a400039c0 2024-04-25T10:20:53.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c0a6450 con 0x7f5a400039c0 2024-04-25T10:20:53.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a5b400640 1 -- 172.21.15.45:0/1995161893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0a6840 con 0x7f5a400039c0 2024-04-25T10:20:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a62e00640 1 -- 172.21.15.45:0/1995161893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 msgr2=0x7f5a40003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1995161893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a40003de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009ae0 tx=0x7f5a4c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:20:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a60e00640 1 -- 172.21.15.45:0/1995161893 reap_dead start 2024-04-25T10:20:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a62e00640 1 -- 172.21.15.45:0/1995161893 shutdown_connections 2024-04-25T10:20:53.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a62e00640 1 -- 172.21.15.45:0/1995161893 >> 172.21.15.45:0/1995161893 conn(0x7f5a5c0d1970 msgr2=0x7f5a5c0d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a62e00640 1 -- 172.21.15.45:0/1995161893 shutdown_connections 2024-04-25T10:20:53.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.959+0000 7f5a62e00640 1 -- 172.21.15.45:0/1995161893 wait complete. 2024-04-25T10:20:53.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 Processor -- start 2024-04-25T10:20:53.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 -- start start 2024-04-25T10:20:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a5c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c154050 0x7f5a5c0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c0c5ae0 0x7f5a5c0b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c168b00 con 0x7f5a400039c0 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c06b180 con 0x7f5a5c0c5ae0 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c168c80 con 0x7f5a5c154050 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a5c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a5c1539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54380/0 (socket says 172.21.15.45:54380) 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 -- 172.21.15.45:0/1508135148 learned_addr learned my addr 172.21.15.45:0/1508135148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a60e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c0c5ae0 0x7f5a5c0b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a53e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c154050 0x7f5a5c0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 -- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c154050 msgr2=0x7f5a5c0c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c154050 0x7f5a5c0c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 -- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c0c5ae0 msgr2=0x7f5a5c0b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c0c5ae0 0x7f5a5c0b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 -- 172.21.15.45:0/1508135148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0c8ae0 con 0x7f5a400039c0 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a60e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c0c5ae0 0x7f5a5c0b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a53e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c154050 0x7f5a5c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:53.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a5be00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a5c1539d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c01cf50 tx=0x7f5a40005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0a6c00 con 0x7f5a400039c0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c0a6da0 con 0x7f5a400039c0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c0c8dc0 con 0x7f5a400039c0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c004260 con 0x7f5a400039c0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.963+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c0c9380 con 0x7f5a400039c0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.967+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c06ce20 con 0x7f5a400039c0 2024-04-25T10:20:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.967+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a4c003620 con 0x7f5a400039c0 2024-04-25T10:20:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.967+0000 7f5a58c00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a38066f60 0x7f5a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.967+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a4c11f130 con 0x7f5a400039c0 2024-04-25T10:20:53.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.967+0000 7f5a53e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a38066f60 0x7f5a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.967+0000 7f5a53e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a38066f60 0x7f5a38069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0ce960 tx=0x7f5a44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:53.971+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a4c0ac020 con 0x7f5a400039c0 2024-04-25T10:20:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.167+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a5c04ef30 con 0x7f5a400039c0 2024-04-25T10:20:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a58c00640 1 -- 172.21.15.45:0/1508135148 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5a4c016780 con 0x7f5a400039c0 2024-04-25T10:20:54.173 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a38066f60 msgr2=0x7f5a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a38066f60 0x7f5a38069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0ce960 tx=0x7f5a44008040 comp rx=0 tx=0).stop 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 msgr2=0x7f5a5c1539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a5c1539d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c01cf50 tx=0x7f5a40005840 comp rx=0 tx=0).stop 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 shutdown_connections 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a38066f60 0x7f5a38069420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a5c0c5ae0 0x7f5a5c0b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a5c154050 0x7f5a5c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 --2- 172.21.15.45:0/1508135148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a400039c0 0x7f5a5c1539d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 >> 172.21.15.45:0/1508135148 conn(0x7f5a5c0d1970 msgr2=0x7f5a5c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 shutdown_connections 2024-04-25T10:20:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:54.171+0000 7f5a62e00640 1 -- 172.21.15.45:0/1508135148 wait complete. 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:20:54.194 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:54.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.195 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:54.204 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42286sec preferred_lft 42286sec 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:20:54.205 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:54.207 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:20:54.207 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:20:54.207 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:54.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.208 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:20:54.266 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:20:54.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.268 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:20:54.325 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:20:54.326 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:20:54.326 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:20:54.326 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:20:54.326 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:20:54.326 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:20:54.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.326 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:20:54.326 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:20:54.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:20:54.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:20:54.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:20:54.327 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:20:54.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.419 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:20:54.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:20:54.428 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:20:54.428 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:54.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:20:54.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:54.488 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:54.540 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:54.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:54.540 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:54.591 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:20:54.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:20:54.681 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:20:54.681 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:20:54.694 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:20:54.694 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:20:54.694 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:20:54.694 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:20:54.694 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:20:54.698 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:20:54.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:54.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.719 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:20:54.780 INFO:teuthology.orchestra.run.smithi045.stdout:client5721 2024-04-25T10:20:54.781 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:20:54.781 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:54.832 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:54.832 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:20:54.832 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:20:54.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:54.833 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> import json 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:20:54.897 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2024-04-25T10:20:54.898 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:20:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:55 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:20:55.020 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:20:55.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:55.021 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T10:20:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T10:20:55.335 INFO:teuthology.orchestra.run.smithi045.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T10:20:57.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:57.657 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:58.859 INFO:teuthology.orchestra.run.smithi045.stderr:Updating files: 52% (729/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 82% (1160/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T10:20:58.867 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T10:20:58.867 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:20:58.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:59.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 Processor -- start 2024-04-25T10:20:59.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 -- start start 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f072015b750 0x7f07200cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f07200b4690 0x7f07200cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07200b2460 con 0x7f07200cd590 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07200b2d70 con 0x7f072015b750 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071f200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0720070fb0 con 0x7f07200b4690 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.007+0000 7f071de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f072015b750 0x7f07200cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f072015b750 0x7f07200cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40888/0 (socket says 172.21.15.45:40888) 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 -- 172.21.15.45:0/3119914781 learned_addr learned my addr 172.21.15.45:0/3119914781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071e800640 1 --2- 172.21.15.45:0/3119914781 >> v2:172.21.15.117:3300/0 conn(0x7f07200b4690 0x7f07200cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 -- 172.21.15.45:0/3119914781 >> v2:172.21.15.117:3300/0 conn(0x7f07200b4690 msgr2=0x7f07200cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 --2- 172.21.15.45:0/3119914781 >> v2:172.21.15.117:3300/0 conn(0x7f07200b4690 0x7f07200cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 -- 172.21.15.45:0/3119914781 >> v1:172.21.15.117:6789/0 conn(0x7f07200cd590 legacy=0x7f07200b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 -- 172.21.15.45:0/3119914781 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07200b2720 con 0x7f072015b750 2024-04-25T10:20:59.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071e800640 1 --2- 172.21.15.45:0/3119914781 >> v2:172.21.15.117:3300/0 conn(0x7f07200b4690 0x7f07200cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:59.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071de00640 1 --2- 172.21.15.45:0/3119914781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f07200cbc30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f07200b2ef0 tx=0x7f071001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd254424dea8049d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071d400640 1 -- 172.21.15.45:0/3119914781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100a5840 con 0x7f072015b750 2024-04-25T10:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071d400640 1 -- 172.21.15.45:0/3119914781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07100a59e0 con 0x7f072015b750 2024-04-25T10:20:59.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.011+0000 7f071d400640 1 -- 172.21.15.45:0/3119914781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100a5db0 con 0x7f072015b750 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- 172.21.15.45:0/3119914781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 msgr2=0x7f07200cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 --2- 172.21.15.45:0/3119914781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f07200cbc30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f07200b2ef0 tx=0x7f071001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- 172.21.15.45:0/3119914781 shutdown_connections 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 --2- 172.21.15.45:0/3119914781 >> v2:172.21.15.117:3300/0 conn(0x7f07200b4690 0x7f07200cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 --2- 172.21.15.45:0/3119914781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f07200cbc30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- 172.21.15.45:0/3119914781 >> 172.21.15.45:0/3119914781 conn(0x7f07200743f0 msgr2=0x7f07200669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- 172.21.15.45:0/3119914781 shutdown_connections 2024-04-25T10:20:59.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- 172.21.15.45:0/3119914781 wait complete. 2024-04-25T10:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 Processor -- start 2024-04-25T10:20:59.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- start start 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07200b4690 0x7f072014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07200cd590 0x7f072014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f072014a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720070ce0 con 0x7f072015b750 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0720070b60 con 0x7f07200cd590 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0720168760 con 0x7f07200b4690 2024-04-25T10:20:59.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f072014a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f072014a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40890/0 (socket says 172.21.15.45:40890) 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 -- 172.21.15.45:0/2092343423 learned_addr learned my addr 172.21.15.45:0/2092343423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f0715400640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07200cd590 0x7f072014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071de00640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07200b4690 0x7f072014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 -- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07200b4690 msgr2=0x7f072014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07200b4690 0x7f072014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 -- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07200cd590 msgr2=0x7f072014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07200cd590 0x7f072014c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 -- 172.21.15.45:0/2092343423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072014cc60 con 0x7f072015b750 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f0715400640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07200cd590 0x7f072014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071e800640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f072014a160 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f071800e810 tx=0x7f071800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718017070 con 0x7f072015b750 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071800c6d0 con 0x7f072015b750 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072014cf40 con 0x7f072015b750 2024-04-25T10:20:59.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.015+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071800f040 con 0x7f072015b750 2024-04-25T10:20:59.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072014d500 con 0x7f072015b750 2024-04-25T10:20:59.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0720075120 con 0x7f072015b750 2024-04-25T10:20:59.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f071801a070 con 0x7f072015b750 2024-04-25T10:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f0716a00640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06fc066f60 0x7f06fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0718084cf0 con 0x7f072015b750 2024-04-25T10:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f071de00640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06fc066f60 0x7f06fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.019+0000 7f071de00640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06fc066f60 0x7f06fc069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07100ae000 tx=0x7f07100a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.023+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0718055c60 con 0x7f072015b750 2024-04-25T10:20:59.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f072015f130 con 0x7f072015b750 2024-04-25T10:20:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f0716a00640 1 -- 172.21.15.45:0/2092343423 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f0718055560 con 0x7f072015b750 2024-04-25T10:20:59.222 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:59.222 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:20:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06fc066f60 msgr2=0x7f06fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06fc066f60 0x7f06fc069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f07100ae000 tx=0x7f07100a7070 comp rx=0 tx=0).stop 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 msgr2=0x7f072014a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f072014a160 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f071800e810 tx=0x7f071800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 shutdown_connections 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06fc066f60 0x7f06fc069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f072015b750 0x7f072014a160 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07200cd590 0x7f072014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 --2- 172.21.15.45:0/2092343423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07200b4690 0x7f072014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 >> 172.21.15.45:0/2092343423 conn(0x7f07200743f0 msgr2=0x7f07200aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 shutdown_connections 2024-04-25T10:20:59.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.219+0000 7f071f200640 1 -- 172.21.15.45:0/2092343423 wait complete. 2024-04-25T10:20:59.243 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:20:59.383 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:20:59.383 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": -2 2024-04-25T10:20:59.383 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:20:59.398 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:20:59.398 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:20:59.399 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:20:59.399 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:20:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.539+0000 7f9925e00640 1 Processor -- start 2024-04-25T10:20:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.539+0000 7f9925e00640 1 -- start start 2024-04-25T10:20:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9925e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9920159540 0x7f99200ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9925e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99200aebd0 0x7f9920071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9925e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9920072020 0x7f992015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9925e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9920159920 con 0x7f9920159540 2024-04-25T10:20:59.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9925e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992015c2f0 con 0x7f9920072020 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9925e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992015c5d0 con 0x7f99200aebd0 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9920159540 0x7f99200ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9920159540 0x7f99200ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40906/0 (socket says 172.21.15.45:40906) 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 -- 172.21.15.45:0/937450067 learned_addr learned my addr 172.21.15.45:0/937450067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9916000640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99200aebd0 0x7f9920071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991f400640 1 --2- 172.21.15.45:0/937450067 >> v2:172.21.15.117:3300/0 conn(0x7f9920072020 0x7f992015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 -- 172.21.15.45:0/937450067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99200aebd0 msgr2=0x7f9920071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99200aebd0 0x7f9920071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 -- 172.21.15.45:0/937450067 >> v2:172.21.15.117:3300/0 conn(0x7f9920072020 msgr2=0x7f992015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 --2- 172.21.15.45:0/937450067 >> v2:172.21.15.117:3300/0 conn(0x7f9920072020 0x7f992015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 -- 172.21.15.45:0/937450067 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992015c8b0 con 0x7f9920159540 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f9916000640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99200aebd0 0x7f9920071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:59.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991f400640 1 --2- 172.21.15.45:0/937450067 >> v2:172.21.15.117:3300/0 conn(0x7f9920072020 0x7f992015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:59.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991ea00640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920159540 0x7f99200ae690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9920071720 tx=0x7f991001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9d62d8af16be74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991e000640 1 -- 172.21.15.45:0/937450067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100a64d0 con 0x7f9920159540 2024-04-25T10:20:59.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991e000640 1 -- 172.21.15.45:0/937450067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99100a6670 con 0x7f9920159540 2024-04-25T10:20:59.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.543+0000 7f991e000640 1 -- 172.21.15.45:0/937450067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100a6a40 con 0x7f9920159540 2024-04-25T10:20:59.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- 172.21.15.45:0/937450067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920159540 msgr2=0x7f99200ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920159540 0x7f99200ae690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9920071720 tx=0x7f991001c730 comp rx=0 tx=0).stop 2024-04-25T10:20:59.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- 172.21.15.45:0/937450067 shutdown_connections 2024-04-25T10:20:59.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- 172.21.15.45:0/937450067 >> v2:172.21.15.117:3300/0 conn(0x7f9920072020 0x7f992015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99200aebd0 0x7f9920071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- 172.21.15.45:0/937450067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920159540 0x7f99200ae690 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- 172.21.15.45:0/937450067 >> 172.21.15.45:0/937450067 conn(0x7f99200669a0 msgr2=0x7f9920066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- 172.21.15.45:0/937450067 shutdown_connections 2024-04-25T10:20:59.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- 172.21.15.45:0/937450067 wait complete. 2024-04-25T10:20:59.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 Processor -- start 2024-04-25T10:20:59.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- start start 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 0x7f99200b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99200aebd0 0x7f99200afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9920159540 0x7f9920147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920070350 con 0x7f9920072020 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920072430 con 0x7f99200aebd0 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9925e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99200704d0 con 0x7f9920159540 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 0x7f99200b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 0x7f99200b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40918/0 (socket says 172.21.15.45:40918) 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 -- 172.21.15.45:0/1252435342 learned_addr learned my addr 172.21.15.45:0/1252435342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f9916000640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99200aebd0 0x7f99200afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991f400640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9920159540 0x7f9920147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 -- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9920159540 msgr2=0x7f9920147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9920159540 0x7f9920147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 -- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99200aebd0 msgr2=0x7f99200afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99200aebd0 0x7f99200afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991ea00640 1 -- 172.21.15.45:0/1252435342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99200b0c00 con 0x7f9920072020 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.547+0000 7f991f400640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9920159540 0x7f9920147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f991ea00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 0x7f99200b41b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f991001ade0 tx=0x7f99100194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9917e00640 1 -- 172.21.15.45:0/1252435342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100a6c30 con 0x7f9920072020 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9917e00640 1 -- 172.21.15.45:0/1252435342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99100a6dd0 con 0x7f9920072020 2024-04-25T10:20:59.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9917e00640 1 -- 172.21.15.45:0/1252435342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100af6c0 con 0x7f9920072020 2024-04-25T10:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99200b0640 con 0x7f9920072020 2024-04-25T10:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99201425c0 con 0x7f9920072020 2024-04-25T10:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9917e00640 1 -- 172.21.15.45:0/1252435342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99100af860 con 0x7f9920072020 2024-04-25T10:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9917e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98fc066f60 0x7f98fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9917e00640 1 -- 172.21.15.45:0/1252435342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f991011f160 con 0x7f9920072020 2024-04-25T10:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f9916000640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98fc066f60 0x7f98fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.551+0000 7f99292f4000 1 Processor -- start 2024-04-25T10:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- start start 2024-04-25T10:20:59.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f9916000640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98fc066f60 0x7f98fc069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99200b16c0 tx=0x7f9908008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 0x55f5068bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bd290 0x55f506872790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 0x55f5068730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5066888a0 con 0x55f506872cd0 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5066c0d10 con 0x55f5068bd290 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5066492d0 con 0x55f5068bc970 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 0x55f5068730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f9916000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bd290 0x55f506872790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 0x55f5068bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 0x55f5068730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40922/0 (socket says 172.21.15.45:40922) 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 0x55f5068bcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37364/0 (socket says 172.21.15.45:37364) 2024-04-25T10:20:59.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 -- 172.21.15.45:0/328613955 learned_addr learned my addr 172.21.15.45:0/328613955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 -- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 msgr2=0x55f5068bcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 0x55f5068bcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 -- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bd290 msgr2=0x55f506872790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bd290 0x55f506872790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 -- 172.21.15.45:0/328613955 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f506707870 con 0x55f506872cd0 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991ea00640 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 0x55f5068bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f9916000640 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bd290 0x55f506872790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f991f400640 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 0x55f5068730b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9918009a90 tx=0x7f9918009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847e4370690421d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f990fe00640 1 -- 172.21.15.45:0/328613955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918019020 con 0x55f506872cd0 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f990fe00640 1 -- 172.21.15.45:0/328613955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918010d60 con 0x55f506872cd0 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f990fe00640 1 -- 172.21.15.45:0/328613955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800f040 con 0x55f506872cd0 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- 172.21.15.45:0/328613955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 msgr2=0x55f5068730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 0x55f5068730b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9918009a90 tx=0x7f9918009f50 comp rx=0 tx=0).stop 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- 172.21.15.45:0/328613955 shutdown_connections 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f506872cd0 0x55f5068730b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bd290 0x55f506872790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 --2- 172.21.15.45:0/328613955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bc970 0x55f5068bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- 172.21.15.45:0/328613955 >> 172.21.15.45:0/328613955 conn(0x55f506879540 msgr2=0x55f506879920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- 172.21.15.45:0/328613955 shutdown_connections 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- 172.21.15.45:0/328613955 wait complete. 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.555+0000 7f99292f4000 1 -- start start 2024-04-25T10:20:59.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 Processor -- start 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f506872cd0 0x55f5068c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 0x55f5068c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bd290 0x55f5068777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f50667bb50 con 0x55f5068bc970 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5066bcdb0 con 0x55f506872cd0 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f506667870 con 0x55f5068bd290 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 10 client.0 _ll_register_callbacks cb 0x55f506774fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:59.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 0x55f5068c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 0x55f5068c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40926/0 (socket says 172.21.15.45:40926) 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 -- 172.21.15.45:0/3842934731 learned_addr learned my addr 172.21.15.45:0/3842934731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f991ea00640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f506872cd0 0x55f5068c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f991f400640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bd290 0x55f5068777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 -- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bd290 msgr2=0x55f5068777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bd290 0x55f5068777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 -- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f506872cd0 msgr2=0x55f5068c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f506872cd0 0x55f5068c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 -- 172.21.15.45:0/3842934731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f506649520 con 0x55f5068bc970 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f9916000640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 0x55f5068c3200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99080156f0 tx=0x7f99080046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e9737f0397b083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f990cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f990cc00640 1 -- 172.21.15.45:0/3842934731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908026070 con 0x55f5068bc970 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f990cc00640 1 -- 172.21.15.45:0/3842934731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99080149f0 con 0x55f5068bc970 2024-04-25T10:20:59.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9918010d60 con 0x55f5068bc970 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f990cc00640 1 -- 172.21.15.45:0/3842934731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9908031050 con 0x55f5068bc970 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 10 client.5739 fetch_fsmap learned FSMap version 27 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55f5066bbd00 con 0x55f5068bc970 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f990cc00640 1 -- 172.21.15.45:0/3842934731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918010d60 con 0x55f5068bc970 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f990cc00640 1 -- 172.21.15.45:0/3842934731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9908034020 con 0x55f5068bc970 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 10 client.5739 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 10 client.5739 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 20 client.5739 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506878550 0x55f506878960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 4 client.5739 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f50665cb10 con 0x55f506878550 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.559+0000 7f991ea00640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506878550 0x55f506878960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f991ea00640 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506878550 0x55f506878960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbff7d6a150a8e51 server_cookie=92fcb2804c65bcf6 in_seq=0 out_seq=0 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f990cc00640 10 client.5739 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f990cc00640 1 -- 172.21.15.45:0/3842934731 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f50665cb10 con 0x55f506878550 2024-04-25T10:20:59.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f990cc00640 10 client.5739 handle_command_reply: tid=0 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 client.5739 shutdown 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 msgr2=0x55f5068c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 0x55f5068c3200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99080156f0 tx=0x7f99080046d0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 shutdown_connections 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506878550 0x55f506878960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f5068bd290 0x55f5068777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bc970 0x55f5068c3200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f990cc00640 0 client.5739 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 --2- 172.21.15.45:0/3842934731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f506872cd0 0x55f5068c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 >> 172.21.15.45:0/3842934731 conn(0x55f506879540 msgr2=0x55f506871780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 shutdown_connections 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 1 -- 172.21.15.45:0/3842934731 wait complete. 2024-04-25T10:20:59.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.563+0000 7f99292f4000 20 client.5739 trim_cache size 0 max 16384 2024-04-25T10:20:59.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 Processor -- start 2024-04-25T10:20:59.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- start start 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f506872320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f5068bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f5068bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920145150 con 0x55f5068bd290 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920144550 con 0x55f5068bc970 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9920144b50 con 0x55f506872cd0 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f5068bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f5068bfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40942/0 (socket says 172.21.15.45:40942) 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 -- 172.21.15.45:0/1570683160 learned_addr learned my addr 172.21.15.45:0/1570683160 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991ea00640 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f506872320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f9916000640 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f5068bf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 -- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 msgr2=0x55f506872320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f506872320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 -- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 msgr2=0x55f5068bf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f5068bf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 -- 172.21.15.45:0/1570683160 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f506744920 con 0x55f5068bd290 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f9916000640 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f5068bf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f991f400640 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f5068bfbc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9918017000 tx=0x7f991800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8910e9a84676dab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f990f400640 1 -- 172.21.15.45:0/1570683160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918019030 con 0x55f5068bd290 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f990f400640 1 -- 172.21.15.45:0/1570683160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918010b60 con 0x55f5068bd290 2024-04-25T10:20:59.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f990f400640 1 -- 172.21.15.45:0/1570683160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918004040 con 0x55f5068bd290 2024-04-25T10:20:59.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- 172.21.15.45:0/1570683160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 msgr2=0x55f5068bfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f5068bfbc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9918017000 tx=0x7f991800e000 comp rx=0 tx=0).stop 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- 172.21.15.45:0/1570683160 shutdown_connections 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f5068bfbc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f5068bf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 --2- 172.21.15.45:0/1570683160 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f506872320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- 172.21.15.45:0/1570683160 >> 172.21.15.45:0/1570683160 conn(0x55f506879540 msgr2=0x55f506876bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- 172.21.15.45:0/1570683160 shutdown_connections 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.579+0000 7f99292f4000 1 -- 172.21.15.45:0/1570683160 wait complete. 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- start start 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 Processor -- start 2024-04-25T10:20:59.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f50691d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f50691fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f506922010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f506684550 con 0x55f5068bd290 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f506677d20 con 0x55f5068bc970 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5066c5340 con 0x55f506872cd0 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 10 client.0 _ll_register_callbacks cb 0x7f9918009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f506922010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f50691d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f506922010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40950/0 (socket says 172.21.15.45:40950) 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 -- 172.21.15.45:0/3316304995 learned_addr learned my addr 172.21.15.45:0/3316304995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f9916000640 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f50691fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 -- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 msgr2=0x55f50691d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f50691d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 -- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 msgr2=0x55f50691fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f50691fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 -- 172.21.15.45:0/3316304995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f50671aa30 con 0x55f5068bd290 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991f400640 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f506922010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9918000c10 tx=0x7f9918013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9774ff99269fd5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 1 -- 172.21.15.45:0/3316304995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918003620 con 0x55f5068bd290 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 1 -- 172.21.15.45:0/3316304995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99180037c0 con 0x55f5068bd290 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9918010b60 con 0x55f5068bd290 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 1 -- 172.21.15.45:0/3316304995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f991803a050 con 0x55f5068bd290 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 10 client.5745 fetch_fsmap learned FSMap version 27 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55f506668150 con 0x55f5068bd290 2024-04-25T10:20:59.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 1 -- 172.21.15.45:0/3316304995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918010b60 con 0x55f5068bd290 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 1 -- 172.21.15.45:0/3316304995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f991803d020 con 0x55f5068bd290 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 10 client.5745 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 10 client.5745 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 20 client.5745 populate_metadata read hostname 'smithi045' 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506922550 0x55f506924940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 4 client.5745 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55f5066ce250 con 0x55f506922550 2024-04-25T10:20:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991ea00640 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506922550 0x55f506924940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f991ea00640 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506922550 0x55f506924940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84413a2b9bd3262 server_cookie=eb90738482e1f591 in_seq=0 out_seq=0 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.583+0000 7f990cc00640 10 client.5745 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f990cc00640 1 -- 172.21.15.45:0/3316304995 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55f5066ce250 con 0x55f506922550 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f990cc00640 10 client.5745 handle_command_reply: tid=0 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 client.5745 shutdown 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 msgr2=0x55f506922010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f506922010 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9918000c10 tx=0x7f9918013000 comp rx=0 tx=0).stop 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 shutdown_connections 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f506922550 0x55f506924940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5068bd290 0x55f506922010 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5068bc970 0x55f50691fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 --2- 172.21.15.45:0/3316304995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f506872cd0 0x55f50691d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 >> 172.21.15.45:0/3316304995 conn(0x55f506879540 msgr2=0x55f506871e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f990cc00640 0 client.5745 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 shutdown_connections 2024-04-25T10:20:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 1 -- 172.21.15.45:0/3316304995 wait complete. 2024-04-25T10:20:59.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f99292f4000 20 client.5745 trim_cache size 0 max 16384 2024-04-25T10:20:59.594 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:20:59.594 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": -2 2024-04-25T10:20:59.594 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98fc066f60 msgr2=0x7f98fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98fc066f60 0x7f98fc069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99200b16c0 tx=0x7f9908008040 comp rx=0 tx=0).stop 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 msgr2=0x7f99200b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 0x7f99200b41b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f991001ade0 tx=0x7f99100194d0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 shutdown_connections 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98fc066f60 0x7f98fc069420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9920159540 0x7f9920147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99200aebd0 0x7f99200afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 --2- 172.21.15.45:0/1252435342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9920072020 0x7f99200b41b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 >> 172.21.15.45:0/1252435342 conn(0x7f99200669a0 msgr2=0x7f99200cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 shutdown_connections 2024-04-25T10:20:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.587+0000 7f9925e00640 1 -- 172.21.15.45:0/1252435342 wait complete. 2024-04-25T10:20:59.608 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:20:59.608 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:20:59.608 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:20:59.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:59.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 Processor -- start 2024-04-25T10:20:59.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 -- start start 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15b9e0 0x7fbd7c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd7c15bdc0 0x7fbd7c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c0ccfb0 con 0x7fbd7c0d1520 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7c0cd290 con 0x7fbd7c15bdc0 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd84600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0cd570 con 0x7fbd7c15b9e0 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd82600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd7c0d1520 0x7fbd7c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42912/0 (socket says 172.21.15.45:42912) 2024-04-25T10:20:59.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd82600640 1 -- 172.21.15.45:0/3475186489 learned_addr learned my addr 172.21.15.45:0/3475186489 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:20:59.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81200640 1 --2- 172.21.15.45:0/3475186489 >> v2:172.21.15.117:3300/0 conn(0x7fbd7c15bdc0 0x7fbd7c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81c00640 1 --2- 172.21.15.45:0/3475186489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15b9e0 0x7fbd7c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd7be00640 1 -- 172.21.15.45:0/3475186489 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648795128 0 0) 0x7fbd7c0ccfb0 con 0x7fbd7c0d1520 2024-04-25T10:20:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd7be00640 1 -- 172.21.15.45:0/3475186489 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd64003530 con 0x7fbd7c0d1520 2024-04-25T10:20:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81c00640 1 -- 172.21.15.45:0/3475186489 >> v2:172.21.15.117:3300/0 conn(0x7fbd7c15bdc0 msgr2=0x7fbd7c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81c00640 1 --2- 172.21.15.45:0/3475186489 >> v2:172.21.15.117:3300/0 conn(0x7fbd7c15bdc0 0x7fbd7c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81c00640 1 -- 172.21.15.45:0/3475186489 >> v1:172.21.15.45:6789/0 conn(0x7fbd7c0d1520 legacy=0x7fbd7c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81c00640 1 -- 172.21.15.45:0/3475186489 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0cd850 con 0x7fbd7c15b9e0 2024-04-25T10:20:59.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.747+0000 7fbd81200640 1 --2- 172.21.15.45:0/3475186489 >> v2:172.21.15.117:3300/0 conn(0x7fbd7c15bdc0 0x7fbd7c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd81c00640 1 --2- 172.21.15.45:0/3475186489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15b9e0 0x7fbd7c0cc410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd700097c0 tx=0x7fbd7001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3719f9a7b843de62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd7be00640 1 -- 172.21.15.45:0/3475186489 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd700b0070 con 0x7fbd7c15b9e0 2024-04-25T10:20:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd7be00640 1 -- 172.21.15.45:0/3475186489 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd700ab600 con 0x7fbd7c15b9e0 2024-04-25T10:20:59.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd7be00640 1 -- 172.21.15.45:0/3475186489 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd700ab9d0 con 0x7fbd7c15b9e0 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 -- 172.21.15.45:0/3475186489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15b9e0 msgr2=0x7fbd7c0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 --2- 172.21.15.45:0/3475186489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15b9e0 0x7fbd7c0cc410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd700097c0 tx=0x7fbd7001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 -- 172.21.15.45:0/3475186489 shutdown_connections 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 --2- 172.21.15.45:0/3475186489 >> v2:172.21.15.117:3300/0 conn(0x7fbd7c15bdc0 0x7fbd7c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 --2- 172.21.15.45:0/3475186489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15b9e0 0x7fbd7c0cc410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 -- 172.21.15.45:0/3475186489 >> 172.21.15.45:0/3475186489 conn(0x7fbd7c09a9d0 msgr2=0x7fbd7c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 -- 172.21.15.45:0/3475186489 shutdown_connections 2024-04-25T10:20:59.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.751+0000 7fbd84600640 1 -- 172.21.15.45:0/3475186489 wait complete. 2024-04-25T10:20:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 Processor -- start 2024-04-25T10:20:59.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- start start 2024-04-25T10:20:59.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 0x7fbd7c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd7c15b9e0 0x7fbd7c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15bdc0 0x7fbd7c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0c65f0 con 0x7fbd7c0d1520 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0c6470 con 0x7fbd7c15b9e0 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0c6770 con 0x7fbd7c15bdc0 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 0x7fbd7c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 0x7fbd7c0c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40956/0 (socket says 172.21.15.45:40956) 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 -- 172.21.15.45:0/900727749 learned_addr learned my addr 172.21.15.45:0/900727749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81200640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd7c15b9e0 0x7fbd7c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd82600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15bdc0 0x7fbd7c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 -- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15bdc0 msgr2=0x7fbd7c0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15bdc0 0x7fbd7c0c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 -- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd7c15b9e0 msgr2=0x7fbd7c155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd7c15b9e0 0x7fbd7c155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 -- 172.21.15.45:0/900727749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0c8f10 con 0x7fbd7c0d1520 2024-04-25T10:20:59.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd81c00640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 0x7fbd7c0c89d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbd7001a0d0 tx=0x7fbd70019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd700b0040 con 0x7fbd7c0d1520 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd700ba070 con 0x7fbd7c0d1520 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c15d540 con 0x7fbd7c0d1520 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70006e20 con 0x7fbd7c0d1520 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c15db00 con 0x7fbd7c0d1520 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.755+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c04ef30 con 0x7fbd7c0d1520 2024-04-25T10:20:59.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.759+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd700b4630 con 0x7fbd7c0d1520 2024-04-25T10:20:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.759+0000 7fbd79600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbd5c066f90 0x7fbd5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:59.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.759+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd7011eb70 con 0x7fbd7c0d1520 2024-04-25T10:20:59.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.759+0000 7fbd81200640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbd5c066f90 0x7fbd5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:59.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.759+0000 7fbd81200640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbd5c066f90 0x7fbd5c069450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009580 tx=0x7fbd68009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:59.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.763+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd70016090 con 0x7fbd7c0d1520 2024-04-25T10:20:59.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd7c15a990 con 0x7fbd7c0d1520 2024-04-25T10:20:59.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd79600640 1 -- 172.21.15.45:0/900727749 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fbd700ef490 con 0x7fbd7c0d1520 2024-04-25T10:20:59.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:20:59.962 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:20:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbd5c066f90 msgr2=0x7fbd5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbd5c066f90 0x7fbd5c069450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009580 tx=0x7fbd68009190 comp rx=0 tx=0).stop 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 msgr2=0x7fbd7c0c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 0x7fbd7c0c89d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbd7001a0d0 tx=0x7fbd70019240 comp rx=0 tx=0).stop 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 shutdown_connections 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbd5c066f90 0x7fbd5c069450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd7c15bdc0 0x7fbd7c0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd7c15b9e0 0x7fbd7c155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 --2- 172.21.15.45:0/900727749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7c0d1520 0x7fbd7c0c89d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 >> 172.21.15.45:0/900727749 conn(0x7fbd7c09a9d0 msgr2=0x7fbd7c15ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 shutdown_connections 2024-04-25T10:20:59.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:20:59.959+0000 7fbd84600640 1 -- 172.21.15.45:0/900727749 wait complete. 2024-04-25T10:20:59.983 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:21:00.319 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:00.319 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:00.319 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:00.334 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:00.334 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:21:00.334 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:00.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:21:00.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 Processor -- start 2024-04-25T10:21:00.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 -- start start 2024-04-25T10:21:00.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdef40aef80 0x7fdef415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdef40b31c0 0x7fdef40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef415b080 con 0x7fdef4070350 2024-04-25T10:21:00.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef415adb0 con 0x7fdef40aef80 2024-04-25T10:21:00.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef9200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef415eb60 con 0x7fdef40b31c0 2024-04-25T10:21:00.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdef40aef80 0x7fdef415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdef40aef80 0x7fdef415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40966/0 (socket says 172.21.15.45:40966) 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef1e00640 1 -- 172.21.15.45:0/872452676 learned_addr learned my addr 172.21.15.45:0/872452676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef2800640 1 --2- 172.21.15.45:0/872452676 >> v2:172.21.15.117:3300/0 conn(0x7fdef40b31c0 0x7fdef40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.475+0000 7fdef1e00640 1 -- 172.21.15.45:0/872452676 >> v2:172.21.15.117:3300/0 conn(0x7fdef40b31c0 msgr2=0x7fdef40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef1e00640 1 --2- 172.21.15.45:0/872452676 >> v2:172.21.15.117:3300/0 conn(0x7fdef40b31c0 0x7fdef40b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef1e00640 1 -- 172.21.15.45:0/872452676 >> v1:172.21.15.45:6789/0 conn(0x7fdef4070350 legacy=0x7fdef415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef1e00640 1 -- 172.21.15.45:0/872452676 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef415ee40 con 0x7fdef40aef80 2024-04-25T10:21:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef1e00640 1 --2- 172.21.15.45:0/872452676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef415e480 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdef40d0750 tx=0x7fdee401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e6d10b5db4b9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef0a00640 1 -- 172.21.15.45:0/872452676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40a79e0 con 0x7fdef40aef80 2024-04-25T10:21:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef0a00640 1 -- 172.21.15.45:0/872452676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee40a7b80 con 0x7fdef40aef80 2024-04-25T10:21:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef0a00640 1 -- 172.21.15.45:0/872452676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40af3f0 con 0x7fdef40aef80 2024-04-25T10:21:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 -- 172.21.15.45:0/872452676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 msgr2=0x7fdef415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 --2- 172.21.15.45:0/872452676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef415e480 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdef40d0750 tx=0x7fdee401ec00 comp rx=0 tx=0).stop 2024-04-25T10:21:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 -- 172.21.15.45:0/872452676 shutdown_connections 2024-04-25T10:21:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 --2- 172.21.15.45:0/872452676 >> v2:172.21.15.117:3300/0 conn(0x7fdef40b31c0 0x7fdef40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 --2- 172.21.15.45:0/872452676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef415e480 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 -- 172.21.15.45:0/872452676 >> 172.21.15.45:0/872452676 conn(0x7fdef40669a0 msgr2=0x7fdef4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 -- 172.21.15.45:0/872452676 shutdown_connections 2024-04-25T10:21:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.479+0000 7fdef9200640 1 -- 172.21.15.45:0/872452676 wait complete. 2024-04-25T10:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 Processor -- start 2024-04-25T10:21:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 -- start start 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef4070350 0x7fdef414d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef40b31c0 0x7fdef4150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef406c250 con 0x7fdef40aef80 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef406c0d0 con 0x7fdef4070350 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef406c3d0 con 0x7fdef40b31c0 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef4070350 0x7fdef414d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef414f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40982/0 (socket says 172.21.15.45:40982) 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 -- 172.21.15.45:0/3946405551 learned_addr learned my addr 172.21.15.45:0/3946405551 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef2800640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef40b31c0 0x7fdef4150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 -- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef40b31c0 msgr2=0x7fdef4150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef40b31c0 0x7fdef4150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 -- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef4070350 msgr2=0x7fdef414d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef4070350 0x7fdef414d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 -- 172.21.15.45:0/3946405551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4151650 con 0x7fdef40aef80 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef1400640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef414f080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c420 tx=0x7fdee000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdeea000640 1 -- 172.21.15.45:0/3946405551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00052d0 con 0x7fdef40aef80 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdeea000640 1 -- 172.21.15.45:0/3946405551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0005470 con 0x7fdef40aef80 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdeea000640 1 -- 172.21.15.45:0/3946405551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0007260 con 0x7fdef40aef80 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef4151930 con 0x7fdef40aef80 2024-04-25T10:21:00.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.483+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef40af760 con 0x7fdef40aef80 2024-04-25T10:21:00.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.487+0000 7fdeea000640 1 -- 172.21.15.45:0/3946405551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdee0007400 con 0x7fdef40aef80 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.487+0000 7fdeea000640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fded4066f60 0x7fded4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.487+0000 7fdeea000640 1 -- 172.21.15.45:0/3946405551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdee0084490 con 0x7fdef40aef80 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.487+0000 7fdefc5d0000 1 Processor -- start 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.487+0000 7fdef1e00640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fded4066f60 0x7fded4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.487+0000 7fdefc5d0000 1 -- start start 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1e00640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fded4066f60 0x7fded4069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdee40b0000 tx=0x7fdee40a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563087767ea0 0x563087768280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 0x56308771e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308756cdb0 con 0x5630877687c0 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630874f5230 con 0x563087767ea0 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563087568e50 con 0x56308771e710 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 0x56308771e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 0x56308771e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40994/0 (socket says 172.21.15.45:40994) 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 -- 172.21.15.45:0/614665912 learned_addr learned my addr 172.21.15.45:0/614665912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1e00640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563087767ea0 0x563087768280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef2800640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 -- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 msgr2=0x56308771eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 -- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563087767ea0 msgr2=0x563087768280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563087767ea0 0x563087768280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 -- 172.21.15.45:0/614665912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56308752e750 con 0x5630877687c0 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef2800640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1e00640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563087767ea0 0x563087768280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdef1400640 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 0x56308771e1d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdee0054d40 tx=0x7fdee0054e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67611a4492845796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdedea00640 1 -- 172.21.15.45:0/614665912 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0052e50 con 0x5630877687c0 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdedea00640 1 -- 172.21.15.45:0/614665912 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee00530b0 con 0x5630877687c0 2024-04-25T10:21:00.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdedea00640 1 -- 172.21.15.45:0/614665912 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00840d0 con 0x5630877687c0 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- 172.21.15.45:0/614665912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 msgr2=0x56308771e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 0x56308771e1d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdee0054d40 tx=0x7fdee0054e70 comp rx=0 tx=0).stop 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- 172.21.15.45:0/614665912 shutdown_connections 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630877687c0 0x56308771e1d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/614665912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563087767ea0 0x563087768280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- 172.21.15.45:0/614665912 >> 172.21.15.45:0/614665912 conn(0x56308771a6c0 msgr2=0x56308771cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- 172.21.15.45:0/614665912 shutdown_connections 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- 172.21.15.45:0/614665912 wait complete. 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- start start 2024-04-25T10:21:00.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 Processor -- start 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 0x5630877512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563087767ea0 0x563087751820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x563087751d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.491+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563087527bf0 con 0x56308771e710 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563087637f90 con 0x5630877687c0 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563087569e20 con 0x563087767ea0 2024-04-25T10:21:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 10 client.0 _ll_register_callbacks cb 0x563087709790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 0x5630877512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 0x5630877512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40996/0 (socket says 172.21.15.45:40996) 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 -- 172.21.15.45:0/4177783434 learned_addr learned my addr 172.21.15.45:0/4177783434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1400640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563087767ea0 0x563087751820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef2800640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x563087751d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 -- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563087767ea0 msgr2=0x563087751820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563087767ea0 0x563087751820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 -- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 msgr2=0x563087751d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x563087751d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 -- 172.21.15.45:0/4177783434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56308750f640 con 0x56308771e710 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef2800640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x563087751d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1400640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563087767ea0 0x563087751820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 0x5630877512e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdee4025210 tx=0x7fdee401d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d501378a31c95884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 1 -- 172.21.15.45:0/4177783434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40b8c00 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 1 -- 172.21.15.45:0/4177783434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdee40b8da0 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee00530b0 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 1 -- 172.21.15.45:0/4177783434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee40c3440 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 10 client.5766 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5630874f5480 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 1 -- 172.21.15.45:0/4177783434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee00530b0 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 1 -- 172.21.15.45:0/4177783434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdee40d6020 con 0x56308771e710 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 10 client.5766 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 10 client.5766 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 20 client.5766 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777bf80 0x56308777e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 4 client.5766 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563087508bb0 con 0x56308777bf80 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1400640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777bf80 0x56308777e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdef1400640 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777bf80 0x56308777e370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d526516239201b server_cookie=44b7053f04980e4b in_seq=0 out_seq=0 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 10 client.5766 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 1 -- 172.21.15.45:0/4177783434 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563087508bb0 con 0x56308777bf80 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 10 client.5766 handle_command_reply: tid=0 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 client.5766 shutdown 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 msgr2=0x5630877512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 0x5630877512e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdee4025210 tx=0x7fdee401d440 comp rx=0 tx=0).stop 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 shutdown_connections 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777bf80 0x56308777e370 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x563087751d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563087767ea0 0x563087751820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4177783434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56308771e710 0x5630877512e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.495+0000 7fded3400640 0 client.5766 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:00.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.499+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 >> 172.21.15.45:0/4177783434 conn(0x56308771a6c0 msgr2=0x56308771d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.499+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 shutdown_connections 2024-04-25T10:21:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.499+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4177783434 wait complete. 2024-04-25T10:21:00.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.499+0000 7fdefc5d0000 20 client.5766 trim_cache size 0 max 16384 2024-04-25T10:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.511+0000 7fdefc5d0000 1 Processor -- start 2024-04-25T10:21:00.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.511+0000 7fdefc5d0000 1 -- start start 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x56308774be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x56308774c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41fedc0 con 0x563087767ea0 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41f6d40 con 0x5630877687c0 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41f6140 con 0x56308771e710 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x56308774be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x56308774be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41012/0 (socket says 172.21.15.45:41012) 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 -- 172.21.15.45:0/3524833421 learned_addr learned my addr 172.21.15.45:0/3524833421 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1e00640 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef2800640 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x56308774c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 -- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 msgr2=0x56308771dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 -- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 msgr2=0x56308774c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x56308774c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 -- 172.21.15.45:0/3524833421 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630875eeef0 con 0x563087767ea0 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdef1400640 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x56308774be80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdee0054eb0 tx=0x7fdee001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3db8fbb58074461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdede000640 1 -- 172.21.15.45:0/3524833421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0051420 con 0x563087767ea0 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdede000640 1 -- 172.21.15.45:0/3524833421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee00840d0 con 0x563087767ea0 2024-04-25T10:21:00.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdede000640 1 -- 172.21.15.45:0/3524833421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00515c0 con 0x563087767ea0 2024-04-25T10:21:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- 172.21.15.45:0/3524833421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 msgr2=0x56308774be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x56308774be80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdee0054eb0 tx=0x7fdee001c000 comp rx=0 tx=0).stop 2024-04-25T10:21:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- 172.21.15.45:0/3524833421 shutdown_connections 2024-04-25T10:21:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630877687c0 0x56308774c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x56308774be80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/3524833421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56308771e710 0x56308771dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- 172.21.15.45:0/3524833421 >> 172.21.15.45:0/3524833421 conn(0x56308771a6c0 msgr2=0x563087767430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- 172.21.15.45:0/3524833421 shutdown_connections 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- 172.21.15.45:0/3524833421 wait complete. 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 -- start start 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.515+0000 7fdefc5d0000 1 Processor -- start 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56308771e710 0x563087779130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x563087779670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5630877687c0 0x56308777f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563087727200 con 0x563087767ea0 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41f6740 con 0x56308771e710 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563087527bf0 con 0x5630877687c0 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 10 client.0 _ll_register_callbacks cb 0x7fdee00076f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x563087779670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x563087779670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41014/0 (socket says 172.21.15.45:41014) 2024-04-25T10:21:00.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 -- 172.21.15.45:0/4108537105 learned_addr learned my addr 172.21.15.45:0/4108537105 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 -- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5630877687c0 msgr2=0x56308777f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5630877687c0 0x56308777f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 -- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56308771e710 msgr2=0x563087779130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56308771e710 0x563087779130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56308771e710 0x563087779130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 -- 172.21.15.45:0/4108537105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56308752c9a0 con 0x563087767ea0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56308771e710 0x563087779130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1400640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x563087779670 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdee0054ab0 tx=0x7fdee0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d009c26f22122b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 1 -- 172.21.15.45:0/4108537105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00530b0 con 0x563087767ea0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 1 -- 172.21.15.45:0/4108537105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdee0054200 con 0x563087767ea0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee00840d0 con 0x563087767ea0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 1 -- 172.21.15.45:0/4108537105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee007b050 con 0x563087767ea0 2024-04-25T10:21:00.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 10 client.5772 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5630872295a0 con 0x563087767ea0 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 1 -- 172.21.15.45:0/4108537105 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee00840d0 con 0x563087767ea0 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 1 -- 172.21.15.45:0/4108537105 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdee0050b00 con 0x563087767ea0 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 10 client.5772 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 10 client.5772 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 20 client.5772 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777f900 0x56308777fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 4 client.5772 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x56308751c080 con 0x56308777f900 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777f900 0x56308777fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdef1e00640 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777f900 0x56308777fd10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b20a2000a573eaa server_cookie=8c0dfe79a54616d3 in_seq=0 out_seq=0 2024-04-25T10:21:00.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 10 client.5772 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 1 -- 172.21.15.45:0/4108537105 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56308751c080 con 0x56308777f900 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 10 client.5772 handle_command_reply: tid=0 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 client.5772 shutdown 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 msgr2=0x563087779670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x563087779670 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdee0054ab0 tx=0x7fdee0000ee0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 shutdown_connections 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56308777f900 0x56308777fd10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5630877687c0 0x56308777f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563087767ea0 0x563087779670 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fded3400640 0 client.5772 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 --2- 172.21.15.45:0/4108537105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56308771e710 0x563087779130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 >> 172.21.15.45:0/4108537105 conn(0x56308771a6c0 msgr2=0x563087766ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 shutdown_connections 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 1 -- 172.21.15.45:0/4108537105 wait complete. 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.519+0000 7fdefc5d0000 20 client.5772 trim_cache size 0 max 16384 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "4613f334-86a8-442f-904e-d51489876fca", 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:00.529 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fded4066f60 msgr2=0x7fded4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fded4066f60 0x7fded4069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdee40b0000 tx=0x7fdee40a9070 comp rx=0 tx=0).stop 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 msgr2=0x7fdef414f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef414f080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c420 tx=0x7fdee000c820 comp rx=0 tx=0).stop 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 shutdown_connections 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fded4066f60 0x7fded4069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdef40b31c0 0x7fdef4150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdef40aef80 0x7fdef414f080 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 --2- 172.21.15.45:0/3946405551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdef4070350 0x7fdef414d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 >> 172.21.15.45:0/3946405551 conn(0x7fdef40669a0 msgr2=0x7fdef40cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 shutdown_connections 2024-04-25T10:21:00.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.523+0000 7fdef9200640 1 -- 172.21.15.45:0/3946405551 wait complete. 2024-04-25T10:21:00.543 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '4613f334-86a8-442f-904e-d51489876fca', 'mode': 'asynchronous'}' 2024-04-25T10:21:00.544 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:21:00.544 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:21:00.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:00.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.679+0000 7f8730600640 1 Processor -- start 2024-04-25T10:21:00.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.679+0000 7f8730600640 1 -- start start 2024-04-25T10:21:00.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8730600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87280d0040 0x7f872815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8730600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f87280b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8730600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8728071050 con 0x7f87281550b0 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8730600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872815d040 con 0x7f87280d0040 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8730600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280712f0 con 0x7f872815b740 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872dc00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f87281550b0 0x7f87280cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42926/0 (socket says 172.21.15.45:42926) 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872dc00640 1 -- 172.21.15.45:0/2657486401 learned_addr learned my addr 172.21.15.45:0/2657486401 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872e600640 1 --2- 172.21.15.45:0/2657486401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f87280b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872d200640 1 --2- 172.21.15.45:0/2657486401 >> v2:172.21.15.117:3300/0 conn(0x7f87280d0040 0x7f872815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8727e00640 1 -- 172.21.15.45:0/2657486401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696991947 0 0) 0x7f8728071050 con 0x7f87281550b0 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f8727e00640 1 -- 172.21.15.45:0/2657486401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8710003530 con 0x7f87281550b0 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872e600640 1 -- 172.21.15.45:0/2657486401 >> v2:172.21.15.117:3300/0 conn(0x7f87280d0040 msgr2=0x7f872815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872e600640 1 --2- 172.21.15.45:0/2657486401 >> v2:172.21.15.117:3300/0 conn(0x7f87280d0040 0x7f872815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872e600640 1 -- 172.21.15.45:0/2657486401 >> v1:172.21.15.45:6789/0 conn(0x7f87281550b0 legacy=0x7f87280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.683+0000 7f872e600640 1 -- 172.21.15.45:0/2657486401 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280715d0 con 0x7f872815b740 2024-04-25T10:21:00.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f872e600640 1 --2- 172.21.15.45:0/2657486401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f87280b3260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8720009d90 tx=0x7f872001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03578644c12fc7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8727e00640 1 -- 172.21.15.45:0/2657486401 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200b2070 con 0x7f872815b740 2024-04-25T10:21:00.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8727e00640 1 -- 172.21.15.45:0/2657486401 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87200a5a00 con 0x7f872815b740 2024-04-25T10:21:00.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8727e00640 1 -- 172.21.15.45:0/2657486401 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200a5dd0 con 0x7f872815b740 2024-04-25T10:21:00.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 -- 172.21.15.45:0/2657486401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 msgr2=0x7f87280b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 --2- 172.21.15.45:0/2657486401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f87280b3260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8720009d90 tx=0x7f872001c940 comp rx=0 tx=0).stop 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 -- 172.21.15.45:0/2657486401 shutdown_connections 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 --2- 172.21.15.45:0/2657486401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f87280b3260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 --2- 172.21.15.45:0/2657486401 >> v2:172.21.15.117:3300/0 conn(0x7f87280d0040 0x7f872815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 -- 172.21.15.45:0/2657486401 >> 172.21.15.45:0/2657486401 conn(0x7f8728066b70 msgr2=0x7f8728066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 -- 172.21.15.45:0/2657486401 shutdown_connections 2024-04-25T10:21:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 -- 172.21.15.45:0/2657486401 wait complete. 2024-04-25T10:21:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 Processor -- start 2024-04-25T10:21:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.687+0000 7f8730600640 1 -- start start 2024-04-25T10:21:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 0x7f8728150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87281550b0 0x7f8728145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f872814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872815bd50 con 0x7f87280d0040 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872815bbd0 con 0x7f87281550b0 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872815c030 con 0x7f872815b740 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 0x7f8728150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 0x7f8728150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41028/0 (socket says 172.21.15.45:41028) 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 -- 172.21.15.45:0/1980934356 learned_addr learned my addr 172.21.15.45:0/1980934356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872e600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f872814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872d200640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87281550b0 0x7f8728145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 -- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 msgr2=0x7f872814f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f872814f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 -- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87281550b0 msgr2=0x7f8728145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87281550b0 0x7f8728145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 -- 172.21.15.45:0/1980934356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728152010 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872d200640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87281550b0 0x7f8728145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f872dc00640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 0x7f8728150740 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c450 tx=0x7f871c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c017070 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c005cf0 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8728150f70 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c007290 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8728151530 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872804ef30 con 0x7f87280d0040 2024-04-25T10:21:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.691+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f871c01a070 con 0x7f87280d0040 2024-04-25T10:21:00.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.695+0000 7f8725600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8708066f60 0x7f8708069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:00.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.695+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f871c084430 con 0x7f87280d0040 2024-04-25T10:21:00.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.695+0000 7f872d200640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8708066f60 0x7f8708069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.695+0000 7f872d200640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8708066f60 0x7f8708069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f872806e5a0 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:00.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.699+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f871c054d50 con 0x7f87280d0040 2024-04-25T10:21:00.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.895+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87280cf190 con 0x7f87280d0040 2024-04-25T10:21:00.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.895+0000 7f8725600640 1 -- 172.21.15.45:0/1980934356 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f871c058df0 con 0x7f87280d0040 2024-04-25T10:21:00.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:00.900 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:00.900 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8708066f60 msgr2=0x7f8708069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8708066f60 0x7f8708069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f872806e5a0 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 msgr2=0x7f8728150740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 0x7f8728150740 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c450 tx=0x7f871c00c910 comp rx=0 tx=0).stop 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 shutdown_connections 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8708066f60 0x7f8708069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f872815b740 0x7f872814f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87281550b0 0x7f8728145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 --2- 172.21.15.45:0/1980934356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87280d0040 0x7f8728150740 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 >> 172.21.15.45:0/1980934356 conn(0x7f8728066b70 msgr2=0x7f872815da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 shutdown_connections 2024-04-25T10:21:00.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:00.899+0000 7f8730600640 1 -- 172.21.15.45:0/1980934356 wait complete. 2024-04-25T10:21:00.922 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:21:01.066 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:01.066 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:01.066 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:01.081 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:01.081 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:21:01.081 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:01.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:21:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1778400640 1 Processor -- start 2024-04-25T10:21:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1778400640 1 -- start start 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1778400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17700aeda0 0x7f1770071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1778400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1770159920 con 0x7f1770159540 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1778400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177015c2f0 con 0x7f1770072020 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1778400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177015c5d0 con 0x7f17700aeda0 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17700aeda0 0x7f1770071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1770159540 0x7f17700ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42932/0 (socket says 172.21.15.45:42932) 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17700aeda0 0x7f1770071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41034/0 (socket says 172.21.15.45:41034) 2024-04-25T10:21:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775a00640 1 -- 172.21.15.45:0/1680280211 learned_addr learned my addr 172.21.15.45:0/1680280211 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:01.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775000640 1 -- 172.21.15.45:0/1680280211 >> v1:172.21.15.117:6789/0 conn(0x7f1770072020 legacy=0x7f1770070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775000640 1 -- 172.21.15.45:0/1680280211 >> v1:172.21.15.45:6789/0 conn(0x7f1770159540 legacy=0x7f17700ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f1775000640 1 -- 172.21.15.45:0/1680280211 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177015c8b0 con 0x7f17700aeda0 2024-04-25T10:21:01.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.223+0000 7f176fe00640 1 -- 172.21.15.45:0/1680280211 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524923072 0 0) 0x7f1770159920 con 0x7f1770159540 2024-04-25T10:21:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1775000640 1 --2- 172.21.15.45:0/1680280211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17700aeda0 0x7f1770071ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f175c0097c0 tx=0x7f175c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fce24bf8e8b256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f176fe00640 1 -- 172.21.15.45:0/1680280211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0aa5f0 con 0x7f17700aeda0 2024-04-25T10:21:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f176fe00640 1 -- 172.21.15.45:0/1680280211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c0af070 con 0x7f17700aeda0 2024-04-25T10:21:01.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f176fe00640 1 -- 172.21.15.45:0/1680280211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0aa9b0 con 0x7f17700aeda0 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 -- 172.21.15.45:0/1680280211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17700aeda0 msgr2=0x7f1770071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 --2- 172.21.15.45:0/1680280211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17700aeda0 0x7f1770071ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f175c0097c0 tx=0x7f175c01c470 comp rx=0 tx=0).stop 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 -- 172.21.15.45:0/1680280211 shutdown_connections 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 --2- 172.21.15.45:0/1680280211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17700aeda0 0x7f1770071ae0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 -- 172.21.15.45:0/1680280211 >> 172.21.15.45:0/1680280211 conn(0x7f17700669a0 msgr2=0x7f1770066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 -- 172.21.15.45:0/1680280211 shutdown_connections 2024-04-25T10:21:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.227+0000 7f1778400640 1 -- 172.21.15.45:0/1680280211 wait complete. 2024-04-25T10:21:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 Processor -- start 2024-04-25T10:21:01.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 -- start start 2024-04-25T10:21:01.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 0x7f177014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17700aeda0 0x7f177014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1770159540 0x7f1770154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1770169940 con 0x7f1770159540 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1770072430 con 0x7f1770072020 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1770169ac0 con 0x7f17700aeda0 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1775a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 0x7f177014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1770159540 0x7f1770154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1775a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 0x7f177014a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54338/0 (socket says 172.21.15.45:54338) 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1775a00640 1 -- 172.21.15.45:0/1357465021 learned_addr learned my addr 172.21.15.45:0/1357465021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1775000640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17700aeda0 0x7f177014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 -- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17700aeda0 msgr2=0x7f177014b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17700aeda0 0x7f177014b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 -- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 msgr2=0x7f177014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 0x7f177014a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 -- 172.21.15.45:0/1357465021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770152a60 con 0x7f1770159540 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1775a00640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 0x7f177014a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:01.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1776400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1770159540 0x7f1770154e30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f176800c700 tx=0x7f176800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f176d600640 1 -- 172.21.15.45:0/1357465021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768003b80 con 0x7f1770159540 2024-04-25T10:21:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f176d600640 1 -- 172.21.15.45:0/1357465021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768003d20 con 0x7f1770159540 2024-04-25T10:21:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f176d600640 1 -- 172.21.15.45:0/1357465021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800aa30 con 0x7f1770159540 2024-04-25T10:21:01.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177014e6e0 con 0x7f1770159540 2024-04-25T10:21:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.231+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177014eca0 con 0x7f1770159540 2024-04-25T10:21:01.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f176d600640 1 -- 172.21.15.45:0/1357465021 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f176800abd0 con 0x7f1770159540 2024-04-25T10:21:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f176d600640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1750066f90 0x7f1750069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f176d600640 1 -- 172.21.15.45:0/1357465021 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1768084d40 con 0x7f1770159540 2024-04-25T10:21:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1750066f90 0x7f1750069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 Processor -- start 2024-04-25T10:21:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 -- start start 2024-04-25T10:21:01.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1750066f90 0x7f1750069450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1764000f70 tx=0x7f1764007ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c8e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c72a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544b2b4a0 con 0x55f544c8e200 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544a93210 con 0x55f544c72f60 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f544a1b980 con 0x55f544c8eb20 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c8e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c8e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41052/0 (socket says 172.21.15.45:41052) 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 -- 172.21.15.45:0/2205216727 learned_addr learned my addr 172.21.15.45:0/2205216727 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775000640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c72a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 -- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 msgr2=0x55f544c72a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1776400640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c72a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 -- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 msgr2=0x55f544c73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c73340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775a00640 1 -- 172.21.15.45:0/2205216727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f544a8e200 con 0x55f544c8e200 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1775000640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c72a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.235+0000 7f1776400640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775a00640 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c8e5e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1764004fa0 tx=0x7f1764013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aeef204aeeaf223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1762000640 1 -- 172.21.15.45:0/2205216727 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764026070 con 0x55f544c8e200 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1762000640 1 -- 172.21.15.45:0/2205216727 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764012350 con 0x55f544c8e200 2024-04-25T10:21:01.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1762000640 1 -- 172.21.15.45:0/2205216727 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764012700 con 0x55f544c8e200 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- 172.21.15.45:0/2205216727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 msgr2=0x55f544c8e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c8e5e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1764004fa0 tx=0x7f1764013000 comp rx=0 tx=0).stop 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- 172.21.15.45:0/2205216727 shutdown_connections 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c72a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- 172.21.15.45:0/2205216727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c8e5e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- 172.21.15.45:0/2205216727 >> 172.21.15.45:0/2205216727 conn(0x55f544c3b4f0 msgr2=0x55f544c4aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- 172.21.15.45:0/2205216727 shutdown_connections 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- 172.21.15.45:0/2205216727 wait complete. 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- start start 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 Processor -- start 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c95980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c95ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544a4dd80 con 0x55f544c8e200 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544a8f2b0 con 0x55f544c72f60 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f544a54830 con 0x55f544c8eb20 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f177b844000 10 client.0 _ll_register_callbacks cb 0x55f544b46070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c95ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c95ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41062/0 (socket says 172.21.15.45:41062) 2024-04-25T10:21:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 -- 172.21.15.45:0/4254336253 learned_addr learned my addr 172.21.15.45:0/4254336253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1776400640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775a00640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c95980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 -- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 msgr2=0x55f544c73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c73340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 -- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 msgr2=0x55f544c95980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.239+0000 7f1775000640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c95980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1775000640 1 -- 172.21.15.45:0/4254336253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f544a5f130 con 0x55f544c8e200 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1775000640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c95ec0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f175c0ac040 tx=0x7f175c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17674a749ef57cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 1 -- 172.21.15.45:0/4254336253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0b5220 con 0x55f544c8e200 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 1 -- 172.21.15.45:0/4254336253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f175c0b5b30 con 0x55f544c8e200 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1764012350 con 0x55f544c8e200 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 1 -- 172.21.15.45:0/4254336253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175c0b5d30 con 0x55f544c8e200 2024-04-25T10:21:01.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 10 client.5793 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55f544b14490 con 0x55f544c8e200 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 1 -- 172.21.15.45:0/4254336253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764012350 con 0x55f544c8e200 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 1 -- 172.21.15.45:0/4254336253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f175c0c9420 con 0x55f544c8e200 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 10 client.5793 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 10 client.5793 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 20 client.5793 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544c740e0 0x55f544c744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 4 client.5793 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f544a2f020 con 0x55f544c740e0 2024-04-25T10:21:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1775a00640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544c740e0 0x55f544c744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1775a00640 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544c740e0 0x55f544c744c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a7306246192a78 server_cookie=94072bf0f5916a0c in_seq=0 out_seq=0 2024-04-25T10:21:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 10 client.5793 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 1 -- 172.21.15.45:0/4254336253 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f544a2f020 con 0x55f544c740e0 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 10 client.5793 handle_command_reply: tid=0 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 client.5793 shutdown 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 msgr2=0x55f544c95ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c95ec0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f175c0ac040 tx=0x7f175c01acb0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 shutdown_connections 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544c740e0 0x55f544c744c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c95ec0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f1756a00640 0 client.5793 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 --2- 172.21.15.45:0/4254336253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c95980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 >> 172.21.15.45:0/4254336253 conn(0x55f544c3b4f0 msgr2=0x55f544c42aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 shutdown_connections 2024-04-25T10:21:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 1 -- 172.21.15.45:0/4254336253 wait complete. 2024-04-25T10:21:01.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.243+0000 7f177b844000 20 client.5793 trim_cache size 0 max 16384 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 Processor -- start 2024-04-25T10:21:01.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 -- start start 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c48e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c49340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1770202ec0 con 0x55f544c8e200 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17701540d0 con 0x55f544c72f60 2024-04-25T10:21:01.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1770153ad0 con 0x55f544c8eb20 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f1775000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c49340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f1775000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c49340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41064/0 (socket says 172.21.15.45:41064) 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f1775000640 1 -- 172.21.15.45:0/4095808441 learned_addr learned my addr 172.21.15.45:0/4095808441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f1775a00640 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c48e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.259+0000 7f1776400640 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 -- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 msgr2=0x55f544c49880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c49880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 -- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 msgr2=0x55f544c48e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c48e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 -- 172.21.15.45:0/4095808441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f544a52a30 con 0x55f544c8e200 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c49340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f175c019080 tx=0x7f175c0b7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930f3d85cee5bd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1761600640 1 -- 172.21.15.45:0/4095808441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c006a30 con 0x55f544c8e200 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1761600640 1 -- 172.21.15.45:0/4095808441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c0cd070 con 0x55f544c8e200 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1761600640 1 -- 172.21.15.45:0/4095808441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0c9c50 con 0x55f544c8e200 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- 172.21.15.45:0/4095808441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 msgr2=0x55f544c49340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c49340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f175c019080 tx=0x7f175c0b7040 comp rx=0 tx=0).stop 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- 172.21.15.45:0/4095808441 shutdown_connections 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8eb20 0x55f544c49880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8e200 0x55f544c49340 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- 172.21.15.45:0/4095808441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c48e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- 172.21.15.45:0/4095808441 >> 172.21.15.45:0/4095808441 conn(0x55f544c3b4f0 msgr2=0x55f544c716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- 172.21.15.45:0/4095808441 shutdown_connections 2024-04-25T10:21:01.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- 172.21.15.45:0/4095808441 wait complete. 2024-04-25T10:21:01.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- start start 2024-04-25T10:21:01.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 Processor -- start 2024-04-25T10:21:01.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8e200 0x55f544c43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 0x55f544c444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544a507c0 con 0x55f544c8eb20 2024-04-25T10:21:01.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544a1f190 con 0x55f544c72f60 2024-04-25T10:21:01.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f544a92520 con 0x55f544c8e200 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f177b844000 10 client.0 _ll_register_callbacks cb 0x7f1764001c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 0x55f544c444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 0x55f544c444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41068/0 (socket says 172.21.15.45:41068) 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 -- 172.21.15.45:0/1757803010 learned_addr learned my addr 172.21.15.45:0/1757803010 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775a00640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8e200 0x55f544c43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 -- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8e200 msgr2=0x55f544c43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8e200 0x55f544c43fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 -- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 msgr2=0x55f544c43650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c43650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 -- 172.21.15.45:0/1757803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f544a5b040 con 0x55f544c8eb20 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775000640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8e200 0x55f544c43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1775a00640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1776400640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 0x55f544c444e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1768001340 tx=0x7f1768059790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982e148d11d87f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.263+0000 7f1756a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 1 -- 172.21.15.45:0/1757803010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768081060 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 1 -- 172.21.15.45:0/1757803010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1768051d60 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f175c0cd070 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 1 -- 172.21.15.45:0/1757803010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f176807cbc0 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 10 client.5799 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55f54474f900 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 1 -- 172.21.15.45:0/1757803010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c0cd070 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 1 -- 172.21.15.45:0/1757803010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1768006b60 con 0x55f544c8eb20 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 10 client.5799 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 10 client.5799 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 20 client.5799 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544cac040 0x55f544cac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 4 client.5799 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:21:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55f544aa0750 con 0x55f544cac040 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1775a00640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544cac040 0x55f544cac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1775a00640 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544cac040 0x55f544cac420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ca58438faa5eb0 server_cookie=573dd35b5b084b09 in_seq=0 out_seq=0 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 10 client.5799 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 1 -- 172.21.15.45:0/1757803010 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f544aa0750 con 0x55f544cac040 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 10 client.5799 handle_command_reply: tid=0 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 client.5799 shutdown 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 msgr2=0x55f544c444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 0x55f544c444e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1768001340 tx=0x7f1768059790 comp rx=0 tx=0).stop 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 shutdown_connections 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55f544cac040 0x55f544cac420 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f544c8eb20 0x55f544c444e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f544c8e200 0x55f544c43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 --2- 172.21.15.45:0/1757803010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f544c72f60 0x55f544c43650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 >> 172.21.15.45:0/1757803010 conn(0x55f544c3b4f0 msgr2=0x55f544c45c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1756a00640 0 client.5799 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 shutdown_connections 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 1 -- 172.21.15.45:0/1757803010 wait complete. 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f177b844000 20 client.5799 trim_cache size 0 max 16384 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:01.275 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "4391bf56-430a-47ee-bd62-826d5d85d110", 2024-04-25T10:21:01.276 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:01.276 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1750066f90 msgr2=0x7f1750069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1750066f90 0x7f1750069450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1764000f70 tx=0x7f1764007ee0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1770159540 msgr2=0x7f1770154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1770159540 0x7f1770154e30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f176800c700 tx=0x7f176800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 shutdown_connections 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1750066f90 0x7f1750069450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1770159540 0x7f1770154e30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17700aeda0 0x7f177014b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 --2- 172.21.15.45:0/1357465021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1770072020 0x7f177014a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 >> 172.21.15.45:0/1357465021 conn(0x7f17700669a0 msgr2=0x7f177006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 shutdown_connections 2024-04-25T10:21:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.267+0000 7f1778400640 1 -- 172.21.15.45:0/1357465021 wait complete. 2024-04-25T10:21:01.291 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '4391bf56-430a-47ee-bd62-826d5d85d110', 'mode': 'asynchronous'}' 2024-04-25T10:21:01.291 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T10:21:01.291 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:21:01.291 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 Processor -- start 2024-04-25T10:21:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 -- start start 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcef015df00 0x7fcef00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcef00d0750 0x7fcef00d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef00d0c00 con 0x7fcef00d1ce0 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00d2490 con 0x7fcef00d0750 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcef6200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00d2770 con 0x7fcef015df00 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcef00d0750 0x7fcef00d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcef00d0750 0x7fcef00d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41082/0 (socket says 172.21.15.45:41082) 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 -- 172.21.15.45:0/1489635239 learned_addr learned my addr 172.21.15.45:0/1489635239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fceefe00640 1 --1- 172.21.15.45:0/1489635239 >> v1:172.21.15.45:6789/0 conn(0x7fcef00d1ce0 0x7fcef00b5610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42934/0 (socket says 172.21.15.45:42934) 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fceef400640 1 --2- 172.21.15.45:0/1489635239 >> v2:172.21.15.117:3300/0 conn(0x7fcef015df00 0x7fcef00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 -- 172.21.15.45:0/1489635239 >> v2:172.21.15.117:3300/0 conn(0x7fcef015df00 msgr2=0x7fcef00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 --2- 172.21.15.45:0/1489635239 >> v2:172.21.15.117:3300/0 conn(0x7fcef015df00 0x7fcef00d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 -- 172.21.15.45:0/1489635239 >> v1:172.21.15.45:6789/0 conn(0x7fcef00d1ce0 legacy=0x7fcef00b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.431+0000 7fcee7e00640 1 -- 172.21.15.45:0/1489635239 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00d2a50 con 0x7fcef00d0750 2024-04-25T10:21:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcee7e00640 1 --2- 172.21.15.45:0/1489635239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef00d17a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcef00d1160 tx=0x7fced801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34258c5449d7ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fceeea00640 1 -- 172.21.15.45:0/1489635239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80aa650 con 0x7fcef00d0750 2024-04-25T10:21:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fceeea00640 1 -- 172.21.15.45:0/1489635239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced80af070 con 0x7fcef00d0750 2024-04-25T10:21:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fceeea00640 1 -- 172.21.15.45:0/1489635239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80aaa10 con 0x7fcef00d0750 2024-04-25T10:21:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 -- 172.21.15.45:0/1489635239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 msgr2=0x7fcef00d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 --2- 172.21.15.45:0/1489635239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef00d17a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcef00d1160 tx=0x7fced801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 -- 172.21.15.45:0/1489635239 shutdown_connections 2024-04-25T10:21:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 --2- 172.21.15.45:0/1489635239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef00d17a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 --2- 172.21.15.45:0/1489635239 >> v2:172.21.15.117:3300/0 conn(0x7fcef015df00 0x7fcef00d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 -- 172.21.15.45:0/1489635239 >> 172.21.15.45:0/1489635239 conn(0x7fcef006b290 msgr2=0x7fcef00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 -- 172.21.15.45:0/1489635239 shutdown_connections 2024-04-25T10:21:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.435+0000 7fcef6200640 1 -- 172.21.15.45:0/1489635239 wait complete. 2024-04-25T10:21:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 Processor -- start 2024-04-25T10:21:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- start start 2024-04-25T10:21:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef0154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef00d1ce0 0x7fcef00b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcef015df00 0x7fcef00c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00d2270 con 0x7fcef00d0750 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00d20f0 con 0x7fcef00d1ce0 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef016ab70 con 0x7fcef015df00 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef0154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef0154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41088/0 (socket says 172.21.15.45:41088) 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 -- 172.21.15.45:0/298920740 learned_addr learned my addr 172.21.15.45:0/298920740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcee7e00640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef00d1ce0 0x7fcef00b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceefe00640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcef015df00 0x7fcef00c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 -- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcef015df00 msgr2=0x7fcef00c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcef015df00 0x7fcef00c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 -- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef00d1ce0 msgr2=0x7fcef00b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef00d1ce0 0x7fcef00b88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 -- 172.21.15.45:0/298920740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef014dca0 con 0x7fcef00d0750 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcee7e00640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef00d1ce0 0x7fcef00b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceefe00640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcef015df00 0x7fcef00c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fceef400640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef0154ff0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcee000e8a0 tx=0x7fcee000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0017070 con 0x7fcef00d0750 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee000c940 con 0x7fcef00d0750 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000ccf0 con 0x7fcef00d0750 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef014df80 con 0x7fcef00d0750 2024-04-25T10:21:01.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef014e540 con 0x7fcef00d0750 2024-04-25T10:21:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.439+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef004ef30 con 0x7fcef00d0750 2024-04-25T10:21:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.443+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcee001a070 con 0x7fcef00d0750 2024-04-25T10:21:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.443+0000 7fcee7400640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcecc066f60 0x7fcecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.443+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcee0084f40 con 0x7fcef00d0750 2024-04-25T10:21:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.443+0000 7fcee7e00640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcecc066f60 0x7fcecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.443+0000 7fcee7e00640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcecc066f60 0x7fcecc069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fced80ae000 tx=0x7fced8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.447+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcee0055f40 con 0x7fcef00d0750 2024-04-25T10:21:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.643+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef014eee0 con 0x7fcef00d0750 2024-04-25T10:21:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.643+0000 7fcee7400640 1 -- 172.21.15.45:0/298920740 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcee0055840 con 0x7fcef00d0750 2024-04-25T10:21:01.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:01.647 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcecc066f60 msgr2=0x7fcecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcecc066f60 0x7fcecc069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fced80ae000 tx=0x7fced8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 msgr2=0x7fcef0154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef0154ff0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcee000e8a0 tx=0x7fcee000eca0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 shutdown_connections 2024-04-25T10:21:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcecc066f60 0x7fcecc069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcef015df00 0x7fcef00c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcef00d1ce0 0x7fcef00b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 --2- 172.21.15.45:0/298920740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcef00d0750 0x7fcef0154ff0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 >> 172.21.15.45:0/298920740 conn(0x7fcef006b290 msgr2=0x7fcef00b9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 shutdown_connections 2024-04-25T10:21:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.647+0000 7fcef6200640 1 -- 172.21.15.45:0/298920740 wait complete. 2024-04-25T10:21:01.668 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:21:01.811 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:01.811 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:01.812 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:01.826 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:01.827 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:21:01.827 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:01.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:21:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 Processor -- start 2024-04-25T10:21:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 -- start start 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd60159540 0x7fcd600ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcd600721f0 0x7fcd60070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd60159920 con 0x7fcd600aebd0 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6015c2f0 con 0x7fcd60159540 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.967+0000 7fcd65e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6015c5d0 con 0x7fcd600721f0 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd60159540 0x7fcd600ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd60159540 0x7fcd600ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41104/0 (socket says 172.21.15.45:41104) 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 -- 172.21.15.45:0/1244731137 learned_addr learned my addr 172.21.15.45:0/1244731137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 -- 172.21.15.45:0/1244731137 >> v2:172.21.15.117:3300/0 conn(0x7fcd600721f0 msgr2=0x7fcd60070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1244731137 >> v2:172.21.15.117:3300/0 conn(0x7fcd600721f0 0x7fcd60070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 -- 172.21.15.45:0/1244731137 >> v1:172.21.15.117:6789/0 conn(0x7fcd600aebd0 legacy=0x7fcd60071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 -- 172.21.15.45:0/1244731137 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6015c8b0 con 0x7fcd60159540 2024-04-25T10:21:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1244731137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd60159540 0x7fcd600ae690 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcd600cd200 tx=0x7fcd5001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cde96d83fae341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5e000640 1 -- 172.21.15.45:0/1244731137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ab5f0 con 0x7fcd60159540 2024-04-25T10:21:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5e000640 1 -- 172.21.15.45:0/1244731137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500b0070 con 0x7fcd60159540 2024-04-25T10:21:01.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.971+0000 7fcd5e000640 1 -- 172.21.15.45:0/1244731137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ab9b0 con 0x7fcd60159540 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- 172.21.15.45:0/1244731137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd60159540 msgr2=0x7fcd600ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 --2- 172.21.15.45:0/1244731137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd60159540 0x7fcd600ae690 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcd600cd200 tx=0x7fcd5001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- 172.21.15.45:0/1244731137 shutdown_connections 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 --2- 172.21.15.45:0/1244731137 >> v2:172.21.15.117:3300/0 conn(0x7fcd600721f0 0x7fcd60070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 --2- 172.21.15.45:0/1244731137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd60159540 0x7fcd600ae690 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- 172.21.15.45:0/1244731137 >> 172.21.15.45:0/1244731137 conn(0x7fcd600669a0 msgr2=0x7fcd60066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- 172.21.15.45:0/1244731137 shutdown_connections 2024-04-25T10:21:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- 172.21.15.45:0/1244731137 wait complete. 2024-04-25T10:21:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 Processor -- start 2024-04-25T10:21:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- start start 2024-04-25T10:21:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 0x7fcd6013ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd600aebd0 0x7fcd600cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd60159540 0x7fcd600cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd60071220 con 0x7fcd600aebd0 2024-04-25T10:21:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600710a0 con 0x7fcd60159540 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd65e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600713a0 con 0x7fcd600721f0 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 0x7fcd6013ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 0x7fcd6013ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37532/0 (socket says 172.21.15.45:37532) 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd56000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd600aebd0 0x7fcd600cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd5ea00640 1 -- 172.21.15.45:0/827840341 learned_addr learned my addr 172.21.15.45:0/827840341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.975+0000 7fcd5f400640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd60159540 0x7fcd600cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd5ea00640 1 -- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd60159540 msgr2=0x7fcd600cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd60159540 0x7fcd600cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd5ea00640 1 -- 172.21.15.45:0/827840341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd600aebd0 msgr2=0x7fcd600cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd600aebd0 0x7fcd600cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd5ea00640 1 -- 172.21.15.45:0/827840341 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd60151010 con 0x7fcd600721f0 2024-04-25T10:21:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd56000640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd600aebd0 0x7fcd600cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 0x7fcd6013ef30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd500ad040 tx=0x7fcd5001ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd57e00640 1 -- 172.21.15.45:0/827840341 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500be5f0 con 0x7fcd600721f0 2024-04-25T10:21:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd57e00640 1 -- 172.21.15.45:0/827840341 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500be7b0 con 0x7fcd600721f0 2024-04-25T10:21:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd601512f0 con 0x7fcd600721f0 2024-04-25T10:21:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd57e00640 1 -- 172.21.15.45:0/827840341 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500beba0 con 0x7fcd600721f0 2024-04-25T10:21:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd601518b0 con 0x7fcd600721f0 2024-04-25T10:21:01.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.979+0000 7fcd57e00640 1 -- 172.21.15.45:0/827840341 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd500b43f0 con 0x7fcd600721f0 2024-04-25T10:21:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd57e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd3c066fb0 0x7fcd3c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd57e00640 1 -- 172.21.15.45:0/827840341 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd5011f3e0 con 0x7fcd600721f0 2024-04-25T10:21:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd56000640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd3c066fb0 0x7fcd3c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 Processor -- start 2024-04-25T10:21:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 -- start start 2024-04-25T10:21:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd56000640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd3c066fb0 0x7fcd3c069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd6014d7b0 tx=0x7fcd48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e000 0x556621d8e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e920 0x556621d43ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d447f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621b91f40 con 0x556621d44410 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621b19fc0 con 0x556621d8e920 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556621b8dfe0 con 0x556621d8e000 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41124/0 (socket says 172.21.15.45:41124) 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 -- 172.21.15.45:0/1965828010 learned_addr learned my addr 172.21.15.45:0/1965828010 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd56000640 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e920 0x556621d43ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e000 0x556621d8e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 -- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e000 msgr2=0x556621d8e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e000 0x556621d8e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 -- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e920 msgr2=0x556621d43ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e920 0x556621d43ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 -- 172.21.15.45:0/1965828010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556621bd9a50 con 0x556621d44410 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d447f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800a430 tx=0x7fcd58009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06192300a0627b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd4fe00640 1 -- 172.21.15.45:0/1965828010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58013790 con 0x556621d44410 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd4fe00640 1 -- 172.21.15.45:0/1965828010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58019070 con 0x556621d44410 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd4fe00640 1 -- 172.21.15.45:0/1965828010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58013b70 con 0x556621d44410 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 -- 172.21.15.45:0/1965828010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 msgr2=0x556621d447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.983+0000 7fcd69292000 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d447f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800a430 tx=0x7fcd58009650 comp rx=0 tx=0).stop 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/1965828010 shutdown_connections 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d447f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e920 0x556621d43ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- 172.21.15.45:0/1965828010 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e000 0x556621d8e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/1965828010 >> 172.21.15.45:0/1965828010 conn(0x556621d403c0 msgr2=0x556621d427b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:01.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/1965828010 shutdown_connections 2024-04-25T10:21:01.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/1965828010 wait complete. 2024-04-25T10:21:01.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- start start 2024-04-25T10:21:01.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 Processor -- start 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d4fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621d50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e920 0x556621d50880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621b96570 con 0x556621d44410 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621b4c5f0 con 0x556621d8e000 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556621b90f10 con 0x556621d8e920 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 10 client.0 _ll_register_callbacks cb 0x556621c467a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d4fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d4fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41126/0 (socket says 172.21.15.45:41126) 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 -- 172.21.15.45:0/3455848877 learned_addr learned my addr 172.21.15.45:0/3455848877 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd56000640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621d50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5f400640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e920 0x556621d50880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 -- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e920 msgr2=0x556621d50880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e920 0x556621d50880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 -- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 msgr2=0x556621d50340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621d50340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 -- 172.21.15.45:0/3455848877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556621b1a210 con 0x556621d44410 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d4fe00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcd5001c9c0 tx=0x7fcd500ef8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34d88fc2185e5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd4cc00640 1 -- 172.21.15.45:0/3455848877 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ec500 con 0x556621d44410 2024-04-25T10:21:01.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd4cc00640 1 -- 172.21.15.45:0/3455848877 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd500ef2e0 con 0x556621d44410 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd58019070 con 0x556621d44410 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd4cc00640 1 -- 172.21.15.45:0/3455848877 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd500ec6a0 con 0x556621d44410 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 10 client.5820 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x556621c16400 con 0x556621d44410 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd4cc00640 1 -- 172.21.15.45:0/3455848877 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58019070 con 0x556621d44410 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd4cc00640 1 -- 172.21.15.45:0/3455848877 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd500ebbe0 con 0x556621d44410 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 10 client.5820 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 10 client.5820 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 20 client.5820 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621d51290 0x556621da02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 4 client.5820 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.987+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556621b2dc90 con 0x556621d51290 2024-04-25T10:21:01.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd56000640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621d51290 0x556621da02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd56000640 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621d51290 0x556621da02c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d023046416cd79 server_cookie=7d56638eefecbd25 in_seq=0 out_seq=0 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd4cc00640 10 client.5820 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd4cc00640 1 -- 172.21.15.45:0/3455848877 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556621b2dc90 con 0x556621d51290 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd4cc00640 10 client.5820 handle_command_reply: tid=0 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 client.5820 shutdown 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 msgr2=0x556621d4fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d4fe00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcd5001c9c0 tx=0x7fcd500ef8b0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 shutdown_connections 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621d51290 0x556621da02c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d8e920 0x556621d50880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621d50340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd4cc00640 0 client.5820 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 --2- 172.21.15.45:0/3455848877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d44410 0x556621d4fe00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:01.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 >> 172.21.15.45:0/3455848877 conn(0x556621d403c0 msgr2=0x556621d43040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 shutdown_connections 2024-04-25T10:21:02.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 1 -- 172.21.15.45:0/3455848877 wait complete. 2024-04-25T10:21:02.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:01.991+0000 7fcd69292000 20 client.5820 trim_cache size 0 max 16384 2024-04-25T10:21:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 Processor -- start 2024-04-25T10:21:02.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- start start 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621deb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621deb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd60202ba0 con 0x556621d8e920 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6014eeb0 con 0x556621d8e000 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6015a6b0 con 0x556621d44410 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621deb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621deb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41130/0 (socket says 172.21.15.45:41130) 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 -- 172.21.15.45:0/1876238215 learned_addr learned my addr 172.21.15.45:0/1876238215 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:02.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd56000640 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621deb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 -- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 msgr2=0x556621deab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 -- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 msgr2=0x556621deb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621deb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 -- 172.21.15.45:0/1876238215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556621becc10 con 0x556621d8e920 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd56000640 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621deb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:02.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621deb610 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcd58016040 tx=0x7fcd58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5f0d58d751d943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd4f400640 1 -- 172.21.15.45:0/1876238215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58013810 con 0x556621d8e920 2024-04-25T10:21:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd4f400640 1 -- 172.21.15.45:0/1876238215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58019070 con 0x556621d8e920 2024-04-25T10:21:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd4f400640 1 -- 172.21.15.45:0/1876238215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58013bf0 con 0x556621d8e920 2024-04-25T10:21:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- 172.21.15.45:0/1876238215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 msgr2=0x556621deb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621deb610 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcd58016040 tx=0x7fcd58013000 comp rx=0 tx=0).stop 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- 172.21.15.45:0/1876238215 shutdown_connections 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621deb610 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621deb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 --2- 172.21.15.45:0/1876238215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- 172.21.15.45:0/1876238215 >> 172.21.15.45:0/1876238215 conn(0x556621d403c0 msgr2=0x556621d4f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- 172.21.15.45:0/1876238215 shutdown_connections 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.007+0000 7fcd69292000 1 -- 172.21.15.45:0/1876238215 wait complete. 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- start start 2024-04-25T10:21:02.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 Processor -- start 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621def020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621da9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556621b8efb0 con 0x556621d8e920 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6014e8b0 con 0x556621d8e000 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556621b96570 con 0x556621d44410 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 10 client.0 _ll_register_callbacks cb 0x556621be42c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621da9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621da9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41132/0 (socket says 172.21.15.45:41132) 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 -- 172.21.15.45:0/1391677818 learned_addr learned my addr 172.21.15.45:0/1391677818 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd56000640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621def020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 -- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 msgr2=0x556621deeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 -- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 msgr2=0x556621def020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:02.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621def020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 -- 172.21.15.45:0/1391677818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556621b26bb0 con 0x556621d8e920 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd56000640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621def020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5f400640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621da9d40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd58015000 tx=0x7fcd58011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64191578c53e8d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd4cc00640 1 -- 172.21.15.45:0/1391677818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58013e50 con 0x556621d8e920 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd4cc00640 1 -- 172.21.15.45:0/1391677818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd58027430 con 0x556621d8e920 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd58019070 con 0x556621d8e920 2024-04-25T10:21:02.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd4cc00640 1 -- 172.21.15.45:0/1391677818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd58027650 con 0x556621d8e920 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 10 client.5826 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x556621b39030 con 0x556621d8e920 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd4cc00640 1 -- 172.21.15.45:0/1391677818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58019070 con 0x556621d8e920 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd4cc00640 1 -- 172.21.15.45:0/1391677818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd580277f0 con 0x556621d8e920 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 10 client.5826 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 10 client.5826 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 20 client.5826 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621daa700 0x556621daab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 4 client.5826 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x556621b9f480 con 0x556621daa700 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.011+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621daa700 0x556621daab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd5ea00640 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621daa700 0x556621daab10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293988005b3a3f3b server_cookie=deaba604b3960157 in_seq=0 out_seq=0 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd4cc00640 10 client.5826 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd4cc00640 1 -- 172.21.15.45:0/1391677818 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556621b9f480 con 0x556621daa700 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd4cc00640 10 client.5826 handle_command_reply: tid=0 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 client.5826 shutdown 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 msgr2=0x556621da9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621da9d40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcd58015000 tx=0x7fcd58011000 comp rx=0 tx=0).stop 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 shutdown_connections 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556621daa700 0x556621daab10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556621d8e920 0x556621da9d40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556621d8e000 0x556621def020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 --2- 172.21.15.45:0/1391677818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556621d44410 0x556621deeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd4cc00640 0 client.5826 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 >> 172.21.15.45:0/1391677818 conn(0x556621d403c0 msgr2=0x556621d4fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 shutdown_connections 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 1 -- 172.21.15.45:0/1391677818 wait complete. 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd69292000 20 client.5826 trim_cache size 0 max 16384 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:02.023 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "2cfdd3c5-6495-4e75-941b-284f7adeeade", 2024-04-25T10:21:02.024 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:02.024 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd3c066fb0 msgr2=0x7fcd3c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd3c066fb0 0x7fcd3c069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd6014d7b0 tx=0x7fcd48009040 comp rx=0 tx=0).stop 2024-04-25T10:21:02.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 msgr2=0x7fcd6013ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 0x7fcd6013ef30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcd500ad040 tx=0x7fcd5001ad20 comp rx=0 tx=0).stop 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 shutdown_connections 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd3c066fb0 0x7fcd3c069470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd60159540 0x7fcd600cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd600aebd0 0x7fcd600cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 --2- 172.21.15.45:0/827840341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd600721f0 0x7fcd6013ef30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 >> 172.21.15.45:0/827840341 conn(0x7fcd600669a0 msgr2=0x7fcd600ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 shutdown_connections 2024-04-25T10:21:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.015+0000 7fcd65e00640 1 -- 172.21.15.45:0/827840341 wait complete. 2024-04-25T10:21:02.038 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '2cfdd3c5-6495-4e75-941b-284f7adeeade', 'mode': 'asynchronous'}' 2024-04-25T10:21:02.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:02.038 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T10:21:02.050 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T10:21:02.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.231+0000 7f6178400640 1 Processor -- start 2024-04-25T10:21:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.231+0000 7f6178400640 1 -- start start 2024-04-25T10:21:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6178400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f617015df60 0x7f61700cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6178400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700ccaa0 con 0x7f617006aea0 2024-04-25T10:21:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6178400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700ccd80 con 0x7f61700d0180 2024-04-25T10:21:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6178400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617015e340 con 0x7f617015df60 2024-04-25T10:21:02.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6176400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f617006aea0 0x7f617006b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42942/0 (socket says 172.21.15.45:42942) 2024-04-25T10:21:02.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6176400640 1 -- 172.21.15.45:0/3850325408 learned_addr learned my addr 172.21.15.45:0/3850325408 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6175a00640 1 --2- 172.21.15.45:0/3850325408 >> v2:172.21.15.117:3300/0 conn(0x7f617015df60 0x7f61700cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794693344 0 0) 0x7f61700ccaa0 con 0x7f617006aea0 2024-04-25T10:21:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6158003530 con 0x7f617006aea0 2024-04-25T10:21:02.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021684696 0 0) 0x7f61700ccd80 con 0x7f61700d0180 2024-04-25T10:21:02.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61700ccaa0 con 0x7f61700d0180 2024-04-25T10:21:02.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6175a00640 1 -- 172.21.15.45:0/3850325408 >> v1:172.21.15.117:6789/0 conn(0x7f61700d0180 legacy=0x7f61700d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6175a00640 1 -- 172.21.15.45:0/3850325408 >> v1:172.21.15.45:6789/0 conn(0x7f617006aea0 legacy=0x7f617006b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.235+0000 7f6175a00640 1 -- 172.21.15.45:0/3850325408 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700cd060 con 0x7f617015df60 2024-04-25T10:21:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6175a00640 1 --2- 172.21.15.45:0/3850325408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617015df60 0x7f61700cfc40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f61640097c0 tx=0x7f616401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ae6505c01e626d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640b0070 con 0x7f617015df60 2024-04-25T10:21:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617006c1d0 con 0x7f617015df60 2024-04-25T10:21:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f616fe00640 1 -- 172.21.15.45:0/3850325408 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640ab820 con 0x7f617015df60 2024-04-25T10:21:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 -- 172.21.15.45:0/3850325408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617015df60 msgr2=0x7f61700cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 --2- 172.21.15.45:0/3850325408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617015df60 0x7f61700cfc40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f61640097c0 tx=0x7f616401c410 comp rx=0 tx=0).stop 2024-04-25T10:21:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 -- 172.21.15.45:0/3850325408 shutdown_connections 2024-04-25T10:21:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 --2- 172.21.15.45:0/3850325408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617015df60 0x7f61700cfc40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 -- 172.21.15.45:0/3850325408 >> 172.21.15.45:0/3850325408 conn(0x7f61700d1910 msgr2=0x7f61700d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 -- 172.21.15.45:0/3850325408 shutdown_connections 2024-04-25T10:21:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 -- 172.21.15.45:0/3850325408 wait complete. 2024-04-25T10:21:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 Processor -- start 2024-04-25T10:21:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.239+0000 7f6178400640 1 -- start start 2024-04-25T10:21:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 0x7f6170151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61700d0180 0x7f61700c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f617015df60 0x7f61700c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700d0ff0 con 0x7f617006aea0 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700d0e70 con 0x7f61700d0180 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617016ba30 con 0x7f617015df60 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 0x7f6170151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 0x7f6170151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41138/0 (socket says 172.21.15.45:41138) 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175000640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61700d0180 0x7f61700c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 -- 172.21.15.45:0/2434136070 learned_addr learned my addr 172.21.15.45:0/2434136070 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6176400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f617015df60 0x7f61700c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 -- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f617015df60 msgr2=0x7f61700c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f617015df60 0x7f61700c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 -- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61700d0180 msgr2=0x7f61700c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61700d0180 0x7f61700c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 -- 172.21.15.45:0/2434136070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700cba40 con 0x7f617006aea0 2024-04-25T10:21:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6175a00640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 0x7f6170151330 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6164019020 tx=0x7f61640191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640b8020 con 0x7f617006aea0 2024-04-25T10:21:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61640abd40 con 0x7f617006aea0 2024-04-25T10:21:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700cb760 con 0x7f617006aea0 2024-04-25T10:21:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640b0040 con 0x7f617006aea0 2024-04-25T10:21:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61700c69b0 con 0x7f617006aea0 2024-04-25T10:21:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.243+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617004ef30 con 0x7f617006aea0 2024-04-25T10:21:02.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.247+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61640bb020 con 0x7f617006aea0 2024-04-25T10:21:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.247+0000 7f616d600640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6150066f60 0x7f6150069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.247+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f616411ed30 con 0x7f617006aea0 2024-04-25T10:21:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.247+0000 7f6175000640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6150066f60 0x7f6150069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.247+0000 7f6175000640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6150066f60 0x7f6150069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f617006fe90 tx=0x7f615c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.251+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61640c2030 con 0x7f617006aea0 2024-04-25T10:21:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61700c91d0 con 0x7f617006aea0 2024-04-25T10:21:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f616d600640 1 -- 172.21.15.45:0/2434136070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6164016090 con 0x7f617006aea0 2024-04-25T10:21:02.451 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:02.451 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6150066f60 msgr2=0x7f6150069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6150066f60 0x7f6150069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f617006fe90 tx=0x7f615c009040 comp rx=0 tx=0).stop 2024-04-25T10:21:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 msgr2=0x7f6170151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 0x7f6170151330 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6164019020 tx=0x7f61640191b0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 shutdown_connections 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6150066f60 0x7f6150069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f617015df60 0x7f61700c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61700d0180 0x7f61700c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 --2- 172.21.15.45:0/2434136070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617006aea0 0x7f6170151330 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 >> 172.21.15.45:0/2434136070 conn(0x7f61700d1910 msgr2=0x7f61700d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 shutdown_connections 2024-04-25T10:21:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.447+0000 7f6178400640 1 -- 172.21.15.45:0/2434136070 wait complete. 2024-04-25T10:21:02.471 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T10:21:02.613 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:02.613 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:02.613 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:02.628 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:02.628 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:02.628 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:02.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:02.637 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:21:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:21:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:21:02.711 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178297 s, 58.8 MB/s 2024-04-25T10:21:02.712 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:21:02.712 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:02.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cfe00640 1 Processor -- start 2024-04-25T10:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cfe00640 1 -- start start 2024-04-25T10:21:02.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cfe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00b5780 con 0x7f61d00b5010 2024-04-25T10:21:02.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cfe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00b5a60 con 0x7f61d006b2c0 2024-04-25T10:21:02.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00b5d40 con 0x7f61d015db40 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61c7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f61d00b5010 0x7f61d015df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42958/0 (socket says 172.21.15.45:42958) 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61c7e00640 1 -- 172.21.15.45:0/3434440253 learned_addr learned my addr 172.21.15.45:0/3434440253 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.899+0000 7f61cea00640 1 --2- 172.21.15.45:0/3434440253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089155350 0 0) 0x7f61d00b5780 con 0x7f61d00b5010 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b4003530 con 0x7f61d00b5010 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164048073 0 0) 0x7f61d00b5a60 con 0x7f61d006b2c0 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d00b5780 con 0x7f61d006b2c0 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61cea00640 1 -- 172.21.15.45:0/3434440253 >> v1:172.21.15.117:6789/0 conn(0x7f61d006b2c0 legacy=0x7f61d00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61cea00640 1 -- 172.21.15.45:0/3434440253 >> v1:172.21.15.45:6789/0 conn(0x7f61d00b5010 legacy=0x7f61d015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61cea00640 1 -- 172.21.15.45:0/3434440253 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00b6020 con 0x7f61d015db40 2024-04-25T10:21:02.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61cea00640 1 --2- 172.21.15.45:0/3434440253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d006ac90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61c00097f0 tx=0x7f61c001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5fc0239cfffbec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b5070 con 0x7f61d015db40 2024-04-25T10:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00a89d0 con 0x7f61d015db40 2024-04-25T10:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61ce000640 1 -- 172.21.15.45:0/3434440253 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a8da0 con 0x7f61d015db40 2024-04-25T10:21:02.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61cfe00640 1 -- 172.21.15.45:0/3434440253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 msgr2=0x7f61d006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.903+0000 7f61cfe00640 1 --2- 172.21.15.45:0/3434440253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d006ac90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61c00097f0 tx=0x7f61c001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- 172.21.15.45:0/3434440253 shutdown_connections 2024-04-25T10:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 --2- 172.21.15.45:0/3434440253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d006ac90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- 172.21.15.45:0/3434440253 >> 172.21.15.45:0/3434440253 conn(0x7f61d00d17b0 msgr2=0x7f61d00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- 172.21.15.45:0/3434440253 shutdown_connections 2024-04-25T10:21:02.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- 172.21.15.45:0/3434440253 wait complete. 2024-04-25T10:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 Processor -- start 2024-04-25T10:21:02.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- start start 2024-04-25T10:21:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 0x7f61d009e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d00b5010 0x7f61d00cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d01527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d006b750 con 0x7f61d006b2c0 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00b5450 con 0x7f61d00b5010 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d006b8d0 con 0x7f61d015db40 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 0x7f61d009e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 0x7f61d009e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41152/0 (socket says 172.21.15.45:41152) 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 -- 172.21.15.45:0/1274696055 learned_addr learned my addr 172.21.15.45:0/1274696055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61c7e00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d00b5010 0x7f61d00cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cf400640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d01527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 -- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 msgr2=0x7f61d01527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d01527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 -- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d00b5010 msgr2=0x7f61d00cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:02.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d00b5010 0x7f61d00cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 -- 172.21.15.45:0/1274696055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d009d2b0 con 0x7f61d006b2c0 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61cea00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 0x7f61d009e350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f61c001b8e0 tx=0x7f61c001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00c1070 con 0x7f61d006b2c0 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.907+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00b0720 con 0x7f61d006b2c0 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d009d590 con 0x7f61d006b2c0 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b5040 con 0x7f61d006b2c0 2024-04-25T10:21:02.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d009db50 con 0x7f61d006b2c0 2024-04-25T10:21:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d0075120 con 0x7f61d006b2c0 2024-04-25T10:21:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61c00c4070 con 0x7f61d006b2c0 2024-04-25T10:21:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61c6a00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61ac066f60 0x7f61ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61c0121d10 con 0x7f61d006b2c0 2024-04-25T10:21:02.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61c7e00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61ac066f60 0x7f61ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:02.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.911+0000 7f61c7e00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61ac066f60 0x7f61ac069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61d009f5f0 tx=0x7f61b8007d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:02.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:02.915+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61c0018830 con 0x7f61d006b2c0 2024-04-25T10:21:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.135+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61d00b2b30 con 0x7f61d006b2c0 2024-04-25T10:21:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61c6a00640 1 -- 172.21.15.45:0/1274696055 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f61c00f2630 con 0x7f61d006b2c0 2024-04-25T10:21:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:03.141 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61ac066f60 msgr2=0x7f61ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61ac066f60 0x7f61ac069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61d009f5f0 tx=0x7f61b8007d80 comp rx=0 tx=0).stop 2024-04-25T10:21:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 msgr2=0x7f61d009e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 0x7f61d009e350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f61c001b8e0 tx=0x7f61c001ba70 comp rx=0 tx=0).stop 2024-04-25T10:21:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 shutdown_connections 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61ac066f60 0x7f61ac069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61d015db40 0x7f61d01527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61d00b5010 0x7f61d00cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 --2- 172.21.15.45:0/1274696055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61d006b2c0 0x7f61d009e350 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 >> 172.21.15.45:0/1274696055 conn(0x7f61d00d17b0 msgr2=0x7f61d006f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 shutdown_connections 2024-04-25T10:21:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.139+0000 7f61cfe00640 1 -- 172.21.15.45:0/1274696055 wait complete. 2024-04-25T10:21:03.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:21:03.303 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:03.304 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:03.304 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:03.318 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:03.319 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:21:03.319 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:21:03.319 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:21:03.319 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:21:03.319 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T10:21:03.319 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:21:03.385 INFO:teuthology.orchestra.run.smithi045.stdout:1099511630187 2024-04-25T10:21:03.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:21:03.394 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:03.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:21:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 Processor -- start 2024-04-25T10:21:03.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 -- start start 2024-04-25T10:21:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e640aeda0 0x7f9e64071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e64072020 0x7f9e64070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e64159920 con 0x7f9e64159540 2024-04-25T10:21:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6415c2f0 con 0x7f9e64072020 2024-04-25T10:21:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6c600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6415c5d0 con 0x7f9e640aeda0 2024-04-25T10:21:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6a600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e64072020 0x7f9e64070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6a600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e64072020 0x7f9e64070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41168/0 (socket says 172.21.15.45:41168) 2024-04-25T10:21:03.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.579+0000 7f9e6a600640 1 -- 172.21.15.45:0/2299370758 learned_addr learned my addr 172.21.15.45:0/2299370758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e69200640 1 --2- 172.21.15.45:0/2299370758 >> v2:172.21.15.117:3300/0 conn(0x7f9e640aeda0 0x7f9e64071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6a600640 1 -- 172.21.15.45:0/2299370758 >> v2:172.21.15.117:3300/0 conn(0x7f9e640aeda0 msgr2=0x7f9e64071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6a600640 1 --2- 172.21.15.45:0/2299370758 >> v2:172.21.15.117:3300/0 conn(0x7f9e640aeda0 0x7f9e64071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6a600640 1 -- 172.21.15.45:0/2299370758 >> v1:172.21.15.117:6789/0 conn(0x7f9e64159540 legacy=0x7f9e640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6a600640 1 -- 172.21.15.45:0/2299370758 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6415c8b0 con 0x7f9e64072020 2024-04-25T10:21:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6a600640 1 --2- 172.21.15.45:0/2299370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64072020 0x7f9e64070350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9e640cd510 tx=0x7f9e5c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8baf191afa7932a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e63e00640 1 -- 172.21.15.45:0/2299370758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0a5860 con 0x7f9e64072020 2024-04-25T10:21:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e63e00640 1 -- 172.21.15.45:0/2299370758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0a5a00 con 0x7f9e64072020 2024-04-25T10:21:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e63e00640 1 -- 172.21.15.45:0/2299370758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0a5dd0 con 0x7f9e64072020 2024-04-25T10:21:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6c600640 1 -- 172.21.15.45:0/2299370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64072020 msgr2=0x7f9e64070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2299370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64072020 0x7f9e64070350 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9e640cd510 tx=0x7f9e5c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6c600640 1 -- 172.21.15.45:0/2299370758 shutdown_connections 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.583+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2299370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64072020 0x7f9e64070350 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2299370758 >> v2:172.21.15.117:3300/0 conn(0x7f9e640aeda0 0x7f9e64071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- 172.21.15.45:0/2299370758 >> 172.21.15.45:0/2299370758 conn(0x7f9e640669a0 msgr2=0x7f9e64066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- 172.21.15.45:0/2299370758 shutdown_connections 2024-04-25T10:21:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- 172.21.15.45:0/2299370758 wait complete. 2024-04-25T10:21:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 Processor -- start 2024-04-25T10:21:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- start start 2024-04-25T10:21:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e64072020 0x7f9e64159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e640aeda0 0x7f9e64153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 0x7f9e6414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640710a0 con 0x7f9e64159540 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6406c0d0 con 0x7f9e640aeda0 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64071220 con 0x7f9e64072020 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 0x7f9e6414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 0x7f9e6414af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41182/0 (socket says 172.21.15.45:41182) 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 -- 172.21.15.45:0/2528902117 learned_addr learned my addr 172.21.15.45:0/2528902117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e69200640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e640aeda0 0x7f9e64153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e69c00640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e64072020 0x7f9e64159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 -- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e64072020 msgr2=0x7f9e64159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e64072020 0x7f9e64159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 -- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e640aeda0 msgr2=0x7f9e64153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e640aeda0 0x7f9e64153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 -- 172.21.15.45:0/2528902117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e641527e0 con 0x7f9e64159540 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.587+0000 7f9e6a600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 0x7f9e6414af70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0195d0 tx=0x7f9e5c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e61600640 1 -- 172.21.15.45:0/2528902117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0bc020 con 0x7f9e64159540 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e61600640 1 -- 172.21.15.45:0/2528902117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0b86b0 con 0x7f9e64159540 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e61600640 1 -- 172.21.15.45:0/2528902117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0b8aa0 con 0x7f9e64159540 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e64152a80 con 0x7f9e64159540 2024-04-25T10:21:03.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e64153040 con 0x7f9e64159540 2024-04-25T10:21:03.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e61600640 1 -- 172.21.15.45:0/2528902117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e5c0bf020 con 0x7f9e64159540 2024-04-25T10:21:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e61600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.591+0000 7f9e61600640 1 -- 172.21.15.45:0/2528902117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e5c120280 con 0x7f9e64159540 2024-04-25T10:21:03.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69c00640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 Processor -- start 2024-04-25T10:21:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- start start 2024-04-25T10:21:03.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69c00640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e58001650 tx=0x7f9e58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6bfc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6bfca10 0x561cb6bfcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6a45f20 con 0x561cb6c46cf0 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb69ce000 con 0x561cb6bfca10 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cb6a41fc0 con 0x561cb6c463d0 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6bfc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6bfc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41190/0 (socket says 172.21.15.45:41190) 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 -- 172.21.15.45:0/3414785661 learned_addr learned my addr 172.21.15.45:0/3414785661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69c00640 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6a600640 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6bfca10 0x561cb6bfcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 -- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 msgr2=0x561cb6c467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 -- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6bfca10 msgr2=0x561cb6bfcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6bfca10 0x561cb6bfcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 -- 172.21.15.45:0/3414785661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb6a21ba0 con 0x561cb6c46cf0 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e69200640 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6bfc4d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007810 tx=0x7f9e50007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcbf7352876969a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e56000640 1 -- 172.21.15.45:0/3414785661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50018020 con 0x561cb6c46cf0 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e56000640 1 -- 172.21.15.45:0/3414785661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50004240 con 0x561cb6c46cf0 2024-04-25T10:21:03.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e56000640 1 -- 172.21.15.45:0/3414785661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50010830 con 0x561cb6c46cf0 2024-04-25T10:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3414785661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 msgr2=0x561cb6bfc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6bfc4d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9e50007810 tx=0x7f9e50007cd0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3414785661 shutdown_connections 2024-04-25T10:21:03.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6bfca10 0x561cb6bfcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6bfc4d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3414785661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3414785661 >> 172.21.15.45:0/3414785661 conn(0x561cb6bfe760 msgr2=0x561cb6bf69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3414785661 shutdown_connections 2024-04-25T10:21:03.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3414785661 wait complete. 2024-04-25T10:21:03.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.595+0000 7f9e6fb76000 1 -- start start 2024-04-25T10:21:03.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 Processor -- start 2024-04-25T10:21:03.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6c4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c4dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c46cf0 0x561cb6c07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6a005d0 con 0x561cb6bfca10 2024-04-25T10:21:03.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6a45230 con 0x561cb6c46cf0 2024-04-25T10:21:03.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cb6a47f70 con 0x561cb6c463d0 2024-04-25T10:21:03.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 10 client.0 _ll_register_callbacks cb 0x561cb6be3a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:03.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6c4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6c4d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41196/0 (socket says 172.21.15.45:41196) 2024-04-25T10:21:03.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 -- 172.21.15.45:0/3799016207 learned_addr learned my addr 172.21.15.45:0/3799016207 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69200640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c4dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6a600640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c46cf0 0x561cb6c07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 -- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 msgr2=0x561cb6c4dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c4dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 -- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c46cf0 msgr2=0x561cb6c07250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c46cf0 0x561cb6c07250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 -- 172.21.15.45:0/3799016207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cb6a8c5a0 con 0x561cb6bfca10 2024-04-25T10:21:03.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69c00640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6c4d730 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9e58003950 tx=0x7f9e58000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7478535140858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 1 -- 172.21.15.45:0/3799016207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58028060 con 0x561cb6bfca10 2024-04-25T10:21:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 1 -- 172.21.15.45:0/3799016207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e58010040 con 0x561cb6bfca10 2024-04-25T10:21:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e50004240 con 0x561cb6bfca10 2024-04-25T10:21:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 1 -- 172.21.15.45:0/3799016207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5800ecc0 con 0x561cb6bfca10 2024-04-25T10:21:03.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 10 client.5850 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x561cb6a1d2b0 con 0x561cb6bfca10 2024-04-25T10:21:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 1 -- 172.21.15.45:0/3799016207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50004240 con 0x561cb6bfca10 2024-04-25T10:21:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 1 -- 172.21.15.45:0/3799016207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e58013420 con 0x561cb6bfca10 2024-04-25T10:21:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 10 client.5850 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 10 client.5850 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:03.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 20 client.5850 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c07f90 0x561cb6c083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 4 client.5850 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cb69e1c70 con 0x561cb6c07f90 2024-04-25T10:21:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69200640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c07f90 0x561cb6c083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e69200640 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c07f90 0x561cb6c083a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763c5d6911417ce0 server_cookie=6c74b9171475fd24 in_seq=0 out_seq=0 2024-04-25T10:21:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.599+0000 7f9e4aa00640 10 client.5850 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e4aa00640 1 -- 172.21.15.45:0/3799016207 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cb69e1c70 con 0x561cb6c07f90 2024-04-25T10:21:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e4aa00640 10 client.5850 handle_command_reply: tid=0 2024-04-25T10:21:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 client.5850 shutdown 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 msgr2=0x561cb6c4d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6c4d730 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9e58003950 tx=0x7f9e58000ee0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 shutdown_connections 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c07f90 0x561cb6c083a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c46cf0 0x561cb6c07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c463d0 0x561cb6c4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3799016207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6c4d730 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e4aa00640 0 client.5850 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 >> 172.21.15.45:0/3799016207 conn(0x561cb6bfe760 msgr2=0x561cb6bfdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 shutdown_connections 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3799016207 wait complete. 2024-04-25T10:21:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.603+0000 7f9e6fb76000 20 client.5850 trim_cache size 0 max 16384 2024-04-25T10:21:03.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 Processor -- start 2024-04-25T10:21:03.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- start start 2024-04-25T10:21:03.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6bfca10 0x561cb6c45da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c06910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6c06e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b3c50 con 0x561cb6c46cf0 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b2e90 con 0x561cb6c463d0 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6414fb60 con 0x561cb6bfca10 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6c06e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6c06e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41208/0 (socket says 172.21.15.45:41208) 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 -- 172.21.15.45:0/3967630832 learned_addr learned my addr 172.21.15.45:0/3967630832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e69c00640 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6bfca10 0x561cb6c45da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e69200640 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c06910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 -- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6bfca10 msgr2=0x561cb6c45da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6bfca10 0x561cb6c45da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 -- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 msgr2=0x561cb6c06910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c06910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 -- 172.21.15.45:0/3967630832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb6bfc840 con 0x561cb6c46cf0 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e69200640 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c06910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6a600640 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6c06e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0f0c30 tx=0x7f9e5c120040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f14b3bc941e500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e55600640 1 -- 172.21.15.45:0/3967630832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0f13a0 con 0x561cb6c46cf0 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e55600640 1 -- 172.21.15.45:0/3967630832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0f0ca0 con 0x561cb6c46cf0 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e55600640 1 -- 172.21.15.45:0/3967630832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0f4df0 con 0x561cb6c46cf0 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3967630832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 msgr2=0x561cb6c06e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6c06e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0f0c30 tx=0x7f9e5c120040 comp rx=0 tx=0).stop 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3967630832 shutdown_connections 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6c46cf0 0x561cb6c06e50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c06910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/3967630832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6bfca10 0x561cb6c45da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3967630832 >> 172.21.15.45:0/3967630832 conn(0x561cb6bfe760 msgr2=0x561cb6c06200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3967630832 shutdown_connections 2024-04-25T10:21:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.615+0000 7f9e6fb76000 1 -- 172.21.15.45:0/3967630832 wait complete. 2024-04-25T10:21:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- start start 2024-04-25T10:21:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 Processor -- start 2024-04-25T10:21:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6ca6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c46cf0 0x561cb6c49620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b28c0 con 0x561cb6bfca10 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb6a45230 con 0x561cb6c463d0 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cb69fcdc0 con 0x561cb6c46cf0 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 10 client.0 _ll_register_callbacks cb 0x561cb6bc4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6ca6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6ca6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41210/0 (socket says 172.21.15.45:41210) 2024-04-25T10:21:03.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 -- 172.21.15.45:0/4026683235 learned_addr learned my addr 172.21.15.45:0/4026683235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6a600640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c46cf0 0x561cb6c49620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69200640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 -- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c46cf0 msgr2=0x561cb6c49620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c46cf0 0x561cb6c49620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 -- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 msgr2=0x561cb6c490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 -- 172.21.15.45:0/4026683235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cb69ce250 con 0x561cb6bfca10 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6a600640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c46cf0 0x561cb6c49620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69200640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69c00640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6ca6200 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9e58008fd0 tx=0x7f9e58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddec0763ee3ca3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e4aa00640 1 -- 172.21.15.45:0/4026683235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800e900 con 0x561cb6bfca10 2024-04-25T10:21:03.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e4aa00640 1 -- 172.21.15.45:0/4026683235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e5802e670 con 0x561cb6bfca10 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e5c0f0ca0 con 0x561cb6bfca10 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e4aa00640 1 -- 172.21.15.45:0/4026683235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5800ee60 con 0x561cb6bfca10 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 10 client.5856 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x561cb69dab90 con 0x561cb6bfca10 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e4aa00640 1 -- 172.21.15.45:0/4026683235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c0f0ca0 con 0x561cb6bfca10 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e4aa00640 1 -- 172.21.15.45:0/4026683235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e5803e020 con 0x561cb6bfca10 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 10 client.5856 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 10 client.5856 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 20 client.5856 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c49b60 0x561cb6c49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 4 client.5856 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561cb6a53460 con 0x561cb6c49b60 2024-04-25T10:21:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.619+0000 7f9e69200640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c49b60 0x561cb6c49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e69200640 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c49b60 0x561cb6c49f70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91eda224c8d808d server_cookie=ce7bb4bffd43e3ca in_seq=0 out_seq=0 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e4aa00640 10 client.5856 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e4aa00640 1 -- 172.21.15.45:0/4026683235 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561cb6a53460 con 0x561cb6c49b60 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e4aa00640 10 client.5856 handle_command_reply: tid=0 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 client.5856 shutdown 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 msgr2=0x561cb6ca6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6ca6200 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9e58008fd0 tx=0x7f9e58013000 comp rx=0 tx=0).stop 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 shutdown_connections 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561cb6c49b60 0x561cb6c49f70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561cb6c46cf0 0x561cb6c49620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561cb6c463d0 0x561cb6c490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 --2- 172.21.15.45:0/4026683235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561cb6bfca10 0x561cb6ca6200 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e4aa00640 0 client.5856 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 >> 172.21.15.45:0/4026683235 conn(0x561cb6bfe760 msgr2=0x561cb6bf69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 shutdown_connections 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 1 -- 172.21.15.45:0/4026683235 wait complete. 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6fb76000 20 client.5856 trim_cache size 0 max 16384 2024-04-25T10:21:03.633 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 msgr2=0x7f9e44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e58001650 tx=0x7f9e58009040 comp rx=0 tx=0).stop 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 msgr2=0x7f9e6414af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 0x7f9e6414af70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0195d0 tx=0x7f9e5c019760 comp rx=0 tx=0).stop 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 shutdown_connections 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e64159540 0x7f9e6414af70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e640aeda0 0x7f9e64153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 --2- 172.21.15.45:0/2528902117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e64072020 0x7f9e64159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 >> 172.21.15.45:0/2528902117 conn(0x7f9e640669a0 msgr2=0x7f9e64066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 shutdown_connections 2024-04-25T10:21:03.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.623+0000 7f9e6c600640 1 -- 172.21.15.45:0/2528902117 wait complete. 2024-04-25T10:21:03.645 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096b.00000000 parent 2024-04-25T10:21:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 Processor -- start 2024-04-25T10:21:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 -- start start 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd113aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cb7a0 0x561dd11ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cbb80 0x561dd1122930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd1122510 con 0x561dd11cb7a0 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd109a9f0 con 0x561dd1139e40 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.687+0000 7f7d09a72080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dd1122690 con 0x561dd11cbb80 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d07600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cbb80 0x561dd1122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd113aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd113aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54504/0 (socket says 172.21.15.45:54504) 2024-04-25T10:21:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d07600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cbb80 0x561dd1122930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37652/0 (socket says 172.21.15.45:37652) 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 -- 172.21.15.45:0/2909346747 learned_addr learned my addr 172.21.15.45:0/2909346747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7cffe00640 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cb7a0 0x561dd11ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 -- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cbb80 msgr2=0x561dd1122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cbb80 0x561dd1122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 -- 172.21.15.45:0/2909346747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cb7a0 msgr2=0x561dd11ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cb7a0 0x561dd11ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 -- 172.21.15.45:0/2909346747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dd11ce8f0 con 0x561dd1139e40 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d06c00640 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd113aea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800b350 tx=0x7f7cf800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715163856775d192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d05800640 1 -- 172.21.15.45:0/2909346747 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800d5d0 con 0x561dd1139e40 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d05800640 1 -- 172.21.15.45:0/2909346747 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8008070 con 0x561dd1139e40 2024-04-25T10:21:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d05800640 1 -- 172.21.15.45:0/2909346747 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800d970 con 0x561dd1139e40 2024-04-25T10:21:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 msgr2=0x561dd113aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd113aea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800b350 tx=0x7f7cf800b810 comp rx=0 tx=0).stop 2024-04-25T10:21:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- 172.21.15.45:0/2909346747 shutdown_connections 2024-04-25T10:21:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cbb80 0x561dd1122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cb7a0 0x561dd11ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- 172.21.15.45:0/2909346747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd113aea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- 172.21.15.45:0/2909346747 >> 172.21.15.45:0/2909346747 conn(0x561dd111e760 msgr2=0x561dd10f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- 172.21.15.45:0/2909346747 shutdown_connections 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- 172.21.15.45:0/2909346747 wait complete. 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 Processor -- start 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- start start 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd11a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cb7a0 0x561dd1122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cbb80 0x561dd1122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd113b810 con 0x561dd11cbb80 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd113b570 con 0x561dd1139e40 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.691+0000 7f7d09a72080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dd113bab0 con 0x561dd11cb7a0 2024-04-25T10:21:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cbb80 0x561dd1122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cb7a0 0x561dd1122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cbb80 0x561dd1122e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41240/0 (socket says 172.21.15.45:41240) 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 -- 172.21.15.45:0/4170847627 learned_addr learned my addr 172.21.15.45:0/4170847627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d06c00640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd11a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 -- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cb7a0 msgr2=0x561dd1122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561dd11cb7a0 0x561dd1122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 -- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 msgr2=0x561dd11a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561dd1139e40 0x561dd11a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 -- 172.21.15.45:0/4170847627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dd11b1af0 con 0x561dd11cbb80 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d07600640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cbb80 0x561dd1122e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a560 tx=0x7f7d00017e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7cff400640 1 -- 172.21.15.45:0/4170847627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000ba020 con 0x561dd11cbb80 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7cff400640 1 -- 172.21.15.45:0/4170847627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d000abc20 con 0x561dd11cbb80 2024-04-25T10:21:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7cff400640 1 -- 172.21.15.45:0/4170847627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000b6450 con 0x561dd11cbb80 2024-04-25T10:21:03.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561dd11b1d90 con 0x561dd11cbb80 2024-04-25T10:21:03.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dd11b2030 con 0x561dd11cbb80 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7cff400640 1 -- 172.21.15.45:0/4170847627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d000bd020 con 0x561dd11cbb80 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.695+0000 7f7cff400640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce4065970 0x7f7ce4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7cff400640 1 -- 172.21.15.45:0/4170847627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d0011e8c0 con 0x561dd11cbb80 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x561dd12650d0 0x561dd12654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d06c00640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce4065970 0x7f7ce4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 --> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] -- osd_op(unknown.0.0:1 7.a 7:51f47cfd:::1000000096b.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50) v9 -- 0x561dd11a94b0 con 0x561dd12650d0 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7cffe00640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x561dd12650d0 0x561dd12654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d06c00640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce4065970 0x7f7ce4067e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8000c10 tx=0x7f7cf8008000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7cffe00640 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x561dd12650d0 0x561dd12654b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7cffe00640 1 -- 172.21.15.45:0/4170847627 <== osd.4 v2:172.21.15.117:6800/728189779 1 ==== osd_op_reply(1 1000000096b.00000000 [rmxattr] v50'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7cf0009980 con 0x561dd12650d0 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x561dd12650d0 msgr2=0x561dd12654b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x561dd12650d0 0x561dd12654b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce4065970 msgr2=0x7f7ce4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce4065970 0x7f7ce4067e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8000c10 tx=0x7f7cf8008000 comp rx=0 tx=0).stop 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cbb80 msgr2=0x561dd1122e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 --2- 172.21.15.45:0/4170847627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561dd11cbb80 0x561dd1122e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a560 tx=0x7f7d00017e40 comp rx=0 tx=0).stop 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d07600640 1 -- 172.21.15.45:0/4170847627 reap_dead start 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 shutdown_connections 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 >> 172.21.15.45:0/4170847627 conn(0x561dd111e760 msgr2=0x561dd113ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 shutdown_connections 2024-04-25T10:21:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.699+0000 7f7d09a72080 1 -- 172.21.15.45:0/4170847627 wait complete. 2024-04-25T10:21:03.708 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:21:03.708 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:03.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:21:03.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 Processor -- start 2024-04-25T10:21:03.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 -- start start 2024-04-25T10:21:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fc806bd10 0x7f0fc8161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0fc8161e50 0x7f0fc815d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc806ac90 con 0x7f0fc80cd9a0 2024-04-25T10:21:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc81622a0 con 0x7f0fc806bd10 2024-04-25T10:21:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcf200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc806b880 con 0x7f0fc8161e50 2024-04-25T10:21:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fbfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fc806bd10 0x7f0fc8161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fbfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fc806bd10 0x7f0fc8161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41256/0 (socket says 172.21.15.45:41256) 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fbfe00640 1 -- 172.21.15.45:0/1763015206 learned_addr learned my addr 172.21.15.45:0/1763015206 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fc7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0fc80cd9a0 0x7f0fc80d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42980/0 (socket says 172.21.15.45:42980) 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.891+0000 7f0fcd200640 1 --2- 172.21.15.45:0/1763015206 >> v2:172.21.15.117:3300/0 conn(0x7f0fc8161e50 0x7f0fc815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fbfe00640 1 -- 172.21.15.45:0/1763015206 >> v2:172.21.15.117:3300/0 conn(0x7f0fc8161e50 msgr2=0x7f0fc815d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/1763015206 >> v2:172.21.15.117:3300/0 conn(0x7f0fc8161e50 0x7f0fc815d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fbfe00640 1 -- 172.21.15.45:0/1763015206 >> v1:172.21.15.45:6789/0 conn(0x7f0fc80cd9a0 legacy=0x7f0fc80d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fbfe00640 1 -- 172.21.15.45:0/1763015206 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc806af50 con 0x7f0fc806bd10 2024-04-25T10:21:03.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/1763015206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8161910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0fc81629b0 tx=0x7f0fb001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b950fd1204a01b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fc7400640 1 -- 172.21.15.45:0/1763015206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00b5070 con 0x7f0fc806bd10 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fc7400640 1 -- 172.21.15.45:0/1763015206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a8a10 con 0x7f0fc806bd10 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fc7400640 1 -- 172.21.15.45:0/1763015206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00a8de0 con 0x7f0fc806bd10 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 -- 172.21.15.45:0/1763015206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 msgr2=0x7f0fc8161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1763015206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8161910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0fc81629b0 tx=0x7f0fb001ed10 comp rx=0 tx=0).stop 2024-04-25T10:21:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 -- 172.21.15.45:0/1763015206 shutdown_connections 2024-04-25T10:21:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1763015206 >> v2:172.21.15.117:3300/0 conn(0x7f0fc8161e50 0x7f0fc815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1763015206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8161910 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 -- 172.21.15.45:0/1763015206 >> 172.21.15.45:0/1763015206 conn(0x7f0fc80d19a0 msgr2=0x7f0fc80d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 -- 172.21.15.45:0/1763015206 shutdown_connections 2024-04-25T10:21:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.895+0000 7f0fcf200640 1 -- 172.21.15.45:0/1763015206 wait complete. 2024-04-25T10:21:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 Processor -- start 2024-04-25T10:21:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 -- start start 2024-04-25T10:21:03.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc80cd9a0 0x7f0fc80c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fc8161e50 0x7f0fc80ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80cdd80 con 0x7f0fc806bd10 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc806b510 con 0x7f0fc80cd9a0 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc8162660 con 0x7f0fc8161e50 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41270/0 (socket says 172.21.15.45:41270) 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 -- 172.21.15.45:0/1049247744 learned_addr learned my addr 172.21.15.45:0/1049247744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcd200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fc8161e50 0x7f0fc80ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc80cd9a0 0x7f0fc80c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 -- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fc8161e50 msgr2=0x7f0fc80ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fc8161e50 0x7f0fc80ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 -- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc80cd9a0 msgr2=0x7f0fc80c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc80cd9a0 0x7f0fc80c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 -- 172.21.15.45:0/1049247744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80c8c10 con 0x7f0fc806bd10 2024-04-25T10:21:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8154170 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800c420 tx=0x7f0fb800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc4c00640 1 -- 172.21.15.45:0/1049247744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80074e0 con 0x7f0fc806bd10 2024-04-25T10:21:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc4c00640 1 -- 172.21.15.45:0/1049247744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb8007680 con 0x7f0fc806bd10 2024-04-25T10:21:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fc4c00640 1 -- 172.21.15.45:0/1049247744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8004a30 con 0x7f0fc806bd10 2024-04-25T10:21:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.899+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc80c8ef0 con 0x7f0fc806bd10 2024-04-25T10:21:03.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc80c94b0 con 0x7f0fc806bd10 2024-04-25T10:21:03.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fc4c00640 1 -- 172.21.15.45:0/1049247744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fb801a020 con 0x7f0fc806bd10 2024-04-25T10:21:03.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fc4c00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fa4066f60 0x7f0fa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fc4c00640 1 -- 172.21.15.45:0/1049247744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fb8084730 con 0x7f0fc806bd10 2024-04-25T10:21:03.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fa4066f60 0x7f0fa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fd26a8000 1 Processor -- start 2024-04-25T10:21:03.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fd26a8000 1 -- start start 2024-04-25T10:21:03.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.903+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fa4066f60 0x7f0fa4069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00b2000 tx=0x7f0fb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c9180363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c918036d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c917e86ce0 con 0x55c918083b10 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c917e0f530 con 0x55c9180831f0 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c917e82d80 con 0x55c918036920 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c9180363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c9180363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41276/0 (socket says 172.21.15.45:41276) 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 -- 172.21.15.45:0/2698575635 learned_addr learned my addr 172.21.15.45:0/2698575635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fcd200640 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c918036d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 -- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 msgr2=0x55c918036d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c918036d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 -- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 msgr2=0x55c9180835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 -- 172.21.15.45:0/2698575635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c917e81cd0 con 0x55c918083b10 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c9180363e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009010 tx=0x7f0fb0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92fe0acc7d2e847 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbcc00640 1 -- 172.21.15.45:0/2698575635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00cb070 con 0x55c918083b10 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbcc00640 1 -- 172.21.15.45:0/2698575635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a8df0 con 0x55c918083b10 2024-04-25T10:21:03.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fbcc00640 1 -- 172.21.15.45:0/2698575635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00b5040 con 0x55c918083b10 2024-04-25T10:21:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- 172.21.15.45:0/2698575635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 msgr2=0x55c9180363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c9180363e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009010 tx=0x7f0fb0004740 comp rx=0 tx=0).stop 2024-04-25T10:21:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- 172.21.15.45:0/2698575635 shutdown_connections 2024-04-25T10:21:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c918036d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c9180363e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/2698575635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- 172.21.15.45:0/2698575635 >> 172.21.15.45:0/2698575635 conn(0x55c918063f90 msgr2=0x55c918066380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- 172.21.15.45:0/2698575635 shutdown_connections 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- 172.21.15.45:0/2698575635 wait complete. 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 -- start start 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.907+0000 7f0fd26a8000 1 Processor -- start 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 0x55c91808a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c91808aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c91803ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c917e41bc0 con 0x55c918036920 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c917e8b310 con 0x55c9180831f0 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c917e530b0 con 0x55c918083b10 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 10 client.0 _ll_register_callbacks cb 0x55c9180244a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 0x55c91808a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 0x55c91808a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41290/0 (socket says 172.21.15.45:41290) 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3523227693 learned_addr learned my addr 172.21.15.45:0/3523227693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c91808aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fcd200640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c91803ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 msgr2=0x55c91803ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c91803ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 msgr2=0x55c91808aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c91808aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3523227693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c917e94950 con 0x55c918036920 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c91808aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 0x55c91808a760 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8054de0 tx=0x7f0fb805d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517ab13a56298496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 1 -- 172.21.15.45:0/3523227693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8080020 con 0x55c918036920 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 1 -- 172.21.15.45:0/3523227693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fb8004dd0 con 0x55c918036920 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fb00a8df0 con 0x55c918036920 2024-04-25T10:21:03.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 1 -- 172.21.15.45:0/3523227693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fb8073170 con 0x55c918036920 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 10 client.5883 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c917f03790 con 0x55c918036920 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 1 -- 172.21.15.45:0/3523227693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a8df0 con 0x55c918036920 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 1 -- 172.21.15.45:0/3523227693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fb805d2b0 con 0x55c918036920 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 10 client.5883 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 10 client.5883 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 20 client.5883 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c91803d790 0x55c91803dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 4 client.5883 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c917e22b80 con 0x55c91803d790 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c91803d790 0x55c91803dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c91803d790 0x55c91803dba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7939add78f1735 server_cookie=d928aa3c14e15895 in_seq=0 out_seq=0 2024-04-25T10:21:03.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 10 client.5883 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 1 -- 172.21.15.45:0/3523227693 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c917e22b80 con 0x55c91803d790 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 10 client.5883 handle_command_reply: tid=0 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 client.5883 shutdown 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 msgr2=0x55c91808a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 0x55c91808a760 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8054de0 tx=0x7f0fb805d810 comp rx=0 tx=0).stop 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 shutdown_connections 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c91803d790 0x55c91803dba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c91803ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c91808aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fb5600640 0 client.5883 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3523227693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918036920 0x55c91808a760 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 >> 172.21.15.45:0/3523227693 conn(0x55c918063f90 msgr2=0x55c918035ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.911+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 shutdown_connections 2024-04-25T10:21:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.915+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3523227693 wait complete. 2024-04-25T10:21:03.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.915+0000 7f0fd26a8000 20 client.5883 trim_cache size 0 max 16384 2024-04-25T10:21:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.927+0000 7f0fd26a8000 1 Processor -- start 2024-04-25T10:21:03.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.927+0000 7f0fd26a8000 1 -- start start 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c918036920 0x55c9180828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 0x55c918034b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c918035080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81ffee0 con 0x55c9180831f0 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81ff080 con 0x55c918036920 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80c6330 con 0x55c918083b10 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 0x55c918034b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 0x55c918034b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41292/0 (socket says 172.21.15.45:41292) 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 -- 172.21.15.45:0/3965309998 learned_addr learned my addr 172.21.15.45:0/3965309998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fcd200640 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c918035080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c918036920 0x55c9180828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 -- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 msgr2=0x55c918035080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c918035080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 -- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c918036920 msgr2=0x55c9180828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c918036920 0x55c9180828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 -- 172.21.15.45:0/3965309998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c917e4c6f0 con 0x55c9180831f0 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 0x55c918034b40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009630 tx=0x7f0fb0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096cfb4d7e2b086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fb7e00640 1 -- 172.21.15.45:0/3965309998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00cb040 con 0x55c9180831f0 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fb7e00640 1 -- 172.21.15.45:0/3965309998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a8df0 con 0x55c9180831f0 2024-04-25T10:21:03.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fb7e00640 1 -- 172.21.15.45:0/3965309998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00b5040 con 0x55c9180831f0 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3965309998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 msgr2=0x55c918034b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 0x55c918034b40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009630 tx=0x7f0fb0009240 comp rx=0 tx=0).stop 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3965309998 shutdown_connections 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918083b10 0x55c918035080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c9180831f0 0x55c918034b40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3965309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c918036920 0x55c9180828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3965309998 >> 172.21.15.45:0/3965309998 conn(0x55c918063f90 msgr2=0x55c918090790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3965309998 shutdown_connections 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3965309998 wait complete. 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 -- start start 2024-04-25T10:21:03.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.931+0000 7f0fd26a8000 1 Processor -- start 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c9180e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c918085f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c918042130 con 0x55c918083b10 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81feab0 con 0x55c9180831f0 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c917e10140 con 0x55c918036920 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 10 client.0 _ll_register_callbacks cb 0x7f0fb00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c9180e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c9180e3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37688/0 (socket says 172.21.15.45:37688) 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3262054608 learned_addr learned my addr 172.21.15.45:0/3262054608 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fcd200640 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c918085f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 msgr2=0x55c9180e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3262054608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 msgr2=0x55c918085f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c918085f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 -- 172.21.15.45:0/3262054608 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c917e42c80 con 0x55c918036920 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fc7e00640 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c9180e3a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8007fd0 tx=0x7f0fb8000bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd65b571cb439f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 1 -- 172.21.15.45:0/3262054608 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8080040 con 0x55c918036920 2024-04-25T10:21:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 1 -- 172.21.15.45:0/3262054608 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fb8050e30 con 0x55c918036920 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fb00a8df0 con 0x55c918036920 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 1 -- 172.21.15.45:0/3262054608 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fb8054c20 con 0x55c918036920 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 10 client.5312 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55c917b43600 con 0x55c918036920 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 1 -- 172.21.15.45:0/3262054608 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00a8df0 con 0x55c918036920 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 1 -- 172.21.15.45:0/3262054608 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0fb8051030 con 0x55c918036920 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 10 client.5312 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 10 client.5312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 20 client.5312 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c9180868f0 0x55c918086cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 4 client.5312 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55c918063de0 con 0x55c9180868f0 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c9180868f0 0x55c918086cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fbfe00640 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c9180868f0 0x55c918086cd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b4313cfb48ca8c server_cookie=9e691a2812c6b73e in_seq=0 out_seq=0 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 10 client.5312 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 1 -- 172.21.15.45:0/3262054608 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c918063de0 con 0x55c9180868f0 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 10 client.5312 handle_command_reply: tid=0 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 client.5312 shutdown 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 msgr2=0x55c9180e3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c9180e3a40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8007fd0 tx=0x7f0fb8000bd0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 shutdown_connections 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55c9180868f0 0x55c918086cd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c918083b10 0x55c918085f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9180831f0 0x55c9180e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 --2- 172.21.15.45:0/3262054608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c918036920 0x55c9180e3a40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fb5600640 0 client.5312 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 >> 172.21.15.45:0/3262054608 conn(0x55c918063f90 msgr2=0x55c91803db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 shutdown_connections 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.935+0000 7f0fd26a8000 1 -- 172.21.15.45:0/3262054608 wait complete. 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fd26a8000 20 client.5312 trim_cache size 0 max 16384 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "2c78f40e-4475-4d32-b5fa-e0837a4bd834", 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:03.944 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:03.945 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:21:03.938+0000 7faeed000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096b [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v622 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5721=0-4194304@1} caps={5721=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5721 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x564ec1c9ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1000000096b:[<0x10000000001/new_file_0 v622>,<0x10000000000/ceph-qa-suite v53>,<0x1/scrub_test_path v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:21:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fa4066f60 msgr2=0x7f0fa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fa4066f60 0x7f0fa4069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00b2000 tx=0x7f0fb0009040 comp rx=0 tx=0).stop 2024-04-25T10:21:03.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 msgr2=0x7f0fc8154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8154170 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800c420 tx=0x7f0fb800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 shutdown_connections 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fa4066f60 0x7f0fa4069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fc8161e50 0x7f0fc80ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fc80cd9a0 0x7f0fc80c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 --2- 172.21.15.45:0/1049247744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fc806bd10 0x7f0fc8154170 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 >> 172.21.15.45:0/1049247744 conn(0x7f0fc80d19a0 msgr2=0x7f0fc80b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 shutdown_connections 2024-04-25T10:21:03.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:03.939+0000 7f0fcf200640 1 -- 172.21.15.45:0/1049247744 wait complete. 2024-04-25T10:21:03.959 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '2c78f40e-4475-4d32-b5fa-e0837a4bd834', 'mode': 'asynchronous'}' 2024-04-25T10:21:03.960 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:21:03.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:21:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 Processor -- start 2024-04-25T10:21:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 -- start start 2024-04-25T10:21:04.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0840aef80 0x7ff08415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0840b31c0 0x7ff0840b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08415b080 con 0x7ff084070350 2024-04-25T10:21:04.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08415adb0 con 0x7ff0840b31c0 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff08be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08415eb60 con 0x7ff0840aef80 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0840b31c0 0x7ff0840b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0840b31c0 0x7ff0840b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41302/0 (socket says 172.21.15.45:41302) 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 -- 172.21.15.45:0/3557484131 learned_addr learned my addr 172.21.15.45:0/3557484131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089400640 1 --2- 172.21.15.45:0/3557484131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0840aef80 0x7ff08415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 -- 172.21.15.45:0/3557484131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0840aef80 msgr2=0x7ff08415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 --2- 172.21.15.45:0/3557484131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0840aef80 0x7ff08415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 -- 172.21.15.45:0/3557484131 >> v1:172.21.15.117:6789/0 conn(0x7ff084070350 legacy=0x7ff08415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.103+0000 7ff089e00640 1 -- 172.21.15.45:0/3557484131 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08415ee40 con 0x7ff0840b31c0 2024-04-25T10:21:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff089e00640 1 --2- 172.21.15.45:0/3557484131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840b31c0 0x7ff0840b3610 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d150 tx=0x7ff07c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c964c64ec981e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff083e00640 1 -- 172.21.15.45:0/3557484131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00b2d0 con 0x7ff0840b31c0 2024-04-25T10:21:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff083e00640 1 -- 172.21.15.45:0/3557484131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c00b470 con 0x7ff0840b31c0 2024-04-25T10:21:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff083e00640 1 -- 172.21.15.45:0/3557484131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0a7c30 con 0x7ff0840b31c0 2024-04-25T10:21:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 -- 172.21.15.45:0/3557484131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840b31c0 msgr2=0x7ff0840b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 --2- 172.21.15.45:0/3557484131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840b31c0 0x7ff0840b3610 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00d150 tx=0x7ff07c01ec30 comp rx=0 tx=0).stop 2024-04-25T10:21:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 -- 172.21.15.45:0/3557484131 shutdown_connections 2024-04-25T10:21:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 --2- 172.21.15.45:0/3557484131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840b31c0 0x7ff0840b3610 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 --2- 172.21.15.45:0/3557484131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0840aef80 0x7ff08415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 -- 172.21.15.45:0/3557484131 >> 172.21.15.45:0/3557484131 conn(0x7ff0840669a0 msgr2=0x7ff084066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 -- 172.21.15.45:0/3557484131 shutdown_connections 2024-04-25T10:21:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.107+0000 7ff08be00640 1 -- 172.21.15.45:0/3557484131 wait complete. 2024-04-25T10:21:04.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 Processor -- start 2024-04-25T10:21:04.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 -- start start 2024-04-25T10:21:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff084070350 0x7ff084153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 0x7ff084150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0840b31c0 0x7ff08415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08406c250 con 0x7ff0840aef80 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08406c0d0 con 0x7ff0840b31c0 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08406c3d0 con 0x7ff084070350 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 0x7ff084150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 0x7ff084150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41310/0 (socket says 172.21.15.45:41310) 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 -- 172.21.15.45:0/1447174113 learned_addr learned my addr 172.21.15.45:0/1447174113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff089400640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff084070350 0x7ff084153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff089e00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0840b31c0 0x7ff08415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 -- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff084070350 msgr2=0x7ff084153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff084070350 0x7ff084153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 -- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0840b31c0 msgr2=0x7ff08415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0840b31c0 0x7ff08415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 -- 172.21.15.45:0/1447174113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0841507d0 con 0x7ff0840aef80 2024-04-25T10:21:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff088a00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 0x7ff084150290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c450 tx=0x7ff07000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff081600640 1 -- 172.21.15.45:0/1447174113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070016020 con 0x7ff0840aef80 2024-04-25T10:21:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff081600640 1 -- 172.21.15.45:0/1447174113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0700052d0 con 0x7ff0840aef80 2024-04-25T10:21:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff081600640 1 -- 172.21.15.45:0/1447174113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070007260 con 0x7ff0840aef80 2024-04-25T10:21:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08415a190 con 0x7ff0840aef80 2024-04-25T10:21:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.111+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08415a750 con 0x7ff0840aef80 2024-04-25T10:21:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff081600640 1 -- 172.21.15.45:0/1447174113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff070007400 con 0x7ff0840aef80 2024-04-25T10:21:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff081600640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff064066f60 0x7ff064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff081600640 1 -- 172.21.15.45:0/1447174113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff070084880 con 0x7ff0840aef80 2024-04-25T10:21:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 Processor -- start 2024-04-25T10:21:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff064066f60 0x7ff064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 -- start start 2024-04-25T10:21:04.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff064066f60 0x7ff064069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff078006aa0 tx=0x7ff078008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206f22f0 0x564e206a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206a80a0 0x564e206a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e204f4ff0 con 0x564e206f19d0 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2047d150 con 0x564e206f22f0 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e204f1090 con 0x564e206a80a0 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41312/0 (socket says 172.21.15.45:41312) 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 -- 172.21.15.45:0/3091804528 learned_addr learned my addr 172.21.15.45:0/3091804528 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff088a00640 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206f22f0 0x564e206a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089e00640 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206a80a0 0x564e206a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 -- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206a80a0 msgr2=0x564e206a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206a80a0 0x564e206a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 -- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206f22f0 msgr2=0x564e206a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206f22f0 0x564e206a7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.115+0000 7ff089400640 1 -- 172.21.15.45:0/3091804528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2068b0d0 con 0x564e206f19d0 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206f22f0 0x564e206a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff089400640 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206f1db0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff0780041f0 tx=0x7ff078013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df6042ced68ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff076000640 1 -- 172.21.15.45:0/3091804528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078027070 con 0x564e206f19d0 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff076000640 1 -- 172.21.15.45:0/3091804528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078003830 con 0x564e206f19d0 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff076000640 1 -- 172.21.15.45:0/3091804528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078003be0 con 0x564e206f19d0 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3091804528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 msgr2=0x564e206f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206f1db0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff0780041f0 tx=0x7ff078013000 comp rx=0 tx=0).stop 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3091804528 shutdown_connections 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206a80a0 0x564e206a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206f22f0 0x564e206a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3091804528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206f1db0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3091804528 >> 172.21.15.45:0/3091804528 conn(0x564e206a4020 msgr2=0x564e206a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3091804528 shutdown_connections 2024-04-25T10:21:04.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3091804528 wait complete. 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- start start 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 Processor -- start 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e206d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e206d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e204b9250 con 0x564e206f19d0 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e204af690 con 0x564e206a80a0 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e2047da40 con 0x564e206f22f0 2024-04-25T10:21:04.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff08f1ff000 10 client.0 _ll_register_callbacks cb 0x564e20692e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206d6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41316/0 (socket says 172.21.15.45:41316) 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 -- 172.21.15.45:0/546030737 learned_addr learned my addr 172.21.15.45:0/546030737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 -- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 msgr2=0x564e206d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff089e00640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e206d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff089400640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e206d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e206d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 -- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 msgr2=0x564e206d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e206d67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff088a00640 1 -- 172.21.15.45:0/546030737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e205a21c0 con 0x564e206f19d0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff089400640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e206d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.119+0000 7ff089e00640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e206d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff088a00640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206d6d30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c420 tx=0x7ff070055380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea7ffee148addcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:04.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 1 -- 172.21.15.45:0/546030737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070080070 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 1 -- 172.21.15.45:0/546030737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff070054680 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff078003830 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 1 -- 172.21.15.45:0/546030737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff070084050 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 10 client.5901 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x564e2059e170 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 1 -- 172.21.15.45:0/546030737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078003830 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 1 -- 172.21.15.45:0/546030737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff070073170 con 0x564e206f19d0 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 10 client.5901 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 10 client.5901 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 20 client.5901 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20701c80 0x564e20704070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 4 client.5901 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:04.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e20490c60 con 0x564e20701c80 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff089400640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20701c80 0x564e20704070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff089400640 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20701c80 0x564e20704070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dfbda8faba54e72 server_cookie=d1d2e314912c8924 in_seq=0 out_seq=0 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 10 client.5901 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 1 -- 172.21.15.45:0/546030737 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e20490c60 con 0x564e20701c80 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 10 client.5901 handle_command_reply: tid=0 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 client.5901 shutdown 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 msgr2=0x564e206d6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206d6d30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c420 tx=0x7ff070055380 comp rx=0 tx=0).stop 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 shutdown_connections 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20701c80 0x564e20704070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e206d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e206d6d30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff06aa00640 0 client.5901 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/546030737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e206d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 >> 172.21.15.45:0/546030737 conn(0x564e206a4020 msgr2=0x564e206a6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 shutdown_connections 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 1 -- 172.21.15.45:0/546030737 wait complete. 2024-04-25T10:21:04.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.123+0000 7ff08f1ff000 20 client.5901 trim_cache size 0 max 16384 2024-04-25T10:21:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 Processor -- start 2024-04-25T10:21:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 -- start start 2024-04-25T10:21:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e2074e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e2074e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e2074ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0842031b0 con 0x564e206f19d0 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084202330 con 0x564e206a80a0 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff084201730 con 0x564e206f22f0 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e2074e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e2074e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41326/0 (socket says 172.21.15.45:41326) 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 -- 172.21.15.45:0/2584864075 learned_addr learned my addr 172.21.15.45:0/2584864075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff089e00640 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e2074ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff089400640 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e2074e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 -- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 msgr2=0x564e2074ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e2074ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 -- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 msgr2=0x564e2074e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e2074e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff088a00640 1 -- 172.21.15.45:0/2584864075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2053bde0 con 0x564e206f19d0 2024-04-25T10:21:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.139+0000 7ff089400640 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e2074e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e2074e840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff070051bb0 tx=0x7ff070057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd4147efbcca196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff075600640 1 -- 172.21.15.45:0/2584864075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070080070 con 0x564e206f19d0 2024-04-25T10:21:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff075600640 1 -- 172.21.15.45:0/2584864075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070055230 con 0x564e206f19d0 2024-04-25T10:21:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff075600640 1 -- 172.21.15.45:0/2584864075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070054680 con 0x564e206f19d0 2024-04-25T10:21:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- 172.21.15.45:0/2584864075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 msgr2=0x564e2074e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e2074e840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff070051bb0 tx=0x7ff070057000 comp rx=0 tx=0).stop 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- 172.21.15.45:0/2584864075 shutdown_connections 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e2074ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e2074e840 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/2584864075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e2074e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- 172.21.15.45:0/2584864075 >> 172.21.15.45:0/2584864075 conn(0x564e206a4020 msgr2=0x564e206f00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- 172.21.15.45:0/2584864075 shutdown_connections 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- 172.21.15.45:0/2584864075 wait complete. 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- start start 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 Processor -- start 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e20752310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e20752850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e20704380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2048fe00 con 0x564e206f19d0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084201d30 con 0x564e206a80a0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e204f4300 con 0x564e206f22f0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 10 client.0 _ll_register_callbacks cb 0x564e20673040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e20752850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e20752850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41342/0 (socket says 172.21.15.45:41342) 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 -- 172.21.15.45:0/3574890938 learned_addr learned my addr 172.21.15.45:0/3574890938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff089e00640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e20704380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff089400640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e20752310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 -- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 msgr2=0x564e20704380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e20704380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 -- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 msgr2=0x564e20752310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e20752310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 -- 172.21.15.45:0/3574890938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e204968e0 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff088a00640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e20752850 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c420 tx=0x7ff070082c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc038495e5e08787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff06aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff06aa00640 1 -- 172.21.15.45:0/3574890938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070080040 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff06aa00640 1 -- 172.21.15.45:0/3574890938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff07007c3b0 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.143+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff070055230 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 1 -- 172.21.15.45:0/3574890938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff07005b260 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 10 client.5907 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x564e2054e8f0 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 1 -- 172.21.15.45:0/3574890938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070055230 con 0x564e206f19d0 2024-04-25T10:21:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 1 -- 172.21.15.45:0/3574890938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff07005b400 con 0x564e206f19d0 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 10 client.5907 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 10 client.5907 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 20 client.5907 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20704d40 0x564e20705150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 4 client.5907 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e20502530 con 0x564e20704d40 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff089400640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20704d40 0x564e20705150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 10 client.5907 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:04.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff089400640 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20704d40 0x564e20705150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99dd198d9dff766b server_cookie=921d63c4c68f075e in_seq=0 out_seq=0 2024-04-25T10:21:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 1 -- 172.21.15.45:0/3574890938 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x564e20502530 con 0x564e20704d40 2024-04-25T10:21:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 10 client.5907 handle_command_reply: tid=0 2024-04-25T10:21:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 client.5907 shutdown 2024-04-25T10:21:04.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 msgr2=0x564e20752850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e20752850 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c420 tx=0x7ff070082c70 comp rx=0 tx=0).stop 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 shutdown_connections 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x564e20704d40 0x564e20705150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564e206f22f0 0x564e20704380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e206f19d0 0x564e20752850 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff06aa00640 0 client.5907 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 --2- 172.21.15.45:0/3574890938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e206a80a0 0x564e20752310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 >> 172.21.15.45:0/3574890938 conn(0x564e206a4020 msgr2=0x564e206f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 shutdown_connections 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 1 -- 172.21.15.45:0/3574890938 wait complete. 2024-04-25T10:21:04.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08f1ff000 20 client.5907 trim_cache size 0 max 16384 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "2c78f40e-4475-4d32-b5fa-e0837a4bd834": { 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "2c78f40e-4475-4d32-b5fa-e0837a4bd834", 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "" 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:04.156 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff064066f60 msgr2=0x7ff064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff064066f60 0x7ff064069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff078006aa0 tx=0x7ff078008040 comp rx=0 tx=0).stop 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 msgr2=0x7ff084150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 0x7ff084150290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c450 tx=0x7ff07000c910 comp rx=0 tx=0).stop 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 shutdown_connections 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff064066f60 0x7ff064069420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0840b31c0 0x7ff08415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0840aef80 0x7ff084150290 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 --2- 172.21.15.45:0/1447174113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff084070350 0x7ff084153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 >> 172.21.15.45:0/1447174113 conn(0x7ff0840669a0 msgr2=0x7ff0841521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 shutdown_connections 2024-04-25T10:21:04.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:04.147+0000 7ff08be00640 1 -- 172.21.15.45:0/1447174113 wait complete. 2024-04-25T10:21:09.172 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:21:09.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:21:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.323+0000 7f1a10a00640 1 Processor -- start 2024-04-25T10:21:09.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.323+0000 7f1a10a00640 1 -- start start 2024-04-25T10:21:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a10a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a0c0b2c70 0x7f1a0c0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b3050 0x7f1a0c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a10a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c06c830 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a10a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c06cb10 con 0x7f1a0c0b2c70 2024-04-25T10:21:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a10a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c06cdf0 con 0x7f1a0c0b3050 2024-04-25T10:21:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a0c0b2c70 0x7f1a0c0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a0c0b2c70 0x7f1a0c0b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50450/0 (socket says 172.21.15.45:50450) 2024-04-25T10:21:09.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 -- 172.21.15.45:0/553258745 learned_addr learned my addr 172.21.15.45:0/553258745 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 -- 172.21.15.45:0/553258745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b3050 msgr2=0x7f1a0c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0b200640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b3050 0x7f1a0c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b3050 0x7f1a0c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 -- 172.21.15.45:0/553258745 >> v1:172.21.15.117:6789/0 conn(0x7f1a0c0b4c80 legacy=0x7f1a0c06d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0a800640 1 -- 172.21.15.45:0/553258745 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c06d0d0 con 0x7f1a0c0b2c70 2024-04-25T10:21:09.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.327+0000 7f1a0b200640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b3050 0x7f1a0c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a0a800640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b2c70 0x7f1a0c0b4740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009e50 tx=0x7f19fc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4997c8e6f1af99dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a09e00640 1 -- 172.21.15.45:0/553258745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0a5820 con 0x7f1a0c0b2c70 2024-04-25T10:21:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a09e00640 1 -- 172.21.15.45:0/553258745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0a59c0 con 0x7f1a0c0b2c70 2024-04-25T10:21:09.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a09e00640 1 -- 172.21.15.45:0/553258745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0a5d90 con 0x7f1a0c0b2c70 2024-04-25T10:21:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 -- 172.21.15.45:0/553258745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b2c70 msgr2=0x7f1a0c0b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b2c70 0x7f1a0c0b4740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009e50 tx=0x7f19fc01c940 comp rx=0 tx=0).stop 2024-04-25T10:21:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 -- 172.21.15.45:0/553258745 shutdown_connections 2024-04-25T10:21:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b3050 0x7f1a0c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 --2- 172.21.15.45:0/553258745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b2c70 0x7f1a0c0b4740 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 -- 172.21.15.45:0/553258745 >> 172.21.15.45:0/553258745 conn(0x7f1a0c0b1710 msgr2=0x7f1a0c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 -- 172.21.15.45:0/553258745 shutdown_connections 2024-04-25T10:21:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 -- 172.21.15.45:0/553258745 wait complete. 2024-04-25T10:21:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 Processor -- start 2024-04-25T10:21:09.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.331+0000 7f1a10a00640 1 -- start start 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b2c70 0x7f1a0c155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a0c0b3050 0x7f1a0c072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 0x7f1a0c1f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c15c4d0 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c15c350 con 0x7f1a0c0b3050 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c15c650 con 0x7f1a0c0b2c70 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 0x7f1a0c1f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 0x7f1a0c1f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50454/0 (socket says 172.21.15.45:50454) 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 -- 172.21.15.45:0/2324091236 learned_addr learned my addr 172.21.15.45:0/2324091236 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0a800640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b2c70 0x7f1a0c155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a01e00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a0c0b3050 0x7f1a0c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 -- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b2c70 msgr2=0x7f1a0c155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b2c70 0x7f1a0c155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 -- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a0c0b3050 msgr2=0x7f1a0c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a0c0b3050 0x7f1a0c072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 -- 172.21.15.45:0/2324091236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c1f7100 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a01e00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a0c0b3050 0x7f1a0c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0a800640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b2c70 0x7f1a0c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a0b200640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 0x7f1a0c1f6920 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c7f0 tx=0x7f1a0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a03400640 1 -- 172.21.15.45:0/2324091236 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04015070 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a03400640 1 -- 172.21.15.45:0/2324091236 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a04007860 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c072640 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a03400640 1 -- 172.21.15.45:0/2324091236 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04007c50 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c071620 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.335+0000 7f1a03400640 1 -- 172.21.15.45:0/2324091236 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a04018070 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a03400640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19e8066f60 0x7f19e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a03400640 1 -- 172.21.15.45:0/2324091236 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a04085270 con 0x7f1a0c0b4c80 2024-04-25T10:21:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 Processor -- start 2024-04-25T10:21:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19e8066f60 0x7f19e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 -- start start 2024-04-25T10:21:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19e8066f60 0x7f19e8069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ae000 tx=0x7f19fc0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149051db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149007b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x557149008480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557148e54ff0 con 0x5571490519d0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557148ddd150 con 0x5571490522f0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557148e51090 con 0x5571490080a0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149051db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149051db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50470/0 (socket says 172.21.15.45:50470) 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 -- 172.21.15.45:0/342281855 learned_addr learned my addr 172.21.15.45:0/342281855 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a01e00640 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149007b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0b200640 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x557149008480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 -- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 msgr2=0x557149008480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x557149008480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 -- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 msgr2=0x557149007b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149007b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 -- 172.21.15.45:0/342281855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557148feb0d0 con 0x5571490519d0 2024-04-25T10:21:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0b200640 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x557149008480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a0a800640 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149051db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f19fc01c980 tx=0x7f19fc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0c32706f5986c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f19fb400640 1 -- 172.21.15.45:0/342281855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0c6070 con 0x5571490519d0 2024-04-25T10:21:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f19fb400640 1 -- 172.21.15.45:0/342281855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0b6620 con 0x5571490519d0 2024-04-25T10:21:09.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f19fb400640 1 -- 172.21.15.45:0/342281855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0b69f0 con 0x5571490519d0 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 -- 172.21.15.45:0/342281855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 msgr2=0x557149051db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.339+0000 7f1a149b1000 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149051db0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f19fc01c980 tx=0x7f19fc004ac0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- 172.21.15.45:0/342281855 shutdown_connections 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x557149008480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149007b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 --2- 172.21.15.45:0/342281855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149051db0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- 172.21.15.45:0/342281855 >> 172.21.15.45:0/342281855 conn(0x557149004020 msgr2=0x557149006440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- 172.21.15.45:0/342281855 shutdown_connections 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- 172.21.15.45:0/342281855 wait complete. 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- start start 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 Processor -- start 2024-04-25T10:21:09.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x5571490367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149036d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x557149037270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557148e19250 con 0x5571490519d0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557148e0f690 con 0x5571490080a0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557148ddda40 con 0x5571490522f0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 10 client.0 _ll_register_callbacks cb 0x557148ff2e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149036d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149036d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50478/0 (socket says 172.21.15.45:50478) 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 -- 172.21.15.45:0/4269555295 learned_addr learned my addr 172.21.15.45:0/4269555295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x5571490367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a0b200640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x557149037270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 -- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 msgr2=0x557149037270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x557149037270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 -- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 msgr2=0x5571490367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x5571490367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 -- 172.21.15.45:0/4269555295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557148f021c0 con 0x5571490519d0 2024-04-25T10:21:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149036d30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001e90 tx=0x7f19f40116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff60f5686fb0b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f19efe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f19efe00640 1 -- 172.21.15.45:0/4269555295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40052e0 con 0x5571490519d0 2024-04-25T10:21:09.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19fc0b6620 con 0x5571490519d0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f19efe00640 1 -- 172.21.15.45:0/4269555295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19f4003600 con 0x5571490519d0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f19efe00640 1 -- 172.21.15.45:0/4269555295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f4022050 con 0x5571490519d0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 10 client.5919 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x557148efe170 con 0x5571490519d0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.343+0000 7f19efe00640 1 -- 172.21.15.45:0/4269555295 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0b6620 con 0x5571490519d0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f19efe00640 1 -- 172.21.15.45:0/4269555295 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19f4025020 con 0x5571490519d0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 10 client.5919 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 10 client.5919 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 20 client.5919 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x557149061c80 0x557149064070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 4 client.5919 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557148df0c60 con 0x557149061c80 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x557149061c80 0x557149064070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x557149061c80 0x557149064070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b60570d3693159 server_cookie=5e79dd0bab5870fa in_seq=0 out_seq=0 2024-04-25T10:21:09.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f19efe00640 10 client.5919 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f19efe00640 1 -- 172.21.15.45:0/4269555295 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557148df0c60 con 0x557149061c80 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f19efe00640 10 client.5919 handle_command_reply: tid=0 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 client.5919 shutdown 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 msgr2=0x557149036d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149036d30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001e90 tx=0x7f19f40116e0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 shutdown_connections 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x557149061c80 0x557149064070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x557149037270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f19efe00640 0 client.5919 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x557149036d30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4269555295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x5571490367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 >> 172.21.15.45:0/4269555295 conn(0x557149004020 msgr2=0x557149006ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 shutdown_connections 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 1 -- 172.21.15.45:0/4269555295 wait complete. 2024-04-25T10:21:09.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.347+0000 7f1a149b1000 20 client.5919 trim_cache size 0 max 16384 2024-04-25T10:21:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 Processor -- start 2024-04-25T10:21:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 -- start start 2024-04-25T10:21:09.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x557149007a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x55714905ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x55714905f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c2029d0 con 0x5571490519d0 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c201bd0 con 0x5571490080a0 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c1fa490 con 0x5571490522f0 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x55714905ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x55714905ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50486/0 (socket says 172.21.15.45:50486) 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 -- 172.21.15.45:0/4287078718 learned_addr learned my addr 172.21.15.45:0/4287078718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x557149007a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a0b200640 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x55714905f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 -- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 msgr2=0x55714905f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x55714905f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 -- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 msgr2=0x557149007a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x557149007a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 -- 172.21.15.45:0/4287078718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557148e9bde0 con 0x5571490519d0 2024-04-25T10:21:09.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x55714905ee40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f19f4007ed0 tx=0x7f19f400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce275af3bb13fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f19faa00640 1 -- 172.21.15.45:0/4287078718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40052e0 con 0x5571490519d0 2024-04-25T10:21:09.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f19faa00640 1 -- 172.21.15.45:0/4287078718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f401a070 con 0x5571490519d0 2024-04-25T10:21:09.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.363+0000 7f19faa00640 1 -- 172.21.15.45:0/4287078718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4003600 con 0x5571490519d0 2024-04-25T10:21:09.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4287078718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 msgr2=0x55714905ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x55714905ee40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f19f4007ed0 tx=0x7f19f400c000 comp rx=0 tx=0).stop 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4287078718 shutdown_connections 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490522f0 0x55714905f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x55714905ee40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4287078718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490080a0 0x557149007a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4287078718 >> 172.21.15.45:0/4287078718 conn(0x557149004020 msgr2=0x55714905e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4287078718 shutdown_connections 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4287078718 wait complete. 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- start start 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 Processor -- start 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x5571490b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x5571490b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149063a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557148defe00 con 0x5571490519d0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c0ae100 con 0x5571490522f0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557148e54300 con 0x5571490080a0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 10 client.0 _ll_register_callbacks cb 0x557148fd3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x5571490b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x5571490b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50490/0 (socket says 172.21.15.45:50490) 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 -- 172.21.15.45:0/4225042460 learned_addr learned my addr 172.21.15.45:0/4225042460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x5571490b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a0b200640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149063a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 -- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 msgr2=0x5571490b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x5571490b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 -- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 msgr2=0x557149063a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149063a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 -- 172.21.15.45:0/4225042460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557148df68e0 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a01e00640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x5571490b1b30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009010 tx=0x7f19f4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3473f3137ee7f73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f19efe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f19efe00640 1 -- 172.21.15.45:0/4225042460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4034060 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f19efe00640 1 -- 172.21.15.45:0/4225042460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19f4010040 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19f401a070 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f19efe00640 1 -- 172.21.15.45:0/4225042460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f4003d20 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 10 client.5925 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x557148eae8f0 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f19efe00640 1 -- 172.21.15.45:0/4225042460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f401a070 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f19efe00640 1 -- 172.21.15.45:0/4225042460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19f403d020 con 0x5571490519d0 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 10 client.5925 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 10 client.5925 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.367+0000 7f1a149b1000 20 client.5925 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5571490643b0 0x5571490647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 4 client.5925 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557148e62530 con 0x5571490643b0 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5571490643b0 0x5571490647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a0a800640 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5571490643b0 0x5571490647c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fda086707552fcf server_cookie=236c076a85089ad in_seq=0 out_seq=0 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f19efe00640 10 client.5925 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f19efe00640 1 -- 172.21.15.45:0/4225042460 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557148e62530 con 0x5571490643b0 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f19efe00640 10 client.5925 handle_command_reply: tid=0 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 client.5925 shutdown 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 msgr2=0x5571490b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x5571490b1b30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009010 tx=0x7f19f4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 shutdown_connections 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5571490643b0 0x5571490647c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571490522f0 0x557149063a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5571490519d0 0x5571490b1b30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f19efe00640 0 client.5925 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 --2- 172.21.15.45:0/4225042460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5571490080a0 0x5571490b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 >> 172.21.15.45:0/4225042460 conn(0x557149004020 msgr2=0x55714905efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 shutdown_connections 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 1 -- 172.21.15.45:0/4225042460 wait complete. 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a149b1000 20 client.5925 trim_cache size 0 max 16384 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:21:09.379 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:09.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19e8066f60 msgr2=0x7f19e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19e8066f60 0x7f19e8069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0ae000 tx=0x7f19fc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:09.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 msgr2=0x7f1a0c1f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 0x7f1a0c1f6920 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c7f0 tx=0x7f1a0400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 shutdown_connections 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19e8066f60 0x7f19e8069420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a0c0b4c80 0x7f1a0c1f6920 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a0c0b3050 0x7f1a0c072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 --2- 172.21.15.45:0/2324091236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a0c0b2c70 0x7f1a0c155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 >> 172.21.15.45:0/2324091236 conn(0x7f1a0c0b1710 msgr2=0x7f1a0c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 shutdown_connections 2024-04-25T10:21:09.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.371+0000 7f1a10a00640 1 -- 172.21.15.45:0/2324091236 wait complete. 2024-04-25T10:21:09.394 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:21:09.394 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:09.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:21:09.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 Processor -- start 2024-04-25T10:21:09.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 -- start start 2024-04-25T10:21:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98f415d9b0 0x7f98f40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415dd90 0x7f98f40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f406d220 con 0x7f98f41620b0 2024-04-25T10:21:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f4162600 con 0x7f98f415d9b0 2024-04-25T10:21:09.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40cad00 con 0x7f98f415dd90 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98f415d9b0 0x7f98f40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98f415d9b0 0x7f98f40cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50494/0 (socket says 172.21.15.45:50494) 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 -- 172.21.15.45:0/3730886909 learned_addr learned my addr 172.21.15.45:0/3730886909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98e9400640 1 --2- 172.21.15.45:0/3730886909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415dd90 0x7f98f40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 -- 172.21.15.45:0/3730886909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415dd90 msgr2=0x7f98f40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 --2- 172.21.15.45:0/3730886909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415dd90 0x7f98f40669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 -- 172.21.15.45:0/3730886909 >> v1:172.21.15.117:6789/0 conn(0x7f98f41620b0 legacy=0x7f98f406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.535+0000 7f98f1e00640 1 -- 172.21.15.45:0/3730886909 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40cafe0 con 0x7f98f415d9b0 2024-04-25T10:21:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f1e00640 1 --2- 172.21.15.45:0/3730886909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f415d9b0 0x7f98f40cc860 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009870 tx=0x7f98e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49819a5d9175c986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f1400640 1 -- 172.21.15.45:0/3730886909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40a7070 con 0x7f98f415d9b0 2024-04-25T10:21:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f1400640 1 -- 172.21.15.45:0/3730886909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e40b0070 con 0x7f98f415d9b0 2024-04-25T10:21:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f1400640 1 -- 172.21.15.45:0/3730886909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40ab890 con 0x7f98f415d9b0 2024-04-25T10:21:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 -- 172.21.15.45:0/3730886909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f415d9b0 msgr2=0x7f98f40cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 --2- 172.21.15.45:0/3730886909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f415d9b0 0x7f98f40cc860 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98e4009870 tx=0x7f98e401c560 comp rx=0 tx=0).stop 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 -- 172.21.15.45:0/3730886909 shutdown_connections 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 --2- 172.21.15.45:0/3730886909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415dd90 0x7f98f40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 --2- 172.21.15.45:0/3730886909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f415d9b0 0x7f98f40cc860 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 -- 172.21.15.45:0/3730886909 >> 172.21.15.45:0/3730886909 conn(0x7f98f40d07e0 msgr2=0x7f98f40d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 -- 172.21.15.45:0/3730886909 shutdown_connections 2024-04-25T10:21:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.539+0000 7f98f3200640 1 -- 172.21.15.45:0/3730886909 wait complete. 2024-04-25T10:21:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 Processor -- start 2024-04-25T10:21:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 -- start start 2024-04-25T10:21:09.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415d9b0 0x7f98f4153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98f415dd90 0x7f98f415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 0x7f98f415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f416aca0 con 0x7f98f41620b0 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40672f0 con 0x7f98f415dd90 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f416ae20 con 0x7f98f415d9b0 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 0x7f98f415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 0x7f98f415ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50502/0 (socket says 172.21.15.45:50502) 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 -- 172.21.15.45:0/3514395800 learned_addr learned my addr 172.21.15.45:0/3514395800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f1e00640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415d9b0 0x7f98f4153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98e9400640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98f415dd90 0x7f98f415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 -- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415d9b0 msgr2=0x7f98f4153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415d9b0 0x7f98f4153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 -- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98f415dd90 msgr2=0x7f98f415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98f415dd90 0x7f98f415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 -- 172.21.15.45:0/3514395800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f4153e50 con 0x7f98f41620b0 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f2800640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 0x7f98f415ce50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c3c0 tx=0x7f98ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98eaa00640 1 -- 172.21.15.45:0/3514395800 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec017070 con 0x7f98f41620b0 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98eaa00640 1 -- 172.21.15.45:0/3514395800 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec0052d0 con 0x7f98f41620b0 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f4158160 con 0x7f98f41620b0 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98eaa00640 1 -- 172.21.15.45:0/3514395800 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec003860 con 0x7f98f41620b0 2024-04-25T10:21:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.543+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f4158720 con 0x7f98f41620b0 2024-04-25T10:21:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98eaa00640 1 -- 172.21.15.45:0/3514395800 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98ec01a070 con 0x7f98f41620b0 2024-04-25T10:21:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98eaa00640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98d0066f60 0x7f98d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98eaa00640 1 -- 172.21.15.45:0/3514395800 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98ec0842b0 con 0x7f98f41620b0 2024-04-25T10:21:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 Processor -- start 2024-04-25T10:21:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98f1e00640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98d0066f60 0x7f98d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 -- start start 2024-04-25T10:21:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86a180 0x55b9bd86a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98f1e00640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98d0066f60 0x7f98d0069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f98e40af000 tx=0x7f98e40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 0x55b9bd8200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd8209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd66df20 con 0x55b9bd86aaa0 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd5f6000 con 0x55b9bd820600 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.547+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd669fc0 con 0x55b9bd86a180 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 0x55b9bd8200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 0x55b9bd8200c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50518/0 (socket says 172.21.15.45:50518) 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 -- 172.21.15.45:0/3238051671 learned_addr learned my addr 172.21.15.45:0/3238051671 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98f1e00640 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86a180 0x55b9bd86a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98f2800640 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd8209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 -- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86a180 msgr2=0x55b9bd86a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86a180 0x55b9bd86a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 -- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 msgr2=0x55b9bd8209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd8209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 -- 172.21.15.45:0/3238051671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9bd649ba0 con 0x55b9bd86aaa0 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 0x55b9bd8200c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0074e0 tx=0x7f98dc0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3124f85c1ffa5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e2a00640 1 -- 172.21.15.45:0/3238051671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0147e0 con 0x55b9bd86aaa0 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e2a00640 1 -- 172.21.15.45:0/3238051671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc01a070 con 0x55b9bd86aaa0 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e2a00640 1 -- 172.21.15.45:0/3238051671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc014ba0 con 0x55b9bd86aaa0 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- 172.21.15.45:0/3238051671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 msgr2=0x55b9bd8200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 0x55b9bd8200c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0074e0 tx=0x7f98dc0078e0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- 172.21.15.45:0/3238051671 shutdown_connections 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd8209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86aaa0 0x55b9bd8200c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/3238051671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86a180 0x55b9bd86a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- 172.21.15.45:0/3238051671 >> 172.21.15.45:0/3238051671 conn(0x55b9bd826760 msgr2=0x55b9bd81e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- 172.21.15.45:0/3238051671 shutdown_connections 2024-04-25T10:21:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- 172.21.15.45:0/3238051671 wait complete. 2024-04-25T10:21:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- start start 2024-04-25T10:21:09.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 Processor -- start 2024-04-25T10:21:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd825660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 0x55b9bd825ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86aaa0 0x55b9bd858450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd72b9c0 con 0x55b9bd86a180 2024-04-25T10:21:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd6285d0 con 0x55b9bd820600 2024-04-25T10:21:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd62fee0 con 0x55b9bd86aaa0 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98fbfd0000 10 client.0 _ll_register_callbacks cb 0x55b9bd80ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.551+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 0x55b9bd825ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 0x55b9bd825ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50522/0 (socket says 172.21.15.45:50522) 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 -- 172.21.15.45:0/2325417407 learned_addr learned my addr 172.21.15.45:0/2325417407 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 -- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86aaa0 msgr2=0x55b9bd858450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86aaa0 0x55b9bd858450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 -- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 msgr2=0x55b9bd825660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd825660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 -- 172.21.15.45:0/2325417407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9bd6b45a0 con 0x55b9bd86a180 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98e9400640 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 0x55b9bd825ba0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b010 tx=0x7f98dc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bff88defae9647c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 1 -- 172.21.15.45:0/2325417407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc015670 con 0x55b9bd86a180 2024-04-25T10:21:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 1 -- 172.21.15.45:0/2325417407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98dc015810 con 0x55b9bd86a180 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98dc01a070 con 0x55b9bd86a180 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 1 -- 172.21.15.45:0/2325417407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98dc032050 con 0x55b9bd86a180 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 10 client.5937 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55b9bd6452b0 con 0x55b9bd86a180 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 1 -- 172.21.15.45:0/2325417407 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc01a070 con 0x55b9bd86a180 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 1 -- 172.21.15.45:0/2325417407 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98dc035020 con 0x55b9bd86a180 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 10 client.5937 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 10 client.5937 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 20 client.5937 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd858e60 0x55b9bd87c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 4 client.5937 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9bd609c70 con 0x55b9bd858e60 2024-04-25T10:21:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98f1e00640 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd858e60 0x55b9bd87c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98f1e00640 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd858e60 0x55b9bd87c440 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b0b3716ae6f1f74 server_cookie=60765679f076d6cf in_seq=0 out_seq=0 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 10 client.5937 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 1 -- 172.21.15.45:0/2325417407 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9bd609c70 con 0x55b9bd858e60 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 10 client.5937 handle_command_reply: tid=0 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 client.5937 shutdown 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 msgr2=0x55b9bd825ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 0x55b9bd825ba0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00b010 tx=0x7f98dc004ac0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 shutdown_connections 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd858e60 0x55b9bd87c440 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd86aaa0 0x55b9bd858450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd86a180 0x55b9bd825ba0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/2325417407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd820600 0x55b9bd825660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98d7400640 0 client.5937 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.555+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 >> 172.21.15.45:0/2325417407 conn(0x55b9bd826760 msgr2=0x55b9bd81f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.559+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 shutdown_connections 2024-04-25T10:21:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.559+0000 7f98fbfd0000 1 -- 172.21.15.45:0/2325417407 wait complete. 2024-04-25T10:21:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.559+0000 7f98fbfd0000 20 client.5937 trim_cache size 0 max 16384 2024-04-25T10:21:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 Processor -- start 2024-04-25T10:21:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- start start 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd826760 0x55b9bd87e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 0x55b9bd81f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd81f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f42034d0 con 0x55b9bd820600 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4202650 con 0x55b9bd86a180 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4201ab0 con 0x55b9bd826760 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 0x55b9bd81f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 0x55b9bd81f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50524/0 (socket says 172.21.15.45:50524) 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 -- 172.21.15.45:0/1166004323 learned_addr learned my addr 172.21.15.45:0/1166004323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98f1e00640 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd826760 0x55b9bd87e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98f2800640 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd81f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 -- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd826760 msgr2=0x55b9bd87e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd826760 0x55b9bd87e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 -- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 msgr2=0x55b9bd81f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd81f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 -- 172.21.15.45:0/1166004323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9bd5f6250 con 0x55b9bd820600 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98f2800640 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd81f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e9400640 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 0x55b9bd81f120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f98dc002240 tx=0x7f98dc020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7121a5ac6b82e5a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e2000640 1 -- 172.21.15.45:0/1166004323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc03b070 con 0x55b9bd820600 2024-04-25T10:21:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e2000640 1 -- 172.21.15.45:0/1166004323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc01a040 con 0x55b9bd820600 2024-04-25T10:21:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98e2000640 1 -- 172.21.15.45:0/1166004323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc014bf0 con 0x55b9bd820600 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1166004323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 msgr2=0x55b9bd81f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 0x55b9bd81f120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f98dc002240 tx=0x7f98dc020040 comp rx=0 tx=0).stop 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1166004323 shutdown_connections 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd81f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd820600 0x55b9bd81f120 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1166004323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd826760 0x55b9bd87e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1166004323 >> 172.21.15.45:0/1166004323 conn(0x55b9bd8585b0 msgr2=0x55b9bd869930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1166004323 shutdown_connections 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1166004323 wait complete. 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 -- start start 2024-04-25T10:21:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.571+0000 7f98fbfd0000 1 Processor -- start 2024-04-25T10:21:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd820600 0x55b9bd8c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 0x55b9bd87cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd87d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd66d230 con 0x55b9bd826760 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4202050 con 0x55b9bd86a180 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9bd626e80 con 0x55b9bd820600 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 10 client.0 _ll_register_callbacks cb 0x55b9bd7ec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 0x55b9bd87cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 0x55b9bd87cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50530/0 (socket says 172.21.15.45:50530) 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 -- 172.21.15.45:0/1872561377 learned_addr learned my addr 172.21.15.45:0/1872561377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98f1e00640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd820600 0x55b9bd8c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98f2800640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd87d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 -- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd820600 msgr2=0x55b9bd8c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd820600 0x55b9bd8c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 -- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 msgr2=0x55b9bd87d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd87d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 -- 172.21.15.45:0/1872561377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9bd602b90 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98e9400640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 0x55b9bd87cc60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f98dc002240 tx=0x7f98dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83fe1e153112e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 1 -- 172.21.15.45:0/1872561377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc03b070 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98dc01a040 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 1 -- 172.21.15.45:0/1872561377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98dc015670 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 1 -- 172.21.15.45:0/1872561377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98dc044050 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 10 client.5943 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55b9bd615010 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 1 -- 172.21.15.45:0/1872561377 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc01a040 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 1 -- 172.21.15.45:0/1872561377 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98dc035040 con 0x55b9bd826760 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 10 client.5943 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 20 client.5943 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd86aaa0 0x55b9bd87d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 4 client.5943 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9bd67b460 con 0x55b9bd86aaa0 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98f1e00640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd86aaa0 0x55b9bd87d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98f1e00640 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd86aaa0 0x55b9bd87d6e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d82a61822e8bfb server_cookie=50ae4820814da67 in_seq=0 out_seq=0 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 10 client.5943 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 1 -- 172.21.15.45:0/1872561377 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55b9bd67b460 con 0x55b9bd86aaa0 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 10 client.5943 handle_command_reply: tid=0 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 client.5943 shutdown 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 msgr2=0x55b9bd87cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 0x55b9bd87cc60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f98dc002240 tx=0x7f98dc00b040 comp rx=0 tx=0).stop 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 shutdown_connections 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b9bd86aaa0 0x55b9bd87d6e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b9bd86a180 0x55b9bd87d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9bd826760 0x55b9bd87cc60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 --2- 172.21.15.45:0/1872561377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b9bd820600 0x55b9bd8c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98d7400640 0 client.5943 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 >> 172.21.15.45:0/1872561377 conn(0x55b9bd8585b0 msgr2=0x55b9bd877770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 shutdown_connections 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 1 -- 172.21.15.45:0/1872561377 wait complete. 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.575+0000 7f98fbfd0000 20 client.5943 trim_cache size 0 max 16384 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2024-04-25T10:21:09.584 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4146253973, 2024-04-25T10:21:09.585 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511630187, 2024-04-25T10:21:09.585 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T10:21:09.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:09.585 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:21:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98d0066f60 msgr2=0x7f98d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98d0066f60 0x7f98d0069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f98e40af000 tx=0x7f98e40a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 msgr2=0x7f98f415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 0x7f98f415ce50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c3c0 tx=0x7f98ec00c880 comp rx=0 tx=0).stop 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 shutdown_connections 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98d0066f60 0x7f98d0069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98f41620b0 0x7f98f415ce50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98f415dd90 0x7f98f415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 --2- 172.21.15.45:0/3514395800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98f415d9b0 0x7f98f4153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 >> 172.21.15.45:0/3514395800 conn(0x7f98f40d07e0 msgr2=0x7f98f406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 shutdown_connections 2024-04-25T10:21:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.579+0000 7f98f3200640 1 -- 172.21.15.45:0/3514395800 wait complete. 2024-04-25T10:21:09.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 Processor -- start 2024-04-25T10:21:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 -- start start 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c880d1820 0x7f4c880d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4c8815e050 0x7f4c880b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c880b13e0 0x7f4c880ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c880b1850 con 0x7f4c880d1820 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c880b1af0 con 0x7f4c8815e050 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880b1dd0 con 0x7f4c880b13e0 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c880d1820 0x7f4c880d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c880d1820 0x7f4c880d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50542/0 (socket says 172.21.15.45:50542) 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 -- 172.21.15.45:0/137990156 learned_addr learned my addr 172.21.15.45:0/137990156 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87400640 1 --2- 172.21.15.45:0/137990156 >> v2:172.21.15.117:3300/0 conn(0x7f4c8815e050 0x7f4c880b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c8d000640 1 --2- 172.21.15.45:0/137990156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c880b13e0 0x7f4c880ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 -- 172.21.15.45:0/137990156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c880b13e0 msgr2=0x7f4c880ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 --2- 172.21.15.45:0/137990156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c880b13e0 0x7f4c880ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 -- 172.21.15.45:0/137990156 >> v2:172.21.15.117:3300/0 conn(0x7f4c8815e050 msgr2=0x7f4c880b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 --2- 172.21.15.45:0/137990156 >> v2:172.21.15.117:3300/0 conn(0x7f4c8815e050 0x7f4c880b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.739+0000 7f4c87e00640 1 -- 172.21.15.45:0/137990156 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880b20b0 con 0x7f4c880d1820 2024-04-25T10:21:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c87e00640 1 --2- 172.21.15.45:0/137990156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880d1820 0x7f4c880d1c00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4c880c5400 tx=0x7f4c7801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da29bde26c3f3b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c86a00640 1 -- 172.21.15.45:0/137990156 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c780b0070 con 0x7f4c880d1820 2024-04-25T10:21:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c86a00640 1 -- 172.21.15.45:0/137990156 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c780ab600 con 0x7f4c880d1820 2024-04-25T10:21:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c86a00640 1 -- 172.21.15.45:0/137990156 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c780ab9d0 con 0x7f4c880d1820 2024-04-25T10:21:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 -- 172.21.15.45:0/137990156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880d1820 msgr2=0x7f4c880d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 --2- 172.21.15.45:0/137990156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880d1820 0x7f4c880d1c00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4c880c5400 tx=0x7f4c7801c260 comp rx=0 tx=0).stop 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 -- 172.21.15.45:0/137990156 shutdown_connections 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 --2- 172.21.15.45:0/137990156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c880b13e0 0x7f4c880ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 --2- 172.21.15.45:0/137990156 >> v2:172.21.15.117:3300/0 conn(0x7f4c8815e050 0x7f4c880b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 --2- 172.21.15.45:0/137990156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880d1820 0x7f4c880d1c00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 -- 172.21.15.45:0/137990156 >> 172.21.15.45:0/137990156 conn(0x7f4c8809a9d0 msgr2=0x7f4c8809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 -- 172.21.15.45:0/137990156 shutdown_connections 2024-04-25T10:21:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 -- 172.21.15.45:0/137990156 wait complete. 2024-04-25T10:21:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 Processor -- start 2024-04-25T10:21:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.743+0000 7f4c8f000640 1 -- start start 2024-04-25T10:21:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 0x7f4c88141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c880d1820 0x7f4c881537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c8815e050 0x7f4c880b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880cfdf0 con 0x7f4c880b13e0 2024-04-25T10:21:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880cfc70 con 0x7f4c880d1820 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880cff70 con 0x7f4c8815e050 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 0x7f4c88141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 0x7f4c88141510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50550/0 (socket says 172.21.15.45:50550) 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 -- 172.21.15.45:0/3282466511 learned_addr learned my addr 172.21.15.45:0/3282466511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87400640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c880d1820 0x7f4c881537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8d000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c8815e050 0x7f4c880b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 -- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c8815e050 msgr2=0x7f4c880b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c8815e050 0x7f4c880b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 -- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c880d1820 msgr2=0x7f4c881537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c880d1820 0x7f4c881537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 -- 172.21.15.45:0/3282466511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880b6cb0 con 0x7f4c880b13e0 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8d000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c8815e050 0x7f4c880b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c87e00640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 0x7f4c88141510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4c780ae040 tx=0x7f4c78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c780b0040 con 0x7f4c880b13e0 2024-04-25T10:21:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c780b9070 con 0x7f4c880b13e0 2024-04-25T10:21:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c780b4680 con 0x7f4c880b13e0 2024-04-25T10:21:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880b6f90 con 0x7f4c880b13e0 2024-04-25T10:21:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c880b7550 con 0x7f4c880b13e0 2024-04-25T10:21:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.747+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c880c7d10 con 0x7f4c880b13e0 2024-04-25T10:21:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.751+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c780b4820 con 0x7f4c880b13e0 2024-04-25T10:21:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.751+0000 7f4c7fe00640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c68066f60 0x7f4c68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.751+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c7811ec50 con 0x7f4c880b13e0 2024-04-25T10:21:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.751+0000 7f4c87400640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c68066f60 0x7f4c68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.751+0000 7f4c87400640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c68066f60 0x7f4c68069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4c8813e7c0 tx=0x7f4c74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:09.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.755+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c78016110 con 0x7f4c880b13e0 2024-04-25T10:21:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c8804ef30 con 0x7f4c880b13e0 2024-04-25T10:21:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c7fe00640 1 -- 172.21.15.45:0/3282466511 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f4c780efd70 con 0x7f4c880b13e0 2024-04-25T10:21:09.954 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:09.954 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:09.954 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c68066f60 msgr2=0x7f4c68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c68066f60 0x7f4c68069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4c8813e7c0 tx=0x7f4c74008040 comp rx=0 tx=0).stop 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 msgr2=0x7f4c88141510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 0x7f4c88141510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4c780ae040 tx=0x7f4c78019000 comp rx=0 tx=0).stop 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 shutdown_connections 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c68066f60 0x7f4c68069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c8815e050 0x7f4c880b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c880d1820 0x7f4c881537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 --2- 172.21.15.45:0/3282466511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c880b13e0 0x7f4c88141510 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 >> 172.21.15.45:0/3282466511 conn(0x7f4c8809a9d0 msgr2=0x7f4c880d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 shutdown_connections 2024-04-25T10:21:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:09.951+0000 7f4c8f000640 1 -- 172.21.15.45:0/3282466511 wait complete. 2024-04-25T10:21:09.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 4146253973 2024-04-25T10:21:10.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 Processor -- start 2024-04-25T10:21:10.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 -- start start 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73700aef80 0x7f737015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7370070350 0x7f737015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737015b080 con 0x7f737015bac0 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737015adb0 con 0x7f7370070350 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7376e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737015eb60 con 0x7f73700aef80 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7367400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7370070350 0x7f737015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7367400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7370070350 0x7f737015b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50552/0 (socket says 172.21.15.45:50552) 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f7367400640 1 -- 172.21.15.45:0/329562494 learned_addr learned my addr 172.21.15.45:0/329562494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.115+0000 7f736fe00640 1 --2- 172.21.15.45:0/329562494 >> v2:172.21.15.117:3300/0 conn(0x7f73700aef80 0x7f737015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7367400640 1 -- 172.21.15.45:0/329562494 >> v2:172.21.15.117:3300/0 conn(0x7f73700aef80 msgr2=0x7f737015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7367400640 1 --2- 172.21.15.45:0/329562494 >> v2:172.21.15.117:3300/0 conn(0x7f73700aef80 0x7f737015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7367400640 1 -- 172.21.15.45:0/329562494 >> v1:172.21.15.117:6789/0 conn(0x7f737015bac0 legacy=0x7f73700b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7367400640 1 -- 172.21.15.45:0/329562494 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737015ee40 con 0x7f7370070350 2024-04-25T10:21:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7367400640 1 --2- 172.21.15.45:0/329562494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f737015b580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f737006e220 tx=0x7f735801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8687536222cb95dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f736f400640 1 -- 172.21.15.45:0/329562494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580aa660 con 0x7f7370070350 2024-04-25T10:21:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f736f400640 1 -- 172.21.15.45:0/329562494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73580af070 con 0x7f7370070350 2024-04-25T10:21:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f736f400640 1 -- 172.21.15.45:0/329562494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580aaa20 con 0x7f7370070350 2024-04-25T10:21:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 -- 172.21.15.45:0/329562494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 msgr2=0x7f737015b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 --2- 172.21.15.45:0/329562494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f737015b580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f737006e220 tx=0x7f735801c520 comp rx=0 tx=0).stop 2024-04-25T10:21:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 -- 172.21.15.45:0/329562494 shutdown_connections 2024-04-25T10:21:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 --2- 172.21.15.45:0/329562494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f737015b580 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 --2- 172.21.15.45:0/329562494 >> v2:172.21.15.117:3300/0 conn(0x7f73700aef80 0x7f737015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 -- 172.21.15.45:0/329562494 >> 172.21.15.45:0/329562494 conn(0x7f73700669a0 msgr2=0x7f7370066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.119+0000 7f7376e00640 1 -- 172.21.15.45:0/329562494 shutdown_connections 2024-04-25T10:21:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- 172.21.15.45:0/329562494 wait complete. 2024-04-25T10:21:10.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 Processor -- start 2024-04-25T10:21:10.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- start start 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f7370138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73700aef80 0x7f7370139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f737015bac0 0x7f737015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737006c250 con 0x7f7370070350 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737006c0d0 con 0x7f73700aef80 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737006c3d0 con 0x7f737015bac0 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f7370138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f7370138c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50564/0 (socket says 172.21.15.45:50564) 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 -- 172.21.15.45:0/1946082932 learned_addr learned my addr 172.21.15.45:0/1946082932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7367400640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73700aef80 0x7f7370139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7374e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f737015bac0 0x7f737015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 -- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f737015bac0 msgr2=0x7f737015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f737015bac0 0x7f737015bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 -- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73700aef80 msgr2=0x7f7370139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73700aef80 0x7f7370139390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 -- 172.21.15.45:0/1946082932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370139de0 con 0x7f7370070350 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7367400640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73700aef80 0x7f7370139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736fe00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f7370138c60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7360010f50 tx=0x7f736000e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736cc00640 1 -- 172.21.15.45:0/1946082932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000b2d0 con 0x7f7370070350 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736cc00640 1 -- 172.21.15.45:0/1946082932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736001b070 con 0x7f7370070350 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737013a0c0 con 0x7f7370070350 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f736cc00640 1 -- 172.21.15.45:0/1946082932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360009760 con 0x7f7370070350 2024-04-25T10:21:10.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.123+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737013a680 con 0x7f7370070350 2024-04-25T10:21:10.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f736cc00640 1 -- 172.21.15.45:0/1946082932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7360014050 con 0x7f7370070350 2024-04-25T10:21:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f736cc00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f734c066f90 0x7f734c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f736cc00640 1 -- 172.21.15.45:0/1946082932 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7360087760 con 0x7f7370070350 2024-04-25T10:21:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f7367400640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f734c066f90 0x7f734c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f737a3af000 1 Processor -- start 2024-04-25T10:21:10.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f737a3af000 1 -- start start 2024-04-25T10:21:10.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.127+0000 7f7367400640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f734c066f90 0x7f734c069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73580ae000 tx=0x7f73580a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb789a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb784dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb784e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb7699f00 con 0x563bb784dff0 2024-04-25T10:21:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb76224a0 con 0x563bb789abe0 2024-04-25T10:21:10.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bb7695fa0 con 0x563bb789a2c0 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb784e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb784e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50576/0 (socket says 172.21.15.45:50576) 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 -- 172.21.15.45:0/165868605 learned_addr learned my addr 172.21.15.45:0/165868605 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7367400640 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb784dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f736fe00640 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb789a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 -- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 msgr2=0x563bb789a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb789a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 -- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 msgr2=0x563bb784dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb784dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 -- 172.21.15.45:0/165868605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb761fd00 con 0x563bb784dff0 2024-04-25T10:21:10.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7374e00640 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb784e3d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f73680014d0 tx=0x7f7368001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b9b85891075aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7364c00640 1 -- 172.21.15.45:0/165868605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368019070 con 0x563bb784dff0 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7364c00640 1 -- 172.21.15.45:0/165868605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73680052e0 con 0x563bb784dff0 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f7364c00640 1 -- 172.21.15.45:0/165868605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368003820 con 0x563bb784dff0 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- 172.21.15.45:0/165868605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 msgr2=0x563bb784e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb784e3d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f73680014d0 tx=0x7f7368001990 comp rx=0 tx=0).stop 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- 172.21.15.45:0/165868605 shutdown_connections 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb784e3d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb784dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- 172.21.15.45:0/165868605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb789a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- 172.21.15.45:0/165868605 >> 172.21.15.45:0/165868605 conn(0x563bb7876e10 msgr2=0x563bb7879200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- 172.21.15.45:0/165868605 shutdown_connections 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- 172.21.15.45:0/165868605 wait complete. 2024-04-25T10:21:10.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 -- start start 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 Processor -- start 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb784dff0 0x563bb78a1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.131+0000 7f737a3af000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 0x563bb78a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb787b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb769e530 con 0x563bb789a2c0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb7654ae0 con 0x563bb789abe0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bb7658e10 con 0x563bb784dff0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 10 client.0 _ll_register_callbacks cb 0x563bb7837750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 0x563bb78a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 0x563bb78a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50578/0 (socket says 172.21.15.45:50578) 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 -- 172.21.15.45:0/406805925 learned_addr learned my addr 172.21.15.45:0/406805925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f736fe00640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb784dff0 0x563bb78a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7374e00640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb787b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 -- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb784dff0 msgr2=0x563bb78a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb784dff0 0x563bb78a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 -- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 msgr2=0x563bb787b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb787b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 -- 172.21.15.45:0/406805925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bb763ca10 con 0x563bb789a2c0 2024-04-25T10:21:10.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f7367400640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 0x563bb78a1b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f73580ac040 tx=0x7f735801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7578acacb5d73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:10.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 1 -- 172.21.15.45:0/406805925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580cc060 con 0x563bb789a2c0 2024-04-25T10:21:10.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73680052e0 con 0x563bb789a2c0 2024-04-25T10:21:10.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 1 -- 172.21.15.45:0/406805925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73580be8e0 con 0x563bb789a2c0 2024-04-25T10:21:10.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 1 -- 172.21.15.45:0/406805925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73580d2050 con 0x563bb789a2c0 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 10 client.5961 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x563bb7694ef0 con 0x563bb789a2c0 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 1 -- 172.21.15.45:0/406805925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73680052e0 con 0x563bb789a2c0 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 1 -- 172.21.15.45:0/406805925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73580d5020 con 0x563bb789a2c0 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 10 client.5961 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 10 client.5961 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 20 client.5961 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb787be40 0x563bb787c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 4 client.5961 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bb7635b20 con 0x563bb787be40 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f736fe00640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb787be40 0x563bb787c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f736fe00640 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb787be40 0x563bb787c220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d422d2a6a3df81b7 server_cookie=a4dc485d60b7382d in_seq=0 out_seq=0 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 10 client.5961 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:10.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 1 -- 172.21.15.45:0/406805925 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563bb7635b20 con 0x563bb787be40 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f735d600640 10 client.5961 handle_command_reply: tid=0 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 client.5961 shutdown 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 msgr2=0x563bb78a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 0x563bb78a1b60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f73580ac040 tx=0x7f735801ad60 comp rx=0 tx=0).stop 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 shutdown_connections 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.135+0000 7f737a3af000 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb787be40 0x563bb787c220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789abe0 0x563bb787b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789a2c0 0x563bb78a1b60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f735d600640 0 client.5961 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 1 --2- 172.21.15.45:0/406805925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb784dff0 0x563bb78a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 >> 172.21.15.45:0/406805925 conn(0x563bb7876e10 msgr2=0x563bb7848da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 shutdown_connections 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 1 -- 172.21.15.45:0/406805925 wait complete. 2024-04-25T10:21:10.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.139+0000 7f737a3af000 20 client.5961 trim_cache size 0 max 16384 2024-04-25T10:21:10.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 Processor -- start 2024-04-25T10:21:10.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- start start 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb784dff0 0x563bb78f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb78f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 0x563bb78f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370158fd0 con 0x563bb789abe0 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701583d0 con 0x563bb784dff0 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73701589d0 con 0x563bb789a2c0 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 0x563bb78f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 0x563bb78f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50592/0 (socket says 172.21.15.45:50592) 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 -- 172.21.15.45:0/3166293494 learned_addr learned my addr 172.21.15.45:0/3166293494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7367400640 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb78f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f736fe00640 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb784dff0 0x563bb78f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 -- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 msgr2=0x563bb78f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb78f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 -- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb784dff0 msgr2=0x563bb78f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb784dff0 0x563bb78f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 -- 172.21.15.45:0/3166293494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb789aa10 con 0x563bb789abe0 2024-04-25T10:21:10.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f7374e00640 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 0x563bb78f7590 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7368009010 tx=0x7f736800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b3c53e4de7c192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f735fe00640 1 -- 172.21.15.45:0/3166293494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368019040 con 0x563bb789abe0 2024-04-25T10:21:10.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f735fe00640 1 -- 172.21.15.45:0/3166293494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368024070 con 0x563bb789abe0 2024-04-25T10:21:10.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f735fe00640 1 -- 172.21.15.45:0/3166293494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368006a20 con 0x563bb789abe0 2024-04-25T10:21:10.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- 172.21.15.45:0/3166293494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 msgr2=0x563bb78f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 0x563bb78f7590 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7368009010 tx=0x7f736800a000 comp rx=0 tx=0).stop 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- 172.21.15.45:0/3166293494 shutdown_connections 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb789abe0 0x563bb78f7590 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789a2c0 0x563bb78f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 --2- 172.21.15.45:0/3166293494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb784dff0 0x563bb78f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- 172.21.15.45:0/3166293494 >> 172.21.15.45:0/3166293494 conn(0x563bb7876e10 msgr2=0x563bb7847e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- 172.21.15.45:0/3166293494 shutdown_connections 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- 172.21.15.45:0/3166293494 wait complete. 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 -- start start 2024-04-25T10:21:10.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.151+0000 7f737a3af000 1 Processor -- start 2024-04-25T10:21:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb78fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789a2c0 0x563bb78fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789abe0 0x563bb78a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb76537c0 con 0x563bb784dff0 2024-04-25T10:21:10.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb7666b90 con 0x563bb789a2c0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bb7699210 con 0x563bb789abe0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 10 client.0 _ll_register_callbacks cb 0x7f7368007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb78fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb78fb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50608/0 (socket says 172.21.15.45:50608) 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 -- 172.21.15.45:0/2452113409 learned_addr learned my addr 172.21.15.45:0/2452113409 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f7374e00640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789abe0 0x563bb78a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f7367400640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789a2c0 0x563bb78fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 -- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789abe0 msgr2=0x563bb78a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789abe0 0x563bb78a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 -- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789a2c0 msgr2=0x563bb78fb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789a2c0 0x563bb78fb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 -- 172.21.15.45:0/2452113409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bb7659790 con 0x563bb784dff0 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f7374e00640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789abe0 0x563bb78a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:10.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f7367400640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789a2c0 0x563bb78fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f736fe00640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb78fb2e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f736000b0a0 tx=0x7f736000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3129b4468d8063d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 1 -- 172.21.15.45:0/2452113409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360060770 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 1 -- 172.21.15.45:0/2452113409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7360056360 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7368024070 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 1 -- 172.21.15.45:0/2452113409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f736007b050 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 10 client.5967 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x563bb7661da0 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 1 -- 172.21.15.45:0/2452113409 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368024070 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 1 -- 172.21.15.45:0/2452113409 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73600546d0 con 0x563bb784dff0 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 10 client.5967 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 10 client.5967 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 20 client.5967 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:10.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb79021b0 0x563bb79045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 4 client.5967 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 4146253973}] 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "damage rm", "damage_id": 4146253973}) v1 -- 0x563bb76a7440 con 0x563bb79021b0 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f7367400640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb79021b0 0x563bb79045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f7367400640 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb79021b0 0x563bb79045a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3884d4344efcf1f3 server_cookie=19434e10be679f7b in_seq=0 out_seq=0 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 10 client.5967 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 1 -- 172.21.15.45:0/2452113409 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x563bb76a7440 con 0x563bb79021b0 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f735d600640 10 client.5967 handle_command_reply: tid=0 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 client.5967 shutdown 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 msgr2=0x563bb78fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb78fb2e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f736000b0a0 tx=0x7f736000e040 comp rx=0 tx=0).stop 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 shutdown_connections 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563bb79021b0 0x563bb79045a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.155+0000 7f737a3af000 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563bb789abe0 0x563bb78a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f737a3af000 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bb789a2c0 0x563bb78fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f737a3af000 1 --2- 172.21.15.45:0/2452113409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563bb784dff0 0x563bb78fb2e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 >> 172.21.15.45:0/2452113409 conn(0x563bb7876e10 msgr2=0x563bb7848690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f735d600640 0 client.5967 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 shutdown_connections 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f737a3af000 1 -- 172.21.15.45:0/2452113409 wait complete. 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f737a3af000 20 client.5967 trim_cache size 0 max 16384 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f734c066f90 msgr2=0x7f734c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f734c066f90 0x7f734c069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73580ae000 tx=0x7f73580a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 msgr2=0x7f7370138c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f7370138c60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7360010f50 tx=0x7f736000e550 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 shutdown_connections 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f734c066f90 0x7f734c069450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f737015bac0 0x7f737015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73700aef80 0x7f7370139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 --2- 172.21.15.45:0/1946082932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7370070350 0x7f7370138c60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 >> 172.21.15.45:0/1946082932 conn(0x7f73700669a0 msgr2=0x7f7370066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 shutdown_connections 2024-04-25T10:21:10.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.159+0000 7f7376e00640 1 -- 172.21.15.45:0/1946082932 wait complete. 2024-04-25T10:21:10.179 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:21:10.180 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.315+0000 7f2597e00640 1 Processor -- start 2024-04-25T10:21:10.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2597e00640 1 -- start start 2024-04-25T10:21:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2597e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2598151930 0x7f2598070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2597e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980711f0 0x7f25980b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2597e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259815bf50 con 0x7f25980b2090 2024-04-25T10:21:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2597e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980b2500 con 0x7f2598151930 2024-04-25T10:21:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2597e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598151d10 con 0x7f25980711f0 2024-04-25T10:21:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2598151930 0x7f2598070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2598151930 0x7f2598070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50624/0 (socket says 172.21.15.45:50624) 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 -- 172.21.15.45:0/2659931445 learned_addr learned my addr 172.21.15.45:0/2659931445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f258fe00640 1 --2- 172.21.15.45:0/2659931445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980711f0 0x7f25980b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 -- 172.21.15.45:0/2659931445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980711f0 msgr2=0x7f25980b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 --2- 172.21.15.45:0/2659931445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980711f0 0x7f25980b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 -- 172.21.15.45:0/2659931445 >> v1:172.21.15.45:6789/0 conn(0x7f25980b2090 legacy=0x7f259815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.319+0000 7f2596a00640 1 -- 172.21.15.45:0/2659931445 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259815b0a0 con 0x7f2598151930 2024-04-25T10:21:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2596a00640 1 --2- 172.21.15.45:0/2659931445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2598151930 0x7f2598070cb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2588009870 tx=0x7f258801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcd7e2470f6f879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2596000640 1 -- 172.21.15.45:0/2659931445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880b0070 con 0x7f2598151930 2024-04-25T10:21:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2596000640 1 -- 172.21.15.45:0/2659931445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25880ab600 con 0x7f2598151930 2024-04-25T10:21:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2596000640 1 -- 172.21.15.45:0/2659931445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880ab9d0 con 0x7f2598151930 2024-04-25T10:21:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 -- 172.21.15.45:0/2659931445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2598151930 msgr2=0x7f2598070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 --2- 172.21.15.45:0/2659931445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2598151930 0x7f2598070cb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2588009870 tx=0x7f258801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 -- 172.21.15.45:0/2659931445 shutdown_connections 2024-04-25T10:21:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 --2- 172.21.15.45:0/2659931445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980711f0 0x7f25980b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 --2- 172.21.15.45:0/2659931445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2598151930 0x7f2598070cb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 -- 172.21.15.45:0/2659931445 >> 172.21.15.45:0/2659931445 conn(0x7f259806ac90 msgr2=0x7f259806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 -- 172.21.15.45:0/2659931445 shutdown_connections 2024-04-25T10:21:10.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 -- 172.21.15.45:0/2659931445 wait complete. 2024-04-25T10:21:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 Processor -- start 2024-04-25T10:21:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.323+0000 7f2597e00640 1 -- start start 2024-04-25T10:21:10.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 0x7f2598150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980b2090 0x7f25981494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2598151930 0x7f259815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b5100 con 0x7f25980711f0 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b3c60 con 0x7f2598151930 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598165710 con 0x7f25980b2090 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 0x7f2598150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 0x7f2598150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50640/0 (socket says 172.21.15.45:50640) 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 -- 172.21.15.45:0/502843292 learned_addr learned my addr 172.21.15.45:0/502843292 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f258fe00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980b2090 0x7f25981494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597400640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2598151930 0x7f259815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 -- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980b2090 msgr2=0x7f25981494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980b2090 0x7f25981494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 -- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2598151930 msgr2=0x7f259815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2598151930 0x7f259815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 -- 172.21.15.45:0/502843292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25981664f0 con 0x7f25980711f0 2024-04-25T10:21:10.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2596a00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 0x7f2598150870 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f25880ae040 tx=0x7f258801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880ba020 con 0x7f25980711f0 2024-04-25T10:21:10.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25880b6620 con 0x7f25980711f0 2024-04-25T10:21:10.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25880b0040 con 0x7f25980711f0 2024-04-25T10:21:10.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2598166790 con 0x7f25980711f0 2024-04-25T10:21:10.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2598166d50 con 0x7f25980711f0 2024-04-25T10:21:10.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.327+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2598074520 con 0x7f25980711f0 2024-04-25T10:21:10.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.331+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25880bd020 con 0x7f25980711f0 2024-04-25T10:21:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.331+0000 7f258ea00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2574066f60 0x7f2574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.331+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f258811f160 con 0x7f25980711f0 2024-04-25T10:21:10.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.331+0000 7f258fe00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2574066f60 0x7f2574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:10.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.331+0000 7f258fe00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2574066f60 0x7f2574069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f259815c850 tx=0x7f2580008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:10.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.335+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25880c4030 con 0x7f25980711f0 2024-04-25T10:21:10.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.531+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2598149ec0 con 0x7f25980711f0 2024-04-25T10:21:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.531+0000 7f258ea00640 1 -- 172.21.15.45:0/502843292 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f2588016150 con 0x7f25980711f0 2024-04-25T10:21:10.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:10.535 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2574066f60 msgr2=0x7f2574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2574066f60 0x7f2574069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f259815c850 tx=0x7f2580008040 comp rx=0 tx=0).stop 2024-04-25T10:21:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 msgr2=0x7f2598150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 0x7f2598150870 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f25880ae040 tx=0x7f258801ad10 comp rx=0 tx=0).stop 2024-04-25T10:21:10.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 shutdown_connections 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2574066f60 0x7f2574069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2598151930 0x7f259815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25980b2090 0x7f25981494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 --2- 172.21.15.45:0/502843292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25980711f0 0x7f2598150870 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 >> 172.21.15.45:0/502843292 conn(0x7f259806ac90 msgr2=0x7f259806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 shutdown_connections 2024-04-25T10:21:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10.535+0000 7f2597e00640 1 -- 172.21.15.45:0/502843292 wait complete. 2024-04-25T10:21:10.556 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:21:10.707 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:10.707 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:10.723 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:10.723 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:10.723 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:21:10.723 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:21:10.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:10.723 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> import json 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:21:10.734 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2024-04-25T10:21:10.735 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:21:10.847 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:10.847 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mode": 16893, 2024-04-25T10:21:10.847 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ino": 1099511627777, 2024-04-25T10:21:10.847 INFO:teuthology.orchestra.run.smithi045.stdout: "st_dev": 44, 2024-04-25T10:21:10.847 INFO:teuthology.orchestra.run.smithi045.stdout: "st_nlink": 16, 2024-04-25T10:21:10.847 INFO:teuthology.orchestra.run.smithi045.stdout: "st_uid": 1000, 2024-04-25T10:21:10.848 INFO:teuthology.orchestra.run.smithi045.stdout: "st_gid": 1271, 2024-04-25T10:21:10.848 INFO:teuthology.orchestra.run.smithi045.stdout: "st_size": 18, 2024-04-25T10:21:10.848 INFO:teuthology.orchestra.run.smithi045.stdout: "st_atime": 1714040455.0714786, 2024-04-25T10:21:10.848 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mtime": 1714040462.6874042, 2024-04-25T10:21:10.848 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ctime": 1714040462.6874042 2024-04-25T10:21:10.848 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:21:10.856 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T10:21:10.857 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:21:10.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:11.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 Processor -- start 2024-04-25T10:21:11.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 -- start start 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4cc4072200 0x7f4cc40725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cc4072b20 0x7f4cc40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc40b1330 con 0x7f4cc40b0b40 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc40cbee0 con 0x7f4cc4072b20 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cc9200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc40b1610 con 0x7f4cc4072200 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.039+0000 7f4cbbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cc4072b20 0x7f4cc40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cc4072b20 0x7f4cc40b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50642/0 (socket says 172.21.15.45:50642) 2024-04-25T10:21:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 -- 172.21.15.45:0/1202228887 learned_addr learned my addr 172.21.15.45:0/1202228887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cc1e00640 1 --2- 172.21.15.45:0/1202228887 >> v2:172.21.15.117:3300/0 conn(0x7f4cc4072200 0x7f4cc40725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 -- 172.21.15.45:0/1202228887 >> v2:172.21.15.117:3300/0 conn(0x7f4cc4072200 msgr2=0x7f4cc40725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 --2- 172.21.15.45:0/1202228887 >> v2:172.21.15.117:3300/0 conn(0x7f4cc4072200 0x7f4cc40725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 -- 172.21.15.45:0/1202228887 >> v1:172.21.15.45:6789/0 conn(0x7f4cc40b0b40 legacy=0x7f4cc40cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 -- 172.21.15.45:0/1202228887 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc40b18f0 con 0x7f4cc4072b20 2024-04-25T10:21:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cbbe00640 1 --2- 172.21.15.45:0/1202228887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072b20 0x7f4cc40b0600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40710f0 tx=0x7f4cac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259a686ae2115827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cc1400640 1 -- 172.21.15.45:0/1202228887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0ab430 con 0x7f4cc4072b20 2024-04-25T10:21:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cc1400640 1 -- 172.21.15.45:0/1202228887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac0b0070 con 0x7f4cc4072b20 2024-04-25T10:21:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cc1400640 1 -- 172.21.15.45:0/1202228887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0ab7f0 con 0x7f4cc4072b20 2024-04-25T10:21:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cc9200640 1 -- 172.21.15.45:0/1202228887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072b20 msgr2=0x7f4cc40b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.043+0000 7f4cc9200640 1 --2- 172.21.15.45:0/1202228887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072b20 0x7f4cc40b0600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40710f0 tx=0x7f4cac01c520 comp rx=0 tx=0).stop 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- 172.21.15.45:0/1202228887 shutdown_connections 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 --2- 172.21.15.45:0/1202228887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072b20 0x7f4cc40b0600 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 --2- 172.21.15.45:0/1202228887 >> v2:172.21.15.117:3300/0 conn(0x7f4cc4072200 0x7f4cc40725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- 172.21.15.45:0/1202228887 >> 172.21.15.45:0/1202228887 conn(0x7f4cc40b1d50 msgr2=0x7f4cc40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- 172.21.15.45:0/1202228887 shutdown_connections 2024-04-25T10:21:11.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- 172.21.15.45:0/1202228887 wait complete. 2024-04-25T10:21:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 Processor -- start 2024-04-25T10:21:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- start start 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 0x7f4cc415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc4072b20 0x7f4cc40c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc40b0b40 0x7f4cc40c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40cd410 con 0x7f4cc4072200 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40cd290 con 0x7f4cc4072b20 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40cd590 con 0x7f4cc40b0b40 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 0x7f4cc415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 0x7f4cc415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50654/0 (socket says 172.21.15.45:50654) 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 -- 172.21.15.45:0/2581603163 learned_addr learned my addr 172.21.15.45:0/2581603163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cbbe00640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc4072b20 0x7f4cc40c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc2800640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc40b0b40 0x7f4cc40c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 -- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc40b0b40 msgr2=0x7f4cc40c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc40b0b40 0x7f4cc40c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 -- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc4072b20 msgr2=0x7f4cc40c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc4072b20 0x7f4cc40c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 -- 172.21.15.45:0/2581603163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4165d50 con 0x7f4cc4072200 2024-04-25T10:21:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc2800640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc40b0b40 0x7f4cc40c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:11.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.047+0000 7f4cc1e00640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 0x7f4cc415ab80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e870 tx=0x7f4cb400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4017070 con 0x7f4cc4072200 2024-04-25T10:21:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb400c940 con 0x7f4cc4072200 2024-04-25T10:21:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400ccf0 con 0x7f4cc4072200 2024-04-25T10:21:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc4166030 con 0x7f4cc4072200 2024-04-25T10:21:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc40c73e0 con 0x7f4cc4072200 2024-04-25T10:21:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc404ef30 con 0x7f4cc4072200 2024-04-25T10:21:11.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cb401a070 con 0x7f4cc4072200 2024-04-25T10:21:11.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cba000640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ca0066f60 0x7f4ca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cb40852b0 con 0x7f4cc4072200 2024-04-25T10:21:11.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cbbe00640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ca0066f60 0x7f4ca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.051+0000 7f4cbbe00640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ca0066f60 0x7f4ca0069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0af000 tx=0x7f4cac0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.055+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cb4055bd0 con 0x7f4cc4072200 2024-04-25T10:21:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.251+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cc40c3830 con 0x7f4cc4072200 2024-04-25T10:21:11.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.251+0000 7f4cba000640 1 -- 172.21.15.45:0/2581603163 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f4cb4059c70 con 0x7f4cc4072200 2024-04-25T10:21:11.255 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:11.256 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:20:50.288582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:11.256 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ca0066f60 msgr2=0x7f4ca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ca0066f60 0x7f4ca0069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0af000 tx=0x7f4cac0a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 msgr2=0x7f4cc415ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 0x7f4cc415ab80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e870 tx=0x7f4cb400ec70 comp rx=0 tx=0).stop 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 shutdown_connections 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ca0066f60 0x7f4ca0069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc40b0b40 0x7f4cc40c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc4072b20 0x7f4cc40c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 --2- 172.21.15.45:0/2581603163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cc4072200 0x7f4cc415ab80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 >> 172.21.15.45:0/2581603163 conn(0x7f4cc40b1d50 msgr2=0x7f4cc40ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 shutdown_connections 2024-04-25T10:21:11.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.255+0000 7f4cc9200640 1 -- 172.21.15.45:0/2581603163 wait complete. 2024-04-25T10:21:11.278 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:21:11.415 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:11.415 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": -2 2024-04-25T10:21:11.415 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:11.431 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:21:11.431 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:21:11.431 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:11.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:21:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.571+0000 7fe9d9000640 1 Processor -- start 2024-04-25T10:21:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.571+0000 7fe9d9000640 1 -- start start 2024-04-25T10:21:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.571+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d4159540 0x7fe9d40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.571+0000 7fe9d9000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d4159920 con 0x7fe9d40721f0 2024-04-25T10:21:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.571+0000 7fe9d9000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d415c2f0 con 0x7fe9d40aebd0 2024-04-25T10:21:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.571+0000 7fe9d9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40af030 con 0x7fe9d4159540 2024-04-25T10:21:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d2800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe9d40721f0 0x7fe9d4070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36514/0 (socket says 172.21.15.45:36514) 2024-04-25T10:21:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d2800640 1 -- 172.21.15.45:0/93592089 learned_addr learned my addr 172.21.15.45:0/93592089 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/93592089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d4159540 0x7fe9d40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1400640 1 -- 172.21.15.45:0/93592089 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248186343 0 0) 0x7fe9d4159920 con 0x7fe9d40721f0 2024-04-25T10:21:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1400640 1 -- 172.21.15.45:0/93592089 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b8003530 con 0x7fe9d40721f0 2024-04-25T10:21:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1e00640 1 -- 172.21.15.45:0/93592089 >> v1:172.21.15.117:6789/0 conn(0x7fe9d40aebd0 legacy=0x7fe9d4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1400640 1 -- 172.21.15.45:0/93592089 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094053447 0 0) 0x7fe9d415c2f0 con 0x7fe9d40aebd0 2024-04-25T10:21:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1e00640 1 -- 172.21.15.45:0/93592089 >> v1:172.21.15.45:6789/0 conn(0x7fe9d40721f0 legacy=0x7fe9d4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1e00640 1 -- 172.21.15.45:0/93592089 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d415c5d0 con 0x7fe9d4159540 2024-04-25T10:21:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.575+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/93592089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d4159540 0x7fe9d40ae690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40097f0 tx=0x7fe9c401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa60b96ebedbe165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d1400640 1 -- 172.21.15.45:0/93592089 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40aa610 con 0x7fe9d4159540 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d1400640 1 -- 172.21.15.45:0/93592089 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c40af070 con 0x7fe9d4159540 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d1400640 1 -- 172.21.15.45:0/93592089 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40aa9d0 con 0x7fe9d4159540 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- 172.21.15.45:0/93592089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d4159540 msgr2=0x7fe9d40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 --2- 172.21.15.45:0/93592089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d4159540 0x7fe9d40ae690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40097f0 tx=0x7fe9c401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- 172.21.15.45:0/93592089 shutdown_connections 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 --2- 172.21.15.45:0/93592089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d4159540 0x7fe9d40ae690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- 172.21.15.45:0/93592089 >> 172.21.15.45:0/93592089 conn(0x7fe9d40669a0 msgr2=0x7fe9d4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- 172.21.15.45:0/93592089 shutdown_connections 2024-04-25T10:21:11.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- 172.21.15.45:0/93592089 wait complete. 2024-04-25T10:21:11.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 Processor -- start 2024-04-25T10:21:11.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- start start 2024-04-25T10:21:11.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d40721f0 0x7fe9d414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 0x7fe9d40afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d4159540 0x7fe9d414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d4169940 con 0x7fe9d40aebd0 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d406c0d0 con 0x7fe9d4159540 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d4169ac0 con 0x7fe9d40721f0 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 0x7fe9d40afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 0x7fe9d40afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50664/0 (socket says 172.21.15.45:50664) 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d1e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d40721f0 0x7fe9d414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9cbe00640 1 -- 172.21.15.45:0/2164014985 learned_addr learned my addr 172.21.15.45:0/2164014985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.579+0000 7fe9d2800640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d4159540 0x7fe9d414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9cbe00640 1 -- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d40721f0 msgr2=0x7fe9d414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d40721f0 0x7fe9d414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9cbe00640 1 -- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d4159540 msgr2=0x7fe9d414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d4159540 0x7fe9d414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9cbe00640 1 -- 172.21.15.45:0/2164014985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40b2c30 con 0x7fe9d40aebd0 2024-04-25T10:21:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 0x7fe9d40afb70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c480 tx=0x7fe9bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9ca000640 1 -- 172.21.15.45:0/2164014985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc005480 con 0x7fe9d40aebd0 2024-04-25T10:21:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9ca000640 1 -- 172.21.15.45:0/2164014985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc003600 con 0x7fe9d40aebd0 2024-04-25T10:21:11.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9ca000640 1 -- 172.21.15.45:0/2164014985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0039f0 con 0x7fe9d40aebd0 2024-04-25T10:21:11.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d40b03a0 con 0x7fe9d40aebd0 2024-04-25T10:21:11.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d40b3490 con 0x7fe9d40aebd0 2024-04-25T10:21:11.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9ca000640 1 -- 172.21.15.45:0/2164014985 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9bc003b90 con 0x7fe9d40aebd0 2024-04-25T10:21:11.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9ca000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.583+0000 7fe9ca000640 1 -- 172.21.15.45:0/2164014985 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9bc084bb0 con 0x7fe9d40aebd0 2024-04-25T10:21:11.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 Processor -- start 2024-04-25T10:21:11.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- start start 2024-04-25T10:21:11.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40ae000 tx=0x7fe9c40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd205c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cd1e718a0 con 0x563cd20a5970 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cd1ea9d10 con 0x563cd20a6290 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cd1e322d0 con 0x563cd205bcd0 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20a5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50676/0 (socket says 172.21.15.45:50676) 2024-04-25T10:21:11.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 -- 172.21.15.45:0/4152974017 learned_addr learned my addr 172.21.15.45:0/4152974017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d2800640 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd205c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 -- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 msgr2=0x563cd205c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd205c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 -- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 msgr2=0x563cd205b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 -- 172.21.15.45:0/4152974017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cd1ef0870 con 0x563cd20a5970 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d2800640 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd205c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:11.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20a5d50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40ac040 tx=0x7fe9c4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8bd99a10ca89c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9c2a00640 1 -- 172.21.15.45:0/4152974017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40bb070 con 0x563cd20a5970 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9c2a00640 1 -- 172.21.15.45:0/4152974017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c40af040 con 0x563cd20a5970 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9c2a00640 1 -- 172.21.15.45:0/4152974017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40aadd0 con 0x563cd20a5970 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- 172.21.15.45:0/4152974017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 msgr2=0x563cd20a5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20a5d50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40ac040 tx=0x7fe9c4004740 comp rx=0 tx=0).stop 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- 172.21.15.45:0/4152974017 shutdown_connections 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd205c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 --2- 172.21.15.45:0/4152974017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20a5d50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- 172.21.15.45:0/4152974017 >> 172.21.15.45:0/4152974017 conn(0x563cd2062540 msgr2=0x563cd2062920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- 172.21.15.45:0/4152974017 shutdown_connections 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.587+0000 7fe9dc532000 1 -- 172.21.15.45:0/4152974017 wait complete. 2024-04-25T10:21:11.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- start start 2024-04-25T10:21:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 Processor -- start 2024-04-25T10:21:11.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd20accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cd1e64b50 con 0x563cd20a5970 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cd1ea5db0 con 0x563cd20a6290 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cd1f2d490 con 0x563cd205bcd0 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 10 client.0 _ll_register_callbacks cb 0x563cd1f5dfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50678/0 (socket says 172.21.15.45:50678) 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 -- 172.21.15.45:0/200148161 learned_addr learned my addr 172.21.15.45:0/200148161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9d2800640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd20accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 -- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 msgr2=0x563cd20accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd20accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 -- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 msgr2=0x563cd205efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 -- 172.21.15.45:0/200148161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cd1e32520 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9d2800640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20ad210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0830d0 tx=0x7fe9bc05dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8414fb8aeaf80ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9b7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9b7400640 1 -- 172.21.15.45:0/200148161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc05d940 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9b7400640 1 -- 172.21.15.45:0/200148161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9bc0549b0 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9c40af040 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9b7400640 1 -- 172.21.15.45:0/200148161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc054bd0 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 10 client.5997 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x563cd1ea4d00 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9b7400640 1 -- 172.21.15.45:0/200148161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c40af040 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9b7400640 1 -- 172.21.15.45:0/200148161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9bc0590b0 con 0x563cd20a5970 2024-04-25T10:21:11.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 10 client.5997 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 10 client.5997 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 20 client.5997 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd205fd20 0x563cd2060100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 4 client.5997 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cd1e45b10 con 0x563cd205fd20 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.591+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd205fd20 0x563cd2060100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd205fd20 0x563cd2060100 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e987e3960ae5758 server_cookie=941a29b4b2ea10c8 in_seq=0 out_seq=0 2024-04-25T10:21:11.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9b7400640 10 client.5997 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9b7400640 1 -- 172.21.15.45:0/200148161 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cd1e45b10 con 0x563cd205fd20 2024-04-25T10:21:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9b7400640 10 client.5997 handle_command_reply: tid=0 2024-04-25T10:21:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 client.5997 shutdown 2024-04-25T10:21:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 msgr2=0x563cd20ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20ad210 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0830d0 tx=0x7fe9bc05dbc0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 shutdown_connections 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd205fd20 0x563cd2060100 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a6290 0x563cd205efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd20ad210 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 --2- 172.21.15.45:0/200148161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd20accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9b7400640 0 client.5997 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 >> 172.21.15.45:0/200148161 conn(0x563cd2062540 msgr2=0x563cd2058770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 shutdown_connections 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 1 -- 172.21.15.45:0/200148161 wait complete. 2024-04-25T10:21:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.595+0000 7fe9dc532000 20 client.5997 trim_cache size 0 max 16384 2024-04-25T10:21:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 Processor -- start 2024-04-25T10:21:11.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- start start 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd205bcd0 0x563cd2101780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd2101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd20a6290 0x563cd2102200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d4202c30 con 0x563cd20a5970 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40b4310 con 0x563cd205bcd0 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d415ba70 con 0x563cd20a6290 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd2101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd2101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50686/0 (socket says 172.21.15.45:50686) 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 -- 172.21.15.45:0/216195503 learned_addr learned my addr 172.21.15.45:0/216195503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd205bcd0 0x563cd2101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9d2800640 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd20a6290 0x563cd2102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 -- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd20a6290 msgr2=0x563cd2102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd20a6290 0x563cd2102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 -- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd205bcd0 msgr2=0x563cd2101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd205bcd0 0x563cd2101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 -- 172.21.15.45:0/216195503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cd1f2d920 con 0x563cd20a5970 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd2101cc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc007d70 tx=0x7fe9bc06a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b365145e96c66a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9c2000640 1 -- 172.21.15.45:0/216195503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc055330 con 0x563cd20a5970 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9c2000640 1 -- 172.21.15.45:0/216195503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc0549b0 con 0x563cd20a5970 2024-04-25T10:21:11.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9c2000640 1 -- 172.21.15.45:0/216195503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0590b0 con 0x563cd20a5970 2024-04-25T10:21:11.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- 172.21.15.45:0/216195503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 msgr2=0x563cd2101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd2101cc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc007d70 tx=0x7fe9bc06a000 comp rx=0 tx=0).stop 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- 172.21.15.45:0/216195503 shutdown_connections 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd20a6290 0x563cd2102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a5970 0x563cd2101cc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 --2- 172.21.15.45:0/216195503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd205bcd0 0x563cd2101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- 172.21.15.45:0/216195503 >> 172.21.15.45:0/216195503 conn(0x563cd2062540 msgr2=0x563cd2059ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- 172.21.15.45:0/216195503 shutdown_connections 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.611+0000 7fe9dc532000 1 -- 172.21.15.45:0/216195503 wait complete. 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- start start 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 Processor -- start 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd2106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a5970 0x563cd21091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 0x563cd210b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40b3d10 con 0x563cd20a6290 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cd1ea5db0 con 0x563cd20a5970 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cd1e6b700 con 0x563cd205bcd0 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 10 client.0 _ll_register_callbacks cb 0x7fe9c4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 0x563cd210b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 0x563cd210b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50698/0 (socket says 172.21.15.45:50698) 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 -- 172.21.15.45:0/3032665709 learned_addr learned my addr 172.21.15.45:0/3032665709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd2106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9cbe00640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a5970 0x563cd21091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 -- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 msgr2=0x563cd2106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd2106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 -- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a5970 msgr2=0x563cd21091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a5970 0x563cd21091e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 -- 172.21.15.45:0/3032665709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cd1f03a30 con 0x563cd20a6290 2024-04-25T10:21:11.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d2800640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 0x563cd210b730 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0166e0 tx=0x7fe9cc016ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f723723f93b9547e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 1 -- 172.21.15.45:0/3032665709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc015890 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 1 -- 172.21.15.45:0/3032665709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9cc010040 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9bc0549b0 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 1 -- 172.21.15.45:0/3032665709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9cc015a30 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 10 client.6003 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x563cd1e51150 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 1 -- 172.21.15.45:0/3032665709 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc0549b0 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 1 -- 172.21.15.45:0/3032665709 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9cc028050 con 0x563cd20a6290 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 10 client.6003 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 10 client.6003 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:11.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 20 client.6003 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:11.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd210bc70 0x563cd210e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 4 client.6003 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:21:11.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x563cd1eb7250 con 0x563cd210bc70 2024-04-25T10:21:11.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd210bc70 0x563cd210e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9d1e00640 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd210bc70 0x563cd210e060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b88f8bcd4e0b5c68 server_cookie=5100f812e93ac82e in_seq=0 out_seq=0 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.615+0000 7fe9b7400640 10 client.6003 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9b7400640 1 -- 172.21.15.45:0/3032665709 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x563cd1eb7250 con 0x563cd210bc70 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9b7400640 10 client.6003 handle_command_reply: tid=0 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 client.6003 shutdown 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 msgr2=0x563cd210b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 0x563cd210b730 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0166e0 tx=0x7fe9cc016ba0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 shutdown_connections 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x563cd210bc70 0x563cd210e060 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563cd20a6290 0x563cd210b730 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563cd20a5970 0x563cd21091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 --2- 172.21.15.45:0/3032665709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563cd205bcd0 0x563cd2106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 >> 172.21.15.45:0/3032665709 conn(0x563cd2062540 msgr2=0x563cd205a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9b7400640 0 client.6003 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 shutdown_connections 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 1 -- 172.21.15.45:0/3032665709 wait complete. 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9dc532000 20 client.6003 trim_cache size 0 max 16384 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:11.625 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": -2 2024-04-25T10:21:11.626 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:11.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9b0066f60 msgr2=0x7fe9b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40ae000 tx=0x7fe9c40a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 msgr2=0x7fe9d40afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 0x7fe9d40afb70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c480 tx=0x7fe9bc00c940 comp rx=0 tx=0).stop 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 shutdown_connections 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d4159540 0x7fe9d414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d40aebd0 0x7fe9d40afb70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 --2- 172.21.15.45:0/2164014985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d40721f0 0x7fe9d414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 >> 172.21.15.45:0/2164014985 conn(0x7fe9d40669a0 msgr2=0x7fe9d415e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 shutdown_connections 2024-04-25T10:21:11.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.619+0000 7fe9d9000640 1 -- 172.21.15.45:0/2164014985 wait complete. 2024-04-25T10:21:11.640 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:21:11.640 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:21:11.640 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:11.640 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:21:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 Processor -- start 2024-04-25T10:21:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 -- start start 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c20070350 0x7f0c2015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200b31c0 0x7f0c200b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2015b080 con 0x7f0c200aef80 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2015adb0 con 0x7f0c20070350 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c24c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2015eb60 con 0x7f0c200b31c0 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c1de00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0c200aef80 0x7f0c2015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36520/0 (socket says 172.21.15.45:36520) 2024-04-25T10:21:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c1de00640 1 -- 172.21.15.45:0/1167366250 learned_addr learned my addr 172.21.15.45:0/1167366250 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c17e00640 1 --2- 172.21.15.45:0/1167366250 >> v2:172.21.15.117:3300/0 conn(0x7f0c20070350 0x7f0c2015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c1e800640 1 --2- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200b31c0 0x7f0c200b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c1d400640 1 -- 172.21.15.45:0/1167366250 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469236738 0 0) 0x7f0c2015b080 con 0x7f0c200aef80 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c1d400640 1 -- 172.21.15.45:0/1167366250 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c04003530 con 0x7f0c200aef80 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c17e00640 1 -- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200b31c0 msgr2=0x7f0c200b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c17e00640 1 --2- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200b31c0 0x7f0c200b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c17e00640 1 -- 172.21.15.45:0/1167366250 >> v1:172.21.15.45:6789/0 conn(0x7f0c200aef80 legacy=0x7f0c2015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.783+0000 7f0c17e00640 1 -- 172.21.15.45:0/1167366250 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2015ee40 con 0x7f0c20070350 2024-04-25T10:21:11.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c17e00640 1 --2- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20070350 0x7f0c2015e480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097c0 tx=0x7f0c0801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a1449feebbd4a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c1d400640 1 -- 172.21.15.45:0/1167366250 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080b1070 con 0x7f0c20070350 2024-04-25T10:21:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c1d400640 1 -- 172.21.15.45:0/1167366250 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c080ac410 con 0x7f0c20070350 2024-04-25T10:21:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c1d400640 1 -- 172.21.15.45:0/1167366250 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080ac7e0 con 0x7f0c20070350 2024-04-25T10:21:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 -- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20070350 msgr2=0x7f0c2015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 --2- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20070350 0x7f0c2015e480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097c0 tx=0x7f0c0801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 -- 172.21.15.45:0/1167366250 shutdown_connections 2024-04-25T10:21:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 --2- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200b31c0 0x7f0c200b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 --2- 172.21.15.45:0/1167366250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20070350 0x7f0c2015e480 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 -- 172.21.15.45:0/1167366250 >> 172.21.15.45:0/1167366250 conn(0x7f0c200669a0 msgr2=0x7f0c20066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 -- 172.21.15.45:0/1167366250 shutdown_connections 2024-04-25T10:21:11.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.787+0000 7f0c24c00640 1 -- 172.21.15.45:0/1167366250 wait complete. 2024-04-25T10:21:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 Processor -- start 2024-04-25T10:21:11.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 -- start start 2024-04-25T10:21:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 0x7f0c20159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200aef80 0x7f0c200af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c200b31c0 0x7f0c2014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2006c250 con 0x7f0c20070350 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2006c0d0 con 0x7f0c200b31c0 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2006c3d0 con 0x7f0c200aef80 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 0x7f0c20159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 0x7f0c20159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50714/0 (socket says 172.21.15.45:50714) 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 -- 172.21.15.45:0/3640526790 learned_addr learned my addr 172.21.15.45:0/3640526790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c17e00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200aef80 0x7f0c200af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1e800640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c200b31c0 0x7f0c2014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 -- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200aef80 msgr2=0x7f0c200af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200aef80 0x7f0c200af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 -- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c200b31c0 msgr2=0x7f0c2014bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c200b31c0 0x7f0c2014bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 -- 172.21.15.45:0/3640526790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20150520 con 0x7f0c20070350 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c1de00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 0x7f0c20159f60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0c100010f0 tx=0x7f0c1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c16000640 1 -- 172.21.15.45:0/3640526790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10017070 con 0x7f0c20070350 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c16000640 1 -- 172.21.15.45:0/3640526790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10005d10 con 0x7f0c20070350 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c16000640 1 -- 172.21.15.45:0/3640526790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007310 con 0x7f0c20070350 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c20150800 con 0x7f0c20070350 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.791+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c20154e90 con 0x7f0c20070350 2024-04-25T10:21:11.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c16000640 1 -- 172.21.15.45:0/3640526790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c1001a070 con 0x7f0c20070350 2024-04-25T10:21:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c16000640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bfc066f60 0x7f0bfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c16000640 1 -- 172.21.15.45:0/3640526790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c10084400 con 0x7f0c20070350 2024-04-25T10:21:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 Processor -- start 2024-04-25T10:21:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c17e00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bfc066f60 0x7f0bfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 -- start start 2024-04-25T10:21:11.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c17e00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bfc066f60 0x7f0bfc069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ae000 tx=0x7f0c08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e535f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 0x5599e53151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e5315710 0x5599e5315af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e5163db0 con 0x5599e535f7c0 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e50ec230 con 0x5599e5315710 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e515fe50 con 0x5599e535eea0 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 0x5599e53151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 0x5599e53151d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50730/0 (socket says 172.21.15.45:50730) 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c17e00640 1 -- 172.21.15.45:0/564551737 learned_addr learned my addr 172.21.15.45:0/564551737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.795+0000 7f0c1de00640 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e535f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c1e800640 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e5315710 0x5599e5315af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 -- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 msgr2=0x5599e535f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e535f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 -- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e5315710 msgr2=0x5599e5315af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e5315710 0x5599e5315af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 -- 172.21.15.45:0/564551737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e5125750 con 0x5599e535f7c0 2024-04-25T10:21:11.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 0x5599e53151d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c0801c4e0 tx=0x7f0c08004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefb149e5026c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c0ea00640 1 -- 172.21.15.45:0/564551737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080c8070 con 0x5599e535f7c0 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c0ea00640 1 -- 172.21.15.45:0/564551737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c080b7530 con 0x5599e535f7c0 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c0ea00640 1 -- 172.21.15.45:0/564551737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080b1040 con 0x5599e535f7c0 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- 172.21.15.45:0/564551737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 msgr2=0x5599e53151d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 0x5599e53151d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c0801c4e0 tx=0x7f0c08004740 comp rx=0 tx=0).stop 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- 172.21.15.45:0/564551737 shutdown_connections 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e5315710 0x5599e5315af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535f7c0 0x5599e53151d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- 172.21.15.45:0/564551737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e535f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- 172.21.15.45:0/564551737 >> 172.21.15.45:0/564551737 conn(0x5599e53116c0 msgr2=0x5599e5313ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- 172.21.15.45:0/564551737 shutdown_connections 2024-04-25T10:21:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- 172.21.15.45:0/564551737 wait complete. 2024-04-25T10:21:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- start start 2024-04-25T10:21:11.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 Processor -- start 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e5315710 0x5599e5343ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 0x5599e5344410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5344950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e511ebf0 con 0x5599e535eea0 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e5162d80 con 0x5599e535f7c0 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e50e8990 con 0x5599e5315710 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c2817e000 10 client.0 _ll_register_callbacks cb 0x5599e5300790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 0x5599e5344410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 0x5599e5344410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50734/0 (socket says 172.21.15.45:50734) 2024-04-25T10:21:11.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c17e00640 1 -- 172.21.15.45:0/2955034870 learned_addr learned my addr 172.21.15.45:0/2955034870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c1de00640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e5315710 0x5599e5343ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.799+0000 7f0c1e800640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5344950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c17e00640 1 -- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e5315710 msgr2=0x5599e5343ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c17e00640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e5315710 0x5599e5343ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c17e00640 1 -- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 msgr2=0x5599e5344950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c17e00640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5344950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c17e00640 1 -- 172.21.15.45:0/2955034870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599e5106640 con 0x5599e535eea0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c17e00640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 0x5599e5344410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0c0801c980 tx=0x7f0c0801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d0134e860a1eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 1 -- 172.21.15.45:0/2955034870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080c8030 con 0x5599e535eea0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 1 -- 172.21.15.45:0/2955034870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c080be040 con 0x5599e535eea0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c080b7530 con 0x5599e535eea0 2024-04-25T10:21:11.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 1 -- 172.21.15.45:0/2955034870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c080b1040 con 0x5599e535eea0 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 10 client.6015 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5599e50ec480 con 0x5599e535eea0 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 1 -- 172.21.15.45:0/2955034870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c080b7530 con 0x5599e535eea0 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 1 -- 172.21.15.45:0/2955034870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c200cfe70 con 0x5599e535eea0 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 10 client.6015 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 20 client.6015 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e536ef40 0x5599e5371330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 4 client.6015 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e50ffbb0 con 0x5599e536ef40 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c1de00640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e536ef40 0x5599e5371330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c1de00640 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e536ef40 0x5599e5371330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e92413ecdb02ef server_cookie=8f01f7f158749e8b in_seq=0 out_seq=0 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 10 client.6015 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 1 -- 172.21.15.45:0/2955034870 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599e50ffbb0 con 0x5599e536ef40 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 10 client.6015 handle_command_reply: tid=0 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 client.6015 shutdown 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 msgr2=0x5599e5344410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 0x5599e5344410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0c0801c980 tx=0x7f0c0801ace0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 shutdown_connections 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e536ef40 0x5599e5371330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5344950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e535eea0 0x5599e5344410 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c03400640 0 client.6015 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 --2- 172.21.15.45:0/2955034870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e5315710 0x5599e5343ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 >> 172.21.15.45:0/2955034870 conn(0x5599e53116c0 msgr2=0x5599e5314340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 shutdown_connections 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 1 -- 172.21.15.45:0/2955034870 wait complete. 2024-04-25T10:21:11.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.803+0000 7f0c2817e000 20 client.6015 trim_cache size 0 max 16384 2024-04-25T10:21:11.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 Processor -- start 2024-04-25T10:21:11.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 -- start start 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e5314c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535eea0 0x5599e5313ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535f7c0 0x5599e5313ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20202e00 con 0x5599e5315710 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20158f60 con 0x5599e535eea0 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20158360 con 0x5599e535f7c0 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c1de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e5314c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c1de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e5314c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50748/0 (socket says 172.21.15.45:50748) 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c1de00640 1 -- 172.21.15.45:0/1620643367 learned_addr learned my addr 172.21.15.45:0/1620643367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c17e00640 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535eea0 0x5599e5313ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.819+0000 7f0c1e800640 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535f7c0 0x5599e5313ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535f7c0 msgr2=0x5599e5313ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535f7c0 0x5599e5313ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535eea0 msgr2=0x5599e5313ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535eea0 0x5599e5313ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/1620643367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e51e5ef0 con 0x5599e5315710 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c17e00640 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535eea0 0x5599e5313ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e5314c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0c10007fd0 tx=0x7f0c100827f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5cdef884f4fd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c0e000640 1 -- 172.21.15.45:0/1620643367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100530b0 con 0x5599e5315710 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c0e000640 1 -- 172.21.15.45:0/1620643367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c100840b0 con 0x5599e5315710 2024-04-25T10:21:11.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c0e000640 1 -- 172.21.15.45:0/1620643367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100074b0 con 0x5599e5315710 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- 172.21.15.45:0/1620643367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 msgr2=0x5599e5314c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e5314c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0c10007fd0 tx=0x7f0c100827f0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- 172.21.15.45:0/1620643367 shutdown_connections 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535f7c0 0x5599e5313ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535eea0 0x5599e5313ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- 172.21.15.45:0/1620643367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e5314c10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- 172.21.15.45:0/1620643367 >> 172.21.15.45:0/1620643367 conn(0x5599e53116c0 msgr2=0x5599e536c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- 172.21.15.45:0/1620643367 shutdown_connections 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- 172.21.15.45:0/1620643367 wait complete. 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- start start 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 Processor -- start 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e53c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e53c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5371640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e512a940 con 0x5599e5315710 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20158960 con 0x5599e535f7c0 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e51630c0 con 0x5599e535eea0 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c2817e000 10 client.0 _ll_register_callbacks cb 0x7f0c04003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e53c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e53c0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50764/0 (socket says 172.21.15.45:50764) 2024-04-25T10:21:11.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/372845988 learned_addr learned my addr 172.21.15.45:0/372845988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c17e00640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e53c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1e800640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5371640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 msgr2=0x5599e53c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e53c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 msgr2=0x5599e5371640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5371640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 -- 172.21.15.45:0/372845988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599e51239a0 con 0x5599e5315710 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1e800640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5371640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c1de00640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e53c0050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0c10051210 tx=0x7f0c10084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb23c0df441dd5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c03400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.823+0000 7f0c03400640 1 -- 172.21.15.45:0/372845988 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10079070 con 0x5599e5315710 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 1 -- 172.21.15.45:0/372845988 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c10051420 con 0x5599e5315710 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c100840b0 con 0x5599e5315710 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 1 -- 172.21.15.45:0/372845988 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c10065050 con 0x5599e5315710 2024-04-25T10:21:11.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 10 client.6021 fetch_fsmap learned FSMap version 27 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5599e4e205a0 con 0x5599e5315710 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 1 -- 172.21.15.45:0/372845988 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c100840b0 con 0x5599e5315710 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 1 -- 172.21.15.45:0/372845988 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c10068020 con 0x5599e5315710 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 10 client.6021 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 20 client.6021 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e5372030 0x5599e5372410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 4 client.6021 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x5599e5113080 con 0x5599e5372030 2024-04-25T10:21:11.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c17e00640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e5372030 0x5599e5372410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c17e00640 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e5372030 0x5599e5372410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f179731ef34b60 server_cookie=9d55a4e6ce264cb4 in_seq=0 out_seq=0 2024-04-25T10:21:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 10 client.6021 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 1 -- 172.21.15.45:0/372845988 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5599e5113080 con 0x5599e5372030 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 10 client.6021 handle_command_reply: tid=0 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 client.6021 shutdown 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 msgr2=0x5599e53c0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e53c0050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0c10051210 tx=0x7f0c10084040 comp rx=0 tx=0).stop 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 shutdown_connections 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5599e5372030 0x5599e5372410 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599e535f7c0 0x5599e5371640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c03400640 0 client.6021 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599e535eea0 0x5599e53c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 --2- 172.21.15.45:0/372845988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599e5315710 0x5599e53c0050 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 >> 172.21.15.45:0/372845988 conn(0x5599e53116c0 msgr2=0x5599e535d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 shutdown_connections 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 1 -- 172.21.15.45:0/372845988 wait complete. 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c2817e000 20 client.6021 trim_cache size 0 max 16384 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "9c5a9117-0391-47be-a3c5-73ce341cdf31", 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:11.835 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bfc066f60 msgr2=0x7f0bfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bfc066f60 0x7f0bfc069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ae000 tx=0x7f0c08008040 comp rx=0 tx=0).stop 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 msgr2=0x7f0c20159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 0x7f0c20159f60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0c100010f0 tx=0x7f0c1000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 shutdown_connections 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bfc066f60 0x7f0bfc069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c200b31c0 0x7f0c2014bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c200aef80 0x7f0c200af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 --2- 172.21.15.45:0/3640526790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c20070350 0x7f0c20159f60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:11.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 >> 172.21.15.45:0/3640526790 conn(0x7f0c200669a0 msgr2=0x7f0c2015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:11.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 shutdown_connections 2024-04-25T10:21:11.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.827+0000 7f0c24c00640 1 -- 172.21.15.45:0/3640526790 wait complete. 2024-04-25T10:21:11.850 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '9c5a9117-0391-47be-a3c5-73ce341cdf31', 'mode': 'asynchronous'}' 2024-04-25T10:21:11.851 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:21:11.851 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:11.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:21:11.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 Processor -- start 2024-04-25T10:21:11.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 -- start start 2024-04-25T10:21:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20fc0aef80 0x7f20fc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc15b080 con 0x7f20fc15bac0 2024-04-25T10:21:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc15adb0 con 0x7f20fc0aef80 2024-04-25T10:21:11.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f2101000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc15eb60 con 0x7f20fc070350 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20fc0aef80 0x7f20fc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20fc0aef80 0x7f20fc15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50768/0 (socket says 172.21.15.45:50768) 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 -- 172.21.15.45:0/3601901170 learned_addr learned my addr 172.21.15.45:0/3601901170 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20fa800640 1 --1- 172.21.15.45:0/3601901170 >> v1:172.21.15.45:6789/0 conn(0x7f20fc15bac0 0x7f20fc0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36528/0 (socket says 172.21.15.45:36528) 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f1400640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 -- 172.21.15.45:0/3601901170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 msgr2=0x7f20fc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 -- 172.21.15.45:0/3601901170 >> v1:172.21.15.45:6789/0 conn(0x7f20fc15bac0 legacy=0x7f20fc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f9e00640 1 -- 172.21.15.45:0/3601901170 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc15ee40 con 0x7f20fc0aef80 2024-04-25T10:21:12.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.995+0000 7f20f1400640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:12.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f20f9e00640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc0aef80 0x7f20fc15e480 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009870 tx=0x7f20ec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1852d18b692b4f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f20f9400640 1 -- 172.21.15.45:0/3601901170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0a9400 con 0x7f20fc0aef80 2024-04-25T10:21:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f20f9400640 1 -- 172.21.15.45:0/3601901170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec0a95a0 con 0x7f20fc0aef80 2024-04-25T10:21:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f20f9400640 1 -- 172.21.15.45:0/3601901170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0a9970 con 0x7f20fc0aef80 2024-04-25T10:21:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 -- 172.21.15.45:0/3601901170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc0aef80 msgr2=0x7f20fc15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc0aef80 0x7f20fc15e480 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009870 tx=0x7f20ec01c560 comp rx=0 tx=0).stop 2024-04-25T10:21:12.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 -- 172.21.15.45:0/3601901170 shutdown_connections 2024-04-25T10:21:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 --2- 172.21.15.45:0/3601901170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc0aef80 0x7f20fc15e480 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 -- 172.21.15.45:0/3601901170 >> 172.21.15.45:0/3601901170 conn(0x7f20fc0669a0 msgr2=0x7f20fc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 -- 172.21.15.45:0/3601901170 shutdown_connections 2024-04-25T10:21:12.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:11.999+0000 7f2101000640 1 -- 172.21.15.45:0/3601901170 wait complete. 2024-04-25T10:21:12.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 Processor -- start 2024-04-25T10:21:12.005 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:21:12.003+0000 7ff8bb000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:21:12.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 -- start start 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc0aef80 0x7f20fc0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc15bac0 0x7f20fc0b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc1679a0 con 0x7f20fc15bac0 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc167820 con 0x7f20fc0aef80 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f2101000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc167b20 con 0x7f20fc070350 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc15bac0 0x7f20fc0b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc14c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43320/0 (socket says 172.21.15.45:43320) 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20f9e00640 1 -- 172.21.15.45:0/1585782063 learned_addr learned my addr 172.21.15.45:0/1585782063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20f1400640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc0aef80 0x7f20fc0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 -- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 msgr2=0x7f20fc14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc14c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 -- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc0aef80 msgr2=0x7f20fc0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc0aef80 0x7f20fc0b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 -- 172.21.15.45:0/1585782063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc151060 con 0x7f20fc15bac0 2024-04-25T10:21:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20f9e00640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:12.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.003+0000 7f20fa800640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc15bac0 0x7f20fc0b06b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c730 tx=0x7f20f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f2a00640 1 -- 172.21.15.45:0/1585782063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4018070 con 0x7f20fc15bac0 2024-04-25T10:21:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f2a00640 1 -- 172.21.15.45:0/1585782063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f4004d70 con 0x7f20fc15bac0 2024-04-25T10:21:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f2a00640 1 -- 172.21.15.45:0/1585782063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400a860 con 0x7f20fc15bac0 2024-04-25T10:21:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc1512e0 con 0x7f20fc15bac0 2024-04-25T10:21:12.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc1518a0 con 0x7f20fc15bac0 2024-04-25T10:21:12.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f2a00640 1 -- 172.21.15.45:0/1585782063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20f400ac10 con 0x7f20fc15bac0 2024-04-25T10:21:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f2a00640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20d8066f60 0x7f20d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f2a00640 1 -- 172.21.15.45:0/1585782063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20f4084660 con 0x7f20fc15bac0 2024-04-25T10:21:12.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.007+0000 7f20f9e00640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20d8066f60 0x7f20d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 Processor -- start 2024-04-25T10:21:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 -- start start 2024-04-25T10:21:12.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f9e00640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20d8066f60 0x7f20d8069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0ab040 tx=0x7f20ec0b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6fe200 0x556b8a6fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6e2f60 0x556b8a6e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8a59b4a0 con 0x556b8a6feb20 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8a503210 con 0x556b8a6e2f60 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b8a48b980 con 0x556b8a6fe200 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6fe200 0x556b8a6fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6e2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50778/0 (socket says 172.21.15.45:50778) 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 -- 172.21.15.45:0/763125189 learned_addr learned my addr 172.21.15.45:0/763125189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20fa800640 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6e2f60 0x556b8a6e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 -- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6fe200 msgr2=0x556b8a6fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6fe200 0x556b8a6fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 -- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6e2f60 msgr2=0x556b8a6e3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6e2f60 0x556b8a6e3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 -- 172.21.15.45:0/763125189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b8a4fe200 con 0x556b8a6feb20 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20fa800640 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6e2f60 0x556b8a6e3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:12.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20f1400640 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6e2a20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009960 tx=0x7f20e4009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbf787d6a04d1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20eaa00640 1 -- 172.21.15.45:0/763125189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4019070 con 0x556b8a6feb20 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20eaa00640 1 -- 172.21.15.45:0/763125189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e4011bc0 con 0x556b8a6feb20 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f20eaa00640 1 -- 172.21.15.45:0/763125189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400f040 con 0x556b8a6feb20 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 -- 172.21.15.45:0/763125189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 msgr2=0x556b8a6e2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.011+0000 7f21044d5000 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6e2a20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009960 tx=0x7f20e4009e20 comp rx=0 tx=0).stop 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- 172.21.15.45:0/763125189 shutdown_connections 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6e2f60 0x556b8a6e3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6e2a20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 --2- 172.21.15.45:0/763125189 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6fe200 0x556b8a6fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- 172.21.15.45:0/763125189 >> 172.21.15.45:0/763125189 conn(0x556b8a6ab4f0 msgr2=0x556b8a6baa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- 172.21.15.45:0/763125189 shutdown_connections 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- 172.21.15.45:0/763125189 wait complete. 2024-04-25T10:21:12.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- start start 2024-04-25T10:21:12.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 Processor -- start 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a705980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a705ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6e3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8a4bdd80 con 0x556b8a6e2f60 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8a4ff2b0 con 0x556b8a6fe200 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b8a4c4830 con 0x556b8a6feb20 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 10 client.0 _ll_register_callbacks cb 0x556b8a5b6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a705980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a705980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50788/0 (socket says 172.21.15.45:50788) 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f9e00640 1 -- 172.21.15.45:0/3336943290 learned_addr learned my addr 172.21.15.45:0/3336943290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a705ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20fa800640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 -- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 msgr2=0x556b8a6e3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6e3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 -- 172.21.15.45:0/3336943290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 msgr2=0x556b8a705980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a705980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 -- 172.21.15.45:0/3336943290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b8a4cf130 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20f1400640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a705ec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20e401b000 tx=0x7f20e4004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d11d72f5845a4274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20df400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20df400640 1 -- 172.21.15.45:0/3336943290 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e402c060 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20df400640 1 -- 172.21.15.45:0/3336943290 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e40119c0 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20e4011bc0 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20df400640 1 -- 172.21.15.45:0/3336943290 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e4032050 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 10 client.5815 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x556b8a584490 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.015+0000 7f20df400640 1 -- 172.21.15.45:0/3336943290 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e4011bc0 con 0x556b8a6fe200 2024-04-25T10:21:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20df400640 1 -- 172.21.15.45:0/3336943290 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20e4035020 con 0x556b8a6fe200 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 10 client.5815 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 10 client.5815 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 20 client.5815 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a6e40b0 0x556b8a6e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 4 client.5815 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b8a49f020 con 0x556b8a6e40b0 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20f9e00640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a6e40b0 0x556b8a6e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20f9e00640 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a6e40b0 0x556b8a6e44c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4707cc91869547 server_cookie=b2e9d95961d46515 in_seq=0 out_seq=0 2024-04-25T10:21:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20df400640 10 client.5815 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20df400640 1 -- 172.21.15.45:0/3336943290 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b8a49f020 con 0x556b8a6e40b0 2024-04-25T10:21:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20df400640 10 client.5815 handle_command_reply: tid=0 2024-04-25T10:21:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 client.5815 shutdown 2024-04-25T10:21:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 msgr2=0x556b8a705ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a705ec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f20e401b000 tx=0x7f20e4004740 comp rx=0 tx=0).stop 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 shutdown_connections 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a6e40b0 0x556b8a6e44c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6e3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a705ec0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 --2- 172.21.15.45:0/3336943290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a705980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 >> 172.21.15.45:0/3336943290 conn(0x556b8a6ab4f0 msgr2=0x556b8a6b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f20df400640 0 client.5815 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 shutdown_connections 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 1 -- 172.21.15.45:0/3336943290 wait complete. 2024-04-25T10:21:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.019+0000 7f21044d5000 20 client.5815 trim_cache size 0 max 16384 2024-04-25T10:21:12.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 Processor -- start 2024-04-25T10:21:12.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 -- start start 2024-04-25T10:21:12.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a6b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc200e20 con 0x556b8a6e2f60 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc153170 con 0x556b8a6fe200 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc14d9b0 con 0x556b8a6feb20 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a6b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a6b8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50794/0 (socket says 172.21.15.45:50794) 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 -- 172.21.15.45:0/1474682759 learned_addr learned my addr 172.21.15.45:0/1474682759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20fa800640 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f1400640 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 -- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 msgr2=0x556b8a6b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 -- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 msgr2=0x556b8a6b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 -- 172.21.15.45:0/1474682759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b8a4c2a30 con 0x556b8a6e2f60 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20f9e00640 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a6b8e30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0a9b20 tx=0x7f20ec0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f635e4d0bfaee61a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20ea000640 1 -- 172.21.15.45:0/1474682759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0d3070 con 0x556b8a6e2f60 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20ea000640 1 -- 172.21.15.45:0/1474682759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec0c8450 con 0x556b8a6e2f60 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.035+0000 7f20ea000640 1 -- 172.21.15.45:0/1474682759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0c8800 con 0x556b8a6e2f60 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/1474682759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 msgr2=0x556b8a6b8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a6b8e30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0a9b20 tx=0x7f20ec0a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/1474682759 shutdown_connections 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6feb20 0x556b8a6b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- 172.21.15.45:0/1474682759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6e2f60 0x556b8a6b8e30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/1474682759 >> 172.21.15.45:0/1474682759 conn(0x556b8a6ab4f0 msgr2=0x556b8a6e16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/1474682759 shutdown_connections 2024-04-25T10:21:12.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/1474682759 wait complete. 2024-04-25T10:21:12.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- start start 2024-04-25T10:21:12.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 Processor -- start 2024-04-25T10:21:12.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6e2f60 0x556b8a6b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8a4ff2b0 con 0x556b8a6feb20 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8a48f190 con 0x556b8a6fe200 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b8a507840 con 0x556b8a6e2f60 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 10 client.0 _ll_register_callbacks cb 0x7f20e4009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6e2f60 0x556b8a6b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20f9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6e2f60 0x556b8a6b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43358/0 (socket says 172.21.15.45:43358) 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20f9e00640 1 -- 172.21.15.45:0/2090541012 learned_addr learned my addr 172.21.15.45:0/2090541012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20f1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 -- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6e2f60 msgr2=0x556b8a6b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6e2f60 0x556b8a6b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 -- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 msgr2=0x556b8a6b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 -- 172.21.15.45:0/2090541012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b8a4cb040 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20fa800640 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6b44e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c700 tx=0x7f20f4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae662e32b32ff22d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20df400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20df400640 1 -- 172.21.15.45:0/2090541012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4081060 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20df400640 1 -- 172.21.15.45:0/2090541012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20f405d3e0 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ec0c8450 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20df400640 1 -- 172.21.15.45:0/2090541012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f406c070 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 10 client.6042 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x556b8a1bf900 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20df400640 1 -- 172.21.15.45:0/2090541012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec0c8450 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f20df400640 1 -- 172.21.15.45:0/2090541012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20f405b0b0 con 0x556b8a6feb20 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 10 client.6042 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 10 client.6042 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 20 client.6042 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:12.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a720450 0x556b8a720830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 4 client.6042 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.039+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x556b8a510750 con 0x556b8a720450 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f20f9e00640 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a720450 0x556b8a720830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f20f9e00640 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a720450 0x556b8a720830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e969fc3ab5e734 server_cookie=7f4ab13304465e34 in_seq=0 out_seq=0 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f20df400640 10 client.6042 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f20df400640 1 -- 172.21.15.45:0/2090541012 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556b8a510750 con 0x556b8a720450 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f20df400640 10 client.6042 handle_command_reply: tid=0 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 client.6042 shutdown 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 msgr2=0x556b8a6b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6b44e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c700 tx=0x7f20f4001260 comp rx=0 tx=0).stop 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 shutdown_connections 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x556b8a720450 0x556b8a720830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556b8a6feb20 0x556b8a6b44e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b8a6fe200 0x556b8a6b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 --2- 172.21.15.45:0/2090541012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556b8a6e2f60 0x556b8a6b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 >> 172.21.15.45:0/2090541012 conn(0x556b8a6ab4f0 msgr2=0x556b8a6b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f20df400640 0 client.6042 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 shutdown_connections 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 1 -- 172.21.15.45:0/2090541012 wait complete. 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f21044d5000 20 client.6042 trim_cache size 0 max 16384 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:12.050 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "f7326f8a-f5e5-4b29-914a-29faca83cebe", 2024-04-25T10:21:12.051 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:12.051 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20d8066f60 msgr2=0x7f20d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20d8066f60 0x7f20d8069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0ab040 tx=0x7f20ec0b3040 comp rx=0 tx=0).stop 2024-04-25T10:21:12.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc15bac0 msgr2=0x7f20fc0b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc15bac0 0x7f20fc0b06b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c730 tx=0x7f20f400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 shutdown_connections 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20d8066f60 0x7f20d8069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20fc15bac0 0x7f20fc0b06b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc0aef80 0x7f20fc0b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 --2- 172.21.15.45:0/1585782063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20fc070350 0x7f20fc14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 >> 172.21.15.45:0/1585782063 conn(0x7f20fc0669a0 msgr2=0x7f20fc06daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 shutdown_connections 2024-04-25T10:21:12.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.043+0000 7f2101000640 1 -- 172.21.15.45:0/1585782063 wait complete. 2024-04-25T10:21:12.065 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'f7326f8a-f5e5-4b29-914a-29faca83cebe', 'mode': 'asynchronous'}' 2024-04-25T10:21:12.065 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:21:12.065 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:12.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:21:12.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 Processor -- start 2024-04-25T10:21:12.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 -- start start 2024-04-25T10:21:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc640ceb80 0x7efc640669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc64066ee0 0x7efc6415c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc64156490 con 0x7efc641560b0 2024-04-25T10:21:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6415f760 con 0x7efc640ceb80 2024-04-25T10:21:12.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc6b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415fa40 con 0x7efc64066ee0 2024-04-25T10:21:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.207+0000 7efc68a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efc641560b0 0x7efc6406ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36532/0 (socket says 172.21.15.45:36532) 2024-04-25T10:21:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc68a00640 1 -- 172.21.15.45:0/824312442 learned_addr learned my addr 172.21.15.45:0/824312442 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc5be00640 1 --2- 172.21.15.45:0/824312442 >> v2:172.21.15.117:3300/0 conn(0x7efc640ceb80 0x7efc640669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc69400640 1 --2- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc64066ee0 0x7efc6415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc63e00640 1 -- 172.21.15.45:0/824312442 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622090352 0 0) 0x7efc64156490 con 0x7efc641560b0 2024-04-25T10:21:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc63e00640 1 -- 172.21.15.45:0/824312442 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc48003530 con 0x7efc641560b0 2024-04-25T10:21:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc5be00640 1 -- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc64066ee0 msgr2=0x7efc6415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc5be00640 1 --2- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc64066ee0 0x7efc6415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc5be00640 1 -- 172.21.15.45:0/824312442 >> v1:172.21.15.45:6789/0 conn(0x7efc641560b0 legacy=0x7efc6406ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc5be00640 1 -- 172.21.15.45:0/824312442 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6415fd20 con 0x7efc640ceb80 2024-04-25T10:21:12.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc5be00640 1 --2- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc640ceb80 0x7efc640669a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0097c0 tx=0x7efc4c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d83eafae961b3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc63e00640 1 -- 172.21.15.45:0/824312442 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0b0070 con 0x7efc640ceb80 2024-04-25T10:21:12.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc63e00640 1 -- 172.21.15.45:0/824312442 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c0ab600 con 0x7efc640ceb80 2024-04-25T10:21:12.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.211+0000 7efc63e00640 1 -- 172.21.15.45:0/824312442 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0ab9b0 con 0x7efc640ceb80 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc640ceb80 msgr2=0x7efc640669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 --2- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc640ceb80 0x7efc640669a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0097c0 tx=0x7efc4c01c410 comp rx=0 tx=0).stop 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- 172.21.15.45:0/824312442 shutdown_connections 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 --2- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc64066ee0 0x7efc6415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 --2- 172.21.15.45:0/824312442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc640ceb80 0x7efc640669a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- 172.21.15.45:0/824312442 >> 172.21.15.45:0/824312442 conn(0x7efc640b2560 msgr2=0x7efc640b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- 172.21.15.45:0/824312442 shutdown_connections 2024-04-25T10:21:12.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- 172.21.15.45:0/824312442 wait complete. 2024-04-25T10:21:12.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 Processor -- start 2024-04-25T10:21:12.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- start start 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 0x7efc6413bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc640ceb80 0x7efc6415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc641560b0 0x7efc6415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc64168960 con 0x7efc64066ee0 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc640672f0 con 0x7efc641560b0 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc6b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc64168ae0 con 0x7efc640ceb80 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 0x7efc6413bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 0x7efc6413bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50816/0 (socket says 172.21.15.45:50816) 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 -- 172.21.15.45:0/2053105469 learned_addr learned my addr 172.21.15.45:0/2053105469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 -- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc640ceb80 msgr2=0x7efc6415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc640ceb80 0x7efc6415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 -- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc641560b0 msgr2=0x7efc6415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc641560b0 0x7efc6415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 -- 172.21.15.45:0/2053105469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc641505a0 con 0x7efc64066ee0 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.215+0000 7efc68a00640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 0x7efc6413bf20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efc5400c3f0 tx=0x7efc5400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.219+0000 7efc61600640 1 -- 172.21.15.45:0/2053105469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54017070 con 0x7efc64066ee0 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.219+0000 7efc61600640 1 -- 172.21.15.45:0/2053105469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc54005cf0 con 0x7efc64066ee0 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.219+0000 7efc61600640 1 -- 172.21.15.45:0/2053105469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540072f0 con 0x7efc64066ee0 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.219+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc64146bd0 con 0x7efc64066ee0 2024-04-25T10:21:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.219+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc64147190 con 0x7efc64066ee0 2024-04-25T10:21:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.219+0000 7efc61600640 1 -- 172.21.15.45:0/2053105469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc5401a070 con 0x7efc64066ee0 2024-04-25T10:21:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc61600640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc40066f60 0x7efc40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc61600640 1 -- 172.21.15.45:0/2053105469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc54084430 con 0x7efc64066ee0 2024-04-25T10:21:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc5be00640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc40066f60 0x7efc40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 Processor -- start 2024-04-25T10:21:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- start start 2024-04-25T10:21:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc5be00640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc40066f60 0x7efc40069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0ad000 tx=0x7efc4c004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 0x55b6aca113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6ac9c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac814f40 con 0x55b6aca11000 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac79cfc0 con 0x55b6ac9c7410 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac810fe0 con 0x55b6aca11920 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 0x55b6aca113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 0x55b6aca113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50824/0 (socket says 172.21.15.45:50824) 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 -- 172.21.15.45:0/2560272460 learned_addr learned my addr 172.21.15.45:0/2560272460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc5be00640 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc69400640 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6ac9c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 -- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 msgr2=0x55b6ac9c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 -- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 msgr2=0x55b6ac9c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6ac9c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 -- 172.21.15.45:0/2560272460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6ac85ca50 con 0x55b6aca11000 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc69400640 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6ac9c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc68a00640 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 0x55b6aca113e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efc54051190 tx=0x7efc540549a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d8d889a57e97a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc59600640 1 -- 172.21.15.45:0/2560272460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5405b0d0 con 0x55b6aca11000 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc59600640 1 -- 172.21.15.45:0/2560272460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc54053030 con 0x55b6aca11000 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc59600640 1 -- 172.21.15.45:0/2560272460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540840c0 con 0x55b6aca11000 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- 172.21.15.45:0/2560272460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 msgr2=0x55b6aca113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 0x55b6aca113e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efc54051190 tx=0x7efc540549a0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- 172.21.15.45:0/2560272460 shutdown_connections 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6ac9c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/2560272460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11000 0x55b6aca113e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- 172.21.15.45:0/2560272460 >> 172.21.15.45:0/2560272460 conn(0x55b6ac9c33c0 msgr2=0x55b6ac9c57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- 172.21.15.45:0/2560272460 shutdown_connections 2024-04-25T10:21:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.223+0000 7efc6e8d9000 1 -- 172.21.15.45:0/2560272460 wait complete. 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- start start 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 Processor -- start 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 0x55b6ac9d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6ac9d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac819570 con 0x55b6ac9c7410 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac7cf5f0 con 0x55b6aca11000 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac811fb0 con 0x55b6aca11920 2024-04-25T10:21:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 10 client.0 _ll_register_callbacks cb 0x55b6ac8c97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 0x55b6ac9d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 0x55b6ac9d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50832/0 (socket says 172.21.15.45:50832) 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 -- 172.21.15.45:0/3018480632 learned_addr learned my addr 172.21.15.45:0/3018480632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc69400640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc5be00640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6ac9d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 -- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 msgr2=0x55b6ac9d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 -- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 msgr2=0x55b6ac9d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6ac9d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 -- 172.21.15.45:0/3018480632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6ac79d210 con 0x55b6ac9c7410 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc68a00640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 0x55b6ac9d2e00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc54054a30 tx=0x7efc54000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88c96be69ca6cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc52000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc52000640 1 -- 172.21.15.45:0/3018480632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540840c0 con 0x55b6ac9c7410 2024-04-25T10:21:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc52000640 1 -- 172.21.15.45:0/3018480632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc54007490 con 0x55b6ac9c7410 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc54053030 con 0x55b6ac9c7410 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc52000640 1 -- 172.21.15.45:0/3018480632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc54074070 con 0x55b6ac9c7410 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 10 client.6054 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55b6ac899400 con 0x55b6ac9c7410 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc52000640 1 -- 172.21.15.45:0/3018480632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc54053030 con 0x55b6ac9c7410 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc52000640 1 -- 172.21.15.45:0/3018480632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc54054180 con 0x55b6ac9c7410 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 10 client.6054 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 10 client.6054 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 20 client.6054 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca212b0 0x55b6aca236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 4 client.6054 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6ac7b0c90 con 0x55b6aca212b0 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.227+0000 7efc5be00640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca212b0 0x55b6aca236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc52000640 10 client.6054 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc5be00640 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca212b0 0x55b6aca236a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241209bb2832295a server_cookie=63292c59bb9a4c54 in_seq=0 out_seq=0 2024-04-25T10:21:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc52000640 1 -- 172.21.15.45:0/3018480632 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6ac7b0c90 con 0x55b6aca212b0 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc52000640 10 client.6054 handle_command_reply: tid=0 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 client.6054 shutdown 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 msgr2=0x55b6ac9d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 0x55b6ac9d2e00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7efc54054a30 tx=0x7efc54000bd0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 shutdown_connections 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca212b0 0x55b6aca236a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11920 0x55b6ac9d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6ac9d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/3018480632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6ac9c7410 0x55b6ac9d2e00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc52000640 0 client.6054 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 >> 172.21.15.45:0/3018480632 conn(0x55b6ac9c33c0 msgr2=0x55b6ac9c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 shutdown_connections 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 1 -- 172.21.15.45:0/3018480632 wait complete. 2024-04-25T10:21:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.231+0000 7efc6e8d9000 20 client.6054 trim_cache size 0 max 16384 2024-04-25T10:21:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 Processor -- start 2024-04-25T10:21:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- start start 2024-04-25T10:21:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6ac9c7410 0x55b6aca6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6aca6e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca6e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc64202ef0 con 0x55b6aca11920 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641488a0 con 0x55b6aca11000 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc640cfe10 con 0x55b6ac9c7410 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca6e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca6e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50848/0 (socket says 172.21.15.45:50848) 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 -- 172.21.15.45:0/4183718186 learned_addr learned my addr 172.21.15.45:0/4183718186 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc68a00640 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6ac9c7410 0x55b6aca6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc5be00640 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6aca6e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 -- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6ac9c7410 msgr2=0x55b6aca6df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6ac9c7410 0x55b6aca6df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 -- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 msgr2=0x55b6aca6e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6aca6e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 -- 172.21.15.45:0/4183718186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6ac86fc10 con 0x55b6aca11920 2024-04-25T10:21:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc5be00640 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6aca6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc69400640 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca6e9f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efc5c007650 tx=0x7efc5c007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dc1ba933ca833b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc58c00640 1 -- 172.21.15.45:0/4183718186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c01c070 con 0x55b6aca11920 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc58c00640 1 -- 172.21.15.45:0/4183718186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c0139b0 con 0x55b6aca11920 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc58c00640 1 -- 172.21.15.45:0/4183718186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c013d60 con 0x55b6aca11920 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- 172.21.15.45:0/4183718186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 msgr2=0x55b6aca6e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca6e9f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efc5c007650 tx=0x7efc5c007b10 comp rx=0 tx=0).stop 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- 172.21.15.45:0/4183718186 shutdown_connections 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca6e9f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6aca11000 0x55b6aca6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/4183718186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6ac9c7410 0x55b6aca6df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- 172.21.15.45:0/4183718186 >> 172.21.15.45:0/4183718186 conn(0x55b6ac9c33c0 msgr2=0x55b6ac9d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- 172.21.15.45:0/4183718186 shutdown_connections 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- 172.21.15.45:0/4183718186 wait complete. 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.247+0000 7efc6e8d9000 1 -- start start 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 Processor -- start 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6aca71cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11000 0x55b6aca721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc641482a0 con 0x55b6aca11920 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac819570 con 0x55b6ac9c7410 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6ac857570 con 0x55b6aca11000 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 10 client.0 _ll_register_callbacks cb 0x55b6ac8672c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca2d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50854/0 (socket says 172.21.15.45:50854) 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 -- 172.21.15.45:0/1752485810 learned_addr learned my addr 172.21.15.45:0/1752485810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc68a00640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6aca71cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc5be00640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11000 0x55b6aca721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 -- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11000 msgr2=0x55b6aca721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11000 0x55b6aca721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 -- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 msgr2=0x55b6aca71cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6aca71cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 -- 172.21.15.45:0/1752485810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6ac7a9bb0 con 0x55b6aca11920 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc69400640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca2d120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efc5c000f90 tx=0x7efc5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4055c32c7be8c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 1 -- 172.21.15.45:0/1752485810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c01c040 con 0x55b6aca11920 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 1 -- 172.21.15.45:0/1752485810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc5c02caa0 con 0x55b6aca11920 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc5c0139b0 con 0x55b6aca11920 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 1 -- 172.21.15.45:0/1752485810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc5c02cca0 con 0x55b6aca11920 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 10 client.6060 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55b6ac7bc030 con 0x55b6aca11920 2024-04-25T10:21:12.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 1 -- 172.21.15.45:0/1752485810 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c0139b0 con 0x55b6aca11920 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 1 -- 172.21.15.45:0/1752485810 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc5c004040 con 0x55b6aca11920 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 10 client.6060 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 10 client.6060 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 20 client.6060 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca2dae0 0x55b6aca2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 4 client.6060 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:21:12.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55b6ac822480 con 0x55b6aca2dae0 2024-04-25T10:21:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc68a00640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca2dae0 0x55b6aca2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc68a00640 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca2dae0 0x55b6aca2def0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb6c2151225dc4b server_cookie=78659602c922a99c in_seq=0 out_seq=0 2024-04-25T10:21:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.251+0000 7efc52000640 10 client.6060 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc52000640 1 -- 172.21.15.45:0/1752485810 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b6ac822480 con 0x55b6aca2dae0 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc52000640 10 client.6060 handle_command_reply: tid=0 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 client.6060 shutdown 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 msgr2=0x55b6aca2d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca2d120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efc5c000f90 tx=0x7efc5c009750 comp rx=0 tx=0).stop 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 shutdown_connections 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55b6aca2dae0 0x55b6aca2def0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b6aca11920 0x55b6aca2d120 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b6aca11000 0x55b6aca721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 --2- 172.21.15.45:0/1752485810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6ac9c7410 0x55b6aca71cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 >> 172.21.15.45:0/1752485810 conn(0x55b6ac9c33c0 msgr2=0x55b6ac9d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc52000640 0 client.6060 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 shutdown_connections 2024-04-25T10:21:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 1 -- 172.21.15.45:0/1752485810 wait complete. 2024-04-25T10:21:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6e8d9000 20 client.6060 trim_cache size 0 max 16384 2024-04-25T10:21:12.261 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:12.261 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:12.261 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "9a5f59ad-7963-4a6f-9f58-0f1be05659ca", 2024-04-25T10:21:12.261 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:12.261 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc40066f60 msgr2=0x7efc40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc40066f60 0x7efc40069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0ad000 tx=0x7efc4c004740 comp rx=0 tx=0).stop 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 msgr2=0x7efc6413bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 0x7efc6413bf20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efc5400c3f0 tx=0x7efc5400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 shutdown_connections 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efc40066f60 0x7efc40069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc641560b0 0x7efc6415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc640ceb80 0x7efc6415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 --2- 172.21.15.45:0/2053105469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64066ee0 0x7efc6413bf20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 >> 172.21.15.45:0/2053105469 conn(0x7efc640b2560 msgr2=0x7efc64145970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 shutdown_connections 2024-04-25T10:21:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.255+0000 7efc6b400640 1 -- 172.21.15.45:0/2053105469 wait complete. 2024-04-25T10:21:12.276 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '9a5f59ad-7963-4a6f-9f58-0f1be05659ca', 'mode': 'asynchronous'}' 2024-04-25T10:21:12.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:12.276 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T10:21:12.289 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T10:21:12.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.467+0000 7fbc7ba00640 1 Processor -- start 2024-04-25T10:21:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.467+0000 7fbc7ba00640 1 -- start start 2024-04-25T10:21:12.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc7ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc7415a9f0 0x7fbc740cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc7ba00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc740cbc80 0x7fbc7406d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc7ba00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc740cc2c0 con 0x7fbc740cf310 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc7ba00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740cf770 con 0x7fbc740cbc80 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc7ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740cc5a0 con 0x7fbc7415a9f0 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc740cbc80 0x7fbc7406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbc740cbc80 0x7fbc7406d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50862/0 (socket says 172.21.15.45:50862) 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 -- 172.21.15.45:0/743601063 learned_addr learned my addr 172.21.15.45:0/743601063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79000640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc7415a9f0 0x7fbc740cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 -- 172.21.15.45:0/743601063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc7415a9f0 msgr2=0x7fbc740cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc7415a9f0 0x7fbc740cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 -- 172.21.15.45:0/743601063 >> v1:172.21.15.117:6789/0 conn(0x7fbc740cf310 legacy=0x7fbc740cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79a00640 1 -- 172.21.15.45:0/743601063 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc740cc880 con 0x7fbc740cbc80 2024-04-25T10:21:12.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.471+0000 7fbc79000640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc7415a9f0 0x7fbc740cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc79a00640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cbc80 0x7fbc7406d460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009e50 tx=0x7fbc6c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a38cce8254797da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc73e00640 1 -- 172.21.15.45:0/743601063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0a5760 con 0x7fbc740cbc80 2024-04-25T10:21:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc73e00640 1 -- 172.21.15.45:0/743601063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c0a5900 con 0x7fbc740cbc80 2024-04-25T10:21:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc73e00640 1 -- 172.21.15.45:0/743601063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0a5cd0 con 0x7fbc740cbc80 2024-04-25T10:21:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- 172.21.15.45:0/743601063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cbc80 msgr2=0x7fbc7406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cbc80 0x7fbc7406d460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009e50 tx=0x7fbc6c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- 172.21.15.45:0/743601063 shutdown_connections 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cbc80 0x7fbc7406d460 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/743601063 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc7415a9f0 0x7fbc740cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- 172.21.15.45:0/743601063 >> 172.21.15.45:0/743601063 conn(0x7fbc740b2410 msgr2=0x7fbc740b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- 172.21.15.45:0/743601063 shutdown_connections 2024-04-25T10:21:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- 172.21.15.45:0/743601063 wait complete. 2024-04-25T10:21:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 Processor -- start 2024-04-25T10:21:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- start start 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc740cbc80 0x7fbc740c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 0x7fbc74140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc7415a9f0 0x7fbc74140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7406e330 con 0x7fbc740cf310 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7406e1b0 con 0x7fbc7415a9f0 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.475+0000 7fbc7ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7406e4b0 con 0x7fbc740cbc80 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 0x7fbc74140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 0x7fbc74140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50870/0 (socket says 172.21.15.45:50870) 2024-04-25T10:21:12.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc79000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc740cbc80 0x7fbc740c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 -- 172.21.15.45:0/2227109944 learned_addr learned my addr 172.21.15.45:0/2227109944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc79a00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc7415a9f0 0x7fbc74140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 -- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc740cbc80 msgr2=0x7fbc740c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc740cbc80 0x7fbc740c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 -- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc7415a9f0 msgr2=0x7fbc74140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc7415a9f0 0x7fbc74140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 -- 172.21.15.45:0/2227109944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74158680 con 0x7fbc740cf310 2024-04-25T10:21:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc6be00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 0x7fbc74140560 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c480 tx=0x7fbc5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c012830 con 0x7fbc740cf310 2024-04-25T10:21:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c017070 con 0x7fbc740cf310 2024-04-25T10:21:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c012c30 con 0x7fbc740cf310 2024-04-25T10:21:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc74158960 con 0x7fbc740cf310 2024-04-25T10:21:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc74158f20 con 0x7fbc740cf310 2024-04-25T10:21:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7404ef30 con 0x7fbc740cf310 2024-04-25T10:21:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.479+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc5c01c460 con 0x7fbc740cf310 2024-04-25T10:21:12.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.483+0000 7fbc71600640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc50066f60 0x7fbc50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.483+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc5c084d60 con 0x7fbc740cf310 2024-04-25T10:21:12.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.483+0000 7fbc79000640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc50066f60 0x7fbc50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.483+0000 7fbc79000640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc50066f60 0x7fbc50069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbc74159ac0 tx=0x7fbc64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.487+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc5c055710 con 0x7fbc740cf310 2024-04-25T10:21:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.679+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc740b1da0 con 0x7fbc740cf310 2024-04-25T10:21:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.679+0000 7fbc71600640 1 -- 172.21.15.45:0/2227109944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fbc5c0597b0 con 0x7fbc740cf310 2024-04-25T10:21:12.684 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:12.684 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:11.903745+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:12.684 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T10:21:12.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc50066f60 msgr2=0x7fbc50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc50066f60 0x7fbc50069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbc74159ac0 tx=0x7fbc64008040 comp rx=0 tx=0).stop 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 msgr2=0x7fbc74140560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 0x7fbc74140560 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00c480 tx=0x7fbc5c00c940 comp rx=0 tx=0).stop 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 shutdown_connections 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc50066f60 0x7fbc50069420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc7415a9f0 0x7fbc74140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc740cf310 0x7fbc74140560 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 --2- 172.21.15.45:0/2227109944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc740cbc80 0x7fbc740c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 >> 172.21.15.45:0/2227109944 conn(0x7fbc740b2410 msgr2=0x7fbc740d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 shutdown_connections 2024-04-25T10:21:12.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:12.683+0000 7fbc7ba00640 1 -- 172.21.15.45:0/2227109944 wait complete. 2024-04-25T10:21:12.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T10:21:12.849 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:12.849 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:12.849 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:12.864 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:12.865 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:12.865 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:12.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:12.874 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:21:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:21:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:21:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224264 s, 46.8 MB/s 2024-04-25T10:21:12.948 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:21:12.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:13.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.127+0000 7fded0200640 1 Processor -- start 2024-04-25T10:21:13.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.127+0000 7fded0200640 1 -- start start 2024-04-25T10:21:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fded0200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 0x7fdec80cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fded0200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdec815f560 0x7fdec815c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fded0200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec815c080 con 0x7fdec80669a0 2024-04-25T10:21:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fded0200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec815fd90 con 0x7fdec815f560 2024-04-25T10:21:13.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fded0200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8160070 con 0x7fdec81594b0 2024-04-25T10:21:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 0x7fdec80cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 0x7fdec80cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43426/0 (socket says 172.21.15.45:43426) 2024-04-25T10:21:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecce00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdec815f560 0x7fdec815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 -- 172.21.15.45:0/2129577370 learned_addr learned my addr 172.21.15.45:0/2129577370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdece200640 1 --1- 172.21.15.45:0/2129577370 >> v1:172.21.15.117:6789/0 conn(0x7fdec80669a0 0x7fdec80ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:60304/0 (socket says 172.21.15.45:60304) 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 -- 172.21.15.45:0/2129577370 >> v2:172.21.15.117:3300/0 conn(0x7fdec815f560 msgr2=0x7fdec815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 --2- 172.21.15.45:0/2129577370 >> v2:172.21.15.117:3300/0 conn(0x7fdec815f560 0x7fdec815c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 -- 172.21.15.45:0/2129577370 >> v1:172.21.15.117:6789/0 conn(0x7fdec80669a0 legacy=0x7fdec80ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdecd800640 1 -- 172.21.15.45:0/2129577370 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec815c2f0 con 0x7fdec81594b0 2024-04-25T10:21:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.131+0000 7fdec7e00640 1 -- 172.21.15.45:0/2129577370 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992921779 0 0) 0x7fdec815c080 con 0x7fdec80669a0 2024-04-25T10:21:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fdecd800640 1 --2- 172.21.15.45:0/2129577370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 0x7fdec80cd370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009870 tx=0x7fdebc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf91e3ff07fd99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fdec7e00640 1 -- 172.21.15.45:0/2129577370 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0aa650 con 0x7fdec81594b0 2024-04-25T10:21:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fdec7e00640 1 -- 172.21.15.45:0/2129577370 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc0af070 con 0x7fdec81594b0 2024-04-25T10:21:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fdec7e00640 1 -- 172.21.15.45:0/2129577370 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0aaa10 con 0x7fdec81594b0 2024-04-25T10:21:13.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 -- 172.21.15.45:0/2129577370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 msgr2=0x7fdec80cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 --2- 172.21.15.45:0/2129577370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 0x7fdec80cd370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009870 tx=0x7fdebc01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 -- 172.21.15.45:0/2129577370 shutdown_connections 2024-04-25T10:21:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 --2- 172.21.15.45:0/2129577370 >> v2:172.21.15.117:3300/0 conn(0x7fdec815f560 0x7fdec815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 --2- 172.21.15.45:0/2129577370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec81594b0 0x7fdec80cd370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 -- 172.21.15.45:0/2129577370 >> 172.21.15.45:0/2129577370 conn(0x7fdec80b24f0 msgr2=0x7fdec80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 -- 172.21.15.45:0/2129577370 shutdown_connections 2024-04-25T10:21:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 -- 172.21.15.45:0/2129577370 wait complete. 2024-04-25T10:21:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 Processor -- start 2024-04-25T10:21:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.135+0000 7fded0200640 1 -- start start 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 0x7fdec814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdec81594b0 0x7fdec80b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec815f560 0x7fdec81573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8159a10 con 0x7fdec815f560 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8159890 con 0x7fdec81594b0 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec815cd50 con 0x7fdec80669a0 2024-04-25T10:21:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecd800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 0x7fdec814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdece200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec815f560 0x7fdec81573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdec81594b0 0x7fdec80b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecd800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 0x7fdec814f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43428/0 (socket says 172.21.15.45:43428) 2024-04-25T10:21:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecd800640 1 -- 172.21.15.45:0/1140926970 learned_addr learned my addr 172.21.15.45:0/1140926970 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 -- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 msgr2=0x7fdec814f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 0x7fdec814f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 -- 172.21.15.45:0/1140926970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec815f560 msgr2=0x7fdec81573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec815f560 0x7fdec81573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 -- 172.21.15.45:0/1140926970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec815d3d0 con 0x7fdec81594b0 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdece200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec815f560 0x7fdec81573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecd800640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 0x7fdec814f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdecce00640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdec81594b0 0x7fdec80b4490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e870 tx=0x7fdeb400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017070 con 0x7fdec81594b0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb400ee60 con 0x7fdec81594b0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec814f790 con 0x7fdec81594b0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400cb60 con 0x7fdec81594b0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec815d950 con 0x7fdec81594b0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec804ef30 con 0x7fdec81594b0 2024-04-25T10:21:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.139+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdeb401a070 con 0x7fdec81594b0 2024-04-25T10:21:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.143+0000 7fdec5600640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdea8066f60 0x7fdea8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.143+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdeb4084e80 con 0x7fdec81594b0 2024-04-25T10:21:13.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.143+0000 7fdecd800640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdea8066f60 0x7fdea8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.143+0000 7fdecd800640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdea8066f60 0x7fdea8069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009870 tx=0x7fdebc0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.147+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdeb40557a0 con 0x7fdec81594b0 2024-04-25T10:21:13.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.339+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdec80b2b70 con 0x7fdec81594b0 2024-04-25T10:21:13.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fdec5600640 1 -- 172.21.15.45:0/1140926970 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fdeb4059840 con 0x7fdec81594b0 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:11.903745+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:13.345 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdea8066f60 msgr2=0x7fdea8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdea8066f60 0x7fdea8069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009870 tx=0x7fdebc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdec81594b0 msgr2=0x7fdec80b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdec81594b0 0x7fdec80b4490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e870 tx=0x7fdeb400ed30 comp rx=0 tx=0).stop 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 shutdown_connections 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdea8066f60 0x7fdea8069420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdec815f560 0x7fdec81573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdec81594b0 0x7fdec80b4490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 --2- 172.21.15.45:0/1140926970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdec80669a0 0x7fdec814f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 >> 172.21.15.45:0/1140926970 conn(0x7fdec80b24f0 msgr2=0x7fdec8165d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 shutdown_connections 2024-04-25T10:21:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.343+0000 7fded0200640 1 -- 172.21.15.45:0/1140926970 wait complete. 2024-04-25T10:21:13.366 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:21:13.506 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:13.506 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:13.506 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:13.521 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:13.522 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:21:13.522 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:21:13.522 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:21:13.522 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:21:13.522 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T10:21:13.522 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:21:13.589 INFO:teuthology.orchestra.run.smithi045.stdout:1099511630189 2024-04-25T10:21:13.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:21:13.598 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:13.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:21:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.779+0000 7f07a5800640 1 Processor -- start 2024-04-25T10:21:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.779+0000 7f07a5800640 1 -- start start 2024-04-25T10:21:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f07a5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015b150 0x7f07a00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f07a5800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07a006c0d0 0x7f07a006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f07a5800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00b2fd0 con 0x7f07a015a0e0 2024-04-25T10:21:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f07a5800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00aef90 con 0x7f07a006c0d0 2024-04-25T10:21:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f07a5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00af1d0 con 0x7f07a015b150 2024-04-25T10:21:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07a006c0d0 0x7f07a006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f07a015a0e0 0x7f07a00b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36542/0 (socket says 172.21.15.45:36542) 2024-04-25T10:21:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079ea00640 1 -- 172.21.15.45:0/3105895863 learned_addr learned my addr 172.21.15.45:0/3105895863 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f0797e00640 1 --2- 172.21.15.45:0/3105895863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015b150 0x7f07a00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079f400640 1 -- 172.21.15.45:0/3105895863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015b150 msgr2=0x7f07a00aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079f400640 1 --2- 172.21.15.45:0/3105895863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015b150 0x7f07a00aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079f400640 1 -- 172.21.15.45:0/3105895863 >> v1:172.21.15.45:6789/0 conn(0x7f07a015a0e0 legacy=0x7f07a00b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079f400640 1 -- 172.21.15.45:0/3105895863 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00b32b0 con 0x7f07a006c0d0 2024-04-25T10:21:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.783+0000 7f079e000640 1 -- 172.21.15.45:0/3105895863 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317000696 0 0) 0x7f07a00b2fd0 con 0x7f07a015a0e0 2024-04-25T10:21:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079f400640 1 --2- 172.21.15.45:0/3105895863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a006c0d0 0x7f07a006c520 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0798009d90 tx=0x7f079801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9951b1052189e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079e000640 1 -- 172.21.15.45:0/3105895863 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980b2070 con 0x7f07a006c0d0 2024-04-25T10:21:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079e000640 1 -- 172.21.15.45:0/3105895863 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07980a57a0 con 0x7f07a006c0d0 2024-04-25T10:21:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079e000640 1 -- 172.21.15.45:0/3105895863 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980a5b70 con 0x7f07a006c0d0 2024-04-25T10:21:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- 172.21.15.45:0/3105895863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a006c0d0 msgr2=0x7f07a006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 --2- 172.21.15.45:0/3105895863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a006c0d0 0x7f07a006c520 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0798009d90 tx=0x7f079801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- 172.21.15.45:0/3105895863 shutdown_connections 2024-04-25T10:21:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 --2- 172.21.15.45:0/3105895863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a006c0d0 0x7f07a006c520 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 --2- 172.21.15.45:0/3105895863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015b150 0x7f07a00aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- 172.21.15.45:0/3105895863 >> 172.21.15.45:0/3105895863 conn(0x7f07a00669a0 msgr2=0x7f07a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- 172.21.15.45:0/3105895863 shutdown_connections 2024-04-25T10:21:13.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- 172.21.15.45:0/3105895863 wait complete. 2024-04-25T10:21:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 Processor -- start 2024-04-25T10:21:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- start start 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a006c0d0 0x7f07a01580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015a0e0 0x7f07a01f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 0x7f07a00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b4620 con 0x7f07a015b150 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b44a0 con 0x7f07a006c0d0 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f07a5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cf700 con 0x7f07a015a0e0 2024-04-25T10:21:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 0x7f07a00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 0x7f07a00ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50884/0 (socket says 172.21.15.45:50884) 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079f400640 1 -- 172.21.15.45:0/47445430 learned_addr learned my addr 172.21.15.45:0/47445430 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.787+0000 7f079ea00640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a006c0d0 0x7f07a01580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f0797e00640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015a0e0 0x7f07a01f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f079f400640 1 -- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015a0e0 msgr2=0x7f07a01f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f079f400640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015a0e0 0x7f07a01f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f079f400640 1 -- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a006c0d0 msgr2=0x7f07a01580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f079f400640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a006c0d0 0x7f07a01580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f079f400640 1 -- 172.21.15.45:0/47445430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0151050 con 0x7f07a015b150 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f079f400640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 0x7f07a00ad840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f079801af90 tx=0x7f0798019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f0796a00640 1 -- 172.21.15.45:0/47445430 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980b2050 con 0x7f07a015b150 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f0796a00640 1 -- 172.21.15.45:0/47445430 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07980be070 con 0x7f07a015b150 2024-04-25T10:21:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f0796a00640 1 -- 172.21.15.45:0/47445430 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798009450 con 0x7f07a015b150 2024-04-25T10:21:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a0151330 con 0x7f07a015b150 2024-04-25T10:21:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a01518f0 con 0x7f07a015b150 2024-04-25T10:21:13.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.791+0000 7f0796a00640 1 -- 172.21.15.45:0/47445430 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07980adc20 con 0x7f07a015b150 2024-04-25T10:21:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f0796a00640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f077c066f60 0x7f077c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f0796a00640 1 -- 172.21.15.45:0/47445430 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0798120140 con 0x7f07a015b150 2024-04-25T10:21:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079ea00640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f077c066f60 0x7f077c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 Processor -- start 2024-04-25T10:21:13.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 -- start start 2024-04-25T10:21:13.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079ea00640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f077c066f60 0x7f077c069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0790007fd0 tx=0x7f0790007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146891560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891aa0 0x5611468470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561146694f20 con 0x561146847600 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114661d000 con 0x561146891180 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561146690fc0 con 0x561146891aa0 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146891560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50898/0 (socket says 172.21.15.45:50898) 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 -- 172.21.15.45:0/3926762719 learned_addr learned my addr 172.21.15.45:0/3926762719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f0797e00640 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891aa0 0x5611468470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 -- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891aa0 msgr2=0x5611468470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891aa0 0x5611468470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 -- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 msgr2=0x561146891560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146891560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 -- 172.21.15.45:0/3926762719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561146670ba0 con 0x561146847600 2024-04-25T10:21:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f079f400640 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468479e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f079811e690 tx=0x7f07980f0460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab5d805461ca5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f078f400640 1 -- 172.21.15.45:0/3926762719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980f4ba0 con 0x561146847600 2024-04-25T10:21:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f078f400640 1 -- 172.21.15.45:0/3926762719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07980b3040 con 0x561146847600 2024-04-25T10:21:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f078f400640 1 -- 172.21.15.45:0/3926762719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980f9160 con 0x561146847600 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 -- 172.21.15.45:0/3926762719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 msgr2=0x5611468479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.795+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468479e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f079811e690 tx=0x7f07980f0460 comp rx=0 tx=0).stop 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- 172.21.15.45:0/3926762719 shutdown_connections 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468479e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891aa0 0x5611468470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/3926762719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146891560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- 172.21.15.45:0/3926762719 >> 172.21.15.45:0/3926762719 conn(0x56114684d760 msgr2=0x5611468459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- 172.21.15.45:0/3926762719 shutdown_connections 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- 172.21.15.45:0/3926762719 wait complete. 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- start start 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 Processor -- start 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146847600 0x56114684c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x56114684cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146891aa0 0x56114687f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611467529c0 con 0x561146891aa0 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114664f5d0 con 0x561146891180 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561146694f20 con 0x561146847600 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 10 client.0 _ll_register_callbacks cb 0x561146832a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x56114684cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f079f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146891aa0 0x56114687f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f079ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146847600 0x56114684c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x56114684cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33084/0 (socket says 172.21.15.45:33084) 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 -- 172.21.15.45:0/443721141 learned_addr learned my addr 172.21.15.45:0/443721141 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 -- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146847600 msgr2=0x56114684c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146847600 0x56114684c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 -- 172.21.15.45:0/443721141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146891aa0 msgr2=0x56114687f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146891aa0 0x56114687f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 -- 172.21.15.45:0/443721141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611466db5a0 con 0x561146891180 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f079f400640 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146891aa0 0x56114687f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0797e00640 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x56114684cbe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0788014770 tx=0x7f0788014c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae091920f6b2acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0783e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0783e00640 1 -- 172.21.15.45:0/443721141 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788022060 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0783e00640 1 -- 172.21.15.45:0/443721141 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0788012ac0 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07980b3040 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0783e00640 1 -- 172.21.15.45:0/443721141 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0788028050 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 10 client.5857 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x56114666c2b0 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0783e00640 1 -- 172.21.15.45:0/443721141 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07980b3040 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f0783e00640 1 -- 172.21.15.45:0/443721141 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f078802b020 con 0x561146891180 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 10 client.5857 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.799+0000 7f07a8d7c000 10 client.5857 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 20 client.5857 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56114687ee60 0x56114687fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 4 client.5857 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:13.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561146630c70 con 0x56114687ee60 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f079ea00640 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56114687ee60 0x56114687fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f079ea00640 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56114687ee60 0x56114687fe60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89df9306c2df2c58 server_cookie=f7cf50f7b8602004 in_seq=0 out_seq=0 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f0783e00640 10 client.5857 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f0783e00640 1 -- 172.21.15.45:0/443721141 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561146630c70 con 0x56114687ee60 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f0783e00640 10 client.5857 handle_command_reply: tid=0 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 client.5857 shutdown 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 msgr2=0x56114684cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x56114684cbe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0788014770 tx=0x7f0788014c30 comp rx=0 tx=0).stop 2024-04-25T10:21:13.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 shutdown_connections 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56114687ee60 0x56114687fe60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146891aa0 0x56114687f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f0783e00640 0 client.5857 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x56114684cbe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/443721141 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146847600 0x56114684c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 >> 172.21.15.45:0/443721141 conn(0x56114684d760 msgr2=0x5611468460b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 shutdown_connections 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 1 -- 172.21.15.45:0/443721141 wait complete. 2024-04-25T10:21:13.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.803+0000 7f07a8d7c000 20 client.5857 trim_cache size 0 max 16384 2024-04-25T10:21:13.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 Processor -- start 2024-04-25T10:21:13.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 -- start start 2024-04-25T10:21:13.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56114684d760 0x5611468a7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x561146846120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146846660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0202f00 con 0x561146847600 2024-04-25T10:21:13.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01f79a0 con 0x561146891180 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0152980 con 0x56114684d760 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x561146846120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x561146846120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50920/0 (socket says 172.21.15.45:50920) 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 -- 172.21.15.45:0/2574493858 learned_addr learned my addr 172.21.15.45:0/2574493858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f079ea00640 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56114684d760 0x5611468a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f079f400640 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146846660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 -- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56114684d760 msgr2=0x5611468a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56114684d760 0x5611468a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 -- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 msgr2=0x561146846660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146846660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.815+0000 7f0797e00640 1 -- 172.21.15.45:0/2574493858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611468801d0 con 0x561146847600 2024-04-25T10:21:13.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x561146846120 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0788014c70 tx=0x7f0788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbbaa37d45da43b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f078ea00640 1 -- 172.21.15.45:0/2574493858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788022070 con 0x561146847600 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f078ea00640 1 -- 172.21.15.45:0/2574493858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788012700 con 0x561146847600 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f078ea00640 1 -- 172.21.15.45:0/2574493858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788014350 con 0x561146847600 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2574493858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 msgr2=0x561146846120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x561146846120 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0788014c70 tx=0x7f0788017000 comp rx=0 tx=0).stop 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2574493858 shutdown_connections 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561146891180 0x561146846660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x561146846120 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2574493858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56114684d760 0x5611468a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2574493858 >> 172.21.15.45:0/2574493858 conn(0x56114687f5b0 msgr2=0x56114689e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2574493858 shutdown_connections 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2574493858 wait complete. 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- start start 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 Processor -- start 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114684d760 0x5611468f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891180 0x5611468a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611466587b0 con 0x561146847600 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01f73a0 con 0x56114684d760 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611467529c0 con 0x561146891180 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f07a8d7c000 10 client.0 _ll_register_callbacks cb 0x561146813600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f079ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114684d760 0x5611468f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f079ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50922/0 (socket says 172.21.15.45:50922) 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f079ea00640 1 -- 172.21.15.45:0/2997292328 learned_addr learned my addr 172.21.15.45:0/2997292328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f079f400640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891180 0x5611468a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 -- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891180 msgr2=0x5611468a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891180 0x5611468a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 -- 172.21.15.45:0/2997292328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 msgr2=0x5611468f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 -- 172.21.15.45:0/2997292328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114661d250 con 0x56114684d760 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f079ea00640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0797e00640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114684d760 0x5611468f18f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5611466ed5a0 tx=0x7f07880046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f1a156856b4623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.819+0000 7f0783e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 1 -- 172.21.15.45:0/2997292328 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788022070 con 0x56114684d760 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0788012700 con 0x56114684d760 2024-04-25T10:21:13.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 1 -- 172.21.15.45:0/2997292328 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0788005bb0 con 0x56114684d760 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 1 -- 172.21.15.45:0/2997292328 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078803c050 con 0x56114684d760 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 10 client.5863 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x561146629b90 con 0x56114684d760 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 1 -- 172.21.15.45:0/2997292328 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788012700 con 0x56114684d760 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 1 -- 172.21.15.45:0/2997292328 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f078803f020 con 0x56114684d760 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 10 client.5863 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 10 client.5863 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 20 client.5863 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561146891aa0 0x5611468a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 4 client.5863 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:21:13.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5611466a2460 con 0x561146891aa0 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f079ea00640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561146891aa0 0x5611468a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f079ea00640 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561146891aa0 0x5611468a6ac0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda879bb6901e89d server_cookie=c99b8ea2e0f80368 in_seq=0 out_seq=0 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 10 client.5863 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 1 -- 172.21.15.45:0/2997292328 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5611466a2460 con 0x561146891aa0 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 10 client.5863 handle_command_reply: tid=0 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 client.5863 shutdown 2024-04-25T10:21:13.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114684d760 msgr2=0x5611468f18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114684d760 0x5611468f18f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5611466ed5a0 tx=0x7f07880046d0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 shutdown_connections 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x561146891aa0 0x5611468a6ac0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561146891180 0x5611468a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114684d760 0x5611468f18f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f0783e00640 0 client.5863 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 --2- 172.21.15.45:0/2997292328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561146847600 0x5611468f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 >> 172.21.15.45:0/2997292328 conn(0x56114687f5b0 msgr2=0x561146845110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 shutdown_connections 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 1 -- 172.21.15.45:0/2997292328 wait complete. 2024-04-25T10:21:13.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a8d7c000 20 client.5863 trim_cache size 0 max 16384 2024-04-25T10:21:13.832 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T10:21:13.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f077c066f60 msgr2=0x7f077c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f077c066f60 0x7f077c069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0790007fd0 tx=0x7f0790007d60 comp rx=0 tx=0).stop 2024-04-25T10:21:13.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 msgr2=0x7f07a00ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 0x7f07a00ad840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f079801af90 tx=0x7f0798019680 comp rx=0 tx=0).stop 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 shutdown_connections 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f077c066f60 0x7f077c069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07a015b150 0x7f07a00ad840 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07a015a0e0 0x7f07a01f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 --2- 172.21.15.45:0/47445430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a006c0d0 0x7f07a01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 >> 172.21.15.45:0/47445430 conn(0x7f07a00669a0 msgr2=0x7f07a015b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 shutdown_connections 2024-04-25T10:21:13.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.823+0000 7f07a5800640 1 -- 172.21.15.45:0/47445430 wait complete. 2024-04-25T10:21:13.846 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096d.00000000 parent 2024-04-25T10:21:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 Processor -- start 2024-04-25T10:21:13.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- start start 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636164fb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164860e0 0x5636164f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164fbf10 0x56361646bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563616513f10 con 0x5636164860e0 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636163e49f0 con 0x5636164fbf10 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563616514090 con 0x563616485d00 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636164fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f3800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164fbf10 0x56361646bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636164fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43486/0 (socket says 172.21.15.45:43486) 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 -- 172.21.15.45:0/2302329608 learned_addr learned my addr 172.21.15.45:0/2302329608 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2400640 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164860e0 0x5636164f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 -- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164fbf10 msgr2=0x56361646bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164fbf10 0x56361646bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 -- 172.21.15.45:0/2302329608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164860e0 msgr2=0x5636164f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164860e0 0x5636164f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 -- 172.21.15.45:0/2302329608 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563616516470 con 0x563616485d00 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f3800640 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164fbf10 0x56361646bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f2e00640 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636164fb280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f97e40099b0 tx=0x7f97e4005a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc42ab79a9081fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f1000640 1 -- 172.21.15.45:0/2302329608 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4019070 con 0x563616485d00 2024-04-25T10:21:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f1000640 1 -- 172.21.15.45:0/2302329608 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4014480 con 0x563616485d00 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f1000640 1 -- 172.21.15.45:0/2302329608 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4008e60 con 0x563616485d00 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 msgr2=0x5636164fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636164fb280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f97e40099b0 tx=0x7f97e4005a10 comp rx=0 tx=0).stop 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- 172.21.15.45:0/2302329608 shutdown_connections 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164fbf10 0x56361646bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164860e0 0x5636164f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/2302329608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636164fb280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- 172.21.15.45:0/2302329608 >> 172.21.15.45:0/2302329608 conn(0x563616468760 msgr2=0x563616512920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- 172.21.15.45:0/2302329608 shutdown_connections 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.891+0000 7f97f5b0f080 1 -- 172.21.15.45:0/2302329608 wait complete. 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 Processor -- start 2024-04-25T10:21:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 -- start start 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636165aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164860e0 0x56361651cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164fbf10 0x563616511e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563616516a20 con 0x5636164fbf10 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636164fb7c0 con 0x5636164860e0 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563616441190 con 0x563616485d00 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164fbf10 0x563616511e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164fbf10 0x563616511e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50934/0 (socket says 172.21.15.45:50934) 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 -- 172.21.15.45:0/1542693456 learned_addr learned my addr 172.21.15.45:0/1542693456 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f2e00640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636165aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f2400640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164860e0 0x56361651cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 -- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 msgr2=0x5636165aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636165aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 -- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164860e0 msgr2=0x56361651cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636164860e0 0x56361651cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 -- 172.21.15.45:0/1542693456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636165199b0 con 0x5636164fbf10 2024-04-25T10:21:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f2e00640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563616485d00 0x5636165aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f3800640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164fbf10 0x563616511e00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0b1040 tx=0x7f97ec01a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97eb400640 1 -- 172.21.15.45:0/1542693456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0bc020 con 0x5636164fbf10 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97eb400640 1 -- 172.21.15.45:0/1542693456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec0b4040 con 0x5636164fbf10 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563616519c50 con 0x5636164fbf10 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97eb400640 1 -- 172.21.15.45:0/1542693456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0b3040 con 0x5636164fbf10 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.895+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56361651a190 con 0x5636164fbf10 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97eb400640 1 -- 172.21.15.45:0/1542693456 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97ec0bf020 con 0x5636164fbf10 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97eb400640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f97d4065940 0x7f97d4067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97eb400640 1 -- 172.21.15.45:0/1542693456 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97ec120a30 con 0x5636164fbf10 2024-04-25T10:21:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97f2e00640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f97d4065940 0x7f97d4067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x5636165194a0 0x56361651a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 --> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] -- osd_op(unknown.0.0:1 7.21 7:844f785e:::1000000096d.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50) v9 -- 0x563616512340 con 0x5636165194a0 2024-04-25T10:21:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97f2400640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x5636165194a0 0x56361651a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97f2e00640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f97d4065940 0x7f97d4067e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97e4017040 tx=0x7f97e400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.899+0000 7f97f2400640 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x5636165194a0 0x56361651a8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f2400640 1 -- 172.21.15.45:0/1542693456 <== osd.6 v2:172.21.15.117:6816/2565393264 1 ==== osd_op_reply(1 1000000096d.00000000 [rmxattr] v50'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f97e000d040 con 0x5636165194a0 2024-04-25T10:21:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x5636165194a0 msgr2=0x56361651a8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x5636165194a0 0x56361651a8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f97d4065940 msgr2=0x7f97d4067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f97d4065940 0x7f97d4067e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97e4017040 tx=0x7f97e400b040 comp rx=0 tx=0).stop 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164fbf10 msgr2=0x563616511e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 --2- 172.21.15.45:0/1542693456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636164fbf10 0x563616511e00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0b1040 tx=0x7f97ec01a640 comp rx=0 tx=0).stop 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f3800640 1 -- 172.21.15.45:0/1542693456 reap_dead start 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 shutdown_connections 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 >> 172.21.15.45:0/1542693456 conn(0x563616468760 msgr2=0x563616515040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 shutdown_connections 2024-04-25T10:21:13.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:13.903+0000 7f97f5b0f080 1 -- 172.21.15.45:0/1542693456 wait complete. 2024-04-25T10:21:13.910 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:21:13.910 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:13.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:21:14.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.051+0000 7fbae6e00640 1 Processor -- start 2024-04-25T10:21:14.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.051+0000 7fbae6e00640 1 -- start start 2024-04-25T10:21:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbae006d850 0x7fbae00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae006e040 con 0x7fbae015a7b0 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae006e320 con 0x7fbae006d850 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00cd0e0 con 0x7fbae0072460 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbae006d850 0x7fbae00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbae006d850 0x7fbae00cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50948/0 (socket says 172.21.15.45:50948) 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 -- 172.21.15.45:0/2509130473 learned_addr learned my addr 172.21.15.45:0/2509130473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae5000640 1 --2- 172.21.15.45:0/2509130473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 -- 172.21.15.45:0/2509130473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 msgr2=0x7fbae006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 --2- 172.21.15.45:0/2509130473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae006d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 -- 172.21.15.45:0/2509130473 >> v1:172.21.15.117:6789/0 conn(0x7fbae015a7b0 legacy=0x7fbae0071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 -- 172.21.15.45:0/2509130473 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00cd3c0 con 0x7fbae006d850 2024-04-25T10:21:14.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.055+0000 7fbae6400640 1 --2- 172.21.15.45:0/2509130473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae006d850 0x7fbae00cd8e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009e50 tx=0x7fbad801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aeac7832bf6b63f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbadfe00640 1 -- 172.21.15.45:0/2509130473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80a5920 con 0x7fbae006d850 2024-04-25T10:21:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbadfe00640 1 -- 172.21.15.45:0/2509130473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad80a5ac0 con 0x7fbae006d850 2024-04-25T10:21:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbadfe00640 1 -- 172.21.15.45:0/2509130473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80ad3f0 con 0x7fbae006d850 2024-04-25T10:21:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- 172.21.15.45:0/2509130473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae006d850 msgr2=0x7fbae00cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 --2- 172.21.15.45:0/2509130473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae006d850 0x7fbae00cd8e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009e50 tx=0x7fbad801ca30 comp rx=0 tx=0).stop 2024-04-25T10:21:14.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- 172.21.15.45:0/2509130473 shutdown_connections 2024-04-25T10:21:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 --2- 172.21.15.45:0/2509130473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae006d850 0x7fbae00cd8e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 --2- 172.21.15.45:0/2509130473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- 172.21.15.45:0/2509130473 >> 172.21.15.45:0/2509130473 conn(0x7fbae00cfe80 msgr2=0x7fbae00d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- 172.21.15.45:0/2509130473 shutdown_connections 2024-04-25T10:21:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- 172.21.15.45:0/2509130473 wait complete. 2024-04-25T10:21:14.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 Processor -- start 2024-04-25T10:21:14.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- start start 2024-04-25T10:21:14.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae006d850 0x7fbae014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 0x7fbae01595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00ce7b0 con 0x7fbae015a7b0 2024-04-25T10:21:14.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00ce630 con 0x7fbae006d850 2024-04-25T10:21:14.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00ce930 con 0x7fbae0072460 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 0x7fbae01595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 0x7fbae01595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50964/0 (socket says 172.21.15.45:50964) 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae6400640 1 -- 172.21.15.45:0/3034989802 learned_addr learned my addr 172.21.15.45:0/3034989802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae5000640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.059+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae006d850 0x7fbae014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6400640 1 -- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 msgr2=0x7fbae0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6400640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae0155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6400640 1 -- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae006d850 msgr2=0x7fbae014b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6400640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae006d850 0x7fbae014b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6400640 1 -- 172.21.15.45:0/3034989802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae014b740 con 0x7fbae015a7b0 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae5000640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6400640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 0x7fbae01595d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbad801a660 tx=0x7fbad80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbadd600640 1 -- 172.21.15.45:0/3034989802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80a5920 con 0x7fbae015a7b0 2024-04-25T10:21:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbadd600640 1 -- 172.21.15.45:0/3034989802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad8007c70 con 0x7fbae015a7b0 2024-04-25T10:21:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbadd600640 1 -- 172.21.15.45:0/3034989802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8009510 con 0x7fbae015a7b0 2024-04-25T10:21:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae0157830 con 0x7fbae015a7b0 2024-04-25T10:21:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae0156790 con 0x7fbae015a7b0 2024-04-25T10:21:14.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.063+0000 7fbadd600640 1 -- 172.21.15.45:0/3034989802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbad80c0020 con 0x7fbae015a7b0 2024-04-25T10:21:14.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbadd600640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbac0066f60 0x7fbac0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbadd600640 1 -- 172.21.15.45:0/3034989802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbad8120090 con 0x7fbae015a7b0 2024-04-25T10:21:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbac0066f60 0x7fbac0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 Processor -- start 2024-04-25T10:21:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- start start 2024-04-25T10:21:14.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbac0066f60 0x7fbac0069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbae015a1a0 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f31f0 0x55fd634f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd632f6ce0 con 0x55fd634f3b10 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6327f530 con 0x55fd634a6920 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd632f2d80 con 0x55fd634f31f0 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50974/0 (socket says 172.21.15.45:50974) 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 -- 172.21.15.45:0/1747392446 learned_addr learned my addr 172.21.15.45:0/1747392446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5a00640 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f31f0 0x55fd634f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae6400640 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 -- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f31f0 msgr2=0x55fd634f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f31f0 0x55fd634f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 -- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 msgr2=0x55fd634a6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 -- 172.21.15.45:0/1747392446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd632f1cd0 con 0x55fd634f3b10 2024-04-25T10:21:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbae5000640 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634a63e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbacc007960 tx=0x7fbacc007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72ed51a8a9985f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbad2000640 1 -- 172.21.15.45:0/1747392446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc018020 con 0x55fd634f3b10 2024-04-25T10:21:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbad2000640 1 -- 172.21.15.45:0/1747392446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00fa80 con 0x55fd634f3b10 2024-04-25T10:21:14.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbad2000640 1 -- 172.21.15.45:0/1747392446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00fe30 con 0x55fd634f3b10 2024-04-25T10:21:14.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/1747392446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 msgr2=0x55fd634a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634a63e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbacc007960 tx=0x7fbacc007e20 comp rx=0 tx=0).stop 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/1747392446 shutdown_connections 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634a63e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/1747392446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f31f0 0x55fd634f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/1747392446 >> 172.21.15.45:0/1747392446 conn(0x55fd634d3f90 msgr2=0x55fd634d6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/1747392446 shutdown_connections 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.067+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/1747392446 wait complete. 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- start start 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 Processor -- start 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634a6920 0x55fd634aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd632b1bc0 con 0x55fd634f3b10 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd632fb310 con 0x55fd634f31f0 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd632c30b0 con 0x55fd634a6920 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 10 client.0 _ll_register_callbacks cb 0x55fd634944a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50978/0 (socket says 172.21.15.45:50978) 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 -- 172.21.15.45:0/3871141453 learned_addr learned my addr 172.21.15.45:0/3871141453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae5000640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 -- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634a6920 msgr2=0x55fd634aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634a6920 0x55fd634aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634a6920 0x55fd634aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 -- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 msgr2=0x55fd634aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 -- 172.21.15.45:0/3871141453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd63304950 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae5000640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634a6920 0x55fd634aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae6400640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634acfb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbad8019640 tx=0x7fbad8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be1ee022c1a727a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbac6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbac6a00640 1 -- 172.21.15.45:0/3871141453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80f0d60 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbac6a00640 1 -- 172.21.15.45:0/3871141453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbad80f8b80 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbacc00fa80 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbac6a00640 1 -- 172.21.15.45:0/3871141453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad80ac3a0 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 10 client.6108 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55fd63373790 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbac6a00640 1 -- 172.21.15.45:0/3871141453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00fa80 con 0x55fd634f3b10 2024-04-25T10:21:14.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbac6a00640 1 -- 172.21.15.45:0/3871141453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbad80efd50 con 0x55fd634f3b10 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 10 client.6108 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 10 client.6108 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 20 client.6108 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd634add50 0x55fd635052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 4 client.6108 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd63292b80 con 0x55fd634add50 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.071+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd634add50 0x55fd635052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd634add50 0x55fd635052a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93db21dfdfe980e server_cookie=bb22c406625eca86 in_seq=0 out_seq=0 2024-04-25T10:21:14.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbac6a00640 10 client.6108 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbac6a00640 1 -- 172.21.15.45:0/3871141453 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd63292b80 con 0x55fd634add50 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbac6a00640 10 client.6108 handle_command_reply: tid=0 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 client.6108 shutdown 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 msgr2=0x55fd634acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634acfb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbad8019640 tx=0x7fbad8009810 comp rx=0 tx=0).stop 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 shutdown_connections 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd634add50 0x55fd635052a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f3b10 0x55fd634acfb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3871141453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634a6920 0x55fd634aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbac6a00640 0 client.6108 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 >> 172.21.15.45:0/3871141453 conn(0x55fd634d3f90 msgr2=0x55fd634a5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 shutdown_connections 2024-04-25T10:21:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3871141453 wait complete. 2024-04-25T10:21:14.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.075+0000 7fbaeb9a4000 20 client.6108 trim_cache size 0 max 16384 2024-04-25T10:21:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 Processor -- start 2024-04-25T10:21:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- start start 2024-04-25T10:21:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 0x55fd634a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd634a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0202900 con 0x55fd634a6920 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0201ac0 con 0x55fd634f31f0 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae015b9c0 con 0x55fd634f3b10 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 0x55fd634a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 0x55fd634a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50988/0 (socket says 172.21.15.45:50988) 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 -- 172.21.15.45:0/3620883960 learned_addr learned my addr 172.21.15.45:0/3620883960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5000640 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae6400640 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd634a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 -- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 msgr2=0x55fd634a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd634a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 -- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 msgr2=0x55fd634a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 -- 172.21.15.45:0/3620883960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd632bc6f0 con 0x55fd634a6920 2024-04-25T10:21:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5000640 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbae5a00640 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 0x55fd634a4b40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbad4005290 tx=0x7fbad4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cc633fafdf7aff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbad1600640 1 -- 172.21.15.45:0/3620883960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4003b00 con 0x55fd634a6920 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbad1600640 1 -- 172.21.15.45:0/3620883960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad4003ca0 con 0x55fd634a6920 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbad1600640 1 -- 172.21.15.45:0/3620883960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4003e50 con 0x55fd634a6920 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3620883960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 msgr2=0x55fd634a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 0x55fd634a4b40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbad4005290 tx=0x7fbad4017040 comp rx=0 tx=0).stop 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3620883960 shutdown_connections 2024-04-25T10:21:14.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd634a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634f31f0 0x55fd634a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/3620883960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634a6920 0x55fd634a4b40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3620883960 >> 172.21.15.45:0/3620883960 conn(0x55fd634d3f90 msgr2=0x55fd635074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3620883960 shutdown_connections 2024-04-25T10:21:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.091+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/3620883960 wait complete. 2024-04-25T10:21:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- start start 2024-04-25T10:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 Processor -- start 2024-04-25T10:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 0x55fd635062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd63506830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd632f3d50 con 0x55fd634f31f0 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae015bfc0 con 0x55fd634a6920 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd632f5cb0 con 0x55fd634f3b10 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 10 client.0 _ll_register_callbacks cb 0x7fbacc007380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 0x55fd635062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 0x55fd635062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50990/0 (socket says 172.21.15.45:50990) 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 -- 172.21.15.45:0/2496733250 learned_addr learned my addr 172.21.15.45:0/2496733250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae6400640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd63506830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5a00640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 -- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 msgr2=0x55fd63506830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd63506830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 -- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 msgr2=0x55fd634a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 -- 172.21.15.45:0/2496733250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd632b2c80 con 0x55fd634f31f0 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5a00640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5000640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 0x55fd635062f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x55fd634a0970 tx=0x7fbacc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b455d4c40cd365e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbac6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbac6a00640 1 -- 172.21.15.45:0/2496733250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc018040 con 0x55fd634f31f0 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbac6a00640 1 -- 172.21.15.45:0/2496733250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbacc00e040 con 0x55fd634f31f0 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbad4003ca0 con 0x55fd634f31f0 2024-04-25T10:21:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbac6a00640 1 -- 172.21.15.45:0/2496733250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc031050 con 0x55fd634f31f0 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 10 client.6114 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55fd62fb3600 con 0x55fd634f31f0 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbac6a00640 1 -- 172.21.15.45:0/2496733250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad4003ca0 con 0x55fd634f31f0 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbac6a00640 1 -- 172.21.15.45:0/2496733250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbacc034020 con 0x55fd634f31f0 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 10 client.6114 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 10 client.6114 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 20 client.6114 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd63553780 0x55fd63553b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 4 client.6114 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55fd634d3de0 con 0x55fd63553780 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.095+0000 7fbae5a00640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd63553780 0x55fd63553b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae5a00640 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd63553780 0x55fd63553b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5266a3ec578ec2fc server_cookie=dd175650d9edd9b0 in_seq=0 out_seq=0 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbac6a00640 10 client.6114 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbac6a00640 1 -- 172.21.15.45:0/2496733250 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fd634d3de0 con 0x55fd63553780 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbac6a00640 10 client.6114 handle_command_reply: tid=0 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 client.6114 shutdown 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 msgr2=0x55fd635062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 0x55fd635062f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x55fd634a0970 tx=0x7fbacc004740 comp rx=0 tx=0).stop 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 shutdown_connections 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55fd63553780 0x55fd63553b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55fd634f3b10 0x55fd63506830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fd634f31f0 0x55fd635062f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 --2- 172.21.15.45:0/2496733250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd634a6920 0x55fd634a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbac6a00640 0 client.6114 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 >> 172.21.15.45:0/2496733250 conn(0x55fd634d3f90 msgr2=0x55fd634ac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 shutdown_connections 2024-04-25T10:21:14.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 1 -- 172.21.15.45:0/2496733250 wait complete. 2024-04-25T10:21:14.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbaeb9a4000 20 client.6114 trim_cache size 0 max 16384 2024-04-25T10:21:14.108 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "9bbea666-d6a1-4281-a8f5-5d9b493624b5", 2024-04-25T10:21:14.108 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:14.108 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbac0066f60 msgr2=0x7fbac0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbac0066f60 0x7fbac0069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbae015a1a0 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2024-04-25T10:21:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 msgr2=0x7fbae01595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 0x7fbae01595d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbad801a660 tx=0x7fbad80197d0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 shutdown_connections 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbac0066f60 0x7fbac0069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae015a7b0 0x7fbae01595d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbae0072460 0x7fbae0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 --2- 172.21.15.45:0/3034989802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbae006d850 0x7fbae014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 >> 172.21.15.45:0/3034989802 conn(0x7fbae00cfe80 msgr2=0x7fbae00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 shutdown_connections 2024-04-25T10:21:14.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.099+0000 7fbae6e00640 1 -- 172.21.15.45:0/3034989802 wait complete. 2024-04-25T10:21:14.111 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:21:14.098+0000 7faeed000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096d [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v650 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5721=0-4194304@1} caps={5721=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5721 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x564ec1c9b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1000000096d:[<0x10000000001/new_file_1 v650>,<0x10000000000/ceph-qa-suite v72>,<0x1/scrub_test_path v23>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:21:14.121 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '9bbea666-d6a1-4281-a8f5-5d9b493624b5', 'mode': 'asynchronous'}' 2024-04-25T10:21:14.122 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:21:14.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:21:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 Processor -- start 2024-04-25T10:21:14.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 -- start start 2024-04-25T10:21:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5df8159540 0x7f5df80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5df80721f0 0x7f5df8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df8159920 con 0x7f5df80aebd0 2024-04-25T10:21:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df815c2f0 con 0x7f5df8159540 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5dfde00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df815c5d0 con 0x7f5df80721f0 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5df8159540 0x7f5df80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5df8159540 0x7f5df80ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50994/0 (socket says 172.21.15.45:50994) 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 -- 172.21.15.45:0/503078309 learned_addr learned my addr 172.21.15.45:0/503078309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df7400640 1 --2- 172.21.15.45:0/503078309 >> v2:172.21.15.117:3300/0 conn(0x7f5df80721f0 0x7f5df8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 -- 172.21.15.45:0/503078309 >> v2:172.21.15.117:3300/0 conn(0x7f5df80721f0 msgr2=0x7f5df8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 --2- 172.21.15.45:0/503078309 >> v2:172.21.15.117:3300/0 conn(0x7f5df80721f0 0x7f5df8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 -- 172.21.15.45:0/503078309 >> v1:172.21.15.117:6789/0 conn(0x7f5df80aebd0 legacy=0x7f5df8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df6a00640 1 -- 172.21.15.45:0/503078309 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df815c8b0 con 0x7f5df8159540 2024-04-25T10:21:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.263+0000 7f5df7400640 1 --2- 172.21.15.45:0/503078309 >> v2:172.21.15.117:3300/0 conn(0x7f5df80721f0 0x7f5df8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5df6a00640 1 --2- 172.21.15.45:0/503078309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df8159540 0x7f5df80ae690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5df80cd200 tx=0x7f5de801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a67c2bc1ad2b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5df6000640 1 -- 172.21.15.45:0/503078309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80ab5f0 con 0x7f5df8159540 2024-04-25T10:21:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5df6000640 1 -- 172.21.15.45:0/503078309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80b0070 con 0x7f5df8159540 2024-04-25T10:21:14.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5df6000640 1 -- 172.21.15.45:0/503078309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80ab9b0 con 0x7f5df8159540 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 -- 172.21.15.45:0/503078309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df8159540 msgr2=0x7f5df80ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 --2- 172.21.15.45:0/503078309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df8159540 0x7f5df80ae690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5df80cd200 tx=0x7f5de801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 -- 172.21.15.45:0/503078309 shutdown_connections 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 --2- 172.21.15.45:0/503078309 >> v2:172.21.15.117:3300/0 conn(0x7f5df80721f0 0x7f5df8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 --2- 172.21.15.45:0/503078309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df8159540 0x7f5df80ae690 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 -- 172.21.15.45:0/503078309 >> 172.21.15.45:0/503078309 conn(0x7f5df80669a0 msgr2=0x7f5df8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 -- 172.21.15.45:0/503078309 shutdown_connections 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.267+0000 7f5dfde00640 1 -- 172.21.15.45:0/503078309 wait complete. 2024-04-25T10:21:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 Processor -- start 2024-04-25T10:21:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 -- start start 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 0x7f5df8154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5df80aebd0 0x7f5df80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5df8159540 0x7f5df8157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8071220 con 0x7f5df80721f0 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80710a0 con 0x7f5df80aebd0 2024-04-25T10:21:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df80713a0 con 0x7f5df8159540 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 0x7f5df8154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 0x7f5df8154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51006/0 (socket says 172.21.15.45:51006) 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 -- 172.21.15.45:0/2636348747 learned_addr learned my addr 172.21.15.45:0/2636348747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dee000640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5df80aebd0 0x7f5df80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df7400640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5df8159540 0x7f5df8157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 -- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5df8159540 msgr2=0x7f5df8157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5df8159540 0x7f5df8157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 -- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5df80aebd0 msgr2=0x7f5df80b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5df80aebd0 0x7f5df80b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 -- 172.21.15.45:0/2636348747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80b13f0 con 0x7f5df80721f0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5df6a00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 0x7f5df8154b40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5de80ad040 tx=0x7f5de801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5defe00640 1 -- 172.21.15.45:0/2636348747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80be5f0 con 0x7f5df80721f0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5defe00640 1 -- 172.21.15.45:0/2636348747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de80be7b0 con 0x7f5df80721f0 2024-04-25T10:21:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5defe00640 1 -- 172.21.15.45:0/2636348747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80beba0 con 0x7f5df80721f0 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df80b16d0 con 0x7f5df80721f0 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.271+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df80b1c90 con 0x7f5df80721f0 2024-04-25T10:21:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5defe00640 1 -- 172.21.15.45:0/2636348747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5de80b43f0 con 0x7f5df80721f0 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5defe00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5dd4066f60 0x7f5dd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5defe00640 1 -- 172.21.15.45:0/2636348747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de811f760 con 0x7f5df80721f0 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5dee000640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5dd4066f60 0x7f5dd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 Processor -- start 2024-04-25T10:21:14.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 -- start start 2024-04-25T10:21:14.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5dee000640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5dd4066f60 0x7f5dd4069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5de0004e30 tx=0x7f5de0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c1cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c1d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf80a69ff0 con 0x55bf80c672f0 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf809f2150 con 0x55bf80c669d0 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.275+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf80a66090 con 0x55bf80c1d0a0 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c1cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c1cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51022/0 (socket says 172.21.15.45:51022) 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 -- 172.21.15.45:0/1247260312 learned_addr learned my addr 172.21.15.45:0/1247260312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5df6a00640 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5df7400640 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c1d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 -- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 msgr2=0x55bf80c1d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c1d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 -- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 msgr2=0x55bf80c66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 -- 172.21.15.45:0/1247260312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf80c000d0 con 0x55bf80c672f0 2024-04-25T10:21:14.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c1cb60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5de00055d0 tx=0x7f5de0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85400c0fcab1bf34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5de7e00640 1 -- 172.21.15.45:0/1247260312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0025070 con 0x55bf80c672f0 2024-04-25T10:21:14.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5de7e00640 1 -- 172.21.15.45:0/1247260312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0015880 con 0x55bf80c672f0 2024-04-25T10:21:14.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5de7e00640 1 -- 172.21.15.45:0/1247260312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0015c30 con 0x55bf80c672f0 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- 172.21.15.45:0/1247260312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 msgr2=0x55bf80c1cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c1cb60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5de00055d0 tx=0x7f5de0011000 comp rx=0 tx=0).stop 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- 172.21.15.45:0/1247260312 shutdown_connections 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c1d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c1cb60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- 172.21.15.45:0/1247260312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- 172.21.15.45:0/1247260312 >> 172.21.15.45:0/1247260312 conn(0x55bf80c19020 msgr2=0x55bf80c1b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- 172.21.15.45:0/1247260312 shutdown_connections 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- 172.21.15.45:0/1247260312 wait complete. 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- start start 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 Processor -- start 2024-04-25T10:21:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 0x55bf80c4b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c4bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c672f0 0x55bf80c4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf80a2e250 con 0x55bf80c1d0a0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf80a24690 con 0x55bf80c669d0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf809f2a40 con 0x55bf80c672f0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5e01373000 10 client.0 _ll_register_callbacks cb 0x55bf80c07e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5df6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 0x55bf80c4b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5df6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 0x55bf80c4b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51032/0 (socket says 172.21.15.45:51032) 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5df6a00640 1 -- 172.21.15.45:0/83699464 learned_addr learned my addr 172.21.15.45:0/83699464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5dee000640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c4bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.279+0000 7f5df7400640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c672f0 0x55bf80c4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df6a00640 1 -- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c672f0 msgr2=0x55bf80c4c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df6a00640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c672f0 0x55bf80c4c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df6a00640 1 -- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 msgr2=0x55bf80c4bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df6a00640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c4bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df6a00640 1 -- 172.21.15.45:0/83699464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf80b171c0 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df7400640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c672f0 0x55bf80c4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5df6a00640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 0x55bf80c4b7f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5de80eff30 tx=0x7f5de80f02f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f63c88082741545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 1 -- 172.21.15.45:0/83699464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80c4040 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 1 -- 172.21.15.45:0/83699464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de80f8400 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5de0015880 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 1 -- 172.21.15.45:0/83699464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5de80f8600 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 10 client.6126 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55bf80b13170 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 1 -- 172.21.15.45:0/83699464 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0015880 con 0x55bf80c1d0a0 2024-04-25T10:21:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 1 -- 172.21.15.45:0/83699464 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5de80ef680 con 0x55bf80c1d0a0 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 10 client.6126 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 10 client.6126 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 20 client.6126 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c76c80 0x55bf80c79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 4 client.6126 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf80a05c60 con 0x55bf80c76c80 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5dee000640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c76c80 0x55bf80c79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5dee000640 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c76c80 0x55bf80c79070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5df0c23b5afdfea8 server_cookie=7552b98fbee01437 in_seq=0 out_seq=0 2024-04-25T10:21:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 10 client.6126 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 1 -- 172.21.15.45:0/83699464 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf80a05c60 con 0x55bf80c76c80 2024-04-25T10:21:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 10 client.6126 handle_command_reply: tid=0 2024-04-25T10:21:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 client.6126 shutdown 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 msgr2=0x55bf80c4b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 0x55bf80c4b7f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5de80eff30 tx=0x7f5de80f02f0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 shutdown_connections 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c76c80 0x55bf80c79070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c672f0 0x55bf80c4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80c4bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 --2- 172.21.15.45:0/83699464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c1d0a0 0x55bf80c4b7f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5de4c00640 0 client.6126 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 >> 172.21.15.45:0/83699464 conn(0x55bf80c19020 msgr2=0x55bf80c1bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 shutdown_connections 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 1 -- 172.21.15.45:0/83699464 wait complete. 2024-04-25T10:21:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.283+0000 7f5e01373000 20 client.6126 trim_cache size 0 max 16384 2024-04-25T10:21:14.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.299+0000 7f5e01373000 1 Processor -- start 2024-04-25T10:21:14.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- start start 2024-04-25T10:21:14.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c65de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 0x55bf80c1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c672f0 0x55bf80c1c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8202ab0 con 0x55bf80c669d0 2024-04-25T10:21:14.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8201c70 con 0x55bf80c672f0 2024-04-25T10:21:14.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8201070 con 0x55bf80c1d0a0 2024-04-25T10:21:14.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 0x55bf80c1c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 0x55bf80c1c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51042/0 (socket says 172.21.15.45:51042) 2024-04-25T10:21:14.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 -- 172.21.15.45:0/4143869611 learned_addr learned my addr 172.21.15.45:0/4143869611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5df7400640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c672f0 0x55bf80c1c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 -- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 msgr2=0x55bf80c65de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5df6a00640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c65de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c65de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 -- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c672f0 msgr2=0x55bf80c1c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c672f0 0x55bf80c1c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 -- 172.21.15.45:0/4143869611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf80ab0de0 con 0x55bf80c669d0 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5df7400640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c672f0 0x55bf80c1c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5df6a00640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c65de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5dee000640 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 0x55bf80c1c420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5de00055a0 tx=0x7f5de0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557e3d14cfc0c55d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5de7400640 1 -- 172.21.15.45:0/4143869611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0025050 con 0x55bf80c669d0 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5de7400640 1 -- 172.21.15.45:0/4143869611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0026040 con 0x55bf80c669d0 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5de7400640 1 -- 172.21.15.45:0/4143869611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de0015aa0 con 0x55bf80c669d0 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- 172.21.15.45:0/4143869611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 msgr2=0x55bf80c1c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 0x55bf80c1c420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5de00055a0 tx=0x7f5de0013000 comp rx=0 tx=0).stop 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- 172.21.15.45:0/4143869611 shutdown_connections 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c672f0 0x55bf80c1c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c669d0 0x55bf80c1c420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 --2- 172.21.15.45:0/4143869611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80c65de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- 172.21.15.45:0/4143869611 >> 172.21.15.45:0/4143869611 conn(0x55bf80c19020 msgr2=0x55bf80c650a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- 172.21.15.45:0/4143869611 shutdown_connections 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- 172.21.15.45:0/4143869611 wait complete. 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 -- start start 2024-04-25T10:21:14.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.303+0000 7f5e01373000 1 Processor -- start 2024-04-25T10:21:14.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80cc7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80cc7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c79380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf80a04e00 con 0x55bf80c672f0 2024-04-25T10:21:14.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df8201670 con 0x55bf80c669d0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf80a69300 con 0x55bf80c1d0a0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 10 client.0 _ll_register_callbacks cb 0x55bf80be8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c79380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c79380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51048/0 (socket says 172.21.15.45:51048) 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 -- 172.21.15.45:0/471978635 learned_addr learned my addr 172.21.15.45:0/471978635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df6a00640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80cc7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5dee000640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80cc7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 -- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 msgr2=0x55bf80cc7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80cc7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 -- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 msgr2=0x55bf80cc7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80cc7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 -- 172.21.15.45:0/471978635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf80a0b8e0 con 0x55bf80c672f0 2024-04-25T10:21:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df6a00640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80cc7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5dee000640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80cc7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df7400640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c79380 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5df000db30 tx=0x7f5df0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86dc1a4b9d4d730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 1 -- 172.21.15.45:0/471978635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df001e060 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 1 -- 172.21.15.45:0/471978635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5df0019400 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5de0026040 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 1 -- 172.21.15.45:0/471978635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df0024050 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 10 client.6132 fetch_fsmap learned FSMap version 28 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55bf80ac38f0 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 1 -- 172.21.15.45:0/471978635 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de0026040 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 1 -- 172.21.15.45:0/471978635 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5df0019600 con 0x55bf80c672f0 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 10 client.6132 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:21:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 10 client.6132 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 20 client.6132 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c79f60 0x55bf80c7a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 4 client.6132 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf80a77530 con 0x55bf80c79f60 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df6a00640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c79f60 0x55bf80c7a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5df6a00640 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c79f60 0x55bf80c7a370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19024090e1b13838 server_cookie=382da10f169bf842 in_seq=0 out_seq=0 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 10 client.6132 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 1 -- 172.21.15.45:0/471978635 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x55bf80a77530 con 0x55bf80c79f60 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 10 client.6132 handle_command_reply: tid=0 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 client.6132 shutdown 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 msgr2=0x55bf80c79380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c79380 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5df000db30 tx=0x7f5df0007f70 comp rx=0 tx=0).stop 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 shutdown_connections 2024-04-25T10:21:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55bf80c79f60 0x55bf80c7a370 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf80c672f0 0x55bf80c79380 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf80c669d0 0x55bf80cc7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 --2- 172.21.15.45:0/471978635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bf80c1d0a0 0x55bf80cc7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 >> 172.21.15.45:0/471978635 conn(0x55bf80c19020 msgr2=0x55bf80c65930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5de4c00640 0 client.6132 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 shutdown_connections 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 1 -- 172.21.15.45:0/471978635 wait complete. 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.307+0000 7f5e01373000 20 client.6132 trim_cache size 0 max 16384 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "9bbea666-d6a1-4281-a8f5-5d9b493624b5": { 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "9bbea666-d6a1-4281-a8f5-5d9b493624b5", 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "" 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:14.321 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5dd4066f60 msgr2=0x7f5dd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5dd4066f60 0x7f5dd4069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5de0004e30 tx=0x7f5de0009040 comp rx=0 tx=0).stop 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 msgr2=0x7f5df8154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 0x7f5df8154b40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5de80ad040 tx=0x7f5de801ad20 comp rx=0 tx=0).stop 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 shutdown_connections 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5dd4066f60 0x7f5dd4069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5df8159540 0x7f5df8157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5df80aebd0 0x7f5df80b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 --2- 172.21.15.45:0/2636348747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df80721f0 0x7f5df8154b40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:14.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 >> 172.21.15.45:0/2636348747 conn(0x7f5df80669a0 msgr2=0x7f5df815aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:14.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 shutdown_connections 2024-04-25T10:21:14.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:14.311+0000 7f5dfde00640 1 -- 172.21.15.45:0/2636348747 wait complete. 2024-04-25T10:21:19.332 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:21:19.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:21:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 Processor -- start 2024-04-25T10:21:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 -- start start 2024-04-25T10:21:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f556c072220 0x7f556c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c06f250 con 0x7f556c071e40 2024-04-25T10:21:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c15f720 con 0x7f556c072220 2024-04-25T10:21:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.483+0000 7f5572200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c15f9c0 con 0x7f556c15bb70 2024-04-25T10:21:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c06e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37136/0 (socket says 172.21.15.45:37136) 2024-04-25T10:21:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 -- 172.21.15.45:0/2494128127 learned_addr learned my addr 172.21.15.45:0/2494128127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f556be00640 1 --2- 172.21.15.45:0/2494128127 >> v2:172.21.15.117:3300/0 conn(0x7f556c072220 0x7f556c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 -- 172.21.15.45:0/2494128127 >> v2:172.21.15.117:3300/0 conn(0x7f556c072220 msgr2=0x7f556c06ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 --2- 172.21.15.45:0/2494128127 >> v2:172.21.15.117:3300/0 conn(0x7f556c072220 0x7f556c06ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 -- 172.21.15.45:0/2494128127 >> v1:172.21.15.117:6789/0 conn(0x7f556c071e40 legacy=0x7f556c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 -- 172.21.15.45:0/2494128127 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c15fca0 con 0x7f556c15bb70 2024-04-25T10:21:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.487+0000 7f5563e00640 1 --2- 172.21.15.45:0/2494128127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c06e7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5554009870 tx=0x7f555401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285b9b4a8af4fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f556aa00640 1 -- 172.21.15.45:0/2494128127 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540a95f0 con 0x7f556c15bb70 2024-04-25T10:21:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f556aa00640 1 -- 172.21.15.45:0/2494128127 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540a9790 con 0x7f556c15bb70 2024-04-25T10:21:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f556aa00640 1 -- 172.21.15.45:0/2494128127 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540a9b60 con 0x7f556c15bb70 2024-04-25T10:21:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- 172.21.15.45:0/2494128127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 msgr2=0x7f556c06e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 --2- 172.21.15.45:0/2494128127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c06e7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5554009870 tx=0x7f555401c530 comp rx=0 tx=0).stop 2024-04-25T10:21:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- 172.21.15.45:0/2494128127 shutdown_connections 2024-04-25T10:21:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 --2- 172.21.15.45:0/2494128127 >> v2:172.21.15.117:3300/0 conn(0x7f556c072220 0x7f556c06ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 --2- 172.21.15.45:0/2494128127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c06e7d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- 172.21.15.45:0/2494128127 >> 172.21.15.45:0/2494128127 conn(0x7f556c06ac90 msgr2=0x7f556c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- 172.21.15.45:0/2494128127 shutdown_connections 2024-04-25T10:21:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- 172.21.15.45:0/2494128127 wait complete. 2024-04-25T10:21:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 Processor -- start 2024-04-25T10:21:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- start start 2024-04-25T10:21:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 0x7f556c145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c072220 0x7f556c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0cf8f0 con 0x7f556c071e40 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0cf770 con 0x7f556c072220 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.491+0000 7f5572200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0cfa70 con 0x7f556c15bb70 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 0x7f556c145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 0x7f556c145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52426/0 (socket says 172.21.15.45:52426) 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 -- 172.21.15.45:0/1472897208 learned_addr learned my addr 172.21.15.45:0/1472897208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 -- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 msgr2=0x7f556c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 -- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c072220 msgr2=0x7f556c155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5563e00640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c072220 0x7f556c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c072220 0x7f556c155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 -- 172.21.15.45:0/1472897208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c1572a0 con 0x7f556c071e40 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f556b400640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 0x7f556c145680 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c3d0 tx=0x7f555c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5563400640 1 -- 172.21.15.45:0/1472897208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c0052d0 con 0x7f556c071e40 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5563400640 1 -- 172.21.15.45:0/1472897208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c005470 con 0x7f556c071e40 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5563400640 1 -- 172.21.15.45:0/1472897208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c007260 con 0x7f556c071e40 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c156200 con 0x7f556c071e40 2024-04-25T10:21:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c1567c0 con 0x7f556c071e40 2024-04-25T10:21:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.495+0000 7f5563400640 1 -- 172.21.15.45:0/1472897208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f555c007400 con 0x7f556c071e40 2024-04-25T10:21:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563400640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5548066f90 0x7f5548069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563400640 1 -- 172.21.15.45:0/1472897208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f555c084480 con 0x7f556c071e40 2024-04-25T10:21:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5548066f90 0x7f5548069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 Processor -- start 2024-04-25T10:21:19.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 -- start start 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5548066f90 0x7f5548069450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55540ae000 tx=0x7f55540b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485611210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x5594855c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x5594855c7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948540fff0 con 0x559485611750 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559485398150 con 0x559485610e30 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55948540c090 con 0x5594855c74b0 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x5594855c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x5594855c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52428/0 (socket says 172.21.15.45:52428) 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 -- 172.21.15.45:0/3692622015 learned_addr learned my addr 172.21.15.45:0/3692622015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f556be00640 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x5594855c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f556b400640 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485611210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 -- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 msgr2=0x5594855c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x5594855c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 -- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 msgr2=0x559485611210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485611210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 -- 172.21.15.45:0/3692622015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594855a60d0 con 0x559485611750 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f5563e00640 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x5594855c6f70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f555401abe0 tx=0x7f5554004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce87b963271ca40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f555be00640 1 -- 172.21.15.45:0/3692622015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540cb070 con 0x559485611750 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f555be00640 1 -- 172.21.15.45:0/3692622015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540bc5c0 con 0x559485611750 2024-04-25T10:21:19.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.499+0000 7f555be00640 1 -- 172.21.15.45:0/3692622015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540bc970 con 0x559485611750 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/3692622015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 msgr2=0x5594855c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x5594855c6f70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f555401abe0 tx=0x7f5554004740 comp rx=0 tx=0).stop 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/3692622015 shutdown_connections 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x5594855c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x5594855c6f70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- 172.21.15.45:0/3692622015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485611210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/3692622015 >> 172.21.15.45:0/3692622015 conn(0x5594855bf020 msgr2=0x5594855c1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/3692622015 shutdown_connections 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/3692622015 wait complete. 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- start start 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 Processor -- start 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 0x5594856107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x5594855cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485611750 0x5594855cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559485414620 con 0x5594855c74b0 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594853ca690 con 0x559485611750 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559485398a40 con 0x559485610e30 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 10 client.0 _ll_register_callbacks cb 0x5594855ade90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 0x5594856107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 0x5594856107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52434/0 (socket says 172.21.15.45:52434) 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 -- 172.21.15.45:0/116969110 learned_addr learned my addr 172.21.15.45:0/116969110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556be00640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485611750 0x5594855cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5563e00640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x5594855cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 -- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 msgr2=0x5594855cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x5594855cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 -- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485611750 msgr2=0x5594855cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485611750 0x5594855cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 -- 172.21.15.45:0/116969110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594854bd1c0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f556b400640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 0x5594856107b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f555c00cd90 tx=0x7f555c054d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68aba63217d41a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5558c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5558c00640 1 -- 172.21.15.45:0/116969110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c05b0c0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5558c00640 1 -- 172.21.15.45:0/116969110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f555c0513f0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55540bc5c0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5558c00640 1 -- 172.21.15.45:0/116969110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f555c0840d0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 10 client.6141 fetch_fsmap learned FSMap version 29 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.503+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5594854b9170 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5558c00640 1 -- 172.21.15.45:0/116969110 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540bc5c0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5558c00640 1 -- 172.21.15.45:0/116969110 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f555c0541d0 con 0x5594855c74b0 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 10 client.6141 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 10 client.6141 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 20 client.6141 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:19.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594855ce640 0x5594856230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 4 client.6141 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594853abc60 con 0x5594855ce640 2024-04-25T10:21:19.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5563e00640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594855ce640 0x5594856230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5563e00640 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594855ce640 0x5594856230f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed39428b5f9e644b server_cookie=93c71c709b1e2d25 in_seq=0 out_seq=0 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5558c00640 10 client.6141 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5558c00640 1 -- 172.21.15.45:0/116969110 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594853abc60 con 0x5594855ce640 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5558c00640 10 client.6141 handle_command_reply: tid=0 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 client.6141 shutdown 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 msgr2=0x5594856107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 0x5594856107b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f555c00cd90 tx=0x7f555c054d80 comp rx=0 tx=0).stop 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 shutdown_connections 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594855ce640 0x5594856230f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485611750 0x5594855cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x5594855cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 --2- 172.21.15.45:0/116969110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594855c74b0 0x5594856107b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5558c00640 0 client.6141 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 >> 172.21.15.45:0/116969110 conn(0x5594855bf020 msgr2=0x5594855c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 shutdown_connections 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 1 -- 172.21.15.45:0/116969110 wait complete. 2024-04-25T10:21:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.507+0000 7f5575693000 20 client.6141 trim_cache size 0 max 16384 2024-04-25T10:21:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 Processor -- start 2024-04-25T10:21:19.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 -- start start 2024-04-25T10:21:19.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x559485625110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485625650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x55948561e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b52c0 con 0x559485611750 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b4480 con 0x559485610e30 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b3880 con 0x5594855c74b0 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x55948561e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x55948561e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52440/0 (socket says 172.21.15.45:52440) 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 -- 172.21.15.45:0/3301123986 learned_addr learned my addr 172.21.15.45:0/3301123986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f5563e00640 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485625650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556b400640 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x559485625110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 -- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 msgr2=0x559485625110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x559485625110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 -- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 msgr2=0x559485625650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485625650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 -- 172.21.15.45:0/3301123986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559485456de0 con 0x559485611750 2024-04-25T10:21:19.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f556be00640 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x55948561e2a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5564003c70 tx=0x7f5564004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa2ef7053a294a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.523+0000 7f555b400640 1 -- 172.21.15.45:0/3301123986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556401b020 con 0x559485611750 2024-04-25T10:21:19.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f555b400640 1 -- 172.21.15.45:0/3301123986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564017680 con 0x559485611750 2024-04-25T10:21:19.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f555b400640 1 -- 172.21.15.45:0/3301123986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564017a90 con 0x559485611750 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- 172.21.15.45:0/3301123986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 msgr2=0x55948561e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x55948561e2a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5564003c70 tx=0x7f5564004410 comp rx=0 tx=0).stop 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- 172.21.15.45:0/3301123986 shutdown_connections 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x55948561e2a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559485610e30 0x559485625650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- 172.21.15.45:0/3301123986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594855c74b0 0x559485625110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- 172.21.15.45:0/3301123986 >> 172.21.15.45:0/3301123986 conn(0x5594855bf020 msgr2=0x55948561db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- 172.21.15.45:0/3301123986 shutdown_connections 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- 172.21.15.45:0/3301123986 wait complete. 2024-04-25T10:21:19.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- start start 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 Processor -- start 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594855c74b0 0x5594856230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x559485623630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x559485623b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594853aae00 con 0x559485611750 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b3e80 con 0x5594855c74b0 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55948540f300 con 0x559485610e30 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 10 client.0 _ll_register_callbacks cb 0x55948558e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x559485623b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x559485623b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52442/0 (socket says 172.21.15.45:52442) 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 -- 172.21.15.45:0/1531590761 learned_addr learned my addr 172.21.15.45:0/1531590761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594855c74b0 0x5594856230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5563e00640 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x559485623630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 -- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 msgr2=0x559485623630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x559485623630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 -- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594855c74b0 msgr2=0x5594856230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594855c74b0 0x5594856230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 -- 172.21.15.45:0/1531590761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594853b18e0 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f556be00640 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x559485623b70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f556400b010 tx=0x7f5564009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cceb034b82223b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5558c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5558c00640 1 -- 172.21.15.45:0/1531590761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556402c680 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5558c00640 1 -- 172.21.15.45:0/1531590761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f556402c820 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5564017680 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.527+0000 7f5558c00640 1 -- 172.21.15.45:0/1531590761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556402ca20 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 10 client.6147 fetch_fsmap learned FSMap version 29 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5594854698f0 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5558c00640 1 -- 172.21.15.45:0/1531590761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564017680 con 0x559485611750 2024-04-25T10:21:19.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5558c00640 1 -- 172.21.15.45:0/1531590761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f556402cbc0 con 0x559485611750 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 10 client.6147 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 10 client.6147 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 20 client.6147 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594856246c0 0x559485678fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 4 client.6147 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55948541d530 con 0x5594856246c0 2024-04-25T10:21:19.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f556b400640 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594856246c0 0x559485678fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f556b400640 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594856246c0 0x559485678fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85fe9a63849be040 server_cookie=517a71e235aceb80 in_seq=0 out_seq=0 2024-04-25T10:21:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5558c00640 10 client.6147 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5558c00640 1 -- 172.21.15.45:0/1531590761 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55948541d530 con 0x5594856246c0 2024-04-25T10:21:19.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5558c00640 10 client.6147 handle_command_reply: tid=0 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 client.6147 shutdown 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 msgr2=0x559485623b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x559485623b70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f556400b010 tx=0x7f5564009750 comp rx=0 tx=0).stop 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 shutdown_connections 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x5594856246c0 0x559485678fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559485611750 0x559485623b70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559485610e30 0x559485623630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 --2- 172.21.15.45:0/1531590761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5594855c74b0 0x5594856230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5558c00640 0 client.6147 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 >> 172.21.15.45:0/1531590761 conn(0x5594855bf020 msgr2=0x55948561e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 shutdown_connections 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 1 -- 172.21.15.45:0/1531590761 wait complete. 2024-04-25T10:21:19.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5575693000 20 client.6147 trim_cache size 0 max 16384 2024-04-25T10:21:19.539 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:21:19.539 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:21:19.539 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5548066f90 msgr2=0x7f5548069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5548066f90 0x7f5548069450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55540ae000 tx=0x7f55540b3040 comp rx=0 tx=0).stop 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 msgr2=0x7f556c145680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 0x7f556c145680 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c3d0 tx=0x7f555c00c890 comp rx=0 tx=0).stop 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 shutdown_connections 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5548066f90 0x7f5548069450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f556c15bb70 0x7f556c14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f556c072220 0x7f556c155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 --2- 172.21.15.45:0/1472897208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f556c071e40 0x7f556c145680 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 >> 172.21.15.45:0/1472897208 conn(0x7f556c06ac90 msgr2=0x7f556c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 shutdown_connections 2024-04-25T10:21:19.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.531+0000 7f5572200640 1 -- 172.21.15.45:0/1472897208 wait complete. 2024-04-25T10:21:19.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:21:19.554 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:19.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:21:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.691+0000 7f3011a00640 1 Processor -- start 2024-04-25T10:21:19.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.691+0000 7f3011a00640 1 -- start start 2024-04-25T10:21:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f3011a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f300c0721f0 0x7f300c166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f3011a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c071360 con 0x7f300c0aeda0 2024-04-25T10:21:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f3011a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c159920 con 0x7f300c159540 2024-04-25T10:21:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f3011a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c15c2f0 con 0x7f300c0721f0 2024-04-25T10:21:19.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f300c0721f0 0x7f300c166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f300c0721f0 0x7f300c166920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52452/0 (socket says 172.21.15.45:52452) 2024-04-25T10:21:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f3002000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f300c0aeda0 0x7f300c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41472/0 (socket says 172.21.15.45:41472) 2024-04-25T10:21:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300b400640 1 -- 172.21.15.45:0/1459000210 learned_addr learned my addr 172.21.15.45:0/1459000210 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300b400640 1 -- 172.21.15.45:0/1459000210 >> v1:172.21.15.117:6789/0 conn(0x7f300c159540 legacy=0x7f300c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300b400640 1 -- 172.21.15.45:0/1459000210 >> v1:172.21.15.45:6789/0 conn(0x7f300c0aeda0 legacy=0x7f300c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300b400640 1 -- 172.21.15.45:0/1459000210 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c15c5d0 con 0x7f300c0721f0 2024-04-25T10:21:19.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.695+0000 7f300a000640 1 -- 172.21.15.45:0/1459000210 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293750508 0 0) 0x7f300c071360 con 0x7f300c0aeda0 2024-04-25T10:21:19.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f300b400640 1 --2- 172.21.15.45:0/1459000210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c166920 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3004009e50 tx=0x7f300401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdadda956b22ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f300a000640 1 -- 172.21.15.45:0/1459000210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040a58d0 con 0x7f300c0721f0 2024-04-25T10:21:19.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f300a000640 1 -- 172.21.15.45:0/1459000210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040a5a70 con 0x7f300c0721f0 2024-04-25T10:21:19.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f300a000640 1 -- 172.21.15.45:0/1459000210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040a5e40 con 0x7f300c0721f0 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 -- 172.21.15.45:0/1459000210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 msgr2=0x7f300c166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 --2- 172.21.15.45:0/1459000210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c166920 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3004009e50 tx=0x7f300401ca00 comp rx=0 tx=0).stop 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 -- 172.21.15.45:0/1459000210 shutdown_connections 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 --2- 172.21.15.45:0/1459000210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c166920 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 -- 172.21.15.45:0/1459000210 >> 172.21.15.45:0/1459000210 conn(0x7f300c0669a0 msgr2=0x7f300c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 -- 172.21.15.45:0/1459000210 shutdown_connections 2024-04-25T10:21:19.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 -- 172.21.15.45:0/1459000210 wait complete. 2024-04-25T10:21:19.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 Processor -- start 2024-04-25T10:21:19.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.699+0000 7f3011a00640 1 -- start start 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c0aeda0 0x7f300c150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f300c159540 0x7f300c0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c169680 con 0x7f300c0721f0 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c06c0d0 con 0x7f300c0aeda0 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c169800 con 0x7f300c159540 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52458/0 (socket says 172.21.15.45:52458) 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 -- 172.21.15.45:0/2013915267 learned_addr learned my addr 172.21.15.45:0/2013915267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3002000640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c0aeda0 0x7f300c150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300b400640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f300c159540 0x7f300c0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 -- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f300c159540 msgr2=0x7f300c0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f300c159540 0x7f300c0cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 -- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c0aeda0 msgr2=0x7f300c150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c0aeda0 0x7f300c150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 -- 172.21.15.45:0/2013915267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c151ee0 con 0x7f300c0721f0 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300b400640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f300c159540 0x7f300c0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f300aa00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c152f40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c3c0 tx=0x7f2ffc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3003400640 1 -- 172.21.15.45:0/2013915267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc017070 con 0x7f300c0721f0 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3003400640 1 -- 172.21.15.45:0/2013915267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0052d0 con 0x7f300c0721f0 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c152180 con 0x7f300c0721f0 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3003400640 1 -- 172.21.15.45:0/2013915267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc003840 con 0x7f300c0721f0 2024-04-25T10:21:19.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c152740 con 0x7f300c0721f0 2024-04-25T10:21:19.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.703+0000 7f3003400640 1 -- 172.21.15.45:0/2013915267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ffc01a070 con 0x7f300c0721f0 2024-04-25T10:21:19.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3003400640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fe8066f60 0x7f2fe8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3003400640 1 -- 172.21.15.45:0/2013915267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ffc084420 con 0x7f300c0721f0 2024-04-25T10:21:19.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3002000640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fe8066f60 0x7f2fe8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 Processor -- start 2024-04-25T10:21:19.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- start start 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3002000640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fe8066f60 0x7f2fe8069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4007fd0 tx=0x7f2ff4007ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f57560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f0d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615f0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e615d5af20 con 0x55e615f0d600 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e615ce3000 con 0x55e615f57180 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e615d56fc0 con 0x55e615f57aa0 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615f0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615f0d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52466/0 (socket says 172.21.15.45:52466) 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 -- 172.21.15.45:0/3430327546 learned_addr learned my addr 172.21.15.45:0/3430327546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300aa00640 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f57560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3002000640 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f0d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 -- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 msgr2=0x55e615f0d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f0d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 -- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 msgr2=0x55e615f57560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f57560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 -- 172.21.15.45:0/3430327546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e615d36ba0 con 0x55e615f0d600 2024-04-25T10:21:19.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f300b400640 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615f0d9e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3004019610 tx=0x7f3004004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0d5cdfb26439a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f2ffb400640 1 -- 172.21.15.45:0/3430327546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040a58d0 con 0x55e615f0d600 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f2ffb400640 1 -- 172.21.15.45:0/3430327546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040a5a70 con 0x55e615f0d600 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f2ffb400640 1 -- 172.21.15.45:0/3430327546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004003be0 con 0x55e615f0d600 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- 172.21.15.45:0/3430327546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 msgr2=0x55e615f0d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615f0d9e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3004019610 tx=0x7f3004004ac0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- 172.21.15.45:0/3430327546 shutdown_connections 2024-04-25T10:21:19.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615f0d9e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f0d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 --2- 172.21.15.45:0/3430327546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f57560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- 172.21.15.45:0/3430327546 >> 172.21.15.45:0/3430327546 conn(0x55e615f13760 msgr2=0x55e615f0b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.707+0000 7f3014de1000 1 -- 172.21.15.45:0/3430327546 shutdown_connections 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- 172.21.15.45:0/3430327546 wait complete. 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- start start 2024-04-25T10:21:19.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 Processor -- start 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f0d600 0x55e615f12640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57180 0x55e615f12b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f45450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e615e189c0 con 0x55e615f57aa0 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e615d155d0 con 0x55e615f0d600 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e615d11dc0 con 0x55e615f57180 2024-04-25T10:21:19.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 10 client.0 _ll_register_callbacks cb 0x55e615ef8a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f45450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f45450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52478/0 (socket says 172.21.15.45:52478) 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 -- 172.21.15.45:0/2830746634 learned_addr learned my addr 172.21.15.45:0/2830746634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3002000640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57180 0x55e615f12b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300aa00640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f0d600 0x55e615f12640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 -- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57180 msgr2=0x55e615f12b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57180 0x55e615f12b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 -- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f0d600 msgr2=0x55e615f12640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f0d600 0x55e615f12640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 -- 172.21.15.45:0/2830746634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e615da15a0 con 0x55e615f57aa0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f300b400640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f45450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3004019610 tx=0x7f300401b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18f7f36a19762c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f2fefe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f2fefe00640 1 -- 172.21.15.45:0/2830746634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30040a58d0 con 0x55e615f57aa0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f2fefe00640 1 -- 172.21.15.45:0/2830746634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3004004040 con 0x55e615f57aa0 2024-04-25T10:21:19.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30040a5a70 con 0x55e615f57aa0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f2fefe00640 1 -- 172.21.15.45:0/2830746634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3004004240 con 0x55e615f57aa0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 10 client.6162 fetch_fsmap learned FSMap version 29 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55e615d322b0 con 0x55e615f57aa0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f2fefe00640 1 -- 172.21.15.45:0/2830746634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040a5a70 con 0x55e615f57aa0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f2fefe00640 1 -- 172.21.15.45:0/2830746634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3004008050 con 0x55e615f57aa0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 10 client.6162 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 10 client.6162 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 20 client.6162 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f44e60 0x55e615f45e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 4 client.6162 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.711+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e615cf6c70 con 0x55e615f44e60 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f300aa00640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f44e60 0x55e615f45e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f300aa00640 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f44e60 0x55e615f45e60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb37f92fa2dea4b server_cookie=fcdc7abd6a8e48c9 in_seq=0 out_seq=0 2024-04-25T10:21:19.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f2fefe00640 10 client.6162 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f2fefe00640 1 -- 172.21.15.45:0/2830746634 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e615cf6c70 con 0x55e615f44e60 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f2fefe00640 10 client.6162 handle_command_reply: tid=0 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 client.6162 shutdown 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 msgr2=0x55e615f45450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f45450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3004019610 tx=0x7f300401b240 comp rx=0 tx=0).stop 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 shutdown_connections 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f44e60 0x55e615f45e60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f45450 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57180 0x55e615f12b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 --2- 172.21.15.45:0/2830746634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f0d600 0x55e615f12640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f2fefe00640 0 client.6162 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 >> 172.21.15.45:0/2830746634 conn(0x55e615f13760 msgr2=0x55e615f0c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 shutdown_connections 2024-04-25T10:21:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 1 -- 172.21.15.45:0/2830746634 wait complete. 2024-04-25T10:21:19.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.715+0000 7f3014de1000 20 client.6162 trim_cache size 0 max 16384 2024-04-25T10:21:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 Processor -- start 2024-04-25T10:21:19.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 -- start start 2024-04-25T10:21:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f0d600 0x55e615f56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f0c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f0c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c2008a0 con 0x55e615f57aa0 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c1f85e0 con 0x55e615f57180 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c15a6b0 con 0x55e615f0d600 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f0c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f0c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52494/0 (socket says 172.21.15.45:52494) 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 -- 172.21.15.45:0/4085467100 learned_addr learned my addr 172.21.15.45:0/4085467100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 -- 172.21.15.45:0/4085467100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f0d600 msgr2=0x55e615f56ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f0d600 0x55e615f56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 -- 172.21.15.45:0/4085467100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 msgr2=0x55e615f0c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f0c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 -- 172.21.15.45:0/4085467100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e615f461d0 con 0x55e615f57aa0 2024-04-25T10:21:19.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f300b400640 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f0c660 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f30040094c0 tx=0x7f30040c1070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73ff6f313de73fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f2ffaa00640 1 -- 172.21.15.45:0/4085467100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004003cb0 con 0x55e615f57aa0 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f2ffaa00640 1 -- 172.21.15.45:0/4085467100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040c3070 con 0x55e615f57aa0 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.727+0000 7f2ffaa00640 1 -- 172.21.15.45:0/4085467100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004004090 con 0x55e615f57aa0 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/4085467100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 msgr2=0x55e615f0c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f0c660 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f30040094c0 tx=0x7f30040c1070 comp rx=0 tx=0).stop 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/4085467100 shutdown_connections 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f57aa0 0x55e615f0c660 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615f0c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- 172.21.15.45:0/4085467100 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f0d600 0x55e615f56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/4085467100 >> 172.21.15.45:0/4085467100 conn(0x55e615f13760 msgr2=0x55e615f64770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/4085467100 shutdown_connections 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/4085467100 wait complete. 2024-04-25T10:21:19.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- start start 2024-04-25T10:21:19.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 Processor -- start 2024-04-25T10:21:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615fbb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615fbb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f76cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c1f7fe0 con 0x55e615f0d600 2024-04-25T10:21:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e615f166f0 con 0x55e615f57180 2024-04-25T10:21:19.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e615d11dc0 con 0x55e615f57aa0 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 10 client.0 _ll_register_callbacks cb 0x55e615ed9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615fbb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615fbb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52510/0 (socket says 172.21.15.45:52510) 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 -- 172.21.15.45:0/2454935869 learned_addr learned my addr 172.21.15.45:0/2454935869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3002000640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615fbb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300b400640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f76cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 -- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 msgr2=0x55e615f76cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f76cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 -- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 msgr2=0x55e615fbb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615fbb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 -- 172.21.15.45:0/2454935869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e615ce3250 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300b400640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f76cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3002000640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615fbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f300aa00640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615fbb420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc054990 tx=0x7f2ffc051060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422dbe3c89933d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f2fefe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f2fefe00640 1 -- 172.21.15.45:0/2454935869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc058de0 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f2fefe00640 1 -- 172.21.15.45:0/2454935869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ffc051300 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30040c3070 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f2fefe00640 1 -- 172.21.15.45:0/2454935869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ffc05d3a0 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 10 client.6168 fetch_fsmap learned FSMap version 29 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55e615cefb90 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.731+0000 7f2fefe00640 1 -- 172.21.15.45:0/2454935869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30040c3070 con 0x55e615f0d600 2024-04-25T10:21:19.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f2fefe00640 1 -- 172.21.15.45:0/2454935869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ffc0840b0 con 0x55e615f0d600 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 10 client.6168 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 10 client.6168 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 20 client.6168 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f77bb0 0x55e615f77fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 4 client.6168 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e615f457e0 con 0x55e615f77bb0 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3002000640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f77bb0 0x55e615f77fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3002000640 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f77bb0 0x55e615f77fc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5cb218edef991a server_cookie=d9d44804983a8eba in_seq=0 out_seq=0 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f2fefe00640 10 client.6168 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f2fefe00640 1 -- 172.21.15.45:0/2454935869 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55e615f457e0 con 0x55e615f77bb0 2024-04-25T10:21:19.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f2fefe00640 10 client.6168 handle_command_reply: tid=0 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 client.6168 shutdown 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 msgr2=0x55e615fbb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615fbb420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc054990 tx=0x7f2ffc051060 comp rx=0 tx=0).stop 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 shutdown_connections 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x55e615f77bb0 0x55e615f77fc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e615f57aa0 0x55e615f76cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e615f57180 0x55e615fbb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 --2- 172.21.15.45:0/2454935869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e615f0d600 0x55e615fbb420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f2fefe00640 0 client.6168 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 >> 172.21.15.45:0/2454935869 conn(0x55e615f13760 msgr2=0x55e615f0b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 shutdown_connections 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 1 -- 172.21.15.45:0/2454935869 wait complete. 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3014de1000 20 client.6168 trim_cache size 0 max 16384 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 2042924602, 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511630189, 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:19.743 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:21:19.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fe8066f60 msgr2=0x7f2fe8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fe8066f60 0x7f2fe8069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4007fd0 tx=0x7f2ff4007ba0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 msgr2=0x7f300c152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c152f40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00c3c0 tx=0x7f2ffc00c880 comp rx=0 tx=0).stop 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 shutdown_connections 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fe8066f60 0x7f2fe8069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f300c159540 0x7f300c0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c0aeda0 0x7f300c150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 --2- 172.21.15.45:0/2013915267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c0721f0 0x7f300c152f40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 >> 172.21.15.45:0/2013915267 conn(0x7f300c0669a0 msgr2=0x7f300c0cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 shutdown_connections 2024-04-25T10:21:19.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.735+0000 7f3011a00640 1 -- 172.21.15.45:0/2013915267 wait complete. 2024-04-25T10:21:19.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:19.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 Processor -- start 2024-04-25T10:21:19.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 -- start start 2024-04-25T10:21:19.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60400d0920 0x7f60400d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60400d1ce0 0x7f60400b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60400d2490 con 0x7f604015df00 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400d2770 con 0x7f60400d1ce0 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6047a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400d2a50 con 0x7f60400d0920 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60400d1ce0 0x7f60400b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60400d1ce0 0x7f60400b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52514/0 (socket says 172.21.15.45:52514) 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 -- 172.21.15.45:0/3760255085 learned_addr learned my addr 172.21.15.45:0/3760255085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f603fe00640 1 --2- 172.21.15.45:0/3760255085 >> v2:172.21.15.117:3300/0 conn(0x7f60400d0920 0x7f60400d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 -- 172.21.15.45:0/3760255085 >> v2:172.21.15.117:3300/0 conn(0x7f60400d0920 msgr2=0x7f60400d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 --2- 172.21.15.45:0/3760255085 >> v2:172.21.15.117:3300/0 conn(0x7f60400d0920 0x7f60400d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 -- 172.21.15.45:0/3760255085 >> v1:172.21.15.117:6789/0 conn(0x7f604015df00 legacy=0x7f60400d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.899+0000 7f6045a00640 1 -- 172.21.15.45:0/3760255085 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400d2d30 con 0x7f60400d1ce0 2024-04-25T10:21:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6045a00640 1 --2- 172.21.15.45:0/3760255085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400b71e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6040067960 tx=0x7f603801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101c2d8d9374e594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f603f400640 1 -- 172.21.15.45:0/3760255085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60380a58a0 con 0x7f60400d1ce0 2024-04-25T10:21:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f603f400640 1 -- 172.21.15.45:0/3760255085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60380a5a40 con 0x7f60400d1ce0 2024-04-25T10:21:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f603f400640 1 -- 172.21.15.45:0/3760255085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60380a5e10 con 0x7f60400d1ce0 2024-04-25T10:21:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 -- 172.21.15.45:0/3760255085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 msgr2=0x7f60400b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 --2- 172.21.15.45:0/3760255085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400b71e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6040067960 tx=0x7f603801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 -- 172.21.15.45:0/3760255085 shutdown_connections 2024-04-25T10:21:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 --2- 172.21.15.45:0/3760255085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400b71e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 --2- 172.21.15.45:0/3760255085 >> v2:172.21.15.117:3300/0 conn(0x7f60400d0920 0x7f60400d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 -- 172.21.15.45:0/3760255085 >> 172.21.15.45:0/3760255085 conn(0x7f604006b290 msgr2=0x7f60400669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 -- 172.21.15.45:0/3760255085 shutdown_connections 2024-04-25T10:21:19.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 -- 172.21.15.45:0/3760255085 wait complete. 2024-04-25T10:21:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 Processor -- start 2024-04-25T10:21:19.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.903+0000 7f6047a00640 1 -- start start 2024-04-25T10:21:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60400d0920 0x7f60400c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604015df00 0x7f60400cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b80b0 con 0x7f60400d1ce0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b7f30 con 0x7f604015df00 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60400b8230 con 0x7f60400d0920 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52516/0 (socket says 172.21.15.45:52516) 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 -- 172.21.15.45:0/31172383 learned_addr learned my addr 172.21.15.45:0/31172383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6045000640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60400d0920 0x7f60400c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6045a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604015df00 0x7f60400cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 -- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60400d0920 msgr2=0x7f60400c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60400d0920 0x7f60400c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 -- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604015df00 msgr2=0x7f60400cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604015df00 0x7f60400cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 -- 172.21.15.45:0/31172383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60401f7580 con 0x7f60400d1ce0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603fe00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400cd3e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e930 tx=0x7f602c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c017070 con 0x7f60400d1ce0 2024-04-25T10:21:19.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c00ca00 con 0x7f60400d1ce0 2024-04-25T10:21:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00cdb0 con 0x7f60400d1ce0 2024-04-25T10:21:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60401f7860 con 0x7f60400d1ce0 2024-04-25T10:21:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60400c9fd0 con 0x7f60400d1ce0 2024-04-25T10:21:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604014cb10 con 0x7f60400d1ce0 2024-04-25T10:21:19.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.907+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f602c01a070 con 0x7f60400d1ce0 2024-04-25T10:21:19.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.911+0000 7f603cc00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:19.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.911+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f602c085390 con 0x7f60400d1ce0 2024-04-25T10:21:19.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.911+0000 7f6045000640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:19.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.911+0000 7f6045000640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6020066f60 0x7f6020069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6034001770 tx=0x7f6034009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:19.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:19.915+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f602c055e00 con 0x7f60400d1ce0 2024-04-25T10:21:20.039 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:21:20.035+0000 7ff8bb000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:21:20.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.119+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f604004ef30 con 0x7f60400d1ce0 2024-04-25T10:21:20.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.119+0000 7f603cc00640 1 -- 172.21.15.45:0/31172383 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f602c059ea0 con 0x7f60400d1ce0 2024-04-25T10:21:20.122 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:20.123 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:19.902145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:20.123 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6020066f60 msgr2=0x7f6020069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6020066f60 0x7f6020069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6034001770 tx=0x7f6034009040 comp rx=0 tx=0).stop 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 msgr2=0x7f60400cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400cd3e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e930 tx=0x7f602c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 shutdown_connections 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f604015df00 0x7f60400cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60400d1ce0 0x7f60400cd3e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 --2- 172.21.15.45:0/31172383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60400d0920 0x7f60400c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 >> 172.21.15.45:0/31172383 conn(0x7f604006b290 msgr2=0x7f60400b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 shutdown_connections 2024-04-25T10:21:20.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.123+0000 7f6047a00640 1 -- 172.21.15.45:0/31172383 wait complete. 2024-04-25T10:21:20.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 2042924602 2024-04-25T10:21:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.283+0000 7f0157600640 1 Processor -- start 2024-04-25T10:21:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.283+0000 7f0157600640 1 -- start start 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0157600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0150159540 0x7f01500ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0157600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01500aebd0 0x7f0150071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0157600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0150071360 con 0x7f0150072020 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0157600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0150159920 con 0x7f01500aebd0 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0157600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015015c2f0 con 0x7f0150159540 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01500aebd0 0x7f0150071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01500aebd0 0x7f0150071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52524/0 (socket says 172.21.15.45:52524) 2024-04-25T10:21:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 -- 172.21.15.45:0/4064553843 learned_addr learned my addr 172.21.15.45:0/4064553843 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0154c00640 1 --2- 172.21.15.45:0/4064553843 >> v2:172.21.15.117:3300/0 conn(0x7f0150159540 0x7f01500ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 -- 172.21.15.45:0/4064553843 >> v2:172.21.15.117:3300/0 conn(0x7f0150159540 msgr2=0x7f01500ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 --2- 172.21.15.45:0/4064553843 >> v2:172.21.15.117:3300/0 conn(0x7f0150159540 0x7f01500ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 -- 172.21.15.45:0/4064553843 >> v1:172.21.15.117:6789/0 conn(0x7f0150072020 legacy=0x7f015015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0147e00640 1 -- 172.21.15.45:0/4064553843 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015015c5d0 con 0x7f01500aebd0 2024-04-25T10:21:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.287+0000 7f0154c00640 1 --2- 172.21.15.45:0/4064553843 >> v2:172.21.15.117:3300/0 conn(0x7f0150159540 0x7f01500ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0147e00640 1 --2- 172.21.15.45:0/4064553843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01500aebd0 0x7f0150071ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f01500b4c20 tx=0x7f013801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a461144f317701a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f014fe00640 1 -- 172.21.15.45:0/4064553843 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380b0070 con 0x7f01500aebd0 2024-04-25T10:21:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f014fe00640 1 -- 172.21.15.45:0/4064553843 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01380ab600 con 0x7f01500aebd0 2024-04-25T10:21:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f014fe00640 1 -- 172.21.15.45:0/4064553843 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380ab9d0 con 0x7f01500aebd0 2024-04-25T10:21:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- 172.21.15.45:0/4064553843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01500aebd0 msgr2=0x7f0150071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 --2- 172.21.15.45:0/4064553843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01500aebd0 0x7f0150071ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f01500b4c20 tx=0x7f013801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- 172.21.15.45:0/4064553843 shutdown_connections 2024-04-25T10:21:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 --2- 172.21.15.45:0/4064553843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01500aebd0 0x7f0150071ae0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 --2- 172.21.15.45:0/4064553843 >> v2:172.21.15.117:3300/0 conn(0x7f0150159540 0x7f01500ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- 172.21.15.45:0/4064553843 >> 172.21.15.45:0/4064553843 conn(0x7f01500669a0 msgr2=0x7f0150066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- 172.21.15.45:0/4064553843 shutdown_connections 2024-04-25T10:21:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- 172.21.15.45:0/4064553843 wait complete. 2024-04-25T10:21:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 Processor -- start 2024-04-25T10:21:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- start start 2024-04-25T10:21:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0150072020 0x7f015014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01500aebd0 0x7f015014d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 0x7f0150151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015006c0d0 con 0x7f0150159540 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150072450 con 0x7f0150072020 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0157600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0150168930 con 0x7f01500aebd0 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0155600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 0x7f0150151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0154c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0150072020 0x7f015014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0155600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 0x7f0150151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52532/0 (socket says 172.21.15.45:52532) 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.291+0000 7f0155600640 1 -- 172.21.15.45:0/1305379845 learned_addr learned my addr 172.21.15.45:0/1305379845 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0147e00640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01500aebd0 0x7f015014d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0155600640 1 -- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01500aebd0 msgr2=0x7f015014d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0155600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01500aebd0 0x7f015014d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0155600640 1 -- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0150072020 msgr2=0x7f015014ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0155600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0150072020 0x7f015014ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0155600640 1 -- 172.21.15.45:0/1305379845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500680c0 con 0x7f0150159540 2024-04-25T10:21:20.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0155600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 0x7f0150151430 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f014800c780 tx=0x7f014800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f014d600640 1 -- 172.21.15.45:0/1305379845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148004b70 con 0x7f0150159540 2024-04-25T10:21:20.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f014d600640 1 -- 172.21.15.45:0/1305379845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148004d10 con 0x7f0150159540 2024-04-25T10:21:20.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f014d600640 1 -- 172.21.15.45:0/1305379845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800aa90 con 0x7f0150159540 2024-04-25T10:21:20.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015014cf90 con 0x7f0150159540 2024-04-25T10:21:20.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015014de20 con 0x7f0150159540 2024-04-25T10:21:20.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f014d600640 1 -- 172.21.15.45:0/1305379845 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f014800ac30 con 0x7f0150159540 2024-04-25T10:21:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.295+0000 7f014d600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f012c066f90 0x7f012c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f014d600640 1 -- 172.21.15.45:0/1305379845 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0148085170 con 0x7f0150159540 2024-04-25T10:21:20.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 Processor -- start 2024-04-25T10:21:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- start start 2024-04-25T10:21:20.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0154c00640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f012c066f90 0x7f012c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f5ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0154c00640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f012c066f90 0x7f012c069450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0140006fd0 tx=0x7f0140008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce990 0x56524f5816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f581fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524f3d1f00 con 0x56524f581be0 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524f35a4a0 con 0x56524f5ce990 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56524f3cdfa0 con 0x56524f5ce070 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f581fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0154c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f5ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f581fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52544/0 (socket says 172.21.15.45:52544) 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 -- 172.21.15.45:0/2348306110 learned_addr learned my addr 172.21.15.45:0/2348306110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0147e00640 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce990 0x56524f5816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 -- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 msgr2=0x56524f5ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f5ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 -- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce990 msgr2=0x56524f5816a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce990 0x56524f5816a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 -- 172.21.15.45:0/2348306110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56524f357d00 con 0x56524f581be0 2024-04-25T10:21:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0155600640 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f581fc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01480835d0 tx=0x7f0148055480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e13efdbe588b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0145600640 1 -- 172.21.15.45:0/2348306110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014806d030 con 0x56524f581be0 2024-04-25T10:21:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0145600640 1 -- 172.21.15.45:0/2348306110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014805e180 con 0x56524f581be0 2024-04-25T10:21:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f0145600640 1 -- 172.21.15.45:0/2348306110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480832d0 con 0x56524f581be0 2024-04-25T10:21:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- 172.21.15.45:0/2348306110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 msgr2=0x56524f581fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f581fc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01480835d0 tx=0x7f0148055480 comp rx=0 tx=0).stop 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- 172.21.15.45:0/2348306110 shutdown_connections 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f581fc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce990 0x56524f5816a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 --2- 172.21.15.45:0/2348306110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f5ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- 172.21.15.45:0/2348306110 >> 172.21.15.45:0/2348306110 conn(0x56524f5aee10 msgr2=0x56524f5b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- 172.21.15.45:0/2348306110 shutdown_connections 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- 172.21.15.45:0/2348306110 wait complete. 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 -- start start 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.299+0000 7f015aa98000 1 Processor -- start 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f5cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f587ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f5881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524f3d6530 con 0x56524f581be0 2024-04-25T10:21:20.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524f38cae0 con 0x56524f5ce070 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56524f390e10 con 0x56524f5ce990 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 10 client.0 _ll_register_callbacks cb 0x56524f56f750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f5cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f5cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52548/0 (socket says 172.21.15.45:52548) 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 -- 172.21.15.45:0/602671163 learned_addr learned my addr 172.21.15.45:0/602671163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 -- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 msgr2=0x56524f5881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0147e00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f587ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0155600640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f5881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f5881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 -- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 msgr2=0x56524f587ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f587ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 -- 172.21.15.45:0/602671163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56524f374a10 con 0x56524f581be0 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0147e00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f587ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0155600640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f5881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0154c00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f5cd800 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0140015a00 tx=0x7f0140004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e932d5806bf740a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f013e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f013e000640 1 -- 172.21.15.45:0/602671163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140025030 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f013e000640 1 -- 172.21.15.45:0/602671163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0140013bb0 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f014805e180 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f013e000640 1 -- 172.21.15.45:0/602671163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0140031050 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 10 client.6186 fetch_fsmap learned FSMap version 30 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x56524f3ccef0 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f013e000640 1 -- 172.21.15.45:0/602671163 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014805e180 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f013e000640 1 -- 172.21.15.45:0/602671163 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0140034020 con 0x56524f581be0 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 10 client.6186 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 10 client.6186 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 20 client.6186 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5d56c0 0x56524f5d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 4 client.6186 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56524f36db20 con 0x56524f5d56c0 2024-04-25T10:21:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.303+0000 7f0147e00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5d56c0 0x56524f5d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f0147e00640 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5d56c0 0x56524f5d5aa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3496155e8052420e server_cookie=78b9bbb7f0c12f44 in_seq=0 out_seq=0 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f013e000640 10 client.6186 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f013e000640 1 -- 172.21.15.45:0/602671163 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56524f36db20 con 0x56524f5d56c0 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f013e000640 10 client.6186 handle_command_reply: tid=0 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 client.6186 shutdown 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 msgr2=0x56524f5cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f5cd800 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0140015a00 tx=0x7f0140004740 comp rx=0 tx=0).stop 2024-04-25T10:21:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 shutdown_connections 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5d56c0 0x56524f5d5aa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f5881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f587ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f013e000640 0 client.6186 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 --2- 172.21.15.45:0/602671163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f5cd800 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 >> 172.21.15.45:0/602671163 conn(0x56524f5aee10 msgr2=0x56524f580da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 shutdown_connections 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 1 -- 172.21.15.45:0/602671163 wait complete. 2024-04-25T10:21:20.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.307+0000 7f015aa98000 20 client.6186 trim_cache size 0 max 16384 2024-04-25T10:21:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 Processor -- start 2024-04-25T10:21:20.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 -- start start 2024-04-25T10:21:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f580510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f580a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f589cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01501ffe10 con 0x56524f581be0 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01501ff240 con 0x56524f5ce070 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01501ff810 con 0x56524f5ce990 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f580510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f580510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52564/0 (socket says 172.21.15.45:52564) 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 -- 172.21.15.45:0/4254567594 learned_addr learned my addr 172.21.15.45:0/4254567594 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0155600640 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f589cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0147e00640 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f580a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 -- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 msgr2=0x56524f589cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f589cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 -- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 msgr2=0x56524f580a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f580a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 -- 172.21.15.45:0/4254567594 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56524f5ce7c0 con 0x56524f581be0 2024-04-25T10:21:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0154c00640 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f580510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f014000ffb0 tx=0x7f0140021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dcc3db680bb42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0144c00640 1 -- 172.21.15.45:0/4254567594 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140039070 con 0x56524f581be0 2024-04-25T10:21:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0144c00640 1 -- 172.21.15.45:0/4254567594 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140026040 con 0x56524f581be0 2024-04-25T10:21:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f0144c00640 1 -- 172.21.15.45:0/4254567594 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140025050 con 0x56524f581be0 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 -- 172.21.15.45:0/4254567594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 msgr2=0x56524f580510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.319+0000 7f015aa98000 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f580510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f014000ffb0 tx=0x7f0140021000 comp rx=0 tx=0).stop 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/4254567594 shutdown_connections 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce990 0x56524f589cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f5ce070 0x56524f580a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- 172.21.15.45:0/4254567594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f581be0 0x56524f580510 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/4254567594 >> 172.21.15.45:0/4254567594 conn(0x56524f5aee10 msgr2=0x56524f57fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/4254567594 shutdown_connections 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/4254567594 wait complete. 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- start start 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 Processor -- start 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f581be0 0x56524f633680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f633bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 0x56524f5eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524f411ca0 con 0x56524f5ce990 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56524f39eb90 con 0x56524f581be0 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56524f581750 con 0x56524f5ce070 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 10 client.0 _ll_register_callbacks cb 0x7f014800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 0x56524f5eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 0x56524f5eed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52574/0 (socket says 172.21.15.45:52574) 2024-04-25T10:21:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 -- 172.21.15.45:0/3952267297 learned_addr learned my addr 172.21.15.45:0/3952267297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0154c00640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f581be0 0x56524f633680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0147e00640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f633bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 -- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 msgr2=0x56524f633bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f633bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 -- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f581be0 msgr2=0x56524f633680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f581be0 0x56524f633680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 -- 172.21.15.45:0/3952267297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56524f391790 con 0x56524f5ce990 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0155600640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 0x56524f5eed00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f014805dc50 tx=0x7f0148058070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1044d362ec1115f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f013e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f013e000640 1 -- 172.21.15.45:0/3952267297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148078060 con 0x56524f5ce990 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f013e000640 1 -- 172.21.15.45:0/3952267297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01480832d0 con 0x56524f5ce990 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0140026040 con 0x56524f5ce990 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f013e000640 1 -- 172.21.15.45:0/3952267297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f014806d040 con 0x56524f5ce990 2024-04-25T10:21:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 10 client.6192 fetch_fsmap learned FSMap version 30 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x56524f399da0 con 0x56524f5ce990 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f013e000640 1 -- 172.21.15.45:0/3952267297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140026040 con 0x56524f5ce990 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f013e000640 1 -- 172.21.15.45:0/3952267297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0148052190 con 0x56524f5ce990 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 10 client.6192 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 10 client.6192 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 20 client.6192 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5ef630 0x56524f5efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 4 client.6192 mds_command: new command op to 5574 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2042924602}] 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 --> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2042924602}) v1 -- 0x56524f3df440 con 0x56524f5ef630 2024-04-25T10:21:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.323+0000 7f0154c00640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5ef630 0x56524f5efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0154c00640 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5ef630 0x56524f5efa10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e601628cb1d8202d server_cookie=9d1b786ef89409c8 in_seq=0 out_seq=0 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f013e000640 10 client.6192 ms_handle_connect on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f013e000640 1 -- 172.21.15.45:0/3952267297 <== mds.0 v2:172.21.15.117:6834/3308184634 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x56524f3df440 con 0x56524f5ef630 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f013e000640 10 client.6192 handle_command_reply: tid=0 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 client.6192 shutdown 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 msgr2=0x56524f5eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 0x56524f5eed00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f014805dc50 tx=0x7f0148058070 comp rx=0 tx=0).stop 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 shutdown_connections 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:6834/3308184634,v1:172.21.15.117:6836/3308184634] conn(0x56524f5ef630 0x56524f5efa10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56524f5ce990 0x56524f5eed00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56524f5ce070 0x56524f633bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f013e000640 0 client.6192 ms_handle_reset on v2:172.21.15.117:6834/3308184634 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 --2- 172.21.15.45:0/3952267297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56524f581be0 0x56524f633680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 >> 172.21.15.45:0/3952267297 conn(0x56524f5aee10 msgr2=0x56524f580690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 shutdown_connections 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 1 -- 172.21.15.45:0/3952267297 wait complete. 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f015aa98000 20 client.6192 trim_cache size 0 max 16384 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f012c066f90 msgr2=0x7f012c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f012c066f90 0x7f012c069450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0140006fd0 tx=0x7f0140008040 comp rx=0 tx=0).stop 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 msgr2=0x7f0150151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 0x7f0150151430 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f014800c780 tx=0x7f014800cc40 comp rx=0 tx=0).stop 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 shutdown_connections 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f012c066f90 0x7f012c069450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0150159540 0x7f0150151430 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01500aebd0 0x7f015014d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 --2- 172.21.15.45:0/1305379845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0150072020 0x7f015014ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 >> 172.21.15.45:0/1305379845 conn(0x7f01500669a0 msgr2=0x7f015015e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 shutdown_connections 2024-04-25T10:21:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.327+0000 7f0157600640 1 -- 172.21.15.45:0/1305379845 wait complete. 2024-04-25T10:21:20.348 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:21:20.349 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 Processor -- start 2024-04-25T10:21:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 -- start start 2024-04-25T10:21:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 0x7f15a815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15a815eea0 0x7f15a80669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a815e400 con 0x7f15a815d9b0 2024-04-25T10:21:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80b02e0 con 0x7f15a815eea0 2024-04-25T10:21:20.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15af200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80afda0 con 0x7f15a80b38d0 2024-04-25T10:21:20.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f159f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 0x7f15a815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15a7e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f15a815d9b0 0x7f15a80af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:55804/0 (socket says 172.21.15.45:55804) 2024-04-25T10:21:20.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15a7e00640 1 -- 172.21.15.45:0/624779024 learned_addr learned my addr 172.21.15.45:0/624779024 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:20.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f15ad200640 1 --2- 172.21.15.45:0/624779024 >> v2:172.21.15.117:3300/0 conn(0x7f15a815eea0 0x7f15a80669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f159f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 0x7f15a815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37286/0 (socket says 172.21.15.45:37286) 2024-04-25T10:21:20.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f159f400640 1 -- 172.21.15.45:0/624779024 >> v2:172.21.15.117:3300/0 conn(0x7f15a815eea0 msgr2=0x7f15a80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:20.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f159f400640 1 --2- 172.21.15.45:0/624779024 >> v2:172.21.15.117:3300/0 conn(0x7f15a815eea0 0x7f15a80669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f159f400640 1 -- 172.21.15.45:0/624779024 >> v1:172.21.15.117:6789/0 conn(0x7f15a815d9b0 legacy=0x7f15a80af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.487+0000 7f159f400640 1 -- 172.21.15.45:0/624779024 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80b0000 con 0x7f15a80b38d0 2024-04-25T10:21:20.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f159f400640 1 --2- 172.21.15.45:0/624779024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 0x7f15a815dd90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f15900097c0 tx=0x7f159001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f4453fc00e906f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15a7400640 1 -- 172.21.15.45:0/624779024 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900b0070 con 0x7f15a80b38d0 2024-04-25T10:21:20.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15a7400640 1 -- 172.21.15.45:0/624779024 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15900ab600 con 0x7f15a80b38d0 2024-04-25T10:21:20.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15a7400640 1 -- 172.21.15.45:0/624779024 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900ab9d0 con 0x7f15a80b38d0 2024-04-25T10:21:20.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 -- 172.21.15.45:0/624779024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 msgr2=0x7f15a815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 --2- 172.21.15.45:0/624779024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 0x7f15a815dd90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f15900097c0 tx=0x7f159001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 -- 172.21.15.45:0/624779024 shutdown_connections 2024-04-25T10:21:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 --2- 172.21.15.45:0/624779024 >> v2:172.21.15.117:3300/0 conn(0x7f15a815eea0 0x7f15a80669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 --2- 172.21.15.45:0/624779024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a80b38d0 0x7f15a815dd90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 -- 172.21.15.45:0/624779024 >> 172.21.15.45:0/624779024 conn(0x7f15a806ac90 msgr2=0x7f15a806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 -- 172.21.15.45:0/624779024 shutdown_connections 2024-04-25T10:21:20.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.491+0000 7f15af200640 1 -- 172.21.15.45:0/624779024 wait complete. 2024-04-25T10:21:20.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 Processor -- start 2024-04-25T10:21:20.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 -- start start 2024-04-25T10:21:20.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 0x7f15a814e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a815d9b0 0x7f15a815fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a815eea0 0x7f15a81fb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80d1120 con 0x7f15a80b38d0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80d0fa0 con 0x7f15a815eea0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80d12a0 con 0x7f15a815d9b0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 0x7f15a814e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 0x7f15a814e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52576/0 (socket says 172.21.15.45:52576) 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 -- 172.21.15.45:0/3142632795 learned_addr learned my addr 172.21.15.45:0/3142632795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f159f400640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a815d9b0 0x7f15a815fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15ad200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a815eea0 0x7f15a81fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 -- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a815d9b0 msgr2=0x7f15a815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a815d9b0 0x7f15a815fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 -- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a815eea0 msgr2=0x7f15a81fb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a815eea0 0x7f15a81fb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 -- 172.21.15.45:0/3142632795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a814da80 con 0x7f15a80b38d0 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f159f400640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a815d9b0 0x7f15a815fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a7e00640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 0x7f15a814e840 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f159800c420 tx=0x7f159800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598017070 con 0x7f15a80b38d0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598005cf0 con 0x7f15a80b38d0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15980072f0 con 0x7f15a80b38d0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a814dd60 con 0x7f15a80b38d0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.495+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a814e320 con 0x7f15a80b38d0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.499+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a804ef30 con 0x7f15a80b38d0 2024-04-25T10:21:20.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.499+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f159801a070 con 0x7f15a80b38d0 2024-04-25T10:21:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.499+0000 7f15a4c00640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1584066f60 0x7f1584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.499+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1598084490 con 0x7f15a80b38d0 2024-04-25T10:21:20.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.499+0000 7f159f400640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1584066f60 0x7f1584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:20.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.499+0000 7f159f400640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1584066f60 0x7f1584069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15900ad000 tx=0x7f1590008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.503+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1598055400 con 0x7f15a80b38d0 2024-04-25T10:21:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a8068190 con 0x7f15a80b38d0 2024-04-25T10:21:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15a4c00640 1 -- 172.21.15.45:0/3142632795 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f1598054d00 con 0x7f15a80b38d0 2024-04-25T10:21:20.702 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:20.702 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:19.902145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1584066f60 msgr2=0x7f1584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1584066f60 0x7f1584069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15900ad000 tx=0x7f1590008040 comp rx=0 tx=0).stop 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 msgr2=0x7f15a814e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 0x7f15a814e840 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f159800c420 tx=0x7f159800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 shutdown_connections 2024-04-25T10:21:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1584066f60 0x7f1584069420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a815eea0 0x7f15a81fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15a815d9b0 0x7f15a815fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 --2- 172.21.15.45:0/3142632795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a80b38d0 0x7f15a814e840 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 >> 172.21.15.45:0/3142632795 conn(0x7f15a806ac90 msgr2=0x7f15a814ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 shutdown_connections 2024-04-25T10:21:20.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:20.699+0000 7f15af200640 1 -- 172.21.15.45:0/3142632795 wait complete. 2024-04-25T10:21:20.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:21:20.874 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-25T10:21:20.875 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0 2024-04-25T10:21:20.875 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-25T10:21:20.890 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:21:20.890 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:21:20.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:20.979 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:21.073 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:21:21.073 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:21.074 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:21.083 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:21.083 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:21.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:21.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 Processor -- start 2024-04-25T10:21:21.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 -- start start 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7b415a760 0x7fd7b40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40b44a0 0x7fd7b40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b4071160 con 0x7fd7b40cd3a0 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b40b34a0 con 0x7fd7b415a760 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40b3780 con 0x7fd7b40b44a0 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7b415a760 0x7fd7b40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7b415a760 0x7fd7b40b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52580/0 (socket says 172.21.15.45:52580) 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 -- 172.21.15.45:0/2999171660 learned_addr learned my addr 172.21.15.45:0/2999171660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:21.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b2800640 1 --1- 172.21.15.45:0/2999171660 >> v1:172.21.15.45:6789/0 conn(0x7fd7b40cd3a0 0x7fd7b406de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41486/0 (socket says 172.21.15.45:41486) 2024-04-25T10:21:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7a9400640 1 --2- 172.21.15.45:0/2999171660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40b44a0 0x7fd7b40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 -- 172.21.15.45:0/2999171660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40b44a0 msgr2=0x7fd7b40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 --2- 172.21.15.45:0/2999171660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40b44a0 0x7fd7b40cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1400640 1 -- 172.21.15.45:0/2999171660 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243569980 0 0) 0x7fd7b4071160 con 0x7fd7b40cd3a0 2024-04-25T10:21:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 -- 172.21.15.45:0/2999171660 >> v1:172.21.15.45:6789/0 conn(0x7fd7b40cd3a0 legacy=0x7fd7b406de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.315+0000 7fd7b1e00640 1 -- 172.21.15.45:0/2999171660 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b40b3a60 con 0x7fd7b415a760 2024-04-25T10:21:21.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b1e00640 1 --2- 172.21.15.45:0/2999171660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b415a760 0x7fd7b40b2ec0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400bb10 tx=0x7fd7a401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d3d31a1244c26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:21.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b1400640 1 -- 172.21.15.45:0/2999171660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b4070 con 0x7fd7b415a760 2024-04-25T10:21:21.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b1400640 1 -- 172.21.15.45:0/2999171660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40a7c70 con 0x7fd7b415a760 2024-04-25T10:21:21.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b1400640 1 -- 172.21.15.45:0/2999171660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40af660 con 0x7fd7b415a760 2024-04-25T10:21:21.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 -- 172.21.15.45:0/2999171660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b415a760 msgr2=0x7fd7b40b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 --2- 172.21.15.45:0/2999171660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b415a760 0x7fd7b40b2ec0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400bb10 tx=0x7fd7a401ec00 comp rx=0 tx=0).stop 2024-04-25T10:21:21.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 -- 172.21.15.45:0/2999171660 shutdown_connections 2024-04-25T10:21:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 --2- 172.21.15.45:0/2999171660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40b44a0 0x7fd7b40cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 --2- 172.21.15.45:0/2999171660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b415a760 0x7fd7b40b2ec0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 -- 172.21.15.45:0/2999171660 >> 172.21.15.45:0/2999171660 conn(0x7fd7b406ae60 msgr2=0x7fd7b406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 -- 172.21.15.45:0/2999171660 shutdown_connections 2024-04-25T10:21:21.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 -- 172.21.15.45:0/2999171660 wait complete. 2024-04-25T10:21:21.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 Processor -- start 2024-04-25T10:21:21.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.319+0000 7fd7b3200640 1 -- start start 2024-04-25T10:21:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 0x7fd7b415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cd3a0 0x7fd7b4159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b415a760 0x7fd7b4159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b415f2b0 con 0x7fd7b40b44a0 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b415f130 con 0x7fd7b40cd3a0 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b406e510 con 0x7fd7b415a760 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 0x7fd7b415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 0x7fd7b415ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52592/0 (socket says 172.21.15.45:52592) 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 -- 172.21.15.45:0/1345326009 learned_addr learned my addr 172.21.15.45:0/1345326009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7a9400640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cd3a0 0x7fd7b4159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b2800640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b415a760 0x7fd7b4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 -- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b415a760 msgr2=0x7fd7b4159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b415a760 0x7fd7b4159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 -- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cd3a0 msgr2=0x7fd7b4159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cd3a0 0x7fd7b4159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 -- 172.21.15.45:0/1345326009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b40cec20 con 0x7fd7b40b44a0 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7a9400640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cd3a0 0x7fd7b4159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:21.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b1e00640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 0x7fd7b415ae50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400bb10 tx=0x7fd7a401b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b4040 con 0x7fd7b40b44a0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40afbf0 con 0x7fd7b40b44a0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4004020 con 0x7fd7b40b44a0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b40cef00 con 0x7fd7b40b44a0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b4141220 con 0x7fd7b40b44a0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.323+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b4075120 con 0x7fd7b40b44a0 2024-04-25T10:21:21.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.327+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7a40b7070 con 0x7fd7b40b44a0 2024-04-25T10:21:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.327+0000 7fd7aaa00640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd790066f60 0x7fd790069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.327+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7a41222c0 con 0x7fd7b40b44a0 2024-04-25T10:21:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.327+0000 7fd7a9400640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd790066f60 0x7fd790069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.327+0000 7fd7a9400640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd790066f60 0x7fd790069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4148d30 tx=0x7fd79c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.331+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7a40c7030 con 0x7fd7b40b44a0 2024-04-25T10:21:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.491+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7b40d08e0 con 0x7fd7b40b44a0 2024-04-25T10:21:21.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7aaa00640 1 -- 172.21.15.45:0/1345326009 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fd7a40be030 con 0x7fd7b40b44a0 2024-04-25T10:21:21.497 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:21.497 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":50,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:48.857409+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd790066f60 msgr2=0x7fd790069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd790066f60 0x7fd790069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4148d30 tx=0x7fd79c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 msgr2=0x7fd7b415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 0x7fd7b415ae50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400bb10 tx=0x7fd7a401b9a0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 shutdown_connections 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd790066f60 0x7fd790069420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b415a760 0x7fd7b4159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cd3a0 0x7fd7b4159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 --2- 172.21.15.45:0/1345326009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7b40b44a0 0x7fd7b415ae50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 >> 172.21.15.45:0/1345326009 conn(0x7fd7b406ae60 msgr2=0x7fd7b41494f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 shutdown_connections 2024-04-25T10:21:21.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.495+0000 7fd7b3200640 1 -- 172.21.15.45:0/1345326009 wait complete. 2024-04-25T10:21:21.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:20:45.743832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:46.252078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:21.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 Processor -- start 2024-04-25T10:21:21.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 -- start start 2024-04-25T10:21:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c159540 0x7f9a9c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9a9c0721f0 0x7f9a9c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c159920 con 0x7f9a9c0aebd0 2024-04-25T10:21:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0af030 con 0x7f9a9c0721f0 2024-04-25T10:21:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9aa3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c15c2f0 con 0x7f9a9c159540 2024-04-25T10:21:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.659+0000 7f9a93e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9a9c0aebd0 0x7f9a9c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41500/0 (socket says 172.21.15.45:41500) 2024-04-25T10:21:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9a93e00640 1 -- 172.21.15.45:0/4189123410 learned_addr learned my addr 172.21.15.45:0/4189123410 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1c00640 1 --2- 172.21.15.45:0/4189123410 >> v2:172.21.15.117:3300/0 conn(0x7f9a9c0721f0 0x7f9a9c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1200640 1 --2- 172.21.15.45:0/4189123410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c159540 0x7f9a9c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9a9be00640 1 -- 172.21.15.45:0/4189123410 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941072669 0 0) 0x7f9a9c159920 con 0x7f9a9c0aebd0 2024-04-25T10:21:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9a9be00640 1 -- 172.21.15.45:0/4189123410 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a80003530 con 0x7f9a9c0aebd0 2024-04-25T10:21:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1200640 1 -- 172.21.15.45:0/4189123410 >> v2:172.21.15.117:3300/0 conn(0x7f9a9c0721f0 msgr2=0x7f9a9c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1200640 1 --2- 172.21.15.45:0/4189123410 >> v2:172.21.15.117:3300/0 conn(0x7f9a9c0721f0 0x7f9a9c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1200640 1 -- 172.21.15.45:0/4189123410 >> v1:172.21.15.45:6789/0 conn(0x7f9a9c0aebd0 legacy=0x7f9a9c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1200640 1 -- 172.21.15.45:0/4189123410 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c15c5d0 con 0x7f9a9c159540 2024-04-25T10:21:21.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1c00640 1 --2- 172.21.15.45:0/4189123410 >> v2:172.21.15.117:3300/0 conn(0x7f9a9c0721f0 0x7f9a9c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.663+0000 7f9aa1200640 1 --2- 172.21.15.45:0/4189123410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c159540 0x7f9a9c0ae690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0097c0 tx=0x7f9a8c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb385530f1e486fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9a9be00640 1 -- 172.21.15.45:0/4189123410 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0a7070 con 0x7f9a9c159540 2024-04-25T10:21:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9a9be00640 1 -- 172.21.15.45:0/4189123410 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c0b0070 con 0x7f9a9c159540 2024-04-25T10:21:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9a9be00640 1 -- 172.21.15.45:0/4189123410 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0ab870 con 0x7f9a9c159540 2024-04-25T10:21:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- 172.21.15.45:0/4189123410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c159540 msgr2=0x7f9a9c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/4189123410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c159540 0x7f9a9c0ae690 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0097c0 tx=0x7f9a8c01c470 comp rx=0 tx=0).stop 2024-04-25T10:21:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- 172.21.15.45:0/4189123410 shutdown_connections 2024-04-25T10:21:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/4189123410 >> v2:172.21.15.117:3300/0 conn(0x7f9a9c0721f0 0x7f9a9c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/4189123410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c159540 0x7f9a9c0ae690 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- 172.21.15.45:0/4189123410 >> 172.21.15.45:0/4189123410 conn(0x7f9a9c0669a0 msgr2=0x7f9a9c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- 172.21.15.45:0/4189123410 shutdown_connections 2024-04-25T10:21:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- 172.21.15.45:0/4189123410 wait complete. 2024-04-25T10:21:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 Processor -- start 2024-04-25T10:21:21.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- start start 2024-04-25T10:21:21.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c0721f0 0x7f9a9c140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a9c0aebd0 0x7f9a9c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 0x7f9a9c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0710a0 con 0x7f9a9c159540 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c06c0d0 con 0x7f9a9c0aebd0 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c071220 con 0x7f9a9c0721f0 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa1c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 0x7f9a9c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa1c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 0x7f9a9c157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52594/0 (socket says 172.21.15.45:52594) 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.667+0000 7f9aa1c00640 1 -- 172.21.15.45:0/1999605901 learned_addr learned my addr 172.21.15.45:0/1999605901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1200640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c0721f0 0x7f9a9c140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a93e00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a9c0aebd0 0x7f9a9c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1c00640 1 -- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c0721f0 msgr2=0x7f9a9c140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c0721f0 0x7f9a9c140f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1c00640 1 -- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a9c0aebd0 msgr2=0x7f9a9c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a9c0aebd0 0x7f9a9c1576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1c00640 1 -- 172.21.15.45:0/1999605901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c0affb0 con 0x7f9a9c159540 2024-04-25T10:21:21.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa1c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 0x7f9a9c157c20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009f00 tx=0x7f9a9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94018030 con 0x7f9a9c159540 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9400ce50 con 0x7f9a9c159540 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94003bc0 con 0x7f9a9c159540 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c141900 con 0x7f9a9c159540 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c1414d0 con 0x7f9a9c159540 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c04ef30 con 0x7f9a9c159540 2024-04-25T10:21:21.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a9401b070 con 0x7f9a9c159540 2024-04-25T10:21:21.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a99600640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a78066f90 0x7f9a78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:21.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.671+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a94085b10 con 0x7f9a9c159540 2024-04-25T10:21:21.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.675+0000 7f9aa1200640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a78066f90 0x7f9a78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:21.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.675+0000 7f9aa1200640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a78066f90 0x7f9a78069450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0af000 tx=0x7f9a8c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:21.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.675+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a94010040 con 0x7f9a9c159540 2024-04-25T10:21:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.839+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a9c0cf470 con 0x7f9a9c159540 2024-04-25T10:21:21.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.839+0000 7f9a99600640 1 -- 172.21.15.45:0/1999605901 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a94056a80 con 0x7f9a9c159540 2024-04-25T10:21:21.844 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:21:21.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a78066f90 msgr2=0x7f9a78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a78066f90 0x7f9a78069450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0af000 tx=0x7f9a8c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 msgr2=0x7f9a9c157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 0x7f9a9c157c20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009f00 tx=0x7f9a9400c490 comp rx=0 tx=0).stop 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 shutdown_connections 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9a78066f90 0x7f9a78069450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a9c159540 0x7f9a9c157c20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a9c0aebd0 0x7f9a9c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 --2- 172.21.15.45:0/1999605901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a9c0721f0 0x7f9a9c140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 >> 172.21.15.45:0/1999605901 conn(0x7f9a9c0669a0 msgr2=0x7f9a9c0ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 shutdown_connections 2024-04-25T10:21:21.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:21.843+0000 7f9aa3c00640 1 -- 172.21.15.45:0/1999605901 wait complete. 2024-04-25T10:21:21.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:21:22.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d3200640 1 Processor -- start 2024-04-25T10:21:22.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d3200640 1 -- start start 2024-04-25T10:21:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d3200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff5cc0b38d0 0x7ff5cc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d3200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc15b080 con 0x7ff5cc0aef80 2024-04-25T10:21:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d3200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc15adb0 con 0x7ff5cc070350 2024-04-25T10:21:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d3200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc15eb60 con 0x7ff5cc0b38d0 2024-04-25T10:21:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5cbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5cc0aef80 0x7ff5cc15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41504/0 (socket says 172.21.15.45:41504) 2024-04-25T10:21:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5cbe00640 1 -- 172.21.15.45:0/2291743497 learned_addr learned my addr 172.21.15.45:0/2291743497 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d1200640 1 --2- 172.21.15.45:0/2291743497 >> v2:172.21.15.45:3300/0 conn(0x7ff5cc0b38d0 0x7ff5cc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d1200640 1 -- 172.21.15.45:0/2291743497 >> v1:172.21.15.117:6789/0 conn(0x7ff5cc070350 legacy=0x7ff5cc0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d1200640 1 -- 172.21.15.45:0/2291743497 >> v1:172.21.15.45:6789/0 conn(0x7ff5cc0aef80 legacy=0x7ff5cc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5d1200640 1 -- 172.21.15.45:0/2291743497 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc15ee40 con 0x7ff5cc0b38d0 2024-04-25T10:21:22.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.007+0000 7ff5cb400640 1 -- 172.21.15.45:0/2291743497 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109921818 0 0) 0x7ff5cc15b080 con 0x7ff5cc0aef80 2024-04-25T10:21:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d1200640 1 --2- 172.21.15.45:0/2291743497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc0b38d0 0x7ff5cc15e480 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009d90 tx=0x7ff5c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d758997e9635047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5cb400640 1 -- 172.21.15.45:0/2291743497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b2070 con 0x7ff5cc0b38d0 2024-04-25T10:21:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5cb400640 1 -- 172.21.15.45:0/2291743497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c40a5a10 con 0x7ff5cc0b38d0 2024-04-25T10:21:22.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5cb400640 1 -- 172.21.15.45:0/2291743497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40a5de0 con 0x7ff5cc0b38d0 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 -- 172.21.15.45:0/2291743497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc0b38d0 msgr2=0x7ff5cc15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 --2- 172.21.15.45:0/2291743497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc0b38d0 0x7ff5cc15e480 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009d90 tx=0x7ff5c401c970 comp rx=0 tx=0).stop 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 -- 172.21.15.45:0/2291743497 shutdown_connections 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 --2- 172.21.15.45:0/2291743497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc0b38d0 0x7ff5cc15e480 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 -- 172.21.15.45:0/2291743497 >> 172.21.15.45:0/2291743497 conn(0x7ff5cc0669a0 msgr2=0x7ff5cc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 -- 172.21.15.45:0/2291743497 shutdown_connections 2024-04-25T10:21:22.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.011+0000 7ff5d3200640 1 -- 172.21.15.45:0/2291743497 wait complete. 2024-04-25T10:21:22.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 Processor -- start 2024-04-25T10:21:22.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- start start 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5cc070350 0x7ff5cc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5cc0aef80 0x7ff5cc1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 0x7ff5cc1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1689b0 con 0x7ff5cc160240 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc168830 con 0x7ff5cc070350 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc168b30 con 0x7ff5cc0aef80 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 0x7ff5cc1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 0x7ff5cc1606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52612/0 (socket says 172.21.15.45:52612) 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 -- 172.21.15.45:0/3784908497 learned_addr learned my addr 172.21.15.45:0/3784908497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5cbe00640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5cc070350 0x7ff5cc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5c3e00640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5cc0aef80 0x7ff5cc1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 -- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5cc0aef80 msgr2=0x7ff5cc1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5cc0aef80 0x7ff5cc1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 -- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5cc070350 msgr2=0x7ff5cc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5cc070350 0x7ff5cc155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 -- 172.21.15.45:0/3784908497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc0b0040 con 0x7ff5cc160240 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d1200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 0x7ff5cc1606f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b0040 tx=0x7ff5c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40bd070 con 0x7ff5cc160240 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4003ba0 con 0x7ff5cc160240 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40b2040 con 0x7ff5cc160240 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc0b0320 con 0x7ff5cc160240 2024-04-25T10:21:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc0b08e0 con 0x7ff5cc160240 2024-04-25T10:21:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.015+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc04ef30 con 0x7ff5cc160240 2024-04-25T10:21:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.019+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5c4004030 con 0x7ff5cc160240 2024-04-25T10:21:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.019+0000 7ff5c8c00640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a8066f90 0x7ff5a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.019+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5c411fde0 con 0x7ff5cc160240 2024-04-25T10:21:22.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.019+0000 7ff5cbe00640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a8066f90 0x7ff5a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.019+0000 7ff5cbe00640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a8066f90 0x7ff5a8069450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc007a90 tx=0x7ff5bc0079e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.023+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5c40c7020 con 0x7ff5cc160240 2024-04-25T10:21:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.183+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5cc0b2fa0 con 0x7ff5cc160240 2024-04-25T10:21:22.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.183+0000 7ff5c8c00640 1 -- 172.21.15.45:0/3784908497 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5c4016560 con 0x7ff5cc160240 2024-04-25T10:21:22.187 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:21:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a8066f90 msgr2=0x7ff5a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a8066f90 0x7ff5a8069450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc007a90 tx=0x7ff5bc0079e0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 msgr2=0x7ff5cc1606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 0x7ff5cc1606f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40b0040 tx=0x7ff5c4019710 comp rx=0 tx=0).stop 2024-04-25T10:21:22.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 shutdown_connections 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5a8066f90 0x7ff5a8069450 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5cc160240 0x7ff5cc1606f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5cc0aef80 0x7ff5cc1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 --2- 172.21.15.45:0/3784908497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5cc070350 0x7ff5cc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 >> 172.21.15.45:0/3784908497 conn(0x7ff5cc0669a0 msgr2=0x7ff5cc15d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 shutdown_connections 2024-04-25T10:21:22.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.187+0000 7ff5d3200640 1 -- 172.21.15.45:0/3784908497 wait complete. 2024-04-25T10:21:22.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:21:22.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb2c600640 1 Processor -- start 2024-04-25T10:21:22.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb2c600640 1 -- start start 2024-04-25T10:21:22.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb2c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb2c600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb24159920 con 0x7fbb240aebd0 2024-04-25T10:21:22.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb2c600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2415c2f0 con 0x7fbb240721f0 2024-04-25T10:21:22.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb2c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb240af030 con 0x7fbb24159540 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb29200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbb240aebd0 0x7fbb24071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41518/0 (socket says 172.21.15.45:41518) 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb29200640 1 -- 172.21.15.45:0/3873565256 learned_addr learned my addr 172.21.15.45:0/3873565256 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb29c00640 1 --2- 172.21.15.45:0/3873565256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79485944 0 0) 0x7fbb24159920 con 0x7fbb240aebd0 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb0c003530 con 0x7fbb240aebd0 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492439342 0 0) 0x7fbb2415c2f0 con 0x7fbb240721f0 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb24159920 con 0x7fbb240721f0 2024-04-25T10:21:22.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb29c00640 1 -- 172.21.15.45:0/3873565256 >> v1:172.21.15.117:6789/0 conn(0x7fbb240721f0 legacy=0x7fbb24070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb29c00640 1 -- 172.21.15.45:0/3873565256 >> v1:172.21.15.45:6789/0 conn(0x7fbb240aebd0 legacy=0x7fbb24071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.351+0000 7fbb29c00640 1 -- 172.21.15.45:0/3873565256 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2415c5d0 con 0x7fbb24159540 2024-04-25T10:21:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb29c00640 1 --2- 172.21.15.45:0/3873565256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097f0 tx=0x7fbb1801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e1215369231ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180aa610 con 0x7fbb24159540 2024-04-25T10:21:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb180af070 con 0x7fbb24159540 2024-04-25T10:21:22.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb23e00640 1 -- 172.21.15.45:0/3873565256 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb180aa9b0 con 0x7fbb24159540 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 -- 172.21.15.45:0/3873565256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb24159540 msgr2=0x7fbb240ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 --2- 172.21.15.45:0/3873565256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097f0 tx=0x7fbb1801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 -- 172.21.15.45:0/3873565256 shutdown_connections 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 --2- 172.21.15.45:0/3873565256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb24159540 0x7fbb240ae690 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 -- 172.21.15.45:0/3873565256 >> 172.21.15.45:0/3873565256 conn(0x7fbb240669a0 msgr2=0x7fbb24066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 -- 172.21.15.45:0/3873565256 shutdown_connections 2024-04-25T10:21:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 -- 172.21.15.45:0/3873565256 wait complete. 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.355+0000 7fbb2c600640 1 Processor -- start 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- start start 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 0x7fbb240cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb240aebd0 0x7fbb240b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24159540 0x7fbb2414d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24169940 con 0x7fbb240721f0 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2406c0d0 con 0x7fbb24159540 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb24169ac0 con 0x7fbb240aebd0 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 0x7fbb240cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 0x7fbb240cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52616/0 (socket says 172.21.15.45:52616) 2024-04-25T10:21:22.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 -- 172.21.15.45:0/4277159986 learned_addr learned my addr 172.21.15.45:0/4277159986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2a600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24159540 0x7fbb2414d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29200640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb240aebd0 0x7fbb240b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 -- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb240aebd0 msgr2=0x7fbb240b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb240aebd0 0x7fbb240b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 -- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24159540 msgr2=0x7fbb2414d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24159540 0x7fbb2414d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 -- 172.21.15.45:0/4277159986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb240b2fe0 con 0x7fbb240721f0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29200640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb240aebd0 0x7fbb240b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb29c00640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 0x7fbb240cbe90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbb180ac040 tx=0x7fbb18019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18003820 con 0x7fbb240721f0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb180039c0 con 0x7fbb240721f0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18003d90 con 0x7fbb240721f0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb240b32c0 con 0x7fbb240721f0 2024-04-25T10:21:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2414ecf0 con 0x7fbb240721f0 2024-04-25T10:21:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.359+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2404ef30 con 0x7fbb240721f0 2024-04-25T10:21:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.363+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb180c2060 con 0x7fbb240721f0 2024-04-25T10:21:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.363+0000 7fbb21600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb04066f60 0x7fbb04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.363+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb1811ea10 con 0x7fbb240721f0 2024-04-25T10:21:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.363+0000 7fbb29200640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb04066f60 0x7fbb04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.363+0000 7fbb29200640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb04066f60 0x7fbb04069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b25b0 tx=0x7fbb10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.367+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb180a7070 con 0x7fbb240721f0 2024-04-25T10:21:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb2414d6a0 con 0x7fbb240721f0 2024-04-25T10:21:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb21600640 1 -- 172.21.15.45:0/4277159986 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb180160d0 con 0x7fbb240721f0 2024-04-25T10:21:22.533 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:21:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb04066f60 msgr2=0x7fbb04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb04066f60 0x7fbb04069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb240b25b0 tx=0x7fbb10009040 comp rx=0 tx=0).stop 2024-04-25T10:21:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 msgr2=0x7fbb240cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 0x7fbb240cbe90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbb180ac040 tx=0x7fbb18019250 comp rx=0 tx=0).stop 2024-04-25T10:21:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 shutdown_connections 2024-04-25T10:21:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb04066f60 0x7fbb04069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24159540 0x7fbb2414d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb240aebd0 0x7fbb240b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 --2- 172.21.15.45:0/4277159986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb240721f0 0x7fbb240cbe90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 >> 172.21.15.45:0/4277159986 conn(0x7fbb240669a0 msgr2=0x7fbb2415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 shutdown_connections 2024-04-25T10:21:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.531+0000 7fbb2c600640 1 -- 172.21.15.45:0/4277159986 wait complete. 2024-04-25T10:21:22.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 Processor -- start 2024-04-25T10:21:22.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 -- start start 2024-04-25T10:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f963015df60 0x7f96300cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f963006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96300ccaa0 con 0x7f96300d0180 2024-04-25T10:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963015e340 con 0x7f963015df60 2024-04-25T10:21:22.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9636000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96300ccd80 con 0x7f963006aea0 2024-04-25T10:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f963006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f963015df60 0x7f96300cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9627e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f96300d0180 0x7f96300d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:55836/0 (socket says 172.21.15.45:55836) 2024-04-25T10:21:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f9627e00640 1 -- 172.21.15.45:0/1993448204 learned_addr learned my addr 172.21.15.45:0/1993448204 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962fe00640 1 -- 172.21.15.45:0/1993448204 >> v2:172.21.15.117:3300/0 conn(0x7f963015df60 msgr2=0x7f96300cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962fe00640 1 --2- 172.21.15.45:0/1993448204 >> v2:172.21.15.117:3300/0 conn(0x7f963015df60 0x7f96300cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962fe00640 1 -- 172.21.15.45:0/1993448204 >> v1:172.21.15.117:6789/0 conn(0x7f96300d0180 legacy=0x7f96300d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962fe00640 1 -- 172.21.15.45:0/1993448204 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300cd060 con 0x7f963006aea0 2024-04-25T10:21:22.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.695+0000 7f962ea00640 1 -- 172.21.15.45:0/1993448204 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157097720 0 0) 0x7f96300ccaa0 con 0x7f96300d0180 2024-04-25T10:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f962fe00640 1 --2- 172.21.15.45:0/1993448204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f963006b2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9628009d90 tx=0x7f962801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6652509f330e5c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f962ea00640 1 -- 172.21.15.45:0/1993448204 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280b2070 con 0x7f963006aea0 2024-04-25T10:21:22.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f962ea00640 1 -- 172.21.15.45:0/1993448204 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96280a5890 con 0x7f963006aea0 2024-04-25T10:21:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f962ea00640 1 -- 172.21.15.45:0/1993448204 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280a5c60 con 0x7f963006aea0 2024-04-25T10:21:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 -- 172.21.15.45:0/1993448204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 msgr2=0x7f963006b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 --2- 172.21.15.45:0/1993448204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f963006b2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9628009d90 tx=0x7f962801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 -- 172.21.15.45:0/1993448204 shutdown_connections 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 --2- 172.21.15.45:0/1993448204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f963006b2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 --2- 172.21.15.45:0/1993448204 >> v2:172.21.15.117:3300/0 conn(0x7f963015df60 0x7f96300cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 -- 172.21.15.45:0/1993448204 >> 172.21.15.45:0/1993448204 conn(0x7f96300d1910 msgr2=0x7f96300d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 -- 172.21.15.45:0/1993448204 shutdown_connections 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.699+0000 7f9636000640 1 -- 172.21.15.45:0/1993448204 wait complete. 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 Processor -- start 2024-04-25T10:21:22.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- start start 2024-04-25T10:21:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f96300c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96300d0180 0x7f96300c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 0x7f96300b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300d0ff0 con 0x7f963015df60 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300d0e70 con 0x7f96300d0180 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963016aa20 con 0x7f963006aea0 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 0x7f96300b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 0x7f96300b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52628/0 (socket says 172.21.15.45:52628) 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 -- 172.21.15.45:0/4094128416 learned_addr learned my addr 172.21.15.45:0/4094128416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9627e00640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96300d0180 0x7f96300c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962f400640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f96300c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 -- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 msgr2=0x7f96300c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f96300c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 -- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96300d0180 msgr2=0x7f96300c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96300d0180 0x7f96300c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 -- 172.21.15.45:0/4094128416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300c9070 con 0x7f963015df60 2024-04-25T10:21:22.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f962fe00640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 0x7f96300b5ec0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f962801af90 tx=0x7f9628019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280b2050 con 0x7f963015df60 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96280be070 con 0x7f963015df60 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96300c9350 con 0x7f963015df60 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96280ad850 con 0x7f963015df60 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96300c9910 con 0x7f963015df60 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.703+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963004ef30 con 0x7f963015df60 2024-04-25T10:21:22.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.707+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96280ad9f0 con 0x7f963015df60 2024-04-25T10:21:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.707+0000 7f9627400640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f960c066f60 0x7f960c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.707+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f962811fff0 con 0x7f963015df60 2024-04-25T10:21:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.707+0000 7f962f400640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f960c066f60 0x7f960c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:22.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.707+0000 7f962f400640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f960c066f60 0x7f960c069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9620009b60 tx=0x7f9620009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.711+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9628016560 con 0x7f963015df60 2024-04-25T10:21:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.883+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96300ca020 con 0x7f963015df60 2024-04-25T10:21:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9627400640 1 -- 172.21.15.45:0/4094128416 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f96280f0950 con 0x7f963015df60 2024-04-25T10:21:22.888 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:21:22.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f960c066f60 msgr2=0x7f960c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f960c066f60 0x7f960c069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9620009b60 tx=0x7f9620009190 comp rx=0 tx=0).stop 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 msgr2=0x7f96300b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 0x7f96300b5ec0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f962801af90 tx=0x7f9628019680 comp rx=0 tx=0).stop 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 shutdown_connections 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f960c066f60 0x7f960c069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f963015df60 0x7f96300b5ec0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96300d0180 0x7f96300c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 --2- 172.21.15.45:0/4094128416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f963006aea0 0x7f96300c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 >> 172.21.15.45:0/4094128416 conn(0x7f96300d1910 msgr2=0x7f96301545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 shutdown_connections 2024-04-25T10:21:22.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:22.887+0000 7f9636000640 1 -- 172.21.15.45:0/4094128416 wait complete. 2024-04-25T10:21:22.910 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:21:22.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:22.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:23.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:21:23.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:23.823 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:21:23.832 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:21:23.833 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:23.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:23.841 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:23.841 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:23.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:21:23.884 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:21:23.884 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:23.894 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:23.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 Processor -- start 2024-04-25T10:21:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 -- start start 2024-04-25T10:21:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff0ec15e050 0x7ff0ec0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ec0b1850 con 0x7ff0ec0d1820 2024-04-25T10:21:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ec0b1af0 con 0x7ff0ec15e050 2024-04-25T10:21:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0f0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0b1dd0 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0e9e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff0ec0d1820 0x7ff0ec0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:55842/0 (socket says 172.21.15.45:55842) 2024-04-25T10:21:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0e9e00640 1 -- 172.21.15.45:0/1744235979 learned_addr learned my addr 172.21.15.45:0/1744235979 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0ea800640 1 --2- 172.21.15.45:0/1744235979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0e1400640 1 --2- 172.21.15.45:0/1744235979 >> v2:172.21.15.117:3300/0 conn(0x7ff0ec15e050 0x7ff0ec0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0e9400640 1 -- 172.21.15.45:0/1744235979 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763383128 0 0) 0x7ff0ec0b1850 con 0x7ff0ec0d1820 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0e9400640 1 -- 172.21.15.45:0/1744235979 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d0003530 con 0x7ff0ec0d1820 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0ea800640 1 -- 172.21.15.45:0/1744235979 >> v2:172.21.15.117:3300/0 conn(0x7ff0ec15e050 msgr2=0x7ff0ec0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0ea800640 1 --2- 172.21.15.45:0/1744235979 >> v2:172.21.15.117:3300/0 conn(0x7ff0ec15e050 0x7ff0ec0b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0ea800640 1 -- 172.21.15.45:0/1744235979 >> v1:172.21.15.117:6789/0 conn(0x7ff0ec0d1820 legacy=0x7ff0ec0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0ea800640 1 -- 172.21.15.45:0/1744235979 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec0b20b0 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.071+0000 7ff0e1400640 1 --2- 172.21.15.45:0/1744235979 >> v2:172.21.15.117:3300/0 conn(0x7ff0ec15e050 0x7ff0ec0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0ea800640 1 --2- 172.21.15.45:0/1744235979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0cfc70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009d90 tx=0x7ff0e401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b95e327ec914610b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0e9400640 1 -- 172.21.15.45:0/1744235979 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40b2070 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0e9400640 1 -- 172.21.15.45:0/1744235979 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e40a59a0 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0e9400640 1 -- 172.21.15.45:0/1744235979 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40a5d70 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 -- 172.21.15.45:0/1744235979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 msgr2=0x7ff0ec0cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/1744235979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0cfc70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009d90 tx=0x7ff0e401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 -- 172.21.15.45:0/1744235979 shutdown_connections 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/1744235979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0cfc70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/1744235979 >> v2:172.21.15.117:3300/0 conn(0x7ff0ec15e050 0x7ff0ec0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 -- 172.21.15.45:0/1744235979 >> 172.21.15.45:0/1744235979 conn(0x7ff0ec09a9d0 msgr2=0x7ff0ec09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 -- 172.21.15.45:0/1744235979 shutdown_connections 2024-04-25T10:21:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.075+0000 7ff0f0a00640 1 -- 172.21.15.45:0/1744235979 wait complete. 2024-04-25T10:21:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 Processor -- start 2024-04-25T10:21:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 -- start start 2024-04-25T10:21:24.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 0x7ff0ec0c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0ec15e050 0x7ff0ec153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0d0d90 con 0x7ff0ec0d1820 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0d0c10 con 0x7ff0ec15e050 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0d0f10 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 0x7ff0ec0c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37402/0 (socket says 172.21.15.45:37402) 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 -- 172.21.15.45:0/3499642964 learned_addr learned my addr 172.21.15.45:0/3499642964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 0x7ff0ec0c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52644/0 (socket says 172.21.15.45:52644) 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0ea800640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0ec15e050 0x7ff0ec153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 -- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0ec15e050 msgr2=0x7ff0ec153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0ec15e050 0x7ff0ec153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 -- 172.21.15.45:0/3499642964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 msgr2=0x7ff0ec0c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 0x7ff0ec0c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 -- 172.21.15.45:0/3499642964 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec06f6d0 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e1400640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 0x7ff0ec0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e9e00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0c89d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00c460 tx=0x7ff0dc00c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc017070 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc005d10 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0072b0 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec15ad70 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.079+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec15b330 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.083+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec15ba40 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.083+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0dc01a070 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.083+0000 7ff0e2a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0c8066f60 0x7ff0c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.083+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0dc084380 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.083+0000 7ff0e1400640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0c8066f60 0x7ff0c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.083+0000 7ff0e1400640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0c8066f60 0x7ff0c8069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0b03b0 tx=0x7ff0d4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.087+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0dc054ca0 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.283+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0ec0cfe10 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.283+0000 7ff0e2a00640 1 -- 172.21.15.45:0/3499642964 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4945 (secure 0 0 0) 0x7ff0dc058d40 con 0x7ff0ec0b13e0 2024-04-25T10:21:24.287 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:24.287 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:23.900522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:24.287 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 31 2024-04-25T10:21:24.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0c8066f60 msgr2=0x7ff0c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0c8066f60 0x7ff0c8069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0b03b0 tx=0x7ff0d4009190 comp rx=0 tx=0).stop 2024-04-25T10:21:24.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 msgr2=0x7ff0ec0c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0c89d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00c460 tx=0x7ff0dc00c920 comp rx=0 tx=0).stop 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 shutdown_connections 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0c8066f60 0x7ff0c8069420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0ec15e050 0x7ff0ec153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0ec0d1820 0x7ff0ec0c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 --2- 172.21.15.45:0/3499642964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0ec0b13e0 0x7ff0ec0c89d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 >> 172.21.15.45:0/3499642964 conn(0x7ff0ec09a9d0 msgr2=0x7ff0ec06fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 shutdown_connections 2024-04-25T10:21:24.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.287+0000 7ff0f0a00640 1 -- 172.21.15.45:0/3499642964 wait complete. 2024-04-25T10:21:24.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 Processor -- start 2024-04-25T10:21:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 -- start start 2024-04-25T10:21:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc9400cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9400d0270 0x7fc9400b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94015bff0 con 0x7fc94015bc10 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9400cd550 con 0x7fc9400d0270 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc946600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400b0da0 con 0x7fc9400cf760 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9400d0270 0x7fc9400b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9400d0270 0x7fc9400b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52648/0 (socket says 172.21.15.45:52648) 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 -- 172.21.15.45:0/3957807709 learned_addr learned my addr 172.21.15.45:0/3957807709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc937e00640 1 --2- 172.21.15.45:0/3957807709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc9400cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 -- 172.21.15.45:0/3957807709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 msgr2=0x7fc9400cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 --2- 172.21.15.45:0/3957807709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc9400cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 -- 172.21.15.45:0/3957807709 >> v1:172.21.15.117:6789/0 conn(0x7fc94015bc10 legacy=0x7fc9400ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.447+0000 7fc945c00640 1 -- 172.21.15.45:0/3957807709 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400b1080 con 0x7fc9400d0270 2024-04-25T10:21:24.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc945c00640 1 --2- 172.21.15.45:0/3957807709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc9400b1880 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc938009e50 tx=0x7fc93801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25696015bca17063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc93fe00640 1 -- 172.21.15.45:0/3957807709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380a5810 con 0x7fc9400d0270 2024-04-25T10:21:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc93fe00640 1 -- 172.21.15.45:0/3957807709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380a59b0 con 0x7fc9400d0270 2024-04-25T10:21:24.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc93fe00640 1 -- 172.21.15.45:0/3957807709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380a5d80 con 0x7fc9400d0270 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 -- 172.21.15.45:0/3957807709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 msgr2=0x7fc9400b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 --2- 172.21.15.45:0/3957807709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc9400b1880 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc938009e50 tx=0x7fc93801ca30 comp rx=0 tx=0).stop 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 -- 172.21.15.45:0/3957807709 shutdown_connections 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 --2- 172.21.15.45:0/3957807709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc9400b1880 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 --2- 172.21.15.45:0/3957807709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc9400cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 -- 172.21.15.45:0/3957807709 >> 172.21.15.45:0/3957807709 conn(0x7fc9400d1750 msgr2=0x7fc9400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 -- 172.21.15.45:0/3957807709 shutdown_connections 2024-04-25T10:21:24.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.451+0000 7fc946600640 1 -- 172.21.15.45:0/3957807709 wait complete. 2024-04-25T10:21:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 Processor -- start 2024-04-25T10:21:24.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 -- start start 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc94015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc94015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94015bc10 0x7fc9400b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400b2960 con 0x7fc9400d0270 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400b27e0 con 0x7fc94015bc10 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400b2ae0 con 0x7fc9400cf760 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc94015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc94015e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52654/0 (socket says 172.21.15.45:52654) 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 -- 172.21.15.45:0/432605437 learned_addr learned my addr 172.21.15.45:0/432605437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc945200640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc94015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc945c00640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94015bc10 0x7fc9400b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 -- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 msgr2=0x7fc94015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc94015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 -- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94015bc10 msgr2=0x7fc9400b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94015bc10 0x7fc9400b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 -- 172.21.15.45:0/432605437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400c8b70 con 0x7fc9400d0270 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc937e00640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc94015e890 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc92800e8a0 tx=0x7fc92800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928017070 con 0x7fc9400d0270 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800c940 con 0x7fc9400d0270 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800ccf0 con 0x7fc9400d0270 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9400c8e50 con 0x7fc9400d0270 2024-04-25T10:21:24.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.455+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9400c96f0 con 0x7fc9400d0270 2024-04-25T10:21:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.459+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9400c9020 con 0x7fc9400d0270 2024-04-25T10:21:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.459+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc92801a070 con 0x7fc9400d0270 2024-04-25T10:21:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.459+0000 7fc93d600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.459+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9280851f0 con 0x7fc9400d0270 2024-04-25T10:21:24.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.459+0000 7fc945200640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.459+0000 7fc945200640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc91c066f60 0x7fc91c069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc930001810 tx=0x7fc930009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.463+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc928055ba0 con 0x7fc9400d0270 2024-04-25T10:21:24.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.627+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc940075120 con 0x7fc9400d0270 2024-04-25T10:21:24.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc93d600640 1 -- 172.21.15.45:0/432605437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fc928059c40 con 0x7fc9400d0270 2024-04-25T10:21:24.633 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:24.633 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":50,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:48.857409+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc91c066f60 msgr2=0x7fc91c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc91c066f60 0x7fc91c069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc930001810 tx=0x7fc930009040 comp rx=0 tx=0).stop 2024-04-25T10:21:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 msgr2=0x7fc94015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc94015e890 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc92800e8a0 tx=0x7fc92800eca0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 shutdown_connections 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc94015bc10 0x7fc9400b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9400d0270 0x7fc94015e890 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 --2- 172.21.15.45:0/432605437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc9400cf760 0x7fc94015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 >> 172.21.15.45:0/432605437 conn(0x7fc9400d1750 msgr2=0x7fc9400cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 shutdown_connections 2024-04-25T10:21:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.631+0000 7fc946600640 1 -- 172.21.15.45:0/432605437 wait complete. 2024-04-25T10:21:24.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:20:45.743832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:46.252078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:24.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:24.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.795+0000 7fbf0f200640 1 Processor -- start 2024-04-25T10:21:24.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.795+0000 7fbf0f200640 1 -- start start 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.795+0000 7fbf0f200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf08066c00 0x7fbf08161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf08066fe0 0x7fbf08072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf0f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf080b40c0 con 0x7fbf080b5560 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf0f200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf08161d60 con 0x7fbf08066c00 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf0f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf080b5b00 con 0x7fbf08066fe0 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf08066c00 0x7fbf08161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf08066c00 0x7fbf08161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52660/0 (socket says 172.21.15.45:52660) 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 -- 172.21.15.45:0/2753958095 learned_addr learned my addr 172.21.15.45:0/2753958095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbeffe00640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf08066fe0 0x7fbf08072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 -- 172.21.15.45:0/2753958095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf08066fe0 msgr2=0x7fbf08072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:24.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf08066fe0 0x7fbf08072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 -- 172.21.15.45:0/2753958095 >> v1:172.21.15.117:6789/0 conn(0x7fbf080b5560 legacy=0x7fbf080ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 -- 172.21.15.45:0/2753958095 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf080b3a00 con 0x7fbf08066c00 2024-04-25T10:21:24.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbeffe00640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf08066fe0 0x7fbf08072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:24.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07e00640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08161820 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870fee8cb1af3032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07400640 1 -- 172.21.15.45:0/2753958095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80b0070 con 0x7fbf08066c00 2024-04-25T10:21:24.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07400640 1 -- 172.21.15.45:0/2753958095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef80ab600 con 0x7fbf08066c00 2024-04-25T10:21:24.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.799+0000 7fbf07400640 1 -- 172.21.15.45:0/2753958095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80ab9d0 con 0x7fbf08066c00 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- 172.21.15.45:0/2753958095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 msgr2=0x7fbf08161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08161820 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009870 tx=0x7fbef801c440 comp rx=0 tx=0).stop 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- 172.21.15.45:0/2753958095 shutdown_connections 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf08066fe0 0x7fbf08072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2753958095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08161820 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- 172.21.15.45:0/2753958095 >> 172.21.15.45:0/2753958095 conn(0x7fbf080b2560 msgr2=0x7fbf080b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- 172.21.15.45:0/2753958095 shutdown_connections 2024-04-25T10:21:24.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- 172.21.15.45:0/2753958095 wait complete. 2024-04-25T10:21:24.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 Processor -- start 2024-04-25T10:21:24.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- start start 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf08066fe0 0x7fbf08158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf080b5560 0x7fbf0813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0815e540 con 0x7fbf08066c00 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf081585e0 con 0x7fbf08066fe0 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf0f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0815e6c0 con 0x7fbf080b5560 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52674/0 (socket says 172.21.15.45:52674) 2024-04-25T10:21:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 -- 172.21.15.45:0/2470600475 learned_addr learned my addr 172.21.15.45:0/2470600475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 -- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf080b5560 msgr2=0x7fbf0813ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf080b5560 0x7fbf0813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 -- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf08066fe0 msgr2=0x7fbf08158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf08066fe0 0x7fbf08158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 -- 172.21.15.45:0/2470600475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08159cc0 con 0x7fbf08066c00 2024-04-25T10:21:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.803+0000 7fbf07e00640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08156560 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbef80ae040 tx=0x7fbef801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80ba020 con 0x7fbf08066c00 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef80b6640 con 0x7fbf08066c00 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80b0040 con 0x7fbf08066c00 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf08159140 con 0x7fbf08066c00 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf08159700 con 0x7fbf08066c00 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0804ef30 con 0x7fbf08066c00 2024-04-25T10:21:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbef80bd020 con 0x7fbf08066c00 2024-04-25T10:21:24.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf04c00640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbee4066f60 0x7fbee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:24.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbef811f280 con 0x7fbf08066c00 2024-04-25T10:21:24.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.807+0000 7fbeffe00640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbee4066f60 0x7fbee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:24.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.811+0000 7fbeffe00640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbee4066f60 0x7fbee4069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbef0006f10 tx=0x7fbef0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:24.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.811+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbef80c4030 con 0x7fbf08066c00 2024-04-25T10:21:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.975+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf081699a0 con 0x7fbf08066c00 2024-04-25T10:21:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.975+0000 7fbf04c00640 1 -- 172.21.15.45:0/2470600475 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf08159cc0 con 0x7fbf08066c00 2024-04-25T10:21:24.979 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:21:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbee4066f60 msgr2=0x7fbee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbee4066f60 0x7fbee4069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbef0006f10 tx=0x7fbef0008040 comp rx=0 tx=0).stop 2024-04-25T10:21:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 msgr2=0x7fbf08156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08156560 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbef80ae040 tx=0x7fbef801ac80 comp rx=0 tx=0).stop 2024-04-25T10:21:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 shutdown_connections 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbee4066f60 0x7fbee4069420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf080b5560 0x7fbf0813ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf08066fe0 0x7fbf08158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 --2- 172.21.15.45:0/2470600475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf08066c00 0x7fbf08156560 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 >> 172.21.15.45:0/2470600475 conn(0x7fbf080b2560 msgr2=0x7fbf080ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 shutdown_connections 2024-04-25T10:21:24.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:24.979+0000 7fbf0f200640 1 -- 172.21.15.45:0/2470600475 wait complete. 2024-04-25T10:21:25.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:21:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f8b200640 1 Processor -- start 2024-04-25T10:21:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f8b200640 1 -- start start 2024-04-25T10:21:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f8b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f840d0d00 0x7f9f840ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f8b200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f840cdab0 con 0x7f9f8406cf40 2024-04-25T10:21:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f8b200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f840cd210 con 0x7f9f840cc200 2024-04-25T10:21:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f8b200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f840cd4f0 con 0x7f9f840d0d00 2024-04-25T10:21:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f83e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f840d0d00 0x7f9f840ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f89200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f8406cf40 0x7f9f8415d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50486/0 (socket says 172.21.15.45:50486) 2024-04-25T10:21:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f89200640 1 -- 172.21.15.45:0/212346095 learned_addr learned my addr 172.21.15.45:0/212346095 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f83e00640 1 -- 172.21.15.45:0/212346095 >> v1:172.21.15.117:6789/0 conn(0x7f9f840cc200 legacy=0x7f9f84162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f83e00640 1 -- 172.21.15.45:0/212346095 >> v1:172.21.15.45:6789/0 conn(0x7f9f8406cf40 legacy=0x7f9f8415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f83e00640 1 -- 172.21.15.45:0/212346095 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840cd7d0 con 0x7f9f840d0d00 2024-04-25T10:21:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.143+0000 7f9f83400640 1 -- 172.21.15.45:0/212346095 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940529766 0 0) 0x7f9f840cdab0 con 0x7f9f8406cf40 2024-04-25T10:21:25.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f83e00640 1 --2- 172.21.15.45:0/212346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840d0d00 0x7f9f840ccc30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9f740097c0 tx=0x7f9f7401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3730c47fe8a708e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f83400640 1 -- 172.21.15.45:0/212346095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f740b0070 con 0x7f9f840d0d00 2024-04-25T10:21:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f83400640 1 -- 172.21.15.45:0/212346095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f740ab600 con 0x7f9f840d0d00 2024-04-25T10:21:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f83400640 1 -- 172.21.15.45:0/212346095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f740a4e60 con 0x7f9f840d0d00 2024-04-25T10:21:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 -- 172.21.15.45:0/212346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840d0d00 msgr2=0x7f9f840ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 --2- 172.21.15.45:0/212346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840d0d00 0x7f9f840ccc30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9f740097c0 tx=0x7f9f7401c3d0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 -- 172.21.15.45:0/212346095 shutdown_connections 2024-04-25T10:21:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 --2- 172.21.15.45:0/212346095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840d0d00 0x7f9f840ccc30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 -- 172.21.15.45:0/212346095 >> 172.21.15.45:0/212346095 conn(0x7f9f84072290 msgr2=0x7f9f840726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 -- 172.21.15.45:0/212346095 shutdown_connections 2024-04-25T10:21:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 -- 172.21.15.45:0/212346095 wait complete. 2024-04-25T10:21:25.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.147+0000 7f9f8b200640 1 Processor -- start 2024-04-25T10:21:25.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- start start 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f8406cf40 0x7f9f84155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 0x7f9f84155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f840d0d00 0x7f9f84155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f84162a10 con 0x7f9f840cc200 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f84162890 con 0x7f9f840d0d00 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8416c690 con 0x7f9f8406cf40 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 0x7f9f84155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 0x7f9f84155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43098/0 (socket says 172.21.15.45:43098) 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 -- 172.21.15.45:0/4189434848 learned_addr learned my addr 172.21.15.45:0/4189434848 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f83e00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f8406cf40 0x7f9f84155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f89200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f840d0d00 0x7f9f84155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 -- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f8406cf40 msgr2=0x7f9f84155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f8406cf40 0x7f9f84155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 -- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f840d0d00 msgr2=0x7f9f84155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f840d0d00 0x7f9f84155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 -- 172.21.15.45:0/4189434848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840c7430 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f7be00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 0x7f9f84155220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b410 tx=0x7f9f6c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c015030 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c004020 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004450 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8415b320 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8415b8a0 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.151+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8404ef30 con 0x7f9f840cc200 2024-04-25T10:21:25.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.155+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f6c018070 con 0x7f9f840cc200 2024-04-25T10:21:25.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.155+0000 7f9f80c00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f60066f60 0x7f9f60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.155+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f6c084ab0 con 0x7f9f840cc200 2024-04-25T10:21:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.155+0000 7f9f83e00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f60066f60 0x7f9f60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.155+0000 7f9f83e00640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f60066f60 0x7f9f60069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f740ad000 tx=0x7f9f740a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.159+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f840c7430 con 0x7f9f840cc200 2024-04-25T10:21:25.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.319+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f840b61a0 con 0x7f9f840cc200 2024-04-25T10:21:25.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f80c00640 1 -- 172.21.15.45:0/4189434848 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f6c0553d0 con 0x7f9f840cc200 2024-04-25T10:21:25.325 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f60066f60 msgr2=0x7f9f60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f60066f60 0x7f9f60069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f740ad000 tx=0x7f9f740a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 msgr2=0x7f9f84155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 0x7f9f84155220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00b410 tx=0x7f9f6c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 shutdown_connections 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f60066f60 0x7f9f60069420 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f840d0d00 0x7f9f84155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f840cc200 0x7f9f84155220 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 --2- 172.21.15.45:0/4189434848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f8406cf40 0x7f9f84155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 >> 172.21.15.45:0/4189434848 conn(0x7f9f84072290 msgr2=0x7f9f84160fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 shutdown_connections 2024-04-25T10:21:25.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.323+0000 7f9f8b200640 1 -- 172.21.15.45:0/4189434848 wait complete. 2024-04-25T10:21:25.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:21:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 Processor -- start 2024-04-25T10:21:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 -- start start 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc56c15e130 0x7fc56c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc56c071bc0 0x7fc56c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c0ad840 con 0x7fc56c0b3480 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c072570 con 0x7fc56c071bc0 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc572c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0d0090 con 0x7fc56c15e130 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc56be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc56c0b3480 0x7fc56c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50490/0 (socket says 172.21.15.45:50490) 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc56be00640 1 -- 172.21.15.45:0/68490193 learned_addr learned my addr 172.21.15.45:0/68490193 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc570c00640 1 --2- 172.21.15.45:0/68490193 >> v2:172.21.15.45:3300/0 conn(0x7fc56c071bc0 0x7fc56c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc563e00640 1 --2- 172.21.15.45:0/68490193 >> v2:172.21.15.117:3300/0 conn(0x7fc56c15e130 0x7fc56c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc56b400640 1 -- 172.21.15.45:0/68490193 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083300512 0 0) 0x7fc56c0ad840 con 0x7fc56c0b3480 2024-04-25T10:21:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc56b400640 1 -- 172.21.15.45:0/68490193 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc550003530 con 0x7fc56c0b3480 2024-04-25T10:21:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc570c00640 1 -- 172.21.15.45:0/68490193 >> v2:172.21.15.117:3300/0 conn(0x7fc56c15e130 msgr2=0x7fc56c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc570c00640 1 --2- 172.21.15.45:0/68490193 >> v2:172.21.15.117:3300/0 conn(0x7fc56c15e130 0x7fc56c0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc570c00640 1 -- 172.21.15.45:0/68490193 >> v1:172.21.15.45:6789/0 conn(0x7fc56c0b3480 legacy=0x7fc56c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.487+0000 7fc570c00640 1 -- 172.21.15.45:0/68490193 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c15e710 con 0x7fc56c071bc0 2024-04-25T10:21:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc570c00640 1 --2- 172.21.15.45:0/68490193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c071bc0 0x7fc56c072030 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0b4c40 tx=0x7fc56401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22230ae64cb57ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc56b400640 1 -- 172.21.15.45:0/68490193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5640b2070 con 0x7fc56c071bc0 2024-04-25T10:21:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc56b400640 1 -- 172.21.15.45:0/68490193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5640a57f0 con 0x7fc56c071bc0 2024-04-25T10:21:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc56b400640 1 -- 172.21.15.45:0/68490193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5640a5ba0 con 0x7fc56c071bc0 2024-04-25T10:21:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 -- 172.21.15.45:0/68490193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c071bc0 msgr2=0x7fc56c072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 --2- 172.21.15.45:0/68490193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c071bc0 0x7fc56c072030 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0b4c40 tx=0x7fc56401c930 comp rx=0 tx=0).stop 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 -- 172.21.15.45:0/68490193 shutdown_connections 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 --2- 172.21.15.45:0/68490193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c071bc0 0x7fc56c072030 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 --2- 172.21.15.45:0/68490193 >> v2:172.21.15.117:3300/0 conn(0x7fc56c15e130 0x7fc56c0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 -- 172.21.15.45:0/68490193 >> 172.21.15.45:0/68490193 conn(0x7fc56c0669a0 msgr2=0x7fc56c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 -- 172.21.15.45:0/68490193 shutdown_connections 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.491+0000 7fc572c00640 1 -- 172.21.15.45:0/68490193 wait complete. 2024-04-25T10:21:25.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 Processor -- start 2024-04-25T10:21:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- start start 2024-04-25T10:21:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc56c071bc0 0x7fc56c157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 0x7fc56c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc56c15e130 0x7fc56c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0ae530 con 0x7fc56c0b3480 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c15bb90 con 0x7fc56c071bc0 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0ae6b0 con 0x7fc56c15e130 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 0x7fc56c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 0x7fc56c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43122/0 (socket says 172.21.15.45:43122) 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 -- 172.21.15.45:0/3277265823 learned_addr learned my addr 172.21.15.45:0/3277265823 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc56be00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc56c071bc0 0x7fc56c157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc570c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc56c15e130 0x7fc56c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 -- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc56c15e130 msgr2=0x7fc56c15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc56c15e130 0x7fc56c15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 -- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc56c071bc0 msgr2=0x7fc56c157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc56c071bc0 0x7fc56c157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 -- 172.21.15.45:0/3277265823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c1553a0 con 0x7fc56c0b3480 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc570c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc56c15e130 0x7fc56c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc56be00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc56c071bc0 0x7fc56c157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc563e00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 0x7fc56c15a2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc55400e870 tx=0x7fc55400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554017070 con 0x7fc56c0b3480 2024-04-25T10:21:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55400c940 con 0x7fc56c0b3480 2024-04-25T10:21:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400ccf0 con 0x7fc56c0b3480 2024-04-25T10:21:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c155680 con 0x7fc56c0b3480 2024-04-25T10:21:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c15ad60 con 0x7fc56c0b3480 2024-04-25T10:21:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.495+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c15b070 con 0x7fc56c0b3480 2024-04-25T10:21:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.499+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc55401a070 con 0x7fc56c0b3480 2024-04-25T10:21:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.499+0000 7fc568c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc548066f60 0x7fc548069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.499+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5540852d0 con 0x7fc56c0b3480 2024-04-25T10:21:25.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.499+0000 7fc56be00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc548066f60 0x7fc548069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.499+0000 7fc56be00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc548066f60 0x7fc548069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000f70 tx=0x7fc55c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.503+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5540562d0 con 0x7fc56c0b3480 2024-04-25T10:21:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.663+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc56c070760 con 0x7fc56c0b3480 2024-04-25T10:21:25.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.663+0000 7fc568c00640 1 -- 172.21.15.45:0/3277265823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc554055bd0 con 0x7fc56c0b3480 2024-04-25T10:21:25.667 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc548066f60 msgr2=0x7fc548069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc548066f60 0x7fc548069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000f70 tx=0x7fc55c009040 comp rx=0 tx=0).stop 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 msgr2=0x7fc56c15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 0x7fc56c15a2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc55400e870 tx=0x7fc55400ec70 comp rx=0 tx=0).stop 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 shutdown_connections 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc548066f60 0x7fc548069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc56c15e130 0x7fc56c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56c0b3480 0x7fc56c15a2e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 --2- 172.21.15.45:0/3277265823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc56c071bc0 0x7fc56c157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 >> 172.21.15.45:0/3277265823 conn(0x7fc56c0669a0 msgr2=0x7fc56c06d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 shutdown_connections 2024-04-25T10:21:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.667+0000 7fc572c00640 1 -- 172.21.15.45:0/3277265823 wait complete. 2024-04-25T10:21:25.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:25.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 Processor -- start 2024-04-25T10:21:25.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 -- start start 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a1415ac30 0x7f7a140cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1406e2a0 0x7f7a140d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1406ecf0 con 0x7f7a1415a850 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1415e8e0 con 0x7f7a1415ac30 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a1ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1406e730 con 0x7f7a1406e2a0 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a19000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7a1415a850 0x7f7a1406db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50494/0 (socket says 172.21.15.45:50494) 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a19000640 1 -- 172.21.15.45:0/625632424 learned_addr learned my addr 172.21.15.45:0/625632424 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13e00640 1 --2- 172.21.15.45:0/625632424 >> v2:172.21.15.45:3300/0 conn(0x7f7a1415ac30 0x7f7a140cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13400640 1 -- 172.21.15.45:0/625632424 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331989296 0 0) 0x7f7a1406ecf0 con 0x7f7a1415a850 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13400640 1 -- 172.21.15.45:0/625632424 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79fc003530 con 0x7f7a1415a850 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a19a00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1406e2a0 0x7f7a140d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13e00640 1 -- 172.21.15.45:0/625632424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1406e2a0 msgr2=0x7f7a140d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13e00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1406e2a0 0x7f7a140d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13e00640 1 -- 172.21.15.45:0/625632424 >> v1:172.21.15.45:6789/0 conn(0x7f7a1415a850 legacy=0x7f7a1406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a13e00640 1 -- 172.21.15.45:0/625632424 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1406ea10 con 0x7f7a1415ac30 2024-04-25T10:21:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.827+0000 7f7a19a00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1406e2a0 0x7f7a140d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a13e00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1415ac30 0x7f7a140cd9e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7a000097c0 tx=0x7f7a0001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a6c2d62de920e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a13400640 1 -- 172.21.15.45:0/625632424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a000b0070 con 0x7f7a1415ac30 2024-04-25T10:21:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a13400640 1 -- 172.21.15.45:0/625632424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a000ab600 con 0x7f7a1415ac30 2024-04-25T10:21:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a13400640 1 -- 172.21.15.45:0/625632424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a000ab9d0 con 0x7f7a1415ac30 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 -- 172.21.15.45:0/625632424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1415ac30 msgr2=0x7f7a140cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1415ac30 0x7f7a140cd9e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7a000097c0 tx=0x7f7a0001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 -- 172.21.15.45:0/625632424 shutdown_connections 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1406e2a0 0x7f7a140d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/625632424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1415ac30 0x7f7a140cd9e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 -- 172.21.15.45:0/625632424 >> 172.21.15.45:0/625632424 conn(0x7f7a1406ac90 msgr2=0x7f7a1406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 -- 172.21.15.45:0/625632424 shutdown_connections 2024-04-25T10:21:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 -- 172.21.15.45:0/625632424 wait complete. 2024-04-25T10:21:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.831+0000 7f7a1ba00640 1 Processor -- start 2024-04-25T10:21:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- start start 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1415a850 0x7f7a1414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1415ac30 0x7f7a14142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 0x7f7a14156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140b4ca0 con 0x7f7a140b0250 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140b4b20 con 0x7f7a1415a850 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140b4e20 con 0x7f7a1415ac30 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 0x7f7a14156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 0x7f7a14156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43148/0 (socket says 172.21.15.45:43148) 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 -- 172.21.15.45:0/3246419195 learned_addr learned my addr 172.21.15.45:0/3246419195 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19000640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1415a850 0x7f7a1414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a13e00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1415ac30 0x7f7a14142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 -- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1415ac30 msgr2=0x7f7a14142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1415ac30 0x7f7a14142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 -- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1415a850 msgr2=0x7f7a1414f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1415a850 0x7f7a1414f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 -- 172.21.15.45:0/3246419195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a14142980 con 0x7f7a140b0250 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19000640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1415a850 0x7f7a1414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a19a00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 0x7f7a14156d40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00cea0 tx=0x7f7a0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c003ba0 con 0x7f7a140b0250 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c003d60 con 0x7f7a140b0250 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00d880 con 0x7f7a140b0250 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a140b0ae0 con 0x7f7a140b0250 2024-04-25T10:21:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a140b10a0 con 0x7f7a140b0250 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.835+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1404ef30 con 0x7f7a140b0250 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.839+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a0c00da20 con 0x7f7a140b0250 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.839+0000 7f7a10c00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79f4066f60 0x7f79f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.839+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a0c085ae0 con 0x7f7a140b0250 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.839+0000 7f7a19000640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79f4066f60 0x7f79f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.839+0000 7f7a19000640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79f4066f60 0x7f79f4069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7a08005890 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:25.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:25.843+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a0c056a50 con 0x7f7a140b0250 2024-04-25T10:21:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.039+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a1406d740 con 0x7f7a140b0250 2024-04-25T10:21:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.039+0000 7f7a10c00640 1 -- 172.21.15.45:0/3246419195 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f7a0c056350 con 0x7f7a140b0250 2024-04-25T10:21:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:26.044 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:23.900522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 31 2024-04-25T10:21:26.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79f4066f60 msgr2=0x7f79f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79f4066f60 0x7f79f4069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7a08005890 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 msgr2=0x7f7a14156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 0x7f7a14156d40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00cea0 tx=0x7f7a0c00cf90 comp rx=0 tx=0).stop 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 shutdown_connections 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79f4066f60 0x7f79f4069420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a140b0250 0x7f7a14156d40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a1415ac30 0x7f7a14142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 --2- 172.21.15.45:0/3246419195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1415a850 0x7f7a1414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 >> 172.21.15.45:0/3246419195 conn(0x7f7a1406ac90 msgr2=0x7f7a141517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 shutdown_connections 2024-04-25T10:21:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.043+0000 7f7a1ba00640 1 -- 172.21.15.45:0/3246419195 wait complete. 2024-04-25T10:21:26.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 Processor -- start 2024-04-25T10:21:26.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 -- start start 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c1550b0 0x7f972c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f972c15b910 0x7f972c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972c071050 con 0x7f972c0cfe70 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972c15bda0 con 0x7f972c15b910 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9733400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c15d040 con 0x7f972c1550b0 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9723e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f972c0cfe70 0x7f972c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50502/0 (socket says 172.21.15.45:50502) 2024-04-25T10:21:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9723e00640 1 -- 172.21.15.45:0/3203335200 learned_addr learned my addr 172.21.15.45:0/3203335200 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9730a00640 1 --2- 172.21.15.45:0/3203335200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c1550b0 0x7f972c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.203+0000 7f9731400640 1 --2- 172.21.15.45:0/3203335200 >> v2:172.21.15.117:3300/0 conn(0x7f972c15b910 0x7f972c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f972be00640 1 -- 172.21.15.45:0/3203335200 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174179584 0 0) 0x7f972c071050 con 0x7f972c0cfe70 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f972be00640 1 -- 172.21.15.45:0/3203335200 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9710003530 con 0x7f972c0cfe70 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f9730a00640 1 -- 172.21.15.45:0/3203335200 >> v2:172.21.15.117:3300/0 conn(0x7f972c15b910 msgr2=0x7f972c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f9730a00640 1 --2- 172.21.15.45:0/3203335200 >> v2:172.21.15.117:3300/0 conn(0x7f972c15b910 0x7f972c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f9730a00640 1 -- 172.21.15.45:0/3203335200 >> v1:172.21.15.45:6789/0 conn(0x7f972c0cfe70 legacy=0x7f972c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f9730a00640 1 -- 172.21.15.45:0/3203335200 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c0712f0 con 0x7f972c1550b0 2024-04-25T10:21:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f9730a00640 1 --2- 172.21.15.45:0/3203335200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c1550b0 0x7f972c0cf930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f971c0097c0 tx=0x7f971c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75dcafbd7fba866b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f972be00640 1 -- 172.21.15.45:0/3203335200 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0b2070 con 0x7f972c1550b0 2024-04-25T10:21:26.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f972be00640 1 -- 172.21.15.45:0/3203335200 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c0ad640 con 0x7f972c1550b0 2024-04-25T10:21:26.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.207+0000 7f972be00640 1 -- 172.21.15.45:0/3203335200 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0ada10 con 0x7f972c1550b0 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- 172.21.15.45:0/3203335200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c1550b0 msgr2=0x7f972c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 --2- 172.21.15.45:0/3203335200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c1550b0 0x7f972c0cf930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f971c0097c0 tx=0x7f971c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- 172.21.15.45:0/3203335200 shutdown_connections 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 --2- 172.21.15.45:0/3203335200 >> v2:172.21.15.117:3300/0 conn(0x7f972c15b910 0x7f972c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 --2- 172.21.15.45:0/3203335200 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c1550b0 0x7f972c0cf930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- 172.21.15.45:0/3203335200 >> 172.21.15.45:0/3203335200 conn(0x7f972c066b70 msgr2=0x7f972c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- 172.21.15.45:0/3203335200 shutdown_connections 2024-04-25T10:21:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- 172.21.15.45:0/3203335200 wait complete. 2024-04-25T10:21:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 Processor -- start 2024-04-25T10:21:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- start start 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c0cfe70 0x7f972c140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 0x7f972c13d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c15b910 0x7f972c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c15c030 con 0x7f972c1550b0 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c0720b0 con 0x7f972c0cfe70 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9733400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c15c1b0 con 0x7f972c15b910 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 0x7f972c13d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 0x7f972c13d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43158/0 (socket says 172.21.15.45:43158) 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 -- 172.21.15.45:0/823518231 learned_addr learned my addr 172.21.15.45:0/823518231 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9731400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c15b910 0x7f972c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9730a00640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c0cfe70 0x7f972c140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 -- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c15b910 msgr2=0x7f972c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c15b910 0x7f972c157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 -- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c0cfe70 msgr2=0x7f972c140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c0cfe70 0x7f972c140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9723e00640 1 -- 172.21.15.45:0/823518231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c140b60 con 0x7f972c1550b0 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.211+0000 7f9731400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c15b910 0x7f972c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9723e00640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 0x7f972c13d310 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f971400c450 tx=0x7f971400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714017070 con 0x7f972c1550b0 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9714005cf0 con 0x7f972c1550b0 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714007290 con 0x7f972c1550b0 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c155490 con 0x7f972c1550b0 2024-04-25T10:21:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c155a50 con 0x7f972c1550b0 2024-04-25T10:21:26.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c04ef30 con 0x7f972c1550b0 2024-04-25T10:21:26.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f971401a070 con 0x7f972c1550b0 2024-04-25T10:21:26.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9729600640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9708066f60 0x7f9708069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9714084460 con 0x7f972c1550b0 2024-04-25T10:21:26.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.215+0000 7f9730a00640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9708066f60 0x7f9708069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.219+0000 7f9730a00640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9708066f60 0x7f9708069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f971c0af000 tx=0x7f971c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.219+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9714055450 con 0x7f972c1550b0 2024-04-25T10:21:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.383+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f972c15d410 con 0x7f972c1550b0 2024-04-25T10:21:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.383+0000 7f9729600640 1 -- 172.21.15.45:0/823518231 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f9714054d50 con 0x7f972c1550b0 2024-04-25T10:21:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:26.388 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":50,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:48.857409+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9708066f60 msgr2=0x7f9708069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9708066f60 0x7f9708069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f971c0af000 tx=0x7f971c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 msgr2=0x7f972c13d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 0x7f972c13d310 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f971400c450 tx=0x7f971400c910 comp rx=0 tx=0).stop 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 shutdown_connections 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9708066f60 0x7f9708069420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f972c15b910 0x7f972c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f972c1550b0 0x7f972c13d310 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 --2- 172.21.15.45:0/823518231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f972c0cfe70 0x7f972c140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 >> 172.21.15.45:0/823518231 conn(0x7f972c066b70 msgr2=0x7f972c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 shutdown_connections 2024-04-25T10:21:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.387+0000 7f9733400640 1 -- 172.21.15.45:0/823518231 wait complete. 2024-04-25T10:21:26.409 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:21:26.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 Processor -- start 2024-04-25T10:21:26.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 -- start start 2024-04-25T10:21:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100cf930 0x7f04100cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f04100ae020 0x7f04100ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04100d0250 con 0x7f0410070c20 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041015bb90 con 0x7f04100ae020 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0417000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04100ad840 con 0x7f04100cf930 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f040f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0410070c20 0x7f041015e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50510/0 (socket says 172.21.15.45:50510) 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f040f400640 1 -- 172.21.15.45:0/2310683000 learned_addr learned my addr 172.21.15.45:0/2310683000 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f0415000640 1 --2- 172.21.15.45:0/2310683000 >> v2:172.21.15.117:3300/0 conn(0x7f04100ae020 0x7f04100ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f040fe00640 1 --2- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100cf930 0x7f04100cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f040ea00640 1 -- 172.21.15.45:0/2310683000 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002863923 0 0) 0x7f04100d0250 con 0x7f0410070c20 2024-04-25T10:21:26.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.547+0000 7f040ea00640 1 -- 172.21.15.45:0/2310683000 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f4003530 con 0x7f0410070c20 2024-04-25T10:21:26.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0415000640 1 -- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100cf930 msgr2=0x7f04100cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:26.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0415000640 1 --2- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100cf930 0x7f04100cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0415000640 1 -- 172.21.15.45:0/2310683000 >> v1:172.21.15.45:6789/0 conn(0x7f0410070c20 legacy=0x7f041015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0415000640 1 -- 172.21.15.45:0/2310683000 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041015e9f0 con 0x7f04100ae020 2024-04-25T10:21:26.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0415000640 1 --2- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04100ae020 0x7f04100ae4b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0408009d90 tx=0x7f040801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bee593eca96261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f040ea00640 1 -- 172.21.15.45:0/2310683000 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04080b2070 con 0x7f04100ae020 2024-04-25T10:21:26.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f040ea00640 1 -- 172.21.15.45:0/2310683000 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04080a5a00 con 0x7f04100ae020 2024-04-25T10:21:26.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f040ea00640 1 -- 172.21.15.45:0/2310683000 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04080a5dd0 con 0x7f04100ae020 2024-04-25T10:21:26.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0417000640 1 -- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04100ae020 msgr2=0x7f04100ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.551+0000 7f0417000640 1 --2- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04100ae020 0x7f04100ae4b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0408009d90 tx=0x7f040801c940 comp rx=0 tx=0).stop 2024-04-25T10:21:26.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- 172.21.15.45:0/2310683000 shutdown_connections 2024-04-25T10:21:26.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 --2- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04100ae020 0x7f04100ae4b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 --2- 172.21.15.45:0/2310683000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100cf930 0x7f04100cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- 172.21.15.45:0/2310683000 >> 172.21.15.45:0/2310683000 conn(0x7f0410066b70 msgr2=0x7f0410066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- 172.21.15.45:0/2310683000 shutdown_connections 2024-04-25T10:21:26.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- 172.21.15.45:0/2310683000 wait complete. 2024-04-25T10:21:26.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 Processor -- start 2024-04-25T10:21:26.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- start start 2024-04-25T10:21:26.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410070c20 0x7f04100ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100ae020 0x7f0410157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 0x7f0410155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041016a8e0 con 0x7f04100cf930 2024-04-25T10:21:26.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041006e130 con 0x7f0410070c20 2024-04-25T10:21:26.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0417000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041016aa60 con 0x7f04100ae020 2024-04-25T10:21:26.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 0x7f0410155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 0x7f0410155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43170/0 (socket says 172.21.15.45:43170) 2024-04-25T10:21:26.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 -- 172.21.15.45:0/321119129 learned_addr learned my addr 172.21.15.45:0/321119129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:26.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f040f400640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100ae020 0x7f0410157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 -- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100ae020 msgr2=0x7f0410157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f040fe00640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410070c20 0x7f04100ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100ae020 0x7f0410157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 -- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410070c20 msgr2=0x7f04100ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410070c20 0x7f04100ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f0415000640 1 -- 172.21.15.45:0/321119129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0410158860 con 0x7f04100cf930 2024-04-25T10:21:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f040f400640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100ae020 0x7f0410157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.555+0000 7f040fe00640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410070c20 0x7f04100ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0415000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 0x7f0410155500 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f04080b0040 tx=0x7f04080196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04080b2050 con 0x7f04100cf930 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04080be070 con 0x7f04100cf930 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04080b9690 con 0x7f04100cf930 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0410158b40 con 0x7f04100cf930 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04100cca60 con 0x7f04100cf930 2024-04-25T10:21:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041004ef30 con 0x7f04100cf930 2024-04-25T10:21:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04080b9830 con 0x7f04100cf930 2024-04-25T10:21:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0407e00640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03f0066f60 0x7f03f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f040811ffe0 con 0x7f04100cf930 2024-04-25T10:21:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.559+0000 7f040fe00640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03f0066f60 0x7f03f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.563+0000 7f040fe00640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03f0066f60 0x7f03f0069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0400009a40 tx=0x7f0400008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.563+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0408016560 con 0x7f04100cf930 2024-04-25T10:21:26.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04100b3f10 con 0x7f04100cf930 2024-04-25T10:21:26.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0407e00640 1 -- 172.21.15.45:0/321119129 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f04080f0ee0 con 0x7f04100cf930 2024-04-25T10:21:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:26.762 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:23.900522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:26.763 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 31 2024-04-25T10:21:26.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03f0066f60 msgr2=0x7f03f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03f0066f60 0x7f03f0069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0400009a40 tx=0x7f0400008040 comp rx=0 tx=0).stop 2024-04-25T10:21:26.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 msgr2=0x7f0410155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 0x7f0410155500 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f04080b0040 tx=0x7f04080196e0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 shutdown_connections 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03f0066f60 0x7f03f0069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04100cf930 0x7f0410155500 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f04100ae020 0x7f0410157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 --2- 172.21.15.45:0/321119129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0410070c20 0x7f04100ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.759+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 >> 172.21.15.45:0/321119129 conn(0x7f0410066b70 msgr2=0x7f041015bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.763+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 shutdown_connections 2024-04-25T10:21:26.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.763+0000 7f0417000640 1 -- 172.21.15.45:0/321119129 wait complete. 2024-04-25T10:21:26.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.927+0000 7fd6b4c00640 1 Processor -- start 2024-04-25T10:21:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.927+0000 7fd6b4c00640 1 -- start start 2024-04-25T10:21:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6b4c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6b015db40 0x7fd6b006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6b4c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b006b2c0 0x7fd6b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6b4c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b00b5780 con 0x7fd6b00b5010 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6b4c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b00b5a60 con 0x7fd6b015db40 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6b4c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b5d40 con 0x7fd6b006b2c0 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6b015db40 0x7fd6b006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6b015db40 0x7fd6b006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43172/0 (socket says 172.21.15.45:43172) 2024-04-25T10:21:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 -- 172.21.15.45:0/3296509443 learned_addr learned my addr 172.21.15.45:0/3296509443 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6afe00640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b006b2c0 0x7fd6b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 -- 172.21.15.45:0/3296509443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b006b2c0 msgr2=0x7fd6b006b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b006b2c0 0x7fd6b006b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 -- 172.21.15.45:0/3296509443 >> v1:172.21.15.117:6789/0 conn(0x7fd6b00b5010 legacy=0x7fd6b015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 -- 172.21.15.45:0/3296509443 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00b6020 con 0x7fd6b015db40 2024-04-25T10:21:26.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6afe00640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b006b2c0 0x7fd6b006b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.931+0000 7fd6af400640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015db40 0x7fd6b006ac90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009e50 tx=0x7fd6a001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5932990ab0b1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6aea00640 1 -- 172.21.15.45:0/3296509443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00b4070 con 0x7fd6b015db40 2024-04-25T10:21:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6aea00640 1 -- 172.21.15.45:0/3296509443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a00af400 con 0x7fd6b015db40 2024-04-25T10:21:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6aea00640 1 -- 172.21.15.45:0/3296509443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00af7d0 con 0x7fd6b015db40 2024-04-25T10:21:26.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- 172.21.15.45:0/3296509443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015db40 msgr2=0x7fd6b006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015db40 0x7fd6b006ac90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009e50 tx=0x7fd6a001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- 172.21.15.45:0/3296509443 shutdown_connections 2024-04-25T10:21:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b006b2c0 0x7fd6b006b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/3296509443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015db40 0x7fd6b006ac90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- 172.21.15.45:0/3296509443 >> 172.21.15.45:0/3296509443 conn(0x7fd6b00d17b0 msgr2=0x7fd6b00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- 172.21.15.45:0/3296509443 shutdown_connections 2024-04-25T10:21:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- 172.21.15.45:0/3296509443 wait complete. 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 Processor -- start 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- start start 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 0x7fd6b00ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b00b5010 0x7fd6b0152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b015db40 0x7fd6b009bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b6820 con 0x7fd6b006b2c0 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b5480 con 0x7fd6b015db40 2024-04-25T10:21:26.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6b4c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b69a0 con 0x7fd6b00b5010 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 0x7fd6b00ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 0x7fd6b00ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43182/0 (socket says 172.21.15.45:43182) 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.935+0000 7fd6af400640 1 -- 172.21.15.45:0/1692433632 learned_addr learned my addr 172.21.15.45:0/1692433632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a6a00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b00b5010 0x7fd6b0152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6afe00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b015db40 0x7fd6b009bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6af400640 1 -- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b00b5010 msgr2=0x7fd6b0152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6af400640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b00b5010 0x7fd6b0152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6af400640 1 -- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b015db40 msgr2=0x7fd6b009bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6af400640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b015db40 0x7fd6b009bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:26.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6af400640 1 -- 172.21.15.45:0/1692433632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b006c0c0 con 0x7fd6b006b2c0 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6afe00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b015db40 0x7fd6b009bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6af400640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 0x7fd6b00ce6e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd6a001d890 tx=0x7fd6a001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00b4040 con 0x7fd6b006b2c0 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a00afe30 con 0x7fd6b006b2c0 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0009510 con 0x7fd6b006b2c0 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b01f3a70 con 0x7fd6b006b2c0 2024-04-25T10:21:26.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b00cf0c0 con 0x7fd6b006b2c0 2024-04-25T10:21:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b0075120 con 0x7fd6b006b2c0 2024-04-25T10:21:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6a00c2020 con 0x7fd6b006b2c0 2024-04-25T10:21:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a7e00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.939+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6a0122db0 con 0x7fd6b006b2c0 2024-04-25T10:21:26.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.943+0000 7fd6a6a00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:26.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.943+0000 7fd6a6a00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0070260 tx=0x7fd698009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:26.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:26.943+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6a0127070 con 0x7fd6b006b2c0 2024-04-25T10:21:27.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6b009caf0 con 0x7fd6b006b2c0 2024-04-25T10:21:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6a7e00640 1 -- 172.21.15.45:0/1692433632 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fd6b01f3a70 con 0x7fd6b006b2c0 2024-04-25T10:21:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:27.146 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:47.262741+0000","modified":"2024-04-25T10:21:23.900522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5574":{"gid":5574,"name":"d","rank":0,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.117:6836/3308184634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3308184634},{"type":"v1","addr":"172.21.15.117:6836","nonce":3308184634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:21:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 31 2024-04-25T10:21:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 msgr2=0x7fd68c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0070260 tx=0x7fd698009040 comp rx=0 tx=0).stop 2024-04-25T10:21:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 msgr2=0x7fd6b00ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 0x7fd6b00ce6e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd6a001d890 tx=0x7fd6a001bf80 comp rx=0 tx=0).stop 2024-04-25T10:21:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 shutdown_connections 2024-04-25T10:21:27.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b015db40 0x7fd6b009bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b00b5010 0x7fd6b0152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 --2- 172.21.15.45:0/1692433632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b006b2c0 0x7fd6b00ce6e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 >> 172.21.15.45:0/1692433632 conn(0x7fd6b00d17b0 msgr2=0x7fd6b015cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 shutdown_connections 2024-04-25T10:21:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.143+0000 7fd6b4c00640 1 -- 172.21.15.45:0/1692433632 wait complete. 2024-04-25T10:21:27.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:27.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 Processor -- start 2024-04-25T10:21:27.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 -- start start 2024-04-25T10:21:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2280d0350 0x7fe2280d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe22806aea0 0x7fe22806b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2280ccaa0 con 0x7fe22815df60 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22815e340 con 0x7fe22806aea0 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280ccd80 con 0x7fe2280d0350 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe227e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe22815df60 0x7fe2280cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50516/0 (socket says 172.21.15.45:50516) 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe227e00640 1 -- 172.21.15.45:0/2278041872 learned_addr learned my addr 172.21.15.45:0/2278041872 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ce00640 1 --2- 172.21.15.45:0/2278041872 >> v2:172.21.15.45:3300/0 conn(0x7fe22806aea0 0x7fe22806b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe227400640 1 --2- 172.21.15.45:0/2278041872 >> v2:172.21.15.117:3300/0 conn(0x7fe2280d0350 0x7fe2280d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe226a00640 1 -- 172.21.15.45:0/2278041872 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808271273 0 0) 0x7fe2280ccaa0 con 0x7fe22815df60 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe226a00640 1 -- 172.21.15.45:0/2278041872 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe20c003530 con 0x7fe22815df60 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ce00640 1 -- 172.21.15.45:0/2278041872 >> v2:172.21.15.117:3300/0 conn(0x7fe2280d0350 msgr2=0x7fe2280d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ce00640 1 --2- 172.21.15.45:0/2278041872 >> v2:172.21.15.117:3300/0 conn(0x7fe2280d0350 0x7fe2280d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ce00640 1 -- 172.21.15.45:0/2278041872 >> v1:172.21.15.45:6789/0 conn(0x7fe22815df60 legacy=0x7fe2280cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.307+0000 7fe22ce00640 1 -- 172.21.15.45:0/2278041872 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280cd060 con 0x7fe22806aea0 2024-04-25T10:21:27.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ce00640 1 --2- 172.21.15.45:0/2278041872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22806aea0 0x7fe22806b2b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe228075210 tx=0x7fe22001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219dabf666933ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe226a00640 1 -- 172.21.15.45:0/2278041872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200b2070 con 0x7fe22806aea0 2024-04-25T10:21:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe226a00640 1 -- 172.21.15.45:0/2278041872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200a5850 con 0x7fe22806aea0 2024-04-25T10:21:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe226a00640 1 -- 172.21.15.45:0/2278041872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200a5c20 con 0x7fe22806aea0 2024-04-25T10:21:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 -- 172.21.15.45:0/2278041872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22806aea0 msgr2=0x7fe22806b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 --2- 172.21.15.45:0/2278041872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22806aea0 0x7fe22806b2b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe228075210 tx=0x7fe22001c8a0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 -- 172.21.15.45:0/2278041872 shutdown_connections 2024-04-25T10:21:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 --2- 172.21.15.45:0/2278041872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22806aea0 0x7fe22806b2b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 --2- 172.21.15.45:0/2278041872 >> v2:172.21.15.117:3300/0 conn(0x7fe2280d0350 0x7fe2280d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 -- 172.21.15.45:0/2278041872 >> 172.21.15.45:0/2278041872 conn(0x7fe2280d1910 msgr2=0x7fe2280d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 -- 172.21.15.45:0/2278041872 shutdown_connections 2024-04-25T10:21:27.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 -- 172.21.15.45:0/2278041872 wait complete. 2024-04-25T10:21:27.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.311+0000 7fe22ee00640 1 Processor -- start 2024-04-25T10:21:27.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- start start 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22806aea0 0x7fe2280ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2280d0350 0x7fe2280c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 0x7fe22814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280d0ff0 con 0x7fe22815df60 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280d0e70 con 0x7fe22806aea0 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22816aa20 con 0x7fe2280d0350 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 0x7fe22814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 0x7fe22814f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43210/0 (socket says 172.21.15.45:43210) 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 -- 172.21.15.45:0/713516066 learned_addr learned my addr 172.21.15.45:0/713516066 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:27.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe227e00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22806aea0 0x7fe2280ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe227400640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2280d0350 0x7fe2280c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 -- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2280d0350 msgr2=0x7fe2280c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2280d0350 0x7fe2280c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 -- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22806aea0 msgr2=0x7fe2280ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22806aea0 0x7fe2280ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 -- 172.21.15.45:0/713516066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280703b0 con 0x7fe22815df60 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe227e00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22806aea0 0x7fe2280ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ce00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 0x7fe22814f000 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe2200194b0 tx=0x7fe220019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200b2050 con 0x7fe22815df60 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200be070 con 0x7fe22815df60 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200ad890 con 0x7fe22815df60 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22814f540 con 0x7fe22815df60 2024-04-25T10:21:27.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2280c9330 con 0x7fe22815df60 2024-04-25T10:21:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.315+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22804ef30 con 0x7fe22815df60 2024-04-25T10:21:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.319+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2200ada30 con 0x7fe22815df60 2024-04-25T10:21:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.319+0000 7fe21fe00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe208066f60 0x7fe208069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.319+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2201200b0 con 0x7fe22815df60 2024-04-25T10:21:27.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.319+0000 7fe227e00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe208066f60 0x7fe208069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.319+0000 7fe227e00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe208066f60 0x7fe208069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe218005890 tx=0x7fe218009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.323+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe220016520 con 0x7fe22815df60 2024-04-25T10:21:27.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.483+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2280d09c0 con 0x7fe22815df60 2024-04-25T10:21:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe21fe00640 1 -- 172.21.15.45:0/713516066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fe2200f10b0 con 0x7fe22815df60 2024-04-25T10:21:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:27.490 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":50,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:20:48.857409+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:45.743832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:46.252078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:27.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe208066f60 msgr2=0x7fe208069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe208066f60 0x7fe208069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe218005890 tx=0x7fe218009040 comp rx=0 tx=0).stop 2024-04-25T10:21:27.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 msgr2=0x7fe22814f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 0x7fe22814f000 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe2200194b0 tx=0x7fe220019640 comp rx=0 tx=0).stop 2024-04-25T10:21:27.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 shutdown_connections 2024-04-25T10:21:27.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe208066f60 0x7fe208069420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe22815df60 0x7fe22814f000 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2280d0350 0x7fe2280c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 --2- 172.21.15.45:0/713516066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe22806aea0 0x7fe2280ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 >> 172.21.15.45:0/713516066 conn(0x7fe2280d1910 msgr2=0x7fe22815cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 shutdown_connections 2024-04-25T10:21:27.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.487+0000 7fe22ee00640 1 -- 172.21.15.45:0/713516066 wait complete. 2024-04-25T10:21:27.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:21:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 Processor -- start 2024-04-25T10:21:27.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 -- start start 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20700cea00 0x7f20700cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20700afde0 0x7f20700cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20700ae610 con 0x7f207015f180 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700d27b0 con 0x7f20700cea00 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2077800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20700cf570 con 0x7f20700afde0 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2067e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20700cea00 0x7f20700cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2074e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f207015f180 0x7f20700ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50526/0 (socket says 172.21.15.45:50526) 2024-04-25T10:21:27.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.651+0000 7f2074e00640 1 -- 172.21.15.45:0/1698155856 learned_addr learned my addr 172.21.15.45:0/1698155856 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2075800640 1 --2- 172.21.15.45:0/1698155856 >> v2:172.21.15.117:3300/0 conn(0x7f20700afde0 0x7f20700cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2067e00640 1 -- 172.21.15.45:0/1698155856 >> v2:172.21.15.117:3300/0 conn(0x7f20700afde0 msgr2=0x7f20700cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2067e00640 1 --2- 172.21.15.45:0/1698155856 >> v2:172.21.15.117:3300/0 conn(0x7f20700afde0 0x7f20700cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2067e00640 1 -- 172.21.15.45:0/1698155856 >> v1:172.21.15.45:6789/0 conn(0x7f207015f180 legacy=0x7f20700ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2067e00640 1 -- 172.21.15.45:0/1698155856 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20700cf850 con 0x7f20700cea00 2024-04-25T10:21:27.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f206fe00640 1 -- 172.21.15.45:0/1698155856 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452236856 0 0) 0x7f20700ae610 con 0x7f207015f180 2024-04-25T10:21:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2067e00640 1 --2- 172.21.15.45:0/1698155856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f20700cc510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f20700cbee0 tx=0x7f205801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a585c3e0cbe9375e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f206fe00640 1 -- 172.21.15.45:0/1698155856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580b2070 con 0x7f20700cea00 2024-04-25T10:21:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f206fe00640 1 -- 172.21.15.45:0/1698155856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20580a9070 con 0x7f20700cea00 2024-04-25T10:21:27.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f206fe00640 1 -- 172.21.15.45:0/1698155856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580ad880 con 0x7f20700cea00 2024-04-25T10:21:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2077800640 1 -- 172.21.15.45:0/1698155856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 msgr2=0x7f20700cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.655+0000 7f2077800640 1 --2- 172.21.15.45:0/1698155856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f20700cc510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f20700cbee0 tx=0x7f205801c450 comp rx=0 tx=0).stop 2024-04-25T10:21:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- 172.21.15.45:0/1698155856 shutdown_connections 2024-04-25T10:21:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 --2- 172.21.15.45:0/1698155856 >> v2:172.21.15.117:3300/0 conn(0x7f20700afde0 0x7f20700cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 --2- 172.21.15.45:0/1698155856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f20700cc510 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- 172.21.15.45:0/1698155856 >> 172.21.15.45:0/1698155856 conn(0x7f20700d1900 msgr2=0x7f20700d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- 172.21.15.45:0/1698155856 shutdown_connections 2024-04-25T10:21:27.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- 172.21.15.45:0/1698155856 wait complete. 2024-04-25T10:21:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 Processor -- start 2024-04-25T10:21:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- start start 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20700afde0 0x7f2070154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f2070155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f207015f180 0x7f2070163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2070168a20 con 0x7f20700cea00 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20701688a0 con 0x7f20700afde0 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2070168ba0 con 0x7f207015f180 2024-04-25T10:21:27.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f2070155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f2070155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43222/0 (socket says 172.21.15.45:43222) 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 -- 172.21.15.45:0/4079819558 learned_addr learned my addr 172.21.15.45:0/4079819558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2075800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f207015f180 0x7f2070163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2074e00640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20700afde0 0x7f2070154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 -- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f207015f180 msgr2=0x7f2070163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f207015f180 0x7f2070163b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 -- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20700afde0 msgr2=0x7f2070154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20700afde0 0x7f2070154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 -- 172.21.15.45:0/4079819558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070166660 con 0x7f20700cea00 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2067e00640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f2070155030 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f20580b0040 tx=0x7f20580191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580b2040 con 0x7f20700cea00 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20580bb070 con 0x7f20700cea00 2024-04-25T10:21:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20580b6890 con 0x7f20700cea00 2024-04-25T10:21:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.659+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20701660a0 con 0x7f20700cea00 2024-04-25T10:21:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.663+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2070164dc0 con 0x7f20700cea00 2024-04-25T10:21:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.663+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207004ef30 con 0x7f20700cea00 2024-04-25T10:21:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.663+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20580ab070 con 0x7f20700cea00 2024-04-25T10:21:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.663+0000 7f206d600640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f204c066f60 0x7f204c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.663+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f205811ef10 con 0x7f20700cea00 2024-04-25T10:21:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.663+0000 7f2074e00640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f204c066f60 0x7f204c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:27.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.667+0000 7f2074e00640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f204c066f60 0x7f204c069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2060000f70 tx=0x7f2060007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:27.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.667+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2058016070 con 0x7f20700cea00 2024-04-25T10:21:27.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:27.739 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:27.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.867+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f20700b2e80 con 0x7f20700cea00 2024-04-25T10:21:27.871 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:21:27.867+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:21:27.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.871+0000 7f206d600640 1 -- 172.21.15.45:0/4079819558 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f20580efe90 con 0x7f20700cea00 2024-04-25T10:21:27.873 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f204c066f60 msgr2=0x7f204c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f204c066f60 0x7f204c069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2060000f70 tx=0x7f2060007c10 comp rx=0 tx=0).stop 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 msgr2=0x7f2070155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f2070155030 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f20580b0040 tx=0x7f20580191f0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 shutdown_connections 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f204c066f60 0x7f204c069420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f207015f180 0x7f2070163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20700cea00 0x7f2070155030 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 --2- 172.21.15.45:0/4079819558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20700afde0 0x7f2070154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 >> 172.21.15.45:0/4079819558 conn(0x7f20700d1900 msgr2=0x7f20701596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:27.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 shutdown_connections 2024-04-25T10:21:27.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:27.875+0000 7f2077800640 1 -- 172.21.15.45:0/4079819558 wait complete. 2024-04-25T10:21:27.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:21:27.951 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:21:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe96b600640 1 Processor -- start 2024-04-25T10:21:28.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe96b600640 1 -- start start 2024-04-25T10:21:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe96b600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe964071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe96b600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe964159920 con 0x7fe964159540 2024-04-25T10:21:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe96b600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96415c2f0 con 0x7fe964072020 2024-04-25T10:21:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe96b600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96415c5d0 con 0x7fe9640aeda0 2024-04-25T10:21:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe968c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe964159540 0x7fe9640ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50534/0 (socket says 172.21.15.45:50534) 2024-04-25T10:21:28.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe968c00640 1 -- 172.21.15.45:0/3787857131 learned_addr learned my addr 172.21.15.45:0/3787857131 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963e00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe964071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190879244 0 0) 0x7fe964159920 con 0x7fe964159540 2024-04-25T10:21:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe94c003530 con 0x7fe964159540 2024-04-25T10:21:28.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969431856 0 0) 0x7fe96415c2f0 con 0x7fe964072020 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe964159920 con 0x7fe964072020 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 646549556 0 0) 0x7fe94c003530 con 0x7fe964159540 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 msgr2=0x7fe964071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe964071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 >> v1:172.21.15.117:6789/0 conn(0x7fe964072020 legacy=0x7fe964070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96415c8b0 con 0x7fe964159540 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fe958001680 con 0x7fe964159540 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963e00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe964071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 >> v1:172.21.15.45:6789/0 conn(0x7fe964159540 legacy=0x7fe9640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c003b80 0x7fe94c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c004810 0x7fe94c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe94c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96415c5d0 con 0x7fe94c004810 2024-04-25T10:21:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c00ba30 con 0x7fe94c003b80 2024-04-25T10:21:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.039+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c00bc30 con 0x7fe9640aeda0 2024-04-25T10:21:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c003b80 0x7fe94c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe963e00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c004810 0x7fe94c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe969600640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe94c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 -- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 msgr2=0x7fe94c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe94c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 -- 172.21.15.45:0/3787857131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c004810 msgr2=0x7fe94c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c004810 0x7fe94c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 -- 172.21.15.45:0/3787857131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c003530 con 0x7fe94c003b80 2024-04-25T10:21:28.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe969600640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640aeda0 0x7fe94c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe968c00640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c003b80 0x7fe94c003f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c160 tx=0x7fe95801cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa87a908b6d14eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958007cd0 con 0x7fe94c003b80 2024-04-25T10:21:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9580a9020 con 0x7fe94c003b80 2024-04-25T10:21:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.043+0000 7fe963400640 1 -- 172.21.15.45:0/3787857131 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958005b00 con 0x7fe94c003b80 2024-04-25T10:21:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c003b80 msgr2=0x7fe94c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 --2- 172.21.15.45:0/3787857131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe94c003b80 0x7fe94c003f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c160 tx=0x7fe95801cbc0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe969600640 1 -- 172.21.15.45:0/3787857131 reap_dead start 2024-04-25T10:21:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- 172.21.15.45:0/3787857131 shutdown_connections 2024-04-25T10:21:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- 172.21.15.45:0/3787857131 >> 172.21.15.45:0/3787857131 conn(0x7fe9640669a0 msgr2=0x7fe964066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- 172.21.15.45:0/3787857131 shutdown_connections 2024-04-25T10:21:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- 172.21.15.45:0/3787857131 wait complete. 2024-04-25T10:21:28.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 Processor -- start 2024-04-25T10:21:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- start start 2024-04-25T10:21:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 0x7fe96415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640b2df0 0x7fe9640b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9640b0430 0x7fe9640b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964072450 con 0x7fe94c003b80 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96415bfa0 con 0x7fe9640b0430 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe96b600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96406c0d0 con 0x7fe9640b2df0 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe968c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 0x7fe96415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe968c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 0x7fe96415e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43240/0 (socket says 172.21.15.45:43240) 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe968c00640 1 -- 172.21.15.45:0/3454811417 learned_addr learned my addr 172.21.15.45:0/3454811417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe963e00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640b2df0 0x7fe9640b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe969600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9640b0430 0x7fe9640b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe968c00640 1 -- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640b2df0 msgr2=0x7fe9640b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.047+0000 7fe968c00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640b2df0 0x7fe9640b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe968c00640 1 -- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9640b0430 msgr2=0x7fe9640b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe968c00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9640b0430 0x7fe9640b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe968c00640 1 -- 172.21.15.45:0/3454811417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640b0dd0 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe968c00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 0x7fe96415e3f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe958007040 tx=0x7fe94c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958001c10 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9580049f0 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958004dc0 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640b3840 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9640b3e00 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96404ef30 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe958007070 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe960c00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe944066f90 0x7fe944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe95811f1b0 con 0x7fe94c003b80 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.051+0000 7fe963e00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe944066f90 0x7fe944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.055+0000 7fe963e00640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe944066f90 0x7fe944069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9640cce90 tx=0x7fe950009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.055+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9580168d0 con 0x7fe94c003b80 2024-04-25T10:21:28.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.255+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe964067280 con 0x7fe94c003b80 2024-04-25T10:21:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.871+0000 7fe960c00640 1 -- 172.21.15.45:0/3454811417 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9580f0150 con 0x7fe94c003b80 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe944066f90 msgr2=0x7fe944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe944066f90 0x7fe944069450 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9640cce90 tx=0x7fe950009190 comp rx=0 tx=0).stop 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 msgr2=0x7fe96415e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 0x7fe96415e3f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe958007040 tx=0x7fe94c005840 comp rx=0 tx=0).stop 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 shutdown_connections 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe944066f90 0x7fe944069450 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9640b0430 0x7fe9640b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9640b2df0 0x7fe9640b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 --2- 172.21.15.45:0/3454811417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe94c003b80 0x7fe96415e3f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 >> 172.21.15.45:0/3454811417 conn(0x7fe9640669a0 msgr2=0x7fe9640ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 shutdown_connections 2024-04-25T10:21:28.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:28.875+0000 7fe96b600640 1 -- 172.21.15.45:0/3454811417 wait complete. 2024-04-25T10:21:28.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:21:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 Processor -- start 2024-04-25T10:21:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 -- start start 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bb015bd10 0x7f3bb00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3bb015c0f0 0x7f3bb00b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb006c0e0 con 0x7f3bb00cf9e0 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb015c6a0 con 0x7f3bb015c0f0 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bb7200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb015c980 con 0x7f3bb015bd10 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3ba7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3bb015c0f0 0x7f3bb00b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3ba7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3bb015c0f0 0x7f3bb00b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43244/0 (socket says 172.21.15.45:43244) 2024-04-25T10:21:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3ba7e00640 1 -- 172.21.15.45:0/248858480 learned_addr learned my addr 172.21.15.45:0/248858480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.047+0000 7f3bafe00640 1 --2- 172.21.15.45:0/248858480 >> v2:172.21.15.117:3300/0 conn(0x7f3bb015bd10 0x7f3bb00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3ba7e00640 1 -- 172.21.15.45:0/248858480 >> v2:172.21.15.117:3300/0 conn(0x7f3bb015bd10 msgr2=0x7f3bb00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3ba7e00640 1 --2- 172.21.15.45:0/248858480 >> v2:172.21.15.117:3300/0 conn(0x7f3bb015bd10 0x7f3bb00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3ba7e00640 1 -- 172.21.15.45:0/248858480 >> v1:172.21.15.117:6789/0 conn(0x7f3bb00cf9e0 legacy=0x7f3bb006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3ba7e00640 1 -- 172.21.15.45:0/248858480 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb015cc60 con 0x7f3bb015c0f0 2024-04-25T10:21:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3ba7e00640 1 --2- 172.21.15.45:0/248858480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb00b27e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0072810 tx=0x7f3b9801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886a21fdbe14600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3baf400640 1 -- 172.21.15.45:0/248858480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980aa670 con 0x7f3bb015c0f0 2024-04-25T10:21:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3baf400640 1 -- 172.21.15.45:0/248858480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b980af070 con 0x7f3bb015c0f0 2024-04-25T10:21:29.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3baf400640 1 -- 172.21.15.45:0/248858480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980aaa30 con 0x7f3bb015c0f0 2024-04-25T10:21:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 -- 172.21.15.45:0/248858480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 msgr2=0x7f3bb00b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 --2- 172.21.15.45:0/248858480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb00b27e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0072810 tx=0x7f3b9801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 -- 172.21.15.45:0/248858480 shutdown_connections 2024-04-25T10:21:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 --2- 172.21.15.45:0/248858480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb00b27e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 --2- 172.21.15.45:0/248858480 >> v2:172.21.15.117:3300/0 conn(0x7f3bb015bd10 0x7f3bb00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 -- 172.21.15.45:0/248858480 >> 172.21.15.45:0/248858480 conn(0x7f3bb0066b70 msgr2=0x7f3bb0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 -- 172.21.15.45:0/248858480 shutdown_connections 2024-04-25T10:21:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.051+0000 7f3bb7200640 1 -- 172.21.15.45:0/248858480 wait complete. 2024-04-25T10:21:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 Processor -- start 2024-04-25T10:21:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 -- start start 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb00cf9e0 0x7f3bb0159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb015bd10 0x7f3bb013fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00b2ea0 con 0x7f3bb015c0f0 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00b2d20 con 0x7f3bb015bd10 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb01692d0 con 0x7f3bb00cf9e0 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb014e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb014e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43246/0 (socket says 172.21.15.45:43246) 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 -- 172.21.15.45:0/3109765260 learned_addr learned my addr 172.21.15.45:0/3109765260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:29.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3ba7e00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb015bd10 0x7f3bb013fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bafe00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb00cf9e0 0x7f3bb0159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 -- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb00cf9e0 msgr2=0x7f3bb0159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb00cf9e0 0x7f3bb0159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 -- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb015bd10 msgr2=0x7f3bb013fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb015bd10 0x7f3bb013fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 -- 172.21.15.45:0/3109765260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb014ece0 con 0x7f3bb015c0f0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3ba7e00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb015bd10 0x7f3bb013fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bafe00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb00cf9e0 0x7f3bb0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb5200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb014e7a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c730 tx=0x7f3ba800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8003b80 con 0x7f3bb015c0f0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba8003d20 con 0x7f3bb015c0f0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800aa30 con 0x7f3bb015c0f0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb0159ea0 con 0x7f3bb015c0f0 2024-04-25T10:21:29.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.055+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb015a460 con 0x7f3bb015c0f0 2024-04-25T10:21:29.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.059+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb004ef30 con 0x7f3bb015c0f0 2024-04-25T10:21:29.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.059+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ba800abd0 con 0x7f3bb015c0f0 2024-04-25T10:21:29.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.059+0000 7f3bacc00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b8c066f60 0x7f3b8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.059+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3ba8084f90 con 0x7f3bb015c0f0 2024-04-25T10:21:29.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.059+0000 7f3bafe00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b8c066f60 0x7f3b8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.059+0000 7f3bafe00640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b8c066f60 0x7f3b8c069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000b9f0 tx=0x7f3ba0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.063+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ba8055ea0 con 0x7f3bb015c0f0 2024-04-25T10:21:29.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:29.231+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3bb006d710 con 0x7f3bb015c0f0 2024-04-25T10:21:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.123+0000 7f3bacc00640 1 -- 172.21.15.45:0/3109765260 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ba80557a0 con 0x7f3bb015c0f0 2024-04-25T10:21:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.127+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b8c066f60 msgr2=0x7f3b8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.127+0000 7f3bb7200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b8c066f60 0x7f3b8c069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000b9f0 tx=0x7f3ba0008040 comp rx=0 tx=0).stop 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.127+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 msgr2=0x7f3bb014e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.127+0000 7f3bb7200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb014e7a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c730 tx=0x7f3ba800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 shutdown_connections 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b8c066f60 0x7f3b8c069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bb015c0f0 0x7f3bb014e7a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bb015bd10 0x7f3bb013fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 --2- 172.21.15.45:0/3109765260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bb00cf9e0 0x7f3bb0159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 >> 172.21.15.45:0/3109765260 conn(0x7f3bb0066b70 msgr2=0x7f3bb0154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 shutdown_connections 2024-04-25T10:21:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.131+0000 7f3bb7200640 1 -- 172.21.15.45:0/3109765260 wait complete. 2024-04-25T10:21:30.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:21:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.307+0000 7f2c7ac00640 1 Processor -- start 2024-04-25T10:21:30.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.307+0000 7f2c7ac00640 1 -- start start 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.307+0000 7f2c7ac00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c740cf040 0x7f2c740cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c7ac00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c7415bc00 0x7f2c740cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c7ac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c74071f80 con 0x7f2c740713a0 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c7ac00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c74072260 con 0x7f2c740cf040 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c7ac00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c740b3700 con 0x7f2c7415bc00 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c740cf040 0x7f2c740cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c740cf040 0x7f2c740cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43256/0 (socket says 172.21.15.45:43256) 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 -- 172.21.15.45:0/199892032 learned_addr learned my addr 172.21.15.45:0/199892032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c6be00640 1 --2- 172.21.15.45:0/199892032 >> v2:172.21.15.117:3300/0 conn(0x7f2c7415bc00 0x7f2c740cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 -- 172.21.15.45:0/199892032 >> v2:172.21.15.117:3300/0 conn(0x7f2c7415bc00 msgr2=0x7f2c740cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 --2- 172.21.15.45:0/199892032 >> v2:172.21.15.117:3300/0 conn(0x7f2c7415bc00 0x7f2c740cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 -- 172.21.15.45:0/199892032 >> v1:172.21.15.45:6789/0 conn(0x7f2c740713a0 legacy=0x7f2c740ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 -- 172.21.15.45:0/199892032 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7415b120 con 0x7f2c740cf040 2024-04-25T10:21:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73e00640 1 --2- 172.21.15.45:0/199892032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740cf040 0x7f2c740cf9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2c7406e840 tx=0x7f2c6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b6f39f51ca68d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73400640 1 -- 172.21.15.45:0/199892032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640ab610 con 0x7f2c740cf040 2024-04-25T10:21:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.311+0000 7f2c73400640 1 -- 172.21.15.45:0/199892032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c640b0070 con 0x7f2c740cf040 2024-04-25T10:21:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73400640 1 -- 172.21.15.45:0/199892032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640ab9d0 con 0x7f2c740cf040 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- 172.21.15.45:0/199892032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740cf040 msgr2=0x7f2c740cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/199892032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740cf040 0x7f2c740cf9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2c7406e840 tx=0x7f2c6401c520 comp rx=0 tx=0).stop 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- 172.21.15.45:0/199892032 shutdown_connections 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/199892032 >> v2:172.21.15.117:3300/0 conn(0x7f2c7415bc00 0x7f2c740cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/199892032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740cf040 0x7f2c740cf9e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- 172.21.15.45:0/199892032 >> 172.21.15.45:0/199892032 conn(0x7f2c74066b70 msgr2=0x7f2c74066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- 172.21.15.45:0/199892032 shutdown_connections 2024-04-25T10:21:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- 172.21.15.45:0/199892032 wait complete. 2024-04-25T10:21:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 Processor -- start 2024-04-25T10:21:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- start start 2024-04-25T10:21:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 0x7f2c740add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c740cf040 0x7f2c74159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7415bc00 0x7f2c7415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740cf5a0 con 0x7f2c740713a0 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740cf420 con 0x7f2c7415bc00 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c7ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7415e500 con 0x7f2c740cf040 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 0x7f2c740add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 0x7f2c740add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43260/0 (socket says 172.21.15.45:43260) 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 -- 172.21.15.45:0/1572750993 learned_addr learned my addr 172.21.15.45:0/1572750993 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c78c00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7415bc00 0x7f2c7415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c6be00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c740cf040 0x7f2c74159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 -- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c740cf040 msgr2=0x7f2c74159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c740cf040 0x7f2c74159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 -- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7415bc00 msgr2=0x7f2c7415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7415bc00 0x7f2c7415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.315+0000 7f2c73e00640 1 -- 172.21.15.45:0/1572750993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74151390 con 0x7f2c740713a0 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c73e00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 0x7f2c740add70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2c640ad040 tx=0x7f2c6401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640be5f0 con 0x7f2c740713a0 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c640be7b0 con 0x7f2c740713a0 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c74151670 con 0x7f2c740713a0 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640beba0 con 0x7f2c740713a0 2024-04-25T10:21:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c740aba80 con 0x7f2c740713a0 2024-04-25T10:21:30.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7404ef30 con 0x7f2c740713a0 2024-04-25T10:21:30.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c640b43f0 con 0x7f2c740713a0 2024-04-25T10:21:30.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c70c00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c50066f60 0x7f2c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:30.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f2c6411f550 con 0x7f2c740713a0 2024-04-25T10:21:30.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.319+0000 7f2c6be00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c50066f60 0x7f2c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:30.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.323+0000 7f2c6be00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c50066f60 0x7f2c50069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c74142ba0 tx=0x7f2c5c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:30.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.323+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c640c4060 con 0x7f2c740713a0 2024-04-25T10:21:30.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:30.487+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c74151840 con 0x7f2c740713a0 2024-04-25T10:21:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.131+0000 7f2c70c00640 1 -- 172.21.15.45:0/1572750993 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2c640a8050 con 0x7f2c740713a0 2024-04-25T10:21:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c50066f60 msgr2=0x7f2c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c50066f60 0x7f2c50069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c74142ba0 tx=0x7f2c5c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 msgr2=0x7f2c740add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 0x7f2c740add70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2c640ad040 tx=0x7f2c6401ad60 comp rx=0 tx=0).stop 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 shutdown_connections 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c50066f60 0x7f2c50069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c7415bc00 0x7f2c7415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c740cf040 0x7f2c74159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 --2- 172.21.15.45:0/1572750993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c740713a0 0x7f2c740add70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 >> 172.21.15.45:0/1572750993 conn(0x7f2c74066b70 msgr2=0x7f2c7415d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 shutdown_connections 2024-04-25T10:21:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.135+0000 7f2c7ac00640 1 -- 172.21.15.45:0/1572750993 wait complete. 2024-04-25T10:21:31.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:21:31.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.311+0000 7f663a800640 1 Processor -- start 2024-04-25T10:21:31.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.311+0000 7f663a800640 1 -- start start 2024-04-25T10:21:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f663a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66340b38d0 0x7f663415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f663a800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663415b080 con 0x7f66340aef80 2024-04-25T10:21:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f663a800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663415adb0 con 0x7f6634070350 2024-04-25T10:21:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f663a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663415eb60 con 0x7f66340b38d0 2024-04-25T10:21:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f66340aef80 0x7f663415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50566/0 (socket says 172.21.15.45:50566) 2024-04-25T10:21:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633400640 1 -- 172.21.15.45:0/2362181486 learned_addr learned my addr 172.21.15.45:0/2362181486 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66340b38d0 0x7f663415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299603338 0 0) 0x7f663415b080 con 0x7f66340aef80 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6618003530 con 0x7f66340aef80 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647377090 0 0) 0x7f663415adb0 con 0x7f6634070350 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f663415b080 con 0x7f6634070350 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975450027 0 0) 0x7f6618003530 con 0x7f66340aef80 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66340b38d0 msgr2=0x7f663415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66340b38d0 0x7f663415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 >> v1:172.21.15.117:6789/0 conn(0x7f6634070350 legacy=0x7f66340b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663415ee40 con 0x7f66340aef80 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6624001a00 con 0x7f66340aef80 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 >> v1:172.21.15.45:6789/0 conn(0x7f66340aef80 legacy=0x7f663415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6618003b80 0x7f6618003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6618004810 0x7f6618004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f661800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663415eb60 con 0x7f6618005190 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661800bf10 con 0x7f6618003b80 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661800c090 con 0x7f6618004810 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f661800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633400640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6618003b80 0x7f6618003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f662aa00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6618004810 0x7f6618004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 -- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6618004810 msgr2=0x7f6618004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6618004810 0x7f6618004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 -- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6618003b80 msgr2=0x7f6618003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6618003b80 0x7f6618003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.315+0000 7f6633e00640 1 -- 172.21.15.45:0/2362181486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618003530 con 0x7f6618005190 2024-04-25T10:21:31.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f6633e00640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f661800b9d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f662c009ab0 tx=0x7f662c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9067014cf37326ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0b2070 con 0x7f6618005190 2024-04-25T10:21:31.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c0a5530 con 0x7f6618005190 2024-04-25T10:21:31.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f6632a00640 1 -- 172.21.15.45:0/2362181486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c0096b0 con 0x7f6618005190 2024-04-25T10:21:31.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f663a800640 1 -- 172.21.15.45:0/2362181486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 msgr2=0x7f661800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f663a800640 1 --2- 172.21.15.45:0/2362181486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f661800b9d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f662c009ab0 tx=0x7f662c01ca50 comp rx=0 tx=0).stop 2024-04-25T10:21:31.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f6633e00640 1 -- 172.21.15.45:0/2362181486 reap_dead start 2024-04-25T10:21:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f663a800640 1 -- 172.21.15.45:0/2362181486 shutdown_connections 2024-04-25T10:21:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f663a800640 1 -- 172.21.15.45:0/2362181486 >> 172.21.15.45:0/2362181486 conn(0x7f66340669a0 msgr2=0x7f6634066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f663a800640 1 -- 172.21.15.45:0/2362181486 shutdown_connections 2024-04-25T10:21:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.319+0000 7f663a800640 1 -- 172.21.15.45:0/2362181486 wait complete. 2024-04-25T10:21:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 Processor -- start 2024-04-25T10:21:31.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- start start 2024-04-25T10:21:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f66340c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66340c70e0 0x7f66340afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f663414f180 0x7f6634157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663415bc90 con 0x7f6618005190 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6634071a00 con 0x7f66340c70e0 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663415e480 con 0x7f663414f180 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f66340c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f66340c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43276/0 (socket says 172.21.15.45:43276) 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 -- 172.21.15.45:0/501862338 learned_addr learned my addr 172.21.15.45:0/501862338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633e00640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f663414f180 0x7f6634157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f662aa00640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66340c70e0 0x7f66340afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 -- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f663414f180 msgr2=0x7f6634157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f663414f180 0x7f6634157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 -- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66340c70e0 msgr2=0x7f66340afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66340c70e0 0x7f66340afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 -- 172.21.15.45:0/501862338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634071830 con 0x7f6618005190 2024-04-25T10:21:31.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f6633400640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f66340c5780 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6624007610 tx=0x7f6624007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662401c070 con 0x7f6618005190 2024-04-25T10:21:31.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662400ae40 con 0x7f6618005190 2024-04-25T10:21:31.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66340b05b0 con 0x7f6618005190 2024-04-25T10:21:31.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624010040 con 0x7f6618005190 2024-04-25T10:21:31.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66340b0ea0 con 0x7f6618005190 2024-04-25T10:21:31.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.323+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66340b09e0 con 0x7f6618005190 2024-04-25T10:21:31.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.327+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f662401f070 con 0x7f6618005190 2024-04-25T10:21:31.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.327+0000 7f662be00640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6610066f90 0x7f6610069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.327+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f66240879e0 con 0x7f6618005190 2024-04-25T10:21:31.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.327+0000 7f662aa00640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6610066f90 0x7f6610069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.327+0000 7f662aa00640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6610066f90 0x7f6610069450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f663415c590 tx=0x7f661c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.331+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66340b09e0 con 0x7f6618005190 2024-04-25T10:21:31.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.503+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f66340b4120 con 0x7f6618005190 2024-04-25T10:21:31.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f662be00640 1 -- 172.21.15.45:0/501862338 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6624058690 con 0x7f6618005190 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6610066f90 msgr2=0x7f6610069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6610066f90 0x7f6610069450 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f663415c590 tx=0x7f661c009040 comp rx=0 tx=0).stop 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 msgr2=0x7f66340c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f66340c5780 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6624007610 tx=0x7f6624007a10 comp rx=0 tx=0).stop 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 shutdown_connections 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6610066f90 0x7f6610069450 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f663414f180 0x7f6634157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66340c70e0 0x7f66340afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 --2- 172.21.15.45:0/501862338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6618005190 0x7f66340c5780 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 >> 172.21.15.45:0/501862338 conn(0x7f66340669a0 msgr2=0x7f663415d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:31.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 shutdown_connections 2024-04-25T10:21:31.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.507+0000 7f663a800640 1 -- 172.21.15.45:0/501862338 wait complete. 2024-04-25T10:21:31.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:21:31.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dca800640 1 Processor -- start 2024-04-25T10:21:31.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dca800640 1 -- start start 2024-04-25T10:21:31.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dca800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7dc40cfd90 0x7f7dc40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dca800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc415f3b0 con 0x7f7dc40cd110 2024-04-25T10:21:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dca800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc40ca810 con 0x7f7dc40b0f70 2024-04-25T10:21:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dca800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc415ef20 con 0x7f7dc40cfd90 2024-04-25T10:21:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dc3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7dc40cd110 0x7f7dc40cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50582/0 (socket says 172.21.15.45:50582) 2024-04-25T10:21:31.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dc3e00640 1 -- 172.21.15.45:0/3638522447 learned_addr learned my addr 172.21.15.45:0/3638522447 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dbaa00640 1 --2- 172.21.15.45:0/3638522447 >> v2:172.21.15.117:3300/0 conn(0x7f7dc40cfd90 0x7f7dc40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.667+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314125578 0 0) 0x7f7dc415f3b0 con 0x7f7dc40cd110 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7da8003530 con 0x7f7dc40cd110 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597610980 0 0) 0x7f7dc40ca810 con 0x7f7dc40b0f70 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dc415f3b0 con 0x7f7dc40b0f70 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803215535 0 0) 0x7f7da8003530 con 0x7f7dc40cd110 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 >> v2:172.21.15.117:3300/0 conn(0x7f7dc40cfd90 msgr2=0x7f7dc40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 --2- 172.21.15.45:0/3638522447 >> v2:172.21.15.117:3300/0 conn(0x7f7dc40cfd90 0x7f7dc40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 >> v1:172.21.15.117:6789/0 conn(0x7f7dc40b0f70 legacy=0x7f7dc40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc415e5f0 con 0x7f7dc40cd110 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f7dbc001a00 con 0x7f7dc40cd110 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 >> v1:172.21.15.45:6789/0 conn(0x7f7dc40cd110 legacy=0x7f7dc40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7da8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7da8004810 0x7f7da8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da80051d0 0x7f7da800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc415ef20 con 0x7f7da80051d0 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da800c070 con 0x7f7da8003b50 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da800c270 con 0x7f7da8004810 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dbaa00640 1 --2- 172.21.15.45:0/3638522447 >> v2:172.21.15.117:3300/0 conn(0x7f7dc40cfd90 0x7f7dc40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7da8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3e00640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da80051d0 0x7f7da800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dbaa00640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7da8004810 0x7f7da8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3400640 1 -- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7da8004810 msgr2=0x7f7da8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7da8004810 0x7f7da8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3400640 1 -- 172.21.15.45:0/3638522447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da80051d0 msgr2=0x7f7da800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da80051d0 0x7f7da800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3400640 1 -- 172.21.15.45:0/3638522447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da8003530 con 0x7f7da8003b50 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3e00640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7da80051d0 0x7f7da800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.671+0000 7f7dc3e00640 1 -- 172.21.15.45:0/3638522447 reap_dead start 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7da8003f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7db40018d0 tx=0x7f7db401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f2558105d186a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4001520 con 0x7f7da8003b50 2024-04-25T10:21:31.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db40052f0 con 0x7f7da8003b50 2024-04-25T10:21:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dc2a00640 1 -- 172.21.15.45:0/3638522447 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4003840 con 0x7f7da8003b50 2024-04-25T10:21:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 msgr2=0x7f7da8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7da8003f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7db40018d0 tx=0x7f7db401c560 comp rx=0 tx=0).stop 2024-04-25T10:21:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- 172.21.15.45:0/3638522447 shutdown_connections 2024-04-25T10:21:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 --2- 172.21.15.45:0/3638522447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7da8003f70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- 172.21.15.45:0/3638522447 >> 172.21.15.45:0/3638522447 conn(0x7f7dc40b1d50 msgr2=0x7f7dc40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- 172.21.15.45:0/3638522447 shutdown_connections 2024-04-25T10:21:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- 172.21.15.45:0/3638522447 wait complete. 2024-04-25T10:21:31.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 Processor -- start 2024-04-25T10:21:31.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- start start 2024-04-25T10:21:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7dc40af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 0x7f7dc40c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc40ad960 0x7f7dc40b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40cd520 con 0x7f7dc41603e0 2024-04-25T10:21:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc415ebb0 con 0x7f7da8003b50 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.675+0000 7f7dca800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40ccbd0 con 0x7f7dc40ad960 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 0x7f7dc40c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 0x7f7dc40c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43288/0 (socket says 172.21.15.45:43288) 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 -- 172.21.15.45:0/3318325839 learned_addr learned my addr 172.21.15.45:0/3318325839 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 -- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc40ad960 msgr2=0x7f7dc40b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc40ad960 0x7f7dc40b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 -- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 msgr2=0x7f7dc40af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7dc40af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7dc40af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 -- 172.21.15.45:0/3318325839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40afdb0 con 0x7f7dc41603e0 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7dc40af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbaa00640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 0x7f7dc40c3c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7dac011cd0 tx=0x7f7dac00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac011e80 con 0x7f7dc41603e0 2024-04-25T10:21:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac004020 con 0x7f7dc41603e0 2024-04-25T10:21:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0043d0 con 0x7f7dc41603e0 2024-04-25T10:21:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc40b0090 con 0x7f7dc41603e0 2024-04-25T10:21:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc40b4be0 con 0x7f7dc41603e0 2024-04-25T10:21:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc404ef30 con 0x7f7dc41603e0 2024-04-25T10:21:31.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.679+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dac012040 con 0x7f7dc41603e0 2024-04-25T10:21:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.683+0000 7f7dbbe00640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7da0066f60 0x7f7da0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.683+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7dac085510 con 0x7f7dc41603e0 2024-04-25T10:21:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.683+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7da0066f60 0x7f7da0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.683+0000 7f7dc3400640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7da0066f60 0x7f7da0069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ad000 tx=0x7f7db40a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:31.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.687+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dac019030 con 0x7f7dc41603e0 2024-04-25T10:21:31.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:31.915+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f7dc40cc9b0 con 0x7f7dc41603e0 2024-04-25T10:21:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dbbe00640 1 -- 172.21.15.45:0/3318325839 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v188) v1 ==== 143+0+0 (secure 0 0 0) 0x7f7dac0566f0 con 0x7f7dc41603e0 2024-04-25T10:21:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7da0066f60 msgr2=0x7f7da0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7da0066f60 0x7f7da0069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ad000 tx=0x7f7db40a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 msgr2=0x7f7dc40c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 0x7f7dc40c3c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7dac011cd0 tx=0x7f7dac00f610 comp rx=0 tx=0).stop 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 shutdown_connections 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7da0066f60 0x7f7da0069420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7dc40ad960 0x7f7dc40b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dc41603e0 0x7f7dc40c3c20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 --2- 172.21.15.45:0/3318325839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7da8003b50 0x7f7dc40af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.135+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 >> 172.21.15.45:0/3318325839 conn(0x7f7dc40b1d50 msgr2=0x7f7dc4072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.139+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 shutdown_connections 2024-04-25T10:21:32.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.139+0000 7f7dca800640 1 -- 172.21.15.45:0/3318325839 wait complete. 2024-04-25T10:21:32.161 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:21:32.161 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:21:32.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:32.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.299+0000 7f18df600640 1 Processor -- start 2024-04-25T10:21:32.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.299+0000 7f18df600640 1 -- start start 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18df600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d815bc40 0x7f18d806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18df600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18d80cda10 0x7f18d80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18df600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18d80cf930 0x7f18d80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18df600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d80b2770 con 0x7f18d80cda10 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18df600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d806c670 con 0x7f18d80cf930 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18df600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d806c950 con 0x7f18d815bc40 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18d80cda10 0x7f18d80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18d80cda10 0x7f18d80b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43304/0 (socket says 172.21.15.45:43304) 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 -- 172.21.15.45:0/3944206955 learned_addr learned my addr 172.21.15.45:0/3944206955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:32.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18dd600640 1 --2- 172.21.15.45:0/3944206955 >> v2:172.21.15.117:3300/0 conn(0x7f18d80cf930 0x7f18d80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18dcc00640 1 --2- 172.21.15.45:0/3944206955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d815bc40 0x7f18d806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 -- 172.21.15.45:0/3944206955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d815bc40 msgr2=0x7f18d806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 --2- 172.21.15.45:0/3944206955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d815bc40 0x7f18d806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 -- 172.21.15.45:0/3944206955 >> v2:172.21.15.117:3300/0 conn(0x7f18d80cf930 msgr2=0x7f18d80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 --2- 172.21.15.45:0/3944206955 >> v2:172.21.15.117:3300/0 conn(0x7f18d80cf930 0x7f18d80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18d7e00640 1 -- 172.21.15.45:0/3944206955 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d806cc30 con 0x7f18d80cda10 2024-04-25T10:21:32.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.303+0000 7f18dd600640 1 --2- 172.21.15.45:0/3944206955 >> v2:172.21.15.117:3300/0 conn(0x7f18d80cf930 0x7f18d80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:32.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18d7e00640 1 --2- 172.21.15.45:0/3944206955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d80cda10 0x7f18d80b4610 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f18d80b3f80 tx=0x7f18c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c379b31dc7f3a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18d7400640 1 -- 172.21.15.45:0/3944206955 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c40b0070 con 0x7f18d80cda10 2024-04-25T10:21:32.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18d7400640 1 -- 172.21.15.45:0/3944206955 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c40ab600 con 0x7f18d80cda10 2024-04-25T10:21:32.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18d7400640 1 -- 172.21.15.45:0/3944206955 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c40ab9d0 con 0x7f18d80cda10 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 -- 172.21.15.45:0/3944206955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d80cda10 msgr2=0x7f18d80b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 --2- 172.21.15.45:0/3944206955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d80cda10 0x7f18d80b4610 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f18d80b3f80 tx=0x7f18c401c520 comp rx=0 tx=0).stop 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 -- 172.21.15.45:0/3944206955 shutdown_connections 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 --2- 172.21.15.45:0/3944206955 >> v2:172.21.15.117:3300/0 conn(0x7f18d80cf930 0x7f18d80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 --2- 172.21.15.45:0/3944206955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d80cda10 0x7f18d80b4610 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 --2- 172.21.15.45:0/3944206955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d815bc40 0x7f18d806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 -- 172.21.15.45:0/3944206955 >> 172.21.15.45:0/3944206955 conn(0x7f18d8066b70 msgr2=0x7f18d8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 -- 172.21.15.45:0/3944206955 shutdown_connections 2024-04-25T10:21:32.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 -- 172.21.15.45:0/3944206955 wait complete. 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 Processor -- start 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.307+0000 7f18df600640 1 -- start start 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d80cda10 0x7f18d8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d80cf930 0x7f18d8149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 0x7f18d8149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d8160230 con 0x7f18d815bc40 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81600b0 con 0x7f18d80cda10 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d81603b0 con 0x7f18d80cf930 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 0x7f18d8149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dcc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d80cda10 0x7f18d8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 0x7f18d8149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43306/0 (socket says 172.21.15.45:43306) 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 -- 172.21.15.45:0/2081775062 learned_addr learned my addr 172.21.15.45:0/2081775062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18d7e00640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d80cf930 0x7f18d8149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 -- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d80cf930 msgr2=0x7f18d8149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d80cf930 0x7f18d8149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 -- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d80cda10 msgr2=0x7f18d8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d80cda10 0x7f18d8155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 -- 172.21.15.45:0/2081775062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d8156310 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18dd600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 0x7f18d8149d60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f18d0007db0 tx=0x7f18d000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d001a070 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d00039a0 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d8148460 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0003d70 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d8148a20 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d804ef30 con 0x7f18d815bc40 2024-04-25T10:21:32.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.311+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18d001d070 con 0x7f18d815bc40 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.315+0000 7f18d4c00640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18b8066f60 0x7f18b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.315+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f18d0084f40 con 0x7f18d815bc40 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.315+0000 7f18dcc00640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18b8066f60 0x7f18b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.315+0000 7f18dcc00640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18b8066f60 0x7f18b8069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f18cc007a60 tx=0x7f18cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.319+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18d0010070 con 0x7f18d815bc40 2024-04-25T10:21:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.483+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18d8155870 con 0x7f18d815bc40 2024-04-25T10:21:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.483+0000 7f18d4c00640 1 -- 172.21.15.45:0/2081775062 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13010 (secure 0 0 0) 0x7f18d0055b60 con 0x7f18d815bc40 2024-04-25T10:21:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:32.486 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":53,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:31.129225+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6836/3308184634":"2024-04-26T10:21:27.871953+0000","172.21.15.117:6834/3308184634":"2024-04-26T10:21:27.871953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.483+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18b8066f60 msgr2=0x7f18b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.483+0000 7f18df600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18b8066f60 0x7f18b8069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f18cc007a60 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.483+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 msgr2=0x7f18d8149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.483+0000 7f18df600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 0x7f18d8149d60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f18d0007db0 tx=0x7f18d000d4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 shutdown_connections 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18b8066f60 0x7f18b8069420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18d815bc40 0x7f18d8149d60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18d80cf930 0x7f18d8149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 --2- 172.21.15.45:0/2081775062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d80cda10 0x7f18d8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 >> 172.21.15.45:0/2081775062 conn(0x7f18d8066b70 msgr2=0x7f18d80cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 shutdown_connections 2024-04-25T10:21:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.487+0000 7f18df600640 1 -- 172.21.15.45:0/2081775062 wait complete. 2024-04-25T10:21:32.507 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:32.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:32.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.647+0000 7fd9bea00640 1 Processor -- start 2024-04-25T10:21:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bea00640 1 -- start start 2024-04-25T10:21:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd9b80aebd0 0x7fd9b8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bea00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b8071360 con 0x7fd9b8072020 2024-04-25T10:21:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bea00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80af030 con 0x7fd9b80aebd0 2024-04-25T10:21:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8159920 con 0x7fd9b8159540 2024-04-25T10:21:32.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9afe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd9b80aebd0 0x7fd9b8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bca00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd9b8072020 0x7fd9b815f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:34046/0 (socket says 172.21.15.45:34046) 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9bca00640 1 -- 172.21.15.45:0/65204802 learned_addr learned my addr 172.21.15.45:0/65204802 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9b7e00640 1 -- 172.21.15.45:0/65204802 >> v2:172.21.15.117:3300/0 conn(0x7fd9b80aebd0 msgr2=0x7fd9b8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9b7e00640 1 --2- 172.21.15.45:0/65204802 >> v2:172.21.15.117:3300/0 conn(0x7fd9b80aebd0 0x7fd9b8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9b7400640 1 -- 172.21.15.45:0/65204802 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470743350 0 0) 0x7fd9b8071360 con 0x7fd9b8072020 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9b7e00640 1 -- 172.21.15.45:0/65204802 >> v1:172.21.15.117:6789/0 conn(0x7fd9b8072020 legacy=0x7fd9b815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.651+0000 7fd9b7e00640 1 -- 172.21.15.45:0/65204802 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b815c2f0 con 0x7fd9b8159540 2024-04-25T10:21:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9b7e00640 1 --2- 172.21.15.45:0/65204802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b80ae690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80097c0 tx=0x7fd9a801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81fcf18002deefe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9b7400640 1 -- 172.21.15.45:0/65204802 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80b2070 con 0x7fd9b8159540 2024-04-25T10:21:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9b7400640 1 -- 172.21.15.45:0/65204802 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a80ad640 con 0x7fd9b8159540 2024-04-25T10:21:32.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9b7400640 1 -- 172.21.15.45:0/65204802 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80ada10 con 0x7fd9b8159540 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 -- 172.21.15.45:0/65204802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 msgr2=0x7fd9b80ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 --2- 172.21.15.45:0/65204802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b80ae690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80097c0 tx=0x7fd9a801c470 comp rx=0 tx=0).stop 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 -- 172.21.15.45:0/65204802 shutdown_connections 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 --2- 172.21.15.45:0/65204802 >> v2:172.21.15.117:3300/0 conn(0x7fd9b80aebd0 0x7fd9b8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 --2- 172.21.15.45:0/65204802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b80ae690 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 -- 172.21.15.45:0/65204802 >> 172.21.15.45:0/65204802 conn(0x7fd9b80669a0 msgr2=0x7fd9b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 -- 172.21.15.45:0/65204802 shutdown_connections 2024-04-25T10:21:32.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 -- 172.21.15.45:0/65204802 wait complete. 2024-04-25T10:21:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.655+0000 7fd9bea00640 1 Processor -- start 2024-04-25T10:21:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- start start 2024-04-25T10:21:32.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 0x7fd9b814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b80aebd0 0x7fd9b814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b806c0d0 con 0x7fd9b8072020 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b815bfa0 con 0x7fd9b80aebd0 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8072430 con 0x7fd9b8159540 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 0x7fd9b814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 0x7fd9b814fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43320/0 (socket says 172.21.15.45:43320) 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 -- 172.21.15.45:0/3438130253 learned_addr learned my addr 172.21.15.45:0/3438130253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9afe00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b80aebd0 0x7fd9b814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bca00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 -- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 msgr2=0x7fd9b8154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b8154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 -- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b80aebd0 msgr2=0x7fd9b814eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b80aebd0 0x7fd9b814eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 -- 172.21.15.45:0/3438130253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8150370 con 0x7fd9b8072020 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b7e00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 0x7fd9b814fd90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd9a801a0a0 tx=0x7fd9a8019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80b2040 con 0x7fd9b8072020 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a80bc070 con 0x7fd9b8072020 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b8150650 con 0x7fd9b8072020 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80b6630 con 0x7fd9b8072020 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b8150c10 con 0x7fd9b8072020 2024-04-25T10:21:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.659+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b804ef30 con 0x7fd9b8072020 2024-04-25T10:21:32.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.663+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9a80b67d0 con 0x7fd9b8072020 2024-04-25T10:21:32.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.663+0000 7fd9b4c00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd994066f60 0x7fd994069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.663+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd9a811ec20 con 0x7fd9b8072020 2024-04-25T10:21:32.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.663+0000 7fd9afe00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd994066f60 0x7fd994069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.663+0000 7fd9afe00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd994066f60 0x7fd994069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd9b815b210 tx=0x7fd9a0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:32.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.667+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9a8016090 con 0x7fd9b8072020 2024-04-25T10:21:32.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.831+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9b81553d0 con 0x7fd9b8072020 2024-04-25T10:21:32.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.831+0000 7fd9b4c00640 1 -- 172.21.15.45:0/3438130253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9a80efe90 con 0x7fd9b8072020 2024-04-25T10:21:32.835 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd994066f60 msgr2=0x7fd994069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd994066f60 0x7fd994069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd9b815b210 tx=0x7fd9a0009190 comp rx=0 tx=0).stop 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 msgr2=0x7fd9b814fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 0x7fd9b814fd90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd9a801a0a0 tx=0x7fd9a8019210 comp rx=0 tx=0).stop 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 shutdown_connections 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd994066f60 0x7fd994069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9b8159540 0x7fd9b8154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b80aebd0 0x7fd9b814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 --2- 172.21.15.45:0/3438130253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9b8072020 0x7fd9b814fd90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 >> 172.21.15.45:0/3438130253 conn(0x7fd9b80669a0 msgr2=0x7fd9b815e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 shutdown_connections 2024-04-25T10:21:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.835+0000 7fd9bea00640 1 -- 172.21.15.45:0/3438130253 wait complete. 2024-04-25T10:21:32.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:21:32.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 Processor -- start 2024-04-25T10:21:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 -- start start 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400b16c0 0x7eff400b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7eff400670b0 0x7eff400ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4015de30 con 0x7eff4015da50 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400b59f0 con 0x7eff400670b0 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400b5cd0 con 0x7eff400b16c0 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff45a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7eff4015da50 0x7eff400669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50586/0 (socket says 172.21.15.45:50586) 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff45a00640 1 -- 172.21.15.45:0/3922777497 learned_addr learned my addr 172.21.15.45:0/3922777497 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46400640 1 --2- 172.21.15.45:0/3922777497 >> v2:172.21.15.45:3300/0 conn(0x7eff400670b0 0x7eff400ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff45000640 1 --2- 172.21.15.45:0/3922777497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400b16c0 0x7eff400b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff3fe00640 1 -- 172.21.15.45:0/3922777497 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648787440 0 0) 0x7eff4015de30 con 0x7eff4015da50 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff3fe00640 1 -- 172.21.15.45:0/3922777497 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff28003530 con 0x7eff4015da50 2024-04-25T10:21:32.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46400640 1 -- 172.21.15.45:0/3922777497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400b16c0 msgr2=0x7eff400b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46400640 1 --2- 172.21.15.45:0/3922777497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400b16c0 0x7eff400b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46400640 1 -- 172.21.15.45:0/3922777497 >> v1:172.21.15.45:6789/0 conn(0x7eff4015da50 legacy=0x7eff400669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.995+0000 7eff46400640 1 -- 172.21.15.45:0/3922777497 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400b5fb0 con 0x7eff400670b0 2024-04-25T10:21:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46400640 1 --2- 172.21.15.45:0/3922777497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff400670b0 0x7eff400ce6b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7eff38009d90 tx=0x7eff3801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f135aa602fce02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff3fe00640 1 -- 172.21.15.45:0/3922777497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380b2070 con 0x7eff400670b0 2024-04-25T10:21:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff3fe00640 1 -- 172.21.15.45:0/3922777497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff380a58b0 con 0x7eff400670b0 2024-04-25T10:21:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff3fe00640 1 -- 172.21.15.45:0/3922777497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380a5c80 con 0x7eff400670b0 2024-04-25T10:21:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 -- 172.21.15.45:0/3922777497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff400670b0 msgr2=0x7eff400ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 --2- 172.21.15.45:0/3922777497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff400670b0 0x7eff400ce6b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7eff38009d90 tx=0x7eff3801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 -- 172.21.15.45:0/3922777497 shutdown_connections 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 --2- 172.21.15.45:0/3922777497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff400670b0 0x7eff400ce6b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 --2- 172.21.15.45:0/3922777497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400b16c0 0x7eff400b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 -- 172.21.15.45:0/3922777497 >> 172.21.15.45:0/3922777497 conn(0x7eff400d0690 msgr2=0x7eff400d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 -- 172.21.15.45:0/3922777497 shutdown_connections 2024-04-25T10:21:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 -- 172.21.15.45:0/3922777497 wait complete. 2024-04-25T10:21:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:32.999+0000 7eff46e00640 1 Processor -- start 2024-04-25T10:21:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- start start 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400670b0 0x7eff40158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff400b16c0 0x7eff40150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 0x7eff400cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cf790 con 0x7eff4015da50 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cf610 con 0x7eff400b16c0 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cf910 con 0x7eff400670b0 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 0x7eff400cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 0x7eff400cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43348/0 (socket says 172.21.15.45:43348) 2024-04-25T10:21:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 -- 172.21.15.45:0/2691430579 learned_addr learned my addr 172.21.15.45:0/2691430579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff45a00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400670b0 0x7eff40158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff45000640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff400b16c0 0x7eff40150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 -- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400670b0 msgr2=0x7eff40158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400670b0 0x7eff40158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 -- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff400b16c0 msgr2=0x7eff40150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff400b16c0 0x7eff40150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 -- 172.21.15.45:0/2691430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400cbab0 con 0x7eff4015da50 2024-04-25T10:21:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46400640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 0x7eff400cb570 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7eff380b0040 tx=0x7eff38019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380b2050 con 0x7eff4015da50 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff380be070 con 0x7eff4015da50 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff40156100 con 0x7eff4015da50 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380ad890 con 0x7eff4015da50 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff401563c0 con 0x7eff4015da50 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.003+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff40075120 con 0x7eff4015da50 2024-04-25T10:21:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.007+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff380ada30 con 0x7eff4015da50 2024-04-25T10:21:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.007+0000 7eff3d600640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff20066f90 0x7eff20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.007+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7eff3811f7a0 con 0x7eff4015da50 2024-04-25T10:21:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.007+0000 7eff45a00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff20066f90 0x7eff20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.007+0000 7eff45a00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff20066f90 0x7eff20069450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff34007fd0 tx=0x7eff34009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:33.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.011+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff380c7020 con 0x7eff4015da50 2024-04-25T10:21:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:33.231+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7eff40155cc0 con 0x7eff4015da50 2024-04-25T10:21:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.143+0000 7eff3d600640 1 -- 172.21.15.45:0/2691430579 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v190) v1 ==== 146+0+0 (secure 0 0 0) 0x7eff400cbab0 con 0x7eff4015da50 2024-04-25T10:21:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.143+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff20066f90 msgr2=0x7eff20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.143+0000 7eff46e00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff20066f90 0x7eff20069450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff34007fd0 tx=0x7eff34009040 comp rx=0 tx=0).stop 2024-04-25T10:21:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.143+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 msgr2=0x7eff400cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.143+0000 7eff46e00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 0x7eff400cb570 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7eff380b0040 tx=0x7eff38019680 comp rx=0 tx=0).stop 2024-04-25T10:21:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 shutdown_connections 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff20066f90 0x7eff20069450 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff4015da50 0x7eff400cb570 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff400b16c0 0x7eff40150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 --2- 172.21.15.45:0/2691430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff400670b0 0x7eff40158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 >> 172.21.15.45:0/2691430579 conn(0x7eff400d0690 msgr2=0x7eff4015c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 shutdown_connections 2024-04-25T10:21:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.147+0000 7eff46e00640 1 -- 172.21.15.45:0/2691430579 wait complete. 2024-04-25T10:21:34.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:21:34.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.315+0000 7f8efc000640 1 Processor -- start 2024-04-25T10:21:34.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.315+0000 7f8efc000640 1 -- start start 2024-04-25T10:21:34.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efc000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ef415a0e0 0x7f8ef415b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efc000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef406c0d0 0x7f8ef406c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efc000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40b2fd0 con 0x7f8ef40aea50 2024-04-25T10:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efc000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40af1d0 con 0x7f8ef415a0e0 2024-04-25T10:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efc000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40b32b0 con 0x7f8ef406c0d0 2024-04-25T10:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ef415a0e0 0x7f8ef415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ef415a0e0 0x7f8ef415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43360/0 (socket says 172.21.15.45:43360) 2024-04-25T10:21:34.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 -- 172.21.15.45:0/2506902362 learned_addr learned my addr 172.21.15.45:0/2506902362 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efa000640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef406c0d0 0x7f8ef406c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 -- 172.21.15.45:0/2506902362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef406c0d0 msgr2=0x7f8ef406c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef406c0d0 0x7f8ef406c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 -- 172.21.15.45:0/2506902362 >> v1:172.21.15.117:6789/0 conn(0x7f8ef40aea50 legacy=0x7f8ef40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8ef9600640 1 -- 172.21.15.45:0/2506902362 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40b3590 con 0x7f8ef415a0e0 2024-04-25T10:21:34.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.319+0000 7f8efa000640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef406c0d0 0x7f8ef406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:34.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8ef9600640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef415a0e0 0x7f8ef415b150 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009870 tx=0x7f8ee801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e90597f5387d703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8ef3e00640 1 -- 172.21.15.45:0/2506902362 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80b0070 con 0x7f8ef415a0e0 2024-04-25T10:21:34.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8ef3e00640 1 -- 172.21.15.45:0/2506902362 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee80ab600 con 0x7f8ef415a0e0 2024-04-25T10:21:34.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8ef3e00640 1 -- 172.21.15.45:0/2506902362 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80a4e60 con 0x7f8ef415a0e0 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 -- 172.21.15.45:0/2506902362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef415a0e0 msgr2=0x7f8ef415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef415a0e0 0x7f8ef415b150 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009870 tx=0x7f8ee801c490 comp rx=0 tx=0).stop 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 -- 172.21.15.45:0/2506902362 shutdown_connections 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef406c0d0 0x7f8ef406c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 --2- 172.21.15.45:0/2506902362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef415a0e0 0x7f8ef415b150 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 -- 172.21.15.45:0/2506902362 >> 172.21.15.45:0/2506902362 conn(0x7f8ef40669a0 msgr2=0x7f8ef4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 -- 172.21.15.45:0/2506902362 shutdown_connections 2024-04-25T10:21:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 -- 172.21.15.45:0/2506902362 wait complete. 2024-04-25T10:21:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 Processor -- start 2024-04-25T10:21:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.323+0000 7f8efc000640 1 -- start start 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef406c0d0 0x7f8ef40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 0x7f8ef415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef415a0e0 0x7f8ef4142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4070770 con 0x7f8ef40aea50 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40705f0 con 0x7f8ef406c0d0 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40708f0 con 0x7f8ef415a0e0 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 0x7f8ef415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 0x7f8ef415b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43366/0 (socket says 172.21.15.45:43366) 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 -- 172.21.15.45:0/847067628 learned_addr learned my addr 172.21.15.45:0/847067628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:34.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efa000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef415a0e0 0x7f8ef4142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef9600640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef406c0d0 0x7f8ef40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 -- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef415a0e0 msgr2=0x7f8ef4142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef415a0e0 0x7f8ef4142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 -- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef406c0d0 msgr2=0x7f8ef40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef406c0d0 0x7f8ef40ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 -- 172.21.15.45:0/847067628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40cd380 con 0x7f8ef40aea50 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef9600640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef406c0d0 0x7f8ef40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efa000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef415a0e0 0x7f8ef4142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef8c00640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 0x7f8ef415b170 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c480 tx=0x7f8ee000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0012830 con 0x7f8ef40aea50 2024-04-25T10:21:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee0017070 con 0x7f8ef40aea50 2024-04-25T10:21:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0012c30 con 0x7f8ef40aea50 2024-04-25T10:21:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef40cd660 con 0x7f8ef40aea50 2024-04-25T10:21:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef40cdc20 con 0x7f8ef40aea50 2024-04-25T10:21:34.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef404ef30 con 0x7f8ef40aea50 2024-04-25T10:21:34.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.327+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ee001c460 con 0x7f8ef40aea50 2024-04-25T10:21:34.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.331+0000 7f8ef1600640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed4066f60 0x7f8ed4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.331+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ee0084bf0 con 0x7f8ef40aea50 2024-04-25T10:21:34.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.331+0000 7f8ef9600640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed4066f60 0x7f8ed4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.331+0000 7f8ef9600640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed4066f60 0x7f8ed4069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80ad000 tx=0x7f8ee8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.335+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ee0055ef0 con 0x7f8ef40aea50 2024-04-25T10:21:34.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.495+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8ef406e1f0 con 0x7f8ef40aea50 2024-04-25T10:21:34.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8ef1600640 1 -- 172.21.15.45:0/847067628 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ee00557f0 con 0x7f8ef40aea50 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed4066f60 msgr2=0x7f8ed4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed4066f60 0x7f8ed4069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80ad000 tx=0x7f8ee8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 msgr2=0x7f8ef415b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 0x7f8ef415b170 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c480 tx=0x7f8ee000c940 comp rx=0 tx=0).stop 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 shutdown_connections 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed4066f60 0x7f8ed4069420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef415a0e0 0x7f8ef4142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40aea50 0x7f8ef415b170 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 --2- 172.21.15.45:0/847067628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef406c0d0 0x7f8ef40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 >> 172.21.15.45:0/847067628 conn(0x7f8ef40669a0 msgr2=0x7f8ef4066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 shutdown_connections 2024-04-25T10:21:34.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.499+0000 7f8efc000640 1 -- 172.21.15.45:0/847067628 wait complete. 2024-04-25T10:21:34.524 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:21:34.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:34.602 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:21:34.697 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:21:34.697 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:34.698 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:34.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:21:34.707 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:21:34.707 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:34.715 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:34.716 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:34.716 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:34.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:21:34.756 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:21:34.756 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:34.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:21:34.766 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:21:34.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:34.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 Processor -- start 2024-04-25T10:21:34.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 -- start start 2024-04-25T10:21:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3000cbc80 0x7fe30006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3000cc2c0 con 0x7fe3000cf310 2024-04-25T10:21:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3000cf770 con 0x7fe3000cbc80 2024-04-25T10:21:34.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe306400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3000cc5a0 con 0x7fe30015a9f0 2024-04-25T10:21:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ff400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ffe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3000cbc80 0x7fe30006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2fea00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe3000cf310 0x7fe3000cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:57458/0 (socket says 172.21.15.45:57458) 2024-04-25T10:21:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2fea00640 1 -- 172.21.15.45:0/1828126098 learned_addr learned my addr 172.21.15.45:0/1828126098 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:34.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ff400640 1 -- 172.21.15.45:0/1828126098 >> v2:172.21.15.117:3300/0 conn(0x7fe3000cbc80 msgr2=0x7fe30006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ff400640 1 --2- 172.21.15.45:0/1828126098 >> v2:172.21.15.117:3300/0 conn(0x7fe3000cbc80 0x7fe30006d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ff400640 1 -- 172.21.15.45:0/1828126098 >> v1:172.21.15.117:6789/0 conn(0x7fe3000cf310 legacy=0x7fe3000cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ff400640 1 -- 172.21.15.45:0/1828126098 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000cc880 con 0x7fe30015a9f0 2024-04-25T10:21:34.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.939+0000 7fe2ffe00640 1 --2- 172.21.15.45:0/1828126098 >> v2:172.21.15.117:3300/0 conn(0x7fe3000cbc80 0x7fe30006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:34.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe2ff400640 1 --2- 172.21.15.45:0/1828126098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000cedd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00097c0 tx=0x7fe2f001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e00a9c56334d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe2fe000640 1 -- 172.21.15.45:0/1828126098 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00b2070 con 0x7fe30015a9f0 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe2fe000640 1 -- 172.21.15.45:0/1828126098 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f00ad640 con 0x7fe30015a9f0 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe2fe000640 1 -- 172.21.15.45:0/1828126098 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00ada10 con 0x7fe30015a9f0 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 -- 172.21.15.45:0/1828126098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 msgr2=0x7fe3000cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 --2- 172.21.15.45:0/1828126098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000cedd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00097c0 tx=0x7fe2f001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 -- 172.21.15.45:0/1828126098 shutdown_connections 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 --2- 172.21.15.45:0/1828126098 >> v2:172.21.15.117:3300/0 conn(0x7fe3000cbc80 0x7fe30006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 --2- 172.21.15.45:0/1828126098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000cedd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 -- 172.21.15.45:0/1828126098 >> 172.21.15.45:0/1828126098 conn(0x7fe3000b2410 msgr2=0x7fe3000b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:34.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 -- 172.21.15.45:0/1828126098 shutdown_connections 2024-04-25T10:21:34.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.943+0000 7fe306400640 1 -- 172.21.15.45:0/1828126098 wait complete. 2024-04-25T10:21:34.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 Processor -- start 2024-04-25T10:21:34.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- start start 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3000cbc80 0x7fe3000b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 0x7fe30013f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30006e330 con 0x7fe3000cf310 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30006e1b0 con 0x7fe3000cbc80 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30006e4b0 con 0x7fe30015a9f0 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 0x7fe30013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2ff400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3000cbc80 0x7fe3000b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 0x7fe30013f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52128/0 (socket says 172.21.15.45:52128) 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 -- 172.21.15.45:0/2058060415 learned_addr learned my addr 172.21.15.45:0/2058060415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:34.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2ffe00640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 -- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 msgr2=0x7fe3000b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 -- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3000cbc80 msgr2=0x7fe3000b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3000cbc80 0x7fe3000b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 -- 172.21.15.45:0/2058060415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000b3e10 con 0x7fe3000cf310 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2fea00640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 0x7fe30013f420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00c420 tx=0x7fe2ec00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec017070 con 0x7fe3000cf310 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec005cf0 con 0x7fe3000cf310 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0072f0 con 0x7fe3000cf310 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3000b40f0 con 0x7fe3000cf310 2024-04-25T10:21:34.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3000b46b0 con 0x7fe3000cf310 2024-04-25T10:21:34.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.947+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30004ef30 con 0x7fe3000cf310 2024-04-25T10:21:34.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.951+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2ec01a070 con 0x7fe3000cf310 2024-04-25T10:21:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.951+0000 7fe2f7400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2e0066f60 0x7fe2e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.951+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2ec083ff0 con 0x7fe3000cf310 2024-04-25T10:21:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.951+0000 7fe2ff400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2e0066f60 0x7fe2e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.951+0000 7fe2ff400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2e0066f60 0x7fe2e0069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00af000 tx=0x7fe2f00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:34.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:34.955+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2ec054ce0 con 0x7fe3000cf310 2024-04-25T10:21:35.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3000cb780 con 0x7fe3000cf310 2024-04-25T10:21:35.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe2f7400640 1 -- 172.21.15.45:0/2058060415 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fe2ec058d80 con 0x7fe3000cf310 2024-04-25T10:21:35.157 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:35.157 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2026530730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2026530730},{"type":"v1","addr":"172.21.15.117:6837","nonce":2026530730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2816002686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2816002686},{"type":"v1","addr":"172.21.15.45:6834","nonce":2816002686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5571,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2577642053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2577642053},{"type":"v1","addr":"172.21.15.45:6835","nonce":2577642053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5636,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2688611015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2688611015},{"type":"v1","addr":"172.21.15.117:6836","nonce":2688611015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[]} 2024-04-25T10:21:35.158 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 33 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2e0066f60 msgr2=0x7fe2e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2e0066f60 0x7fe2e0069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00af000 tx=0x7fe2f00a7070 comp rx=0 tx=0).stop 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 msgr2=0x7fe30013f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 0x7fe30013f420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00c420 tx=0x7fe2ec00c820 comp rx=0 tx=0).stop 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 shutdown_connections 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2e0066f60 0x7fe2e0069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30015a9f0 0x7fe3000b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3000cf310 0x7fe30013f420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 --2- 172.21.15.45:0/2058060415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3000cbc80 0x7fe3000b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 >> 172.21.15.45:0/2058060415 conn(0x7fe3000b2410 msgr2=0x7fe3000c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 shutdown_connections 2024-04-25T10:21:35.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.155+0000 7fe306400640 1 -- 172.21.15.45:0/2058060415 wait complete. 2024-04-25T10:21:35.180 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:21:35.180 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:21:35.180 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:21:35.180 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:35.180 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:21:35.180 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:21:35.181 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:21:35.181 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:35.181 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:21:35.181 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:21:35.181 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:21:35.181 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:35.181 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:21:35.181 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:21:35.182 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:21:35.182 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:35.193 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:21:35.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:21:35.193 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:21:35.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:21:35.195 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:21:35.196 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:21:35.197 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:21:35.197 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:21:35.243 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:21:35.243 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:35.244 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:21:35.244 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:35.245 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:21:35.245 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:35.247 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:21:35.247 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:35.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:21:35.333 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:21:35.333 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:21:35.336 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:21:35.337 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:21:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 Processor -- start 2024-04-25T10:21:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 -- start start 2024-04-25T10:21:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6d0015bc40 0x7f6d0006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d000cda10 0x7f6d000b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d000b2770 con 0x7f6d000cf930 2024-04-25T10:21:35.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0006c670 con 0x7f6d0015bc40 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d08600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0006c950 con 0x7f6d000cda10 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d000cda10 0x7f6d000b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6d0015bc40 0x7f6d0006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d06600640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6d000cf930 0x7f6d0015f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:57462/0 (socket says 172.21.15.45:57462) 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d000cda10 0x7f6d000b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:59410/0 (socket says 172.21.15.45:59410) 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d06600640 1 -- 172.21.15.45:0/499401153 learned_addr learned my addr 172.21.15.45:0/499401153 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05c00640 1 -- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d000cda10 msgr2=0x7f6d000b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05c00640 1 --2- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d000cda10 0x7f6d000b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05c00640 1 -- 172.21.15.45:0/499401153 >> v1:172.21.15.117:6789/0 conn(0x7f6d000cf930 legacy=0x7f6d0015f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6d05c00640 1 -- 172.21.15.45:0/499401153 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0006cc30 con 0x7f6d0015bc40 2024-04-25T10:21:35.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.391+0000 7f6cffe00640 1 -- 172.21.15.45:0/499401153 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269903100 0 0) 0x7f6d000b2770 con 0x7f6d000cf930 2024-04-25T10:21:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d05c00640 1 --2- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d0015bc40 0x7f6d0006c090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40097c0 tx=0x7f6cf401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53fa487a5970ffa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6cffe00640 1 -- 172.21.15.45:0/499401153 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40b1070 con 0x7f6d0015bc40 2024-04-25T10:21:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6cffe00640 1 -- 172.21.15.45:0/499401153 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40ac410 con 0x7f6d0015bc40 2024-04-25T10:21:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6cffe00640 1 -- 172.21.15.45:0/499401153 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40ac7e0 con 0x7f6d0015bc40 2024-04-25T10:21:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 -- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d0015bc40 msgr2=0x7f6d0006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 --2- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d0015bc40 0x7f6d0006c090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40097c0 tx=0x7f6cf401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 -- 172.21.15.45:0/499401153 shutdown_connections 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 --2- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d000cda10 0x7f6d000b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 --2- 172.21.15.45:0/499401153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d0015bc40 0x7f6d0006c090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 -- 172.21.15.45:0/499401153 >> 172.21.15.45:0/499401153 conn(0x7f6d00066b70 msgr2=0x7f6d00066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 -- 172.21.15.45:0/499401153 shutdown_connections 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.395+0000 7f6d08600640 1 -- 172.21.15.45:0/499401153 wait complete. 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 Processor -- start 2024-04-25T10:21:35.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 -- start start 2024-04-25T10:21:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 0x7f6d00147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d000cf930 0x7f6d00070520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d0015bc40 0x7f6d000ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d00168b50 con 0x7f6d000cda10 2024-04-25T10:21:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001689d0 con 0x7f6d000cf930 2024-04-25T10:21:35.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d00168cd0 con 0x7f6d0015bc40 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 0x7f6d00147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 0x7f6d00147200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52172/0 (socket says 172.21.15.45:52172) 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 -- 172.21.15.45:0/1144420346 learned_addr learned my addr 172.21.15.45:0/1144420346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d06600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d0015bc40 0x7f6d000ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05200640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d000cf930 0x7f6d00070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 -- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d0015bc40 msgr2=0x7f6d000ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d0015bc40 0x7f6d000ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 -- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d000cf930 msgr2=0x7f6d00070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d000cf930 0x7f6d00070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 -- 172.21.15.45:0/1144420346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00147740 con 0x7f6d000cda10 2024-04-25T10:21:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d05c00640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 0x7f6d00147200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6cf401a0d0 tx=0x7f6cf4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40b1040 con 0x7f6d000cda10 2024-04-25T10:21:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40bc070 con 0x7f6d000cda10 2024-04-25T10:21:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d00070fa0 con 0x7f6d000cda10 2024-04-25T10:21:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40b5410 con 0x7f6d000cda10 2024-04-25T10:21:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.399+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0015e600 con 0x7f6d000cda10 2024-04-25T10:21:35.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.403+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0004ef30 con 0x7f6d000cda10 2024-04-25T10:21:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.403+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cf40b55b0 con 0x7f6d000cda10 2024-04-25T10:21:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.403+0000 7f6cfd600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ce0066f60 0x7f6ce0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:35.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.403+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6cf411e720 con 0x7f6d000cda10 2024-04-25T10:21:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.403+0000 7f6d05200640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ce0066f60 0x7f6ce0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:35.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.403+0000 7f6d05200640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ce0066f60 0x7f6ce0069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6d000ae3d0 tx=0x7f6cec009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.407+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cf4016090 con 0x7f6d000cda10 2024-04-25T10:21:35.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:35.587+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6d0006d960 con 0x7f6d000cda10 2024-04-25T10:21:36.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6cfd600640 1 -- 172.21.15.45:0/1144420346 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6cf40ef340 con 0x7f6d000cda10 2024-04-25T10:21:36.529 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:21:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ce0066f60 msgr2=0x7f6ce0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ce0066f60 0x7f6ce0069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6d000ae3d0 tx=0x7f6cec009190 comp rx=0 tx=0).stop 2024-04-25T10:21:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 msgr2=0x7f6d00147200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 0x7f6d00147200 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6cf401a0d0 tx=0x7f6cf4019240 comp rx=0 tx=0).stop 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 shutdown_connections 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ce0066f60 0x7f6ce0069420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d0015bc40 0x7f6d000ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d000cf930 0x7f6d00070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 --2- 172.21.15.45:0/1144420346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d000cda10 0x7f6d00147200 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 >> 172.21.15.45:0/1144420346 conn(0x7f6d00066b70 msgr2=0x7f6d000cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.527+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 shutdown_connections 2024-04-25T10:21:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.531+0000 7f6d08600640 1 -- 172.21.15.45:0/1144420346 wait complete. 2024-04-25T10:21:36.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:21:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 Processor -- start 2024-04-25T10:21:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 -- start start 2024-04-25T10:21:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f333415a780 0x7f333415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33340cff00 0x7f33340b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333406de60 con 0x7f33340cf3b0 2024-04-25T10:21:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333415f2f0 con 0x7f333415a780 2024-04-25T10:21:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3333200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333406cdc0 con 0x7f33340cff00 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f333415a780 0x7f333415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f333415a780 0x7f333415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52180/0 (socket says 172.21.15.45:52180) 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 -- 172.21.15.45:0/396353582 learned_addr learned my addr 172.21.15.45:0/396353582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3332800640 1 --2- 172.21.15.45:0/396353582 >> v2:172.21.15.117:3300/0 conn(0x7f33340cff00 0x7f33340b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 -- 172.21.15.45:0/396353582 >> v2:172.21.15.117:3300/0 conn(0x7f33340cff00 msgr2=0x7f33340b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 --2- 172.21.15.45:0/396353582 >> v2:172.21.15.117:3300/0 conn(0x7f33340cff00 0x7f33340b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 -- 172.21.15.45:0/396353582 >> v1:172.21.15.117:6789/0 conn(0x7f33340cf3b0 legacy=0x7f33340cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.699+0000 7f3331e00640 1 -- 172.21.15.45:0/396353582 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333406d0a0 con 0x7f333415a780 2024-04-25T10:21:36.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3331e00640 1 --2- 172.21.15.45:0/396353582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333415a780 0x7f333415edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f33340cc220 tx=0x7f332401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43be1e9d5ea07c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:36.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3330a00640 1 -- 172.21.15.45:0/396353582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240b0070 con 0x7f333415a780 2024-04-25T10:21:36.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3330a00640 1 -- 172.21.15.45:0/396353582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33240ab600 con 0x7f333415a780 2024-04-25T10:21:36.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3330a00640 1 -- 172.21.15.45:0/396353582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240ab9d0 con 0x7f333415a780 2024-04-25T10:21:36.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 -- 172.21.15.45:0/396353582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333415a780 msgr2=0x7f333415edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 --2- 172.21.15.45:0/396353582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333415a780 0x7f333415edb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f33340cc220 tx=0x7f332401c520 comp rx=0 tx=0).stop 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 -- 172.21.15.45:0/396353582 shutdown_connections 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 --2- 172.21.15.45:0/396353582 >> v2:172.21.15.117:3300/0 conn(0x7f33340cff00 0x7f33340b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 --2- 172.21.15.45:0/396353582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f333415a780 0x7f333415edb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 -- 172.21.15.45:0/396353582 >> 172.21.15.45:0/396353582 conn(0x7f33340743f0 msgr2=0x7f33340669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 -- 172.21.15.45:0/396353582 shutdown_connections 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 -- 172.21.15.45:0/396353582 wait complete. 2024-04-25T10:21:36.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 Processor -- start 2024-04-25T10:21:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.703+0000 7f3333200640 1 -- start start 2024-04-25T10:21:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 0x7f3334139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340cff00 0x7f333414d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333415a780 0x7f333414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b3330 con 0x7f33340cf3b0 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b31b0 con 0x7f33340cff00 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b34b0 con 0x7f333415a780 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 0x7f3334139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 0x7f3334139ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52194/0 (socket says 172.21.15.45:52194) 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 -- 172.21.15.45:0/1018363162 learned_addr learned my addr 172.21.15.45:0/1018363162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331400640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340cff00 0x7f333414d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3332800640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333415a780 0x7f333414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 -- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333415a780 msgr2=0x7f333414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333415a780 0x7f333414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 -- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340cff00 msgr2=0x7f333414d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340cff00 0x7f333414d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 -- 172.21.15.45:0/1018363162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333415bb40 con 0x7f33340cf3b0 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331400640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340cff00 0x7f333414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3331e00640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 0x7f3334139ce0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f33240ae040 tx=0x7f33240192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240b0040 con 0x7f33340cf3b0 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33240b4410 con 0x7f33340cf3b0 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240b4820 con 0x7f33340cf3b0 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333415be20 con 0x7f33340cf3b0 2024-04-25T10:21:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333415c3e0 con 0x7f33340cf3b0 2024-04-25T10:21:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3334075120 con 0x7f33340cf3b0 2024-04-25T10:21:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.707+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33240bc020 con 0x7f33340cf3b0 2024-04-25T10:21:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.711+0000 7f332a000640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3314066f90 0x7f3314069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.711+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f332411dfc0 con 0x7f33340cf3b0 2024-04-25T10:21:36.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.711+0000 7f3331400640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3314066f90 0x7f3314069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:36.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.711+0000 7f3331400640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3314066f90 0x7f3314069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f33340b0c40 tx=0x7f3320009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:36.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.715+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33240c3020 con 0x7f33340cf3b0 2024-04-25T10:21:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.931+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f333406c8b0 con 0x7f33340cf3b0 2024-04-25T10:21:36.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.931+0000 7f332a000640 1 -- 172.21.15.45:0/1018363162 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3324016110 con 0x7f33340cf3b0 2024-04-25T10:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.935 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:21:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:21:36.937 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.938 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.939 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.940 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.941 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.943 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:21:36.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:21:36.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.947 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:21:36.949 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:21:36.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.951 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.953 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:36.954 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:21:36.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:21:36.956 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3314066f90 msgr2=0x7f3314069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3314066f90 0x7f3314069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f33340b0c40 tx=0x7f3320009040 comp rx=0 tx=0).stop 2024-04-25T10:21:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 msgr2=0x7f3334139ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 0x7f3334139ce0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f33240ae040 tx=0x7f33240192c0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 shutdown_connections 2024-04-25T10:21:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3314066f90 0x7f3314069450 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f333415a780 0x7f333414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33340cff00 0x7f333414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 --2- 172.21.15.45:0/1018363162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33340cf3b0 0x7f3334139ce0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 >> 172.21.15.45:0/1018363162 conn(0x7f33340743f0 msgr2=0x7f33340712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 shutdown_connections 2024-04-25T10:21:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:36.935+0000 7f3333200640 1 -- 172.21.15.45:0/1018363162 wait complete. 2024-04-25T10:21:36.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.099+0000 7fe394800640 1 Processor -- start 2024-04-25T10:21:37.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.099+0000 7fe394800640 1 -- start start 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe394800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe38c1550b0 0x7fe38c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe394800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe38c0cfe70 0x7fe38c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe394800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c15b740 0x7fe38c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe394800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38c071050 con 0x7fe38c1550b0 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe394800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38c0712f0 con 0x7fe38c0cfe70 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe394800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0715d0 con 0x7fe38c15b740 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe38c1550b0 0x7fe38c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe38c1550b0 0x7fe38c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52196/0 (socket says 172.21.15.45:52196) 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 -- 172.21.15.45:0/1039122467 learned_addr learned my addr 172.21.15.45:0/1039122467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:37.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391400640 1 --2- 172.21.15.45:0/1039122467 >> v2:172.21.15.117:3300/0 conn(0x7fe38c0cfe70 0x7fe38c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe392800640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c15b740 0x7fe38c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 -- 172.21.15.45:0/1039122467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c15b740 msgr2=0x7fe38c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c15b740 0x7fe38c06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 -- 172.21.15.45:0/1039122467 >> v2:172.21.15.117:3300/0 conn(0x7fe38c0cfe70 msgr2=0x7fe38c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 --2- 172.21.15.45:0/1039122467 >> v2:172.21.15.117:3300/0 conn(0x7fe38c0cfe70 0x7fe38c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 -- 172.21.15.45:0/1039122467 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0718b0 con 0x7fe38c1550b0 2024-04-25T10:21:37.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe392800640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c15b740 0x7fe38c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.103+0000 7fe391e00640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c1550b0 0x7fe38c0cf930 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe380009870 tx=0x7fe38001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9382b4f106620384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe390a00640 1 -- 172.21.15.45:0/1039122467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3800b0070 con 0x7fe38c1550b0 2024-04-25T10:21:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe390a00640 1 -- 172.21.15.45:0/1039122467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3800ab600 con 0x7fe38c1550b0 2024-04-25T10:21:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe390a00640 1 -- 172.21.15.45:0/1039122467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3800ab9d0 con 0x7fe38c1550b0 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- 172.21.15.45:0/1039122467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c1550b0 msgr2=0x7fe38c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c1550b0 0x7fe38c0cf930 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe380009870 tx=0x7fe38001c260 comp rx=0 tx=0).stop 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- 172.21.15.45:0/1039122467 shutdown_connections 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c15b740 0x7fe38c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- 172.21.15.45:0/1039122467 >> v2:172.21.15.117:3300/0 conn(0x7fe38c0cfe70 0x7fe38c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- 172.21.15.45:0/1039122467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c1550b0 0x7fe38c0cf930 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- 172.21.15.45:0/1039122467 >> 172.21.15.45:0/1039122467 conn(0x7fe38c066b70 msgr2=0x7fe38c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- 172.21.15.45:0/1039122467 shutdown_connections 2024-04-25T10:21:37.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- 172.21.15.45:0/1039122467 wait complete. 2024-04-25T10:21:37.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 Processor -- start 2024-04-25T10:21:37.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- start start 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c0cfe70 0x7fe38c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38c1550b0 0x7fe38c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 0x7fe38c148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0b3260 con 0x7fe38c15b740 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c15bd60 con 0x7fe38c1550b0 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe394800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0b33e0 con 0x7fe38c0cfe70 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.107+0000 7fe392800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 0x7fe38c148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 0x7fe38c148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52202/0 (socket says 172.21.15.45:52202) 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 -- 172.21.15.45:0/1930927693 learned_addr learned my addr 172.21.15.45:0/1930927693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:37.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe391400640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38c1550b0 0x7fe38c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe391e00640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c0cfe70 0x7fe38c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 -- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c0cfe70 msgr2=0x7fe38c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c0cfe70 0x7fe38c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 -- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38c1550b0 msgr2=0x7fe38c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38c1550b0 0x7fe38c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 -- 172.21.15.45:0/1930927693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0cd740 con 0x7fe38c15b740 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe391e00640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c0cfe70 0x7fe38c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe392800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 0x7fe38c148d70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe388007ea0 tx=0x7fe388007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38801a070 con 0x7fe38c15b740 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388003bc0 con 0x7fe38c15b740 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38800de70 con 0x7fe38c15b740 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c0cd9e0 con 0x7fe38c15b740 2024-04-25T10:21:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c0cdfa0 con 0x7fe38c15b740 2024-04-25T10:21:37.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c04ef30 con 0x7fe38c15b740 2024-04-25T10:21:37.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe38801d070 con 0x7fe38c15b740 2024-04-25T10:21:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe386000640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe370066f60 0x7fe370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.111+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe388084f60 con 0x7fe38c15b740 2024-04-25T10:21:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.115+0000 7fe391e00640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe370066f60 0x7fe370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.115+0000 7fe391e00640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe370066f60 0x7fe370069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3800ad000 tx=0x7fe3800a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.115+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe388010070 con 0x7fe38c15b740 2024-04-25T10:21:37.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.279+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe38c0ce170 con 0x7fe38c15b740 2024-04-25T10:21:37.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.279+0000 7fe386000640 1 -- 172.21.15.45:0/1930927693 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+12879 (secure 0 0 0) 0x7fe388055b40 con 0x7fe38c15b740 2024-04-25T10:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:37.283 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":58,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:36.527436+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe370066f60 msgr2=0x7fe370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe370066f60 0x7fe370069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3800ad000 tx=0x7fe3800a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 msgr2=0x7fe38c148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 0x7fe38c148d70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe388007ea0 tx=0x7fe388007f90 comp rx=0 tx=0).stop 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 shutdown_connections 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe370066f60 0x7fe370069420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe38c15b740 0x7fe38c148d70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe38c1550b0 0x7fe38c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 --2- 172.21.15.45:0/1930927693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe38c0cfe70 0x7fe38c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 >> 172.21.15.45:0/1930927693 conn(0x7fe38c066b70 msgr2=0x7fe38c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 shutdown_connections 2024-04-25T10:21:37.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.283+0000 7fe394800640 1 -- 172.21.15.45:0/1930927693 wait complete. 2024-04-25T10:21:37.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:37.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:37.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 Processor -- start 2024-04-25T10:21:37.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 -- start start 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f956015f1b0 0x7f956006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956006ed00 0x7f95600ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f95600ce540 0x7f95600ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956006d950 con 0x7f956015f1b0 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956006dc20 con 0x7f95600ce540 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9565c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956006df00 con 0x7f956006ed00 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f956015f1b0 0x7f956006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f956015f1b0 0x7f956006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52208/0 (socket says 172.21.15.45:52208) 2024-04-25T10:21:37.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 -- 172.21.15.45:0/3624611172 learned_addr learned my addr 172.21.15.45:0/3624611172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9557e00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956006ed00 0x7f95600ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955f400640 1 --2- 172.21.15.45:0/3624611172 >> v2:172.21.15.117:3300/0 conn(0x7f95600ce540 0x7f95600ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 -- 172.21.15.45:0/3624611172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956006ed00 msgr2=0x7f95600ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956006ed00 0x7f95600ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 -- 172.21.15.45:0/3624611172 >> v2:172.21.15.117:3300/0 conn(0x7f95600ce540 msgr2=0x7f95600ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 --2- 172.21.15.45:0/3624611172 >> v2:172.21.15.117:3300/0 conn(0x7f95600ce540 0x7f95600ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f955ea00640 1 -- 172.21.15.45:0/3624611172 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956006e1e0 con 0x7f956015f1b0 2024-04-25T10:21:37.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.447+0000 7f9557e00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956006ed00 0x7f95600ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f955ea00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956015f1b0 0x7f956006e7c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9550009870 tx=0x7f955001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b0f9f39d139cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f955e000640 1 -- 172.21.15.45:0/3624611172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95500b0070 con 0x7f956015f1b0 2024-04-25T10:21:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f955e000640 1 -- 172.21.15.45:0/3624611172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95500ab600 con 0x7f956015f1b0 2024-04-25T10:21:37.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f955e000640 1 -- 172.21.15.45:0/3624611172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95500ab9d0 con 0x7f956015f1b0 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 -- 172.21.15.45:0/3624611172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956015f1b0 msgr2=0x7f956006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956015f1b0 0x7f956006e7c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9550009870 tx=0x7f955001c1d0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 -- 172.21.15.45:0/3624611172 shutdown_connections 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 --2- 172.21.15.45:0/3624611172 >> v2:172.21.15.117:3300/0 conn(0x7f95600ce540 0x7f95600ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956006ed00 0x7f95600ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 --2- 172.21.15.45:0/3624611172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f956015f1b0 0x7f956006e7c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 -- 172.21.15.45:0/3624611172 >> 172.21.15.45:0/3624611172 conn(0x7f9560072290 msgr2=0x7f95600726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 -- 172.21.15.45:0/3624611172 shutdown_connections 2024-04-25T10:21:37.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.451+0000 7f9565c00640 1 -- 172.21.15.45:0/3624611172 wait complete. 2024-04-25T10:21:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 Processor -- start 2024-04-25T10:21:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- start start 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956006ed00 0x7f95600998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 0x7f95600c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956015f1b0 0x7f95600c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cb150 con 0x7f95600ce540 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cafd0 con 0x7f956006ed00 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cb2d0 con 0x7f956015f1b0 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 0x7f95600c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 0x7f95600c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52222/0 (socket says 172.21.15.45:52222) 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 -- 172.21.15.45:0/395494443 learned_addr learned my addr 172.21.15.45:0/395494443 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f955ea00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956006ed00 0x7f95600998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f955f400640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956015f1b0 0x7f95600c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 -- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956015f1b0 msgr2=0x7f95600c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956015f1b0 0x7f95600c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 -- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956006ed00 msgr2=0x7f95600998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956006ed00 0x7f95600998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 -- 172.21.15.45:0/395494443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95601f4e40 con 0x7f95600ce540 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f955ea00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956006ed00 0x7f95600998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9557e00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 0x7f95600c9310 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f954800e840 tx=0x7f954800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954800f040 con 0x7f95600ce540 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9548004030 con 0x7f95600ce540 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548004400 con 0x7f95600ce540 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9560159f90 con 0x7f95600ce540 2024-04-25T10:21:37.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956015a550 con 0x7f95600ce540 2024-04-25T10:21:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.455+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956004ef30 con 0x7f95600ce540 2024-04-25T10:21:37.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.459+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f954800c6d0 con 0x7f95600ce540 2024-04-25T10:21:37.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.459+0000 7f9556a00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f953c066f60 0x7f953c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.459+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95480849e0 con 0x7f95600ce540 2024-04-25T10:21:37.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.459+0000 7f955ea00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f953c066f60 0x7f953c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.459+0000 7f955ea00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f953c066f60 0x7f953c069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95500ad000 tx=0x7f9550008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.463+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9548017020 con 0x7f95600ce540 2024-04-25T10:21:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.623+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95600b61a0 con 0x7f95600ce540 2024-04-25T10:21:37.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.623+0000 7f9556a00640 1 -- 172.21.15.45:0/395494443 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9548055ca0 con 0x7f95600ce540 2024-04-25T10:21:37.628 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f953c066f60 msgr2=0x7f953c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f953c066f60 0x7f953c069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95500ad000 tx=0x7f9550008040 comp rx=0 tx=0).stop 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 msgr2=0x7f95600c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 0x7f95600c9310 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f954800e840 tx=0x7f954800ed00 comp rx=0 tx=0).stop 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 shutdown_connections 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f953c066f60 0x7f953c069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f956015f1b0 0x7f95600c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95600ce540 0x7f95600c9310 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 --2- 172.21.15.45:0/395494443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956006ed00 0x7f95600998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 >> 172.21.15.45:0/395494443 conn(0x7f9560072290 msgr2=0x7f95600c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 shutdown_connections 2024-04-25T10:21:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.627+0000 7f9565c00640 1 -- 172.21.15.45:0/395494443 wait complete. 2024-04-25T10:21:37.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:21:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 Processor -- start 2024-04-25T10:21:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 -- start start 2024-04-25T10:21:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 0x7f1f200b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f200b13e0 0x7f1f200cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200b1870 con 0x7f1f200d1820 2024-04-25T10:21:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200b1af0 con 0x7f1f200b13e0 2024-04-25T10:21:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f25600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200b1dd0 con 0x7f1f2015e050 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 0x7f1f200b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 0x7f1f200b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:59470/0 (socket says 172.21.15.45:59470) 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 -- 172.21.15.45:0/4192791638 learned_addr learned my addr 172.21.15.45:0/4192791638 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f1f400640 1 --2- 172.21.15.45:0/4192791638 >> v2:172.21.15.117:3300/0 conn(0x7f1f200b13e0 0x7f1f200cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 -- 172.21.15.45:0/4192791638 >> v2:172.21.15.117:3300/0 conn(0x7f1f200b13e0 msgr2=0x7f1f200cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 --2- 172.21.15.45:0/4192791638 >> v2:172.21.15.117:3300/0 conn(0x7f1f200b13e0 0x7f1f200cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 -- 172.21.15.45:0/4192791638 >> v1:172.21.15.117:6789/0 conn(0x7f1f200d1820 legacy=0x7f1f200d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.787+0000 7f1f16000640 1 -- 172.21.15.45:0/4192791638 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200b20b0 con 0x7f1f2015e050 2024-04-25T10:21:37.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f16000640 1 --2- 172.21.15.45:0/4192791638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 0x7f1f200b0ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009870 tx=0x7f1f0801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862ad46133115c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f1e000640 1 -- 172.21.15.45:0/4192791638 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080aa430 con 0x7f1f2015e050 2024-04-25T10:21:37.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f1e000640 1 -- 172.21.15.45:0/4192791638 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f080af070 con 0x7f1f2015e050 2024-04-25T10:21:37.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f1e000640 1 -- 172.21.15.45:0/4192791638 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080aa7f0 con 0x7f1f2015e050 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 -- 172.21.15.45:0/4192791638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 msgr2=0x7f1f200b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 --2- 172.21.15.45:0/4192791638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 0x7f1f200b0ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009870 tx=0x7f1f0801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 -- 172.21.15.45:0/4192791638 shutdown_connections 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 --2- 172.21.15.45:0/4192791638 >> v2:172.21.15.117:3300/0 conn(0x7f1f200b13e0 0x7f1f200cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 --2- 172.21.15.45:0/4192791638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015e050 0x7f1f200b0ea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 -- 172.21.15.45:0/4192791638 >> 172.21.15.45:0/4192791638 conn(0x7f1f2009a9d0 msgr2=0x7f1f2009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 -- 172.21.15.45:0/4192791638 shutdown_connections 2024-04-25T10:21:37.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.791+0000 7f1f25600640 1 -- 172.21.15.45:0/4192791638 wait complete. 2024-04-25T10:21:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 Processor -- start 2024-04-25T10:21:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 -- start start 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 0x7f1f20159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200d1820 0x7f1f200c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015e050 0x7f1f2014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200d0d90 con 0x7f1f200b13e0 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200d0c10 con 0x7f1f2015e050 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200d0f10 con 0x7f1f200d1820 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 0x7f1f20159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 0x7f1f20159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52234/0 (socket says 172.21.15.45:52234) 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 -- 172.21.15.45:0/2611281228 learned_addr learned my addr 172.21.15.45:0/2611281228 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f16000640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200d1820 0x7f1f200c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1f400640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015e050 0x7f1f2014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 -- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200d1820 msgr2=0x7f1f200c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200d1820 0x7f1f200c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 -- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015e050 msgr2=0x7f1f2014ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015e050 0x7f1f2014ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 -- 172.21.15.45:0/2611281228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200cdd50 con 0x7f1f200b13e0 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f1ea00640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 0x7f1f20159170 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c760 tx=0x7f1f1000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10016020 con 0x7f1f200b13e0 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10006d60 con 0x7f1f200b13e0 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003bc0 con 0x7f1f200b13e0 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f200ce030 con 0x7f1f200b13e0 2024-04-25T10:21:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.795+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2014fe60 con 0x7f1f200b13e0 2024-04-25T10:21:37.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.799+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2004ef30 con 0x7f1f200b13e0 2024-04-25T10:21:37.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.799+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f1000a850 con 0x7f1f200b13e0 2024-04-25T10:21:37.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.799+0000 7f1f17400640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1efc066f60 0x7f1efc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:37.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.799+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f100848b0 con 0x7f1f200b13e0 2024-04-25T10:21:37.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.799+0000 7f1f16000640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1efc066f60 0x7f1efc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:37.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.799+0000 7f1f16000640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1efc066f60 0x7f1efc069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f080ae000 tx=0x7f1f080a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:37.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.803+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f10020020 con 0x7f1f200b13e0 2024-04-25T10:21:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f200ce200 con 0x7f1f200b13e0 2024-04-25T10:21:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f17400640 1 -- 172.21.15.45:0/2611281228 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1f200b1870 con 0x7f1f200b13e0 2024-04-25T10:21:37.969 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:21:37.970 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:21:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1efc066f60 msgr2=0x7f1efc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1efc066f60 0x7f1efc069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f080ae000 tx=0x7f1f080a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 msgr2=0x7f1f20159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 0x7f1f20159170 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c760 tx=0x7f1f1000cc20 comp rx=0 tx=0).stop 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 shutdown_connections 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1efc066f60 0x7f1efc069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015e050 0x7f1f2014ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200d1820 0x7f1f200c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 --2- 172.21.15.45:0/2611281228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f200b13e0 0x7f1f20159170 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 >> 172.21.15.45:0/2611281228 conn(0x7f1f2009a9d0 msgr2=0x7f1f2006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 shutdown_connections 2024-04-25T10:21:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:37.967+0000 7f1f25600640 1 -- 172.21.15.45:0/2611281228 wait complete. 2024-04-25T10:21:37.991 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:21:37.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:21:38.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.131+0000 7fc4e7600640 1 Processor -- start 2024-04-25T10:21:38.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.131+0000 7fc4e7600640 1 -- start start 2024-04-25T10:21:38.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4e7600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e006ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4e7600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4e00b26b0 0x7fc4e00cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4e7600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e00b2f50 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4e7600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e006bfc0 con 0x7fc4e00b26b0 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4e7600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00d0270 con 0x7fc4e006b6a0 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4e00b26b0 0x7fc4e00cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4e00b26b0 0x7fc4e00cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52242/0 (socket says 172.21.15.45:52242) 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 -- 172.21.15.45:0/404495014 learned_addr learned my addr 172.21.15.45:0/404495014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4e4c00640 1 --2- 172.21.15.45:0/404495014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e006ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 -- 172.21.15.45:0/404495014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 msgr2=0x7fc4e006ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 --2- 172.21.15.45:0/404495014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e006ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 -- 172.21.15.45:0/404495014 >> v1:172.21.15.117:6789/0 conn(0x7fc4e00b3fc0 legacy=0x7fc4e00b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.135+0000 7fc4dfe00640 1 -- 172.21.15.45:0/404495014 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e006b040 con 0x7fc4e00b26b0 2024-04-25T10:21:38.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4dfe00640 1 --2- 172.21.15.45:0/404495014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b26b0 0x7fc4e00cfd30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009870 tx=0x7fc4cc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdb91b5e90aac06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:38.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4df400640 1 -- 172.21.15.45:0/404495014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0b2070 con 0x7fc4e00b26b0 2024-04-25T10:21:38.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4df400640 1 -- 172.21.15.45:0/404495014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc0a9070 con 0x7fc4e00b26b0 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4df400640 1 -- 172.21.15.45:0/404495014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0ad880 con 0x7fc4e00b26b0 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 -- 172.21.15.45:0/404495014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b26b0 msgr2=0x7fc4e00cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 --2- 172.21.15.45:0/404495014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b26b0 0x7fc4e00cfd30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009870 tx=0x7fc4cc01c560 comp rx=0 tx=0).stop 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 -- 172.21.15.45:0/404495014 shutdown_connections 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 --2- 172.21.15.45:0/404495014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b26b0 0x7fc4e00cfd30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 --2- 172.21.15.45:0/404495014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e006ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 -- 172.21.15.45:0/404495014 >> 172.21.15.45:0/404495014 conn(0x7fc4e00b34a0 msgr2=0x7fc4e00b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 -- 172.21.15.45:0/404495014 shutdown_connections 2024-04-25T10:21:38.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 -- 172.21.15.45:0/404495014 wait complete. 2024-04-25T10:21:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 Processor -- start 2024-04-25T10:21:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.139+0000 7fc4e7600640 1 -- start start 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e0144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e00b26b0 0x7fc4e015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 0x7fc4e00b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e015f0c0 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e015ef40 con 0x7fc4e00b26b0 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e016b800 con 0x7fc4e006b6a0 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 0x7fc4e00b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 0x7fc4e00b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52256/0 (socket says 172.21.15.45:52256) 2024-04-25T10:21:38.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 -- 172.21.15.45:0/1195902049 learned_addr learned my addr 172.21.15.45:0/1195902049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4dfe00640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e00b26b0 0x7fc4e015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e4c00640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e0144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 -- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 msgr2=0x7fc4e0144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e0144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 -- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e00b26b0 msgr2=0x7fc4e015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e00b26b0 0x7fc4e015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 -- 172.21.15.45:0/1195902049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00b73f0 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e5600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 0x7fc4e00b6710 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c770 tx=0x7fc4d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8003b80 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d8003d20 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e00b7110 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800aa90 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e0143980 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e004ef30 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.143+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4d800ac30 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.147+0000 7fc4dcc00640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4c0066f90 0x7fc4c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:38.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.147+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4d8084b40 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.147+0000 7fc4e4c00640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4c0066f90 0x7fc4c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:38.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.147+0000 7fc4e4c00640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4c0066f90 0x7fc4c0069450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00b4680 tx=0x7fc4d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:38.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.147+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4e00b2f50 con 0x7fc4e00b3fc0 2024-04-25T10:21:38.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:38.311+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc4e0072850 con 0x7fc4e00b3fc0 2024-04-25T10:21:39.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.163+0000 7fc4dcc00640 1 -- 172.21.15.45:0/1195902049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4d80557b0 con 0x7fc4e00b3fc0 2024-04-25T10:21:39.168 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:21:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4c0066f90 msgr2=0x7fc4c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4c0066f90 0x7fc4c0069450 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00b4680 tx=0x7fc4d4008040 comp rx=0 tx=0).stop 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 msgr2=0x7fc4e00b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 0x7fc4e00b6710 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800c770 tx=0x7fc4d800cc30 comp rx=0 tx=0).stop 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 shutdown_connections 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4c0066f90 0x7fc4c0069450 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e00b3fc0 0x7fc4e00b6710 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4e00b26b0 0x7fc4e015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 --2- 172.21.15.45:0/1195902049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4e006b6a0 0x7fc4e0144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 >> 172.21.15.45:0/1195902049 conn(0x7fc4e00b34a0 msgr2=0x7fc4e006fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 shutdown_connections 2024-04-25T10:21:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.167+0000 7fc4e7600640 1 -- 172.21.15.45:0/1195902049 wait complete. 2024-04-25T10:21:39.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:21:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.347+0000 7f8ad8c00640 1 Processor -- start 2024-04-25T10:21:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.347+0000 7f8ad8c00640 1 -- start start 2024-04-25T10:21:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ad40d1510 0x7f8ad40b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad8c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ad40d18f0 0x7f8ad406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad8c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40d0170 con 0x7f8ad40d18f0 2024-04-25T10:21:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad8c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40d03e0 con 0x7f8ad40d1510 2024-04-25T10:21:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad8c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40d06c0 con 0x7f8ad40cfbb0 2024-04-25T10:21:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ad40d18f0 0x7f8ad406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ad40d18f0 0x7f8ad406bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52272/0 (socket says 172.21.15.45:52272) 2024-04-25T10:21:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 -- 172.21.15.45:0/467699262 learned_addr learned my addr 172.21.15.45:0/467699262 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8acaa00640 1 --2- 172.21.15.45:0/467699262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3400640 1 --2- 172.21.15.45:0/467699262 >> v2:172.21.15.117:3300/0 conn(0x7f8ad40d1510 0x7f8ad40b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 -- 172.21.15.45:0/467699262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 msgr2=0x7f8ad40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 --2- 172.21.15.45:0/467699262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 -- 172.21.15.45:0/467699262 >> v2:172.21.15.117:3300/0 conn(0x7f8ad40d1510 msgr2=0x7f8ad40b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 --2- 172.21.15.45:0/467699262 >> v2:172.21.15.117:3300/0 conn(0x7f8ad40d1510 0x7f8ad40b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.351+0000 7f8ad3e00640 1 -- 172.21.15.45:0/467699262 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad406b5f0 con 0x7f8ad40d18f0 2024-04-25T10:21:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad3e00640 1 --2- 172.21.15.45:0/467699262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad406bb10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4072fd0 tx=0x7f8acc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8d8212e0a0b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad2a00640 1 -- 172.21.15.45:0/467699262 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0a5860 con 0x7f8ad40d18f0 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad2a00640 1 -- 172.21.15.45:0/467699262 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc0a5a00 con 0x7f8ad40d18f0 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad2a00640 1 -- 172.21.15.45:0/467699262 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0a5dd0 con 0x7f8ad40d18f0 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 -- 172.21.15.45:0/467699262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 msgr2=0x7f8ad406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/467699262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad406bb10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4072fd0 tx=0x7f8acc01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 -- 172.21.15.45:0/467699262 shutdown_connections 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/467699262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad406bb10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/467699262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/467699262 >> v2:172.21.15.117:3300/0 conn(0x7f8ad40d1510 0x7f8ad40b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 -- 172.21.15.45:0/467699262 >> 172.21.15.45:0/467699262 conn(0x7f8ad4072820 msgr2=0x7f8ad409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 -- 172.21.15.45:0/467699262 shutdown_connections 2024-04-25T10:21:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 -- 172.21.15.45:0/467699262 wait complete. 2024-04-25T10:21:39.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 Processor -- start 2024-04-25T10:21:39.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.355+0000 7f8ad8c00640 1 -- start start 2024-04-25T10:21:39.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad4157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad40d1510 0x7f8ad40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad40b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40b28c0 con 0x7f8ad40d18f0 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40b2740 con 0x7f8ad40d1510 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40d1d80 con 0x7f8ad40cfbb0 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad40b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad40b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52284/0 (socket says 172.21.15.45:52284) 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 -- 172.21.15.45:0/2190985932 learned_addr learned my addr 172.21.15.45:0/2190985932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3400640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad4157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8acaa00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad40d1510 0x7f8ad40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 -- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 msgr2=0x7f8ad4157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad4157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 -- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad40d1510 msgr2=0x7f8ad40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad40d1510 0x7f8ad40b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 -- 172.21.15.45:0/2190985932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4157590 con 0x7f8ad40d18f0 2024-04-25T10:21:39.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad3e00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad40b6020 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0af040 tx=0x7f8acc019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0b8440 con 0x7f8ad40d18f0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc0b8620 con 0x7f8ad40d18f0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad415dc40 con 0x7f8ad40d18f0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0b8a10 con 0x7f8ad40d18f0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad415e200 con 0x7f8ad40d18f0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.359+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad4075120 con 0x7f8ad40d18f0 2024-04-25T10:21:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.363+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8acc0c1020 con 0x7f8ad40d18f0 2024-04-25T10:21:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.363+0000 7f8acbe00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ab0066f60 0x7f8ab0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.363+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f8acc11f3f0 con 0x7f8ad40d18f0 2024-04-25T10:21:39.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.363+0000 7f8ad3400640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ab0066f60 0x7f8ab0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:39.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.363+0000 7f8ad3400640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ab0066f60 0x7f8ab0069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009b60 tx=0x7f8ac4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:39.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.367+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8acc0165e0 con 0x7f8ad40d18f0 2024-04-25T10:21:39.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:39.527+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8ad415d560 con 0x7f8ad40d18f0 2024-04-25T10:21:40.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.167+0000 7f8acbe00640 1 -- 172.21.15.45:0/2190985932 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8acc0a59d0 con 0x7f8ad40d18f0 2024-04-25T10:21:40.171 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:21:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ab0066f60 msgr2=0x7f8ab0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ab0066f60 0x7f8ab0069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009b60 tx=0x7f8ac4009190 comp rx=0 tx=0).stop 2024-04-25T10:21:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 msgr2=0x7f8ad40b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad40b6020 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0af040 tx=0x7f8acc019760 comp rx=0 tx=0).stop 2024-04-25T10:21:40.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 shutdown_connections 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ab0066f60 0x7f8ab0069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad40d18f0 0x7f8ad40b6020 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ad40d1510 0x7f8ad40b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 --2- 172.21.15.45:0/2190985932 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ad40cfbb0 0x7f8ad4157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 >> 172.21.15.45:0/2190985932 conn(0x7f8ad4072820 msgr2=0x7f8ad409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 shutdown_connections 2024-04-25T10:21:40.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.171+0000 7f8ad8c00640 1 -- 172.21.15.45:0/2190985932 wait complete. 2024-04-25T10:21:40.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:21:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 Processor -- start 2024-04-25T10:21:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 -- start start 2024-04-25T10:21:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1de415dd30 0x7f1de406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de40d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1de40ce220 0x7f1de40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de415e110 con 0x7f1de415dd30 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de406b1d0 con 0x7f1de40ce220 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dec400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de406b470 con 0x7f1de40cfb30 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1de415dd30 0x7f1de406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1de415dd30 0x7f1de406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52298/0 (socket says 172.21.15.45:52298) 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 -- 172.21.15.45:0/1249215511 learned_addr learned my addr 172.21.15.45:0/1249215511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9000640 1 --2- 172.21.15.45:0/1249215511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de40d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1dea400640 1 --2- 172.21.15.45:0/1249215511 >> v2:172.21.15.117:3300/0 conn(0x7f1de40ce220 0x7f1de40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 -- 172.21.15.45:0/1249215511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 msgr2=0x7f1de40d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 --2- 172.21.15.45:0/1249215511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de40d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 -- 172.21.15.45:0/1249215511 >> v2:172.21.15.117:3300/0 conn(0x7f1de40ce220 msgr2=0x7f1de40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:40.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 --2- 172.21.15.45:0/1249215511 >> v2:172.21.15.117:3300/0 conn(0x7f1de40ce220 0x7f1de40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.351+0000 7f1de9a00640 1 -- 172.21.15.45:0/1249215511 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de406b750 con 0x7f1de415dd30 2024-04-25T10:21:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1de9a00640 1 --2- 172.21.15.45:0/1249215511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de406ac90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1de40c8a20 tx=0x7f1dd801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163b9dbd6ef3e9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1de3e00640 1 -- 172.21.15.45:0/1249215511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd80b1070 con 0x7f1de415dd30 2024-04-25T10:21:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1de3e00640 1 -- 172.21.15.45:0/1249215511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd80ac630 con 0x7f1de415dd30 2024-04-25T10:21:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1de3e00640 1 -- 172.21.15.45:0/1249215511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd80a8050 con 0x7f1de415dd30 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 -- 172.21.15.45:0/1249215511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 msgr2=0x7f1de406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 --2- 172.21.15.45:0/1249215511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de406ac90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1de40c8a20 tx=0x7f1dd801c520 comp rx=0 tx=0).stop 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 -- 172.21.15.45:0/1249215511 shutdown_connections 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 --2- 172.21.15.45:0/1249215511 >> v2:172.21.15.117:3300/0 conn(0x7f1de40ce220 0x7f1de40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 --2- 172.21.15.45:0/1249215511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de40d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 --2- 172.21.15.45:0/1249215511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de406ac90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 -- 172.21.15.45:0/1249215511 >> 172.21.15.45:0/1249215511 conn(0x7f1de409a9d0 msgr2=0x7f1de409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 -- 172.21.15.45:0/1249215511 shutdown_connections 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.355+0000 7f1dec400640 1 -- 172.21.15.45:0/1249215511 wait complete. 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 Processor -- start 2024-04-25T10:21:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 -- start start 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1de40ce220 0x7f1de4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de4151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de4162360 con 0x7f1de415dd30 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de41621e0 con 0x7f1de40ce220 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de41624e0 con 0x7f1de40cfb30 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52308/0 (socket says 172.21.15.45:52308) 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 -- 172.21.15.45:0/218428221 learned_addr learned my addr 172.21.15.45:0/218428221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1de9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1de40ce220 0x7f1de4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1de9000640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de4151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 -- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 msgr2=0x7f1de4151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de4151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 -- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1de40ce220 msgr2=0x7f1de4158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1de40ce220 0x7f1de4158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 -- 172.21.15.45:0/218428221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4157b70 con 0x7f1de415dd30 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dea400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de41568b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009fd0 tx=0x7f1ddc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc014420 con 0x7f1de415dd30 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc019070 con 0x7f1de415dd30 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de4157e50 con 0x7f1de415dd30 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc014800 con 0x7f1de415dd30 2024-04-25T10:21:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.359+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de4158410 con 0x7f1de415dd30 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.363+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de404ef30 con 0x7f1de415dd30 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.363+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ddc0149a0 con 0x7f1de415dd30 2024-04-25T10:21:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.363+0000 7f1de1600640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1dc4066f90 0x7f1dc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.363+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1ddc086130 con 0x7f1de415dd30 2024-04-25T10:21:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.363+0000 7f1de9a00640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1dc4066f90 0x7f1dc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.363+0000 7f1de9a00640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1dc4066f90 0x7f1dc4069450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80ae000 tx=0x7f1dd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:40.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.367+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ddc056bc0 con 0x7f1de415dd30 2024-04-25T10:21:40.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:40.563+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1de4151940 con 0x7f1de415dd30 2024-04-25T10:21:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.191+0000 7f1de1600640 1 -- 172.21.15.45:0/218428221 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v35) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1ddc05ac60 con 0x7f1de415dd30 2024-04-25T10:21:41.194 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T10:21:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.195+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1dc4066f90 msgr2=0x7f1dc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.195+0000 7f1dec400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1dc4066f90 0x7f1dc4069450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80ae000 tx=0x7f1dd8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.195+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 msgr2=0x7f1de41568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.195+0000 7f1dec400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de41568b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc009fd0 tx=0x7f1ddc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 shutdown_connections 2024-04-25T10:21:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1dc4066f90 0x7f1dc4069450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1de415dd30 0x7f1de41568b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1de40cfb30 0x7f1de4151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 --2- 172.21.15.45:0/218428221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1de40ce220 0x7f1de4158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 >> 172.21.15.45:0/218428221 conn(0x7f1de409a9d0 msgr2=0x7f1de40cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 shutdown_connections 2024-04-25T10:21:41.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.199+0000 7f1dec400640 1 -- 172.21.15.45:0/218428221 wait complete. 2024-04-25T10:21:41.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 Processor -- start 2024-04-25T10:21:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 -- start start 2024-04-25T10:21:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01f015b9e0 0x7f01f00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f015bdc0 0x7f01f00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01f00d1520 0x7f01f00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00ccfb0 con 0x7f01f015b9e0 2024-04-25T10:21:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00cd290 con 0x7f01f00d1520 2024-04-25T10:21:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01f4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00cd570 con 0x7f01f015bdc0 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01f015b9e0 0x7f01f00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01f015b9e0 0x7f01f00cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52324/0 (socket says 172.21.15.45:52324) 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 -- 172.21.15.45:0/2367868820 learned_addr learned my addr 172.21.15.45:0/2367868820 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ee800640 1 --2- 172.21.15.45:0/2367868820 >> v2:172.21.15.117:3300/0 conn(0x7f01f00d1520 0x7f01f00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01e5400640 1 --2- 172.21.15.45:0/2367868820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f015bdc0 0x7f01f00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 -- 172.21.15.45:0/2367868820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f015bdc0 msgr2=0x7f01f00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 --2- 172.21.15.45:0/2367868820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f015bdc0 0x7f01f00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 -- 172.21.15.45:0/2367868820 >> v2:172.21.15.117:3300/0 conn(0x7f01f00d1520 msgr2=0x7f01f00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 --2- 172.21.15.45:0/2367868820 >> v2:172.21.15.117:3300/0 conn(0x7f01f00d1520 0x7f01f00d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.379+0000 7f01ede00640 1 -- 172.21.15.45:0/2367868820 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00cd850 con 0x7f01f015b9e0 2024-04-25T10:21:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01ede00640 1 --2- 172.21.15.45:0/2367868820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00cc410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f01f00b3180 tx=0x7f01e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cdbb97367b715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01ed400640 1 -- 172.21.15.45:0/2367868820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00b0070 con 0x7f01f015b9e0 2024-04-25T10:21:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01ed400640 1 -- 172.21.15.45:0/2367868820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e00ab600 con 0x7f01f015b9e0 2024-04-25T10:21:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01ed400640 1 -- 172.21.15.45:0/2367868820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00ab9d0 con 0x7f01f015b9e0 2024-04-25T10:21:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 -- 172.21.15.45:0/2367868820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 msgr2=0x7f01f00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2367868820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00cc410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f01f00b3180 tx=0x7f01e001c520 comp rx=0 tx=0).stop 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 -- 172.21.15.45:0/2367868820 shutdown_connections 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2367868820 >> v2:172.21.15.117:3300/0 conn(0x7f01f00d1520 0x7f01f00d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2367868820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f015bdc0 0x7f01f00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2367868820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00cc410 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 -- 172.21.15.45:0/2367868820 >> 172.21.15.45:0/2367868820 conn(0x7f01f009a9d0 msgr2=0x7f01f009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 -- 172.21.15.45:0/2367868820 shutdown_connections 2024-04-25T10:21:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.383+0000 7f01f4e00640 1 -- 172.21.15.45:0/2367868820 wait complete. 2024-04-25T10:21:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 Processor -- start 2024-04-25T10:21:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 -- start start 2024-04-25T10:21:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f00d1520 0x7f01f00b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f015bdc0 0x7f01f006f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00cebd0 con 0x7f01f015b9e0 2024-04-25T10:21:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00cea50 con 0x7f01f015bdc0 2024-04-25T10:21:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00ced50 con 0x7f01f00d1520 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52330/0 (socket says 172.21.15.45:52330) 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 -- 172.21.15.45:0/2352825858 learned_addr learned my addr 172.21.15.45:0/2352825858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01ede00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f00d1520 0x7f01f00b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01ee800640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f015bdc0 0x7f01f006f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 -- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f00d1520 msgr2=0x7f01f00b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f00d1520 0x7f01f00b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 -- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f015bdc0 msgr2=0x7f01f006f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f015bdc0 0x7f01f006f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 -- 172.21.15.45:0/2352825858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00b6ba0 con 0x7f01f015b9e0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e5400640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00c8f20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d5f0 tx=0x7f01d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80125f0 con 0x7f01f015b9e0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8019070 con 0x7f01f015b9e0 2024-04-25T10:21:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80129d0 con 0x7f01f015b9e0 2024-04-25T10:21:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f00b6e20 con 0x7f01f015b9e0 2024-04-25T10:21:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.387+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f00b73e0 con 0x7f01f015b9e0 2024-04-25T10:21:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.391+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f004ef30 con 0x7f01f015b9e0 2024-04-25T10:21:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.391+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01d8012b70 con 0x7f01f015b9e0 2024-04-25T10:21:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.391+0000 7f01e6a00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01cc066f60 0x7f01cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.391+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01d80853b0 con 0x7f01f015b9e0 2024-04-25T10:21:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.391+0000 7f01ede00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01cc066f60 0x7f01cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.391+0000 7f01ede00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01cc066f60 0x7f01cc069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01e00ad000 tx=0x7f01e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.395+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01f00b6ba0 con 0x7f01f015b9e0 2024-04-25T10:21:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01f00cc850 con 0x7f01f015b9e0 2024-04-25T10:21:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01e6a00640 1 -- 172.21.15.45:0/2352825858 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f01d8055cd0 con 0x7f01f015b9e0 2024-04-25T10:21:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:40.567353+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01cc066f60 msgr2=0x7f01cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01cc066f60 0x7f01cc069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01e00ad000 tx=0x7f01e0008040 comp rx=0 tx=0).stop 2024-04-25T10:21:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 msgr2=0x7f01f00c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00c8f20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d5f0 tx=0x7f01d800dab0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 shutdown_connections 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01cc066f60 0x7f01cc069420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f015bdc0 0x7f01f006f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f015b9e0 0x7f01f00c8f20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 --2- 172.21.15.45:0/2352825858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f00d1520 0x7f01f00b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 >> 172.21.15.45:0/2352825858 conn(0x7f01f009a9d0 msgr2=0x7f01f00cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 shutdown_connections 2024-04-25T10:21:41.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.559+0000 7f01f4e00640 1 -- 172.21.15.45:0/2352825858 wait complete. 2024-04-25T10:21:41.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:41.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.719+0000 7f0e15400640 1 Processor -- start 2024-04-25T10:21:41.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.719+0000 7f0e15400640 1 -- start start 2024-04-25T10:21:41.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e15400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0e1006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e15400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e100b2770 con 0x7f0e1015bc40 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e15400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1006c670 con 0x7f0e100cda10 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e15400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006c950 con 0x7f0e100d0040 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0e1015bc40 0x7f0e100b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49382/0 (socket says 172.21.15.45:49382) 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0e800640 1 -- 172.21.15.45:0/3609959502 learned_addr learned my addr 172.21.15.45:0/3609959502 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0e1006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646126257 0 0) 0x7f0e100b2770 con 0x7f0e1015bc40 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df4003530 con 0x7f0e1015bc40 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15621752 0 0) 0x7f0df4003530 con 0x7f0e1015bc40 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 msgr2=0x7f0e1006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0e1006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 >> v1:172.21.15.117:6789/0 conn(0x7f0e100cda10 legacy=0x7f0e100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1006cc30 con 0x7f0e1015bc40 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f0e00001680 con 0x7f0e1015bc40 2024-04-25T10:21:41.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 >> v1:172.21.15.45:6789/0 conn(0x7f0e1015bc40 legacy=0x7f0e100b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0df40039f0 0x7f0df4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0df4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006c950 con 0x7f0df4004d50 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400bb90 con 0x7f0df40039f0 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400bd90 con 0x7f0e100d0040 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0e800640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0df40039f0 0x7f0df4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e05e00640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0df4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 -- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 msgr2=0x7f0df4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0df4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 -- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0df40039f0 msgr2=0x7f0df4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0df40039f0 0x7f0df4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e0f200640 1 -- 172.21.15.45:0/3609959502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4003530 con 0x7f0df4004d50 2024-04-25T10:21:41.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.723+0000 7f0e05e00640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e100d0040 0x7f0df4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:41.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e0f200640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0e100b4510 tx=0x7f0e0801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71fc1cc2f5c73a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e080b2070 con 0x7f0df4004d50 2024-04-25T10:21:41.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e080a5c30 con 0x7f0df4004d50 2024-04-25T10:21:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e0de00640 1 -- 172.21.15.45:0/3609959502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08009c10 con 0x7f0df4004d50 2024-04-25T10:21:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 -- 172.21.15.45:0/3609959502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0df4004d50 msgr2=0x7f0df40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 --2- 172.21.15.45:0/3609959502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0df4004d50 0x7f0df40051d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0e100b4510 tx=0x7f0e0801cc20 comp rx=0 tx=0).stop 2024-04-25T10:21:41.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e0f200640 1 -- 172.21.15.45:0/3609959502 reap_dead start 2024-04-25T10:21:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 -- 172.21.15.45:0/3609959502 shutdown_connections 2024-04-25T10:21:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 -- 172.21.15.45:0/3609959502 >> 172.21.15.45:0/3609959502 conn(0x7f0e10066b70 msgr2=0x7f0e10066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 -- 172.21.15.45:0/3609959502 shutdown_connections 2024-04-25T10:21:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 -- 172.21.15.45:0/3609959502 wait complete. 2024-04-25T10:21:41.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.727+0000 7f0e15400640 1 Processor -- start 2024-04-25T10:21:41.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- start start 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0df4004d50 0x7f0e1014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e100af3e0 0x7f0e100b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 0x7f0e100717e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006c210 con 0x7f0e10071360 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006c090 con 0x7f0e100af3e0 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006c390 con 0x7f0df4004d50 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 0x7f0e100717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 0x7f0e100717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52362/0 (socket says 172.21.15.45:52362) 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 -- 172.21.15.45:0/661582068 learned_addr learned my addr 172.21.15.45:0/661582068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e05e00640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e100af3e0 0x7f0e100b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0e800640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0df4004d50 0x7f0e1014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 -- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0df4004d50 msgr2=0x7f0e1014ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0df4004d50 0x7f0e1014ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 -- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e100af3e0 msgr2=0x7f0e100b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e100af3e0 0x7f0e100b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 -- 172.21.15.45:0/661582068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100af9a0 con 0x7f0e10071360 2024-04-25T10:21:41.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0e800640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0df4004d50 0x7f0e1014ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:41.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e05e00640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e100af3e0 0x7f0e100b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e0f200640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 0x7f0e100717e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0e08002240 tx=0x7f0e08009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e080b2040 con 0x7f0e10071360 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e080bc070 con 0x7f0e10071360 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00007e60 con 0x7f0e10071360 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100afc80 con 0x7f0e10071360 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e100b0240 con 0x7f0e10071360 2024-04-25T10:21:41.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.731+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1004ef30 con 0x7f0e10071360 2024-04-25T10:21:41.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.735+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e080ae950 con 0x7f0e10071360 2024-04-25T10:21:41.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.735+0000 7f0e07400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0dec066f90 0x7f0dec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:41.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.735+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e08120170 con 0x7f0e10071360 2024-04-25T10:21:41.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.735+0000 7f0e0e800640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0dec066f90 0x7f0dec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:41.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.735+0000 7f0e0e800640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0dec066f90 0x7f0dec069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e00004880 tx=0x7f0e000047f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:41.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.739+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e08016930 con 0x7f0e10071360 2024-04-25T10:21:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.903+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e101578a0 con 0x7f0e10071360 2024-04-25T10:21:41.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.903+0000 7f0e07400640 1 -- 172.21.15.45:0/661582068 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f0e080f1160 con 0x7f0e10071360 2024-04-25T10:21:41.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:41.907 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:40.567353+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:41.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0dec066f90 msgr2=0x7f0dec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0dec066f90 0x7f0dec069450 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e00004880 tx=0x7f0e000047f0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 msgr2=0x7f0e100717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 0x7f0e100717e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0e08002240 tx=0x7f0e08009750 comp rx=0 tx=0).stop 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 shutdown_connections 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0dec066f90 0x7f0dec069450 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e10071360 0x7f0e100717e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e100af3e0 0x7f0e100b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 --2- 172.21.15.45:0/661582068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0df4004d50 0x7f0e1014ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 >> 172.21.15.45:0/661582068 conn(0x7f0e10066b70 msgr2=0x7f0e100b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 shutdown_connections 2024-04-25T10:21:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:41.907+0000 7f0e15400640 1 -- 172.21.15.45:0/661582068 wait complete. 2024-04-25T10:21:41.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:21:42.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 Processor -- start 2024-04-25T10:21:42.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 -- start start 2024-04-25T10:21:42.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6610163140 0x7f66100b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66100cf640 0x7f66100cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6610071d90 con 0x7f66100aeef0 2024-04-25T10:21:42.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66100b2e80 con 0x7f6610163140 2024-04-25T10:21:42.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6615000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100d0010 con 0x7f66100cf640 2024-04-25T10:21:42.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f660e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66100cf640 0x7f66100cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6610163140 0x7f66100b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6610163140 0x7f66100b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40378/0 (socket says 172.21.15.45:40378) 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f660e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66100cf640 0x7f66100cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:59574/0 (socket says 172.21.15.45:59574) 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f660de00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f66100aeef0 0x7f66100b2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:57516/0 (socket says 172.21.15.45:57516) 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 -- 172.21.15.45:0/2902545364 learned_addr learned my addr 172.21.15.45:0/2902545364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 -- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66100cf640 msgr2=0x7f66100cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 --2- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66100cf640 0x7f66100cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 -- 172.21.15.45:0/2902545364 >> v1:172.21.15.117:6789/0 conn(0x7f66100aeef0 legacy=0x7f66100b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.071+0000 7f6607e00640 1 -- 172.21.15.45:0/2902545364 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661006c170 con 0x7f6610163140 2024-04-25T10:21:42.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6607e00640 1 --2- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6610163140 0x7f66100b43a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009870 tx=0x7f65f801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a715eaec84e600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:42.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f660d400640 1 -- 172.21.15.45:0/2902545364 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80b2070 con 0x7f6610163140 2024-04-25T10:21:42.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f660d400640 1 -- 172.21.15.45:0/2902545364 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f80a9070 con 0x7f6610163140 2024-04-25T10:21:42.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f660d400640 1 -- 172.21.15.45:0/2902545364 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80ad880 con 0x7f6610163140 2024-04-25T10:21:42.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 -- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6610163140 msgr2=0x7f66100b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 --2- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6610163140 0x7f66100b43a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009870 tx=0x7f65f801c560 comp rx=0 tx=0).stop 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 -- 172.21.15.45:0/2902545364 shutdown_connections 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 --2- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66100cf640 0x7f66100cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 --2- 172.21.15.45:0/2902545364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6610163140 0x7f66100b43a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 -- 172.21.15.45:0/2902545364 >> 172.21.15.45:0/2902545364 conn(0x7f66100669a0 msgr2=0x7f6610066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 -- 172.21.15.45:0/2902545364 shutdown_connections 2024-04-25T10:21:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.075+0000 7f6615000640 1 -- 172.21.15.45:0/2902545364 wait complete. 2024-04-25T10:21:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 Processor -- start 2024-04-25T10:21:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 -- start start 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 0x7f6610160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66100cf640 0x7f6610143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6610163140 0x7f66101435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661006ce60 con 0x7f66100aeef0 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66101635c0 con 0x7f66100cf640 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661006cfe0 con 0x7f6610163140 2024-04-25T10:21:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 0x7f6610160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 0x7f6610160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52376/0 (socket says 172.21.15.45:52376) 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 -- 172.21.15.45:0/2255438882 learned_addr learned my addr 172.21.15.45:0/2255438882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6607e00640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66100cf640 0x7f6610143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660e800640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6610163140 0x7f66101435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 -- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6610163140 msgr2=0x7f66101435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6610163140 0x7f66101435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 -- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66100cf640 msgr2=0x7f6610143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66100cf640 0x7f6610143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 -- 172.21.15.45:0/2255438882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610160e30 con 0x7f66100aeef0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f660de00640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 0x7f6610160530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f660000c3a0 tx=0x7f660000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66000074e0 con 0x7f66100aeef0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6600007680 con 0x7f66100aeef0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600004a30 con 0x7f66100aeef0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6610161110 con 0x7f66100aeef0 2024-04-25T10:21:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.079+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6610160a70 con 0x7f66100aeef0 2024-04-25T10:21:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.083+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661004ef30 con 0x7f66100aeef0 2024-04-25T10:21:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.083+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f660001a020 con 0x7f66100aeef0 2024-04-25T10:21:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.083+0000 7f6606000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65ec066f60 0x7f65ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.083+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6600084690 con 0x7f66100aeef0 2024-04-25T10:21:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.083+0000 7f6607e00640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65ec066f60 0x7f65ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.083+0000 7f6607e00640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65ec066f60 0x7f65ec069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65f80af000 tx=0x7f65f80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:42.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.087+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6610071d90 con 0x7f66100aeef0 2024-04-25T10:21:42.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:42.291+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f66100b3870 con 0x7f66100aeef0 2024-04-25T10:21:43.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.207+0000 7f6606000640 1 -- 172.21.15.45:0/2255438882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6600055040 con 0x7f66100aeef0 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65ec066f60 msgr2=0x7f65ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65ec066f60 0x7f65ec069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65f80af000 tx=0x7f65f80a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 msgr2=0x7f6610160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 0x7f6610160530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f660000c3a0 tx=0x7f660000c860 comp rx=0 tx=0).stop 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 shutdown_connections 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65ec066f60 0x7f65ec069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6610163140 0x7f66101435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66100cf640 0x7f6610143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 --2- 172.21.15.45:0/2255438882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66100aeef0 0x7f6610160530 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 >> 172.21.15.45:0/2255438882 conn(0x7f66100669a0 msgr2=0x7f6610162610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 shutdown_connections 2024-04-25T10:21:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.211+0000 7f6615000640 1 -- 172.21.15.45:0/2255438882 wait complete. 2024-04-25T10:21:43.237 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:21:43.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:21:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efece800640 1 Processor -- start 2024-04-25T10:21:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efece800640 1 -- start start 2024-04-25T10:21:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efece800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efec80cc200 0x7efec80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efece800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec80cdab0 con 0x7efec806cf40 2024-04-25T10:21:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efece800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec80cd210 con 0x7efec80d0d00 2024-04-25T10:21:43.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efece800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec80cd4f0 con 0x7efec80cc200 2024-04-25T10:21:43.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efec7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efec806cf40 0x7efec815d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49388/0 (socket says 172.21.15.45:49388) 2024-04-25T10:21:43.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efec7e00640 1 -- 172.21.15.45:0/2257804862 learned_addr learned my addr 172.21.15.45:0/2257804862 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:43.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.383+0000 7efebea00640 1 --2- 172.21.15.45:0/2257804862 >> v2:172.21.15.117:3300/0 conn(0x7efec80cc200 0x7efec80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efec6a00640 1 -- 172.21.15.45:0/2257804862 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062413117 0 0) 0x7efec80cdab0 con 0x7efec806cf40 2024-04-25T10:21:43.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efec6a00640 1 -- 172.21.15.45:0/2257804862 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeac003530 con 0x7efec806cf40 2024-04-25T10:21:43.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efebea00640 1 -- 172.21.15.45:0/2257804862 >> v1:172.21.15.117:6789/0 conn(0x7efec80d0d00 legacy=0x7efec8162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efebea00640 1 -- 172.21.15.45:0/2257804862 >> v1:172.21.15.45:6789/0 conn(0x7efec806cf40 legacy=0x7efec815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efebea00640 1 -- 172.21.15.45:0/2257804862 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80cd7d0 con 0x7efec80cc200 2024-04-25T10:21:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efebea00640 1 --2- 172.21.15.45:0/2257804862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec80cc200 0x7efec80ccc30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097c0 tx=0x7efeb001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c02f5035c438c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efec6a00640 1 -- 172.21.15.45:0/2257804862 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00b0070 con 0x7efec80cc200 2024-04-25T10:21:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efec6a00640 1 -- 172.21.15.45:0/2257804862 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb00ab600 con 0x7efec80cc200 2024-04-25T10:21:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.387+0000 7efec6a00640 1 -- 172.21.15.45:0/2257804862 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00ab9d0 con 0x7efec80cc200 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- 172.21.15.45:0/2257804862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec80cc200 msgr2=0x7efec80ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 --2- 172.21.15.45:0/2257804862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec80cc200 0x7efec80ccc30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efeb00097c0 tx=0x7efeb001c410 comp rx=0 tx=0).stop 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- 172.21.15.45:0/2257804862 shutdown_connections 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 --2- 172.21.15.45:0/2257804862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec80cc200 0x7efec80ccc30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- 172.21.15.45:0/2257804862 >> 172.21.15.45:0/2257804862 conn(0x7efec8072290 msgr2=0x7efec80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- 172.21.15.45:0/2257804862 shutdown_connections 2024-04-25T10:21:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- 172.21.15.45:0/2257804862 wait complete. 2024-04-25T10:21:43.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 Processor -- start 2024-04-25T10:21:43.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- start start 2024-04-25T10:21:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec806cf40 0x7efec8099730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efec80cc200 0x7efec80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 0x7efec80c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8162a10 con 0x7efec80d0d00 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec8162890 con 0x7efec806cf40 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efece800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec816c690 con 0x7efec80cc200 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 0x7efec80c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec806cf40 0x7efec8099730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 0x7efec80c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52382/0 (socket says 172.21.15.45:52382) 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec806cf40 0x7efec8099730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40406/0 (socket says 172.21.15.45:40406) 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 -- 172.21.15.45:0/1393650997 learned_addr learned my addr 172.21.15.45:0/1393650997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efebea00640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efec80cc200 0x7efec80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 -- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efec80cc200 msgr2=0x7efec80c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efec80cc200 0x7efec80c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 -- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec806cf40 msgr2=0x7efec8099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:43.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec806cf40 0x7efec8099730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:43.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.391+0000 7efec7e00640 1 -- 172.21.15.45:0/1393650997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80cb630 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efec7e00640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 0x7efec80c7a60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efec000caa0 tx=0x7efec000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0017070 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec000ac00 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0012400 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8157dd0 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec8158390 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec804ef30 con 0x7efec80d0d00 2024-04-25T10:21:43.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efec001a070 con 0x7efec80d0d00 2024-04-25T10:21:43.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efebfe00640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efea4066f90 0x7efea4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efec0084d30 con 0x7efec80d0d00 2024-04-25T10:21:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.395+0000 7efec7400640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efea4066f90 0x7efea4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.399+0000 7efec7400640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efea4066f90 0x7efea4069450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efeb8007fd0 tx=0x7efeb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:43.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.399+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efec0055650 con 0x7efec80d0d00 2024-04-25T10:21:43.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:43.599+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efec8072ff0 con 0x7efec80d0d00 2024-04-25T10:21:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.207+0000 7efebfe00640 1 -- 172.21.15.45:0/1393650997 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7efec00596f0 con 0x7efec80d0d00 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efea4066f90 msgr2=0x7efea4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efea4066f90 0x7efea4069450 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efeb8007fd0 tx=0x7efeb8009040 comp rx=0 tx=0).stop 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 msgr2=0x7efec80c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 0x7efec80c7a60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efec000caa0 tx=0x7efec000cf60 comp rx=0 tx=0).stop 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 shutdown_connections 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efea4066f90 0x7efea4069450 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efec80d0d00 0x7efec80c7a60 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efec80cc200 0x7efec80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 --2- 172.21.15.45:0/1393650997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efec806cf40 0x7efec8099730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 >> 172.21.15.45:0/1393650997 conn(0x7efec8072290 msgr2=0x7efec815e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:44.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 shutdown_connections 2024-04-25T10:21:44.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.211+0000 7efece800640 1 -- 172.21.15.45:0/1393650997 wait complete. 2024-04-25T10:21:44.237 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:21:44.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 Processor -- start 2024-04-25T10:21:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 -- start start 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbeb415bc40 0x7fbeb406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 0x7fbeb40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb40b2770 con 0x7fbeb40cda10 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb406c670 con 0x7fbeb415bc40 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbebc200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb406c950 con 0x7fbeb40cf930 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbeb8e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbeb40cda10 0x7fbeb40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49390/0 (socket says 172.21.15.45:49390) 2024-04-25T10:21:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbeb8e00640 1 -- 172.21.15.45:0/200380496 learned_addr learned my addr 172.21.15.45:0/200380496 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbeba200640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 0x7fbeb40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbeb9800640 1 --2- 172.21.15.45:0/200380496 >> v2:172.21.15.117:3300/0 conn(0x7fbeb415bc40 0x7fbeb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507076337 0 0) 0x7fbeb40b2770 con 0x7fbeb40cda10 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.378+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9c003530 con 0x7fbeb40cda10 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2905547158 0 0) 0x7fbe9c003530 con 0x7fbeb40cda10 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 msgr2=0x7fbeb40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 0x7fbeb40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 >> v2:172.21.15.117:3300/0 conn(0x7fbeb415bc40 msgr2=0x7fbeb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 --2- 172.21.15.45:0/200380496 >> v2:172.21.15.117:3300/0 conn(0x7fbeb415bc40 0x7fbeb406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb406cc30 con 0x7fbeb40cda10 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fbea0001680 con 0x7fbeb40cda10 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 >> v1:172.21.15.45:6789/0 conn(0x7fbeb40cda10 legacy=0x7fbeb40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 0x7fbe9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c004d50 0x7fbe9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb406c670 con 0x7fbe9c004d50 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb406c950 con 0x7fbe9c0039c0 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c00b9d0 con 0x7fbeb40cf930 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c004d50 0x7fbe9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb9800640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb8e00640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 0x7fbe9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 -- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 msgr2=0x7fbe9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:44.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbeb40cf930 0x7fbe9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 -- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe9c0039c0 msgr2=0x7fbe9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:44.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe9c0039c0 0x7fbe9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 -- 172.21.15.45:0/200380496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c003530 con 0x7fbe9c004d50 2024-04-25T10:21:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeba200640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c004d50 0x7fbe9c0051d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009fc0 tx=0x7fbeac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e4d3544c4a6092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0b2070 con 0x7fbe9c004d50 2024-04-25T10:21:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac0a5b80 con 0x7fbe9c004d50 2024-04-25T10:21:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.382+0000 7fbeb3e00640 1 -- 172.21.15.45:0/200380496 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac009c10 con 0x7fbe9c004d50 2024-04-25T10:21:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- 172.21.15.45:0/200380496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c004d50 msgr2=0x7fbe9c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 --2- 172.21.15.45:0/200380496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c004d50 0x7fbe9c0051d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009fc0 tx=0x7fbeac01cc80 comp rx=0 tx=0).stop 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeba200640 1 -- 172.21.15.45:0/200380496 reap_dead start 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- 172.21.15.45:0/200380496 shutdown_connections 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- 172.21.15.45:0/200380496 >> 172.21.15.45:0/200380496 conn(0x7fbeb4066b70 msgr2=0x7fbeb4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- 172.21.15.45:0/200380496 shutdown_connections 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- 172.21.15.45:0/200380496 wait complete. 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 Processor -- start 2024-04-25T10:21:44.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- start start 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe9c004d50 0x7fbeb4149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 0x7fbeb4071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbeb41e5e80 0x7fbeb40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40cff60 con 0x7fbeb414c670 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40cfde0 con 0x7fbeb41e5e80 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbebc200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40d00e0 con 0x7fbe9c004d50 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 0x7fbeb4071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 0x7fbeb4071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52392/0 (socket says 172.21.15.45:52392) 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 -- 172.21.15.45:0/1472373517 learned_addr learned my addr 172.21.15.45:0/1472373517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeba200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbeb41e5e80 0x7fbeb40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb9800640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe9c004d50 0x7fbeb4149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 -- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe9c004d50 msgr2=0x7fbeb4149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe9c004d50 0x7fbeb4149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 -- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbeb41e5e80 msgr2=0x7fbeb40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbeb41e5e80 0x7fbeb40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.386+0000 7fbeb8e00640 1 -- 172.21.15.45:0/1472373517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb41e4de0 con 0x7fbeb414c670 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb8e00640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 0x7fbeb4071730 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbea0011920 tx=0x7fbea0011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0010040 con 0x7fbeb414c670 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea001a070 con 0x7fbeb414c670 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000fd80 con 0x7fbeb414c670 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb41e50c0 con 0x7fbeb414c670 2024-04-25T10:21:44.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb41e5680 con 0x7fbeb414c670 2024-04-25T10:21:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb404ef30 con 0x7fbeb414c670 2024-04-25T10:21:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbea001f460 con 0x7fbeb414c670 2024-04-25T10:21:44.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb1600640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbe94066f90 0x7fbe94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea00882e0 con 0x7fbeb414c670 2024-04-25T10:21:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.390+0000 7fbeb9800640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbe94066f90 0x7fbe94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.394+0000 7fbeb9800640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbe94066f90 0x7fbe94069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009b20 tx=0x7fbea8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.394+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbea0059250 con 0x7fbeb414c670 2024-04-25T10:21:44.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeb40b2770 con 0x7fbeb414c670 2024-04-25T10:21:44.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbeb1600640 1 -- 172.21.15.45:0/1472373517 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fbea0058b50 con 0x7fbeb414c670 2024-04-25T10:21:44.598 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:44.598 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2258543519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2258543519},{"type":"v1","addr":"172.21.15.45:6835","nonce":2258543519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3808318987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3808318987},{"type":"v1","addr":"172.21.15.117:6836","nonce":3808318987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6384,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/317823919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":317823919},{"type":"v1","addr":"172.21.15.45:6834","nonce":317823919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:43.601865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3997755731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3997755731},{"type":"v1","addr":"172.21.15.117:6837","nonce":3997755731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:21:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 39 2024-04-25T10:21:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbe94066f90 msgr2=0x7fbe94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbe94066f90 0x7fbe94069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009b20 tx=0x7fbea8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 msgr2=0x7fbeb4071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 0x7fbeb4071730 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbea0011920 tx=0x7fbea0011de0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 shutdown_connections 2024-04-25T10:21:44.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbe94066f90 0x7fbe94069450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbeb41e5e80 0x7fbeb40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbeb414c670 0x7fbeb4071730 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 --2- 172.21.15.45:0/1472373517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe9c004d50 0x7fbeb4149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 >> 172.21.15.45:0/1472373517 conn(0x7fbeb4066b70 msgr2=0x7fbeb40b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 shutdown_connections 2024-04-25T10:21:44.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.594+0000 7fbebc200640 1 -- 172.21.15.45:0/1472373517 wait complete. 2024-04-25T10:21:44.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:44.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.754+0000 7f3b27a00640 1 Processor -- start 2024-04-25T10:21:44.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.754+0000 7f3b27a00640 1 -- start start 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b27a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b201550b0 0x7f3b200cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b27a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b200cfe70 0x7f3b2015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b27a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b2015b740 0x7f3b2006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b27a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b20071050 con 0x7f3b200cfe70 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b27a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200712f0 con 0x7f3b2015b740 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b27a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200715d0 con 0x7f3b201550b0 2024-04-25T10:21:44.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b200cfe70 0x7f3b2015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b200cfe70 0x7f3b2015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52404/0 (socket says 172.21.15.45:52404) 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b25a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b2015b740 0x7f3b2006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 -- 172.21.15.45:0/3107956838 learned_addr learned my addr 172.21.15.45:0/3107956838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b25000640 1 --2- 172.21.15.45:0/3107956838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b201550b0 0x7f3b200cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 -- 172.21.15.45:0/3107956838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b201550b0 msgr2=0x7f3b200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 --2- 172.21.15.45:0/3107956838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b201550b0 0x7f3b200cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 -- 172.21.15.45:0/3107956838 >> v2:172.21.15.117:3300/0 conn(0x7f3b2015b740 msgr2=0x7f3b2006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 --2- 172.21.15.45:0/3107956838 >> v2:172.21.15.117:3300/0 conn(0x7f3b2015b740 0x7f3b2006d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.758+0000 7f3b17e00640 1 -- 172.21.15.45:0/3107956838 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200718b0 con 0x7f3b200cfe70 2024-04-25T10:21:44.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b17e00640 1 --2- 172.21.15.45:0/3107956838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200cfe70 0x7f3b2015b200 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3b200d04d0 tx=0x7f3b0801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f545bf2516fcfd6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b1fe00640 1 -- 172.21.15.45:0/3107956838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b0070 con 0x7f3b200cfe70 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b1fe00640 1 -- 172.21.15.45:0/3107956838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080ab600 con 0x7f3b200cfe70 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b1fe00640 1 -- 172.21.15.45:0/3107956838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080ab9d0 con 0x7f3b200cfe70 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 -- 172.21.15.45:0/3107956838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200cfe70 msgr2=0x7f3b2015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 --2- 172.21.15.45:0/3107956838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200cfe70 0x7f3b2015b200 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3b200d04d0 tx=0x7f3b0801c260 comp rx=0 tx=0).stop 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 -- 172.21.15.45:0/3107956838 shutdown_connections 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 --2- 172.21.15.45:0/3107956838 >> v2:172.21.15.117:3300/0 conn(0x7f3b2015b740 0x7f3b2006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 --2- 172.21.15.45:0/3107956838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b200cfe70 0x7f3b2015b200 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 --2- 172.21.15.45:0/3107956838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b201550b0 0x7f3b200cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 -- 172.21.15.45:0/3107956838 >> 172.21.15.45:0/3107956838 conn(0x7f3b20066b70 msgr2=0x7f3b20066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 -- 172.21.15.45:0/3107956838 shutdown_connections 2024-04-25T10:21:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 -- 172.21.15.45:0/3107956838 wait complete. 2024-04-25T10:21:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 Processor -- start 2024-04-25T10:21:44.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.762+0000 7f3b27a00640 1 -- start start 2024-04-25T10:21:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b200cfe70 0x7f3b2015d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 0x7f3b2015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b2015b740 0x7f3b20155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b3260 con 0x7f3b201550b0 2024-04-25T10:21:44.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2015bd60 con 0x7f3b200cfe70 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b33e0 con 0x7f3b2015b740 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 0x7f3b2015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 0x7f3b2015c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52408/0 (socket says 172.21.15.45:52408) 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 -- 172.21.15.45:0/787637258 learned_addr learned my addr 172.21.15.45:0/787637258 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b25a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b2015b740 0x7f3b20155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b25000640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b200cfe70 0x7f3b2015d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 -- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b2015b740 msgr2=0x7f3b20155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b2015b740 0x7f3b20155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 -- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b200cfe70 msgr2=0x7f3b2015d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b200cfe70 0x7f3b2015d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 -- 172.21.15.45:0/787637258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2015c780 con 0x7f3b201550b0 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b25000640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b200cfe70 0x7f3b2015d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:44.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b17e00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 0x7f3b2015c010 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3b080ae040 tx=0x7f3b0801aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080ba020 con 0x7f3b201550b0 2024-04-25T10:21:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b080b6420 con 0x7f3b201550b0 2024-04-25T10:21:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b0040 con 0x7f3b201550b0 2024-04-25T10:21:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2015ca60 con 0x7f3b201550b0 2024-04-25T10:21:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2015d020 con 0x7f3b201550b0 2024-04-25T10:21:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2004ef30 con 0x7f3b201550b0 2024-04-25T10:21:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.766+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b080bd020 con 0x7f3b201550b0 2024-04-25T10:21:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.770+0000 7f3b1d600640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.770+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b0811f1e0 con 0x7f3b201550b0 2024-04-25T10:21:44.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.770+0000 7f3b25000640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:44.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.770+0000 7f3b25000640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3afc066f60 0x7f3afc069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009ae0 tx=0x7f3b10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:44.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.774+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b080c4030 con 0x7f3b201550b0 2024-04-25T10:21:44.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.934+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b20068020 con 0x7f3b201550b0 2024-04-25T10:21:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b1d600640 1 -- 172.21.15.45:0/787637258 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f3b08016110 con 0x7f3b201550b0 2024-04-25T10:21:44.941 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:44.941 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:42.194238+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3afc066f60 msgr2=0x7f3afc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3afc066f60 0x7f3afc069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3b10009ae0 tx=0x7f3b10008040 comp rx=0 tx=0).stop 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 msgr2=0x7f3b2015c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 0x7f3b2015c010 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3b080ae040 tx=0x7f3b0801aaa0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 shutdown_connections 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3afc066f60 0x7f3afc069420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b2015b740 0x7f3b20155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b201550b0 0x7f3b2015c010 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 --2- 172.21.15.45:0/787637258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b200cfe70 0x7f3b2015d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 >> 172.21.15.45:0/787637258 conn(0x7f3b20066b70 msgr2=0x7f3b20150020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:44.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 shutdown_connections 2024-04-25T10:21:44.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:44.938+0000 7f3b27a00640 1 -- 172.21.15.45:0/787637258 wait complete. 2024-04-25T10:21:44.962 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:21:44.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:21:45.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.098+0000 7f2f0f000640 1 Processor -- start 2024-04-25T10:21:45.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.098+0000 7f2f0f000640 1 -- start start 2024-04-25T10:21:45.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f0f000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f081550b0 0x7f2f080cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f0f000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f0815b910 0x7f2f080b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f0f000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f08071050 con 0x7f2f080cfe70 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f0f000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0815bd80 con 0x7f2f081550b0 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f0f000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0815d040 con 0x7f2f0815b910 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f07e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f081550b0 0x7f2f080cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2eff400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2f080cfe70 0x7f2f0815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60094/0 (socket says 172.21.15.45:60094) 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f07e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f081550b0 0x7f2f080cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46452/0 (socket says 172.21.15.45:46452) 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2eff400640 1 -- 172.21.15.45:0/1271459894 learned_addr learned my addr 172.21.15.45:0/1271459894 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f0d000640 1 --2- 172.21.15.45:0/1271459894 >> v2:172.21.15.117:3300/0 conn(0x7f2f0815b910 0x7f2f080b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f07e00640 1 -- 172.21.15.45:0/1271459894 >> v2:172.21.15.117:3300/0 conn(0x7f2f0815b910 msgr2=0x7f2f080b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:45.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f07e00640 1 --2- 172.21.15.45:0/1271459894 >> v2:172.21.15.117:3300/0 conn(0x7f2f0815b910 0x7f2f080b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f07e00640 1 -- 172.21.15.45:0/1271459894 >> v1:172.21.15.45:6789/0 conn(0x7f2f080cfe70 legacy=0x7f2f0815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.102+0000 7f2f07e00640 1 -- 172.21.15.45:0/1271459894 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080712f0 con 0x7f2f081550b0 2024-04-25T10:21:45.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f07e00640 1 --2- 172.21.15.45:0/1271459894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f081550b0 0x7f2f080cf930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800ba60 tx=0x7f2ef801eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3ca8b07ddc7ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f07400640 1 -- 172.21.15.45:0/1271459894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b4070 con 0x7f2f081550b0 2024-04-25T10:21:45.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f07400640 1 -- 172.21.15.45:0/1271459894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80a7af0 con 0x7f2f081550b0 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f07400640 1 -- 172.21.15.45:0/1271459894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80af440 con 0x7f2f081550b0 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 -- 172.21.15.45:0/1271459894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f081550b0 msgr2=0x7f2f080cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 --2- 172.21.15.45:0/1271459894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f081550b0 0x7f2f080cf930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800ba60 tx=0x7f2ef801eb30 comp rx=0 tx=0).stop 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 -- 172.21.15.45:0/1271459894 shutdown_connections 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 --2- 172.21.15.45:0/1271459894 >> v2:172.21.15.117:3300/0 conn(0x7f2f0815b910 0x7f2f080b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 --2- 172.21.15.45:0/1271459894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f081550b0 0x7f2f080cf930 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 -- 172.21.15.45:0/1271459894 >> 172.21.15.45:0/1271459894 conn(0x7f2f08066b70 msgr2=0x7f2f08066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 -- 172.21.15.45:0/1271459894 shutdown_connections 2024-04-25T10:21:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 -- 172.21.15.45:0/1271459894 wait complete. 2024-04-25T10:21:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 Processor -- start 2024-04-25T10:21:45.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.106+0000 7f2f0f000640 1 -- start start 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f080cfe70 0x7f2f0813f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f081550b0 0x7f2f08152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 0x7f2f0813c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0815c030 con 0x7f2f0815b910 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080720b0 con 0x7f2f081550b0 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0815c1b0 con 0x7f2f080cfe70 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 0x7f2f0813c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 0x7f2f0813c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46462/0 (socket says 172.21.15.45:46462) 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 -- 172.21.15.45:0/2760500654 learned_addr learned my addr 172.21.15.45:0/2760500654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f07e00640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f080cfe70 0x7f2f0813f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2eff400640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f081550b0 0x7f2f08152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 -- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f080cfe70 msgr2=0x7f2f0813f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f080cfe70 0x7f2f0813f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 -- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f081550b0 msgr2=0x7f2f08152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f081550b0 0x7f2f08152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 -- 172.21.15.45:0/2760500654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0813ce40 con 0x7f2f0815b910 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0d000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 0x7f2f0813c900 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007e60 tx=0x7f2f0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00014420 con 0x7f2f0815b910 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00019070 con 0x7f2f0815b910 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00014820 con 0x7f2f0815b910 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f08153650 con 0x7f2f0815b910 2024-04-25T10:21:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08153c10 con 0x7f2f0815b910 2024-04-25T10:21:45.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0804ef30 con 0x7f2f0815b910 2024-04-25T10:21:45.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.110+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f000149c0 con 0x7f2f0815b910 2024-04-25T10:21:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.114+0000 7f2f04c00640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.114+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f00085970 con 0x7f2f0815b910 2024-04-25T10:21:45.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.114+0000 7f2f07e00640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.114+0000 7f2f07e00640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee4066f60 0x7f2ee4069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80b1000 tx=0x7f2ef8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.118+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f000568e0 con 0x7f2f0815b910 2024-04-25T10:21:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.274+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f080cbd40 con 0x7f2ee4066f60 2024-04-25T10:21:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.282+0000 7f2f04c00640 1 -- 172.21.15.45:0/2760500654 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251958 (secure 0 0 0) 0x7f2f080cbd40 con 0x7f2ee4066f60 2024-04-25T10:21:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:45.291 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee4066f60 msgr2=0x7f2ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee4066f60 0x7f2ee4069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80b1000 tx=0x7f2ef8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 msgr2=0x7f2f0813c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 0x7f2f0813c900 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007e60 tx=0x7f2f0000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 shutdown_connections 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee4066f60 0x7f2ee4069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f0815b910 0x7f2f0813c900 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f081550b0 0x7f2f08152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 --2- 172.21.15.45:0/2760500654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f080cfe70 0x7f2f0813f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 >> 172.21.15.45:0/2760500654 conn(0x7f2f08066b70 msgr2=0x7f2f080b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 shutdown_connections 2024-04-25T10:21:45.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.286+0000 7f2f0f000640 1 -- 172.21.15.45:0/2760500654 wait complete. 2024-04-25T10:21:45.310 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-25T10:21:44.924130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316080,"kb_used_data":6972,"kb_used_omap":12,"kb_used_meta":308979,"kb_avail":754658640,"statfs":{"total":773094113280,"available":772770447360,"internally_reserved":0,"allocated":7139328,"data_stored":3645256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":316395056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.755256"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220875+0000","last_change":"2024-04-25T10:21:41.212340+0000","last_active":"2024-04-25T10:21:42.220875+0000","last_peered":"2024-04-25T10:21:42.220875+0000","last_clean":"2024-04-25T10:21:42.220875+0000","last_became_active":"2024-04-25T10:21:41.211835+0000","last_became_peered":"2024-04-25T10:21:41.211835+0000","last_unstale":"2024-04-25T10:21:42.220875+0000","last_undegraded":"2024-04-25T10:21:42.220875+0000","last_fullsized":"2024-04-25T10:21:42.220875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:05.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220697+0000","last_change":"2024-04-25T10:21:41.208111+0000","last_active":"2024-04-25T10:21:42.220697+0000","last_peered":"2024-04-25T10:21:42.220697+0000","last_clean":"2024-04-25T10:21:42.220697+0000","last_became_active":"2024-04-25T10:21:41.207841+0000","last_became_peered":"2024-04-25T10:21:41.207841+0000","last_unstale":"2024-04-25T10:21:42.220697+0000","last_undegraded":"2024-04-25T10:21:42.220697+0000","last_fullsized":"2024-04-25T10:21:42.220697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:39.616257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202982+0000","last_change":"2024-04-25T10:21:41.206781+0000","last_active":"2024-04-25T10:21:42.202982+0000","last_peered":"2024-04-25T10:21:42.202982+0000","last_clean":"2024-04-25T10:21:42.202982+0000","last_became_active":"2024-04-25T10:21:41.206456+0000","last_became_peered":"2024-04-25T10:21:41.206456+0000","last_unstale":"2024-04-25T10:21:42.202982+0000","last_undegraded":"2024-04-25T10:21:42.202982+0000","last_fullsized":"2024-04-25T10:21:42.202982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:30.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221002+0000","last_change":"2024-04-25T10:21:41.215021+0000","last_active":"2024-04-25T10:21:42.221002+0000","last_peered":"2024-04-25T10:21:42.221002+0000","last_clean":"2024-04-25T10:21:42.221002+0000","last_became_active":"2024-04-25T10:21:41.214730+0000","last_became_peered":"2024-04-25T10:21:41.214730+0000","last_unstale":"2024-04-25T10:21:42.221002+0000","last_undegraded":"2024-04-25T10:21:42.221002+0000","last_fullsized":"2024-04-25T10:21:42.221002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:05.397077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220895+0000","last_change":"2024-04-25T10:21:41.207350+0000","last_active":"2024-04-25T10:21:42.220895+0000","last_peered":"2024-04-25T10:21:42.220895+0000","last_clean":"2024-04-25T10:21:42.220895+0000","last_became_active":"2024-04-25T10:21:41.207048+0000","last_became_peered":"2024-04-25T10:21:41.207048+0000","last_unstale":"2024-04-25T10:21:42.220895+0000","last_undegraded":"2024-04-25T10:21:42.220895+0000","last_fullsized":"2024-04-25T10:21:42.220895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:28.894445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220091+0000","last_change":"2024-04-25T10:21:41.218816+0000","last_active":"2024-04-25T10:21:42.220091+0000","last_peered":"2024-04-25T10:21:42.220091+0000","last_clean":"2024-04-25T10:21:42.220091+0000","last_became_active":"2024-04-25T10:21:41.218525+0000","last_became_peered":"2024-04-25T10:21:41.218525+0000","last_unstale":"2024-04-25T10:21:42.220091+0000","last_undegraded":"2024-04-25T10:21:42.220091+0000","last_fullsized":"2024-04-25T10:21:42.220091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:15.787793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221477+0000","last_change":"2024-04-25T10:21:41.195363+0000","last_active":"2024-04-25T10:21:42.221477+0000","last_peered":"2024-04-25T10:21:42.221477+0000","last_clean":"2024-04-25T10:21:42.221477+0000","last_became_active":"2024-04-25T10:21:41.195081+0000","last_became_peered":"2024-04-25T10:21:41.195081+0000","last_unstale":"2024-04-25T10:21:42.221477+0000","last_undegraded":"2024-04-25T10:21:42.221477+0000","last_fullsized":"2024-04-25T10:21:42.221477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:32.078454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202859+0000","last_change":"2024-04-25T10:21:41.206833+0000","last_active":"2024-04-25T10:21:42.202859+0000","last_peered":"2024-04-25T10:21:42.202859+0000","last_clean":"2024-04-25T10:21:42.202859+0000","last_became_active":"2024-04-25T10:21:41.206135+0000","last_became_peered":"2024-04-25T10:21:41.206135+0000","last_unstale":"2024-04-25T10:21:42.202859+0000","last_undegraded":"2024-04-25T10:21:42.202859+0000","last_fullsized":"2024-04-25T10:21:42.202859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:31.909824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222504+0000","last_change":"2024-04-25T10:21:41.211889+0000","last_active":"2024-04-25T10:21:42.222504+0000","last_peered":"2024-04-25T10:21:42.222504+0000","last_clean":"2024-04-25T10:21:42.222504+0000","last_became_active":"2024-04-25T10:21:41.211674+0000","last_became_peered":"2024-04-25T10:21:41.211674+0000","last_unstale":"2024-04-25T10:21:42.222504+0000","last_undegraded":"2024-04-25T10:21:42.222504+0000","last_fullsized":"2024-04-25T10:21:42.222504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.396942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220417+0000","last_change":"2024-04-25T10:21:41.206297+0000","last_active":"2024-04-25T10:21:42.220417+0000","last_peered":"2024-04-25T10:21:42.220417+0000","last_clean":"2024-04-25T10:21:42.220417+0000","last_became_active":"2024-04-25T10:21:41.205625+0000","last_became_peered":"2024-04-25T10:21:41.205625+0000","last_unstale":"2024-04-25T10:21:42.220417+0000","last_undegraded":"2024-04-25T10:21:42.220417+0000","last_fullsized":"2024-04-25T10:21:42.220417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:06.849937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203579+0000","last_change":"2024-04-25T10:21:41.212278+0000","last_active":"2024-04-25T10:21:42.203579+0000","last_peered":"2024-04-25T10:21:42.203579+0000","last_clean":"2024-04-25T10:21:42.203579+0000","last_became_active":"2024-04-25T10:21:41.211922+0000","last_became_peered":"2024-04-25T10:21:41.211922+0000","last_unstale":"2024-04-25T10:21:42.203579+0000","last_undegraded":"2024-04-25T10:21:42.203579+0000","last_fullsized":"2024-04-25T10:21:42.203579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:27.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221391+0000","last_change":"2024-04-25T10:21:41.211113+0000","last_active":"2024-04-25T10:21:42.221391+0000","last_peered":"2024-04-25T10:21:42.221391+0000","last_clean":"2024-04-25T10:21:42.221391+0000","last_became_active":"2024-04-25T10:21:41.210833+0000","last_became_peered":"2024-04-25T10:21:41.210833+0000","last_unstale":"2024-04-25T10:21:42.221391+0000","last_undegraded":"2024-04-25T10:21:42.221391+0000","last_fullsized":"2024-04-25T10:21:42.221391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:05.133982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222311+0000","last_change":"2024-04-25T10:21:41.209511+0000","last_active":"2024-04-25T10:21:42.222311+0000","last_peered":"2024-04-25T10:21:42.222311+0000","last_clean":"2024-04-25T10:21:42.222311+0000","last_became_active":"2024-04-25T10:21:41.208333+0000","last_became_peered":"2024-04-25T10:21:41.208333+0000","last_unstale":"2024-04-25T10:21:42.222311+0000","last_undegraded":"2024-04-25T10:21:42.222311+0000","last_fullsized":"2024-04-25T10:21:42.222311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:47.847107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202788+0000","last_change":"2024-04-25T10:21:41.206327+0000","last_active":"2024-04-25T10:21:42.202788+0000","last_peered":"2024-04-25T10:21:42.202788+0000","last_clean":"2024-04-25T10:21:42.202788+0000","last_became_active":"2024-04-25T10:21:41.206007+0000","last_became_peered":"2024-04-25T10:21:41.206007+0000","last_unstale":"2024-04-25T10:21:42.202788+0000","last_undegraded":"2024-04-25T10:21:42.202788+0000","last_fullsized":"2024-04-25T10:21:42.202788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:41.542390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222110+0000","last_change":"2024-04-25T10:21:41.210444+0000","last_active":"2024-04-25T10:21:42.222110+0000","last_peered":"2024-04-25T10:21:42.222110+0000","last_clean":"2024-04-25T10:21:42.222110+0000","last_became_active":"2024-04-25T10:21:41.209106+0000","last_became_peered":"2024-04-25T10:21:41.209106+0000","last_unstale":"2024-04-25T10:21:42.222110+0000","last_undegraded":"2024-04-25T10:21:42.222110+0000","last_fullsized":"2024-04-25T10:21:42.222110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:56.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219686+0000","last_change":"2024-04-25T10:21:41.206620+0000","last_active":"2024-04-25T10:21:42.219686+0000","last_peered":"2024-04-25T10:21:42.219686+0000","last_clean":"2024-04-25T10:21:42.219686+0000","last_became_active":"2024-04-25T10:21:41.206300+0000","last_became_peered":"2024-04-25T10:21:41.206300+0000","last_unstale":"2024-04-25T10:21:42.219686+0000","last_undegraded":"2024-04-25T10:21:42.219686+0000","last_fullsized":"2024-04-25T10:21:42.219686+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:34.174885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221033+0000","last_change":"2024-04-25T10:21:41.210846+0000","last_active":"2024-04-25T10:21:42.221033+0000","last_peered":"2024-04-25T10:21:42.221033+0000","last_clean":"2024-04-25T10:21:42.221033+0000","last_became_active":"2024-04-25T10:21:41.207207+0000","last_became_peered":"2024-04-25T10:21:41.207207+0000","last_unstale":"2024-04-25T10:21:42.221033+0000","last_undegraded":"2024-04-25T10:21:42.221033+0000","last_fullsized":"2024-04-25T10:21:42.221033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:21.681543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203137+0000","last_change":"2024-04-25T10:21:41.203604+0000","last_active":"2024-04-25T10:21:42.203137+0000","last_peered":"2024-04-25T10:21:42.203137+0000","last_clean":"2024-04-25T10:21:42.203137+0000","last_became_active":"2024-04-25T10:21:41.203236+0000","last_became_peered":"2024-04-25T10:21:41.203236+0000","last_unstale":"2024-04-25T10:21:42.203137+0000","last_undegraded":"2024-04-25T10:21:42.203137+0000","last_fullsized":"2024-04-25T10:21:42.203137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.849653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220598+0000","last_change":"2024-04-25T10:21:41.219420+0000","last_active":"2024-04-25T10:21:42.220598+0000","last_peered":"2024-04-25T10:21:42.220598+0000","last_clean":"2024-04-25T10:21:42.220598+0000","last_became_active":"2024-04-25T10:21:41.219116+0000","last_became_peered":"2024-04-25T10:21:41.219116+0000","last_unstale":"2024-04-25T10:21:42.220598+0000","last_undegraded":"2024-04-25T10:21:42.220598+0000","last_fullsized":"2024-04-25T10:21:42.220598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:22.475805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203004+0000","last_change":"2024-04-25T10:21:41.212213+0000","last_active":"2024-04-25T10:21:42.203004+0000","last_peered":"2024-04-25T10:21:42.203004+0000","last_clean":"2024-04-25T10:21:42.203004+0000","last_became_active":"2024-04-25T10:21:41.211847+0000","last_became_peered":"2024-04-25T10:21:41.211847+0000","last_unstale":"2024-04-25T10:21:42.203004+0000","last_undegraded":"2024-04-25T10:21:42.203004+0000","last_fullsized":"2024-04-25T10:21:42.203004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:37.486523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222619+0000","last_change":"2024-04-25T10:21:41.216433+0000","last_active":"2024-04-25T10:21:42.222619+0000","last_peered":"2024-04-25T10:21:42.222619+0000","last_clean":"2024-04-25T10:21:42.222619+0000","last_became_active":"2024-04-25T10:21:41.216161+0000","last_became_peered":"2024-04-25T10:21:41.216161+0000","last_unstale":"2024-04-25T10:21:42.222619+0000","last_undegraded":"2024-04-25T10:21:42.222619+0000","last_fullsized":"2024-04-25T10:21:42.222619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:15.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203954+0000","last_change":"2024-04-25T10:21:41.206838+0000","last_active":"2024-04-25T10:21:42.203954+0000","last_peered":"2024-04-25T10:21:42.203954+0000","last_clean":"2024-04-25T10:21:42.203954+0000","last_became_active":"2024-04-25T10:21:41.206177+0000","last_became_peered":"2024-04-25T10:21:41.206177+0000","last_unstale":"2024-04-25T10:21:42.203954+0000","last_undegraded":"2024-04-25T10:21:42.203954+0000","last_fullsized":"2024-04-25T10:21:42.203954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:12.761480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221615+0000","last_change":"2024-04-25T10:21:41.209818+0000","last_active":"2024-04-25T10:21:42.221615+0000","last_peered":"2024-04-25T10:21:42.221615+0000","last_clean":"2024-04-25T10:21:42.221615+0000","last_became_active":"2024-04-25T10:21:41.209524+0000","last_became_peered":"2024-04-25T10:21:41.209524+0000","last_unstale":"2024-04-25T10:21:42.221615+0000","last_undegraded":"2024-04-25T10:21:42.221615+0000","last_fullsized":"2024-04-25T10:21:42.221615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:59.050656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220634+0000","last_change":"2024-04-25T10:21:41.208575+0000","last_active":"2024-04-25T10:21:42.220634+0000","last_peered":"2024-04-25T10:21:42.220634+0000","last_clean":"2024-04-25T10:21:42.220634+0000","last_became_active":"2024-04-25T10:21:41.208286+0000","last_became_peered":"2024-04-25T10:21:41.208286+0000","last_unstale":"2024-04-25T10:21:42.220634+0000","last_undegraded":"2024-04-25T10:21:42.220634+0000","last_fullsized":"2024-04-25T10:21:42.220634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:13.496108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220219+0000","last_change":"2024-04-25T10:21:41.223922+0000","last_active":"2024-04-25T10:21:42.220219+0000","last_peered":"2024-04-25T10:21:42.220219+0000","last_clean":"2024-04-25T10:21:42.220219+0000","last_became_active":"2024-04-25T10:21:41.223620+0000","last_became_peered":"2024-04-25T10:21:41.223620+0000","last_unstale":"2024-04-25T10:21:42.220219+0000","last_undegraded":"2024-04-25T10:21:42.220219+0000","last_fullsized":"2024-04-25T10:21:42.220219+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:37.823772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222545+0000","last_change":"2024-04-25T10:21:41.208628+0000","last_active":"2024-04-25T10:21:42.222545+0000","last_peered":"2024-04-25T10:21:42.222545+0000","last_clean":"2024-04-25T10:21:42.222545+0000","last_became_active":"2024-04-25T10:21:41.205592+0000","last_became_peered":"2024-04-25T10:21:41.205592+0000","last_unstale":"2024-04-25T10:21:42.222545+0000","last_undegraded":"2024-04-25T10:21:42.222545+0000","last_fullsized":"2024-04-25T10:21:42.222545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:33.499184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220833+0000","last_change":"2024-04-25T10:21:41.215862+0000","last_active":"2024-04-25T10:21:42.220833+0000","last_peered":"2024-04-25T10:21:42.220833+0000","last_clean":"2024-04-25T10:21:42.220833+0000","last_became_active":"2024-04-25T10:21:41.215510+0000","last_became_peered":"2024-04-25T10:21:41.215510+0000","last_unstale":"2024-04-25T10:21:42.220833+0000","last_undegraded":"2024-04-25T10:21:42.220833+0000","last_fullsized":"2024-04-25T10:21:42.220833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:44.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204247+0000","last_change":"2024-04-25T10:21:41.214827+0000","last_active":"2024-04-25T10:21:42.204247+0000","last_peered":"2024-04-25T10:21:42.204247+0000","last_clean":"2024-04-25T10:21:42.204247+0000","last_became_active":"2024-04-25T10:21:41.214163+0000","last_became_peered":"2024-04-25T10:21:41.214163+0000","last_unstale":"2024-04-25T10:21:42.204247+0000","last_undegraded":"2024-04-25T10:21:42.204247+0000","last_fullsized":"2024-04-25T10:21:42.204247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:07.422717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204181+0000","last_change":"2024-04-25T10:21:41.199841+0000","last_active":"2024-04-25T10:21:42.204181+0000","last_peered":"2024-04-25T10:21:42.204181+0000","last_clean":"2024-04-25T10:21:42.204181+0000","last_became_active":"2024-04-25T10:21:41.198765+0000","last_became_peered":"2024-04-25T10:21:41.198765+0000","last_unstale":"2024-04-25T10:21:42.204181+0000","last_undegraded":"2024-04-25T10:21:42.204181+0000","last_fullsized":"2024-04-25T10:21:42.204181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:21.239625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220604+0000","last_change":"2024-04-25T10:21:41.200274+0000","last_active":"2024-04-25T10:21:42.220604+0000","last_peered":"2024-04-25T10:21:42.220604+0000","last_clean":"2024-04-25T10:21:42.220604+0000","last_became_active":"2024-04-25T10:21:41.199183+0000","last_became_peered":"2024-04-25T10:21:41.199183+0000","last_unstale":"2024-04-25T10:21:42.220604+0000","last_undegraded":"2024-04-25T10:21:42.220604+0000","last_fullsized":"2024-04-25T10:21:42.220604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:00.753360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219729+0000","last_change":"2024-04-25T10:21:41.221460+0000","last_active":"2024-04-25T10:21:42.219729+0000","last_peered":"2024-04-25T10:21:42.219729+0000","last_clean":"2024-04-25T10:21:42.219729+0000","last_became_active":"2024-04-25T10:21:41.221138+0000","last_became_peered":"2024-04-25T10:21:41.221138+0000","last_unstale":"2024-04-25T10:21:42.219729+0000","last_undegraded":"2024-04-25T10:21:42.219729+0000","last_fullsized":"2024-04-25T10:21:42.219729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:50.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202859+0000","last_change":"2024-04-25T10:21:41.208704+0000","last_active":"2024-04-25T10:21:42.202859+0000","last_peered":"2024-04-25T10:21:42.202859+0000","last_clean":"2024-04-25T10:21:42.202859+0000","last_became_active":"2024-04-25T10:21:41.208386+0000","last_became_peered":"2024-04-25T10:21:41.208386+0000","last_unstale":"2024-04-25T10:21:42.202859+0000","last_undegraded":"2024-04-25T10:21:42.202859+0000","last_fullsized":"2024-04-25T10:21:42.202859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.951240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220500+0000","last_change":"2024-04-25T10:21:41.204958+0000","last_active":"2024-04-25T10:21:42.220500+0000","last_peered":"2024-04-25T10:21:42.220500+0000","last_clean":"2024-04-25T10:21:42.220500+0000","last_became_active":"2024-04-25T10:21:41.204281+0000","last_became_peered":"2024-04-25T10:21:41.204281+0000","last_unstale":"2024-04-25T10:21:42.220500+0000","last_undegraded":"2024-04-25T10:21:42.220500+0000","last_fullsized":"2024-04-25T10:21:42.220500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:48.353976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203884+0000","last_change":"2024-04-25T10:21:40.177066+0000","last_active":"2024-04-25T10:21:42.203884+0000","last_peered":"2024-04-25T10:21:42.203884+0000","last_clean":"2024-04-25T10:21:42.203884+0000","last_became_active":"2024-04-25T10:21:40.176671+0000","last_became_peered":"2024-04-25T10:21:40.176671+0000","last_unstale":"2024-04-25T10:21:42.203884+0000","last_undegraded":"2024-04-25T10:21:42.203884+0000","last_fullsized":"2024-04-25T10:21:42.203884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:59.242042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221068+0000","last_change":"2024-04-25T10:21:41.212112+0000","last_active":"2024-04-25T10:21:42.221068+0000","last_peered":"2024-04-25T10:21:42.221068+0000","last_clean":"2024-04-25T10:21:42.221068+0000","last_became_active":"2024-04-25T10:21:41.211385+0000","last_became_peered":"2024-04-25T10:21:41.211385+0000","last_unstale":"2024-04-25T10:21:42.221068+0000","last_undegraded":"2024-04-25T10:21:42.221068+0000","last_fullsized":"2024-04-25T10:21:42.221068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:16.576949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220808+0000","last_change":"2024-04-25T10:21:40.196085+0000","last_active":"2024-04-25T10:21:42.220808+0000","last_peered":"2024-04-25T10:21:42.220808+0000","last_clean":"2024-04-25T10:21:42.220808+0000","last_became_active":"2024-04-25T10:21:40.195474+0000","last_became_peered":"2024-04-25T10:21:40.195474+0000","last_unstale":"2024-04-25T10:21:42.220808+0000","last_undegraded":"2024-04-25T10:21:42.220808+0000","last_fullsized":"2024-04-25T10:21:42.220808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:16.619511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204298+0000","last_change":"2024-04-25T10:21:41.215196+0000","last_active":"2024-04-25T10:21:42.204298+0000","last_peered":"2024-04-25T10:21:42.204298+0000","last_clean":"2024-04-25T10:21:42.204298+0000","last_became_active":"2024-04-25T10:21:41.214839+0000","last_became_peered":"2024-04-25T10:21:41.214839+0000","last_unstale":"2024-04-25T10:21:42.204298+0000","last_undegraded":"2024-04-25T10:21:42.204298+0000","last_fullsized":"2024-04-25T10:21:42.204298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:30.349114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220619+0000","last_change":"2024-04-25T10:21:40.202185+0000","last_active":"2024-04-25T10:21:42.220619+0000","last_peered":"2024-04-25T10:21:42.220619+0000","last_clean":"2024-04-25T10:21:42.220619+0000","last_became_active":"2024-04-25T10:21:40.201908+0000","last_became_peered":"2024-04-25T10:21:40.201908+0000","last_unstale":"2024-04-25T10:21:42.220619+0000","last_undegraded":"2024-04-25T10:21:42.220619+0000","last_fullsized":"2024-04-25T10:21:42.220619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:21.464946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220403+0000","last_change":"2024-04-25T10:21:41.206287+0000","last_active":"2024-04-25T10:21:42.220403+0000","last_peered":"2024-04-25T10:21:42.220403+0000","last_clean":"2024-04-25T10:21:42.220403+0000","last_became_active":"2024-04-25T10:21:41.205981+0000","last_became_peered":"2024-04-25T10:21:41.205981+0000","last_unstale":"2024-04-25T10:21:42.220403+0000","last_undegraded":"2024-04-25T10:21:42.220403+0000","last_fullsized":"2024-04-25T10:21:42.220403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:15.136959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221095+0000","last_change":"2024-04-25T10:21:40.193443+0000","last_active":"2024-04-25T10:21:42.221095+0000","last_peered":"2024-04-25T10:21:42.221095+0000","last_clean":"2024-04-25T10:21:42.221095+0000","last_became_active":"2024-04-25T10:21:40.193096+0000","last_became_peered":"2024-04-25T10:21:40.193096+0000","last_unstale":"2024-04-25T10:21:42.221095+0000","last_undegraded":"2024-04-25T10:21:42.221095+0000","last_fullsized":"2024-04-25T10:21:42.221095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:51.990070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203229+0000","last_change":"2024-04-25T10:21:41.192606+0000","last_active":"2024-04-25T10:21:42.203229+0000","last_peered":"2024-04-25T10:21:42.203229+0000","last_clean":"2024-04-25T10:21:42.203229+0000","last_became_active":"2024-04-25T10:21:41.192240+0000","last_became_peered":"2024-04-25T10:21:41.192240+0000","last_unstale":"2024-04-25T10:21:42.203229+0000","last_undegraded":"2024-04-25T10:21:42.203229+0000","last_fullsized":"2024-04-25T10:21:42.203229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:40.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202883+0000","last_change":"2024-04-25T10:21:40.186905+0000","last_active":"2024-04-25T10:21:42.202883+0000","last_peered":"2024-04-25T10:21:42.202883+0000","last_clean":"2024-04-25T10:21:42.202883+0000","last_became_active":"2024-04-25T10:21:40.185516+0000","last_became_peered":"2024-04-25T10:21:40.185516+0000","last_unstale":"2024-04-25T10:21:42.202883+0000","last_undegraded":"2024-04-25T10:21:42.202883+0000","last_fullsized":"2024-04-25T10:21:42.202883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:41.704477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203224+0000","last_change":"2024-04-25T10:21:41.208531+0000","last_active":"2024-04-25T10:21:42.203224+0000","last_peered":"2024-04-25T10:21:42.203224+0000","last_clean":"2024-04-25T10:21:42.203224+0000","last_became_active":"2024-04-25T10:21:41.208252+0000","last_became_peered":"2024-04-25T10:21:41.208252+0000","last_unstale":"2024-04-25T10:21:42.203224+0000","last_undegraded":"2024-04-25T10:21:42.203224+0000","last_fullsized":"2024-04-25T10:21:42.203224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.828723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220210+0000","last_change":"2024-04-25T10:21:40.201628+0000","last_active":"2024-04-25T10:21:42.220210+0000","last_peered":"2024-04-25T10:21:42.220210+0000","last_clean":"2024-04-25T10:21:42.220210+0000","last_became_active":"2024-04-25T10:21:40.201263+0000","last_became_peered":"2024-04-25T10:21:40.201263+0000","last_unstale":"2024-04-25T10:21:42.220210+0000","last_undegraded":"2024-04-25T10:21:42.220210+0000","last_fullsized":"2024-04-25T10:21:42.220210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:43.455535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202832+0000","last_change":"2024-04-25T10:21:41.198686+0000","last_active":"2024-04-25T10:21:42.202832+0000","last_peered":"2024-04-25T10:21:42.202832+0000","last_clean":"2024-04-25T10:21:42.202832+0000","last_became_active":"2024-04-25T10:21:41.198390+0000","last_became_peered":"2024-04-25T10:21:41.198390+0000","last_unstale":"2024-04-25T10:21:42.202832+0000","last_undegraded":"2024-04-25T10:21:42.202832+0000","last_fullsized":"2024-04-25T10:21:42.202832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:32.035479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221107+0000","last_change":"2024-04-25T10:21:40.196432+0000","last_active":"2024-04-25T10:21:42.221107+0000","last_peered":"2024-04-25T10:21:42.221107+0000","last_clean":"2024-04-25T10:21:42.221107+0000","last_became_active":"2024-04-25T10:21:40.196148+0000","last_became_peered":"2024-04-25T10:21:40.196148+0000","last_unstale":"2024-04-25T10:21:42.221107+0000","last_undegraded":"2024-04-25T10:21:42.221107+0000","last_fullsized":"2024-04-25T10:21:42.221107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:23.242048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220717+0000","last_change":"2024-04-25T10:21:41.205986+0000","last_active":"2024-04-25T10:21:42.220717+0000","last_peered":"2024-04-25T10:21:42.220717+0000","last_clean":"2024-04-25T10:21:42.220717+0000","last_became_active":"2024-04-25T10:21:41.204808+0000","last_became_peered":"2024-04-25T10:21:41.204808+0000","last_unstale":"2024-04-25T10:21:42.220717+0000","last_undegraded":"2024-04-25T10:21:42.220717+0000","last_fullsized":"2024-04-25T10:21:42.220717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:33.499184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221108+0000","last_change":"2024-04-25T10:21:40.194950+0000","last_active":"2024-04-25T10:21:42.221108+0000","last_peered":"2024-04-25T10:21:42.221108+0000","last_clean":"2024-04-25T10:21:42.221108+0000","last_became_active":"2024-04-25T10:21:40.194767+0000","last_became_peered":"2024-04-25T10:21:40.194767+0000","last_unstale":"2024-04-25T10:21:42.221108+0000","last_undegraded":"2024-04-25T10:21:42.221108+0000","last_fullsized":"2024-04-25T10:21:42.221108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:02.827283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222658+0000","last_change":"2024-04-25T10:21:41.219108+0000","last_active":"2024-04-25T10:21:42.222658+0000","last_peered":"2024-04-25T10:21:42.222658+0000","last_clean":"2024-04-25T10:21:42.222658+0000","last_became_active":"2024-04-25T10:21:41.218738+0000","last_became_peered":"2024-04-25T10:21:41.218738+0000","last_unstale":"2024-04-25T10:21:42.222658+0000","last_undegraded":"2024-04-25T10:21:42.222658+0000","last_fullsized":"2024-04-25T10:21:42.222658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:15.787793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222131+0000","last_change":"2024-04-25T10:21:40.197183+0000","last_active":"2024-04-25T10:21:42.222131+0000","last_peered":"2024-04-25T10:21:42.222131+0000","last_clean":"2024-04-25T10:21:42.222131+0000","last_became_active":"2024-04-25T10:21:40.196934+0000","last_became_peered":"2024-04-25T10:21:40.196934+0000","last_unstale":"2024-04-25T10:21:42.222131+0000","last_undegraded":"2024-04-25T10:21:42.222131+0000","last_fullsized":"2024-04-25T10:21:42.222131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:16.619511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203234+0000","last_change":"2024-04-25T10:21:41.212995+0000","last_active":"2024-04-25T10:21:42.203234+0000","last_peered":"2024-04-25T10:21:42.203234+0000","last_clean":"2024-04-25T10:21:42.203234+0000","last_became_active":"2024-04-25T10:21:41.212730+0000","last_became_peered":"2024-04-25T10:21:41.212730+0000","last_unstale":"2024-04-25T10:21:42.203234+0000","last_undegraded":"2024-04-25T10:21:42.203234+0000","last_fullsized":"2024-04-25T10:21:42.203234+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:30.657137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203984+0000","last_change":"2024-04-25T10:21:40.196263+0000","last_active":"2024-04-25T10:21:42.203984+0000","last_peered":"2024-04-25T10:21:42.203984+0000","last_clean":"2024-04-25T10:21:42.203984+0000","last_became_active":"2024-04-25T10:21:40.193856+0000","last_became_peered":"2024-04-25T10:21:40.193856+0000","last_unstale":"2024-04-25T10:21:42.203984+0000","last_undegraded":"2024-04-25T10:21:42.203984+0000","last_fullsized":"2024-04-25T10:21:42.203984+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:04.248491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204436+0000","last_change":"2024-04-25T10:21:41.215056+0000","last_active":"2024-04-25T10:21:42.204436+0000","last_peered":"2024-04-25T10:21:42.204436+0000","last_clean":"2024-04-25T10:21:42.204436+0000","last_became_active":"2024-04-25T10:21:41.214536+0000","last_became_peered":"2024-04-25T10:21:41.214536+0000","last_unstale":"2024-04-25T10:21:42.204436+0000","last_undegraded":"2024-04-25T10:21:42.204436+0000","last_fullsized":"2024-04-25T10:21:42.204436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:15.245969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221220+0000","last_change":"2024-04-25T10:21:40.198205+0000","last_active":"2024-04-25T10:21:42.221220+0000","last_peered":"2024-04-25T10:21:42.221220+0000","last_clean":"2024-04-25T10:21:42.221220+0000","last_became_active":"2024-04-25T10:21:40.197954+0000","last_became_peered":"2024-04-25T10:21:40.197954+0000","last_unstale":"2024-04-25T10:21:42.221220+0000","last_undegraded":"2024-04-25T10:21:42.221220+0000","last_fullsized":"2024-04-25T10:21:42.221220+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:39.438339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221062+0000","last_change":"2024-04-25T10:21:41.198083+0000","last_active":"2024-04-25T10:21:42.221062+0000","last_peered":"2024-04-25T10:21:42.221062+0000","last_clean":"2024-04-25T10:21:42.221062+0000","last_became_active":"2024-04-25T10:21:41.196937+0000","last_became_peered":"2024-04-25T10:21:41.196937+0000","last_unstale":"2024-04-25T10:21:42.221062+0000","last_undegraded":"2024-04-25T10:21:42.221062+0000","last_fullsized":"2024-04-25T10:21:42.221062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:50.865221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222445+0000","last_change":"2024-04-25T10:21:40.188594+0000","last_active":"2024-04-25T10:21:42.222445+0000","last_peered":"2024-04-25T10:21:42.222445+0000","last_clean":"2024-04-25T10:21:42.222445+0000","last_became_active":"2024-04-25T10:21:40.188126+0000","last_became_peered":"2024-04-25T10:21:40.188126+0000","last_unstale":"2024-04-25T10:21:42.222445+0000","last_undegraded":"2024-04-25T10:21:42.222445+0000","last_fullsized":"2024-04-25T10:21:42.222445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:01.501495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221101+0000","last_change":"2024-04-25T10:21:41.201759+0000","last_active":"2024-04-25T10:21:42.221101+0000","last_peered":"2024-04-25T10:21:42.221101+0000","last_clean":"2024-04-25T10:21:42.221101+0000","last_became_active":"2024-04-25T10:21:41.201482+0000","last_became_peered":"2024-04-25T10:21:41.201482+0000","last_unstale":"2024-04-25T10:21:42.221101+0000","last_undegraded":"2024-04-25T10:21:42.221101+0000","last_fullsized":"2024-04-25T10:21:42.221101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.081194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204087+0000","last_change":"2024-04-25T10:21:40.192607+0000","last_active":"2024-04-25T10:21:42.204087+0000","last_peered":"2024-04-25T10:21:42.204087+0000","last_clean":"2024-04-25T10:21:42.204087+0000","last_became_active":"2024-04-25T10:21:40.192317+0000","last_became_peered":"2024-04-25T10:21:40.192317+0000","last_unstale":"2024-04-25T10:21:42.204087+0000","last_undegraded":"2024-04-25T10:21:42.204087+0000","last_fullsized":"2024-04-25T10:21:42.204087+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:27.883586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202723+0000","last_change":"2024-04-25T10:21:41.206297+0000","last_active":"2024-04-25T10:21:42.202723+0000","last_peered":"2024-04-25T10:21:42.202723+0000","last_clean":"2024-04-25T10:21:42.202723+0000","last_became_active":"2024-04-25T10:21:41.205953+0000","last_became_peered":"2024-04-25T10:21:41.205953+0000","last_unstale":"2024-04-25T10:21:42.202723+0000","last_undegraded":"2024-04-25T10:21:42.202723+0000","last_fullsized":"2024-04-25T10:21:42.202723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:32.394769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220716+0000","last_change":"2024-04-25T10:21:40.195906+0000","last_active":"2024-04-25T10:21:42.220716+0000","last_peered":"2024-04-25T10:21:42.220716+0000","last_clean":"2024-04-25T10:21:42.220716+0000","last_became_active":"2024-04-25T10:21:40.190663+0000","last_became_peered":"2024-04-25T10:21:40.190663+0000","last_unstale":"2024-04-25T10:21:42.220716+0000","last_undegraded":"2024-04-25T10:21:42.220716+0000","last_fullsized":"2024-04-25T10:21:42.220716+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:30.898965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220812+0000","last_change":"2024-04-25T10:21:41.201179+0000","last_active":"2024-04-25T10:21:42.220812+0000","last_peered":"2024-04-25T10:21:42.220812+0000","last_clean":"2024-04-25T10:21:42.220812+0000","last_became_active":"2024-04-25T10:21:41.199882+0000","last_became_peered":"2024-04-25T10:21:41.199882+0000","last_unstale":"2024-04-25T10:21:42.220812+0000","last_undegraded":"2024-04-25T10:21:42.220812+0000","last_fullsized":"2024-04-25T10:21:42.220812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:51.486160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221843+0000","last_change":"2024-04-25T10:21:40.197623+0000","last_active":"2024-04-25T10:21:42.221843+0000","last_peered":"2024-04-25T10:21:42.221843+0000","last_clean":"2024-04-25T10:21:42.221843+0000","last_became_active":"2024-04-25T10:21:40.196276+0000","last_became_peered":"2024-04-25T10:21:40.196276+0000","last_unstale":"2024-04-25T10:21:42.221843+0000","last_undegraded":"2024-04-25T10:21:42.221843+0000","last_fullsized":"2024-04-25T10:21:42.221843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:18.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221753+0000","last_change":"2024-04-25T10:21:41.209510+0000","last_active":"2024-04-25T10:21:42.221753+0000","last_peered":"2024-04-25T10:21:42.221753+0000","last_clean":"2024-04-25T10:21:42.221753+0000","last_became_active":"2024-04-25T10:21:41.209211+0000","last_became_peered":"2024-04-25T10:21:41.209211+0000","last_unstale":"2024-04-25T10:21:42.221753+0000","last_undegraded":"2024-04-25T10:21:42.221753+0000","last_fullsized":"2024-04-25T10:21:42.221753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:00.812567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":22,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220781+0000","last_change":"2024-04-25T10:21:40.190478+0000","last_active":"2024-04-25T10:21:42.220781+0000","last_peered":"2024-04-25T10:21:42.220781+0000","last_clean":"2024-04-25T10:21:42.220781+0000","last_became_active":"2024-04-25T10:21:40.190156+0000","last_became_peered":"2024-04-25T10:21:40.190156+0000","last_unstale":"2024-04-25T10:21:42.220781+0000","last_undegraded":"2024-04-25T10:21:42.220781+0000","last_fullsized":"2024-04-25T10:21:42.220781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:54.141917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203224+0000","last_change":"2024-04-25T10:21:41.198753+0000","last_active":"2024-04-25T10:21:42.203224+0000","last_peered":"2024-04-25T10:21:42.203224+0000","last_clean":"2024-04-25T10:21:42.203224+0000","last_became_active":"2024-04-25T10:21:41.198469+0000","last_became_peered":"2024-04-25T10:21:41.198469+0000","last_unstale":"2024-04-25T10:21:42.203224+0000","last_undegraded":"2024-04-25T10:21:42.203224+0000","last_fullsized":"2024-04-25T10:21:42.203224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:34.347853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220330+0000","last_change":"2024-04-25T10:21:40.199590+0000","last_active":"2024-04-25T10:21:42.220330+0000","last_peered":"2024-04-25T10:21:42.220330+0000","last_clean":"2024-04-25T10:21:42.220330+0000","last_became_active":"2024-04-25T10:21:40.198614+0000","last_became_peered":"2024-04-25T10:21:40.198614+0000","last_unstale":"2024-04-25T10:21:42.220330+0000","last_undegraded":"2024-04-25T10:21:42.220330+0000","last_fullsized":"2024-04-25T10:21:42.220330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:06.527101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203210+0000","last_change":"2024-04-25T10:21:41.212992+0000","last_active":"2024-04-25T10:21:42.203210+0000","last_peered":"2024-04-25T10:21:42.203210+0000","last_clean":"2024-04-25T10:21:42.203210+0000","last_became_active":"2024-04-25T10:21:41.212625+0000","last_became_peered":"2024-04-25T10:21:41.212625+0000","last_unstale":"2024-04-25T10:21:42.203210+0000","last_undegraded":"2024-04-25T10:21:42.203210+0000","last_fullsized":"2024-04-25T10:21:42.203210+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:31.483195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204061+0000","last_change":"2024-04-25T10:21:40.196071+0000","last_active":"2024-04-25T10:21:42.204061+0000","last_peered":"2024-04-25T10:21:42.204061+0000","last_clean":"2024-04-25T10:21:42.204061+0000","last_became_active":"2024-04-25T10:21:40.193492+0000","last_became_peered":"2024-04-25T10:21:40.193492+0000","last_unstale":"2024-04-25T10:21:42.204061+0000","last_undegraded":"2024-04-25T10:21:42.204061+0000","last_fullsized":"2024-04-25T10:21:42.204061+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:20.670684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220093+0000","last_change":"2024-04-25T10:21:41.210304+0000","last_active":"2024-04-25T10:21:42.220093+0000","last_peered":"2024-04-25T10:21:42.220093+0000","last_clean":"2024-04-25T10:21:42.220093+0000","last_became_active":"2024-04-25T10:21:41.209988+0000","last_became_peered":"2024-04-25T10:21:41.209988+0000","last_unstale":"2024-04-25T10:21:42.220093+0000","last_undegraded":"2024-04-25T10:21:42.220093+0000","last_fullsized":"2024-04-25T10:21:42.220093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.396942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203116+0000","last_change":"2024-04-25T10:21:40.201434+0000","last_active":"2024-04-25T10:21:42.203116+0000","last_peered":"2024-04-25T10:21:42.203116+0000","last_clean":"2024-04-25T10:21:42.203116+0000","last_became_active":"2024-04-25T10:21:40.200888+0000","last_became_peered":"2024-04-25T10:21:40.200888+0000","last_unstale":"2024-04-25T10:21:42.203116+0000","last_undegraded":"2024-04-25T10:21:42.203116+0000","last_fullsized":"2024-04-25T10:21:42.203116+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:33.164026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221195+0000","last_change":"2024-04-25T10:21:41.215278+0000","last_active":"2024-04-25T10:21:42.221195+0000","last_peered":"2024-04-25T10:21:42.221195+0000","last_clean":"2024-04-25T10:21:42.221195+0000","last_became_active":"2024-04-25T10:21:41.214964+0000","last_became_peered":"2024-04-25T10:21:41.214964+0000","last_unstale":"2024-04-25T10:21:42.221195+0000","last_undegraded":"2024-04-25T10:21:42.221195+0000","last_fullsized":"2024-04-25T10:21:42.221195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:35.721778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204368+0000","last_change":"2024-04-25T10:21:40.199586+0000","last_active":"2024-04-25T10:21:42.204368+0000","last_peered":"2024-04-25T10:21:42.204368+0000","last_clean":"2024-04-25T10:21:42.204368+0000","last_became_active":"2024-04-25T10:21:40.199305+0000","last_became_peered":"2024-04-25T10:21:40.199305+0000","last_unstale":"2024-04-25T10:21:42.204368+0000","last_undegraded":"2024-04-25T10:21:42.204368+0000","last_fullsized":"2024-04-25T10:21:42.204368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:47.343118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221194+0000","last_change":"2024-04-25T10:21:41.211428+0000","last_active":"2024-04-25T10:21:42.221194+0000","last_peered":"2024-04-25T10:21:42.221194+0000","last_clean":"2024-04-25T10:21:42.221194+0000","last_became_active":"2024-04-25T10:21:41.211142+0000","last_became_peered":"2024-04-25T10:21:41.211142+0000","last_unstale":"2024-04-25T10:21:42.221194+0000","last_undegraded":"2024-04-25T10:21:42.221194+0000","last_fullsized":"2024-04-25T10:21:42.221194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:50.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203339+0000","last_change":"2024-04-25T10:21:40.191205+0000","last_active":"2024-04-25T10:21:42.203339+0000","last_peered":"2024-04-25T10:21:42.203339+0000","last_clean":"2024-04-25T10:21:42.203339+0000","last_became_active":"2024-04-25T10:21:40.190776+0000","last_became_peered":"2024-04-25T10:21:40.190776+0000","last_unstale":"2024-04-25T10:21:42.203339+0000","last_undegraded":"2024-04-25T10:21:42.203339+0000","last_fullsized":"2024-04-25T10:21:42.203339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:09.663619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203958+0000","last_change":"2024-04-25T10:21:41.199613+0000","last_active":"2024-04-25T10:21:42.203958+0000","last_peered":"2024-04-25T10:21:42.203958+0000","last_clean":"2024-04-25T10:21:42.203958+0000","last_became_active":"2024-04-25T10:21:41.198421+0000","last_became_peered":"2024-04-25T10:21:41.198421+0000","last_unstale":"2024-04-25T10:21:42.203958+0000","last_undegraded":"2024-04-25T10:21:42.203958+0000","last_fullsized":"2024-04-25T10:21:42.203958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.670568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203343+0000","last_change":"2024-04-25T10:21:40.183797+0000","last_active":"2024-04-25T10:21:42.203343+0000","last_peered":"2024-04-25T10:21:42.203343+0000","last_clean":"2024-04-25T10:21:42.203343+0000","last_became_active":"2024-04-25T10:21:40.183503+0000","last_became_peered":"2024-04-25T10:21:40.183503+0000","last_unstale":"2024-04-25T10:21:42.203343+0000","last_undegraded":"2024-04-25T10:21:42.203343+0000","last_fullsized":"2024-04-25T10:21:42.203343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:14.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202915+0000","last_change":"2024-04-25T10:21:41.205059+0000","last_active":"2024-04-25T10:21:42.202915+0000","last_peered":"2024-04-25T10:21:42.202915+0000","last_clean":"2024-04-25T10:21:42.202915+0000","last_became_active":"2024-04-25T10:21:41.204776+0000","last_became_peered":"2024-04-25T10:21:41.204776+0000","last_unstale":"2024-04-25T10:21:42.202915+0000","last_undegraded":"2024-04-25T10:21:42.202915+0000","last_fullsized":"2024-04-25T10:21:42.202915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:53.000929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220802+0000","last_change":"2024-04-25T10:21:40.198245+0000","last_active":"2024-04-25T10:21:42.220802+0000","last_peered":"2024-04-25T10:21:42.220802+0000","last_clean":"2024-04-25T10:21:42.220802+0000","last_became_active":"2024-04-25T10:21:40.196154+0000","last_became_peered":"2024-04-25T10:21:40.196154+0000","last_unstale":"2024-04-25T10:21:42.220802+0000","last_undegraded":"2024-04-25T10:21:42.220802+0000","last_fullsized":"2024-04-25T10:21:42.220802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:30.617232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203301+0000","last_change":"2024-04-25T10:21:41.206073+0000","last_active":"2024-04-25T10:21:42.203301+0000","last_peered":"2024-04-25T10:21:42.203301+0000","last_clean":"2024-04-25T10:21:42.203301+0000","last_became_active":"2024-04-25T10:21:41.205704+0000","last_became_peered":"2024-04-25T10:21:41.205704+0000","last_unstale":"2024-04-25T10:21:42.203301+0000","last_undegraded":"2024-04-25T10:21:42.203301+0000","last_fullsized":"2024-04-25T10:21:42.203301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:31.909824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220448+0000","last_change":"2024-04-25T10:21:40.196050+0000","last_active":"2024-04-25T10:21:42.220448+0000","last_peered":"2024-04-25T10:21:42.220448+0000","last_clean":"2024-04-25T10:21:42.220448+0000","last_became_active":"2024-04-25T10:21:40.195717+0000","last_became_peered":"2024-04-25T10:21:40.195717+0000","last_unstale":"2024-04-25T10:21:42.220448+0000","last_undegraded":"2024-04-25T10:21:42.220448+0000","last_fullsized":"2024-04-25T10:21:42.220448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:20.770611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203291+0000","last_change":"2024-04-25T10:21:41.198144+0000","last_active":"2024-04-25T10:21:42.203291+0000","last_peered":"2024-04-25T10:21:42.203291+0000","last_clean":"2024-04-25T10:21:42.203291+0000","last_became_active":"2024-04-25T10:21:41.197830+0000","last_became_peered":"2024-04-25T10:21:41.197830+0000","last_unstale":"2024-04-25T10:21:42.203291+0000","last_undegraded":"2024-04-25T10:21:42.203291+0000","last_fullsized":"2024-04-25T10:21:42.203291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:55.152776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222018+0000","last_change":"2024-04-25T10:21:40.201632+0000","last_active":"2024-04-25T10:21:42.222018+0000","last_peered":"2024-04-25T10:21:42.222018+0000","last_clean":"2024-04-25T10:21:42.222018+0000","last_became_active":"2024-04-25T10:21:40.201331+0000","last_became_peered":"2024-04-25T10:21:40.201331+0000","last_unstale":"2024-04-25T10:21:42.222018+0000","last_undegraded":"2024-04-25T10:21:42.222018+0000","last_fullsized":"2024-04-25T10:21:42.222018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:45.965146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220879+0000","last_change":"2024-04-25T10:21:41.207659+0000","last_active":"2024-04-25T10:21:42.220879+0000","last_peered":"2024-04-25T10:21:42.220879+0000","last_clean":"2024-04-25T10:21:42.220879+0000","last_became_active":"2024-04-25T10:21:41.207300+0000","last_became_peered":"2024-04-25T10:21:41.207300+0000","last_unstale":"2024-04-25T10:21:42.220879+0000","last_undegraded":"2024-04-25T10:21:42.220879+0000","last_fullsized":"2024-04-25T10:21:42.220879+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:46.976004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221492+0000","last_change":"2024-04-25T10:21:40.186060+0000","last_active":"2024-04-25T10:21:42.221492+0000","last_peered":"2024-04-25T10:21:42.221492+0000","last_clean":"2024-04-25T10:21:42.221492+0000","last_became_active":"2024-04-25T10:21:40.185767+0000","last_became_peered":"2024-04-25T10:21:40.185767+0000","last_unstale":"2024-04-25T10:21:42.221492+0000","last_undegraded":"2024-04-25T10:21:42.221492+0000","last_fullsized":"2024-04-25T10:21:42.221492+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:04.614877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203885+0000","last_change":"2024-04-25T10:21:41.213606+0000","last_active":"2024-04-25T10:21:42.203885+0000","last_peered":"2024-04-25T10:21:42.203885+0000","last_clean":"2024-04-25T10:21:42.203885+0000","last_became_active":"2024-04-25T10:21:41.213253+0000","last_became_peered":"2024-04-25T10:21:41.213253+0000","last_unstale":"2024-04-25T10:21:42.203885+0000","last_undegraded":"2024-04-25T10:21:42.203885+0000","last_fullsized":"2024-04-25T10:21:42.203885+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.553902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202982+0000","last_change":"2024-04-25T10:21:40.198944+0000","last_active":"2024-04-25T10:21:42.202982+0000","last_peered":"2024-04-25T10:21:42.202982+0000","last_clean":"2024-04-25T10:21:42.202982+0000","last_became_active":"2024-04-25T10:21:40.198666+0000","last_became_peered":"2024-04-25T10:21:40.198666+0000","last_unstale":"2024-04-25T10:21:42.202982+0000","last_undegraded":"2024-04-25T10:21:42.202982+0000","last_fullsized":"2024-04-25T10:21:42.202982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.026517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219943+0000","last_change":"2024-04-25T10:21:41.217766+0000","last_active":"2024-04-25T10:21:42.219943+0000","last_peered":"2024-04-25T10:21:42.219943+0000","last_clean":"2024-04-25T10:21:42.219943+0000","last_became_active":"2024-04-25T10:21:41.211493+0000","last_became_peered":"2024-04-25T10:21:41.211493+0000","last_unstale":"2024-04-25T10:21:42.219943+0000","last_undegraded":"2024-04-25T10:21:42.219943+0000","last_fullsized":"2024-04-25T10:21:42.219943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:15.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220089+0000","last_change":"2024-04-25T10:21:40.198938+0000","last_active":"2024-04-25T10:21:42.220089+0000","last_peered":"2024-04-25T10:21:42.220089+0000","last_clean":"2024-04-25T10:21:42.220089+0000","last_became_active":"2024-04-25T10:21:40.198640+0000","last_became_peered":"2024-04-25T10:21:40.198640+0000","last_unstale":"2024-04-25T10:21:42.220089+0000","last_undegraded":"2024-04-25T10:21:42.220089+0000","last_fullsized":"2024-04-25T10:21:42.220089+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:30.617232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222531+0000","last_change":"2024-04-25T10:21:41.214282+0000","last_active":"2024-04-25T10:21:42.222531+0000","last_peered":"2024-04-25T10:21:42.222531+0000","last_clean":"2024-04-25T10:21:42.222531+0000","last_became_active":"2024-04-25T10:21:41.213972+0000","last_became_peered":"2024-04-25T10:21:41.213972+0000","last_unstale":"2024-04-25T10:21:42.222531+0000","last_undegraded":"2024-04-25T10:21:42.222531+0000","last_fullsized":"2024-04-25T10:21:42.222531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:34.174885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202757+0000","last_change":"2024-04-25T10:21:40.201536+0000","last_active":"2024-04-25T10:21:42.202757+0000","last_peered":"2024-04-25T10:21:42.202757+0000","last_clean":"2024-04-25T10:21:42.202757+0000","last_became_active":"2024-04-25T10:21:40.201220+0000","last_became_peered":"2024-04-25T10:21:40.201220+0000","last_unstale":"2024-04-25T10:21:42.202757+0000","last_undegraded":"2024-04-25T10:21:42.202757+0000","last_fullsized":"2024-04-25T10:21:42.202757+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:46.836248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203023+0000","last_change":"2024-04-25T10:21:41.192855+0000","last_active":"2024-04-25T10:21:42.203023+0000","last_peered":"2024-04-25T10:21:42.203023+0000","last_clean":"2024-04-25T10:21:42.203023+0000","last_became_active":"2024-04-25T10:21:41.192413+0000","last_became_peered":"2024-04-25T10:21:41.192413+0000","last_unstale":"2024-04-25T10:21:42.203023+0000","last_undegraded":"2024-04-25T10:21:42.203023+0000","last_fullsized":"2024-04-25T10:21:42.203023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:19.769433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219890+0000","last_change":"2024-04-25T10:21:40.198377+0000","last_active":"2024-04-25T10:21:42.219890+0000","last_peered":"2024-04-25T10:21:42.219890+0000","last_clean":"2024-04-25T10:21:42.219890+0000","last_became_active":"2024-04-25T10:21:40.198112+0000","last_became_peered":"2024-04-25T10:21:40.198112+0000","last_unstale":"2024-04-25T10:21:42.219890+0000","last_undegraded":"2024-04-25T10:21:42.219890+0000","last_fullsized":"2024-04-25T10:21:42.219890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:38.605398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219898+0000","last_change":"2024-04-25T10:21:41.218098+0000","last_active":"2024-04-25T10:21:42.219898+0000","last_peered":"2024-04-25T10:21:42.219898+0000","last_clean":"2024-04-25T10:21:42.219898+0000","last_became_active":"2024-04-25T10:21:41.217814+0000","last_became_peered":"2024-04-25T10:21:41.217814+0000","last_unstale":"2024-04-25T10:21:42.219898+0000","last_undegraded":"2024-04-25T10:21:42.219898+0000","last_fullsized":"2024-04-25T10:21:42.219898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:05.133982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203129+0000","last_change":"2024-04-25T10:21:40.198104+0000","last_active":"2024-04-25T10:21:42.203129+0000","last_peered":"2024-04-25T10:21:42.203129+0000","last_clean":"2024-04-25T10:21:42.203129+0000","last_became_active":"2024-04-25T10:21:40.197665+0000","last_became_peered":"2024-04-25T10:21:40.197665+0000","last_unstale":"2024-04-25T10:21:42.203129+0000","last_undegraded":"2024-04-25T10:21:42.203129+0000","last_fullsized":"2024-04-25T10:21:42.203129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:05.839079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203307+0000","last_change":"2024-04-25T10:21:41.207095+0000","last_active":"2024-04-25T10:21:42.203307+0000","last_peered":"2024-04-25T10:21:42.203307+0000","last_clean":"2024-04-25T10:21:42.203307+0000","last_became_active":"2024-04-25T10:21:41.206524+0000","last_became_peered":"2024-04-25T10:21:41.206524+0000","last_unstale":"2024-04-25T10:21:42.203307+0000","last_undegraded":"2024-04-25T10:21:42.203307+0000","last_fullsized":"2024-04-25T10:21:42.203307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.692831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222270+0000","last_change":"2024-04-25T10:21:40.194319+0000","last_active":"2024-04-25T10:21:42.222270+0000","last_peered":"2024-04-25T10:21:42.222270+0000","last_clean":"2024-04-25T10:21:42.222270+0000","last_became_active":"2024-04-25T10:21:40.193961+0000","last_became_peered":"2024-04-25T10:21:40.193961+0000","last_unstale":"2024-04-25T10:21:42.222270+0000","last_undegraded":"2024-04-25T10:21:42.222270+0000","last_fullsized":"2024-04-25T10:21:42.222270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:36.719653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220851+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:21:42.220851+0000","last_peered":"2024-04-25T10:21:42.220851+0000","last_clean":"2024-04-25T10:21:42.220851+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:21:42.220851+0000","last_undegraded":"2024-04-25T10:21:42.220851+0000","last_fullsized":"2024-04-25T10:21:42.220851+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":745472,"data_stored":308342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36064,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335776,"statfs":{"total":96636764160,"available":96599834624,"internally_reserved":0,"allocated":741376,"data_stored":308272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40812,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331028,"statfs":{"total":96636764160,"available":96594972672,"internally_reserved":0,"allocated":1335296,"data_stored":898652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":741376,"data_stored":308340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40804,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331036,"statfs":{"total":96636764160,"available":96594980864,"internally_reserved":0,"allocated":1335296,"data_stored":899200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":749568,"data_stored":309862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58272,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313568,"statfs":{"total":96636764160,"available":96577093632,"internally_reserved":0,"allocated":733184,"data_stored":302566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:21:45.315 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:21:45.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 Processor -- start 2024-04-25T10:21:45.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 -- start start 2024-04-25T10:21:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83100d0350 0x7f83100d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831006aea0 0x7f831006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83100ccaa0 con 0x7f831015df60 2024-04-25T10:21:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831015e340 con 0x7f83100d0350 2024-04-25T10:21:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8316c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83100ccd80 con 0x7f831006aea0 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8314c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831006aea0 0x7f831006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f8307e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83100d0350 0x7f83100d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f830fe00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f831015df60 0x7f83100cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:49914/0 (socket says 172.21.15.45:49914) 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.494+0000 7f830fe00640 1 -- 172.21.15.45:0/550305162 learned_addr learned my addr 172.21.15.45:0/550305162 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f8307e00640 1 -- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831006aea0 msgr2=0x7f831006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f8307e00640 1 --2- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831006aea0 0x7f831006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f8307e00640 1 -- 172.21.15.45:0/550305162 >> v1:172.21.15.117:6789/0 conn(0x7f831015df60 legacy=0x7f83100cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f8307e00640 1 -- 172.21.15.45:0/550305162 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100cd060 con 0x7f83100d0350 2024-04-25T10:21:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f830f400640 1 -- 172.21.15.45:0/550305162 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492238615 0 0) 0x7f83100ccaa0 con 0x7f831015df60 2024-04-25T10:21:45.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f8314c00640 1 --2- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831006aea0 0x7f831006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f8307e00640 1 --2- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83100d0350 0x7f83100d07b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097c0 tx=0x7f82f801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3153bb3c1b04893d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f830f400640 1 -- 172.21.15.45:0/550305162 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80b0070 con 0x7f83100d0350 2024-04-25T10:21:45.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f830f400640 1 -- 172.21.15.45:0/550305162 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f80ab600 con 0x7f83100d0350 2024-04-25T10:21:45.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.498+0000 7f830f400640 1 -- 172.21.15.45:0/550305162 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80ab9d0 con 0x7f83100d0350 2024-04-25T10:21:45.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83100d0350 msgr2=0x7f83100d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 --2- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83100d0350 0x7f83100d07b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097c0 tx=0x7f82f801c410 comp rx=0 tx=0).stop 2024-04-25T10:21:45.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- 172.21.15.45:0/550305162 shutdown_connections 2024-04-25T10:21:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 --2- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831006aea0 0x7f831006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 --2- 172.21.15.45:0/550305162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83100d0350 0x7f83100d07b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- 172.21.15.45:0/550305162 >> 172.21.15.45:0/550305162 conn(0x7f83100d1910 msgr2=0x7f83100d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- 172.21.15.45:0/550305162 shutdown_connections 2024-04-25T10:21:45.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- 172.21.15.45:0/550305162 wait complete. 2024-04-25T10:21:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 Processor -- start 2024-04-25T10:21:45.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- start start 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f831006aea0 0x7f8310153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 0x7f83100b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831015df60 0x7f83100b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100d0ff0 con 0x7f83100d0350 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100d0e70 con 0x7f831006aea0 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8316c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831016aa20 con 0x7f831015df60 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 0x7f83100b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 0x7f83100b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46472/0 (socket says 172.21.15.45:46472) 2024-04-25T10:21:45.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 -- 172.21.15.45:0/3078750507 learned_addr learned my addr 172.21.15.45:0/3078750507 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f830fe00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f831006aea0 0x7f8310153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8314c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831015df60 0x7f83100b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 -- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831015df60 msgr2=0x7f83100b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831015df60 0x7f83100b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 -- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f831006aea0 msgr2=0x7f8310153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f831006aea0 0x7f8310153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8307e00640 1 -- 172.21.15.45:0/3078750507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100cae40 con 0x7f83100d0350 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f830fe00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f831006aea0 0x7f8310153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.502+0000 7f8314c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831015df60 0x7f83100b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:45.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f8307e00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 0x7f83100b68a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097c0 tx=0x7f82f801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80bc070 con 0x7f83100d0350 2024-04-25T10:21:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f80b1040 con 0x7f83100d0350 2024-04-25T10:21:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83100cb120 con 0x7f83100d0350 2024-04-25T10:21:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80b0050 con 0x7f83100d0350 2024-04-25T10:21:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83100cb6e0 con 0x7f83100d0350 2024-04-25T10:21:45.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831004ef30 con 0x7f83100d0350 2024-04-25T10:21:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82f80bf070 con 0x7f83100d0350 2024-04-25T10:21:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830cc00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82ec066f60 0x7f82ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82f811ea40 con 0x7f83100d0350 2024-04-25T10:21:45.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.506+0000 7f830fe00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82ec066f60 0x7f82ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.510+0000 7f830fe00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82ec066f60 0x7f82ec069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8300000f70 tx=0x7f8300007c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.510+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82f8016090 con 0x7f83100d0350 2024-04-25T10:21:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.670+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83100cab40 con 0x7f82ec066f60 2024-04-25T10:21:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.678+0000 7f830cc00640 1 -- 172.21.15.45:0/3078750507 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251958 (secure 0 0 0) 0x7f83100cab40 con 0x7f82ec066f60 2024-04-25T10:21:45.683 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:45.685 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:21:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82ec066f60 msgr2=0x7f82ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82ec066f60 0x7f82ec069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8300000f70 tx=0x7f8300007c60 comp rx=0 tx=0).stop 2024-04-25T10:21:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 msgr2=0x7f83100b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 0x7f83100b68a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f82f80097c0 tx=0x7f82f801ac50 comp rx=0 tx=0).stop 2024-04-25T10:21:45.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 shutdown_connections 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82ec066f60 0x7f82ec069420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f831015df60 0x7f83100b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83100d0350 0x7f83100b68a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 --2- 172.21.15.45:0/3078750507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f831006aea0 0x7f8310153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 >> 172.21.15.45:0/3078750507 conn(0x7f83100d1910 msgr2=0x7f831006fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 shutdown_connections 2024-04-25T10:21:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.682+0000 7f8316c00640 1 -- 172.21.15.45:0/3078750507 wait complete. 2024-04-25T10:21:45.704 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-25T10:21:44.924130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316080,"kb_used_data":6972,"kb_used_omap":12,"kb_used_meta":308979,"kb_avail":754658640,"statfs":{"total":773094113280,"available":772770447360,"internally_reserved":0,"allocated":7139328,"data_stored":3645256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":316395056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.755256"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220875+0000","last_change":"2024-04-25T10:21:41.212340+0000","last_active":"2024-04-25T10:21:42.220875+0000","last_peered":"2024-04-25T10:21:42.220875+0000","last_clean":"2024-04-25T10:21:42.220875+0000","last_became_active":"2024-04-25T10:21:41.211835+0000","last_became_peered":"2024-04-25T10:21:41.211835+0000","last_unstale":"2024-04-25T10:21:42.220875+0000","last_undegraded":"2024-04-25T10:21:42.220875+0000","last_fullsized":"2024-04-25T10:21:42.220875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:05.259350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220697+0000","last_change":"2024-04-25T10:21:41.208111+0000","last_active":"2024-04-25T10:21:42.220697+0000","last_peered":"2024-04-25T10:21:42.220697+0000","last_clean":"2024-04-25T10:21:42.220697+0000","last_became_active":"2024-04-25T10:21:41.207841+0000","last_became_peered":"2024-04-25T10:21:41.207841+0000","last_unstale":"2024-04-25T10:21:42.220697+0000","last_undegraded":"2024-04-25T10:21:42.220697+0000","last_fullsized":"2024-04-25T10:21:42.220697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:39.616257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202982+0000","last_change":"2024-04-25T10:21:41.206781+0000","last_active":"2024-04-25T10:21:42.202982+0000","last_peered":"2024-04-25T10:21:42.202982+0000","last_clean":"2024-04-25T10:21:42.202982+0000","last_became_active":"2024-04-25T10:21:41.206456+0000","last_became_peered":"2024-04-25T10:21:41.206456+0000","last_unstale":"2024-04-25T10:21:42.202982+0000","last_undegraded":"2024-04-25T10:21:42.202982+0000","last_fullsized":"2024-04-25T10:21:42.202982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:30.050000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221002+0000","last_change":"2024-04-25T10:21:41.215021+0000","last_active":"2024-04-25T10:21:42.221002+0000","last_peered":"2024-04-25T10:21:42.221002+0000","last_clean":"2024-04-25T10:21:42.221002+0000","last_became_active":"2024-04-25T10:21:41.214730+0000","last_became_peered":"2024-04-25T10:21:41.214730+0000","last_unstale":"2024-04-25T10:21:42.221002+0000","last_undegraded":"2024-04-25T10:21:42.221002+0000","last_fullsized":"2024-04-25T10:21:42.221002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:05.397077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220895+0000","last_change":"2024-04-25T10:21:41.207350+0000","last_active":"2024-04-25T10:21:42.220895+0000","last_peered":"2024-04-25T10:21:42.220895+0000","last_clean":"2024-04-25T10:21:42.220895+0000","last_became_active":"2024-04-25T10:21:41.207048+0000","last_became_peered":"2024-04-25T10:21:41.207048+0000","last_unstale":"2024-04-25T10:21:42.220895+0000","last_undegraded":"2024-04-25T10:21:42.220895+0000","last_fullsized":"2024-04-25T10:21:42.220895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:28.894445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220091+0000","last_change":"2024-04-25T10:21:41.218816+0000","last_active":"2024-04-25T10:21:42.220091+0000","last_peered":"2024-04-25T10:21:42.220091+0000","last_clean":"2024-04-25T10:21:42.220091+0000","last_became_active":"2024-04-25T10:21:41.218525+0000","last_became_peered":"2024-04-25T10:21:41.218525+0000","last_unstale":"2024-04-25T10:21:42.220091+0000","last_undegraded":"2024-04-25T10:21:42.220091+0000","last_fullsized":"2024-04-25T10:21:42.220091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:15.787793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221477+0000","last_change":"2024-04-25T10:21:41.195363+0000","last_active":"2024-04-25T10:21:42.221477+0000","last_peered":"2024-04-25T10:21:42.221477+0000","last_clean":"2024-04-25T10:21:42.221477+0000","last_became_active":"2024-04-25T10:21:41.195081+0000","last_became_peered":"2024-04-25T10:21:41.195081+0000","last_unstale":"2024-04-25T10:21:42.221477+0000","last_undegraded":"2024-04-25T10:21:42.221477+0000","last_fullsized":"2024-04-25T10:21:42.221477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:32.078454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202859+0000","last_change":"2024-04-25T10:21:41.206833+0000","last_active":"2024-04-25T10:21:42.202859+0000","last_peered":"2024-04-25T10:21:42.202859+0000","last_clean":"2024-04-25T10:21:42.202859+0000","last_became_active":"2024-04-25T10:21:41.206135+0000","last_became_peered":"2024-04-25T10:21:41.206135+0000","last_unstale":"2024-04-25T10:21:42.202859+0000","last_undegraded":"2024-04-25T10:21:42.202859+0000","last_fullsized":"2024-04-25T10:21:42.202859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:31.909824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222504+0000","last_change":"2024-04-25T10:21:41.211889+0000","last_active":"2024-04-25T10:21:42.222504+0000","last_peered":"2024-04-25T10:21:42.222504+0000","last_clean":"2024-04-25T10:21:42.222504+0000","last_became_active":"2024-04-25T10:21:41.211674+0000","last_became_peered":"2024-04-25T10:21:41.211674+0000","last_unstale":"2024-04-25T10:21:42.222504+0000","last_undegraded":"2024-04-25T10:21:42.222504+0000","last_fullsized":"2024-04-25T10:21:42.222504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.396942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220417+0000","last_change":"2024-04-25T10:21:41.206297+0000","last_active":"2024-04-25T10:21:42.220417+0000","last_peered":"2024-04-25T10:21:42.220417+0000","last_clean":"2024-04-25T10:21:42.220417+0000","last_became_active":"2024-04-25T10:21:41.205625+0000","last_became_peered":"2024-04-25T10:21:41.205625+0000","last_unstale":"2024-04-25T10:21:42.220417+0000","last_undegraded":"2024-04-25T10:21:42.220417+0000","last_fullsized":"2024-04-25T10:21:42.220417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:06.849937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203579+0000","last_change":"2024-04-25T10:21:41.212278+0000","last_active":"2024-04-25T10:21:42.203579+0000","last_peered":"2024-04-25T10:21:42.203579+0000","last_clean":"2024-04-25T10:21:42.203579+0000","last_became_active":"2024-04-25T10:21:41.211922+0000","last_became_peered":"2024-04-25T10:21:41.211922+0000","last_unstale":"2024-04-25T10:21:42.203579+0000","last_undegraded":"2024-04-25T10:21:42.203579+0000","last_fullsized":"2024-04-25T10:21:42.203579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:27.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221391+0000","last_change":"2024-04-25T10:21:41.211113+0000","last_active":"2024-04-25T10:21:42.221391+0000","last_peered":"2024-04-25T10:21:42.221391+0000","last_clean":"2024-04-25T10:21:42.221391+0000","last_became_active":"2024-04-25T10:21:41.210833+0000","last_became_peered":"2024-04-25T10:21:41.210833+0000","last_unstale":"2024-04-25T10:21:42.221391+0000","last_undegraded":"2024-04-25T10:21:42.221391+0000","last_fullsized":"2024-04-25T10:21:42.221391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:05.133982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222311+0000","last_change":"2024-04-25T10:21:41.209511+0000","last_active":"2024-04-25T10:21:42.222311+0000","last_peered":"2024-04-25T10:21:42.222311+0000","last_clean":"2024-04-25T10:21:42.222311+0000","last_became_active":"2024-04-25T10:21:41.208333+0000","last_became_peered":"2024-04-25T10:21:41.208333+0000","last_unstale":"2024-04-25T10:21:42.222311+0000","last_undegraded":"2024-04-25T10:21:42.222311+0000","last_fullsized":"2024-04-25T10:21:42.222311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:47.847107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202788+0000","last_change":"2024-04-25T10:21:41.206327+0000","last_active":"2024-04-25T10:21:42.202788+0000","last_peered":"2024-04-25T10:21:42.202788+0000","last_clean":"2024-04-25T10:21:42.202788+0000","last_became_active":"2024-04-25T10:21:41.206007+0000","last_became_peered":"2024-04-25T10:21:41.206007+0000","last_unstale":"2024-04-25T10:21:42.202788+0000","last_undegraded":"2024-04-25T10:21:42.202788+0000","last_fullsized":"2024-04-25T10:21:42.202788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:41.542390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222110+0000","last_change":"2024-04-25T10:21:41.210444+0000","last_active":"2024-04-25T10:21:42.222110+0000","last_peered":"2024-04-25T10:21:42.222110+0000","last_clean":"2024-04-25T10:21:42.222110+0000","last_became_active":"2024-04-25T10:21:41.209106+0000","last_became_peered":"2024-04-25T10:21:41.209106+0000","last_unstale":"2024-04-25T10:21:42.222110+0000","last_undegraded":"2024-04-25T10:21:42.222110+0000","last_fullsized":"2024-04-25T10:21:42.222110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:56.037376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219686+0000","last_change":"2024-04-25T10:21:41.206620+0000","last_active":"2024-04-25T10:21:42.219686+0000","last_peered":"2024-04-25T10:21:42.219686+0000","last_clean":"2024-04-25T10:21:42.219686+0000","last_became_active":"2024-04-25T10:21:41.206300+0000","last_became_peered":"2024-04-25T10:21:41.206300+0000","last_unstale":"2024-04-25T10:21:42.219686+0000","last_undegraded":"2024-04-25T10:21:42.219686+0000","last_fullsized":"2024-04-25T10:21:42.219686+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:34.174885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221033+0000","last_change":"2024-04-25T10:21:41.210846+0000","last_active":"2024-04-25T10:21:42.221033+0000","last_peered":"2024-04-25T10:21:42.221033+0000","last_clean":"2024-04-25T10:21:42.221033+0000","last_became_active":"2024-04-25T10:21:41.207207+0000","last_became_peered":"2024-04-25T10:21:41.207207+0000","last_unstale":"2024-04-25T10:21:42.221033+0000","last_undegraded":"2024-04-25T10:21:42.221033+0000","last_fullsized":"2024-04-25T10:21:42.221033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:21.681543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203137+0000","last_change":"2024-04-25T10:21:41.203604+0000","last_active":"2024-04-25T10:21:42.203137+0000","last_peered":"2024-04-25T10:21:42.203137+0000","last_clean":"2024-04-25T10:21:42.203137+0000","last_became_active":"2024-04-25T10:21:41.203236+0000","last_became_peered":"2024-04-25T10:21:41.203236+0000","last_unstale":"2024-04-25T10:21:42.203137+0000","last_undegraded":"2024-04-25T10:21:42.203137+0000","last_fullsized":"2024-04-25T10:21:42.203137+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.849653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220598+0000","last_change":"2024-04-25T10:21:41.219420+0000","last_active":"2024-04-25T10:21:42.220598+0000","last_peered":"2024-04-25T10:21:42.220598+0000","last_clean":"2024-04-25T10:21:42.220598+0000","last_became_active":"2024-04-25T10:21:41.219116+0000","last_became_peered":"2024-04-25T10:21:41.219116+0000","last_unstale":"2024-04-25T10:21:42.220598+0000","last_undegraded":"2024-04-25T10:21:42.220598+0000","last_fullsized":"2024-04-25T10:21:42.220598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:22.475805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203004+0000","last_change":"2024-04-25T10:21:41.212213+0000","last_active":"2024-04-25T10:21:42.203004+0000","last_peered":"2024-04-25T10:21:42.203004+0000","last_clean":"2024-04-25T10:21:42.203004+0000","last_became_active":"2024-04-25T10:21:41.211847+0000","last_became_peered":"2024-04-25T10:21:41.211847+0000","last_unstale":"2024-04-25T10:21:42.203004+0000","last_undegraded":"2024-04-25T10:21:42.203004+0000","last_fullsized":"2024-04-25T10:21:42.203004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:37.486523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222619+0000","last_change":"2024-04-25T10:21:41.216433+0000","last_active":"2024-04-25T10:21:42.222619+0000","last_peered":"2024-04-25T10:21:42.222619+0000","last_clean":"2024-04-25T10:21:42.222619+0000","last_became_active":"2024-04-25T10:21:41.216161+0000","last_became_peered":"2024-04-25T10:21:41.216161+0000","last_unstale":"2024-04-25T10:21:42.222619+0000","last_undegraded":"2024-04-25T10:21:42.222619+0000","last_fullsized":"2024-04-25T10:21:42.222619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:15.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203954+0000","last_change":"2024-04-25T10:21:41.206838+0000","last_active":"2024-04-25T10:21:42.203954+0000","last_peered":"2024-04-25T10:21:42.203954+0000","last_clean":"2024-04-25T10:21:42.203954+0000","last_became_active":"2024-04-25T10:21:41.206177+0000","last_became_peered":"2024-04-25T10:21:41.206177+0000","last_unstale":"2024-04-25T10:21:42.203954+0000","last_undegraded":"2024-04-25T10:21:42.203954+0000","last_fullsized":"2024-04-25T10:21:42.203954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:12.761480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221615+0000","last_change":"2024-04-25T10:21:41.209818+0000","last_active":"2024-04-25T10:21:42.221615+0000","last_peered":"2024-04-25T10:21:42.221615+0000","last_clean":"2024-04-25T10:21:42.221615+0000","last_became_active":"2024-04-25T10:21:41.209524+0000","last_became_peered":"2024-04-25T10:21:41.209524+0000","last_unstale":"2024-04-25T10:21:42.221615+0000","last_undegraded":"2024-04-25T10:21:42.221615+0000","last_fullsized":"2024-04-25T10:21:42.221615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:59.050656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220634+0000","last_change":"2024-04-25T10:21:41.208575+0000","last_active":"2024-04-25T10:21:42.220634+0000","last_peered":"2024-04-25T10:21:42.220634+0000","last_clean":"2024-04-25T10:21:42.220634+0000","last_became_active":"2024-04-25T10:21:41.208286+0000","last_became_peered":"2024-04-25T10:21:41.208286+0000","last_unstale":"2024-04-25T10:21:42.220634+0000","last_undegraded":"2024-04-25T10:21:42.220634+0000","last_fullsized":"2024-04-25T10:21:42.220634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:13.496108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220219+0000","last_change":"2024-04-25T10:21:41.223922+0000","last_active":"2024-04-25T10:21:42.220219+0000","last_peered":"2024-04-25T10:21:42.220219+0000","last_clean":"2024-04-25T10:21:42.220219+0000","last_became_active":"2024-04-25T10:21:41.223620+0000","last_became_peered":"2024-04-25T10:21:41.223620+0000","last_unstale":"2024-04-25T10:21:42.220219+0000","last_undegraded":"2024-04-25T10:21:42.220219+0000","last_fullsized":"2024-04-25T10:21:42.220219+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:37.823772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222545+0000","last_change":"2024-04-25T10:21:41.208628+0000","last_active":"2024-04-25T10:21:42.222545+0000","last_peered":"2024-04-25T10:21:42.222545+0000","last_clean":"2024-04-25T10:21:42.222545+0000","last_became_active":"2024-04-25T10:21:41.205592+0000","last_became_peered":"2024-04-25T10:21:41.205592+0000","last_unstale":"2024-04-25T10:21:42.222545+0000","last_undegraded":"2024-04-25T10:21:42.222545+0000","last_fullsized":"2024-04-25T10:21:42.222545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:33.499184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220833+0000","last_change":"2024-04-25T10:21:41.215862+0000","last_active":"2024-04-25T10:21:42.220833+0000","last_peered":"2024-04-25T10:21:42.220833+0000","last_clean":"2024-04-25T10:21:42.220833+0000","last_became_active":"2024-04-25T10:21:41.215510+0000","last_became_peered":"2024-04-25T10:21:41.215510+0000","last_unstale":"2024-04-25T10:21:42.220833+0000","last_undegraded":"2024-04-25T10:21:42.220833+0000","last_fullsized":"2024-04-25T10:21:42.220833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:44.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204247+0000","last_change":"2024-04-25T10:21:41.214827+0000","last_active":"2024-04-25T10:21:42.204247+0000","last_peered":"2024-04-25T10:21:42.204247+0000","last_clean":"2024-04-25T10:21:42.204247+0000","last_became_active":"2024-04-25T10:21:41.214163+0000","last_became_peered":"2024-04-25T10:21:41.214163+0000","last_unstale":"2024-04-25T10:21:42.204247+0000","last_undegraded":"2024-04-25T10:21:42.204247+0000","last_fullsized":"2024-04-25T10:21:42.204247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:07.422717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204181+0000","last_change":"2024-04-25T10:21:41.199841+0000","last_active":"2024-04-25T10:21:42.204181+0000","last_peered":"2024-04-25T10:21:42.204181+0000","last_clean":"2024-04-25T10:21:42.204181+0000","last_became_active":"2024-04-25T10:21:41.198765+0000","last_became_peered":"2024-04-25T10:21:41.198765+0000","last_unstale":"2024-04-25T10:21:42.204181+0000","last_undegraded":"2024-04-25T10:21:42.204181+0000","last_fullsized":"2024-04-25T10:21:42.204181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:21.239625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220604+0000","last_change":"2024-04-25T10:21:41.200274+0000","last_active":"2024-04-25T10:21:42.220604+0000","last_peered":"2024-04-25T10:21:42.220604+0000","last_clean":"2024-04-25T10:21:42.220604+0000","last_became_active":"2024-04-25T10:21:41.199183+0000","last_became_peered":"2024-04-25T10:21:41.199183+0000","last_unstale":"2024-04-25T10:21:42.220604+0000","last_undegraded":"2024-04-25T10:21:42.220604+0000","last_fullsized":"2024-04-25T10:21:42.220604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:00.753360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219729+0000","last_change":"2024-04-25T10:21:41.221460+0000","last_active":"2024-04-25T10:21:42.219729+0000","last_peered":"2024-04-25T10:21:42.219729+0000","last_clean":"2024-04-25T10:21:42.219729+0000","last_became_active":"2024-04-25T10:21:41.221138+0000","last_became_peered":"2024-04-25T10:21:41.221138+0000","last_unstale":"2024-04-25T10:21:42.219729+0000","last_undegraded":"2024-04-25T10:21:42.219729+0000","last_fullsized":"2024-04-25T10:21:42.219729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:50.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202859+0000","last_change":"2024-04-25T10:21:41.208704+0000","last_active":"2024-04-25T10:21:42.202859+0000","last_peered":"2024-04-25T10:21:42.202859+0000","last_clean":"2024-04-25T10:21:42.202859+0000","last_became_active":"2024-04-25T10:21:41.208386+0000","last_became_peered":"2024-04-25T10:21:41.208386+0000","last_unstale":"2024-04-25T10:21:42.202859+0000","last_undegraded":"2024-04-25T10:21:42.202859+0000","last_fullsized":"2024-04-25T10:21:42.202859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.951240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220500+0000","last_change":"2024-04-25T10:21:41.204958+0000","last_active":"2024-04-25T10:21:42.220500+0000","last_peered":"2024-04-25T10:21:42.220500+0000","last_clean":"2024-04-25T10:21:42.220500+0000","last_became_active":"2024-04-25T10:21:41.204281+0000","last_became_peered":"2024-04-25T10:21:41.204281+0000","last_unstale":"2024-04-25T10:21:42.220500+0000","last_undegraded":"2024-04-25T10:21:42.220500+0000","last_fullsized":"2024-04-25T10:21:42.220500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:48.353976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203884+0000","last_change":"2024-04-25T10:21:40.177066+0000","last_active":"2024-04-25T10:21:42.203884+0000","last_peered":"2024-04-25T10:21:42.203884+0000","last_clean":"2024-04-25T10:21:42.203884+0000","last_became_active":"2024-04-25T10:21:40.176671+0000","last_became_peered":"2024-04-25T10:21:40.176671+0000","last_unstale":"2024-04-25T10:21:42.203884+0000","last_undegraded":"2024-04-25T10:21:42.203884+0000","last_fullsized":"2024-04-25T10:21:42.203884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:59.242042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221068+0000","last_change":"2024-04-25T10:21:41.212112+0000","last_active":"2024-04-25T10:21:42.221068+0000","last_peered":"2024-04-25T10:21:42.221068+0000","last_clean":"2024-04-25T10:21:42.221068+0000","last_became_active":"2024-04-25T10:21:41.211385+0000","last_became_peered":"2024-04-25T10:21:41.211385+0000","last_unstale":"2024-04-25T10:21:42.221068+0000","last_undegraded":"2024-04-25T10:21:42.221068+0000","last_fullsized":"2024-04-25T10:21:42.221068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:16.576949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220808+0000","last_change":"2024-04-25T10:21:40.196085+0000","last_active":"2024-04-25T10:21:42.220808+0000","last_peered":"2024-04-25T10:21:42.220808+0000","last_clean":"2024-04-25T10:21:42.220808+0000","last_became_active":"2024-04-25T10:21:40.195474+0000","last_became_peered":"2024-04-25T10:21:40.195474+0000","last_unstale":"2024-04-25T10:21:42.220808+0000","last_undegraded":"2024-04-25T10:21:42.220808+0000","last_fullsized":"2024-04-25T10:21:42.220808+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:16.619511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204298+0000","last_change":"2024-04-25T10:21:41.215196+0000","last_active":"2024-04-25T10:21:42.204298+0000","last_peered":"2024-04-25T10:21:42.204298+0000","last_clean":"2024-04-25T10:21:42.204298+0000","last_became_active":"2024-04-25T10:21:41.214839+0000","last_became_peered":"2024-04-25T10:21:41.214839+0000","last_unstale":"2024-04-25T10:21:42.204298+0000","last_undegraded":"2024-04-25T10:21:42.204298+0000","last_fullsized":"2024-04-25T10:21:42.204298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:30.349114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220619+0000","last_change":"2024-04-25T10:21:40.202185+0000","last_active":"2024-04-25T10:21:42.220619+0000","last_peered":"2024-04-25T10:21:42.220619+0000","last_clean":"2024-04-25T10:21:42.220619+0000","last_became_active":"2024-04-25T10:21:40.201908+0000","last_became_peered":"2024-04-25T10:21:40.201908+0000","last_unstale":"2024-04-25T10:21:42.220619+0000","last_undegraded":"2024-04-25T10:21:42.220619+0000","last_fullsized":"2024-04-25T10:21:42.220619+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:21.464946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220403+0000","last_change":"2024-04-25T10:21:41.206287+0000","last_active":"2024-04-25T10:21:42.220403+0000","last_peered":"2024-04-25T10:21:42.220403+0000","last_clean":"2024-04-25T10:21:42.220403+0000","last_became_active":"2024-04-25T10:21:41.205981+0000","last_became_peered":"2024-04-25T10:21:41.205981+0000","last_unstale":"2024-04-25T10:21:42.220403+0000","last_undegraded":"2024-04-25T10:21:42.220403+0000","last_fullsized":"2024-04-25T10:21:42.220403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:15.136959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221095+0000","last_change":"2024-04-25T10:21:40.193443+0000","last_active":"2024-04-25T10:21:42.221095+0000","last_peered":"2024-04-25T10:21:42.221095+0000","last_clean":"2024-04-25T10:21:42.221095+0000","last_became_active":"2024-04-25T10:21:40.193096+0000","last_became_peered":"2024-04-25T10:21:40.193096+0000","last_unstale":"2024-04-25T10:21:42.221095+0000","last_undegraded":"2024-04-25T10:21:42.221095+0000","last_fullsized":"2024-04-25T10:21:42.221095+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:51.990070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203229+0000","last_change":"2024-04-25T10:21:41.192606+0000","last_active":"2024-04-25T10:21:42.203229+0000","last_peered":"2024-04-25T10:21:42.203229+0000","last_clean":"2024-04-25T10:21:42.203229+0000","last_became_active":"2024-04-25T10:21:41.192240+0000","last_became_peered":"2024-04-25T10:21:41.192240+0000","last_unstale":"2024-04-25T10:21:42.203229+0000","last_undegraded":"2024-04-25T10:21:42.203229+0000","last_fullsized":"2024-04-25T10:21:42.203229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:40.449197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202883+0000","last_change":"2024-04-25T10:21:40.186905+0000","last_active":"2024-04-25T10:21:42.202883+0000","last_peered":"2024-04-25T10:21:42.202883+0000","last_clean":"2024-04-25T10:21:42.202883+0000","last_became_active":"2024-04-25T10:21:40.185516+0000","last_became_peered":"2024-04-25T10:21:40.185516+0000","last_unstale":"2024-04-25T10:21:42.202883+0000","last_undegraded":"2024-04-25T10:21:42.202883+0000","last_fullsized":"2024-04-25T10:21:42.202883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:41.704477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203224+0000","last_change":"2024-04-25T10:21:41.208531+0000","last_active":"2024-04-25T10:21:42.203224+0000","last_peered":"2024-04-25T10:21:42.203224+0000","last_clean":"2024-04-25T10:21:42.203224+0000","last_became_active":"2024-04-25T10:21:41.208252+0000","last_became_peered":"2024-04-25T10:21:41.208252+0000","last_unstale":"2024-04-25T10:21:42.203224+0000","last_undegraded":"2024-04-25T10:21:42.203224+0000","last_fullsized":"2024-04-25T10:21:42.203224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:15.828723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220210+0000","last_change":"2024-04-25T10:21:40.201628+0000","last_active":"2024-04-25T10:21:42.220210+0000","last_peered":"2024-04-25T10:21:42.220210+0000","last_clean":"2024-04-25T10:21:42.220210+0000","last_became_active":"2024-04-25T10:21:40.201263+0000","last_became_peered":"2024-04-25T10:21:40.201263+0000","last_unstale":"2024-04-25T10:21:42.220210+0000","last_undegraded":"2024-04-25T10:21:42.220210+0000","last_fullsized":"2024-04-25T10:21:42.220210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:43.455535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202832+0000","last_change":"2024-04-25T10:21:41.198686+0000","last_active":"2024-04-25T10:21:42.202832+0000","last_peered":"2024-04-25T10:21:42.202832+0000","last_clean":"2024-04-25T10:21:42.202832+0000","last_became_active":"2024-04-25T10:21:41.198390+0000","last_became_peered":"2024-04-25T10:21:41.198390+0000","last_unstale":"2024-04-25T10:21:42.202832+0000","last_undegraded":"2024-04-25T10:21:42.202832+0000","last_fullsized":"2024-04-25T10:21:42.202832+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:32.035479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221107+0000","last_change":"2024-04-25T10:21:40.196432+0000","last_active":"2024-04-25T10:21:42.221107+0000","last_peered":"2024-04-25T10:21:42.221107+0000","last_clean":"2024-04-25T10:21:42.221107+0000","last_became_active":"2024-04-25T10:21:40.196148+0000","last_became_peered":"2024-04-25T10:21:40.196148+0000","last_unstale":"2024-04-25T10:21:42.221107+0000","last_undegraded":"2024-04-25T10:21:42.221107+0000","last_fullsized":"2024-04-25T10:21:42.221107+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:23.242048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220717+0000","last_change":"2024-04-25T10:21:41.205986+0000","last_active":"2024-04-25T10:21:42.220717+0000","last_peered":"2024-04-25T10:21:42.220717+0000","last_clean":"2024-04-25T10:21:42.220717+0000","last_became_active":"2024-04-25T10:21:41.204808+0000","last_became_peered":"2024-04-25T10:21:41.204808+0000","last_unstale":"2024-04-25T10:21:42.220717+0000","last_undegraded":"2024-04-25T10:21:42.220717+0000","last_fullsized":"2024-04-25T10:21:42.220717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:33.499184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221108+0000","last_change":"2024-04-25T10:21:40.194950+0000","last_active":"2024-04-25T10:21:42.221108+0000","last_peered":"2024-04-25T10:21:42.221108+0000","last_clean":"2024-04-25T10:21:42.221108+0000","last_became_active":"2024-04-25T10:21:40.194767+0000","last_became_peered":"2024-04-25T10:21:40.194767+0000","last_unstale":"2024-04-25T10:21:42.221108+0000","last_undegraded":"2024-04-25T10:21:42.221108+0000","last_fullsized":"2024-04-25T10:21:42.221108+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:02.827283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222658+0000","last_change":"2024-04-25T10:21:41.219108+0000","last_active":"2024-04-25T10:21:42.222658+0000","last_peered":"2024-04-25T10:21:42.222658+0000","last_clean":"2024-04-25T10:21:42.222658+0000","last_became_active":"2024-04-25T10:21:41.218738+0000","last_became_peered":"2024-04-25T10:21:41.218738+0000","last_unstale":"2024-04-25T10:21:42.222658+0000","last_undegraded":"2024-04-25T10:21:42.222658+0000","last_fullsized":"2024-04-25T10:21:42.222658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:15.787793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222131+0000","last_change":"2024-04-25T10:21:40.197183+0000","last_active":"2024-04-25T10:21:42.222131+0000","last_peered":"2024-04-25T10:21:42.222131+0000","last_clean":"2024-04-25T10:21:42.222131+0000","last_became_active":"2024-04-25T10:21:40.196934+0000","last_became_peered":"2024-04-25T10:21:40.196934+0000","last_unstale":"2024-04-25T10:21:42.222131+0000","last_undegraded":"2024-04-25T10:21:42.222131+0000","last_fullsized":"2024-04-25T10:21:42.222131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:16.619511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203234+0000","last_change":"2024-04-25T10:21:41.212995+0000","last_active":"2024-04-25T10:21:42.203234+0000","last_peered":"2024-04-25T10:21:42.203234+0000","last_clean":"2024-04-25T10:21:42.203234+0000","last_became_active":"2024-04-25T10:21:41.212730+0000","last_became_peered":"2024-04-25T10:21:41.212730+0000","last_unstale":"2024-04-25T10:21:42.203234+0000","last_undegraded":"2024-04-25T10:21:42.203234+0000","last_fullsized":"2024-04-25T10:21:42.203234+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:30.657137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203984+0000","last_change":"2024-04-25T10:21:40.196263+0000","last_active":"2024-04-25T10:21:42.203984+0000","last_peered":"2024-04-25T10:21:42.203984+0000","last_clean":"2024-04-25T10:21:42.203984+0000","last_became_active":"2024-04-25T10:21:40.193856+0000","last_became_peered":"2024-04-25T10:21:40.193856+0000","last_unstale":"2024-04-25T10:21:42.203984+0000","last_undegraded":"2024-04-25T10:21:42.203984+0000","last_fullsized":"2024-04-25T10:21:42.203984+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:04.248491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204436+0000","last_change":"2024-04-25T10:21:41.215056+0000","last_active":"2024-04-25T10:21:42.204436+0000","last_peered":"2024-04-25T10:21:42.204436+0000","last_clean":"2024-04-25T10:21:42.204436+0000","last_became_active":"2024-04-25T10:21:41.214536+0000","last_became_peered":"2024-04-25T10:21:41.214536+0000","last_unstale":"2024-04-25T10:21:42.204436+0000","last_undegraded":"2024-04-25T10:21:42.204436+0000","last_fullsized":"2024-04-25T10:21:42.204436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:15.245969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221220+0000","last_change":"2024-04-25T10:21:40.198205+0000","last_active":"2024-04-25T10:21:42.221220+0000","last_peered":"2024-04-25T10:21:42.221220+0000","last_clean":"2024-04-25T10:21:42.221220+0000","last_became_active":"2024-04-25T10:21:40.197954+0000","last_became_peered":"2024-04-25T10:21:40.197954+0000","last_unstale":"2024-04-25T10:21:42.221220+0000","last_undegraded":"2024-04-25T10:21:42.221220+0000","last_fullsized":"2024-04-25T10:21:42.221220+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:39.438339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221062+0000","last_change":"2024-04-25T10:21:41.198083+0000","last_active":"2024-04-25T10:21:42.221062+0000","last_peered":"2024-04-25T10:21:42.221062+0000","last_clean":"2024-04-25T10:21:42.221062+0000","last_became_active":"2024-04-25T10:21:41.196937+0000","last_became_peered":"2024-04-25T10:21:41.196937+0000","last_unstale":"2024-04-25T10:21:42.221062+0000","last_undegraded":"2024-04-25T10:21:42.221062+0000","last_fullsized":"2024-04-25T10:21:42.221062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:50.865221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222445+0000","last_change":"2024-04-25T10:21:40.188594+0000","last_active":"2024-04-25T10:21:42.222445+0000","last_peered":"2024-04-25T10:21:42.222445+0000","last_clean":"2024-04-25T10:21:42.222445+0000","last_became_active":"2024-04-25T10:21:40.188126+0000","last_became_peered":"2024-04-25T10:21:40.188126+0000","last_unstale":"2024-04-25T10:21:42.222445+0000","last_undegraded":"2024-04-25T10:21:42.222445+0000","last_fullsized":"2024-04-25T10:21:42.222445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:01.501495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221101+0000","last_change":"2024-04-25T10:21:41.201759+0000","last_active":"2024-04-25T10:21:42.221101+0000","last_peered":"2024-04-25T10:21:42.221101+0000","last_clean":"2024-04-25T10:21:42.221101+0000","last_became_active":"2024-04-25T10:21:41.201482+0000","last_became_peered":"2024-04-25T10:21:41.201482+0000","last_unstale":"2024-04-25T10:21:42.221101+0000","last_undegraded":"2024-04-25T10:21:42.221101+0000","last_fullsized":"2024-04-25T10:21:42.221101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.081194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204087+0000","last_change":"2024-04-25T10:21:40.192607+0000","last_active":"2024-04-25T10:21:42.204087+0000","last_peered":"2024-04-25T10:21:42.204087+0000","last_clean":"2024-04-25T10:21:42.204087+0000","last_became_active":"2024-04-25T10:21:40.192317+0000","last_became_peered":"2024-04-25T10:21:40.192317+0000","last_unstale":"2024-04-25T10:21:42.204087+0000","last_undegraded":"2024-04-25T10:21:42.204087+0000","last_fullsized":"2024-04-25T10:21:42.204087+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:27.883586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202723+0000","last_change":"2024-04-25T10:21:41.206297+0000","last_active":"2024-04-25T10:21:42.202723+0000","last_peered":"2024-04-25T10:21:42.202723+0000","last_clean":"2024-04-25T10:21:42.202723+0000","last_became_active":"2024-04-25T10:21:41.205953+0000","last_became_peered":"2024-04-25T10:21:41.205953+0000","last_unstale":"2024-04-25T10:21:42.202723+0000","last_undegraded":"2024-04-25T10:21:42.202723+0000","last_fullsized":"2024-04-25T10:21:42.202723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:32.394769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220716+0000","last_change":"2024-04-25T10:21:40.195906+0000","last_active":"2024-04-25T10:21:42.220716+0000","last_peered":"2024-04-25T10:21:42.220716+0000","last_clean":"2024-04-25T10:21:42.220716+0000","last_became_active":"2024-04-25T10:21:40.190663+0000","last_became_peered":"2024-04-25T10:21:40.190663+0000","last_unstale":"2024-04-25T10:21:42.220716+0000","last_undegraded":"2024-04-25T10:21:42.220716+0000","last_fullsized":"2024-04-25T10:21:42.220716+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:30.898965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220812+0000","last_change":"2024-04-25T10:21:41.201179+0000","last_active":"2024-04-25T10:21:42.220812+0000","last_peered":"2024-04-25T10:21:42.220812+0000","last_clean":"2024-04-25T10:21:42.220812+0000","last_became_active":"2024-04-25T10:21:41.199882+0000","last_became_peered":"2024-04-25T10:21:41.199882+0000","last_unstale":"2024-04-25T10:21:42.220812+0000","last_undegraded":"2024-04-25T10:21:42.220812+0000","last_fullsized":"2024-04-25T10:21:42.220812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:51.486160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221843+0000","last_change":"2024-04-25T10:21:40.197623+0000","last_active":"2024-04-25T10:21:42.221843+0000","last_peered":"2024-04-25T10:21:42.221843+0000","last_clean":"2024-04-25T10:21:42.221843+0000","last_became_active":"2024-04-25T10:21:40.196276+0000","last_became_peered":"2024-04-25T10:21:40.196276+0000","last_unstale":"2024-04-25T10:21:42.221843+0000","last_undegraded":"2024-04-25T10:21:42.221843+0000","last_fullsized":"2024-04-25T10:21:42.221843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:18.758575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221753+0000","last_change":"2024-04-25T10:21:41.209510+0000","last_active":"2024-04-25T10:21:42.221753+0000","last_peered":"2024-04-25T10:21:42.221753+0000","last_clean":"2024-04-25T10:21:42.221753+0000","last_became_active":"2024-04-25T10:21:41.209211+0000","last_became_peered":"2024-04-25T10:21:41.209211+0000","last_unstale":"2024-04-25T10:21:42.221753+0000","last_undegraded":"2024-04-25T10:21:42.221753+0000","last_fullsized":"2024-04-25T10:21:42.221753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:00.812567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":22,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220781+0000","last_change":"2024-04-25T10:21:40.190478+0000","last_active":"2024-04-25T10:21:42.220781+0000","last_peered":"2024-04-25T10:21:42.220781+0000","last_clean":"2024-04-25T10:21:42.220781+0000","last_became_active":"2024-04-25T10:21:40.190156+0000","last_became_peered":"2024-04-25T10:21:40.190156+0000","last_unstale":"2024-04-25T10:21:42.220781+0000","last_undegraded":"2024-04-25T10:21:42.220781+0000","last_fullsized":"2024-04-25T10:21:42.220781+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:54.141917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203224+0000","last_change":"2024-04-25T10:21:41.198753+0000","last_active":"2024-04-25T10:21:42.203224+0000","last_peered":"2024-04-25T10:21:42.203224+0000","last_clean":"2024-04-25T10:21:42.203224+0000","last_became_active":"2024-04-25T10:21:41.198469+0000","last_became_peered":"2024-04-25T10:21:41.198469+0000","last_unstale":"2024-04-25T10:21:42.203224+0000","last_undegraded":"2024-04-25T10:21:42.203224+0000","last_fullsized":"2024-04-25T10:21:42.203224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:34.347853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220330+0000","last_change":"2024-04-25T10:21:40.199590+0000","last_active":"2024-04-25T10:21:42.220330+0000","last_peered":"2024-04-25T10:21:42.220330+0000","last_clean":"2024-04-25T10:21:42.220330+0000","last_became_active":"2024-04-25T10:21:40.198614+0000","last_became_peered":"2024-04-25T10:21:40.198614+0000","last_unstale":"2024-04-25T10:21:42.220330+0000","last_undegraded":"2024-04-25T10:21:42.220330+0000","last_fullsized":"2024-04-25T10:21:42.220330+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:06.527101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203210+0000","last_change":"2024-04-25T10:21:41.212992+0000","last_active":"2024-04-25T10:21:42.203210+0000","last_peered":"2024-04-25T10:21:42.203210+0000","last_clean":"2024-04-25T10:21:42.203210+0000","last_became_active":"2024-04-25T10:21:41.212625+0000","last_became_peered":"2024-04-25T10:21:41.212625+0000","last_unstale":"2024-04-25T10:21:42.203210+0000","last_undegraded":"2024-04-25T10:21:42.203210+0000","last_fullsized":"2024-04-25T10:21:42.203210+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:31.483195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204061+0000","last_change":"2024-04-25T10:21:40.196071+0000","last_active":"2024-04-25T10:21:42.204061+0000","last_peered":"2024-04-25T10:21:42.204061+0000","last_clean":"2024-04-25T10:21:42.204061+0000","last_became_active":"2024-04-25T10:21:40.193492+0000","last_became_peered":"2024-04-25T10:21:40.193492+0000","last_unstale":"2024-04-25T10:21:42.204061+0000","last_undegraded":"2024-04-25T10:21:42.204061+0000","last_fullsized":"2024-04-25T10:21:42.204061+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:20.670684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220093+0000","last_change":"2024-04-25T10:21:41.210304+0000","last_active":"2024-04-25T10:21:42.220093+0000","last_peered":"2024-04-25T10:21:42.220093+0000","last_clean":"2024-04-25T10:21:42.220093+0000","last_became_active":"2024-04-25T10:21:41.209988+0000","last_became_peered":"2024-04-25T10:21:41.209988+0000","last_unstale":"2024-04-25T10:21:42.220093+0000","last_undegraded":"2024-04-25T10:21:42.220093+0000","last_fullsized":"2024-04-25T10:21:42.220093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:19.396942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203116+0000","last_change":"2024-04-25T10:21:40.201434+0000","last_active":"2024-04-25T10:21:42.203116+0000","last_peered":"2024-04-25T10:21:42.203116+0000","last_clean":"2024-04-25T10:21:42.203116+0000","last_became_active":"2024-04-25T10:21:40.200888+0000","last_became_peered":"2024-04-25T10:21:40.200888+0000","last_unstale":"2024-04-25T10:21:42.203116+0000","last_undegraded":"2024-04-25T10:21:42.203116+0000","last_fullsized":"2024-04-25T10:21:42.203116+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:33.164026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221195+0000","last_change":"2024-04-25T10:21:41.215278+0000","last_active":"2024-04-25T10:21:42.221195+0000","last_peered":"2024-04-25T10:21:42.221195+0000","last_clean":"2024-04-25T10:21:42.221195+0000","last_became_active":"2024-04-25T10:21:41.214964+0000","last_became_peered":"2024-04-25T10:21:41.214964+0000","last_unstale":"2024-04-25T10:21:42.221195+0000","last_undegraded":"2024-04-25T10:21:42.221195+0000","last_fullsized":"2024-04-25T10:21:42.221195+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:35.721778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.204368+0000","last_change":"2024-04-25T10:21:40.199586+0000","last_active":"2024-04-25T10:21:42.204368+0000","last_peered":"2024-04-25T10:21:42.204368+0000","last_clean":"2024-04-25T10:21:42.204368+0000","last_became_active":"2024-04-25T10:21:40.199305+0000","last_became_peered":"2024-04-25T10:21:40.199305+0000","last_unstale":"2024-04-25T10:21:42.204368+0000","last_undegraded":"2024-04-25T10:21:42.204368+0000","last_fullsized":"2024-04-25T10:21:42.204368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:47.343118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221194+0000","last_change":"2024-04-25T10:21:41.211428+0000","last_active":"2024-04-25T10:21:42.221194+0000","last_peered":"2024-04-25T10:21:42.221194+0000","last_clean":"2024-04-25T10:21:42.221194+0000","last_became_active":"2024-04-25T10:21:41.211142+0000","last_became_peered":"2024-04-25T10:21:41.211142+0000","last_unstale":"2024-04-25T10:21:42.221194+0000","last_undegraded":"2024-04-25T10:21:42.221194+0000","last_fullsized":"2024-04-25T10:21:42.221194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:50.881767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203339+0000","last_change":"2024-04-25T10:21:40.191205+0000","last_active":"2024-04-25T10:21:42.203339+0000","last_peered":"2024-04-25T10:21:42.203339+0000","last_clean":"2024-04-25T10:21:42.203339+0000","last_became_active":"2024-04-25T10:21:40.190776+0000","last_became_peered":"2024-04-25T10:21:40.190776+0000","last_unstale":"2024-04-25T10:21:42.203339+0000","last_undegraded":"2024-04-25T10:21:42.203339+0000","last_fullsized":"2024-04-25T10:21:42.203339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:09.663619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203958+0000","last_change":"2024-04-25T10:21:41.199613+0000","last_active":"2024-04-25T10:21:42.203958+0000","last_peered":"2024-04-25T10:21:42.203958+0000","last_clean":"2024-04-25T10:21:42.203958+0000","last_became_active":"2024-04-25T10:21:41.198421+0000","last_became_peered":"2024-04-25T10:21:41.198421+0000","last_unstale":"2024-04-25T10:21:42.203958+0000","last_undegraded":"2024-04-25T10:21:42.203958+0000","last_fullsized":"2024-04-25T10:21:42.203958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:22.670568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203343+0000","last_change":"2024-04-25T10:21:40.183797+0000","last_active":"2024-04-25T10:21:42.203343+0000","last_peered":"2024-04-25T10:21:42.203343+0000","last_clean":"2024-04-25T10:21:42.203343+0000","last_became_active":"2024-04-25T10:21:40.183503+0000","last_became_peered":"2024-04-25T10:21:40.183503+0000","last_unstale":"2024-04-25T10:21:42.203343+0000","last_undegraded":"2024-04-25T10:21:42.203343+0000","last_fullsized":"2024-04-25T10:21:42.203343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:14.126101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202915+0000","last_change":"2024-04-25T10:21:41.205059+0000","last_active":"2024-04-25T10:21:42.202915+0000","last_peered":"2024-04-25T10:21:42.202915+0000","last_clean":"2024-04-25T10:21:42.202915+0000","last_became_active":"2024-04-25T10:21:41.204776+0000","last_became_peered":"2024-04-25T10:21:41.204776+0000","last_unstale":"2024-04-25T10:21:42.202915+0000","last_undegraded":"2024-04-25T10:21:42.202915+0000","last_fullsized":"2024-04-25T10:21:42.202915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:53.000929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220802+0000","last_change":"2024-04-25T10:21:40.198245+0000","last_active":"2024-04-25T10:21:42.220802+0000","last_peered":"2024-04-25T10:21:42.220802+0000","last_clean":"2024-04-25T10:21:42.220802+0000","last_became_active":"2024-04-25T10:21:40.196154+0000","last_became_peered":"2024-04-25T10:21:40.196154+0000","last_unstale":"2024-04-25T10:21:42.220802+0000","last_undegraded":"2024-04-25T10:21:42.220802+0000","last_fullsized":"2024-04-25T10:21:42.220802+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:30.617232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203301+0000","last_change":"2024-04-25T10:21:41.206073+0000","last_active":"2024-04-25T10:21:42.203301+0000","last_peered":"2024-04-25T10:21:42.203301+0000","last_clean":"2024-04-25T10:21:42.203301+0000","last_became_active":"2024-04-25T10:21:41.205704+0000","last_became_peered":"2024-04-25T10:21:41.205704+0000","last_unstale":"2024-04-25T10:21:42.203301+0000","last_undegraded":"2024-04-25T10:21:42.203301+0000","last_fullsized":"2024-04-25T10:21:42.203301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:31.909824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220448+0000","last_change":"2024-04-25T10:21:40.196050+0000","last_active":"2024-04-25T10:21:42.220448+0000","last_peered":"2024-04-25T10:21:42.220448+0000","last_clean":"2024-04-25T10:21:42.220448+0000","last_became_active":"2024-04-25T10:21:40.195717+0000","last_became_peered":"2024-04-25T10:21:40.195717+0000","last_unstale":"2024-04-25T10:21:42.220448+0000","last_undegraded":"2024-04-25T10:21:42.220448+0000","last_fullsized":"2024-04-25T10:21:42.220448+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:20.770611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203291+0000","last_change":"2024-04-25T10:21:41.198144+0000","last_active":"2024-04-25T10:21:42.203291+0000","last_peered":"2024-04-25T10:21:42.203291+0000","last_clean":"2024-04-25T10:21:42.203291+0000","last_became_active":"2024-04-25T10:21:41.197830+0000","last_became_peered":"2024-04-25T10:21:41.197830+0000","last_unstale":"2024-04-25T10:21:42.203291+0000","last_undegraded":"2024-04-25T10:21:42.203291+0000","last_fullsized":"2024-04-25T10:21:42.203291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:55.152776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222018+0000","last_change":"2024-04-25T10:21:40.201632+0000","last_active":"2024-04-25T10:21:42.222018+0000","last_peered":"2024-04-25T10:21:42.222018+0000","last_clean":"2024-04-25T10:21:42.222018+0000","last_became_active":"2024-04-25T10:21:40.201331+0000","last_became_peered":"2024-04-25T10:21:40.201331+0000","last_unstale":"2024-04-25T10:21:42.222018+0000","last_undegraded":"2024-04-25T10:21:42.222018+0000","last_fullsized":"2024-04-25T10:21:42.222018+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:45.965146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220879+0000","last_change":"2024-04-25T10:21:41.207659+0000","last_active":"2024-04-25T10:21:42.220879+0000","last_peered":"2024-04-25T10:21:42.220879+0000","last_clean":"2024-04-25T10:21:42.220879+0000","last_became_active":"2024-04-25T10:21:41.207300+0000","last_became_peered":"2024-04-25T10:21:41.207300+0000","last_unstale":"2024-04-25T10:21:42.220879+0000","last_undegraded":"2024-04-25T10:21:42.220879+0000","last_fullsized":"2024-04-25T10:21:42.220879+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:46.976004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.221492+0000","last_change":"2024-04-25T10:21:40.186060+0000","last_active":"2024-04-25T10:21:42.221492+0000","last_peered":"2024-04-25T10:21:42.221492+0000","last_clean":"2024-04-25T10:21:42.221492+0000","last_became_active":"2024-04-25T10:21:40.185767+0000","last_became_peered":"2024-04-25T10:21:40.185767+0000","last_unstale":"2024-04-25T10:21:42.221492+0000","last_undegraded":"2024-04-25T10:21:42.221492+0000","last_fullsized":"2024-04-25T10:21:42.221492+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:04.614877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203885+0000","last_change":"2024-04-25T10:21:41.213606+0000","last_active":"2024-04-25T10:21:42.203885+0000","last_peered":"2024-04-25T10:21:42.203885+0000","last_clean":"2024-04-25T10:21:42.203885+0000","last_became_active":"2024-04-25T10:21:41.213253+0000","last_became_peered":"2024-04-25T10:21:41.213253+0000","last_unstale":"2024-04-25T10:21:42.203885+0000","last_undegraded":"2024-04-25T10:21:42.203885+0000","last_fullsized":"2024-04-25T10:21:42.203885+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.553902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202982+0000","last_change":"2024-04-25T10:21:40.198944+0000","last_active":"2024-04-25T10:21:42.202982+0000","last_peered":"2024-04-25T10:21:42.202982+0000","last_clean":"2024-04-25T10:21:42.202982+0000","last_became_active":"2024-04-25T10:21:40.198666+0000","last_became_peered":"2024-04-25T10:21:40.198666+0000","last_unstale":"2024-04-25T10:21:42.202982+0000","last_undegraded":"2024-04-25T10:21:42.202982+0000","last_fullsized":"2024-04-25T10:21:42.202982+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:55.026517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219943+0000","last_change":"2024-04-25T10:21:41.217766+0000","last_active":"2024-04-25T10:21:42.219943+0000","last_peered":"2024-04-25T10:21:42.219943+0000","last_clean":"2024-04-25T10:21:42.219943+0000","last_became_active":"2024-04-25T10:21:41.211493+0000","last_became_peered":"2024-04-25T10:21:41.211493+0000","last_unstale":"2024-04-25T10:21:42.219943+0000","last_undegraded":"2024-04-25T10:21:42.219943+0000","last_fullsized":"2024-04-25T10:21:42.219943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:15.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220089+0000","last_change":"2024-04-25T10:21:40.198938+0000","last_active":"2024-04-25T10:21:42.220089+0000","last_peered":"2024-04-25T10:21:42.220089+0000","last_clean":"2024-04-25T10:21:42.220089+0000","last_became_active":"2024-04-25T10:21:40.198640+0000","last_became_peered":"2024-04-25T10:21:40.198640+0000","last_unstale":"2024-04-25T10:21:42.220089+0000","last_undegraded":"2024-04-25T10:21:42.220089+0000","last_fullsized":"2024-04-25T10:21:42.220089+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:30.617232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222531+0000","last_change":"2024-04-25T10:21:41.214282+0000","last_active":"2024-04-25T10:21:42.222531+0000","last_peered":"2024-04-25T10:21:42.222531+0000","last_clean":"2024-04-25T10:21:42.222531+0000","last_became_active":"2024-04-25T10:21:41.213972+0000","last_became_peered":"2024-04-25T10:21:41.213972+0000","last_unstale":"2024-04-25T10:21:42.222531+0000","last_undegraded":"2024-04-25T10:21:42.222531+0000","last_fullsized":"2024-04-25T10:21:42.222531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:34.174885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.202757+0000","last_change":"2024-04-25T10:21:40.201536+0000","last_active":"2024-04-25T10:21:42.202757+0000","last_peered":"2024-04-25T10:21:42.202757+0000","last_clean":"2024-04-25T10:21:42.202757+0000","last_became_active":"2024-04-25T10:21:40.201220+0000","last_became_peered":"2024-04-25T10:21:40.201220+0000","last_unstale":"2024-04-25T10:21:42.202757+0000","last_undegraded":"2024-04-25T10:21:42.202757+0000","last_fullsized":"2024-04-25T10:21:42.202757+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:46.836248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203023+0000","last_change":"2024-04-25T10:21:41.192855+0000","last_active":"2024-04-25T10:21:42.203023+0000","last_peered":"2024-04-25T10:21:42.203023+0000","last_clean":"2024-04-25T10:21:42.203023+0000","last_became_active":"2024-04-25T10:21:41.192413+0000","last_became_peered":"2024-04-25T10:21:41.192413+0000","last_unstale":"2024-04-25T10:21:42.203023+0000","last_undegraded":"2024-04-25T10:21:42.203023+0000","last_fullsized":"2024-04-25T10:21:42.203023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:19.769433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219890+0000","last_change":"2024-04-25T10:21:40.198377+0000","last_active":"2024-04-25T10:21:42.219890+0000","last_peered":"2024-04-25T10:21:42.219890+0000","last_clean":"2024-04-25T10:21:42.219890+0000","last_became_active":"2024-04-25T10:21:40.198112+0000","last_became_peered":"2024-04-25T10:21:40.198112+0000","last_unstale":"2024-04-25T10:21:42.219890+0000","last_undegraded":"2024-04-25T10:21:42.219890+0000","last_fullsized":"2024-04-25T10:21:42.219890+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:38.605398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.219898+0000","last_change":"2024-04-25T10:21:41.218098+0000","last_active":"2024-04-25T10:21:42.219898+0000","last_peered":"2024-04-25T10:21:42.219898+0000","last_clean":"2024-04-25T10:21:42.219898+0000","last_became_active":"2024-04-25T10:21:41.217814+0000","last_became_peered":"2024-04-25T10:21:41.217814+0000","last_unstale":"2024-04-25T10:21:42.219898+0000","last_undegraded":"2024-04-25T10:21:42.219898+0000","last_fullsized":"2024-04-25T10:21:42.219898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:05.133982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203129+0000","last_change":"2024-04-25T10:21:40.198104+0000","last_active":"2024-04-25T10:21:42.203129+0000","last_peered":"2024-04-25T10:21:42.203129+0000","last_clean":"2024-04-25T10:21:42.203129+0000","last_became_active":"2024-04-25T10:21:40.197665+0000","last_became_peered":"2024-04-25T10:21:40.197665+0000","last_unstale":"2024-04-25T10:21:42.203129+0000","last_undegraded":"2024-04-25T10:21:42.203129+0000","last_fullsized":"2024-04-25T10:21:42.203129+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:05.839079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.203307+0000","last_change":"2024-04-25T10:21:41.207095+0000","last_active":"2024-04-25T10:21:42.203307+0000","last_peered":"2024-04-25T10:21:42.203307+0000","last_clean":"2024-04-25T10:21:42.203307+0000","last_became_active":"2024-04-25T10:21:41.206524+0000","last_became_peered":"2024-04-25T10:21:41.206524+0000","last_unstale":"2024-04-25T10:21:42.203307+0000","last_undegraded":"2024-04-25T10:21:42.203307+0000","last_fullsized":"2024-04-25T10:21:42.203307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:40.168541+0000","last_clean_scrub_stamp":"2024-04-25T10:21:40.168541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.692831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.222270+0000","last_change":"2024-04-25T10:21:40.194319+0000","last_active":"2024-04-25T10:21:42.222270+0000","last_peered":"2024-04-25T10:21:42.222270+0000","last_clean":"2024-04-25T10:21:42.222270+0000","last_became_active":"2024-04-25T10:21:40.193961+0000","last_became_peered":"2024-04-25T10:21:40.193961+0000","last_unstale":"2024-04-25T10:21:42.222270+0000","last_undegraded":"2024-04-25T10:21:42.222270+0000","last_fullsized":"2024-04-25T10:21:42.222270+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:21:39.157682+0000","last_clean_scrub_stamp":"2024-04-25T10:21:39.157682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:36.719653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":247,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T10:21:42.220851+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:21:42.220851+0000","last_peered":"2024-04-25T10:21:42.220851+0000","last_clean":"2024-04-25T10:21:42.220851+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:21:42.220851+0000","last_undegraded":"2024-04-25T10:21:42.220851+0000","last_fullsized":"2024-04-25T10:21:42.220851+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":745472,"data_stored":308342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36064,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335776,"statfs":{"total":96636764160,"available":96599834624,"internally_reserved":0,"allocated":741376,"data_stored":308272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40812,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331028,"statfs":{"total":96636764160,"available":96594972672,"internally_reserved":0,"allocated":1335296,"data_stored":898652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40224,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331616,"statfs":{"total":96636764160,"available":96595574784,"internally_reserved":0,"allocated":741376,"data_stored":308340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":310022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40804,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331036,"statfs":{"total":96636764160,"available":96594980864,"internally_reserved":0,"allocated":1335296,"data_stored":899200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":749568,"data_stored":309862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58272,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313568,"statfs":{"total":96636764160,"available":96577093632,"internally_reserved":0,"allocated":733184,"data_stored":302566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:21:45.708 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:21:45.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:21:45.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.890+0000 7fa6b2c00640 1 Processor -- start 2024-04-25T10:21:45.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.890+0000 7fa6b2c00640 1 -- start start 2024-04-25T10:21:45.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b2c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6ac0b7320 0x7fa6ac0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b2c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6ac06bdb0 0x7fa6ac15d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b2c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac0b4950 con 0x7fa6ac06af50 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b2c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b5280 con 0x7fa6ac06bdb0 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b2c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b4c30 con 0x7fa6ac0b7320 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6ac06bdb0 0x7fa6ac15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa6ac06bdb0 0x7fa6ac15d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46488/0 (socket says 172.21.15.45:46488) 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 -- 172.21.15.45:0/268510744 learned_addr learned my addr 172.21.15.45:0/268510744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6a3400640 1 --1- 172.21.15.45:0/268510744 >> v1:172.21.15.45:6789/0 conn(0x7fa6ac06af50 0x7fa6ac06b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60096/0 (socket says 172.21.15.45:60096) 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6abe00640 1 --2- 172.21.15.45:0/268510744 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac0b7320 0x7fa6ac0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 -- 172.21.15.45:0/268510744 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac0b7320 msgr2=0x7fa6ac0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 --2- 172.21.15.45:0/268510744 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac0b7320 0x7fa6ac0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 -- 172.21.15.45:0/268510744 >> v1:172.21.15.45:6789/0 conn(0x7fa6ac06af50 legacy=0x7fa6ac06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.894+0000 7fa6b0c00640 1 -- 172.21.15.45:0/268510744 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0b4f10 con 0x7fa6ac06bdb0 2024-04-25T10:21:45.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b0c00640 1 --2- 172.21.15.45:0/268510744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac15d640 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0b4540 tx=0x7fa6a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160c250f9679fd3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6ab400640 1 -- 172.21.15.45:0/268510744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40a5730 con 0x7fa6ac06bdb0 2024-04-25T10:21:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6ab400640 1 -- 172.21.15.45:0/268510744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a40a58d0 con 0x7fa6ac06bdb0 2024-04-25T10:21:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6ab400640 1 -- 172.21.15.45:0/268510744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40a5ca0 con 0x7fa6ac06bdb0 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 -- 172.21.15.45:0/268510744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06bdb0 msgr2=0x7fa6ac15d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/268510744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac15d640 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0b4540 tx=0x7fa6a401c960 comp rx=0 tx=0).stop 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 -- 172.21.15.45:0/268510744 shutdown_connections 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/268510744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac15d640 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/268510744 >> v2:172.21.15.117:3300/0 conn(0x7fa6ac0b7320 0x7fa6ac0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 -- 172.21.15.45:0/268510744 >> 172.21.15.45:0/268510744 conn(0x7fa6ac0d19d0 msgr2=0x7fa6ac0d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 -- 172.21.15.45:0/268510744 shutdown_connections 2024-04-25T10:21:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 -- 172.21.15.45:0/268510744 wait complete. 2024-04-25T10:21:45.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 Processor -- start 2024-04-25T10:21:45.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.898+0000 7fa6b2c00640 1 -- start start 2024-04-25T10:21:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 0x7fa6ac15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac0b7320 0x7fa6ac154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b5a80 con 0x7fa6ac06af50 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b5900 con 0x7fa6ac06bdb0 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b5c00 con 0x7fa6ac0b7320 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 0x7fa6ac15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 0x7fa6ac15af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46490/0 (socket says 172.21.15.45:46490) 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 -- 172.21.15.45:0/3552954948 learned_addr learned my addr 172.21.15.45:0/3552954948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6a3400640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b0c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac0b7320 0x7fa6ac154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 -- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac0b7320 msgr2=0x7fa6ac154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac0b7320 0x7fa6ac154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 -- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac06bdb0 msgr2=0x7fa6ac159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 -- 172.21.15.45:0/3552954948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac154f80 con 0x7fa6ac06af50 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6abe00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 0x7fa6ac15af90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00d620 tx=0x7fa69c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00b620 con 0x7fa6ac06af50 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c019070 con 0x7fa6ac06af50 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00b9e0 con 0x7fa6ac06af50 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac15b5a0 con 0x7fa6ac06af50 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac15bb60 con 0x7fa6ac06af50 2024-04-25T10:21:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac04ef30 con 0x7fa6ac06af50 2024-04-25T10:21:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.902+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa69c00bb80 con 0x7fa6ac06af50 2024-04-25T10:21:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.906+0000 7fa6a8c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa688066f60 0x7fa688069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.906+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa69c085550 con 0x7fa6ac06af50 2024-04-25T10:21:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.906+0000 7fa6a3400640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa688066f60 0x7fa688069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:45.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.906+0000 7fa6a3400640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa688066f60 0x7fa688069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0d2420 tx=0x7fa694007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:45.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:45.910+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa69c056430 con 0x7fa6ac06af50 2024-04-25T10:21:46.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.130+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa6ac0b7950 con 0x7fa6ac06af50 2024-04-25T10:21:46.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.130+0000 7fa6a8c00640 1 -- 172.21.15.45:0/3552954948 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa69c055d30 con 0x7fa6ac06af50 2024-04-25T10:21:46.135 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:21:46.135 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:21:46.135 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:21:46.135 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:21:46.135 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:21:46.136 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa688066f60 msgr2=0x7fa688069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa688066f60 0x7fa688069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0d2420 tx=0x7fa694007cf0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 msgr2=0x7fa6ac15af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 0x7fa6ac15af90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00d620 tx=0x7fa69c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 shutdown_connections 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa688066f60 0x7fa688069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6ac0b7320 0x7fa6ac154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6ac06bdb0 0x7fa6ac159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 --2- 172.21.15.45:0/3552954948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6ac06af50 0x7fa6ac15af90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 >> 172.21.15.45:0/3552954948 conn(0x7fa6ac0d19d0 msgr2=0x7fa6ac0cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 shutdown_connections 2024-04-25T10:21:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.134+0000 7fa6b2c00640 1 -- 172.21.15.45:0/3552954948 wait complete. 2024-04-25T10:21:46.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:46.161 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:21:46.157+0000 7fd487400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:21:46.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 Processor -- start 2024-04-25T10:21:46.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 -- start start 2024-04-25T10:21:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7aac15e050 0x7f7aac0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac0b1870 con 0x7f7aac0b13e0 2024-04-25T10:21:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac0b1af0 con 0x7f7aac15e050 2024-04-25T10:21:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7ab0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0b1dd0 con 0x7f7aac0d1820 2024-04-25T10:21:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7aac15e050 0x7f7aac0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7aac15e050 0x7f7aac0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46492/0 (socket says 172.21.15.45:46492) 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 -- 172.21.15.45:0/1457623730 learned_addr learned my addr 172.21.15.45:0/1457623730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa9e00640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 -- 172.21.15.45:0/1457623730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 msgr2=0x7f7aac0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 -- 172.21.15.45:0/1457623730 >> v1:172.21.15.117:6789/0 conn(0x7f7aac0b13e0 legacy=0x7f7aac0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa1400640 1 -- 172.21.15.45:0/1457623730 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0b20b0 con 0x7f7aac15e050 2024-04-25T10:21:46.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.294+0000 7f7aa9e00640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:46.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7aa1400640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac0b0ea0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e50 tx=0x7f7a9401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce65378574aec14a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7aa9400640 1 -- 172.21.15.45:0/1457623730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940a5900 con 0x7f7aac15e050 2024-04-25T10:21:46.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7aa9400640 1 -- 172.21.15.45:0/1457623730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a940a5aa0 con 0x7f7aac15e050 2024-04-25T10:21:46.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7aa9400640 1 -- 172.21.15.45:0/1457623730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940ad3f0 con 0x7f7aac15e050 2024-04-25T10:21:46.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1457623730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 msgr2=0x7f7aac0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac0b0ea0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e50 tx=0x7f7a9401ca30 comp rx=0 tx=0).stop 2024-04-25T10:21:46.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1457623730 shutdown_connections 2024-04-25T10:21:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac0b0ea0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1457623730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1457623730 >> 172.21.15.45:0/1457623730 conn(0x7f7aac09a9d0 msgr2=0x7f7aac09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1457623730 shutdown_connections 2024-04-25T10:21:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.298+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1457623730 wait complete. 2024-04-25T10:21:46.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 Processor -- start 2024-04-25T10:21:46.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- start start 2024-04-25T10:21:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac0b13e0 0x7f7aac149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac16ab20 con 0x7f7aac15e050 2024-04-25T10:21:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac16a9a0 con 0x7f7aac0b13e0 2024-04-25T10:21:46.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac16aca0 con 0x7f7aac0d1820 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac15a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46504/0 (socket says 172.21.15.45:46504) 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 -- 172.21.15.45:0/1218270936 learned_addr learned my addr 172.21.15.45:0/1218270936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aa9e00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac0b13e0 0x7f7aac149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aa1400640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 -- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 msgr2=0x7f7aac159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 -- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac0b13e0 msgr2=0x7f7aac149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac0b13e0 0x7f7aac149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 -- 172.21.15.45:0/1218270936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac14a350 con 0x7f7aac15e050 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aaa800640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac15a4a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c430 tx=0x7f7aa400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4005220 con 0x7f7aac15e050 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa40053c0 con 0x7f7aac15e050 2024-04-25T10:21:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4007270 con 0x7f7aac15e050 2024-04-25T10:21:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac0af390 con 0x7f7aac15e050 2024-04-25T10:21:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac0af950 con 0x7f7aac15e050 2024-04-25T10:21:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.302+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac0aff60 con 0x7f7aac15e050 2024-04-25T10:21:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.306+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7aa4019020 con 0x7f7aac15e050 2024-04-25T10:21:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.306+0000 7f7aa2a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7a88066fb0 0x7f7a88069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.306+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa40841a0 con 0x7f7aac15e050 2024-04-25T10:21:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.306+0000 7f7aa9e00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7a88066fb0 0x7f7a88069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.306+0000 7f7aa9e00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7a88066fb0 0x7f7a88069470 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00ab60 tx=0x7f7a9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.310+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7aac0b1870 con 0x7f7aac15e050 2024-04-25T10:21:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.506+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aac04ef30 con 0x7f7aac15e050 2024-04-25T10:21:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.506+0000 7f7aa2a00640 1 -- 172.21.15.45:0/1218270936 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f7aa4054c40 con 0x7f7aac15e050 2024-04-25T10:21:46.511 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:46.511 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2258543519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2258543519},{"type":"v1","addr":"172.21.15.45:6835","nonce":2258543519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3808318987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3808318987},{"type":"v1","addr":"172.21.15.117:6836","nonce":3808318987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6384,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/317823919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":317823919},{"type":"v1","addr":"172.21.15.45:6834","nonce":317823919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:43.601865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3997755731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3997755731},{"type":"v1","addr":"172.21.15.117:6837","nonce":3997755731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:21:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 39 2024-04-25T10:21:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7a88066fb0 msgr2=0x7f7a88069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7a88066fb0 0x7f7a88069470 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00ab60 tx=0x7f7a9c008040 comp rx=0 tx=0).stop 2024-04-25T10:21:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 msgr2=0x7f7aac15a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac15a4a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c430 tx=0x7f7aa400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 shutdown_connections 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7a88066fb0 0x7f7a88069470 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac15e050 0x7f7aac15a4a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7aac0d1820 0x7f7aac159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 --2- 172.21.15.45:0/1218270936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aac0b13e0 0x7f7aac149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 >> 172.21.15.45:0/1218270936 conn(0x7f7aac09a9d0 msgr2=0x7f7aac0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 shutdown_connections 2024-04-25T10:21:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:46.510+0000 7f7ab0a00640 1 -- 172.21.15.45:0/1218270936 wait complete. 2024-04-25T10:21:46.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:21:40.567701+0000', 'modified': '2024-04-25T10:21:43.601865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6393': {'gid': 6393, 'name': 'd', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/3997755731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 3997755731}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 3997755731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:21:46.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:21:46.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:21:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.074+0000 7f991fe00640 1 Processor -- start 2024-04-25T10:21:47.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.074+0000 7f991fe00640 1 -- start start 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f991806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99180cda10 0x7f99180b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f99180cf930 0x7f99180cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991fe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99180b2770 con 0x7f99180cda10 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991fe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991806c670 con 0x7f99180cf930 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991806c950 con 0x7f991815bc40 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99180cda10 0x7f99180b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99180cda10 0x7f99180b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46516/0 (socket says 172.21.15.45:46516) 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 -- 172.21.15.45:0/3361435880 learned_addr learned my addr 172.21.15.45:0/3361435880 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991d400640 1 --2- 172.21.15.45:0/3361435880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f991806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991de00640 1 --2- 172.21.15.45:0/3361435880 >> v2:172.21.15.117:3300/0 conn(0x7f99180cf930 0x7f99180cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 -- 172.21.15.45:0/3361435880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 msgr2=0x7f991806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 --2- 172.21.15.45:0/3361435880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f991806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 -- 172.21.15.45:0/3361435880 >> v2:172.21.15.117:3300/0 conn(0x7f99180cf930 msgr2=0x7f99180cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 --2- 172.21.15.45:0/3361435880 >> v2:172.21.15.117:3300/0 conn(0x7f99180cf930 0x7f99180cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.078+0000 7f991ca00640 1 -- 172.21.15.45:0/3361435880 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991806cc30 con 0x7f99180cda10 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991ca00640 1 --2- 172.21.15.45:0/3361435880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99180b4610 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f99180b3f80 tx=0x7f990401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315040c3d257ba8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f9917e00640 1 -- 172.21.15.45:0/3361435880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99040b0070 con 0x7f99180cda10 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f9917e00640 1 -- 172.21.15.45:0/3361435880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99040ab600 con 0x7f99180cda10 2024-04-25T10:21:47.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f9917e00640 1 -- 172.21.15.45:0/3361435880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99040ab9d0 con 0x7f99180cda10 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- 172.21.15.45:0/3361435880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 msgr2=0x7f99180b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- 172.21.15.45:0/3361435880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99180b4610 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f99180b3f80 tx=0x7f990401c520 comp rx=0 tx=0).stop 2024-04-25T10:21:47.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- 172.21.15.45:0/3361435880 shutdown_connections 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- 172.21.15.45:0/3361435880 >> v2:172.21.15.117:3300/0 conn(0x7f99180cf930 0x7f99180cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- 172.21.15.45:0/3361435880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99180b4610 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- 172.21.15.45:0/3361435880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f991806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- 172.21.15.45:0/3361435880 >> 172.21.15.45:0/3361435880 conn(0x7f9918066b70 msgr2=0x7f9918066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- 172.21.15.45:0/3361435880 shutdown_connections 2024-04-25T10:21:47.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- 172.21.15.45:0/3361435880 wait complete. 2024-04-25T10:21:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 Processor -- start 2024-04-25T10:21:47.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- start start 2024-04-25T10:21:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99181fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99180cf930 0x7f9918155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f9918143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918160230 con 0x7f99180cda10 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99181600b0 con 0x7f99180cf930 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.082+0000 7f991fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99181603b0 con 0x7f991815bc40 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99181fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99181fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46530/0 (socket says 172.21.15.45:46530) 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 -- 172.21.15.45:0/2995041794 learned_addr learned my addr 172.21.15.45:0/2995041794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991ca00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99180cf930 0x7f9918155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 -- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 msgr2=0x7f9918143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f9918143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 -- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99180cf930 msgr2=0x7f9918155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99180cf930 0x7f9918155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 -- 172.21.15.45:0/2995041794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991815ad80 con 0x7f99180cda10 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991ca00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99180cf930 0x7f9918155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991d400640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99181fb1e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f990c00e930 tx=0x7f990c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f9915600640 1 -- 172.21.15.45:0/2995041794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c017070 con 0x7f99180cda10 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f9915600640 1 -- 172.21.15.45:0/2995041794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00c980 con 0x7f99180cda10 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991815e970 con 0x7f99180cda10 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f9915600640 1 -- 172.21.15.45:0/2995041794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00cd30 con 0x7f99180cda10 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9918155da0 con 0x7f99180cda10 2024-04-25T10:21:47.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.086+0000 7f9915600640 1 -- 172.21.15.45:0/2995041794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f990c01a070 con 0x7f99180cda10 2024-04-25T10:21:47.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9915600640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98f8066f90 0x7f98f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9915600640 1 -- 172.21.15.45:0/2995041794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f990c0850a0 con 0x7f99180cda10 2024-04-25T10:21:47.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 Processor -- start 2024-04-25T10:21:47.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98f8066f90 0x7f98f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- start start 2024-04-25T10:21:47.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98f8066f90 0x7f98f8069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f99040ad000 tx=0x7f9904008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaceced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abace849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abace85310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abacd6c5f0 con 0x55abacecf240 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abaccd3da0 con 0x55abacece920 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abacc5c150 con 0x55abace84f30 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abace849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abace849f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46540/0 (socket says 172.21.15.45:46540) 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 -- 172.21.15.45:0/4117339509 learned_addr learned my addr 172.21.15.45:0/4117339509 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991d400640 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaceced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991de00640 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abace85310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 -- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 msgr2=0x55abace85310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abace85310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 -- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 msgr2=0x55abaceced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaceced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 -- 172.21.15.45:0/4117339509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abacc75ca0 con 0x55abacecf240 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991de00640 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abace85310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f991ca00640 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abace849f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9904008010 tx=0x7f99040a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9a59665e5e0fab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f990a000640 1 -- 172.21.15.45:0/4117339509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99040c6070 con 0x55abacecf240 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f990a000640 1 -- 172.21.15.45:0/4117339509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99040b5460 con 0x55abacecf240 2024-04-25T10:21:47.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f990a000640 1 -- 172.21.15.45:0/4117339509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99040b0040 con 0x55abacecf240 2024-04-25T10:21:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- 172.21.15.45:0/4117339509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 msgr2=0x55abace849f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abace849f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9904008010 tx=0x7f99040a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- 172.21.15.45:0/4117339509 shutdown_connections 2024-04-25T10:21:47.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abace85310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abace849f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 --2- 172.21.15.45:0/4117339509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaceced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- 172.21.15.45:0/4117339509 >> 172.21.15.45:0/4117339509 conn(0x55abace8b2a0 msgr2=0x55abace832d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- 172.21.15.45:0/4117339509 shutdown_connections 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- 172.21.15.45:0/4117339509 wait complete. 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.090+0000 7f9923397000 1 -- start start 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 Processor -- start 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 0x55abaced5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaced61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacecf240 0x55abace88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abacc8e990 con 0x55abace84f30 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abacccfe40 con 0x55abacece920 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abacc5f900 con 0x55abacecf240 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 10 client.0 _ll_register_callbacks cb 0x55abacd871c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:47.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 0x55abaced5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 0x55abaced5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46554/0 (socket says 172.21.15.45:46554) 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 -- 172.21.15.45:0/2088699230 learned_addr learned my addr 172.21.15.45:0/2088699230 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991de00640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacecf240 0x55abace88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991ca00640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaced61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 -- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacecf240 msgr2=0x55abace88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacecf240 0x55abace88aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 -- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 msgr2=0x55abaced61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaced61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 -- 172.21.15.45:0/2088699230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abacc8c630 con 0x55abace84f30 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991ca00640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaced61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991d400640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 0x55abaced5c80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f990c0097c0 tx=0x7f990c059b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ec352f957ffc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f98fea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f98fea00640 1 -- 172.21.15.45:0/2088699230 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c080070 con 0x55abace84f30 2024-04-25T10:21:47.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f98fea00640 1 -- 172.21.15.45:0/2088699230 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f990c055020 con 0x55abace84f30 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99040b5460 con 0x55abace84f30 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f98fea00640 1 -- 172.21.15.45:0/2088699230 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f990c021040 con 0x55abace84f30 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 10 client.6507 fetch_fsmap learned FSMap version 39 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55abacc5c3a0 con 0x55abace84f30 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f98fea00640 1 -- 172.21.15.45:0/2088699230 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99040b5460 con 0x55abace84f30 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f98fea00640 1 -- 172.21.15.45:0/2088699230 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f990c024060 con 0x55abace84f30 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 10 client.6507 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 10 client.6507 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 20 client.6507 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abace8a160 0x55abacedfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 4 client.6507 mds_command: new command op to 6393 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:47.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 --> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abacc6fa50 con 0x55abace8a160 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.094+0000 7f991ca00640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abace8a160 0x55abacedfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f98fea00640 10 client.6507 ms_handle_connect on v2:172.21.15.117:6835/3997755731 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f991ca00640 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abace8a160 0x55abacedfc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f2da5647ee6e3b server_cookie=471fc191cf3cb81c in_seq=0 out_seq=0 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f98fea00640 1 -- 172.21.15.45:0/2088699230 <== mds.0 v2:172.21.15.117:6835/3997755731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abacc6fa50 con 0x55abace8a160 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f98fea00640 10 client.6507 handle_command_reply: tid=0 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 client.6507 shutdown 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 msgr2=0x55abaced5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 0x55abaced5c80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f990c0097c0 tx=0x7f990c059b20 comp rx=0 tx=0).stop 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 shutdown_connections 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abace8a160 0x55abacedfc70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacecf240 0x55abace88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abaced61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 --2- 172.21.15.45:0/2088699230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abace84f30 0x55abaced5c80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 >> 172.21.15.45:0/2088699230 conn(0x55abace8b2a0 msgr2=0x55abace839e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f98fea00640 0 client.6507 ms_handle_reset on v2:172.21.15.117:6835/3997755731 2024-04-25T10:21:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 shutdown_connections 2024-04-25T10:21:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 1 -- 172.21.15.45:0/2088699230 wait complete. 2024-04-25T10:21:47.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.098+0000 7f9923397000 20 client.6507 trim_cache size 0 max 16384 2024-04-25T10:21:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 Processor -- start 2024-04-25T10:21:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 -- start start 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abace84f30 0x55abacecd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacece920 0x55abacecd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abaced88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918201fe0 con 0x55abacecf240 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918158af0 con 0x55abace84f30 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9918201a40 con 0x55abacece920 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abaced88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abaced88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46562/0 (socket says 172.21.15.45:46562) 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 -- 172.21.15.45:0/684162487 learned_addr learned my addr 172.21.15.45:0/684162487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991ca00640 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacece920 0x55abacecd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991d400640 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abace84f30 0x55abacecd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 -- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacece920 msgr2=0x55abacecd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacece920 0x55abacecd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 -- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abace84f30 msgr2=0x55abacecd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abace84f30 0x55abacecd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 -- 172.21.15.45:0/684162487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abacecf070 con 0x55abacecf240 2024-04-25T10:21:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f991de00640 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abaced88b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f99100083d0 tx=0x7f991000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12937002d129f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9909600640 1 -- 172.21.15.45:0/684162487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100167f0 con 0x55abacecf240 2024-04-25T10:21:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9909600640 1 -- 172.21.15.45:0/684162487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991001c070 con 0x55abacecf240 2024-04-25T10:21:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.114+0000 7f9909600640 1 -- 172.21.15.45:0/684162487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910016c10 con 0x55abacecf240 2024-04-25T10:21:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/684162487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 msgr2=0x55abaced88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abaced88b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f99100083d0 tx=0x7f991000b040 comp rx=0 tx=0).stop 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/684162487 shutdown_connections 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abaced88b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abacece920 0x55abacecd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- 172.21.15.45:0/684162487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abace84f30 0x55abacecd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/684162487 >> 172.21.15.45:0/684162487 conn(0x55abace8b2a0 msgr2=0x55abace88220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/684162487 shutdown_connections 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/684162487 wait complete. 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- start start 2024-04-25T10:21:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 Processor -- start 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abacedb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abacf301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abacf30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abaccd83d0 con 0x55abacecf240 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abacd8fd50 con 0x55abacece920 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abacc6e980 con 0x55abace84f30 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 10 client.0 _ll_register_callbacks cb 0x7f99040ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abacf30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abacf30710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46570/0 (socket says 172.21.15.45:46570) 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 -- 172.21.15.45:0/3437092771 learned_addr learned my addr 172.21.15.45:0/3437092771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991d400640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abacedb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 -- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 msgr2=0x55abacedb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abacedb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991ca00640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abacf301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 -- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 msgr2=0x55abacf301d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abacf301d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 -- 172.21.15.45:0/3437092771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abacd55600 con 0x55abacecf240 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991d400640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abacedb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991ca00640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abacf301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f991de00640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abacf30710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f991000d010 tx=0x7f9910009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167bbe1d520e65be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f98fea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f98fea00640 1 -- 172.21.15.45:0/3437092771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910017680 con 0x55abacecf240 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f98fea00640 1 -- 172.21.15.45:0/3437092771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9910017820 con 0x55abacecf240 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991001c070 con 0x55abacecf240 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f98fea00640 1 -- 172.21.15.45:0/3437092771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9910031050 con 0x55abacecf240 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 10 client.6513 fetch_fsmap learned FSMap version 39 2024-04-25T10:21:47.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55abacc689b0 con 0x55abacecf240 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f98fea00640 1 -- 172.21.15.45:0/3437092771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991001c070 con 0x55abacecf240 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f98fea00640 1 -- 172.21.15.45:0/3437092771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9910034020 con 0x55abacecf240 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 10 client.6513 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 10 client.6513 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 20 client.6513 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abacf13d40 0x55abacf14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 4 client.6513 mds_command: new command op to 6393 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.118+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 --> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55abacce12e0 con 0x55abacf13d40 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991d400640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abacf13d40 0x55abacf14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991d400640 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abacf13d40 0x55abacf14120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42e56f4ebe799e5 server_cookie=72a0bc59e2da3dd7 in_seq=0 out_seq=0 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f98fea00640 10 client.6513 ms_handle_connect on v2:172.21.15.117:6835/3997755731 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f98fea00640 1 -- 172.21.15.45:0/3437092771 <== mds.0 v2:172.21.15.117:6835/3997755731 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55abacce12e0 con 0x55abacf13d40 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f98fea00640 10 client.6513 handle_command_reply: tid=0 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 client.6513 shutdown 2024-04-25T10:21:47.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 msgr2=0x55abacf30710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abacf30710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f991000d010 tx=0x7f9910009750 comp rx=0 tx=0).stop 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 shutdown_connections 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:6835/3997755731,v1:172.21.15.117:6837/3997755731] conn(0x55abacf13d40 0x55abacf14120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abacecf240 0x55abacf30710 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abacece920 0x55abacf301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f98fea00640 0 client.6513 ms_handle_reset on v2:172.21.15.117:6835/3997755731 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 --2- 172.21.15.45:0/3437092771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abace84f30 0x55abacedb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 >> 172.21.15.45:0/3437092771 conn(0x55abace8b2a0 msgr2=0x55abaced67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 shutdown_connections 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 1 -- 172.21.15.45:0/3437092771 wait complete. 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f9923397000 20 client.6513 trim_cache size 0 max 16384 2024-04-25T10:21:47.129 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6393, 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9168955299999997, 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 39, 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 62, 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 62, 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.783344312000001 2024-04-25T10:21:47.130 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98f8066f90 msgr2=0x7f98f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98f8066f90 0x7f98f8069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f99040ad000 tx=0x7f9904008040 comp rx=0 tx=0).stop 2024-04-25T10:21:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 msgr2=0x7f99181fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99181fb1e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f990c00e930 tx=0x7f990c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 shutdown_connections 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98f8066f90 0x7f98f8069450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f991815bc40 0x7f9918143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99180cf930 0x7f9918155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 --2- 172.21.15.45:0/2995041794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99180cda10 0x7f99181fb1e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 >> 172.21.15.45:0/2995041794 conn(0x7f9918066b70 msgr2=0x7f991815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 shutdown_connections 2024-04-25T10:21:47.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.122+0000 7f991fe00640 1 -- 172.21.15.45:0/2995041794 wait complete. 2024-04-25T10:21:47.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:21:47.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.282+0000 7f8ff2600640 1 Processor -- start 2024-04-25T10:21:47.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.282+0000 7f8ff2600640 1 -- start start 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8ff2600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8fec0b0f70 0x7f8fec0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8ff2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fec0cd2e0 0x7f8fec0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8ff2600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec15f3b0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8ff2600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec15ef20 con 0x7f8fec0cd2e0 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8ff2600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec0ca810 con 0x7f8fec0b0f70 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8febe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fec0cd2e0 0x7f8fec0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8fe2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8fec0cfd90 0x7f8fec0ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60108/0 (socket says 172.21.15.45:60108) 2024-04-25T10:21:47.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8fe2a00640 1 -- 172.21.15.45:0/1909497452 learned_addr learned my addr 172.21.15.45:0/1909497452 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8feb400640 1 --2- 172.21.15.45:0/1909497452 >> v2:172.21.15.117:3300/0 conn(0x7f8fec0b0f70 0x7f8fec0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8febe00640 1 -- 172.21.15.45:0/1909497452 >> v2:172.21.15.117:3300/0 conn(0x7f8fec0b0f70 msgr2=0x7f8fec0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8febe00640 1 --2- 172.21.15.45:0/1909497452 >> v2:172.21.15.117:3300/0 conn(0x7f8fec0b0f70 0x7f8fec0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8febe00640 1 -- 172.21.15.45:0/1909497452 >> v1:172.21.15.45:6789/0 conn(0x7f8fec0cfd90 legacy=0x7f8fec0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8febe00640 1 -- 172.21.15.45:0/1909497452 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0b1890 con 0x7f8fec0cd2e0 2024-04-25T10:21:47.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8feb400640 1 --2- 172.21.15.45:0/1909497452 >> v2:172.21.15.117:3300/0 conn(0x7f8fec0b0f70 0x7f8fec0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.286+0000 7f8feaa00640 1 -- 172.21.15.45:0/1909497452 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673970463 0 0) 0x7f8fec15f3b0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8febe00640 1 --2- 172.21.15.45:0/1909497452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cd2e0 0x7f8fec0cba20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8fec071230 tx=0x7f8fe401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0b4bfecf29de44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8feaa00640 1 -- 172.21.15.45:0/1909497452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40b2070 con 0x7f8fec0cd2e0 2024-04-25T10:21:47.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8feaa00640 1 -- 172.21.15.45:0/1909497452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe40a57f0 con 0x7f8fec0cd2e0 2024-04-25T10:21:47.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8feaa00640 1 -- 172.21.15.45:0/1909497452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40a5bc0 con 0x7f8fec0cd2e0 2024-04-25T10:21:47.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 -- 172.21.15.45:0/1909497452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cd2e0 msgr2=0x7f8fec0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1909497452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cd2e0 0x7f8fec0cba20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8fec071230 tx=0x7f8fe401c930 comp rx=0 tx=0).stop 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 -- 172.21.15.45:0/1909497452 shutdown_connections 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1909497452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cd2e0 0x7f8fec0cba20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1909497452 >> v2:172.21.15.117:3300/0 conn(0x7f8fec0b0f70 0x7f8fec0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 -- 172.21.15.45:0/1909497452 >> 172.21.15.45:0/1909497452 conn(0x7f8fec0b1d50 msgr2=0x7f8fec0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 -- 172.21.15.45:0/1909497452 shutdown_connections 2024-04-25T10:21:47.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 -- 172.21.15.45:0/1909497452 wait complete. 2024-04-25T10:21:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 Processor -- start 2024-04-25T10:21:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 -- start start 2024-04-25T10:21:47.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0b0f70 0x7f8fec0c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.290+0000 7f8ff2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0cd2e0 0x7f8fec156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 0x7f8fec156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec161b50 con 0x7f8fec0cfd90 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1619d0 con 0x7f8fec0b0f70 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec161cd0 con 0x7f8fec0cd2e0 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 0x7f8fec156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 0x7f8fec156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46586/0 (socket says 172.21.15.45:46586) 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 -- 172.21.15.45:0/1869557358 learned_addr learned my addr 172.21.15.45:0/1869557358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8feb400640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0b0f70 0x7f8fec0c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 -- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0cd2e0 msgr2=0x7f8fec156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0cd2e0 0x7f8fec156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 -- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0b0f70 msgr2=0x7f8fec0c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0b0f70 0x7f8fec0c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 -- 172.21.15.45:0/1869557358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0c63c0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8feb400640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0b0f70 0x7f8fec0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8febe00640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 0x7f8fec156ea0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40b0040 tx=0x7f8fe40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40b2050 con 0x7f8fec0cfd90 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe40b3040 con 0x7f8fec0cfd90 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4003ba0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec0c66a0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec14ff00 con 0x7f8fec0cfd90 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec04ef30 con 0x7f8fec0cfd90 2024-04-25T10:21:47.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.294+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fe40c1060 con 0x7f8fec0cfd90 2024-04-25T10:21:47.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.298+0000 7f8fe3e00640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc8066f60 0x7f8fc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:47.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.298+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe411f430 con 0x7f8fec0cfd90 2024-04-25T10:21:47.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.298+0000 7f8feb400640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc8066f60 0x7f8fc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:47.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.298+0000 7f8feb400640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc8066f60 0x7f8fc8069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009980 tx=0x7f8fdc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:47.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.302+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fe4016520 con 0x7f8fec0cfd90 2024-04-25T10:21:47.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8fec0cc9b0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1869557358 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8fe40f03a0 con 0x7f8fec0cfd90 2024-04-25T10:21:47.502 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc8066f60 msgr2=0x7f8fc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc8066f60 0x7f8fc8069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009980 tx=0x7f8fdc008040 comp rx=0 tx=0).stop 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 msgr2=0x7f8fec156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 0x7f8fec156ea0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40b0040 tx=0x7f8fe40196d0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 shutdown_connections 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc8066f60 0x7f8fc8069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec0cfd90 0x7f8fec156ea0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fec0cd2e0 0x7f8fec156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 --2- 172.21.15.45:0/1869557358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fec0b0f70 0x7f8fec0c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 >> 172.21.15.45:0/1869557358 conn(0x7f8fec0b1d50 msgr2=0x7f8fec072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 shutdown_connections 2024-04-25T10:21:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:47.498+0000 7f8ff2600640 1 -- 172.21.15.45:0/1869557358 wait complete. 2024-04-25T10:21:47.522 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:21:47.522 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:21:47.523 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:47.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.524 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:21:47.535 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:21:47.535 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:21:47.535 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:21:47.535 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.535 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:21:47.536 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.536 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42232sec preferred_lft 42232sec 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:21:47.537 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:21:47.538 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:21:47.539 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:47.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.540 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:21:47.593 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:21:47.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.595 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:21:47.645 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:21:47.645 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:21:47.645 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:21:47.645 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:21:47.645 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:21:47.645 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:21:47.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.646 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:21:47.646 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:21:47.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:21:47.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:21:47.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:21:47.646 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:21:47.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.737 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:21:47.747 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:21:47.747 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:21:47.747 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:47.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:21:47.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:21:47.808 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:47.860 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:47.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:21:47.860 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:47.911 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:21:47.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:47.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:21:48.004 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:21:48.004 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:21:48.017 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:21:48.017 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:21:48.017 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:21:48.017 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:21:48.017 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:21:48.021 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:21:48.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:48.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:48.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:48.042 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:21:48.103 INFO:teuthology.orchestra.run.smithi045.stdout:client6525 2024-04-25T10:21:48.103 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:48.104 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:48.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:48.152 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T10:21:48.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:48.215 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T10:21:48.267 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:21:48.267 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:21:48.267 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:21:48.267 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:21:48.267 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T10:21:48.267 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:21:48.378 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-04-25T10:21:48.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:21:48.387 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:48.387 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:48.440 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:48.441 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:48.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:21:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.670+0000 7f66eec00640 1 Processor -- start 2024-04-25T10:21:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.670+0000 7f66eec00640 1 -- start start 2024-04-25T10:21:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66eec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e80d0350 0x7f66e80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66eec00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66e806aea0 0x7f66e806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66eec00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e80ccaa0 con 0x7f66e815df60 2024-04-25T10:21:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66eec00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e815e340 con 0x7f66e806aea0 2024-04-25T10:21:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66eec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80ccd80 con 0x7f66e80d0350 2024-04-25T10:21:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66ecc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66e806aea0 0x7f66e806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66e7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f66e815df60 0x7f66e80cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60124/0 (socket says 172.21.15.45:60124) 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66e7e00640 1 -- 172.21.15.45:0/942618516 learned_addr learned my addr 172.21.15.45:0/942618516 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66df400640 1 --2- 172.21.15.45:0/942618516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e80d0350 0x7f66e80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66ecc00640 1 -- 172.21.15.45:0/942618516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e80d0350 msgr2=0x7f66e80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66ecc00640 1 --2- 172.21.15.45:0/942618516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e80d0350 0x7f66e80d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66ecc00640 1 -- 172.21.15.45:0/942618516 >> v1:172.21.15.45:6789/0 conn(0x7f66e815df60 legacy=0x7f66e80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.674+0000 7f66ecc00640 1 -- 172.21.15.45:0/942618516 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80cd060 con 0x7f66e806aea0 2024-04-25T10:21:48.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66ecc00640 1 --2- 172.21.15.45:0/942618516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e806aea0 0x7f66e806b310 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009d90 tx=0x7f66e001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8136fbe1fccbc56a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:48.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66e7400640 1 -- 172.21.15.45:0/942618516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00b2070 con 0x7f66e806aea0 2024-04-25T10:21:48.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66e7400640 1 -- 172.21.15.45:0/942618516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e00a5890 con 0x7f66e806aea0 2024-04-25T10:21:48.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66e7400640 1 -- 172.21.15.45:0/942618516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00a5c60 con 0x7f66e806aea0 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 -- 172.21.15.45:0/942618516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e806aea0 msgr2=0x7f66e806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 --2- 172.21.15.45:0/942618516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e806aea0 0x7f66e806b310 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009d90 tx=0x7f66e001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 -- 172.21.15.45:0/942618516 shutdown_connections 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 --2- 172.21.15.45:0/942618516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e806aea0 0x7f66e806b310 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 --2- 172.21.15.45:0/942618516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e80d0350 0x7f66e80d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 -- 172.21.15.45:0/942618516 >> 172.21.15.45:0/942618516 conn(0x7f66e80d1910 msgr2=0x7f66e80d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 -- 172.21.15.45:0/942618516 shutdown_connections 2024-04-25T10:21:48.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 -- 172.21.15.45:0/942618516 wait complete. 2024-04-25T10:21:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 Processor -- start 2024-04-25T10:21:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.678+0000 7f66eec00640 1 -- start start 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e806aea0 0x7f66e8157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e80d0350 0x7f66e8152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 0x7f66e815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80d0ff0 con 0x7f66e815df60 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80d0e70 con 0x7f66e80d0350 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e816aa20 con 0x7f66e806aea0 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 0x7f66e815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 0x7f66e815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46606/0 (socket says 172.21.15.45:46606) 2024-04-25T10:21:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 -- 172.21.15.45:0/4151366915 learned_addr learned my addr 172.21.15.45:0/4151366915 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66e7e00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e806aea0 0x7f66e8157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66df400640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e80d0350 0x7f66e8152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 -- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e806aea0 msgr2=0x7f66e8157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e806aea0 0x7f66e8157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 -- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e80d0350 msgr2=0x7f66e8152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e80d0350 0x7f66e8152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 -- 172.21.15.45:0/4151366915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80c9e00 con 0x7f66e815df60 2024-04-25T10:21:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66ecc00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 0x7f66e815cb60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f66e00b0040 tx=0x7f66e0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00b2050 con 0x7f66e815df60 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e00be070 con 0x7f66e815df60 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00ada70 con 0x7f66e815df60 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e8153c50 con 0x7f66e815df60 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e80ca0a0 con 0x7f66e815df60 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e804ef30 con 0x7f66e815df60 2024-04-25T10:21:48.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.682+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66e00a5710 con 0x7f66e815df60 2024-04-25T10:21:48.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.686+0000 7f66e4c00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66c4066f60 0x7f66c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:48.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.686+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66e0120090 con 0x7f66e815df60 2024-04-25T10:21:48.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.686+0000 7f66e7e00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66c4066f60 0x7f66c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:48.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.686+0000 7f66e7e00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66c4066f60 0x7f66c4069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000f70 tx=0x7f66d8007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.690+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66e0016560 con 0x7f66e815df60 2024-04-25T10:21:48.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.850+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66e80c8780 con 0x7f66e815df60 2024-04-25T10:21:48.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.850+0000 7f66e4c00640 1 -- 172.21.15.45:0/4151366915 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f66e00f0a30 con 0x7f66e815df60 2024-04-25T10:21:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":62,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:42.194238+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66c4066f60 msgr2=0x7f66c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66c4066f60 0x7f66c4069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000f70 tx=0x7f66d8007e90 comp rx=0 tx=0).stop 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 msgr2=0x7f66e815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 0x7f66e815cb60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f66e00b0040 tx=0x7f66e0019680 comp rx=0 tx=0).stop 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 shutdown_connections 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f66c4066f60 0x7f66c4069420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e815df60 0x7f66e815cb60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66e80d0350 0x7f66e8152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 --2- 172.21.15.45:0/4151366915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66e806aea0 0x7f66e8157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:48.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 >> 172.21.15.45:0/4151366915 conn(0x7f66e80d1910 msgr2=0x7f66e8068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:48.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 shutdown_connections 2024-04-25T10:21:48.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:48.854+0000 7f66eec00640 1 -- 172.21.15.45:0/4151366915 wait complete. 2024-04-25T10:21:48.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:21:38.360777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:21:39.578182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:21:48.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:21:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 Processor -- start 2024-04-25T10:21:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 -- start start 2024-04-25T10:21:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6dcc06af70 0x7f6dcc06b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071950 0x7f6dcc071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc06b890 con 0x7f6dcc0cf800 2024-04-25T10:21:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc0cfc90 con 0x7f6dcc06af70 2024-04-25T10:21:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dcb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0cff70 con 0x7f6dcc071950 2024-04-25T10:21:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dc1400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6dcc0cf800 0x7f6dcc06e060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60138/0 (socket says 172.21.15.45:60138) 2024-04-25T10:21:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dc1400640 1 -- 172.21.15.45:0/4159075307 learned_addr learned my addr 172.21.15.45:0/4159075307 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dc9e00640 1 --2- 172.21.15.45:0/4159075307 >> v2:172.21.15.117:3300/0 conn(0x7f6dcc06af70 0x7f6dcc06b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dca800640 1 --2- 172.21.15.45:0/4159075307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071950 0x7f6dcc071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dc9400640 1 -- 172.21.15.45:0/4159075307 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6606208 0 0) 0x7f6dcc06b890 con 0x7f6dcc0cf800 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dc9400640 1 -- 172.21.15.45:0/4159075307 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db0003530 con 0x7f6dcc0cf800 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dca800640 1 -- 172.21.15.45:0/4159075307 >> v2:172.21.15.117:3300/0 conn(0x7f6dcc06af70 msgr2=0x7f6dcc06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dca800640 1 --2- 172.21.15.45:0/4159075307 >> v2:172.21.15.117:3300/0 conn(0x7f6dcc06af70 0x7f6dcc06b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dca800640 1 -- 172.21.15.45:0/4159075307 >> v1:172.21.15.45:6789/0 conn(0x7f6dcc0cf800 legacy=0x7f6dcc06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dca800640 1 -- 172.21.15.45:0/4159075307 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0d0250 con 0x7f6dcc071950 2024-04-25T10:21:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.018+0000 7f6dc9e00640 1 --2- 172.21.15.45:0/4159075307 >> v2:172.21.15.117:3300/0 conn(0x7f6dcc06af70 0x7f6dcc06b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:49.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dca800640 1 --2- 172.21.15.45:0/4159075307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071950 0x7f6dcc071dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009d90 tx=0x7f6dc401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3194afaf6ae24d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dc9400640 1 -- 172.21.15.45:0/4159075307 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40b2070 con 0x7f6dcc071950 2024-04-25T10:21:49.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dc9400640 1 -- 172.21.15.45:0/4159075307 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc40a58e0 con 0x7f6dcc071950 2024-04-25T10:21:49.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dc9400640 1 -- 172.21.15.45:0/4159075307 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40a5cb0 con 0x7f6dcc071950 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 -- 172.21.15.45:0/4159075307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071950 msgr2=0x7f6dcc071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 --2- 172.21.15.45:0/4159075307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071950 0x7f6dcc071dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009d90 tx=0x7f6dc401c970 comp rx=0 tx=0).stop 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 -- 172.21.15.45:0/4159075307 shutdown_connections 2024-04-25T10:21:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 --2- 172.21.15.45:0/4159075307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071950 0x7f6dcc071dc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 --2- 172.21.15.45:0/4159075307 >> v2:172.21.15.117:3300/0 conn(0x7f6dcc06af70 0x7f6dcc06b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 -- 172.21.15.45:0/4159075307 >> 172.21.15.45:0/4159075307 conn(0x7f6dcc073d10 msgr2=0x7f6dcc074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 -- 172.21.15.45:0/4159075307 shutdown_connections 2024-04-25T10:21:49.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.022+0000 7f6dcb200640 1 -- 172.21.15.45:0/4159075307 wait complete. 2024-04-25T10:21:49.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 Processor -- start 2024-04-25T10:21:49.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 -- start start 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc06af70 0x7f6dcc13a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc071950 0x7f6dcc13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 0x7f6dcc0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc168650 con 0x7f6dcc140350 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1684d0 con 0x7f6dcc071950 2024-04-25T10:21:49.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1687d0 con 0x7f6dcc06af70 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 0x7f6dcc0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 0x7f6dcc0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46610/0 (socket says 172.21.15.45:46610) 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 -- 172.21.15.45:0/3519983345 learned_addr learned my addr 172.21.15.45:0/3519983345 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dc9e00640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc06af70 0x7f6dcc13a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dc1400640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc071950 0x7f6dcc13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 -- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc06af70 msgr2=0x7f6dcc13a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc06af70 0x7f6dcc13a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 -- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc071950 msgr2=0x7f6dcc13fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc071950 0x7f6dcc13fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 -- 172.21.15.45:0/3519983345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc140980 con 0x7f6dcc140350 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dc1400640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc071950 0x7f6dcc13fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dca800640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 0x7f6dcc0b2760 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009d90 tx=0x7f6dc4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40b2050 con 0x7f6dcc140350 2024-04-25T10:21:49.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc40be070 con 0x7f6dcc140350 2024-04-25T10:21:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40ada90 con 0x7f6dcc140350 2024-04-25T10:21:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc140c60 con 0x7f6dcc140350 2024-04-25T10:21:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.026+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc141220 con 0x7f6dcc140350 2024-04-25T10:21:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.030+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc075120 con 0x7f6dcc140350 2024-04-25T10:21:49.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.030+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6dc40a5760 con 0x7f6dcc140350 2024-04-25T10:21:49.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.030+0000 7f6dc2a00640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6da8066f60 0x7f6da8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.030+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dc4120160 con 0x7f6dcc140350 2024-04-25T10:21:49.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.030+0000 7f6dc9e00640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6da8066f60 0x7f6da8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.030+0000 7f6dc9e00640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6da8066f60 0x7f6da8069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009580 tx=0x7f6dbc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.034+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6dc4016560 con 0x7f6dcc140350 2024-04-25T10:21:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6dcc144580 con 0x7f6dcc140350 2024-04-25T10:21:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dc2a00640 1 -- 172.21.15.45:0/3519983345 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6dc40f10d0 con 0x7f6dcc140350 2024-04-25T10:21:49.202 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:21:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6da8066f60 msgr2=0x7f6da8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6da8066f60 0x7f6da8069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009580 tx=0x7f6dbc009190 comp rx=0 tx=0).stop 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 msgr2=0x7f6dcc0b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 0x7f6dcc0b2760 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009d90 tx=0x7f6dc4019710 comp rx=0 tx=0).stop 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 shutdown_connections 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6da8066f60 0x7f6da8069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.198+0000 7f6dcb200640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dcc140350 0x7f6dcc0b2760 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.202+0000 7f6dcb200640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc071950 0x7f6dcc13fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.202+0000 7f6dcb200640 1 --2- 172.21.15.45:0/3519983345 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc06af70 0x7f6dcc13a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.202+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 >> 172.21.15.45:0/3519983345 conn(0x7f6dcc073d10 msgr2=0x7f6dcc06b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.202+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 shutdown_connections 2024-04-25T10:21:49.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.202+0000 7f6dcb200640 1 -- 172.21.15.45:0/3519983345 wait complete. 2024-04-25T10:21:49.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:21:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 Processor -- start 2024-04-25T10:21:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 -- start start 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f914815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91480b31c0 0x7f91480b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914815b080 con 0x7f9148070350 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914815adb0 con 0x7f91480b31c0 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f914d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914815eb60 con 0x7f91480aef80 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9147400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91480b31c0 0x7f91480b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f913e000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9148070350 0x7f914815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60152/0 (socket says 172.21.15.45:60152) 2024-04-25T10:21:49.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f913e000640 1 -- 172.21.15.45:0/4010837466 learned_addr learned my addr 172.21.15.45:0/4010837466 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9146a00640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f914815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9147400640 1 -- 172.21.15.45:0/4010837466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 msgr2=0x7f914815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9147400640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f914815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9147400640 1 -- 172.21.15.45:0/4010837466 >> v1:172.21.15.45:6789/0 conn(0x7f9148070350 legacy=0x7f914815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9147400640 1 -- 172.21.15.45:0/4010837466 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914815ee40 con 0x7f91480b31c0 2024-04-25T10:21:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.366+0000 7f9146a00640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f914815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f9147400640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91480b31c0 0x7f91480b3610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9140009d90 tx=0x7f914001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea3176a1023986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f9146000640 1 -- 172.21.15.45:0/4010837466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91400b2070 con 0x7f91480b31c0 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f9146000640 1 -- 172.21.15.45:0/4010837466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91400a5940 con 0x7f91480b31c0 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f9146000640 1 -- 172.21.15.45:0/4010837466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91400a5d10 con 0x7f91480b31c0 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 -- 172.21.15.45:0/4010837466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91480b31c0 msgr2=0x7f91480b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91480b31c0 0x7f91480b3610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9140009d90 tx=0x7f914001c970 comp rx=0 tx=0).stop 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 -- 172.21.15.45:0/4010837466 shutdown_connections 2024-04-25T10:21:49.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91480b31c0 0x7f91480b3610 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 --2- 172.21.15.45:0/4010837466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f914815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 -- 172.21.15.45:0/4010837466 >> 172.21.15.45:0/4010837466 conn(0x7f91480669a0 msgr2=0x7f9148066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 -- 172.21.15.45:0/4010837466 shutdown_connections 2024-04-25T10:21:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.370+0000 7f914d800640 1 -- 172.21.15.45:0/4010837466 wait complete. 2024-04-25T10:21:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 Processor -- start 2024-04-25T10:21:49.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 -- start start 2024-04-25T10:21:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 0x7f9148149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f9148155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480b31c0 0x7f91480ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481679a0 con 0x7f9148070350 2024-04-25T10:21:49.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148167820 con 0x7f91480b31c0 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9148167b20 con 0x7f91480aef80 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 0x7f9148149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 0x7f9148149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46624/0 (socket says 172.21.15.45:46624) 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 -- 172.21.15.45:0/933641701 learned_addr learned my addr 172.21.15.45:0/933641701 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 -- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 msgr2=0x7f9148155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f913e000640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f9148155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9147400640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480b31c0 0x7f91480ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f9148155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 -- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480b31c0 msgr2=0x7f91480ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480b31c0 0x7f91480ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 -- 172.21.15.45:0/933641701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148151760 con 0x7f9148070350 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f913e000640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f9148155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9147400640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480b31c0 0x7f91480ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:49.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f9146a00640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 0x7f9148149b40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f913800e810 tx=0x7f913800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138017070 con 0x7f9148070350 2024-04-25T10:21:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800c8d0 con 0x7f9148070350 2024-04-25T10:21:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9148151a40 con 0x7f9148070350 2024-04-25T10:21:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800f040 con 0x7f9148070350 2024-04-25T10:21:49.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.374+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9148152000 con 0x7f9148070350 2024-04-25T10:21:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.378+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914804ef30 con 0x7f9148070350 2024-04-25T10:21:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.378+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f913801a070 con 0x7f9148070350 2024-04-25T10:21:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.378+0000 7f913f400640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9124066f60 0x7f9124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.378+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9138084c70 con 0x7f9148070350 2024-04-25T10:21:49.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.378+0000 7f913e000640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9124066f60 0x7f9124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.378+0000 7f913e000640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9124066f60 0x7f9124069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9130007fd0 tx=0x7f9130009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.382+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9138055c70 con 0x7f9148070350 2024-04-25T10:21:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.542+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91480b2fa0 con 0x7f9148070350 2024-04-25T10:21:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.542+0000 7f913f400640 1 -- 172.21.15.45:0/933641701 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9138055570 con 0x7f9148070350 2024-04-25T10:21:49.547 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9124066f60 msgr2=0x7f9124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9124066f60 0x7f9124069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9130007fd0 tx=0x7f9130009040 comp rx=0 tx=0).stop 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 msgr2=0x7f9148149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 0x7f9148149b40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f913800e810 tx=0x7f913800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 shutdown_connections 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9124066f60 0x7f9124069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91480b31c0 0x7f91480ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91480aef80 0x7f9148155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 --2- 172.21.15.45:0/933641701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9148070350 0x7f9148149b40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 >> 172.21.15.45:0/933641701 conn(0x7f91480669a0 msgr2=0x7f91480cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 shutdown_connections 2024-04-25T10:21:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.546+0000 7f914d800640 1 -- 172.21.15.45:0/933641701 wait complete. 2024-04-25T10:21:49.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:21:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 Processor -- start 2024-04-25T10:21:49.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 -- start start 2024-04-25T10:21:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f39cc159540 0x7f39cc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39cc0aebd0 0x7f39cc071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc071360 con 0x7f39cc072020 2024-04-25T10:21:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc159920 con 0x7f39cc159540 2024-04-25T10:21:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39d0c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc15c2f0 con 0x7f39cc0aebd0 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f39cc159540 0x7f39cc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f39cc159540 0x7f39cc0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46634/0 (socket says 172.21.15.45:46634) 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 -- 172.21.15.45:0/2080997523 learned_addr learned my addr 172.21.15.45:0/2080997523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9400640 1 --2- 172.21.15.45:0/2080997523 >> v2:172.21.15.117:3300/0 conn(0x7f39cc0aebd0 0x7f39cc071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 -- 172.21.15.45:0/2080997523 >> v2:172.21.15.117:3300/0 conn(0x7f39cc0aebd0 msgr2=0x7f39cc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 --2- 172.21.15.45:0/2080997523 >> v2:172.21.15.117:3300/0 conn(0x7f39cc0aebd0 0x7f39cc071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 -- 172.21.15.45:0/2080997523 >> v1:172.21.15.117:6789/0 conn(0x7f39cc072020 legacy=0x7f39cc15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.710+0000 7f39c9e00640 1 -- 172.21.15.45:0/2080997523 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc15c5d0 con 0x7f39cc159540 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39c9e00640 1 --2- 172.21.15.45:0/2080997523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc159540 0x7f39cc0ae690 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0cd4e0 tx=0x7f39bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dad4b8d61ce40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39c8a00640 1 -- 172.21.15.45:0/2080997523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0b0070 con 0x7f39cc159540 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39c8a00640 1 -- 172.21.15.45:0/2080997523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc0ab600 con 0x7f39cc159540 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39c8a00640 1 -- 172.21.15.45:0/2080997523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0ab9d0 con 0x7f39cc159540 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 -- 172.21.15.45:0/2080997523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc159540 msgr2=0x7f39cc0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 --2- 172.21.15.45:0/2080997523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc159540 0x7f39cc0ae690 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0cd4e0 tx=0x7f39bc01c520 comp rx=0 tx=0).stop 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 -- 172.21.15.45:0/2080997523 shutdown_connections 2024-04-25T10:21:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 --2- 172.21.15.45:0/2080997523 >> v2:172.21.15.117:3300/0 conn(0x7f39cc0aebd0 0x7f39cc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 --2- 172.21.15.45:0/2080997523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc159540 0x7f39cc0ae690 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 -- 172.21.15.45:0/2080997523 >> 172.21.15.45:0/2080997523 conn(0x7f39cc0669a0 msgr2=0x7f39cc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 -- 172.21.15.45:0/2080997523 shutdown_connections 2024-04-25T10:21:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 -- 172.21.15.45:0/2080997523 wait complete. 2024-04-25T10:21:49.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 Processor -- start 2024-04-25T10:21:49.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.714+0000 7f39d0c00640 1 -- start start 2024-04-25T10:21:49.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 0x7f39cc150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39cc0aebd0 0x7f39cc13f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39cc159540 0x7f39cc1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc072430 con 0x7f39cc072020 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc06c0d0 con 0x7f39cc0aebd0 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc168930 con 0x7f39cc159540 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 0x7f39cc150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 0x7f39cc150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46644/0 (socket says 172.21.15.45:46644) 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 -- 172.21.15.45:0/856348872 learned_addr learned my addr 172.21.15.45:0/856348872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 -- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39cc159540 msgr2=0x7f39cc1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9400640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39cc0aebd0 0x7f39cc13f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39ca800640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39cc159540 0x7f39cc1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39cc159540 0x7f39cc1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 -- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39cc0aebd0 msgr2=0x7f39cc13f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39cc0aebd0 0x7f39cc13f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 -- 172.21.15.45:0/856348872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc14fe20 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9400640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39cc0aebd0 0x7f39cc13f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39ca800640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39cc159540 0x7f39cc1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c9e00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 0x7f39cc150ea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0ae040 tx=0x7f39bc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0bb070 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc0b6430 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc1500e0 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0b0040 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc1506a0 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.718+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc04ef30 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.722+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39bc0be070 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.722+0000 7f39c2000640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f39ac066f60 0x7f39ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.722+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39bc11e600 con 0x7f39cc072020 2024-04-25T10:21:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.722+0000 7f39c9400640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f39ac066f60 0x7f39ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.722+0000 7f39c9400640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f39ac066f60 0x7f39ac069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0af610 tx=0x7f39b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.726+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39bc016110 con 0x7f39cc072020 2024-04-25T10:21:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39cc0cf470 con 0x7f39cc072020 2024-04-25T10:21:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39c2000640 1 -- 172.21.15.45:0/856348872 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39bc0ef570 con 0x7f39cc072020 2024-04-25T10:21:49.893 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f39ac066f60 msgr2=0x7f39ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f39ac066f60 0x7f39ac069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0af610 tx=0x7f39b8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 msgr2=0x7f39cc150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 0x7f39cc150ea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0ae040 tx=0x7f39bc0192c0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 shutdown_connections 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f39ac066f60 0x7f39ac069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39cc159540 0x7f39cc1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39cc0aebd0 0x7f39cc13f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 --2- 172.21.15.45:0/856348872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39cc072020 0x7f39cc150ea0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 >> 172.21.15.45:0/856348872 conn(0x7f39cc0669a0 msgr2=0x7f39cc0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 shutdown_connections 2024-04-25T10:21:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:49.890+0000 7f39d0c00640 1 -- 172.21.15.45:0/856348872 wait complete. 2024-04-25T10:21:49.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:21:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 Processor -- start 2024-04-25T10:21:50.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 -- start start 2024-04-25T10:21:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faadc15da50 0x7faadc0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc0b16c0 0x7faadc0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faadc066ee0 0x7faadc06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faadc15de30 con 0x7faadc15da50 2024-04-25T10:21:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faadc0b59f0 con 0x7faadc066ee0 2024-04-25T10:21:50.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faadb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0b5cd0 con 0x7faadc0b16c0 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faadc15da50 0x7faadc0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faadc15da50 0x7faadc0b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46660/0 (socket says 172.21.15.45:46660) 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 -- 172.21.15.45:0/1286802763 learned_addr learned my addr 172.21.15.45:0/1286802763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad3e00640 1 --2- 172.21.15.45:0/1286802763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc0b16c0 0x7faadc0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faada800640 1 --2- 172.21.15.45:0/1286802763 >> v2:172.21.15.117:3300/0 conn(0x7faadc066ee0 0x7faadc06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 -- 172.21.15.45:0/1286802763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc0b16c0 msgr2=0x7faadc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 --2- 172.21.15.45:0/1286802763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc0b16c0 0x7faadc0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 -- 172.21.15.45:0/1286802763 >> v2:172.21.15.117:3300/0 conn(0x7faadc066ee0 msgr2=0x7faadc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 --2- 172.21.15.45:0/1286802763 >> v2:172.21.15.117:3300/0 conn(0x7faadc066ee0 0x7faadc06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faad9e00640 1 -- 172.21.15.45:0/1286802763 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0b5fb0 con 0x7faadc15da50 2024-04-25T10:21:50.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.078+0000 7faada800640 1 --2- 172.21.15.45:0/1286802763 >> v2:172.21.15.117:3300/0 conn(0x7faadc066ee0 0x7faadc06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:50.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faad9e00640 1 --2- 172.21.15.45:0/1286802763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc15da50 0x7faadc0b1d10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faadc0cdc10 tx=0x7faacc01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e0e4eb4a297340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faad9400640 1 -- 172.21.15.45:0/1286802763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc0b0070 con 0x7faadc15da50 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faad9400640 1 -- 172.21.15.45:0/1286802763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc0ab600 con 0x7faadc15da50 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faad9400640 1 -- 172.21.15.45:0/1286802763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc0ab9d0 con 0x7faadc15da50 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 -- 172.21.15.45:0/1286802763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc15da50 msgr2=0x7faadc0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 --2- 172.21.15.45:0/1286802763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc15da50 0x7faadc0b1d10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7faadc0cdc10 tx=0x7faacc01c1d0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 -- 172.21.15.45:0/1286802763 shutdown_connections 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 --2- 172.21.15.45:0/1286802763 >> v2:172.21.15.117:3300/0 conn(0x7faadc066ee0 0x7faadc06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 --2- 172.21.15.45:0/1286802763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc0b16c0 0x7faadc0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 --2- 172.21.15.45:0/1286802763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc15da50 0x7faadc0b1d10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 -- 172.21.15.45:0/1286802763 >> 172.21.15.45:0/1286802763 conn(0x7faadc0d0690 msgr2=0x7faadc0d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 -- 172.21.15.45:0/1286802763 shutdown_connections 2024-04-25T10:21:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.082+0000 7faadb200640 1 -- 172.21.15.45:0/1286802763 wait complete. 2024-04-25T10:21:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 Processor -- start 2024-04-25T10:21:50.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- start start 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc066ee0 0x7faadc0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 0x7faadc14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faadc15da50 0x7faadc14db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0cea40 con 0x7faadc0b16c0 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0ce8c0 con 0x7faadc15da50 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0cebc0 con 0x7faadc066ee0 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 0x7faadc14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 0x7faadc14c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46672/0 (socket says 172.21.15.45:46672) 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 -- 172.21.15.45:0/1250841928 learned_addr learned my addr 172.21.15.45:0/1250841928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad9e00640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc066ee0 0x7faadc0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faada800640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faadc15da50 0x7faadc14db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 -- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc066ee0 msgr2=0x7faadc0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc066ee0 0x7faadc0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 -- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faadc15da50 msgr2=0x7faadc14db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faadc15da50 0x7faadc14db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 -- 172.21.15.45:0/1250841928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc1558b0 con 0x7faadc0b16c0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad3e00640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 0x7faadc14c3a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faac400e930 tx=0x7faac400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac4017070 con 0x7faadc0b16c0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac400c9a0 con 0x7faadc0b16c0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc155b90 con 0x7faadc0b16c0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac400cd50 con 0x7faadc0b16c0 2024-04-25T10:21:50.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc156150 con 0x7faadc0b16c0 2024-04-25T10:21:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.086+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc075120 con 0x7faadc0b16c0 2024-04-25T10:21:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.090+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faac401a070 con 0x7faadc0b16c0 2024-04-25T10:21:50.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.090+0000 7faad2000640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faab8066f60 0x7faab8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.090+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faac4085350 con 0x7faadc0b16c0 2024-04-25T10:21:50.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.090+0000 7faad9e00640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faab8066f60 0x7faab8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.090+0000 7faad9e00640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faab8066f60 0x7faab8069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faacc0ad000 tx=0x7faacc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.098+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faac4055d30 con 0x7faadc0b16c0 2024-04-25T10:21:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.270+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faadc153ed0 con 0x7faadc0b16c0 2024-04-25T10:21:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faad2000640 1 -- 172.21.15.45:0/1250841928 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7faac4059dd0 con 0x7faadc0b16c0 2024-04-25T10:21:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:21:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faab8066f60 msgr2=0x7faab8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faab8066f60 0x7faab8069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faacc0ad000 tx=0x7faacc008040 comp rx=0 tx=0).stop 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 msgr2=0x7faadc14c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 0x7faadc14c3a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faac400e930 tx=0x7faac400edf0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 shutdown_connections 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faab8066f60 0x7faab8069420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faadc15da50 0x7faadc14db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faadc0b16c0 0x7faadc14c3a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 --2- 172.21.15.45:0/1250841928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faadc066ee0 0x7faadc0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 >> 172.21.15.45:0/1250841928 conn(0x7faadc0d0690 msgr2=0x7faadc15b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 shutdown_connections 2024-04-25T10:21:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.274+0000 7faadb200640 1 -- 172.21.15.45:0/1250841928 wait complete. 2024-04-25T10:21:50.298 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:21:50.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:50.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:50.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:21:50.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:50.355 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:21:50.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6562000640 1 Processor -- start 2024-04-25T10:21:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6562000640 1 -- start start 2024-04-25T10:21:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6562000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f655c15e050 0x7f655c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6562000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c0b1af0 con 0x7f655c0d1820 2024-04-25T10:21:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6562000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655c0b1dd0 con 0x7f655c0b13e0 2024-04-25T10:21:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6562000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655c0b20b0 con 0x7f655c15e050 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6552a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f655c15e050 0x7f655c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f655b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f655c0d1820 0x7f655c0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60166/0 (socket says 172.21.15.45:60166) 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f655b400640 1 -- 172.21.15.45:0/2788855575 learned_addr learned my addr 172.21.15.45:0/2788855575 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6552a00640 1 -- 172.21.15.45:0/2788855575 >> v1:172.21.15.117:6789/0 conn(0x7f655c0b13e0 legacy=0x7f655c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6552a00640 1 -- 172.21.15.45:0/2788855575 >> v1:172.21.15.45:6789/0 conn(0x7f655c0d1820 legacy=0x7f655c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f6552a00640 1 -- 172.21.15.45:0/2788855575 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0b2390 con 0x7f655c15e050 2024-04-25T10:21:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.550+0000 7f655aa00640 1 -- 172.21.15.45:0/2788855575 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717531266 0 0) 0x7f655c0b1af0 con 0x7f655c0d1820 2024-04-25T10:21:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6552a00640 1 --2- 172.21.15.45:0/2788855575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c0b0ea0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f65440097c0 tx=0x7f654401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d0abb90232ab0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f655aa00640 1 -- 172.21.15.45:0/2788855575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440b1070 con 0x7f655c15e050 2024-04-25T10:21:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f655aa00640 1 -- 172.21.15.45:0/2788855575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65440ac410 con 0x7f655c15e050 2024-04-25T10:21:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f655aa00640 1 -- 172.21.15.45:0/2788855575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440ac7e0 con 0x7f655c15e050 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 -- 172.21.15.45:0/2788855575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 msgr2=0x7f655c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 --2- 172.21.15.45:0/2788855575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c0b0ea0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f65440097c0 tx=0x7f654401c410 comp rx=0 tx=0).stop 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 -- 172.21.15.45:0/2788855575 shutdown_connections 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 --2- 172.21.15.45:0/2788855575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c0b0ea0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 -- 172.21.15.45:0/2788855575 >> 172.21.15.45:0/2788855575 conn(0x7f655c09a9d0 msgr2=0x7f655c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 -- 172.21.15.45:0/2788855575 shutdown_connections 2024-04-25T10:21:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 -- 172.21.15.45:0/2788855575 wait complete. 2024-04-25T10:21:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 Processor -- start 2024-04-25T10:21:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.554+0000 7f6562000640 1 -- start start 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f655c0b13e0 0x7f655c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f655c0d1820 0x7f655c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0d0fa0 con 0x7f655c15e050 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c0d0e20 con 0x7f655c0b13e0 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c16b9b0 con 0x7f655c0d1820 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c15f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46690/0 (socket says 172.21.15.45:46690) 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 -- 172.21.15.45:0/555010871 learned_addr learned my addr 172.21.15.45:0/555010871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6552a00640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f655c0d1820 0x7f655c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655b400640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f655c0b13e0 0x7f655c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 -- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f655c0d1820 msgr2=0x7f655c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f655c0d1820 0x7f655c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 -- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f655c0b13e0 msgr2=0x7f655c155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f655c0b13e0 0x7f655c155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 -- 172.21.15.45:0/555010871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0d27a0 con 0x7f655c15e050 2024-04-25T10:21:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f655be00640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c15f6d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f655400c770 tx=0x7f655400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554003b80 con 0x7f655c15e050 2024-04-25T10:21:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554003d20 con 0x7f655c15e050 2024-04-25T10:21:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655400aa90 con 0x7f655c15e050 2024-04-25T10:21:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c06c7e0 con 0x7f655c15e050 2024-04-25T10:21:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c1565e0 con 0x7f655c15e050 2024-04-25T10:21:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655c14c9e0 con 0x7f655c15e050 2024-04-25T10:21:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.558+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f655400ac30 con 0x7f655c15e050 2024-04-25T10:21:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.562+0000 7f6553e00640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6538066f60 0x7f6538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.562+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6554084e30 con 0x7f655c15e050 2024-04-25T10:21:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.562+0000 7f655b400640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6538066f60 0x7f6538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.562+0000 7f655b400640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6538066f60 0x7f6538069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f654c000f70 tx=0x7f654c007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.566+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f655c14c9e0 con 0x7f655c15e050 2024-04-25T10:21:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.758+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f655c06b480 con 0x7f655c15e050 2024-04-25T10:21:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.758+0000 7f6553e00640 1 -- 172.21.15.45:0/555010871 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f65540557e0 con 0x7f655c15e050 2024-04-25T10:21:50.763 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:50.763 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5696,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2258543519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2258543519},{"type":"v1","addr":"172.21.15.45:6835","nonce":2258543519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5699,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3808318987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3808318987},{"type":"v1","addr":"172.21.15.117:6836","nonce":3808318987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":6384,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/317823919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":317823919},{"type":"v1","addr":"172.21.15.45:6834","nonce":317823919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:43.601865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/3997755731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3997755731},{"type":"v1","addr":"172.21.15.117:6837","nonce":3997755731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:21:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 39 2024-04-25T10:21:50.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6538066f60 msgr2=0x7f6538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6538066f60 0x7f6538069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f654c000f70 tx=0x7f654c007240 comp rx=0 tx=0).stop 2024-04-25T10:21:50.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 msgr2=0x7f655c15f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:50.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c15f6d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f655400c770 tx=0x7f655400cc30 comp rx=0 tx=0).stop 2024-04-25T10:21:50.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 shutdown_connections 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6538066f60 0x7f6538069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f655c15e050 0x7f655c15f6d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f655c0d1820 0x7f655c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 --2- 172.21.15.45:0/555010871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f655c0b13e0 0x7f655c155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 >> 172.21.15.45:0/555010871 conn(0x7f655c09a9d0 msgr2=0x7f655c15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 shutdown_connections 2024-04-25T10:21:50.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:50.762+0000 7f6562000640 1 -- 172.21.15.45:0/555010871 wait complete. 2024-04-25T10:21:50.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:21:50.957 INFO:teuthology.orchestra.run.smithi117.stdout:{"message":"","return_code":0} 2024-04-25T10:21:50.958 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:21:50.958 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:21:50.958 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:50.959 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:21:50.959 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:50.959 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:21:50.959 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:50.960 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:21:50.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:21:50.972 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:21:50.972 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:50.973 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:21:50.973 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:50.974 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:21:50.974 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:50.975 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:21:50.975 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:50.975 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T10:21:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.062+0000 7f7814d26080 1 Processor -- start 2024-04-25T10:21:51.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.062+0000 7f7814d26080 1 -- start start 2024-04-25T10:21:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8820 0x562983cda5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 0x562983cb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983d862b0 0x562983cb1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562983cdd510 con 0x562983cf8c00 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562983c559f0 con 0x562983cf8820 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562983cdd690 con 0x562983d862b0 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 0x562983cb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 0x562983cb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46698/0 (socket says 172.21.15.45:46698) 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 -- 172.21.15.45:0/1048574735 learned_addr learned my addr 172.21.15.45:0/1048574735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7812a00640 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983d862b0 0x562983cb1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7812000640 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8820 0x562983cda5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 -- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983d862b0 msgr2=0x562983cb1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983d862b0 0x562983cb1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 -- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8820 msgr2=0x562983cda5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8820 0x562983cda5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 -- 172.21.15.45:0/1048574735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562983d8a360 con 0x562983cf8c00 2024-04-25T10:21:51.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7811600640 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 0x562983cb1680 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f780000b2f0 tx=0x7f780000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6612a3e18b4174a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f780be00640 1 -- 172.21.15.45:0/1048574735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780000d510 con 0x562983cf8c00 2024-04-25T10:21:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f780be00640 1 -- 172.21.15.45:0/1048574735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7800008070 con 0x562983cf8c00 2024-04-25T10:21:51.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f780be00640 1 -- 172.21.15.45:0/1048574735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780000d8d0 con 0x562983cf8c00 2024-04-25T10:21:51.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- 172.21.15.45:0/1048574735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 msgr2=0x562983cb1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 0x562983cb1680 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f780000b2f0 tx=0x7f780000b7b0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- 172.21.15.45:0/1048574735 shutdown_connections 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983d862b0 0x562983cb1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983cf8c00 0x562983cb1680 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- 172.21.15.45:0/1048574735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8820 0x562983cda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- 172.21.15.45:0/1048574735 >> 172.21.15.45:0/1048574735 conn(0x562983cd9760 msgr2=0x562983d8f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- 172.21.15.45:0/1048574735 shutdown_connections 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- 172.21.15.45:0/1048574735 wait complete. 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 Processor -- start 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- start start 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983cf8820 0x562983e1f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8c00 0x562983cf7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983d862b0 0x562983d83ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562983d87970 con 0x562983d862b0 2024-04-25T10:21:51.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562983d8a600 con 0x562983cf8c00 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7814d26080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562983d87c10 con 0x562983cf8820 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7812a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983d862b0 0x562983d83ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7812000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983cf8820 0x562983e1f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.066+0000 7f7812a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983d862b0 0x562983d83ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46712/0 (socket says 172.21.15.45:46712) 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 -- 172.21.15.45:0/1947278215 learned_addr learned my addr 172.21.15.45:0/1947278215 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7811600640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8c00 0x562983cf7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 -- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983cf8820 msgr2=0x562983e1f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983cf8820 0x562983e1f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 -- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8c00 msgr2=0x562983cf7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562983cf8c00 0x562983cf7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 -- 172.21.15.45:0/1947278215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562983d813a0 con 0x562983d862b0 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812000640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562983cf8820 0x562983e1f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7812a00640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983d862b0 0x562983d83ba0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f780c018df0 tx=0x7f780c017f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f780aa00640 1 -- 172.21.15.45:0/1947278215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0aab60 con 0x562983d862b0 2024-04-25T10:21:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f780aa00640 1 -- 172.21.15.45:0/1947278215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c0aad00 con 0x562983d862b0 2024-04-25T10:21:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562983d81640 con 0x562983d862b0 2024-04-25T10:21:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f780aa00640 1 -- 172.21.15.45:0/1947278215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0c0680 con 0x562983d862b0 2024-04-25T10:21:51.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562983d81b80 con 0x562983d862b0 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f780aa00640 1 -- 172.21.15.45:0/1947278215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f780c0b5410 con 0x562983d862b0 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f780aa00640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77f4065970 0x7f77f4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.070+0000 7f780aa00640 1 -- 172.21.15.45:0/1947278215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f780c11f400 con 0x562983d862b0 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x562983cf7fe0 0x562983e22920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7812000640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77f4065970 0x7f77f4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 --> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] -- osd_op(unknown.0.0:1 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62) v9 -- 0x562983d840e0 con 0x562983cf7fe0 2024-04-25T10:21:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7811600640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x562983cf7fe0 0x562983e22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7812000640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77f4065970 0x7f77f4067e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7804005d00 tx=0x7f780400c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7811600640 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x562983cf7fe0 0x562983e22920 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7811600640 1 -- 172.21.15.45:0/1947278215 <== osd.3 v2:172.21.15.45:6800/3296322295 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v62'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f780000d610 con 0x562983cf7fe0 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x562983cf7fe0 msgr2=0x562983e22920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x562983cf7fe0 0x562983e22920 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77f4065970 msgr2=0x7f77f4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77f4065970 0x7f77f4067e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7804005d00 tx=0x7f780400c040 comp rx=0 tx=0).stop 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983d862b0 msgr2=0x562983d83ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 --2- 172.21.15.45:0/1947278215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562983d862b0 0x562983d83ba0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f780c018df0 tx=0x7f780c017f60 comp rx=0 tx=0).stop 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7812a00640 1 -- 172.21.15.45:0/1947278215 reap_dead start 2024-04-25T10:21:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 shutdown_connections 2024-04-25T10:21:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 >> 172.21.15.45:0/1947278215 conn(0x562983cd9760 msgr2=0x562983cda690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 shutdown_connections 2024-04-25T10:21:51.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.074+0000 7f7814d26080 1 -- 172.21.15.45:0/1947278215 wait complete. 2024-04-25T10:21:51.083 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T10:21:51.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T10:21:51.084 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T10:21:51.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T10:21:51.085 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T10:21:51.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T10:21:51.085 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T10:21:51.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T10:21:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 Processor -- start 2024-04-25T10:21:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 -- start start 2024-04-25T10:21:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 Processor -- start 2024-04-25T10:21:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 -- start start 2024-04-25T10:21:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 Processor -- start 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 -- start start 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4c40d1c30 0x7fc4c40cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c40ccfb0 con 0x7fc4c415bdc0 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c40cd290 con 0x7fc4c40d1c30 2024-04-25T10:21:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4cc600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40cd570 con 0x7fc4c415b9e0 2024-04-25T10:21:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 Processor -- start 2024-04-25T10:21:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa0b8161120 0x7fa0b8071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 -- start start 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0b806e5c0 0x7fa0b80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b806eb30 con 0x7fa0b80669a0 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80cd5d0 con 0x7fa0b806e5c0 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0be600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b806ee10 con 0x7fa0b8161120 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4ca600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4c40d1c30 0x7fc4c40cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c40cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:60526/0 (socket says 172.21.15.45:60526) 2024-04-25T10:21:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 -- 172.21.15.45:0/3883104217 learned_addr learned my addr 172.21.15.45:0/3883104217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa0b8161120 0x7fa0b8071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa0b8161120 0x7fa0b8071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56058/0 (socket says 172.21.15.45:56058) 2024-04-25T10:21:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 -- 172.21.15.45:0/1055816286 learned_addr learned my addr 172.21.15.45:0/1055816286 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b6a00640 1 --2- 172.21.15.45:0/1055816286 >> v2:172.21.15.45:3300/0 conn(0x7fa0b806e5c0 0x7fa0b80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 -- 172.21.15.45:0/3883104217 >> v2:172.21.15.117:3300/0 conn(0x7fc4c40d1c30 msgr2=0x7fc4c40cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 --2- 172.21.15.45:0/3883104217 >> v2:172.21.15.117:3300/0 conn(0x7fc4c40d1c30 0x7fc4c40cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 -- 172.21.15.45:0/3883104217 >> v1:172.21.15.117:6789/0 conn(0x7fc4c415bdc0 legacy=0x7fc4c40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4c9c00640 1 -- 172.21.15.45:0/3883104217 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c40cd850 con 0x7fc4c415b9e0 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fc4ca600640 1 --2- 172.21.15.45:0/3883104217 >> v2:172.21.15.117:3300/0 conn(0x7fc4c40d1c30 0x7fc4c40cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 -- 172.21.15.45:0/1055816286 >> v2:172.21.15.45:3300/0 conn(0x7fa0b806e5c0 msgr2=0x7fa0b80cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 --2- 172.21.15.45:0/1055816286 >> v2:172.21.15.45:3300/0 conn(0x7fa0b806e5c0 0x7fa0b80cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 -- 172.21.15.45:0/1055816286 >> v1:172.21.15.45:6789/0 conn(0x7fa0b80669a0 legacy=0x7fa0b8161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa0b7400640 1 -- 172.21.15.45:0/1055816286 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b806f0f0 con 0x7fa0b8161120 2024-04-25T10:21:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa9a0161120 0x7fa9a0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa9a006e5c0 0x7fa9a00cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a006eb30 con 0x7fa9a00669a0 2024-04-25T10:21:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a00cd5d0 con 0x7fa9a006e5c0 2024-04-25T10:21:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa9a5a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a006ee10 con 0x7fa9a0161120 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa9a0161120 0x7fa9a0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa9a0161120 0x7fa9a0071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56070/0 (socket says 172.21.15.45:56070) 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99ea00640 1 -- 172.21.15.45:0/2208204517 learned_addr learned my addr 172.21.15.45:0/2208204517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99e000640 1 --2- 172.21.15.45:0/2208204517 >> v2:172.21.15.45:3300/0 conn(0x7fa9a006e5c0 0x7fa9a00cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61340670b0 0x7f6134165ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f61340c3f30 0x7f61341663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6134072070 con 0x7f61340d0120 2024-04-25T10:21:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61340ccd90 con 0x7f61340c3f30 2024-04-25T10:21:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7f6138e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6134072350 con 0x7f61340670b0 2024-04-25T10:21:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99ea00640 1 -- 172.21.15.45:0/2208204517 >> v2:172.21.15.45:3300/0 conn(0x7fa9a006e5c0 msgr2=0x7fa9a00cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99ea00640 1 --2- 172.21.15.45:0/2208204517 >> v2:172.21.15.45:3300/0 conn(0x7fa9a006e5c0 0x7fa9a00cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.274+0000 7fa99ea00640 1 -- 172.21.15.45:0/2208204517 >> v1:172.21.15.45:6789/0 conn(0x7fa9a00669a0 legacy=0x7fa9a0161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6131e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f61340d0120 0x7f61340669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:49958/0 (socket says 172.21.15.45:49958) 2024-04-25T10:21:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6131e00640 1 -- 172.21.15.45:0/4257281035 learned_addr learned my addr 172.21.15.45:0/4257281035 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f612be00640 1 --2- 172.21.15.45:0/4257281035 >> v2:172.21.15.117:3300/0 conn(0x7f61340670b0 0x7f6134165ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6132800640 1 --2- 172.21.15.45:0/4257281035 >> v2:172.21.15.45:3300/0 conn(0x7f61340c3f30 0x7f61341663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fa99ea00640 1 -- 172.21.15.45:0/2208204517 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a006f0f0 con 0x7fa9a0161120 2024-04-25T10:21:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6131400640 1 -- 172.21.15.45:0/4257281035 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181408013 0 0) 0x7f6134072070 con 0x7f61340d0120 2024-04-25T10:21:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6131400640 1 -- 172.21.15.45:0/4257281035 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6118003530 con 0x7f61340d0120 2024-04-25T10:21:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f612be00640 1 -- 172.21.15.45:0/4257281035 >> v2:172.21.15.45:3300/0 conn(0x7f61340c3f30 msgr2=0x7f61341663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6131400640 1 -- 172.21.15.45:0/4257281035 <== mon.1 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2710965315 0 0) 0x7f6118003530 con 0x7f61340d0120 2024-04-25T10:21:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f612be00640 1 --2- 172.21.15.45:0/4257281035 >> v2:172.21.15.45:3300/0 conn(0x7f61340c3f30 0x7f61341663e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f612be00640 1 -- 172.21.15.45:0/4257281035 >> v1:172.21.15.117:6789/0 conn(0x7f61340d0120 legacy=0x7f61340669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f612be00640 1 -- 172.21.15.45:0/4257281035 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613406e6f0 con 0x7f61340670b0 2024-04-25T10:21:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f6132800640 1 --2- 172.21.15.45:0/4257281035 >> v2:172.21.15.45:3300/0 conn(0x7f61340c3f30 0x7f61341663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4c9c00640 1 --2- 172.21.15.45:0/3883104217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c40cc410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfdc5c7a790f4a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4c3e00640 1 -- 172.21.15.45:0/3883104217 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80b2070 con 0x7fc4c415b9e0 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4c3e00640 1 -- 172.21.15.45:0/3883104217 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b80a9070 con 0x7fc4c415b9e0 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4c3e00640 1 -- 172.21.15.45:0/3883104217 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80ad880 con 0x7fc4c415b9e0 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 -- 172.21.15.45:0/3883104217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 msgr2=0x7fc4c40cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 --2- 172.21.15.45:0/3883104217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c40cc410 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009870 tx=0x7fc4b801c560 comp rx=0 tx=0).stop 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 -- 172.21.15.45:0/3883104217 shutdown_connections 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 --2- 172.21.15.45:0/3883104217 >> v2:172.21.15.117:3300/0 conn(0x7fc4c40d1c30 0x7fc4c40cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 --2- 172.21.15.45:0/3883104217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c40cc410 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 -- 172.21.15.45:0/3883104217 >> 172.21.15.45:0/3883104217 conn(0x7fc4c409a9d0 msgr2=0x7fc4c409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 -- 172.21.15.45:0/3883104217 shutdown_connections 2024-04-25T10:21:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fc4cc600640 1 -- 172.21.15.45:0/3883104217 wait complete. 2024-04-25T10:21:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7fa0b7400640 1 --2- 172.21.15.45:0/1055816286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b8161120 0x7fa0b8071fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009870 tx=0x7fa0a801c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b101b25108c251b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.278+0000 7f612be00640 1 --2- 172.21.15.45:0/4257281035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134165ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61340b3460 tx=0x7f611c01c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=284d20929df70e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0b6000640 1 -- 172.21.15.45:0/1055816286 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80b1070 con 0x7fa0b8161120 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0b6000640 1 -- 172.21.15.45:0/1055816286 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a80ac620 con 0x7fa0b8161120 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0b6000640 1 -- 172.21.15.45:0/1055816286 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80ac9f0 con 0x7fa0b8161120 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6131400640 1 -- 172.21.15.45:0/4257281035 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b0070 con 0x7f61340670b0 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6131400640 1 -- 172.21.15.45:0/4257281035 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c0ab400 con 0x7f61340670b0 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6131400640 1 -- 172.21.15.45:0/4257281035 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124001680 con 0x7f61340670b0 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 -- 172.21.15.45:0/4257281035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 msgr2=0x7f6134165ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 --2- 172.21.15.45:0/4257281035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134165ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f61340b3460 tx=0x7f611c01c280 comp rx=0 tx=0).stop 2024-04-25T10:21:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 -- 172.21.15.45:0/1055816286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b8161120 msgr2=0x7fa0b8071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 --2- 172.21.15.45:0/1055816286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b8161120 0x7fa0b8071fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009870 tx=0x7fa0a801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 -- 172.21.15.45:0/4257281035 shutdown_connections 2024-04-25T10:21:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 --2- 172.21.15.45:0/4257281035 >> v2:172.21.15.45:3300/0 conn(0x7f61340c3f30 0x7f61341663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 --2- 172.21.15.45:0/4257281035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134165ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 -- 172.21.15.45:0/4257281035 >> 172.21.15.45:0/4257281035 conn(0x7f61340b2380 msgr2=0x7f61340b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa99ea00640 1 --2- 172.21.15.45:0/2208204517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0161120 0x7fa9a0071fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa990009870 tx=0x7fa99001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881b19d4360f4a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 -- 172.21.15.45:0/1055816286 shutdown_connections 2024-04-25T10:21:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 --2- 172.21.15.45:0/1055816286 >> v2:172.21.15.45:3300/0 conn(0x7fa0b806e5c0 0x7fa0b80cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 Processor -- start 2024-04-25T10:21:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 --2- 172.21.15.45:0/1055816286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b8161120 0x7fa0b8071fe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 -- 172.21.15.45:0/1055816286 >> 172.21.15.45:0/1055816286 conn(0x7fa0b80b2380 msgr2=0x7fa0b80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 -- 172.21.15.45:0/1055816286 shutdown_connections 2024-04-25T10:21:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 -- start start 2024-04-25T10:21:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 -- 172.21.15.45:0/1055816286 wait complete. 2024-04-25T10:21:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa99d600640 1 -- 172.21.15.45:0/2208204517 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900ab600 con 0x7fa9a0161120 2024-04-25T10:21:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa99d600640 1 -- 172.21.15.45:0/2208204517 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9900b0070 con 0x7fa9a0161120 2024-04-25T10:21:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa99d600640 1 -- 172.21.15.45:0/2208204517 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900ab9c0 con 0x7fa9a0161120 2024-04-25T10:21:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 -- 172.21.15.45:0/4257281035 shutdown_connections 2024-04-25T10:21:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 -- 172.21.15.45:0/4257281035 wait complete. 2024-04-25T10:21:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2208204517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0161120 msgr2=0x7fa9a0071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2208204517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0161120 0x7fa9a0071fe0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa990009870 tx=0x7fa99001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2208204517 shutdown_connections 2024-04-25T10:21:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2208204517 >> v2:172.21.15.45:3300/0 conn(0x7fa9a006e5c0 0x7fa9a00cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2208204517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a0161120 0x7fa9a0071fe0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2208204517 >> 172.21.15.45:0/2208204517 conn(0x7fa9a00b2380 msgr2=0x7fa9a00b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 0x7fc4c40cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c409b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4c415bdc0 0x7fc4c409bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40c6630 con 0x7fc4c40d1c30 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40c64b0 con 0x7fc4c415bdc0 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4cc600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c40c67b0 con 0x7fc4c415b9e0 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2208204517 shutdown_connections 2024-04-25T10:21:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2208204517 wait complete. 2024-04-25T10:21:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 0x7fc4c40cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 0x7fc4c40cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46742/0 (socket says 172.21.15.45:46742) 2024-04-25T10:21:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 -- 172.21.15.45:0/889476689 learned_addr learned my addr 172.21.15.45:0/889476689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9200640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c409b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4ca600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4c415bdc0 0x7fc4c409bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 Processor -- start 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 -- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 msgr2=0x7fc4c409b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c409b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 -- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4c415bdc0 msgr2=0x7fc4c409bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4c415bdc0 0x7fc4c409bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4c9c00640 1 -- 172.21.15.45:0/889476689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c409d5e0 con 0x7fc4c40d1c30 2024-04-25T10:21:51.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fa0be600640 1 -- start start 2024-04-25T10:21:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7fc4ca600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4c415bdc0 0x7fc4c409bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 Processor -- start 2024-04-25T10:21:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.282+0000 7f6138e00640 1 -- start start 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4c9c00640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 0x7fc4c40cb170 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80b0040 tx=0x7fc4b8019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80b2040 con 0x7fc4c40d1c30 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b80adae0 con 0x7fc4c40d1c30 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80b6440 con 0x7fc4c40d1c30 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c409d8c0 con 0x7fc4c40d1c30 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c409de80 con 0x7fc4c40d1c30 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b80669a0 0x7fa0b8155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b806e5c0 0x7fa0b80c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 0x7fa0b80b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 Processor -- start 2024-04-25T10:21:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80ce120 con 0x7fa0b806e5c0 2024-04-25T10:21:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80cdfa0 con 0x7fa0b80669a0 2024-04-25T10:21:51.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8161500 con 0x7fa0b8161120 2024-04-25T10:21:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 -- start start 2024-04-25T10:21:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 0x7fa0b80b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 0x7fa0b80b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:60544/0 (socket says 172.21.15.45:60544) 2024-04-25T10:21:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61340c3f30 0x7f61340c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61340d0120 0x7f6134157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 -- 172.21.15.45:0/3954510151 learned_addr learned my addr 172.21.15.45:0/3954510151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6134071e30 con 0x7f61340c3f30 2024-04-25T10:21:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6134071cb0 con 0x7f61340670b0 2024-04-25T10:21:51.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61340d0500 con 0x7f61340d0120 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7400640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b80669a0 0x7fa0b8155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b6a00640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b806e5c0 0x7fa0b80c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 -- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b80669a0 msgr2=0x7fa0b8155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b80669a0 0x7fa0b8155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 -- 172.21.15.45:0/3954510151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b806e5c0 msgr2=0x7fa0b80c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b806e5c0 0x7fa0b80c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 -- 172.21.15.45:0/3954510151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80c50c0 con 0x7fa0b8161120 2024-04-25T10:21:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f612be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61340c3f30 0x7f61340c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f612be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61340c3f30 0x7f61340c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46766/0 (socket says 172.21.15.45:46766) 2024-04-25T10:21:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f612be00640 1 -- 172.21.15.45:0/1748741002 learned_addr learned my addr 172.21.15.45:0/1748741002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6132800640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61340d0120 0x7f6134157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a00052a0 con 0x7fc4c40d1c30 2024-04-25T10:21:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0b7e00640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 0x7fa0b80b3780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7b0 tx=0x7fa0b000cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 0x7fa9a00c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9a006e5c0 0x7fa9a00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a0161120 0x7fa9a014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00ce120 con 0x7fa9a0161120 2024-04-25T10:21:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0004b70 con 0x7fa0b8161120 2024-04-25T10:21:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0004d10 con 0x7fa0b8161120 2024-04-25T10:21:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000aa90 con 0x7fa0b8161120 2024-04-25T10:21:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00cdfa0 con 0x7fa9a00669a0 2024-04-25T10:21:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa9a5a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a0161500 con 0x7fa9a006e5c0 2024-04-25T10:21:51.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b80c53a0 con 0x7fa0b8161120 2024-04-25T10:21:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b80c5960 con 0x7fa0b8161120 2024-04-25T10:21:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 0x7fa9a00c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 0x7fa9a00c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56096/0 (socket says 172.21.15.45:56096) 2024-04-25T10:21:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 -- 172.21.15.45:0/2113681195 learned_addr learned my addr 172.21.15.45:0/2113681195 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99e000640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9a006e5c0 0x7fa9a00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99f400640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a0161120 0x7fa9a014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 -- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9a006e5c0 msgr2=0x7fa9a00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9a006e5c0 0x7fa9a00c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 -- 172.21.15.45:0/2113681195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a0161120 msgr2=0x7fa9a014cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a0161120 0x7fa9a014cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7fa99ea00640 1 -- 172.21.15.45:0/2113681195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00c6fe0 con 0x7fa9a00669a0 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 -- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61340d0120 msgr2=0x7f6134157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4b80bd020 con 0x7fc4c40d1c30 2024-04-25T10:21:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61340d0120 0x7f6134157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 -- 172.21.15.45:0/1748741002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61340c3f30 msgr2=0x7f61340c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61340c3f30 0x7f61340c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 -- 172.21.15.45:0/1748741002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134156e50 con 0x7f61340670b0 2024-04-25T10:21:51.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6131e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134156910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6124001390 tx=0x7f612400ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124017070 con 0x7f61340670b0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124005cf0 con 0x7f61340670b0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124007290 con 0x7f61340670b0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61340b06b0 con 0x7f61340670b0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.286+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61340b0c70 con 0x7f61340670b0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa99ea00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 0x7fa9a00c5ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa9900ad040 tx=0x7fa99001ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804ef30 con 0x7fa0b8161120 2024-04-25T10:21:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613404ef30 con 0x7f61340670b0 2024-04-25T10:21:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900be5f0 con 0x7fa9a00669a0 2024-04-25T10:21:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9900be790 con 0x7fa9a00669a0 2024-04-25T10:21:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a0150c10 con 0x7fa9a00669a0 2024-04-25T10:21:51.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0b000ac30 con 0x7fa0b8161120 2024-04-25T10:21:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fc4c1600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4a4066f60 0x7fc4a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4b811e3c0 con 0x7fc4c40d1c30 2024-04-25T10:21:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a01511d0 con 0x7fa9a00669a0 2024-04-25T10:21:51.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900beb80 con 0x7fa9a00669a0 2024-04-25T10:21:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fc4c9200640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4a4066f60 0x7fc4a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f612401a070 con 0x7f61340670b0 2024-04-25T10:21:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9780052a0 con 0x7fa9a00669a0 2024-04-25T10:21:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa0af400640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa098066f60 0x7fa098069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0b0085580 con 0x7fa0b8161120 2024-04-25T10:21:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fa0b7400640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa098066f60 0x7fa098069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7fc4c9200640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4a4066f60 0x7fc4a4069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4c406fdb0 tx=0x7fc4b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.290+0000 7f612a000640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6110067030 0x7f61100694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61240844f0 con 0x7f61340670b0 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7f612be00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6110067030 0x7f61100694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fa0b7400640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa098066f60 0x7fa098069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80ae000 tx=0x7fa0a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9900aa070 con 0x7fa9a00669a0 2024-04-25T10:21:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7f612be00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6110067030 0x7f61100694f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f611c000c10 tx=0x7f611c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fa996a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa980066fe0 0x7fa9800694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa99011f480 con 0x7fa9a00669a0 2024-04-25T10:21:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fa99e000640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa980066fe0 0x7fa9800694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fa99e000640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa980066fe0 0x7fa9800694a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b13b0 tx=0x7fa98c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.294+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4b80c4020 con 0x7fc4c40d1c30 2024-04-25T10:21:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.302+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0b00564f0 con 0x7fa0b8161120 2024-04-25T10:21:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.302+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6124055570 con 0x7f61340670b0 2024-04-25T10:21:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.306+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9900c4060 con 0x7fa9a00669a0 2024-04-25T10:21:51.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.514+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7fc4a0005590 con 0x7fc4c40d1c30 2024-04-25T10:21:51.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.526+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fa0b80b67f0 con 0x7fa0b8161120 2024-04-25T10:21:51.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.538+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f6134068cb0 con 0x7f61340670b0 2024-04-25T10:21:51.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:51.542+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fa978005590 con 0x7fa9a00669a0 2024-04-25T10:21:52.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.242+0000 7fc4c1600640 1 -- 172.21.15.45:0/889476689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 5696 v40) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc4b8016150 con 0x7fc4c40d1c30 2024-04-25T10:21:52.247 INFO:teuthology.orchestra.run.smithi045.stderr:failed mds gid 5696 2024-04-25T10:21:52.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4a4066f60 msgr2=0x7fc4a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4a4066f60 0x7fc4a4069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4c406fdb0 tx=0x7fc4b0009040 comp rx=0 tx=0).stop 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 msgr2=0x7fc4c40cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 0x7fc4c40cb170 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80b0040 tx=0x7fc4b8019300 comp rx=0 tx=0).stop 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 shutdown_connections 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4a4066f60 0x7fc4a4069420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4c415bdc0 0x7fc4c409bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4c415b9e0 0x7fc4c409b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 --2- 172.21.15.45:0/889476689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c40d1c30 0x7fc4c40cb170 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 >> 172.21.15.45:0/889476689 conn(0x7fc4c409a9d0 msgr2=0x7fc4c40c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 shutdown_connections 2024-04-25T10:21:52.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.246+0000 7fc4cc600640 1 -- 172.21.15.45:0/889476689 wait complete. 2024-04-25T10:21:52.254 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:21:52.250+0000 7ff8b4c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:21:52.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.258+0000 7f612a000640 1 -- 172.21.15.45:0/1748741002 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6393 v41) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6124054e70 con 0x7f61340670b0 2024-04-25T10:21:52.264 INFO:teuthology.orchestra.run.smithi045.stderr:failed mds gid 6393 2024-04-25T10:21:52.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.258+0000 7fa0af400640 1 -- 172.21.15.45:0/3954510151 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5699 v41) v1 ==== 95+0+0 (secure 0 0 0) 0x7fa0b0055df0 con 0x7fa0b8161120 2024-04-25T10:21:52.264 INFO:teuthology.orchestra.run.smithi045.stderr:failed mds gid 5699 2024-04-25T10:21:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa098066f60 msgr2=0x7fa098069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa098066f60 0x7fa098069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80ae000 tx=0x7fa0a8008040 comp rx=0 tx=0).stop 2024-04-25T10:21:52.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 msgr2=0x7fa0b80b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 0x7fa0b80b3780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7b0 tx=0x7fa0b000cc70 comp rx=0 tx=0).stop 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6110067030 msgr2=0x7f61100694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6110067030 0x7f61100694f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f611c000c10 tx=0x7f611c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 msgr2=0x7f6134156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134156910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6124001390 tx=0x7f612400ca30 comp rx=0 tx=0).stop 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 shutdown_connections 2024-04-25T10:21:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa098066f60 0x7fa098069420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b8161120 0x7fa0b80b3780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b806e5c0 0x7fa0b80c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 shutdown_connections 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6110067030 0x7f61100694f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61340d0120 0x7f6134157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61340c3f30 0x7f61340c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 --2- 172.21.15.45:0/1748741002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61340670b0 0x7f6134156910 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 >> 172.21.15.45:0/1748741002 conn(0x7f61340b2380 msgr2=0x7f613415efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 --2- 172.21.15.45:0/3954510151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b80669a0 0x7fa0b8155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 >> 172.21.15.45:0/3954510151 conn(0x7fa0b80b2380 msgr2=0x7fa0b80aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 shutdown_connections 2024-04-25T10:21:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 shutdown_connections 2024-04-25T10:21:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7fa0be600640 1 -- 172.21.15.45:0/3954510151 wait complete. 2024-04-25T10:21:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:52.262+0000 7f6138e00640 1 -- 172.21.15.45:0/1748741002 wait complete. 2024-04-25T10:21:52.281 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:21:52.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:21:52.284 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:21:52.284 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:52.301 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:21:52.301 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:21:52.302 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:21:52.302 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:21:52.305 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:21:52.305 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:52.306 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:21:52.307 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:52.375 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:21:52.392 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:21:52.393 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:21:53.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.258+0000 7fa996a00640 1 -- 172.21.15.45:0/2113681195 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6384 v42) v1 ==== 95+0+0 (secure 0 0 0) 0x7fa990016110 con 0x7fa9a00669a0 2024-04-25T10:21:53.263 INFO:teuthology.orchestra.run.smithi045.stderr:failed mds gid 6384 2024-04-25T10:21:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa980066fe0 msgr2=0x7fa9800694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa980066fe0 0x7fa9800694a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b13b0 tx=0x7fa98c009190 comp rx=0 tx=0).stop 2024-04-25T10:21:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 msgr2=0x7fa9a00c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 0x7fa9a00c5ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa9900ad040 tx=0x7fa99001ad30 comp rx=0 tx=0).stop 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 shutdown_connections 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa980066fe0 0x7fa9800694a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9a0161120 0x7fa9a014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9a006e5c0 0x7fa9a00c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 --2- 172.21.15.45:0/2113681195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9a00669a0 0x7fa9a00c5ae0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 >> 172.21.15.45:0/2113681195 conn(0x7fa9a00b2380 msgr2=0x7fa9a00ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 shutdown_connections 2024-04-25T10:21:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.262+0000 7fa9a5a00640 1 -- 172.21.15.45:0/2113681195 wait complete. 2024-04-25T10:21:53.291 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:21:53.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:21:53.338 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:21:53.339 DEBUG:teuthology.parallel:result is None 2024-04-25T10:21:53.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:53.430 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:21:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 Processor -- start 2024-04-25T10:21:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 -- start start 2024-04-25T10:21:53.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f64a015bc40 0x7f64a006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cda10 0x7f64a00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64a00cf930 0x7f64a00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00b2770 con 0x7f64a00cf930 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a006c670 con 0x7f64a015bc40 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a6e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a006c950 con 0x7f64a00cda10 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64a00cf930 0x7f64a00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f64a00cf930 0x7f64a00cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46814/0 (socket says 172.21.15.45:46814) 2024-04-25T10:21:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.486+0000 7f64a4e00640 1 -- 172.21.15.45:0/4246636088 learned_addr learned my addr 172.21.15.45:0/4246636088 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f6497400640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cda10 0x7f64a00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f649fe00640 1 --2- 172.21.15.45:0/4246636088 >> v2:172.21.15.117:3300/0 conn(0x7f64a015bc40 0x7f64a006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a4e00640 1 -- 172.21.15.45:0/4246636088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cda10 msgr2=0x7f64a00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a4e00640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cda10 0x7f64a00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a4e00640 1 -- 172.21.15.45:0/4246636088 >> v2:172.21.15.117:3300/0 conn(0x7f64a015bc40 msgr2=0x7f64a006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a4e00640 1 --2- 172.21.15.45:0/4246636088 >> v2:172.21.15.117:3300/0 conn(0x7f64a015bc40 0x7f64a006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a4e00640 1 -- 172.21.15.45:0/4246636088 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a006cc30 con 0x7f64a00cf930 2024-04-25T10:21:53.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f6497400640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cda10 0x7f64a00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:53.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a4e00640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cf930 0x7f64a00cfde0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f64a00d0800 tx=0x7f649801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119aa9cf3b9a9831 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:53.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f649f400640 1 -- 172.21.15.45:0/4246636088 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980a5670 con 0x7f64a00cf930 2024-04-25T10:21:53.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f649f400640 1 -- 172.21.15.45:0/4246636088 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64980a5810 con 0x7f64a00cf930 2024-04-25T10:21:53.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f649f400640 1 -- 172.21.15.45:0/4246636088 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980a5be0 con 0x7f64a00cf930 2024-04-25T10:21:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a6e00640 1 -- 172.21.15.45:0/4246636088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cf930 msgr2=0x7f64a00cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.490+0000 7f64a6e00640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cf930 0x7f64a00cfde0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f64a00d0800 tx=0x7f649801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- 172.21.15.45:0/4246636088 shutdown_connections 2024-04-25T10:21:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cf930 0x7f64a00cfde0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 --2- 172.21.15.45:0/4246636088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cda10 0x7f64a00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 --2- 172.21.15.45:0/4246636088 >> v2:172.21.15.117:3300/0 conn(0x7f64a015bc40 0x7f64a006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- 172.21.15.45:0/4246636088 >> 172.21.15.45:0/4246636088 conn(0x7f64a0066b70 msgr2=0x7f64a0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- 172.21.15.45:0/4246636088 shutdown_connections 2024-04-25T10:21:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- 172.21.15.45:0/4246636088 wait complete. 2024-04-25T10:21:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 Processor -- start 2024-04-25T10:21:53.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- start start 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 0x7f64a015ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cf930 0x7f64a00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a015bc40 0x7f64a00add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a0160230 con 0x7f64a00cda10 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01600b0 con 0x7f64a015bc40 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01603b0 con 0x7f64a00cf930 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 0x7f64a015ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 0x7f64a015ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46826/0 (socket says 172.21.15.45:46826) 2024-04-25T10:21:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 -- 172.21.15.45:0/1016732370 learned_addr learned my addr 172.21.15.45:0/1016732370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f6497400640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cf930 0x7f64a00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a4e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a015bc40 0x7f64a00add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 -- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cf930 msgr2=0x7f64a00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cf930 0x7f64a00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 -- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a015bc40 msgr2=0x7f64a00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a015bc40 0x7f64a00add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 -- 172.21.15.45:0/1016732370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a015f100 con 0x7f64a00cda10 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649fe00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 0x7f64a015ebc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f649000d800 tx=0x7f649000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000b6c0 con 0x7f64a00cda10 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6490019070 con 0x7f64a00cda10 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000ba60 con 0x7f64a00cda10 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.494+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a00721b0 con 0x7f64a00cda10 2024-04-25T10:21:53.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a015d710 con 0x7f64a00cda10 2024-04-25T10:21:53.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a004ef30 con 0x7f64a00cda10 2024-04-25T10:21:53.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f649000bc00 con 0x7f64a00cda10 2024-04-25T10:21:53.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f649cc00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f647c066f60 0x7f647c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:53.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6490085410 con 0x7f64a00cda10 2024-04-25T10:21:53.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f6497400640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f647c066f60 0x7f647c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:53.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.498+0000 7f6497400640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f647c066f60 0x7f647c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f64a014efc0 tx=0x7f6488008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.502+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64a015f100 con 0x7f64a00cda10 2024-04-25T10:21:53.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.702+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64a00cf4e0 con 0x7f64a00cda10 2024-04-25T10:21:53.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.702+0000 7f649cc00640 1 -- 172.21.15.45:0/1016732370 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4154 (secure 0 0 0) 0x7f6490055d60 con 0x7f64a00cda10 2024-04-25T10:21:53.706 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:53.706 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":43,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:53.262402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:21:53.707 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 43 2024-04-25T10:21:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.702+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f647c066f60 msgr2=0x7f647c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.702+0000 7f64a6e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f647c066f60 0x7f647c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f64a014efc0 tx=0x7f6488008040 comp rx=0 tx=0).stop 2024-04-25T10:21:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.702+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 msgr2=0x7f64a015ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.702+0000 7f64a6e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 0x7f64a015ebc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f649000d800 tx=0x7f649000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 shutdown_connections 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f647c066f60 0x7f647c069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a015bc40 0x7f64a00add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64a00cf930 0x7f64a00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 --2- 172.21.15.45:0/1016732370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a00cda10 0x7f64a015ebc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 >> 172.21.15.45:0/1016732370 conn(0x7f64a0066b70 msgr2=0x7f64a00b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 shutdown_connections 2024-04-25T10:21:53.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:53.706+0000 7f64a6e00640 1 -- 172.21.15.45:0/1016732370 wait complete. 2024-04-25T10:21:54.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:21:40.567701+0000', 'modified': '2024-04-25T10:21:53.262402+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6349}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6349': {'gid': 6349, 'name': 'd', 'rank': 0, 'incarnation': 43, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/4277203907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 4277203907}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 4277203907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:21:54.131 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6349:up:replay 2024-04-25T10:21:55.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6141400640 1 Processor -- start 2024-04-25T10:21:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6141400640 1 -- start start 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6141400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f613c0cca10 0x7f613c165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6141400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c15e520 con 0x7f613c0cf1b0 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6141400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c0cf6b0 con 0x7f613c1437b0 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6141400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0cea50 con 0x7f613c0cca10 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f613b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f613c0cf1b0 0x7f613c0cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53230/0 (socket says 172.21.15.45:53230) 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f613b400640 1 -- 172.21.15.45:0/1171775841 learned_addr learned my addr 172.21.15.45:0/1171775841 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.282+0000 7f6140a00640 1 --2- 172.21.15.45:0/1171775841 >> v2:172.21.15.117:3300/0 conn(0x7f613c0cca10 0x7f613c165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839001599 0 0) 0x7f613c15e520 con 0x7f613c0cf1b0 2024-04-25T10:21:55.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6120005740 con 0x7f613c0cf1b0 2024-04-25T10:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330044541 0 0) 0x7f613c0cf6b0 con 0x7f613c1437b0 2024-04-25T10:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f613c15e520 con 0x7f613c1437b0 2024-04-25T10:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851460867 0 0) 0x7f6120005740 con 0x7f613c0cf1b0 2024-04-25T10:21:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 >> v2:172.21.15.117:3300/0 conn(0x7f613c0cca10 msgr2=0x7f613c165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 --2- 172.21.15.45:0/1171775841 >> v2:172.21.15.117:3300/0 conn(0x7f613c0cca10 0x7f613c165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 >> v1:172.21.15.117:6789/0 conn(0x7f613c1437b0 legacy=0x7f613c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c15e7c0 con 0x7f613c0cf1b0 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6128001680 con 0x7f613c0cf1b0 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 >> v1:172.21.15.45:6789/0 conn(0x7f613c0cf1b0 legacy=0x7f613c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120005db0 0x7f61200061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6120006a20 0x7f6120006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61200073e0 0x7f612000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0cea50 con 0x7f6120006a20 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612000e2b0 con 0x7f61200073e0 2024-04-25T10:21:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612000e4b0 con 0x7f6120005db0 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613b400640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6120006a20 0x7f6120006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613be00640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120005db0 0x7f61200061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f6140a00640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61200073e0 0x7f612000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613b400640 1 -- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120005db0 msgr2=0x7f61200061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613b400640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120005db0 0x7f61200061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613b400640 1 -- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61200073e0 msgr2=0x7f612000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613b400640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61200073e0 0x7f612000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f613b400640 1 -- 172.21.15.45:0/1171775841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120005740 con 0x7f6120006a20 2024-04-25T10:21:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.286+0000 7f6140a00640 1 -- 172.21.15.45:0/1171775841 reap_dead start 2024-04-25T10:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f613b400640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6120006a20 0x7f6120006ea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6128007fb0 tx=0x7f612801cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d6cbeb5894541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128007bd0 con 0x7f6120006a20 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61280a9020 con 0x7f6120006a20 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f613aa00640 1 -- 172.21.15.45:0/1171775841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128005b00 con 0x7f6120006a20 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- 172.21.15.45:0/1171775841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6120006a20 msgr2=0x7f6120006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6120006a20 0x7f6120006ea0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6128007fb0 tx=0x7f612801cb20 comp rx=0 tx=0).stop 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- 172.21.15.45:0/1171775841 shutdown_connections 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 --2- 172.21.15.45:0/1171775841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6120006a20 0x7f6120006ea0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- 172.21.15.45:0/1171775841 >> 172.21.15.45:0/1171775841 conn(0x7f613c0b16a0 msgr2=0x7f613c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- 172.21.15.45:0/1171775841 shutdown_connections 2024-04-25T10:21:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- 172.21.15.45:0/1171775841 wait complete. 2024-04-25T10:21:55.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 Processor -- start 2024-04-25T10:21:55.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- start start 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120006a20 0x7f613c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f613c1f4750 0x7f613c15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 0x7f613c0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c15f2a0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c15ed80 con 0x7f6120006a20 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.290+0000 7f6141400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0b2790 con 0x7f613c1f4750 2024-04-25T10:21:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 0x7f613c0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 0x7f613c0b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52010/0 (socket says 172.21.15.45:52010) 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 -- 172.21.15.45:0/2937651081 learned_addr learned my addr 172.21.15.45:0/2937651081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f613b400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f613c1f4750 0x7f613c15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f613be00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120006a20 0x7f613c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 -- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f613c1f4750 msgr2=0x7f613c15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f613c1f4750 0x7f613c15ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 -- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120006a20 msgr2=0x7f613c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120006a20 0x7f613c15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 -- 172.21.15.45:0/2937651081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0b29e0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f613be00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120006a20 0x7f613c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6140a00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 0x7f613c0b3a80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f613c0cbc90 tx=0x7f61340115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:55.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134011db0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613401b070 con 0x7f613c1f4bd0 2024-04-25T10:21:55.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340039a0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c0b2cc0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c0b3280 con 0x7f613c1f4bd0 2024-04-25T10:21:55.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c075120 con 0x7f613c1f4bd0 2024-04-25T10:21:55.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.294+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6134003b40 con 0x7f613c1f4bd0 2024-04-25T10:21:55.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.298+0000 7f6133e00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f611c066f60 0x7f611c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:55.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.298+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f61340858e0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.298+0000 7f613be00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f611c066f60 0x7f611c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:55.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.298+0000 7f613be00640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f611c066f60 0x7f611c069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f612c005630 tx=0x7f612c00f3d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:55.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.302+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61340567f0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.494+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f613c0cd820 con 0x7f613c1f4bd0 2024-04-25T10:21:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.494+0000 7f6133e00640 1 -- 172.21.15.45:0/2937651081 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f61340560f0 con 0x7f613c1f4bd0 2024-04-25T10:21:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:55.499 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:54.277681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:21:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 45 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f611c066f60 msgr2=0x7f611c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f611c066f60 0x7f611c069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f612c005630 tx=0x7f612c00f3d0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 msgr2=0x7f613c0b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 0x7f613c0b3a80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f613c0cbc90 tx=0x7f61340115a0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 shutdown_connections 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f611c066f60 0x7f611c069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f613c1f4bd0 0x7f613c0b3a80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f613c1f4750 0x7f613c15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 --2- 172.21.15.45:0/2937651081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120006a20 0x7f613c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 >> 172.21.15.45:0/2937651081 conn(0x7f613c0b16a0 msgr2=0x7f613c0b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 shutdown_connections 2024-04-25T10:21:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:55.498+0000 7f6141400640 1 -- 172.21.15.45:0/2937651081 wait complete. 2024-04-25T10:21:55.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:21:40.567701+0000', 'modified': '2024-04-25T10:21:54.277681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6349}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6349': {'gid': 6349, 'name': 'd', 'rank': 0, 'incarnation': 43, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.117:6837/4277203907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 4277203907}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 4277203907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:21:55.924 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6349:up:rejoin 2024-04-25T10:21:56.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:21:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 Processor -- start 2024-04-25T10:21:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 -- start start 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40c15a780 0x7fd40c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c0cff00 0x7fd40c0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40c06de60 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40c15f2f0 con 0x7fd40c15a780 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c06cdc0 con 0x7fd40c0cff00 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40c15a780 0x7fd40c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40c15a780 0x7fd40c15edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52014/0 (socket says 172.21.15.45:52014) 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 -- 172.21.15.45:0/3271168209 learned_addr learned my addr 172.21.15.45:0/3271168209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40b400640 1 --2- 172.21.15.45:0/3271168209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c0cff00 0x7fd40c0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 -- 172.21.15.45:0/3271168209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c0cff00 msgr2=0x7fd40c0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 --2- 172.21.15.45:0/3271168209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c0cff00 0x7fd40c0b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 -- 172.21.15.45:0/3271168209 >> v1:172.21.15.117:6789/0 conn(0x7fd40c0cf3b0 legacy=0x7fd40c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.074+0000 7fd40aa00640 1 -- 172.21.15.45:0/3271168209 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c06d0a0 con 0x7fd40c15a780 2024-04-25T10:21:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40aa00640 1 --2- 172.21.15.45:0/3271168209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15a780 0x7fd40c15edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009870 tx=0x7fd3fc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3edd63f576584ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd409600640 1 -- 172.21.15.45:0/3271168209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0b2070 con 0x7fd40c15a780 2024-04-25T10:21:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd409600640 1 -- 172.21.15.45:0/3271168209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0a9070 con 0x7fd40c15a780 2024-04-25T10:21:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd409600640 1 -- 172.21.15.45:0/3271168209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0ad880 con 0x7fd40c15a780 2024-04-25T10:21:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 -- 172.21.15.45:0/3271168209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15a780 msgr2=0x7fd40c15edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 --2- 172.21.15.45:0/3271168209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15a780 0x7fd40c15edb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009870 tx=0x7fd3fc01c560 comp rx=0 tx=0).stop 2024-04-25T10:21:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 -- 172.21.15.45:0/3271168209 shutdown_connections 2024-04-25T10:21:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 --2- 172.21.15.45:0/3271168209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c0cff00 0x7fd40c0b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 --2- 172.21.15.45:0/3271168209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c15a780 0x7fd40c15edb0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 -- 172.21.15.45:0/3271168209 >> 172.21.15.45:0/3271168209 conn(0x7fd40c0743f0 msgr2=0x7fd40c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 -- 172.21.15.45:0/3271168209 shutdown_connections 2024-04-25T10:21:57.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 -- 172.21.15.45:0/3271168209 wait complete. 2024-04-25T10:21:57.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.078+0000 7fd40be00640 1 Processor -- start 2024-04-25T10:21:57.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- start start 2024-04-25T10:21:57.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 0x7fd40c13bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0cff00 0x7fd40c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c15a780 0x7fd40c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b3330 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b31b0 con 0x7fd40c0cff00 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c0b34b0 con 0x7fd40c15a780 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 0x7fd40c13bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 0x7fd40c13bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52026/0 (socket says 172.21.15.45:52026) 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 -- 172.21.15.45:0/3158416340 learned_addr learned my addr 172.21.15.45:0/3158416340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40a000640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0cff00 0x7fd40c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40b400640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c15a780 0x7fd40c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 -- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c15a780 msgr2=0x7fd40c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c15a780 0x7fd40c150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 -- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0cff00 msgr2=0x7fd40c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0cff00 0x7fd40c1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 -- 172.21.15.45:0/3158416340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c13c450 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40b400640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c15a780 0x7fd40c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40aa00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 0x7fd40c13bf10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0b0040 tx=0x7fd3fc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0b2040 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0bb070 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c14fd70 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0b6440 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c14f290 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.082+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40c075120 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.086+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3fc0ab070 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.086+0000 7fd402a00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3ec066f90 0x7fd3ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.086+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd3fc11e970 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.086+0000 7fd40a000640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3ec066f90 0x7fd3ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.086+0000 7fd40a000640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3ec066f90 0x7fd3ec069450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd40c1510a0 tx=0x7fd3f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.090+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3fc016150 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.290+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd40c070d90 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.290+0000 7fd402a00640 1 -- 172.21.15.45:0/3158416340 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fd3fc0ef910 con 0x7fd40c0cf3b0 2024-04-25T10:21:57.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:21:57.296 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:55.281089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:21:57.296 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2024-04-25T10:21:57.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3ec066f90 msgr2=0x7fd3ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3ec066f90 0x7fd3ec069450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd40c1510a0 tx=0x7fd3f8009040 comp rx=0 tx=0).stop 2024-04-25T10:21:57.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 msgr2=0x7fd40c13bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 0x7fd40c13bf10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0b0040 tx=0x7fd3fc019300 comp rx=0 tx=0).stop 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 shutdown_connections 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3ec066f90 0x7fd3ec069450 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd40c15a780 0x7fd40c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd40c0cff00 0x7fd40c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 --2- 172.21.15.45:0/3158416340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40c0cf3b0 0x7fd40c13bf10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 >> 172.21.15.45:0/3158416340 conn(0x7fd40c0743f0 msgr2=0x7fd40c071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 shutdown_connections 2024-04-25T10:21:57.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.294+0000 7fd40be00640 1 -- 172.21.15.45:0/3158416340 wait complete. 2024-04-25T10:21:57.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:21:40.567701+0000', 'modified': '2024-04-25T10:21:55.281089+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6349}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6349': {'gid': 6349, 'name': 'd', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6837/4277203907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 4277203907}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 4277203907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:21:57.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:21:57.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:21:57.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:57.800 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:57.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f16200640 1 Processor -- start 2024-04-25T10:21:57.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f16200640 1 -- start start 2024-04-25T10:21:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f16200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f1015b910 0x7f8f1015c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f16200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1015bd60 con 0x7f8f100d0040 2024-04-25T10:21:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f16200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f10071050 con 0x7f8f101550b0 2024-04-25T10:21:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f16200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f100712f0 con 0x7f8f1015b910 2024-04-25T10:21:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f0fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f1015b910 0x7f8f1015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f0fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f1015b910 0x7f8f1015c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52040/0 (socket says 172.21.15.45:52040) 2024-04-25T10:21:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f0fe00640 1 -- 172.21.15.45:0/1128228214 learned_addr learned my addr 172.21.15.45:0/1128228214 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f0fe00640 1 -- 172.21.15.45:0/1128228214 >> v1:172.21.15.117:6789/0 conn(0x7f8f101550b0 legacy=0x7f8f100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f0fe00640 1 -- 172.21.15.45:0/1128228214 >> v1:172.21.15.45:6789/0 conn(0x7f8f100d0040 legacy=0x7f8f1015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.866+0000 7f8f0fe00640 1 -- 172.21.15.45:0/1128228214 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100715d0 con 0x7f8f1015b910 2024-04-25T10:21:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/1128228214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1015b910 0x7f8f1015c030 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009e50 tx=0x7f8f0801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e09a49f3e07a57b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f0e000640 1 -- 172.21.15.45:0/1128228214 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f080a5880 con 0x7f8f1015b910 2024-04-25T10:21:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f0e000640 1 -- 172.21.15.45:0/1128228214 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f080a5a20 con 0x7f8f1015b910 2024-04-25T10:21:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f0e000640 1 -- 172.21.15.45:0/1128228214 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f080a5df0 con 0x7f8f1015b910 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 -- 172.21.15.45:0/1128228214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1015b910 msgr2=0x7f8f1015c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 --2- 172.21.15.45:0/1128228214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1015b910 0x7f8f1015c030 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009e50 tx=0x7f8f0801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 -- 172.21.15.45:0/1128228214 shutdown_connections 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 --2- 172.21.15.45:0/1128228214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f1015b910 0x7f8f1015c030 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 -- 172.21.15.45:0/1128228214 >> 172.21.15.45:0/1128228214 conn(0x7f8f10066b70 msgr2=0x7f8f10066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 -- 172.21.15.45:0/1128228214 shutdown_connections 2024-04-25T10:21:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 -- 172.21.15.45:0/1128228214 wait complete. 2024-04-25T10:21:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.870+0000 7f8f16200640 1 Processor -- start 2024-04-25T10:21:57.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 -- start start 2024-04-25T10:21:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 0x7f8f10147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f101550b0 0x7f8f1014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f1015b910 0x7f8f1015d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1015cf00 con 0x7f8f100d0040 2024-04-25T10:21:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1015cd80 con 0x7f8f101550b0 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1015d080 con 0x7f8f1015b910 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 0x7f8f10147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 0x7f8f10147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52052/0 (socket says 172.21.15.45:52052) 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 -- 172.21.15.45:0/1225187505 learned_addr learned my addr 172.21.15.45:0/1225187505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f101550b0 0x7f8f1014fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f1015b910 0x7f8f1015d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 -- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f1015b910 msgr2=0x7f8f1015d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f1015b910 0x7f8f1015d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 -- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f101550b0 msgr2=0x7f8f1014fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f101550b0 0x7f8f1014fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 -- 172.21.15.45:0/1225187505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1014be50 con 0x7f8f100d0040 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f101550b0 0x7f8f1014fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f0f400640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 0x7f8f10147ac0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c450 tx=0x7f8f0000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f07400640 1 -- 172.21.15.45:0/1225187505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00012830 con 0x7f8f100d0040 2024-04-25T10:21:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f07400640 1 -- 172.21.15.45:0/1225187505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00017070 con 0x7f8f100d0040 2024-04-25T10:21:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f07400640 1 -- 172.21.15.45:0/1225187505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00012c30 con 0x7f8f100d0040 2024-04-25T10:21:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1014f010 con 0x7f8f100d0040 2024-04-25T10:21:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.874+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1014f5d0 con 0x7f8f100d0040 2024-04-25T10:21:57.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f07400640 1 -- 172.21.15.45:0/1225187505 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f0001c460 con 0x7f8f100d0040 2024-04-25T10:21:57.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f07400640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f07400640 1 -- 172.21.15.45:0/1225187505 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8f00084db0 con 0x7f8f100d0040 2024-04-25T10:21:57.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 Processor -- start 2024-04-25T10:21:57.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 -- start start 2024-04-25T10:21:57.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8f1015e330 tx=0x7f8efc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd5920 0x563238fd5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238f8b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238f8c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563238e735f0 con 0x563238fd6240 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563238ddada0 con 0x563238fd5920 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563238d63150 con 0x563238f8bf30 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238f8b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238f8b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52066/0 (socket says 172.21.15.45:52066) 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3500464297 learned_addr learned my addr 172.21.15.45:0/3500464297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238f8c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0f400640 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd5920 0x563238fd5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 msgr2=0x563238f8c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238f8c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd5920 msgr2=0x563238fd5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd5920 0x563238fd5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3500464297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563238d7cca0 con 0x563238fd6240 2024-04-25T10:21:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.878+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238f8b9f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005280 tx=0x7f8efc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2761448f2833aa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8efbe00640 1 -- 172.21.15.45:0/3500464297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00e500 con 0x563238fd6240 2024-04-25T10:21:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8efbe00640 1 -- 172.21.15.45:0/3500464297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00e6a0 con 0x563238fd6240 2024-04-25T10:21:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8efbe00640 1 -- 172.21.15.45:0/3500464297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00ea50 con 0x563238fd6240 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/3500464297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 msgr2=0x563238f8b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238f8b9f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005280 tx=0x7f8efc011000 comp rx=0 tx=0).stop 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/3500464297 shutdown_connections 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238f8c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238f8b9f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3500464297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd5920 0x563238fd5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/3500464297 >> 172.21.15.45:0/3500464297 conn(0x563238f922a0 msgr2=0x563238f8a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/3500464297 shutdown_connections 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/3500464297 wait complete. 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- start start 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 Processor -- start 2024-04-25T10:21:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 0x563238fdcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238fd5920 0x563238fdd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563238f8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563238d95990 con 0x563238f8bf30 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563238dd6e40 con 0x563238fd6240 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563238d66900 con 0x563238fd5920 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 10 client.0 _ll_register_callbacks cb 0x563238e8e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 0x563238fdcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 0x563238fdcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52068/0 (socket says 172.21.15.45:52068) 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 -- 172.21.15.45:0/1407110620 learned_addr learned my addr 172.21.15.45:0/1407110620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563238f8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238fd5920 0x563238fdd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 -- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238fd5920 msgr2=0x563238fdd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238fd5920 0x563238fdd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 -- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 msgr2=0x563238f8faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563238f8faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 -- 172.21.15.45:0/1407110620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563238d93630 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563238f8faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f0f400640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 0x563238fdcd60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8f00055350 tx=0x7f8f000598e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7d33e30bc18df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8ef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8ef8c00640 1 -- 172.21.15.45:0/1407110620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00081060 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8ef8c00640 1 -- 172.21.15.45:0/1407110620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8f0007cdd0 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8efc00e6a0 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8ef8c00640 1 -- 172.21.15.45:0/1407110620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f0006d070 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 10 client.6654 fetch_fsmap learned FSMap version 46 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.882+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x563238d633a0 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8ef8c00640 1 -- 172.21.15.45:0/1407110620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00e6a0 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8ef8c00640 1 -- 172.21.15.45:0/1407110620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f00054aa0 con 0x563238f8bf30 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 10 client.6654 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 10 client.6654 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 20 client.6654 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563238f91160 0x563238fe6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 4 client.6654 mds_command: new command op to 6349 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 --> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563238d76a50 con 0x563238f91160 2024-04-25T10:21:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563238f91160 0x563238fe6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563238f91160 0x563238fe6dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3553b2f1423d1c server_cookie=fd249a22e7780b5 in_seq=0 out_seq=0 2024-04-25T10:21:57.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8ef8c00640 10 client.6654 ms_handle_connect on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8ef8c00640 1 -- 172.21.15.45:0/1407110620 <== mds.0 v2:172.21.15.117:6835/4277203907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563238d76a50 con 0x563238f91160 2024-04-25T10:21:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8ef8c00640 10 client.6654 handle_command_reply: tid=0 2024-04-25T10:21:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 client.6654 shutdown 2024-04-25T10:21:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 msgr2=0x563238fdcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 0x563238fdcd60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8f00055350 tx=0x7f8f000598e0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 shutdown_connections 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563238f91160 0x563238fe6dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563238f8faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238fd5920 0x563238fdd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 --2- 172.21.15.45:0/1407110620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238f8bf30 0x563238fdcd60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 >> 172.21.15.45:0/1407110620 conn(0x563238f922a0 msgr2=0x563238f8a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8ef8c00640 0 client.6654 ms_handle_reset on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 shutdown_connections 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 1 -- 172.21.15.45:0/1407110620 wait complete. 2024-04-25T10:21:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.886+0000 7f8f1974d000 20 client.6654 trim_cache size 0 max 16384 2024-04-25T10:21:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 Processor -- start 2024-04-25T10:21:57.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 -- start start 2024-04-25T10:21:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563239031480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 0x5632390319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563239031f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f10201ef0 con 0x563238fd5920 2024-04-25T10:21:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100cd5e0 con 0x563238fd6240 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f101590e0 con 0x563238f8bf30 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 0x5632390319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 0x5632390319c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52072/0 (socket says 172.21.15.45:52072) 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.902+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3936551725 learned_addr learned my addr 172.21.15.45:0/3936551725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0f400640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563239031480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 msgr2=0x563239031480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563239031f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563239031480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 msgr2=0x563239031f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563239031f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/3936551725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563238fd6070 con 0x563238fd5920 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0f400640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563239031480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:57.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563239031f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 0x5632390319c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8efc014000 tx=0x7f8efc005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f06563fb62324f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8efb400640 1 -- 172.21.15.45:0/3936551725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00e9d0 con 0x563238fd5920 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8efb400640 1 -- 172.21.15.45:0/3936551725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc025030 con 0x563238fd5920 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8efb400640 1 -- 172.21.15.45:0/3936551725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00ed70 con 0x563238fd5920 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- 172.21.15.45:0/3936551725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 msgr2=0x5632390319c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 0x5632390319c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8efc014000 tx=0x7f8efc005000 comp rx=0 tx=0).stop 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- 172.21.15.45:0/3936551725 shutdown_connections 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563238fd6240 0x563239031f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd5920 0x5632390319c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- 172.21.15.45:0/3936551725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563239031480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- 172.21.15.45:0/3936551725 >> 172.21.15.45:0/3936551725 conn(0x563238f922a0 msgr2=0x563238f8f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- 172.21.15.45:0/3936551725 shutdown_connections 2024-04-25T10:21:57.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- 172.21.15.45:0/3936551725 wait complete. 2024-04-25T10:21:57.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- start start 2024-04-25T10:21:57.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 Processor -- start 2024-04-25T10:21:57.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238fe2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238fe2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563239036a70 0x563239036e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563238ddf3d0 con 0x563238fd6240 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563238e96d50 con 0x563239036a70 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563238dda0b0 con 0x563238f8bf30 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f1974d000 10 client.0 _ll_register_callbacks cb 0x7f8efc001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238fe2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238fe2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52076/0 (socket says 172.21.15.45:52076) 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/2661210860 learned_addr learned my addr 172.21.15.45:0/2661210860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0f400640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238fe2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0fe00640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563239036a70 0x563239036e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 msgr2=0x563238fe2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238fe2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563239036a70 msgr2=0x563239036e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563239036a70 0x563239036e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0ea00640 1 -- 172.21.15.45:0/2661210860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563238e5c600 con 0x563238fd6240 2024-04-25T10:21:57.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.906+0000 7f8f0f400640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238fe2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f0ea00640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238fe2650 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005220 tx=0x7f8efc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec3b6592ae3fa96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 1 -- 172.21.15.45:0/2661210860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00e9d0 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8efc025030 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 1 -- 172.21.15.45:0/2661210860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8efc00eb70 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 1 -- 172.21.15.45:0/2661210860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8efc03f050 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 10 client.6660 fetch_fsmap learned FSMap version 46 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x563238d6f9b0 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 1 -- 172.21.15.45:0/2661210860 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc025030 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 1 -- 172.21.15.45:0/2661210860 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8efc042020 con 0x563238fd6240 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 10 client.6660 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 10 client.6660 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 20 client.6660 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563239037580 0x563238ff0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:57.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 4 client.6660 mds_command: new command op to 6349 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 --> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563238de82e0 con 0x563239037580 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f0f400640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563239037580 0x563238ff0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f0f400640 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563239037580 0x563238ff0840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a083bfb7d525a4 server_cookie=11d783fa70d5a9dd in_seq=0 out_seq=0 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 10 client.6660 ms_handle_connect on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 1 -- 172.21.15.45:0/2661210860 <== mds.0 v2:172.21.15.117:6835/4277203907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563238de82e0 con 0x563239037580 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 10 client.6660 handle_command_reply: tid=0 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 client.6660 shutdown 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 msgr2=0x563238fe2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238fe2650 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005220 tx=0x7f8efc013000 comp rx=0 tx=0).stop 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 shutdown_connections 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x563239037580 0x563238ff0840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563239036a70 0x563239036e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238fd6240 0x563238fe2650 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8ef8c00640 0 client.6660 ms_handle_reset on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 --2- 172.21.15.45:0/2661210860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563238f8bf30 0x563238fe2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 >> 172.21.15.45:0/2661210860 conn(0x563238f922a0 msgr2=0x563238fdd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 shutdown_connections 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 1 -- 172.21.15.45:0/2661210860 wait complete. 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f1974d000 20 client.6660 trim_cache size 0 max 16384 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6349, 2024-04-25T10:21:57.922 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.6450229639999998, 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 46, 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 66, 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 66, 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 5.5179133990000002 2024-04-25T10:21:57.923 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ef0066f60 msgr2=0x7f8ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8f1015e330 tx=0x7f8efc009040 comp rx=0 tx=0).stop 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 msgr2=0x7f8f10147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 0x7f8f10147ac0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c450 tx=0x7f8f0000c910 comp rx=0 tx=0).stop 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 shutdown_connections 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f1015b910 0x7f8f1015d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f101550b0 0x7f8f1014fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 --2- 172.21.15.45:0/1225187505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f100d0040 0x7f8f10147ac0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 >> 172.21.15.45:0/1225187505 conn(0x7f8f10066b70 msgr2=0x7f8f100ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:57.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 shutdown_connections 2024-04-25T10:21:57.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:57.910+0000 7f8f16200640 1 -- 172.21.15.45:0/1225187505 wait complete. 2024-04-25T10:21:57.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.110+0000 7fc446a00640 1 Processor -- start 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.110+0000 7fc446a00640 1 -- start start 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc446a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4400d0040 0x7fc44015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc446a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc44015b740 0x7fc4400b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc446a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc440071050 con 0x7fc4401550b0 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc446a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44015d040 con 0x7fc44015b740 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc446a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4400712f0 con 0x7fc4400d0040 2024-04-25T10:21:58.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc444a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc44015b740 0x7fc4400b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc43fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4401550b0 0x7fc4400cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53248/0 (socket says 172.21.15.45:53248) 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc43fe00640 1 -- 172.21.15.45:0/357861329 learned_addr learned my addr 172.21.15.45:0/357861329 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc444a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc44015b740 0x7fc4400b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52086/0 (socket says 172.21.15.45:52086) 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc437e00640 1 --2- 172.21.15.45:0/357861329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4400d0040 0x7fc44015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc444a00640 1 -- 172.21.15.45:0/357861329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4400d0040 msgr2=0x7fc44015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc444a00640 1 --2- 172.21.15.45:0/357861329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4400d0040 0x7fc44015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc444a00640 1 -- 172.21.15.45:0/357861329 >> v1:172.21.15.45:6789/0 conn(0x7fc4401550b0 legacy=0x7fc4400cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc444a00640 1 -- 172.21.15.45:0/357861329 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400715d0 con 0x7fc44015b740 2024-04-25T10:21:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.114+0000 7fc43f400640 1 -- 172.21.15.45:0/357861329 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066886516 0 0) 0x7fc440071050 con 0x7fc4401550b0 2024-04-25T10:21:58.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc444a00640 1 --2- 172.21.15.45:0/357861329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44015b740 0x7fc4400b3260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc438009d90 tx=0x7fc43801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88f433f68d65d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:58.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc43f400640 1 -- 172.21.15.45:0/357861329 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380b2070 con 0x7fc44015b740 2024-04-25T10:21:58.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc43f400640 1 -- 172.21.15.45:0/357861329 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4380a5a00 con 0x7fc44015b740 2024-04-25T10:21:58.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc43f400640 1 -- 172.21.15.45:0/357861329 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4380a5dd0 con 0x7fc44015b740 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 -- 172.21.15.45:0/357861329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44015b740 msgr2=0x7fc4400b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 --2- 172.21.15.45:0/357861329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44015b740 0x7fc4400b3260 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc438009d90 tx=0x7fc43801c940 comp rx=0 tx=0).stop 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 -- 172.21.15.45:0/357861329 shutdown_connections 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 --2- 172.21.15.45:0/357861329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc44015b740 0x7fc4400b3260 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 --2- 172.21.15.45:0/357861329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4400d0040 0x7fc44015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 -- 172.21.15.45:0/357861329 >> 172.21.15.45:0/357861329 conn(0x7fc440066b70 msgr2=0x7fc440066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 -- 172.21.15.45:0/357861329 shutdown_connections 2024-04-25T10:21:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 -- 172.21.15.45:0/357861329 wait complete. 2024-04-25T10:21:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 Processor -- start 2024-04-25T10:21:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.118+0000 7fc446a00640 1 -- start start 2024-04-25T10:21:58.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 0x7fc440159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4401550b0 0x7fc440151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc44015b740 0x7fc4401fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015bd70 con 0x7fc4400d0040 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015bbf0 con 0x7fc44015b740 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44015c030 con 0x7fc4401550b0 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 0x7fc440159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 0x7fc440159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52102/0 (socket says 172.21.15.45:52102) 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 -- 172.21.15.45:0/1648274859 learned_addr learned my addr 172.21.15.45:0/1648274859 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc437e00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4401550b0 0x7fc440151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc444a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc44015b740 0x7fc4401fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 -- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4401550b0 msgr2=0x7fc440151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4401550b0 0x7fc440151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 -- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc44015b740 msgr2=0x7fc4401fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:58.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc44015b740 0x7fc4401fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 -- 172.21.15.45:0/1648274859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440153f00 con 0x7fc4400d0040 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc444a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc44015b740 0x7fc4401fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43fe00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 0x7fc440159cb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc4300011f0 tx=0x7fc43000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430018070 con 0x7fc4400d0040 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300074e0 con 0x7fc4400d0040 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430004a10 con 0x7fc4400d0040 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4401541e0 con 0x7fc4400d0040 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc440153ae0 con 0x7fc4400d0040 2024-04-25T10:21:58.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44004ef30 con 0x7fc4400d0040 2024-04-25T10:21:58.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.122+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc43001b070 con 0x7fc4400d0040 2024-04-25T10:21:58.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.126+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc41c066f60 0x7fc41c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:58.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.126+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc430084440 con 0x7fc4400d0040 2024-04-25T10:21:58.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.126+0000 7fc437e00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc41c066f60 0x7fc41c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:58.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.126+0000 7fc437e00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc41c066f60 0x7fc41c069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc4401fa1d0 tx=0x7fc428009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:58.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.130+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc430054d00 con 0x7fc4400d0040 2024-04-25T10:21:58.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.326+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc44006c3c0 con 0x7fc4400d0040 2024-04-25T10:21:58.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.326+0000 7fc43cc00640 1 -- 172.21.15.45:0/1648274859 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc430058da0 con 0x7fc4400d0040 2024-04-25T10:21:58.332 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc41c066f60 msgr2=0x7fc41c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc41c066f60 0x7fc41c069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc4401fa1d0 tx=0x7fc428009190 comp rx=0 tx=0).stop 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 msgr2=0x7fc440159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 0x7fc440159cb0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc4300011f0 tx=0x7fc43000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 shutdown_connections 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc41c066f60 0x7fc41c069420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc44015b740 0x7fc4401fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4401550b0 0x7fc440151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 --2- 172.21.15.45:0/1648274859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4400d0040 0x7fc440159cb0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:58.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 >> 172.21.15.45:0/1648274859 conn(0x7fc440066b70 msgr2=0x7fc44006c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:58.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 shutdown_connections 2024-04-25T10:21:58.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:58.330+0000 7fc446a00640 1 -- 172.21.15.45:0/1648274859 wait complete. 2024-04-25T10:21:58.352 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:21:58.353 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:58.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.353 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:21:58.362 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:21:58.362 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:21:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:21:58.364 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:21:58.365 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:21:58.366 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:21:58.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.367 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:21:58.426 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:21:58.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.428 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:21:58.481 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:21:58.482 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:21:58.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.482 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:21:58.482 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:21:58.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:21:58.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:21:58.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:21:58.482 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:21:58.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.578 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:21:58.587 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:21:58.587 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:21:58.587 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:58.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:21:58.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:21:58.648 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:58.700 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:21:58.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:21:58.700 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:58.751 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:21:58.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:21:58.841 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:21:58.841 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:21:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:21:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:21:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:21:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:21:58.855 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:21:58.859 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:21:58.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:21:58.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.880 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:21:58.935 INFO:teuthology.orchestra.run.smithi045.stdout:client6672 2024-04-25T10:21:58.935 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:21:58.935 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:21:58.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:58.984 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:21:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T10:21:59.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:21:59.039 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T10:21:59.039 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:21:59.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T10:21:59.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.222+0000 7f89af000640 1 Processor -- start 2024-04-25T10:21:59.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.222+0000 7f89af000640 1 -- start start 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.222+0000 7f89af000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89a815c0f0 0x7f89a80b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.222+0000 7f89af000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a806c0e0 con 0x7f89a80cf9e0 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.222+0000 7f89af000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a815c6a0 con 0x7f89a815bd10 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.222+0000 7f89af000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a815c980 con 0x7f89a815c0f0 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f899f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89a815c0f0 0x7f89a80b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f89ad000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f89a80cf9e0 0x7f89a806c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53264/0 (socket says 172.21.15.45:53264) 2024-04-25T10:21:59.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f89ad000640 1 -- 172.21.15.45:0/330718258 learned_addr learned my addr 172.21.15.45:0/330718258 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:21:59.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f899f400640 1 -- 172.21.15.45:0/330718258 >> v1:172.21.15.117:6789/0 conn(0x7f89a815bd10 legacy=0x7f89a80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f89a7400640 1 -- 172.21.15.45:0/330718258 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219102680 0 0) 0x7f89a806c0e0 con 0x7f89a80cf9e0 2024-04-25T10:21:59.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f899f400640 1 -- 172.21.15.45:0/330718258 >> v1:172.21.15.45:6789/0 conn(0x7f89a80cf9e0 legacy=0x7f89a806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f899f400640 1 -- 172.21.15.45:0/330718258 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a815cc60 con 0x7f89a815c0f0 2024-04-25T10:21:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f899f400640 1 --2- 172.21.15.45:0/330718258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a815c0f0 0x7f89a80b27e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f89900097c0 tx=0x7f899001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9565aaac35e81bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f89a7400640 1 -- 172.21.15.45:0/330718258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900b1070 con 0x7f89a815c0f0 2024-04-25T10:21:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f89a7400640 1 -- 172.21.15.45:0/330718258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89900ac410 con 0x7f89a815c0f0 2024-04-25T10:21:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.226+0000 7f89a7400640 1 -- 172.21.15.45:0/330718258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900ac7e0 con 0x7f89a815c0f0 2024-04-25T10:21:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- 172.21.15.45:0/330718258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a815c0f0 msgr2=0x7f89a80b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 --2- 172.21.15.45:0/330718258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a815c0f0 0x7f89a80b27e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f89900097c0 tx=0x7f899001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- 172.21.15.45:0/330718258 shutdown_connections 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 --2- 172.21.15.45:0/330718258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a815c0f0 0x7f89a80b27e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- 172.21.15.45:0/330718258 >> 172.21.15.45:0/330718258 conn(0x7f89a8066b70 msgr2=0x7f89a8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- 172.21.15.45:0/330718258 shutdown_connections 2024-04-25T10:21:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- 172.21.15.45:0/330718258 wait complete. 2024-04-25T10:21:59.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 Processor -- start 2024-04-25T10:21:59.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- start start 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 0x7f89a814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a815bd10 0x7f89a80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a815c0f0 0x7f89a80718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a816a460 con 0x7f89a80cf9e0 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a816a2e0 con 0x7f89a815bd10 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89af000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a816a5e0 con 0x7f89a815c0f0 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 0x7f89a814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 0x7f89a814ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52126/0 (socket says 172.21.15.45:52126) 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 -- 172.21.15.45:0/721161309 learned_addr learned my addr 172.21.15.45:0/721161309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f899f400640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a815bd10 0x7f89a80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89ad000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a815c0f0 0x7f89a80718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 -- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a815c0f0 msgr2=0x7f89a80718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a815c0f0 0x7f89a80718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 -- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a815bd10 msgr2=0x7f89a80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a815bd10 0x7f89a80713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.230+0000 7f89a7e00640 1 -- 172.21.15.45:0/721161309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a81538a0 con 0x7f89a80cf9e0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a7e00640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 0x7f89a814ec20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f899800c500 tx=0x7f899800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a4c00640 1 -- 172.21.15.45:0/721161309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998016020 con 0x7f89a80cf9e0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a4c00640 1 -- 172.21.15.45:0/721161309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998005220 con 0x7f89a80cf9e0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a4c00640 1 -- 172.21.15.45:0/721161309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998007030 con 0x7f89a80cf9e0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a8153b80 con 0x7f89a80cf9e0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a8154140 con 0x7f89a80cf9e0 2024-04-25T10:21:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a4c00640 1 -- 172.21.15.45:0/721161309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89980071f0 con 0x7f89a80cf9e0 2024-04-25T10:21:59.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a4c00640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8984066f60 0x7f8984069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.234+0000 7f89a4c00640 1 -- 172.21.15.45:0/721161309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89980849f0 con 0x7f89a80cf9e0 2024-04-25T10:21:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 Processor -- start 2024-04-25T10:21:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f899f400640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8984066f60 0x7f8984069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- start start 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f899f400640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8984066f60 0x7f8984069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f89900ae000 tx=0x7f89900a6040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f36f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f370160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f254c70 con 0x555d1f3b6e60 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f1bb040 con 0x555d1f36fd80 2024-04-25T10:21:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1f143040 con 0x555d1f3b7780 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52140/0 (socket says 172.21.15.45:52140) 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 -- 172.21.15.45:0/4064081288 learned_addr learned my addr 172.21.15.45:0/4064081288 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89ad000640 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f370160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f899f400640 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f36f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 -- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 msgr2=0x555d1f36f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f36f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 -- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 msgr2=0x555d1f370160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f370160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 -- 172.21.15.45:0/4064081288 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1f172240 con 0x555d1f3b6e60 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89a7e00640 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3b7240 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f899805d3f0 tx=0x7f8998055400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4918d2c6550990cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f899cc00640 1 -- 172.21.15.45:0/4064081288 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998059160 con 0x555d1f3b6e60 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f899cc00640 1 -- 172.21.15.45:0/4064081288 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998053360 con 0x555d1f3b6e60 2024-04-25T10:21:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f899cc00640 1 -- 172.21.15.45:0/4064081288 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998051930 con 0x555d1f3b6e60 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- 172.21.15.45:0/4064081288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 msgr2=0x555d1f3b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3b7240 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f899805d3f0 tx=0x7f8998055400 comp rx=0 tx=0).stop 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- 172.21.15.45:0/4064081288 shutdown_connections 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f370160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f36f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 --2- 172.21.15.45:0/4064081288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3b7240 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- 172.21.15.45:0/4064081288 >> 172.21.15.45:0/4064081288 conn(0x555d1f367100 msgr2=0x555d1f36e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- 172.21.15.45:0/4064081288 shutdown_connections 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- 172.21.15.45:0/4064081288 wait complete. 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 -- start start 2024-04-25T10:21:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.238+0000 7f89b2578000 1 Processor -- start 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f36fd80 0x555d1f378210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f378750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 0x555d1f378c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f175540 con 0x555d1f3b7780 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f1b70e0 con 0x555d1f3b6e60 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1f1877c0 con 0x555d1f36fd80 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 10 client.0 _ll_register_callbacks cb 0x555d1f26f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 0x555d1f378c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 0x555d1f378c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52142/0 (socket says 172.21.15.45:52142) 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 -- 172.21.15.45:0/2198216527 learned_addr learned my addr 172.21.15.45:0/2198216527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f899f400640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f378750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89a7e00640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f36fd80 0x555d1f378210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 -- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f36fd80 msgr2=0x555d1f378210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f36fd80 0x555d1f378210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 -- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 msgr2=0x555d1f378750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f378750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 -- 172.21.15.45:0/2198216527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d1f183410 con 0x555d1f3b7780 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89ad000640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 0x555d1f378c90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f89a0004f40 tx=0x7f89a00074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e986eca80c1e223d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 1 -- 172.21.15.45:0/2198216527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0013600 con 0x555d1f3b7780 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 1 -- 172.21.15.45:0/2198216527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89a000c7e0 con 0x555d1f3b7780 2024-04-25T10:21:59.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8998053360 con 0x555d1f3b7780 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 1 -- 172.21.15.45:0/2198216527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a000ca20 con 0x555d1f3b7780 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 10 client.6687 fetch_fsmap learned FSMap version 46 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x555d1f143290 con 0x555d1f3b7780 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 1 -- 172.21.15.45:0/2198216527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998053360 con 0x555d1f3b7780 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 1 -- 172.21.15.45:0/2198216527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89a000cbc0 con 0x555d1f3b7780 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 10 client.6687 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 10 client.6687 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 20 client.6687 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f379560 0x555d1f3c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 4 client.6687 mds_command: new command op to 6349 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 --> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d1f156d70 con 0x555d1f379560 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89a7e00640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f379560 0x555d1f3c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f89a7e00640 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f379560 0x555d1f3c85e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dc8dcaa252f988 server_cookie=99e5bfb90d877fe1 in_seq=0 out_seq=0 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.242+0000 7f8995600640 10 client.6687 ms_handle_connect on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f8995600640 1 -- 172.21.15.45:0/2198216527 <== mds.0 v2:172.21.15.117:6835/4277203907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d1f156d70 con 0x555d1f379560 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f8995600640 10 client.6687 handle_command_reply: tid=0 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 client.6687 shutdown 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 msgr2=0x555d1f378c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 0x555d1f378c90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f89a0004f40 tx=0x7f89a00074f0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 shutdown_connections 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f379560 0x555d1f3c85e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b7780 0x555d1f378c90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f8995600640 0 client.6687 ms_handle_reset on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f378750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 --2- 172.21.15.45:0/2198216527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f36fd80 0x555d1f378210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 >> 172.21.15.45:0/2198216527 conn(0x555d1f367100 msgr2=0x555d1f36e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 shutdown_connections 2024-04-25T10:21:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 1 -- 172.21.15.45:0/2198216527 wait complete. 2024-04-25T10:21:59.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.246+0000 7f89b2578000 20 client.6687 trim_cache size 0 max 16384 2024-04-25T10:21:59.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 Processor -- start 2024-04-25T10:21:59.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- start start 2024-04-25T10:21:59.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f3ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8203540 con 0x555d1f3b6e60 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8202740 con 0x555d1f36fd80 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8201c40 con 0x555d1f3b7780 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52146/0 (socket says 172.21.15.45:52146) 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 -- 172.21.15.45:0/2396292540 learned_addr learned my addr 172.21.15.45:0/2396292540 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89ad000640 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89a7e00640 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f3ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 -- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 msgr2=0x555d1f3c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 -- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 msgr2=0x555d1f3ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f3ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 -- 172.21.15.45:0/2396292540 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1f23d890 con 0x555d1f3b6e60 2024-04-25T10:21:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f899f400640 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3c85e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8990009580 tx=0x7f89900a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784300170c6b8f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f8997e00640 1 -- 172.21.15.45:0/2396292540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900ca070 con 0x555d1f3b6e60 2024-04-25T10:21:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f8997e00640 1 -- 172.21.15.45:0/2396292540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89900044c0 con 0x555d1f3b6e60 2024-04-25T10:21:59.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f8997e00640 1 -- 172.21.15.45:0/2396292540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900b1040 con 0x555d1f3b6e60 2024-04-25T10:21:59.268 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:21:59.265+0000 7fcf88600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:21:59.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- 172.21.15.45:0/2396292540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 msgr2=0x555d1f3c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3c85e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8990009580 tx=0x7f89900a7000 comp rx=0 tx=0).stop 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- 172.21.15.45:0/2396292540 shutdown_connections 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f3b6e60 0x555d1f3c85e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 --2- 172.21.15.45:0/2396292540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f36fd80 0x555d1f3ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- 172.21.15.45:0/2396292540 >> 172.21.15.45:0/2396292540 conn(0x555d1f367100 msgr2=0x555d1f3b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- 172.21.15.45:0/2396292540 shutdown_connections 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.262+0000 7f89b2578000 1 -- 172.21.15.45:0/2396292540 wait complete. 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- start start 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 Processor -- start 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 0x555d1f417cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f418230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f23d400 con 0x555d1f36fd80 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8202140 con 0x555d1f3b6e60 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1f1bf670 con 0x555d1f3b7780 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 10 client.0 _ll_register_callbacks cb 0x7f8998071f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 0x555d1f417cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 0x555d1f417cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52150/0 (socket says 172.21.15.45:52150) 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 -- 172.21.15.45:0/534817715 learned_addr learned my addr 172.21.15.45:0/534817715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f899f400640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f418230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89ad000640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 -- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 msgr2=0x555d1f3d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 -- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 msgr2=0x555d1f418230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f418230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 -- 172.21.15.45:0/534817715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d1f14fe50 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89a7e00640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 0x555d1f417cf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8998055250 tx=0x7f8998058070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51c4ed43687fbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 1 -- 172.21.15.45:0/534817715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998051930 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 1 -- 172.21.15.45:0/534817715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89980074e0 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89900044c0 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 1 -- 172.21.15.45:0/534817715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f899807b050 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 10 client.6693 fetch_fsmap learned FSMap version 46 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x555d1f161f80 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 1 -- 172.21.15.45:0/534817715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89900044c0 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 1 -- 172.21.15.45:0/534817715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89980730e0 con 0x555d1f36fd80 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 10 client.6693 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:21:59.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 10 client.6693 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 20 client.6693 populate_metadata read hostname 'smithi045' 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f3d3aa0 0x555d1f3d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 4 client.6693 mds_command: new command op to 6349 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 --> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x555d1f1c8580 con 0x555d1f3d3aa0 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f899f400640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f3d3aa0 0x555d1f3d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f899f400640 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f3d3aa0 0x555d1f3d3eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f589dc16883153 server_cookie=449ad2de0ea9db24 in_seq=0 out_seq=0 2024-04-25T10:21:59.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.266+0000 7f8995600640 10 client.6693 ms_handle_connect on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:59.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f8995600640 1 -- 172.21.15.45:0/534817715 <== mds.0 v2:172.21.15.117:6835/4277203907 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555d1f1c8580 con 0x555d1f3d3aa0 2024-04-25T10:21:59.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f8995600640 10 client.6693 handle_command_reply: tid=0 2024-04-25T10:21:59.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 client.6693 shutdown 2024-04-25T10:21:59.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 msgr2=0x555d1f417cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 0x555d1f417cf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8998055250 tx=0x7f8998058070 comp rx=0 tx=0).stop 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 shutdown_connections 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:6835/4277203907,v1:172.21.15.117:6837/4277203907] conn(0x555d1f3d3aa0 0x555d1f3d3eb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555d1f3b7780 0x555d1f3d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555d1f3b6e60 0x555d1f418230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 --2- 172.21.15.45:0/534817715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555d1f36fd80 0x555d1f417cf0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 >> 172.21.15.45:0/534817715 conn(0x555d1f367100 msgr2=0x555d1f3b5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f8995600640 0 client.6693 ms_handle_reset on v2:172.21.15.117:6835/4277203907 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 shutdown_connections 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 1 -- 172.21.15.45:0/534817715 wait complete. 2024-04-25T10:21:59.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89b2578000 20 client.6693 trim_cache size 0 max 16384 2024-04-25T10:21:59.278 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:21:59.278 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:21:59.278 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "13c40de2-ea7a-4c7f-aa0d-ba40bb7066a9", 2024-04-25T10:21:59.278 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:21:59.278 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8984066f60 msgr2=0x7f8984069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8984066f60 0x7f8984069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f89900ae000 tx=0x7f89900a6040 comp rx=0 tx=0).stop 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 msgr2=0x7f89a814ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 0x7f89a814ec20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f899800c500 tx=0x7f899800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 shutdown_connections 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8984066f60 0x7f8984069420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a815c0f0 0x7f89a80718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a815bd10 0x7f89a80713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 --2- 172.21.15.45:0/721161309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a80cf9e0 0x7f89a814ec20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 >> 172.21.15.45:0/721161309 conn(0x7f89a8066b70 msgr2=0x7f89a8068bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 shutdown_connections 2024-04-25T10:21:59.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:21:59.270+0000 7f89af000640 1 -- 172.21.15.45:0/721161309 wait complete. 2024-04-25T10:21:59.294 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '13c40de2-ea7a-4c7f-aa0d-ba40bb7066a9', 'mode': 'asynchronous'}' 2024-04-25T10:22:09.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:09.295 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:22:09.310 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:22:09.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:09.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:09.485 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:22:09.485 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:09.485 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:09.495 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:09.495 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:09.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 Processor -- start 2024-04-25T10:22:09.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 -- start start 2024-04-25T10:22:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f00b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f31f00b2fc0 0x7f31f006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f0070ed0 con 0x7f31f015a0e0 2024-04-25T10:22:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f00711b0 con 0x7f31f00b2fc0 2024-04-25T10:22:09.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31f6000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0071490 con 0x7f31f0071f70 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31efe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f31f00b2fc0 0x7f31f006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31efe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f31f00b2fc0 0x7f31f006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49214/0 (socket says 172.21.15.45:49214) 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31efe00640 1 -- 172.21.15.45:0/3982543903 learned_addr learned my addr 172.21.15.45:0/3982543903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.734+0000 7f31e6a00640 1 --2- 172.21.15.45:0/3982543903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f00b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31efe00640 1 -- 172.21.15.45:0/3982543903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 msgr2=0x7f31f00b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31efe00640 1 --2- 172.21.15.45:0/3982543903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f00b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31efe00640 1 -- 172.21.15.45:0/3982543903 >> v1:172.21.15.45:6789/0 conn(0x7f31f015a0e0 legacy=0x7f31f015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31efe00640 1 -- 172.21.15.45:0/3982543903 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f0071770 con 0x7f31f00b2fc0 2024-04-25T10:22:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31efe00640 1 --2- 172.21.15.45:0/3982543903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f006c090 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f31e800af60 tx=0x7f31e801db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86c588d00a52e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31eea00640 1 -- 172.21.15.45:0/3982543903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800a2d0 con 0x7f31f00b2fc0 2024-04-25T10:22:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31eea00640 1 -- 172.21.15.45:0/3982543903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80a6880 con 0x7f31f00b2fc0 2024-04-25T10:22:09.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31eea00640 1 -- 172.21.15.45:0/3982543903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80a6c50 con 0x7f31f00b2fc0 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31f6000640 1 -- 172.21.15.45:0/3982543903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 msgr2=0x7f31f006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31f6000640 1 --2- 172.21.15.45:0/3982543903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f006c090 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f31e800af60 tx=0x7f31e801db30 comp rx=0 tx=0).stop 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31f6000640 1 -- 172.21.15.45:0/3982543903 shutdown_connections 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31f6000640 1 --2- 172.21.15.45:0/3982543903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f006c090 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31f6000640 1 --2- 172.21.15.45:0/3982543903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f00b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.738+0000 7f31f6000640 1 -- 172.21.15.45:0/3982543903 >> 172.21.15.45:0/3982543903 conn(0x7f31f0066b70 msgr2=0x7f31f0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 -- 172.21.15.45:0/3982543903 shutdown_connections 2024-04-25T10:22:09.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 -- 172.21.15.45:0/3982543903 wait complete. 2024-04-25T10:22:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 Processor -- start 2024-04-25T10:22:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 -- start start 2024-04-25T10:22:09.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f0155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f0145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f015a0e0 0x7f31f00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0072350 con 0x7f31f00b2fc0 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f0070b50 con 0x7f31f015a0e0 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31f6000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00724d0 con 0x7f31f0071f70 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f0145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f0145d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49218/0 (socket says 172.21.15.45:49218) 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 -- 172.21.15.45:0/4160400783 learned_addr learned my addr 172.21.15.45:0/4160400783 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31ef400640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31efe00640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f015a0e0 0x7f31f00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 -- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 msgr2=0x7f31f0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 -- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f015a0e0 msgr2=0x7f31f00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f015a0e0 0x7f31f00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 -- 172.21.15.45:0/4160400783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f01473b0 con 0x7f31f00b2fc0 2024-04-25T10:22:09.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e6a00640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f0145d30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f31d800d800 tx=0x7f31d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.742+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800b6c0 con 0x7f31f00b2fc0 2024-04-25T10:22:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8019070 con 0x7f31f00b2fc0 2024-04-25T10:22:09.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800ba60 con 0x7f31f00b2fc0 2024-04-25T10:22:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0147690 con 0x7f31f00b2fc0 2024-04-25T10:22:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f015ab90 con 0x7f31f00b2fc0 2024-04-25T10:22:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f004ef30 con 0x7f31f00b2fc0 2024-04-25T10:22:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31d800bc00 con 0x7f31f00b2fc0 2024-04-25T10:22:09.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31e7e00640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31cc066f60 0x7f31cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31d8085430 con 0x7f31f00b2fc0 2024-04-25T10:22:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.746+0000 7f31ef400640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31cc066f60 0x7f31cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.750+0000 7f31ef400640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31cc066f60 0x7f31cc069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f31e0001770 tx=0x7f31e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:09.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.750+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31d8055d80 con 0x7f31f00b2fc0 2024-04-25T10:22:09.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.914+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31f00703e0 con 0x7f31f00b2fc0 2024-04-25T10:22:09.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.914+0000 7f31e7e00640 1 -- 172.21.15.45:0/4160400783 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f31d805c0d0 con 0x7f31f00b2fc0 2024-04-25T10:22:09.919 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:09.919 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:52.261160+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3997755731":"2024-04-26T10:21:52.255019+0000","172.21.15.117:6835/3997755731":"2024-04-26T10:21:52.255019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31cc066f60 msgr2=0x7f31cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31cc066f60 0x7f31cc069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f31e0001770 tx=0x7f31e0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:09.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 msgr2=0x7f31f0145d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f0145d30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f31d800d800 tx=0x7f31d800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 shutdown_connections 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31cc066f60 0x7f31cc069420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31f015a0e0 0x7f31f00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f00b2fc0 0x7f31f0145d30 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 --2- 172.21.15.45:0/4160400783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31f0071f70 0x7f31f0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 >> 172.21.15.45:0/4160400783 conn(0x7f31f0066b70 msgr2=0x7f31f014b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 shutdown_connections 2024-04-25T10:22:09.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:09.918+0000 7f31f6000640 1 -- 172.21.15.45:0/4160400783 wait complete. 2024-04-25T10:22:09.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:21:38.360777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:21:39.578182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:09.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 Processor -- start 2024-04-25T10:22:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 -- start start 2024-04-25T10:22:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bf4070350 0x7f5bf415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40b31c0 0x7f5bf40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf415b080 con 0x7f5bf40aef80 2024-04-25T10:22:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf415adb0 con 0x7f5bf4070350 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfc800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf415eb60 con 0x7f5bf40b31c0 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bf4070350 0x7f5bf415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bf4070350 0x7f5bf415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49228/0 (socket says 172.21.15.45:49228) 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 -- 172.21.15.45:0/2925887984 learned_addr learned my addr 172.21.15.45:0/2925887984 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfa800640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40b31c0 0x7f5bf40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 -- 172.21.15.45:0/2925887984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40b31c0 msgr2=0x7f5bf40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40b31c0 0x7f5bf40b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 -- 172.21.15.45:0/2925887984 >> v1:172.21.15.117:6789/0 conn(0x7f5bf40aef80 legacy=0x7f5bf415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bf9400640 1 -- 172.21.15.45:0/2925887984 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf415ee40 con 0x7f5bf4070350 2024-04-25T10:22:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.082+0000 7f5bfa800640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40b31c0 0x7f5bf40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:10.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bf9400640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf415e480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5be400bb10 tx=0x7f5be401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9262b9ec55b02590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bf8a00640 1 -- 172.21.15.45:0/2925887984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40b4070 con 0x7f5bf4070350 2024-04-25T10:22:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bf8a00640 1 -- 172.21.15.45:0/2925887984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40a7c10 con 0x7f5bf4070350 2024-04-25T10:22:10.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bf8a00640 1 -- 172.21.15.45:0/2925887984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40af440 con 0x7f5bf4070350 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 -- 172.21.15.45:0/2925887984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 msgr2=0x7f5bf415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf415e480 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5be400bb10 tx=0x7f5be401ec30 comp rx=0 tx=0).stop 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 -- 172.21.15.45:0/2925887984 shutdown_connections 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40b31c0 0x7f5bf40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2925887984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf415e480 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 -- 172.21.15.45:0/2925887984 >> 172.21.15.45:0/2925887984 conn(0x7f5bf40669a0 msgr2=0x7f5bf4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 -- 172.21.15.45:0/2925887984 shutdown_connections 2024-04-25T10:22:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 -- 172.21.15.45:0/2925887984 wait complete. 2024-04-25T10:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.086+0000 7f5bfc800640 1 Processor -- start 2024-04-25T10:22:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- start start 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40aef80 0x7f5bf41575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf40b31c0 0x7f5bf40ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406c250 con 0x7f5bf4070350 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406c0d0 con 0x7f5bf40b31c0 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf406c3d0 con 0x7f5bf40aef80 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf414cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49238/0 (socket says 172.21.15.45:49238) 2024-04-25T10:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 -- 172.21.15.45:0/2841688439 learned_addr learned my addr 172.21.15.45:0/2841688439 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9400640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40aef80 0x7f5bf41575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfa800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf40b31c0 0x7f5bf40ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 -- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40aef80 msgr2=0x7f5bf41575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40aef80 0x7f5bf41575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 -- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf40b31c0 msgr2=0x7f5bf40ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf40b31c0 0x7f5bf40ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 -- 172.21.15.45:0/2841688439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4157e30 con 0x7f5bf4070350 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bf9e00640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf414cea0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5be8007fd0 tx=0x7f5be800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8017070 con 0x7f5bf4070350 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be80052d0 con 0x7f5bf4070350 2024-04-25T10:22:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003840 con 0x7f5bf4070350 2024-04-25T10:22:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf4158110 con 0x7f5bf4070350 2024-04-25T10:22:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf41586d0 con 0x7f5bf4070350 2024-04-25T10:22:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.090+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf404ef30 con 0x7f5bf4070350 2024-04-25T10:22:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.094+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5be801a070 con 0x7f5bf4070350 2024-04-25T10:22:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.094+0000 7f5bee000640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5bd8066f60 0x7f5bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.094+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5be8084420 con 0x7f5bf4070350 2024-04-25T10:22:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.094+0000 7f5bf9400640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5bd8066f60 0x7f5bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.094+0000 7f5bf9400640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5bd8066f60 0x7f5bd8069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5be40b1000 tx=0x7f5be4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.098+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5be8005480 con 0x7f5bf4070350 2024-04-25T10:22:10.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.262+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bf40b2fa0 con 0x7f5bf4070350 2024-04-25T10:22:10.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.262+0000 7f5bee000640 1 -- 172.21.15.45:0/2841688439 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5be8054ce0 con 0x7f5bf4070350 2024-04-25T10:22:10.266 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:22:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5bd8066f60 msgr2=0x7f5bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5bd8066f60 0x7f5bd8069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5be40b1000 tx=0x7f5be4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 msgr2=0x7f5bf414cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf414cea0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5be8007fd0 tx=0x7f5be800c890 comp rx=0 tx=0).stop 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 shutdown_connections 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5bd8066f60 0x7f5bd8069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bf40b31c0 0x7f5bf40ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bf40aef80 0x7f5bf41575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 --2- 172.21.15.45:0/2841688439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf4070350 0x7f5bf414cea0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 >> 172.21.15.45:0/2841688439 conn(0x7f5bf40669a0 msgr2=0x7f5bf415d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 shutdown_connections 2024-04-25T10:22:10.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.266+0000 7f5bfc800640 1 -- 172.21.15.45:0/2841688439 wait complete. 2024-04-25T10:22:10.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:22:10.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 Processor -- start 2024-04-25T10:22:10.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 -- start start 2024-04-25T10:22:10.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff9a0159540 0x7ff9a00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a00aebd0 0x7ff9a0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9a0072020 0x7ff9a015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a0159920 con 0x7ff9a0072020 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a015c2f0 con 0x7ff9a0159540 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.426+0000 7ff9a6c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015c5d0 con 0x7ff9a00aebd0 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9a0072020 0x7ff9a015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9a0072020 0x7ff9a015f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49240/0 (socket says 172.21.15.45:49240) 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 -- 172.21.15.45:0/1918610507 learned_addr learned my addr 172.21.15.45:0/1918610507 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff99fe00640 1 --2- 172.21.15.45:0/1918610507 >> v2:172.21.15.117:3300/0 conn(0x7ff9a0159540 0x7ff9a00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 -- 172.21.15.45:0/1918610507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a00aebd0 msgr2=0x7ff9a0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff997e00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a00aebd0 0x7ff9a0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a00aebd0 0x7ff9a0071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 -- 172.21.15.45:0/1918610507 >> v2:172.21.15.117:3300/0 conn(0x7ff9a0159540 msgr2=0x7ff9a00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 --2- 172.21.15.45:0/1918610507 >> v2:172.21.15.117:3300/0 conn(0x7ff9a0159540 0x7ff9a00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 -- 172.21.15.45:0/1918610507 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a015c8b0 con 0x7ff9a0072020 2024-04-25T10:22:10.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff99fe00640 1 --2- 172.21.15.45:0/1918610507 >> v2:172.21.15.117:3300/0 conn(0x7ff9a0159540 0x7ff9a00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:10.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff997e00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a00aebd0 0x7ff9a0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:10.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff9a4c00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a0072020 0x7ff9a015f590 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00b4c20 tx=0x7ff99801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a925f330d06cd836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff99f400640 1 -- 172.21.15.45:0/1918610507 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980ac020 con 0x7ff9a0072020 2024-04-25T10:22:10.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff99f400640 1 -- 172.21.15.45:0/1918610507 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9980a6450 con 0x7ff9a0072020 2024-04-25T10:22:10.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.430+0000 7ff99f400640 1 -- 172.21.15.45:0/1918610507 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980a6820 con 0x7ff9a0072020 2024-04-25T10:22:10.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- 172.21.15.45:0/1918610507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a0072020 msgr2=0x7ff9a015f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a0072020 0x7ff9a015f590 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00b4c20 tx=0x7ff99801c730 comp rx=0 tx=0).stop 2024-04-25T10:22:10.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- 172.21.15.45:0/1918610507 shutdown_connections 2024-04-25T10:22:10.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a0072020 0x7ff9a015f590 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/1918610507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a00aebd0 0x7ff9a0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/1918610507 >> v2:172.21.15.117:3300/0 conn(0x7ff9a0159540 0x7ff9a00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- 172.21.15.45:0/1918610507 >> 172.21.15.45:0/1918610507 conn(0x7ff9a00669a0 msgr2=0x7ff9a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- 172.21.15.45:0/1918610507 shutdown_connections 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- 172.21.15.45:0/1918610507 wait complete. 2024-04-25T10:22:10.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 Processor -- start 2024-04-25T10:22:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- start start 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a0072020 0x7ff9a014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 0x7ff9a014f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0159540 0x7ff9a00cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0070350 con 0x7ff9a00aebd0 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006c0d0 con 0x7ff9a0159540 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a6c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00704d0 con 0x7ff9a0072020 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 0x7ff9a014f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 0x7ff9a014f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49246/0 (socket says 172.21.15.45:49246) 2024-04-25T10:22:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 -- 172.21.15.45:0/2140724472 learned_addr learned my addr 172.21.15.45:0/2140724472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a4c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0159540 0x7ff9a00cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff99fe00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a0072020 0x7ff9a014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 -- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a0072020 msgr2=0x7ff9a014b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a0072020 0x7ff9a014b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 -- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0159540 msgr2=0x7ff9a00cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0159540 0x7ff9a00cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 -- 172.21.15.45:0/2140724472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a014fb40 con 0x7ff9a00aebd0 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff9a4c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0159540 0x7ff9a00cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff99fe00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a0072020 0x7ff9a014b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.434+0000 7ff997e00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 0x7ff9a014f240 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c7f0 tx=0x7ff98800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988007a30 con 0x7ff9a00aebd0 2024-04-25T10:22:10.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988007bd0 con 0x7ff9a00aebd0 2024-04-25T10:22:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880113f0 con 0x7ff9a00aebd0 2024-04-25T10:22:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a014fe20 con 0x7ff9a00aebd0 2024-04-25T10:22:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a01503e0 con 0x7ff9a00aebd0 2024-04-25T10:22:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a004ef30 con 0x7ff9a00aebd0 2024-04-25T10:22:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff988011590 con 0x7ff9a00aebd0 2024-04-25T10:22:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99cc00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff97c066f60 0x7ff97c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff9880852e0 con 0x7ff9a00aebd0 2024-04-25T10:22:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.438+0000 7ff99fe00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff97c066f60 0x7ff97c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.442+0000 7ff99fe00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff97c066f60 0x7ff97c069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b840 tx=0x7ff990008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.442+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff988015020 con 0x7ff9a00aebd0 2024-04-25T10:22:10.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.602+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9a00ccef0 con 0x7ff9a00aebd0 2024-04-25T10:22:10.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.602+0000 7ff99cc00640 1 -- 172.21.15.45:0/2140724472 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff988056280 con 0x7ff9a00aebd0 2024-04-25T10:22:10.608 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:22:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff97c066f60 msgr2=0x7ff97c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff97c066f60 0x7ff97c069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b840 tx=0x7ff990008040 comp rx=0 tx=0).stop 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 msgr2=0x7ff9a014f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 0x7ff9a014f240 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c7f0 tx=0x7ff98800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 shutdown_connections 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff97c066f60 0x7ff97c069420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9a0159540 0x7ff9a00cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9a00aebd0 0x7ff9a014f240 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 --2- 172.21.15.45:0/2140724472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9a0072020 0x7ff9a014b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 >> 172.21.15.45:0/2140724472 conn(0x7ff9a00669a0 msgr2=0x7ff9a006de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 shutdown_connections 2024-04-25T10:22:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.606+0000 7ff9a6c00640 1 -- 172.21.15.45:0/2140724472 wait complete. 2024-04-25T10:22:10.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:22:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 Processor -- start 2024-04-25T10:22:10.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 -- start start 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe154159540 0x7fe1540ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe1540721f0 0x7fe154070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe154159920 con 0x7fe1540aebd0 2024-04-25T10:22:10.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540af030 con 0x7fe1540721f0 2024-04-25T10:22:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe15ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15415c2f0 con 0x7fe154159540 2024-04-25T10:22:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe14be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe1540aebd0 0x7fe154071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33946/0 (socket says 172.21.15.45:33946) 2024-04-25T10:22:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe14be00640 1 -- 172.21.15.45:0/1687681551 learned_addr learned my addr 172.21.15.45:0/1687681551 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:10.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153e00640 1 --2- 172.21.15.45:0/1687681551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe154159540 0x7fe1540ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe158e00640 1 --2- 172.21.15.45:0/1687681551 >> v2:172.21.15.117:3300/0 conn(0x7fe1540721f0 0x7fe154070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153400640 1 -- 172.21.15.45:0/1687681551 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267610175 0 0) 0x7fe154159920 con 0x7fe1540aebd0 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153400640 1 -- 172.21.15.45:0/1687681551 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe138003530 con 0x7fe1540aebd0 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153e00640 1 -- 172.21.15.45:0/1687681551 >> v2:172.21.15.117:3300/0 conn(0x7fe1540721f0 msgr2=0x7fe154070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153e00640 1 --2- 172.21.15.45:0/1687681551 >> v2:172.21.15.117:3300/0 conn(0x7fe1540721f0 0x7fe154070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153e00640 1 -- 172.21.15.45:0/1687681551 >> v1:172.21.15.45:6789/0 conn(0x7fe1540aebd0 legacy=0x7fe154071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.770+0000 7fe153e00640 1 -- 172.21.15.45:0/1687681551 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15415c5d0 con 0x7fe154159540 2024-04-25T10:22:10.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe153e00640 1 --2- 172.21.15.45:0/1687681551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe154159540 0x7fe1540ae690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1440097c0 tx=0x7fe14401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f7ebba034dab45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe153400640 1 -- 172.21.15.45:0/1687681551 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440a7070 con 0x7fe154159540 2024-04-25T10:22:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe153400640 1 -- 172.21.15.45:0/1687681551 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1440b0070 con 0x7fe154159540 2024-04-25T10:22:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe153400640 1 -- 172.21.15.45:0/1687681551 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440ab890 con 0x7fe154159540 2024-04-25T10:22:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 -- 172.21.15.45:0/1687681551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe154159540 msgr2=0x7fe1540ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1687681551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe154159540 0x7fe1540ae690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1440097c0 tx=0x7fe14401c470 comp rx=0 tx=0).stop 2024-04-25T10:22:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 -- 172.21.15.45:0/1687681551 shutdown_connections 2024-04-25T10:22:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1687681551 >> v2:172.21.15.117:3300/0 conn(0x7fe1540721f0 0x7fe154070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1687681551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe154159540 0x7fe1540ae690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 -- 172.21.15.45:0/1687681551 >> 172.21.15.45:0/1687681551 conn(0x7fe1540669a0 msgr2=0x7fe154066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 -- 172.21.15.45:0/1687681551 shutdown_connections 2024-04-25T10:22:10.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.774+0000 7fe15ae00640 1 -- 172.21.15.45:0/1687681551 wait complete. 2024-04-25T10:22:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 Processor -- start 2024-04-25T10:22:10.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- start start 2024-04-25T10:22:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 0x7fe15414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1540aebd0 0x7fe15414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe154159540 0x7fe154151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540710a0 con 0x7fe1540721f0 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15406c0d0 con 0x7fe154159540 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe154071220 con 0x7fe1540aebd0 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 0x7fe15414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 0x7fe15414d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49254/0 (socket says 172.21.15.45:49254) 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 -- 172.21.15.45:0/1753916790 learned_addr learned my addr 172.21.15.45:0/1753916790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe14be00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1540aebd0 0x7fe15414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe158e00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe154159540 0x7fe154151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 -- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1540aebd0 msgr2=0x7fe15414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1540aebd0 0x7fe15414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 -- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe154159540 msgr2=0x7fe154151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe154159540 0x7fe154151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 -- 172.21.15.45:0/1753916790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15414fab0 con 0x7fe1540721f0 2024-04-25T10:22:10.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe153e00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 0x7fe15414d410 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe1440097c0 tx=0x7fe14401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440bf610 con 0x7fe1540721f0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1440abe50 con 0x7fe1540721f0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440bfa10 con 0x7fe1540721f0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15414ef30 con 0x7fe1540721f0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15414f4f0 con 0x7fe1540721f0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.778+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15414d950 con 0x7fe1540721f0 2024-04-25T10:22:10.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.782+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1440bd020 con 0x7fe1540721f0 2024-04-25T10:22:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.782+0000 7fe150c00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe130066f60 0x7fe130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.782+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe14411f9d0 con 0x7fe1540721f0 2024-04-25T10:22:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.782+0000 7fe14be00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe130066f60 0x7fe130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.782+0000 7fe14be00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe130066f60 0x7fe130069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe13c007fd0 tx=0x7fe13c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.786+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe144016090 con 0x7fe1540721f0 2024-04-25T10:22:10.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.946+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1540cf470 con 0x7fe1540721f0 2024-04-25T10:22:10.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe150c00640 1 -- 172.21.15.45:0/1753916790 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe1440f0440 con 0x7fe1540721f0 2024-04-25T10:22:10.952 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:22:10.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe130066f60 msgr2=0x7fe130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe130066f60 0x7fe130069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe13c007fd0 tx=0x7fe13c009040 comp rx=0 tx=0).stop 2024-04-25T10:22:10.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 msgr2=0x7fe15414d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 0x7fe15414d410 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe1440097c0 tx=0x7fe14401acb0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 shutdown_connections 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe130066f60 0x7fe130069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe154159540 0x7fe154151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1540aebd0 0x7fe15414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 --2- 172.21.15.45:0/1753916790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1540721f0 0x7fe15414d410 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 >> 172.21.15.45:0/1753916790 conn(0x7fe1540669a0 msgr2=0x7fe15415d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 shutdown_connections 2024-04-25T10:22:10.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:10.950+0000 7fe15ae00640 1 -- 172.21.15.45:0/1753916790 wait complete. 2024-04-25T10:22:10.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:22:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.110+0000 7facfe800640 1 Processor -- start 2024-04-25T10:22:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.110+0000 7facfe800640 1 -- start start 2024-04-25T10:22:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facfe800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7facf80d0350 0x7facf80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facfe800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf80ccaa0 con 0x7facf806aea0 2024-04-25T10:22:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facfe800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf80ccd80 con 0x7facf815df60 2024-04-25T10:22:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facfe800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf815e340 con 0x7facf80d0350 2024-04-25T10:22:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7facf806aea0 0x7facf806b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33948/0 (socket says 172.21.15.45:33948) 2024-04-25T10:22:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7e00640 1 -- 172.21.15.45:0/3082404620 learned_addr learned my addr 172.21.15.45:0/3082404620 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7faceea00640 1 --2- 172.21.15.45:0/3082404620 >> v2:172.21.15.117:3300/0 conn(0x7facf80d0350 0x7facf80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423846255 0 0) 0x7facf80ccaa0 con 0x7facf806aea0 2024-04-25T10:22:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facdc003530 con 0x7facf806aea0 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706014455 0 0) 0x7facdc003530 con 0x7facf806aea0 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 >> v2:172.21.15.117:3300/0 conn(0x7facf80d0350 msgr2=0x7facf80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 --2- 172.21.15.45:0/3082404620 >> v2:172.21.15.117:3300/0 conn(0x7facf80d0350 0x7facf80d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 >> v1:172.21.15.117:6789/0 conn(0x7facf815df60 legacy=0x7facf80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf80cd060 con 0x7facf806aea0 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7facf0001a00 con 0x7facf806aea0 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 >> v1:172.21.15.45:6789/0 conn(0x7facf806aea0 legacy=0x7facf806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facdc0039c0 0x7facdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc004810 0x7facdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0051d0 0x7facdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc00c140 con 0x7facdc0039c0 2024-04-25T10:22:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc00c2f0 con 0x7facdc0051d0 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc00c4f0 con 0x7facdc004810 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7400640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facdc0039c0 0x7facdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7faceea00640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc004810 0x7facdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7e00640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0051d0 0x7facdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7400640 1 -- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc004810 msgr2=0x7facdc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7400640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facdc004810 0x7facdc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7400640 1 -- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0051d0 msgr2=0x7facdc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7400640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0051d0 0x7facdc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7400640 1 -- 172.21.15.45:0/3082404620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc003530 con 0x7facdc0039c0 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7e00640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0051d0 0x7facdc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.114+0000 7facf7e00640 1 -- 172.21.15.45:0/3082404620 reap_dead start 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facf7400640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facdc0039c0 0x7facdc003de0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7face8001780 tx=0x7face801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d364a4199b357d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80013a0 con 0x7facdc0039c0 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face80052f0 con 0x7facdc0039c0 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facf6a00640 1 -- 172.21.15.45:0/3082404620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8003840 con 0x7facdc0039c0 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 -- 172.21.15.45:0/3082404620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facdc0039c0 msgr2=0x7facdc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facdc0039c0 0x7facdc003de0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7face8001780 tx=0x7face801c400 comp rx=0 tx=0).stop 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 -- 172.21.15.45:0/3082404620 shutdown_connections 2024-04-25T10:22:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 --2- 172.21.15.45:0/3082404620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facdc0039c0 0x7facdc003de0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 -- 172.21.15.45:0/3082404620 >> 172.21.15.45:0/3082404620 conn(0x7facf80d1910 msgr2=0x7facf80d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 -- 172.21.15.45:0/3082404620 shutdown_connections 2024-04-25T10:22:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.118+0000 7facfe800640 1 -- 172.21.15.45:0/3082404620 wait complete. 2024-04-25T10:22:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 Processor -- start 2024-04-25T10:22:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- start start 2024-04-25T10:22:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0039c0 0x7facf81fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 0x7facf8160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facf80c6cf0 0x7facf80c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80d0ff0 con 0x7facf80caef0 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80d0e70 con 0x7facdc0039c0 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf806ba90 con 0x7facf80c6cf0 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 0x7facf8160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 0x7facf8160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49270/0 (socket says 172.21.15.45:49270) 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 -- 172.21.15.45:0/1067142679 learned_addr learned my addr 172.21.15.45:0/1067142679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facf7e00640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facf80c6cf0 0x7facf80c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 -- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facf80c6cf0 msgr2=0x7facf80c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facf7400640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0039c0 0x7facf81fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facf80c6cf0 0x7facf80c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 -- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0039c0 msgr2=0x7facf81fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0039c0 0x7facf81fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 -- 172.21.15.45:0/1067142679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf81fb130 con 0x7facf80caef0 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facf7e00640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facf80c6cf0 0x7facf80c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facf7400640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0039c0 0x7facf81fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7faceea00640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 0x7facf8160620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7face0011aa0 tx=0x7face0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face000fb80 con 0x7facf80caef0 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face000fd20 con 0x7facf80caef0 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf81f6f00 con 0x7facf80caef0 2024-04-25T10:22:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0014680 con 0x7facf80caef0 2024-04-25T10:22:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf81f74c0 con 0x7facf80caef0 2024-04-25T10:22:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.122+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf81f7b80 con 0x7facf80caef0 2024-04-25T10:22:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.126+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7face0012040 con 0x7facf80caef0 2024-04-25T10:22:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.126+0000 7facefe00640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7facd4066f60 0x7facd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.126+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7face0085780 con 0x7facf80caef0 2024-04-25T10:22:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.126+0000 7facf7400640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7facd4066f60 0x7facd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.126+0000 7facf7400640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7facd4066f60 0x7facd4069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7face8001750 tx=0x7face80011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:11.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.130+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7facf81f7b80 con 0x7facf80caef0 2024-04-25T10:22:11.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.302+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7facf804ef30 con 0x7facf80caef0 2024-04-25T10:22:11.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.302+0000 7facefe00640 1 -- 172.21.15.45:0/1067142679 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v66) v1 ==== 80+0+124 (secure 0 0 0) 0x7face0019030 con 0x7facf80caef0 2024-04-25T10:22:11.308 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.117:6837/3997755731 2024-04-26T10:21:52.255019+0000 2024-04-25T10:22:11.308 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.117:6835/3997755731 2024-04-26T10:21:52.255019+0000 2024-04-25T10:22:11.308 INFO:teuthology.orchestra.run.smithi045.stderr:listed 2 entries 2024-04-25T10:22:11.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7facd4066f60 msgr2=0x7facd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7facd4066f60 0x7facd4069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7face8001750 tx=0x7face80011a0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 msgr2=0x7facf8160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 0x7facf8160620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7face0011aa0 tx=0x7face0011f60 comp rx=0 tx=0).stop 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 shutdown_connections 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7facd4066f60 0x7facd4069420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facf80c6cf0 0x7facf80c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf80caef0 0x7facf8160620 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 --2- 172.21.15.45:0/1067142679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facdc0039c0 0x7facf81fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 >> 172.21.15.45:0/1067142679 conn(0x7facf80d1910 msgr2=0x7facf80d1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:11.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 shutdown_connections 2024-04-25T10:22:11.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.306+0000 7facfe800640 1 -- 172.21.15.45:0/1067142679 wait complete. 2024-04-25T10:22:11.330 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:22:11.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:22:11.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:22:11.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.359 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:22:11.412 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:22:11.413 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:11.422 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:11.423 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.463 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:11.464 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:22:11.464 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:11.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:11.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.642+0000 7feada000640 1 Processor -- start 2024-04-25T10:22:11.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.642+0000 7feada000640 1 -- start start 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feada000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fead40cda10 0x7fead406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feada000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fead40cf930 0x7fead40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feada000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40b2770 con 0x7fead415bc40 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feada000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead406c670 con 0x7fead40cda10 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feada000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead406c950 con 0x7fead40cf930 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fead40cda10 0x7fead406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fead40cda10 0x7fead406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49280/0 (socket says 172.21.15.45:49280) 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 -- 172.21.15.45:0/1785403976 learned_addr learned my addr 172.21.15.45:0/1785403976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7fead3e00640 1 --2- 172.21.15.45:0/1785403976 >> v2:172.21.15.117:3300/0 conn(0x7fead40cf930 0x7fead40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 -- 172.21.15.45:0/1785403976 >> v2:172.21.15.117:3300/0 conn(0x7fead40cf930 msgr2=0x7fead40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 --2- 172.21.15.45:0/1785403976 >> v2:172.21.15.117:3300/0 conn(0x7fead40cf930 0x7fead40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 -- 172.21.15.45:0/1785403976 >> v1:172.21.15.117:6789/0 conn(0x7fead415bc40 legacy=0x7fead40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 -- 172.21.15.45:0/1785403976 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead406cc30 con 0x7fead40cda10 2024-04-25T10:22:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.646+0000 7feacbe00640 1 --2- 172.21.15.45:0/1785403976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cda10 0x7fead406c090 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fead40ae4c0 tx=0x7feabc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cd208a92f2fce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7fead2a00640 1 -- 172.21.15.45:0/1785403976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0b0070 con 0x7fead40cda10 2024-04-25T10:22:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7fead2a00640 1 -- 172.21.15.45:0/1785403976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc0ab600 con 0x7fead40cda10 2024-04-25T10:22:11.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7fead2a00640 1 -- 172.21.15.45:0/1785403976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0ab9d0 con 0x7fead40cda10 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- 172.21.15.45:0/1785403976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cda10 msgr2=0x7fead406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 --2- 172.21.15.45:0/1785403976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cda10 0x7fead406c090 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fead40ae4c0 tx=0x7feabc01c450 comp rx=0 tx=0).stop 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- 172.21.15.45:0/1785403976 shutdown_connections 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 --2- 172.21.15.45:0/1785403976 >> v2:172.21.15.117:3300/0 conn(0x7fead40cf930 0x7fead40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 --2- 172.21.15.45:0/1785403976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cda10 0x7fead406c090 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- 172.21.15.45:0/1785403976 >> 172.21.15.45:0/1785403976 conn(0x7fead4066b70 msgr2=0x7fead4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- 172.21.15.45:0/1785403976 shutdown_connections 2024-04-25T10:22:11.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- 172.21.15.45:0/1785403976 wait complete. 2024-04-25T10:22:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 Processor -- start 2024-04-25T10:22:11.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- start start 2024-04-25T10:22:11.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead40cda10 0x7fead4158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 0x7fead415d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead415bc40 0x7fead4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b3850 con 0x7fead40cf930 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b36d0 con 0x7fead40cda10 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feada000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b39d0 con 0x7fead415bc40 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feacbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 0x7fead415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feacbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 0x7fead415d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49284/0 (socket says 172.21.15.45:49284) 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feacbe00640 1 -- 172.21.15.45:0/3342493667 learned_addr learned my addr 172.21.15.45:0/3342493667 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7fead3400640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead40cda10 0x7fead4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7fead3e00640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead415bc40 0x7fead4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.650+0000 7feacbe00640 1 -- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead415bc40 msgr2=0x7fead4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacbe00640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead415bc40 0x7fead4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacbe00640 1 -- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead40cda10 msgr2=0x7fead4158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacbe00640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead40cda10 0x7fead4158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacbe00640 1 -- 172.21.15.45:0/3342493667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4152530 con 0x7fead40cf930 2024-04-25T10:22:11.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7fead3400640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead40cda10 0x7fead4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacbe00640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 0x7fead415d520 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7feabc0ae040 tx=0x7feabc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0b0040 con 0x7fead40cf930 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc0b9070 con 0x7fead40cf930 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0b48a0 con 0x7fead40cf930 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead415dc20 con 0x7fead40cf930 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead415e1e0 con 0x7fead40cf930 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead404ef30 con 0x7fead40cf930 2024-04-25T10:22:11.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feabc0b4a40 con 0x7fead40cf930 2024-04-25T10:22:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacb400640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feab0066f60 0x7feab0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7feabc11ee40 con 0x7fead40cf930 2024-04-25T10:22:11.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.654+0000 7fead3400640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feab0066f60 0x7feab0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:11.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.658+0000 7fead3400640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feab0066f60 0x7feab0069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feac40098b0 tx=0x7feac4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:11.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.658+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feabc0160d0 con 0x7fead40cf930 2024-04-25T10:22:11.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.854+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fead406c1a0 con 0x7fead40cf930 2024-04-25T10:22:11.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.854+0000 7feacb400640 1 -- 172.21.15.45:0/3342493667 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4945 (secure 0 0 0) 0x7feabc0efe70 con 0x7fead40cf930 2024-04-25T10:22:11.859 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:11.859 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:55.281089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:22:11.859 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feab0066f60 msgr2=0x7feab0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feab0066f60 0x7feab0069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feac40098b0 tx=0x7feac4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 msgr2=0x7fead415d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 0x7fead415d520 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7feabc0ae040 tx=0x7feabc0191f0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 shutdown_connections 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feab0066f60 0x7feab0069420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fead415bc40 0x7fead4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40cf930 0x7fead415d520 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 --2- 172.21.15.45:0/3342493667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fead40cda10 0x7fead4158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 >> 172.21.15.45:0/3342493667 conn(0x7fead4066b70 msgr2=0x7fead4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:11.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 shutdown_connections 2024-04-25T10:22:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:11.858+0000 7feada000640 1 -- 172.21.15.45:0/3342493667 wait complete. 2024-04-25T10:22:11.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d7400640 1 Processor -- start 2024-04-25T10:22:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d7400640 1 -- start start 2024-04-25T10:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f27d015dab0 0x7f27d0162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d7400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d00b0210 con 0x7f27d00669a0 2024-04-25T10:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d7400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d00aff30 con 0x7f27d00b57e0 2024-04-25T10:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d7400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d0161840 con 0x7f27d015dab0 2024-04-25T10:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d5400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f27d00669a0 0x7f27d0066dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33962/0 (socket says 172.21.15.45:33962) 2024-04-25T10:22:12.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d5400640 1 -- 172.21.15.45:0/823478563 learned_addr learned my addr 172.21.15.45:0/823478563 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d4a00640 1 --2- 172.21.15.45:0/823478563 >> v2:172.21.15.117:3300/0 conn(0x7f27d015dab0 0x7f27d0162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813843234 0 0) 0x7f27d00b0210 con 0x7f27d00669a0 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8003530 con 0x7f27d00669a0 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793313784 0 0) 0x7f27b8003530 con 0x7f27d00669a0 2024-04-25T10:22:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 >> v2:172.21.15.117:3300/0 conn(0x7f27d015dab0 msgr2=0x7f27d0162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 --2- 172.21.15.45:0/823478563 >> v2:172.21.15.117:3300/0 conn(0x7f27d015dab0 0x7f27d0162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 >> v1:172.21.15.117:6789/0 conn(0x7f27d00b57e0 legacy=0x7f27d00af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0161b20 con 0x7f27d00669a0 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f27c8001a00 con 0x7f27d00669a0 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 >> v1:172.21.15.45:6789/0 conn(0x7f27d00669a0 legacy=0x7f27d0066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8003a10 0x7f27b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0161840 con 0x7f27b8005220 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b800c170 con 0x7f27b8003a10 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b800c370 con 0x7f27b8004810 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d5400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27d4a00640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8003a10 0x7f27b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.018+0000 7f27cfe00640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 -- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b8004810 msgr2=0x7f27b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27b8004810 0x7f27b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 -- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8003a10 msgr2=0x7f27b8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8003a10 0x7f27b8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 -- 172.21.15.45:0/823478563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8003530 con 0x7f27b8005220 2024-04-25T10:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 -- 172.21.15.45:0/823478563 reap_dead start 2024-04-25T10:22:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d5400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f27c800a740 tx=0x7f27c801f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3cc5e300d7c1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c800a390 con 0x7f27b8005220 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c80ac650 con 0x7f27b8005220 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27cf400640 1 -- 172.21.15.45:0/823478563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80aca40 con 0x7f27b8005220 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 -- 172.21.15.45:0/823478563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b8005220 msgr2=0x7f27b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f27c800a740 tx=0x7f27c801f9a0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 -- 172.21.15.45:0/823478563 shutdown_connections 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 --2- 172.21.15.45:0/823478563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27b8005220 0x7f27b800b9d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 -- 172.21.15.45:0/823478563 >> 172.21.15.45:0/823478563 conn(0x7f27d006ac90 msgr2=0x7f27d006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 -- 172.21.15.45:0/823478563 shutdown_connections 2024-04-25T10:22:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.022+0000 7f27d7400640 1 -- 172.21.15.45:0/823478563 wait complete. 2024-04-25T10:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 Processor -- start 2024-04-25T10:22:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 -- start start 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8005220 0x7f27d00b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 0x7f27d0147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d015a7c0 0x7f27d00b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01624a0 con 0x7f27d00b1b30 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0162320 con 0x7f27b8005220 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0162620 con 0x7f27d015a7c0 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 0x7f27d0147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 0x7f27d0147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49308/0 (socket says 172.21.15.45:49308) 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 -- 172.21.15.45:0/1170133155 learned_addr learned my addr 172.21.15.45:0/1170133155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d4a00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8005220 0x7f27d00b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d5400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d015a7c0 0x7f27d00b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 -- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d015a7c0 msgr2=0x7f27d00b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d015a7c0 0x7f27d00b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 -- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8005220 msgr2=0x7f27d00b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8005220 0x7f27d00b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 -- 172.21.15.45:0/1170133155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00b0a90 con 0x7f27d00b1b30 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27cfe00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 0x7f27d0147420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00f650 tx=0x7f27bc00fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0052e0 con 0x7f27d00b1b30 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc018070 con 0x7f27d00b1b30 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc003820 con 0x7f27d00b1b30 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d00b0d70 con 0x7f27d00b1b30 2024-04-25T10:22:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.026+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d00b1330 con 0x7f27d00b1b30 2024-04-25T10:22:12.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.030+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d004ef30 con 0x7f27d00b1b30 2024-04-25T10:22:12.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.030+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27bc0039c0 con 0x7f27d00b1b30 2024-04-25T10:22:12.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.030+0000 7f27ccc00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f27b0066f60 0x7f27b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.030+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f27bc084e20 con 0x7f27d00b1b30 2024-04-25T10:22:12.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.030+0000 7f27d4a00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f27b0066f60 0x7f27b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.030+0000 7f27d4a00640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f27b0066f60 0x7f27b0069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009b60 tx=0x7f27c4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.034+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27bc055d30 con 0x7f27d00b1b30 2024-04-25T10:22:12.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.198+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27d00719b0 con 0x7f27d00b1b30 2024-04-25T10:22:12.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27ccc00640 1 -- 172.21.15.45:0/1170133155 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f27bc005480 con 0x7f27d00b1b30 2024-04-25T10:22:12.205 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:12.205 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:52.261160+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3997755731":"2024-04-26T10:21:52.255019+0000","172.21.15.117:6835/3997755731":"2024-04-26T10:21:52.255019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f27b0066f60 msgr2=0x7f27b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f27b0066f60 0x7f27b0069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009b60 tx=0x7f27c4009190 comp rx=0 tx=0).stop 2024-04-25T10:22:12.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 msgr2=0x7f27d0147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 0x7f27d0147420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00f650 tx=0x7f27bc00fa50 comp rx=0 tx=0).stop 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 shutdown_connections 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f27b0066f60 0x7f27b0069420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d015a7c0 0x7f27d00b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d00b1b30 0x7f27d0147420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 --2- 172.21.15.45:0/1170133155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27b8005220 0x7f27d00b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 >> 172.21.15.45:0/1170133155 conn(0x7f27d006ac90 msgr2=0x7f27d0071e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 shutdown_connections 2024-04-25T10:22:12.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.202+0000 7f27d7400640 1 -- 172.21.15.45:0/1170133155 wait complete. 2024-04-25T10:22:12.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:21:38.360777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:21:39.578182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:12.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:12.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 Processor -- start 2024-04-25T10:22:12.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 -- start start 2024-04-25T10:22:12.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 0x7f8af8161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8af80b5560 0x7f8af80ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af80b40c0 con 0x7f8af8066c00 2024-04-25T10:22:12.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af8161d60 con 0x7f8af80b5560 2024-04-25T10:22:12.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8afd400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80b5b00 con 0x7f8af8066fe0 2024-04-25T10:22:12.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 0x7f8af8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.370+0000 7f8af7200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8af80b5560 0x7f8af80ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8af80b5560 0x7f8af80ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33874/0 (socket says 172.21.15.45:33874) 2024-04-25T10:22:12.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 0x7f8af8161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53796/0 (socket says 172.21.15.45:53796) 2024-04-25T10:22:12.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 -- 172.21.15.45:0/534066408 learned_addr learned my addr 172.21.15.45:0/534066408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:12.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 -- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 msgr2=0x7f8af8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 --2- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 0x7f8af8161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 -- 172.21.15.45:0/534066408 >> v1:172.21.15.117:6789/0 conn(0x7f8af8066c00 legacy=0x7f8af8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 -- 172.21.15.45:0/534066408 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80b3a00 con 0x7f8af80b5560 2024-04-25T10:22:12.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8aefe00640 1 --2- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 0x7f8af8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:12.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af7200640 1 --2- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af80ae850 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009e50 tx=0x7f8af001c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59df4775bc1d6ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af5e00640 1 -- 172.21.15.45:0/534066408 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00a56e0 con 0x7f8af80b5560 2024-04-25T10:22:12.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af5e00640 1 -- 172.21.15.45:0/534066408 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00a5880 con 0x7f8af80b5560 2024-04-25T10:22:12.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.374+0000 7f8af5e00640 1 -- 172.21.15.45:0/534066408 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af00a5c50 con 0x7f8af80b5560 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 msgr2=0x7f8af80ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 --2- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af80ae850 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009e50 tx=0x7f8af001c910 comp rx=0 tx=0).stop 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- 172.21.15.45:0/534066408 shutdown_connections 2024-04-25T10:22:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 --2- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af80ae850 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 --2- 172.21.15.45:0/534066408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066fe0 0x7f8af8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- 172.21.15.45:0/534066408 >> 172.21.15.45:0/534066408 conn(0x7f8af80b2560 msgr2=0x7f8af80b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- 172.21.15.45:0/534066408 shutdown_connections 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- 172.21.15.45:0/534066408 wait complete. 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 Processor -- start 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- start start 2024-04-25T10:22:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066c00 0x7f8af8148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 0x7f8af814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af8155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af815e540 con 0x7f8af8066fe0 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af81585a0 con 0x7f8af80b5560 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8afd400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af815e6c0 con 0x7f8af8066c00 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 0x7f8af814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 0x7f8af814faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49314/0 (socket says 172.21.15.45:49314) 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 -- 172.21.15.45:0/1625112848 learned_addr learned my addr 172.21.15.45:0/1625112848 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8af7200640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af8155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8af6800640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066c00 0x7f8af8148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 -- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066c00 msgr2=0x7f8af8148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066c00 0x7f8af8148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 -- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 msgr2=0x7f8af8155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af8155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 -- 172.21.15.45:0/1625112848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af81fa590 con 0x7f8af8066fe0 2024-04-25T10:22:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.378+0000 7f8aefe00640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 0x7f8af814faa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000ebe0 tx=0x7f8ae000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0017070 con 0x7f8af8066fe0 2024-04-25T10:22:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000cc30 con 0x7f8af8066fe0 2024-04-25T10:22:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af81fa870 con 0x7f8af8066fe0 2024-04-25T10:22:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0013470 con 0x7f8af8066fe0 2024-04-25T10:22:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af81fae30 con 0x7f8af8066fe0 2024-04-25T10:22:12.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af804ef30 con 0x7f8af8066fe0 2024-04-25T10:22:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ae001a070 con 0x7f8af8066fe0 2024-04-25T10:22:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8aeea00640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad4066f90 0x7f8ad4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8ae0085330 con 0x7f8af8066fe0 2024-04-25T10:22:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.382+0000 7f8af6800640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad4066f90 0x7f8ad4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.386+0000 7f8af6800640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad4066f90 0x7f8ad4069450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8001810 tx=0x7f8ae8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.386+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ae0055bf0 con 0x7f8af8066fe0 2024-04-25T10:22:12.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8af8071c20 con 0x7f8af8066fe0 2024-04-25T10:22:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8aeea00640 1 -- 172.21.15.45:0/1625112848 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ae0059c90 con 0x7f8af8066fe0 2024-04-25T10:22:12.553 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad4066f90 msgr2=0x7f8ad4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad4066f90 0x7f8ad4069450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8001810 tx=0x7f8ae8009040 comp rx=0 tx=0).stop 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 msgr2=0x7f8af814faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 0x7f8af814faa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000ebe0 tx=0x7f8ae000c520 comp rx=0 tx=0).stop 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 shutdown_connections 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ad4066f90 0x7f8ad4069450 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8af80b5560 0x7f8af8155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af8066fe0 0x7f8af814faa0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 --2- 172.21.15.45:0/1625112848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8af8066c00 0x7f8af8148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 >> 172.21.15.45:0/1625112848 conn(0x7f8af80b2560 msgr2=0x7f8af806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 shutdown_connections 2024-04-25T10:22:12.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.550+0000 7f8afd400640 1 -- 172.21.15.45:0/1625112848 wait complete. 2024-04-25T10:22:12.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f1302000640 1 Processor -- start 2024-04-25T10:22:12.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f1302000640 1 -- start start 2024-04-25T10:22:12.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f1302000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12fc0721f0 0x7f12fc166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f1302000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc071360 con 0x7f12fc0aeda0 2024-04-25T10:22:12.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f1302000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc159920 con 0x7f12fc159540 2024-04-25T10:22:12.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f1302000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc15c2f0 con 0x7f12fc0721f0 2024-04-25T10:22:12.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12f2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f12fc0aeda0 0x7f12fc071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33978/0 (socket says 172.21.15.45:33978) 2024-04-25T10:22:12.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12f2a00640 1 -- 172.21.15.45:0/1020281107 learned_addr learned my addr 172.21.15.45:0/1020281107 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:12.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12fbe00640 1 --2- 172.21.15.45:0/1020281107 >> v2:172.21.15.117:3300/0 conn(0x7f12fc0721f0 0x7f12fc166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12faa00640 1 -- 172.21.15.45:0/1020281107 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787703556 0 0) 0x7f12fc071360 con 0x7f12fc0aeda0 2024-04-25T10:22:12.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12faa00640 1 -- 172.21.15.45:0/1020281107 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e0003530 con 0x7f12fc0aeda0 2024-04-25T10:22:12.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12fbe00640 1 -- 172.21.15.45:0/1020281107 >> v1:172.21.15.117:6789/0 conn(0x7f12fc159540 legacy=0x7f12fc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12fbe00640 1 -- 172.21.15.45:0/1020281107 >> v1:172.21.15.45:6789/0 conn(0x7f12fc0aeda0 legacy=0x7f12fc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.714+0000 7f12fbe00640 1 -- 172.21.15.45:0/1020281107 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc15c5d0 con 0x7f12fc0721f0 2024-04-25T10:22:12.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f12fbe00640 1 --2- 172.21.15.45:0/1020281107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0721f0 0x7f12fc166920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009d90 tx=0x7f12f401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faabb48353054bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f12faa00640 1 -- 172.21.15.45:0/1020281107 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40b2070 con 0x7f12fc0721f0 2024-04-25T10:22:12.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f12faa00640 1 -- 172.21.15.45:0/1020281107 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f40a5a00 con 0x7f12fc0721f0 2024-04-25T10:22:12.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f12faa00640 1 -- 172.21.15.45:0/1020281107 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40a5dd0 con 0x7f12fc0721f0 2024-04-25T10:22:12.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 -- 172.21.15.45:0/1020281107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0721f0 msgr2=0x7f12fc166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 --2- 172.21.15.45:0/1020281107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0721f0 0x7f12fc166920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009d90 tx=0x7f12f401c940 comp rx=0 tx=0).stop 2024-04-25T10:22:12.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 -- 172.21.15.45:0/1020281107 shutdown_connections 2024-04-25T10:22:12.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 --2- 172.21.15.45:0/1020281107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0721f0 0x7f12fc166920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 -- 172.21.15.45:0/1020281107 >> 172.21.15.45:0/1020281107 conn(0x7f12fc0669a0 msgr2=0x7f12fc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 -- 172.21.15.45:0/1020281107 shutdown_connections 2024-04-25T10:22:12.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.718+0000 7f1302000640 1 -- 172.21.15.45:0/1020281107 wait complete. 2024-04-25T10:22:12.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 Processor -- start 2024-04-25T10:22:12.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- start start 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 0x7f12fc150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12fc0aeda0 0x7f12fc152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc159540 0x7f12fc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc169680 con 0x7f12fc0721f0 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc06c0d0 con 0x7f12fc159540 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc169800 con 0x7f12fc0aeda0 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 0x7f12fc150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 0x7f12fc150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49322/0 (socket says 172.21.15.45:49322) 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 -- 172.21.15.45:0/3926963445 learned_addr learned my addr 172.21.15.45:0/3926963445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:12.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fbe00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc159540 0x7f12fc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12f2a00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12fc0aeda0 0x7f12fc152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 -- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12fc0aeda0 msgr2=0x7f12fc152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12fc0aeda0 0x7f12fc152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 -- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc159540 msgr2=0x7f12fc0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc159540 0x7f12fc0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 -- 172.21.15.45:0/3926963445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc1527d0 con 0x7f12fc0721f0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fbe00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc159540 0x7f12fc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12f2a00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12fc0aeda0 0x7f12fc152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12fb400640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 0x7f12fc150440 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c3c0 tx=0x7f12ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec017070 con 0x7f12fc0721f0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec0052d0 con 0x7f12fc0721f0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc1511f0 con 0x7f12fc0721f0 2024-04-25T10:22:12.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec003840 con 0x7f12fc0721f0 2024-04-25T10:22:12.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc1517b0 con 0x7f12fc0721f0 2024-04-25T10:22:12.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.722+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc04ef30 con 0x7f12fc0721f0 2024-04-25T10:22:12.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.726+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12ec01a070 con 0x7f12fc0721f0 2024-04-25T10:22:12.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.726+0000 7f12f3e00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12d8066f60 0x7f12d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.726+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f12ec0844d0 con 0x7f12fc0721f0 2024-04-25T10:22:12.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.726+0000 7f12f2a00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12d8066f60 0x7f12d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.726+0000 7f12f2a00640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12d8066f60 0x7f12d8069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f12fc166df0 tx=0x7f12e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.730+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12ec005480 con 0x7f12fc0721f0 2024-04-25T10:22:12.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.894+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12fc0b4860 con 0x7f12fc0721f0 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.894+0000 7f12f3e00640 1 -- 172.21.15.45:0/3926963445 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12ec055360 con 0x7f12fc0721f0 2024-04-25T10:22:12.898 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12d8066f60 msgr2=0x7f12d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12d8066f60 0x7f12d8069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f12fc166df0 tx=0x7f12e4009040 comp rx=0 tx=0).stop 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 msgr2=0x7f12fc150440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 0x7f12fc150440 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c3c0 tx=0x7f12ec00c880 comp rx=0 tx=0).stop 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 shutdown_connections 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12d8066f60 0x7f12d8069420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc159540 0x7f12fc0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12fc0aeda0 0x7f12fc152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 --2- 172.21.15.45:0/3926963445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12fc0721f0 0x7f12fc150440 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 >> 172.21.15.45:0/3926963445 conn(0x7f12fc0669a0 msgr2=0x7f12fc0ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 shutdown_connections 2024-04-25T10:22:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:12.898+0000 7f1302000640 1 -- 172.21.15.45:0/3926963445 wait complete. 2024-04-25T10:22:12.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:22:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 Processor -- start 2024-04-25T10:22:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 -- start start 2024-04-25T10:22:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec00cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7ec0071bc0 0x7f7ec0072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec00ad840 con 0x7f7ec00b3480 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00d0090 con 0x7f7ec0071bc0 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ec6000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015bb90 con 0x7f7ec015e130 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec00cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec00cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53810/0 (socket says 172.21.15.45:53810) 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebfe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7ec0071bc0 0x7f7ec0072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 -- 172.21.15.45:0/243934064 learned_addr learned my addr 172.21.15.45:0/243934064 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 -- 172.21.15.45:0/243934064 >> v2:172.21.15.117:3300/0 conn(0x7f7ec0071bc0 msgr2=0x7f7ec0072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 --2- 172.21.15.45:0/243934064 >> v2:172.21.15.117:3300/0 conn(0x7f7ec0071bc0 0x7f7ec0072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 -- 172.21.15.45:0/243934064 >> v1:172.21.15.117:6789/0 conn(0x7f7ec00b3480 legacy=0x7f7ec0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.062+0000 7f7ebea00640 1 -- 172.21.15.45:0/243934064 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec015e710 con 0x7f7ec015e130 2024-04-25T10:22:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ebea00640 1 --2- 172.21.15.45:0/243934064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec00cfb50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009870 tx=0x7f7eac01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92ef63bef969c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ebe000640 1 -- 172.21.15.45:0/243934064 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0b2070 con 0x7f7ec015e130 2024-04-25T10:22:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ebe000640 1 -- 172.21.15.45:0/243934064 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0a9070 con 0x7f7ec015e130 2024-04-25T10:22:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ebe000640 1 -- 172.21.15.45:0/243934064 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ad880 con 0x7f7ec015e130 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 -- 172.21.15.45:0/243934064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 msgr2=0x7f7ec00cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 --2- 172.21.15.45:0/243934064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec00cfb50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009870 tx=0x7f7eac01c560 comp rx=0 tx=0).stop 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 -- 172.21.15.45:0/243934064 shutdown_connections 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 --2- 172.21.15.45:0/243934064 >> v2:172.21.15.117:3300/0 conn(0x7f7ec0071bc0 0x7f7ec0072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 --2- 172.21.15.45:0/243934064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec00cfb50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 -- 172.21.15.45:0/243934064 >> 172.21.15.45:0/243934064 conn(0x7f7ec00669a0 msgr2=0x7f7ec0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 -- 172.21.15.45:0/243934064 shutdown_connections 2024-04-25T10:22:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.066+0000 7f7ec6000640 1 -- 172.21.15.45:0/243934064 wait complete. 2024-04-25T10:22:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 Processor -- start 2024-04-25T10:22:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- start start 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ec0071bc0 0x7f7ec00afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 0x7f7ec0153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ae6b0 con 0x7f7ec00b3480 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ae530 con 0x7f7ec0071bc0 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00ae830 con 0x7f7ec015e130 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 0x7f7ec0153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 0x7f7ec0153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49330/0 (socket says 172.21.15.45:49330) 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 -- 172.21.15.45:0/2721846476 learned_addr learned my addr 172.21.15.45:0/2721846476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebfe00640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebf400640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ec0071bc0 0x7f7ec00afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 -- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 msgr2=0x7f7ec015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec015a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 -- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ec0071bc0 msgr2=0x7f7ec00afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ec0071bc0 0x7f7ec00afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 -- 172.21.15.45:0/2721846476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00b02d0 con 0x7f7ec00b3480 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebf400640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ec0071bc0 0x7f7ec00afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ebea00640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 0x7f7ec0153890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7eac01ac10 tx=0x7f7eac019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0bb070 con 0x7f7ec00b3480 2024-04-25T10:22:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0add40 con 0x7f7ec00b3480 2024-04-25T10:22:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0b2040 con 0x7f7ec00b3480 2024-04-25T10:22:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec00b05b0 con 0x7f7ec00b3480 2024-04-25T10:22:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec00b0b70 con 0x7f7ec00b3480 2024-04-25T10:22:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.070+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec004ef30 con 0x7f7ec00b3480 2024-04-25T10:22:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.074+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7eac0ab070 con 0x7f7ec00b3480 2024-04-25T10:22:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.074+0000 7f7eb7400640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ea0066f60 0x7f7ea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.074+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7eac11ddd0 con 0x7f7ec00b3480 2024-04-25T10:22:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.074+0000 7f7ebf400640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ea0066f60 0x7f7ea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.074+0000 7f7ebf400640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ea0066f60 0x7f7ea0069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0168800 tx=0x7f7eb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.078+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7eac016150 con 0x7f7ec00b3480 2024-04-25T10:22:13.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ec0070f70 con 0x7f7ec00b3480 2024-04-25T10:22:13.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7eb7400640 1 -- 172.21.15.45:0/2721846476 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7eac0ee720 con 0x7f7ec00b3480 2024-04-25T10:22:13.246 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ea0066f60 msgr2=0x7f7ea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ea0066f60 0x7f7ea0069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0168800 tx=0x7f7eb0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 msgr2=0x7f7ec0153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 0x7f7ec0153890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7eac01ac10 tx=0x7f7eac019300 comp rx=0 tx=0).stop 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 shutdown_connections 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ea0066f60 0x7f7ea0069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ec015e130 0x7f7ec015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ec00b3480 0x7f7ec0153890 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 --2- 172.21.15.45:0/2721846476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ec0071bc0 0x7f7ec00afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 >> 172.21.15.45:0/2721846476 conn(0x7f7ec00669a0 msgr2=0x7f7ec0154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 shutdown_connections 2024-04-25T10:22:13.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.242+0000 7f7ec6000640 1 -- 172.21.15.45:0/2721846476 wait complete. 2024-04-25T10:22:13.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82e4c00640 1 Processor -- start 2024-04-25T10:22:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82e4c00640 1 -- start start 2024-04-25T10:22:13.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82e4c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82e00b15b0 0x7f82e0168990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82e4c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00b1af0 con 0x7f82e00d1820 2024-04-25T10:22:13.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82e4c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00b1dd0 con 0x7f82e015e050 2024-04-25T10:22:13.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82e4c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e00b20b0 con 0x7f82e00b15b0 2024-04-25T10:22:13.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82dde00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f82e00d1820 0x7f82e00d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33982/0 (socket says 172.21.15.45:33982) 2024-04-25T10:22:13.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82dde00640 1 -- 172.21.15.45:0/3925223700 learned_addr learned my addr 172.21.15.45:0/3925223700 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:13.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82de800640 1 --2- 172.21.15.45:0/3925223700 >> v2:172.21.15.45:3300/0 conn(0x7f82e00b15b0 0x7f82e0168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82dd400640 1 -- 172.21.15.45:0/3925223700 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513649163 0 0) 0x7f82e00b1af0 con 0x7f82e00d1820 2024-04-25T10:22:13.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82dd400640 1 -- 172.21.15.45:0/3925223700 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c4003530 con 0x7f82e00d1820 2024-04-25T10:22:13.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82de800640 1 -- 172.21.15.45:0/3925223700 >> v1:172.21.15.117:6789/0 conn(0x7f82e015e050 legacy=0x7f82e00b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82de800640 1 -- 172.21.15.45:0/3925223700 >> v1:172.21.15.45:6789/0 conn(0x7f82e00d1820 legacy=0x7f82e00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.406+0000 7f82de800640 1 -- 172.21.15.45:0/3925223700 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00b2390 con 0x7f82e00b15b0 2024-04-25T10:22:13.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82de800640 1 --2- 172.21.15.45:0/3925223700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e0168990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009d90 tx=0x7f82d801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4224d1f1f63edec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82dd400640 1 -- 172.21.15.45:0/3925223700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80b2070 con 0x7f82e00b15b0 2024-04-25T10:22:13.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82dd400640 1 -- 172.21.15.45:0/3925223700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d80a58b0 con 0x7f82e00b15b0 2024-04-25T10:22:13.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82dd400640 1 -- 172.21.15.45:0/3925223700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80a5c60 con 0x7f82e00b15b0 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 -- 172.21.15.45:0/3925223700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 msgr2=0x7f82e0168990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 --2- 172.21.15.45:0/3925223700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e0168990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f82d8009d90 tx=0x7f82d801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 -- 172.21.15.45:0/3925223700 shutdown_connections 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 --2- 172.21.15.45:0/3925223700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e0168990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 -- 172.21.15.45:0/3925223700 >> 172.21.15.45:0/3925223700 conn(0x7f82e009a9d0 msgr2=0x7f82e009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 -- 172.21.15.45:0/3925223700 shutdown_connections 2024-04-25T10:22:13.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.410+0000 7f82e4c00640 1 -- 172.21.15.45:0/3925223700 wait complete. 2024-04-25T10:22:13.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 Processor -- start 2024-04-25T10:22:13.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- start start 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82e00d1820 0x7f82e015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82e015e050 0x7f82e0158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e016bda0 con 0x7f82e00b15b0 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e016bc20 con 0x7f82e015e050 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e016bf20 con 0x7f82e00d1820 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e015bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49354/0 (socket says 172.21.15.45:49354) 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 -- 172.21.15.45:0/2704249878 learned_addr learned my addr 172.21.15.45:0/2704249878 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:13.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82de800640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82e015e050 0x7f82e0158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82d5400640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82e00d1820 0x7f82e015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 -- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82e00d1820 msgr2=0x7f82e015f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82e00d1820 0x7f82e015f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 -- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82e015e050 msgr2=0x7f82e0158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82e015e050 0x7f82e0158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 -- 172.21.15.45:0/2704249878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e0160d90 con 0x7f82e00b15b0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82d5400640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82e00d1820 0x7f82e015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82dde00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e015bbc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c420 tx=0x7f82d000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0017070 con 0x7f82e00b15b0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0005cf0 con 0x7f82e00b15b0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00072d0 con 0x7f82e00b15b0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e015fcf0 con 0x7f82e00b15b0 2024-04-25T10:22:13.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e01602b0 con 0x7f82e00b15b0 2024-04-25T10:22:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.414+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e004ef30 con 0x7f82e00b15b0 2024-04-25T10:22:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.418+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82d001a070 con 0x7f82e00b15b0 2024-04-25T10:22:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.418+0000 7f82d6a00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82bc066f90 0x7f82bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.418+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f82d00843c0 con 0x7f82e00b15b0 2024-04-25T10:22:13.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.418+0000 7f82d5400640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82bc066f90 0x7f82bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.418+0000 7f82d5400640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82bc066f90 0x7f82bc069450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f82e006f070 tx=0x7f82c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.422+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82d0054c80 con 0x7f82e00b15b0 2024-04-25T10:22:13.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.614+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82e006b480 con 0x7f82e00b15b0 2024-04-25T10:22:13.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.614+0000 7f82d6a00640 1 -- 172.21.15.45:0/2704249878 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f82d0058d20 con 0x7f82e00b15b0 2024-04-25T10:22:13.619 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:13.619 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:55.281089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:22:13.619 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82bc066f90 msgr2=0x7f82bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82bc066f90 0x7f82bc069450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f82e006f070 tx=0x7f82c8009040 comp rx=0 tx=0).stop 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 msgr2=0x7f82e015bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e015bbc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c420 tx=0x7f82d000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 shutdown_connections 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f82bc066f90 0x7f82bc069450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82e015e050 0x7f82e0158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82e00d1820 0x7f82e015f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 --2- 172.21.15.45:0/2704249878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e00b15b0 0x7f82e015bbc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 >> 172.21.15.45:0/2704249878 conn(0x7f82e009a9d0 msgr2=0x7f82e006f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 shutdown_connections 2024-04-25T10:22:13.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.618+0000 7f82e4c00640 1 -- 172.21.15.45:0/2704249878 wait complete. 2024-04-25T10:22:13.640 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:13.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 Processor -- start 2024-04-25T10:22:13.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 -- start start 2024-04-25T10:22:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f818c1550b0 0x7f818c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f818c15b740 0x7f818c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c071050 con 0x7f818c1550b0 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c0712f0 con 0x7f818c15b740 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8193c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0715d0 con 0x7f818c0cfe70 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f818c1550b0 0x7f818c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f818c1550b0 0x7f818c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49356/0 (socket says 172.21.15.45:49356) 2024-04-25T10:22:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 -- 172.21.15.45:0/4257104681 learned_addr learned my addr 172.21.15.45:0/4257104681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 -- 172.21.15.45:0/4257104681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 msgr2=0x7f818c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 --2- 172.21.15.45:0/4257104681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 -- 172.21.15.45:0/4257104681 >> v2:172.21.15.117:3300/0 conn(0x7f818c15b740 msgr2=0x7f818c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191c00640 1 --2- 172.21.15.45:0/4257104681 >> v2:172.21.15.117:3300/0 conn(0x7f818c15b740 0x7f818c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 --2- 172.21.15.45:0/4257104681 >> v2:172.21.15.117:3300/0 conn(0x7f818c15b740 0x7f818c06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191200640 1 -- 172.21.15.45:0/4257104681 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0718b0 con 0x7f818c1550b0 2024-04-25T10:22:13.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.778+0000 7f8191c00640 1 --2- 172.21.15.45:0/4257104681 >> v2:172.21.15.117:3300/0 conn(0x7f818c15b740 0x7f818c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8191200640 1 --2- 172.21.15.45:0/4257104681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c0cf930 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f818c0d04d0 tx=0x7f817c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711df45c06b572de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f818be00640 1 -- 172.21.15.45:0/4257104681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0b0070 con 0x7f818c1550b0 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f818be00640 1 -- 172.21.15.45:0/4257104681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c0ab600 con 0x7f818c1550b0 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f818be00640 1 -- 172.21.15.45:0/4257104681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0ab9d0 con 0x7f818c1550b0 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 -- 172.21.15.45:0/4257104681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 msgr2=0x7f818c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 --2- 172.21.15.45:0/4257104681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c0cf930 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f818c0d04d0 tx=0x7f817c01c260 comp rx=0 tx=0).stop 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 -- 172.21.15.45:0/4257104681 shutdown_connections 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 --2- 172.21.15.45:0/4257104681 >> v2:172.21.15.117:3300/0 conn(0x7f818c15b740 0x7f818c06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 --2- 172.21.15.45:0/4257104681 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 --2- 172.21.15.45:0/4257104681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c0cf930 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 -- 172.21.15.45:0/4257104681 >> 172.21.15.45:0/4257104681 conn(0x7f818c066b70 msgr2=0x7f818c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 -- 172.21.15.45:0/4257104681 shutdown_connections 2024-04-25T10:22:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.782+0000 7f8193c00640 1 -- 172.21.15.45:0/4257104681 wait complete. 2024-04-25T10:22:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 Processor -- start 2024-04-25T10:22:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 -- start start 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c1400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f818c15b740 0x7f818c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b3260 con 0x7f818c1550b0 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c15bd60 con 0x7f818c15b740 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b33e0 con 0x7f818c0cfe70 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c1400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c1400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49364/0 (socket says 172.21.15.45:49364) 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 -- 172.21.15.45:0/177313133 learned_addr learned my addr 172.21.15.45:0/177313133 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8191200640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8191c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f818c15b740 0x7f818c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 -- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 msgr2=0x7f818c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 -- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f818c15b740 msgr2=0x7f818c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f818c15b740 0x7f818c1572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 -- 172.21.15.45:0/177313133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c157b50 con 0x7f818c1550b0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8183e00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c1400b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f817400b550 tx=0x7f817400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174014070 con 0x7f818c1550b0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174010890 con 0x7f818c1550b0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174010c80 con 0x7f818c1550b0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c157e30 con 0x7f818c1550b0 2024-04-25T10:22:13.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.786+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c157820 con 0x7f818c1550b0 2024-04-25T10:22:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.790+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c158350 con 0x7f818c1550b0 2024-04-25T10:22:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.790+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8174017070 con 0x7f818c1550b0 2024-04-25T10:22:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.790+0000 7f8189600640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8168066f60 0x7f8168069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.790+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8174084d60 con 0x7f818c1550b0 2024-04-25T10:22:13.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.790+0000 7f8191200640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8168066f60 0x7f8168069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.790+0000 7f8191200640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8168066f60 0x7f8168069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f817c0ad000 tx=0x7f817c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.794+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f818c158350 con 0x7f818c1550b0 2024-04-25T10:22:13.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.958+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f818c04ef30 con 0x7f818c1550b0 2024-04-25T10:22:13.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.958+0000 7f8189600640 1 -- 172.21.15.45:0/177313133 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f8174010e20 con 0x7f818c1550b0 2024-04-25T10:22:13.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:13.962 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:52.261160+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3997755731":"2024-04-26T10:21:52.255019+0000","172.21.15.117:6835/3997755731":"2024-04-26T10:21:52.255019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.958+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8168066f60 msgr2=0x7f8168069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.958+0000 7f8193c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8168066f60 0x7f8168069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f817c0ad000 tx=0x7f817c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.958+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 msgr2=0x7f818c1400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.958+0000 7f8193c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c1400b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f817400b550 tx=0x7f817400ba10 comp rx=0 tx=0).stop 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 shutdown_connections 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8168066f60 0x7f8168069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f818c15b740 0x7f818c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818c1550b0 0x7f818c1400b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 --2- 172.21.15.45:0/177313133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f818c0cfe70 0x7f818c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 >> 172.21.15.45:0/177313133 conn(0x7f818c066b70 msgr2=0x7f818c150150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 shutdown_connections 2024-04-25T10:22:13.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:13.962+0000 7f8193c00640 1 -- 172.21.15.45:0/177313133 wait complete. 2024-04-25T10:22:13.983 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:22:13.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:14.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.118+0000 7f68a2200640 1 Processor -- start 2024-04-25T10:22:14.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.118+0000 7f68a2200640 1 -- start start 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f68a2200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f689c1550b0 0x7f689c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f68a2200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c15b910 0x7f689c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f68a2200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c071050 con 0x7f689c0cfe70 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f68a2200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689c15bda0 con 0x7f689c1550b0 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f68a2200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689c15d040 con 0x7f689c15b910 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f6892a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f689c0cfe70 0x7f689c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33990/0 (socket says 172.21.15.45:33990) 2024-04-25T10:22:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f6892a00640 1 -- 172.21.15.45:0/2980967789 learned_addr learned my addr 172.21.15.45:0/2980967789 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689b400640 1 --2- 172.21.15.45:0/2980967789 >> v2:172.21.15.117:3300/0 conn(0x7f689c1550b0 0x7f689c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689be00640 1 --2- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c15b910 0x7f689c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689aa00640 1 -- 172.21.15.45:0/2980967789 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262963857 0 0) 0x7f689c071050 con 0x7f689c0cfe70 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689aa00640 1 -- 172.21.15.45:0/2980967789 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6880003530 con 0x7f689c0cfe70 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689b400640 1 -- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c15b910 msgr2=0x7f689c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689aa00640 1 -- 172.21.15.45:0/2980967789 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3984414417 0 0) 0x7f6880003530 con 0x7f689c0cfe70 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689b400640 1 --2- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c15b910 0x7f689c0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689b400640 1 -- 172.21.15.45:0/2980967789 >> v1:172.21.15.45:6789/0 conn(0x7f689c0cfe70 legacy=0x7f689c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.122+0000 7f689b400640 1 -- 172.21.15.45:0/2980967789 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c0712f0 con 0x7f689c1550b0 2024-04-25T10:22:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f689b400640 1 --2- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c1550b0 0x7f689c0cf930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f688c0097f0 tx=0x7f688c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8437a29130478414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f689aa00640 1 -- 172.21.15.45:0/2980967789 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c0b0070 con 0x7f689c1550b0 2024-04-25T10:22:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f689aa00640 1 -- 172.21.15.45:0/2980967789 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c0ab400 con 0x7f689c1550b0 2024-04-25T10:22:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f689aa00640 1 -- 172.21.15.45:0/2980967789 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884001680 con 0x7f689c1550b0 2024-04-25T10:22:14.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 -- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c1550b0 msgr2=0x7f689c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 --2- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c1550b0 0x7f689c0cf930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f688c0097f0 tx=0x7f688c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 -- 172.21.15.45:0/2980967789 shutdown_connections 2024-04-25T10:22:14.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 --2- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c15b910 0x7f689c0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 --2- 172.21.15.45:0/2980967789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c1550b0 0x7f689c0cf930 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 -- 172.21.15.45:0/2980967789 >> 172.21.15.45:0/2980967789 conn(0x7f689c066b70 msgr2=0x7f689c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 -- 172.21.15.45:0/2980967789 shutdown_connections 2024-04-25T10:22:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 -- 172.21.15.45:0/2980967789 wait complete. 2024-04-25T10:22:14.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.126+0000 7f68a2200640 1 Processor -- start 2024-04-25T10:22:14.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- start start 2024-04-25T10:22:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c0cfe70 0x7f689c140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 0x7f689c143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c15b910 0x7f689c155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c15c030 con 0x7f689c1550b0 2024-04-25T10:22:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0720b0 con 0x7f689c15b910 2024-04-25T10:22:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689c15c1b0 con 0x7f689c0cfe70 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 0x7f689c143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 0x7f689c143980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49378/0 (socket says 172.21.15.45:49378) 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 -- 172.21.15.45:0/320297585 learned_addr learned my addr 172.21.15.45:0/320297585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f689b400640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c0cfe70 0x7f689c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f689be00640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c15b910 0x7f689c155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 -- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c0cfe70 msgr2=0x7f689c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c0cfe70 0x7f689c140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 -- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c15b910 msgr2=0x7f689c155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c15b910 0x7f689c155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 -- 172.21.15.45:0/320297585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c140c40 con 0x7f689c1550b0 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f689b400640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c0cfe70 0x7f689c140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6892a00640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 0x7f689c143980 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6884001390 tx=0x7f688400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884017070 con 0x7f689c1550b0 2024-04-25T10:22:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884005ae0 con 0x7f689c1550b0 2024-04-25T10:22:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884007070 con 0x7f689c1550b0 2024-04-25T10:22:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c143ec0 con 0x7f689c1550b0 2024-04-25T10:22:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c1416b0 con 0x7f689c1550b0 2024-04-25T10:22:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.130+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689c04ef30 con 0x7f689c1550b0 2024-04-25T10:22:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.134+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f688401a070 con 0x7f689c1550b0 2024-04-25T10:22:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.134+0000 7f6893e00640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.134+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f68840845c0 con 0x7f689c1550b0 2024-04-25T10:22:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.134+0000 7f689b400640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.134+0000 7f689b400640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f688c0ad000 tx=0x7f688c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.138+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6884054e80 con 0x7f689c1550b0 2024-04-25T10:22:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f689c0cbc30 con 0x7f689c1550b0 2024-04-25T10:22:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f6893e00640 1 -- 172.21.15.45:0/320297585 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f6884058f20 con 0x7f689c1550b0 2024-04-25T10:22:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:14.334 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:55.281089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:22:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6878066f60 msgr2=0x7f6878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f688c0ad000 tx=0x7f688c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 msgr2=0x7f689c143980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 0x7f689c143980 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6884001390 tx=0x7f688400cbe0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 shutdown_connections 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f689c15b910 0x7f689c155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f689c1550b0 0x7f689c143980 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 --2- 172.21.15.45:0/320297585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f689c0cfe70 0x7f689c140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.330+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 >> 172.21.15.45:0/320297585 conn(0x7f689c066b70 msgr2=0x7f689c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.334+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 shutdown_connections 2024-04-25T10:22:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.334+0000 7f68a2200640 1 -- 172.21.15.45:0/320297585 wait complete. 2024-04-25T10:22:14.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:14.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 Processor -- start 2024-04-25T10:22:14.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 -- start start 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600d0040 0x7ffb6015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb6015b740 0x7ffb600b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb60071050 con 0x7ffb601550b0 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6015d040 con 0x7ffb6015b740 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb65000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600712f0 con 0x7ffb600d0040 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb6015b740 0x7ffb600b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb6015b740 0x7ffb600b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49390/0 (socket says 172.21.15.45:49390) 2024-04-25T10:22:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 -- 172.21.15.45:0/4094567481 learned_addr learned my addr 172.21.15.45:0/4094567481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb57e00640 1 --2- 172.21.15.45:0/4094567481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600d0040 0x7ffb6015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 -- 172.21.15.45:0/4094567481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600d0040 msgr2=0x7ffb6015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 --2- 172.21.15.45:0/4094567481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600d0040 0x7ffb6015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 -- 172.21.15.45:0/4094567481 >> v1:172.21.15.117:6789/0 conn(0x7ffb601550b0 legacy=0x7ffb600cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.494+0000 7ffb5e800640 1 -- 172.21.15.45:0/4094567481 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600715d0 con 0x7ffb6015b740 2024-04-25T10:22:14.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb5e800640 1 --2- 172.21.15.45:0/4094567481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb600b3260 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009e50 tx=0x7ffb5801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f870e8eb97dc454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb5d400640 1 -- 172.21.15.45:0/4094567481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb580a58d0 con 0x7ffb6015b740 2024-04-25T10:22:14.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb5d400640 1 -- 172.21.15.45:0/4094567481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb580a5a70 con 0x7ffb6015b740 2024-04-25T10:22:14.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb5d400640 1 -- 172.21.15.45:0/4094567481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb580a5e40 con 0x7ffb6015b740 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 -- 172.21.15.45:0/4094567481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 msgr2=0x7ffb600b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 --2- 172.21.15.45:0/4094567481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb600b3260 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009e50 tx=0x7ffb5801ca00 comp rx=0 tx=0).stop 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 -- 172.21.15.45:0/4094567481 shutdown_connections 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 --2- 172.21.15.45:0/4094567481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb600b3260 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 --2- 172.21.15.45:0/4094567481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600d0040 0x7ffb6015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 -- 172.21.15.45:0/4094567481 >> 172.21.15.45:0/4094567481 conn(0x7ffb60066b70 msgr2=0x7ffb60066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 -- 172.21.15.45:0/4094567481 shutdown_connections 2024-04-25T10:22:14.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.498+0000 7ffb65000640 1 -- 172.21.15.45:0/4094567481 wait complete. 2024-04-25T10:22:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 Processor -- start 2024-04-25T10:22:14.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 -- start start 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600d0040 0x7ffb60156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb601550b0 0x7ffb601431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb60155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015bd70 con 0x7ffb6015b740 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015bbf0 con 0x7ffb600d0040 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015c030 con 0x7ffb601550b0 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb60155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb60155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49402/0 (socket says 172.21.15.45:49402) 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 -- 172.21.15.45:0/2702750182 learned_addr learned my addr 172.21.15.45:0/2702750182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb57e00640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb601550b0 0x7ffb601431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5de00640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600d0040 0x7ffb60156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 -- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb601550b0 msgr2=0x7ffb601431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb601550b0 0x7ffb601431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 -- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600d0040 msgr2=0x7ffb60156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600d0040 0x7ffb60156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 -- 172.21.15.45:0/2702750182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60155f50 con 0x7ffb6015b740 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb5e800640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb60155490 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009e50 tx=0x7ffb580197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb580a58d0 con 0x7ffb6015b740 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb580be070 con 0x7ffb6015b740 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58009510 con 0x7ffb6015b740 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb60156230 con 0x7ffb6015b740 2024-04-25T10:22:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.502+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601567f0 con 0x7ffb6015b740 2024-04-25T10:22:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.506+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6004ef30 con 0x7ffb6015b740 2024-04-25T10:22:14.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.506+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb580b86b0 con 0x7ffb6015b740 2024-04-25T10:22:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.506+0000 7ffb56000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb3c066f60 0x7ffb3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.506+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffb581206b0 con 0x7ffb6015b740 2024-04-25T10:22:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.506+0000 7ffb5de00640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb3c066f60 0x7ffb3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.506+0000 7ffb5de00640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb3c066f60 0x7ffb3c069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001810 tx=0x7ffb50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.510+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb58016620 con 0x7ffb6015b740 2024-04-25T10:22:14.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb6015b290 con 0x7ffb6015b740 2024-04-25T10:22:14.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb56000640 1 -- 172.21.15.45:0/2702750182 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4945 (secure 0 0 0) 0x7ffb580f0ee0 con 0x7ffb6015b740 2024-04-25T10:22:14.709 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:14.709 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:21:40.567701+0000","modified":"2024-04-25T10:21:55.281089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6349},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6349":{"gid":6349,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/4277203907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4277203907},{"type":"v1","addr":"172.21.15.117:6837","nonce":4277203907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:22:14.709 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 46 2024-04-25T10:22:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb3c066f60 msgr2=0x7ffb3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb3c066f60 0x7ffb3c069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001810 tx=0x7ffb50009040 comp rx=0 tx=0).stop 2024-04-25T10:22:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 msgr2=0x7ffb60155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb60155490 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ffb58009e50 tx=0x7ffb580197a0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 shutdown_connections 2024-04-25T10:22:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb3c066f60 0x7ffb3c069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb6015b740 0x7ffb60155490 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb601550b0 0x7ffb601431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 --2- 172.21.15.45:0/2702750182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600d0040 0x7ffb60156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 >> 172.21.15.45:0/2702750182 conn(0x7ffb60066b70 msgr2=0x7ffb6015dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 shutdown_connections 2024-04-25T10:22:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.706+0000 7ffb65000640 1 -- 172.21.15.45:0/2702750182 wait complete. 2024-04-25T10:22:14.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.866+0000 7f22dec00640 1 Processor -- start 2024-04-25T10:22:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.866+0000 7f22dec00640 1 -- start start 2024-04-25T10:22:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8066c00 0x7f22d815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dec00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22d80b5560 0x7f22d8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dec00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80b5c20 con 0x7f22d8066fe0 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dec00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80b59d0 con 0x7f22d80b5560 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80af6e0 con 0x7f22d8066c00 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22d80b5560 0x7f22d8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22d80b5560 0x7f22d8157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49410/0 (socket says 172.21.15.45:49410) 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 -- 172.21.15.45:0/1391464842 learned_addr learned my addr 172.21.15.45:0/1391464842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22d7e00640 1 --2- 172.21.15.45:0/1391464842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8066c00 0x7f22d815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 -- 172.21.15.45:0/1391464842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8066c00 msgr2=0x7f22d815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 --2- 172.21.15.45:0/1391464842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8066c00 0x7f22d815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 -- 172.21.15.45:0/1391464842 >> v1:172.21.15.117:6789/0 conn(0x7f22d8066fe0 legacy=0x7f22d806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.870+0000 7f22dcc00640 1 -- 172.21.15.45:0/1391464842 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80af9c0 con 0x7f22d80b5560 2024-04-25T10:22:14.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dcc00640 1 --2- 172.21.15.45:0/1391464842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d80b5560 0x7f22d8157b50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009e50 tx=0x7f22d001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70abda82827fad0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22d7400640 1 -- 172.21.15.45:0/1391464842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00a58d0 con 0x7f22d80b5560 2024-04-25T10:22:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22d7400640 1 -- 172.21.15.45:0/1391464842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d00a5a70 con 0x7f22d80b5560 2024-04-25T10:22:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22d7400640 1 -- 172.21.15.45:0/1391464842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d00a5e40 con 0x7f22d80b5560 2024-04-25T10:22:14.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 -- 172.21.15.45:0/1391464842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d80b5560 msgr2=0x7f22d8157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 --2- 172.21.15.45:0/1391464842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d80b5560 0x7f22d8157b50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009e50 tx=0x7f22d001ca00 comp rx=0 tx=0).stop 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 -- 172.21.15.45:0/1391464842 shutdown_connections 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 --2- 172.21.15.45:0/1391464842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d80b5560 0x7f22d8157b50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 --2- 172.21.15.45:0/1391464842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8066c00 0x7f22d815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 -- 172.21.15.45:0/1391464842 >> 172.21.15.45:0/1391464842 conn(0x7f22d80b24f0 msgr2=0x7f22d80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 -- 172.21.15.45:0/1391464842 shutdown_connections 2024-04-25T10:22:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 -- 172.21.15.45:0/1391464842 wait complete. 2024-04-25T10:22:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 Processor -- start 2024-04-25T10:22:14.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.874+0000 7f22dec00640 1 -- start start 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 0x7f22d814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8066fe0 0x7f22d8144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d80b5560 0x7f22d81509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80af1e0 con 0x7f22d8066c00 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80af060 con 0x7f22d8066fe0 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d806ac90 con 0x7f22d80b5560 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 0x7f22d814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 0x7f22d814e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49026/0 (socket says 172.21.15.45:49026) 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 -- 172.21.15.45:0/2041854220 learned_addr learned my addr 172.21.15.45:0/2041854220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22cfe00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8066fe0 0x7f22d8144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dcc00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d80b5560 0x7f22d81509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 -- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d80b5560 msgr2=0x7f22d81509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d80b5560 0x7f22d81509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 -- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8066fe0 msgr2=0x7f22d8144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8066fe0 0x7f22d8144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 -- 172.21.15.45:0/2041854220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d814f560 con 0x7f22d8066c00 2024-04-25T10:22:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22cfe00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8066fe0 0x7f22d8144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:14.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d7e00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 0x7f22d814e6d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f22c800d830 tx=0x7f22c800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c800b6c0 con 0x7f22d8066c00 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8019070 con 0x7f22d8066c00 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c800bae0 con 0x7f22d8066c00 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d814f840 con 0x7f22d8066c00 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d814fe00 con 0x7f22d8066c00 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d804ef30 con 0x7f22d8066c00 2024-04-25T10:22:14.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.878+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22c801e460 con 0x7f22d8066c00 2024-04-25T10:22:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.882+0000 7f22d4c00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b4066f60 0x7f22b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.882+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f22c80857c0 con 0x7f22d8066c00 2024-04-25T10:22:14.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.882+0000 7f22cfe00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b4066f60 0x7f22b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.882+0000 7f22cfe00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b4066f60 0x7f22b4069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f22c0001710 tx=0x7f22c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:14.886+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22c80566d0 con 0x7f22d8066c00 2024-04-25T10:22:15.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.046+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22d81501a0 con 0x7f22d8066c00 2024-04-25T10:22:15.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22d4c00640 1 -- 172.21.15.45:0/2041854220 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f22c8055fd0 con 0x7f22d8066c00 2024-04-25T10:22:15.053 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:21:52.261160+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:21:38.360777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T10:21:39.578182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3997755731":"2024-04-26T10:21:52.255019+0000","172.21.15.117:6835/3997755731":"2024-04-26T10:21:52.255019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b4066f60 msgr2=0x7f22b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b4066f60 0x7f22b4069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f22c0001710 tx=0x7f22c0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 msgr2=0x7f22d814e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 0x7f22d814e6d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f22c800d830 tx=0x7f22c800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 shutdown_connections 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b4066f60 0x7f22b4069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d80b5560 0x7f22d81509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8066fe0 0x7f22d8144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 --2- 172.21.15.45:0/2041854220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d8066c00 0x7f22d814e6d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 >> 172.21.15.45:0/2041854220 conn(0x7f22d80b24f0 msgr2=0x7f22d806de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 shutdown_connections 2024-04-25T10:22:15.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.050+0000 7f22dec00640 1 -- 172.21.15.45:0/2041854220 wait complete. 2024-04-25T10:22:15.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:22:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 Processor -- start 2024-04-25T10:22:15.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 -- start start 2024-04-25T10:22:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1c0072580 0x7fa1c015b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 0x7fa1c0070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c00b3400 con 0x7fa1c00b44a0 2024-04-25T10:22:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c0072960 con 0x7fa1c0072580 2024-04-25T10:22:15.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00b36e0 con 0x7fa1c006e670 2024-04-25T10:22:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 0x7fa1c0070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 0x7fa1c0070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:39752/0 (socket says 172.21.15.45:39752) 2024-04-25T10:22:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 -- 172.21.15.45:0/2515319943 learned_addr learned my addr 172.21.15.45:0/2515319943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bea00640 1 --2- 172.21.15.45:0/2515319943 >> v2:172.21.15.117:3300/0 conn(0x7fa1c0072580 0x7fa1c015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 -- 172.21.15.45:0/2515319943 >> v2:172.21.15.117:3300/0 conn(0x7fa1c0072580 msgr2=0x7fa1c015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 --2- 172.21.15.45:0/2515319943 >> v2:172.21.15.117:3300/0 conn(0x7fa1c0072580 0x7fa1c015b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 -- 172.21.15.45:0/2515319943 >> v1:172.21.15.117:6789/0 conn(0x7fa1c00b44a0 legacy=0x7fa1c006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.214+0000 7fa1bf400640 1 -- 172.21.15.45:0/2515319943 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00b39c0 con 0x7fa1c006e670 2024-04-25T10:22:15.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bf400640 1 --2- 172.21.15.45:0/2515319943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 0x7fa1c0070ac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009e50 tx=0x7fa1b801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22a9e002f2e5f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1be000640 1 -- 172.21.15.45:0/2515319943 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80a5810 con 0x7fa1c006e670 2024-04-25T10:22:15.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1be000640 1 -- 172.21.15.45:0/2515319943 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b80a59b0 con 0x7fa1c006e670 2024-04-25T10:22:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1be000640 1 -- 172.21.15.45:0/2515319943 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80a5d80 con 0x7fa1c006e670 2024-04-25T10:22:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 -- 172.21.15.45:0/2515319943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 msgr2=0x7fa1c0070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/2515319943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 0x7fa1c0070ac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009e50 tx=0x7fa1b801ca30 comp rx=0 tx=0).stop 2024-04-25T10:22:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 -- 172.21.15.45:0/2515319943 shutdown_connections 2024-04-25T10:22:15.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/2515319943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c006e670 0x7fa1c0070ac0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/2515319943 >> v2:172.21.15.117:3300/0 conn(0x7fa1c0072580 0x7fa1c015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 -- 172.21.15.45:0/2515319943 >> 172.21.15.45:0/2515319943 conn(0x7fa1c006b4e0 msgr2=0x7fa1c00b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 -- 172.21.15.45:0/2515319943 shutdown_connections 2024-04-25T10:22:15.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.218+0000 7fa1bfe00640 1 -- 172.21.15.45:0/2515319943 wait complete. 2024-04-25T10:22:15.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 Processor -- start 2024-04-25T10:22:15.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 -- start start 2024-04-25T10:22:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1c006e670 0x7fa1c0148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c0072580 0x7fa1c0167f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c00b44a0 0x7fa1c0154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c01659f0 con 0x7fa1c00b44a0 2024-04-25T10:22:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c0165870 con 0x7fa1c006e670 2024-04-25T10:22:15.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c0165b70 con 0x7fa1c0072580 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c00b44a0 0x7fa1c0154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c0072580 0x7fa1c0167f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c0072580 0x7fa1c0167f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:39758/0 (socket says 172.21.15.45:39758) 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1b7e00640 1 -- 172.21.15.45:0/1427798229 learned_addr learned my addr 172.21.15.45:0/1427798229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1c006e670 0x7fa1c0148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 -- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c0072580 msgr2=0x7fa1c0167f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c0072580 0x7fa1c0167f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 -- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1c006e670 msgr2=0x7fa1c0148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1c006e670 0x7fa1c0148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 -- 172.21.15.45:0/1427798229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c01523d0 con 0x7fa1c00b44a0 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bf400640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c00b44a0 0x7fa1c0154450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80ab000 tx=0x7fa1b80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8009510 con 0x7fa1c00b44a0 2024-04-25T10:22:15.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b80b8680 con 0x7fa1c00b44a0 2024-04-25T10:22:15.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c0152670 con 0x7fa1c00b44a0 2024-04-25T10:22:15.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b80b8a50 con 0x7fa1c00b44a0 2024-04-25T10:22:15.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.222+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c0152c30 con 0x7fa1c00b44a0 2024-04-25T10:22:15.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.226+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c0075120 con 0x7fa1c00b44a0 2024-04-25T10:22:15.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.226+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1b80c0020 con 0x7fa1c00b44a0 2024-04-25T10:22:15.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.226+0000 7fa1b6a00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa19c066f60 0x7fa19c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.226+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa1b811fbe0 con 0x7fa1c00b44a0 2024-04-25T10:22:15.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.226+0000 7fa1bea00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa19c066f60 0x7fa19c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.226+0000 7fa1bea00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa19c066f60 0x7fa19c069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1c014fd70 tx=0x7fa1b0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.230+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1b8016620 con 0x7fa1c00b44a0 2024-04-25T10:22:15.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.430+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa1c006ee10 con 0x7fa1c00b44a0 2024-04-25T10:22:15.435 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:22:15.430+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:22:15.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.434+0000 7fa1b6a00640 1 -- 172.21.15.45:0/1427798229 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa1b80a5980 con 0x7fa1c00b44a0 2024-04-25T10:22:15.438 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa19c066f60 msgr2=0x7fa19c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa19c066f60 0x7fa19c069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1c014fd70 tx=0x7fa1b0009190 comp rx=0 tx=0).stop 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c00b44a0 msgr2=0x7fa1c0154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c00b44a0 0x7fa1c0154450 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80ab000 tx=0x7fa1b80197d0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 shutdown_connections 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa19c066f60 0x7fa19c069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c00b44a0 0x7fa1c0154450 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1c0072580 0x7fa1c0167f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 --2- 172.21.15.45:0/1427798229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1c006e670 0x7fa1c0148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 >> 172.21.15.45:0/1427798229 conn(0x7fa1c006b4e0 msgr2=0x7fa1c00d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 shutdown_connections 2024-04-25T10:22:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.438+0000 7fa1bfe00640 1 -- 172.21.15.45:0/1427798229 wait complete. 2024-04-25T10:22:15.451 INFO:tasks.ceph.mds.d.smithi117.stderr: -1879> 2024-04-25T10:21:59.265+0000 7fcf88600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:22:15.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:22:15.496 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:22:15.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.598+0000 7f962b000640 1 Processor -- start 2024-04-25T10:22:15.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 -- start start 2024-04-25T10:22:15.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f962406cf40 0x7f962415e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f962406db90 0x7f962406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f962415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962415de80 con 0x7f962406db90 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962415e440 con 0x7f962406cf40 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f962b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962415ee10 con 0x7f962406d320 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f962406db90 0x7f962406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f962406db90 0x7f962406ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49042/0 (socket says 172.21.15.45:49042) 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 -- 172.21.15.45:0/1714852803 learned_addr learned my addr 172.21.15.45:0/1714852803 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:15.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f9629000640 1 --2- 172.21.15.45:0/1714852803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f962415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f9623e00640 1 --2- 172.21.15.45:0/1714852803 >> v2:172.21.15.117:3300/0 conn(0x7f962406cf40 0x7f962415e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 -- 172.21.15.45:0/1714852803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 msgr2=0x7f962415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 --2- 172.21.15.45:0/1714852803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f962415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 -- 172.21.15.45:0/1714852803 >> v2:172.21.15.117:3300/0 conn(0x7f962406cf40 msgr2=0x7f962415e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 --2- 172.21.15.45:0/1714852803 >> v2:172.21.15.117:3300/0 conn(0x7f962406cf40 0x7f962415e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.602+0000 7f961b400640 1 -- 172.21.15.45:0/1714852803 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240cddf0 con 0x7f962406db90 2024-04-25T10:22:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f961b400640 1 --2- 172.21.15.45:0/1714852803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f962406ea80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f962406f750 tx=0x7f960c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7425c908ec294f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f9623400640 1 -- 172.21.15.45:0/1714852803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0b0070 con 0x7f962406db90 2024-04-25T10:22:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f9623400640 1 -- 172.21.15.45:0/1714852803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c0ab600 con 0x7f962406db90 2024-04-25T10:22:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f9623400640 1 -- 172.21.15.45:0/1714852803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0ab9d0 con 0x7f962406db90 2024-04-25T10:22:15.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 -- 172.21.15.45:0/1714852803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 msgr2=0x7f962406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 --2- 172.21.15.45:0/1714852803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f962406ea80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f962406f750 tx=0x7f960c01c490 comp rx=0 tx=0).stop 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 -- 172.21.15.45:0/1714852803 shutdown_connections 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 --2- 172.21.15.45:0/1714852803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f962415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 --2- 172.21.15.45:0/1714852803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f962406ea80 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 --2- 172.21.15.45:0/1714852803 >> v2:172.21.15.117:3300/0 conn(0x7f962406cf40 0x7f962415e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 -- 172.21.15.45:0/1714852803 >> 172.21.15.45:0/1714852803 conn(0x7f9624072290 msgr2=0x7f96240726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 -- 172.21.15.45:0/1714852803 shutdown_connections 2024-04-25T10:22:15.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 -- 172.21.15.45:0/1714852803 wait complete. 2024-04-25T10:22:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 Processor -- start 2024-04-25T10:22:15.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.606+0000 7f962b000640 1 -- start start 2024-04-25T10:22:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f962406cf40 0x7f9624150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f96240c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f9624097f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962406d8b0 con 0x7f962406db90 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962406d730 con 0x7f962406cf40 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962415acf0 con 0x7f962406d320 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f9624097f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f9624097f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49050/0 (socket says 172.21.15.45:49050) 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 -- 172.21.15.45:0/1489181552 learned_addr learned my addr 172.21.15.45:0/1489181552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9623e00640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f962406cf40 0x7f9624150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f961b400640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f96240c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 -- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 msgr2=0x7f96240c0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f96240c0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 -- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f962406cf40 msgr2=0x7f9624150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f962406cf40 0x7f9624150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 -- 172.21.15.45:0/1489181552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240ca390 con 0x7f962406db90 2024-04-25T10:22:15.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9629000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f9624097f90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f961c007e70 tx=0x7f961c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c014620 con 0x7f962406db90 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c019070 con 0x7f962406db90 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c014a20 con 0x7f962406db90 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624151390 con 0x7f962406db90 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96240977b0 con 0x7f962406db90 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962404ef30 con 0x7f962406db90 2024-04-25T10:22:15.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.610+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f961c014bc0 con 0x7f962406db90 2024-04-25T10:22:15.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.614+0000 7f9620c00640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9600066f60 0x7f9600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.614+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f961c085880 con 0x7f962406db90 2024-04-25T10:22:15.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.614+0000 7f9623e00640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9600066f60 0x7f9600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.614+0000 7f9623e00640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9600066f60 0x7f9600069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f96140045f0 tx=0x7f9614008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.618+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f961c056180 con 0x7f962406db90 2024-04-25T10:22:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:15.814+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f96240d1000 con 0x7f962406db90 2024-04-25T10:22:16.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.434+0000 7f9620c00640 1 -- 172.21.15.45:0/1489181552 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f961c05a220 con 0x7f962406db90 2024-04-25T10:22:16.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9600066f60 msgr2=0x7f9600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9600066f60 0x7f9600069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f96140045f0 tx=0x7f9614008040 comp rx=0 tx=0).stop 2024-04-25T10:22:16.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 msgr2=0x7f9624097f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f9624097f90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f961c007e70 tx=0x7f961c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 shutdown_connections 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9600066f60 0x7f9600069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f962406db90 0x7f9624097f90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f962406d320 0x7f96240c0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 --2- 172.21.15.45:0/1489181552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f962406cf40 0x7f9624150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 >> 172.21.15.45:0/1489181552 conn(0x7f9624072290 msgr2=0x7f96240ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 shutdown_connections 2024-04-25T10:22:16.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.438+0000 7f962b000640 1 -- 172.21.15.45:0/1489181552 wait complete. 2024-04-25T10:22:16.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:22:16.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57ac400640 1 Processor -- start 2024-04-25T10:22:16.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57ac400640 1 -- start start 2024-04-25T10:22:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57ac400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57a415c0f0 0x7f57a40b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57ac400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a406c0e0 con 0x7f57a415bd10 2024-04-25T10:22:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57ac400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a415c6a0 con 0x7f57a40cf9e0 2024-04-25T10:22:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57ac400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a415c980 con 0x7f57a415c0f0 2024-04-25T10:22:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a9000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57a415c0f0 0x7f57a40b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a9000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57a415c0f0 0x7f57a40b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49054/0 (socket says 172.21.15.45:49054) 2024-04-25T10:22:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a9000640 1 -- 172.21.15.45:0/2697318135 learned_addr learned my addr 172.21.15.45:0/2697318135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a9000640 1 -- 172.21.15.45:0/2697318135 >> v1:172.21.15.117:6789/0 conn(0x7f57a40cf9e0 legacy=0x7f57a406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a9000640 1 -- 172.21.15.45:0/2697318135 >> v1:172.21.15.45:6789/0 conn(0x7f57a415bd10 legacy=0x7f57a40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a3e00640 1 -- 172.21.15.45:0/2697318135 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102980666 0 0) 0x7f57a406c0e0 con 0x7f57a415bd10 2024-04-25T10:22:16.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.614+0000 7f57a9000640 1 -- 172.21.15.45:0/2697318135 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a415cc60 con 0x7f57a415c0f0 2024-04-25T10:22:16.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57a9000640 1 --2- 172.21.15.45:0/2697318135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a415c0f0 0x7f57a40b27e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0931452ce13d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57a3e00640 1 -- 172.21.15.45:0/2697318135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900ab430 con 0x7f57a415c0f0 2024-04-25T10:22:16.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57a3e00640 1 -- 172.21.15.45:0/2697318135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57900b0070 con 0x7f57a415c0f0 2024-04-25T10:22:16.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57a3e00640 1 -- 172.21.15.45:0/2697318135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900ab7f0 con 0x7f57a415c0f0 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 -- 172.21.15.45:0/2697318135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a415c0f0 msgr2=0x7f57a40b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 --2- 172.21.15.45:0/2697318135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a415c0f0 0x7f57a40b27e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579001c520 comp rx=0 tx=0).stop 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 -- 172.21.15.45:0/2697318135 shutdown_connections 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 --2- 172.21.15.45:0/2697318135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a415c0f0 0x7f57a40b27e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 -- 172.21.15.45:0/2697318135 >> 172.21.15.45:0/2697318135 conn(0x7f57a4066b70 msgr2=0x7f57a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 -- 172.21.15.45:0/2697318135 shutdown_connections 2024-04-25T10:22:16.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.618+0000 7f57ac400640 1 -- 172.21.15.45:0/2697318135 wait complete. 2024-04-25T10:22:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 Processor -- start 2024-04-25T10:22:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- start start 2024-04-25T10:22:16.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 0x7f57a41465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57a415bd10 0x7f57a4157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a415c0f0 0x7f57a41486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a416a460 con 0x7f57a40cf9e0 2024-04-25T10:22:16.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a416a2e0 con 0x7f57a415c0f0 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a416a5e0 con 0x7f57a415bd10 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 0x7f57a41465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 0x7f57a41465c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49062/0 (socket says 172.21.15.45:49062) 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 -- 172.21.15.45:0/2560584698 learned_addr learned my addr 172.21.15.45:0/2560584698 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9000640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57a415bd10 0x7f57a4157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57aa400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a415c0f0 0x7f57a41486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 -- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57a415bd10 msgr2=0x7f57a4157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57a415bd10 0x7f57a4157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 -- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a415c0f0 msgr2=0x7f57a41486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a415c0f0 0x7f57a41486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 -- 172.21.15.45:0/2560584698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a4149620 con 0x7f57a40cf9e0 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a9a00640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 0x7f57a41465c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f579800c4d0 tx=0x7f579800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798017070 con 0x7f57a40cf9e0 2024-04-25T10:22:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798005480 con 0x7f57a40cf9e0 2024-04-25T10:22:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798003840 con 0x7f57a40cf9e0 2024-04-25T10:22:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a4149900 con 0x7f57a40cf9e0 2024-04-25T10:22:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a414c700 con 0x7f57a40cf9e0 2024-04-25T10:22:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.622+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a404ef30 con 0x7f57a40cf9e0 2024-04-25T10:22:16.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.626+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f579801a070 con 0x7f57a40cf9e0 2024-04-25T10:22:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.626+0000 7f57a1600640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5784066f60 0x7f5784069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.626+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f57980844d0 con 0x7f57a40cf9e0 2024-04-25T10:22:16.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.626+0000 7f57a9000640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5784066f60 0x7f5784069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.626+0000 7f57a9000640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5784066f60 0x7f5784069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f57900af000 tx=0x7f57900a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.630+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57980554e0 con 0x7f57a40cf9e0 2024-04-25T10:22:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:16.794+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57a41678a0 con 0x7f57a40cf9e0 2024-04-25T10:22:17.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.442+0000 7f57a1600640 1 -- 172.21.15.45:0/2560584698 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5798054de0 con 0x7f57a40cf9e0 2024-04-25T10:22:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:22:17.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5784066f60 msgr2=0x7f5784069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5784066f60 0x7f5784069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f57900af000 tx=0x7f57900a7000 comp rx=0 tx=0).stop 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 msgr2=0x7f57a41465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 0x7f57a41465c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f579800c4d0 tx=0x7f579800c990 comp rx=0 tx=0).stop 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 shutdown_connections 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5784066f60 0x7f5784069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a415c0f0 0x7f57a41486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57a415bd10 0x7f57a4157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 --2- 172.21.15.45:0/2560584698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57a40cf9e0 0x7f57a41465c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 >> 172.21.15.45:0/2560584698 conn(0x7f57a4066b70 msgr2=0x7f57a40b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 shutdown_connections 2024-04-25T10:22:17.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.446+0000 7f57ac400640 1 -- 172.21.15.45:0/2560584698 wait complete. 2024-04-25T10:22:17.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:22:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 Processor -- start 2024-04-25T10:22:17.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 -- start start 2024-04-25T10:22:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f372006b800 0x7f37200ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f372006b0d0 0x7f37200cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006bbe0 0x7f37200cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200cde90 con 0x7f372006b0d0 2024-04-25T10:22:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200cfdb0 con 0x7f372006b800 2024-04-25T10:22:17.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3725200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200d0090 con 0x7f372006bbe0 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f372006b0d0 0x7f37200cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f372006b0d0 0x7f37200cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49078/0 (socket says 172.21.15.45:49078) 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 -- 172.21.15.45:0/1501615476 learned_addr learned my addr 172.21.15.45:0/1501615476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f371e800640 1 --2- 172.21.15.45:0/1501615476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006bbe0 0x7f37200cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 -- 172.21.15.45:0/1501615476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006bbe0 msgr2=0x7f37200cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 --2- 172.21.15.45:0/1501615476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006bbe0 0x7f37200cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 -- 172.21.15.45:0/1501615476 >> v2:172.21.15.117:3300/0 conn(0x7f372006b800 msgr2=0x7f37200ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 --2- 172.21.15.45:0/1501615476 >> v2:172.21.15.117:3300/0 conn(0x7f372006b800 0x7f37200ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.618+0000 7f3715400640 1 -- 172.21.15.45:0/1501615476 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200d0370 con 0x7f372006b0d0 2024-04-25T10:22:17.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3715400640 1 --2- 172.21.15.45:0/1501615476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006b0d0 0x7f37200cd410 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3720071ac0 tx=0x7f370801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2b19593c6ee26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f371d400640 1 -- 172.21.15.45:0/1501615476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080b5070 con 0x7f372006b0d0 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f371d400640 1 -- 172.21.15.45:0/1501615476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37080a89d0 con 0x7f372006b0d0 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f371d400640 1 -- 172.21.15.45:0/1501615476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080a8da0 con 0x7f372006b0d0 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 -- 172.21.15.45:0/1501615476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006b0d0 msgr2=0x7f37200cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 --2- 172.21.15.45:0/1501615476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006b0d0 0x7f37200cd410 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3720071ac0 tx=0x7f370801ed10 comp rx=0 tx=0).stop 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 -- 172.21.15.45:0/1501615476 shutdown_connections 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 --2- 172.21.15.45:0/1501615476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006bbe0 0x7f37200cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 --2- 172.21.15.45:0/1501615476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006b0d0 0x7f37200cd410 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 --2- 172.21.15.45:0/1501615476 >> v2:172.21.15.117:3300/0 conn(0x7f372006b800 0x7f37200ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 -- 172.21.15.45:0/1501615476 >> 172.21.15.45:0/1501615476 conn(0x7f37200d19d0 msgr2=0x7f37200d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 -- 172.21.15.45:0/1501615476 shutdown_connections 2024-04-25T10:22:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.622+0000 7f3725200640 1 -- 172.21.15.45:0/1501615476 wait complete. 2024-04-25T10:22:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 Processor -- start 2024-04-25T10:22:17.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 -- start start 2024-04-25T10:22:17.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006b0d0 0x7f372014bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372006b800 0x7f37200af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 0x7f37200af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200d0e50 con 0x7f372006bbe0 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200d0930 con 0x7f372006b800 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37201f85b0 con 0x7f372006b0d0 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 0x7f37200af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 0x7f37200af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49080/0 (socket says 172.21.15.45:49080) 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 -- 172.21.15.45:0/524779812 learned_addr learned my addr 172.21.15.45:0/524779812 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3715400640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372006b800 0x7f37200af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371de00640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006b0d0 0x7f372014bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 -- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006b0d0 msgr2=0x7f372014bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006b0d0 0x7f372014bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 -- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372006b800 msgr2=0x7f37200af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372006b800 0x7f37200af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 -- 172.21.15.45:0/524779812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37201f77f0 con 0x7f372006bbe0 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371de00640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006b0d0 0x7f372014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:17.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f371e800640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 0x7f37200af8b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3718007e90 tx=0x7f371800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718014420 con 0x7f372006bbe0 2024-04-25T10:22:17.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3718019070 con 0x7f372006bbe0 2024-04-25T10:22:17.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718014800 con 0x7f372006bbe0 2024-04-25T10:22:17.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37201f7ad0 con 0x7f372006bbe0 2024-04-25T10:22:17.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.626+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37201f8090 con 0x7f372006bbe0 2024-04-25T10:22:17.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.630+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372004ef30 con 0x7f372006bbe0 2024-04-25T10:22:17.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.630+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37180149a0 con 0x7f372006bbe0 2024-04-25T10:22:17.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.630+0000 7f3716a00640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36fc066f60 0x7f36fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.630+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f3718085810 con 0x7f372006bbe0 2024-04-25T10:22:17.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.630+0000 7f371de00640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36fc066f60 0x7f36fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.630+0000 7f371de00640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36fc066f60 0x7f36fc069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f371000a8b0 tx=0x7f3710008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.634+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3718011260 con 0x7f372006bbe0 2024-04-25T10:22:17.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:17.802+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37200b7950 con 0x7f372006bbe0 2024-04-25T10:22:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.442+0000 7f3716a00640 1 -- 172.21.15.45:0/524779812 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3718056880 con 0x7f372006bbe0 2024-04-25T10:22:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:22:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36fc066f60 msgr2=0x7f36fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36fc066f60 0x7f36fc069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f371000a8b0 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 msgr2=0x7f37200af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 0x7f37200af8b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3718007e90 tx=0x7f371800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 shutdown_connections 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36fc066f60 0x7f36fc069420 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f371000a8b0 tx=0x7f3710008040 comp rx=0 tx=0).stop 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f372006bbe0 0x7f37200af8b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f372006b800 0x7f37200af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 --2- 172.21.15.45:0/524779812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f372006b0d0 0x7f372014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.446+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 >> 172.21.15.45:0/524779812 conn(0x7f37200d19d0 msgr2=0x7f37200b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.450+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 shutdown_connections 2024-04-25T10:22:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.450+0000 7f3725200640 1 -- 172.21.15.45:0/524779812 wait complete. 2024-04-25T10:22:18.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:22:18.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.622+0000 7f1fefa00640 1 Processor -- start 2024-04-25T10:22:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.622+0000 7f1fefa00640 1 -- start start 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.622+0000 7f1fefa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fe80d0d00 0x7f1fe80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.622+0000 7f1fefa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1fe80cc200 0x7f1fe8162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fefa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80cdab0 con 0x7f1fe81626c0 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fefa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe80cd210 con 0x7f1fe80cc200 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fefa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cd4f0 con 0x7f1fe80d0d00 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1fe81626c0 0x7f1fe806cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44898/0 (socket says 172.21.15.45:44898) 2024-04-25T10:22:18.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 -- 172.21.15.45:0/2058868353 learned_addr learned my addr 172.21.15.45:0/2058868353 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fdfe00640 1 --2- 172.21.15.45:0/2058868353 >> v2:172.21.15.117:3300/0 conn(0x7f1fe80cc200 0x7f1fe8162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fed000640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fe80d0d00 0x7f1fe80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454146527 0 0) 0x7f1fe80cdab0 con 0x7f1fe81626c0 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fcc003530 con 0x7f1fe81626c0 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4087340374 0 0) 0x7f1fcc003530 con 0x7f1fe81626c0 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fe80d0d00 msgr2=0x7f1fe80ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fe80d0d00 0x7f1fe80ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 >> v2:172.21.15.117:3300/0 conn(0x7f1fe80cc200 msgr2=0x7f1fe8162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 --2- 172.21.15.45:0/2058868353 >> v2:172.21.15.117:3300/0 conn(0x7f1fe80cc200 0x7f1fe8162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80cd7d0 con 0x7f1fe81626c0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f1fe0001a00 con 0x7f1fe81626c0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 >> v1:172.21.15.45:6789/0 conn(0x7f1fe81626c0 legacy=0x7f1fe806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0039c0 0x7f1fcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc0043b0 0x7f1fcc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fcc004fa0 0x7f1fcc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cd210 con 0x7f1fcc004fa0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80cd4f0 con 0x7f1fcc0043b0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc00bf40 con 0x7f1fcc0039c0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fcc004fa0 0x7f1fcc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fed000640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0039c0 0x7f1fcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1fdfe00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc0043b0 0x7f1fcc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 -- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0039c0 msgr2=0x7f1fcc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc0039c0 0x7f1fcc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 -- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc0043b0 msgr2=0x7f1fcc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fcc0043b0 0x7f1fcc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.626+0000 7f1feda00640 1 -- 172.21.15.45:0/2058868353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc003530 con 0x7f1fcc004fa0 2024-04-25T10:22:18.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1feda00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fcc004fa0 0x7f1fcc00b9d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000a8a0 tx=0x7f1fe001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e624bf0d57b9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000a4c0 con 0x7f1fcc004fa0 2024-04-25T10:22:18.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe00ad020 con 0x7f1fcc004fa0 2024-04-25T10:22:18.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fe7e00640 1 -- 172.21.15.45:0/2058868353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00a9740 con 0x7f1fcc004fa0 2024-04-25T10:22:18.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 -- 172.21.15.45:0/2058868353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fcc004fa0 msgr2=0x7f1fcc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 --2- 172.21.15.45:0/2058868353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fcc004fa0 0x7f1fcc00b9d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000a8a0 tx=0x7f1fe001fac0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1feda00640 1 -- 172.21.15.45:0/2058868353 reap_dead start 2024-04-25T10:22:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 -- 172.21.15.45:0/2058868353 shutdown_connections 2024-04-25T10:22:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 -- 172.21.15.45:0/2058868353 >> 172.21.15.45:0/2058868353 conn(0x7f1fe8072290 msgr2=0x7f1fe80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 -- 172.21.15.45:0/2058868353 shutdown_connections 2024-04-25T10:22:18.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 -- 172.21.15.45:0/2058868353 wait complete. 2024-04-25T10:22:18.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 Processor -- start 2024-04-25T10:22:18.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.630+0000 7f1fefa00640 1 -- start start 2024-04-25T10:22:18.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc004fa0 0x7f1fe80c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 0x7f1fe80af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe80c5e40 0x7f1fe80c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe806d650 con 0x7f1fe80aed00 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe806dc70 con 0x7f1fe80c5e40 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe806d7d0 con 0x7f1fcc004fa0 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 0x7f1fe80af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 0x7f1fe80af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49092/0 (socket says 172.21.15.45:49092) 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 -- 172.21.15.45:0/4062271493 learned_addr learned my addr 172.21.15.45:0/4062271493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1feda00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe80c5e40 0x7f1fe80c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fed000640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc004fa0 0x7f1fe80c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 -- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc004fa0 msgr2=0x7f1fe80c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc004fa0 0x7f1fe80c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 -- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe80c5e40 msgr2=0x7f1fe80c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe80c5e40 0x7f1fe80c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 -- 172.21.15.45:0/4062271493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80c5c10 con 0x7f1fe80aed00 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fdfe00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 0x7f1fe80af160 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0011a10 tx=0x7f1fd0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0019070 con 0x7f1fe80aed00 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0004240 con 0x7f1fe80aed00 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000f880 con 0x7f1fe80aed00 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe80c4e50 con 0x7f1fe80aed00 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe80c5410 con 0x7f1fe80aed00 2024-04-25T10:22:18.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe804ef30 con 0x7f1fe80aed00 2024-04-25T10:22:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.634+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fd001c070 con 0x7f1fe80aed00 2024-04-25T10:22:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.638+0000 7f1fe5600640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1fc4066f90 0x7f1fc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.638+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1fd0085330 con 0x7f1fe80aed00 2024-04-25T10:22:18.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.638+0000 7f1fed000640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1fc4066f90 0x7f1fc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.638+0000 7f1fed000640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1fc4066f90 0x7f1fc4069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80cdd40 tx=0x7f1fd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.642+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fd00564b0 con 0x7f1fe80aed00 2024-04-25T10:22:18.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.810+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1fe80af6a0 con 0x7f1fe80aed00 2024-04-25T10:22:18.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.814+0000 7f1fe5600640 1 -- 172.21.15.45:0/4062271493 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1fd0055db0 con 0x7f1fe80aed00 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1fc4066f90 msgr2=0x7f1fc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1fc4066f90 0x7f1fc4069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80cdd40 tx=0x7f1fd8008040 comp rx=0 tx=0).stop 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 msgr2=0x7f1fe80af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 0x7f1fe80af160 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0011a10 tx=0x7f1fd0011ed0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 shutdown_connections 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1fc4066f90 0x7f1fc4069450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fe80c5e40 0x7f1fe80c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fe80aed00 0x7f1fe80af160 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 --2- 172.21.15.45:0/4062271493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1fcc004fa0 0x7f1fe80c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 >> 172.21.15.45:0/4062271493 conn(0x7f1fe8072290 msgr2=0x7f1fe80cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 shutdown_connections 2024-04-25T10:22:18.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.818+0000 7f1fefa00640 1 -- 172.21.15.45:0/4062271493 wait complete. 2024-04-25T10:22:18.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:22:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.978+0000 7eff84c00640 1 Processor -- start 2024-04-25T10:22:18.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.978+0000 7eff84c00640 1 -- start start 2024-04-25T10:22:18.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff84c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7eff801550b0 0x7eff800cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff84c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff84c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff80071050 con 0x7eff800cfe70 2024-04-25T10:22:18.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff84c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8015bda0 con 0x7eff801550b0 2024-04-25T10:22:18.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff84c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015d040 con 0x7eff8015b910 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7eff801550b0 0x7eff800cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7eff801550b0 0x7eff800cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49104/0 (socket says 172.21.15.45:49104) 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 -- 172.21.15.45:0/3342987018 learned_addr learned my addr 172.21.15.45:0/3342987018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7e800640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 -- 172.21.15.45:0/3342987018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 msgr2=0x7eff800b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 -- 172.21.15.45:0/3342987018 >> v1:172.21.15.117:6789/0 conn(0x7eff800cfe70 legacy=0x7eff8015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7de00640 1 -- 172.21.15.45:0/3342987018 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800712f0 con 0x7eff801550b0 2024-04-25T10:22:18.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.982+0000 7eff7e800640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:18.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff7de00640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff801550b0 0x7eff800cf930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7eff700098a0 tx=0x7eff7001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4895bf599c68f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff7d400640 1 -- 172.21.15.45:0/3342987018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700b1070 con 0x7eff801550b0 2024-04-25T10:22:18.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff7d400640 1 -- 172.21.15.45:0/3342987018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff700ac410 con 0x7eff801550b0 2024-04-25T10:22:18.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff7d400640 1 -- 172.21.15.45:0/3342987018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700ac7e0 con 0x7eff801550b0 2024-04-25T10:22:18.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 -- 172.21.15.45:0/3342987018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff801550b0 msgr2=0x7eff800cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff801550b0 0x7eff800cf930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7eff700098a0 tx=0x7eff7001c500 comp rx=0 tx=0).stop 2024-04-25T10:22:18.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 -- 172.21.15.45:0/3342987018 shutdown_connections 2024-04-25T10:22:18.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 --2- 172.21.15.45:0/3342987018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff801550b0 0x7eff800cf930 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 -- 172.21.15.45:0/3342987018 >> 172.21.15.45:0/3342987018 conn(0x7eff80066b70 msgr2=0x7eff80066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 -- 172.21.15.45:0/3342987018 shutdown_connections 2024-04-25T10:22:18.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 -- 172.21.15.45:0/3342987018 wait complete. 2024-04-25T10:22:18.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 Processor -- start 2024-04-25T10:22:18.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 -- start start 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.986+0000 7eff84c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 0x7eff8014b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff801550b0 0x7eff800ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015c030 con 0x7eff800cfe70 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800720b0 con 0x7eff801550b0 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8015c1b0 con 0x7eff8015b910 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 0x7eff8014b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 0x7eff8014b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49120/0 (socket says 172.21.15.45:49120) 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 -- 172.21.15.45:0/2783292167 learned_addr learned my addr 172.21.15.45:0/2783292167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7e800640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff75400640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff801550b0 0x7eff800ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 -- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 msgr2=0x7eff800add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 -- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff801550b0 msgr2=0x7eff800ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff801550b0 0x7eff800ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 -- 172.21.15.45:0/2783292167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff801fb490 con 0x7eff800cfe70 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff7de00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 0x7eff8014b0d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7eff700af040 tx=0x7eff700192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700bc070 con 0x7eff800cfe70 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff700acda0 con 0x7eff800cfe70 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8014cd60 con 0x7eff800cfe70 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700b1040 con 0x7eff800cfe70 2024-04-25T10:22:18.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff80159040 con 0x7eff800cfe70 2024-04-25T10:22:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8004ef30 con 0x7eff800cfe70 2024-04-25T10:22:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff700bf070 con 0x7eff800cfe70 2024-04-25T10:22:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.990+0000 7eff76a00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.994+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7eff7011e420 con 0x7eff800cfe70 2024-04-25T10:22:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.994+0000 7eff75400640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.994+0000 7eff75400640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff5c066f60 0x7eff5c069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff800cbbe0 tx=0x7eff68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:18.994+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff801fb490 con 0x7eff800cfe70 2024-04-25T10:22:19.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.222+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7eff8015b140 con 0x7eff800cfe70 2024-04-25T10:22:19.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.454+0000 7eff76a00640 1 -- 172.21.15.45:0/2783292167 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v230) v1 ==== 143+0+0 (secure 0 0 0) 0x7eff700160f0 con 0x7eff800cfe70 2024-04-25T10:22:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff5c066f60 msgr2=0x7eff5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff5c066f60 0x7eff5c069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7eff800cbbe0 tx=0x7eff68009040 comp rx=0 tx=0).stop 2024-04-25T10:22:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 msgr2=0x7eff8014b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 0x7eff8014b0d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7eff700af040 tx=0x7eff700192a0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 shutdown_connections 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff8015b910 0x7eff800add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff801550b0 0x7eff800ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 --2- 172.21.15.45:0/2783292167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff800cfe70 0x7eff8014b0d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 >> 172.21.15.45:0/2783292167 conn(0x7eff80066b70 msgr2=0x7eff8014e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 shutdown_connections 2024-04-25T10:22:19.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.458+0000 7eff84c00640 1 -- 172.21.15.45:0/2783292167 wait complete. 2024-04-25T10:22:19.481 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:22:19.481 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T10:22:19.481 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:22:19.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.618+0000 7ff614e00640 1 Processor -- start 2024-04-25T10:22:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff614e00640 1 -- start start 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff614e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6101550b0 0x7ff6100cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff614e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff61015b910 0x7ff6100b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff614e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff610071050 con 0x7ff6100cfe70 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff614e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61015bd80 con 0x7ff6101550b0 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff614e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61015d040 con 0x7ff61015b910 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6101550b0 0x7ff6100cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60d400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff6100cfe70 0x7ff61015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44908/0 (socket says 172.21.15.45:44908) 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60d400640 1 -- 172.21.15.45:0/2191817697 learned_addr learned my addr 172.21.15.45:0/2191817697 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60e800640 1 --2- 172.21.15.45:0/2191817697 >> v2:172.21.15.117:3300/0 conn(0x7ff61015b910 0x7ff6100b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60de00640 1 -- 172.21.15.45:0/2191817697 >> v2:172.21.15.117:3300/0 conn(0x7ff61015b910 msgr2=0x7ff6100b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60ca00640 1 -- 172.21.15.45:0/2191817697 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781881936 0 0) 0x7ff610071050 con 0x7ff6100cfe70 2024-04-25T10:22:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60de00640 1 --2- 172.21.15.45:0/2191817697 >> v2:172.21.15.117:3300/0 conn(0x7ff61015b910 0x7ff6100b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60de00640 1 -- 172.21.15.45:0/2191817697 >> v1:172.21.15.45:6789/0 conn(0x7ff6100cfe70 legacy=0x7ff61015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.622+0000 7ff60de00640 1 -- 172.21.15.45:0/2191817697 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100712f0 con 0x7ff6101550b0 2024-04-25T10:22:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff60de00640 1 --2- 172.21.15.45:0/2191817697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6101550b0 0x7ff6100cf930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097c0 tx=0x7ff60001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7817b9cb5af09ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff60ca00640 1 -- 172.21.15.45:0/2191817697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000b0070 con 0x7ff6101550b0 2024-04-25T10:22:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff60ca00640 1 -- 172.21.15.45:0/2191817697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600004230 con 0x7ff6101550b0 2024-04-25T10:22:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff60ca00640 1 -- 172.21.15.45:0/2191817697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600003620 con 0x7ff6101550b0 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 -- 172.21.15.45:0/2191817697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6101550b0 msgr2=0x7ff6100cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 --2- 172.21.15.45:0/2191817697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6101550b0 0x7ff6100cf930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097c0 tx=0x7ff60001c460 comp rx=0 tx=0).stop 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 -- 172.21.15.45:0/2191817697 shutdown_connections 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 --2- 172.21.15.45:0/2191817697 >> v2:172.21.15.117:3300/0 conn(0x7ff61015b910 0x7ff6100b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 --2- 172.21.15.45:0/2191817697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6101550b0 0x7ff6100cf930 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 -- 172.21.15.45:0/2191817697 >> 172.21.15.45:0/2191817697 conn(0x7ff610066b70 msgr2=0x7ff610066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 -- 172.21.15.45:0/2191817697 shutdown_connections 2024-04-25T10:22:19.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 -- 172.21.15.45:0/2191817697 wait complete. 2024-04-25T10:22:19.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 Processor -- start 2024-04-25T10:22:19.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.626+0000 7ff614e00640 1 -- start start 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6100cfe70 0x7ff61014d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6101550b0 0x7ff610145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 0x7ff610130c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015c030 con 0x7ff61015b910 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100720b0 con 0x7ff6100cfe70 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015c1b0 con 0x7ff6101550b0 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 0x7ff610130c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 0x7ff610130c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49130/0 (socket says 172.21.15.45:49130) 2024-04-25T10:22:19.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 -- 172.21.15.45:0/1242120638 learned_addr learned my addr 172.21.15.45:0/1242120638 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60de00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6100cfe70 0x7ff61014d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60d400640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6101550b0 0x7ff610145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 -- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6101550b0 msgr2=0x7ff610145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6101550b0 0x7ff610145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 -- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6100cfe70 msgr2=0x7ff61014d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6100cfe70 0x7ff61014d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 -- 172.21.15.45:0/1242120638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6101316b0 con 0x7ff61015b910 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60de00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6100cfe70 0x7ff61014d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:19.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff60e800640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 0x7ff610130c50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff608007e60 tx=0x7ff60800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608014420 con 0x7ff61015b910 2024-04-25T10:22:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608019070 con 0x7ff61015b910 2024-04-25T10:22:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff610133a80 con 0x7ff61015b910 2024-04-25T10:22:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608014820 con 0x7ff61015b910 2024-04-25T10:22:19.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff610145250 con 0x7ff61015b910 2024-04-25T10:22:19.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff610144d60 con 0x7ff61015b910 2024-04-25T10:22:19.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.630+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6080149c0 con 0x7ff61015b910 2024-04-25T10:22:19.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.634+0000 7ff606000640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f0066f90 0x7ff5f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.634+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff608085680 con 0x7ff61015b910 2024-04-25T10:22:19.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.634+0000 7ff60de00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f0066f90 0x7ff5f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.634+0000 7ff60de00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f0066f90 0x7ff5f0069450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff6000ad000 tx=0x7ff600008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.638+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff608056890 con 0x7ff61015b910 2024-04-25T10:22:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.798+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff61004ef30 con 0x7ff61015b910 2024-04-25T10:22:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff606000640 1 -- 172.21.15.45:0/1242120638 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+13142 (secure 0 0 0) 0x7ff608056190 con 0x7ff61015b910 2024-04-25T10:22:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:19.805 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":69,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:22:18.446669+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/4277203907":"2024-04-26T10:22:15.436795+0000","172.21.15.117:6835/4277203907":"2024-04-26T10:22:15.436795+0000","172.21.15.117:6837/3997755731":"2024-04-26T10:21:52.255019+0000","172.21.15.117:6835/3997755731":"2024-04-26T10:21:52.255019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f0066f90 msgr2=0x7ff5f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f0066f90 0x7ff5f0069450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff6000ad000 tx=0x7ff600008040 comp rx=0 tx=0).stop 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 msgr2=0x7ff610130c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 0x7ff610130c50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff608007e60 tx=0x7ff60800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 shutdown_connections 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f0066f90 0x7ff5f0069450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61015b910 0x7ff610130c50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6101550b0 0x7ff610145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 --2- 172.21.15.45:0/1242120638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6100cfe70 0x7ff61014d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 >> 172.21.15.45:0/1242120638 conn(0x7ff610066b70 msgr2=0x7ff61006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 shutdown_connections 2024-04-25T10:22:19.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.802+0000 7ff614e00640 1 -- 172.21.15.45:0/1242120638 wait complete. 2024-04-25T10:22:19.826 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:19.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:19.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 Processor -- start 2024-04-25T10:22:19.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 -- start start 2024-04-25T10:22:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5464159540 0x7f54640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 0x7f5464071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5464071360 con 0x7f5464072020 2024-04-25T10:22:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5464159920 con 0x7f5464159540 2024-04-25T10:22:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5468e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546415c2f0 con 0x7f54640aebd0 2024-04-25T10:22:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5464072020 0x7f546415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44920/0 (socket says 172.21.15.45:44920) 2024-04-25T10:22:19.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 -- 172.21.15.45:0/2757790449 learned_addr learned my addr 172.21.15.45:0/2757790449 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461e00640 1 --2- 172.21.15.45:0/2757790449 >> v2:172.21.15.117:3300/0 conn(0x7f5464159540 0x7f54640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f545be00640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 0x7f5464071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352482303 0 0) 0x7f5464071360 con 0x7f5464072020 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5448003530 con 0x7f5464072020 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1691731276 0 0) 0x7f5448003530 con 0x7f5464072020 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 msgr2=0x7f5464071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 0x7f5464071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 >> v2:172.21.15.117:3300/0 conn(0x7f5464159540 msgr2=0x7f54640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 --2- 172.21.15.45:0/2757790449 >> v2:172.21.15.117:3300/0 conn(0x7f5464159540 0x7f54640ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546415c5d0 con 0x7f5464072020 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f545c001a20 con 0x7f5464072020 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461e00640 1 --2- 172.21.15.45:0/2757790449 >> v2:172.21.15.117:3300/0 conn(0x7f5464159540 0x7f54640ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 >> v1:172.21.15.45:6789/0 conn(0x7f5464072020 legacy=0x7f546415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 0x7f54480039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5448003f40 0x7f5448004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f54480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464159920 con 0x7f5448004d50 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546415c2f0 con 0x7f5448003f40 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544800b9d0 con 0x7f54640aebd0 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f54480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f545be00640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5448003f40 0x7f5448004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5461e00640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 0x7f54480039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 -- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 msgr2=0x7f54480039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640aebd0 0x7f54480039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 -- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5448003f40 msgr2=0x7f5448004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5448003f40 0x7f5448004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.970+0000 7f5462800640 1 -- 172.21.15.45:0/2757790449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448003530 con 0x7f5448004d50 2024-04-25T10:22:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5462800640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f54480051d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f545c00a880 tx=0x7f545c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bee772119ef347 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c00a4d0 con 0x7f5448004d50 2024-04-25T10:22:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c0ad630 con 0x7f5448004d50 2024-04-25T10:22:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5461400640 1 -- 172.21.15.45:0/2757790449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0ada20 con 0x7f5448004d50 2024-04-25T10:22:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5468e00640 1 -- 172.21.15.45:0/2757790449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 msgr2=0x7f54480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5468e00640 1 --2- 172.21.15.45:0/2757790449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f54480051d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f545c00a880 tx=0x7f545c01d320 comp rx=0 tx=0).stop 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5462800640 1 -- 172.21.15.45:0/2757790449 reap_dead start 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5468e00640 1 -- 172.21.15.45:0/2757790449 shutdown_connections 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5468e00640 1 -- 172.21.15.45:0/2757790449 >> 172.21.15.45:0/2757790449 conn(0x7f54640669a0 msgr2=0x7f5464066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5468e00640 1 -- 172.21.15.45:0/2757790449 shutdown_connections 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.974+0000 7f5468e00640 1 -- 172.21.15.45:0/2757790449 wait complete. 2024-04-25T10:22:19.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 Processor -- start 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 -- start start 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f5464152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640b1610 0x7f54640b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54641f4d00 0x7f54641f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464072470 con 0x7f5448004d50 2024-04-25T10:22:19.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546415cb90 con 0x7f54641f4d00 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546415bfa0 con 0x7f54640b1610 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f5464152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f5464152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49146/0 (socket says 172.21.15.45:49146) 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 -- 172.21.15.45:0/921164222 learned_addr learned my addr 172.21.15.45:0/921164222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5462800640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54641f4d00 0x7f54641f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 -- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640b1610 msgr2=0x7f54640b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640b1610 0x7f54640b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 -- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54641f4d00 msgr2=0x7f54641f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54641f4d00 0x7f54641f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 -- 172.21.15.45:0/921164222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5464152bd0 con 0x7f5448004d50 2024-04-25T10:22:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5462800640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54641f4d00 0x7f54641f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5461e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f5464152690 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5454011d00 tx=0x7f545400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454004020 con 0x7f5448004d50 2024-04-25T10:22:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54540041c0 con 0x7f5448004d50 2024-04-25T10:22:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54641f16f0 con 0x7f5448004d50 2024-04-25T10:22:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454003620 con 0x7f5448004d50 2024-04-25T10:22:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.978+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54641f1cb0 con 0x7f5448004d50 2024-04-25T10:22:19.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.982+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546404ef30 con 0x7f5448004d50 2024-04-25T10:22:19.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.982+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5454012040 con 0x7f5448004d50 2024-04-25T10:22:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.982+0000 7f545a000640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5440066f90 0x7f5440069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.982+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f54540854f0 con 0x7f5448004d50 2024-04-25T10:22:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.982+0000 7f545be00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5440066f90 0x7f5440069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.982+0000 7f545be00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5440066f90 0x7f5440069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f54640b1380 tx=0x7f544c009340 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:19.986+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5454019030 con 0x7f5448004d50 2024-04-25T10:22:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.150+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f546406d7f0 con 0x7f5448004d50 2024-04-25T10:22:20.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.150+0000 7f545a000640 1 -- 172.21.15.45:0/921164222 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5454056700 con 0x7f5448004d50 2024-04-25T10:22:20.156 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5440066f90 msgr2=0x7f5440069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5440066f90 0x7f5440069450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f54640b1380 tx=0x7f544c009340 comp rx=0 tx=0).stop 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 msgr2=0x7f5464152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f5464152690 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5454011d00 tx=0x7f545400f610 comp rx=0 tx=0).stop 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 shutdown_connections 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5440066f90 0x7f5440069450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54641f4d00 0x7f54641f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54640b1610 0x7f54640b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 --2- 172.21.15.45:0/921164222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5448004d50 0x7f5464152690 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 >> 172.21.15.45:0/921164222 conn(0x7f54640669a0 msgr2=0x7f546406de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 shutdown_connections 2024-04-25T10:22:20.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.154+0000 7f5468e00640 1 -- 172.21.15.45:0/921164222 wait complete. 2024-04-25T10:22:20.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:22:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 Processor -- start 2024-04-25T10:22:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 -- start start 2024-04-25T10:22:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3f00cda10 0x7fd3f006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00b2770 con 0x7fd3f015bc40 2024-04-25T10:22:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f006c670 con 0x7fd3f00cda10 2024-04-25T10:22:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3f4c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f006c950 con 0x7fd3f00cf930 2024-04-25T10:22:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3ede00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd3f015bc40 0x7fd3f00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44928/0 (socket says 172.21.15.45:44928) 2024-04-25T10:22:20.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.314+0000 7fd3ede00640 1 -- 172.21.15.45:0/920631496 learned_addr learned my addr 172.21.15.45:0/920631496 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3e5400640 1 --2- 172.21.15.45:0/920631496 >> v2:172.21.15.117:3300/0 conn(0x7fd3f00cda10 0x7fd3f006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3ee800640 1 --2- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3ed400640 1 -- 172.21.15.45:0/920631496 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185750220 0 0) 0x7fd3f00b2770 con 0x7fd3f015bc40 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3ed400640 1 -- 172.21.15.45:0/920631496 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d4003530 con 0x7fd3f015bc40 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3e5400640 1 -- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 msgr2=0x7fd3f00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3e5400640 1 --2- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3e5400640 1 -- 172.21.15.45:0/920631496 >> v1:172.21.15.45:6789/0 conn(0x7fd3f015bc40 legacy=0x7fd3f00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3e5400640 1 -- 172.21.15.45:0/920631496 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f006cc30 con 0x7fd3f00cda10 2024-04-25T10:22:20.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.318+0000 7fd3e5400640 1 --2- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f006c090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80097c0 tx=0x7fd3d801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ee4af0f33cbbd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3ed400640 1 -- 172.21.15.45:0/920631496 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80b0070 con 0x7fd3f00cda10 2024-04-25T10:22:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3ed400640 1 -- 172.21.15.45:0/920631496 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d80ab600 con 0x7fd3f00cda10 2024-04-25T10:22:20.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3ed400640 1 -- 172.21.15.45:0/920631496 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80ab9d0 con 0x7fd3f00cda10 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 msgr2=0x7fd3f006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f006c090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80097c0 tx=0x7fd3d801c410 comp rx=0 tx=0).stop 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- 172.21.15.45:0/920631496 shutdown_connections 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/920631496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f006c090 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- 172.21.15.45:0/920631496 >> 172.21.15.45:0/920631496 conn(0x7fd3f0066b70 msgr2=0x7fd3f0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- 172.21.15.45:0/920631496 shutdown_connections 2024-04-25T10:22:20.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- 172.21.15.45:0/920631496 wait complete. 2024-04-25T10:22:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 Processor -- start 2024-04-25T10:22:20.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- start start 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f0146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 0x7fd3f00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00b3890 con 0x7fd3f015bc40 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00b3710 con 0x7fd3f00cda10 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.322+0000 7fd3f4c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00b3a10 con 0x7fd3f00cf930 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 0x7fd3f00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 0x7fd3f00ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49162/0 (socket says 172.21.15.45:49162) 2024-04-25T10:22:20.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 -- 172.21.15.45:0/3536400485 learned_addr learned my addr 172.21.15.45:0/3536400485 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ede00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e5400640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f0146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 -- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 msgr2=0x7fd3f0146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f0146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 -- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 msgr2=0x7fd3f00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 -- 172.21.15.45:0/3536400485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f01466a0 con 0x7fd3f015bc40 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3ee800640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 0x7fd3f00ad840 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e7f0 tx=0x7fd3e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8017070 con 0x7fd3f015bc40 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e800c940 con 0x7fd3f015bc40 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800eea0 con 0x7fd3f015bc40 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f00afac0 con 0x7fd3f015bc40 2024-04-25T10:22:20.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f00b0080 con 0x7fd3f015bc40 2024-04-25T10:22:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f004ef30 con 0x7fd3f015bc40 2024-04-25T10:22:20.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3e801a070 con 0x7fd3f015bc40 2024-04-25T10:22:20.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e6a00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3cc066f90 0x7fd3cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.326+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd3e8084fe0 con 0x7fd3f015bc40 2024-04-25T10:22:20.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.330+0000 7fd3ede00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3cc066f90 0x7fd3cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.330+0000 7fd3ede00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3cc066f90 0x7fd3cc069450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0005c10 tx=0x7fd3e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.330+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3e8056280 con 0x7fd3f015bc40 2024-04-25T10:22:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:20.554+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fd3f006d960 con 0x7fd3f015bc40 2024-04-25T10:22:21.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3e6a00640 1 -- 172.21.15.45:0/3536400485 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v232) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd3e8055b80 con 0x7fd3f015bc40 2024-04-25T10:22:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3cc066f90 msgr2=0x7fd3cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3cc066f90 0x7fd3cc069450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0005c10 tx=0x7fd3e0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 msgr2=0x7fd3f00ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 0x7fd3f00ad840 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e7f0 tx=0x7fd3e800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 shutdown_connections 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3cc066f90 0x7fd3cc069450 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3f015bc40 0x7fd3f00ad840 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3f00cf930 0x7fd3f0146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 --2- 172.21.15.45:0/3536400485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3f00cda10 0x7fd3f00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 >> 172.21.15.45:0/3536400485 conn(0x7fd3f0066b70 msgr2=0x7fd3f0155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 shutdown_connections 2024-04-25T10:22:21.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.466+0000 7fd3f4c00640 1 -- 172.21.15.45:0/3536400485 wait complete. 2024-04-25T10:22:21.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:22:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.638+0000 7fb25b000640 1 Processor -- start 2024-04-25T10:22:21.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.638+0000 7fb25b000640 1 -- start start 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb25b000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2540cd9a0 0x7fb2540d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb25b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb25406bd10 0x7fb254161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb25b000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25406ac90 con 0x7fb2540d0ba0 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb25b000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb25406b880 con 0x7fb2540cd9a0 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb25b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25406af50 con 0x7fb25406bd10 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2540cd9a0 0x7fb2540d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2540cd9a0 0x7fb2540d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49172/0 (socket says 172.21.15.45:49172) 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb259000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb2540d0ba0 0x7fb25415d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44938/0 (socket says 172.21.15.45:44938) 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 -- 172.21.15.45:0/3839324941 learned_addr learned my addr 172.21.15.45:0/3839324941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb24b400640 1 --2- 172.21.15.45:0/3839324941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb25406bd10 0x7fb254161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 -- 172.21.15.45:0/3839324941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb25406bd10 msgr2=0x7fb254161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 --2- 172.21.15.45:0/3839324941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb25406bd10 0x7fb254161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 -- 172.21.15.45:0/3839324941 >> v1:172.21.15.45:6789/0 conn(0x7fb2540d0ba0 legacy=0x7fb25415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253e00640 1 -- 172.21.15.45:0/3839324941 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25406b230 con 0x7fb2540cd9a0 2024-04-25T10:22:21.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.642+0000 7fb253400640 1 -- 172.21.15.45:0/3839324941 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15746001 0 0) 0x7fb25406ac90 con 0x7fb2540d0ba0 2024-04-25T10:22:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb253e00640 1 --2- 172.21.15.45:0/3839324941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb2540d0660 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009870 tx=0x7fb23c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d3be900bbc90d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb253400640 1 -- 172.21.15.45:0/3839324941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0b5070 con 0x7fb2540cd9a0 2024-04-25T10:22:21.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb253400640 1 -- 172.21.15.45:0/3839324941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c0a8a50 con 0x7fb2540cd9a0 2024-04-25T10:22:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb253400640 1 -- 172.21.15.45:0/3839324941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0a8e20 con 0x7fb2540cd9a0 2024-04-25T10:22:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- 172.21.15.45:0/3839324941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 msgr2=0x7fb2540d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 --2- 172.21.15.45:0/3839324941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb2540d0660 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009870 tx=0x7fb23c01ed50 comp rx=0 tx=0).stop 2024-04-25T10:22:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- 172.21.15.45:0/3839324941 shutdown_connections 2024-04-25T10:22:21.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 --2- 172.21.15.45:0/3839324941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb25406bd10 0x7fb254161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 --2- 172.21.15.45:0/3839324941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb2540d0660 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- 172.21.15.45:0/3839324941 >> 172.21.15.45:0/3839324941 conn(0x7fb2540d19a0 msgr2=0x7fb2540d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- 172.21.15.45:0/3839324941 shutdown_connections 2024-04-25T10:22:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- 172.21.15.45:0/3839324941 wait complete. 2024-04-25T10:22:21.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 Processor -- start 2024-04-25T10:22:21.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- start start 2024-04-25T10:22:21.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb25406bd10 0x7fb2540b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb25415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2540d0ba0 0x7fb2540c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb254162660 con 0x7fb2540cd9a0 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2540cdd80 con 0x7fb25406bd10 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.646+0000 7fb25b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb254161e50 con 0x7fb2540d0ba0 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb25415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb25415d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49182/0 (socket says 172.21.15.45:49182) 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 -- 172.21.15.45:0/3697219747 learned_addr learned my addr 172.21.15.45:0/3697219747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb259000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2540d0ba0 0x7fb2540c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb253e00640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb25406bd10 0x7fb2540b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 -- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2540d0ba0 msgr2=0x7fb2540c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2540d0ba0 0x7fb2540c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 -- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb25406bd10 msgr2=0x7fb2540b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb25406bd10 0x7fb2540b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 -- 172.21.15.45:0/3697219747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2540b5200 con 0x7fb2540cd9a0 2024-04-25T10:22:21.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb24b400640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb25415d050 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb24400e960 tx=0x7fb24400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb244017070 con 0x7fb2540cd9a0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24400ca00 con 0x7fb2540cd9a0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24400cdb0 con 0x7fb2540cd9a0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2540b54e0 con 0x7fb2540cd9a0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25415d590 con 0x7fb2540cd9a0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25404ef30 con 0x7fb2540cd9a0 2024-04-25T10:22:21.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.650+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb24401a070 con 0x7fb2540cd9a0 2024-04-25T10:22:21.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.654+0000 7fb250c00640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb230066f60 0x7fb230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.654+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb2440851b0 con 0x7fb2540cd9a0 2024-04-25T10:22:21.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.654+0000 7fb253e00640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb230066f60 0x7fb230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.654+0000 7fb253e00640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb230066f60 0x7fb230069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b2000 tx=0x7fb23c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.658+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb244056450 con 0x7fb2540cd9a0 2024-04-25T10:22:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.818+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb2540b67b0 con 0x7fb2540cd9a0 2024-04-25T10:22:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb250c00640 1 -- 172.21.15.45:0/3697219747 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb244055d50 con 0x7fb2540cd9a0 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb230066f60 msgr2=0x7fb230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb230066f60 0x7fb230069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b2000 tx=0x7fb23c008040 comp rx=0 tx=0).stop 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 msgr2=0x7fb25415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb25415d050 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb24400e960 tx=0x7fb24400ee20 comp rx=0 tx=0).stop 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 shutdown_connections 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb230066f60 0x7fb230069420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2540d0ba0 0x7fb2540c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2540cd9a0 0x7fb25415d050 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 --2- 172.21.15.45:0/3697219747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb25406bd10 0x7fb2540b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 >> 172.21.15.45:0/3697219747 conn(0x7fb2540d19a0 msgr2=0x7fb2540cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 shutdown_connections 2024-04-25T10:22:21.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:21.822+0000 7fb25b000640 1 -- 172.21.15.45:0/3697219747 wait complete. 2024-04-25T10:22:21.847 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:22:21.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:21.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:21.989 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:22:21.989 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:21.989 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:21.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:21.999 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:22:21.999 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:22.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:22.008 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:22.008 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:22.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:22.048 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:22:22.048 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:22.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:22.058 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:22:22.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 Processor -- start 2024-04-25T10:22:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 -- start start 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b3406c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b340ccf50 0x7f8b340b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3406d360 con 0x7f8b3406ced0 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3406d640 con 0x7f8b340ccf50 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b3a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406d920 con 0x7f8b341620b0 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b340ccf50 0x7f8b340b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b340ccf50 0x7f8b340b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49194/0 (socket says 172.21.15.45:49194) 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b33e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8b3406ced0 0x7f8b340cc840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44948/0 (socket says 172.21.15.45:44948) 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 -- 172.21.15.45:0/742168540 learned_addr learned my addr 172.21.15.45:0/742168540 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39000640 1 --2- 172.21.15.45:0/742168540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b3406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 -- 172.21.15.45:0/742168540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 msgr2=0x7f8b3406c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 --2- 172.21.15.45:0/742168540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b3406c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 -- 172.21.15.45:0/742168540 >> v1:172.21.15.45:6789/0 conn(0x7f8b3406ced0 legacy=0x7f8b340cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b39a00640 1 -- 172.21.15.45:0/742168540 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3406dc00 con 0x7f8b340ccf50 2024-04-25T10:22:22.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.230+0000 7f8b33400640 1 -- 172.21.15.45:0/742168540 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241990519 0 0) 0x7f8b3406d360 con 0x7f8b3406ced0 2024-04-25T10:22:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b39a00640 1 --2- 172.21.15.45:0/742168540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340b3b80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009e50 tx=0x7f8b2c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fc8763a847a92a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b33400640 1 -- 172.21.15.45:0/742168540 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0a5820 con 0x7f8b340ccf50 2024-04-25T10:22:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b33400640 1 -- 172.21.15.45:0/742168540 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0a59c0 con 0x7f8b340ccf50 2024-04-25T10:22:22.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b33400640 1 -- 172.21.15.45:0/742168540 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0a5d90 con 0x7f8b340ccf50 2024-04-25T10:22:22.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 -- 172.21.15.45:0/742168540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 msgr2=0x7f8b340b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 --2- 172.21.15.45:0/742168540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340b3b80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009e50 tx=0x7f8b2c01c9d0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 -- 172.21.15.45:0/742168540 shutdown_connections 2024-04-25T10:22:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 --2- 172.21.15.45:0/742168540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340b3b80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 --2- 172.21.15.45:0/742168540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b3406c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 -- 172.21.15.45:0/742168540 >> 172.21.15.45:0/742168540 conn(0x7f8b340b1a20 msgr2=0x7f8b340b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 -- 172.21.15.45:0/742168540 shutdown_connections 2024-04-25T10:22:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.234+0000 7f8b3a400640 1 -- 172.21.15.45:0/742168540 wait complete. 2024-04-25T10:22:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 Processor -- start 2024-04-25T10:22:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- start start 2024-04-25T10:22:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b34150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34163860 0x7f8b3415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340cd970 con 0x7f8b340ccf50 2024-04-25T10:22:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340cd7f0 con 0x7f8b34163860 2024-04-25T10:22:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b48d0 con 0x7f8b341620b0 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49208/0 (socket says 172.21.15.45:49208) 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 -- 172.21.15.45:0/549939378 learned_addr learned my addr 172.21.15.45:0/549939378 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b33e00640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b34150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39a00640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34163860 0x7f8b3415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 -- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 msgr2=0x7f8b34150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b34150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 -- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34163860 msgr2=0x7f8b3415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34163860 0x7f8b3415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 -- 172.21.15.45:0/549939378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340adf60 con 0x7f8b340ccf50 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39a00640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34163860 0x7f8b3415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b39000640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340af000 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d5f0 tx=0x7f8b2800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280125f0 con 0x7f8b340ccf50 2024-04-25T10:22:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b28019070 con 0x7f8b340ccf50 2024-04-25T10:22:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b340ae240 con 0x7f8b340ccf50 2024-04-25T10:22:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280129b0 con 0x7f8b340ccf50 2024-04-25T10:22:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340ae800 con 0x7f8b340ccf50 2024-04-25T10:22:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.238+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b34075120 con 0x7f8b340ccf50 2024-04-25T10:22:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.242+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b28012b50 con 0x7f8b340ccf50 2024-04-25T10:22:22.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.242+0000 7f8b30c00640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b14066f60 0x7f8b14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.242+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8b28085240 con 0x7f8b340ccf50 2024-04-25T10:22:22.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.242+0000 7f8b33e00640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b14066f60 0x7f8b14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.242+0000 7f8b33e00640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b14066f60 0x7f8b14069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8b34154d70 tx=0x7f8b20007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:22.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.246+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b28056450 con 0x7f8b340ccf50 2024-04-25T10:22:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.438+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b34160ea0 con 0x7f8b340ccf50 2024-04-25T10:22:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b30c00640 1 -- 172.21.15.45:0/549939378 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f8b28055d50 con 0x7f8b340ccf50 2024-04-25T10:22:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:22.445 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1270289893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1270289893},{"type":"v1","addr":"172.21.15.45:6833","nonce":1270289893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6346,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2300391527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2300391527},{"type":"v1","addr":"172.21.15.117:6836","nonce":2300391527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/427022352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":427022352},{"type":"v1","addr":"172.21.15.117:6837","nonce":427022352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6621,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1063940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1063940451},{"type":"v1","addr":"172.21.15.45:6835","nonce":1063940451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[]} 2024-04-25T10:22:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 48 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b14066f60 msgr2=0x7f8b14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b14066f60 0x7f8b14069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8b34154d70 tx=0x7f8b20007b60 comp rx=0 tx=0).stop 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 msgr2=0x7f8b340af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340af000 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d5f0 tx=0x7f8b2800dab0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 shutdown_connections 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b14066f60 0x7f8b14069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b34163860 0x7f8b3415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b341620b0 0x7f8b34150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 --2- 172.21.15.45:0/549939378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b340ccf50 0x7f8b340af000 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 >> 172.21.15.45:0/549939378 conn(0x7f8b340b1a20 msgr2=0x7f8b340709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 shutdown_connections 2024-04-25T10:22:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.442+0000 7f8b3a400640 1 -- 172.21.15.45:0/549939378 wait complete. 2024-04-25T10:22:22.466 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:22:22.466 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:22:22.466 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:22:22.466 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:22.467 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:22:22.467 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:22:22.467 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:22:22.467 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:22.467 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:22:22.467 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:22:22.467 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:22:22.468 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:22.468 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:22:22.468 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:22:22.468 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:22:22.468 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:22.481 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:22:22.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:22:22.482 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:22:22.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:22:22.485 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:22:22.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:22:22.486 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:22:22.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:22:22.526 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:22:22.526 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:22.528 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:22:22.528 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:22.532 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:22:22.532 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:22.534 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:22:22.534 DEBUG:teuthology.parallel:result is None 2024-04-25T10:22:22.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:22:22.614 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:22:22.615 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:22:22.623 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:22:22.623 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:22:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.678+0000 7f7292e00640 1 Processor -- start 2024-04-25T10:22:22.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.678+0000 7f7292e00640 1 -- start start 2024-04-25T10:22:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f7292e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f728c1550b0 0x7f728c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f7292e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c071050 con 0x7f728c0cfe70 2024-04-25T10:22:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f7292e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c15d040 con 0x7f728c15b910 2024-04-25T10:22:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f7292e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c15bd80 con 0x7f728c1550b0 2024-04-25T10:22:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f728be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f728c1550b0 0x7f728c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f7283400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f728c0cfe70 0x7f728c15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44974/0 (socket says 172.21.15.45:44974) 2024-04-25T10:22:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f7283400640 1 -- 172.21.15.45:0/2340167385 learned_addr learned my addr 172.21.15.45:0/2340167385 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f728be00640 1 -- 172.21.15.45:0/2340167385 >> v1:172.21.15.117:6789/0 conn(0x7f728c15b910 legacy=0x7f728c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f728be00640 1 -- 172.21.15.45:0/2340167385 >> v1:172.21.15.45:6789/0 conn(0x7f728c0cfe70 legacy=0x7f728c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f728b400640 1 -- 172.21.15.45:0/2340167385 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267854863 0 0) 0x7f728c071050 con 0x7f728c0cfe70 2024-04-25T10:22:22.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f728be00640 1 -- 172.21.15.45:0/2340167385 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0712f0 con 0x7f728c1550b0 2024-04-25T10:22:22.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.682+0000 7f728be00640 1 --2- 172.21.15.45:0/2340167385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c1550b0 0x7f728c0cf930 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097f0 tx=0x7f727c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34144b28f0546d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f728b400640 1 -- 172.21.15.45:0/2340167385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b0070 con 0x7f728c1550b0 2024-04-25T10:22:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f728b400640 1 -- 172.21.15.45:0/2340167385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0ab600 con 0x7f728c1550b0 2024-04-25T10:22:22.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f728b400640 1 -- 172.21.15.45:0/2340167385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0ab9d0 con 0x7f728c1550b0 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- 172.21.15.45:0/2340167385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c1550b0 msgr2=0x7f728c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 --2- 172.21.15.45:0/2340167385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c1550b0 0x7f728c0cf930 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f727c0097f0 tx=0x7f727c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- 172.21.15.45:0/2340167385 shutdown_connections 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 --2- 172.21.15.45:0/2340167385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c1550b0 0x7f728c0cf930 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- 172.21.15.45:0/2340167385 >> 172.21.15.45:0/2340167385 conn(0x7f728c066b70 msgr2=0x7f728c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- 172.21.15.45:0/2340167385 shutdown_connections 2024-04-25T10:22:22.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- 172.21.15.45:0/2340167385 wait complete. 2024-04-25T10:22:22.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 Processor -- start 2024-04-25T10:22:22.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- start start 2024-04-25T10:22:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 0x7f728c145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c1550b0 0x7f728c150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f728c15b910 0x7f728c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c1684c0 con 0x7f728c0cfe70 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0720b0 con 0x7f728c1550b0 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7292e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c168640 con 0x7f728c15b910 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f728be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 0x7f728c145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f728be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 0x7f728c145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49268/0 (socket says 172.21.15.45:49268) 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f728be00640 1 -- 172.21.15.45:0/3434156070 learned_addr learned my addr 172.21.15.45:0/3434156070 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7283400640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c1550b0 0x7f728c150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.686+0000 7f7290e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f728c15b910 0x7f728c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f728be00640 1 -- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f728c15b910 msgr2=0x7f728c14ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f728be00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f728c15b910 0x7f728c14ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f728be00640 1 -- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c1550b0 msgr2=0x7f728c150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f728be00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c1550b0 0x7f728c150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:22.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f728be00640 1 -- 172.21.15.45:0/3434156070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c14f330 con 0x7f728c0cfe70 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f728be00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 0x7f728c145590 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ae040 tx=0x7f727c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0bb070 con 0x7f728c0cfe70 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c004020 con 0x7f728c0cfe70 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b0040 con 0x7f728c0cfe70 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c153fe0 con 0x7f728c0cfe70 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c1545a0 con 0x7f728c0cfe70 2024-04-25T10:22:22.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c14f500 con 0x7f728c0cfe70 2024-04-25T10:22:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f727c003620 con 0x7f728c0cfe70 2024-04-25T10:22:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7288c00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7268066f90 0x7f7268069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.690+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f727c11f3f0 con 0x7f728c0cfe70 2024-04-25T10:22:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.694+0000 7f7283400640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7268066f90 0x7f7268069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:22.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.694+0000 7f7283400640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7268066f90 0x7f7268069450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f728c146260 tx=0x7f7274007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:22.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.694+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f727c016090 con 0x7f728c0cfe70 2024-04-25T10:22:22.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:22.870+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f728c0cbd70 con 0x7f728c0cfe70 2024-04-25T10:22:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.478+0000 7f7288c00640 1 -- 172.21.15.45:0/3434156070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7f727c0f0160 con 0x7f728c0cfe70 2024-04-25T10:22:23.482 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7268066f90 msgr2=0x7f7268069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7268066f90 0x7f7268069450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f728c146260 tx=0x7f7274007240 comp rx=0 tx=0).stop 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 msgr2=0x7f728c145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 0x7f728c145590 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ae040 tx=0x7f727c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 shutdown_connections 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7268066f90 0x7f7268069450 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f728c15b910 0x7f728c14ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c1550b0 0x7f728c150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 --2- 172.21.15.45:0/3434156070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c0cfe70 0x7f728c145590 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 >> 172.21.15.45:0/3434156070 conn(0x7f728c066b70 msgr2=0x7f728c0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 shutdown_connections 2024-04-25T10:22:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.482+0000 7f7292e00640 1 -- 172.21.15.45:0/3434156070 wait complete. 2024-04-25T10:22:23.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:22:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 Processor -- start 2024-04-25T10:22:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 -- start start 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fe40cfe70 0x7f8fe415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe4071050 con 0x7f8fe415b740 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe415d040 con 0x7f8fe40cfe70 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fea800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe40712f0 con 0x7f8fe41550b0 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fe40cfe70 0x7f8fe415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8fe40cfe70 0x7f8fe415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49276/0 (socket says 172.21.15.45:49276) 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 -- 172.21.15.45:0/3843697338 learned_addr learned my addr 172.21.15.45:0/3843697338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fe3400640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 -- 172.21.15.45:0/3843697338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 msgr2=0x7f8fe40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 -- 172.21.15.45:0/3843697338 >> v1:172.21.15.117:6789/0 conn(0x7f8fe415b740 legacy=0x7f8fe406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fdbe00640 1 -- 172.21.15.45:0/3843697338 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe40715d0 con 0x7f8fe40cfe70 2024-04-25T10:22:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.654+0000 7f8fe3400640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fdbe00640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe40cfe70 0x7f8fe415b200 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00bb10 tx=0x7f8fcc01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b9b4289ce18fac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fe2a00640 1 -- 172.21.15.45:0/3843697338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0b4070 con 0x7f8fe40cfe70 2024-04-25T10:22:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fe2a00640 1 -- 172.21.15.45:0/3843697338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc0a7b10 con 0x7f8fe40cfe70 2024-04-25T10:22:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fe2a00640 1 -- 172.21.15.45:0/3843697338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0af440 con 0x7f8fe40cfe70 2024-04-25T10:22:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 -- 172.21.15.45:0/3843697338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe40cfe70 msgr2=0x7f8fe415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe40cfe70 0x7f8fe415b200 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00bb10 tx=0x7f8fcc01ec30 comp rx=0 tx=0).stop 2024-04-25T10:22:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 -- 172.21.15.45:0/3843697338 shutdown_connections 2024-04-25T10:22:23.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe40cfe70 0x7f8fe415b200 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 --2- 172.21.15.45:0/3843697338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 -- 172.21.15.45:0/3843697338 >> 172.21.15.45:0/3843697338 conn(0x7f8fe4066b70 msgr2=0x7f8fe4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:23.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 -- 172.21.15.45:0/3843697338 shutdown_connections 2024-04-25T10:22:23.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.658+0000 7f8fea800640 1 -- 172.21.15.45:0/3843697338 wait complete. 2024-04-25T10:22:23.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 Processor -- start 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- start start 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe40cfe70 0x7f8fe4146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 0x7f8fe4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415bcd0 con 0x7f8fe415b740 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415bb50 con 0x7f8fe40cfe70 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415c030 con 0x7f8fe41550b0 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 0x7f8fe4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 0x7f8fe4153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49286/0 (socket says 172.21.15.45:49286) 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1024908578 learned_addr learned my addr 172.21.15.45:0/1024908578 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3400640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe40cfe70 0x7f8fe4146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 msgr2=0x7f8fe415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe40cfe70 msgr2=0x7f8fe4146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe40cfe70 0x7f8fe4146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 -- 172.21.15.45:0/1024908578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe41f9d40 con 0x7f8fe415b740 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fe3e00640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 0x7f8fe4153ca0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c730 tx=0x7f8fdc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc016020 con 0x7f8fe415b740 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fdc006d60 con 0x7f8fe415b740 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc003bc0 con 0x7f8fe415b740 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe41f9fe0 con 0x7f8fe415b740 2024-04-25T10:22:23.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe41fa5a0 con 0x7f8fe415b740 2024-04-25T10:22:23.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.662+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe404ef30 con 0x7f8fe415b740 2024-04-25T10:22:23.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.666+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fdc0050d0 con 0x7f8fe415b740 2024-04-25T10:22:23.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.666+0000 7f8fdb400640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc0066f90 0x7f8fc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:23.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.666+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8fdc0847a0 con 0x7f8fe415b740 2024-04-25T10:22:23.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.666+0000 7f8fe3400640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc0066f90 0x7f8fc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:23.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.666+0000 7f8fe3400640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc0066f90 0x7f8fc0069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4153240 tx=0x7f8fd4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:23.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.670+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fdc020020 con 0x7f8fe415b740 2024-04-25T10:22:23.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8fe415df00 con 0x7f8fe415b740 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fdb400640 1 -- 172.21.15.45:0/1024908578 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8fdc0554a0 con 0x7f8fe415b740 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.894 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:22:23.895 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:22:23.896 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:22:23.897 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.899 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.900 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:22:23.901 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.902 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:22:23.904 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.905 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:23.907 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:22:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:22:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc0066f90 msgr2=0x7f8fc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc0066f90 0x7f8fc0069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4153240 tx=0x7f8fd4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 msgr2=0x7f8fe4153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 0x7f8fe4153ca0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00c730 tx=0x7f8fdc00cb30 comp rx=0 tx=0).stop 2024-04-25T10:22:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 shutdown_connections 2024-04-25T10:22:23.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8fc0066f90 0x7f8fc0069450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe415b740 0x7f8fe4153ca0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fe41550b0 0x7f8fe415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 --2- 172.21.15.45:0/1024908578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fe40cfe70 0x7f8fe4146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.890+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 >> 172.21.15.45:0/1024908578 conn(0x7f8fe4066b70 msgr2=0x7f8fe40b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.894+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 shutdown_connections 2024-04-25T10:22:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:23.894+0000 7f8fea800640 1 -- 172.21.15.45:0/1024908578 wait complete. 2024-04-25T10:22:23.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:24.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 Processor -- start 2024-04-25T10:22:24.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 -- start start 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f055c1550b0 0x7f055c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c071050 con 0x7f055c15b740 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c15d040 con 0x7f055c1550b0 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0563e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0712f0 con 0x7f055c0cfe70 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f055c1550b0 0x7f055c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f055c1550b0 0x7f055c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49294/0 (socket says 172.21.15.45:49294) 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 -- 172.21.15.45:0/2690124588 learned_addr learned my addr 172.21.15.45:0/2690124588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0560a00640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 -- 172.21.15.45:0/2690124588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 msgr2=0x7f055c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 -- 172.21.15.45:0/2690124588 >> v1:172.21.15.45:6789/0 conn(0x7f055c15b740 legacy=0x7f055c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0561400640 1 -- 172.21.15.45:0/2690124588 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0715d0 con 0x7f055c1550b0 2024-04-25T10:22:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f055be00640 1 -- 172.21.15.45:0/2690124588 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803838080 0 0) 0x7f055c071050 con 0x7f055c15b740 2024-04-25T10:22:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.054+0000 7f0560a00640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0561400640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c0cf930 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0550009870 tx=0x7f055001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cac8d38362631e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f055be00640 1 -- 172.21.15.45:0/2690124588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500b0070 con 0x7f055c1550b0 2024-04-25T10:22:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f055be00640 1 -- 172.21.15.45:0/2690124588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500ab600 con 0x7f055c1550b0 2024-04-25T10:22:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f055be00640 1 -- 172.21.15.45:0/2690124588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500ab9d0 con 0x7f055c1550b0 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 -- 172.21.15.45:0/2690124588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 msgr2=0x7f055c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c0cf930 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0550009870 tx=0x7f055001c560 comp rx=0 tx=0).stop 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 -- 172.21.15.45:0/2690124588 shutdown_connections 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 --2- 172.21.15.45:0/2690124588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c0cf930 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 -- 172.21.15.45:0/2690124588 >> 172.21.15.45:0/2690124588 conn(0x7f055c066b70 msgr2=0x7f055c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 -- 172.21.15.45:0/2690124588 shutdown_connections 2024-04-25T10:22:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.058+0000 7f0563e00640 1 -- 172.21.15.45:0/2690124588 wait complete. 2024-04-25T10:22:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 Processor -- start 2024-04-25T10:22:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- start start 2024-04-25T10:22:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f055c15b740 0x7f055c14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c15bd10 con 0x7f055c1550b0 2024-04-25T10:22:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c15bb90 con 0x7f055c15b740 2024-04-25T10:22:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c15c030 con 0x7f055c0cfe70 2024-04-25T10:22:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0561400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c14d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49306/0 (socket says 172.21.15.45:49306) 2024-04-25T10:22:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 -- 172.21.15.45:0/3707504218 learned_addr learned my addr 172.21.15.45:0/3707504218 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0561e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f055c15b740 0x7f055c14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 -- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 msgr2=0x7f055c149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 -- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f055c15b740 msgr2=0x7f055c14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f055c15b740 0x7f055c14dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 -- 172.21.15.45:0/3707504218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c14e660 con 0x7f055c1550b0 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0561400640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0560a00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c14d6e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f054800c7c0 tx=0x7f054800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548007a30 con 0x7f055c1550b0 2024-04-25T10:22:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0548007bd0 con 0x7f055c1550b0 2024-04-25T10:22:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05480113f0 con 0x7f055c1550b0 2024-04-25T10:22:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c149780 con 0x7f055c1550b0 2024-04-25T10:22:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c0cccd0 con 0x7f055c1550b0 2024-04-25T10:22:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.062+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c04ef30 con 0x7f055c1550b0 2024-04-25T10:22:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.066+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0548011590 con 0x7f055c1550b0 2024-04-25T10:22:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.066+0000 7f0559600640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f053c066f60 0x7f053c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.066+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0548084e40 con 0x7f055c1550b0 2024-04-25T10:22:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.066+0000 7f0561400640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f053c066f60 0x7f053c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.066+0000 7f0561400640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f053c066f60 0x7f053c069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f05500ad000 tx=0x7f05500a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.070+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f055c14e660 con 0x7f055c1550b0 2024-04-25T10:22:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.230+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f055c0b1080 con 0x7f055c1550b0 2024-04-25T10:22:24.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0559600640 1 -- 172.21.15.45:0/3707504218 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f0548015020 con 0x7f055c1550b0 2024-04-25T10:22:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:22:23.491312+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f053c066f60 msgr2=0x7f053c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f053c066f60 0x7f053c069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f05500ad000 tx=0x7f05500a6000 comp rx=0 tx=0).stop 2024-04-25T10:22:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 msgr2=0x7f055c14d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c14d6e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f054800c7c0 tx=0x7f054800cc80 comp rx=0 tx=0).stop 2024-04-25T10:22:24.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 shutdown_connections 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f053c066f60 0x7f053c069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f055c15b740 0x7f055c14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f055c1550b0 0x7f055c14d6e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 --2- 172.21.15.45:0/3707504218 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f055c0cfe70 0x7f055c149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 >> 172.21.15.45:0/3707504218 conn(0x7f055c066b70 msgr2=0x7f055c15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 shutdown_connections 2024-04-25T10:22:24.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.234+0000 7f0563e00640 1 -- 172.21.15.45:0/3707504218 wait complete. 2024-04-25T10:22:24.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:24.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.398+0000 7f4913000640 1 Processor -- start 2024-04-25T10:22:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.398+0000 7f4913000640 1 -- start start 2024-04-25T10:22:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4913000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f490c0cde90 0x7f490c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4913000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06be90 con 0x7f490c0b1f50 2024-04-25T10:22:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4913000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06b8d0 con 0x7f490c0b23c0 2024-04-25T10:22:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4913000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c0d0b50 con 0x7f490c0cde90 2024-04-25T10:22:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4911c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f490c0b1f50 0x7f490c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44988/0 (socket says 172.21.15.45:44988) 2024-04-25T10:22:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4911c00640 1 -- 172.21.15.45:0/1870809867 learned_addr learned my addr 172.21.15.45:0/1870809867 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4912600640 1 --2- 172.21.15.45:0/1870809867 >> v2:172.21.15.117:3300/0 conn(0x7f490c0cde90 0x7f490c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f490be00640 1 -- 172.21.15.45:0/1870809867 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471653611 0 0) 0x7f4900001120 con 0x7f490c0b1f50 2024-04-25T10:22:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f490be00640 1 -- 172.21.15.45:0/1870809867 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f4003530 con 0x7f490c0b1f50 2024-04-25T10:22:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4912600640 1 -- 172.21.15.45:0/1870809867 >> v1:172.21.15.117:6789/0 conn(0x7f490c0b23c0 legacy=0x7f490c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4912600640 1 -- 172.21.15.45:0/1870809867 >> v1:172.21.15.45:6789/0 conn(0x7f490c0b1f50 legacy=0x7f490c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.402+0000 7f4912600640 1 -- 172.21.15.45:0/1870809867 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c06bbb0 con 0x7f490c0cde90 2024-04-25T10:22:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4912600640 1 --2- 172.21.15.45:0/1870809867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0cde90 0x7f490c0b3dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4904009d90 tx=0x7f490401c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74df662299d8ed99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f490be00640 1 -- 172.21.15.45:0/1870809867 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49040b2070 con 0x7f490c0cde90 2024-04-25T10:22:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f490be00640 1 -- 172.21.15.45:0/1870809867 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49040a5820 con 0x7f490c0cde90 2024-04-25T10:22:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f490be00640 1 -- 172.21.15.45:0/1870809867 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49040a5bf0 con 0x7f490c0cde90 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 -- 172.21.15.45:0/1870809867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0cde90 msgr2=0x7f490c0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 --2- 172.21.15.45:0/1870809867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0cde90 0x7f490c0b3dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4904009d90 tx=0x7f490401c850 comp rx=0 tx=0).stop 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 -- 172.21.15.45:0/1870809867 shutdown_connections 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 --2- 172.21.15.45:0/1870809867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0cde90 0x7f490c0b3dd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 -- 172.21.15.45:0/1870809867 >> 172.21.15.45:0/1870809867 conn(0x7f490c0b32b0 msgr2=0x7f490c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 -- 172.21.15.45:0/1870809867 shutdown_connections 2024-04-25T10:22:24.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 -- 172.21.15.45:0/1870809867 wait complete. 2024-04-25T10:22:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 Processor -- start 2024-04-25T10:22:24.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.406+0000 7f4913000640 1 -- start start 2024-04-25T10:22:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0b1f50 0x7f490c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 0x7f490c0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490c0cde90 0x7f490c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c16ba20 con 0x7f490c0b23c0 2024-04-25T10:22:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0d04d0 con 0x7f490c0b1f50 2024-04-25T10:22:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c16bba0 con 0x7f490c0cde90 2024-04-25T10:22:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 0x7f490c0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 0x7f490c0af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49318/0 (socket says 172.21.15.45:49318) 2024-04-25T10:22:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 -- 172.21.15.45:0/1169657489 learned_addr learned my addr 172.21.15.45:0/1169657489 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 -- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490c0cde90 msgr2=0x7f490c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490c0cde90 0x7f490c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 -- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0b1f50 msgr2=0x7f490c157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0b1f50 0x7f490c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 -- 172.21.15.45:0/1169657489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0b0030 con 0x7f490c0b23c0 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4911200640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 0x7f490c0af5e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f48f800c3a0 tx=0x7f48f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80052d0 con 0x7f490c0b23c0 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f8005470 con 0x7f490c0b23c0 2024-04-25T10:22:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c0b0310 con 0x7f490c0b23c0 2024-04-25T10:22:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8007260 con 0x7f490c0b23c0 2024-04-25T10:22:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c0b08d0 con 0x7f490c0b23c0 2024-04-25T10:22:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c075120 con 0x7f490c0b23c0 2024-04-25T10:22:24.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.410+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48f8007400 con 0x7f490c0b23c0 2024-04-25T10:22:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.414+0000 7f4909600640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48ec066f90 0x7f48ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.414+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f48f8084170 con 0x7f490c0b23c0 2024-04-25T10:22:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.414+0000 7f4911c00640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48ec066f90 0x7f48ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.414+0000 7f4911c00640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48ec066f90 0x7f48ec069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4900005890 tx=0x7f4900001160 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.418+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48f80554d0 con 0x7f490c0b23c0 2024-04-25T10:22:24.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f490c06eeb0 con 0x7f490c0b23c0 2024-04-25T10:22:24.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4909600640 1 -- 172.21.15.45:0/1169657489 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48f8054dd0 con 0x7f490c0b23c0 2024-04-25T10:22:24.586 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48ec066f90 msgr2=0x7f48ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48ec066f90 0x7f48ec069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4900005890 tx=0x7f4900001160 comp rx=0 tx=0).stop 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 msgr2=0x7f490c0af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 0x7f490c0af5e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f48f800c3a0 tx=0x7f48f800c860 comp rx=0 tx=0).stop 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 shutdown_connections 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48ec066f90 0x7f48ec069450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f490c0cde90 0x7f490c0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f490c0b23c0 0x7f490c0af5e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 --2- 172.21.15.45:0/1169657489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f490c0b1f50 0x7f490c157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 >> 172.21.15.45:0/1169657489 conn(0x7f490c0b32b0 msgr2=0x7f490c06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:24.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 shutdown_connections 2024-04-25T10:22:24.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.582+0000 7f4913000640 1 -- 172.21.15.45:0/1169657489 wait complete. 2024-04-25T10:22:24.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:22:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 Processor -- start 2024-04-25T10:22:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 -- start start 2024-04-25T10:22:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05c4156150 0x7f05c4071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05c40670b0 0x7f05c40af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c4156530 con 0x7f05c415c080 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c415d220 con 0x7f05c4156150 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05cc400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c415c640 con 0x7f05c40670b0 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05c4156150 0x7f05c4071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05c4156150 0x7f05c4071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49324/0 (socket says 172.21.15.45:49324) 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 -- 172.21.15.45:0/3211589731 learned_addr learned my addr 172.21.15.45:0/3211589731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05ca400640 1 --2- 172.21.15.45:0/3211589731 >> v2:172.21.15.117:3300/0 conn(0x7f05c40670b0 0x7f05c40af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 -- 172.21.15.45:0/3211589731 >> v2:172.21.15.117:3300/0 conn(0x7f05c40670b0 msgr2=0x7f05c40af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 --2- 172.21.15.45:0/3211589731 >> v2:172.21.15.117:3300/0 conn(0x7f05c40670b0 0x7f05c40af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 -- 172.21.15.45:0/3211589731 >> v1:172.21.15.117:6789/0 conn(0x7f05c415c080 legacy=0x7f05c40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.746+0000 7f05c9a00640 1 -- 172.21.15.45:0/3211589731 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c415c920 con 0x7f05c4156150 2024-04-25T10:22:24.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05c9a00640 1 --2- 172.21.15.45:0/3211589731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c4071f30 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f05c406bcb0 tx=0x7f05b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12fd510b1ded1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05c3e00640 1 -- 172.21.15.45:0/3211589731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80aa650 con 0x7f05c4156150 2024-04-25T10:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05c3e00640 1 -- 172.21.15.45:0/3211589731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b80af070 con 0x7f05c4156150 2024-04-25T10:22:24.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05c3e00640 1 -- 172.21.15.45:0/3211589731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80aaa10 con 0x7f05c4156150 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 -- 172.21.15.45:0/3211589731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 msgr2=0x7f05c4071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 --2- 172.21.15.45:0/3211589731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c4071f30 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f05c406bcb0 tx=0x7f05b801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 -- 172.21.15.45:0/3211589731 shutdown_connections 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 --2- 172.21.15.45:0/3211589731 >> v2:172.21.15.117:3300/0 conn(0x7f05c40670b0 0x7f05c40af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 --2- 172.21.15.45:0/3211589731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c4071f30 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 -- 172.21.15.45:0/3211589731 >> 172.21.15.45:0/3211589731 conn(0x7f05c406ac90 msgr2=0x7f05c406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 -- 172.21.15.45:0/3211589731 shutdown_connections 2024-04-25T10:22:24.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.750+0000 7f05cc400640 1 -- 172.21.15.45:0/3211589731 wait complete. 2024-04-25T10:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 Processor -- start 2024-04-25T10:22:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- start start 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40670b0 0x7f05c40b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c41fa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415c080 0x7f05c41fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415d880 con 0x7f05c4156150 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415d700 con 0x7f05c415c080 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415da00 con 0x7f05c40670b0 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c41fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c41fa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49334/0 (socket says 172.21.15.45:49334) 2024-04-25T10:22:24.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 -- 172.21.15.45:0/250042824 learned_addr learned my addr 172.21.15.45:0/250042824 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9a00640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40670b0 0x7f05c40b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05ca400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415c080 0x7f05c41fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 -- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40670b0 msgr2=0x7f05c40b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40670b0 0x7f05c40b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 -- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415c080 msgr2=0x7f05c41fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415c080 0x7f05c41fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 -- 172.21.15.45:0/250042824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c41fb0e0 con 0x7f05c4156150 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05ca400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415c080 0x7f05c41fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:24.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c9000640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c41fa370 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c3a0 tx=0x7f05b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b00052d0 con 0x7f05c4156150 2024-04-25T10:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0005470 con 0x7f05c4156150 2024-04-25T10:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0007260 con 0x7f05c4156150 2024-04-25T10:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c40ab440 con 0x7f05c4156150 2024-04-25T10:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c40aba00 con 0x7f05c4156150 2024-04-25T10:22:24.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.754+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c404ef30 con 0x7f05c4156150 2024-04-25T10:22:24.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.758+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05b0007400 con 0x7f05c4156150 2024-04-25T10:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.758+0000 7f05c1600640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f05a4066f60 0x7f05a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.758+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05b0084160 con 0x7f05c4156150 2024-04-25T10:22:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.758+0000 7f05c9a00640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f05a4066f60 0x7f05a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:24.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.758+0000 7f05c9a00640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f05a4066f60 0x7f05a4069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f05b80ae000 tx=0x7f05b80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:24.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.762+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05b0054dd0 con 0x7f05c4156150 2024-04-25T10:22:24.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.926+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f05c40d0ea0 con 0x7f05c4156150 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.926+0000 7f05c1600640 1 -- 172.21.15.45:0/250042824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7f05b0058e70 con 0x7f05c4156150 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:22:24.930 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.926+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f05a4066f60 msgr2=0x7f05a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.926+0000 7f05cc400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f05a4066f60 0x7f05a4069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f05b80ae000 tx=0x7f05b80a7000 comp rx=0 tx=0).stop 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.926+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 msgr2=0x7f05c41fa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.926+0000 7f05cc400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c41fa370 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c3a0 tx=0x7f05b000c860 comp rx=0 tx=0).stop 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 shutdown_connections 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f05a4066f60 0x7f05a4069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415c080 0x7f05c41fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c4156150 0x7f05c41fa370 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 --2- 172.21.15.45:0/250042824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40670b0 0x7f05c40b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:24.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 >> 172.21.15.45:0/250042824 conn(0x7f05c406ac90 msgr2=0x7f05c4154fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:24.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 shutdown_connections 2024-04-25T10:22:24.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:24.930+0000 7f05cc400640 1 -- 172.21.15.45:0/250042824 wait complete. 2024-04-25T10:22:24.951 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:22:24.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:22:25.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 Processor -- start 2024-04-25T10:22:25.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 -- start start 2024-04-25T10:22:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f718815c0f0 0x7f71880b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71880cf9e0 0x7f71880cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718806c0e0 con 0x7f718815bd10 2024-04-25T10:22:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71880d0390 con 0x7f718815c0f0 2024-04-25T10:22:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718815c6a0 con 0x7f71880cf9e0 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71880cf9e0 0x7f71880cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718d000640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f718815bd10 0x7f71880b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:58916/0 (socket says 172.21.15.45:58916) 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f7187e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f718815c0f0 0x7f71880b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718d000640 1 -- 172.21.15.45:0/2891628166 learned_addr learned my addr 172.21.15.45:0/2891628166 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718da00640 1 -- 172.21.15.45:0/2891628166 >> v2:172.21.15.117:3300/0 conn(0x7f718815c0f0 msgr2=0x7f71880b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718da00640 1 --2- 172.21.15.45:0/2891628166 >> v2:172.21.15.117:3300/0 conn(0x7f718815c0f0 0x7f71880b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718da00640 1 -- 172.21.15.45:0/2891628166 >> v1:172.21.15.117:6789/0 conn(0x7f718815bd10 legacy=0x7f71880b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f7187400640 1 -- 172.21.15.45:0/2891628166 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303865720 0 0) 0x7f718806c0e0 con 0x7f718815bd10 2024-04-25T10:22:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.094+0000 7f718da00640 1 -- 172.21.15.45:0/2891628166 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718815c980 con 0x7f71880cf9e0 2024-04-25T10:22:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718da00640 1 --2- 172.21.15.45:0/2891628166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71880cf9e0 0x7f71880cfe50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7180009d90 tx=0x7f718001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe80a5d866119146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f7187400640 1 -- 172.21.15.45:0/2891628166 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71800b2070 con 0x7f71880cf9e0 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f7187400640 1 -- 172.21.15.45:0/2891628166 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71800a5920 con 0x7f71880cf9e0 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f7187400640 1 -- 172.21.15.45:0/2891628166 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71800a5cf0 con 0x7f71880cf9e0 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 -- 172.21.15.45:0/2891628166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71880cf9e0 msgr2=0x7f71880cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 --2- 172.21.15.45:0/2891628166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71880cf9e0 0x7f71880cfe50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7180009d90 tx=0x7f718001c970 comp rx=0 tx=0).stop 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 -- 172.21.15.45:0/2891628166 shutdown_connections 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 --2- 172.21.15.45:0/2891628166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71880cf9e0 0x7f71880cfe50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 --2- 172.21.15.45:0/2891628166 >> v2:172.21.15.117:3300/0 conn(0x7f718815c0f0 0x7f71880b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 -- 172.21.15.45:0/2891628166 >> 172.21.15.45:0/2891628166 conn(0x7f7188066b70 msgr2=0x7f7188066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 -- 172.21.15.45:0/2891628166 shutdown_connections 2024-04-25T10:22:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.098+0000 7f718fa00640 1 -- 172.21.15.45:0/2891628166 wait complete. 2024-04-25T10:22:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 Processor -- start 2024-04-25T10:22:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- start start 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 0x7f7188152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f718815bd10 0x7f718814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f718815c0f0 0x7f71880b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718806cf70 con 0x7f71880cf9e0 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718806cdf0 con 0x7f718815c0f0 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7188168ac0 con 0x7f718815bd10 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 0x7f7188152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 0x7f7188152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46108/0 (socket says 172.21.15.45:46108) 2024-04-25T10:22:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 -- 172.21.15.45:0/2397214343 learned_addr learned my addr 172.21.15.45:0/2397214343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f7187e00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f718815bd10 0x7f718814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718da00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f718815c0f0 0x7f71880b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 -- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f718815bd10 msgr2=0x7f718814d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f718815bd10 0x7f718814d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 -- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f718815c0f0 msgr2=0x7f71880b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f718815c0f0 0x7f71880b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 -- 172.21.15.45:0/2397214343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71880af2d0 con 0x7f71880cf9e0 2024-04-25T10:22:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718d000640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 0x7f7188152340 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c520 tx=0x7f717c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c018070 con 0x7f71880cf9e0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c0074e0 con 0x7f71880cf9e0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71880af5b0 con 0x7f71880cf9e0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0049f0 con 0x7f71880cf9e0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718815ad70 con 0x7f71880cf9e0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.102+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71880af7d0 con 0x7f71880cf9e0 2024-04-25T10:22:25.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.106+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f717c01b070 con 0x7f71880cf9e0 2024-04-25T10:22:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.106+0000 7f7184c00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7168066f60 0x7f7168069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.106+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f717c0840e0 con 0x7f71880cf9e0 2024-04-25T10:22:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.106+0000 7f7187e00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7168066f60 0x7f7168069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.106+0000 7f7187e00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7168066f60 0x7f7168069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7188156d70 tx=0x7f7174009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:25.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.110+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f717c054de0 con 0x7f71880cf9e0 2024-04-25T10:22:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.274+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f718804ef30 con 0x7f71880cf9e0 2024-04-25T10:22:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.486+0000 7f7184c00640 1 -- 172.21.15.45:0/2397214343 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f717c058e80 con 0x7f71880cf9e0 2024-04-25T10:22:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:22:25.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7168066f60 msgr2=0x7f7168069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7168066f60 0x7f7168069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7188156d70 tx=0x7f7174009190 comp rx=0 tx=0).stop 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 msgr2=0x7f7188152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 0x7f7188152340 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c520 tx=0x7f717c00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 shutdown_connections 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7168066f60 0x7f7168069420 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f718815c0f0 0x7f71880b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f718815bd10 0x7f718814d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 --2- 172.21.15.45:0/2397214343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71880cf9e0 0x7f7188152340 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 >> 172.21.15.45:0/2397214343 conn(0x7f7188066b70 msgr2=0x7f718815d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 shutdown_connections 2024-04-25T10:22:25.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.490+0000 7f718fa00640 1 -- 172.21.15.45:0/2397214343 wait complete. 2024-04-25T10:22:25.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:22:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 Processor -- start 2024-04-25T10:22:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 -- start start 2024-04-25T10:22:25.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5da815bbb0 0x7f5da80cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5da80b4660 0x7f5da806c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da806ca90 con 0x7f5da80cf960 2024-04-25T10:22:25.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da806d050 con 0x7f5da815bbb0 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.662+0000 7f5daec00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da806cd70 con 0x7f5da80b4660 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5da815bbb0 0x7f5da80cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5da815bbb0 0x7f5da80cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46118/0 (socket says 172.21.15.45:46118) 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 -- 172.21.15.45:0/3375532667 learned_addr learned my addr 172.21.15.45:0/3375532667 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5d9f400640 1 --2- 172.21.15.45:0/3375532667 >> v2:172.21.15.117:3300/0 conn(0x7f5da80b4660 0x7f5da806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 -- 172.21.15.45:0/3375532667 >> v2:172.21.15.117:3300/0 conn(0x7f5da80b4660 msgr2=0x7f5da806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 --2- 172.21.15.45:0/3375532667 >> v2:172.21.15.117:3300/0 conn(0x7f5da80b4660 0x7f5da806c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 -- 172.21.15.45:0/3375532667 >> v1:172.21.15.117:6789/0 conn(0x7f5da80cf960 legacy=0x7f5da80b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 -- 172.21.15.45:0/3375532667 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da80b2880 con 0x7f5da815bbb0 2024-04-25T10:22:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5d9f400640 1 --2- 172.21.15.45:0/3375532667 >> v2:172.21.15.117:3300/0 conn(0x7f5da80b4660 0x7f5da806c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7e00640 1 --2- 172.21.15.45:0/3375532667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da80cda80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5da80b25a0 tx=0x7f5d9801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130c5ac2540418ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7400640 1 -- 172.21.15.45:0/3375532667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980b1070 con 0x7f5da815bbb0 2024-04-25T10:22:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7400640 1 -- 172.21.15.45:0/3375532667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d980ac630 con 0x7f5da815bbb0 2024-04-25T10:22:25.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.666+0000 7f5da7400640 1 -- 172.21.15.45:0/3375532667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980a8050 con 0x7f5da815bbb0 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- 172.21.15.45:0/3375532667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 msgr2=0x7f5da80cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 --2- 172.21.15.45:0/3375532667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da80cda80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5da80b25a0 tx=0x7f5d9801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- 172.21.15.45:0/3375532667 shutdown_connections 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 --2- 172.21.15.45:0/3375532667 >> v2:172.21.15.117:3300/0 conn(0x7f5da80b4660 0x7f5da806c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 --2- 172.21.15.45:0/3375532667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da80cda80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- 172.21.15.45:0/3375532667 >> 172.21.15.45:0/3375532667 conn(0x7f5da8066b70 msgr2=0x7f5da8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:25.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- 172.21.15.45:0/3375532667 shutdown_connections 2024-04-25T10:22:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- 172.21.15.45:0/3375532667 wait complete. 2024-04-25T10:22:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 Processor -- start 2024-04-25T10:22:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- start start 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5da80b4660 0x7f5da813f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5da80cf960 0x7f5da8153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da80ce140 con 0x7f5da815bbb0 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da80cdfc0 con 0x7f5da80b4660 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5daec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da806c660 con 0x7f5da80cf960 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da8153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46130/0 (socket says 172.21.15.45:46130) 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 -- 172.21.15.45:0/2494471568 learned_addr learned my addr 172.21.15.45:0/2494471568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5da7e00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5da80b4660 0x7f5da813f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5d9f400640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5da80cf960 0x7f5da8153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 -- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5da80cf960 msgr2=0x7f5da8153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5da80cf960 0x7f5da8153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 -- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5da80b4660 msgr2=0x7f5da813f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5da80b4660 0x7f5da813f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5dacc00640 1 -- 172.21.15.45:0/2494471568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da813f6f0 con 0x7f5da815bbb0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.670+0000 7f5da7e00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5da80b4660 0x7f5da813f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5dacc00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da8153d60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5da000c780 tx=0x7f5da000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0004b70 con 0x7f5da815bbb0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da0004d10 con 0x7f5da815bbb0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da8155950 con 0x7f5da815bbb0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da000aa90 con 0x7f5da815bbb0 2024-04-25T10:22:25.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da8155f10 con 0x7f5da815bbb0 2024-04-25T10:22:25.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da804ef30 con 0x7f5da815bbb0 2024-04-25T10:22:25.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5da000ac30 con 0x7f5da815bbb0 2024-04-25T10:22:25.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da4c00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d84066f90 0x7f5d84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:25.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5da0085140 con 0x7f5da815bbb0 2024-04-25T10:22:25.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.674+0000 7f5da7e00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d84066f90 0x7f5d84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:25.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.678+0000 7f5da7e00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d84066f90 0x7f5d84069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5d980ae000 tx=0x7f5d98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.682+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5da0020070 con 0x7f5da815bbb0 2024-04-25T10:22:25.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:25.842+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5da80cf2f0 con 0x7f5da815bbb0 2024-04-25T10:22:26.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.490+0000 7f5da4c00640 1 -- 172.21.15.45:0/2494471568 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5da0056290 con 0x7f5da815bbb0 2024-04-25T10:22:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:22:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d84066f90 msgr2=0x7f5d84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d84066f90 0x7f5d84069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5d980ae000 tx=0x7f5d98008040 comp rx=0 tx=0).stop 2024-04-25T10:22:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 msgr2=0x7f5da8153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da8153d60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5da000c780 tx=0x7f5da000cc40 comp rx=0 tx=0).stop 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 shutdown_connections 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d84066f90 0x7f5d84069450 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5da815bbb0 0x7f5da8153d60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5da80cf960 0x7f5da8153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 --2- 172.21.15.45:0/2494471568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5da80b4660 0x7f5da813f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 >> 172.21.15.45:0/2494471568 conn(0x7f5da8066b70 msgr2=0x7f5da8151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:26.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 shutdown_connections 2024-04-25T10:22:26.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.494+0000 7f5daec00640 1 -- 172.21.15.45:0/2494471568 wait complete. 2024-04-25T10:22:26.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:22:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.678+0000 7fbb23c00640 1 Processor -- start 2024-04-25T10:22:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.678+0000 7fbb23c00640 1 -- start start 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb23c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb1c0cc200 0x7fbb1c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb23c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0cdab0 con 0x7fbb1c06cf40 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb23c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0cd210 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb23c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cd4f0 con 0x7fbb1c0cc200 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb1c0cc200 0x7fbb1c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb1c0cc200 0x7fbb1c0ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46132/0 (socket says 172.21.15.45:46132) 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 -- 172.21.15.45:0/2405778825 learned_addr learned my addr 172.21.15.45:0/2405778825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:26.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb21c00640 1 --1- 172.21.15.45:0/2405778825 >> v1:172.21.15.45:6789/0 conn(0x7fbb1c06cf40 0x7fbb1c15d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59082/0 (socket says 172.21.15.45:59082) 2024-04-25T10:22:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 -- 172.21.15.45:0/2405778825 >> v1:172.21.15.117:6789/0 conn(0x7fbb1c0d0d00 legacy=0x7fbb1c162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 -- 172.21.15.45:0/2405778825 >> v1:172.21.15.45:6789/0 conn(0x7fbb1c06cf40 legacy=0x7fbb1c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 -- 172.21.15.45:0/2405778825 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0cd7d0 con 0x7fbb1c0cc200 2024-04-25T10:22:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb13e00640 1 --2- 172.21.15.45:0/2405778825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0ccc30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009870 tx=0x7fbb0401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eed24380a1625ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:26.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb1be00640 1 -- 172.21.15.45:0/2405778825 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040b0070 con 0x7fbb1c0cc200 2024-04-25T10:22:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb1be00640 1 -- 172.21.15.45:0/2405778825 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb040ab600 con 0x7fbb1c0cc200 2024-04-25T10:22:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.682+0000 7fbb1be00640 1 -- 172.21.15.45:0/2405778825 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040ab9d0 con 0x7fbb1c0cc200 2024-04-25T10:22:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- 172.21.15.45:0/2405778825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0cc200 msgr2=0x7fbb1c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 --2- 172.21.15.45:0/2405778825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0ccc30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009870 tx=0x7fbb0401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- 172.21.15.45:0/2405778825 shutdown_connections 2024-04-25T10:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 --2- 172.21.15.45:0/2405778825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0ccc30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- 172.21.15.45:0/2405778825 >> 172.21.15.45:0/2405778825 conn(0x7fbb1c072290 msgr2=0x7fbb1c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- 172.21.15.45:0/2405778825 shutdown_connections 2024-04-25T10:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- 172.21.15.45:0/2405778825 wait complete. 2024-04-25T10:22:26.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 Processor -- start 2024-04-25T10:22:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- start start 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb1c06cf40 0x7fbb1c0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 0x7fbb1c15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c162a10 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c162890 con 0x7fbb1c0cc200 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb23c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c16c690 con 0x7fbb1c06cf40 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 0x7fbb1c15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 0x7fbb1c15bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46146/0 (socket says 172.21.15.45:46146) 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 -- 172.21.15.45:0/4042541159 learned_addr learned my addr 172.21.15.45:0/4042541159 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb13e00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21200640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb1c06cf40 0x7fbb1c0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 -- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb1c06cf40 msgr2=0x7fbb1c0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb1c06cf40 0x7fbb1c0999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 -- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb1c0cc200 msgr2=0x7fbb1c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.686+0000 7fbb21c00640 1 -- 172.21.15.45:0/4042541159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c1615b0 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb21c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 0x7fbb1c15bb90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c730 tx=0x7fbb1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14017070 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400ce40 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c15b310 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400a840 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c161240 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c15b830 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.690+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb1401a070 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.694+0000 7fbb19600640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbaf8066f60 0x7fbaf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.694+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbb14084950 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.694+0000 7fbb21200640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbaf8066f60 0x7fbaf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.694+0000 7fbb21200640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbaf8066f60 0x7fbaf8069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c15fb40 tx=0x7fbb0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:26.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.694+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb14055350 con 0x7fbb1c0d0d00 2024-04-25T10:22:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:26.894+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbb1c04ef30 con 0x7fbb1c0d0d00 2024-04-25T10:22:27.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.514+0000 7fbb19600640 1 -- 172.21.15.45:0/4042541159 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbb140593f0 con 0x7fbb1c0d0d00 2024-04-25T10:22:27.519 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T10:22:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbaf8066f60 msgr2=0x7fbaf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbaf8066f60 0x7fbaf8069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c15fb40 tx=0x7fbb0c009040 comp rx=0 tx=0).stop 2024-04-25T10:22:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 msgr2=0x7fbb1c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 0x7fbb1c15bb90 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c730 tx=0x7fbb1400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 shutdown_connections 2024-04-25T10:22:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbaf8066f60 0x7fbaf8069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb1c0d0d00 0x7fbb1c15bb90 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb1c0cc200 0x7fbb1c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 --2- 172.21.15.45:0/4042541159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb1c06cf40 0x7fbb1c0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 >> 172.21.15.45:0/4042541159 conn(0x7fbb1c072290 msgr2=0x7fbb1c0ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:27.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 shutdown_connections 2024-04-25T10:22:27.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.522+0000 7fbb23c00640 1 -- 172.21.15.45:0/4042541159 wait complete. 2024-04-25T10:22:27.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.710+0000 7fb6bdc00640 1 Processor -- start 2024-04-25T10:22:27.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.710+0000 7fb6bdc00640 1 -- start start 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6bdc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6b80d0040 0x7fb6b815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6bdc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b815b740 0x7fb6b80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6bdc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b8071050 con 0x7fb6b81550b0 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6bdc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b815d040 con 0x7fb6b80d0040 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6bdc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80712f0 con 0x7fb6b815b740 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b6a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6b81550b0 0x7fb6b80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59090/0 (socket says 172.21.15.45:59090) 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b6a00640 1 -- 172.21.15.45:0/2953955037 learned_addr learned my addr 172.21.15.45:0/2953955037 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b7400640 1 --2- 172.21.15.45:0/2953955037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b815b740 0x7fb6b80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6afe00640 1 --2- 172.21.15.45:0/2953955037 >> v2:172.21.15.117:3300/0 conn(0x7fb6b80d0040 0x7fb6b815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b6000640 1 -- 172.21.15.45:0/2953955037 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522818466 0 0) 0x7fb6b8071050 con 0x7fb6b81550b0 2024-04-25T10:22:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b6000640 1 -- 172.21.15.45:0/2953955037 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb69c003530 con 0x7fb6b81550b0 2024-04-25T10:22:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b7400640 1 -- 172.21.15.45:0/2953955037 >> v2:172.21.15.117:3300/0 conn(0x7fb6b80d0040 msgr2=0x7fb6b815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b7400640 1 --2- 172.21.15.45:0/2953955037 >> v2:172.21.15.117:3300/0 conn(0x7fb6b80d0040 0x7fb6b815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b7400640 1 -- 172.21.15.45:0/2953955037 >> v1:172.21.15.45:6789/0 conn(0x7fb6b81550b0 legacy=0x7fb6b80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.714+0000 7fb6b7400640 1 -- 172.21.15.45:0/2953955037 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80715d0 con 0x7fb6b815b740 2024-04-25T10:22:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6b7400640 1 --2- 172.21.15.45:0/2953955037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b815b740 0x7fb6b80b3260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009d90 tx=0x7fb6b001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb78be507a4d0242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:27.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6b6000640 1 -- 172.21.15.45:0/2953955037 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00b2070 con 0x7fb6b815b740 2024-04-25T10:22:27.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6b6000640 1 -- 172.21.15.45:0/2953955037 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b00a5a00 con 0x7fb6b815b740 2024-04-25T10:22:27.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6b6000640 1 -- 172.21.15.45:0/2953955037 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00a5db0 con 0x7fb6b815b740 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2953955037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b815b740 msgr2=0x7fb6b80b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2953955037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b815b740 0x7fb6b80b3260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009d90 tx=0x7fb6b001c940 comp rx=0 tx=0).stop 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2953955037 shutdown_connections 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2953955037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b815b740 0x7fb6b80b3260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2953955037 >> v2:172.21.15.117:3300/0 conn(0x7fb6b80d0040 0x7fb6b815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2953955037 >> 172.21.15.45:0/2953955037 conn(0x7fb6b8066b70 msgr2=0x7fb6b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2953955037 shutdown_connections 2024-04-25T10:22:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2953955037 wait complete. 2024-04-25T10:22:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 Processor -- start 2024-04-25T10:22:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.718+0000 7fb6bdc00640 1 -- start start 2024-04-25T10:22:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b80d0040 0x7fb6b80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 0x7fb6b80cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b815b740 0x7fb6b8155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b815bd50 con 0x7fb6b81550b0 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b815bbd0 con 0x7fb6b815b740 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b815c030 con 0x7fb6b80d0040 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 0x7fb6b80cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 0x7fb6b80cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46148/0 (socket says 172.21.15.45:46148) 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 -- 172.21.15.45:0/2465196267 learned_addr learned my addr 172.21.15.45:0/2465196267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6b6a00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b80d0040 0x7fb6b80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6b7400640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b815b740 0x7fb6b8155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 -- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b80d0040 msgr2=0x7fb6b80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b80d0040 0x7fb6b80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 -- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b815b740 msgr2=0x7fb6b8155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b815b740 0x7fb6b8155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 -- 172.21.15.45:0/2465196267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80cdba0 con 0x7fb6b81550b0 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6b7400640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b815b740 0x7fb6b8155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6afe00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 0x7fb6b80cec40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e960 tx=0x7fb6a000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0017070 con 0x7fb6b81550b0 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a000c9a0 con 0x7fb6b81550b0 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000cd50 con 0x7fb6b81550b0 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b80cde80 con 0x7fb6b81550b0 2024-04-25T10:22:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b80ce440 con 0x7fb6b81550b0 2024-04-25T10:22:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b804ef30 con 0x7fb6b81550b0 2024-04-25T10:22:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.722+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6a001a070 con 0x7fb6b81550b0 2024-04-25T10:22:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.726+0000 7fb6aea00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb694066f60 0x7fb694069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.726+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6a00853d0 con 0x7fb6b81550b0 2024-04-25T10:22:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.726+0000 7fb6b6a00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb694066f60 0x7fb694069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.726+0000 7fb6b6a00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb694066f60 0x7fb694069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8007fd0 tx=0x7fb6a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.730+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6a0055cf0 con 0x7fb6b81550b0 2024-04-25T10:22:27.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:27.860 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:27.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.894+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6b806b390 con 0x7fb6b81550b0 2024-04-25T10:22:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.894+0000 7fb6aea00640 1 -- 172.21.15.45:0/2465196267 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16609 (secure 0 0 0) 0x7fb6a0059d90 con 0x7fb6b81550b0 2024-04-25T10:22:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:27.898 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":76,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:22:26.898465+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb694066f60 msgr2=0x7fb694069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb694066f60 0x7fb694069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8007fd0 tx=0x7fb6a8009040 comp rx=0 tx=0).stop 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 msgr2=0x7fb6b80cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 0x7fb6b80cec40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e960 tx=0x7fb6a000ee20 comp rx=0 tx=0).stop 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 shutdown_connections 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb694066f60 0x7fb694069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6b815b740 0x7fb6b8155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b81550b0 0x7fb6b80cec40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 --2- 172.21.15.45:0/2465196267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6b80d0040 0x7fb6b80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 >> 172.21.15.45:0/2465196267 conn(0x7fb6b8066b70 msgr2=0x7fb6b806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 shutdown_connections 2024-04-25T10:22:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:27.898+0000 7fb6bdc00640 1 -- 172.21.15.45:0/2465196267 wait complete. 2024-04-25T10:22:27.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 Processor -- start 2024-04-25T10:22:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 -- start start 2024-04-25T10:22:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff90815c080 0x7ff9080669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff908066ee0 0x7ff9080af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff908156530 con 0x7ff908156150 2024-04-25T10:22:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff908067350 con 0x7ff908066ee0 2024-04-25T10:22:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90ec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815d220 con 0x7ff90815c080 2024-04-25T10:22:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff907400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff90815c080 0x7ff9080669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90cc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff908066ee0 0x7ff9080af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff907e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff908156150 0x7ff908071f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:58940/0 (socket says 172.21.15.45:58940) 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff907400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff90815c080 0x7ff9080669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:49212/0 (socket says 172.21.15.45:49212) 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90cc00640 1 --2- 172.21.15.45:0/1390044484 >> v2:172.21.15.117:3300/0 conn(0x7ff908066ee0 0x7ff9080af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45788/0 (socket says 172.21.15.45:45788) 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff907e00640 1 -- 172.21.15.45:0/1390044484 learned_addr learned my addr 172.21.15.45:0/1390044484 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90cc00640 1 -- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff90815c080 msgr2=0x7ff9080669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90cc00640 1 --2- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff90815c080 0x7ff9080669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90cc00640 1 -- 172.21.15.45:0/1390044484 >> v1:172.21.15.117:6789/0 conn(0x7ff908156150 legacy=0x7ff908071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.102+0000 7ff90cc00640 1 -- 172.21.15.45:0/1390044484 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90815c640 con 0x7ff908066ee0 2024-04-25T10:22:28.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90cc00640 1 --2- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff908066ee0 0x7ff9080af490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff900009d90 tx=0x7ff90001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5b562f4d82a0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff906a00640 1 -- 172.21.15.45:0/1390044484 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000b2070 con 0x7ff908066ee0 2024-04-25T10:22:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff906a00640 1 -- 172.21.15.45:0/1390044484 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9000a5910 con 0x7ff908066ee0 2024-04-25T10:22:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff906a00640 1 -- 172.21.15.45:0/1390044484 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9000a5ce0 con 0x7ff908066ee0 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 -- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff908066ee0 msgr2=0x7ff9080af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 --2- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff908066ee0 0x7ff9080af490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff900009d90 tx=0x7ff90001c940 comp rx=0 tx=0).stop 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 -- 172.21.15.45:0/1390044484 shutdown_connections 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 --2- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff908066ee0 0x7ff9080af490 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 --2- 172.21.15.45:0/1390044484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff90815c080 0x7ff9080669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 -- 172.21.15.45:0/1390044484 >> 172.21.15.45:0/1390044484 conn(0x7ff90806ac90 msgr2=0x7ff90806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 -- 172.21.15.45:0/1390044484 shutdown_connections 2024-04-25T10:22:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.106+0000 7ff90ec00640 1 -- 172.21.15.45:0/1390044484 wait complete. 2024-04-25T10:22:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 Processor -- start 2024-04-25T10:22:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- start start 2024-04-25T10:22:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff908066ee0 0x7ff90814a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 0x7ff908155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff90815c080 0x7ff908153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815d880 con 0x7ff908156150 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815d700 con 0x7ff90815c080 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90815da00 con 0x7ff908066ee0 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 0x7ff908155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 0x7ff908155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46162/0 (socket says 172.21.15.45:46162) 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 -- 172.21.15.45:0/4003272193 learned_addr learned my addr 172.21.15.45:0/4003272193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 -- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff908066ee0 msgr2=0x7ff90814a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907e00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff908066ee0 0x7ff90814a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90cc00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff90815c080 0x7ff908153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff908066ee0 0x7ff90814a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 -- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff90815c080 msgr2=0x7ff908153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff90815c080 0x7ff908153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 -- 172.21.15.45:0/4003272193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90814a7e0 con 0x7ff908156150 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90cc00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff90815c080 0x7ff908153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907e00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff908066ee0 0x7ff90814a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff907400640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 0x7ff908155030 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e870 tx=0x7ff8f400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4017070 con 0x7ff908156150 2024-04-25T10:22:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f400c940 con 0x7ff908156150 2024-04-25T10:22:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff908153f90 con 0x7ff908156150 2024-04-25T10:22:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400ccf0 con 0x7ff908156150 2024-04-25T10:22:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff908154550 con 0x7ff908156150 2024-04-25T10:22:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.110+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90804ef30 con 0x7ff908156150 2024-04-25T10:22:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.114+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8f401a070 con 0x7ff908156150 2024-04-25T10:22:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.114+0000 7ff8ffe00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.114+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8f4084e40 con 0x7ff908156150 2024-04-25T10:22:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.114+0000 7ff907e00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.114+0000 7ff907e00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff90806f1c0 tx=0x7ff8f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.118+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff90814a7e0 con 0x7ff908156150 2024-04-25T10:22:28.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.282+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff90806e090 con 0x7ff908156150 2024-04-25T10:22:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.282+0000 7ff8ffe00640 1 -- 172.21.15.45:0/4003272193 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16609 (secure 0 0 0) 0x7ff8f4055ee0 con 0x7ff908156150 2024-04-25T10:22:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:28.288 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":76,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:22:26.898465+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff8e8066f60 msgr2=0x7ff8e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff90806f1c0 tx=0x7ff8f8009040 comp rx=0 tx=0).stop 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 msgr2=0x7ff908155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 0x7ff908155030 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e870 tx=0x7ff8f400ec70 comp rx=0 tx=0).stop 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 shutdown_connections 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff90815c080 0x7ff908153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff908156150 0x7ff908155030 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 --2- 172.21.15.45:0/4003272193 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff908066ee0 0x7ff90814a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 >> 172.21.15.45:0/4003272193 conn(0x7ff90806ac90 msgr2=0x7ff9080b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 shutdown_connections 2024-04-25T10:22:28.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.286+0000 7ff90ec00640 1 -- 172.21.15.45:0/4003272193 wait complete. 2024-04-25T10:22:28.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:22:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.450+0000 7f1d08a00640 1 Processor -- start 2024-04-25T10:22:28.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.450+0000 7f1d08a00640 1 -- start start 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040721f0 0x7f1d04166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d08a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04071360 con 0x7f1d040aeda0 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d08a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04159920 con 0x7f1d04159540 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d08a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0415c2f0 con 0x7f1d040721f0 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1cf9400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1d040aeda0 0x7f1d04071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59106/0 (socket says 172.21.15.45:59106) 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1cf9400640 1 -- 172.21.15.45:0/1776104516 learned_addr learned my addr 172.21.15.45:0/1776104516 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d02800640 1 --2- 172.21.15.45:0/1776104516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040721f0 0x7f1d04166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270558266 0 0) 0x7f1d04071360 con 0x7f1d040aeda0 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce8003530 con 0x7f1d040aeda0 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305538228 0 0) 0x7f1d04159920 con 0x7f1d04159540 2024-04-25T10:22:28.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04071360 con 0x7f1d04159540 2024-04-25T10:22:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d02800640 1 -- 172.21.15.45:0/1776104516 >> v1:172.21.15.117:6789/0 conn(0x7f1d04159540 legacy=0x7f1d040ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d02800640 1 -- 172.21.15.45:0/1776104516 >> v1:172.21.15.45:6789/0 conn(0x7f1d040aeda0 legacy=0x7f1d04071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d02800640 1 -- 172.21.15.45:0/1776104516 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0415c5d0 con 0x7f1d040721f0 2024-04-25T10:22:28.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.454+0000 7f1d02800640 1 --2- 172.21.15.45:0/1776104516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040721f0 0x7f1d04166920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0097f0 tx=0x7f1cfc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb3d68510c88ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:28.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0aa610 con 0x7f1d040721f0 2024-04-25T10:22:28.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0af070 con 0x7f1d040721f0 2024-04-25T10:22:28.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d01400640 1 -- 172.21.15.45:0/1776104516 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0aa9d0 con 0x7f1d040721f0 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- 172.21.15.45:0/1776104516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040721f0 msgr2=0x7f1d04166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 --2- 172.21.15.45:0/1776104516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040721f0 0x7f1d04166920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0097f0 tx=0x7f1cfc01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- 172.21.15.45:0/1776104516 shutdown_connections 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 --2- 172.21.15.45:0/1776104516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040721f0 0x7f1d04166920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- 172.21.15.45:0/1776104516 >> 172.21.15.45:0/1776104516 conn(0x7f1d040669a0 msgr2=0x7f1d04066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- 172.21.15.45:0/1776104516 shutdown_connections 2024-04-25T10:22:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- 172.21.15.45:0/1776104516 wait complete. 2024-04-25T10:22:28.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 Processor -- start 2024-04-25T10:22:28.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- start start 2024-04-25T10:22:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d040721f0 0x7f1d04158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040aeda0 0x7f1d040cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 0x7f1d040cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04169680 con 0x7f1d04159540 2024-04-25T10:22:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0406c0d0 con 0x7f1d040721f0 2024-04-25T10:22:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d08a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04169800 con 0x7f1d040aeda0 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d02800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 0x7f1d040cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d02800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 0x7f1d040cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46168/0 (socket says 172.21.15.45:46168) 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d02800640 1 -- 172.21.15.45:0/2810257722 learned_addr learned my addr 172.21.15.45:0/2810257722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1cf9400640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040aeda0 0x7f1d040cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.458+0000 7f1d01e00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d040721f0 0x7f1d04158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d02800640 1 -- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040aeda0 msgr2=0x7f1d040cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d02800640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040aeda0 0x7f1d040cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d02800640 1 -- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d040721f0 msgr2=0x7f1d04158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d02800640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d040721f0 0x7f1d04158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d02800640 1 -- 172.21.15.45:0/2810257722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04152f90 con 0x7f1d04159540 2024-04-25T10:22:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d02800640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 0x7f1d040cfbd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0097f0 tx=0x7f1cfc01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0be410 con 0x7f1d04159540 2024-04-25T10:22:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc0be5b0 con 0x7f1d04159540 2024-04-25T10:22:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0be980 con 0x7f1d04159540 2024-04-25T10:22:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d04153270 con 0x7f1d04159540 2024-04-25T10:22:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d04153830 con 0x7f1d04159540 2024-04-25T10:22:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0404ef30 con 0x7f1d04159540 2024-04-25T10:22:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cfc0bc020 con 0x7f1d04159540 2024-04-25T10:22:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1cfaa00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ce0066f60 0x7f1ce0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.462+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cfc11ee50 con 0x7f1d04159540 2024-04-25T10:22:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.466+0000 7f1d01e00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ce0066f60 0x7f1ce0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.466+0000 7f1d01e00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ce0066f60 0x7f1ce0069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007fd0 tx=0x7f1cf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:28.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.466+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cfc0c5070 con 0x7f1d04159540 2024-04-25T10:22:28.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:28.670+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1d0406b390 con 0x7f1d04159540 2024-04-25T10:22:29.488 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:22:29.486+0000 7fbcae000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:22:29.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.534+0000 7f1cfaa00640 1 -- 172.21.15.45:0/2810257722 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1cfc0a7070 con 0x7f1d04159540 2024-04-25T10:22:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ce0066f60 msgr2=0x7f1ce0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ce0066f60 0x7f1ce0069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007fd0 tx=0x7f1cf4009040 comp rx=0 tx=0).stop 2024-04-25T10:22:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 msgr2=0x7f1d040cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 0x7f1d040cfbd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0097f0 tx=0x7f1cfc01acf0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 shutdown_connections 2024-04-25T10:22:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1ce0066f60 0x7f1ce0069420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d04159540 0x7f1d040cfbd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d040aeda0 0x7f1d040cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 --2- 172.21.15.45:0/2810257722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d040721f0 0x7f1d04158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 >> 172.21.15.45:0/2810257722 conn(0x7f1d040669a0 msgr2=0x7f1d04070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 shutdown_connections 2024-04-25T10:22:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.538+0000 7f1d08a00640 1 -- 172.21.15.45:0/2810257722 wait complete. 2024-04-25T10:22:29.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:22:29.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:22:29.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 Processor -- start 2024-04-25T10:22:29.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 -- start start 2024-04-25T10:22:29.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf90070350 0x7fdf9015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:29.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900b31c0 0x7fdf900b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:29.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9015b080 con 0x7fdf900aef80 2024-04-25T10:22:29.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9015adb0 con 0x7fdf90070350 2024-04-25T10:22:29.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf96200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9015eb60 con 0x7fdf900b31c0 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf87e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf90070350 0x7fdf9015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf8f400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fdf900aef80 0x7fdf9015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:58958/0 (socket says 172.21.15.45:58958) 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf8f400640 1 -- 172.21.15.45:0/3828741747 learned_addr learned my addr 172.21.15.45:0/3828741747 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf8fe00640 1 --2- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900b31c0 0x7fdf900b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf87e00640 1 -- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900b31c0 msgr2=0x7fdf900b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf87e00640 1 --2- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900b31c0 0x7fdf900b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf87e00640 1 -- 172.21.15.45:0/3828741747 >> v1:172.21.15.117:6789/0 conn(0x7fdf900aef80 legacy=0x7fdf9015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.710+0000 7fdf87e00640 1 -- 172.21.15.45:0/3828741747 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9015ee40 con 0x7fdf90070350 2024-04-25T10:22:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf87e00640 1 --2- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf9015e480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097c0 tx=0x7fdf7801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc7ca8f8e6cf8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf8ea00640 1 -- 172.21.15.45:0/3828741747 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780b1070 con 0x7fdf90070350 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf8ea00640 1 -- 172.21.15.45:0/3828741747 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf780ac410 con 0x7fdf90070350 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf8ea00640 1 -- 172.21.15.45:0/3828741747 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780ac7e0 con 0x7fdf90070350 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 -- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 msgr2=0x7fdf9015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 --2- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf9015e480 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdf780097c0 tx=0x7fdf7801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 -- 172.21.15.45:0/3828741747 shutdown_connections 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 --2- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900b31c0 0x7fdf900b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 --2- 172.21.15.45:0/3828741747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf9015e480 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 -- 172.21.15.45:0/3828741747 >> 172.21.15.45:0/3828741747 conn(0x7fdf900669a0 msgr2=0x7fdf90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 -- 172.21.15.45:0/3828741747 shutdown_connections 2024-04-25T10:22:29.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.714+0000 7fdf96200640 1 -- 172.21.15.45:0/3828741747 wait complete. 2024-04-25T10:22:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 Processor -- start 2024-04-25T10:22:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 -- start start 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf90136210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900aef80 0x7fdf9014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 0x7fdf9014c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901679a0 con 0x7fdf900b31c0 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90167820 con 0x7fdf90070350 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90167b20 con 0x7fdf900aef80 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 0x7fdf9014c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 0x7fdf9014c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46184/0 (socket says 172.21.15.45:46184) 2024-04-25T10:22:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 -- 172.21.15.45:0/164697885 learned_addr learned my addr 172.21.15.45:0/164697885 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf87e00640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900aef80 0x7fdf9014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8f400640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf90136210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 -- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900aef80 msgr2=0x7fdf9014bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900aef80 0x7fdf9014bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 -- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 msgr2=0x7fdf90136210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf90136210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 -- 172.21.15.45:0/164697885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9015c0f0 con 0x7fdf900b31c0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf87e00640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900aef80 0x7fdf9014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf8fe00640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 0x7fdf9014c0d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009d30 tx=0x7fdf8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88018020 con 0x7fdf900b31c0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88014480 con 0x7fdf900b31c0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88014870 con 0x7fdf900b31c0 2024-04-25T10:22:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9015c3d0 con 0x7fdf900b31c0 2024-04-25T10:22:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.718+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9015c990 con 0x7fdf900b31c0 2024-04-25T10:22:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.722+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9004ef30 con 0x7fdf900b31c0 2024-04-25T10:22:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.722+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf8801b020 con 0x7fdf900b31c0 2024-04-25T10:22:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.722+0000 7fdf87400640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf6c066f60 0x7fdf6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.722+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf88085970 con 0x7fdf900b31c0 2024-04-25T10:22:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.722+0000 7fdf8f400640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf6c066f60 0x7fdf6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.722+0000 7fdf8f400640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf6c066f60 0x7fdf6c069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000f70 tx=0x7fdf80007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:29.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.726+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf88022070 con 0x7fdf900b31c0 2024-04-25T10:22:29.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:29.922+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdf900b2320 con 0x7fdf900b31c0 2024-04-25T10:22:30.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.534+0000 7fdf87400640 1 -- 172.21.15.45:0/164697885 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf88010040 con 0x7fdf900b31c0 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf6c066f60 msgr2=0x7fdf6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf6c066f60 0x7fdf6c069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000f70 tx=0x7fdf80007e90 comp rx=0 tx=0).stop 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 msgr2=0x7fdf9014c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 0x7fdf9014c0d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009d30 tx=0x7fdf8800c490 comp rx=0 tx=0).stop 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 shutdown_connections 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf6c066f60 0x7fdf6c069420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf900b31c0 0x7fdf9014c0d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf900aef80 0x7fdf9014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 --2- 172.21.15.45:0/164697885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf90070350 0x7fdf90136210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 >> 172.21.15.45:0/164697885 conn(0x7fdf900669a0 msgr2=0x7fdf900b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:30.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 shutdown_connections 2024-04-25T10:22:30.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.538+0000 7fdf96200640 1 -- 172.21.15.45:0/164697885 wait complete. 2024-04-25T10:22:30.565 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:22:30.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:30.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.706+0000 7fa72e800640 1 Processor -- start 2024-04-25T10:22:30.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.706+0000 7fa72e800640 1 -- start start 2024-04-25T10:22:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa72e800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7281550b0 0x7fa7280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa72e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa72815b910 0x7fa7280b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa72e800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa728071050 con 0x7fa7280cfe70 2024-04-25T10:22:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa72e800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72815bd80 con 0x7fa72815b910 2024-04-25T10:22:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa72e800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72815d040 con 0x7fa7281550b0 2024-04-25T10:22:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa727e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa72815b910 0x7fa7280b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa71ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa7280cfe70 0x7fa72815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59118/0 (socket says 172.21.15.45:59118) 2024-04-25T10:22:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa71ea00640 1 -- 172.21.15.45:0/3857650331 learned_addr learned my addr 172.21.15.45:0/3857650331 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa727400640 1 --2- 172.21.15.45:0/3857650331 >> v2:172.21.15.117:3300/0 conn(0x7fa7281550b0 0x7fa7280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa727e00640 1 -- 172.21.15.45:0/3857650331 >> v2:172.21.15.117:3300/0 conn(0x7fa7281550b0 msgr2=0x7fa7280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa727e00640 1 --2- 172.21.15.45:0/3857650331 >> v2:172.21.15.117:3300/0 conn(0x7fa7281550b0 0x7fa7280cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa727e00640 1 -- 172.21.15.45:0/3857650331 >> v1:172.21.15.45:6789/0 conn(0x7fa7280cfe70 legacy=0x7fa72815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa727e00640 1 -- 172.21.15.45:0/3857650331 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280712f0 con 0x7fa72815b910 2024-04-25T10:22:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.710+0000 7fa726a00640 1 -- 172.21.15.45:0/3857650331 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476148848 0 0) 0x7fa728071050 con 0x7fa7280cfe70 2024-04-25T10:22:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa727e00640 1 --2- 172.21.15.45:0/3857650331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa72815b910 0x7fa7280b3260 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d90 tx=0x7fa72001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142254b7e346fb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa726a00640 1 -- 172.21.15.45:0/3857650331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200b2070 con 0x7fa72815b910 2024-04-25T10:22:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa726a00640 1 -- 172.21.15.45:0/3857650331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200a58e0 con 0x7fa72815b910 2024-04-25T10:22:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa726a00640 1 -- 172.21.15.45:0/3857650331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200a5cb0 con 0x7fa72815b910 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- 172.21.15.45:0/3857650331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa72815b910 msgr2=0x7fa7280b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 --2- 172.21.15.45:0/3857650331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa72815b910 0x7fa7280b3260 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa720009d90 tx=0x7fa72001c930 comp rx=0 tx=0).stop 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- 172.21.15.45:0/3857650331 shutdown_connections 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 --2- 172.21.15.45:0/3857650331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa72815b910 0x7fa7280b3260 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 --2- 172.21.15.45:0/3857650331 >> v2:172.21.15.117:3300/0 conn(0x7fa7281550b0 0x7fa7280cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- 172.21.15.45:0/3857650331 >> 172.21.15.45:0/3857650331 conn(0x7fa728066b70 msgr2=0x7fa728066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- 172.21.15.45:0/3857650331 shutdown_connections 2024-04-25T10:22:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- 172.21.15.45:0/3857650331 wait complete. 2024-04-25T10:22:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 Processor -- start 2024-04-25T10:22:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- start start 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 0x7fa7281483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7281550b0 0x7fa72814eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa72815b910 0x7fa7280cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72815c030 con 0x7fa7280cfe70 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280720b0 con 0x7fa7281550b0 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa72e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72815c1b0 con 0x7fa72815b910 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa727400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 0x7fa7281483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa727400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 0x7fa7281483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46202/0 (socket says 172.21.15.45:46202) 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa727400640 1 -- 172.21.15.45:0/2012088547 learned_addr learned my addr 172.21.15.45:0/2012088547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.714+0000 7fa71ea00640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7281550b0 0x7fa72814eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727e00640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa72815b910 0x7fa7280cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727400640 1 -- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa72815b910 msgr2=0x7fa7280cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727400640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa72815b910 0x7fa7280cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727400640 1 -- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7281550b0 msgr2=0x7fa72814eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727400640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7281550b0 0x7fa72814eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727400640 1 -- 172.21.15.45:0/2012088547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728151490 con 0x7fa7280cfe70 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727e00640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa72815b910 0x7fa7280cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa727400640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 0x7fa7281483c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa71800d800 tx=0x7fa71800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71800b660 con 0x7fa7280cfe70 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718019070 con 0x7fa7280cfe70 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71800ba40 con 0x7fa7280cfe70 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72814d330 con 0x7fa7280cfe70 2024-04-25T10:22:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72814d8f0 con 0x7fa7280cfe70 2024-04-25T10:22:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa728151770 con 0x7fa7280cfe70 2024-04-25T10:22:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.718+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa71800bbe0 con 0x7fa7280cfe70 2024-04-25T10:22:30.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.722+0000 7fa71fe00640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa704066f60 0x7fa704069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:30.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.722+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7180853b0 con 0x7fa7280cfe70 2024-04-25T10:22:30.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.722+0000 7fa71ea00640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa704066f60 0x7fa704069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:30.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.722+0000 7fa71ea00640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa704066f60 0x7fa704069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa72815e290 tx=0x7fa710007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.726+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7180563b0 con 0x7fa7280cfe70 2024-04-25T10:22:30.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.918+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72814d100 con 0x7fa7280cfe70 2024-04-25T10:22:30.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa71fe00640 1 -- 172.21.15.45:0/2012088547 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa718055cb0 con 0x7fa7280cfe70 2024-04-25T10:22:30.924 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:30.924 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:22:26.898819+0000","modified":"2024-04-25T10:22:29.928612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3661919331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3661919331},{"type":"v1","addr":"172.21.15.45:6835","nonce":3661919331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:22:30.925 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 55 2024-04-25T10:22:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa704066f60 msgr2=0x7fa704069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa704066f60 0x7fa704069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa72815e290 tx=0x7fa710007d60 comp rx=0 tx=0).stop 2024-04-25T10:22:30.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 msgr2=0x7fa7281483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 0x7fa7281483c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa71800d800 tx=0x7fa71800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 shutdown_connections 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa704066f60 0x7fa704069420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa72815b910 0x7fa7280cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7281550b0 0x7fa72814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 --2- 172.21.15.45:0/2012088547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7280cfe70 0x7fa7281483c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 >> 172.21.15.45:0/2012088547 conn(0x7fa728066b70 msgr2=0x7fa7280b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 shutdown_connections 2024-04-25T10:22:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:30.922+0000 7fa72e800640 1 -- 172.21.15.45:0/2012088547 wait complete. 2024-04-25T10:22:30.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:31.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.082+0000 7feb0c200640 1 Processor -- start 2024-04-25T10:22:31.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.082+0000 7feb0c200640 1 -- start start 2024-04-25T10:22:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb0c200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb041550b0 0x7feb040cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb0c200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040cfe70 0x7feb0415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb0c200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb04071050 con 0x7feb0415b740 2024-04-25T10:22:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb0c200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0415d040 con 0x7feb041550b0 2024-04-25T10:22:31.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb0c200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040712f0 con 0x7feb040cfe70 2024-04-25T10:22:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb08e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040cfe70 0x7feb0415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb09800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb041550b0 0x7feb040cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb08e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040cfe70 0x7feb0415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:49262/0 (socket says 172.21.15.45:49262) 2024-04-25T10:22:31.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb08e00640 1 -- 172.21.15.45:0/3644788946 learned_addr learned my addr 172.21.15.45:0/3644788946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:31.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb0a200640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7feb0415b740 0x7feb0406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:58962/0 (socket says 172.21.15.45:58962) 2024-04-25T10:22:31.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb09800640 1 -- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040cfe70 msgr2=0x7feb0415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:31.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb09800640 1 --2- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040cfe70 0x7feb0415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb09800640 1 -- 172.21.15.45:0/3644788946 >> v1:172.21.15.117:6789/0 conn(0x7feb0415b740 legacy=0x7feb0406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.086+0000 7feb09800640 1 -- 172.21.15.45:0/3644788946 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb040715d0 con 0x7feb041550b0 2024-04-25T10:22:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb09800640 1 --2- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb041550b0 0x7feb040cf930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feaf80097c0 tx=0x7feaf801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b774a342b74543bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb03e00640 1 -- 172.21.15.45:0/3644788946 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b0070 con 0x7feb041550b0 2024-04-25T10:22:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb03e00640 1 -- 172.21.15.45:0/3644788946 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80ab600 con 0x7feb041550b0 2024-04-25T10:22:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb03e00640 1 -- 172.21.15.45:0/3644788946 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80ab9d0 con 0x7feb041550b0 2024-04-25T10:22:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 -- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb041550b0 msgr2=0x7feb040cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 --2- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb041550b0 0x7feb040cf930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feaf80097c0 tx=0x7feaf801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 -- 172.21.15.45:0/3644788946 shutdown_connections 2024-04-25T10:22:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 --2- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb040cfe70 0x7feb0415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 --2- 172.21.15.45:0/3644788946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb041550b0 0x7feb040cf930 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 -- 172.21.15.45:0/3644788946 >> 172.21.15.45:0/3644788946 conn(0x7feb04066b70 msgr2=0x7feb04066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 -- 172.21.15.45:0/3644788946 shutdown_connections 2024-04-25T10:22:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 -- 172.21.15.45:0/3644788946 wait complete. 2024-04-25T10:22:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 Processor -- start 2024-04-25T10:22:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.090+0000 7feb0c200640 1 -- start start 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 0x7feb040cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb041550b0 0x7feb041f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415b740 0x7feb04158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415bcd0 con 0x7feb040cfe70 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415bb50 con 0x7feb0415b740 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0415c030 con 0x7feb041550b0 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 0x7feb040cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 0x7feb040cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46212/0 (socket says 172.21.15.45:46212) 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 -- 172.21.15.45:0/2548110090 learned_addr learned my addr 172.21.15.45:0/2548110090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb08e00640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb041550b0 0x7feb041f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0a200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415b740 0x7feb04158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 -- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb041550b0 msgr2=0x7feb041f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb041550b0 0x7feb041f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 -- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415b740 msgr2=0x7feb04158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415b740 0x7feb04158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 -- 172.21.15.45:0/2548110090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04159ab0 con 0x7feb040cfe70 2024-04-25T10:22:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb09800640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 0x7feb040cf100 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ae040 tx=0x7feaf801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b0050 con 0x7feb040cfe70 2024-04-25T10:22:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf80bc070 con 0x7feb040cfe70 2024-04-25T10:22:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb04159d50 con 0x7feb040cfe70 2024-04-25T10:22:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80b5870 con 0x7feb040cfe70 2024-04-25T10:22:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb041496f0 con 0x7feb040cfe70 2024-04-25T10:22:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0404ef30 con 0x7feb040cfe70 2024-04-25T10:22:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.094+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaf80b5a10 con 0x7feb040cfe70 2024-04-25T10:22:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.098+0000 7feb01600640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.098+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaf811f670 con 0x7feb040cfe70 2024-04-25T10:22:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.098+0000 7feb08e00640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.098+0000 7feb08e00640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb0413aa50 tx=0x7feaf0009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.102+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf8016090 con 0x7feb040cfe70 2024-04-25T10:22:31.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.266+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb0414c520 con 0x7feb040cfe70 2024-04-25T10:22:31.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.266+0000 7feb01600640 1 -- 172.21.15.45:0/2548110090 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16596 (secure 0 0 0) 0x7feaf80eff90 con 0x7feb040cfe70 2024-04-25T10:22:31.270 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:31.270 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":77,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:22:28.515150+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.266+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 msgr2=0x7feae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.266+0000 7feb0c200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7feb0413aa50 tx=0x7feaf0009240 comp rx=0 tx=0).stop 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.266+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 msgr2=0x7feb040cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.266+0000 7feb0c200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 0x7feb040cf100 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ae040 tx=0x7feaf801ace0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 shutdown_connections 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae4066f60 0x7feae4069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0415b740 0x7feb04158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb041550b0 0x7feb041f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 --2- 172.21.15.45:0/2548110090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb040cfe70 0x7feb040cf100 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 >> 172.21.15.45:0/2548110090 conn(0x7feb04066b70 msgr2=0x7feb0414fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 shutdown_connections 2024-04-25T10:22:31.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.270+0000 7feb0c200640 1 -- 172.21.15.45:0/2548110090 wait complete. 2024-04-25T10:22:31.291 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:22:31.291 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:31.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 Processor -- start 2024-04-25T10:22:31.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 -- start start 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f218406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2184071f50 0x7f21840669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218406f200 con 0x7f21840afda0 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218406d380 con 0x7f2184071f50 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2188e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21840ccc10 con 0x7f21840af9c0 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2184071f50 0x7f21840669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2184071f50 0x7f21840669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46226/0 (socket says 172.21.15.45:46226) 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 -- 172.21.15.45:0/3317071915 learned_addr learned my addr 172.21.15.45:0/3317071915 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2179400640 1 --1- 172.21.15.45:0/3317071915 >> v1:172.21.15.45:6789/0 conn(0x7f21840afda0 0x7f21840cc3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59132/0 (socket says 172.21.15.45:59132) 2024-04-25T10:22:31.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2181e00640 1 --2- 172.21.15.45:0/3317071915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f218406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 -- 172.21.15.45:0/3317071915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 msgr2=0x7f218406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:31.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 --2- 172.21.15.45:0/3317071915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f218406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 -- 172.21.15.45:0/3317071915 >> v1:172.21.15.45:6789/0 conn(0x7f21840afda0 legacy=0x7f21840cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.430+0000 7f2182800640 1 -- 172.21.15.45:0/3317071915 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840ccef0 con 0x7f2184071f50 2024-04-25T10:22:31.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2182800640 1 --2- 172.21.15.45:0/3317071915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840669a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f217c009e50 tx=0x7f217c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375eb03b758e1fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2181400640 1 -- 172.21.15.45:0/3317071915 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c0a56b0 con 0x7f2184071f50 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2181400640 1 -- 172.21.15.45:0/3317071915 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c0a5850 con 0x7f2184071f50 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2181400640 1 -- 172.21.15.45:0/3317071915 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c0a5c20 con 0x7f2184071f50 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 -- 172.21.15.45:0/3317071915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 msgr2=0x7f21840669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 --2- 172.21.15.45:0/3317071915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840669a0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f217c009e50 tx=0x7f217c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 -- 172.21.15.45:0/3317071915 shutdown_connections 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 --2- 172.21.15.45:0/3317071915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840669a0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 --2- 172.21.15.45:0/3317071915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f218406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 -- 172.21.15.45:0/3317071915 >> 172.21.15.45:0/3317071915 conn(0x7f21840b2410 msgr2=0x7f21840b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:31.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 -- 172.21.15.45:0/3317071915 shutdown_connections 2024-04-25T10:22:31.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.434+0000 7f2188e00640 1 -- 172.21.15.45:0/3317071915 wait complete. 2024-04-25T10:22:31.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 Processor -- start 2024-04-25T10:22:31.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 -- start start 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f21840c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21840afda0 0x7f21840c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21841604a0 con 0x7f2184071f50 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2184160320 con 0x7f21840afda0 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218415b4e0 con 0x7f21840af9c0 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46240/0 (socket says 172.21.15.45:46240) 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 -- 172.21.15.45:0/1992366615 learned_addr learned my addr 172.21.15.45:0/1992366615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2182800640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21840afda0 0x7f21840c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 -- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 msgr2=0x7f21840c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f21840c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 -- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21840afda0 msgr2=0x7f21840c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21840afda0 0x7f21840c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 -- 172.21.15.45:0/1992366615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840b4150 con 0x7f2184071f50 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2182800640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21840afda0 0x7f21840c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2181e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840b3ac0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f217400e930 tx=0x7f217400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174017070 con 0x7f2184071f50 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217400ca00 con 0x7f2184071f50 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400cdb0 con 0x7f2184071f50 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21840b4430 con 0x7f2184071f50 2024-04-25T10:22:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.438+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21840b49f0 con 0x7f2184071f50 2024-04-25T10:22:31.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.442+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218404ef30 con 0x7f2184071f50 2024-04-25T10:22:31.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.442+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f217401a070 con 0x7f2184071f50 2024-04-25T10:22:31.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.442+0000 7f217aa00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2160066f60 0x7f2160069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.442+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2174085160 con 0x7f2184071f50 2024-04-25T10:22:31.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.442+0000 7f2179400640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2160066f60 0x7f2160069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.442+0000 7f2179400640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2160066f60 0x7f2160069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f216c000f70 tx=0x7f216c007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.446+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2174056190 con 0x7f2184071f50 2024-04-25T10:22:31.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.606+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2184071420 con 0x7f2160066f60 2024-04-25T10:22:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.610+0000 7f217aa00640 1 -- 172.21.15.45:0/1992366615 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252064 (secure 0 0 0) 0x7f2184071420 con 0x7f2160066f60 2024-04-25T10:22:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2160066f60 msgr2=0x7f2160069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2160066f60 0x7f2160069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f216c000f70 tx=0x7f216c007a10 comp rx=0 tx=0).stop 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 msgr2=0x7f21840b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840b3ac0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f217400e930 tx=0x7f217400edf0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 shutdown_connections 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2160066f60 0x7f2160069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21840afda0 0x7f21840c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21840af9c0 0x7f21840c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 --2- 172.21.15.45:0/1992366615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2184071f50 0x7f21840b3ac0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 >> 172.21.15.45:0/1992366615 conn(0x7f21840b2410 msgr2=0x7f21840b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 shutdown_connections 2024-04-25T10:22:31.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.614+0000 7f2188e00640 1 -- 172.21.15.45:0/1992366615 wait complete. 2024-04-25T10:22:31.638 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T10:22:30.935190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":325812,"kb_used_data":8392,"kb_used_omap":12,"kb_used_meta":317299,"kb_avail":754648908,"statfs":{"total":773094113280,"available":772760481792,"internally_reserved":0,"allocated":8593408,"data_stored":4301156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":324914736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.440749"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531052+0000","last_change":"2024-04-25T10:22:27.529213+0000","last_active":"2024-04-25T10:22:28.531052+0000","last_peered":"2024-04-25T10:22:28.531052+0000","last_clean":"2024-04-25T10:22:28.531052+0000","last_became_active":"2024-04-25T10:22:27.528941+0000","last_became_peered":"2024-04-25T10:22:27.528941+0000","last_unstale":"2024-04-25T10:22:28.531052+0000","last_undegraded":"2024-04-25T10:22:28.531052+0000","last_fullsized":"2024-04-25T10:22:28.531052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:32.517081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522761+0000","last_change":"2024-04-25T10:22:27.547825+0000","last_active":"2024-04-25T10:22:28.522761+0000","last_peered":"2024-04-25T10:22:28.522761+0000","last_clean":"2024-04-25T10:22:28.522761+0000","last_became_active":"2024-04-25T10:22:27.546835+0000","last_became_peered":"2024-04-25T10:22:27.546835+0000","last_unstale":"2024-04-25T10:22:28.522761+0000","last_undegraded":"2024-04-25T10:22:28.522761+0000","last_fullsized":"2024-04-25T10:22:28.522761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:08.994301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727910+0000","last_change":"2024-04-25T10:22:27.520176+0000","last_active":"2024-04-25T10:22:28.727910+0000","last_peered":"2024-04-25T10:22:28.727910+0000","last_clean":"2024-04-25T10:22:28.727910+0000","last_became_active":"2024-04-25T10:22:27.519856+0000","last_became_peered":"2024-04-25T10:22:27.519856+0000","last_unstale":"2024-04-25T10:22:28.727910+0000","last_undegraded":"2024-04-25T10:22:28.727910+0000","last_fullsized":"2024-04-25T10:22:28.727910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.670849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727893+0000","last_change":"2024-04-25T10:22:27.521233+0000","last_active":"2024-04-25T10:22:28.727893+0000","last_peered":"2024-04-25T10:22:28.727893+0000","last_clean":"2024-04-25T10:22:28.727893+0000","last_became_active":"2024-04-25T10:22:27.520962+0000","last_became_peered":"2024-04-25T10:22:27.520962+0000","last_unstale":"2024-04-25T10:22:28.727893+0000","last_undegraded":"2024-04-25T10:22:28.727893+0000","last_fullsized":"2024-04-25T10:22:28.727893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:17.806191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.521992+0000","last_change":"2024-04-25T10:22:27.539324+0000","last_active":"2024-04-25T10:22:28.521992+0000","last_peered":"2024-04-25T10:22:28.521992+0000","last_clean":"2024-04-25T10:22:28.521992+0000","last_became_active":"2024-04-25T10:22:27.539045+0000","last_became_peered":"2024-04-25T10:22:27.539045+0000","last_unstale":"2024-04-25T10:22:28.521992+0000","last_undegraded":"2024-04-25T10:22:28.521992+0000","last_fullsized":"2024-04-25T10:22:28.521992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:53.106627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727803+0000","last_change":"2024-04-25T10:22:27.551534+0000","last_active":"2024-04-25T10:22:28.727803+0000","last_peered":"2024-04-25T10:22:28.727803+0000","last_clean":"2024-04-25T10:22:28.727803+0000","last_became_active":"2024-04-25T10:22:27.551259+0000","last_became_peered":"2024-04-25T10:22:27.551259+0000","last_unstale":"2024-04-25T10:22:28.727803+0000","last_undegraded":"2024-04-25T10:22:28.727803+0000","last_fullsized":"2024-04-25T10:22:28.727803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.532569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519471+0000","last_change":"2024-04-25T10:22:27.532076+0000","last_active":"2024-04-25T10:22:28.519471+0000","last_peered":"2024-04-25T10:22:28.519471+0000","last_clean":"2024-04-25T10:22:28.519471+0000","last_became_active":"2024-04-25T10:22:27.531610+0000","last_became_peered":"2024-04-25T10:22:27.531610+0000","last_unstale":"2024-04-25T10:22:28.519471+0000","last_undegraded":"2024-04-25T10:22:28.519471+0000","last_fullsized":"2024-04-25T10:22:28.519471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.670849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530541+0000","last_change":"2024-04-25T10:22:27.527219+0000","last_active":"2024-04-25T10:22:28.530541+0000","last_peered":"2024-04-25T10:22:28.530541+0000","last_clean":"2024-04-25T10:22:28.530541+0000","last_became_active":"2024-04-25T10:22:27.526931+0000","last_became_peered":"2024-04-25T10:22:27.526931+0000","last_unstale":"2024-04-25T10:22:28.530541+0000","last_undegraded":"2024-04-25T10:22:28.530541+0000","last_fullsized":"2024-04-25T10:22:28.530541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:50.540203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727851+0000","last_change":"2024-04-25T10:22:27.524376+0000","last_active":"2024-04-25T10:22:28.727851+0000","last_peered":"2024-04-25T10:22:28.727851+0000","last_clean":"2024-04-25T10:22:28.727851+0000","last_became_active":"2024-04-25T10:22:27.523830+0000","last_became_peered":"2024-04-25T10:22:27.523830+0000","last_unstale":"2024-04-25T10:22:28.727851+0000","last_undegraded":"2024-04-25T10:22:28.727851+0000","last_fullsized":"2024-04-25T10:22:28.727851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:10.172650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.520182+0000","last_change":"2024-04-25T10:22:27.531332+0000","last_active":"2024-04-25T10:22:28.520182+0000","last_peered":"2024-04-25T10:22:28.520182+0000","last_clean":"2024-04-25T10:22:28.520182+0000","last_became_active":"2024-04-25T10:22:27.530328+0000","last_became_peered":"2024-04-25T10:22:27.530328+0000","last_unstale":"2024-04-25T10:22:28.520182+0000","last_undegraded":"2024-04-25T10:22:28.520182+0000","last_fullsized":"2024-04-25T10:22:28.520182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:05.015827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531327+0000","last_change":"2024-04-25T10:22:27.527162+0000","last_active":"2024-04-25T10:22:28.531327+0000","last_peered":"2024-04-25T10:22:28.531327+0000","last_clean":"2024-04-25T10:22:28.531327+0000","last_became_active":"2024-04-25T10:22:27.526804+0000","last_became_peered":"2024-04-25T10:22:27.526804+0000","last_unstale":"2024-04-25T10:22:28.531327+0000","last_undegraded":"2024-04-25T10:22:28.531327+0000","last_fullsized":"2024-04-25T10:22:28.531327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:56.706006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.533258+0000","last_change":"2024-04-25T10:22:27.529596+0000","last_active":"2024-04-25T10:22:27.533258+0000","last_peered":"2024-04-25T10:22:27.533258+0000","last_clean":"2024-04-25T10:22:27.533258+0000","last_became_active":"2024-04-25T10:22:27.529306+0000","last_became_peered":"2024-04-25T10:22:27.529306+0000","last_unstale":"2024-04-25T10:22:27.533258+0000","last_undegraded":"2024-04-25T10:22:27.533258+0000","last_fullsized":"2024-04-25T10:22:27.533258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:34.712976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.729053+0000","last_change":"2024-04-25T10:22:27.522735+0000","last_active":"2024-04-25T10:22:28.729053+0000","last_peered":"2024-04-25T10:22:28.729053+0000","last_clean":"2024-04-25T10:22:28.729053+0000","last_became_active":"2024-04-25T10:22:27.522097+0000","last_became_peered":"2024-04-25T10:22:27.522097+0000","last_unstale":"2024-04-25T10:22:28.729053+0000","last_undegraded":"2024-04-25T10:22:28.729053+0000","last_fullsized":"2024-04-25T10:22:28.729053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:39.323925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.538423+0000","last_change":"2024-04-25T10:22:27.536278+0000","last_active":"2024-04-25T10:22:27.538423+0000","last_peered":"2024-04-25T10:22:27.538423+0000","last_clean":"2024-04-25T10:22:27.538423+0000","last_became_active":"2024-04-25T10:22:27.535980+0000","last_became_peered":"2024-04-25T10:22:27.535980+0000","last_unstale":"2024-04-25T10:22:27.538423+0000","last_undegraded":"2024-04-25T10:22:27.538423+0000","last_fullsized":"2024-04-25T10:22:27.538423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:43.686247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.533928+0000","last_change":"2024-04-25T10:22:27.530550+0000","last_active":"2024-04-25T10:22:27.533928+0000","last_peered":"2024-04-25T10:22:27.533928+0000","last_clean":"2024-04-25T10:22:27.533928+0000","last_became_active":"2024-04-25T10:22:27.528935+0000","last_became_peered":"2024-04-25T10:22:27.528935+0000","last_unstale":"2024-04-25T10:22:27.533928+0000","last_undegraded":"2024-04-25T10:22:27.533928+0000","last_fullsized":"2024-04-25T10:22:27.533928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:13.188900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530541+0000","last_change":"2024-04-25T10:22:27.545207+0000","last_active":"2024-04-25T10:22:28.530541+0000","last_peered":"2024-04-25T10:22:28.530541+0000","last_clean":"2024-04-25T10:22:28.530541+0000","last_became_active":"2024-04-25T10:22:27.544920+0000","last_became_peered":"2024-04-25T10:22:27.544920+0000","last_unstale":"2024-04-25T10:22:28.530541+0000","last_undegraded":"2024-04-25T10:22:28.530541+0000","last_fullsized":"2024-04-25T10:22:28.530541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:40.523825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522793+0000","last_change":"2024-04-25T10:22:27.522799+0000","last_active":"2024-04-25T10:22:28.522793+0000","last_peered":"2024-04-25T10:22:28.522793+0000","last_clean":"2024-04-25T10:22:28.522793+0000","last_became_active":"2024-04-25T10:22:27.522539+0000","last_became_peered":"2024-04-25T10:22:27.522539+0000","last_unstale":"2024-04-25T10:22:28.522793+0000","last_undegraded":"2024-04-25T10:22:28.522793+0000","last_fullsized":"2024-04-25T10:22:28.522793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:10.006564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522645+0000","last_change":"2024-04-25T10:22:27.520044+0000","last_active":"2024-04-25T10:22:28.522645+0000","last_peered":"2024-04-25T10:22:28.522645+0000","last_clean":"2024-04-25T10:22:28.522645+0000","last_became_active":"2024-04-25T10:22:27.519782+0000","last_became_peered":"2024-04-25T10:22:27.519782+0000","last_unstale":"2024-04-25T10:22:28.522645+0000","last_undegraded":"2024-04-25T10:22:28.522645+0000","last_fullsized":"2024-04-25T10:22:28.522645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:25.652315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531141+0000","last_change":"2024-04-25T10:22:27.522915+0000","last_active":"2024-04-25T10:22:28.531141+0000","last_peered":"2024-04-25T10:22:28.531141+0000","last_clean":"2024-04-25T10:22:28.531141+0000","last_became_active":"2024-04-25T10:22:27.522500+0000","last_became_peered":"2024-04-25T10:22:27.522500+0000","last_unstale":"2024-04-25T10:22:28.531141+0000","last_undegraded":"2024-04-25T10:22:28.531141+0000","last_fullsized":"2024-04-25T10:22:28.531141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.431390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522239+0000","last_change":"2024-04-25T10:22:27.532976+0000","last_active":"2024-04-25T10:22:28.522239+0000","last_peered":"2024-04-25T10:22:28.522239+0000","last_clean":"2024-04-25T10:22:28.522239+0000","last_became_active":"2024-04-25T10:22:27.532680+0000","last_became_peered":"2024-04-25T10:22:27.532680+0000","last_unstale":"2024-04-25T10:22:28.522239+0000","last_undegraded":"2024-04-25T10:22:28.522239+0000","last_fullsized":"2024-04-25T10:22:28.522239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:45.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522588+0000","last_change":"2024-04-25T10:22:27.537777+0000","last_active":"2024-04-25T10:22:28.522588+0000","last_peered":"2024-04-25T10:22:28.522588+0000","last_clean":"2024-04-25T10:22:28.522588+0000","last_became_active":"2024-04-25T10:22:27.535147+0000","last_became_peered":"2024-04-25T10:22:27.535147+0000","last_unstale":"2024-04-25T10:22:28.522588+0000","last_undegraded":"2024-04-25T10:22:28.522588+0000","last_fullsized":"2024-04-25T10:22:28.522588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:24.917800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727806+0000","last_change":"2024-04-25T10:22:27.531095+0000","last_active":"2024-04-25T10:22:28.727806+0000","last_peered":"2024-04-25T10:22:28.727806+0000","last_clean":"2024-04-25T10:22:28.727806+0000","last_became_active":"2024-04-25T10:22:27.530511+0000","last_became_peered":"2024-04-25T10:22:27.530511+0000","last_unstale":"2024-04-25T10:22:28.727806+0000","last_undegraded":"2024-04-25T10:22:28.727806+0000","last_fullsized":"2024-04-25T10:22:28.727806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:55.195531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519472+0000","last_change":"2024-04-25T10:22:27.545792+0000","last_active":"2024-04-25T10:22:28.519472+0000","last_peered":"2024-04-25T10:22:28.519472+0000","last_clean":"2024-04-25T10:22:28.519472+0000","last_became_active":"2024-04-25T10:22:27.545451+0000","last_became_peered":"2024-04-25T10:22:27.545451+0000","last_unstale":"2024-04-25T10:22:28.519472+0000","last_undegraded":"2024-04-25T10:22:28.519472+0000","last_fullsized":"2024-04-25T10:22:28.519472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:16.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530735+0000","last_change":"2024-04-25T10:22:27.539944+0000","last_active":"2024-04-25T10:22:28.530735+0000","last_peered":"2024-04-25T10:22:28.530735+0000","last_clean":"2024-04-25T10:22:28.530735+0000","last_became_active":"2024-04-25T10:22:27.539090+0000","last_became_peered":"2024-04-25T10:22:27.539090+0000","last_unstale":"2024-04-25T10:22:28.530735+0000","last_undegraded":"2024-04-25T10:22:28.530735+0000","last_fullsized":"2024-04-25T10:22:28.530735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:31.963964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519998+0000","last_change":"2024-04-25T10:22:27.538284+0000","last_active":"2024-04-25T10:22:28.519998+0000","last_peered":"2024-04-25T10:22:28.519998+0000","last_clean":"2024-04-25T10:22:28.519998+0000","last_became_active":"2024-04-25T10:22:27.538003+0000","last_became_peered":"2024-04-25T10:22:27.538003+0000","last_unstale":"2024-04-25T10:22:28.519998+0000","last_undegraded":"2024-04-25T10:22:28.519998+0000","last_fullsized":"2024-04-25T10:22:28.519998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:06.092429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530897+0000","last_change":"2024-04-25T10:22:27.522458+0000","last_active":"2024-04-25T10:22:28.530897+0000","last_peered":"2024-04-25T10:22:28.530897+0000","last_clean":"2024-04-25T10:22:28.530897+0000","last_became_active":"2024-04-25T10:22:27.522134+0000","last_became_peered":"2024-04-25T10:22:27.522134+0000","last_unstale":"2024-04-25T10:22:28.530897+0000","last_undegraded":"2024-04-25T10:22:28.530897+0000","last_fullsized":"2024-04-25T10:22:28.530897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:08.564452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522277+0000","last_change":"2024-04-25T10:22:27.524055+0000","last_active":"2024-04-25T10:22:28.522277+0000","last_peered":"2024-04-25T10:22:28.522277+0000","last_clean":"2024-04-25T10:22:28.522277+0000","last_became_active":"2024-04-25T10:22:27.521372+0000","last_became_peered":"2024-04-25T10:22:27.521372+0000","last_unstale":"2024-04-25T10:22:28.522277+0000","last_undegraded":"2024-04-25T10:22:28.522277+0000","last_fullsized":"2024-04-25T10:22:28.522277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:36.171433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522545+0000","last_change":"2024-04-25T10:22:27.541204+0000","last_active":"2024-04-25T10:22:28.522545+0000","last_peered":"2024-04-25T10:22:28.522545+0000","last_clean":"2024-04-25T10:22:28.522545+0000","last_became_active":"2024-04-25T10:22:27.538442+0000","last_became_peered":"2024-04-25T10:22:27.538442+0000","last_unstale":"2024-04-25T10:22:28.522545+0000","last_undegraded":"2024-04-25T10:22:28.522545+0000","last_fullsized":"2024-04-25T10:22:28.522545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:19.112842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727703+0000","last_change":"2024-04-25T10:22:27.523039+0000","last_active":"2024-04-25T10:22:28.727703+0000","last_peered":"2024-04-25T10:22:28.727703+0000","last_clean":"2024-04-25T10:22:28.727703+0000","last_became_active":"2024-04-25T10:22:27.522481+0000","last_became_peered":"2024-04-25T10:22:27.522481+0000","last_unstale":"2024-04-25T10:22:28.727703+0000","last_undegraded":"2024-04-25T10:22:28.727703+0000","last_fullsized":"2024-04-25T10:22:28.727703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:16.372996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530681+0000","last_change":"2024-04-25T10:22:27.524761+0000","last_active":"2024-04-25T10:22:28.530681+0000","last_peered":"2024-04-25T10:22:28.530681+0000","last_clean":"2024-04-25T10:22:28.530681+0000","last_became_active":"2024-04-25T10:22:27.524434+0000","last_became_peered":"2024-04-25T10:22:27.524434+0000","last_unstale":"2024-04-25T10:22:28.530681+0000","last_undegraded":"2024-04-25T10:22:28.530681+0000","last_fullsized":"2024-04-25T10:22:28.530681+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:23.136134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522401+0000","last_change":"2024-04-25T10:22:27.536748+0000","last_active":"2024-04-25T10:22:28.522401+0000","last_peered":"2024-04-25T10:22:28.522401+0000","last_clean":"2024-04-25T10:22:28.522401+0000","last_became_active":"2024-04-25T10:22:27.536193+0000","last_became_peered":"2024-04-25T10:22:27.536193+0000","last_unstale":"2024-04-25T10:22:28.522401+0000","last_undegraded":"2024-04-25T10:22:28.522401+0000","last_fullsized":"2024-04-25T10:22:28.522401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:19.765870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522323+0000","last_change":"2024-04-25T10:22:27.538504+0000","last_active":"2024-04-25T10:22:28.522323+0000","last_peered":"2024-04-25T10:22:28.522323+0000","last_clean":"2024-04-25T10:22:28.522323+0000","last_became_active":"2024-04-25T10:22:27.534304+0000","last_became_peered":"2024-04-25T10:22:27.534304+0000","last_unstale":"2024-04-25T10:22:28.522323+0000","last_undegraded":"2024-04-25T10:22:28.522323+0000","last_fullsized":"2024-04-25T10:22:28.522323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:26.886634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530986+0000","last_change":"2024-04-25T10:22:27.540348+0000","last_active":"2024-04-25T10:22:28.530986+0000","last_peered":"2024-04-25T10:22:28.530986+0000","last_clean":"2024-04-25T10:22:28.530986+0000","last_became_active":"2024-04-25T10:22:27.539557+0000","last_became_peered":"2024-04-25T10:22:27.539557+0000","last_unstale":"2024-04-25T10:22:28.530986+0000","last_undegraded":"2024-04-25T10:22:28.530986+0000","last_fullsized":"2024-04-25T10:22:28.530986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.500005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728141+0000","last_change":"2024-04-25T10:22:26.516020+0000","last_active":"2024-04-25T10:22:28.728141+0000","last_peered":"2024-04-25T10:22:28.728141+0000","last_clean":"2024-04-25T10:22:28.728141+0000","last_became_active":"2024-04-25T10:22:26.515759+0000","last_became_peered":"2024-04-25T10:22:26.515759+0000","last_unstale":"2024-04-25T10:22:28.728141+0000","last_undegraded":"2024-04-25T10:22:28.728141+0000","last_fullsized":"2024-04-25T10:22:28.728141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:07.590515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728034+0000","last_change":"2024-04-25T10:22:27.524144+0000","last_active":"2024-04-25T10:22:28.728034+0000","last_peered":"2024-04-25T10:22:28.728034+0000","last_clean":"2024-04-25T10:22:28.728034+0000","last_became_active":"2024-04-25T10:22:27.523463+0000","last_became_peered":"2024-04-25T10:22:27.523463+0000","last_unstale":"2024-04-25T10:22:28.728034+0000","last_undegraded":"2024-04-25T10:22:28.728034+0000","last_fullsized":"2024-04-25T10:22:28.728034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:41.475772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522752+0000","last_change":"2024-04-25T10:22:26.511037+0000","last_active":"2024-04-25T10:22:28.522752+0000","last_peered":"2024-04-25T10:22:28.522752+0000","last_clean":"2024-04-25T10:22:28.522752+0000","last_became_active":"2024-04-25T10:22:26.510718+0000","last_became_peered":"2024-04-25T10:22:26.510718+0000","last_unstale":"2024-04-25T10:22:28.522752+0000","last_undegraded":"2024-04-25T10:22:28.522752+0000","last_fullsized":"2024-04-25T10:22:28.522752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:53.906477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728054+0000","last_change":"2024-04-25T10:22:27.543573+0000","last_active":"2024-04-25T10:22:28.728054+0000","last_peered":"2024-04-25T10:22:28.728054+0000","last_clean":"2024-04-25T10:22:28.728054+0000","last_became_active":"2024-04-25T10:22:27.543258+0000","last_became_peered":"2024-04-25T10:22:27.543258+0000","last_unstale":"2024-04-25T10:22:28.728054+0000","last_undegraded":"2024-04-25T10:22:28.728054+0000","last_fullsized":"2024-04-25T10:22:28.728054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:37.697538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531328+0000","last_change":"2024-04-25T10:22:26.528433+0000","last_active":"2024-04-25T10:22:28.531328+0000","last_peered":"2024-04-25T10:22:28.531328+0000","last_clean":"2024-04-25T10:22:28.531328+0000","last_became_active":"2024-04-25T10:22:26.528149+0000","last_became_peered":"2024-04-25T10:22:26.528149+0000","last_unstale":"2024-04-25T10:22:28.531328+0000","last_undegraded":"2024-04-25T10:22:28.531328+0000","last_fullsized":"2024-04-25T10:22:28.531328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:55.748507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519923+0000","last_change":"2024-04-25T10:22:27.535896+0000","last_active":"2024-04-25T10:22:28.519923+0000","last_peered":"2024-04-25T10:22:28.519923+0000","last_clean":"2024-04-25T10:22:28.519923+0000","last_became_active":"2024-04-25T10:22:27.531993+0000","last_became_peered":"2024-04-25T10:22:27.531993+0000","last_unstale":"2024-04-25T10:22:28.519923+0000","last_undegraded":"2024-04-25T10:22:28.519923+0000","last_fullsized":"2024-04-25T10:22:28.519923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:39.323925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728043+0000","last_change":"2024-04-25T10:22:26.517778+0000","last_active":"2024-04-25T10:22:28.728043+0000","last_peered":"2024-04-25T10:22:28.728043+0000","last_clean":"2024-04-25T10:22:28.728043+0000","last_became_active":"2024-04-25T10:22:26.517503+0000","last_became_peered":"2024-04-25T10:22:26.517503+0000","last_unstale":"2024-04-25T10:22:28.728043+0000","last_undegraded":"2024-04-25T10:22:28.728043+0000","last_fullsized":"2024-04-25T10:22:28.728043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.624430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530685+0000","last_change":"2024-04-25T10:22:27.530913+0000","last_active":"2024-04-25T10:22:28.530685+0000","last_peered":"2024-04-25T10:22:28.530685+0000","last_clean":"2024-04-25T10:22:28.530685+0000","last_became_active":"2024-04-25T10:22:27.530621+0000","last_became_peered":"2024-04-25T10:22:27.530621+0000","last_unstale":"2024-04-25T10:22:28.530685+0000","last_undegraded":"2024-04-25T10:22:28.530685+0000","last_fullsized":"2024-04-25T10:22:28.530685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.372674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.553244+0000","last_change":"2024-04-25T10:22:26.519810+0000","last_active":"2024-04-25T10:22:27.553244+0000","last_peered":"2024-04-25T10:22:27.553244+0000","last_clean":"2024-04-25T10:22:27.553244+0000","last_became_active":"2024-04-25T10:22:26.519347+0000","last_became_peered":"2024-04-25T10:22:26.519347+0000","last_unstale":"2024-04-25T10:22:27.553244+0000","last_undegraded":"2024-04-25T10:22:27.553244+0000","last_fullsized":"2024-04-25T10:22:27.553244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:16.615456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727894+0000","last_change":"2024-04-25T10:22:27.541723+0000","last_active":"2024-04-25T10:22:28.727894+0000","last_peered":"2024-04-25T10:22:28.727894+0000","last_clean":"2024-04-25T10:22:28.727894+0000","last_became_active":"2024-04-25T10:22:27.541273+0000","last_became_peered":"2024-04-25T10:22:27.541273+0000","last_unstale":"2024-04-25T10:22:28.727894+0000","last_undegraded":"2024-04-25T10:22:28.727894+0000","last_fullsized":"2024-04-25T10:22:28.727894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:50.444166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519566+0000","last_change":"2024-04-25T10:22:26.519410+0000","last_active":"2024-04-25T10:22:28.519566+0000","last_peered":"2024-04-25T10:22:28.519566+0000","last_clean":"2024-04-25T10:22:28.519566+0000","last_became_active":"2024-04-25T10:22:26.519108+0000","last_became_peered":"2024-04-25T10:22:26.519108+0000","last_unstale":"2024-04-25T10:22:28.519566+0000","last_undegraded":"2024-04-25T10:22:28.519566+0000","last_fullsized":"2024-04-25T10:22:28.519566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:50.573948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727802+0000","last_change":"2024-04-25T10:22:27.530306+0000","last_active":"2024-04-25T10:22:28.727802+0000","last_peered":"2024-04-25T10:22:28.727802+0000","last_clean":"2024-04-25T10:22:28.727802+0000","last_became_active":"2024-04-25T10:22:27.529600+0000","last_became_peered":"2024-04-25T10:22:27.529600+0000","last_unstale":"2024-04-25T10:22:28.727802+0000","last_undegraded":"2024-04-25T10:22:28.727802+0000","last_fullsized":"2024-04-25T10:22:28.727802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:47.135564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728679+0000","last_change":"2024-04-25T10:22:26.517926+0000","last_active":"2024-04-25T10:22:28.728679+0000","last_peered":"2024-04-25T10:22:28.728679+0000","last_clean":"2024-04-25T10:22:28.728679+0000","last_became_active":"2024-04-25T10:22:26.517562+0000","last_became_peered":"2024-04-25T10:22:26.517562+0000","last_unstale":"2024-04-25T10:22:28.728679+0000","last_undegraded":"2024-04-25T10:22:28.728679+0000","last_fullsized":"2024-04-25T10:22:28.728679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:00.560567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.530422+0000","last_change":"2024-04-25T10:22:27.523607+0000","last_active":"2024-04-25T10:22:27.530422+0000","last_peered":"2024-04-25T10:22:27.530422+0000","last_clean":"2024-04-25T10:22:27.530422+0000","last_became_active":"2024-04-25T10:22:27.523107+0000","last_became_peered":"2024-04-25T10:22:27.523107+0000","last_unstale":"2024-04-25T10:22:27.530422+0000","last_undegraded":"2024-04-25T10:22:27.530422+0000","last_fullsized":"2024-04-25T10:22:27.530422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:56.381637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.518190+0000","last_change":"2024-04-25T10:22:26.513960+0000","last_active":"2024-04-25T10:22:27.518190+0000","last_peered":"2024-04-25T10:22:27.518190+0000","last_clean":"2024-04-25T10:22:27.518190+0000","last_became_active":"2024-04-25T10:22:26.513670+0000","last_became_peered":"2024-04-25T10:22:26.513670+0000","last_unstale":"2024-04-25T10:22:27.518190+0000","last_undegraded":"2024-04-25T10:22:27.518190+0000","last_fullsized":"2024-04-25T10:22:27.518190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:03.778988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.520309+0000","last_change":"2024-04-25T10:22:27.545987+0000","last_active":"2024-04-25T10:22:28.520309+0000","last_peered":"2024-04-25T10:22:28.520309+0000","last_clean":"2024-04-25T10:22:28.520309+0000","last_became_active":"2024-04-25T10:22:27.545427+0000","last_became_peered":"2024-04-25T10:22:27.545427+0000","last_unstale":"2024-04-25T10:22:28.520309+0000","last_undegraded":"2024-04-25T10:22:28.520309+0000","last_fullsized":"2024-04-25T10:22:28.520309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:37.876899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530894+0000","last_change":"2024-04-25T10:22:26.527205+0000","last_active":"2024-04-25T10:22:28.530894+0000","last_peered":"2024-04-25T10:22:28.530894+0000","last_clean":"2024-04-25T10:22:28.530894+0000","last_became_active":"2024-04-25T10:22:26.526865+0000","last_became_peered":"2024-04-25T10:22:26.526865+0000","last_unstale":"2024-04-25T10:22:28.530894+0000","last_undegraded":"2024-04-25T10:22:28.530894+0000","last_fullsized":"2024-04-25T10:22:28.530894+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:26.869712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522826+0000","last_change":"2024-04-25T10:22:27.535815+0000","last_active":"2024-04-25T10:22:28.522826+0000","last_peered":"2024-04-25T10:22:28.522826+0000","last_clean":"2024-04-25T10:22:28.522826+0000","last_became_active":"2024-04-25T10:22:27.535478+0000","last_became_peered":"2024-04-25T10:22:27.535478+0000","last_unstale":"2024-04-25T10:22:28.522826+0000","last_undegraded":"2024-04-25T10:22:28.522826+0000","last_fullsized":"2024-04-25T10:22:28.522826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:08.598913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522854+0000","last_change":"2024-04-25T10:22:26.522527+0000","last_active":"2024-04-25T10:22:28.522854+0000","last_peered":"2024-04-25T10:22:28.522854+0000","last_clean":"2024-04-25T10:22:28.522854+0000","last_became_active":"2024-04-25T10:22:26.522210+0000","last_became_peered":"2024-04-25T10:22:26.522210+0000","last_unstale":"2024-04-25T10:22:28.522854+0000","last_undegraded":"2024-04-25T10:22:28.522854+0000","last_fullsized":"2024-04-25T10:22:28.522854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:07.985166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530898+0000","last_change":"2024-04-25T10:22:27.523140+0000","last_active":"2024-04-25T10:22:28.530898+0000","last_peered":"2024-04-25T10:22:28.530898+0000","last_clean":"2024-04-25T10:22:28.530898+0000","last_became_active":"2024-04-25T10:22:27.522776+0000","last_became_peered":"2024-04-25T10:22:27.522776+0000","last_unstale":"2024-04-25T10:22:28.530898+0000","last_undegraded":"2024-04-25T10:22:28.530898+0000","last_fullsized":"2024-04-25T10:22:28.530898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.796012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.547792+0000","last_change":"2024-04-25T10:22:26.511939+0000","last_active":"2024-04-25T10:22:27.547792+0000","last_peered":"2024-04-25T10:22:27.547792+0000","last_clean":"2024-04-25T10:22:27.547792+0000","last_became_active":"2024-04-25T10:22:26.511607+0000","last_became_peered":"2024-04-25T10:22:26.511607+0000","last_unstale":"2024-04-25T10:22:27.547792+0000","last_undegraded":"2024-04-25T10:22:27.547792+0000","last_fullsized":"2024-04-25T10:22:27.547792+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:46.854374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522307+0000","last_change":"2024-04-25T10:22:27.524309+0000","last_active":"2024-04-25T10:22:28.522307+0000","last_peered":"2024-04-25T10:22:28.522307+0000","last_clean":"2024-04-25T10:22:28.522307+0000","last_became_active":"2024-04-25T10:22:27.521343+0000","last_became_peered":"2024-04-25T10:22:27.521343+0000","last_unstale":"2024-04-25T10:22:28.522307+0000","last_undegraded":"2024-04-25T10:22:28.522307+0000","last_fullsized":"2024-04-25T10:22:28.522307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:12.954149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728238+0000","last_change":"2024-04-25T10:22:26.517908+0000","last_active":"2024-04-25T10:22:28.728238+0000","last_peered":"2024-04-25T10:22:28.728238+0000","last_clean":"2024-04-25T10:22:28.728238+0000","last_became_active":"2024-04-25T10:22:26.517616+0000","last_became_peered":"2024-04-25T10:22:26.517616+0000","last_unstale":"2024-04-25T10:22:28.728238+0000","last_undegraded":"2024-04-25T10:22:28.728238+0000","last_fullsized":"2024-04-25T10:22:28.728238+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:46.067958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728010+0000","last_change":"2024-04-25T10:22:27.530845+0000","last_active":"2024-04-25T10:22:28.728010+0000","last_peered":"2024-04-25T10:22:28.728010+0000","last_clean":"2024-04-25T10:22:28.728010+0000","last_became_active":"2024-04-25T10:22:27.530176+0000","last_became_peered":"2024-04-25T10:22:27.530176+0000","last_unstale":"2024-04-25T10:22:28.728010+0000","last_undegraded":"2024-04-25T10:22:28.728010+0000","last_fullsized":"2024-04-25T10:22:28.728010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:08.917931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728123+0000","last_change":"2024-04-25T10:22:26.526208+0000","last_active":"2024-04-25T10:22:28.728123+0000","last_peered":"2024-04-25T10:22:28.728123+0000","last_clean":"2024-04-25T10:22:28.728123+0000","last_became_active":"2024-04-25T10:22:26.525912+0000","last_became_peered":"2024-04-25T10:22:26.525912+0000","last_unstale":"2024-04-25T10:22:28.728123+0000","last_undegraded":"2024-04-25T10:22:28.728123+0000","last_fullsized":"2024-04-25T10:22:28.728123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:50.711675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519710+0000","last_change":"2024-04-25T10:22:27.529937+0000","last_active":"2024-04-25T10:22:28.519710+0000","last_peered":"2024-04-25T10:22:28.519710+0000","last_clean":"2024-04-25T10:22:28.519710+0000","last_became_active":"2024-04-25T10:22:27.529634+0000","last_became_peered":"2024-04-25T10:22:27.529634+0000","last_unstale":"2024-04-25T10:22:28.519710+0000","last_undegraded":"2024-04-25T10:22:28.519710+0000","last_fullsized":"2024-04-25T10:22:28.519710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:41.475772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519553+0000","last_change":"2024-04-25T10:22:26.519164+0000","last_active":"2024-04-25T10:22:28.519553+0000","last_peered":"2024-04-25T10:22:28.519553+0000","last_clean":"2024-04-25T10:22:28.519553+0000","last_became_active":"2024-04-25T10:22:26.518878+0000","last_became_peered":"2024-04-25T10:22:26.518878+0000","last_unstale":"2024-04-25T10:22:28.519553+0000","last_undegraded":"2024-04-25T10:22:28.519553+0000","last_fullsized":"2024-04-25T10:22:28.519553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:14.209043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.521893+0000","last_change":"2024-04-25T10:22:27.530002+0000","last_active":"2024-04-25T10:22:28.521893+0000","last_peered":"2024-04-25T10:22:28.521893+0000","last_clean":"2024-04-25T10:22:28.521893+0000","last_became_active":"2024-04-25T10:22:27.529388+0000","last_became_peered":"2024-04-25T10:22:27.529388+0000","last_unstale":"2024-04-25T10:22:28.521893+0000","last_undegraded":"2024-04-25T10:22:28.521893+0000","last_fullsized":"2024-04-25T10:22:28.521893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:56.756906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519588+0000","last_change":"2024-04-25T10:22:26.517915+0000","last_active":"2024-04-25T10:22:28.519588+0000","last_peered":"2024-04-25T10:22:28.519588+0000","last_clean":"2024-04-25T10:22:28.519588+0000","last_became_active":"2024-04-25T10:22:26.517710+0000","last_became_peered":"2024-04-25T10:22:26.517710+0000","last_unstale":"2024-04-25T10:22:28.519588+0000","last_undegraded":"2024-04-25T10:22:28.519588+0000","last_fullsized":"2024-04-25T10:22:28.519588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:52.164535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727648+0000","last_change":"2024-04-25T10:22:27.523422+0000","last_active":"2024-04-25T10:22:28.727648+0000","last_peered":"2024-04-25T10:22:28.727648+0000","last_clean":"2024-04-25T10:22:28.727648+0000","last_became_active":"2024-04-25T10:22:27.523023+0000","last_became_peered":"2024-04-25T10:22:27.523023+0000","last_unstale":"2024-04-25T10:22:28.727648+0000","last_undegraded":"2024-04-25T10:22:28.727648+0000","last_fullsized":"2024-04-25T10:22:28.727648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:18.358476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519306+0000","last_change":"2024-04-25T10:22:26.512094+0000","last_active":"2024-04-25T10:22:28.519306+0000","last_peered":"2024-04-25T10:22:28.519306+0000","last_clean":"2024-04-25T10:22:28.519306+0000","last_became_active":"2024-04-25T10:22:26.511120+0000","last_became_peered":"2024-04-25T10:22:26.511120+0000","last_unstale":"2024-04-25T10:22:28.519306+0000","last_undegraded":"2024-04-25T10:22:28.519306+0000","last_fullsized":"2024-04-25T10:22:28.519306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:00.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522870+0000","last_change":"2024-04-25T10:22:27.536272+0000","last_active":"2024-04-25T10:22:28.522870+0000","last_peered":"2024-04-25T10:22:28.522870+0000","last_clean":"2024-04-25T10:22:28.522870+0000","last_became_active":"2024-04-25T10:22:27.534471+0000","last_became_peered":"2024-04-25T10:22:27.534471+0000","last_unstale":"2024-04-25T10:22:28.522870+0000","last_undegraded":"2024-04-25T10:22:28.522870+0000","last_fullsized":"2024-04-25T10:22:28.522870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:03.173613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530967+0000","last_change":"2024-04-25T10:22:26.526525+0000","last_active":"2024-04-25T10:22:28.530967+0000","last_peered":"2024-04-25T10:22:28.530967+0000","last_clean":"2024-04-25T10:22:28.530967+0000","last_became_active":"2024-04-25T10:22:26.525984+0000","last_became_peered":"2024-04-25T10:22:26.525984+0000","last_unstale":"2024-04-25T10:22:28.530967+0000","last_undegraded":"2024-04-25T10:22:28.530967+0000","last_fullsized":"2024-04-25T10:22:28.530967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:52.098228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522837+0000","last_change":"2024-04-25T10:22:27.535755+0000","last_active":"2024-04-25T10:22:28.522837+0000","last_peered":"2024-04-25T10:22:28.522837+0000","last_clean":"2024-04-25T10:22:28.522837+0000","last_became_active":"2024-04-25T10:22:27.535458+0000","last_became_peered":"2024-04-25T10:22:27.535458+0000","last_unstale":"2024-04-25T10:22:28.522837+0000","last_undegraded":"2024-04-25T10:22:28.522837+0000","last_fullsized":"2024-04-25T10:22:28.522837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:02.947463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522927+0000","last_change":"2024-04-25T10:22:26.510224+0000","last_active":"2024-04-25T10:22:28.522927+0000","last_peered":"2024-04-25T10:22:28.522927+0000","last_clean":"2024-04-25T10:22:28.522927+0000","last_became_active":"2024-04-25T10:22:26.509906+0000","last_became_peered":"2024-04-25T10:22:26.509906+0000","last_unstale":"2024-04-25T10:22:28.522927+0000","last_undegraded":"2024-04-25T10:22:28.522927+0000","last_fullsized":"2024-04-25T10:22:28.522927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:06.554223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727944+0000","last_change":"2024-04-25T10:22:27.530945+0000","last_active":"2024-04-25T10:22:28.727944+0000","last_peered":"2024-04-25T10:22:28.727944+0000","last_clean":"2024-04-25T10:22:28.727944+0000","last_became_active":"2024-04-25T10:22:27.530665+0000","last_became_peered":"2024-04-25T10:22:27.530665+0000","last_unstale":"2024-04-25T10:22:28.727944+0000","last_undegraded":"2024-04-25T10:22:28.727944+0000","last_fullsized":"2024-04-25T10:22:28.727944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:16.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522576+0000","last_change":"2024-04-25T10:22:26.528258+0000","last_active":"2024-04-25T10:22:28.522576+0000","last_peered":"2024-04-25T10:22:28.522576+0000","last_clean":"2024-04-25T10:22:28.522576+0000","last_became_active":"2024-04-25T10:22:26.527909+0000","last_became_peered":"2024-04-25T10:22:26.527909+0000","last_unstale":"2024-04-25T10:22:28.522576+0000","last_undegraded":"2024-04-25T10:22:28.522576+0000","last_fullsized":"2024-04-25T10:22:28.522576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:04.007429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728072+0000","last_change":"2024-04-25T10:22:27.520973+0000","last_active":"2024-04-25T10:22:28.728072+0000","last_peered":"2024-04-25T10:22:28.728072+0000","last_clean":"2024-04-25T10:22:28.728072+0000","last_became_active":"2024-04-25T10:22:27.520714+0000","last_became_peered":"2024-04-25T10:22:27.520714+0000","last_unstale":"2024-04-25T10:22:28.728072+0000","last_undegraded":"2024-04-25T10:22:28.728072+0000","last_fullsized":"2024-04-25T10:22:28.728072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728135+0000","last_change":"2024-04-25T10:22:26.525845+0000","last_active":"2024-04-25T10:22:28.728135+0000","last_peered":"2024-04-25T10:22:28.728135+0000","last_clean":"2024-04-25T10:22:28.728135+0000","last_became_active":"2024-04-25T10:22:26.525559+0000","last_became_peered":"2024-04-25T10:22:26.525559+0000","last_unstale":"2024-04-25T10:22:28.728135+0000","last_undegraded":"2024-04-25T10:22:28.728135+0000","last_fullsized":"2024-04-25T10:22:28.728135+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:07.985903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727968+0000","last_change":"2024-04-25T10:22:27.523903+0000","last_active":"2024-04-25T10:22:28.727968+0000","last_peered":"2024-04-25T10:22:28.727968+0000","last_clean":"2024-04-25T10:22:28.727968+0000","last_became_active":"2024-04-25T10:22:27.523627+0000","last_became_peered":"2024-04-25T10:22:27.523627+0000","last_unstale":"2024-04-25T10:22:28.727968+0000","last_undegraded":"2024-04-25T10:22:28.727968+0000","last_fullsized":"2024-04-25T10:22:28.727968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:07.127333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728105+0000","last_change":"2024-04-25T10:22:26.521395+0000","last_active":"2024-04-25T10:22:28.728105+0000","last_peered":"2024-04-25T10:22:28.728105+0000","last_clean":"2024-04-25T10:22:28.728105+0000","last_became_active":"2024-04-25T10:22:26.519955+0000","last_became_peered":"2024-04-25T10:22:26.519955+0000","last_unstale":"2024-04-25T10:22:28.728105+0000","last_undegraded":"2024-04-25T10:22:28.728105+0000","last_fullsized":"2024-04-25T10:22:28.728105+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:18.104444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728014+0000","last_change":"2024-04-25T10:22:27.528088+0000","last_active":"2024-04-25T10:22:28.728014+0000","last_peered":"2024-04-25T10:22:28.728014+0000","last_clean":"2024-04-25T10:22:28.728014+0000","last_became_active":"2024-04-25T10:22:27.527794+0000","last_became_peered":"2024-04-25T10:22:27.527794+0000","last_unstale":"2024-04-25T10:22:28.728014+0000","last_undegraded":"2024-04-25T10:22:28.728014+0000","last_fullsized":"2024-04-25T10:22:28.728014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:26.886634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522447+0000","last_change":"2024-04-25T10:22:26.516828+0000","last_active":"2024-04-25T10:22:28.522447+0000","last_peered":"2024-04-25T10:22:28.522447+0000","last_clean":"2024-04-25T10:22:28.522447+0000","last_became_active":"2024-04-25T10:22:26.516390+0000","last_became_peered":"2024-04-25T10:22:26.516390+0000","last_unstale":"2024-04-25T10:22:28.522447+0000","last_undegraded":"2024-04-25T10:22:28.522447+0000","last_fullsized":"2024-04-25T10:22:28.522447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:25.763795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.529334+0000","last_change":"2024-04-25T10:22:27.523385+0000","last_active":"2024-04-25T10:22:27.529334+0000","last_peered":"2024-04-25T10:22:27.529334+0000","last_clean":"2024-04-25T10:22:27.529334+0000","last_became_active":"2024-04-25T10:22:27.522814+0000","last_became_peered":"2024-04-25T10:22:27.522814+0000","last_unstale":"2024-04-25T10:22:27.529334+0000","last_undegraded":"2024-04-25T10:22:27.529334+0000","last_fullsized":"2024-04-25T10:22:27.529334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:37.528737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.521778+0000","last_change":"2024-04-25T10:22:26.510683+0000","last_active":"2024-04-25T10:22:28.521778+0000","last_peered":"2024-04-25T10:22:28.521778+0000","last_clean":"2024-04-25T10:22:28.521778+0000","last_became_active":"2024-04-25T10:22:26.510377+0000","last_became_peered":"2024-04-25T10:22:26.510377+0000","last_unstale":"2024-04-25T10:22:28.521778+0000","last_undegraded":"2024-04-25T10:22:28.521778+0000","last_fullsized":"2024-04-25T10:22:28.521778+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:16.923299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522655+0000","last_change":"2024-04-25T10:22:27.523260+0000","last_active":"2024-04-25T10:22:28.522655+0000","last_peered":"2024-04-25T10:22:28.522655+0000","last_clean":"2024-04-25T10:22:28.522655+0000","last_became_active":"2024-04-25T10:22:27.522433+0000","last_became_peered":"2024-04-25T10:22:27.522433+0000","last_unstale":"2024-04-25T10:22:28.522655+0000","last_undegraded":"2024-04-25T10:22:28.522655+0000","last_fullsized":"2024-04-25T10:22:28.522655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:36.566530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531107+0000","last_change":"2024-04-25T10:22:26.526296+0000","last_active":"2024-04-25T10:22:28.531107+0000","last_peered":"2024-04-25T10:22:28.531107+0000","last_clean":"2024-04-25T10:22:28.531107+0000","last_became_active":"2024-04-25T10:22:26.525710+0000","last_became_peered":"2024-04-25T10:22:26.525710+0000","last_unstale":"2024-04-25T10:22:28.531107+0000","last_undegraded":"2024-04-25T10:22:28.531107+0000","last_fullsized":"2024-04-25T10:22:28.531107+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:16.280940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.520124+0000","last_change":"2024-04-25T10:22:27.530737+0000","last_active":"2024-04-25T10:22:28.520124+0000","last_peered":"2024-04-25T10:22:28.520124+0000","last_clean":"2024-04-25T10:22:28.520124+0000","last_became_active":"2024-04-25T10:22:27.530433+0000","last_became_peered":"2024-04-25T10:22:27.530433+0000","last_unstale":"2024-04-25T10:22:28.520124+0000","last_undegraded":"2024-04-25T10:22:28.520124+0000","last_fullsized":"2024-04-25T10:22:28.520124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:26.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"77'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.532329+0000","last_change":"2024-04-25T10:22:26.524426+0000","last_active":"2024-04-25T10:22:28.532329+0000","last_peered":"2024-04-25T10:22:28.532329+0000","last_clean":"2024-04-25T10:22:28.532329+0000","last_became_active":"2024-04-25T10:22:26.524142+0000","last_became_peered":"2024-04-25T10:22:26.524142+0000","last_unstale":"2024-04-25T10:22:28.532329+0000","last_undegraded":"2024-04-25T10:22:28.532329+0000","last_fullsized":"2024-04-25T10:22:28.532329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:05.584260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522603+0000","last_change":"2024-04-25T10:22:27.544246+0000","last_active":"2024-04-25T10:22:28.522603+0000","last_peered":"2024-04-25T10:22:28.522603+0000","last_clean":"2024-04-25T10:22:28.522603+0000","last_became_active":"2024-04-25T10:22:27.537554+0000","last_became_peered":"2024-04-25T10:22:27.537554+0000","last_unstale":"2024-04-25T10:22:28.522603+0000","last_undegraded":"2024-04-25T10:22:28.522603+0000","last_fullsized":"2024-04-25T10:22:28.522603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:11.632829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.518789+0000","last_change":"2024-04-25T10:22:26.516019+0000","last_active":"2024-04-25T10:22:27.518789+0000","last_peered":"2024-04-25T10:22:27.518789+0000","last_clean":"2024-04-25T10:22:27.518789+0000","last_became_active":"2024-04-25T10:22:26.515308+0000","last_became_peered":"2024-04-25T10:22:26.515308+0000","last_unstale":"2024-04-25T10:22:27.518789+0000","last_undegraded":"2024-04-25T10:22:27.518789+0000","last_fullsized":"2024-04-25T10:22:27.518789+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:11.276185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519908+0000","last_change":"2024-04-25T10:22:27.539298+0000","last_active":"2024-04-25T10:22:28.519908+0000","last_peered":"2024-04-25T10:22:28.519908+0000","last_clean":"2024-04-25T10:22:28.519908+0000","last_became_active":"2024-04-25T10:22:27.538523+0000","last_became_peered":"2024-04-25T10:22:27.538523+0000","last_unstale":"2024-04-25T10:22:28.519908+0000","last_undegraded":"2024-04-25T10:22:28.519908+0000","last_fullsized":"2024-04-25T10:22:28.519908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.520984+0000","last_change":"2024-04-25T10:22:26.511178+0000","last_active":"2024-04-25T10:22:27.520984+0000","last_peered":"2024-04-25T10:22:27.520984+0000","last_clean":"2024-04-25T10:22:27.520984+0000","last_became_active":"2024-04-25T10:22:26.510867+0000","last_became_peered":"2024-04-25T10:22:26.510867+0000","last_unstale":"2024-04-25T10:22:27.520984+0000","last_undegraded":"2024-04-25T10:22:27.520984+0000","last_fullsized":"2024-04-25T10:22:27.520984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:36.386576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522124+0000","last_change":"2024-04-25T10:22:27.526582+0000","last_active":"2024-04-25T10:22:28.522124+0000","last_peered":"2024-04-25T10:22:28.522124+0000","last_clean":"2024-04-25T10:22:28.522124+0000","last_became_active":"2024-04-25T10:22:27.526260+0000","last_became_peered":"2024-04-25T10:22:27.526260+0000","last_unstale":"2024-04-25T10:22:28.522124+0000","last_undegraded":"2024-04-25T10:22:28.522124+0000","last_fullsized":"2024-04-25T10:22:28.522124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:52.310123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.519321+0000","last_change":"2024-04-25T10:22:26.514828+0000","last_active":"2024-04-25T10:22:27.519321+0000","last_peered":"2024-04-25T10:22:27.519321+0000","last_clean":"2024-04-25T10:22:27.519321+0000","last_became_active":"2024-04-25T10:22:26.514510+0000","last_became_peered":"2024-04-25T10:22:26.514510+0000","last_unstale":"2024-04-25T10:22:27.519321+0000","last_undegraded":"2024-04-25T10:22:27.519321+0000","last_fullsized":"2024-04-25T10:22:27.519321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:26.590540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530915+0000","last_change":"2024-04-25T10:22:27.539492+0000","last_active":"2024-04-25T10:22:28.530915+0000","last_peered":"2024-04-25T10:22:28.530915+0000","last_clean":"2024-04-25T10:22:28.530915+0000","last_became_active":"2024-04-25T10:22:27.539141+0000","last_became_peered":"2024-04-25T10:22:27.539141+0000","last_unstale":"2024-04-25T10:22:28.530915+0000","last_undegraded":"2024-04-25T10:22:28.530915+0000","last_fullsized":"2024-04-25T10:22:28.530915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:03.004996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522127+0000","last_change":"2024-04-25T10:22:26.520118+0000","last_active":"2024-04-25T10:22:28.522127+0000","last_peered":"2024-04-25T10:22:28.522127+0000","last_clean":"2024-04-25T10:22:28.522127+0000","last_became_active":"2024-04-25T10:22:26.519549+0000","last_became_peered":"2024-04-25T10:22:26.519549+0000","last_unstale":"2024-04-25T10:22:28.522127+0000","last_undegraded":"2024-04-25T10:22:28.522127+0000","last_fullsized":"2024-04-25T10:22:28.522127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:58.079090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728041+0000","last_change":"2024-04-25T10:22:27.536986+0000","last_active":"2024-04-25T10:22:28.728041+0000","last_peered":"2024-04-25T10:22:28.728041+0000","last_clean":"2024-04-25T10:22:28.728041+0000","last_became_active":"2024-04-25T10:22:27.536592+0000","last_became_peered":"2024-04-25T10:22:27.536592+0000","last_unstale":"2024-04-25T10:22:28.728041+0000","last_undegraded":"2024-04-25T10:22:28.728041+0000","last_fullsized":"2024-04-25T10:22:28.728041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:23.543952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522656+0000","last_change":"2024-04-25T10:22:26.515436+0000","last_active":"2024-04-25T10:22:28.522656+0000","last_peered":"2024-04-25T10:22:28.522656+0000","last_clean":"2024-04-25T10:22:28.522656+0000","last_became_active":"2024-04-25T10:22:26.515138+0000","last_became_peered":"2024-04-25T10:22:26.515138+0000","last_unstale":"2024-04-25T10:22:28.522656+0000","last_undegraded":"2024-04-25T10:22:28.522656+0000","last_fullsized":"2024-04-25T10:22:28.522656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:36.868500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522344+0000","last_change":"2024-04-25T10:22:27.536479+0000","last_active":"2024-04-25T10:22:28.522344+0000","last_peered":"2024-04-25T10:22:28.522344+0000","last_clean":"2024-04-25T10:22:28.522344+0000","last_became_active":"2024-04-25T10:22:27.534838+0000","last_became_peered":"2024-04-25T10:22:27.534838+0000","last_unstale":"2024-04-25T10:22:28.522344+0000","last_undegraded":"2024-04-25T10:22:28.522344+0000","last_fullsized":"2024-04-25T10:22:28.522344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:23.543952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727800+0000","last_change":"2024-04-25T10:22:26.517623+0000","last_active":"2024-04-25T10:22:28.727800+0000","last_peered":"2024-04-25T10:22:28.727800+0000","last_clean":"2024-04-25T10:22:28.727800+0000","last_became_active":"2024-04-25T10:22:26.517337+0000","last_became_peered":"2024-04-25T10:22:26.517337+0000","last_unstale":"2024-04-25T10:22:28.727800+0000","last_undegraded":"2024-04-25T10:22:28.727800+0000","last_fullsized":"2024-04-25T10:22:28.727800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:23.909402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522004+0000","last_change":"2024-04-25T10:22:27.521993+0000","last_active":"2024-04-25T10:22:28.522004+0000","last_peered":"2024-04-25T10:22:28.522004+0000","last_clean":"2024-04-25T10:22:28.522004+0000","last_became_active":"2024-04-25T10:22:27.521701+0000","last_became_peered":"2024-04-25T10:22:27.521701+0000","last_unstale":"2024-04-25T10:22:28.522004+0000","last_undegraded":"2024-04-25T10:22:28.522004+0000","last_fullsized":"2024-04-25T10:22:28.522004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:17.289338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519659+0000","last_change":"2024-04-25T10:22:26.511919+0000","last_active":"2024-04-25T10:22:28.519659+0000","last_peered":"2024-04-25T10:22:28.519659+0000","last_clean":"2024-04-25T10:22:28.519659+0000","last_became_active":"2024-04-25T10:22:26.511659+0000","last_became_peered":"2024-04-25T10:22:26.511659+0000","last_unstale":"2024-04-25T10:22:28.519659+0000","last_undegraded":"2024-04-25T10:22:28.519659+0000","last_fullsized":"2024-04-25T10:22:28.519659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:33.668574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519332+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:22:28.519332+0000","last_peered":"2024-04-25T10:22:28.519332+0000","last_clean":"2024-04-25T10:22:28.519332+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:22:28.519332+0000","last_undegraded":"2024-04-25T10:22:28.519332+0000","last_fullsized":"2024-04-25T10:22:28.519332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":929792,"data_stored":391062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509500,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":391096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40988,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330852,"statfs":{"total":96636764160,"available":96594792448,"internally_reserved":0,"allocated":1515520,"data_stored":979908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44568,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327272,"statfs":{"total":96636764160,"available":96591126528,"internally_reserved":0,"allocated":929792,"data_stored":389576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":390032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40976,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330864,"statfs":{"total":96636764160,"available":96594804736,"internally_reserved":0,"allocated":1511424,"data_stored":979818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509500,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":389520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62616,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309224,"statfs":{"total":96636764160,"available":96572645376,"internally_reserved":0,"allocated":929792,"data_stored":390144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:31.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3f200640 1 Processor -- start 2024-04-25T10:22:31.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3f200640 1 -- start start 2024-04-25T10:22:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 0x7f7e40168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3f200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e400cbff0 con 0x7f7e4015dc10 2024-04-25T10:22:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e400cc2d0 con 0x7f7e4015dff0 2024-04-25T10:22:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b45a0 con 0x7f7e4006bdb0 2024-04-25T10:22:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3de00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7e4015dc10 0x7f7e400d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59140/0 (socket says 172.21.15.45:59140) 2024-04-25T10:22:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3de00640 1 -- 172.21.15.45:0/3404074036 learned_addr learned my addr 172.21.15.45:0/3404074036 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 0x7f7e40168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858386189 0 0) 0x7f7e400cbff0 con 0x7f7e4015dc10 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e24003530 con 0x7f7e4015dc10 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843778933 0 0) 0x7f7e24003530 con 0x7f7e4015dc10 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 msgr2=0x7f7e40168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 0x7f7e40168950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 >> v1:172.21.15.117:6789/0 conn(0x7f7e4015dff0 legacy=0x7f7e4006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400cc5b0 con 0x7f7e4015dc10 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f7e30001680 con 0x7f7e4015dc10 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 >> v1:172.21.15.45:6789/0 conn(0x7f7e4015dc10 legacy=0x7f7e400d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e240039f0 0x7f7e24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 0x7f7e24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400b9d0 con 0x7f7e24004d50 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400bb80 con 0x7f7e240039f0 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2400bd80 con 0x7f7e4006bdb0 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3de00640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e240039f0 0x7f7e24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3d400640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 0x7f7e24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 -- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 msgr2=0x7f7e24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e4006bdb0 0x7f7e24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 -- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e240039f0 msgr2=0x7f7e24003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e240039f0 0x7f7e24003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3e800640 1 -- 172.21.15.45:0/3404074036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e24003530 con 0x7f7e24004d50 2024-04-25T10:22:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.822+0000 7f7e3de00640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e240039f0 0x7f7e24003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:31.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e240051d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7e40074a20 tx=0x7f7e3801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9835d4fc8b8f2e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380b2070 con 0x7f7e24004d50 2024-04-25T10:22:31.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e380a5ae0 con 0x7f7e24004d50 2024-04-25T10:22:31.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3ca00640 1 -- 172.21.15.45:0/3404074036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38009c10 con 0x7f7e24004d50 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3f200640 1 -- 172.21.15.45:0/3404074036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 msgr2=0x7f7e240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3404074036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e240051d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7e40074a20 tx=0x7f7e3801cbe0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3e800640 1 -- 172.21.15.45:0/3404074036 reap_dead start 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3f200640 1 -- 172.21.15.45:0/3404074036 shutdown_connections 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3f200640 1 -- 172.21.15.45:0/3404074036 >> 172.21.15.45:0/3404074036 conn(0x7f7e400d1750 msgr2=0x7f7e400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3f200640 1 -- 172.21.15.45:0/3404074036 shutdown_connections 2024-04-25T10:22:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.826+0000 7f7e3f200640 1 -- 172.21.15.45:0/3404074036 wait complete. 2024-04-25T10:22:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 Processor -- start 2024-04-25T10:22:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 -- start start 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e400b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e400cb1c0 0x7f7e401f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e401f6d10 0x7f7e401f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400d0bb0 con 0x7f7e24004d50 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400cbda0 con 0x7f7e400cb1c0 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400cddc0 con 0x7f7e401f6d10 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e400b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e400b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46254/0 (socket says 172.21.15.45:46254) 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 -- 172.21.15.45:0/3608337911 learned_addr learned my addr 172.21.15.45:0/3608337911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e401f6d10 0x7f7e401f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3d400640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e400cb1c0 0x7f7e401f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 -- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e401f6d10 msgr2=0x7f7e401f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e401f6d10 0x7f7e401f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 -- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e400cb1c0 msgr2=0x7f7e401f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e400cb1c0 0x7f7e401f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 -- 172.21.15.45:0/3608337911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400b6990 con 0x7f7e24004d50 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3e800640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e401f6d10 0x7f7e401f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3de00640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e400b6100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7e30011950 tx=0x7f7e30011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30010040 con 0x7f7e24004d50 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3001a070 con 0x7f7e24004d50 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000fe20 con 0x7f7e24004d50 2024-04-25T10:22:31.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e400c88c0 con 0x7f7e24004d50 2024-04-25T10:22:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.830+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e400c8e80 con 0x7f7e24004d50 2024-04-25T10:22:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.834+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e40075120 con 0x7f7e24004d50 2024-04-25T10:22:31.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.834+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e3001f460 con 0x7f7e24004d50 2024-04-25T10:22:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.834+0000 7f7e36000640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e20066f60 0x7f7e20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.834+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e30088320 con 0x7f7e24004d50 2024-04-25T10:22:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.834+0000 7f7e3d400640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e20066f60 0x7f7e20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:31.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.834+0000 7f7e3d400640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e20066f60 0x7f7e20069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ca380 tx=0x7f7e2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:31.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.838+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e30059320 con 0x7f7e24004d50 2024-04-25T10:22:32.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:31.998+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e4015e6c0 con 0x7f7e20066f60 2024-04-25T10:22:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.002+0000 7f7e36000640 1 -- 172.21.15.45:0/3608337911 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252064 (secure 0 0 0) 0x7f7e4015e6c0 con 0x7f7e20066f60 2024-04-25T10:22:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:32.011 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:22:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e20066f60 msgr2=0x7f7e20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e20066f60 0x7f7e20069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ca380 tx=0x7f7e2c009040 comp rx=0 tx=0).stop 2024-04-25T10:22:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 msgr2=0x7f7e400b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e400b6100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7e30011950 tx=0x7f7e30011e10 comp rx=0 tx=0).stop 2024-04-25T10:22:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 shutdown_connections 2024-04-25T10:22:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e20066f60 0x7f7e20069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e401f6d10 0x7f7e401f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e400cb1c0 0x7f7e401f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 --2- 172.21.15.45:0/3608337911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e24004d50 0x7f7e400b6100 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 >> 172.21.15.45:0/3608337911 conn(0x7f7e400d1750 msgr2=0x7f7e400d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 shutdown_connections 2024-04-25T10:22:32.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.006+0000 7f7e3f200640 1 -- 172.21.15.45:0/3608337911 wait complete. 2024-04-25T10:22:32.030 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T10:22:30.935190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":325812,"kb_used_data":8392,"kb_used_omap":12,"kb_used_meta":317299,"kb_avail":754648908,"statfs":{"total":773094113280,"available":772760481792,"internally_reserved":0,"allocated":8593408,"data_stored":4301156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":324914736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.440749"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531052+0000","last_change":"2024-04-25T10:22:27.529213+0000","last_active":"2024-04-25T10:22:28.531052+0000","last_peered":"2024-04-25T10:22:28.531052+0000","last_clean":"2024-04-25T10:22:28.531052+0000","last_became_active":"2024-04-25T10:22:27.528941+0000","last_became_peered":"2024-04-25T10:22:27.528941+0000","last_unstale":"2024-04-25T10:22:28.531052+0000","last_undegraded":"2024-04-25T10:22:28.531052+0000","last_fullsized":"2024-04-25T10:22:28.531052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:32.517081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522761+0000","last_change":"2024-04-25T10:22:27.547825+0000","last_active":"2024-04-25T10:22:28.522761+0000","last_peered":"2024-04-25T10:22:28.522761+0000","last_clean":"2024-04-25T10:22:28.522761+0000","last_became_active":"2024-04-25T10:22:27.546835+0000","last_became_peered":"2024-04-25T10:22:27.546835+0000","last_unstale":"2024-04-25T10:22:28.522761+0000","last_undegraded":"2024-04-25T10:22:28.522761+0000","last_fullsized":"2024-04-25T10:22:28.522761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:08.994301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727910+0000","last_change":"2024-04-25T10:22:27.520176+0000","last_active":"2024-04-25T10:22:28.727910+0000","last_peered":"2024-04-25T10:22:28.727910+0000","last_clean":"2024-04-25T10:22:28.727910+0000","last_became_active":"2024-04-25T10:22:27.519856+0000","last_became_peered":"2024-04-25T10:22:27.519856+0000","last_unstale":"2024-04-25T10:22:28.727910+0000","last_undegraded":"2024-04-25T10:22:28.727910+0000","last_fullsized":"2024-04-25T10:22:28.727910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.670849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727893+0000","last_change":"2024-04-25T10:22:27.521233+0000","last_active":"2024-04-25T10:22:28.727893+0000","last_peered":"2024-04-25T10:22:28.727893+0000","last_clean":"2024-04-25T10:22:28.727893+0000","last_became_active":"2024-04-25T10:22:27.520962+0000","last_became_peered":"2024-04-25T10:22:27.520962+0000","last_unstale":"2024-04-25T10:22:28.727893+0000","last_undegraded":"2024-04-25T10:22:28.727893+0000","last_fullsized":"2024-04-25T10:22:28.727893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:17.806191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.521992+0000","last_change":"2024-04-25T10:22:27.539324+0000","last_active":"2024-04-25T10:22:28.521992+0000","last_peered":"2024-04-25T10:22:28.521992+0000","last_clean":"2024-04-25T10:22:28.521992+0000","last_became_active":"2024-04-25T10:22:27.539045+0000","last_became_peered":"2024-04-25T10:22:27.539045+0000","last_unstale":"2024-04-25T10:22:28.521992+0000","last_undegraded":"2024-04-25T10:22:28.521992+0000","last_fullsized":"2024-04-25T10:22:28.521992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:53.106627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727803+0000","last_change":"2024-04-25T10:22:27.551534+0000","last_active":"2024-04-25T10:22:28.727803+0000","last_peered":"2024-04-25T10:22:28.727803+0000","last_clean":"2024-04-25T10:22:28.727803+0000","last_became_active":"2024-04-25T10:22:27.551259+0000","last_became_peered":"2024-04-25T10:22:27.551259+0000","last_unstale":"2024-04-25T10:22:28.727803+0000","last_undegraded":"2024-04-25T10:22:28.727803+0000","last_fullsized":"2024-04-25T10:22:28.727803+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.532569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519471+0000","last_change":"2024-04-25T10:22:27.532076+0000","last_active":"2024-04-25T10:22:28.519471+0000","last_peered":"2024-04-25T10:22:28.519471+0000","last_clean":"2024-04-25T10:22:28.519471+0000","last_became_active":"2024-04-25T10:22:27.531610+0000","last_became_peered":"2024-04-25T10:22:27.531610+0000","last_unstale":"2024-04-25T10:22:28.519471+0000","last_undegraded":"2024-04-25T10:22:28.519471+0000","last_fullsized":"2024-04-25T10:22:28.519471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.670849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530541+0000","last_change":"2024-04-25T10:22:27.527219+0000","last_active":"2024-04-25T10:22:28.530541+0000","last_peered":"2024-04-25T10:22:28.530541+0000","last_clean":"2024-04-25T10:22:28.530541+0000","last_became_active":"2024-04-25T10:22:27.526931+0000","last_became_peered":"2024-04-25T10:22:27.526931+0000","last_unstale":"2024-04-25T10:22:28.530541+0000","last_undegraded":"2024-04-25T10:22:28.530541+0000","last_fullsized":"2024-04-25T10:22:28.530541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:50.540203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727851+0000","last_change":"2024-04-25T10:22:27.524376+0000","last_active":"2024-04-25T10:22:28.727851+0000","last_peered":"2024-04-25T10:22:28.727851+0000","last_clean":"2024-04-25T10:22:28.727851+0000","last_became_active":"2024-04-25T10:22:27.523830+0000","last_became_peered":"2024-04-25T10:22:27.523830+0000","last_unstale":"2024-04-25T10:22:28.727851+0000","last_undegraded":"2024-04-25T10:22:28.727851+0000","last_fullsized":"2024-04-25T10:22:28.727851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:10.172650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.520182+0000","last_change":"2024-04-25T10:22:27.531332+0000","last_active":"2024-04-25T10:22:28.520182+0000","last_peered":"2024-04-25T10:22:28.520182+0000","last_clean":"2024-04-25T10:22:28.520182+0000","last_became_active":"2024-04-25T10:22:27.530328+0000","last_became_peered":"2024-04-25T10:22:27.530328+0000","last_unstale":"2024-04-25T10:22:28.520182+0000","last_undegraded":"2024-04-25T10:22:28.520182+0000","last_fullsized":"2024-04-25T10:22:28.520182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:05.015827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531327+0000","last_change":"2024-04-25T10:22:27.527162+0000","last_active":"2024-04-25T10:22:28.531327+0000","last_peered":"2024-04-25T10:22:28.531327+0000","last_clean":"2024-04-25T10:22:28.531327+0000","last_became_active":"2024-04-25T10:22:27.526804+0000","last_became_peered":"2024-04-25T10:22:27.526804+0000","last_unstale":"2024-04-25T10:22:28.531327+0000","last_undegraded":"2024-04-25T10:22:28.531327+0000","last_fullsized":"2024-04-25T10:22:28.531327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:56.706006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.533258+0000","last_change":"2024-04-25T10:22:27.529596+0000","last_active":"2024-04-25T10:22:27.533258+0000","last_peered":"2024-04-25T10:22:27.533258+0000","last_clean":"2024-04-25T10:22:27.533258+0000","last_became_active":"2024-04-25T10:22:27.529306+0000","last_became_peered":"2024-04-25T10:22:27.529306+0000","last_unstale":"2024-04-25T10:22:27.533258+0000","last_undegraded":"2024-04-25T10:22:27.533258+0000","last_fullsized":"2024-04-25T10:22:27.533258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:34.712976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.729053+0000","last_change":"2024-04-25T10:22:27.522735+0000","last_active":"2024-04-25T10:22:28.729053+0000","last_peered":"2024-04-25T10:22:28.729053+0000","last_clean":"2024-04-25T10:22:28.729053+0000","last_became_active":"2024-04-25T10:22:27.522097+0000","last_became_peered":"2024-04-25T10:22:27.522097+0000","last_unstale":"2024-04-25T10:22:28.729053+0000","last_undegraded":"2024-04-25T10:22:28.729053+0000","last_fullsized":"2024-04-25T10:22:28.729053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:39.323925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.538423+0000","last_change":"2024-04-25T10:22:27.536278+0000","last_active":"2024-04-25T10:22:27.538423+0000","last_peered":"2024-04-25T10:22:27.538423+0000","last_clean":"2024-04-25T10:22:27.538423+0000","last_became_active":"2024-04-25T10:22:27.535980+0000","last_became_peered":"2024-04-25T10:22:27.535980+0000","last_unstale":"2024-04-25T10:22:27.538423+0000","last_undegraded":"2024-04-25T10:22:27.538423+0000","last_fullsized":"2024-04-25T10:22:27.538423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:43.686247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.533928+0000","last_change":"2024-04-25T10:22:27.530550+0000","last_active":"2024-04-25T10:22:27.533928+0000","last_peered":"2024-04-25T10:22:27.533928+0000","last_clean":"2024-04-25T10:22:27.533928+0000","last_became_active":"2024-04-25T10:22:27.528935+0000","last_became_peered":"2024-04-25T10:22:27.528935+0000","last_unstale":"2024-04-25T10:22:27.533928+0000","last_undegraded":"2024-04-25T10:22:27.533928+0000","last_fullsized":"2024-04-25T10:22:27.533928+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:13.188900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530541+0000","last_change":"2024-04-25T10:22:27.545207+0000","last_active":"2024-04-25T10:22:28.530541+0000","last_peered":"2024-04-25T10:22:28.530541+0000","last_clean":"2024-04-25T10:22:28.530541+0000","last_became_active":"2024-04-25T10:22:27.544920+0000","last_became_peered":"2024-04-25T10:22:27.544920+0000","last_unstale":"2024-04-25T10:22:28.530541+0000","last_undegraded":"2024-04-25T10:22:28.530541+0000","last_fullsized":"2024-04-25T10:22:28.530541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:40.523825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522793+0000","last_change":"2024-04-25T10:22:27.522799+0000","last_active":"2024-04-25T10:22:28.522793+0000","last_peered":"2024-04-25T10:22:28.522793+0000","last_clean":"2024-04-25T10:22:28.522793+0000","last_became_active":"2024-04-25T10:22:27.522539+0000","last_became_peered":"2024-04-25T10:22:27.522539+0000","last_unstale":"2024-04-25T10:22:28.522793+0000","last_undegraded":"2024-04-25T10:22:28.522793+0000","last_fullsized":"2024-04-25T10:22:28.522793+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:10.006564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522645+0000","last_change":"2024-04-25T10:22:27.520044+0000","last_active":"2024-04-25T10:22:28.522645+0000","last_peered":"2024-04-25T10:22:28.522645+0000","last_clean":"2024-04-25T10:22:28.522645+0000","last_became_active":"2024-04-25T10:22:27.519782+0000","last_became_peered":"2024-04-25T10:22:27.519782+0000","last_unstale":"2024-04-25T10:22:28.522645+0000","last_undegraded":"2024-04-25T10:22:28.522645+0000","last_fullsized":"2024-04-25T10:22:28.522645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:25.652315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531141+0000","last_change":"2024-04-25T10:22:27.522915+0000","last_active":"2024-04-25T10:22:28.531141+0000","last_peered":"2024-04-25T10:22:28.531141+0000","last_clean":"2024-04-25T10:22:28.531141+0000","last_became_active":"2024-04-25T10:22:27.522500+0000","last_became_peered":"2024-04-25T10:22:27.522500+0000","last_unstale":"2024-04-25T10:22:28.531141+0000","last_undegraded":"2024-04-25T10:22:28.531141+0000","last_fullsized":"2024-04-25T10:22:28.531141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:45.431390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522239+0000","last_change":"2024-04-25T10:22:27.532976+0000","last_active":"2024-04-25T10:22:28.522239+0000","last_peered":"2024-04-25T10:22:28.522239+0000","last_clean":"2024-04-25T10:22:28.522239+0000","last_became_active":"2024-04-25T10:22:27.532680+0000","last_became_peered":"2024-04-25T10:22:27.532680+0000","last_unstale":"2024-04-25T10:22:28.522239+0000","last_undegraded":"2024-04-25T10:22:28.522239+0000","last_fullsized":"2024-04-25T10:22:28.522239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:45.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522588+0000","last_change":"2024-04-25T10:22:27.537777+0000","last_active":"2024-04-25T10:22:28.522588+0000","last_peered":"2024-04-25T10:22:28.522588+0000","last_clean":"2024-04-25T10:22:28.522588+0000","last_became_active":"2024-04-25T10:22:27.535147+0000","last_became_peered":"2024-04-25T10:22:27.535147+0000","last_unstale":"2024-04-25T10:22:28.522588+0000","last_undegraded":"2024-04-25T10:22:28.522588+0000","last_fullsized":"2024-04-25T10:22:28.522588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:24.917800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727806+0000","last_change":"2024-04-25T10:22:27.531095+0000","last_active":"2024-04-25T10:22:28.727806+0000","last_peered":"2024-04-25T10:22:28.727806+0000","last_clean":"2024-04-25T10:22:28.727806+0000","last_became_active":"2024-04-25T10:22:27.530511+0000","last_became_peered":"2024-04-25T10:22:27.530511+0000","last_unstale":"2024-04-25T10:22:28.727806+0000","last_undegraded":"2024-04-25T10:22:28.727806+0000","last_fullsized":"2024-04-25T10:22:28.727806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:55.195531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519472+0000","last_change":"2024-04-25T10:22:27.545792+0000","last_active":"2024-04-25T10:22:28.519472+0000","last_peered":"2024-04-25T10:22:28.519472+0000","last_clean":"2024-04-25T10:22:28.519472+0000","last_became_active":"2024-04-25T10:22:27.545451+0000","last_became_peered":"2024-04-25T10:22:27.545451+0000","last_unstale":"2024-04-25T10:22:28.519472+0000","last_undegraded":"2024-04-25T10:22:28.519472+0000","last_fullsized":"2024-04-25T10:22:28.519472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:16.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530735+0000","last_change":"2024-04-25T10:22:27.539944+0000","last_active":"2024-04-25T10:22:28.530735+0000","last_peered":"2024-04-25T10:22:28.530735+0000","last_clean":"2024-04-25T10:22:28.530735+0000","last_became_active":"2024-04-25T10:22:27.539090+0000","last_became_peered":"2024-04-25T10:22:27.539090+0000","last_unstale":"2024-04-25T10:22:28.530735+0000","last_undegraded":"2024-04-25T10:22:28.530735+0000","last_fullsized":"2024-04-25T10:22:28.530735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:31.963964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519998+0000","last_change":"2024-04-25T10:22:27.538284+0000","last_active":"2024-04-25T10:22:28.519998+0000","last_peered":"2024-04-25T10:22:28.519998+0000","last_clean":"2024-04-25T10:22:28.519998+0000","last_became_active":"2024-04-25T10:22:27.538003+0000","last_became_peered":"2024-04-25T10:22:27.538003+0000","last_unstale":"2024-04-25T10:22:28.519998+0000","last_undegraded":"2024-04-25T10:22:28.519998+0000","last_fullsized":"2024-04-25T10:22:28.519998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:06.092429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530897+0000","last_change":"2024-04-25T10:22:27.522458+0000","last_active":"2024-04-25T10:22:28.530897+0000","last_peered":"2024-04-25T10:22:28.530897+0000","last_clean":"2024-04-25T10:22:28.530897+0000","last_became_active":"2024-04-25T10:22:27.522134+0000","last_became_peered":"2024-04-25T10:22:27.522134+0000","last_unstale":"2024-04-25T10:22:28.530897+0000","last_undegraded":"2024-04-25T10:22:28.530897+0000","last_fullsized":"2024-04-25T10:22:28.530897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:08.564452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522277+0000","last_change":"2024-04-25T10:22:27.524055+0000","last_active":"2024-04-25T10:22:28.522277+0000","last_peered":"2024-04-25T10:22:28.522277+0000","last_clean":"2024-04-25T10:22:28.522277+0000","last_became_active":"2024-04-25T10:22:27.521372+0000","last_became_peered":"2024-04-25T10:22:27.521372+0000","last_unstale":"2024-04-25T10:22:28.522277+0000","last_undegraded":"2024-04-25T10:22:28.522277+0000","last_fullsized":"2024-04-25T10:22:28.522277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:36.171433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522545+0000","last_change":"2024-04-25T10:22:27.541204+0000","last_active":"2024-04-25T10:22:28.522545+0000","last_peered":"2024-04-25T10:22:28.522545+0000","last_clean":"2024-04-25T10:22:28.522545+0000","last_became_active":"2024-04-25T10:22:27.538442+0000","last_became_peered":"2024-04-25T10:22:27.538442+0000","last_unstale":"2024-04-25T10:22:28.522545+0000","last_undegraded":"2024-04-25T10:22:28.522545+0000","last_fullsized":"2024-04-25T10:22:28.522545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:19.112842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727703+0000","last_change":"2024-04-25T10:22:27.523039+0000","last_active":"2024-04-25T10:22:28.727703+0000","last_peered":"2024-04-25T10:22:28.727703+0000","last_clean":"2024-04-25T10:22:28.727703+0000","last_became_active":"2024-04-25T10:22:27.522481+0000","last_became_peered":"2024-04-25T10:22:27.522481+0000","last_unstale":"2024-04-25T10:22:28.727703+0000","last_undegraded":"2024-04-25T10:22:28.727703+0000","last_fullsized":"2024-04-25T10:22:28.727703+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:16.372996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530681+0000","last_change":"2024-04-25T10:22:27.524761+0000","last_active":"2024-04-25T10:22:28.530681+0000","last_peered":"2024-04-25T10:22:28.530681+0000","last_clean":"2024-04-25T10:22:28.530681+0000","last_became_active":"2024-04-25T10:22:27.524434+0000","last_became_peered":"2024-04-25T10:22:27.524434+0000","last_unstale":"2024-04-25T10:22:28.530681+0000","last_undegraded":"2024-04-25T10:22:28.530681+0000","last_fullsized":"2024-04-25T10:22:28.530681+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:23.136134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522401+0000","last_change":"2024-04-25T10:22:27.536748+0000","last_active":"2024-04-25T10:22:28.522401+0000","last_peered":"2024-04-25T10:22:28.522401+0000","last_clean":"2024-04-25T10:22:28.522401+0000","last_became_active":"2024-04-25T10:22:27.536193+0000","last_became_peered":"2024-04-25T10:22:27.536193+0000","last_unstale":"2024-04-25T10:22:28.522401+0000","last_undegraded":"2024-04-25T10:22:28.522401+0000","last_fullsized":"2024-04-25T10:22:28.522401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:19.765870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522323+0000","last_change":"2024-04-25T10:22:27.538504+0000","last_active":"2024-04-25T10:22:28.522323+0000","last_peered":"2024-04-25T10:22:28.522323+0000","last_clean":"2024-04-25T10:22:28.522323+0000","last_became_active":"2024-04-25T10:22:27.534304+0000","last_became_peered":"2024-04-25T10:22:27.534304+0000","last_unstale":"2024-04-25T10:22:28.522323+0000","last_undegraded":"2024-04-25T10:22:28.522323+0000","last_fullsized":"2024-04-25T10:22:28.522323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:26.886634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530986+0000","last_change":"2024-04-25T10:22:27.540348+0000","last_active":"2024-04-25T10:22:28.530986+0000","last_peered":"2024-04-25T10:22:28.530986+0000","last_clean":"2024-04-25T10:22:28.530986+0000","last_became_active":"2024-04-25T10:22:27.539557+0000","last_became_peered":"2024-04-25T10:22:27.539557+0000","last_unstale":"2024-04-25T10:22:28.530986+0000","last_undegraded":"2024-04-25T10:22:28.530986+0000","last_fullsized":"2024-04-25T10:22:28.530986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:45.500005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728141+0000","last_change":"2024-04-25T10:22:26.516020+0000","last_active":"2024-04-25T10:22:28.728141+0000","last_peered":"2024-04-25T10:22:28.728141+0000","last_clean":"2024-04-25T10:22:28.728141+0000","last_became_active":"2024-04-25T10:22:26.515759+0000","last_became_peered":"2024-04-25T10:22:26.515759+0000","last_unstale":"2024-04-25T10:22:28.728141+0000","last_undegraded":"2024-04-25T10:22:28.728141+0000","last_fullsized":"2024-04-25T10:22:28.728141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:07.590515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728034+0000","last_change":"2024-04-25T10:22:27.524144+0000","last_active":"2024-04-25T10:22:28.728034+0000","last_peered":"2024-04-25T10:22:28.728034+0000","last_clean":"2024-04-25T10:22:28.728034+0000","last_became_active":"2024-04-25T10:22:27.523463+0000","last_became_peered":"2024-04-25T10:22:27.523463+0000","last_unstale":"2024-04-25T10:22:28.728034+0000","last_undegraded":"2024-04-25T10:22:28.728034+0000","last_fullsized":"2024-04-25T10:22:28.728034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:41.475772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522752+0000","last_change":"2024-04-25T10:22:26.511037+0000","last_active":"2024-04-25T10:22:28.522752+0000","last_peered":"2024-04-25T10:22:28.522752+0000","last_clean":"2024-04-25T10:22:28.522752+0000","last_became_active":"2024-04-25T10:22:26.510718+0000","last_became_peered":"2024-04-25T10:22:26.510718+0000","last_unstale":"2024-04-25T10:22:28.522752+0000","last_undegraded":"2024-04-25T10:22:28.522752+0000","last_fullsized":"2024-04-25T10:22:28.522752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:53.906477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728054+0000","last_change":"2024-04-25T10:22:27.543573+0000","last_active":"2024-04-25T10:22:28.728054+0000","last_peered":"2024-04-25T10:22:28.728054+0000","last_clean":"2024-04-25T10:22:28.728054+0000","last_became_active":"2024-04-25T10:22:27.543258+0000","last_became_peered":"2024-04-25T10:22:27.543258+0000","last_unstale":"2024-04-25T10:22:28.728054+0000","last_undegraded":"2024-04-25T10:22:28.728054+0000","last_fullsized":"2024-04-25T10:22:28.728054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:37.697538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531328+0000","last_change":"2024-04-25T10:22:26.528433+0000","last_active":"2024-04-25T10:22:28.531328+0000","last_peered":"2024-04-25T10:22:28.531328+0000","last_clean":"2024-04-25T10:22:28.531328+0000","last_became_active":"2024-04-25T10:22:26.528149+0000","last_became_peered":"2024-04-25T10:22:26.528149+0000","last_unstale":"2024-04-25T10:22:28.531328+0000","last_undegraded":"2024-04-25T10:22:28.531328+0000","last_fullsized":"2024-04-25T10:22:28.531328+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:55.748507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519923+0000","last_change":"2024-04-25T10:22:27.535896+0000","last_active":"2024-04-25T10:22:28.519923+0000","last_peered":"2024-04-25T10:22:28.519923+0000","last_clean":"2024-04-25T10:22:28.519923+0000","last_became_active":"2024-04-25T10:22:27.531993+0000","last_became_peered":"2024-04-25T10:22:27.531993+0000","last_unstale":"2024-04-25T10:22:28.519923+0000","last_undegraded":"2024-04-25T10:22:28.519923+0000","last_fullsized":"2024-04-25T10:22:28.519923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:39.323925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728043+0000","last_change":"2024-04-25T10:22:26.517778+0000","last_active":"2024-04-25T10:22:28.728043+0000","last_peered":"2024-04-25T10:22:28.728043+0000","last_clean":"2024-04-25T10:22:28.728043+0000","last_became_active":"2024-04-25T10:22:26.517503+0000","last_became_peered":"2024-04-25T10:22:26.517503+0000","last_unstale":"2024-04-25T10:22:28.728043+0000","last_undegraded":"2024-04-25T10:22:28.728043+0000","last_fullsized":"2024-04-25T10:22:28.728043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:10.624430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530685+0000","last_change":"2024-04-25T10:22:27.530913+0000","last_active":"2024-04-25T10:22:28.530685+0000","last_peered":"2024-04-25T10:22:28.530685+0000","last_clean":"2024-04-25T10:22:28.530685+0000","last_became_active":"2024-04-25T10:22:27.530621+0000","last_became_peered":"2024-04-25T10:22:27.530621+0000","last_unstale":"2024-04-25T10:22:28.530685+0000","last_undegraded":"2024-04-25T10:22:28.530685+0000","last_fullsized":"2024-04-25T10:22:28.530685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.372674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.553244+0000","last_change":"2024-04-25T10:22:26.519810+0000","last_active":"2024-04-25T10:22:27.553244+0000","last_peered":"2024-04-25T10:22:27.553244+0000","last_clean":"2024-04-25T10:22:27.553244+0000","last_became_active":"2024-04-25T10:22:26.519347+0000","last_became_peered":"2024-04-25T10:22:26.519347+0000","last_unstale":"2024-04-25T10:22:27.553244+0000","last_undegraded":"2024-04-25T10:22:27.553244+0000","last_fullsized":"2024-04-25T10:22:27.553244+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:16.615456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727894+0000","last_change":"2024-04-25T10:22:27.541723+0000","last_active":"2024-04-25T10:22:28.727894+0000","last_peered":"2024-04-25T10:22:28.727894+0000","last_clean":"2024-04-25T10:22:28.727894+0000","last_became_active":"2024-04-25T10:22:27.541273+0000","last_became_peered":"2024-04-25T10:22:27.541273+0000","last_unstale":"2024-04-25T10:22:28.727894+0000","last_undegraded":"2024-04-25T10:22:28.727894+0000","last_fullsized":"2024-04-25T10:22:28.727894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:50.444166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519566+0000","last_change":"2024-04-25T10:22:26.519410+0000","last_active":"2024-04-25T10:22:28.519566+0000","last_peered":"2024-04-25T10:22:28.519566+0000","last_clean":"2024-04-25T10:22:28.519566+0000","last_became_active":"2024-04-25T10:22:26.519108+0000","last_became_peered":"2024-04-25T10:22:26.519108+0000","last_unstale":"2024-04-25T10:22:28.519566+0000","last_undegraded":"2024-04-25T10:22:28.519566+0000","last_fullsized":"2024-04-25T10:22:28.519566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:50.573948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727802+0000","last_change":"2024-04-25T10:22:27.530306+0000","last_active":"2024-04-25T10:22:28.727802+0000","last_peered":"2024-04-25T10:22:28.727802+0000","last_clean":"2024-04-25T10:22:28.727802+0000","last_became_active":"2024-04-25T10:22:27.529600+0000","last_became_peered":"2024-04-25T10:22:27.529600+0000","last_unstale":"2024-04-25T10:22:28.727802+0000","last_undegraded":"2024-04-25T10:22:28.727802+0000","last_fullsized":"2024-04-25T10:22:28.727802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:47.135564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728679+0000","last_change":"2024-04-25T10:22:26.517926+0000","last_active":"2024-04-25T10:22:28.728679+0000","last_peered":"2024-04-25T10:22:28.728679+0000","last_clean":"2024-04-25T10:22:28.728679+0000","last_became_active":"2024-04-25T10:22:26.517562+0000","last_became_peered":"2024-04-25T10:22:26.517562+0000","last_unstale":"2024-04-25T10:22:28.728679+0000","last_undegraded":"2024-04-25T10:22:28.728679+0000","last_fullsized":"2024-04-25T10:22:28.728679+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:00.560567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.530422+0000","last_change":"2024-04-25T10:22:27.523607+0000","last_active":"2024-04-25T10:22:27.530422+0000","last_peered":"2024-04-25T10:22:27.530422+0000","last_clean":"2024-04-25T10:22:27.530422+0000","last_became_active":"2024-04-25T10:22:27.523107+0000","last_became_peered":"2024-04-25T10:22:27.523107+0000","last_unstale":"2024-04-25T10:22:27.530422+0000","last_undegraded":"2024-04-25T10:22:27.530422+0000","last_fullsized":"2024-04-25T10:22:27.530422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:56.381637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.518190+0000","last_change":"2024-04-25T10:22:26.513960+0000","last_active":"2024-04-25T10:22:27.518190+0000","last_peered":"2024-04-25T10:22:27.518190+0000","last_clean":"2024-04-25T10:22:27.518190+0000","last_became_active":"2024-04-25T10:22:26.513670+0000","last_became_peered":"2024-04-25T10:22:26.513670+0000","last_unstale":"2024-04-25T10:22:27.518190+0000","last_undegraded":"2024-04-25T10:22:27.518190+0000","last_fullsized":"2024-04-25T10:22:27.518190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:03.778988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.520309+0000","last_change":"2024-04-25T10:22:27.545987+0000","last_active":"2024-04-25T10:22:28.520309+0000","last_peered":"2024-04-25T10:22:28.520309+0000","last_clean":"2024-04-25T10:22:28.520309+0000","last_became_active":"2024-04-25T10:22:27.545427+0000","last_became_peered":"2024-04-25T10:22:27.545427+0000","last_unstale":"2024-04-25T10:22:28.520309+0000","last_undegraded":"2024-04-25T10:22:28.520309+0000","last_fullsized":"2024-04-25T10:22:28.520309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:37.876899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530894+0000","last_change":"2024-04-25T10:22:26.527205+0000","last_active":"2024-04-25T10:22:28.530894+0000","last_peered":"2024-04-25T10:22:28.530894+0000","last_clean":"2024-04-25T10:22:28.530894+0000","last_became_active":"2024-04-25T10:22:26.526865+0000","last_became_peered":"2024-04-25T10:22:26.526865+0000","last_unstale":"2024-04-25T10:22:28.530894+0000","last_undegraded":"2024-04-25T10:22:28.530894+0000","last_fullsized":"2024-04-25T10:22:28.530894+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:26.869712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522826+0000","last_change":"2024-04-25T10:22:27.535815+0000","last_active":"2024-04-25T10:22:28.522826+0000","last_peered":"2024-04-25T10:22:28.522826+0000","last_clean":"2024-04-25T10:22:28.522826+0000","last_became_active":"2024-04-25T10:22:27.535478+0000","last_became_peered":"2024-04-25T10:22:27.535478+0000","last_unstale":"2024-04-25T10:22:28.522826+0000","last_undegraded":"2024-04-25T10:22:28.522826+0000","last_fullsized":"2024-04-25T10:22:28.522826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:08.598913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522854+0000","last_change":"2024-04-25T10:22:26.522527+0000","last_active":"2024-04-25T10:22:28.522854+0000","last_peered":"2024-04-25T10:22:28.522854+0000","last_clean":"2024-04-25T10:22:28.522854+0000","last_became_active":"2024-04-25T10:22:26.522210+0000","last_became_peered":"2024-04-25T10:22:26.522210+0000","last_unstale":"2024-04-25T10:22:28.522854+0000","last_undegraded":"2024-04-25T10:22:28.522854+0000","last_fullsized":"2024-04-25T10:22:28.522854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:07.985166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530898+0000","last_change":"2024-04-25T10:22:27.523140+0000","last_active":"2024-04-25T10:22:28.530898+0000","last_peered":"2024-04-25T10:22:28.530898+0000","last_clean":"2024-04-25T10:22:28.530898+0000","last_became_active":"2024-04-25T10:22:27.522776+0000","last_became_peered":"2024-04-25T10:22:27.522776+0000","last_unstale":"2024-04-25T10:22:28.530898+0000","last_undegraded":"2024-04-25T10:22:28.530898+0000","last_fullsized":"2024-04-25T10:22:28.530898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:16.796012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.547792+0000","last_change":"2024-04-25T10:22:26.511939+0000","last_active":"2024-04-25T10:22:27.547792+0000","last_peered":"2024-04-25T10:22:27.547792+0000","last_clean":"2024-04-25T10:22:27.547792+0000","last_became_active":"2024-04-25T10:22:26.511607+0000","last_became_peered":"2024-04-25T10:22:26.511607+0000","last_unstale":"2024-04-25T10:22:27.547792+0000","last_undegraded":"2024-04-25T10:22:27.547792+0000","last_fullsized":"2024-04-25T10:22:27.547792+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:46.854374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522307+0000","last_change":"2024-04-25T10:22:27.524309+0000","last_active":"2024-04-25T10:22:28.522307+0000","last_peered":"2024-04-25T10:22:28.522307+0000","last_clean":"2024-04-25T10:22:28.522307+0000","last_became_active":"2024-04-25T10:22:27.521343+0000","last_became_peered":"2024-04-25T10:22:27.521343+0000","last_unstale":"2024-04-25T10:22:28.522307+0000","last_undegraded":"2024-04-25T10:22:28.522307+0000","last_fullsized":"2024-04-25T10:22:28.522307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:12.954149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728238+0000","last_change":"2024-04-25T10:22:26.517908+0000","last_active":"2024-04-25T10:22:28.728238+0000","last_peered":"2024-04-25T10:22:28.728238+0000","last_clean":"2024-04-25T10:22:28.728238+0000","last_became_active":"2024-04-25T10:22:26.517616+0000","last_became_peered":"2024-04-25T10:22:26.517616+0000","last_unstale":"2024-04-25T10:22:28.728238+0000","last_undegraded":"2024-04-25T10:22:28.728238+0000","last_fullsized":"2024-04-25T10:22:28.728238+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:46.067958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728010+0000","last_change":"2024-04-25T10:22:27.530845+0000","last_active":"2024-04-25T10:22:28.728010+0000","last_peered":"2024-04-25T10:22:28.728010+0000","last_clean":"2024-04-25T10:22:28.728010+0000","last_became_active":"2024-04-25T10:22:27.530176+0000","last_became_peered":"2024-04-25T10:22:27.530176+0000","last_unstale":"2024-04-25T10:22:28.728010+0000","last_undegraded":"2024-04-25T10:22:28.728010+0000","last_fullsized":"2024-04-25T10:22:28.728010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:08.917931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728123+0000","last_change":"2024-04-25T10:22:26.526208+0000","last_active":"2024-04-25T10:22:28.728123+0000","last_peered":"2024-04-25T10:22:28.728123+0000","last_clean":"2024-04-25T10:22:28.728123+0000","last_became_active":"2024-04-25T10:22:26.525912+0000","last_became_peered":"2024-04-25T10:22:26.525912+0000","last_unstale":"2024-04-25T10:22:28.728123+0000","last_undegraded":"2024-04-25T10:22:28.728123+0000","last_fullsized":"2024-04-25T10:22:28.728123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:50.711675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519710+0000","last_change":"2024-04-25T10:22:27.529937+0000","last_active":"2024-04-25T10:22:28.519710+0000","last_peered":"2024-04-25T10:22:28.519710+0000","last_clean":"2024-04-25T10:22:28.519710+0000","last_became_active":"2024-04-25T10:22:27.529634+0000","last_became_peered":"2024-04-25T10:22:27.529634+0000","last_unstale":"2024-04-25T10:22:28.519710+0000","last_undegraded":"2024-04-25T10:22:28.519710+0000","last_fullsized":"2024-04-25T10:22:28.519710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:41.475772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519553+0000","last_change":"2024-04-25T10:22:26.519164+0000","last_active":"2024-04-25T10:22:28.519553+0000","last_peered":"2024-04-25T10:22:28.519553+0000","last_clean":"2024-04-25T10:22:28.519553+0000","last_became_active":"2024-04-25T10:22:26.518878+0000","last_became_peered":"2024-04-25T10:22:26.518878+0000","last_unstale":"2024-04-25T10:22:28.519553+0000","last_undegraded":"2024-04-25T10:22:28.519553+0000","last_fullsized":"2024-04-25T10:22:28.519553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:14.209043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.521893+0000","last_change":"2024-04-25T10:22:27.530002+0000","last_active":"2024-04-25T10:22:28.521893+0000","last_peered":"2024-04-25T10:22:28.521893+0000","last_clean":"2024-04-25T10:22:28.521893+0000","last_became_active":"2024-04-25T10:22:27.529388+0000","last_became_peered":"2024-04-25T10:22:27.529388+0000","last_unstale":"2024-04-25T10:22:28.521893+0000","last_undegraded":"2024-04-25T10:22:28.521893+0000","last_fullsized":"2024-04-25T10:22:28.521893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:56.756906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519588+0000","last_change":"2024-04-25T10:22:26.517915+0000","last_active":"2024-04-25T10:22:28.519588+0000","last_peered":"2024-04-25T10:22:28.519588+0000","last_clean":"2024-04-25T10:22:28.519588+0000","last_became_active":"2024-04-25T10:22:26.517710+0000","last_became_peered":"2024-04-25T10:22:26.517710+0000","last_unstale":"2024-04-25T10:22:28.519588+0000","last_undegraded":"2024-04-25T10:22:28.519588+0000","last_fullsized":"2024-04-25T10:22:28.519588+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:52.164535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727648+0000","last_change":"2024-04-25T10:22:27.523422+0000","last_active":"2024-04-25T10:22:28.727648+0000","last_peered":"2024-04-25T10:22:28.727648+0000","last_clean":"2024-04-25T10:22:28.727648+0000","last_became_active":"2024-04-25T10:22:27.523023+0000","last_became_peered":"2024-04-25T10:22:27.523023+0000","last_unstale":"2024-04-25T10:22:28.727648+0000","last_undegraded":"2024-04-25T10:22:28.727648+0000","last_fullsized":"2024-04-25T10:22:28.727648+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:18.358476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519306+0000","last_change":"2024-04-25T10:22:26.512094+0000","last_active":"2024-04-25T10:22:28.519306+0000","last_peered":"2024-04-25T10:22:28.519306+0000","last_clean":"2024-04-25T10:22:28.519306+0000","last_became_active":"2024-04-25T10:22:26.511120+0000","last_became_peered":"2024-04-25T10:22:26.511120+0000","last_unstale":"2024-04-25T10:22:28.519306+0000","last_undegraded":"2024-04-25T10:22:28.519306+0000","last_fullsized":"2024-04-25T10:22:28.519306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:00.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522870+0000","last_change":"2024-04-25T10:22:27.536272+0000","last_active":"2024-04-25T10:22:28.522870+0000","last_peered":"2024-04-25T10:22:28.522870+0000","last_clean":"2024-04-25T10:22:28.522870+0000","last_became_active":"2024-04-25T10:22:27.534471+0000","last_became_peered":"2024-04-25T10:22:27.534471+0000","last_unstale":"2024-04-25T10:22:28.522870+0000","last_undegraded":"2024-04-25T10:22:28.522870+0000","last_fullsized":"2024-04-25T10:22:28.522870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:03.173613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530967+0000","last_change":"2024-04-25T10:22:26.526525+0000","last_active":"2024-04-25T10:22:28.530967+0000","last_peered":"2024-04-25T10:22:28.530967+0000","last_clean":"2024-04-25T10:22:28.530967+0000","last_became_active":"2024-04-25T10:22:26.525984+0000","last_became_peered":"2024-04-25T10:22:26.525984+0000","last_unstale":"2024-04-25T10:22:28.530967+0000","last_undegraded":"2024-04-25T10:22:28.530967+0000","last_fullsized":"2024-04-25T10:22:28.530967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:52.098228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522837+0000","last_change":"2024-04-25T10:22:27.535755+0000","last_active":"2024-04-25T10:22:28.522837+0000","last_peered":"2024-04-25T10:22:28.522837+0000","last_clean":"2024-04-25T10:22:28.522837+0000","last_became_active":"2024-04-25T10:22:27.535458+0000","last_became_peered":"2024-04-25T10:22:27.535458+0000","last_unstale":"2024-04-25T10:22:28.522837+0000","last_undegraded":"2024-04-25T10:22:28.522837+0000","last_fullsized":"2024-04-25T10:22:28.522837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:02.947463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522927+0000","last_change":"2024-04-25T10:22:26.510224+0000","last_active":"2024-04-25T10:22:28.522927+0000","last_peered":"2024-04-25T10:22:28.522927+0000","last_clean":"2024-04-25T10:22:28.522927+0000","last_became_active":"2024-04-25T10:22:26.509906+0000","last_became_peered":"2024-04-25T10:22:26.509906+0000","last_unstale":"2024-04-25T10:22:28.522927+0000","last_undegraded":"2024-04-25T10:22:28.522927+0000","last_fullsized":"2024-04-25T10:22:28.522927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:06.554223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727944+0000","last_change":"2024-04-25T10:22:27.530945+0000","last_active":"2024-04-25T10:22:28.727944+0000","last_peered":"2024-04-25T10:22:28.727944+0000","last_clean":"2024-04-25T10:22:28.727944+0000","last_became_active":"2024-04-25T10:22:27.530665+0000","last_became_peered":"2024-04-25T10:22:27.530665+0000","last_unstale":"2024-04-25T10:22:28.727944+0000","last_undegraded":"2024-04-25T10:22:28.727944+0000","last_fullsized":"2024-04-25T10:22:28.727944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:16.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522576+0000","last_change":"2024-04-25T10:22:26.528258+0000","last_active":"2024-04-25T10:22:28.522576+0000","last_peered":"2024-04-25T10:22:28.522576+0000","last_clean":"2024-04-25T10:22:28.522576+0000","last_became_active":"2024-04-25T10:22:26.527909+0000","last_became_peered":"2024-04-25T10:22:26.527909+0000","last_unstale":"2024-04-25T10:22:28.522576+0000","last_undegraded":"2024-04-25T10:22:28.522576+0000","last_fullsized":"2024-04-25T10:22:28.522576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:04.007429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728072+0000","last_change":"2024-04-25T10:22:27.520973+0000","last_active":"2024-04-25T10:22:28.728072+0000","last_peered":"2024-04-25T10:22:28.728072+0000","last_clean":"2024-04-25T10:22:28.728072+0000","last_became_active":"2024-04-25T10:22:27.520714+0000","last_became_peered":"2024-04-25T10:22:27.520714+0000","last_unstale":"2024-04-25T10:22:28.728072+0000","last_undegraded":"2024-04-25T10:22:28.728072+0000","last_fullsized":"2024-04-25T10:22:28.728072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728135+0000","last_change":"2024-04-25T10:22:26.525845+0000","last_active":"2024-04-25T10:22:28.728135+0000","last_peered":"2024-04-25T10:22:28.728135+0000","last_clean":"2024-04-25T10:22:28.728135+0000","last_became_active":"2024-04-25T10:22:26.525559+0000","last_became_peered":"2024-04-25T10:22:26.525559+0000","last_unstale":"2024-04-25T10:22:28.728135+0000","last_undegraded":"2024-04-25T10:22:28.728135+0000","last_fullsized":"2024-04-25T10:22:28.728135+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:07.985903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727968+0000","last_change":"2024-04-25T10:22:27.523903+0000","last_active":"2024-04-25T10:22:28.727968+0000","last_peered":"2024-04-25T10:22:28.727968+0000","last_clean":"2024-04-25T10:22:28.727968+0000","last_became_active":"2024-04-25T10:22:27.523627+0000","last_became_peered":"2024-04-25T10:22:27.523627+0000","last_unstale":"2024-04-25T10:22:28.727968+0000","last_undegraded":"2024-04-25T10:22:28.727968+0000","last_fullsized":"2024-04-25T10:22:28.727968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:07.127333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728105+0000","last_change":"2024-04-25T10:22:26.521395+0000","last_active":"2024-04-25T10:22:28.728105+0000","last_peered":"2024-04-25T10:22:28.728105+0000","last_clean":"2024-04-25T10:22:28.728105+0000","last_became_active":"2024-04-25T10:22:26.519955+0000","last_became_peered":"2024-04-25T10:22:26.519955+0000","last_unstale":"2024-04-25T10:22:28.728105+0000","last_undegraded":"2024-04-25T10:22:28.728105+0000","last_fullsized":"2024-04-25T10:22:28.728105+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:18.104444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728014+0000","last_change":"2024-04-25T10:22:27.528088+0000","last_active":"2024-04-25T10:22:28.728014+0000","last_peered":"2024-04-25T10:22:28.728014+0000","last_clean":"2024-04-25T10:22:28.728014+0000","last_became_active":"2024-04-25T10:22:27.527794+0000","last_became_peered":"2024-04-25T10:22:27.527794+0000","last_unstale":"2024-04-25T10:22:28.728014+0000","last_undegraded":"2024-04-25T10:22:28.728014+0000","last_fullsized":"2024-04-25T10:22:28.728014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:26.886634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522447+0000","last_change":"2024-04-25T10:22:26.516828+0000","last_active":"2024-04-25T10:22:28.522447+0000","last_peered":"2024-04-25T10:22:28.522447+0000","last_clean":"2024-04-25T10:22:28.522447+0000","last_became_active":"2024-04-25T10:22:26.516390+0000","last_became_peered":"2024-04-25T10:22:26.516390+0000","last_unstale":"2024-04-25T10:22:28.522447+0000","last_undegraded":"2024-04-25T10:22:28.522447+0000","last_fullsized":"2024-04-25T10:22:28.522447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:25.763795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.529334+0000","last_change":"2024-04-25T10:22:27.523385+0000","last_active":"2024-04-25T10:22:27.529334+0000","last_peered":"2024-04-25T10:22:27.529334+0000","last_clean":"2024-04-25T10:22:27.529334+0000","last_became_active":"2024-04-25T10:22:27.522814+0000","last_became_peered":"2024-04-25T10:22:27.522814+0000","last_unstale":"2024-04-25T10:22:27.529334+0000","last_undegraded":"2024-04-25T10:22:27.529334+0000","last_fullsized":"2024-04-25T10:22:27.529334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:37.528737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.521778+0000","last_change":"2024-04-25T10:22:26.510683+0000","last_active":"2024-04-25T10:22:28.521778+0000","last_peered":"2024-04-25T10:22:28.521778+0000","last_clean":"2024-04-25T10:22:28.521778+0000","last_became_active":"2024-04-25T10:22:26.510377+0000","last_became_peered":"2024-04-25T10:22:26.510377+0000","last_unstale":"2024-04-25T10:22:28.521778+0000","last_undegraded":"2024-04-25T10:22:28.521778+0000","last_fullsized":"2024-04-25T10:22:28.521778+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:16.923299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522655+0000","last_change":"2024-04-25T10:22:27.523260+0000","last_active":"2024-04-25T10:22:28.522655+0000","last_peered":"2024-04-25T10:22:28.522655+0000","last_clean":"2024-04-25T10:22:28.522655+0000","last_became_active":"2024-04-25T10:22:27.522433+0000","last_became_peered":"2024-04-25T10:22:27.522433+0000","last_unstale":"2024-04-25T10:22:28.522655+0000","last_undegraded":"2024-04-25T10:22:28.522655+0000","last_fullsized":"2024-04-25T10:22:28.522655+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:36.566530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.531107+0000","last_change":"2024-04-25T10:22:26.526296+0000","last_active":"2024-04-25T10:22:28.531107+0000","last_peered":"2024-04-25T10:22:28.531107+0000","last_clean":"2024-04-25T10:22:28.531107+0000","last_became_active":"2024-04-25T10:22:26.525710+0000","last_became_peered":"2024-04-25T10:22:26.525710+0000","last_unstale":"2024-04-25T10:22:28.531107+0000","last_undegraded":"2024-04-25T10:22:28.531107+0000","last_fullsized":"2024-04-25T10:22:28.531107+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:16.280940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.520124+0000","last_change":"2024-04-25T10:22:27.530737+0000","last_active":"2024-04-25T10:22:28.520124+0000","last_peered":"2024-04-25T10:22:28.520124+0000","last_clean":"2024-04-25T10:22:28.520124+0000","last_became_active":"2024-04-25T10:22:27.530433+0000","last_became_peered":"2024-04-25T10:22:27.530433+0000","last_unstale":"2024-04-25T10:22:28.520124+0000","last_undegraded":"2024-04-25T10:22:28.520124+0000","last_fullsized":"2024-04-25T10:22:28.520124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:26.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"77'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.532329+0000","last_change":"2024-04-25T10:22:26.524426+0000","last_active":"2024-04-25T10:22:28.532329+0000","last_peered":"2024-04-25T10:22:28.532329+0000","last_clean":"2024-04-25T10:22:28.532329+0000","last_became_active":"2024-04-25T10:22:26.524142+0000","last_became_peered":"2024-04-25T10:22:26.524142+0000","last_unstale":"2024-04-25T10:22:28.532329+0000","last_undegraded":"2024-04-25T10:22:28.532329+0000","last_fullsized":"2024-04-25T10:22:28.532329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:05.584260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522603+0000","last_change":"2024-04-25T10:22:27.544246+0000","last_active":"2024-04-25T10:22:28.522603+0000","last_peered":"2024-04-25T10:22:28.522603+0000","last_clean":"2024-04-25T10:22:28.522603+0000","last_became_active":"2024-04-25T10:22:27.537554+0000","last_became_peered":"2024-04-25T10:22:27.537554+0000","last_unstale":"2024-04-25T10:22:28.522603+0000","last_undegraded":"2024-04-25T10:22:28.522603+0000","last_fullsized":"2024-04-25T10:22:28.522603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:11.632829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.518789+0000","last_change":"2024-04-25T10:22:26.516019+0000","last_active":"2024-04-25T10:22:27.518789+0000","last_peered":"2024-04-25T10:22:27.518789+0000","last_clean":"2024-04-25T10:22:27.518789+0000","last_became_active":"2024-04-25T10:22:26.515308+0000","last_became_peered":"2024-04-25T10:22:26.515308+0000","last_unstale":"2024-04-25T10:22:27.518789+0000","last_undegraded":"2024-04-25T10:22:27.518789+0000","last_fullsized":"2024-04-25T10:22:27.518789+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:11.276185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519908+0000","last_change":"2024-04-25T10:22:27.539298+0000","last_active":"2024-04-25T10:22:28.519908+0000","last_peered":"2024-04-25T10:22:28.519908+0000","last_clean":"2024-04-25T10:22:28.519908+0000","last_became_active":"2024-04-25T10:22:27.538523+0000","last_became_peered":"2024-04-25T10:22:27.538523+0000","last_unstale":"2024-04-25T10:22:28.519908+0000","last_undegraded":"2024-04-25T10:22:28.519908+0000","last_fullsized":"2024-04-25T10:22:28.519908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:18.232820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.520984+0000","last_change":"2024-04-25T10:22:26.511178+0000","last_active":"2024-04-25T10:22:27.520984+0000","last_peered":"2024-04-25T10:22:27.520984+0000","last_clean":"2024-04-25T10:22:27.520984+0000","last_became_active":"2024-04-25T10:22:26.510867+0000","last_became_peered":"2024-04-25T10:22:26.510867+0000","last_unstale":"2024-04-25T10:22:27.520984+0000","last_undegraded":"2024-04-25T10:22:27.520984+0000","last_fullsized":"2024-04-25T10:22:27.520984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:36.386576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522124+0000","last_change":"2024-04-25T10:22:27.526582+0000","last_active":"2024-04-25T10:22:28.522124+0000","last_peered":"2024-04-25T10:22:28.522124+0000","last_clean":"2024-04-25T10:22:28.522124+0000","last_became_active":"2024-04-25T10:22:27.526260+0000","last_became_peered":"2024-04-25T10:22:27.526260+0000","last_unstale":"2024-04-25T10:22:28.522124+0000","last_undegraded":"2024-04-25T10:22:28.522124+0000","last_fullsized":"2024-04-25T10:22:28.522124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:52.310123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:22:27.519321+0000","last_change":"2024-04-25T10:22:26.514828+0000","last_active":"2024-04-25T10:22:27.519321+0000","last_peered":"2024-04-25T10:22:27.519321+0000","last_clean":"2024-04-25T10:22:27.519321+0000","last_became_active":"2024-04-25T10:22:26.514510+0000","last_became_peered":"2024-04-25T10:22:26.514510+0000","last_unstale":"2024-04-25T10:22:27.519321+0000","last_undegraded":"2024-04-25T10:22:27.519321+0000","last_fullsized":"2024-04-25T10:22:27.519321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:26.590540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.530915+0000","last_change":"2024-04-25T10:22:27.539492+0000","last_active":"2024-04-25T10:22:28.530915+0000","last_peered":"2024-04-25T10:22:28.530915+0000","last_clean":"2024-04-25T10:22:28.530915+0000","last_became_active":"2024-04-25T10:22:27.539141+0000","last_became_peered":"2024-04-25T10:22:27.539141+0000","last_unstale":"2024-04-25T10:22:28.530915+0000","last_undegraded":"2024-04-25T10:22:28.530915+0000","last_fullsized":"2024-04-25T10:22:28.530915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:03.004996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522127+0000","last_change":"2024-04-25T10:22:26.520118+0000","last_active":"2024-04-25T10:22:28.522127+0000","last_peered":"2024-04-25T10:22:28.522127+0000","last_clean":"2024-04-25T10:22:28.522127+0000","last_became_active":"2024-04-25T10:22:26.519549+0000","last_became_peered":"2024-04-25T10:22:26.519549+0000","last_unstale":"2024-04-25T10:22:28.522127+0000","last_undegraded":"2024-04-25T10:22:28.522127+0000","last_fullsized":"2024-04-25T10:22:28.522127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:58.079090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.728041+0000","last_change":"2024-04-25T10:22:27.536986+0000","last_active":"2024-04-25T10:22:28.728041+0000","last_peered":"2024-04-25T10:22:28.728041+0000","last_clean":"2024-04-25T10:22:28.728041+0000","last_became_active":"2024-04-25T10:22:27.536592+0000","last_became_peered":"2024-04-25T10:22:27.536592+0000","last_unstale":"2024-04-25T10:22:28.728041+0000","last_undegraded":"2024-04-25T10:22:28.728041+0000","last_fullsized":"2024-04-25T10:22:28.728041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:23.543952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522656+0000","last_change":"2024-04-25T10:22:26.515436+0000","last_active":"2024-04-25T10:22:28.522656+0000","last_peered":"2024-04-25T10:22:28.522656+0000","last_clean":"2024-04-25T10:22:28.522656+0000","last_became_active":"2024-04-25T10:22:26.515138+0000","last_became_peered":"2024-04-25T10:22:26.515138+0000","last_unstale":"2024-04-25T10:22:28.522656+0000","last_undegraded":"2024-04-25T10:22:28.522656+0000","last_fullsized":"2024-04-25T10:22:28.522656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:36.868500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522344+0000","last_change":"2024-04-25T10:22:27.536479+0000","last_active":"2024-04-25T10:22:28.522344+0000","last_peered":"2024-04-25T10:22:28.522344+0000","last_clean":"2024-04-25T10:22:28.522344+0000","last_became_active":"2024-04-25T10:22:27.534838+0000","last_became_peered":"2024-04-25T10:22:27.534838+0000","last_unstale":"2024-04-25T10:22:28.522344+0000","last_undegraded":"2024-04-25T10:22:28.522344+0000","last_fullsized":"2024-04-25T10:22:28.522344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:23.543952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.727800+0000","last_change":"2024-04-25T10:22:26.517623+0000","last_active":"2024-04-25T10:22:28.727800+0000","last_peered":"2024-04-25T10:22:28.727800+0000","last_clean":"2024-04-25T10:22:28.727800+0000","last_became_active":"2024-04-25T10:22:26.517337+0000","last_became_peered":"2024-04-25T10:22:26.517337+0000","last_unstale":"2024-04-25T10:22:28.727800+0000","last_undegraded":"2024-04-25T10:22:28.727800+0000","last_fullsized":"2024-04-25T10:22:28.727800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:23.909402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.522004+0000","last_change":"2024-04-25T10:22:27.521993+0000","last_active":"2024-04-25T10:22:28.522004+0000","last_peered":"2024-04-25T10:22:28.522004+0000","last_clean":"2024-04-25T10:22:28.522004+0000","last_became_active":"2024-04-25T10:22:27.521701+0000","last_became_peered":"2024-04-25T10:22:27.521701+0000","last_unstale":"2024-04-25T10:22:28.522004+0000","last_undegraded":"2024-04-25T10:22:28.522004+0000","last_fullsized":"2024-04-25T10:22:28.522004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:26.491537+0000","last_clean_scrub_stamp":"2024-04-25T10:22:26.491537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:17.289338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519659+0000","last_change":"2024-04-25T10:22:26.511919+0000","last_active":"2024-04-25T10:22:28.519659+0000","last_peered":"2024-04-25T10:22:28.519659+0000","last_clean":"2024-04-25T10:22:28.519659+0000","last_became_active":"2024-04-25T10:22:26.511659+0000","last_became_peered":"2024-04-25T10:22:26.511659+0000","last_unstale":"2024-04-25T10:22:28.519659+0000","last_undegraded":"2024-04-25T10:22:28.519659+0000","last_fullsized":"2024-04-25T10:22:28.519659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:22:25.483139+0000","last_clean_scrub_stamp":"2024-04-25T10:22:25.483139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:33.668574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:22:28.519332+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:22:28.519332+0000","last_peered":"2024-04-25T10:22:28.519332+0000","last_clean":"2024-04-25T10:22:28.519332+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:22:28.519332+0000","last_undegraded":"2024-04-25T10:22:28.519332+0000","last_fullsized":"2024-04-25T10:22:28.519332+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509499,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":929792,"data_stored":391062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509500,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":391096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40988,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330852,"statfs":{"total":96636764160,"available":96594792448,"internally_reserved":0,"allocated":1515520,"data_stored":979908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44568,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327272,"statfs":{"total":96636764160,"available":96591126528,"internally_reserved":0,"allocated":929792,"data_stored":389576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":390032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40976,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330864,"statfs":{"total":96636764160,"available":96594804736,"internally_reserved":0,"allocated":1511424,"data_stored":979818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509500,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":921600,"data_stored":389520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62616,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309224,"statfs":{"total":96636764160,"available":96572645376,"internally_reserved":0,"allocated":929792,"data_stored":390144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:32.034 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:22:32.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:22:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 Processor -- start 2024-04-25T10:22:32.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 -- start start 2024-04-25T10:22:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73c0161300 0x7f73c00d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c00d0c60 0x7f73c006b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c00b10b0 con 0x7f73c006b8b0 2024-04-25T10:22:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c00b1390 con 0x7f73c0161300 2024-04-25T10:22:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c8800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c00b1670 con 0x7f73c00d0c60 2024-04-25T10:22:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73c0161300 0x7f73c00d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c6800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f73c006b8b0 0x7f73c0161a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59144/0 (socket says 172.21.15.45:59144) 2024-04-25T10:22:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c6800640 1 -- 172.21.15.45:0/3319787579 learned_addr learned my addr 172.21.15.45:0/3319787579 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c5400640 1 --2- 172.21.15.45:0/3319787579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c00d0c60 0x7f73c006b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c5e00640 1 -- 172.21.15.45:0/3319787579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c00d0c60 msgr2=0x7f73c006b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c5e00640 1 --2- 172.21.15.45:0/3319787579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c00d0c60 0x7f73c006b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c5e00640 1 -- 172.21.15.45:0/3319787579 >> v1:172.21.15.45:6789/0 conn(0x7f73c006b8b0 legacy=0x7f73c0161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c4a00640 1 -- 172.21.15.45:0/3319787579 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22333816 0 0) 0x7f73c00b10b0 con 0x7f73c006b8b0 2024-04-25T10:22:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.218+0000 7f73c5e00640 1 -- 172.21.15.45:0/3319787579 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c00b1950 con 0x7f73c0161300 2024-04-25T10:22:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c5e00640 1 --2- 172.21.15.45:0/3319787579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c0161300 0x7f73c00d0720 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f73b40097c0 tx=0x7f73b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b478575bbf9bccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c4a00640 1 -- 172.21.15.45:0/3319787579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40b0070 con 0x7f73c0161300 2024-04-25T10:22:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c4a00640 1 -- 172.21.15.45:0/3319787579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b40ab600 con 0x7f73c0161300 2024-04-25T10:22:32.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c4a00640 1 -- 172.21.15.45:0/3319787579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40ab9d0 con 0x7f73c0161300 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 -- 172.21.15.45:0/3319787579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c0161300 msgr2=0x7f73c00d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 --2- 172.21.15.45:0/3319787579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c0161300 0x7f73c00d0720 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f73b40097c0 tx=0x7f73b401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 -- 172.21.15.45:0/3319787579 shutdown_connections 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 --2- 172.21.15.45:0/3319787579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c00d0c60 0x7f73c006b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 --2- 172.21.15.45:0/3319787579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c0161300 0x7f73c00d0720 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 -- 172.21.15.45:0/3319787579 >> 172.21.15.45:0/3319787579 conn(0x7f73c00d19d0 msgr2=0x7f73c00d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 -- 172.21.15.45:0/3319787579 shutdown_connections 2024-04-25T10:22:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 -- 172.21.15.45:0/3319787579 wait complete. 2024-04-25T10:22:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 Processor -- start 2024-04-25T10:22:32.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.222+0000 7f73c8800640 1 -- start start 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 0x7f73c0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c00d0c60 0x7f73c014b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c0161300 0x7f73c00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c016b160 con 0x7f73c006b8b0 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c016afe0 con 0x7f73c00d0c60 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c016b2e0 con 0x7f73c0161300 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 0x7f73c0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 0x7f73c0159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46278/0 (socket says 172.21.15.45:46278) 2024-04-25T10:22:32.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 -- 172.21.15.45:0/391057755 learned_addr learned my addr 172.21.15.45:0/391057755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5400640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c00d0c60 0x7f73c014b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c6800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c0161300 0x7f73c00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 -- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c0161300 msgr2=0x7f73c00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c0161300 0x7f73c00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 -- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c00d0c60 msgr2=0x7f73c014b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c00d0c60 0x7f73c014b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 -- 172.21.15.45:0/391057755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c014c980 con 0x7f73c006b8b0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c6800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c0161300 0x7f73c00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c5e00640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 0x7f73c0159d20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f73b40190b0 tx=0x7f73b4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40b9020 con 0x7f73c006b8b0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b40b5680 con 0x7f73c006b8b0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40b0040 con 0x7f73c006b8b0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c014cc60 con 0x7f73c006b8b0 2024-04-25T10:22:32.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c00c8aa0 con 0x7f73c006b8b0 2024-04-25T10:22:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c004ef30 con 0x7f73c006b8b0 2024-04-25T10:22:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.226+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73b40bc020 con 0x7f73c006b8b0 2024-04-25T10:22:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.230+0000 7f73ba000640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73a4066f60 0x7f73a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.230+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73b411ecd0 con 0x7f73c006b8b0 2024-04-25T10:22:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.230+0000 7f73c5400640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73a4066f60 0x7f73a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.230+0000 7f73c5400640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73a4066f60 0x7f73a4069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f73c00b48d0 tx=0x7f73b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.234+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73b4016090 con 0x7f73c006b8b0 2024-04-25T10:22:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.454+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f73c014ce30 con 0x7f73c006b8b0 2024-04-25T10:22:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73ba000640 1 -- 172.21.15.45:0/391057755 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f73b40efcd0 con 0x7f73c006b8b0 2024-04-25T10:22:32.461 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:22:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:22:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:22:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:22:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:22:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73a4066f60 msgr2=0x7f73a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73a4066f60 0x7f73a4069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f73c00b48d0 tx=0x7f73b0008040 comp rx=0 tx=0).stop 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 msgr2=0x7f73c0159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 0x7f73c0159d20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f73b40190b0 tx=0x7f73b4019240 comp rx=0 tx=0).stop 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 shutdown_connections 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f73a4066f60 0x7f73a4069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73c0161300 0x7f73c00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73c00d0c60 0x7f73c014b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 --2- 172.21.15.45:0/391057755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73c006b8b0 0x7f73c0159d20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 >> 172.21.15.45:0/391057755 conn(0x7f73c00d19d0 msgr2=0x7f73c015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 shutdown_connections 2024-04-25T10:22:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.458+0000 7f73c8800640 1 -- 172.21.15.45:0/391057755 wait complete. 2024-04-25T10:22:32.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:32.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.618+0000 7f96a0400640 1 Processor -- start 2024-04-25T10:22:32.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.618+0000 7f96a0400640 1 -- start start 2024-04-25T10:22:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f96a0400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f96981550b0 0x7f96980cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f96a0400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96980cfe70 0x7f969815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f96a0400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f969815b740 0x7f969806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f96a0400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9698071050 con 0x7f969815b740 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f96a0400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980712f0 con 0x7f96981550b0 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f96a0400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980715d0 con 0x7f96980cfe70 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f969815b740 0x7f969806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f969815b740 0x7f969806d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46288/0 (socket says 172.21.15.45:46288) 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 -- 172.21.15.45:0/457052518 learned_addr learned my addr 172.21.15.45:0/457052518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969d000640 1 --2- 172.21.15.45:0/457052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96980cfe70 0x7f969815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969da00640 1 --2- 172.21.15.45:0/457052518 >> v2:172.21.15.117:3300/0 conn(0x7f96981550b0 0x7f96980cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 -- 172.21.15.45:0/457052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96980cfe70 msgr2=0x7f969815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 --2- 172.21.15.45:0/457052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96980cfe70 0x7f969815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 -- 172.21.15.45:0/457052518 >> v2:172.21.15.117:3300/0 conn(0x7f96981550b0 msgr2=0x7f96980cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 --2- 172.21.15.45:0/457052518 >> v2:172.21.15.117:3300/0 conn(0x7f96981550b0 0x7f96980cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.622+0000 7f969e400640 1 -- 172.21.15.45:0/457052518 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980718b0 con 0x7f969815b740 2024-04-25T10:22:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f969e400640 1 --2- 172.21.15.45:0/457052518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969815b740 0x7f969806d1a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9698067980 tx=0x7f969001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5747fd1b155cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f9697e00640 1 -- 172.21.15.45:0/457052518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900ac020 con 0x7f969815b740 2024-04-25T10:22:32.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f9697e00640 1 -- 172.21.15.45:0/457052518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96900a6440 con 0x7f969815b740 2024-04-25T10:22:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f9697e00640 1 -- 172.21.15.45:0/457052518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900a6810 con 0x7f969815b740 2024-04-25T10:22:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 -- 172.21.15.45:0/457052518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969815b740 msgr2=0x7f969806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 --2- 172.21.15.45:0/457052518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969815b740 0x7f969806d1a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9698067980 tx=0x7f969001c730 comp rx=0 tx=0).stop 2024-04-25T10:22:32.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 -- 172.21.15.45:0/457052518 shutdown_connections 2024-04-25T10:22:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 --2- 172.21.15.45:0/457052518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f969815b740 0x7f969806d1a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 --2- 172.21.15.45:0/457052518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96980cfe70 0x7f969815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 --2- 172.21.15.45:0/457052518 >> v2:172.21.15.117:3300/0 conn(0x7f96981550b0 0x7f96980cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 -- 172.21.15.45:0/457052518 >> 172.21.15.45:0/457052518 conn(0x7f9698066b70 msgr2=0x7f9698066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 -- 172.21.15.45:0/457052518 shutdown_connections 2024-04-25T10:22:32.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 -- 172.21.15.45:0/457052518 wait complete. 2024-04-25T10:22:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 Processor -- start 2024-04-25T10:22:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.626+0000 7f96a0400640 1 -- start start 2024-04-25T10:22:32.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 0x7f969815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96981550b0 0x7f9698156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969815b740 0x7f969814a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b33e0 con 0x7f96980cfe70 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b3260 con 0x7f96981550b0 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b3560 con 0x7f969815b740 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 0x7f969815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 0x7f969815c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46304/0 (socket says 172.21.15.45:46304) 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 -- 172.21.15.45:0/1266605821 learned_addr learned my addr 172.21.15.45:0/1266605821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969d000640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96981550b0 0x7f9698156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969e400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969815b740 0x7f969814a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 -- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969815b740 msgr2=0x7f969814a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969815b740 0x7f969814a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 -- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96981550b0 msgr2=0x7f9698156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96981550b0 0x7f9698156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 -- 172.21.15.45:0/1266605821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96981579d0 con 0x7f96980cfe70 2024-04-25T10:22:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f969da00640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 0x7f969815c270 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f968c00d800 tx=0x7f968c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00b660 con 0x7f96980cfe70 2024-04-25T10:22:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c019070 con 0x7f96980cfe70 2024-04-25T10:22:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9698157cb0 con 0x7f96980cfe70 2024-04-25T10:22:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00ba00 con 0x7f96980cfe70 2024-04-25T10:22:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96981506a0 con 0x7f96980cfe70 2024-04-25T10:22:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969804ef30 con 0x7f96980cfe70 2024-04-25T10:22:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.630+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f968c00bba0 con 0x7f96980cfe70 2024-04-25T10:22:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.634+0000 7f9695600640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9678066f90 0x7f9678069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.634+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f968c085330 con 0x7f96980cfe70 2024-04-25T10:22:32.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.634+0000 7f969d000640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9678066f90 0x7f9678069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.634+0000 7f969d000640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9678066f90 0x7f9678069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96980cfdc0 tx=0x7f9684008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:32.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.638+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f968c0563d0 con 0x7f96980cfe70 2024-04-25T10:22:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.834+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f969815b4a0 con 0x7f96980cfe70 2024-04-25T10:22:32.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.834+0000 7f9695600640 1 -- 172.21.15.45:0/1266605821 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f968c05c0d0 con 0x7f96980cfe70 2024-04-25T10:22:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:22:32.839 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:22:26.898819+0000","modified":"2024-04-25T10:22:29.928612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3661919331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3661919331},{"type":"v1","addr":"172.21.15.45:6835","nonce":3661919331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:22:32.839 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 55 2024-04-25T10:22:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9678066f90 msgr2=0x7f9678069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9678066f90 0x7f9678069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96980cfdc0 tx=0x7f9684008040 comp rx=0 tx=0).stop 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 msgr2=0x7f969815c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 0x7f969815c270 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f968c00d800 tx=0x7f968c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 shutdown_connections 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9678066f90 0x7f9678069450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969815b740 0x7f969814a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96981550b0 0x7f9698156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 --2- 172.21.15.45:0/1266605821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96980cfe70 0x7f969815c270 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 >> 172.21.15.45:0/1266605821 conn(0x7f9698066b70 msgr2=0x7f96980afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 shutdown_connections 2024-04-25T10:22:32.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:32.838+0000 7f96a0400640 1 -- 172.21.15.45:0/1266605821 wait complete. 2024-04-25T10:22:33.262 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:22:26.898819+0000', 'modified': '2024-04-25T10:22:29.928612+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6888': {'gid': 6888, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/3661919331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 3661919331}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3661919331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:22:33.262 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:22:33.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:22:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.402+0000 7ffa32000640 1 Processor -- start 2024-04-25T10:22:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.402+0000 7ffa32000640 1 -- start start 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa32000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa2c15b750 0x7ffa2c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa32000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0cd590 0x7ffa2c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa32000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa2c0b3f80 0x7ffa2c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa32000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c0b2d70 con 0x7ffa2c15b750 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa32000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c0b2460 con 0x7ffa2c0b3f80 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa32000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c070fb0 con 0x7ffa2c0cd590 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa2c15b750 0x7ffa2c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa2c15b750 0x7ffa2c0cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46308/0 (socket says 172.21.15.45:46308) 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 -- 172.21.15.45:0/1718110902 learned_addr learned my addr 172.21.15.45:0/1718110902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa31600640 1 --2- 172.21.15.45:0/1718110902 >> v2:172.21.15.117:3300/0 conn(0x7ffa2c0b3f80 0x7ffa2c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa2be00640 1 --2- 172.21.15.45:0/1718110902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0cd590 0x7ffa2c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 -- 172.21.15.45:0/1718110902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0cd590 msgr2=0x7ffa2c0cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 --2- 172.21.15.45:0/1718110902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0cd590 0x7ffa2c0cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 -- 172.21.15.45:0/1718110902 >> v2:172.21.15.117:3300/0 conn(0x7ffa2c0b3f80 msgr2=0x7ffa2c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 --2- 172.21.15.45:0/1718110902 >> v2:172.21.15.117:3300/0 conn(0x7ffa2c0b3f80 0x7ffa2c0b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa30c00640 1 -- 172.21.15.45:0/1718110902 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0b2720 con 0x7ffa2c15b750 2024-04-25T10:22:33.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.406+0000 7ffa31600640 1 --2- 172.21.15.45:0/1718110902 >> v2:172.21.15.117:3300/0 conn(0x7ffa2c0b3f80 0x7ffa2c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa30c00640 1 --2- 172.21.15.45:0/1718110902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0cbc30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0ae7e0 tx=0x7ffa2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8d67ac9aedf170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa2b400640 1 -- 172.21.15.45:0/1718110902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa200b2070 con 0x7ffa2c15b750 2024-04-25T10:22:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa2b400640 1 -- 172.21.15.45:0/1718110902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa200ad640 con 0x7ffa2c15b750 2024-04-25T10:22:33.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa2b400640 1 -- 172.21.15.45:0/1718110902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa200ada10 con 0x7ffa2c15b750 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 -- 172.21.15.45:0/1718110902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 msgr2=0x7ffa2c0cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 --2- 172.21.15.45:0/1718110902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0cbc30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0ae7e0 tx=0x7ffa2001c520 comp rx=0 tx=0).stop 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 -- 172.21.15.45:0/1718110902 shutdown_connections 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 --2- 172.21.15.45:0/1718110902 >> v2:172.21.15.117:3300/0 conn(0x7ffa2c0b3f80 0x7ffa2c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 --2- 172.21.15.45:0/1718110902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0cd590 0x7ffa2c0cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 --2- 172.21.15.45:0/1718110902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0cbc30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 -- 172.21.15.45:0/1718110902 >> 172.21.15.45:0/1718110902 conn(0x7ffa2c0743f0 msgr2=0x7ffa2c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 -- 172.21.15.45:0/1718110902 shutdown_connections 2024-04-25T10:22:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 -- 172.21.15.45:0/1718110902 wait complete. 2024-04-25T10:22:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 Processor -- start 2024-04-25T10:22:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.410+0000 7ffa32000640 1 -- start start 2024-04-25T10:22:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0b3f80 0x7ffa2c159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c0cd590 0x7ffa2c0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c070ce0 con 0x7ffa2c15b750 2024-04-25T10:22:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c070b60 con 0x7ffa2c0cd590 2024-04-25T10:22:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c15f3c0 con 0x7ffa2c0b3f80 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46310/0 (socket says 172.21.15.45:46310) 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 -- 172.21.15.45:0/2922404449 learned_addr learned my addr 172.21.15.45:0/2922404449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa2be00640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c0cd590 0x7ffa2c0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa30c00640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0b3f80 0x7ffa2c159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 -- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0b3f80 msgr2=0x7ffa2c159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0b3f80 0x7ffa2c159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 -- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c0cd590 msgr2=0x7ffa2c0aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c0cd590 0x7ffa2c0aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 -- 172.21.15.45:0/2922404449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0ab040 con 0x7ffa2c15b750 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa31600640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0aa9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffa24007e10 tx=0x7ffa2400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa28c00640 1 -- 172.21.15.45:0/2922404449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2401a070 con 0x7ffa2c15b750 2024-04-25T10:22:33.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa28c00640 1 -- 172.21.15.45:0/2922404449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24003ba0 con 0x7ffa2c15b750 2024-04-25T10:22:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa28c00640 1 -- 172.21.15.45:0/2922404449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24004030 con 0x7ffa2c15b750 2024-04-25T10:22:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c0ab320 con 0x7ffa2c15b750 2024-04-25T10:22:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c0ab8e0 con 0x7ffa2c15b750 2024-04-25T10:22:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.414+0000 7ffa28c00640 1 -- 172.21.15.45:0/2922404449 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa2401d070 con 0x7ffa2c15b750 2024-04-25T10:22:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa28c00640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffa0c066f90 0x7ffa0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa28c00640 1 -- 172.21.15.45:0/2922404449 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa24085290 con 0x7ffa2c15b750 2024-04-25T10:22:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa30c00640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffa0c066f90 0x7ffa0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 Processor -- start 2024-04-25T10:22:33.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- start start 2024-04-25T10:22:33.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa30c00640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffa0c066f90 0x7ffa0c069450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa200af000 tx=0x7ffa20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e059310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3df405f0 con 0x559b3e058f30 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3dea7da0 con 0x559b3e0a3240 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b3de30150 con 0x559b3e0a2920 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa30c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e059310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa30c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:49382/0 (socket says 172.21.15.45:49382) 2024-04-25T10:22:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa30c00640 1 -- 172.21.15.45:0/690159166 learned_addr learned my addr 172.21.15.45:0/690159166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 -- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 msgr2=0x559b3e0a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 -- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 msgr2=0x559b3e0589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 -- 172.21.15.45:0/690159166 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3de49ca0 con 0x559b3e058f30 2024-04-25T10:22:33.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa2be00640 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa31600640 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e059310 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ffa24083720 tx=0x7ffa24055aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3322663f6244c49b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa1d600640 1 -- 172.21.15.45:0/690159166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa240850c0 con 0x559b3e058f30 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa1d600640 1 -- 172.21.15.45:0/690159166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24053eb0 con 0x559b3e058f30 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa1d600640 1 -- 172.21.15.45:0/690159166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2405dd80 con 0x559b3e058f30 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- 172.21.15.45:0/690159166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 msgr2=0x559b3e059310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e059310 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ffa24083720 tx=0x7ffa24055aa0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- 172.21.15.45:0/690159166 shutdown_connections 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e059310 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/690159166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- 172.21.15.45:0/690159166 >> 172.21.15.45:0/690159166 conn(0x559b3e05f2a0 msgr2=0x559b3e0572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- 172.21.15.45:0/690159166 shutdown_connections 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.418+0000 7ffa36b3c000 1 -- 172.21.15.45:0/690159166 wait complete. 2024-04-25T10:22:33.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- start start 2024-04-25T10:22:33.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 Processor -- start 2024-04-25T10:22:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e0aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e05caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3de62990 con 0x559b3e058f30 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3dea3e40 con 0x559b3e0a3240 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b3de33900 con 0x559b3e0a2920 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 10 client.0 _ll_register_callbacks cb 0x559b3df5b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e0aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e0aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46316/0 (socket says 172.21.15.45:46316) 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 -- 172.21.15.45:0/3350935592 learned_addr learned my addr 172.21.15.45:0/3350935592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa2be00640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa31600640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e05caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 -- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 msgr2=0x559b3e0aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 -- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 msgr2=0x559b3e05caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e05caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 -- 172.21.15.45:0/3350935592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b3de60630 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa30c00640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e0aa0e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ffa2001abd0 tx=0x7ffa2001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fd2771ce5f88ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa12000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa12000640 1 -- 172.21.15.45:0/3350935592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa200b2040 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa12000640 1 -- 172.21.15.45:0/3350935592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa200b7b30 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa24053eb0 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa12000640 1 -- 172.21.15.45:0/3350935592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa200b7d30 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 10 client.7020 fetch_fsmap learned FSMap version 55 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x559b3de303a0 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa12000640 1 -- 172.21.15.45:0/3350935592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24053eb0 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa12000640 1 -- 172.21.15.45:0/3350935592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffa200ce450 con 0x559b3e058f30 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 10 client.7020 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 10 client.7020 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 20 client.7020 populate_metadata read hostname 'smithi045' 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e05e160 0x559b3e0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 4 client.7020 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b3de43a50 con 0x559b3e05e160 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.422+0000 7ffa2be00640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e05e160 0x559b3e0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa2be00640 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e05e160 0x559b3e0b4140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c79a0f4d8884175 server_cookie=1482d0535e6b4954 in_seq=0 out_seq=0 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa12000640 10 client.7020 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa12000640 1 -- 172.21.15.45:0/3350935592 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b3de43a50 con 0x559b3e05e160 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa12000640 10 client.7020 handle_command_reply: tid=0 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 client.7020 shutdown 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 msgr2=0x559b3e0aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e0aa0e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ffa2001abd0 tx=0x7ffa2001ad60 comp rx=0 tx=0).stop 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 shutdown_connections 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e05e160 0x559b3e0b4140 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e05caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a2920 0x559b3e0aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/3350935592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e058f30 0x559b3e0aa0e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa12000640 0 client.7020 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:22:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 >> 172.21.15.45:0/3350935592 conn(0x559b3e05f2a0 msgr2=0x559b3e0579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 shutdown_connections 2024-04-25T10:22:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 1 -- 172.21.15.45:0/3350935592 wait complete. 2024-04-25T10:22:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.426+0000 7ffa36b3c000 20 client.7020 trim_cache size 0 max 16384 2024-04-25T10:22:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 Processor -- start 2024-04-25T10:22:33.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 -- start start 2024-04-25T10:22:33.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e058f30 0x559b3e0fdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a3240 0x559b3e0fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0ad070 con 0x559b3e0a2920 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c06ce30 con 0x559b3e058f30 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0aca70 con 0x559b3e0a3240 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46322/0 (socket says 172.21.15.45:46322) 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 -- 172.21.15.45:0/245527118 learned_addr learned my addr 172.21.15.45:0/245527118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa30c00640 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e058f30 0x559b3e0fdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa31600640 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a3240 0x559b3e0fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 -- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a3240 msgr2=0x559b3e0fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a3240 0x559b3e0fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 -- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e058f30 msgr2=0x559b3e0fdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e058f30 0x559b3e0fdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 -- 172.21.15.45:0/245527118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b3e0a3070 con 0x559b3e0a2920 2024-04-25T10:22:33.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa2be00640 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0fe150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c06dec0 tx=0x7ffa18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0a6f8106139ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa1cc00640 1 -- 172.21.15.45:0/245527118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18017190 con 0x559b3e0a2920 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa1cc00640 1 -- 172.21.15.45:0/245527118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18021030 con 0x559b3e0a2920 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa1cc00640 1 -- 172.21.15.45:0/245527118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18017b30 con 0x559b3e0a2920 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 -- 172.21.15.45:0/245527118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 msgr2=0x559b3e0fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.442+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0fe150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c06dec0 tx=0x7ffa18008040 comp rx=0 tx=0).stop 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- 172.21.15.45:0/245527118 shutdown_connections 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e0a3240 0x559b3e0fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0fe150 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/245527118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e058f30 0x559b3e0fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- 172.21.15.45:0/245527118 >> 172.21.15.45:0/245527118 conn(0x559b3e05f2a0 msgr2=0x559b3e05b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- 172.21.15.45:0/245527118 shutdown_connections 2024-04-25T10:22:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- 172.21.15.45:0/245527118 wait complete. 2024-04-25T10:22:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- start start 2024-04-25T10:22:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 Processor -- start 2024-04-25T10:22:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 0x559b3e0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3deac3d0 con 0x559b3e0a2920 2024-04-25T10:22:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b3df63d50 con 0x559b3e0a3240 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b3de42980 con 0x559b3e058f30 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 10 client.0 _ll_register_callbacks cb 0x7ffa200b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa30c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 0x559b3e0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46332/0 (socket says 172.21.15.45:46332) 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 -- 172.21.15.45:0/2895672583 learned_addr learned my addr 172.21.15.45:0/2895672583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa30c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 0x559b3e0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:49410/0 (socket says 172.21.15.45:49410) 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa31600640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 -- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 msgr2=0x559b3e0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 0x559b3e0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 -- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 msgr2=0x559b3e0e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 -- 172.21.15.45:0/2895672583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b3df29600 con 0x559b3e0a2920 2024-04-25T10:22:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa30c00640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 0x559b3e0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa2be00640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0af050 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ffa18000c10 tx=0x7ffa18018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808273e7e70746c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa12000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa12000640 1 -- 172.21.15.45:0/2895672583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004250 con 0x559b3e0a2920 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa12000640 1 -- 172.21.15.45:0/2895672583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa180043f0 con 0x559b3e0a2920 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa18021030 con 0x559b3e0a2920 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa12000640 1 -- 172.21.15.45:0/2895672583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa18037050 con 0x559b3e0a2920 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 10 client.7026 fetch_fsmap learned FSMap version 55 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x559b3de3c9b0 con 0x559b3e0a2920 2024-04-25T10:22:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa12000640 1 -- 172.21.15.45:0/2895672583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18021030 con 0x559b3e0a2920 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa12000640 1 -- 172.21.15.45:0/2895672583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffa1803a020 con 0x559b3e0a2920 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 10 client.7026 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 10 client.7026 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.446+0000 7ffa36b3c000 20 client.7026 populate_metadata read hostname 'smithi045' 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e0e77b0 0x559b3e0e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 4 client.7026 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559b3deb52e0 con 0x559b3e0e77b0 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa30c00640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e0e77b0 0x559b3e0e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa30c00640 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e0e77b0 0x559b3e0e7b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dedea68edb9ac76f server_cookie=826692272e728593 in_seq=0 out_seq=0 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa12000640 10 client.7026 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa12000640 1 -- 172.21.15.45:0/2895672583 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559b3deb52e0 con 0x559b3e0e77b0 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa12000640 10 client.7026 handle_command_reply: tid=0 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 client.7026 shutdown 2024-04-25T10:22:33.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 msgr2=0x559b3e0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0af050 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ffa18000c10 tx=0x7ffa18018040 comp rx=0 tx=0).stop 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 shutdown_connections 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x559b3e0e77b0 0x559b3e0e7b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b3e0a3240 0x559b3e0e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b3e0a2920 0x559b3e0af050 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 --2- 172.21.15.45:0/2895672583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559b3e058f30 0x559b3e0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa12000640 0 client.7026 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 >> 172.21.15.45:0/2895672583 conn(0x559b3e05f2a0 msgr2=0x559b3e05bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 shutdown_connections 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 1 -- 172.21.15.45:0/2895672583 wait complete. 2024-04-25T10:22:33.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa36b3c000 20 client.7026 trim_cache size 0 max 16384 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6888, 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9357449820000001, 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 55, 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 77, 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:22:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.833839526 2024-04-25T10:22:33.460 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:22:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffa0c066f90 msgr2=0x7ffa0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffa0c066f90 0x7ffa0c069450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa200af000 tx=0x7ffa20009040 comp rx=0 tx=0).stop 2024-04-25T10:22:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 msgr2=0x7ffa2c0aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0aa9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffa24007e10 tx=0x7ffa2400d4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 shutdown_connections 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffa0c066f90 0x7ffa0c069450 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa2c15b750 0x7ffa2c0aa9c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa2c0cd590 0x7ffa2c0aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 --2- 172.21.15.45:0/2922404449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa2c0b3f80 0x7ffa2c159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 >> 172.21.15.45:0/2922404449 conn(0x7ffa2c0743f0 msgr2=0x7ffa2c154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 shutdown_connections 2024-04-25T10:22:33.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.450+0000 7ffa32000640 1 -- 172.21.15.45:0/2922404449 wait complete. 2024-04-25T10:22:33.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:22:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.610+0000 7fb131400640 1 Processor -- start 2024-04-25T10:22:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.610+0000 7fb131400640 1 -- start start 2024-04-25T10:22:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb131400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb131400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb12c0b2fc0 0x7fb12c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb131400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12c070ed0 con 0x7fb12c15a0e0 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb131400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12c0711b0 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb131400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c071490 con 0x7fb12c071f70 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12a800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb12c15a0e0 0x7fb12c15b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59154/0 (socket says 172.21.15.45:59154) 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12a800640 1 -- 172.21.15.45:0/3179648012 learned_addr learned my addr 172.21.15.45:0/3179648012 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12b200640 1 --2- 172.21.15.45:0/3179648012 >> v2:172.21.15.45:3300/0 conn(0x7fb12c0b2fc0 0x7fb12c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb129e00640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb129400640 1 -- 172.21.15.45:0/3179648012 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143287711 0 0) 0x7fb12c070ed0 con 0x7fb12c15a0e0 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb129400640 1 -- 172.21.15.45:0/3179648012 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb110003450 con 0x7fb12c15a0e0 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12b200640 1 -- 172.21.15.45:0/3179648012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 msgr2=0x7fb12c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12b200640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12b200640 1 -- 172.21.15.45:0/3179648012 >> v1:172.21.15.45:6789/0 conn(0x7fb12c15a0e0 legacy=0x7fb12c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12b200640 1 -- 172.21.15.45:0/3179648012 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c071770 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb129e00640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb12b200640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c06c090 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb1240097c0 tx=0x7fb12401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84df7ef24692dbca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.614+0000 7fb129400640 1 -- 172.21.15.45:0/3179648012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240b0070 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb129400640 1 -- 172.21.15.45:0/3179648012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1240ab600 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb129400640 1 -- 172.21.15.45:0/3179648012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240ab9d0 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- 172.21.15.45:0/3179648012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 msgr2=0x7fb12c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c06c090 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb1240097c0 tx=0x7fb12401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- 172.21.15.45:0/3179648012 shutdown_connections 2024-04-25T10:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c06c090 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 --2- 172.21.15.45:0/3179648012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- 172.21.15.45:0/3179648012 >> 172.21.15.45:0/3179648012 conn(0x7fb12c066b70 msgr2=0x7fb12c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- 172.21.15.45:0/3179648012 shutdown_connections 2024-04-25T10:22:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- 172.21.15.45:0/3179648012 wait complete. 2024-04-25T10:22:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 Processor -- start 2024-04-25T10:22:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- start start 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c14a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c15a0e0 0x7fb12c156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c072350 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c070b50 con 0x7fb12c15a0e0 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb131400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0724d0 con 0x7fb12c071f70 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb129e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb129e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46336/0 (socket says 172.21.15.45:46336) 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb129e00640 1 -- 172.21.15.45:0/2345871329 learned_addr learned my addr 172.21.15.45:0/2345871329 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb12b200640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c15a0e0 0x7fb12c156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.618+0000 7fb12a800640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c14a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb129e00640 1 -- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 msgr2=0x7fb12c14a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb129e00640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c14a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb129e00640 1 -- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c15a0e0 msgr2=0x7fb12c156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb129e00640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c15a0e0 0x7fb12c156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb129e00640 1 -- 172.21.15.45:0/2345871329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c1fa010 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb12b200640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c15a0e0 0x7fb12c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb129e00640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c158da0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e810 tx=0x7fb11800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118017070 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11800c8d0 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11800f040 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c1fad90 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c1fa590 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c04ef30 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb11801a070 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb122a00640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb10c066f90 0x7fb10c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb118084be0 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.622+0000 7fb12a800640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb10c066f90 0x7fb10c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:33.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.626+0000 7fb12a800640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb10c066f90 0x7fb10c069450 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb11c005c10 tx=0x7fb11c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.626+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb118055be0 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.822+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb12c0ae790 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.822+0000 7fb122a00640 1 -- 172.21.15.45:0/2345871329 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1180554e0 con 0x7fb12c0b2fc0 2024-04-25T10:22:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:22:33.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb10c066f90 msgr2=0x7fb10c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb10c066f90 0x7fb10c069450 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb11c005c10 tx=0x7fb11c009040 comp rx=0 tx=0).stop 2024-04-25T10:22:33.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 msgr2=0x7fb12c158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:33.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c158da0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e810 tx=0x7fb11800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 shutdown_connections 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb10c066f90 0x7fb10c069450 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb12c15a0e0 0x7fb12c156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb12c0b2fc0 0x7fb12c158da0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 --2- 172.21.15.45:0/2345871329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12c071f70 0x7fb12c14a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 >> 172.21.15.45:0/2345871329 conn(0x7fb12c066b70 msgr2=0x7fb12c14fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 shutdown_connections 2024-04-25T10:22:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:33.826+0000 7fb131400640 1 -- 172.21.15.45:0/2345871329 wait complete. 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:22:33.849 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:22:33.850 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:22:33.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:33.850 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:22:33.860 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:22:33.861 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:22:33.862 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:22:33.862 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:22:33.862 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:22:33.862 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:22:33.862 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:22:33.862 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42186sec preferred_lft 42186sec 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:22:33.863 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:22:33.864 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:22:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:22:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:22:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.864 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:22:33.864 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:22:33.865 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:22:33.865 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.865 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:22:33.865 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.865 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:22:33.865 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:22:33.866 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:22:33.866 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.866 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:22:33.866 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:22:33.866 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:22:33.866 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:22:33.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:33.867 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:22:33.925 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:22:33.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:33.928 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:22:33.984 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:22:33.985 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:22:33.985 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:22:33.985 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:22:33.985 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:22:33.985 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:22:33.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:33.986 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:22:33.986 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:22:33.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:22:33.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:22:33.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:22:33.986 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:22:34.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:34.082 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:22:34.091 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:22:34.091 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:22:34.091 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:22:34.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:34.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:22:34.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:22:34.151 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:22:34.200 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:22:34.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:22:34.200 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:22:34.255 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:22:34.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:34.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:22:34.345 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:22:34.345 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:22:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:22:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:22:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:22:34.358 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:22:34.358 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:22:34.361 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:22:34.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:34.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:22:34.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:34.383 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:22:34.439 INFO:teuthology.orchestra.run.smithi045.stdout:client6158 2024-04-25T10:22:34.440 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:34.440 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:34.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:22:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.678+0000 7fb129200640 1 Processor -- start 2024-04-25T10:22:34.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb129200640 1 -- start start 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb129200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1240b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb129200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1240cf9e0 0x7fb1240cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb129200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12406c0e0 con 0x7fb12415c0f0 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb129200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240b2ef0 con 0x7fb1240cf9e0 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb129200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240d0390 con 0x7fb12415bd10 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb12415c0f0 0x7fb1240b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59158/0 (socket says 172.21.15.45:59158) 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 -- 172.21.15.45:0/1552306126 learned_addr learned my addr 172.21.15.45:0/1552306126 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb122800640 1 --2- 172.21.15.45:0/1552306126 >> v2:172.21.15.117:3300/0 conn(0x7fb1240cf9e0 0x7fb1240cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121e00640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1240b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413974747 0 0) 0x7fb12406c0e0 con 0x7fb12415c0f0 2024-04-25T10:22:34.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb108003530 con 0x7fb12415c0f0 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572465384 0 0) 0x7fb108003530 con 0x7fb12415c0f0 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 msgr2=0x7fb1240b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1240b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 >> v2:172.21.15.117:3300/0 conn(0x7fb1240cf9e0 msgr2=0x7fb1240cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 --2- 172.21.15.45:0/1552306126 >> v2:172.21.15.117:3300/0 conn(0x7fb1240cf9e0 0x7fb1240cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12415c6a0 con 0x7fb12415c0f0 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fb10c001680 con 0x7fb12415c0f0 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb122800640 1 --2- 172.21.15.45:0/1552306126 >> v2:172.21.15.117:3300/0 conn(0x7fb1240cf9e0 0x7fb1240cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 >> v1:172.21.15.45:6789/0 conn(0x7fb12415c0f0 legacy=0x7fb1240b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1080039c0 0x7fb108003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb108004810 0x7fb108004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b2ef0 con 0x7fb108004810 2024-04-25T10:22:34.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240d0390 con 0x7fb1080039c0 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10800b9d0 con 0x7fb12415bd10 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb108004810 0x7fb108004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb121e00640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1080039c0 0x7fb108003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb122800640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 -- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 msgr2=0x7fb1080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 -- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1080039c0 msgr2=0x7fb108003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1080039c0 0x7fb108003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb11be00640 1 -- 172.21.15.45:0/1552306126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108003530 con 0x7fb108004810 2024-04-25T10:22:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.682+0000 7fb122800640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb12415bd10 0x7fb1080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb11be00640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb108004810 0x7fb108004c90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c2c0 tx=0x7fb10c01f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5275b3dc1110ecc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c007d60 con 0x7fb108004810 2024-04-25T10:22:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c005b00 con 0x7fb108004810 2024-04-25T10:22:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb121400640 1 -- 172.21.15.45:0/1552306126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c007070 con 0x7fb108004810 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb129200640 1 -- 172.21.15.45:0/1552306126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb108004810 msgr2=0x7fb108004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb129200640 1 --2- 172.21.15.45:0/1552306126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb108004810 0x7fb108004c90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c2c0 tx=0x7fb10c01f4b0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb122800640 1 -- 172.21.15.45:0/1552306126 reap_dead start 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb129200640 1 -- 172.21.15.45:0/1552306126 shutdown_connections 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb129200640 1 -- 172.21.15.45:0/1552306126 >> 172.21.15.45:0/1552306126 conn(0x7fb124066b70 msgr2=0x7fb124066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb129200640 1 -- 172.21.15.45:0/1552306126 shutdown_connections 2024-04-25T10:22:34.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.686+0000 7fb129200640 1 -- 172.21.15.45:0/1552306126 wait complete. 2024-04-25T10:22:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 Processor -- start 2024-04-25T10:22:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 -- start start 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb108004810 0x7fb1241421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 0x7fb124153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb124158cc0 0x7fb124159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12415cf40 con 0x7fb124151b80 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12415cc60 con 0x7fb108004810 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b4610 con 0x7fb124158cc0 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 0x7fb124153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 0x7fb124153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46352/0 (socket says 172.21.15.45:46352) 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 -- 172.21.15.45:0/1369312911 learned_addr learned my addr 172.21.15.45:0/1369312911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb122800640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb124158cc0 0x7fb124159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb121e00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb108004810 0x7fb1241421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 -- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb124158cc0 msgr2=0x7fb124159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb124158cc0 0x7fb124159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 -- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb108004810 msgr2=0x7fb1241421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb108004810 0x7fb1241421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 -- 172.21.15.45:0/1369312911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124142730 con 0x7fb124151b80 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb121e00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb108004810 0x7fb1241421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11be00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 0x7fb124153e70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb10c026020 tx=0x7fb108005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c005cb0 con 0x7fb124151b80 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c005e50 con 0x7fb124151b80 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c004a30 con 0x7fb124151b80 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124152640 con 0x7fb124151b80 2024-04-25T10:22:34.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.690+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb124152360 con 0x7fb124151b80 2024-04-25T10:22:34.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.694+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12404ef30 con 0x7fb124151b80 2024-04-25T10:22:34.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.694+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb10c0ba030 con 0x7fb124151b80 2024-04-25T10:22:34.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.694+0000 7fb11a000640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb100066f60 0x7fb100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:34.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.694+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb10c121e50 con 0x7fb124151b80 2024-04-25T10:22:34.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.694+0000 7fb121e00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb100066f60 0x7fb100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:34.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.694+0000 7fb121e00640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb100066f60 0x7fb100069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb114009a80 tx=0x7fb114008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:34.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.698+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb10c0c1020 con 0x7fb124151b80 2024-04-25T10:22:34.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:34.894+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb1240691a0 con 0x7fb124151b80 2024-04-25T10:22:35.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.558+0000 7fb11a000640 1 -- 172.21.15.45:0/1369312911 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v56) v1 ==== 167+0+0 (secure 0 0 0) 0x7fb10c0191c0 con 0x7fb124151b80 2024-04-25T10:22:35.561 INFO:teuthology.orchestra.run.smithi045.stderr:enabled new snapshots 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb100066f60 msgr2=0x7fb100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb100066f60 0x7fb100069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb114009a80 tx=0x7fb114008040 comp rx=0 tx=0).stop 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 msgr2=0x7fb124153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 0x7fb124153e70 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb10c026020 tx=0x7fb108005840 comp rx=0 tx=0).stop 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 shutdown_connections 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb100066f60 0x7fb100069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:35.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb124158cc0 0x7fb124159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:35.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb124151b80 0x7fb124153e70 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:35.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 --2- 172.21.15.45:0/1369312911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb108004810 0x7fb1241421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:35.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 >> 172.21.15.45:0/1369312911 conn(0x7fb124066b70 msgr2=0x7fb1240ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:35.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 shutdown_connections 2024-04-25T10:22:35.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:35.562+0000 7fb129200640 1 -- 172.21.15.45:0/1369312911 wait complete. 2024-04-25T10:22:35.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:35.589 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:22:35.606 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:22:35.606 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:35.659 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:22:35.659 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:22:35.659 DEBUG:teuthology.orchestra.run.smithi045:> import uuid 2024-04-25T10:22:35.659 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> n = 2000 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> create_hard_links = False 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> if 3 > 0: 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> create_hard_links = True 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> dpath = os.path.dirname(path) 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> fnameprefix = os.path.basename(path) 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> for i in range(n): 2024-04-25T10:22:35.660 DEBUG:teuthology.orchestra.run.smithi045:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> f.write(f"{i}") 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> f.flush() 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(f.fileno()) 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> os.unlink(fpath) 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(dirfd) 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> if create_hard_links: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> for j in range(3): 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(dirfd) 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> finally: 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> os.close(dirfd) 2024-04-25T10:22:35.661 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:22:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:22:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:22:57.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:57.368 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:22:57.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:57.435 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:22:57.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:57.916 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:24.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:24.433 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:23:24.455 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:23:24.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:23:27.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:27.979 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff48d000640 1 Processor -- start 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff48d000640 1 -- start start 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff48d000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff48808b260 0x7ff4880cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff48d000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48808bd90 con 0x7ff48816a590 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff48d000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48808b7d0 con 0x7ff48816a970 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff48d000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48816e770 con 0x7ff48808b260 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff487400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff48816a590 0x7ff48816ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37306/0 (socket says 172.21.15.45:37306) 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff487400640 1 -- 172.21.15.45:0/3145327495 learned_addr learned my addr 172.21.15.45:0/3145327495 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff487e00640 1 --2- 172.21.15.45:0/3145327495 >> v2:172.21.15.117:3300/0 conn(0x7ff48808b260 0x7ff4880cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:24.633+0000 7ff486a00640 1 -- 172.21.15.45:0/3145327495 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687622761 0 0) 0x7ff48808bd90 con 0x7ff48816a590 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 -- 172.21.15.45:0/3145327495 >> v2:172.21.15.117:3300/0 conn(0x7ff48808b260 msgr2=0x7ff4880cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 --2- 172.21.15.45:0/3145327495 >> v2:172.21.15.117:3300/0 conn(0x7ff48808b260 0x7ff4880cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 -- 172.21.15.45:0/3145327495 >> v1:172.21.15.117:6789/0 conn(0x7ff48816a970 legacy=0x7ff48806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 -- 172.21.15.45:0/3145327495 >> v1:172.21.15.45:6789/0 conn(0x7ff48816a590 legacy=0x7ff48816ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 --2- 172.21.15.45:0/3145327495 >> v2:172.21.15.45:3300/0 conn(0x7ff4640035f0 0x7ff464003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 --2- 172.21.15.45:0/3145327495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464003fa0 0x7ff46400cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 -- 172.21.15.45:0/3145327495 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48816e770 con 0x7ff4640035f0 2024-04-25T10:23:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff485600640 1 -- 172.21.15.45:0/3145327495 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff464006720 con 0x7ff464003fa0 2024-04-25T10:23:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff487400640 1 --2- 172.21.15.45:0/3145327495 >> v2:172.21.15.45:3300/0 conn(0x7ff4640035f0 0x7ff464003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3145327495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464003fa0 0x7ff46400cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff487400640 1 -- 172.21.15.45:0/3145327495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464003fa0 msgr2=0x7ff46400cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff487400640 1 --2- 172.21.15.45:0/3145327495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff464003fa0 0x7ff46400cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.213+0000 7ff487400640 1 -- 172.21.15.45:0/3145327495 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48808bab0 con 0x7ff4640035f0 2024-04-25T10:23:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.217+0000 7ff487400640 1 --2- 172.21.15.45:0/3145327495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4640035f0 0x7ff464003a60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff478001940 tx=0x7ff47801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0bdb10d6896223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.217+0000 7ff486a00640 1 -- 172.21.15.45:0/3145327495 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780af070 con 0x7ff4640035f0 2024-04-25T10:23:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.217+0000 7ff486a00640 1 -- 172.21.15.45:0/3145327495 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478005ae0 con 0x7ff4640035f0 2024-04-25T10:23:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.217+0000 7ff486a00640 1 -- 172.21.15.45:0/3145327495 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478007070 con 0x7ff4640035f0 2024-04-25T10:23:28.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 -- 172.21.15.45:0/3145327495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4640035f0 msgr2=0x7ff464003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 --2- 172.21.15.45:0/3145327495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4640035f0 0x7ff464003a60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff478001940 tx=0x7ff47801c710 comp rx=0 tx=0).stop 2024-04-25T10:23:28.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff487e00640 1 -- 172.21.15.45:0/3145327495 reap_dead start 2024-04-25T10:23:28.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 -- 172.21.15.45:0/3145327495 shutdown_connections 2024-04-25T10:23:28.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 --2- 172.21.15.45:0/3145327495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4640035f0 0x7ff464003a60 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 -- 172.21.15.45:0/3145327495 >> 172.21.15.45:0/3145327495 conn(0x7ff488066da0 msgr2=0x7ff4880671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 -- 172.21.15.45:0/3145327495 shutdown_connections 2024-04-25T10:23:28.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 -- 172.21.15.45:0/3145327495 wait complete. 2024-04-25T10:23:28.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 Processor -- start 2024-04-25T10:23:28.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.221+0000 7ff48d000640 1 -- start start 2024-04-25T10:23:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4640035f0 0x7ff48815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 0x7ff4880cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff48815e1b0 0x7ff488089c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881786a0 con 0x7ff4880ccf40 2024-04-25T10:23:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880b3320 con 0x7ff4640035f0 2024-04-25T10:23:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488178820 con 0x7ff48815e1b0 2024-04-25T10:23:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 0x7ff4880cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 0x7ff4880cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41042/0 (socket says 172.21.15.45:41042) 2024-04-25T10:23:28.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 -- 172.21.15.45:0/3773474776 learned_addr learned my addr 172.21.15.45:0/3773474776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff487e00640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff48815e1b0 0x7ff488089c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff487400640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4640035f0 0x7ff48815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 -- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff48815e1b0 msgr2=0x7ff488089c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff48815e1b0 0x7ff488089c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 -- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4640035f0 msgr2=0x7ff48815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4640035f0 0x7ff48815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 -- 172.21.15.45:0/3773474776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880cbc00 con 0x7ff4880ccf40 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 0x7ff4880cb6c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff47000f840 tx=0x7ff47000fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47f400640 1 -- 172.21.15.45:0/3773474776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000fe90 con 0x7ff4880ccf40 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47f400640 1 -- 172.21.15.45:0/3773474776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff470018070 con 0x7ff4880ccf40 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff47f400640 1 -- 172.21.15.45:0/3773474776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000d730 con 0x7ff4880ccf40 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4880cbea0 con 0x7ff4880ccf40 2024-04-25T10:23:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.225+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4880cc460 con 0x7ff4880ccf40 2024-04-25T10:23:28.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.229+0000 7ff47f400640 1 -- 172.21.15.45:0/3773474776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff47000d8d0 con 0x7ff4880ccf40 2024-04-25T10:23:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.229+0000 7ff47f400640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4640707f0 0x7ff464072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.229+0000 7ff47f400640 1 -- 172.21.15.45:0/3773474776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff470084ee0 con 0x7ff4880ccf40 2024-04-25T10:23:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.229+0000 7ff491aac000 1 Processor -- start 2024-04-25T10:23:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.229+0000 7ff491aac000 1 -- start start 2024-04-25T10:23:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.229+0000 7ff487400640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4640707f0 0x7ff464072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487400640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4640707f0 0x7ff464072cb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff4780ac000 tx=0x7ff4780a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1aa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599da08d1f0 con 0x5599da1aad90 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d9ff3e90 con 0x5599da1efa40 2024-04-25T10:23:28.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d9f7bf50 con 0x5599da1f0360 2024-04-25T10:23:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1ab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41058/0 (socket says 172.21.15.45:41058) 2024-04-25T10:23:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 -- 172.21.15.45:0/2704903041 learned_addr learned my addr 172.21.15.45:0/2704903041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487400640 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1efe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36146/0 (socket says 172.21.15.45:36146) 2024-04-25T10:23:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff47fe00640 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 -- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 msgr2=0x5599da1aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 -- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 msgr2=0x5599da1efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 -- 172.21.15.45:0/2704903041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d9fcb8b0 con 0x5599da1aad90 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff47fe00640 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff487e00640 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1ab170 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff488071d90 tx=0x7ff480009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba6a74de5cafd37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff477e00640 1 -- 172.21.15.45:0/2704903041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480007510 con 0x5599da1aad90 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff477e00640 1 -- 172.21.15.45:0/2704903041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48001a070 con 0x5599da1aad90 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff477e00640 1 -- 172.21.15.45:0/2704903041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4800078f0 con 0x5599da1aad90 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- 172.21.15.45:0/2704903041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 msgr2=0x5599da1ab170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1ab170 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff488071d90 tx=0x7ff480009c70 comp rx=0 tx=0).stop 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- 172.21.15.45:0/2704903041 shutdown_connections 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1ab170 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 --2- 172.21.15.45:0/2704903041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- 172.21.15.45:0/2704903041 >> 172.21.15.45:0/2704903041 conn(0x5599da1a6d40 msgr2=0x5599da1a9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- 172.21.15.45:0/2704903041 shutdown_connections 2024-04-25T10:23:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- 172.21.15.45:0/2704903041 wait complete. 2024-04-25T10:23:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.233+0000 7ff491aac000 1 -- start start 2024-04-25T10:23:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 Processor -- start 2024-04-25T10:23:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d9fae540 con 0x5599da1aad90 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d9feff30 con 0x5599da1efa40 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d9fd2010 con 0x5599da1f0360 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 10 client.0 _ll_register_callbacks cb 0x5599da0a8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1f71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41066/0 (socket says 172.21.15.45:41066) 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 -- 172.21.15.45:0/3150466621 learned_addr learned my addr 172.21.15.45:0/3150466621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487e00640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 -- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 msgr2=0x5599da1b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 -- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 msgr2=0x5599da1f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 -- 172.21.15.45:0/3150466621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599d9f7c1a0 con 0x5599da1aad90 2024-04-25T10:23:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff487400640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1f71c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff4780011f0 tx=0x7ff47801af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fc00bc873c0375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff474c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff474c00640 1 -- 172.21.15.45:0/3150466621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780c4070 con 0x5599da1aad90 2024-04-25T10:23:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff474c00640 1 -- 172.21.15.45:0/3150466621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff4780c0460 con 0x5599da1aad90 2024-04-25T10:23:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff48001a070 con 0x5599da1aad90 2024-04-25T10:23:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff474c00640 1 -- 172.21.15.45:0/3150466621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4780cf050 con 0x5599da1aad90 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 10 client.7056 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5599da076840 con 0x5599da1aad90 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff474c00640 1 -- 172.21.15.45:0/3150466621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48001a070 con 0x5599da1aad90 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff474c00640 1 -- 172.21.15.45:0/3150466621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff4780d2020 con 0x5599da1aad90 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 10 client.7056 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 10 client.7056 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 20 client.7056 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1b1bf0 0x5599da1b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 4 client.7056 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.237+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599d9f8fbe0 con 0x5599da1b1bf0 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1b1bf0 0x5599da1b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff47fe00640 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1b1bf0 0x5599da1b2000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0075ca6a69ff91 server_cookie=34b8a11b37d05dd4 in_seq=0 out_seq=0 2024-04-25T10:23:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff474c00640 10 client.7056 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff474c00640 1 -- 172.21.15.45:0/3150466621 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599d9f8fbe0 con 0x5599da1b1bf0 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff474c00640 10 client.7056 handle_command_reply: tid=0 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 client.7056 shutdown 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 msgr2=0x5599da1f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1f71c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff4780011f0 tx=0x7ff47801af50 comp rx=0 tx=0).stop 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 shutdown_connections 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1b1bf0 0x5599da1b2000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da1b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff474c00640 0 client.7056 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da1f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 --2- 172.21.15.45:0/3150466621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da1f71c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 >> 172.21.15.45:0/3150466621 conn(0x5599da1a6d40 msgr2=0x5599da1a99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 shutdown_connections 2024-04-25T10:23:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 1 -- 172.21.15.45:0/3150466621 wait complete. 2024-04-25T10:23:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.241+0000 7ff491aac000 20 client.7056 trim_cache size 0 max 16384 2024-04-25T10:23:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 Processor -- start 2024-04-25T10:23:28.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 -- start start 2024-04-25T10:23:28.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da24c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da24c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da24cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48820ff30 con 0x5599da1aad90 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48820f110 con 0x5599da1efa40 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881604f0 con 0x5599da1f0360 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da24c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da24c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41074/0 (socket says 172.21.15.45:41074) 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 -- 172.21.15.45:0/1437416928 learned_addr learned my addr 172.21.15.45:0/1437416928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff47fe00640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da24c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487e00640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da24cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 -- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 msgr2=0x5599da24cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da24cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 -- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 msgr2=0x5599da24c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da24c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 -- 172.21.15.45:0/1437416928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599da04da10 con 0x5599da1aad90 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff47fe00640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da24c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487e00640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da24cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:28.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff487400640 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da24c080 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff47801c9b0 tx=0x7ff4780c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e87e1b31bc7e26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff477400640 1 -- 172.21.15.45:0/1437416928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780c4040 con 0x5599da1aad90 2024-04-25T10:23:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff477400640 1 -- 172.21.15.45:0/1437416928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4780c0890 con 0x5599da1aad90 2024-04-25T10:23:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.257+0000 7ff477400640 1 -- 172.21.15.45:0/1437416928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780b9040 con 0x5599da1aad90 2024-04-25T10:23:28.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- 172.21.15.45:0/1437416928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 msgr2=0x5599da24c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da24c080 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff47801c9b0 tx=0x7ff4780c0000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- 172.21.15.45:0/1437416928 shutdown_connections 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1f0360 0x5599da24cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1efa40 0x5599da24c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- 172.21.15.45:0/1437416928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1aad90 0x5599da24c080 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- 172.21.15.45:0/1437416928 >> 172.21.15.45:0/1437416928 conn(0x5599da1a6d40 msgr2=0x5599da1ee1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- 172.21.15.45:0/1437416928 shutdown_connections 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- 172.21.15.45:0/1437416928 wait complete. 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- start start 2024-04-25T10:23:28.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 Processor -- start 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 0x5599da2502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1efa40 0x5599da2507e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1f0360 0x5599da1f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d9ff2e60 con 0x5599da1f0360 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48816c750 con 0x5599da1aad90 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d9feff30 con 0x5599da1efa40 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff491aac000 10 client.0 _ll_register_callbacks cb 0x5599d9fb6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1f0360 0x5599da1f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 0x5599da2502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1efa40 0x5599da2507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 0x5599da2502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36168/0 (socket says 172.21.15.45:36168) 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 -- 172.21.15.45:0/4036426031 learned_addr learned my addr 172.21.15.45:0/4036426031 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 -- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1efa40 msgr2=0x5599da2507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1efa40 0x5599da2507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 -- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1f0360 msgr2=0x5599da1f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1f0360 0x5599da1f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487400640 1 -- 172.21.15.45:0/4036426031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599d9f88b00 con 0x5599da1aad90 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.261+0000 7ff487e00640 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1f0360 0x5599da1f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff487400640 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 0x5599da2502a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff478001940 tx=0x7ff478007f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de55d22caf5f3c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 1 -- 172.21.15.45:0/4036426031 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780b9070 con 0x5599da1aad90 2024-04-25T10:23:28.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 1 -- 172.21.15.45:0/4036426031 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff4780d4ae0 con 0x5599da1aad90 2024-04-25T10:23:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4780c0890 con 0x5599da1aad90 2024-04-25T10:23:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 1 -- 172.21.15.45:0/4036426031 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4780c4070 con 0x5599da1aad90 2024-04-25T10:23:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 10 client.6718 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:28.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5599d9f9af80 con 0x5599da1aad90 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 1 -- 172.21.15.45:0/4036426031 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4780c0890 con 0x5599da1aad90 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 1 -- 172.21.15.45:0/4036426031 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff4780d2070 con 0x5599da1aad90 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 10 client.6718 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 10 client.6718 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 20 client.6718 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1f5890 0x5599da1f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 4 client.6718 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5599da0013d0 con 0x5599da1f5890 2024-04-25T10:23:28.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff47fe00640 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1f5890 0x5599da1f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff47fe00640 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1f5890 0x5599da1f5ca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede34ab912f40aa0 server_cookie=fff5571e0883a301 in_seq=0 out_seq=0 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 10 client.6718 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 1 -- 172.21.15.45:0/4036426031 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17924 (crc 0 0 0) 0x5599da0013d0 con 0x5599da1f5890 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 10 client.6718 handle_command_reply: tid=0 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 client.6718 shutdown 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 msgr2=0x5599da2502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 0x5599da2502a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff478001940 tx=0x7ff478007f50 comp rx=0 tx=0).stop 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 shutdown_connections 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5599da1f5890 0x5599da1f5ca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5599da1f0360 0x5599da1f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5599da1efa40 0x5599da2507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 --2- 172.21.15.45:0/4036426031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599da1aad90 0x5599da2502a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 >> 172.21.15.45:0/4036426031 conn(0x5599da1a6d40 msgr2=0x5599da1eea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff474c00640 0 client.6718 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 shutdown_connections 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 1 -- 172.21.15.45:0/4036426031 wait complete. 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.265+0000 7ff491aac000 20 client.6718 trim_cache size 0 max 16384 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2269, 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2262, 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 660486, 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 24941064, 2024-04-25T10:23:28.274 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 8, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.243006309, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.109140214, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.057132167, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.060280850, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2261, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.418026766, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000184885 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000305, 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000152 2024-04-25T10:23:28.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 114944, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 19472 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 30385, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 51280, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 10878629, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 56121336, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 8, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 4.781889494, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 2.870099710, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 2.371157923, 2024-04-25T10:23:28.276 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.051593023, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 51279, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.649236788, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000032162 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 72376, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.017216602, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000237 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 108368 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:23:28.277 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2517, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2517, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 842008, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 37130893, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.299204179, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.148205804, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.059951824, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.071078181, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2516, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.466013482, 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000185219 2024-04-25T10:23:28.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:23:28.279 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4323, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 13.204686395, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003054519 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1822, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.637683888, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000349991 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T10:23:28.280 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 26273, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 26273, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 26273, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 241.836952543, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.009204771 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 12, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 19755, 2024-04-25T10:23:28.281 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 19755, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 19755, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 2, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 2, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 2, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 49066, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 48519, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T10:23:28.282 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 156, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:23:28.283 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 1437, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 4068, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 4546, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:23:28.284 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 30, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 14092 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 1613, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 4141, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 2000, 2024-04-25T10:23:28.285 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 2244, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.286 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 28737, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 28737, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 45, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 45, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 4, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T10:23:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 95419335, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4162, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 79.720400135, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.019154348 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1627, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 2014, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 387, 2024-04-25T10:23:28.288 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 13, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 13, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 19368, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 19756, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 388, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 2, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 3701, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 3699, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 109788, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215340 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 29833, 2024-04-25T10:23:28.289 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 26273, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 3, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2000, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.341677269, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000670838 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 28, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003748177, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000133863 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.290 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6000, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 7.046492249, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001174415 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7988, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 6.668837352, 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000834856 2024-04-25T10:23:28.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000062156, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000062156 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003260683, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003260683 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:28.294 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.006099852, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.006099852 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.093919533, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.011739941 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T10:23:28.295 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2244, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 210.487101911, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.093799956 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.296 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.010646796, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.010646796 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.297 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000235401, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000235401 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8000, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 16.174871164, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002021858 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.300 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 18919, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 18919, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 53, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:23:28.301 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:23:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T10:23:28.303 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 19414138, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 4873, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T10:23:28.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 5504, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 79, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 20639358, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 814701, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T10:23:28.305 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 6379, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 91658336, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 37, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 6379, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6379, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 25.863618986, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.004054494 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6379, 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 6474 2024-04-25T10:23:28.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 6379, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 41, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 6338, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 11, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 41, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 4203, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 27, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 1613, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T10:23:28.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 420, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 22, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T10:23:28.308 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T10:23:28.309 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 79, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 3, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 8, 2024-04-25T10:23:28.310 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 8, 2024-04-25T10:23:28.313 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T10:23:28.313 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T10:23:28.313 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 106, 2024-04-25T10:23:28.313 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 31 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T10:23:28.314 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 32, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 2000, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 2000, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 35166, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 9211040, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 35166, 2024-04-25T10:23:28.315 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 35166, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9211040, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:28.316 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 6379, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 91873240, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 6379, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 91873240, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T10:23:28.317 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 6379, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 6379, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 6379, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 6379, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 2000, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 214904, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 2000, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 41, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 214904, 2024-04-25T10:23:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x55d6dc22ca00": { 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 60, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 28737, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 91225031, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 28737, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:23:28.320 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4162, 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 91224971, 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.321 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4640707f0 msgr2=0x7ff464072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4640707f0 0x7ff464072cb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff4780ac000 tx=0x7ff4780a7000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 msgr2=0x7ff4880cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 0x7ff4880cb6c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff47000f840 tx=0x7ff47000fc40 comp rx=0 tx=0).stop 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 shutdown_connections 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4640707f0 0x7ff464072cb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff48815e1b0 0x7ff488089c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4880ccf40 0x7ff4880cb6c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 --2- 172.21.15.45:0/3773474776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4640035f0 0x7ff48815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 >> 172.21.15.45:0/3773474776 conn(0x7ff488066da0 msgr2=0x7ff4880af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 shutdown_connections 2024-04-25T10:23:28.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.269+0000 7ff48d000640 1 -- 172.21.15.45:0/3773474776 wait complete. 2024-04-25T10:23:28.324 INFO:tasks.cephfs.test_scrub_checks:num of strays: 1613 2024-04-25T10:23:28.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:28.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T10:23:28.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.473+0000 7fa46ea00640 1 Processor -- start 2024-04-25T10:23:28.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.473+0000 7fa46ea00640 1 -- start start 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa46ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa46815bd10 0x7fa4680b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa46ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa46815c0f0 0x7fa4680b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa46ea00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46806c0e0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa46ea00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46815c6a0 con 0x7fa46815c0f0 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa46ea00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46815c980 con 0x7fa46815bd10 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa46815c0f0 0x7fa4680b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa46815c0f0 0x7fa4680b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41104/0 (socket says 172.21.15.45:41104) 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 -- 172.21.15.45:0/4193944396 learned_addr learned my addr 172.21.15.45:0/4193944396 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa467e00640 1 --2- 172.21.15.45:0/4193944396 >> v2:172.21.15.117:3300/0 conn(0x7fa46815bd10 0x7fa4680b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 -- 172.21.15.45:0/4193944396 >> v2:172.21.15.117:3300/0 conn(0x7fa46815bd10 msgr2=0x7fa4680b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 --2- 172.21.15.45:0/4193944396 >> v2:172.21.15.117:3300/0 conn(0x7fa46815bd10 0x7fa4680b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 -- 172.21.15.45:0/4193944396 >> v1:172.21.15.117:6789/0 conn(0x7fa4680cf9e0 legacy=0x7fa46806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.477+0000 7fa45f400640 1 -- 172.21.15.45:0/4193944396 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46815cc60 con 0x7fa46815c0f0 2024-04-25T10:23:28.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa45f400640 1 --2- 172.21.15.45:0/4193944396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa46815c0f0 0x7fa4680b27e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa468072810 tx=0x7fa45001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5632e3aaf5c3b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa467400640 1 -- 172.21.15.45:0/4193944396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500aa670 con 0x7fa46815c0f0 2024-04-25T10:23:28.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa467400640 1 -- 172.21.15.45:0/4193944396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500af070 con 0x7fa46815c0f0 2024-04-25T10:23:28.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa467400640 1 -- 172.21.15.45:0/4193944396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500aaa30 con 0x7fa46815c0f0 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- 172.21.15.45:0/4193944396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa46815c0f0 msgr2=0x7fa4680b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 --2- 172.21.15.45:0/4193944396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa46815c0f0 0x7fa4680b27e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa468072810 tx=0x7fa45001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- 172.21.15.45:0/4193944396 shutdown_connections 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 --2- 172.21.15.45:0/4193944396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa46815c0f0 0x7fa4680b27e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 --2- 172.21.15.45:0/4193944396 >> v2:172.21.15.117:3300/0 conn(0x7fa46815bd10 0x7fa4680b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- 172.21.15.45:0/4193944396 >> 172.21.15.45:0/4193944396 conn(0x7fa468066b70 msgr2=0x7fa468066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- 172.21.15.45:0/4193944396 shutdown_connections 2024-04-25T10:23:28.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- 172.21.15.45:0/4193944396 wait complete. 2024-04-25T10:23:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 Processor -- start 2024-04-25T10:23:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- start start 2024-04-25T10:23:28.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 0x7fa468159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa46815bd10 0x7fa46813fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46815c0f0 0x7fa46814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680b2ea0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680b2d20 con 0x7fa46815c0f0 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.481+0000 7fa46ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4681692d0 con 0x7fa46815bd10 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 0x7fa468159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 0x7fa468159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41108/0 (socket says 172.21.15.45:41108) 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 -- 172.21.15.45:0/1258639516 learned_addr learned my addr 172.21.15.45:0/1258639516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa46ca00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46815c0f0 0x7fa46814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa45f400640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa46815bd10 0x7fa46813fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 -- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa46815bd10 msgr2=0x7fa46813fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa46815bd10 0x7fa46813fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 -- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46815c0f0 msgr2=0x7fa46814e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46815c0f0 0x7fa46814e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 -- 172.21.15.45:0/1258639516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46814ece0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa467e00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 0x7fa468159820 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d5f0 tx=0x7fa45800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa464c00640 1 -- 172.21.15.45:0/1258639516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580125f0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa464c00640 1 -- 172.21.15.45:0/1258639516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458019070 con 0x7fa4680cf9e0 2024-04-25T10:23:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa464c00640 1 -- 172.21.15.45:0/1258639516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580129b0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa468159ea0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46815a460 con 0x7fa4680cf9e0 2024-04-25T10:23:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.485+0000 7fa464c00640 1 -- 172.21.15.45:0/1258639516 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa458012b50 con 0x7fa4680cf9e0 2024-04-25T10:23:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa464c00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa444066f60 0x7fa444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa464c00640 1 -- 172.21.15.45:0/1258639516 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4580853d0 con 0x7fa4680cf9e0 2024-04-25T10:23:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa45f400640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa444066f60 0x7fa444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 Processor -- start 2024-04-25T10:23:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- start start 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa45f400640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa444066f60 0x7fa444069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa4500ae000 tx=0x7fa4500a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d61e0 0x55996c28b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c28b650 0x55996c28ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996c0d9dd0 con 0x55996c2d58c0 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996c062010 con 0x55996c28b650 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55996c0d5e70 con 0x55996c2d61e0 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2d5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41112/0 (socket says 172.21.15.45:41112) 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 -- 172.21.15.45:0/3247400972 learned_addr learned my addr 172.21.15.45:0/3247400972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa45f400640 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d61e0 0x55996c28b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa46ca00640 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c28b650 0x55996c28ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 -- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d61e0 msgr2=0x55996c28b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d61e0 0x55996c28b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 -- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c28b650 msgr2=0x55996c28ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c28b650 0x55996c28ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 -- 172.21.15.45:0/3247400972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996c0929c0 con 0x55996c2d58c0 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa467e00640 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2d5ca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa458083790 tx=0x7fa4580556d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47325d4d5a858fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa45cc00640 1 -- 172.21.15.45:0/3247400972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45805e310 con 0x55996c2d58c0 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa45cc00640 1 -- 172.21.15.45:0/3247400972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45807e070 con 0x55996c2d58c0 2024-04-25T10:23:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa45cc00640 1 -- 172.21.15.45:0/3247400972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458053fb0 con 0x55996c2d58c0 2024-04-25T10:23:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- 172.21.15.45:0/3247400972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 msgr2=0x55996c2d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2d5ca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa458083790 tx=0x7fa4580556d0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- 172.21.15.45:0/3247400972 shutdown_connections 2024-04-25T10:23:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c28b650 0x55996c28ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d61e0 0x55996c28b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 --2- 172.21.15.45:0/3247400972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2d5ca0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- 172.21.15.45:0/3247400972 >> 172.21.15.45:0/3247400972 conn(0x55996c173870 msgr2=0x55996c2899f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- 172.21.15.45:0/3247400972 shutdown_connections 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.489+0000 7fa471e00000 1 -- 172.21.15.45:0/3247400972 wait complete. 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- start start 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 Processor -- start 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c28b650 0x55996c28ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c2d61e0 0x55996c2909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996c094420 con 0x55996c2d58c0 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996c294c20 con 0x55996c2d61e0 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55996c15c470 con 0x55996c28b650 2024-04-25T10:23:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 10 client.0 _ll_register_callbacks cb 0x55996c276f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa467e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c28b650 0x55996c28ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2904a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41122/0 (socket says 172.21.15.45:41122) 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 -- 172.21.15.45:0/3698187580 learned_addr learned my addr 172.21.15.45:0/3698187580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa46ca00640 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c2d61e0 0x55996c2909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 -- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c28b650 msgr2=0x55996c28ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c28b650 0x55996c28ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 -- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c2d61e0 msgr2=0x55996c2909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c2d61e0 0x55996c2909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 -- 172.21.15.45:0/3698187580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55996c062260 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa45f400640 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2904a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa450009630 tx=0x7fa45001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d1274be3929977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 1 -- 172.21.15.45:0/3698187580 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500aaca0 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 1 -- 172.21.15.45:0/3698187580 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4500cab10 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa45807e070 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 1 -- 172.21.15.45:0/3698187580 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4500aae40 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 10 client.7074 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55996c094b30 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 1 -- 172.21.15.45:0/3698187580 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45807e070 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 1 -- 172.21.15.45:0/3698187580 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4500d1050 con 0x55996c2d58c0 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 10 client.7074 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 10 client.7074 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 20 client.7074 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2913f0 0x55996c2e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 4 client.7074 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55996c075c80 con 0x55996c2913f0 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa467e00640 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2913f0 0x55996c2e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa467e00640 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2913f0 0x55996c2e6960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecfbc51756ae7c9 server_cookie=19a6594609c4fc32 in_seq=0 out_seq=0 2024-04-25T10:23:28.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.493+0000 7fa455600640 10 client.7074 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa455600640 1 -- 172.21.15.45:0/3698187580 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55996c075c80 con 0x55996c2913f0 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa455600640 10 client.7074 handle_command_reply: tid=0 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 client.7074 shutdown 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 msgr2=0x55996c2904a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2904a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa450009630 tx=0x7fa45001ad30 comp rx=0 tx=0).stop 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 shutdown_connections 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2913f0 0x55996c2e6960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c2d61e0 0x55996c2909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c2d58c0 0x55996c2904a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 --2- 172.21.15.45:0/3698187580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c28b650 0x55996c28ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa455600640 0 client.7074 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 >> 172.21.15.45:0/3698187580 conn(0x55996c173870 msgr2=0x55996c2899f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 shutdown_connections 2024-04-25T10:23:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 1 -- 172.21.15.45:0/3698187580 wait complete. 2024-04-25T10:23:28.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.497+0000 7fa471e00000 20 client.7074 trim_cache size 0 max 16384 2024-04-25T10:23:28.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 Processor -- start 2024-04-25T10:23:28.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- start start 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c331c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c332150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c332690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa468203790 con 0x55996c28b650 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4682028b0 con 0x55996c173870 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680b1020 con 0x55996c2d58c0 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c332150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c332150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41136/0 (socket says 172.21.15.45:41136) 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 -- 172.21.15.45:0/1362219667 learned_addr learned my addr 172.21.15.45:0/1362219667 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa46ca00640 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c332690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa467e00640 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c331c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 -- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 msgr2=0x55996c332690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c332690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 -- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 msgr2=0x55996c331c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c331c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 -- 172.21.15.45:0/1362219667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996c15f6d0 con 0x55996c28b650 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa45f400640 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c332150 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa450019100 tx=0x7fa4500c4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d89e04917a1d84b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa457e00640 1 -- 172.21.15.45:0/1362219667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500aaca0 con 0x55996c28b650 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa457e00640 1 -- 172.21.15.45:0/1362219667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500af040 con 0x55996c28b650 2024-04-25T10:23:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa457e00640 1 -- 172.21.15.45:0/1362219667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa450003930 con 0x55996c28b650 2024-04-25T10:23:28.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- 172.21.15.45:0/1362219667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 msgr2=0x55996c332150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c332150 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa450019100 tx=0x7fa4500c4000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- 172.21.15.45:0/1362219667 shutdown_connections 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c332690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c332150 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 --2- 172.21.15.45:0/1362219667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c331c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- 172.21.15.45:0/1362219667 >> 172.21.15.45:0/1362219667 conn(0x55996c2b7610 msgr2=0x55996c2d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- 172.21.15.45:0/1362219667 shutdown_connections 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- 172.21.15.45:0/1362219667 wait complete. 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.513+0000 7fa471e00000 1 -- start start 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 Processor -- start 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c3369c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c336f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c31a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996c294c20 con 0x55996c28b650 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa468202310 con 0x55996c173870 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55996c0d90e0 con 0x55996c2d58c0 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 10 client.0 _ll_register_callbacks cb 0x55996c076200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c336f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c336f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41150/0 (socket says 172.21.15.45:41150) 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa467e00640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c3369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 -- 172.21.15.45:0/1874184644 learned_addr learned my addr 172.21.15.45:0/1874184644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa46ca00640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c31a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 -- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 msgr2=0x55996c31a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c31a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 -- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 msgr2=0x55996c3369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c3369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 -- 172.21.15.45:0/1874184644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55996c15c900 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa45f400640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c336f00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa45001ca60 tx=0x7fa450008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b8b511800dfe32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 1 -- 172.21.15.45:0/1874184644 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4500d7070 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 1 -- 172.21.15.45:0/1874184644 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa4500d3680 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4500af040 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 1 -- 172.21.15.45:0/1874184644 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4500e2050 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 10 client.7080 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55996c080f20 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 1 -- 172.21.15.45:0/1874184644 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4500af040 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 1 -- 172.21.15.45:0/1874184644 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4500e5020 con 0x55996c28b650 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 10 client.7080 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 10 client.7080 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 20 client.7080 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2d61e0 0x55996c31ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 4 client.7080 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55996c0e7310 con 0x55996c2d61e0 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa467e00640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2d61e0 0x55996c31ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa455600640 10 client.7080 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.517+0000 7fa467e00640 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2d61e0 0x55996c31ab00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772c376d9326d0a3 server_cookie=96c41c296f1bc6b0 in_seq=0 out_seq=0 2024-04-25T10:23:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa455600640 1 -- 172.21.15.45:0/1874184644 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55996c0e7310 con 0x55996c2d61e0 2024-04-25T10:23:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa455600640 10 client.7080 handle_command_reply: tid=0 2024-04-25T10:23:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 client.7080 shutdown 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 msgr2=0x55996c336f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c336f00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa45001ca60 tx=0x7fa450008040 comp rx=0 tx=0).stop 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 shutdown_connections 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55996c2d61e0 0x55996c31ab00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55996c2d58c0 0x55996c31a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55996c28b650 0x55996c336f00 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 --2- 172.21.15.45:0/1874184644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55996c173870 0x55996c3369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 >> 172.21.15.45:0/1874184644 conn(0x55996c2b7610 msgr2=0x55996c2d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa455600640 0 client.7080 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 shutdown_connections 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 1 -- 172.21.15.45:0/1874184644 wait complete. 2024-04-25T10:23:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.525+0000 7fa471e00000 20 client.7080 trim_cache size 0 max 16384 2024-04-25T10:23:28.533 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:23:28.533 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:23:28.533 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "7f391cbd-463d-40e1-ad6e-8d0f2144c73e", 2024-04-25T10:23:28.533 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:23:28.533 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:23:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa444066f60 msgr2=0x7fa444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa444066f60 0x7fa444069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa4500ae000 tx=0x7fa4500a7000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 msgr2=0x7fa468159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 0x7fa468159820 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa45800d5f0 tx=0x7fa45800dab0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 shutdown_connections 2024-04-25T10:23:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa444066f60 0x7fa444069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa46815c0f0 0x7fa46814e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa46815bd10 0x7fa46813fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 --2- 172.21.15.45:0/1258639516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4680cf9e0 0x7fa468159820 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 >> 172.21.15.45:0/1258639516 conn(0x7fa468066b70 msgr2=0x7fa468154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 shutdown_connections 2024-04-25T10:23:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.529+0000 7fa46ea00640 1 -- 172.21.15.45:0/1258639516 wait complete. 2024-04-25T10:23:28.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:28.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:23:28.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 Processor -- start 2024-04-25T10:23:28.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 -- start start 2024-04-25T10:23:28.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7298071db0 0x7f72980b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72980cf690 0x7f72980cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7298072620 con 0x7f72980aed00 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729806dae0 con 0x7f72980cf690 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729fe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72980d0020 con 0x7f7298071db0 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72980cf690 0x7f72980cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72980cf690 0x7f72980cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41156/0 (socket says 172.21.15.45:41156) 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729de00640 1 -- 172.21.15.45:0/3519607278 learned_addr learned my addr 172.21.15.45:0/3519607278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.717+0000 7f729ca00640 1 --2- 172.21.15.45:0/3519607278 >> v2:172.21.15.117:3300/0 conn(0x7f7298071db0 0x7f72980b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.817+0000 7f729de00640 1 -- 172.21.15.45:0/3519607278 >> v2:172.21.15.117:3300/0 conn(0x7f7298071db0 msgr2=0x7f72980b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.817+0000 7f729de00640 1 --2- 172.21.15.45:0/3519607278 >> v2:172.21.15.117:3300/0 conn(0x7f7298071db0 0x7f72980b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.817+0000 7f729de00640 1 -- 172.21.15.45:0/3519607278 >> v1:172.21.15.45:6789/0 conn(0x7f72980aed00 legacy=0x7f72980b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.817+0000 7f729de00640 1 -- 172.21.15.45:0/3519607278 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298072340 con 0x7f72980cf690 2024-04-25T10:23:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729de00640 1 --2- 172.21.15.45:0/3519607278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980cf690 0x7f72980cfae0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f72980676b0 tx=0x7f729001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a4ec6068a21a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f7297e00640 1 -- 172.21.15.45:0/3519607278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900b2070 con 0x7f72980cf690 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f7297e00640 1 -- 172.21.15.45:0/3519607278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72900a5b40 con 0x7f72980cf690 2024-04-25T10:23:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f7297e00640 1 -- 172.21.15.45:0/3519607278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900ad440 con 0x7f72980cf690 2024-04-25T10:23:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 -- 172.21.15.45:0/3519607278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980cf690 msgr2=0x7f72980cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 --2- 172.21.15.45:0/3519607278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980cf690 0x7f72980cfae0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f72980676b0 tx=0x7f729001cc70 comp rx=0 tx=0).stop 2024-04-25T10:23:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 -- 172.21.15.45:0/3519607278 shutdown_connections 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 --2- 172.21.15.45:0/3519607278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980cf690 0x7f72980cfae0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 --2- 172.21.15.45:0/3519607278 >> v2:172.21.15.117:3300/0 conn(0x7f7298071db0 0x7f72980b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 -- 172.21.15.45:0/3519607278 >> 172.21.15.45:0/3519607278 conn(0x7f72980b5480 msgr2=0x7f72980669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 -- 172.21.15.45:0/3519607278 shutdown_connections 2024-04-25T10:23:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.825+0000 7f729fe00640 1 -- 172.21.15.45:0/3519607278 wait complete. 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 Processor -- start 2024-04-25T10:23:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 -- start start 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298071db0 0x7f7298162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 0x7f7298160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72980cf690 0x7f7298168d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72980af260 con 0x7f72980aed00 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72980af0e0 con 0x7f7298071db0 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729816b7a0 con 0x7f72980cf690 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 0x7f7298160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 0x7f7298160b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41160/0 (socket says 172.21.15.45:41160) 2024-04-25T10:23:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 -- 172.21.15.45:0/3174170523 learned_addr learned my addr 172.21.15.45:0/3174170523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729de00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72980cf690 0x7f7298168d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729d400640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298071db0 0x7f7298162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 -- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72980cf690 msgr2=0x7f7298168d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72980cf690 0x7f7298168d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 -- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298071db0 msgr2=0x7f7298162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298071db0 0x7f7298162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 -- 172.21.15.45:0/3174170523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72981613b0 con 0x7f72980aed00 2024-04-25T10:23:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.829+0000 7f729ca00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 0x7f7298160b60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f728400fa60 tx=0x7f728400ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.833+0000 7f7295600640 1 -- 172.21.15.45:0/3174170523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400dcd0 con 0x7f72980aed00 2024-04-25T10:23:28.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.833+0000 7f7295600640 1 -- 172.21.15.45:0/3174170523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728401a480 con 0x7f72980aed00 2024-04-25T10:23:28.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.833+0000 7f7295600640 1 -- 172.21.15.45:0/3174170523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728401a830 con 0x7f72980aed00 2024-04-25T10:23:28.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.833+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298161690 con 0x7f72980aed00 2024-04-25T10:23:28.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.833+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7298161c50 con 0x7f72980aed00 2024-04-25T10:23:28.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.833+0000 7f7295600640 1 -- 172.21.15.45:0/3174170523 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7284011450 con 0x7f72980aed00 2024-04-25T10:23:28.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f7295600640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7278066f60 0x7f7278069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f7295600640 1 -- 172.21.15.45:0/3174170523 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7284085950 con 0x7f72980aed00 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 Processor -- start 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729d400640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7278066f60 0x7f7278069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 -- start start 2024-04-25T10:23:28.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729d400640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7278066f60 0x7f7278069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7298165210 tx=0x7f728c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff51b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff52480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd9eff0 con 0x557d1ff9c2f0 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd27150 con 0x557d1ff520a0 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd9b090 con 0x557d1ff9b9d0 2024-04-25T10:23:28.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff51b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff51b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41172/0 (socket says 172.21.15.45:41172) 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 -- 172.21.15.45:0/3389390296 learned_addr learned my addr 172.21.15.45:0/3389390296 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729d400640 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729de00640 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff52480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 -- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 msgr2=0x557d1ff9bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff9bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 -- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 msgr2=0x557d1ff52480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff52480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729ca00640 1 -- 172.21.15.45:0/3389390296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d1ff350d0 con 0x557d1ff9c2f0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.837+0000 7f729de00640 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff52480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f729ca00640 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff51b60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f728400fff0 tx=0x7f7284056ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfceb0977620085a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f728a000640 1 -- 172.21.15.45:0/3389390296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728401f030 con 0x557d1ff9c2f0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f728a000640 1 -- 172.21.15.45:0/3389390296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728405a1b0 con 0x557d1ff9c2f0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f728a000640 1 -- 172.21.15.45:0/3389390296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728405e9b0 con 0x557d1ff9c2f0 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 -- 172.21.15.45:0/3389390296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 msgr2=0x557d1ff51b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff51b60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f728400fff0 tx=0x7f7284056ae0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 -- 172.21.15.45:0/3389390296 shutdown_connections 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff52480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff51b60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 --2- 172.21.15.45:0/3389390296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 -- 172.21.15.45:0/3389390296 >> 172.21.15.45:0/3389390296 conn(0x557d1ff4e020 msgr2=0x557d1ff50440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 -- 172.21.15.45:0/3389390296 shutdown_connections 2024-04-25T10:23:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 -- 172.21.15.45:0/3389390296 wait complete. 2024-04-25T10:23:28.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 -- start start 2024-04-25T10:23:28.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.841+0000 7f72a333b000 1 Processor -- start 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff80d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd63250 con 0x557d1ff9c2f0 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd59690 con 0x557d1ff520a0 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd27a40 con 0x557d1ff9b9d0 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 10 client.0 _ll_register_callbacks cb 0x557d1ff3ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff807f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36242/0 (socket says 172.21.15.45:36242) 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 -- 172.21.15.45:0/3358909003 learned_addr learned my addr 172.21.15.45:0/3358909003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729de00640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729ca00640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff80d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 -- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 msgr2=0x557d1ff80d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff80d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 -- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 msgr2=0x557d1ff81270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff81270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 -- 172.21.15.45:0/3358909003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d1fe4c1c0 con 0x557d1ff520a0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f729d400640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff807f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f728c003980 tx=0x7f728c000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca58b1cfe0a4362c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f727ea00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f727ea00640 1 -- 172.21.15.45:0/3358909003 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c028060 con 0x557d1ff520a0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f727ea00640 1 -- 172.21.15.45:0/3358909003 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f728c010040 con 0x557d1ff520a0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f728405a1b0 con 0x557d1ff520a0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f727ea00640 1 -- 172.21.15.45:0/3358909003 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f728c00ecc0 con 0x557d1ff520a0 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 10 client.6751 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:28.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x557d1fe48170 con 0x557d1ff520a0 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f727ea00640 1 -- 172.21.15.45:0/3358909003 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728405a1b0 con 0x557d1ff520a0 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f727ea00640 1 -- 172.21.15.45:0/3358909003 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f728c013420 con 0x557d1ff520a0 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 10 client.6751 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 10 client.6751 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 20 client.6751 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffabc80 0x557d1ffae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 4 client.6751 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.845+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d1fd3ac60 con 0x557d1ffabc80 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.849+0000 7f729ca00640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffabc80 0x557d1ffae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.849+0000 7f729ca00640 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffabc80 0x557d1ffae070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3693ac415ac190 server_cookie=45408b3c8be9829f in_seq=0 out_seq=0 2024-04-25T10:23:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.849+0000 7f727ea00640 10 client.6751 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.849+0000 7f727ea00640 1 -- 172.21.15.45:0/3358909003 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d1fd3ac60 con 0x557d1ffabc80 2024-04-25T10:23:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f727ea00640 10 client.6751 handle_command_reply: tid=0 2024-04-25T10:23:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 client.6751 shutdown 2024-04-25T10:23:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 msgr2=0x557d1ff807f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff807f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f728c003980 tx=0x7f728c000ee0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 shutdown_connections 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffabc80 0x557d1ffae070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9c2f0 0x557d1ff81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9b9d0 0x557d1ff80d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f727ea00640 0 client.6751 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 --2- 172.21.15.45:0/3358909003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1ff807f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 >> 172.21.15.45:0/3358909003 conn(0x557d1ff4e020 msgr2=0x557d1ff50ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 shutdown_connections 2024-04-25T10:23:28.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 1 -- 172.21.15.45:0/3358909003 wait complete. 2024-04-25T10:23:28.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.853+0000 7f72a333b000 20 client.6751 trim_cache size 0 max 16384 2024-04-25T10:23:28.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.881+0000 7f72a333b000 1 Processor -- start 2024-04-25T10:23:28.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.881+0000 7f72a333b000 1 -- start start 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.881+0000 7f72a333b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 0x557d1fff8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff9b9d0 0x557d1fff8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1fff8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298213ea0 con 0x557d1ff520a0 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298213140 con 0x557d1ff9b9d0 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72981533b0 con 0x557d1ff9c2f0 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 0x557d1fff8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 0x557d1fff8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41190/0 (socket says 172.21.15.45:41190) 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 -- 172.21.15.45:0/3988174183 learned_addr learned my addr 172.21.15.45:0/3988174183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729ca00640 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff9b9d0 0x557d1fff8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729de00640 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1fff8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 -- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 msgr2=0x557d1fff8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1fff8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 -- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff9b9d0 msgr2=0x557d1fff8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff9b9d0 0x557d1fff8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 -- 172.21.15.45:0/3988174183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d1fde5de0 con 0x557d1ff520a0 2024-04-25T10:23:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f729d400640 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 0x557d1fff8300 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f728c00e2e0 tx=0x7f728c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0068d4c5c36116d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f7289600640 1 -- 172.21.15.45:0/3988174183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00ee60 con 0x557d1ff520a0 2024-04-25T10:23:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f7289600640 1 -- 172.21.15.45:0/3988174183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c019070 con 0x557d1ff520a0 2024-04-25T10:23:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f7289600640 1 -- 172.21.15.45:0/3988174183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c013420 con 0x557d1ff520a0 2024-04-25T10:23:28.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- 172.21.15.45:0/3988174183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 msgr2=0x557d1fff8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 0x557d1fff8300 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f728c00e2e0 tx=0x7f728c01d000 comp rx=0 tx=0).stop 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- 172.21.15.45:0/3988174183 shutdown_connections 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1fff8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff9b9d0 0x557d1fff8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 --2- 172.21.15.45:0/3988174183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff520a0 0x557d1fff8300 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- 172.21.15.45:0/3988174183 >> 172.21.15.45:0/3988174183 conn(0x557d1ff4e020 msgr2=0x557d1ffa8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- 172.21.15.45:0/3988174183 shutdown_connections 2024-04-25T10:23:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- 172.21.15.45:0/3988174183 wait complete. 2024-04-25T10:23:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.885+0000 7f72a333b000 1 -- start start 2024-04-25T10:23:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 Processor -- start 2024-04-25T10:23:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1fffc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 0x557d1fffc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1ffae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd39e00 con 0x557d1ff9b9d0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981539b0 con 0x557d1ff520a0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d1fd9e300 con 0x557d1ff9c2f0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 10 client.0 _ll_register_callbacks cb 0x557d1ff1d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 0x557d1fffc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 0x557d1fffc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41200/0 (socket says 172.21.15.45:41200) 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 -- 172.21.15.45:0/389872606 learned_addr learned my addr 172.21.15.45:0/389872606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729d400640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1fffc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729de00640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1ffae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 -- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 msgr2=0x557d1ffae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1ffae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 -- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 msgr2=0x557d1fffc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1fffc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 -- 172.21.15.45:0/389872606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d1fd408e0 con 0x557d1ff9b9d0 2024-04-25T10:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f729ca00640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 0x557d1fffc850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f728400fa30 tx=0x7f7284000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6e024778f90605 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:28.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f727ea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f727ea00640 1 -- 172.21.15.45:0/389872606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728401f040 con 0x557d1ff9b9d0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f727ea00640 1 -- 172.21.15.45:0/389872606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7284052910 con 0x557d1ff9b9d0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f728c019070 con 0x557d1ff9b9d0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f727ea00640 1 -- 172.21.15.45:0/389872606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f728405a1b0 con 0x557d1ff9b9d0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 10 client.7101 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x557d1fdf88f0 con 0x557d1ff9b9d0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.889+0000 7f727ea00640 1 -- 172.21.15.45:0/389872606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284072030 con 0x557d1ff9b9d0 2024-04-25T10:23:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f727ea00640 1 -- 172.21.15.45:0/389872606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f728400e040 con 0x557d1ff9b9d0 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 10 client.7101 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 10 client.7101 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 20 client.7101 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffaecb0 0x557d1ffaf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 4 client.7101 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d1fdac530 con 0x557d1ffaecb0 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f729d400640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffaecb0 0x557d1ffaf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f729d400640 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffaecb0 0x557d1ffaf0c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627fac504e166c67 server_cookie=ff454e2d410db9a2 in_seq=0 out_seq=0 2024-04-25T10:23:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f727ea00640 10 client.7101 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f727ea00640 1 -- 172.21.15.45:0/389872606 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557d1fdac530 con 0x557d1ffaecb0 2024-04-25T10:23:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f727ea00640 10 client.7101 handle_command_reply: tid=0 2024-04-25T10:23:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 client.7101 shutdown 2024-04-25T10:23:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 msgr2=0x557d1fffc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 0x557d1fffc850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f728400fa30 tx=0x7f7284000bd0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 shutdown_connections 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x557d1ffaecb0 0x557d1ffaf0c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d1ff9c2f0 0x557d1ffae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d1ff9b9d0 0x557d1fffc850 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 --2- 172.21.15.45:0/389872606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d1ff520a0 0x557d1fffc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 >> 172.21.15.45:0/389872606 conn(0x557d1ff4e020 msgr2=0x557d1ff9a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f727ea00640 0 client.7101 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 shutdown_connections 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 1 -- 172.21.15.45:0/389872606 wait complete. 2024-04-25T10:23:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f72a333b000 20 client.7101 trim_cache size 0 max 16384 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (761 inodes in the stack)", 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: "7f391cbd-463d-40e1-ad6e-8d0f2144c73e": { 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "~mds0", 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "7f391cbd-463d-40e1-ad6e-8d0f2144c73e", 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:28.900 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:23:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7278066f60 msgr2=0x7f7278069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f729fe00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7278066f60 0x7f7278069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7298165210 tx=0x7f728c009040 comp rx=0 tx=0).stop 2024-04-25T10:23:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 msgr2=0x7f7298160b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.893+0000 7f729fe00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 0x7f7298160b60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f728400fa60 tx=0x7f728400ff20 comp rx=0 tx=0).stop 2024-04-25T10:23:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 shutdown_connections 2024-04-25T10:23:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7278066f60 0x7f7278069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72980cf690 0x7f7298168d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72980aed00 0x7f7298160b60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 --2- 172.21.15.45:0/3174170523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298071db0 0x7f7298162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:28.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 >> 172.21.15.45:0/3174170523 conn(0x7f72980b5480 msgr2=0x7f729816a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:28.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 shutdown_connections 2024-04-25T10:23:28.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:28.897+0000 7f729fe00640 1 -- 172.21.15.45:0/3174170523 wait complete. 2024-04-25T10:23:28.929 INFO:tasks.cephfs.filesystem:scrub status for tag:7f391cbd-463d-40e1-ad6e-8d0f2144c73e - {'path': '~mds0', 'tag': '7f391cbd-463d-40e1-ad6e-8d0f2144c73e', 'options': 'recursive'} 2024-04-25T10:23:58.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:23:58.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:23:59.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 Processor -- start 2024-04-25T10:23:59.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 -- start start 2024-04-25T10:23:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f318015bca0 0x7f31800cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31800d07b0 0x7f31800ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31800cc9d0 con 0x7f318015f760 2024-04-25T10:23:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318015c080 con 0x7f318015bca0 2024-04-25T10:23:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f3185000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31800cddf0 con 0x7f31800d07b0 2024-04-25T10:23:59.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f318015bca0 0x7f31800cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f318015bca0 0x7f31800cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46326/0 (socket says 172.21.15.45:46326) 2024-04-25T10:23:59.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 -- 172.21.15.45:0/3507189790 learned_addr learned my addr 172.21.15.45:0/3507189790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317e800640 1 --2- 172.21.15.45:0/3507189790 >> v2:172.21.15.117:3300/0 conn(0x7f31800d07b0 0x7f31800ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 -- 172.21.15.45:0/3507189790 >> v2:172.21.15.117:3300/0 conn(0x7f31800d07b0 msgr2=0x7f31800ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 --2- 172.21.15.45:0/3507189790 >> v2:172.21.15.117:3300/0 conn(0x7f31800d07b0 0x7f31800ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 -- 172.21.15.45:0/3507189790 >> v1:172.21.15.45:6789/0 conn(0x7f318015f760 legacy=0x7f31800d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317de00640 1 -- 172.21.15.45:0/3507189790 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318015fc40 con 0x7f318015bca0 2024-04-25T10:23:59.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.085+0000 7f317e800640 1 --2- 172.21.15.45:0/3507189790 >> v2:172.21.15.117:3300/0 conn(0x7f31800d07b0 0x7f31800ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f317de00640 1 --2- 172.21.15.45:0/3507189790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800cc490 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f31800749a0 tx=0x7f317001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621cbbb9105f9b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f317d400640 1 -- 172.21.15.45:0/3507189790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700aa650 con 0x7f318015bca0 2024-04-25T10:23:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f317d400640 1 -- 172.21.15.45:0/3507189790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31700af070 con 0x7f318015bca0 2024-04-25T10:23:59.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f317d400640 1 -- 172.21.15.45:0/3507189790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700aaa10 con 0x7f318015bca0 2024-04-25T10:23:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 -- 172.21.15.45:0/3507189790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 msgr2=0x7f31800cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 --2- 172.21.15.45:0/3507189790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800cc490 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f31800749a0 tx=0x7f317001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 -- 172.21.15.45:0/3507189790 shutdown_connections 2024-04-25T10:23:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 --2- 172.21.15.45:0/3507189790 >> v2:172.21.15.117:3300/0 conn(0x7f31800d07b0 0x7f31800ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 --2- 172.21.15.45:0/3507189790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800cc490 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 -- 172.21.15.45:0/3507189790 >> 172.21.15.45:0/3507189790 conn(0x7f3180072290 msgr2=0x7f31800726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 -- 172.21.15.45:0/3507189790 shutdown_connections 2024-04-25T10:23:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.089+0000 7f3185000640 1 -- 172.21.15.45:0/3507189790 wait complete. 2024-04-25T10:23:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 Processor -- start 2024-04-25T10:23:59.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 -- start start 2024-04-25T10:23:59.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31800d07b0 0x7f3180153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318015f760 0x7f31800aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31800cefd0 con 0x7f318015bca0 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31800d0f00 con 0x7f31800d07b0 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31800cf150 con 0x7f318015f760 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46340/0 (socket says 172.21.15.45:46340) 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 -- 172.21.15.45:0/3799103134 learned_addr learned my addr 172.21.15.45:0/3799103134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f317de00640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31800d07b0 0x7f3180153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f317e800640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318015f760 0x7f31800aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 -- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318015f760 msgr2=0x7f31800aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318015f760 0x7f31800aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 -- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31800d07b0 msgr2=0x7f3180153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31800d07b0 0x7f3180153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 -- 172.21.15.45:0/3799103134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180154280 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3175400640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800ae620 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f316800c760 tx=0x7f316800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3176a00640 1 -- 172.21.15.45:0/3799103134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168016020 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3176a00640 1 -- 172.21.15.45:0/3799103134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3168006d80 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3180165890 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3176a00640 1 -- 172.21.15.45:0/3799103134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168003bc0 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.093+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3180165e50 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3176a00640 1 -- 172.21.15.45:0/3799103134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f316800a7f0 con 0x7f318015bca0 2024-04-25T10:23:59.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3176a00640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f315c066f90 0x7f315c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3176a00640 1 -- 172.21.15.45:0/3799103134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3168084ba0 con 0x7f318015bca0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317de00640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f315c066f90 0x7f315c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 Processor -- start 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 -- start start 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317de00640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f315c066f90 0x7f315c069450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f31700ae000 tx=0x7f31700a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4aaddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a63b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a64480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c48b0ff0 con 0x5629c4a640a0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c4839150 con 0x5629c4aae2f0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629c48ad090 con 0x5629c4aad9d0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a64480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a64480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46342/0 (socket says 172.21.15.45:46342) 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 -- 172.21.15.45:0/1450016881 learned_addr learned my addr 172.21.15.45:0/1450016881 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317de00640 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4aaddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f3175400640 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a63b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 -- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 msgr2=0x5629c4aaddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4aaddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 -- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 msgr2=0x5629c4a63b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a63b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.097+0000 7f317e800640 1 -- 172.21.15.45:0/1450016881 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c4a470d0 con 0x5629c4a640a0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317e800640 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a64480 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3178009930 tx=0x7f3178009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cf7e170580f32b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f316ea00640 1 -- 172.21.15.45:0/1450016881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178019020 con 0x5629c4a640a0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f316ea00640 1 -- 172.21.15.45:0/1450016881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178004230 con 0x5629c4a640a0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f316ea00640 1 -- 172.21.15.45:0/1450016881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317800c070 con 0x5629c4a640a0 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- 172.21.15.45:0/1450016881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 msgr2=0x5629c4a64480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a64480 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3178009930 tx=0x7f3178009df0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- 172.21.15.45:0/1450016881 shutdown_connections 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a64480 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a63b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- 172.21.15.45:0/1450016881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4aaddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- 172.21.15.45:0/1450016881 >> 172.21.15.45:0/1450016881 conn(0x5629c4a60020 msgr2=0x5629c4a62440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- 172.21.15.45:0/1450016881 shutdown_connections 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- 172.21.15.45:0/1450016881 wait complete. 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- start start 2024-04-25T10:23:59.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 Processor -- start 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4a92d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c4875250 con 0x5629c4a640a0 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c486b690 con 0x5629c4aae2f0 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629c4839a40 con 0x5629c4aad9d0 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3188476000 10 client.0 _ll_register_callbacks cb 0x5629c4a4ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a927f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46356/0 (socket says 172.21.15.45:46356) 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 -- 172.21.15.45:0/3818688803 learned_addr learned my addr 172.21.15.45:0/3818688803 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317e800640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3175400640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4a92d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 -- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 msgr2=0x5629c4a92d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4a92d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 -- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 msgr2=0x5629c4a93270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a93270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 -- 172.21.15.45:0/3818688803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629c495e1c0 con 0x5629c4a640a0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3175400640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4a92d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f317de00640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a927f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f317001c530 tx=0x7f317001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269cc37a61589a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.101+0000 7f3163400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 1 -- 172.21.15.45:0/3818688803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700cd060 con 0x5629c4a640a0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 1 -- 172.21.15.45:0/3818688803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31700c08a0 con 0x5629c4a640a0 2024-04-25T10:23:59.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3178004230 con 0x5629c4a640a0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 1 -- 172.21.15.45:0/3818688803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31700d3050 con 0x5629c4a640a0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 10 client.7116 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5629c495a170 con 0x5629c4a640a0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 1 -- 172.21.15.45:0/3818688803 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178004230 con 0x5629c4a640a0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 1 -- 172.21.15.45:0/3818688803 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f31700d6020 con 0x5629c4a640a0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 10 client.7116 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 10 client.7116 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 20 client.7116 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4abdc80 0x5629c4ac0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 4 client.7116 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629c484cc60 con 0x5629c4abdc80 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3175400640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4abdc80 0x5629c4ac0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3175400640 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4abdc80 0x5629c4ac0070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43b73b22566e13a server_cookie=f7db22eac83a294e in_seq=0 out_seq=0 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 10 client.7116 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 1 -- 172.21.15.45:0/3818688803 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629c484cc60 con 0x5629c4abdc80 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 10 client.7116 handle_command_reply: tid=0 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 client.7116 shutdown 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 msgr2=0x5629c4a927f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a927f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f317001c530 tx=0x7f317001ad30 comp rx=0 tx=0).stop 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 shutdown_connections 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4abdc80 0x5629c4ac0070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4aae2f0 0x5629c4a93270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aad9d0 0x5629c4a92d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 --2- 172.21.15.45:0/3818688803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4a640a0 0x5629c4a927f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3163400640 0 client.7116 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 >> 172.21.15.45:0/3818688803 conn(0x5629c4a60020 msgr2=0x5629c4a62ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 shutdown_connections 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 1 -- 172.21.15.45:0/3818688803 wait complete. 2024-04-25T10:23:59.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.105+0000 7f3188476000 20 client.7116 trim_cache size 0 max 16384 2024-04-25T10:23:59.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 Processor -- start 2024-04-25T10:23:59.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 -- start start 2024-04-25T10:23:59.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4b0a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4b0a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4b0ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3180202e70 con 0x5629c4aad9d0 2024-04-25T10:23:59.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31800c9d50 con 0x5629c4a640a0 2024-04-25T10:23:59.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31800c9150 con 0x5629c4aae2f0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4b0a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4b0a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46372/0 (socket says 172.21.15.45:46372) 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 -- 172.21.15.45:0/710730081 learned_addr learned my addr 172.21.15.45:0/710730081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f317e800640 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4b0ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f317de00640 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4b0a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 -- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 msgr2=0x5629c4b0ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4b0ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 -- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 msgr2=0x5629c4b0a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4b0a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 -- 172.21.15.45:0/710730081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c48f7de0 con 0x5629c4aad9d0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f3175400640 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4b0a630 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3168055480 tx=0x7f3168005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf851fc07be01f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f316e000640 1 -- 172.21.15.45:0/710730081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168080030 con 0x5629c4aad9d0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f316e000640 1 -- 172.21.15.45:0/710730081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31680537f0 con 0x5629c4aad9d0 2024-04-25T10:23:59.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.121+0000 7f316e000640 1 -- 172.21.15.45:0/710730081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168051b60 con 0x5629c4aad9d0 2024-04-25T10:23:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/710730081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 msgr2=0x5629c4b0a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4b0a630 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3168055480 tx=0x7f3168005040 comp rx=0 tx=0).stop 2024-04-25T10:23:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/710730081 shutdown_connections 2024-04-25T10:23:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4b0ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4b0a630 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- 172.21.15.45:0/710730081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4b0a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/710730081 >> 172.21.15.45:0/710730081 conn(0x5629c4a60020 msgr2=0x5629c4abafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/710730081 shutdown_connections 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/710730081 wait complete. 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- start start 2024-04-25T10:23:59.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 Processor -- start 2024-04-25T10:23:59.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4ac0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4ac08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4ac0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c484be00 con 0x5629c4aad9d0 2024-04-25T10:23:59.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31800c9750 con 0x5629c4a640a0 2024-04-25T10:23:59.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629c48b0300 con 0x5629c4aae2f0 2024-04-25T10:23:59.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 10 client.0 _ll_register_callbacks cb 0x5629c4a2f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:59.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4ac08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4ac08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46386/0 (socket says 172.21.15.45:46386) 2024-04-25T10:23:59.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 -- 172.21.15.45:0/2650233217 learned_addr learned my addr 172.21.15.45:0/2650233217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f317e800640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4ac0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f317de00640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4ac0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 -- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 msgr2=0x5629c4ac0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4ac0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 -- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 msgr2=0x5629c4ac0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4ac0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 -- 172.21.15.45:0/2650233217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629c48528e0 con 0x5629c4aad9d0 2024-04-25T10:23:59.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f317de00640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4ac0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:59.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3175400640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4ac08c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f31680551f0 tx=0x7f3170004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc767367374e72fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3163400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:59.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3163400640 1 -- 172.21.15.45:0/2650233217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168080030 con 0x5629c4aad9d0 2024-04-25T10:23:59.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3163400640 1 -- 172.21.15.45:0/2650233217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f316806e040 con 0x5629c4aad9d0 2024-04-25T10:23:59.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31680537f0 con 0x5629c4aad9d0 2024-04-25T10:23:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3163400640 1 -- 172.21.15.45:0/2650233217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31680831a0 con 0x5629c4aad9d0 2024-04-25T10:23:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 10 client.7122 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5629c490a8f0 con 0x5629c4aad9d0 2024-04-25T10:23:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3163400640 1 -- 172.21.15.45:0/2650233217 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31680537f0 con 0x5629c4aad9d0 2024-04-25T10:23:59.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3163400640 1 -- 172.21.15.45:0/2650233217 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3168054940 con 0x5629c4aad9d0 2024-04-25T10:23:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 10 client.7122 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 10 client.7122 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 20 client.7122 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4ac17f0 0x5629c4b15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 4 client.7122 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:23:59.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.125+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5629c48be530 con 0x5629c4ac17f0 2024-04-25T10:23:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f317de00640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4ac17f0 0x5629c4b15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f317de00640 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4ac17f0 0x5629c4b15f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567578ab1ab5c1b6 server_cookie=796db80d3ee7e134 in_seq=0 out_seq=0 2024-04-25T10:23:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3163400640 10 client.7122 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3163400640 1 -- 172.21.15.45:0/2650233217 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5629c48be530 con 0x5629c4ac17f0 2024-04-25T10:23:59.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3163400640 10 client.7122 handle_command_reply: tid=0 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 client.7122 shutdown 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 msgr2=0x5629c4ac08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4ac08c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f31680551f0 tx=0x7f3170004740 comp rx=0 tx=0).stop 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 shutdown_connections 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x5629c4ac17f0 0x5629c4b15f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5629c4aae2f0 0x5629c4ac0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5629c4aad9d0 0x5629c4ac08c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 --2- 172.21.15.45:0/2650233217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5629c4a640a0 0x5629c4ac0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3163400640 0 client.7122 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 >> 172.21.15.45:0/2650233217 conn(0x5629c4a60020 msgr2=0x5629c4aac0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 shutdown_connections 2024-04-25T10:23:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 1 -- 172.21.15.45:0/2650233217 wait complete. 2024-04-25T10:23:59.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3188476000 20 client.7122 trim_cache size 0 max 16384 2024-04-25T10:23:59.146 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:23:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:23:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:23:59.147 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:23:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f315c066f90 msgr2=0x7f315c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f315c066f90 0x7f315c069450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f31700ae000 tx=0x7f31700a7000 comp rx=0 tx=0).stop 2024-04-25T10:23:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 msgr2=0x7f31800ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800ae620 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f316800c760 tx=0x7f316800cc20 comp rx=0 tx=0).stop 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 shutdown_connections 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f315c066f90 0x7f315c069450 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f318015f760 0x7f31800aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f318015bca0 0x7f31800ae620 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 --2- 172.21.15.45:0/3799103134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31800d07b0 0x7f3180153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 >> 172.21.15.45:0/3799103134 conn(0x7f3180072290 msgr2=0x7f31800b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 shutdown_connections 2024-04-25T10:23:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.129+0000 7f3185000640 1 -- 172.21.15.45:0/3799103134 wait complete. 2024-04-25T10:23:59.152 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:23:59.152 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:23:59.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:23:59.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:59.265 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:23:59.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 Processor -- start 2024-04-25T10:23:59.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 -- start start 2024-04-25T10:23:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 0x7f3994162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39940670b0 0x7f39940b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399415f0b0 con 0x7f39940b5480 2024-04-25T10:23:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3994161800 con 0x7f39940670b0 2024-04-25T10:23:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399c800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b3b20 con 0x7f399415dcc0 2024-04-25T10:23:59.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3999400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f39940b5480 0x7f39940669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40162/0 (socket says 172.21.15.45:40162) 2024-04-25T10:23:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3999400640 1 -- 172.21.15.45:0/4243984231 learned_addr learned my addr 172.21.15.45:0/4243984231 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:23:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3999e00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 0x7f3994162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f399a800640 1 --2- 172.21.15.45:0/4243984231 >> v2:172.21.15.117:3300/0 conn(0x7f39940670b0 0x7f39940b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519432354 0 0) 0x7f399415f0b0 con 0x7f39940b5480 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f397c003530 con 0x7f39940b5480 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695278083 0 0) 0x7f397c003530 con 0x7f39940b5480 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 msgr2=0x7f3994162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 0x7f3994162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 >> v2:172.21.15.117:3300/0 conn(0x7f39940670b0 msgr2=0x7f39940b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 --2- 172.21.15.45:0/4243984231 >> v2:172.21.15.117:3300/0 conn(0x7f39940670b0 0x7f39940b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940b3e00 con 0x7f39940b5480 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f3984001680 con 0x7f39940b5480 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 >> v1:172.21.15.45:6789/0 conn(0x7f39940b5480 legacy=0x7f39940669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f397c0039c0 0x7f397c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f397c004810 0x7f397c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 0x7f397c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994161800 con 0x7f397c0039c0 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b3b20 con 0x7f397c004810 2024-04-25T10:23:59.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.293+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c00b9d0 con 0x7f399415dcc0 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f397c0039c0 0x7f397c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999400640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f397c004810 0x7f397c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f399a800640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 0x7f397c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 -- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 msgr2=0x7f397c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f399415dcc0 0x7f397c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 -- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f397c004810 msgr2=0x7f397c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f397c004810 0x7f397c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 -- 172.21.15.45:0/4243984231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c003530 con 0x7f397c0039c0 2024-04-25T10:23:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3999e00640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f397c0039c0 0x7f397c003de0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b10 tx=0x7f398801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754e4662f6a6c230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880b2070 con 0x7f397c0039c0 2024-04-25T10:23:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880a5550 con 0x7f397c0039c0 2024-04-25T10:23:59.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.297+0000 7f3998a00640 1 -- 172.21.15.45:0/4243984231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880096b0 con 0x7f397c0039c0 2024-04-25T10:23:59.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- 172.21.15.45:0/4243984231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f397c0039c0 msgr2=0x7f397c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 --2- 172.21.15.45:0/4243984231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f397c0039c0 0x7f397c003de0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b10 tx=0x7f398801ca40 comp rx=0 tx=0).stop 2024-04-25T10:23:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399a800640 1 -- 172.21.15.45:0/4243984231 reap_dead start 2024-04-25T10:23:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- 172.21.15.45:0/4243984231 shutdown_connections 2024-04-25T10:23:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- 172.21.15.45:0/4243984231 >> 172.21.15.45:0/4243984231 conn(0x7f399406ac90 msgr2=0x7f399406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- 172.21.15.45:0/4243984231 shutdown_connections 2024-04-25T10:23:59.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- 172.21.15.45:0/4243984231 wait complete. 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 Processor -- start 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- start start 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f397c0039c0 0x7f399414e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39940cd280 0x7f39940b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 0x7f399415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b3900 con 0x7f399415c310 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b3780 con 0x7f39940cd280 2024-04-25T10:23:59.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399c800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940d0d40 con 0x7f397c0039c0 2024-04-25T10:23:59.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 0x7f399415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 0x7f399415c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46394/0 (socket says 172.21.15.45:46394) 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f399a800640 1 -- 172.21.15.45:0/4222292312 learned_addr learned my addr 172.21.15.45:0/4222292312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f3999400640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39940cd280 0x7f39940b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.301+0000 7f3999e00640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f397c0039c0 0x7f399414e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399a800640 1 -- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f397c0039c0 msgr2=0x7f399414e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399a800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f397c0039c0 0x7f399414e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399a800640 1 -- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39940cd280 msgr2=0x7f39940b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399a800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39940cd280 0x7f39940b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399a800640 1 -- 172.21.15.45:0/4222292312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940c59b0 con 0x7f399415c310 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f3999400640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39940cd280 0x7f39940b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399a800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 0x7f399415c790 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f39940b2430 tx=0x7f399000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f398e000640 1 -- 172.21.15.45:0/4222292312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990015430 con 0x7f399415c310 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f398e000640 1 -- 172.21.15.45:0/4222292312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399001d070 con 0x7f399415c310 2024-04-25T10:23:59.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f398e000640 1 -- 172.21.15.45:0/4222292312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990015810 con 0x7f399415c310 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940b1410 con 0x7f399415c310 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39940cdb20 con 0x7f399415c310 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f398e000640 1 -- 172.21.15.45:0/4222292312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39900159b0 con 0x7f399415c310 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f398e000640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3978066f90 0x7f3978069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.305+0000 7f398e000640 1 -- 172.21.15.45:0/4222292312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3990085a40 con 0x7f399415c310 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3999e00640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3978066f90 0x7f3978069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 Processor -- start 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- start start 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3999e00640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3978066f90 0x7f3978069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f39880af000 tx=0x7f39880046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca4889ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48859850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca4885a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4873c1f0 con 0x55ca48859d90 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca486a2e90 con 0x55ca4889ea40 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca4862af50 con 0x55ca4889f360 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca4885a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca4885a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46400/0 (socket says 172.21.15.45:46400) 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 -- 172.21.15.45:0/2369863412 learned_addr learned my addr 172.21.15.45:0/2369863412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3999400640 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48859850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3999e00640 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca4889ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 -- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 msgr2=0x55ca48859850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48859850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 -- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 msgr2=0x55ca4889ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca4889ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 -- 172.21.15.45:0/2369863412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4867a8b0 con 0x55ca48859d90 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399a800640 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca4885a170 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f399005e430 tx=0x7f3990055ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83ddd842a40441a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3982a00640 1 -- 172.21.15.45:0/2369863412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399005a3a0 con 0x55ca48859d90 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3982a00640 1 -- 172.21.15.45:0/2369863412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399005e960 con 0x55ca48859d90 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f3982a00640 1 -- 172.21.15.45:0/2369863412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399005a070 con 0x55ca48859d90 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- 172.21.15.45:0/2369863412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 msgr2=0x55ca4885a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca4885a170 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f399005e430 tx=0x7f3990055ec0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- 172.21.15.45:0/2369863412 shutdown_connections 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca4885a170 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48859850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 --2- 172.21.15.45:0/2369863412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca4889ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- 172.21.15.45:0/2369863412 >> 172.21.15.45:0/2369863412 conn(0x55ca48855d40 msgr2=0x55ca48858130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- 172.21.15.45:0/2369863412 shutdown_connections 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- 172.21.15.45:0/2369863412 wait complete. 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.309+0000 7f399fc26000 1 -- start start 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 Processor -- start 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca48859d90 0x55ca488a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 0x55ca488a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca4885fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4865d540 con 0x55ca4889ea40 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4869ef30 con 0x55ca48859d90 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca48664e50 con 0x55ca4889f360 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 10 client.0 _ll_register_callbacks cb 0x55ca48757180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 0x55ca488a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 0x55ca488a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46412/0 (socket says 172.21.15.45:46412) 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 -- 172.21.15.45:0/2284726835 learned_addr learned my addr 172.21.15.45:0/2284726835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399a800640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca4885fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999e00640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca48859d90 0x55ca488a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 -- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 msgr2=0x55ca4885fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca4885fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 -- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca48859d90 msgr2=0x55ca488a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca48859d90 0x55ca488a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 -- 172.21.15.45:0/2284726835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca4862b1a0 con 0x55ca4889ea40 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999400640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 0x55ca488a6700 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3984004ea0 tx=0x7f3984004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae5b0d9efe76981 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 1 -- 172.21.15.45:0/2284726835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398401a070 con 0x55ca4889ea40 2024-04-25T10:23:59.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 1 -- 172.21.15.45:0/2284726835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f398401b920 con 0x55ca4889ea40 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f399005e960 con 0x55ca4889ea40 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 1 -- 172.21.15.45:0/2284726835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3984026050 con 0x55ca4889ea40 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 10 client.7134 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55ca48725840 con 0x55ca4889ea40 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 1 -- 172.21.15.45:0/2284726835 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399005e960 con 0x55ca4889ea40 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 1 -- 172.21.15.45:0/2284726835 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3984029020 con 0x55ca4889ea40 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 10 client.7134 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 10 client.7134 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 20 client.7134 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca48860a80 0x55ca48860e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 4 client.7134 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca4863ebe0 con 0x55ca48860a80 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999e00640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca48860a80 0x55ca48860e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3999e00640 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca48860a80 0x55ca48860e60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231b40ac17f95d07 server_cookie=54dc4e7043d39baa in_seq=0 out_seq=0 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.313+0000 7f3977400640 10 client.7134 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f3977400640 1 -- 172.21.15.45:0/2284726835 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca4863ebe0 con 0x55ca48860a80 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f3977400640 10 client.7134 handle_command_reply: tid=0 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 client.7134 shutdown 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 msgr2=0x55ca488a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 0x55ca488a6700 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3984004ea0 tx=0x7f3984004f90 comp rx=0 tx=0).stop 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 shutdown_connections 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca48860a80 0x55ca48860e60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca4885fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca4889ea40 0x55ca488a6700 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 --2- 172.21.15.45:0/2284726835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca48859d90 0x55ca488a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f3977400640 0 client.7134 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 >> 172.21.15.45:0/2284726835 conn(0x55ca48855d40 msgr2=0x55ca488589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 shutdown_connections 2024-04-25T10:23:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 1 -- 172.21.15.45:0/2284726835 wait complete. 2024-04-25T10:23:59.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.317+0000 7f399fc26000 20 client.7134 trim_cache size 0 max 16384 2024-04-25T10:23:59.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.329+0000 7f399fc26000 1 Processor -- start 2024-04-25T10:23:59.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.329+0000 7f399fc26000 1 -- start start 2024-04-25T10:23:59.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca48858130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48858670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994203750 con 0x55ca48859d90 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994202990 con 0x55ca4889ea40 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c6c20 con 0x55ca4889f360 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46414/0 (socket says 172.21.15.45:46414) 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 -- 172.21.15.45:0/45091911 learned_addr learned my addr 172.21.15.45:0/45091911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 -- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 msgr2=0x55ca48858670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48858670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 -- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 msgr2=0x55ca48858130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999400640 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca48858130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca48858130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 -- 172.21.15.45:0/45091911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4885cad0 con 0x55ca48859d90 2024-04-25T10:23:59.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999400640 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca48858130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:23:59.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488596b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b10 tx=0x7f3988004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae66196aa89de7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3982000640 1 -- 172.21.15.45:0/45091911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880b2040 con 0x55ca48859d90 2024-04-25T10:23:59.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3982000640 1 -- 172.21.15.45:0/45091911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880ce030 con 0x55ca48859d90 2024-04-25T10:23:59.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3982000640 1 -- 172.21.15.45:0/45091911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880a5d20 con 0x55ca48859d90 2024-04-25T10:23:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- 172.21.15.45:0/45091911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 msgr2=0x55ca488596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488596b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b10 tx=0x7f3988004740 comp rx=0 tx=0).stop 2024-04-25T10:23:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- 172.21.15.45:0/45091911 shutdown_connections 2024-04-25T10:23:59.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889f360 0x55ca48858670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889ea40 0x55ca48858130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- 172.21.15.45:0/45091911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488596b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- 172.21.15.45:0/45091911 >> 172.21.15.45:0/45091911 conn(0x55ca48855d40 msgr2=0x55ca4889d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- 172.21.15.45:0/45091911 shutdown_connections 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- 172.21.15.45:0/45091911 wait complete. 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- start start 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 Processor -- start 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889ea40 0x55ca488ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889f360 0x55ca488a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4869ef30 con 0x55ca48859d90 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994202490 con 0x55ca4889f360 2024-04-25T10:23:59.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca48666720 con 0x55ca4889ea40 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f399fc26000 10 client.0 _ll_register_callbacks cb 0x55ca48665a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.333+0000 7f3999e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46430/0 (socket says 172.21.15.45:46430) 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 -- 172.21.15.45:0/2398042550 learned_addr learned my addr 172.21.15.45:0/2398042550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399a800640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889f360 0x55ca488a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999400640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889ea40 0x55ca488ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 -- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889ea40 msgr2=0x55ca488ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889ea40 0x55ca488ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 -- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889f360 msgr2=0x55ca488a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889f360 0x55ca488a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 -- 172.21.15.45:0/2398042550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca486fca10 con 0x55ca48859d90 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399a800640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889f360 0x55ca488a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999e00640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488ff4b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3988000c10 tx=0x7f39880197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacc67f7784c706e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 1 -- 172.21.15.45:0/2398042550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880d7020 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 1 -- 172.21.15.45:0/2398042550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f39880a5a60 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39880ce030 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 1 -- 172.21.15.45:0/2398042550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39880e1050 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 10 client.7140 fetch_fsmap learned FSMap version 56 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55ca48637b00 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 1 -- 172.21.15.45:0/2398042550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880ce030 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 1 -- 172.21.15.45:0/2398042550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39880e4020 con 0x55ca48859d90 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 10 client.7140 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 10 client.7140 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 20 client.7140 populate_metadata read hostname 'smithi045' 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca488a4890 0x55ca488a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 4 client.7140 mds_command: new command op to 6888 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:23:59.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 --> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55ca486b03d0 con 0x55ca488a4890 2024-04-25T10:23:59.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999400640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca488a4890 0x55ca488a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3999400640 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca488a4890 0x55ca488a4c70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca488f441a4c95a3 server_cookie=776bed713fb57a0d in_seq=0 out_seq=0 2024-04-25T10:23:59.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 10 client.7140 ms_handle_connect on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 1 -- 172.21.15.45:0/2398042550 <== mds.0 v2:172.21.15.45:6833/3661919331 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17935 (crc 0 0 0) 0x55ca486b03d0 con 0x55ca488a4890 2024-04-25T10:23:59.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 10 client.7140 handle_command_reply: tid=0 2024-04-25T10:23:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 client.7140 shutdown 2024-04-25T10:23:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 msgr2=0x55ca488ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488ff4b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3988000c10 tx=0x7f39880197e0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 shutdown_connections 2024-04-25T10:23:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:6833/3661919331,v1:172.21.15.45:6835/3661919331] conn(0x55ca488a4890 0x55ca488a4c70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca4889f360 0x55ca488a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ca4889ea40 0x55ca488ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f3977400640 0 client.7140 ms_handle_reset on v2:172.21.15.45:6833/3661919331 2024-04-25T10:23:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 --2- 172.21.15.45:0/2398042550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ca48859d90 0x55ca488ff4b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 >> 172.21.15.45:0/2398042550 conn(0x55ca48855d40 msgr2=0x55ca4889d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 shutdown_connections 2024-04-25T10:23:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 1 -- 172.21.15.45:0/2398042550 wait complete. 2024-04-25T10:23:59.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.337+0000 7f399fc26000 20 client.7140 trim_cache size 0 max 16384 2024-04-25T10:23:59.350 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:23:59.350 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:23:59.351 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3281, 2024-04-25T10:23:59.351 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 3274, 2024-04-25T10:23:59.351 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 917404, 2024-04-25T10:23:59.351 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 25368608, 2024-04-25T10:23:59.351 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 16, 2024-04-25T10:23:59.351 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 6, 2024-04-25T10:23:59.352 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.334359372, 2024-04-25T10:23:59.352 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.144801587, 2024-04-25T10:23:59.352 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.080513266, 2024-04-25T10:23:59.352 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.083985615, 2024-04-25T10:23:59.352 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:23:59.352 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3273, 2024-04-25T10:23:59.353 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.690988745, 2024-04-25T10:23:59.353 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000211117 2024-04-25T10:23:59.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.353 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:23:59.353 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 10, 2024-04-25T10:23:59.353 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000001516, 2024-04-25T10:23:59.354 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000151 2024-04-25T10:23:59.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.354 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 118976, 2024-04-25T10:23:59.354 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 25888 2024-04-25T10:23:59.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.354 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:23:59.355 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 31280, 2024-04-25T10:23:59.355 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 52150, 2024-04-25T10:23:59.355 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 11111183, 2024-04-25T10:23:59.355 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 57388077, 2024-04-25T10:23:59.355 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 8, 2024-04-25T10:23:59.355 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:23:59.356 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 4.875258767, 2024-04-25T10:23:59.356 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 2.913401140, 2024-04-25T10:23:59.356 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 2.389563746, 2024-04-25T10:23:59.356 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.075887974, 2024-04-25T10:23:59.356 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:23:59.356 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 52149, 2024-04-25T10:23:59.357 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.837770590, 2024-04-25T10:23:59.357 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000035240 2024-04-25T10:23:59.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.357 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:23:59.357 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 72411, 2024-04-25T10:23:59.357 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.017222921, 2024-04-25T10:23:59.358 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000237 2024-04-25T10:23:59.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.358 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T10:23:59.358 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 119120 2024-04-25T10:23:59.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.359 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:23:59.359 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3923, 2024-04-25T10:23:59.359 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 3923, 2024-04-25T10:23:59.360 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1362675, 2024-04-25T10:23:59.360 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 37756217, 2024-04-25T10:23:59.360 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-25T10:23:59.360 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 4, 2024-04-25T10:23:59.360 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.622394030, 2024-04-25T10:23:59.361 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.198064111, 2024-04-25T10:23:59.361 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.088527274, 2024-04-25T10:23:59.361 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.306837759, 2024-04-25T10:23:59.361 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:23:59.361 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3922, 2024-04-25T10:23:59.362 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.818759866, 2024-04-25T10:23:59.362 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000208760 2024-04-25T10:23:59.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.362 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:23:59.362 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.362 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.363 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.363 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-04-25T10:23:59.363 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T10:23:59.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.363 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T10:23:59.364 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T10:23:59.364 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T10:23:59.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.364 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T10:23:59.364 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:23:59.364 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:23:59.365 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 5312, 2024-04-25T10:23:59.365 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 13.848668179, 2024-04-25T10:23:59.365 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002607053 2024-04-25T10:23:59.365 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.365 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T10:23:59.366 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T10:23:59.366 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T10:23:59.366 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2954, 2024-04-25T10:23:59.366 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.918661824, 2024-04-25T10:23:59.366 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000310989 2024-04-25T10:23:59.366 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.367 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T10:23:59.367 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 26273, 2024-04-25T10:23:59.367 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 26273, 2024-04-25T10:23:59.367 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T10:23:59.367 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 26273, 2024-04-25T10:23:59.368 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 241.836952543, 2024-04-25T10:23:59.368 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.009204771 2024-04-25T10:23:59.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.368 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T10:23:59.368 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 12, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 17755, 2024-04-25T10:23:59.369 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 0, 2024-04-25T10:23:59.370 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 17755, 2024-04-25T10:23:59.370 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-25T10:23:59.370 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 17755, 2024-04-25T10:23:59.370 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-25T10:23:59.370 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 2, 2024-04-25T10:23:59.371 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 2, 2024-04-25T10:23:59.371 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 2, 2024-04-25T10:23:59.371 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 49067, 2024-04-25T10:23:59.371 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 48520, 2024-04-25T10:23:59.371 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 156, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-25T10:23:59.372 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-25T10:23:59.373 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-25T10:23:59.373 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-25T10:23:59.373 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T10:23:59.373 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 1624, 2024-04-25T10:23:59.373 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 1624, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 11, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T10:23:59.374 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 1613, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 1437, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 4068, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 4546, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 30, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 14092 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T10:23:59.375 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 4141, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 3613, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 2244, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:23:59.376 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:23:59.377 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 30355, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 30355, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T10:23:59.378 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 46, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 46, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 4, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 96391296, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4166, 2024-04-25T10:23:59.379 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 79.751110411, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.019143329 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 14, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 2015, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 2001, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 13, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 13, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 17755, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 19756, 2024-04-25T10:23:59.380 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 2001, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 2, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 3701, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 3699, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 95952, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215340 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 29833, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 26273, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 5, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-25T10:23:59.381 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2000, 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.341677269, 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000670838 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 28, 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003748177, 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000133863 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.382 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6000, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 7.046492249, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001174415 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7988, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 6.668837352, 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000834856 2024-04-25T10:23:59.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.384 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T10:23:59.385 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000062156, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000062156 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.003260683, 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003260683 2024-04-25T10:23:59.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.006099852, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.006099852 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T10:23:59.387 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8, 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.093919533, 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.011739941 2024-04-25T10:23:59.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2244, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 210.487101911, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.093799956 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T10:23:59.389 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.010646796, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.010646796 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000235401, 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000235401 2024-04-25T10:23:59.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T10:23:59.392 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.393 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 8000, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 16.174871164, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002021858 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 1, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 0, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 18919, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 18919, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 83, 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T10:23:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:23:59.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:23:59.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T10:23:59.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 250419, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 86, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:23:59.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 5608, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 81, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 13371316, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 660335, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.400 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 9637, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 92803241, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 37, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 9637, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 9637, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 37.575441277, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003899080 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T10:23:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 9637, 2024-04-25T10:23:59.402 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 9732 2024-04-25T10:23:59.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.402 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 9637, 2024-04-25T10:23:59.402 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 1673, 2024-04-25T10:23:59.402 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 7964, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 11, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 48, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 4213, 2024-04-25T10:23:59.403 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 29, 2024-04-25T10:23:59.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T10:23:59.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T10:23:59.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 1613, 2024-04-25T10:23:59.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T10:23:59.404 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 2033, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 1625, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 22, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T10:23:59.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:23:59.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T10:23:59.407 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T10:23:59.408 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 81, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 5, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 8, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 8, 2024-04-25T10:23:59.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 107, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 31 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T10:23:59.410 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 32, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 3613, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 3613, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:23:59.411 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 38479, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 9922346, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 38479, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 38479, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9922346, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 9637, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 93181058, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 9637, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 93181058, 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:59.413 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T10:23:59.414 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 9637, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 9637, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 9637, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9637, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:59.415 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 3613, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 377817, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 3613, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 47, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 377817, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x55d6dc22ca00": { 2024-04-25T10:23:59.416 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 30355, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 92196992, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 30355, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4166, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 92196992, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T10:23:59.417 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T10:23:59.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:23:59.418 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:23:59.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3978066f90 msgr2=0x7f3978069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3978066f90 0x7f3978069450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f39880af000 tx=0x7f39880046d0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 msgr2=0x7f399415c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 0x7f399415c790 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f39940b2430 tx=0x7f399000f580 comp rx=0 tx=0).stop 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 shutdown_connections 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3978066f90 0x7f3978069450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f399415c310 0x7f399415c790 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39940cd280 0x7f39940b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 --2- 172.21.15.45:0/4222292312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f397c0039c0 0x7f399414e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 >> 172.21.15.45:0/4222292312 conn(0x7f399406ac90 msgr2=0x7f399415f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 shutdown_connections 2024-04-25T10:23:59.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.341+0000 7f399c800640 1 -- 172.21.15.45:0/4222292312 wait complete. 2024-04-25T10:23:59.421 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:23:59.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:23:59.530 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:23:59.592 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:23:59.592 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:23:59.593 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:59.602 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:23:59.602 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:23:59.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:23:59.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 Processor -- start 2024-04-25T10:23:59.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 -- start start 2024-04-25T10:23:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fddf40b2fa0 0x7fddf40b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf406df80 0x7fddf406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40af690 con 0x7fddf415ddf0 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf415e1d0 con 0x7fddf40b2fa0 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40ae250 con 0x7fddf406df80 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf1e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fddf415ddf0 0x7fddf40aef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40174/0 (socket says 172.21.15.45:40174) 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf1e00640 1 -- 172.21.15.45:0/218981041 learned_addr learned my addr 172.21.15.45:0/218981041 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fdde9400640 1 --2- 172.21.15.45:0/218981041 >> v2:172.21.15.117:3300/0 conn(0x7fddf40b2fa0 0x7fddf40b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf2800640 1 --2- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf406df80 0x7fddf406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf1400640 1 -- 172.21.15.45:0/218981041 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074072160 0 0) 0x7fddf40af690 con 0x7fddf415ddf0 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fddf1400640 1 -- 172.21.15.45:0/218981041 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd8003530 con 0x7fddf415ddf0 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fdde9400640 1 -- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf406df80 msgr2=0x7fddf406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fdde9400640 1 --2- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf406df80 0x7fddf406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fdde9400640 1 -- 172.21.15.45:0/218981041 >> v1:172.21.15.45:6789/0 conn(0x7fddf415ddf0 legacy=0x7fddf40aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.833+0000 7fdde9400640 1 -- 172.21.15.45:0/218981041 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40b39f0 con 0x7fddf40b2fa0 2024-04-25T10:23:59.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fdde9400640 1 --2- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf40b2fa0 0x7fddf40b3400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0097c0 tx=0x7fdddc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca97850598e1a463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf1400640 1 -- 172.21.15.45:0/218981041 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0b0070 con 0x7fddf40b2fa0 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf1400640 1 -- 172.21.15.45:0/218981041 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc0ab600 con 0x7fddf40b2fa0 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf1400640 1 -- 172.21.15.45:0/218981041 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0ab9d0 con 0x7fddf40b2fa0 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 -- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf40b2fa0 msgr2=0x7fddf40b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 --2- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf40b2fa0 0x7fddf40b3400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0097c0 tx=0x7fdddc01c410 comp rx=0 tx=0).stop 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 -- 172.21.15.45:0/218981041 shutdown_connections 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 --2- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf406df80 0x7fddf406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 --2- 172.21.15.45:0/218981041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf40b2fa0 0x7fddf40b3400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 -- 172.21.15.45:0/218981041 >> 172.21.15.45:0/218981041 conn(0x7fddf40b1ca0 msgr2=0x7fddf40b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 -- 172.21.15.45:0/218981041 shutdown_connections 2024-04-25T10:23:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.837+0000 7fddf9000640 1 -- 172.21.15.45:0/218981041 wait complete. 2024-04-25T10:23:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 Processor -- start 2024-04-25T10:23:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- start start 2024-04-25T10:23:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 0x7fddf4153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf40b2fa0 0x7fddf41603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf415ddf0 0x7fddf4160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf406f520 con 0x7fddf406df80 2024-04-25T10:23:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf406f3a0 con 0x7fddf415ddf0 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf416aa60 con 0x7fddf40b2fa0 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 0x7fddf4153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 0x7fddf4153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46432/0 (socket says 172.21.15.45:46432) 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 -- 172.21.15.45:0/3185481812 learned_addr learned my addr 172.21.15.45:0/3185481812 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf2800640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf415ddf0 0x7fddf4160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fdde9400640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf40b2fa0 0x7fddf41603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 -- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf40b2fa0 msgr2=0x7fddf41603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf40b2fa0 0x7fddf41603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 -- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf415ddf0 msgr2=0x7fddf4160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf415ddf0 0x7fddf4160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 -- 172.21.15.45:0/3185481812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4153890 con 0x7fddf406df80 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf1e00640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 0x7fddf4153aa0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fdde400caa0 tx=0x7fdde400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4017070 con 0x7fddf406df80 2024-04-25T10:23:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde400ac00 con 0x7fddf406df80 2024-04-25T10:23:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4012400 con 0x7fddf406df80 2024-04-25T10:23:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf415ed60 con 0x7fddf406df80 2024-04-25T10:23:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf415f320 con 0x7fddf406df80 2024-04-25T10:23:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.841+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf404ef30 con 0x7fddf406df80 2024-04-25T10:23:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.845+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdde401a070 con 0x7fddf406df80 2024-04-25T10:23:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.845+0000 7fddeaa00640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddd0066f90 0x7fddd0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:23:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.845+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdde4084e00 con 0x7fddf406df80 2024-04-25T10:23:59.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.845+0000 7fdde9400640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddd0066f90 0x7fddd0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:23:59.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.845+0000 7fdde9400640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddd0066f90 0x7fddd0069450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0ad000 tx=0x7fdddc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:23:59.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:23:59.849+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdde40556c0 con 0x7fddf406df80 2024-04-25T10:24:00.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.013+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddf406e480 con 0x7fddf406df80 2024-04-25T10:24:00.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.013+0000 7fddeaa00640 1 -- 172.21.15.45:0/3185481812 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16759 (secure 0 0 0) 0x7fdde4059760 con 0x7fddf406df80 2024-04-25T10:24:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:00.019 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:23:40.632586+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":80,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.0","mappings":[{"from":2,"to":7}]},{"pgid":"11.3d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddd0066f90 msgr2=0x7fddd0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddd0066f90 0x7fddd0069450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0ad000 tx=0x7fdddc008040 comp rx=0 tx=0).stop 2024-04-25T10:24:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 msgr2=0x7fddf4153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 0x7fddf4153aa0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fdde400caa0 tx=0x7fdde400cf60 comp rx=0 tx=0).stop 2024-04-25T10:24:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 shutdown_connections 2024-04-25T10:24:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fddd0066f90 0x7fddd0069450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddf415ddf0 0x7fddf4160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddf40b2fa0 0x7fddf41603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 --2- 172.21.15.45:0/3185481812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf406df80 0x7fddf4153aa0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 >> 172.21.15.45:0/3185481812 conn(0x7fddf40b1ca0 msgr2=0x7fddf415aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 shutdown_connections 2024-04-25T10:24:00.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.017+0000 7fddf9000640 1 -- 172.21.15.45:0/3185481812 wait complete. 2024-04-25T10:24:00.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:22:25.323207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:22:25.891841+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 80, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:00.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 Processor -- start 2024-04-25T10:24:00.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 -- start start 2024-04-25T10:24:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ebc0aef80 0x7f2ebc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc15b080 con 0x7f2ebc070350 2024-04-25T10:24:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc15adb0 con 0x7f2ebc0aef80 2024-04-25T10:24:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc15eb60 con 0x7f2ebc0b31c0 2024-04-25T10:24:00.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ebc0aef80 0x7f2ebc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ebc0aef80 0x7f2ebc15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46442/0 (socket says 172.21.15.45:46442) 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 -- 172.21.15.45:0/2513545772 learned_addr learned my addr 172.21.15.45:0/2513545772 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1c00640 1 --2- 172.21.15.45:0/2513545772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 -- 172.21.15.45:0/2513545772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 msgr2=0x7f2ebc0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 --2- 172.21.15.45:0/2513545772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 -- 172.21.15.45:0/2513545772 >> v1:172.21.15.117:6789/0 conn(0x7f2ebc070350 legacy=0x7f2ebc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.181+0000 7f2ec1200640 1 -- 172.21.15.45:0/2513545772 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc15ee40 con 0x7f2ebc0aef80 2024-04-25T10:24:00.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec1200640 1 --2- 172.21.15.45:0/2513545772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc15e480 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009870 tx=0x7f2eac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f5c083688da608 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ebbe00640 1 -- 172.21.15.45:0/2513545772 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0a7070 con 0x7f2ebc0aef80 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ebbe00640 1 -- 172.21.15.45:0/2513545772 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac0b0070 con 0x7f2ebc0aef80 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ebbe00640 1 -- 172.21.15.45:0/2513545772 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0ab890 con 0x7f2ebc0aef80 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2513545772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc0aef80 msgr2=0x7f2ebc15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2513545772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc15e480 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009870 tx=0x7f2eac01c560 comp rx=0 tx=0).stop 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2513545772 shutdown_connections 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2513545772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2513545772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc15e480 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2513545772 >> 172.21.15.45:0/2513545772 conn(0x7f2ebc0669a0 msgr2=0x7f2ebc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2513545772 shutdown_connections 2024-04-25T10:24:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.185+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2513545772 wait complete. 2024-04-25T10:24:00.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 Processor -- start 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- start start 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 0x7f2ebc159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06c250 con 0x7f2ebc070350 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06c0d0 con 0x7f2ebc0aef80 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc06c3d0 con 0x7f2ebc0b31c0 2024-04-25T10:24:00.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 0x7f2ebc159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 0x7f2ebc159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46450/0 (socket says 172.21.15.45:46450) 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 -- 172.21.15.45:0/2991042904 learned_addr learned my addr 172.21.15.45:0/2991042904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2eb3e00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 -- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 msgr2=0x7f2ebc157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 -- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0aef80 msgr2=0x7f2ebc140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 -- 172.21.15.45:0/2991042904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0adb00 con 0x7f2ebc070350 2024-04-25T10:24:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec1200640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 0x7f2ebc159ea0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2eac01a190 tx=0x7f2eac01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0be5f0 con 0x7f2ebc070350 2024-04-25T10:24:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac0be7b0 con 0x7f2ebc070350 2024-04-25T10:24:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc0ae080 con 0x7f2ebc070350 2024-04-25T10:24:00.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0beba0 con 0x7f2ebc070350 2024-04-25T10:24:00.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc154e80 con 0x7f2ebc070350 2024-04-25T10:24:00.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.189+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc04ef30 con 0x7f2ebc070350 2024-04-25T10:24:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.193+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eac0b43f0 con 0x7f2ebc070350 2024-04-25T10:24:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.193+0000 7f2eb9600640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e98066f60 0x7f2e98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.193+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2eac11fdf0 con 0x7f2ebc070350 2024-04-25T10:24:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.193+0000 7f2eb3e00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e98066f60 0x7f2e98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.193+0000 7f2eb3e00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e98066f60 0x7f2e98069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc143c10 tx=0x7f2ea4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.197+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2eac0c4060 con 0x7f2ebc070350 2024-04-25T10:24:00.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.357+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ebc0b2fa0 con 0x7f2ebc070350 2024-04-25T10:24:00.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.357+0000 7f2eb9600640 1 -- 172.21.15.45:0/2991042904 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2eac016150 con 0x7f2ebc070350 2024-04-25T10:24:00.363 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:24:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e98066f60 msgr2=0x7f2e98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e98066f60 0x7f2e98069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc143c10 tx=0x7f2ea4009040 comp rx=0 tx=0).stop 2024-04-25T10:24:00.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 msgr2=0x7f2ebc159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 0x7f2ebc159ea0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2eac01a190 tx=0x7f2eac01ada0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 shutdown_connections 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2e98066f60 0x7f2e98069420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ebc0b31c0 0x7f2ebc157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ebc0aef80 0x7f2ebc140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 --2- 172.21.15.45:0/2991042904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ebc070350 0x7f2ebc159ea0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 >> 172.21.15.45:0/2991042904 conn(0x7f2ebc0669a0 msgr2=0x7f2ebc15d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 shutdown_connections 2024-04-25T10:24:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.361+0000 7f2ec3c00640 1 -- 172.21.15.45:0/2991042904 wait complete. 2024-04-25T10:24:00.384 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:24:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.521+0000 7fe29da00640 1 Processor -- start 2024-04-25T10:24:00.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.521+0000 7fe29da00640 1 -- start start 2024-04-25T10:24:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe29da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2980aeda0 0x7fe298071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe29da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe298072020 0x7fe298070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe29da00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe298159920 con 0x7fe298159540 2024-04-25T10:24:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe29da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29815c2f0 con 0x7fe2980aeda0 2024-04-25T10:24:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe29da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29815c5d0 con 0x7fe298072020 2024-04-25T10:24:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe296a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe298159540 0x7fe2980ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40178/0 (socket says 172.21.15.45:40178) 2024-04-25T10:24:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe296a00640 1 -- 172.21.15.45:0/2890600753 learned_addr learned my addr 172.21.15.45:0/2890600753 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe28e000640 1 --2- 172.21.15.45:0/2890600753 >> v2:172.21.15.117:3300/0 conn(0x7fe2980aeda0 0x7fe298071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe297400640 1 --2- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe298072020 0x7fe298070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe296000640 1 -- 172.21.15.45:0/2890600753 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855096192 0 0) 0x7fe298159920 con 0x7fe298159540 2024-04-25T10:24:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe296000640 1 -- 172.21.15.45:0/2890600753 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27c003530 con 0x7fe298159540 2024-04-25T10:24:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe28e000640 1 -- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe298072020 msgr2=0x7fe298070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe28e000640 1 --2- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe298072020 0x7fe298070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe28e000640 1 -- 172.21.15.45:0/2890600753 >> v1:172.21.15.45:6789/0 conn(0x7fe298159540 legacy=0x7fe2980ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.525+0000 7fe28e000640 1 -- 172.21.15.45:0/2890600753 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29815c8b0 con 0x7fe2980aeda0 2024-04-25T10:24:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe28e000640 1 --2- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2980aeda0 0x7fe298071ae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097c0 tx=0x7fe28001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=310fad16341afffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe296000640 1 -- 172.21.15.45:0/2890600753 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2800aa5f0 con 0x7fe2980aeda0 2024-04-25T10:24:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe296000640 1 -- 172.21.15.45:0/2890600753 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2800af070 con 0x7fe2980aeda0 2024-04-25T10:24:00.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe296000640 1 -- 172.21.15.45:0/2890600753 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2800aa990 con 0x7fe2980aeda0 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 -- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2980aeda0 msgr2=0x7fe298071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 --2- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2980aeda0 0x7fe298071ae0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097c0 tx=0x7fe28001c470 comp rx=0 tx=0).stop 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 -- 172.21.15.45:0/2890600753 shutdown_connections 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 --2- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe298072020 0x7fe298070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 --2- 172.21.15.45:0/2890600753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2980aeda0 0x7fe298071ae0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 -- 172.21.15.45:0/2890600753 >> 172.21.15.45:0/2890600753 conn(0x7fe2980669a0 msgr2=0x7fe298066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:00.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 -- 172.21.15.45:0/2890600753 shutdown_connections 2024-04-25T10:24:00.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 -- 172.21.15.45:0/2890600753 wait complete. 2024-04-25T10:24:00.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 Processor -- start 2024-04-25T10:24:00.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.529+0000 7fe29da00640 1 -- start start 2024-04-25T10:24:00.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe298072020 0x7fe298152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2980aeda0 0x7fe298151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 0x7fe29814c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2980710a0 con 0x7fe298159540 2024-04-25T10:24:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298072430 con 0x7fe298072020 2024-04-25T10:24:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe298071220 con 0x7fe2980aeda0 2024-04-25T10:24:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 0x7fe29814c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 0x7fe29814c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46466/0 (socket says 172.21.15.45:46466) 2024-04-25T10:24:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 -- 172.21.15.45:0/3809933918 learned_addr learned my addr 172.21.15.45:0/3809933918 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe28e000640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2980aeda0 0x7fe298151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe296a00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe298072020 0x7fe298152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 -- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2980aeda0 msgr2=0x7fe298151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2980aeda0 0x7fe298151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 -- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe298072020 msgr2=0x7fe298152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe298072020 0x7fe298152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 -- 172.21.15.45:0/3809933918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29814de70 con 0x7fe298159540 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe297400640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 0x7fe29814c500 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe29000efc0 tx=0x7fe29000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290018020 con 0x7fe298159540 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe290014480 con 0x7fe298159540 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290014870 con 0x7fe298159540 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2981510d0 con 0x7fe298159540 2024-04-25T10:24:00.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe298151690 con 0x7fe298159540 2024-04-25T10:24:00.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29804ef30 con 0x7fe298159540 2024-04-25T10:24:00.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.533+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe29001b020 con 0x7fe298159540 2024-04-25T10:24:00.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.537+0000 7fe28f400640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe274066f60 0x7fe274069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.537+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe290085d90 con 0x7fe298159540 2024-04-25T10:24:00.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.537+0000 7fe296a00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe274066f60 0x7fe274069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.537+0000 7fe296a00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe274066f60 0x7fe274069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe288009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.541+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe290022030 con 0x7fe298159540 2024-04-25T10:24:00.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe29806d7f0 con 0x7fe298159540 2024-04-25T10:24:00.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe28f400640 1 -- 172.21.15.45:0/3809933918 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe290010040 con 0x7fe298159540 2024-04-25T10:24:00.706 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:24:00.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe274066f60 msgr2=0x7fe274069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe274066f60 0x7fe274069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe288009040 comp rx=0 tx=0).stop 2024-04-25T10:24:00.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 msgr2=0x7fe29814c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 0x7fe29814c500 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe29000efc0 tx=0x7fe29000c490 comp rx=0 tx=0).stop 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 shutdown_connections 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe274066f60 0x7fe274069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe298159540 0x7fe29814c500 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2980aeda0 0x7fe298151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 --2- 172.21.15.45:0/3809933918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe298072020 0x7fe298152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 >> 172.21.15.45:0/3809933918 conn(0x7fe2980669a0 msgr2=0x7fe29815fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 shutdown_connections 2024-04-25T10:24:00.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.701+0000 7fe29da00640 1 -- 172.21.15.45:0/3809933918 wait complete. 2024-04-25T10:24:00.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.865+0000 7f692e000640 1 Processor -- start 2024-04-25T10:24:00.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.865+0000 7f692e000640 1 -- start start 2024-04-25T10:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f69280b1f50 0x7f69280b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692e000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692806be90 con 0x7f69280cd780 2024-04-25T10:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692e000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692806b8d0 con 0x7f69280b3dd0 2024-04-25T10:24:00.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692e000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69280d0b50 con 0x7f69280b1f50 2024-04-25T10:24:00.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692d600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f69280cd780 0x7f69280cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40186/0 (socket says 172.21.15.45:40186) 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692d600640 1 -- 172.21.15.45:0/3097308950 learned_addr learned my addr 172.21.15.45:0/3097308950 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692cc00640 1 --2- 172.21.15.45:0/3097308950 >> v2:172.21.15.117:3300/0 conn(0x7f69280b1f50 0x7f69280b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663825114 0 0) 0x7f69200014a0 con 0x7f69280cd780 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f690c003530 con 0x7f69280cd780 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792879459 0 0) 0x7f692806b8d0 con 0x7f69280b3dd0 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69200014a0 con 0x7f69280b3dd0 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692cc00640 1 -- 172.21.15.45:0/3097308950 >> v1:172.21.15.117:6789/0 conn(0x7f69280b3dd0 legacy=0x7f692806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692cc00640 1 -- 172.21.15.45:0/3097308950 >> v1:172.21.15.45:6789/0 conn(0x7f69280cd780 legacy=0x7f69280cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.869+0000 7f692cc00640 1 -- 172.21.15.45:0/3097308950 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692806bbb0 con 0x7f69280b1f50 2024-04-25T10:24:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692cc00640 1 --2- 172.21.15.45:0/3097308950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280b1f50 0x7f69280b23c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f69180097c0 tx=0x7f691801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dd83eba289dacce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180b0070 con 0x7f69280b1f50 2024-04-25T10:24:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69180ab600 con 0x7f69280b1f50 2024-04-25T10:24:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f6927e00640 1 -- 172.21.15.45:0/3097308950 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180ab9d0 con 0x7f69280b1f50 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 -- 172.21.15.45:0/3097308950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280b1f50 msgr2=0x7f69280b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 --2- 172.21.15.45:0/3097308950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280b1f50 0x7f69280b23c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f69180097c0 tx=0x7f691801c380 comp rx=0 tx=0).stop 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 -- 172.21.15.45:0/3097308950 shutdown_connections 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 --2- 172.21.15.45:0/3097308950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280b1f50 0x7f69280b23c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 -- 172.21.15.45:0/3097308950 >> 172.21.15.45:0/3097308950 conn(0x7f69280b32b0 msgr2=0x7f69280b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 -- 172.21.15.45:0/3097308950 shutdown_connections 2024-04-25T10:24:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 -- 172.21.15.45:0/3097308950 wait complete. 2024-04-25T10:24:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 Processor -- start 2024-04-25T10:24:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.873+0000 7f692e000640 1 -- start start 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 0x7f6928154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69280b3dd0 0x7f69280b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280cd780 0x7f692815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692816c230 con 0x7f69280b1f50 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69280d0550 con 0x7f69280cd780 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692816c3b0 con 0x7f69280b3dd0 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 0x7f6928154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 0x7f6928154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46472/0 (socket says 172.21.15.45:46472) 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 -- 172.21.15.45:0/1649761860 learned_addr learned my addr 172.21.15.45:0/1649761860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692d600640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280cd780 0x7f692815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f691fe00640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69280b3dd0 0x7f69280b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 -- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69280b3dd0 msgr2=0x7f69280b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69280b3dd0 0x7f69280b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 -- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280cd780 msgr2=0x7f692815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280cd780 0x7f692815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 -- 172.21.15.45:0/1649761860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69280c9710 con 0x7f69280b1f50 2024-04-25T10:24:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692d600640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280cd780 0x7f692815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692cc00640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 0x7f6928154860 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6918018f90 tx=0x7f691801abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180b0050 con 0x7f69280b1f50 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69180bc070 con 0x7f69280b1f50 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918006e20 con 0x7f69280b1f50 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69280c99f0 con 0x7f69280b1f50 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69280c9fb0 con 0x7f69280b1f50 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6928075120 con 0x7f69280b1f50 2024-04-25T10:24:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.877+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69180b5830 con 0x7f69280b1f50 2024-04-25T10:24:00.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.881+0000 7f6925600640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6904066f60 0x7f6904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:00.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.881+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f691811f610 con 0x7f69280b1f50 2024-04-25T10:24:00.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.881+0000 7f691fe00640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6904066f60 0x7f6904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.881+0000 7f691fe00640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6904066f60 0x7f6904069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f69280c8fc0 tx=0x7f6910009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:00.885+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6918016090 con 0x7f69280b1f50 2024-04-25T10:24:01.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.045+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69280b13f0 con 0x7f69280b1f50 2024-04-25T10:24:01.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f6925600640 1 -- 172.21.15.45:0/1649761860 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69180f0520 con 0x7f69280b1f50 2024-04-25T10:24:01.052 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6904066f60 msgr2=0x7f6904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6904066f60 0x7f6904069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f69280c8fc0 tx=0x7f6910009040 comp rx=0 tx=0).stop 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 msgr2=0x7f6928154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 0x7f6928154860 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6918018f90 tx=0x7f691801abc0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 shutdown_connections 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6904066f60 0x7f6904069420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69280cd780 0x7f692815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69280b3dd0 0x7f69280b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 --2- 172.21.15.45:0/1649761860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69280b1f50 0x7f6928154860 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 >> 172.21.15.45:0/1649761860 conn(0x7f69280b32b0 msgr2=0x7f69280ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:01.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 shutdown_connections 2024-04-25T10:24:01.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.049+0000 7f692e000640 1 -- 172.21.15.45:0/1649761860 wait complete. 2024-04-25T10:24:01.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:24:01.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.209+0000 7f5145800640 1 Processor -- start 2024-04-25T10:24:01.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.209+0000 7f5145800640 1 -- start start 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5145800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51400d0040 0x7f514015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5145800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f514015b740 0x7f51400b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5145800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5140071050 con 0x7f51401550b0 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5145800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514015d040 con 0x7f51400d0040 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5145800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51400712f0 con 0x7f514015b740 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51400d0040 0x7f514015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f51401550b0 0x7f51400cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40200/0 (socket says 172.21.15.45:40200) 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51400d0040 0x7f514015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46474/0 (socket says 172.21.15.45:46474) 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513ea00640 1 -- 172.21.15.45:0/4274854595 learned_addr learned my addr 172.21.15.45:0/4274854595 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:01.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513f400640 1 --2- 172.21.15.45:0/4274854595 >> v2:172.21.15.117:3300/0 conn(0x7f514015b740 0x7f51400b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 -- 172.21.15.45:0/4274854595 >> v2:172.21.15.117:3300/0 conn(0x7f514015b740 msgr2=0x7f51400b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:01.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 --2- 172.21.15.45:0/4274854595 >> v2:172.21.15.117:3300/0 conn(0x7f514015b740 0x7f51400b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 -- 172.21.15.45:0/4274854595 >> v1:172.21.15.45:6789/0 conn(0x7f51401550b0 legacy=0x7f51400cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 -- 172.21.15.45:0/4274854595 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400715d0 con 0x7f51400d0040 2024-04-25T10:24:01.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513e000640 1 -- 172.21.15.45:0/4274854595 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983581725 0 0) 0x7f5140071050 con 0x7f51401550b0 2024-04-25T10:24:01.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f5136000640 1 --2- 172.21.15.45:0/4274854595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51400d0040 0x7f514015b200 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f51400d04d0 tx=0x7f512801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d024f5a01463463e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:01.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513e000640 1 -- 172.21.15.45:0/4274854595 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280ab650 con 0x7f51400d0040 2024-04-25T10:24:01.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513e000640 1 -- 172.21.15.45:0/4274854595 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51280b0070 con 0x7f51400d0040 2024-04-25T10:24:01.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.213+0000 7f513e000640 1 -- 172.21.15.45:0/4274854595 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280aba10 con 0x7f51400d0040 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- 172.21.15.45:0/4274854595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51400d0040 msgr2=0x7f514015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 --2- 172.21.15.45:0/4274854595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51400d0040 0x7f514015b200 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f51400d04d0 tx=0x7f512801c450 comp rx=0 tx=0).stop 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- 172.21.15.45:0/4274854595 shutdown_connections 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 --2- 172.21.15.45:0/4274854595 >> v2:172.21.15.117:3300/0 conn(0x7f514015b740 0x7f51400b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 --2- 172.21.15.45:0/4274854595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51400d0040 0x7f514015b200 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- 172.21.15.45:0/4274854595 >> 172.21.15.45:0/4274854595 conn(0x7f5140066b70 msgr2=0x7f5140066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- 172.21.15.45:0/4274854595 shutdown_connections 2024-04-25T10:24:01.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- 172.21.15.45:0/4274854595 wait complete. 2024-04-25T10:24:01.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 Processor -- start 2024-04-25T10:24:01.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- start start 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51400d0040 0x7f514013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51401550b0 0x7f5140140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 0x7f514006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514015bcd0 con 0x7f514015b740 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514015bb50 con 0x7f51401550b0 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5145800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514015c030 con 0x7f51400d0040 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f513f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 0x7f514006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f513f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 0x7f514006d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46488/0 (socket says 172.21.15.45:46488) 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f513f400640 1 -- 172.21.15.45:0/1078483741 learned_addr learned my addr 172.21.15.45:0/1078483741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f5136000640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51401550b0 0x7f5140140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.217+0000 7f513ea00640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51400d0040 0x7f514013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513f400640 1 -- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51400d0040 msgr2=0x7f514013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513f400640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51400d0040 0x7f514013fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513f400640 1 -- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51401550b0 msgr2=0x7f5140140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513f400640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51401550b0 0x7f5140140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513f400640 1 -- 172.21.15.45:0/1078483741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5140140330 con 0x7f514015b740 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5136000640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51401550b0 0x7f5140140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513f400640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 0x7f514006d5f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f513800efc0 tx=0x7f513800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138018020 con 0x7f514015b740 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138014480 con 0x7f514015b740 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51401410a0 con 0x7f514015b740 2024-04-25T10:24:01.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138014870 con 0x7f514015b740 2024-04-25T10:24:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5140141660 con 0x7f514015b740 2024-04-25T10:24:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514004ef30 con 0x7f514015b740 2024-04-25T10:24:01.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f513801b020 con 0x7f514015b740 2024-04-25T10:24:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5137400640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f511c066f60 0x7f511c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5138085e80 con 0x7f514015b740 2024-04-25T10:24:01.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.221+0000 7f513ea00640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f511c066f60 0x7f511c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.225+0000 7f513ea00640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f511c066f60 0x7f511c069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5130000f70 tx=0x7f5130007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:01.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.225+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5138022030 con 0x7f514015b740 2024-04-25T10:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51400adc00 con 0x7f514015b740 2024-04-25T10:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5137400640 1 -- 172.21.15.45:0/1078483741 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5138010040 con 0x7f514015b740 2024-04-25T10:24:01.405 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:24:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f511c066f60 msgr2=0x7f511c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f511c066f60 0x7f511c069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5130000f70 tx=0x7f5130007b60 comp rx=0 tx=0).stop 2024-04-25T10:24:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 msgr2=0x7f514006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 0x7f514006d5f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f513800efc0 tx=0x7f513800c490 comp rx=0 tx=0).stop 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 shutdown_connections 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f511c066f60 0x7f511c069420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f514015b740 0x7f514006d5f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51401550b0 0x7f5140140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 --2- 172.21.15.45:0/1078483741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51400d0040 0x7f514013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 >> 172.21.15.45:0/1078483741 conn(0x7f5140066b70 msgr2=0x7f51400cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 shutdown_connections 2024-04-25T10:24:01.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.401+0000 7f5145800640 1 -- 172.21.15.45:0/1078483741 wait complete. 2024-04-25T10:24:01.426 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:24:01.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:01.426 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:01.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:24:01.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:01.464 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:24:01.521 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:24:01.521 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:01.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:01.531 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:24:01.531 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:01.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:24:01.572 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:24:01.572 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:01.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:01.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 Processor -- start 2024-04-25T10:24:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 -- start start 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5db415dff0 0x7f5db406b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db406bbe0 0x7f5db40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40cbff0 con 0x7f5db415dc10 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db40b45a0 con 0x7f5db415dff0 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5dba600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cbda0 con 0x7f5db406bbe0 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5db415dff0 0x7f5db406b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5db415dff0 0x7f5db406b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46502/0 (socket says 172.21.15.45:46502) 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 -- 172.21.15.45:0/1223781389 learned_addr learned my addr 172.21.15.45:0/1223781389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db9c00640 1 --2- 172.21.15.45:0/1223781389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db406bbe0 0x7f5db40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 -- 172.21.15.45:0/1223781389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db406bbe0 msgr2=0x7f5db40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:01.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 --2- 172.21.15.45:0/1223781389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db406bbe0 0x7f5db40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 -- 172.21.15.45:0/1223781389 >> v1:172.21.15.45:6789/0 conn(0x7f5db415dc10 legacy=0x7f5db40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3400640 1 -- 172.21.15.45:0/1223781389 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21543282 0 0) 0x7f5db40cbff0 con 0x7f5db415dc10 2024-04-25T10:24:01.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.753+0000 7f5db3e00640 1 -- 172.21.15.45:0/1223781389 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40cc2d0 con 0x7f5db415dff0 2024-04-25T10:24:01.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5db3e00640 1 --2- 172.21.15.45:0/1223781389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dff0 0x7f5db406b6a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009870 tx=0x7f5da001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dd68b5de9934f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:01.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5db3400640 1 -- 172.21.15.45:0/1223781389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da00ab650 con 0x7f5db415dff0 2024-04-25T10:24:01.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5db3400640 1 -- 172.21.15.45:0/1223781389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da00b0070 con 0x7f5db415dff0 2024-04-25T10:24:01.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5db3400640 1 -- 172.21.15.45:0/1223781389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da00aba10 con 0x7f5db415dff0 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 -- 172.21.15.45:0/1223781389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dff0 msgr2=0x7f5db406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 --2- 172.21.15.45:0/1223781389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dff0 0x7f5db406b6a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009870 tx=0x7f5da001c560 comp rx=0 tx=0).stop 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 -- 172.21.15.45:0/1223781389 shutdown_connections 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 --2- 172.21.15.45:0/1223781389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db406bbe0 0x7f5db40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 --2- 172.21.15.45:0/1223781389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dff0 0x7f5db406b6a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 -- 172.21.15.45:0/1223781389 >> 172.21.15.45:0/1223781389 conn(0x7f5db40d1750 msgr2=0x7f5db40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:01.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 -- 172.21.15.45:0/1223781389 shutdown_connections 2024-04-25T10:24:01.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 -- 172.21.15.45:0/1223781389 wait complete. 2024-04-25T10:24:01.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.757+0000 7f5dba600640 1 Processor -- start 2024-04-25T10:24:01.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- start start 2024-04-25T10:24:01.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db406bbe0 0x7f5db414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 0x7f5db4155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db415dff0 0x7f5db414db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db406b1e0 con 0x7f5db415dc10 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db406b060 con 0x7f5db406bbe0 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cdb00 con 0x7f5db415dff0 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 0x7f5db4155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 0x7f5db4155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46508/0 (socket says 172.21.15.45:46508) 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 -- 172.21.15.45:0/3145664823 learned_addr learned my addr 172.21.15.45:0/3145664823 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db9200640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db406bbe0 0x7f5db414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db9c00640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db415dff0 0x7f5db414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 -- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db415dff0 msgr2=0x7f5db414db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db415dff0 0x7f5db414db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 -- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db406bbe0 msgr2=0x7f5db414f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db406bbe0 0x7f5db414f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 -- 172.21.15.45:0/3145664823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4154770 con 0x7f5db415dc10 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db3e00640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 0x7f5db4155a30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f5da001ac10 tx=0x7f5da0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da00bf610 con 0x7f5db415dc10 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da00bf7b0 con 0x7f5db415dc10 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da00bfba0 con 0x7f5db415dc10 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db4154a50 con 0x7f5db415dc10 2024-04-25T10:24:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4155010 con 0x7f5db415dc10 2024-04-25T10:24:01.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.761+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db4075120 con 0x7f5db415dc10 2024-04-25T10:24:01.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.765+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5da00bd020 con 0x7f5db415dc10 2024-04-25T10:24:01.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.765+0000 7f5db0c00640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d94066f60 0x7f5d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:01.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.765+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5da011f450 con 0x7f5db415dc10 2024-04-25T10:24:01.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.765+0000 7f5db9200640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d94066f60 0x7f5d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:01.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.765+0000 7f5db9200640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d94066f60 0x7f5d94069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5da8000f70 tx=0x7f5da8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:01.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.769+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5da00c6030 con 0x7f5db415dc10 2024-04-25T10:24:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.961+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5db4068650 con 0x7f5db415dc10 2024-04-25T10:24:01.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.961+0000 7f5db0c00640 1 -- 172.21.15.45:0/3145664823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5da0016150 con 0x7f5db415dc10 2024-04-25T10:24:01.967 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:01.967 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:22:26.898819+0000","modified":"2024-04-25T10:22:34.900344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3661919331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3661919331},{"type":"v1","addr":"172.21.15.45:6835","nonce":3661919331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:24:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2024-04-25T10:24:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d94066f60 msgr2=0x7f5d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d94066f60 0x7f5d94069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5da8000f70 tx=0x7f5da8009040 comp rx=0 tx=0).stop 2024-04-25T10:24:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 msgr2=0x7f5db4155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 0x7f5db4155a30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f5da001ac10 tx=0x7f5da0019300 comp rx=0 tx=0).stop 2024-04-25T10:24:01.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 shutdown_connections 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5d94066f60 0x7f5d94069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5db415dff0 0x7f5db414db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5db415dc10 0x7f5db4155a30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 --2- 172.21.15.45:0/3145664823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5db406bbe0 0x7f5db414f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 >> 172.21.15.45:0/3145664823 conn(0x7f5db40d1750 msgr2=0x7f5db40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 shutdown_connections 2024-04-25T10:24:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:01.965+0000 7f5dba600640 1 -- 172.21.15.45:0/3145664823 wait complete. 2024-04-25T10:24:01.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.125+0000 7fe2e5400640 1 Processor -- start 2024-04-25T10:24:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.125+0000 7fe2e5400640 1 -- start start 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2e5400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe2e01550b0 0x7fe2e00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2e5400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2e00cfe70 0x7fe2e015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2e5400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e0071050 con 0x7fe2e015b740 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2e5400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e015d040 con 0x7fe2e01550b0 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2e5400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00712f0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2de800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe2e01550b0 0x7fe2e00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2df200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe2e015b740 0x7fe2e006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40214/0 (socket says 172.21.15.45:40214) 2024-04-25T10:24:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2df200640 1 -- 172.21.15.45:0/4136416317 learned_addr learned my addr 172.21.15.45:0/4136416317 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2d5e00640 1 --2- 172.21.15.45:0/4136416317 >> v2:172.21.15.117:3300/0 conn(0x7fe2e00cfe70 0x7fe2e015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2dde00640 1 -- 172.21.15.45:0/4136416317 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314504439 0 0) 0x7fe2e0071050 con 0x7fe2e015b740 2024-04-25T10:24:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2d5e00640 1 -- 172.21.15.45:0/4136416317 >> v2:172.21.15.45:3300/0 conn(0x7fe2e01550b0 msgr2=0x7fe2e00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2d5e00640 1 --2- 172.21.15.45:0/4136416317 >> v2:172.21.15.45:3300/0 conn(0x7fe2e01550b0 0x7fe2e00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2d5e00640 1 -- 172.21.15.45:0/4136416317 >> v1:172.21.15.45:6789/0 conn(0x7fe2e015b740 legacy=0x7fe2e006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2d5e00640 1 -- 172.21.15.45:0/4136416317 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00715d0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.129+0000 7fe2de800640 1 --2- 172.21.15.45:0/4136416317 >> v2:172.21.15.45:3300/0 conn(0x7fe2e01550b0 0x7fe2e00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2d5e00640 1 --2- 172.21.15.45:0/4136416317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e00cfe70 0x7fe2e015b200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80097c0 tx=0x7fe2c801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5105fe2f7ccf21f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2dde00640 1 -- 172.21.15.45:0/4136416317 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80b0070 con 0x7fe2e00cfe70 2024-04-25T10:24:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2dde00640 1 -- 172.21.15.45:0/4136416317 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c80ab600 con 0x7fe2e00cfe70 2024-04-25T10:24:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2dde00640 1 -- 172.21.15.45:0/4136416317 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ab9d0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 -- 172.21.15.45:0/4136416317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e00cfe70 msgr2=0x7fe2e015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 --2- 172.21.15.45:0/4136416317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e00cfe70 0x7fe2e015b200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80097c0 tx=0x7fe2c801c460 comp rx=0 tx=0).stop 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 -- 172.21.15.45:0/4136416317 shutdown_connections 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 --2- 172.21.15.45:0/4136416317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e00cfe70 0x7fe2e015b200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 --2- 172.21.15.45:0/4136416317 >> v2:172.21.15.45:3300/0 conn(0x7fe2e01550b0 0x7fe2e00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 -- 172.21.15.45:0/4136416317 >> 172.21.15.45:0/4136416317 conn(0x7fe2e0066b70 msgr2=0x7fe2e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 -- 172.21.15.45:0/4136416317 shutdown_connections 2024-04-25T10:24:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 -- 172.21.15.45:0/4136416317 wait complete. 2024-04-25T10:24:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 Processor -- start 2024-04-25T10:24:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.133+0000 7fe2e5400640 1 -- start start 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 0x7fe2e01491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e01550b0 0x7fe2e00ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2e015b740 0x7fe2e00ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015bcf0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015bb70 con 0x7fe2e01550b0 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015c030 con 0x7fe2e015b740 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 0x7fe2e01491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 0x7fe2e01491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46526/0 (socket says 172.21.15.45:46526) 2024-04-25T10:24:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 -- 172.21.15.45:0/3447780412 learned_addr learned my addr 172.21.15.45:0/3447780412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d5e00640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e01550b0 0x7fe2e00ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2df200640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2e015b740 0x7fe2e00ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 -- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2e015b740 msgr2=0x7fe2e00ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2e015b740 0x7fe2e00ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 -- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e01550b0 msgr2=0x7fe2e00ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e01550b0 0x7fe2e00ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 -- 172.21.15.45:0/3447780412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e01496f0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2df200640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2e015b740 0x7fe2e00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2de800640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 0x7fe2e01491b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009f60 tx=0x7fe2d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0018020 con 0x7fe2e00cfe70 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0014680 con 0x7fe2e00cfe70 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00cd220 con 0x7fe2e00cfe70 2024-04-25T10:24:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0014a50 con 0x7fe2e00cfe70 2024-04-25T10:24:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00cd7e0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e004ef30 con 0x7fe2e00cfe70 2024-04-25T10:24:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2d001b020 con 0x7fe2e00cfe70 2024-04-25T10:24:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d7400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2bc066f60 0x7fe2bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe2d00864c0 con 0x7fe2e00cfe70 2024-04-25T10:24:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d5e00640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2bc066f60 0x7fe2bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.141+0000 7fe2d5e00640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2bc066f60 0x7fe2bc069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad000 tx=0x7fe2c80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.145+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2d0022030 con 0x7fe2e00cfe70 2024-04-25T10:24:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.305+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2e015a260 con 0x7fe2e00cfe70 2024-04-25T10:24:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.305+0000 7fe2d7400640 1 -- 172.21.15.45:0/3447780412 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16759 (secure 0 0 0) 0x7fe2d0010040 con 0x7fe2e00cfe70 2024-04-25T10:24:02.312 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:02.312 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:23:40.632586+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":80,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.0","mappings":[{"from":2,"to":7}]},{"pgid":"11.3d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2bc066f60 msgr2=0x7fe2bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2bc066f60 0x7fe2bc069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad000 tx=0x7fe2c80a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 msgr2=0x7fe2e01491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 0x7fe2e01491b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009f60 tx=0x7fe2d0009f90 comp rx=0 tx=0).stop 2024-04-25T10:24:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 shutdown_connections 2024-04-25T10:24:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe2bc066f60 0x7fe2bc069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe2e015b740 0x7fe2e00ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2e01550b0 0x7fe2e00ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 --2- 172.21.15.45:0/3447780412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2e00cfe70 0x7fe2e01491b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 >> 172.21.15.45:0/3447780412 conn(0x7fe2e0066b70 msgr2=0x7fe2e015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 shutdown_connections 2024-04-25T10:24:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.309+0000 7fe2e5400640 1 -- 172.21.15.45:0/3447780412 wait complete. 2024-04-25T10:24:02.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:22:25.323207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:22:25.891841+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 80, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:02.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 Processor -- start 2024-04-25T10:24:02.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- start start 2024-04-25T10:24:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f000b1f50 0x7f2f000b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000cd780 0x7f2f000cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0006be90 con 0x7f2f000b3dd0 2024-04-25T10:24:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f000d0b50 con 0x7f2f000b1f50 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0006b680 con 0x7f2f000cd780 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f000b1f50 0x7f2f000b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2f000b1f50 0x7f2f000b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46536/0 (socket says 172.21.15.45:46536) 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 -- 172.21.15.45:0/3589280018 learned_addr learned my addr 172.21.15.45:0/3589280018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06200640 1 --2- 172.21.15.45:0/3589280018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000cd780 0x7f2f000cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 -- 172.21.15.45:0/3589280018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000cd780 msgr2=0x7f2f000cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 --2- 172.21.15.45:0/3589280018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000cd780 0x7f2f000cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 -- 172.21.15.45:0/3589280018 >> v1:172.21.15.117:6789/0 conn(0x7f2f000b3dd0 legacy=0x7f2f0006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 -- 172.21.15.45:0/3589280018 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0006b8d0 con 0x7f2f000b1f50 2024-04-25T10:24:02.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f05800640 1 --2- 172.21.15.45:0/3589280018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b1f50 0x7f2f000b23c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009870 tx=0x7f2ef401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d066e6fce7b5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2effe00640 1 -- 172.21.15.45:0/3589280018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40b0070 con 0x7f2f000b1f50 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2effe00640 1 -- 172.21.15.45:0/3589280018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4004230 con 0x7f2f000b1f50 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2effe00640 1 -- 172.21.15.45:0/3589280018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4003620 con 0x7f2f000b1f50 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- 172.21.15.45:0/3589280018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b1f50 msgr2=0x7f2f000b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 --2- 172.21.15.45:0/3589280018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b1f50 0x7f2f000b23c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009870 tx=0x7f2ef401c440 comp rx=0 tx=0).stop 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- 172.21.15.45:0/3589280018 shutdown_connections 2024-04-25T10:24:02.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 --2- 172.21.15.45:0/3589280018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000cd780 0x7f2f000cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 --2- 172.21.15.45:0/3589280018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b1f50 0x7f2f000b23c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- 172.21.15.45:0/3589280018 >> 172.21.15.45:0/3589280018 conn(0x7f2f000b32b0 msgr2=0x7f2f000b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- 172.21.15.45:0/3589280018 shutdown_connections 2024-04-25T10:24:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.477+0000 7f2f06c00640 1 -- 172.21.15.45:0/3589280018 wait complete. 2024-04-25T10:24:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 Processor -- start 2024-04-25T10:24:02.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 -- start start 2024-04-25T10:24:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000b1f50 0x7f2f00148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 0x7f2f000c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000cd780 0x7f2f00156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000d03c0 con 0x7f2f000b3dd0 2024-04-25T10:24:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000d0240 con 0x7f2f000cd780 2024-04-25T10:24:02.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000d0540 con 0x7f2f000b1f50 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 0x7f2f000c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 0x7f2f000c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46552/0 (socket says 172.21.15.45:46552) 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 -- 172.21.15.45:0/2442604001 learned_addr learned my addr 172.21.15.45:0/2442604001 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f05800640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000b1f50 0x7f2f00148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06200640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000cd780 0x7f2f00156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 -- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000b1f50 msgr2=0x7f2f00148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000b1f50 0x7f2f00148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 -- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000cd780 msgr2=0x7f2f00156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000cd780 0x7f2f00156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 -- 172.21.15.45:0/2442604001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00149230 con 0x7f2f000b3dd0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f04e00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 0x7f2f000c8830 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c370 tx=0x7f2eec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0052d0 con 0x7f2f000b3dd0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec005470 con 0x7f2f000b3dd0 2024-04-25T10:24:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec007260 con 0x7f2f000b3dd0 2024-04-25T10:24:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.481+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f000cb4a0 con 0x7f2f000b3dd0 2024-04-25T10:24:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0015fcf0 con 0x7f2f000b3dd0 2024-04-25T10:24:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0015f800 con 0x7f2f000b3dd0 2024-04-25T10:24:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eec007400 con 0x7f2f000b3dd0 2024-04-25T10:24:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2efd600640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee0066f90 0x7f2ee0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2eec084430 con 0x7f2f000b3dd0 2024-04-25T10:24:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2f05800640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee0066f90 0x7f2ee0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.485+0000 7f2f05800640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee0066f90 0x7f2ee0069450 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40ad000 tx=0x7f2ef40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.489+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f0006be90 con 0x7f2f000b3dd0 2024-04-25T10:24:02.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.657+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f000cba70 con 0x7f2f000b3dd0 2024-04-25T10:24:02.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.657+0000 7f2efd600640 1 -- 172.21.15.45:0/2442604001 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2eec054d90 con 0x7f2f000b3dd0 2024-04-25T10:24:02.663 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee0066f90 msgr2=0x7f2ee0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee0066f90 0x7f2ee0069450 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40ad000 tx=0x7f2ef40a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 msgr2=0x7f2f000c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 0x7f2f000c8830 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c370 tx=0x7f2eec00c830 comp rx=0 tx=0).stop 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 shutdown_connections 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ee0066f90 0x7f2ee0069450 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000cd780 0x7f2f00156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000b3dd0 0x7f2f000c8830 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 --2- 172.21.15.45:0/2442604001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f000b1f50 0x7f2f00148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 >> 172.21.15.45:0/2442604001 conn(0x7f2f000b32b0 msgr2=0x7f2f000ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:02.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 shutdown_connections 2024-04-25T10:24:02.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.661+0000 7f2f06c00640 1 -- 172.21.15.45:0/2442604001 wait complete. 2024-04-25T10:24:02.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:24:02.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3717800640 1 Processor -- start 2024-04-25T10:24:02.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3717800640 1 -- start start 2024-04-25T10:24:02.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3717800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f371015bbd0 0x7f37100d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3717800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37100cfba0 con 0x7f37100d0630 2024-04-25T10:24:02.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3717800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37100b4a00 con 0x7f37100cf530 2024-04-25T10:24:02.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3717800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37100cf970 con 0x7f371015bbd0 2024-04-25T10:24:02.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3714e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f371015bbd0 0x7f37100d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3714e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f371015bbd0 0x7f37100d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46566/0 (socket says 172.21.15.45:46566) 2024-04-25T10:24:02.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3714e00640 1 -- 172.21.15.45:0/2636102597 learned_addr learned my addr 172.21.15.45:0/2636102597 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:02.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3714e00640 1 -- 172.21.15.45:0/2636102597 >> v1:172.21.15.117:6789/0 conn(0x7f37100cf530 legacy=0x7f37100cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3714e00640 1 -- 172.21.15.45:0/2636102597 >> v1:172.21.15.45:6789/0 conn(0x7f37100d0630 legacy=0x7f37100b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f3714e00640 1 -- 172.21.15.45:0/2636102597 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100b4ce0 con 0x7f371015bbd0 2024-04-25T10:24:02.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.825+0000 7f370f400640 1 -- 172.21.15.45:0/2636102597 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462132118 0 0) 0x7f37100cfba0 con 0x7f37100d0630 2024-04-25T10:24:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3714e00640 1 --2- 172.21.15.45:0/2636102597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371015bbd0 0x7f37100d2450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f370401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcb9825d7b6a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f370f400640 1 -- 172.21.15.45:0/2636102597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040a7070 con 0x7f371015bbd0 2024-04-25T10:24:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f370f400640 1 -- 172.21.15.45:0/2636102597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37040b0070 con 0x7f371015bbd0 2024-04-25T10:24:02.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f370f400640 1 -- 172.21.15.45:0/2636102597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040ab890 con 0x7f371015bbd0 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 -- 172.21.15.45:0/2636102597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371015bbd0 msgr2=0x7f37100d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 --2- 172.21.15.45:0/2636102597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371015bbd0 0x7f37100d2450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f370401c560 comp rx=0 tx=0).stop 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 -- 172.21.15.45:0/2636102597 shutdown_connections 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 --2- 172.21.15.45:0/2636102597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f371015bbd0 0x7f37100d2450 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 -- 172.21.15.45:0/2636102597 >> 172.21.15.45:0/2636102597 conn(0x7f37100d1960 msgr2=0x7f37100d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 -- 172.21.15.45:0/2636102597 shutdown_connections 2024-04-25T10:24:02.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.829+0000 7f3717800640 1 -- 172.21.15.45:0/2636102597 wait complete. 2024-04-25T10:24:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 Processor -- start 2024-04-25T10:24:02.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 -- start start 2024-04-25T10:24:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 0x7f371006af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37100d0630 0x7f37100ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371015bbd0 0x7f3710156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37100b6790 con 0x7f37100cf530 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37100b6610 con 0x7f371015bbd0 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37101698a0 con 0x7f37100d0630 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 0x7f371006af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 0x7f371006af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46574/0 (socket says 172.21.15.45:46574) 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 -- 172.21.15.45:0/1217353341 learned_addr learned my addr 172.21.15.45:0/1217353341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 -- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37100d0630 msgr2=0x7f37100ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37100d0630 0x7f37100ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 -- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371015bbd0 msgr2=0x7f3710156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371015bbd0 0x7f3710156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 -- 172.21.15.45:0/1217353341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37101571b0 con 0x7f37100cf530 2024-04-25T10:24:02.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3714e00640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 0x7f371006af60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f37040ad040 tx=0x7f370401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040b4800 con 0x7f37100cf530 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37040b49c0 con 0x7f37100cf530 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37040b4db0 con 0x7f37100cf530 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371015e0d0 con 0x7f37100cf530 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.833+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371015d310 con 0x7f37100cf530 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.837+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371004ef30 con 0x7f37100cf530 2024-04-25T10:24:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.837+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37040b3380 con 0x7f37100cf530 2024-04-25T10:24:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.837+0000 7f370cc00640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f0066f60 0x7f36f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.837+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f370411ef00 con 0x7f37100cf530 2024-04-25T10:24:02.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.837+0000 7f370fe00640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f0066f60 0x7f36f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:02.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.837+0000 7f370fe00640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f0066f60 0x7f36f0069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f70 tx=0x7f36fc0079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:02.845+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37040c7020 con 0x7f37100cf530 2024-04-25T10:24:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f371006d510 con 0x7f37100cf530 2024-04-25T10:24:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f370cc00640 1 -- 172.21.15.45:0/1217353341 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3704016150 con 0x7f37100cf530 2024-04-25T10:24:03.009 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f0066f60 msgr2=0x7f36f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f0066f60 0x7f36f0069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f70 tx=0x7f36fc0079c0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 msgr2=0x7f371006af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 0x7f371006af60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f37040ad040 tx=0x7f370401ada0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 shutdown_connections 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f0066f60 0x7f36f0069420 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f371015bbd0 0x7f3710156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37100d0630 0x7f37100ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 --2- 172.21.15.45:0/1217353341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37100cf530 0x7f371006af60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 >> 172.21.15.45:0/1217353341 conn(0x7f37100d1960 msgr2=0x7f37100ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 shutdown_connections 2024-04-25T10:24:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.005+0000 7f3717800640 1 -- 172.21.15.45:0/1217353341 wait complete. 2024-04-25T10:24:03.030 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:24:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.173+0000 7f004be00640 1 Processor -- start 2024-04-25T10:24:03.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.173+0000 7f004be00640 1 -- start start 2024-04-25T10:24:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce290 0x7f004c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0cfde0 con 0x7f004c15f420 2024-04-25T10:24:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0d0080 con 0x7f004c0ce670 2024-04-25T10:24:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0d0360 con 0x7f004c0ce290 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f004c15f420 0x7f004c0b4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40244/0 (socket says 172.21.15.45:40244) 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004b400640 1 -- 172.21.15.45:0/4175458515 learned_addr learned my addr 172.21.15.45:0/4175458515 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004aa00640 1 --2- 172.21.15.45:0/4175458515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce290 0x7f004c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004a000640 1 -- 172.21.15.45:0/4175458515 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099818241 0 0) 0x7f004c0cfde0 con 0x7f004c15f420 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004a000640 1 -- 172.21.15.45:0/4175458515 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0030003530 con 0x7f004c15f420 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004a000640 1 -- 172.21.15.45:0/4175458515 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550134002 0 0) 0x7f004c0d0080 con 0x7f004c0ce670 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004aa00640 1 -- 172.21.15.45:0/4175458515 >> v1:172.21.15.117:6789/0 conn(0x7f004c0ce670 legacy=0x7f004c15ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004aa00640 1 -- 172.21.15.45:0/4175458515 >> v1:172.21.15.45:6789/0 conn(0x7f004c15f420 legacy=0x7f004c0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004aa00640 1 -- 172.21.15.45:0/4175458515 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0d0640 con 0x7f004c0ce290 2024-04-25T10:24:03.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004aa00640 1 --2- 172.21.15.45:0/4175458515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce290 0x7f004c0b3dd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f003c009dc0 tx=0x7f003c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3e701d479e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004a000640 1 -- 172.21.15.45:0/4175458515 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0b2070 con 0x7f004c0ce290 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004a000640 1 -- 172.21.15.45:0/4175458515 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c0a58f0 con 0x7f004c0ce290 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004a000640 1 -- 172.21.15.45:0/4175458515 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0a5cc0 con 0x7f004c0ce290 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- 172.21.15.45:0/4175458515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce290 msgr2=0x7f004c0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 --2- 172.21.15.45:0/4175458515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce290 0x7f004c0b3dd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f003c009dc0 tx=0x7f003c01c900 comp rx=0 tx=0).stop 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- 172.21.15.45:0/4175458515 shutdown_connections 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 --2- 172.21.15.45:0/4175458515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce290 0x7f004c0b3dd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- 172.21.15.45:0/4175458515 >> 172.21.15.45:0/4175458515 conn(0x7f004c0b32b0 msgr2=0x7f004c0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- 172.21.15.45:0/4175458515 shutdown_connections 2024-04-25T10:24:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- 172.21.15.45:0/4175458515 wait complete. 2024-04-25T10:24:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 Processor -- start 2024-04-25T10:24:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.177+0000 7f004be00640 1 -- start start 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004c0ce290 0x7f004c157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce670 0x7f004c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 0x7f004c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0b5b40 con 0x7f004c15f420 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0b59c0 con 0x7f004c0ce290 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c169920 con 0x7f004c0ce670 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 0x7f004c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 0x7f004c1599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46582/0 (socket says 172.21.15.45:46582) 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 -- 172.21.15.45:0/2682921609 learned_addr learned my addr 172.21.15.45:0/2682921609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:03.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f0042000640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce670 0x7f004c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004aa00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004c0ce290 0x7f004c157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 -- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce670 msgr2=0x7f004c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce670 0x7f004c1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 -- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004c0ce290 msgr2=0x7f004c157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004c0ce290 0x7f004c157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 -- 172.21.15.45:0/2682921609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c09f240 con 0x7f004c15f420 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004b400640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 0x7f004c1599a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f004400c3e0 tx=0x7f004400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044017070 con 0x7f004c15f420 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044005cf0 con 0x7f004c15f420 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c09f520 con 0x7f004c15f420 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00440072d0 con 0x7f004c15f420 2024-04-25T10:24:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c1584a0 con 0x7f004c15f420 2024-04-25T10:24:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c157210 con 0x7f004c15f420 2024-04-25T10:24:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.181+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f004401a070 con 0x7f004c15f420 2024-04-25T10:24:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.185+0000 7f0043400640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0028066f60 0x7f0028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.185+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f00440842f0 con 0x7f004c15f420 2024-04-25T10:24:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.185+0000 7f004aa00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0028066f60 0x7f0028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.185+0000 7f004aa00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0028066f60 0x7f0028069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f003c0af000 tx=0x7f003c004ac0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.189+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f004c157210 con 0x7f004c15f420 2024-04-25T10:24:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f004c075120 con 0x7f004c15f420 2024-04-25T10:24:03.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f0043400640 1 -- 172.21.15.45:0/2682921609 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0044054c40 con 0x7f004c15f420 2024-04-25T10:24:03.353 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:24:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0028066f60 msgr2=0x7f0028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0028066f60 0x7f0028069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f003c0af000 tx=0x7f003c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 msgr2=0x7f004c1599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 0x7f004c1599a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f004400c3e0 tx=0x7f004400c8a0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 shutdown_connections 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0028066f60 0x7f0028069420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004c15f420 0x7f004c1599a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f004c0ce670 0x7f004c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 --2- 172.21.15.45:0/2682921609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004c0ce290 0x7f004c157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 >> 172.21.15.45:0/2682921609 conn(0x7f004c0b32b0 msgr2=0x7f004c15dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 shutdown_connections 2024-04-25T10:24:03.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.349+0000 7f004be00640 1 -- 172.21.15.45:0/2682921609 wait complete. 2024-04-25T10:24:03.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:03.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 Processor -- start 2024-04-25T10:24:03.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 -- start start 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0cf930 0x7f465c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f465c070c20 0x7f465c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f465c0ae020 0x7f465c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c15bb90 con 0x7f465c0ae020 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c0d0250 con 0x7f465c070c20 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.513+0000 7f4662e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0ad840 con 0x7f465c0cf930 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f465c0ae020 0x7f465c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f465c0ae020 0x7f465c0ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46598/0 (socket says 172.21.15.45:46598) 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 -- 172.21.15.45:0/3594577139 learned_addr learned my addr 172.21.15.45:0/3594577139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f465be00640 1 --2- 172.21.15.45:0/3594577139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0cf930 0x7f465c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4653e00640 1 --2- 172.21.15.45:0/3594577139 >> v2:172.21.15.117:3300/0 conn(0x7f465c070c20 0x7f465c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 -- 172.21.15.45:0/3594577139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0cf930 msgr2=0x7f465c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 --2- 172.21.15.45:0/3594577139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0cf930 0x7f465c0cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 -- 172.21.15.45:0/3594577139 >> v2:172.21.15.117:3300/0 conn(0x7f465c070c20 msgr2=0x7f465c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 --2- 172.21.15.45:0/3594577139 >> v2:172.21.15.117:3300/0 conn(0x7f465c070c20 0x7f465c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 -- 172.21.15.45:0/3594577139 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c15e9f0 con 0x7f465c0ae020 2024-04-25T10:24:03.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4653e00640 1 --2- 172.21.15.45:0/3594577139 >> v2:172.21.15.117:3300/0 conn(0x7f465c070c20 0x7f465c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:03.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f4660e00640 1 --2- 172.21.15.45:0/3594577139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c0ae4b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f465c0d0540 tx=0x7f465401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bc43d2bf23098a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f465b400640 1 -- 172.21.15.45:0/3594577139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540a5840 con 0x7f465c0ae020 2024-04-25T10:24:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f465b400640 1 -- 172.21.15.45:0/3594577139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46540a59e0 con 0x7f465c0ae020 2024-04-25T10:24:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.517+0000 7f465b400640 1 -- 172.21.15.45:0/3594577139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46540a5db0 con 0x7f465c0ae020 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- 172.21.15.45:0/3594577139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 msgr2=0x7f465c0ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- 172.21.15.45:0/3594577139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c0ae4b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f465c0d0540 tx=0x7f465401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- 172.21.15.45:0/3594577139 shutdown_connections 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- 172.21.15.45:0/3594577139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c0ae4b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- 172.21.15.45:0/3594577139 >> v2:172.21.15.117:3300/0 conn(0x7f465c070c20 0x7f465c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- 172.21.15.45:0/3594577139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c0cf930 0x7f465c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- 172.21.15.45:0/3594577139 >> 172.21.15.45:0/3594577139 conn(0x7f465c066b70 msgr2=0x7f465c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- 172.21.15.45:0/3594577139 shutdown_connections 2024-04-25T10:24:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- 172.21.15.45:0/3594577139 wait complete. 2024-04-25T10:24:03.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 Processor -- start 2024-04-25T10:24:03.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- start start 2024-04-25T10:24:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c070c20 0x7f465c160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c0cf930 0x7f465c06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c15e6b0 con 0x7f465c0ae020 2024-04-25T10:24:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c06e130 con 0x7f465c0cf930 2024-04-25T10:24:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4662e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c161790 con 0x7f465c070c20 2024-04-25T10:24:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f465be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c070c20 0x7f465c160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4660e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c0cf930 0x7f465c06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f465be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c070c20 0x7f465c160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:42984/0 (socket says 172.21.15.45:42984) 2024-04-25T10:24:03.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f465be00640 1 -- 172.21.15.45:0/2532550205 learned_addr learned my addr 172.21.15.45:0/2532550205 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.521+0000 7f4653e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4653e00640 1 -- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c070c20 msgr2=0x7f465c160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4653e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c070c20 0x7f465c160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4653e00640 1 -- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c0cf930 msgr2=0x7f465c06c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4653e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c0cf930 0x7f465c06c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4653e00640 1 -- 172.21.15.45:0/2532550205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c1524e0 con 0x7f465c0ae020 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4653e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c1519a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f464400e610 tx=0x7f464400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400f040 con 0x7f465c0ae020 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644004230 con 0x7f465c0ae020 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644003620 con 0x7f465c0ae020 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c1527c0 con 0x7f465c0ae020 2024-04-25T10:24:03.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c152d80 con 0x7f465c0ae020 2024-04-25T10:24:03.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c04ef30 con 0x7f465c0ae020 2024-04-25T10:24:03.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46440037c0 con 0x7f465c0ae020 2024-04-25T10:24:03.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4658c00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4638066f60 0x7f4638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.525+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4644084b90 con 0x7f465c0ae020 2024-04-25T10:24:03.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.529+0000 7f465be00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4638066f60 0x7f4638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.529+0000 7f465be00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4638066f60 0x7f4638069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f464c009b80 tx=0x7f464c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.533+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4644017020 con 0x7f465c0ae020 2024-04-25T10:24:03.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.725+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f465c151ee0 con 0x7f465c0ae020 2024-04-25T10:24:03.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4658c00640 1 -- 172.21.15.45:0/2532550205 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f46440554e0 con 0x7f465c0ae020 2024-04-25T10:24:03.728 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:03.728 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:22:26.898819+0000","modified":"2024-04-25T10:22:34.900344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3661919331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3661919331},{"type":"v1","addr":"172.21.15.45:6835","nonce":3661919331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:24:03.728 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4638066f60 msgr2=0x7f4638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4638066f60 0x7f4638069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f464c009b80 tx=0x7f464c009190 comp rx=0 tx=0).stop 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 msgr2=0x7f465c1519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c1519a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f464400e610 tx=0x7f464400ead0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 shutdown_connections 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4638066f60 0x7f4638069420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f465c0cf930 0x7f465c06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f465c0ae020 0x7f465c1519a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 --2- 172.21.15.45:0/2532550205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f465c070c20 0x7f465c160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 >> 172.21.15.45:0/2532550205 conn(0x7f465c066b70 msgr2=0x7f465c068c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 shutdown_connections 2024-04-25T10:24:03.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.729+0000 7f4662e00640 1 -- 172.21.15.45:0/2532550205 wait complete. 2024-04-25T10:24:03.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:03.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.889+0000 7f879c800640 1 Processor -- start 2024-04-25T10:24:03.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.889+0000 7f879c800640 1 -- start start 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f879c800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8794156150 0x7f8794071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f879c800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87940670b0 0x7f87940af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f879c800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8794156530 con 0x7f879415c080 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f879c800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879415d220 con 0x7f8794156150 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f879c800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879415c640 con 0x7f87940670b0 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8794156150 0x7f8794071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8794156150 0x7f8794071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46614/0 (socket says 172.21.15.45:46614) 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 -- 172.21.15.45:0/260058322 learned_addr learned my addr 172.21.15.45:0/260058322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f879415c080 0x7f87940669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40254/0 (socket says 172.21.15.45:40254) 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f879a800640 1 --2- 172.21.15.45:0/260058322 >> v2:172.21.15.117:3300/0 conn(0x7f87940670b0 0x7f87940af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 -- 172.21.15.45:0/260058322 >> v2:172.21.15.117:3300/0 conn(0x7f87940670b0 msgr2=0x7f87940af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 --2- 172.21.15.45:0/260058322 >> v2:172.21.15.117:3300/0 conn(0x7f87940670b0 0x7f87940af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 -- 172.21.15.45:0/260058322 >> v1:172.21.15.45:6789/0 conn(0x7f879415c080 legacy=0x7f87940669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8799e00640 1 -- 172.21.15.45:0/260058322 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879415c920 con 0x7f8794156150 2024-04-25T10:24:03.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.893+0000 7f8798a00640 1 -- 172.21.15.45:0/260058322 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901823665 0 0) 0x7f8794156530 con 0x7f879415c080 2024-04-25T10:24:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f8799e00640 1 --2- 172.21.15.45:0/260058322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f8794071f30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f879406bcb0 tx=0x7f878801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a9f4c59d6288db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f8798a00640 1 -- 172.21.15.45:0/260058322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880aa650 con 0x7f8794156150 2024-04-25T10:24:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f8798a00640 1 -- 172.21.15.45:0/260058322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87880af070 con 0x7f8794156150 2024-04-25T10:24:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f8798a00640 1 -- 172.21.15.45:0/260058322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880aaa10 con 0x7f8794156150 2024-04-25T10:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- 172.21.15.45:0/260058322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 msgr2=0x7f8794071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 --2- 172.21.15.45:0/260058322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f8794071f30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f879406bcb0 tx=0x7f878801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- 172.21.15.45:0/260058322 shutdown_connections 2024-04-25T10:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 --2- 172.21.15.45:0/260058322 >> v2:172.21.15.117:3300/0 conn(0x7f87940670b0 0x7f87940af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 --2- 172.21.15.45:0/260058322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f8794071f30 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- 172.21.15.45:0/260058322 >> 172.21.15.45:0/260058322 conn(0x7f879406ac90 msgr2=0x7f879406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:03.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- 172.21.15.45:0/260058322 shutdown_connections 2024-04-25T10:24:03.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- 172.21.15.45:0/260058322 wait complete. 2024-04-25T10:24:03.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 Processor -- start 2024-04-25T10:24:03.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- start start 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87940670b0 0x7f87941f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f87941f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879415c080 0x7f8794154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879415d880 con 0x7f8794156150 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879415d700 con 0x7f879415c080 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.897+0000 7f879c800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879415da00 con 0x7f87940670b0 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f87941f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f87941f9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46628/0 (socket says 172.21.15.45:46628) 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 -- 172.21.15.45:0/2971820589 learned_addr learned my addr 172.21.15.45:0/2971820589 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f879a800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879415c080 0x7f8794154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799e00640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87940670b0 0x7f87941f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 -- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87940670b0 msgr2=0x7f87941f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87940670b0 0x7f87941f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 -- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879415c080 msgr2=0x7f8794154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879415c080 0x7f8794154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 -- 172.21.15.45:0/2971820589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794153980 con 0x7f8794156150 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f8799400640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f87941f9870 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f878400c4b0 tx=0x7f878400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784018070 con 0x7f8794156150 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784007500 con 0x7f8794156150 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784004a30 con 0x7f8794156150 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879414f160 con 0x7f8794156150 2024-04-25T10:24:03.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879414f720 con 0x7f8794156150 2024-04-25T10:24:03.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879404ef30 con 0x7f8794156150 2024-04-25T10:24:03.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8784004bd0 con 0x7f8794156150 2024-04-25T10:24:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.901+0000 7f878e000640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8778066f60 0x7f8778069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.905+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8784084240 con 0x7f8794156150 2024-04-25T10:24:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.905+0000 7f8799e00640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8778066f60 0x7f8778069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:03.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.905+0000 7f8799e00640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8778066f60 0x7f8778069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f87880ae000 tx=0x7f87880a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:03.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:03.905+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8784054b90 con 0x7f8794156150 2024-04-25T10:24:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.073+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f879414e6b0 con 0x7f8794156150 2024-04-25T10:24:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.073+0000 7f878e000640 1 -- 172.21.15.45:0/2971820589 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16759 (secure 0 0 0) 0x7f8784058c30 con 0x7f8794156150 2024-04-25T10:24:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:04.074 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:23:40.632586+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":80,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.0","mappings":[{"from":2,"to":7}]},{"pgid":"11.3d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8778066f60 msgr2=0x7f8778069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8778066f60 0x7f8778069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f87880ae000 tx=0x7f87880a7000 comp rx=0 tx=0).stop 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 msgr2=0x7f87941f9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f87941f9870 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f878400c4b0 tx=0x7f878400c970 comp rx=0 tx=0).stop 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 shutdown_connections 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8778066f60 0x7f8778069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f879415c080 0x7f8794154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794156150 0x7f87941f9870 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 --2- 172.21.15.45:0/2971820589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87940670b0 0x7f87941f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 >> 172.21.15.45:0/2971820589 conn(0x7f879406ac90 msgr2=0x7f879406d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 shutdown_connections 2024-04-25T10:24:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.077+0000 7f879c800640 1 -- 172.21.15.45:0/2971820589 wait complete. 2024-04-25T10:24:04.095 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:24:04.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.233+0000 7eff64400640 1 Processor -- start 2024-04-25T10:24:04.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.233+0000 7eff64400640 1 -- start start 2024-04-25T10:24:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff64400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff64400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7eff5c0cda10 0x7eff5c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff64400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c0b2770 con 0x7eff5c0cf930 2024-04-25T10:24:04.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff64400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c06c670 con 0x7eff5c0cda10 2024-04-25T10:24:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff64400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c06c950 con 0x7eff5c15bc40 2024-04-25T10:24:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff61000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7eff5c0cda10 0x7eff5c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff62400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7eff5c0cf930 0x7eff5c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40268/0 (socket says 172.21.15.45:40268) 2024-04-25T10:24:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff62400640 1 -- 172.21.15.45:0/267127599 learned_addr learned my addr 172.21.15.45:0/267127599 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:04.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff61a00640 1 --2- 172.21.15.45:0/267127599 >> v2:172.21.15.117:3300/0 conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff61000640 1 -- 172.21.15.45:0/267127599 >> v2:172.21.15.117:3300/0 conn(0x7eff5c15bc40 msgr2=0x7eff5c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff61000640 1 --2- 172.21.15.45:0/267127599 >> v2:172.21.15.117:3300/0 conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff61000640 1 -- 172.21.15.45:0/267127599 >> v1:172.21.15.45:6789/0 conn(0x7eff5c0cf930 legacy=0x7eff5c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff61000640 1 -- 172.21.15.45:0/267127599 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c06cc30 con 0x7eff5c0cda10 2024-04-25T10:24:04.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.237+0000 7eff5be00640 1 -- 172.21.15.45:0/267127599 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691957840 0 0) 0x7eff5c0b2770 con 0x7eff5c0cf930 2024-04-25T10:24:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff61000640 1 --2- 172.21.15.45:0/267127599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c0cda10 0x7eff5c0b4610 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0d0800 tx=0x7eff4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dfa7c3e58139c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff5be00640 1 -- 172.21.15.45:0/267127599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480b1070 con 0x7eff5c0cda10 2024-04-25T10:24:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff5be00640 1 -- 172.21.15.45:0/267127599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff480ac5f0 con 0x7eff5c0cda10 2024-04-25T10:24:04.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff5be00640 1 -- 172.21.15.45:0/267127599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480ac9c0 con 0x7eff5c0cda10 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- 172.21.15.45:0/267127599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c0cda10 msgr2=0x7eff5c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 --2- 172.21.15.45:0/267127599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c0cda10 0x7eff5c0b4610 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0d0800 tx=0x7eff4801c480 comp rx=0 tx=0).stop 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- 172.21.15.45:0/267127599 shutdown_connections 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 --2- 172.21.15.45:0/267127599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c0cda10 0x7eff5c0b4610 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 --2- 172.21.15.45:0/267127599 >> v2:172.21.15.117:3300/0 conn(0x7eff5c15bc40 0x7eff5c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- 172.21.15.45:0/267127599 >> 172.21.15.45:0/267127599 conn(0x7eff5c066b70 msgr2=0x7eff5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- 172.21.15.45:0/267127599 shutdown_connections 2024-04-25T10:24:04.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- 172.21.15.45:0/267127599 wait complete. 2024-04-25T10:24:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 Processor -- start 2024-04-25T10:24:04.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- start start 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff5c0cda10 0x7eff5c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff5c0cf930 0x7eff5c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 0x7eff5c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c168b50 con 0x7eff5c15bc40 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c1689d0 con 0x7eff5c0cda10 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.241+0000 7eff64400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c168cd0 con 0x7eff5c0cf930 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 0x7eff5c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 0x7eff5c0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46636/0 (socket says 172.21.15.45:46636) 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 -- 172.21.15.45:0/1348841707 learned_addr learned my addr 172.21.15.45:0/1348841707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff61a00640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff5c0cda10 0x7eff5c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff61000640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff5c0cf930 0x7eff5c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 -- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff5c0cf930 msgr2=0x7eff5c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff5c0cf930 0x7eff5c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 -- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff5c0cda10 msgr2=0x7eff5c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff5c0cda10 0x7eff5c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 -- 172.21.15.45:0/1348841707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c14ad40 con 0x7eff5c15bc40 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff61a00640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff5c0cda10 0x7eff5c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff62400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 0x7eff5c0add80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c700 tx=0x7eff5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54018070 con 0x7eff5c15bc40 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5400ce10 con 0x7eff5c15bc40 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c1548c0 con 0x7eff5c15bc40 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400a860 con 0x7eff5c15bc40 2024-04-25T10:24:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c154e80 con 0x7eff5c15bc40 2024-04-25T10:24:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c04ef30 con 0x7eff5c15bc40 2024-04-25T10:24:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.245+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff5401b070 con 0x7eff5c15bc40 2024-04-25T10:24:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.249+0000 7eff59600640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff3c066f60 0x7eff3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.249+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff540845f0 con 0x7eff5c15bc40 2024-04-25T10:24:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.249+0000 7eff61a00640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff3c066f60 0x7eff3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.249+0000 7eff61a00640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff3c066f60 0x7eff3c069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0aeb70 tx=0x7eff50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.253+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff54055590 con 0x7eff5c15bc40 2024-04-25T10:24:04.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.445+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff5c15f400 con 0x7eff5c15bc40 2024-04-25T10:24:04.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.445+0000 7eff59600640 1 -- 172.21.15.45:0/1348841707 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4943 (secure 0 0 0) 0x7eff54054e90 con 0x7eff5c15bc40 2024-04-25T10:24:04.448 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:04.448 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:22:26.898819+0000","modified":"2024-04-25T10:22:34.900344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3661919331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3661919331},{"type":"v1","addr":"172.21.15.45:6835","nonce":3661919331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:24:04.448 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2024-04-25T10:24:04.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff3c066f60 msgr2=0x7eff3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff3c066f60 0x7eff3c069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0aeb70 tx=0x7eff50008040 comp rx=0 tx=0).stop 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 msgr2=0x7eff5c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 0x7eff5c0add80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c700 tx=0x7eff5400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 shutdown_connections 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7eff3c066f60 0x7eff3c069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff5c15bc40 0x7eff5c0add80 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff5c0cf930 0x7eff5c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 --2- 172.21.15.45:0/1348841707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff5c0cda10 0x7eff5c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 >> 172.21.15.45:0/1348841707 conn(0x7eff5c066b70 msgr2=0x7eff5c14c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 shutdown_connections 2024-04-25T10:24:04.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.449+0000 7eff64400640 1 -- 172.21.15.45:0/1348841707 wait complete. 2024-04-25T10:24:04.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:04.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 Processor -- start 2024-04-25T10:24:04.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 -- start start 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe0e415bb60 0x7fe0e406e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe0e406d270 0x7fe0e40cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e406e860 con 0x7fe0e40cc9c0 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e40ce2b0 con 0x7fe0e406d270 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e9a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e406ee20 con 0x7fe0e415bb60 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0da000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe0e40cc9c0 0x7fe0e406cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40284/0 (socket says 172.21.15.45:40284) 2024-04-25T10:24:04.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0da000640 1 -- 172.21.15.45:0/4233326485 learned_addr learned my addr 172.21.15.45:0/4233326485 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:04.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e3400640 1 --2- 172.21.15.45:0/4233326485 >> v2:172.21.15.45:3300/0 conn(0x7fe0e406d270 0x7fe0e40cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e2a00640 1 --2- 172.21.15.45:0/4233326485 >> v2:172.21.15.117:3300/0 conn(0x7fe0e415bb60 0x7fe0e406e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e3400640 1 -- 172.21.15.45:0/4233326485 >> v2:172.21.15.117:3300/0 conn(0x7fe0e415bb60 msgr2=0x7fe0e406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:04.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e3400640 1 --2- 172.21.15.45:0/4233326485 >> v2:172.21.15.117:3300/0 conn(0x7fe0e415bb60 0x7fe0e406e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e3400640 1 -- 172.21.15.45:0/4233326485 >> v1:172.21.15.45:6789/0 conn(0x7fe0e40cc9c0 legacy=0x7fe0e406cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.609+0000 7fe0e3400640 1 -- 172.21.15.45:0/4233326485 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e406eb40 con 0x7fe0e406d270 2024-04-25T10:24:04.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e3400640 1 --2- 172.21.15.45:0/4233326485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e406d270 0x7fe0e40cece0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe0e406d8d0 tx=0x7fe0dc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ac3b5150244361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e2000640 1 -- 172.21.15.45:0/4233326485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0b1070 con 0x7fe0e406d270 2024-04-25T10:24:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e2000640 1 -- 172.21.15.45:0/4233326485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc0ac620 con 0x7fe0e406d270 2024-04-25T10:24:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e2000640 1 -- 172.21.15.45:0/4233326485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0ac9f0 con 0x7fe0e406d270 2024-04-25T10:24:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 -- 172.21.15.45:0/4233326485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e406d270 msgr2=0x7fe0e40cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/4233326485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e406d270 0x7fe0e40cece0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe0e406d8d0 tx=0x7fe0dc01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 -- 172.21.15.45:0/4233326485 shutdown_connections 2024-04-25T10:24:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/4233326485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e406d270 0x7fe0e40cece0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/4233326485 >> v2:172.21.15.117:3300/0 conn(0x7fe0e415bb60 0x7fe0e406e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 -- 172.21.15.45:0/4233326485 >> 172.21.15.45:0/4233326485 conn(0x7fe0e40cffb0 msgr2=0x7fe0e40d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 -- 172.21.15.45:0/4233326485 shutdown_connections 2024-04-25T10:24:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.613+0000 7fe0e9a00640 1 -- 172.21.15.45:0/4233326485 wait complete. 2024-04-25T10:24:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 Processor -- start 2024-04-25T10:24:04.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 -- start start 2024-04-25T10:24:04.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e406d270 0x7fe0e4150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0e40cc9c0 0x7fe0e4156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 0x7fe0e4156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40cfa30 con 0x7fe0e415bb60 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e415bf40 con 0x7fe0e406d270 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e40cfbb0 con 0x7fe0e40cc9c0 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 0x7fe0e4156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 0x7fe0e4156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46652/0 (socket says 172.21.15.45:46652) 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 -- 172.21.15.45:0/1310716484 learned_addr learned my addr 172.21.15.45:0/1310716484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0da000640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0e40cc9c0 0x7fe0e4156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e2a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e406d270 0x7fe0e4150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 -- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0e40cc9c0 msgr2=0x7fe0e4156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0e40cc9c0 0x7fe0e4156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 -- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e406d270 msgr2=0x7fe0e4150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e406d270 0x7fe0e4150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 -- 172.21.15.45:0/1310716484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e41518e0 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e3400640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 0x7fe0e4156bf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0af040 tx=0x7fe0dc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0b9070 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc0ace50 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0b1040 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4151bc0 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.617+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e4158720 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.621+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e404ef30 con 0x7fe0e415bb60 2024-04-25T10:24:04.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.621+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0dc0bc070 con 0x7fe0e415bb60 2024-04-25T10:24:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.621+0000 7fe0db400640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0c0066f90 0x7fe0c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.621+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe0dc11ec90 con 0x7fe0e415bb60 2024-04-25T10:24:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.621+0000 7fe0e2a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0c0066f90 0x7fe0c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.621+0000 7fe0e2a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0c0066f90 0x7fe0c0069450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009ac0 tx=0x7fe0d4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.625+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0dc016070 con 0x7fe0e415bb60 2024-04-25T10:24:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0e41596c0 con 0x7fe0e415bb60 2024-04-25T10:24:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0db400640 1 -- 172.21.15.45:0/1310716484 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe0dc0ef5f0 con 0x7fe0e415bb60 2024-04-25T10:24:04.822 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:04.822 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:22:26.898819+0000","modified":"2024-04-25T10:22:34.900344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6888":{"gid":6888,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3661919331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3661919331},{"type":"v1","addr":"172.21.15.45:6835","nonce":3661919331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:24:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 56 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0c0066f90 msgr2=0x7fe0c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0c0066f90 0x7fe0c0069450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009ac0 tx=0x7fe0d4009190 comp rx=0 tx=0).stop 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 msgr2=0x7fe0e4156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 0x7fe0e4156bf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0af040 tx=0x7fe0dc019160 comp rx=0 tx=0).stop 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.821+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 shutdown_connections 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0c0066f90 0x7fe0c0069450 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e415bb60 0x7fe0e4156bf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0e40cc9c0 0x7fe0e4156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 --2- 172.21.15.45:0/1310716484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0e406d270 0x7fe0e4150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 >> 172.21.15.45:0/1310716484 conn(0x7fe0e40cffb0 msgr2=0x7fe0e41431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 shutdown_connections 2024-04-25T10:24:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.825+0000 7fe0e9a00640 1 -- 172.21.15.45:0/1310716484 wait complete. 2024-04-25T10:24:04.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 Processor -- start 2024-04-25T10:24:04.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 -- start start 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9798146ed0 0x7f97981472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97980726b0 0x7f9798071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979806c600 con 0x7f9798071c80 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97980720f0 con 0x7f97980726b0 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f979ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9798161980 con 0x7f9798146ed0 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f9796800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9798071c80 0x7f979806c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44570/0 (socket says 172.21.15.45:44570) 2024-04-25T10:24:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f9796800640 1 -- 172.21.15.45:0/34286475 learned_addr learned my addr 172.21.15.45:0/34286475 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f978d400640 1 --2- 172.21.15.45:0/34286475 >> v2:172.21.15.117:3300/0 conn(0x7f97980726b0 0x7f9798071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f9795e00640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9798146ed0 0x7f97981472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f9795400640 1 -- 172.21.15.45:0/34286475 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650403769 0 0) 0x7f979806c600 con 0x7f9798071c80 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f9795400640 1 -- 172.21.15.45:0/34286475 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f977c003530 con 0x7f9798071c80 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f978d400640 1 -- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9798146ed0 msgr2=0x7f97981472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f978d400640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9798146ed0 0x7f97981472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f978d400640 1 -- 172.21.15.45:0/34286475 >> v1:172.21.15.45:6789/0 conn(0x7f9798071c80 legacy=0x7f979806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f978d400640 1 -- 172.21.15.45:0/34286475 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798161bd0 con 0x7f97980726b0 2024-04-25T10:24:04.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.985+0000 7f9795e00640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9798146ed0 0x7f97981472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f978d400640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980726b0 0x7f9798071740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9798161b00 tx=0x7f978001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc2a58958f952b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f9795400640 1 -- 172.21.15.45:0/34286475 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800a7070 con 0x7f97980726b0 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f9795400640 1 -- 172.21.15.45:0/34286475 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97800b0070 con 0x7f97980726b0 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f9795400640 1 -- 172.21.15.45:0/34286475 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800ab870 con 0x7f97980726b0 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 -- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980726b0 msgr2=0x7f9798071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980726b0 0x7f9798071740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9798161b00 tx=0x7f978001c470 comp rx=0 tx=0).stop 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 -- 172.21.15.45:0/34286475 shutdown_connections 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97980726b0 0x7f9798071740 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 --2- 172.21.15.45:0/34286475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9798146ed0 0x7f97981472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 -- 172.21.15.45:0/34286475 >> 172.21.15.45:0/34286475 conn(0x7f97980669a0 msgr2=0x7f9798066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 -- 172.21.15.45:0/34286475 shutdown_connections 2024-04-25T10:24:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.989+0000 7f979ca00640 1 -- 172.21.15.45:0/34286475 wait complete. 2024-04-25T10:24:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 Processor -- start 2024-04-25T10:24:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 -- start start 2024-04-25T10:24:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9798071c80 0x7f979815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980726b0 0x7f9798139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 0x7f979813a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798147bf0 con 0x7f9798146ed0 2024-04-25T10:24:04.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798147a70 con 0x7f9798071c80 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980ae850 con 0x7f97980726b0 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 0x7f979813a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 0x7f979813a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48806/0 (socket says 172.21.15.45:48806) 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 -- 172.21.15.45:0/405571739 learned_addr learned my addr 172.21.15.45:0/405571739 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9795e00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9798071c80 0x7f979815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f978d400640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980726b0 0x7f9798139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 -- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980726b0 msgr2=0x7f9798139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980726b0 0x7f9798139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 -- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9798071c80 msgr2=0x7f979815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9798071c80 0x7f979815d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 -- 172.21.15.45:0/405571739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979813ce70 con 0x7f9798146ed0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f9796800640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 0x7f979813a1a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f979000c480 tx=0x7f979000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790017070 con 0x7f9798146ed0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790005cf0 con 0x7f9798146ed0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97981423d0 con 0x7f9798146ed0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900072b0 con 0x7f9798146ed0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.993+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9798145650 con 0x7f9798146ed0 2024-04-25T10:24:04.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.997+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97981425a0 con 0x7f9798146ed0 2024-04-25T10:24:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.997+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f979001a070 con 0x7f9798146ed0 2024-04-25T10:24:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.997+0000 7f978ea00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066fb0 0x7f9774069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.997+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9790084400 con 0x7f9798146ed0 2024-04-25T10:24:04.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.997+0000 7f9795e00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066fb0 0x7f9774069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:04.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:04.997+0000 7f9795e00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066fb0 0x7f9774069470 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9788009bd0 tx=0x7f978800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.001+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f979813ce70 con 0x7f9798146ed0 2024-04-25T10:24:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.165+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f979804ef30 con 0x7f9798146ed0 2024-04-25T10:24:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.165+0000 7f978ea00640 1 -- 172.21.15.45:0/405571739 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16759 (secure 0 0 0) 0x7f9790054dd0 con 0x7f9798146ed0 2024-04-25T10:24:05.167 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:05.167 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:23:40.632586+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:22:25.323207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T10:22:25.891841+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":80,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.0","mappings":[{"from":2,"to":7}]},{"pgid":"11.3d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066fb0 msgr2=0x7f9774069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066fb0 0x7f9774069470 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9788009bd0 tx=0x7f978800b040 comp rx=0 tx=0).stop 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 msgr2=0x7f979813a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 0x7f979813a1a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f979000c480 tx=0x7f979000c940 comp rx=0 tx=0).stop 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 shutdown_connections 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9774066fb0 0x7f9774069470 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9798146ed0 0x7f979813a1a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97980726b0 0x7f9798139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 --2- 172.21.15.45:0/405571739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9798071c80 0x7f979815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 >> 172.21.15.45:0/405571739 conn(0x7f97980669a0 msgr2=0x7f97980ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 shutdown_connections 2024-04-25T10:24:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.169+0000 7f979ca00640 1 -- 172.21.15.45:0/405571739 wait complete. 2024-04-25T10:24:05.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:24:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6ebac00640 1 Processor -- start 2024-04-25T10:24:05.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6ebac00640 1 -- start start 2024-04-25T10:24:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6ebac00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6eb40d0040 0x7f6eb406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6ebac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40b2770 con 0x7f6eb40cda10 2024-04-25T10:24:05.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6ebac00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb406c670 con 0x7f6eb415bc40 2024-04-25T10:24:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6ebac00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb406c950 con 0x7f6eb40d0040 2024-04-25T10:24:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eabe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6eb40cda10 0x7f6eb40cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44578/0 (socket says 172.21.15.45:44578) 2024-04-25T10:24:05.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eabe00640 1 -- 172.21.15.45:0/2316360965 learned_addr learned my addr 172.21.15.45:0/2316360965 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703980047 0 0) 0x7f6eb40b2770 con 0x7f6eb40cda10 2024-04-25T10:24:05.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e98003530 con 0x7f6eb40cda10 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb8c00640 1 --2- 172.21.15.45:0/2316360965 >> v2:172.21.15.117:3300/0 conn(0x7f6eb40d0040 0x7f6eb406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3381422129 0 0) 0x7f6e98003530 con 0x7f6eb40cda10 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 >> v2:172.21.15.117:3300/0 conn(0x7f6eb40d0040 msgr2=0x7f6eb406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 --2- 172.21.15.45:0/2316360965 >> v2:172.21.15.117:3300/0 conn(0x7f6eb40d0040 0x7f6eb406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 >> v1:172.21.15.117:6789/0 conn(0x7f6eb415bc40 legacy=0x7f6eb40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb406cc30 con 0x7f6eb40cda10 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6e9c001680 con 0x7f6eb40cda10 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 >> v1:172.21.15.45:6789/0 conn(0x7f6eb40cda10 legacy=0x7f6eb40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.329+0000 7f6eb3400640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98003a10 0x7f6e98003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6e98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e98005220 0x7f6e9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb406c950 con 0x7f6e98004810 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800c170 con 0x7f6e98003a10 2024-04-25T10:24:05.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800c370 con 0x7f6e98005220 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6e98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3e00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98003a10 0x7f6e98003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb8c00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e98005220 0x7f6e9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 -- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e98005220 msgr2=0x7f6e9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e98005220 0x7f6e9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 -- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98003a10 msgr2=0x7f6e98003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e98003a10 0x7f6e98003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 -- 172.21.15.45:0/2316360965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98003530 con 0x7f6e98004810 2024-04-25T10:24:05.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb8c00640 1 -- 172.21.15.45:0/2316360965 reap_dead start 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eabe00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6e98004c90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c0e0 tx=0x7f6e9c01cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8768bd623d323ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c007c30 con 0x7f6e98004810 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c0a9020 con 0x7f6e98004810 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.333+0000 7f6eb3400640 1 -- 172.21.15.45:0/2316360965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c005ae0 con 0x7f6e98004810 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- 172.21.15.45:0/2316360965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 msgr2=0x7f6e98004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6e98004c90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00c0e0 tx=0x7f6e9c01cae0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- 172.21.15.45:0/2316360965 shutdown_connections 2024-04-25T10:24:05.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 --2- 172.21.15.45:0/2316360965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6e98004c90 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- 172.21.15.45:0/2316360965 >> 172.21.15.45:0/2316360965 conn(0x7f6eb4066b70 msgr2=0x7f6eb4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- 172.21.15.45:0/2316360965 shutdown_connections 2024-04-25T10:24:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- 172.21.15.45:0/2316360965 wait complete. 2024-04-25T10:24:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 Processor -- start 2024-04-25T10:24:05.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- start start 2024-04-25T10:24:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6eb414d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb415c190 0x7f6eb415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb41e6640 0x7f6eb41e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40b49a0 con 0x7f6e98004810 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40b4820 con 0x7f6eb415c190 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6ebac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb40b4b20 con 0x7f6eb41e6640 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6eb414d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6eb414d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48820/0 (socket says 172.21.15.45:48820) 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 -- 172.21.15.45:0/1814116934 learned_addr learned my addr 172.21.15.45:0/1814116934 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eabe00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb415c190 0x7f6eb415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb8c00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb41e6640 0x7f6eb41e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 -- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb41e6640 msgr2=0x7f6eb41e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb41e6640 0x7f6eb41e6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 -- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb415c190 msgr2=0x7f6eb415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb415c190 0x7f6eb415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eb3e00640 1 -- 172.21.15.45:0/1814116934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb41e55a0 con 0x7f6e98004810 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.337+0000 7f6eabe00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb415c190 0x7f6eb415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb3e00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6eb414d920 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4007820 tx=0x7f6ea400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40050d0 con 0x7f6e98004810 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4018070 con 0x7f6e98004810 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb41e5880 con 0x7f6e98004810 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4005490 con 0x7f6e98004810 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb41e5e40 con 0x7f6e98004810 2024-04-25T10:24:05.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb404ef30 con 0x7f6e98004810 2024-04-25T10:24:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ea400d730 con 0x7f6e98004810 2024-04-25T10:24:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb0c00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e90067100 0x7f6e900695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6ea4084ef0 con 0x7f6e98004810 2024-04-25T10:24:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.341+0000 7f6eabe00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e90067100 0x7f6e900695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.345+0000 7f6eabe00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e90067100 0x7f6e900695c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0aa000 tx=0x7f6eb406c840 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.345+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ea4055e00 con 0x7f6e98004810 2024-04-25T10:24:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.541+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6eb40af1c0 con 0x7f6e98004810 2024-04-25T10:24:05.543 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:24:05.545+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:24:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.545+0000 7f6eb0c00640 1 -- 172.21.15.45:0/1814116934 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6ea4055700 con 0x7f6e98004810 2024-04-25T10:24:05.545 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:24:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e90067100 msgr2=0x7f6e900695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e90067100 0x7f6e900695c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0aa000 tx=0x7f6eb406c840 comp rx=0 tx=0).stop 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 msgr2=0x7f6eb414d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6eb414d920 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4007820 tx=0x7f6ea400fc00 comp rx=0 tx=0).stop 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 shutdown_connections 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6e90067100 0x7f6e900695c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6eb41e6640 0x7f6eb41e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6eb415c190 0x7f6eb415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 --2- 172.21.15.45:0/1814116934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e98004810 0x7f6eb414d920 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 >> 172.21.15.45:0/1814116934 conn(0x7f6eb4066b70 msgr2=0x7f6eb40b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 shutdown_connections 2024-04-25T10:24:05.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.549+0000 7f6ebac00640 1 -- 172.21.15.45:0/1814116934 wait complete. 2024-04-25T10:24:05.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:24:05.630 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:24:05.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.725+0000 7f1669000640 1 Processor -- start 2024-04-25T10:24:05.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.725+0000 7f1669000640 1 -- start start 2024-04-25T10:24:05.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.725+0000 7f1669000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16640aef80 0x7f166415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1669000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16640b31c0 0x7f16640b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1669000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166415b080 con 0x7f1664070350 2024-04-25T10:24:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1669000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166415adb0 con 0x7f16640aef80 2024-04-25T10:24:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1669000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166415eb60 con 0x7f16640b31c0 2024-04-25T10:24:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16640aef80 0x7f166415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16640aef80 0x7f166415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48846/0 (socket says 172.21.15.45:48846) 2024-04-25T10:24:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 -- 172.21.15.45:0/626890604 learned_addr learned my addr 172.21.15.45:0/626890604 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1659400640 1 --1- 172.21.15.45:0/626890604 >> v1:172.21.15.45:6789/0 conn(0x7f1664070350 0x7f166415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44602/0 (socket says 172.21.15.45:44602) 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1662800640 1 --2- 172.21.15.45:0/626890604 >> v2:172.21.15.117:3300/0 conn(0x7f16640b31c0 0x7f16640b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 -- 172.21.15.45:0/626890604 >> v2:172.21.15.117:3300/0 conn(0x7f16640b31c0 msgr2=0x7f16640b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 --2- 172.21.15.45:0/626890604 >> v2:172.21.15.117:3300/0 conn(0x7f16640b31c0 0x7f16640b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 -- 172.21.15.45:0/626890604 >> v1:172.21.15.45:6789/0 conn(0x7f1664070350 legacy=0x7f166415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 -- 172.21.15.45:0/626890604 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166415ee40 con 0x7f16640aef80 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1662800640 1 --2- 172.21.15.45:0/626890604 >> v2:172.21.15.117:3300/0 conn(0x7f16640b31c0 0x7f16640b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:05.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661400640 1 -- 172.21.15.45:0/626890604 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805363865 0 0) 0x7f166415b080 con 0x7f1664070350 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661e00640 1 --2- 172.21.15.45:0/626890604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16640aef80 0x7f166415e480 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f16640d0750 tx=0x7f165401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99fcafa2d124ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661400640 1 -- 172.21.15.45:0/626890604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16540ab430 con 0x7f16640aef80 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661400640 1 -- 172.21.15.45:0/626890604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16540b0070 con 0x7f16640aef80 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.729+0000 7f1661400640 1 -- 172.21.15.45:0/626890604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16540ab7f0 con 0x7f16640aef80 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- 172.21.15.45:0/626890604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16640aef80 msgr2=0x7f166415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 --2- 172.21.15.45:0/626890604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16640aef80 0x7f166415e480 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f16640d0750 tx=0x7f165401c520 comp rx=0 tx=0).stop 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- 172.21.15.45:0/626890604 shutdown_connections 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 --2- 172.21.15.45:0/626890604 >> v2:172.21.15.117:3300/0 conn(0x7f16640b31c0 0x7f16640b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 --2- 172.21.15.45:0/626890604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16640aef80 0x7f166415e480 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- 172.21.15.45:0/626890604 >> 172.21.15.45:0/626890604 conn(0x7f16640669a0 msgr2=0x7f1664066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:05.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- 172.21.15.45:0/626890604 shutdown_connections 2024-04-25T10:24:05.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- 172.21.15.45:0/626890604 wait complete. 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 Processor -- start 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- start start 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 0x7f1664159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16640aef80 0x7f16640cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16640b31c0 0x7f16641536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16641679a0 con 0x7f1664070350 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1664167820 con 0x7f16640aef80 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1669000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1664167b20 con 0x7f16640b31c0 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 0x7f1664159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 0x7f1664159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48860/0 (socket says 172.21.15.45:48860) 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 -- 172.21.15.45:0/559217128 learned_addr learned my addr 172.21.15.45:0/559217128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1662800640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16640b31c0 0x7f16641536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1659400640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16640aef80 0x7f16640cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 -- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16640b31c0 msgr2=0x7f16641536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16640b31c0 0x7f16641536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 -- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16640aef80 msgr2=0x7f16640cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16640aef80 0x7f16640cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1661e00640 1 -- 172.21.15.45:0/559217128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664071830 con 0x7f1664070350 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1662800640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16640b31c0 0x7f16641536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.733+0000 7f1659400640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16640aef80 0x7f16640cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f1661e00640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 0x7f1664159a10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f16540ad040 tx=0x7f165401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16540ba070 con 0x7f1664070350 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16540abd10 con 0x7f1664070350 2024-04-25T10:24:05.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1654004220 con 0x7f1664070350 2024-04-25T10:24:05.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16640cdb80 con 0x7f1664070350 2024-04-25T10:24:05.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16640ad8e0 con 0x7f1664070350 2024-04-25T10:24:05.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166404ef30 con 0x7f1664070350 2024-04-25T10:24:05.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1654003620 con 0x7f1664070350 2024-04-25T10:24:05.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f165aa00640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1640066f60 0x7f1640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:05.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f16540a8030 con 0x7f1664070350 2024-04-25T10:24:05.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.737+0000 7f1659400640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1640066f60 0x7f1640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:05.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.741+0000 7f1659400640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1640066f60 0x7f1640069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f164c005fd0 tx=0x7f164c005b80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.741+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16540c3060 con 0x7f1664070350 2024-04-25T10:24:05.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:05.937+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f16640b2fa0 con 0x7f1664070350 2024-04-25T10:24:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f165aa00640 1 -- 172.21.15.45:0/559217128 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1654016110 con 0x7f1664070350 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1640066f60 msgr2=0x7f1640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1640066f60 0x7f1640069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f164c005fd0 tx=0x7f164c005b80 comp rx=0 tx=0).stop 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 msgr2=0x7f1664159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 0x7f1664159a10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f16540ad040 tx=0x7f165401ad60 comp rx=0 tx=0).stop 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 shutdown_connections 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1640066f60 0x7f1640069420 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16640b31c0 0x7f16641536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16640aef80 0x7f16640cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 --2- 172.21.15.45:0/559217128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1664070350 0x7f1664159a10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 >> 172.21.15.45:0/559217128 conn(0x7f16640669a0 msgr2=0x7f16640b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 shutdown_connections 2024-04-25T10:24:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.549+0000 7f1669000640 1 -- 172.21.15.45:0/559217128 wait complete. 2024-04-25T10:24:06.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:24:06.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.721+0000 7f5b5ae00640 1 Processor -- start 2024-04-25T10:24:06.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b5ae00640 1 -- start start 2024-04-25T10:24:06.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b5ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b540cf040 0x7f5b540cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b5ae00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b54071f80 con 0x7f5b54071ab0 2024-04-25T10:24:06.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b5ae00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5415b120 con 0x7f5b5415bc00 2024-04-25T10:24:06.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b5ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54072260 con 0x7f5b540cf040 2024-04-25T10:24:06.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b58e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5b54071ab0 0x7f5b540ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44614/0 (socket says 172.21.15.45:44614) 2024-04-25T10:24:06.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b58e00640 1 -- 172.21.15.45:0/2284767601 learned_addr learned my addr 172.21.15.45:0/2284767601 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:06.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53e00640 1 --2- 172.21.15.45:0/2284767601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b540cf040 0x7f5b540cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962838147 0 0) 0x7f5b54071f80 con 0x7f5b54071ab0 2024-04-25T10:24:06.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b38005740 con 0x7f5b54071ab0 2024-04-25T10:24:06.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637482975 0 0) 0x7f5b5415b120 con 0x7f5b5415bc00 2024-04-25T10:24:06.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b54071f80 con 0x7f5b5415bc00 2024-04-25T10:24:06.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53e00640 1 -- 172.21.15.45:0/2284767601 >> v1:172.21.15.117:6789/0 conn(0x7f5b5415bc00 legacy=0x7f5b540713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53e00640 1 -- 172.21.15.45:0/2284767601 >> v1:172.21.15.45:6789/0 conn(0x7f5b54071ab0 legacy=0x7f5b540ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695086183 0 0) 0x7f5b38005740 con 0x7f5b54071ab0 2024-04-25T10:24:06.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.725+0000 7f5b53e00640 1 -- 172.21.15.45:0/2284767601 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5415b400 con 0x7f5b540cf040 2024-04-25T10:24:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b53e00640 1 --2- 172.21.15.45:0/2284767601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b540cf040 0x7f5b540cf9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b440097f0 tx=0x7f5b4401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d367cfd13b7a276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b0070 con 0x7f5b540cf040 2024-04-25T10:24:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b440ab400 con 0x7f5b540cf040 2024-04-25T10:24:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b53400640 1 -- 172.21.15.45:0/2284767601 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c001a00 con 0x7f5b540cf040 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 -- 172.21.15.45:0/2284767601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b540cf040 msgr2=0x7f5b540cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/2284767601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b540cf040 0x7f5b540cf9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b440097f0 tx=0x7f5b4401c400 comp rx=0 tx=0).stop 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 -- 172.21.15.45:0/2284767601 shutdown_connections 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/2284767601 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b540cf040 0x7f5b540cf9e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 -- 172.21.15.45:0/2284767601 >> 172.21.15.45:0/2284767601 conn(0x7f5b54066b70 msgr2=0x7f5b54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 -- 172.21.15.45:0/2284767601 shutdown_connections 2024-04-25T10:24:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 -- 172.21.15.45:0/2284767601 wait complete. 2024-04-25T10:24:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.729+0000 7f5b5ae00640 1 Processor -- start 2024-04-25T10:24:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- start start 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 0x7f5b5414bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b540cf040 0x7f5b54154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b5415bc00 0x7f5b540afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5406c370 con 0x7f5b54071ab0 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b540cf630 con 0x7f5b540cf040 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5406c4f0 con 0x7f5b5415bc00 2024-04-25T10:24:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 0x7f5b5414bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 0x7f5b5414bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48872/0 (socket says 172.21.15.45:48872) 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 -- 172.21.15.45:0/4257857870 learned_addr learned my addr 172.21.15.45:0/4257857870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b4b400640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b540cf040 0x7f5b54154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b58e00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b5415bc00 0x7f5b540afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 -- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b5415bc00 msgr2=0x7f5b540afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b5415bc00 0x7f5b540afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 -- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b540cf040 msgr2=0x7f5b54154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b540cf040 0x7f5b54154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 -- 172.21.15.45:0/4257857870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5414c530 con 0x7f5b54071ab0 2024-04-25T10:24:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b53e00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 0x7f5b5414bff0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5b440ae040 tx=0x7f5b4401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440bb020 con 0x7f5b54071ab0 2024-04-25T10:24:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b440b1040 con 0x7f5b54071ab0 2024-04-25T10:24:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5415cbb0 con 0x7f5b54071ab0 2024-04-25T10:24:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b0050 con 0x7f5b54071ab0 2024-04-25T10:24:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5415d170 con 0x7f5b54071ab0 2024-04-25T10:24:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.733+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5404ef30 con 0x7f5b54071ab0 2024-04-25T10:24:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.737+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b440be070 con 0x7f5b54071ab0 2024-04-25T10:24:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.737+0000 7f5b50c00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b30066f60 0x7f5b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.737+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5b4411e6e0 con 0x7f5b54071ab0 2024-04-25T10:24:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.737+0000 7f5b4b400640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b30066f60 0x7f5b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.737+0000 7f5b4b400640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b30066f60 0x7f5b30069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b540b4070 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.741+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b44015ff0 con 0x7f5b54071ab0 2024-04-25T10:24:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:06.905+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5b540b2a10 con 0x7f5b54071ab0 2024-04-25T10:24:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.761+0000 7f5b50c00640 1 -- 172.21.15.45:0/4257857870 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5b440ef650 con 0x7f5b54071ab0 2024-04-25T10:24:07.763 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:24:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b30066f60 msgr2=0x7f5b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b30066f60 0x7f5b30069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b540b4070 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2024-04-25T10:24:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 msgr2=0x7f5b5414bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:07.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 0x7f5b5414bff0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5b440ae040 tx=0x7f5b4401ac40 comp rx=0 tx=0).stop 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 shutdown_connections 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b30066f60 0x7f5b30069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b5415bc00 0x7f5b540afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b540cf040 0x7f5b54154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 --2- 172.21.15.45:0/4257857870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54071ab0 0x7f5b5414bff0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 >> 172.21.15.45:0/4257857870 conn(0x7f5b54066b70 msgr2=0x7f5b5406d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 shutdown_connections 2024-04-25T10:24:07.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.765+0000 7f5b5ae00640 1 -- 172.21.15.45:0/4257857870 wait complete. 2024-04-25T10:24:07.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:24:07.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.945+0000 7f5923200640 1 Processor -- start 2024-04-25T10:24:07.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.945+0000 7f5923200640 1 -- start start 2024-04-25T10:24:07.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5923200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f591c071860 0x7f591c070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:07.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f591c06c3c0 0x7f591c06c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:33674/0 (socket says 172.21.15.45:33674) 2024-04-25T10:24:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5923200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f591c15a950 0x7f591c0b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5923200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c15ae00 con 0x7f591c06c3c0 2024-04-25T10:24:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921e00640 1 -- 172.21.15.45:0/3571793604 learned_addr learned my addr 172.21.15.45:0/3571793604 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0707c0 con 0x7f591c071860 2024-04-25T10:24:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c070aa0 con 0x7f591c15a950 2024-04-25T10:24:07.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921400640 1 --2- 172.21.15.45:0/3571793604 >> v2:172.21.15.45:3300/0 conn(0x7f591c071860 0x7f591c070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5920a00640 1 -- 172.21.15.45:0/3571793604 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355505999 0 0) 0x7f591c15ae00 con 0x7f591c06c3c0 2024-04-25T10:24:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5920a00640 1 -- 172.21.15.45:0/3571793604 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5904003450 con 0x7f591c06c3c0 2024-04-25T10:24:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921400640 1 -- 172.21.15.45:0/3571793604 >> v2:172.21.15.117:3300/0 conn(0x7f591c15a950 msgr2=0x7f591c0b2c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:24:07.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921400640 1 --2- 172.21.15.45:0/3571793604 >> v2:172.21.15.117:3300/0 conn(0x7f591c15a950 0x7f591c0b2c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921400640 1 -- 172.21.15.45:0/3571793604 >> v1:172.21.15.117:6789/0 conn(0x7f591c06c3c0 legacy=0x7f591c06c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:07.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.949+0000 7f5921400640 1 -- 172.21.15.45:0/3571793604 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c070d80 con 0x7f591c071860 2024-04-25T10:24:07.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5921400640 1 --2- 172.21.15.45:0/3571793604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c071860 0x7f591c070130 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f590c009fd0 tx=0x7f590c020820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54aabf5559456ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5920a00640 1 -- 172.21.15.45:0/3571793604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0b6070 con 0x7f591c071860 2024-04-25T10:24:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5920a00640 1 -- 172.21.15.45:0/3571793604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c0a97e0 con 0x7f591c071860 2024-04-25T10:24:07.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5920a00640 1 -- 172.21.15.45:0/3571793604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0a9bb0 con 0x7f591c071860 2024-04-25T10:24:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c071860 msgr2=0x7f591c070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 --2- 172.21.15.45:0/3571793604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c071860 0x7f591c070130 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f590c009fd0 tx=0x7f590c020820 comp rx=0 tx=0).stop 2024-04-25T10:24:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 shutdown_connections 2024-04-25T10:24:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 --2- 172.21.15.45:0/3571793604 >> v2:172.21.15.117:3300/0 conn(0x7f591c15a950 0x7f591c0b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 --2- 172.21.15.45:0/3571793604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c071860 0x7f591c070130 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 >> 172.21.15.45:0/3571793604 conn(0x7f591c066da0 msgr2=0x7f591c0671b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 shutdown_connections 2024-04-25T10:24:07.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.957+0000 7f5923200640 1 -- 172.21.15.45:0/3571793604 wait complete. 2024-04-25T10:24:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 Processor -- start 2024-04-25T10:24:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 -- start start 2024-04-25T10:24:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c06c3c0 0x7f591c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c071860 0x7f591c1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 0x7f591c149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c166a50 con 0x7f591c15a950 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1668d0 con 0x7f591c071860 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c166bd0 con 0x7f591c06c3c0 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 0x7f591c149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 0x7f591c149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48886/0 (socket says 172.21.15.45:48886) 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 -- 172.21.15.45:0/4170751771 learned_addr learned my addr 172.21.15.45:0/4170751771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5921400640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c071860 0x7f591c1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5921e00640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c06c3c0 0x7f591c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 -- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c06c3c0 msgr2=0x7f591c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c06c3c0 0x7f591c15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 -- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c071860 msgr2=0x7f591c1498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c071860 0x7f591c1498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 -- 172.21.15.45:0/4170751771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c15cc50 con 0x7f591c15a950 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5922800640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 0x7f591c149e20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f591800b990 tx=0x7f591800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918019070 con 0x7f591c15a950 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591800cba0 con 0x7f591c15a950 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918014440 con 0x7f591c15a950 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c15cf30 con 0x7f591c15a950 2024-04-25T10:24:07.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.961+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c15d4f0 con 0x7f591c15a950 2024-04-25T10:24:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.965+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c075120 con 0x7f591c15a950 2024-04-25T10:24:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.965+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f591801c070 con 0x7f591c15a950 2024-04-25T10:24:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.965+0000 7f5916000640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5900066f60 0x7f5900069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:07.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.965+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f5918084c40 con 0x7f591c15a950 2024-04-25T10:24:07.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.965+0000 7f5921e00640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5900066f60 0x7f5900069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:07.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.965+0000 7f5921e00640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5900066f60 0x7f5900069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f59100011f0 tx=0x7f5910007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:07.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:07.969+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59180556c0 con 0x7f591c15a950 2024-04-25T10:24:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.133+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f591c0b27b0 con 0x7f591c15a950 2024-04-25T10:24:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.769+0000 7f5916000640 1 -- 172.21.15.45:0/4170751771 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5918059760 con 0x7f591c15a950 2024-04-25T10:24:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:24:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5900066f60 msgr2=0x7f5900069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5900066f60 0x7f5900069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f59100011f0 tx=0x7f5910007240 comp rx=0 tx=0).stop 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 msgr2=0x7f591c149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 0x7f591c149e20 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f591800b990 tx=0x7f591800be50 comp rx=0 tx=0).stop 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 shutdown_connections 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5900066f60 0x7f5900069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f591c15a950 0x7f591c149e20 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f591c071860 0x7f591c1498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 --2- 172.21.15.45:0/4170751771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f591c06c3c0 0x7f591c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 >> 172.21.15.45:0/4170751771 conn(0x7f591c066da0 msgr2=0x7f591c0cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 shutdown_connections 2024-04-25T10:24:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.773+0000 7f5923200640 1 -- 172.21.15.45:0/4170751771 wait complete. 2024-04-25T10:24:08.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 Processor -- start 2024-04-25T10:24:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 -- start start 2024-04-25T10:24:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5dd415dd30 0x7f5dd406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5dd40d1fb0 0x7f5dd40b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd406b1d0 con 0x7f5dd40cfb30 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd415e110 con 0x7f5dd415dd30 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd9600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd406b470 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5dd415dd30 0x7f5dd406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5dd415dd30 0x7f5dd406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48890/0 (socket says 172.21.15.45:48890) 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 -- 172.21.15.45:0/2938988041 learned_addr learned my addr 172.21.15.45:0/2938988041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 -- 172.21.15.45:0/2938988041 >> v2:172.21.15.117:3300/0 conn(0x7f5dd40d1fb0 msgr2=0x7f5dd40b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 --2- 172.21.15.45:0/2938988041 >> v2:172.21.15.117:3300/0 conn(0x7f5dd40d1fb0 0x7f5dd40b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 -- 172.21.15.45:0/2938988041 >> v1:172.21.15.117:6789/0 conn(0x7f5dd40cfb30 legacy=0x7f5dd40d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.949+0000 7f5dd2a00640 1 -- 172.21.15.45:0/2938988041 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd406b750 con 0x7f5dd415dd30 2024-04-25T10:24:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd2a00640 1 --2- 172.21.15.45:0/2938988041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd415dd30 0x7f5dd406ac90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40b1520 tx=0x7f5dc401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad58449a46a16183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd2000640 1 -- 172.21.15.45:0/2938988041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40b1070 con 0x7f5dd415dd30 2024-04-25T10:24:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd2000640 1 -- 172.21.15.45:0/2938988041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc40ac630 con 0x7f5dd415dd30 2024-04-25T10:24:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd2000640 1 -- 172.21.15.45:0/2938988041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40a8050 con 0x7f5dd415dd30 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 -- 172.21.15.45:0/2938988041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd415dd30 msgr2=0x7f5dd406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 --2- 172.21.15.45:0/2938988041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd415dd30 0x7f5dd406ac90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40b1520 tx=0x7f5dc401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 -- 172.21.15.45:0/2938988041 shutdown_connections 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 --2- 172.21.15.45:0/2938988041 >> v2:172.21.15.117:3300/0 conn(0x7f5dd40d1fb0 0x7f5dd40b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 --2- 172.21.15.45:0/2938988041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd415dd30 0x7f5dd406ac90 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 -- 172.21.15.45:0/2938988041 >> 172.21.15.45:0/2938988041 conn(0x7f5dd409a9d0 msgr2=0x7f5dd409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 -- 172.21.15.45:0/2938988041 shutdown_connections 2024-04-25T10:24:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.953+0000 7f5dd9600640 1 -- 172.21.15.45:0/2938988041 wait complete. 2024-04-25T10:24:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 Processor -- start 2024-04-25T10:24:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 -- start start 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd40cfb30 0x7f5dd40af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 0x7f5dd414f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd415dd30 0x7f5dd414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd416ab60 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b28d0 con 0x7f5dd415dd30 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd416ace0 con 0x7f5dd40cfb30 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 0x7f5dd414f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 0x7f5dd414f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48892/0 (socket says 172.21.15.45:48892) 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 -- 172.21.15.45:0/147489220 learned_addr learned my addr 172.21.15.45:0/147489220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd2a00640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd40cfb30 0x7f5dd40af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd3400640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd415dd30 0x7f5dd414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 -- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd40cfb30 msgr2=0x7f5dd40af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd40cfb30 0x7f5dd40af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 -- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd415dd30 msgr2=0x7f5dd414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd415dd30 0x7f5dd414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 -- 172.21.15.45:0/147489220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40b3120 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd3400640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd415dd30 0x7f5dd414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd2a00640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd40cfb30 0x7f5dd40af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dca000640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 0x7f5dd414f8d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c490 tx=0x7f5dbc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc003600 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc003a30 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd4159d10 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.957+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd415b370 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.961+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd404ef30 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.961+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dbc01a070 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.961+0000 7f5dcb400640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5db0066f90 0x7f5db0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.961+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5dbc084180 con 0x7f5dd40d1fb0 2024-04-25T10:24:08.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.961+0000 7f5dd2a00640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5db0066f90 0x7f5db0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:08.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.961+0000 7f5dd2a00640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5db0066f90 0x7f5db0069450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40ae000 tx=0x7f5dc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:08.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:08.965+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dbc055480 con 0x7f5dd40d1fb0 2024-04-25T10:24:09.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.129+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5dd415ae50 con 0x7f5dd40d1fb0 2024-04-25T10:24:09.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dcb400640 1 -- 172.21.15.45:0/147489220 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5dbc054d80 con 0x7f5dd40d1fb0 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5db0066f90 msgr2=0x7f5db0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5db0066f90 0x7f5db0069450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40ae000 tx=0x7f5dc4008040 comp rx=0 tx=0).stop 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 msgr2=0x7f5dd414f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 0x7f5dd414f8d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c490 tx=0x7f5dbc001e80 comp rx=0 tx=0).stop 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 shutdown_connections 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5db0066f90 0x7f5db0069450 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5dd415dd30 0x7f5dd414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5dd40d1fb0 0x7f5dd414f8d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 --2- 172.21.15.45:0/147489220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5dd40cfb30 0x7f5dd40af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 >> 172.21.15.45:0/147489220 conn(0x7f5dd409a9d0 msgr2=0x7f5dd40cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:09.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 shutdown_connections 2024-04-25T10:24:09.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.133+0000 7f5dd9600640 1 -- 172.21.15.45:0/147489220 wait complete. 2024-04-25T10:24:09.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:24:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8992e00640 1 Processor -- start 2024-04-25T10:24:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8992e00640 1 -- start start 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8992e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f898c0cda10 0x7f898c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8992e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c0b2770 con 0x7f898c15bc40 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8992e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c06c670 con 0x7f898c0cf930 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8992e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c06c950 con 0x7f898c0cda10 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8983e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f898c0cda10 0x7f898c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8983e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f898c0cda10 0x7f898c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48894/0 (socket says 172.21.15.45:48894) 2024-04-25T10:24:09.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8983e00640 1 -- 172.21.15.45:0/3924750444 learned_addr learned my addr 172.21.15.45:0/3924750444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:09.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8983e00640 1 -- 172.21.15.45:0/3924750444 >> v1:172.21.15.117:6789/0 conn(0x7f898c0cf930 legacy=0x7f898c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8983e00640 1 -- 172.21.15.45:0/3924750444 >> v1:172.21.15.45:6789/0 conn(0x7f898c15bc40 legacy=0x7f898c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.297+0000 7f8983e00640 1 -- 172.21.15.45:0/3924750444 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c06cc30 con 0x7f898c0cda10 2024-04-25T10:24:09.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8983e00640 1 --2- 172.21.15.45:0/3924750444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c0cda10 0x7f898c06c090 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8974009870 tx=0x7f897401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3dc8394751abbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f898b400640 1 -- 172.21.15.45:0/3924750444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740b0070 con 0x7f898c0cda10 2024-04-25T10:24:09.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f898b400640 1 -- 172.21.15.45:0/3924750444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89740ab600 con 0x7f898c0cda10 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f898b400640 1 -- 172.21.15.45:0/3924750444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740ab9d0 con 0x7f898c0cda10 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 -- 172.21.15.45:0/3924750444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c0cda10 msgr2=0x7f898c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 --2- 172.21.15.45:0/3924750444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c0cda10 0x7f898c06c090 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8974009870 tx=0x7f897401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 -- 172.21.15.45:0/3924750444 shutdown_connections 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 --2- 172.21.15.45:0/3924750444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c0cda10 0x7f898c06c090 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 -- 172.21.15.45:0/3924750444 >> 172.21.15.45:0/3924750444 conn(0x7f898c066b70 msgr2=0x7f898c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 -- 172.21.15.45:0/3924750444 shutdown_connections 2024-04-25T10:24:09.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.301+0000 7f8992e00640 1 -- 172.21.15.45:0/3924750444 wait complete. 2024-04-25T10:24:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 Processor -- start 2024-04-25T10:24:09.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- start start 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0cda10 0x7f898c14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0cf930 0x7f898c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 0x7f898c151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c169d70 con 0x7f898c15bc40 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c169bf0 con 0x7f898c0cda10 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c169ef0 con 0x7f898c0cf930 2024-04-25T10:24:09.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 0x7f898c151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 0x7f898c151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48906/0 (socket says 172.21.15.45:48906) 2024-04-25T10:24:09.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 -- 172.21.15.45:0/2663068234 learned_addr learned my addr 172.21.15.45:0/2663068234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:09.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f898be00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0cda10 0x7f898c14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8983e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0cf930 0x7f898c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 -- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0cf930 msgr2=0x7f898c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0cf930 0x7f898c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 -- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0cda10 msgr2=0x7f898c14d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0cda10 0x7f898c14d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 -- 172.21.15.45:0/2663068234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c15b4c0 con 0x7f898c15bc40 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f898be00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0cda10 0x7f898c14d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8990e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 0x7f898c151f60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f898400c770 tx=0x7f898400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984003b80 con 0x7f898c15bc40 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984003d20 con 0x7f898c15bc40 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c14e2e0 con 0x7f898c15bc40 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400aa90 con 0x7f898c15bc40 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c15d4d0 con 0x7f898c15bc40 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.305+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c04ef30 con 0x7f898c15bc40 2024-04-25T10:24:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.309+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f898400ac30 con 0x7f898c15bc40 2024-04-25T10:24:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.309+0000 7f8988c00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8968066f90 0x7f8968069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.309+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8984084d20 con 0x7f898c15bc40 2024-04-25T10:24:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.309+0000 7f898be00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8968066f90 0x7f8968069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.309+0000 7f898be00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8968066f90 0x7f8968069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f898c067130 tx=0x7f897c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:09.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.313+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8984055f90 con 0x7f898c15bc40 2024-04-25T10:24:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.549+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f898c158ef0 con 0x7f898c15bc40 2024-04-25T10:24:09.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.773+0000 7f8988c00640 1 -- 172.21.15.45:0/2663068234 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v301) v1 ==== 158+0+0 (secure 0 0 0) 0x7f8984055890 con 0x7f898c15bc40 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8968066f90 msgr2=0x7f8968069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8968066f90 0x7f8968069450 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f898c067130 tx=0x7f897c009040 comp rx=0 tx=0).stop 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 msgr2=0x7f898c151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 0x7f898c151f60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f898400c770 tx=0x7f898400cc30 comp rx=0 tx=0).stop 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 shutdown_connections 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8968066f90 0x7f8968069450 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f898c15bc40 0x7f898c151f60 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f898c0cf930 0x7f898c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 --2- 172.21.15.45:0/2663068234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f898c0cda10 0x7f898c14d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 >> 172.21.15.45:0/2663068234 conn(0x7f898c066b70 msgr2=0x7f898c159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:09.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 shutdown_connections 2024-04-25T10:24:09.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.777+0000 7f8992e00640 1 -- 172.21.15.45:0/2663068234 wait complete. 2024-04-25T10:24:09.798 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T10:24:09.798 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:24:09.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:09.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.937+0000 7fe118600640 1 Processor -- start 2024-04-25T10:24:09.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.937+0000 7fe118600640 1 -- start start 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe118600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe110156150 0x7fe110071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe118600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe11015c080 0x7fe1100669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe118600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110066ee0 0x7fe1100af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe118600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe110067330 con 0x7fe11015c080 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe118600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe110156530 con 0x7fe110156150 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe118600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015d220 con 0x7fe110066ee0 2024-04-25T10:24:09.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe11015c080 0x7fe1100669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe11015c080 0x7fe1100669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48912/0 (socket says 172.21.15.45:48912) 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 -- 172.21.15.45:0/3975073405 learned_addr learned my addr 172.21.15.45:0/3975073405 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115c00640 1 --2- 172.21.15.45:0/3975073405 >> v2:172.21.15.117:3300/0 conn(0x7fe110156150 0x7fe110071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe116600640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110066ee0 0x7fe1100af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 -- 172.21.15.45:0/3975073405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110066ee0 msgr2=0x7fe1100af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110066ee0 0x7fe1100af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 -- 172.21.15.45:0/3975073405 >> v2:172.21.15.117:3300/0 conn(0x7fe110156150 msgr2=0x7fe110071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 --2- 172.21.15.45:0/3975073405 >> v2:172.21.15.117:3300/0 conn(0x7fe110156150 0x7fe110071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115200640 1 -- 172.21.15.45:0/3975073405 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11015c640 con 0x7fe11015c080 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe115c00640 1 --2- 172.21.15.45:0/3975073405 >> v2:172.21.15.117:3300/0 conn(0x7fe110156150 0x7fe110071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:09.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.941+0000 7fe116600640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110066ee0 0x7fe1100af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:09.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe115200640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe1100669a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe110070ab0 tx=0x7fe0fc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a19f23e12ad5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:09.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe10fe00640 1 -- 172.21.15.45:0/3975073405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0aa610 con 0x7fe11015c080 2024-04-25T10:24:09.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe10fe00640 1 -- 172.21.15.45:0/3975073405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc0af070 con 0x7fe11015c080 2024-04-25T10:24:09.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe10fe00640 1 -- 172.21.15.45:0/3975073405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0aa9d0 con 0x7fe11015c080 2024-04-25T10:24:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 -- 172.21.15.45:0/3975073405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 msgr2=0x7fe1100669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe1100669a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe110070ab0 tx=0x7fe0fc01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 -- 172.21.15.45:0/3975073405 shutdown_connections 2024-04-25T10:24:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110066ee0 0x7fe1100af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 --2- 172.21.15.45:0/3975073405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe1100669a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 --2- 172.21.15.45:0/3975073405 >> v2:172.21.15.117:3300/0 conn(0x7fe110156150 0x7fe110071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 -- 172.21.15.45:0/3975073405 >> 172.21.15.45:0/3975073405 conn(0x7fe11006ac90 msgr2=0x7fe11006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:09.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 -- 172.21.15.45:0/3975073405 shutdown_connections 2024-04-25T10:24:09.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 -- 172.21.15.45:0/3975073405 wait complete. 2024-04-25T10:24:09.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 Processor -- start 2024-04-25T10:24:09.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.945+0000 7fe118600640 1 -- start start 2024-04-25T10:24:09.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110066ee0 0x7fe110155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110156150 0x7fe110142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe110146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11006d430 con 0x7fe11015c080 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110072470 con 0x7fe110066ee0 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11006d5b0 con 0x7fe110156150 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe110146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe110146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48922/0 (socket says 172.21.15.45:48922) 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 -- 172.21.15.45:0/1432622514 learned_addr learned my addr 172.21.15.45:0/1432622514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe115c00640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110066ee0 0x7fe110155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe115200640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110156150 0x7fe110142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 -- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110156150 msgr2=0x7fe110142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110156150 0x7fe110142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 -- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110066ee0 msgr2=0x7fe110155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:09.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110066ee0 0x7fe110155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:09.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 -- 172.21.15.45:0/1432622514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1100b4c50 con 0x7fe11015c080 2024-04-25T10:24:09.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe116600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe110146100 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe10800cef0 tx=0x7fe10800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10801a070 con 0x7fe11015c080 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10800d840 con 0x7fe11015c080 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11014b8c0 con 0x7fe11015c080 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800dc30 con 0x7fe11015c080 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe110142e40 con 0x7fe11015c080 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11014b6b0 con 0x7fe11015c080 2024-04-25T10:24:09.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.949+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe10801d070 con 0x7fe11015c080 2024-04-25T10:24:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.953+0000 7fe10d600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0f0066f90 0x7fe0f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.953+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe1080854c0 con 0x7fe11015c080 2024-04-25T10:24:09.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.953+0000 7fe115c00640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0f0066f90 0x7fe0f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:09.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.953+0000 7fe115c00640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0f0066f90 0x7fe0f0069450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe1100af6b0 tx=0x7fe104008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:09.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:09.957+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe108056170 con 0x7fe11015c080 2024-04-25T10:24:10.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.117+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe11006d9f0 con 0x7fe11015c080 2024-04-25T10:24:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe10d600640 1 -- 172.21.15.45:0/1432622514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+13009 (secure 0 0 0) 0x7fe10805a210 con 0x7fe11015c080 2024-04-25T10:24:10.120 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:10.120 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":84,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:08.764790+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/3661919331":"2024-04-26T10:24:05.545025+0000","172.21.15.45:6833/3661919331":"2024-04-26T10:24:05.545025+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0f0066f90 msgr2=0x7fe0f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0f0066f90 0x7fe0f0069450 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe1100af6b0 tx=0x7fe104008040 comp rx=0 tx=0).stop 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 msgr2=0x7fe110146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe110146100 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe10800cef0 tx=0x7fe10800a710 comp rx=0 tx=0).stop 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 shutdown_connections 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe0f0066f90 0x7fe0f0069450 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11015c080 0x7fe110146100 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe110156150 0x7fe110142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 --2- 172.21.15.45:0/1432622514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe110066ee0 0x7fe110155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 >> 172.21.15.45:0/1432622514 conn(0x7fe11006ac90 msgr2=0x7fe11006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 shutdown_connections 2024-04-25T10:24:10.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.121+0000 7fe118600640 1 -- 172.21.15.45:0/1432622514 wait complete. 2024-04-25T10:24:10.141 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:10.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:10.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 Processor -- start 2024-04-25T10:24:10.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 -- start start 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc130159540 0x7fc1300ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1300aebd0 0x7fc130071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc130071360 con 0x7fc130072020 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc130159920 con 0x7fc130159540 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc137400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13015c2f0 con 0x7fc1300aebd0 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc130159540 0x7fc1300ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc130159540 0x7fc1300ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48930/0 (socket says 172.21.15.45:48930) 2024-04-25T10:24:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 -- 172.21.15.45:0/2226362212 learned_addr learned my addr 172.21.15.45:0/2226362212 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:10.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc127e00640 1 --2- 172.21.15.45:0/2226362212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1300aebd0 0x7fc130071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 -- 172.21.15.45:0/2226362212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1300aebd0 msgr2=0x7fc130071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:10.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 --2- 172.21.15.45:0/2226362212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1300aebd0 0x7fc130071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 -- 172.21.15.45:0/2226362212 >> v1:172.21.15.117:6789/0 conn(0x7fc130072020 legacy=0x7fc13015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.285+0000 7fc134a00640 1 -- 172.21.15.45:0/2226362212 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13015c5d0 con 0x7fc130159540 2024-04-25T10:24:10.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc134a00640 1 --2- 172.21.15.45:0/2226362212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ae690 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc120009870 tx=0x7fc12001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828fb418c6b4ea0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:10.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc12fe00640 1 -- 172.21.15.45:0/2226362212 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1200b0070 con 0x7fc130159540 2024-04-25T10:24:10.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc12fe00640 1 -- 172.21.15.45:0/2226362212 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1200ab600 con 0x7fc130159540 2024-04-25T10:24:10.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc12fe00640 1 -- 172.21.15.45:0/2226362212 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1200ab9d0 con 0x7fc130159540 2024-04-25T10:24:10.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 -- 172.21.15.45:0/2226362212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 msgr2=0x7fc1300ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 --2- 172.21.15.45:0/2226362212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ae690 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc120009870 tx=0x7fc12001c560 comp rx=0 tx=0).stop 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 -- 172.21.15.45:0/2226362212 shutdown_connections 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 --2- 172.21.15.45:0/2226362212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1300aebd0 0x7fc130071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 --2- 172.21.15.45:0/2226362212 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ae690 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 -- 172.21.15.45:0/2226362212 >> 172.21.15.45:0/2226362212 conn(0x7fc1300669a0 msgr2=0x7fc130066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 -- 172.21.15.45:0/2226362212 shutdown_connections 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.289+0000 7fc137400640 1 -- 172.21.15.45:0/2226362212 wait complete. 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 Processor -- start 2024-04-25T10:24:10.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 -- start start 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc130072020 0x7fc130140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1300aebd0 0x7fc1300af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc130072430 con 0x7fc130159540 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13006c0d0 con 0x7fc1300aebd0 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc130168930 con 0x7fc130072020 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48944/0 (socket says 172.21.15.45:48944) 2024-04-25T10:24:10.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 -- 172.21.15.45:0/546085573 learned_addr learned my addr 172.21.15.45:0/546085573 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:10.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc134a00640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc130072020 0x7fc130140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc127e00640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1300aebd0 0x7fc1300af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 -- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc130072020 msgr2=0x7fc130140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc130072020 0x7fc130140240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 -- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1300aebd0 msgr2=0x7fc1300af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1300aebd0 0x7fc1300af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 -- 172.21.15.45:0/546085573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1300b2870 con 0x7fc130159540 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc135400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ad840 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c760 tx=0x7fc12800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128003ba0 con 0x7fc130159540 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc128003d40 con 0x7fc130159540 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12800aa90 con 0x7fc130159540 2024-04-25T10:24:10.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1301511c0 con 0x7fc130159540 2024-04-25T10:24:10.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.293+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc130151780 con 0x7fc130159540 2024-04-25T10:24:10.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.297+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13004ef30 con 0x7fc130159540 2024-04-25T10:24:10.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.297+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc12800ac30 con 0x7fc130159540 2024-04-25T10:24:10.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.297+0000 7fc12d600640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc10c066f90 0x7fc10c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.297+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc128084ac0 con 0x7fc130159540 2024-04-25T10:24:10.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.297+0000 7fc134a00640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc10c066f90 0x7fc10c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.301+0000 7fc134a00640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc10c066f90 0x7fc10c069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc1200ad000 tx=0x7fc120008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:10.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.301+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1280556e0 con 0x7fc130159540 2024-04-25T10:24:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc130166700 con 0x7fc130159540 2024-04-25T10:24:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc12d600640 1 -- 172.21.15.45:0/546085573 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc128059780 con 0x7fc130159540 2024-04-25T10:24:10.468 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:24:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc10c066f90 msgr2=0x7fc10c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc10c066f90 0x7fc10c069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc1200ad000 tx=0x7fc120008040 comp rx=0 tx=0).stop 2024-04-25T10:24:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 msgr2=0x7fc1300ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ad840 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c760 tx=0x7fc12800cc20 comp rx=0 tx=0).stop 2024-04-25T10:24:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 shutdown_connections 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc10c066f90 0x7fc10c069450 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc130159540 0x7fc1300ad840 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1300aebd0 0x7fc1300af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 --2- 172.21.15.45:0/546085573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc130072020 0x7fc130140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 >> 172.21.15.45:0/546085573 conn(0x7fc1300669a0 msgr2=0x7fc130066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 shutdown_connections 2024-04-25T10:24:10.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.469+0000 7fc137400640 1 -- 172.21.15.45:0/546085573 wait complete. 2024-04-25T10:24:10.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:24:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.629+0000 7f5109c00640 1 Processor -- start 2024-04-25T10:24:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.629+0000 7f5109c00640 1 -- start start 2024-04-25T10:24:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5109c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f510415df60 0x7f51040cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5109c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51040d0180 0x7f51040d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5109c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51040d0b20 con 0x7f51040cdb40 2024-04-25T10:24:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5109c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510415e340 con 0x7f51040d0180 2024-04-25T10:24:10.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5109c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040ccaa0 con 0x7f510415df60 2024-04-25T10:24:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5103400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f51040cdb40 0x7f51040cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44640/0 (socket says 172.21.15.45:44640) 2024-04-25T10:24:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5103400640 1 -- 172.21.15.45:0/1798805270 learned_addr learned my addr 172.21.15.45:0/1798805270 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102a00640 1 --2- 172.21.15.45:0/1798805270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f510415df60 0x7f51040cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f50fbe00640 1 --2- 172.21.15.45:0/1798805270 >> v2:172.21.15.117:3300/0 conn(0x7f51040d0180 0x7f51040d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102000640 1 -- 172.21.15.45:0/1798805270 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519542122 0 0) 0x7f51040d0b20 con 0x7f51040cdb40 2024-04-25T10:24:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102000640 1 -- 172.21.15.45:0/1798805270 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e8003530 con 0x7f51040cdb40 2024-04-25T10:24:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102a00640 1 -- 172.21.15.45:0/1798805270 >> v2:172.21.15.117:3300/0 conn(0x7f51040d0180 msgr2=0x7f51040d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102a00640 1 --2- 172.21.15.45:0/1798805270 >> v2:172.21.15.117:3300/0 conn(0x7f51040d0180 0x7f51040d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102a00640 1 -- 172.21.15.45:0/1798805270 >> v1:172.21.15.45:6789/0 conn(0x7f51040cdb40 legacy=0x7f51040cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.633+0000 7f5102a00640 1 -- 172.21.15.45:0/1798805270 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040ccd80 con 0x7f510415df60 2024-04-25T10:24:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5102a00640 1 --2- 172.21.15.45:0/1798805270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f510415df60 0x7f51040cfc40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f50f40097c0 tx=0x7f50f401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab21ee0954a067fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5102000640 1 -- 172.21.15.45:0/1798805270 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f40b0070 con 0x7f510415df60 2024-04-25T10:24:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5102000640 1 -- 172.21.15.45:0/1798805270 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f40ab600 con 0x7f510415df60 2024-04-25T10:24:10.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5102000640 1 -- 172.21.15.45:0/1798805270 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f40ab9d0 con 0x7f510415df60 2024-04-25T10:24:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 -- 172.21.15.45:0/1798805270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f510415df60 msgr2=0x7f51040cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 --2- 172.21.15.45:0/1798805270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f510415df60 0x7f51040cfc40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f50f40097c0 tx=0x7f50f401c410 comp rx=0 tx=0).stop 2024-04-25T10:24:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 -- 172.21.15.45:0/1798805270 shutdown_connections 2024-04-25T10:24:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 --2- 172.21.15.45:0/1798805270 >> v2:172.21.15.117:3300/0 conn(0x7f51040d0180 0x7f51040d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 --2- 172.21.15.45:0/1798805270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f510415df60 0x7f51040cfc40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 -- 172.21.15.45:0/1798805270 >> 172.21.15.45:0/1798805270 conn(0x7f51040d1910 msgr2=0x7f51040d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 -- 172.21.15.45:0/1798805270 shutdown_connections 2024-04-25T10:24:10.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 -- 172.21.15.45:0/1798805270 wait complete. 2024-04-25T10:24:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 Processor -- start 2024-04-25T10:24:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.637+0000 7f5109c00640 1 -- start start 2024-04-25T10:24:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51040cdb40 0x7f51040c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51040d0180 0x7f5104153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 0x7f51040ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040d0ff0 con 0x7f510415df60 2024-04-25T10:24:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040d0e70 con 0x7f51040d0180 2024-04-25T10:24:10.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040cead0 con 0x7f51040cdb40 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 0x7f51040ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5102a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51040cdb40 0x7f51040c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 0x7f51040ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48956/0 (socket says 172.21.15.45:48956) 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5102a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51040cdb40 0x7f51040c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:38750/0 (socket says 172.21.15.45:38750) 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 -- 172.21.15.45:0/725114551 learned_addr learned my addr 172.21.15.45:0/725114551 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f50fbe00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51040d0180 0x7f5104153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 -- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51040cdb40 msgr2=0x7f51040c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51040cdb40 0x7f51040c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 -- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51040d0180 msgr2=0x7f5104153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51040d0180 0x7f5104153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 -- 172.21.15.45:0/725114551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040ce1c0 con 0x7f510415df60 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5103400640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 0x7f51040ca840 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00cb40 tx=0x7f50fc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc017070 con 0x7f510415df60 2024-04-25T10:24:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc00aca0 con 0x7f510415df60 2024-04-25T10:24:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc012620 con 0x7f510415df60 2024-04-25T10:24:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51040cafe0 con 0x7f510415df60 2024-04-25T10:24:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51040cb5a0 con 0x7f510415df60 2024-04-25T10:24:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510404ef30 con 0x7f510415df60 2024-04-25T10:24:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.641+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50fc01a070 con 0x7f510415df60 2024-04-25T10:24:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.645+0000 7f50faa00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50e0066f60 0x7f50e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.645+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f50fc084b20 con 0x7f510415df60 2024-04-25T10:24:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.645+0000 7f5102a00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50e0066f60 0x7f50e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.645+0000 7f5102a00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50e0066f60 0x7f50e0069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50f40ad000 tx=0x7f50f40a4f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.649+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50fc055760 con 0x7f510415df60 2024-04-25T10:24:10.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:10.865+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f510406bb90 con 0x7f510415df60 2024-04-25T10:24:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f50faa00640 1 -- 172.21.15.45:0/725114551 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v303) v1 ==== 147+0+0 (secure 0 0 0) 0x7f50fc059800 con 0x7f510415df60 2024-04-25T10:24:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50e0066f60 msgr2=0x7f50e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50e0066f60 0x7f50e0069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50f40ad000 tx=0x7f50f40a4f90 comp rx=0 tx=0).stop 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 msgr2=0x7f51040ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 0x7f51040ca840 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00cb40 tx=0x7f50fc00a060 comp rx=0 tx=0).stop 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 shutdown_connections 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50e0066f60 0x7f50e0069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510415df60 0x7f51040ca840 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51040d0180 0x7f5104153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 --2- 172.21.15.45:0/725114551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51040cdb40 0x7f51040c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 >> 172.21.15.45:0/725114551 conn(0x7f51040d1910 msgr2=0x7f51040cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 shutdown_connections 2024-04-25T10:24:11.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.785+0000 7f5109c00640 1 -- 172.21.15.45:0/725114551 wait complete. 2024-04-25T10:24:11.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:24:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d58a00640 1 Processor -- start 2024-04-25T10:24:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d58a00640 1 -- start start 2024-04-25T10:24:11.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d58a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d5406e5e0 0x7f6d540af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d58a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d540b3820 con 0x7f6d541610b0 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d58a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d540b3a90 con 0x7f6d540670b0 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d58a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d540b3d70 con 0x7f6d5406e5e0 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d51e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6d541610b0 0x7f6d540669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44646/0 (socket says 172.21.15.45:44646) 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d51e00640 1 -- 172.21.15.45:0/4201560358 learned_addr learned my addr 172.21.15.45:0/4201560358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d52800640 1 --2- 172.21.15.45:0/4201560358 >> v2:172.21.15.45:3300/0 conn(0x7f6d5406e5e0 0x7f6d540af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d51400640 1 -- 172.21.15.45:0/4201560358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329373121 0 0) 0x7f6d540b3820 con 0x7f6d541610b0 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d51400640 1 -- 172.21.15.45:0/4201560358 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d38003530 con 0x7f6d541610b0 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d52800640 1 -- 172.21.15.45:0/4201560358 >> v1:172.21.15.117:6789/0 conn(0x7f6d540670b0 legacy=0x7f6d54161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d52800640 1 -- 172.21.15.45:0/4201560358 >> v1:172.21.15.45:6789/0 conn(0x7f6d541610b0 legacy=0x7f6d540669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.961+0000 7f6d52800640 1 -- 172.21.15.45:0/4201560358 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d540afee0 con 0x7f6d5406e5e0 2024-04-25T10:24:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d52800640 1 --2- 172.21.15.45:0/4201560358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d5406e5e0 0x7f6d540af600 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d90 tx=0x7f6d4c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72793ed0a232976f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d51400640 1 -- 172.21.15.45:0/4201560358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0b2070 con 0x7f6d5406e5e0 2024-04-25T10:24:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d51400640 1 -- 172.21.15.45:0/4201560358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0a5830 con 0x7f6d5406e5e0 2024-04-25T10:24:11.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d51400640 1 -- 172.21.15.45:0/4201560358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0a5be0 con 0x7f6d5406e5e0 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 -- 172.21.15.45:0/4201560358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d5406e5e0 msgr2=0x7f6d540af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 --2- 172.21.15.45:0/4201560358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d5406e5e0 0x7f6d540af600 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009d90 tx=0x7f6d4c01c970 comp rx=0 tx=0).stop 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 -- 172.21.15.45:0/4201560358 shutdown_connections 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 --2- 172.21.15.45:0/4201560358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d5406e5e0 0x7f6d540af600 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 -- 172.21.15.45:0/4201560358 >> 172.21.15.45:0/4201560358 conn(0x7f6d5406ac90 msgr2=0x7f6d5406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 -- 172.21.15.45:0/4201560358 shutdown_connections 2024-04-25T10:24:11.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.965+0000 7f6d58a00640 1 -- 172.21.15.45:0/4201560358 wait complete. 2024-04-25T10:24:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 Processor -- start 2024-04-25T10:24:11.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 -- start start 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 0x7f6d540cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5406e5e0 0x7f6d541565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d541610b0 0x7f6d5414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5416bda0 con 0x7f6d540670b0 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5416bc20 con 0x7f6d541610b0 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5416bf20 con 0x7f6d5406e5e0 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 0x7f6d540cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 0x7f6d540cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48966/0 (socket says 172.21.15.45:48966) 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 -- 172.21.15.45:0/1633453691 learned_addr learned my addr 172.21.15.45:0/1633453691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d4be00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5406e5e0 0x7f6d541565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d52800640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d541610b0 0x7f6d5414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 -- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5406e5e0 msgr2=0x7f6d541565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5406e5e0 0x7f6d541565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 -- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d541610b0 msgr2=0x7f6d5414ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d541610b0 0x7f6d5414ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 -- 172.21.15.45:0/1633453691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d54156ff0 con 0x7f6d540670b0 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d51e00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 0x7f6d540cd4e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c4d0 tx=0x7f6d4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44017070 con 0x7f6d540670b0 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d44005cf0 con 0x7f6d540670b0 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d44007270 con 0x7f6d540670b0 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d541572d0 con 0x7f6d540670b0 2024-04-25T10:24:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.969+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d54157890 con 0x7f6d540670b0 2024-04-25T10:24:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.973+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5404ef30 con 0x7f6d540670b0 2024-04-25T10:24:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.973+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d4401a070 con 0x7f6d540670b0 2024-04-25T10:24:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.973+0000 7f6d4a000640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d30066f60 0x7f6d30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:11.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.973+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d44084090 con 0x7f6d540670b0 2024-04-25T10:24:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.973+0000 7f6d4be00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d30066f60 0x7f6d30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.973+0000 7f6d4be00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d30066f60 0x7f6d30069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6d54152fd0 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:11.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:11.977+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d44054d40 con 0x7f6d540670b0 2024-04-25T10:24:12.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.137+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6d540724e0 con 0x7f6d540670b0 2024-04-25T10:24:12.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.141+0000 7f6d4a000640 1 -- 172.21.15.45:0/1633453691 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d44058de0 con 0x7f6d540670b0 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d30066f60 msgr2=0x7f6d30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d30066f60 0x7f6d30069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6d54152fd0 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 msgr2=0x7f6d540cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 0x7f6d540cd4e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c4d0 tx=0x7f6d4400c990 comp rx=0 tx=0).stop 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 shutdown_connections 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d30066f60 0x7f6d30069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d541610b0 0x7f6d5414ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d5406e5e0 0x7f6d541565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 --2- 172.21.15.45:0/1633453691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d540670b0 0x7f6d540cd4e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 >> 172.21.15.45:0/1633453691 conn(0x7f6d5406ac90 msgr2=0x7f6d540af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 shutdown_connections 2024-04-25T10:24:12.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.145+0000 7f6d58a00640 1 -- 172.21.15.45:0/1633453691 wait complete. 2024-04-25T10:24:12.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:24:12.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:12.258 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:12.332 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:24:12.332 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:24:12.332 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:12.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:24:12.341 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:24:12.341 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:12.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:12.350 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:24:12.350 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:12.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:24:12.391 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:24:12.391 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:12.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:24:12.407 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:24:12.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:12.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.573+0000 7f8f69a00640 1 Processor -- start 2024-04-25T10:24:12.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.573+0000 7f8f69a00640 1 -- start start 2024-04-25T10:24:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f69a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8f6415b910 0x7f8f6415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f69a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6415bd60 con 0x7f8f641550b0 2024-04-25T10:24:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f69a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f64071050 con 0x7f8f640d0040 2024-04-25T10:24:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f69a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f640712f0 con 0x7f8f6415b910 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f62a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8f641550b0 0x7f8f640cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44658/0 (socket says 172.21.15.45:44658) 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f62a00640 1 -- 172.21.15.45:0/72973068 learned_addr learned my addr 172.21.15.45:0/72973068 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f63400640 1 --2- 172.21.15.45:0/72973068 >> v2:172.21.15.45:3300/0 conn(0x7f8f6415b910 0x7f8f6415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f62000640 1 -- 172.21.15.45:0/72973068 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939392535 0 0) 0x7f8f6415bd60 con 0x7f8f641550b0 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f62000640 1 -- 172.21.15.45:0/72973068 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f48005740 con 0x7f8f641550b0 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f63400640 1 -- 172.21.15.45:0/72973068 >> v1:172.21.15.117:6789/0 conn(0x7f8f640d0040 legacy=0x7f8f6415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f63400640 1 -- 172.21.15.45:0/72973068 >> v1:172.21.15.45:6789/0 conn(0x7f8f641550b0 legacy=0x7f8f640cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.577+0000 7f8f63400640 1 -- 172.21.15.45:0/72973068 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640715d0 con 0x7f8f6415b910 2024-04-25T10:24:12.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f63400640 1 --2- 172.21.15.45:0/72973068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6415b910 0x7f8f6415c030 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009d90 tx=0x7f8f5c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba06b5b5352af31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f62000640 1 -- 172.21.15.45:0/72973068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0b2070 con 0x7f8f6415b910 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f62000640 1 -- 172.21.15.45:0/72973068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c0a5a00 con 0x7f8f6415b910 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f62000640 1 -- 172.21.15.45:0/72973068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0a5dd0 con 0x7f8f6415b910 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 -- 172.21.15.45:0/72973068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6415b910 msgr2=0x7f8f6415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 --2- 172.21.15.45:0/72973068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6415b910 0x7f8f6415c030 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009d90 tx=0x7f8f5c01c940 comp rx=0 tx=0).stop 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 -- 172.21.15.45:0/72973068 shutdown_connections 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 --2- 172.21.15.45:0/72973068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f6415b910 0x7f8f6415c030 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 -- 172.21.15.45:0/72973068 >> 172.21.15.45:0/72973068 conn(0x7f8f64066b70 msgr2=0x7f8f64066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 -- 172.21.15.45:0/72973068 shutdown_connections 2024-04-25T10:24:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 -- 172.21.15.45:0/72973068 wait complete. 2024-04-25T10:24:12.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 Processor -- start 2024-04-25T10:24:12.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.581+0000 7f8f69a00640 1 -- start start 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f640d0040 0x7f8f64152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 0x7f8f6413e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6415b910 0x7f8f640cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6415cf00 con 0x7f8f641550b0 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6415cd80 con 0x7f8f640d0040 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6415d080 con 0x7f8f6415b910 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 0x7f8f6413e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 0x7f8f6413e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48970/0 (socket says 172.21.15.45:48970) 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 -- 172.21.15.45:0/1389952243 learned_addr learned my addr 172.21.15.45:0/1389952243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f62a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f640d0040 0x7f8f64152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f63400640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6415b910 0x7f8f640cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 -- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6415b910 msgr2=0x7f8f640cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6415b910 0x7f8f640cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 -- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f640d0040 msgr2=0x7f8f64152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f640d0040 0x7f8f64152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 -- 172.21.15.45:0/1389952243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64153610 con 0x7f8f641550b0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f63400640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6415b910 0x7f8f640cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5be00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 0x7f8f6413e070 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00c3a0 tx=0x7f8f4c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0052d0 con 0x7f8f641550b0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c005470 con 0x7f8f641550b0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f641538f0 con 0x7f8f641550b0 2024-04-25T10:24:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c007260 con 0x7f8f641550b0 2024-04-25T10:24:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f64153eb0 con 0x7f8f641550b0 2024-04-25T10:24:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6404ef30 con 0x7f8f641550b0 2024-04-25T10:24:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.585+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f4c007400 con 0x7f8f641550b0 2024-04-25T10:24:12.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.589+0000 7f8f5aa00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8f40066f60 0x7f8f40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:12.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.589+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f4c0841b0 con 0x7f8f641550b0 2024-04-25T10:24:12.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.589+0000 7f8f62a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8f40066f60 0x7f8f40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:12.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.589+0000 7f8f62a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8f40066f60 0x7f8f40069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f540011f0 tx=0x7f8f54007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.593+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f4c054dd0 con 0x7f8f641550b0 2024-04-25T10:24:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.789+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f640af7b0 con 0x7f8f641550b0 2024-04-25T10:24:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.789+0000 7f8f5aa00640 1 -- 172.21.15.45:0/1389952243 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f8f4c058e70 con 0x7f8f641550b0 2024-04-25T10:24:12.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:12.791 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3962183680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3962183680},{"type":"v1","addr":"172.21.15.117:6837","nonce":3962183680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6562,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/523480210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":523480210},{"type":"v1","addr":"172.21.15.117:6836","nonce":523480210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/4272996482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4272996482},{"type":"v1","addr":"172.21.15.45:6834","nonce":4272996482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2701347193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2701347193},{"type":"v1","addr":"172.21.15.45:6835","nonce":2701347193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[]} 2024-04-25T10:24:12.792 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 58 2024-04-25T10:24:12.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8f40066f60 msgr2=0x7f8f40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8f40066f60 0x7f8f40069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f540011f0 tx=0x7f8f54007d40 comp rx=0 tx=0).stop 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 msgr2=0x7f8f6413e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 0x7f8f6413e070 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00c3a0 tx=0x7f8f4c00c860 comp rx=0 tx=0).stop 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 shutdown_connections 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8f40066f60 0x7f8f40069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f6415b910 0x7f8f640cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f641550b0 0x7f8f6413e070 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 --2- 172.21.15.45:0/1389952243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f640d0040 0x7f8f64152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 >> 172.21.15.45:0/1389952243 conn(0x7f8f64066b70 msgr2=0x7f8f640affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 shutdown_connections 2024-04-25T10:24:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:12.793+0000 7f8f69a00640 1 -- 172.21.15.45:0/1389952243 wait complete. 2024-04-25T10:24:12.814 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:24:12.814 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:24:12.814 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:24:12.814 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:12.817 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:24:12.817 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:24:12.818 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:24:12.818 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:12.818 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:24:12.818 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:24:12.818 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:24:12.818 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:12.819 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:24:12.819 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:24:12.819 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:24:12.819 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:24:12.825 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:24:12.825 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:24:12.831 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:24:12.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:24:12.836 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:24:12.836 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:24:12.838 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:24:12.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:24:12.873 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:24:12.874 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:12.876 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:24:12.876 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:12.886 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:24:12.887 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:12.889 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:24:12.889 DEBUG:teuthology.parallel:result is None 2024-04-25T10:24:12.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:24:12.961 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:24:12.962 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:24:12.980 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:24:12.980 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:24:13.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 Processor -- start 2024-04-25T10:24:13.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 -- start start 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c081550b0 0x7f1c080cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1c080cfe70 0x7f1c0815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c0806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c08071050 con 0x7f1c081550b0 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080712f0 con 0x7f1c080cfe70 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c10400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080715d0 con 0x7f1c0815b740 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c081550b0 0x7f1c080cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c081550b0 0x7f1c080cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48992/0 (socket says 172.21.15.45:48992) 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 -- 172.21.15.45:0/2395287448 learned_addr learned my addr 172.21.15.45:0/2395287448 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0e400640 1 --2- 172.21.15.45:0/2395287448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c0806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0d000640 1 --2- 172.21.15.45:0/2395287448 >> v2:172.21.15.117:3300/0 conn(0x7f1c080cfe70 0x7f1c0815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 -- 172.21.15.45:0/2395287448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 msgr2=0x7f1c0806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 --2- 172.21.15.45:0/2395287448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c0806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 -- 172.21.15.45:0/2395287448 >> v2:172.21.15.117:3300/0 conn(0x7f1c080cfe70 msgr2=0x7f1c0815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 --2- 172.21.15.45:0/2395287448 >> v2:172.21.15.117:3300/0 conn(0x7f1c080cfe70 0x7f1c0815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.037+0000 7f1c0da00640 1 -- 172.21.15.45:0/2395287448 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080718b0 con 0x7f1c081550b0 2024-04-25T10:24:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c0da00640 1 --2- 172.21.15.45:0/2395287448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c081550b0 0x7f1c080cf930 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009870 tx=0x7f1bfc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f40bb4788c1e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c07e00640 1 -- 172.21.15.45:0/2395287448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0b0070 con 0x7f1c081550b0 2024-04-25T10:24:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c07e00640 1 -- 172.21.15.45:0/2395287448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc0ab600 con 0x7f1c081550b0 2024-04-25T10:24:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c07e00640 1 -- 172.21.15.45:0/2395287448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0ab9d0 con 0x7f1c081550b0 2024-04-25T10:24:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 -- 172.21.15.45:0/2395287448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c081550b0 msgr2=0x7f1c080cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 --2- 172.21.15.45:0/2395287448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c081550b0 0x7f1c080cf930 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009870 tx=0x7f1bfc01c260 comp rx=0 tx=0).stop 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 -- 172.21.15.45:0/2395287448 shutdown_connections 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 --2- 172.21.15.45:0/2395287448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c0806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 --2- 172.21.15.45:0/2395287448 >> v2:172.21.15.117:3300/0 conn(0x7f1c080cfe70 0x7f1c0815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 --2- 172.21.15.45:0/2395287448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c081550b0 0x7f1c080cf930 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 -- 172.21.15.45:0/2395287448 >> 172.21.15.45:0/2395287448 conn(0x7f1c08066b70 msgr2=0x7f1c08066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 -- 172.21.15.45:0/2395287448 shutdown_connections 2024-04-25T10:24:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.041+0000 7f1c10400640 1 -- 172.21.15.45:0/2395287448 wait complete. 2024-04-25T10:24:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 Processor -- start 2024-04-25T10:24:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 -- start start 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 0x7f1c080ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c081550b0 0x7f1c081f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c080cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080b3260 con 0x7f1c080cfe70 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0815bd60 con 0x7f1c081550b0 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080b33e0 con 0x7f1c0815b740 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 0x7f1c080ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 0x7f1c080ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48996/0 (socket says 172.21.15.45:48996) 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 -- 172.21.15.45:0/825811099 learned_addr learned my addr 172.21.15.45:0/825811099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0e400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c080cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0d000640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c081550b0 0x7f1c081f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 -- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 msgr2=0x7f1c080cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c080cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 -- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c081550b0 msgr2=0x7f1c081f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c081550b0 0x7f1c081f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 -- 172.21.15.45:0/825811099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080cece0 con 0x7f1c080cfe70 2024-04-25T10:24:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c0da00640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 0x7f1c080ce7a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0ae040 tx=0x7f1bfc01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0ba020 con 0x7f1c080cfe70 2024-04-25T10:24:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc0b6420 con 0x7f1c080cfe70 2024-04-25T10:24:13.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c081f5ac0 con 0x7f1c080cfe70 2024-04-25T10:24:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.045+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0b0040 con 0x7f1c080cfe70 2024-04-25T10:24:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c081f8530 con 0x7f1c080cfe70 2024-04-25T10:24:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0804ef30 con 0x7f1c080cfe70 2024-04-25T10:24:13.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bfc0bd020 con 0x7f1c080cfe70 2024-04-25T10:24:13.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c05600640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1be8066f60 0x7f1be8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1bfc11ed90 con 0x7f1c080cfe70 2024-04-25T10:24:13.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c0d000640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1be8066f60 0x7f1be8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.049+0000 7f1c0d000640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1be8066f60 0x7f1be8069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1c0814e7a0 tx=0x7f1bf4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.053+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bfc0c4030 con 0x7f1c080cfe70 2024-04-25T10:24:13.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.245+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1c0815abe0 con 0x7f1c080cfe70 2024-04-25T10:24:13.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.797+0000 7f1c05600640 1 -- 172.21.15.45:0/825811099 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1c081f5ac0 con 0x7f1c080cfe70 2024-04-25T10:24:13.798 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1be8066f60 msgr2=0x7f1be8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1be8066f60 0x7f1be8069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1c0814e7a0 tx=0x7f1bf4009190 comp rx=0 tx=0).stop 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 msgr2=0x7f1c080ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 0x7f1c080ce7a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc0ae040 tx=0x7f1bfc01aaa0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 shutdown_connections 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1be8066f60 0x7f1be8069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c0815b740 0x7f1c080cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c081550b0 0x7f1c081f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 --2- 172.21.15.45:0/825811099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c080cfe70 0x7f1c080ce7a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 >> 172.21.15.45:0/825811099 conn(0x7f1c08066b70 msgr2=0x7f1c080b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 shutdown_connections 2024-04-25T10:24:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.801+0000 7f1c10400640 1 -- 172.21.15.45:0/825811099 wait complete. 2024-04-25T10:24:13.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:24:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57c400640 1 Processor -- start 2024-04-25T10:24:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57c400640 1 -- start start 2024-04-25T10:24:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57c400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff57415b910 0x7ff57415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57c400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57415bd60 con 0x7ff5740d0040 2024-04-25T10:24:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57c400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff574071050 con 0x7ff5741550b0 2024-04-25T10:24:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57c400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740712f0 con 0x7ff57415b910 2024-04-25T10:24:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff579000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5740d0040 0x7ff57415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44690/0 (socket says 172.21.15.45:44690) 2024-04-25T10:24:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff579000640 1 -- 172.21.15.45:0/3483403323 learned_addr learned my addr 172.21.15.45:0/3483403323 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff57a400640 1 --2- 172.21.15.45:0/3483403323 >> v2:172.21.15.117:3300/0 conn(0x7ff57415b910 0x7ff57415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608893906 0 0) 0x7ff57415bd60 con 0x7ff5740d0040 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff55c005740 con 0x7ff5740d0040 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101810731 0 0) 0x7ff55c005740 con 0x7ff5740d0040 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 >> v2:172.21.15.117:3300/0 conn(0x7ff57415b910 msgr2=0x7ff57415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 --2- 172.21.15.45:0/3483403323 >> v2:172.21.15.117:3300/0 conn(0x7ff57415b910 0x7ff57415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 >> v1:172.21.15.117:6789/0 conn(0x7ff5741550b0 legacy=0x7ff5740cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740715d0 con 0x7ff5740d0040 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7ff560001680 con 0x7ff5740d0040 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 >> v1:172.21.15.45:6789/0 conn(0x7ff5740d0040 legacy=0x7ff57415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.973+0000 7ff573e00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff55c005bd0 0x7ff55c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55c006a20 0x7ff55c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c0073e0 0x7ff55c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740712f0 con 0x7ff55c005bd0 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c00e280 con 0x7ff55c0073e0 2024-04-25T10:24:13.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c00e480 con 0x7ff55c006a20 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff55c005bd0 0x7ff55c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579000640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55c006a20 0x7ff55c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff57a400640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c0073e0 0x7ff55c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 -- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55c006a20 msgr2=0x7ff55c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55c006a20 0x7ff55c006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 -- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c0073e0 msgr2=0x7ff55c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c0073e0 0x7ff55c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 -- 172.21.15.45:0/3483403323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c005740 con 0x7ff55c005bd0 2024-04-25T10:24:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff57a400640 1 -- 172.21.15.45:0/3483403323 reap_dead start 2024-04-25T10:24:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff579a00640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff55c005bd0 0x7ff55c005ff0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff568001780 tx=0x7ff56801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db551e0df5134c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680013a0 con 0x7ff55c005bd0 2024-04-25T10:24:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5680052f0 con 0x7ff55c005bd0 2024-04-25T10:24:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.977+0000 7ff573e00640 1 -- 172.21.15.45:0/3483403323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568003840 con 0x7ff55c005bd0 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- 172.21.15.45:0/3483403323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff55c005bd0 msgr2=0x7ff55c005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff55c005bd0 0x7ff55c005ff0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff568001780 tx=0x7ff56801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- 172.21.15.45:0/3483403323 shutdown_connections 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 --2- 172.21.15.45:0/3483403323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff55c005bd0 0x7ff55c005ff0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- 172.21.15.45:0/3483403323 >> 172.21.15.45:0/3483403323 conn(0x7ff574066b70 msgr2=0x7ff574066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- 172.21.15.45:0/3483403323 shutdown_connections 2024-04-25T10:24:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- 172.21.15.45:0/3483403323 wait complete. 2024-04-25T10:24:13.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 Processor -- start 2024-04-25T10:24:13.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- start start 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c005bd0 0x7ff5741f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff574158150 0x7ff5741ec600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 0x7ff574152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57415b790 con 0x7ff574152970 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57415b610 con 0x7ff55c005bd0 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57c400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cfd40 con 0x7ff574158150 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 0x7ff574152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 0x7ff574152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49010/0 (socket says 172.21.15.45:49010) 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57a400640 1 -- 172.21.15.45:0/1978749723 learned_addr learned my addr 172.21.15.45:0/1978749723 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff579000640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff574158150 0x7ff5741ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff579a00640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c005bd0 0x7ff5741f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57a400640 1 -- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff574158150 msgr2=0x7ff5741ec600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.981+0000 7ff57a400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff574158150 0x7ff5741ec600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57a400640 1 -- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c005bd0 msgr2=0x7ff5741f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57a400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c005bd0 0x7ff5741f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57a400640 1 -- 172.21.15.45:0/1978749723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5741f7e30 con 0x7ff574152970 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57a400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 0x7ff574152df0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff5740af390 tx=0x7ff56c0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003c70 con 0x7ff574152970 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c01b070 con 0x7ff574152970 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5741f6e30 con 0x7ff574152970 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c011950 con 0x7ff574152970 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5741ed130 con 0x7ff574152970 2024-04-25T10:24:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57404ef30 con 0x7ff574152970 2024-04-25T10:24:13.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff56c011af0 con 0x7ff574152970 2024-04-25T10:24:13.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff571600640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f90 0x7ff554069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:13.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff56c085160 con 0x7ff574152970 2024-04-25T10:24:13.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.985+0000 7ff579a00640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f90 0x7ff554069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:13.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.989+0000 7ff579a00640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f90 0x7ff554069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff568001750 tx=0x7ff5680011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:13.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:13.989+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5741f7e30 con 0x7ff574152970 2024-04-25T10:24:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.209+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff574154870 con 0x7ff574152970 2024-04-25T10:24:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff571600640 1 -- 172.21.15.45:0/1978749723 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff56c056550 con 0x7ff574152970 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:24:14.214 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:24:14.215 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:24:14.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:24:14.220 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:24:14.221 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:24:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:24:14.224 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:24:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:14.225 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:24:14.225 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f90 msgr2=0x7ff554069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f90 0x7ff554069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff568001750 tx=0x7ff5680011a0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 msgr2=0x7ff574152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 0x7ff574152df0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff5740af390 tx=0x7ff56c0111d0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 shutdown_connections 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff554066f90 0x7ff554069450 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff574152970 0x7ff574152df0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff574158150 0x7ff5741ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 --2- 172.21.15.45:0/1978749723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55c005bd0 0x7ff5741f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 >> 172.21.15.45:0/1978749723 conn(0x7ff574066b70 msgr2=0x7ff57415bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 shutdown_connections 2024-04-25T10:24:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.213+0000 7ff57c400640 1 -- 172.21.15.45:0/1978749723 wait complete. 2024-04-25T10:24:14.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:14.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.373+0000 7fd857600640 1 Processor -- start 2024-04-25T10:24:14.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.373+0000 7fd857600640 1 -- start start 2024-04-25T10:24:14.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd857600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8500cda10 0x7fd85006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd857600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8500cf930 0x7fd8500cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd857600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500b2770 con 0x7fd85015bc40 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd857600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85006c670 con 0x7fd8500cda10 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd857600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85006c950 con 0x7fd8500cf930 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8500cda10 0x7fd85006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd854c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd85015bc40 0x7fd8500b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44692/0 (socket says 172.21.15.45:44692) 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd854c00640 1 -- 172.21.15.45:0/3437608527 learned_addr learned my addr 172.21.15.45:0/3437608527 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd855600640 1 --2- 172.21.15.45:0/3437608527 >> v2:172.21.15.117:3300/0 conn(0x7fd8500cf930 0x7fd8500cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84fe00640 1 -- 172.21.15.45:0/3437608527 >> v2:172.21.15.117:3300/0 conn(0x7fd8500cf930 msgr2=0x7fd8500cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84fe00640 1 --2- 172.21.15.45:0/3437608527 >> v2:172.21.15.117:3300/0 conn(0x7fd8500cf930 0x7fd8500cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84fe00640 1 -- 172.21.15.45:0/3437608527 >> v1:172.21.15.45:6789/0 conn(0x7fd85015bc40 legacy=0x7fd8500b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84fe00640 1 -- 172.21.15.45:0/3437608527 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85006cc30 con 0x7fd8500cda10 2024-04-25T10:24:14.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84f400640 1 -- 172.21.15.45:0/3437608527 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920301166 0 0) 0x7fd8500b2770 con 0x7fd85015bc40 2024-04-25T10:24:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84fe00640 1 --2- 172.21.15.45:0/3437608527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85006c090 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ae4c0 tx=0x7fd83c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d3b573530df7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84f400640 1 -- 172.21.15.45:0/3437608527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0b0070 con 0x7fd8500cda10 2024-04-25T10:24:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84f400640 1 -- 172.21.15.45:0/3437608527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c0ab600 con 0x7fd8500cda10 2024-04-25T10:24:14.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.377+0000 7fd84f400640 1 -- 172.21.15.45:0/3437608527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0ab9d0 con 0x7fd8500cda10 2024-04-25T10:24:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- 172.21.15.45:0/3437608527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 msgr2=0x7fd85006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 --2- 172.21.15.45:0/3437608527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85006c090 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ae4c0 tx=0x7fd83c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- 172.21.15.45:0/3437608527 shutdown_connections 2024-04-25T10:24:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 --2- 172.21.15.45:0/3437608527 >> v2:172.21.15.117:3300/0 conn(0x7fd8500cf930 0x7fd8500cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 --2- 172.21.15.45:0/3437608527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85006c090 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- 172.21.15.45:0/3437608527 >> 172.21.15.45:0/3437608527 conn(0x7fd850066b70 msgr2=0x7fd850066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- 172.21.15.45:0/3437608527 shutdown_connections 2024-04-25T10:24:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- 172.21.15.45:0/3437608527 wait complete. 2024-04-25T10:24:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 Processor -- start 2024-04-25T10:24:14.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- start start 2024-04-25T10:24:14.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85014c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8500cf930 0x7fd8500ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd85015bc40 0x7fd8500add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b3850 con 0x7fd8500cda10 2024-04-25T10:24:14.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b36d0 con 0x7fd8500cf930 2024-04-25T10:24:14.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd857600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b39d0 con 0x7fd85015bc40 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85014c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85014c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49036/0 (socket says 172.21.15.45:49036) 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 -- 172.21.15.45:0/623089689 learned_addr learned my addr 172.21.15.45:0/623089689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd84fe00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8500cf930 0x7fd8500ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd855600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd85015bc40 0x7fd8500add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 -- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd85015bc40 msgr2=0x7fd8500add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd85015bc40 0x7fd8500add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 -- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8500cf930 msgr2=0x7fd8500ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8500cf930 0x7fd8500ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.381+0000 7fd854c00640 1 -- 172.21.15.45:0/623089689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850154c10 con 0x7fd8500cda10 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd854c00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85014c390 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c480 tx=0x7fd84400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844018070 con 0x7fd8500cda10 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844007500 con 0x7fd8500cda10 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844004a30 con 0x7fd8500cda10 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850154ef0 con 0x7fd8500cda10 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8501554b0 con 0x7fd8500cda10 2024-04-25T10:24:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85004ef30 con 0x7fd8500cda10 2024-04-25T10:24:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd844004bd0 con 0x7fd8500cda10 2024-04-25T10:24:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84cc00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd844083fd0 con 0x7fd8500cda10 2024-04-25T10:24:14.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.385+0000 7fd84fe00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.389+0000 7fd84fe00640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd830066f60 0x7fd830069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0ad000 tx=0x7fd83c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.389+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd844055320 con 0x7fd8500cda10 2024-04-25T10:24:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.553+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85006d960 con 0x7fd8500cda10 2024-04-25T10:24:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.553+0000 7fd84cc00640 1 -- 172.21.15.45:0/623089689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+12880 (secure 0 0 0) 0x7fd844054c20 con 0x7fd8500cda10 2024-04-25T10:24:14.555 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:14.555 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:13.807772+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd830066f60 msgr2=0x7fd830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd830066f60 0x7fd830069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0ad000 tx=0x7fd83c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 msgr2=0x7fd85014c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85014c390 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c480 tx=0x7fd84400c940 comp rx=0 tx=0).stop 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 shutdown_connections 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd85015bc40 0x7fd8500add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8500cf930 0x7fd8500ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 --2- 172.21.15.45:0/623089689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8500cda10 0x7fd85014c390 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 >> 172.21.15.45:0/623089689 conn(0x7fd850066b70 msgr2=0x7fd8500ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 shutdown_connections 2024-04-25T10:24:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.557+0000 7fd857600640 1 -- 172.21.15.45:0/623089689 wait complete. 2024-04-25T10:24:14.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:14.576 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.717+0000 7f404da00640 1 Processor -- start 2024-04-25T10:24:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 -- start start 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4048159540 0x7f40480ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f40480aebd0 0x7f4048071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404815f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4048159920 con 0x7f4048159540 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404815c2f0 con 0x7f40480aebd0 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f404da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404815c5d0 con 0x7f4048072020 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4048159540 0x7f40480ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4048159540 0x7f40480ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49042/0 (socket says 172.21.15.45:49042) 2024-04-25T10:24:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 -- 172.21.15.45:0/2388276857 learned_addr learned my addr 172.21.15.45:0/2388276857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f403fe00640 1 --2- 172.21.15.45:0/2388276857 >> v2:172.21.15.117:3300/0 conn(0x7f40480aebd0 0x7f4048071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4047400640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 -- 172.21.15.45:0/2388276857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 msgr2=0x7f404815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 -- 172.21.15.45:0/2388276857 >> v2:172.21.15.117:3300/0 conn(0x7f40480aebd0 msgr2=0x7f4048071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 --2- 172.21.15.45:0/2388276857 >> v2:172.21.15.117:3300/0 conn(0x7f40480aebd0 0x7f4048071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4046a00640 1 -- 172.21.15.45:0/2388276857 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404815c8b0 con 0x7f4048159540 2024-04-25T10:24:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.721+0000 7f4047400640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:14.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f4046a00640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40480ae690 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f40480cd4e0 tx=0x7f403801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a7ec0fa658f17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:14.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f4046000640 1 -- 172.21.15.45:0/2388276857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380b0070 con 0x7f4048159540 2024-04-25T10:24:14.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f4046000640 1 -- 172.21.15.45:0/2388276857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40380ab600 con 0x7f4048159540 2024-04-25T10:24:14.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f4046000640 1 -- 172.21.15.45:0/2388276857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380ab9d0 con 0x7f4048159540 2024-04-25T10:24:14.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 -- 172.21.15.45:0/2388276857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 msgr2=0x7f40480ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40480ae690 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f40480cd4e0 tx=0x7f403801c260 comp rx=0 tx=0).stop 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 -- 172.21.15.45:0/2388276857 shutdown_connections 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 --2- 172.21.15.45:0/2388276857 >> v2:172.21.15.117:3300/0 conn(0x7f40480aebd0 0x7f4048071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 --2- 172.21.15.45:0/2388276857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40480ae690 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 -- 172.21.15.45:0/2388276857 >> 172.21.15.45:0/2388276857 conn(0x7f40480669a0 msgr2=0x7f4048066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 -- 172.21.15.45:0/2388276857 shutdown_connections 2024-04-25T10:24:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 -- 172.21.15.45:0/2388276857 wait complete. 2024-04-25T10:24:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 Processor -- start 2024-04-25T10:24:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.725+0000 7f404da00640 1 -- start start 2024-04-25T10:24:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404813f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40480aebd0 0x7f4048154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40481551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048070350 con 0x7f4048159540 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048072430 con 0x7f40480aebd0 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40480704d0 con 0x7f4048072020 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40481551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4046a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404813f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40481551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49054/0 (socket says 172.21.15.45:49054) 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 -- 172.21.15.45:0/899513703 learned_addr learned my addr 172.21.15.45:0/899513703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f403fe00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40480aebd0 0x7f4048154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 -- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 msgr2=0x7f404813f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404813f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 -- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40480aebd0 msgr2=0x7f4048154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40480aebd0 0x7f4048154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 -- 172.21.15.45:0/899513703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40481faff0 con 0x7f4048159540 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f4047400640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40481551c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f404000cf20 tx=0x7f404000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404001a070 con 0x7f4048159540 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404000d840 con 0x7f4048159540 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40481fa230 con 0x7f4048159540 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404000dc30 con 0x7f4048159540 2024-04-25T10:24:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40481fa7f0 con 0x7f4048159540 2024-04-25T10:24:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.729+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404804ef30 con 0x7f4048159540 2024-04-25T10:24:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.733+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f404001d070 con 0x7f4048159540 2024-04-25T10:24:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.733+0000 7f403ea00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.733+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40400854d0 con 0x7f4048159540 2024-04-25T10:24:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.733+0000 7f4046a00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.733+0000 7f4046a00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4024066f60 0x7f4024069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f40380ad000 tx=0x7f4038008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.737+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40400561d0 con 0x7f4048159540 2024-04-25T10:24:14.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.901+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4048166700 con 0x7f4048159540 2024-04-25T10:24:14.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.901+0000 7f403ea00640 1 -- 172.21.15.45:0/899513703 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f404005a270 con 0x7f4048159540 2024-04-25T10:24:14.903 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4024066f60 msgr2=0x7f4024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4024066f60 0x7f4024069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f40380ad000 tx=0x7f4038008040 comp rx=0 tx=0).stop 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 msgr2=0x7f40481551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40481551c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f404000cf20 tx=0x7f404000a7d0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 shutdown_connections 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4024066f60 0x7f4024069420 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4048159540 0x7f40481551c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40480aebd0 0x7f4048154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 --2- 172.21.15.45:0/899513703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4048072020 0x7f404813f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 >> 172.21.15.45:0/899513703 conn(0x7f40480669a0 msgr2=0x7f404815e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 shutdown_connections 2024-04-25T10:24:14.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:14.905+0000 7f404da00640 1 -- 172.21.15.45:0/899513703 wait complete. 2024-04-25T10:24:14.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:24:15.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 Processor -- start 2024-04-25T10:24:15.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 -- start start 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61980cfd90 0x7f61980ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f61980cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619815f3b0 con 0x7f61980b0f70 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619815ef20 con 0x7f61980cfd90 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f619dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61980ca810 con 0x7f61980cd110 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f6196a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f61980b0f70 0x7f61980b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58416/0 (socket says 172.21.15.45:58416) 2024-04-25T10:24:15.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f6196a00640 1 -- 172.21.15.45:0/2034508860 learned_addr learned my addr 172.21.15.45:0/2034508860 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:15.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f618fe00640 1 --2- 172.21.15.45:0/2034508860 >> v2:172.21.15.117:3300/0 conn(0x7f61980cfd90 0x7f61980ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f6197400640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f61980cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904959577 0 0) 0x7f619815f3b0 con 0x7f61980b0f70 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.065+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f617c003530 con 0x7f61980b0f70 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559968979 0 0) 0x7f617c003530 con 0x7f61980b0f70 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 msgr2=0x7f61980cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f61980cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 >> v2:172.21.15.117:3300/0 conn(0x7f61980cfd90 msgr2=0x7f61980ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 --2- 172.21.15.45:0/2034508860 >> v2:172.21.15.117:3300/0 conn(0x7f61980cfd90 0x7f61980ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619815e5f0 con 0x7f61980b0f70 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6188001680 con 0x7f61980b0f70 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 >> v1:172.21.15.45:6789/0 conn(0x7f61980b0f70 legacy=0x7f61980b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617c0039c0 0x7f617c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f617c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f617c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619815ef20 con 0x7f617c004d50 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980ca810 con 0x7f617c0039c0 2024-04-25T10:24:15.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c00b9d0 con 0x7f61980cd110 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f617c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196a00640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617c0039c0 0x7f617c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f618fe00640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f617c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 -- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 msgr2=0x7f617c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f617c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 -- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617c0039c0 msgr2=0x7f617c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617c0039c0 0x7f617c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 -- 172.21.15.45:0/2034508860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c003530 con 0x7f617c004d50 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f618fe00640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61980cd110 0x7f617c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6196a00640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f617c0039c0 0x7f617c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.069+0000 7f6197400640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f617c0051d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6190009fc0 tx=0x7f619001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad47e713e236e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61900b2070 con 0x7f617c004d50 2024-04-25T10:24:15.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61900a5c90 con 0x7f617c004d50 2024-04-25T10:24:15.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196000640 1 -- 172.21.15.45:0/2034508860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190009c10 con 0x7f617c004d50 2024-04-25T10:24:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- 172.21.15.45:0/2034508860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 msgr2=0x7f617c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 --2- 172.21.15.45:0/2034508860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f617c0051d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6190009fc0 tx=0x7f619001cc80 comp rx=0 tx=0).stop 2024-04-25T10:24:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6197400640 1 -- 172.21.15.45:0/2034508860 reap_dead start 2024-04-25T10:24:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- 172.21.15.45:0/2034508860 shutdown_connections 2024-04-25T10:24:15.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- 172.21.15.45:0/2034508860 >> 172.21.15.45:0/2034508860 conn(0x7f61980b1d50 msgr2=0x7f61980b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- 172.21.15.45:0/2034508860 shutdown_connections 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- 172.21.15.45:0/2034508860 wait complete. 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 Processor -- start 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- start start 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f61980b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619814ea30 0x7f619814eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61980c5fe0 0x7f61980c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980cd560 con 0x7f617c004d50 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619815ebb0 con 0x7f61980c5fe0 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f619dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61980cba20 con 0x7f619814ea30 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f61980b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f61980b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43198/0 (socket says 172.21.15.45:43198) 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196a00640 1 -- 172.21.15.45:0/2521917550 learned_addr learned my addr 172.21.15.45:0/2521917550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:15.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6197400640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61980c5fe0 0x7f61980c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f618fe00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619814ea30 0x7f619814eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196a00640 1 -- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619814ea30 msgr2=0x7f619814eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196a00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619814ea30 0x7f619814eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.073+0000 7f6196a00640 1 -- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61980c5fe0 msgr2=0x7f61980c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f6196a00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61980c5fe0 0x7f61980c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f6196a00640 1 -- 172.21.15.45:0/2521917550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619814f3f0 con 0x7f617c004d50 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f6197400640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61980c5fe0 0x7f61980c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f6196a00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f61980b50b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f618800e850 tx=0x7f618800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800c870 con 0x7f617c004d50 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6188018070 con 0x7f617c004d50 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800cc10 con 0x7f617c004d50 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61981f7240 con 0x7f617c004d50 2024-04-25T10:24:15.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61981f7800 con 0x7f617c004d50 2024-04-25T10:24:15.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619804ef30 con 0x7f617c004d50 2024-04-25T10:24:15.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f618801d460 con 0x7f617c004d50 2024-04-25T10:24:15.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618ea00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6174066f60 0x7f6174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6188084cb0 con 0x7f617c004d50 2024-04-25T10:24:15.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.077+0000 7f618fe00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6174066f60 0x7f6174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.081+0000 7f618fe00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6174066f60 0x7f6174069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f61980b5b00 tx=0x7f6180008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.081+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6188055f70 con 0x7f617c004d50 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.245+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61980cda20 con 0x7f617c004d50 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.245+0000 7f618ea00640 1 -- 172.21.15.45:0/2521917550 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6188055870 con 0x7f617c004d50 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:15.247 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6174066f60 msgr2=0x7f6174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6174066f60 0x7f6174069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f61980b5b00 tx=0x7f6180008040 comp rx=0 tx=0).stop 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 msgr2=0x7f61980b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f61980b50b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f618800e850 tx=0x7f618800ed10 comp rx=0 tx=0).stop 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 shutdown_connections 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6174066f60 0x7f6174069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61980c5fe0 0x7f61980c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619814ea30 0x7f619814eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 --2- 172.21.15.45:0/2521917550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f617c004d50 0x7f61980b50b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 >> 172.21.15.45:0/2521917550 conn(0x7f61980b1d50 msgr2=0x7f619806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 shutdown_connections 2024-04-25T10:24:15.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.249+0000 7f619dc00640 1 -- 172.21.15.45:0/2521917550 wait complete. 2024-04-25T10:24:15.268 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:24:15.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:24:15.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 Processor -- start 2024-04-25T10:24:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 -- start start 2024-04-25T10:24:15.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd0015b420 0x7fdd000d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000d06b0 0x7fdd0015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd00154cd0 con 0x7fdd001548f0 2024-04-25T10:24:15.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000b3be0 con 0x7fdd0015b420 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdd06600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0015c120 con 0x7fdd000d06b0 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd0015b420 0x7fdd000d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdd0015b420 0x7fdd000d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43206/0 (socket says 172.21.15.45:43206) 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcff400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd001548f0 0x7fdd000b30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58424/0 (socket says 172.21.15.45:58424) 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 -- 172.21.15.45:0/3996203172 learned_addr learned my addr 172.21.15.45:0/3996203172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcffe00640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000d06b0 0x7fdd0015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 -- 172.21.15.45:0/3996203172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000d06b0 msgr2=0x7fdd0015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000d06b0 0x7fdd0015c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 -- 172.21.15.45:0/3996203172 >> v1:172.21.15.45:6789/0 conn(0x7fdd001548f0 legacy=0x7fdd000b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcf6a00640 1 -- 172.21.15.45:0/3996203172 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000b3900 con 0x7fdd0015b420 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcffe00640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000d06b0 0x7fdd0015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.413+0000 7fdcfea00640 1 -- 172.21.15.45:0/3996203172 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173159241 0 0) 0x7fdd00154cd0 con 0x7fdd001548f0 2024-04-25T10:24:15.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdcf6a00640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015b420 0x7fdd000d0170 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdce800aa10 tx=0x7fdce801d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf083c19cf62f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdcfea00640 1 -- 172.21.15.45:0/3996203172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce80b3070 con 0x7fdd0015b420 2024-04-25T10:24:15.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdcfea00640 1 -- 172.21.15.45:0/3996203172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce80a67c0 con 0x7fdd0015b420 2024-04-25T10:24:15.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdcfea00640 1 -- 172.21.15.45:0/3996203172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800b040 con 0x7fdd0015b420 2024-04-25T10:24:15.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 -- 172.21.15.45:0/3996203172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015b420 msgr2=0x7fdd000d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015b420 0x7fdd000d0170 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fdce800aa10 tx=0x7fdce801d8a0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 -- 172.21.15.45:0/3996203172 shutdown_connections 2024-04-25T10:24:15.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000d06b0 0x7fdd0015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 --2- 172.21.15.45:0/3996203172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015b420 0x7fdd000d0170 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 -- 172.21.15.45:0/3996203172 >> 172.21.15.45:0/3996203172 conn(0x7fdd000b5610 msgr2=0x7fdd000b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:15.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 -- 172.21.15.45:0/3996203172 shutdown_connections 2024-04-25T10:24:15.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.417+0000 7fdd06600640 1 -- 172.21.15.45:0/3996203172 wait complete. 2024-04-25T10:24:15.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 Processor -- start 2024-04-25T10:24:15.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- start start 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 0x7fdd0014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd001548f0 0x7fdd0014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd0015b420 0x7fdd0014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0015cec0 con 0x7fdd000d06b0 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0015cd40 con 0x7fdd001548f0 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0015d040 con 0x7fdd0015b420 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 0x7fdd0014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 0x7fdd0014a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43222/0 (socket says 172.21.15.45:43222) 2024-04-25T10:24:15.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 -- 172.21.15.45:0/52001844 learned_addr learned my addr 172.21.15.45:0/52001844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcffe00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd0015b420 0x7fdd0014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 -- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd0015b420 msgr2=0x7fdd0014c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd0015b420 0x7fdd0014c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcf6a00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd001548f0 0x7fdd0014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 -- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd001548f0 msgr2=0x7fdd0014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd001548f0 0x7fdd0014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 -- 172.21.15.45:0/52001844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0014af10 con 0x7fdd000d06b0 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcffe00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd0015b420 0x7fdd0014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcf6a00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd001548f0 0x7fdd0014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:15.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcff400640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 0x7fdd0014a5a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00011f0 tx=0x7fdcf000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0018070 con 0x7fdd000d06b0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf00074e0 con 0x7fdd000d06b0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004a10 con 0x7fdd000d06b0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0014b1f0 con 0x7fdd000d06b0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0014b7b0 con 0x7fdd000d06b0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.421+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0004ef30 con 0x7fdd000d06b0 2024-04-25T10:24:15.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.425+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcf0004bb0 con 0x7fdd000d06b0 2024-04-25T10:24:15.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.425+0000 7fdcf7e00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcdc066f60 0x7fdcdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.425+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdcf0084170 con 0x7fdd000d06b0 2024-04-25T10:24:15.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.425+0000 7fdcf6a00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcdc066f60 0x7fdcdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.425+0000 7fdcf6a00640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcdc066f60 0x7fdcdc069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdce80b0000 tx=0x7fdce800c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.429+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcf0055430 con 0x7fdd000d06b0 2024-04-25T10:24:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.589+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdd000b18a0 con 0x7fdd000d06b0 2024-04-25T10:24:15.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.801+0000 7fdcf7e00640 1 -- 172.21.15.45:0/52001844 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdcf0054d30 con 0x7fdd000d06b0 2024-04-25T10:24:15.802 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcdc066f60 msgr2=0x7fdcdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcdc066f60 0x7fdcdc069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdce80b0000 tx=0x7fdce800c040 comp rx=0 tx=0).stop 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 msgr2=0x7fdd0014a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 0x7fdd0014a5a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00011f0 tx=0x7fdcf000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 shutdown_connections 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcdc066f60 0x7fdcdc069420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd0015b420 0x7fdd0014c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd001548f0 0x7fdd0014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 --2- 172.21.15.45:0/52001844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd000d06b0 0x7fdd0014a5a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 >> 172.21.15.45:0/52001844 conn(0x7fdd000b5610 msgr2=0x7fdd001515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 shutdown_connections 2024-04-25T10:24:15.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.805+0000 7fdd06600640 1 -- 172.21.15.45:0/52001844 wait complete. 2024-04-25T10:24:15.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:24:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.977+0000 7ff61fe00640 1 Processor -- start 2024-04-25T10:24:15.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.977+0000 7ff61fe00640 1 -- start start 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff618154f70 0x7ff6180cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff6180b3ba0 0x7ff6180ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61fe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61815e6e0 con 0x7ff6180cfea0 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61fe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61815e9c0 con 0x7ff618154f70 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61fe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61815eca0 con 0x7ff6180b3ba0 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff618154f70 0x7ff6180cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff618154f70 0x7ff6180cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43234/0 (socket says 172.21.15.45:43234) 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 -- 172.21.15.45:0/2622242220 learned_addr learned my addr 172.21.15.45:0/2622242220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:15.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61de00640 1 --2- 172.21.15.45:0/2622242220 >> v2:172.21.15.117:3300/0 conn(0x7ff6180b3ba0 0x7ff6180ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 -- 172.21.15.45:0/2622242220 >> v2:172.21.15.117:3300/0 conn(0x7ff6180b3ba0 msgr2=0x7ff6180ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 --2- 172.21.15.45:0/2622242220 >> v2:172.21.15.117:3300/0 conn(0x7ff6180b3ba0 0x7ff6180ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 -- 172.21.15.45:0/2622242220 >> v1:172.21.15.117:6789/0 conn(0x7ff6180cfea0 legacy=0x7ff6180b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:15.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 -- 172.21.15.45:0/2622242220 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61815ef80 con 0x7ff618154f70 2024-04-25T10:24:15.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.981+0000 7ff61d400640 1 --2- 172.21.15.45:0/2622242220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618154f70 0x7ff6180cf960 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff61815af50 tx=0x7ff60c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8914761198576db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff617e00640 1 -- 172.21.15.45:0/2622242220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0b4070 con 0x7ff618154f70 2024-04-25T10:24:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff617e00640 1 -- 172.21.15.45:0/2622242220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c0a7a80 con 0x7ff618154f70 2024-04-25T10:24:15.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff617e00640 1 -- 172.21.15.45:0/2622242220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a7e50 con 0x7ff618154f70 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- 172.21.15.45:0/2622242220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618154f70 msgr2=0x7ff6180cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 --2- 172.21.15.45:0/2622242220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618154f70 0x7ff6180cf960 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff61815af50 tx=0x7ff60c01ebb0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- 172.21.15.45:0/2622242220 shutdown_connections 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 --2- 172.21.15.45:0/2622242220 >> v2:172.21.15.117:3300/0 conn(0x7ff6180b3ba0 0x7ff6180ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 --2- 172.21.15.45:0/2622242220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff618154f70 0x7ff6180cf960 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- 172.21.15.45:0/2622242220 >> 172.21.15.45:0/2622242220 conn(0x7ff618066b70 msgr2=0x7ff618066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- 172.21.15.45:0/2622242220 shutdown_connections 2024-04-25T10:24:15.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- 172.21.15.45:0/2622242220 wait complete. 2024-04-25T10:24:15.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 Processor -- start 2024-04-25T10:24:15.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- start start 2024-04-25T10:24:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6180b3ba0 0x7ff6180adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 0x7ff618157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff618154f70 0x7ff6181578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806cd00 con 0x7ff6180cfea0 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806cb80 con 0x7ff618154f70 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806ce80 con 0x7ff6180b3ba0 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 0x7ff618157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 0x7ff618157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43238/0 (socket says 172.21.15.45:43238) 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.985+0000 7ff61ca00640 1 -- 172.21.15.45:0/488084278 learned_addr learned my addr 172.21.15.45:0/488084278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61de00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff618154f70 0x7ff6181578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61d400640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6180b3ba0 0x7ff6180adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61ca00640 1 -- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6180b3ba0 msgr2=0x7ff6180adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61ca00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6180b3ba0 0x7ff6180adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61ca00640 1 -- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff618154f70 msgr2=0x7ff6181578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61ca00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff618154f70 0x7ff6181578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:15.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61ca00640 1 -- 172.21.15.45:0/488084278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180ae100 con 0x7ff6180cfea0 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61de00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff618154f70 0x7ff6181578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61d400640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6180b3ba0 0x7ff6180adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61ca00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 0x7ff618157360 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff60400b440 tx=0x7ff60400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604015020 con 0x7ff6180cfea0 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604011890 con 0x7ff6180cfea0 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6180cdee0 con 0x7ff6180cfea0 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604011c80 con 0x7ff6180cfea0 2024-04-25T10:24:15.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618151e80 con 0x7ff6180cfea0 2024-04-25T10:24:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61804ef30 con 0x7ff6180cfea0 2024-04-25T10:24:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.989+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff604018020 con 0x7ff6180cfea0 2024-04-25T10:24:15.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.993+0000 7ff615600640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.993+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff604084d10 con 0x7ff6180cfea0 2024-04-25T10:24:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.993+0000 7ff61d400640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.993+0000 7ff61d400640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b1000 tx=0x7ff60c0a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:15.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:15.997+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff60408a020 con 0x7ff6180cfea0 2024-04-25T10:24:16.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.161+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff6180b2050 con 0x7ff6180cfea0 2024-04-25T10:24:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.809+0000 7ff615600640 1 -- 172.21.15.45:0/488084278 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff60401f030 con 0x7ff6180cfea0 2024-04-25T10:24:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:24:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 msgr2=0x7ff5f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b1000 tx=0x7ff60c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 msgr2=0x7ff618157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 0x7ff618157360 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff60400b440 tx=0x7ff60400b900 comp rx=0 tx=0).stop 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 shutdown_connections 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff618154f70 0x7ff6181578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180cfea0 0x7ff618157360 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 --2- 172.21.15.45:0/488084278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6180b3ba0 0x7ff6180adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:16.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 >> 172.21.15.45:0/488084278 conn(0x7ff618066b70 msgr2=0x7ff61815d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:16.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 shutdown_connections 2024-04-25T10:24:16.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:16.813+0000 7ff61fe00640 1 -- 172.21.15.45:0/488084278 wait complete. 2024-04-25T10:24:16.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:24:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.001+0000 7fdb05c00640 1 Processor -- start 2024-04-25T10:24:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.001+0000 7fdb05c00640 1 -- start start 2024-04-25T10:24:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdb05c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb0015dbe0 0x7fdb000cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb000b2860 0x7fdb00161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdb05c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb000b2cc0 con 0x7fdb0006afe0 2024-04-25T10:24:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdb05c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb000b5640 con 0x7fdb0015dbe0 2024-04-25T10:24:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdb05c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000b5920 con 0x7fdb000b2860 2024-04-25T10:24:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb0015dbe0 0x7fdb000cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:17.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb0015dbe0 0x7fdb000cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43244/0 (socket says 172.21.15.45:43244) 2024-04-25T10:24:17.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 -- 172.21.15.45:0/1288040737 learned_addr learned my addr 172.21.15.45:0/1288040737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:17.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaf7e00640 1 --2- 172.21.15.45:0/1288040737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb000b2860 0x7fdb00161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:17.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaff400640 1 -- 172.21.15.45:0/1288040737 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604036382 0 0) 0x7fdaf80014a0 con 0x7fdb0006afe0 2024-04-25T10:24:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 -- 172.21.15.45:0/1288040737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb000b2860 msgr2=0x7fdb00161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 --2- 172.21.15.45:0/1288040737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb000b2860 0x7fdb00161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 -- 172.21.15.45:0/1288040737 >> v1:172.21.15.45:6789/0 conn(0x7fdb0006afe0 legacy=0x7fdb000cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.005+0000 7fdaffe00640 1 -- 172.21.15.45:0/1288040737 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000b5c00 con 0x7fdb0015dbe0 2024-04-25T10:24:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdaffe00640 1 --2- 172.21.15.45:0/1288040737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0015dbe0 0x7fdb000cc500 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009870 tx=0x7fdaf001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f669b56c5268c5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdaff400640 1 -- 172.21.15.45:0/1288040737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00a7070 con 0x7fdb0015dbe0 2024-04-25T10:24:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdaff400640 1 -- 172.21.15.45:0/1288040737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf00b0070 con 0x7fdb0015dbe0 2024-04-25T10:24:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdaff400640 1 -- 172.21.15.45:0/1288040737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00ab890 con 0x7fdb0015dbe0 2024-04-25T10:24:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- 172.21.15.45:0/1288040737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0015dbe0 msgr2=0x7fdb000cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 --2- 172.21.15.45:0/1288040737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0015dbe0 0x7fdb000cc500 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009870 tx=0x7fdaf001c560 comp rx=0 tx=0).stop 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- 172.21.15.45:0/1288040737 shutdown_connections 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 --2- 172.21.15.45:0/1288040737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb000b2860 0x7fdb00161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 --2- 172.21.15.45:0/1288040737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0015dbe0 0x7fdb000cc500 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- 172.21.15.45:0/1288040737 >> 172.21.15.45:0/1288040737 conn(0x7fdb00073d10 msgr2=0x7fdb00074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- 172.21.15.45:0/1288040737 shutdown_connections 2024-04-25T10:24:17.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- 172.21.15.45:0/1288040737 wait complete. 2024-04-25T10:24:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 Processor -- start 2024-04-25T10:24:17.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- start start 2024-04-25T10:24:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 0x7fdb0015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb000b2860 0x7fdb00151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb0015dbe0 0x7fdb0015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00162540 con 0x7fdb0006afe0 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb001623c0 con 0x7fdb000b2860 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.009+0000 7fdb05c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0016b190 con 0x7fdb0015dbe0 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 0x7fdb0015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 0x7fdb0015fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43252/0 (socket says 172.21.15.45:43252) 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 -- 172.21.15.45:0/2702272889 learned_addr learned my addr 172.21.15.45:0/2702272889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaf7e00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb000b2860 0x7fdb00151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdb05200640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb0015dbe0 0x7fdb0015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 -- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb0015dbe0 msgr2=0x7fdb0015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb0015dbe0 0x7fdb0015dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 -- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb000b2860 msgr2=0x7fdb00151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb000b2860 0x7fdb00151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 -- 172.21.15.45:0/2702272889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0015ecb0 con 0x7fdb0006afe0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdb05200640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb0015dbe0 0x7fdb0015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdaffe00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 0x7fdb0015fd50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdaf001a190 tx=0x7fdaf001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00be5f0 con 0x7fdb0006afe0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf00be7b0 con 0x7fdb0006afe0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00beba0 con 0x7fdb0006afe0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0015ef90 con 0x7fdb0006afe0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0015f550 con 0x7fdb0006afe0 2024-04-25T10:24:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb00075120 con 0x7fdb0006afe0 2024-04-25T10:24:17.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.013+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdaf00c9430 con 0x7fdb0006afe0 2024-04-25T10:24:17.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.017+0000 7fdafcc00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdadc066f60 0x7fdadc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.017+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdaf011f980 con 0x7fdb0006afe0 2024-04-25T10:24:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.017+0000 7fdaf7e00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdadc066f60 0x7fdadc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:17.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.017+0000 7fdaf7e00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdadc066f60 0x7fdadc069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb0006e7f0 tx=0x7fdae8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:17.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.021+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdaf0016150 con 0x7fdb0006afe0 2024-04-25T10:24:17.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.213+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdb00160ff0 con 0x7fdb0006afe0 2024-04-25T10:24:17.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.825+0000 7fdafcc00640 1 -- 172.21.15.45:0/2702272889 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdaf00f04a0 con 0x7fdb0006afe0 2024-04-25T10:24:17.829 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T10:24:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.837+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdadc066f60 msgr2=0x7fdadc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.837+0000 7fdb05c00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdadc066f60 0x7fdadc069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb0006e7f0 tx=0x7fdae8009190 comp rx=0 tx=0).stop 2024-04-25T10:24:17.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.837+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 msgr2=0x7fdb0015fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:17.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.837+0000 7fdb05c00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 0x7fdb0015fd50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdaf001a190 tx=0x7fdaf001ada0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 shutdown_connections 2024-04-25T10:24:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdadc066f60 0x7fdadc069420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb0015dbe0 0x7fdb0015dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb000b2860 0x7fdb00151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 --2- 172.21.15.45:0/2702272889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb0006afe0 0x7fdb0015fd50 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:17.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 >> 172.21.15.45:0/2702272889 conn(0x7fdb00073d10 msgr2=0x7fdb00152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 shutdown_connections 2024-04-25T10:24:17.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:17.841+0000 7fdb05c00640 1 -- 172.21.15.45:0/2702272889 wait complete. 2024-04-25T10:24:17.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48eaa00640 1 Processor -- start 2024-04-25T10:24:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48eaa00640 1 -- start start 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48eaa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48e4157a40 0x7f48e415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48eaa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e4157e20 con 0x7f48e40b2b70 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48eaa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e406c7c0 con 0x7f48e406dfb0 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48eaa00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e406caa0 con 0x7f48e4157a40 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48e9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48e4157a40 0x7f48e415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48e9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f48e4157a40 0x7f48e415aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43268/0 (socket says 172.21.15.45:43268) 2024-04-25T10:24:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48e9600640 1 -- 172.21.15.45:0/2959096258 learned_addr learned my addr 172.21.15.45:0/2959096258 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48ea000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f48e40b2b70 0x7f48e4070150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58438/0 (socket says 172.21.15.45:58438) 2024-04-25T10:24:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48e9600640 1 -- 172.21.15.45:0/2959096258 >> v1:172.21.15.117:6789/0 conn(0x7f48e406dfb0 legacy=0x7f48e40b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48e9600640 1 -- 172.21.15.45:0/2959096258 >> v1:172.21.15.45:6789/0 conn(0x7f48e40b2b70 legacy=0x7f48e4070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.017+0000 7f48e9600640 1 -- 172.21.15.45:0/2959096258 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e406cd80 con 0x7f48e4157a40 2024-04-25T10:24:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48e9600640 1 --2- 172.21.15.45:0/2959096258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e4157a40 0x7f48e415aeb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f48d800bb10 tx=0x7f48d801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a1d13b8cd89aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48e3e00640 1 -- 172.21.15.45:0/2959096258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80b4070 con 0x7f48e4157a40 2024-04-25T10:24:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48e3e00640 1 -- 172.21.15.45:0/2959096258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d80a7b00 con 0x7f48e4157a40 2024-04-25T10:24:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48e3e00640 1 -- 172.21.15.45:0/2959096258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80af440 con 0x7f48e4157a40 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 -- 172.21.15.45:0/2959096258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e4157a40 msgr2=0x7f48e415aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 --2- 172.21.15.45:0/2959096258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e4157a40 0x7f48e415aeb0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f48d800bb10 tx=0x7f48d801ec30 comp rx=0 tx=0).stop 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 -- 172.21.15.45:0/2959096258 shutdown_connections 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 --2- 172.21.15.45:0/2959096258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e4157a40 0x7f48e415aeb0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 -- 172.21.15.45:0/2959096258 >> 172.21.15.45:0/2959096258 conn(0x7f48e40743f0 msgr2=0x7f48e40669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 -- 172.21.15.45:0/2959096258 shutdown_connections 2024-04-25T10:24:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.021+0000 7f48eaa00640 1 -- 172.21.15.45:0/2959096258 wait complete. 2024-04-25T10:24:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 Processor -- start 2024-04-25T10:24:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- start start 2024-04-25T10:24:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48e406dfb0 0x7f48e4144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 0x7f48e40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48e4157a40 0x7f48e40b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e40709e0 con 0x7f48e40b2b70 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4070860 con 0x7f48e406dfb0 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e4070b60 con 0x7f48e4157a40 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 0x7f48e40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 0x7f48e40aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43274/0 (socket says 172.21.15.45:43274) 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 -- 172.21.15.45:0/1881526280 learned_addr learned my addr 172.21.15.45:0/1881526280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 -- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48e4157a40 msgr2=0x7f48e40b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48e4157a40 0x7f48e40b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 -- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48e406dfb0 msgr2=0x7f48e4144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e9600640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48e406dfb0 0x7f48e4144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48e406dfb0 0x7f48e4144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 -- 172.21.15.45:0/1881526280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e40af810 con 0x7f48e40b2b70 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e9600640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48e406dfb0 0x7f48e4144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e8c00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 0x7f48e40aed90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c370 tx=0x7f48d000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00052d0 con 0x7f48e40b2b70 2024-04-25T10:24:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0005470 con 0x7f48e40b2b70 2024-04-25T10:24:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0007260 con 0x7f48e40b2b70 2024-04-25T10:24:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e40afaf0 con 0x7f48e40b2b70 2024-04-25T10:24:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e40b00b0 con 0x7f48e40b2b70 2024-04-25T10:24:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.025+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e4075120 con 0x7f48e40b2b70 2024-04-25T10:24:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.029+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48d0007400 con 0x7f48e40b2b70 2024-04-25T10:24:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.029+0000 7f48e1600640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48c4066f60 0x7f48c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.029+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d00844a0 con 0x7f48e40b2b70 2024-04-25T10:24:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.029+0000 7f48e9600640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48c4066f60 0x7f48c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.029+0000 7f48e9600640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48c4066f60 0x7f48c4069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f48d80b1000 tx=0x7f48d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.033+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48d00554a0 con 0x7f48e40b2b70 2024-04-25T10:24:18.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.197+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48e4071600 con 0x7f48e40b2b70 2024-04-25T10:24:18.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.197+0000 7f48e1600640 1 -- 172.21.15.45:0/1881526280 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16673 (secure 0 0 0) 0x7f48d0054da0 con 0x7f48e40b2b70 2024-04-25T10:24:18.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:18.199 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:17.216388+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48c4066f60 msgr2=0x7f48c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48c4066f60 0x7f48c4069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f48d80b1000 tx=0x7f48d8008040 comp rx=0 tx=0).stop 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 msgr2=0x7f48e40aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 0x7f48e40aed90 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c370 tx=0x7f48d000c830 comp rx=0 tx=0).stop 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 shutdown_connections 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48c4066f60 0x7f48c4069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48e4157a40 0x7f48e40b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48e40b2b70 0x7f48e40aed90 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 --2- 172.21.15.45:0/1881526280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48e406dfb0 0x7f48e4144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 >> 172.21.15.45:0/1881526280 conn(0x7f48e40743f0 msgr2=0x7f48e40ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:18.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 shutdown_connections 2024-04-25T10:24:18.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.201+0000 7f48eaa00640 1 -- 172.21.15.45:0/1881526280 wait complete. 2024-04-25T10:24:18.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 Processor -- start 2024-04-25T10:24:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 -- start start 2024-04-25T10:24:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f732015db40 0x7f732006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73200b5010 0x7f732015df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f732006b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732006bc70 con 0x7f732015db40 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732015e460 con 0x7f73200b5010 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7326600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b5780 con 0x7f732006b2c0 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f732015db40 0x7f732006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f732015db40 0x7f732006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43284/0 (socket says 172.21.15.45:43284) 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 -- 172.21.15.45:0/1535260151 learned_addr learned my addr 172.21.15.45:0/1535260151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325c00640 1 --2- 172.21.15.45:0/1535260151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f732006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7317e00640 1 --2- 172.21.15.45:0/1535260151 >> v2:172.21.15.117:3300/0 conn(0x7f73200b5010 0x7f732015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 -- 172.21.15.45:0/1535260151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 msgr2=0x7f732006b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 --2- 172.21.15.45:0/1535260151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f732006b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 -- 172.21.15.45:0/1535260151 >> v2:172.21.15.117:3300/0 conn(0x7f73200b5010 msgr2=0x7f732015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 --2- 172.21.15.45:0/1535260151 >> v2:172.21.15.117:3300/0 conn(0x7f73200b5010 0x7f732015df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7325200640 1 -- 172.21.15.45:0/1535260151 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200b5a60 con 0x7f732015db40 2024-04-25T10:24:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.361+0000 7f7317e00640 1 --2- 172.21.15.45:0/1535260151 >> v2:172.21.15.117:3300/0 conn(0x7f73200b5010 0x7f732015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7325200640 1 --2- 172.21.15.45:0/1535260151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732015db40 0x7f732006ac90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7320068400 tx=0x7f731001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2f463d209e330e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f731fe00640 1 -- 172.21.15.45:0/1535260151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73100b5070 con 0x7f732015db40 2024-04-25T10:24:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f731fe00640 1 -- 172.21.15.45:0/1535260151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73100a89d0 con 0x7f732015db40 2024-04-25T10:24:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f731fe00640 1 -- 172.21.15.45:0/1535260151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73100a8da0 con 0x7f732015db40 2024-04-25T10:24:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 -- 172.21.15.45:0/1535260151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732015db40 msgr2=0x7f732006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 --2- 172.21.15.45:0/1535260151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732015db40 0x7f732006ac90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7320068400 tx=0x7f731001ed10 comp rx=0 tx=0).stop 2024-04-25T10:24:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 -- 172.21.15.45:0/1535260151 shutdown_connections 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 --2- 172.21.15.45:0/1535260151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f732006b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 --2- 172.21.15.45:0/1535260151 >> v2:172.21.15.117:3300/0 conn(0x7f73200b5010 0x7f732015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 --2- 172.21.15.45:0/1535260151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f732015db40 0x7f732006ac90 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 -- 172.21.15.45:0/1535260151 >> 172.21.15.45:0/1535260151 conn(0x7f73200d17b0 msgr2=0x7f73200d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 -- 172.21.15.45:0/1535260151 shutdown_connections 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.365+0000 7f7326600640 1 -- 172.21.15.45:0/1535260151 wait complete. 2024-04-25T10:24:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 Processor -- start 2024-04-25T10:24:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 -- start start 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f7320145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 0x7f73200c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732015db40 0x7f7320147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b6300 con 0x7f73200b5010 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b6020 con 0x7f732015db40 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b5480 con 0x7f732006b2c0 2024-04-25T10:24:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 0x7f73200c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 0x7f73200c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43292/0 (socket says 172.21.15.45:43292) 2024-04-25T10:24:18.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 -- 172.21.15.45:0/1156075754 learned_addr learned my addr 172.21.15.45:0/1156075754 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:18.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7325200640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f7320145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7325c00640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732015db40 0x7f7320147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 -- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 msgr2=0x7f7320145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f7320145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 -- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732015db40 msgr2=0x7f7320147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732015db40 0x7f7320147920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 -- 172.21.15.45:0/1156075754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200cb7f0 con 0x7f73200b5010 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7317e00640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 0x7f73200c8be0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f730800c7c0 tx=0x7f730800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308007a30 con 0x7f73200b5010 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308007bd0 con 0x7f73200b5010 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080113f0 con 0x7f73200b5010 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73200cd060 con 0x7f73200b5010 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.369+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73200c9120 con 0x7f73200b5010 2024-04-25T10:24:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.373+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7320075120 con 0x7f73200b5010 2024-04-25T10:24:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.373+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7308011590 con 0x7f73200b5010 2024-04-25T10:24:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.373+0000 7f731d600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72fc066f60 0x7f72fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.373+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7308085090 con 0x7f73200b5010 2024-04-25T10:24:18.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.373+0000 7f7325200640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72fc066f60 0x7f72fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.373+0000 7f7325200640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72fc066f60 0x7f72fc069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f73100b2000 tx=0x7f7310009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.377+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7308015020 con 0x7f73200b5010 2024-04-25T10:24:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.541+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73200b3950 con 0x7f73200b5010 2024-04-25T10:24:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.541+0000 7f731d600640 1 -- 172.21.15.45:0/1156075754 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16673 (secure 0 0 0) 0x7f7308055ae0 con 0x7f73200b5010 2024-04-25T10:24:18.546 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:18.546 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:17.216388+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72fc066f60 msgr2=0x7f72fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72fc066f60 0x7f72fc069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f73100b2000 tx=0x7f7310009040 comp rx=0 tx=0).stop 2024-04-25T10:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 msgr2=0x7f73200c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 0x7f73200c8be0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f730800c7c0 tx=0x7f730800cc80 comp rx=0 tx=0).stop 2024-04-25T10:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 shutdown_connections 2024-04-25T10:24:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72fc066f60 0x7f72fc069420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732015db40 0x7f7320147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73200b5010 0x7f73200c8be0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 --2- 172.21.15.45:0/1156075754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f732006b2c0 0x7f7320145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 >> 172.21.15.45:0/1156075754 conn(0x7f73200d17b0 msgr2=0x7f73200b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 shutdown_connections 2024-04-25T10:24:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.545+0000 7f7326600640 1 -- 172.21.15.45:0/1156075754 wait complete. 2024-04-25T10:24:18.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:24:18.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.705+0000 7f7f41e00640 1 Processor -- start 2024-04-25T10:24:18.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.705+0000 7f7f41e00640 1 -- start start 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f41e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0d0d00 0x7f7f3c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f41e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c0cdab0 con 0x7f7f3c0cc200 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f41e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3c0cd210 con 0x7f7f3c06cf40 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f41e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c0cd4f0 con 0x7f7f3c0d0d00 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f33e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7f3c0cc200 0x7f7f3c162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58452/0 (socket says 172.21.15.45:58452) 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f33e00640 1 -- 172.21.15.45:0/3412540715 learned_addr learned my addr 172.21.15.45:0/3412540715 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3aa00640 1 --2- 172.21.15.45:0/3412540715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0d0d00 0x7f7f3c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3a000640 1 -- 172.21.15.45:0/3412540715 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260731814 0 0) 0x7f7f3c0cdab0 con 0x7f7f3c0cc200 2024-04-25T10:24:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3a000640 1 -- 172.21.15.45:0/3412540715 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f20003530 con 0x7f7f3c0cc200 2024-04-25T10:24:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3aa00640 1 -- 172.21.15.45:0/3412540715 >> v1:172.21.15.117:6789/0 conn(0x7f7f3c06cf40 legacy=0x7f7f3c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3a000640 1 -- 172.21.15.45:0/3412540715 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913338518 0 0) 0x7f7f3c0cd210 con 0x7f7f3c06cf40 2024-04-25T10:24:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3aa00640 1 -- 172.21.15.45:0/3412540715 >> v1:172.21.15.45:6789/0 conn(0x7f7f3c0cc200 legacy=0x7f7f3c162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.709+0000 7f7f3aa00640 1 -- 172.21.15.45:0/3412540715 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0cd7d0 con 0x7f7f3c0d0d00 2024-04-25T10:24:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f3aa00640 1 --2- 172.21.15.45:0/3412540715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0d0d00 0x7f7f3c0ccc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097f0 tx=0x7f7f2c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f961dcdf2b1fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f3a000640 1 -- 172.21.15.45:0/3412540715 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0b0070 con 0x7f7f3c0d0d00 2024-04-25T10:24:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f3a000640 1 -- 172.21.15.45:0/3412540715 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0ab600 con 0x7f7f3c0d0d00 2024-04-25T10:24:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f3a000640 1 -- 172.21.15.45:0/3412540715 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0ab9d0 con 0x7f7f3c0d0d00 2024-04-25T10:24:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 -- 172.21.15.45:0/3412540715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0d0d00 msgr2=0x7f7f3c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 --2- 172.21.15.45:0/3412540715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0d0d00 0x7f7f3c0ccc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097f0 tx=0x7f7f2c01c370 comp rx=0 tx=0).stop 2024-04-25T10:24:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 -- 172.21.15.45:0/3412540715 shutdown_connections 2024-04-25T10:24:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 --2- 172.21.15.45:0/3412540715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0d0d00 0x7f7f3c0ccc30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 -- 172.21.15.45:0/3412540715 >> 172.21.15.45:0/3412540715 conn(0x7f7f3c072290 msgr2=0x7f7f3c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 -- 172.21.15.45:0/3412540715 shutdown_connections 2024-04-25T10:24:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 -- 172.21.15.45:0/3412540715 wait complete. 2024-04-25T10:24:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 Processor -- start 2024-04-25T10:24:18.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.713+0000 7f7f41e00640 1 -- start start 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 0x7f7f3c0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0cc200 0x7f7f3c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f3c0d0d00 0x7f7f3c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c162a10 con 0x7f7f3c06cf40 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c162890 con 0x7f7f3c0d0d00 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c16c690 con 0x7f7f3c0cc200 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 0x7f7f3c0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 0x7f7f3c0ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43300/0 (socket says 172.21.15.45:43300) 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 -- 172.21.15.45:0/846066031 learned_addr learned my addr 172.21.15.45:0/846066031 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 -- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0cc200 msgr2=0x7f7f3c0ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0cc200 0x7f7f3c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 -- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f3c0d0d00 msgr2=0x7f7f3c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f3c0d0d00 0x7f7f3c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 -- 172.21.15.45:0/846066031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0adec0 con 0x7f7f3c06cf40 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f3aa00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 0x7f7f3c0ad030 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097f0 tx=0x7f7f2c01abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0bb020 con 0x7f7f3c06cf40 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0b1040 con 0x7f7f3c06cf40 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0b0050 con 0x7f7f3c06cf40 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c0ae1a0 con 0x7f7f3c06cf40 2024-04-25T10:24:18.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c0ae760 con 0x7f7f3c06cf40 2024-04-25T10:24:18.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c04ef30 con 0x7f7f3c06cf40 2024-04-25T10:24:18.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.717+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f2c0be070 con 0x7f7f3c06cf40 2024-04-25T10:24:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.721+0000 7f7f33400640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7f18066f60 0x7f7f18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.721+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f2c11e6a0 con 0x7f7f3c06cf40 2024-04-25T10:24:18.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.721+0000 7f7f33e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7f18066f60 0x7f7f18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:18.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.721+0000 7f7f33e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7f18066f60 0x7f7f18069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f24007fd0 tx=0x7f7f24007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:18.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.725+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f2c015ff0 con 0x7f7f3c06cf40 2024-04-25T10:24:18.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:18.921+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7f3c072ff0 con 0x7f7f3c06cf40 2024-04-25T10:24:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.841+0000 7f7f33400640 1 -- 172.21.15.45:0/846066031 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7f2c0ef6b0 con 0x7f7f3c06cf40 2024-04-25T10:24:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7f18066f60 msgr2=0x7f7f18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7f18066f60 0x7f7f18069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7f24007fd0 tx=0x7f7f24007900 comp rx=0 tx=0).stop 2024-04-25T10:24:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 msgr2=0x7f7f3c0ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 0x7f7f3c0ad030 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097f0 tx=0x7f7f2c01abb0 comp rx=0 tx=0).stop 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 shutdown_connections 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7f18066f60 0x7f7f18069420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f3c0d0d00 0x7f7f3c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f3c0cc200 0x7f7f3c0ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 --2- 172.21.15.45:0/846066031 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f3c06cf40 0x7f7f3c0ad030 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 >> 172.21.15.45:0/846066031 conn(0x7f7f3c072290 msgr2=0x7f7f3c162180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 shutdown_connections 2024-04-25T10:24:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:19.845+0000 7f7f41e00640 1 -- 172.21.15.45:0/846066031 wait complete. 2024-04-25T10:24:19.870 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:24:19.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:24:20.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.017+0000 7f443e800640 1 Processor -- start 2024-04-25T10:24:20.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.017+0000 7f443e800640 1 -- start start 2024-04-25T10:24:20.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f443e800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44380cc200 0x7f44380ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:20.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f443e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 0x7f443806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:20.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f443e800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44380cdab0 con 0x7f44380d0d00 2024-04-25T10:24:20.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f443e800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443806d8f0 con 0x7f44380cc200 2024-04-25T10:24:20.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f443e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44380cd210 con 0x7f443806cf40 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 0x7f443806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f442fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44380cc200 0x7f44380ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 0x7f443806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:36496/0 (socket says 172.21.15.45:36496) 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 -- 172.21.15.45:0/3765876139 learned_addr learned my addr 172.21.15.45:0/3765876139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437400640 1 --1- 172.21.15.45:0/3765876139 >> v1:172.21.15.117:6789/0 conn(0x7f44380d0d00 0x7f4438162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:38358/0 (socket says 172.21.15.45:38358) 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 -- 172.21.15.45:0/3765876139 >> v2:172.21.15.117:3300/0 conn(0x7f44380cc200 msgr2=0x7f44380ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 --2- 172.21.15.45:0/3765876139 >> v2:172.21.15.117:3300/0 conn(0x7f44380cc200 0x7f44380ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4436a00640 1 -- 172.21.15.45:0/3765876139 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954127839 0 0) 0x7f44380cdab0 con 0x7f44380d0d00 2024-04-25T10:24:20.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 -- 172.21.15.45:0/3765876139 >> v1:172.21.15.117:6789/0 conn(0x7f44380d0d00 legacy=0x7f4438162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:20.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.021+0000 7f4437e00640 1 -- 172.21.15.45:0/3765876139 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380cd4f0 con 0x7f443806cf40 2024-04-25T10:24:20.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f4437e00640 1 --2- 172.21.15.45:0/3765876139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 0x7f443806d3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4430009e50 tx=0x7f443001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ec3eb8b865454b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f4436a00640 1 -- 172.21.15.45:0/3765876139 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44300a5750 con 0x7f443806cf40 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f4436a00640 1 -- 172.21.15.45:0/3765876139 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44300a58f0 con 0x7f443806cf40 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f4436a00640 1 -- 172.21.15.45:0/3765876139 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44300a5cc0 con 0x7f443806cf40 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 -- 172.21.15.45:0/3765876139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 msgr2=0x7f443806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 --2- 172.21.15.45:0/3765876139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 0x7f443806d3b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4430009e50 tx=0x7f443001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 -- 172.21.15.45:0/3765876139 shutdown_connections 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 --2- 172.21.15.45:0/3765876139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443806cf40 0x7f443806d3b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 --2- 172.21.15.45:0/3765876139 >> v2:172.21.15.117:3300/0 conn(0x7f44380cc200 0x7f44380ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 -- 172.21.15.45:0/3765876139 >> 172.21.15.45:0/3765876139 conn(0x7f4438072290 msgr2=0x7f44380726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:20.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 -- 172.21.15.45:0/3765876139 shutdown_connections 2024-04-25T10:24:20.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 -- 172.21.15.45:0/3765876139 wait complete. 2024-04-25T10:24:20.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 Processor -- start 2024-04-25T10:24:20.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.025+0000 7f443e800640 1 -- start start 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443806cf40 0x7f44380999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 0x7f44380c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d0d00 0x7f44380c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438162a10 con 0x7f44380cc200 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438162890 con 0x7f443806cf40 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443815e040 con 0x7f44380d0d00 2024-04-25T10:24:20.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 0x7f44380c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 0x7f44380c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43310/0 (socket says 172.21.15.45:43310) 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 -- 172.21.15.45:0/4219540303 learned_addr learned my addr 172.21.15.45:0/4219540303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f4437400640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443806cf40 0x7f44380999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f4437e00640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d0d00 0x7f44380c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 -- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d0d00 msgr2=0x7f44380c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d0d00 0x7f44380c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 -- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443806cf40 msgr2=0x7f44380999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443806cf40 0x7f44380999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 -- 172.21.15.45:0/4219540303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380c5580 con 0x7f44380cc200 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442fe00640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 0x7f44380c6620 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f442000e930 tx=0x7f442000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420017070 con 0x7f44380cc200 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442000ca00 con 0x7f44380cc200 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000cdb0 con 0x7f44380cc200 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44380c5860 con 0x7f44380cc200 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44380c5e20 con 0x7f44380cc200 2024-04-25T10:24:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443804ef30 con 0x7f44380cc200 2024-04-25T10:24:20.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.029+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f442001a070 con 0x7f44380cc200 2024-04-25T10:24:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.033+0000 7f442f400640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.033+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44200852f0 con 0x7f44380cc200 2024-04-25T10:24:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.033+0000 7f4437400640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.033+0000 7f4437400640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4428000f70 tx=0x7f4428007df0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:20.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.037+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4420055e00 con 0x7f44380cc200 2024-04-25T10:24:20.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.233+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4438072ff0 con 0x7f44380cc200 2024-04-25T10:24:20.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f442f400640 1 -- 172.21.15.45:0/4219540303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4420085110 con 0x7f44380cc200 2024-04-25T10:24:20.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 msgr2=0x7f4414069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4428000f70 tx=0x7f4428007df0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 msgr2=0x7f44380c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 0x7f44380c6620 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f442000e930 tx=0x7f442000edf0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 shutdown_connections 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d0d00 0x7f44380c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44380cc200 0x7f44380c6620 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 --2- 172.21.15.45:0/4219540303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f443806cf40 0x7f44380999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.845+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 >> 172.21.15.45:0/4219540303 conn(0x7f4438072290 msgr2=0x7f4438155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.849+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 shutdown_connections 2024-04-25T10:24:20.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:20.849+0000 7f443e800640 1 -- 172.21.15.45:0/4219540303 wait complete. 2024-04-25T10:24:20.871 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:24:20.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 Processor -- start 2024-04-25T10:24:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 -- start start 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50180cda10 0x7f501806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50180cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50180b2770 con 0x7f501815bc40 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501806c670 con 0x7f50180cda10 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501806c950 con 0x7f50180cf930 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50180cda10 0x7f501806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50180cda10 0x7f501806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43318/0 (socket says 172.21.15.45:43318) 2024-04-25T10:24:21.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f5017e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f501815bc40 0x7f50180b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58468/0 (socket says 172.21.15.45:58468) 2024-04-25T10:24:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 -- 172.21.15.45:0/1461867499 learned_addr learned my addr 172.21.15.45:0/1461867499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ce00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50180cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 -- 172.21.15.45:0/1461867499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 msgr2=0x7f50180cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50180cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 -- 172.21.15.45:0/1461867499 >> v1:172.21.15.45:6789/0 conn(0x7f501815bc40 legacy=0x7f50180b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f500fe00640 1 -- 172.21.15.45:0/1461867499 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501806cc30 con 0x7f50180cda10 2024-04-25T10:24:21.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.017+0000 7f501ce00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50180cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f500fe00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50180cda10 0x7f501806c090 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5000009870 tx=0x7f500001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f9f1eff70e168b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f5017400640 1 -- 172.21.15.45:0/1461867499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000b0070 con 0x7f50180cda10 2024-04-25T10:24:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f5017400640 1 -- 172.21.15.45:0/1461867499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50000ab600 con 0x7f50180cda10 2024-04-25T10:24:21.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f5017400640 1 -- 172.21.15.45:0/1461867499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000ab9d0 con 0x7f50180cda10 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 -- 172.21.15.45:0/1461867499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50180cda10 msgr2=0x7f501806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50180cda10 0x7f501806c090 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5000009870 tx=0x7f500001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 -- 172.21.15.45:0/1461867499 shutdown_connections 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50180cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 --2- 172.21.15.45:0/1461867499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50180cda10 0x7f501806c090 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 -- 172.21.15.45:0/1461867499 >> 172.21.15.45:0/1461867499 conn(0x7f5018066b70 msgr2=0x7f5018066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 -- 172.21.15.45:0/1461867499 shutdown_connections 2024-04-25T10:24:21.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.021+0000 7f501ee00640 1 -- 172.21.15.45:0/1461867499 wait complete. 2024-04-25T10:24:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 Processor -- start 2024-04-25T10:24:21.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- start start 2024-04-25T10:24:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50180cda10 0x7f5018165b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50181532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 0x7f50181f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50180b3890 con 0x7f501815bc40 2024-04-25T10:24:21.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50180b3710 con 0x7f50180cda10 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50180b3a10 con 0x7f50180cf930 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 0x7f50181f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 0x7f50181f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43324/0 (socket says 172.21.15.45:43324) 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 -- 172.21.15.45:0/594174099 learned_addr learned my addr 172.21.15.45:0/594174099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f500fe00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50181532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f5017e00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50180cda10 0x7f5018165b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 -- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 msgr2=0x7f50181532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50181532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 -- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50180cda10 msgr2=0x7f5018165b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50180cda10 0x7f5018165b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 -- 172.21.15.45:0/594174099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018166070 con 0x7f501815bc40 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ce00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 0x7f50181f6a80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f501000cd50 tx=0x7f501000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501000f680 con 0x7f501815bc40 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501000f840 con 0x7f501815bc40 2024-04-25T10:24:21.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501000fc30 con 0x7f501815bc40 2024-04-25T10:24:21.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5018166340 con 0x7f501815bc40 2024-04-25T10:24:21.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5018158430 con 0x7f501815bc40 2024-04-25T10:24:21.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.025+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5018157f70 con 0x7f501815bc40 2024-04-25T10:24:21.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.029+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f501000e3e0 con 0x7f501815bc40 2024-04-25T10:24:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.029+0000 7f5014c00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ff4066f60 0x7f4ff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.029+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5010085dc0 con 0x7f501815bc40 2024-04-25T10:24:21.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.029+0000 7f5017e00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ff4066f60 0x7f4ff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.029+0000 7f5017e00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ff4066f60 0x7f4ff4069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f50180ae770 tx=0x7f5008007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.033+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5018157f70 con 0x7f501815bc40 2024-04-25T10:24:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f501804ef30 con 0x7f501815bc40 2024-04-25T10:24:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f5014c00640 1 -- 172.21.15.45:0/594174099 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5010014030 con 0x7f501815bc40 2024-04-25T10:24:21.229 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:21.230 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ff4066f60 msgr2=0x7f4ff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ff4066f60 0x7f4ff4069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f50180ae770 tx=0x7f5008007cf0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 msgr2=0x7f50181f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 0x7f50181f6a80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f501000cd50 tx=0x7f501000a650 comp rx=0 tx=0).stop 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 shutdown_connections 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4ff4066f60 0x7f4ff4069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f501815bc40 0x7f50181f6a80 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50180cf930 0x7f50181532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 --2- 172.21.15.45:0/594174099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50180cda10 0x7f5018165b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 >> 172.21.15.45:0/594174099 conn(0x7f5018066b70 msgr2=0x7f5018066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 shutdown_connections 2024-04-25T10:24:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.229+0000 7f501ee00640 1 -- 172.21.15.45:0/594174099 wait complete. 2024-04-25T10:24:21.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.389+0000 7f3241400640 1 Processor -- start 2024-04-25T10:24:21.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3241400640 1 -- start start 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3241400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3241400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f323c15b740 0x7f323c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3241400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c071050 con 0x7f323c1550b0 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3241400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c15d040 con 0x7f323c15b740 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3241400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0712f0 con 0x7f323c0d0040 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f323c15b740 0x7f323c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f323c15b740 0x7f323c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43340/0 (socket says 172.21.15.45:43340) 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 -- 172.21.15.45:0/3353773201 learned_addr learned my addr 172.21.15.45:0/3353773201 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3233e00640 1 --2- 172.21.15.45:0/3353773201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 -- 172.21.15.45:0/3353773201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 msgr2=0x7f323c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 --2- 172.21.15.45:0/3353773201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c15b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 -- 172.21.15.45:0/3353773201 >> v1:172.21.15.117:6789/0 conn(0x7f323c1550b0 legacy=0x7f323c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f323b200640 1 -- 172.21.15.45:0/3353773201 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0715d0 con 0x7f323c15b740 2024-04-25T10:24:21.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.393+0000 7f3239e00640 1 -- 172.21.15.45:0/3353773201 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812767266 0 0) 0x7f323c071050 con 0x7f323c1550b0 2024-04-25T10:24:21.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f323b200640 1 --2- 172.21.15.45:0/3353773201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0b3260 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e50 tx=0x7f323401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629b7d6e92237ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3239e00640 1 -- 172.21.15.45:0/3353773201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340a58d0 con 0x7f323c15b740 2024-04-25T10:24:21.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3239e00640 1 -- 172.21.15.45:0/3353773201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340a5a70 con 0x7f323c15b740 2024-04-25T10:24:21.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3239e00640 1 -- 172.21.15.45:0/3353773201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340a5e40 con 0x7f323c15b740 2024-04-25T10:24:21.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 -- 172.21.15.45:0/3353773201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 msgr2=0x7f323c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 --2- 172.21.15.45:0/3353773201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0b3260 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e50 tx=0x7f323401ca00 comp rx=0 tx=0).stop 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 -- 172.21.15.45:0/3353773201 shutdown_connections 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 --2- 172.21.15.45:0/3353773201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0b3260 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 --2- 172.21.15.45:0/3353773201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c15b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 -- 172.21.15.45:0/3353773201 >> 172.21.15.45:0/3353773201 conn(0x7f323c066b70 msgr2=0x7f323c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 -- 172.21.15.45:0/3353773201 shutdown_connections 2024-04-25T10:24:21.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 -- 172.21.15.45:0/3353773201 wait complete. 2024-04-25T10:24:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 Processor -- start 2024-04-25T10:24:21.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.397+0000 7f3241400640 1 -- start start 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f323c1550b0 0x7f323c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c15bd70 con 0x7f323c15b740 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c15bbf0 con 0x7f323c1550b0 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c15c030 con 0x7f323c0d0040 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43344/0 (socket says 172.21.15.45:43344) 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 -- 172.21.15.45:0/1142887414 learned_addr learned my addr 172.21.15.45:0/1142887414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3233e00640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f323c1550b0 0x7f323c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323a800640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 -- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 msgr2=0x7f323c158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 -- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f323c1550b0 msgr2=0x7f323c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f323c1550b0 0x7f323c145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 -- 172.21.15.45:0/1142887414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c159300 con 0x7f323c15b740 2024-04-25T10:24:21.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f323b200640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0ad840 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f32340af040 tx=0x7f32340197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340a58d0 con 0x7f323c15b740 2024-04-25T10:24:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340be070 con 0x7f323c15b740 2024-04-25T10:24:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234009510 con 0x7f323c15b740 2024-04-25T10:24:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c1595e0 con 0x7f323c15b740 2024-04-25T10:24:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c159ba0 con 0x7f323c15b740 2024-04-25T10:24:21.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c04ef30 con 0x7f323c15b740 2024-04-25T10:24:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.401+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32340b8470 con 0x7f323c15b740 2024-04-25T10:24:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.405+0000 7f3232a00640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.405+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3234120540 con 0x7f323c15b740 2024-04-25T10:24:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.405+0000 7f323a800640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.405+0000 7f323a800640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3218066f60 0x7f3218069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f322c000f70 tx=0x7f322c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.409+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3234016620 con 0x7f323c15b740 2024-04-25T10:24:21.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.569+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f323c0cc270 con 0x7f323c15b740 2024-04-25T10:24:21.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.569+0000 7f3232a00640 1 -- 172.21.15.45:0/1142887414 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16660 (secure 0 0 0) 0x7f32340f5e20 con 0x7f323c15b740 2024-04-25T10:24:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:18.823568+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3218066f60 msgr2=0x7f3218069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3218066f60 0x7f3218069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f322c000f70 tx=0x7f322c009040 comp rx=0 tx=0).stop 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 msgr2=0x7f323c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0ad840 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f32340af040 tx=0x7f32340197a0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 shutdown_connections 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3218066f60 0x7f3218069420 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f323c15b740 0x7f323c0ad840 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f323c1550b0 0x7f323c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 --2- 172.21.15.45:0/1142887414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f323c0d0040 0x7f323c158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 >> 172.21.15.45:0/1142887414 conn(0x7f323c066b70 msgr2=0x7f323c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 shutdown_connections 2024-04-25T10:24:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.573+0000 7f3241400640 1 -- 172.21.15.45:0/1142887414 wait complete. 2024-04-25T10:24:21.593 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:24:21.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:24:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 Processor -- start 2024-04-25T10:24:21.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 -- start start 2024-04-25T10:24:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f08340ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f083415f570 0x7f08340ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08340d15e0 con 0x7f08340c57d0 2024-04-25T10:24:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08340cfef0 con 0x7f083415f570 2024-04-25T10:24:21.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f083b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08340d0140 con 0x7f083415f190 2024-04-25T10:24:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f0839400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f08340c57d0 0x7f08340c5c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58476/0 (socket says 172.21.15.45:58476) 2024-04-25T10:24:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.733+0000 7f0839400640 1 -- 172.21.15.45:0/4130311048 learned_addr learned my addr 172.21.15.45:0/4130311048 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0838a00640 1 --2- 172.21.15.45:0/4130311048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f08340ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0833e00640 1 --2- 172.21.15.45:0/4130311048 >> v2:172.21.15.117:3300/0 conn(0x7f083415f570 0x7f08340ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0833400640 1 -- 172.21.15.45:0/4130311048 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286903517 0 0) 0x7f082c0014a0 con 0x7f08340c57d0 2024-04-25T10:24:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0833400640 1 -- 172.21.15.45:0/4130311048 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f081c003530 con 0x7f08340c57d0 2024-04-25T10:24:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0838a00640 1 -- 172.21.15.45:0/4130311048 >> v2:172.21.15.117:3300/0 conn(0x7f083415f570 msgr2=0x7f08340ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0838a00640 1 --2- 172.21.15.45:0/4130311048 >> v2:172.21.15.117:3300/0 conn(0x7f083415f570 0x7f08340ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0838a00640 1 -- 172.21.15.45:0/4130311048 >> v1:172.21.15.45:6789/0 conn(0x7f08340c57d0 legacy=0x7f08340c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0838a00640 1 -- 172.21.15.45:0/4130311048 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08340d0420 con 0x7f083415f190 2024-04-25T10:24:21.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.737+0000 7f0838a00640 1 --2- 172.21.15.45:0/4130311048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f08340ce290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08280097c0 tx=0x7f082801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef7bc5798ff95fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f0833400640 1 -- 172.21.15.45:0/4130311048 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280b0070 con 0x7f083415f190 2024-04-25T10:24:21.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f0833400640 1 -- 172.21.15.45:0/4130311048 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08280ab600 con 0x7f083415f190 2024-04-25T10:24:21.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f0833400640 1 -- 172.21.15.45:0/4130311048 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280ab9d0 con 0x7f083415f190 2024-04-25T10:24:21.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- 172.21.15.45:0/4130311048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 msgr2=0x7f08340ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 --2- 172.21.15.45:0/4130311048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f08340ce290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f08280097c0 tx=0x7f082801c350 comp rx=0 tx=0).stop 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- 172.21.15.45:0/4130311048 shutdown_connections 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 --2- 172.21.15.45:0/4130311048 >> v2:172.21.15.117:3300/0 conn(0x7f083415f570 0x7f08340ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 --2- 172.21.15.45:0/4130311048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f08340ce290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- 172.21.15.45:0/4130311048 >> 172.21.15.45:0/4130311048 conn(0x7f08340d1910 msgr2=0x7f08340d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- 172.21.15.45:0/4130311048 shutdown_connections 2024-04-25T10:24:21.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- 172.21.15.45:0/4130311048 wait complete. 2024-04-25T10:24:21.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 Processor -- start 2024-04-25T10:24:21.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- start start 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 0x7f08340c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f083415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f083415f570 0x7f08341577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0834168a50 con 0x7f08340c57d0 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08340d2640 con 0x7f083415f570 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f083b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0834168bd0 con 0x7f083415f190 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f0838a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 0x7f08340c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f0838a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 0x7f08340c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43348/0 (socket says 172.21.15.45:43348) 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.741+0000 7f0838a00640 1 -- 172.21.15.45:0/214877640 learned_addr learned my addr 172.21.15.45:0/214877640 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:21.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0833e00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f083415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0839400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f083415f570 0x7f08341577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0838a00640 1 -- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 msgr2=0x7f083415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0838a00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f083415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0838a00640 1 -- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f083415f570 msgr2=0x7f08341577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0838a00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f083415f570 0x7f08341577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0838a00640 1 -- 172.21.15.45:0/214877640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08340c9c40 con 0x7f08340c57d0 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0839400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f083415f570 0x7f08341577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:21.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0838a00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 0x7f08340c93a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f08280ae040 tx=0x7f08280190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280b0040 con 0x7f08340c57d0 2024-04-25T10:24:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08280b9070 con 0x7f08340c57d0 2024-04-25T10:24:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08340c9f20 con 0x7f08340c57d0 2024-04-25T10:24:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08280b4650 con 0x7f08340c57d0 2024-04-25T10:24:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08340ca4e0 con 0x7f08340c57d0 2024-04-25T10:24:21.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083404ef30 con 0x7f08340c57d0 2024-04-25T10:24:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08280b47f0 con 0x7f08340c57d0 2024-04-25T10:24:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0830c00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0814066f60 0x7f0814069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f082811eec0 con 0x7f08340c57d0 2024-04-25T10:24:21.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.745+0000 7f0833e00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0814066f60 0x7f0814069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:21.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.749+0000 7f0833e00640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0814066f60 0x7f0814069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0834158d40 tx=0x7f0820008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:21.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.749+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0828016090 con 0x7f08340c57d0 2024-04-25T10:24:21.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.909+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f083415ba30 con 0x7f0814066f60 2024-04-25T10:24:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.917+0000 7f0830c00640 1 -- 172.21.15.45:0/214877640 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252042 (secure 0 0 0) 0x7f083415ba30 con 0x7f0814066f60 2024-04-25T10:24:21.923 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:21.925 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:24:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0814066f60 msgr2=0x7f0814069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0814066f60 0x7f0814069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0834158d40 tx=0x7f0820008040 comp rx=0 tx=0).stop 2024-04-25T10:24:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 msgr2=0x7f08340c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 0x7f08340c93a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f08280ae040 tx=0x7f08280190f0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 shutdown_connections 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0814066f60 0x7f0814069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f083415f570 0x7f08341577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f083415f190 0x7f083415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 --2- 172.21.15.45:0/214877640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08340c57d0 0x7f08340c93a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:21.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 >> 172.21.15.45:0/214877640 conn(0x7f08340d1910 msgr2=0x7f083406ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 shutdown_connections 2024-04-25T10:24:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:21.921+0000 7f083b400640 1 -- 172.21.15.45:0/214877640 wait complete. 2024-04-25T10:24:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-04-25T10:24:20.962314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389848,"kb_used_data":9880,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754584872,"statfs":{"total":773094113280,"available":772694908928,"internally_reserved":0,"allocated":10117120,"data_stored":4996635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157751"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.836258+0000","last_change":"2024-04-25T10:24:17.832232+0000","last_active":"2024-04-25T10:24:17.836258+0000","last_peered":"2024-04-25T10:24:17.836258+0000","last_clean":"2024-04-25T10:24:17.836258+0000","last_became_active":"2024-04-25T10:24:17.831574+0000","last_became_peered":"2024-04-25T10:24:17.831574+0000","last_unstale":"2024-04-25T10:24:17.836258+0000","last_undegraded":"2024-04-25T10:24:17.836258+0000","last_fullsized":"2024-04-25T10:24:17.836258+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:02.003009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.835743+0000","last_change":"2024-04-25T10:24:17.833876+0000","last_active":"2024-04-25T10:24:17.835743+0000","last_peered":"2024-04-25T10:24:17.835743+0000","last_clean":"2024-04-25T10:24:17.835743+0000","last_became_active":"2024-04-25T10:24:17.833547+0000","last_became_peered":"2024-04-25T10:24:17.833547+0000","last_unstale":"2024-04-25T10:24:17.835743+0000","last_undegraded":"2024-04-25T10:24:17.835743+0000","last_fullsized":"2024-04-25T10:24:17.835743+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:33.654465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827910+0000","last_change":"2024-04-25T10:24:17.836719+0000","last_active":"2024-04-25T10:24:18.827910+0000","last_peered":"2024-04-25T10:24:18.827910+0000","last_clean":"2024-04-25T10:24:18.827910+0000","last_became_active":"2024-04-25T10:24:17.836068+0000","last_became_peered":"2024-04-25T10:24:17.836068+0000","last_unstale":"2024-04-25T10:24:18.827910+0000","last_undegraded":"2024-04-25T10:24:18.827910+0000","last_fullsized":"2024-04-25T10:24:18.827910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:39.411945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829699+0000","last_change":"2024-04-25T10:24:17.835190+0000","last_active":"2024-04-25T10:24:18.829699+0000","last_peered":"2024-04-25T10:24:18.829699+0000","last_clean":"2024-04-25T10:24:18.829699+0000","last_became_active":"2024-04-25T10:24:17.834919+0000","last_became_peered":"2024-04-25T10:24:17.834919+0000","last_unstale":"2024-04-25T10:24:18.829699+0000","last_undegraded":"2024-04-25T10:24:18.829699+0000","last_fullsized":"2024-04-25T10:24:18.829699+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:46.366686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.845253+0000","last_change":"2024-04-25T10:24:17.839206+0000","last_active":"2024-04-25T10:24:17.845253+0000","last_peered":"2024-04-25T10:24:17.845253+0000","last_clean":"2024-04-25T10:24:17.845253+0000","last_became_active":"2024-04-25T10:24:17.838936+0000","last_became_peered":"2024-04-25T10:24:17.838936+0000","last_unstale":"2024-04-25T10:24:17.845253+0000","last_undegraded":"2024-04-25T10:24:17.845253+0000","last_fullsized":"2024-04-25T10:24:17.845253+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:11.614791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827572+0000","last_change":"2024-04-25T10:24:17.831573+0000","last_active":"2024-04-25T10:24:18.827572+0000","last_peered":"2024-04-25T10:24:18.827572+0000","last_clean":"2024-04-25T10:24:18.827572+0000","last_became_active":"2024-04-25T10:24:17.831251+0000","last_became_peered":"2024-04-25T10:24:17.831251+0000","last_unstale":"2024-04-25T10:24:18.827572+0000","last_undegraded":"2024-04-25T10:24:18.827572+0000","last_fullsized":"2024-04-25T10:24:18.827572+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:13.268129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838808+0000","last_change":"2024-04-25T10:24:17.837356+0000","last_active":"2024-04-25T10:24:17.838808+0000","last_peered":"2024-04-25T10:24:17.838808+0000","last_clean":"2024-04-25T10:24:17.838808+0000","last_became_active":"2024-04-25T10:24:17.837023+0000","last_became_peered":"2024-04-25T10:24:17.837023+0000","last_unstale":"2024-04-25T10:24:17.838808+0000","last_undegraded":"2024-04-25T10:24:17.838808+0000","last_fullsized":"2024-04-25T10:24:17.838808+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:03.490041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.853224+0000","last_change":"2024-04-25T10:24:17.849684+0000","last_active":"2024-04-25T10:24:17.853224+0000","last_peered":"2024-04-25T10:24:17.853224+0000","last_clean":"2024-04-25T10:24:17.853224+0000","last_became_active":"2024-04-25T10:24:17.846322+0000","last_became_peered":"2024-04-25T10:24:17.846322+0000","last_unstale":"2024-04-25T10:24:17.853224+0000","last_undegraded":"2024-04-25T10:24:17.853224+0000","last_fullsized":"2024-04-25T10:24:17.853224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:25.140910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.836236+0000","last_change":"2024-04-25T10:24:17.826600+0000","last_active":"2024-04-25T10:24:17.836236+0000","last_peered":"2024-04-25T10:24:17.836236+0000","last_clean":"2024-04-25T10:24:17.836236+0000","last_became_active":"2024-04-25T10:24:17.826329+0000","last_became_peered":"2024-04-25T10:24:17.826329+0000","last_unstale":"2024-04-25T10:24:17.836236+0000","last_undegraded":"2024-04-25T10:24:17.836236+0000","last_fullsized":"2024-04-25T10:24:17.836236+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:58.745217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828011+0000","last_change":"2024-04-25T10:24:17.830454+0000","last_active":"2024-04-25T10:24:18.828011+0000","last_peered":"2024-04-25T10:24:18.828011+0000","last_clean":"2024-04-25T10:24:18.828011+0000","last_became_active":"2024-04-25T10:24:17.830117+0000","last_became_peered":"2024-04-25T10:24:17.830117+0000","last_unstale":"2024-04-25T10:24:18.828011+0000","last_undegraded":"2024-04-25T10:24:18.828011+0000","last_fullsized":"2024-04-25T10:24:18.828011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830474+0000","last_change":"2024-04-25T10:24:17.846808+0000","last_active":"2024-04-25T10:24:18.830474+0000","last_peered":"2024-04-25T10:24:18.830474+0000","last_clean":"2024-04-25T10:24:18.830474+0000","last_became_active":"2024-04-25T10:24:17.846484+0000","last_became_peered":"2024-04-25T10:24:17.846484+0000","last_unstale":"2024-04-25T10:24:18.830474+0000","last_undegraded":"2024-04-25T10:24:18.830474+0000","last_fullsized":"2024-04-25T10:24:18.830474+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:53.427172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.847011+0000","last_change":"2024-04-25T10:24:17.841833+0000","last_active":"2024-04-25T10:24:17.847011+0000","last_peered":"2024-04-25T10:24:17.847011+0000","last_clean":"2024-04-25T10:24:17.847011+0000","last_became_active":"2024-04-25T10:24:17.841470+0000","last_became_peered":"2024-04-25T10:24:17.841470+0000","last_unstale":"2024-04-25T10:24:17.847011+0000","last_undegraded":"2024-04-25T10:24:17.847011+0000","last_fullsized":"2024-04-25T10:24:17.847011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.507016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838383+0000","last_change":"2024-04-25T10:24:17.832792+0000","last_active":"2024-04-25T10:24:17.838383+0000","last_peered":"2024-04-25T10:24:17.838383+0000","last_clean":"2024-04-25T10:24:17.838383+0000","last_became_active":"2024-04-25T10:24:17.832489+0000","last_became_peered":"2024-04-25T10:24:17.832489+0000","last_unstale":"2024-04-25T10:24:17.838383+0000","last_undegraded":"2024-04-25T10:24:17.838383+0000","last_fullsized":"2024-04-25T10:24:17.838383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:52.688025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838949+0000","last_change":"2024-04-25T10:24:17.835599+0000","last_active":"2024-04-25T10:24:17.838949+0000","last_peered":"2024-04-25T10:24:17.838949+0000","last_clean":"2024-04-25T10:24:17.838949+0000","last_became_active":"2024-04-25T10:24:17.835068+0000","last_became_peered":"2024-04-25T10:24:17.835068+0000","last_unstale":"2024-04-25T10:24:17.838949+0000","last_undegraded":"2024-04-25T10:24:17.838949+0000","last_fullsized":"2024-04-25T10:24:17.838949+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:38.685064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.835119+0000","last_change":"2024-04-25T10:24:17.829392+0000","last_active":"2024-04-25T10:24:17.835119+0000","last_peered":"2024-04-25T10:24:17.835119+0000","last_clean":"2024-04-25T10:24:17.835119+0000","last_became_active":"2024-04-25T10:24:17.828557+0000","last_became_peered":"2024-04-25T10:24:17.828557+0000","last_unstale":"2024-04-25T10:24:17.835119+0000","last_undegraded":"2024-04-25T10:24:17.835119+0000","last_fullsized":"2024-04-25T10:24:17.835119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:54.913191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827452+0000","last_change":"2024-04-25T10:24:17.838925+0000","last_active":"2024-04-25T10:24:18.827452+0000","last_peered":"2024-04-25T10:24:18.827452+0000","last_clean":"2024-04-25T10:24:18.827452+0000","last_became_active":"2024-04-25T10:24:17.836659+0000","last_became_peered":"2024-04-25T10:24:17.836659+0000","last_unstale":"2024-04-25T10:24:18.827452+0000","last_undegraded":"2024-04-25T10:24:18.827452+0000","last_fullsized":"2024-04-25T10:24:18.827452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:17.675318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.847077+0000","last_change":"2024-04-25T10:24:17.839641+0000","last_active":"2024-04-25T10:24:17.847077+0000","last_peered":"2024-04-25T10:24:17.847077+0000","last_clean":"2024-04-25T10:24:17.847077+0000","last_became_active":"2024-04-25T10:24:17.839371+0000","last_became_peered":"2024-04-25T10:24:17.839371+0000","last_unstale":"2024-04-25T10:24:17.847077+0000","last_undegraded":"2024-04-25T10:24:17.847077+0000","last_fullsized":"2024-04-25T10:24:17.847077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:32.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830185+0000","last_change":"2024-04-25T10:24:17.840179+0000","last_active":"2024-04-25T10:24:18.830185+0000","last_peered":"2024-04-25T10:24:18.830185+0000","last_clean":"2024-04-25T10:24:18.830185+0000","last_became_active":"2024-04-25T10:24:17.839444+0000","last_became_peered":"2024-04-25T10:24:17.839444+0000","last_unstale":"2024-04-25T10:24:18.830185+0000","last_undegraded":"2024-04-25T10:24:18.830185+0000","last_fullsized":"2024-04-25T10:24:18.830185+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:32.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844129+0000","last_change":"2024-04-25T10:24:17.840824+0000","last_active":"2024-04-25T10:24:17.844129+0000","last_peered":"2024-04-25T10:24:17.844129+0000","last_clean":"2024-04-25T10:24:17.844129+0000","last_became_active":"2024-04-25T10:24:17.839970+0000","last_became_peered":"2024-04-25T10:24:17.839970+0000","last_unstale":"2024-04-25T10:24:17.844129+0000","last_undegraded":"2024-04-25T10:24:17.844129+0000","last_fullsized":"2024-04-25T10:24:17.844129+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:58.876843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844262+0000","last_change":"2024-04-25T10:24:17.839899+0000","last_active":"2024-04-25T10:24:17.844262+0000","last_peered":"2024-04-25T10:24:17.844262+0000","last_clean":"2024-04-25T10:24:17.844262+0000","last_became_active":"2024-04-25T10:24:17.839506+0000","last_became_peered":"2024-04-25T10:24:17.839506+0000","last_unstale":"2024-04-25T10:24:17.844262+0000","last_undegraded":"2024-04-25T10:24:17.844262+0000","last_fullsized":"2024-04-25T10:24:17.844262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:07.108402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829056+0000","last_change":"2024-04-25T10:24:17.825028+0000","last_active":"2024-04-25T10:24:18.829056+0000","last_peered":"2024-04-25T10:24:18.829056+0000","last_clean":"2024-04-25T10:24:18.829056+0000","last_became_active":"2024-04-25T10:24:17.824766+0000","last_became_peered":"2024-04-25T10:24:17.824766+0000","last_unstale":"2024-04-25T10:24:18.829056+0000","last_undegraded":"2024-04-25T10:24:18.829056+0000","last_fullsized":"2024-04-25T10:24:18.829056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.418507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838971+0000","last_change":"2024-04-25T10:24:17.836259+0000","last_active":"2024-04-25T10:24:17.838971+0000","last_peered":"2024-04-25T10:24:17.838971+0000","last_clean":"2024-04-25T10:24:17.838971+0000","last_became_active":"2024-04-25T10:24:17.835970+0000","last_became_peered":"2024-04-25T10:24:17.835970+0000","last_unstale":"2024-04-25T10:24:17.838971+0000","last_undegraded":"2024-04-25T10:24:17.838971+0000","last_fullsized":"2024-04-25T10:24:17.838971+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.448524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846657+0000","last_change":"2024-04-25T10:24:17.841926+0000","last_active":"2024-04-25T10:24:17.846657+0000","last_peered":"2024-04-25T10:24:17.846657+0000","last_clean":"2024-04-25T10:24:17.846657+0000","last_became_active":"2024-04-25T10:24:17.841520+0000","last_became_peered":"2024-04-25T10:24:17.841520+0000","last_unstale":"2024-04-25T10:24:17.846657+0000","last_undegraded":"2024-04-25T10:24:17.846657+0000","last_fullsized":"2024-04-25T10:24:17.846657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.485882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828860+0000","last_change":"2024-04-25T10:24:17.839112+0000","last_active":"2024-04-25T10:24:18.828860+0000","last_peered":"2024-04-25T10:24:18.828860+0000","last_clean":"2024-04-25T10:24:18.828860+0000","last_became_active":"2024-04-25T10:24:17.838373+0000","last_became_peered":"2024-04-25T10:24:17.838373+0000","last_unstale":"2024-04-25T10:24:18.828860+0000","last_undegraded":"2024-04-25T10:24:18.828860+0000","last_fullsized":"2024-04-25T10:24:18.828860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:08.432023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830376+0000","last_change":"2024-04-25T10:24:17.849004+0000","last_active":"2024-04-25T10:24:18.830376+0000","last_peered":"2024-04-25T10:24:18.830376+0000","last_clean":"2024-04-25T10:24:18.830376+0000","last_became_active":"2024-04-25T10:24:17.848680+0000","last_became_peered":"2024-04-25T10:24:17.848680+0000","last_unstale":"2024-04-25T10:24:18.830376+0000","last_undegraded":"2024-04-25T10:24:18.830376+0000","last_fullsized":"2024-04-25T10:24:18.830376+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:21.203313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833574+0000","last_change":"2024-04-25T10:24:17.830053+0000","last_active":"2024-04-25T10:24:17.833574+0000","last_peered":"2024-04-25T10:24:17.833574+0000","last_clean":"2024-04-25T10:24:17.833574+0000","last_became_active":"2024-04-25T10:24:17.829191+0000","last_became_peered":"2024-04-25T10:24:17.829191+0000","last_unstale":"2024-04-25T10:24:17.833574+0000","last_undegraded":"2024-04-25T10:24:17.833574+0000","last_fullsized":"2024-04-25T10:24:17.833574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.150590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833536+0000","last_change":"2024-04-25T10:24:17.826289+0000","last_active":"2024-04-25T10:24:17.833536+0000","last_peered":"2024-04-25T10:24:17.833536+0000","last_clean":"2024-04-25T10:24:17.833536+0000","last_became_active":"2024-04-25T10:24:17.825209+0000","last_became_peered":"2024-04-25T10:24:17.825209+0000","last_unstale":"2024-04-25T10:24:17.833536+0000","last_undegraded":"2024-04-25T10:24:17.833536+0000","last_fullsized":"2024-04-25T10:24:17.833536+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:51.816635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.834476+0000","last_change":"2024-04-25T10:24:17.825774+0000","last_active":"2024-04-25T10:24:17.834476+0000","last_peered":"2024-04-25T10:24:17.834476+0000","last_clean":"2024-04-25T10:24:17.834476+0000","last_became_active":"2024-04-25T10:24:17.825479+0000","last_became_peered":"2024-04-25T10:24:17.825479+0000","last_unstale":"2024-04-25T10:24:17.834476+0000","last_undegraded":"2024-04-25T10:24:17.834476+0000","last_fullsized":"2024-04-25T10:24:17.834476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:19.500246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829213+0000","last_change":"2024-04-25T10:24:17.824823+0000","last_active":"2024-04-25T10:24:18.829213+0000","last_peered":"2024-04-25T10:24:18.829213+0000","last_clean":"2024-04-25T10:24:18.829213+0000","last_became_active":"2024-04-25T10:24:17.824534+0000","last_became_peered":"2024-04-25T10:24:17.824534+0000","last_unstale":"2024-04-25T10:24:18.829213+0000","last_undegraded":"2024-04-25T10:24:18.829213+0000","last_fullsized":"2024-04-25T10:24:18.829213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844166+0000","last_change":"2024-04-25T10:24:17.840826+0000","last_active":"2024-04-25T10:24:17.844166+0000","last_peered":"2024-04-25T10:24:17.844166+0000","last_clean":"2024-04-25T10:24:17.844166+0000","last_became_active":"2024-04-25T10:24:17.840550+0000","last_became_peered":"2024-04-25T10:24:17.840550+0000","last_unstale":"2024-04-25T10:24:17.844166+0000","last_undegraded":"2024-04-25T10:24:17.844166+0000","last_fullsized":"2024-04-25T10:24:17.844166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:53.317387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827395+0000","last_change":"2024-04-25T10:24:17.826375+0000","last_active":"2024-04-25T10:24:18.827395+0000","last_peered":"2024-04-25T10:24:18.827395+0000","last_clean":"2024-04-25T10:24:18.827395+0000","last_became_active":"2024-04-25T10:24:17.826112+0000","last_became_peered":"2024-04-25T10:24:17.826112+0000","last_unstale":"2024-04-25T10:24:18.827395+0000","last_undegraded":"2024-04-25T10:24:18.827395+0000","last_fullsized":"2024-04-25T10:24:18.827395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.218872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827472+0000","last_change":"2024-04-25T10:24:17.828435+0000","last_active":"2024-04-25T10:24:18.827472+0000","last_peered":"2024-04-25T10:24:18.827472+0000","last_clean":"2024-04-25T10:24:18.827472+0000","last_became_active":"2024-04-25T10:24:17.828154+0000","last_became_peered":"2024-04-25T10:24:17.828154+0000","last_unstale":"2024-04-25T10:24:18.827472+0000","last_undegraded":"2024-04-25T10:24:18.827472+0000","last_fullsized":"2024-04-25T10:24:18.827472+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:07.810903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846589+0000","last_change":"2024-04-25T10:24:17.835706+0000","last_active":"2024-04-25T10:24:17.846589+0000","last_peered":"2024-04-25T10:24:17.846589+0000","last_clean":"2024-04-25T10:24:17.846589+0000","last_became_active":"2024-04-25T10:24:17.835428+0000","last_became_peered":"2024-04-25T10:24:17.835428+0000","last_unstale":"2024-04-25T10:24:17.846589+0000","last_undegraded":"2024-04-25T10:24:17.846589+0000","last_fullsized":"2024-04-25T10:24:17.846589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:58.752127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.855921+0000","last_change":"2024-04-25T10:24:16.831295+0000","last_active":"2024-04-25T10:24:17.855921+0000","last_peered":"2024-04-25T10:24:17.855921+0000","last_clean":"2024-04-25T10:24:17.855921+0000","last_became_active":"2024-04-25T10:24:16.830078+0000","last_became_peered":"2024-04-25T10:24:16.830078+0000","last_unstale":"2024-04-25T10:24:17.855921+0000","last_undegraded":"2024-04-25T10:24:17.855921+0000","last_fullsized":"2024-04-25T10:24:17.855921+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:49.339835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.836277+0000","last_change":"2024-04-25T10:24:17.832378+0000","last_active":"2024-04-25T10:24:17.836277+0000","last_peered":"2024-04-25T10:24:17.836277+0000","last_clean":"2024-04-25T10:24:17.836277+0000","last_became_active":"2024-04-25T10:24:17.832031+0000","last_became_peered":"2024-04-25T10:24:17.832031+0000","last_unstale":"2024-04-25T10:24:17.836277+0000","last_undegraded":"2024-04-25T10:24:17.836277+0000","last_fullsized":"2024-04-25T10:24:17.836277+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:40.852593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828024+0000","last_change":"2024-04-25T10:24:16.833266+0000","last_active":"2024-04-25T10:24:18.828024+0000","last_peered":"2024-04-25T10:24:18.828024+0000","last_clean":"2024-04-25T10:24:18.828024+0000","last_became_active":"2024-04-25T10:24:16.832972+0000","last_became_peered":"2024-04-25T10:24:16.832972+0000","last_unstale":"2024-04-25T10:24:18.828024+0000","last_undegraded":"2024-04-25T10:24:18.828024+0000","last_fullsized":"2024-04-25T10:24:18.828024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:01.710862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846988+0000","last_change":"2024-04-25T10:24:17.838921+0000","last_active":"2024-04-25T10:24:17.846988+0000","last_peered":"2024-04-25T10:24:17.846988+0000","last_clean":"2024-04-25T10:24:17.846988+0000","last_became_active":"2024-04-25T10:24:17.838644+0000","last_became_peered":"2024-04-25T10:24:17.838644+0000","last_unstale":"2024-04-25T10:24:17.846988+0000","last_undegraded":"2024-04-25T10:24:17.846988+0000","last_fullsized":"2024-04-25T10:24:17.846988+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:08.432023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830427+0000","last_change":"2024-04-25T10:24:16.826200+0000","last_active":"2024-04-25T10:24:18.830427+0000","last_peered":"2024-04-25T10:24:18.830427+0000","last_clean":"2024-04-25T10:24:18.830427+0000","last_became_active":"2024-04-25T10:24:16.825913+0000","last_became_peered":"2024-04-25T10:24:16.825913+0000","last_unstale":"2024-04-25T10:24:18.830427+0000","last_undegraded":"2024-04-25T10:24:18.830427+0000","last_fullsized":"2024-04-25T10:24:18.830427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:34.737370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846904+0000","last_change":"2024-04-25T10:24:17.843409+0000","last_active":"2024-04-25T10:24:17.846904+0000","last_peered":"2024-04-25T10:24:17.846904+0000","last_clean":"2024-04-25T10:24:17.846904+0000","last_became_active":"2024-04-25T10:24:17.843020+0000","last_became_peered":"2024-04-25T10:24:17.843020+0000","last_unstale":"2024-04-25T10:24:17.846904+0000","last_undegraded":"2024-04-25T10:24:17.846904+0000","last_fullsized":"2024-04-25T10:24:17.846904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:22.276355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829714+0000","last_change":"2024-04-25T10:24:16.829898+0000","last_active":"2024-04-25T10:24:18.829714+0000","last_peered":"2024-04-25T10:24:18.829714+0000","last_clean":"2024-04-25T10:24:18.829714+0000","last_became_active":"2024-04-25T10:24:16.829634+0000","last_became_peered":"2024-04-25T10:24:16.829634+0000","last_unstale":"2024-04-25T10:24:18.829714+0000","last_undegraded":"2024-04-25T10:24:18.829714+0000","last_fullsized":"2024-04-25T10:24:18.829714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:36.590024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829470+0000","last_change":"2024-04-25T10:24:17.843673+0000","last_active":"2024-04-25T10:24:18.829470+0000","last_peered":"2024-04-25T10:24:18.829470+0000","last_clean":"2024-04-25T10:24:18.829470+0000","last_became_active":"2024-04-25T10:24:17.843408+0000","last_became_peered":"2024-04-25T10:24:17.843408+0000","last_unstale":"2024-04-25T10:24:18.829470+0000","last_undegraded":"2024-04-25T10:24:18.829470+0000","last_fullsized":"2024-04-25T10:24:18.829470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:37.628056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.856059+0000","last_change":"2024-04-25T10:24:16.839407+0000","last_active":"2024-04-25T10:24:17.856059+0000","last_peered":"2024-04-25T10:24:17.856059+0000","last_clean":"2024-04-25T10:24:17.856059+0000","last_became_active":"2024-04-25T10:24:16.839119+0000","last_became_peered":"2024-04-25T10:24:16.839119+0000","last_unstale":"2024-04-25T10:24:17.856059+0000","last_undegraded":"2024-04-25T10:24:17.856059+0000","last_fullsized":"2024-04-25T10:24:17.856059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:47.398559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827755+0000","last_change":"2024-04-25T10:24:17.843580+0000","last_active":"2024-04-25T10:24:18.827755+0000","last_peered":"2024-04-25T10:24:18.827755+0000","last_clean":"2024-04-25T10:24:18.827755+0000","last_became_active":"2024-04-25T10:24:17.843315+0000","last_became_peered":"2024-04-25T10:24:17.843315+0000","last_unstale":"2024-04-25T10:24:18.827755+0000","last_undegraded":"2024-04-25T10:24:18.827755+0000","last_fullsized":"2024-04-25T10:24:18.827755+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:46.295720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827671+0000","last_change":"2024-04-25T10:24:16.834136+0000","last_active":"2024-04-25T10:24:18.827671+0000","last_peered":"2024-04-25T10:24:18.827671+0000","last_clean":"2024-04-25T10:24:18.827671+0000","last_became_active":"2024-04-25T10:24:16.833616+0000","last_became_peered":"2024-04-25T10:24:16.833616+0000","last_unstale":"2024-04-25T10:24:18.827671+0000","last_undegraded":"2024-04-25T10:24:18.827671+0000","last_fullsized":"2024-04-25T10:24:18.827671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:20.196903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844379+0000","last_change":"2024-04-25T10:24:17.842529+0000","last_active":"2024-04-25T10:24:17.844379+0000","last_peered":"2024-04-25T10:24:17.844379+0000","last_clean":"2024-04-25T10:24:17.844379+0000","last_became_active":"2024-04-25T10:24:17.842192+0000","last_became_peered":"2024-04-25T10:24:17.842192+0000","last_unstale":"2024-04-25T10:24:17.844379+0000","last_undegraded":"2024-04-25T10:24:17.844379+0000","last_fullsized":"2024-04-25T10:24:17.844379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.448524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.825746+0000","last_change":"2024-04-25T10:24:16.839002+0000","last_active":"2024-04-25T10:24:17.825746+0000","last_peered":"2024-04-25T10:24:17.825746+0000","last_clean":"2024-04-25T10:24:17.825746+0000","last_became_active":"2024-04-25T10:24:16.838731+0000","last_became_peered":"2024-04-25T10:24:16.838731+0000","last_unstale":"2024-04-25T10:24:17.825746+0000","last_undegraded":"2024-04-25T10:24:17.825746+0000","last_fullsized":"2024-04-25T10:24:17.825746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:08.232629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.840502+0000","last_change":"2024-04-25T10:24:17.837160+0000","last_active":"2024-04-25T10:24:17.840502+0000","last_peered":"2024-04-25T10:24:17.840502+0000","last_clean":"2024-04-25T10:24:17.840502+0000","last_became_active":"2024-04-25T10:24:17.836613+0000","last_became_peered":"2024-04-25T10:24:17.836613+0000","last_unstale":"2024-04-25T10:24:17.840502+0000","last_undegraded":"2024-04-25T10:24:17.840502+0000","last_fullsized":"2024-04-25T10:24:17.840502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:22.276355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.828058+0000","last_change":"2024-04-25T10:24:16.837830+0000","last_active":"2024-04-25T10:24:17.828058+0000","last_peered":"2024-04-25T10:24:17.828058+0000","last_clean":"2024-04-25T10:24:17.828058+0000","last_became_active":"2024-04-25T10:24:16.837559+0000","last_became_peered":"2024-04-25T10:24:16.837559+0000","last_unstale":"2024-04-25T10:24:17.828058+0000","last_undegraded":"2024-04-25T10:24:17.828058+0000","last_fullsized":"2024-04-25T10:24:17.828058+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844355+0000","last_change":"2024-04-25T10:24:17.841454+0000","last_active":"2024-04-25T10:24:17.844355+0000","last_peered":"2024-04-25T10:24:17.844355+0000","last_clean":"2024-04-25T10:24:17.844355+0000","last_became_active":"2024-04-25T10:24:17.840918+0000","last_became_peered":"2024-04-25T10:24:17.840918+0000","last_unstale":"2024-04-25T10:24:17.844355+0000","last_undegraded":"2024-04-25T10:24:17.844355+0000","last_fullsized":"2024-04-25T10:24:17.844355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:35.812396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.825245+0000","last_change":"2024-04-25T10:24:16.823038+0000","last_active":"2024-04-25T10:24:17.825245+0000","last_peered":"2024-04-25T10:24:17.825245+0000","last_clean":"2024-04-25T10:24:17.825245+0000","last_became_active":"2024-04-25T10:24:16.822756+0000","last_became_peered":"2024-04-25T10:24:16.822756+0000","last_unstale":"2024-04-25T10:24:17.825245+0000","last_undegraded":"2024-04-25T10:24:17.825245+0000","last_fullsized":"2024-04-25T10:24:17.825245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:08.232629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846252+0000","last_change":"2024-04-25T10:24:17.844134+0000","last_active":"2024-04-25T10:24:17.846252+0000","last_peered":"2024-04-25T10:24:17.846252+0000","last_clean":"2024-04-25T10:24:17.846252+0000","last_became_active":"2024-04-25T10:24:17.843775+0000","last_became_peered":"2024-04-25T10:24:17.843775+0000","last_unstale":"2024-04-25T10:24:17.846252+0000","last_undegraded":"2024-04-25T10:24:17.846252+0000","last_fullsized":"2024-04-25T10:24:17.846252+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:58.876843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828107+0000","last_change":"2024-04-25T10:24:16.839195+0000","last_active":"2024-04-25T10:24:18.828107+0000","last_peered":"2024-04-25T10:24:18.828107+0000","last_clean":"2024-04-25T10:24:18.828107+0000","last_became_active":"2024-04-25T10:24:16.838251+0000","last_became_peered":"2024-04-25T10:24:16.838251+0000","last_unstale":"2024-04-25T10:24:18.828107+0000","last_undegraded":"2024-04-25T10:24:18.828107+0000","last_fullsized":"2024-04-25T10:24:18.828107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:05.395985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830265+0000","last_change":"2024-04-25T10:24:17.842559+0000","last_active":"2024-04-25T10:24:18.830265+0000","last_peered":"2024-04-25T10:24:18.830265+0000","last_clean":"2024-04-25T10:24:18.830265+0000","last_became_active":"2024-04-25T10:24:17.841787+0000","last_became_peered":"2024-04-25T10:24:17.841787+0000","last_unstale":"2024-04-25T10:24:18.830265+0000","last_undegraded":"2024-04-25T10:24:18.830265+0000","last_fullsized":"2024-04-25T10:24:18.830265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.485882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.854772+0000","last_change":"2024-04-25T10:24:16.832777+0000","last_active":"2024-04-25T10:24:17.854772+0000","last_peered":"2024-04-25T10:24:17.854772+0000","last_clean":"2024-04-25T10:24:17.854772+0000","last_became_active":"2024-04-25T10:24:16.832360+0000","last_became_peered":"2024-04-25T10:24:16.832360+0000","last_unstale":"2024-04-25T10:24:17.854772+0000","last_undegraded":"2024-04-25T10:24:17.854772+0000","last_fullsized":"2024-04-25T10:24:17.854772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:59.312544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.848927+0000","last_change":"2024-04-25T10:24:17.841427+0000","last_active":"2024-04-25T10:24:17.848927+0000","last_peered":"2024-04-25T10:24:17.848927+0000","last_clean":"2024-04-25T10:24:17.848927+0000","last_became_active":"2024-04-25T10:24:17.841148+0000","last_became_peered":"2024-04-25T10:24:17.841148+0000","last_unstale":"2024-04-25T10:24:17.848927+0000","last_undegraded":"2024-04-25T10:24:17.848927+0000","last_fullsized":"2024-04-25T10:24:17.848927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:21.366821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.859164+0000","last_change":"2024-04-25T10:24:16.835769+0000","last_active":"2024-04-25T10:24:17.859164+0000","last_peered":"2024-04-25T10:24:17.859164+0000","last_clean":"2024-04-25T10:24:17.859164+0000","last_became_active":"2024-04-25T10:24:16.835478+0000","last_became_peered":"2024-04-25T10:24:16.835478+0000","last_unstale":"2024-04-25T10:24:17.859164+0000","last_undegraded":"2024-04-25T10:24:17.859164+0000","last_fullsized":"2024-04-25T10:24:17.859164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:14.683268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830009+0000","last_change":"2024-04-25T10:24:17.846547+0000","last_active":"2024-04-25T10:24:18.830009+0000","last_peered":"2024-04-25T10:24:18.830009+0000","last_clean":"2024-04-25T10:24:18.830009+0000","last_became_active":"2024-04-25T10:24:17.846259+0000","last_became_peered":"2024-04-25T10:24:17.846259+0000","last_unstale":"2024-04-25T10:24:18.830009+0000","last_undegraded":"2024-04-25T10:24:18.830009+0000","last_fullsized":"2024-04-25T10:24:18.830009+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:27.970876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829263+0000","last_change":"2024-04-25T10:24:16.825392+0000","last_active":"2024-04-25T10:24:18.829263+0000","last_peered":"2024-04-25T10:24:18.829263+0000","last_clean":"2024-04-25T10:24:18.829263+0000","last_became_active":"2024-04-25T10:24:16.825103+0000","last_became_peered":"2024-04-25T10:24:16.825103+0000","last_unstale":"2024-04-25T10:24:18.829263+0000","last_undegraded":"2024-04-25T10:24:18.829263+0000","last_fullsized":"2024-04-25T10:24:18.829263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:57.729076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.850724+0000","last_change":"2024-04-25T10:24:17.843715+0000","last_active":"2024-04-25T10:24:17.850724+0000","last_peered":"2024-04-25T10:24:17.850724+0000","last_clean":"2024-04-25T10:24:17.850724+0000","last_became_active":"2024-04-25T10:24:17.843418+0000","last_became_peered":"2024-04-25T10:24:17.843418+0000","last_unstale":"2024-04-25T10:24:17.850724+0000","last_undegraded":"2024-04-25T10:24:17.850724+0000","last_fullsized":"2024-04-25T10:24:17.850724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:07.108402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.822677+0000","last_change":"2024-04-25T10:24:16.835544+0000","last_active":"2024-04-25T10:24:17.822677+0000","last_peered":"2024-04-25T10:24:17.822677+0000","last_clean":"2024-04-25T10:24:17.822677+0000","last_became_active":"2024-04-25T10:24:16.835191+0000","last_became_peered":"2024-04-25T10:24:16.835191+0000","last_unstale":"2024-04-25T10:24:17.822677+0000","last_undegraded":"2024-04-25T10:24:17.822677+0000","last_fullsized":"2024-04-25T10:24:17.822677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:40.610067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844848+0000","last_change":"2024-04-25T10:24:17.843357+0000","last_active":"2024-04-25T10:24:17.844848+0000","last_peered":"2024-04-25T10:24:17.844848+0000","last_clean":"2024-04-25T10:24:17.844848+0000","last_became_active":"2024-04-25T10:24:17.842848+0000","last_became_peered":"2024-04-25T10:24:17.842848+0000","last_unstale":"2024-04-25T10:24:17.844848+0000","last_undegraded":"2024-04-25T10:24:17.844848+0000","last_fullsized":"2024-04-25T10:24:17.844848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:22.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827377+0000","last_change":"2024-04-25T10:24:16.833067+0000","last_active":"2024-04-25T10:24:18.827377+0000","last_peered":"2024-04-25T10:24:18.827377+0000","last_clean":"2024-04-25T10:24:18.827377+0000","last_became_active":"2024-04-25T10:24:16.830978+0000","last_became_peered":"2024-04-25T10:24:16.830978+0000","last_unstale":"2024-04-25T10:24:18.827377+0000","last_undegraded":"2024-04-25T10:24:18.827377+0000","last_fullsized":"2024-04-25T10:24:18.827377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:10.479472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828589+0000","last_change":"2024-04-25T10:24:17.839023+0000","last_active":"2024-04-25T10:24:18.828589+0000","last_peered":"2024-04-25T10:24:18.828589+0000","last_clean":"2024-04-25T10:24:18.828589+0000","last_became_active":"2024-04-25T10:24:17.838545+0000","last_became_peered":"2024-04-25T10:24:17.838545+0000","last_unstale":"2024-04-25T10:24:18.828589+0000","last_undegraded":"2024-04-25T10:24:18.828589+0000","last_fullsized":"2024-04-25T10:24:18.828589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.545041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828976+0000","last_change":"2024-04-25T10:24:16.831790+0000","last_active":"2024-04-25T10:24:18.828976+0000","last_peered":"2024-04-25T10:24:18.828976+0000","last_clean":"2024-04-25T10:24:18.828976+0000","last_became_active":"2024-04-25T10:24:16.831478+0000","last_became_peered":"2024-04-25T10:24:16.831478+0000","last_unstale":"2024-04-25T10:24:18.828976+0000","last_undegraded":"2024-04-25T10:24:18.828976+0000","last_fullsized":"2024-04-25T10:24:18.828976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:52.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846617+0000","last_change":"2024-04-25T10:24:17.839467+0000","last_active":"2024-04-25T10:24:17.846617+0000","last_peered":"2024-04-25T10:24:17.846617+0000","last_clean":"2024-04-25T10:24:17.846617+0000","last_became_active":"2024-04-25T10:24:17.839155+0000","last_became_peered":"2024-04-25T10:24:17.839155+0000","last_unstale":"2024-04-25T10:24:17.846617+0000","last_undegraded":"2024-04-25T10:24:17.846617+0000","last_fullsized":"2024-04-25T10:24:17.846617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:27.970876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.855707+0000","last_change":"2024-04-25T10:24:16.838820+0000","last_active":"2024-04-25T10:24:17.855707+0000","last_peered":"2024-04-25T10:24:17.855707+0000","last_clean":"2024-04-25T10:24:17.855707+0000","last_became_active":"2024-04-25T10:24:16.838346+0000","last_became_peered":"2024-04-25T10:24:16.838346+0000","last_unstale":"2024-04-25T10:24:17.855707+0000","last_undegraded":"2024-04-25T10:24:17.855707+0000","last_fullsized":"2024-04-25T10:24:17.855707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:45.952893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828223+0000","last_change":"2024-04-25T10:24:17.839246+0000","last_active":"2024-04-25T10:24:18.828223+0000","last_peered":"2024-04-25T10:24:18.828223+0000","last_clean":"2024-04-25T10:24:18.828223+0000","last_became_active":"2024-04-25T10:24:17.836460+0000","last_became_peered":"2024-04-25T10:24:17.836460+0000","last_unstale":"2024-04-25T10:24:18.828223+0000","last_undegraded":"2024-04-25T10:24:18.828223+0000","last_fullsized":"2024-04-25T10:24:18.828223+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:37.628056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.855966+0000","last_change":"2024-04-25T10:24:16.838997+0000","last_active":"2024-04-25T10:24:17.855966+0000","last_peered":"2024-04-25T10:24:17.855966+0000","last_clean":"2024-04-25T10:24:17.855966+0000","last_became_active":"2024-04-25T10:24:16.838764+0000","last_became_peered":"2024-04-25T10:24:16.838764+0000","last_unstale":"2024-04-25T10:24:17.855966+0000","last_undegraded":"2024-04-25T10:24:17.855966+0000","last_fullsized":"2024-04-25T10:24:17.855966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:39.846183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.840871+0000","last_change":"2024-04-25T10:24:17.837341+0000","last_active":"2024-04-25T10:24:17.840871+0000","last_peered":"2024-04-25T10:24:17.840871+0000","last_clean":"2024-04-25T10:24:17.840871+0000","last_became_active":"2024-04-25T10:24:17.837057+0000","last_became_peered":"2024-04-25T10:24:17.837057+0000","last_unstale":"2024-04-25T10:24:17.840871+0000","last_undegraded":"2024-04-25T10:24:17.840871+0000","last_fullsized":"2024-04-25T10:24:17.840871+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.830004+0000","last_change":"2024-04-25T10:24:16.834313+0000","last_active":"2024-04-25T10:24:17.830004+0000","last_peered":"2024-04-25T10:24:17.830004+0000","last_clean":"2024-04-25T10:24:17.830004+0000","last_became_active":"2024-04-25T10:24:16.834043+0000","last_became_peered":"2024-04-25T10:24:16.834043+0000","last_unstale":"2024-04-25T10:24:17.830004+0000","last_undegraded":"2024-04-25T10:24:17.830004+0000","last_fullsized":"2024-04-25T10:24:17.830004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:29.829805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.843440+0000","last_change":"2024-04-25T10:24:17.841359+0000","last_active":"2024-04-25T10:24:17.843440+0000","last_peered":"2024-04-25T10:24:17.843440+0000","last_clean":"2024-04-25T10:24:17.843440+0000","last_became_active":"2024-04-25T10:24:17.841089+0000","last_became_peered":"2024-04-25T10:24:17.841089+0000","last_unstale":"2024-04-25T10:24:17.843440+0000","last_undegraded":"2024-04-25T10:24:17.843440+0000","last_fullsized":"2024-04-25T10:24:17.843440+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:13.982822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.852787+0000","last_change":"2024-04-25T10:24:16.833962+0000","last_active":"2024-04-25T10:24:17.852787+0000","last_peered":"2024-04-25T10:24:17.852787+0000","last_clean":"2024-04-25T10:24:17.852787+0000","last_became_active":"2024-04-25T10:24:16.833348+0000","last_became_peered":"2024-04-25T10:24:16.833348+0000","last_unstale":"2024-04-25T10:24:17.852787+0000","last_undegraded":"2024-04-25T10:24:17.852787+0000","last_fullsized":"2024-04-25T10:24:17.852787+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:57.738807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846469+0000","last_change":"2024-04-25T10:24:17.839492+0000","last_active":"2024-04-25T10:24:17.846469+0000","last_peered":"2024-04-25T10:24:17.846469+0000","last_clean":"2024-04-25T10:24:17.846469+0000","last_became_active":"2024-04-25T10:24:17.839209+0000","last_became_peered":"2024-04-25T10:24:17.839209+0000","last_unstale":"2024-04-25T10:24:17.846469+0000","last_undegraded":"2024-04-25T10:24:17.846469+0000","last_fullsized":"2024-04-25T10:24:17.846469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:22.179797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.822840+0000","last_change":"2024-04-25T10:24:16.825025+0000","last_active":"2024-04-25T10:24:17.822840+0000","last_peered":"2024-04-25T10:24:17.822840+0000","last_clean":"2024-04-25T10:24:17.822840+0000","last_became_active":"2024-04-25T10:24:16.824727+0000","last_became_peered":"2024-04-25T10:24:16.824727+0000","last_unstale":"2024-04-25T10:24:17.822840+0000","last_undegraded":"2024-04-25T10:24:17.822840+0000","last_fullsized":"2024-04-25T10:24:17.822840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:45.838531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828849+0000","last_change":"2024-04-25T10:24:17.833116+0000","last_active":"2024-04-25T10:24:18.828849+0000","last_peered":"2024-04-25T10:24:18.828849+0000","last_clean":"2024-04-25T10:24:18.828849+0000","last_became_active":"2024-04-25T10:24:17.832837+0000","last_became_peered":"2024-04-25T10:24:17.832837+0000","last_unstale":"2024-04-25T10:24:18.828849+0000","last_undegraded":"2024-04-25T10:24:18.828849+0000","last_fullsized":"2024-04-25T10:24:18.828849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:39.411945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829737+0000","last_change":"2024-04-25T10:24:16.833133+0000","last_active":"2024-04-25T10:24:18.829737+0000","last_peered":"2024-04-25T10:24:18.829737+0000","last_clean":"2024-04-25T10:24:18.829737+0000","last_became_active":"2024-04-25T10:24:16.832832+0000","last_became_peered":"2024-04-25T10:24:16.832832+0000","last_unstale":"2024-04-25T10:24:18.829737+0000","last_undegraded":"2024-04-25T10:24:18.829737+0000","last_fullsized":"2024-04-25T10:24:18.829737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:34.805986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.841273+0000","last_change":"2024-04-25T10:24:17.837357+0000","last_active":"2024-04-25T10:24:17.841273+0000","last_peered":"2024-04-25T10:24:17.841273+0000","last_clean":"2024-04-25T10:24:17.841273+0000","last_became_active":"2024-04-25T10:24:17.837024+0000","last_became_peered":"2024-04-25T10:24:17.837024+0000","last_unstale":"2024-04-25T10:24:17.841273+0000","last_undegraded":"2024-04-25T10:24:17.841273+0000","last_fullsized":"2024-04-25T10:24:17.841273+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.545041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.826946+0000","last_change":"2024-04-25T10:24:16.834378+0000","last_active":"2024-04-25T10:24:17.826946+0000","last_peered":"2024-04-25T10:24:17.826946+0000","last_clean":"2024-04-25T10:24:17.826946+0000","last_became_active":"2024-04-25T10:24:16.833856+0000","last_became_peered":"2024-04-25T10:24:16.833856+0000","last_unstale":"2024-04-25T10:24:17.826946+0000","last_undegraded":"2024-04-25T10:24:17.826946+0000","last_fullsized":"2024-04-25T10:24:17.826946+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:18.493835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833226+0000","last_change":"2024-04-25T10:24:17.824314+0000","last_active":"2024-04-25T10:24:17.833226+0000","last_peered":"2024-04-25T10:24:17.833226+0000","last_clean":"2024-04-25T10:24:17.833226+0000","last_became_active":"2024-04-25T10:24:17.824030+0000","last_became_peered":"2024-04-25T10:24:17.824030+0000","last_unstale":"2024-04-25T10:24:17.833226+0000","last_undegraded":"2024-04-25T10:24:17.833226+0000","last_fullsized":"2024-04-25T10:24:17.833226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:29.735646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.852583+0000","last_change":"2024-04-25T10:24:16.837771+0000","last_active":"2024-04-25T10:24:17.852583+0000","last_peered":"2024-04-25T10:24:17.852583+0000","last_clean":"2024-04-25T10:24:17.852583+0000","last_became_active":"2024-04-25T10:24:16.837466+0000","last_became_peered":"2024-04-25T10:24:16.837466+0000","last_unstale":"2024-04-25T10:24:17.852583+0000","last_undegraded":"2024-04-25T10:24:17.852583+0000","last_fullsized":"2024-04-25T10:24:17.852583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:00.996599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.845797+0000","last_change":"2024-04-25T10:24:17.838722+0000","last_active":"2024-04-25T10:24:17.845797+0000","last_peered":"2024-04-25T10:24:17.845797+0000","last_clean":"2024-04-25T10:24:17.845797+0000","last_became_active":"2024-04-25T10:24:17.838126+0000","last_became_peered":"2024-04-25T10:24:17.838126+0000","last_unstale":"2024-04-25T10:24:17.845797+0000","last_undegraded":"2024-04-25T10:24:17.845797+0000","last_fullsized":"2024-04-25T10:24:17.845797+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:58.758804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.842754+0000","last_change":"2024-04-25T10:24:16.828101+0000","last_active":"2024-04-25T10:24:18.842754+0000","last_peered":"2024-04-25T10:24:18.842754+0000","last_clean":"2024-04-25T10:24:18.842754+0000","last_became_active":"2024-04-25T10:24:16.827618+0000","last_became_peered":"2024-04-25T10:24:16.827618+0000","last_unstale":"2024-04-25T10:24:18.842754+0000","last_undegraded":"2024-04-25T10:24:18.842754+0000","last_fullsized":"2024-04-25T10:24:18.842754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:21.823062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833404+0000","last_change":"2024-04-25T10:24:17.826359+0000","last_active":"2024-04-25T10:24:17.833404+0000","last_peered":"2024-04-25T10:24:17.833404+0000","last_clean":"2024-04-25T10:24:17.833404+0000","last_became_active":"2024-04-25T10:24:17.826039+0000","last_became_peered":"2024-04-25T10:24:17.826039+0000","last_unstale":"2024-04-25T10:24:17.833404+0000","last_undegraded":"2024-04-25T10:24:17.833404+0000","last_fullsized":"2024-04-25T10:24:17.833404+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.751829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.827168+0000","last_change":"2024-04-25T10:24:16.834986+0000","last_active":"2024-04-25T10:24:17.827168+0000","last_peered":"2024-04-25T10:24:17.827168+0000","last_clean":"2024-04-25T10:24:17.827168+0000","last_became_active":"2024-04-25T10:24:16.834284+0000","last_became_peered":"2024-04-25T10:24:16.834284+0000","last_unstale":"2024-04-25T10:24:17.827168+0000","last_undegraded":"2024-04-25T10:24:17.827168+0000","last_fullsized":"2024-04-25T10:24:17.827168+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:06.947896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827991+0000","last_change":"2024-04-25T10:24:17.844975+0000","last_active":"2024-04-25T10:24:18.827991+0000","last_peered":"2024-04-25T10:24:18.827991+0000","last_clean":"2024-04-25T10:24:18.827991+0000","last_became_active":"2024-04-25T10:24:17.844661+0000","last_became_peered":"2024-04-25T10:24:17.844661+0000","last_unstale":"2024-04-25T10:24:18.827991+0000","last_undegraded":"2024-04-25T10:24:18.827991+0000","last_fullsized":"2024-04-25T10:24:18.827991+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.418507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827641+0000","last_change":"2024-04-25T10:24:16.836994+0000","last_active":"2024-04-25T10:24:18.827641+0000","last_peered":"2024-04-25T10:24:18.827641+0000","last_clean":"2024-04-25T10:24:18.827641+0000","last_became_active":"2024-04-25T10:24:16.836698+0000","last_became_peered":"2024-04-25T10:24:16.836698+0000","last_unstale":"2024-04-25T10:24:18.827641+0000","last_undegraded":"2024-04-25T10:24:18.827641+0000","last_fullsized":"2024-04-25T10:24:18.827641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:24.134500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.837857+0000","last_change":"2024-04-25T10:24:17.834703+0000","last_active":"2024-04-25T10:24:17.837857+0000","last_peered":"2024-04-25T10:24:17.837857+0000","last_clean":"2024-04-25T10:24:17.837857+0000","last_became_active":"2024-04-25T10:24:17.834398+0000","last_became_peered":"2024-04-25T10:24:17.834398+0000","last_unstale":"2024-04-25T10:24:17.837857+0000","last_undegraded":"2024-04-25T10:24:17.837857+0000","last_fullsized":"2024-04-25T10:24:17.837857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:38.685064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.822898+0000","last_change":"2024-04-25T10:24:16.829669+0000","last_active":"2024-04-25T10:24:17.822898+0000","last_peered":"2024-04-25T10:24:17.822898+0000","last_clean":"2024-04-25T10:24:17.822898+0000","last_became_active":"2024-04-25T10:24:16.829361+0000","last_became_peered":"2024-04-25T10:24:16.829361+0000","last_unstale":"2024-04-25T10:24:17.822898+0000","last_undegraded":"2024-04-25T10:24:17.822898+0000","last_fullsized":"2024-04-25T10:24:17.822898+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:19.008841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830195+0000","last_change":"2024-04-25T10:24:17.846629+0000","last_active":"2024-04-25T10:24:18.830195+0000","last_peered":"2024-04-25T10:24:18.830195+0000","last_clean":"2024-04-25T10:24:18.830195+0000","last_became_active":"2024-04-25T10:24:17.846209+0000","last_became_peered":"2024-04-25T10:24:17.846209+0000","last_unstale":"2024-04-25T10:24:18.830195+0000","last_undegraded":"2024-04-25T10:24:18.830195+0000","last_fullsized":"2024-04-25T10:24:18.830195+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.507016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827781+0000","last_change":"2024-04-25T10:24:16.833901+0000","last_active":"2024-04-25T10:24:18.827781+0000","last_peered":"2024-04-25T10:24:18.827781+0000","last_clean":"2024-04-25T10:24:18.827781+0000","last_became_active":"2024-04-25T10:24:16.833168+0000","last_became_peered":"2024-04-25T10:24:16.833168+0000","last_unstale":"2024-04-25T10:24:18.827781+0000","last_undegraded":"2024-04-25T10:24:18.827781+0000","last_fullsized":"2024-04-25T10:24:18.827781+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:52.420761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.837855+0000","last_change":"2024-04-25T10:24:17.832713+0000","last_active":"2024-04-25T10:24:17.837855+0000","last_peered":"2024-04-25T10:24:17.837855+0000","last_clean":"2024-04-25T10:24:17.837855+0000","last_became_active":"2024-04-25T10:24:17.832419+0000","last_became_peered":"2024-04-25T10:24:17.832419+0000","last_unstale":"2024-04-25T10:24:17.837855+0000","last_undegraded":"2024-04-25T10:24:17.837855+0000","last_fullsized":"2024-04-25T10:24:17.837855+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:30.836215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.821240+0000","last_change":"2024-04-25T10:24:16.833871+0000","last_active":"2024-04-25T10:24:17.821240+0000","last_peered":"2024-04-25T10:24:17.821240+0000","last_clean":"2024-04-25T10:24:17.821240+0000","last_became_active":"2024-04-25T10:24:16.833579+0000","last_became_peered":"2024-04-25T10:24:16.833579+0000","last_unstale":"2024-04-25T10:24:17.821240+0000","last_undegraded":"2024-04-25T10:24:17.821240+0000","last_fullsized":"2024-04-25T10:24:17.821240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:25.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.834819+0000","last_change":"2024-04-25T10:24:17.828443+0000","last_active":"2024-04-25T10:24:17.834819+0000","last_peered":"2024-04-25T10:24:17.834819+0000","last_clean":"2024-04-25T10:24:17.834819+0000","last_became_active":"2024-04-25T10:24:17.828171+0000","last_became_peered":"2024-04-25T10:24:17.828171+0000","last_unstale":"2024-04-25T10:24:17.834819+0000","last_undegraded":"2024-04-25T10:24:17.834819+0000","last_fullsized":"2024-04-25T10:24:17.834819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:54.380310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.827934+0000","last_change":"2024-04-25T10:24:16.831898+0000","last_active":"2024-04-25T10:24:17.827934+0000","last_peered":"2024-04-25T10:24:17.827934+0000","last_clean":"2024-04-25T10:24:17.827934+0000","last_became_active":"2024-04-25T10:24:16.831530+0000","last_became_peered":"2024-04-25T10:24:16.831530+0000","last_unstale":"2024-04-25T10:24:17.827934+0000","last_undegraded":"2024-04-25T10:24:17.827934+0000","last_fullsized":"2024-04-25T10:24:17.827934+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:14.013649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830258+0000","last_change":"2024-04-25T10:24:17.843609+0000","last_active":"2024-04-25T10:24:18.830258+0000","last_peered":"2024-04-25T10:24:18.830258+0000","last_clean":"2024-04-25T10:24:18.830258+0000","last_became_active":"2024-04-25T10:24:17.843102+0000","last_became_peered":"2024-04-25T10:24:17.843102+0000","last_unstale":"2024-04-25T10:24:18.830258+0000","last_undegraded":"2024-04-25T10:24:18.830258+0000","last_fullsized":"2024-04-25T10:24:18.830258+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:25.140910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.854198+0000","last_change":"2024-04-25T10:24:16.836725+0000","last_active":"2024-04-25T10:24:17.854198+0000","last_peered":"2024-04-25T10:24:17.854198+0000","last_clean":"2024-04-25T10:24:17.854198+0000","last_became_active":"2024-04-25T10:24:16.831837+0000","last_became_peered":"2024-04-25T10:24:16.831837+0000","last_unstale":"2024-04-25T10:24:17.854198+0000","last_undegraded":"2024-04-25T10:24:17.854198+0000","last_fullsized":"2024-04-25T10:24:17.854198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:08.878687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":302,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.821710+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:24:17.821710+0000","last_peered":"2024-04-25T10:24:17.821710+0000","last_clean":"2024-04-25T10:24:17.821710+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:24:17.821710+0000","last_undegraded":"2024-04-25T10:24:17.821710+0000","last_fullsized":"2024-04-25T10:24:17.821710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509521,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46164,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325676,"statfs":{"total":96636764160,"available":96589492224,"internally_reserved":0,"allocated":1114112,"data_stored":477306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46164,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325676,"statfs":{"total":96636764160,"available":96589492224,"internally_reserved":0,"allocated":1118208,"data_stored":477908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50912,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320928,"statfs":{"total":96636764160,"available":96584630272,"internally_reserved":0,"allocated":1720320,"data_stored":1069250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509521,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54488,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317352,"statfs":{"total":96636764160,"available":96580968448,"internally_reserved":0,"allocated":1122304,"data_stored":477910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58640,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313200,"statfs":{"total":96636764160,"available":96576716800,"internally_reserved":0,"allocated":1110016,"data_stored":477236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509522,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50896,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320944,"statfs":{"total":96636764160,"available":96584646656,"internally_reserved":0,"allocated":1708032,"data_stored":1067694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509522,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":478870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1097728,"data_stored":470461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:24:21.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:24:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.125+0000 7fa11c000640 1 Processor -- start 2024-04-25T10:24:22.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.125+0000 7fa11c000640 1 -- start start 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11c000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1140cf930 0x7fa1140cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11c000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1140ae020 0x7fa1140ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11c000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1140d0250 con 0x7fa114070c20 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11c000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11415bb90 con 0x7fa1140ae020 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11c000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1140ad840 con 0x7fa1140cf930 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1140ae020 0x7fa1140ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1140ae020 0x7fa1140ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43362/0 (socket says 172.21.15.45:43362) 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 -- 172.21.15.45:0/1216787614 learned_addr learned my addr 172.21.15.45:0/1216787614 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa118c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa114070c20 0x7fa11415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58490/0 (socket says 172.21.15.45:58490) 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa119600640 1 --2- 172.21.15.45:0/1216787614 >> v2:172.21.15.117:3300/0 conn(0x7fa1140cf930 0x7fa1140cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 -- 172.21.15.45:0/1216787614 >> v2:172.21.15.117:3300/0 conn(0x7fa1140cf930 msgr2=0x7fa1140cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 --2- 172.21.15.45:0/1216787614 >> v2:172.21.15.117:3300/0 conn(0x7fa1140cf930 0x7fa1140cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 -- 172.21.15.45:0/1216787614 >> v1:172.21.15.45:6789/0 conn(0x7fa114070c20 legacy=0x7fa11415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.129+0000 7fa11a000640 1 -- 172.21.15.45:0/1216787614 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11415e9f0 con 0x7fa1140ae020 2024-04-25T10:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11a000640 1 --2- 172.21.15.45:0/1216787614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1140ae020 0x7fa1140ae4b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa1140aec50 tx=0x7fa10c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72138ef67bd13403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa113e00640 1 -- 172.21.15.45:0/1216787614 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0a5880 con 0x7fa1140ae020 2024-04-25T10:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa113e00640 1 -- 172.21.15.45:0/1216787614 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0a5a20 con 0x7fa1140ae020 2024-04-25T10:24:22.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa113e00640 1 -- 172.21.15.45:0/1216787614 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0a5df0 con 0x7fa1140ae020 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 -- 172.21.15.45:0/1216787614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1140ae020 msgr2=0x7fa1140ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 --2- 172.21.15.45:0/1216787614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1140ae020 0x7fa1140ae4b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa1140aec50 tx=0x7fa10c01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 -- 172.21.15.45:0/1216787614 shutdown_connections 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 --2- 172.21.15.45:0/1216787614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1140ae020 0x7fa1140ae4b0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 --2- 172.21.15.45:0/1216787614 >> v2:172.21.15.117:3300/0 conn(0x7fa1140cf930 0x7fa1140cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 -- 172.21.15.45:0/1216787614 >> 172.21.15.45:0/1216787614 conn(0x7fa114066b70 msgr2=0x7fa114066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 -- 172.21.15.45:0/1216787614 shutdown_connections 2024-04-25T10:24:22.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 -- 172.21.15.45:0/1216787614 wait complete. 2024-04-25T10:24:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 Processor -- start 2024-04-25T10:24:22.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 -- start start 2024-04-25T10:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.133+0000 7fa11c000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 0x7fa114153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1140ae020 0x7fa114156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1140cf930 0x7fa1141f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11416a8e0 con 0x7fa114070c20 2024-04-25T10:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11406e130 con 0x7fa1140ae020 2024-04-25T10:24:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11416aa60 con 0x7fa1140cf930 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 0x7fa114153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 0x7fa114153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43376/0 (socket says 172.21.15.45:43376) 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 -- 172.21.15.45:0/781233946 learned_addr learned my addr 172.21.15.45:0/781233946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 -- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1140cf930 msgr2=0x7fa1141f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1140cf930 0x7fa1141f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 -- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1140ae020 msgr2=0x7fa114156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa118c00640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1140ae020 0x7fa114156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1140ae020 0x7fa114156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 -- 172.21.15.45:0/781233946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114155b90 con 0x7fa114070c20 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa118c00640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1140ae020 0x7fa114156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa119600640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 0x7fa114153e10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa10800d620 tx=0x7fa10800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800b620 con 0x7fa114070c20 2024-04-25T10:24:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa108019070 con 0x7fa114070c20 2024-04-25T10:24:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800b9e0 con 0x7fa114070c20 2024-04-25T10:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1141fab60 con 0x7fa114070c20 2024-04-25T10:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1141561c0 con 0x7fa114070c20 2024-04-25T10:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11404ef30 con 0x7fa114070c20 2024-04-25T10:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.137+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa10800bb80 con 0x7fa114070c20 2024-04-25T10:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.141+0000 7fa111600640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0f4066f60 0x7fa0f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.141+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa108085490 con 0x7fa114070c20 2024-04-25T10:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.141+0000 7fa118c00640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0f4066f60 0x7fa0f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.141+0000 7fa118c00640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0f4066f60 0x7fa0f4069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa100007fd0 tx=0x7fa100009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.145+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa108056400 con 0x7fa114070c20 2024-04-25T10:24:22.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.301+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa114066f50 con 0x7fa0f4066f60 2024-04-25T10:24:22.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.305+0000 7fa111600640 1 -- 172.21.15.45:0/781233946 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252042 (secure 0 0 0) 0x7fa114066f50 con 0x7fa0f4066f60 2024-04-25T10:24:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:22.311 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0f4066f60 msgr2=0x7fa0f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0f4066f60 0x7fa0f4069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa100007fd0 tx=0x7fa100009040 comp rx=0 tx=0).stop 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 msgr2=0x7fa114153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 0x7fa114153e10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa10800d620 tx=0x7fa10800dae0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 shutdown_connections 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0f4066f60 0x7fa0f4069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1140cf930 0x7fa1141f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1140ae020 0x7fa114156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 --2- 172.21.15.45:0/781233946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa114070c20 0x7fa114153e10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 >> 172.21.15.45:0/781233946 conn(0x7fa114066b70 msgr2=0x7fa1140b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:22.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 shutdown_connections 2024-04-25T10:24:22.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.309+0000 7fa11c000640 1 -- 172.21.15.45:0/781233946 wait complete. 2024-04-25T10:24:22.330 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-04-25T10:24:20.962314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389848,"kb_used_data":9880,"kb_used_omap":12,"kb_used_meta":379827,"kb_avail":754584872,"statfs":{"total":773094113280,"available":772694908928,"internally_reserved":0,"allocated":10117120,"data_stored":4996635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":388943408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157751"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.836258+0000","last_change":"2024-04-25T10:24:17.832232+0000","last_active":"2024-04-25T10:24:17.836258+0000","last_peered":"2024-04-25T10:24:17.836258+0000","last_clean":"2024-04-25T10:24:17.836258+0000","last_became_active":"2024-04-25T10:24:17.831574+0000","last_became_peered":"2024-04-25T10:24:17.831574+0000","last_unstale":"2024-04-25T10:24:17.836258+0000","last_undegraded":"2024-04-25T10:24:17.836258+0000","last_fullsized":"2024-04-25T10:24:17.836258+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:02.003009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.835743+0000","last_change":"2024-04-25T10:24:17.833876+0000","last_active":"2024-04-25T10:24:17.835743+0000","last_peered":"2024-04-25T10:24:17.835743+0000","last_clean":"2024-04-25T10:24:17.835743+0000","last_became_active":"2024-04-25T10:24:17.833547+0000","last_became_peered":"2024-04-25T10:24:17.833547+0000","last_unstale":"2024-04-25T10:24:17.835743+0000","last_undegraded":"2024-04-25T10:24:17.835743+0000","last_fullsized":"2024-04-25T10:24:17.835743+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:33.654465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827910+0000","last_change":"2024-04-25T10:24:17.836719+0000","last_active":"2024-04-25T10:24:18.827910+0000","last_peered":"2024-04-25T10:24:18.827910+0000","last_clean":"2024-04-25T10:24:18.827910+0000","last_became_active":"2024-04-25T10:24:17.836068+0000","last_became_peered":"2024-04-25T10:24:17.836068+0000","last_unstale":"2024-04-25T10:24:18.827910+0000","last_undegraded":"2024-04-25T10:24:18.827910+0000","last_fullsized":"2024-04-25T10:24:18.827910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:39.411945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829699+0000","last_change":"2024-04-25T10:24:17.835190+0000","last_active":"2024-04-25T10:24:18.829699+0000","last_peered":"2024-04-25T10:24:18.829699+0000","last_clean":"2024-04-25T10:24:18.829699+0000","last_became_active":"2024-04-25T10:24:17.834919+0000","last_became_peered":"2024-04-25T10:24:17.834919+0000","last_unstale":"2024-04-25T10:24:18.829699+0000","last_undegraded":"2024-04-25T10:24:18.829699+0000","last_fullsized":"2024-04-25T10:24:18.829699+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:46.366686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.845253+0000","last_change":"2024-04-25T10:24:17.839206+0000","last_active":"2024-04-25T10:24:17.845253+0000","last_peered":"2024-04-25T10:24:17.845253+0000","last_clean":"2024-04-25T10:24:17.845253+0000","last_became_active":"2024-04-25T10:24:17.838936+0000","last_became_peered":"2024-04-25T10:24:17.838936+0000","last_unstale":"2024-04-25T10:24:17.845253+0000","last_undegraded":"2024-04-25T10:24:17.845253+0000","last_fullsized":"2024-04-25T10:24:17.845253+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:11.614791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827572+0000","last_change":"2024-04-25T10:24:17.831573+0000","last_active":"2024-04-25T10:24:18.827572+0000","last_peered":"2024-04-25T10:24:18.827572+0000","last_clean":"2024-04-25T10:24:18.827572+0000","last_became_active":"2024-04-25T10:24:17.831251+0000","last_became_peered":"2024-04-25T10:24:17.831251+0000","last_unstale":"2024-04-25T10:24:18.827572+0000","last_undegraded":"2024-04-25T10:24:18.827572+0000","last_fullsized":"2024-04-25T10:24:18.827572+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:13.268129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838808+0000","last_change":"2024-04-25T10:24:17.837356+0000","last_active":"2024-04-25T10:24:17.838808+0000","last_peered":"2024-04-25T10:24:17.838808+0000","last_clean":"2024-04-25T10:24:17.838808+0000","last_became_active":"2024-04-25T10:24:17.837023+0000","last_became_peered":"2024-04-25T10:24:17.837023+0000","last_unstale":"2024-04-25T10:24:17.838808+0000","last_undegraded":"2024-04-25T10:24:17.838808+0000","last_fullsized":"2024-04-25T10:24:17.838808+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:03.490041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.853224+0000","last_change":"2024-04-25T10:24:17.849684+0000","last_active":"2024-04-25T10:24:17.853224+0000","last_peered":"2024-04-25T10:24:17.853224+0000","last_clean":"2024-04-25T10:24:17.853224+0000","last_became_active":"2024-04-25T10:24:17.846322+0000","last_became_peered":"2024-04-25T10:24:17.846322+0000","last_unstale":"2024-04-25T10:24:17.853224+0000","last_undegraded":"2024-04-25T10:24:17.853224+0000","last_fullsized":"2024-04-25T10:24:17.853224+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:25.140910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.836236+0000","last_change":"2024-04-25T10:24:17.826600+0000","last_active":"2024-04-25T10:24:17.836236+0000","last_peered":"2024-04-25T10:24:17.836236+0000","last_clean":"2024-04-25T10:24:17.836236+0000","last_became_active":"2024-04-25T10:24:17.826329+0000","last_became_peered":"2024-04-25T10:24:17.826329+0000","last_unstale":"2024-04-25T10:24:17.836236+0000","last_undegraded":"2024-04-25T10:24:17.836236+0000","last_fullsized":"2024-04-25T10:24:17.836236+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:58.745217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828011+0000","last_change":"2024-04-25T10:24:17.830454+0000","last_active":"2024-04-25T10:24:18.828011+0000","last_peered":"2024-04-25T10:24:18.828011+0000","last_clean":"2024-04-25T10:24:18.828011+0000","last_became_active":"2024-04-25T10:24:17.830117+0000","last_became_peered":"2024-04-25T10:24:17.830117+0000","last_unstale":"2024-04-25T10:24:18.828011+0000","last_undegraded":"2024-04-25T10:24:18.828011+0000","last_fullsized":"2024-04-25T10:24:18.828011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830474+0000","last_change":"2024-04-25T10:24:17.846808+0000","last_active":"2024-04-25T10:24:18.830474+0000","last_peered":"2024-04-25T10:24:18.830474+0000","last_clean":"2024-04-25T10:24:18.830474+0000","last_became_active":"2024-04-25T10:24:17.846484+0000","last_became_peered":"2024-04-25T10:24:17.846484+0000","last_unstale":"2024-04-25T10:24:18.830474+0000","last_undegraded":"2024-04-25T10:24:18.830474+0000","last_fullsized":"2024-04-25T10:24:18.830474+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:53.427172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.847011+0000","last_change":"2024-04-25T10:24:17.841833+0000","last_active":"2024-04-25T10:24:17.847011+0000","last_peered":"2024-04-25T10:24:17.847011+0000","last_clean":"2024-04-25T10:24:17.847011+0000","last_became_active":"2024-04-25T10:24:17.841470+0000","last_became_peered":"2024-04-25T10:24:17.841470+0000","last_unstale":"2024-04-25T10:24:17.847011+0000","last_undegraded":"2024-04-25T10:24:17.847011+0000","last_fullsized":"2024-04-25T10:24:17.847011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.507016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838383+0000","last_change":"2024-04-25T10:24:17.832792+0000","last_active":"2024-04-25T10:24:17.838383+0000","last_peered":"2024-04-25T10:24:17.838383+0000","last_clean":"2024-04-25T10:24:17.838383+0000","last_became_active":"2024-04-25T10:24:17.832489+0000","last_became_peered":"2024-04-25T10:24:17.832489+0000","last_unstale":"2024-04-25T10:24:17.838383+0000","last_undegraded":"2024-04-25T10:24:17.838383+0000","last_fullsized":"2024-04-25T10:24:17.838383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:52.688025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838949+0000","last_change":"2024-04-25T10:24:17.835599+0000","last_active":"2024-04-25T10:24:17.838949+0000","last_peered":"2024-04-25T10:24:17.838949+0000","last_clean":"2024-04-25T10:24:17.838949+0000","last_became_active":"2024-04-25T10:24:17.835068+0000","last_became_peered":"2024-04-25T10:24:17.835068+0000","last_unstale":"2024-04-25T10:24:17.838949+0000","last_undegraded":"2024-04-25T10:24:17.838949+0000","last_fullsized":"2024-04-25T10:24:17.838949+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:38.685064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.835119+0000","last_change":"2024-04-25T10:24:17.829392+0000","last_active":"2024-04-25T10:24:17.835119+0000","last_peered":"2024-04-25T10:24:17.835119+0000","last_clean":"2024-04-25T10:24:17.835119+0000","last_became_active":"2024-04-25T10:24:17.828557+0000","last_became_peered":"2024-04-25T10:24:17.828557+0000","last_unstale":"2024-04-25T10:24:17.835119+0000","last_undegraded":"2024-04-25T10:24:17.835119+0000","last_fullsized":"2024-04-25T10:24:17.835119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:54.913191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827452+0000","last_change":"2024-04-25T10:24:17.838925+0000","last_active":"2024-04-25T10:24:18.827452+0000","last_peered":"2024-04-25T10:24:18.827452+0000","last_clean":"2024-04-25T10:24:18.827452+0000","last_became_active":"2024-04-25T10:24:17.836659+0000","last_became_peered":"2024-04-25T10:24:17.836659+0000","last_unstale":"2024-04-25T10:24:18.827452+0000","last_undegraded":"2024-04-25T10:24:18.827452+0000","last_fullsized":"2024-04-25T10:24:18.827452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:17.675318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.847077+0000","last_change":"2024-04-25T10:24:17.839641+0000","last_active":"2024-04-25T10:24:17.847077+0000","last_peered":"2024-04-25T10:24:17.847077+0000","last_clean":"2024-04-25T10:24:17.847077+0000","last_became_active":"2024-04-25T10:24:17.839371+0000","last_became_peered":"2024-04-25T10:24:17.839371+0000","last_unstale":"2024-04-25T10:24:17.847077+0000","last_undegraded":"2024-04-25T10:24:17.847077+0000","last_fullsized":"2024-04-25T10:24:17.847077+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:32.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830185+0000","last_change":"2024-04-25T10:24:17.840179+0000","last_active":"2024-04-25T10:24:18.830185+0000","last_peered":"2024-04-25T10:24:18.830185+0000","last_clean":"2024-04-25T10:24:18.830185+0000","last_became_active":"2024-04-25T10:24:17.839444+0000","last_became_peered":"2024-04-25T10:24:17.839444+0000","last_unstale":"2024-04-25T10:24:18.830185+0000","last_undegraded":"2024-04-25T10:24:18.830185+0000","last_fullsized":"2024-04-25T10:24:18.830185+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:32.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844129+0000","last_change":"2024-04-25T10:24:17.840824+0000","last_active":"2024-04-25T10:24:17.844129+0000","last_peered":"2024-04-25T10:24:17.844129+0000","last_clean":"2024-04-25T10:24:17.844129+0000","last_became_active":"2024-04-25T10:24:17.839970+0000","last_became_peered":"2024-04-25T10:24:17.839970+0000","last_unstale":"2024-04-25T10:24:17.844129+0000","last_undegraded":"2024-04-25T10:24:17.844129+0000","last_fullsized":"2024-04-25T10:24:17.844129+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:58.876843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844262+0000","last_change":"2024-04-25T10:24:17.839899+0000","last_active":"2024-04-25T10:24:17.844262+0000","last_peered":"2024-04-25T10:24:17.844262+0000","last_clean":"2024-04-25T10:24:17.844262+0000","last_became_active":"2024-04-25T10:24:17.839506+0000","last_became_peered":"2024-04-25T10:24:17.839506+0000","last_unstale":"2024-04-25T10:24:17.844262+0000","last_undegraded":"2024-04-25T10:24:17.844262+0000","last_fullsized":"2024-04-25T10:24:17.844262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:07.108402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829056+0000","last_change":"2024-04-25T10:24:17.825028+0000","last_active":"2024-04-25T10:24:18.829056+0000","last_peered":"2024-04-25T10:24:18.829056+0000","last_clean":"2024-04-25T10:24:18.829056+0000","last_became_active":"2024-04-25T10:24:17.824766+0000","last_became_peered":"2024-04-25T10:24:17.824766+0000","last_unstale":"2024-04-25T10:24:18.829056+0000","last_undegraded":"2024-04-25T10:24:18.829056+0000","last_fullsized":"2024-04-25T10:24:18.829056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.418507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.838971+0000","last_change":"2024-04-25T10:24:17.836259+0000","last_active":"2024-04-25T10:24:17.838971+0000","last_peered":"2024-04-25T10:24:17.838971+0000","last_clean":"2024-04-25T10:24:17.838971+0000","last_became_active":"2024-04-25T10:24:17.835970+0000","last_became_peered":"2024-04-25T10:24:17.835970+0000","last_unstale":"2024-04-25T10:24:17.838971+0000","last_undegraded":"2024-04-25T10:24:17.838971+0000","last_fullsized":"2024-04-25T10:24:17.838971+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.448524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846657+0000","last_change":"2024-04-25T10:24:17.841926+0000","last_active":"2024-04-25T10:24:17.846657+0000","last_peered":"2024-04-25T10:24:17.846657+0000","last_clean":"2024-04-25T10:24:17.846657+0000","last_became_active":"2024-04-25T10:24:17.841520+0000","last_became_peered":"2024-04-25T10:24:17.841520+0000","last_unstale":"2024-04-25T10:24:17.846657+0000","last_undegraded":"2024-04-25T10:24:17.846657+0000","last_fullsized":"2024-04-25T10:24:17.846657+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.485882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828860+0000","last_change":"2024-04-25T10:24:17.839112+0000","last_active":"2024-04-25T10:24:18.828860+0000","last_peered":"2024-04-25T10:24:18.828860+0000","last_clean":"2024-04-25T10:24:18.828860+0000","last_became_active":"2024-04-25T10:24:17.838373+0000","last_became_peered":"2024-04-25T10:24:17.838373+0000","last_unstale":"2024-04-25T10:24:18.828860+0000","last_undegraded":"2024-04-25T10:24:18.828860+0000","last_fullsized":"2024-04-25T10:24:18.828860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:08.432023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830376+0000","last_change":"2024-04-25T10:24:17.849004+0000","last_active":"2024-04-25T10:24:18.830376+0000","last_peered":"2024-04-25T10:24:18.830376+0000","last_clean":"2024-04-25T10:24:18.830376+0000","last_became_active":"2024-04-25T10:24:17.848680+0000","last_became_peered":"2024-04-25T10:24:17.848680+0000","last_unstale":"2024-04-25T10:24:18.830376+0000","last_undegraded":"2024-04-25T10:24:18.830376+0000","last_fullsized":"2024-04-25T10:24:18.830376+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:21.203313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833574+0000","last_change":"2024-04-25T10:24:17.830053+0000","last_active":"2024-04-25T10:24:17.833574+0000","last_peered":"2024-04-25T10:24:17.833574+0000","last_clean":"2024-04-25T10:24:17.833574+0000","last_became_active":"2024-04-25T10:24:17.829191+0000","last_became_peered":"2024-04-25T10:24:17.829191+0000","last_unstale":"2024-04-25T10:24:17.833574+0000","last_undegraded":"2024-04-25T10:24:17.833574+0000","last_fullsized":"2024-04-25T10:24:17.833574+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:15.150590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833536+0000","last_change":"2024-04-25T10:24:17.826289+0000","last_active":"2024-04-25T10:24:17.833536+0000","last_peered":"2024-04-25T10:24:17.833536+0000","last_clean":"2024-04-25T10:24:17.833536+0000","last_became_active":"2024-04-25T10:24:17.825209+0000","last_became_peered":"2024-04-25T10:24:17.825209+0000","last_unstale":"2024-04-25T10:24:17.833536+0000","last_undegraded":"2024-04-25T10:24:17.833536+0000","last_fullsized":"2024-04-25T10:24:17.833536+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:51.816635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.834476+0000","last_change":"2024-04-25T10:24:17.825774+0000","last_active":"2024-04-25T10:24:17.834476+0000","last_peered":"2024-04-25T10:24:17.834476+0000","last_clean":"2024-04-25T10:24:17.834476+0000","last_became_active":"2024-04-25T10:24:17.825479+0000","last_became_peered":"2024-04-25T10:24:17.825479+0000","last_unstale":"2024-04-25T10:24:17.834476+0000","last_undegraded":"2024-04-25T10:24:17.834476+0000","last_fullsized":"2024-04-25T10:24:17.834476+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:19.500246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829213+0000","last_change":"2024-04-25T10:24:17.824823+0000","last_active":"2024-04-25T10:24:18.829213+0000","last_peered":"2024-04-25T10:24:18.829213+0000","last_clean":"2024-04-25T10:24:18.829213+0000","last_became_active":"2024-04-25T10:24:17.824534+0000","last_became_peered":"2024-04-25T10:24:17.824534+0000","last_unstale":"2024-04-25T10:24:18.829213+0000","last_undegraded":"2024-04-25T10:24:18.829213+0000","last_fullsized":"2024-04-25T10:24:18.829213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.181450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844166+0000","last_change":"2024-04-25T10:24:17.840826+0000","last_active":"2024-04-25T10:24:17.844166+0000","last_peered":"2024-04-25T10:24:17.844166+0000","last_clean":"2024-04-25T10:24:17.844166+0000","last_became_active":"2024-04-25T10:24:17.840550+0000","last_became_peered":"2024-04-25T10:24:17.840550+0000","last_unstale":"2024-04-25T10:24:17.844166+0000","last_undegraded":"2024-04-25T10:24:17.844166+0000","last_fullsized":"2024-04-25T10:24:17.844166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:53.317387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827395+0000","last_change":"2024-04-25T10:24:17.826375+0000","last_active":"2024-04-25T10:24:18.827395+0000","last_peered":"2024-04-25T10:24:18.827395+0000","last_clean":"2024-04-25T10:24:18.827395+0000","last_became_active":"2024-04-25T10:24:17.826112+0000","last_became_peered":"2024-04-25T10:24:17.826112+0000","last_unstale":"2024-04-25T10:24:18.827395+0000","last_undegraded":"2024-04-25T10:24:18.827395+0000","last_fullsized":"2024-04-25T10:24:18.827395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.218872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827472+0000","last_change":"2024-04-25T10:24:17.828435+0000","last_active":"2024-04-25T10:24:18.827472+0000","last_peered":"2024-04-25T10:24:18.827472+0000","last_clean":"2024-04-25T10:24:18.827472+0000","last_became_active":"2024-04-25T10:24:17.828154+0000","last_became_peered":"2024-04-25T10:24:17.828154+0000","last_unstale":"2024-04-25T10:24:18.827472+0000","last_undegraded":"2024-04-25T10:24:18.827472+0000","last_fullsized":"2024-04-25T10:24:18.827472+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:07.810903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846589+0000","last_change":"2024-04-25T10:24:17.835706+0000","last_active":"2024-04-25T10:24:17.846589+0000","last_peered":"2024-04-25T10:24:17.846589+0000","last_clean":"2024-04-25T10:24:17.846589+0000","last_became_active":"2024-04-25T10:24:17.835428+0000","last_became_peered":"2024-04-25T10:24:17.835428+0000","last_unstale":"2024-04-25T10:24:17.846589+0000","last_undegraded":"2024-04-25T10:24:17.846589+0000","last_fullsized":"2024-04-25T10:24:17.846589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:58.752127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.855921+0000","last_change":"2024-04-25T10:24:16.831295+0000","last_active":"2024-04-25T10:24:17.855921+0000","last_peered":"2024-04-25T10:24:17.855921+0000","last_clean":"2024-04-25T10:24:17.855921+0000","last_became_active":"2024-04-25T10:24:16.830078+0000","last_became_peered":"2024-04-25T10:24:16.830078+0000","last_unstale":"2024-04-25T10:24:17.855921+0000","last_undegraded":"2024-04-25T10:24:17.855921+0000","last_fullsized":"2024-04-25T10:24:17.855921+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:49.339835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.836277+0000","last_change":"2024-04-25T10:24:17.832378+0000","last_active":"2024-04-25T10:24:17.836277+0000","last_peered":"2024-04-25T10:24:17.836277+0000","last_clean":"2024-04-25T10:24:17.836277+0000","last_became_active":"2024-04-25T10:24:17.832031+0000","last_became_peered":"2024-04-25T10:24:17.832031+0000","last_unstale":"2024-04-25T10:24:17.836277+0000","last_undegraded":"2024-04-25T10:24:17.836277+0000","last_fullsized":"2024-04-25T10:24:17.836277+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:40.852593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828024+0000","last_change":"2024-04-25T10:24:16.833266+0000","last_active":"2024-04-25T10:24:18.828024+0000","last_peered":"2024-04-25T10:24:18.828024+0000","last_clean":"2024-04-25T10:24:18.828024+0000","last_became_active":"2024-04-25T10:24:16.832972+0000","last_became_peered":"2024-04-25T10:24:16.832972+0000","last_unstale":"2024-04-25T10:24:18.828024+0000","last_undegraded":"2024-04-25T10:24:18.828024+0000","last_fullsized":"2024-04-25T10:24:18.828024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:01.710862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846988+0000","last_change":"2024-04-25T10:24:17.838921+0000","last_active":"2024-04-25T10:24:17.846988+0000","last_peered":"2024-04-25T10:24:17.846988+0000","last_clean":"2024-04-25T10:24:17.846988+0000","last_became_active":"2024-04-25T10:24:17.838644+0000","last_became_peered":"2024-04-25T10:24:17.838644+0000","last_unstale":"2024-04-25T10:24:17.846988+0000","last_undegraded":"2024-04-25T10:24:17.846988+0000","last_fullsized":"2024-04-25T10:24:17.846988+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:08.432023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830427+0000","last_change":"2024-04-25T10:24:16.826200+0000","last_active":"2024-04-25T10:24:18.830427+0000","last_peered":"2024-04-25T10:24:18.830427+0000","last_clean":"2024-04-25T10:24:18.830427+0000","last_became_active":"2024-04-25T10:24:16.825913+0000","last_became_peered":"2024-04-25T10:24:16.825913+0000","last_unstale":"2024-04-25T10:24:18.830427+0000","last_undegraded":"2024-04-25T10:24:18.830427+0000","last_fullsized":"2024-04-25T10:24:18.830427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:34.737370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846904+0000","last_change":"2024-04-25T10:24:17.843409+0000","last_active":"2024-04-25T10:24:17.846904+0000","last_peered":"2024-04-25T10:24:17.846904+0000","last_clean":"2024-04-25T10:24:17.846904+0000","last_became_active":"2024-04-25T10:24:17.843020+0000","last_became_peered":"2024-04-25T10:24:17.843020+0000","last_unstale":"2024-04-25T10:24:17.846904+0000","last_undegraded":"2024-04-25T10:24:17.846904+0000","last_fullsized":"2024-04-25T10:24:17.846904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:22.276355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829714+0000","last_change":"2024-04-25T10:24:16.829898+0000","last_active":"2024-04-25T10:24:18.829714+0000","last_peered":"2024-04-25T10:24:18.829714+0000","last_clean":"2024-04-25T10:24:18.829714+0000","last_became_active":"2024-04-25T10:24:16.829634+0000","last_became_peered":"2024-04-25T10:24:16.829634+0000","last_unstale":"2024-04-25T10:24:18.829714+0000","last_undegraded":"2024-04-25T10:24:18.829714+0000","last_fullsized":"2024-04-25T10:24:18.829714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:36.590024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829470+0000","last_change":"2024-04-25T10:24:17.843673+0000","last_active":"2024-04-25T10:24:18.829470+0000","last_peered":"2024-04-25T10:24:18.829470+0000","last_clean":"2024-04-25T10:24:18.829470+0000","last_became_active":"2024-04-25T10:24:17.843408+0000","last_became_peered":"2024-04-25T10:24:17.843408+0000","last_unstale":"2024-04-25T10:24:18.829470+0000","last_undegraded":"2024-04-25T10:24:18.829470+0000","last_fullsized":"2024-04-25T10:24:18.829470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:37.628056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.856059+0000","last_change":"2024-04-25T10:24:16.839407+0000","last_active":"2024-04-25T10:24:17.856059+0000","last_peered":"2024-04-25T10:24:17.856059+0000","last_clean":"2024-04-25T10:24:17.856059+0000","last_became_active":"2024-04-25T10:24:16.839119+0000","last_became_peered":"2024-04-25T10:24:16.839119+0000","last_unstale":"2024-04-25T10:24:17.856059+0000","last_undegraded":"2024-04-25T10:24:17.856059+0000","last_fullsized":"2024-04-25T10:24:17.856059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:47.398559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827755+0000","last_change":"2024-04-25T10:24:17.843580+0000","last_active":"2024-04-25T10:24:18.827755+0000","last_peered":"2024-04-25T10:24:18.827755+0000","last_clean":"2024-04-25T10:24:18.827755+0000","last_became_active":"2024-04-25T10:24:17.843315+0000","last_became_peered":"2024-04-25T10:24:17.843315+0000","last_unstale":"2024-04-25T10:24:18.827755+0000","last_undegraded":"2024-04-25T10:24:18.827755+0000","last_fullsized":"2024-04-25T10:24:18.827755+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:46.295720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827671+0000","last_change":"2024-04-25T10:24:16.834136+0000","last_active":"2024-04-25T10:24:18.827671+0000","last_peered":"2024-04-25T10:24:18.827671+0000","last_clean":"2024-04-25T10:24:18.827671+0000","last_became_active":"2024-04-25T10:24:16.833616+0000","last_became_peered":"2024-04-25T10:24:16.833616+0000","last_unstale":"2024-04-25T10:24:18.827671+0000","last_undegraded":"2024-04-25T10:24:18.827671+0000","last_fullsized":"2024-04-25T10:24:18.827671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:20.196903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844379+0000","last_change":"2024-04-25T10:24:17.842529+0000","last_active":"2024-04-25T10:24:17.844379+0000","last_peered":"2024-04-25T10:24:17.844379+0000","last_clean":"2024-04-25T10:24:17.844379+0000","last_became_active":"2024-04-25T10:24:17.842192+0000","last_became_peered":"2024-04-25T10:24:17.842192+0000","last_unstale":"2024-04-25T10:24:17.844379+0000","last_undegraded":"2024-04-25T10:24:17.844379+0000","last_fullsized":"2024-04-25T10:24:17.844379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:13.448524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.825746+0000","last_change":"2024-04-25T10:24:16.839002+0000","last_active":"2024-04-25T10:24:17.825746+0000","last_peered":"2024-04-25T10:24:17.825746+0000","last_clean":"2024-04-25T10:24:17.825746+0000","last_became_active":"2024-04-25T10:24:16.838731+0000","last_became_peered":"2024-04-25T10:24:16.838731+0000","last_unstale":"2024-04-25T10:24:17.825746+0000","last_undegraded":"2024-04-25T10:24:17.825746+0000","last_fullsized":"2024-04-25T10:24:17.825746+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:08.232629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.840502+0000","last_change":"2024-04-25T10:24:17.837160+0000","last_active":"2024-04-25T10:24:17.840502+0000","last_peered":"2024-04-25T10:24:17.840502+0000","last_clean":"2024-04-25T10:24:17.840502+0000","last_became_active":"2024-04-25T10:24:17.836613+0000","last_became_peered":"2024-04-25T10:24:17.836613+0000","last_unstale":"2024-04-25T10:24:17.840502+0000","last_undegraded":"2024-04-25T10:24:17.840502+0000","last_fullsized":"2024-04-25T10:24:17.840502+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:22.276355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.828058+0000","last_change":"2024-04-25T10:24:16.837830+0000","last_active":"2024-04-25T10:24:17.828058+0000","last_peered":"2024-04-25T10:24:17.828058+0000","last_clean":"2024-04-25T10:24:17.828058+0000","last_became_active":"2024-04-25T10:24:16.837559+0000","last_became_peered":"2024-04-25T10:24:16.837559+0000","last_unstale":"2024-04-25T10:24:17.828058+0000","last_undegraded":"2024-04-25T10:24:17.828058+0000","last_fullsized":"2024-04-25T10:24:17.828058+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:53.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844355+0000","last_change":"2024-04-25T10:24:17.841454+0000","last_active":"2024-04-25T10:24:17.844355+0000","last_peered":"2024-04-25T10:24:17.844355+0000","last_clean":"2024-04-25T10:24:17.844355+0000","last_became_active":"2024-04-25T10:24:17.840918+0000","last_became_peered":"2024-04-25T10:24:17.840918+0000","last_unstale":"2024-04-25T10:24:17.844355+0000","last_undegraded":"2024-04-25T10:24:17.844355+0000","last_fullsized":"2024-04-25T10:24:17.844355+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:35.812396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.825245+0000","last_change":"2024-04-25T10:24:16.823038+0000","last_active":"2024-04-25T10:24:17.825245+0000","last_peered":"2024-04-25T10:24:17.825245+0000","last_clean":"2024-04-25T10:24:17.825245+0000","last_became_active":"2024-04-25T10:24:16.822756+0000","last_became_peered":"2024-04-25T10:24:16.822756+0000","last_unstale":"2024-04-25T10:24:17.825245+0000","last_undegraded":"2024-04-25T10:24:17.825245+0000","last_fullsized":"2024-04-25T10:24:17.825245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:08.232629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846252+0000","last_change":"2024-04-25T10:24:17.844134+0000","last_active":"2024-04-25T10:24:17.846252+0000","last_peered":"2024-04-25T10:24:17.846252+0000","last_clean":"2024-04-25T10:24:17.846252+0000","last_became_active":"2024-04-25T10:24:17.843775+0000","last_became_peered":"2024-04-25T10:24:17.843775+0000","last_unstale":"2024-04-25T10:24:17.846252+0000","last_undegraded":"2024-04-25T10:24:17.846252+0000","last_fullsized":"2024-04-25T10:24:17.846252+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:58.876843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828107+0000","last_change":"2024-04-25T10:24:16.839195+0000","last_active":"2024-04-25T10:24:18.828107+0000","last_peered":"2024-04-25T10:24:18.828107+0000","last_clean":"2024-04-25T10:24:18.828107+0000","last_became_active":"2024-04-25T10:24:16.838251+0000","last_became_peered":"2024-04-25T10:24:16.838251+0000","last_unstale":"2024-04-25T10:24:18.828107+0000","last_undegraded":"2024-04-25T10:24:18.828107+0000","last_fullsized":"2024-04-25T10:24:18.828107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:05.395985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830265+0000","last_change":"2024-04-25T10:24:17.842559+0000","last_active":"2024-04-25T10:24:18.830265+0000","last_peered":"2024-04-25T10:24:18.830265+0000","last_clean":"2024-04-25T10:24:18.830265+0000","last_became_active":"2024-04-25T10:24:17.841787+0000","last_became_peered":"2024-04-25T10:24:17.841787+0000","last_unstale":"2024-04-25T10:24:18.830265+0000","last_undegraded":"2024-04-25T10:24:18.830265+0000","last_fullsized":"2024-04-25T10:24:18.830265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.485882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.854772+0000","last_change":"2024-04-25T10:24:16.832777+0000","last_active":"2024-04-25T10:24:17.854772+0000","last_peered":"2024-04-25T10:24:17.854772+0000","last_clean":"2024-04-25T10:24:17.854772+0000","last_became_active":"2024-04-25T10:24:16.832360+0000","last_became_peered":"2024-04-25T10:24:16.832360+0000","last_unstale":"2024-04-25T10:24:17.854772+0000","last_undegraded":"2024-04-25T10:24:17.854772+0000","last_fullsized":"2024-04-25T10:24:17.854772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:59.312544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.848927+0000","last_change":"2024-04-25T10:24:17.841427+0000","last_active":"2024-04-25T10:24:17.848927+0000","last_peered":"2024-04-25T10:24:17.848927+0000","last_clean":"2024-04-25T10:24:17.848927+0000","last_became_active":"2024-04-25T10:24:17.841148+0000","last_became_peered":"2024-04-25T10:24:17.841148+0000","last_unstale":"2024-04-25T10:24:17.848927+0000","last_undegraded":"2024-04-25T10:24:17.848927+0000","last_fullsized":"2024-04-25T10:24:17.848927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:21.366821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.859164+0000","last_change":"2024-04-25T10:24:16.835769+0000","last_active":"2024-04-25T10:24:17.859164+0000","last_peered":"2024-04-25T10:24:17.859164+0000","last_clean":"2024-04-25T10:24:17.859164+0000","last_became_active":"2024-04-25T10:24:16.835478+0000","last_became_peered":"2024-04-25T10:24:16.835478+0000","last_unstale":"2024-04-25T10:24:17.859164+0000","last_undegraded":"2024-04-25T10:24:17.859164+0000","last_fullsized":"2024-04-25T10:24:17.859164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:14.683268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830009+0000","last_change":"2024-04-25T10:24:17.846547+0000","last_active":"2024-04-25T10:24:18.830009+0000","last_peered":"2024-04-25T10:24:18.830009+0000","last_clean":"2024-04-25T10:24:18.830009+0000","last_became_active":"2024-04-25T10:24:17.846259+0000","last_became_peered":"2024-04-25T10:24:17.846259+0000","last_unstale":"2024-04-25T10:24:18.830009+0000","last_undegraded":"2024-04-25T10:24:18.830009+0000","last_fullsized":"2024-04-25T10:24:18.830009+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:27.970876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829263+0000","last_change":"2024-04-25T10:24:16.825392+0000","last_active":"2024-04-25T10:24:18.829263+0000","last_peered":"2024-04-25T10:24:18.829263+0000","last_clean":"2024-04-25T10:24:18.829263+0000","last_became_active":"2024-04-25T10:24:16.825103+0000","last_became_peered":"2024-04-25T10:24:16.825103+0000","last_unstale":"2024-04-25T10:24:18.829263+0000","last_undegraded":"2024-04-25T10:24:18.829263+0000","last_fullsized":"2024-04-25T10:24:18.829263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:57.729076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.850724+0000","last_change":"2024-04-25T10:24:17.843715+0000","last_active":"2024-04-25T10:24:17.850724+0000","last_peered":"2024-04-25T10:24:17.850724+0000","last_clean":"2024-04-25T10:24:17.850724+0000","last_became_active":"2024-04-25T10:24:17.843418+0000","last_became_peered":"2024-04-25T10:24:17.843418+0000","last_unstale":"2024-04-25T10:24:17.850724+0000","last_undegraded":"2024-04-25T10:24:17.850724+0000","last_fullsized":"2024-04-25T10:24:17.850724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:07.108402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.822677+0000","last_change":"2024-04-25T10:24:16.835544+0000","last_active":"2024-04-25T10:24:17.822677+0000","last_peered":"2024-04-25T10:24:17.822677+0000","last_clean":"2024-04-25T10:24:17.822677+0000","last_became_active":"2024-04-25T10:24:16.835191+0000","last_became_peered":"2024-04-25T10:24:16.835191+0000","last_unstale":"2024-04-25T10:24:17.822677+0000","last_undegraded":"2024-04-25T10:24:17.822677+0000","last_fullsized":"2024-04-25T10:24:17.822677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:40.610067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.844848+0000","last_change":"2024-04-25T10:24:17.843357+0000","last_active":"2024-04-25T10:24:17.844848+0000","last_peered":"2024-04-25T10:24:17.844848+0000","last_clean":"2024-04-25T10:24:17.844848+0000","last_became_active":"2024-04-25T10:24:17.842848+0000","last_became_peered":"2024-04-25T10:24:17.842848+0000","last_unstale":"2024-04-25T10:24:17.844848+0000","last_undegraded":"2024-04-25T10:24:17.844848+0000","last_fullsized":"2024-04-25T10:24:17.844848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:22.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827377+0000","last_change":"2024-04-25T10:24:16.833067+0000","last_active":"2024-04-25T10:24:18.827377+0000","last_peered":"2024-04-25T10:24:18.827377+0000","last_clean":"2024-04-25T10:24:18.827377+0000","last_became_active":"2024-04-25T10:24:16.830978+0000","last_became_peered":"2024-04-25T10:24:16.830978+0000","last_unstale":"2024-04-25T10:24:18.827377+0000","last_undegraded":"2024-04-25T10:24:18.827377+0000","last_fullsized":"2024-04-25T10:24:18.827377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:10.479472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828589+0000","last_change":"2024-04-25T10:24:17.839023+0000","last_active":"2024-04-25T10:24:18.828589+0000","last_peered":"2024-04-25T10:24:18.828589+0000","last_clean":"2024-04-25T10:24:18.828589+0000","last_became_active":"2024-04-25T10:24:17.838545+0000","last_became_peered":"2024-04-25T10:24:17.838545+0000","last_unstale":"2024-04-25T10:24:18.828589+0000","last_undegraded":"2024-04-25T10:24:18.828589+0000","last_fullsized":"2024-04-25T10:24:18.828589+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.545041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828976+0000","last_change":"2024-04-25T10:24:16.831790+0000","last_active":"2024-04-25T10:24:18.828976+0000","last_peered":"2024-04-25T10:24:18.828976+0000","last_clean":"2024-04-25T10:24:18.828976+0000","last_became_active":"2024-04-25T10:24:16.831478+0000","last_became_peered":"2024-04-25T10:24:16.831478+0000","last_unstale":"2024-04-25T10:24:18.828976+0000","last_undegraded":"2024-04-25T10:24:18.828976+0000","last_fullsized":"2024-04-25T10:24:18.828976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:52.310977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846617+0000","last_change":"2024-04-25T10:24:17.839467+0000","last_active":"2024-04-25T10:24:17.846617+0000","last_peered":"2024-04-25T10:24:17.846617+0000","last_clean":"2024-04-25T10:24:17.846617+0000","last_became_active":"2024-04-25T10:24:17.839155+0000","last_became_peered":"2024-04-25T10:24:17.839155+0000","last_unstale":"2024-04-25T10:24:17.846617+0000","last_undegraded":"2024-04-25T10:24:17.846617+0000","last_fullsized":"2024-04-25T10:24:17.846617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:27.970876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.855707+0000","last_change":"2024-04-25T10:24:16.838820+0000","last_active":"2024-04-25T10:24:17.855707+0000","last_peered":"2024-04-25T10:24:17.855707+0000","last_clean":"2024-04-25T10:24:17.855707+0000","last_became_active":"2024-04-25T10:24:16.838346+0000","last_became_peered":"2024-04-25T10:24:16.838346+0000","last_unstale":"2024-04-25T10:24:17.855707+0000","last_undegraded":"2024-04-25T10:24:17.855707+0000","last_fullsized":"2024-04-25T10:24:17.855707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:45.952893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828223+0000","last_change":"2024-04-25T10:24:17.839246+0000","last_active":"2024-04-25T10:24:18.828223+0000","last_peered":"2024-04-25T10:24:18.828223+0000","last_clean":"2024-04-25T10:24:18.828223+0000","last_became_active":"2024-04-25T10:24:17.836460+0000","last_became_peered":"2024-04-25T10:24:17.836460+0000","last_unstale":"2024-04-25T10:24:18.828223+0000","last_undegraded":"2024-04-25T10:24:18.828223+0000","last_fullsized":"2024-04-25T10:24:18.828223+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:37.628056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.855966+0000","last_change":"2024-04-25T10:24:16.838997+0000","last_active":"2024-04-25T10:24:17.855966+0000","last_peered":"2024-04-25T10:24:17.855966+0000","last_clean":"2024-04-25T10:24:17.855966+0000","last_became_active":"2024-04-25T10:24:16.838764+0000","last_became_peered":"2024-04-25T10:24:16.838764+0000","last_unstale":"2024-04-25T10:24:17.855966+0000","last_undegraded":"2024-04-25T10:24:17.855966+0000","last_fullsized":"2024-04-25T10:24:17.855966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:39.846183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.840871+0000","last_change":"2024-04-25T10:24:17.837341+0000","last_active":"2024-04-25T10:24:17.840871+0000","last_peered":"2024-04-25T10:24:17.840871+0000","last_clean":"2024-04-25T10:24:17.840871+0000","last_became_active":"2024-04-25T10:24:17.837057+0000","last_became_peered":"2024-04-25T10:24:17.837057+0000","last_unstale":"2024-04-25T10:24:17.840871+0000","last_undegraded":"2024-04-25T10:24:17.840871+0000","last_fullsized":"2024-04-25T10:24:17.840871+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.508264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.830004+0000","last_change":"2024-04-25T10:24:16.834313+0000","last_active":"2024-04-25T10:24:17.830004+0000","last_peered":"2024-04-25T10:24:17.830004+0000","last_clean":"2024-04-25T10:24:17.830004+0000","last_became_active":"2024-04-25T10:24:16.834043+0000","last_became_peered":"2024-04-25T10:24:16.834043+0000","last_unstale":"2024-04-25T10:24:17.830004+0000","last_undegraded":"2024-04-25T10:24:17.830004+0000","last_fullsized":"2024-04-25T10:24:17.830004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:29.829805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.843440+0000","last_change":"2024-04-25T10:24:17.841359+0000","last_active":"2024-04-25T10:24:17.843440+0000","last_peered":"2024-04-25T10:24:17.843440+0000","last_clean":"2024-04-25T10:24:17.843440+0000","last_became_active":"2024-04-25T10:24:17.841089+0000","last_became_peered":"2024-04-25T10:24:17.841089+0000","last_unstale":"2024-04-25T10:24:17.843440+0000","last_undegraded":"2024-04-25T10:24:17.843440+0000","last_fullsized":"2024-04-25T10:24:17.843440+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:13.982822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.852787+0000","last_change":"2024-04-25T10:24:16.833962+0000","last_active":"2024-04-25T10:24:17.852787+0000","last_peered":"2024-04-25T10:24:17.852787+0000","last_clean":"2024-04-25T10:24:17.852787+0000","last_became_active":"2024-04-25T10:24:16.833348+0000","last_became_peered":"2024-04-25T10:24:16.833348+0000","last_unstale":"2024-04-25T10:24:17.852787+0000","last_undegraded":"2024-04-25T10:24:17.852787+0000","last_fullsized":"2024-04-25T10:24:17.852787+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:57.738807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.846469+0000","last_change":"2024-04-25T10:24:17.839492+0000","last_active":"2024-04-25T10:24:17.846469+0000","last_peered":"2024-04-25T10:24:17.846469+0000","last_clean":"2024-04-25T10:24:17.846469+0000","last_became_active":"2024-04-25T10:24:17.839209+0000","last_became_peered":"2024-04-25T10:24:17.839209+0000","last_unstale":"2024-04-25T10:24:17.846469+0000","last_undegraded":"2024-04-25T10:24:17.846469+0000","last_fullsized":"2024-04-25T10:24:17.846469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:22.179797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.822840+0000","last_change":"2024-04-25T10:24:16.825025+0000","last_active":"2024-04-25T10:24:17.822840+0000","last_peered":"2024-04-25T10:24:17.822840+0000","last_clean":"2024-04-25T10:24:17.822840+0000","last_became_active":"2024-04-25T10:24:16.824727+0000","last_became_peered":"2024-04-25T10:24:16.824727+0000","last_unstale":"2024-04-25T10:24:17.822840+0000","last_undegraded":"2024-04-25T10:24:17.822840+0000","last_fullsized":"2024-04-25T10:24:17.822840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:45.838531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.828849+0000","last_change":"2024-04-25T10:24:17.833116+0000","last_active":"2024-04-25T10:24:18.828849+0000","last_peered":"2024-04-25T10:24:18.828849+0000","last_clean":"2024-04-25T10:24:18.828849+0000","last_became_active":"2024-04-25T10:24:17.832837+0000","last_became_peered":"2024-04-25T10:24:17.832837+0000","last_unstale":"2024-04-25T10:24:18.828849+0000","last_undegraded":"2024-04-25T10:24:18.828849+0000","last_fullsized":"2024-04-25T10:24:18.828849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:39.411945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.829737+0000","last_change":"2024-04-25T10:24:16.833133+0000","last_active":"2024-04-25T10:24:18.829737+0000","last_peered":"2024-04-25T10:24:18.829737+0000","last_clean":"2024-04-25T10:24:18.829737+0000","last_became_active":"2024-04-25T10:24:16.832832+0000","last_became_peered":"2024-04-25T10:24:16.832832+0000","last_unstale":"2024-04-25T10:24:18.829737+0000","last_undegraded":"2024-04-25T10:24:18.829737+0000","last_fullsized":"2024-04-25T10:24:18.829737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:34.805986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.841273+0000","last_change":"2024-04-25T10:24:17.837357+0000","last_active":"2024-04-25T10:24:17.841273+0000","last_peered":"2024-04-25T10:24:17.841273+0000","last_clean":"2024-04-25T10:24:17.841273+0000","last_became_active":"2024-04-25T10:24:17.837024+0000","last_became_peered":"2024-04-25T10:24:17.837024+0000","last_unstale":"2024-04-25T10:24:17.841273+0000","last_undegraded":"2024-04-25T10:24:17.841273+0000","last_fullsized":"2024-04-25T10:24:17.841273+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.545041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.826946+0000","last_change":"2024-04-25T10:24:16.834378+0000","last_active":"2024-04-25T10:24:17.826946+0000","last_peered":"2024-04-25T10:24:17.826946+0000","last_clean":"2024-04-25T10:24:17.826946+0000","last_became_active":"2024-04-25T10:24:16.833856+0000","last_became_peered":"2024-04-25T10:24:16.833856+0000","last_unstale":"2024-04-25T10:24:17.826946+0000","last_undegraded":"2024-04-25T10:24:17.826946+0000","last_fullsized":"2024-04-25T10:24:17.826946+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:18.493835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833226+0000","last_change":"2024-04-25T10:24:17.824314+0000","last_active":"2024-04-25T10:24:17.833226+0000","last_peered":"2024-04-25T10:24:17.833226+0000","last_clean":"2024-04-25T10:24:17.833226+0000","last_became_active":"2024-04-25T10:24:17.824030+0000","last_became_peered":"2024-04-25T10:24:17.824030+0000","last_unstale":"2024-04-25T10:24:17.833226+0000","last_undegraded":"2024-04-25T10:24:17.833226+0000","last_fullsized":"2024-04-25T10:24:17.833226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:29.735646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.852583+0000","last_change":"2024-04-25T10:24:16.837771+0000","last_active":"2024-04-25T10:24:17.852583+0000","last_peered":"2024-04-25T10:24:17.852583+0000","last_clean":"2024-04-25T10:24:17.852583+0000","last_became_active":"2024-04-25T10:24:16.837466+0000","last_became_peered":"2024-04-25T10:24:16.837466+0000","last_unstale":"2024-04-25T10:24:17.852583+0000","last_undegraded":"2024-04-25T10:24:17.852583+0000","last_fullsized":"2024-04-25T10:24:17.852583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:00.996599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.845797+0000","last_change":"2024-04-25T10:24:17.838722+0000","last_active":"2024-04-25T10:24:17.845797+0000","last_peered":"2024-04-25T10:24:17.845797+0000","last_clean":"2024-04-25T10:24:17.845797+0000","last_became_active":"2024-04-25T10:24:17.838126+0000","last_became_peered":"2024-04-25T10:24:17.838126+0000","last_unstale":"2024-04-25T10:24:17.845797+0000","last_undegraded":"2024-04-25T10:24:17.845797+0000","last_fullsized":"2024-04-25T10:24:17.845797+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:58.758804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.842754+0000","last_change":"2024-04-25T10:24:16.828101+0000","last_active":"2024-04-25T10:24:18.842754+0000","last_peered":"2024-04-25T10:24:18.842754+0000","last_clean":"2024-04-25T10:24:18.842754+0000","last_became_active":"2024-04-25T10:24:16.827618+0000","last_became_peered":"2024-04-25T10:24:16.827618+0000","last_unstale":"2024-04-25T10:24:18.842754+0000","last_undegraded":"2024-04-25T10:24:18.842754+0000","last_fullsized":"2024-04-25T10:24:18.842754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:21.823062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.833404+0000","last_change":"2024-04-25T10:24:17.826359+0000","last_active":"2024-04-25T10:24:17.833404+0000","last_peered":"2024-04-25T10:24:17.833404+0000","last_clean":"2024-04-25T10:24:17.833404+0000","last_became_active":"2024-04-25T10:24:17.826039+0000","last_became_peered":"2024-04-25T10:24:17.826039+0000","last_unstale":"2024-04-25T10:24:17.833404+0000","last_undegraded":"2024-04-25T10:24:17.833404+0000","last_fullsized":"2024-04-25T10:24:17.833404+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.751829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.827168+0000","last_change":"2024-04-25T10:24:16.834986+0000","last_active":"2024-04-25T10:24:17.827168+0000","last_peered":"2024-04-25T10:24:17.827168+0000","last_clean":"2024-04-25T10:24:17.827168+0000","last_became_active":"2024-04-25T10:24:16.834284+0000","last_became_peered":"2024-04-25T10:24:16.834284+0000","last_unstale":"2024-04-25T10:24:17.827168+0000","last_undegraded":"2024-04-25T10:24:17.827168+0000","last_fullsized":"2024-04-25T10:24:17.827168+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:06.947896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827991+0000","last_change":"2024-04-25T10:24:17.844975+0000","last_active":"2024-04-25T10:24:18.827991+0000","last_peered":"2024-04-25T10:24:18.827991+0000","last_clean":"2024-04-25T10:24:18.827991+0000","last_became_active":"2024-04-25T10:24:17.844661+0000","last_became_peered":"2024-04-25T10:24:17.844661+0000","last_unstale":"2024-04-25T10:24:18.827991+0000","last_undegraded":"2024-04-25T10:24:18.827991+0000","last_fullsized":"2024-04-25T10:24:18.827991+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:04.418507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827641+0000","last_change":"2024-04-25T10:24:16.836994+0000","last_active":"2024-04-25T10:24:18.827641+0000","last_peered":"2024-04-25T10:24:18.827641+0000","last_clean":"2024-04-25T10:24:18.827641+0000","last_became_active":"2024-04-25T10:24:16.836698+0000","last_became_peered":"2024-04-25T10:24:16.836698+0000","last_unstale":"2024-04-25T10:24:18.827641+0000","last_undegraded":"2024-04-25T10:24:18.827641+0000","last_fullsized":"2024-04-25T10:24:18.827641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:24.134500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.837857+0000","last_change":"2024-04-25T10:24:17.834703+0000","last_active":"2024-04-25T10:24:17.837857+0000","last_peered":"2024-04-25T10:24:17.837857+0000","last_clean":"2024-04-25T10:24:17.837857+0000","last_became_active":"2024-04-25T10:24:17.834398+0000","last_became_peered":"2024-04-25T10:24:17.834398+0000","last_unstale":"2024-04-25T10:24:17.837857+0000","last_undegraded":"2024-04-25T10:24:17.837857+0000","last_fullsized":"2024-04-25T10:24:17.837857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:38.685064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.822898+0000","last_change":"2024-04-25T10:24:16.829669+0000","last_active":"2024-04-25T10:24:17.822898+0000","last_peered":"2024-04-25T10:24:17.822898+0000","last_clean":"2024-04-25T10:24:17.822898+0000","last_became_active":"2024-04-25T10:24:16.829361+0000","last_became_peered":"2024-04-25T10:24:16.829361+0000","last_unstale":"2024-04-25T10:24:17.822898+0000","last_undegraded":"2024-04-25T10:24:17.822898+0000","last_fullsized":"2024-04-25T10:24:17.822898+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:19.008841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830195+0000","last_change":"2024-04-25T10:24:17.846629+0000","last_active":"2024-04-25T10:24:18.830195+0000","last_peered":"2024-04-25T10:24:18.830195+0000","last_clean":"2024-04-25T10:24:18.830195+0000","last_became_active":"2024-04-25T10:24:17.846209+0000","last_became_peered":"2024-04-25T10:24:17.846209+0000","last_unstale":"2024-04-25T10:24:18.830195+0000","last_undegraded":"2024-04-25T10:24:18.830195+0000","last_fullsized":"2024-04-25T10:24:18.830195+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:18.507016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.827781+0000","last_change":"2024-04-25T10:24:16.833901+0000","last_active":"2024-04-25T10:24:18.827781+0000","last_peered":"2024-04-25T10:24:18.827781+0000","last_clean":"2024-04-25T10:24:18.827781+0000","last_became_active":"2024-04-25T10:24:16.833168+0000","last_became_peered":"2024-04-25T10:24:16.833168+0000","last_unstale":"2024-04-25T10:24:18.827781+0000","last_undegraded":"2024-04-25T10:24:18.827781+0000","last_fullsized":"2024-04-25T10:24:18.827781+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:52.420761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.837855+0000","last_change":"2024-04-25T10:24:17.832713+0000","last_active":"2024-04-25T10:24:17.837855+0000","last_peered":"2024-04-25T10:24:17.837855+0000","last_clean":"2024-04-25T10:24:17.837855+0000","last_became_active":"2024-04-25T10:24:17.832419+0000","last_became_peered":"2024-04-25T10:24:17.832419+0000","last_unstale":"2024-04-25T10:24:17.837855+0000","last_undegraded":"2024-04-25T10:24:17.837855+0000","last_fullsized":"2024-04-25T10:24:17.837855+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:30.836215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.821240+0000","last_change":"2024-04-25T10:24:16.833871+0000","last_active":"2024-04-25T10:24:17.821240+0000","last_peered":"2024-04-25T10:24:17.821240+0000","last_clean":"2024-04-25T10:24:17.821240+0000","last_became_active":"2024-04-25T10:24:16.833579+0000","last_became_peered":"2024-04-25T10:24:16.833579+0000","last_unstale":"2024-04-25T10:24:17.821240+0000","last_undegraded":"2024-04-25T10:24:17.821240+0000","last_fullsized":"2024-04-25T10:24:17.821240+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:25.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.834819+0000","last_change":"2024-04-25T10:24:17.828443+0000","last_active":"2024-04-25T10:24:17.834819+0000","last_peered":"2024-04-25T10:24:17.834819+0000","last_clean":"2024-04-25T10:24:17.834819+0000","last_became_active":"2024-04-25T10:24:17.828171+0000","last_became_peered":"2024-04-25T10:24:17.828171+0000","last_unstale":"2024-04-25T10:24:17.834819+0000","last_undegraded":"2024-04-25T10:24:17.834819+0000","last_fullsized":"2024-04-25T10:24:17.834819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:54.380310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.827934+0000","last_change":"2024-04-25T10:24:16.831898+0000","last_active":"2024-04-25T10:24:17.827934+0000","last_peered":"2024-04-25T10:24:17.827934+0000","last_clean":"2024-04-25T10:24:17.827934+0000","last_became_active":"2024-04-25T10:24:16.831530+0000","last_became_peered":"2024-04-25T10:24:16.831530+0000","last_unstale":"2024-04-25T10:24:17.827934+0000","last_undegraded":"2024-04-25T10:24:17.827934+0000","last_fullsized":"2024-04-25T10:24:17.827934+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:14.013649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T10:24:18.830258+0000","last_change":"2024-04-25T10:24:17.843609+0000","last_active":"2024-04-25T10:24:18.830258+0000","last_peered":"2024-04-25T10:24:18.830258+0000","last_clean":"2024-04-25T10:24:18.830258+0000","last_became_active":"2024-04-25T10:24:17.843102+0000","last_became_peered":"2024-04-25T10:24:17.843102+0000","last_unstale":"2024-04-25T10:24:18.830258+0000","last_undegraded":"2024-04-25T10:24:18.830258+0000","last_fullsized":"2024-04-25T10:24:18.830258+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:16.803928+0000","last_clean_scrub_stamp":"2024-04-25T10:24:16.803928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:25.140910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.854198+0000","last_change":"2024-04-25T10:24:16.836725+0000","last_active":"2024-04-25T10:24:17.854198+0000","last_peered":"2024-04-25T10:24:17.854198+0000","last_clean":"2024-04-25T10:24:17.854198+0000","last_became_active":"2024-04-25T10:24:16.831837+0000","last_became_peered":"2024-04-25T10:24:16.831837+0000","last_unstale":"2024-04-25T10:24:17.854198+0000","last_undegraded":"2024-04-25T10:24:17.854198+0000","last_fullsized":"2024-04-25T10:24:17.854198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:24:15.797518+0000","last_clean_scrub_stamp":"2024-04-25T10:24:15.797518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:08.878687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":302,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:24:17.821710+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:24:17.821710+0000","last_peered":"2024-04-25T10:24:17.821710+0000","last_clean":"2024-04-25T10:24:17.821710+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:24:17.821710+0000","last_undegraded":"2024-04-25T10:24:17.821710+0000","last_fullsized":"2024-04-25T10:24:17.821710+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509521,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46164,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325676,"statfs":{"total":96636764160,"available":96589492224,"internally_reserved":0,"allocated":1114112,"data_stored":477306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509522,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46164,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325676,"statfs":{"total":96636764160,"available":96589492224,"internally_reserved":0,"allocated":1118208,"data_stored":477908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50912,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320928,"statfs":{"total":96636764160,"available":96584630272,"internally_reserved":0,"allocated":1720320,"data_stored":1069250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509521,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54488,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317352,"statfs":{"total":96636764160,"available":96580968448,"internally_reserved":0,"allocated":1122304,"data_stored":477910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509521,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58640,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313200,"statfs":{"total":96636764160,"available":96576716800,"internally_reserved":0,"allocated":1110016,"data_stored":477236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509522,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50896,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320944,"statfs":{"total":96636764160,"available":96584646656,"internally_reserved":0,"allocated":1708032,"data_stored":1067694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509522,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":478870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54460,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317380,"statfs":{"total":96636764160,"available":96580997120,"internally_reserved":0,"allocated":1097728,"data_stored":470461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:24:22.334 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:24:22.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:24:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.521+0000 7fc325c00640 1 Processor -- start 2024-04-25T10:24:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.521+0000 7fc325c00640 1 -- start start 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.521+0000 7fc325c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc32015bbb0 0x7fc3200cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc325c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32006ca90 con 0x7fc3200cf960 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc325c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32006cd70 con 0x7fc3200b4660 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc325c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32006d050 con 0x7fc32015bbb0 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc32015bbb0 0x7fc3200cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc3200cf960 0x7fc3200b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58502/0 (socket says 172.21.15.45:58502) 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc32015bbb0 0x7fc3200cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43388/0 (socket says 172.21.15.45:43388) 2024-04-25T10:24:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31f400640 1 -- 172.21.15.45:0/4221582744 learned_addr learned my addr 172.21.15.45:0/4221582744 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31ea00640 1 -- 172.21.15.45:0/4221582744 >> v1:172.21.15.117:6789/0 conn(0x7fc3200b4660 legacy=0x7fc32006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31ea00640 1 -- 172.21.15.45:0/4221582744 >> v1:172.21.15.45:6789/0 conn(0x7fc3200cf960 legacy=0x7fc3200b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31ea00640 1 -- 172.21.15.45:0/4221582744 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200b2880 con 0x7fc32015bbb0 2024-04-25T10:24:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31e000640 1 -- 172.21.15.45:0/4221582744 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624221239 0 0) 0x7fc32006ca90 con 0x7fc3200cf960 2024-04-25T10:24:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31ea00640 1 --2- 172.21.15.45:0/4221582744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc32015bbb0 0x7fc3200cda80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc3100097c0 tx=0x7fc31001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71990c2c27b0e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31e000640 1 -- 172.21.15.45:0/4221582744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100b0070 con 0x7fc32015bbb0 2024-04-25T10:24:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31e000640 1 -- 172.21.15.45:0/4221582744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3100ab600 con 0x7fc32015bbb0 2024-04-25T10:24:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.525+0000 7fc31e000640 1 -- 172.21.15.45:0/4221582744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100ab9d0 con 0x7fc32015bbb0 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- 172.21.15.45:0/4221582744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc32015bbb0 msgr2=0x7fc3200cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 --2- 172.21.15.45:0/4221582744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc32015bbb0 0x7fc3200cda80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc3100097c0 tx=0x7fc31001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- 172.21.15.45:0/4221582744 shutdown_connections 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 --2- 172.21.15.45:0/4221582744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc32015bbb0 0x7fc3200cda80 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- 172.21.15.45:0/4221582744 >> 172.21.15.45:0/4221582744 conn(0x7fc320066b70 msgr2=0x7fc320066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- 172.21.15.45:0/4221582744 shutdown_connections 2024-04-25T10:24:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- 172.21.15.45:0/4221582744 wait complete. 2024-04-25T10:24:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 Processor -- start 2024-04-25T10:24:22.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- start start 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3200b4660 0x7fc320167340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 0x7fc3201fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32015bbb0 0x7fc3200ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200ce140 con 0x7fc3200cf960 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200cdfc0 con 0x7fc32015bbb0 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc325c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32016a180 con 0x7fc3200b4660 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 0x7fc3201fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 0x7fc3201fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43392/0 (socket says 172.21.15.45:43392) 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 -- 172.21.15.45:0/2499403011 learned_addr learned my addr 172.21.15.45:0/2499403011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc31f400640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32015bbb0 0x7fc3200ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc31ea00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3200b4660 0x7fc320167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 -- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3200b4660 msgr2=0x7fc320167340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3200b4660 0x7fc320167340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 -- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32015bbb0 msgr2=0x7fc3200ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32015bbb0 0x7fc3200ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 -- 172.21.15.45:0/2499403011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3201fb8f0 con 0x7fc3200cf960 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.529+0000 7fc316000640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 0x7fc3201fb3b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c390 tx=0x7fc30800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308017070 con 0x7fc3200cf960 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3080052d0 con 0x7fc3200cf960 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308003820 con 0x7fc3200cf960 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3201606d0 con 0x7fc3200cf960 2024-04-25T10:24:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc320160990 con 0x7fc3200cf960 2024-04-25T10:24:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32004ef30 con 0x7fc3200cf960 2024-04-25T10:24:22.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc30801a070 con 0x7fc3200cf960 2024-04-25T10:24:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc317400640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2fc066f60 0x7fc2fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc308084440 con 0x7fc3200cf960 2024-04-25T10:24:22.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.533+0000 7fc31ea00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2fc066f60 0x7fc2fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.537+0000 7fc31ea00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2fc066f60 0x7fc2fc069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc3100ad000 tx=0x7fc310008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.537+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc308005480 con 0x7fc3200cf960 2024-04-25T10:24:22.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.756+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc32006d980 con 0x7fc3200cf960 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc317400640 1 -- 172.21.15.45:0/2499403011 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc3080553b0 con 0x7fc3200cf960 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:24:22.760 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2fc066f60 msgr2=0x7fc2fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2fc066f60 0x7fc2fc069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc3100ad000 tx=0x7fc310008040 comp rx=0 tx=0).stop 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 msgr2=0x7fc3201fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 0x7fc3201fb3b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c390 tx=0x7fc30800c850 comp rx=0 tx=0).stop 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 shutdown_connections 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2fc066f60 0x7fc2fc069420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc32015bbb0 0x7fc3200ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3200cf960 0x7fc3201fb3b0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 --2- 172.21.15.45:0/2499403011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3200b4660 0x7fc320167340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 >> 172.21.15.45:0/2499403011 conn(0x7fc320066b70 msgr2=0x7fc32015fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:22.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 shutdown_connections 2024-04-25T10:24:22.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.760+0000 7fc325c00640 1 -- 172.21.15.45:0/2499403011 wait complete. 2024-04-25T10:24:22.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:22.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.920+0000 7f5b8bc00640 1 Processor -- start 2024-04-25T10:24:22.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.920+0000 7f5b8bc00640 1 -- start start 2024-04-25T10:24:22.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b8bc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5b840ae020 0x7f5b840ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b8bc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b840ad840 con 0x7f5b84070c20 2024-04-25T10:24:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b8bc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8415e9f0 con 0x7f5b840cf930 2024-04-25T10:24:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b8bc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8415bb90 con 0x7f5b840ae020 2024-04-25T10:24:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b83e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5b84070c20 0x7f5b8415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58512/0 (socket says 172.21.15.45:58512) 2024-04-25T10:24:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b83e00640 1 -- 172.21.15.45:0/450619756 learned_addr learned my addr 172.21.15.45:0/450619756 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b89c00640 1 --2- 172.21.15.45:0/450619756 >> v2:172.21.15.117:3300/0 conn(0x7f5b840ae020 0x7f5b840ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b83400640 1 -- 172.21.15.45:0/450619756 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453840182 0 0) 0x7f5b840ad840 con 0x7f5b84070c20 2024-04-25T10:24:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b83400640 1 -- 172.21.15.45:0/450619756 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6c003530 con 0x7f5b84070c20 2024-04-25T10:24:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b89c00640 1 -- 172.21.15.45:0/450619756 >> v1:172.21.15.117:6789/0 conn(0x7f5b840cf930 legacy=0x7f5b840cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b83400640 1 -- 172.21.15.45:0/450619756 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200976788 0 0) 0x7f5b8415e9f0 con 0x7f5b840cf930 2024-04-25T10:24:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b89c00640 1 -- 172.21.15.45:0/450619756 >> v1:172.21.15.45:6789/0 conn(0x7f5b84070c20 legacy=0x7f5b8415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.924+0000 7f5b89c00640 1 -- 172.21.15.45:0/450619756 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8415ecd0 con 0x7f5b840ae020 2024-04-25T10:24:22.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b89c00640 1 --2- 172.21.15.45:0/450619756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840ae020 0x7f5b840ae4b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009d90 tx=0x7f5b7c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ff3a736cb3cd54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b83400640 1 -- 172.21.15.45:0/450619756 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0b2070 con 0x7f5b840ae020 2024-04-25T10:24:22.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b83400640 1 -- 172.21.15.45:0/450619756 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0a5a00 con 0x7f5b840ae020 2024-04-25T10:24:22.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b83400640 1 -- 172.21.15.45:0/450619756 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0a5db0 con 0x7f5b840ae020 2024-04-25T10:24:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 -- 172.21.15.45:0/450619756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840ae020 msgr2=0x7f5b840ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/450619756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840ae020 0x7f5b840ae4b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009d90 tx=0x7f5b7c01c940 comp rx=0 tx=0).stop 2024-04-25T10:24:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 -- 172.21.15.45:0/450619756 shutdown_connections 2024-04-25T10:24:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/450619756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840ae020 0x7f5b840ae4b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 -- 172.21.15.45:0/450619756 >> 172.21.15.45:0/450619756 conn(0x7f5b84066b70 msgr2=0x7f5b84066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:22.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 -- 172.21.15.45:0/450619756 shutdown_connections 2024-04-25T10:24:22.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 -- 172.21.15.45:0/450619756 wait complete. 2024-04-25T10:24:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 Processor -- start 2024-04-25T10:24:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.928+0000 7f5b8bc00640 1 -- start start 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 0x7f5b840cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b840ae020 0x7f5b84151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840cf930 0x7f5b8415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8416ba70 con 0x7f5b84070c20 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8416b8f0 con 0x7f5b840cf930 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8416bbf0 con 0x7f5b840ae020 2024-04-25T10:24:22.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 0x7f5b840cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 0x7f5b840cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43406/0 (socket says 172.21.15.45:43406) 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 -- 172.21.15.45:0/2928682453 learned_addr learned my addr 172.21.15.45:0/2928682453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b83e00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b840ae020 0x7f5b84151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89c00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840cf930 0x7f5b8415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 -- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b840ae020 msgr2=0x7f5b84151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b840ae020 0x7f5b84151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 -- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840cf930 msgr2=0x7f5b8415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840cf930 0x7f5b8415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 -- 172.21.15.45:0/2928682453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840cca30 con 0x7f5b84070c20 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b89200640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 0x7f5b840cebb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c440 tx=0x7f5b7800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78005480 con 0x7f5b84070c20 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78003600 con 0x7f5b84070c20 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b780039f0 con 0x7f5b84070c20 2024-04-25T10:24:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b840ccd10 con 0x7f5b84070c20 2024-04-25T10:24:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b840cd2d0 con 0x7f5b84070c20 2024-04-25T10:24:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8404ef30 con 0x7f5b84070c20 2024-04-25T10:24:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.932+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b78003b90 con 0x7f5b84070c20 2024-04-25T10:24:22.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.936+0000 7f5b80c00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b64066f90 0x7f5b64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:22.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.936+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b78084bc0 con 0x7f5b84070c20 2024-04-25T10:24:22.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.936+0000 7f5b83e00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b64066f90 0x7f5b64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.936+0000 7f5b83e00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b64066f90 0x7f5b64069450 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b840cbf40 tx=0x7f5b70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:22.940+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b780182b0 con 0x7f5b84070c20 2024-04-25T10:24:23.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.132+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b840cf4e0 con 0x7f5b84070c20 2024-04-25T10:24:23.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.132+0000 7f5b80c00640 1 -- 172.21.15.45:0/2928682453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5b78055bc0 con 0x7f5b84070c20 2024-04-25T10:24:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:23.135 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:23.135 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b64066f90 msgr2=0x7f5b64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b64066f90 0x7f5b64069450 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5b840cbf40 tx=0x7f5b70009040 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 msgr2=0x7f5b840cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 0x7f5b840cebb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c440 tx=0x7f5b7800c900 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 shutdown_connections 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b64066f90 0x7f5b64069450 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b840cf930 0x7f5b8415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b840ae020 0x7f5b84151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 --2- 172.21.15.45:0/2928682453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b84070c20 0x7f5b840cebb0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 >> 172.21.15.45:0/2928682453 conn(0x7f5b84066b70 msgr2=0x7f5b84066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 shutdown_connections 2024-04-25T10:24:23.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.136+0000 7f5b8bc00640 1 -- 172.21.15.45:0/2928682453 wait complete. 2024-04-25T10:24:23.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:24:17.216714+0000', 'modified': '2024-04-25T10:24:20.235839+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7344}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7344': {'gid': 7344, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6834/4102215497', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 4102215497}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 4102215497}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:24:23.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:24:23.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:24:23.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 Processor -- start 2024-04-25T10:24:23.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 -- start start 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff37415df60 0x7ff3740cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 0x7ff37406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3740ccaa0 con 0x7ff3740d0180 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37415e340 con 0x7ff37415df60 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff37aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740ccd80 con 0x7ff37406aea0 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff36b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff3740d0180 0x7ff3740d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58524/0 (socket says 172.21.15.45:58524) 2024-04-25T10:24:23.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff36b400640 1 -- 172.21.15.45:0/1197625023 learned_addr learned my addr 172.21.15.45:0/1197625023 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff378a00640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 0x7ff37406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064789127 0 0) 0x7ff3740ccaa0 con 0x7ff3740d0180 2024-04-25T10:24:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff373e00640 1 --2- 172.21.15.45:0/1197625023 >> v2:172.21.15.117:3300/0 conn(0x7ff37415df60 0x7ff3740cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.704+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff358003530 con 0x7ff3740d0180 2024-04-25T10:24:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980493457 0 0) 0x7ff358003530 con 0x7ff3740d0180 2024-04-25T10:24:23.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 msgr2=0x7ff37406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 0x7ff37406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 >> v2:172.21.15.117:3300/0 conn(0x7ff37415df60 msgr2=0x7ff3740cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 --2- 172.21.15.45:0/1197625023 >> v2:172.21.15.117:3300/0 conn(0x7ff37415df60 0x7ff3740cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740cd060 con 0x7ff3740d0180 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7ff35c001680 con 0x7ff3740d0180 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 >> v1:172.21.15.45:6789/0 conn(0x7ff3740d0180 legacy=0x7ff3740d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3580039c0 0x7ff358003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff358004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 0x7ff3580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740ccd80 con 0x7ff358004810 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35800b9d0 con 0x7ff3580039c0 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35800bb50 con 0x7ff37406aea0 2024-04-25T10:24:23.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff36b400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff358004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373e00640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3580039c0 0x7ff358003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff378a00640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 0x7ff3580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff36b400640 1 -- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 msgr2=0x7ff3580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff36b400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff37406aea0 0x7ff3580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff36b400640 1 -- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3580039c0 msgr2=0x7ff358003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff36b400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3580039c0 0x7ff358003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff36b400640 1 -- 172.21.15.45:0/1197625023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff358003530 con 0x7ff358004810 2024-04-25T10:24:23.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.708+0000 7ff373e00640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3580039c0 0x7ff358003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff36b400640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff358004c90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00c210 tx=0x7ff35c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a66a9a40a7e4eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c007d80 con 0x7ff358004810 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c0a9020 con 0x7ff358004810 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff373400640 1 -- 172.21.15.45:0/1197625023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c005b00 con 0x7ff358004810 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- 172.21.15.45:0/1197625023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 msgr2=0x7ff358004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 --2- 172.21.15.45:0/1197625023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff358004c90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00c210 tx=0x7ff35c01cbe0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff378a00640 1 -- 172.21.15.45:0/1197625023 reap_dead start 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- 172.21.15.45:0/1197625023 shutdown_connections 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- 172.21.15.45:0/1197625023 >> 172.21.15.45:0/1197625023 conn(0x7ff3740d1910 msgr2=0x7ff3740d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- 172.21.15.45:0/1197625023 shutdown_connections 2024-04-25T10:24:23.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- 172.21.15.45:0/1197625023 wait complete. 2024-04-25T10:24:23.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 Processor -- start 2024-04-25T10:24:23.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- start start 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff37414fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3740ca110 0x7ff374159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3740b6840 0x7ff37414f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740d0ff0 con 0x7ff358004810 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740d0e70 con 0x7ff3740b6840 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff37aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37406bac0 con 0x7ff3740ca110 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff37414fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff37414fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43410/0 (socket says 172.21.15.45:43410) 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.712+0000 7ff373e00640 1 -- 172.21.15.45:0/2278674579 learned_addr learned my addr 172.21.15.45:0/2278674579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff36b400640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3740ca110 0x7ff374159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff378a00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3740b6840 0x7ff37414f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff373e00640 1 -- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3740ca110 msgr2=0x7ff374159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff373e00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3740ca110 0x7ff374159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff373e00640 1 -- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3740b6840 msgr2=0x7ff37414f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff373e00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3740b6840 0x7ff37414f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff373e00640 1 -- 172.21.15.45:0/2278674579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740ca590 con 0x7ff358004810 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff373e00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff37414fe40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff36400f460 tx=0x7ff36400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff370c00640 1 -- 172.21.15.45:0/2278674579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364003ba0 con 0x7ff358004810 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff370c00640 1 -- 172.21.15.45:0/2278674579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364003d40 con 0x7ff358004810 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff370c00640 1 -- 172.21.15.45:0/2278674579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364004240 con 0x7ff358004810 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff374160170 con 0x7ff358004810 2024-04-25T10:24:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff374160730 con 0x7ff358004810 2024-04-25T10:24:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff370c00640 1 -- 172.21.15.45:0/2278674579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3640043e0 con 0x7ff358004810 2024-04-25T10:24:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff370c00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff350066fe0 0x7ff3500694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.716+0000 7ff370c00640 1 -- 172.21.15.45:0/2278674579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff364086190 con 0x7ff358004810 2024-04-25T10:24:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff36b400640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff350066fe0 0x7ff3500694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 Processor -- start 2024-04-25T10:24:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- start start 2024-04-25T10:24:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff36b400640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff350066fe0 0x7ff3500694a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0aa000 tx=0x7ff35c001a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1ce920 0x5617ae1ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae185310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ae06c5f0 con 0x5617ae184f30 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617adfd3da0 con 0x5617ae1cf240 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617adf5c150 con 0x5617ae1ce920 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae185310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae185310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43414/0 (socket says 172.21.15.45:43414) 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 -- 172.21.15.45:0/236546328 learned_addr learned my addr 172.21.15.45:0/236546328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff373e00640 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1ce920 0x5617ae1ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff36b400640 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 -- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1ce920 msgr2=0x5617ae1ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1ce920 0x5617ae1ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 -- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 msgr2=0x5617ae1849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 -- 172.21.15.45:0/236546328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617adf75ca0 con 0x5617ae184f30 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff36b400640 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff378a00640 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae185310 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff374068970 tx=0x7ff36c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15fe073f59552ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff368c00640 1 -- 172.21.15.45:0/236546328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364009c10 con 0x5617ae184f30 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff368c00640 1 -- 172.21.15.45:0/236546328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c014b50 con 0x5617ae184f30 2024-04-25T10:24:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff368c00640 1 -- 172.21.15.45:0/236546328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c004020 con 0x5617ae184f30 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- 172.21.15.45:0/236546328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 msgr2=0x5617ae185310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae185310 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff374068970 tx=0x7ff36c009190 comp rx=0 tx=0).stop 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- 172.21.15.45:0/236546328 shutdown_connections 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae185310 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 --2- 172.21.15.45:0/236546328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1ce920 0x5617ae1ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- 172.21.15.45:0/236546328 >> 172.21.15.45:0/236546328 conn(0x5617ae18b2a0 msgr2=0x5617ae1832d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- 172.21.15.45:0/236546328 shutdown_connections 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- 172.21.15.45:0/236546328 wait complete. 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 -- start start 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.720+0000 7ff37deda000 1 Processor -- start 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae1d5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1cf240 0x5617ae188aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617adf8e990 con 0x5617ae184f30 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617adfcfe40 con 0x5617ae1ce920 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617adf5f900 con 0x5617ae1cf240 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 10 client.0 _ll_register_callbacks cb 0x5617ae0871c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae1d5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae1d5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43420/0 (socket says 172.21.15.45:43420) 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 -- 172.21.15.45:0/1774763788 learned_addr learned my addr 172.21.15.45:0/1774763788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff36b400640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff378a00640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1cf240 0x5617ae188aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 -- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1cf240 msgr2=0x5617ae188aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1cf240 0x5617ae188aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 -- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 msgr2=0x5617ae1d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 -- 172.21.15.45:0/1774763788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617adf8c630 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff373e00640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae1d5d60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff36400fe20 tx=0x7ff364009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d437f36ef9c19542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 1 -- 172.21.15.45:0/1774763788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364082060 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 1 -- 172.21.15.45:0/1774763788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff364055ed0 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff36c014b50 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 1 -- 172.21.15.45:0/1774763788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff364073050 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 10 client.7497 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5617adf5c3a0 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 1 -- 172.21.15.45:0/1774763788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c014b50 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 1 -- 172.21.15.45:0/1774763788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff36405baf0 con 0x5617ae184f30 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 10 client.7497 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 10 client.7497 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 20 client.7497 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae18faf0 0x5617ae1894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 4 client.7497 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:23.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617adf6fa50 con 0x5617ae18faf0 2024-04-25T10:24:23.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff36b400640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae18faf0 0x5617ae1894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff36b400640 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae18faf0 0x5617ae1894b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e80c509d64902b server_cookie=da49824236eb1f1f in_seq=0 out_seq=0 2024-04-25T10:24:23.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.724+0000 7ff361600640 10 client.7497 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:23.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff361600640 1 -- 172.21.15.45:0/1774763788 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617adf6fa50 con 0x5617ae18faf0 2024-04-25T10:24:23.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff361600640 10 client.7497 handle_command_reply: tid=0 2024-04-25T10:24:23.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 client.7497 shutdown 2024-04-25T10:24:23.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 msgr2=0x5617ae1d5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae1d5d60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff36400fe20 tx=0x7ff364009d30 comp rx=0 tx=0).stop 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 shutdown_connections 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae1cf240 0x5617ae188aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae18faf0 0x5617ae1894b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 --2- 172.21.15.45:0/1774763788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae184f30 0x5617ae1d5d60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 >> 172.21.15.45:0/1774763788 conn(0x5617ae18b2a0 msgr2=0x5617ae1839e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff361600640 0 client.7497 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 shutdown_connections 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 1 -- 172.21.15.45:0/1774763788 wait complete. 2024-04-25T10:24:23.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.728+0000 7ff37deda000 20 client.7497 trim_cache size 0 max 16384 2024-04-25T10:24:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 Processor -- start 2024-04-25T10:24:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- start start 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae187320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 0x5617ae187860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1d6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374202450 con 0x5617ae1ce920 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740c9cc0 con 0x5617ae1cf240 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3741582d0 con 0x5617ae184f30 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 0x5617ae187860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 0x5617ae187860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43426/0 (socket says 172.21.15.45:43426) 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 -- 172.21.15.45:0/4285432311 learned_addr learned my addr 172.21.15.45:0/4285432311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff373e00640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae187320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 -- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 msgr2=0x5617ae187320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff378a00640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae187320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 -- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 msgr2=0x5617ae1d6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1d6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 -- 172.21.15.45:0/4285432311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617ae1cf070 con 0x5617ae1ce920 2024-04-25T10:24:23.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff373e00640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae187320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff378a00640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1d6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff36b400640 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 0x5617ae187860 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0ab040 tx=0x7ff35c000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7736fa0817d820d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff363e00640 1 -- 172.21.15.45:0/4285432311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0cd070 con 0x5617ae1ce920 2024-04-25T10:24:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff363e00640 1 -- 172.21.15.45:0/4285432311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c0a9030 con 0x5617ae1ce920 2024-04-25T10:24:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff363e00640 1 -- 172.21.15.45:0/4285432311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0077a0 con 0x5617ae1ce920 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- 172.21.15.45:0/4285432311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 msgr2=0x5617ae187860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 0x5617ae187860 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0ab040 tx=0x7ff35c000ff0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- 172.21.15.45:0/4285432311 shutdown_connections 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1cf240 0x5617ae1d6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1ce920 0x5617ae187860 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 --2- 172.21.15.45:0/4285432311 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae187320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- 172.21.15.45:0/4285432311 >> 172.21.15.45:0/4285432311 conn(0x5617ae18b2a0 msgr2=0x5617ae188750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- 172.21.15.45:0/4285432311 shutdown_connections 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.744+0000 7ff37deda000 1 -- 172.21.15.45:0/4285432311 wait complete. 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- start start 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 Processor -- start 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae22ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 0x5617ae1da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617adfd83d0 con 0x5617ae1cf240 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617ae08fd50 con 0x5617ae1ce920 2024-04-25T10:24:23.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617adf6e980 con 0x5617ae184f30 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 10 client.0 _ll_register_callbacks cb 0x7ff36c006c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 0x5617ae1da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 0x5617ae1da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43442/0 (socket says 172.21.15.45:43442) 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 -- 172.21.15.45:0/2165685047 learned_addr learned my addr 172.21.15.45:0/2165685047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff36b400640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff373e00640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae22ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 -- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 msgr2=0x5617ae22ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae22ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 -- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 msgr2=0x5617ae1d9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 -- 172.21.15.45:0/2165685047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617ae055600 con 0x5617ae1cf240 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff378a00640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 0x5617ae1da090 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff36c008010 tx=0x7ff36c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b2d45c165ef77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff361600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff361600640 1 -- 172.21.15.45:0/2165685047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003820 con 0x5617ae1cf240 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff361600640 1 -- 172.21.15.45:0/2165685047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff36c0039c0 con 0x5617ae1cf240 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff35c0a9030 con 0x5617ae1cf240 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff361600640 1 -- 172.21.15.45:0/2165685047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff36c006de0 con 0x5617ae1cf240 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 10 client.7503 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5617adf689b0 con 0x5617ae1cf240 2024-04-25T10:24:23.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff361600640 1 -- 172.21.15.45:0/2165685047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c0a9030 con 0x5617ae1cf240 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff361600640 1 -- 172.21.15.45:0/2165685047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff36c003bc0 con 0x5617ae1cf240 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 10 client.7503 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 10 client.7503 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 20 client.7503 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae1ea480 0x5617ae1ea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 4 client.7503 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:24:23.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5617adfe12e0 con 0x5617ae1ea480 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.748+0000 7ff373e00640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae1ea480 0x5617ae1ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff373e00640 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae1ea480 0x5617ae1ea860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84369537a624dbb4 server_cookie=81e4562b641b3a35 in_seq=0 out_seq=0 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff361600640 10 client.7503 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff361600640 1 -- 172.21.15.45:0/2165685047 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5617adfe12e0 con 0x5617ae1ea480 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff361600640 10 client.7503 handle_command_reply: tid=0 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 client.7503 shutdown 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 msgr2=0x5617ae1da090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 0x5617ae1da090 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff36c008010 tx=0x7ff36c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 shutdown_connections 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617ae1cf240 0x5617ae1da090 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5617ae1ea480 0x5617ae1ea860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617ae1ce920 0x5617ae1d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 --2- 172.21.15.45:0/2165685047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5617ae184f30 0x5617ae22ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff361600640 0 client.7503 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 >> 172.21.15.45:0/2165685047 conn(0x5617ae18b2a0 msgr2=0x5617ae187bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 shutdown_connections 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 1 -- 172.21.15.45:0/2165685047 wait complete. 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37deda000 20 client.7503 trim_cache size 0 max 16384 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:24:23.759 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7344, 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9262741859999997, 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 65, 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 92, 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.786853299000001 2024-04-25T10:24:23.760 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:24:23.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff350066fe0 msgr2=0x7ff3500694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff350066fe0 0x7ff3500694a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0aa000 tx=0x7ff35c001a50 comp rx=0 tx=0).stop 2024-04-25T10:24:23.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 msgr2=0x7ff37414fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff37414fe40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff36400f460 tx=0x7ff36400f920 comp rx=0 tx=0).stop 2024-04-25T10:24:23.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 shutdown_connections 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff350066fe0 0x7ff3500694a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3740b6840 0x7ff37414f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3740ca110 0x7ff374159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 --2- 172.21.15.45:0/2278674579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff358004810 0x7ff37414fe40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 >> 172.21.15.45:0/2278674579 conn(0x7ff3740d1910 msgr2=0x7ff3740cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 shutdown_connections 2024-04-25T10:24:23.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.752+0000 7ff37aa00640 1 -- 172.21.15.45:0/2278674579 wait complete. 2024-04-25T10:24:23.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:24:23.802 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-04-25T10:24:23.800+0000 7fb56e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:24:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 Processor -- start 2024-04-25T10:24:23.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 -- start start 2024-04-25T10:24:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe51c15b9e0 0x7fe51c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 0x7fe51c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c0ccfb0 con 0x7fe51c15bdc0 2024-04-25T10:24:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c0cd290 con 0x7fe51c15b9e0 2024-04-25T10:24:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe521c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0cd570 con 0x7fe51c0d1c30 2024-04-25T10:24:23.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe513e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe51c15bdc0 0x7fe51c0d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58534/0 (socket says 172.21.15.45:58534) 2024-04-25T10:24:23.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe513e00640 1 -- 172.21.15.45:0/2420009639 learned_addr learned my addr 172.21.15.45:0/2420009639 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51aa00640 1 --2- 172.21.15.45:0/2420009639 >> v2:172.21.15.117:3300/0 conn(0x7fe51c15b9e0 0x7fe51c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51b400640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 0x7fe51c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173851788 0 0) 0x7fe51c0ccfb0 con 0x7fe51c15bdc0 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe500003530 con 0x7fe51c15bdc0 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898050350 0 0) 0x7fe500003530 con 0x7fe51c15bdc0 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 msgr2=0x7fe51c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 0x7fe51c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 >> v2:172.21.15.117:3300/0 conn(0x7fe51c15b9e0 msgr2=0x7fe51c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 --2- 172.21.15.45:0/2420009639 >> v2:172.21.15.117:3300/0 conn(0x7fe51c15b9e0 0x7fe51c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0cd850 con 0x7fe51c15bdc0 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fe5040016a0 con 0x7fe51c15bdc0 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 >> v1:172.21.15.45:6789/0 conn(0x7fe51c15bdc0 legacy=0x7fe51c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5000039c0 0x7fe500003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe500004810 0x7fe500004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 0x7fe5000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0cd290 con 0x7fe5000039c0 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0cd570 con 0x7fe500004810 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.912+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50000ba20 con 0x7fe51c0d1c30 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5000039c0 0x7fe500003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe513e00640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe500004810 0x7fe500004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51b400640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 0x7fe5000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 -- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 msgr2=0x7fe5000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c0d1c30 0x7fe5000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 -- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe500004810 msgr2=0x7fe500004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe500004810 0x7fe500004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 -- 172.21.15.45:0/2420009639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe500003530 con 0x7fe5000039c0 2024-04-25T10:24:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51aa00640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5000039c0 0x7fe500003de0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009b10 tx=0x7fe50c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eefe0fab18b16a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0096b0 con 0x7fe5000039c0 2024-04-25T10:24:23.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c0a8410 con 0x7fe5000039c0 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.916+0000 7fe51a000640 1 -- 172.21.15.45:0/2420009639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0a8800 con 0x7fe5000039c0 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- 172.21.15.45:0/2420009639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5000039c0 msgr2=0x7fe500003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 --2- 172.21.15.45:0/2420009639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5000039c0 0x7fe500003de0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009b10 tx=0x7fe50c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 -- 172.21.15.45:0/2420009639 reap_dead start 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- 172.21.15.45:0/2420009639 shutdown_connections 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- 172.21.15.45:0/2420009639 >> 172.21.15.45:0/2420009639 conn(0x7fe51c09a9d0 msgr2=0x7fe51c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- 172.21.15.45:0/2420009639 shutdown_connections 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- 172.21.15.45:0/2420009639 wait complete. 2024-04-25T10:24:23.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 Processor -- start 2024-04-25T10:24:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- start start 2024-04-25T10:24:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5000039c0 0x7fe51c14bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c06bce0 0x7fe51c06c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 0x7fe51c0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c65f0 con 0x7fe51c0b46a0 2024-04-25T10:24:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c6470 con 0x7fe5000039c0 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe521c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c6770 con 0x7fe51c06bce0 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 0x7fe51c0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 0x7fe51c0b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43458/0 (socket says 172.21.15.45:43458) 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 -- 172.21.15.45:0/1412187865 learned_addr learned my addr 172.21.15.45:0/1412187865 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51aa00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5000039c0 0x7fe51c14bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe513e00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c06bce0 0x7fe51c06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 -- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c06bce0 msgr2=0x7fe51c06c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c06bce0 0x7fe51c06c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 -- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5000039c0 msgr2=0x7fe51c14bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5000039c0 0x7fe51c14bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe51b400640 1 -- 172.21.15.45:0/1412187865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c157000 con 0x7fe51c0b46a0 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.920+0000 7fe513e00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c06bce0 0x7fe51c06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe51b400640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 0x7fe51c0b4b20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0b3180 tx=0x7fe51400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51400fd80 con 0x7fe51c0b46a0 2024-04-25T10:24:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51401d070 con 0x7fe51c0b46a0 2024-04-25T10:24:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514015670 con 0x7fe51c0b46a0 2024-04-25T10:24:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c14c270 con 0x7fe51c0b46a0 2024-04-25T10:24:23.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c0b5560 con 0x7fe51c0b46a0 2024-04-25T10:24:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c0b2f90 con 0x7fe51c0b46a0 2024-04-25T10:24:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe514015810 con 0x7fe51c0b46a0 2024-04-25T10:24:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe512a00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4f8066f90 0x7fe4f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5140858f0 con 0x7fe51c0b46a0 2024-04-25T10:24:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.924+0000 7fe51aa00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4f8066f90 0x7fe4f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.928+0000 7fe51aa00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4f8066f90 0x7fe4f8069450 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c0b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:23.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:23.928+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe51c157000 con 0x7fe51c0b46a0 2024-04-25T10:24:24.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.128+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe51c04ef30 con 0x7fe51c0b46a0 2024-04-25T10:24:24.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.128+0000 7fe512a00640 1 -- 172.21.15.45:0/1412187865 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe514010040 con 0x7fe51c0b46a0 2024-04-25T10:24:24.130 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:24:24.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4f8066f90 msgr2=0x7fe4f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:24.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4f8066f90 0x7fe4f8069450 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0af000 tx=0x7fe50c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:24:24.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 msgr2=0x7fe51c0b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:24.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 0x7fe51c0b4b20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0b3180 tx=0x7fe51400f580 comp rx=0 tx=0).stop 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 shutdown_connections 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4f8066f90 0x7fe4f8069450 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe51c0b46a0 0x7fe51c0b4b20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe51c06bce0 0x7fe51c06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 --2- 172.21.15.45:0/1412187865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5000039c0 0x7fe51c14bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 >> 172.21.15.45:0/1412187865 conn(0x7fe51c09a9d0 msgr2=0x7fe51c157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 shutdown_connections 2024-04-25T10:24:24.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:24.132+0000 7fe521c00640 1 -- 172.21.15.45:0/1412187865 wait complete. 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:24:24.152 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:24:24.153 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:24:24.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.153 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:24:24.204 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42076sec preferred_lft 42076sec 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:24.205 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:24:24.206 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.207 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:24:24.207 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:24:24.207 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:24:24.207 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:24:24.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.208 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:24:24.269 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:24:24.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.271 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:24:24.324 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:24:24.324 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:24:24.325 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:24:24.325 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:24:24.325 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:24:24.325 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:24:24.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.325 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:24:24.325 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:24.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:24:24.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:24:24.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:24:24.325 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:24.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.417 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:24:24.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:24:24.426 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:24:24.427 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:24:24.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:24:24.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:24:24.487 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:24.539 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:24:24.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:24:24.540 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:24.591 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:24:24.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:24:24.690 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:24:24.690 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:24:24.701 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:24:24.701 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:24:24.701 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:24:24.702 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:24:24.702 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:24:24.705 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:24:24.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:24:24.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.726 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:24:24.786 INFO:teuthology.orchestra.run.smithi045.stdout:client6449 2024-04-25T10:24:24.787 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:24:24.787 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:24.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:24.840 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:24.840 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:24.840 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:24:24.840 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:24.840 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:24.841 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:24.841 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:24.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:24.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:24.929 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204557 s, 51.3 MB/s 2024-04-25T10:24:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:24.948 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171696 s, 61.1 MB/s 2024-04-25T10:24:24.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:24.967 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01678 s, 62.5 MB/s 2024-04-25T10:24:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:24.984 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156934 s, 66.8 MB/s 2024-04-25T10:24:25.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.002 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155344 s, 67.5 MB/s 2024-04-25T10:24:25.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232012 s, 45.2 MB/s 2024-04-25T10:24:25.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.045 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155284 s, 67.5 MB/s 2024-04-25T10:24:25.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142462 s, 73.6 MB/s 2024-04-25T10:24:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014333 s, 73.2 MB/s 2024-04-25T10:24:25.094 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.094 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.095 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164235 s, 63.8 MB/s 2024-04-25T10:24:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144563 s, 72.5 MB/s 2024-04-25T10:24:25.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.127 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.127 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142243 s, 73.7 MB/s 2024-04-25T10:24:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175492 s, 59.8 MB/s 2024-04-25T10:24:25.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.168 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200438 s, 52.3 MB/s 2024-04-25T10:24:25.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145847 s, 71.9 MB/s 2024-04-25T10:24:25.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179547 s, 58.4 MB/s 2024-04-25T10:24:25.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.224 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017596 s, 59.6 MB/s 2024-04-25T10:24:25.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.241 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154069 s, 68.1 MB/s 2024-04-25T10:24:25.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.260 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169948 s, 61.7 MB/s 2024-04-25T10:24:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149225 s, 70.3 MB/s 2024-04-25T10:24:25.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144462 s, 72.6 MB/s 2024-04-25T10:24:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134013 s, 78.2 MB/s 2024-04-25T10:24:25.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.325 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158333 s, 66.2 MB/s 2024-04-25T10:24:25.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165693 s, 63.3 MB/s 2024-04-25T10:24:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152094 s, 68.9 MB/s 2024-04-25T10:24:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156672 s, 66.9 MB/s 2024-04-25T10:24:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152307 s, 68.8 MB/s 2024-04-25T10:24:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163457 s, 64.2 MB/s 2024-04-25T10:24:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177992 s, 58.9 MB/s 2024-04-25T10:24:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136169 s, 77.0 MB/s 2024-04-25T10:24:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139744 s, 75.0 MB/s 2024-04-25T10:24:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139301 s, 75.3 MB/s 2024-04-25T10:24:25.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:25.481 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152298 s, 68.9 MB/s 2024-04-25T10:24:25.571 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.572 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138522 s, 75.7 MB/s 2024-04-25T10:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.588 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144711 s, 72.5 MB/s 2024-04-25T10:24:25.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.604 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139341 s, 75.3 MB/s 2024-04-25T10:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154961 s, 67.7 MB/s 2024-04-25T10:24:25.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135216 s, 77.5 MB/s 2024-04-25T10:24:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137005 s, 76.5 MB/s 2024-04-25T10:24:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141469 s, 74.1 MB/s 2024-04-25T10:24:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133637 s, 78.5 MB/s 2024-04-25T10:24:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.701 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155732 s, 67.3 MB/s 2024-04-25T10:24:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149719 s, 70.0 MB/s 2024-04-25T10:24:25.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.733 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138913 s, 75.5 MB/s 2024-04-25T10:24:25.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013927 s, 75.3 MB/s 2024-04-25T10:24:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.766 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153416 s, 68.3 MB/s 2024-04-25T10:24:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143505 s, 73.1 MB/s 2024-04-25T10:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155695 s, 67.3 MB/s 2024-04-25T10:24:25.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181285 s, 57.8 MB/s 2024-04-25T10:24:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215674 s, 48.6 MB/s 2024-04-25T10:24:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157211 s, 66.7 MB/s 2024-04-25T10:24:25.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:25.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:25.876 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139959 s, 74.9 MB/s 2024-04-25T10:24:25.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.013 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139374 s, 75.2 MB/s 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149537 s, 70.1 MB/s 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013614 s, 77.0 MB/s 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013859 s, 75.7 MB/s 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149872 s, 70.0 MB/s 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132332 s, 79.2 MB/s 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151633 s, 69.2 MB/s 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150883 s, 69.5 MB/s 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135818 s, 77.2 MB/s 2024-04-25T10:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014391 s, 72.9 MB/s 2024-04-25T10:24:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.057 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181226 s, 57.9 MB/s 2024-04-25T10:24:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014529 s, 72.2 MB/s 2024-04-25T10:24:26.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:26.074 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:26.074 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:26.074 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:24:26.074 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:26.074 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:26.074 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:26.075 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151048 s, 69.4 MB/s 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.161 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152569 s, 68.7 MB/s 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130058 s, 80.6 MB/s 2024-04-25T10:24:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158018 s, 66.4 MB/s 2024-04-25T10:24:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155864 s, 67.3 MB/s 2024-04-25T10:24:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.227 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141742 s, 74.0 MB/s 2024-04-25T10:24:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138209 s, 75.9 MB/s 2024-04-25T10:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162055 s, 64.7 MB/s 2024-04-25T10:24:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146715 s, 71.5 MB/s 2024-04-25T10:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139195 s, 75.3 MB/s 2024-04-25T10:24:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.309 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145276 s, 72.2 MB/s 2024-04-25T10:24:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01507 s, 69.6 MB/s 2024-04-25T10:24:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132815 s, 79.0 MB/s 2024-04-25T10:24:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136157 s, 77.0 MB/s 2024-04-25T10:24:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137218 s, 76.4 MB/s 2024-04-25T10:24:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136037 s, 77.1 MB/s 2024-04-25T10:24:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.404 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153667 s, 68.2 MB/s 2024-04-25T10:24:26.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.421 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154164 s, 68.0 MB/s 2024-04-25T10:24:26.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.437 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136692 s, 76.7 MB/s 2024-04-25T10:24:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.454 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154668 s, 67.8 MB/s 2024-04-25T10:24:26.469 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.470 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137609 s, 76.2 MB/s 2024-04-25T10:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143152 s, 73.2 MB/s 2024-04-25T10:24:26.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145942 s, 71.8 MB/s 2024-04-25T10:24:26.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.519 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146852 s, 71.4 MB/s 2024-04-25T10:24:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015004 s, 69.9 MB/s 2024-04-25T10:24:26.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146403 s, 71.6 MB/s 2024-04-25T10:24:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.572 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181505 s, 57.8 MB/s 2024-04-25T10:24:26.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015995 s, 65.6 MB/s 2024-04-25T10:24:26.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.605 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133675 s, 78.4 MB/s 2024-04-25T10:24:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144782 s, 72.4 MB/s 2024-04-25T10:24:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133155 s, 78.7 MB/s 2024-04-25T10:24:26.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.653 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147792 s, 70.9 MB/s 2024-04-25T10:24:26.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:26.654 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:26.724 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.724 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153874 s, 68.1 MB/s 2024-04-25T10:24:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015572 s, 67.3 MB/s 2024-04-25T10:24:26.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013761 s, 76.2 MB/s 2024-04-25T10:24:26.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129175 s, 81.2 MB/s 2024-04-25T10:24:26.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.788 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137097 s, 76.5 MB/s 2024-04-25T10:24:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140648 s, 74.6 MB/s 2024-04-25T10:24:26.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129687 s, 80.9 MB/s 2024-04-25T10:24:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172425 s, 60.8 MB/s 2024-04-25T10:24:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.853 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140018 s, 74.9 MB/s 2024-04-25T10:24:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.871 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163247 s, 64.2 MB/s 2024-04-25T10:24:26.891 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184619 s, 56.8 MB/s 2024-04-25T10:24:26.908 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.908 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.908 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146824 s, 71.4 MB/s 2024-04-25T10:24:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134301 s, 78.1 MB/s 2024-04-25T10:24:26.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014262 s, 73.5 MB/s 2024-04-25T10:24:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133294 s, 78.7 MB/s 2024-04-25T10:24:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.971 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014366 s, 73.0 MB/s 2024-04-25T10:24:26.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:26.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:26.988 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149151 s, 70.3 MB/s 2024-04-25T10:24:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147379 s, 71.1 MB/s 2024-04-25T10:24:27.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147076 s, 71.3 MB/s 2024-04-25T10:24:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153259 s, 68.4 MB/s 2024-04-25T10:24:27.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136712 s, 76.7 MB/s 2024-04-25T10:24:27.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147582 s, 71.1 MB/s 2024-04-25T10:24:27.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.088 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167014 s, 62.8 MB/s 2024-04-25T10:24:27.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.104 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138569 s, 75.7 MB/s 2024-04-25T10:24:27.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141857 s, 73.9 MB/s 2024-04-25T10:24:27.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.136 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141686 s, 74.0 MB/s 2024-04-25T10:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190066 s, 55.2 MB/s 2024-04-25T10:24:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141082 s, 74.3 MB/s 2024-04-25T10:24:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155165 s, 67.6 MB/s 2024-04-25T10:24:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138988 s, 75.4 MB/s 2024-04-25T10:24:27.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.222 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137074 s, 76.5 MB/s 2024-04-25T10:24:27.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186991 s, 56.1 MB/s 2024-04-25T10:24:27.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:27.243 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:27.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145825 s, 71.9 MB/s 2024-04-25T10:24:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013013 s, 80.6 MB/s 2024-04-25T10:24:27.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133664 s, 78.4 MB/s 2024-04-25T10:24:27.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.366 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183206 s, 57.2 MB/s 2024-04-25T10:24:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.383 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148821 s, 70.5 MB/s 2024-04-25T10:24:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133922 s, 78.3 MB/s 2024-04-25T10:24:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136641 s, 76.7 MB/s 2024-04-25T10:24:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188954 s, 55.5 MB/s 2024-04-25T10:24:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155269 s, 67.5 MB/s 2024-04-25T10:24:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133326 s, 78.6 MB/s 2024-04-25T10:24:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141719 s, 74.0 MB/s 2024-04-25T10:24:27.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134551 s, 77.9 MB/s 2024-04-25T10:24:27.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.514 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149239 s, 70.3 MB/s 2024-04-25T10:24:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188535 s, 55.6 MB/s 2024-04-25T10:24:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156205 s, 67.1 MB/s 2024-04-25T10:24:27.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134934 s, 77.7 MB/s 2024-04-25T10:24:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137422 s, 76.3 MB/s 2024-04-25T10:24:27.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145545 s, 72.0 MB/s 2024-04-25T10:24:27.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.616 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145913 s, 71.9 MB/s 2024-04-25T10:24:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147044 s, 71.3 MB/s 2024-04-25T10:24:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151462 s, 69.2 MB/s 2024-04-25T10:24:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147854 s, 70.9 MB/s 2024-04-25T10:24:27.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135884 s, 77.2 MB/s 2024-04-25T10:24:27.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148791 s, 70.5 MB/s 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.713 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138374 s, 75.8 MB/s 2024-04-25T10:24:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132528 s, 79.1 MB/s 2024-04-25T10:24:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148427 s, 70.6 MB/s 2024-04-25T10:24:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180088 s, 58.2 MB/s 2024-04-25T10:24:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.780 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128746 s, 81.4 MB/s 2024-04-25T10:24:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137016 s, 76.5 MB/s 2024-04-25T10:24:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140478 s, 74.6 MB/s 2024-04-25T10:24:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135715 s, 77.3 MB/s 2024-04-25T10:24:27.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:27.828 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148131 s, 70.8 MB/s 2024-04-25T10:24:27.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135952 s, 77.1 MB/s 2024-04-25T10:24:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134321 s, 78.1 MB/s 2024-04-25T10:24:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148709 s, 70.5 MB/s 2024-04-25T10:24:27.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.962 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133526 s, 78.5 MB/s 2024-04-25T10:24:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150186 s, 69.8 MB/s 2024-04-25T10:24:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:27.996 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014987 s, 70.0 MB/s 2024-04-25T10:24:28.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152161 s, 68.9 MB/s 2024-04-25T10:24:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151802 s, 69.1 MB/s 2024-04-25T10:24:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018235 s, 57.5 MB/s 2024-04-25T10:24:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145801 s, 71.9 MB/s 2024-04-25T10:24:28.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144945 s, 72.3 MB/s 2024-04-25T10:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203639 s, 51.5 MB/s 2024-04-25T10:24:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135158 s, 77.6 MB/s 2024-04-25T10:24:28.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148054 s, 70.8 MB/s 2024-04-25T10:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.152 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132607 s, 79.1 MB/s 2024-04-25T10:24:28.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152745 s, 68.6 MB/s 2024-04-25T10:24:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180189 s, 58.2 MB/s 2024-04-25T10:24:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015395 s, 68.1 MB/s 2024-04-25T10:24:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015011 s, 69.9 MB/s 2024-04-25T10:24:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139489 s, 75.2 MB/s 2024-04-25T10:24:28.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.255 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014164 s, 74.0 MB/s 2024-04-25T10:24:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.271 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144755 s, 72.4 MB/s 2024-04-25T10:24:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.285 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128405 s, 81.7 MB/s 2024-04-25T10:24:28.302 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153163 s, 68.5 MB/s 2024-04-25T10:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139925 s, 74.9 MB/s 2024-04-25T10:24:28.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.334 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133724 s, 78.4 MB/s 2024-04-25T10:24:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142649 s, 73.5 MB/s 2024-04-25T10:24:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146947 s, 71.4 MB/s 2024-04-25T10:24:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147778 s, 71.0 MB/s 2024-04-25T10:24:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136045 s, 77.1 MB/s 2024-04-25T10:24:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013373 s, 78.4 MB/s 2024-04-25T10:24:28.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:28.415 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:28.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179522 s, 58.4 MB/s 2024-04-25T10:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134676 s, 77.9 MB/s 2024-04-25T10:24:28.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.517 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133757 s, 78.4 MB/s 2024-04-25T10:24:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180976 s, 57.9 MB/s 2024-04-25T10:24:28.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135236 s, 77.5 MB/s 2024-04-25T10:24:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135788 s, 77.2 MB/s 2024-04-25T10:24:28.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.584 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139474 s, 75.2 MB/s 2024-04-25T10:24:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130462 s, 80.4 MB/s 2024-04-25T10:24:28.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.614 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135183 s, 77.6 MB/s 2024-04-25T10:24:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157194 s, 66.7 MB/s 2024-04-25T10:24:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138891 s, 75.5 MB/s 2024-04-25T10:24:28.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013793 s, 76.0 MB/s 2024-04-25T10:24:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130548 s, 80.3 MB/s 2024-04-25T10:24:28.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.694 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145796 s, 71.9 MB/s 2024-04-25T10:24:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135086 s, 77.6 MB/s 2024-04-25T10:24:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136225 s, 77.0 MB/s 2024-04-25T10:24:28.740 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.740 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140147 s, 74.8 MB/s 2024-04-25T10:24:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013642 s, 76.9 MB/s 2024-04-25T10:24:28.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.777 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187797 s, 55.8 MB/s 2024-04-25T10:24:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.793 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149822 s, 70.0 MB/s 2024-04-25T10:24:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.810 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144693 s, 72.5 MB/s 2024-04-25T10:24:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131754 s, 79.6 MB/s 2024-04-25T10:24:28.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.841 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142148 s, 73.8 MB/s 2024-04-25T10:24:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138926 s, 75.5 MB/s 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144612 s, 72.5 MB/s 2024-04-25T10:24:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135979 s, 77.1 MB/s 2024-04-25T10:24:28.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134809 s, 77.8 MB/s 2024-04-25T10:24:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013933 s, 75.3 MB/s 2024-04-25T10:24:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129174 s, 81.2 MB/s 2024-04-25T10:24:28.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.950 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142314 s, 73.7 MB/s 2024-04-25T10:24:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01363 s, 76.9 MB/s 2024-04-25T10:24:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138659 s, 75.6 MB/s 2024-04-25T10:24:28.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:28.982 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:28.982 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:28.983 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:24:28.983 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:28.983 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:28.983 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:28.983 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:29.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01379 s, 76.0 MB/s 2024-04-25T10:24:29.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179131 s, 58.5 MB/s 2024-04-25T10:24:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134317 s, 78.1 MB/s 2024-04-25T10:24:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129278 s, 81.1 MB/s 2024-04-25T10:24:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135359 s, 77.5 MB/s 2024-04-25T10:24:29.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.133 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150076 s, 69.9 MB/s 2024-04-25T10:24:29.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134187 s, 78.1 MB/s 2024-04-25T10:24:29.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190315 s, 55.1 MB/s 2024-04-25T10:24:29.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.188 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179077 s, 58.6 MB/s 2024-04-25T10:24:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125525 s, 83.5 MB/s 2024-04-25T10:24:29.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150024 s, 69.9 MB/s 2024-04-25T10:24:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147165 s, 71.3 MB/s 2024-04-25T10:24:29.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141136 s, 74.3 MB/s 2024-04-25T10:24:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136156 s, 77.0 MB/s 2024-04-25T10:24:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133932 s, 78.3 MB/s 2024-04-25T10:24:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148015 s, 70.8 MB/s 2024-04-25T10:24:29.314 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.314 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.315 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139553 s, 75.1 MB/s 2024-04-25T10:24:29.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:29.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138301 s, 75.8 MB/s 2024-04-25T10:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135443 s, 77.4 MB/s 2024-04-25T10:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130177 s, 80.5 MB/s 2024-04-25T10:24:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134573 s, 77.9 MB/s 2024-04-25T10:24:29.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.391 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135096 s, 77.6 MB/s 2024-04-25T10:24:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132352 s, 79.2 MB/s 2024-04-25T10:24:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135513 s, 77.4 MB/s 2024-04-25T10:24:29.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.437 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131849 s, 79.5 MB/s 2024-04-25T10:24:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.453 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143061 s, 73.3 MB/s 2024-04-25T10:24:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125117 s, 83.8 MB/s 2024-04-25T10:24:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133291 s, 78.7 MB/s 2024-04-25T10:24:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154121 s, 68.0 MB/s 2024-04-25T10:24:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133952 s, 78.3 MB/s 2024-04-25T10:24:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.529 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012896 s, 81.3 MB/s 2024-04-25T10:24:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148657 s, 70.5 MB/s 2024-04-25T10:24:29.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:29.548 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:29.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.619 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134339 s, 78.1 MB/s 2024-04-25T10:24:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.634 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132541 s, 79.1 MB/s 2024-04-25T10:24:29.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138734 s, 75.6 MB/s 2024-04-25T10:24:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148152 s, 70.8 MB/s 2024-04-25T10:24:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128581 s, 81.6 MB/s 2024-04-25T10:24:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013495 s, 77.7 MB/s 2024-04-25T10:24:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.717 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191137 s, 54.9 MB/s 2024-04-25T10:24:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137287 s, 76.4 MB/s 2024-04-25T10:24:29.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.748 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137787 s, 76.1 MB/s 2024-04-25T10:24:29.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.768 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177302 s, 59.1 MB/s 2024-04-25T10:24:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.783 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141141 s, 74.3 MB/s 2024-04-25T10:24:29.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182722 s, 57.4 MB/s 2024-04-25T10:24:29.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.821 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158591 s, 66.1 MB/s 2024-04-25T10:24:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013377 s, 78.4 MB/s 2024-04-25T10:24:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131297 s, 79.9 MB/s 2024-04-25T10:24:29.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135797 s, 77.2 MB/s 2024-04-25T10:24:29.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.887 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187102 s, 56.0 MB/s 2024-04-25T10:24:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129366 s, 81.1 MB/s 2024-04-25T10:24:29.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136071 s, 77.1 MB/s 2024-04-25T10:24:29.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.935 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137913 s, 76.0 MB/s 2024-04-25T10:24:29.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144368 s, 72.6 MB/s 2024-04-25T10:24:29.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.964 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128663 s, 81.5 MB/s 2024-04-25T10:24:29.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:29.984 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:29.984 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176622 s, 59.4 MB/s 2024-04-25T10:24:30.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.003 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179698 s, 58.4 MB/s 2024-04-25T10:24:30.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.020 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146913 s, 71.4 MB/s 2024-04-25T10:24:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130234 s, 80.5 MB/s 2024-04-25T10:24:30.049 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130249 s, 80.5 MB/s 2024-04-25T10:24:30.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181039 s, 57.9 MB/s 2024-04-25T10:24:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133635 s, 78.5 MB/s 2024-04-25T10:24:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131927 s, 79.5 MB/s 2024-04-25T10:24:30.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015208 s, 68.9 MB/s 2024-04-25T10:24:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188811 s, 55.5 MB/s 2024-04-25T10:24:30.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:30.139 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:30.207 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.207 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.207 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139598 s, 75.1 MB/s 2024-04-25T10:24:30.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145938 s, 71.9 MB/s 2024-04-25T10:24:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136898 s, 76.6 MB/s 2024-04-25T10:24:30.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.257 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172255 s, 60.9 MB/s 2024-04-25T10:24:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139514 s, 75.2 MB/s 2024-04-25T10:24:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011823 s, 88.7 MB/s 2024-04-25T10:24:30.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.307 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018107 s, 57.9 MB/s 2024-04-25T10:24:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140601 s, 74.6 MB/s 2024-04-25T10:24:30.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.339 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146678 s, 71.5 MB/s 2024-04-25T10:24:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139332 s, 75.3 MB/s 2024-04-25T10:24:30.370 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.370 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.371 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014098 s, 74.4 MB/s 2024-04-25T10:24:30.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.389 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170678 s, 61.4 MB/s 2024-04-25T10:24:30.405 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147423 s, 71.1 MB/s 2024-04-25T10:24:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142246 s, 73.7 MB/s 2024-04-25T10:24:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132164 s, 79.3 MB/s 2024-04-25T10:24:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013083 s, 80.1 MB/s 2024-04-25T10:24:30.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013487 s, 77.7 MB/s 2024-04-25T10:24:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.483 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137271 s, 76.4 MB/s 2024-04-25T10:24:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135131 s, 77.6 MB/s 2024-04-25T10:24:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142458 s, 73.6 MB/s 2024-04-25T10:24:30.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.529 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133845 s, 78.3 MB/s 2024-04-25T10:24:30.544 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.544 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.544 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132266 s, 79.3 MB/s 2024-04-25T10:24:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134384 s, 78.0 MB/s 2024-04-25T10:24:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142819 s, 73.4 MB/s 2024-04-25T10:24:30.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.596 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184076 s, 57.0 MB/s 2024-04-25T10:24:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143503 s, 73.1 MB/s 2024-04-25T10:24:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016622 s, 63.1 MB/s 2024-04-25T10:24:30.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133189 s, 78.7 MB/s 2024-04-25T10:24:30.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.660 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131269 s, 79.9 MB/s 2024-04-25T10:24:30.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.675 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133487 s, 78.6 MB/s 2024-04-25T10:24:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131098 s, 80.0 MB/s 2024-04-25T10:24:30.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.706 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135334 s, 77.5 MB/s 2024-04-25T10:24:30.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:30.707 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:30.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.777 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014846 s, 70.6 MB/s 2024-04-25T10:24:30.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144962 s, 72.3 MB/s 2024-04-25T10:24:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.809 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144273 s, 72.7 MB/s 2024-04-25T10:24:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144162 s, 72.7 MB/s 2024-04-25T10:24:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.840 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137095 s, 76.5 MB/s 2024-04-25T10:24:30.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.859 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172367 s, 60.8 MB/s 2024-04-25T10:24:30.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189986 s, 55.2 MB/s 2024-04-25T10:24:30.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143584 s, 73.0 MB/s 2024-04-25T10:24:30.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.912 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143379 s, 73.1 MB/s 2024-04-25T10:24:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.928 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139216 s, 75.3 MB/s 2024-04-25T10:24:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.943 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134628 s, 77.9 MB/s 2024-04-25T10:24:30.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015561 s, 67.4 MB/s 2024-04-25T10:24:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.977 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150874 s, 69.5 MB/s 2024-04-25T10:24:30.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:30.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:30.994 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149251 s, 70.3 MB/s 2024-04-25T10:24:31.010 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.010 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.010 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142758 s, 73.5 MB/s 2024-04-25T10:24:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145274 s, 72.2 MB/s 2024-04-25T10:24:31.042 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.042 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.042 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137136 s, 76.5 MB/s 2024-04-25T10:24:31.058 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149645 s, 70.1 MB/s 2024-04-25T10:24:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.078 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176603 s, 59.4 MB/s 2024-04-25T10:24:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.093 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129448 s, 81.0 MB/s 2024-04-25T10:24:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137077 s, 76.5 MB/s 2024-04-25T10:24:31.125 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.125 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.125 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151222 s, 69.3 MB/s 2024-04-25T10:24:31.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.141 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135985 s, 77.1 MB/s 2024-04-25T10:24:31.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138286 s, 75.8 MB/s 2024-04-25T10:24:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017002 s, 61.7 MB/s 2024-04-25T10:24:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125189 s, 83.8 MB/s 2024-04-25T10:24:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148144 s, 70.8 MB/s 2024-04-25T10:24:31.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151116 s, 69.4 MB/s 2024-04-25T10:24:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.239 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147392 s, 71.1 MB/s 2024-04-25T10:24:31.255 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.255 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.255 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143805 s, 72.9 MB/s 2024-04-25T10:24:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169832 s, 61.7 MB/s 2024-04-25T10:24:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01533 s, 68.4 MB/s 2024-04-25T10:24:31.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:31.292 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:31.292 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:31.292 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:24:31.292 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:31.293 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:31.293 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:31.293 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145524 s, 72.1 MB/s 2024-04-25T10:24:31.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.380 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146438 s, 71.6 MB/s 2024-04-25T10:24:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01484 s, 70.7 MB/s 2024-04-25T10:24:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.412 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135932 s, 77.1 MB/s 2024-04-25T10:24:31.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.429 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149541 s, 70.1 MB/s 2024-04-25T10:24:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.444 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131533 s, 79.7 MB/s 2024-04-25T10:24:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.460 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140445 s, 74.7 MB/s 2024-04-25T10:24:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.476 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152671 s, 68.7 MB/s 2024-04-25T10:24:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.491 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129791 s, 80.8 MB/s 2024-04-25T10:24:31.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141758 s, 74.0 MB/s 2024-04-25T10:24:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148024 s, 70.8 MB/s 2024-04-25T10:24:31.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.541 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147236 s, 71.2 MB/s 2024-04-25T10:24:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01273 s, 82.4 MB/s 2024-04-25T10:24:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134786 s, 77.8 MB/s 2024-04-25T10:24:31.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.589 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01644 s, 63.8 MB/s 2024-04-25T10:24:31.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.604 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133987 s, 78.3 MB/s 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127401 s, 82.3 MB/s 2024-04-25T10:24:31.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147571 s, 71.1 MB/s 2024-04-25T10:24:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133313 s, 78.7 MB/s 2024-04-25T10:24:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.666 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145235 s, 72.2 MB/s 2024-04-25T10:24:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135511 s, 77.4 MB/s 2024-04-25T10:24:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149082 s, 70.3 MB/s 2024-04-25T10:24:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140538 s, 74.6 MB/s 2024-04-25T10:24:31.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130298 s, 80.5 MB/s 2024-04-25T10:24:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.744 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133935 s, 78.3 MB/s 2024-04-25T10:24:31.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.759 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133862 s, 78.3 MB/s 2024-04-25T10:24:31.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.775 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.775 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135482 s, 77.4 MB/s 2024-04-25T10:24:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.790 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138254 s, 75.8 MB/s 2024-04-25T10:24:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135948 s, 77.1 MB/s 2024-04-25T10:24:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140937 s, 74.4 MB/s 2024-04-25T10:24:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.838 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149744 s, 70.0 MB/s 2024-04-25T10:24:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.854 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137439 s, 76.3 MB/s 2024-04-25T10:24:31.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:31.855 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:31.855 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:31.855 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:24:31.855 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:31.856 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:31.856 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:31.856 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150761 s, 69.6 MB/s 2024-04-25T10:24:31.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.943 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013511 s, 77.6 MB/s 2024-04-25T10:24:31.958 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.959 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.959 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136328 s, 76.9 MB/s 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149378 s, 70.2 MB/s 2024-04-25T10:24:31.990 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:31.991 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:31.991 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136975 s, 76.6 MB/s 2024-04-25T10:24:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.007 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143528 s, 73.1 MB/s 2024-04-25T10:24:32.024 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.024 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.024 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157572 s, 66.5 MB/s 2024-04-25T10:24:32.040 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.040 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.040 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138252 s, 75.8 MB/s 2024-04-25T10:24:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.057 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155261 s, 67.5 MB/s 2024-04-25T10:24:32.072 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133688 s, 78.4 MB/s 2024-04-25T10:24:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.087 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129412 s, 81.0 MB/s 2024-04-25T10:24:32.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155373 s, 67.5 MB/s 2024-04-25T10:24:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150846 s, 69.5 MB/s 2024-04-25T10:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134065 s, 78.2 MB/s 2024-04-25T10:24:32.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.153 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143657 s, 73.0 MB/s 2024-04-25T10:24:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140067 s, 74.9 MB/s 2024-04-25T10:24:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.185 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142238 s, 73.7 MB/s 2024-04-25T10:24:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161807 s, 64.8 MB/s 2024-04-25T10:24:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.219 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151077 s, 69.4 MB/s 2024-04-25T10:24:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176811 s, 59.3 MB/s 2024-04-25T10:24:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138179 s, 75.9 MB/s 2024-04-25T10:24:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131613 s, 79.7 MB/s 2024-04-25T10:24:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135269 s, 77.5 MB/s 2024-04-25T10:24:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.300 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137942 s, 76.0 MB/s 2024-04-25T10:24:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141167 s, 74.3 MB/s 2024-04-25T10:24:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179584 s, 58.4 MB/s 2024-04-25T10:24:32.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141555 s, 74.1 MB/s 2024-04-25T10:24:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153237 s, 68.4 MB/s 2024-04-25T10:24:32.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130321 s, 80.5 MB/s 2024-04-25T10:24:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134254 s, 78.1 MB/s 2024-04-25T10:24:32.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138475 s, 75.7 MB/s 2024-04-25T10:24:32.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139196 s, 75.3 MB/s 2024-04-25T10:24:32.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:32.432 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:32.432 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:32.433 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:24:32.433 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:32.433 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:32.433 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:32.433 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:32.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191238 s, 54.8 MB/s 2024-04-25T10:24:32.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.531 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144008 s, 72.8 MB/s 2024-04-25T10:24:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132711 s, 79.0 MB/s 2024-04-25T10:24:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012971 s, 80.8 MB/s 2024-04-25T10:24:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.576 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133975 s, 78.3 MB/s 2024-04-25T10:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.591 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132749 s, 79.0 MB/s 2024-04-25T10:24:32.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135323 s, 77.5 MB/s 2024-04-25T10:24:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013276 s, 79.0 MB/s 2024-04-25T10:24:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137435 s, 76.3 MB/s 2024-04-25T10:24:32.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013377 s, 78.4 MB/s 2024-04-25T10:24:32.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.672 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176432 s, 59.4 MB/s 2024-04-25T10:24:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.686 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129391 s, 81.0 MB/s 2024-04-25T10:24:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.701 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133753 s, 78.4 MB/s 2024-04-25T10:24:32.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150513 s, 69.7 MB/s 2024-04-25T10:24:32.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.738 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176863 s, 59.3 MB/s 2024-04-25T10:24:32.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136745 s, 76.7 MB/s 2024-04-25T10:24:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.770 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144546 s, 72.5 MB/s 2024-04-25T10:24:32.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148764 s, 70.5 MB/s 2024-04-25T10:24:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01752 s, 59.9 MB/s 2024-04-25T10:24:32.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188158 s, 55.7 MB/s 2024-04-25T10:24:32.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162173 s, 64.7 MB/s 2024-04-25T10:24:32.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131705 s, 79.6 MB/s 2024-04-25T10:24:32.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192568 s, 54.5 MB/s 2024-04-25T10:24:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.897 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014068 s, 74.5 MB/s 2024-04-25T10:24:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175241 s, 59.8 MB/s 2024-04-25T10:24:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.931 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133389 s, 78.6 MB/s 2024-04-25T10:24:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189742 s, 55.3 MB/s 2024-04-25T10:24:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144763 s, 72.4 MB/s 2024-04-25T10:24:32.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:32.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:32.985 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148893 s, 70.4 MB/s 2024-04-25T10:24:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01645 s, 63.7 MB/s 2024-04-25T10:24:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154563 s, 67.8 MB/s 2024-04-25T10:24:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138961 s, 75.5 MB/s 2024-04-25T10:24:33.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:33.038 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:33.038 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:33.039 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:24:33.039 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:33.039 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:33.039 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:33.039 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:33.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135852 s, 77.2 MB/s 2024-04-25T10:24:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140931 s, 74.4 MB/s 2024-04-25T10:24:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137904 s, 76.0 MB/s 2024-04-25T10:24:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186559 s, 56.2 MB/s 2024-04-25T10:24:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126355 s, 83.0 MB/s 2024-04-25T10:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144236 s, 72.7 MB/s 2024-04-25T10:24:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144497 s, 72.6 MB/s 2024-04-25T10:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137435 s, 76.3 MB/s 2024-04-25T10:24:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150617 s, 69.6 MB/s 2024-04-25T10:24:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190545 s, 55.0 MB/s 2024-04-25T10:24:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147943 s, 70.9 MB/s 2024-04-25T10:24:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147945 s, 70.9 MB/s 2024-04-25T10:24:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.309 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153976 s, 68.1 MB/s 2024-04-25T10:24:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139273 s, 75.3 MB/s 2024-04-25T10:24:33.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152136 s, 68.9 MB/s 2024-04-25T10:24:33.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135406 s, 77.4 MB/s 2024-04-25T10:24:33.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.375 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149271 s, 70.2 MB/s 2024-04-25T10:24:33.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177119 s, 59.2 MB/s 2024-04-25T10:24:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130603 s, 80.3 MB/s 2024-04-25T10:24:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137699 s, 76.1 MB/s 2024-04-25T10:24:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137178 s, 76.4 MB/s 2024-04-25T10:24:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.456 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150424 s, 69.7 MB/s 2024-04-25T10:24:33.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.476 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184152 s, 56.9 MB/s 2024-04-25T10:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.491 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013367 s, 78.4 MB/s 2024-04-25T10:24:33.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141565 s, 74.1 MB/s 2024-04-25T10:24:33.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.523 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01332 s, 78.7 MB/s 2024-04-25T10:24:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148347 s, 70.7 MB/s 2024-04-25T10:24:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133668 s, 78.4 MB/s 2024-04-25T10:24:33.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133831 s, 78.4 MB/s 2024-04-25T10:24:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.589 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173085 s, 60.6 MB/s 2024-04-25T10:24:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.605 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146396 s, 71.6 MB/s 2024-04-25T10:24:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153076 s, 68.5 MB/s 2024-04-25T10:24:33.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:33.623 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:33.624 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:33.624 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:24:33.624 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:33.624 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:33.624 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:33.624 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131366 s, 79.8 MB/s 2024-04-25T10:24:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.714 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141287 s, 74.2 MB/s 2024-04-25T10:24:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132449 s, 79.2 MB/s 2024-04-25T10:24:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015095 s, 69.5 MB/s 2024-04-25T10:24:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.766 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176001 s, 59.6 MB/s 2024-04-25T10:24:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.782 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148047 s, 70.8 MB/s 2024-04-25T10:24:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175929 s, 59.6 MB/s 2024-04-25T10:24:33.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.817 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137385 s, 76.3 MB/s 2024-04-25T10:24:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143305 s, 73.2 MB/s 2024-04-25T10:24:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152062 s, 69.0 MB/s 2024-04-25T10:24:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138759 s, 75.6 MB/s 2024-04-25T10:24:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169241 s, 62.0 MB/s 2024-04-25T10:24:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.901 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013529 s, 77.5 MB/s 2024-04-25T10:24:33.920 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.921 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180084 s, 58.2 MB/s 2024-04-25T10:24:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.937 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145285 s, 72.2 MB/s 2024-04-25T10:24:33.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.953 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148982 s, 70.4 MB/s 2024-04-25T10:24:33.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.969 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139896 s, 75.0 MB/s 2024-04-25T10:24:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138572 s, 75.7 MB/s 2024-04-25T10:24:34.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.001 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143455 s, 73.1 MB/s 2024-04-25T10:24:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150034 s, 69.9 MB/s 2024-04-25T10:24:34.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143168 s, 73.2 MB/s 2024-04-25T10:24:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176387 s, 59.4 MB/s 2024-04-25T10:24:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137361 s, 76.3 MB/s 2024-04-25T10:24:34.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137236 s, 76.4 MB/s 2024-04-25T10:24:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134766 s, 77.8 MB/s 2024-04-25T10:24:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151074 s, 69.4 MB/s 2024-04-25T10:24:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188055 s, 55.8 MB/s 2024-04-25T10:24:34.152 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135897 s, 77.2 MB/s 2024-04-25T10:24:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146622 s, 71.5 MB/s 2024-04-25T10:24:34.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131889 s, 79.5 MB/s 2024-04-25T10:24:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184319 s, 56.9 MB/s 2024-04-25T10:24:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143982 s, 72.8 MB/s 2024-04-25T10:24:34.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:34.222 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:34.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.296 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192265 s, 54.5 MB/s 2024-04-25T10:24:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148801 s, 70.5 MB/s 2024-04-25T10:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015319 s, 68.4 MB/s 2024-04-25T10:24:34.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.345 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140773 s, 74.5 MB/s 2024-04-25T10:24:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144778 s, 72.4 MB/s 2024-04-25T10:24:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.377 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135487 s, 77.4 MB/s 2024-04-25T10:24:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186881 s, 56.1 MB/s 2024-04-25T10:24:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138699 s, 75.6 MB/s 2024-04-25T10:24:34.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.429 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146223 s, 71.7 MB/s 2024-04-25T10:24:34.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.444 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135282 s, 77.5 MB/s 2024-04-25T10:24:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129407 s, 81.0 MB/s 2024-04-25T10:24:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140041 s, 74.9 MB/s 2024-04-25T10:24:34.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.492 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152836 s, 68.6 MB/s 2024-04-25T10:24:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.509 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151933 s, 69.0 MB/s 2024-04-25T10:24:34.526 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.526 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.526 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145547 s, 72.0 MB/s 2024-04-25T10:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.542 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148662 s, 70.5 MB/s 2024-04-25T10:24:34.557 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.557 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.557 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137702 s, 76.1 MB/s 2024-04-25T10:24:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.578 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182301 s, 57.5 MB/s 2024-04-25T10:24:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013713 s, 76.5 MB/s 2024-04-25T10:24:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138335 s, 75.8 MB/s 2024-04-25T10:24:34.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180444 s, 58.1 MB/s 2024-04-25T10:24:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.644 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138972 s, 75.5 MB/s 2024-04-25T10:24:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148432 s, 70.6 MB/s 2024-04-25T10:24:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136604 s, 76.8 MB/s 2024-04-25T10:24:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149606 s, 70.1 MB/s 2024-04-25T10:24:34.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146497 s, 71.6 MB/s 2024-04-25T10:24:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.730 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190044 s, 55.2 MB/s 2024-04-25T10:24:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128412 s, 81.7 MB/s 2024-04-25T10:24:34.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.762 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150511 s, 69.7 MB/s 2024-04-25T10:24:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132146 s, 79.3 MB/s 2024-04-25T10:24:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.793 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.794 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150719 s, 69.6 MB/s 2024-04-25T10:24:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182798 s, 57.4 MB/s 2024-04-25T10:24:34.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:34.815 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:34.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.887 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180129 s, 58.2 MB/s 2024-04-25T10:24:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015165 s, 69.1 MB/s 2024-04-25T10:24:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175817 s, 59.6 MB/s 2024-04-25T10:24:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.939 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137812 s, 76.1 MB/s 2024-04-25T10:24:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134205 s, 78.1 MB/s 2024-04-25T10:24:34.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136013 s, 77.1 MB/s 2024-04-25T10:24:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140704 s, 74.5 MB/s 2024-04-25T10:24:35.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.000 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130701 s, 80.2 MB/s 2024-04-25T10:24:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154147 s, 68.0 MB/s 2024-04-25T10:24:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.038 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183837 s, 57.0 MB/s 2024-04-25T10:24:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138537 s, 75.7 MB/s 2024-04-25T10:24:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142381 s, 73.6 MB/s 2024-04-25T10:24:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139242 s, 75.3 MB/s 2024-04-25T10:24:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013297 s, 78.9 MB/s 2024-04-25T10:24:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166966 s, 62.8 MB/s 2024-04-25T10:24:35.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135511 s, 77.4 MB/s 2024-04-25T10:24:35.149 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.150 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.150 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133715 s, 78.4 MB/s 2024-04-25T10:24:35.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170445 s, 61.5 MB/s 2024-04-25T10:24:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155411 s, 67.5 MB/s 2024-04-25T10:24:35.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131427 s, 79.8 MB/s 2024-04-25T10:24:35.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.218 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153786 s, 68.2 MB/s 2024-04-25T10:24:35.234 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148107 s, 70.8 MB/s 2024-04-25T10:24:35.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.251 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141543 s, 74.1 MB/s 2024-04-25T10:24:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.266 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132754 s, 79.0 MB/s 2024-04-25T10:24:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133197 s, 78.7 MB/s 2024-04-25T10:24:35.297 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.298 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144064 s, 72.8 MB/s 2024-04-25T10:24:35.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.314 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.314 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014686 s, 71.4 MB/s 2024-04-25T10:24:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164829 s, 63.6 MB/s 2024-04-25T10:24:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.349 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144412 s, 72.6 MB/s 2024-04-25T10:24:35.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.365 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147984 s, 70.9 MB/s 2024-04-25T10:24:35.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.381 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144327 s, 72.7 MB/s 2024-04-25T10:24:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.396 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133551 s, 78.5 MB/s 2024-04-25T10:24:35.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:35.398 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:35.398 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:35.399 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:24:35.399 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:35.399 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:35.399 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:35.399 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141145 s, 74.3 MB/s 2024-04-25T10:24:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.482 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131959 s, 79.5 MB/s 2024-04-25T10:24:35.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146373 s, 71.6 MB/s 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.516 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161861 s, 64.8 MB/s 2024-04-25T10:24:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173933 s, 60.3 MB/s 2024-04-25T10:24:35.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149373 s, 70.2 MB/s 2024-04-25T10:24:35.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136349 s, 76.9 MB/s 2024-04-25T10:24:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137566 s, 76.2 MB/s 2024-04-25T10:24:35.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.598 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130637 s, 80.3 MB/s 2024-04-25T10:24:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143579 s, 73.0 MB/s 2024-04-25T10:24:35.629 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134104 s, 78.2 MB/s 2024-04-25T10:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136902 s, 76.6 MB/s 2024-04-25T10:24:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179111 s, 58.5 MB/s 2024-04-25T10:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135003 s, 77.7 MB/s 2024-04-25T10:24:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.695 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132152 s, 79.3 MB/s 2024-04-25T10:24:35.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.712 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014664 s, 71.5 MB/s 2024-04-25T10:24:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126644 s, 82.8 MB/s 2024-04-25T10:24:35.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.742 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141237 s, 74.2 MB/s 2024-04-25T10:24:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177105 s, 59.2 MB/s 2024-04-25T10:24:35.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.777 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133905 s, 78.3 MB/s 2024-04-25T10:24:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136736 s, 76.7 MB/s 2024-04-25T10:24:35.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.808 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138631 s, 75.6 MB/s 2024-04-25T10:24:35.823 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.823 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.823 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139529 s, 75.2 MB/s 2024-04-25T10:24:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145727 s, 72.0 MB/s 2024-04-25T10:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139274 s, 75.3 MB/s 2024-04-25T10:24:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.871 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137805 s, 76.1 MB/s 2024-04-25T10:24:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.888 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147599 s, 71.0 MB/s 2024-04-25T10:24:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.903 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133335 s, 78.6 MB/s 2024-04-25T10:24:35.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177257 s, 59.2 MB/s 2024-04-25T10:24:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.937 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130693 s, 80.2 MB/s 2024-04-25T10:24:35.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.952 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133251 s, 78.7 MB/s 2024-04-25T10:24:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150745 s, 69.6 MB/s 2024-04-25T10:24:35.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:35.971 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:36.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.039 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140987 s, 74.4 MB/s 2024-04-25T10:24:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.055 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.055 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134501 s, 78.0 MB/s 2024-04-25T10:24:36.072 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.072 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.072 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154175 s, 68.0 MB/s 2024-04-25T10:24:36.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.087 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129872 s, 80.7 MB/s 2024-04-25T10:24:36.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.103 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150836 s, 69.5 MB/s 2024-04-25T10:24:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145993 s, 71.8 MB/s 2024-04-25T10:24:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015166 s, 69.1 MB/s 2024-04-25T10:24:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143393 s, 73.1 MB/s 2024-04-25T10:24:36.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.170 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152543 s, 68.7 MB/s 2024-04-25T10:24:36.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.185 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133711 s, 78.4 MB/s 2024-04-25T10:24:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144144 s, 72.7 MB/s 2024-04-25T10:24:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135658 s, 77.3 MB/s 2024-04-25T10:24:36.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.233 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138977 s, 75.4 MB/s 2024-04-25T10:24:36.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.248 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129601 s, 80.9 MB/s 2024-04-25T10:24:36.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.265 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151086 s, 69.4 MB/s 2024-04-25T10:24:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135306 s, 77.5 MB/s 2024-04-25T10:24:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.296 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143654 s, 73.0 MB/s 2024-04-25T10:24:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133863 s, 78.3 MB/s 2024-04-25T10:24:36.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.332 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188835 s, 55.5 MB/s 2024-04-25T10:24:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124495 s, 84.2 MB/s 2024-04-25T10:24:36.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.366 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179661 s, 58.4 MB/s 2024-04-25T10:24:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145568 s, 72.0 MB/s 2024-04-25T10:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141278 s, 74.2 MB/s 2024-04-25T10:24:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148226 s, 70.7 MB/s 2024-04-25T10:24:36.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.435 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187691 s, 55.9 MB/s 2024-04-25T10:24:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151705 s, 69.1 MB/s 2024-04-25T10:24:36.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132799 s, 79.0 MB/s 2024-04-25T10:24:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131824 s, 79.5 MB/s 2024-04-25T10:24:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138775 s, 75.6 MB/s 2024-04-25T10:24:36.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.514 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145991 s, 71.8 MB/s 2024-04-25T10:24:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.532 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167569 s, 62.6 MB/s 2024-04-25T10:24:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149867 s, 70.0 MB/s 2024-04-25T10:24:36.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:36.551 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133582 s, 78.5 MB/s 2024-04-25T10:24:36.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135255 s, 77.5 MB/s 2024-04-25T10:24:36.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169804 s, 61.8 MB/s 2024-04-25T10:24:36.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014928 s, 70.2 MB/s 2024-04-25T10:24:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.687 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143879 s, 72.9 MB/s 2024-04-25T10:24:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01344 s, 78.0 MB/s 2024-04-25T10:24:36.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.719 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014183 s, 73.9 MB/s 2024-04-25T10:24:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.735 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146218 s, 71.7 MB/s 2024-04-25T10:24:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171579 s, 61.1 MB/s 2024-04-25T10:24:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.766 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113119 s, 92.7 MB/s 2024-04-25T10:24:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014486 s, 72.4 MB/s 2024-04-25T10:24:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.798 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136389 s, 76.9 MB/s 2024-04-25T10:24:36.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.815 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151529 s, 69.2 MB/s 2024-04-25T10:24:36.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.832 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142867 s, 73.4 MB/s 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.847 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142071 s, 73.8 MB/s 2024-04-25T10:24:36.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.863 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145536 s, 72.0 MB/s 2024-04-25T10:24:36.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147028 s, 71.3 MB/s 2024-04-25T10:24:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144641 s, 72.5 MB/s 2024-04-25T10:24:36.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.913 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139207 s, 75.3 MB/s 2024-04-25T10:24:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136323 s, 76.9 MB/s 2024-04-25T10:24:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193034 s, 54.3 MB/s 2024-04-25T10:24:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153261 s, 68.4 MB/s 2024-04-25T10:24:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.981 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135138 s, 77.6 MB/s 2024-04-25T10:24:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135151 s, 77.6 MB/s 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.011 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132199 s, 79.3 MB/s 2024-04-25T10:24:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134521 s, 77.9 MB/s 2024-04-25T10:24:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184124 s, 56.9 MB/s 2024-04-25T10:24:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.065 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160291 s, 65.4 MB/s 2024-04-25T10:24:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.080 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137841 s, 76.1 MB/s 2024-04-25T10:24:37.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.096 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136672 s, 76.7 MB/s 2024-04-25T10:24:37.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.111 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131141 s, 80.0 MB/s 2024-04-25T10:24:37.127 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.128 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.128 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150352 s, 69.7 MB/s 2024-04-25T10:24:37.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:37.129 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:37.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.198 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132686 s, 79.0 MB/s 2024-04-25T10:24:37.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.219 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191421 s, 54.8 MB/s 2024-04-25T10:24:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152401 s, 68.8 MB/s 2024-04-25T10:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.257 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189245 s, 55.4 MB/s 2024-04-25T10:24:37.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.272 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137679 s, 76.2 MB/s 2024-04-25T10:24:37.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.288 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137224 s, 76.4 MB/s 2024-04-25T10:24:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147362 s, 71.2 MB/s 2024-04-25T10:24:37.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.321 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015294 s, 68.6 MB/s 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134352 s, 78.0 MB/s 2024-04-25T10:24:37.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135939 s, 77.1 MB/s 2024-04-25T10:24:37.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142128 s, 73.8 MB/s 2024-04-25T10:24:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154038 s, 68.1 MB/s 2024-04-25T10:24:37.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.400 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129788 s, 80.8 MB/s 2024-04-25T10:24:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199484 s, 52.6 MB/s 2024-04-25T10:24:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.442 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190436 s, 55.1 MB/s 2024-04-25T10:24:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147667 s, 71.0 MB/s 2024-04-25T10:24:37.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.474 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136049 s, 77.1 MB/s 2024-04-25T10:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.491 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149008 s, 70.4 MB/s 2024-04-25T10:24:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190955 s, 54.9 MB/s 2024-04-25T10:24:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134731 s, 77.8 MB/s 2024-04-25T10:24:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129939 s, 80.7 MB/s 2024-04-25T10:24:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147795 s, 70.9 MB/s 2024-04-25T10:24:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151026 s, 69.4 MB/s 2024-04-25T10:24:37.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.591 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013668 s, 76.7 MB/s 2024-04-25T10:24:37.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140584 s, 74.6 MB/s 2024-04-25T10:24:37.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.622 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137981 s, 76.0 MB/s 2024-04-25T10:24:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142783 s, 73.4 MB/s 2024-04-25T10:24:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.655 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141464 s, 74.1 MB/s 2024-04-25T10:24:37.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.675 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189716 s, 55.3 MB/s 2024-04-25T10:24:37.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177408 s, 59.1 MB/s 2024-04-25T10:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.712 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015192 s, 69.0 MB/s 2024-04-25T10:24:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013433 s, 78.1 MB/s 2024-04-25T10:24:37.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:37.728 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:37.729 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:37.729 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:24:37.729 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:37.729 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:37.729 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:37.729 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:37.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.798 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134051 s, 78.2 MB/s 2024-04-25T10:24:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141894 s, 73.9 MB/s 2024-04-25T10:24:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.834 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179261 s, 58.5 MB/s 2024-04-25T10:24:37.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150502 s, 69.7 MB/s 2024-04-25T10:24:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133926 s, 78.3 MB/s 2024-04-25T10:24:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153524 s, 68.3 MB/s 2024-04-25T10:24:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.899 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135376 s, 77.5 MB/s 2024-04-25T10:24:37.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151208 s, 69.3 MB/s 2024-04-25T10:24:37.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.933 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152291 s, 68.9 MB/s 2024-04-25T10:24:37.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.948 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013038 s, 80.4 MB/s 2024-04-25T10:24:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141461 s, 74.1 MB/s 2024-04-25T10:24:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:37.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:37.984 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175685 s, 59.7 MB/s 2024-04-25T10:24:38.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.001 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153544 s, 68.3 MB/s 2024-04-25T10:24:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.023 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189817 s, 55.2 MB/s 2024-04-25T10:24:38.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.042 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164816 s, 63.6 MB/s 2024-04-25T10:24:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160775 s, 65.2 MB/s 2024-04-25T10:24:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145763 s, 71.9 MB/s 2024-04-25T10:24:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.092 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138475 s, 75.7 MB/s 2024-04-25T10:24:38.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.106 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128755 s, 81.4 MB/s 2024-04-25T10:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135181 s, 77.6 MB/s 2024-04-25T10:24:38.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136516 s, 76.8 MB/s 2024-04-25T10:24:38.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.153 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137927 s, 76.0 MB/s 2024-04-25T10:24:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155265 s, 67.5 MB/s 2024-04-25T10:24:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135376 s, 77.5 MB/s 2024-04-25T10:24:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136503 s, 76.8 MB/s 2024-04-25T10:24:38.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.217 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136828 s, 76.6 MB/s 2024-04-25T10:24:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.233 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.234 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148648 s, 70.5 MB/s 2024-04-25T10:24:38.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.249 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137097 s, 76.5 MB/s 2024-04-25T10:24:38.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181053 s, 57.9 MB/s 2024-04-25T10:24:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.283 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132753 s, 79.0 MB/s 2024-04-25T10:24:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185546 s, 56.5 MB/s 2024-04-25T10:24:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152691 s, 68.7 MB/s 2024-04-25T10:24:38.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:38.323 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:38.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.400 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185813 s, 56.4 MB/s 2024-04-25T10:24:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171334 s, 61.2 MB/s 2024-04-25T10:24:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.439 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187421 s, 55.9 MB/s 2024-04-25T10:24:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.455 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136792 s, 76.7 MB/s 2024-04-25T10:24:38.469 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.470 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013382 s, 78.4 MB/s 2024-04-25T10:24:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151208 s, 69.3 MB/s 2024-04-25T10:24:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180245 s, 58.2 MB/s 2024-04-25T10:24:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01363 s, 76.9 MB/s 2024-04-25T10:24:38.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014336 s, 73.1 MB/s 2024-04-25T10:24:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132064 s, 79.4 MB/s 2024-04-25T10:24:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117 s, 89.6 MB/s 2024-04-25T10:24:38.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150648 s, 69.6 MB/s 2024-04-25T10:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145702 s, 72.0 MB/s 2024-04-25T10:24:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.615 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01373 s, 76.4 MB/s 2024-04-25T10:24:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131522 s, 79.7 MB/s 2024-04-25T10:24:38.650 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.650 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.650 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179875 s, 58.3 MB/s 2024-04-25T10:24:38.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152265 s, 68.9 MB/s 2024-04-25T10:24:38.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143832 s, 72.9 MB/s 2024-04-25T10:24:38.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131523 s, 79.7 MB/s 2024-04-25T10:24:38.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181268 s, 57.8 MB/s 2024-04-25T10:24:38.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.734 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137115 s, 76.5 MB/s 2024-04-25T10:24:38.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134909 s, 77.7 MB/s 2024-04-25T10:24:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136037 s, 77.1 MB/s 2024-04-25T10:24:38.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.781 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147832 s, 70.9 MB/s 2024-04-25T10:24:38.794 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123621 s, 84.8 MB/s 2024-04-25T10:24:38.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132293 s, 79.3 MB/s 2024-04-25T10:24:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.826 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143927 s, 72.9 MB/s 2024-04-25T10:24:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136655 s, 76.7 MB/s 2024-04-25T10:24:38.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.857 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142504 s, 73.6 MB/s 2024-04-25T10:24:38.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.874 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151382 s, 69.3 MB/s 2024-04-25T10:24:38.895 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.895 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.895 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185091 s, 56.7 MB/s 2024-04-25T10:24:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.910 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135261 s, 77.5 MB/s 2024-04-25T10:24:38.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:38.911 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:38.912 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:38.912 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:24:38.912 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:38.912 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:38.912 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:38.912 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:38.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:38.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:38.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141155 s, 74.3 MB/s 2024-04-25T10:24:39.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.004 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145918 s, 71.9 MB/s 2024-04-25T10:24:39.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.019 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013056 s, 80.3 MB/s 2024-04-25T10:24:39.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.035 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143145 s, 73.3 MB/s 2024-04-25T10:24:39.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.050 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131896 s, 79.5 MB/s 2024-04-25T10:24:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.065 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013388 s, 78.3 MB/s 2024-04-25T10:24:39.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.084 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173365 s, 60.5 MB/s 2024-04-25T10:24:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118985 s, 88.1 MB/s 2024-04-25T10:24:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.113 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139031 s, 75.4 MB/s 2024-04-25T10:24:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.133 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173289 s, 60.5 MB/s 2024-04-25T10:24:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013639 s, 76.9 MB/s 2024-04-25T10:24:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.163 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013101 s, 80.0 MB/s 2024-04-25T10:24:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134599 s, 77.9 MB/s 2024-04-25T10:24:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134635 s, 77.9 MB/s 2024-04-25T10:24:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.210 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145944 s, 71.8 MB/s 2024-04-25T10:24:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145835 s, 71.9 MB/s 2024-04-25T10:24:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138077 s, 75.9 MB/s 2024-04-25T10:24:39.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141263 s, 74.2 MB/s 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.279 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188399 s, 55.7 MB/s 2024-04-25T10:24:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138938 s, 75.5 MB/s 2024-04-25T10:24:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135818 s, 77.2 MB/s 2024-04-25T10:24:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148356 s, 70.7 MB/s 2024-04-25T10:24:39.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013179 s, 79.6 MB/s 2024-04-25T10:24:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135353 s, 77.5 MB/s 2024-04-25T10:24:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.373 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141219 s, 74.3 MB/s 2024-04-25T10:24:39.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.390 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147069 s, 71.3 MB/s 2024-04-25T10:24:39.405 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143292 s, 73.2 MB/s 2024-04-25T10:24:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144399 s, 72.6 MB/s 2024-04-25T10:24:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130549 s, 80.3 MB/s 2024-04-25T10:24:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137024 s, 76.5 MB/s 2024-04-25T10:24:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142925 s, 73.4 MB/s 2024-04-25T10:24:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.483 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132381 s, 79.2 MB/s 2024-04-25T10:24:39.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:39.485 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:39.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.554 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134122 s, 78.2 MB/s 2024-04-25T10:24:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135676 s, 77.3 MB/s 2024-04-25T10:24:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.589 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018014 s, 58.2 MB/s 2024-04-25T10:24:39.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.604 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133439 s, 78.6 MB/s 2024-04-25T10:24:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.620 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135055 s, 77.6 MB/s 2024-04-25T10:24:39.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134223 s, 78.1 MB/s 2024-04-25T10:24:39.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149702 s, 70.0 MB/s 2024-04-25T10:24:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133852 s, 78.3 MB/s 2024-04-25T10:24:39.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144405 s, 72.6 MB/s 2024-04-25T10:24:39.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013482 s, 77.8 MB/s 2024-04-25T10:24:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015077 s, 69.5 MB/s 2024-04-25T10:24:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013575 s, 77.2 MB/s 2024-04-25T10:24:39.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137749 s, 76.1 MB/s 2024-04-25T10:24:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128289 s, 81.7 MB/s 2024-04-25T10:24:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150545 s, 69.7 MB/s 2024-04-25T10:24:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.793 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132869 s, 78.9 MB/s 2024-04-25T10:24:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.808 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135306 s, 77.5 MB/s 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.825 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146757 s, 71.4 MB/s 2024-04-25T10:24:39.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.841 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140026 s, 74.9 MB/s 2024-04-25T10:24:39.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.856 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133039 s, 78.8 MB/s 2024-04-25T10:24:39.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.871 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135792 s, 77.2 MB/s 2024-04-25T10:24:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.892 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183346 s, 57.2 MB/s 2024-04-25T10:24:39.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139138 s, 75.4 MB/s 2024-04-25T10:24:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141986 s, 73.9 MB/s 2024-04-25T10:24:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118136 s, 88.8 MB/s 2024-04-25T10:24:39.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.953 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141939 s, 73.9 MB/s 2024-04-25T10:24:39.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.968 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137587 s, 76.2 MB/s 2024-04-25T10:24:39.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184739 s, 56.8 MB/s 2024-04-25T10:24:40.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148912 s, 70.4 MB/s 2024-04-25T10:24:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139372 s, 75.2 MB/s 2024-04-25T10:24:40.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.041 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182819 s, 57.4 MB/s 2024-04-25T10:24:40.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.057 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139476 s, 75.2 MB/s 2024-04-25T10:24:40.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:40.058 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:40.058 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:40.058 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:24:40.058 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:40.059 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:40.059 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:40.059 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:40.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.126 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134056 s, 78.2 MB/s 2024-04-25T10:24:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.146 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017981 s, 58.3 MB/s 2024-04-25T10:24:40.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.162 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145568 s, 72.0 MB/s 2024-04-25T10:24:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.182 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185001 s, 56.7 MB/s 2024-04-25T10:24:40.199 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.200 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154532 s, 67.9 MB/s 2024-04-25T10:24:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014525 s, 72.2 MB/s 2024-04-25T10:24:40.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143044 s, 73.3 MB/s 2024-04-25T10:24:40.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.248 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141096 s, 74.3 MB/s 2024-04-25T10:24:40.264 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.265 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152839 s, 68.6 MB/s 2024-04-25T10:24:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.285 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184223 s, 56.9 MB/s 2024-04-25T10:24:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.301 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139252 s, 75.3 MB/s 2024-04-25T10:24:40.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137448 s, 76.3 MB/s 2024-04-25T10:24:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181903 s, 57.6 MB/s 2024-04-25T10:24:40.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01441 s, 72.8 MB/s 2024-04-25T10:24:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142268 s, 73.7 MB/s 2024-04-25T10:24:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132429 s, 79.2 MB/s 2024-04-25T10:24:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132523 s, 79.1 MB/s 2024-04-25T10:24:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.418 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181499 s, 57.8 MB/s 2024-04-25T10:24:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.434 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142018 s, 73.8 MB/s 2024-04-25T10:24:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.450 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137142 s, 76.5 MB/s 2024-04-25T10:24:40.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.466 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148179 s, 70.8 MB/s 2024-04-25T10:24:40.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.481 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136055 s, 77.1 MB/s 2024-04-25T10:24:40.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157499 s, 66.6 MB/s 2024-04-25T10:24:40.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.519 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184735 s, 56.8 MB/s 2024-04-25T10:24:40.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.537 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151112 s, 69.4 MB/s 2024-04-25T10:24:40.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149483 s, 70.1 MB/s 2024-04-25T10:24:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.573 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180553 s, 58.1 MB/s 2024-04-25T10:24:40.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.592 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177122 s, 59.2 MB/s 2024-04-25T10:24:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151392 s, 69.3 MB/s 2024-04-25T10:24:40.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144133 s, 72.8 MB/s 2024-04-25T10:24:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.642 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150231 s, 69.8 MB/s 2024-04-25T10:24:40.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.662 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180296 s, 58.2 MB/s 2024-04-25T10:24:40.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:40.663 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:40.663 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:40.663 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:24:40.663 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:40.664 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:40.664 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:40.664 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:40.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.738 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132532 s, 79.1 MB/s 2024-04-25T10:24:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012911 s, 81.2 MB/s 2024-04-25T10:24:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175274 s, 59.8 MB/s 2024-04-25T10:24:40.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.790 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157969 s, 66.4 MB/s 2024-04-25T10:24:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.805 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132629 s, 79.1 MB/s 2024-04-25T10:24:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134916 s, 77.7 MB/s 2024-04-25T10:24:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144163 s, 72.7 MB/s 2024-04-25T10:24:40.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.852 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133266 s, 78.7 MB/s 2024-04-25T10:24:40.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133509 s, 78.5 MB/s 2024-04-25T10:24:40.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013718 s, 76.4 MB/s 2024-04-25T10:24:40.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.899 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136793 s, 76.7 MB/s 2024-04-25T10:24:40.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.913 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131422 s, 79.8 MB/s 2024-04-25T10:24:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.930 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141154 s, 74.3 MB/s 2024-04-25T10:24:40.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.945 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142379 s, 73.6 MB/s 2024-04-25T10:24:40.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128868 s, 81.4 MB/s 2024-04-25T10:24:40.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178119 s, 58.9 MB/s 2024-04-25T10:24:40.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:40.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:40.996 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144407 s, 72.6 MB/s 2024-04-25T10:24:41.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.013 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156572 s, 67.0 MB/s 2024-04-25T10:24:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150761 s, 69.6 MB/s 2024-04-25T10:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.046 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138976 s, 75.5 MB/s 2024-04-25T10:24:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.067 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191849 s, 54.7 MB/s 2024-04-25T10:24:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138835 s, 75.5 MB/s 2024-04-25T10:24:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133997 s, 78.3 MB/s 2024-04-25T10:24:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136202 s, 77.0 MB/s 2024-04-25T10:24:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150473 s, 69.7 MB/s 2024-04-25T10:24:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153914 s, 68.1 MB/s 2024-04-25T10:24:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018963 s, 55.3 MB/s 2024-04-25T10:24:41.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137159 s, 76.4 MB/s 2024-04-25T10:24:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146899 s, 71.4 MB/s 2024-04-25T10:24:41.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.217 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148304 s, 70.7 MB/s 2024-04-25T10:24:41.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.231 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132025 s, 79.4 MB/s 2024-04-25T10:24:41.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.247 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013759 s, 76.2 MB/s 2024-04-25T10:24:41.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:41.249 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:41.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.322 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127901 s, 82.0 MB/s 2024-04-25T10:24:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.339 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153135 s, 68.5 MB/s 2024-04-25T10:24:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167802 s, 62.5 MB/s 2024-04-25T10:24:41.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141297 s, 74.2 MB/s 2024-04-25T10:24:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189541 s, 55.3 MB/s 2024-04-25T10:24:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.410 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133698 s, 78.4 MB/s 2024-04-25T10:24:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178571 s, 58.7 MB/s 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.444 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132886 s, 78.9 MB/s 2024-04-25T10:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141065 s, 74.3 MB/s 2024-04-25T10:24:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125938 s, 83.3 MB/s 2024-04-25T10:24:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187946 s, 55.8 MB/s 2024-04-25T10:24:41.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.509 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117712 s, 89.1 MB/s 2024-04-25T10:24:41.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138603 s, 75.7 MB/s 2024-04-25T10:24:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147908 s, 70.9 MB/s 2024-04-25T10:24:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187202 s, 56.0 MB/s 2024-04-25T10:24:41.576 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.576 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.576 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122891 s, 85.3 MB/s 2024-04-25T10:24:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.591 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132724 s, 79.0 MB/s 2024-04-25T10:24:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135043 s, 77.6 MB/s 2024-04-25T10:24:41.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.627 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.627 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188081 s, 55.8 MB/s 2024-04-25T10:24:41.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014114 s, 74.3 MB/s 2024-04-25T10:24:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013045 s, 80.4 MB/s 2024-04-25T10:24:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134583 s, 77.9 MB/s 2024-04-25T10:24:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.692 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178815 s, 58.6 MB/s 2024-04-25T10:24:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153756 s, 68.2 MB/s 2024-04-25T10:24:41.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138188 s, 75.9 MB/s 2024-04-25T10:24:41.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143394 s, 73.1 MB/s 2024-04-25T10:24:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151589 s, 69.2 MB/s 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.773 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131726 s, 79.6 MB/s 2024-04-25T10:24:41.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.788 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136314 s, 76.9 MB/s 2024-04-25T10:24:41.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134198 s, 78.1 MB/s 2024-04-25T10:24:41.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014266 s, 73.5 MB/s 2024-04-25T10:24:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151588 s, 69.2 MB/s 2024-04-25T10:24:41.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:41.838 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:41.838 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:41.839 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:24:41.839 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:41.839 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:41.839 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:41.839 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:41.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137165 s, 76.4 MB/s 2024-04-25T10:24:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133849 s, 78.3 MB/s 2024-04-25T10:24:41.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.938 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140176 s, 74.8 MB/s 2024-04-25T10:24:41.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.953 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131248 s, 79.9 MB/s 2024-04-25T10:24:41.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.968 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137026 s, 76.5 MB/s 2024-04-25T10:24:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:41.984 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136169 s, 77.0 MB/s 2024-04-25T10:24:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.001 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147985 s, 70.9 MB/s 2024-04-25T10:24:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132935 s, 78.9 MB/s 2024-04-25T10:24:42.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134755 s, 77.8 MB/s 2024-04-25T10:24:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143452 s, 73.1 MB/s 2024-04-25T10:24:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.062 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133731 s, 78.4 MB/s 2024-04-25T10:24:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179394 s, 58.5 MB/s 2024-04-25T10:24:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136192 s, 77.0 MB/s 2024-04-25T10:24:42.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.114 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014982 s, 70.0 MB/s 2024-04-25T10:24:42.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162579 s, 64.5 MB/s 2024-04-25T10:24:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187239 s, 56.0 MB/s 2024-04-25T10:24:42.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145791 s, 71.9 MB/s 2024-04-25T10:24:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144097 s, 72.8 MB/s 2024-04-25T10:24:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151545 s, 69.2 MB/s 2024-04-25T10:24:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.218 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133726 s, 78.4 MB/s 2024-04-25T10:24:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.235 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154213 s, 68.0 MB/s 2024-04-25T10:24:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.255 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186896 s, 56.1 MB/s 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.272 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151828 s, 69.1 MB/s 2024-04-25T10:24:42.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.288 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135207 s, 77.6 MB/s 2024-04-25T10:24:42.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.302 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.302 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125928 s, 83.3 MB/s 2024-04-25T10:24:42.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148222 s, 70.7 MB/s 2024-04-25T10:24:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.335 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145273 s, 72.2 MB/s 2024-04-25T10:24:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133924 s, 78.3 MB/s 2024-04-25T10:24:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134532 s, 77.9 MB/s 2024-04-25T10:24:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.386 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180793 s, 58.0 MB/s 2024-04-25T10:24:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137493 s, 76.3 MB/s 2024-04-25T10:24:42.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.421 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180403 s, 58.1 MB/s 2024-04-25T10:24:42.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:42.422 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:42.422 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:42.422 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:24:42.422 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:42.422 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:42.423 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:42.423 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:42.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.494 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134038 s, 78.2 MB/s 2024-04-25T10:24:42.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.511 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014476 s, 72.4 MB/s 2024-04-25T10:24:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132577 s, 79.1 MB/s 2024-04-25T10:24:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146418 s, 71.6 MB/s 2024-04-25T10:24:42.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.560 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168945 s, 62.1 MB/s 2024-04-25T10:24:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151414 s, 69.3 MB/s 2024-04-25T10:24:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135344 s, 77.5 MB/s 2024-04-25T10:24:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141934 s, 73.9 MB/s 2024-04-25T10:24:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134631 s, 77.9 MB/s 2024-04-25T10:24:42.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133622 s, 78.5 MB/s 2024-04-25T10:24:42.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.654 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134482 s, 78.0 MB/s 2024-04-25T10:24:42.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143801 s, 72.9 MB/s 2024-04-25T10:24:42.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128515 s, 81.6 MB/s 2024-04-25T10:24:42.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130615 s, 80.3 MB/s 2024-04-25T10:24:42.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.717 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014614 s, 71.8 MB/s 2024-04-25T10:24:42.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.732 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136983 s, 76.5 MB/s 2024-04-25T10:24:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144663 s, 72.5 MB/s 2024-04-25T10:24:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131562 s, 79.7 MB/s 2024-04-25T10:24:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.781 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158059 s, 66.3 MB/s 2024-04-25T10:24:42.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.796 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134071 s, 78.2 MB/s 2024-04-25T10:24:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.812 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142318 s, 73.7 MB/s 2024-04-25T10:24:42.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.829 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.829 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149732 s, 70.0 MB/s 2024-04-25T10:24:42.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151776 s, 69.1 MB/s 2024-04-25T10:24:42.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.861 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135228 s, 77.5 MB/s 2024-04-25T10:24:42.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148977 s, 70.4 MB/s 2024-04-25T10:24:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139608 s, 75.1 MB/s 2024-04-25T10:24:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131469 s, 79.8 MB/s 2024-04-25T10:24:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145493 s, 72.1 MB/s 2024-04-25T10:24:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013534 s, 77.5 MB/s 2024-04-25T10:24:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142874 s, 73.4 MB/s 2024-04-25T10:24:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140055 s, 74.9 MB/s 2024-04-25T10:24:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244669 s, 42.9 MB/s 2024-04-25T10:24:43.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:24:43.001 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:24:43.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.074 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132844 s, 78.9 MB/s 2024-04-25T10:24:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122402 s, 85.7 MB/s 2024-04-25T10:24:43.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.103 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135229 s, 77.5 MB/s 2024-04-25T10:24:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146948 s, 71.4 MB/s 2024-04-25T10:24:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.140 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177614 s, 59.0 MB/s 2024-04-25T10:24:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013577 s, 77.2 MB/s 2024-04-25T10:24:43.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136286 s, 76.9 MB/s 2024-04-25T10:24:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139045 s, 75.4 MB/s 2024-04-25T10:24:43.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163701 s, 64.1 MB/s 2024-04-25T10:24:43.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132303 s, 79.3 MB/s 2024-04-25T10:24:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134025 s, 78.2 MB/s 2024-04-25T10:24:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130625 s, 80.3 MB/s 2024-04-25T10:24:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172315 s, 60.9 MB/s 2024-04-25T10:24:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154452 s, 67.9 MB/s 2024-04-25T10:24:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173619 s, 60.4 MB/s 2024-04-25T10:24:43.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.322 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150039 s, 69.9 MB/s 2024-04-25T10:24:43.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.337 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128387 s, 81.7 MB/s 2024-04-25T10:24:43.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138889 s, 75.5 MB/s 2024-04-25T10:24:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141024 s, 74.4 MB/s 2024-04-25T10:24:43.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153242 s, 68.4 MB/s 2024-04-25T10:24:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132145 s, 79.4 MB/s 2024-04-25T10:24:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.416 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136209 s, 77.0 MB/s 2024-04-25T10:24:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134375 s, 78.0 MB/s 2024-04-25T10:24:43.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.447 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136005 s, 77.1 MB/s 2024-04-25T10:24:43.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.462 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131124 s, 80.0 MB/s 2024-04-25T10:24:43.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.479 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141518 s, 74.1 MB/s 2024-04-25T10:24:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012804 s, 81.9 MB/s 2024-04-25T10:24:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118538 s, 88.5 MB/s 2024-04-25T10:24:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135747 s, 77.2 MB/s 2024-04-25T10:24:43.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149223 s, 70.3 MB/s 2024-04-25T10:24:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151062 s, 69.4 MB/s 2024-04-25T10:24:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:24:43.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:24:43.574 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139802 s, 75.0 MB/s 2024-04-25T10:24:43.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:43.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:24:43.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 Processor -- start 2024-04-25T10:24:43.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 -- start start 2024-04-25T10:24:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb1a80d0090 0x7fb1a815be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a815ac40 0x7fb1a80b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a815b100 con 0x7fb1a80721f0 2024-04-25T10:24:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a815bc80 con 0x7fb1a80d0090 2024-04-25T10:24:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1acc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a815c390 con 0x7fb1a815ac40 2024-04-25T10:24:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb1a80721f0 0x7fb1a815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37804/0 (socket says 172.21.15.45:37804) 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7e00640 1 -- 172.21.15.45:0/1812867365 learned_addr learned my addr 172.21.15.45:0/1812867365 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7400640 1 --2- 172.21.15.45:0/1812867365 >> v2:172.21.15.117:3300/0 conn(0x7fb1a80d0090 0x7fb1a815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb19fe00640 1 --2- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a815ac40 0x7fb1a80b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a6a00640 1 -- 172.21.15.45:0/1812867365 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070395943 0 0) 0x7fb1a815b100 con 0x7fb1a80721f0 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a6a00640 1 -- 172.21.15.45:0/1812867365 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb18c005740 con 0x7fb1a80721f0 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7400640 1 -- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a815ac40 msgr2=0x7fb1a80b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7400640 1 --2- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a815ac40 0x7fb1a80b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7400640 1 -- 172.21.15.45:0/1812867365 >> v1:172.21.15.45:6789/0 conn(0x7fb1a80721f0 legacy=0x7fb1a815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.760+0000 7fb1a7400640 1 -- 172.21.15.45:0/1812867365 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a815b3e0 con 0x7fb1a80d0090 2024-04-25T10:24:43.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1a7400640 1 --2- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a80d0090 0x7fb1a815be50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb1980097c0 tx=0x7fb19801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ca6fcc3bbc57b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1a6a00640 1 -- 172.21.15.45:0/1812867365 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980b0070 con 0x7fb1a80d0090 2024-04-25T10:24:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1a6a00640 1 -- 172.21.15.45:0/1812867365 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1980ab600 con 0x7fb1a80d0090 2024-04-25T10:24:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1a6a00640 1 -- 172.21.15.45:0/1812867365 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980ab9d0 con 0x7fb1a80d0090 2024-04-25T10:24:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 -- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a80d0090 msgr2=0x7fb1a815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a80d0090 0x7fb1a815be50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb1980097c0 tx=0x7fb19801c380 comp rx=0 tx=0).stop 2024-04-25T10:24:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 -- 172.21.15.45:0/1812867365 shutdown_connections 2024-04-25T10:24:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a815ac40 0x7fb1a80b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1812867365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a80d0090 0x7fb1a815be50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 -- 172.21.15.45:0/1812867365 >> 172.21.15.45:0/1812867365 conn(0x7fb1a8074450 msgr2=0x7fb1a80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 -- 172.21.15.45:0/1812867365 shutdown_connections 2024-04-25T10:24:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.764+0000 7fb1acc00640 1 -- 172.21.15.45:0/1812867365 wait complete. 2024-04-25T10:24:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 Processor -- start 2024-04-25T10:24:43.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 -- start start 2024-04-25T10:24:43.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 0x7fb1a8140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a80d0090 0x7fb1a80a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a815ac40 0x7fb1a80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a806c9c0 con 0x7fb1a80721f0 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a80b3a10 con 0x7fb1a815ac40 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a806cb40 con 0x7fb1a80d0090 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 0x7fb1a8140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 0x7fb1a8140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52010/0 (socket says 172.21.15.45:52010) 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 -- 172.21.15.45:0/1665517690 learned_addr learned my addr 172.21.15.45:0/1665517690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a815ac40 0x7fb1a80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb19fe00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a80d0090 0x7fb1a80a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 -- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a80d0090 msgr2=0x7fb1a80a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a80d0090 0x7fb1a80a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 -- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a815ac40 msgr2=0x7fb1a80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a815ac40 0x7fb1a80b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 -- 172.21.15.45:0/1665517690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8141500 con 0x7fb1a80721f0 2024-04-25T10:24:43.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb19fe00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a80d0090 0x7fb1a80a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:43.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a815ac40 0x7fb1a80b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:43.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1a7400640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 0x7fb1a8140fc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb198019fb0 tx=0x7fb198019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb19f400640 1 -- 172.21.15.45:0/1665517690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980b8020 con 0x7fb1a80721f0 2024-04-25T10:24:43.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb19f400640 1 -- 172.21.15.45:0/1665517690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1980b4410 con 0x7fb1a80721f0 2024-04-25T10:24:43.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb19f400640 1 -- 172.21.15.45:0/1665517690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980b0040 con 0x7fb1a80721f0 2024-04-25T10:24:43.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a80a9970 con 0x7fb1a80721f0 2024-04-25T10:24:43.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.768+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a80ac7b0 con 0x7fb1a80721f0 2024-04-25T10:24:43.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb19f400640 1 -- 172.21.15.45:0/1665517690 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1980bb020 con 0x7fb1a80721f0 2024-04-25T10:24:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb19f400640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb184066f60 0x7fb184069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb19f400640 1 -- 172.21.15.45:0/1665517690 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb19811eb50 con 0x7fb1a80721f0 2024-04-25T10:24:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb19fe00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb184066f60 0x7fb184069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 Processor -- start 2024-04-25T10:24:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 -- start start 2024-04-25T10:24:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb19fe00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb184066f60 0x7fb184069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80b1280 tx=0x7fb190008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 0x55f4318067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f43160ad50 con 0x55f4318063f0 2024-04-25T10:24:43.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f431592e40 con 0x55f431806d10 2024-04-25T10:24:43.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f431606df0 con 0x55f4317bd520 2024-04-25T10:24:43.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 0x55f4318067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 0x55f4318067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52012/0 (socket says 172.21.15.45:52012) 2024-04-25T10:24:43.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 -- 172.21.15.45:0/3058977081 learned_addr learned my addr 172.21.15.45:0/3058977081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:43.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 -- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 msgr2=0x55f4317bd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 -- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 msgr2=0x55f4317bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 -- 172.21.15.45:0/3058977081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4315a7d90 con 0x55f4318063f0 2024-04-25T10:24:43.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.772+0000 7fb1a7400640 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 0x55f4318067d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb1980ef3e0 tx=0x7fb198006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65f1ee1c44f0987 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb197e00640 1 -- 172.21.15.45:0/3058977081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980ef450 con 0x55f4318063f0 2024-04-25T10:24:43.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb197e00640 1 -- 172.21.15.45:0/3058977081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1980f35a0 con 0x55f4318063f0 2024-04-25T10:24:43.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb197e00640 1 -- 172.21.15.45:0/3058977081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980f7b60 con 0x55f4318063f0 2024-04-25T10:24:43.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3058977081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 msgr2=0x55f4318067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 0x55f4318067d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fb1980ef3e0 tx=0x7fb198006e20 comp rx=0 tx=0).stop 2024-04-25T10:24:43.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3058977081 shutdown_connections 2024-04-25T10:24:43.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3058977081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4318063f0 0x55f4318067d0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3058977081 >> 172.21.15.45:0/3058977081 conn(0x55f4317b94d0 msgr2=0x55f4317bb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3058977081 shutdown_connections 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3058977081 wait complete. 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- start start 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 Processor -- start 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 0x55f431805d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4318063f0 0x55f4317c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4315b0610 con 0x55f4317bd520 2024-04-25T10:24:43.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4315c59a0 con 0x55f431806d10 2024-04-25T10:24:43.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f43160f380 con 0x55f4318063f0 2024-04-25T10:24:43.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 10 client.0 _ll_register_callbacks cb 0x55f4316bf0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:43.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 0x55f431805d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 0x55f431805d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52024/0 (socket says 172.21.15.45:52024) 2024-04-25T10:24:43.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 -- 172.21.15.45:0/3884719297 learned_addr learned my addr 172.21.15.45:0/3884719297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:43.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb19fe00640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4318063f0 0x55f4317c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 -- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4318063f0 msgr2=0x55f4317c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4318063f0 0x55f4317c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 -- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 msgr2=0x55f4317c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 -- 172.21.15.45:0/3884719297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f431605d40 con 0x55f4317bd520 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1a7400640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 0x55f431805d10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb1980ef3e0 tx=0x7fb1980abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b3d51b600c1a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb194c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb194c00640 1 -- 172.21.15.45:0/3884719297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19811b070 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb194c00640 1 -- 172.21.15.45:0/3884719297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1980f76d0 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.776+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1980f35a0 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 1 -- 172.21.15.45:0/3884719297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1980f3370 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 10 client.7521 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55f43168df00 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 1 -- 172.21.15.45:0/3884719297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1980f35a0 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 1 -- 172.21.15.45:0/3884719297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1980b3050 con 0x55f4317bd520 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 10 client.7521 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 10 client.7521 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 20 client.7521 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f431807b30 0x55f431817b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 4 client.7521 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:43.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4315a6a20 con 0x55f431807b30 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb19fe00640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f431807b30 0x55f431817b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb19fe00640 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f431807b30 0x55f431817b00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2c5c2197fad60a server_cookie=7c192b9dc09a3d36 in_seq=0 out_seq=0 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 10 client.7521 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 1 -- 172.21.15.45:0/3884719297 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4315a6a20 con 0x55f431807b30 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 10 client.7521 handle_command_reply: tid=0 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 client.7521 shutdown 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 msgr2=0x55f431805d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 0x55f431805d10 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb1980ef3e0 tx=0x7fb1980abf90 comp rx=0 tx=0).stop 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 shutdown_connections 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f431807b30 0x55f431817b00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f431806d10 0x55f4317c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4318063f0 0x55f4317c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb194c00640 0 client.7521 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:43.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/3884719297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f4317bd520 0x55f431805d10 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 >> 172.21.15.45:0/3884719297 conn(0x55f4317b94d0 msgr2=0x55f4317bc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 shutdown_connections 2024-04-25T10:24:43.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 1 -- 172.21.15.45:0/3884719297 wait complete. 2024-04-25T10:24:43.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.780+0000 7fb1b16c2000 20 client.7521 trim_cache size 0 max 16384 2024-04-25T10:24:43.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 Processor -- start 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 -- start start 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431803dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431804310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8203170 con 0x55f431806d10 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a82022f0 con 0x55f4318063f0 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a82016f0 con 0x55f4317bd520 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431804310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431804310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52030/0 (socket says 172.21.15.45:52030) 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 -- 172.21.15.45:0/52670664 learned_addr learned my addr 172.21.15.45:0/52670664 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb19fe00640 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431803dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7400640 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 -- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 msgr2=0x55f4317bcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 -- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 msgr2=0x55f431803dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431803dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 -- 172.21.15.45:0/52670664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f431664330 con 0x55f431806d10 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431804310 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00144a0 tx=0x7fb1a0014960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caa9588f3d6b06f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb197400640 1 -- 172.21.15.45:0/52670664 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a001a070 con 0x55f431806d10 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb197400640 1 -- 172.21.15.45:0/52670664 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a00167b0 con 0x55f431806d10 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.796+0000 7fb197400640 1 -- 172.21.15.45:0/52670664 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0016b60 con 0x55f431806d10 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/52670664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 msgr2=0x55f431804310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431804310 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00144a0 tx=0x7fb1a0014960 comp rx=0 tx=0).stop 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/52670664 shutdown_connections 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431804310 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431803dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/52670664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f4317bcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/52670664 >> 172.21.15.45:0/52670664 conn(0x55f4317b94d0 msgr2=0x55f4317c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/52670664 shutdown_connections 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/52670664 wait complete. 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- start start 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 Processor -- start 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f431818f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431823590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431823ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4315c1dc0 con 0x55f431806d10 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a8201cf0 con 0x55f4318063f0 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f43160a060 con 0x55f4317bd520 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 10 client.0 _ll_register_callbacks cb 0x7fb198009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431823ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431823ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52046/0 (socket says 172.21.15.45:52046) 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f431818f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 -- 172.21.15.45:0/4199063191 learned_addr learned my addr 172.21.15.45:0/4199063191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb19fe00640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431823590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 -- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 msgr2=0x55f431818f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f431818f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 -- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 msgr2=0x55f431823590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431823590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 -- 172.21.15.45:0/4199063191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4315d75f0 con 0x55f431806d10 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb19fe00640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431823590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1a7e00640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431823ad0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00149a0 tx=0x7fb1a0001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b653e2fda420d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb194c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb194c00640 1 -- 172.21.15.45:0/4199063191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a001a040 con 0x55f431806d10 2024-04-25T10:24:43.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb194c00640 1 -- 172.21.15.45:0/4199063191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1a002aaa0 con 0x55f431806d10 2024-04-25T10:24:43.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1a00167b0 con 0x55f431806d10 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb194c00640 1 -- 172.21.15.45:0/4199063191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1a0031050 con 0x55f431806d10 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 10 client.7527 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.800+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55f4315b1fe0 con 0x55f431806d10 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb194c00640 1 -- 172.21.15.45:0/4199063191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a00167b0 con 0x55f431806d10 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb194c00640 1 -- 172.21.15.45:0/4199063191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1a0034020 con 0x55f431806d10 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1b16c2000 10 client.7527 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1b16c2000 10 client.7527 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1b16c2000 20 client.7527 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:43.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f43186c9f0 0x55f43186ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1b16c2000 4 client.7527 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55f431618290 con 0x55f43186c9f0 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1a7400640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f43186c9f0 0x55f43186ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb1a7400640 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f43186c9f0 0x55f43186ede0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5b3293f13ebb92 server_cookie=58264553908b25a7 in_seq=0 out_seq=0 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.804+0000 7fb194c00640 10 client.7527 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb194c00640 1 -- 172.21.15.45:0/4199063191 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f431618290 con 0x55f43186c9f0 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb194c00640 10 client.7527 handle_command_reply: tid=0 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 client.7527 shutdown 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 msgr2=0x55f431823ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431823ad0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00149a0 tx=0x7fb1a0001260 comp rx=0 tx=0).stop 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 shutdown_connections 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f431806d10 0x55f431823ad0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55f43186c9f0 0x55f43186ede0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f4318063f0 0x55f431823590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 --2- 172.21.15.45:0/4199063191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f4317bd520 0x55f431818f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 >> 172.21.15.45:0/4199063191 conn(0x55f4317b94d0 msgr2=0x55f4317c1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb194c00640 0 client.7527 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 shutdown_connections 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 1 -- 172.21.15.45:0/4199063191 wait complete. 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1b16c2000 20 client.7527 trim_cache size 0 max 16384 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:24:43.811 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "b88af451-20f3-4462-aeb7-214ecf0560fc", 2024-04-25T10:24:43.812 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:24:43.812 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:24:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb184066f60 msgr2=0x7fb184069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb184066f60 0x7fb184069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80b1280 tx=0x7fb190008040 comp rx=0 tx=0).stop 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 msgr2=0x7fb1a8140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 0x7fb1a8140fc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb198019fb0 tx=0x7fb198019120 comp rx=0 tx=0).stop 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 shutdown_connections 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb184066f60 0x7fb184069420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1a815ac40 0x7fb1a80b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1a80d0090 0x7fb1a80a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 --2- 172.21.15.45:0/1665517690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1a80721f0 0x7fb1a8140fc0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 >> 172.21.15.45:0/1665517690 conn(0x7fb1a8074450 msgr2=0x7fb1a815d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 shutdown_connections 2024-04-25T10:24:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:43.808+0000 7fb1acc00640 1 -- 172.21.15.45:0/1665517690 wait complete. 2024-04-25T10:24:43.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:43.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T10:24:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 Processor -- start 2024-04-25T10:24:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 -- start start 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b2c15dc90 0x7f8b2c0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b2c06b400 0x7f8b2c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c0cff30 con 0x7f8b2c0cca70 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c06ac90 con 0x7f8b2c15dc90 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2be00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0d0210 con 0x7f8b2c06b400 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b2c15dc90 0x7f8b2c0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b2c15dc90 0x7f8b2c0cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52050/0 (socket says 172.21.15.45:52050) 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 -- 172.21.15.45:0/3327762982 learned_addr learned my addr 172.21.15.45:0/3327762982 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2b400640 1 --2- 172.21.15.45:0/3327762982 >> v2:172.21.15.117:3300/0 conn(0x7f8b2c06b400 0x7f8b2c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 -- 172.21.15.45:0/3327762982 >> v2:172.21.15.117:3300/0 conn(0x7f8b2c06b400 msgr2=0x7f8b2c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/3327762982 >> v2:172.21.15.117:3300/0 conn(0x7f8b2c06b400 0x7f8b2c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 -- 172.21.15.45:0/3327762982 >> v1:172.21.15.117:6789/0 conn(0x7f8b2c0cca70 legacy=0x7f8b2c0ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.016+0000 7f8b2aa00640 1 -- 172.21.15.45:0/3327762982 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0d04f0 con 0x7f8b2c15dc90 2024-04-25T10:24:44.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/3327762982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c15dc90 0x7f8b2c0cc530 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0d16e0 tx=0x7f8b1c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a158dbff1ffc25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2a000640 1 -- 172.21.15.45:0/3327762982 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0ab660 con 0x7f8b2c15dc90 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2a000640 1 -- 172.21.15.45:0/3327762982 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c0b0070 con 0x7f8b2c15dc90 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2a000640 1 -- 172.21.15.45:0/3327762982 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0aba20 con 0x7f8b2c15dc90 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 -- 172.21.15.45:0/3327762982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c15dc90 msgr2=0x7f8b2c0cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 --2- 172.21.15.45:0/3327762982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c15dc90 0x7f8b2c0cc530 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0d16e0 tx=0x7f8b1c01c460 comp rx=0 tx=0).stop 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 -- 172.21.15.45:0/3327762982 shutdown_connections 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 --2- 172.21.15.45:0/3327762982 >> v2:172.21.15.117:3300/0 conn(0x7f8b2c06b400 0x7f8b2c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 --2- 172.21.15.45:0/3327762982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c15dc90 0x7f8b2c0cc530 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 -- 172.21.15.45:0/3327762982 >> 172.21.15.45:0/3327762982 conn(0x7f8b2c0d1740 msgr2=0x7f8b2c0d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 -- 172.21.15.45:0/3327762982 shutdown_connections 2024-04-25T10:24:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.020+0000 7f8b2be00640 1 -- 172.21.15.45:0/3327762982 wait complete. 2024-04-25T10:24:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 Processor -- start 2024-04-25T10:24:44.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 -- start start 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 0x7f8b2c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2c0cca70 0x7f8b2c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b2c15dc90 0x7f8b2c14c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c15e070 con 0x7f8b2c06b400 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0cef60 con 0x7f8b2c0cca70 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c15e1f0 con 0x7f8b2c15dc90 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 0x7f8b2c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 0x7f8b2c15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52056/0 (socket says 172.21.15.45:52056) 2024-04-25T10:24:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 -- 172.21.15.45:0/4109375975 learned_addr learned my addr 172.21.15.45:0/4109375975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2b400640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b2c15dc90 0x7f8b2c14c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b23e00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2c0cca70 0x7f8b2c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 -- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b2c15dc90 msgr2=0x7f8b2c14c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b2c15dc90 0x7f8b2c14c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 -- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2c0cca70 msgr2=0x7f8b2c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2c0cca70 0x7f8b2c0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 -- 172.21.15.45:0/4109375975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c15c8a0 con 0x7f8b2c06b400 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b23e00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2c0cca70 0x7f8b2c0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 0x7f8b2c15ae90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ad040 tx=0x7f8b1c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b23400640 1 -- 172.21.15.45:0/4109375975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0be5f0 con 0x7f8b2c06b400 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b23400640 1 -- 172.21.15.45:0/4109375975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c0be7b0 con 0x7f8b2c06b400 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b23400640 1 -- 172.21.15.45:0/4109375975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0beba0 con 0x7f8b2c06b400 2024-04-25T10:24:44.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c15b520 con 0x7f8b2c06b400 2024-04-25T10:24:44.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.024+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c15bae0 con 0x7f8b2c06b400 2024-04-25T10:24:44.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b23400640 1 -- 172.21.15.45:0/4109375975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b1c0b43f0 con 0x7f8b2c06b400 2024-04-25T10:24:44.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b23400640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b08066f60 0x7f8b08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b23400640 1 -- 172.21.15.45:0/4109375975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b1c11fc10 con 0x7f8b2c06b400 2024-04-25T10:24:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b23e00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b08066f60 0x7f8b08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b352fe000 1 Processor -- start 2024-04-25T10:24:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b352fe000 1 -- start start 2024-04-25T10:24:44.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.028+0000 7f8b23e00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b08066f60 0x7f8b08069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c15d300 tx=0x7f8b14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc620e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc4be1f0 con 0x55d1bc620a40 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc424e90 con 0x55d1bc5dbd90 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc3acf50 con 0x55d1bc621360 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc620e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc620e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52072/0 (socket says 172.21.15.45:52072) 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 -- 172.21.15.45:0/167708268 learned_addr learned my addr 172.21.15.45:0/167708268 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b23e00640 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 -- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 msgr2=0x55d1bc5db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5db850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 -- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 msgr2=0x55d1bc5dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 -- 172.21.15.45:0/167708268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1bc3fc8b0 con 0x55d1bc620a40 2024-04-25T10:24:44.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc620e20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0f0350 tx=0x7f8b1c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12f5675d0f0d795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b1be00640 1 -- 172.21.15.45:0/167708268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0c4050 con 0x55d1bc620a40 2024-04-25T10:24:44.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b1be00640 1 -- 172.21.15.45:0/167708268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c0b0070 con 0x55d1bc620a40 2024-04-25T10:24:44.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b1be00640 1 -- 172.21.15.45:0/167708268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0f8cb0 con 0x55d1bc620a40 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- 172.21.15.45:0/167708268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 msgr2=0x55d1bc620e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc620e20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0f0350 tx=0x7f8b1c019200 comp rx=0 tx=0).stop 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- 172.21.15.45:0/167708268 shutdown_connections 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- 172.21.15.45:0/167708268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc620e20 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- 172.21.15.45:0/167708268 >> 172.21.15.45:0/167708268 conn(0x55d1bc5d7d40 msgr2=0x55d1bc5da130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- 172.21.15.45:0/167708268 shutdown_connections 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- 172.21.15.45:0/167708268 wait complete. 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- start start 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 Processor -- start 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc620a40 0x55d1bc5e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc3df540 con 0x55d1bc5dbd90 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc420f30 con 0x55d1bc620a40 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc3d9cb0 con 0x55d1bc621360 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.032+0000 7f8b352fe000 10 client.0 _ll_register_callbacks cb 0x55d1bc4d9180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5e1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52080/0 (socket says 172.21.15.45:52080) 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc620a40 0x55d1bc5e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2522860317 learned_addr learned my addr 172.21.15.45:0/2522860317 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2b400640 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 msgr2=0x55d1bc5e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc620a40 msgr2=0x55d1bc5e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc620a40 0x55d1bc5e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2522860317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1bc3ad1a0 con 0x55d1bc5dbd90 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5e1450 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c097f30 tx=0x7f8b1c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d0b81a9a8d8183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 1 -- 172.21.15.45:0/2522860317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c11c060 con 0x55d1bc5dbd90 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 1 -- 172.21.15.45:0/2522860317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b1c0f8820 con 0x55d1bc5dbd90 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b1c0b0070 con 0x55d1bc5dbd90 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 1 -- 172.21.15.45:0/2522860317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b1c0fd050 con 0x55d1bc5dbd90 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 10 client.7539 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:44.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d1bc4a7840 con 0x55d1bc5dbd90 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 1 -- 172.21.15.45:0/2522860317 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c0b0070 con 0x55d1bc5dbd90 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 1 -- 172.21.15.45:0/2522860317 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b1c100020 con 0x55d1bc5dbd90 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 10 client.7539 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 10 client.7539 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 20 client.7539 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc62fce0 0x55d1bc6320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 4 client.7539 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1bc3c0be0 con 0x55d1bc62fce0 2024-04-25T10:24:44.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b23e00640 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc62fce0 0x55d1bc6320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b23e00640 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc62fce0 0x55d1bc6320d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e25f8e7e574de8 server_cookie=9b22c319a1e51e31 in_seq=0 out_seq=0 2024-04-25T10:24:44.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 10 client.7539 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 1 -- 172.21.15.45:0/2522860317 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d1bc3c0be0 con 0x55d1bc62fce0 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 10 client.7539 handle_command_reply: tid=0 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 client.7539 shutdown 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 msgr2=0x55d1bc5e1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5e1450 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c097f30 tx=0x7f8b1c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 shutdown_connections 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc5e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc62fce0 0x55d1bc6320d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc620a40 0x55d1bc5e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2522860317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc5e1450 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b18c00640 0 client.7539 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 >> 172.21.15.45:0/2522860317 conn(0x55d1bc5d7d40 msgr2=0x55d1bc5da9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 shutdown_connections 2024-04-25T10:24:44.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 1 -- 172.21.15.45:0/2522860317 wait complete. 2024-04-25T10:24:44.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.036+0000 7f8b352fe000 20 client.7539 trim_cache size 0 max 16384 2024-04-25T10:24:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 Processor -- start 2024-04-25T10:24:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 -- start start 2024-04-25T10:24:44.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc634620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc681f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c202e20 con 0x55d1bc620a40 2024-04-25T10:24:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c158ca0 con 0x55d1bc5dbd90 2024-04-25T10:24:44.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0c96a0 con 0x55d1bc621360 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b23e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc634620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6340e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33010/0 (socket says 172.21.15.45:33010) 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b2b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc681f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.052+0000 7f8b2aa00640 1 -- 172.21.15.45:0/1298024777 learned_addr learned my addr 172.21.15.45:0/1298024777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 msgr2=0x55d1bc681f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc681f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/1298024777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 msgr2=0x55d1bc634620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc634620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/1298024777 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1bc47ea10 con 0x55d1bc5dbd90 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6340e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0f0350 tx=0x7f8b1c0c0000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8783696d6db622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b1b400640 1 -- 172.21.15.45:0/1298024777 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0c4070 con 0x55d1bc5dbd90 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b1b400640 1 -- 172.21.15.45:0/1298024777 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c0b0070 con 0x55d1bc5dbd90 2024-04-25T10:24:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b1b400640 1 -- 172.21.15.45:0/1298024777 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0f8a20 con 0x55d1bc5dbd90 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 msgr2=0x55d1bc6340e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6340e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0f0350 tx=0x7f8b1c0c0000 comp rx=0 tx=0).stop 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- 172.21.15.45:0/1298024777 shutdown_connections 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc621360 0x55d1bc681f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc620a40 0x55d1bc634620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- 172.21.15.45:0/1298024777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6340e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- 172.21.15.45:0/1298024777 >> 172.21.15.45:0/1298024777 conn(0x55d1bc5d7d40 msgr2=0x55d1bc61f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- 172.21.15.45:0/1298024777 shutdown_connections 2024-04-25T10:24:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- 172.21.15.45:0/1298024777 wait complete. 2024-04-25T10:24:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- start start 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 Processor -- start 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc620a40 0x55d1bc680a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc621360 0x55d1bc63bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc4241a0 con 0x55d1bc5dbd90 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c1586a0 con 0x55d1bc621360 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1bc3cbc00 con 0x55d1bc620a40 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b352fe000 10 client.0 _ll_register_callbacks cb 0x55d1bc3e7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6804d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52088/0 (socket says 172.21.15.45:52088) 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2858323313 learned_addr learned my addr 172.21.15.45:0/2858323313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2b400640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc621360 0x55d1bc63bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b23e00640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc620a40 0x55d1bc680a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc620a40 msgr2=0x55d1bc680a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc620a40 0x55d1bc680a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc621360 msgr2=0x55d1bc63bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc621360 0x55d1bc63bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 -- 172.21.15.45:0/2858323313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1bc3b9b00 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b2aa00640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6804d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0f0210 tx=0x7f8b1c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4278ba45fe38f504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.056+0000 7f8b18c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 1 -- 172.21.15.45:0/2858323313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0c4040 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 1 -- 172.21.15.45:0/2858323313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b1c0ec3a0 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b1c0b0070 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 1 -- 172.21.15.45:0/2858323313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b1c0ec5e0 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 10 client.7545 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d1bc3cbf80 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 1 -- 172.21.15.45:0/2858323313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c0b0070 con 0x55d1bc5dbd90 2024-04-25T10:24:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 1 -- 172.21.15.45:0/2858323313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b1c100040 con 0x55d1bc5dbd90 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 10 client.7545 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 10 client.7545 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 20 client.7545 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc63c540 0x55d1bc63c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 4 client.7545 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55d1bc4323d0 con 0x55d1bc63c540 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b23e00640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc63c540 0x55d1bc63c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b23e00640 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc63c540 0x55d1bc63c920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e213b3fb6f7416 server_cookie=dade6582923e9037 in_seq=0 out_seq=0 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 10 client.7545 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 1 -- 172.21.15.45:0/2858323313 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55d1bc4323d0 con 0x55d1bc63c540 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 10 client.7545 handle_command_reply: tid=0 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 client.7545 shutdown 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 msgr2=0x55d1bc6804d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6804d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0f0210 tx=0x7f8b1c009240 comp rx=0 tx=0).stop 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 shutdown_connections 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x55d1bc63c540 0x55d1bc63c920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1bc621360 0x55d1bc63bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55d1bc620a40 0x55d1bc680a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b18c00640 0 client.7545 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 --2- 172.21.15.45:0/2858323313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1bc5dbd90 0x55d1bc6804d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 >> 172.21.15.45:0/2858323313 conn(0x55d1bc5d7d40 msgr2=0x55d1bc61fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 shutdown_connections 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 1 -- 172.21.15.45:0/2858323313 wait complete. 2024-04-25T10:24:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b352fe000 20 client.7545 trim_cache size 0 max 16384 2024-04-25T10:24:44.066 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:24:44.066 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:24:44.066 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b08066f60 msgr2=0x7f8b08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b08066f60 0x7f8b08069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c15d300 tx=0x7f8b14009040 comp rx=0 tx=0).stop 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 msgr2=0x7f8b2c15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 0x7f8b2c15ae90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0ad040 tx=0x7f8b1c01aca0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 shutdown_connections 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b08066f60 0x7f8b08069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b2c15dc90 0x7f8b2c14c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b2c0cca70 0x7f8b2c0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 --2- 172.21.15.45:0/4109375975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c06b400 0x7f8b2c15ae90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 >> 172.21.15.45:0/4109375975 conn(0x7f8b2c0d1740 msgr2=0x7f8b2c0d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 shutdown_connections 2024-04-25T10:24:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.060+0000 7f8b2be00640 1 -- 172.21.15.45:0/4109375975 wait complete. 2024-04-25T10:24:44.081 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:24:44.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:24:44.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 Processor -- start 2024-04-25T10:24:44.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 -- start start 2024-04-25T10:24:44.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb03c0aef80 0x7fb03c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb03c070350 0x7fb03c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c15bac0 0x7fb03c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c15b080 con 0x7fb03c070350 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c15adb0 con 0x7fb03c0aef80 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb042600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15eb60 con 0x7fb03c15bac0 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb03c070350 0x7fb03c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb03c070350 0x7fb03c15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52102/0 (socket says 172.21.15.45:52102) 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 -- 172.21.15.45:0/44146987 learned_addr learned my addr 172.21.15.45:0/44146987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb03be00640 1 --2- 172.21.15.45:0/44146987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c15bac0 0x7fb03c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb03b400640 1 --2- 172.21.15.45:0/44146987 >> v2:172.21.15.117:3300/0 conn(0x7fb03c0aef80 0x7fb03c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 -- 172.21.15.45:0/44146987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c15bac0 msgr2=0x7fb03c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 --2- 172.21.15.45:0/44146987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c15bac0 0x7fb03c0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 -- 172.21.15.45:0/44146987 >> v2:172.21.15.117:3300/0 conn(0x7fb03c0aef80 msgr2=0x7fb03c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 --2- 172.21.15.45:0/44146987 >> v2:172.21.15.117:3300/0 conn(0x7fb03c0aef80 0x7fb03c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb032a00640 1 -- 172.21.15.45:0/44146987 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c15ee40 con 0x7fb03c070350 2024-04-25T10:24:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.224+0000 7fb03b400640 1 --2- 172.21.15.45:0/44146987 >> v2:172.21.15.117:3300/0 conn(0x7fb03c0aef80 0x7fb03c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:44.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb032a00640 1 --2- 172.21.15.45:0/44146987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c070350 0x7fb03c15b580 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb03c06e220 tx=0x7fb02401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c4703cd239b8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb03aa00640 1 -- 172.21.15.45:0/44146987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240b4070 con 0x7fb03c070350 2024-04-25T10:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb03aa00640 1 -- 172.21.15.45:0/44146987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0240a7a80 con 0x7fb03c070350 2024-04-25T10:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb03aa00640 1 -- 172.21.15.45:0/44146987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240a7e50 con 0x7fb03c070350 2024-04-25T10:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 -- 172.21.15.45:0/44146987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c070350 msgr2=0x7fb03c15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 --2- 172.21.15.45:0/44146987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c070350 0x7fb03c15b580 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb03c06e220 tx=0x7fb02401ebf0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 -- 172.21.15.45:0/44146987 shutdown_connections 2024-04-25T10:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 --2- 172.21.15.45:0/44146987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c15bac0 0x7fb03c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 --2- 172.21.15.45:0/44146987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c070350 0x7fb03c15b580 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 --2- 172.21.15.45:0/44146987 >> v2:172.21.15.117:3300/0 conn(0x7fb03c0aef80 0x7fb03c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 -- 172.21.15.45:0/44146987 >> 172.21.15.45:0/44146987 conn(0x7fb03c0669a0 msgr2=0x7fb03c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 -- 172.21.15.45:0/44146987 shutdown_connections 2024-04-25T10:24:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 -- 172.21.15.45:0/44146987 wait complete. 2024-04-25T10:24:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 Processor -- start 2024-04-25T10:24:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.228+0000 7fb042600640 1 -- start start 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c070350 0x7fb03c14a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0aef80 0x7fb03c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 0x7fb03c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c165ba0 con 0x7fb03c15bac0 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c165a20 con 0x7fb03c070350 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c165d20 con 0x7fb03c0aef80 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 0x7fb03c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 0x7fb03c0cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52116/0 (socket says 172.21.15.45:52116) 2024-04-25T10:24:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 -- 172.21.15.45:0/811935363 learned_addr learned my addr 172.21.15.45:0/811935363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb032a00640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0aef80 0x7fb03c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03b400640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c070350 0x7fb03c14a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 -- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0aef80 msgr2=0x7fb03c15ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0aef80 0x7fb03c15ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 -- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c070350 msgr2=0x7fb03c14a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c070350 0x7fb03c14a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 -- 172.21.15.45:0/811935363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c160830 con 0x7fb03c15bac0 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb032a00640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0aef80 0x7fb03c15ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb03be00640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 0x7fb03c0cd110 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d30 tx=0x7fb03400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb033e00640 1 -- 172.21.15.45:0/811935363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034018020 con 0x7fb03c15bac0 2024-04-25T10:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb033e00640 1 -- 172.21.15.45:0/811935363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034014480 con 0x7fb03c15bac0 2024-04-25T10:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb033e00640 1 -- 172.21.15.45:0/811935363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034014870 con 0x7fb03c15bac0 2024-04-25T10:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c160b10 con 0x7fb03c15bac0 2024-04-25T10:24:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.232+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c1610d0 con 0x7fb03c15bac0 2024-04-25T10:24:44.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb033e00640 1 -- 172.21.15.45:0/811935363 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb03401b020 con 0x7fb03c15bac0 2024-04-25T10:24:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb033e00640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb018066f60 0x7fb018069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb033e00640 1 -- 172.21.15.45:0/811935363 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb034085a80 con 0x7fb03c15bac0 2024-04-25T10:24:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 Processor -- start 2024-04-25T10:24:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb018066f60 0x7fb018069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 -- start start 2024-04-25T10:24:44.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb018066f60 0x7fb018069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098d0 tx=0x7fb02c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b3cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134af2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134af3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56313493fff0 con 0x563134b3c9d0 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631348c8150 con 0x563134b3d2f0 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56313493c090 con 0x563134af30a0 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b3cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b3cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52126/0 (socket says 172.21.15.45:52126) 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 -- 172.21.15.45:0/2366940019 learned_addr learned my addr 172.21.15.45:0/2366940019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03be00640 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134af3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb032a00640 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134af2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 -- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 msgr2=0x563134af3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134af3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 -- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 msgr2=0x563134af2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134af2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 -- 172.21.15.45:0/2366940019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563134ad60d0 con 0x563134b3c9d0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb03b400640 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b3cdb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb02c011a20 tx=0x7fb02c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315f5d24123b0e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb02be00640 1 -- 172.21.15.45:0/2366940019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00f040 con 0x563134b3c9d0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb02be00640 1 -- 172.21.15.45:0/2366940019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c026070 con 0x563134b3c9d0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.236+0000 7fb02be00640 1 -- 172.21.15.45:0/2366940019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c015dc0 con 0x563134b3c9d0 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2366940019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 msgr2=0x563134b3cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b3cdb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb02c011a20 tx=0x7fb02c014000 comp rx=0 tx=0).stop 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2366940019 shutdown_connections 2024-04-25T10:24:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134af3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134af2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2366940019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b3cdb0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2366940019 >> 172.21.15.45:0/2366940019 conn(0x563134aef020 msgr2=0x563134af1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2366940019 shutdown_connections 2024-04-25T10:24:44.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2366940019 wait complete. 2024-04-25T10:24:44.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- start start 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 Processor -- start 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b22270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563134904250 con 0x563134b3d2f0 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631348fa690 con 0x563134af30a0 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631348c8a40 con 0x563134b3c9d0 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 10 client.0 _ll_register_callbacks cb 0x563134adde90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b22270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb032a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b21d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b22270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52138/0 (socket says 172.21.15.45:52138) 2024-04-25T10:24:44.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 -- 172.21.15.45:0/2874907735 learned_addr learned my addr 172.21.15.45:0/2874907735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 -- 172.21.15.45:0/2874907735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 msgr2=0x563134b21d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b21d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 -- 172.21.15.45:0/2874907735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 msgr2=0x563134b217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 -- 172.21.15.45:0/2874907735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631349ed1c0 con 0x563134b3d2f0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb03be00640 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b22270 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb034052770 tx=0x7fb03405a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fed4382e456a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb028c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb028c00640 1 -- 172.21.15.45:0/2874907735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03405e770 con 0x563134b3d2f0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb028c00640 1 -- 172.21.15.45:0/2874907735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb034055760 con 0x563134b3d2f0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb02c026070 con 0x563134b3d2f0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb028c00640 1 -- 172.21.15.45:0/2874907735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0340546d0 con 0x563134b3d2f0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 10 client.7557 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5631349e9170 con 0x563134b3d2f0 2024-04-25T10:24:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb028c00640 1 -- 172.21.15.45:0/2874907735 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c026070 con 0x563134b3d2f0 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb028c00640 1 -- 172.21.15.45:0/2874907735 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb034074170 con 0x563134b3d2f0 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 10 client.7557 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 10 client.7557 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 20 client.7557 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134b4cc80 0x563134b4f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 4 client.7557 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.240+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631348dbc60 con 0x563134b4cc80 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb03b400640 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134b4cc80 0x563134b4f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb03b400640 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134b4cc80 0x563134b4f070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7fc8234a5a5443 server_cookie=bbf8777ffafbd900 in_seq=0 out_seq=0 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb028c00640 10 client.7557 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb028c00640 1 -- 172.21.15.45:0/2874907735 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631348dbc60 con 0x563134b4cc80 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb028c00640 10 client.7557 handle_command_reply: tid=0 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 client.7557 shutdown 2024-04-25T10:24:44.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 msgr2=0x563134b22270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b22270 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb034052770 tx=0x7fb03405a470 comp rx=0 tx=0).stop 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 shutdown_connections 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b22270 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b21d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134b4cc80 0x563134b4f070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 --2- 172.21.15.45:0/2874907735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 >> 172.21.15.45:0/2874907735 conn(0x563134aef020 msgr2=0x563134af1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb028c00640 0 client.7557 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 shutdown_connections 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 1 -- 172.21.15.45:0/2874907735 wait complete. 2024-04-25T10:24:44.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.244+0000 7fb0459d0000 20 client.7557 trim_cache size 0 max 16384 2024-04-25T10:24:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 Processor -- start 2024-04-25T10:24:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- start start 2024-04-25T10:24:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b99d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c203610 con 0x563134b3d2f0 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c163f10 con 0x563134af30a0 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0b1a40 con 0x563134b3c9d0 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b99d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b99d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52140/0 (socket says 172.21.15.45:52140) 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 -- 172.21.15.45:0/780066823 learned_addr learned my addr 172.21.15.45:0/780066823 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb032a00640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03b400640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 -- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 msgr2=0x563134b99840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b99840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 -- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 msgr2=0x563134b99300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b99300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 -- 172.21.15.45:0/780066823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563134986de0 con 0x563134b3d2f0 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb032a00640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03b400640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b99300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb03be00640 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b99d80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb03405e4d0 tx=0x7fb03406b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73aada5d5d22b66b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb02b400640 1 -- 172.21.15.45:0/780066823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034056310 con 0x563134b3d2f0 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb02b400640 1 -- 172.21.15.45:0/780066823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034082070 con 0x563134b3d2f0 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb02b400640 1 -- 172.21.15.45:0/780066823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03405e570 con 0x563134b3d2f0 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- 172.21.15.45:0/780066823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 msgr2=0x563134b99d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b99d80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb03405e4d0 tx=0x7fb03406b000 comp rx=0 tx=0).stop 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- 172.21.15.45:0/780066823 shutdown_connections 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3d2f0 0x563134b99d80 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134b3c9d0 0x563134b99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 --2- 172.21.15.45:0/780066823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134af30a0 0x563134b99300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- 172.21.15.45:0/780066823 >> 172.21.15.45:0/780066823 conn(0x563134aef020 msgr2=0x563134b49fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- 172.21.15.45:0/780066823 shutdown_connections 2024-04-25T10:24:44.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- 172.21.15.45:0/780066823 wait complete. 2024-04-25T10:24:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 -- start start 2024-04-25T10:24:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.260+0000 7fb0459d0000 1 Processor -- start 2024-04-25T10:24:44.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134b4f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b4f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134b4fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631348dae00 con 0x563134b3c9d0 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c163910 con 0x563134b3d2f0 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56313493f300 con 0x563134af30a0 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 10 client.0 _ll_register_callbacks cb 0x563134abe040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b4f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b4f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52144/0 (socket says 172.21.15.45:52144) 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 -- 172.21.15.45:0/1777807791 learned_addr learned my addr 172.21.15.45:0/1777807791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:44.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb03be00640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134b4fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb03b400640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134b4f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 -- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 msgr2=0x563134b4f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134b4f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 -- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 msgr2=0x563134b4fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134b4fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 -- 172.21.15.45:0/1777807791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631348e18e0 con 0x563134b3c9d0 2024-04-25T10:24:44.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb032a00640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b4f8c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b2c0 tx=0x7fb02401d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90d313e9e05f229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 1 -- 172.21.15.45:0/1777807791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240b4040 con 0x563134b3c9d0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 1 -- 172.21.15.45:0/1777807791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0240afd60 con 0x563134b3c9d0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb034082070 con 0x563134b3c9d0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 1 -- 172.21.15.45:0/1777807791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0240b9410 con 0x563134b3c9d0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 10 client.7563 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5631349998f0 con 0x563134b3c9d0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 1 -- 172.21.15.45:0/1777807791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034082070 con 0x563134b3c9d0 2024-04-25T10:24:44.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 1 -- 172.21.15.45:0/1777807791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb03c0cfe70 con 0x563134b3c9d0 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 10 client.7563 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 10 client.7563 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 20 client.7563 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134ba2f50 0x563134ba5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 4 client.7563 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56313494d530 con 0x563134ba2f50 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb03b400640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134ba2f50 0x563134ba5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb03b400640 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134ba2f50 0x563134ba5340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2fd9d9824f9386 server_cookie=34e1397689833b0 in_seq=0 out_seq=0 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 10 client.7563 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 1 -- 172.21.15.45:0/1777807791 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x56313494d530 con 0x563134ba2f50 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb028c00640 10 client.7563 handle_command_reply: tid=0 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 client.7563 shutdown 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 msgr2=0x563134b4f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b4f8c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b2c0 tx=0x7fb02401d430 comp rx=0 tx=0).stop 2024-04-25T10:24:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 shutdown_connections 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x563134ba2f50 0x563134ba5340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.264+0000 7fb0459d0000 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563134b3d2f0 0x563134b4fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb0459d0000 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563134b3c9d0 0x563134b4f8c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb0459d0000 1 --2- 172.21.15.45:0/1777807791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563134af30a0 0x563134b4f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb028c00640 0 client.7563 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 >> 172.21.15.45:0/1777807791 conn(0x563134aef020 msgr2=0x563134b3b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 shutdown_connections 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb0459d0000 1 -- 172.21.15.45:0/1777807791 wait complete. 2024-04-25T10:24:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb0459d0000 20 client.7563 trim_cache size 0 max 16384 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: "b88af451-20f3-4462-aeb7-214ecf0560fc": { 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "b88af451-20f3-4462-aeb7-214ecf0560fc", 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:24:44.275 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb018066f60 msgr2=0x7fb018069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb018066f60 0x7fb018069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098d0 tx=0x7fb02c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 msgr2=0x7fb03c0cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 0x7fb03c0cd110 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d30 tx=0x7fb03400c490 comp rx=0 tx=0).stop 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 shutdown_connections 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb018066f60 0x7fb018069420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15bac0 0x7fb03c0cd110 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0aef80 0x7fb03c15ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 --2- 172.21.15.45:0/811935363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c070350 0x7fb03c14a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 >> 172.21.15.45:0/811935363 conn(0x7fb03c0669a0 msgr2=0x7fb03c15ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 shutdown_connections 2024-04-25T10:24:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:44.268+0000 7fb042600640 1 -- 172.21.15.45:0/811935363 wait complete. 2024-04-25T10:24:49.289 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:24:49.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:24:49.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda95e00640 1 Processor -- start 2024-04-25T10:24:49.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda95e00640 1 -- start start 2024-04-25T10:24:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda95e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda900721f0 0x7fda90166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda95e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda90071360 con 0x7fda90159540 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda95e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda90159920 con 0x7fda900aeda0 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda95e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9015c2f0 con 0x7fda900721f0 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda90159540 0x7fda900ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48452/0 (socket says 172.21.15.45:48452) 2024-04-25T10:24:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8ea00640 1 -- 172.21.15.45:0/3442545004 learned_addr learned my addr 172.21.15.45:0/3442545004 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8f400640 1 --2- 172.21.15.45:0/3442545004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda900721f0 0x7fda90166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8e000640 1 -- 172.21.15.45:0/3442545004 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645072583 0 0) 0x7fda90071360 con 0x7fda90159540 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8e000640 1 -- 172.21.15.45:0/3442545004 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda74003530 con 0x7fda90159540 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8e000640 1 -- 172.21.15.45:0/3442545004 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497554991 0 0) 0x7fda90159920 con 0x7fda900aeda0 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8f400640 1 -- 172.21.15.45:0/3442545004 >> v1:172.21.15.117:6789/0 conn(0x7fda900aeda0 legacy=0x7fda90071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8f400640 1 -- 172.21.15.45:0/3442545004 >> v1:172.21.15.45:6789/0 conn(0x7fda90159540 legacy=0x7fda900ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.444+0000 7fda8f400640 1 -- 172.21.15.45:0/3442545004 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9015c5d0 con 0x7fda900721f0 2024-04-25T10:24:49.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda8f400640 1 --2- 172.21.15.45:0/3442545004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda900721f0 0x7fda90166920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda88009dc0 tx=0x7fda8801c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b69bf5c497f6d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda8e000640 1 -- 172.21.15.45:0/3442545004 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda880b2070 con 0x7fda900721f0 2024-04-25T10:24:49.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda8e000640 1 -- 172.21.15.45:0/3442545004 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda880a5620 con 0x7fda900721f0 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda8e000640 1 -- 172.21.15.45:0/3442545004 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800016a0 con 0x7fda900721f0 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 -- 172.21.15.45:0/3442545004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda900721f0 msgr2=0x7fda90166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 --2- 172.21.15.45:0/3442545004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda900721f0 0x7fda90166920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda88009dc0 tx=0x7fda8801c7a0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 -- 172.21.15.45:0/3442545004 shutdown_connections 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 --2- 172.21.15.45:0/3442545004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda900721f0 0x7fda90166920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 -- 172.21.15.45:0/3442545004 >> 172.21.15.45:0/3442545004 conn(0x7fda900669a0 msgr2=0x7fda90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 -- 172.21.15.45:0/3442545004 shutdown_connections 2024-04-25T10:24:49.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.448+0000 7fda95e00640 1 -- 172.21.15.45:0/3442545004 wait complete. 2024-04-25T10:24:49.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 Processor -- start 2024-04-25T10:24:49.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 -- start start 2024-04-25T10:24:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 0x7fda9014bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda900aeda0 0x7fda900afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda90159540 0x7fda900b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90169680 con 0x7fda900721f0 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9006c0d0 con 0x7fda900aeda0 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda90169800 con 0x7fda90159540 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 0x7fda9014bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 0x7fda9014bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36300/0 (socket says 172.21.15.45:36300) 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 -- 172.21.15.45:0/2973599178 learned_addr learned my addr 172.21.15.45:0/2973599178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8f400640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda90159540 0x7fda900b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda87e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda900aeda0 0x7fda900afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 -- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda90159540 msgr2=0x7fda900b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda90159540 0x7fda900b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 -- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda900aeda0 msgr2=0x7fda900afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda900aeda0 0x7fda900afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 -- 172.21.15.45:0/2973599178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900b1050 con 0x7fda900721f0 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda87e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda900aeda0 0x7fda900afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 0x7fda9014bd90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c7d0 tx=0x7fda8000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda87400640 1 -- 172.21.15.45:0/2973599178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80018070 con 0x7fda900721f0 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda87400640 1 -- 172.21.15.45:0/2973599178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80007500 con 0x7fda900721f0 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda900af360 con 0x7fda900721f0 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda87400640 1 -- 172.21.15.45:0/2973599178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80004a30 con 0x7fda900721f0 2024-04-25T10:24:49.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.452+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda900af920 con 0x7fda900721f0 2024-04-25T10:24:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda87400640 1 -- 172.21.15.45:0/2973599178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda8001b070 con 0x7fda900721f0 2024-04-25T10:24:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda87400640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda6c066f60 0x7fda6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda87400640 1 -- 172.21.15.45:0/2973599178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda80084760 con 0x7fda900721f0 2024-04-25T10:24:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda87e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda6c066f60 0x7fda6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 Processor -- start 2024-04-25T10:24:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 -- start start 2024-04-25T10:24:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda87e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda6c066f60 0x7fda6c069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda90166e30 tx=0x7fda78009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef22f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef1e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ef02dff0 con 0x5636ef22ee30 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eefb6150 con 0x5636ef22f750 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636ef02a090 con 0x5636ef1e54b0 2024-04-25T10:24:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.456+0000 7fda8ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef22f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef22f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36312/0 (socket says 172.21.15.45:36312) 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 -- 172.21.15.45:0/334469096 learned_addr learned my addr 172.21.15.45:0/334469096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8f400640 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef1e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda87e00640 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 -- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 msgr2=0x5636ef1e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef1e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 -- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 msgr2=0x5636ef1e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 -- 172.21.15.45:0/334469096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ef1c40d0 con 0x5636ef22ee30 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda8ea00640 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef22f210 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fda8000a430 tx=0x7fda80054a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88e6793382d35a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda7fe00640 1 -- 172.21.15.45:0/334469096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8005d6e0 con 0x5636ef22ee30 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda7fe00640 1 -- 172.21.15.45:0/334469096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda800532d0 con 0x5636ef22ee30 2024-04-25T10:24:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda7fe00640 1 -- 172.21.15.45:0/334469096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8005d250 con 0x5636ef22ee30 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- 172.21.15.45:0/334469096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 msgr2=0x5636ef22f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef22f210 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fda8000a430 tx=0x7fda80054a30 comp rx=0 tx=0).stop 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- 172.21.15.45:0/334469096 shutdown_connections 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef1e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- 172.21.15.45:0/334469096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef22f210 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- 172.21.15.45:0/334469096 >> 172.21.15.45:0/334469096 conn(0x5636ef1dd020 msgr2=0x5636ef1df440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- 172.21.15.45:0/334469096 shutdown_connections 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- 172.21.15.45:0/334469096 wait complete. 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- start start 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 Processor -- start 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef22e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef1eb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ef032620 con 0x5636ef22ee30 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eefe8690 con 0x5636ef22f750 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636eefb6a40 con 0x5636ef1e54b0 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda99271000 10 client.0 _ll_register_callbacks cb 0x5636ef1cbe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef1eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef1eb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36314/0 (socket says 172.21.15.45:36314) 2024-04-25T10:24:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.460+0000 7fda87e00640 1 -- 172.21.15.45:0/2774025268 learned_addr learned my addr 172.21.15.45:0/2774025268 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef22e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda8f400640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda87e00640 1 -- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 msgr2=0x5636ef22e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda87e00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef22e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda87e00640 1 -- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 msgr2=0x5636ef1eb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda87e00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1eb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda87e00640 1 -- 172.21.15.45:0/2774025268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636ef0db1c0 con 0x5636ef22ee30 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef22e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda87e00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef1eb360 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fda90070130 tx=0x7fda78001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106ac065df5021fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 1 -- 172.21.15.45:0/2774025268 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78027060 con 0x5636ef22ee30 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 1 -- 172.21.15.45:0/2774025268 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda78022b00 con 0x5636ef22ee30 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda800532d0 con 0x5636ef22ee30 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 1 -- 172.21.15.45:0/2774025268 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7800de80 con 0x5636ef22ee30 2024-04-25T10:24:49.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 10 client.7575 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5636ef0d7170 con 0x5636ef22ee30 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 1 -- 172.21.15.45:0/2774025268 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda800532d0 con 0x5636ef22ee30 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 1 -- 172.21.15.45:0/2774025268 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda78012620 con 0x5636ef22ee30 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 10 client.7575 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 10 client.7575 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 20 client.7575 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef1ec640 0x5636ef2410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 4 client.7575 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636eefc9c60 con 0x5636ef1ec640 2024-04-25T10:24:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef1ec640 0x5636ef2410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda8ea00640 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef1ec640 0x5636ef2410f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70fbbe817b765bd server_cookie=9624322eec26f423 in_seq=0 out_seq=0 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 10 client.7575 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 1 -- 172.21.15.45:0/2774025268 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5636eefc9c60 con 0x5636ef1ec640 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 10 client.7575 handle_command_reply: tid=0 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 client.7575 shutdown 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 msgr2=0x5636ef1eb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef1eb360 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fda90070130 tx=0x7fda78001090 comp rx=0 tx=0).stop 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 shutdown_connections 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef1ec640 0x5636ef2410f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef1eb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef1eb360 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 --2- 172.21.15.45:0/2774025268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef22e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda7cc00640 0 client.7575 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 >> 172.21.15.45:0/2774025268 conn(0x5636ef1dd020 msgr2=0x5636ef1e6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 shutdown_connections 2024-04-25T10:24:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 1 -- 172.21.15.45:0/2774025268 wait complete. 2024-04-25T10:24:49.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.464+0000 7fda99271000 20 client.7575 trim_cache size 0 max 16384 2024-04-25T10:24:49.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 Processor -- start 2024-04-25T10:24:49.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 -- start start 2024-04-25T10:24:49.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef20db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef243110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef243650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90157300 con 0x5636ef22ee30 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900b44b0 con 0x5636ef22f750 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9014cef0 con 0x5636ef1e54b0 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef243110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef243110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36330/0 (socket says 172.21.15.45:36330) 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 -- 172.21.15.45:0/1917902082 learned_addr learned my addr 172.21.15.45:0/1917902082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 -- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 msgr2=0x5636ef20db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda8f400640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef243650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda8ea00640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef20db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef20db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 -- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 msgr2=0x5636ef243650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef243650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda87e00640 1 -- 172.21.15.45:0/1917902082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ef074de0 con 0x5636ef22ee30 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda8f400640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef243650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.480+0000 7fda8ea00640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef20db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda87e00640 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef243110 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fda78009010 tx=0x7fda78016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119fc106d7f4b4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda7f400640 1 -- 172.21.15.45:0/1917902082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78031030 con 0x5636ef22ee30 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda7f400640 1 -- 172.21.15.45:0/1917902082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda78018040 con 0x5636ef22ee30 2024-04-25T10:24:49.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda7f400640 1 -- 172.21.15.45:0/1917902082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7800d570 con 0x5636ef22ee30 2024-04-25T10:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- 172.21.15.45:0/1917902082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 msgr2=0x5636ef243110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef243110 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fda78009010 tx=0x7fda78016070 comp rx=0 tx=0).stop 2024-04-25T10:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- 172.21.15.45:0/1917902082 shutdown_connections 2024-04-25T10:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22f750 0x5636ef243650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef22ee30 0x5636ef243110 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- 172.21.15.45:0/1917902082 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef1e54b0 0x5636ef20db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- 172.21.15.45:0/1917902082 >> 172.21.15.45:0/1917902082 conn(0x5636ef1dd020 msgr2=0x5636ef23bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- 172.21.15.45:0/1917902082 shutdown_connections 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- 172.21.15.45:0/1917902082 wait complete. 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- start start 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 Processor -- start 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 0x5636ef243650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22ee30 0x5636ef2410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef22f750 0x5636ef241630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636eefc8e00 con 0x5636ef1e54b0 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900b3eb0 con 0x5636ef22ee30 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636ef02d300 con 0x5636ef22f750 2024-04-25T10:24:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 10 client.0 _ll_register_callbacks cb 0x5636ef1ac040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 0x5636ef243650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 0x5636ef243650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36334/0 (socket says 172.21.15.45:36334) 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 -- 172.21.15.45:0/208143557 learned_addr learned my addr 172.21.15.45:0/208143557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda87e00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22ee30 0x5636ef2410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8f400640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef22f750 0x5636ef241630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 -- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef22f750 msgr2=0x5636ef241630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef22f750 0x5636ef241630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 -- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22ee30 msgr2=0x5636ef2410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22ee30 0x5636ef2410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 -- 172.21.15.45:0/208143557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636eefcf8e0 con 0x5636ef1e54b0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda87e00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22ee30 0x5636ef2410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda8ea00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 0x5636ef243650 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fda80051430 tx=0x7fda80000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5b8e13ca2eb2f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:24:49.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda7cc00640 1 -- 172.21.15.45:0/208143557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80079060 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda7cc00640 1 -- 172.21.15.45:0/208143557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda80051640 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.484+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda78018040 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 1 -- 172.21.15.45:0/208143557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda8005d6e0 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 10 client.7581 fetch_fsmap learned FSMap version 65 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x5636ef0878f0 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 1 -- 172.21.15.45:0/208143557 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda78018040 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 1 -- 172.21.15.45:0/208143557 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda8005d250 con 0x5636ef1e54b0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 10 client.7581 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 10 client.7581 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 20 client.7581 populate_metadata read hostname 'smithi045' 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef241ff0 0x5636ef2423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 4 client.7581 mds_command: new command op to 7344 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 --> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5636ef03b530 con 0x5636ef241ff0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda87e00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef241ff0 0x5636ef2423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda87e00640 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef241ff0 0x5636ef2423d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc9444eae249b48 server_cookie=33eba2ea967bcfd7 in_seq=0 out_seq=0 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 10 client.7581 ms_handle_connect on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 1 -- 172.21.15.45:0/208143557 <== mds.0 v2:172.21.15.45:6832/4102215497 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5636ef03b530 con 0x5636ef241ff0 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 10 client.7581 handle_command_reply: tid=0 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 client.7581 shutdown 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 msgr2=0x5636ef243650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 0x5636ef243650 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fda80051430 tx=0x7fda80000ee0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 shutdown_connections 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5636ef22f750 0x5636ef241630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:6832/4102215497,v1:172.21.15.45:6834/4102215497] conn(0x5636ef241ff0 0x5636ef2423d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5636ef22ee30 0x5636ef2410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 --2- 172.21.15.45:0/208143557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636ef1e54b0 0x5636ef243650 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda7cc00640 0 client.7581 ms_handle_reset on v2:172.21.15.45:6832/4102215497 2024-04-25T10:24:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 >> 172.21.15.45:0/208143557 conn(0x5636ef1dd020 msgr2=0x5636ef23c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 shutdown_connections 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 1 -- 172.21.15.45:0/208143557 wait complete. 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda99271000 20 client.7581 trim_cache size 0 max 16384 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:24:49.494 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:24:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda6c066f60 msgr2=0x7fda6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda6c066f60 0x7fda6c069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda90166e30 tx=0x7fda78009040 comp rx=0 tx=0).stop 2024-04-25T10:24:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 msgr2=0x7fda9014bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 0x7fda9014bd90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c7d0 tx=0x7fda8000cc90 comp rx=0 tx=0).stop 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 shutdown_connections 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda6c066f60 0x7fda6c069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda90159540 0x7fda900b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda900aeda0 0x7fda900afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 --2- 172.21.15.45:0/2973599178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda900721f0 0x7fda9014bd90 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 >> 172.21.15.45:0/2973599178 conn(0x7fda900669a0 msgr2=0x7fda900cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 shutdown_connections 2024-04-25T10:24:49.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.488+0000 7fda95e00640 1 -- 172.21.15.45:0/2973599178 wait complete. 2024-04-25T10:24:49.509 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:24:49.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:49.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 Processor -- start 2024-04-25T10:24:49.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 -- start start 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60ec15bc40 0x7f60ec06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60ec0cda10 0x7f60ec0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec0b2770 con 0x7f60ec0cf930 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec06c670 con 0x7f60ec15bc40 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60f2000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec06c950 con 0x7f60ec0cda10 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60eb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60ec15bc40 0x7f60ec06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60eb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60ec15bc40 0x7f60ec06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36348/0 (socket says 172.21.15.45:36348) 2024-04-25T10:24:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60eb400640 1 -- 172.21.15.45:0/4266351060 learned_addr learned my addr 172.21.15.45:0/4266351060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.648+0000 7f60ebe00640 1 --1- 172.21.15.45:0/4266351060 >> v1:172.21.15.45:6789/0 conn(0x7f60ec0cf930 0x7f60ec15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48462/0 (socket says 172.21.15.45:48462) 2024-04-25T10:24:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60e2a00640 1 --2- 172.21.15.45:0/4266351060 >> v2:172.21.15.117:3300/0 conn(0x7f60ec0cda10 0x7f60ec0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eb400640 1 -- 172.21.15.45:0/4266351060 >> v2:172.21.15.117:3300/0 conn(0x7f60ec0cda10 msgr2=0x7f60ec0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eb400640 1 --2- 172.21.15.45:0/4266351060 >> v2:172.21.15.117:3300/0 conn(0x7f60ec0cda10 0x7f60ec0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eb400640 1 -- 172.21.15.45:0/4266351060 >> v1:172.21.15.45:6789/0 conn(0x7f60ec0cf930 legacy=0x7f60ec15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eb400640 1 -- 172.21.15.45:0/4266351060 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec06cc30 con 0x7f60ec15bc40 2024-04-25T10:24:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eaa00640 1 -- 172.21.15.45:0/4266351060 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025881574 0 0) 0x7f60ec0b2770 con 0x7f60ec0cf930 2024-04-25T10:24:49.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60e2a00640 1 --2- 172.21.15.45:0/4266351060 >> v2:172.21.15.117:3300/0 conn(0x7f60ec0cda10 0x7f60ec0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eb400640 1 --2- 172.21.15.45:0/4266351060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec15bc40 0x7f60ec06c090 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0726f0 tx=0x7f60dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8444fb8788de328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eaa00640 1 -- 172.21.15.45:0/4266351060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0ab430 con 0x7f60ec15bc40 2024-04-25T10:24:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eaa00640 1 -- 172.21.15.45:0/4266351060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc0b0070 con 0x7f60ec15bc40 2024-04-25T10:24:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60eaa00640 1 -- 172.21.15.45:0/4266351060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0ab7f0 con 0x7f60ec15bc40 2024-04-25T10:24:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60f2000640 1 -- 172.21.15.45:0/4266351060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec15bc40 msgr2=0x7f60ec06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.652+0000 7f60f2000640 1 --2- 172.21.15.45:0/4266351060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec15bc40 0x7f60ec06c090 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0726f0 tx=0x7f60dc01c520 comp rx=0 tx=0).stop 2024-04-25T10:24:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- 172.21.15.45:0/4266351060 shutdown_connections 2024-04-25T10:24:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 --2- 172.21.15.45:0/4266351060 >> v2:172.21.15.117:3300/0 conn(0x7f60ec0cda10 0x7f60ec0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 --2- 172.21.15.45:0/4266351060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec15bc40 0x7f60ec06c090 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- 172.21.15.45:0/4266351060 >> 172.21.15.45:0/4266351060 conn(0x7f60ec066b70 msgr2=0x7f60ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- 172.21.15.45:0/4266351060 shutdown_connections 2024-04-25T10:24:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- 172.21.15.45:0/4266351060 wait complete. 2024-04-25T10:24:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 Processor -- start 2024-04-25T10:24:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- start start 2024-04-25T10:24:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 0x7f60ec160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ec0cf930 0x7f60ec156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ec15bc40 0x7f60ec157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec168b50 con 0x7f60ec0cda10 2024-04-25T10:24:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec1689d0 con 0x7f60ec0cf930 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec168cd0 con 0x7f60ec15bc40 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 0x7f60ec160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 0x7f60ec160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36360/0 (socket says 172.21.15.45:36360) 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 -- 172.21.15.45:0/2145731904 learned_addr learned my addr 172.21.15.45:0/2145731904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 -- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ec15bc40 msgr2=0x7f60ec157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ec15bc40 0x7f60ec157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 -- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ec0cf930 msgr2=0x7f60ec156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ec0cf930 0x7f60ec156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 -- 172.21.15.45:0/2145731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec1569c0 con 0x7f60ec0cda10 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60eb400640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 0x7f60ec160dd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0ad040 tx=0x7f60dc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0ba070 con 0x7f60ec0cda10 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc0abdd0 con 0x7f60ec0cda10 2024-04-25T10:24:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc004230 con 0x7f60ec0cda10 2024-04-25T10:24:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.656+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec1f7d70 con 0x7f60ec0cda10 2024-04-25T10:24:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec1f8680 con 0x7f60ec0cda10 2024-04-25T10:24:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec04ef30 con 0x7f60ec0cda10 2024-04-25T10:24:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60dc003620 con 0x7f60ec0cda10 2024-04-25T10:24:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60e3e00640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f60c8066f60 0x7f60c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60dc0a8030 con 0x7f60ec0cda10 2024-04-25T10:24:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60e2a00640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f60c8066f60 0x7f60c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.660+0000 7f60e2a00640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f60c8066f60 0x7f60c8069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f60ec1fa6e0 tx=0x7f60d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.664+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60dc016110 con 0x7f60ec0cda10 2024-04-25T10:24:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.864+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60ec06d960 con 0x7f60ec0cda10 2024-04-25T10:24:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.864+0000 7f60e3e00640 1 -- 172.21.15.45:0/2145731904 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f60dc0f0260 con 0x7f60ec0cda10 2024-04-25T10:24:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:49.868 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f60c8066f60 msgr2=0x7f60c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f60c8066f60 0x7f60c8069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f60ec1fa6e0 tx=0x7f60d4008040 comp rx=0 tx=0).stop 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 msgr2=0x7f60ec160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 0x7f60ec160dd0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0ad040 tx=0x7f60dc01ad60 comp rx=0 tx=0).stop 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 shutdown_connections 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f60c8066f60 0x7f60c8069420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ec15bc40 0x7f60ec157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ec0cf930 0x7f60ec156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 --2- 172.21.15.45:0/2145731904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ec0cda10 0x7f60ec160dd0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 >> 172.21.15.45:0/2145731904 conn(0x7f60ec066b70 msgr2=0x7f60ec0aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 shutdown_connections 2024-04-25T10:24:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:49.868+0000 7f60f2000640 1 -- 172.21.15.45:0/2145731904 wait complete. 2024-04-25T10:24:49.889 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:24:49.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:24:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.028+0000 7f8e1be00640 1 Processor -- start 2024-04-25T10:24:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.028+0000 7f8e1be00640 1 -- start start 2024-04-25T10:24:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e1be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8e140ae020 0x7f8e140ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e1be00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e140ad840 con 0x7f8e14070c20 2024-04-25T10:24:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e1be00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1415e9f0 con 0x7f8e140cf930 2024-04-25T10:24:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e1be00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1415bb90 con 0x7f8e140ae020 2024-04-25T10:24:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e18a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8e14070c20 0x7f8e1415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48474/0 (socket says 172.21.15.45:48474) 2024-04-25T10:24:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e18a00640 1 -- 172.21.15.45:0/3215223155 learned_addr learned my addr 172.21.15.45:0/3215223155 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e19e00640 1 --2- 172.21.15.45:0/3215223155 >> v2:172.21.15.117:3300/0 conn(0x7f8e140ae020 0x7f8e140ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285790838 0 0) 0x7f8e140ad840 con 0x7f8e14070c20 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dfc003530 con 0x7f8e14070c20 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380701784 0 0) 0x7f8e1415e9f0 con 0x7f8e140cf930 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e140ad840 con 0x7f8e140cf930 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298051345 0 0) 0x7f8dfc003530 con 0x7f8e14070c20 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e19e00640 1 -- 172.21.15.45:0/3215223155 >> v1:172.21.15.117:6789/0 conn(0x7f8e140cf930 legacy=0x7f8e140cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e19e00640 1 -- 172.21.15.45:0/3215223155 >> v1:172.21.15.45:6789/0 conn(0x7f8e14070c20 legacy=0x7f8e1415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.032+0000 7f8e19e00640 1 -- 172.21.15.45:0/3215223155 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1415ecd0 con 0x7f8e140ae020 2024-04-25T10:24:50.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.036+0000 7f8e19e00640 1 --2- 172.21.15.45:0/3215223155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e140ae4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009d90 tx=0x7f8e0c01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6275ed94ced7f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.036+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0b2070 con 0x7f8e140ae020 2024-04-25T10:24:50.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.036+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0a56a0 con 0x7f8e140ae020 2024-04-25T10:24:50.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.036+0000 7f8e13e00640 1 -- 172.21.15.45:0/3215223155 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00001680 con 0x7f8e140ae020 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- 172.21.15.45:0/3215223155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 msgr2=0x7f8e140ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 --2- 172.21.15.45:0/3215223155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e140ae4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009d90 tx=0x7f8e0c01c860 comp rx=0 tx=0).stop 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- 172.21.15.45:0/3215223155 shutdown_connections 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 --2- 172.21.15.45:0/3215223155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e140ae4b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- 172.21.15.45:0/3215223155 >> 172.21.15.45:0/3215223155 conn(0x7f8e14066b70 msgr2=0x7f8e14066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- 172.21.15.45:0/3215223155 shutdown_connections 2024-04-25T10:24:50.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- 172.21.15.45:0/3215223155 wait complete. 2024-04-25T10:24:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 Processor -- start 2024-04-25T10:24:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- start start 2024-04-25T10:24:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 0x7f8e1415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e1415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e140cf930 0x7f8e141400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1416ba70 con 0x7f8e14070c20 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1416b8f0 con 0x7f8e140ae020 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e1be00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1416bbf0 con 0x7f8e140cf930 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 0x7f8e1415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 0x7f8e1415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36374/0 (socket says 172.21.15.45:36374) 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 -- 172.21.15.45:0/825131404 learned_addr learned my addr 172.21.15.45:0/825131404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e18a00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e1415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19e00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e140cf930 0x7f8e141400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 -- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e140cf930 msgr2=0x7f8e141400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e140cf930 0x7f8e141400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 -- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 msgr2=0x7f8e1415c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e1415c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 -- 172.21.15.45:0/825131404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1413fcf0 con 0x7f8e14070c20 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e18a00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e1415c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.040+0000 7f8e19400640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 0x7f8e1415bd10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c430 tx=0x7f8e0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08017070 con 0x7f8e14070c20 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e080052f0 con 0x7f8e14070c20 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e14159730 con 0x7f8e14070c20 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08003840 con 0x7f8e14070c20 2024-04-25T10:24:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e14140600 con 0x7f8e14070c20 2024-04-25T10:24:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1404ef30 con 0x7f8e14070c20 2024-04-25T10:24:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e0801a070 con 0x7f8e14070c20 2024-04-25T10:24:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e11600640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df4066f60 0x7f8df4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e08084390 con 0x7f8e14070c20 2024-04-25T10:24:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.044+0000 7f8e18a00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df4066f60 0x7f8df4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.048+0000 7f8e18a00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df4066f60 0x7f8df4069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e140ccc90 tx=0x7f8e00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:50.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.048+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e08054cb0 con 0x7f8e14070c20 2024-04-25T10:24:50.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.272+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8e140aec50 con 0x7f8e14070c20 2024-04-25T10:24:50.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.272+0000 7f8e11600640 1 -- 172.21.15.45:0/825131404 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f8e08058d50 con 0x7f8e14070c20 2024-04-25T10:24:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:50.274 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":434,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076607605,"bytes_used":2559803392,"bytes_avail":770534309888,"bytes_total":773094113280,"write_bytes_sec":46500125,"read_op_per_sec":0,"write_op_per_sec":46},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7344}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T10:24:48.969514+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df4066f60 msgr2=0x7f8df4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df4066f60 0x7f8df4069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e140ccc90 tx=0x7f8e00009040 comp rx=0 tx=0).stop 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 msgr2=0x7f8e1415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 0x7f8e1415bd10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c430 tx=0x7f8e0800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 shutdown_connections 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df4066f60 0x7f8df4069420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e140cf930 0x7f8e141400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e140ae020 0x7f8e1415c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 --2- 172.21.15.45:0/825131404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e14070c20 0x7f8e1415bd10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 >> 172.21.15.45:0/825131404 conn(0x7f8e14066b70 msgr2=0x7f8e140b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:50.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 shutdown_connections 2024-04-25T10:24:50.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:50.276+0000 7f8e1be00640 1 -- 172.21.15.45:0/825131404 wait complete. 2024-04-25T10:24:50.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:24:51.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:51.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 Processor -- start 2024-04-25T10:24:51.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 -- start start 2024-04-25T10:24:51.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce2c15bc40 0x7fce2c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce2c0cda10 0x7fce2c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c0b2770 con 0x7fce2c0cf930 2024-04-25T10:24:51.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c06c670 con 0x7fce2c0cda10 2024-04-25T10:24:51.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce30e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c06c950 con 0x7fce2c15bc40 2024-04-25T10:24:51.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce2c0cda10 0x7fce2c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce2c0cda10 0x7fce2c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36386/0 (socket says 172.21.15.45:36386) 2024-04-25T10:24:51.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 -- 172.21.15.45:0/2986586813 learned_addr learned my addr 172.21.15.45:0/2986586813 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29e00640 1 --2- 172.21.15.45:0/2986586813 >> v2:172.21.15.117:3300/0 conn(0x7fce2c15bc40 0x7fce2c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 -- 172.21.15.45:0/2986586813 >> v2:172.21.15.117:3300/0 conn(0x7fce2c15bc40 msgr2=0x7fce2c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 --2- 172.21.15.45:0/2986586813 >> v2:172.21.15.117:3300/0 conn(0x7fce2c15bc40 0x7fce2c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 -- 172.21.15.45:0/2986586813 >> v1:172.21.15.117:6789/0 conn(0x7fce2c0cf930 legacy=0x7fce2c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.448+0000 7fce29400640 1 -- 172.21.15.45:0/2986586813 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c06cc30 con 0x7fce2c0cda10 2024-04-25T10:24:51.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce29400640 1 --2- 172.21.15.45:0/2986586813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cda10 0x7fce2c0b4610 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0d0800 tx=0x7fce1801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19215258d0c5b428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce28a00640 1 -- 172.21.15.45:0/2986586813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce180ab430 con 0x7fce2c0cda10 2024-04-25T10:24:51.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce28a00640 1 -- 172.21.15.45:0/2986586813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce180b0070 con 0x7fce2c0cda10 2024-04-25T10:24:51.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce28a00640 1 -- 172.21.15.45:0/2986586813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce180ab7f0 con 0x7fce2c0cda10 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 -- 172.21.15.45:0/2986586813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cda10 msgr2=0x7fce2c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 --2- 172.21.15.45:0/2986586813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cda10 0x7fce2c0b4610 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0d0800 tx=0x7fce1801c520 comp rx=0 tx=0).stop 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 -- 172.21.15.45:0/2986586813 shutdown_connections 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 --2- 172.21.15.45:0/2986586813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cda10 0x7fce2c0b4610 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 --2- 172.21.15.45:0/2986586813 >> v2:172.21.15.117:3300/0 conn(0x7fce2c15bc40 0x7fce2c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 -- 172.21.15.45:0/2986586813 >> 172.21.15.45:0/2986586813 conn(0x7fce2c066b70 msgr2=0x7fce2c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 -- 172.21.15.45:0/2986586813 shutdown_connections 2024-04-25T10:24:51.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.452+0000 7fce30e00640 1 -- 172.21.15.45:0/2986586813 wait complete. 2024-04-25T10:24:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 Processor -- start 2024-04-25T10:24:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 -- start start 2024-04-25T10:24:51.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce2c0cda10 0x7fce2c1f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 0x7fce2c1f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce2c15bc40 0x7fce2c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c168b50 con 0x7fce2c0cf930 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1689d0 con 0x7fce2c15bc40 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c168cd0 con 0x7fce2c0cda10 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 0x7fce2c1f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 0x7fce2c1f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36392/0 (socket says 172.21.15.45:36392) 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 -- 172.21.15.45:0/4132304387 learned_addr learned my addr 172.21.15.45:0/4132304387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce2c0cda10 0x7fce2c1f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce2a800640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce2c15bc40 0x7fce2c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 -- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce2c0cda10 msgr2=0x7fce2c1f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce2c0cda10 0x7fce2c1f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 -- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce2c15bc40 msgr2=0x7fce2c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce2c15bc40 0x7fce2c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 -- 172.21.15.45:0/4132304387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0704b0 con 0x7fce2c0cf930 2024-04-25T10:24:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce2a800640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce2c15bc40 0x7fce2c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:51.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce29400640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 0x7fce2c1f9f90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fce180ad040 tx=0x7fce1801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce180be5f0 con 0x7fce2c0cf930 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce180be790 con 0x7fce2c0cf930 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c070750 con 0x7fce2c0cf930 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce180beb80 con 0x7fce2c0cf930 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.456+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c070d10 con 0x7fce2c0cf930 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.460+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c04ef30 con 0x7fce2c0cf930 2024-04-25T10:24:51.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.460+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce180b43f0 con 0x7fce2c0cf930 2024-04-25T10:24:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.460+0000 7fce22000640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce0c066f60 0x7fce0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.460+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce180a8070 con 0x7fce2c0cf930 2024-04-25T10:24:51.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.460+0000 7fce29e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce0c066f60 0x7fce0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.460+0000 7fce29e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce0c066f60 0x7fce0c069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007a60 tx=0x7fce1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.464+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce18016110 con 0x7fce2c0cf930 2024-04-25T10:24:51.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.656+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce2c0af1c0 con 0x7fce2c0cf930 2024-04-25T10:24:51.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce22000640 1 -- 172.21.15.45:0/4132304387 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fce180f00c0 con 0x7fce2c0cf930 2024-04-25T10:24:51.660 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:51.660 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:51.660 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:51.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce0c066f60 msgr2=0x7fce0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce0c066f60 0x7fce0c069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007a60 tx=0x7fce1c008040 comp rx=0 tx=0).stop 2024-04-25T10:24:51.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 msgr2=0x7fce2c1f9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 0x7fce2c1f9f90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fce180ad040 tx=0x7fce1801ad60 comp rx=0 tx=0).stop 2024-04-25T10:24:51.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 shutdown_connections 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce0c066f60 0x7fce0c069420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce2c15bc40 0x7fce2c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce2c0cf930 0x7fce2c1f9f90 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 --2- 172.21.15.45:0/4132304387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce2c0cda10 0x7fce2c1f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 >> 172.21.15.45:0/4132304387 conn(0x7fce2c066b70 msgr2=0x7fce2c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 shutdown_connections 2024-04-25T10:24:51.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.660+0000 7fce30e00640 1 -- 172.21.15.45:0/4132304387 wait complete. 2024-04-25T10:24:51.681 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:24:51.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:24:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4699e00640 1 Processor -- start 2024-04-25T10:24:51.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4699e00640 1 -- start start 2024-04-25T10:24:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4699e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46940cda10 0x7f469406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4699e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46940b2770 con 0x7f469415bc40 2024-04-25T10:24:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4699e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469406c670 con 0x7f46940cf930 2024-04-25T10:24:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4699e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469406c950 con 0x7f46940cda10 2024-04-25T10:24:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f468a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f46940cda10 0x7f469406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4692a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f469415bc40 0x7f46940b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48476/0 (socket says 172.21.15.45:48476) 2024-04-25T10:24:51.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4692a00640 1 -- 172.21.15.45:0/3989519862 learned_addr learned my addr 172.21.15.45:0/3989519862 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:51.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f468a000640 1 -- 172.21.15.45:0/3989519862 >> v1:172.21.15.117:6789/0 conn(0x7f46940cf930 legacy=0x7f46940b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:51.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f468a000640 1 -- 172.21.15.45:0/3989519862 >> v1:172.21.15.45:6789/0 conn(0x7f469415bc40 legacy=0x7f46940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f468a000640 1 -- 172.21.15.45:0/3989519862 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469406cc30 con 0x7f46940cda10 2024-04-25T10:24:51.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.820+0000 7f4692000640 1 -- 172.21.15.45:0/3989519862 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635371869 0 0) 0x7f46940b2770 con 0x7f469415bc40 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f468a000640 1 --2- 172.21.15.45:0/3989519862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469406c090 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097c0 tx=0x7f467c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5ded0a3975b1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4692000640 1 -- 172.21.15.45:0/3989519862 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0b0070 con 0x7f46940cda10 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4692000640 1 -- 172.21.15.45:0/3989519862 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0ab600 con 0x7f46940cda10 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4692000640 1 -- 172.21.15.45:0/3989519862 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0ab9d0 con 0x7f46940cda10 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 -- 172.21.15.45:0/3989519862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 msgr2=0x7f469406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 --2- 172.21.15.45:0/3989519862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469406c090 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097c0 tx=0x7f467c01c410 comp rx=0 tx=0).stop 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 -- 172.21.15.45:0/3989519862 shutdown_connections 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 --2- 172.21.15.45:0/3989519862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469406c090 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 -- 172.21.15.45:0/3989519862 >> 172.21.15.45:0/3989519862 conn(0x7f4694066b70 msgr2=0x7f4694066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 -- 172.21.15.45:0/3989519862 shutdown_connections 2024-04-25T10:24:51.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.824+0000 7f4699e00640 1 -- 172.21.15.45:0/3989519862 wait complete. 2024-04-25T10:24:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 Processor -- start 2024-04-25T10:24:51.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 -- start start 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46940cf930 0x7f4694155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f469415bc40 0x7f4694155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694169d70 con 0x7f46940cda10 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694169bf0 con 0x7f46940cf930 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4694169ef0 con 0x7f469415bc40 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469414ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36398/0 (socket says 172.21.15.45:36398) 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 -- 172.21.15.45:0/1609212415 learned_addr learned my addr 172.21.15.45:0/1609212415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4693400640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f469415bc40 0x7f4694155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f468a000640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46940cf930 0x7f4694155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 -- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f469415bc40 msgr2=0x7f4694155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f469415bc40 0x7f4694155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 -- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46940cf930 msgr2=0x7f4694155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46940cf930 0x7f4694155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 -- 172.21.15.45:0/1609212415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940b01a0 con 0x7f46940cda10 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4692a00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469414ad80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f468400c4a0 tx=0x7f468400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684018070 con 0x7f46940cda10 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840074e0 con 0x7f46940cda10 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684004a30 con 0x7f46940cda10 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469414f400 con 0x7f46940cda10 2024-04-25T10:24:51.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.828+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469414b2c0 con 0x7f46940cda10 2024-04-25T10:24:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.832+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469404ef30 con 0x7f46940cda10 2024-04-25T10:24:51.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.832+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f468401b070 con 0x7f46940cda10 2024-04-25T10:24:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.832+0000 7f468be00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4670066f60 0x7f4670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.832+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46840843a0 con 0x7f46940cda10 2024-04-25T10:24:51.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.832+0000 7f468a000640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4670066f60 0x7f4670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:51.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.832+0000 7f468a000640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4670066f60 0x7f4670069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f467c0ad000 tx=0x7f467c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:51.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:51.836+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46840553a0 con 0x7f46940cda10 2024-04-25T10:24:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f469406d960 con 0x7f46940cda10 2024-04-25T10:24:52.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f468be00640 1 -- 172.21.15.45:0/1609212415 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f4684054ca0 con 0x7f46940cda10 2024-04-25T10:24:52.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:52.061 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":436,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076607605,"bytes_used":2559803392,"bytes_avail":770534309888,"bytes_total":773094113280,"write_bytes_sec":22597750,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7344}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T10:24:48.969514+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:24:52.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4670066f60 msgr2=0x7f4670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4670066f60 0x7f4670069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f467c0ad000 tx=0x7f467c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 msgr2=0x7f469414ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469414ad80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f468400c4a0 tx=0x7f468400c960 comp rx=0 tx=0).stop 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 shutdown_connections 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4670066f60 0x7f4670069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f469415bc40 0x7f4694155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46940cf930 0x7f4694155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 --2- 172.21.15.45:0/1609212415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46940cda10 0x7f469414ad80 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 >> 172.21.15.45:0/1609212415 conn(0x7f4694066b70 msgr2=0x7f469415af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 shutdown_connections 2024-04-25T10:24:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:52.060+0000 7f4699e00640 1 -- 172.21.15.45:0/1609212415 wait complete. 2024-04-25T10:24:52.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:24:53.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:53.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.232+0000 7f0cc3a00640 1 Processor -- start 2024-04-25T10:24:53.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.232+0000 7f0cc3a00640 1 -- start start 2024-04-25T10:24:53.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc3a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0cbc15bc40 0x7f0cbc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc3a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0cbc0cda10 0x7f0cbc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc3a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cbc0b2770 con 0x7f0cbc0cf930 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc3a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc06c670 con 0x7f0cbc0cda10 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc3a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc06c950 con 0x7f0cbc15bc40 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0cbc0cda10 0x7f0cbc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0cbc0cda10 0x7f0cbc0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36402/0 (socket says 172.21.15.45:36402) 2024-04-25T10:24:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 -- 172.21.15.45:0/2827516716 learned_addr learned my addr 172.21.15.45:0/2827516716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:53.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc1000640 1 --2- 172.21.15.45:0/2827516716 >> v2:172.21.15.117:3300/0 conn(0x7f0cbc15bc40 0x7f0cbc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 -- 172.21.15.45:0/2827516716 >> v2:172.21.15.117:3300/0 conn(0x7f0cbc15bc40 msgr2=0x7f0cbc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 --2- 172.21.15.45:0/2827516716 >> v2:172.21.15.117:3300/0 conn(0x7f0cbc15bc40 0x7f0cbc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 -- 172.21.15.45:0/2827516716 >> v1:172.21.15.117:6789/0 conn(0x7f0cbc0cf930 legacy=0x7f0cbc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cbbe00640 1 -- 172.21.15.45:0/2827516716 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc06cc30 con 0x7f0cbc0cda10 2024-04-25T10:24:53.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.236+0000 7f0cc1000640 1 --2- 172.21.15.45:0/2827516716 >> v2:172.21.15.117:3300/0 conn(0x7f0cbc15bc40 0x7f0cbc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cbbe00640 1 --2- 172.21.15.45:0/2827516716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc0b4610 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0d0800 tx=0x7f0ca801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543b888ad015f62e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cbb400640 1 -- 172.21.15.45:0/2827516716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80ab5f0 con 0x7f0cbc0cda10 2024-04-25T10:24:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cbb400640 1 -- 172.21.15.45:0/2827516716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca80b0070 con 0x7f0cbc0cda10 2024-04-25T10:24:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cbb400640 1 -- 172.21.15.45:0/2827516716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80ab9b0 con 0x7f0cbc0cda10 2024-04-25T10:24:53.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 -- 172.21.15.45:0/2827516716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc0cda10 msgr2=0x7f0cbc0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/2827516716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc0b4610 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0d0800 tx=0x7f0ca801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 -- 172.21.15.45:0/2827516716 shutdown_connections 2024-04-25T10:24:53.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/2827516716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc0b4610 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/2827516716 >> v2:172.21.15.117:3300/0 conn(0x7f0cbc15bc40 0x7f0cbc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 -- 172.21.15.45:0/2827516716 >> 172.21.15.45:0/2827516716 conn(0x7f0cbc066b70 msgr2=0x7f0cbc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 -- 172.21.15.45:0/2827516716 shutdown_connections 2024-04-25T10:24:53.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 -- 172.21.15.45:0/2827516716 wait complete. 2024-04-25T10:24:53.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 Processor -- start 2024-04-25T10:24:53.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.240+0000 7f0cc3a00640 1 -- start start 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cbc0cf930 0x7f0cbc15ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 0x7f0cbc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc168b50 con 0x7f0cbc15bc40 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc1689d0 con 0x7f0cbc0cda10 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc168cd0 con 0x7f0cbc0cf930 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 0x7f0cbc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 0x7f0cbc0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36412/0 (socket says 172.21.15.45:36412) 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 -- 172.21.15.45:0/295833916 learned_addr learned my addr 172.21.15.45:0/295833916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1000640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cbbe00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cbc0cf930 0x7f0cbc15ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 -- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cbc0cf930 msgr2=0x7f0cbc15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cbc0cf930 0x7f0cbc15ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 -- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cbc0cda10 msgr2=0x7f0cbc15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc15e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 -- 172.21.15.45:0/295833916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc15c3e0 con 0x7f0cbc15bc40 2024-04-25T10:24:53.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc1a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 0x7f0cbc0ad840 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c7b0 tx=0x7f0cb400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4017070 con 0x7f0cbc15bc40 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb4003db0 con 0x7f0cbc15bc40 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40052b0 con 0x7f0cbc15bc40 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc15c6c0 con 0x7f0cbc15bc40 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc15cc80 con 0x7f0cbc15bc40 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc04ef30 con 0x7f0cbc15bc40 2024-04-25T10:24:53.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.244+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cb4005450 con 0x7f0cbc15bc40 2024-04-25T10:24:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.248+0000 7f0cb8c00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c9c066f60 0x7f0c9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.248+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cb4084540 con 0x7f0cbc15bc40 2024-04-25T10:24:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.248+0000 7f0cc1000640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c9c066f60 0x7f0c9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.248+0000 7f0cc1000640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c9c066f60 0x7f0c9c069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000ab60 tx=0x7f0cb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.252+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0cb4054ef0 con 0x7f0cbc15bc40 2024-04-25T10:24:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cbc0af980 con 0x7f0cbc15bc40 2024-04-25T10:24:53.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cb8c00640 1 -- 172.21.15.45:0/295833916 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0cb4058f90 con 0x7f0cbc15bc40 2024-04-25T10:24:53.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:53.445 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c9c066f60 msgr2=0x7f0c9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c9c066f60 0x7f0c9c069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000ab60 tx=0x7f0cb0008040 comp rx=0 tx=0).stop 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 msgr2=0x7f0cbc0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 0x7f0cbc0ad840 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c7b0 tx=0x7f0cb400cc70 comp rx=0 tx=0).stop 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 shutdown_connections 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c9c066f60 0x7f0c9c069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0cbc15bc40 0x7f0cbc0ad840 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cbc0cf930 0x7f0cbc15ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 --2- 172.21.15.45:0/295833916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cbc0cda10 0x7f0cbc15e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 >> 172.21.15.45:0/295833916 conn(0x7f0cbc066b70 msgr2=0x7f0cbc0ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 shutdown_connections 2024-04-25T10:24:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.444+0000 7f0cc3a00640 1 -- 172.21.15.45:0/295833916 wait complete. 2024-04-25T10:24:53.467 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:24:53.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:24:53.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 Processor -- start 2024-04-25T10:24:53.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 -- start start 2024-04-25T10:24:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b40071e50 0x7f8b401472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b400726b0 0x7f8b400ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4006c600 con 0x7f8b40146ed0 2024-04-25T10:24:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b40161980 con 0x7f8b400726b0 2024-04-25T10:24:53.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.604+0000 7f8b47400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40161bd0 con 0x7f8b40071e50 2024-04-25T10:24:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b400726b0 0x7f8b400ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b400726b0 0x7f8b400ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36422/0 (socket says 172.21.15.45:36422) 2024-04-25T10:24:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 -- 172.21.15.45:0/630770009 learned_addr learned my addr 172.21.15.45:0/630770009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b37e00640 1 --2- 172.21.15.45:0/630770009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b40071e50 0x7f8b401472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 -- 172.21.15.45:0/630770009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b40071e50 msgr2=0x7f8b401472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:53.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 --2- 172.21.15.45:0/630770009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b40071e50 0x7f8b401472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 -- 172.21.15.45:0/630770009 >> v1:172.21.15.117:6789/0 conn(0x7f8b40146ed0 legacy=0x7f8b40071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 -- 172.21.15.45:0/630770009 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40161eb0 con 0x7f8b400726b0 2024-04-25T10:24:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b45400640 1 --2- 172.21.15.45:0/630770009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b400726b0 0x7f8b400ad840 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009e50 tx=0x7f8b3801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fdeef40eb0a2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b3fe00640 1 -- 172.21.15.45:0/630770009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a58d0 con 0x7f8b400726b0 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b3fe00640 1 -- 172.21.15.45:0/630770009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b380a5a70 con 0x7f8b400726b0 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.608+0000 7f8b3fe00640 1 -- 172.21.15.45:0/630770009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a5e40 con 0x7f8b400726b0 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- 172.21.15.45:0/630770009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b400726b0 msgr2=0x7f8b400ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 --2- 172.21.15.45:0/630770009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b400726b0 0x7f8b400ad840 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009e50 tx=0x7f8b3801ca00 comp rx=0 tx=0).stop 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- 172.21.15.45:0/630770009 shutdown_connections 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 --2- 172.21.15.45:0/630770009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b400726b0 0x7f8b400ad840 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 --2- 172.21.15.45:0/630770009 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b40071e50 0x7f8b401472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- 172.21.15.45:0/630770009 >> 172.21.15.45:0/630770009 conn(0x7f8b400669a0 msgr2=0x7f8b40066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- 172.21.15.45:0/630770009 shutdown_connections 2024-04-25T10:24:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- 172.21.15.45:0/630770009 wait complete. 2024-04-25T10:24:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 Processor -- start 2024-04-25T10:24:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- start start 2024-04-25T10:24:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 0x7f8b4013c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b400726b0 0x7f8b40140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b40146ed0 0x7f8b4015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400ae590 con 0x7f8b40071e50 2024-04-25T10:24:53.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b40147c80 con 0x7f8b40146ed0 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b47400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400ae710 con 0x7f8b400726b0 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 0x7f8b4013c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 0x7f8b4013c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36424/0 (socket says 172.21.15.45:36424) 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 -- 172.21.15.45:0/125713312 learned_addr learned my addr 172.21.15.45:0/125713312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b37e00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b400726b0 0x7f8b40140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 -- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b400726b0 msgr2=0x7f8b40140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b400726b0 0x7f8b40140640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 -- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b40146ed0 msgr2=0x7f8b4015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b40146ed0 0x7f8b4015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 -- 172.21.15.45:0/125713312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40143150 con 0x7f8b40071e50 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b37e00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b400726b0 0x7f8b40140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b44a00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 0x7f8b4013c580 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c3a0 tx=0x7f8b3000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.612+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b300052d0 con 0x7f8b40071e50 2024-04-25T10:24:53.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30005470 con 0x7f8b40071e50 2024-04-25T10:24:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007260 con 0x7f8b40071e50 2024-04-25T10:24:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b40142e70 con 0x7f8b40071e50 2024-04-25T10:24:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4013bd80 con 0x7f8b40071e50 2024-04-25T10:24:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4004ef30 con 0x7f8b40071e50 2024-04-25T10:24:53.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b30007400 con 0x7f8b40071e50 2024-04-25T10:24:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b3d600640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b1c066f60 0x7f8b1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b30084560 con 0x7f8b40071e50 2024-04-25T10:24:53.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.616+0000 7f8b37e00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b1c066f60 0x7f8b1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:53.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.620+0000 7f8b37e00640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b1c066f60 0x7f8b1c069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b4013dd40 tx=0x7f8b28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:53.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.620+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b300554d0 con 0x7f8b40071e50 2024-04-25T10:24:53.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.836+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8b400b55f0 con 0x7f8b40071e50 2024-04-25T10:24:53.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b3d600640 1 -- 172.21.15.45:0/125713312 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f8b30054dd0 con 0x7f8b40071e50 2024-04-25T10:24:53.840 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:53.840 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":438,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076607605,"bytes_used":2559803392,"bytes_avail":770534309888,"bytes_total":773094113280,"write_bytes_sec":22597750,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7344}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T10:24:48.969514+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b1c066f60 msgr2=0x7f8b1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b1c066f60 0x7f8b1c069420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b4013dd40 tx=0x7f8b28008040 comp rx=0 tx=0).stop 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 msgr2=0x7f8b4013c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 0x7f8b4013c580 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c3a0 tx=0x7f8b3000c860 comp rx=0 tx=0).stop 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 shutdown_connections 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b1c066f60 0x7f8b1c069420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b40146ed0 0x7f8b4015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b400726b0 0x7f8b40140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 --2- 172.21.15.45:0/125713312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b40071e50 0x7f8b4013c580 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 >> 172.21.15.45:0/125713312 conn(0x7f8b400669a0 msgr2=0x7f8b40140f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 shutdown_connections 2024-04-25T10:24:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:53.840+0000 7f8b47400640 1 -- 172.21.15.45:0/125713312 wait complete. 2024-04-25T10:24:53.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:24:54.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:55.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.016+0000 7fbdff400640 1 Processor -- start 2024-04-25T10:24:55.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.016+0000 7fbdff400640 1 -- start start 2024-04-25T10:24:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdff400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdff400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbdf80b2fc0 0x7fbdf806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdff400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf8070ed0 con 0x7fbdf815a0e0 2024-04-25T10:24:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdff400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf80711b0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdff400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8071490 con 0x7fbdf8071f70 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfd400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbdf80b2fc0 0x7fbdf806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfca00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fbdf815a0e0 0x7fbdf815b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:53924/0 (socket says 172.21.15.45:53924) 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfca00640 1 -- 172.21.15.45:0/1761271565 learned_addr learned my addr 172.21.15.45:0/1761271565 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfd400640 1 -- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 msgr2=0x7fbdf80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfd400640 1 --2- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf80b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfd400640 1 -- 172.21.15.45:0/1761271565 >> v1:172.21.15.117:6789/0 conn(0x7fbdf815a0e0 legacy=0x7fbdf815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdfd400640 1 -- 172.21.15.45:0/1761271565 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8071770 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdf7400640 1 -- 172.21.15.45:0/1761271565 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562811609 0 0) 0x7fbdf8070ed0 con 0x7fbdf815a0e0 2024-04-25T10:24:55.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.020+0000 7fbdf7e00640 1 --2- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf80b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:24:55.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdfd400640 1 --2- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf806c090 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009d90 tx=0x7fbdf001da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1a512a6b280c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdf7400640 1 -- 172.21.15.45:0/1761271565 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00b3070 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdf7400640 1 -- 172.21.15.45:0/1761271565 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf00a69f0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdf7400640 1 -- 172.21.15.45:0/1761271565 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00a6dc0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 -- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf80b2fc0 msgr2=0x7fbdf806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 --2- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf806c090 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009d90 tx=0x7fbdf001da80 comp rx=0 tx=0).stop 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 -- 172.21.15.45:0/1761271565 shutdown_connections 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 --2- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf806c090 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 --2- 172.21.15.45:0/1761271565 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf80b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 -- 172.21.15.45:0/1761271565 >> 172.21.15.45:0/1761271565 conn(0x7fbdf8066b70 msgr2=0x7fbdf8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 -- 172.21.15.45:0/1761271565 shutdown_connections 2024-04-25T10:24:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 -- 172.21.15.45:0/1761271565 wait complete. 2024-04-25T10:24:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 Processor -- start 2024-04-25T10:24:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.024+0000 7fbdff400640 1 -- start start 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf81f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf815a0e0 0x7fbdf8152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8072350 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8070b50 con 0x7fbdf815a0e0 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80724d0 con 0x7fbdf8071f70 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf81f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf81f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54860/0 (socket says 172.21.15.45:54860) 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 -- 172.21.15.45:0/357100186 learned_addr learned my addr 172.21.15.45:0/357100186 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdfca00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdfd400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf815a0e0 0x7fbdf8152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 -- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 msgr2=0x7fbdf814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 -- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf815a0e0 msgr2=0x7fbdf8152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf815a0e0 0x7fbdf8152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 -- 172.21.15.45:0/357100186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf81f6bf0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf7e00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf81f66b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fbde400ec10 tx=0x7fbde400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4017070 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde400cbd0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf8151ff0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4013470 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf8151230 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf804ef30 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.028+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbde401a070 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.032+0000 7fbdf4c00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdd8066f90 0x7fbdd8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.032+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbde4084f10 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.032+0000 7fbdfca00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdd8066f90 0x7fbdd8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.032+0000 7fbdfca00640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdd8066f90 0x7fbdd8069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f70 tx=0x7fbdec007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.036+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbdf81f6bf0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.236+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdf80703e0 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.236+0000 7fbdf4c00640 1 -- 172.21.15.45:0/357100186 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fbde4055840 con 0x7fbdf80b2fc0 2024-04-25T10:24:55.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:55.238 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.236+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdd8066f90 msgr2=0x7fbdd8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.236+0000 7fbdff400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdd8066f90 0x7fbdd8069450 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f70 tx=0x7fbdec007240 comp rx=0 tx=0).stop 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.236+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 msgr2=0x7fbdf81f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.236+0000 7fbdff400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf81f66b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fbde400ec10 tx=0x7fbde400c520 comp rx=0 tx=0).stop 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 shutdown_connections 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdd8066f90 0x7fbdd8069450 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdf815a0e0 0x7fbdf8152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdf80b2fc0 0x7fbdf81f66b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 --2- 172.21.15.45:0/357100186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdf8071f70 0x7fbdf814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 >> 172.21.15.45:0/357100186 conn(0x7fbdf8066b70 msgr2=0x7fbdf80af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 shutdown_connections 2024-04-25T10:24:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.240+0000 7fbdff400640 1 -- 172.21.15.45:0/357100186 wait complete. 2024-04-25T10:24:55.259 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:24:55.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:24:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.396+0000 7f64e8200640 1 Processor -- start 2024-04-25T10:24:55.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.396+0000 7f64e8200640 1 -- start start 2024-04-25T10:24:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e8200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f64e00d1820 0x7f64e00d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e8200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00b1af0 con 0x7f64e015e050 2024-04-25T10:24:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e8200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00b1dd0 con 0x7f64e00b15b0 2024-04-25T10:24:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e8200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e00b20b0 con 0x7f64e00d1820 2024-04-25T10:24:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e4e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f64e015e050 0x7f64e00b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54660/0 (socket says 172.21.15.45:54660) 2024-04-25T10:24:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e4e00640 1 -- 172.21.15.45:0/3251228798 learned_addr learned my addr 172.21.15.45:0/3251228798 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e5800640 1 --2- 172.21.15.45:0/3251228798 >> v2:172.21.15.117:3300/0 conn(0x7f64e00d1820 0x7f64e00d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827044268 0 0) 0x7f64e00b1af0 con 0x7f64e015e050 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c8003530 con 0x7f64e015e050 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110800670 0 0) 0x7f64c8003530 con 0x7f64e015e050 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 >> v2:172.21.15.117:3300/0 conn(0x7f64e00d1820 msgr2=0x7f64e00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 --2- 172.21.15.45:0/3251228798 >> v2:172.21.15.117:3300/0 conn(0x7f64e00d1820 0x7f64e00d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 >> v1:172.21.15.117:6789/0 conn(0x7f64e00b15b0 legacy=0x7f64e00cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00b2390 con 0x7f64e015e050 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f64cc001680 con 0x7f64e015e050 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 >> v1:172.21.15.45:6789/0 conn(0x7f64e015e050 legacy=0x7f64e00b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c80039c0 0x7f64c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80043c0 0x7f64c8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64c8005090 0x7f64c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b20b0 con 0x7f64c8005090 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c800bff0 con 0x7f64c80043c0 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c800c1f0 con 0x7f64c80039c0 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e6200640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64c8005090 0x7f64c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e5800640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c80039c0 0x7f64c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e4e00640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80043c0 0x7f64c8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e6200640 1 -- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c80039c0 msgr2=0x7f64c8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e6200640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c80039c0 0x7f64c8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e6200640 1 -- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80043c0 msgr2=0x7f64c8004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e6200640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80043c0 0x7f64c8004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.400+0000 7f64e6200640 1 -- 172.21.15.45:0/3251228798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8003530 con 0x7f64c8005090 2024-04-25T10:24:55.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e6200640 1 -- 172.21.15.45:0/3251228798 reap_dead start 2024-04-25T10:24:55.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e6200640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64c8005090 0x7f64c800b9d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f64d8001f60 tx=0x7f64d801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04459958d9cbb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8001b60 con 0x7f64c8005090 2024-04-25T10:24:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d8003b80 con 0x7f64c8005090 2024-04-25T10:24:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64dfe00640 1 -- 172.21.15.45:0/3251228798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80050d0 con 0x7f64c8005090 2024-04-25T10:24:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 -- 172.21.15.45:0/3251228798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64c8005090 msgr2=0x7f64c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64c8005090 0x7f64c800b9d0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f64d8001f60 tx=0x7f64d801c880 comp rx=0 tx=0).stop 2024-04-25T10:24:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 -- 172.21.15.45:0/3251228798 shutdown_connections 2024-04-25T10:24:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 --2- 172.21.15.45:0/3251228798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64c8005090 0x7f64c800b9d0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 -- 172.21.15.45:0/3251228798 >> 172.21.15.45:0/3251228798 conn(0x7f64e009a9d0 msgr2=0x7f64e009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 -- 172.21.15.45:0/3251228798 shutdown_connections 2024-04-25T10:24:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.404+0000 7f64e8200640 1 -- 172.21.15.45:0/3251228798 wait complete. 2024-04-25T10:24:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 Processor -- start 2024-04-25T10:24:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- start start 2024-04-25T10:24:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c8005090 0x7f64e0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 0x7f64e0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e00ce100 0x7f64e00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b1020 con 0x7f64e0157960 2024-04-25T10:24:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b0ea0 con 0x7f64e00ce100 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b11a0 con 0x7f64c8005090 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 0x7f64e0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 0x7f64e0156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54876/0 (socket says 172.21.15.45:54876) 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 -- 172.21.15.45:0/2243922759 learned_addr learned my addr 172.21.15.45:0/2243922759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e5800640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c8005090 0x7f64e0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e6200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e00ce100 0x7f64e00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 -- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c8005090 msgr2=0x7f64e0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c8005090 0x7f64e0155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 -- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e00ce100 msgr2=0x7f64e00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e00ce100 0x7f64e00ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 -- 172.21.15.45:0/2243922759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e01f7010 con 0x7f64e0157960 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e6200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e00ce100 0x7f64e00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e5800640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c8005090 0x7f64e0155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e4e00640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 0x7f64e0156ea0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f64cc012510 tx=0x7f64cc0129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc01c070 con 0x7f64e0157960 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc014770 con 0x7f64e0157960 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e01f72f0 con 0x7f64e0157960 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc014b60 con 0x7f64e0157960 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e01f78b0 con 0x7f64e0157960 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.408+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e004ef30 con 0x7f64e0157960 2024-04-25T10:24:55.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.412+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64cc01f070 con 0x7f64e0157960 2024-04-25T10:24:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.412+0000 7f64dd600640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f64c0066f60 0x7f64c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.412+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64cc087a00 con 0x7f64e0157960 2024-04-25T10:24:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.412+0000 7f64e5800640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f64c0066f60 0x7f64c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:55.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.412+0000 7f64e5800640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f64c0066f60 0x7f64c0069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009b60 tx=0x7f64d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.416+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64cc058a30 con 0x7f64e0157960 2024-04-25T10:24:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.636+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f64e006b480 con 0x7f64e0157960 2024-04-25T10:24:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.636+0000 7f64dd600640 1 -- 172.21.15.45:0/2243922759 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f64cc058330 con 0x7f64e0157960 2024-04-25T10:24:55.638 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:55.638 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":439,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076607605,"bytes_used":2559803392,"bytes_avail":770534309888,"bytes_total":773094113280,"write_bytes_sec":22597780,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7344}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T10:24:48.969514+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f64c0066f60 msgr2=0x7f64c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f64c0066f60 0x7f64c0069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009b60 tx=0x7f64d4008040 comp rx=0 tx=0).stop 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 msgr2=0x7f64e0156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 0x7f64e0156ea0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f64cc012510 tx=0x7f64cc0129d0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 shutdown_connections 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f64c0066f60 0x7f64c0069420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64e00ce100 0x7f64e00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64e0157960 0x7f64e0156ea0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 --2- 172.21.15.45:0/2243922759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c8005090 0x7f64e0155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 >> 172.21.15.45:0/2243922759 conn(0x7f64e009a9d0 msgr2=0x7f64e00af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 shutdown_connections 2024-04-25T10:24:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:55.640+0000 7f64e8200640 1 -- 172.21.15.45:0/2243922759 wait complete. 2024-04-25T10:24:55.659 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:24:56.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:24:56.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e3800640 1 Processor -- start 2024-04-25T10:24:56.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e3800640 1 -- start start 2024-04-25T10:24:56.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e3800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f86dc1550b0 0x7f86dc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:56.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e3800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc071050 con 0x7f86dc0cfe70 2024-04-25T10:24:56.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e3800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc15d040 con 0x7f86dc15b910 2024-04-25T10:24:56.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e3800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc15bd80 con 0x7f86dc1550b0 2024-04-25T10:24:56.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e0e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f86dc1550b0 0x7f86dc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:56.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86d3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f86dc0cfe70 0x7f86dc15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54676/0 (socket says 172.21.15.45:54676) 2024-04-25T10:24:56.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86d3e00640 1 -- 172.21.15.45:0/2782660604 learned_addr learned my addr 172.21.15.45:0/2782660604 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:56.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.812+0000 7f86e0e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f86dc1550b0 0x7f86dc0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54890/0 (socket says 172.21.15.45:54890) 2024-04-25T10:24:56.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86e0e00640 1 -- 172.21.15.45:0/2782660604 >> v1:172.21.15.117:6789/0 conn(0x7f86dc15b910 legacy=0x7f86dc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:56.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86e0e00640 1 -- 172.21.15.45:0/2782660604 >> v1:172.21.15.45:6789/0 conn(0x7f86dc0cfe70 legacy=0x7f86dc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:56.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86e0e00640 1 -- 172.21.15.45:0/2782660604 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0712f0 con 0x7f86dc1550b0 2024-04-25T10:24:56.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86e0e00640 1 --2- 172.21.15.45:0/2782660604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc1550b0 0x7f86dc0cf930 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097f0 tx=0x7f86cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61737906427772e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:56.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86dbe00640 1 -- 172.21.15.45:0/2782660604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0b0070 con 0x7f86dc1550b0 2024-04-25T10:24:56.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86dbe00640 1 -- 172.21.15.45:0/2782660604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc0ab600 con 0x7f86dc1550b0 2024-04-25T10:24:56.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86dbe00640 1 -- 172.21.15.45:0/2782660604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0ab9d0 con 0x7f86dc1550b0 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86e3800640 1 -- 172.21.15.45:0/2782660604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc1550b0 msgr2=0x7f86dc0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.816+0000 7f86e3800640 1 --2- 172.21.15.45:0/2782660604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc1550b0 0x7f86dc0cf930 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097f0 tx=0x7f86cc01c460 comp rx=0 tx=0).stop 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- 172.21.15.45:0/2782660604 shutdown_connections 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 --2- 172.21.15.45:0/2782660604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc1550b0 0x7f86dc0cf930 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- 172.21.15.45:0/2782660604 >> 172.21.15.45:0/2782660604 conn(0x7f86dc066b70 msgr2=0x7f86dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- 172.21.15.45:0/2782660604 shutdown_connections 2024-04-25T10:24:56.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- 172.21.15.45:0/2782660604 wait complete. 2024-04-25T10:24:56.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 Processor -- start 2024-04-25T10:24:56.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- start start 2024-04-25T10:24:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 0x7f86dc0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86dc1550b0 0x7f86dc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86dc15b910 0x7f86dc14c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc06c090 con 0x7f86dc0cfe70 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0720b0 con 0x7f86dc1550b0 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e3800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc06c210 con 0x7f86dc15b910 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 0x7f86dc0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 0x7f86dc0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54898/0 (socket says 172.21.15.45:54898) 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 -- 172.21.15.45:0/4199637299 learned_addr learned my addr 172.21.15.45:0/4199637299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86d3e00640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86dc1550b0 0x7f86dc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e1800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86dc15b910 0x7f86dc14c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 -- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86dc15b910 msgr2=0x7f86dc14c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86dc15b910 0x7f86dc14c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 -- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86dc1550b0 msgr2=0x7f86dc147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86dc1550b0 0x7f86dc147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 -- 172.21.15.45:0/4199637299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc148490 con 0x7f86dc0cfe70 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.820+0000 7f86e0e00640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 0x7f86dc0cd4f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0ae040 tx=0x7f86cc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0b0050 con 0x7f86dc0cfe70 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc0bc070 con 0x7f86dc0cfe70 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0b5a50 con 0x7f86dc0cfe70 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc16a260 con 0x7f86dc0cfe70 2024-04-25T10:24:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc169190 con 0x7f86dc0cfe70 2024-04-25T10:24:56.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc04ef30 con 0x7f86dc0cfe70 2024-04-25T10:24:56.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86cc0ab4a0 con 0x7f86dc0cfe70 2024-04-25T10:24:56.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d9600640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f86b8066f60 0x7f86b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:56.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86cc11f840 con 0x7f86dc0cfe70 2024-04-25T10:24:56.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.824+0000 7f86d3e00640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f86b8066f60 0x7f86b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:56.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.828+0000 7f86d3e00640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f86b8066f60 0x7f86b8069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f86dc14cd80 tx=0x7f86c4007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:56.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:56.828+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86cc016050 con 0x7f86dc0cfe70 2024-04-25T10:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.028+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86dc14fe80 con 0x7f86dc0cfe70 2024-04-25T10:24:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.028+0000 7f86d9600640 1 -- 172.21.15.45:0/4199637299 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f86cc0f07b0 con 0x7f86dc0cfe70 2024-04-25T10:24:57.031 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:57.031 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:24:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:24:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f86b8066f60 msgr2=0x7f86b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f86b8066f60 0x7f86b8069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f86dc14cd80 tx=0x7f86c4007e90 comp rx=0 tx=0).stop 2024-04-25T10:24:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 msgr2=0x7f86dc0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 0x7f86dc0cd4f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0ae040 tx=0x7f86cc01aca0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 shutdown_connections 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f86b8066f60 0x7f86b8069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86dc15b910 0x7f86dc14c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86dc1550b0 0x7f86dc147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 --2- 172.21.15.45:0/4199637299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86dc0cfe70 0x7f86dc0cd4f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 >> 172.21.15.45:0/4199637299 conn(0x7f86dc066b70 msgr2=0x7f86dc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 shutdown_connections 2024-04-25T10:24:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.032+0000 7f86e3800640 1 -- 172.21.15.45:0/4199637299 wait complete. 2024-04-25T10:24:57.052 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:24:57.052 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:24:57.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 Processor -- start 2024-04-25T10:24:57.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 -- start start 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6f81550b0 0x7fd6f80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6f80cfe70 0x7fd6f815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f8071050 con 0x7fd6f815b740 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f815d040 con 0x7fd6f80cfe70 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6fcc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80712f0 con 0x7fd6f81550b0 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6f80cfe70 0x7fd6f815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6f80cfe70 0x7fd6f815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54912/0 (socket says 172.21.15.45:54912) 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 -- 172.21.15.45:0/3354421645 learned_addr learned my addr 172.21.15.45:0/3354421645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6f5e00640 1 --2- 172.21.15.45:0/3354421645 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81550b0 0x7fd6f80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 -- 172.21.15.45:0/3354421645 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81550b0 msgr2=0x7fd6f80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 --2- 172.21.15.45:0/3354421645 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81550b0 0x7fd6f80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 -- 172.21.15.45:0/3354421645 >> v1:172.21.15.117:6789/0 conn(0x7fd6f815b740 legacy=0x7fd6f806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.192+0000 7fd6efe00640 1 -- 172.21.15.45:0/3354421645 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80715d0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6efe00640 1 --2- 172.21.15.45:0/3354421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f815b200 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9d30b0a95dc8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6f5400640 1 -- 172.21.15.45:0/3354421645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b0070 con 0x7fd6f80cfe70 2024-04-25T10:24:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6f5400640 1 -- 172.21.15.45:0/3354421645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e00ab600 con 0x7fd6f80cfe70 2024-04-25T10:24:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6f5400640 1 -- 172.21.15.45:0/3354421645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00ab9d0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 -- 172.21.15.45:0/3354421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 msgr2=0x7fd6f815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/3354421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f815b200 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e001c520 comp rx=0 tx=0).stop 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 -- 172.21.15.45:0/3354421645 shutdown_connections 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/3354421645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f815b200 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/3354421645 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81550b0 0x7fd6f80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 -- 172.21.15.45:0/3354421645 >> 172.21.15.45:0/3354421645 conn(0x7fd6f8066b70 msgr2=0x7fd6f8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 -- 172.21.15.45:0/3354421645 shutdown_connections 2024-04-25T10:24:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.196+0000 7fd6fcc00640 1 -- 172.21.15.45:0/3354421645 wait complete. 2024-04-25T10:24:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 Processor -- start 2024-04-25T10:24:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- start start 2024-04-25T10:24:57.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f813faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f81550b0 0x7fd6f80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f815b740 0x7fd6f80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f815bcd0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f815bb50 con 0x7fd6f815b740 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f815c030 con 0x7fd6f81550b0 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f813faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f813faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54916/0 (socket says 172.21.15.45:54916) 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 -- 172.21.15.45:0/1152253451 learned_addr learned my addr 172.21.15.45:0/1152253451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6efe00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f81550b0 0x7fd6f80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f6800640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f815b740 0x7fd6f80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 -- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f81550b0 msgr2=0x7fd6f80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f81550b0 0x7fd6f80b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 -- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f815b740 msgr2=0x7fd6f80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f815b740 0x7fd6f80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 -- 172.21.15.45:0/1152253451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8140410 con 0x7fd6f80cfe70 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6efe00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f81550b0 0x7fd6f80b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6f5e00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f813faa0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800e9e0 tx=0x7fd6e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8017070 con 0x7fd6f80cfe70 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800c9d0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800cda0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80ce750 con 0x7fd6f80cfe70 2024-04-25T10:24:57.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f8140990 con 0x7fd6f80cfe70 2024-04-25T10:24:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.200+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f804ef30 con 0x7fd6f80cfe70 2024-04-25T10:24:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.204+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6e801a070 con 0x7fd6f80cfe70 2024-04-25T10:24:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.204+0000 7fd6ee000640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6d4066f60 0x7fd6d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.204+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6e8085540 con 0x7fd6f80cfe70 2024-04-25T10:24:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.204+0000 7fd6efe00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6d4066f60 0x7fd6d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.204+0000 7fd6efe00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6d4066f60 0x7fd6d4069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ad000 tx=0x7fd6e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:57.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.208+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6e80564b0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd6f80008e0 con 0x7fd6f80cfe70 2024-04-25T10:24:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6ee000640 1 -- 172.21.15.45:0/1152253451 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1574 (secure 0 0 0) 0x7fd6e805ae20 con 0x7fd6f80cfe70 2024-04-25T10:24:57.426 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:57.426 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076611830,"bytes_used":2559811584,"bytes_avail":770534301696,"bytes_total":773094113280,"write_bytes_sec":21112827,"read_op_per_sec":0,"write_op_per_sec":21},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7344}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:24:54.971509+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:24:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6d4066f60 msgr2=0x7fd6d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6d4066f60 0x7fd6d4069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ad000 tx=0x7fd6e0008040 comp rx=0 tx=0).stop 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 msgr2=0x7fd6f813faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f813faa0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800e9e0 tx=0x7fd6e800eea0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 shutdown_connections 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6d4066f60 0x7fd6d4069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f815b740 0x7fd6f80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f81550b0 0x7fd6f80b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 --2- 172.21.15.45:0/1152253451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6f80cfe70 0x7fd6f813faa0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.424+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 >> 172.21.15.45:0/1152253451 conn(0x7fd6f8066b70 msgr2=0x7fd6f806c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.428+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 shutdown_connections 2024-04-25T10:24:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.428+0000 7fd6fcc00640 1 -- 172.21.15.45:0/1152253451 wait complete. 2024-04-25T10:24:57.446 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:24:57.447 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:24:57.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:57.510 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:24:57.607 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:24:57.607 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:24:57.607 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:57.617 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:24:57.617 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:24:57.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:24:57.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.852+0000 7fb029600640 1 Processor -- start 2024-04-25T10:24:57.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.852+0000 7fb029600640 1 -- start start 2024-04-25T10:24:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb029600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb02415db40 0x7fb02406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb029600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb02415df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb029600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02415e460 con 0x7fb02406b2c0 2024-04-25T10:24:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb029600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0240b5780 con 0x7fb02415db40 2024-04-25T10:24:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb029600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240b5a60 con 0x7fb0240b5010 2024-04-25T10:24:57.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb02415db40 0x7fb02406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb02415db40 0x7fb02406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54930/0 (socket says 172.21.15.45:54930) 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb028c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb02406b2c0 0x7fb0240b6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54682/0 (socket says 172.21.15.45:54682) 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 -- 172.21.15.45:0/2007324534 learned_addr learned my addr 172.21.15.45:0/2007324534 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb01b400640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb02415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 -- 172.21.15.45:0/2007324534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 msgr2=0x7fb02415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb02415df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 -- 172.21.15.45:0/2007324534 >> v1:172.21.15.45:6789/0 conn(0x7fb02406b2c0 legacy=0x7fb0240b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 -- 172.21.15.45:0/2007324534 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240b5d40 con 0x7fb02415db40 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023400640 1 -- 172.21.15.45:0/2007324534 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667943547 0 0) 0x7fb02415e460 con 0x7fb02406b2c0 2024-04-25T10:24:57.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb01b400640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb02415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:57.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.856+0000 7fb023e00640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb02406ac90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb014009e50 tx=0x7fb01401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70658f64f5bdddbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:57.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb023400640 1 -- 172.21.15.45:0/2007324534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b4070 con 0x7fb02415db40 2024-04-25T10:24:57.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb023400640 1 -- 172.21.15.45:0/2007324534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0140af400 con 0x7fb02415db40 2024-04-25T10:24:57.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb023400640 1 -- 172.21.15.45:0/2007324534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140af7d0 con 0x7fb02415db40 2024-04-25T10:24:57.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- 172.21.15.45:0/2007324534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 msgr2=0x7fb02406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb02406ac90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb014009e50 tx=0x7fb01401f210 comp rx=0 tx=0).stop 2024-04-25T10:24:57.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- 172.21.15.45:0/2007324534 shutdown_connections 2024-04-25T10:24:57.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb02415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 --2- 172.21.15.45:0/2007324534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb02406ac90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- 172.21.15.45:0/2007324534 >> 172.21.15.45:0/2007324534 conn(0x7fb0240d17b0 msgr2=0x7fb0240d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:57.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- 172.21.15.45:0/2007324534 shutdown_connections 2024-04-25T10:24:57.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- 172.21.15.45:0/2007324534 wait complete. 2024-04-25T10:24:57.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 Processor -- start 2024-04-25T10:24:57.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- start start 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02406b2c0 0x7fb024151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb0240c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb0240c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240b5450 con 0x7fb02415db40 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240b6300 con 0x7fb02406b2c0 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb029600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02406b730 con 0x7fb0240b5010 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb0240c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb0240c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54934/0 (socket says 172.21.15.45:54934) 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 -- 172.21.15.45:0/1614594794 learned_addr learned my addr 172.21.15.45:0/1614594794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb01b400640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb0240c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb023e00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02406b2c0 0x7fb024151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 -- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 msgr2=0x7fb0240c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb0240c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 -- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02406b2c0 msgr2=0x7fb024151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02406b2c0 0x7fb024151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb028c00640 1 -- 172.21.15.45:0/1614594794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240ccab0 con 0x7fb02415db40 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.860+0000 7fb023e00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02406b2c0 0x7fb024151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb028c00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb0240c9890 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c480 tx=0x7fb01c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c018070 con 0x7fb02415db40 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c0074e0 con 0x7fb02415db40 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c004a30 con 0x7fb02415db40 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0240ccd90 con 0x7fb02415db40 2024-04-25T10:24:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0240cd350 con 0x7fb02415db40 2024-04-25T10:24:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0240cd550 con 0x7fb02415db40 2024-04-25T10:24:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb01c01b070 con 0x7fb02415db40 2024-04-25T10:24:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb020c00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb000066f60 0x7fb000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.864+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb01c0842a0 con 0x7fb02415db40 2024-04-25T10:24:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.868+0000 7fb023e00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb000066f60 0x7fb000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:57.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.868+0000 7fb023e00640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb000066f60 0x7fb000069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b1000 tx=0x7fb0140aa000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:57.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:57.868+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0240cd550 con 0x7fb02415db40 2024-04-25T10:24:58.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.036+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0240b34e0 con 0x7fb02415db40 2024-04-25T10:24:58.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.036+0000 7fb020c00640 1 -- 172.21.15.45:0/1614594794 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16660 (secure 0 0 0) 0x7fb01c055340 con 0x7fb02415db40 2024-04-25T10:24:58.039 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:24:58.040 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:18.823568+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:24:58.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb000066f60 msgr2=0x7fb000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb000066f60 0x7fb000069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0140b1000 tx=0x7fb0140aa000 comp rx=0 tx=0).stop 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 msgr2=0x7fb0240c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb0240c9890 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c480 tx=0x7fb01c00c940 comp rx=0 tx=0).stop 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 shutdown_connections 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb000066f60 0x7fb000069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02415db40 0x7fb0240c9890 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0240b5010 0x7fb0240c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 --2- 172.21.15.45:0/1614594794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02406b2c0 0x7fb024151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 >> 172.21.15.45:0/1614594794 conn(0x7fb0240d17b0 msgr2=0x7fb02415cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 shutdown_connections 2024-04-25T10:24:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.040+0000 7fb029600640 1 -- 172.21.15.45:0/1614594794 wait complete. 2024-04-25T10:24:58.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:24:15.637327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:24:16.209015+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:24:58.062 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:24:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.204+0000 7fa6fca00640 1 Processor -- start 2024-04-25T10:24:58.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.204+0000 7fa6fca00640 1 -- start start 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.204+0000 7fa6fca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6f80aef80 0x7fa6f815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.204+0000 7fa6fca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 0x7fa6f815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6fca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f815b080 con 0x7fa6f815bac0 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6fca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f815adb0 con 0x7fa6f80aef80 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6fca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815eb60 con 0x7fa6f8070350 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 0x7fa6f815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 0x7fa6f815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53706/0 (socket says 172.21.15.45:53706) 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 -- 172.21.15.45:0/3563936755 learned_addr learned my addr 172.21.15.45:0/3563936755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6f5e00640 1 --2- 172.21.15.45:0/3563936755 >> v2:172.21.15.117:3300/0 conn(0x7fa6f80aef80 0x7fa6f815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 -- 172.21.15.45:0/3563936755 >> v2:172.21.15.117:3300/0 conn(0x7fa6f80aef80 msgr2=0x7fa6f815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:58.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 --2- 172.21.15.45:0/3563936755 >> v2:172.21.15.117:3300/0 conn(0x7fa6f80aef80 0x7fa6f815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 -- 172.21.15.45:0/3563936755 >> v1:172.21.15.117:6789/0 conn(0x7fa6f815bac0 legacy=0x7fa6f80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 -- 172.21.15.45:0/3563936755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f815ee40 con 0x7fa6f8070350 2024-04-25T10:24:58.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.208+0000 7fa6efe00640 1 --2- 172.21.15.45:0/3563936755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 0x7fa6f815b580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009870 tx=0x7fa6e001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe0a20621dbabd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6f5400640 1 -- 172.21.15.45:0/3563936755 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00a9400 con 0x7fa6f8070350 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6f5400640 1 -- 172.21.15.45:0/3563936755 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e00a95a0 con 0x7fa6f8070350 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6f5400640 1 -- 172.21.15.45:0/3563936755 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00a9970 con 0x7fa6f8070350 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- 172.21.15.45:0/3563936755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 msgr2=0x7fa6f815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 --2- 172.21.15.45:0/3563936755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 0x7fa6f815b580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009870 tx=0x7fa6e001c560 comp rx=0 tx=0).stop 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- 172.21.15.45:0/3563936755 shutdown_connections 2024-04-25T10:24:58.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 --2- 172.21.15.45:0/3563936755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f8070350 0x7fa6f815b580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 --2- 172.21.15.45:0/3563936755 >> v2:172.21.15.117:3300/0 conn(0x7fa6f80aef80 0x7fa6f815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- 172.21.15.45:0/3563936755 >> 172.21.15.45:0/3563936755 conn(0x7fa6f80669a0 msgr2=0x7fa6f8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:58.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- 172.21.15.45:0/3563936755 shutdown_connections 2024-04-25T10:24:58.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- 172.21.15.45:0/3563936755 wait complete. 2024-04-25T10:24:58.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 Processor -- start 2024-04-25T10:24:58.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- start start 2024-04-25T10:24:58.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8070350 0x7fa6f8143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 0x7fa6f80ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f815bac0 0x7fa6f80b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f806c0d0 con 0x7fa6f80aef80 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b3ae0 con 0x7fa6f8070350 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6fca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f806c250 con 0x7fa6f815bac0 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 0x7fa6f80ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 0x7fa6f80ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54946/0 (socket says 172.21.15.45:54946) 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6efe00640 1 -- 172.21.15.45:0/1621906240 learned_addr learned my addr 172.21.15.45:0/1621906240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6f5e00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8070350 0x7fa6f8143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.212+0000 7fa6f6800640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f815bac0 0x7fa6f80b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6efe00640 1 -- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f815bac0 msgr2=0x7fa6f80b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6efe00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f815bac0 0x7fa6f80b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6efe00640 1 -- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8070350 msgr2=0x7fa6f8143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6efe00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8070350 0x7fa6f8143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6efe00640 1 -- 172.21.15.45:0/1621906240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80affe0 con 0x7fa6f80aef80 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6f6800640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f815bac0 0x7fa6f80b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:58.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6efe00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 0x7fa6f80ce360 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001a190 tx=0x7fa6e001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00a9400 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e00b1800 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b1bb0 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f80b02c0 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f80ad840 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f804ef30 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6e00a8360 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6ee000640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6d4066f60 0x7fa6d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.216+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6e011ece0 con 0x7fa6f80aef80 2024-04-25T10:24:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.220+0000 7fa6f5e00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6d4066f60 0x7fa6d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.220+0000 7fa6f5e00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6d4066f60 0x7fa6d4069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005bd0 tx=0x7fa6e800b750 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.220+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6e00c2070 con 0x7fa6f80aef80 2024-04-25T10:24:58.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.384+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6f80b2fa0 con 0x7fa6f80aef80 2024-04-25T10:24:58.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.384+0000 7fa6ee000640 1 -- 172.21.15.45:0/1621906240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6e00ad020 con 0x7fa6f80aef80 2024-04-25T10:24:58.386 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.384+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6d4066f60 msgr2=0x7fa6d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.384+0000 7fa6fca00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6d4066f60 0x7fa6d4069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005bd0 tx=0x7fa6e800b750 comp rx=0 tx=0).stop 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 msgr2=0x7fa6f80ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 0x7fa6f80ce360 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa6e001a190 tx=0x7fa6e001ada0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 shutdown_connections 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6d4066f60 0x7fa6d4069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6f815bac0 0x7fa6f80b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6f80aef80 0x7fa6f80ce360 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 --2- 172.21.15.45:0/1621906240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8070350 0x7fa6f8143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 >> 172.21.15.45:0/1621906240 conn(0x7fa6f80669a0 msgr2=0x7fa6f8156280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 shutdown_connections 2024-04-25T10:24:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.388+0000 7fa6fca00640 1 -- 172.21.15.45:0/1621906240 wait complete. 2024-04-25T10:24:58.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:24:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 Processor -- start 2024-04-25T10:24:58.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 -- start start 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb0c0d0d00 0x7feb0c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb0c06cf40 0x7feb0c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0cdab0 con 0x7feb0c0cc200 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c06d8b0 con 0x7feb0c06cf40 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.548+0000 7feb10e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0cd210 con 0x7feb0c0d0d00 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb0a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb0c06cf40 0x7feb0c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb03e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feb0c0cc200 0x7feb0c162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54686/0 (socket says 172.21.15.45:54686) 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb03e00640 1 -- 172.21.15.45:0/2638791881 learned_addr learned my addr 172.21.15.45:0/2638791881 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb09e00640 1 --2- 172.21.15.45:0/2638791881 >> v2:172.21.15.117:3300/0 conn(0x7feb0c0d0d00 0x7feb0c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb0a800640 1 -- 172.21.15.45:0/2638791881 >> v2:172.21.15.117:3300/0 conn(0x7feb0c0d0d00 msgr2=0x7feb0c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb0a800640 1 --2- 172.21.15.45:0/2638791881 >> v2:172.21.15.117:3300/0 conn(0x7feb0c0d0d00 0x7feb0c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb0a800640 1 -- 172.21.15.45:0/2638791881 >> v1:172.21.15.45:6789/0 conn(0x7feb0c0cc200 legacy=0x7feb0c162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb0a800640 1 -- 172.21.15.45:0/2638791881 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0cd4f0 con 0x7feb0c06cf40 2024-04-25T10:24:58.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb09400640 1 -- 172.21.15.45:0/2638791881 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139714863 0 0) 0x7feb0c0cdab0 con 0x7feb0c0cc200 2024-04-25T10:24:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb0a800640 1 --2- 172.21.15.45:0/2638791881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c06d370 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cdd40 tx=0x7feb0401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21939af43cf64c45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb09400640 1 -- 172.21.15.45:0/2638791881 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040b2070 con 0x7feb0c06cf40 2024-04-25T10:24:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb09400640 1 -- 172.21.15.45:0/2638791881 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb040a5850 con 0x7feb0c06cf40 2024-04-25T10:24:58.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.552+0000 7feb09400640 1 -- 172.21.15.45:0/2638791881 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040a5c20 con 0x7feb0c06cf40 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- 172.21.15.45:0/2638791881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 msgr2=0x7feb0c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 --2- 172.21.15.45:0/2638791881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c06d370 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cdd40 tx=0x7feb0401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- 172.21.15.45:0/2638791881 shutdown_connections 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 --2- 172.21.15.45:0/2638791881 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c06d370 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 --2- 172.21.15.45:0/2638791881 >> v2:172.21.15.117:3300/0 conn(0x7feb0c0d0d00 0x7feb0c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- 172.21.15.45:0/2638791881 >> 172.21.15.45:0/2638791881 conn(0x7feb0c072290 msgr2=0x7feb0c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- 172.21.15.45:0/2638791881 shutdown_connections 2024-04-25T10:24:58.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- 172.21.15.45:0/2638791881 wait complete. 2024-04-25T10:24:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 Processor -- start 2024-04-25T10:24:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- start start 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c099290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb0c0cc200 0x7feb0c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0c0d0d00 0x7feb0c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c162a10 con 0x7feb0c06cf40 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c162890 con 0x7feb0c0d0d00 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb10e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c15e040 con 0x7feb0c0cc200 2024-04-25T10:24:58.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c099290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c099290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54960/0 (socket says 172.21.15.45:54960) 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 -- 172.21.15.45:0/4112454486 learned_addr learned my addr 172.21.15.45:0/4112454486 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb0a800640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0c0d0d00 0x7feb0c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb03e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb0c0cc200 0x7feb0c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 -- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb0c0cc200 msgr2=0x7feb0c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb0c0cc200 0x7feb0c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 -- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0c0d0d00 msgr2=0x7feb0c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0c0d0d00 0x7feb0c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb09e00640 1 -- 172.21.15.45:0/4112454486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c15b130 con 0x7feb0c06cf40 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.556+0000 7feb03e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb0c0cc200 0x7feb0c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb09e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c099290 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7feafc00d5f0 tx=0x7feafc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0125f0 con 0x7feb0c06cf40 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc019070 con 0x7feb0c06cf40 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0129b0 con 0x7feb0c06cf40 2024-04-25T10:24:58.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0c5220 con 0x7feb0c06cf40 2024-04-25T10:24:58.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c15cdb0 con 0x7feb0c06cf40 2024-04-25T10:24:58.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c04ef30 con 0x7feb0c06cf40 2024-04-25T10:24:58.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feafc012b50 con 0x7feb0c06cf40 2024-04-25T10:24:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb02000640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae8066f60 0x7feae8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feafc0853b0 con 0x7feb0c06cf40 2024-04-25T10:24:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.560+0000 7feb03e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae8066f60 0x7feae8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.564+0000 7feb03e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae8066f60 0x7feae8069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feaf4007fd0 tx=0x7feaf4007c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.564+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feafc0563b0 con 0x7feb0c06cf40 2024-04-25T10:24:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.724+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb0c072ff0 con 0x7feb0c06cf40 2024-04-25T10:24:58.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.724+0000 7feb02000640 1 -- 172.21.15.45:0/4112454486 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7feafc055cb0 con 0x7feb0c06cf40 2024-04-25T10:24:58.727 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae8066f60 msgr2=0x7feae8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae8066f60 0x7feae8069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feaf4007fd0 tx=0x7feaf4007c20 comp rx=0 tx=0).stop 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 msgr2=0x7feb0c099290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c099290 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7feafc00d5f0 tx=0x7feafc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 shutdown_connections 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feae8066f60 0x7feae8069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb0c0d0d00 0x7feb0c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb0c0cc200 0x7feb0c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 --2- 172.21.15.45:0/4112454486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c06cf40 0x7feb0c099290 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 >> 172.21.15.45:0/4112454486 conn(0x7feb0c072290 msgr2=0x7feb0c0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:58.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 shutdown_connections 2024-04-25T10:24:58.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.728+0000 7feb10e00640 1 -- 172.21.15.45:0/4112454486 wait complete. 2024-04-25T10:24:58.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:24:58.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa24e000640 1 Processor -- start 2024-04-25T10:24:58.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa24e000640 1 -- start start 2024-04-25T10:24:58.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa24e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa24815dd30 0x7fa24806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa24e000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24806b1d0 con 0x7fa2480d1fb0 2024-04-25T10:24:58.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa24e000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24806b470 con 0x7fa2480cfb30 2024-04-25T10:24:58.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa24e000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24815e110 con 0x7fa24815dd30 2024-04-25T10:24:58.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa247e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa2480d1fb0 0x7fa2480b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54692/0 (socket says 172.21.15.45:54692) 2024-04-25T10:24:58.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa247e00640 1 -- 172.21.15.45:0/1192439839 learned_addr learned my addr 172.21.15.45:0/1192439839 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:58.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa247400640 1 --2- 172.21.15.45:0/1192439839 >> v2:172.21.15.117:3300/0 conn(0x7fa24815dd30 0x7fa24806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa246a00640 1 -- 172.21.15.45:0/1192439839 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591966475 0 0) 0x7fa24806b1d0 con 0x7fa2480d1fb0 2024-04-25T10:24:58.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa246a00640 1 -- 172.21.15.45:0/1192439839 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa22c003530 con 0x7fa2480d1fb0 2024-04-25T10:24:58.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa247400640 1 -- 172.21.15.45:0/1192439839 >> v1:172.21.15.117:6789/0 conn(0x7fa2480cfb30 legacy=0x7fa2480d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa247400640 1 -- 172.21.15.45:0/1192439839 >> v1:172.21.15.45:6789/0 conn(0x7fa2480d1fb0 legacy=0x7fa2480b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.892+0000 7fa247400640 1 -- 172.21.15.45:0/1192439839 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24806b750 con 0x7fa24815dd30 2024-04-25T10:24:58.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa247400640 1 --2- 172.21.15.45:0/1192439839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24815dd30 0x7fa24806ac90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa2380097c0 tx=0x7fa23801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17e0298c9c4d513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa246a00640 1 -- 172.21.15.45:0/1192439839 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2380b1070 con 0x7fa24815dd30 2024-04-25T10:24:58.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa246a00640 1 -- 172.21.15.45:0/1192439839 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2380ac5f0 con 0x7fa24815dd30 2024-04-25T10:24:58.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa246a00640 1 -- 172.21.15.45:0/1192439839 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2380ac9c0 con 0x7fa24815dd30 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 -- 172.21.15.45:0/1192439839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24815dd30 msgr2=0x7fa24806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 --2- 172.21.15.45:0/1192439839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24815dd30 0x7fa24806ac90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa2380097c0 tx=0x7fa23801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 -- 172.21.15.45:0/1192439839 shutdown_connections 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 --2- 172.21.15.45:0/1192439839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24815dd30 0x7fa24806ac90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 -- 172.21.15.45:0/1192439839 >> 172.21.15.45:0/1192439839 conn(0x7fa24809a9d0 msgr2=0x7fa24809adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 -- 172.21.15.45:0/1192439839 shutdown_connections 2024-04-25T10:24:58.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.896+0000 7fa24e000640 1 -- 172.21.15.45:0/1192439839 wait complete. 2024-04-25T10:24:58.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 Processor -- start 2024-04-25T10:24:58.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 -- start start 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2480cfb30 0x7fa2480c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2480d1fb0 0x7fa2480cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 0x7fa2480c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24816bf00 con 0x7fa24815dd30 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24816bd80 con 0x7fa2480cfb30 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24816c080 con 0x7fa2480d1fb0 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 0x7fa2480c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 0x7fa2480c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54962/0 (socket says 172.21.15.45:54962) 2024-04-25T10:24:58.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 -- 172.21.15.45:0/4136040631 learned_addr learned my addr 172.21.15.45:0/4136040631 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247400640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2480cfb30 0x7fa2480c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa23ea00640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2480d1fb0 0x7fa2480cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 -- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2480d1fb0 msgr2=0x7fa2480cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2480d1fb0 0x7fa2480cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 -- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2480cfb30 msgr2=0x7fa2480c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2480cfb30 0x7fa2480c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:58.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 -- 172.21.15.45:0/4136040631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2480afa90 con 0x7fa24815dd30 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa247e00640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 0x7fa2480c6b00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa24000cb60 tx=0x7fa24000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240017070 con 0x7fa24815dd30 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000ad40 con 0x7fa24815dd30 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240012600 con 0x7fa24815dd30 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2480afd70 con 0x7fa24815dd30 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.900+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2480b0330 con 0x7fa24815dd30 2024-04-25T10:24:58.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.904+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24804ef30 con 0x7fa24815dd30 2024-04-25T10:24:58.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.904+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa24001a070 con 0x7fa24815dd30 2024-04-25T10:24:58.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.904+0000 7fa23fe00640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa224066f60 0x7fa224069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:58.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.904+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa240084d70 con 0x7fa24815dd30 2024-04-25T10:24:58.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.904+0000 7fa247400640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa224066f60 0x7fa224069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:58.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.904+0000 7fa247400640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa224066f60 0x7fa224069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa2380ae000 tx=0x7fa2380a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:58.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:58.908+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa240055690 con 0x7fa24815dd30 2024-04-25T10:24:59.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.072+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2480d1580 con 0x7fa24815dd30 2024-04-25T10:24:59.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.072+0000 7fa23fe00640 1 -- 172.21.15.45:0/4136040631 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa240059730 con 0x7fa24815dd30 2024-04-25T10:24:59.074 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.072+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa224066f60 msgr2=0x7fa224069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.072+0000 7fa24e000640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa224066f60 0x7fa224069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa2380ae000 tx=0x7fa2380a7000 comp rx=0 tx=0).stop 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.072+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 msgr2=0x7fa2480c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.072+0000 7fa24e000640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 0x7fa2480c6b00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa24000cb60 tx=0x7fa24000a430 comp rx=0 tx=0).stop 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 shutdown_connections 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa224066f60 0x7fa224069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa24815dd30 0x7fa2480c6b00 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2480d1fb0 0x7fa2480cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 --2- 172.21.15.45:0/4136040631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2480cfb30 0x7fa2480c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 >> 172.21.15.45:0/4136040631 conn(0x7fa24809a9d0 msgr2=0x7fa2480cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 shutdown_connections 2024-04-25T10:24:59.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.076+0000 7fa24e000640 1 -- 172.21.15.45:0/4136040631 wait complete. 2024-04-25T10:24:59.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:24:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.236+0000 7fe5aca00640 1 Processor -- start 2024-04-25T10:24:59.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.236+0000 7fe5aca00640 1 -- start start 2024-04-25T10:24:59.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.236+0000 7fe5aca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5a80d0040 0x7fe5a815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.236+0000 7fe5aca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a8071050 con 0x7fe5a81550b0 2024-04-25T10:24:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.236+0000 7fe5aca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a815d040 con 0x7fe5a815b740 2024-04-25T10:24:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.236+0000 7fe5aca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a80712f0 con 0x7fe5a80d0040 2024-04-25T10:24:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe5a5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5a81550b0 0x7fe5a80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54698/0 (socket says 172.21.15.45:54698) 2024-04-25T10:24:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe5a5e00640 1 -- 172.21.15.45:0/2246694445 learned_addr learned my addr 172.21.15.45:0/2246694445 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:24:59.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe59fe00640 1 --2- 172.21.15.45:0/2246694445 >> v2:172.21.15.117:3300/0 conn(0x7fe5a80d0040 0x7fe5a815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe5a5400640 1 -- 172.21.15.45:0/2246694445 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253544087 0 0) 0x7fe5a8071050 con 0x7fe5a81550b0 2024-04-25T10:24:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe5a5400640 1 -- 172.21.15.45:0/2246694445 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe58c005740 con 0x7fe5a81550b0 2024-04-25T10:24:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe59fe00640 1 -- 172.21.15.45:0/2246694445 >> v1:172.21.15.117:6789/0 conn(0x7fe5a815b740 legacy=0x7fe5a80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe59fe00640 1 -- 172.21.15.45:0/2246694445 >> v1:172.21.15.45:6789/0 conn(0x7fe5a81550b0 legacy=0x7fe5a80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe59fe00640 1 -- 172.21.15.45:0/2246694445 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a80715d0 con 0x7fe5a80d0040 2024-04-25T10:24:59.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.240+0000 7fe59fe00640 1 --2- 172.21.15.45:0/2246694445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a80d0040 0x7fe5a815b200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5900097c0 tx=0x7fe59001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958d92012ca55de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:59.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5a5400640 1 -- 172.21.15.45:0/2246694445 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900b0070 con 0x7fe5a80d0040 2024-04-25T10:24:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5a5400640 1 -- 172.21.15.45:0/2246694445 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5900ab600 con 0x7fe5a80d0040 2024-04-25T10:24:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5a5400640 1 -- 172.21.15.45:0/2246694445 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900ab9d0 con 0x7fe5a80d0040 2024-04-25T10:24:59.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- 172.21.15.45:0/2246694445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a80d0040 msgr2=0x7fe5a815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 --2- 172.21.15.45:0/2246694445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a80d0040 0x7fe5a815b200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5900097c0 tx=0x7fe59001c470 comp rx=0 tx=0).stop 2024-04-25T10:24:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- 172.21.15.45:0/2246694445 shutdown_connections 2024-04-25T10:24:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 --2- 172.21.15.45:0/2246694445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a80d0040 0x7fe5a815b200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- 172.21.15.45:0/2246694445 >> 172.21.15.45:0/2246694445 conn(0x7fe5a8066b70 msgr2=0x7fe5a8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- 172.21.15.45:0/2246694445 shutdown_connections 2024-04-25T10:24:59.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- 172.21.15.45:0/2246694445 wait complete. 2024-04-25T10:24:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 Processor -- start 2024-04-25T10:24:59.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- start start 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 0x7fe5a8140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81550b0 0x7fe5a8155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5a815b740 0x7fe5a80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a815bcd0 con 0x7fe5a80d0040 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a815bb50 con 0x7fe5a81550b0 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5aca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a806c090 con 0x7fe5a815b740 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5a5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 0x7fe5a8140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5a5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 0x7fe5a8140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54966/0 (socket says 172.21.15.45:54966) 2024-04-25T10:24:59.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.244+0000 7fe5a5e00640 1 -- 172.21.15.45:0/954007575 learned_addr learned my addr 172.21.15.45:0/954007575 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59fe00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81550b0 0x7fe5a8155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a6800640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5a815b740 0x7fe5a80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a5e00640 1 -- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5a815b740 msgr2=0x7fe5a80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a5e00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5a815b740 0x7fe5a80cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a5e00640 1 -- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81550b0 msgr2=0x7fe5a8155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a5e00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81550b0 0x7fe5a8155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a5e00640 1 -- 172.21.15.45:0/954007575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8141560 con 0x7fe5a80d0040 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5a5e00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 0x7fe5a8140ab0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe59800c4d0 tx=0x7fe59800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598017070 con 0x7fe5a80d0040 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe598005cf0 con 0x7fe5a80d0040 2024-04-25T10:24:59.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a8141840 con 0x7fe5a80d0040 2024-04-25T10:24:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598007270 con 0x7fe5a80d0040 2024-04-25T10:24:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a8141e00 con 0x7fe5a80d0040 2024-04-25T10:24:59.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a804ef30 con 0x7fe5a80d0040 2024-04-25T10:24:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe59801a070 con 0x7fe5a80d0040 2024-04-25T10:24:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59e000640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe584066f60 0x7fe584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:24:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.248+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe598084390 con 0x7fe5a80d0040 2024-04-25T10:24:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.252+0000 7fe59fe00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe584066f60 0x7fe584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:24:59.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.252+0000 7fe59fe00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe584066f60 0x7fe584069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe5900ad000 tx=0x7fe590008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:24:59.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.252+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe598054d40 con 0x7fe5a80d0040 2024-04-25T10:24:59.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:59.376 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:24:59.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.428+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5a806c900 con 0x7fe5a80d0040 2024-04-25T10:24:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.428+0000 7fe59e000640 1 -- 172.21.15.45:0/954007575 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe598058de0 con 0x7fe5a80d0040 2024-04-25T10:24:59.431 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:24:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe584066f60 msgr2=0x7fe584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe584066f60 0x7fe584069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe5900ad000 tx=0x7fe590008040 comp rx=0 tx=0).stop 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 msgr2=0x7fe5a8140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 0x7fe5a8140ab0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe59800c4d0 tx=0x7fe59800c990 comp rx=0 tx=0).stop 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 shutdown_connections 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe584066f60 0x7fe584069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5a815b740 0x7fe5a80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81550b0 0x7fe5a8155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 --2- 172.21.15.45:0/954007575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a80d0040 0x7fe5a8140ab0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:24:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 >> 172.21.15.45:0/954007575 conn(0x7fe5a8066b70 msgr2=0x7fe5a806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:24:59.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 shutdown_connections 2024-04-25T10:24:59.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:24:59.432+0000 7fe5aca00640 1 -- 172.21.15.45:0/954007575 wait complete. 2024-04-25T10:24:59.452 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:24:59.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:24:59.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:00.207 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:25:00.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:00.207 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:25:00.216 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:25:00.216 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:00.225 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:00.226 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:00.226 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:00.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:00.267 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:25:00.268 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:00.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:00.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:00.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.452+0000 7f0783000640 1 Processor -- start 2024-04-25T10:25:00.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0783000640 1 -- start start 2024-04-25T10:25:00.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0783000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c156130 0x7f077c071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0783000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c15cdb0 con 0x7f077c0cff80 2024-04-25T10:25:00.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0783000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0724a0 con 0x7f077c06d390 2024-04-25T10:25:00.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0783000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15bff0 con 0x7f077c156130 2024-04-25T10:25:00.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0782600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f077c0cff80 0x7f077c0b16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54704/0 (socket says 172.21.15.45:54704) 2024-04-25T10:25:00.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0782600640 1 -- 172.21.15.45:0/3293805623 learned_addr learned my addr 172.21.15.45:0/3293805623 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:00.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0781c00640 1 --2- 172.21.15.45:0/3293805623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c156130 0x7f077c071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702748284 0 0) 0x7f077c15cdb0 con 0x7f077c0cff80 2024-04-25T10:25:00.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0764003530 con 0x7f077c0cff80 2024-04-25T10:25:00.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336390481 0 0) 0x7f077c0724a0 con 0x7f077c06d390 2024-04-25T10:25:00.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077c15cdb0 con 0x7f077c06d390 2024-04-25T10:25:00.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0781c00640 1 -- 172.21.15.45:0/3293805623 >> v1:172.21.15.117:6789/0 conn(0x7f077c06d390 legacy=0x7f077c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0781c00640 1 -- 172.21.15.45:0/3293805623 >> v1:172.21.15.45:6789/0 conn(0x7f077c0cff80 legacy=0x7f077c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.456+0000 7f0781c00640 1 -- 172.21.15.45:0/3293805623 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c15c2d0 con 0x7f077c156130 2024-04-25T10:25:00.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0781c00640 1 --2- 172.21.15.45:0/3293805623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c156130 0x7f077c071dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07700097f0 tx=0x7f077001c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85671aa2d2712eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:00.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700ab620 con 0x7f077c156130 2024-04-25T10:25:00.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07700b0070 con 0x7f077c156130 2024-04-25T10:25:00.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f077be00640 1 -- 172.21.15.45:0/3293805623 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700ab9e0 con 0x7f077c156130 2024-04-25T10:25:00.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 -- 172.21.15.45:0/3293805623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c156130 msgr2=0x7f077c071dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 --2- 172.21.15.45:0/3293805623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c156130 0x7f077c071dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07700097f0 tx=0x7f077001c420 comp rx=0 tx=0).stop 2024-04-25T10:25:00.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 -- 172.21.15.45:0/3293805623 shutdown_connections 2024-04-25T10:25:00.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 --2- 172.21.15.45:0/3293805623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c156130 0x7f077c071dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 -- 172.21.15.45:0/3293805623 >> 172.21.15.45:0/3293805623 conn(0x7f077c073d10 msgr2=0x7f077c0740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:00.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 -- 172.21.15.45:0/3293805623 shutdown_connections 2024-04-25T10:25:00.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 -- 172.21.15.45:0/3293805623 wait complete. 2024-04-25T10:25:00.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 Processor -- start 2024-04-25T10:25:00.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.460+0000 7f0783000640 1 -- start start 2024-04-25T10:25:00.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f077c06d390 0x7f077c153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c0cff80 0x7f077c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 0x7f077c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c169a00 con 0x7f077c156130 2024-04-25T10:25:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c169880 con 0x7f077c06d390 2024-04-25T10:25:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c169b80 con 0x7f077c0cff80 2024-04-25T10:25:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 0x7f077c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0781200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c0cff80 0x7f077c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 0x7f077c0b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54978/0 (socket says 172.21.15.45:54978) 2024-04-25T10:25:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 -- 172.21.15.45:0/3180627761 learned_addr learned my addr 172.21.15.45:0/3180627761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0781c00640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f077c06d390 0x7f077c153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 -- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c0cff80 msgr2=0x7f077c154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c0cff80 0x7f077c154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 -- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f077c06d390 msgr2=0x7f077c153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f077c06d390 0x7f077c153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 -- 172.21.15.45:0/3180627761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c153740 con 0x7f077c156130 2024-04-25T10:25:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0782600640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 0x7f077c0b4e50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f077400caa0 tx=0x7f077400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774017070 con 0x7f077c156130 2024-04-25T10:25:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400ac20 con 0x7f077c156130 2024-04-25T10:25:00.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774012400 con 0x7f077c156130 2024-04-25T10:25:00.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c153a10 con 0x7f077c156130 2024-04-25T10:25:00.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c1542b0 con 0x7f077c156130 2024-04-25T10:25:00.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.464+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c06bca0 con 0x7f077c156130 2024-04-25T10:25:00.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.468+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f077401a070 con 0x7f077c156130 2024-04-25T10:25:00.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.468+0000 7f0779600640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f075c066f60 0x7f075c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.468+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0774084db0 con 0x7f077c156130 2024-04-25T10:25:00.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.468+0000 7f0781c00640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f075c066f60 0x7f075c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.468+0000 7f0781c00640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f075c066f60 0x7f075c069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f07700af000 tx=0x7f07700b6040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:00.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.472+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0774089070 con 0x7f077c156130 2024-04-25T10:25:00.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.664+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f077c0d0400 con 0x7f077c156130 2024-04-25T10:25:00.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0779600640 1 -- 172.21.15.45:0/3180627761 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f07740556d0 con 0x7f077c156130 2024-04-25T10:25:00.668 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:00.668 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:25:00.668 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f075c066f60 msgr2=0x7f075c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f075c066f60 0x7f075c069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f07700af000 tx=0x7f07700b6040 comp rx=0 tx=0).stop 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 msgr2=0x7f077c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 0x7f077c0b4e50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f077400caa0 tx=0x7f077400cf60 comp rx=0 tx=0).stop 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 shutdown_connections 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f075c066f60 0x7f075c069420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077c156130 0x7f077c0b4e50 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f077c0cff80 0x7f077c154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 --2- 172.21.15.45:0/3180627761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f077c06d390 0x7f077c153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 >> 172.21.15.45:0/3180627761 conn(0x7f077c073d10 msgr2=0x7f077c150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:00.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 shutdown_connections 2024-04-25T10:25:00.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.668+0000 7f0783000640 1 -- 172.21.15.45:0/3180627761 wait complete. 2024-04-25T10:25:00.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 Processor -- start 2024-04-25T10:25:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 -- start start 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7140b0f70 0x7fc7140b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc7140cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71415f3b0 con 0x7fc7140cfd90 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71415ef20 con 0x7fc7140b0f70 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc719200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140ca810 con 0x7fc7140cd2e0 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7140b0f70 0x7fc7140b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7140b0f70 0x7fc7140b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54986/0 (socket says 172.21.15.45:54986) 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 -- 172.21.15.45:0/556466263 learned_addr learned my addr 172.21.15.45:0/556466263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc712800640 1 --2- 172.21.15.45:0/556466263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc7140cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 -- 172.21.15.45:0/556466263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 msgr2=0x7fc7140cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 --2- 172.21.15.45:0/556466263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc7140cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 -- 172.21.15.45:0/556466263 >> v1:172.21.15.117:6789/0 conn(0x7fc7140cfd90 legacy=0x7fc7140ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.828+0000 7fc711e00640 1 -- 172.21.15.45:0/556466263 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140b1890 con 0x7fc7140b0f70 2024-04-25T10:25:00.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc711e00640 1 --2- 172.21.15.45:0/556466263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140b0f70 0x7fc7140b1350 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc704009870 tx=0x7fc70401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ea07064b3fcc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:00.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc711400640 1 -- 172.21.15.45:0/556466263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040b2070 con 0x7fc7140b0f70 2024-04-25T10:25:00.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc711400640 1 -- 172.21.15.45:0/556466263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7040a9070 con 0x7fc7140b0f70 2024-04-25T10:25:00.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc711400640 1 -- 172.21.15.45:0/556466263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040ad880 con 0x7fc7140b0f70 2024-04-25T10:25:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 -- 172.21.15.45:0/556466263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140b0f70 msgr2=0x7fc7140b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 --2- 172.21.15.45:0/556466263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140b0f70 0x7fc7140b1350 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc704009870 tx=0x7fc70401c560 comp rx=0 tx=0).stop 2024-04-25T10:25:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 -- 172.21.15.45:0/556466263 shutdown_connections 2024-04-25T10:25:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 --2- 172.21.15.45:0/556466263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc7140cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 --2- 172.21.15.45:0/556466263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140b0f70 0x7fc7140b1350 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 -- 172.21.15.45:0/556466263 >> 172.21.15.45:0/556466263 conn(0x7fc7140b1d50 msgr2=0x7fc7140b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:00.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 -- 172.21.15.45:0/556466263 shutdown_connections 2024-04-25T10:25:00.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.832+0000 7fc719200640 1 -- 172.21.15.45:0/556466263 wait complete. 2024-04-25T10:25:00.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 Processor -- start 2024-04-25T10:25:00.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 -- start start 2024-04-25T10:25:00.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7140b0f70 0x7fc7140cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc714099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 0x7fc7140c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714161b50 con 0x7fc7140cfd90 2024-04-25T10:25:00.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141619d0 con 0x7fc7140b0f70 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc714161cd0 con 0x7fc7140cd2e0 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 0x7fc7140c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 0x7fc7140c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55000/0 (socket says 172.21.15.45:55000) 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 -- 172.21.15.45:0/2982270412 learned_addr learned my addr 172.21.15.45:0/2982270412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc711e00640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7140b0f70 0x7fc7140cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc709400640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc714099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 -- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 msgr2=0x7fc714099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc714099670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 -- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7140b0f70 msgr2=0x7fc7140cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7140b0f70 0x7fc7140cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 -- 172.21.15.45:0/2982270412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714098330 con 0x7fc7140cfd90 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc711e00640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7140b0f70 0x7fc7140cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc712800640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 0x7fc7140c8e70 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d800 tx=0x7fc70c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00b6c0 con 0x7fc7140cfd90 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c019070 con 0x7fc7140cfd90 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7140985d0 con 0x7fc7140cfd90 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00baa0 con 0x7fc7140cfd90 2024-04-25T10:25:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.836+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc714098b90 con 0x7fc7140cfd90 2024-04-25T10:25:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.840+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71404ef30 con 0x7fc7140cfd90 2024-04-25T10:25:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.840+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc70c00bc40 con 0x7fc7140cfd90 2024-04-25T10:25:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.840+0000 7fc70aa00640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc6f0066f60 0x7fc6f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.840+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc70c085530 con 0x7fc7140cfd90 2024-04-25T10:25:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.840+0000 7fc711e00640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc6f0066f60 0x7fc6f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:00.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.840+0000 7fc711e00640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc6f0066f60 0x7fc6f0069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc7040af000 tx=0x7fc7040a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:00.844+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc70c0564a0 con 0x7fc7140cfd90 2024-04-25T10:25:01.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.008+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7140cda20 con 0x7fc7140cfd90 2024-04-25T10:25:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.008+0000 7fc70aa00640 1 -- 172.21.15.45:0/2982270412 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16660 (secure 0 0 0) 0x7fc70c055da0 con 0x7fc7140cfd90 2024-04-25T10:25:01.010 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:01.010 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:18.823568+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.008+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc6f0066f60 msgr2=0x7fc6f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.008+0000 7fc719200640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc6f0066f60 0x7fc6f0069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc7040af000 tx=0x7fc7040a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.008+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 msgr2=0x7fc7140c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.008+0000 7fc719200640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 0x7fc7140c8e70 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d800 tx=0x7fc70c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 shutdown_connections 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc6f0066f60 0x7fc6f0069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7140cfd90 0x7fc7140c8e70 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7140cd2e0 0x7fc714099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 --2- 172.21.15.45:0/2982270412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7140b0f70 0x7fc7140cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 >> 172.21.15.45:0/2982270412 conn(0x7fc7140b1d50 msgr2=0x7fc714160c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 shutdown_connections 2024-04-25T10:25:01.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.012+0000 7fc719200640 1 -- 172.21.15.45:0/2982270412 wait complete. 2024-04-25T10:25:01.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:24:15.637327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:24:16.209015+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:01.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:01.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.172+0000 7efe57800640 1 Processor -- start 2024-04-25T10:25:01.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe57800640 1 -- start start 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe57800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500afc40 0x7efe5015af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe57800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe500b4270 0x7efe500b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe57800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5015b7b0 con 0x7efe50071360 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe57800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe500b4c20 con 0x7efe500b4270 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe57800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5015f450 con 0x7efe500afc40 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe500b4270 0x7efe500b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efe500b4270 0x7efe500b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55012/0 (socket says 172.21.15.45:55012) 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 -- 172.21.15.45:0/4164563166 learned_addr learned my addr 172.21.15.45:0/4164563166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe4fe00640 1 --1- 172.21.15.45:0/4164563166 >> v1:172.21.15.45:6789/0 conn(0x7efe50071360 0x7efe5015ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54712/0 (socket says 172.21.15.45:54712) 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe54e00640 1 --2- 172.21.15.45:0/4164563166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500afc40 0x7efe5015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 -- 172.21.15.45:0/4164563166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500afc40 msgr2=0x7efe5015af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 --2- 172.21.15.45:0/4164563166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500afc40 0x7efe5015af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 -- 172.21.15.45:0/4164563166 >> v1:172.21.15.45:6789/0 conn(0x7efe50071360 legacy=0x7efe5015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.176+0000 7efe55800640 1 -- 172.21.15.45:0/4164563166 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5015ba90 con 0x7efe500b4270 2024-04-25T10:25:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe55800640 1 --2- 172.21.15.45:0/4164563166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe500b4270 0x7efe500b46e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7efe4800d150 tx=0x7efe4801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4c384b02d9772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe4f400640 1 -- 172.21.15.45:0/4164563166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4800b2d0 con 0x7efe500b4270 2024-04-25T10:25:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe4f400640 1 -- 172.21.15.45:0/4164563166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4800b470 con 0x7efe500b4270 2024-04-25T10:25:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe4f400640 1 -- 172.21.15.45:0/4164563166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe480a7b80 con 0x7efe500b4270 2024-04-25T10:25:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 -- 172.21.15.45:0/4164563166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe500b4270 msgr2=0x7efe500b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 --2- 172.21.15.45:0/4164563166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe500b4270 0x7efe500b46e0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7efe4800d150 tx=0x7efe4801eba0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 -- 172.21.15.45:0/4164563166 shutdown_connections 2024-04-25T10:25:01.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 --2- 172.21.15.45:0/4164563166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe500b4270 0x7efe500b46e0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 --2- 172.21.15.45:0/4164563166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500afc40 0x7efe5015af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 -- 172.21.15.45:0/4164563166 >> 172.21.15.45:0/4164563166 conn(0x7efe5006ac90 msgr2=0x7efe5006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:01.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 -- 172.21.15.45:0/4164563166 shutdown_connections 2024-04-25T10:25:01.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 -- 172.21.15.45:0/4164563166 wait complete. 2024-04-25T10:25:01.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 Processor -- start 2024-04-25T10:25:01.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.180+0000 7efe57800640 1 -- start start 2024-04-25T10:25:01.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 0x7efe50149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500afc40 0x7efe500b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500b4270 0x7efe50151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe500b01a0 con 0x7efe50071360 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe500b0020 con 0x7efe500afc40 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5015c870 con 0x7efe500b4270 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 0x7efe50149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 0x7efe50149b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55022/0 (socket says 172.21.15.45:55022) 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 -- 172.21.15.45:0/1245930671 learned_addr learned my addr 172.21.15.45:0/1245930671 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe4fe00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500afc40 0x7efe500b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe55800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500b4270 0x7efe50151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 -- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500b4270 msgr2=0x7efe50151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500b4270 0x7efe50151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 -- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500afc40 msgr2=0x7efe500b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500afc40 0x7efe500b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 -- 172.21.15.45:0/1245930671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe50151f00 con 0x7efe50071360 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe55800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500b4270 0x7efe50151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe4fe00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500afc40 0x7efe500b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe54e00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 0x7efe50149b80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7efe4400e930 tx=0x7efe4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44017070 con 0x7efe50071360 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4400ca00 con 0x7efe50071360 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400cdb0 con 0x7efe50071360 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe500b0570 con 0x7efe50071360 2024-04-25T10:25:01.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe500b0b30 con 0x7efe50071360 2024-04-25T10:25:01.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.184+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5004ef30 con 0x7efe50071360 2024-04-25T10:25:01.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.188+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe4401a070 con 0x7efe50071360 2024-04-25T10:25:01.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.188+0000 7efe4cc00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efe30066f60 0x7efe30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.188+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe44085440 con 0x7efe50071360 2024-04-25T10:25:01.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.188+0000 7efe4fe00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efe30066f60 0x7efe30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.188+0000 7efe4fe00640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efe30066f60 0x7efe30069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efe3c007fd0 tx=0x7efe3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.192+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe44056500 con 0x7efe50071360 2024-04-25T10:25:01.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe500d09e0 con 0x7efe50071360 2024-04-25T10:25:01.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe4cc00640 1 -- 172.21.15.45:0/1245930671 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe44055e00 con 0x7efe50071360 2024-04-25T10:25:01.356 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:25:01.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efe30066f60 msgr2=0x7efe30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efe30066f60 0x7efe30069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efe3c007fd0 tx=0x7efe3c009040 comp rx=0 tx=0).stop 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 msgr2=0x7efe50149b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 0x7efe50149b80 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7efe4400e930 tx=0x7efe4400edf0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 shutdown_connections 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efe30066f60 0x7efe30069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe500b4270 0x7efe50151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe500afc40 0x7efe500b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 --2- 172.21.15.45:0/1245930671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe50071360 0x7efe50149b80 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 >> 172.21.15.45:0/1245930671 conn(0x7efe5006ac90 msgr2=0x7efe500b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 shutdown_connections 2024-04-25T10:25:01.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.356+0000 7efe57800640 1 -- 172.21.15.45:0/1245930671 wait complete. 2024-04-25T10:25:01.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 Processor -- start 2024-04-25T10:25:01.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 -- start start 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efbfc15bb10 0x7efbfc0d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efbfc15bef0 0x7efbfc0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc0d0960 con 0x7efbfc0cd1f0 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0cc470 con 0x7efbfc15bef0 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.520+0000 7efc01c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0d0680 con 0x7efbfc15bb10 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efbfc15bef0 0x7efbfc0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efbfc15bef0 0x7efbfc0cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55028/0 (socket says 172.21.15.45:55028) 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 -- 172.21.15.45:0/2985904202 learned_addr learned my addr 172.21.15.45:0/2985904202 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbfaa00640 1 --2- 172.21.15.45:0/2985904202 >> v2:172.21.15.117:3300/0 conn(0x7efbfc15bb10 0x7efbfc0d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 -- 172.21.15.45:0/2985904202 >> v2:172.21.15.117:3300/0 conn(0x7efbfc15bb10 msgr2=0x7efbfc0d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:01.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 --2- 172.21.15.45:0/2985904202 >> v2:172.21.15.117:3300/0 conn(0x7efbfc15bb10 0x7efbfc0d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 -- 172.21.15.45:0/2985904202 >> v1:172.21.15.117:6789/0 conn(0x7efbfc0cd1f0 legacy=0x7efbfc15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 -- 172.21.15.45:0/2985904202 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0cc710 con 0x7efbfc15bef0 2024-04-25T10:25:01.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbfaa00640 1 --2- 172.21.15.45:0/2985904202 >> v2:172.21.15.117:3300/0 conn(0x7efbfc15bb10 0x7efbfc0d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:01.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbf2000640 1 --2- 172.21.15.45:0/2985904202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bef0 0x7efbfc0cff50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0cf4a0 tx=0x7efbe401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168c9be3f86b14b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbfa000640 1 -- 172.21.15.45:0/2985904202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe40ab610 con 0x7efbfc15bef0 2024-04-25T10:25:01.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbfa000640 1 -- 172.21.15.45:0/2985904202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe40b0070 con 0x7efbfc15bef0 2024-04-25T10:25:01.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.524+0000 7efbfa000640 1 -- 172.21.15.45:0/2985904202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe40ab9d0 con 0x7efbfc15bef0 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- 172.21.15.45:0/2985904202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bef0 msgr2=0x7efbfc0cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 --2- 172.21.15.45:0/2985904202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bef0 0x7efbfc0cff50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0cf4a0 tx=0x7efbe401c520 comp rx=0 tx=0).stop 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- 172.21.15.45:0/2985904202 shutdown_connections 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 --2- 172.21.15.45:0/2985904202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bef0 0x7efbfc0cff50 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 --2- 172.21.15.45:0/2985904202 >> v2:172.21.15.117:3300/0 conn(0x7efbfc15bb10 0x7efbfc0d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- 172.21.15.45:0/2985904202 >> 172.21.15.45:0/2985904202 conn(0x7efbfc0d19a0 msgr2=0x7efbfc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- 172.21.15.45:0/2985904202 shutdown_connections 2024-04-25T10:25:01.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- 172.21.15.45:0/2985904202 wait complete. 2024-04-25T10:25:01.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 Processor -- start 2024-04-25T10:25:01.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- start start 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbfc0cd1f0 0x7efbfc159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 0x7efbfc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbfc15bef0 0x7efbfc146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc15ff20 con 0x7efbfc15bb10 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0c5f10 con 0x7efbfc0cd1f0 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efc01c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc1600a0 con 0x7efbfc15bef0 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 0x7efbfc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 0x7efbfc0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55032/0 (socket says 172.21.15.45:55032) 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 -- 172.21.15.45:0/3378619472 learned_addr learned my addr 172.21.15.45:0/3378619472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:01.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbfaa00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbfc0cd1f0 0x7efbfc159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbfb400640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbfc15bef0 0x7efbfc146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 -- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbfc15bef0 msgr2=0x7efbfc146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbfc15bef0 0x7efbfc146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 -- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbfc0cd1f0 msgr2=0x7efbfc159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbfc0cd1f0 0x7efbfc159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.528+0000 7efbf2000640 1 -- 172.21.15.45:0/3378619472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc159f40 con 0x7efbfc15bb10 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf2000640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 0x7efbfc0ca3c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7efbe40ad040 tx=0x7efbe401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe40be5f0 con 0x7efbfc15bb10 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe40be790 con 0x7efbfc15bb10 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe40beb80 con 0x7efbfc15bb10 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc15a220 con 0x7efbfc15bb10 2024-04-25T10:25:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc15a7e0 con 0x7efbfc15bb10 2024-04-25T10:25:01.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc04ef30 con 0x7efbfc15bb10 2024-04-25T10:25:01.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efbe40b43f0 con 0x7efbfc15bb10 2024-04-25T10:25:01.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf3400640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efbd8066f60 0x7efbd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efbe411fc70 con 0x7efbfc15bb10 2024-04-25T10:25:01.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.532+0000 7efbfaa00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efbd8066f60 0x7efbd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.536+0000 7efbfaa00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efbd8066f60 0x7efbd8069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efbec009fd0 tx=0x7efbec009870 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.536+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efbe40c4060 con 0x7efbfc15bb10 2024-04-25T10:25:01.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efbfc0c4f40 con 0x7efbfc15bb10 2024-04-25T10:25:01.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efbf3400640 1 -- 172.21.15.45:0/3378619472 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7efbe40a8050 con 0x7efbfc15bb10 2024-04-25T10:25:01.701 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efbd8066f60 msgr2=0x7efbd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efbd8066f60 0x7efbd8069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efbec009fd0 tx=0x7efbec009870 comp rx=0 tx=0).stop 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 msgr2=0x7efbfc0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 0x7efbfc0ca3c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7efbe40ad040 tx=0x7efbe401ad60 comp rx=0 tx=0).stop 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 shutdown_connections 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efbd8066f60 0x7efbd8069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efbfc15bef0 0x7efbfc146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbfc15bb10 0x7efbfc0ca3c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 --2- 172.21.15.45:0/3378619472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efbfc0cd1f0 0x7efbfc159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 >> 172.21.15.45:0/3378619472 conn(0x7efbfc0d19a0 msgr2=0x7efbfc154780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:01.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 shutdown_connections 2024-04-25T10:25:01.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.700+0000 7efc01c00640 1 -- 172.21.15.45:0/3378619472 wait complete. 2024-04-25T10:25:01.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 Processor -- start 2024-04-25T10:25:01.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 -- start start 2024-04-25T10:25:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e1c15a780 0x7f8e1c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8e1c0cf3b0 0x7f8e1c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0b3470 con 0x7f8e1c0cfd30 2024-04-25T10:25:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c06de60 con 0x7f8e1c15a780 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.864+0000 7f8e1b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c15f2f0 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e1c15a780 0x7f8e1c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e1c15a780 0x7f8e1c15edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55040/0 (socket says 172.21.15.45:55040) 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 -- 172.21.15.45:0/1238681295 learned_addr learned my addr 172.21.15.45:0/1238681295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e11400640 1 --2- 172.21.15.45:0/1238681295 >> v2:172.21.15.117:3300/0 conn(0x7f8e1c0cf3b0 0x7f8e1c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 -- 172.21.15.45:0/1238681295 >> v2:172.21.15.117:3300/0 conn(0x7f8e1c0cf3b0 msgr2=0x7f8e1c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 --2- 172.21.15.45:0/1238681295 >> v2:172.21.15.117:3300/0 conn(0x7f8e1c0cf3b0 0x7f8e1c0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 -- 172.21.15.45:0/1238681295 >> v1:172.21.15.117:6789/0 conn(0x7f8e1c0cfd30 legacy=0x7f8e1c071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 -- 172.21.15.45:0/1238681295 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c06cdc0 con 0x7f8e1c15a780 2024-04-25T10:25:01.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19e00640 1 --2- 172.21.15.45:0/1238681295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c15a780 0x7f8e1c15edb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0677f0 tx=0x7f8e0c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324626686df2a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19400640 1 -- 172.21.15.45:0/1238681295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0a56c0 con 0x7f8e1c15a780 2024-04-25T10:25:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19400640 1 -- 172.21.15.45:0/1238681295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0a5860 con 0x7f8e1c15a780 2024-04-25T10:25:01.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.868+0000 7f8e19400640 1 -- 172.21.15.45:0/1238681295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0a5c30 con 0x7f8e1c15a780 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- 172.21.15.45:0/1238681295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c15a780 msgr2=0x7f8e1c15edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 --2- 172.21.15.45:0/1238681295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c15a780 0x7f8e1c15edb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0677f0 tx=0x7f8e0c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- 172.21.15.45:0/1238681295 shutdown_connections 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 --2- 172.21.15.45:0/1238681295 >> v2:172.21.15.117:3300/0 conn(0x7f8e1c0cf3b0 0x7f8e1c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 --2- 172.21.15.45:0/1238681295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c15a780 0x7f8e1c15edb0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- 172.21.15.45:0/1238681295 >> 172.21.15.45:0/1238681295 conn(0x7f8e1c0743f0 msgr2=0x7f8e1c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- 172.21.15.45:0/1238681295 shutdown_connections 2024-04-25T10:25:01.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- 172.21.15.45:0/1238681295 wait complete. 2024-04-25T10:25:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 Processor -- start 2024-04-25T10:25:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- start start 2024-04-25T10:25:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 0x7f8e1c165d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c0cfd30 0x7f8e1c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1c15a780 0x7f8e1c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c15ab60 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c0d0140 con 0x7f8e1c0cfd30 2024-04-25T10:25:01.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c165730 con 0x7f8e1c15a780 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 0x7f8e1c165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 0x7f8e1c165d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55050/0 (socket says 172.21.15.45:55050) 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 -- 172.21.15.45:0/392059249 learned_addr learned my addr 172.21.15.45:0/392059249 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e11400640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c0cfd30 0x7f8e1c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e1a800640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1c15a780 0x7f8e1c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 -- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1c15a780 msgr2=0x7f8e1c1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1c15a780 0x7f8e1c1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 -- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c0cfd30 msgr2=0x7f8e1c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c0cfd30 0x7f8e1c1586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.872+0000 7f8e19e00640 1 -- 172.21.15.45:0/392059249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c1662d0 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e19e00640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 0x7f8e1c165d90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0af040 tx=0x7f8e0c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c0bc020 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c0b8670 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c009510 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c166ff0 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c1675b0 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c075120 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e0c0bf020 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e12a00640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df8066f90 0x7f8df8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e0c11f750 con 0x7f8e1c0cf3b0 2024-04-25T10:25:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.876+0000 7f8e11400640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df8066f90 0x7f8df8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.880+0000 7f8e11400640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df8066f90 0x7f8df8069450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c1573d0 tx=0x7f8e04009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:01.880+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e0c0c7020 con 0x7f8e1c0cf3b0 2024-04-25T10:25:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.044+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e1c1513c0 con 0x7f8e1c0cf3b0 2024-04-25T10:25:02.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.044+0000 7f8e12a00640 1 -- 172.21.15.45:0/392059249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e0c0165e0 con 0x7f8e1c0cf3b0 2024-04-25T10:25:02.047 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:25:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df8066f90 msgr2=0x7f8df8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df8066f90 0x7f8df8069450 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c1573d0 tx=0x7f8e04009190 comp rx=0 tx=0).stop 2024-04-25T10:25:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 msgr2=0x7f8e1c165d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 0x7f8e1c165d90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0af040 tx=0x7f8e0c019790 comp rx=0 tx=0).stop 2024-04-25T10:25:02.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 shutdown_connections 2024-04-25T10:25:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8df8066f90 0x7f8df8069450 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e1c15a780 0x7f8e1c1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1c0cfd30 0x7f8e1c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 --2- 172.21.15.45:0/392059249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e1c0cf3b0 0x7f8e1c165d90 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 >> 172.21.15.45:0/392059249 conn(0x7f8e1c0743f0 msgr2=0x7f8e1c070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:02.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 shutdown_connections 2024-04-25T10:25:02.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.048+0000 7f8e1b200640 1 -- 172.21.15.45:0/392059249 wait complete. 2024-04-25T10:25:02.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:02.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eefe00640 1 Processor -- start 2024-04-25T10:25:02.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eefe00640 1 -- start start 2024-04-25T10:25:02.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eefe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3ee8066fe0 0x7f3ee815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eefe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80b5970 con 0x7f3ee80b5560 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eefe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80b5c20 con 0x7f3ee8066c00 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eefe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee80af6e0 con 0x7f3ee8066fe0 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eede00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3ee80b5560 0x7f3ee8157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54726/0 (socket says 172.21.15.45:54726) 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eede00640 1 -- 172.21.15.45:0/3710589706 learned_addr learned my addr 172.21.15.45:0/3710589706 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.208+0000 7f3eeca00640 1 --2- 172.21.15.45:0/3710589706 >> v2:172.21.15.117:3300/0 conn(0x7f3ee8066fe0 0x7f3ee815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272340646 0 0) 0x7f3ee80b5970 con 0x7f3ee80b5560 2024-04-25T10:25:02.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed0003530 con 0x7f3ee80b5560 2024-04-25T10:25:02.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337956466 0 0) 0x7f3ed0003530 con 0x7f3ee80b5560 2024-04-25T10:25:02.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 >> v2:172.21.15.117:3300/0 conn(0x7f3ee8066fe0 msgr2=0x7f3ee815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:02.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 --2- 172.21.15.45:0/3710589706 >> v2:172.21.15.117:3300/0 conn(0x7f3ee8066fe0 0x7f3ee815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 >> v1:172.21.15.117:6789/0 conn(0x7f3ee8066c00 legacy=0x7f3ee806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80af9c0 con 0x7f3ee80b5560 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f3ee0001a00 con 0x7f3ee80b5560 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 >> v1:172.21.15.45:6789/0 conn(0x7f3ee80b5560 legacy=0x7f3ee8157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed00039c0 0x7f3ed0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0004810 0x7f3ed0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ed000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80af6e0 con 0x7f3ed00051d0 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed000c070 con 0x7f3ed0004810 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed000c270 con 0x7f3ed00039c0 2024-04-25T10:25:02.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ed000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eeca00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0004810 0x7f3ed0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eed400640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed00039c0 0x7f3ed0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 -- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed00039c0 msgr2=0x7f3ed0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ed00039c0 0x7f3ed0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 -- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0004810 msgr2=0x7f3ed0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0004810 0x7f3ed0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 -- 172.21.15.45:0/3710589706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0003530 con 0x7f3ed00051d0 2024-04-25T10:25:02.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eeca00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ed0004810 0x7f3ed0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.212+0000 7f3eede00640 1 -- 172.21.15.45:0/3710589706 reap_dead start 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eede00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ed000b9d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000a840 tx=0x7f3ee001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9019b357ce363ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed40096b0 con 0x7f3ed00051d0 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee00ac640 con 0x7f3ed00051d0 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3ee7e00640 1 -- 172.21.15.45:0/3710589706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000a460 con 0x7f3ed00051d0 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- 172.21.15.45:0/3710589706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 msgr2=0x7f3ed000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ed000b9d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000a840 tx=0x7f3ee001d360 comp rx=0 tx=0).stop 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- 172.21.15.45:0/3710589706 shutdown_connections 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3710589706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ed000b9d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- 172.21.15.45:0/3710589706 >> 172.21.15.45:0/3710589706 conn(0x7f3ee80b24f0 msgr2=0x7f3ee80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- 172.21.15.45:0/3710589706 shutdown_connections 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- 172.21.15.45:0/3710589706 wait complete. 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 Processor -- start 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- start start 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ee8153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee8155c90 0x7f3ee81560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee8151360 0x7f3ee81517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee815e110 con 0x7f3ed00051d0 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80aff80 con 0x7f3ee8151360 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eefe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee815e290 con 0x7f3ee8155c90 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eed400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ee8153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eed400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ee8153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55076/0 (socket says 172.21.15.45:55076) 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.216+0000 7f3eed400640 1 -- 172.21.15.45:0/3869919474 learned_addr learned my addr 172.21.15.45:0/3869919474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eeca00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee8155c90 0x7f3ee81560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eede00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee8151360 0x7f3ee81517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eed400640 1 -- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee8155c90 msgr2=0x7f3ee81560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eed400640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee8155c90 0x7f3ee81560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eed400640 1 -- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee8151360 msgr2=0x7f3ee81517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eed400640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee8151360 0x7f3ee81517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eed400640 1 -- 172.21.15.45:0/3869919474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8156790 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eed400640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ee8153160 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00f540 tx=0x7f3edc00f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0052e0 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc018070 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc005480 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee8156a70 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee8157030 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8157410 con 0x7f3ed00051d0 2024-04-25T10:25:02.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.220+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3edc003800 con 0x7f3ed00051d0 2024-04-25T10:25:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.224+0000 7f3ee5600640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ec8066f60 0x7f3ec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.224+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3edc084cf0 con 0x7f3ed00051d0 2024-04-25T10:25:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.224+0000 7f3eeca00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ec8066f60 0x7f3ec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.224+0000 7f3eeca00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ec8066f60 0x7f3ec8069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3ee81f6fe0 tx=0x7f3ed4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.228+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ee8157410 con 0x7f3ed00051d0 2024-04-25T10:25:02.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.420+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ee804ef30 con 0x7f3ed00051d0 2024-04-25T10:25:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.420+0000 7f3ee5600640 1 -- 172.21.15.45:0/3869919474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3edc055c60 con 0x7f3ed00051d0 2024-04-25T10:25:02.423 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:02.423 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:25:02.423 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ec8066f60 msgr2=0x7f3ec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ec8066f60 0x7f3ec8069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3ee81f6fe0 tx=0x7f3ed4009190 comp rx=0 tx=0).stop 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 msgr2=0x7f3ee8153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ee8153160 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00f540 tx=0x7f3edc00f940 comp rx=0 tx=0).stop 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 shutdown_connections 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ec8066f60 0x7f3ec8069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee8151360 0x7f3ee81517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee8155c90 0x7f3ee81560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 --2- 172.21.15.45:0/3869919474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ed00051d0 0x7f3ee8153160 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 >> 172.21.15.45:0/3869919474 conn(0x7f3ee80b24f0 msgr2=0x7f3ee80aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 shutdown_connections 2024-04-25T10:25:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.424+0000 7f3eefe00640 1 -- 172.21.15.45:0/3869919474 wait complete. 2024-04-25T10:25:02.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8efa200640 1 Processor -- start 2024-04-25T10:25:02.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8efa200640 1 -- start start 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8efa200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ef40afda0 0x7f8ef406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8efa200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef406f200 con 0x7f8ef40af9c0 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8efa200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef40ccc10 con 0x7f8ef4071f50 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8efa200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef406d380 con 0x7f8ef40afda0 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8eebe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ef40afda0 0x7f8ef406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8ef3400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8ef40af9c0 0x7f8ef40cc3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54736/0 (socket says 172.21.15.45:54736) 2024-04-25T10:25:02.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8ef3400640 1 -- 172.21.15.45:0/1305559101 learned_addr learned my addr 172.21.15.45:0/1305559101 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8eebe00640 1 -- 172.21.15.45:0/1305559101 >> v1:172.21.15.117:6789/0 conn(0x7f8ef4071f50 legacy=0x7f8ef40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8eebe00640 1 -- 172.21.15.45:0/1305559101 >> v1:172.21.15.45:6789/0 conn(0x7f8ef40af9c0 legacy=0x7f8ef40cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8eebe00640 1 -- 172.21.15.45:0/1305559101 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40ccef0 con 0x7f8ef40afda0 2024-04-25T10:25:02.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.584+0000 7f8ef2a00640 1 -- 172.21.15.45:0/1305559101 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794719961 0 0) 0x7f8ee4001120 con 0x7f8ef40af9c0 2024-04-25T10:25:02.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8eebe00640 1 --2- 172.21.15.45:0/1305559101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40afda0 0x7f8ef406ddd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0097c0 tx=0x7f8edc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438f78c1f24579fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8ef2a00640 1 -- 172.21.15.45:0/1305559101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0b1070 con 0x7f8ef40afda0 2024-04-25T10:25:02.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8ef2a00640 1 -- 172.21.15.45:0/1305559101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc0a8050 con 0x7f8ef40afda0 2024-04-25T10:25:02.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8ef2a00640 1 -- 172.21.15.45:0/1305559101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0ac860 con 0x7f8ef40afda0 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 -- 172.21.15.45:0/1305559101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40afda0 msgr2=0x7f8ef406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 --2- 172.21.15.45:0/1305559101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40afda0 0x7f8ef406ddd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0097c0 tx=0x7f8edc01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 -- 172.21.15.45:0/1305559101 shutdown_connections 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 --2- 172.21.15.45:0/1305559101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40afda0 0x7f8ef406ddd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 -- 172.21.15.45:0/1305559101 >> 172.21.15.45:0/1305559101 conn(0x7f8ef40b2410 msgr2=0x7f8ef40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 -- 172.21.15.45:0/1305559101 shutdown_connections 2024-04-25T10:25:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.588+0000 7f8efa200640 1 -- 172.21.15.45:0/1305559101 wait complete. 2024-04-25T10:25:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 Processor -- start 2024-04-25T10:25:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 -- start start 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4071f50 0x7f8ef4157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 0x7f8ef4150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef40afda0 0x7f8ef41528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef41600d0 con 0x7f8ef40af9c0 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4067300 con 0x7f8ef4071f50 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4160250 con 0x7f8ef40afda0 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 0x7f8ef4150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 0x7f8ef4150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55088/0 (socket says 172.21.15.45:55088) 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 -- 172.21.15.45:0/3960184012 learned_addr learned my addr 172.21.15.45:0/3960184012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8ef3400640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4071f50 0x7f8ef4157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8ef3e00640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef40afda0 0x7f8ef41528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 -- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef40afda0 msgr2=0x7f8ef41528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef40afda0 0x7f8ef41528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 -- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4071f50 msgr2=0x7f8ef4157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4071f50 0x7f8ef4157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 -- 172.21.15.45:0/3960184012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4151450 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eebe00640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 0x7f8ef4150f10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0097c0 tx=0x7f8edc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0bd070 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc0ace20 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef4158e60 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0b1040 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.592+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4152050 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.596+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef404ef30 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.596+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8edc0c0070 con 0x7f8ef40af9c0 2024-04-25T10:25:02.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.596+0000 7f8eeb400640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed0066f60 0x7f8ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.596+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8edc11eb60 con 0x7f8ef40af9c0 2024-04-25T10:25:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.596+0000 7f8ef3400640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed0066f60 0x7f8ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.596+0000 7f8ef3400640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed0066f60 0x7f8ed0069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4001390 tx=0x7f8ee4001320 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.600+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8edc016090 con 0x7f8ef40af9c0 2024-04-25T10:25:02.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.760+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ef4071420 con 0x7f8ef40af9c0 2024-04-25T10:25:02.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8eeb400640 1 -- 172.21.15.45:0/3960184012 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16660 (secure 0 0 0) 0x7f8edc0ef480 con 0x7f8ef40af9c0 2024-04-25T10:25:02.765 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:02.765 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:18.823568+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed0066f60 msgr2=0x7f8ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed0066f60 0x7f8ed0069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4001390 tx=0x7f8ee4001320 comp rx=0 tx=0).stop 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 msgr2=0x7f8ef4150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 0x7f8ef4150f10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0097c0 tx=0x7f8edc01ac20 comp rx=0 tx=0).stop 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 shutdown_connections 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ed0066f60 0x7f8ed0069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ef40afda0 0x7f8ef41528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ef40af9c0 0x7f8ef4150f10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 --2- 172.21.15.45:0/3960184012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ef4071f50 0x7f8ef4157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 >> 172.21.15.45:0/3960184012 conn(0x7f8ef40b2410 msgr2=0x7f8ef406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:02.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 shutdown_connections 2024-04-25T10:25:02.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.764+0000 7f8efa200640 1 -- 172.21.15.45:0/3960184012 wait complete. 2024-04-25T10:25:02.787 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:25:02.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.928+0000 7f2c16400640 1 Processor -- start 2024-04-25T10:25:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.928+0000 7f2c16400640 1 -- start start 2024-04-25T10:25:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c16400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c101550b0 0x7f2c100cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c16400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c10071050 con 0x7f2c1015b910 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c16400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1015d040 con 0x7f2c100cfe70 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c16400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1015bd80 con 0x7f2c101550b0 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c101550b0 0x7f2c100cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2c1015b910 0x7f2c100b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54748/0 (socket says 172.21.15.45:54748) 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c101550b0 0x7f2c100cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55094/0 (socket says 172.21.15.45:55094) 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0fe00640 1 -- 172.21.15.45:0/2146710357 learned_addr learned my addr 172.21.15.45:0/2146710357 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:02.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0f400640 1 -- 172.21.15.45:0/2146710357 >> v1:172.21.15.117:6789/0 conn(0x7f2c100cfe70 legacy=0x7f2c1015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0f400640 1 -- 172.21.15.45:0/2146710357 >> v1:172.21.15.45:6789/0 conn(0x7f2c1015b910 legacy=0x7f2c100b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0ea00640 1 -- 172.21.15.45:0/2146710357 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329079383 0 0) 0x7f2c10071050 con 0x7f2c1015b910 2024-04-25T10:25:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0f400640 1 -- 172.21.15.45:0/2146710357 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c100712f0 con 0x7f2c101550b0 2024-04-25T10:25:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0f400640 1 --2- 172.21.15.45:0/2146710357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100cf930 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097f0 tx=0x7f2c0001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85690a8cd0c54680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0ea00640 1 -- 172.21.15.45:0/2146710357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000b0070 con 0x7f2c101550b0 2024-04-25T10:25:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0ea00640 1 -- 172.21.15.45:0/2146710357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c000ab600 con 0x7f2c101550b0 2024-04-25T10:25:02.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.932+0000 7f2c0ea00640 1 -- 172.21.15.45:0/2146710357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000ab9d0 con 0x7f2c101550b0 2024-04-25T10:25:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- 172.21.15.45:0/2146710357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 msgr2=0x7f2c100cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 --2- 172.21.15.45:0/2146710357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100cf930 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f2c000097f0 tx=0x7f2c0001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- 172.21.15.45:0/2146710357 shutdown_connections 2024-04-25T10:25:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 --2- 172.21.15.45:0/2146710357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100cf930 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- 172.21.15.45:0/2146710357 >> 172.21.15.45:0/2146710357 conn(0x7f2c10066b70 msgr2=0x7f2c10066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- 172.21.15.45:0/2146710357 shutdown_connections 2024-04-25T10:25:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- 172.21.15.45:0/2146710357 wait complete. 2024-04-25T10:25:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 Processor -- start 2024-04-25T10:25:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- start start 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c100cfe70 0x7f2c10159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c1015b910 0x7f2c100afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101684c0 con 0x7f2c101550b0 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c100720b0 con 0x7f2c100cfe70 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c16400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c10168640 con 0x7f2c1015b910 2024-04-25T10:25:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55102/0 (socket says 172.21.15.45:55102) 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 -- 172.21.15.45:0/2967661969 learned_addr learned my addr 172.21.15.45:0/2967661969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c0f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c100cfe70 0x7f2c10159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c0fe00640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c1015b910 0x7f2c100afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 -- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c1015b910 msgr2=0x7f2c100afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c1015b910 0x7f2c100afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 -- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c100cfe70 msgr2=0x7f2c10159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c100cfe70 0x7f2c10159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.936+0000 7f2c06a00640 1 -- 172.21.15.45:0/2967661969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c101599f0 con 0x7f2c101550b0 2024-04-25T10:25:02.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c06a00640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100af630 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800c3c0 tx=0x7f2bf800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8017070 con 0x7f2c101550b0 2024-04-25T10:25:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf80052d0 con 0x7f2c101550b0 2024-04-25T10:25:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8003840 con 0x7f2c101550b0 2024-04-25T10:25:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c10159cd0 con 0x7f2c101550b0 2024-04-25T10:25:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1014f730 con 0x7f2c101550b0 2024-04-25T10:25:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1004ef30 con 0x7f2c101550b0 2024-04-25T10:25:02.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bf801a070 con 0x7f2c101550b0 2024-04-25T10:25:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c07e00640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2bec066f60 0x7f2bec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bf8084370 con 0x7f2c101550b0 2024-04-25T10:25:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.940+0000 7f2c0f400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2bec066f60 0x7f2bec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:02.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.944+0000 7f2c0f400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2bec066f60 0x7f2bec069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ad000 tx=0x7f2c000a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:02.944+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bf8005480 con 0x7f2c101550b0 2024-04-25T10:25:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c1015ce00 con 0x7f2c101550b0 2024-04-25T10:25:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c07e00640 1 -- 172.21.15.45:0/2967661969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2bf8054c90 con 0x7f2c101550b0 2024-04-25T10:25:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:03.141 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:25:03.142 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:25:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2bec066f60 msgr2=0x7f2bec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2bec066f60 0x7f2bec069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2c000ad000 tx=0x7f2c000a6000 comp rx=0 tx=0).stop 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 msgr2=0x7f2c100af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100af630 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800c3c0 tx=0x7f2bf800c880 comp rx=0 tx=0).stop 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 shutdown_connections 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2bec066f60 0x7f2bec069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c1015b910 0x7f2c100afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c101550b0 0x7f2c100af630 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 --2- 172.21.15.45:0/2967661969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c100cfe70 0x7f2c10159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.140+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 >> 172.21.15.45:0/2967661969 conn(0x7f2c10066b70 msgr2=0x7f2c1006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.144+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 shutdown_connections 2024-04-25T10:25:03.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.144+0000 7f2c16400640 1 -- 172.21.15.45:0/2967661969 wait complete. 2024-04-25T10:25:03.163 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.300+0000 7f63df400640 1 Processor -- start 2024-04-25T10:25:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.300+0000 7f63df400640 1 -- start start 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63df400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63d81550b0 0x7f63d80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63df400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f63d80cfe70 0x7f63d815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63df400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d8071050 con 0x7f63d815b740 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63df400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d815d040 con 0x7f63d80cfe70 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63df400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80712f0 con 0x7f63d81550b0 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dd400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63d815b740 0x7f63d806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54764/0 (socket says 172.21.15.45:54764) 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dd400640 1 -- 172.21.15.45:0/2924327321 learned_addr learned my addr 172.21.15.45:0/2924327321 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dca00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63d81550b0 0x7f63d80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 --2- 172.21.15.45:0/2924327321 >> v2:172.21.15.117:3300/0 conn(0x7f63d80cfe70 0x7f63d815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655524306 0 0) 0x7f63d8071050 con 0x7f63d815b740 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63bc003530 con 0x7f63d815b740 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3360655565 0 0) 0x7f63bc003530 con 0x7f63d815b740 2024-04-25T10:25:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63d81550b0 msgr2=0x7f63d80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63d81550b0 0x7f63d80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 >> v2:172.21.15.117:3300/0 conn(0x7f63d80cfe70 msgr2=0x7f63d815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 --2- 172.21.15.45:0/2924327321 >> v2:172.21.15.117:3300/0 conn(0x7f63d80cfe70 0x7f63d815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80715d0 con 0x7f63d815b740 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f63d0001a00 con 0x7f63d815b740 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dca00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63d81550b0 0x7f63d80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 >> v1:172.21.15.45:6789/0 conn(0x7f63d815b740 legacy=0x7f63d806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc0039c0 0x7f63bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63bc004810 0x7f63bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63bc0051d0 0x7f63bc00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815d040 con 0x7f63bc004810 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80712f0 con 0x7f63bc0051d0 2024-04-25T10:25:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc00e120 con 0x7f63bc0039c0 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 --2- 172.21.15.45:0/2924327321 >> v2:172.21.15.117:3300/0 conn(0x7f63d80cfe70 0x7f63d815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dca00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc0039c0 0x7f63bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63bc004810 0x7f63bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dd400640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63bc0051d0 0x7f63bc00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 -- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc0039c0 msgr2=0x7f63bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc0039c0 0x7f63bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 -- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63bc0051d0 msgr2=0x7f63bc00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63bc0051d0 0x7f63bc00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63cfe00640 1 -- 172.21.15.45:0/2924327321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc003530 con 0x7f63bc004810 2024-04-25T10:25:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.304+0000 7f63dca00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc0039c0 0x7f63bc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:03.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63cfe00640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63bc004810 0x7f63bc004c90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009cc0 tx=0x7f63c001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a169cd794cbd02bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0009860 con 0x7f63bc004810 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0004020 con 0x7f63bc004810 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63d7e00640 1 -- 172.21.15.45:0/2924327321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004450 con 0x7f63bc004810 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63df400640 1 -- 172.21.15.45:0/2924327321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63bc004810 msgr2=0x7f63bc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63df400640 1 --2- 172.21.15.45:0/2924327321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63bc004810 0x7f63bc004c90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009cc0 tx=0x7f63c001cc50 comp rx=0 tx=0).stop 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63dd400640 1 -- 172.21.15.45:0/2924327321 reap_dead start 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63df400640 1 -- 172.21.15.45:0/2924327321 shutdown_connections 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63df400640 1 -- 172.21.15.45:0/2924327321 >> 172.21.15.45:0/2924327321 conn(0x7f63d8066b70 msgr2=0x7f63d8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63df400640 1 -- 172.21.15.45:0/2924327321 shutdown_connections 2024-04-25T10:25:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.308+0000 7f63df400640 1 -- 172.21.15.45:0/2924327321 wait complete. 2024-04-25T10:25:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 Processor -- start 2024-04-25T10:25:03.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 -- start start 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc004810 0x7f63d814a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 0x7f63d81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d81577f0 0x7f63d80b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80720b0 con 0x7f63d81f3b80 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8071b90 con 0x7f63d81577f0 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815bbd0 con 0x7f63bc004810 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 0x7f63d81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 0x7f63d81572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55106/0 (socket says 172.21.15.45:55106) 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 -- 172.21.15.45:0/3379761292 learned_addr learned my addr 172.21.15.45:0/3379761292 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63dca00640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc004810 0x7f63d814a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63dd400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d81577f0 0x7f63d80b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 -- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc004810 msgr2=0x7f63d814a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc004810 0x7f63d814a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 -- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d81577f0 msgr2=0x7f63d80b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d81577f0 0x7f63d80b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 -- 172.21.15.45:0/3379761292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d81f4b90 con 0x7f63d81f3b80 2024-04-25T10:25:03.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63cfe00640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 0x7f63d81572b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f63c00af040 tx=0x7f63bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c00bb020 con 0x7f63d81f3b80 2024-04-25T10:25:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0005e30 con 0x7f63d81f3b80 2024-04-25T10:25:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c00a5a00 con 0x7f63d81f3b80 2024-04-25T10:25:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d81f4010 con 0x7f63d81f3b80 2024-04-25T10:25:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.312+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d81f45d0 con 0x7f63d81f3b80 2024-04-25T10:25:03.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.316+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d804ef30 con 0x7f63d81f3b80 2024-04-25T10:25:03.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.316+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63c00be020 con 0x7f63d81f3b80 2024-04-25T10:25:03.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.316+0000 7f63d5600640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63b4066f90 0x7f63b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.316+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63c011fa00 con 0x7f63d81f3b80 2024-04-25T10:25:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.316+0000 7f63dca00640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63b4066f90 0x7f63b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.316+0000 7f63dca00640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63b4066f90 0x7f63b4069450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f63d8067980 tx=0x7f63c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:03.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.320+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63c00c5030 con 0x7f63d81f3b80 2024-04-25T10:25:03.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.516+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63d815ac40 con 0x7f63d81f3b80 2024-04-25T10:25:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63d5600640 1 -- 172.21.15.45:0/3379761292 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f63c00169a0 con 0x7f63d81f3b80 2024-04-25T10:25:03.520 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:03.520 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:24:17.216714+0000","modified":"2024-04-25T10:24:20.235839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7344},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7344":{"gid":7344,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/4102215497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4102215497},{"type":"v1","addr":"172.21.15.45:6834","nonce":4102215497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:25:03.520 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63b4066f90 msgr2=0x7f63b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63b4066f90 0x7f63b4069450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f63d8067980 tx=0x7f63c8008040 comp rx=0 tx=0).stop 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 msgr2=0x7f63d81572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 0x7f63d81572b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f63c00af040 tx=0x7f63bc005840 comp rx=0 tx=0).stop 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 shutdown_connections 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63b4066f90 0x7f63b4069450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63d81577f0 0x7f63d80b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63d81f3b80 0x7f63d81572b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 --2- 172.21.15.45:0/3379761292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63bc004810 0x7f63d814a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 >> 172.21.15.45:0/3379761292 conn(0x7f63d8066b70 msgr2=0x7f63d8154640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 shutdown_connections 2024-04-25T10:25:03.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.520+0000 7f63df400640 1 -- 172.21.15.45:0/3379761292 wait complete. 2024-04-25T10:25:03.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9299400640 1 Processor -- start 2024-04-25T10:25:03.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9299400640 1 -- start start 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9299400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92940cda10 0x7f929406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9299400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92940b2770 con 0x7f929415bc40 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9299400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929406c670 con 0x7f92940cf930 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9299400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929406c950 con 0x7f92940cda10 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9292800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f929415bc40 0x7f92940b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54776/0 (socket says 172.21.15.45:54776) 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9292800640 1 -- 172.21.15.45:0/1468497225 learned_addr learned my addr 172.21.15.45:0/1468497225 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:03.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9291e00640 1 --2- 172.21.15.45:0/1468497225 >> v2:172.21.15.45:3300/0 conn(0x7f92940cda10 0x7f929406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9291e00640 1 -- 172.21.15.45:0/1468497225 >> v1:172.21.15.117:6789/0 conn(0x7f92940cf930 legacy=0x7f92940b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:03.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9291e00640 1 -- 172.21.15.45:0/1468497225 >> v1:172.21.15.45:6789/0 conn(0x7f929415bc40 legacy=0x7f92940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.684+0000 7f9291e00640 1 -- 172.21.15.45:0/1468497225 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929406cc30 con 0x7f92940cda10 2024-04-25T10:25:03.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9291e00640 1 --2- 172.21.15.45:0/1468497225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f929406c090 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f92800097c0 tx=0x7f928001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8555ec2082ad3735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:03.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9291400640 1 -- 172.21.15.45:0/1468497225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92800b0070 con 0x7f92940cda10 2024-04-25T10:25:03.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9291400640 1 -- 172.21.15.45:0/1468497225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92800ab600 con 0x7f92940cda10 2024-04-25T10:25:03.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9291400640 1 -- 172.21.15.45:0/1468497225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92800ab9d0 con 0x7f92940cda10 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 -- 172.21.15.45:0/1468497225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 msgr2=0x7f929406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 --2- 172.21.15.45:0/1468497225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f929406c090 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f92800097c0 tx=0x7f928001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 -- 172.21.15.45:0/1468497225 shutdown_connections 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 --2- 172.21.15.45:0/1468497225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f929406c090 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 -- 172.21.15.45:0/1468497225 >> 172.21.15.45:0/1468497225 conn(0x7f9294066b70 msgr2=0x7f9294066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 -- 172.21.15.45:0/1468497225 shutdown_connections 2024-04-25T10:25:03.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 -- 172.21.15.45:0/1468497225 wait complete. 2024-04-25T10:25:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 Processor -- start 2024-04-25T10:25:03.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.688+0000 7f9299400640 1 -- start start 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f9294143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92940cf930 0x7f9294145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f929415bc40 0x7f929415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9294169d70 con 0x7f92940cda10 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9294169bf0 con 0x7f929415bc40 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9294169ef0 con 0x7f92940cf930 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f9294143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f9294143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55126/0 (socket says 172.21.15.45:55126) 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 -- 172.21.15.45:0/241344889 learned_addr learned my addr 172.21.15.45:0/241344889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9291e00640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92940cf930 0x7f9294145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9293200640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f929415bc40 0x7f929415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 -- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92940cf930 msgr2=0x7f9294145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92940cf930 0x7f9294145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 -- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f929415bc40 msgr2=0x7f929415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f929415bc40 0x7f929415e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 -- 172.21.15.45:0/241344889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92941439b0 con 0x7f92940cda10 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9292800640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f9294143470 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f928400c420 tx=0x7f928400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284017070 con 0x7f92940cda10 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9284005cf0 con 0x7f92940cda10 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92840072f0 con 0x7f92940cda10 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9294157260 con 0x7f92940cda10 2024-04-25T10:25:03.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9294157820 con 0x7f92940cda10 2024-04-25T10:25:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929404ef30 con 0x7f92940cda10 2024-04-25T10:25:03.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.692+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f928401a070 con 0x7f92940cda10 2024-04-25T10:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.696+0000 7f928aa00640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9274066f60 0x7f9274069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.696+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9284084410 con 0x7f92940cda10 2024-04-25T10:25:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.696+0000 7f9291e00640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9274066f60 0x7f9274069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.696+0000 7f9291e00640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9274066f60 0x7f9274069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92800ad000 tx=0x7f9280008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.700+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9284055400 con 0x7f92940cda10 2024-04-25T10:25:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.860+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92940af1c0 con 0x7f92940cda10 2024-04-25T10:25:03.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f928aa00640 1 -- 172.21.15.45:0/241344889 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16660 (secure 0 0 0) 0x7f9284054d00 con 0x7f92940cda10 2024-04-25T10:25:03.864 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:03.864 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:24:18.823568+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:24:15.637327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T10:24:16.209015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9274066f60 msgr2=0x7f9274069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9274066f60 0x7f9274069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92800ad000 tx=0x7f9280008040 comp rx=0 tx=0).stop 2024-04-25T10:25:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 msgr2=0x7f9294143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f9294143470 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f928400c420 tx=0x7f928400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 shutdown_connections 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9274066f60 0x7f9274069420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f929415bc40 0x7f929415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92940cf930 0x7f9294145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 --2- 172.21.15.45:0/241344889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92940cda10 0x7f9294143470 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 >> 172.21.15.45:0/241344889 conn(0x7f9294066b70 msgr2=0x7f929415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 shutdown_connections 2024-04-25T10:25:03.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:03.864+0000 7f9299400640 1 -- 172.21.15.45:0/241344889 wait complete. 2024-04-25T10:25:03.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:25:04.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.024+0000 7f3525400640 1 Processor -- start 2024-04-25T10:25:04.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f3525400640 1 -- start start 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f3525400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f352015bc40 0x7f352006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f3525400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200b2770 con 0x7f35200cda10 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f3525400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352006c670 con 0x7f35200cf930 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f3525400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352006c950 con 0x7f352015bc40 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f351e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f352015bc40 0x7f352006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f351e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f352015bc40 0x7f352006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55132/0 (socket says 172.21.15.45:55132) 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f351e800640 1 -- 172.21.15.45:0/45561740 learned_addr learned my addr 172.21.15.45:0/45561740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:04.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f3517e00640 1 --1- 172.21.15.45:0/45561740 >> v1:172.21.15.45:6789/0 conn(0x7f35200cda10 0x7f35200b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54792/0 (socket says 172.21.15.45:54792) 2024-04-25T10:25:04.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f351e800640 1 -- 172.21.15.45:0/45561740 >> v1:172.21.15.117:6789/0 conn(0x7f35200cf930 legacy=0x7f35200b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f351e800640 1 -- 172.21.15.45:0/45561740 >> v1:172.21.15.45:6789/0 conn(0x7f35200cda10 legacy=0x7f35200b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.028+0000 7f351e800640 1 -- 172.21.15.45:0/45561740 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352006cc30 con 0x7f352015bc40 2024-04-25T10:25:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f351e800640 1 --2- 172.21.15.45:0/45561740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015bc40 0x7f352006c090 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3510009870 tx=0x7f351001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c481cc813934a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f351de00640 1 -- 172.21.15.45:0/45561740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100b0070 con 0x7f352015bc40 2024-04-25T10:25:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f351de00640 1 -- 172.21.15.45:0/45561740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35100ab600 con 0x7f352015bc40 2024-04-25T10:25:04.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f351de00640 1 -- 172.21.15.45:0/45561740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100ab9d0 con 0x7f352015bc40 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 -- 172.21.15.45:0/45561740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015bc40 msgr2=0x7f352006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 --2- 172.21.15.45:0/45561740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015bc40 0x7f352006c090 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3510009870 tx=0x7f351001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 -- 172.21.15.45:0/45561740 shutdown_connections 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 --2- 172.21.15.45:0/45561740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015bc40 0x7f352006c090 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 -- 172.21.15.45:0/45561740 >> 172.21.15.45:0/45561740 conn(0x7f3520066b70 msgr2=0x7f3520066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 -- 172.21.15.45:0/45561740 shutdown_connections 2024-04-25T10:25:04.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 -- 172.21.15.45:0/45561740 wait complete. 2024-04-25T10:25:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 Processor -- start 2024-04-25T10:25:04.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.032+0000 7f3525400640 1 -- start start 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 0x7f35200b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35200cf930 0x7f352013cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f352015bc40 0x7f35201f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520169d70 con 0x7f35200cda10 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520169bf0 con 0x7f352015bc40 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3520169ef0 con 0x7f35200cf930 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 0x7f35200b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 0x7f35200b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55146/0 (socket says 172.21.15.45:55146) 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 -- 172.21.15.45:0/3648219109 learned_addr learned my addr 172.21.15.45:0/3648219109 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:04.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351f200640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f352015bc40 0x7f35201f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3517e00640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35200cf930 0x7f352013cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 -- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35200cf930 msgr2=0x7f352013cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35200cf930 0x7f352013cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 -- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f352015bc40 msgr2=0x7f35201f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f352015bc40 0x7f35201f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 -- 172.21.15.45:0/3648219109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35201fb440 con 0x7f35200cda10 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351f200640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f352015bc40 0x7f35201f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f351e800640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 0x7f35200b1de0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f35100ae040 tx=0x7f351001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100ba020 con 0x7f35200cda10 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35100b6640 con 0x7f35200cda10 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100b0040 con 0x7f35200cda10 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35201fb720 con 0x7f35200cda10 2024-04-25T10:25:04.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35201fbce0 con 0x7f35200cda10 2024-04-25T10:25:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352004ef30 con 0x7f35200cda10 2024-04-25T10:25:04.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.036+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35100bd020 con 0x7f35200cda10 2024-04-25T10:25:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.040+0000 7f3516a00640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34fc066f60 0x7f34fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.040+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f351011f3b0 con 0x7f35200cda10 2024-04-25T10:25:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.040+0000 7f3517e00640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34fc066f60 0x7f34fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.040+0000 7f3517e00640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34fc066f60 0x7f34fc069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3520067660 tx=0x7f3508007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.044+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35100c4030 con 0x7f35200cda10 2024-04-25T10:25:04.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.240+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f352006d960 con 0x7f35200cda10 2024-04-25T10:25:04.244 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:25:04.244+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:25:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.244+0000 7f3516a00640 1 -- 172.21.15.45:0/3648219109 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3510016150 con 0x7f35200cda10 2024-04-25T10:25:04.246 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:25:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34fc066f60 msgr2=0x7f34fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34fc066f60 0x7f34fc069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3520067660 tx=0x7f3508007c90 comp rx=0 tx=0).stop 2024-04-25T10:25:04.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 msgr2=0x7f35200b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 0x7f35200b1de0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f35100ae040 tx=0x7f351001ad10 comp rx=0 tx=0).stop 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 shutdown_connections 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34fc066f60 0x7f34fc069420 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f352015bc40 0x7f35201f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35200cf930 0x7f352013cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 --2- 172.21.15.45:0/3648219109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200cda10 0x7f35200b1de0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 >> 172.21.15.45:0/3648219109 conn(0x7f3520066b70 msgr2=0x7f3520156540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 shutdown_connections 2024-04-25T10:25:04.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.248+0000 7f3525400640 1 -- 172.21.15.45:0/3648219109 wait complete. 2024-04-25T10:25:04.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:25:04.326 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:25:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.420+0000 7fb613400640 1 Processor -- start 2024-04-25T10:25:04.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.420+0000 7fb613400640 1 -- start start 2024-04-25T10:25:04.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb613400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb60c0aeda0 0x7fb60c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb613400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c072020 0x7fb60c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb613400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c159920 con 0x7fb60c159540 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb613400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c15c2f0 con 0x7fb60c0aeda0 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb613400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c15c5d0 con 0x7fb60c072020 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb60c0aeda0 0x7fb60c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb60c0aeda0 0x7fb60c071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55184/0 (socket says 172.21.15.45:55184) 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 -- 172.21.15.45:0/3055756196 learned_addr learned my addr 172.21.15.45:0/3055756196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 -- 172.21.15.45:0/3055756196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c072020 msgr2=0x7fb60c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 --2- 172.21.15.45:0/3055756196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c072020 0x7fb60c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 -- 172.21.15.45:0/3055756196 >> v1:172.21.15.117:6789/0 conn(0x7fb60c159540 legacy=0x7fb60c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:04.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.424+0000 7fb603e00640 1 -- 172.21.15.45:0/3055756196 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c15c8b0 con 0x7fb60c0aeda0 2024-04-25T10:25:04.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb603e00640 1 --2- 172.21.15.45:0/3055756196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c0aeda0 0x7fb60c071ae0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009870 tx=0x7fb5f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a2517b8ea11e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:04.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb60be00640 1 -- 172.21.15.45:0/3055756196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40a7070 con 0x7fb60c0aeda0 2024-04-25T10:25:04.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb60be00640 1 -- 172.21.15.45:0/3055756196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f40b0070 con 0x7fb60c0aeda0 2024-04-25T10:25:04.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb60be00640 1 -- 172.21.15.45:0/3055756196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40ab890 con 0x7fb60c0aeda0 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 -- 172.21.15.45:0/3055756196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c0aeda0 msgr2=0x7fb60c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 --2- 172.21.15.45:0/3055756196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c0aeda0 0x7fb60c071ae0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009870 tx=0x7fb5f401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 -- 172.21.15.45:0/3055756196 shutdown_connections 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 --2- 172.21.15.45:0/3055756196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c072020 0x7fb60c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 --2- 172.21.15.45:0/3055756196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c0aeda0 0x7fb60c071ae0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 -- 172.21.15.45:0/3055756196 >> 172.21.15.45:0/3055756196 conn(0x7fb60c0669a0 msgr2=0x7fb60c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 -- 172.21.15.45:0/3055756196 shutdown_connections 2024-04-25T10:25:04.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 -- 172.21.15.45:0/3055756196 wait complete. 2024-04-25T10:25:04.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 Processor -- start 2024-04-25T10:25:04.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 -- start start 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c072020 0x7fb60c140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c0aeda0 0x7fb60c14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.428+0000 7fb613400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 0x7fb60c150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb613400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0710a0 con 0x7fb60c159540 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb613400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c06c0d0 con 0x7fb60c072020 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb613400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c071220 con 0x7fb60c0aeda0 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 0x7fb60c150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 0x7fb60c150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55194/0 (socket says 172.21.15.45:55194) 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 -- 172.21.15.45:0/321558596 learned_addr learned my addr 172.21.15.45:0/321558596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb610a00640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c072020 0x7fb60c140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb603e00640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c0aeda0 0x7fb60c14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 -- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c0aeda0 msgr2=0x7fb60c14e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c0aeda0 0x7fb60c14e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 -- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c072020 msgr2=0x7fb60c140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c072020 0x7fb60c140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 -- 172.21.15.45:0/321558596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c140aa0 con 0x7fb60c159540 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb611400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 0x7fb60c150360 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb60400b9a0 tx=0x7fb60400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604019070 con 0x7fb60c159540 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60400cbc0 con 0x7fb60c159540 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c151420 con 0x7fb60c159540 2024-04-25T10:25:04.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604014440 con 0x7fb60c159540 2024-04-25T10:25:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c151cc0 con 0x7fb60c159540 2024-04-25T10:25:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c04ef30 con 0x7fb60c159540 2024-04-25T10:25:04.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.432+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb60401c070 con 0x7fb60c159540 2024-04-25T10:25:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.436+0000 7fb609600640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5e8066f90 0x7fb5e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.436+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb604084e60 con 0x7fb60c159540 2024-04-25T10:25:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.436+0000 7fb610a00640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5e8066f90 0x7fb5e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:04.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.436+0000 7fb610a00640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5e8066f90 0x7fb5e8069450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc007fd0 tx=0x7fb5fc007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:04.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.440+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb604055690 con 0x7fb60c159540 2024-04-25T10:25:04.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:04.640+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb60c166700 con 0x7fb60c159540 2024-04-25T10:25:05.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb609600640 1 -- 172.21.15.45:0/321558596 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb604059730 con 0x7fb60c159540 2024-04-25T10:25:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5e8066f90 msgr2=0x7fb5e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5e8066f90 0x7fb5e8069450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc007fd0 tx=0x7fb5fc007e40 comp rx=0 tx=0).stop 2024-04-25T10:25:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 msgr2=0x7fb60c150360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 0x7fb60c150360 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb60400b9a0 tx=0x7fb60400be60 comp rx=0 tx=0).stop 2024-04-25T10:25:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 shutdown_connections 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5e8066f90 0x7fb5e8069450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb60c159540 0x7fb60c150360 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb60c0aeda0 0x7fb60c14e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 --2- 172.21.15.45:0/321558596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb60c072020 0x7fb60c140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 >> 172.21.15.45:0/321558596 conn(0x7fb60c0669a0 msgr2=0x7fb60c15f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 shutdown_connections 2024-04-25T10:25:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.248+0000 7fb613400640 1 -- 172.21.15.45:0/321558596 wait complete. 2024-04-25T10:25:05.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:25:05.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 Processor -- start 2024-04-25T10:25:05.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 -- start start 2024-04-25T10:25:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c8015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c801618a0 0x7f2c800b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8015e9b0 con 0x7f2c800b4d80 2024-04-25T10:25:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8015f250 con 0x7f2c801618a0 2024-04-25T10:25:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.460+0000 7f2c86600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800afce0 con 0x7f2c80162310 2024-04-25T10:25:05.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2c800b4d80 0x7f2c80072250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56252/0 (socket says 172.21.15.45:56252) 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7fe00640 1 -- 172.21.15.45:0/3409489075 learned_addr learned my addr 172.21.15.45:0/3409489075 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3409489075 >> v2:172.21.15.117:3300/0 conn(0x7f2c801618a0 0x7f2c800b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c8015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366886539 0 0) 0x7f2c8015e9b0 con 0x7f2c800b4d80 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c64003530 con 0x7f2c800b4d80 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3498276152 0 0) 0x7f2c64003530 con 0x7f2c800b4d80 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 msgr2=0x7f2c8015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c8015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 >> v2:172.21.15.117:3300/0 conn(0x7f2c801618a0 msgr2=0x7f2c800b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 --2- 172.21.15.45:0/3409489075 >> v2:172.21.15.117:3300/0 conn(0x7f2c801618a0 0x7f2c800b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8015ec90 con 0x7f2c800b4d80 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f2c780016a0 con 0x7f2c800b4d80 2024-04-25T10:25:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c8015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 >> v1:172.21.15.45:6789/0 conn(0x7f2c800b4d80 legacy=0x7f2c80072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64003f50 0x7f2c64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c64004d50 0x7f2c640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015f250 con 0x7f2c64003f50 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800afce0 con 0x7f2c64004d50 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6400bad0 con 0x7f2c80162310 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3409489075 >> v2:172.21.15.117:3300/0 conn(0x7f2c801618a0 0x7f2c800b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64003f50 0x7f2c64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c7fe00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c64004d50 0x7f2c640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 -- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 msgr2=0x7f2c640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c80162310 0x7f2c640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 -- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c64004d50 msgr2=0x7f2c640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c64004d50 0x7f2c640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.464+0000 7f2c76a00640 1 -- 172.21.15.45:0/3409489075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64003530 con 0x7f2c64003f50 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64003f50 0x7f2c64004810 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009cc0 tx=0x7f2c6801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401f7789a345329e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68009860 con 0x7f2c64003f50 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68004020 con 0x7f2c64003f50 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c7ea00640 1 -- 172.21.15.45:0/3409489075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68004450 con 0x7f2c64003f50 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 -- 172.21.15.45:0/3409489075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64003f50 msgr2=0x7f2c64004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 --2- 172.21.15.45:0/3409489075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c64003f50 0x7f2c64004810 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009cc0 tx=0x7f2c6801cc20 comp rx=0 tx=0).stop 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c7fe00640 1 -- 172.21.15.45:0/3409489075 reap_dead start 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 -- 172.21.15.45:0/3409489075 shutdown_connections 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 -- 172.21.15.45:0/3409489075 >> 172.21.15.45:0/3409489075 conn(0x7f2c800b5610 msgr2=0x7f2c800b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 -- 172.21.15.45:0/3409489075 shutdown_connections 2024-04-25T10:25:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 -- 172.21.15.45:0/3409489075 wait complete. 2024-04-25T10:25:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 Processor -- start 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.468+0000 7f2c86600640 1 -- start start 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c64003f50 0x7f2c8014d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 0x7f2c8014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c800aed60 0x7f2c800af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800b5190 con 0x7f2c80150de0 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801626f0 con 0x7f2c800aed60 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80168a20 con 0x7f2c64003f50 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 0x7f2c8014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 0x7f2c8014f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41496/0 (socket says 172.21.15.45:41496) 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 -- 172.21.15.45:0/3750759620 learned_addr learned my addr 172.21.15.45:0/3750759620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c64003f50 0x7f2c8014d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c7fe00640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c800aed60 0x7f2c800af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 -- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c64003f50 msgr2=0x7f2c8014d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c64003f50 0x7f2c8014d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 -- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c800aed60 msgr2=0x7f2c800af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c800aed60 0x7f2c800af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 -- 172.21.15.45:0/3750759620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c801513f0 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c7fe00640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c800aed60 0x7f2c800af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c64003f50 0x7f2c8014d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c76a00640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 0x7f2c8014f550 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2c680af040 tx=0x7f2c64005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c680bb020 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c680059a0 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c800ae280 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68005d50 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c800ae840 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c801687d0 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.472+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c680be020 con 0x7f2c80150de0 2024-04-25T10:25:05.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.476+0000 7f2c77e00640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c5c066f60 0x7f2c5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.476+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2c6811f7f0 con 0x7f2c80150de0 2024-04-25T10:25:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.476+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c5c066f60 0x7f2c5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.476+0000 7f2c7f400640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c5c066f60 0x7f2c5c069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2c70012290 tx=0x7f2c7000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:05.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.480+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c680c5030 con 0x7f2c80150de0 2024-04-25T10:25:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:05.640+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c8004ef30 con 0x7f2c80150de0 2024-04-25T10:25:06.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.248+0000 7f2c77e00640 1 -- 172.21.15.45:0/3750759620 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2c680169a0 con 0x7f2c80150de0 2024-04-25T10:25:06.250 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:25:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c5c066f60 msgr2=0x7f2c5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c5c066f60 0x7f2c5c069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2c70012290 tx=0x7f2c7000b040 comp rx=0 tx=0).stop 2024-04-25T10:25:06.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 msgr2=0x7f2c8014f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 0x7f2c8014f550 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2c680af040 tx=0x7f2c64005840 comp rx=0 tx=0).stop 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 shutdown_connections 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c5c066f60 0x7f2c5c069420 secure :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2c70012290 tx=0x7f2c7000b040 comp rx=0 tx=0).stop 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c800aed60 0x7f2c800af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c80150de0 0x7f2c8014f550 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 --2- 172.21.15.45:0/3750759620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c64003f50 0x7f2c8014d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 >> 172.21.15.45:0/3750759620 conn(0x7f2c800b5610 msgr2=0x7f2c800b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 shutdown_connections 2024-04-25T10:25:06.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.252+0000 7f2c86600640 1 -- 172.21.15.45:0/3750759620 wait complete. 2024-04-25T10:25:06.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:25:06.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.424+0000 7fb271c00640 1 Processor -- start 2024-04-25T10:25:06.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.424+0000 7fb271c00640 1 -- start start 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb271c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c0cf9e0 0x7fb26c0cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb271c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c06c0e0 con 0x7fb26c15c0f0 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb271c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c15c6a0 con 0x7fb26c15bd10 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb271c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0b2ef0 con 0x7fb26c0cf9e0 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb263e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb26c15c0f0 0x7fb26c0b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56266/0 (socket says 172.21.15.45:56266) 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb263e00640 1 -- 172.21.15.45:0/3138043656 learned_addr learned my addr 172.21.15.45:0/3138043656 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:06.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26b400640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c0cf9e0 0x7fb26c0cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975553344 0 0) 0x7fb26c06c0e0 con 0x7fb26c15c0f0 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb250003530 con 0x7fb26c15c0f0 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443905936 0 0) 0x7fb26c15c6a0 con 0x7fb26c15bd10 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26c06c0e0 con 0x7fb26c15bd10 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240483474 0 0) 0x7fb250003530 con 0x7fb26c15c0f0 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c0cf9e0 msgr2=0x7fb26c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c0cf9e0 0x7fb26c0cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 >> v1:172.21.15.117:6789/0 conn(0x7fb26c15bd10 legacy=0x7fb26c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c15c980 con 0x7fb26c15c0f0 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fb254001680 con 0x7fb26c15c0f0 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 >> v1:172.21.15.45:6789/0 conn(0x7fb26c15c0f0 legacy=0x7fb26c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb250003b80 0x7fb250003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb250004810 0x7fb250004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2500051d0 0x7fb25000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0b2ef0 con 0x7fb250003b80 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25000bff0 con 0x7fb2500051d0 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25000c1f0 con 0x7fb250004810 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26b400640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c0cf9e0 0x7fb26c0cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb250003b80 0x7fb250003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb263e00640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb250004810 0x7fb250004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26b400640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2500051d0 0x7fb25000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26aa00640 1 -- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb250004810 msgr2=0x7fb250004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:06.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb250004810 0x7fb250004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26aa00640 1 -- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2500051d0 msgr2=0x7fb25000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:06.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2500051d0 0x7fb25000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.428+0000 7fb26aa00640 1 -- 172.21.15.45:0/3138043656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250003530 con 0x7fb250003b80 2024-04-25T10:25:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb250003b80 0x7fb250003f80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0018c0 tx=0x7fb25c01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447689c4f2c4eec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c001510 con 0x7fb250003b80 2024-04-25T10:25:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c0052d0 con 0x7fb250003b80 2024-04-25T10:25:06.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb26a000640 1 -- 172.21.15.45:0/3138043656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003820 con 0x7fb250003b80 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 -- 172.21.15.45:0/3138043656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb250003b80 msgr2=0x7fb250003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 --2- 172.21.15.45:0/3138043656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb250003b80 0x7fb250003f80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0018c0 tx=0x7fb25c01c580 comp rx=0 tx=0).stop 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb26b400640 1 -- 172.21.15.45:0/3138043656 reap_dead start 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 -- 172.21.15.45:0/3138043656 shutdown_connections 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 -- 172.21.15.45:0/3138043656 >> 172.21.15.45:0/3138043656 conn(0x7fb26c066b70 msgr2=0x7fb26c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 -- 172.21.15.45:0/3138043656 shutdown_connections 2024-04-25T10:25:06.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 -- 172.21.15.45:0/3138043656 wait complete. 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 Processor -- start 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.432+0000 7fb271c00640 1 -- start start 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb250003b80 0x7fb26c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 0x7fb26c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c159db0 0x7fb26c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c06d0a0 con 0x7fb26c1540d0 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0d0390 con 0x7fb250003b80 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c167ac0 con 0x7fb26c159db0 2024-04-25T10:25:06.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 0x7fb26c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 0x7fb26c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41522/0 (socket says 172.21.15.45:41522) 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 -- 172.21.15.45:0/3622152401 learned_addr learned my addr 172.21.15.45:0/3622152401 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb250003b80 0x7fb26c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb26b400640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c159db0 0x7fb26c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 -- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c159db0 msgr2=0x7fb26c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c159db0 0x7fb26c155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 -- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb250003b80 msgr2=0x7fb26c1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb250003b80 0x7fb26c1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 -- 172.21.15.45:0/3622152401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c152cf0 con 0x7fb26c1540d0 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb26b400640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c159db0 0x7fb26c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb263e00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 0x7fb26c159870 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb254000fc0 tx=0x7fb25400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:06.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400c6c0 con 0x7fb26c1540d0 2024-04-25T10:25:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb254018070 con 0x7fb26c1540d0 2024-04-25T10:25:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c1568f0 con 0x7fb26c1540d0 2024-04-25T10:25:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400cac0 con 0x7fb26c1540d0 2024-04-25T10:25:06.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c1538d0 con 0x7fb26c1540d0 2024-04-25T10:25:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c15a1c0 con 0x7fb26c1540d0 2024-04-25T10:25:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.436+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb25400cc60 con 0x7fb26c1540d0 2024-04-25T10:25:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.440+0000 7fb262a00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb248066f90 0x7fb248069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.440+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fb254084c60 con 0x7fb26c1540d0 2024-04-25T10:25:06.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.440+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb248066f90 0x7fb248069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.440+0000 7fb26aa00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb248066f90 0x7fb248069450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ac000 tx=0x7fb25c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:06.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.444+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb26c15a1c0 con 0x7fb26c1540d0 2024-04-25T10:25:06.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:06.608+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb26c04ef30 con 0x7fb26c1540d0 2024-04-25T10:25:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.252+0000 7fb262a00640 1 -- 172.21.15.45:0/3622152401 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb254055d30 con 0x7fb26c1540d0 2024-04-25T10:25:07.255 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:25:07.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb248066f90 msgr2=0x7fb248069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb248066f90 0x7fb248069450 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ac000 tx=0x7fb25c009040 comp rx=0 tx=0).stop 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 msgr2=0x7fb26c159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 0x7fb26c159870 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb254000fc0 tx=0x7fb25400ec10 comp rx=0 tx=0).stop 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 shutdown_connections 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb248066f90 0x7fb248069450 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0ac000 tx=0x7fb25c009040 comp rx=0 tx=0).stop 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb26c159db0 0x7fb26c155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c1540d0 0x7fb26c159870 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 --2- 172.21.15.45:0/3622152401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb250003b80 0x7fb26c1527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 >> 172.21.15.45:0/3622152401 conn(0x7fb26c066b70 msgr2=0x7fb26c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 shutdown_connections 2024-04-25T10:25:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.256+0000 7fb271c00640 1 -- 172.21.15.45:0/3622152401 wait complete. 2024-04-25T10:25:07.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:25:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.460+0000 7fccb3000640 1 Processor -- start 2024-04-25T10:25:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.460+0000 7fccb3000640 1 -- start start 2024-04-25T10:25:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb3000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac070350 0x7fccac15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb3000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fccac0b31c0 0x7fccac0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb3000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac15b080 con 0x7fccac0aef80 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb3000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac15adb0 con 0x7fccac0b31c0 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb3000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15eb60 con 0x7fccac070350 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fccac0b31c0 0x7fccac0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fccac0b31c0 0x7fccac0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41536/0 (socket says 172.21.15.45:41536) 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 -- 172.21.15.45:0/3672142858 learned_addr learned my addr 172.21.15.45:0/3672142858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 -- 172.21.15.45:0/3672142858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac070350 msgr2=0x7fccac15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 --2- 172.21.15.45:0/3672142858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac070350 0x7fccac15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 -- 172.21.15.45:0/3672142858 >> v1:172.21.15.117:6789/0 conn(0x7fccac0aef80 legacy=0x7fccac15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.464+0000 7fccb1000640 1 -- 172.21.15.45:0/3672142858 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac15ee40 con 0x7fccac0b31c0 2024-04-25T10:25:07.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb1000640 1 --2- 172.21.15.45:0/3672142858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0b31c0 0x7fccac0b3610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fcca400d150 tx=0x7fcca401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c304a180bdfec92c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccab400640 1 -- 172.21.15.45:0/3672142858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400b2d0 con 0x7fccac0b31c0 2024-04-25T10:25:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccab400640 1 -- 172.21.15.45:0/3672142858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca400b470 con 0x7fccac0b31c0 2024-04-25T10:25:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccab400640 1 -- 172.21.15.45:0/3672142858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40a7bb0 con 0x7fccac0b31c0 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- 172.21.15.45:0/3672142858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0b31c0 msgr2=0x7fccac0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 --2- 172.21.15.45:0/3672142858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0b31c0 0x7fccac0b3610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fcca400d150 tx=0x7fcca401ebf0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- 172.21.15.45:0/3672142858 shutdown_connections 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 --2- 172.21.15.45:0/3672142858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0b31c0 0x7fccac0b3610 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 --2- 172.21.15.45:0/3672142858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac070350 0x7fccac15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- 172.21.15.45:0/3672142858 >> 172.21.15.45:0/3672142858 conn(0x7fccac0669a0 msgr2=0x7fccac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- 172.21.15.45:0/3672142858 shutdown_connections 2024-04-25T10:25:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- 172.21.15.45:0/3672142858 wait complete. 2024-04-25T10:25:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 Processor -- start 2024-04-25T10:25:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- start start 2024-04-25T10:25:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccac070350 0x7fccac0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 0x7fccac1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac0b31c0 0x7fccac159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac06c250 con 0x7fccac0aef80 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac06c0d0 con 0x7fccac070350 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccb3000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac06c3d0 con 0x7fccac0b31c0 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fcca3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 0x7fccac1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fccabe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccac070350 0x7fccac0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fcca3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 0x7fccac1528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41540/0 (socket says 172.21.15.45:41540) 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.468+0000 7fcca3400640 1 -- 172.21.15.45:0/591835748 learned_addr learned my addr 172.21.15.45:0/591835748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fccb1000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac0b31c0 0x7fccac159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca3400640 1 -- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac0b31c0 msgr2=0x7fccac159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca3400640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac0b31c0 0x7fccac159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca3400640 1 -- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccac070350 msgr2=0x7fccac0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca3400640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccac070350 0x7fccac0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca3400640 1 -- 172.21.15.45:0/591835748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0af920 con 0x7fccac0aef80 2024-04-25T10:25:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca3400640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 0x7fccac1528a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400b550 tx=0x7fcc9400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:07.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94014070 con 0x7fccac0aef80 2024-04-25T10:25:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc94010890 con 0x7fccac0aef80 2024-04-25T10:25:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac0afba0 con 0x7fccac0aef80 2024-04-25T10:25:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94010c80 con 0x7fccac0aef80 2024-04-25T10:25:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac0b0160 con 0x7fccac0aef80 2024-04-25T10:25:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac04ef30 con 0x7fccac0aef80 2024-04-25T10:25:07.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc94017070 con 0x7fccac0aef80 2024-04-25T10:25:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca8c00640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc88066f60 0x7fcc88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.472+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcc94084a00 con 0x7fccac0aef80 2024-04-25T10:25:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.476+0000 7fccabe00640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc88066f60 0x7fcc88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.476+0000 7fccabe00640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc88066f60 0x7fcc88069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c007fd0 tx=0x7fcc9c007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:07.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.476+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fccac0af920 con 0x7fccac0aef80 2024-04-25T10:25:07.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.640+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fccac0b2fa0 con 0x7fccac0aef80 2024-04-25T10:25:07.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fcca8c00640 1 -- 172.21.15.45:0/591835748 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcc94010e20 con 0x7fccac0aef80 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc88066f60 msgr2=0x7fcc88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc88066f60 0x7fcc88069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c007fd0 tx=0x7fcc9c007e40 comp rx=0 tx=0).stop 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 msgr2=0x7fccac1528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 0x7fccac1528a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400b550 tx=0x7fcc9400ba10 comp rx=0 tx=0).stop 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 shutdown_connections 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcc88066f60 0x7fcc88069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccac0b31c0 0x7fccac159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccac0aef80 0x7fccac1528a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 --2- 172.21.15.45:0/591835748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccac070350 0x7fccac0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 >> 172.21.15.45:0/591835748 conn(0x7fccac0669a0 msgr2=0x7fccac15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 shutdown_connections 2024-04-25T10:25:07.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.644+0000 7fccb3000640 1 -- 172.21.15.45:0/591835748 wait complete. 2024-04-25T10:25:07.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:25:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 Processor -- start 2024-04-25T10:25:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 -- start start 2024-04-25T10:25:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95f80b35d0 0x7f95f8161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f815ed90 0x7f95f8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f8162350 con 0x7f95f8143d40 2024-04-25T10:25:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f8162110 con 0x7f95f80b35d0 2024-04-25T10:25:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95fd000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f8144120 con 0x7f95f815ed90 2024-04-25T10:25:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95f80b35d0 0x7f95f8161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95f80b35d0 0x7f95f8161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41548/0 (socket says 172.21.15.45:41548) 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 -- 172.21.15.45:0/3466172656 learned_addr learned my addr 172.21.15.45:0/3466172656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f7e00640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f815ed90 0x7f95f8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 -- 172.21.15.45:0/3466172656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f815ed90 msgr2=0x7f95f8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f815ed90 0x7f95f8161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 -- 172.21.15.45:0/3466172656 >> v1:172.21.15.117:6789/0 conn(0x7f95f8143d40 legacy=0x7f95f80b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f6a00640 1 -- 172.21.15.45:0/3466172656 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f815e2b0 con 0x7f95f80b35d0 2024-04-25T10:25:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.808+0000 7f95f7e00640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f815ed90 0x7f95f8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95f6a00640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f80b35d0 0x7f95f8161690 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009870 tx=0x7f95e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc7728f81473d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95f6000640 1 -- 172.21.15.45:0/3466172656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b0070 con 0x7f95f80b35d0 2024-04-25T10:25:07.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95f6000640 1 -- 172.21.15.45:0/3466172656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e40ab600 con 0x7f95f80b35d0 2024-04-25T10:25:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95f6000640 1 -- 172.21.15.45:0/3466172656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40ab9d0 con 0x7f95f80b35d0 2024-04-25T10:25:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 -- 172.21.15.45:0/3466172656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f80b35d0 msgr2=0x7f95f8161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f80b35d0 0x7f95f8161690 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009870 tx=0x7f95e401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 -- 172.21.15.45:0/3466172656 shutdown_connections 2024-04-25T10:25:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f815ed90 0x7f95f8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 --2- 172.21.15.45:0/3466172656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f80b35d0 0x7f95f8161690 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 -- 172.21.15.45:0/3466172656 >> 172.21.15.45:0/3466172656 conn(0x7f95f806cd20 msgr2=0x7f95f806d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 -- 172.21.15.45:0/3466172656 shutdown_connections 2024-04-25T10:25:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 -- 172.21.15.45:0/3466172656 wait complete. 2024-04-25T10:25:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 Processor -- start 2024-04-25T10:25:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.812+0000 7f95fd000640 1 -- start start 2024-04-25T10:25:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f80b35d0 0x7f95f813b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95f8143d40 0x7f95f813b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 0x7f95f813c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80ae3e0 con 0x7f95f815ed90 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80ae260 con 0x7f95f8143d40 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80ae560 con 0x7f95f80b35d0 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 0x7f95f813c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f80b35d0 0x7f95f813b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 0x7f95f813c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41562/0 (socket says 172.21.15.45:41562) 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 -- 172.21.15.45:0/2595954541 learned_addr learned my addr 172.21.15.45:0/2595954541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f6a00640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95f8143d40 0x7f95f813b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 -- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f80b35d0 msgr2=0x7f95f813b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f80b35d0 0x7f95f813b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 -- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95f8143d40 msgr2=0x7f95f813b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95f8143d40 0x7f95f813b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 -- 172.21.15.45:0/2595954541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8141fc0 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95f7e00640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 0x7f95f813c030 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009fd0 tx=0x7f95f000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f001a070 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f0003bc0 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0004030 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f81422a0 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f813aba0 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.816+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f80745a0 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.820+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95f001d070 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.820+0000 7f95ef400640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95d8066f90 0x7f95d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.820+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f95f0084ff0 con 0x7f95f815ed90 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.820+0000 7f95f7400640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95d8066f90 0x7f95d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:07.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.820+0000 7f95f7400640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95d8066f90 0x7f95d8069450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007fd0 tx=0x7f95e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:07.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:07.824+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95f0010070 con 0x7f95f815ed90 2024-04-25T10:25:08.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.040+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f95f8075120 con 0x7f95f815ed90 2024-04-25T10:25:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95ef400640 1 -- 172.21.15.45:0/2595954541 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v347) v1 ==== 144+0+0 (secure 0 0 0) 0x7f95f0055c50 con 0x7f95f815ed90 2024-04-25T10:25:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95d8066f90 msgr2=0x7f95d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95d8066f90 0x7f95d8069450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007fd0 tx=0x7f95e8009040 comp rx=0 tx=0).stop 2024-04-25T10:25:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 msgr2=0x7f95f813c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 0x7f95f813c030 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009fd0 tx=0x7f95f000d560 comp rx=0 tx=0).stop 2024-04-25T10:25:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 shutdown_connections 2024-04-25T10:25:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95d8066f90 0x7f95d8069450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95f815ed90 0x7f95f813c030 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95f8143d40 0x7f95f813b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 --2- 172.21.15.45:0/2595954541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95f80b35d0 0x7f95f813b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 >> 172.21.15.45:0/2595954541 conn(0x7f95f806cd20 msgr2=0x7f95f815f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 shutdown_connections 2024-04-25T10:25:08.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.256+0000 7f95fd000640 1 -- 172.21.15.45:0/2595954541 wait complete. 2024-04-25T10:25:08.279 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:25:08.280 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:25:08.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 Processor -- start 2024-04-25T10:25:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 -- start start 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feca00d0040 0x7feca015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feca015b740 0x7feca00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca0071050 con 0x7feca01550b0 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca015d040 con 0x7feca015b740 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7feca5200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca00712f0 con 0x7feca00d0040 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7fec9e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feca015b740 0x7feca00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7fec9e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feca015b740 0x7feca00b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41564/0 (socket says 172.21.15.45:41564) 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7fec9e800640 1 -- 172.21.15.45:0/3198574171 learned_addr learned my addr 172.21.15.45:0/3198574171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.420+0000 7fec95400640 1 --2- 172.21.15.45:0/3198574171 >> v2:172.21.15.117:3300/0 conn(0x7feca00d0040 0x7feca015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9e800640 1 -- 172.21.15.45:0/3198574171 >> v2:172.21.15.117:3300/0 conn(0x7feca00d0040 msgr2=0x7feca015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9e800640 1 --2- 172.21.15.45:0/3198574171 >> v2:172.21.15.117:3300/0 conn(0x7feca00d0040 0x7feca015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9e800640 1 -- 172.21.15.45:0/3198574171 >> v1:172.21.15.117:6789/0 conn(0x7feca01550b0 legacy=0x7feca00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9e800640 1 -- 172.21.15.45:0/3198574171 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00715d0 con 0x7feca015b740 2024-04-25T10:25:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9e800640 1 --2- 172.21.15.45:0/3198574171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca015b740 0x7feca00b3260 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7feca0070160 tx=0x7fec9801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a6959162b4fef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9d400640 1 -- 172.21.15.45:0/3198574171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980a7aa0 con 0x7feca015b740 2024-04-25T10:25:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9d400640 1 -- 172.21.15.45:0/3198574171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec980a7c40 con 0x7feca015b740 2024-04-25T10:25:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7fec9d400640 1 -- 172.21.15.45:0/3198574171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980af3f0 con 0x7feca015b740 2024-04-25T10:25:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 -- 172.21.15.45:0/3198574171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca015b740 msgr2=0x7feca00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 --2- 172.21.15.45:0/3198574171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca015b740 0x7feca00b3260 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7feca0070160 tx=0x7fec9801ebd0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 -- 172.21.15.45:0/3198574171 shutdown_connections 2024-04-25T10:25:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 --2- 172.21.15.45:0/3198574171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca015b740 0x7feca00b3260 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 --2- 172.21.15.45:0/3198574171 >> v2:172.21.15.117:3300/0 conn(0x7feca00d0040 0x7feca015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 -- 172.21.15.45:0/3198574171 >> 172.21.15.45:0/3198574171 conn(0x7feca0066b70 msgr2=0x7feca0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.424+0000 7feca5200640 1 -- 172.21.15.45:0/3198574171 shutdown_connections 2024-04-25T10:25:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- 172.21.15.45:0/3198574171 wait complete. 2024-04-25T10:25:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 Processor -- start 2024-04-25T10:25:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- start start 2024-04-25T10:25:08.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca00d0040 0x7feca0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 0x7feca013d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feca015b740 0x7feca006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca015bd10 con 0x7feca01550b0 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca015bb90 con 0x7feca00d0040 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca015c030 con 0x7feca015b740 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 0x7feca013d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 0x7feca013d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41568/0 (socket says 172.21.15.45:41568) 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 -- 172.21.15.45:0/2143883882 learned_addr learned my addr 172.21.15.45:0/2143883882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec9de00640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca00d0040 0x7feca0152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec9e800640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feca015b740 0x7feca006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 -- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feca015b740 msgr2=0x7feca006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feca015b740 0x7feca006d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 -- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca00d0040 msgr2=0x7feca0152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca00d0040 0x7feca0152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 -- 172.21.15.45:0/2143883882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca013da50 con 0x7feca01550b0 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec9de00640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca00d0040 0x7feca0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec95400640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 0x7feca013d510 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e930 tx=0x7fec8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88017070 con 0x7feca01550b0 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8800ca00 con 0x7feca01550b0 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8800cdb0 con 0x7feca01550b0 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca0158240 con 0x7feca01550b0 2024-04-25T10:25:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.428+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca0151840 con 0x7feca01550b0 2024-04-25T10:25:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.432+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca004ef30 con 0x7feca01550b0 2024-04-25T10:25:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.432+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec8801a070 con 0x7feca01550b0 2024-04-25T10:25:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.432+0000 7fec96a00640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec7c066f60 0x7fec7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.432+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fec88084e00 con 0x7feca01550b0 2024-04-25T10:25:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.432+0000 7fec9de00640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec7c066f60 0x7fec7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.432+0000 7fec9de00640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec7c066f60 0x7fec7c069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fec90001710 tx=0x7fec90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.436+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec88056130 con 0x7feca01550b0 2024-04-25T10:25:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.600+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feca00ae7b0 con 0x7feca01550b0 2024-04-25T10:25:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.600+0000 7fec96a00640 1 -- 172.21.15.45:0/2143883882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+13009 (secure 0 0 0) 0x7fec88055a30 con 0x7feca01550b0 2024-04-25T10:25:08.603 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:08.603 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:07.251514+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/4102215497":"2024-04-26T10:25:04.245641+0000","172.21.15.45:6832/4102215497":"2024-04-26T10:25:04.245641+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec7c066f60 msgr2=0x7fec7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec7c066f60 0x7fec7c069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fec90001710 tx=0x7fec90009040 comp rx=0 tx=0).stop 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 msgr2=0x7feca013d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 0x7feca013d510 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e930 tx=0x7fec8800edf0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 shutdown_connections 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec7c066f60 0x7fec7c069420 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feca015b740 0x7feca006d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feca01550b0 0x7feca013d510 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 --2- 172.21.15.45:0/2143883882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feca00d0040 0x7feca0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 >> 172.21.15.45:0/2143883882 conn(0x7feca0066b70 msgr2=0x7feca00afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 shutdown_connections 2024-04-25T10:25:08.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.604+0000 7feca5200640 1 -- 172.21.15.45:0/2143883882 wait complete. 2024-04-25T10:25:08.624 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:08.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:08.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 Processor -- start 2024-04-25T10:25:08.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 -- start start 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59d40cd9a0 0x7f59d40d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f59d406bd10 0x7f59d4161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d406ac90 con 0x7f59d40d0ba0 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d406b880 con 0x7f59d40cd9a0 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59da800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d406af50 con 0x7f59d406bd10 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59d40cd9a0 0x7f59d40d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59d40cd9a0 0x7f59d40d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41572/0 (socket says 172.21.15.45:41572) 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 -- 172.21.15.45:0/3962156770 learned_addr learned my addr 172.21.15.45:0/3962156770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59caa00640 1 --2- 172.21.15.45:0/3962156770 >> v2:172.21.15.117:3300/0 conn(0x7f59d406bd10 0x7f59d4161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 -- 172.21.15.45:0/3962156770 >> v2:172.21.15.117:3300/0 conn(0x7f59d406bd10 msgr2=0x7f59d4161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 --2- 172.21.15.45:0/3962156770 >> v2:172.21.15.117:3300/0 conn(0x7f59d406bd10 0x7f59d4161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 -- 172.21.15.45:0/3962156770 >> v1:172.21.15.117:6789/0 conn(0x7f59d40d0ba0 legacy=0x7f59d415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.768+0000 7f59d3400640 1 -- 172.21.15.45:0/3962156770 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d406b230 con 0x7f59d40cd9a0 2024-04-25T10:25:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59d3400640 1 --2- 172.21.15.45:0/3962156770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d40cd9a0 0x7f59d40d0660 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f59d4073300 tx=0x7f59c401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90ff65ae9b13672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59d2a00640 1 -- 172.21.15.45:0/3962156770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40b5070 con 0x7f59d40cd9a0 2024-04-25T10:25:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59d2a00640 1 -- 172.21.15.45:0/3962156770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40a8a10 con 0x7f59d40cd9a0 2024-04-25T10:25:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59d2a00640 1 -- 172.21.15.45:0/3962156770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40a8de0 con 0x7f59d40cd9a0 2024-04-25T10:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 -- 172.21.15.45:0/3962156770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d40cd9a0 msgr2=0x7f59d40d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 --2- 172.21.15.45:0/3962156770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d40cd9a0 0x7f59d40d0660 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f59d4073300 tx=0x7f59c401ed10 comp rx=0 tx=0).stop 2024-04-25T10:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 -- 172.21.15.45:0/3962156770 shutdown_connections 2024-04-25T10:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 --2- 172.21.15.45:0/3962156770 >> v2:172.21.15.117:3300/0 conn(0x7f59d406bd10 0x7f59d4161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 --2- 172.21.15.45:0/3962156770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d40cd9a0 0x7f59d40d0660 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 -- 172.21.15.45:0/3962156770 >> 172.21.15.45:0/3962156770 conn(0x7f59d40d19a0 msgr2=0x7f59d40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 -- 172.21.15.45:0/3962156770 shutdown_connections 2024-04-25T10:25:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.772+0000 7f59da800640 1 -- 172.21.15.45:0/3962156770 wait complete. 2024-04-25T10:25:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 Processor -- start 2024-04-25T10:25:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- start start 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 0x7f59d40c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59d40cd9a0 0x7f59d409ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d40d0ba0 0x7f59d4154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4162660 con 0x7f59d406bd10 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40cdd80 con 0x7f59d40d0ba0 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4161e50 con 0x7f59d40cd9a0 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 0x7f59d40c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 0x7f59d40c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41586/0 (socket says 172.21.15.45:41586) 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 -- 172.21.15.45:0/3684691741 learned_addr learned my addr 172.21.15.45:0/3684691741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3e00640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d40d0ba0 0x7f59d4154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59caa00640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59d40cd9a0 0x7f59d409ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 -- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59d40cd9a0 msgr2=0x7f59d409ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59d40cd9a0 0x7f59d409ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 -- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d40d0ba0 msgr2=0x7f59d4154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d40d0ba0 0x7f59d4154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 -- 172.21.15.45:0/3684691741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d409ba70 con 0x7f59d406bd10 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59caa00640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59d40cd9a0 0x7f59d409ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59d3400640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 0x7f59d40c5860 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f59c401b920 tx=0x7f59c401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40b5070 con 0x7f59d406bd10 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c40b0a50 con 0x7f59d406bd10 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d409bd50 con 0x7f59d406bd10 2024-04-25T10:25:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40b0e40 con 0x7f59d406bd10 2024-04-25T10:25:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d409e540 con 0x7f59d406bd10 2024-04-25T10:25:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.776+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d404ef30 con 0x7f59d406bd10 2024-04-25T10:25:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.780+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59c40c2060 con 0x7f59d406bd10 2024-04-25T10:25:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.780+0000 7f59cbe00640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59b0066f60 0x7f59b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.780+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f59c4121ec0 con 0x7f59d406bd10 2024-04-25T10:25:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.780+0000 7f59caa00640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59b0066f60 0x7f59b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.780+0000 7f59caa00640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59b0066f60 0x7f59b0069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ccf30 tx=0x7f59bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.784+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59c4018870 con 0x7f59d406bd10 2024-04-25T10:25:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.948+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59d40b7920 con 0x7f59d406bd10 2024-04-25T10:25:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.948+0000 7f59cbe00640 1 -- 172.21.15.45:0/3684691741 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59c40f3130 con 0x7f59d406bd10 2024-04-25T10:25:08.952 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59b0066f60 msgr2=0x7f59b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59b0066f60 0x7f59b0069420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ccf30 tx=0x7f59bc008040 comp rx=0 tx=0).stop 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 msgr2=0x7f59d40c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 0x7f59d40c5860 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f59c401b920 tx=0x7f59c401bab0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 shutdown_connections 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59b0066f60 0x7f59b0069420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d40d0ba0 0x7f59d4154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59d40cd9a0 0x7f59d409ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 --2- 172.21.15.45:0/3684691741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59d406bd10 0x7f59d40c5860 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 >> 172.21.15.45:0/3684691741 conn(0x7f59d40d19a0 msgr2=0x7f59d406f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 shutdown_connections 2024-04-25T10:25:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:08.952+0000 7f59da800640 1 -- 172.21.15.45:0/3684691741 wait complete. 2024-04-25T10:25:08.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:25:09.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 Processor -- start 2024-04-25T10:25:09.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 -- start start 2024-04-25T10:25:09.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9be415b9e0 0x7f9be40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:09.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40d1c30 0x7f9be40cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:09.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be40ccfb0 con 0x7f9be415bdc0 2024-04-25T10:25:09.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be40cd290 con 0x7f9be415b9e0 2024-04-25T10:25:09.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.112+0000 7f9bea200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40cd570 con 0x7f9be40d1c30 2024-04-25T10:25:09.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9be415b9e0 0x7f9be40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:09.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9bdaa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9be415bdc0 0x7f9be40d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56276/0 (socket says 172.21.15.45:56276) 2024-04-25T10:25:09.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9bdaa00640 1 -- 172.21.15.45:0/2484203576 learned_addr learned my addr 172.21.15.45:0/2484203576 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:09.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3e00640 1 --2- 172.21.15.45:0/2484203576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40d1c30 0x7f9be40cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:09.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3400640 1 -- 172.21.15.45:0/2484203576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40d1c30 msgr2=0x7f9be40cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:09.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3400640 1 --2- 172.21.15.45:0/2484203576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40d1c30 0x7f9be40cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:09.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3400640 1 -- 172.21.15.45:0/2484203576 >> v1:172.21.15.45:6789/0 conn(0x7f9be415bdc0 legacy=0x7f9be40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:09.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be2a00640 1 -- 172.21.15.45:0/2484203576 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174816534 0 0) 0x7f9be40ccfb0 con 0x7f9be415bdc0 2024-04-25T10:25:09.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3400640 1 -- 172.21.15.45:0/2484203576 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40cd850 con 0x7f9be415b9e0 2024-04-25T10:25:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be3400640 1 --2- 172.21.15.45:0/2484203576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be40cc410 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097c0 tx=0x7f9bd401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af37720705634bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be2a00640 1 -- 172.21.15.45:0/2484203576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40b0070 con 0x7f9be415b9e0 2024-04-25T10:25:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be2a00640 1 -- 172.21.15.45:0/2484203576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40ab600 con 0x7f9be415b9e0 2024-04-25T10:25:09.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.116+0000 7f9be2a00640 1 -- 172.21.15.45:0/2484203576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40ab9d0 con 0x7f9be415b9e0 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- 172.21.15.45:0/2484203576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 msgr2=0x7f9be40cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 --2- 172.21.15.45:0/2484203576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be40cc410 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097c0 tx=0x7f9bd401c410 comp rx=0 tx=0).stop 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- 172.21.15.45:0/2484203576 shutdown_connections 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 --2- 172.21.15.45:0/2484203576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40d1c30 0x7f9be40cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 --2- 172.21.15.45:0/2484203576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be40cc410 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- 172.21.15.45:0/2484203576 >> 172.21.15.45:0/2484203576 conn(0x7f9be409a9d0 msgr2=0x7f9be409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- 172.21.15.45:0/2484203576 shutdown_connections 2024-04-25T10:25:09.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- 172.21.15.45:0/2484203576 wait complete. 2024-04-25T10:25:09.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 Processor -- start 2024-04-25T10:25:09.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- start start 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40d1c30 0x7f9be40b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be415bdc0 0x7f9be40ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40c65f0 con 0x7f9be415b9e0 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40c6470 con 0x7f9be40d1c30 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bea200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40c6770 con 0x7f9be415bdc0 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be415a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41598/0 (socket says 172.21.15.45:41598) 2024-04-25T10:25:09.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 -- 172.21.15.45:0/4049038531 learned_addr learned my addr 172.21.15.45:0/4049038531 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9be3e00640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be415bdc0 0x7f9be40ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9be3400640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40d1c30 0x7f9be40b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 -- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be415bdc0 msgr2=0x7f9be40ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be415bdc0 0x7f9be40ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 -- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40d1c30 msgr2=0x7f9be40b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40d1c30 0x7f9be40b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 -- 172.21.15.45:0/4049038531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40c9e70 con 0x7f9be415b9e0 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9be3e00640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be415bdc0 0x7f9be40ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:09.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.120+0000 7f9bdaa00640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be415a580 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc007fd0 tx=0x7f9bcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc017070 con 0x7f9be415b9e0 2024-04-25T10:25:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc005d10 con 0x7f9be415b9e0 2024-04-25T10:25:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc007310 con 0x7f9be415b9e0 2024-04-25T10:25:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be40b4ee0 con 0x7f9be415b9e0 2024-04-25T10:25:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be40b54a0 con 0x7f9be415b9e0 2024-04-25T10:25:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be404ef30 con 0x7f9be415b9e0 2024-04-25T10:25:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bcc01a070 con 0x7f9be415b9e0 2024-04-25T10:25:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bdbe00640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc0066f60 0x7f9bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9bcc0840e0 con 0x7f9be415b9e0 2024-04-25T10:25:09.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.124+0000 7f9be3400640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc0066f60 0x7f9bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:09.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.128+0000 7f9be3400640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc0066f60 0x7f9bc0069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40ad000 tx=0x7f9bd40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:09.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.128+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bcc054d00 con 0x7f9be415b9e0 2024-04-25T10:25:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:09.348+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f9be40b2ac0 con 0x7f9be415b9e0 2024-04-25T10:25:10.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bdbe00640 1 -- 172.21.15.45:0/4049038531 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v349) v1 ==== 158+0+0 (secure 0 0 0) 0x7f9bcc058da0 con 0x7f9be415b9e0 2024-04-25T10:25:10.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc0066f60 msgr2=0x7f9bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc0066f60 0x7f9bc0069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40ad000 tx=0x7f9bd40a6000 comp rx=0 tx=0).stop 2024-04-25T10:25:10.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 msgr2=0x7f9be415a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be415a580 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc007fd0 tx=0x7f9bcc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 shutdown_connections 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9bc0066f60 0x7f9bc0069420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be415bdc0 0x7f9be40ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9be415b9e0 0x7f9be415a580 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 --2- 172.21.15.45:0/4049038531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40d1c30 0x7f9be40b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 >> 172.21.15.45:0/4049038531 conn(0x7f9be409a9d0 msgr2=0x7f9be406ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.264+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 shutdown_connections 2024-04-25T10:25:10.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.268+0000 7f9bea200640 1 -- 172.21.15.45:0/4049038531 wait complete. 2024-04-25T10:25:10.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:25:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 Processor -- start 2024-04-25T10:25:10.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 -- start start 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9c40cc200 0x7fb9c40ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c406cf40 0x7fb9c406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c40cdab0 con 0x7fb9c40d0d00 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c406d8f0 con 0x7fb9c40cc200 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c9a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40cd210 con 0x7fb9c406cf40 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9bbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb9c40cc200 0x7fb9c40ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb9c40d0d00 0x7fb9c4162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56288/0 (socket says 172.21.15.45:56288) 2024-04-25T10:25:10.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.440+0000 7fb9c2a00640 1 -- 172.21.15.45:0/2910752000 learned_addr learned my addr 172.21.15.45:0/2910752000 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c3400640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c406cf40 0x7fb9c406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9bbe00640 1 -- 172.21.15.45:0/2910752000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c406cf40 msgr2=0x7fb9c406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9bbe00640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c406cf40 0x7fb9c406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9bbe00640 1 -- 172.21.15.45:0/2910752000 >> v1:172.21.15.45:6789/0 conn(0x7fb9c40d0d00 legacy=0x7fb9c4162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9bbe00640 1 -- 172.21.15.45:0/2910752000 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c40cd4f0 con 0x7fb9c40cc200 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c3400640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c406cf40 0x7fb9c406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:10.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c2000640 1 -- 172.21.15.45:0/2910752000 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838392646 0 0) 0x7fb9c40cdab0 con 0x7fb9c40d0d00 2024-04-25T10:25:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9bbe00640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c40cc200 0x7fb9c40ccc30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097c0 tx=0x7fb9ac01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7062e6189856c81b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:10.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c2000640 1 -- 172.21.15.45:0/2910752000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0b0070 con 0x7fb9c40cc200 2024-04-25T10:25:10.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c2000640 1 -- 172.21.15.45:0/2910752000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac0ab600 con 0x7fb9c40cc200 2024-04-25T10:25:10.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c2000640 1 -- 172.21.15.45:0/2910752000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0ab9d0 con 0x7fb9c40cc200 2024-04-25T10:25:10.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c9a00640 1 -- 172.21.15.45:0/2910752000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c40cc200 msgr2=0x7fb9c40ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c40cc200 0x7fb9c40ccc30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0097c0 tx=0x7fb9ac01c410 comp rx=0 tx=0).stop 2024-04-25T10:25:10.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c9a00640 1 -- 172.21.15.45:0/2910752000 shutdown_connections 2024-04-25T10:25:10.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c406cf40 0x7fb9c406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/2910752000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c40cc200 0x7fb9c40ccc30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.444+0000 7fb9c9a00640 1 -- 172.21.15.45:0/2910752000 >> 172.21.15.45:0/2910752000 conn(0x7fb9c4072290 msgr2=0x7fb9c40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:10.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- 172.21.15.45:0/2910752000 shutdown_connections 2024-04-25T10:25:10.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- 172.21.15.45:0/2910752000 wait complete. 2024-04-25T10:25:10.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 Processor -- start 2024-04-25T10:25:10.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- start start 2024-04-25T10:25:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 0x7fb9c415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c40cc200 0x7fb9c415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9c40d0d00 0x7fb9c41579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4162a10 con 0x7fb9c406cf40 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4162890 con 0x7fb9c40d0d00 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c415e040 con 0x7fb9c40cc200 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 0x7fb9c415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 0x7fb9c415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41614/0 (socket says 172.21.15.45:41614) 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 -- 172.21.15.45:0/3747347758 learned_addr learned my addr 172.21.15.45:0/3747347758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9bbe00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c40cc200 0x7fb9c415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c3400640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9c40d0d00 0x7fb9c41579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 -- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c40cc200 msgr2=0x7fb9c415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c40cc200 0x7fb9c415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 -- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9c40d0d00 msgr2=0x7fb9c41579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9c40d0d00 0x7fb9c41579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 -- 172.21.15.45:0/3747347758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c40c9dd0 con 0x7fb9c406cf40 2024-04-25T10:25:10.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c3400640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9c40d0d00 0x7fb9c41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c2a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 0x7fb9c415ccd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c4b0 tx=0x7fb9b400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4018070 con 0x7fb9c406cf40 2024-04-25T10:25:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4007500 con 0x7fb9c406cf40 2024-04-25T10:25:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4004a30 con 0x7fb9c406cf40 2024-04-25T10:25:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c40ca090 con 0x7fb9c406cf40 2024-04-25T10:25:10.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.448+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c40ad310 con 0x7fb9c406cf40 2024-04-25T10:25:10.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.452+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c404ef30 con 0x7fb9c406cf40 2024-04-25T10:25:10.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.452+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9b401b070 con 0x7fb9c406cf40 2024-04-25T10:25:10.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.452+0000 7fb9baa00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb9a0066f90 0x7fb9a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:10.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.452+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb9b4083fc0 con 0x7fb9c406cf40 2024-04-25T10:25:10.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.452+0000 7fb9bbe00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb9a0066f90 0x7fb9a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:10.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.452+0000 7fb9bbe00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb9a0066f90 0x7fb9a0069450 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0ad000 tx=0x7fb9ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:10.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.456+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9b4054be0 con 0x7fb9c406cf40 2024-04-25T10:25:10.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.620+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb9c4072ff0 con 0x7fb9c406cf40 2024-04-25T10:25:10.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9baa00640 1 -- 172.21.15.45:0/3747347758 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb9b4058c80 con 0x7fb9c406cf40 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb9a0066f90 msgr2=0x7fb9a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb9a0066f90 0x7fb9a0069450 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0ad000 tx=0x7fb9ac008040 comp rx=0 tx=0).stop 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 msgr2=0x7fb9c415ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 0x7fb9c415ccd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400c4b0 tx=0x7fb9b400c970 comp rx=0 tx=0).stop 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 shutdown_connections 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb9a0066f90 0x7fb9a0069450 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9c40d0d00 0x7fb9c41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9c40cc200 0x7fb9c415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 --2- 172.21.15.45:0/3747347758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c406cf40 0x7fb9c415ccd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 >> 172.21.15.45:0/3747347758 conn(0x7fb9c4072290 msgr2=0x7fb9c40cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 shutdown_connections 2024-04-25T10:25:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:10.624+0000 7fb9c9a00640 1 -- 172.21.15.45:0/3747347758 wait complete. 2024-04-25T10:25:10.646 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:25:10.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:10.758 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:10.851 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:25:10.851 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:10.851 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:10.860 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:10.860 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:25:10.860 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:10.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:10.869 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:10.869 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:10.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:10.911 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:25:10.911 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:10.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:10.924 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:25:10.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.092+0000 7f9281000640 1 Processor -- start 2024-04-25T10:25:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.092+0000 7f9281000640 1 -- start start 2024-04-25T10:25:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9281000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c1550b0 0x7f927c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9281000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f927c0cfe70 0x7f927c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9281000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c071050 con 0x7f927c15b740 2024-04-25T10:25:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9281000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c15d040 con 0x7f927c0cfe70 2024-04-25T10:25:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9281000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0712f0 con 0x7f927c1550b0 2024-04-25T10:25:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f927c0cfe70 0x7f927c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f927c0cfe70 0x7f927c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39084/0 (socket says 172.21.15.45:39084) 2024-04-25T10:25:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9279e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c1550b0 0x7f927c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 -- 172.21.15.45:0/4288787004 learned_addr learned my addr 172.21.15.45:0/4288787004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 -- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c1550b0 msgr2=0x7f927c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 --2- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c1550b0 0x7f927c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 -- 172.21.15.45:0/4288787004 >> v1:172.21.15.117:6789/0 conn(0x7f927c15b740 legacy=0x7f927c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.096+0000 7f9273e00640 1 -- 172.21.15.45:0/4288787004 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c0715d0 con 0x7f927c0cfe70 2024-04-25T10:25:11.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9273e00640 1 --2- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c0cfe70 0x7f927c15b200 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f926400bb10 tx=0x7f926401ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1254608e79cdac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9279400640 1 -- 172.21.15.45:0/4288787004 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640b4070 con 0x7f927c0cfe70 2024-04-25T10:25:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9279400640 1 -- 172.21.15.45:0/4288787004 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640a7b10 con 0x7f927c0cfe70 2024-04-25T10:25:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9279400640 1 -- 172.21.15.45:0/4288787004 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640af440 con 0x7f927c0cfe70 2024-04-25T10:25:11.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 -- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c0cfe70 msgr2=0x7f927c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 --2- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c0cfe70 0x7f927c15b200 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f926400bb10 tx=0x7f926401ec30 comp rx=0 tx=0).stop 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 -- 172.21.15.45:0/4288787004 shutdown_connections 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 --2- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c0cfe70 0x7f927c15b200 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 --2- 172.21.15.45:0/4288787004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c1550b0 0x7f927c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 -- 172.21.15.45:0/4288787004 >> 172.21.15.45:0/4288787004 conn(0x7f927c066b70 msgr2=0x7f927c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 -- 172.21.15.45:0/4288787004 shutdown_connections 2024-04-25T10:25:11.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 -- 172.21.15.45:0/4288787004 wait complete. 2024-04-25T10:25:11.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.100+0000 7f9281000640 1 Processor -- start 2024-04-25T10:25:11.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- start start 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c0cfe70 0x7f927c136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 0x7f927c137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c15b740 0x7f927c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c15bcd0 con 0x7f927c1550b0 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c15bb50 con 0x7f927c15b740 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c15c030 con 0x7f927c0cfe70 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 0x7f927c137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 0x7f927c137530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41626/0 (socket says 172.21.15.45:41626) 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 -- 172.21.15.45:0/1394170922 learned_addr learned my addr 172.21.15.45:0/1394170922 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9279e00640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c0cfe70 0x7f927c136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f927a800640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c15b740 0x7f927c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 -- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c0cfe70 msgr2=0x7f927c136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c0cfe70 0x7f927c136c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 -- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c15b740 msgr2=0x7f927c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c15b740 0x7f927c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 -- 172.21.15.45:0/1394170922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c15ce70 con 0x7f927c1550b0 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9273e00640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 0x7f927c137530 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f926400b2c0 tx=0x7f926401d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640bf070 con 0x7f927c1550b0 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640afe40 con 0x7f927c1550b0 2024-04-25T10:25:11.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c0ce380 con 0x7f927c1550b0 2024-04-25T10:25:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640b4070 con 0x7f927c1550b0 2024-04-25T10:25:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c159e20 con 0x7f927c1550b0 2024-04-25T10:25:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.104+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c04ef30 con 0x7f927c1550b0 2024-04-25T10:25:11.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.108+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92640c2070 con 0x7f927c1550b0 2024-04-25T10:25:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.108+0000 7f9272000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9258066f60 0x7f9258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.108+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f926411f370 con 0x7f927c1550b0 2024-04-25T10:25:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.108+0000 7f9279e00640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9258066f60 0x7f9258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.108+0000 7f9279e00640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9258066f60 0x7f9258069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f926c009b60 tx=0x7f926c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:11.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.112+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9264018820 con 0x7f927c1550b0 2024-04-25T10:25:11.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.308+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f927c06b390 con 0x7f927c1550b0 2024-04-25T10:25:11.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.308+0000 7f9272000640 1 -- 172.21.15.45:0/1394170922 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f92640f0020 con 0x7f927c1550b0 2024-04-25T10:25:11.311 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:11.312 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1755169344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1755169344},{"type":"v1","addr":"172.21.15.117:6837","nonce":1755169344}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6946,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3818679388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3818679388},{"type":"v1","addr":"172.21.15.45:6835","nonce":3818679388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2326048296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2326048296},{"type":"v1","addr":"172.21.15.117:6836","nonce":2326048296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7767,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/456498889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":456498889},{"type":"v1","addr":"172.21.15.45:6834","nonce":456498889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[]} 2024-04-25T10:25:11.312 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 67 2024-04-25T10:25:11.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9258066f60 msgr2=0x7f9258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9258066f60 0x7f9258069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f926c009b60 tx=0x7f926c009190 comp rx=0 tx=0).stop 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 msgr2=0x7f927c137530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 0x7f927c137530 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f926400b2c0 tx=0x7f926401d470 comp rx=0 tx=0).stop 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 shutdown_connections 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9258066f60 0x7f9258069420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f927c15b740 0x7f927c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c1550b0 0x7f927c137530 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 --2- 172.21.15.45:0/1394170922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f927c0cfe70 0x7f927c136c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 >> 172.21.15.45:0/1394170922 conn(0x7f927c066b70 msgr2=0x7f927c0af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 shutdown_connections 2024-04-25T10:25:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.312+0000 7f9281000640 1 -- 172.21.15.45:0/1394170922 wait complete. 2024-04-25T10:25:11.333 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:25:11.333 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:25:11.333 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:25:11.333 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:11.334 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:25:11.334 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:25:11.334 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:25:11.334 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:11.334 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:25:11.334 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:25:11.335 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:25:11.335 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:11.335 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:25:11.335 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:25:11.335 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:25:11.335 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:25:11.346 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:25:11.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:25:11.347 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:25:11.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:25:11.350 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:25:11.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:25:11.351 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:25:11.351 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:25:11.394 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:25:11.394 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:11.395 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:25:11.395 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:11.395 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:25:11.396 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:11.396 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:25:11.396 DEBUG:teuthology.parallel:result is None 2024-04-25T10:25:11.397 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:25:11.481 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:25:11.482 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:25:11.485 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:25:11.485 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:25:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4671a00640 1 Processor -- start 2024-04-25T10:25:11.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4671a00640 1 -- start start 2024-04-25T10:25:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4671a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f466c0d1820 0x7f466c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4671a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0b1af0 con 0x7f466c15e050 2024-04-25T10:25:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4671a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0b1dd0 con 0x7f466c0b15b0 2024-04-25T10:25:11.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4671a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c0b20b0 con 0x7f466c0d1820 2024-04-25T10:25:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f466c0d1820 0x7f466c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f466c0d1820 0x7f466c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41656/0 (socket says 172.21.15.45:41656) 2024-04-25T10:25:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466aa00640 1 -- 172.21.15.45:0/1218595695 learned_addr learned my addr 172.21.15.45:0/1218595695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f4662000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f466c15e050 0x7f466c0b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56310/0 (socket says 172.21.15.45:56310) 2024-04-25T10:25:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466aa00640 1 -- 172.21.15.45:0/1218595695 >> v1:172.21.15.117:6789/0 conn(0x7f466c0b15b0 legacy=0x7f466c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466aa00640 1 -- 172.21.15.45:0/1218595695 >> v1:172.21.15.45:6789/0 conn(0x7f466c15e050 legacy=0x7f466c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466aa00640 1 -- 172.21.15.45:0/1218595695 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0b2390 con 0x7f466c0d1820 2024-04-25T10:25:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.544+0000 7f466a000640 1 -- 172.21.15.45:0/1218595695 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878967759 0 0) 0x7f466c0b1af0 con 0x7f466c15e050 2024-04-25T10:25:11.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f466aa00640 1 --2- 172.21.15.45:0/1218595695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0d1820 0x7f466c0d1c00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f465c009870 tx=0x7f465c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75662263337ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:11.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f466a000640 1 -- 172.21.15.45:0/1218595695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0a9640 con 0x7f466c0d1820 2024-04-25T10:25:11.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f466a000640 1 -- 172.21.15.45:0/1218595695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0a97e0 con 0x7f466c0d1820 2024-04-25T10:25:11.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f466a000640 1 -- 172.21.15.45:0/1218595695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0a9bb0 con 0x7f466c0d1820 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 -- 172.21.15.45:0/1218595695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0d1820 msgr2=0x7f466c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 --2- 172.21.15.45:0/1218595695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0d1820 0x7f466c0d1c00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f465c009870 tx=0x7f465c01c560 comp rx=0 tx=0).stop 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 -- 172.21.15.45:0/1218595695 shutdown_connections 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 --2- 172.21.15.45:0/1218595695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0d1820 0x7f466c0d1c00 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 -- 172.21.15.45:0/1218595695 >> 172.21.15.45:0/1218595695 conn(0x7f466c09a9d0 msgr2=0x7f466c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 -- 172.21.15.45:0/1218595695 shutdown_connections 2024-04-25T10:25:11.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.548+0000 7f4671a00640 1 -- 172.21.15.45:0/1218595695 wait complete. 2024-04-25T10:25:11.551 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:25:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 Processor -- start 2024-04-25T10:25:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- start start 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 0x7f466c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f466c0d1820 0x7f466c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f466c15e050 0x7f466c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0d0f60 con 0x7f466c0b15b0 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0d0de0 con 0x7f466c0d1820 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c16b9b0 con 0x7f466c15e050 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 0x7f466c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4662000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f466c0d1820 0x7f466c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 0x7f466c153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41672/0 (socket says 172.21.15.45:41672) 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 -- 172.21.15.45:0/1612564657 learned_addr learned my addr 172.21.15.45:0/1612564657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466b400640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f466c15e050 0x7f466c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 -- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f466c15e050 msgr2=0x7f466c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f466c15e050 0x7f466c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 -- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f466c0d1820 msgr2=0x7f466c0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f466c0d1820 0x7f466c0b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 -- 172.21.15.45:0/1612564657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c155220 con 0x7f466c0b15b0 2024-04-25T10:25:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f466aa00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 0x7f466c153ce0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f465c019170 tx=0x7f465c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b1880 con 0x7f466c0b15b0 2024-04-25T10:25:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0b1a20 con 0x7f466c0b15b0 2024-04-25T10:25:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b1e10 con 0x7f466c0b15b0 2024-04-25T10:25:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c155500 con 0x7f466c0b15b0 2024-04-25T10:25:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c0c5430 con 0x7f466c0b15b0 2024-04-25T10:25:11.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.552+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c04ef30 con 0x7f466c0b15b0 2024-04-25T10:25:11.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.556+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f465c0bc450 con 0x7f466c0b15b0 2024-04-25T10:25:11.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.556+0000 7f4663400640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4648066f60 0x7f4648069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:11.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.556+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f465c11f590 con 0x7f466c0b15b0 2024-04-25T10:25:11.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.556+0000 7f4662000640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4648066f60 0x7f4648069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:11.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.556+0000 7f4662000640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4648066f60 0x7f4648069420 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f466c0b4820 tx=0x7f4654007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:11.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.560+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f465c0c3030 con 0x7f466c0b15b0 2024-04-25T10:25:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:11.744+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f466c06b480 con 0x7f466c0b15b0 2024-04-25T10:25:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.648+0000 7f4663400640 1 -- 172.21.15.45:0/1612564657 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7f465c0ad020 con 0x7f466c0b15b0 2024-04-25T10:25:12.651 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4648066f60 msgr2=0x7f4648069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4648066f60 0x7f4648069420 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f466c0b4820 tx=0x7f4654007240 comp rx=0 tx=0).stop 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 msgr2=0x7f466c153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 0x7f466c153ce0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f465c019170 tx=0x7f465c01ada0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 shutdown_connections 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4648066f60 0x7f4648069420 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f466c15e050 0x7f466c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f466c0d1820 0x7f466c0b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 --2- 172.21.15.45:0/1612564657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f466c0b15b0 0x7f466c153ce0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 >> 172.21.15.45:0/1612564657 conn(0x7f466c09a9d0 msgr2=0x7f466c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 shutdown_connections 2024-04-25T10:25:12.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.652+0000 7f4671a00640 1 -- 172.21.15.45:0/1612564657 wait complete. 2024-04-25T10:25:12.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:25:12.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a9200640 1 Processor -- start 2024-04-25T10:25:12.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a9200640 1 -- start start 2024-04-25T10:25:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a9200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38a415bc40 0x7f38a406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a9200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a40b2770 con 0x7f38a40cda10 2024-04-25T10:25:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a9200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a406c670 con 0x7f38a40cf930 2024-04-25T10:25:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a9200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a406c950 con 0x7f38a415bc40 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38a415bc40 0x7f38a406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38a415bc40 0x7f38a406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41678/0 (socket says 172.21.15.45:41678) 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38a40cda10 0x7f38a40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56324/0 (socket says 172.21.15.45:56324) 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1e00640 1 -- 172.21.15.45:0/661749033 learned_addr learned my addr 172.21.15.45:0/661749033 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1e00640 1 -- 172.21.15.45:0/661749033 >> v1:172.21.15.117:6789/0 conn(0x7f38a40cf930 legacy=0x7f38a40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1e00640 1 -- 172.21.15.45:0/661749033 >> v1:172.21.15.45:6789/0 conn(0x7f38a40cda10 legacy=0x7f38a40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a1e00640 1 -- 172.21.15.45:0/661749033 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a406cc30 con 0x7f38a415bc40 2024-04-25T10:25:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.824+0000 7f38a0a00640 1 -- 172.21.15.45:0/661749033 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984772153 0 0) 0x7f38a40b2770 con 0x7f38a40cda10 2024-04-25T10:25:12.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a1e00640 1 --2- 172.21.15.45:0/661749033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a406c090 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3894009870 tx=0x7f389401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de545be26f6aeb84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a0a00640 1 -- 172.21.15.45:0/661749033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940b0070 con 0x7f38a415bc40 2024-04-25T10:25:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a0a00640 1 -- 172.21.15.45:0/661749033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38940ab600 con 0x7f38a415bc40 2024-04-25T10:25:12.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a0a00640 1 -- 172.21.15.45:0/661749033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940ab9d0 con 0x7f38a415bc40 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 -- 172.21.15.45:0/661749033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 msgr2=0x7f38a406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 --2- 172.21.15.45:0/661749033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a406c090 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3894009870 tx=0x7f389401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 -- 172.21.15.45:0/661749033 shutdown_connections 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 --2- 172.21.15.45:0/661749033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a406c090 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 -- 172.21.15.45:0/661749033 >> 172.21.15.45:0/661749033 conn(0x7f38a4066b70 msgr2=0x7f38a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 -- 172.21.15.45:0/661749033 shutdown_connections 2024-04-25T10:25:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 -- 172.21.15.45:0/661749033 wait complete. 2024-04-25T10:25:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 Processor -- start 2024-04-25T10:25:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.828+0000 7f38a9200640 1 -- start start 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a40cda10 0x7f38a4148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38a40cf930 0x7f38a40b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a414cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4169d70 con 0x7f38a415bc40 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4169bf0 con 0x7f38a40cda10 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4169ef0 con 0x7f38a40cf930 2024-04-25T10:25:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a414cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a414cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41690/0 (socket says 172.21.15.45:41690) 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 -- 172.21.15.45:0/3776531984 learned_addr learned my addr 172.21.15.45:0/3776531984 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a1e00640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a40cda10 0x7f38a4148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a1400640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38a40cf930 0x7f38a40b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 -- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38a40cf930 msgr2=0x7f38a40b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38a40cf930 0x7f38a40b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 -- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a40cda10 msgr2=0x7f38a4148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a40cda10 0x7f38a4148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 -- 172.21.15.45:0/3776531984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a41f74d0 con 0x7f38a415bc40 2024-04-25T10:25:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a2800640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a414cb00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c770 tx=0x7f389c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c003b80 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c003d20 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a41f77b0 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00aa90 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a41f7d70 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.832+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a404ef30 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.836+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f389c00ac30 con 0x7f38a415bc40 2024-04-25T10:25:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.836+0000 7f389a000640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3884066f60 0x7f3884069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.836+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f389c084ae0 con 0x7f38a415bc40 2024-04-25T10:25:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.836+0000 7f38a1e00640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3884066f60 0x7f3884069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.836+0000 7f38a1e00640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3884066f60 0x7f3884069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f38940ad000 tx=0x7f38940a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:12.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:12.840+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f389c0557e0 con 0x7f38a415bc40 2024-04-25T10:25:13.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f38a40af1c0 con 0x7f38a415bc40 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f389a000640 1 -- 172.21.15.45:0/3776531984 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f389c059880 con 0x7f38a415bc40 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:25:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:25:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:25:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:25:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:25:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:25:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:25:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:25:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:25:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:13.077 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:25:13.077 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3884066f60 msgr2=0x7f3884069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3884066f60 0x7f3884069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f38940ad000 tx=0x7f38940a6000 comp rx=0 tx=0).stop 2024-04-25T10:25:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 msgr2=0x7f38a414cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a414cb00 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c770 tx=0x7f389c00cc30 comp rx=0 tx=0).stop 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 shutdown_connections 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3884066f60 0x7f3884069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a415bc40 0x7f38a414cb00 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38a40cf930 0x7f38a40b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 --2- 172.21.15.45:0/3776531984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38a40cda10 0x7f38a4148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 >> 172.21.15.45:0/3776531984 conn(0x7f38a4066b70 msgr2=0x7f38a40af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 shutdown_connections 2024-04-25T10:25:13.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.056+0000 7f38a9200640 1 -- 172.21.15.45:0/3776531984 wait complete. 2024-04-25T10:25:13.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe7800640 1 Processor -- start 2024-04-25T10:25:13.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe7800640 1 -- start start 2024-04-25T10:25:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe7800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe00ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe7800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00d15e0 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe7800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe00d0140 con 0x7fbfe015f570 2024-04-25T10:25:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe7800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00cfef0 con 0x7fbfe015f190 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe5800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbfe00cd5e0 0x7fbfe00c5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56330/0 (socket says 172.21.15.45:56330) 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.220+0000 7fbfe5800640 1 -- 172.21.15.45:0/3071338695 learned_addr learned my addr 172.21.15.45:0/3071338695 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfe4e00640 1 --2- 172.21.15.45:0/3071338695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe00ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303152213 0 0) 0x7fbfd80014a0 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc4003530 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344146778 0 0) 0x7fbfe00d0140 con 0x7fbfe015f570 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd80014a0 con 0x7fbfe015f570 2024-04-25T10:25:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfe4e00640 1 -- 172.21.15.45:0/3071338695 >> v1:172.21.15.117:6789/0 conn(0x7fbfe015f570 legacy=0x7fbfe00c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfe4e00640 1 -- 172.21.15.45:0/3071338695 >> v1:172.21.15.45:6789/0 conn(0x7fbfe00cd5e0 legacy=0x7fbfe00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfe4e00640 1 -- 172.21.15.45:0/3071338695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00d0420 con 0x7fbfe015f190 2024-04-25T10:25:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfe4e00640 1 --2- 172.21.15.45:0/3071338695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe00ce290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00097f0 tx=0x7fbfd001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf633069b4a240b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00b0070 con 0x7fbfe015f190 2024-04-25T10:25:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd00a4e50 con 0x7fbfe015f190 2024-04-25T10:25:13.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.224+0000 7fbfdfe00640 1 -- 172.21.15.45:0/3071338695 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00ab800 con 0x7fbfe015f190 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- 172.21.15.45:0/3071338695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 msgr2=0x7fbfe00ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 --2- 172.21.15.45:0/3071338695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe00ce290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00097f0 tx=0x7fbfd001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- 172.21.15.45:0/3071338695 shutdown_connections 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 --2- 172.21.15.45:0/3071338695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe00ce290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- 172.21.15.45:0/3071338695 >> 172.21.15.45:0/3071338695 conn(0x7fbfe00d1910 msgr2=0x7fbfe00d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- 172.21.15.45:0/3071338695 shutdown_connections 2024-04-25T10:25:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- 172.21.15.45:0/3071338695 wait complete. 2024-04-25T10:25:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 Processor -- start 2024-04-25T10:25:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- start start 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 0x7fbfe0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe015f570 0x7fbfe00b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00d25b0 con 0x7fbfe015f570 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00d2430 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe7800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe016a270 con 0x7fbfe015f190 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 0x7fbfe0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe015f570 0x7fbfe00b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 0x7fbfe0155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39136/0 (socket says 172.21.15.45:39136) 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 -- 172.21.15.45:0/1346629825 learned_addr learned my addr 172.21.15.45:0/1346629825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfd7e00640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 -- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 msgr2=0x7fbfe006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe006bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 -- 172.21.15.45:0/1346629825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe015f570 msgr2=0x7fbfe00b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe015f570 0x7fbfe00b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.228+0000 7fbfe4e00640 1 -- 172.21.15.45:0/1346629825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe01523a0 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfe4e00640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 0x7fbfe0155a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00ae040 tx=0x7fbfd0019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00b9020 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0003820 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00b0040 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe0152680 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe0152c40 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe004ef30 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfd00bc020 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfdd600640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfbc066f60 0x7fbfbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.232+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbfd011eb40 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.236+0000 7fbfd7e00640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfbc066f60 0x7fbfbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.236+0000 7fbfd7e00640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfbc066f60 0x7fbfbc069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80017c0 tx=0x7fbfc8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.236+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfd00c3030 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.400+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfe00d13a0 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfdd600640 1 -- 172.21.15.45:0/1346629825 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+12881 (secure 0 0 0) 0x7fbfd00160d0 con 0x7fbfe00cd5e0 2024-04-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:12.650481+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfbc066f60 msgr2=0x7fbfbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfbc066f60 0x7fbfbc069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80017c0 tx=0x7fbfc8009040 comp rx=0 tx=0).stop 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 msgr2=0x7fbfe0155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 0x7fbfe0155a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00ae040 tx=0x7fbfd0019160 comp rx=0 tx=0).stop 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 shutdown_connections 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfbc066f60 0x7fbfbc069420 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe015f570 0x7fbfe00b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe015f190 0x7fbfe006bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 --2- 172.21.15.45:0/1346629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe00cd5e0 0x7fbfe0155a60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 >> 172.21.15.45:0/1346629825 conn(0x7fbfe00d1910 msgr2=0x7fbfe0156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:13.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 shutdown_connections 2024-04-25T10:25:13.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.404+0000 7fbfe7800640 1 -- 172.21.15.45:0/1346629825 wait complete. 2024-04-25T10:25:13.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:13.426 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 Processor -- start 2024-04-25T10:25:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 -- start start 2024-04-25T10:25:13.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5abc15df20 0x7f5abc0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0b1970 0x7f5abc0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5abc0aeb50 0x7f5abc0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0b1e10 con 0x7f5abc0aeb50 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0b20f0 con 0x7f5abc15df20 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ac2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0b23d0 con 0x7f5abc0b1970 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5abc0aeb50 0x7f5abc0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5abc0aeb50 0x7f5abc0aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41706/0 (socket says 172.21.15.45:41706) 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 -- 172.21.15.45:0/598291293 learned_addr learned my addr 172.21.15.45:0/598291293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5ab3e00640 1 --2- 172.21.15.45:0/598291293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0b1970 0x7f5abc0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abb400640 1 --2- 172.21.15.45:0/598291293 >> v2:172.21.15.117:3300/0 conn(0x7f5abc15df20 0x7f5abc0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 -- 172.21.15.45:0/598291293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0b1970 msgr2=0x7f5abc0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 --2- 172.21.15.45:0/598291293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0b1970 0x7f5abc0ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 -- 172.21.15.45:0/598291293 >> v2:172.21.15.117:3300/0 conn(0x7f5abc15df20 msgr2=0x7f5abc0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 --2- 172.21.15.45:0/598291293 >> v2:172.21.15.117:3300/0 conn(0x7f5abc15df20 0x7f5abc0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.568+0000 7f5abbe00640 1 -- 172.21.15.45:0/598291293 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0b26b0 con 0x7f5abc0aeb50 2024-04-25T10:25:13.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5abbe00640 1 --2- 172.21.15.45:0/598291293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc0aeb50 0x7f5abc0aefc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0cf0a0 tx=0x7f5ab401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4484a42aae5a80aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5abaa00640 1 -- 172.21.15.45:0/598291293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40b4070 con 0x7f5abc0aeb50 2024-04-25T10:25:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5abaa00640 1 -- 172.21.15.45:0/598291293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab40af400 con 0x7f5abc0aeb50 2024-04-25T10:25:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5abaa00640 1 -- 172.21.15.45:0/598291293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40af7d0 con 0x7f5abc0aeb50 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 -- 172.21.15.45:0/598291293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc0aeb50 msgr2=0x7f5abc0aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 --2- 172.21.15.45:0/598291293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc0aeb50 0x7f5abc0aefc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5abc0cf0a0 tx=0x7f5ab401f1e0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 -- 172.21.15.45:0/598291293 shutdown_connections 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 --2- 172.21.15.45:0/598291293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc0aeb50 0x7f5abc0aefc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 --2- 172.21.15.45:0/598291293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0b1970 0x7f5abc0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 --2- 172.21.15.45:0/598291293 >> v2:172.21.15.117:3300/0 conn(0x7f5abc15df20 0x7f5abc0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 -- 172.21.15.45:0/598291293 >> 172.21.15.45:0/598291293 conn(0x7f5abc0d1960 msgr2=0x7f5abc0d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 -- 172.21.15.45:0/598291293 shutdown_connections 2024-04-25T10:25:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.572+0000 7f5ac2600640 1 -- 172.21.15.45:0/598291293 wait complete. 2024-04-25T10:25:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 Processor -- start 2024-04-25T10:25:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 -- start start 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0aeb50 0x7f5abc159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5abc0b1970 0x7f5abc159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 0x7f5abc0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0cfb60 con 0x7f5abc15df20 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc15e300 con 0x7f5abc0b1970 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0cfce0 con 0x7f5abc0aeb50 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 0x7f5abc0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 0x7f5abc0b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41714/0 (socket says 172.21.15.45:41714) 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 -- 172.21.15.45:0/887555275 learned_addr learned my addr 172.21.15.45:0/887555275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ab3e00640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5abc0b1970 0x7f5abc159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abb400640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0aeb50 0x7f5abc159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 -- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0aeb50 msgr2=0x7f5abc159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0aeb50 0x7f5abc159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 -- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5abc0b1970 msgr2=0x7f5abc159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5abc0b1970 0x7f5abc159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 -- 172.21.15.45:0/887555275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0b6d00 con 0x7f5abc15df20 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abb400640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0aeb50 0x7f5abc159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5abbe00640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 0x7f5abc0b5e30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40b2040 tx=0x7f5ab401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40b4040 con 0x7f5abc15df20 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab40c0070 con 0x7f5abc15df20 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4009510 con 0x7f5abc15df20 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc0b6fe0 con 0x7f5abc15df20 2024-04-25T10:25:13.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.576+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc0b72c0 con 0x7f5abc15df20 2024-04-25T10:25:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.580+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc04ef30 con 0x7f5abc15df20 2024-04-25T10:25:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.580+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ab40c5460 con 0x7f5abc15df20 2024-04-25T10:25:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.580+0000 7f5ab3400640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.580+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ab4123070 con 0x7f5abc15df20 2024-04-25T10:25:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.580+0000 7f5abb400640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.580+0000 7f5abb400640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a98066f60 0x7f5a98069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009720 tx=0x7f5aac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.584+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ab4018d40 con 0x7f5abc15df20 2024-04-25T10:25:13.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.748+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5abc06fcb0 con 0x7f5abc15df20 2024-04-25T10:25:13.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.748+0000 7f5ab3400640 1 -- 172.21.15.45:0/887555275 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ab40f4480 con 0x7f5abc15df20 2024-04-25T10:25:13.750 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a98066f60 msgr2=0x7f5a98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a98066f60 0x7f5a98069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009720 tx=0x7f5aac008040 comp rx=0 tx=0).stop 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 msgr2=0x7f5abc0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 0x7f5abc0b5e30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40b2040 tx=0x7f5ab401bf80 comp rx=0 tx=0).stop 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 shutdown_connections 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5abc15df20 0x7f5abc0b5e30 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5abc0b1970 0x7f5abc159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 --2- 172.21.15.45:0/887555275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5abc0aeb50 0x7f5abc159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 >> 172.21.15.45:0/887555275 conn(0x7f5abc0d1960 msgr2=0x7f5abc0b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 shutdown_connections 2024-04-25T10:25:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.752+0000 7f5ac2600640 1 -- 172.21.15.45:0/887555275 wait complete. 2024-04-25T10:25:13.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:25:13.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 Processor -- start 2024-04-25T10:25:13.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 -- start start 2024-04-25T10:25:13.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 0x7ff1f00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 0x7ff1f00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f00ccfb0 con 0x7ff1f00d1520 2024-04-25T10:25:13.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f00cd290 con 0x7ff1f015bdc0 2024-04-25T10:25:13.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1f6400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00cd570 con 0x7ff1f015b9e0 2024-04-25T10:25:13.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 0x7ff1f00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1e6a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 0x7ff1f00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 0x7ff1f00cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:42318/0 (socket says 172.21.15.45:42318) 2024-04-25T10:25:13.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1e6a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 0x7ff1f00cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39160/0 (socket says 172.21.15.45:39160) 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1efe00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff1f00d1520 0x7ff1f00d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:48840/0 (socket says 172.21.15.45:48840) 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 -- 172.21.15.45:0/2685678252 learned_addr learned my addr 172.21.15.45:0/2685678252 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 -- 172.21.15.45:0/2685678252 >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 msgr2=0x7ff1f00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 --2- 172.21.15.45:0/2685678252 >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 0x7ff1f00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 -- 172.21.15.45:0/2685678252 >> v1:172.21.15.117:6789/0 conn(0x7ff1f00d1520 legacy=0x7ff1f00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1ef400640 1 -- 172.21.15.45:0/2685678252 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00cd850 con 0x7ff1f015b9e0 2024-04-25T10:25:13.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.912+0000 7ff1e6a00640 1 --2- 172.21.15.45:0/2685678252 >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 0x7ff1f00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:13.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1ef400640 1 --2- 172.21.15.45:0/2685678252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 0x7ff1f00cc410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eeb78c94ee264e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1eea00640 1 -- 172.21.15.45:0/2685678252 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00b2070 con 0x7ff1f015b9e0 2024-04-25T10:25:13.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1eea00640 1 -- 172.21.15.45:0/2685678252 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e00a9070 con 0x7ff1f015b9e0 2024-04-25T10:25:13.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1eea00640 1 -- 172.21.15.45:0/2685678252 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00ad880 con 0x7ff1f015b9e0 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 -- 172.21.15.45:0/2685678252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 msgr2=0x7ff1f00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2685678252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 0x7ff1f00cc410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e001c560 comp rx=0 tx=0).stop 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 -- 172.21.15.45:0/2685678252 shutdown_connections 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2685678252 >> v2:172.21.15.117:3300/0 conn(0x7ff1f015bdc0 0x7ff1f00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2685678252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015b9e0 0x7ff1f00cc410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 -- 172.21.15.45:0/2685678252 >> 172.21.15.45:0/2685678252 conn(0x7ff1f009a9d0 msgr2=0x7ff1f009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 -- 172.21.15.45:0/2685678252 shutdown_connections 2024-04-25T10:25:13.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.916+0000 7ff1f6400640 1 -- 172.21.15.45:0/2685678252 wait complete. 2024-04-25T10:25:13.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 Processor -- start 2024-04-25T10:25:13.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 -- start start 2024-04-25T10:25:13.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 0x7ff1f01506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f015b9e0 0x7ff1f015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015bdc0 0x7ff1f015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c65f0 con 0x7ff1f00d1520 2024-04-25T10:25:13.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c6470 con 0x7ff1f015b9e0 2024-04-25T10:25:13.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c6770 con 0x7ff1f015bdc0 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 0x7ff1f01506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 0x7ff1f01506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41730/0 (socket says 172.21.15.45:41730) 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 -- 172.21.15.45:0/2917572188 learned_addr learned my addr 172.21.15.45:0/2917572188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1e6a00640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f015b9e0 0x7ff1f015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1efe00640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015bdc0 0x7ff1f015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 -- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015bdc0 msgr2=0x7ff1f015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015bdc0 0x7ff1f015abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 -- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f015b9e0 msgr2=0x7ff1f015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f015b9e0 0x7ff1f015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 -- 172.21.15.45:0/2917572188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00cb580 con 0x7ff1f00d1520 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1ef400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 0x7ff1f01506e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00b0040 tx=0x7ff1e0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00bb070 con 0x7ff1f00d1520 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e00adce0 con 0x7ff1f00d1520 2024-04-25T10:25:13.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00b2040 con 0x7ff1f00d1520 2024-04-25T10:25:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f00cb860 con 0x7ff1f00d1520 2024-04-25T10:25:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.920+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f0154920 con 0x7ff1f00d1520 2024-04-25T10:25:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.924+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f004ef30 con 0x7ff1f00d1520 2024-04-25T10:25:13.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.924+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1e00ab070 con 0x7ff1f00d1520 2024-04-25T10:25:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.924+0000 7ff1e7e00640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1cc066f60 0x7ff1cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.924+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff1e011da10 con 0x7ff1f00d1520 2024-04-25T10:25:13.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.924+0000 7ff1e6a00640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1cc066f60 0x7ff1cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:13.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.924+0000 7ff1e6a00640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1cc066f60 0x7ff1cc069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009580 tx=0x7ff1d8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:13.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:13.928+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1e0016150 con 0x7ff1f00d1520 2024-04-25T10:25:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff1f00cec80 con 0x7ff1f00d1520 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1e7e00640 1 -- 172.21.15.45:0/2917572188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff1e00ee680 con 0x7ff1f00d1520 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:14.094 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:25:14.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1cc066f60 msgr2=0x7ff1cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:14.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1cc066f60 0x7ff1cc069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009580 tx=0x7ff1d8009190 comp rx=0 tx=0).stop 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 msgr2=0x7ff1f01506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 0x7ff1f01506e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00b0040 tx=0x7ff1e0019300 comp rx=0 tx=0).stop 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 shutdown_connections 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1cc066f60 0x7ff1cc069420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1f015bdc0 0x7ff1f015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1f015b9e0 0x7ff1f015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 --2- 172.21.15.45:0/2917572188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f00d1520 0x7ff1f01506e0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.092+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 >> 172.21.15.45:0/2917572188 conn(0x7ff1f009a9d0 msgr2=0x7ff1f006e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.096+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 shutdown_connections 2024-04-25T10:25:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.096+0000 7ff1f6400640 1 -- 172.21.15.45:0/2917572188 wait complete. 2024-04-25T10:25:14.114 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:25:14.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:25:14.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 Processor -- start 2024-04-25T10:25:14.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 -- start start 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18480cf040 0x7f18480cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f184815bc00 0x7f18480cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1848071f80 con 0x7f18480713a0 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1848072260 con 0x7f18480cf040 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f184e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18480b3700 con 0x7f184815bc00 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f18480713a0 0x7f18480ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56332/0 (socket says 172.21.15.45:56332) 2024-04-25T10:25:14.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847e00640 1 -- 172.21.15.45:0/2192618610 learned_addr learned my addr 172.21.15.45:0/2192618610 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847400640 1 --2- 172.21.15.45:0/2192618610 >> v2:172.21.15.117:3300/0 conn(0x7f18480cf040 0x7f18480cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f183ea00640 1 --2- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f184815bc00 0x7f18480cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:14.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1846a00640 1 -- 172.21.15.45:0/2192618610 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644760891 0 0) 0x7f1848071f80 con 0x7f18480713a0 2024-04-25T10:25:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1846a00640 1 -- 172.21.15.45:0/2192618610 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f182c003530 con 0x7f18480713a0 2024-04-25T10:25:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847400640 1 -- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f184815bc00 msgr2=0x7f18480cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847400640 1 --2- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f184815bc00 0x7f18480cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847400640 1 -- 172.21.15.45:0/2192618610 >> v1:172.21.15.45:6789/0 conn(0x7f18480713a0 legacy=0x7f18480ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:14.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.260+0000 7f1847400640 1 -- 172.21.15.45:0/2192618610 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184815b120 con 0x7f18480cf040 2024-04-25T10:25:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f1847400640 1 --2- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480cf040 0x7f18480cf9e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1838009d90 tx=0x7f183801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffed6e3ffa107727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f1846a00640 1 -- 172.21.15.45:0/2192618610 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380b2070 con 0x7f18480cf040 2024-04-25T10:25:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f1846a00640 1 -- 172.21.15.45:0/2192618610 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18380a5920 con 0x7f18480cf040 2024-04-25T10:25:14.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f1846a00640 1 -- 172.21.15.45:0/2192618610 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380a5cf0 con 0x7f18480cf040 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 -- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480cf040 msgr2=0x7f18480cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 --2- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480cf040 0x7f18480cf9e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1838009d90 tx=0x7f183801c970 comp rx=0 tx=0).stop 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 -- 172.21.15.45:0/2192618610 shutdown_connections 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 --2- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f184815bc00 0x7f18480cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 --2- 172.21.15.45:0/2192618610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480cf040 0x7f18480cf9e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 -- 172.21.15.45:0/2192618610 >> 172.21.15.45:0/2192618610 conn(0x7f1848066b70 msgr2=0x7f1848066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 -- 172.21.15.45:0/2192618610 shutdown_connections 2024-04-25T10:25:14.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.264+0000 7f184e000640 1 -- 172.21.15.45:0/2192618610 wait complete. 2024-04-25T10:25:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 Processor -- start 2024-04-25T10:25:14.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 -- start start 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480713a0 0x7f184813e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18480cf040 0x7f18480b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 0x7f184813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18480cf5a0 con 0x7f184815bc00 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18480cf420 con 0x7f18480713a0 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184815e500 con 0x7f18480cf040 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 0x7f184813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 0x7f184813ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41746/0 (socket says 172.21.15.45:41746) 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 -- 172.21.15.45:0/1607746403 learned_addr learned my addr 172.21.15.45:0/1607746403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:14.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f183ea00640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18480cf040 0x7f18480b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847400640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480713a0 0x7f184813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 -- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18480cf040 msgr2=0x7f18480b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18480cf040 0x7f18480b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 -- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480713a0 msgr2=0x7f184813e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480713a0 0x7f184813e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 -- 172.21.15.45:0/1607746403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18480af810 con 0x7f184815bc00 2024-04-25T10:25:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f1847e00640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 0x7f184813ed30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f184000c400 tx=0x7f184000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:14.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840017070 con 0x7f184815bc00 2024-04-25T10:25:14.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840005cf0 con 0x7f184815bc00 2024-04-25T10:25:14.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18400072d0 con 0x7f184815bc00 2024-04-25T10:25:14.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18480afaf0 con 0x7f184815bc00 2024-04-25T10:25:14.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.268+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18480b00b0 con 0x7f184815bc00 2024-04-25T10:25:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.272+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184804ef30 con 0x7f184815bc00 2024-04-25T10:25:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.272+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1840007470 con 0x7f184815bc00 2024-04-25T10:25:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.272+0000 7f183fe00640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1824066f60 0x7f1824069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.272+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1840083fb0 con 0x7f184815bc00 2024-04-25T10:25:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.272+0000 7f1847400640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1824066f60 0x7f1824069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.272+0000 7f1847400640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1824066f60 0x7f1824069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f18380af000 tx=0x7f18380a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.276+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1840054ce0 con 0x7f184815bc00 2024-04-25T10:25:14.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:14.440+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f18480b40a0 con 0x7f184815bc00 2024-04-25T10:25:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.288+0000 7f183fe00640 1 -- 172.21.15.45:0/1607746403 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1840058d80 con 0x7f184815bc00 2024-04-25T10:25:15.290 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:25:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1824066f60 msgr2=0x7f1824069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1824066f60 0x7f1824069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f18380af000 tx=0x7f18380a7070 comp rx=0 tx=0).stop 2024-04-25T10:25:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 msgr2=0x7f184813ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:15.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 0x7f184813ed30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f184000c400 tx=0x7f184000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 shutdown_connections 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1824066f60 0x7f1824069420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f184815bc00 0x7f184813ed30 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18480cf040 0x7f18480b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 --2- 172.21.15.45:0/1607746403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18480713a0 0x7f184813e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 >> 172.21.15.45:0/1607746403 conn(0x7f1848066b70 msgr2=0x7f184806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 shutdown_connections 2024-04-25T10:25:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.292+0000 7f184e000640 1 -- 172.21.15.45:0/1607746403 wait complete. 2024-04-25T10:25:15.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:25:15.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 Processor -- start 2024-04-25T10:25:15.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 -- start start 2024-04-25T10:25:15.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6d4161300 0x7fd6d40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6d40d0c60 0x7fd6d406b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d406b8b0 0x7fd6d4161a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d40b10b0 con 0x7fd6d40d0c60 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d40b1390 con 0x7fd6d4161300 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d8a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40b1670 con 0x7fd6d406b8b0 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6d40d0c60 0x7fd6d406b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6d40d0c60 0x7fd6d406b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51894/0 (socket says 172.21.15.45:51894) 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 -- 172.21.15.45:0/2093979613 learned_addr learned my addr 172.21.15.45:0/2093979613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d1e00640 1 --2- 172.21.15.45:0/2093979613 >> v2:172.21.15.117:3300/0 conn(0x7fd6d4161300 0x7fd6d40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6d2800640 1 --2- 172.21.15.45:0/2093979613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d406b8b0 0x7fd6d4161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 -- 172.21.15.45:0/2093979613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d406b8b0 msgr2=0x7fd6d4161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/2093979613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d406b8b0 0x7fd6d4161a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 -- 172.21.15.45:0/2093979613 >> v2:172.21.15.117:3300/0 conn(0x7fd6d4161300 msgr2=0x7fd6d40d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:15.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/2093979613 >> v2:172.21.15.117:3300/0 conn(0x7fd6d4161300 0x7fd6d40d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.472+0000 7fd6cbe00640 1 -- 172.21.15.45:0/2093979613 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40b1950 con 0x7fd6d40d0c60 2024-04-25T10:25:15.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/2093979613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d40d0c60 0x7fd6d406b370 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4071ce0 tx=0x7fd6bc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68903d0108ead8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:15.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d1400640 1 -- 172.21.15.45:0/2093979613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0b0070 con 0x7fd6d40d0c60 2024-04-25T10:25:15.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d1400640 1 -- 172.21.15.45:0/2093979613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0ab600 con 0x7fd6d40d0c60 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d1400640 1 -- 172.21.15.45:0/2093979613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0ab9d0 con 0x7fd6d40d0c60 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 -- 172.21.15.45:0/2093979613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d40d0c60 msgr2=0x7fd6d406b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/2093979613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d40d0c60 0x7fd6d406b370 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4071ce0 tx=0x7fd6bc01c260 comp rx=0 tx=0).stop 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 -- 172.21.15.45:0/2093979613 shutdown_connections 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/2093979613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d406b8b0 0x7fd6d4161a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/2093979613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d40d0c60 0x7fd6d406b370 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/2093979613 >> v2:172.21.15.117:3300/0 conn(0x7fd6d4161300 0x7fd6d40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 -- 172.21.15.45:0/2093979613 >> 172.21.15.45:0/2093979613 conn(0x7fd6d40d19d0 msgr2=0x7fd6d40d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 -- 172.21.15.45:0/2093979613 shutdown_connections 2024-04-25T10:25:15.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.476+0000 7fd6d8a00640 1 -- 172.21.15.45:0/2093979613 wait complete. 2024-04-25T10:25:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 Processor -- start 2024-04-25T10:25:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 -- start start 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 0x7fd6d4159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d40d0c60 0x7fd6d4159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4161300 0x7fd6d40b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40b2240 con 0x7fd6d406b8b0 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40b20c0 con 0x7fd6d4161300 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40b23c0 con 0x7fd6d40d0c60 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 0x7fd6d4159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 0x7fd6d4159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51904/0 (socket says 172.21.15.45:51904) 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 -- 172.21.15.45:0/1373284324 learned_addr learned my addr 172.21.15.45:0/1373284324 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d40d0c60 0x7fd6d4159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d2800640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4161300 0x7fd6d40b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 -- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d40d0c60 msgr2=0x7fd6d4159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d40d0c60 0x7fd6d4159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 -- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4161300 msgr2=0x7fd6d40b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4161300 0x7fd6d40b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 -- 172.21.15.45:0/1373284324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d415a5d0 con 0x7fd6d406b8b0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d40d0c60 0x7fd6d4159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d1e00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 0x7fd6d4159520 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400d620 tx=0x7fd6c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400b620 con 0x7fd6d406b8b0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4019070 con 0x7fd6d406b8b0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400ba00 con 0x7fd6d406b8b0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d415a8b0 con 0x7fd6d406b8b0 2024-04-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.480+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d415ae70 con 0x7fd6d406b8b0 2024-04-25T10:25:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.484+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d404ef30 con 0x7fd6d406b8b0 2024-04-25T10:25:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.484+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6c400bba0 con 0x7fd6d406b8b0 2024-04-25T10:25:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.484+0000 7fd6ca000640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6b0066f90 0x7fd6b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.484+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd6c4085220 con 0x7fd6d406b8b0 2024-04-25T10:25:15.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.484+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6b0066f90 0x7fd6b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:15.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.484+0000 7fd6cbe00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6b0066f90 0x7fd6b0069450 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0ad000 tx=0x7fd6bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:15.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.488+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6c4056400 con 0x7fd6d406b8b0 2024-04-25T10:25:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:15.652+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd6d40d1510 con 0x7fd6d406b8b0 2024-04-25T10:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.292+0000 7fd6ca000640 1 -- 172.21.15.45:0/1373284324 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd6c4055d00 con 0x7fd6d406b8b0 2024-04-25T10:25:16.293 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6b0066f90 msgr2=0x7fd6b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6b0066f90 0x7fd6b0069450 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0ad000 tx=0x7fd6bc008040 comp rx=0 tx=0).stop 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 msgr2=0x7fd6d4159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 0x7fd6d4159520 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400d620 tx=0x7fd6c400dae0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 shutdown_connections 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6b0066f90 0x7fd6b0069450 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6d4161300 0x7fd6d40b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6d40d0c60 0x7fd6d4159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 --2- 172.21.15.45:0/1373284324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6d406b8b0 0x7fd6d4159520 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 >> 172.21.15.45:0/1373284324 conn(0x7fd6d40d19d0 msgr2=0x7fd6d406f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 shutdown_connections 2024-04-25T10:25:16.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.296+0000 7fd6d8a00640 1 -- 172.21.15.45:0/1373284324 wait complete. 2024-04-25T10:25:16.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:25:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.484+0000 7fdd07a00640 1 Processor -- start 2024-04-25T10:25:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.484+0000 7fdd07a00640 1 -- start start 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd07a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd0015fce0 0x7fdd000b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd07a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd00160650 con 0x7fdd000b2980 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd07a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd00160930 con 0x7fdd0015f550 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd07a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000cd0e0 con 0x7fdd0015fce0 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdcffe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd000b2980 0x7fdd0015ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50466/0 (socket says 172.21.15.45:50466) 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdcffe00640 1 -- 172.21.15.45:0/223358305 learned_addr learned my addr 172.21.15.45:0/223358305 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd05000640 1 --2- 172.21.15.45:0/223358305 >> v2:172.21.15.117:3300/0 conn(0x7fdd0015fce0 0x7fdd000b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdcff400640 1 -- 172.21.15.45:0/223358305 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181994213 0 0) 0x7fdd00160650 con 0x7fdd000b2980 2024-04-25T10:25:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdcff400640 1 -- 172.21.15.45:0/223358305 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce8003530 con 0x7fdd000b2980 2024-04-25T10:25:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd05000640 1 -- 172.21.15.45:0/223358305 >> v1:172.21.15.117:6789/0 conn(0x7fdd0015f550 legacy=0x7fdd000cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdcff400640 1 -- 172.21.15.45:0/223358305 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604793162 0 0) 0x7fdd00160930 con 0x7fdd0015f550 2024-04-25T10:25:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd05000640 1 -- 172.21.15.45:0/223358305 >> v1:172.21.15.45:6789/0 conn(0x7fdd000b2980 legacy=0x7fdd0015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.488+0000 7fdd05000640 1 -- 172.21.15.45:0/223358305 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000cd3c0 con 0x7fdd0015fce0 2024-04-25T10:25:16.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd05000640 1 --2- 172.21.15.45:0/223358305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015fce0 0x7fdd000b2440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40097c0 tx=0x7fdcf401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=217e76c96557d6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:16.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdcff400640 1 -- 172.21.15.45:0/223358305 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40aa5f0 con 0x7fdd0015fce0 2024-04-25T10:25:16.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdcff400640 1 -- 172.21.15.45:0/223358305 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf40af070 con 0x7fdd0015fce0 2024-04-25T10:25:16.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdcff400640 1 -- 172.21.15.45:0/223358305 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf40aa9b0 con 0x7fdd0015fce0 2024-04-25T10:25:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 -- 172.21.15.45:0/223358305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015fce0 msgr2=0x7fdd000b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 --2- 172.21.15.45:0/223358305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015fce0 0x7fdd000b2440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40097c0 tx=0x7fdcf401c470 comp rx=0 tx=0).stop 2024-04-25T10:25:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 -- 172.21.15.45:0/223358305 shutdown_connections 2024-04-25T10:25:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 --2- 172.21.15.45:0/223358305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015fce0 0x7fdd000b2440 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 -- 172.21.15.45:0/223358305 >> 172.21.15.45:0/223358305 conn(0x7fdd000b3430 msgr2=0x7fdd000b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 -- 172.21.15.45:0/223358305 shutdown_connections 2024-04-25T10:25:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.492+0000 7fdd07a00640 1 -- 172.21.15.45:0/223358305 wait complete. 2024-04-25T10:25:16.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 Processor -- start 2024-04-25T10:25:16.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- start start 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000b2980 0x7fdd000aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015f550 0x7fdd00148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 0x7fdd000b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000d08e0 con 0x7fdd0015fce0 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000d0760 con 0x7fdd0015f550 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00169dd0 con 0x7fdd000b2980 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 0x7fdd000b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 0x7fdd000b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51916/0 (socket says 172.21.15.45:51916) 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 -- 172.21.15.45:0/3228206643 learned_addr learned my addr 172.21.15.45:0/3228206643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05000640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000b2980 0x7fdd000aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:16.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdcffe00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015f550 0x7fdd00148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 -- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000b2980 msgr2=0x7fdd000aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000b2980 0x7fdd000aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 -- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015f550 msgr2=0x7fdd00148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015f550 0x7fdd00148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 -- 172.21.15.45:0/3228206643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00160790 con 0x7fdd0015fce0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdcffe00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015f550 0x7fdd00148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd05a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 0x7fdd000b4450 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c780 tx=0x7fdcf800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8003db0 con 0x7fdd0015fce0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf8005070 con 0x7fdd0015fce0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8005460 con 0x7fdd0015fce0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd000b6550 con 0x7fdd0015fce0 2024-04-25T10:25:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd000b6db0 con 0x7fdd0015fce0 2024-04-25T10:25:16.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.496+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0004ef30 con 0x7fdd0015fce0 2024-04-25T10:25:16.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.500+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcf8005600 con 0x7fdd0015fce0 2024-04-25T10:25:16.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.500+0000 7fdcfcc00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdce0066f60 0x7fdce0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:16.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.500+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdcf8084f10 con 0x7fdd0015fce0 2024-04-25T10:25:16.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.500+0000 7fdd05000640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdce0066f60 0x7fdce0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:16.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.500+0000 7fdd05000640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdce0066f60 0x7fdce0069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40ae000 tx=0x7fdcf40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:16.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.504+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcf8055ee0 con 0x7fdd0015fce0 2024-04-25T10:25:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:16.696+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdd000cf290 con 0x7fdd0015fce0 2024-04-25T10:25:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.320+0000 7fdcfcc00640 1 -- 172.21.15.45:0/3228206643 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v69) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdcf80557e0 con 0x7fdd0015fce0 2024-04-25T10:25:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.332+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdce0066f60 msgr2=0x7fdce0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.332+0000 7fdd07a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdce0066f60 0x7fdce0069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40ae000 tx=0x7fdcf40a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.332+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 msgr2=0x7fdd000b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.332+0000 7fdd07a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 0x7fdd000b4450 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c780 tx=0x7fdcf800cc40 comp rx=0 tx=0).stop 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 shutdown_connections 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdce0066f60 0x7fdce0069420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd0015fce0 0x7fdd000b4450 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd0015f550 0x7fdd00148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 --2- 172.21.15.45:0/3228206643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd000b2980 0x7fdd000aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 >> 172.21.15.45:0/3228206643 conn(0x7fdd000b3430 msgr2=0x7fdd0015ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:17.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 shutdown_connections 2024-04-25T10:25:17.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.336+0000 7fdd07a00640 1 -- 172.21.15.45:0/3228206643 wait complete. 2024-04-25T10:25:17.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.504+0000 7f595a000640 1 Processor -- start 2024-04-25T10:25:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.504+0000 7f595a000640 1 -- start start 2024-04-25T10:25:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.504+0000 7f595a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59540cda10 0x7f595406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f595a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59540cf930 0x7f59540cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f595a000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540b2770 con 0x7f595415bc40 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f595a000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595406c670 con 0x7f59540cf930 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f595a000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595406c950 con 0x7f59540cda10 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f595415bc40 0x7f59540b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50470/0 (socket says 172.21.15.45:50470) 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953400640 1 -- 172.21.15.45:0/3918635357 learned_addr learned my addr 172.21.15.45:0/3918635357 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953e00640 1 --2- 172.21.15.45:0/3918635357 >> v2:172.21.15.45:3300/0 conn(0x7f59540cf930 0x7f59540cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f594be00640 1 --2- 172.21.15.45:0/3918635357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59540cda10 0x7f595406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5952a00640 1 -- 172.21.15.45:0/3918635357 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105758924 0 0) 0x7f59540b2770 con 0x7f595415bc40 2024-04-25T10:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5952a00640 1 -- 172.21.15.45:0/3918635357 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5938003530 con 0x7f595415bc40 2024-04-25T10:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953e00640 1 -- 172.21.15.45:0/3918635357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59540cda10 msgr2=0x7f595406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953e00640 1 --2- 172.21.15.45:0/3918635357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59540cda10 0x7f595406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953e00640 1 -- 172.21.15.45:0/3918635357 >> v1:172.21.15.45:6789/0 conn(0x7f595415bc40 legacy=0x7f59540b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953e00640 1 -- 172.21.15.45:0/3918635357 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595406cc30 con 0x7f59540cf930 2024-04-25T10:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5953e00640 1 --2- 172.21.15.45:0/3918635357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cf930 0x7f59540cfde0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f594c009d90 tx=0x7f594c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d11aab43178019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.508+0000 7f5952a00640 1 -- 172.21.15.45:0/3918635357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0b2070 con 0x7f59540cf930 2024-04-25T10:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5952a00640 1 -- 172.21.15.45:0/3918635357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c0a57a0 con 0x7f59540cf930 2024-04-25T10:25:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5952a00640 1 -- 172.21.15.45:0/3918635357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0a5b70 con 0x7f59540cf930 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- 172.21.15.45:0/3918635357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cf930 msgr2=0x7f59540cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 --2- 172.21.15.45:0/3918635357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cf930 0x7f59540cfde0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f594c009d90 tx=0x7f594c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- 172.21.15.45:0/3918635357 shutdown_connections 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 --2- 172.21.15.45:0/3918635357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cf930 0x7f59540cfde0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 --2- 172.21.15.45:0/3918635357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59540cda10 0x7f595406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- 172.21.15.45:0/3918635357 >> 172.21.15.45:0/3918635357 conn(0x7f5954066b70 msgr2=0x7f5954066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:17.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- 172.21.15.45:0/3918635357 shutdown_connections 2024-04-25T10:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- 172.21.15.45:0/3918635357 wait complete. 2024-04-25T10:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 Processor -- start 2024-04-25T10:25:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- start start 2024-04-25T10:25:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 0x7f59540b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59540cf930 0x7f595415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595415bc40 0x7f595414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b3850 con 0x7f59540cda10 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b36d0 con 0x7f59540cf930 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f595a000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540b39d0 con 0x7f595415bc40 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 0x7f59540b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 0x7f59540b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51926/0 (socket says 172.21.15.45:51926) 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 -- 172.21.15.45:0/861460705 learned_addr learned my addr 172.21.15.45:0/861460705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f594be00640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59540cf930 0x7f595415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953e00640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595415bc40 0x7f595414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 -- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595415bc40 msgr2=0x7f595414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595415bc40 0x7f595414e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 -- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59540cf930 msgr2=0x7f595415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.512+0000 7f5953400640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59540cf930 0x7f595415d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f5953400640 1 -- 172.21.15.45:0/861460705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595414e700 con 0x7f59540cda10 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f5953400640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 0x7f59540b0ff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f594400c3c0 tx=0x7f594400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944017070 con 0x7f59540cda10 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5944005cf0 con 0x7f59540cda10 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540b15d0 con 0x7f59540cda10 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440072f0 con 0x7f59540cda10 2024-04-25T10:25:17.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540af480 con 0x7f59540cda10 2024-04-25T10:25:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595404ef30 con 0x7f59540cda10 2024-04-25T10:25:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f594401a070 con 0x7f59540cda10 2024-04-25T10:25:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f594b400640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5930066f60 0x7f5930069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.516+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59440844d0 con 0x7f59540cda10 2024-04-25T10:25:17.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.520+0000 7f594be00640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5930066f60 0x7f5930069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.520+0000 7f594be00640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5930066f60 0x7f5930069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5954150fc0 tx=0x7f593c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:17.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.520+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5944055440 con 0x7f59540cda10 2024-04-25T10:25:17.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.684+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5954160150 con 0x7f59540cda10 2024-04-25T10:25:17.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f594b400640 1 -- 172.21.15.45:0/861460705 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f5944054d40 con 0x7f59540cda10 2024-04-25T10:25:17.688 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:17.688 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:16.701730+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5930066f60 msgr2=0x7f5930069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5930066f60 0x7f5930069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5954150fc0 tx=0x7f593c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 msgr2=0x7f59540b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 0x7f59540b0ff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f594400c3c0 tx=0x7f594400c880 comp rx=0 tx=0).stop 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 shutdown_connections 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5930066f60 0x7f5930069420 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595415bc40 0x7f595414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59540cf930 0x7f595415d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 --2- 172.21.15.45:0/861460705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59540cda10 0x7f59540b0ff0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 >> 172.21.15.45:0/861460705 conn(0x7f5954066b70 msgr2=0x7f595415f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 shutdown_connections 2024-04-25T10:25:17.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.688+0000 7f595a000640 1 -- 172.21.15.45:0/861460705 wait complete. 2024-04-25T10:25:17.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:17.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.848+0000 7fdf31800640 1 Processor -- start 2024-04-25T10:25:17.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 -- start start 2024-04-25T10:25:17.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdf2c0d1820 0x7fdf2c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf2c15e050 0x7fdf2c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b1850 con 0x7fdf2c15e050 2024-04-25T10:25:17.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b1af0 con 0x7fdf2c0d1820 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf31800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b1dd0 con 0x7fdf2c0b13e0 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf2c15e050 0x7fdf2c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdf2c15e050 0x7fdf2c0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51930/0 (socket says 172.21.15.45:51930) 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 -- 172.21.15.45:0/2368927775 learned_addr learned my addr 172.21.15.45:0/2368927775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf2aa00640 1 --2- 172.21.15.45:0/2368927775 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c0d1820 0x7fdf2c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf2b400640 1 --2- 172.21.15.45:0/2368927775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 -- 172.21.15.45:0/2368927775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 msgr2=0x7fdf2c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 --2- 172.21.15.45:0/2368927775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c0ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 -- 172.21.15.45:0/2368927775 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c0d1820 msgr2=0x7fdf2c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 --2- 172.21.15.45:0/2368927775 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c0d1820 0x7fdf2c0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.852+0000 7fdf23e00640 1 -- 172.21.15.45:0/2368927775 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0b20b0 con 0x7fdf2c15e050 2024-04-25T10:25:17.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf23e00640 1 --2- 172.21.15.45:0/2368927775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0b0ea0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c070e90 tx=0x7fdf1401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742e831c628b3b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:17.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf2a000640 1 -- 172.21.15.45:0/2368927775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140b0070 con 0x7fdf2c15e050 2024-04-25T10:25:17.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf2a000640 1 -- 172.21.15.45:0/2368927775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf140ab600 con 0x7fdf2c15e050 2024-04-25T10:25:17.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf2a000640 1 -- 172.21.15.45:0/2368927775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140ab9d0 con 0x7fdf2c15e050 2024-04-25T10:25:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 -- 172.21.15.45:0/2368927775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 msgr2=0x7fdf2c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 --2- 172.21.15.45:0/2368927775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0b0ea0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c070e90 tx=0x7fdf1401c260 comp rx=0 tx=0).stop 2024-04-25T10:25:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 -- 172.21.15.45:0/2368927775 shutdown_connections 2024-04-25T10:25:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 --2- 172.21.15.45:0/2368927775 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 --2- 172.21.15.45:0/2368927775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0b0ea0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 --2- 172.21.15.45:0/2368927775 >> v2:172.21.15.117:3300/0 conn(0x7fdf2c0d1820 0x7fdf2c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 -- 172.21.15.45:0/2368927775 >> 172.21.15.45:0/2368927775 conn(0x7fdf2c09a9d0 msgr2=0x7fdf2c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:17.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 -- 172.21.15.45:0/2368927775 shutdown_connections 2024-04-25T10:25:17.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 -- 172.21.15.45:0/2368927775 wait complete. 2024-04-25T10:25:17.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 Processor -- start 2024-04-25T10:25:17.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.856+0000 7fdf31800640 1 -- start start 2024-04-25T10:25:17.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0d1820 0x7fdf2c151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cfdf0 con 0x7fdf2c15e050 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cfc70 con 0x7fdf2c0d1820 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cff70 con 0x7fdf2c0b13e0 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51944/0 (socket says 172.21.15.45:51944) 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 -- 172.21.15.45:0/374580280 learned_addr learned my addr 172.21.15.45:0/374580280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2aa00640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf23e00640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0d1820 0x7fdf2c151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 -- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 msgr2=0x7fdf2c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 -- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0d1820 msgr2=0x7fdf2c151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0d1820 0x7fdf2c151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:17.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 -- 172.21.15.45:0/374580280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0b44d0 con 0x7fdf2c15e050 2024-04-25T10:25:17.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf2b400640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0c5f90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007e40 tx=0x7fdf2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:17.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24014620 con 0x7fdf2c15e050 2024-04-25T10:25:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24019070 con 0x7fdf2c15e050 2024-04-25T10:25:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0b6480 con 0x7fdf2c15e050 2024-04-25T10:25:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24014a20 con 0x7fdf2c15e050 2024-04-25T10:25:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0b6a40 con 0x7fdf2c15e050 2024-04-25T10:25:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c04ef30 con 0x7fdf2c15e050 2024-04-25T10:25:17.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.860+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf24014bc0 con 0x7fdf2c15e050 2024-04-25T10:25:17.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.864+0000 7fdf22a00640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:17.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.864+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf240857b0 con 0x7fdf2c15e050 2024-04-25T10:25:17.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.864+0000 7fdf2aa00640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:17.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.864+0000 7fdf2aa00640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c15c5f0 tx=0x7fdf1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:17.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:17.868+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf24056160 con 0x7fdf2c15e050 2024-04-25T10:25:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.036+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2c06ea80 con 0x7fdf2c15e050 2024-04-25T10:25:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf22a00640 1 -- 172.21.15.45:0/374580280 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16620 (secure 0 0 0) 0x7fdf2405a200 con 0x7fdf2c15e050 2024-04-25T10:25:18.040 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:18.040 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:16.701730+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 msgr2=0x7fdf08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c15c5f0 tx=0x7fdf1c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 msgr2=0x7fdf2c0c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0c5f90 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007e40 tx=0x7fdf2400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 shutdown_connections 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf08066f60 0x7fdf08069420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c15e050 0x7fdf2c0c5f90 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c0d1820 0x7fdf2c151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 --2- 172.21.15.45:0/374580280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf2c0b13e0 0x7fdf2c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 >> 172.21.15.45:0/374580280 conn(0x7fdf2c09a9d0 msgr2=0x7fdf2c06fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 shutdown_connections 2024-04-25T10:25:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.040+0000 7fdf31800640 1 -- 172.21.15.45:0/374580280 wait complete. 2024-04-25T10:25:18.062 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:25:18.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd066600640 1 Processor -- start 2024-04-25T10:25:18.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd066600640 1 -- start start 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd066600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0600cfb30 0x7fd0600d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd066600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06006b470 con 0x7fd0600ce220 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd066600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06006b750 con 0x7fd06015dd30 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd066600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06015e110 con 0x7fd0600cfb30 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd057e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0600cfb30 0x7fd0600d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd05fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd0600ce220 0x7fd0600b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50476/0 (socket says 172.21.15.45:50476) 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd05fe00640 1 -- 172.21.15.45:0/4260480152 learned_addr learned my addr 172.21.15.45:0/4260480152 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd05ea00640 1 -- 172.21.15.45:0/4260480152 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215244118 0 0) 0x7fd06006b470 con 0x7fd0600ce220 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd057e00640 1 -- 172.21.15.45:0/4260480152 >> v1:172.21.15.117:6789/0 conn(0x7fd06015dd30 legacy=0x7fd06006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd057e00640 1 -- 172.21.15.45:0/4260480152 >> v1:172.21.15.45:6789/0 conn(0x7fd0600ce220 legacy=0x7fd0600b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.204+0000 7fd057e00640 1 -- 172.21.15.45:0/4260480152 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06006ba30 con 0x7fd0600cfb30 2024-04-25T10:25:18.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd057e00640 1 --2- 172.21.15.45:0/4260480152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600d18a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd0480097c0 tx=0x7fd04801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0020c2e8d115fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:18.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd05ea00640 1 -- 172.21.15.45:0/4260480152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480b0070 con 0x7fd0600cfb30 2024-04-25T10:25:18.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd05ea00640 1 -- 172.21.15.45:0/4260480152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480ab600 con 0x7fd0600cfb30 2024-04-25T10:25:18.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd05ea00640 1 -- 172.21.15.45:0/4260480152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480ab9d0 con 0x7fd0600cfb30 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 -- 172.21.15.45:0/4260480152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 msgr2=0x7fd0600d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 --2- 172.21.15.45:0/4260480152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600d18a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd0480097c0 tx=0x7fd04801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 -- 172.21.15.45:0/4260480152 shutdown_connections 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 --2- 172.21.15.45:0/4260480152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600d18a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 -- 172.21.15.45:0/4260480152 >> 172.21.15.45:0/4260480152 conn(0x7fd06009a9d0 msgr2=0x7fd06009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 -- 172.21.15.45:0/4260480152 shutdown_connections 2024-04-25T10:25:18.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.208+0000 7fd066600640 1 -- 172.21.15.45:0/4260480152 wait complete. 2024-04-25T10:25:18.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 Processor -- start 2024-04-25T10:25:18.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 -- start start 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0600ce220 0x7fd06014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06015dd30 0x7fd0600c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06016bf00 con 0x7fd0600cfb30 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06016bd80 con 0x7fd06015dd30 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06016c080 con 0x7fd0600ce220 2024-04-25T10:25:18.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51966/0 (socket says 172.21.15.45:51966) 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 -- 172.21.15.45:0/68970939 learned_addr learned my addr 172.21.15.45:0/68970939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd05fe00640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06015dd30 0x7fd0600c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd05f400640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0600ce220 0x7fd06014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 -- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0600ce220 msgr2=0x7fd06014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0600ce220 0x7fd06014e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 -- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06015dd30 msgr2=0x7fd0600c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06015dd30 0x7fd0600c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 -- 172.21.15.45:0/68970939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600c61a0 con 0x7fd0600cfb30 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057e00640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600c7780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd0480190b0 tx=0x7fd04801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480b0050 con 0x7fd0600cfb30 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480bc070 con 0x7fd0600cfb30 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480aac00 con 0x7fd0600cfb30 2024-04-25T10:25:18.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600ae930 con 0x7fd0600cfb30 2024-04-25T10:25:18.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.212+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0600aeef0 con 0x7fd0600cfb30 2024-04-25T10:25:18.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.216+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06004ef30 con 0x7fd0600cfb30 2024-04-25T10:25:18.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.216+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0480c1460 con 0x7fd0600cfb30 2024-04-25T10:25:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.216+0000 7fd057400640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd03c066f90 0x7fd03c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.216+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd04811fb90 con 0x7fd0600cfb30 2024-04-25T10:25:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.216+0000 7fd05f400640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd03c066f90 0x7fd03c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:18.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.216+0000 7fd05f400640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd03c066f90 0x7fd03c069450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd060070050 tx=0x7fd050009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:18.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.220+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd048016090 con 0x7fd0600cfb30 2024-04-25T10:25:18.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:25:18.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:18.420+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd060068190 con 0x7fd0600cfb30 2024-04-25T10:25:19.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.332+0000 7fd057400640 1 -- 172.21.15.45:0/68970939 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd0480f0b90 con 0x7fd0600cfb30 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd03c066f90 msgr2=0x7fd03c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd03c066f90 0x7fd03c069450 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd060070050 tx=0x7fd050009040 comp rx=0 tx=0).stop 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 msgr2=0x7fd0600c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600c7780 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd0480190b0 tx=0x7fd04801ace0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 shutdown_connections 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd03c066f90 0x7fd03c069450 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd06015dd30 0x7fd0600c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0600cfb30 0x7fd0600c7780 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 --2- 172.21.15.45:0/68970939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0600ce220 0x7fd06014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 >> 172.21.15.45:0/68970939 conn(0x7fd06009a9d0 msgr2=0x7fd0600cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:19.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 shutdown_connections 2024-04-25T10:25:19.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.336+0000 7fd066600640 1 -- 172.21.15.45:0/68970939 wait complete. 2024-04-25T10:25:19.360 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:25:19.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:25:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5778000640 1 Processor -- start 2024-04-25T10:25:19.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5778000640 1 -- start start 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5778000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f57700d1830 0x7f57700d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5778000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700b1770 con 0x7f577015bc10 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5778000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577015fec0 con 0x7f57700cf980 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5778000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57701601a0 con 0x7f57700d1830 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5775600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f577015bc10 0x7f577015f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50478/0 (socket says 172.21.15.45:50478) 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5775600640 1 -- 172.21.15.45:0/1691315824 learned_addr learned my addr 172.21.15.45:0/1691315824 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5776000640 1 --2- 172.21.15.45:0/1691315824 >> v2:172.21.15.45:3300/0 conn(0x7f57700d1830 0x7f57700d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5776000640 1 -- 172.21.15.45:0/1691315824 >> v1:172.21.15.117:6789/0 conn(0x7f57700cf980 legacy=0x7f57700b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f576fe00640 1 -- 172.21.15.45:0/1691315824 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129143816 0 0) 0x7f57700b1770 con 0x7f577015bc10 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5776000640 1 -- 172.21.15.45:0/1691315824 >> v1:172.21.15.45:6789/0 conn(0x7f577015bc10 legacy=0x7f577015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.508+0000 7f5776000640 1 -- 172.21.15.45:0/1691315824 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700cc5b0 con 0x7f57700d1830 2024-04-25T10:25:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5776000640 1 --2- 172.21.15.45:0/1691315824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57700d1830 0x7f57700d1c10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f576801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c75f08e94321bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f576fe00640 1 -- 172.21.15.45:0/1691315824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680b2070 con 0x7f57700d1830 2024-04-25T10:25:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f576fe00640 1 -- 172.21.15.45:0/1691315824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57680a57b0 con 0x7f57700d1830 2024-04-25T10:25:19.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f576fe00640 1 -- 172.21.15.45:0/1691315824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680a5b80 con 0x7f57700d1830 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 -- 172.21.15.45:0/1691315824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57700d1830 msgr2=0x7f57700d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 --2- 172.21.15.45:0/1691315824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57700d1830 0x7f57700d1c10 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f576801c970 comp rx=0 tx=0).stop 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 -- 172.21.15.45:0/1691315824 shutdown_connections 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 --2- 172.21.15.45:0/1691315824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57700d1830 0x7f57700d1c10 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 -- 172.21.15.45:0/1691315824 >> 172.21.15.45:0/1691315824 conn(0x7f577009a9d0 msgr2=0x7f577009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 -- 172.21.15.45:0/1691315824 shutdown_connections 2024-04-25T10:25:19.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.512+0000 7f5778000640 1 -- 172.21.15.45:0/1691315824 wait complete. 2024-04-25T10:25:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 Processor -- start 2024-04-25T10:25:19.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 -- start start 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57700cf980 0x7f5770165a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57700d1830 0x7f5770152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 0x7f57700b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57700cd740 con 0x7f577015bc10 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57700cd5c0 con 0x7f57700d1830 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5770169970 con 0x7f57700cf980 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 0x7f57700b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 0x7f57700b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51986/0 (socket says 172.21.15.45:51986) 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 -- 172.21.15.45:0/3421899622 learned_addr learned my addr 172.21.15.45:0/3421899622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5774c00640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57700d1830 0x7f5770152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:19.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5775600640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57700cf980 0x7f5770165a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 -- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57700cf980 msgr2=0x7f5770165a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57700cf980 0x7f5770165a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 -- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57700d1830 msgr2=0x7f5770152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57700d1830 0x7f5770152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 -- 172.21.15.45:0/3421899622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57701661b0 con 0x7f577015bc10 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5774c00640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57700d1830 0x7f5770152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5776000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 0x7f57700b5fb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f5768019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680bd070 con 0x7f577015bc10 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5768003ba0 con 0x7f577015bc10 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680b2040 con 0x7f577015bc10 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5770166490 con 0x7f577015bc10 2024-04-25T10:25:19.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.516+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577015c980 con 0x7f577015bc10 2024-04-25T10:25:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.520+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577004ef30 con 0x7f577015bc10 2024-04-25T10:25:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.520+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5768004030 con 0x7f577015bc10 2024-04-25T10:25:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.520+0000 7f576d600640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5750066f60 0x7f5750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:19.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.520+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f576811ff90 con 0x7f577015bc10 2024-04-25T10:25:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.520+0000 7f5775600640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5750066f60 0x7f5750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:19.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.520+0000 7f5775600640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5750066f60 0x7f5750069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5764007fd0 tx=0x7f5764007ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:19.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.524+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57680c7020 con 0x7f577015bc10 2024-04-25T10:25:19.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:19.720+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f57700b7920 con 0x7f577015bc10 2024-04-25T10:25:20.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f576d600640 1 -- 172.21.15.45:0/3421899622 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5768016560 con 0x7f577015bc10 2024-04-25T10:25:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5750066f60 msgr2=0x7f5750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5750066f60 0x7f5750069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5764007fd0 tx=0x7f5764007ba0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 msgr2=0x7f57700b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 0x7f57700b5fb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f5768019710 comp rx=0 tx=0).stop 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 shutdown_connections 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5750066f60 0x7f5750069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f577015bc10 0x7f57700b5fb0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57700d1830 0x7f5770152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 --2- 172.21.15.45:0/3421899622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57700cf980 0x7f5770165a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 >> 172.21.15.45:0/3421899622 conn(0x7f577009a9d0 msgr2=0x7f5770153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 shutdown_connections 2024-04-25T10:25:20.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.336+0000 7f5778000640 1 -- 172.21.15.45:0/3421899622 wait complete. 2024-04-25T10:25:20.361 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:25:20.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 Processor -- start 2024-04-25T10:25:20.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 -- start start 2024-04-25T10:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f53e00ccb20 0x7f53e00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e015bdc0 0x7f53e00d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00ccfb0 con 0x7f53e015b9e0 2024-04-25T10:25:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00cd290 con 0x7f53e00ccb20 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e7a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00cd570 con 0x7f53e015bdc0 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f53e00ccb20 0x7f53e00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f53e00ccb20 0x7f53e00d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52002/0 (socket says 172.21.15.45:52002) 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 -- 172.21.15.45:0/5736461 learned_addr learned my addr 172.21.15.45:0/5736461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53e5a00640 1 --2- 172.21.15.45:0/5736461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e015bdc0 0x7f53e00d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 -- 172.21.15.45:0/5736461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e015bdc0 msgr2=0x7f53e00d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 --2- 172.21.15.45:0/5736461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e015bdc0 0x7f53e00d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 -- 172.21.15.45:0/5736461 >> v1:172.21.15.117:6789/0 conn(0x7f53e015b9e0 legacy=0x7f53e00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.504+0000 7f53dfe00640 1 -- 172.21.15.45:0/5736461 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00cd850 con 0x7f53e00ccb20 2024-04-25T10:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53dfe00640 1 --2- 172.21.15.45:0/5736461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e00ccb20 0x7f53e00d1520 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009870 tx=0x7f53cc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c329493a8a71dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53df400640 1 -- 172.21.15.45:0/5736461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0b2070 con 0x7f53e00ccb20 2024-04-25T10:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53df400640 1 -- 172.21.15.45:0/5736461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0a9070 con 0x7f53e00ccb20 2024-04-25T10:25:20.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53df400640 1 -- 172.21.15.45:0/5736461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0ad880 con 0x7f53e00ccb20 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 -- 172.21.15.45:0/5736461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e00ccb20 msgr2=0x7f53e00d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 --2- 172.21.15.45:0/5736461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e00ccb20 0x7f53e00d1520 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009870 tx=0x7f53cc01c560 comp rx=0 tx=0).stop 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 -- 172.21.15.45:0/5736461 shutdown_connections 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 --2- 172.21.15.45:0/5736461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e015bdc0 0x7f53e00d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 --2- 172.21.15.45:0/5736461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e00ccb20 0x7f53e00d1520 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 -- 172.21.15.45:0/5736461 >> 172.21.15.45:0/5736461 conn(0x7f53e009a9d0 msgr2=0x7f53e009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 -- 172.21.15.45:0/5736461 shutdown_connections 2024-04-25T10:25:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 -- 172.21.15.45:0/5736461 wait complete. 2024-04-25T10:25:20.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 Processor -- start 2024-04-25T10:25:20.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.508+0000 7f53e7a00640 1 -- start start 2024-04-25T10:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e015b9e0 0x7f53e014aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 0x7f53e015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e006c1e0 0x7f53e0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c65f0 con 0x7f53e015bdc0 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c6470 con 0x7f53e015b9e0 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00c6770 con 0x7f53e006c1e0 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 0x7f53e015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 0x7f53e015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52006/0 (socket says 172.21.15.45:52006) 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 -- 172.21.15.45:0/1421025027 learned_addr learned my addr 172.21.15.45:0/1421025027 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e5a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e006c1e0 0x7f53e0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e5000640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e015b9e0 0x7f53e014aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 -- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e006c1e0 msgr2=0x7f53e0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e006c1e0 0x7f53e0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 -- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e015b9e0 msgr2=0x7f53e014aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e015b9e0 0x7f53e014aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 -- 172.21.15.45:0/1421025027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e006cbc0 con 0x7f53e015bdc0 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dfe00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 0x7f53e015adf0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0b0040 tx=0x7f53cc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0b2040 con 0x7f53e015bdc0 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0bb070 con 0x7f53e015bdc0 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0b6440 con 0x7f53e015bdc0 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e015a030 con 0x7f53e015bdc0 2024-04-25T10:25:20.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e015a5f0 con 0x7f53e015bdc0 2024-04-25T10:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.512+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e004ef30 con 0x7f53e015bdc0 2024-04-25T10:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.516+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53cc0ab070 con 0x7f53e015bdc0 2024-04-25T10:25:20.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.516+0000 7f53dcc00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f53c0066f60 0x7f53c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.516+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53cc11f340 con 0x7f53e015bdc0 2024-04-25T10:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.516+0000 7f53e5000640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f53c0066f60 0x7f53c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.516+0000 7f53e5000640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f53c0066f60 0x7f53c0069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f53d4007fd0 tx=0x7f53d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:20.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.520+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53cc016150 con 0x7f53e015bdc0 2024-04-25T10:25:20.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.716+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53e006f850 con 0x7f53e015bdc0 2024-04-25T10:25:20.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.716+0000 7f53dcc00640 1 -- 172.21.15.45:0/1421025027 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f53cc0efde0 con 0x7f53e015bdc0 2024-04-25T10:25:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:20.718 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:20.718 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f53c0066f60 msgr2=0x7f53c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f53c0066f60 0x7f53c0069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f53d4007fd0 tx=0x7f53d4009040 comp rx=0 tx=0).stop 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 msgr2=0x7f53e015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 0x7f53e015adf0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0b0040 tx=0x7f53cc019300 comp rx=0 tx=0).stop 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 shutdown_connections 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f53c0066f60 0x7f53c0069420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53e006c1e0 0x7f53e0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53e015bdc0 0x7f53e015adf0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 --2- 172.21.15.45:0/1421025027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53e015b9e0 0x7f53e014aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 >> 172.21.15.45:0/1421025027 conn(0x7f53e009a9d0 msgr2=0x7f53e006f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 shutdown_connections 2024-04-25T10:25:20.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.720+0000 7f53e7a00640 1 -- 172.21.15.45:0/1421025027 wait complete. 2024-04-25T10:25:20.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 Processor -- start 2024-04-25T10:25:20.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 -- start start 2024-04-25T10:25:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effc406e120 0x7effc40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effc40cc5d0 0x7effc40cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc4071f70 con 0x7effc415bcf0 2024-04-25T10:25:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc40ccf70 con 0x7effc406e120 2024-04-25T10:25:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc9600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc4072250 con 0x7effc40cc5d0 2024-04-25T10:25:20.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effc406e120 0x7effc40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effc406e120 0x7effc40b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52016/0 (socket says 172.21.15.45:52016) 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 -- 172.21.15.45:0/654216889 learned_addr learned my addr 172.21.15.45:0/654216889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effc3400640 1 --2- 172.21.15.45:0/654216889 >> v2:172.21.15.117:3300/0 conn(0x7effc40cc5d0 0x7effc40cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 -- 172.21.15.45:0/654216889 >> v2:172.21.15.117:3300/0 conn(0x7effc40cc5d0 msgr2=0x7effc40cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 --2- 172.21.15.45:0/654216889 >> v2:172.21.15.117:3300/0 conn(0x7effc40cc5d0 0x7effc40cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 -- 172.21.15.45:0/654216889 >> v1:172.21.15.45:6789/0 conn(0x7effc415bcf0 legacy=0x7effc406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.880+0000 7effba000640 1 -- 172.21.15.45:0/654216889 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4072530 con 0x7effc406e120 2024-04-25T10:25:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effba000640 1 --2- 172.21.15.45:0/654216889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc406e120 0x7effc40b5560 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7effc40709d0 tx=0x7effac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4124c5b825cf989b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc2000640 1 -- 172.21.15.45:0/654216889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0b0070 con 0x7effc406e120 2024-04-25T10:25:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc2000640 1 -- 172.21.15.45:0/654216889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac0ab600 con 0x7effc406e120 2024-04-25T10:25:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc2000640 1 -- 172.21.15.45:0/654216889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0ab9d0 con 0x7effc406e120 2024-04-25T10:25:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 -- 172.21.15.45:0/654216889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc406e120 msgr2=0x7effc40b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 --2- 172.21.15.45:0/654216889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc406e120 0x7effc40b5560 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7effc40709d0 tx=0x7effac01c520 comp rx=0 tx=0).stop 2024-04-25T10:25:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 -- 172.21.15.45:0/654216889 shutdown_connections 2024-04-25T10:25:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 --2- 172.21.15.45:0/654216889 >> v2:172.21.15.117:3300/0 conn(0x7effc40cc5d0 0x7effc40cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 --2- 172.21.15.45:0/654216889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc406e120 0x7effc40b5560 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 -- 172.21.15.45:0/654216889 >> 172.21.15.45:0/654216889 conn(0x7effc40b1780 msgr2=0x7effc40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 -- 172.21.15.45:0/654216889 shutdown_connections 2024-04-25T10:25:20.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.884+0000 7effc9600640 1 -- 172.21.15.45:0/654216889 wait complete. 2024-04-25T10:25:20.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 Processor -- start 2024-04-25T10:25:20.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- start start 2024-04-25T10:25:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc406e120 0x7effc40ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc40cc5d0 0x7effc41f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 0x7effc41532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4168b20 con 0x7effc415bcf0 2024-04-25T10:25:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4160600 con 0x7effc40cc5d0 2024-04-25T10:25:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc4168ca0 con 0x7effc406e120 2024-04-25T10:25:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 0x7effc41532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 0x7effc41532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52026/0 (socket says 172.21.15.45:52026) 2024-04-25T10:25:20.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 -- 172.21.15.45:0/1903999270 learned_addr learned my addr 172.21.15.45:0/1903999270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:20.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc2a00640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc406e120 0x7effc40ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effba000640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc40cc5d0 0x7effc41f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 -- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc406e120 msgr2=0x7effc40ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc406e120 0x7effc40ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 -- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc40cc5d0 msgr2=0x7effc41f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:20.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc40cc5d0 0x7effc41f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 -- 172.21.15.45:0/1903999270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40adf10 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effba000640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc40cc5d0 0x7effc41f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc3400640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 0x7effc41532d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7effbc007e60 tx=0x7effbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc014420 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc019070 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc014820 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc40ae1f0 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc40ae7b0 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.888+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc40ae9f0 con 0x7effc415bcf0 2024-04-25T10:25:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.892+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effbc0149c0 con 0x7effc415bcf0 2024-04-25T10:25:20.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.892+0000 7effbb400640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effa0066f60 0x7effa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:20.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.892+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effbc085970 con 0x7effc415bcf0 2024-04-25T10:25:20.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.892+0000 7effc2a00640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effa0066f60 0x7effa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:20.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.892+0000 7effc2a00640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effa0066f60 0x7effa0069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7effb4001650 tx=0x7effb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:20.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:20.896+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effbc0568e0 con 0x7effc415bcf0 2024-04-25T10:25:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.060+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effc404ef30 con 0x7effc415bcf0 2024-04-25T10:25:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.060+0000 7effbb400640 1 -- 172.21.15.45:0/1903999270 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16607 (secure 0 0 0) 0x7effbc0561e0 con 0x7effc415bcf0 2024-04-25T10:25:21.062 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:21.062 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:18.307135+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effa0066f60 msgr2=0x7effa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effa0066f60 0x7effa0069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7effb4001650 tx=0x7effb4009040 comp rx=0 tx=0).stop 2024-04-25T10:25:21.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 msgr2=0x7effc41532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 0x7effc41532d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7effbc007e60 tx=0x7effbc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 shutdown_connections 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7effa0066f60 0x7effa0069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effc415bcf0 0x7effc41532d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effc40cc5d0 0x7effc41f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 --2- 172.21.15.45:0/1903999270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effc406e120 0x7effc40ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 >> 172.21.15.45:0/1903999270 conn(0x7effc40b1780 msgr2=0x7effc40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 shutdown_connections 2024-04-25T10:25:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.064+0000 7effc9600640 1 -- 172.21.15.45:0/1903999270 wait complete. 2024-04-25T10:25:21.084 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:25:21.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:25:21.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7488c00640 1 Processor -- start 2024-04-25T10:25:21.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7488c00640 1 -- start start 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7488c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74840d3490 0x7f74840d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7488c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840d23f0 con 0x7f748415dbd0 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7488c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840d26d0 con 0x7f74840d0af0 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7488c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840d29b0 con 0x7f74840d3490 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7483400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f748415dbd0 0x7f74840d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50506/0 (socket says 172.21.15.45:50506) 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7483400640 1 -- 172.21.15.45:0/4001357870 learned_addr learned my addr 172.21.15.45:0/4001357870 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7483e00640 1 --2- 172.21.15.45:0/4001357870 >> v2:172.21.15.45:3300/0 conn(0x7f74840d3490 0x7f74840d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7483e00640 1 -- 172.21.15.45:0/4001357870 >> v1:172.21.15.117:6789/0 conn(0x7f74840d0af0 legacy=0x7f74840b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7483e00640 1 -- 172.21.15.45:0/4001357870 >> v1:172.21.15.45:6789/0 conn(0x7f748415dbd0 legacy=0x7f74840d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.224+0000 7f7483e00640 1 -- 172.21.15.45:0/4001357870 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840d2c90 con 0x7f74840d3490 2024-04-25T10:25:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7483e00640 1 --2- 172.21.15.45:0/4001357870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74840d3490 0x7f74840d1d10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f747c009d90 tx=0x7f747c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff5d152a72c6dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7482a00640 1 -- 172.21.15.45:0/4001357870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c0b2070 con 0x7f74840d3490 2024-04-25T10:25:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7482a00640 1 -- 172.21.15.45:0/4001357870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c0a5830 con 0x7f74840d3490 2024-04-25T10:25:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7482a00640 1 -- 172.21.15.45:0/4001357870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c0a5c00 con 0x7f74840d3490 2024-04-25T10:25:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 -- 172.21.15.45:0/4001357870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74840d3490 msgr2=0x7f74840d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 --2- 172.21.15.45:0/4001357870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74840d3490 0x7f74840d1d10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f747c009d90 tx=0x7f747c01c970 comp rx=0 tx=0).stop 2024-04-25T10:25:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 -- 172.21.15.45:0/4001357870 shutdown_connections 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 --2- 172.21.15.45:0/4001357870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74840d3490 0x7f74840d1d10 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 -- 172.21.15.45:0/4001357870 >> 172.21.15.45:0/4001357870 conn(0x7f74840669a0 msgr2=0x7f7484066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 -- 172.21.15.45:0/4001357870 shutdown_connections 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.228+0000 7f7488c00640 1 -- 172.21.15.45:0/4001357870 wait complete. 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 Processor -- start 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 -- start start 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840d0af0 0x7f74840c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74840d3490 0x7f74840caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 0x7f748414e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748416bb30 con 0x7f748415dbd0 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748416b9b0 con 0x7f74840d3490 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748416bcb0 con 0x7f74840d0af0 2024-04-25T10:25:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 0x7f748414e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 0x7f748414e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52044/0 (socket says 172.21.15.45:52044) 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 -- 172.21.15.45:0/1219708618 learned_addr learned my addr 172.21.15.45:0/1219708618 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483400640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840d0af0 0x7f74840c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f747be00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74840d3490 0x7f74840caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 -- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840d0af0 msgr2=0x7f74840c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840d0af0 0x7f74840c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 -- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74840d3490 msgr2=0x7f74840caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74840d3490 0x7f74840caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 -- 172.21.15.45:0/1219708618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484153b10 con 0x7f748415dbd0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7483e00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 0x7f748414e660 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f747c0b0040 tx=0x7f747c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c0b2050 con 0x7f748415dbd0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c0be070 con 0x7f748415dbd0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c0adb10 con 0x7f748415dbd0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7484153df0 con 0x7f748415dbd0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.232+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841543b0 con 0x7f748415dbd0 2024-04-25T10:25:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.236+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7484075120 con 0x7f748415dbd0 2024-04-25T10:25:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.236+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f747c0adcb0 con 0x7f748415dbd0 2024-04-25T10:25:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.236+0000 7f747b400640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7460066f60 0x7f7460069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.236+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f747c120320 con 0x7f748415dbd0 2024-04-25T10:25:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.236+0000 7f7483400640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7460066f60 0x7f7460069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.236+0000 7f7483400640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7460066f60 0x7f7460069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7474005890 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.240+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f747c016560 con 0x7f748415dbd0 2024-04-25T10:25:21.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.400+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74840b3a90 con 0x7f7460066f60 2024-04-25T10:25:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.408+0000 7f747b400640 1 -- 172.21.15.45:0/1219708618 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252430 (secure 0 0 0) 0x7f74840b3a90 con 0x7f7460066f60 2024-04-25T10:25:21.411 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:25:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7460066f60 msgr2=0x7f7460069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7460066f60 0x7f7460069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7474005890 tx=0x7f7474009040 comp rx=0 tx=0).stop 2024-04-25T10:25:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 msgr2=0x7f748414e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 0x7f748414e660 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f747c0b0040 tx=0x7f747c019710 comp rx=0 tx=0).stop 2024-04-25T10:25:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 shutdown_connections 2024-04-25T10:25:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7460066f60 0x7f7460069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f748415dbd0 0x7f748414e660 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74840d3490 0x7f74840caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 --2- 172.21.15.45:0/1219708618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840d0af0 0x7f74840c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 >> 172.21.15.45:0/1219708618 conn(0x7f74840669a0 msgr2=0x7f74840b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 shutdown_connections 2024-04-25T10:25:21.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.412+0000 7f7488c00640 1 -- 172.21.15.45:0/1219708618 wait complete. 2024-04-25T10:25:21.431 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T10:25:20.976471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":403580,"kb_used_data":11136,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754571140,"statfs":{"total":773094113280,"available":772680847360,"internally_reserved":0,"allocated":11403264,"data_stored":5564164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682954"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333903+0000","last_change":"2024-04-25T10:25:17.329721+0000","last_active":"2024-04-25T10:25:17.333903+0000","last_peered":"2024-04-25T10:25:17.333903+0000","last_clean":"2024-04-25T10:25:17.333903+0000","last_became_active":"2024-04-25T10:25:17.326581+0000","last_became_peered":"2024-04-25T10:25:17.326581+0000","last_unstale":"2024-04-25T10:25:17.333903+0000","last_undegraded":"2024-04-25T10:25:17.333903+0000","last_fullsized":"2024-04-25T10:25:17.333903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:51.845697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.342328+0000","last_change":"2024-04-25T10:25:17.339795+0000","last_active":"2024-04-25T10:25:17.342328+0000","last_peered":"2024-04-25T10:25:17.342328+0000","last_clean":"2024-04-25T10:25:17.342328+0000","last_became_active":"2024-04-25T10:25:17.336746+0000","last_became_peered":"2024-04-25T10:25:17.336746+0000","last_unstale":"2024-04-25T10:25:17.342328+0000","last_undegraded":"2024-04-25T10:25:17.342328+0000","last_fullsized":"2024-04-25T10:25:17.342328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:49.834031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.331309+0000","last_change":"2024-04-25T10:25:17.324366+0000","last_active":"2024-04-25T10:25:17.331309+0000","last_peered":"2024-04-25T10:25:17.331309+0000","last_clean":"2024-04-25T10:25:17.331309+0000","last_became_active":"2024-04-25T10:25:17.324035+0000","last_became_peered":"2024-04-25T10:25:17.324035+0000","last_unstale":"2024-04-25T10:25:17.331309+0000","last_undegraded":"2024-04-25T10:25:17.331309+0000","last_fullsized":"2024-04-25T10:25:17.331309+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:58.594993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346165+0000","last_change":"2024-04-25T10:25:17.342861+0000","last_active":"2024-04-25T10:25:17.346165+0000","last_peered":"2024-04-25T10:25:17.346165+0000","last_clean":"2024-04-25T10:25:17.346165+0000","last_became_active":"2024-04-25T10:25:17.341100+0000","last_became_peered":"2024-04-25T10:25:17.341100+0000","last_unstale":"2024-04-25T10:25:17.346165+0000","last_undegraded":"2024-04-25T10:25:17.346165+0000","last_fullsized":"2024-04-25T10:25:17.346165+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:46.447089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333353+0000","last_change":"2024-04-25T10:25:17.324832+0000","last_active":"2024-04-25T10:25:17.333353+0000","last_peered":"2024-04-25T10:25:17.333353+0000","last_clean":"2024-04-25T10:25:17.333353+0000","last_became_active":"2024-04-25T10:25:17.324426+0000","last_became_peered":"2024-04-25T10:25:17.324426+0000","last_unstale":"2024-04-25T10:25:17.333353+0000","last_undegraded":"2024-04-25T10:25:17.333353+0000","last_fullsized":"2024-04-25T10:25:17.333353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:57.300709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.319931+0000","last_change":"2024-04-25T10:25:17.317381+0000","last_active":"2024-04-25T10:25:17.319931+0000","last_peered":"2024-04-25T10:25:17.319931+0000","last_clean":"2024-04-25T10:25:17.319931+0000","last_became_active":"2024-04-25T10:25:17.316947+0000","last_became_peered":"2024-04-25T10:25:17.316947+0000","last_unstale":"2024-04-25T10:25:17.319931+0000","last_undegraded":"2024-04-25T10:25:17.319931+0000","last_fullsized":"2024-04-25T10:25:17.319931+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:59.838154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333581+0000","last_change":"2024-04-25T10:25:17.331046+0000","last_active":"2024-04-25T10:25:17.333581+0000","last_peered":"2024-04-25T10:25:17.333581+0000","last_clean":"2024-04-25T10:25:17.333581+0000","last_became_active":"2024-04-25T10:25:17.330758+0000","last_became_peered":"2024-04-25T10:25:17.330758+0000","last_unstale":"2024-04-25T10:25:17.333581+0000","last_undegraded":"2024-04-25T10:25:17.333581+0000","last_fullsized":"2024-04-25T10:25:17.333581+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:55.524198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333394+0000","last_change":"2024-04-25T10:25:17.328466+0000","last_active":"2024-04-25T10:25:17.333394+0000","last_peered":"2024-04-25T10:25:17.333394+0000","last_clean":"2024-04-25T10:25:17.333394+0000","last_became_active":"2024-04-25T10:25:17.327799+0000","last_became_peered":"2024-04-25T10:25:17.327799+0000","last_unstale":"2024-04-25T10:25:17.333394+0000","last_undegraded":"2024-04-25T10:25:17.333394+0000","last_fullsized":"2024-04-25T10:25:17.333394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:31.032462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335354+0000","last_change":"2024-04-25T10:25:17.333819+0000","last_active":"2024-04-25T10:25:17.335354+0000","last_peered":"2024-04-25T10:25:17.335354+0000","last_clean":"2024-04-25T10:25:17.335354+0000","last_became_active":"2024-04-25T10:25:17.333479+0000","last_became_peered":"2024-04-25T10:25:17.333479+0000","last_unstale":"2024-04-25T10:25:17.335354+0000","last_undegraded":"2024-04-25T10:25:17.335354+0000","last_fullsized":"2024-04-25T10:25:17.335354+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:24.749760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338971+0000","last_change":"2024-04-25T10:25:17.338102+0000","last_active":"2024-04-25T10:25:17.338971+0000","last_peered":"2024-04-25T10:25:17.338971+0000","last_clean":"2024-04-25T10:25:17.338971+0000","last_became_active":"2024-04-25T10:25:17.337792+0000","last_became_peered":"2024-04-25T10:25:17.337792+0000","last_unstale":"2024-04-25T10:25:17.338971+0000","last_undegraded":"2024-04-25T10:25:17.338971+0000","last_fullsized":"2024-04-25T10:25:17.338971+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:43.585040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329754+0000","last_change":"2024-04-25T10:25:17.323439+0000","last_active":"2024-04-25T10:25:17.329754+0000","last_peered":"2024-04-25T10:25:17.329754+0000","last_clean":"2024-04-25T10:25:17.329754+0000","last_became_active":"2024-04-25T10:25:17.323141+0000","last_became_peered":"2024-04-25T10:25:17.323141+0000","last_unstale":"2024-04-25T10:25:17.329754+0000","last_undegraded":"2024-04-25T10:25:17.329754+0000","last_fullsized":"2024-04-25T10:25:17.329754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:30.922148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311449+0000","last_change":"2024-04-25T10:25:17.335766+0000","last_active":"2024-04-25T10:25:18.311449+0000","last_peered":"2024-04-25T10:25:18.311449+0000","last_clean":"2024-04-25T10:25:18.311449+0000","last_became_active":"2024-04-25T10:25:17.334231+0000","last_became_peered":"2024-04-25T10:25:17.334231+0000","last_unstale":"2024-04-25T10:25:18.311449+0000","last_undegraded":"2024-04-25T10:25:18.311449+0000","last_fullsized":"2024-04-25T10:25:18.311449+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:09.758846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337459+0000","last_change":"2024-04-25T10:25:17.330919+0000","last_active":"2024-04-25T10:25:17.337459+0000","last_peered":"2024-04-25T10:25:17.337459+0000","last_clean":"2024-04-25T10:25:17.337459+0000","last_became_active":"2024-04-25T10:25:17.330617+0000","last_became_peered":"2024-04-25T10:25:17.330617+0000","last_unstale":"2024-04-25T10:25:17.337459+0000","last_undegraded":"2024-04-25T10:25:17.337459+0000","last_fullsized":"2024-04-25T10:25:17.337459+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:45.783505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335135+0000","last_change":"2024-04-25T10:25:17.332791+0000","last_active":"2024-04-25T10:25:17.335135+0000","last_peered":"2024-04-25T10:25:17.335135+0000","last_clean":"2024-04-25T10:25:17.335135+0000","last_became_active":"2024-04-25T10:25:17.332536+0000","last_became_peered":"2024-04-25T10:25:17.332536+0000","last_unstale":"2024-04-25T10:25:17.335135+0000","last_undegraded":"2024-04-25T10:25:17.335135+0000","last_fullsized":"2024-04-25T10:25:17.335135+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:04.009172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337659+0000","last_change":"2024-04-25T10:25:17.334746+0000","last_active":"2024-04-25T10:25:17.337659+0000","last_peered":"2024-04-25T10:25:17.337659+0000","last_clean":"2024-04-25T10:25:17.337659+0000","last_became_active":"2024-04-25T10:25:17.334455+0000","last_became_peered":"2024-04-25T10:25:17.334455+0000","last_unstale":"2024-04-25T10:25:17.337659+0000","last_undegraded":"2024-04-25T10:25:17.337659+0000","last_fullsized":"2024-04-25T10:25:17.337659+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:52.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.326271+0000","last_change":"2024-04-25T10:25:17.325237+0000","last_active":"2024-04-25T10:25:17.326271+0000","last_peered":"2024-04-25T10:25:17.326271+0000","last_clean":"2024-04-25T10:25:17.326271+0000","last_became_active":"2024-04-25T10:25:17.324309+0000","last_became_peered":"2024-04-25T10:25:17.324309+0000","last_unstale":"2024-04-25T10:25:17.326271+0000","last_undegraded":"2024-04-25T10:25:17.326271+0000","last_fullsized":"2024-04-25T10:25:17.326271+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:01.383498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316873+0000","last_change":"2024-04-25T10:25:17.328951+0000","last_active":"2024-04-25T10:25:18.316873+0000","last_peered":"2024-04-25T10:25:18.316873+0000","last_clean":"2024-04-25T10:25:18.316873+0000","last_became_active":"2024-04-25T10:25:17.328711+0000","last_became_peered":"2024-04-25T10:25:17.328711+0000","last_unstale":"2024-04-25T10:25:18.316873+0000","last_undegraded":"2024-04-25T10:25:18.316873+0000","last_fullsized":"2024-04-25T10:25:18.316873+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.086283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.339935+0000","last_change":"2024-04-25T10:25:17.337124+0000","last_active":"2024-04-25T10:25:17.339935+0000","last_peered":"2024-04-25T10:25:17.339935+0000","last_clean":"2024-04-25T10:25:17.339935+0000","last_became_active":"2024-04-25T10:25:17.336323+0000","last_became_peered":"2024-04-25T10:25:17.336323+0000","last_unstale":"2024-04-25T10:25:17.339935+0000","last_undegraded":"2024-04-25T10:25:17.339935+0000","last_fullsized":"2024-04-25T10:25:17.339935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:44.663051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315761+0000","last_change":"2024-04-25T10:25:17.336196+0000","last_active":"2024-04-25T10:25:18.315761+0000","last_peered":"2024-04-25T10:25:18.315761+0000","last_clean":"2024-04-25T10:25:18.315761+0000","last_became_active":"2024-04-25T10:25:17.335900+0000","last_became_peered":"2024-04-25T10:25:17.335900+0000","last_unstale":"2024-04-25T10:25:18.315761+0000","last_undegraded":"2024-04-25T10:25:18.315761+0000","last_fullsized":"2024-04-25T10:25:18.315761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:39.657975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311485+0000","last_change":"2024-04-25T10:25:17.331578+0000","last_active":"2024-04-25T10:25:18.311485+0000","last_peered":"2024-04-25T10:25:18.311485+0000","last_clean":"2024-04-25T10:25:18.311485+0000","last_became_active":"2024-04-25T10:25:17.331086+0000","last_became_peered":"2024-04-25T10:25:17.331086+0000","last_unstale":"2024-04-25T10:25:18.311485+0000","last_undegraded":"2024-04-25T10:25:18.311485+0000","last_fullsized":"2024-04-25T10:25:18.311485+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:39.185655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338715+0000","last_change":"2024-04-25T10:25:17.328619+0000","last_active":"2024-04-25T10:25:17.338715+0000","last_peered":"2024-04-25T10:25:17.338715+0000","last_clean":"2024-04-25T10:25:17.338715+0000","last_became_active":"2024-04-25T10:25:17.328330+0000","last_became_peered":"2024-04-25T10:25:17.328330+0000","last_unstale":"2024-04-25T10:25:17.338715+0000","last_undegraded":"2024-04-25T10:25:17.338715+0000","last_fullsized":"2024-04-25T10:25:17.338715+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:39.857188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.331340+0000","last_change":"2024-04-25T10:25:17.322870+0000","last_active":"2024-04-25T10:25:17.331340+0000","last_peered":"2024-04-25T10:25:17.331340+0000","last_clean":"2024-04-25T10:25:17.331340+0000","last_became_active":"2024-04-25T10:25:17.322603+0000","last_became_peered":"2024-04-25T10:25:17.322603+0000","last_unstale":"2024-04-25T10:25:17.331340+0000","last_undegraded":"2024-04-25T10:25:17.331340+0000","last_fullsized":"2024-04-25T10:25:17.331340+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:12.033703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333864+0000","last_change":"2024-04-25T10:25:17.329470+0000","last_active":"2024-04-25T10:25:17.333864+0000","last_peered":"2024-04-25T10:25:17.333864+0000","last_clean":"2024-04-25T10:25:17.333864+0000","last_became_active":"2024-04-25T10:25:17.329158+0000","last_became_peered":"2024-04-25T10:25:17.329158+0000","last_unstale":"2024-04-25T10:25:17.333864+0000","last_undegraded":"2024-04-25T10:25:17.333864+0000","last_fullsized":"2024-04-25T10:25:17.333864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:43.585040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337526+0000","last_change":"2024-04-25T10:25:17.334866+0000","last_active":"2024-04-25T10:25:17.337526+0000","last_peered":"2024-04-25T10:25:17.337526+0000","last_clean":"2024-04-25T10:25:17.337526+0000","last_became_active":"2024-04-25T10:25:17.334598+0000","last_became_peered":"2024-04-25T10:25:17.334598+0000","last_unstale":"2024-04-25T10:25:17.337526+0000","last_undegraded":"2024-04-25T10:25:17.337526+0000","last_fullsized":"2024-04-25T10:25:17.337526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:49.470748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.336386+0000","last_change":"2024-04-25T10:25:17.334784+0000","last_active":"2024-04-25T10:25:17.336386+0000","last_peered":"2024-04-25T10:25:17.336386+0000","last_clean":"2024-04-25T10:25:17.336386+0000","last_became_active":"2024-04-25T10:25:17.333271+0000","last_became_peered":"2024-04-25T10:25:17.333271+0000","last_unstale":"2024-04-25T10:25:17.336386+0000","last_undegraded":"2024-04-25T10:25:17.336386+0000","last_fullsized":"2024-04-25T10:25:17.336386+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:18.095808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.340298+0000","last_change":"2024-04-25T10:25:17.339553+0000","last_active":"2024-04-25T10:25:17.340298+0000","last_peered":"2024-04-25T10:25:17.340298+0000","last_clean":"2024-04-25T10:25:17.340298+0000","last_became_active":"2024-04-25T10:25:17.338777+0000","last_became_peered":"2024-04-25T10:25:17.338777+0000","last_unstale":"2024-04-25T10:25:17.340298+0000","last_undegraded":"2024-04-25T10:25:17.340298+0000","last_fullsized":"2024-04-25T10:25:17.340298+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:44.669235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.330514+0000","last_change":"2024-04-25T10:25:17.326313+0000","last_active":"2024-04-25T10:25:17.330514+0000","last_peered":"2024-04-25T10:25:17.330514+0000","last_clean":"2024-04-25T10:25:17.330514+0000","last_became_active":"2024-04-25T10:25:17.326034+0000","last_became_peered":"2024-04-25T10:25:17.326034+0000","last_unstale":"2024-04-25T10:25:17.330514+0000","last_undegraded":"2024-04-25T10:25:17.330514+0000","last_fullsized":"2024-04-25T10:25:17.330514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:20.854606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337756+0000","last_change":"2024-04-25T10:25:17.330778+0000","last_active":"2024-04-25T10:25:17.337756+0000","last_peered":"2024-04-25T10:25:17.337756+0000","last_clean":"2024-04-25T10:25:17.337756+0000","last_became_active":"2024-04-25T10:25:17.329730+0000","last_became_peered":"2024-04-25T10:25:17.329730+0000","last_unstale":"2024-04-25T10:25:17.337756+0000","last_undegraded":"2024-04-25T10:25:17.337756+0000","last_fullsized":"2024-04-25T10:25:17.337756+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:28.279914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.334635+0000","last_change":"2024-04-25T10:25:17.329304+0000","last_active":"2024-04-25T10:25:17.334635+0000","last_peered":"2024-04-25T10:25:17.334635+0000","last_clean":"2024-04-25T10:25:17.334635+0000","last_became_active":"2024-04-25T10:25:17.328892+0000","last_became_peered":"2024-04-25T10:25:17.328892+0000","last_unstale":"2024-04-25T10:25:17.334635+0000","last_undegraded":"2024-04-25T10:25:17.334635+0000","last_fullsized":"2024-04-25T10:25:17.334635+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:12.755915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.334014+0000","last_change":"2024-04-25T10:25:17.326968+0000","last_active":"2024-04-25T10:25:17.334014+0000","last_peered":"2024-04-25T10:25:17.334014+0000","last_clean":"2024-04-25T10:25:17.334014+0000","last_became_active":"2024-04-25T10:25:17.326698+0000","last_became_peered":"2024-04-25T10:25:17.326698+0000","last_unstale":"2024-04-25T10:25:17.334014+0000","last_undegraded":"2024-04-25T10:25:17.334014+0000","last_fullsized":"2024-04-25T10:25:17.334014+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.298688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333915+0000","last_change":"2024-04-25T10:25:17.325291+0000","last_active":"2024-04-25T10:25:17.333915+0000","last_peered":"2024-04-25T10:25:17.333915+0000","last_clean":"2024-04-25T10:25:17.333915+0000","last_became_active":"2024-04-25T10:25:17.324722+0000","last_became_peered":"2024-04-25T10:25:17.324722+0000","last_unstale":"2024-04-25T10:25:17.333915+0000","last_undegraded":"2024-04-25T10:25:17.333915+0000","last_fullsized":"2024-04-25T10:25:17.333915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:40.388203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333907+0000","last_change":"2024-04-25T10:25:17.327182+0000","last_active":"2024-04-25T10:25:17.333907+0000","last_peered":"2024-04-25T10:25:17.333907+0000","last_clean":"2024-04-25T10:25:17.333907+0000","last_became_active":"2024-04-25T10:25:17.326923+0000","last_became_peered":"2024-04-25T10:25:17.326923+0000","last_unstale":"2024-04-25T10:25:17.333907+0000","last_undegraded":"2024-04-25T10:25:17.333907+0000","last_fullsized":"2024-04-25T10:25:17.333907+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:15.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316972+0000","last_change":"2024-04-25T10:25:17.328691+0000","last_active":"2024-04-25T10:25:18.316972+0000","last_peered":"2024-04-25T10:25:18.316972+0000","last_clean":"2024-04-25T10:25:18.316972+0000","last_became_active":"2024-04-25T10:25:17.328397+0000","last_became_peered":"2024-04-25T10:25:17.328397+0000","last_unstale":"2024-04-25T10:25:18.316972+0000","last_undegraded":"2024-04-25T10:25:18.316972+0000","last_fullsized":"2024-04-25T10:25:18.316972+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.921086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.347644+0000","last_change":"2024-04-25T10:25:16.319616+0000","last_active":"2024-04-25T10:25:17.347644+0000","last_peered":"2024-04-25T10:25:17.347644+0000","last_clean":"2024-04-25T10:25:17.347644+0000","last_became_active":"2024-04-25T10:25:16.319346+0000","last_became_peered":"2024-04-25T10:25:16.319346+0000","last_unstale":"2024-04-25T10:25:17.347644+0000","last_undegraded":"2024-04-25T10:25:17.347644+0000","last_fullsized":"2024-04-25T10:25:17.347644+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:16.152573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338899+0000","last_change":"2024-04-25T10:25:17.336908+0000","last_active":"2024-04-25T10:25:17.338899+0000","last_peered":"2024-04-25T10:25:17.338899+0000","last_clean":"2024-04-25T10:25:17.338899+0000","last_became_active":"2024-04-25T10:25:17.335590+0000","last_became_peered":"2024-04-25T10:25:17.335590+0000","last_unstale":"2024-04-25T10:25:17.338899+0000","last_undegraded":"2024-04-25T10:25:17.338899+0000","last_fullsized":"2024-04-25T10:25:17.338899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:03.906293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346128+0000","last_change":"2024-04-25T10:25:16.304038+0000","last_active":"2024-04-25T10:25:17.346128+0000","last_peered":"2024-04-25T10:25:17.346128+0000","last_clean":"2024-04-25T10:25:17.346128+0000","last_became_active":"2024-04-25T10:25:16.303742+0000","last_became_peered":"2024-04-25T10:25:16.303742+0000","last_unstale":"2024-04-25T10:25:17.346128+0000","last_undegraded":"2024-04-25T10:25:17.346128+0000","last_fullsized":"2024-04-25T10:25:17.346128+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:19.132756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333821+0000","last_change":"2024-04-25T10:25:17.327281+0000","last_active":"2024-04-25T10:25:17.333821+0000","last_peered":"2024-04-25T10:25:17.333821+0000","last_clean":"2024-04-25T10:25:17.333821+0000","last_became_active":"2024-04-25T10:25:17.326740+0000","last_became_peered":"2024-04-25T10:25:17.326740+0000","last_unstale":"2024-04-25T10:25:17.333821+0000","last_undegraded":"2024-04-25T10:25:17.333821+0000","last_fullsized":"2024-04-25T10:25:17.333821+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:18.095808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.345330+0000","last_change":"2024-04-25T10:25:16.323667+0000","last_active":"2024-04-25T10:25:17.345330+0000","last_peered":"2024-04-25T10:25:17.345330+0000","last_clean":"2024-04-25T10:25:17.345330+0000","last_became_active":"2024-04-25T10:25:16.323402+0000","last_became_peered":"2024-04-25T10:25:16.323402+0000","last_unstale":"2024-04-25T10:25:17.345330+0000","last_undegraded":"2024-04-25T10:25:17.345330+0000","last_fullsized":"2024-04-25T10:25:17.345330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:27.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.336393+0000","last_change":"2024-04-25T10:25:17.332748+0000","last_active":"2024-04-25T10:25:17.336393+0000","last_peered":"2024-04-25T10:25:17.336393+0000","last_clean":"2024-04-25T10:25:17.336393+0000","last_became_active":"2024-04-25T10:25:17.332376+0000","last_became_peered":"2024-04-25T10:25:17.332376+0000","last_unstale":"2024-04-25T10:25:17.336393+0000","last_undegraded":"2024-04-25T10:25:17.336393+0000","last_fullsized":"2024-04-25T10:25:17.336393+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:15.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.348127+0000","last_change":"2024-04-25T10:25:16.320132+0000","last_active":"2024-04-25T10:25:17.348127+0000","last_peered":"2024-04-25T10:25:17.348127+0000","last_clean":"2024-04-25T10:25:17.348127+0000","last_became_active":"2024-04-25T10:25:16.319848+0000","last_became_peered":"2024-04-25T10:25:16.319848+0000","last_unstale":"2024-04-25T10:25:17.348127+0000","last_undegraded":"2024-04-25T10:25:17.348127+0000","last_fullsized":"2024-04-25T10:25:17.348127+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:08.748315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311147+0000","last_change":"2024-04-25T10:25:17.318982+0000","last_active":"2024-04-25T10:25:18.311147+0000","last_peered":"2024-04-25T10:25:18.311147+0000","last_clean":"2024-04-25T10:25:18.311147+0000","last_became_active":"2024-04-25T10:25:17.318434+0000","last_became_peered":"2024-04-25T10:25:17.318434+0000","last_unstale":"2024-04-25T10:25:18.311147+0000","last_undegraded":"2024-04-25T10:25:18.311147+0000","last_fullsized":"2024-04-25T10:25:18.311147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:20.540499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.347442+0000","last_change":"2024-04-25T10:25:16.320289+0000","last_active":"2024-04-25T10:25:17.347442+0000","last_peered":"2024-04-25T10:25:17.347442+0000","last_clean":"2024-04-25T10:25:17.347442+0000","last_became_active":"2024-04-25T10:25:16.320014+0000","last_became_peered":"2024-04-25T10:25:16.320014+0000","last_unstale":"2024-04-25T10:25:17.347442+0000","last_undegraded":"2024-04-25T10:25:17.347442+0000","last_fullsized":"2024-04-25T10:25:17.347442+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.363040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.320299+0000","last_change":"2024-04-25T10:25:17.319275+0000","last_active":"2024-04-25T10:25:17.320299+0000","last_peered":"2024-04-25T10:25:17.320299+0000","last_clean":"2024-04-25T10:25:17.320299+0000","last_became_active":"2024-04-25T10:25:17.319009+0000","last_became_peered":"2024-04-25T10:25:17.319009+0000","last_unstale":"2024-04-25T10:25:17.320299+0000","last_undegraded":"2024-04-25T10:25:17.320299+0000","last_fullsized":"2024-04-25T10:25:17.320299+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:00.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.344750+0000","last_change":"2024-04-25T10:25:16.320146+0000","last_active":"2024-04-25T10:25:17.344750+0000","last_peered":"2024-04-25T10:25:17.344750+0000","last_clean":"2024-04-25T10:25:17.344750+0000","last_became_active":"2024-04-25T10:25:16.319848+0000","last_became_peered":"2024-04-25T10:25:16.319848+0000","last_unstale":"2024-04-25T10:25:17.344750+0000","last_undegraded":"2024-04-25T10:25:17.344750+0000","last_fullsized":"2024-04-25T10:25:17.344750+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:35.964647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337030+0000","last_change":"2024-04-25T10:25:17.335019+0000","last_active":"2024-04-25T10:25:17.337030+0000","last_peered":"2024-04-25T10:25:17.337030+0000","last_clean":"2024-04-25T10:25:17.337030+0000","last_became_active":"2024-04-25T10:25:17.334722+0000","last_became_peered":"2024-04-25T10:25:17.334722+0000","last_unstale":"2024-04-25T10:25:17.337030+0000","last_undegraded":"2024-04-25T10:25:17.337030+0000","last_fullsized":"2024-04-25T10:25:17.337030+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:38.899730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315468+0000","last_change":"2024-04-25T10:25:16.305938+0000","last_active":"2024-04-25T10:25:17.315468+0000","last_peered":"2024-04-25T10:25:17.315468+0000","last_clean":"2024-04-25T10:25:17.315468+0000","last_became_active":"2024-04-25T10:25:16.305631+0000","last_became_peered":"2024-04-25T10:25:16.305631+0000","last_unstale":"2024-04-25T10:25:17.315468+0000","last_undegraded":"2024-04-25T10:25:17.315468+0000","last_fullsized":"2024-04-25T10:25:17.315468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:31.120957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311082+0000","last_change":"2024-04-25T10:25:17.318938+0000","last_active":"2024-04-25T10:25:18.311082+0000","last_peered":"2024-04-25T10:25:18.311082+0000","last_clean":"2024-04-25T10:25:18.311082+0000","last_became_active":"2024-04-25T10:25:17.318621+0000","last_became_peered":"2024-04-25T10:25:17.318621+0000","last_unstale":"2024-04-25T10:25:18.311082+0000","last_undegraded":"2024-04-25T10:25:18.311082+0000","last_fullsized":"2024-04-25T10:25:18.311082+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:37.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315616+0000","last_change":"2024-04-25T10:25:16.323148+0000","last_active":"2024-04-25T10:25:18.315616+0000","last_peered":"2024-04-25T10:25:18.315616+0000","last_clean":"2024-04-25T10:25:18.315616+0000","last_became_active":"2024-04-25T10:25:16.322285+0000","last_became_peered":"2024-04-25T10:25:16.322285+0000","last_unstale":"2024-04-25T10:25:18.315616+0000","last_undegraded":"2024-04-25T10:25:18.315616+0000","last_fullsized":"2024-04-25T10:25:18.315616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:20.657052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.342350+0000","last_change":"2024-04-25T10:25:17.340096+0000","last_active":"2024-04-25T10:25:17.342350+0000","last_peered":"2024-04-25T10:25:17.342350+0000","last_clean":"2024-04-25T10:25:17.342350+0000","last_became_active":"2024-04-25T10:25:17.335792+0000","last_became_peered":"2024-04-25T10:25:17.335792+0000","last_unstale":"2024-04-25T10:25:17.342350+0000","last_undegraded":"2024-04-25T10:25:17.342350+0000","last_fullsized":"2024-04-25T10:25:17.342350+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:09.372882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.313996+0000","last_change":"2024-04-25T10:25:16.318782+0000","last_active":"2024-04-25T10:25:17.313996+0000","last_peered":"2024-04-25T10:25:17.313996+0000","last_clean":"2024-04-25T10:25:17.313996+0000","last_became_active":"2024-04-25T10:25:16.318450+0000","last_became_peered":"2024-04-25T10:25:16.318450+0000","last_unstale":"2024-04-25T10:25:17.313996+0000","last_undegraded":"2024-04-25T10:25:17.313996+0000","last_fullsized":"2024-04-25T10:25:17.313996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:28.869952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329867+0000","last_change":"2024-04-25T10:25:17.327183+0000","last_active":"2024-04-25T10:25:17.329867+0000","last_peered":"2024-04-25T10:25:17.329867+0000","last_clean":"2024-04-25T10:25:17.329867+0000","last_became_active":"2024-04-25T10:25:17.326912+0000","last_became_peered":"2024-04-25T10:25:17.326912+0000","last_unstale":"2024-04-25T10:25:17.329867+0000","last_undegraded":"2024-04-25T10:25:17.329867+0000","last_fullsized":"2024-04-25T10:25:17.329867+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:50.045084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.322708+0000","last_change":"2024-04-25T10:25:16.321087+0000","last_active":"2024-04-25T10:25:17.322708+0000","last_peered":"2024-04-25T10:25:17.322708+0000","last_clean":"2024-04-25T10:25:17.322708+0000","last_became_active":"2024-04-25T10:25:16.320801+0000","last_became_peered":"2024-04-25T10:25:16.320801+0000","last_unstale":"2024-04-25T10:25:17.322708+0000","last_undegraded":"2024-04-25T10:25:17.322708+0000","last_fullsized":"2024-04-25T10:25:17.322708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:38.175123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338448+0000","last_change":"2024-04-25T10:25:17.335665+0000","last_active":"2024-04-25T10:25:17.338448+0000","last_peered":"2024-04-25T10:25:17.338448+0000","last_clean":"2024-04-25T10:25:17.338448+0000","last_became_active":"2024-04-25T10:25:17.334863+0000","last_became_peered":"2024-04-25T10:25:17.334863+0000","last_unstale":"2024-04-25T10:25:17.338448+0000","last_undegraded":"2024-04-25T10:25:17.338448+0000","last_fullsized":"2024-04-25T10:25:17.338448+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:41.465106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346158+0000","last_change":"2024-04-25T10:25:16.313497+0000","last_active":"2024-04-25T10:25:17.346158+0000","last_peered":"2024-04-25T10:25:17.346158+0000","last_clean":"2024-04-25T10:25:17.346158+0000","last_became_active":"2024-04-25T10:25:16.313064+0000","last_became_peered":"2024-04-25T10:25:16.313064+0000","last_unstale":"2024-04-25T10:25:17.346158+0000","last_undegraded":"2024-04-25T10:25:17.346158+0000","last_fullsized":"2024-04-25T10:25:17.346158+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:56.918562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337493+0000","last_change":"2024-04-25T10:25:17.334281+0000","last_active":"2024-04-25T10:25:17.337493+0000","last_peered":"2024-04-25T10:25:17.337493+0000","last_clean":"2024-04-25T10:25:17.337493+0000","last_became_active":"2024-04-25T10:25:17.334014+0000","last_became_peered":"2024-04-25T10:25:17.334014+0000","last_unstale":"2024-04-25T10:25:17.337493+0000","last_undegraded":"2024-04-25T10:25:17.337493+0000","last_fullsized":"2024-04-25T10:25:17.337493+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:19.243975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311558+0000","last_change":"2024-04-25T10:25:16.313694+0000","last_active":"2024-04-25T10:25:18.311558+0000","last_peered":"2024-04-25T10:25:18.311558+0000","last_clean":"2024-04-25T10:25:18.311558+0000","last_became_active":"2024-04-25T10:25:16.313397+0000","last_became_peered":"2024-04-25T10:25:16.313397+0000","last_unstale":"2024-04-25T10:25:18.311558+0000","last_undegraded":"2024-04-25T10:25:18.311558+0000","last_fullsized":"2024-04-25T10:25:18.311558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:44.772974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335159+0000","last_change":"2024-04-25T10:25:17.333171+0000","last_active":"2024-04-25T10:25:17.335159+0000","last_peered":"2024-04-25T10:25:17.335159+0000","last_clean":"2024-04-25T10:25:17.335159+0000","last_became_active":"2024-04-25T10:25:17.332219+0000","last_became_peered":"2024-04-25T10:25:17.332219+0000","last_unstale":"2024-04-25T10:25:17.335159+0000","last_undegraded":"2024-04-25T10:25:17.335159+0000","last_fullsized":"2024-04-25T10:25:17.335159+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.183487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315985+0000","last_change":"2024-04-25T10:25:16.307483+0000","last_active":"2024-04-25T10:25:18.315985+0000","last_peered":"2024-04-25T10:25:18.315985+0000","last_clean":"2024-04-25T10:25:18.315985+0000","last_became_active":"2024-04-25T10:25:16.307160+0000","last_became_peered":"2024-04-25T10:25:16.307160+0000","last_unstale":"2024-04-25T10:25:18.315985+0000","last_undegraded":"2024-04-25T10:25:18.315985+0000","last_fullsized":"2024-04-25T10:25:18.315985+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:19.844075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338835+0000","last_change":"2024-04-25T10:25:17.334693+0000","last_active":"2024-04-25T10:25:17.338835+0000","last_peered":"2024-04-25T10:25:17.338835+0000","last_clean":"2024-04-25T10:25:17.338835+0000","last_became_active":"2024-04-25T10:25:17.329717+0000","last_became_peered":"2024-04-25T10:25:17.329717+0000","last_unstale":"2024-04-25T10:25:17.338835+0000","last_undegraded":"2024-04-25T10:25:17.338835+0000","last_fullsized":"2024-04-25T10:25:17.338835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:03.223023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.319934+0000","last_change":"2024-04-25T10:25:16.320139+0000","last_active":"2024-04-25T10:25:17.319934+0000","last_peered":"2024-04-25T10:25:17.319934+0000","last_clean":"2024-04-25T10:25:17.319934+0000","last_became_active":"2024-04-25T10:25:16.319639+0000","last_became_peered":"2024-04-25T10:25:16.319639+0000","last_unstale":"2024-04-25T10:25:17.319934+0000","last_undegraded":"2024-04-25T10:25:17.319934+0000","last_fullsized":"2024-04-25T10:25:17.319934+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:36.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.322970+0000","last_change":"2024-04-25T10:25:17.320968+0000","last_active":"2024-04-25T10:25:17.322970+0000","last_peered":"2024-04-25T10:25:17.322970+0000","last_clean":"2024-04-25T10:25:17.322970+0000","last_became_active":"2024-04-25T10:25:17.318899+0000","last_became_peered":"2024-04-25T10:25:17.318899+0000","last_unstale":"2024-04-25T10:25:17.322970+0000","last_undegraded":"2024-04-25T10:25:17.322970+0000","last_fullsized":"2024-04-25T10:25:17.322970+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:05.310522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311124+0000","last_change":"2024-04-25T10:25:16.318497+0000","last_active":"2024-04-25T10:25:18.311124+0000","last_peered":"2024-04-25T10:25:18.311124+0000","last_clean":"2024-04-25T10:25:18.311124+0000","last_became_active":"2024-04-25T10:25:16.318225+0000","last_became_peered":"2024-04-25T10:25:16.318225+0000","last_unstale":"2024-04-25T10:25:18.311124+0000","last_undegraded":"2024-04-25T10:25:18.311124+0000","last_fullsized":"2024-04-25T10:25:18.311124+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:16.152573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333938+0000","last_change":"2024-04-25T10:25:17.325542+0000","last_active":"2024-04-25T10:25:17.333938+0000","last_peered":"2024-04-25T10:25:17.333938+0000","last_clean":"2024-04-25T10:25:17.333938+0000","last_became_active":"2024-04-25T10:25:17.325047+0000","last_became_peered":"2024-04-25T10:25:17.325047+0000","last_unstale":"2024-04-25T10:25:17.333938+0000","last_undegraded":"2024-04-25T10:25:17.333938+0000","last_fullsized":"2024-04-25T10:25:17.333938+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:50.280414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.350031+0000","last_change":"2024-04-25T10:25:16.322326+0000","last_active":"2024-04-25T10:25:17.350031+0000","last_peered":"2024-04-25T10:25:17.350031+0000","last_clean":"2024-04-25T10:25:17.350031+0000","last_became_active":"2024-04-25T10:25:16.321967+0000","last_became_peered":"2024-04-25T10:25:16.321967+0000","last_unstale":"2024-04-25T10:25:17.350031+0000","last_undegraded":"2024-04-25T10:25:17.350031+0000","last_fullsized":"2024-04-25T10:25:17.350031+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:19.529968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329419+0000","last_change":"2024-04-25T10:25:17.326600+0000","last_active":"2024-04-25T10:25:17.329419+0000","last_peered":"2024-04-25T10:25:17.329419+0000","last_clean":"2024-04-25T10:25:17.329419+0000","last_became_active":"2024-04-25T10:25:17.325400+0000","last_became_peered":"2024-04-25T10:25:17.325400+0000","last_unstale":"2024-04-25T10:25:17.329419+0000","last_undegraded":"2024-04-25T10:25:17.329419+0000","last_fullsized":"2024-04-25T10:25:17.329419+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.717156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311481+0000","last_change":"2024-04-25T10:25:16.316378+0000","last_active":"2024-04-25T10:25:18.311481+0000","last_peered":"2024-04-25T10:25:18.311481+0000","last_clean":"2024-04-25T10:25:18.311481+0000","last_became_active":"2024-04-25T10:25:16.315414+0000","last_became_peered":"2024-04-25T10:25:16.315414+0000","last_unstale":"2024-04-25T10:25:18.311481+0000","last_undegraded":"2024-04-25T10:25:18.311481+0000","last_fullsized":"2024-04-25T10:25:18.311481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:00.696127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329797+0000","last_change":"2024-04-25T10:25:17.327310+0000","last_active":"2024-04-25T10:25:17.329797+0000","last_peered":"2024-04-25T10:25:17.329797+0000","last_clean":"2024-04-25T10:25:17.329797+0000","last_became_active":"2024-04-25T10:25:17.322279+0000","last_became_peered":"2024-04-25T10:25:17.322279+0000","last_unstale":"2024-04-25T10:25:17.329797+0000","last_undegraded":"2024-04-25T10:25:17.329797+0000","last_fullsized":"2024-04-25T10:25:17.329797+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:41.678714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346022+0000","last_change":"2024-04-25T10:25:16.318068+0000","last_active":"2024-04-25T10:25:17.346022+0000","last_peered":"2024-04-25T10:25:17.346022+0000","last_clean":"2024-04-25T10:25:17.346022+0000","last_became_active":"2024-04-25T10:25:16.316424+0000","last_became_peered":"2024-04-25T10:25:16.316424+0000","last_unstale":"2024-04-25T10:25:17.346022+0000","last_undegraded":"2024-04-25T10:25:17.346022+0000","last_fullsized":"2024-04-25T10:25:17.346022+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:31.992394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311800+0000","last_change":"2024-04-25T10:25:17.323925+0000","last_active":"2024-04-25T10:25:18.311800+0000","last_peered":"2024-04-25T10:25:18.311800+0000","last_clean":"2024-04-25T10:25:18.311800+0000","last_became_active":"2024-04-25T10:25:17.323658+0000","last_became_peered":"2024-04-25T10:25:17.323658+0000","last_unstale":"2024-04-25T10:25:18.311800+0000","last_undegraded":"2024-04-25T10:25:18.311800+0000","last_fullsized":"2024-04-25T10:25:18.311800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:15.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315616+0000","last_change":"2024-04-25T10:25:16.308940+0000","last_active":"2024-04-25T10:25:18.315616+0000","last_peered":"2024-04-25T10:25:18.315616+0000","last_clean":"2024-04-25T10:25:18.315616+0000","last_became_active":"2024-04-25T10:25:16.306608+0000","last_became_peered":"2024-04-25T10:25:16.306608+0000","last_unstale":"2024-04-25T10:25:18.315616+0000","last_undegraded":"2024-04-25T10:25:18.315616+0000","last_fullsized":"2024-04-25T10:25:18.315616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:10.092046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316450+0000","last_change":"2024-04-25T10:25:17.324978+0000","last_active":"2024-04-25T10:25:18.316450+0000","last_peered":"2024-04-25T10:25:18.316450+0000","last_clean":"2024-04-25T10:25:18.316450+0000","last_became_active":"2024-04-25T10:25:17.324708+0000","last_became_peered":"2024-04-25T10:25:17.324708+0000","last_unstale":"2024-04-25T10:25:18.316450+0000","last_undegraded":"2024-04-25T10:25:18.316450+0000","last_fullsized":"2024-04-25T10:25:18.316450+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:02.232621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.347671+0000","last_change":"2024-04-25T10:25:16.320860+0000","last_active":"2024-04-25T10:25:17.347671+0000","last_peered":"2024-04-25T10:25:17.347671+0000","last_clean":"2024-04-25T10:25:17.347671+0000","last_became_active":"2024-04-25T10:25:16.320590+0000","last_became_peered":"2024-04-25T10:25:16.320590+0000","last_unstale":"2024-04-25T10:25:17.347671+0000","last_undegraded":"2024-04-25T10:25:17.347671+0000","last_fullsized":"2024-04-25T10:25:17.347671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:52.846423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315875+0000","last_change":"2024-04-25T10:25:17.331982+0000","last_active":"2024-04-25T10:25:18.315875+0000","last_peered":"2024-04-25T10:25:18.315875+0000","last_clean":"2024-04-25T10:25:18.315875+0000","last_became_active":"2024-04-25T10:25:17.331671+0000","last_became_peered":"2024-04-25T10:25:17.331671+0000","last_unstale":"2024-04-25T10:25:18.315875+0000","last_undegraded":"2024-04-25T10:25:18.315875+0000","last_fullsized":"2024-04-25T10:25:18.315875+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:21.860431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.321533+0000","last_change":"2024-04-25T10:25:16.317019+0000","last_active":"2024-04-25T10:25:17.321533+0000","last_peered":"2024-04-25T10:25:17.321533+0000","last_clean":"2024-04-25T10:25:17.321533+0000","last_became_active":"2024-04-25T10:25:16.316561+0000","last_became_peered":"2024-04-25T10:25:16.316561+0000","last_unstale":"2024-04-25T10:25:17.321533+0000","last_undegraded":"2024-04-25T10:25:17.321533+0000","last_fullsized":"2024-04-25T10:25:17.321533+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:57.236059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337759+0000","last_change":"2024-04-25T10:25:17.335378+0000","last_active":"2024-04-25T10:25:17.337759+0000","last_peered":"2024-04-25T10:25:17.337759+0000","last_clean":"2024-04-25T10:25:17.337759+0000","last_became_active":"2024-04-25T10:25:17.335110+0000","last_became_peered":"2024-04-25T10:25:17.335110+0000","last_unstale":"2024-04-25T10:25:17.337759+0000","last_undegraded":"2024-04-25T10:25:17.337759+0000","last_fullsized":"2024-04-25T10:25:17.337759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:03.954804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.321329+0000","last_change":"2024-04-25T10:25:16.317460+0000","last_active":"2024-04-25T10:25:17.321329+0000","last_peered":"2024-04-25T10:25:17.321329+0000","last_clean":"2024-04-25T10:25:17.321329+0000","last_became_active":"2024-04-25T10:25:16.315646+0000","last_became_peered":"2024-04-25T10:25:16.315646+0000","last_unstale":"2024-04-25T10:25:17.321329+0000","last_undegraded":"2024-04-25T10:25:17.321329+0000","last_fullsized":"2024-04-25T10:25:17.321329+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:38.175123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337556+0000","last_change":"2024-04-25T10:25:17.332074+0000","last_active":"2024-04-25T10:25:17.337556+0000","last_peered":"2024-04-25T10:25:17.337556+0000","last_clean":"2024-04-25T10:25:17.337556+0000","last_became_active":"2024-04-25T10:25:17.331819+0000","last_became_peered":"2024-04-25T10:25:17.331819+0000","last_unstale":"2024-04-25T10:25:17.337556+0000","last_undegraded":"2024-04-25T10:25:17.337556+0000","last_fullsized":"2024-04-25T10:25:17.337556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:43.126962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315023+0000","last_change":"2024-04-25T10:25:16.315905+0000","last_active":"2024-04-25T10:25:17.315023+0000","last_peered":"2024-04-25T10:25:17.315023+0000","last_clean":"2024-04-25T10:25:17.315023+0000","last_became_active":"2024-04-25T10:25:16.315165+0000","last_became_peered":"2024-04-25T10:25:16.315165+0000","last_unstale":"2024-04-25T10:25:17.315023+0000","last_undegraded":"2024-04-25T10:25:17.315023+0000","last_fullsized":"2024-04-25T10:25:17.315023+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:22.650760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.317121+0000","last_change":"2024-04-25T10:25:17.326354+0000","last_active":"2024-04-25T10:25:18.317121+0000","last_peered":"2024-04-25T10:25:18.317121+0000","last_clean":"2024-04-25T10:25:18.317121+0000","last_became_active":"2024-04-25T10:25:17.325977+0000","last_became_peered":"2024-04-25T10:25:17.325977+0000","last_unstale":"2024-04-25T10:25:18.317121+0000","last_undegraded":"2024-04-25T10:25:18.317121+0000","last_fullsized":"2024-04-25T10:25:18.317121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:31.628621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311632+0000","last_change":"2024-04-25T10:25:16.320302+0000","last_active":"2024-04-25T10:25:18.311632+0000","last_peered":"2024-04-25T10:25:18.311632+0000","last_clean":"2024-04-25T10:25:18.311632+0000","last_became_active":"2024-04-25T10:25:16.319851+0000","last_became_peered":"2024-04-25T10:25:16.319851+0000","last_unstale":"2024-04-25T10:25:18.311632+0000","last_undegraded":"2024-04-25T10:25:18.311632+0000","last_fullsized":"2024-04-25T10:25:18.311632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.745384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316962+0000","last_change":"2024-04-25T10:25:17.335191+0000","last_active":"2024-04-25T10:25:18.316962+0000","last_peered":"2024-04-25T10:25:18.316962+0000","last_clean":"2024-04-25T10:25:18.316962+0000","last_became_active":"2024-04-25T10:25:17.334226+0000","last_became_peered":"2024-04-25T10:25:17.334226+0000","last_unstale":"2024-04-25T10:25:18.316962+0000","last_undegraded":"2024-04-25T10:25:18.316962+0000","last_fullsized":"2024-04-25T10:25:18.316962+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:49.895711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.343042+0000","last_change":"2024-04-25T10:25:16.318770+0000","last_active":"2024-04-25T10:25:18.343042+0000","last_peered":"2024-04-25T10:25:18.343042+0000","last_clean":"2024-04-25T10:25:18.343042+0000","last_became_active":"2024-04-25T10:25:16.318503+0000","last_became_peered":"2024-04-25T10:25:16.318503+0000","last_unstale":"2024-04-25T10:25:18.343042+0000","last_undegraded":"2024-04-25T10:25:18.343042+0000","last_fullsized":"2024-04-25T10:25:18.343042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.034553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.331194+0000","last_change":"2024-04-25T10:25:17.325010+0000","last_active":"2024-04-25T10:25:17.331194+0000","last_peered":"2024-04-25T10:25:17.331194+0000","last_clean":"2024-04-25T10:25:17.331194+0000","last_became_active":"2024-04-25T10:25:17.324685+0000","last_became_peered":"2024-04-25T10:25:17.324685+0000","last_unstale":"2024-04-25T10:25:17.331194+0000","last_undegraded":"2024-04-25T10:25:17.331194+0000","last_fullsized":"2024-04-25T10:25:17.331194+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:17.262400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316721+0000","last_change":"2024-04-25T10:25:16.315435+0000","last_active":"2024-04-25T10:25:18.316721+0000","last_peered":"2024-04-25T10:25:18.316721+0000","last_clean":"2024-04-25T10:25:18.316721+0000","last_became_active":"2024-04-25T10:25:16.315149+0000","last_became_peered":"2024-04-25T10:25:16.315149+0000","last_unstale":"2024-04-25T10:25:18.316721+0000","last_undegraded":"2024-04-25T10:25:18.316721+0000","last_fullsized":"2024-04-25T10:25:18.316721+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:57.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.332819+0000","last_change":"2024-04-25T10:25:17.330232+0000","last_active":"2024-04-25T10:25:17.332819+0000","last_peered":"2024-04-25T10:25:17.332819+0000","last_clean":"2024-04-25T10:25:17.332819+0000","last_became_active":"2024-04-25T10:25:17.329884+0000","last_became_peered":"2024-04-25T10:25:17.329884+0000","last_unstale":"2024-04-25T10:25:17.332819+0000","last_undegraded":"2024-04-25T10:25:17.332819+0000","last_fullsized":"2024-04-25T10:25:17.332819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:00.936902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.323096+0000","last_change":"2024-04-25T10:25:16.320521+0000","last_active":"2024-04-25T10:25:17.323096+0000","last_peered":"2024-04-25T10:25:17.323096+0000","last_clean":"2024-04-25T10:25:17.323096+0000","last_became_active":"2024-04-25T10:25:16.320102+0000","last_became_peered":"2024-04-25T10:25:16.320102+0000","last_unstale":"2024-04-25T10:25:17.323096+0000","last_undegraded":"2024-04-25T10:25:17.323096+0000","last_fullsized":"2024-04-25T10:25:17.323096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:20.657052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316797+0000","last_change":"2024-04-25T10:25:17.325481+0000","last_active":"2024-04-25T10:25:18.316797+0000","last_peered":"2024-04-25T10:25:18.316797+0000","last_clean":"2024-04-25T10:25:18.316797+0000","last_became_active":"2024-04-25T10:25:17.325202+0000","last_became_peered":"2024-04-25T10:25:17.325202+0000","last_unstale":"2024-04-25T10:25:18.316797+0000","last_undegraded":"2024-04-25T10:25:18.316797+0000","last_fullsized":"2024-04-25T10:25:18.316797+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:56.549749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315868+0000","last_change":"2024-04-25T10:25:16.323722+0000","last_active":"2024-04-25T10:25:17.315868+0000","last_peered":"2024-04-25T10:25:17.315868+0000","last_clean":"2024-04-25T10:25:17.315868+0000","last_became_active":"2024-04-25T10:25:16.323243+0000","last_became_peered":"2024-04-25T10:25:16.323243+0000","last_unstale":"2024-04-25T10:25:17.315868+0000","last_undegraded":"2024-04-25T10:25:17.315868+0000","last_fullsized":"2024-04-25T10:25:17.315868+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:42.451132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335733+0000","last_change":"2024-04-25T10:25:17.334306+0000","last_active":"2024-04-25T10:25:17.335733+0000","last_peered":"2024-04-25T10:25:17.335733+0000","last_clean":"2024-04-25T10:25:17.335733+0000","last_became_active":"2024-04-25T10:25:17.333999+0000","last_became_peered":"2024-04-25T10:25:17.333999+0000","last_unstale":"2024-04-25T10:25:17.335733+0000","last_undegraded":"2024-04-25T10:25:17.335733+0000","last_fullsized":"2024-04-25T10:25:17.335733+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:06.505620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.322168+0000","last_change":"2024-04-25T10:25:16.313109+0000","last_active":"2024-04-25T10:25:17.322168+0000","last_peered":"2024-04-25T10:25:17.322168+0000","last_clean":"2024-04-25T10:25:17.322168+0000","last_became_active":"2024-04-25T10:25:16.312845+0000","last_became_peered":"2024-04-25T10:25:16.312845+0000","last_unstale":"2024-04-25T10:25:17.322168+0000","last_undegraded":"2024-04-25T10:25:17.322168+0000","last_fullsized":"2024-04-25T10:25:17.322168+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:10.092046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315509+0000","last_change":"2024-04-25T10:25:17.320561+0000","last_active":"2024-04-25T10:25:18.315509+0000","last_peered":"2024-04-25T10:25:18.315509+0000","last_clean":"2024-04-25T10:25:18.315509+0000","last_became_active":"2024-04-25T10:25:17.318491+0000","last_became_peered":"2024-04-25T10:25:17.318491+0000","last_unstale":"2024-04-25T10:25:18.315509+0000","last_undegraded":"2024-04-25T10:25:18.315509+0000","last_fullsized":"2024-04-25T10:25:18.315509+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:14.515182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.321333+0000","last_change":"2024-04-25T10:25:16.316870+0000","last_active":"2024-04-25T10:25:17.321333+0000","last_peered":"2024-04-25T10:25:17.321333+0000","last_clean":"2024-04-25T10:25:17.321333+0000","last_became_active":"2024-04-25T10:25:16.316543+0000","last_became_peered":"2024-04-25T10:25:16.316543+0000","last_unstale":"2024-04-25T10:25:17.321333+0000","last_undegraded":"2024-04-25T10:25:17.321333+0000","last_fullsized":"2024-04-25T10:25:17.321333+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:00.696127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311060+0000","last_change":"2024-04-25T10:25:17.327417+0000","last_active":"2024-04-25T10:25:18.311060+0000","last_peered":"2024-04-25T10:25:18.311060+0000","last_clean":"2024-04-25T10:25:18.311060+0000","last_became_active":"2024-04-25T10:25:17.327126+0000","last_became_peered":"2024-04-25T10:25:17.327126+0000","last_unstale":"2024-04-25T10:25:18.311060+0000","last_undegraded":"2024-04-25T10:25:18.311060+0000","last_fullsized":"2024-04-25T10:25:18.311060+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:53.856954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315042+0000","last_change":"2024-04-25T10:25:16.315447+0000","last_active":"2024-04-25T10:25:17.315042+0000","last_peered":"2024-04-25T10:25:17.315042+0000","last_clean":"2024-04-25T10:25:17.315042+0000","last_became_active":"2024-04-25T10:25:16.314834+0000","last_became_peered":"2024-04-25T10:25:16.314834+0000","last_unstale":"2024-04-25T10:25:17.315042+0000","last_undegraded":"2024-04-25T10:25:17.315042+0000","last_fullsized":"2024-04-25T10:25:17.315042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:26.048826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.341256+0000","last_change":"2024-04-25T10:25:17.338235+0000","last_active":"2024-04-25T10:25:17.341256+0000","last_peered":"2024-04-25T10:25:17.341256+0000","last_clean":"2024-04-25T10:25:17.341256+0000","last_became_active":"2024-04-25T10:25:17.337852+0000","last_became_peered":"2024-04-25T10:25:17.337852+0000","last_unstale":"2024-04-25T10:25:17.341256+0000","last_undegraded":"2024-04-25T10:25:17.341256+0000","last_fullsized":"2024-04-25T10:25:17.341256+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:35.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311692+0000","last_change":"2024-04-25T10:25:16.314529+0000","last_active":"2024-04-25T10:25:18.311692+0000","last_peered":"2024-04-25T10:25:18.311692+0000","last_clean":"2024-04-25T10:25:18.311692+0000","last_became_active":"2024-04-25T10:25:16.314235+0000","last_became_peered":"2024-04-25T10:25:16.314235+0000","last_unstale":"2024-04-25T10:25:18.311692+0000","last_undegraded":"2024-04-25T10:25:18.311692+0000","last_fullsized":"2024-04-25T10:25:18.311692+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:06.288157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.310963+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:25:18.310963+0000","last_peered":"2024-04-25T10:25:18.310963+0000","last_clean":"2024-04-25T10:25:18.310963+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:25:18.310963+0000","last_undegraded":"2024-04-25T10:25:18.310963+0000","last_fullsized":"2024-04-25T10:25:18.310963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509533,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46312,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325528,"statfs":{"total":96636764160,"available":96589340672,"internally_reserved":0,"allocated":1269760,"data_stored":546333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509534,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50464,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321376,"statfs":{"total":96636764160,"available":96585089024,"internally_reserved":0,"allocated":1265664,"data_stored":545751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55240,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316600,"statfs":{"total":96636764160,"available":96580198400,"internally_reserved":0,"allocated":1892352,"data_stored":1144087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54640,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317200,"statfs":{"total":96636764160,"available":96580812800,"internally_reserved":0,"allocated":1277952,"data_stored":545945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58796,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313044,"statfs":{"total":96636764160,"available":96576557056,"internally_reserved":0,"allocated":1269760,"data_stored":545821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509534,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":1892352,"data_stored":1144143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":545751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54632,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317208,"statfs":{"total":96636764160,"available":96580820992,"internally_reserved":0,"allocated":1269760,"data_stored":546333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:25:21.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:25:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b02400640 1 Processor -- start 2024-04-25T10:25:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b02400640 1 -- start start 2024-04-25T10:25:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b02400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 0x7f7afc15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b02400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc0b5200 con 0x7f7afc0cf9b0 2024-04-25T10:25:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b02400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc0b38b0 con 0x7f7afc071120 2024-04-25T10:25:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b02400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc15ef10 con 0x7f7afc0b3360 2024-04-25T10:25:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7afc0cf9b0 0x7f7afc070a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50514/0 (socket says 172.21.15.45:50514) 2024-04-25T10:25:21.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 -- 172.21.15.45:0/2837788747 learned_addr learned my addr 172.21.15.45:0/2837788747 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01a00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 0x7f7afc15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690187815 0 0) 0x7f7afc0b5200 con 0x7f7afc0cf9b0 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ae0003530 con 0x7f7afc0cf9b0 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1960202 0 0) 0x7f7ae0003530 con 0x7f7afc0cf9b0 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 msgr2=0x7f7afc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 0x7f7afc15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 >> v1:172.21.15.117:6789/0 conn(0x7f7afc071120 legacy=0x7f7afc1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0b3b90 con 0x7f7afc0cf9b0 2024-04-25T10:25:21.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f7aec0016a0 con 0x7f7afc0cf9b0 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 >> v1:172.21.15.45:6789/0 conn(0x7f7afc0cf9b0 legacy=0x7f7afc070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7ae0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae0004810 0x7f7ae0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 0x7f7ae00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc15ef10 con 0x7f7ae00039f0 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae000ba80 con 0x7f7ae0004810 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae000bc80 con 0x7f7afc0b3360 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7ae0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7af3e00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae0004810 0x7f7ae0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01a00640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 0x7f7ae00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 -- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 msgr2=0x7f7ae00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc0b3360 0x7f7ae00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 -- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae0004810 msgr2=0x7f7ae0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae0004810 0x7f7ae0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.616+0000 7f7b01000640 1 -- 172.21.15.45:0/2837788747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0003530 con 0x7f7ae00039f0 2024-04-25T10:25:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b01000640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7ae0003e10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7afc15bb00 tx=0x7f7aec01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908fa4c3cad84c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:21.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec007d40 con 0x7f7ae00039f0 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec0a9070 con 0x7f7ae00039f0 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7afbe00640 1 -- 172.21.15.45:0/2837788747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005d40 con 0x7f7ae00039f0 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b02400640 1 -- 172.21.15.45:0/2837788747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 msgr2=0x7f7ae0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b02400640 1 --2- 172.21.15.45:0/2837788747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7ae0003e10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7afc15bb00 tx=0x7f7aec01cc20 comp rx=0 tx=0).stop 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b01a00640 1 -- 172.21.15.45:0/2837788747 reap_dead start 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b02400640 1 -- 172.21.15.45:0/2837788747 shutdown_connections 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b02400640 1 -- 172.21.15.45:0/2837788747 >> 172.21.15.45:0/2837788747 conn(0x7f7afc0cfe80 msgr2=0x7f7afc0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b02400640 1 -- 172.21.15.45:0/2837788747 shutdown_connections 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.620+0000 7f7b02400640 1 -- 172.21.15.45:0/2837788747 wait complete. 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 Processor -- start 2024-04-25T10:25:21.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 -- start start 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7afc13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc06e560 0x7f7afc158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7afc15f0d0 0x7f7afc15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc169cc0 con 0x7f7ae00039f0 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc169b40 con 0x7f7afc15f0d0 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc169e40 con 0x7f7afc06e560 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7afc13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7afc13e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52054/0 (socket says 172.21.15.45:52054) 2024-04-25T10:25:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 -- 172.21.15.45:0/2596602757 learned_addr learned my addr 172.21.15.45:0/2596602757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01a00640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7afc15f0d0 0x7f7afc15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7af3e00640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc06e560 0x7f7afc158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 -- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc06e560 msgr2=0x7f7afc158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc06e560 0x7f7afc158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 -- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7afc15f0d0 msgr2=0x7f7afc15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7afc15f0d0 0x7f7afc15f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 -- 172.21.15.45:0/2596602757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc1598a0 con 0x7f7ae00039f0 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01a00640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7afc15f0d0 0x7f7afc15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b01000640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7afc13e8b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7aec01c690 tx=0x7f7ae0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005d20 con 0x7f7ae00039f0 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec001460 con 0x7f7ae00039f0 2024-04-25T10:25:21.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0a5810 con 0x7f7ae00039f0 2024-04-25T10:25:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc159b80 con 0x7f7ae00039f0 2024-04-25T10:25:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.624+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc15a140 con 0x7f7ae00039f0 2024-04-25T10:25:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.628+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc075120 con 0x7f7ae00039f0 2024-04-25T10:25:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.628+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7aec0a59b0 con 0x7f7ae00039f0 2024-04-25T10:25:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.628+0000 7f7af9600640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ad8066f60 0x7f7ad8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.628+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aec11e7d0 con 0x7f7ae00039f0 2024-04-25T10:25:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.628+0000 7f7af3e00640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ad8066f60 0x7f7ad8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.628+0000 7f7af3e00640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ad8066f60 0x7f7ad8069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7afc160c50 tx=0x7f7ae4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:21.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.632+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7aec016930 con 0x7f7ae00039f0 2024-04-25T10:25:21.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.792+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7afc0b2f30 con 0x7f7ad8066f60 2024-04-25T10:25:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.800+0000 7f7af9600640 1 -- 172.21.15.45:0/2596602757 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252430 (secure 0 0 0) 0x7f7afc0b2f30 con 0x7f7ad8066f60 2024-04-25T10:25:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:21.804 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:25:21.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ad8066f60 msgr2=0x7f7ad8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ad8066f60 0x7f7ad8069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7afc160c50 tx=0x7f7ae4009040 comp rx=0 tx=0).stop 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 msgr2=0x7f7afc13e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7afc13e8b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7aec01c690 tx=0x7f7ae0005840 comp rx=0 tx=0).stop 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 shutdown_connections 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ad8066f60 0x7f7ad8069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7afc15f0d0 0x7f7afc15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7afc06e560 0x7f7afc158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 --2- 172.21.15.45:0/2596602757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae00039f0 0x7f7afc13e8b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 >> 172.21.15.45:0/2596602757 conn(0x7f7afc0cfe80 msgr2=0x7f7afc15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 shutdown_connections 2024-04-25T10:25:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:21.804+0000 7f7b02400640 1 -- 172.21.15.45:0/2596602757 wait complete. 2024-04-25T10:25:21.824 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T10:25:20.976471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":403580,"kb_used_data":11136,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754571140,"statfs":{"total":773094113280,"available":772680847360,"internally_reserved":0,"allocated":11403264,"data_stored":5564164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682954"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333903+0000","last_change":"2024-04-25T10:25:17.329721+0000","last_active":"2024-04-25T10:25:17.333903+0000","last_peered":"2024-04-25T10:25:17.333903+0000","last_clean":"2024-04-25T10:25:17.333903+0000","last_became_active":"2024-04-25T10:25:17.326581+0000","last_became_peered":"2024-04-25T10:25:17.326581+0000","last_unstale":"2024-04-25T10:25:17.333903+0000","last_undegraded":"2024-04-25T10:25:17.333903+0000","last_fullsized":"2024-04-25T10:25:17.333903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:51.845697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.342328+0000","last_change":"2024-04-25T10:25:17.339795+0000","last_active":"2024-04-25T10:25:17.342328+0000","last_peered":"2024-04-25T10:25:17.342328+0000","last_clean":"2024-04-25T10:25:17.342328+0000","last_became_active":"2024-04-25T10:25:17.336746+0000","last_became_peered":"2024-04-25T10:25:17.336746+0000","last_unstale":"2024-04-25T10:25:17.342328+0000","last_undegraded":"2024-04-25T10:25:17.342328+0000","last_fullsized":"2024-04-25T10:25:17.342328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:49.834031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.331309+0000","last_change":"2024-04-25T10:25:17.324366+0000","last_active":"2024-04-25T10:25:17.331309+0000","last_peered":"2024-04-25T10:25:17.331309+0000","last_clean":"2024-04-25T10:25:17.331309+0000","last_became_active":"2024-04-25T10:25:17.324035+0000","last_became_peered":"2024-04-25T10:25:17.324035+0000","last_unstale":"2024-04-25T10:25:17.331309+0000","last_undegraded":"2024-04-25T10:25:17.331309+0000","last_fullsized":"2024-04-25T10:25:17.331309+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:58.594993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346165+0000","last_change":"2024-04-25T10:25:17.342861+0000","last_active":"2024-04-25T10:25:17.346165+0000","last_peered":"2024-04-25T10:25:17.346165+0000","last_clean":"2024-04-25T10:25:17.346165+0000","last_became_active":"2024-04-25T10:25:17.341100+0000","last_became_peered":"2024-04-25T10:25:17.341100+0000","last_unstale":"2024-04-25T10:25:17.346165+0000","last_undegraded":"2024-04-25T10:25:17.346165+0000","last_fullsized":"2024-04-25T10:25:17.346165+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:46.447089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333353+0000","last_change":"2024-04-25T10:25:17.324832+0000","last_active":"2024-04-25T10:25:17.333353+0000","last_peered":"2024-04-25T10:25:17.333353+0000","last_clean":"2024-04-25T10:25:17.333353+0000","last_became_active":"2024-04-25T10:25:17.324426+0000","last_became_peered":"2024-04-25T10:25:17.324426+0000","last_unstale":"2024-04-25T10:25:17.333353+0000","last_undegraded":"2024-04-25T10:25:17.333353+0000","last_fullsized":"2024-04-25T10:25:17.333353+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:57.300709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.319931+0000","last_change":"2024-04-25T10:25:17.317381+0000","last_active":"2024-04-25T10:25:17.319931+0000","last_peered":"2024-04-25T10:25:17.319931+0000","last_clean":"2024-04-25T10:25:17.319931+0000","last_became_active":"2024-04-25T10:25:17.316947+0000","last_became_peered":"2024-04-25T10:25:17.316947+0000","last_unstale":"2024-04-25T10:25:17.319931+0000","last_undegraded":"2024-04-25T10:25:17.319931+0000","last_fullsized":"2024-04-25T10:25:17.319931+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:59.838154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333581+0000","last_change":"2024-04-25T10:25:17.331046+0000","last_active":"2024-04-25T10:25:17.333581+0000","last_peered":"2024-04-25T10:25:17.333581+0000","last_clean":"2024-04-25T10:25:17.333581+0000","last_became_active":"2024-04-25T10:25:17.330758+0000","last_became_peered":"2024-04-25T10:25:17.330758+0000","last_unstale":"2024-04-25T10:25:17.333581+0000","last_undegraded":"2024-04-25T10:25:17.333581+0000","last_fullsized":"2024-04-25T10:25:17.333581+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:55.524198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333394+0000","last_change":"2024-04-25T10:25:17.328466+0000","last_active":"2024-04-25T10:25:17.333394+0000","last_peered":"2024-04-25T10:25:17.333394+0000","last_clean":"2024-04-25T10:25:17.333394+0000","last_became_active":"2024-04-25T10:25:17.327799+0000","last_became_peered":"2024-04-25T10:25:17.327799+0000","last_unstale":"2024-04-25T10:25:17.333394+0000","last_undegraded":"2024-04-25T10:25:17.333394+0000","last_fullsized":"2024-04-25T10:25:17.333394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:31.032462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335354+0000","last_change":"2024-04-25T10:25:17.333819+0000","last_active":"2024-04-25T10:25:17.335354+0000","last_peered":"2024-04-25T10:25:17.335354+0000","last_clean":"2024-04-25T10:25:17.335354+0000","last_became_active":"2024-04-25T10:25:17.333479+0000","last_became_peered":"2024-04-25T10:25:17.333479+0000","last_unstale":"2024-04-25T10:25:17.335354+0000","last_undegraded":"2024-04-25T10:25:17.335354+0000","last_fullsized":"2024-04-25T10:25:17.335354+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:24.749760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338971+0000","last_change":"2024-04-25T10:25:17.338102+0000","last_active":"2024-04-25T10:25:17.338971+0000","last_peered":"2024-04-25T10:25:17.338971+0000","last_clean":"2024-04-25T10:25:17.338971+0000","last_became_active":"2024-04-25T10:25:17.337792+0000","last_became_peered":"2024-04-25T10:25:17.337792+0000","last_unstale":"2024-04-25T10:25:17.338971+0000","last_undegraded":"2024-04-25T10:25:17.338971+0000","last_fullsized":"2024-04-25T10:25:17.338971+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:43.585040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329754+0000","last_change":"2024-04-25T10:25:17.323439+0000","last_active":"2024-04-25T10:25:17.329754+0000","last_peered":"2024-04-25T10:25:17.329754+0000","last_clean":"2024-04-25T10:25:17.329754+0000","last_became_active":"2024-04-25T10:25:17.323141+0000","last_became_peered":"2024-04-25T10:25:17.323141+0000","last_unstale":"2024-04-25T10:25:17.329754+0000","last_undegraded":"2024-04-25T10:25:17.329754+0000","last_fullsized":"2024-04-25T10:25:17.329754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:30.922148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311449+0000","last_change":"2024-04-25T10:25:17.335766+0000","last_active":"2024-04-25T10:25:18.311449+0000","last_peered":"2024-04-25T10:25:18.311449+0000","last_clean":"2024-04-25T10:25:18.311449+0000","last_became_active":"2024-04-25T10:25:17.334231+0000","last_became_peered":"2024-04-25T10:25:17.334231+0000","last_unstale":"2024-04-25T10:25:18.311449+0000","last_undegraded":"2024-04-25T10:25:18.311449+0000","last_fullsized":"2024-04-25T10:25:18.311449+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:09.758846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337459+0000","last_change":"2024-04-25T10:25:17.330919+0000","last_active":"2024-04-25T10:25:17.337459+0000","last_peered":"2024-04-25T10:25:17.337459+0000","last_clean":"2024-04-25T10:25:17.337459+0000","last_became_active":"2024-04-25T10:25:17.330617+0000","last_became_peered":"2024-04-25T10:25:17.330617+0000","last_unstale":"2024-04-25T10:25:17.337459+0000","last_undegraded":"2024-04-25T10:25:17.337459+0000","last_fullsized":"2024-04-25T10:25:17.337459+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:45.783505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335135+0000","last_change":"2024-04-25T10:25:17.332791+0000","last_active":"2024-04-25T10:25:17.335135+0000","last_peered":"2024-04-25T10:25:17.335135+0000","last_clean":"2024-04-25T10:25:17.335135+0000","last_became_active":"2024-04-25T10:25:17.332536+0000","last_became_peered":"2024-04-25T10:25:17.332536+0000","last_unstale":"2024-04-25T10:25:17.335135+0000","last_undegraded":"2024-04-25T10:25:17.335135+0000","last_fullsized":"2024-04-25T10:25:17.335135+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:04.009172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337659+0000","last_change":"2024-04-25T10:25:17.334746+0000","last_active":"2024-04-25T10:25:17.337659+0000","last_peered":"2024-04-25T10:25:17.337659+0000","last_clean":"2024-04-25T10:25:17.337659+0000","last_became_active":"2024-04-25T10:25:17.334455+0000","last_became_peered":"2024-04-25T10:25:17.334455+0000","last_unstale":"2024-04-25T10:25:17.337659+0000","last_undegraded":"2024-04-25T10:25:17.337659+0000","last_fullsized":"2024-04-25T10:25:17.337659+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:52.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.326271+0000","last_change":"2024-04-25T10:25:17.325237+0000","last_active":"2024-04-25T10:25:17.326271+0000","last_peered":"2024-04-25T10:25:17.326271+0000","last_clean":"2024-04-25T10:25:17.326271+0000","last_became_active":"2024-04-25T10:25:17.324309+0000","last_became_peered":"2024-04-25T10:25:17.324309+0000","last_unstale":"2024-04-25T10:25:17.326271+0000","last_undegraded":"2024-04-25T10:25:17.326271+0000","last_fullsized":"2024-04-25T10:25:17.326271+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:01.383498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316873+0000","last_change":"2024-04-25T10:25:17.328951+0000","last_active":"2024-04-25T10:25:18.316873+0000","last_peered":"2024-04-25T10:25:18.316873+0000","last_clean":"2024-04-25T10:25:18.316873+0000","last_became_active":"2024-04-25T10:25:17.328711+0000","last_became_peered":"2024-04-25T10:25:17.328711+0000","last_unstale":"2024-04-25T10:25:18.316873+0000","last_undegraded":"2024-04-25T10:25:18.316873+0000","last_fullsized":"2024-04-25T10:25:18.316873+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.086283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.339935+0000","last_change":"2024-04-25T10:25:17.337124+0000","last_active":"2024-04-25T10:25:17.339935+0000","last_peered":"2024-04-25T10:25:17.339935+0000","last_clean":"2024-04-25T10:25:17.339935+0000","last_became_active":"2024-04-25T10:25:17.336323+0000","last_became_peered":"2024-04-25T10:25:17.336323+0000","last_unstale":"2024-04-25T10:25:17.339935+0000","last_undegraded":"2024-04-25T10:25:17.339935+0000","last_fullsized":"2024-04-25T10:25:17.339935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:44.663051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315761+0000","last_change":"2024-04-25T10:25:17.336196+0000","last_active":"2024-04-25T10:25:18.315761+0000","last_peered":"2024-04-25T10:25:18.315761+0000","last_clean":"2024-04-25T10:25:18.315761+0000","last_became_active":"2024-04-25T10:25:17.335900+0000","last_became_peered":"2024-04-25T10:25:17.335900+0000","last_unstale":"2024-04-25T10:25:18.315761+0000","last_undegraded":"2024-04-25T10:25:18.315761+0000","last_fullsized":"2024-04-25T10:25:18.315761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:39.657975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311485+0000","last_change":"2024-04-25T10:25:17.331578+0000","last_active":"2024-04-25T10:25:18.311485+0000","last_peered":"2024-04-25T10:25:18.311485+0000","last_clean":"2024-04-25T10:25:18.311485+0000","last_became_active":"2024-04-25T10:25:17.331086+0000","last_became_peered":"2024-04-25T10:25:17.331086+0000","last_unstale":"2024-04-25T10:25:18.311485+0000","last_undegraded":"2024-04-25T10:25:18.311485+0000","last_fullsized":"2024-04-25T10:25:18.311485+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:39.185655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338715+0000","last_change":"2024-04-25T10:25:17.328619+0000","last_active":"2024-04-25T10:25:17.338715+0000","last_peered":"2024-04-25T10:25:17.338715+0000","last_clean":"2024-04-25T10:25:17.338715+0000","last_became_active":"2024-04-25T10:25:17.328330+0000","last_became_peered":"2024-04-25T10:25:17.328330+0000","last_unstale":"2024-04-25T10:25:17.338715+0000","last_undegraded":"2024-04-25T10:25:17.338715+0000","last_fullsized":"2024-04-25T10:25:17.338715+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:39.857188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.331340+0000","last_change":"2024-04-25T10:25:17.322870+0000","last_active":"2024-04-25T10:25:17.331340+0000","last_peered":"2024-04-25T10:25:17.331340+0000","last_clean":"2024-04-25T10:25:17.331340+0000","last_became_active":"2024-04-25T10:25:17.322603+0000","last_became_peered":"2024-04-25T10:25:17.322603+0000","last_unstale":"2024-04-25T10:25:17.331340+0000","last_undegraded":"2024-04-25T10:25:17.331340+0000","last_fullsized":"2024-04-25T10:25:17.331340+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:12.033703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333864+0000","last_change":"2024-04-25T10:25:17.329470+0000","last_active":"2024-04-25T10:25:17.333864+0000","last_peered":"2024-04-25T10:25:17.333864+0000","last_clean":"2024-04-25T10:25:17.333864+0000","last_became_active":"2024-04-25T10:25:17.329158+0000","last_became_peered":"2024-04-25T10:25:17.329158+0000","last_unstale":"2024-04-25T10:25:17.333864+0000","last_undegraded":"2024-04-25T10:25:17.333864+0000","last_fullsized":"2024-04-25T10:25:17.333864+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:43.585040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337526+0000","last_change":"2024-04-25T10:25:17.334866+0000","last_active":"2024-04-25T10:25:17.337526+0000","last_peered":"2024-04-25T10:25:17.337526+0000","last_clean":"2024-04-25T10:25:17.337526+0000","last_became_active":"2024-04-25T10:25:17.334598+0000","last_became_peered":"2024-04-25T10:25:17.334598+0000","last_unstale":"2024-04-25T10:25:17.337526+0000","last_undegraded":"2024-04-25T10:25:17.337526+0000","last_fullsized":"2024-04-25T10:25:17.337526+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:49.470748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.336386+0000","last_change":"2024-04-25T10:25:17.334784+0000","last_active":"2024-04-25T10:25:17.336386+0000","last_peered":"2024-04-25T10:25:17.336386+0000","last_clean":"2024-04-25T10:25:17.336386+0000","last_became_active":"2024-04-25T10:25:17.333271+0000","last_became_peered":"2024-04-25T10:25:17.333271+0000","last_unstale":"2024-04-25T10:25:17.336386+0000","last_undegraded":"2024-04-25T10:25:17.336386+0000","last_fullsized":"2024-04-25T10:25:17.336386+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:18.095808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.340298+0000","last_change":"2024-04-25T10:25:17.339553+0000","last_active":"2024-04-25T10:25:17.340298+0000","last_peered":"2024-04-25T10:25:17.340298+0000","last_clean":"2024-04-25T10:25:17.340298+0000","last_became_active":"2024-04-25T10:25:17.338777+0000","last_became_peered":"2024-04-25T10:25:17.338777+0000","last_unstale":"2024-04-25T10:25:17.340298+0000","last_undegraded":"2024-04-25T10:25:17.340298+0000","last_fullsized":"2024-04-25T10:25:17.340298+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:44.669235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.330514+0000","last_change":"2024-04-25T10:25:17.326313+0000","last_active":"2024-04-25T10:25:17.330514+0000","last_peered":"2024-04-25T10:25:17.330514+0000","last_clean":"2024-04-25T10:25:17.330514+0000","last_became_active":"2024-04-25T10:25:17.326034+0000","last_became_peered":"2024-04-25T10:25:17.326034+0000","last_unstale":"2024-04-25T10:25:17.330514+0000","last_undegraded":"2024-04-25T10:25:17.330514+0000","last_fullsized":"2024-04-25T10:25:17.330514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:20.854606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337756+0000","last_change":"2024-04-25T10:25:17.330778+0000","last_active":"2024-04-25T10:25:17.337756+0000","last_peered":"2024-04-25T10:25:17.337756+0000","last_clean":"2024-04-25T10:25:17.337756+0000","last_became_active":"2024-04-25T10:25:17.329730+0000","last_became_peered":"2024-04-25T10:25:17.329730+0000","last_unstale":"2024-04-25T10:25:17.337756+0000","last_undegraded":"2024-04-25T10:25:17.337756+0000","last_fullsized":"2024-04-25T10:25:17.337756+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:28.279914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.334635+0000","last_change":"2024-04-25T10:25:17.329304+0000","last_active":"2024-04-25T10:25:17.334635+0000","last_peered":"2024-04-25T10:25:17.334635+0000","last_clean":"2024-04-25T10:25:17.334635+0000","last_became_active":"2024-04-25T10:25:17.328892+0000","last_became_peered":"2024-04-25T10:25:17.328892+0000","last_unstale":"2024-04-25T10:25:17.334635+0000","last_undegraded":"2024-04-25T10:25:17.334635+0000","last_fullsized":"2024-04-25T10:25:17.334635+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:12.755915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.334014+0000","last_change":"2024-04-25T10:25:17.326968+0000","last_active":"2024-04-25T10:25:17.334014+0000","last_peered":"2024-04-25T10:25:17.334014+0000","last_clean":"2024-04-25T10:25:17.334014+0000","last_became_active":"2024-04-25T10:25:17.326698+0000","last_became_peered":"2024-04-25T10:25:17.326698+0000","last_unstale":"2024-04-25T10:25:17.334014+0000","last_undegraded":"2024-04-25T10:25:17.334014+0000","last_fullsized":"2024-04-25T10:25:17.334014+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.298688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333915+0000","last_change":"2024-04-25T10:25:17.325291+0000","last_active":"2024-04-25T10:25:17.333915+0000","last_peered":"2024-04-25T10:25:17.333915+0000","last_clean":"2024-04-25T10:25:17.333915+0000","last_became_active":"2024-04-25T10:25:17.324722+0000","last_became_peered":"2024-04-25T10:25:17.324722+0000","last_unstale":"2024-04-25T10:25:17.333915+0000","last_undegraded":"2024-04-25T10:25:17.333915+0000","last_fullsized":"2024-04-25T10:25:17.333915+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:40.388203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333907+0000","last_change":"2024-04-25T10:25:17.327182+0000","last_active":"2024-04-25T10:25:17.333907+0000","last_peered":"2024-04-25T10:25:17.333907+0000","last_clean":"2024-04-25T10:25:17.333907+0000","last_became_active":"2024-04-25T10:25:17.326923+0000","last_became_peered":"2024-04-25T10:25:17.326923+0000","last_unstale":"2024-04-25T10:25:17.333907+0000","last_undegraded":"2024-04-25T10:25:17.333907+0000","last_fullsized":"2024-04-25T10:25:17.333907+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:15.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316972+0000","last_change":"2024-04-25T10:25:17.328691+0000","last_active":"2024-04-25T10:25:18.316972+0000","last_peered":"2024-04-25T10:25:18.316972+0000","last_clean":"2024-04-25T10:25:18.316972+0000","last_became_active":"2024-04-25T10:25:17.328397+0000","last_became_peered":"2024-04-25T10:25:17.328397+0000","last_unstale":"2024-04-25T10:25:18.316972+0000","last_undegraded":"2024-04-25T10:25:18.316972+0000","last_fullsized":"2024-04-25T10:25:18.316972+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.921086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.347644+0000","last_change":"2024-04-25T10:25:16.319616+0000","last_active":"2024-04-25T10:25:17.347644+0000","last_peered":"2024-04-25T10:25:17.347644+0000","last_clean":"2024-04-25T10:25:17.347644+0000","last_became_active":"2024-04-25T10:25:16.319346+0000","last_became_peered":"2024-04-25T10:25:16.319346+0000","last_unstale":"2024-04-25T10:25:17.347644+0000","last_undegraded":"2024-04-25T10:25:17.347644+0000","last_fullsized":"2024-04-25T10:25:17.347644+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:16.152573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338899+0000","last_change":"2024-04-25T10:25:17.336908+0000","last_active":"2024-04-25T10:25:17.338899+0000","last_peered":"2024-04-25T10:25:17.338899+0000","last_clean":"2024-04-25T10:25:17.338899+0000","last_became_active":"2024-04-25T10:25:17.335590+0000","last_became_peered":"2024-04-25T10:25:17.335590+0000","last_unstale":"2024-04-25T10:25:17.338899+0000","last_undegraded":"2024-04-25T10:25:17.338899+0000","last_fullsized":"2024-04-25T10:25:17.338899+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:03.906293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346128+0000","last_change":"2024-04-25T10:25:16.304038+0000","last_active":"2024-04-25T10:25:17.346128+0000","last_peered":"2024-04-25T10:25:17.346128+0000","last_clean":"2024-04-25T10:25:17.346128+0000","last_became_active":"2024-04-25T10:25:16.303742+0000","last_became_peered":"2024-04-25T10:25:16.303742+0000","last_unstale":"2024-04-25T10:25:17.346128+0000","last_undegraded":"2024-04-25T10:25:17.346128+0000","last_fullsized":"2024-04-25T10:25:17.346128+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:19.132756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333821+0000","last_change":"2024-04-25T10:25:17.327281+0000","last_active":"2024-04-25T10:25:17.333821+0000","last_peered":"2024-04-25T10:25:17.333821+0000","last_clean":"2024-04-25T10:25:17.333821+0000","last_became_active":"2024-04-25T10:25:17.326740+0000","last_became_peered":"2024-04-25T10:25:17.326740+0000","last_unstale":"2024-04-25T10:25:17.333821+0000","last_undegraded":"2024-04-25T10:25:17.333821+0000","last_fullsized":"2024-04-25T10:25:17.333821+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:18.095808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.345330+0000","last_change":"2024-04-25T10:25:16.323667+0000","last_active":"2024-04-25T10:25:17.345330+0000","last_peered":"2024-04-25T10:25:17.345330+0000","last_clean":"2024-04-25T10:25:17.345330+0000","last_became_active":"2024-04-25T10:25:16.323402+0000","last_became_peered":"2024-04-25T10:25:16.323402+0000","last_unstale":"2024-04-25T10:25:17.345330+0000","last_undegraded":"2024-04-25T10:25:17.345330+0000","last_fullsized":"2024-04-25T10:25:17.345330+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:27.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.336393+0000","last_change":"2024-04-25T10:25:17.332748+0000","last_active":"2024-04-25T10:25:17.336393+0000","last_peered":"2024-04-25T10:25:17.336393+0000","last_clean":"2024-04-25T10:25:17.336393+0000","last_became_active":"2024-04-25T10:25:17.332376+0000","last_became_peered":"2024-04-25T10:25:17.332376+0000","last_unstale":"2024-04-25T10:25:17.336393+0000","last_undegraded":"2024-04-25T10:25:17.336393+0000","last_fullsized":"2024-04-25T10:25:17.336393+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:15.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.348127+0000","last_change":"2024-04-25T10:25:16.320132+0000","last_active":"2024-04-25T10:25:17.348127+0000","last_peered":"2024-04-25T10:25:17.348127+0000","last_clean":"2024-04-25T10:25:17.348127+0000","last_became_active":"2024-04-25T10:25:16.319848+0000","last_became_peered":"2024-04-25T10:25:16.319848+0000","last_unstale":"2024-04-25T10:25:17.348127+0000","last_undegraded":"2024-04-25T10:25:17.348127+0000","last_fullsized":"2024-04-25T10:25:17.348127+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:08.748315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311147+0000","last_change":"2024-04-25T10:25:17.318982+0000","last_active":"2024-04-25T10:25:18.311147+0000","last_peered":"2024-04-25T10:25:18.311147+0000","last_clean":"2024-04-25T10:25:18.311147+0000","last_became_active":"2024-04-25T10:25:17.318434+0000","last_became_peered":"2024-04-25T10:25:17.318434+0000","last_unstale":"2024-04-25T10:25:18.311147+0000","last_undegraded":"2024-04-25T10:25:18.311147+0000","last_fullsized":"2024-04-25T10:25:18.311147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:20.540499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.347442+0000","last_change":"2024-04-25T10:25:16.320289+0000","last_active":"2024-04-25T10:25:17.347442+0000","last_peered":"2024-04-25T10:25:17.347442+0000","last_clean":"2024-04-25T10:25:17.347442+0000","last_became_active":"2024-04-25T10:25:16.320014+0000","last_became_peered":"2024-04-25T10:25:16.320014+0000","last_unstale":"2024-04-25T10:25:17.347442+0000","last_undegraded":"2024-04-25T10:25:17.347442+0000","last_fullsized":"2024-04-25T10:25:17.347442+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:08.363040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.320299+0000","last_change":"2024-04-25T10:25:17.319275+0000","last_active":"2024-04-25T10:25:17.320299+0000","last_peered":"2024-04-25T10:25:17.320299+0000","last_clean":"2024-04-25T10:25:17.320299+0000","last_became_active":"2024-04-25T10:25:17.319009+0000","last_became_peered":"2024-04-25T10:25:17.319009+0000","last_unstale":"2024-04-25T10:25:17.320299+0000","last_undegraded":"2024-04-25T10:25:17.320299+0000","last_fullsized":"2024-04-25T10:25:17.320299+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:00.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.344750+0000","last_change":"2024-04-25T10:25:16.320146+0000","last_active":"2024-04-25T10:25:17.344750+0000","last_peered":"2024-04-25T10:25:17.344750+0000","last_clean":"2024-04-25T10:25:17.344750+0000","last_became_active":"2024-04-25T10:25:16.319848+0000","last_became_peered":"2024-04-25T10:25:16.319848+0000","last_unstale":"2024-04-25T10:25:17.344750+0000","last_undegraded":"2024-04-25T10:25:17.344750+0000","last_fullsized":"2024-04-25T10:25:17.344750+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:35.964647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337030+0000","last_change":"2024-04-25T10:25:17.335019+0000","last_active":"2024-04-25T10:25:17.337030+0000","last_peered":"2024-04-25T10:25:17.337030+0000","last_clean":"2024-04-25T10:25:17.337030+0000","last_became_active":"2024-04-25T10:25:17.334722+0000","last_became_peered":"2024-04-25T10:25:17.334722+0000","last_unstale":"2024-04-25T10:25:17.337030+0000","last_undegraded":"2024-04-25T10:25:17.337030+0000","last_fullsized":"2024-04-25T10:25:17.337030+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:38.899730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315468+0000","last_change":"2024-04-25T10:25:16.305938+0000","last_active":"2024-04-25T10:25:17.315468+0000","last_peered":"2024-04-25T10:25:17.315468+0000","last_clean":"2024-04-25T10:25:17.315468+0000","last_became_active":"2024-04-25T10:25:16.305631+0000","last_became_peered":"2024-04-25T10:25:16.305631+0000","last_unstale":"2024-04-25T10:25:17.315468+0000","last_undegraded":"2024-04-25T10:25:17.315468+0000","last_fullsized":"2024-04-25T10:25:17.315468+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:31.120957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311082+0000","last_change":"2024-04-25T10:25:17.318938+0000","last_active":"2024-04-25T10:25:18.311082+0000","last_peered":"2024-04-25T10:25:18.311082+0000","last_clean":"2024-04-25T10:25:18.311082+0000","last_became_active":"2024-04-25T10:25:17.318621+0000","last_became_peered":"2024-04-25T10:25:17.318621+0000","last_unstale":"2024-04-25T10:25:18.311082+0000","last_undegraded":"2024-04-25T10:25:18.311082+0000","last_fullsized":"2024-04-25T10:25:18.311082+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:37.482567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315616+0000","last_change":"2024-04-25T10:25:16.323148+0000","last_active":"2024-04-25T10:25:18.315616+0000","last_peered":"2024-04-25T10:25:18.315616+0000","last_clean":"2024-04-25T10:25:18.315616+0000","last_became_active":"2024-04-25T10:25:16.322285+0000","last_became_peered":"2024-04-25T10:25:16.322285+0000","last_unstale":"2024-04-25T10:25:18.315616+0000","last_undegraded":"2024-04-25T10:25:18.315616+0000","last_fullsized":"2024-04-25T10:25:18.315616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:20.657052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.342350+0000","last_change":"2024-04-25T10:25:17.340096+0000","last_active":"2024-04-25T10:25:17.342350+0000","last_peered":"2024-04-25T10:25:17.342350+0000","last_clean":"2024-04-25T10:25:17.342350+0000","last_became_active":"2024-04-25T10:25:17.335792+0000","last_became_peered":"2024-04-25T10:25:17.335792+0000","last_unstale":"2024-04-25T10:25:17.342350+0000","last_undegraded":"2024-04-25T10:25:17.342350+0000","last_fullsized":"2024-04-25T10:25:17.342350+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:09.372882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.313996+0000","last_change":"2024-04-25T10:25:16.318782+0000","last_active":"2024-04-25T10:25:17.313996+0000","last_peered":"2024-04-25T10:25:17.313996+0000","last_clean":"2024-04-25T10:25:17.313996+0000","last_became_active":"2024-04-25T10:25:16.318450+0000","last_became_peered":"2024-04-25T10:25:16.318450+0000","last_unstale":"2024-04-25T10:25:17.313996+0000","last_undegraded":"2024-04-25T10:25:17.313996+0000","last_fullsized":"2024-04-25T10:25:17.313996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:28.869952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329867+0000","last_change":"2024-04-25T10:25:17.327183+0000","last_active":"2024-04-25T10:25:17.329867+0000","last_peered":"2024-04-25T10:25:17.329867+0000","last_clean":"2024-04-25T10:25:17.329867+0000","last_became_active":"2024-04-25T10:25:17.326912+0000","last_became_peered":"2024-04-25T10:25:17.326912+0000","last_unstale":"2024-04-25T10:25:17.329867+0000","last_undegraded":"2024-04-25T10:25:17.329867+0000","last_fullsized":"2024-04-25T10:25:17.329867+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:50.045084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.322708+0000","last_change":"2024-04-25T10:25:16.321087+0000","last_active":"2024-04-25T10:25:17.322708+0000","last_peered":"2024-04-25T10:25:17.322708+0000","last_clean":"2024-04-25T10:25:17.322708+0000","last_became_active":"2024-04-25T10:25:16.320801+0000","last_became_peered":"2024-04-25T10:25:16.320801+0000","last_unstale":"2024-04-25T10:25:17.322708+0000","last_undegraded":"2024-04-25T10:25:17.322708+0000","last_fullsized":"2024-04-25T10:25:17.322708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:38.175123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338448+0000","last_change":"2024-04-25T10:25:17.335665+0000","last_active":"2024-04-25T10:25:17.338448+0000","last_peered":"2024-04-25T10:25:17.338448+0000","last_clean":"2024-04-25T10:25:17.338448+0000","last_became_active":"2024-04-25T10:25:17.334863+0000","last_became_peered":"2024-04-25T10:25:17.334863+0000","last_unstale":"2024-04-25T10:25:17.338448+0000","last_undegraded":"2024-04-25T10:25:17.338448+0000","last_fullsized":"2024-04-25T10:25:17.338448+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:41.465106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346158+0000","last_change":"2024-04-25T10:25:16.313497+0000","last_active":"2024-04-25T10:25:17.346158+0000","last_peered":"2024-04-25T10:25:17.346158+0000","last_clean":"2024-04-25T10:25:17.346158+0000","last_became_active":"2024-04-25T10:25:16.313064+0000","last_became_peered":"2024-04-25T10:25:16.313064+0000","last_unstale":"2024-04-25T10:25:17.346158+0000","last_undegraded":"2024-04-25T10:25:17.346158+0000","last_fullsized":"2024-04-25T10:25:17.346158+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:56.918562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337493+0000","last_change":"2024-04-25T10:25:17.334281+0000","last_active":"2024-04-25T10:25:17.337493+0000","last_peered":"2024-04-25T10:25:17.337493+0000","last_clean":"2024-04-25T10:25:17.337493+0000","last_became_active":"2024-04-25T10:25:17.334014+0000","last_became_peered":"2024-04-25T10:25:17.334014+0000","last_unstale":"2024-04-25T10:25:17.337493+0000","last_undegraded":"2024-04-25T10:25:17.337493+0000","last_fullsized":"2024-04-25T10:25:17.337493+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:19.243975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311558+0000","last_change":"2024-04-25T10:25:16.313694+0000","last_active":"2024-04-25T10:25:18.311558+0000","last_peered":"2024-04-25T10:25:18.311558+0000","last_clean":"2024-04-25T10:25:18.311558+0000","last_became_active":"2024-04-25T10:25:16.313397+0000","last_became_peered":"2024-04-25T10:25:16.313397+0000","last_unstale":"2024-04-25T10:25:18.311558+0000","last_undegraded":"2024-04-25T10:25:18.311558+0000","last_fullsized":"2024-04-25T10:25:18.311558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:44.772974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335159+0000","last_change":"2024-04-25T10:25:17.333171+0000","last_active":"2024-04-25T10:25:17.335159+0000","last_peered":"2024-04-25T10:25:17.335159+0000","last_clean":"2024-04-25T10:25:17.335159+0000","last_became_active":"2024-04-25T10:25:17.332219+0000","last_became_peered":"2024-04-25T10:25:17.332219+0000","last_unstale":"2024-04-25T10:25:17.335159+0000","last_undegraded":"2024-04-25T10:25:17.335159+0000","last_fullsized":"2024-04-25T10:25:17.335159+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:33.183487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315985+0000","last_change":"2024-04-25T10:25:16.307483+0000","last_active":"2024-04-25T10:25:18.315985+0000","last_peered":"2024-04-25T10:25:18.315985+0000","last_clean":"2024-04-25T10:25:18.315985+0000","last_became_active":"2024-04-25T10:25:16.307160+0000","last_became_peered":"2024-04-25T10:25:16.307160+0000","last_unstale":"2024-04-25T10:25:18.315985+0000","last_undegraded":"2024-04-25T10:25:18.315985+0000","last_fullsized":"2024-04-25T10:25:18.315985+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:19.844075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.338835+0000","last_change":"2024-04-25T10:25:17.334693+0000","last_active":"2024-04-25T10:25:17.338835+0000","last_peered":"2024-04-25T10:25:17.338835+0000","last_clean":"2024-04-25T10:25:17.338835+0000","last_became_active":"2024-04-25T10:25:17.329717+0000","last_became_peered":"2024-04-25T10:25:17.329717+0000","last_unstale":"2024-04-25T10:25:17.338835+0000","last_undegraded":"2024-04-25T10:25:17.338835+0000","last_fullsized":"2024-04-25T10:25:17.338835+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:03.223023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.319934+0000","last_change":"2024-04-25T10:25:16.320139+0000","last_active":"2024-04-25T10:25:17.319934+0000","last_peered":"2024-04-25T10:25:17.319934+0000","last_clean":"2024-04-25T10:25:17.319934+0000","last_became_active":"2024-04-25T10:25:16.319639+0000","last_became_peered":"2024-04-25T10:25:16.319639+0000","last_unstale":"2024-04-25T10:25:17.319934+0000","last_undegraded":"2024-04-25T10:25:17.319934+0000","last_fullsized":"2024-04-25T10:25:17.319934+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:36.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.322970+0000","last_change":"2024-04-25T10:25:17.320968+0000","last_active":"2024-04-25T10:25:17.322970+0000","last_peered":"2024-04-25T10:25:17.322970+0000","last_clean":"2024-04-25T10:25:17.322970+0000","last_became_active":"2024-04-25T10:25:17.318899+0000","last_became_peered":"2024-04-25T10:25:17.318899+0000","last_unstale":"2024-04-25T10:25:17.322970+0000","last_undegraded":"2024-04-25T10:25:17.322970+0000","last_fullsized":"2024-04-25T10:25:17.322970+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:05.310522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311124+0000","last_change":"2024-04-25T10:25:16.318497+0000","last_active":"2024-04-25T10:25:18.311124+0000","last_peered":"2024-04-25T10:25:18.311124+0000","last_clean":"2024-04-25T10:25:18.311124+0000","last_became_active":"2024-04-25T10:25:16.318225+0000","last_became_peered":"2024-04-25T10:25:16.318225+0000","last_unstale":"2024-04-25T10:25:18.311124+0000","last_undegraded":"2024-04-25T10:25:18.311124+0000","last_fullsized":"2024-04-25T10:25:18.311124+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:16.152573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.333938+0000","last_change":"2024-04-25T10:25:17.325542+0000","last_active":"2024-04-25T10:25:17.333938+0000","last_peered":"2024-04-25T10:25:17.333938+0000","last_clean":"2024-04-25T10:25:17.333938+0000","last_became_active":"2024-04-25T10:25:17.325047+0000","last_became_peered":"2024-04-25T10:25:17.325047+0000","last_unstale":"2024-04-25T10:25:17.333938+0000","last_undegraded":"2024-04-25T10:25:17.333938+0000","last_fullsized":"2024-04-25T10:25:17.333938+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:50.280414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.350031+0000","last_change":"2024-04-25T10:25:16.322326+0000","last_active":"2024-04-25T10:25:17.350031+0000","last_peered":"2024-04-25T10:25:17.350031+0000","last_clean":"2024-04-25T10:25:17.350031+0000","last_became_active":"2024-04-25T10:25:16.321967+0000","last_became_peered":"2024-04-25T10:25:16.321967+0000","last_unstale":"2024-04-25T10:25:17.350031+0000","last_undegraded":"2024-04-25T10:25:17.350031+0000","last_fullsized":"2024-04-25T10:25:17.350031+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:19.529968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329419+0000","last_change":"2024-04-25T10:25:17.326600+0000","last_active":"2024-04-25T10:25:17.329419+0000","last_peered":"2024-04-25T10:25:17.329419+0000","last_clean":"2024-04-25T10:25:17.329419+0000","last_became_active":"2024-04-25T10:25:17.325400+0000","last_became_peered":"2024-04-25T10:25:17.325400+0000","last_unstale":"2024-04-25T10:25:17.329419+0000","last_undegraded":"2024-04-25T10:25:17.329419+0000","last_fullsized":"2024-04-25T10:25:17.329419+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.717156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311481+0000","last_change":"2024-04-25T10:25:16.316378+0000","last_active":"2024-04-25T10:25:18.311481+0000","last_peered":"2024-04-25T10:25:18.311481+0000","last_clean":"2024-04-25T10:25:18.311481+0000","last_became_active":"2024-04-25T10:25:16.315414+0000","last_became_peered":"2024-04-25T10:25:16.315414+0000","last_unstale":"2024-04-25T10:25:18.311481+0000","last_undegraded":"2024-04-25T10:25:18.311481+0000","last_fullsized":"2024-04-25T10:25:18.311481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:00.696127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.329797+0000","last_change":"2024-04-25T10:25:17.327310+0000","last_active":"2024-04-25T10:25:17.329797+0000","last_peered":"2024-04-25T10:25:17.329797+0000","last_clean":"2024-04-25T10:25:17.329797+0000","last_became_active":"2024-04-25T10:25:17.322279+0000","last_became_peered":"2024-04-25T10:25:17.322279+0000","last_unstale":"2024-04-25T10:25:17.329797+0000","last_undegraded":"2024-04-25T10:25:17.329797+0000","last_fullsized":"2024-04-25T10:25:17.329797+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:41.678714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.346022+0000","last_change":"2024-04-25T10:25:16.318068+0000","last_active":"2024-04-25T10:25:17.346022+0000","last_peered":"2024-04-25T10:25:17.346022+0000","last_clean":"2024-04-25T10:25:17.346022+0000","last_became_active":"2024-04-25T10:25:16.316424+0000","last_became_peered":"2024-04-25T10:25:16.316424+0000","last_unstale":"2024-04-25T10:25:17.346022+0000","last_undegraded":"2024-04-25T10:25:17.346022+0000","last_fullsized":"2024-04-25T10:25:17.346022+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:31.992394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311800+0000","last_change":"2024-04-25T10:25:17.323925+0000","last_active":"2024-04-25T10:25:18.311800+0000","last_peered":"2024-04-25T10:25:18.311800+0000","last_clean":"2024-04-25T10:25:18.311800+0000","last_became_active":"2024-04-25T10:25:17.323658+0000","last_became_peered":"2024-04-25T10:25:17.323658+0000","last_unstale":"2024-04-25T10:25:18.311800+0000","last_undegraded":"2024-04-25T10:25:18.311800+0000","last_fullsized":"2024-04-25T10:25:18.311800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:15.222453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315616+0000","last_change":"2024-04-25T10:25:16.308940+0000","last_active":"2024-04-25T10:25:18.315616+0000","last_peered":"2024-04-25T10:25:18.315616+0000","last_clean":"2024-04-25T10:25:18.315616+0000","last_became_active":"2024-04-25T10:25:16.306608+0000","last_became_peered":"2024-04-25T10:25:16.306608+0000","last_unstale":"2024-04-25T10:25:18.315616+0000","last_undegraded":"2024-04-25T10:25:18.315616+0000","last_fullsized":"2024-04-25T10:25:18.315616+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:10.092046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316450+0000","last_change":"2024-04-25T10:25:17.324978+0000","last_active":"2024-04-25T10:25:18.316450+0000","last_peered":"2024-04-25T10:25:18.316450+0000","last_clean":"2024-04-25T10:25:18.316450+0000","last_became_active":"2024-04-25T10:25:17.324708+0000","last_became_peered":"2024-04-25T10:25:17.324708+0000","last_unstale":"2024-04-25T10:25:18.316450+0000","last_undegraded":"2024-04-25T10:25:18.316450+0000","last_fullsized":"2024-04-25T10:25:18.316450+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:02.232621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.347671+0000","last_change":"2024-04-25T10:25:16.320860+0000","last_active":"2024-04-25T10:25:17.347671+0000","last_peered":"2024-04-25T10:25:17.347671+0000","last_clean":"2024-04-25T10:25:17.347671+0000","last_became_active":"2024-04-25T10:25:16.320590+0000","last_became_peered":"2024-04-25T10:25:16.320590+0000","last_unstale":"2024-04-25T10:25:17.347671+0000","last_undegraded":"2024-04-25T10:25:17.347671+0000","last_fullsized":"2024-04-25T10:25:17.347671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:52.846423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315875+0000","last_change":"2024-04-25T10:25:17.331982+0000","last_active":"2024-04-25T10:25:18.315875+0000","last_peered":"2024-04-25T10:25:18.315875+0000","last_clean":"2024-04-25T10:25:18.315875+0000","last_became_active":"2024-04-25T10:25:17.331671+0000","last_became_peered":"2024-04-25T10:25:17.331671+0000","last_unstale":"2024-04-25T10:25:18.315875+0000","last_undegraded":"2024-04-25T10:25:18.315875+0000","last_fullsized":"2024-04-25T10:25:18.315875+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:21.860431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.321533+0000","last_change":"2024-04-25T10:25:16.317019+0000","last_active":"2024-04-25T10:25:17.321533+0000","last_peered":"2024-04-25T10:25:17.321533+0000","last_clean":"2024-04-25T10:25:17.321533+0000","last_became_active":"2024-04-25T10:25:16.316561+0000","last_became_peered":"2024-04-25T10:25:16.316561+0000","last_unstale":"2024-04-25T10:25:17.321533+0000","last_undegraded":"2024-04-25T10:25:17.321533+0000","last_fullsized":"2024-04-25T10:25:17.321533+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:57.236059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337759+0000","last_change":"2024-04-25T10:25:17.335378+0000","last_active":"2024-04-25T10:25:17.337759+0000","last_peered":"2024-04-25T10:25:17.337759+0000","last_clean":"2024-04-25T10:25:17.337759+0000","last_became_active":"2024-04-25T10:25:17.335110+0000","last_became_peered":"2024-04-25T10:25:17.335110+0000","last_unstale":"2024-04-25T10:25:17.337759+0000","last_undegraded":"2024-04-25T10:25:17.337759+0000","last_fullsized":"2024-04-25T10:25:17.337759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:03.954804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.321329+0000","last_change":"2024-04-25T10:25:16.317460+0000","last_active":"2024-04-25T10:25:17.321329+0000","last_peered":"2024-04-25T10:25:17.321329+0000","last_clean":"2024-04-25T10:25:17.321329+0000","last_became_active":"2024-04-25T10:25:16.315646+0000","last_became_peered":"2024-04-25T10:25:16.315646+0000","last_unstale":"2024-04-25T10:25:17.321329+0000","last_undegraded":"2024-04-25T10:25:17.321329+0000","last_fullsized":"2024-04-25T10:25:17.321329+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:38.175123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.337556+0000","last_change":"2024-04-25T10:25:17.332074+0000","last_active":"2024-04-25T10:25:17.337556+0000","last_peered":"2024-04-25T10:25:17.337556+0000","last_clean":"2024-04-25T10:25:17.337556+0000","last_became_active":"2024-04-25T10:25:17.331819+0000","last_became_peered":"2024-04-25T10:25:17.331819+0000","last_unstale":"2024-04-25T10:25:17.337556+0000","last_undegraded":"2024-04-25T10:25:17.337556+0000","last_fullsized":"2024-04-25T10:25:17.337556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:43.126962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315023+0000","last_change":"2024-04-25T10:25:16.315905+0000","last_active":"2024-04-25T10:25:17.315023+0000","last_peered":"2024-04-25T10:25:17.315023+0000","last_clean":"2024-04-25T10:25:17.315023+0000","last_became_active":"2024-04-25T10:25:16.315165+0000","last_became_peered":"2024-04-25T10:25:16.315165+0000","last_unstale":"2024-04-25T10:25:17.315023+0000","last_undegraded":"2024-04-25T10:25:17.315023+0000","last_fullsized":"2024-04-25T10:25:17.315023+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:22.650760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.317121+0000","last_change":"2024-04-25T10:25:17.326354+0000","last_active":"2024-04-25T10:25:18.317121+0000","last_peered":"2024-04-25T10:25:18.317121+0000","last_clean":"2024-04-25T10:25:18.317121+0000","last_became_active":"2024-04-25T10:25:17.325977+0000","last_became_peered":"2024-04-25T10:25:17.325977+0000","last_unstale":"2024-04-25T10:25:18.317121+0000","last_undegraded":"2024-04-25T10:25:18.317121+0000","last_fullsized":"2024-04-25T10:25:18.317121+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:31.628621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311632+0000","last_change":"2024-04-25T10:25:16.320302+0000","last_active":"2024-04-25T10:25:18.311632+0000","last_peered":"2024-04-25T10:25:18.311632+0000","last_clean":"2024-04-25T10:25:18.311632+0000","last_became_active":"2024-04-25T10:25:16.319851+0000","last_became_peered":"2024-04-25T10:25:16.319851+0000","last_unstale":"2024-04-25T10:25:18.311632+0000","last_undegraded":"2024-04-25T10:25:18.311632+0000","last_fullsized":"2024-04-25T10:25:18.311632+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.745384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316962+0000","last_change":"2024-04-25T10:25:17.335191+0000","last_active":"2024-04-25T10:25:18.316962+0000","last_peered":"2024-04-25T10:25:18.316962+0000","last_clean":"2024-04-25T10:25:18.316962+0000","last_became_active":"2024-04-25T10:25:17.334226+0000","last_became_peered":"2024-04-25T10:25:17.334226+0000","last_unstale":"2024-04-25T10:25:18.316962+0000","last_undegraded":"2024-04-25T10:25:18.316962+0000","last_fullsized":"2024-04-25T10:25:18.316962+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:49.895711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.343042+0000","last_change":"2024-04-25T10:25:16.318770+0000","last_active":"2024-04-25T10:25:18.343042+0000","last_peered":"2024-04-25T10:25:18.343042+0000","last_clean":"2024-04-25T10:25:18.343042+0000","last_became_active":"2024-04-25T10:25:16.318503+0000","last_became_peered":"2024-04-25T10:25:16.318503+0000","last_unstale":"2024-04-25T10:25:18.343042+0000","last_undegraded":"2024-04-25T10:25:18.343042+0000","last_fullsized":"2024-04-25T10:25:18.343042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.034553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.331194+0000","last_change":"2024-04-25T10:25:17.325010+0000","last_active":"2024-04-25T10:25:17.331194+0000","last_peered":"2024-04-25T10:25:17.331194+0000","last_clean":"2024-04-25T10:25:17.331194+0000","last_became_active":"2024-04-25T10:25:17.324685+0000","last_became_peered":"2024-04-25T10:25:17.324685+0000","last_unstale":"2024-04-25T10:25:17.331194+0000","last_undegraded":"2024-04-25T10:25:17.331194+0000","last_fullsized":"2024-04-25T10:25:17.331194+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:17.262400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316721+0000","last_change":"2024-04-25T10:25:16.315435+0000","last_active":"2024-04-25T10:25:18.316721+0000","last_peered":"2024-04-25T10:25:18.316721+0000","last_clean":"2024-04-25T10:25:18.316721+0000","last_became_active":"2024-04-25T10:25:16.315149+0000","last_became_peered":"2024-04-25T10:25:16.315149+0000","last_unstale":"2024-04-25T10:25:18.316721+0000","last_undegraded":"2024-04-25T10:25:18.316721+0000","last_fullsized":"2024-04-25T10:25:18.316721+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:57.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.332819+0000","last_change":"2024-04-25T10:25:17.330232+0000","last_active":"2024-04-25T10:25:17.332819+0000","last_peered":"2024-04-25T10:25:17.332819+0000","last_clean":"2024-04-25T10:25:17.332819+0000","last_became_active":"2024-04-25T10:25:17.329884+0000","last_became_peered":"2024-04-25T10:25:17.329884+0000","last_unstale":"2024-04-25T10:25:17.332819+0000","last_undegraded":"2024-04-25T10:25:17.332819+0000","last_fullsized":"2024-04-25T10:25:17.332819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:00.936902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.323096+0000","last_change":"2024-04-25T10:25:16.320521+0000","last_active":"2024-04-25T10:25:17.323096+0000","last_peered":"2024-04-25T10:25:17.323096+0000","last_clean":"2024-04-25T10:25:17.323096+0000","last_became_active":"2024-04-25T10:25:16.320102+0000","last_became_peered":"2024-04-25T10:25:16.320102+0000","last_unstale":"2024-04-25T10:25:17.323096+0000","last_undegraded":"2024-04-25T10:25:17.323096+0000","last_fullsized":"2024-04-25T10:25:17.323096+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:20.657052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.316797+0000","last_change":"2024-04-25T10:25:17.325481+0000","last_active":"2024-04-25T10:25:18.316797+0000","last_peered":"2024-04-25T10:25:18.316797+0000","last_clean":"2024-04-25T10:25:18.316797+0000","last_became_active":"2024-04-25T10:25:17.325202+0000","last_became_peered":"2024-04-25T10:25:17.325202+0000","last_unstale":"2024-04-25T10:25:18.316797+0000","last_undegraded":"2024-04-25T10:25:18.316797+0000","last_fullsized":"2024-04-25T10:25:18.316797+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:56.549749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315868+0000","last_change":"2024-04-25T10:25:16.323722+0000","last_active":"2024-04-25T10:25:17.315868+0000","last_peered":"2024-04-25T10:25:17.315868+0000","last_clean":"2024-04-25T10:25:17.315868+0000","last_became_active":"2024-04-25T10:25:16.323243+0000","last_became_peered":"2024-04-25T10:25:16.323243+0000","last_unstale":"2024-04-25T10:25:17.315868+0000","last_undegraded":"2024-04-25T10:25:17.315868+0000","last_fullsized":"2024-04-25T10:25:17.315868+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:42.451132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.335733+0000","last_change":"2024-04-25T10:25:17.334306+0000","last_active":"2024-04-25T10:25:17.335733+0000","last_peered":"2024-04-25T10:25:17.335733+0000","last_clean":"2024-04-25T10:25:17.335733+0000","last_became_active":"2024-04-25T10:25:17.333999+0000","last_became_peered":"2024-04-25T10:25:17.333999+0000","last_unstale":"2024-04-25T10:25:17.335733+0000","last_undegraded":"2024-04-25T10:25:17.335733+0000","last_fullsized":"2024-04-25T10:25:17.335733+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:06.505620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.322168+0000","last_change":"2024-04-25T10:25:16.313109+0000","last_active":"2024-04-25T10:25:17.322168+0000","last_peered":"2024-04-25T10:25:17.322168+0000","last_clean":"2024-04-25T10:25:17.322168+0000","last_became_active":"2024-04-25T10:25:16.312845+0000","last_became_peered":"2024-04-25T10:25:16.312845+0000","last_unstale":"2024-04-25T10:25:17.322168+0000","last_undegraded":"2024-04-25T10:25:17.322168+0000","last_fullsized":"2024-04-25T10:25:17.322168+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:10.092046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.315509+0000","last_change":"2024-04-25T10:25:17.320561+0000","last_active":"2024-04-25T10:25:18.315509+0000","last_peered":"2024-04-25T10:25:18.315509+0000","last_clean":"2024-04-25T10:25:18.315509+0000","last_became_active":"2024-04-25T10:25:17.318491+0000","last_became_peered":"2024-04-25T10:25:17.318491+0000","last_unstale":"2024-04-25T10:25:18.315509+0000","last_undegraded":"2024-04-25T10:25:18.315509+0000","last_fullsized":"2024-04-25T10:25:18.315509+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:14.515182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.321333+0000","last_change":"2024-04-25T10:25:16.316870+0000","last_active":"2024-04-25T10:25:17.321333+0000","last_peered":"2024-04-25T10:25:17.321333+0000","last_clean":"2024-04-25T10:25:17.321333+0000","last_became_active":"2024-04-25T10:25:16.316543+0000","last_became_peered":"2024-04-25T10:25:16.316543+0000","last_unstale":"2024-04-25T10:25:17.321333+0000","last_undegraded":"2024-04-25T10:25:17.321333+0000","last_fullsized":"2024-04-25T10:25:17.321333+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:00.696127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311060+0000","last_change":"2024-04-25T10:25:17.327417+0000","last_active":"2024-04-25T10:25:18.311060+0000","last_peered":"2024-04-25T10:25:18.311060+0000","last_clean":"2024-04-25T10:25:18.311060+0000","last_became_active":"2024-04-25T10:25:17.327126+0000","last_became_peered":"2024-04-25T10:25:17.327126+0000","last_unstale":"2024-04-25T10:25:18.311060+0000","last_undegraded":"2024-04-25T10:25:18.311060+0000","last_fullsized":"2024-04-25T10:25:18.311060+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:53.856954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.315042+0000","last_change":"2024-04-25T10:25:16.315447+0000","last_active":"2024-04-25T10:25:17.315042+0000","last_peered":"2024-04-25T10:25:17.315042+0000","last_clean":"2024-04-25T10:25:17.315042+0000","last_became_active":"2024-04-25T10:25:16.314834+0000","last_became_peered":"2024-04-25T10:25:16.314834+0000","last_unstale":"2024-04-25T10:25:17.315042+0000","last_undegraded":"2024-04-25T10:25:17.315042+0000","last_fullsized":"2024-04-25T10:25:17.315042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:26.048826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:25:17.341256+0000","last_change":"2024-04-25T10:25:17.338235+0000","last_active":"2024-04-25T10:25:17.341256+0000","last_peered":"2024-04-25T10:25:17.341256+0000","last_clean":"2024-04-25T10:25:17.341256+0000","last_became_active":"2024-04-25T10:25:17.337852+0000","last_became_peered":"2024-04-25T10:25:17.337852+0000","last_unstale":"2024-04-25T10:25:17.341256+0000","last_undegraded":"2024-04-25T10:25:17.341256+0000","last_fullsized":"2024-04-25T10:25:17.341256+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:16.291713+0000","last_clean_scrub_stamp":"2024-04-25T10:25:16.291713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:35.675365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.311692+0000","last_change":"2024-04-25T10:25:16.314529+0000","last_active":"2024-04-25T10:25:18.311692+0000","last_peered":"2024-04-25T10:25:18.311692+0000","last_clean":"2024-04-25T10:25:18.311692+0000","last_became_active":"2024-04-25T10:25:16.314235+0000","last_became_peered":"2024-04-25T10:25:16.314235+0000","last_unstale":"2024-04-25T10:25:18.311692+0000","last_undegraded":"2024-04-25T10:25:18.311692+0000","last_fullsized":"2024-04-25T10:25:18.311692+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:25:15.281182+0000","last_clean_scrub_stamp":"2024-04-25T10:25:15.281182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:06.288157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":327,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:25:18.310963+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:25:18.310963+0000","last_peered":"2024-04-25T10:25:18.310963+0000","last_clean":"2024-04-25T10:25:18.310963+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:25:18.310963+0000","last_undegraded":"2024-04-25T10:25:18.310963+0000","last_fullsized":"2024-04-25T10:25:18.310963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509533,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46312,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325528,"statfs":{"total":96636764160,"available":96589340672,"internally_reserved":0,"allocated":1269760,"data_stored":546333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509534,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50464,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321376,"statfs":{"total":96636764160,"available":96585089024,"internally_reserved":0,"allocated":1265664,"data_stored":545751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55240,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316600,"statfs":{"total":96636764160,"available":96580198400,"internally_reserved":0,"allocated":1892352,"data_stored":1144087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509533,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54640,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317200,"statfs":{"total":96636764160,"available":96580812800,"internally_reserved":0,"allocated":1277952,"data_stored":545945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509533,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58796,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313044,"statfs":{"total":96636764160,"available":96576557056,"internally_reserved":0,"allocated":1269760,"data_stored":545821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509534,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":1892352,"data_stored":1144143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":545751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54632,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317208,"statfs":{"total":96636764160,"available":96580820992,"internally_reserved":0,"allocated":1269760,"data_stored":546333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:25:21.828 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:25:21.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:25:22.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.012+0000 7f356fa00640 1 Processor -- start 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.012+0000 7f356fa00640 1 -- start start 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356fa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35680af7c0 0x7f3568071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35680670b0 0x7f35681618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356fa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356815e3c0 con 0x7f356815b800 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356fa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680afc50 con 0x7f35680af7c0 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356815ecf0 con 0x7f35680670b0 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f3567e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35680af7c0 0x7f3568071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35680670b0 0x7f35681618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356d000640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f356815b800 0x7f35680669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:56182/0 (socket says 172.21.15.45:56182) 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356d000640 1 -- 172.21.15.45:0/1550626040 learned_addr learned my addr 172.21.15.45:0/1550626040 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356da00640 1 -- 172.21.15.45:0/1550626040 >> v2:172.21.15.117:3300/0 conn(0x7f35680af7c0 msgr2=0x7f3568071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:22.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356da00640 1 --2- 172.21.15.45:0/1550626040 >> v2:172.21.15.117:3300/0 conn(0x7f35680af7c0 0x7f3568071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356da00640 1 -- 172.21.15.45:0/1550626040 >> v1:172.21.15.117:6789/0 conn(0x7f356815b800 legacy=0x7f35680669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f356da00640 1 -- 172.21.15.45:0/1550626040 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356815e6a0 con 0x7f35680670b0 2024-04-25T10:25:22.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f3567e00640 1 --2- 172.21.15.45:0/1550626040 >> v2:172.21.15.117:3300/0 conn(0x7f35680af7c0 0x7f3568071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:22.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.016+0000 7f3567400640 1 -- 172.21.15.45:0/1550626040 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277160143 0 0) 0x7f356815e3c0 con 0x7f356815b800 2024-04-25T10:25:22.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356da00640 1 --2- 172.21.15.45:0/1550626040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35680670b0 0x7f35681618a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3560009d90 tx=0x7f356001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f6d8f93463204d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f3567400640 1 -- 172.21.15.45:0/1550626040 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600b2070 con 0x7f35680670b0 2024-04-25T10:25:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f3567400640 1 -- 172.21.15.45:0/1550626040 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35600a5990 con 0x7f35680670b0 2024-04-25T10:25:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f3567400640 1 -- 172.21.15.45:0/1550626040 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600a5d60 con 0x7f35680670b0 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- 172.21.15.45:0/1550626040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35680670b0 msgr2=0x7f35681618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 --2- 172.21.15.45:0/1550626040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35680670b0 0x7f35681618a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3560009d90 tx=0x7f356001c940 comp rx=0 tx=0).stop 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- 172.21.15.45:0/1550626040 shutdown_connections 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 --2- 172.21.15.45:0/1550626040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35680670b0 0x7f35681618a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 --2- 172.21.15.45:0/1550626040 >> v2:172.21.15.117:3300/0 conn(0x7f35680af7c0 0x7f3568071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- 172.21.15.45:0/1550626040 >> 172.21.15.45:0/1550626040 conn(0x7f35680d0820 msgr2=0x7f35680d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- 172.21.15.45:0/1550626040 shutdown_connections 2024-04-25T10:25:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- 172.21.15.45:0/1550626040 wait complete. 2024-04-25T10:25:22.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 Processor -- start 2024-04-25T10:25:22.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- start start 2024-04-25T10:25:22.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 0x7f35681501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356815b800 0x7f35680b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3568148850 0x7f35680b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356816ac40 con 0x7f35680670b0 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356815bbe0 con 0x7f3568148850 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.020+0000 7f356fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356816adc0 con 0x7f356815b800 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 0x7f35681501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 0x7f35681501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52058/0 (socket says 172.21.15.45:52058) 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 -- 172.21.15.45:0/529461971 learned_addr learned my addr 172.21.15.45:0/529461971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f3567e00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356815b800 0x7f35680b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356da00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3568148850 0x7f35680b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 -- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356815b800 msgr2=0x7f35680b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356815b800 0x7f35680b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 -- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3568148850 msgr2=0x7f35680b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3568148850 0x7f35680b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 -- 172.21.15.45:0/529461971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568137bc0 con 0x7f35680670b0 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356da00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3568148850 0x7f35680b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356d000640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 0x7f35681501e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c4a0 tx=0x7f355c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c018070 con 0x7f35680670b0 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c0074e0 con 0x7f35680670b0 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c004a30 con 0x7f35680670b0 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3568137040 con 0x7f35680670b0 2024-04-25T10:25:22.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3568137600 con 0x7f35680670b0 2024-04-25T10:25:22.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356804ef30 con 0x7f35680670b0 2024-04-25T10:25:22.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.024+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f355c004bd0 con 0x7f35680670b0 2024-04-25T10:25:22.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.028+0000 7f3564c00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3548066f90 0x7f3548069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.028+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f355c084250 con 0x7f35680670b0 2024-04-25T10:25:22.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.028+0000 7f3567e00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3548066f90 0x7f3548069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.028+0000 7f3567e00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3548066f90 0x7f3548069450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3554009580 tx=0x7f3554009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:22.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.032+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f355c054b70 con 0x7f35680670b0 2024-04-25T10:25:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.248+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3568148ce0 con 0x7f35680670b0 2024-04-25T10:25:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f3564c00640 1 -- 172.21.15.45:0/529461971 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f355c058c10 con 0x7f35680670b0 2024-04-25T10:25:22.252 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:25:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:25:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:25:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:25:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:25:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3548066f90 msgr2=0x7f3548069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3548066f90 0x7f3548069450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3554009580 tx=0x7f3554009190 comp rx=0 tx=0).stop 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 msgr2=0x7f35681501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 0x7f35681501e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c4a0 tx=0x7f355c00c960 comp rx=0 tx=0).stop 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 shutdown_connections 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3548066f90 0x7f3548069450 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3568148850 0x7f35680b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356815b800 0x7f35680b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 --2- 172.21.15.45:0/529461971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35680670b0 0x7f35681501e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 >> 172.21.15.45:0/529461971 conn(0x7f35680d0820 msgr2=0x7f3568150ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:22.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 shutdown_connections 2024-04-25T10:25:22.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.252+0000 7f356fa00640 1 -- 172.21.15.45:0/529461971 wait complete. 2024-04-25T10:25:22.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:22.278 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:25:22.276+0000 7f6d09000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:25:22.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.412+0000 7f371ea00640 1 Processor -- start 2024-04-25T10:25:22.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.412+0000 7f371ea00640 1 -- start start 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f371815bc40 0x7f371806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37180cf930 0x7f37180cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ea00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37180b2770 con 0x7f37180cda10 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ea00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371806c670 con 0x7f37180cf930 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ea00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371806c950 con 0x7f371815bc40 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37180cf930 0x7f37180cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37180cf930 0x7f37180cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52060/0 (socket says 172.21.15.45:52060) 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 -- 172.21.15.45:0/1430159702 learned_addr learned my addr 172.21.15.45:0/1430159702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f3717e00640 1 --2- 172.21.15.45:0/1430159702 >> v2:172.21.15.117:3300/0 conn(0x7f371815bc40 0x7f371806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 -- 172.21.15.45:0/1430159702 >> v2:172.21.15.117:3300/0 conn(0x7f371815bc40 msgr2=0x7f371806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:22.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 --2- 172.21.15.45:0/1430159702 >> v2:172.21.15.117:3300/0 conn(0x7f371815bc40 0x7f371806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 -- 172.21.15.45:0/1430159702 >> v1:172.21.15.117:6789/0 conn(0x7f37180cda10 legacy=0x7f37180b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.416+0000 7f371ca00640 1 -- 172.21.15.45:0/1430159702 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371806cc30 con 0x7f37180cf930 2024-04-25T10:25:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ca00640 1 --2- 172.21.15.45:0/1430159702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cf930 0x7f37180cfdc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f37180d0800 tx=0x7f371001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5409eb175e68183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f3717400640 1 -- 172.21.15.45:0/1430159702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100a5620 con 0x7f37180cf930 2024-04-25T10:25:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f3717400640 1 -- 172.21.15.45:0/1430159702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37100a57c0 con 0x7f37180cf930 2024-04-25T10:25:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f3717400640 1 -- 172.21.15.45:0/1430159702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100a5b90 con 0x7f37180cf930 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 -- 172.21.15.45:0/1430159702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cf930 msgr2=0x7f37180cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 --2- 172.21.15.45:0/1430159702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cf930 0x7f37180cfdc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f37180d0800 tx=0x7f371001c960 comp rx=0 tx=0).stop 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 -- 172.21.15.45:0/1430159702 shutdown_connections 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 --2- 172.21.15.45:0/1430159702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cf930 0x7f37180cfdc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 --2- 172.21.15.45:0/1430159702 >> v2:172.21.15.117:3300/0 conn(0x7f371815bc40 0x7f371806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 -- 172.21.15.45:0/1430159702 >> 172.21.15.45:0/1430159702 conn(0x7f3718066b70 msgr2=0x7f3718066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 -- 172.21.15.45:0/1430159702 shutdown_connections 2024-04-25T10:25:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 -- 172.21.15.45:0/1430159702 wait complete. 2024-04-25T10:25:22.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 Processor -- start 2024-04-25T10:25:22.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.420+0000 7f371ea00640 1 -- start start 2024-04-25T10:25:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 0x7f37181579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cf930 0x7f37181fb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f371815bc40 0x7f3718165f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37180b3850 con 0x7f37180cda10 2024-04-25T10:25:22.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37180b36d0 con 0x7f37180cf930 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37180b39d0 con 0x7f371815bc40 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 0x7f37181579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 0x7f37181579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52070/0 (socket says 172.21.15.45:52070) 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 -- 172.21.15.45:0/1951744946 learned_addr learned my addr 172.21.15.45:0/1951744946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f370f400640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cf930 0x7f37181fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ca00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f371815bc40 0x7f3718165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 -- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f371815bc40 msgr2=0x7f3718165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f371815bc40 0x7f3718165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 -- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cf930 msgr2=0x7f37181fb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cf930 0x7f37181fb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 -- 172.21.15.45:0/1951744946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37181f7290 con 0x7f37180cda10 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f370f400640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cf930 0x7f37181fb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3717e00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 0x7f37181579c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f370800d620 tx=0x7f370800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:22.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800b620 con 0x7f37180cda10 2024-04-25T10:25:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3708019070 con 0x7f37180cda10 2024-04-25T10:25:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800ba20 con 0x7f37180cda10 2024-04-25T10:25:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37181f7570 con 0x7f37180cda10 2024-04-25T10:25:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37181f8050 con 0x7f37180cda10 2024-04-25T10:25:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371804ef30 con 0x7f37180cda10 2024-04-25T10:25:22.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.424+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f370800bbc0 con 0x7f37180cda10 2024-04-25T10:25:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.428+0000 7f3714c00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f90 0x7f36f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.428+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37080854f0 con 0x7f37180cda10 2024-04-25T10:25:22.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.428+0000 7f370f400640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f90 0x7f36f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:22.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.428+0000 7f370f400640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f90 0x7f36f4069450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f37180cbda0 tx=0x7f3700009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:22.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.432+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3708056460 con 0x7f37180cda10 2024-04-25T10:25:22.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.624+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37180af1c0 con 0x7f37180cda10 2024-04-25T10:25:22.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f3714c00640 1 -- 172.21.15.45:0/1951744946 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3708055d60 con 0x7f37180cda10 2024-04-25T10:25:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:22.628 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:22.628 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f90 msgr2=0x7f36f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f90 0x7f36f4069450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f37180cbda0 tx=0x7f3700009040 comp rx=0 tx=0).stop 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 msgr2=0x7f37181579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 0x7f37181579c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f370800d620 tx=0x7f370800dae0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 shutdown_connections 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f90 0x7f36f4069450 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f371815bc40 0x7f3718165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cf930 0x7f37181fb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 --2- 172.21.15.45:0/1951744946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180cda10 0x7f37181579c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:22.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 >> 172.21.15.45:0/1951744946 conn(0x7f3718066b70 msgr2=0x7f37180b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:22.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 shutdown_connections 2024-04-25T10:25:22.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:22.628+0000 7f371ea00640 1 -- 172.21.15.45:0/1951744946 wait complete. 2024-04-25T10:25:23.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:25:16.702073+0000', 'modified': '2024-04-25T10:25:19.723604+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7851}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7851': {'gid': 7851, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/4218207272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 4218207272}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4218207272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:25:23.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:25:23.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:25:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.196+0000 7f7e26a00640 1 Processor -- start 2024-04-25T10:25:23.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.196+0000 7f7e26a00640 1 -- start start 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e26a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e201550b0 0x7f7e200cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e26a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 0x7f7e200b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e26a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e20071050 con 0x7f7e200cfe70 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e26a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2015bda0 con 0x7f7e201550b0 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e26a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015d040 con 0x7f7e2015b910 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e17400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7e200cfe70 0x7f7e2015b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50530/0 (socket says 172.21.15.45:50530) 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e17400640 1 -- 172.21.15.45:0/256408733 learned_addr learned my addr 172.21.15.45:0/256408733 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:23.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 0x7f7e200b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/256408733 >> v2:172.21.15.117:3300/0 conn(0x7f7e201550b0 0x7f7e200cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706608039 0 0) 0x7f7e20071050 con 0x7f7e200cfe70 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e04005740 con 0x7f7e200cfe70 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332885950 0 0) 0x7f7e04005740 con 0x7f7e200cfe70 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 msgr2=0x7f7e200b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 0x7f7e200b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 >> v2:172.21.15.117:3300/0 conn(0x7f7e201550b0 msgr2=0x7f7e200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 --2- 172.21.15.45:0/256408733 >> v2:172.21.15.117:3300/0 conn(0x7f7e201550b0 0x7f7e200cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e200712f0 con 0x7f7e200cfe70 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f7e08001680 con 0x7f7e200cfe70 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/256408733 >> v2:172.21.15.117:3300/0 conn(0x7f7e201550b0 0x7f7e200cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 >> v1:172.21.15.45:6789/0 conn(0x7f7e200cfe70 legacy=0x7f7e2015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 0x7f7e04005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e04006150 0x7f7e04006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e04006f60 0x7f7e040073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2015d040 con 0x7f7e04006f60 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0400dbe0 con 0x7f7e04006150 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0400dd60 con 0x7f7e2015b910 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e04006f60 0x7f7e040073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 0x7f7e04005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e17400640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e04006150 0x7f7e04006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 -- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 msgr2=0x7f7e04005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e2015b910 0x7f7e04005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 -- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e04006150 msgr2=0x7f7e04006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e04006150 0x7f7e04006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.200+0000 7f7e24a00640 1 -- 172.21.15.45:0/256408733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e04005740 con 0x7f7e04006f60 2024-04-25T10:25:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e24a00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e04006f60 0x7f7e040073e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009fc0 tx=0x7f7e1801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dc20898a4924f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180b2070 con 0x7f7e04006f60 2024-04-25T10:25:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180a5c90 con 0x7f7e04006f60 2024-04-25T10:25:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e1f400640 1 -- 172.21.15.45:0/256408733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18009c10 con 0x7f7e04006f60 2024-04-25T10:25:23.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e26a00640 1 -- 172.21.15.45:0/256408733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e04006f60 msgr2=0x7f7e040073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e26a00640 1 --2- 172.21.15.45:0/256408733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e04006f60 0x7f7e040073e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009fc0 tx=0x7f7e1801cc20 comp rx=0 tx=0).stop 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e24a00640 1 -- 172.21.15.45:0/256408733 reap_dead start 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e26a00640 1 -- 172.21.15.45:0/256408733 shutdown_connections 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e26a00640 1 -- 172.21.15.45:0/256408733 >> 172.21.15.45:0/256408733 conn(0x7f7e20066b70 msgr2=0x7f7e20066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e26a00640 1 -- 172.21.15.45:0/256408733 shutdown_connections 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.204+0000 7f7e26a00640 1 -- 172.21.15.45:0/256408733 wait complete. 2024-04-25T10:25:23.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 Processor -- start 2024-04-25T10:25:23.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 -- start start 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e04006f60 0x7f7e201e9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e20152290 0x7f7e201e6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 0x7f7e201f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20071b90 con 0x7f7e201f2920 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200718b0 con 0x7f7e20152290 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e200720b0 con 0x7f7e04006f60 2024-04-25T10:25:23.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 0x7f7e201f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 0x7f7e201f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52094/0 (socket says 172.21.15.45:52094) 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 -- 172.21.15.45:0/3246216498 learned_addr learned my addr 172.21.15.45:0/3246216498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e04006f60 0x7f7e201e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e17400640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e20152290 0x7f7e201e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 -- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e04006f60 msgr2=0x7f7e201e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e04006f60 0x7f7e201e9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 -- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e20152290 msgr2=0x7f7e201e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e20152290 0x7f7e201e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 -- 172.21.15.45:0/3246216498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20157310 con 0x7f7e201f2920 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e17400640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e20152290 0x7f7e201e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e24a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 0x7f7e201f2da0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7e18002240 tx=0x7f7e18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e1cc00640 1 -- 172.21.15.45:0/3246216498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180bd070 con 0x7f7e201f2920 2024-04-25T10:25:23.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e1cc00640 1 -- 172.21.15.45:0/3246216498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180a5a90 con 0x7f7e201f2920 2024-04-25T10:25:23.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e200cc7f0 con 0x7f7e201f2920 2024-04-25T10:25:23.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e1cc00640 1 -- 172.21.15.45:0/3246216498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180b2040 con 0x7f7e201f2920 2024-04-25T10:25:23.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.208+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e200ccdb0 con 0x7f7e201f2920 2024-04-25T10:25:23.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1cc00640 1 -- 172.21.15.45:0/3246216498 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e180c0070 con 0x7f7e201f2920 2024-04-25T10:25:23.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1cc00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7dfc066f90 0x7f7dfc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1cc00640 1 -- 172.21.15.45:0/3246216498 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e1811ee70 con 0x7f7e201f2920 2024-04-25T10:25:23.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7dfc066f90 0x7f7dfc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 Processor -- start 2024-04-25T10:25:23.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 -- start start 2024-04-25T10:25:23.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7dfc066f90 0x7f7dfc069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009d30 tx=0x7f7e10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9919d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 0x5576b98cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b98d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b97b75f0 con 0x5576b98cff30 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b971eda0 con 0x5576b9919920 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576b96a7150 con 0x5576b991a240 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9919d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e17400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 0x5576b98cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b98d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e17400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 0x5576b98cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:47748/0 (socket says 172.21.15.45:47748) 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e17400640 1 -- 172.21.15.45:0/3857660129 learned_addr learned my addr 172.21.15.45:0/3857660129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.212+0000 7f7e1fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9919d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35620/0 (socket says 172.21.15.45:35620) 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 -- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 msgr2=0x5576b98cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 0x5576b98cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 -- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 msgr2=0x5576b9919d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9919d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 -- 172.21.15.45:0/3857660129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576b96c0ca0 con 0x5576b98cff30 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e17400640 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 0x5576b98cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b98d0310 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7e180987b0 tx=0x7f7e180ef1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af73322cac9998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e14c00640 1 -- 172.21.15.45:0/3857660129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180f3830 con 0x5576b98cff30 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e14c00640 1 -- 172.21.15.45:0/3857660129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180f7df0 con 0x5576b98cff30 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e14c00640 1 -- 172.21.15.45:0/3857660129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180aecd0 con 0x5576b98cff30 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- 172.21.15.45:0/3857660129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 msgr2=0x5576b98d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b98d0310 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7e180987b0 tx=0x7f7e180ef1b0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- 172.21.15.45:0/3857660129 shutdown_connections 2024-04-25T10:25:23.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b98d0310 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b991a240 0x5576b98cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/3857660129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9919d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- 172.21.15.45:0/3857660129 >> 172.21.15.45:0/3857660129 conn(0x5576b98d62a0 msgr2=0x5576b98ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- 172.21.15.45:0/3857660129 shutdown_connections 2024-04-25T10:25:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- 172.21.15.45:0/3857660129 wait complete. 2024-04-25T10:25:23.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- start start 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 Processor -- start 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b98cff30 0x5576b9920c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b99211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b98d3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b96d9990 con 0x5576b991a240 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b971ae40 con 0x5576b98cff30 2024-04-25T10:25:23.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576b96aa900 con 0x5576b9919920 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e29ebc000 10 client.0 _ll_register_callbacks cb 0x5576b97d21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b98d3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b98d3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52110/0 (socket says 172.21.15.45:52110) 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e24a00640 1 -- 172.21.15.45:0/2916065069 learned_addr learned my addr 172.21.15.45:0/2916065069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b98cff30 0x5576b9920c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.216+0000 7f7e17400640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b99211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e24a00640 1 -- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 msgr2=0x5576b99211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e24a00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b99211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e24a00640 1 -- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b98cff30 msgr2=0x5576b9920c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e24a00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b98cff30 0x5576b9920c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e24a00640 1 -- 172.21.15.45:0/2916065069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576b96d7630 con 0x5576b991a240 2024-04-25T10:25:23.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b98cff30 0x5576b9920c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e17400640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b99211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e24a00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b98d3aa0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7e1801c690 tx=0x7f7e1801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e98fc8bc1b3668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 1 -- 172.21.15.45:0/2916065069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180f3830 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e180f7df0 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 1 -- 172.21.15.45:0/2916065069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e18107070 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 1 -- 172.21.15.45:0/2916065069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e18107270 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 10 client.7992 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5576b96a73a0 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 1 -- 172.21.15.45:0/2916065069 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180f7df0 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 1 -- 172.21.15.45:0/2916065069 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e180b5050 con 0x5576b991a240 2024-04-25T10:25:23.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 10 client.7992 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 10 client.7992 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 20 client.7992 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b9919500 0x5576b98d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 4 client.7992 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576b96baa50 con 0x5576b9919500 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b9919500 0x5576b98d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b9919500 0x5576b98d44b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b96e3c21034ab9 server_cookie=1f1a199358a240b1 in_seq=0 out_seq=0 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 10 client.7992 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 1 -- 172.21.15.45:0/2916065069 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576b96baa50 con 0x5576b9919500 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 10 client.7992 handle_command_reply: tid=0 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 client.7992 shutdown 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 msgr2=0x5576b98d3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b98d3aa0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7e1801c690 tx=0x7f7e1801cc20 comp rx=0 tx=0).stop 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 shutdown_connections 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b98d3aa0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b99211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b9919500 0x5576b98d44b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2916065069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b98cff30 0x5576b9920c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 >> 172.21.15.45:0/2916065069 conn(0x5576b98d62a0 msgr2=0x5576b98ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e0d600640 0 client.7992 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 shutdown_connections 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2916065069 wait complete. 2024-04-25T10:25:23.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.220+0000 7f7e29ebc000 20 client.7992 trim_cache size 0 max 16384 2024-04-25T10:25:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 Processor -- start 2024-04-25T10:25:23.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 -- start start 2024-04-25T10:25:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b98cff30 0x5576b9905ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9906010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b992bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201ff3f0 con 0x5576b991a240 2024-04-25T10:25:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20157060 con 0x5576b9919920 2024-04-25T10:25:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.236+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e201fedf0 con 0x5576b98cff30 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b992bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e17400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9906010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e1fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b98cff30 0x5576b9905ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b992bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52122/0 (socket says 172.21.15.45:52122) 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 -- 172.21.15.45:0/139193475 learned_addr learned my addr 172.21.15.45:0/139193475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 -- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b98cff30 msgr2=0x5576b9905ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b98cff30 0x5576b9905ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 -- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 msgr2=0x5576b9906010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9906010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 -- 172.21.15.45:0/139193475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576b991a070 con 0x5576b991a240 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e17400640 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9906010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e24a00640 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b992bdf0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7e1801ca50 tx=0x7f7e180f2070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f54be714c290b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e0fe00640 1 -- 172.21.15.45:0/139193475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180f7df0 con 0x5576b991a240 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e0fe00640 1 -- 172.21.15.45:0/139193475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180f3830 con 0x5576b991a240 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e0fe00640 1 -- 172.21.15.45:0/139193475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18009040 con 0x5576b991a240 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- 172.21.15.45:0/139193475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 msgr2=0x5576b992bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b992bdf0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7e1801ca50 tx=0x7f7e180f2070 comp rx=0 tx=0).stop 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- 172.21.15.45:0/139193475 shutdown_connections 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b991a240 0x5576b992bdf0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b9919920 0x5576b9906010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/139193475 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b98cff30 0x5576b9905ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- 172.21.15.45:0/139193475 >> 172.21.15.45:0/139193475 conn(0x5576b98d62a0 msgr2=0x5576b9930050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- 172.21.15.45:0/139193475 shutdown_connections 2024-04-25T10:25:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- 172.21.15.45:0/139193475 wait complete. 2024-04-25T10:25:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- start start 2024-04-25T10:25:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 Processor -- start 2024-04-25T10:25:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b992eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b992f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b991a240 0x5576b992f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b97233d0 con 0x5576b98cff30 2024-04-25T10:25:23.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b97dad50 con 0x5576b991a240 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576b96b9980 con 0x5576b9919920 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e29ebc000 10 client.0 _ll_register_callbacks cb 0x7f7e180bffa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e1fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b992eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e1fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b992eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52132/0 (socket says 172.21.15.45:52132) 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.240+0000 7f7e1fe00640 1 -- 172.21.15.45:0/2080413621 learned_addr learned my addr 172.21.15.45:0/2080413621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e17400640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b992f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e24a00640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b991a240 0x5576b992f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e1fe00640 1 -- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 msgr2=0x5576b992f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b992f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e1fe00640 1 -- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b991a240 msgr2=0x5576b992f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b991a240 0x5576b992f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e1fe00640 1 -- 172.21.15.45:0/2080413621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576b97a0600 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e1fe00640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b992eb20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x5576b98d9400 tx=0x7f7e10004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e203f71e5c9d82a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 1 -- 172.21.15.45:0/2080413621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1002c1b0 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 1 -- 172.21.15.45:0/2080413621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e10036ae0 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e180f3830 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 1 -- 172.21.15.45:0/2080413621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e10036d20 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 10 client.7998 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5576b96b39b0 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 1 -- 172.21.15.45:0/2080413621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180f3830 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 1 -- 172.21.15.45:0/2080413621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7e100059a0 con 0x5576b98cff30 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 10 client.7998 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 10 client.7998 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 20 client.7998 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b99688e0 0x5576b992fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 4 client.7998 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5576b972c2e0 con 0x5576b99688e0 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e17400640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b99688e0 0x5576b992fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e17400640 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b99688e0 0x5576b992fae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1206c56d2cfc2ca6 server_cookie=469b52f164589baf in_seq=0 out_seq=0 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 10 client.7998 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 1 -- 172.21.15.45:0/2080413621 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5576b972c2e0 con 0x5576b99688e0 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 10 client.7998 handle_command_reply: tid=0 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 client.7998 shutdown 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 msgr2=0x5576b992eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b992eb20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x5576b98d9400 tx=0x7f7e10004740 comp rx=0 tx=0).stop 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 shutdown_connections 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5576b99688e0 0x5576b992fae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576b991a240 0x5576b992f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576b9919920 0x5576b992f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 --2- 172.21.15.45:0/2080413621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576b98cff30 0x5576b992eb20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 >> 172.21.15.45:0/2080413621 conn(0x5576b98d62a0 msgr2=0x5576b98d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e0d600640 0 client.7998 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 shutdown_connections 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 1 -- 172.21.15.45:0/2080413621 wait complete. 2024-04-25T10:25:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e29ebc000 20 client.7998 trim_cache size 0 max 16384 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7851, 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9211132209999997, 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 73, 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 104, 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 104, 2024-04-25T10:25:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.760269488 2024-04-25T10:25:23.254 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:23.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7dfc066f90 msgr2=0x7f7dfc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7dfc066f90 0x7f7dfc069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009d30 tx=0x7f7e10008040 comp rx=0 tx=0).stop 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 msgr2=0x7f7e201f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 0x7f7e201f2da0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7e18002240 tx=0x7f7e18009750 comp rx=0 tx=0).stop 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 shutdown_connections 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7dfc066f90 0x7f7dfc069450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e201f2920 0x7f7e201f2da0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e20152290 0x7f7e201e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 --2- 172.21.15.45:0/3246216498 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e04006f60 0x7f7e201e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 >> 172.21.15.45:0/3246216498 conn(0x7f7e20066b70 msgr2=0x7f7e200b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 shutdown_connections 2024-04-25T10:25:23.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.244+0000 7f7e26a00640 1 -- 172.21.15.45:0/3246216498 wait complete. 2024-04-25T10:25:23.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:25:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.408+0000 7ff7c6600640 1 Processor -- start 2024-04-25T10:25:23.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.408+0000 7ff7c6600640 1 -- start start 2024-04-25T10:25:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7c6600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7c00cdba0 0x7ff7c00b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7c6600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c0072110 con 0x7ff7c0144270 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7c6600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c00cd930 con 0x7ff7c00ce030 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7c6600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c0144650 con 0x7ff7c00cdba0 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7b7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7c00cdba0 0x7ff7c00b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7b7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7c00cdba0 0x7ff7c00b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52142/0 (socket says 172.21.15.45:52142) 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7b7e00640 1 -- 172.21.15.45:0/1135878921 learned_addr learned my addr 172.21.15.45:0/1135878921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7c5200640 1 --1- 172.21.15.45:0/1135878921 >> v1:172.21.15.45:6789/0 conn(0x7ff7c0144270 0x7ff7c0071a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50536/0 (socket says 172.21.15.45:50536) 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7b7e00640 1 -- 172.21.15.45:0/1135878921 >> v1:172.21.15.117:6789/0 conn(0x7ff7c00ce030 legacy=0x7ff7c006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7b7e00640 1 -- 172.21.15.45:0/1135878921 >> v1:172.21.15.45:6789/0 conn(0x7ff7c0144270 legacy=0x7ff7c0071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7b7e00640 1 -- 172.21.15.45:0/1135878921 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c0071240 con 0x7ff7c00cdba0 2024-04-25T10:25:23.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.412+0000 7ff7bfe00640 1 -- 172.21.15.45:0/1135878921 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060843132 0 0) 0x7ff7c0072110 con 0x7ff7c0144270 2024-04-25T10:25:23.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7b7e00640 1 --2- 172.21.15.45:0/1135878921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00cdba0 0x7ff7c00b5850 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800bb10 tx=0x7ff7a801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc3f9de3785e7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7bfe00640 1 -- 172.21.15.45:0/1135878921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80b4070 con 0x7ff7c00cdba0 2024-04-25T10:25:23.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7bfe00640 1 -- 172.21.15.45:0/1135878921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a80a7b40 con 0x7ff7c00cdba0 2024-04-25T10:25:23.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7bfe00640 1 -- 172.21.15.45:0/1135878921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80af440 con 0x7ff7c00cdba0 2024-04-25T10:25:23.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 -- 172.21.15.45:0/1135878921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00cdba0 msgr2=0x7ff7c00b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 --2- 172.21.15.45:0/1135878921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00cdba0 0x7ff7c00b5850 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800bb10 tx=0x7ff7a801eb70 comp rx=0 tx=0).stop 2024-04-25T10:25:23.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 -- 172.21.15.45:0/1135878921 shutdown_connections 2024-04-25T10:25:23.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 --2- 172.21.15.45:0/1135878921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00cdba0 0x7ff7c00b5850 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 -- 172.21.15.45:0/1135878921 >> 172.21.15.45:0/1135878921 conn(0x7ff7c00d0690 msgr2=0x7ff7c00d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 -- 172.21.15.45:0/1135878921 shutdown_connections 2024-04-25T10:25:23.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 -- 172.21.15.45:0/1135878921 wait complete. 2024-04-25T10:25:23.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 Processor -- start 2024-04-25T10:25:23.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.416+0000 7ff7c6600640 1 -- start start 2024-04-25T10:25:23.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7c00cdba0 0x7ff7c015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 0x7ff7c0141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7c0144270 0x7ff7c00b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c006ecb0 con 0x7ff7c00ce030 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00b61a0 con 0x7ff7c0144270 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c006ee30 con 0x7ff7c00cdba0 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 0x7ff7c0141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 0x7ff7c0141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52146/0 (socket says 172.21.15.45:52146) 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 -- 172.21.15.45:0/2247373736 learned_addr learned my addr 172.21.15.45:0/2247373736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c5c00640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7c0144270 0x7ff7c00b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c5200640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7c00cdba0 0x7ff7c015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 -- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7c00cdba0 msgr2=0x7ff7c015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7c00cdba0 0x7ff7c015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 -- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7c0144270 msgr2=0x7ff7c00b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7c0144270 0x7ff7c00b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 -- 172.21.15.45:0/2247373736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c015aeb0 con 0x7ff7c00ce030 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7b7e00640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 0x7ff7c0141f50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800b2c0 tx=0x7ff7a801b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80bf070 con 0x7ff7c00ce030 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a80b6050 con 0x7ff7c00ce030 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80b4030 con 0x7ff7c00ce030 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c00c5750 con 0x7ff7c00ce030 2024-04-25T10:25:23.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c00c5d10 con 0x7ff7c00ce030 2024-04-25T10:25:23.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c0075120 con 0x7ff7c00ce030 2024-04-25T10:25:23.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.420+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7a80c2070 con 0x7ff7c00ce030 2024-04-25T10:25:23.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.424+0000 7ff7bd600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff79c066f60 0x7ff79c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:23.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.424+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7a811f270 con 0x7ff7c00ce030 2024-04-25T10:25:23.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.424+0000 7ff7c5200640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff79c066f60 0x7ff79c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:23.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.424+0000 7ff7c5200640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff79c066f60 0x7ff79c069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0007fd0 tx=0x7ff7b0007ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:23.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.428+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7c015aeb0 con 0x7ff7c00ce030 2024-04-25T10:25:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.620+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7c006c2d0 con 0x7ff7c00ce030 2024-04-25T10:25:23.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7bd600640 1 -- 172.21.15.45:0/2247373736 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff7a8018820 con 0x7ff7c00ce030 2024-04-25T10:25:23.624 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:25:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff79c066f60 msgr2=0x7ff79c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff79c066f60 0x7ff79c069420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0007fd0 tx=0x7ff7b0007ba0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 msgr2=0x7ff7c0141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 0x7ff7c0141f50 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800b2c0 tx=0x7ff7a801b910 comp rx=0 tx=0).stop 2024-04-25T10:25:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 shutdown_connections 2024-04-25T10:25:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff79c066f60 0x7ff79c069420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7c0144270 0x7ff7c00b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7c00ce030 0x7ff7c0141f50 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 --2- 172.21.15.45:0/2247373736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7c00cdba0 0x7ff7c015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 >> 172.21.15.45:0/2247373736 conn(0x7ff7c00d0690 msgr2=0x7ff7c00d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 shutdown_connections 2024-04-25T10:25:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:23.624+0000 7ff7c6600640 1 -- 172.21.15.45:0/2247373736 wait complete. 2024-04-25T10:25:23.645 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:25:23.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:25:23.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:25:23.645 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:25:23.646 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:23.646 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:25:23.646 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:25:23.646 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:25:23.646 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:25:23.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:23.646 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:25:23.655 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:25:23.655 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:25:23.655 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:25:23.655 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.655 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:25:23.655 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42016sec preferred_lft 42016sec 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:25:23.656 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:25:23.657 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:25:23.658 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:25:23.658 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.658 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:25:23.658 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:25:23.658 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:25:23.658 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:25:23.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:23.659 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:25:23.725 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:25:23.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:23.727 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:25:23.784 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:25:23.785 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:25:23.785 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:25:23.785 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:25:23.785 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:25:23.785 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:25:23.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:23.785 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:25:23.785 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:23.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:25:23.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:25:23.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:25:23.786 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:23.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:23.881 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:25:23.891 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:25:23.891 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:25:23.891 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:25:23.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:23.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:25:23.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:25:23.955 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:24.007 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:25:24.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:25:24.007 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:24.059 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:25:24.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:24.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:25:24.152 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:25:24.152 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:25:24.167 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:25:24.167 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:25:24.170 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:25:24.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:24.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:24.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:24.191 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:25:24.256 INFO:teuthology.orchestra.run.smithi045.stdout:client7513 2024-04-25T10:25:24.256 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:24.256 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:24.312 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:24.312 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T10:25:24.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:24.312 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:24.312 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:24.312 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:25:24.312 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:24.312 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:24.312 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:24.313 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222392 s, 47.1 MB/s 2024-04-25T10:25:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155705 s, 67.3 MB/s 2024-04-25T10:25:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161793 s, 64.8 MB/s 2024-04-25T10:25:24.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184361 s, 56.9 MB/s 2024-04-25T10:25:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149389 s, 70.2 MB/s 2024-04-25T10:25:24.493 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.493 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.493 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161412 s, 65.0 MB/s 2024-04-25T10:25:24.516 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.517 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215522 s, 48.7 MB/s 2024-04-25T10:25:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193168 s, 54.3 MB/s 2024-04-25T10:25:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134531 s, 77.9 MB/s 2024-04-25T10:25:24.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149489 s, 70.1 MB/s 2024-04-25T10:25:24.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188152 s, 55.7 MB/s 2024-04-25T10:25:24.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143415 s, 73.1 MB/s 2024-04-25T10:25:24.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.623 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145912 s, 71.9 MB/s 2024-04-25T10:25:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153618 s, 68.3 MB/s 2024-04-25T10:25:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.658 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166113 s, 63.1 MB/s 2024-04-25T10:25:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168742 s, 62.1 MB/s 2024-04-25T10:25:24.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151081 s, 69.4 MB/s 2024-04-25T10:25:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144504 s, 72.6 MB/s 2024-04-25T10:25:24.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158994 s, 66.0 MB/s 2024-04-25T10:25:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.745 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149441 s, 70.2 MB/s 2024-04-25T10:25:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.762 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016235 s, 64.6 MB/s 2024-04-25T10:25:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167523 s, 62.6 MB/s 2024-04-25T10:25:24.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160512 s, 65.3 MB/s 2024-04-25T10:25:24.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.816 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158358 s, 66.2 MB/s 2024-04-25T10:25:24.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.833 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145042 s, 72.3 MB/s 2024-04-25T10:25:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189934 s, 55.2 MB/s 2024-04-25T10:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.869 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141014 s, 74.4 MB/s 2024-04-25T10:25:24.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.887 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154341 s, 67.9 MB/s 2024-04-25T10:25:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185763 s, 56.4 MB/s 2024-04-25T10:25:24.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132619 s, 79.1 MB/s 2024-04-25T10:25:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146125 s, 71.8 MB/s 2024-04-25T10:25:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:24.955 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145245 s, 72.2 MB/s 2024-04-25T10:25:24.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:24.956 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:25.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.029 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161327 s, 65.0 MB/s 2024-04-25T10:25:25.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.046 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146923 s, 71.4 MB/s 2024-04-25T10:25:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157051 s, 66.8 MB/s 2024-04-25T10:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141087 s, 74.3 MB/s 2024-04-25T10:25:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018549 s, 56.5 MB/s 2024-04-25T10:25:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150692 s, 69.6 MB/s 2024-04-25T10:25:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150371 s, 69.7 MB/s 2024-04-25T10:25:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016048 s, 65.3 MB/s 2024-04-25T10:25:25.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.168 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014814 s, 70.8 MB/s 2024-04-25T10:25:25.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184268 s, 56.9 MB/s 2024-04-25T10:25:25.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157933 s, 66.4 MB/s 2024-04-25T10:25:25.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.221 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.221 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013604 s, 77.1 MB/s 2024-04-25T10:25:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149581 s, 70.1 MB/s 2024-04-25T10:25:25.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196598 s, 53.3 MB/s 2024-04-25T10:25:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.276 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159794 s, 65.6 MB/s 2024-04-25T10:25:25.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146637 s, 71.5 MB/s 2024-04-25T10:25:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134094 s, 78.2 MB/s 2024-04-25T10:25:25.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155784 s, 67.3 MB/s 2024-04-25T10:25:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163867 s, 64.0 MB/s 2024-04-25T10:25:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142202 s, 73.7 MB/s 2024-04-25T10:25:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.378 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158695 s, 66.1 MB/s 2024-04-25T10:25:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184451 s, 56.8 MB/s 2024-04-25T10:25:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012819 s, 81.8 MB/s 2024-04-25T10:25:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147902 s, 70.9 MB/s 2024-04-25T10:25:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147436 s, 71.1 MB/s 2024-04-25T10:25:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178622 s, 58.7 MB/s 2024-04-25T10:25:25.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195099 s, 53.7 MB/s 2024-04-25T10:25:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015131 s, 69.3 MB/s 2024-04-25T10:25:25.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018438 s, 56.9 MB/s 2024-04-25T10:25:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.540 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145637 s, 72.0 MB/s 2024-04-25T10:25:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142352 s, 73.7 MB/s 2024-04-25T10:25:25.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.574 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156187 s, 67.1 MB/s 2024-04-25T10:25:25.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:25.575 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:25.575 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:25.575 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:25:25.575 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:25.575 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:25.575 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:25.576 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:25.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.648 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014981 s, 70.0 MB/s 2024-04-25T10:25:25.664 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149486 s, 70.1 MB/s 2024-04-25T10:25:25.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153375 s, 68.4 MB/s 2024-04-25T10:25:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139602 s, 75.1 MB/s 2024-04-25T10:25:25.713 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.713 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.713 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140909 s, 74.4 MB/s 2024-04-25T10:25:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166777 s, 62.9 MB/s 2024-04-25T10:25:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151272 s, 69.3 MB/s 2024-04-25T10:25:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.765 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146152 s, 71.7 MB/s 2024-04-25T10:25:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015167 s, 69.1 MB/s 2024-04-25T10:25:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.798 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143134 s, 73.3 MB/s 2024-04-25T10:25:25.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144184 s, 72.7 MB/s 2024-04-25T10:25:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148682 s, 70.5 MB/s 2024-04-25T10:25:25.849 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.849 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.849 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015435 s, 67.9 MB/s 2024-04-25T10:25:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142326 s, 73.7 MB/s 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145854 s, 71.9 MB/s 2024-04-25T10:25:25.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.898 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154631 s, 67.8 MB/s 2024-04-25T10:25:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150901 s, 69.5 MB/s 2024-04-25T10:25:25.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.933 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161113 s, 65.1 MB/s 2024-04-25T10:25:25.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147618 s, 71.0 MB/s 2024-04-25T10:25:25.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.965 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144704 s, 72.5 MB/s 2024-04-25T10:25:25.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:25.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:25.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156738 s, 66.9 MB/s 2024-04-25T10:25:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.000 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014687 s, 71.4 MB/s 2024-04-25T10:25:26.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139607 s, 75.1 MB/s 2024-04-25T10:25:26.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145096 s, 72.3 MB/s 2024-04-25T10:25:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146408 s, 71.6 MB/s 2024-04-25T10:25:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.064 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144594 s, 72.5 MB/s 2024-04-25T10:25:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145997 s, 71.8 MB/s 2024-04-25T10:25:26.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169015 s, 62.0 MB/s 2024-04-25T10:25:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139863 s, 75.0 MB/s 2024-04-25T10:25:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142719 s, 73.5 MB/s 2024-04-25T10:25:26.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.147 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151011 s, 69.4 MB/s 2024-04-25T10:25:26.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.163 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139426 s, 75.2 MB/s 2024-04-25T10:25:26.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:26.164 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:26.164 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:26.165 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:25:26.165 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:26.165 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:26.165 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:26.165 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136569 s, 76.8 MB/s 2024-04-25T10:25:26.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.250 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135275 s, 77.5 MB/s 2024-04-25T10:25:26.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.267 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150326 s, 69.8 MB/s 2024-04-25T10:25:26.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.282 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139179 s, 75.3 MB/s 2024-04-25T10:25:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188403 s, 55.7 MB/s 2024-04-25T10:25:26.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.319 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145147 s, 72.2 MB/s 2024-04-25T10:25:26.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.334 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131128 s, 80.0 MB/s 2024-04-25T10:25:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.356 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203482 s, 51.5 MB/s 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140411 s, 74.7 MB/s 2024-04-25T10:25:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235892 s, 44.5 MB/s 2024-04-25T10:25:26.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013776 s, 76.1 MB/s 2024-04-25T10:25:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131832 s, 79.5 MB/s 2024-04-25T10:25:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139728 s, 75.0 MB/s 2024-04-25T10:25:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.462 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148783 s, 70.5 MB/s 2024-04-25T10:25:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.476 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013285 s, 78.9 MB/s 2024-04-25T10:25:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.494 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156223 s, 67.1 MB/s 2024-04-25T10:25:26.511 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156389 s, 67.0 MB/s 2024-04-25T10:25:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.530 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167363 s, 62.7 MB/s 2024-04-25T10:25:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139922 s, 74.9 MB/s 2024-04-25T10:25:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143191 s, 73.2 MB/s 2024-04-25T10:25:26.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145139 s, 72.2 MB/s 2024-04-25T10:25:26.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.596 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152763 s, 68.6 MB/s 2024-04-25T10:25:26.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014866 s, 70.5 MB/s 2024-04-25T10:25:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142959 s, 73.3 MB/s 2024-04-25T10:25:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130838 s, 80.1 MB/s 2024-04-25T10:25:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.660 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148886 s, 70.4 MB/s 2024-04-25T10:25:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014617 s, 71.7 MB/s 2024-04-25T10:25:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124459 s, 84.3 MB/s 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.707 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144235 s, 72.7 MB/s 2024-04-25T10:25:26.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.722 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130384 s, 80.4 MB/s 2024-04-25T10:25:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161553 s, 64.9 MB/s 2024-04-25T10:25:26.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.757 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146697 s, 71.5 MB/s 2024-04-25T10:25:26.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:26.758 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:26.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.828 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137408 s, 76.3 MB/s 2024-04-25T10:25:26.848 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.848 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.848 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183582 s, 57.1 MB/s 2024-04-25T10:25:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132087 s, 79.4 MB/s 2024-04-25T10:25:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136145 s, 77.0 MB/s 2024-04-25T10:25:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135965 s, 77.1 MB/s 2024-04-25T10:25:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.911 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156221 s, 67.1 MB/s 2024-04-25T10:25:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188543 s, 55.6 MB/s 2024-04-25T10:25:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013552 s, 77.4 MB/s 2024-04-25T10:25:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.962 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012824 s, 81.8 MB/s 2024-04-25T10:25:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015031 s, 69.8 MB/s 2024-04-25T10:25:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017882 s, 58.6 MB/s 2024-04-25T10:25:27.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138933 s, 75.5 MB/s 2024-04-25T10:25:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135568 s, 77.3 MB/s 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.046 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146634 s, 71.5 MB/s 2024-04-25T10:25:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181222 s, 57.9 MB/s 2024-04-25T10:25:27.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015139 s, 69.3 MB/s 2024-04-25T10:25:27.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139366 s, 75.2 MB/s 2024-04-25T10:25:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.114 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138564 s, 75.7 MB/s 2024-04-25T10:25:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.135 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189488 s, 55.3 MB/s 2024-04-25T10:25:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137093 s, 76.5 MB/s 2024-04-25T10:25:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169937 s, 61.7 MB/s 2024-04-25T10:25:27.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.185 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01366 s, 76.8 MB/s 2024-04-25T10:25:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184832 s, 56.7 MB/s 2024-04-25T10:25:27.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159634 s, 65.7 MB/s 2024-04-25T10:25:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152296 s, 68.9 MB/s 2024-04-25T10:25:27.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.257 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150177 s, 69.8 MB/s 2024-04-25T10:25:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.273 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146836 s, 71.4 MB/s 2024-04-25T10:25:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.290 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147377 s, 71.1 MB/s 2024-04-25T10:25:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.310 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189144 s, 55.4 MB/s 2024-04-25T10:25:27.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.325 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131789 s, 79.6 MB/s 2024-04-25T10:25:27.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.341 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143703 s, 73.0 MB/s 2024-04-25T10:25:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131719 s, 79.6 MB/s 2024-04-25T10:25:27.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:27.358 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:27.358 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:27.358 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:25:27.358 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:27.359 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:27.359 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:27.359 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013903 s, 75.4 MB/s 2024-04-25T10:25:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187387 s, 56.0 MB/s 2024-04-25T10:25:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150141 s, 69.8 MB/s 2024-04-25T10:25:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141816 s, 73.9 MB/s 2024-04-25T10:25:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151482 s, 69.2 MB/s 2024-04-25T10:25:27.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150791 s, 69.5 MB/s 2024-04-25T10:25:27.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.530 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136742 s, 76.7 MB/s 2024-04-25T10:25:27.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013544 s, 77.4 MB/s 2024-04-25T10:25:27.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150321 s, 69.8 MB/s 2024-04-25T10:25:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139787 s, 75.0 MB/s 2024-04-25T10:25:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144745 s, 72.4 MB/s 2024-04-25T10:25:27.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.615 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184838 s, 56.7 MB/s 2024-04-25T10:25:27.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152103 s, 68.9 MB/s 2024-04-25T10:25:27.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.648 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141418 s, 74.1 MB/s 2024-04-25T10:25:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141937 s, 73.9 MB/s 2024-04-25T10:25:27.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149423 s, 70.2 MB/s 2024-04-25T10:25:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017926 s, 58.5 MB/s 2024-04-25T10:25:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.717 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151438 s, 69.2 MB/s 2024-04-25T10:25:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144471 s, 72.6 MB/s 2024-04-25T10:25:27.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.748 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133494 s, 78.5 MB/s 2024-04-25T10:25:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138484 s, 75.7 MB/s 2024-04-25T10:25:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.777 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115364 s, 90.9 MB/s 2024-04-25T10:25:27.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01334 s, 78.6 MB/s 2024-04-25T10:25:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018602 s, 56.4 MB/s 2024-04-25T10:25:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144569 s, 72.5 MB/s 2024-04-25T10:25:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153549 s, 68.3 MB/s 2024-04-25T10:25:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172276 s, 60.9 MB/s 2024-04-25T10:25:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.882 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015185 s, 69.1 MB/s 2024-04-25T10:25:27.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.899 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147256 s, 71.2 MB/s 2024-04-25T10:25:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138316 s, 75.8 MB/s 2024-04-25T10:25:27.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152177 s, 68.9 MB/s 2024-04-25T10:25:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133536 s, 78.5 MB/s 2024-04-25T10:25:27.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:27.948 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:28.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133866 s, 78.3 MB/s 2024-04-25T10:25:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.035 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155594 s, 67.4 MB/s 2024-04-25T10:25:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140705 s, 74.5 MB/s 2024-04-25T10:25:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.067 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139423 s, 75.2 MB/s 2024-04-25T10:25:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.087 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182915 s, 57.3 MB/s 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141834 s, 73.9 MB/s 2024-04-25T10:25:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137158 s, 76.5 MB/s 2024-04-25T10:25:28.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.135 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147919 s, 70.9 MB/s 2024-04-25T10:25:28.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186941 s, 56.1 MB/s 2024-04-25T10:25:28.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117354 s, 89.4 MB/s 2024-04-25T10:25:28.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133435 s, 78.6 MB/s 2024-04-25T10:25:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176796 s, 59.3 MB/s 2024-04-25T10:25:28.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133608 s, 78.5 MB/s 2024-04-25T10:25:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190506 s, 55.0 MB/s 2024-04-25T10:25:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147278 s, 71.2 MB/s 2024-04-25T10:25:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171978 s, 61.0 MB/s 2024-04-25T10:25:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151994 s, 69.0 MB/s 2024-04-25T10:25:28.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.309 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150865 s, 69.5 MB/s 2024-04-25T10:25:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164873 s, 63.6 MB/s 2024-04-25T10:25:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132401 s, 79.2 MB/s 2024-04-25T10:25:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013563 s, 77.3 MB/s 2024-04-25T10:25:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165394 s, 63.4 MB/s 2024-04-25T10:25:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151343 s, 69.3 MB/s 2024-04-25T10:25:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129864 s, 80.7 MB/s 2024-04-25T10:25:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142477 s, 73.6 MB/s 2024-04-25T10:25:28.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.445 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018807 s, 55.8 MB/s 2024-04-25T10:25:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.461 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147285 s, 71.2 MB/s 2024-04-25T10:25:28.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.478 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149222 s, 70.3 MB/s 2024-04-25T10:25:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151659 s, 69.1 MB/s 2024-04-25T10:25:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.510 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133597 s, 78.5 MB/s 2024-04-25T10:25:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151272 s, 69.3 MB/s 2024-04-25T10:25:28.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140585 s, 74.6 MB/s 2024-04-25T10:25:28.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:28.544 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:28.544 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:28.544 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:25:28.544 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:28.544 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:28.545 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:28.545 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:28.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.614 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131157 s, 79.9 MB/s 2024-04-25T10:25:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.631 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149036 s, 70.4 MB/s 2024-04-25T10:25:28.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.648 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150347 s, 69.7 MB/s 2024-04-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176594 s, 59.4 MB/s 2024-04-25T10:25:28.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014436 s, 72.6 MB/s 2024-04-25T10:25:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179734 s, 58.3 MB/s 2024-04-25T10:25:28.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.719 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136714 s, 76.7 MB/s 2024-04-25T10:25:28.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151827 s, 69.1 MB/s 2024-04-25T10:25:28.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.752 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145486 s, 72.1 MB/s 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133152 s, 78.8 MB/s 2024-04-25T10:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179689 s, 58.4 MB/s 2024-04-25T10:25:28.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138511 s, 75.7 MB/s 2024-04-25T10:25:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141489 s, 74.1 MB/s 2024-04-25T10:25:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130828 s, 80.1 MB/s 2024-04-25T10:25:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153338 s, 68.4 MB/s 2024-04-25T10:25:28.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.868 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149038 s, 70.4 MB/s 2024-04-25T10:25:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132399 s, 79.2 MB/s 2024-04-25T10:25:28.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.903 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185096 s, 56.7 MB/s 2024-04-25T10:25:28.918 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.918 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.918 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134678 s, 77.9 MB/s 2024-04-25T10:25:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136081 s, 77.1 MB/s 2024-04-25T10:25:28.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.950 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014523 s, 72.2 MB/s 2024-04-25T10:25:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132453 s, 79.2 MB/s 2024-04-25T10:25:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124754 s, 84.1 MB/s 2024-04-25T10:25:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150965 s, 69.5 MB/s 2024-04-25T10:25:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140788 s, 74.5 MB/s 2024-04-25T10:25:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178625 s, 58.7 MB/s 2024-04-25T10:25:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151372 s, 69.3 MB/s 2024-04-25T10:25:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.065 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142523 s, 73.6 MB/s 2024-04-25T10:25:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188103 s, 55.7 MB/s 2024-04-25T10:25:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150232 s, 69.8 MB/s 2024-04-25T10:25:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132959 s, 78.9 MB/s 2024-04-25T10:25:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181781 s, 57.7 MB/s 2024-04-25T10:25:29.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:29.139 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155101 s, 67.6 MB/s 2024-04-25T10:25:29.233 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.233 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.233 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188018 s, 55.8 MB/s 2024-04-25T10:25:29.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.250 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143806 s, 72.9 MB/s 2024-04-25T10:25:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175649 s, 59.7 MB/s 2024-04-25T10:25:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.285 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148311 s, 70.7 MB/s 2024-04-25T10:25:29.302 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.302 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.302 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014857 s, 70.6 MB/s 2024-04-25T10:25:29.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142072 s, 73.8 MB/s 2024-04-25T10:25:29.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.338 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184943 s, 56.7 MB/s 2024-04-25T10:25:29.354 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.355 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143617 s, 73.0 MB/s 2024-04-25T10:25:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172856 s, 60.7 MB/s 2024-04-25T10:25:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.390 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141686 s, 74.0 MB/s 2024-04-25T10:25:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138932 s, 75.5 MB/s 2024-04-25T10:25:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137691 s, 76.2 MB/s 2024-04-25T10:25:29.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:29.434 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209421 s, 50.1 MB/s 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.461 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155905 s, 67.3 MB/s 2024-04-25T10:25:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140844 s, 74.4 MB/s 2024-04-25T10:25:29.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136708 s, 76.7 MB/s 2024-04-25T10:25:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148712 s, 70.5 MB/s 2024-04-25T10:25:29.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.525 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014145 s, 74.1 MB/s 2024-04-25T10:25:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129681 s, 80.9 MB/s 2024-04-25T10:25:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187968 s, 55.8 MB/s 2024-04-25T10:25:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138788 s, 75.6 MB/s 2024-04-25T10:25:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185095 s, 56.7 MB/s 2024-04-25T10:25:29.613 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.613 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.613 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142841 s, 73.4 MB/s 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157044 s, 66.8 MB/s 2024-04-25T10:25:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149324 s, 70.2 MB/s 2024-04-25T10:25:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183916 s, 57.0 MB/s 2024-04-25T10:25:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148398 s, 70.7 MB/s 2024-04-25T10:25:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121365 s, 86.4 MB/s 2024-04-25T10:25:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150422 s, 69.7 MB/s 2024-04-25T10:25:29.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140055 s, 74.9 MB/s 2024-04-25T10:25:29.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.748 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158348 s, 66.2 MB/s 2024-04-25T10:25:29.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:29.750 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:29.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152477 s, 68.8 MB/s 2024-04-25T10:25:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144981 s, 72.3 MB/s 2024-04-25T10:25:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172758 s, 60.7 MB/s 2024-04-25T10:25:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142929 s, 73.4 MB/s 2024-04-25T10:25:29.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.887 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131247 s, 79.9 MB/s 2024-04-25T10:25:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013173 s, 79.6 MB/s 2024-04-25T10:25:29.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.919 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01534 s, 68.4 MB/s 2024-04-25T10:25:29.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.935 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141558 s, 74.1 MB/s 2024-04-25T10:25:29.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.951 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143086 s, 73.3 MB/s 2024-04-25T10:25:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.968 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150872 s, 69.5 MB/s 2024-04-25T10:25:29.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137758 s, 76.1 MB/s 2024-04-25T10:25:29.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:29.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:29.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136543 s, 76.8 MB/s 2024-04-25T10:25:30.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138848 s, 75.5 MB/s 2024-04-25T10:25:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135659 s, 77.3 MB/s 2024-04-25T10:25:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153824 s, 68.2 MB/s 2024-04-25T10:25:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.064 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148631 s, 70.5 MB/s 2024-04-25T10:25:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013229 s, 79.3 MB/s 2024-04-25T10:25:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150855 s, 69.5 MB/s 2024-04-25T10:25:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.111 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138525 s, 75.7 MB/s 2024-04-25T10:25:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140725 s, 74.5 MB/s 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182902 s, 57.3 MB/s 2024-04-25T10:25:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.165 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.165 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155908 s, 67.3 MB/s 2024-04-25T10:25:30.181 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.181 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.181 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149605 s, 70.1 MB/s 2024-04-25T10:25:30.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.197 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130899 s, 80.1 MB/s 2024-04-25T10:25:30.214 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.215 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164385 s, 63.8 MB/s 2024-04-25T10:25:30.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.231 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143166 s, 73.2 MB/s 2024-04-25T10:25:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.251 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186966 s, 56.1 MB/s 2024-04-25T10:25:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150727 s, 69.6 MB/s 2024-04-25T10:25:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159036 s, 65.9 MB/s 2024-04-25T10:25:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121664 s, 86.2 MB/s 2024-04-25T10:25:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179732 s, 58.3 MB/s 2024-04-25T10:25:30.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.337 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152641 s, 68.7 MB/s 2024-04-25T10:25:30.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:30.338 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149142 s, 70.3 MB/s 2024-04-25T10:25:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130797 s, 80.2 MB/s 2024-04-25T10:25:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167911 s, 62.4 MB/s 2024-04-25T10:25:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150752 s, 69.6 MB/s 2024-04-25T10:25:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141668 s, 74.0 MB/s 2024-04-25T10:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.491 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151022 s, 69.4 MB/s 2024-04-25T10:25:30.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130224 s, 80.5 MB/s 2024-04-25T10:25:30.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154903 s, 67.7 MB/s 2024-04-25T10:25:30.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140264 s, 74.8 MB/s 2024-04-25T10:25:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.555 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143007 s, 73.3 MB/s 2024-04-25T10:25:30.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133156 s, 78.7 MB/s 2024-04-25T10:25:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.587 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141464 s, 74.1 MB/s 2024-04-25T10:25:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178682 s, 58.7 MB/s 2024-04-25T10:25:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131251 s, 79.9 MB/s 2024-04-25T10:25:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154662 s, 67.8 MB/s 2024-04-25T10:25:30.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.655 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142257 s, 73.7 MB/s 2024-04-25T10:25:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.672 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154825 s, 67.7 MB/s 2024-04-25T10:25:30.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.689 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152037 s, 69.0 MB/s 2024-04-25T10:25:30.704 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.704 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.705 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137135 s, 76.5 MB/s 2024-04-25T10:25:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147272 s, 71.2 MB/s 2024-04-25T10:25:30.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.738 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149012 s, 70.4 MB/s 2024-04-25T10:25:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135441 s, 77.4 MB/s 2024-04-25T10:25:30.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.769 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139687 s, 75.1 MB/s 2024-04-25T10:25:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146394 s, 71.6 MB/s 2024-04-25T10:25:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.801 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.801 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013684 s, 76.6 MB/s 2024-04-25T10:25:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.817 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014984 s, 70.0 MB/s 2024-04-25T10:25:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147686 s, 71.0 MB/s 2024-04-25T10:25:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141222 s, 74.3 MB/s 2024-04-25T10:25:30.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.866 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141781 s, 74.0 MB/s 2024-04-25T10:25:30.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.882 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142816 s, 73.4 MB/s 2024-04-25T10:25:30.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.898 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149626 s, 70.1 MB/s 2024-04-25T10:25:30.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.914 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141542 s, 74.1 MB/s 2024-04-25T10:25:30.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:30.916 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:30.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:30.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:30.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140483 s, 74.6 MB/s 2024-04-25T10:25:31.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.003 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149762 s, 70.0 MB/s 2024-04-25T10:25:31.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.019 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133255 s, 78.7 MB/s 2024-04-25T10:25:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147637 s, 71.0 MB/s 2024-04-25T10:25:31.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014212 s, 73.8 MB/s 2024-04-25T10:25:31.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.067 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133356 s, 78.6 MB/s 2024-04-25T10:25:31.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.084 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148347 s, 70.7 MB/s 2024-04-25T10:25:31.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.100 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147476 s, 71.1 MB/s 2024-04-25T10:25:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132832 s, 78.9 MB/s 2024-04-25T10:25:31.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145983 s, 71.8 MB/s 2024-04-25T10:25:31.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.147 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139435 s, 75.2 MB/s 2024-04-25T10:25:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.162 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131988 s, 79.4 MB/s 2024-04-25T10:25:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014732 s, 71.2 MB/s 2024-04-25T10:25:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.195 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150308 s, 69.8 MB/s 2024-04-25T10:25:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.211 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014175 s, 74.0 MB/s 2024-04-25T10:25:31.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147011 s, 71.3 MB/s 2024-04-25T10:25:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143528 s, 73.1 MB/s 2024-04-25T10:25:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131872 s, 79.5 MB/s 2024-04-25T10:25:31.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.279 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184099 s, 57.0 MB/s 2024-04-25T10:25:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144602 s, 72.5 MB/s 2024-04-25T10:25:31.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.315 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018148 s, 57.8 MB/s 2024-04-25T10:25:31.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.333 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163674 s, 64.1 MB/s 2024-04-25T10:25:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138659 s, 75.6 MB/s 2024-04-25T10:25:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136452 s, 76.8 MB/s 2024-04-25T10:25:31.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186096 s, 56.3 MB/s 2024-04-25T10:25:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.402 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156621 s, 66.9 MB/s 2024-04-25T10:25:31.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149288 s, 70.2 MB/s 2024-04-25T10:25:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.434 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136132 s, 77.0 MB/s 2024-04-25T10:25:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148207 s, 70.8 MB/s 2024-04-25T10:25:31.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.466 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133283 s, 78.7 MB/s 2024-04-25T10:25:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.483 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152832 s, 68.6 MB/s 2024-04-25T10:25:31.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.504 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184929 s, 56.7 MB/s 2024-04-25T10:25:31.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:31.505 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:31.505 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:31.505 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:25:31.505 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:31.505 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:31.506 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:31.506 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.574 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132617 s, 79.1 MB/s 2024-04-25T10:25:31.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183783 s, 57.1 MB/s 2024-04-25T10:25:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135657 s, 77.3 MB/s 2024-04-25T10:25:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133468 s, 78.6 MB/s 2024-04-25T10:25:31.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154766 s, 67.8 MB/s 2024-04-25T10:25:31.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188969 s, 55.5 MB/s 2024-04-25T10:25:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.678 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135836 s, 77.2 MB/s 2024-04-25T10:25:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184153 s, 56.9 MB/s 2024-04-25T10:25:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01374 s, 76.3 MB/s 2024-04-25T10:25:31.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.730 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143995 s, 72.8 MB/s 2024-04-25T10:25:31.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.748 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154034 s, 68.1 MB/s 2024-04-25T10:25:31.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132841 s, 78.9 MB/s 2024-04-25T10:25:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.782 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175555 s, 59.7 MB/s 2024-04-25T10:25:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150508 s, 69.7 MB/s 2024-04-25T10:25:31.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137033 s, 76.5 MB/s 2024-04-25T10:25:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.832 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149805 s, 70.0 MB/s 2024-04-25T10:25:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129998 s, 80.7 MB/s 2024-04-25T10:25:31.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.866 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179347 s, 58.5 MB/s 2024-04-25T10:25:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.882 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013786 s, 76.1 MB/s 2024-04-25T10:25:31.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.897 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141924 s, 73.9 MB/s 2024-04-25T10:25:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.913 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013827 s, 75.8 MB/s 2024-04-25T10:25:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141105 s, 74.3 MB/s 2024-04-25T10:25:31.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.945 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145401 s, 72.1 MB/s 2024-04-25T10:25:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.961 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013413 s, 78.2 MB/s 2024-04-25T10:25:31.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.980 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179791 s, 58.3 MB/s 2024-04-25T10:25:31.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:31.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:31.997 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152725 s, 68.7 MB/s 2024-04-25T10:25:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133415 s, 78.6 MB/s 2024-04-25T10:25:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129251 s, 81.1 MB/s 2024-04-25T10:25:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188154 s, 55.7 MB/s 2024-04-25T10:25:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.068 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184106 s, 57.0 MB/s 2024-04-25T10:25:32.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162437 s, 64.6 MB/s 2024-04-25T10:25:32.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171612 s, 61.1 MB/s 2024-04-25T10:25:32.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:32.106 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:32.107 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:32.107 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:25:32.107 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:32.107 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:32.107 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:32.107 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149904 s, 69.9 MB/s 2024-04-25T10:25:32.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.197 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190075 s, 55.2 MB/s 2024-04-25T10:25:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144586 s, 72.5 MB/s 2024-04-25T10:25:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137867 s, 76.1 MB/s 2024-04-25T10:25:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.246 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160275 s, 65.4 MB/s 2024-04-25T10:25:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175705 s, 59.7 MB/s 2024-04-25T10:25:32.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.282 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144416 s, 72.6 MB/s 2024-04-25T10:25:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.298 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144307 s, 72.7 MB/s 2024-04-25T10:25:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159895 s, 65.6 MB/s 2024-04-25T10:25:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146778 s, 71.4 MB/s 2024-04-25T10:25:32.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.348 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143724 s, 73.0 MB/s 2024-04-25T10:25:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.370 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.370 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191548 s, 54.7 MB/s 2024-04-25T10:25:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138783 s, 75.6 MB/s 2024-04-25T10:25:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137617 s, 76.2 MB/s 2024-04-25T10:25:32.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.416 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134422 s, 78.0 MB/s 2024-04-25T10:25:32.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.431 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131705 s, 79.6 MB/s 2024-04-25T10:25:32.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.447 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147872 s, 70.9 MB/s 2024-04-25T10:25:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139027 s, 75.4 MB/s 2024-04-25T10:25:32.479 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.479 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.480 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147499 s, 71.1 MB/s 2024-04-25T10:25:32.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134434 s, 78.0 MB/s 2024-04-25T10:25:32.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.510 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140823 s, 74.5 MB/s 2024-04-25T10:25:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.528 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151416 s, 69.3 MB/s 2024-04-25T10:25:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.547 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176241 s, 59.5 MB/s 2024-04-25T10:25:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146908 s, 71.4 MB/s 2024-04-25T10:25:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142997 s, 73.3 MB/s 2024-04-25T10:25:32.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.594 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128739 s, 81.4 MB/s 2024-04-25T10:25:32.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.611 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155831 s, 67.3 MB/s 2024-04-25T10:25:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179952 s, 58.3 MB/s 2024-04-25T10:25:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136597 s, 76.8 MB/s 2024-04-25T10:25:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114893 s, 91.3 MB/s 2024-04-25T10:25:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018616 s, 56.3 MB/s 2024-04-25T10:25:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.696 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014781 s, 70.9 MB/s 2024-04-25T10:25:32.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:32.698 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:32.698 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:32.698 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:25:32.698 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:32.698 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:32.698 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:32.699 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.771 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181371 s, 57.8 MB/s 2024-04-25T10:25:32.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.791 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183182 s, 57.2 MB/s 2024-04-25T10:25:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.806 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138346 s, 75.8 MB/s 2024-04-25T10:25:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.821 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128122 s, 81.8 MB/s 2024-04-25T10:25:32.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136758 s, 76.7 MB/s 2024-04-25T10:25:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143438 s, 73.1 MB/s 2024-04-25T10:25:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.871 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162392 s, 64.6 MB/s 2024-04-25T10:25:32.891 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.891 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.891 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186138 s, 56.3 MB/s 2024-04-25T10:25:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.906 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132409 s, 79.2 MB/s 2024-04-25T10:25:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142697 s, 73.5 MB/s 2024-04-25T10:25:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.939 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145312 s, 72.2 MB/s 2024-04-25T10:25:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131431 s, 79.8 MB/s 2024-04-25T10:25:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135356 s, 77.5 MB/s 2024-04-25T10:25:32.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:32.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:32.985 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144932 s, 72.3 MB/s 2024-04-25T10:25:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.001 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144132 s, 72.8 MB/s 2024-04-25T10:25:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131797 s, 79.6 MB/s 2024-04-25T10:25:33.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.033 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147283 s, 71.2 MB/s 2024-04-25T10:25:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.048 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139042 s, 75.4 MB/s 2024-04-25T10:25:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183481 s, 57.1 MB/s 2024-04-25T10:25:33.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134952 s, 77.7 MB/s 2024-04-25T10:25:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188272 s, 55.7 MB/s 2024-04-25T10:25:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011552 s, 90.8 MB/s 2024-04-25T10:25:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145969 s, 71.8 MB/s 2024-04-25T10:25:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151163 s, 69.4 MB/s 2024-04-25T10:25:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.172 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195216 s, 53.7 MB/s 2024-04-25T10:25:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151596 s, 69.2 MB/s 2024-04-25T10:25:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137825 s, 76.1 MB/s 2024-04-25T10:25:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144505 s, 72.6 MB/s 2024-04-25T10:25:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.236 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138294 s, 75.8 MB/s 2024-04-25T10:25:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017913 s, 58.5 MB/s 2024-04-25T10:25:33.271 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.272 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139346 s, 75.2 MB/s 2024-04-25T10:25:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194637 s, 53.9 MB/s 2024-04-25T10:25:33.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:33.294 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:33.295 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:33.295 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:25:33.295 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:33.295 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:33.295 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:33.295 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.363 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140018 s, 74.9 MB/s 2024-04-25T10:25:33.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.380 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152649 s, 68.7 MB/s 2024-04-25T10:25:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.400 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184167 s, 56.9 MB/s 2024-04-25T10:25:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.417 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151172 s, 69.4 MB/s 2024-04-25T10:25:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131922 s, 79.5 MB/s 2024-04-25T10:25:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017735 s, 59.1 MB/s 2024-04-25T10:25:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.468 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150933 s, 69.5 MB/s 2024-04-25T10:25:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146641 s, 71.5 MB/s 2024-04-25T10:25:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.500 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139408 s, 75.2 MB/s 2024-04-25T10:25:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.517 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148324 s, 70.7 MB/s 2024-04-25T10:25:33.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.534 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146458 s, 71.6 MB/s 2024-04-25T10:25:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140538 s, 74.6 MB/s 2024-04-25T10:25:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139019 s, 75.4 MB/s 2024-04-25T10:25:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150854 s, 69.5 MB/s 2024-04-25T10:25:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.598 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140886 s, 74.4 MB/s 2024-04-25T10:25:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147326 s, 71.2 MB/s 2024-04-25T10:25:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180591 s, 58.1 MB/s 2024-04-25T10:25:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175724 s, 59.7 MB/s 2024-04-25T10:25:33.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.668 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013447 s, 78.0 MB/s 2024-04-25T10:25:33.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139124 s, 75.4 MB/s 2024-04-25T10:25:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136079 s, 77.1 MB/s 2024-04-25T10:25:33.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.717 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149839 s, 70.0 MB/s 2024-04-25T10:25:33.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139564 s, 75.1 MB/s 2024-04-25T10:25:33.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.752 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018243 s, 57.5 MB/s 2024-04-25T10:25:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133541 s, 78.5 MB/s 2024-04-25T10:25:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.783 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138242 s, 75.9 MB/s 2024-04-25T10:25:33.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013946 s, 75.2 MB/s 2024-04-25T10:25:33.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134496 s, 78.0 MB/s 2024-04-25T10:25:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.830 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142775 s, 73.4 MB/s 2024-04-25T10:25:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147148 s, 71.3 MB/s 2024-04-25T10:25:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018686 s, 56.1 MB/s 2024-04-25T10:25:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142741 s, 73.5 MB/s 2024-04-25T10:25:33.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:33.884 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:33.884 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:33.884 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:25:33.884 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:33.885 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:33.885 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:33.885 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173878 s, 60.3 MB/s 2024-04-25T10:25:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138986 s, 75.4 MB/s 2024-04-25T10:25:33.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:33.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:33.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139206 s, 75.3 MB/s 2024-04-25T10:25:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138655 s, 75.6 MB/s 2024-04-25T10:25:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153323 s, 68.4 MB/s 2024-04-25T10:25:34.038 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.039 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148431 s, 70.6 MB/s 2024-04-25T10:25:34.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.055 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141069 s, 74.3 MB/s 2024-04-25T10:25:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179788 s, 58.3 MB/s 2024-04-25T10:25:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148313 s, 70.7 MB/s 2024-04-25T10:25:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145261 s, 72.2 MB/s 2024-04-25T10:25:34.124 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.124 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015564 s, 67.4 MB/s 2024-04-25T10:25:34.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142512 s, 73.6 MB/s 2024-04-25T10:25:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136197 s, 77.0 MB/s 2024-04-25T10:25:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148496 s, 70.6 MB/s 2024-04-25T10:25:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136581 s, 76.8 MB/s 2024-04-25T10:25:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013866 s, 75.6 MB/s 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175515 s, 59.7 MB/s 2024-04-25T10:25:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150611 s, 69.6 MB/s 2024-04-25T10:25:34.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193079 s, 54.3 MB/s 2024-04-25T10:25:34.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.277 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136394 s, 76.9 MB/s 2024-04-25T10:25:34.297 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.297 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.297 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185582 s, 56.5 MB/s 2024-04-25T10:25:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143121 s, 73.3 MB/s 2024-04-25T10:25:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117036 s, 89.6 MB/s 2024-04-25T10:25:34.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149771 s, 70.0 MB/s 2024-04-25T10:25:34.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132761 s, 79.0 MB/s 2024-04-25T10:25:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.373 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130641 s, 80.3 MB/s 2024-04-25T10:25:34.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.389 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145927 s, 71.9 MB/s 2024-04-25T10:25:34.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147292 s, 71.2 MB/s 2024-04-25T10:25:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018497 s, 56.7 MB/s 2024-04-25T10:25:34.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.443 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143509 s, 73.1 MB/s 2024-04-25T10:25:34.460 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.460 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.460 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159215 s, 65.9 MB/s 2024-04-25T10:25:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135676 s, 77.3 MB/s 2024-04-25T10:25:34.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:34.477 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:34.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.555 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018046 s, 58.1 MB/s 2024-04-25T10:25:34.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.573 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162544 s, 64.5 MB/s 2024-04-25T10:25:34.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.592 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176034 s, 59.6 MB/s 2024-04-25T10:25:34.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135779 s, 77.2 MB/s 2024-04-25T10:25:34.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185311 s, 56.6 MB/s 2024-04-25T10:25:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136248 s, 77.0 MB/s 2024-04-25T10:25:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132922 s, 78.9 MB/s 2024-04-25T10:25:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.676 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154115 s, 68.0 MB/s 2024-04-25T10:25:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149898 s, 70.0 MB/s 2024-04-25T10:25:34.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.707 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012888 s, 81.4 MB/s 2024-04-25T10:25:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018994 s, 55.2 MB/s 2024-04-25T10:25:34.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.744 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142281 s, 73.7 MB/s 2024-04-25T10:25:34.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.761 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152124 s, 68.9 MB/s 2024-04-25T10:25:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159605 s, 65.7 MB/s 2024-04-25T10:25:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175261 s, 59.8 MB/s 2024-04-25T10:25:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.812 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124716 s, 84.1 MB/s 2024-04-25T10:25:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130014 s, 80.7 MB/s 2024-04-25T10:25:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156543 s, 67.0 MB/s 2024-04-25T10:25:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.862 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148361 s, 70.7 MB/s 2024-04-25T10:25:34.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.877 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131073 s, 80.0 MB/s 2024-04-25T10:25:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.890 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118802 s, 88.3 MB/s 2024-04-25T10:25:34.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133798 s, 78.4 MB/s 2024-04-25T10:25:34.921 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.921 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.921 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138156 s, 75.9 MB/s 2024-04-25T10:25:34.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.938 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149909 s, 69.9 MB/s 2024-04-25T10:25:34.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.953 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140459 s, 74.7 MB/s 2024-04-25T10:25:34.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141808 s, 73.9 MB/s 2024-04-25T10:25:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145801 s, 71.9 MB/s 2024-04-25T10:25:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178641 s, 58.7 MB/s 2024-04-25T10:25:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142267 s, 73.7 MB/s 2024-04-25T10:25:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.036 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131294 s, 79.9 MB/s 2024-04-25T10:25:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144195 s, 72.7 MB/s 2024-04-25T10:25:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152211 s, 68.9 MB/s 2024-04-25T10:25:35.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:35.071 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:35.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.142 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136512 s, 76.8 MB/s 2024-04-25T10:25:35.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.159 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146123 s, 71.8 MB/s 2024-04-25T10:25:35.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.174 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139633 s, 75.1 MB/s 2024-04-25T10:25:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170971 s, 61.3 MB/s 2024-04-25T10:25:35.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.212 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169974 s, 61.7 MB/s 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157512 s, 66.6 MB/s 2024-04-25T10:25:35.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.246 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142415 s, 73.6 MB/s 2024-04-25T10:25:35.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.262 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014068 s, 74.5 MB/s 2024-04-25T10:25:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147407 s, 71.1 MB/s 2024-04-25T10:25:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.294 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143079 s, 73.3 MB/s 2024-04-25T10:25:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148029 s, 70.8 MB/s 2024-04-25T10:25:35.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140787 s, 74.5 MB/s 2024-04-25T10:25:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145474 s, 72.1 MB/s 2024-04-25T10:25:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143913 s, 72.9 MB/s 2024-04-25T10:25:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158141 s, 66.3 MB/s 2024-04-25T10:25:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145313 s, 72.2 MB/s 2024-04-25T10:25:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013396 s, 78.3 MB/s 2024-04-25T10:25:35.425 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.425 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.425 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148849 s, 70.4 MB/s 2024-04-25T10:25:35.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.441 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144799 s, 72.4 MB/s 2024-04-25T10:25:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.456 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01335 s, 78.5 MB/s 2024-04-25T10:25:35.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.472 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133909 s, 78.3 MB/s 2024-04-25T10:25:35.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136475 s, 76.8 MB/s 2024-04-25T10:25:35.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.504 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151241 s, 69.3 MB/s 2024-04-25T10:25:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.519 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132689 s, 79.0 MB/s 2024-04-25T10:25:35.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150128 s, 69.8 MB/s 2024-04-25T10:25:35.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140466 s, 74.6 MB/s 2024-04-25T10:25:35.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145218 s, 72.2 MB/s 2024-04-25T10:25:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.585 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155159 s, 67.6 MB/s 2024-04-25T10:25:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145349 s, 72.1 MB/s 2024-04-25T10:25:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013495 s, 77.7 MB/s 2024-04-25T10:25:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.633 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147933 s, 70.9 MB/s 2024-04-25T10:25:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136958 s, 76.6 MB/s 2024-04-25T10:25:35.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:35.650 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:35.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.722 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171513 s, 61.1 MB/s 2024-04-25T10:25:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154228 s, 68.0 MB/s 2024-04-25T10:25:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144265 s, 72.7 MB/s 2024-04-25T10:25:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145118 s, 72.3 MB/s 2024-04-25T10:25:35.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013159 s, 79.7 MB/s 2024-04-25T10:25:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151949 s, 69.0 MB/s 2024-04-25T10:25:35.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139772 s, 75.0 MB/s 2024-04-25T10:25:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139705 s, 75.1 MB/s 2024-04-25T10:25:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134581 s, 77.9 MB/s 2024-04-25T10:25:35.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.865 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129197 s, 81.2 MB/s 2024-04-25T10:25:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152854 s, 68.6 MB/s 2024-04-25T10:25:35.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.898 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132427 s, 79.2 MB/s 2024-04-25T10:25:35.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154475 s, 67.9 MB/s 2024-04-25T10:25:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152126 s, 68.9 MB/s 2024-04-25T10:25:35.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.948 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142259 s, 73.7 MB/s 2024-04-25T10:25:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.965 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149373 s, 70.2 MB/s 2024-04-25T10:25:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132179 s, 79.3 MB/s 2024-04-25T10:25:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:35.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:35.997 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149114 s, 70.3 MB/s 2024-04-25T10:25:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152313 s, 68.8 MB/s 2024-04-25T10:25:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.032 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173876 s, 60.3 MB/s 2024-04-25T10:25:36.048 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.048 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.048 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133902 s, 78.3 MB/s 2024-04-25T10:25:36.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.063 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135273 s, 77.5 MB/s 2024-04-25T10:25:36.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.080 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152051 s, 69.0 MB/s 2024-04-25T10:25:36.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.095 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135224 s, 77.5 MB/s 2024-04-25T10:25:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136892 s, 76.6 MB/s 2024-04-25T10:25:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149459 s, 70.2 MB/s 2024-04-25T10:25:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.144 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143062 s, 73.3 MB/s 2024-04-25T10:25:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.160 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014603 s, 71.8 MB/s 2024-04-25T10:25:36.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138348 s, 75.8 MB/s 2024-04-25T10:25:36.192 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.192 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.192 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01476 s, 71.0 MB/s 2024-04-25T10:25:36.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.208 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141227 s, 74.2 MB/s 2024-04-25T10:25:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141816 s, 73.9 MB/s 2024-04-25T10:25:36.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:36.225 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:36.225 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:36.225 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:25:36.225 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:36.225 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:36.226 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:36.226 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:36.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.301 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198052 s, 52.9 MB/s 2024-04-25T10:25:36.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137572 s, 76.2 MB/s 2024-04-25T10:25:36.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.333 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151135 s, 69.4 MB/s 2024-04-25T10:25:36.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.350 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014301 s, 73.3 MB/s 2024-04-25T10:25:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.369 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179441 s, 58.4 MB/s 2024-04-25T10:25:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147892 s, 70.9 MB/s 2024-04-25T10:25:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189906 s, 55.2 MB/s 2024-04-25T10:25:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150405 s, 69.7 MB/s 2024-04-25T10:25:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159884 s, 65.6 MB/s 2024-04-25T10:25:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142628 s, 73.5 MB/s 2024-04-25T10:25:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183181 s, 57.2 MB/s 2024-04-25T10:25:36.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.494 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150609 s, 69.6 MB/s 2024-04-25T10:25:36.511 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.511 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.511 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150073 s, 69.9 MB/s 2024-04-25T10:25:36.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.527 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142781 s, 73.4 MB/s 2024-04-25T10:25:36.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.545 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152176 s, 68.9 MB/s 2024-04-25T10:25:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153172 s, 68.5 MB/s 2024-04-25T10:25:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142154 s, 73.8 MB/s 2024-04-25T10:25:36.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182016 s, 57.6 MB/s 2024-04-25T10:25:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179031 s, 58.6 MB/s 2024-04-25T10:25:36.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.634 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141082 s, 74.3 MB/s 2024-04-25T10:25:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180553 s, 58.1 MB/s 2024-04-25T10:25:36.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140193 s, 74.8 MB/s 2024-04-25T10:25:36.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132231 s, 79.3 MB/s 2024-04-25T10:25:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128501 s, 81.6 MB/s 2024-04-25T10:25:36.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.716 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151246 s, 69.3 MB/s 2024-04-25T10:25:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.733 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146306 s, 71.7 MB/s 2024-04-25T10:25:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184417 s, 56.9 MB/s 2024-04-25T10:25:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139126 s, 75.4 MB/s 2024-04-25T10:25:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.786 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153085 s, 68.5 MB/s 2024-04-25T10:25:36.801 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138323 s, 75.8 MB/s 2024-04-25T10:25:36.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152355 s, 68.8 MB/s 2024-04-25T10:25:36.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.840 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188881 s, 55.5 MB/s 2024-04-25T10:25:36.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:36.841 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175394 s, 59.8 MB/s 2024-04-25T10:25:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142459 s, 73.6 MB/s 2024-04-25T10:25:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134074 s, 78.2 MB/s 2024-04-25T10:25:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.966 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146491 s, 71.6 MB/s 2024-04-25T10:25:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:36.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181352 s, 57.8 MB/s 2024-04-25T10:25:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148796 s, 70.5 MB/s 2024-04-25T10:25:37.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154194 s, 68.0 MB/s 2024-04-25T10:25:37.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.036 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142526 s, 73.6 MB/s 2024-04-25T10:25:37.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133762 s, 78.4 MB/s 2024-04-25T10:25:37.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.067 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140872 s, 74.4 MB/s 2024-04-25T10:25:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.082 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013734 s, 76.3 MB/s 2024-04-25T10:25:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140859 s, 74.4 MB/s 2024-04-25T10:25:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137335 s, 76.4 MB/s 2024-04-25T10:25:37.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.130 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145092 s, 72.3 MB/s 2024-04-25T10:25:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.145 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130265 s, 80.5 MB/s 2024-04-25T10:25:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132078 s, 79.4 MB/s 2024-04-25T10:25:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.180 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184521 s, 56.8 MB/s 2024-04-25T10:25:37.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.196 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142647 s, 73.5 MB/s 2024-04-25T10:25:37.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.212 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139391 s, 75.2 MB/s 2024-04-25T10:25:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139186 s, 75.3 MB/s 2024-04-25T10:25:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.244 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148999 s, 70.4 MB/s 2024-04-25T10:25:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.260 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147737 s, 71.0 MB/s 2024-04-25T10:25:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192914 s, 54.4 MB/s 2024-04-25T10:25:37.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164145 s, 63.9 MB/s 2024-04-25T10:25:37.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166823 s, 62.9 MB/s 2024-04-25T10:25:37.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.333 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.334 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131633 s, 79.7 MB/s 2024-04-25T10:25:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.349 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136499 s, 76.8 MB/s 2024-04-25T10:25:37.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131455 s, 79.8 MB/s 2024-04-25T10:25:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.385 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197802 s, 53.0 MB/s 2024-04-25T10:25:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143144 s, 73.3 MB/s 2024-04-25T10:25:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.418 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149008 s, 70.4 MB/s 2024-04-25T10:25:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138162 s, 75.9 MB/s 2024-04-25T10:25:37.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:37.435 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133031 s, 78.8 MB/s 2024-04-25T10:25:37.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138324 s, 75.8 MB/s 2024-04-25T10:25:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.542 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178417 s, 58.8 MB/s 2024-04-25T10:25:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150725 s, 69.6 MB/s 2024-04-25T10:25:37.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129842 s, 80.8 MB/s 2024-04-25T10:25:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150795 s, 69.5 MB/s 2024-04-25T10:25:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189261 s, 55.4 MB/s 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015002 s, 69.9 MB/s 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132938 s, 78.9 MB/s 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014872 s, 70.5 MB/s 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013698 s, 76.5 MB/s 2024-04-25T10:25:37.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013337 s, 78.6 MB/s 2024-04-25T10:25:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128883 s, 81.4 MB/s 2024-04-25T10:25:37.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146468 s, 71.6 MB/s 2024-04-25T10:25:37.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.736 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124968 s, 83.9 MB/s 2024-04-25T10:25:37.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184839 s, 56.7 MB/s 2024-04-25T10:25:37.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144697 s, 72.5 MB/s 2024-04-25T10:25:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124778 s, 84.0 MB/s 2024-04-25T10:25:37.801 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134935 s, 77.7 MB/s 2024-04-25T10:25:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140211 s, 74.8 MB/s 2024-04-25T10:25:37.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.833 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134639 s, 77.9 MB/s 2024-04-25T10:25:37.849 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.849 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.849 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149019 s, 70.4 MB/s 2024-04-25T10:25:37.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.869 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.869 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177056 s, 59.2 MB/s 2024-04-25T10:25:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134248 s, 78.1 MB/s 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180388 s, 58.1 MB/s 2024-04-25T10:25:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136135 s, 77.0 MB/s 2024-04-25T10:25:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130186 s, 80.5 MB/s 2024-04-25T10:25:37.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131441 s, 79.8 MB/s 2024-04-25T10:25:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188007 s, 55.8 MB/s 2024-04-25T10:25:37.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:37.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146728 s, 71.5 MB/s 2024-04-25T10:25:38.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.003 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148292 s, 70.7 MB/s 2024-04-25T10:25:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138411 s, 75.8 MB/s 2024-04-25T10:25:38.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:38.020 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:38.020 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:38.021 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:25:38.021 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:38.021 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:38.021 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:38.021 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.095 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140744 s, 74.5 MB/s 2024-04-25T10:25:38.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.112 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.112 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014915 s, 70.3 MB/s 2024-04-25T10:25:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.127 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132787 s, 79.0 MB/s 2024-04-25T10:25:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145494 s, 72.1 MB/s 2024-04-25T10:25:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153468 s, 68.3 MB/s 2024-04-25T10:25:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182631 s, 57.4 MB/s 2024-04-25T10:25:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148035 s, 70.8 MB/s 2024-04-25T10:25:38.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.211 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134317 s, 78.1 MB/s 2024-04-25T10:25:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142594 s, 73.5 MB/s 2024-04-25T10:25:38.243 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.243 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.244 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141712 s, 74.0 MB/s 2024-04-25T10:25:38.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135789 s, 77.2 MB/s 2024-04-25T10:25:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013898 s, 75.4 MB/s 2024-04-25T10:25:38.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.291 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151227 s, 69.3 MB/s 2024-04-25T10:25:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177674 s, 59.0 MB/s 2024-04-25T10:25:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147813 s, 70.9 MB/s 2024-04-25T10:25:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129484 s, 81.0 MB/s 2024-04-25T10:25:38.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135693 s, 77.3 MB/s 2024-04-25T10:25:38.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.373 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135185 s, 77.6 MB/s 2024-04-25T10:25:38.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.392 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177347 s, 59.1 MB/s 2024-04-25T10:25:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126486 s, 82.9 MB/s 2024-04-25T10:25:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013454 s, 77.9 MB/s 2024-04-25T10:25:38.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135423 s, 77.4 MB/s 2024-04-25T10:25:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127804 s, 82.0 MB/s 2024-04-25T10:25:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134678 s, 77.9 MB/s 2024-04-25T10:25:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.488 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188763 s, 55.5 MB/s 2024-04-25T10:25:38.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.505 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150576 s, 69.6 MB/s 2024-04-25T10:25:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.520 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.521 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141515 s, 74.1 MB/s 2024-04-25T10:25:38.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.537 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014476 s, 72.4 MB/s 2024-04-25T10:25:38.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139051 s, 75.4 MB/s 2024-04-25T10:25:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.567 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012914 s, 81.2 MB/s 2024-04-25T10:25:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.587 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180179 s, 58.2 MB/s 2024-04-25T10:25:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.603 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135522 s, 77.4 MB/s 2024-04-25T10:25:38.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:38.604 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:38.605 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:38.605 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:25:38.605 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:38.605 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:38.605 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:38.605 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:38.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137118 s, 76.5 MB/s 2024-04-25T10:25:38.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.694 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133661 s, 78.5 MB/s 2024-04-25T10:25:38.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.708 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126849 s, 82.7 MB/s 2024-04-25T10:25:38.724 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.724 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.724 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139954 s, 74.9 MB/s 2024-04-25T10:25:38.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157003 s, 66.8 MB/s 2024-04-25T10:25:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129388 s, 81.0 MB/s 2024-04-25T10:25:38.773 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.773 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.773 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151243 s, 69.3 MB/s 2024-04-25T10:25:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.790 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150647 s, 69.6 MB/s 2024-04-25T10:25:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.805 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131072 s, 80.0 MB/s 2024-04-25T10:25:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144774 s, 72.4 MB/s 2024-04-25T10:25:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.841 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183458 s, 57.2 MB/s 2024-04-25T10:25:38.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.856 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.857 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131039 s, 80.0 MB/s 2024-04-25T10:25:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135319 s, 77.5 MB/s 2024-04-25T10:25:38.888 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152384 s, 68.8 MB/s 2024-04-25T10:25:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186058 s, 56.4 MB/s 2024-04-25T10:25:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013975 s, 75.0 MB/s 2024-04-25T10:25:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.944 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177216 s, 59.2 MB/s 2024-04-25T10:25:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139136 s, 75.4 MB/s 2024-04-25T10:25:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191567 s, 54.7 MB/s 2024-04-25T10:25:38.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:38.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:38.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149889 s, 70.0 MB/s 2024-04-25T10:25:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153678 s, 68.2 MB/s 2024-04-25T10:25:39.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.030 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132044 s, 79.4 MB/s 2024-04-25T10:25:39.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154665 s, 67.8 MB/s 2024-04-25T10:25:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139476 s, 75.2 MB/s 2024-04-25T10:25:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.078 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135976 s, 77.1 MB/s 2024-04-25T10:25:39.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185628 s, 56.5 MB/s 2024-04-25T10:25:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145304 s, 72.2 MB/s 2024-04-25T10:25:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152317 s, 68.8 MB/s 2024-04-25T10:25:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.149 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014767 s, 71.0 MB/s 2024-04-25T10:25:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.164 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132895 s, 78.9 MB/s 2024-04-25T10:25:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143781 s, 72.9 MB/s 2024-04-25T10:25:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.200 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181031 s, 57.9 MB/s 2024-04-25T10:25:39.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:39.201 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191159 s, 54.9 MB/s 2024-04-25T10:25:39.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139643 s, 75.1 MB/s 2024-04-25T10:25:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183082 s, 57.3 MB/s 2024-04-25T10:25:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137697 s, 76.2 MB/s 2024-04-25T10:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128668 s, 81.5 MB/s 2024-04-25T10:25:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188549 s, 55.6 MB/s 2024-04-25T10:25:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.379 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136385 s, 76.9 MB/s 2024-04-25T10:25:39.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.396 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149455 s, 70.2 MB/s 2024-04-25T10:25:39.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.412 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144301 s, 72.7 MB/s 2024-04-25T10:25:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145048 s, 72.3 MB/s 2024-04-25T10:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156274 s, 67.1 MB/s 2024-04-25T10:25:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129919 s, 80.7 MB/s 2024-04-25T10:25:39.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148822 s, 70.5 MB/s 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135505 s, 77.4 MB/s 2024-04-25T10:25:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131852 s, 79.5 MB/s 2024-04-25T10:25:39.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.525 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153406 s, 68.4 MB/s 2024-04-25T10:25:39.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.545 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185834 s, 56.4 MB/s 2024-04-25T10:25:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.560 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127535 s, 82.2 MB/s 2024-04-25T10:25:39.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155552 s, 67.4 MB/s 2024-04-25T10:25:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150466 s, 69.7 MB/s 2024-04-25T10:25:39.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.614 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180693 s, 58.0 MB/s 2024-04-25T10:25:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141922 s, 73.9 MB/s 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.646 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144229 s, 72.7 MB/s 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.662 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136319 s, 76.9 MB/s 2024-04-25T10:25:39.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131705 s, 79.6 MB/s 2024-04-25T10:25:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144787 s, 72.4 MB/s 2024-04-25T10:25:39.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140741 s, 74.5 MB/s 2024-04-25T10:25:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185634 s, 56.5 MB/s 2024-04-25T10:25:39.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157956 s, 66.4 MB/s 2024-04-25T10:25:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.762 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134097 s, 78.2 MB/s 2024-04-25T10:25:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148508 s, 70.6 MB/s 2024-04-25T10:25:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146455 s, 71.6 MB/s 2024-04-25T10:25:39.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:39.797 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:39.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138531 s, 75.7 MB/s 2024-04-25T10:25:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137447 s, 76.3 MB/s 2024-04-25T10:25:39.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.897 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129244 s, 81.1 MB/s 2024-04-25T10:25:39.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.914 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014526 s, 72.2 MB/s 2024-04-25T10:25:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.930 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142439 s, 73.6 MB/s 2024-04-25T10:25:39.946 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015167 s, 69.1 MB/s 2024-04-25T10:25:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.962 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133984 s, 78.3 MB/s 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134946 s, 77.7 MB/s 2024-04-25T10:25:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:39.992 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131251 s, 79.9 MB/s 2024-04-25T10:25:40.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211776 s, 49.5 MB/s 2024-04-25T10:25:40.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136852 s, 76.6 MB/s 2024-04-25T10:25:40.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143678 s, 73.0 MB/s 2024-04-25T10:25:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137744 s, 76.1 MB/s 2024-04-25T10:25:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.079 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144162 s, 72.7 MB/s 2024-04-25T10:25:40.093 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.094 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.094 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130916 s, 80.1 MB/s 2024-04-25T10:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140204 s, 74.8 MB/s 2024-04-25T10:25:40.124 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.124 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.125 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137235 s, 76.4 MB/s 2024-04-25T10:25:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.145 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180919 s, 58.0 MB/s 2024-04-25T10:25:40.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.161 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145403 s, 72.1 MB/s 2024-04-25T10:25:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.180 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175677 s, 59.7 MB/s 2024-04-25T10:25:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.197 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150138 s, 69.8 MB/s 2024-04-25T10:25:40.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.212 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136704 s, 76.7 MB/s 2024-04-25T10:25:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014219 s, 73.7 MB/s 2024-04-25T10:25:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.245 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147587 s, 71.0 MB/s 2024-04-25T10:25:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146141 s, 71.8 MB/s 2024-04-25T10:25:40.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.280 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016212 s, 64.7 MB/s 2024-04-25T10:25:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.296 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142675 s, 73.5 MB/s 2024-04-25T10:25:40.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138805 s, 75.5 MB/s 2024-04-25T10:25:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.331 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179333 s, 58.5 MB/s 2024-04-25T10:25:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180685 s, 58.0 MB/s 2024-04-25T10:25:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148838 s, 70.5 MB/s 2024-04-25T10:25:40.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143099 s, 73.3 MB/s 2024-04-25T10:25:40.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:40.386 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:40.386 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:40.386 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:25:40.386 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:40.387 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:40.387 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:40.387 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:40.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.454 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013377 s, 78.4 MB/s 2024-04-25T10:25:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014884 s, 70.4 MB/s 2024-04-25T10:25:40.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140692 s, 74.5 MB/s 2024-04-25T10:25:40.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013578 s, 77.2 MB/s 2024-04-25T10:25:40.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.518 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138335 s, 75.8 MB/s 2024-04-25T10:25:40.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.533 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135153 s, 77.6 MB/s 2024-04-25T10:25:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150566 s, 69.6 MB/s 2024-04-25T10:25:40.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.567 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149031 s, 70.4 MB/s 2024-04-25T10:25:40.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.584 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151986 s, 69.0 MB/s 2024-04-25T10:25:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.600 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146281 s, 71.7 MB/s 2024-04-25T10:25:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157918 s, 66.4 MB/s 2024-04-25T10:25:40.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156315 s, 67.1 MB/s 2024-04-25T10:25:40.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152656 s, 68.7 MB/s 2024-04-25T10:25:40.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.669 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147087 s, 71.3 MB/s 2024-04-25T10:25:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188454 s, 55.6 MB/s 2024-04-25T10:25:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154594 s, 67.8 MB/s 2024-04-25T10:25:40.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144182 s, 72.7 MB/s 2024-04-25T10:25:40.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.742 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177967 s, 58.9 MB/s 2024-04-25T10:25:40.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014681 s, 71.4 MB/s 2024-04-25T10:25:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182316 s, 57.5 MB/s 2024-04-25T10:25:40.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152284 s, 68.9 MB/s 2024-04-25T10:25:40.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.812 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144533 s, 72.5 MB/s 2024-04-25T10:25:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138162 s, 75.9 MB/s 2024-04-25T10:25:40.842 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.842 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.842 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133332 s, 78.6 MB/s 2024-04-25T10:25:40.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.858 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.858 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133911 s, 78.3 MB/s 2024-04-25T10:25:40.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.873 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133377 s, 78.6 MB/s 2024-04-25T10:25:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.890 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146827 s, 71.4 MB/s 2024-04-25T10:25:40.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.906 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147031 s, 71.3 MB/s 2024-04-25T10:25:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150088 s, 69.9 MB/s 2024-04-25T10:25:40.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.938 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133616 s, 78.5 MB/s 2024-04-25T10:25:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134067 s, 78.2 MB/s 2024-04-25T10:25:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155297 s, 67.5 MB/s 2024-04-25T10:25:40.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:40.971 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:40.972 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:40.972 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:25:40.972 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:40.972 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:40.972 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:40.972 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:41.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.045 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.045 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152472 s, 68.8 MB/s 2024-04-25T10:25:41.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.061 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146269 s, 71.7 MB/s 2024-04-25T10:25:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.081 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185184 s, 56.6 MB/s 2024-04-25T10:25:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131497 s, 79.7 MB/s 2024-04-25T10:25:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148238 s, 70.7 MB/s 2024-04-25T10:25:41.128 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.129 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139087 s, 75.4 MB/s 2024-04-25T10:25:41.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.146 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149557 s, 70.1 MB/s 2024-04-25T10:25:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183654 s, 57.1 MB/s 2024-04-25T10:25:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.180 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130917 s, 80.1 MB/s 2024-04-25T10:25:41.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.196 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132822 s, 78.9 MB/s 2024-04-25T10:25:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014107 s, 74.3 MB/s 2024-04-25T10:25:41.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.227 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139691 s, 75.1 MB/s 2024-04-25T10:25:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.243 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143217 s, 73.2 MB/s 2024-04-25T10:25:41.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132358 s, 79.2 MB/s 2024-04-25T10:25:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142034 s, 73.8 MB/s 2024-04-25T10:25:41.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.291 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151995 s, 69.0 MB/s 2024-04-25T10:25:41.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.307 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014386 s, 72.9 MB/s 2024-04-25T10:25:41.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.323 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014354 s, 73.1 MB/s 2024-04-25T10:25:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.340 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148913 s, 70.4 MB/s 2024-04-25T10:25:41.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142037 s, 73.8 MB/s 2024-04-25T10:25:41.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.377 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186314 s, 56.3 MB/s 2024-04-25T10:25:41.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140509 s, 74.6 MB/s 2024-04-25T10:25:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.409 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151584 s, 69.2 MB/s 2024-04-25T10:25:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147841 s, 70.9 MB/s 2024-04-25T10:25:41.440 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.441 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131406 s, 79.8 MB/s 2024-04-25T10:25:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.458 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152237 s, 68.9 MB/s 2024-04-25T10:25:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143816 s, 72.9 MB/s 2024-04-25T10:25:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182653 s, 57.4 MB/s 2024-04-25T10:25:41.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.510 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.510 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137614 s, 76.2 MB/s 2024-04-25T10:25:41.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.527 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151742 s, 69.1 MB/s 2024-04-25T10:25:41.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220809 s, 47.5 MB/s 2024-04-25T10:25:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017582 s, 59.6 MB/s 2024-04-25T10:25:41.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:41.573 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:41.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137838 s, 76.1 MB/s 2024-04-25T10:25:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188522 s, 55.6 MB/s 2024-04-25T10:25:41.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138211 s, 75.9 MB/s 2024-04-25T10:25:41.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133778 s, 78.4 MB/s 2024-04-25T10:25:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183431 s, 57.2 MB/s 2024-04-25T10:25:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142951 s, 73.4 MB/s 2024-04-25T10:25:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015067 s, 69.6 MB/s 2024-04-25T10:25:41.766 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.766 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.766 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135096 s, 77.6 MB/s 2024-04-25T10:25:41.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.785 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164514 s, 63.7 MB/s 2024-04-25T10:25:41.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.800 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137391 s, 76.3 MB/s 2024-04-25T10:25:41.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.816 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136236 s, 77.0 MB/s 2024-04-25T10:25:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140295 s, 74.7 MB/s 2024-04-25T10:25:41.848 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.848 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.848 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144523 s, 72.6 MB/s 2024-04-25T10:25:41.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.863 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013372 s, 78.4 MB/s 2024-04-25T10:25:41.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.879 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146087 s, 71.8 MB/s 2024-04-25T10:25:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177795 s, 59.0 MB/s 2024-04-25T10:25:41.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142358 s, 73.7 MB/s 2024-04-25T10:25:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178616 s, 58.7 MB/s 2024-04-25T10:25:41.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.955 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179825 s, 58.3 MB/s 2024-04-25T10:25:41.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140194 s, 74.8 MB/s 2024-04-25T10:25:41.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:41.990 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:41.990 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176358 s, 59.5 MB/s 2024-04-25T10:25:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.006 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140336 s, 74.7 MB/s 2024-04-25T10:25:42.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139323 s, 75.3 MB/s 2024-04-25T10:25:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.042 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192159 s, 54.6 MB/s 2024-04-25T10:25:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.061 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168062 s, 62.4 MB/s 2024-04-25T10:25:42.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.081 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182321 s, 57.5 MB/s 2024-04-25T10:25:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.097 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143338 s, 73.2 MB/s 2024-04-25T10:25:42.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188204 s, 55.7 MB/s 2024-04-25T10:25:42.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.134 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146187 s, 71.7 MB/s 2024-04-25T10:25:42.150 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014682 s, 71.4 MB/s 2024-04-25T10:25:42.165 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.165 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.165 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01305 s, 80.4 MB/s 2024-04-25T10:25:42.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.186 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185948 s, 56.4 MB/s 2024-04-25T10:25:42.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:42.187 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:42.187 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:42.187 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:25:42.188 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:42.188 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:42.188 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:42.188 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:42.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132121 s, 79.4 MB/s 2024-04-25T10:25:42.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145821 s, 71.9 MB/s 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152416 s, 68.8 MB/s 2024-04-25T10:25:42.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.312 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176893 s, 59.3 MB/s 2024-04-25T10:25:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162486 s, 64.5 MB/s 2024-04-25T10:25:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137897 s, 76.0 MB/s 2024-04-25T10:25:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146342 s, 71.7 MB/s 2024-04-25T10:25:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171142 s, 61.3 MB/s 2024-04-25T10:25:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.400 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167553 s, 62.6 MB/s 2024-04-25T10:25:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150422 s, 69.7 MB/s 2024-04-25T10:25:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187042 s, 56.1 MB/s 2024-04-25T10:25:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015379 s, 68.2 MB/s 2024-04-25T10:25:42.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.472 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155141 s, 67.6 MB/s 2024-04-25T10:25:42.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152016 s, 69.0 MB/s 2024-04-25T10:25:42.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.510 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190154 s, 55.1 MB/s 2024-04-25T10:25:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.526 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143423 s, 73.1 MB/s 2024-04-25T10:25:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155553 s, 67.4 MB/s 2024-04-25T10:25:42.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.559 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014112 s, 74.3 MB/s 2024-04-25T10:25:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138784 s, 75.6 MB/s 2024-04-25T10:25:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153232 s, 68.4 MB/s 2024-04-25T10:25:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159076 s, 65.9 MB/s 2024-04-25T10:25:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135414 s, 77.4 MB/s 2024-04-25T10:25:42.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.641 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152162 s, 68.9 MB/s 2024-04-25T10:25:42.664 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215068 s, 48.8 MB/s 2024-04-25T10:25:42.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144444 s, 72.6 MB/s 2024-04-25T10:25:42.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147882 s, 70.9 MB/s 2024-04-25T10:25:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149928 s, 69.9 MB/s 2024-04-25T10:25:42.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144039 s, 72.8 MB/s 2024-04-25T10:25:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013754 s, 76.2 MB/s 2024-04-25T10:25:42.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.768 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187056 s, 56.1 MB/s 2024-04-25T10:25:42.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.784 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.784 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014285 s, 73.4 MB/s 2024-04-25T10:25:42.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177197 s, 59.2 MB/s 2024-04-25T10:25:42.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:42.804 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:25:42.804 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:25:42.804 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:25:42.804 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:25:42.804 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:25:42.805 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:25:42.805 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:25:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013535 s, 77.5 MB/s 2024-04-25T10:25:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146438 s, 71.6 MB/s 2024-04-25T10:25:42.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.912 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145958 s, 71.8 MB/s 2024-04-25T10:25:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123807 s, 84.7 MB/s 2024-04-25T10:25:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143536 s, 73.1 MB/s 2024-04-25T10:25:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014452 s, 72.6 MB/s 2024-04-25T10:25:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143893 s, 72.9 MB/s 2024-04-25T10:25:42.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:42.990 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:42.990 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137341 s, 76.3 MB/s 2024-04-25T10:25:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.006 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144505 s, 72.6 MB/s 2024-04-25T10:25:43.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.026 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180263 s, 58.2 MB/s 2024-04-25T10:25:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.043 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154497 s, 67.9 MB/s 2024-04-25T10:25:43.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149971 s, 69.9 MB/s 2024-04-25T10:25:43.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148621 s, 70.6 MB/s 2024-04-25T10:25:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191527 s, 54.7 MB/s 2024-04-25T10:25:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145369 s, 72.1 MB/s 2024-04-25T10:25:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142876 s, 73.4 MB/s 2024-04-25T10:25:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015318 s, 68.5 MB/s 2024-04-25T10:25:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134466 s, 78.0 MB/s 2024-04-25T10:25:43.181 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.181 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.181 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169788 s, 61.8 MB/s 2024-04-25T10:25:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.196 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135357 s, 77.5 MB/s 2024-04-25T10:25:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014904 s, 70.4 MB/s 2024-04-25T10:25:43.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.232 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174043 s, 60.2 MB/s 2024-04-25T10:25:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138452 s, 75.7 MB/s 2024-04-25T10:25:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187796 s, 55.8 MB/s 2024-04-25T10:25:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.286 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156512 s, 67.0 MB/s 2024-04-25T10:25:43.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159811 s, 65.6 MB/s 2024-04-25T10:25:43.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.323 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173499 s, 60.4 MB/s 2024-04-25T10:25:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153494 s, 68.3 MB/s 2024-04-25T10:25:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.356 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140555 s, 74.6 MB/s 2024-04-25T10:25:43.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.372 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140775 s, 74.5 MB/s 2024-04-25T10:25:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130884 s, 80.1 MB/s 2024-04-25T10:25:43.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:25:43.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:25:43.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183414 s, 57.2 MB/s 2024-04-25T10:25:43.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:43.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:25:43.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 Processor -- start 2024-04-25T10:25:43.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 -- start start 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f618c0cf040 0x7f618c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f618c071ab0 0x7f618c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c071f80 con 0x7f618c15bc00 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c072260 con 0x7f618c071ab0 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6190a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0b3700 con 0x7f618c0cf040 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f618c071ab0 0x7f618c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f618c071ab0 0x7f618c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39566/0 (socket says 172.21.15.45:39566) 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 -- 172.21.15.45:0/3904426883 learned_addr learned my addr 172.21.15.45:0/3904426883 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f6189e00640 1 --2- 172.21.15.45:0/3904426883 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cf040 0x7f618c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 -- 172.21.15.45:0/3904426883 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cf040 msgr2=0x7f618c0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 --2- 172.21.15.45:0/3904426883 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cf040 0x7f618c0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 -- 172.21.15.45:0/3904426883 >> v1:172.21.15.117:6789/0 conn(0x7f618c15bc00 legacy=0x7f618c0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.595+0000 7f618a800640 1 -- 172.21.15.45:0/3904426883 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c15b120 con 0x7f618c071ab0 2024-04-25T10:25:43.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f618a800640 1 --2- 172.21.15.45:0/3904426883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c071ab0 0x7f618c0cff20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f618c0d05d0 tx=0x7f618401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290c6fc23ac9417b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6189400640 1 -- 172.21.15.45:0/3904426883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5760 con 0x7f618c071ab0 2024-04-25T10:25:43.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6189400640 1 -- 172.21.15.45:0/3904426883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840a5900 con 0x7f618c071ab0 2024-04-25T10:25:43.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6189400640 1 -- 172.21.15.45:0/3904426883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5cd0 con 0x7f618c071ab0 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 -- 172.21.15.45:0/3904426883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c071ab0 msgr2=0x7f618c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 --2- 172.21.15.45:0/3904426883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c071ab0 0x7f618c0cff20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f618c0d05d0 tx=0x7f618401c9b0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 -- 172.21.15.45:0/3904426883 shutdown_connections 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 --2- 172.21.15.45:0/3904426883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c071ab0 0x7f618c0cff20 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 --2- 172.21.15.45:0/3904426883 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cf040 0x7f618c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 -- 172.21.15.45:0/3904426883 >> 172.21.15.45:0/3904426883 conn(0x7f618c066b70 msgr2=0x7f618c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 -- 172.21.15.45:0/3904426883 shutdown_connections 2024-04-25T10:25:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.599+0000 7f6190a00640 1 -- 172.21.15.45:0/3904426883 wait complete. 2024-04-25T10:25:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 Processor -- start 2024-04-25T10:25:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 -- start start 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c071ab0 0x7f618c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0cf040 0x7f618c15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 0x7f618c153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0aee10 con 0x7f618c15bc00 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0aec90 con 0x7f618c071ab0 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0aef90 con 0x7f618c0cf040 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 0x7f618c153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 0x7f618c153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39572/0 (socket says 172.21.15.45:39572) 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 -- 172.21.15.45:0/868326846 learned_addr learned my addr 172.21.15.45:0/868326846 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6189e00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c071ab0 0x7f618c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6181400640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0cf040 0x7f618c15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 -- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0cf040 msgr2=0x7f618c15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0cf040 0x7f618c15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 -- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c071ab0 msgr2=0x7f618c151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c071ab0 0x7f618c151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 -- 172.21.15.45:0/868326846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c15acb0 con 0x7f618c15bc00 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f618a800640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 0x7f618c153f70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f61840af040 tx=0x7f6184019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6182a00640 1 -- 172.21.15.45:0/868326846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840bc020 con 0x7f618c15bc00 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6182a00640 1 -- 172.21.15.45:0/868326846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840b8690 con 0x7f618c15bc00 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6182a00640 1 -- 172.21.15.45:0/868326846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840b8a80 con 0x7f618c15bc00 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c1531b0 con 0x7f618c15bc00 2024-04-25T10:25:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.603+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c153770 con 0x7f618c15bc00 2024-04-25T10:25:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6182a00640 1 -- 172.21.15.45:0/868326846 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61840bf020 con 0x7f618c15bc00 2024-04-25T10:25:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6182a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f90 0x7f6168069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6182a00640 1 -- 172.21.15.45:0/868326846 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f6184120070 con 0x7f618c15bc00 2024-04-25T10:25:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6189e00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f90 0x7f6168069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 Processor -- start 2024-04-25T10:25:43.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 -- start start 2024-04-25T10:25:43.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6189e00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f90 0x7f6168069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b0fc0 tx=0x7f617c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a53d10 0x5600b5a09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 0x5600b5a0a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b5857d50 con 0x5600b5a0a520 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b57dfe40 con 0x5600b5a533f0 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600b5853df0 con 0x5600b5a53d10 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 0x5600b5a0a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6189e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f6181400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a53d10 0x5600b5a09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 0x5600b5a0a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39578/0 (socket says 172.21.15.45:39578) 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 -- 172.21.15.45:0/155180751 learned_addr learned my addr 172.21.15.45:0/155180751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 -- 172.21.15.45:0/155180751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a53d10 msgr2=0x5600b5a09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a53d10 0x5600b5a09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 -- 172.21.15.45:0/155180751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 msgr2=0x5600b5a537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.607+0000 7f618a800640 1 -- 172.21.15.45:0/155180751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600b57f4d90 con 0x5600b5a0a520 2024-04-25T10:25:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 0x5600b5a0a900 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f61840ed000 tx=0x7f6184009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac164e2f288122f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f617aa00640 1 -- 172.21.15.45:0/155180751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840f0ba0 con 0x5600b5a0a520 2024-04-25T10:25:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f617aa00640 1 -- 172.21.15.45:0/155180751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840f4c40 con 0x5600b5a0a520 2024-04-25T10:25:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f617aa00640 1 -- 172.21.15.45:0/155180751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840f9200 con 0x5600b5a0a520 2024-04-25T10:25:43.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- 172.21.15.45:0/155180751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 msgr2=0x5600b5a0a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 0x5600b5a0a900 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f61840ed000 tx=0x7f6184009af0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- 172.21.15.45:0/155180751 shutdown_connections 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a0a520 0x5600b5a0a900 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a53d10 0x5600b5a09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- 172.21.15.45:0/155180751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- 172.21.15.45:0/155180751 >> 172.21.15.45:0/155180751 conn(0x5600b5a064d0 msgr2=0x5600b5a088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- 172.21.15.45:0/155180751 shutdown_connections 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- 172.21.15.45:0/155180751 wait complete. 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- start start 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 Processor -- start 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a0a520 0x5600b5a5a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a14bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b57fd610 con 0x5600b5a53d10 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b58129a0 con 0x5600b5a533f0 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600b585c380 con 0x5600b5a0a520 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6193f8c000 10 client.0 _ll_register_callbacks cb 0x5600b590c0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a14bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a14bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39588/0 (socket says 172.21.15.45:39588) 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 -- 172.21.15.45:0/180048539 learned_addr learned my addr 172.21.15.45:0/180048539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6189e00640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a0a520 0x5600b5a5a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f6181400640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 -- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a0a520 msgr2=0x5600b5a5a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a0a520 0x5600b5a5a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 -- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 msgr2=0x5600b5a5aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a5aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 -- 172.21.15.45:0/180048539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600b5852d40 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f618a800640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a14bf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f618401b060 tx=0x7f6184009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d845308f6bdce64f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f616f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.611+0000 7f616f400640 1 -- 172.21.15.45:0/180048539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840f1000 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 1 -- 172.21.15.45:0/180048539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f61840f8d70 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61840f4c40 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 1 -- 172.21.15.45:0/180048539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61840f9200 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 10 client.8019 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5600b58daf00 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 1 -- 172.21.15.45:0/180048539 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840f4c40 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 1 -- 172.21.15.45:0/180048539 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61840f5b60 con 0x5600b5a53d10 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 10 client.8019 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 10 client.8019 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 20 client.8019 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a15960 0x5600b5a15d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 4 client.8019 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600b57f3a20 con 0x5600b5a15960 2024-04-25T10:25:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6189e00640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a15960 0x5600b5a15d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6189e00640 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a15960 0x5600b5a15d40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24a5a9075d89148 server_cookie=5c28c2baff6f602f in_seq=0 out_seq=0 2024-04-25T10:25:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 10 client.8019 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 1 -- 172.21.15.45:0/180048539 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600b57f3a20 con 0x5600b5a15960 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 10 client.8019 handle_command_reply: tid=0 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 client.8019 shutdown 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 msgr2=0x5600b5a14bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a14bf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f618401b060 tx=0x7f6184009810 comp rx=0 tx=0).stop 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 shutdown_connections 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a14bf0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a15960 0x5600b5a15d40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a533f0 0x5600b5a5aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 --2- 172.21.15.45:0/180048539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a0a520 0x5600b5a5a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f616f400640 0 client.8019 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 >> 172.21.15.45:0/180048539 conn(0x5600b5a064d0 msgr2=0x5600b5a09150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 shutdown_connections 2024-04-25T10:25:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 1 -- 172.21.15.45:0/180048539 wait complete. 2024-04-25T10:25:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.615+0000 7f6193f8c000 20 client.8019 trim_cache size 0 max 16384 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 Processor -- start 2024-04-25T10:25:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 -- start start 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a09f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a08e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c202d60 con 0x5600b5a53d10 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1670a0 con 0x5600b5a0a520 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1f75b0 con 0x5600b5a533f0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a08e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39592/0 (socket says 172.21.15.45:39592) 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 -- 172.21.15.45:0/3791998253 learned_addr learned my addr 172.21.15.45:0/3791998253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6189e00640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a09f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6181400640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 -- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 msgr2=0x5600b5a088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 -- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 msgr2=0x5600b5a09f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a09f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 -- 172.21.15.45:0/3791998253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600b58b1330 con 0x5600b5a53d10 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6189e00640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a09f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f6181400640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f618a800640 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a08e00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f618401cf20 tx=0x7f618411d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e03652389be1fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f617a000640 1 -- 172.21.15.45:0/3791998253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840f4c40 con 0x5600b5a53d10 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f617a000640 1 -- 172.21.15.45:0/3791998253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184102070 con 0x5600b5a53d10 2024-04-25T10:25:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.631+0000 7f617a000640 1 -- 172.21.15.45:0/3791998253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840f8f90 con 0x5600b5a53d10 2024-04-25T10:25:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3791998253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 msgr2=0x5600b5a08e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a08e00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f618401cf20 tx=0x7f618411d000 comp rx=0 tx=0).stop 2024-04-25T10:25:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3791998253 shutdown_connections 2024-04-25T10:25:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a08e00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3791998253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a09f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3791998253 >> 172.21.15.45:0/3791998253 conn(0x5600b5a064d0 msgr2=0x5600b5a60c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3791998253 shutdown_connections 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3791998253 wait complete. 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- start start 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 Processor -- start 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a57150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a57690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600b580edc0 con 0x5600b5a53d10 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c166aa0 con 0x5600b5a0a520 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600b5857060 con 0x5600b5a533f0 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 10 client.0 _ll_register_callbacks cb 0x7f6184009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a70c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39596/0 (socket says 172.21.15.45:39596) 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 -- 172.21.15.45:0/3026684028 learned_addr learned my addr 172.21.15.45:0/3026684028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6189e00640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a57150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6181400640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a57690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 -- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 msgr2=0x5600b5a57690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a57690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 -- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 msgr2=0x5600b5a57150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a57150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 -- 172.21.15.45:0/3026684028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600b58245f0 con 0x5600b5a53d10 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6189e00640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f618a800640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a70c00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f61840ed520 tx=0x7f61840c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f99d4513a1e2aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f616f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f616f400640 1 -- 172.21.15.45:0/3026684028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840f0ba0 con 0x5600b5a53d10 2024-04-25T10:25:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f616f400640 1 -- 172.21.15.45:0/3026684028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f61840f1010 con 0x5600b5a53d10 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6184102070 con 0x5600b5a53d10 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f616f400640 1 -- 172.21.15.45:0/3026684028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61840f4c40 con 0x5600b5a53d10 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 10 client.8025 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5600b57fefe0 con 0x5600b5a53d10 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f616f400640 1 -- 172.21.15.45:0/3026684028 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184102070 con 0x5600b5a53d10 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f616f400640 1 -- 172.21.15.45:0/3026684028 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61840efff0 con 0x5600b5a53d10 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 10 client.8025 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 10 client.8025 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 20 client.8025 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a71140 0x5600b5a71520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 4 client.8025 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.635+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5600b5865290 con 0x5600b5a71140 2024-04-25T10:25:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6189e00640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a71140 0x5600b5a71520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6189e00640 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a71140 0x5600b5a71520 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e39c04eaa1b1907 server_cookie=9c21def7f6e5c762 in_seq=0 out_seq=0 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f616f400640 10 client.8025 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f616f400640 1 -- 172.21.15.45:0/3026684028 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5600b5865290 con 0x5600b5a71140 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f616f400640 10 client.8025 handle_command_reply: tid=0 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 client.8025 shutdown 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 msgr2=0x5600b5a70c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a70c00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f61840ed520 tx=0x7f61840c2000 comp rx=0 tx=0).stop 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 shutdown_connections 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600b5a53d10 0x5600b5a70c00 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5600b5a533f0 0x5600b5a57690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5600b5a71140 0x5600b5a71520 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 --2- 172.21.15.45:0/3026684028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600b5a0a520 0x5600b5a57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 >> 172.21.15.45:0/3026684028 conn(0x5600b5a064d0 msgr2=0x5600b5a51f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f616f400640 0 client.8025 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 shutdown_connections 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 1 -- 172.21.15.45:0/3026684028 wait complete. 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.639+0000 7f6193f8c000 20 client.8025 trim_cache size 0 max 16384 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "d842100b-563d-4e46-beda-2619600fc8ce", 2024-04-25T10:25:43.645 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:25:43.646 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:43.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f90 msgr2=0x7f6168069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f90 0x7f6168069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b0fc0 tx=0x7f617c008040 comp rx=0 tx=0).stop 2024-04-25T10:25:43.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 msgr2=0x7f618c153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 0x7f618c153f70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f61840af040 tx=0x7f6184019750 comp rx=0 tx=0).stop 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 shutdown_connections 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f90 0x7f6168069450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c15bc00 0x7f618c153f70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0cf040 0x7f618c15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 --2- 172.21.15.45:0/868326846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c071ab0 0x7f618c151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 >> 172.21.15.45:0/868326846 conn(0x7f618c066b70 msgr2=0x7f618c15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 shutdown_connections 2024-04-25T10:25:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.643+0000 7f6190a00640 1 -- 172.21.15.45:0/868326846 wait complete. 2024-04-25T10:25:43.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:43.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T10:25:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.843+0000 7f35a2600640 1 Processor -- start 2024-04-25T10:25:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.843+0000 7f35a2600640 1 -- start start 2024-04-25T10:25:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f35a2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0cc200 0x7f359c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f35a2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f359c06cf40 0x7f359c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f35a2600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0cdab0 con 0x7f359c0d0d00 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f35a2600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c0cd210 con 0x7f359c06cf40 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f35a2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0cd4f0 con 0x7f359c0cc200 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f359c06cf40 0x7f359c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f359c06cf40 0x7f359c06d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39610/0 (socket says 172.21.15.45:39610) 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 -- 172.21.15.45:0/1982709412 learned_addr learned my addr 172.21.15.45:0/1982709412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f3592a00640 1 --2- 172.21.15.45:0/1982709412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0cc200 0x7f359c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 -- 172.21.15.45:0/1982709412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0cc200 msgr2=0x7f359c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 --2- 172.21.15.45:0/1982709412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0cc200 0x7f359c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 -- 172.21.15.45:0/1982709412 >> v1:172.21.15.117:6789/0 conn(0x7f359c0d0d00 legacy=0x7f359c162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359be00640 1 -- 172.21.15.45:0/1982709412 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0cd7d0 con 0x7f359c06cf40 2024-04-25T10:25:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.847+0000 7f359aa00640 1 -- 172.21.15.45:0/1982709412 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92956512 0 0) 0x7f359c0cdab0 con 0x7f359c0d0d00 2024-04-25T10:25:43.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f359be00640 1 --2- 172.21.15.45:0/1982709412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c06d3d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3594009e50 tx=0x7f359401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c45707ad4a32bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f359aa00640 1 -- 172.21.15.45:0/1982709412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940a5660 con 0x7f359c06cf40 2024-04-25T10:25:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f359aa00640 1 -- 172.21.15.45:0/1982709412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35940a5800 con 0x7f359c06cf40 2024-04-25T10:25:43.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f359aa00640 1 -- 172.21.15.45:0/1982709412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940a5bd0 con 0x7f359c06cf40 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- 172.21.15.45:0/1982709412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 msgr2=0x7f359c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 --2- 172.21.15.45:0/1982709412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c06d3d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3594009e50 tx=0x7f359401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- 172.21.15.45:0/1982709412 shutdown_connections 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 --2- 172.21.15.45:0/1982709412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c06d3d0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 --2- 172.21.15.45:0/1982709412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0cc200 0x7f359c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- 172.21.15.45:0/1982709412 >> 172.21.15.45:0/1982709412 conn(0x7f359c072290 msgr2=0x7f359c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- 172.21.15.45:0/1982709412 shutdown_connections 2024-04-25T10:25:43.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- 172.21.15.45:0/1982709412 wait complete. 2024-04-25T10:25:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 Processor -- start 2024-04-25T10:25:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- start start 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c0997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359c0cc200 0x7f359c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0d0d00 0x7f359c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c15e040 con 0x7f359c06cf40 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c162aa0 con 0x7f359c0cc200 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.851+0000 7f35a2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c15e1c0 con 0x7f359c0d0d00 2024-04-25T10:25:43.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c0997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c0997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39622/0 (socket says 172.21.15.45:39622) 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 -- 172.21.15.45:0/1062811937 learned_addr learned my addr 172.21.15.45:0/1062811937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f3592a00640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359c0cc200 0x7f359c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359be00640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0d0d00 0x7f359c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 -- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0d0d00 msgr2=0x7f359c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0d0d00 0x7f359c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 -- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359c0cc200 msgr2=0x7f359c155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359c0cc200 0x7f359c155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 -- 172.21.15.45:0/1062811937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0ad1d0 con 0x7f359c06cf40 2024-04-25T10:25:43.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f359b400640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c0997e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f358c001ec0 tx=0x7f358c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f3593e00640 1 -- 172.21.15.45:0/1062811937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c017070 con 0x7f359c06cf40 2024-04-25T10:25:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f3593e00640 1 -- 172.21.15.45:0/1062811937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c003600 con 0x7f359c06cf40 2024-04-25T10:25:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c0ad4b0 con 0x7f359c06cf40 2024-04-25T10:25:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f3593e00640 1 -- 172.21.15.45:0/1062811937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0039f0 con 0x7f359c06cf40 2024-04-25T10:25:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.855+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c0ada70 con 0x7f359c06cf40 2024-04-25T10:25:43.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3593e00640 1 -- 172.21.15.45:0/1062811937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f358c01a070 con 0x7f359c06cf40 2024-04-25T10:25:43.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3593e00640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3578066f60 0x7f3578069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3593e00640 1 -- 172.21.15.45:0/1062811937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f358c0843f0 con 0x7f359c06cf40 2024-04-25T10:25:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 Processor -- start 2024-04-25T10:25:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3578066f60 0x7f3578069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 -- start start 2024-04-25T10:25:43.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3578066f60 0x7f3578069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f359c0cba40 tx=0x7f3584009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f2930e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f28eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f28ebd90 0x55f9f28ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f27ce1f0 con 0x55f9f2931360 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2734e90 con 0x55f9f28ebd90 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9f26bcf50 con 0x55f9f2930a40 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f28eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f28eb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39632/0 (socket says 172.21.15.45:39632) 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 -- 172.21.15.45:0/1318101891 learned_addr learned my addr 172.21.15.45:0/1318101891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f359b400640 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f2930e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f359be00640 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f28ebd90 0x55f9f28ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 -- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 msgr2=0x55f9f2930e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f2930e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 -- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f28ebd90 msgr2=0x55f9f28ec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f28ebd90 0x55f9f28ec170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 -- 172.21.15.45:0/1318101891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9f270c8b0 con 0x55f9f2931360 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f3592a00640 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f28eb850 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3584006c10 tx=0x7f3584014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50be2ceda7f7de92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f358be00640 1 -- 172.21.15.45:0/1318101891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358401b070 con 0x55f9f2931360 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f358be00640 1 -- 172.21.15.45:0/1318101891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584027070 con 0x55f9f2931360 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.859+0000 7f358be00640 1 -- 172.21.15.45:0/1318101891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584003a30 con 0x55f9f2931360 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/1318101891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 msgr2=0x55f9f28eb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f28eb850 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3584006c10 tx=0x7f3584014000 comp rx=0 tx=0).stop 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/1318101891 shutdown_connections 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f28ebd90 0x55f9f28ec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f28eb850 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- 172.21.15.45:0/1318101891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f2930e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/1318101891 >> 172.21.15.45:0/1318101891 conn(0x55f9f28e7d40 msgr2=0x55f9f28ea130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/1318101891 shutdown_connections 2024-04-25T10:25:43.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/1318101891 wait complete. 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- start start 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 Processor -- start 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f29381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f2938700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2931360 0x55f9f28f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f26ef540 con 0x55f9f28ebd90 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2730f30 con 0x55f9f2930a40 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9f28d25b0 con 0x55f9f2931360 2024-04-25T10:25:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 10 client.0 _ll_register_callbacks cb 0x55f9f27e9180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f29381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2931360 0x55f9f28f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3592a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f2938700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3592a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f2938700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37456/0 (socket says 172.21.15.45:37456) 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2931360 0x55f9f28f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46708/0 (socket says 172.21.15.45:46708) 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3592a00640 1 -- 172.21.15.45:0/599861436 learned_addr learned my addr 172.21.15.45:0/599861436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 -- 172.21.15.45:0/599861436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2931360 msgr2=0x55f9f28f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2931360 0x55f9f28f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 -- 172.21.15.45:0/599861436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 msgr2=0x55f9f2938700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f2938700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 -- 172.21.15.45:0/599861436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9f26bd1a0 con 0x55f9f28ebd90 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f359b400640 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f29381c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f358c082910 tx=0x7f358c05b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b671a6e6e54efac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3588c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:43.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3588c00640 1 -- 172.21.15.45:0/599861436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c003b90 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3588c00640 1 -- 172.21.15.45:0/599861436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f358c003d30 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3584027070 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3588c00640 1 -- 172.21.15.45:0/599861436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f358c0541a0 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 10 client.8037 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55f9f27b7840 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3588c00640 1 -- 172.21.15.45:0/599861436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584027070 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f3588c00640 1 -- 172.21.15.45:0/599861436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f358c082ce0 con 0x55f9f28ebd90 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 10 client.8037 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 10 client.8037 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 20 client.8037 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f28f2c40 0x55f9f28f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 4 client.8037 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.863+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9f26d0be0 con 0x55f9f28f2c40 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f3592a00640 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f28f2c40 0x55f9f28f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f3592a00640 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f28f2c40 0x55f9f28f3020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5821fad9dd317204 server_cookie=49548b3875af79a6 in_seq=0 out_seq=0 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f3588c00640 10 client.8037 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f3588c00640 1 -- 172.21.15.45:0/599861436 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9f26d0be0 con 0x55f9f28f2c40 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f3588c00640 10 client.8037 handle_command_reply: tid=0 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 client.8037 shutdown 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 msgr2=0x55f9f29381c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f29381c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f358c082910 tx=0x7f358c05b1f0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 shutdown_connections 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2931360 0x55f9f28f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f28f2c40 0x55f9f28f3020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f2938700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 --2- 172.21.15.45:0/599861436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f29381c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f3588c00640 0 client.8037 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 >> 172.21.15.45:0/599861436 conn(0x55f9f28e7d40 msgr2=0x55f9f28ea9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 shutdown_connections 2024-04-25T10:25:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 1 -- 172.21.15.45:0/599861436 wait complete. 2024-04-25T10:25:43.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.867+0000 7f35a5aec000 20 client.8037 trim_cache size 0 max 16384 2024-04-25T10:25:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 Processor -- start 2024-04-25T10:25:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- start start 2024-04-25T10:25:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f298d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f298d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2931360 0x55f9f298dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c202c90 con 0x55f9f28ebd90 2024-04-25T10:25:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c201e90 con 0x55f9f2931360 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c201290 con 0x55f9f2930a40 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f298d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f298d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39652/0 (socket says 172.21.15.45:39652) 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 -- 172.21.15.45:0/712788583 learned_addr learned my addr 172.21.15.45:0/712788583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f3592a00640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f298d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 -- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 msgr2=0x55f9f298d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359be00640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2931360 0x55f9f298dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f298d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 -- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2931360 msgr2=0x55f9f298dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2931360 0x55f9f298dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 -- 172.21.15.45:0/712788583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9f278ea10 con 0x55f9f28ebd90 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f3592a00640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f298d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359be00640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2931360 0x55f9f298dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f359b400640 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f298d290 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f358c05ce80 tx=0x7f358c057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ae563b9c88c84b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f358b400640 1 -- 172.21.15.45:0/712788583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c054390 con 0x55f9f28ebd90 2024-04-25T10:25:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f358b400640 1 -- 172.21.15.45:0/712788583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c075020 con 0x55f9f28ebd90 2024-04-25T10:25:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f358b400640 1 -- 172.21.15.45:0/712788583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c003b90 con 0x55f9f28ebd90 2024-04-25T10:25:43.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- 172.21.15.45:0/712788583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 msgr2=0x55f9f298d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f298d290 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f358c05ce80 tx=0x7f358c057000 comp rx=0 tx=0).stop 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- 172.21.15.45:0/712788583 shutdown_connections 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2931360 0x55f9f298dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f2930a40 0x55f9f298d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 --2- 172.21.15.45:0/712788583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f28ebd90 0x55f9f298d290 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- 172.21.15.45:0/712788583 >> 172.21.15.45:0/712788583 conn(0x55f9f28e7d40 msgr2=0x55f9f293e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- 172.21.15.45:0/712788583 shutdown_connections 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.883+0000 7f35a5aec000 1 -- 172.21.15.45:0/712788583 wait complete. 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- start start 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 Processor -- start 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f28ebd90 0x55f9f2991090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f29915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f29356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9f2713010 con 0x55f9f2931360 2024-04-25T10:25:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c201890 con 0x55f9f2930a40 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9f27341a0 con 0x55f9f28ebd90 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 10 client.0 _ll_register_callbacks cb 0x55f9f26f7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f29356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3592a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f29915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f29356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39660/0 (socket says 172.21.15.45:39660) 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 -- 172.21.15.45:0/866392700 learned_addr learned my addr 172.21.15.45:0/866392700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359b400640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f28ebd90 0x55f9f2991090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 -- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f28ebd90 msgr2=0x55f9f2991090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f28ebd90 0x55f9f2991090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 -- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 msgr2=0x55f9f29915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f29915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 -- 172.21.15.45:0/866392700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9f26c9b00 con 0x55f9f2931360 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3592a00640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f29915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:43.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359b400640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f28ebd90 0x55f9f2991090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359be00640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f29356b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f359401a5d0 tx=0x7f359401b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562f979dc4590cad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3588c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3588c00640 1 -- 172.21.15.45:0/866392700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940c5070 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3588c00640 1 -- 172.21.15.45:0/866392700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f35940a5a20 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f358c075020 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3588c00640 1 -- 172.21.15.45:0/866392700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35940d0050 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 10 client.8043 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55f9f26dbf80 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3588c00640 1 -- 172.21.15.45:0/866392700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c075020 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f3588c00640 1 -- 172.21.15.45:0/866392700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f35940d3020 con 0x55f9f2931360 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 10 client.8043 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 10 client.8043 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 20 client.8043 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:43.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f29363b0 0x55f9f29367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:43.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 4 client.8043 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:25:43.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55f9f27423d0 con 0x55f9f29363b0 2024-04-25T10:25:43.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.887+0000 7f359b400640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f29363b0 0x55f9f29367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:43.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f3588c00640 10 client.8043 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f359b400640 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f29363b0 0x55f9f29367c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf4aba82290dd4b2 server_cookie=246f74e6910a66cd in_seq=0 out_seq=0 2024-04-25T10:25:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f3588c00640 1 -- 172.21.15.45:0/866392700 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55f9f27423d0 con 0x55f9f29363b0 2024-04-25T10:25:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f3588c00640 10 client.8043 handle_command_reply: tid=0 2024-04-25T10:25:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 client.8043 shutdown 2024-04-25T10:25:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 msgr2=0x55f9f29356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f29356b0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f359401a5d0 tx=0x7f359401b1e0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 shutdown_connections 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f9f2931360 0x55f9f29356b0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f9f29363b0 0x55f9f29367c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9f2930a40 0x55f9f29915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 --2- 172.21.15.45:0/866392700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f9f28ebd90 0x55f9f2991090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f3588c00640 0 client.8043 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 >> 172.21.15.45:0/866392700 conn(0x55f9f28e7d40 msgr2=0x55f9f292f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 shutdown_connections 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 1 -- 172.21.15.45:0/866392700 wait complete. 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a5aec000 20 client.8043 trim_cache size 0 max 16384 2024-04-25T10:25:43.897 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:43.898 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:25:43.898 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3578066f60 msgr2=0x7f3578069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3578066f60 0x7f3578069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f359c0cba40 tx=0x7f3584009190 comp rx=0 tx=0).stop 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 msgr2=0x7f359c0997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c0997e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f358c001ec0 tx=0x7f358c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 shutdown_connections 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3578066f60 0x7f3578069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f359c0d0d00 0x7f359c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359c0cc200 0x7f359c155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 --2- 172.21.15.45:0/1062811937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359c06cf40 0x7f359c0997e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 >> 172.21.15.45:0/1062811937 conn(0x7f359c072290 msgr2=0x7f359c15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 shutdown_connections 2024-04-25T10:25:43.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:43.891+0000 7f35a2600640 1 -- 172.21.15.45:0/1062811937 wait complete. 2024-04-25T10:25:43.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:43.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:25:44.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 Processor -- start 2024-04-25T10:25:44.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 -- start start 2024-04-25T10:25:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 0x7f22cc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22cc0cd7f0 0x7f22cc15d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22cc0cc080 con 0x7f22cc06aea0 2024-04-25T10:25:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0cc320 con 0x7f22cc0cd7f0 2024-04-25T10:25:44.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d3000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0cc600 con 0x7f22cc0b2e70 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 0x7f22cc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d1000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22cc0cd7f0 0x7f22cc15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 0x7f22cc0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:46744/0 (socket says 172.21.15.45:46744) 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 -- 172.21.15.45:0/3402745894 learned_addr learned my addr 172.21.15.45:0/3402745894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22d1000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22cc0cd7f0 0x7f22cc15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37468/0 (socket says 172.21.15.45:37468) 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 -- 172.21.15.45:0/3402745894 >> v2:172.21.15.117:3300/0 conn(0x7f22cc0cd7f0 msgr2=0x7f22cc15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 --2- 172.21.15.45:0/3402745894 >> v2:172.21.15.117:3300/0 conn(0x7f22cc0cd7f0 0x7f22cc15d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 -- 172.21.15.45:0/3402745894 >> v1:172.21.15.117:6789/0 conn(0x7f22cc06aea0 legacy=0x7f22cc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.055+0000 7f22cbe00640 1 -- 172.21.15.45:0/3402745894 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0cc8e0 con 0x7f22cc0b2e70 2024-04-25T10:25:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22cbe00640 1 --2- 172.21.15.45:0/3402745894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 0x7f22cc0b47e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0cc250 tx=0x7f22bc01ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a623e0d4628f03f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22caa00640 1 -- 172.21.15.45:0/3402745894 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0b5070 con 0x7f22cc0b2e70 2024-04-25T10:25:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22caa00640 1 -- 172.21.15.45:0/3402745894 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc0a8a30 con 0x7f22cc0b2e70 2024-04-25T10:25:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22caa00640 1 -- 172.21.15.45:0/3402745894 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0a8e00 con 0x7f22cc0b2e70 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 -- 172.21.15.45:0/3402745894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 msgr2=0x7f22cc0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 --2- 172.21.15.45:0/3402745894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 0x7f22cc0b47e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0cc250 tx=0x7f22bc01ed30 comp rx=0 tx=0).stop 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 -- 172.21.15.45:0/3402745894 shutdown_connections 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 --2- 172.21.15.45:0/3402745894 >> v2:172.21.15.117:3300/0 conn(0x7f22cc0cd7f0 0x7f22cc15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 --2- 172.21.15.45:0/3402745894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0b2e70 0x7f22cc0b47e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 -- 172.21.15.45:0/3402745894 >> 172.21.15.45:0/3402745894 conn(0x7f22cc0b34a0 msgr2=0x7f22cc0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 -- 172.21.15.45:0/3402745894 shutdown_connections 2024-04-25T10:25:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 -- 172.21.15.45:0/3402745894 wait complete. 2024-04-25T10:25:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.059+0000 7f22d3000640 1 Processor -- start 2024-04-25T10:25:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 -- start start 2024-04-25T10:25:44.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 0x7f22cc155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0b2e70 0x7f22cc0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0cd7f0 0x7f22cc0b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0cdc80 con 0x7f22cc06aea0 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0b3250 con 0x7f22cc0b2e70 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0cde00 con 0x7f22cc0cd7f0 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 0x7f22cc155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 0x7f22cc155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39668/0 (socket says 172.21.15.45:39668) 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 -- 172.21.15.45:0/2509925529 learned_addr learned my addr 172.21.15.45:0/2509925529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d1000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0cd7f0 0x7f22cc0b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cb400640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0b2e70 0x7f22cc0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 -- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0cd7f0 msgr2=0x7f22cc0b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0cd7f0 0x7f22cc0b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 -- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0b2e70 msgr2=0x7f22cc0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0b2e70 0x7f22cc0b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 -- 172.21.15.45:0/2509925529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc1564a0 con 0x7f22cc06aea0 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22cbe00640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 0x7f22cc155f60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0b3040 tx=0x7f22bc01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22c3e00640 1 -- 172.21.15.45:0/2509925529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0b5070 con 0x7f22cc06aea0 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22c3e00640 1 -- 172.21.15.45:0/2509925529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc0b0a50 con 0x7f22cc06aea0 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22c3e00640 1 -- 172.21.15.45:0/2509925529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0b0e40 con 0x7f22cc06aea0 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc0c6130 con 0x7f22cc06aea0 2024-04-25T10:25:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.063+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc0c66f0 con 0x7f22cc06aea0 2024-04-25T10:25:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22c3e00640 1 -- 172.21.15.45:0/2509925529 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22bc0c2060 con 0x7f22cc06aea0 2024-04-25T10:25:44.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22c3e00640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22ac066f60 0x7f22ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22c3e00640 1 -- 172.21.15.45:0/2509925529 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f22bc122060 con 0x7f22cc06aea0 2024-04-25T10:25:44.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 Processor -- start 2024-04-25T10:25:44.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22cb400640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22ac066f60 0x7f22ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 -- start start 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22cb400640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22ac066f60 0x7f22ac069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22cc1f70b0 tx=0x7f22b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9afedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ab4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ab5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f9901ff0 con 0x55f0f9aff2f0 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f988a150 con 0x55f0f9ab50a0 2024-04-25T10:25:44.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0f98fe090 con 0x55f0f9afe9d0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22cb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ab4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22cb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ab4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39682/0 (socket says 172.21.15.45:39682) 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22cb400640 1 -- 172.21.15.45:0/633257002 learned_addr learned my addr 172.21.15.45:0/633257002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22cbe00640 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9afedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.067+0000 7f22d1000640 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ab5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 -- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 msgr2=0x55f0f9afedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9afedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 -- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 msgr2=0x55f0f9ab5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ab5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 -- 172.21.15.45:0/633257002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0f9a980d0 con 0x55f0f9aff2f0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cbe00640 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9afedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ab4b60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f22b800fae0 tx=0x7f22b8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eadd167109d5de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22c0c00640 1 -- 172.21.15.45:0/633257002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8025070 con 0x55f0f9aff2f0 2024-04-25T10:25:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22c0c00640 1 -- 172.21.15.45:0/633257002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8015bf0 con 0x55f0f9aff2f0 2024-04-25T10:25:44.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22c0c00640 1 -- 172.21.15.45:0/633257002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8020400 con 0x55f0f9aff2f0 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- 172.21.15.45:0/633257002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 msgr2=0x55f0f9ab4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ab4b60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f22b800fae0 tx=0x7f22b8000ee0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- 172.21.15.45:0/633257002 shutdown_connections 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ab5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ab4b60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- 172.21.15.45:0/633257002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9afedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- 172.21.15.45:0/633257002 >> 172.21.15.45:0/633257002 conn(0x55f0f9ab1020 msgr2=0x55f0f9ab3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- 172.21.15.45:0/633257002 shutdown_connections 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- 172.21.15.45:0/633257002 wait complete. 2024-04-25T10:25:44.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- start start 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 Processor -- start 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ae37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9ae3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ae4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f98c6250 con 0x55f0f9ab50a0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f98bc690 con 0x55f0f9aff2f0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0f988aa40 con 0x55f0f9afe9d0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d63da000 10 client.0 _ll_register_callbacks cb 0x55f0f9a9fe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ae37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ae37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39690/0 (socket says 172.21.15.45:39690) 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cbe00640 1 -- 172.21.15.45:0/3161992687 learned_addr learned my addr 172.21.15.45:0/3161992687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22d1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ae4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.071+0000 7f22cb400640 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9ae3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cbe00640 1 -- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 msgr2=0x55f0f9ae3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cbe00640 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9ae3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cbe00640 1 -- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 msgr2=0x55f0f9ae4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cbe00640 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ae4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cbe00640 1 -- 172.21.15.45:0/3161992687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0f99af1c0 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cbe00640 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ae37f0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0eee20 tx=0x7f22bc0a8af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20971db0c01623d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 1 -- 172.21.15.45:0/3161992687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0fad90 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 1 -- 172.21.15.45:0/3161992687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f22bc0f1e10 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b8015bf0 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 1 -- 172.21.15.45:0/3161992687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22bc0fb020 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 10 client.8052 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55f0f99ab170 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 1 -- 172.21.15.45:0/3161992687 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8015bf0 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 1 -- 172.21.15.45:0/3161992687 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22bc0ee710 con 0x55f0f9ab50a0 2024-04-25T10:25:44.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 10 client.8052 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 10 client.8052 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 20 client.8052 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b0ec80 0x55f0f9b11070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 4 client.8052 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0f989dc60 con 0x55f0f9b0ec80 2024-04-25T10:25:44.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cb400640 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b0ec80 0x55f0f9b11070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22cb400640 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b0ec80 0x55f0f9b11070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed1e2106f5b8430 server_cookie=20a0275b6271a35 in_seq=0 out_seq=0 2024-04-25T10:25:44.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 10 client.8052 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 1 -- 172.21.15.45:0/3161992687 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0f989dc60 con 0x55f0f9b0ec80 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 10 client.8052 handle_command_reply: tid=0 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 client.8052 shutdown 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 msgr2=0x55f0f9ae37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ae37f0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0eee20 tx=0x7f22bc0a8af0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 shutdown_connections 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b0ec80 0x55f0f9b11070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9ae4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9afe9d0 0x55f0f9ae3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 --2- 172.21.15.45:0/3161992687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9ab50a0 0x55f0f9ae37f0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22b5600640 0 client.8052 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 >> 172.21.15.45:0/3161992687 conn(0x55f0f9ab1020 msgr2=0x55f0f9ab3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 shutdown_connections 2024-04-25T10:25:44.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 1 -- 172.21.15.45:0/3161992687 wait complete. 2024-04-25T10:25:44.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.075+0000 7f22d63da000 20 client.8052 trim_cache size 0 max 16384 2024-04-25T10:25:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 Processor -- start 2024-04-25T10:25:44.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 -- start start 2024-04-25T10:25:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9b5aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9aff2f0 0x55f0f9b5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc1ffee0 con 0x55f0f9afe9d0 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc15b140 con 0x55f0f9ab50a0 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0c9560 con 0x55f0f9aff2f0 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39706/0 (socket says 172.21.15.45:39706) 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 -- 172.21.15.45:0/1243314269 learned_addr learned my addr 172.21.15.45:0/1243314269 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d1000640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9aff2f0 0x55f0f9b5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cbe00640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9b5aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 -- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9aff2f0 msgr2=0x55f0f9b5b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9aff2f0 0x55f0f9b5b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 -- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 msgr2=0x55f0f9b5aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9b5aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cb400640 1 -- 172.21.15.45:0/1243314269 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0f9948de0 con 0x55f0f9afe9d0 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22d1000640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9aff2f0 0x55f0f9b5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.091+0000 7f22cbe00640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9b5aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b5b420 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f22b8023040 tx=0x7f22b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8732a588ff5b4ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22b7e00640 1 -- 172.21.15.45:0/1243314269 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8025040 con 0x55f0f9afe9d0 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22b7e00640 1 -- 172.21.15.45:0/1243314269 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b802e020 con 0x55f0f9afe9d0 2024-04-25T10:25:44.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22b7e00640 1 -- 172.21.15.45:0/1243314269 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8030d00 con 0x55f0f9afe9d0 2024-04-25T10:25:44.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- 172.21.15.45:0/1243314269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 msgr2=0x55f0f9b5b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b5b420 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f22b8023040 tx=0x7f22b8014000 comp rx=0 tx=0).stop 2024-04-25T10:25:44.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- 172.21.15.45:0/1243314269 shutdown_connections 2024-04-25T10:25:44.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9aff2f0 0x55f0f9b5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b5b420 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- 172.21.15.45:0/1243314269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9ab50a0 0x55f0f9b5aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- 172.21.15.45:0/1243314269 >> 172.21.15.45:0/1243314269 conn(0x55f0f9ab1020 msgr2=0x55f0f9afd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- 172.21.15.45:0/1243314269 shutdown_connections 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- 172.21.15.45:0/1243314269 wait complete. 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- start start 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 Processor -- start 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9ab50a0 0x55f0f9b117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b11d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9b12260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f989ce00 con 0x55f0f9afe9d0 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc15ab40 con 0x55f0f9aff2f0 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0f9901300 con 0x55f0f9ab50a0 2024-04-25T10:25:44.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 10 client.0 _ll_register_callbacks cb 0x55f0f9a80040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b11d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9ab50a0 0x55f0f9b117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b11d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39710/0 (socket says 172.21.15.45:39710) 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 -- 172.21.15.45:0/2183946446 learned_addr learned my addr 172.21.15.45:0/2183946446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d1000640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9b12260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 -- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9ab50a0 msgr2=0x55f0f9b117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9ab50a0 0x55f0f9b117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 -- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 msgr2=0x55f0f9b12260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9b12260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 -- 172.21.15.45:0/2183946446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0f98a38e0 con 0x55f0f9afe9d0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cbe00640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9ab50a0 0x55f0f9b117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22cb400640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b11d20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f22b8020000 tx=0x7f22b8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b2509692caaef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22b5600640 1 -- 172.21.15.45:0/2183946446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8036070 con 0x55f0f9afe9d0 2024-04-25T10:25:44.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22b5600640 1 -- 172.21.15.45:0/2183946446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f22b8030b00 con 0x55f0f9afe9d0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.095+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b802e020 con 0x55f0f9afe9d0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 1 -- 172.21.15.45:0/2183946446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22b8025040 con 0x55f0f9afe9d0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 10 client.8058 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55f0f995b8f0 con 0x55f0f9afe9d0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 1 -- 172.21.15.45:0/2183946446 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b802e020 con 0x55f0f9afe9d0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 1 -- 172.21.15.45:0/2183946446 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22b8044080 con 0x55f0f9afe9d0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 10 client.8058 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 10 client.8058 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 20 client.8058 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b12c50 0x55f0f9b66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 4 client.8058 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f0f990f530 con 0x55f0f9b12c50 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22cbe00640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b12c50 0x55f0f9b66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22cbe00640 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b12c50 0x55f0f9b66f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb970755826bacd3 server_cookie=b807cce9301f81f5 in_seq=0 out_seq=0 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 10 client.8058 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 1 -- 172.21.15.45:0/2183946446 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55f0f990f530 con 0x55f0f9b12c50 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 10 client.8058 handle_command_reply: tid=0 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 client.8058 shutdown 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 msgr2=0x55f0f9b11d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b11d20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f22b8020000 tx=0x7f22b8000bd0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 shutdown_connections 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x55f0f9b12c50 0x55f0f9b66f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0f9aff2f0 0x55f0f9b12260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f0f9afe9d0 0x55f0f9b11d20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 --2- 172.21.15.45:0/2183946446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f0f9ab50a0 0x55f0f9b117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22b5600640 0 client.8058 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 >> 172.21.15.45:0/2183946446 conn(0x55f0f9ab1020 msgr2=0x55f0f9afd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 shutdown_connections 2024-04-25T10:25:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 1 -- 172.21.15.45:0/2183946446 wait complete. 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d63da000 20 client.8058 trim_cache size 0 max 16384 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: "d842100b-563d-4e46-beda-2619600fc8ce": { 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "d842100b-563d-4e46-beda-2619600fc8ce", 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:44.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:25:44.107 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22ac066f60 msgr2=0x7f22ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22ac066f60 0x7f22ac069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22cc1f70b0 tx=0x7f22b8009040 comp rx=0 tx=0).stop 2024-04-25T10:25:44.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 msgr2=0x7f22cc155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 0x7f22cc155f60 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0b3040 tx=0x7f22bc01bad0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 shutdown_connections 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22ac066f60 0x7f22ac069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22cc0cd7f0 0x7f22cc0b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22cc0b2e70 0x7f22cc0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 --2- 172.21.15.45:0/2509925529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22cc06aea0 0x7f22cc155f60 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 >> 172.21.15.45:0/2509925529 conn(0x7f22cc0b34a0 msgr2=0x7f22cc06f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 shutdown_connections 2024-04-25T10:25:44.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.099+0000 7f22d3000640 1 -- 172.21.15.45:0/2509925529 wait complete. 2024-04-25T10:25:44.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f438cc00640 1 Processor -- start 2024-04-25T10:25:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f438cc00640 1 -- start start 2024-04-25T10:25:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f438cc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43880cef90 0x7f4388070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f438cc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880b4470 con 0x7f438806ca50 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f438cc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880b4710 con 0x7f43880cf930 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f438cc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43880b4cd0 con 0x7f43880cef90 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4386800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43880cef90 0x7f4388070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4386800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43880cef90 0x7f4388070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39712/0 (socket says 172.21.15.45:39712) 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4386800640 1 -- 172.21.15.45:0/4277316516 learned_addr learned my addr 172.21.15.45:0/4277316516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f437fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f438806ca50 0x7f4388070310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56560/0 (socket says 172.21.15.45:56560) 2024-04-25T10:25:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4386800640 1 -- 172.21.15.45:0/4277316516 >> v1:172.21.15.117:6789/0 conn(0x7f43880cf930 legacy=0x7f43880cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4385400640 1 -- 172.21.15.45:0/4277316516 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118780637 0 0) 0x7f43880b4470 con 0x7f438806ca50 2024-04-25T10:25:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4386800640 1 -- 172.21.15.45:0/4277316516 >> v1:172.21.15.45:6789/0 conn(0x7f438806ca50 legacy=0x7f4388070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.259+0000 7f4386800640 1 -- 172.21.15.45:0/4277316516 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880b49f0 con 0x7f43880cef90 2024-04-25T10:25:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f4386800640 1 --2- 172.21.15.45:0/4277316516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43880cef90 0x7f4388070a20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4380009e50 tx=0x7f438001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f2baf977db37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f4385400640 1 -- 172.21.15.45:0/4277316516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43800a5660 con 0x7f43880cef90 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f4385400640 1 -- 172.21.15.45:0/4277316516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43800a5800 con 0x7f43880cef90 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f4385400640 1 -- 172.21.15.45:0/4277316516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43800a5bd0 con 0x7f43880cef90 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 -- 172.21.15.45:0/4277316516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43880cef90 msgr2=0x7f4388070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 --2- 172.21.15.45:0/4277316516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43880cef90 0x7f4388070a20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4380009e50 tx=0x7f438001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 -- 172.21.15.45:0/4277316516 shutdown_connections 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 --2- 172.21.15.45:0/4277316516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43880cef90 0x7f4388070a20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 -- 172.21.15.45:0/4277316516 >> 172.21.15.45:0/4277316516 conn(0x7f4388066b70 msgr2=0x7f4388066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 -- 172.21.15.45:0/4277316516 shutdown_connections 2024-04-25T10:25:44.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.263+0000 7f438cc00640 1 -- 172.21.15.45:0/4277316516 wait complete. 2024-04-25T10:25:44.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 Processor -- start 2024-04-25T10:25:44.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 -- start start 2024-04-25T10:25:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 0x7f4388153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43880cef90 0x7f4388160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43880cf930 0x7f4388160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438806ce90 con 0x7f438806ca50 2024-04-25T10:25:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4388072050 con 0x7f43880cf930 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880b2e30 con 0x7f43880cef90 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 0x7f4388153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 0x7f4388153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39728/0 (socket says 172.21.15.45:39728) 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 -- 172.21.15.45:0/1960823814 learned_addr learned my addr 172.21.15.45:0/1960823814 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4386800640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43880cf930 0x7f4388160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f437fe00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43880cef90 0x7f4388160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 -- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43880cef90 msgr2=0x7f4388160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43880cef90 0x7f4388160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 -- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43880cf930 msgr2=0x7f4388160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43880cf930 0x7f4388160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 -- 172.21.15.45:0/1960823814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438815d430 con 0x7f438806ca50 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f437fe00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43880cef90 0x7f4388160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f4385e00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 0x7f4388153d70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f437800c370 tx=0x7f437800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43780052d0 con 0x7f438806ca50 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378005470 con 0x7f438806ca50 2024-04-25T10:25:44.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378007260 con 0x7f438806ca50 2024-04-25T10:25:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43881542b0 con 0x7f438806ca50 2024-04-25T10:25:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.267+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438815ce70 con 0x7f438806ca50 2024-04-25T10:25:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.271+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438804ef30 con 0x7f438806ca50 2024-04-25T10:25:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.271+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4378007400 con 0x7f438806ca50 2024-04-25T10:25:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.271+0000 7f437e000640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4364066f60 0x7f4364069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.271+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f43780844e0 con 0x7f438806ca50 2024-04-25T10:25:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.271+0000 7f437fe00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4364066f60 0x7f4364069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.271+0000 7f437fe00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4364066f60 0x7f4364069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f43880ce530 tx=0x7f4370007c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.275+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4378054da0 con 0x7f438806ca50 2024-04-25T10:25:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438815df80 con 0x7f438806ca50 2024-04-25T10:25:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f437e000640 1 -- 172.21.15.45:0/1960823814 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4378058e40 con 0x7f438806ca50 2024-04-25T10:25:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:44.477 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:44.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4364066f60 msgr2=0x7f4364069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4364066f60 0x7f4364069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f43880ce530 tx=0x7f4370007c80 comp rx=0 tx=0).stop 2024-04-25T10:25:44.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 msgr2=0x7f4388153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 0x7f4388153d70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f437800c370 tx=0x7f437800c830 comp rx=0 tx=0).stop 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 shutdown_connections 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4364066f60 0x7f4364069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43880cf930 0x7f4388160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43880cef90 0x7f4388160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 --2- 172.21.15.45:0/1960823814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f438806ca50 0x7f4388153d70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 >> 172.21.15.45:0/1960823814 conn(0x7f4388066b70 msgr2=0x7f43880cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 shutdown_connections 2024-04-25T10:25:44.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.475+0000 7f438cc00640 1 -- 172.21.15.45:0/1960823814 wait complete. 2024-04-25T10:25:44.498 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:44.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:44.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.635+0000 7fc32fe00640 1 Processor -- start 2024-04-25T10:25:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.635+0000 7fc32fe00640 1 -- start start 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc3300adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32fe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300b34f0 con 0x7fc3301582f0 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32fe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300714f0 con 0x7fc330070be0 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300b37a0 con 0x7fc330071060 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc3301582f0 0x7fc3300b2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56570/0 (socket says 172.21.15.45:56570) 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32ea00640 1 -- 172.21.15.45:0/952581905 learned_addr learned my addr 172.21.15.45:0/952581905 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32f400640 1 --2- 172.21.15.45:0/952581905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc3300adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32f400640 1 -- 172.21.15.45:0/952581905 >> v1:172.21.15.117:6789/0 conn(0x7fc330070be0 legacy=0x7fc3300ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32e000640 1 -- 172.21.15.45:0/952581905 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537904272 0 0) 0x7fc3300b34f0 con 0x7fc3301582f0 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32f400640 1 -- 172.21.15.45:0/952581905 >> v1:172.21.15.45:6789/0 conn(0x7fc3301582f0 legacy=0x7fc3300b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.639+0000 7fc32f400640 1 -- 172.21.15.45:0/952581905 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3300b3a80 con 0x7fc330071060 2024-04-25T10:25:44.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32f400640 1 --2- 172.21.15.45:0/952581905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc3300adf50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc328009dc0 tx=0x7fc32801c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e2e1bd371b9f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32e000640 1 -- 172.21.15.45:0/952581905 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280b2070 con 0x7fc330071060 2024-04-25T10:25:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32e000640 1 -- 172.21.15.45:0/952581905 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3280a5770 con 0x7fc330071060 2024-04-25T10:25:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32e000640 1 -- 172.21.15.45:0/952581905 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280a5b40 con 0x7fc330071060 2024-04-25T10:25:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- 172.21.15.45:0/952581905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 msgr2=0x7fc3300adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 --2- 172.21.15.45:0/952581905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc3300adf50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc328009dc0 tx=0x7fc32801c840 comp rx=0 tx=0).stop 2024-04-25T10:25:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- 172.21.15.45:0/952581905 shutdown_connections 2024-04-25T10:25:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 --2- 172.21.15.45:0/952581905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc3300adf50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- 172.21.15.45:0/952581905 >> 172.21.15.45:0/952581905 conn(0x7fc3300b1ae0 msgr2=0x7fc3300b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- 172.21.15.45:0/952581905 shutdown_connections 2024-04-25T10:25:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- 172.21.15.45:0/952581905 wait complete. 2024-04-25T10:25:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 Processor -- start 2024-04-25T10:25:44.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- start start 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 0x7fc33014ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc330155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3301582f0 0x7fc330160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33016b9f0 con 0x7fc3301582f0 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33016b870 con 0x7fc330070be0 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33016bb70 con 0x7fc330071060 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 0x7fc33014ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3301582f0 0x7fc330160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 0x7fc33014ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37524/0 (socket says 172.21.15.45:37524) 2024-04-25T10:25:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc32ea00640 1 -- 172.21.15.45:0/1430672987 learned_addr learned my addr 172.21.15.45:0/1430672987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.643+0000 7fc326000640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc330155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32f400640 1 -- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 msgr2=0x7fc330155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32f400640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc330155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32f400640 1 -- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 msgr2=0x7fc33014ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32f400640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 0x7fc33014ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32f400640 1 -- 172.21.15.45:0/1430672987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33015fa90 con 0x7fc3301582f0 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32ea00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 0x7fc33014ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32f400640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3301582f0 0x7fc330160380 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc3280ac000 tx=0x7fc3300aeca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280b2040 con 0x7fc3301582f0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3280ad440 con 0x7fc3301582f0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3280ad7f0 con 0x7fc3301582f0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33015fd70 con 0x7fc3301582f0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33015b4a0 con 0x7fc3301582f0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc330074520 con 0x7fc3301582f0 2024-04-25T10:25:44.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3280c1020 con 0x7fc3301582f0 2024-04-25T10:25:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc327400640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc30c066f60 0x7fc30c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.647+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fc32811f700 con 0x7fc3301582f0 2024-04-25T10:25:44.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.651+0000 7fc32ea00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc30c066f60 0x7fc30c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:44.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.651+0000 7fc32ea00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc30c066f60 0x7fc30c069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc320000f70 tx=0x7fc320007ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:44.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.651+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3280164c0 con 0x7fc3301582f0 2024-04-25T10:25:44.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc33015ca10 con 0x7fc3301582f0 2024-04-25T10:25:44.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc327400640 1 -- 172.21.15.45:0/1430672987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7fc3280effc0 con 0x7fc3301582f0 2024-04-25T10:25:44.873 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:44.873 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":832,"data_bytes":849273048,"bytes_used":2074083328,"bytes_avail":771020029952,"bytes_total":773094113280,"write_bytes_sec":55059968,"read_op_per_sec":0,"write_op_per_sec":54},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:25:04.973359+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc30c066f60 msgr2=0x7fc30c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc30c066f60 0x7fc30c069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc320000f70 tx=0x7fc320007ee0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3301582f0 msgr2=0x7fc330160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3301582f0 0x7fc330160380 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc3280ac000 tx=0x7fc3300aeca0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 shutdown_connections 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc30c066f60 0x7fc30c069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3301582f0 0x7fc330160380 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc330071060 0x7fc330155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 --2- 172.21.15.45:0/1430672987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc330070be0 0x7fc33014ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 >> 172.21.15.45:0/1430672987 conn(0x7fc3300b1ae0 msgr2=0x7fc3300ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 shutdown_connections 2024-04-25T10:25:44.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:44.871+0000 7fc32fe00640 1 -- 172.21.15.45:0/1430672987 wait complete. 2024-04-25T10:25:44.894 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:25:45.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0689e00640 1 Processor -- start 2024-04-25T10:25:46.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0689e00640 1 -- start start 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0689e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06840cda10 0x7f068406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0689e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840b2770 con 0x7f068415bc40 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0689e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068406c670 con 0x7f06840cf930 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0689e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068406c950 con 0x7f06840cda10 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f067a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06840cda10 0x7f068406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0682a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f068415bc40 0x7f06840b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42038/0 (socket says 172.21.15.45:42038) 2024-04-25T10:25:46.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.047+0000 7f0682a00640 1 -- 172.21.15.45:0/760777995 learned_addr learned my addr 172.21.15.45:0/760777995 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f067a000640 1 -- 172.21.15.45:0/760777995 >> v1:172.21.15.117:6789/0 conn(0x7f06840cf930 legacy=0x7f06840b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f067a000640 1 -- 172.21.15.45:0/760777995 >> v1:172.21.15.45:6789/0 conn(0x7f068415bc40 legacy=0x7f06840b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f067a000640 1 -- 172.21.15.45:0/760777995 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068406cc30 con 0x7f06840cda10 2024-04-25T10:25:46.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f0682000640 1 -- 172.21.15.45:0/760777995 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542367108 0 0) 0x7f06840b2770 con 0x7f068415bc40 2024-04-25T10:25:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f067a000640 1 --2- 172.21.15.45:0/760777995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068406c090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f066c009d90 tx=0x7f066c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd47bee5ad84549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f0682000640 1 -- 172.21.15.45:0/760777995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0b2070 con 0x7f06840cda10 2024-04-25T10:25:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f0682000640 1 -- 172.21.15.45:0/760777995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c0a57b0 con 0x7f06840cda10 2024-04-25T10:25:46.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f0682000640 1 -- 172.21.15.45:0/760777995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0a5b80 con 0x7f06840cda10 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f0689e00640 1 -- 172.21.15.45:0/760777995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 msgr2=0x7f068406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.051+0000 7f0689e00640 1 --2- 172.21.15.45:0/760777995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068406c090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f066c009d90 tx=0x7f066c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- 172.21.15.45:0/760777995 shutdown_connections 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 --2- 172.21.15.45:0/760777995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068406c090 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- 172.21.15.45:0/760777995 >> 172.21.15.45:0/760777995 conn(0x7f0684066b70 msgr2=0x7f0684066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- 172.21.15.45:0/760777995 shutdown_connections 2024-04-25T10:25:46.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- 172.21.15.45:0/760777995 wait complete. 2024-04-25T10:25:46.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 Processor -- start 2024-04-25T10:25:46.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- start start 2024-04-25T10:25:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06840cf930 0x7f06840ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068415bc40 0x7f06840add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0684169d70 con 0x7f06840cda10 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0684169bf0 con 0x7f068415bc40 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0689e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0684169ef0 con 0x7f06840cf930 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068414eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44950/0 (socket says 172.21.15.45:44950) 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 -- 172.21.15.45:0/494819926 learned_addr learned my addr 172.21.15.45:0/494819926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f067a000640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06840cf930 0x7f06840ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0683400640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068415bc40 0x7f06840add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 -- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06840cf930 msgr2=0x7f06840ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06840cf930 0x7f06840ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 -- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068415bc40 msgr2=0x7f06840add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068415bc40 0x7f06840add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 -- 172.21.15.45:0/494819926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0684071580 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0683400640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068415bc40 0x7f06840add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f0682a00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068414eb90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f067400c4a0 tx=0x7f067400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.055+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674018070 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06740074e0 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674004a30 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0684071860 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0684071e20 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068404ef30 con 0x7f06840cda10 2024-04-25T10:25:46.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0674004bd0 con 0x7f06840cda10 2024-04-25T10:25:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067be00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0660066f60 0x7f0660069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f0674084310 con 0x7f06840cda10 2024-04-25T10:25:46.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067a000640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0660066f60 0x7f0660069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.059+0000 7f067a000640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0660066f60 0x7f0660069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f066c0af000 tx=0x7f066c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.063+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0674054c60 con 0x7f06840cda10 2024-04-25T10:25:46.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.263+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f068406d960 con 0x7f06840cda10 2024-04-25T10:25:46.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.263+0000 7f067be00640 1 -- 172.21.15.45:0/494819926 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0674058d00 con 0x7f06840cda10 2024-04-25T10:25:46.266 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:46.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:46.267 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:46.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0660066f60 msgr2=0x7f0660069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0660066f60 0x7f0660069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f066c0af000 tx=0x7f066c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 msgr2=0x7f068414eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068414eb90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f067400c4a0 tx=0x7f067400c960 comp rx=0 tx=0).stop 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 shutdown_connections 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0660066f60 0x7f0660069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f068415bc40 0x7f06840add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06840cf930 0x7f06840ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 --2- 172.21.15.45:0/494819926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06840cda10 0x7f068414eb90 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 >> 172.21.15.45:0/494819926 conn(0x7f0684066b70 msgr2=0x7f068415af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 shutdown_connections 2024-04-25T10:25:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.267+0000 7f0689e00640 1 -- 172.21.15.45:0/494819926 wait complete. 2024-04-25T10:25:46.289 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:46.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:46.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d4600640 1 Processor -- start 2024-04-25T10:25:46.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d4600640 1 -- start start 2024-04-25T10:25:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d4600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0cc0d0040 0x7fb0cc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d4600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0cc0b2770 con 0x7fb0cc0cda10 2024-04-25T10:25:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d4600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0cc06c670 con 0x7fb0cc15bc40 2024-04-25T10:25:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d4600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0cc06c950 con 0x7fb0cc0d0040 2024-04-25T10:25:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0cc0d0040 0x7fb0cc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d1200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb0cc0cda10 0x7fb0cc0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42054/0 (socket says 172.21.15.45:42054) 2024-04-25T10:25:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d1200640 1 -- 172.21.15.45:0/4039085635 learned_addr learned my addr 172.21.15.45:0/4039085635 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d2600640 1 -- 172.21.15.45:0/4039085635 >> v1:172.21.15.117:6789/0 conn(0x7fb0cc15bc40 legacy=0x7fb0cc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d2600640 1 -- 172.21.15.45:0/4039085635 >> v1:172.21.15.45:6789/0 conn(0x7fb0cc0cda10 legacy=0x7fb0cc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0cbe00640 1 -- 172.21.15.45:0/4039085635 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684723238 0 0) 0x7fb0cc0b2770 con 0x7fb0cc0cda10 2024-04-25T10:25:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.427+0000 7fb0d2600640 1 -- 172.21.15.45:0/4039085635 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc06cc30 con 0x7fb0cc0d0040 2024-04-25T10:25:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d2600640 1 --2- 172.21.15.45:0/4039085635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc06c090 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009d90 tx=0x7fb0c401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363101a63e036884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0cbe00640 1 -- 172.21.15.45:0/4039085635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40b2070 con 0x7fb0cc0d0040 2024-04-25T10:25:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0cbe00640 1 -- 172.21.15.45:0/4039085635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c40a58b0 con 0x7fb0cc0d0040 2024-04-25T10:25:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0cbe00640 1 -- 172.21.15.45:0/4039085635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40a5c80 con 0x7fb0cc0d0040 2024-04-25T10:25:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 -- 172.21.15.45:0/4039085635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc0d0040 msgr2=0x7fb0cc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 --2- 172.21.15.45:0/4039085635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc06c090 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009d90 tx=0x7fb0c401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 -- 172.21.15.45:0/4039085635 shutdown_connections 2024-04-25T10:25:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 --2- 172.21.15.45:0/4039085635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc06c090 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 -- 172.21.15.45:0/4039085635 >> 172.21.15.45:0/4039085635 conn(0x7fb0cc066b70 msgr2=0x7fb0cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 -- 172.21.15.45:0/4039085635 shutdown_connections 2024-04-25T10:25:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.431+0000 7fb0d4600640 1 -- 172.21.15.45:0/4039085635 wait complete. 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 Processor -- start 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 -- start start 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0cc0cda10 0x7fb0cc155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 0x7fb0cc148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc169980 con 0x7fb0cc15bc40 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc169800 con 0x7fb0cc0d0040 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc169b00 con 0x7fb0cc0cda10 2024-04-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 0x7fb0cc148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d1c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0cc0cda10 0x7fb0cc155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 0x7fb0cc148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44968/0 (socket says 172.21.15.45:44968) 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 -- 172.21.15.45:0/1160159817 learned_addr learned my addr 172.21.15.45:0/1160159817 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d1200640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 -- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0cc0cda10 msgr2=0x7fb0cc155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0cc0cda10 0x7fb0cc155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 -- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0cc0d0040 msgr2=0x7fb0cc154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 -- 172.21.15.45:0/1160159817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc1558e0 con 0x7fb0cc15bc40 2024-04-25T10:25:46.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d2600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 0x7fb0cc148e60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b0040 tx=0x7fb0c4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40b2050 con 0x7fb0cc15bc40 2024-04-25T10:25:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c40be070 con 0x7fb0cc15bc40 2024-04-25T10:25:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40ade40 con 0x7fb0cc15bc40 2024-04-25T10:25:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc154070 con 0x7fb0cc15bc40 2024-04-25T10:25:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.435+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc1493a0 con 0x7fb0cc15bc40 2024-04-25T10:25:46.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.439+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc04ef30 con 0x7fb0cc15bc40 2024-04-25T10:25:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.439+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0c40c3460 con 0x7fb0cc15bc40 2024-04-25T10:25:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.439+0000 7fb0c9600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0ac066f60 0x7fb0ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.439+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fb0c4120330 con 0x7fb0cc15bc40 2024-04-25T10:25:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.439+0000 7fb0d1c00640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0ac066f60 0x7fb0ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.443+0000 7fb0d1c00640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0ac066f60 0x7fb0ac069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0007fd0 tx=0x7fb0c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:46.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.443+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0c4016560 con 0x7fb0cc15bc40 2024-04-25T10:25:46.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.675+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb0cc1550e0 con 0x7fb0cc15bc40 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.675+0000 7fb0c9600640 1 -- 172.21.15.45:0/1160159817 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1607 (secure 0 0 0) 0x7fb0c40f0d40 con 0x7fb0cc15bc40 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:46.679 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1044,"data_bytes":1072181731,"bytes_used":2555744256,"bytes_avail":770538369024,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":67648877,"read_op_per_sec":0,"write_op_per_sec":67},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:25:04.973359+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0ac066f60 msgr2=0x7fb0ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0ac066f60 0x7fb0ac069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0007fd0 tx=0x7fb0c0009040 comp rx=0 tx=0).stop 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 msgr2=0x7fb0cc148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 0x7fb0cc148e60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b0040 tx=0x7fb0c4019680 comp rx=0 tx=0).stop 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 shutdown_connections 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0ac066f60 0x7fb0ac069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0cc15bc40 0x7fb0cc148e60 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0cc0d0040 0x7fb0cc154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 --2- 172.21.15.45:0/1160159817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0cc0cda10 0x7fb0cc155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 >> 172.21.15.45:0/1160159817 conn(0x7fb0cc066b70 msgr2=0x7fb0cc0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 shutdown_connections 2024-04-25T10:25:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:46.679+0000 7fb0d4600640 1 -- 172.21.15.45:0/1160159817 wait complete. 2024-04-25T10:25:46.700 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:25:47.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 Processor -- start 2024-04-25T10:25:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 -- start start 2024-04-25T10:25:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23bc15dce0 0x7f23bc0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc06eec0 0x7f23bc06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc15e0c0 con 0x7f23bc0ad850 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc0674d0 con 0x7f23bc15dce0 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23c0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0671f0 con 0x7f23bc06eec0 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23bc15dce0 0x7f23bc0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23ba800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23bc0ad850 0x7f23bc0adc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42058/0 (socket says 172.21.15.45:42058) 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23ba800640 1 -- 172.21.15.45:0/1845867996 learned_addr learned my addr 172.21.15.45:0/1845867996 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b1400640 1 --2- 172.21.15.45:0/1845867996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc06eec0 0x7f23bc06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9400640 1 -- 172.21.15.45:0/1845867996 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087022281 0 0) 0x7f23bc15e0c0 con 0x7f23bc0ad850 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9400640 1 -- 172.21.15.45:0/1845867996 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a0003530 con 0x7f23bc0ad850 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9e00640 1 -- 172.21.15.45:0/1845867996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc06eec0 msgr2=0x7f23bc06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9e00640 1 --2- 172.21.15.45:0/1845867996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc06eec0 0x7f23bc06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9e00640 1 -- 172.21.15.45:0/1845867996 >> v1:172.21.15.45:6789/0 conn(0x7f23bc0ad850 legacy=0x7f23bc0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.851+0000 7f23b9e00640 1 -- 172.21.15.45:0/1845867996 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc06d4b0 con 0x7f23bc15dce0 2024-04-25T10:25:47.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23b9e00640 1 --2- 172.21.15.45:0/1845867996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc15dce0 0x7f23bc0669a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f23a40097c0 tx=0x7f23a401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3169aea35ceadbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23b9400640 1 -- 172.21.15.45:0/1845867996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b0070 con 0x7f23bc15dce0 2024-04-25T10:25:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23b9400640 1 -- 172.21.15.45:0/1845867996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40ab600 con 0x7f23bc15dce0 2024-04-25T10:25:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23b9400640 1 -- 172.21.15.45:0/1845867996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40ab9d0 con 0x7f23bc15dce0 2024-04-25T10:25:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 -- 172.21.15.45:0/1845867996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc15dce0 msgr2=0x7f23bc0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1845867996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc15dce0 0x7f23bc0669a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f23a40097c0 tx=0x7f23a401c380 comp rx=0 tx=0).stop 2024-04-25T10:25:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 -- 172.21.15.45:0/1845867996 shutdown_connections 2024-04-25T10:25:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1845867996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc06eec0 0x7f23bc06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1845867996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc15dce0 0x7f23bc0669a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 -- 172.21.15.45:0/1845867996 >> 172.21.15.45:0/1845867996 conn(0x7f23bc0b2410 msgr2=0x7f23bc0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 -- 172.21.15.45:0/1845867996 shutdown_connections 2024-04-25T10:25:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.855+0000 7f23c0c00640 1 -- 172.21.15.45:0/1845867996 wait complete. 2024-04-25T10:25:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 Processor -- start 2024-04-25T10:25:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- start start 2024-04-25T10:25:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 0x7f23bc14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc0ad850 0x7f23bc0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23bc15dce0 0x7f23bc0b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0d0320 con 0x7f23bc06eec0 2024-04-25T10:25:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0d04a0 con 0x7f23bc15dce0 2024-04-25T10:25:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc16abc0 con 0x7f23bc0ad850 2024-04-25T10:25:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 0x7f23bc14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 0x7f23bc14bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44994/0 (socket says 172.21.15.45:44994) 2024-04-25T10:25:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 -- 172.21.15.45:0/1101660138 learned_addr learned my addr 172.21.15.45:0/1101660138 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23ba800640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23bc15dce0 0x7f23bc0b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b1400640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc0ad850 0x7f23bc0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 -- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc0ad850 msgr2=0x7f23bc0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc0ad850 0x7f23bc0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 -- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23bc15dce0 msgr2=0x7f23bc0b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23bc15dce0 0x7f23bc0b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 -- 172.21.15.45:0/1101660138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc15acc0 con 0x7f23bc06eec0 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b1400640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc0ad850 0x7f23bc0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b9e00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 0x7f23bc14bc50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f23a4019fb0 tx=0x7f23a4019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b0040 con 0x7f23bc06eec0 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40b9070 con 0x7f23bc06eec0 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b4650 con 0x7f23bc06eec0 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc15afa0 con 0x7f23bc06eec0 2024-04-25T10:25:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc162ac0 con 0x7f23bc06eec0 2024-04-25T10:25:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.859+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc04ef30 con 0x7f23bc06eec0 2024-04-25T10:25:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.863+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23a40b47f0 con 0x7f23bc06eec0 2024-04-25T10:25:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.863+0000 7f23b2a00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.863+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f23a411ee00 con 0x7f23bc06eec0 2024-04-25T10:25:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.863+0000 7f23b1400640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.863+0000 7f23b1400640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f23bc15f970 tx=0x7f23ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:47.867+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23a4016090 con 0x7f23bc06eec0 2024-04-25T10:25:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.067+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23bc160460 con 0x7f23bc06eec0 2024-04-25T10:25:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.067+0000 7f23b2a00640 1 -- 172.21.15.45:0/1101660138 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f23a40efd10 con 0x7f23bc06eec0 2024-04-25T10:25:48.070 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 msgr2=0x7f2398069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f23bc15f970 tx=0x7f23ac008040 comp rx=0 tx=0).stop 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 msgr2=0x7f23bc14bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 0x7f23bc14bc50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f23a4019fb0 tx=0x7f23a4019120 comp rx=0 tx=0).stop 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 shutdown_connections 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23bc15dce0 0x7f23bc0b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23bc0ad850 0x7f23bc0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 --2- 172.21.15.45:0/1101660138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23bc06eec0 0x7f23bc14bc50 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 >> 172.21.15.45:0/1101660138 conn(0x7f23bc0b2410 msgr2=0x7f23bc15e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 shutdown_connections 2024-04-25T10:25:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.071+0000 7f23c0c00640 1 -- 172.21.15.45:0/1101660138 wait complete. 2024-04-25T10:25:48.091 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:48.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 Processor -- start 2024-04-25T10:25:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 -- start start 2024-04-25T10:25:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f450c0d1820 0x7f450c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0b15b0 0x7f450c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0b1af0 con 0x7f450c15e050 2024-04-25T10:25:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c0b1dd0 con 0x7f450c0d1820 2024-04-25T10:25:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4513a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b20b0 con 0x7f450c0b15b0 2024-04-25T10:25:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0b15b0 0x7f450c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f450c0d1820 0x7f450c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0b15b0 0x7f450c0cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57042/0 (socket says 172.21.15.45:57042) 2024-04-25T10:25:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f450c0d1820 0x7f450c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37136/0 (socket says 172.21.15.45:37136) 2024-04-25T10:25:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511a00640 1 -- 172.21.15.45:0/3575257293 learned_addr learned my addr 172.21.15.45:0/3575257293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4503e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f450c15e050 0x7f450c0b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:37670/0 (socket says 172.21.15.45:37670) 2024-04-25T10:25:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511000640 1 -- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0b15b0 msgr2=0x7f450c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511000640 1 --2- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0b15b0 0x7f450c0cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511000640 1 -- 172.21.15.45:0/3575257293 >> v1:172.21.15.117:6789/0 conn(0x7f450c15e050 legacy=0x7f450c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f450be00640 1 -- 172.21.15.45:0/3575257293 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714042401 0 0) 0x7f450c0b1af0 con 0x7f450c15e050 2024-04-25T10:25:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.231+0000 7f4511000640 1 -- 172.21.15.45:0/3575257293 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0b2390 con 0x7f450c0d1820 2024-04-25T10:25:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4511000640 1 --2- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c0d1820 0x7f450c0d1c00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5cd478f18f8aa9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f450be00640 1 -- 172.21.15.45:0/3575257293 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0aa620 con 0x7f450c0d1820 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f450be00640 1 -- 172.21.15.45:0/3575257293 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc0af070 con 0x7f450c0d1820 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f450be00640 1 -- 172.21.15.45:0/3575257293 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0aa9e0 con 0x7f450c0d1820 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 -- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c0d1820 msgr2=0x7f450c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 --2- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c0d1820 0x7f450c0d1c00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 -- 172.21.15.45:0/3575257293 shutdown_connections 2024-04-25T10:25:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 --2- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0b15b0 0x7f450c0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 --2- 172.21.15.45:0/3575257293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c0d1820 0x7f450c0d1c00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 -- 172.21.15.45:0/3575257293 >> 172.21.15.45:0/3575257293 conn(0x7f450c09a9d0 msgr2=0x7f450c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 -- 172.21.15.45:0/3575257293 shutdown_connections 2024-04-25T10:25:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.235+0000 7f4513a00640 1 -- 172.21.15.45:0/3575257293 wait complete. 2024-04-25T10:25:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 Processor -- start 2024-04-25T10:25:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 -- start start 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 0x7f450c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0d1820 0x7f450c0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c15e050 0x7f450c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d0d90 con 0x7f450c0b15b0 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d0c10 con 0x7f450c15e050 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4513a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d0f10 con 0x7f450c0d1820 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 0x7f450c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 0x7f450c0b44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45006/0 (socket says 172.21.15.45:45006) 2024-04-25T10:25:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 -- 172.21.15.45:0/491629465 learned_addr learned my addr 172.21.15.45:0/491629465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4503e00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0d1820 0x7f450c0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c15e050 0x7f450c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 -- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0d1820 msgr2=0x7f450c0b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0d1820 0x7f450c0b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 -- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c15e050 msgr2=0x7f450c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c15e050 0x7f450c15cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 -- 172.21.15.45:0/491629465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0b6940 con 0x7f450c0b15b0 2024-04-25T10:25:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.239+0000 7f4511000640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 0x7f450c0b44d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0aabc0 con 0x7f450c0b15b0 2024-04-25T10:25:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc0aad60 con 0x7f450c0b15b0 2024-04-25T10:25:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004240 con 0x7f450c0b15b0 2024-04-25T10:25:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c0b4ad0 con 0x7f450c0b15b0 2024-04-25T10:25:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c0b5090 con 0x7f450c0b15b0 2024-04-25T10:25:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c04ef30 con 0x7f450c0b15b0 2024-04-25T10:25:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44fc003620 con 0x7f450c0b15b0 2024-04-25T10:25:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4509600640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e8066f90 0x7f44e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f44fc11f2f0 con 0x7f450c0b15b0 2024-04-25T10:25:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.243+0000 7f4503e00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e8066f90 0x7f44e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.247+0000 7f4503e00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e8066f90 0x7f44e8069450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f450c161680 tx=0x7f44f4007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.247+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44fc016090 con 0x7f450c0b15b0 2024-04-25T10:25:48.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f450c06d4d0 con 0x7f450c0b15b0 2024-04-25T10:25:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4509600640 1 -- 172.21.15.45:0/491629465 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1607 (secure 0 0 0) 0x7f44fc0efd60 con 0x7f450c0b15b0 2024-04-25T10:25:48.475 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:48.475 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1044,"data_bytes":1072181731,"bytes_used":2555744256,"bytes_avail":770538369024,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":32054743,"read_op_per_sec":0,"write_op_per_sec":32},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:25:04.973359+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e8066f90 msgr2=0x7f44e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e8066f90 0x7f44e8069450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f450c161680 tx=0x7f44f4007d60 comp rx=0 tx=0).stop 2024-04-25T10:25:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 msgr2=0x7f450c0b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 0x7f450c0b44d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc01ace0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 shutdown_connections 2024-04-25T10:25:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f44e8066f90 0x7f44e8069450 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f450c15e050 0x7f450c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f450c0d1820 0x7f450c0b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 --2- 172.21.15.45:0/491629465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f450c0b15b0 0x7f450c0b44d0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 >> 172.21.15.45:0/491629465 conn(0x7f450c09a9d0 msgr2=0x7f450c0d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 shutdown_connections 2024-04-25T10:25:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:48.471+0000 7f4513a00640 1 -- 172.21.15.45:0/491629465 wait complete. 2024-04-25T10:25:48.494 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:25:49.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:49.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 Processor -- start 2024-04-25T10:25:49.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 -- start start 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f154815bc40 0x7f154806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15480cda10 0x7f15480b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15480cf930 0x7f15480cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15480b2770 con 0x7f154815bc40 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154806c670 con 0x7f15480cda10 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f1550600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154806c950 con 0x7f15480cf930 2024-04-25T10:25:49.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f154815bc40 0x7f154806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f154815bc40 0x7f154806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45016/0 (socket says 172.21.15.45:45016) 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 -- 172.21.15.45:0/3066350812 learned_addr learned my addr 172.21.15.45:0/3066350812 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154e600640 1 --2- 172.21.15.45:0/3066350812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15480cf930 0x7f15480cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154d200640 1 --2- 172.21.15.45:0/3066350812 >> v2:172.21.15.117:3300/0 conn(0x7f15480cda10 0x7f15480b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 -- 172.21.15.45:0/3066350812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15480cf930 msgr2=0x7f15480cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 --2- 172.21.15.45:0/3066350812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15480cf930 0x7f15480cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 -- 172.21.15.45:0/3066350812 >> v2:172.21.15.117:3300/0 conn(0x7f15480cda10 msgr2=0x7f15480b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 --2- 172.21.15.45:0/3066350812 >> v2:172.21.15.117:3300/0 conn(0x7f15480cda10 0x7f15480b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.647+0000 7f154dc00640 1 -- 172.21.15.45:0/3066350812 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154806cc30 con 0x7f154815bc40 2024-04-25T10:25:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f154dc00640 1 --2- 172.21.15.45:0/3066350812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f154815bc40 0x7f154806c090 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f15480726f0 tx=0x7f153c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d1edb53355087f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1547e00640 1 -- 172.21.15.45:0/3066350812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0b0070 con 0x7f154815bc40 2024-04-25T10:25:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1547e00640 1 -- 172.21.15.45:0/3066350812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0ab600 con 0x7f154815bc40 2024-04-25T10:25:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1547e00640 1 -- 172.21.15.45:0/3066350812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0ab9d0 con 0x7f154815bc40 2024-04-25T10:25:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 -- 172.21.15.45:0/3066350812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f154815bc40 msgr2=0x7f154806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 --2- 172.21.15.45:0/3066350812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f154815bc40 0x7f154806c090 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f15480726f0 tx=0x7f153c01c520 comp rx=0 tx=0).stop 2024-04-25T10:25:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 -- 172.21.15.45:0/3066350812 shutdown_connections 2024-04-25T10:25:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 --2- 172.21.15.45:0/3066350812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15480cf930 0x7f15480cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 --2- 172.21.15.45:0/3066350812 >> v2:172.21.15.117:3300/0 conn(0x7f15480cda10 0x7f15480b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 --2- 172.21.15.45:0/3066350812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f154815bc40 0x7f154806c090 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 -- 172.21.15.45:0/3066350812 >> 172.21.15.45:0/3066350812 conn(0x7f1548066b70 msgr2=0x7f1548066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 -- 172.21.15.45:0/3066350812 shutdown_connections 2024-04-25T10:25:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.651+0000 7f1550600640 1 -- 172.21.15.45:0/3066350812 wait complete. 2024-04-25T10:25:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 Processor -- start 2024-04-25T10:25:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 -- start start 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 0x7f154813bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15480cf930 0x7f154814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f154815bc40 0x7f1548153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548160270 con 0x7f15480cda10 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481600f0 con 0x7f15480cf930 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15481603f0 con 0x7f154815bc40 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 0x7f154813bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 0x7f154813bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45032/0 (socket says 172.21.15.45:45032) 2024-04-25T10:25:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 -- 172.21.15.45:0/4289482185 learned_addr learned my addr 172.21.15.45:0/4289482185 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154d200640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15480cf930 0x7f154814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154e600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f154815bc40 0x7f1548153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 -- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f154815bc40 msgr2=0x7f1548153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f154815bc40 0x7f1548153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 -- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15480cf930 msgr2=0x7f154814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15480cf930 0x7f154814a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 -- 172.21.15.45:0/4289482185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1548156a80 con 0x7f15480cda10 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f154dc00640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 0x7f154813bfb0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f153c0ae040 tx=0x7f153c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0ba020 con 0x7f15480cda10 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0b6420 con 0x7f15480cda10 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c0b0040 con 0x7f15480cda10 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154814aac0 con 0x7f15480cda10 2024-04-25T10:25:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.655+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154814b080 con 0x7f15480cda10 2024-04-25T10:25:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.659+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154804ef30 con 0x7f15480cda10 2024-04-25T10:25:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.659+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f153c0bd020 con 0x7f15480cda10 2024-04-25T10:25:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.659+0000 7f1545600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1528066f60 0x7f1528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.659+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f153c11f200 con 0x7f15480cda10 2024-04-25T10:25:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.659+0000 7f154d200640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1528066f60 0x7f1528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.659+0000 7f154d200640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1528066f60 0x7f1528069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f154815f4c0 tx=0x7f1534008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:49.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.663+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f153c0c4030 con 0x7f15480cda10 2024-04-25T10:25:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.855+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f154806d960 con 0x7f15480cda10 2024-04-25T10:25:49.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.855+0000 7f1545600640 1 -- 172.21.15.45:0/4289482185 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1548156a80 con 0x7f15480cda10 2024-04-25T10:25:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:49.859 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:49.859 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1528066f60 msgr2=0x7f1528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1528066f60 0x7f1528069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f154815f4c0 tx=0x7f1534008040 comp rx=0 tx=0).stop 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 msgr2=0x7f154813bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 0x7f154813bfb0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f153c0ae040 tx=0x7f153c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 shutdown_connections 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1528066f60 0x7f1528069420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f154815bc40 0x7f1548153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15480cf930 0x7f154814a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 --2- 172.21.15.45:0/4289482185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15480cda10 0x7f154813bfb0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 >> 172.21.15.45:0/4289482185 conn(0x7f1548066b70 msgr2=0x7f15480b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 shutdown_connections 2024-04-25T10:25:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:49.859+0000 7f1550600640 1 -- 172.21.15.45:0/4289482185 wait complete. 2024-04-25T10:25:49.879 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:49.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f064ac00640 1 Processor -- start 2024-04-25T10:25:50.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f064ac00640 1 -- start start 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f064ac00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f064415f180 0x7f06440ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f064ac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06440ae610 con 0x7f06440d0630 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f064ac00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06440d27b0 con 0x7f06440ce830 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f064ac00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440cec60 con 0x7f064415f180 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f0643e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f064415f180 0x7f06440ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f063b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06440d0630 0x7f06440aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42062/0 (socket says 172.21.15.45:42062) 2024-04-25T10:25:50.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f063b400640 1 -- 172.21.15.45:0/2810470253 learned_addr learned my addr 172.21.15.45:0/2810470253 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f0643e00640 1 -- 172.21.15.45:0/2810470253 >> v1:172.21.15.117:6789/0 conn(0x7f06440ce830 legacy=0x7f06440af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f0643e00640 1 -- 172.21.15.45:0/2810470253 >> v1:172.21.15.45:6789/0 conn(0x7f06440d0630 legacy=0x7f06440aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.019+0000 7f0643e00640 1 -- 172.21.15.45:0/2810470253 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440cf570 con 0x7f064415f180 2024-04-25T10:25:50.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f0643e00640 1 --2- 172.21.15.45:0/2810470253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064415f180 0x7f06440ce2f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f06340097f0 tx=0x7f063401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3637d0183b9f6b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f0643400640 1 -- 172.21.15.45:0/2810470253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340b1070 con 0x7f064415f180 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f0643400640 1 -- 172.21.15.45:0/2810470253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06340ac600 con 0x7f064415f180 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f0643400640 1 -- 172.21.15.45:0/2810470253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340ac9d0 con 0x7f064415f180 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 -- 172.21.15.45:0/2810470253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064415f180 msgr2=0x7f06440ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 --2- 172.21.15.45:0/2810470253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064415f180 0x7f06440ce2f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f06340097f0 tx=0x7f063401c3a0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 -- 172.21.15.45:0/2810470253 shutdown_connections 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 --2- 172.21.15.45:0/2810470253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064415f180 0x7f06440ce2f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 -- 172.21.15.45:0/2810470253 >> 172.21.15.45:0/2810470253 conn(0x7f06440d1900 msgr2=0x7f06440d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 -- 172.21.15.45:0/2810470253 shutdown_connections 2024-04-25T10:25:50.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.023+0000 7f064ac00640 1 -- 172.21.15.45:0/2810470253 wait complete. 2024-04-25T10:25:50.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 Processor -- start 2024-04-25T10:25:50.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- start start 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06440ce830 0x7f06441f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 0x7f0644167110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064415f180 0x7f064415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440b0000 con 0x7f06440d0630 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064415f560 con 0x7f06440ce830 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440d23f0 con 0x7f064415f180 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 0x7f0644167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 0x7f0644167110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45056/0 (socket says 172.21.15.45:45056) 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 -- 172.21.15.45:0/1558993689 learned_addr learned my addr 172.21.15.45:0/1558993689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f0643e00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06440ce830 0x7f06441f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f0648c00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064415f180 0x7f064415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 -- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064415f180 msgr2=0x7f064415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064415f180 0x7f064415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 -- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06440ce830 msgr2=0x7f06441f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06440ce830 0x7f06441f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 -- 172.21.15.45:0/1558993689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0644167650 con 0x7f06440d0630 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f063b400640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 0x7f0644167110 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f062c00ca70 tx=0x7f062c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c017070 con 0x7f06440d0630 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c00abd0 con 0x7f06440d0630 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c012400 con 0x7f06440d0630 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06441f7e60 con 0x7f06440d0630 2024-04-25T10:25:50.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441f8420 con 0x7f06440d0630 2024-04-25T10:25:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.027+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064404ef30 con 0x7f06440d0630 2024-04-25T10:25:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.031+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f062c01a070 con 0x7f06440d0630 2024-04-25T10:25:50.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.031+0000 7f0640c00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0620066f60 0x7f0620069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.031+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f062c084cf0 con 0x7f06440d0630 2024-04-25T10:25:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.031+0000 7f0643e00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0620066f60 0x7f0620069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.031+0000 7f0643e00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0620066f60 0x7f0620069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f06340ae000 tx=0x7f06340a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.035+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f062c055640 con 0x7f06440d0630 2024-04-25T10:25:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f064415e7b0 con 0x7f06440d0630 2024-04-25T10:25:50.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f0640c00640 1 -- 172.21.15.45:0/1558993689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1792 (secure 0 0 0) 0x7f062c0596e0 con 0x7f06440d0630 2024-04-25T10:25:50.256 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:50.257 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076585911,"bytes_used":2578268160,"bytes_avail":770515845120,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":32606653,"read_op_per_sec":0,"write_op_per_sec":33},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T10:25:48.982836+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0620066f60 msgr2=0x7f0620069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0620066f60 0x7f0620069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f06340ae000 tx=0x7f06340a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:50.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 msgr2=0x7f0644167110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 0x7f0644167110 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f062c00ca70 tx=0x7f062c00cf30 comp rx=0 tx=0).stop 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 shutdown_connections 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0620066f60 0x7f0620069420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064415f180 0x7f064415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06440d0630 0x7f0644167110 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 --2- 172.21.15.45:0/1558993689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06440ce830 0x7f06441f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 >> 172.21.15.45:0/1558993689 conn(0x7f06440d1900 msgr2=0x7f064415da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 shutdown_connections 2024-04-25T10:25:50.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.255+0000 7f064ac00640 1 -- 172.21.15.45:0/1558993689 wait complete. 2024-04-25T10:25:50.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:25:50.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:50.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T10:25:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 Processor -- start 2024-04-25T10:25:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 -- start start 2024-04-25T10:25:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c0aef80 0x7f440c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440c070350 0x7f440c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f440c15bac0 0x7f440c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c15b080 con 0x7f440c070350 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c15adb0 con 0x7f440c15bac0 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4413800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15eb60 con 0x7f440c0aef80 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440c070350 0x7f440c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440c070350 0x7f440c15b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45068/0 (socket says 172.21.15.45:45068) 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 -- 172.21.15.45:0/1075499033 learned_addr learned my addr 172.21.15.45:0/1075499033 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4410e00640 1 --2- 172.21.15.45:0/1075499033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c0aef80 0x7f440c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4411800640 1 --2- 172.21.15.45:0/1075499033 >> v2:172.21.15.117:3300/0 conn(0x7f440c15bac0 0x7f440c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 -- 172.21.15.45:0/1075499033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c0aef80 msgr2=0x7f440c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 --2- 172.21.15.45:0/1075499033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c0aef80 0x7f440c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 -- 172.21.15.45:0/1075499033 >> v2:172.21.15.117:3300/0 conn(0x7f440c15bac0 msgr2=0x7f440c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 --2- 172.21.15.45:0/1075499033 >> v2:172.21.15.117:3300/0 conn(0x7f440c15bac0 0x7f440c0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.419+0000 7f4403e00640 1 -- 172.21.15.45:0/1075499033 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c15ee40 con 0x7f440c070350 2024-04-25T10:25:50.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4403e00640 1 --2- 172.21.15.45:0/1075499033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c15b580 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cc330 tx=0x7f43f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485f074a0c344a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f440be00640 1 -- 172.21.15.45:0/1075499033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40ab5f0 con 0x7f440c070350 2024-04-25T10:25:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f440be00640 1 -- 172.21.15.45:0/1075499033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f40b0070 con 0x7f440c070350 2024-04-25T10:25:50.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f440be00640 1 -- 172.21.15.45:0/1075499033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40ab9b0 con 0x7f440c070350 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 -- 172.21.15.45:0/1075499033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 msgr2=0x7f440c15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 --2- 172.21.15.45:0/1075499033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c15b580 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cc330 tx=0x7f43f401c520 comp rx=0 tx=0).stop 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 -- 172.21.15.45:0/1075499033 shutdown_connections 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 --2- 172.21.15.45:0/1075499033 >> v2:172.21.15.117:3300/0 conn(0x7f440c15bac0 0x7f440c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 --2- 172.21.15.45:0/1075499033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c15b580 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 --2- 172.21.15.45:0/1075499033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c0aef80 0x7f440c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 -- 172.21.15.45:0/1075499033 >> 172.21.15.45:0/1075499033 conn(0x7f440c0669a0 msgr2=0x7f440c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 -- 172.21.15.45:0/1075499033 shutdown_connections 2024-04-25T10:25:50.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.423+0000 7f4413800640 1 -- 172.21.15.45:0/1075499033 wait complete. 2024-04-25T10:25:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 Processor -- start 2024-04-25T10:25:50.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 -- start start 2024-04-25T10:25:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f440c0aef80 0x7f440c0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c15bac0 0x7f440c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c06c460 con 0x7f440c070350 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c06c2e0 con 0x7f440c0aef80 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c06c5e0 con 0x7f440c15bac0 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45080/0 (socket says 172.21.15.45:45080) 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 -- 172.21.15.45:0/2841544030 learned_addr learned my addr 172.21.15.45:0/2841544030 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4403e00640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f440c0aef80 0x7f440c0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4411800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c15bac0 0x7f440c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 -- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c15bac0 msgr2=0x7f440c153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c15bac0 0x7f440c153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 -- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f440c0aef80 msgr2=0x7f440c0b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f440c0aef80 0x7f440c0b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 -- 172.21.15.45:0/2841544030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c153f20 con 0x7f440c070350 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4410e00640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c154fc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00d830 tx=0x7f43fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4409600640 1 -- 172.21.15.45:0/2841544030 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc018020 con 0x7f440c070350 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4409600640 1 -- 172.21.15.45:0/2841544030 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc0096a0 con 0x7f440c070350 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c154200 con 0x7f440c070350 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4409600640 1 -- 172.21.15.45:0/2841544030 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc009a50 con 0x7f440c070350 2024-04-25T10:25:50.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.427+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c154aa0 con 0x7f440c070350 2024-04-25T10:25:50.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4409600640 1 -- 172.21.15.45:0/2841544030 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43fc01b020 con 0x7f440c070350 2024-04-25T10:25:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4409600640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43e8066f60 0x7f43e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4409600640 1 -- 172.21.15.45:0/2841544030 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f43fc0857a0 con 0x7f440c070350 2024-04-25T10:25:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 Processor -- start 2024-04-25T10:25:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 -- start start 2024-04-25T10:25:50.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4403e00640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43e8066f60 0x7f43e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4403e00640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43e8066f60 0x7f43e8069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f43f40af000 tx=0x7f43f40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e2abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ac2f0 0x56434e261b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2620a0 0x56434e262480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434e0aeff0 con 0x56434e2ab9d0 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434e037150 con 0x56434e2620a0 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56434e0ab090 con 0x56434e2ac2f0 2024-04-25T10:25:50.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4410e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e2abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4410e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e2abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45082/0 (socket says 172.21.15.45:45082) 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4410e00640 1 -- 172.21.15.45:0/261889084 learned_addr learned my addr 172.21.15.45:0/261889084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.431+0000 7f4403e00640 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ac2f0 0x56434e261b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4411800640 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2620a0 0x56434e262480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4410e00640 1 -- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ac2f0 msgr2=0x56434e261b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4410e00640 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ac2f0 0x56434e261b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4410e00640 1 -- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2620a0 msgr2=0x56434e262480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4410e00640 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2620a0 0x56434e262480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4410e00640 1 -- 172.21.15.45:0/261889084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56434e2450d0 con 0x56434e2ab9d0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4410e00640 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e2abdb0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f43fc004780 tx=0x7f43fc056760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ab41ea443d6ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4401600640 1 -- 172.21.15.45:0/261889084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc082070 con 0x56434e2ab9d0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4401600640 1 -- 172.21.15.45:0/261889084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc05e6c0 con 0x56434e2ab9d0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4401600640 1 -- 172.21.15.45:0/261889084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc06d030 con 0x56434e2ab9d0 2024-04-25T10:25:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- 172.21.15.45:0/261889084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 msgr2=0x56434e2abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e2abdb0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f43fc004780 tx=0x7f43fc056760 comp rx=0 tx=0).stop 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- 172.21.15.45:0/261889084 shutdown_connections 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2620a0 0x56434e262480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ac2f0 0x56434e261b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- 172.21.15.45:0/261889084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e2abdb0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- 172.21.15.45:0/261889084 >> 172.21.15.45:0/261889084 conn(0x56434e25e020 msgr2=0x56434e260440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- 172.21.15.45:0/261889084 shutdown_connections 2024-04-25T10:25:50.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- 172.21.15.45:0/261889084 wait complete. 2024-04-25T10:25:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- start start 2024-04-25T10:25:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 Processor -- start 2024-04-25T10:25:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e2907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e290d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e291270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434e073250 con 0x56434e2ab9d0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434e069690 con 0x56434e2ac2f0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56434e037a40 con 0x56434e2620a0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4416d36000 10 client.0 _ll_register_callbacks cb 0x56434e24ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4403e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e290d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4403e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e290d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45098/0 (socket says 172.21.15.45:45098) 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.435+0000 7f4403e00640 1 -- 172.21.15.45:0/3007548073 learned_addr learned my addr 172.21.15.45:0/3007548073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4410e00640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e2907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4411800640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e291270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4403e00640 1 -- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 msgr2=0x56434e2907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4403e00640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e2907f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4403e00640 1 -- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 msgr2=0x56434e291270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4403e00640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e291270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4403e00640 1 -- 172.21.15.45:0/3007548073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56434e15c1c0 con 0x56434e2ab9d0 2024-04-25T10:25:50.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4403e00640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e290d30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f43f40ad040 tx=0x7f43f401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898334e764021a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 1 -- 172.21.15.45:0/3007548073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40cd060 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 1 -- 172.21.15.45:0/3007548073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f43f40bf610 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43fc05e6c0 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 1 -- 172.21.15.45:0/3007548073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f40d3050 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 10 client.8130 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x56434e158170 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 1 -- 172.21.15.45:0/3007548073 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc05e6c0 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 1 -- 172.21.15.45:0/3007548073 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f43f40d6020 con 0x56434e2ab9d0 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 10 client.8130 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 10 client.8130 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 20 client.8130 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bbc80 0x56434e2be070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 4 client.8130 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56434e04ac60 con 0x56434e2bbc80 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4410e00640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bbc80 0x56434e2be070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4410e00640 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bbc80 0x56434e2be070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6fcf2bb35579a20 server_cookie=1520b4ad57e959fa in_seq=0 out_seq=0 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 10 client.8130 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 1 -- 172.21.15.45:0/3007548073 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56434e04ac60 con 0x56434e2bbc80 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 10 client.8130 handle_command_reply: tid=0 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 client.8130 shutdown 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 msgr2=0x56434e290d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e290d30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f43f40ad040 tx=0x7f43f401ad60 comp rx=0 tx=0).stop 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 shutdown_connections 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bbc80 0x56434e2be070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e291270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e290d30 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 --2- 172.21.15.45:0/3007548073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e2907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f43fa000640 0 client.8130 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 >> 172.21.15.45:0/3007548073 conn(0x56434e25e020 msgr2=0x56434e260ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 shutdown_connections 2024-04-25T10:25:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 1 -- 172.21.15.45:0/3007548073 wait complete. 2024-04-25T10:25:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.439+0000 7f4416d36000 20 client.8130 trim_cache size 0 max 16384 2024-04-25T10:25:50.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 Processor -- start 2024-04-25T10:25:50.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 -- start start 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 0x56434e2aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ab9d0 0x56434e261420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e261960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c203680 con 0x56434e2620a0 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c2029b0 con 0x56434e2ac2f0 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1fa860 con 0x56434e2ab9d0 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4410e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 0x56434e2aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4410e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 0x56434e2aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45114/0 (socket says 172.21.15.45:45114) 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4410e00640 1 -- 172.21.15.45:0/807590086 learned_addr learned my addr 172.21.15.45:0/807590086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4411800640 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e261960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.455+0000 7f4403e00640 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ab9d0 0x56434e261420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 -- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ab9d0 msgr2=0x56434e261420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ab9d0 0x56434e261420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 -- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 msgr2=0x56434e261960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e261960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 -- 172.21.15.45:0/807590086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56434e0f5de0 con 0x56434e2620a0 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 0x56434e2aae10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0560e0 tx=0x7f43fc058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6c472349b6dc51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4400c00640 1 -- 172.21.15.45:0/807590086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc082040 con 0x56434e2620a0 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4400c00640 1 -- 172.21.15.45:0/807590086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc052590 con 0x56434e2620a0 2024-04-25T10:25:50.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4400c00640 1 -- 172.21.15.45:0/807590086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc06d040 con 0x56434e2620a0 2024-04-25T10:25:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- 172.21.15.45:0/807590086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 msgr2=0x56434e2aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 0x56434e2aae10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0560e0 tx=0x7f43fc058000 comp rx=0 tx=0).stop 2024-04-25T10:25:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- 172.21.15.45:0/807590086 shutdown_connections 2024-04-25T10:25:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e261960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2ab9d0 0x56434e261420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- 172.21.15.45:0/807590086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2620a0 0x56434e2aae10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- 172.21.15.45:0/807590086 >> 172.21.15.45:0/807590086 conn(0x56434e25e020 msgr2=0x56434e2aa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- 172.21.15.45:0/807590086 shutdown_connections 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- 172.21.15.45:0/807590086 wait complete. 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- start start 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 Processor -- start 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e30c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e30c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e2be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56434e049e00 con 0x56434e2ab9d0 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1fae60 con 0x56434e2ac2f0 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56434e0ae300 con 0x56434e2620a0 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4416d36000 10 client.0 _ll_register_callbacks cb 0x56434e22d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e30c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e30c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45118/0 (socket says 172.21.15.45:45118) 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 -- 172.21.15.45:0/3762399479 learned_addr learned my addr 172.21.15.45:0/3762399479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4410e00640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e30c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4411800640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e2be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 -- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 msgr2=0x56434e30c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e30c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 -- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 msgr2=0x56434e2be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e2be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.459+0000 7f4403e00640 1 -- 172.21.15.45:0/3762399479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56434e0508e0 con 0x56434e2ab9d0 2024-04-25T10:25:50.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4403e00640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e30c850 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f43f401ca00 tx=0x7f43f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494174442b6bb965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 1 -- 172.21.15.45:0/3762399479 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40ba070 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 1 -- 172.21.15.45:0/3762399479 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f43f40bf610 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43fc052590 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 1 -- 172.21.15.45:0/3762399479 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43f40e9050 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 10 client.8136 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x56434e1088f0 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 1 -- 172.21.15.45:0/3762399479 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc052590 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 1 -- 172.21.15.45:0/3762399479 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f43f40ec020 con 0x56434e2ab9d0 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 10 client.8136 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 10 client.8136 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 20 client.8136 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bef20 0x56434e2bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 4 client.8136 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x56434e0bc530 con 0x56434e2bef20 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4410e00640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bef20 0x56434e2bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4410e00640 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bef20 0x56434e2bf330 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19672c79a395a4fa server_cookie=94356a1eacc0abe2 in_seq=0 out_seq=0 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 10 client.8136 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 1 -- 172.21.15.45:0/3762399479 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56434e0bc530 con 0x56434e2bef20 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 10 client.8136 handle_command_reply: tid=0 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 client.8136 shutdown 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 msgr2=0x56434e30c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e30c850 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f43f401ca00 tx=0x7f43f4008040 comp rx=0 tx=0).stop 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 shutdown_connections 2024-04-25T10:25:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x56434e2bef20 0x56434e2bf330 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56434e2ac2f0 0x56434e2be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56434e2ab9d0 0x56434e30c850 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 --2- 172.21.15.45:0/3762399479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56434e2620a0 0x56434e30c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f43fa000640 0 client.8136 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 >> 172.21.15.45:0/3762399479 conn(0x56434e25e020 msgr2=0x56434e2aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 shutdown_connections 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 1 -- 172.21.15.45:0/3762399479 wait complete. 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4416d36000 20 client.8136 trim_cache size 0 max 16384 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:25:50.471 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:50.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43e8066f60 msgr2=0x7f43e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43e8066f60 0x7f43e8069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f43f40af000 tx=0x7f43f40a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 msgr2=0x7f440c154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c154fc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00d830 tx=0x7f43fc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 shutdown_connections 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43e8066f60 0x7f43e8069420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f440c15bac0 0x7f440c153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f440c0aef80 0x7f440c0b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 --2- 172.21.15.45:0/2841544030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440c070350 0x7f440c154fc0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 >> 172.21.15.45:0/2841544030 conn(0x7f440c0669a0 msgr2=0x7f440c15cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 shutdown_connections 2024-04-25T10:25:50.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.463+0000 7f4413800640 1 -- 172.21.15.45:0/2841544030 wait complete. 2024-04-25T10:25:50.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:50.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:25:50.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 Processor -- start 2024-04-25T10:25:50.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 -- start start 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9e58070350 0x7f9e5815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e580b31c0 0x7f9e580b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5815b080 con 0x7f9e580aef80 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5815adb0 con 0x7f9e580b31c0 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5fa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5815eb60 con 0x7f9e58070350 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e580b31c0 0x7f9e580b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e580b31c0 0x7f9e580b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45122/0 (socket says 172.21.15.45:45122) 2024-04-25T10:25:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e5da00640 1 -- 172.21.15.45:0/2790997894 learned_addr learned my addr 172.21.15.45:0/2790997894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.627+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/2790997894 >> v2:172.21.15.117:3300/0 conn(0x7f9e58070350 0x7f9e5815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5da00640 1 -- 172.21.15.45:0/2790997894 >> v2:172.21.15.117:3300/0 conn(0x7f9e58070350 msgr2=0x7f9e5815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5da00640 1 --2- 172.21.15.45:0/2790997894 >> v2:172.21.15.117:3300/0 conn(0x7f9e58070350 0x7f9e5815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5da00640 1 -- 172.21.15.45:0/2790997894 >> v1:172.21.15.117:6789/0 conn(0x7f9e580aef80 legacy=0x7f9e5815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5da00640 1 -- 172.21.15.45:0/2790997894 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5815ee40 con 0x7f9e580b31c0 2024-04-25T10:25:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5da00640 1 --2- 172.21.15.45:0/2790997894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580b31c0 0x7f9e580b3610 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9e5806e390 tx=0x7f9e5001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d048e3f301baa361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e57e00640 1 -- 172.21.15.45:0/2790997894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a57d0 con 0x7f9e580b31c0 2024-04-25T10:25:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e57e00640 1 -- 172.21.15.45:0/2790997894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500a5970 con 0x7f9e580b31c0 2024-04-25T10:25:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e57e00640 1 -- 172.21.15.45:0/2790997894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a5d40 con 0x7f9e580b31c0 2024-04-25T10:25:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 -- 172.21.15.45:0/2790997894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580b31c0 msgr2=0x7f9e580b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/2790997894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580b31c0 0x7f9e580b3610 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9e5806e390 tx=0x7f9e5001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 -- 172.21.15.45:0/2790997894 shutdown_connections 2024-04-25T10:25:50.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/2790997894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580b31c0 0x7f9e580b3610 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/2790997894 >> v2:172.21.15.117:3300/0 conn(0x7f9e58070350 0x7f9e5815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 -- 172.21.15.45:0/2790997894 >> 172.21.15.45:0/2790997894 conn(0x7f9e580669a0 msgr2=0x7f9e58066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 -- 172.21.15.45:0/2790997894 shutdown_connections 2024-04-25T10:25:50.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.631+0000 7f9e5fa00640 1 -- 172.21.15.45:0/2790997894 wait complete. 2024-04-25T10:25:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 Processor -- start 2024-04-25T10:25:50.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 -- start start 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58070350 0x7f9e580b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 0x7f9e58155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580b31c0 0x7f9e58162b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581679a0 con 0x7f9e580aef80 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58167820 con 0x7f9e580b31c0 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58167b20 con 0x7f9e58070350 2024-04-25T10:25:50.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 0x7f9e58155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 0x7f9e58155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45132/0 (socket says 172.21.15.45:45132) 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 -- 172.21.15.45:0/1400786282 learned_addr learned my addr 172.21.15.45:0/1400786282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5da00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580b31c0 0x7f9e58162b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5d000640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58070350 0x7f9e580b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 -- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58070350 msgr2=0x7f9e580b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58070350 0x7f9e580b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 -- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580b31c0 msgr2=0x7f9e58162b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580b31c0 0x7f9e58162b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 -- 172.21.15.45:0/1400786282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e58160710 con 0x7f9e580aef80 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5d000640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58070350 0x7f9e580b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 0x7f9e58155b50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e930 tx=0x7f9e4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e55600640 1 -- 172.21.15.45:0/1400786282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40017070 con 0x7f9e580aef80 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e55600640 1 -- 172.21.15.45:0/1400786282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000ca00 con 0x7f9e580aef80 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e55600640 1 -- 172.21.15.45:0/1400786282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000cdb0 con 0x7f9e580aef80 2024-04-25T10:25:50.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e58160a50 con 0x7f9e580aef80 2024-04-25T10:25:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.635+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580b1cf0 con 0x7f9e580aef80 2024-04-25T10:25:50.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e55600640 1 -- 172.21.15.45:0/1400786282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e4001a070 con 0x7f9e580aef80 2024-04-25T10:25:50.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e55600640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e34066f60 0x7f9e34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e55600640 1 -- 172.21.15.45:0/1400786282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f9e40085080 con 0x7f9e580aef80 2024-04-25T10:25:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e62eb6000 1 Processor -- start 2024-04-25T10:25:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e5d000640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e34066f60 0x7f9e34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e62eb6000 1 -- start start 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.639+0000 7f9e5d000640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e34066f60 0x7f9e34069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9e5806d720 tx=0x7f9e48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad8bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad41b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad42480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574bab8eff0 con 0x5574bad8b9d0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574bab17150 con 0x5574bad8c2f0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574bab8b090 con 0x5574bad420a0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad8bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad8bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45136/0 (socket says 172.21.15.45:45136) 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 -- 172.21.15.45:0/277771109 learned_addr learned my addr 172.21.15.45:0/277771109 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5da00640 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad42480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad41b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 -- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 msgr2=0x5574bad42480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad42480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 -- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 msgr2=0x5574bad41b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad41b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 -- 172.21.15.45:0/277771109 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574bad250d0 con 0x5574bad8b9d0 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad41b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e5d000640 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad8bdb0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9e480050d0 tx=0x7f9e48011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21dac4cebc29d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e4d600640 1 -- 172.21.15.45:0/277771109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003600 con 0x5574bad8b9d0 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e4d600640 1 -- 172.21.15.45:0/277771109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480037a0 con 0x5574bad8b9d0 2024-04-25T10:25:50.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e4d600640 1 -- 172.21.15.45:0/277771109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003b50 con 0x5574bad8b9d0 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- 172.21.15.45:0/277771109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 msgr2=0x5574bad8bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad8bdb0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9e480050d0 tx=0x7f9e48011000 comp rx=0 tx=0).stop 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- 172.21.15.45:0/277771109 shutdown_connections 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad42480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad41b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/277771109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad8bdb0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- 172.21.15.45:0/277771109 >> 172.21.15.45:0/277771109 conn(0x5574bad3e020 msgr2=0x5574bad40440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- 172.21.15.45:0/277771109 shutdown_connections 2024-04-25T10:25:50.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- 172.21.15.45:0/277771109 wait complete. 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 -- start start 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 Processor -- start 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.643+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad70d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574bab53250 con 0x5574bad8c2f0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574bab49690 con 0x5574bad420a0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574bab17a40 con 0x5574bad8b9d0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 10 client.0 _ll_register_callbacks cb 0x5574bad2ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad70d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37254/0 (socket says 172.21.15.45:37254) 2024-04-25T10:25:50.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad70d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57142/0 (socket says 172.21.15.45:57142) 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad71270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45142/0 (socket says 172.21.15.45:45142) 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5d000640 1 -- 172.21.15.45:0/101301457 learned_addr learned my addr 172.21.15.45:0/101301457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 -- 172.21.15.45:0/101301457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 msgr2=0x5574bad70d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad70d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 -- 172.21.15.45:0/101301457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 msgr2=0x5574bad707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 -- 172.21.15.45:0/101301457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574bac3c1c0 con 0x5574bad8c2f0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5da00640 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad71270 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9e500ab000 tx=0x7f9e50004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f18fabd459986a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 1 -- 172.21.15.45:0/101301457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a59c0 con 0x5574bad8c2f0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 1 -- 172.21.15.45:0/101301457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f9e500a5b60 con 0x5574bad8c2f0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e480037a0 con 0x5574bad8c2f0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 1 -- 172.21.15.45:0/101301457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e500a5d60 con 0x5574bad8c2f0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 10 client.8148 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5574bac38170 con 0x5574bad8c2f0 2024-04-25T10:25:50.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 1 -- 172.21.15.45:0/101301457 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480037a0 con 0x5574bad8c2f0 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 1 -- 172.21.15.45:0/101301457 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e580cfe70 con 0x5574bad8c2f0 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 10 client.8148 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 10 client.8148 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 20 client.8148 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574bad9bc80 0x5574bad9e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 4 client.8148 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574bab2ac60 con 0x5574bad9bc80 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5d000640 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574bad9bc80 0x5574bad9e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e5d000640 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574bad9bc80 0x5574bad9e070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bda324434ce00b6 server_cookie=12e08aec5bff82e1 in_seq=0 out_seq=0 2024-04-25T10:25:50.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 10 client.8148 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 1 -- 172.21.15.45:0/101301457 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574bab2ac60 con 0x5574bad9bc80 2024-04-25T10:25:50.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 10 client.8148 handle_command_reply: tid=0 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 client.8148 shutdown 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 msgr2=0x5574bad71270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad71270 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9e500ab000 tx=0x7f9e50004a50 comp rx=0 tx=0).stop 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 shutdown_connections 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad71270 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad70d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574bad9bc80 0x5574bad9e070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/101301457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 >> 172.21.15.45:0/101301457 conn(0x5574bad3e020 msgr2=0x5574bad40ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e46000640 0 client.8148 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 shutdown_connections 2024-04-25T10:25:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.647+0000 7f9e62eb6000 1 -- 172.21.15.45:0/101301457 wait complete. 2024-04-25T10:25:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.651+0000 7f9e62eb6000 20 client.8148 trim_cache size 0 max 16384 2024-04-25T10:25:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 Processor -- start 2024-04-25T10:25:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 -- start start 2024-04-25T10:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad40410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad40950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad40e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e582031e0 con 0x5574bad8c2f0 2024-04-25T10:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58202420 con 0x5574bad420a0 2024-04-25T10:25:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.663+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580b0290 con 0x5574bad8b9d0 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad40e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad40e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45156/0 (socket says 172.21.15.45:45156) 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 -- 172.21.15.45:0/3171482867 learned_addr learned my addr 172.21.15.45:0/3171482867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5d000640 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad40410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad40950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 -- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 msgr2=0x5574bad40950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad40950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 -- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 msgr2=0x5574bad40410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad40410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 -- 172.21.15.45:0/3171482867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574babd5de0 con 0x5574bad8c2f0 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5d000640 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:50.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e5da00640 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad40e90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9e5001a620 tx=0x7f9e500a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07394d3469453e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e4cc00640 1 -- 172.21.15.45:0/3171482867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50009af0 con 0x5574bad8c2f0 2024-04-25T10:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e4cc00640 1 -- 172.21.15.45:0/3171482867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500d0070 con 0x5574bad8c2f0 2024-04-25T10:25:50.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e4cc00640 1 -- 172.21.15.45:0/3171482867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a5a10 con 0x5574bad8c2f0 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3171482867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 msgr2=0x5574bad40e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad40e90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9e5001a620 tx=0x7f9e500a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3171482867 shutdown_connections 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8c2f0 0x5574bad40e90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad8b9d0 0x5574bad40950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3171482867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad420a0 0x5574bad40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3171482867 >> 172.21.15.45:0/3171482867 conn(0x5574bad3e020 msgr2=0x5574bad8a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3171482867 shutdown_connections 2024-04-25T10:25:50.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3171482867 wait complete. 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- start start 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 Processor -- start 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad9ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad9f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad9f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574bab29e00 con 0x5574bad8b9d0 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580b0890 con 0x5574bad8c2f0 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574bab8e300 con 0x5574bad420a0 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e62eb6000 10 client.0 _ll_register_callbacks cb 0x5574bad0d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.667+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad9f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad9f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45170/0 (socket says 172.21.15.45:45170) 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 -- 172.21.15.45:0/3292396138 learned_addr learned my addr 172.21.15.45:0/3292396138 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5da00640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad9f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5d000640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad9ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 -- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 msgr2=0x5574bad9ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad9ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 -- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 msgr2=0x5574bad9f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad9f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 -- 172.21.15.45:0/3292396138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574bab308e0 con 0x5574bad8b9d0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e4fe00640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad9f110 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c520 tx=0x7f9e4005e1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b6c800b7301862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 1 -- 172.21.15.45:0/3292396138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40072060 con 0x5574bad8b9d0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 1 -- 172.21.15.45:0/3292396138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f9e400551b0 con 0x5574bad8b9d0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e500d0070 con 0x5574bad8b9d0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 1 -- 172.21.15.45:0/3292396138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e40078050 con 0x5574bad8b9d0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 10 client.8154 fetch_fsmap learned FSMap version 73 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5574babe88f0 con 0x5574bad8b9d0 2024-04-25T10:25:50.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 1 -- 172.21.15.45:0/3292396138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500d0070 con 0x5574bad8b9d0 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 1 -- 172.21.15.45:0/3292396138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e4007b020 con 0x5574bad8b9d0 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 10 client.8154 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 10 client.8154 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 20 client.8154 populate_metadata read hostname 'smithi045' 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574badf1f50 0x5574badf4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 4 client.8154 mds_command: new command op to 7851 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 --> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574bab9c530 con 0x5574badf1f50 2024-04-25T10:25:50.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5d000640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574badf1f50 0x5574badf4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5d000640 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574badf1f50 0x5574badf4340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c118abd7711407b4 server_cookie=5b5b7d9839b8bbe4 in_seq=0 out_seq=0 2024-04-25T10:25:50.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 10 client.8154 ms_handle_connect on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 1 -- 172.21.15.45:0/3292396138 <== mds.0 v2:172.21.15.45:6833/4218207272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5574bab9c530 con 0x5574badf1f50 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 10 client.8154 handle_command_reply: tid=0 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 client.8154 shutdown 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 msgr2=0x5574bad9f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad9f110 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c520 tx=0x7f9e4005e1b0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 shutdown_connections 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:6833/4218207272,v1:172.21.15.45:6835/4218207272] conn(0x5574badf1f50 0x5574badf4340 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574bad8c2f0 0x5574bad9f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5574bad8b9d0 0x5574bad9f110 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e46000640 0 client.8154 ms_handle_reset on v2:172.21.15.45:6833/4218207272 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 --2- 172.21.15.45:0/3292396138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5574bad420a0 0x5574bad9ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 >> 172.21.15.45:0/3292396138 conn(0x5574bad3e020 msgr2=0x5574bad8a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 shutdown_connections 2024-04-25T10:25:50.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 1 -- 172.21.15.45:0/3292396138 wait complete. 2024-04-25T10:25:50.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e62eb6000 20 client.8154 trim_cache size 0 max 16384 2024-04-25T10:25:50.679 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:25:50.679 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:25:50.679 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:25:50.679 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e34066f60 msgr2=0x7f9e34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e34066f60 0x7f9e34069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9e5806d720 tx=0x7f9e48009040 comp rx=0 tx=0).stop 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 msgr2=0x7f9e58155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 0x7f9e58155b50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e930 tx=0x7f9e4000edf0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 shutdown_connections 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9e34066f60 0x7f9e34069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e580b31c0 0x7f9e58162b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e580aef80 0x7f9e58155b50 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 --2- 172.21.15.45:0/1400786282 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e58070350 0x7f9e580b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 >> 172.21.15.45:0/1400786282 conn(0x7f9e580669a0 msgr2=0x7f9e580cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 shutdown_connections 2024-04-25T10:25:50.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.671+0000 7f9e5fa00640 1 -- 172.21.15.45:0/1400786282 wait complete. 2024-04-25T10:25:50.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:50.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2095600640 1 Processor -- start 2024-04-25T10:25:50.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2095600640 1 -- start start 2024-04-25T10:25:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2095600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f20900cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2095600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20900ccaa0 con 0x7f20900d0180 2024-04-25T10:25:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2095600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20900ccd80 con 0x7f209006aea0 2024-04-25T10:25:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2095600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209015e340 con 0x7f209015df60 2024-04-25T10:25:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2087e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f20900d0180 0x7f20900d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42074/0 (socket says 172.21.15.45:42074) 2024-04-25T10:25:50.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f2087e00640 1 -- 172.21.15.45:0/1206081423 learned_addr learned my addr 172.21.15.45:0/1206081423 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f208ea00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f20900cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.831+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663964829 0 0) 0x7f20900ccaa0 con 0x7f20900d0180 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2074003530 con 0x7f20900d0180 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065576470 0 0) 0x7f2074003530 con 0x7f20900d0180 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 msgr2=0x7f20900cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f20900cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 >> v1:172.21.15.117:6789/0 conn(0x7f209006aea0 legacy=0x7f209006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900cd060 con 0x7f20900d0180 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f2078001680 con 0x7f20900d0180 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 >> v1:172.21.15.45:6789/0 conn(0x7f20900d0180 legacy=0x7f20900d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20740039f0 0x7f2074003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f2074004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074004d50 0x7f20740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207400ba80 con 0x7f20740039f0 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207400bc00 con 0x7f2074004d50 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207400be00 con 0x7f209015df60 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208ea00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20740039f0 0x7f2074003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208f400640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074004d50 0x7f20740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f2087e00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f2074004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208ea00640 1 -- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 msgr2=0x7f2074004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208ea00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f2074004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208ea00640 1 -- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074004d50 msgr2=0x7f20740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208ea00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074004d50 0x7f20740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208ea00640 1 -- 172.21.15.45:0/1206081423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074003530 con 0x7f20740039f0 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f2087e00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015df60 0x7f2074004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.835+0000 7f208f400640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2074004d50 0x7f20740051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208ea00640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20740039f0 0x7f2074003dd0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2080009ae0 tx=0x7f208001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f852ff8840e1407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800096b0 con 0x7f20740039f0 2024-04-25T10:25:50.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20800a6450 con 0x7f20740039f0 2024-04-25T10:25:50.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208e000640 1 -- 172.21.15.45:0/1206081423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800a6840 con 0x7f20740039f0 2024-04-25T10:25:50.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- 172.21.15.45:0/1206081423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20740039f0 msgr2=0x7f2074003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 --2- 172.21.15.45:0/1206081423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20740039f0 0x7f2074003dd0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2080009ae0 tx=0x7f208001ca70 comp rx=0 tx=0).stop 2024-04-25T10:25:50.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208f400640 1 -- 172.21.15.45:0/1206081423 reap_dead start 2024-04-25T10:25:50.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- 172.21.15.45:0/1206081423 shutdown_connections 2024-04-25T10:25:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- 172.21.15.45:0/1206081423 >> 172.21.15.45:0/1206081423 conn(0x7f20900d1910 msgr2=0x7f20900d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- 172.21.15.45:0/1206081423 shutdown_connections 2024-04-25T10:25:50.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- 172.21.15.45:0/1206081423 wait complete. 2024-04-25T10:25:50.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 Processor -- start 2024-04-25T10:25:50.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- start start 2024-04-25T10:25:50.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20740039f0 0x7f2090155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 0x7f209015e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015ed40 0x7f2090157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20900d0ff0 con 0x7f2090155c70 2024-04-25T10:25:50.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20900d0e70 con 0x7f20740039f0 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2095600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2090169a20 con 0x7f209015ed40 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2087e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 0x7f209015e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2087e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 0x7f209015e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45196/0 (socket says 172.21.15.45:45196) 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f2087e00640 1 -- 172.21.15.45:0/1949272235 learned_addr learned my addr 172.21.15.45:0/1949272235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208ea00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20740039f0 0x7f2090155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.839+0000 7f208f400640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015ed40 0x7f2090157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2087e00640 1 -- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015ed40 msgr2=0x7f2090157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2087e00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015ed40 0x7f2090157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2087e00640 1 -- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20740039f0 msgr2=0x7f2090155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2087e00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20740039f0 0x7f2090155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2087e00640 1 -- 172.21.15.45:0/1949272235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209015b640 con 0x7f2090155c70 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f208ea00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20740039f0 0x7f2090155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2087e00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 0x7f209015e800 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f207800e8e0 tx=0x7f207800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078019070 con 0x7f2090155c70 2024-04-25T10:25:50.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207800c850 con 0x7f2090155c70 2024-04-25T10:25:50.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207800cc40 con 0x7f2090155c70 2024-04-25T10:25:50.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2090156b40 con 0x7f2090155c70 2024-04-25T10:25:50.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2090155960 con 0x7f2090155c70 2024-04-25T10:25:50.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209004ef30 con 0x7f2090155c70 2024-04-25T10:25:50.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f207801c070 con 0x7f2090155c70 2024-04-25T10:25:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2086a00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f206c066f60 0x7f206c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.843+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f2078084f70 con 0x7f2090155c70 2024-04-25T10:25:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.847+0000 7f208ea00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f206c066f60 0x7f206c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.847+0000 7f208ea00640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f206c066f60 0x7f206c069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20800ad000 tx=0x7f20800b2040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:50.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:50.847+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20780558b0 con 0x7f2090155c70 2024-04-25T10:25:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f209015f120 con 0x7f2090155c70 2024-04-25T10:25:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2086a00640 1 -- 172.21.15.45:0/1949272235 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2078059950 con 0x7f2090155c70 2024-04-25T10:25:51.048 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:51.048 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:51.048 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f206c066f60 msgr2=0x7f206c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f206c066f60 0x7f206c069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20800ad000 tx=0x7f20800b2040 comp rx=0 tx=0).stop 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 msgr2=0x7f209015e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 0x7f209015e800 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f207800e8e0 tx=0x7f207800eda0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 shutdown_connections 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f206c066f60 0x7f206c069420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f209015ed40 0x7f2090157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2090155c70 0x7f209015e800 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 --2- 172.21.15.45:0/1949272235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20740039f0 0x7f2090155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 >> 172.21.15.45:0/1949272235 conn(0x7f20900d1910 msgr2=0x7f209006b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 shutdown_connections 2024-04-25T10:25:51.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.047+0000 7f2095600640 1 -- 172.21.15.45:0/1949272235 wait complete. 2024-04-25T10:25:51.069 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:51.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:51.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f106b800640 1 Processor -- start 2024-04-25T10:25:51.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f106b800640 1 -- start start 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f106b800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10640670b0 0x7f106415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f106b800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106415e400 con 0x7f10640b38d0 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f106b800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640b0000 con 0x7f106415d9b0 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f106b800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10640b02e0 con 0x7f10640670b0 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f1069800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10640670b0 0x7f106415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f105be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f10640b38d0 0x7f10640669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42076/0 (socket says 172.21.15.45:42076) 2024-04-25T10:25:51.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f105be00640 1 -- 172.21.15.45:0/3439402221 learned_addr learned my addr 172.21.15.45:0/3439402221 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f1063e00640 1 -- 172.21.15.45:0/3439402221 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577797809 0 0) 0x7f106415e400 con 0x7f10640b38d0 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f1069800640 1 -- 172.21.15.45:0/3439402221 >> v1:172.21.15.117:6789/0 conn(0x7f106415d9b0 legacy=0x7f10640af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f1069800640 1 -- 172.21.15.45:0/3439402221 >> v1:172.21.15.45:6789/0 conn(0x7f10640b38d0 legacy=0x7f10640669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.207+0000 7f1069800640 1 -- 172.21.15.45:0/3439402221 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106415e6a0 con 0x7f10640670b0 2024-04-25T10:25:51.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f1069800640 1 --2- 172.21.15.45:0/3439402221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640670b0 0x7f106415dd90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f105c009d90 tx=0x7f105c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38533fb961ad1532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f1063e00640 1 -- 172.21.15.45:0/3439402221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0b2070 con 0x7f10640670b0 2024-04-25T10:25:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f1063e00640 1 -- 172.21.15.45:0/3439402221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c0a5940 con 0x7f10640670b0 2024-04-25T10:25:51.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f1063e00640 1 -- 172.21.15.45:0/3439402221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0a5d10 con 0x7f10640670b0 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 -- 172.21.15.45:0/3439402221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640670b0 msgr2=0x7f106415dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 --2- 172.21.15.45:0/3439402221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640670b0 0x7f106415dd90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f105c009d90 tx=0x7f105c01c970 comp rx=0 tx=0).stop 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 -- 172.21.15.45:0/3439402221 shutdown_connections 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 --2- 172.21.15.45:0/3439402221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640670b0 0x7f106415dd90 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 -- 172.21.15.45:0/3439402221 >> 172.21.15.45:0/3439402221 conn(0x7f106406ac90 msgr2=0x7f106406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 -- 172.21.15.45:0/3439402221 shutdown_connections 2024-04-25T10:25:51.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.211+0000 7f106b800640 1 -- 172.21.15.45:0/3439402221 wait complete. 2024-04-25T10:25:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 Processor -- start 2024-04-25T10:25:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- start start 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10640670b0 0x7f106414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 0x7f10641fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106415d9b0 0x7f10641fab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106416bb30 con 0x7f10640b38d0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106416b9b0 con 0x7f106415d9b0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106416bcb0 con 0x7f10640670b0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 0x7f10641fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 0x7f10641fa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45228/0 (socket says 172.21.15.45:45228) 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 -- 172.21.15.45:0/1768207860 learned_addr learned my addr 172.21.15.45:0/1768207860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1068e00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10640670b0 0x7f106414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1069800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106415d9b0 0x7f10641fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 -- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10640670b0 msgr2=0x7f106414bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10640670b0 0x7f106414bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 -- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106415d9b0 msgr2=0x7f10641fab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106415d9b0 0x7f10641fab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 -- 172.21.15.45:0/1768207860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064160e80 con 0x7f10640b38d0 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1069800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106415d9b0 0x7f10641fab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:51.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1068e00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10640670b0 0x7f106414bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f105be00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 0x7f10641fa5e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c480 tx=0x7f104c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c005cf0 con 0x7f10640b38d0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c007070 con 0x7f10640b38d0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0074e0 con 0x7f10640b38d0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1064161160 con 0x7f10640b38d0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1064154610 con 0x7f10640b38d0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.215+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106404ef30 con 0x7f10640b38d0 2024-04-25T10:25:51.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.219+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f104c01b400 con 0x7f10640b38d0 2024-04-25T10:25:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.219+0000 7f1061600640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1040066f90 0x7f1040069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.219+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f104c084d50 con 0x7f10640b38d0 2024-04-25T10:25:51.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.219+0000 7f1068e00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1040066f90 0x7f1040069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:51.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.219+0000 7f1068e00640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1040066f90 0x7f1040069450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1054000f70 tx=0x7f1054009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:51.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.223+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f104c055cf0 con 0x7f10640b38d0 2024-04-25T10:25:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.439+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f10640af7e0 con 0x7f10640b38d0 2024-04-25T10:25:51.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.439+0000 7f1061600640 1 -- 172.21.15.45:0/1768207860 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1792 (secure 0 0 0) 0x7f104c0555f0 con 0x7f10640b38d0 2024-04-25T10:25:51.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:51.443 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":495,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076585911,"bytes_used":2578268160,"bytes_avail":770515845120,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":32606653,"read_op_per_sec":0,"write_op_per_sec":33},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T10:25:48.982836+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1040066f90 msgr2=0x7f1040069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1040066f90 0x7f1040069450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1054000f70 tx=0x7f1054009040 comp rx=0 tx=0).stop 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 msgr2=0x7f10641fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 0x7f10641fa5e0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c480 tx=0x7f104c00c940 comp rx=0 tx=0).stop 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 shutdown_connections 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1040066f90 0x7f1040069450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106415d9b0 0x7f10641fab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10640b38d0 0x7f10641fa5e0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 --2- 172.21.15.45:0/1768207860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10640670b0 0x7f106414bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:51.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 >> 172.21.15.45:0/1768207860 conn(0x7f106406ac90 msgr2=0x7f10640ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:51.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 shutdown_connections 2024-04-25T10:25:51.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:51.443+0000 7f106b800640 1 -- 172.21.15.45:0/1768207860 wait complete. 2024-04-25T10:25:51.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:25:52.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:52.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.615+0000 7f06a4e00640 1 Processor -- start 2024-04-25T10:25:52.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.615+0000 7f06a4e00640 1 -- start start 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f06a4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a0066c00 0x7f06a015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f06a4e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f06a00b5560 0x7f06a0157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f06a4e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a00b5970 con 0x7f06a0066fe0 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f06a4e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00b5c20 con 0x7f06a0066c00 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f06a4e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a00af6e0 con 0x7f06a00b5560 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a0066c00 0x7f06a015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06a0066c00 0x7f06a015e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45230/0 (socket says 172.21.15.45:45230) 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 -- 172.21.15.45:0/1925838898 learned_addr learned my addr 172.21.15.45:0/1925838898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:52.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f0695400640 1 --1- 172.21.15.45:0/1925838898 >> v1:172.21.15.45:6789/0 conn(0x7f06a0066fe0 0x7f06a006ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42084/0 (socket says 172.21.15.45:42084) 2024-04-25T10:25:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069e800640 1 --2- 172.21.15.45:0/1925838898 >> v2:172.21.15.117:3300/0 conn(0x7f06a00b5560 0x7f06a0157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 -- 172.21.15.45:0/1925838898 >> v2:172.21.15.117:3300/0 conn(0x7f06a00b5560 msgr2=0x7f06a0157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 --2- 172.21.15.45:0/1925838898 >> v2:172.21.15.117:3300/0 conn(0x7f06a00b5560 0x7f06a0157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 -- 172.21.15.45:0/1925838898 >> v1:172.21.15.45:6789/0 conn(0x7f06a0066fe0 legacy=0x7f06a006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.619+0000 7f069de00640 1 -- 172.21.15.45:0/1925838898 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00af9c0 con 0x7f06a0066c00 2024-04-25T10:25:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f069de00640 1 --2- 172.21.15.45:0/1925838898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a015e110 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b3360 tx=0x7f069001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cfb3c24b09fbfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f069d400640 1 -- 172.21.15.45:0/1925838898 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900aa650 con 0x7f06a0066c00 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f069d400640 1 -- 172.21.15.45:0/1925838898 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06900af070 con 0x7f06a0066c00 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f069d400640 1 -- 172.21.15.45:0/1925838898 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900aaa10 con 0x7f06a0066c00 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 -- 172.21.15.45:0/1925838898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 msgr2=0x7f06a015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1925838898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a015e110 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b3360 tx=0x7f069001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 -- 172.21.15.45:0/1925838898 shutdown_connections 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1925838898 >> v2:172.21.15.117:3300/0 conn(0x7f06a00b5560 0x7f06a0157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1925838898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a015e110 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 -- 172.21.15.45:0/1925838898 >> 172.21.15.45:0/1925838898 conn(0x7f06a00b24f0 msgr2=0x7f06a00b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 -- 172.21.15.45:0/1925838898 shutdown_connections 2024-04-25T10:25:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 -- 172.21.15.45:0/1925838898 wait complete. 2024-04-25T10:25:52.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 Processor -- start 2024-04-25T10:25:52.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.623+0000 7f06a4e00640 1 -- start start 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a0066fe0 0x7f06a015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a00b5560 0x7f06a0146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00af1e0 con 0x7f06a0066c00 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00af060 con 0x7f06a00b5560 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a006ac90 con 0x7f06a0066fe0 2024-04-25T10:25:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a014ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45242/0 (socket says 172.21.15.45:45242) 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 -- 172.21.15.45:0/1090507429 learned_addr learned my addr 172.21.15.45:0/1090507429 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f0695400640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a0066fe0 0x7f06a015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069e800640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a00b5560 0x7f06a0146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 -- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a0066fe0 msgr2=0x7f06a015c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a0066fe0 0x7f06a015c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 -- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a00b5560 msgr2=0x7f06a0146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a00b5560 0x7f06a0146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 -- 172.21.15.45:0/1090507429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a015cba0 con 0x7f06a0066c00 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f069de00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a014ebe0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f069001a120 tx=0x7f069001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900bd5f0 con 0x7f06a0066c00 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06900a6040 con 0x7f06a0066c00 2024-04-25T10:25:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900bd9b0 con 0x7f06a0066c00 2024-04-25T10:25:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a015ce50 con 0x7f06a0066c00 2024-04-25T10:25:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a015d410 con 0x7f06a0066c00 2024-04-25T10:25:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a015d650 con 0x7f06a0066c00 2024-04-25T10:25:52.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.627+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06900b33f0 con 0x7f06a0066c00 2024-04-25T10:25:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.631+0000 7f0696a00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f067c066f60 0x7f067c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.631+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f069011f100 con 0x7f06a0066c00 2024-04-25T10:25:52.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.631+0000 7f0695400640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f067c066f60 0x7f067c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.631+0000 7f0695400640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f067c066f60 0x7f067c069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f06a015bca0 tx=0x7f0688007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:52.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.635+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06a015d650 con 0x7f06a0066c00 2024-04-25T10:25:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a015e5d0 con 0x7f06a0066c00 2024-04-25T10:25:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f0696a00640 1 -- 172.21.15.45:0/1090507429 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0690016110 con 0x7f06a0066c00 2024-04-25T10:25:52.833 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:52.833 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f067c066f60 msgr2=0x7f067c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f067c066f60 0x7f067c069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f06a015bca0 tx=0x7f0688007cd0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 msgr2=0x7f06a014ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a014ebe0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f069001a120 tx=0x7f069001ad30 comp rx=0 tx=0).stop 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 shutdown_connections 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f067c066f60 0x7f067c069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06a00b5560 0x7f06a0146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06a0066fe0 0x7f06a015c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 --2- 172.21.15.45:0/1090507429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06a0066c00 0x7f06a014ebe0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 >> 172.21.15.45:0/1090507429 conn(0x7f06a00b24f0 msgr2=0x7f06a006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 shutdown_connections 2024-04-25T10:25:52.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.831+0000 7f06a4e00640 1 -- 172.21.15.45:0/1090507429 wait complete. 2024-04-25T10:25:52.855 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:52.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:52.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 Processor -- start 2024-04-25T10:25:52.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 -- start start 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7900ccb20 0x7fb7900d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb79015bdc0 0x7fb7900d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7900ccfb0 con 0x7fb79015b9e0 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7900cd290 con 0x7fb79015bdc0 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb795800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900cd570 con 0x7fb7900ccb20 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb79015bdc0 0x7fb7900d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb79015bdc0 0x7fb7900d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45256/0 (socket says 172.21.15.45:45256) 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 -- 172.21.15.45:0/2299236803 learned_addr learned my addr 172.21.15.45:0/2299236803 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:52.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb787e00640 1 --2- 172.21.15.45:0/2299236803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7900ccb20 0x7fb7900d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 -- 172.21.15.45:0/2299236803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7900ccb20 msgr2=0x7fb7900d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 --2- 172.21.15.45:0/2299236803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7900ccb20 0x7fb7900d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 -- 172.21.15.45:0/2299236803 >> v1:172.21.15.117:6789/0 conn(0x7fb79015b9e0 legacy=0x7fb7900cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:52.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.995+0000 7fb78f400640 1 -- 172.21.15.45:0/2299236803 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7900cd850 con 0x7fb79015bdc0 2024-04-25T10:25:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb78f400640 1 --2- 172.21.15.45:0/2299236803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015bdc0 0x7fb7900d1a60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb788009e50 tx=0x7fb78801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66d5d1312405865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb78e000640 1 -- 172.21.15.45:0/2299236803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880a5700 con 0x7fb79015bdc0 2024-04-25T10:25:53.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb78e000640 1 -- 172.21.15.45:0/2299236803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7880a58a0 con 0x7fb79015bdc0 2024-04-25T10:25:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb78e000640 1 -- 172.21.15.45:0/2299236803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880a5c70 con 0x7fb79015bdc0 2024-04-25T10:25:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 -- 172.21.15.45:0/2299236803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015bdc0 msgr2=0x7fb7900d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 --2- 172.21.15.45:0/2299236803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015bdc0 0x7fb7900d1a60 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb788009e50 tx=0x7fb78801ca30 comp rx=0 tx=0).stop 2024-04-25T10:25:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 -- 172.21.15.45:0/2299236803 shutdown_connections 2024-04-25T10:25:53.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 --2- 172.21.15.45:0/2299236803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015bdc0 0x7fb7900d1a60 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 --2- 172.21.15.45:0/2299236803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7900ccb20 0x7fb7900d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 -- 172.21.15.45:0/2299236803 >> 172.21.15.45:0/2299236803 conn(0x7fb79009a9d0 msgr2=0x7fb79009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 -- 172.21.15.45:0/2299236803 shutdown_connections 2024-04-25T10:25:53.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 -- 172.21.15.45:0/2299236803 wait complete. 2024-04-25T10:25:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 Processor -- start 2024-04-25T10:25:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:52.999+0000 7fb795800640 1 -- start start 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 0x7fb7901f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb79015bdc0 0x7fb7901532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb790153800 0x7fb79014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900c65b0 con 0x7fb79015b9e0 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900c6430 con 0x7fb79015bdc0 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7900c6730 con 0x7fb790153800 2024-04-25T10:25:53.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 0x7fb7901f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 0x7fb7901f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45262/0 (socket says 172.21.15.45:45262) 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 -- 172.21.15.45:0/2815505375 learned_addr learned my addr 172.21.15.45:0/2815505375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb787e00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb79015bdc0 0x7fb7901532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78f400640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb790153800 0x7fb79014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 -- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb790153800 msgr2=0x7fb79014c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb790153800 0x7fb79014c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 -- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb79015bdc0 msgr2=0x7fb7901532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb79015bdc0 0x7fb7901532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 -- 172.21.15.45:0/2815505375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79014cd90 con 0x7fb79015b9e0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb78ea00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 0x7fb7901f7a70 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c450 tx=0x7fb78000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780016020 con 0x7fb79015b9e0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7800052d0 con 0x7fb79015b9e0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780007260 con 0x7fb79015b9e0 2024-04-25T10:25:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79014d070 con 0x7fb79015b9e0 2024-04-25T10:25:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79014c6c0 con 0x7fb79015b9e0 2024-04-25T10:25:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79004ef30 con 0x7fb79015b9e0 2024-04-25T10:25:53.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.003+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb780007400 con 0x7fb79015b9e0 2024-04-25T10:25:53.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.007+0000 7fb786a00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb76c066f60 0x7fb76c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:53.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.007+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fb780084980 con 0x7fb79015b9e0 2024-04-25T10:25:53.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.007+0000 7fb787e00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb76c066f60 0x7fb76c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.007+0000 7fb787e00640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb76c066f60 0x7fb76c069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb7900b2bc0 tx=0x7fb778009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:53.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.011+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb780020020 con 0x7fb79015b9e0 2024-04-25T10:25:53.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb790153be0 con 0x7fb79015b9e0 2024-04-25T10:25:53.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb786a00640 1 -- 172.21.15.45:0/2815505375 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1792 (secure 0 0 0) 0x7fb780005480 con 0x7fb79015b9e0 2024-04-25T10:25:53.237 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:53.237 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076585911,"bytes_used":2578268160,"bytes_avail":770515845120,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":31356072,"read_op_per_sec":0,"write_op_per_sec":32},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T10:25:48.982836+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:53.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb76c066f60 msgr2=0x7fb76c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb76c066f60 0x7fb76c069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb7900b2bc0 tx=0x7fb778009190 comp rx=0 tx=0).stop 2024-04-25T10:25:53.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 msgr2=0x7fb7901f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 0x7fb7901f7a70 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c450 tx=0x7fb78000c910 comp rx=0 tx=0).stop 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 shutdown_connections 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb76c066f60 0x7fb76c069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb790153800 0x7fb79014c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb79015bdc0 0x7fb7901532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 --2- 172.21.15.45:0/2815505375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb79015b9e0 0x7fb7901f7a70 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 >> 172.21.15.45:0/2815505375 conn(0x7fb79009a9d0 msgr2=0x7fb7900ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 shutdown_connections 2024-04-25T10:25:53.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:53.235+0000 7fb795800640 1 -- 172.21.15.45:0/2815505375 wait complete. 2024-04-25T10:25:53.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:25:54.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 Processor -- start 2024-04-25T10:25:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 -- start start 2024-04-25T10:25:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9e8070c20 0x7ff9e815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff9e80ae020 0x7ff9e80ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e80ad840 con 0x7ff9e80cf930 2024-04-25T10:25:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e815bb90 con 0x7ff9e8070c20 2024-04-25T10:25:54.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9f0000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e815e9f0 con 0x7ff9e80ae020 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ecc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9e8070c20 0x7ff9e815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ed600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff9e80cf930 0x7ff9e80cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42088/0 (socket says 172.21.15.45:42088) 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ed600640 1 -- 172.21.15.45:0/1511804073 learned_addr learned my addr 172.21.15.45:0/1511804073 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ecc00640 1 -- 172.21.15.45:0/1511804073 >> v2:172.21.15.117:3300/0 conn(0x7ff9e80ae020 msgr2=0x7ff9e80ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ee000640 1 --2- 172.21.15.45:0/1511804073 >> v2:172.21.15.117:3300/0 conn(0x7ff9e80ae020 0x7ff9e80ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ecc00640 1 --2- 172.21.15.45:0/1511804073 >> v2:172.21.15.117:3300/0 conn(0x7ff9e80ae020 0x7ff9e80ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ecc00640 1 -- 172.21.15.45:0/1511804073 >> v1:172.21.15.45:6789/0 conn(0x7ff9e80cf930 legacy=0x7ff9e80cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ecc00640 1 -- 172.21.15.45:0/1511804073 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e815ecd0 con 0x7ff9e8070c20 2024-04-25T10:25:54.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.411+0000 7ff9ee000640 1 --2- 172.21.15.45:0/1511804073 >> v2:172.21.15.117:3300/0 conn(0x7ff9e80ae020 0x7ff9e80ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:54.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9ecc00640 1 --2- 172.21.15.45:0/1511804073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815e170 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b2820 tx=0x7ff9d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b95cdcde19fae8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9e7e00640 1 -- 172.21.15.45:0/1511804073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40ab650 con 0x7ff9e8070c20 2024-04-25T10:25:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9e7e00640 1 -- 172.21.15.45:0/1511804073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d40b0070 con 0x7ff9e8070c20 2024-04-25T10:25:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9e7e00640 1 -- 172.21.15.45:0/1511804073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40aba10 con 0x7ff9e8070c20 2024-04-25T10:25:54.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 -- 172.21.15.45:0/1511804073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 msgr2=0x7ff9e815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 --2- 172.21.15.45:0/1511804073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815e170 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b2820 tx=0x7ff9d401c450 comp rx=0 tx=0).stop 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 -- 172.21.15.45:0/1511804073 shutdown_connections 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 --2- 172.21.15.45:0/1511804073 >> v2:172.21.15.117:3300/0 conn(0x7ff9e80ae020 0x7ff9e80ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 --2- 172.21.15.45:0/1511804073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815e170 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 -- 172.21.15.45:0/1511804073 >> 172.21.15.45:0/1511804073 conn(0x7ff9e8066b70 msgr2=0x7ff9e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 -- 172.21.15.45:0/1511804073 shutdown_connections 2024-04-25T10:25:54.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.415+0000 7ff9f0000640 1 -- 172.21.15.45:0/1511804073 wait complete. 2024-04-25T10:25:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 Processor -- start 2024-04-25T10:25:54.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- start start 2024-04-25T10:25:54.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9e80ae020 0x7ff9e815df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e80cf930 0x7ff9e8152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e816a8e0 con 0x7ff9e8070c20 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e815e6b0 con 0x7ff9e80cf930 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e816aa60 con 0x7ff9e80ae020 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45276/0 (socket says 172.21.15.45:45276) 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 -- 172.21.15.45:0/2879450669 learned_addr learned my addr 172.21.15.45:0/2879450669 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 -- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9e80ae020 msgr2=0x7ff9e815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9e80ae020 0x7ff9e815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 -- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e80cf930 msgr2=0x7ff9e8152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ee000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e80cf930 0x7ff9e8152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e80cf930 0x7ff9e8152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 -- 172.21.15.45:0/2879450669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e815bf60 con 0x7ff9e8070c20 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ee000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e80cf930 0x7ff9e8152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9ed600640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815d9f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00c420 tx=0x7ff9dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc017070 con 0x7ff9e8070c20 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc005cf0 con 0x7ff9e8070c20 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0072f0 con 0x7ff9e8070c20 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e815c240 con 0x7ff9e8070c20 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e80cc110 con 0x7ff9e8070c20 2024-04-25T10:25:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.419+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e804ef30 con 0x7ff9e8070c20 2024-04-25T10:25:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.423+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9dc01a070 con 0x7ff9e8070c20 2024-04-25T10:25:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.423+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9c8066f60 0x7ff9c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.423+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7ff9dc084460 con 0x7ff9e8070c20 2024-04-25T10:25:54.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.423+0000 7ff9ecc00640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9c8066f60 0x7ff9c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.423+0000 7ff9ecc00640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9c8066f60 0x7ff9c8069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40af000 tx=0x7ff9d4004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.427+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9dc055400 con 0x7ff9e8070c20 2024-04-25T10:25:54.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.623+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9e80cbd40 con 0x7ff9e8070c20 2024-04-25T10:25:54.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.623+0000 7ff9e5600640 1 -- 172.21.15.45:0/2879450669 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff9dc054d00 con 0x7ff9e8070c20 2024-04-25T10:25:54.626 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:54.626 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:54.627 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:54.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9c8066f60 msgr2=0x7ff9c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9c8066f60 0x7ff9c8069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40af000 tx=0x7ff9d4004740 comp rx=0 tx=0).stop 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 msgr2=0x7ff9e815d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815d9f0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00c420 tx=0x7ff9dc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 shutdown_connections 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9c8066f60 0x7ff9c8069420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e80cf930 0x7ff9e8152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9e80ae020 0x7ff9e815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 --2- 172.21.15.45:0/2879450669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e8070c20 0x7ff9e815d9f0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 >> 172.21.15.45:0/2879450669 conn(0x7ff9e8066b70 msgr2=0x7ff9e80b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 shutdown_connections 2024-04-25T10:25:54.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.627+0000 7ff9f0000640 1 -- 172.21.15.45:0/2879450669 wait complete. 2024-04-25T10:25:54.648 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:25:54.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:25:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.783+0000 7fdc94600640 1 Processor -- start 2024-04-25T10:25:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc94600640 1 -- start start 2024-04-25T10:25:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc94600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdc8c0d1820 0x7fdc8c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc94600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0b1af0 con 0x7fdc8c0b15b0 2024-04-25T10:25:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc94600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0b1dd0 con 0x7fdc8c15e050 2024-04-25T10:25:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc94600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b20b0 con 0x7fdc8c0d1820 2024-04-25T10:25:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc92600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdc8c0b15b0 0x7fdc8c0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42096/0 (socket says 172.21.15.45:42096) 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc92600640 1 -- 172.21.15.45:0/3513693574 learned_addr learned my addr 172.21.15.45:0/3513693574 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91c00640 1 --2- 172.21.15.45:0/3513693574 >> v2:172.21.15.117:3300/0 conn(0x7fdc8c0d1820 0x7fdc8c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264974767 0 0) 0x7fdc8c0b1af0 con 0x7fdc8c0b15b0 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc74003530 con 0x7fdc8c0b15b0 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954351973 0 0) 0x7fdc8c0b1dd0 con 0x7fdc8c15e050 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc8c0b1af0 con 0x7fdc8c15e050 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195059704 0 0) 0x7fdc74003530 con 0x7fdc8c0b15b0 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 >> v2:172.21.15.117:3300/0 conn(0x7fdc8c0d1820 msgr2=0x7fdc8c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 --2- 172.21.15.45:0/3513693574 >> v2:172.21.15.117:3300/0 conn(0x7fdc8c0d1820 0x7fdc8c0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 >> v1:172.21.15.117:6789/0 conn(0x7fdc8c15e050 legacy=0x7fdc8c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0b2390 con 0x7fdc8c0b15b0 2024-04-25T10:25:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fdc84001a00 con 0x7fdc8c0b15b0 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91c00640 1 --2- 172.21.15.45:0/3513693574 >> v2:172.21.15.117:3300/0 conn(0x7fdc8c0d1820 0x7fdc8c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 >> v1:172.21.15.45:6789/0 conn(0x7fdc8c0b15b0 legacy=0x7fdc8c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc74003ba0 0x7fdc74003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc740051d0 0x7fdc7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b20b0 con 0x7fdc74004810 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7400c0a0 con 0x7fdc740051d0 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7400c2a0 con 0x7fdc74003ba0 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91200640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc92600640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc740051d0 0x7fdc7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91c00640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc74003ba0 0x7fdc74003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91200640 1 -- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc74003ba0 msgr2=0x7fdc74003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91200640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc74003ba0 0x7fdc74003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91200640 1 -- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc740051d0 msgr2=0x7fdc7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91200640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc740051d0 0x7fdc7400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc91200640 1 -- 172.21.15.45:0/3513693574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc74003530 con 0x7fdc74004810 2024-04-25T10:25:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.787+0000 7fdc92600640 1 -- 172.21.15.45:0/3513693574 reap_dead start 2024-04-25T10:25:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc91200640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c070e60 tx=0x7fdc7801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8af276d040cf8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78001520 con 0x7fdc74004810 2024-04-25T10:25:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc780052f0 con 0x7fdc74004810 2024-04-25T10:25:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc8be00640 1 -- 172.21.15.45:0/3513693574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780a8910 con 0x7fdc74004810 2024-04-25T10:25:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 -- 172.21.15.45:0/3513693574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004810 msgr2=0x7fdc74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c070e60 tx=0x7fdc7801ecb0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 -- 172.21.15.45:0/3513693574 shutdown_connections 2024-04-25T10:25:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 --2- 172.21.15.45:0/3513693574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74004810 0x7fdc74004c90 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 -- 172.21.15.45:0/3513693574 >> 172.21.15.45:0/3513693574 conn(0x7fdc8c09a9d0 msgr2=0x7fdc8c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 -- 172.21.15.45:0/3513693574 shutdown_connections 2024-04-25T10:25:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.791+0000 7fdc94600640 1 -- 172.21.15.45:0/3513693574 wait complete. 2024-04-25T10:25:54.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 Processor -- start 2024-04-25T10:25:54.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- start start 2024-04-25T10:25:54.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74004810 0x7fdc8c0cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 0x7fdc8c0cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc8c0cc920 0x7fdc8c1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b1020 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b0ea0 con 0x7fdc74004810 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b11a0 con 0x7fdc8c0cc920 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 0x7fdc8c0cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 0x7fdc8c0cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45302/0 (socket says 172.21.15.45:45302) 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 -- 172.21.15.45:0/1595168025 learned_addr learned my addr 172.21.15.45:0/1595168025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91c00640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74004810 0x7fdc8c0cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc92600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc8c0cc920 0x7fdc8c1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 -- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc8c0cc920 msgr2=0x7fdc8c1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc8c0cc920 0x7fdc8c1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 -- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74004810 msgr2=0x7fdc8c0cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74004810 0x7fdc8c0cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 -- 172.21.15.45:0/1595168025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0cce30 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91c00640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74004810 0x7fdc8c0cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc91200640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 0x7fdc8c0cbfa0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fdc7801eae0 tx=0x7fdc74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780bd070 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc780054a0 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78001500 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c0cd110 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c15fba0 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.795+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c04ef30 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.799+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc780067e0 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.799+0000 7fdc89600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdc6c066f90 0x7fdc6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.799+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fdc78121700 con 0x7fdc8c0b53b0 2024-04-25T10:25:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.799+0000 7fdc91c00640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdc6c066f90 0x7fdc6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:54.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.799+0000 7fdc91c00640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdc6c066f90 0x7fdc6c069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c06a000 tx=0x7fdc80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:54.803+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc780189c0 con 0x7fdc8c0b53b0 2024-04-25T10:25:55.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.019+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdc8c06b480 con 0x7fdc8c0b53b0 2024-04-25T10:25:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc89600640 1 -- 172.21.15.45:0/1595168025 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1736 (secure 0 0 0) 0x7fdc780f26a0 con 0x7fdc8c0b53b0 2024-04-25T10:25:55.024 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:55.024 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076585911,"bytes_used":2578268160,"bytes_avail":770515845120,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":22729473,"read_op_per_sec":0,"write_op_per_sec":23},"fsmap":{"epoch":73,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7851}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:25:52.983902+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:25:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdc6c066f90 msgr2=0x7fdc6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdc6c066f90 0x7fdc6c069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c06a000 tx=0x7fdc80008040 comp rx=0 tx=0).stop 2024-04-25T10:25:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 msgr2=0x7fdc8c0cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 0x7fdc8c0cbfa0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fdc7801eae0 tx=0x7fdc74005840 comp rx=0 tx=0).stop 2024-04-25T10:25:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 shutdown_connections 2024-04-25T10:25:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdc6c066f90 0x7fdc6c069450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc8c0cc920 0x7fdc8c1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc8c0b53b0 0x7fdc8c0cbfa0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 --2- 172.21.15.45:0/1595168025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc74004810 0x7fdc8c0cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 >> 172.21.15.45:0/1595168025 conn(0x7fdc8c09a9d0 msgr2=0x7fdc8c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 shutdown_connections 2024-04-25T10:25:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.023+0000 7fdc94600640 1 -- 172.21.15.45:0/1595168025 wait complete. 2024-04-25T10:25:55.044 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:25:55.045 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:25:55.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:55.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:25:55.247 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:25:55.247 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:55.247 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:55.256 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:55.256 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:55.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:55.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.487+0000 7fd111c00640 1 Processor -- start 2024-04-25T10:25:55.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.487+0000 7fd111c00640 1 -- start start 2024-04-25T10:25:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd111c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 0x7fd10c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd111c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0b37e0 con 0x7fd10c15ddc0 2024-04-25T10:25:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd111c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0b3a60 con 0x7fd10c162290 2024-04-25T10:25:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd111c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15f1b0 con 0x7fd10c0ccf10 2024-04-25T10:25:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd10c15ddc0 0x7fd10c0cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53156/0 (socket says 172.21.15.45:53156) 2024-04-25T10:25:55.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 -- 172.21.15.45:0/1239091952 learned_addr learned my addr 172.21.15.45:0/1239091952 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414594267 0 0) 0x7fd0fc001120 con 0x7fd10c15ddc0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f0003530 con 0x7fd10c15ddc0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd103400640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 0x7fd10c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2953862638 0 0) 0x7fd0f0003530 con 0x7fd10c15ddc0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 msgr2=0x7fd10c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 0x7fd10c0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 >> v1:172.21.15.117:6789/0 conn(0x7fd10c162290 legacy=0x7fd10c06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0b3d40 con 0x7fd10c15ddc0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fd0fc001850 con 0x7fd10c15ddc0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 >> v1:172.21.15.45:6789/0 conn(0x7fd10c15ddc0 legacy=0x7fd10c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f00039f0 0x7fd0f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 0x7fd0f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0004d50 0x7fd0f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c15f1b0 con 0x7fd0f00039f0 2024-04-25T10:25:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f000bab0 con 0x7fd0f0004d50 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f000bcb0 con 0x7fd10c0ccf10 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f00039f0 0x7fd0f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd103400640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 0x7fd0f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd111200640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0004d50 0x7fd0f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 -- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 msgr2=0x7fd0f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0ccf10 0x7fd0f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 -- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0004d50 msgr2=0x7fd0f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f0004d50 0x7fd0f00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.491+0000 7fd10be00640 1 -- 172.21.15.45:0/1239091952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0003530 con 0x7fd0f00039f0 2024-04-25T10:25:55.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd10be00640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f00039f0 0x7fd0f0003e10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c2f0 tx=0x7fd0fc01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d620cbb9adbe7de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc007e20 con 0x7fd0f00039f0 2024-04-25T10:25:55.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc0a9070 con 0x7fd0f00039f0 2024-04-25T10:25:55.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd10b400640 1 -- 172.21.15.45:0/1239091952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc005d60 con 0x7fd0f00039f0 2024-04-25T10:25:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111c00640 1 -- 172.21.15.45:0/1239091952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f00039f0 msgr2=0x7fd0f0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111c00640 1 --2- 172.21.15.45:0/1239091952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f00039f0 0x7fd0f0003e10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c2f0 tx=0x7fd0fc01cb80 comp rx=0 tx=0).stop 2024-04-25T10:25:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111200640 1 -- 172.21.15.45:0/1239091952 reap_dead start 2024-04-25T10:25:55.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111c00640 1 -- 172.21.15.45:0/1239091952 shutdown_connections 2024-04-25T10:25:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111c00640 1 -- 172.21.15.45:0/1239091952 >> 172.21.15.45:0/1239091952 conn(0x7fd10c0b1a20 msgr2=0x7fd10c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111c00640 1 -- 172.21.15.45:0/1239091952 shutdown_connections 2024-04-25T10:25:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.495+0000 7fd111c00640 1 -- 172.21.15.45:0/1239091952 wait complete. 2024-04-25T10:25:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 Processor -- start 2024-04-25T10:25:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- start start 2024-04-25T10:25:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f00039f0 0x7fd10c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0cf910 0x7fd10c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 0x7fd10c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1699f0 con 0x7fd10c15a550 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0b4c30 con 0x7fd0f00039f0 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c169b70 con 0x7fd10c0cf910 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 0x7fd10c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 0x7fd10c15a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41802/0 (socket says 172.21.15.45:41802) 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 -- 172.21.15.45:0/2285307921 learned_addr learned my addr 172.21.15.45:0/2285307921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd103400640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0cf910 0x7fd10c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd10be00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f00039f0 0x7fd10c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 -- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0cf910 msgr2=0x7fd10c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0cf910 0x7fd10c0cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 -- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f00039f0 msgr2=0x7fd10c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f00039f0 0x7fd10c159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 -- 172.21.15.45:0/2285307921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0d0480 con 0x7fd10c15a550 2024-04-25T10:25:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111200640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 0x7fd10c15a9d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd10c070970 tx=0x7fd10400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10401a070 con 0x7fd10c15a550 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd104006d70 con 0x7fd10c15a550 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c0d0760 con 0x7fd10c15a550 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104003ba0 con 0x7fd10c15a550 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c0b1660 con 0x7fd10c15a550 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.499+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c075120 con 0x7fd10c15a550 2024-04-25T10:25:55.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.503+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd104005070 con 0x7fd10c15a550 2024-04-25T10:25:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.503+0000 7fd108c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0e8066f90 0x7fd0e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.503+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fd104084640 con 0x7fd10c15a550 2024-04-25T10:25:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.503+0000 7fd10be00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0e8066f90 0x7fd0e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.503+0000 7fd10be00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0e8066f90 0x7fd0e8069450 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0ab000 tx=0x7fd0fc0b1040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.507+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd104055550 con 0x7fd10c15a550 2024-04-25T10:25:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.667+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd10c0d0930 con 0x7fd10c15a550 2024-04-25T10:25:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.667+0000 7fd108c00640 1 -- 172.21.15.45:0/2285307921 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16751 (secure 0 0 0) 0x7fd104054e50 con 0x7fd10c15a550 2024-04-25T10:25:55.671 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:55.671 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:43.660339+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0e8066f90 msgr2=0x7fd0e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0e8066f90 0x7fd0e8069450 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc0ab000 tx=0x7fd0fc0b1040 comp rx=0 tx=0).stop 2024-04-25T10:25:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 msgr2=0x7fd10c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 0x7fd10c15a9d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd10c070970 tx=0x7fd10400fcd0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 shutdown_connections 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0e8066f90 0x7fd0e8069450 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd10c15a550 0x7fd10c15a9d0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd10c0cf910 0x7fd10c0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 --2- 172.21.15.45:0/2285307921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0f00039f0 0x7fd10c159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 >> 172.21.15.45:0/2285307921 conn(0x7fd10c0b1a20 msgr2=0x7fd10c15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 shutdown_connections 2024-04-25T10:25:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.671+0000 7fd111c00640 1 -- 172.21.15.45:0/2285307921 wait complete. 2024-04-25T10:25:55.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:25:14.488727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:25:15.698603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:55.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe036200640 1 Processor -- start 2024-04-25T10:25:55.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe036200640 1 -- start start 2024-04-25T10:25:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe036200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe030070350 0x7fe03015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe036200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03015b080 con 0x7fe0300b31c0 2024-04-25T10:25:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe036200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03015adb0 con 0x7fe0300aef80 2024-04-25T10:25:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe036200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03015eb60 con 0x7fe030070350 2024-04-25T10:25:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe02fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe0300b31c0 0x7fe0300cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53172/0 (socket says 172.21.15.45:53172) 2024-04-25T10:25:55.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.835+0000 7fe02fe00640 1 -- 172.21.15.45:0/271103750 learned_addr learned my addr 172.21.15.45:0/271103750 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe027e00640 1 --2- 172.21.15.45:0/271103750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe030070350 0x7fe03015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546561113 0 0) 0x7fe03015b080 con 0x7fe0300b31c0 2024-04-25T10:25:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe014003530 con 0x7fe0300b31c0 2024-04-25T10:25:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423384259 0 0) 0x7fe03015adb0 con 0x7fe0300aef80 2024-04-25T10:25:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe03015b080 con 0x7fe0300aef80 2024-04-25T10:25:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1610515787 0 0) 0x7fe014003530 con 0x7fe0300b31c0 2024-04-25T10:25:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe027e00640 1 -- 172.21.15.45:0/271103750 >> v1:172.21.15.117:6789/0 conn(0x7fe0300aef80 legacy=0x7fe03015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe027e00640 1 -- 172.21.15.45:0/271103750 >> v1:172.21.15.45:6789/0 conn(0x7fe0300b31c0 legacy=0x7fe0300cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe027e00640 1 -- 172.21.15.45:0/271103750 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03015ee40 con 0x7fe030070350 2024-04-25T10:25:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe027e00640 1 --2- 172.21.15.45:0/271103750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe030070350 0x7fe03015e480 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0180097f0 tx=0x7fe01801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950a21ca443ac875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180b2070 con 0x7fe030070350 2024-04-25T10:25:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0180a9070 con 0x7fe030070350 2024-04-25T10:25:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.839+0000 7fe02ea00640 1 -- 172.21.15.45:0/271103750 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028001a00 con 0x7fe030070350 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- 172.21.15.45:0/271103750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe030070350 msgr2=0x7fe03015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 --2- 172.21.15.45:0/271103750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe030070350 0x7fe03015e480 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0180097f0 tx=0x7fe01801c400 comp rx=0 tx=0).stop 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- 172.21.15.45:0/271103750 shutdown_connections 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 --2- 172.21.15.45:0/271103750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe030070350 0x7fe03015e480 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- 172.21.15.45:0/271103750 >> 172.21.15.45:0/271103750 conn(0x7fe0300669a0 msgr2=0x7fe030066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- 172.21.15.45:0/271103750 shutdown_connections 2024-04-25T10:25:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- 172.21.15.45:0/271103750 wait complete. 2024-04-25T10:25:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 Processor -- start 2024-04-25T10:25:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- start start 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe030070350 0x7fe030153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 0x7fe0301527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300b31c0 0x7fe030151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300cfef0 con 0x7fe0300aef80 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300cfd70 con 0x7fe030070350 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe036200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0300b3610 con 0x7fe0300b31c0 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 0x7fe0301527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 0x7fe0301527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41804/0 (socket says 172.21.15.45:41804) 2024-04-25T10:25:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 -- 172.21.15.45:0/3460511890 learned_addr learned my addr 172.21.15.45:0/3460511890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe02f400640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe030070350 0x7fe030153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe02fe00640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300b31c0 0x7fe030151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 -- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300b31c0 msgr2=0x7fe030151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300b31c0 0x7fe030151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 -- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe030070350 msgr2=0x7fe030153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe030070350 0x7fe030153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.843+0000 7fe027e00640 1 -- 172.21.15.45:0/3460511890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe030156a50 con 0x7fe0300aef80 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027e00640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 0x7fe0301527e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe0180b0040 tx=0x7fe0180191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180bd070 con 0x7fe0300aef80 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0180adcf0 con 0x7fe0300aef80 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180b2040 con 0x7fe0300aef80 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe030156d30 con 0x7fe0300aef80 2024-04-25T10:25:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030151750 con 0x7fe0300aef80 2024-04-25T10:25:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03004ef30 con 0x7fe0300aef80 2024-04-25T10:25:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0180ab070 con 0x7fe0300aef80 2024-04-25T10:25:55.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027400640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe00c066f90 0x7fe00c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:55.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fe01811dc00 con 0x7fe0300aef80 2024-04-25T10:25:55.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.847+0000 7fe02f400640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe00c066f90 0x7fe00c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.851+0000 7fe02f400640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe00c066f90 0x7fe00c069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe0200011f0 tx=0x7fe020009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:55.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:55.851+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe018124080 con 0x7fe0300aef80 2024-04-25T10:25:56.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe030068020 con 0x7fe0300aef80 2024-04-25T10:25:56.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe027400640 1 -- 172.21.15.45:0/3460511890 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe018015ff0 con 0x7fe0300aef80 2024-04-25T10:25:56.020 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe00c066f90 msgr2=0x7fe00c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe00c066f90 0x7fe00c069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe0200011f0 tx=0x7fe020009040 comp rx=0 tx=0).stop 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 msgr2=0x7fe0301527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 0x7fe0301527e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe0180b0040 tx=0x7fe0180191a0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 shutdown_connections 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe00c066f90 0x7fe00c069450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0300b31c0 0x7fe030151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0300aef80 0x7fe0301527e0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 --2- 172.21.15.45:0/3460511890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe030070350 0x7fe030153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 >> 172.21.15.45:0/3460511890 conn(0x7fe0300669a0 msgr2=0x7fe0300cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 shutdown_connections 2024-04-25T10:25:56.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.019+0000 7fe036200640 1 -- 172.21.15.45:0/3460511890 wait complete. 2024-04-25T10:25:56.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:56.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb50400640 1 Processor -- start 2024-04-25T10:25:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb50400640 1 -- start start 2024-04-25T10:25:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb50400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffb48070350 0x7ffb4815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb50400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4815b080 con 0x7ffb480b31c0 2024-04-25T10:25:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb50400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4815adb0 con 0x7ffb480aef80 2024-04-25T10:25:56.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb50400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4815eb60 con 0x7ffb48070350 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb4e400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffb480b31c0 0x7ffb480cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53180/0 (socket says 172.21.15.45:53180) 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb4e400640 1 -- 172.21.15.45:0/917715778 learned_addr learned my addr 172.21.15.45:0/917715778 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb4d000640 1 --2- 172.21.15.45:0/917715778 >> v2:172.21.15.117:3300/0 conn(0x7ffb48070350 0x7ffb4815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb47e00640 1 -- 172.21.15.45:0/917715778 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343306491 0 0) 0x7ffb4815b080 con 0x7ffb480b31c0 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb47e00640 1 -- 172.21.15.45:0/917715778 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb30005740 con 0x7ffb480b31c0 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb47e00640 1 -- 172.21.15.45:0/917715778 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3269803532 0 0) 0x7ffb30005740 con 0x7ffb480b31c0 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb4d000640 1 -- 172.21.15.45:0/917715778 >> v1:172.21.15.117:6789/0 conn(0x7ffb480aef80 legacy=0x7ffb4815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb4d000640 1 -- 172.21.15.45:0/917715778 >> v1:172.21.15.45:6789/0 conn(0x7ffb480b31c0 legacy=0x7ffb480cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.183+0000 7ffb4d000640 1 -- 172.21.15.45:0/917715778 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4815ee40 con 0x7ffb48070350 2024-04-25T10:25:56.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb4d000640 1 --2- 172.21.15.45:0/917715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb48070350 0x7ffb4815e480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ffb340097c0 tx=0x7ffb3401c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d11a5270e9e6a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb47e00640 1 -- 172.21.15.45:0/917715778 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb340b0070 con 0x7ffb48070350 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb47e00640 1 -- 172.21.15.45:0/917715778 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb340ab400 con 0x7ffb48070350 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb47e00640 1 -- 172.21.15.45:0/917715778 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40001a00 con 0x7ffb48070350 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 -- 172.21.15.45:0/917715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb48070350 msgr2=0x7ffb4815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 --2- 172.21.15.45:0/917715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb48070350 0x7ffb4815e480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ffb340097c0 tx=0x7ffb3401c270 comp rx=0 tx=0).stop 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 -- 172.21.15.45:0/917715778 shutdown_connections 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 --2- 172.21.15.45:0/917715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb48070350 0x7ffb4815e480 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 -- 172.21.15.45:0/917715778 >> 172.21.15.45:0/917715778 conn(0x7ffb480669a0 msgr2=0x7ffb48066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 -- 172.21.15.45:0/917715778 shutdown_connections 2024-04-25T10:25:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.187+0000 7ffb50400640 1 -- 172.21.15.45:0/917715778 wait complete. 2024-04-25T10:25:56.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 Processor -- start 2024-04-25T10:25:56.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 -- start start 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48070350 0x7ffb480afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480aef80 0x7ffb48158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 0x7ffb4814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480cfef0 con 0x7ffb480b31c0 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480cfd70 con 0x7ffb480aef80 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb480b3610 con 0x7ffb48070350 2024-04-25T10:25:56.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 0x7ffb4814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 0x7ffb4814c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41810/0 (socket says 172.21.15.45:41810) 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 -- 172.21.15.45:0/3308587420 learned_addr learned my addr 172.21.15.45:0/3308587420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4d000640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480aef80 0x7ffb48158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4da00640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48070350 0x7ffb480afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 -- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48070350 msgr2=0x7ffb480afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48070350 0x7ffb480afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 -- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480aef80 msgr2=0x7ffb48158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480aef80 0x7ffb48158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 -- 172.21.15.45:0/3308587420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb48159430 con 0x7ffb480b31c0 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4d000640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480aef80 0x7ffb48158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb4e400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 0x7ffb4814c470 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000caf0 tx=0x7ffb400021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb40018070 con 0x7ffb480b31c0 2024-04-25T10:25:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4000a700 con 0x7ffb480b31c0 2024-04-25T10:25:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4000aaf0 con 0x7ffb480b31c0 2024-04-25T10:25:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb48159710 con 0x7ffb480b31c0 2024-04-25T10:25:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.191+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb48159cd0 con 0x7ffb480b31c0 2024-04-25T10:25:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.195+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4804ef30 con 0x7ffb480b31c0 2024-04-25T10:25:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.195+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb4000ac90 con 0x7ffb480b31c0 2024-04-25T10:25:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.195+0000 7ffb45600640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb28066f60 0x7ffb28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.195+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7ffb400847b0 con 0x7ffb480b31c0 2024-04-25T10:25:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.195+0000 7ffb4da00640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb28066f60 0x7ffb28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.195+0000 7ffb4da00640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb28066f60 0x7ffb28069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0017b0 tx=0x7ffb3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.199+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb400558b0 con 0x7ffb480b31c0 2024-04-25T10:25:56.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.359+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb480b2fa0 con 0x7ffb480b31c0 2024-04-25T10:25:56.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb45600640 1 -- 172.21.15.45:0/3308587420 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb400551b0 con 0x7ffb480b31c0 2024-04-25T10:25:56.364 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb28066f60 msgr2=0x7ffb28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb28066f60 0x7ffb28069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0017b0 tx=0x7ffb3c009040 comp rx=0 tx=0).stop 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 msgr2=0x7ffb4814c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 0x7ffb4814c470 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000caf0 tx=0x7ffb400021e0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 shutdown_connections 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb28066f60 0x7ffb28069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb480b31c0 0x7ffb4814c470 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb480aef80 0x7ffb48158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 --2- 172.21.15.45:0/3308587420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb48070350 0x7ffb480afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 >> 172.21.15.45:0/3308587420 conn(0x7ffb480669a0 msgr2=0x7ffb48155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 shutdown_connections 2024-04-25T10:25:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.363+0000 7ffb50400640 1 -- 172.21.15.45:0/3308587420 wait complete. 2024-04-25T10:25:56.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:56.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 Processor -- start 2024-04-25T10:25:56.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 -- start start 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca5015a0e0 0x7fca5015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fca500aea50 0x7fca500b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca500b2fd0 con 0x7fca500b42d0 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca500af1d0 con 0x7fca5015a0e0 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca56200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca500b32b0 con 0x7fca500aea50 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca4f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca5015a0e0 0x7fca5015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.527+0000 7fca4fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fca500b42d0 0x7fca5006c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53186/0 (socket says 172.21.15.45:53186) 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4fe00640 1 -- 172.21.15.45:0/1811295173 learned_addr learned my addr 172.21.15.45:0/1811295173 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca46a00640 1 --2- 172.21.15.45:0/1811295173 >> v2:172.21.15.117:3300/0 conn(0x7fca500aea50 0x7fca500b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4f400640 1 -- 172.21.15.45:0/1811295173 >> v2:172.21.15.117:3300/0 conn(0x7fca500aea50 msgr2=0x7fca500b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4f400640 1 --2- 172.21.15.45:0/1811295173 >> v2:172.21.15.117:3300/0 conn(0x7fca500aea50 0x7fca500b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4f400640 1 -- 172.21.15.45:0/1811295173 >> v1:172.21.15.45:6789/0 conn(0x7fca500b42d0 legacy=0x7fca5006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4ea00640 1 -- 172.21.15.45:0/1811295173 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157486070 0 0) 0x7fca500b2fd0 con 0x7fca500b42d0 2024-04-25T10:25:56.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4f400640 1 -- 172.21.15.45:0/1811295173 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca500b3590 con 0x7fca5015a0e0 2024-04-25T10:25:56.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca46a00640 1 --2- 172.21.15.45:0/1811295173 >> v2:172.21.15.117:3300/0 conn(0x7fca500aea50 0x7fca500b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:25:56.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4f400640 1 --2- 172.21.15.45:0/1811295173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5015a0e0 0x7fca5015b150 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fca5015dae0 tx=0x7fca4001d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a8c42e3d2e44f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4ea00640 1 -- 172.21.15.45:0/1811295173 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400a6590 con 0x7fca5015a0e0 2024-04-25T10:25:56.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4ea00640 1 -- 172.21.15.45:0/1811295173 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca400a6730 con 0x7fca5015a0e0 2024-04-25T10:25:56.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.531+0000 7fca4ea00640 1 -- 172.21.15.45:0/1811295173 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400a6b00 con 0x7fca5015a0e0 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- 172.21.15.45:0/1811295173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5015a0e0 msgr2=0x7fca5015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 --2- 172.21.15.45:0/1811295173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5015a0e0 0x7fca5015b150 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fca5015dae0 tx=0x7fca4001d930 comp rx=0 tx=0).stop 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- 172.21.15.45:0/1811295173 shutdown_connections 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 --2- 172.21.15.45:0/1811295173 >> v2:172.21.15.117:3300/0 conn(0x7fca500aea50 0x7fca500b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 --2- 172.21.15.45:0/1811295173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5015a0e0 0x7fca5015b150 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- 172.21.15.45:0/1811295173 >> 172.21.15.45:0/1811295173 conn(0x7fca500669a0 msgr2=0x7fca50066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- 172.21.15.45:0/1811295173 shutdown_connections 2024-04-25T10:25:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- 172.21.15.45:0/1811295173 wait complete. 2024-04-25T10:25:56.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 Processor -- start 2024-04-25T10:25:56.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- start start 2024-04-25T10:25:56.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca500aea50 0x7fca50144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 0x7fca50150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca5015a0e0 0x7fca50145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50070770 con 0x7fca500b42d0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca500705f0 con 0x7fca500aea50 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca500708f0 con 0x7fca5015a0e0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 0x7fca50150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 0x7fca50150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41836/0 (socket says 172.21.15.45:41836) 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 -- 172.21.15.45:0/2735384796 learned_addr learned my addr 172.21.15.45:0/2735384796 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca4f400640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca500aea50 0x7fca50144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca4fe00640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca5015a0e0 0x7fca50145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 -- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca5015a0e0 msgr2=0x7fca50145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca5015a0e0 0x7fca50145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 -- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca500aea50 msgr2=0x7fca50144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca500aea50 0x7fca50144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 -- 172.21.15.45:0/2735384796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50147ec0 con 0x7fca500b42d0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca46a00640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 0x7fca50150200 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c7c0 tx=0x7fca3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38007a30 con 0x7fca500b42d0 2024-04-25T10:25:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38007bd0 con 0x7fca500b42d0 2024-04-25T10:25:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380113f0 con 0x7fca500b42d0 2024-04-25T10:25:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.535+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca501481a0 con 0x7fca500b42d0 2024-04-25T10:25:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca50148760 con 0x7fca500b42d0 2024-04-25T10:25:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca50145550 con 0x7fca500b42d0 2024-04-25T10:25:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca38011590 con 0x7fca500b42d0 2024-04-25T10:25:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca47e00640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca2c066f90 0x7fca2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fca380852f0 con 0x7fca500b42d0 2024-04-25T10:25:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca4f400640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca2c066f90 0x7fca2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.539+0000 7fca4f400640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca2c066f90 0x7fca2c069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fca400af000 tx=0x7fca400a8070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.543+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca38015020 con 0x7fca500b42d0 2024-04-25T10:25:56.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.707+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca5006e1f0 con 0x7fca500b42d0 2024-04-25T10:25:56.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.707+0000 7fca47e00640 1 -- 172.21.15.45:0/2735384796 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca38056200 con 0x7fca500b42d0 2024-04-25T10:25:56.710 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca2c066f90 msgr2=0x7fca2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca2c066f90 0x7fca2c069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fca400af000 tx=0x7fca400a8070 comp rx=0 tx=0).stop 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 msgr2=0x7fca50150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 0x7fca50150200 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c7c0 tx=0x7fca3800cc80 comp rx=0 tx=0).stop 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 shutdown_connections 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca2c066f90 0x7fca2c069450 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca5015a0e0 0x7fca50145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca500b42d0 0x7fca50150200 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 --2- 172.21.15.45:0/2735384796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca500aea50 0x7fca50144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 >> 172.21.15.45:0/2735384796 conn(0x7fca500669a0 msgr2=0x7fca500ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 shutdown_connections 2024-04-25T10:25:56.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.711+0000 7fca56200640 1 -- 172.21.15.45:0/2735384796 wait complete. 2024-04-25T10:25:56.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:25:56.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 Processor -- start 2024-04-25T10:25:56.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 -- start start 2024-04-25T10:25:56.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 0x7fa18415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa184071f70 0x7fa1840b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18415b6e0 con 0x7fa1840b2fc0 2024-04-25T10:25:56.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa184070ed0 con 0x7fa184071f70 2024-04-25T10:25:56.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa18ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840711b0 con 0x7fa18415a0e0 2024-04-25T10:25:56.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa183e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 0x7fa18415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa17b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa184071f70 0x7fa1840b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa183e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 0x7fa18415b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:36828/0 (socket says 172.21.15.45:36828) 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.871+0000 7fa183e00640 1 -- 172.21.15.45:0/278582898 learned_addr learned my addr 172.21.15.45:0/278582898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183e00640 1 -- 172.21.15.45:0/278582898 >> v2:172.21.15.117:3300/0 conn(0x7fa184071f70 msgr2=0x7fa1840b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183e00640 1 --2- 172.21.15.45:0/278582898 >> v2:172.21.15.117:3300/0 conn(0x7fa184071f70 0x7fa1840b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183e00640 1 -- 172.21.15.45:0/278582898 >> v1:172.21.15.117:6789/0 conn(0x7fa1840b2fc0 legacy=0x7fa18406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183e00640 1 -- 172.21.15.45:0/278582898 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184071490 con 0x7fa18415a0e0 2024-04-25T10:25:56.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa17b400640 1 --2- 172.21.15.45:0/278582898 >> v2:172.21.15.117:3300/0 conn(0x7fa184071f70 0x7fa1840b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:25:56.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183e00640 1 --2- 172.21.15.45:0/278582898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 0x7fa18415b1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa174009870 tx=0x7fa17401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c5fc4d635690843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183400640 1 -- 172.21.15.45:0/278582898 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740ab5f0 con 0x7fa18415a0e0 2024-04-25T10:25:56.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183400640 1 -- 172.21.15.45:0/278582898 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1740b0070 con 0x7fa18415a0e0 2024-04-25T10:25:56.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa183400640 1 -- 172.21.15.45:0/278582898 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740ab9b0 con 0x7fa18415a0e0 2024-04-25T10:25:56.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa18ae00640 1 -- 172.21.15.45:0/278582898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 msgr2=0x7fa18415b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.875+0000 7fa18ae00640 1 --2- 172.21.15.45:0/278582898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 0x7fa18415b1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa174009870 tx=0x7fa17401c560 comp rx=0 tx=0).stop 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- 172.21.15.45:0/278582898 shutdown_connections 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 --2- 172.21.15.45:0/278582898 >> v2:172.21.15.117:3300/0 conn(0x7fa184071f70 0x7fa1840b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 --2- 172.21.15.45:0/278582898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa18415a0e0 0x7fa18415b1a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- 172.21.15.45:0/278582898 >> 172.21.15.45:0/278582898 conn(0x7fa184066b70 msgr2=0x7fa184066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- 172.21.15.45:0/278582898 shutdown_connections 2024-04-25T10:25:56.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- 172.21.15.45:0/278582898 wait complete. 2024-04-25T10:25:56.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 Processor -- start 2024-04-25T10:25:56.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- start start 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184071f70 0x7fa184139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1840b2fc0 0x7fa18413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 0x7fa1841f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18415a4c0 con 0x7fa18415a0e0 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840af030 con 0x7fa184071f70 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa18ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa184070b50 con 0x7fa1840b2fc0 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 0x7fa1841f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 0x7fa1841f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41842/0 (socket says 172.21.15.45:41842) 2024-04-25T10:25:56.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 -- 172.21.15.45:0/1290354592 learned_addr learned my addr 172.21.15.45:0/1290354592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa17b400640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1840b2fc0 0x7fa18413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa183e00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184071f70 0x7fa184139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 -- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1840b2fc0 msgr2=0x7fa18413c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1840b2fc0 0x7fa18413c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 -- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184071f70 msgr2=0x7fa184139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184071f70 0x7fa184139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 -- 172.21.15.45:0/1290354592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18413cf60 con 0x7fa18415a0e0 2024-04-25T10:25:56.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.879+0000 7fa188e00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 0x7fa1841f7270 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c770 tx=0x7fa17c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c003b80 con 0x7fa18415a0e0 2024-04-25T10:25:56.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c003d20 con 0x7fa18415a0e0 2024-04-25T10:25:56.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00aa90 con 0x7fa18415a0e0 2024-04-25T10:25:56.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18413d240 con 0x7fa18415a0e0 2024-04-25T10:25:56.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1841f6a70 con 0x7fa18415a0e0 2024-04-25T10:25:56.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18404ef30 con 0x7fa18415a0e0 2024-04-25T10:25:56.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa17c00ac30 con 0x7fa18415a0e0 2024-04-25T10:25:56.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa180c00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa160066f60 0x7fa160069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:56.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fa17c084fe0 con 0x7fa18415a0e0 2024-04-25T10:25:56.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.883+0000 7fa183e00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa160066f60 0x7fa160069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:56.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.887+0000 7fa183e00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa160066f60 0x7fa160069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa1740af000 tx=0x7fa1740a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:56.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:56.887+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa18415b6e0 con 0x7fa18415a0e0 2024-04-25T10:25:57.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa18415e160 con 0x7fa18415a0e0 2024-04-25T10:25:57.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa180c00640 1 -- 172.21.15.45:0/1290354592 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa17c055f90 con 0x7fa18415a0e0 2024-04-25T10:25:57.065 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:25:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa160066f60 msgr2=0x7fa160069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa160066f60 0x7fa160069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa1740af000 tx=0x7fa1740a7000 comp rx=0 tx=0).stop 2024-04-25T10:25:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 msgr2=0x7fa1841f7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 0x7fa1841f7270 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c770 tx=0x7fa17c00cc30 comp rx=0 tx=0).stop 2024-04-25T10:25:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 shutdown_connections 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa160066f60 0x7fa160069420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18415a0e0 0x7fa1841f7270 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1840b2fc0 0x7fa18413c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 --2- 172.21.15.45:0/1290354592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184071f70 0x7fa184139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 >> 172.21.15.45:0/1290354592 conn(0x7fa184066b70 msgr2=0x7fa184159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 shutdown_connections 2024-04-25T10:25:57.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:57.063+0000 7fa18ae00640 1 -- 172.21.15.45:0/1290354592 wait complete. 2024-04-25T10:25:57.086 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:25:57.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:57.087 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:25:57.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:25:57.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:25:57.851 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:25:57.860 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:25:57.861 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:57.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:57.870 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:25:57.870 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:57.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:25:57.919 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:25:57.919 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:25:57.928 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:25:57.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:58.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f498ca00640 1 Processor -- start 2024-04-25T10:25:58.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f498ca00640 1 -- start start 2024-04-25T10:25:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f498ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49880669a0 0x7f4988066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f498ca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49880ad840 con 0x7f4988161600 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f498ca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498806e7a0 con 0x7f4988157f40 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f498ca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49880af9b0 con 0x7f49880669a0 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4987e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4988161600 0x7f4988161a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53202/0 (socket says 172.21.15.45:53202) 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4987e00640 1 -- 172.21.15.45:0/2554338679 learned_addr learned my addr 172.21.15.45:0/2554338679 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 --2- 172.21.15.45:0/2554338679 >> v2:172.21.15.117:3300/0 conn(0x7f49880669a0 0x7f4988066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324260275 0 0) 0x7f49800014a0 con 0x7f4988161600 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f496c003530 con 0x7f4988161600 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1309797202 0 0) 0x7f496c003530 con 0x7f4988161600 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 >> v2:172.21.15.117:3300/0 conn(0x7f49880669a0 msgr2=0x7f4988066e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 --2- 172.21.15.45:0/2554338679 >> v2:172.21.15.117:3300/0 conn(0x7f49880669a0 0x7f4988066e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 >> v1:172.21.15.117:6789/0 conn(0x7f4988157f40 legacy=0x7f49880aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498815e120 con 0x7f4988161600 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f4980001bd0 con 0x7f4988161600 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 >> v1:172.21.15.45:6789/0 conn(0x7f4988161600 legacy=0x7f4988161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c0039c0 0x7f496c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f496c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0051d0 0x7f496c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49880af9b0 con 0x7f496c004810 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c00c020 con 0x7f496c0039c0 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496c00c220 con 0x7f496c0051d0 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f496c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4987400640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c0039c0 0x7f496c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4987e00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0051d0 0x7f496c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 -- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0051d0 msgr2=0x7f496c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0051d0 0x7f496c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 -- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c0039c0 msgr2=0x7f496c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c0039c0 0x7f496c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f497fe00640 1 -- 172.21.15.45:0/2554338679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c003530 con 0x7f496c004810 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.099+0000 7f4987e00640 1 -- 172.21.15.45:0/2554338679 reap_dead start 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f497fe00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f496c004c90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4970009ae0 tx=0x7f497001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e4f2ff9487b2d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700b2070 con 0x7f496c004810 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49700a5660 con 0x7f496c004810 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f4986a00640 1 -- 172.21.15.45:0/2554338679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700096b0 con 0x7f496c004810 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 -- 172.21.15.45:0/2554338679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 msgr2=0x7f496c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f496c004c90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4970009ae0 tx=0x7f497001ca70 comp rx=0 tx=0).stop 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 -- 172.21.15.45:0/2554338679 shutdown_connections 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 --2- 172.21.15.45:0/2554338679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f496c004c90 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 -- 172.21.15.45:0/2554338679 >> 172.21.15.45:0/2554338679 conn(0x7f4988074310 msgr2=0x7f498806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 -- 172.21.15.45:0/2554338679 shutdown_connections 2024-04-25T10:25:58.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.103+0000 7f498ca00640 1 -- 172.21.15.45:0/2554338679 wait complete. 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 Processor -- start 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 -- start start 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f498813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49880b4cc0 0x7f49880cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4988142440 0x7f49881428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815e9c0 con 0x7f496c004810 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815e6e0 con 0x7f49880b4cc0 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498815eee0 con 0x7f4988142440 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f498813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f498813fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41858/0 (socket says 172.21.15.45:41858) 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 -- 172.21.15.45:0/2640056302 learned_addr learned my addr 172.21.15.45:0/2640056302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f497fe00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49880b4cc0 0x7f49880cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987e00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4988142440 0x7f49881428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 -- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4988142440 msgr2=0x7f49881428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4988142440 0x7f49881428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 -- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49880b4cc0 msgr2=0x7f49880cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49880b4cc0 0x7f49880cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 -- 172.21.15.45:0/2640056302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49881400e0 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f4987400640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f498813fba0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4978011610 tx=0x7f4978011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978019070 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4978015820 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978010040 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49881f3c10 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.107+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49880b41e0 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.111+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49880b3fd0 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.111+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f497801c070 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.111+0000 7f497f400640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4964066f90 0x7f4964069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.111+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f4978087bc0 con 0x7f496c004810 2024-04-25T10:25:58.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.111+0000 7f497fe00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4964066f90 0x7f4964069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.111+0000 7f497fe00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4964066f90 0x7f4964069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f49700af000 tx=0x7f49700046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.115+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4978058490 con 0x7f496c004810 2024-04-25T10:25:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.311+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4988075120 con 0x7f496c004810 2024-04-25T10:25:58.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.311+0000 7f497f400640 1 -- 172.21.15.45:0/2640056302 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f497805c530 con 0x7f496c004810 2024-04-25T10:25:58.315 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:58.315 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:25:58.315 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:25:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4964066f90 msgr2=0x7f4964069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4964066f90 0x7f4964069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f49700af000 tx=0x7f49700046d0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 msgr2=0x7f498813fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f498813fba0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4978011610 tx=0x7f4978011ad0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 shutdown_connections 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4964066f90 0x7f4964069450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4988142440 0x7f49881428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49880b4cc0 0x7f49880cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 --2- 172.21.15.45:0/2640056302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c004810 0x7f498813fba0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 >> 172.21.15.45:0/2640056302 conn(0x7f4988074310 msgr2=0x7f49880aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 shutdown_connections 2024-04-25T10:25:58.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.315+0000 7f498ca00640 1 -- 172.21.15.45:0/2640056302 wait complete. 2024-04-25T10:25:58.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:25:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 Processor -- start 2024-04-25T10:25:58.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 -- start start 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc78c0cdb60 0x7fc78c06aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c06bcb0 0x7fc78c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c0cdf40 con 0x7fc78c0cbdf0 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78c06b3e0 con 0x7fc78c0cdb60 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc791000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0b1230 con 0x7fc78c06bcb0 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc78c0cdb60 0x7fc78c06aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc78c0cdb60 0x7fc78c06aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41874/0 (socket says 172.21.15.45:41874) 2024-04-25T10:25:58.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 -- 172.21.15.45:0/3323277549 learned_addr learned my addr 172.21.15.45:0/3323277549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78be00640 1 --2- 172.21.15.45:0/3323277549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c06bcb0 0x7fc78c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 -- 172.21.15.45:0/3323277549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c06bcb0 msgr2=0x7fc78c0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 --2- 172.21.15.45:0/3323277549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c06bcb0 0x7fc78c0b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 -- 172.21.15.45:0/3323277549 >> v1:172.21.15.45:6789/0 conn(0x7fc78c0cbdf0 legacy=0x7fc78c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78b400640 1 -- 172.21.15.45:0/3323277549 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c0b1510 con 0x7fc78c0cdb60 2024-04-25T10:25:58.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.479+0000 7fc78aa00640 1 -- 172.21.15.45:0/3323277549 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424121916 0 0) 0x7fc78c0cdf40 con 0x7fc78c0cbdf0 2024-04-25T10:25:58.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc78b400640 1 --2- 172.21.15.45:0/3323277549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c06aea0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009870 tx=0x7fc77c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c6557fb1190958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc78aa00640 1 -- 172.21.15.45:0/3323277549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0a95f0 con 0x7fc78c0cdb60 2024-04-25T10:25:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc78aa00640 1 -- 172.21.15.45:0/3323277549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c0a9790 con 0x7fc78c0cdb60 2024-04-25T10:25:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc78aa00640 1 -- 172.21.15.45:0/3323277549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0a9b60 con 0x7fc78c0cdb60 2024-04-25T10:25:58.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 -- 172.21.15.45:0/3323277549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 msgr2=0x7fc78c06aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 --2- 172.21.15.45:0/3323277549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c06aea0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009870 tx=0x7fc77c01c530 comp rx=0 tx=0).stop 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 -- 172.21.15.45:0/3323277549 shutdown_connections 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 --2- 172.21.15.45:0/3323277549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c06bcb0 0x7fc78c0b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 --2- 172.21.15.45:0/3323277549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c06aea0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 -- 172.21.15.45:0/3323277549 >> 172.21.15.45:0/3323277549 conn(0x7fc78c0d17a0 msgr2=0x7fc78c0d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 -- 172.21.15.45:0/3323277549 shutdown_connections 2024-04-25T10:25:58.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 -- 172.21.15.45:0/3323277549 wait complete. 2024-04-25T10:25:58.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 Processor -- start 2024-04-25T10:25:58.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.483+0000 7fc791000640 1 -- start start 2024-04-25T10:25:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc78c06bcb0 0x7fc78c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c0cbdf0 0x7fc78c154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c15d590 con 0x7fc78c0cdb60 2024-04-25T10:25:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c15d410 con 0x7fc78c06bcb0 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c15d710 con 0x7fc78c0cbdf0 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41876/0 (socket says 172.21.15.45:41876) 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 -- 172.21.15.45:0/2433623043 learned_addr learned my addr 172.21.15.45:0/2433623043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc783e00640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c0cbdf0 0x7fc78c154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78b400640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc78c06bcb0 0x7fc78c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 -- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c0cbdf0 msgr2=0x7fc78c154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c0cbdf0 0x7fc78c154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 -- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc78c06bcb0 msgr2=0x7fc78c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc78c06bcb0 0x7fc78c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 -- 172.21.15.45:0/2433623043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c150fa0 con 0x7fc78c0cdb60 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc78be00640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c155030 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc784007e70 tx=0x7fc78400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784014620 con 0x7fc78c0cdb60 2024-04-25T10:25:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784019070 con 0x7fc78c0cdb60 2024-04-25T10:25:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784014a20 con 0x7fc78c0cdb60 2024-04-25T10:25:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c156ad0 con 0x7fc78c0cdb60 2024-04-25T10:25:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c157080 con 0x7fc78c0cdb60 2024-04-25T10:25:58.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.487+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c075120 con 0x7fc78c0cdb60 2024-04-25T10:25:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.491+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc784014bc0 con 0x7fc78c0cdb60 2024-04-25T10:25:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.491+0000 7fc783400640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc768066fb0 0x7fc768069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.491+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fc784086700 con 0x7fc78c0cdb60 2024-04-25T10:25:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.491+0000 7fc78b400640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc768066fb0 0x7fc768069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.491+0000 7fc78b400640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc768066fb0 0x7fc768069470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0ae000 tx=0x7fc77c0b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.495+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc784057820 con 0x7fc78c0cdb60 2024-04-25T10:25:58.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.651+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc78c06c740 con 0x7fc78c0cdb60 2024-04-25T10:25:58.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc783400640 1 -- 172.21.15.45:0/2433623043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16751 (secure 0 0 0) 0x7fc784057120 con 0x7fc78c0cdb60 2024-04-25T10:25:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:25:58.657 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:43.660339+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:25:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc768066fb0 msgr2=0x7fc768069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc768066fb0 0x7fc768069470 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0ae000 tx=0x7fc77c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 msgr2=0x7fc78c155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c155030 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc784007e70 tx=0x7fc78400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 shutdown_connections 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc768066fb0 0x7fc768069470 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc78c0cdb60 0x7fc78c155030 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78c0cbdf0 0x7fc78c154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 --2- 172.21.15.45:0/2433623043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc78c06bcb0 0x7fc78c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 >> 172.21.15.45:0/2433623043 conn(0x7fc78c0d17a0 msgr2=0x7fc78c0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 shutdown_connections 2024-04-25T10:25:58.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.655+0000 7fc791000640 1 -- 172.21.15.45:0/2433623043 wait complete. 2024-04-25T10:25:58.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:25:14.488727+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:25:15.698603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:25:58.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:25:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 Processor -- start 2024-04-25T10:25:58.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 -- start start 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5d4159540 0x7fa5d40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 0x7fa5d4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d4159920 con 0x7fa5d40aebd0 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d415c2f0 con 0x7fa5d4159540 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5dc800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415c5d0 con 0x7fa5d40721f0 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa5d40aebd0 0x7fa5d4071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53212/0 (socket says 172.21.15.45:53212) 2024-04-25T10:25:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9400640 1 -- 172.21.15.45:0/321352791 learned_addr learned my addr 172.21.15.45:0/321352791 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5da800640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 0x7fa5d4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/321352791 >> v2:172.21.15.117:3300/0 conn(0x7fa5d4159540 0x7fa5d40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236943482 0 0) 0x7fa5d4159920 con 0x7fa5d40aebd0 2024-04-25T10:25:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc003530 con 0x7fa5d40aebd0 2024-04-25T10:25:58.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3687681018 0 0) 0x7fa5bc003530 con 0x7fa5d40aebd0 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 msgr2=0x7fa5d4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 0x7fa5d4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 >> v2:172.21.15.117:3300/0 conn(0x7fa5d4159540 msgr2=0x7fa5d40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 --2- 172.21.15.45:0/321352791 >> v2:172.21.15.117:3300/0 conn(0x7fa5d4159540 0x7fa5d40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d415c8b0 con 0x7fa5d40aebd0 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fa5c4001680 con 0x7fa5d40aebd0 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 >> v1:172.21.15.45:6789/0 conn(0x7fa5d40aebd0 legacy=0x7fa5d4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc0039c0 0x7fa5bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc004810 0x7fa5bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 0x7fa5bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415c2f0 con 0x7fa5bc0039c0 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415c5d0 con 0x7fa5bc004810 2024-04-25T10:25:58.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc00b9d0 con 0x7fa5d40721f0 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc0039c0 0x7fa5bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9400640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc004810 0x7fa5bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5da800640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 0x7fa5bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 -- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 msgr2=0x7fa5bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5d40721f0 0x7fa5bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 -- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc004810 msgr2=0x7fa5bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc004810 0x7fa5bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.823+0000 7fa5d9e00640 1 -- 172.21.15.45:0/321352791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc003530 con 0x7fa5bc0039c0 2024-04-25T10:25:58.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc0039c0 0x7fa5bc003de0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009ae0 tx=0x7fa5c801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c00f2451440fd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80096b0 con 0x7fa5bc0039c0 2024-04-25T10:25:58.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c80a64c0 con 0x7fa5bc0039c0 2024-04-25T10:25:58.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5d8a00640 1 -- 172.21.15.45:0/321352791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80a66b0 con 0x7fa5bc0039c0 2024-04-25T10:25:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5dc800640 1 -- 172.21.15.45:0/321352791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc0039c0 msgr2=0x7fa5bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5dc800640 1 --2- 172.21.15.45:0/321352791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc0039c0 0x7fa5bc003de0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009ae0 tx=0x7fa5c801ca50 comp rx=0 tx=0).stop 2024-04-25T10:25:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5da800640 1 -- 172.21.15.45:0/321352791 reap_dead start 2024-04-25T10:25:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5dc800640 1 -- 172.21.15.45:0/321352791 shutdown_connections 2024-04-25T10:25:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5dc800640 1 -- 172.21.15.45:0/321352791 >> 172.21.15.45:0/321352791 conn(0x7fa5d40669a0 msgr2=0x7fa5d4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:58.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5dc800640 1 -- 172.21.15.45:0/321352791 shutdown_connections 2024-04-25T10:25:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.827+0000 7fa5dc800640 1 -- 172.21.15.45:0/321352791 wait complete. 2024-04-25T10:25:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 Processor -- start 2024-04-25T10:25:58.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- start start 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0039c0 0x7fa5d4148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d41f6db0 0x7fa5d40afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 0x7fa5d40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4070ce0 con 0x7fa5d40affe0 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4070b60 con 0x7fa5d41f6db0 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4070e60 con 0x7fa5bc0039c0 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 0x7fa5d40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 0x7fa5d40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41894/0 (socket says 172.21.15.45:41894) 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 -- 172.21.15.45:0/2384713944 learned_addr learned my addr 172.21.15.45:0/2384713944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5d9400640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d41f6db0 0x7fa5d40afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0039c0 0x7fa5d4148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 -- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0039c0 msgr2=0x7fa5d4148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0039c0 0x7fa5d4148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 -- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d41f6db0 msgr2=0x7fa5d40afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d41f6db0 0x7fa5d40afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 -- 172.21.15.45:0/2384713944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4157120 con 0x7fa5d40affe0 2024-04-25T10:25:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5da800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 0x7fa5d40b3d40 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0007d10 tx=0x7fa5d00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0011de0 con 0x7fa5d40affe0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d001b070 con 0x7fa5d40affe0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00039a0 con 0x7fa5d40affe0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d4157400 con 0x7fa5d40affe0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41579c0 con 0x7fa5d40affe0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.831+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d404ef30 con 0x7fa5d40affe0 2024-04-25T10:25:58.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.835+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5d0003b40 con 0x7fa5d40affe0 2024-04-25T10:25:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.835+0000 7fa5ce000640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b8066f90 0x7fa5b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.835+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fa5d0085800 con 0x7fa5d40affe0 2024-04-25T10:25:58.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.835+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b8066f90 0x7fa5b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:58.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.835+0000 7fa5d9e00640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b8066f90 0x7fa5b8069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80af000 tx=0x7fa5c80046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:58.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:58.839+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5d0025020 con 0x7fa5d40affe0 2024-04-25T10:25:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.003+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5d406d7f0 con 0x7fa5d40affe0 2024-04-25T10:25:59.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.003+0000 7fa5ce000640 1 -- 172.21.15.45:0/2384713944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5d00560c0 con 0x7fa5d40affe0 2024-04-25T10:25:59.006 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b8066f90 msgr2=0x7fa5b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b8066f90 0x7fa5b8069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80af000 tx=0x7fa5c80046d0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 msgr2=0x7fa5d40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 0x7fa5d40b3d40 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0007d10 tx=0x7fa5d00115a0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 shutdown_connections 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5b8066f90 0x7fa5b8069450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d40affe0 0x7fa5d40b3d40 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5d41f6db0 0x7fa5d40afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 --2- 172.21.15.45:0/2384713944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0039c0 0x7fa5d4148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 >> 172.21.15.45:0/2384713944 conn(0x7fa5d40669a0 msgr2=0x7fa5d406de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 shutdown_connections 2024-04-25T10:25:59.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.007+0000 7fa5dc800640 1 -- 172.21.15.45:0/2384713944 wait complete. 2024-04-25T10:25:59.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:25:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.167+0000 7ff599600640 1 Processor -- start 2024-04-25T10:25:59.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.167+0000 7ff599600640 1 -- start start 2024-04-25T10:25:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff599600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff59415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff599600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59415b080 con 0x7ff5940aef80 2024-04-25T10:25:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff599600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59415adb0 con 0x7ff5940b31c0 2024-04-25T10:25:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff599600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59415eb60 con 0x7ff594070350 2024-04-25T10:25:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff5940aef80 0x7ff59415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53220/0 (socket says 172.21.15.45:53220) 2024-04-25T10:25:59.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592a00640 1 -- 172.21.15.45:0/3755066346 learned_addr learned my addr 172.21.15.45:0/3755066346 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff58a000640 1 --2- 172.21.15.45:0/3755066346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff59415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 400869607 0 0) 0x7ff59415b080 con 0x7ff5940aef80 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff578005740 con 0x7ff5940aef80 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582014681 0 0) 0x7ff59415adb0 con 0x7ff5940b31c0 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff59415b080 con 0x7ff5940b31c0 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573678368 0 0) 0x7ff578005740 con 0x7ff5940aef80 2024-04-25T10:25:59.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff58a000640 1 -- 172.21.15.45:0/3755066346 >> v1:172.21.15.117:6789/0 conn(0x7ff5940b31c0 legacy=0x7ff5940cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff58a000640 1 -- 172.21.15.45:0/3755066346 >> v1:172.21.15.45:6789/0 conn(0x7ff5940aef80 legacy=0x7ff59415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.171+0000 7ff58a000640 1 -- 172.21.15.45:0/3755066346 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59415ee40 con 0x7ff594070350 2024-04-25T10:25:59.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff58a000640 1 --2- 172.21.15.45:0/3755066346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff59415e480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8591197bdc35cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b0070 con 0x7ff594070350 2024-04-25T10:25:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c0ab400 con 0x7ff594070350 2024-04-25T10:25:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff592000640 1 -- 172.21.15.45:0/3755066346 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584001680 con 0x7ff594070350 2024-04-25T10:25:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 -- 172.21.15.45:0/3755066346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 msgr2=0x7ff59415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 --2- 172.21.15.45:0/3755066346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff59415e480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c01c400 comp rx=0 tx=0).stop 2024-04-25T10:25:59.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 -- 172.21.15.45:0/3755066346 shutdown_connections 2024-04-25T10:25:59.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 --2- 172.21.15.45:0/3755066346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff59415e480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 -- 172.21.15.45:0/3755066346 >> 172.21.15.45:0/3755066346 conn(0x7ff5940669a0 msgr2=0x7ff594066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 -- 172.21.15.45:0/3755066346 shutdown_connections 2024-04-25T10:25:59.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 -- 172.21.15.45:0/3755066346 wait complete. 2024-04-25T10:25:59.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 Processor -- start 2024-04-25T10:25:59.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.175+0000 7ff599600640 1 -- start start 2024-04-25T10:25:59.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff5940ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 0x7ff5941519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5940b31c0 0x7ff594151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cfef0 con 0x7ff5940aef80 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cfd70 con 0x7ff5940b31c0 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940b35d0 con 0x7ff594070350 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 0x7ff5941519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 0x7ff5941519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41904/0 (socket says 172.21.15.45:41904) 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 -- 172.21.15.45:0/2026835583 learned_addr learned my addr 172.21.15.45:0/2026835583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff592a00640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff5940ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff593400640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5940b31c0 0x7ff594151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 -- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 msgr2=0x7ff5940ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff5940ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 -- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5940b31c0 msgr2=0x7ff594151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5940b31c0 0x7ff594151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 -- 172.21.15.45:0/2026835583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594156a80 con 0x7ff5940aef80 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff592a00640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff5940ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58a000640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 0x7ff5941519d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b8070 con 0x7ff5940aef80 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c004020 con 0x7ff5940aef80 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b0040 con 0x7ff5940aef80 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff594156d00 con 0x7ff5940aef80 2024-04-25T10:25:59.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5941572c0 con 0x7ff5940aef80 2024-04-25T10:25:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.179+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59404ef30 con 0x7ff5940aef80 2024-04-25T10:25:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.183+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff57c003620 con 0x7ff5940aef80 2024-04-25T10:25:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.183+0000 7ff58b400640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff570066f60 0x7ff570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.183+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7ff57c11ea00 con 0x7ff5940aef80 2024-04-25T10:25:59.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.183+0000 7ff592a00640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff570066f60 0x7ff570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.183+0000 7ff592a00640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff570066f60 0x7ff570069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff584001000 tx=0x7ff584009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.187+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff594156a80 con 0x7ff5940aef80 2024-04-25T10:25:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff59415b620 con 0x7ff5940aef80 2024-04-25T10:25:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff58b400640 1 -- 172.21.15.45:0/2026835583 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff57c015ff0 con 0x7ff5940aef80 2024-04-25T10:25:59.349 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:25:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff570066f60 msgr2=0x7ff570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff570066f60 0x7ff570069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff584001000 tx=0x7ff584009040 comp rx=0 tx=0).stop 2024-04-25T10:25:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 msgr2=0x7ff5941519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 0x7ff5941519d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c0191a0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 shutdown_connections 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff570066f60 0x7ff570069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5940b31c0 0x7ff594151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5940aef80 0x7ff5941519d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 --2- 172.21.15.45:0/2026835583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff594070350 0x7ff5940ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 >> 172.21.15.45:0/2026835583 conn(0x7ff5940669a0 msgr2=0x7ff5940d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 shutdown_connections 2024-04-25T10:25:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.347+0000 7ff599600640 1 -- 172.21.15.45:0/2026835583 wait complete. 2024-04-25T10:25:59.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:25:59.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:59.487 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:25:59.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.511+0000 7f17b7a00640 1 Processor -- start 2024-04-25T10:25:59.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.511+0000 7f17b7a00640 1 -- start start 2024-04-25T10:25:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b7a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f17b015dd30 0x7f17b006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b7a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 0x7f17b00d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b7a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b006b1d0 con 0x7f17b00ce220 2024-04-25T10:25:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b7a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b015e110 con 0x7f17b015dd30 2024-04-25T10:25:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b7a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b00b2940 con 0x7f17b00cfb30 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b5000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f17b015dd30 0x7f17b006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 0x7f17b00d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 0x7f17b00d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:36938/0 (socket says 172.21.15.45:36938) 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 -- 172.21.15.45:0/2238597166 learned_addr learned my addr 172.21.15.45:0/2238597166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17b5a00640 1 --1- 172.21.15.45:0/2238597166 >> v1:172.21.15.117:6789/0 conn(0x7f17b00ce220 0x7f17b01617c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:51910/0 (socket says 172.21.15.45:51910) 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 -- 172.21.15.45:0/2238597166 >> v2:172.21.15.117:3300/0 conn(0x7f17b015dd30 msgr2=0x7f17b006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 --2- 172.21.15.45:0/2238597166 >> v2:172.21.15.117:3300/0 conn(0x7f17b015dd30 0x7f17b006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 -- 172.21.15.45:0/2238597166 >> v1:172.21.15.117:6789/0 conn(0x7f17b00ce220 legacy=0x7f17b01617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.515+0000 7f17a7e00640 1 -- 172.21.15.45:0/2238597166 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b006b470 con 0x7f17b00cfb30 2024-04-25T10:25:59.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17a7e00640 1 --2- 172.21.15.45:0/2238597166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 0x7f17b00d18a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1798009870 tx=0x7f179801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8fe368e0f6a4096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17afe00640 1 -- 172.21.15.45:0/2238597166 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17980b2070 con 0x7f17b00cfb30 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17afe00640 1 -- 172.21.15.45:0/2238597166 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17980a9070 con 0x7f17b00cfb30 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17afe00640 1 -- 172.21.15.45:0/2238597166 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17980ad880 con 0x7f17b00cfb30 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 -- 172.21.15.45:0/2238597166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 msgr2=0x7f17b00d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 --2- 172.21.15.45:0/2238597166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 0x7f17b00d18a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1798009870 tx=0x7f179801c560 comp rx=0 tx=0).stop 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 -- 172.21.15.45:0/2238597166 shutdown_connections 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 --2- 172.21.15.45:0/2238597166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b00cfb30 0x7f17b00d18a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 --2- 172.21.15.45:0/2238597166 >> v2:172.21.15.117:3300/0 conn(0x7f17b015dd30 0x7f17b006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 -- 172.21.15.45:0/2238597166 >> 172.21.15.45:0/2238597166 conn(0x7f17b009a9d0 msgr2=0x7f17b009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 -- 172.21.15.45:0/2238597166 shutdown_connections 2024-04-25T10:25:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 -- 172.21.15.45:0/2238597166 wait complete. 2024-04-25T10:25:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 Processor -- start 2024-04-25T10:25:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 -- start start 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b00ce220 0x7f17b0156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.519+0000 7f17b7a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 0x7f17b0159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b015dd30 0x7f17b0154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b016ace0 con 0x7f17b00cfb30 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b016ab60 con 0x7f17b00ce220 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b016ae60 con 0x7f17b015dd30 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 0x7f17b0159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 0x7f17b0159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41920/0 (socket says 172.21.15.45:41920) 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 -- 172.21.15.45:0/3368671140 learned_addr learned my addr 172.21.15.45:0/3368671140 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b5a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b015dd30 0x7f17b0154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b5000640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b00ce220 0x7f17b0156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 -- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b015dd30 msgr2=0x7f17b0154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b015dd30 0x7f17b0154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 -- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b00ce220 msgr2=0x7f17b0156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b00ce220 0x7f17b0156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 -- 172.21.15.45:0/3368671140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0159d80 con 0x7f17b00cfb30 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17a7e00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 0x7f17b0159840 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f179801a190 tx=0x7f1798019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17980bb070 con 0x7f17b00cfb30 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17980add00 con 0x7f17b00cfb30 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17980b2040 con 0x7f17b00cfb30 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b015a050 con 0x7f17b00cfb30 2024-04-25T10:25:59.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b015a610 con 0x7f17b00cfb30 2024-04-25T10:25:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b004ef30 con 0x7f17b00cfb30 2024-04-25T10:25:59.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.523+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17980ab070 con 0x7f17b00cfb30 2024-04-25T10:25:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.527+0000 7f17ad600640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f178c066f60 0x7f178c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.527+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f179811dfd0 con 0x7f17b00cfb30 2024-04-25T10:25:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.527+0000 7f17b5000640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f178c066f60 0x7f178c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.527+0000 7f17b5000640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f178c066f60 0x7f178c069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f17b0159180 tx=0x7f17a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.527+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1798016150 con 0x7f17b00cfb30 2024-04-25T10:25:59.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.691+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17b00b7920 con 0x7f17b00cfb30 2024-04-25T10:25:59.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.691+0000 7f17ad600640 1 -- 172.21.15.45:0/3368671140 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f17980eef70 con 0x7f17b00cfb30 2024-04-25T10:25:59.694 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f178c066f60 msgr2=0x7f178c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f178c066f60 0x7f178c069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f17b0159180 tx=0x7f17a0008040 comp rx=0 tx=0).stop 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 msgr2=0x7f17b0159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 0x7f17b0159840 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f179801a190 tx=0x7f1798019300 comp rx=0 tx=0).stop 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 shutdown_connections 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f178c066f60 0x7f178c069420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f17b015dd30 0x7f17b0154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17b00cfb30 0x7f17b0159840 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 --2- 172.21.15.45:0/3368671140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f17b00ce220 0x7f17b0156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 >> 172.21.15.45:0/3368671140 conn(0x7f17b009a9d0 msgr2=0x7f17b006fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 shutdown_connections 2024-04-25T10:25:59.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.695+0000 7f17b7a00640 1 -- 172.21.15.45:0/3368671140 wait complete. 2024-04-25T10:25:59.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:25:59.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 Processor -- start 2024-04-25T10:25:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 -- start start 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c0cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fae3c070c20 0x7fae3c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0d0250 con 0x7fae3c0ae020 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c15bb90 con 0x7fae3c070c20 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae43400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0ad840 con 0x7fae3c0cf930 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae41400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fae3c0ae020 0x7fae3c0b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53234/0 (socket says 172.21.15.45:53234) 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae41400640 1 -- 172.21.15.45:0/2985619760 learned_addr learned my addr 172.21.15.45:0/2985619760 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae33e00640 1 --2- 172.21.15.45:0/2985619760 >> v2:172.21.15.117:3300/0 conn(0x7fae3c070c20 0x7fae3c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae40a00640 1 --2- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c0cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae3be00640 1 -- 172.21.15.45:0/2985619760 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80217557 0 0) 0x7fae3c0d0250 con 0x7fae3c0ae020 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae3be00640 1 -- 172.21.15.45:0/2985619760 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae20003530 con 0x7fae3c0ae020 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae33e00640 1 -- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 msgr2=0x7fae3c0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae33e00640 1 --2- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c0cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae33e00640 1 -- 172.21.15.45:0/2985619760 >> v1:172.21.15.45:6789/0 conn(0x7fae3c0ae020 legacy=0x7fae3c0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae3be00640 1 -- 172.21.15.45:0/2985619760 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1374120126 0 0) 0x7fae20003530 con 0x7fae3c0ae020 2024-04-25T10:25:59.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.855+0000 7fae33e00640 1 -- 172.21.15.45:0/2985619760 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c15e9f0 con 0x7fae3c070c20 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae33e00640 1 --2- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 0x7fae3c15e170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fae240097c0 tx=0x7fae2401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5abd07c1759da61b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae3be00640 1 -- 172.21.15.45:0/2985619760 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240b0070 con 0x7fae3c070c20 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae3be00640 1 -- 172.21.15.45:0/2985619760 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae240ab400 con 0x7fae3c070c20 2024-04-25T10:25:59.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae3be00640 1 -- 172.21.15.45:0/2985619760 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34001a00 con 0x7fae3c070c20 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 -- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 msgr2=0x7fae3c15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 --2- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 0x7fae3c15e170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fae240097c0 tx=0x7fae2401c390 comp rx=0 tx=0).stop 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 -- 172.21.15.45:0/2985619760 shutdown_connections 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 --2- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 0x7fae3c15e170 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 --2- 172.21.15.45:0/2985619760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 -- 172.21.15.45:0/2985619760 >> 172.21.15.45:0/2985619760 conn(0x7fae3c066b70 msgr2=0x7fae3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:25:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 -- 172.21.15.45:0/2985619760 shutdown_connections 2024-04-25T10:25:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.859+0000 7fae43400640 1 -- 172.21.15.45:0/2985619760 wait complete. 2024-04-25T10:25:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 Processor -- start 2024-04-25T10:25:59.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 -- start start 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 0x7fae3c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 0x7fae3c153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c15e6b0 con 0x7fae3c0ae020 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c06e130 con 0x7fae3c070c20 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c16b0f0 con 0x7fae3c0cf930 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 0x7fae3c153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 0x7fae3c153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41924/0 (socket says 172.21.15.45:41924) 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 -- 172.21.15.45:0/1050803437 learned_addr learned my addr 172.21.15.45:0/1050803437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:25:59.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 -- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 msgr2=0x7fae3c1558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 -- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 msgr2=0x7fae3c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 0x7fae3c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 -- 172.21.15.45:0/1050803437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c152500 con 0x7fae3c0ae020 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae33e00640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 0x7fae3c153b10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fae240ae040 tx=0x7fae24019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240b0040 con 0x7fae3c0ae020 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae240b9070 con 0x7fae3c0ae020 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae240b48b0 con 0x7fae3c0ae020 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c1527e0 con 0x7fae3c0ae020 2024-04-25T10:25:59.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.863+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c143fc0 con 0x7fae3c0ae020 2024-04-25T10:25:59.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.867+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c04ef30 con 0x7fae3c0ae020 2024-04-25T10:25:59.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.867+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae240b4a50 con 0x7fae3c0ae020 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.867+0000 7fae39600640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fae18066f90 0x7fae18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.867+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fae2411efe0 con 0x7fae3c0ae020 2024-04-25T10:25:59.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.867+0000 7fae40a00640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fae18066f90 0x7fae18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:25:59.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.867+0000 7fae40a00640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fae18066f90 0x7fae18069450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fae2c006aa0 tx=0x7fae2c006a30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:25:59.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:25:59.871+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae24015fb0 con 0x7fae3c0ae020 2024-04-25T10:26:00.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.067+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae3c0b3cb0 con 0x7fae3c0ae020 2024-04-25T10:26:00.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.067+0000 7fae39600640 1 -- 172.21.15.45:0/1050803437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fae240effa0 con 0x7fae3c0ae020 2024-04-25T10:26:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:00.070 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:26:00.070 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.067+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fae18066f90 msgr2=0x7fae18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.067+0000 7fae43400640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fae18066f90 0x7fae18069450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fae2c006aa0 tx=0x7fae2c006a30 comp rx=0 tx=0).stop 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.067+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 msgr2=0x7fae3c153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.067+0000 7fae43400640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 0x7fae3c153b10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fae240ae040 tx=0x7fae24019130 comp rx=0 tx=0).stop 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 shutdown_connections 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fae18066f90 0x7fae18069450 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae3c0cf930 0x7fae3c1558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae3c0ae020 0x7fae3c153b10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 --2- 172.21.15.45:0/1050803437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae3c070c20 0x7fae3c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 >> 172.21.15.45:0/1050803437 conn(0x7fae3c066b70 msgr2=0x7fae3c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 shutdown_connections 2024-04-25T10:26:00.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.071+0000 7fae43400640 1 -- 172.21.15.45:0/1050803437 wait complete. 2024-04-25T10:26:00.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:00.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 Processor -- start 2024-04-25T10:26:00.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 -- start start 2024-04-25T10:26:00.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb03c0ad850 0x7fb03c0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c15e0c0 con 0x7fb03c15dce0 2024-04-25T10:26:00.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c0674d0 con 0x7fb03c0ad850 2024-04-25T10:26:00.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.231+0000 7fb044400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0671f0 con 0x7fb03c06eec0 2024-04-25T10:26:00.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb042400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb03c0ad850 0x7fb03c0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb041a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb03c15dce0 0x7fb03c0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:51916/0 (socket says 172.21.15.45:51916) 2024-04-25T10:26:00.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb041a00640 1 -- 172.21.15.45:0/3502658262 learned_addr learned my addr 172.21.15.45:0/3502658262 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:00.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb041000640 1 --2- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb042400640 1 -- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 msgr2=0x7fb03c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:00.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb042400640 1 --2- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb042400640 1 -- 172.21.15.45:0/3502658262 >> v1:172.21.15.117:6789/0 conn(0x7fb03c15dce0 legacy=0x7fb03c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb03be00640 1 -- 172.21.15.45:0/3502658262 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263847136 0 0) 0x7fb03c15e0c0 con 0x7fb03c15dce0 2024-04-25T10:26:00.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb042400640 1 -- 172.21.15.45:0/3502658262 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c06d4b0 con 0x7fb03c0ad850 2024-04-25T10:26:00.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb042400640 1 --2- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c0adc70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb03401c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2121fa45f6e552e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb03be00640 1 -- 172.21.15.45:0/3502658262 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340b2070 con 0x7fb03c0ad850 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb03be00640 1 -- 172.21.15.45:0/3502658262 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0340a57a0 con 0x7fb03c0ad850 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.235+0000 7fb03be00640 1 -- 172.21.15.45:0/3502658262 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340a5b70 con 0x7fb03c0ad850 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 msgr2=0x7fb03c0adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 --2- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c0adc70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb03401c850 comp rx=0 tx=0).stop 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- 172.21.15.45:0/3502658262 shutdown_connections 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 --2- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c0adc70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 --2- 172.21.15.45:0/3502658262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- 172.21.15.45:0/3502658262 >> 172.21.15.45:0/3502658262 conn(0x7fb03c0b2410 msgr2=0x7fb03c0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- 172.21.15.45:0/3502658262 shutdown_connections 2024-04-25T10:26:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- 172.21.15.45:0/3502658262 wait complete. 2024-04-25T10:26:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 Processor -- start 2024-04-25T10:26:00.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- start start 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 0x7fb03c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0d04a0 con 0x7fb03c15dce0 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0d0320 con 0x7fb03c0ad850 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb044400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c16abc0 con 0x7fb03c06eec0 2024-04-25T10:26:00.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb042400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 0x7fb03c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb041a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb042400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 0x7fb03c0c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41926/0 (socket says 172.21.15.45:41926) 2024-04-25T10:26:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb042400640 1 -- 172.21.15.45:0/3679887341 learned_addr learned my addr 172.21.15.45:0/3679887341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:00.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.239+0000 7fb041000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb042400640 1 -- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 msgr2=0x7fb03c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb042400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c15c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb042400640 1 -- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 msgr2=0x7fb03c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb042400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb042400640 1 -- 172.21.15.45:0/3679887341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0cbca0 con 0x7fb03c15dce0 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb042400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 0x7fb03c0c7a30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb03401af00 tx=0x7fb0340195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340b2050 con 0x7fb03c15dce0 2024-04-25T10:26:00.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0340be070 con 0x7fb03c15dce0 2024-04-25T10:26:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c0c8760 con 0x7fb03c15dce0 2024-04-25T10:26:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340ad850 con 0x7fb03c15dce0 2024-04-25T10:26:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c0c8d20 con 0x7fb03c15dce0 2024-04-25T10:26:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c04ef30 con 0x7fb03c15dce0 2024-04-25T10:26:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0340ad9f0 con 0x7fb03c15dce0 2024-04-25T10:26:00.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb039600640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb01c066f90 0x7fb01c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fb03411f9f0 con 0x7fb03c15dce0 2024-04-25T10:26:00.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.243+0000 7fb041a00640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb01c066f90 0x7fb01c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.247+0000 7fb041a00640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb01c066f90 0x7fb01c069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb030000f70 tx=0x7fb030007240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.247+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0340c7020 con 0x7fb03c15dce0 2024-04-25T10:26:00.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.411+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb03c0af770 con 0x7fb03c15dce0 2024-04-25T10:26:00.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb039600640 1 -- 172.21.15.45:0/3679887341 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16751 (secure 0 0 0) 0x7fb034016560 con 0x7fb03c15dce0 2024-04-25T10:26:00.418 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:00.419 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:43.660339+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:00.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb01c066f90 msgr2=0x7fb01c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb01c066f90 0x7fb01c069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb030000f70 tx=0x7fb030007240 comp rx=0 tx=0).stop 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 msgr2=0x7fb03c0c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 0x7fb03c0c7a30 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb03401af00 tx=0x7fb0340195f0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 shutdown_connections 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb01c066f90 0x7fb01c069450 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb03c15dce0 0x7fb03c0c7a30 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0ad850 0x7fb03c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 --2- 172.21.15.45:0/3679887341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c06eec0 0x7fb03c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 >> 172.21.15.45:0/3679887341 conn(0x7fb03c0b2410 msgr2=0x7fb03c0d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 shutdown_connections 2024-04-25T10:26:00.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.415+0000 7fb044400640 1 -- 172.21.15.45:0/3679887341 wait complete. 2024-04-25T10:26:00.438 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:26:00.438 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:00.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 Processor -- start 2024-04-25T10:26:00.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 -- start start 2024-04-25T10:26:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f874015bc40 0x7f874006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87400cf930 0x7f87400cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b2770 con 0x7f87400cda10 2024-04-25T10:26:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874006c670 con 0x7f87400cf930 2024-04-25T10:26:00.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.575+0000 7f8745600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874006c950 con 0x7f874015bc40 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f87400cf930 0x7f87400cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f8736000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f87400cda10 0x7f87400b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53236/0 (socket says 172.21.15.45:53236) 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f8736000640 1 -- 172.21.15.45:0/3896597518 learned_addr learned my addr 172.21.15.45:0/3896597518 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873ea00640 1 --2- 172.21.15.45:0/3896597518 >> v2:172.21.15.117:3300/0 conn(0x7f874015bc40 0x7f874006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873f400640 1 -- 172.21.15.45:0/3896597518 >> v2:172.21.15.117:3300/0 conn(0x7f874015bc40 msgr2=0x7f874006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873f400640 1 --2- 172.21.15.45:0/3896597518 >> v2:172.21.15.117:3300/0 conn(0x7f874015bc40 0x7f874006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873f400640 1 -- 172.21.15.45:0/3896597518 >> v1:172.21.15.45:6789/0 conn(0x7f87400cda10 legacy=0x7f87400b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873f400640 1 -- 172.21.15.45:0/3896597518 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874006cc30 con 0x7f87400cf930 2024-04-25T10:26:00.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873e000640 1 -- 172.21.15.45:0/3896597518 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825960401 0 0) 0x7f87400b2770 con 0x7f87400cda10 2024-04-25T10:26:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873f400640 1 --2- 172.21.15.45:0/3896597518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87400cf930 0x7f87400cfdc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f87400d0800 tx=0x7f873801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6626dd476be78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873e000640 1 -- 172.21.15.45:0/3896597518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380b2070 con 0x7f87400cf930 2024-04-25T10:26:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873e000640 1 -- 172.21.15.45:0/3896597518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380a5760 con 0x7f87400cf930 2024-04-25T10:26:00.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.579+0000 7f873e000640 1 -- 172.21.15.45:0/3896597518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380a5b30 con 0x7f87400cf930 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- 172.21.15.45:0/3896597518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87400cf930 msgr2=0x7f87400cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 --2- 172.21.15.45:0/3896597518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87400cf930 0x7f87400cfdc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f87400d0800 tx=0x7f873801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- 172.21.15.45:0/3896597518 shutdown_connections 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 --2- 172.21.15.45:0/3896597518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87400cf930 0x7f87400cfdc0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 --2- 172.21.15.45:0/3896597518 >> v2:172.21.15.117:3300/0 conn(0x7f874015bc40 0x7f874006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- 172.21.15.45:0/3896597518 >> 172.21.15.45:0/3896597518 conn(0x7f8740066b70 msgr2=0x7f8740066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- 172.21.15.45:0/3896597518 shutdown_connections 2024-04-25T10:26:00.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- 172.21.15.45:0/3896597518 wait complete. 2024-04-25T10:26:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 Processor -- start 2024-04-25T10:26:00.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- start start 2024-04-25T10:26:00.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87400cda10 0x7f8740152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87400cf930 0x7f874015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 0x7f874015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b3850 con 0x7f874015bc40 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b36d0 con 0x7f87400cda10 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8745600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b39d0 con 0x7f87400cf930 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 0x7f874015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 0x7f874015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41940/0 (socket says 172.21.15.45:41940) 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 -- 172.21.15.45:0/1114812132 learned_addr learned my addr 172.21.15.45:0/1114812132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873ea00640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87400cda10 0x7f8740152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8736000640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87400cf930 0x7f874015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 -- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87400cf930 msgr2=0x7f874015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87400cf930 0x7f874015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 -- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87400cda10 msgr2=0x7f8740152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87400cda10 0x7f8740152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f873f400640 1 -- 172.21.15.45:0/1114812132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400b0f50 con 0x7f874015bc40 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.583+0000 7f8736000640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87400cf930 0x7f874015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:00.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f873f400640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 0x7f874015a650 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f87380194b0 tx=0x7f8738019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87380b2050 con 0x7f874015bc40 2024-04-25T10:26:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380be070 con 0x7f874015bc40 2024-04-25T10:26:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738009450 con 0x7f874015bc40 2024-04-25T10:26:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8740153060 con 0x7f874015bc40 2024-04-25T10:26:00.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87400b14d0 con 0x7f874015bc40 2024-04-25T10:26:00.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874004ef30 con 0x7f874015bc40 2024-04-25T10:26:00.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87380adc20 con 0x7f874015bc40 2024-04-25T10:26:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8737400640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f871c066f60 0x7f871c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f8738120cb0 con 0x7f874015bc40 2024-04-25T10:26:00.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.587+0000 7f873ea00640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f871c066f60 0x7f871c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.591+0000 7f873ea00640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f871c066f60 0x7f871c069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87300079c0 tx=0x7f8730008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.591+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8738016520 con 0x7f874015bc40 2024-04-25T10:26:00.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.787+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87400b2770 con 0x7f874015bc40 2024-04-25T10:26:00.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8737400640 1 -- 172.21.15.45:0/1114812132 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f87380f1cd0 con 0x7f874015bc40 2024-04-25T10:26:00.792 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:00.792 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:26:00.792 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:26:00.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f871c066f60 msgr2=0x7f871c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f871c066f60 0x7f871c069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87300079c0 tx=0x7f8730008040 comp rx=0 tx=0).stop 2024-04-25T10:26:00.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 msgr2=0x7f874015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 0x7f874015a650 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f87380194b0 tx=0x7f8738019640 comp rx=0 tx=0).stop 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 shutdown_connections 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f871c066f60 0x7f871c069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f874015bc40 0x7f874015a650 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87400cf930 0x7f874015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 --2- 172.21.15.45:0/1114812132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87400cda10 0x7f8740152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 >> 172.21.15.45:0/1114812132 conn(0x7f8740066b70 msgr2=0x7f8740066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 shutdown_connections 2024-04-25T10:26:00.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.791+0000 7f8745600640 1 -- 172.21.15.45:0/1114812132 wait complete. 2024-04-25T10:26:00.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.951+0000 7fd3d4a00640 1 Processor -- start 2024-04-25T10:26:00.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.951+0000 7fd3d4a00640 1 -- start start 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3d4a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd3d01550b0 0x7fd3d00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d00cfe70 0x7fd3d015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3d4a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d0071050 con 0x7fd3d015b740 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3d4a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d015d040 con 0x7fd3d01550b0 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00712f0 con 0x7fd3d00cfe70 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd3d01550b0 0x7fd3d00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd3d01550b0 0x7fd3d00cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41944/0 (socket says 172.21.15.45:41944) 2024-04-25T10:26:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 -- 172.21.15.45:0/3224571907 learned_addr learned my addr 172.21.15.45:0/3224571907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 -- 172.21.15.45:0/3224571907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d00cfe70 msgr2=0x7fd3d015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cd400640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d00cfe70 0x7fd3d015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cca00640 1 -- 172.21.15.45:0/3224571907 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640824913 0 0) 0x7fd3d0071050 con 0x7fd3d015b740 2024-04-25T10:26:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d00cfe70 0x7fd3d015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 -- 172.21.15.45:0/3224571907 >> v1:172.21.15.45:6789/0 conn(0x7fd3d015b740 legacy=0x7fd3d006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cde00640 1 -- 172.21.15.45:0/3224571907 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d00715d0 con 0x7fd3d01550b0 2024-04-25T10:26:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.955+0000 7fd3cd400640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d00cfe70 0x7fd3d015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:00.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cde00640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00cf930 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009870 tx=0x7fd3c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc31d80ba7fa09d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cca00640 1 -- 172.21.15.45:0/3224571907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00b0070 con 0x7fd3d01550b0 2024-04-25T10:26:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cca00640 1 -- 172.21.15.45:0/3224571907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c00ab600 con 0x7fd3d01550b0 2024-04-25T10:26:00.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cca00640 1 -- 172.21.15.45:0/3224571907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00ab9d0 con 0x7fd3d01550b0 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3224571907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 msgr2=0x7fd3d00cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00cf930 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009870 tx=0x7fd3c001c560 comp rx=0 tx=0).stop 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3224571907 shutdown_connections 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d00cfe70 0x7fd3d015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3224571907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00cf930 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3224571907 >> 172.21.15.45:0/3224571907 conn(0x7fd3d0066b70 msgr2=0x7fd3d0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3224571907 shutdown_connections 2024-04-25T10:26:00.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3224571907 wait complete. 2024-04-25T10:26:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 Processor -- start 2024-04-25T10:26:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- start start 2024-04-25T10:26:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d00cfe70 0x7fd3d014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d015b740 0x7fd3d00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d015bd10 con 0x7fd3d01550b0 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d015bb90 con 0x7fd3d00cfe70 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3d4a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d015c030 con 0x7fd3d015b740 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41952/0 (socket says 172.21.15.45:41952) 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.959+0000 7fd3cd400640 1 -- 172.21.15.45:0/3618392146 learned_addr learned my addr 172.21.15.45:0/3618392146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cde00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d00cfe70 0x7fd3d014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cd400640 1 -- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d015b740 msgr2=0x7fd3d00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cd400640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d015b740 0x7fd3d00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cd400640 1 -- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d00cfe70 msgr2=0x7fd3d014d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cd400640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d00cfe70 0x7fd3d014d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cd400640 1 -- 172.21.15.45:0/3618392146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d0153cc0 con 0x7fd3d01550b0 2024-04-25T10:26:00.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3cd400640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00b3710 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c7c0 tx=0x7fd3bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc007a30 con 0x7fd3d01550b0 2024-04-25T10:26:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc007bd0 con 0x7fd3d01550b0 2024-04-25T10:26:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0113f0 con 0x7fd3d01550b0 2024-04-25T10:26:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d0153fa0 con 0x7fd3d01550b0 2024-04-25T10:26:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d0154560 con 0x7fd3d01550b0 2024-04-25T10:26:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d004ef30 con 0x7fd3d01550b0 2024-04-25T10:26:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3bc011590 con 0x7fd3d01550b0 2024-04-25T10:26:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3c6000640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3b0066f90 0x7fd3b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.963+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fd3bc0852a0 con 0x7fd3d01550b0 2024-04-25T10:26:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.967+0000 7fd3cde00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3b0066f90 0x7fd3b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.967+0000 7fd3cde00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3b0066f90 0x7fd3b0069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00ad000 tx=0x7fd3c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:00.967+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3bc015020 con 0x7fd3d01550b0 2024-04-25T10:26:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3d0151ed0 con 0x7fd3d01550b0 2024-04-25T10:26:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3c6000640 1 -- 172.21.15.45:0/3618392146 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd3bc056240 con 0x7fd3d01550b0 2024-04-25T10:26:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:01.166 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:25:16.702073+0000","modified":"2024-04-25T10:25:19.723604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/4218207272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4218207272},{"type":"v1","addr":"172.21.15.45:6835","nonce":4218207272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:26:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3b0066f90 msgr2=0x7fd3b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3b0066f90 0x7fd3b0069450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00ad000 tx=0x7fd3c0008040 comp rx=0 tx=0).stop 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 msgr2=0x7fd3d00b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00b3710 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c7c0 tx=0x7fd3bc00cc80 comp rx=0 tx=0).stop 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 shutdown_connections 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3b0066f90 0x7fd3b0069450 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3d015b740 0x7fd3d00b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3d01550b0 0x7fd3d00b3710 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 --2- 172.21.15.45:0/3618392146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3d00cfe70 0x7fd3d014d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.163+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 >> 172.21.15.45:0/3618392146 conn(0x7fd3d0066b70 msgr2=0x7fd3d00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.167+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 shutdown_connections 2024-04-25T10:26:01.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.167+0000 7fd3d4a00640 1 -- 172.21.15.45:0/3618392146 wait complete. 2024-04-25T10:26:01.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:01.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 Processor -- start 2024-04-25T10:26:01.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 -- start start 2024-04-25T10:26:01.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40415df60 0x7fd4040cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040d0180 0x7fd4040d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4040d0b20 con 0x7fd4040cdb40 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd40415e340 con 0x7fd40415df60 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40c200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040ccaa0 con 0x7fd4040d0180 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd409800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd40415df60 0x7fd4040cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40a200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd4040cdb40 0x7fd4040cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53246/0 (socket says 172.21.15.45:53246) 2024-04-25T10:26:01.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd40a200640 1 -- 172.21.15.45:0/3289444480 learned_addr learned my addr 172.21.15.45:0/3289444480 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd408e00640 1 --2- 172.21.15.45:0/3289444480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040d0180 0x7fd4040d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd409800640 1 -- 172.21.15.45:0/3289444480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040d0180 msgr2=0x7fd4040d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd409800640 1 --2- 172.21.15.45:0/3289444480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040d0180 0x7fd4040d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd409800640 1 -- 172.21.15.45:0/3289444480 >> v1:172.21.15.45:6789/0 conn(0x7fd4040cdb40 legacy=0x7fd4040cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd409800640 1 -- 172.21.15.45:0/3289444480 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4040ccd80 con 0x7fd40415df60 2024-04-25T10:26:01.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.327+0000 7fd403e00640 1 -- 172.21.15.45:0/3289444480 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011701215 0 0) 0x7fd4040d0b20 con 0x7fd4040cdb40 2024-04-25T10:26:01.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd409800640 1 --2- 172.21.15.45:0/3289444480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd4040cfc40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80097c0 tx=0x7fd3f801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9be4facd3cbfed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd403e00640 1 -- 172.21.15.45:0/3289444480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80b0070 con 0x7fd40415df60 2024-04-25T10:26:01.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd403e00640 1 -- 172.21.15.45:0/3289444480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f80ab600 con 0x7fd40415df60 2024-04-25T10:26:01.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd403e00640 1 -- 172.21.15.45:0/3289444480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80ab9d0 con 0x7fd40415df60 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 -- 172.21.15.45:0/3289444480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 msgr2=0x7fd4040cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 --2- 172.21.15.45:0/3289444480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd4040cfc40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80097c0 tx=0x7fd3f801c410 comp rx=0 tx=0).stop 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 -- 172.21.15.45:0/3289444480 shutdown_connections 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 --2- 172.21.15.45:0/3289444480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040d0180 0x7fd4040d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 --2- 172.21.15.45:0/3289444480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd4040cfc40 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 -- 172.21.15.45:0/3289444480 >> 172.21.15.45:0/3289444480 conn(0x7fd4040d1910 msgr2=0x7fd4040d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 -- 172.21.15.45:0/3289444480 shutdown_connections 2024-04-25T10:26:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.331+0000 7fd40c200640 1 -- 172.21.15.45:0/3289444480 wait complete. 2024-04-25T10:26:01.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 Processor -- start 2024-04-25T10:26:01.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 -- start start 2024-04-25T10:26:01.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040cdb40 0x7fd404153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4040d0180 0x7fd404155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd40415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040d0ff0 con 0x7fd40415df60 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040d0e70 con 0x7fd4040d0180 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040cead0 con 0x7fd4040cdb40 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd40415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd40415bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41982/0 (socket says 172.21.15.45:41982) 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd409800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040cdb40 0x7fd404153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 -- 172.21.15.45:0/4226616389 learned_addr learned my addr 172.21.15.45:0/4226616389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd408e00640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4040d0180 0x7fd404155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 -- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040cdb40 msgr2=0x7fd404153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040cdb40 0x7fd404153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 -- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4040d0180 msgr2=0x7fd404155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4040d0180 0x7fd404155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 -- 172.21.15.45:0/4226616389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4040cc3e0 con 0x7fd40415df60 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd408e00640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4040d0180 0x7fd404155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40a200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd40415bd10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c7a0 tx=0x7fd3fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc018070 con 0x7fd40415df60 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc00a640 con 0x7fd40415df60 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00aab0 con 0x7fd40415df60 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40406cbe0 con 0x7fd40415df60 2024-04-25T10:26:01.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.335+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041554a0 con 0x7fd40415df60 2024-04-25T10:26:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.339+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40404ef30 con 0x7fd40415df60 2024-04-25T10:26:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.339+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3fc01b070 con 0x7fd40415df60 2024-04-25T10:26:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.339+0000 7fd401600640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3e4066f90 0x7fd3e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.339+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fd3fc0845e0 con 0x7fd40415df60 2024-04-25T10:26:01.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.339+0000 7fd409800640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3e4066f90 0x7fd3e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.339+0000 7fd409800640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3e4066f90 0x7fd3e4069450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80ad000 tx=0x7fd3f80a4f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.343+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3fc054f30 con 0x7fd40415df60 2024-04-25T10:26:01.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.503+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd404155fa0 con 0x7fd40415df60 2024-04-25T10:26:01.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd401600640 1 -- 172.21.15.45:0/4226616389 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16751 (secure 0 0 0) 0x7fd3fc058fd0 con 0x7fd40415df60 2024-04-25T10:26:01.509 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:01.509 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:25:43.660339+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:25:14.488727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T10:25:15.698603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.e","mappings":[{"from":5,"to":0}]},{"pgid":"14.1a","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3e4066f90 msgr2=0x7fd3e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3e4066f90 0x7fd3e4069450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80ad000 tx=0x7fd3f80a4f90 comp rx=0 tx=0).stop 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 msgr2=0x7fd40415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd40415bd10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c7a0 tx=0x7fd3fc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 shutdown_connections 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3e4066f90 0x7fd3e4069450 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd40415df60 0x7fd40415bd10 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4040d0180 0x7fd404155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 --2- 172.21.15.45:0/4226616389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4040cdb40 0x7fd404153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 >> 172.21.15.45:0/4226616389 conn(0x7fd4040d1910 msgr2=0x7fd40406e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 shutdown_connections 2024-04-25T10:26:01.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.507+0000 7fd40c200640 1 -- 172.21.15.45:0/4226616389 wait complete. 2024-04-25T10:26:01.530 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:26:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.667+0000 7fe8bc200640 1 Processor -- start 2024-04-25T10:26:01.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.667+0000 7fe8bc200640 1 -- start start 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8bc200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8b40cfd90 0x7fe8b40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8bc200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 0x7fe8b40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8bc200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b415f3b0 con 0x7fe8b40b0f70 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8bc200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b415ef20 con 0x7fe8b40cfd90 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8bc200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40ca810 con 0x7fe8b40cd110 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe8b40b0f70 0x7fe8b40b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53250/0 (socket says 172.21.15.45:53250) 2024-04-25T10:26:01.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 -- 172.21.15.45:0/1240338662 learned_addr learned my addr 172.21.15.45:0/1240338662 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204499000 0 0) 0x7fe8b415f3b0 con 0x7fe8b40b0f70 2024-04-25T10:26:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe89c003530 con 0x7fe8b40b0f70 2024-04-25T10:26:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/1240338662 >> v2:172.21.15.117:3300/0 conn(0x7fe8b40cfd90 0x7fe8b40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8ba200640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 0x7fe8b40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108198790 0 0) 0x7fe89c003530 con 0x7fe8b40b0f70 2024-04-25T10:26:01.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 msgr2=0x7fe8b40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 0x7fe8b40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 >> v2:172.21.15.117:3300/0 conn(0x7fe8b40cfd90 msgr2=0x7fe8b40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 --2- 172.21.15.45:0/1240338662 >> v2:172.21.15.117:3300/0 conn(0x7fe8b40cfd90 0x7fe8b40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b415e5f0 con 0x7fe8b40b0f70 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fe8a8001680 con 0x7fe8b40b0f70 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 >> v1:172.21.15.45:6789/0 conn(0x7fe8b40b0f70 legacy=0x7fe8b40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe89c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe89c004810 0x7fe89c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 0x7fe89c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b415ef20 con 0x7fe89c0039c0 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40ca810 con 0x7fe89c004810 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c00b9d0 con 0x7fe8b40cd110 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe89c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe89c004810 0x7fe89c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8ba200640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 0x7fe89c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 -- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 msgr2=0x7fe89c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b40cd110 0x7fe89c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 -- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe89c004810 msgr2=0x7fe89c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe89c004810 0x7fe89c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b9800640 1 -- 172.21.15.45:0/1240338662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c003530 con 0x7fe89c0039c0 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.671+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe89c004810 0x7fe89c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8b9800640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe89c003de0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c240 tx=0x7fe8a801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccf16f2ec9bac81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8007d40 con 0x7fe89c0039c0 2024-04-25T10:26:01.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a80a9020 con 0x7fe89c0039c0 2024-04-25T10:26:01.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8b3e00640 1 -- 172.21.15.45:0/1240338662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005b00 con 0x7fe89c0039c0 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 -- 172.21.15.45:0/1240338662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 msgr2=0x7fe89c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 --2- 172.21.15.45:0/1240338662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe89c003de0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c240 tx=0x7fe8a801cca0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8ba200640 1 -- 172.21.15.45:0/1240338662 reap_dead start 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 -- 172.21.15.45:0/1240338662 shutdown_connections 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 -- 172.21.15.45:0/1240338662 >> 172.21.15.45:0/1240338662 conn(0x7fe8b40b1d50 msgr2=0x7fe8b40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 -- 172.21.15.45:0/1240338662 shutdown_connections 2024-04-25T10:26:01.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 -- 172.21.15.45:0/1240338662 wait complete. 2024-04-25T10:26:01.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 Processor -- start 2024-04-25T10:26:01.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.675+0000 7fe8bc200640 1 -- start start 2024-04-25T10:26:01.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe8b40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b41f8000 0x7fe8b40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b40b3460 0x7fe8b4157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40cc3b0 con 0x7fe89c0039c0 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40cc230 con 0x7fe8b40b3460 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40cc530 con 0x7fe8b41f8000 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe8b40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe8b40b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41996/0 (socket says 172.21.15.45:41996) 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 -- 172.21.15.45:0/2238170094 learned_addr learned my addr 172.21.15.45:0/2238170094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b41f8000 0x7fe8b40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8ba200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b40b3460 0x7fe8b4157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 -- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b41f8000 msgr2=0x7fe8b40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b41f8000 0x7fe8b40b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 -- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b40b3460 msgr2=0x7fe8b4157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b40b3460 0x7fe8b4157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 -- 172.21.15.45:0/2238170094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40b3f70 con 0x7fe89c0039c0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b41f8000 0x7fe8b40b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b9800640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe8b40b4d30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c240 tx=0x7fe89c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80049f0 con 0x7fe89c0039c0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a80b2070 con 0x7fe89c0039c0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8004d90 con 0x7fe89c0039c0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b40b3c90 con 0x7fe89c0039c0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b40c78f0 con 0x7fe89c0039c0 2024-04-25T10:26:01.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.679+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b404ef30 con 0x7fe89c0039c0 2024-04-25T10:26:01.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.683+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8a80a56a0 con 0x7fe89c0039c0 2024-04-25T10:26:01.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.683+0000 7fe8b1600640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe894066f90 0x7fe894069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:01.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.683+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fe8a811e300 con 0x7fe89c0039c0 2024-04-25T10:26:01.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.683+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe894066f90 0x7fe894069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:01.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.683+0000 7fe8b8e00640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe894066f90 0x7fe894069450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe8b41f8400 tx=0x7fe8a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:01.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.687+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8a80169b0 con 0x7fe89c0039c0 2024-04-25T10:26:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.883+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe8b40cda20 con 0x7fe89c0039c0 2024-04-25T10:26:01.887 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:26:01.883+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:26:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.887+0000 7fe8b1600640 1 -- 172.21.15.45:0/2238170094 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8a80ef220 con 0x7fe89c0039c0 2024-04-25T10:26:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:26:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe894066f90 msgr2=0x7fe894069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe894066f90 0x7fe894069450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe8b41f8400 tx=0x7fe8a0008040 comp rx=0 tx=0).stop 2024-04-25T10:26:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 msgr2=0x7fe8b40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe8b40b4d30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c240 tx=0x7fe89c005840 comp rx=0 tx=0).stop 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 shutdown_connections 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe894066f90 0x7fe894069450 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b40b3460 0x7fe8b4157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b41f8000 0x7fe8b40b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 --2- 172.21.15.45:0/2238170094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe89c0039c0 0x7fe8b40b4d30 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 >> 172.21.15.45:0/2238170094 conn(0x7fe8b40b1d50 msgr2=0x7fe8b406dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 shutdown_connections 2024-04-25T10:26:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:01.891+0000 7fe8bc200640 1 -- 172.21.15.45:0/2238170094 wait complete. 2024-04-25T10:26:01.923 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:26:01.973 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:26:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.063+0000 7fa434e00640 1 Processor -- start 2024-04-25T10:26:02.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.063+0000 7fa434e00640 1 -- start start 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa434e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4300cd040 0x7fa4300cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa434e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cefa0 0x7fa4300cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa434e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4300b2e80 con 0x7fa43015b870 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa434e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4300b3a90 con 0x7fa4300cd040 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa434e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300b3160 con 0x7fa4300cefa0 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4300cd040 0x7fa4300cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4300cd040 0x7fa4300cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42012/0 (socket says 172.21.15.45:42012) 2024-04-25T10:26:02.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 -- 172.21.15.45:0/1729603076 learned_addr learned my addr 172.21.15.45:0/1729603076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 -- 172.21.15.45:0/1729603076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cefa0 msgr2=0x7fa4300cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa42e800640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cefa0 0x7fa4300cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cefa0 0x7fa4300cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 -- 172.21.15.45:0/1729603076 >> v1:172.21.15.117:6789/0 conn(0x7fa43015b870 legacy=0x7fa43015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa427e00640 1 -- 172.21.15.45:0/1729603076 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4300b3440 con 0x7fa4300cd040 2024-04-25T10:26:02.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.067+0000 7fa42e800640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cefa0 0x7fa4300cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa427e00640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cd040 0x7fa4300cd4d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa430067380 tx=0x7fa41801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4522f4015ef858cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa42d400640 1 -- 172.21.15.45:0/1729603076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180b0070 con 0x7fa4300cd040 2024-04-25T10:26:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa42d400640 1 -- 172.21.15.45:0/1729603076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4180ab600 con 0x7fa4300cd040 2024-04-25T10:26:02.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa42d400640 1 -- 172.21.15.45:0/1729603076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180ab9d0 con 0x7fa4300cd040 2024-04-25T10:26:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 -- 172.21.15.45:0/1729603076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cd040 msgr2=0x7fa4300cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cd040 0x7fa4300cd4d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa430067380 tx=0x7fa41801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 -- 172.21.15.45:0/1729603076 shutdown_connections 2024-04-25T10:26:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cefa0 0x7fa4300cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 --2- 172.21.15.45:0/1729603076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cd040 0x7fa4300cd4d0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 -- 172.21.15.45:0/1729603076 >> 172.21.15.45:0/1729603076 conn(0x7fa4300b21c0 msgr2=0x7fa4300669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 -- 172.21.15.45:0/1729603076 shutdown_connections 2024-04-25T10:26:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 -- 172.21.15.45:0/1729603076 wait complete. 2024-04-25T10:26:02.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 Processor -- start 2024-04-25T10:26:02.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.071+0000 7fa434e00640 1 -- start start 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cd040 0x7fa43006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 0x7fa43015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43015b870 0x7fa43014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300b48b0 con 0x7fa4300cefa0 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300b4730 con 0x7fa43015b870 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300b4a30 con 0x7fa4300cd040 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 0x7fa43015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 0x7fa43015c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42020/0 (socket says 172.21.15.45:42020) 2024-04-25T10:26:02.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 -- 172.21.15.45:0/634954944 learned_addr learned my addr 172.21.15.45:0/634954944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa42e800640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43015b870 0x7fa43014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa42de00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cd040 0x7fa43006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 -- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cd040 msgr2=0x7fa43006e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cd040 0x7fa43006e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 -- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43015b870 msgr2=0x7fa43014a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43015b870 0x7fa43014a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 -- 172.21.15.45:0/634954944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43015cce0 con 0x7fa4300cefa0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa42de00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cd040 0x7fa43006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa427e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 0x7fa43015c570 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa41801cd80 tx=0x7fa41801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180b0040 con 0x7fa4300cefa0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4180bc070 con 0x7fa4300cefa0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4180b6880 con 0x7fa4300cefa0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43015cfc0 con 0x7fa4300cefa0 2024-04-25T10:26:02.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43015d580 con 0x7fa4300cefa0 2024-04-25T10:26:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43004ef30 con 0x7fa4300cefa0 2024-04-25T10:26:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.075+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4180b6a20 con 0x7fa4300cefa0 2024-04-25T10:26:02.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.079+0000 7fa426000640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa40c066f60 0x7fa40c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.079+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fa41811f560 con 0x7fa4300cefa0 2024-04-25T10:26:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.079+0000 7fa42de00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa40c066f60 0x7fa40c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:02.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.079+0000 7fa42de00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa40c066f60 0x7fa40c069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa430166080 tx=0x7fa420009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:02.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.083+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4180160d0 con 0x7fa4300cefa0 2024-04-25T10:26:02.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.279+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa430156900 con 0x7fa4300cefa0 2024-04-25T10:26:02.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.887+0000 7fa426000640 1 -- 172.21.15.45:0/634954944 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4180f0550 con 0x7fa4300cefa0 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa40c066f60 msgr2=0x7fa40c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa40c066f60 0x7fa40c069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa430166080 tx=0x7fa420009040 comp rx=0 tx=0).stop 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 msgr2=0x7fa43015c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 0x7fa43015c570 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa41801cd80 tx=0x7fa41801ad20 comp rx=0 tx=0).stop 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 shutdown_connections 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa40c066f60 0x7fa40c069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43015b870 0x7fa43014a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4300cefa0 0x7fa43015c570 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 --2- 172.21.15.45:0/634954944 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4300cd040 0x7fa43006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 >> 172.21.15.45:0/634954944 conn(0x7fa4300b21c0 msgr2=0x7fa43014ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 shutdown_connections 2024-04-25T10:26:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:02.891+0000 7fa434e00640 1 -- 172.21.15.45:0/634954944 wait complete. 2024-04-25T10:26:02.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:26:03.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.059+0000 7fce9ec00640 1 Processor -- start 2024-04-25T10:26:03.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.059+0000 7fce9ec00640 1 -- start start 2024-04-25T10:26:03.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce9ec00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fce980cf040 0x7fce980cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce9ec00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce98071f80 con 0x7fce9815bc00 2024-04-25T10:26:03.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce9ec00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9815b120 con 0x7fce98071ab0 2024-04-25T10:26:03.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce9ec00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce98072260 con 0x7fce980cf040 2024-04-25T10:26:03.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fce9815bc00 0x7fce980713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53258/0 (socket says 172.21.15.45:53258) 2024-04-25T10:26:03.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 -- 172.21.15.45:0/829300980 learned_addr learned my addr 172.21.15.45:0/829300980 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97e00640 1 --2- 172.21.15.45:0/829300980 >> v2:172.21.15.117:3300/0 conn(0x7fce980cf040 0x7fce980cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107846910 0 0) 0x7fce98071f80 con 0x7fce9815bc00 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce7c005740 con 0x7fce9815bc00 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3466667643 0 0) 0x7fce7c005740 con 0x7fce9815bc00 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 >> v2:172.21.15.117:3300/0 conn(0x7fce980cf040 msgr2=0x7fce980cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 --2- 172.21.15.45:0/829300980 >> v2:172.21.15.117:3300/0 conn(0x7fce980cf040 0x7fce980cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 >> v1:172.21.15.117:6789/0 conn(0x7fce98071ab0 legacy=0x7fce980ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9815b400 con 0x7fce9815bc00 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fce80001680 con 0x7fce9815bc00 2024-04-25T10:26:03.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 >> v1:172.21.15.45:6789/0 conn(0x7fce9815bc00 legacy=0x7fce980713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c005bd0 0x7fce7c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce7c006a20 0x7fce7c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce7c0073e0 0x7fce7c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c00e250 con 0x7fce7c006a20 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c00e400 con 0x7fce7c0073e0 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c00e600 con 0x7fce7c005bd0 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce7c006a20 0x7fce7c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce9cc00640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce7c0073e0 0x7fce7c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce97e00640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c005bd0 0x7fce7c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 -- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c005bd0 msgr2=0x7fce7c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:03.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c005bd0 0x7fce7c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 -- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce7c0073e0 msgr2=0x7fce7c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce7c0073e0 0x7fce7c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce8f400640 1 -- 172.21.15.45:0/829300980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c005740 con 0x7fce7c006a20 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.063+0000 7fce9cc00640 1 -- 172.21.15.45:0/829300980 reap_dead start 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce8f400640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce7c006a20 0x7fce7c006ea0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fce98070530 tx=0x7fce8001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2888546d7ac4dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80007d00 con 0x7fce7c006a20 2024-04-25T10:26:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80005b00 con 0x7fce7c006a20 2024-04-25T10:26:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce97400640 1 -- 172.21.15.45:0/829300980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80007070 con 0x7fce7c006a20 2024-04-25T10:26:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 -- 172.21.15.45:0/829300980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce7c006a20 msgr2=0x7fce7c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce7c006a20 0x7fce7c006ea0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fce98070530 tx=0x7fce8001f370 comp rx=0 tx=0).stop 2024-04-25T10:26:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 -- 172.21.15.45:0/829300980 shutdown_connections 2024-04-25T10:26:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 --2- 172.21.15.45:0/829300980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce7c006a20 0x7fce7c006ea0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 -- 172.21.15.45:0/829300980 >> 172.21.15.45:0/829300980 conn(0x7fce98066b70 msgr2=0x7fce98066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 -- 172.21.15.45:0/829300980 shutdown_connections 2024-04-25T10:26:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 -- 172.21.15.45:0/829300980 wait complete. 2024-04-25T10:26:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 Processor -- start 2024-04-25T10:26:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.067+0000 7fce9ec00640 1 -- start start 2024-04-25T10:26:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c006a20 0x7fce98144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce98146f20 0x7fce98145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 0x7fce98143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9815aec0 con 0x7fce98143930 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce980b3700 con 0x7fce98146f20 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce980cf420 con 0x7fce7c006a20 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 0x7fce98143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 0x7fce98143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42036/0 (socket says 172.21.15.45:42036) 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 -- 172.21.15.45:0/883754379 learned_addr learned my addr 172.21.15.45:0/883754379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce97e00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c006a20 0x7fce98144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce8f400640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce98146f20 0x7fce98145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 -- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c006a20 msgr2=0x7fce98144410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c006a20 0x7fce98144410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 -- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce98146f20 msgr2=0x7fce98145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce98146f20 0x7fce98145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 -- 172.21.15.45:0/883754379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce98144a90 con 0x7fce98143930 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce8f400640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce98146f20 0x7fce98145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9cc00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 0x7fce98143d90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fce98067980 tx=0x7fce9000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90019070 con 0x7fce98143930 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce90006d70 con 0x7fce98143930 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90003ba0 con 0x7fce98143930 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce98144d70 con 0x7fce98143930 2024-04-25T10:26:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce98145330 con 0x7fce98143930 2024-04-25T10:26:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce98145820 con 0x7fce98143930 2024-04-25T10:26:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.071+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce9001c070 con 0x7fce98143930 2024-04-25T10:26:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.075+0000 7fce94c00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce74066f90 0x7fce74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.075+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5882+0+0 (secure 0 0 0) 0x7fce90084610 con 0x7fce98143930 2024-04-25T10:26:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.075+0000 7fce97e00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce74066f90 0x7fce74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:03.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.075+0000 7fce97e00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce74066f90 0x7fce74069450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fce9806a000 tx=0x7fce88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:03.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.079+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce90054f10 con 0x7fce98143930 2024-04-25T10:26:03.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.243+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce980b2a10 con 0x7fce98143930 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce94c00640 1 -- 172.21.15.45:0/883754379 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce90058fb0 con 0x7fce98143930 2024-04-25T10:26:03.896 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:26:03.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce74066f90 msgr2=0x7fce74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce74066f90 0x7fce74069450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fce9806a000 tx=0x7fce88008040 comp rx=0 tx=0).stop 2024-04-25T10:26:03.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 msgr2=0x7fce98143d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 0x7fce98143d90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fce98067980 tx=0x7fce9000e9e0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 shutdown_connections 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fce74066f90 0x7fce74069450 secure :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fce9806a000 tx=0x7fce88008040 comp rx=0 tx=0).stop 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce98143930 0x7fce98143d90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce98146f20 0x7fce98145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 --2- 172.21.15.45:0/883754379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce7c006a20 0x7fce98144410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.895+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 >> 172.21.15.45:0/883754379 conn(0x7fce98066b70 msgr2=0x7fce980cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.899+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 shutdown_connections 2024-04-25T10:26:03.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:03.899+0000 7fce9ec00640 1 -- 172.21.15.45:0/883754379 wait complete. 2024-04-25T10:26:03.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:26:04.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 Processor -- start 2024-04-25T10:26:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 -- start start 2024-04-25T10:26:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 0x7fb06c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb06c15bc00 0x7fb06c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c071f80 con 0x7fb06c0713a0 2024-04-25T10:26:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06c072260 con 0x7fb06c15bc00 2024-04-25T10:26:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb074200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0b3700 con 0x7fb06c0cf040 2024-04-25T10:26:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 0x7fb06c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb070e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb06c15bc00 0x7fb06c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 0x7fb06c0cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37116/0 (socket says 172.21.15.45:37116) 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 -- 172.21.15.45:0/804801690 learned_addr learned my addr 172.21.15.45:0/804801690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb072200640 1 --1- 172.21.15.45:0/804801690 >> v1:172.21.15.117:6789/0 conn(0x7fb06c0713a0 0x7fb06c0ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:51930/0 (socket says 172.21.15.45:51930) 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 -- 172.21.15.45:0/804801690 >> v2:172.21.15.117:3300/0 conn(0x7fb06c15bc00 msgr2=0x7fb06c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 --2- 172.21.15.45:0/804801690 >> v2:172.21.15.117:3300/0 conn(0x7fb06c15bc00 0x7fb06c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 -- 172.21.15.45:0/804801690 >> v1:172.21.15.117:6789/0 conn(0x7fb06c0713a0 legacy=0x7fb06c0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb071800640 1 -- 172.21.15.45:0/804801690 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c15b120 con 0x7fb06c0cf040 2024-04-25T10:26:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.071+0000 7fb06be00640 1 -- 172.21.15.45:0/804801690 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713497500 0 0) 0x7fb06c071f80 con 0x7fb06c0713a0 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb071800640 1 --2- 172.21.15.45:0/804801690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 0x7fb06c0cf9e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb060009870 tx=0x7fb06001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3815f29171cd2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb06be00640 1 -- 172.21.15.45:0/804801690 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600ab650 con 0x7fb06c0cf040 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb06be00640 1 -- 172.21.15.45:0/804801690 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0600b0070 con 0x7fb06c0cf040 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb06be00640 1 -- 172.21.15.45:0/804801690 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600aba10 con 0x7fb06c0cf040 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 -- 172.21.15.45:0/804801690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 msgr2=0x7fb06c0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 --2- 172.21.15.45:0/804801690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 0x7fb06c0cf9e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb060009870 tx=0x7fb06001c560 comp rx=0 tx=0).stop 2024-04-25T10:26:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 -- 172.21.15.45:0/804801690 shutdown_connections 2024-04-25T10:26:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 --2- 172.21.15.45:0/804801690 >> v2:172.21.15.117:3300/0 conn(0x7fb06c15bc00 0x7fb06c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 --2- 172.21.15.45:0/804801690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0cf040 0x7fb06c0cf9e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 -- 172.21.15.45:0/804801690 >> 172.21.15.45:0/804801690 conn(0x7fb06c066b70 msgr2=0x7fb06c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 -- 172.21.15.45:0/804801690 shutdown_connections 2024-04-25T10:26:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.075+0000 7fb074200640 1 -- 172.21.15.45:0/804801690 wait complete. 2024-04-25T10:26:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 Processor -- start 2024-04-25T10:26:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 -- start start 2024-04-25T10:26:04.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0713a0 0x7fb06c158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 0x7fb06c15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb06c15bc00 0x7fb06c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0cf5a0 con 0x7fb06c0cf040 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0cf420 con 0x7fb06c15bc00 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c15e500 con 0x7fb06c0713a0 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 0x7fb06c15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 0x7fb06c15ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42048/0 (socket says 172.21.15.45:42048) 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 -- 172.21.15.45:0/2264385056 learned_addr learned my addr 172.21.15.45:0/2264385056 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb071800640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0713a0 0x7fb06c158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb072200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb06c15bc00 0x7fb06c15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 -- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0713a0 msgr2=0x7fb06c158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0713a0 0x7fb06c158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 -- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb06c15bc00 msgr2=0x7fb06c15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb06c15bc00 0x7fb06c15d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 -- 172.21.15.45:0/2264385056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c159ee0 con 0x7fb06c0cf040 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb070e00640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 0x7fb06c15ce60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb05800e930 tx=0x7fb05800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058017070 con 0x7fb06c0cf040 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05800ca00 con 0x7fb06c0cf040 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c1555d0 con 0x7fb06c0cf040 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800cdb0 con 0x7fb06c0cf040 2024-04-25T10:26:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.079+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c155b90 con 0x7fb06c0cf040 2024-04-25T10:26:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.083+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06c04ef30 con 0x7fb06c0cf040 2024-04-25T10:26:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.083+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb05801a070 con 0x7fb06c0cf040 2024-04-25T10:26:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.083+0000 7fb069600640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb04c066f60 0x7fb04c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.083+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fb058085200 con 0x7fb06c0cf040 2024-04-25T10:26:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.083+0000 7fb071800640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb04c066f60 0x7fb04c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.083+0000 7fb071800640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb04c066f60 0x7fb04c069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb0600af000 tx=0x7fb0600a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.087+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb058055dd0 con 0x7fb06c0cf040 2024-04-25T10:26:04.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.251+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb06c0b3e00 con 0x7fb06c0cf040 2024-04-25T10:26:04.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.895+0000 7fb069600640 1 -- 172.21.15.45:0/2264385056 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb058059e70 con 0x7fb06c0cf040 2024-04-25T10:26:04.898 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:26:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb04c066f60 msgr2=0x7fb04c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb04c066f60 0x7fb04c069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb0600af000 tx=0x7fb0600a7000 comp rx=0 tx=0).stop 2024-04-25T10:26:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 msgr2=0x7fb06c15ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:04.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 0x7fb06c15ce60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb05800e930 tx=0x7fb05800edf0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 shutdown_connections 2024-04-25T10:26:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb04c066f60 0x7fb04c069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb06c15bc00 0x7fb06c15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb06c0cf040 0x7fb06c15ce60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 --2- 172.21.15.45:0/2264385056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb06c0713a0 0x7fb06c158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:04.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 >> 172.21.15.45:0/2264385056 conn(0x7fb06c066b70 msgr2=0x7fb06c0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:04.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 shutdown_connections 2024-04-25T10:26:04.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:04.899+0000 7fb074200640 1 -- 172.21.15.45:0/2264385056 wait complete. 2024-04-25T10:26:04.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:26:05.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 Processor -- start 2024-04-25T10:26:05.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 -- start start 2024-04-25T10:26:05.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 0x7f87a015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 0x7f87a0072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a00ad840 con 0x7f87a00cfb50 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a015bb90 con 0x7f87a00b3480 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f87a5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015e710 con 0x7f87a0071bc0 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 0x7f87a0072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 0x7f87a015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879e000640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f87a00cfb50 0x7f87a0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:46800/0 (socket says 172.21.15.45:46800) 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 0x7f87a015e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51998/0 (socket says 172.21.15.45:51998) 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879e000640 1 -- 172.21.15.45:0/1561799752 learned_addr learned my addr 172.21.15.45:0/1561799752 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:05.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 0x7f87a0072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:51564/0 (socket says 172.21.15.45:51564) 2024-04-25T10:26:05.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879f400640 1 -- 172.21.15.45:0/1561799752 >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 msgr2=0x7f87a015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:05.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879f400640 1 --2- 172.21.15.45:0/1561799752 >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 0x7f87a015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879f400640 1 -- 172.21.15.45:0/1561799752 >> v1:172.21.15.117:6789/0 conn(0x7f87a00cfb50 legacy=0x7f87a0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879d600640 1 -- 172.21.15.45:0/1561799752 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983432438 0 0) 0x7f87a00ad840 con 0x7f87a00cfb50 2024-04-25T10:26:05.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879f400640 1 -- 172.21.15.45:0/1561799752 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a015e9f0 con 0x7f87a0071bc0 2024-04-25T10:26:05.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.083+0000 7f879ea00640 1 --2- 172.21.15.45:0/1561799752 >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 0x7f87a015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:05.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f879f400640 1 --2- 172.21.15.45:0/1561799752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 0x7f87a0072050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8798009d90 tx=0x7f879801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfde1c025e8cd658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f879d600640 1 -- 172.21.15.45:0/1561799752 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980b2070 con 0x7f87a0071bc0 2024-04-25T10:26:05.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f879d600640 1 -- 172.21.15.45:0/1561799752 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87980a5830 con 0x7f87a0071bc0 2024-04-25T10:26:05.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f879d600640 1 -- 172.21.15.45:0/1561799752 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980a5c00 con 0x7f87a0071bc0 2024-04-25T10:26:05.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 -- 172.21.15.45:0/1561799752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 msgr2=0x7f87a0072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 --2- 172.21.15.45:0/1561799752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 0x7f87a0072050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8798009d90 tx=0x7f879801c970 comp rx=0 tx=0).stop 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 -- 172.21.15.45:0/1561799752 shutdown_connections 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 --2- 172.21.15.45:0/1561799752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a0071bc0 0x7f87a0072050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 --2- 172.21.15.45:0/1561799752 >> v2:172.21.15.117:3300/0 conn(0x7f87a00b3480 0x7f87a015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 -- 172.21.15.45:0/1561799752 >> 172.21.15.45:0/1561799752 conn(0x7f87a00669a0 msgr2=0x7f87a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 -- 172.21.15.45:0/1561799752 shutdown_connections 2024-04-25T10:26:05.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.087+0000 7f87a5800640 1 -- 172.21.15.45:0/1561799752 wait complete. 2024-04-25T10:26:05.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 Processor -- start 2024-04-25T10:26:05.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- start start 2024-04-25T10:26:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 0x7f87a01517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a00b3480 0x7f87a015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a00cfb50 0x7f87a014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00ae6b0 con 0x7f87a0071bc0 2024-04-25T10:26:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00ae530 con 0x7f87a00cfb50 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00ae830 con 0x7f87a00b3480 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 0x7f87a01517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 0x7f87a01517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48014/0 (socket says 172.21.15.45:48014) 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 -- 172.21.15.45:0/4253448197 learned_addr learned my addr 172.21.15.45:0/4253448197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879f400640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a00cfb50 0x7f87a014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879e000640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a00b3480 0x7f87a015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 -- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a00b3480 msgr2=0x7f87a015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a00b3480 0x7f87a015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 -- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a00cfb50 msgr2=0x7f87a014c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a00cfb50 0x7f87a014c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 -- 172.21.15.45:0/4253448197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0151e40 con 0x7f87a0071bc0 2024-04-25T10:26:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879e000640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a00b3480 0x7f87a015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f879ea00640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 0x7f87a01517d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f879000ec10 tx=0x7f879000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790017070 con 0x7f87a0071bc0 2024-04-25T10:26:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879000cc30 con 0x7f87a0071bc0 2024-04-25T10:26:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790013470 con 0x7f87a0071bc0 2024-04-25T10:26:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a0152120 con 0x7f87a0071bc0 2024-04-25T10:26:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a01526e0 con 0x7f87a0071bc0 2024-04-25T10:26:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.091+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a004ef30 con 0x7f87a0071bc0 2024-04-25T10:26:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.095+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f879001a070 con 0x7f87a0071bc0 2024-04-25T10:26:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.095+0000 7f8796a00640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8780066f60 0x7f8780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.095+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8790084fe0 con 0x7f87a0071bc0 2024-04-25T10:26:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.095+0000 7f879e000640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8780066f60 0x7f8780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.095+0000 7f879e000640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8780066f60 0x7f8780069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87a01627e0 tx=0x7f878c007dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.099+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8790055cd0 con 0x7f87a0071bc0 2024-04-25T10:26:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.259+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f87a0071400 con 0x7f87a0071bc0 2024-04-25T10:26:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.263+0000 7f8796a00640 1 -- 172.21.15.45:0/4253448197 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8790059d70 con 0x7f87a0071bc0 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8780066f60 msgr2=0x7f8780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8780066f60 0x7f8780069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87a01627e0 tx=0x7f878c007dd0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 msgr2=0x7f87a01517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 0x7f87a01517d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f879000ec10 tx=0x7f879000c520 comp rx=0 tx=0).stop 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 shutdown_connections 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8780066f60 0x7f8780069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87a00cfb50 0x7f87a014c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87a00b3480 0x7f87a015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 --2- 172.21.15.45:0/4253448197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87a0071bc0 0x7f87a01517d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 >> 172.21.15.45:0/4253448197 conn(0x7f87a00669a0 msgr2=0x7f87a0158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 shutdown_connections 2024-04-25T10:26:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.267+0000 7f87a5800640 1 -- 172.21.15.45:0/4253448197 wait complete. 2024-04-25T10:26:05.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:26:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.427+0000 7f6a30400640 1 Processor -- start 2024-04-25T10:26:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.427+0000 7f6a30400640 1 -- start start 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a30400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a281550b0 0x7f6a280cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a30400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a280cfe70 0x7f6a2815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a30400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a28071050 con 0x7f6a2815b740 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a30400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2815d040 con 0x7f6a281550b0 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a30400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a280712f0 con 0x7f6a280cfe70 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a281550b0 0x7f6a280cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2e400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6a2815b740 0x7f6a2806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:46802/0 (socket says 172.21.15.45:46802) 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a281550b0 0x7f6a280cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52024/0 (socket says 172.21.15.45:52024) 2024-04-25T10:26:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2e400640 1 -- 172.21.15.45:0/1928209597 learned_addr learned my addr 172.21.15.45:0/1928209597 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2d000640 1 --2- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a280cfe70 0x7f6a2815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 -- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a280cfe70 msgr2=0x7f6a2815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 --2- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a280cfe70 0x7f6a2815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 -- 172.21.15.45:0/1928209597 >> v1:172.21.15.117:6789/0 conn(0x7f6a2815b740 legacy=0x7f6a2806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 -- 172.21.15.45:0/1928209597 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a280715d0 con 0x7f6a281550b0 2024-04-25T10:26:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.431+0000 7f6a2da00640 1 --2- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a281550b0 0x7f6a280cf930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097c0 tx=0x7f6a1c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb5357e90c54a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a27e00640 1 -- 172.21.15.45:0/1928209597 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b0070 con 0x7f6a281550b0 2024-04-25T10:26:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a27e00640 1 -- 172.21.15.45:0/1928209597 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0ab600 con 0x7f6a281550b0 2024-04-25T10:26:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a27e00640 1 -- 172.21.15.45:0/1928209597 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0ab9d0 con 0x7f6a281550b0 2024-04-25T10:26:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a281550b0 msgr2=0x7f6a280cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 --2- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a281550b0 0x7f6a280cf930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097c0 tx=0x7f6a1c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- 172.21.15.45:0/1928209597 shutdown_connections 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 --2- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a280cfe70 0x7f6a2815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 --2- 172.21.15.45:0/1928209597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a281550b0 0x7f6a280cf930 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- 172.21.15.45:0/1928209597 >> 172.21.15.45:0/1928209597 conn(0x7f6a28066b70 msgr2=0x7f6a28066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- 172.21.15.45:0/1928209597 shutdown_connections 2024-04-25T10:26:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- 172.21.15.45:0/1928209597 wait complete. 2024-04-25T10:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 Processor -- start 2024-04-25T10:26:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- start start 2024-04-25T10:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 0x7f6a28142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a281550b0 0x7f6a281f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a2815b740 0x7f6a28140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2815bcd0 con 0x7f6a280cfe70 2024-04-25T10:26:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2815bb50 con 0x7f6a2815b740 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a30400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2815c030 con 0x7f6a281550b0 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a2da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 0x7f6a28142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a2da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 0x7f6a28142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48018/0 (socket says 172.21.15.45:48018) 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a2da00640 1 -- 172.21.15.45:0/3149629105 learned_addr learned my addr 172.21.15.45:0/3149629105 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a2d000640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a281550b0 0x7f6a281f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.435+0000 7f6a2e400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a2815b740 0x7f6a28140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2da00640 1 -- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a281550b0 msgr2=0x7f6a281f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2da00640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a281550b0 0x7f6a281f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2da00640 1 -- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a2815b740 msgr2=0x7f6a28140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2da00640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a2815b740 0x7f6a28140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2da00640 1 -- 172.21.15.45:0/3149629105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a281f7e90 con 0x7f6a280cfe70 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2d000640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a281550b0 0x7f6a281f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a2da00640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 0x7f6a28142bb0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0ae040 tx=0x7f6a1c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b0050 con 0x7f6a280cfe70 2024-04-25T10:26:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0bc070 con 0x7f6a280cfe70 2024-04-25T10:26:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a28140880 con 0x7f6a280cfe70 2024-04-25T10:26:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b5870 con 0x7f6a280cfe70 2024-04-25T10:26:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a28143520 con 0x7f6a280cfe70 2024-04-25T10:26:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2804ef30 con 0x7f6a280cfe70 2024-04-25T10:26:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a1c0b5a10 con 0x7f6a280cfe70 2024-04-25T10:26:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a25600640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a08066f60 0x7f6a08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.439+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6a1c11f3c0 con 0x7f6a280cfe70 2024-04-25T10:26:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.443+0000 7f6a2d000640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a08066f60 0x7f6a08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.443+0000 7f6a2d000640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a08066f60 0x7f6a08069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400bfd0 tx=0x7f6a1400b750 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.443+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a281f7e90 con 0x7f6a280cfe70 2024-04-25T10:26:05.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.663+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f6a280b35a0 con 0x7f6a280cfe70 2024-04-25T10:26:05.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a25600640 1 -- 172.21.15.45:0/3149629105 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v394) v1 ==== 155+0+0 (secure 0 0 0) 0x7f6a1c016090 con 0x7f6a280cfe70 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a08066f60 msgr2=0x7f6a08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a08066f60 0x7f6a08069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400bfd0 tx=0x7f6a1400b750 comp rx=0 tx=0).stop 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 msgr2=0x7f6a28142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 0x7f6a28142bb0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0ae040 tx=0x7f6a1c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 shutdown_connections 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a08066f60 0x7f6a08069420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a2815b740 0x7f6a28140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a281550b0 0x7f6a281f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 --2- 172.21.15.45:0/3149629105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a280cfe70 0x7f6a28142bb0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 >> 172.21.15.45:0/3149629105 conn(0x7f6a28066b70 msgr2=0x7f6a2806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:05.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 shutdown_connections 2024-04-25T10:26:05.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:05.911+0000 7f6a30400640 1 -- 172.21.15.45:0/3149629105 wait complete. 2024-04-25T10:26:05.936 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:26:05.936 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:26:05.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 Processor -- start 2024-04-25T10:26:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 -- start start 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f152c15bc40 0x7f152c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f152c0cda10 0x7f152c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cf930 0x7f152c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c0b2770 con 0x7f152c0cda10 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c06c670 con 0x7f152c15bc40 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1532a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c06c950 con 0x7f152c0cf930 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f152c0cda10 0x7f152c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f152c0cda10 0x7f152c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48022/0 (socket says 172.21.15.45:48022) 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 -- 172.21.15.45:0/1770584247 learned_addr learned my addr 172.21.15.45:0/1770584247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f152be00640 1 --2- 172.21.15.45:0/1770584247 >> v2:172.21.15.117:3300/0 conn(0x7f152c15bc40 0x7f152c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1530a00640 1 --2- 172.21.15.45:0/1770584247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cf930 0x7f152c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 -- 172.21.15.45:0/1770584247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cf930 msgr2=0x7f152c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 --2- 172.21.15.45:0/1770584247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cf930 0x7f152c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 -- 172.21.15.45:0/1770584247 >> v2:172.21.15.117:3300/0 conn(0x7f152c15bc40 msgr2=0x7f152c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 --2- 172.21.15.45:0/1770584247 >> v2:172.21.15.117:3300/0 conn(0x7f152c15bc40 0x7f152c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.091+0000 7f1523e00640 1 -- 172.21.15.45:0/1770584247 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c06cc30 con 0x7f152c0cda10 2024-04-25T10:26:06.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1523e00640 1 --2- 172.21.15.45:0/1770584247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cda10 0x7f152c0b4610 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f152c0d0800 tx=0x7f151401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ffee1428d182a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f152b400640 1 -- 172.21.15.45:0/1770584247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140b0070 con 0x7f152c0cda10 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f152b400640 1 -- 172.21.15.45:0/1770584247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15140ab600 con 0x7f152c0cda10 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f152b400640 1 -- 172.21.15.45:0/1770584247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140ab9d0 con 0x7f152c0cda10 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 -- 172.21.15.45:0/1770584247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cda10 msgr2=0x7f152c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 --2- 172.21.15.45:0/1770584247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cda10 0x7f152c0b4610 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f152c0d0800 tx=0x7f151401c520 comp rx=0 tx=0).stop 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 -- 172.21.15.45:0/1770584247 shutdown_connections 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 --2- 172.21.15.45:0/1770584247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cf930 0x7f152c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 --2- 172.21.15.45:0/1770584247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cda10 0x7f152c0b4610 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 --2- 172.21.15.45:0/1770584247 >> v2:172.21.15.117:3300/0 conn(0x7f152c15bc40 0x7f152c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 -- 172.21.15.45:0/1770584247 >> 172.21.15.45:0/1770584247 conn(0x7f152c066b70 msgr2=0x7f152c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 -- 172.21.15.45:0/1770584247 shutdown_connections 2024-04-25T10:26:06.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.095+0000 7f1532a00640 1 -- 172.21.15.45:0/1770584247 wait complete. 2024-04-25T10:26:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 Processor -- start 2024-04-25T10:26:06.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 -- start start 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cda10 0x7f152c152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 0x7f152c0b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c15bc40 0x7f152c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c160270 con 0x7f152c0cf930 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1600f0 con 0x7f152c15bc40 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1603f0 con 0x7f152c0cda10 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 0x7f152c0b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 0x7f152c0b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48036/0 (socket says 172.21.15.45:48036) 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 -- 172.21.15.45:0/1968543821 learned_addr learned my addr 172.21.15.45:0/1968543821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1530a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c15bc40 0x7f152c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f152be00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cda10 0x7f152c152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 -- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cda10 msgr2=0x7f152c152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cda10 0x7f152c152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 -- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c15bc40 msgr2=0x7f152c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c15bc40 0x7f152c0b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 -- 172.21.15.45:0/1968543821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c15e2e0 con 0x7f152c0cf930 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1530a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c15bc40 0x7f152c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1523e00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 0x7f152c0b2c70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f151401abd0 tx=0x7f15140192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140ba070 con 0x7f152c0cf930 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15140b4620 con 0x7f152c0cf930 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140b0040 con 0x7f152c0cf930 2024-04-25T10:26:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c15e5c0 con 0x7f152c0cf930 2024-04-25T10:26:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.099+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c15eb80 con 0x7f152c0cf930 2024-04-25T10:26:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.103+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c04ef30 con 0x7f152c0cf930 2024-04-25T10:26:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.103+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15140ab4c0 con 0x7f152c0cf930 2024-04-25T10:26:06.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.103+0000 7f1528c00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1508066f60 0x7f1508069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.103+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f151411dfa0 con 0x7f152c0cf930 2024-04-25T10:26:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.103+0000 7f152be00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1508066f60 0x7f1508069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.103+0000 7f152be00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1508066f60 0x7f1508069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f151c0079c0 tx=0x7f151c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.107+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1514016110 con 0x7f152c0cf930 2024-04-25T10:26:06.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.271+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f152c06d960 con 0x7f152c0cf930 2024-04-25T10:26:06.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.271+0000 7f1528c00640 1 -- 172.21.15.45:0/1968543821 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+13018 (secure 0 0 0) 0x7f15140eebc0 con 0x7f152c0cf930 2024-04-25T10:26:06.276 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:06.276 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":109,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:04.897653+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/4218207272":"2024-04-26T10:26:01.888639+0000","172.21.15.45:6833/4218207272":"2024-04-26T10:26:01.888639+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1508066f60 msgr2=0x7f1508069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1508066f60 0x7f1508069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f151c0079c0 tx=0x7f151c008040 comp rx=0 tx=0).stop 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 msgr2=0x7f152c0b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 0x7f152c0b2c70 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f151401abd0 tx=0x7f15140192c0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 shutdown_connections 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1508066f60 0x7f1508069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f152c15bc40 0x7f152c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f152c0cf930 0x7f152c0b2c70 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 --2- 172.21.15.45:0/1968543821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f152c0cda10 0x7f152c152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 >> 172.21.15.45:0/1968543821 conn(0x7f152c066b70 msgr2=0x7f152c15af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 shutdown_connections 2024-04-25T10:26:06.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.275+0000 7f1532a00640 1 -- 172.21.15.45:0/1968543821 wait complete. 2024-04-25T10:26:06.297 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:26:06.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:26:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c34800640 1 Processor -- start 2024-04-25T10:26:06.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c34800640 1 -- start start 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c34800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c2c159540 0x7f4c2c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c34800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c159920 con 0x7f4c2c0721f0 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c34800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c15c2f0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c34800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c15c5d0 con 0x7f4c2c159540 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c31e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c2c159540 0x7f4c2c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c32800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4c2c0721f0 0x7f4c2c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36138/0 (socket says 172.21.15.45:36138) 2024-04-25T10:26:06.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.439+0000 7f4c32800640 1 -- 172.21.15.45:0/214881130 learned_addr learned my addr 172.21.15.45:0/214881130 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:06.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c31e00640 1 -- 172.21.15.45:0/214881130 >> v1:172.21.15.117:6789/0 conn(0x7f4c2c0aebd0 legacy=0x7f4c2c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c31e00640 1 -- 172.21.15.45:0/214881130 >> v1:172.21.15.45:6789/0 conn(0x7f4c2c0721f0 legacy=0x7f4c2c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c31e00640 1 -- 172.21.15.45:0/214881130 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c15c8b0 con 0x7f4c2c159540 2024-04-25T10:26:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c31e00640 1 --2- 172.21.15.45:0/214881130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c159540 0x7f4c2c0ae690 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4c200097c0 tx=0x7f4c2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993ed1f9f19e0900 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c30a00640 1 -- 172.21.15.45:0/214881130 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200b1070 con 0x7f4c2c159540 2024-04-25T10:26:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c30a00640 1 -- 172.21.15.45:0/214881130 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c200ac410 con 0x7f4c2c159540 2024-04-25T10:26:06.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c30a00640 1 -- 172.21.15.45:0/214881130 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200ac7e0 con 0x7f4c2c159540 2024-04-25T10:26:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c34800640 1 -- 172.21.15.45:0/214881130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c159540 msgr2=0x7f4c2c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.443+0000 7f4c34800640 1 --2- 172.21.15.45:0/214881130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c159540 0x7f4c2c0ae690 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4c200097c0 tx=0x7f4c2001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- 172.21.15.45:0/214881130 shutdown_connections 2024-04-25T10:26:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 --2- 172.21.15.45:0/214881130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c159540 0x7f4c2c0ae690 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- 172.21.15.45:0/214881130 >> 172.21.15.45:0/214881130 conn(0x7f4c2c0669a0 msgr2=0x7f4c2c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- 172.21.15.45:0/214881130 shutdown_connections 2024-04-25T10:26:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- 172.21.15.45:0/214881130 wait complete. 2024-04-25T10:26:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 Processor -- start 2024-04-25T10:26:06.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- start start 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0721f0 0x7f4c2c15e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 0x7f4c2c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2c159540 0x7f4c2c1f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c169ac0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c169940 con 0x7f4c2c0721f0 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c34800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c169c40 con 0x7f4c2c159540 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 0x7f4c2c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 0x7f4c2c1505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48054/0 (socket says 172.21.15.45:48054) 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 -- 172.21.15.45:0/1480166449 learned_addr learned my addr 172.21.15.45:0/1480166449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:06.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31e00640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0721f0 0x7f4c2c15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c32800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2c159540 0x7f4c2c1f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 -- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2c159540 msgr2=0x7f4c2c1f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2c159540 0x7f4c2c1f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 -- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0721f0 msgr2=0x7f4c2c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0721f0 0x7f4c2c15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 -- 172.21.15.45:0/1480166449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c14f510 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.447+0000 7f4c31400640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 0x7f4c2c1505b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00c420 tx=0x7f4c1c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0052d0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c005470 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c14f7f0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c007260 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c14fdb0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c04ef30 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c1c007400 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c26000640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c10066f90 0x7f4c10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4c1c0841f0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.451+0000 7f4c31e00640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c10066f90 0x7f4c10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.455+0000 7f4c31e00640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c10066f90 0x7f4c10069450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4c200ae000 tx=0x7f4c20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.455+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c1c0554f0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c2c06de90 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c26000640 1 -- 172.21.15.45:0/1480166449 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c1c054df0 con 0x7f4c2c0aebd0 2024-04-25T10:26:06.621 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c10066f90 msgr2=0x7f4c10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c10066f90 0x7f4c10069450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4c200ae000 tx=0x7f4c20008040 comp rx=0 tx=0).stop 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 msgr2=0x7f4c2c1505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 0x7f4c2c1505b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00c420 tx=0x7f4c1c00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 shutdown_connections 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4c10066f90 0x7f4c10069450 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c2c159540 0x7f4c2c1f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c2c0aebd0 0x7f4c2c1505b0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 --2- 172.21.15.45:0/1480166449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c2c0721f0 0x7f4c2c15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 >> 172.21.15.45:0/1480166449 conn(0x7f4c2c0669a0 msgr2=0x7f4c2c15d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 shutdown_connections 2024-04-25T10:26:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.619+0000 7f4c34800640 1 -- 172.21.15.45:0/1480166449 wait complete. 2024-04-25T10:26:06.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:26:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1824c00640 1 Processor -- start 2024-04-25T10:26:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1824c00640 1 -- start start 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1824c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18200d0040 0x7f182015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1824c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1820071050 con 0x7f182015b740 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1824c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182015d040 con 0x7f18201550b0 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1824c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18200712f0 con 0x7f18200d0040 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f181e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f182015b740 0x7f18200b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36142/0 (socket says 172.21.15.45:36142) 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f181e800640 1 -- 172.21.15.45:0/75717907 learned_addr learned my addr 172.21.15.45:0/75717907 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1815400640 1 --2- 172.21.15.45:0/75717907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18200d0040 0x7f182015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f181d400640 1 -- 172.21.15.45:0/75717907 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159664804 0 0) 0x7f1820071050 con 0x7f182015b740 2024-04-25T10:26:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f181d400640 1 -- 172.21.15.45:0/75717907 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1804005740 con 0x7f182015b740 2024-04-25T10:26:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f181d400640 1 -- 172.21.15.45:0/75717907 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997995317 0 0) 0x7f182015d040 con 0x7f18201550b0 2024-04-25T10:26:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1815400640 1 -- 172.21.15.45:0/75717907 >> v1:172.21.15.117:6789/0 conn(0x7f18201550b0 legacy=0x7f18200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1815400640 1 -- 172.21.15.45:0/75717907 >> v1:172.21.15.45:6789/0 conn(0x7f182015b740 legacy=0x7f18200b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.783+0000 7f1815400640 1 -- 172.21.15.45:0/75717907 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18200715d0 con 0x7f18200d0040 2024-04-25T10:26:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1815400640 1 --2- 172.21.15.45:0/75717907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18200d0040 0x7f182015b200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18080097f0 tx=0x7f180801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8093b253bef49611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f181d400640 1 -- 172.21.15.45:0/75717907 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080b1070 con 0x7f18200d0040 2024-04-25T10:26:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f181d400640 1 -- 172.21.15.45:0/75717907 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18080ac410 con 0x7f18200d0040 2024-04-25T10:26:06.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f181d400640 1 -- 172.21.15.45:0/75717907 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080ac7e0 con 0x7f18200d0040 2024-04-25T10:26:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 -- 172.21.15.45:0/75717907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18200d0040 msgr2=0x7f182015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 --2- 172.21.15.45:0/75717907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18200d0040 0x7f182015b200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18080097f0 tx=0x7f180801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 -- 172.21.15.45:0/75717907 shutdown_connections 2024-04-25T10:26:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 --2- 172.21.15.45:0/75717907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18200d0040 0x7f182015b200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 -- 172.21.15.45:0/75717907 >> 172.21.15.45:0/75717907 conn(0x7f1820066b70 msgr2=0x7f1820066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 -- 172.21.15.45:0/75717907 shutdown_connections 2024-04-25T10:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.787+0000 7f1824c00640 1 -- 172.21.15.45:0/75717907 wait complete. 2024-04-25T10:26:06.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 Processor -- start 2024-04-25T10:26:06.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- start start 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 0x7f1820157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18201550b0 0x7f1820159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f182015b740 0x7f1820158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182015bcf0 con 0x7f18200d0040 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182015bb70 con 0x7f18201550b0 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182006c090 con 0x7f182015b740 2024-04-25T10:26:06.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 0x7f1820157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 0x7f1820157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48062/0 (socket says 172.21.15.45:48062) 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 -- 172.21.15.45:0/2154909482 learned_addr learned my addr 172.21.15.45:0/2154909482 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1815400640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18201550b0 0x7f1820159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181e800640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f182015b740 0x7f1820158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 -- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f182015b740 msgr2=0x7f1820158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f182015b740 0x7f1820158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 -- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18201550b0 msgr2=0x7f1820159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18201550b0 0x7f1820159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 -- 172.21.15.45:0/2154909482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18201560c0 con 0x7f18200d0040 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181e800640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f182015b740 0x7f1820158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f181de00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 0x7f1820157f80 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f181000c480 tx=0x7f181000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810005480 con 0x7f18200d0040 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1810003600 con 0x7f18200d0040 2024-04-25T10:26:06.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100039f0 con 0x7f18200d0040 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1820159ce0 con 0x7f18200d0040 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820156380 con 0x7f18200d0040 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.791+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182004ef30 con 0x7f18200d0040 2024-04-25T10:26:06.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.795+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1810003b90 con 0x7f18200d0040 2024-04-25T10:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.795+0000 7f1816a00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f17fc066f90 0x7f17fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.795+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1810085430 con 0x7f18200d0040 2024-04-25T10:26:06.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.795+0000 7f1815400640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f17fc066f90 0x7f17fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:06.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.795+0000 7f1815400640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f17fc066f90 0x7f17fc069450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f18080ae000 tx=0x7f1808008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:06.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:06.799+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18100567b0 con 0x7f18200d0040 2024-04-25T10:26:07.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.035+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f182006d1c0 con 0x7f18200d0040 2024-04-25T10:26:07.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1816a00640 1 -- 172.21.15.45:0/2154909482 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v396) v1 ==== 169+0+0 (secure 0 0 0) 0x7f18100560b0 con 0x7f18200d0040 2024-04-25T10:26:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f17fc066f90 msgr2=0x7f17fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f17fc066f90 0x7f17fc069450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f18080ae000 tx=0x7f1808008040 comp rx=0 tx=0).stop 2024-04-25T10:26:07.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 msgr2=0x7f1820157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 0x7f1820157f80 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f181000c480 tx=0x7f181000c940 comp rx=0 tx=0).stop 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 shutdown_connections 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f17fc066f90 0x7f17fc069450 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f182015b740 0x7f1820158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18201550b0 0x7f1820159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 --2- 172.21.15.45:0/2154909482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18200d0040 0x7f1820157f80 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 >> 172.21.15.45:0/2154909482 conn(0x7f1820066b70 msgr2=0x7f18200b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 shutdown_connections 2024-04-25T10:26:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:07.919+0000 7f1824c00640 1 -- 172.21.15.45:0/2154909482 wait complete. 2024-04-25T10:26:07.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:26:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.095+0000 7f57e2e00640 1 Processor -- start 2024-04-25T10:26:08.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.095+0000 7f57e2e00640 1 -- start start 2024-04-25T10:26:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e2e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57dc15a5d0 0x7f57dc0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e2e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc0b5a10 con 0x7f57dc06d260 2024-04-25T10:26:08.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e2e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc15e660 con 0x7f57dc06d6e0 2024-04-25T10:26:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e2e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc15e940 con 0x7f57dc15a5d0 2024-04-25T10:26:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f57dc06d260 0x7f57dc06b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36154/0 (socket says 172.21.15.45:36154) 2024-04-25T10:26:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1000640 1 -- 172.21.15.45:0/4289828675 learned_addr learned my addr 172.21.15.45:0/4289828675 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4289828675 >> v2:172.21.15.117:3300/0 conn(0x7f57dc15a5d0 0x7f57dc0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412776952 0 0) 0x7f57dc0b5a10 con 0x7f57dc06d260 2024-04-25T10:26:08.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c4005740 con 0x7f57dc06d260 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1965785969 0 0) 0x7f57c4005740 con 0x7f57dc06d260 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 >> v2:172.21.15.117:3300/0 conn(0x7f57dc15a5d0 msgr2=0x7f57dc0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 --2- 172.21.15.45:0/4289828675 >> v2:172.21.15.117:3300/0 conn(0x7f57dc15a5d0 0x7f57dc0d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 >> v1:172.21.15.117:6789/0 conn(0x7f57dc06d6e0 legacy=0x7f57dc0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc15ec20 con 0x7f57dc06d260 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f57c8001680 con 0x7f57dc06d260 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 >> v1:172.21.15.45:6789/0 conn(0x7f57dc06d260 legacy=0x7f57dc06b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c4005c40 0x7f57c4006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c4006a20 0x7f57c4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c40073e0 0x7f57c400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc15e940 con 0x7f57c4005c40 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c400e310 con 0x7f57c4006a20 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c400e510 con 0x7f57c40073e0 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c4005c40 0x7f57c4006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1000640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c4006a20 0x7f57c4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e2400640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c40073e0 0x7f57c400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 -- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c40073e0 msgr2=0x7f57c400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c40073e0 0x7f57c400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 -- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c4006a20 msgr2=0x7f57c4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c4006a20 0x7f57c4006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1a00640 1 -- 172.21.15.45:0/4289828675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4005740 con 0x7f57c4005c40 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e1000640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57c4006a20 0x7f57c4006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:08.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.099+0000 7f57e2400640 1 -- 172.21.15.45:0/4289828675 reap_dead start 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c4005c40 0x7f57c4006060 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009ae0 tx=0x7f57d001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af0795db79f40fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d00b2070 con 0x7f57c4005c40 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d00a5540 con 0x7f57c4005c40 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57dbe00640 1 -- 172.21.15.45:0/4289828675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d00096b0 con 0x7f57c4005c40 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 -- 172.21.15.45:0/4289828675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c4005c40 msgr2=0x7f57c4006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c4005c40 0x7f57c4006060 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009ae0 tx=0x7f57d001ca90 comp rx=0 tx=0).stop 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 -- 172.21.15.45:0/4289828675 shutdown_connections 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4289828675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57c4005c40 0x7f57c4006060 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 -- 172.21.15.45:0/4289828675 >> 172.21.15.45:0/4289828675 conn(0x7f57dc073d10 msgr2=0x7f57dc074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 -- 172.21.15.45:0/4289828675 shutdown_connections 2024-04-25T10:26:08.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 -- 172.21.15.45:0/4289828675 wait complete. 2024-04-25T10:26:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.103+0000 7f57e2e00640 1 Processor -- start 2024-04-25T10:26:08.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- start start 2024-04-25T10:26:08.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c4005c40 0x7f57dc14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57dc14b290 0x7f57dc0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 0x7f57dc0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc071840 con 0x7f57dc1412d0 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0716c0 con 0x7f57dc14b290 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0719c0 con 0x7f57c4005c40 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 0x7f57dc0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 0x7f57dc0ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48090/0 (socket says 172.21.15.45:48090) 2024-04-25T10:26:08.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 -- 172.21.15.45:0/4055648876 learned_addr learned my addr 172.21.15.45:0/4055648876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c4005c40 0x7f57dc14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e1000640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57dc14b290 0x7f57dc0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 -- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c4005c40 msgr2=0x7f57dc14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c4005c40 0x7f57dc14e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 -- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57dc14b290 msgr2=0x7f57dc0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57dc14b290 0x7f57dc0ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 -- 172.21.15.45:0/4055648876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc14dfd0 con 0x7f57dc1412d0 2024-04-25T10:26:08.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2400640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 0x7f57dc0ce760 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f57dc06cbd0 tx=0x7f57d400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4003980 con 0x7f57dc1412d0 2024-04-25T10:26:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4018070 con 0x7f57dc1412d0 2024-04-25T10:26:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4003d80 con 0x7f57dc1412d0 2024-04-25T10:26:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc14e2b0 con 0x7f57dc1412d0 2024-04-25T10:26:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc14f7a0 con 0x7f57dc1412d0 2024-04-25T10:26:08.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.107+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc06bc20 con 0x7f57dc1412d0 2024-04-25T10:26:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.111+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57d40050d0 con 0x7f57dc1412d0 2024-04-25T10:26:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.111+0000 7f57d9600640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f57bc066f90 0x7f57bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.111+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f57d4084a20 con 0x7f57dc1412d0 2024-04-25T10:26:08.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.111+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f57bc066f90 0x7f57bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.111+0000 7f57e1a00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f57bc066f90 0x7f57bc069450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f57d00af000 tx=0x7f57d00046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.115+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57d4055d20 con 0x7f57dc1412d0 2024-04-25T10:26:08.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.283+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f57dc075120 con 0x7f57dc1412d0 2024-04-25T10:26:08.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57d9600640 1 -- 172.21.15.45:0/4055648876 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f57d4055620 con 0x7f57dc1412d0 2024-04-25T10:26:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f57bc066f90 msgr2=0x7f57bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f57bc066f90 0x7f57bc069450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f57d00af000 tx=0x7f57d00046d0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 msgr2=0x7f57dc0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 0x7f57dc0ce760 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f57dc06cbd0 tx=0x7f57d400ec10 comp rx=0 tx=0).stop 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 shutdown_connections 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f57bc066f90 0x7f57bc069450 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f57dc1412d0 0x7f57dc0ce760 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57dc14b290 0x7f57dc0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 --2- 172.21.15.45:0/4055648876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57c4005c40 0x7f57dc14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 >> 172.21.15.45:0/4055648876 conn(0x7f57dc073d10 msgr2=0x7f57dc0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 shutdown_connections 2024-04-25T10:26:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.287+0000 7f57e2e00640 1 -- 172.21.15.45:0/4055648876 wait complete. 2024-04-25T10:26:08.311 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:26:08.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:26:08.383 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:26:08.494 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:26:08.494 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:26:08.494 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:08.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:26:08.504 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:26:08.504 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:08.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:26:08.514 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:26:08.514 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:08.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:26:08.555 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:26:08.555 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:08.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:26:08.564 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:26:08.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e1e400640 1 Processor -- start 2024-04-25T10:26:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e1e400640 1 -- start start 2024-04-25T10:26:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e1e400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e180cea00 0x7f6e18166890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e1e400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e180d06a0 con 0x7f6e1815bc00 2024-04-25T10:26:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e1e400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e180b4960 con 0x7f6e180d2490 2024-04-25T10:26:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e1e400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e180b4c40 con 0x7f6e180cea00 2024-04-25T10:26:08.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e17400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6e1815bc00 0x7f6e180cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36160/0 (socket says 172.21.15.45:36160) 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e17400640 1 -- 172.21.15.45:0/2469924888 learned_addr learned my addr 172.21.15.45:0/2469924888 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e17e00640 1 --2- 172.21.15.45:0/2469924888 >> v2:172.21.15.45:3300/0 conn(0x7f6e180cea00 0x7f6e18166890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e17e00640 1 -- 172.21.15.45:0/2469924888 >> v1:172.21.15.117:6789/0 conn(0x7f6e180d2490 legacy=0x7f6e180ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e17e00640 1 -- 172.21.15.45:0/2469924888 >> v1:172.21.15.45:6789/0 conn(0x7f6e1815bc00 legacy=0x7f6e180cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e16a00640 1 -- 172.21.15.45:0/2469924888 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863207785 0 0) 0x7f6e180d06a0 con 0x7f6e1815bc00 2024-04-25T10:26:08.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.739+0000 7f6e17e00640 1 -- 172.21.15.45:0/2469924888 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e180b4f20 con 0x7f6e180cea00 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e17e00640 1 --2- 172.21.15.45:0/2469924888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e180cea00 0x7f6e18166890 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009d90 tx=0x7f6e1001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b359e4c2d88215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e16a00640 1 -- 172.21.15.45:0/2469924888 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e100b2070 con 0x7f6e180cea00 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e16a00640 1 -- 172.21.15.45:0/2469924888 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e100a57f0 con 0x7f6e180cea00 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e16a00640 1 -- 172.21.15.45:0/2469924888 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e100a5bc0 con 0x7f6e180cea00 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 -- 172.21.15.45:0/2469924888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e180cea00 msgr2=0x7f6e18166890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 --2- 172.21.15.45:0/2469924888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e180cea00 0x7f6e18166890 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009d90 tx=0x7f6e1001c930 comp rx=0 tx=0).stop 2024-04-25T10:26:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 -- 172.21.15.45:0/2469924888 shutdown_connections 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 --2- 172.21.15.45:0/2469924888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e180cea00 0x7f6e18166890 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 -- 172.21.15.45:0/2469924888 >> 172.21.15.45:0/2469924888 conn(0x7f6e180d1900 msgr2=0x7f6e180d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 -- 172.21.15.45:0/2469924888 shutdown_connections 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.743+0000 7f6e1e400640 1 -- 172.21.15.45:0/2469924888 wait complete. 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 Processor -- start 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- start start 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e180cea00 0x7f6e181579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e180d2490 0x7f6e1815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 0x7f6e180c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18169fa0 con 0x7f6e1815bc00 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18169e20 con 0x7f6e180d2490 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1816a120 con 0x7f6e180cea00 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 0x7f6e180c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 0x7f6e180c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48102/0 (socket says 172.21.15.45:48102) 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 -- 172.21.15.45:0/1514161105 learned_addr learned my addr 172.21.15.45:0/1514161105 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e180cea00 0x7f6e181579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e0fe00640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e180d2490 0x7f6e1815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 -- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e180cea00 msgr2=0x7f6e181579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e180cea00 0x7f6e181579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 -- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e180d2490 msgr2=0x7f6e1815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e180d2490 0x7f6e1815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 -- 172.21.15.45:0/1514161105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e181580f0 con 0x7f6e1815bc00 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e17e00640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 0x7f6e180c7600 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6e100b0040 tx=0x7f6e100196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e100bd020 con 0x7f6e1815bc00 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e100b8670 con 0x7f6e1815bc00 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e181583d0 con 0x7f6e1815bc00 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e100b2040 con 0x7f6e1815bc00 2024-04-25T10:26:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e18158990 con 0x7f6e1815bc00 2024-04-25T10:26:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.747+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1804ef30 con 0x7f6e1815bc00 2024-04-25T10:26:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.751+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e100c0020 con 0x7f6e1815bc00 2024-04-25T10:26:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.751+0000 7f6e0f400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6df4066f60 0x7f6df4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:08.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.751+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6e1011fe30 con 0x7f6e1815bc00 2024-04-25T10:26:08.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.751+0000 7f6e17400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6df4066f60 0x7f6df4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:08.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.751+0000 7f6e17400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6df4066f60 0x7f6df4069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6e08000f70 tx=0x7f6e08007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:08.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.755+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e100c7030 con 0x7f6e1815bc00 2024-04-25T10:26:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.951+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e18157590 con 0x7f6e1815bc00 2024-04-25T10:26:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.951+0000 7f6e0f400640 1 -- 172.21.15.45:0/1514161105 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6e10016520 con 0x7f6e1815bc00 2024-04-25T10:26:08.956 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:08.956 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2505561570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2505561570},{"type":"v1","addr":"172.21.15.117:6837","nonce":2505561570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7375,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/3486706251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3486706251},{"type":"v1","addr":"172.21.15.117:6836","nonce":3486706251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":7848,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1661089677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1661089677},{"type":"v1","addr":"172.21.15.45:6834","nonce":1661089677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3415348450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3415348450},{"type":"v1","addr":"172.21.15.45:6835","nonce":3415348450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[]} 2024-04-25T10:26:08.956 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 75 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6df4066f60 msgr2=0x7f6df4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6df4066f60 0x7f6df4069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6e08000f70 tx=0x7f6e08007c10 comp rx=0 tx=0).stop 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 msgr2=0x7f6e180c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 0x7f6e180c7600 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6e100b0040 tx=0x7f6e100196d0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 shutdown_connections 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6df4066f60 0x7f6df4069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e1815bc00 0x7f6e180c7600 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e180d2490 0x7f6e1815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 --2- 172.21.15.45:0/1514161105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6e180cea00 0x7f6e181579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 >> 172.21.15.45:0/1514161105 conn(0x7f6e180d1900 msgr2=0x7f6e1806e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 shutdown_connections 2024-04-25T10:26:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:08.955+0000 7f6e1e400640 1 -- 172.21.15.45:0/1514161105 wait complete. 2024-04-25T10:26:08.979 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:26:08.979 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:26:08.979 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:26:08.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:08.979 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:26:08.979 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:26:08.980 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:26:08.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:08.980 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:26:08.980 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:26:08.980 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:26:08.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:08.980 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:26:08.980 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:26:08.981 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:26:08.981 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:26:08.990 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:26:08.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:26:08.992 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:26:08.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:26:08.996 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:26:08.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:26:08.997 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:26:08.997 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:26:09.042 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:26:09.042 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:26:09.043 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:09.043 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:09.044 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:26:09.044 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:09.045 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:26:09.045 DEBUG:teuthology.parallel:result is None 2024-04-25T10:26:09.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:26:09.129 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:26:09.131 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:26:09.131 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:26:09.133 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:26:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff812400640 1 Processor -- start 2024-04-25T10:26:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff812400640 1 -- start start 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff812400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff80c0b14f0 0x7ff80c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff812400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c074690 con 0x7ff80c074160 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff812400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c0afd60 con 0x7ff80c15bb20 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff812400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c0afb10 con 0x7ff80c0b14f0 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff803e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff80c074160 0x7ff80c0b0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36174/0 (socket says 172.21.15.45:36174) 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff803e00640 1 -- 172.21.15.45:0/1022082796 learned_addr learned my addr 172.21.15.45:0/1022082796 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff80be00640 1 --2- 172.21.15.45:0/1022082796 >> v2:172.21.15.117:3300/0 conn(0x7ff80c0b14f0 0x7ff80c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff80aa00640 1 -- 172.21.15.45:0/1022082796 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402835025 0 0) 0x7ff80c074690 con 0x7ff80c074160 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff80aa00640 1 -- 172.21.15.45:0/1022082796 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7f0003530 con 0x7ff80c074160 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff80be00640 1 -- 172.21.15.45:0/1022082796 >> v1:172.21.15.117:6789/0 conn(0x7ff80c15bb20 legacy=0x7ff80c072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff80be00640 1 -- 172.21.15.45:0/1022082796 >> v1:172.21.15.45:6789/0 conn(0x7ff80c074160 legacy=0x7ff80c0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.191+0000 7ff80be00640 1 -- 172.21.15.45:0/1022082796 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c0b0040 con 0x7ff80c0b14f0 2024-04-25T10:26:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff80be00640 1 --2- 172.21.15.45:0/1022082796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c0cbb70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0afc90 tx=0x7ff80401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee44bff50e1da25c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff80aa00640 1 -- 172.21.15.45:0/1022082796 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040b2070 con 0x7ff80c0b14f0 2024-04-25T10:26:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff80aa00640 1 -- 172.21.15.45:0/1022082796 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8040a5810 con 0x7ff80c0b14f0 2024-04-25T10:26:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff80aa00640 1 -- 172.21.15.45:0/1022082796 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040a5be0 con 0x7ff80c0b14f0 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 -- 172.21.15.45:0/1022082796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 msgr2=0x7ff80c0cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 --2- 172.21.15.45:0/1022082796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c0cbb70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0afc90 tx=0x7ff80401c950 comp rx=0 tx=0).stop 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 -- 172.21.15.45:0/1022082796 shutdown_connections 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 --2- 172.21.15.45:0/1022082796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c0cbb70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 -- 172.21.15.45:0/1022082796 >> 172.21.15.45:0/1022082796 conn(0x7ff80c0b1d50 msgr2=0x7ff80c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 -- 172.21.15.45:0/1022082796 shutdown_connections 2024-04-25T10:26:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 -- 172.21.15.45:0/1022082796 wait complete. 2024-04-25T10:26:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 Processor -- start 2024-04-25T10:26:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.195+0000 7ff812400640 1 -- start start 2024-04-25T10:26:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c074160 0x7ff80c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 0x7ff80c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c169e70 con 0x7ff80c15bb20 2024-04-25T10:26:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c169cf0 con 0x7ff80c0b14f0 2024-04-25T10:26:09.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c169ff0 con 0x7ff80c074160 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 0x7ff80c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 0x7ff80c0b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48138/0 (socket says 172.21.15.45:48138) 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 -- 172.21.15.45:0/1199823239 learned_addr learned my addr 172.21.15.45:0/1199823239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff803e00640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80b400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c074160 0x7ff80c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 -- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c074160 msgr2=0x7ff80c1565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c074160 0x7ff80c1565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 -- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 msgr2=0x7ff80c099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 -- 172.21.15.45:0/1199823239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c158040 con 0x7ff80c15bb20 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80b400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c074160 0x7ff80c1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:09.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff80be00640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 0x7ff80c0b5010 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff8040b0040 tx=0x7ff8040196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040b2050 con 0x7ff80c15bb20 2024-04-25T10:26:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8040be070 con 0x7ff80c15bb20 2024-04-25T10:26:09.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c158320 con 0x7ff80c15bb20 2024-04-25T10:26:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040ad890 con 0x7ff80c15bb20 2024-04-25T10:26:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c1588e0 con 0x7ff80c15bb20 2024-04-25T10:26:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.199+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c04ef30 con 0x7ff80c15bb20 2024-04-25T10:26:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.203+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8040ada30 con 0x7ff80c15bb20 2024-04-25T10:26:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.203+0000 7ff803400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff7e8066f90 0x7ff7e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.203+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff80411fda0 con 0x7ff80c15bb20 2024-04-25T10:26:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.203+0000 7ff80b400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff7e8066f90 0x7ff7e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:09.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.203+0000 7ff80b400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff7e8066f90 0x7ff7e8069450 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc001710 tx=0x7ff7fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:09.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.207+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff804016540 con 0x7ff80c15bb20 2024-04-25T10:26:09.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.383+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff80c0b5e70 con 0x7ff80c15bb20 2024-04-25T10:26:09.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.931+0000 7ff803400640 1 -- 172.21.15.45:0/1199823239 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff8040f1140 con 0x7ff80c15bb20 2024-04-25T10:26:09.934 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff7e8066f90 msgr2=0x7ff7e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff7e8066f90 0x7ff7e8069450 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc001710 tx=0x7ff7fc009040 comp rx=0 tx=0).stop 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 msgr2=0x7ff80c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 0x7ff80c0b5010 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff8040b0040 tx=0x7ff8040196f0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 shutdown_connections 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff7e8066f90 0x7ff7e8069450 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80c15bb20 0x7ff80c0b5010 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0b14f0 0x7ff80c099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 --2- 172.21.15.45:0/1199823239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c074160 0x7ff80c1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:09.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 >> 172.21.15.45:0/1199823239 conn(0x7ff80c0b1d50 msgr2=0x7ff80c0ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:09.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 shutdown_connections 2024-04-25T10:26:09.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:09.935+0000 7ff812400640 1 -- 172.21.15.45:0/1199823239 wait complete. 2024-04-25T10:26:09.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:26:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa502400640 1 Processor -- start 2024-04-25T10:26:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa502400640 1 -- start start 2024-04-25T10:26:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa502400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4fc0af9c0 0x7fa4fc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa502400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc06f200 con 0x7fa4fc0afda0 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa502400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc0ccc10 con 0x7fa4fc071f50 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa502400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc06d380 con 0x7fa4fc0af9c0 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4fb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4fc0af9c0 0x7fa4fc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4fb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4fc0af9c0 0x7fa4fc06ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48146/0 (socket says 172.21.15.45:48146) 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4fb400640 1 -- 172.21.15.45:0/509386018 learned_addr learned my addr 172.21.15.45:0/509386018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4fb400640 1 -- 172.21.15.45:0/509386018 >> v1:172.21.15.117:6789/0 conn(0x7fa4fc071f50 legacy=0x7fa4fc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4fb400640 1 -- 172.21.15.45:0/509386018 >> v1:172.21.15.45:6789/0 conn(0x7fa4fc0afda0 legacy=0x7fa4fc0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4faa00640 1 -- 172.21.15.45:0/509386018 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602886298 0 0) 0x7fa4e4001120 con 0x7fa4fc0afda0 2024-04-25T10:26:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.103+0000 7fa4fb400640 1 -- 172.21.15.45:0/509386018 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc0ccef0 con 0x7fa4fc0af9c0 2024-04-25T10:26:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa4fb400640 1 --2- 172.21.15.45:0/509386018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc0af9c0 0x7fa4fc06ddd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009870 tx=0x7fa4ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9c80cca5284de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa4faa00640 1 -- 172.21.15.45:0/509386018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0aa600 con 0x7fa4fc0af9c0 2024-04-25T10:26:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa4faa00640 1 -- 172.21.15.45:0/509386018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0af070 con 0x7fa4fc0af9c0 2024-04-25T10:26:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa4faa00640 1 -- 172.21.15.45:0/509386018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0aa9c0 con 0x7fa4fc0af9c0 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 -- 172.21.15.45:0/509386018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc0af9c0 msgr2=0x7fa4fc06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 --2- 172.21.15.45:0/509386018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc0af9c0 0x7fa4fc06ddd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009870 tx=0x7fa4ec01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 -- 172.21.15.45:0/509386018 shutdown_connections 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 --2- 172.21.15.45:0/509386018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc0af9c0 0x7fa4fc06ddd0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 -- 172.21.15.45:0/509386018 >> 172.21.15.45:0/509386018 conn(0x7fa4fc0b2410 msgr2=0x7fa4fc0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 -- 172.21.15.45:0/509386018 shutdown_connections 2024-04-25T10:26:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.107+0000 7fa502400640 1 -- 172.21.15.45:0/509386018 wait complete. 2024-04-25T10:26:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 Processor -- start 2024-04-25T10:26:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- start start 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 0x7fa4fc0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4fc0af9c0 0x7fa4fc0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc0afda0 0x7fa4fc14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc1600d0 con 0x7fa4fc071f50 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc067300 con 0x7fa4fc0afda0 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc160250 con 0x7fa4fc0af9c0 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 0x7fa4fc0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 0x7fa4fc0c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48152/0 (socket says 172.21.15.45:48152) 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 -- 172.21.15.45:0/3214632000 learned_addr learned my addr 172.21.15.45:0/3214632000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4f2a00640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4fc0af9c0 0x7fa4fc0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fbe00640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc0afda0 0x7fa4fc14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 -- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4fc0af9c0 msgr2=0x7fa4fc0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4fc0af9c0 0x7fa4fc0cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 -- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc0afda0 msgr2=0x7fa4fc14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc0afda0 0x7fa4fc14df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 -- 172.21.15.45:0/3214632000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc0d02e0 con 0x7fa4fc071f50 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4fb400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 0x7fa4fc0c6e60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0ac040 tx=0x7fa4ec01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0bd5f0 con 0x7fa4fc071f50 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0bd7b0 con 0x7fa4fc071f50 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc0cf460 con 0x7fa4fc071f50 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0bdba0 con 0x7fa4fc071f50 2024-04-25T10:26:10.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc0ce980 con 0x7fa4fc071f50 2024-04-25T10:26:10.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.111+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc04ef30 con 0x7fa4fc071f50 2024-04-25T10:26:10.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.115+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4ec0a9070 con 0x7fa4fc071f50 2024-04-25T10:26:10.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.115+0000 7fa4f3e00640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa4d8066f90 0x7fa4d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.115+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa4ec11f8b0 con 0x7fa4fc071f50 2024-04-25T10:26:10.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.115+0000 7fa4f2a00640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa4d8066f90 0x7fa4d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.115+0000 7fa4f2a00640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa4d8066f90 0x7fa4d8069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc1f9f70 tx=0x7fa4e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.119+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4ec0c3060 con 0x7fa4fc071f50 2024-04-25T10:26:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa4fc071420 con 0x7fa4fc071f50 2024-04-25T10:26:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa4f3e00640 1 -- 172.21.15.45:0/3214632000 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa4ec016150 con 0x7fa4fc071f50 2024-04-25T10:26:10.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:10.341 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:10.341 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:26:10.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.342 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:26:10.343 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.344 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.345 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.346 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:26:10.348 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.350 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:26:10.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:26:10.352 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:26:10.354 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:10.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa4d8066f90 msgr2=0x7fa4d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa4d8066f90 0x7fa4d8069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc1f9f70 tx=0x7fa4e4009040 comp rx=0 tx=0).stop 2024-04-25T10:26:10.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 msgr2=0x7fa4fc0c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 0x7fa4fc0c6e60 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0ac040 tx=0x7fa4ec01ace0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 shutdown_connections 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa4d8066f90 0x7fa4d8069450 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4fc0afda0 0x7fa4fc14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4fc0af9c0 0x7fa4fc0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 --2- 172.21.15.45:0/3214632000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4fc071f50 0x7fa4fc0c6e60 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 >> 172.21.15.45:0/3214632000 conn(0x7fa4fc0b2410 msgr2=0x7fa4fc0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 shutdown_connections 2024-04-25T10:26:10.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.339+0000 7fa502400640 1 -- 172.21.15.45:0/3214632000 wait complete. 2024-04-25T10:26:10.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 Processor -- start 2024-04-25T10:26:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 -- start start 2024-04-25T10:26:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f77a015b9b0 0x7f77a00ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a00ce700 0x7f77a00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77a00cf380 0x7f77a00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a00d2600 con 0x7f77a00cf380 2024-04-25T10:26:10.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a00cd120 con 0x7f77a015b9b0 2024-04-25T10:26:10.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a5600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00cd400 con 0x7f77a00ce700 2024-04-25T10:26:10.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77a00cf380 0x7f77a00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77a00cf380 0x7f77a00cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48164/0 (socket says 172.21.15.45:48164) 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 -- 172.21.15.45:0/3219730908 learned_addr learned my addr 172.21.15.45:0/3219730908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f779fe00640 1 --2- 172.21.15.45:0/3219730908 >> v2:172.21.15.117:3300/0 conn(0x7f77a015b9b0 0x7f77a00ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f7797e00640 1 --2- 172.21.15.45:0/3219730908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a00ce700 0x7f77a00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 -- 172.21.15.45:0/3219730908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a00ce700 msgr2=0x7f77a00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 --2- 172.21.15.45:0/3219730908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a00ce700 0x7f77a00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 -- 172.21.15.45:0/3219730908 >> v2:172.21.15.117:3300/0 conn(0x7f77a015b9b0 msgr2=0x7f77a00ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 --2- 172.21.15.45:0/3219730908 >> v2:172.21.15.117:3300/0 conn(0x7f77a015b9b0 0x7f77a00ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.503+0000 7f77a4c00640 1 -- 172.21.15.45:0/3219730908 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a00b47a0 con 0x7f77a00cf380 2024-04-25T10:26:10.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.507+0000 7f77a4c00640 1 --2- 172.21.15.45:0/3219730908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00cf380 0x7f77a00cf7f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f77a0070b60 tx=0x7f779801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2a65fc02754196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.507+0000 7f779f400640 1 -- 172.21.15.45:0/3219730908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77980b4070 con 0x7f77a00cf380 2024-04-25T10:26:10.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.507+0000 7f779f400640 1 -- 172.21.15.45:0/3219730908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77980af400 con 0x7f77a00cf380 2024-04-25T10:26:10.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.507+0000 7f779f400640 1 -- 172.21.15.45:0/3219730908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77980af7d0 con 0x7f77a00cf380 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.507+0000 7f77a5600640 1 -- 172.21.15.45:0/3219730908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00cf380 msgr2=0x7f77a00cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.507+0000 7f77a5600640 1 --2- 172.21.15.45:0/3219730908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00cf380 0x7f77a00cf7f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f77a0070b60 tx=0x7f779801f1e0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- 172.21.15.45:0/3219730908 shutdown_connections 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 --2- 172.21.15.45:0/3219730908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00cf380 0x7f77a00cf7f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 --2- 172.21.15.45:0/3219730908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a00ce700 0x7f77a00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 --2- 172.21.15.45:0/3219730908 >> v2:172.21.15.117:3300/0 conn(0x7f77a015b9b0 0x7f77a00ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- 172.21.15.45:0/3219730908 >> 172.21.15.45:0/3219730908 conn(0x7f77a00d17b0 msgr2=0x7f77a00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- 172.21.15.45:0/3219730908 shutdown_connections 2024-04-25T10:26:10.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- 172.21.15.45:0/3219730908 wait complete. 2024-04-25T10:26:10.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 Processor -- start 2024-04-25T10:26:10.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- start start 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 0x7f77a009b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77a00cf380 0x7f77a0153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a015b9b0 0x7f77a00c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a015bf10 con 0x7f77a00ce700 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a015bd90 con 0x7f77a00cf380 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a5600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00ccd30 con 0x7f77a015b9b0 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 0x7f77a009b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 0x7f77a009b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48172/0 (socket says 172.21.15.45:48172) 2024-04-25T10:26:10.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 -- 172.21.15.45:0/3873195663 learned_addr learned my addr 172.21.15.45:0/3873195663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f7797e00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77a00cf380 0x7f77a0153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a4c00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a015b9b0 0x7f77a00c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 -- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a015b9b0 msgr2=0x7f77a00c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a015b9b0 0x7f77a00c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 -- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77a00cf380 msgr2=0x7f77a0153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77a00cf380 0x7f77a0153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 -- 172.21.15.45:0/3873195663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0157e30 con 0x7f77a00ce700 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f77a4c00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a015b9b0 0x7f77a00c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f7797e00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77a00cf380 0x7f77a0153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779fe00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 0x7f77a009b8f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f779000d5f0 tx=0x7f779000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.511+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77900125f0 con 0x7f77a00ce700 2024-04-25T10:26:10.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7790019070 con 0x7f77a00ce700 2024-04-25T10:26:10.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77900129b0 con 0x7f77a00ce700 2024-04-25T10:26:10.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a00c8d10 con 0x7f77a00ce700 2024-04-25T10:26:10.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a00c92d0 con 0x7f77a00ce700 2024-04-25T10:26:10.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a0075120 con 0x7f77a00ce700 2024-04-25T10:26:10.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7790012b50 con 0x7f77a00ce700 2024-04-25T10:26:10.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f779cc00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f777c066f90 0x7f777c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7790085150 con 0x7f77a00ce700 2024-04-25T10:26:10.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.515+0000 7f7797e00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f777c066f90 0x7f777c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.519+0000 7f7797e00640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f777c066f90 0x7f777c069450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7788004660 tx=0x7f7788008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.519+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7790056410 con 0x7f77a00ce700 2024-04-25T10:26:10.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a00008e0 con 0x7f77a00ce700 2024-04-25T10:26:10.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f779cc00640 1 -- 172.21.15.45:0/3873195663 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f7790055d10 con 0x7f77a00ce700 2024-04-25T10:26:10.685 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:10.685 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":113,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:09.942634+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f777c066f90 msgr2=0x7f777c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f777c066f90 0x7f777c069450 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7788004660 tx=0x7f7788008040 comp rx=0 tx=0).stop 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 msgr2=0x7f77a009b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 0x7f77a009b8f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f779000d5f0 tx=0x7f779000dab0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 shutdown_connections 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f777c066f90 0x7f777c069450 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77a015b9b0 0x7f77a00c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77a00cf380 0x7f77a0153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 --2- 172.21.15.45:0/3873195663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77a00ce700 0x7f77a009b8f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 >> 172.21.15.45:0/3873195663 conn(0x7f77a00d17b0 msgr2=0x7f77a00d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 shutdown_connections 2024-04-25T10:26:10.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.683+0000 7f77a5600640 1 -- 172.21.15.45:0/3873195663 wait complete. 2024-04-25T10:26:10.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:26:10.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:26:10.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dc7200640 1 Processor -- start 2024-04-25T10:26:10.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dc7200640 1 -- start start 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dc7200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8dc015a0e0 0x7f8dc015b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dc7200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00b2fd0 con 0x7f8dc006c0d0 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dc7200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc00b32b0 con 0x7f8dc00aea50 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dc7200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00af1d0 con 0x7f8dc015a0e0 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dbfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8dc015a0e0 0x7f8dc015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dbfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8dc015a0e0 0x7f8dc015b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48186/0 (socket says 172.21.15.45:48186) 2024-04-25T10:26:10.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.847+0000 7f8dbfe00640 1 -- 172.21.15.45:0/4267402547 learned_addr learned my addr 172.21.15.45:0/4267402547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:10.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbfe00640 1 -- 172.21.15.45:0/4267402547 >> v1:172.21.15.117:6789/0 conn(0x7f8dc00aea50 legacy=0x7f8dc00b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbfe00640 1 -- 172.21.15.45:0/4267402547 >> v1:172.21.15.45:6789/0 conn(0x7f8dc006c0d0 legacy=0x7f8dc00705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbfe00640 1 -- 172.21.15.45:0/4267402547 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00b3590 con 0x7f8dc015a0e0 2024-04-25T10:26:10.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbf400640 1 -- 172.21.15.45:0/4267402547 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980151625 0 0) 0x7f8dc00b2fd0 con 0x7f8dc006c0d0 2024-04-25T10:26:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbfe00640 1 --2- 172.21.15.45:0/4267402547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc015a0e0 0x7f8dc015b150 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009e50 tx=0x7f8db001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4a0f14112258c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbf400640 1 -- 172.21.15.45:0/4267402547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00a55d0 con 0x7f8dc015a0e0 2024-04-25T10:26:10.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbf400640 1 -- 172.21.15.45:0/4267402547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db00a5770 con 0x7f8dc015a0e0 2024-04-25T10:26:10.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dbf400640 1 -- 172.21.15.45:0/4267402547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00a5b40 con 0x7f8dc015a0e0 2024-04-25T10:26:10.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 -- 172.21.15.45:0/4267402547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc015a0e0 msgr2=0x7f8dc015b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 --2- 172.21.15.45:0/4267402547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc015a0e0 0x7f8dc015b150 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009e50 tx=0x7f8db001c910 comp rx=0 tx=0).stop 2024-04-25T10:26:10.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 -- 172.21.15.45:0/4267402547 shutdown_connections 2024-04-25T10:26:10.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 --2- 172.21.15.45:0/4267402547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc015a0e0 0x7f8dc015b150 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 -- 172.21.15.45:0/4267402547 >> 172.21.15.45:0/4267402547 conn(0x7f8dc00669a0 msgr2=0x7f8dc0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:10.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 -- 172.21.15.45:0/4267402547 shutdown_connections 2024-04-25T10:26:10.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.851+0000 7f8dc7200640 1 -- 172.21.15.45:0/4267402547 wait complete. 2024-04-25T10:26:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 Processor -- start 2024-04-25T10:26:10.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 -- start start 2024-04-25T10:26:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dc006c0d0 0x7f8dc014c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 0x7f8dc00aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8dc015a0e0 0x7f8dc00ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0070e80 con 0x7f8dc00aea50 2024-04-25T10:26:10.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0070d00 con 0x7f8dc006c0d0 2024-04-25T10:26:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc006c4e0 con 0x7f8dc015a0e0 2024-04-25T10:26:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 0x7f8dc00aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 0x7f8dc00aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48194/0 (socket says 172.21.15.45:48194) 2024-04-25T10:26:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 -- 172.21.15.45:0/2469996793 learned_addr learned my addr 172.21.15.45:0/2469996793 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dbfe00640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dc006c0d0 0x7f8dc014c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc5200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8dc015a0e0 0x7f8dc00ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 -- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8dc015a0e0 msgr2=0x7f8dc00ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8dc015a0e0 0x7f8dc00ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 -- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dc006c0d0 msgr2=0x7f8dc014c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dc006c0d0 0x7f8dc014c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 -- 172.21.15.45:0/2469996793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00adc20 con 0x7f8dc00aea50 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8db7400640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 0x7f8dc00aae30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c500 tx=0x7f8da800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8016020 con 0x7f8dc00aea50 2024-04-25T10:26:10.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da8005220 con 0x7f8dc00aea50 2024-04-25T10:26:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8007030 con 0x7f8dc00aea50 2024-04-25T10:26:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00adf00 con 0x7f8dc00aea50 2024-04-25T10:26:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.855+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00af500 con 0x7f8dc00aea50 2024-04-25T10:26:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.859+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc00af700 con 0x7f8dc00aea50 2024-04-25T10:26:10.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.859+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8da80071d0 con 0x7f8dc00aea50 2024-04-25T10:26:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.859+0000 7f8dbcc00640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d9c066f60 0x7f8d9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.859+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8da8084610 con 0x7f8dc00aea50 2024-04-25T10:26:10.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.859+0000 7f8dbfe00640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d9c066f60 0x7f8d9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:10.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.859+0000 7f8dbfe00640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d9c066f60 0x7f8d9c069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8db00ae000 tx=0x7f8db00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:10.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:10.863+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dc00af700 con 0x7f8dc00aea50 2024-04-25T10:26:11.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.027+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dc004ef30 con 0x7f8dc00aea50 2024-04-25T10:26:11.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.027+0000 7f8dbcc00640 1 -- 172.21.15.45:0/2469996793 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8da8020020 con 0x7f8dc00aea50 2024-04-25T10:26:11.030 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:26:11.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d9c066f60 msgr2=0x7f8d9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d9c066f60 0x7f8d9c069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8db00ae000 tx=0x7f8db00a7070 comp rx=0 tx=0).stop 2024-04-25T10:26:11.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 msgr2=0x7f8dc00aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 0x7f8dc00aae30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c500 tx=0x7f8da800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 shutdown_connections 2024-04-25T10:26:11.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d9c066f60 0x7f8d9c069420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8dc015a0e0 0x7f8dc00ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8dc00aea50 0x7f8dc00aae30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 --2- 172.21.15.45:0/2469996793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dc006c0d0 0x7f8dc014c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 >> 172.21.15.45:0/2469996793 conn(0x7f8dc00669a0 msgr2=0x7f8dc015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 shutdown_connections 2024-04-25T10:26:11.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.031+0000 7f8dc7200640 1 -- 172.21.15.45:0/2469996793 wait complete. 2024-04-25T10:26:11.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:26:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 Processor -- start 2024-04-25T10:26:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 -- start start 2024-04-25T10:26:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97740d1820 0x7f97740d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97740b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97740b13e0 0x7f97740ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97740b1870 con 0x7f97740d1820 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97740b1af0 con 0x7f97740b13e0 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f977aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740b1dd0 con 0x7f977415e050 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97740d1820 0x7f97740d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9778a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97740b13e0 0x7f97740ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f97740d1820 0x7f97740d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48196/0 (socket says 172.21.15.45:48196) 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 -- 172.21.15.45:0/1487217889 learned_addr learned my addr 172.21.15.45:0/1487217889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f976be00640 1 --2- 172.21.15.45:0/1487217889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97740b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 -- 172.21.15.45:0/1487217889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 msgr2=0x7f97740b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 --2- 172.21.15.45:0/1487217889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97740b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 -- 172.21.15.45:0/1487217889 >> v2:172.21.15.117:3300/0 conn(0x7f97740b13e0 msgr2=0x7f97740ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 --2- 172.21.15.45:0/1487217889 >> v2:172.21.15.117:3300/0 conn(0x7f97740b13e0 0x7f97740ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.191+0000 7f9773e00640 1 -- 172.21.15.45:0/1487217889 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740b20b0 con 0x7f97740d1820 2024-04-25T10:26:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f9773e00640 1 --2- 172.21.15.45:0/1487217889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f97740d1c00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f977406ce10 tx=0x7f976401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5bccaaec487f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f9773400640 1 -- 172.21.15.45:0/1487217889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640b0070 con 0x7f97740d1820 2024-04-25T10:26:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f9773400640 1 -- 172.21.15.45:0/1487217889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97640ab600 con 0x7f97740d1820 2024-04-25T10:26:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f9773400640 1 -- 172.21.15.45:0/1487217889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640ab9d0 con 0x7f97740d1820 2024-04-25T10:26:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 -- 172.21.15.45:0/1487217889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 msgr2=0x7f97740d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 --2- 172.21.15.45:0/1487217889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f97740d1c00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f977406ce10 tx=0x7f976401c260 comp rx=0 tx=0).stop 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 -- 172.21.15.45:0/1487217889 shutdown_connections 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 --2- 172.21.15.45:0/1487217889 >> v2:172.21.15.117:3300/0 conn(0x7f97740b13e0 0x7f97740ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 --2- 172.21.15.45:0/1487217889 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97740b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 --2- 172.21.15.45:0/1487217889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f97740d1c00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 -- 172.21.15.45:0/1487217889 >> 172.21.15.45:0/1487217889 conn(0x7f977409a9d0 msgr2=0x7f977409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 -- 172.21.15.45:0/1487217889 shutdown_connections 2024-04-25T10:26:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.195+0000 7f977aa00640 1 -- 172.21.15.45:0/1487217889 wait complete. 2024-04-25T10:26:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 Processor -- start 2024-04-25T10:26:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 -- start start 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97740b13e0 0x7f977414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f9774152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97741531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97740cfdf0 con 0x7f97740d1820 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97740cfc70 con 0x7f97740b13e0 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97740cff70 con 0x7f977415e050 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f9774152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f9774152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48206/0 (socket says 172.21.15.45:48206) 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 -- 172.21.15.45:0/840985227 learned_addr learned my addr 172.21.15.45:0/840985227 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f9778a00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97741531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f9773e00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97740b13e0 0x7f977414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 -- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 msgr2=0x7f97741531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97741531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 -- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97740b13e0 msgr2=0x7f977414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97740b13e0 0x7f977414e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 -- 172.21.15.45:0/840985227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9774154b80 con 0x7f97740d1820 2024-04-25T10:26:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f976be00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f9774152cb0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e870 tx=0x7f975c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c017070 con 0x7f97740d1820 2024-04-25T10:26:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c00c940 con 0x7f97740d1820 2024-04-25T10:26:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00cd10 con 0x7f97740d1820 2024-04-25T10:26:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.199+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9774154e60 con 0x7f97740d1820 2024-04-25T10:26:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977414ee50 con 0x7f97740d1820 2024-04-25T10:26:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977404ef30 con 0x7f97740d1820 2024-04-25T10:26:11.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f975c01a070 con 0x7f97740d1820 2024-04-25T10:26:11.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f9770c00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9750066f60 0x7f9750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f975c084ec0 con 0x7f97740d1820 2024-04-25T10:26:11.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f9773e00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9750066f60 0x7f9750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.203+0000 7f9773e00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9750066f60 0x7f9750069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97640ad000 tx=0x7f9764008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.207+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f975c056210 con 0x7f97740d1820 2024-04-25T10:26:11.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.371+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f97740d1600 con 0x7f97740d1820 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.371+0000 7f9770c00640 1 -- 172.21.15.45:0/840985227 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+65 (secure 0 0 0) 0x7f975c055b10 con 0x7f97740d1820 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:11.375 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:26:11.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9750066f60 msgr2=0x7f9750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9750066f60 0x7f9750069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f97640ad000 tx=0x7f9764008040 comp rx=0 tx=0).stop 2024-04-25T10:26:11.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 msgr2=0x7f9774152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f9774152cb0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e870 tx=0x7f975c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:26:11.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 shutdown_connections 2024-04-25T10:26:11.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9750066f60 0x7f9750069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f977415e050 0x7f97741531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97740d1820 0x7f9774152cb0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 --2- 172.21.15.45:0/840985227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97740b13e0 0x7f977414e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 >> 172.21.15.45:0/840985227 conn(0x7f977409a9d0 msgr2=0x7f9774153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 shutdown_connections 2024-04-25T10:26:11.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.375+0000 7f977aa00640 1 -- 172.21.15.45:0/840985227 wait complete. 2024-04-25T10:26:11.396 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:26:11.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:26:11.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 Processor -- start 2024-04-25T10:26:11.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 -- start start 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc06806b800 0x7fc0680ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc06806bbe0 0x7fc06815d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680cfdb0 con 0x7fc06806b0d0 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680cd030 con 0x7fc06806bbe0 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc06dc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680d0090 con 0x7fc06806b800 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc06806bbe0 0x7fc06815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc06806bbe0 0x7fc06815d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48222/0 (socket says 172.21.15.45:48222) 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 -- 172.21.15.45:0/1708789695 learned_addr learned my addr 172.21.15.45:0/1708789695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc066a00640 1 --2- 172.21.15.45:0/1708789695 >> v2:172.21.15.117:3300/0 conn(0x7fc06806b800 0x7fc0680ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 -- 172.21.15.45:0/1708789695 >> v2:172.21.15.117:3300/0 conn(0x7fc06806b800 msgr2=0x7fc0680ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 --2- 172.21.15.45:0/1708789695 >> v2:172.21.15.117:3300/0 conn(0x7fc06806b800 0x7fc0680ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 -- 172.21.15.45:0/1708789695 >> v1:172.21.15.117:6789/0 conn(0x7fc06806b0d0 legacy=0x7fc0680cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.539+0000 7fc067400640 1 -- 172.21.15.45:0/1708789695 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680d0370 con 0x7fc06806bbe0 2024-04-25T10:26:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc067400640 1 --2- 172.21.15.45:0/1708789695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806bbe0 0x7fc06815d640 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc068071ac0 tx=0x7fc06001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8da9998ebcd0fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc066000640 1 -- 172.21.15.45:0/1708789695 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600b4070 con 0x7fc06806bbe0 2024-04-25T10:26:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc066000640 1 -- 172.21.15.45:0/1708789695 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0600af400 con 0x7fc06806bbe0 2024-04-25T10:26:11.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc066000640 1 -- 172.21.15.45:0/1708789695 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600af7d0 con 0x7fc06806bbe0 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 -- 172.21.15.45:0/1708789695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806bbe0 msgr2=0x7fc06815d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1708789695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806bbe0 0x7fc06815d640 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc068071ac0 tx=0x7fc06001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 -- 172.21.15.45:0/1708789695 shutdown_connections 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1708789695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806bbe0 0x7fc06815d640 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1708789695 >> v2:172.21.15.117:3300/0 conn(0x7fc06806b800 0x7fc0680ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 -- 172.21.15.45:0/1708789695 >> 172.21.15.45:0/1708789695 conn(0x7fc0680d19d0 msgr2=0x7fc0680d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 -- 172.21.15.45:0/1708789695 shutdown_connections 2024-04-25T10:26:11.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.543+0000 7fc06dc00640 1 -- 172.21.15.45:0/1708789695 wait complete. 2024-04-25T10:26:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 Processor -- start 2024-04-25T10:26:11.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 -- start start 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc06806b0d0 0x7fc0680caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 0x7fc0680c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc06806bbe0 0x7fc0680b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680d0e50 con 0x7fc06806b800 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680d0930 con 0x7fc06806b0d0 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680cb010 con 0x7fc06806bbe0 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 0x7fc0680c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc066a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc06806b0d0 0x7fc0680caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 0x7fc0680c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48226/0 (socket says 172.21.15.45:48226) 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 -- 172.21.15.45:0/1021242403 learned_addr learned my addr 172.21.15.45:0/1021242403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:11.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc067400640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc06806bbe0 0x7fc0680b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 -- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc06806bbe0 msgr2=0x7fc0680b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc06806bbe0 0x7fc0680b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 -- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc06806b0d0 msgr2=0x7fc0680caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc06806b0d0 0x7fc0680caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 -- 172.21.15.45:0/1021242403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06806c860 con 0x7fc06806b800 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc066a00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc06806b0d0 0x7fc0680caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05e000640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 0x7fc0680c9ce0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c450 tx=0x7fc05000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050016020 con 0x7fc06806b800 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0500052d0 con 0x7fc06806b800 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050007260 con 0x7fc06806b800 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0680ca220 con 0x7fc06806b800 2024-04-25T10:26:11.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.547+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680c8f20 con 0x7fc06806b800 2024-04-25T10:26:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.551+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06804ef30 con 0x7fc06806b800 2024-04-25T10:26:11.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.551+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc050007400 con 0x7fc06806b800 2024-04-25T10:26:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.551+0000 7fc05f400640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc044066f60 0x7fc044069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.551+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc050084530 con 0x7fc06806b800 2024-04-25T10:26:11.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.551+0000 7fc066a00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc044066f60 0x7fc044069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:11.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.551+0000 7fc066a00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc044066f60 0x7fc044069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc0680ca850 tx=0x7fc058009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:11.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.555+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc050020020 con 0x7fc06806b800 2024-04-25T10:26:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.719+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc0680b7950 con 0x7fc06806b800 2024-04-25T10:26:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.939+0000 7fc05f400640 1 -- 172.21.15.45:0/1021242403 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc0680ca220 con 0x7fc06806b800 2024-04-25T10:26:11.942 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:26:11.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc044066f60 msgr2=0x7fc044069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc044066f60 0x7fc044069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc0680ca850 tx=0x7fc058009190 comp rx=0 tx=0).stop 2024-04-25T10:26:11.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 msgr2=0x7fc0680c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:11.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 0x7fc0680c9ce0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c450 tx=0x7fc05000c910 comp rx=0 tx=0).stop 2024-04-25T10:26:11.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 shutdown_connections 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc044066f60 0x7fc044069420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc06806bbe0 0x7fc0680b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06806b800 0x7fc0680c9ce0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 --2- 172.21.15.45:0/1021242403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc06806b0d0 0x7fc0680caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 >> 172.21.15.45:0/1021242403 conn(0x7fc0680d19d0 msgr2=0x7fc0680ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 shutdown_connections 2024-04-25T10:26:11.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:11.943+0000 7fc06dc00640 1 -- 172.21.15.45:0/1021242403 wait complete. 2024-04-25T10:26:11.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:26:12.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f602aa00640 1 Processor -- start 2024-04-25T10:26:12.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f602aa00640 1 -- start start 2024-04-25T10:26:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f602aa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60240b4660 0x7f602406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f602aa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602406ca90 con 0x7f602415bbb0 2024-04-25T10:26:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f602aa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602406cd70 con 0x7f60240cf960 2024-04-25T10:26:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f602aa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602406d050 con 0x7f60240b4660 2024-04-25T10:26:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f602415bbb0 0x7f60240cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36220/0 (socket says 172.21.15.45:36220) 2024-04-25T10:26:12.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 -- 172.21.15.45:0/1122215566 learned_addr learned my addr 172.21.15.45:0/1122215566 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023400640 1 --2- 172.21.15.45:0/1122215566 >> v2:172.21.15.117:3300/0 conn(0x7f60240b4660 0x7f602406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307888479 0 0) 0x7f602406ca90 con 0x7f602415bbb0 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6008003530 con 0x7f602415bbb0 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3183828594 0 0) 0x7f6008003530 con 0x7f602415bbb0 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 >> v2:172.21.15.117:3300/0 conn(0x7f60240b4660 msgr2=0x7f602406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 --2- 172.21.15.45:0/1122215566 >> v2:172.21.15.117:3300/0 conn(0x7f60240b4660 0x7f602406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 >> v1:172.21.15.117:6789/0 conn(0x7f60240cf960 legacy=0x7f60240b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240b2880 con 0x7f602415bbb0 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6014001680 con 0x7f602415bbb0 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 >> v1:172.21.15.45:6789/0 conn(0x7f602415bbb0 legacy=0x7f60240cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080039c0 0x7f6008003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080051d0 0x7f600800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602406d050 con 0x7f60080039c0 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600800c070 con 0x7f60080051d0 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600800c270 con 0x7f6008004810 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080039c0 0x7f6008003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023400640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6028a00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080051d0 0x7f600800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 -- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6008004810 msgr2=0x7f6008004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6008004810 0x7f6008004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 -- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080051d0 msgr2=0x7f600800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080051d0 0x7f600800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6023e00640 1 -- 172.21.15.45:0/1122215566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008003530 con 0x7f60080039c0 2024-04-25T10:26:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.115+0000 7f6028a00640 1 -- 172.21.15.45:0/1122215566 reap_dead start 2024-04-25T10:26:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f6023e00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080039c0 0x7f6008003de0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f601400c1c0 tx=0x7f601401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9058672cf688138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014007d00 con 0x7f60080039c0 2024-04-25T10:26:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60140a9020 con 0x7f60080039c0 2024-04-25T10:26:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f6022a00640 1 -- 172.21.15.45:0/1122215566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014005ae0 con 0x7f60080039c0 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 -- 172.21.15.45:0/1122215566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080039c0 msgr2=0x7f6008003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080039c0 0x7f6008003de0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f601400c1c0 tx=0x7f601401cc60 comp rx=0 tx=0).stop 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 -- 172.21.15.45:0/1122215566 shutdown_connections 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 --2- 172.21.15.45:0/1122215566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080039c0 0x7f6008003de0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 -- 172.21.15.45:0/1122215566 >> 172.21.15.45:0/1122215566 conn(0x7f6024066b70 msgr2=0x7f6024066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 -- 172.21.15.45:0/1122215566 shutdown_connections 2024-04-25T10:26:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.119+0000 7f602aa00640 1 -- 172.21.15.45:0/1122215566 wait complete. 2024-04-25T10:26:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 Processor -- start 2024-04-25T10:26:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 -- start start 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080039c0 0x7f60241f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f602414a380 0x7f60240703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 0x7f6024152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602406c2a0 con 0x7f6024152670 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602406c120 con 0x7f60080039c0 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602406c420 con 0x7f602414a380 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 0x7f6024152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 0x7f6024152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48248/0 (socket says 172.21.15.45:48248) 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 -- 172.21.15.45:0/3883726178 learned_addr learned my addr 172.21.15.45:0/3883726178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6023e00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080039c0 0x7f60241f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6023400640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f602414a380 0x7f60240703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 -- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f602414a380 msgr2=0x7f60240703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f602414a380 0x7f60240703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 -- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080039c0 msgr2=0x7f60241f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080039c0 0x7f60241f6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 -- 172.21.15.45:0/3883726178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6024070bf0 con 0x7f6024152670 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f6028a00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 0x7f6024152af0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f60240b1470 tx=0x7f601c00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c01a070 con 0x7f6024152670 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00fe50 con 0x7f6024152670 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c003980 con 0x7f6024152670 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6024070ed0 con 0x7f6024152670 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.123+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6024071490 con 0x7f6024152670 2024-04-25T10:26:12.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.127+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602404ef30 con 0x7f6024152670 2024-04-25T10:26:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.127+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f601c003b20 con 0x7f6024152670 2024-04-25T10:26:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.127+0000 7f601be00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6004066f90 0x7f6004069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.127+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f601c084380 con 0x7f6024152670 2024-04-25T10:26:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.127+0000 7f6023e00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6004066f90 0x7f6004069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.127+0000 7f6023e00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6004066f90 0x7f6004069450 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f60140aa000 tx=0x7f602406cc60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.131+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f601c054f90 con 0x7f6024152670 2024-04-25T10:26:12.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.295+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f602406d980 con 0x7f6024152670 2024-04-25T10:26:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.943+0000 7f601be00640 1 -- 172.21.15.45:0/3883726178 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7f601c059030 con 0x7f6024152670 2024-04-25T10:26:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6004066f90 msgr2=0x7f6004069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6004066f90 0x7f6004069450 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f60140aa000 tx=0x7f602406cc60 comp rx=0 tx=0).stop 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 msgr2=0x7f6024152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 0x7f6024152af0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f60240b1470 tx=0x7f601c00fb40 comp rx=0 tx=0).stop 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 shutdown_connections 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6004066f90 0x7f6004069450 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6024152670 0x7f6024152af0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f602414a380 0x7f60240703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 --2- 172.21.15.45:0/3883726178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60080039c0 0x7f60241f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 >> 172.21.15.45:0/3883726178 conn(0x7f6024066b70 msgr2=0x7f60240afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 shutdown_connections 2024-04-25T10:26:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:12.947+0000 7f602aa00640 1 -- 172.21.15.45:0/3883726178 wait complete. 2024-04-25T10:26:12.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:26:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 Processor -- start 2024-04-25T10:26:13.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 -- start start 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd9d8159540 0x7fd9d80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d80aebd0 0x7fd9d8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d8072020 0x7fd9d815f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d8159920 con 0x7fd9d8072020 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d815c2f0 con 0x7fd9d8159540 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9de000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d815c5d0 con 0x7fd9d80aebd0 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d8072020 0x7fd9d815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9cea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d80aebd0 0x7fd9d8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd9d8072020 0x7fd9d815f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48252/0 (socket says 172.21.15.45:48252) 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3343511575 learned_addr learned my addr 172.21.15.45:0/3343511575 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7400640 1 --2- 172.21.15.45:0/3343511575 >> v2:172.21.15.117:3300/0 conn(0x7fd9d8159540 0x7fd9d80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3343511575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d80aebd0 msgr2=0x7fd9d8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 --2- 172.21.15.45:0/3343511575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d80aebd0 0x7fd9d8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3343511575 >> v2:172.21.15.117:3300/0 conn(0x7fd9d8159540 msgr2=0x7fd9d80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 --2- 172.21.15.45:0/3343511575 >> v2:172.21.15.117:3300/0 conn(0x7fd9d8159540 0x7fd9d80ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.135+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3343511575 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d815c8b0 con 0x7fd9d8072020 2024-04-25T10:26:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9d7e00640 1 --2- 172.21.15.45:0/3343511575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8072020 0x7fd9d815f590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80b4c20 tx=0x7fd9d001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13e651389c4e4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9d6a00640 1 -- 172.21.15.45:0/3343511575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00ac020 con 0x7fd9d8072020 2024-04-25T10:26:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9d6a00640 1 -- 172.21.15.45:0/3343511575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d00a6450 con 0x7fd9d8072020 2024-04-25T10:26:13.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9d6a00640 1 -- 172.21.15.45:0/3343511575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00a6820 con 0x7fd9d8072020 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 -- 172.21.15.45:0/3343511575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8072020 msgr2=0x7fd9d815f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 --2- 172.21.15.45:0/3343511575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8072020 0x7fd9d815f590 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80b4c20 tx=0x7fd9d001c730 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 -- 172.21.15.45:0/3343511575 shutdown_connections 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 --2- 172.21.15.45:0/3343511575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8072020 0x7fd9d815f590 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 --2- 172.21.15.45:0/3343511575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d80aebd0 0x7fd9d8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 --2- 172.21.15.45:0/3343511575 >> v2:172.21.15.117:3300/0 conn(0x7fd9d8159540 0x7fd9d80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 -- 172.21.15.45:0/3343511575 >> 172.21.15.45:0/3343511575 conn(0x7fd9d80669a0 msgr2=0x7fd9d8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 -- 172.21.15.45:0/3343511575 shutdown_connections 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.139+0000 7fd9de000640 1 -- 172.21.15.45:0/3343511575 wait complete. 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 Processor -- start 2024-04-25T10:26:13.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 -- start start 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d8072020 0x7fd9d8152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d80aebd0 0x7fd9d80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 0x7fd9d80cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8070350 con 0x7fd9d8159540 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d806c0d0 con 0x7fd9d80aebd0 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80704d0 con 0x7fd9d8072020 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 0x7fd9d80cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 0x7fd9d80cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48268/0 (socket says 172.21.15.45:48268) 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3439110781 learned_addr learned my addr 172.21.15.45:0/3439110781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9cea00640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d80aebd0 0x7fd9d80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7400640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d8072020 0x7fd9d8152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d8072020 msgr2=0x7fd9d8152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d8072020 0x7fd9d8152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d80aebd0 msgr2=0x7fd9d80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d80aebd0 0x7fd9d80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 -- 172.21.15.45:0/3439110781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d81f4de0 con 0x7fd9d8159540 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9cea00640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d80aebd0 0x7fd9d80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7400640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d8072020 0x7fd9d8152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9d7e00640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 0x7fd9d80cdea0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd9d001ade0 tx=0x7fd9d00194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00ac040 con 0x7fd9d8159540 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d00a6cf0 con 0x7fd9d8159540 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d81f50c0 con 0x7fd9d8159540 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0009510 con 0x7fd9d8159540 2024-04-25T10:26:13.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.143+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d81f5680 con 0x7fd9d8159540 2024-04-25T10:26:13.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.147+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d804ef30 con 0x7fd9d8159540 2024-04-25T10:26:13.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.147+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9d00af020 con 0x7fd9d8159540 2024-04-25T10:26:13.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.147+0000 7fd9cfe00640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd9b4066f60 0x7fd9b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:13.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.147+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd9d011f760 con 0x7fd9d8159540 2024-04-25T10:26:13.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.151+0000 7fd9d7400640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd9b4066f60 0x7fd9b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:13.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.151+0000 7fd9d7400640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd9b4066f60 0x7fd9b4069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80099c0 tx=0x7fd9c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:13.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.151+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9d00165e0 con 0x7fd9d8159540 2024-04-25T10:26:13.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.351+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9d80ccef0 con 0x7fd9d8159540 2024-04-25T10:26:13.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.963+0000 7fd9cfe00640 1 -- 172.21.15.45:0/3439110781 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd9d00f6070 con 0x7fd9d8159540 2024-04-25T10:26:13.967 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T10:26:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd9b4066f60 msgr2=0x7fd9b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd9b4066f60 0x7fd9b4069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80099c0 tx=0x7fd9c8008040 comp rx=0 tx=0).stop 2024-04-25T10:26:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 msgr2=0x7fd9d80cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 0x7fd9d80cdea0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd9d001ade0 tx=0x7fd9d00194d0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 shutdown_connections 2024-04-25T10:26:13.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd9b4066f60 0x7fd9b4069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9d8159540 0x7fd9d80cdea0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9d80aebd0 0x7fd9d80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 --2- 172.21.15.45:0/3439110781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9d8072020 0x7fd9d8152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.967+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 >> 172.21.15.45:0/3439110781 conn(0x7fd9d80669a0 msgr2=0x7fd9d806de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.971+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 shutdown_connections 2024-04-25T10:26:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:13.971+0000 7fd9de000640 1 -- 172.21.15.45:0/3439110781 wait complete. 2024-04-25T10:26:14.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:14.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 Processor -- start 2024-04-25T10:26:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 -- start start 2024-04-25T10:26:14.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5c015df00 0x7fc5c00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c00d2490 con 0x7fc5c00d1ce0 2024-04-25T10:26:14.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c00d2770 con 0x7fc5c015df00 2024-04-25T10:26:14.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5c4a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c00d2a50 con 0x7fc5c00d0750 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5c015df00 0x7fc5c00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5b5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc5c015df00 0x7fc5c00d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52276/0 (socket says 172.21.15.45:52276) 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 -- 172.21.15.45:0/3328020635 learned_addr learned my addr 172.21.15.45:0/3328020635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 -- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 msgr2=0x7fc5c00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 --2- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 -- 172.21.15.45:0/3328020635 >> v1:172.21.15.117:6789/0 conn(0x7fc5c00d1ce0 legacy=0x7fc5c00b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.163+0000 7fc5bde00640 1 -- 172.21.15.45:0/3328020635 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00d2d30 con 0x7fc5c015df00 2024-04-25T10:26:14.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5bde00640 1 --2- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c015df00 0x7fc5c00d0210 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009e50 tx=0x7fc5b001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9890991198229b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5bd400640 1 -- 172.21.15.45:0/3328020635 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00a58f0 con 0x7fc5c015df00 2024-04-25T10:26:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5bd400640 1 -- 172.21.15.45:0/3328020635 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b00a5a90 con 0x7fc5c015df00 2024-04-25T10:26:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5bd400640 1 -- 172.21.15.45:0/3328020635 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00a5e60 con 0x7fc5c015df00 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 -- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c015df00 msgr2=0x7fc5c00d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c015df00 0x7fc5c00d0210 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009e50 tx=0x7fc5b001ca00 comp rx=0 tx=0).stop 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 -- 172.21.15.45:0/3328020635 shutdown_connections 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/3328020635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c015df00 0x7fc5c00d0210 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 -- 172.21.15.45:0/3328020635 >> 172.21.15.45:0/3328020635 conn(0x7fc5c006b290 msgr2=0x7fc5c00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 -- 172.21.15.45:0/3328020635 shutdown_connections 2024-04-25T10:26:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.167+0000 7fc5c4a00640 1 -- 172.21.15.45:0/3328020635 wait complete. 2024-04-25T10:26:14.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 Processor -- start 2024-04-25T10:26:14.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 -- start start 2024-04-25T10:26:14.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c00d1ce0 0x7fc5c0153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5c015df00 0x7fc5c0154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c00d2270 con 0x7fc5c015df00 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c00d20f0 con 0x7fc5c00d1ce0 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c016ab70 con 0x7fc5c00d0750 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5be800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5c015df00 0x7fc5c0154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5b5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c00d1ce0 0x7fc5c0153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5b5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c00d1ce0 0x7fc5c0153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52290/0 (socket says 172.21.15.45:52290) 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:51868/0 (socket says 172.21.15.45:51868) 2024-04-25T10:26:14.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5b5400640 1 -- 172.21.15.45:0/930546101 learned_addr learned my addr 172.21.15.45:0/930546101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:14.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 -- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c00d1ce0 msgr2=0x7fc5c0153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c00d1ce0 0x7fc5c0153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 -- 172.21.15.45:0/930546101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5c015df00 msgr2=0x7fc5c0154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5c015df00 0x7fc5c0154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 -- 172.21.15.45:0/930546101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00c9b30 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5bde00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00c8410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00ab000 tx=0x7fc5b00197a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00a58f0 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b0007c90 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0009510 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c00c9e10 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.171+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c00b3de0 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.175+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c004ef30 con 0x7fc5c00d0750 2024-04-25T10:26:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.175+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5b00c0020 con 0x7fc5c00d0750 2024-04-25T10:26:14.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.175+0000 7fc5b6a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc59c066f60 0x7fc59c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.175+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5b0120310 con 0x7fc5c00d0750 2024-04-25T10:26:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.175+0000 7fc5b5400640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc59c066f60 0x7fc59c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.175+0000 7fc5b5400640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc59c066f60 0x7fc59c069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0153220 tx=0x7fc5a8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.179+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5b0016620 con 0x7fc5c00d0750 2024-04-25T10:26:14.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.343+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5c00c9fe0 con 0x7fc5c00d0750 2024-04-25T10:26:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.343+0000 7fc5b6a00640 1 -- 172.21.15.45:0/930546101 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16684 (secure 0 0 0) 0x7fc5b00f0e90 con 0x7fc5c00d0750 2024-04-25T10:26:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:14.346 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":116,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:13.356048+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc59c066f60 msgr2=0x7fc59c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc59c066f60 0x7fc59c069420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0153220 tx=0x7fc5a8009190 comp rx=0 tx=0).stop 2024-04-25T10:26:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 msgr2=0x7fc5c00c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00c8410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00ab000 tx=0x7fc5b00197a0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 shutdown_connections 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc59c066f60 0x7fc59c069420 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5c015df00 0x7fc5c0154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5c00d1ce0 0x7fc5c0153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 --2- 172.21.15.45:0/930546101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5c00d0750 0x7fc5c00c8410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 >> 172.21.15.45:0/930546101 conn(0x7fc5c006b290 msgr2=0x7fc5c0152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 shutdown_connections 2024-04-25T10:26:14.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.347+0000 7fc5c4a00640 1 -- 172.21.15.45:0/930546101 wait complete. 2024-04-25T10:26:14.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.507+0000 7f63a9200640 1 Processor -- start 2024-04-25T10:26:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.507+0000 7f63a9200640 1 -- start start 2024-04-25T10:26:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a9200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63a41550b0 0x7f63a40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a9200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f63a415b910 0x7f63a40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a9200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a4071050 con 0x7f63a40cfe70 2024-04-25T10:26:14.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a9200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a415bd80 con 0x7f63a41550b0 2024-04-25T10:26:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a9200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a415d040 con 0x7f63a415b910 2024-04-25T10:26:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63a41550b0 0x7f63a40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f6399400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63a40cfe70 0x7f63a415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36222/0 (socket says 172.21.15.45:36222) 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f6399400640 1 -- 172.21.15.45:0/52404312 learned_addr learned my addr 172.21.15.45:0/52404312 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63a41550b0 0x7f63a40cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48292/0 (socket says 172.21.15.45:48292) 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1e00640 1 -- 172.21.15.45:0/52404312 >> v2:172.21.15.117:3300/0 conn(0x7f63a415b910 msgr2=0x7f63a40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a2800640 1 --2- 172.21.15.45:0/52404312 >> v2:172.21.15.117:3300/0 conn(0x7f63a415b910 0x7f63a40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1e00640 1 --2- 172.21.15.45:0/52404312 >> v2:172.21.15.117:3300/0 conn(0x7f63a415b910 0x7f63a40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1400640 1 -- 172.21.15.45:0/52404312 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254615835 0 0) 0x7f63a4071050 con 0x7f63a40cfe70 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1e00640 1 -- 172.21.15.45:0/52404312 >> v1:172.21.15.45:6789/0 conn(0x7f63a40cfe70 legacy=0x7f63a415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a1e00640 1 -- 172.21.15.45:0/52404312 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40712f0 con 0x7f63a41550b0 2024-04-25T10:26:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.511+0000 7f63a2800640 1 --2- 172.21.15.45:0/52404312 >> v2:172.21.15.117:3300/0 conn(0x7f63a415b910 0x7f63a40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a1e00640 1 --2- 172.21.15.45:0/52404312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a40cf930 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f63940097c0 tx=0x7f639401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f0b964e4762b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a1400640 1 -- 172.21.15.45:0/52404312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940b0070 con 0x7f63a41550b0 2024-04-25T10:26:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a1400640 1 -- 172.21.15.45:0/52404312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394004230 con 0x7f63a41550b0 2024-04-25T10:26:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a1400640 1 -- 172.21.15.45:0/52404312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394003620 con 0x7f63a41550b0 2024-04-25T10:26:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 -- 172.21.15.45:0/52404312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 msgr2=0x7f63a40cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 --2- 172.21.15.45:0/52404312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a40cf930 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f63940097c0 tx=0x7f639401c460 comp rx=0 tx=0).stop 2024-04-25T10:26:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 -- 172.21.15.45:0/52404312 shutdown_connections 2024-04-25T10:26:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 --2- 172.21.15.45:0/52404312 >> v2:172.21.15.117:3300/0 conn(0x7f63a415b910 0x7f63a40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 --2- 172.21.15.45:0/52404312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a40cf930 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 -- 172.21.15.45:0/52404312 >> 172.21.15.45:0/52404312 conn(0x7f63a4066b70 msgr2=0x7f63a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 -- 172.21.15.45:0/52404312 shutdown_connections 2024-04-25T10:26:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 -- 172.21.15.45:0/52404312 wait complete. 2024-04-25T10:26:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 Processor -- start 2024-04-25T10:26:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.515+0000 7f63a9200640 1 -- start start 2024-04-25T10:26:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a40cfe70 0x7f63a415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a4159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a415b910 0x7f63a4137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a415c030 con 0x7f63a41550b0 2024-04-25T10:26:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a40720b0 con 0x7f63a415b910 2024-04-25T10:26:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a415c1b0 con 0x7f63a40cfe70 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a4159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a4159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48308/0 (socket says 172.21.15.45:48308) 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 -- 172.21.15.45:0/3215031762 learned_addr learned my addr 172.21.15.45:0/3215031762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a2800640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a415b910 0x7f63a4137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a1e00640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a40cfe70 0x7f63a415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 -- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a40cfe70 msgr2=0x7f63a415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a40cfe70 0x7f63a415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 -- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a415b910 msgr2=0x7f63a4137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a415b910 0x7f63a4137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 -- 172.21.15.45:0/3215031762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a414fb50 con 0x7f63a41550b0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f6399400640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a4159410 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c890 tx=0x7f638c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c018070 con 0x7f63a41550b0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00a640 con 0x7f63a41550b0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a414f2d0 con 0x7f63a41550b0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00aa30 con 0x7f63a41550b0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a414f870 con 0x7f63a41550b0 2024-04-25T10:26:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a404ef30 con 0x7f63a41550b0 2024-04-25T10:26:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.519+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f638c01b070 con 0x7f63a41550b0 2024-04-25T10:26:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.523+0000 7f639aa00640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6380066f60 0x7f6380069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.523+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f638c084750 con 0x7f63a41550b0 2024-04-25T10:26:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.523+0000 7f63a1e00640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6380066f60 0x7f6380069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.523+0000 7f63a1e00640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6380066f60 0x7f6380069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f63940ad000 tx=0x7f6394008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.527+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63a414fb50 con 0x7f63a41550b0 2024-04-25T10:26:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.691+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63a40cc300 con 0x7f63a41550b0 2024-04-25T10:26:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f639aa00640 1 -- 172.21.15.45:0/3215031762 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f638c054fe0 con 0x7f63a41550b0 2024-04-25T10:26:14.696 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:14.696 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":116,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:13.356048+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6380066f60 msgr2=0x7f6380069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6380066f60 0x7f6380069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f63940ad000 tx=0x7f6394008040 comp rx=0 tx=0).stop 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 msgr2=0x7f63a4159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a4159410 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c890 tx=0x7f638c00cd50 comp rx=0 tx=0).stop 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 shutdown_connections 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6380066f60 0x7f6380069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63a415b910 0x7f63a4137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63a41550b0 0x7f63a4159410 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 --2- 172.21.15.45:0/3215031762 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63a40cfe70 0x7f63a415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 >> 172.21.15.45:0/3215031762 conn(0x7f63a4066b70 msgr2=0x7f63a406c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 shutdown_connections 2024-04-25T10:26:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.695+0000 7f63a9200640 1 -- 172.21.15.45:0/3215031762 wait complete. 2024-04-25T10:26:14.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:26:14.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6847800640 1 Processor -- start 2024-04-25T10:26:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6847800640 1 -- start start 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6847800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6840070350 0x7f684015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6847800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684015b080 con 0x7f68400aef80 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6847800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684015adb0 con 0x7f68400b31c0 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6847800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684015eb60 con 0x7f6840070350 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6844e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f68400aef80 0x7f684015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36224/0 (socket says 172.21.15.45:36224) 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f6844e00640 1 -- 172.21.15.45:0/2033587015 learned_addr learned my addr 172.21.15.45:0/2033587015 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.859+0000 7f683fe00640 1 --2- 172.21.15.45:0/2033587015 >> v2:172.21.15.45:3300/0 conn(0x7f6840070350 0x7f684015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683fe00640 1 -- 172.21.15.45:0/2033587015 >> v1:172.21.15.117:6789/0 conn(0x7f68400b31c0 legacy=0x7f68400cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683fe00640 1 -- 172.21.15.45:0/2033587015 >> v1:172.21.15.45:6789/0 conn(0x7f68400aef80 legacy=0x7f684015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683f400640 1 -- 172.21.15.45:0/2033587015 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766184696 0 0) 0x7f684015b080 con 0x7f68400aef80 2024-04-25T10:26:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683fe00640 1 -- 172.21.15.45:0/2033587015 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684015ee40 con 0x7f6840070350 2024-04-25T10:26:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683fe00640 1 --2- 172.21.15.45:0/2033587015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f684015e480 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f682c0097c0 tx=0x7f682c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c222a019d99bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683f400640 1 -- 172.21.15.45:0/2033587015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0b2070 con 0x7f6840070350 2024-04-25T10:26:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683f400640 1 -- 172.21.15.45:0/2033587015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0ad680 con 0x7f6840070350 2024-04-25T10:26:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f683f400640 1 -- 172.21.15.45:0/2033587015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0ada50 con 0x7f6840070350 2024-04-25T10:26:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 -- 172.21.15.45:0/2033587015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 msgr2=0x7f684015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 --2- 172.21.15.45:0/2033587015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f684015e480 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f682c0097c0 tx=0x7f682c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 -- 172.21.15.45:0/2033587015 shutdown_connections 2024-04-25T10:26:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 --2- 172.21.15.45:0/2033587015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f684015e480 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 -- 172.21.15.45:0/2033587015 >> 172.21.15.45:0/2033587015 conn(0x7f68400669a0 msgr2=0x7f6840066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 -- 172.21.15.45:0/2033587015 shutdown_connections 2024-04-25T10:26:14.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.863+0000 7f6847800640 1 -- 172.21.15.45:0/2033587015 wait complete. 2024-04-25T10:26:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 Processor -- start 2024-04-25T10:26:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 -- start start 2024-04-25T10:26:14.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f68400b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68400aef80 0x7f6840144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68400b31c0 0x7f684015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400cfef0 con 0x7f6840070350 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68400cfd70 con 0x7f68400aef80 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68400b35d0 con 0x7f68400b31c0 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f68400b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f68400b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48330/0 (socket says 172.21.15.45:48330) 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 -- 172.21.15.45:0/267700267 learned_addr learned my addr 172.21.15.45:0/267700267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f683fe00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68400aef80 0x7f6840144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6845800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68400b31c0 0x7f684015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 -- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68400b31c0 msgr2=0x7f684015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68400b31c0 0x7f684015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 -- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68400aef80 msgr2=0x7f6840144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68400aef80 0x7f6840144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 -- 172.21.15.45:0/267700267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840145030 con 0x7f6840070350 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f683fe00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68400aef80 0x7f6840144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:14.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6844e00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f68400b1b20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f683400c4a0 tx=0x7f683400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834018070 con 0x7f6840070350 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68340074e0 con 0x7f6840070350 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834004a30 con 0x7f6840070350 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.867+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68400b21b0 con 0x7f6840070350 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68400af580 con 0x7f6840070350 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684004ef30 con 0x7f6840070350 2024-04-25T10:26:14.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6834004bd0 con 0x7f6840070350 2024-04-25T10:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f683cc00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6820066f60 0x7f6820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6834084400 con 0x7f6840070350 2024-04-25T10:26:14.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f683fe00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6820066f60 0x7f6820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:14.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.871+0000 7f683fe00640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6820066f60 0x7f6820069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f682c0af000 tx=0x7f682c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:14.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:14.875+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6840145030 con 0x7f6840070350 2024-04-25T10:26:15.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.079+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f68400b2fa0 con 0x7f6840070350 2024-04-25T10:26:15.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f683cc00640 1 -- 172.21.15.45:0/267700267 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6834055370 con 0x7f6840070350 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6820066f60 msgr2=0x7f6820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6820066f60 0x7f6820069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f682c0af000 tx=0x7f682c008040 comp rx=0 tx=0).stop 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 msgr2=0x7f68400b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f68400b1b20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f683400c4a0 tx=0x7f683400c960 comp rx=0 tx=0).stop 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 shutdown_connections 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6820066f60 0x7f6820069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68400b31c0 0x7f684015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68400aef80 0x7f6840144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 --2- 172.21.15.45:0/267700267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6840070350 0x7f68400b1b20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 >> 172.21.15.45:0/267700267 conn(0x7f68400669a0 msgr2=0x7f6840151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 shutdown_connections 2024-04-25T10:26:16.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:15.995+0000 7f6847800640 1 -- 172.21.15.45:0/267700267 wait complete. 2024-04-25T10:26:16.023 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:26:16.023 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:26:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f7712000640 1 Processor -- start 2024-04-25T10:26:16.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f7712000640 1 -- start start 2024-04-25T10:26:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f7712000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f7712000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c071360 con 0x7f770c159540 2024-04-25T10:26:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f7712000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770c159920 con 0x7f770c0aeda0 2024-04-25T10:26:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f7712000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c15c2f0 con 0x7f770c0721f0 2024-04-25T10:26:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f770c159540 0x7f770c0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60806/0 (socket says 172.21.15.45:60806) 2024-04-25T10:26:16.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770b400640 1 -- 172.21.15.45:0/3038113542 learned_addr learned my addr 172.21.15.45:0/3038113542 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770be00640 1 --2- 172.21.15.45:0/3038113542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424625387 0 0) 0x7f770c071360 con 0x7f770c159540 2024-04-25T10:26:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76f0003530 con 0x7f770c159540 2024-04-25T10:26:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161034983 0 0) 0x7f770c159920 con 0x7f770c0aeda0 2024-04-25T10:26:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f770c071360 con 0x7f770c0aeda0 2024-04-25T10:26:16.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770be00640 1 -- 172.21.15.45:0/3038113542 >> v1:172.21.15.117:6789/0 conn(0x7f770c0aeda0 legacy=0x7f770c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770be00640 1 -- 172.21.15.45:0/3038113542 >> v1:172.21.15.45:6789/0 conn(0x7f770c159540 legacy=0x7f770c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.171+0000 7f770be00640 1 -- 172.21.15.45:0/3038113542 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c15c5d0 con 0x7f770c0721f0 2024-04-25T10:26:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f770be00640 1 --2- 172.21.15.45:0/3038113542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c166920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7704009dc0 tx=0x7f770401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77255c6e0f1ac441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040b2070 con 0x7f770c0721f0 2024-04-25T10:26:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77040a5a20 con 0x7f770c0721f0 2024-04-25T10:26:16.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f770aa00640 1 -- 172.21.15.45:0/3038113542 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040a5dd0 con 0x7f770c0721f0 2024-04-25T10:26:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 -- 172.21.15.45:0/3038113542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 msgr2=0x7f770c166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 --2- 172.21.15.45:0/3038113542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c166920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7704009dc0 tx=0x7f770401c960 comp rx=0 tx=0).stop 2024-04-25T10:26:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 -- 172.21.15.45:0/3038113542 shutdown_connections 2024-04-25T10:26:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 --2- 172.21.15.45:0/3038113542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c166920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 -- 172.21.15.45:0/3038113542 >> 172.21.15.45:0/3038113542 conn(0x7f770c0669a0 msgr2=0x7f770c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 -- 172.21.15.45:0/3038113542 shutdown_connections 2024-04-25T10:26:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.175+0000 7f7712000640 1 -- 172.21.15.45:0/3038113542 wait complete. 2024-04-25T10:26:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 Processor -- start 2024-04-25T10:26:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 -- start start 2024-04-25T10:26:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f770c0aeda0 0x7f770c1f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:16.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 0x7f770c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c169680 con 0x7f770c159540 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c06c0d0 con 0x7f770c0aeda0 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c169800 con 0x7f770c0721f0 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 0x7f770c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 0x7f770c15a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37584/0 (socket says 172.21.15.45:37584) 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 -- 172.21.15.45:0/2118427518 learned_addr learned my addr 172.21.15.45:0/2118427518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 -- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 msgr2=0x7f770c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 -- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f770c0aeda0 msgr2=0x7f770c1f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f770c0aeda0 0x7f770c1f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 -- 172.21.15.45:0/2118427518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0b1be0 con 0x7f770c159540 2024-04-25T10:26:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f770be00640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 0x7f770c15a360 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f770401a590 tx=0x7f7704019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040b2050 con 0x7f770c159540 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77040b3040 con 0x7f770c159540 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704004040 con 0x7f770c159540 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c0b1e80 con 0x7f770c159540 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.179+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c0b2440 con 0x7f770c159540 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.183+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c04ef30 con 0x7f770c159540 2024-04-25T10:26:16.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.183+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77040c0020 con 0x7f770c159540 2024-04-25T10:26:16.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.183+0000 7f7703400640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76e8066f60 0x7f76e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:16.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.183+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f770411f810 con 0x7f770c159540 2024-04-25T10:26:16.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.183+0000 7f770b400640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76e8066f60 0x7f76e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:16.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.187+0000 7f770b400640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76e8066f60 0x7f76e8069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f76fc007fd0 tx=0x7f76fc007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.187+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77040c7020 con 0x7f770c159540 2024-04-25T10:26:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.387+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f770c0aeb80 con 0x7f770c159540 2024-04-25T10:26:16.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.995+0000 7f7703400640 1 -- 172.21.15.45:0/2118427518 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7704016580 con 0x7f770c159540 2024-04-25T10:26:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76e8066f60 msgr2=0x7f76e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76e8066f60 0x7f76e8069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f76fc007fd0 tx=0x7f76fc007960 comp rx=0 tx=0).stop 2024-04-25T10:26:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 msgr2=0x7f770c15a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 0x7f770c15a360 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f770401a590 tx=0x7f7704019700 comp rx=0 tx=0).stop 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 shutdown_connections 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76e8066f60 0x7f76e8069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f770c159540 0x7f770c15a360 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f770c0aeda0 0x7f770c1f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 --2- 172.21.15.45:0/2118427518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f770c0721f0 0x7f770c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 >> 172.21.15.45:0/2118427518 conn(0x7f770c0669a0 msgr2=0x7f770c070b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 shutdown_connections 2024-04-25T10:26:17.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:16.999+0000 7f7712000640 1 -- 172.21.15.45:0/2118427518 wait complete. 2024-04-25T10:26:17.025 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:26:17.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 Processor -- start 2024-04-25T10:26:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 -- start start 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c1550b0 0x7ff60c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff60c15b910 0x7ff60c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c071050 con 0x7ff60c0cfe70 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c15bda0 con 0x7ff60c15b910 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff611400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15d040 con 0x7ff60c1550b0 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff60c15b910 0x7ff60c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff60c15b910 0x7ff60c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37598/0 (socket says 172.21.15.45:37598) 2024-04-25T10:26:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 -- 172.21.15.45:0/2099577653 learned_addr learned my addr 172.21.15.45:0/2099577653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60a800640 1 --2- 172.21.15.45:0/2099577653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c1550b0 0x7ff60c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 -- 172.21.15.45:0/2099577653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c1550b0 msgr2=0x7ff60c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 --2- 172.21.15.45:0/2099577653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c1550b0 0x7ff60c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 -- 172.21.15.45:0/2099577653 >> v1:172.21.15.45:6789/0 conn(0x7ff60c0cfe70 legacy=0x7ff60c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.171+0000 7ff60b200640 1 -- 172.21.15.45:0/2099577653 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0712f0 con 0x7ff60c15b910 2024-04-25T10:26:17.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff60b200640 1 --2- 172.21.15.45:0/2099577653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c15b910 0x7ff60c0b3260 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff604009870 tx=0x7ff60401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3904b061e19de01d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff609e00640 1 -- 172.21.15.45:0/2099577653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040ab650 con 0x7ff60c15b910 2024-04-25T10:26:17.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff609e00640 1 -- 172.21.15.45:0/2099577653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6040b0070 con 0x7ff60c15b910 2024-04-25T10:26:17.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff609e00640 1 -- 172.21.15.45:0/2099577653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040aba10 con 0x7ff60c15b910 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 -- 172.21.15.45:0/2099577653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c15b910 msgr2=0x7ff60c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 --2- 172.21.15.45:0/2099577653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c15b910 0x7ff60c0b3260 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff604009870 tx=0x7ff60401c560 comp rx=0 tx=0).stop 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 -- 172.21.15.45:0/2099577653 shutdown_connections 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 --2- 172.21.15.45:0/2099577653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c15b910 0x7ff60c0b3260 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 --2- 172.21.15.45:0/2099577653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c1550b0 0x7ff60c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 -- 172.21.15.45:0/2099577653 >> 172.21.15.45:0/2099577653 conn(0x7ff60c066b70 msgr2=0x7ff60c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 -- 172.21.15.45:0/2099577653 shutdown_connections 2024-04-25T10:26:17.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.175+0000 7ff611400640 1 -- 172.21.15.45:0/2099577653 wait complete. 2024-04-25T10:26:17.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 Processor -- start 2024-04-25T10:26:17.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- start start 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 0x7ff60c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff60c1550b0 0x7ff60c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c15b910 0x7ff60c1f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15c030 con 0x7ff60c0cfe70 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0720b0 con 0x7ff60c1550b0 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15c1b0 con 0x7ff60c15b910 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 0x7ff60c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 0x7ff60c0cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37606/0 (socket says 172.21.15.45:37606) 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 -- 172.21.15.45:0/3033328074 learned_addr learned my addr 172.21.15.45:0/3033328074 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff601e00640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff60c1550b0 0x7ff60c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60b200640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c15b910 0x7ff60c1f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 -- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c15b910 msgr2=0x7ff60c1f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c15b910 0x7ff60c1f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 -- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff60c1550b0 msgr2=0x7ff60c1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff60c1550b0 0x7ff60c1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 -- 172.21.15.45:0/3033328074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c13fdc0 con 0x7ff60c0cfe70 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff60a800640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 0x7ff60c0cecc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d830 tx=0x7ff5fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00b6c0 con 0x7ff60c0cfe70 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc019070 con 0x7ff60c0cfe70 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00ba60 con 0x7ff60c0cfe70 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1400a0 con 0x7ff60c0cfe70 2024-04-25T10:26:17.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c151e90 con 0x7ff60c0cfe70 2024-04-25T10:26:17.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.179+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c04ef30 con 0x7ff60c0cfe70 2024-04-25T10:26:17.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.183+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5fc01e460 con 0x7ff60c0cfe70 2024-04-25T10:26:17.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.183+0000 7ff603400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5e8066f60 0x7ff5e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.183+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5fc085760 con 0x7ff60c0cfe70 2024-04-25T10:26:17.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.183+0000 7ff601e00640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5e8066f60 0x7ff5e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.183+0000 7ff601e00640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5e8066f60 0x7ff5e8069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff60c1fa530 tx=0x7ff5f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.187+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5fc0566d0 con 0x7ff60c0cfe70 2024-04-25T10:26:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.383+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff60c154870 con 0x7ff60c0cfe70 2024-04-25T10:26:17.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.383+0000 7ff603400640 1 -- 172.21.15.45:0/3033328074 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff5fc055fd0 con 0x7ff60c0cfe70 2024-04-25T10:26:17.387 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:17.387 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5e8066f60 msgr2=0x7ff5e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5e8066f60 0x7ff5e8069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff60c1fa530 tx=0x7ff5f4009040 comp rx=0 tx=0).stop 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 msgr2=0x7ff60c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 0x7ff60c0cecc0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d830 tx=0x7ff5fc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 shutdown_connections 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5e8066f60 0x7ff5e8069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff60c15b910 0x7ff60c1f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff60c1550b0 0x7ff60c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 --2- 172.21.15.45:0/3033328074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff60c0cfe70 0x7ff60c0cecc0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 >> 172.21.15.45:0/3033328074 conn(0x7ff60c066b70 msgr2=0x7ff60c0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 shutdown_connections 2024-04-25T10:26:17.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.387+0000 7ff611400640 1 -- 172.21.15.45:0/3033328074 wait complete. 2024-04-25T10:26:17.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:26:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90bca00640 1 Processor -- start 2024-04-25T10:26:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90bca00640 1 -- start start 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90bca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b80d0040 0x7f90b806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90bca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b80b2770 con 0x7f90b815bc40 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90bca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b806c670 con 0x7f90b80cda10 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90bca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b806c950 con 0x7f90b80d0040 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b80d0040 0x7f90b806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f90b815bc40 0x7f90b80b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60828/0 (socket says 172.21.15.45:60828) 2024-04-25T10:26:17.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b5e00640 1 -- 172.21.15.45:0/2694487445 learned_addr learned my addr 172.21.15.45:0/2694487445 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b6800640 1 -- 172.21.15.45:0/2694487445 >> v1:172.21.15.117:6789/0 conn(0x7f90b80cda10 legacy=0x7f90b80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b6800640 1 -- 172.21.15.45:0/2694487445 >> v1:172.21.15.45:6789/0 conn(0x7f90b815bc40 legacy=0x7f90b80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b6800640 1 -- 172.21.15.45:0/2694487445 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b806cc30 con 0x7f90b80d0040 2024-04-25T10:26:17.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.547+0000 7f90b5400640 1 -- 172.21.15.45:0/2694487445 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993116881 0 0) 0x7f90b80b2770 con 0x7f90b815bc40 2024-04-25T10:26:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90b6800640 1 --2- 172.21.15.45:0/2694487445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b80d0040 0x7f90b806c090 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d90 tx=0x7f90b001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e057e4a5536c3ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90b5400640 1 -- 172.21.15.45:0/2694487445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00b2070 con 0x7f90b80d0040 2024-04-25T10:26:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90b5400640 1 -- 172.21.15.45:0/2694487445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00a58b0 con 0x7f90b80d0040 2024-04-25T10:26:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90b5400640 1 -- 172.21.15.45:0/2694487445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00a5c80 con 0x7f90b80d0040 2024-04-25T10:26:17.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 -- 172.21.15.45:0/2694487445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b80d0040 msgr2=0x7f90b806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 --2- 172.21.15.45:0/2694487445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b80d0040 0x7f90b806c090 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d90 tx=0x7f90b001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 -- 172.21.15.45:0/2694487445 shutdown_connections 2024-04-25T10:26:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 --2- 172.21.15.45:0/2694487445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b80d0040 0x7f90b806c090 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 -- 172.21.15.45:0/2694487445 >> 172.21.15.45:0/2694487445 conn(0x7f90b8066b70 msgr2=0x7f90b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 -- 172.21.15.45:0/2694487445 shutdown_connections 2024-04-25T10:26:17.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.551+0000 7f90bca00640 1 -- 172.21.15.45:0/2694487445 wait complete. 2024-04-25T10:26:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 Processor -- start 2024-04-25T10:26:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- start start 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b80cda10 0x7f90b80a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b80d0040 0x7f90b813bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 0x7f90b8159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8169980 con 0x7f90b815bc40 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8169800 con 0x7f90b80d0040 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8169b00 con 0x7f90b80cda10 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 0x7f90b8159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 0x7f90b8159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37622/0 (socket says 172.21.15.45:37622) 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 -- 172.21.15.45:0/968992655 learned_addr learned my addr 172.21.15.45:0/968992655 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b5e00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b80cda10 0x7f90b80a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90ad400640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b80d0040 0x7f90b813bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 -- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b80cda10 msgr2=0x7f90b80a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b80cda10 0x7f90b80a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 -- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b80d0040 msgr2=0x7f90b813bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b80d0040 0x7f90b813bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 -- 172.21.15.45:0/968992655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b813c930 con 0x7f90b815bc40 2024-04-25T10:26:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90b6800640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 0x7f90b8159570 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d90 tx=0x7f90b0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00bd030 con 0x7f90b815bc40 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00ad870 con 0x7f90b815bc40 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00b2040 con 0x7f90b815bc40 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b813cc10 con 0x7f90b815bc40 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b813d1d0 con 0x7f90b815bc40 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.555+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b804ef30 con 0x7f90b815bc40 2024-04-25T10:26:17.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.559+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90b00c0070 con 0x7f90b815bc40 2024-04-25T10:26:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.559+0000 7f90aea00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9094066f60 0x7f9094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.559+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90b011fef0 con 0x7f90b815bc40 2024-04-25T10:26:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.559+0000 7f90b5e00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9094066f60 0x7f9094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.559+0000 7f90b5e00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9094066f60 0x7f9094069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007fd0 tx=0x7f90a8007ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.563+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90b00c7070 con 0x7f90b815bc40 2024-04-25T10:26:17.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.735+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90b80cf1b0 con 0x7f90b815bc40 2024-04-25T10:26:17.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.735+0000 7f90aea00640 1 -- 172.21.15.45:0/968992655 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f90b0016560 con 0x7f90b815bc40 2024-04-25T10:26:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:17.739 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":117,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:14.978281+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9094066f60 msgr2=0x7f9094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9094066f60 0x7f9094069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007fd0 tx=0x7f90a8007ba0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 msgr2=0x7f90b8159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 0x7f90b8159570 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009d90 tx=0x7f90b0019680 comp rx=0 tx=0).stop 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 shutdown_connections 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9094066f60 0x7f9094069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b815bc40 0x7f90b8159570 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b80d0040 0x7f90b813bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 --2- 172.21.15.45:0/968992655 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b80cda10 0x7f90b80a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 >> 172.21.15.45:0/968992655 conn(0x7f90b8066b70 msgr2=0x7f90b8066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 shutdown_connections 2024-04-25T10:26:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.739+0000 7f90bca00640 1 -- 172.21.15.45:0/968992655 wait complete. 2024-04-25T10:26:17.761 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:26:17.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.899+0000 7f76a5000640 1 Processor -- start 2024-04-25T10:26:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.899+0000 7f76a5000640 1 -- start start 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f76a5000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76a015b9e0 0x7f76a00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f76a5000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f76a00d1c30 0x7f76a00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f76a5000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a00ccfb0 con 0x7f76a015bdc0 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f76a5000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00cd290 con 0x7f76a015b9e0 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f76a5000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00cd570 con 0x7f76a00d1c30 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76a015b9e0 0x7f76a00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76a015b9e0 0x7f76a00cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37624/0 (socket says 172.21.15.45:37624) 2024-04-25T10:26:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 -- 172.21.15.45:0/3466334455 learned_addr learned my addr 172.21.15.45:0/3466334455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769e800640 1 --2- 172.21.15.45:0/3466334455 >> v2:172.21.15.117:3300/0 conn(0x7f76a00d1c30 0x7f76a00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 -- 172.21.15.45:0/3466334455 >> v2:172.21.15.117:3300/0 conn(0x7f76a00d1c30 msgr2=0x7f76a00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 --2- 172.21.15.45:0/3466334455 >> v2:172.21.15.117:3300/0 conn(0x7f76a00d1c30 0x7f76a00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 -- 172.21.15.45:0/3466334455 >> v1:172.21.15.117:6789/0 conn(0x7f76a015bdc0 legacy=0x7f76a00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 -- 172.21.15.45:0/3466334455 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00cd850 con 0x7f76a015b9e0 2024-04-25T10:26:17.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769e800640 1 --2- 172.21.15.45:0/3466334455 >> v2:172.21.15.117:3300/0 conn(0x7f76a00d1c30 0x7f76a00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.903+0000 7f769de00640 1 --2- 172.21.15.45:0/3466334455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015b9e0 0x7f76a00cc410 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f76a0070e60 tx=0x7f769001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255b506a068c2642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769d400640 1 -- 172.21.15.45:0/3466334455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900b0070 con 0x7f76a015b9e0 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769d400640 1 -- 172.21.15.45:0/3466334455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900ab600 con 0x7f76a015b9e0 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769d400640 1 -- 172.21.15.45:0/3466334455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900ab9d0 con 0x7f76a015b9e0 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- 172.21.15.45:0/3466334455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015b9e0 msgr2=0x7f76a00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 --2- 172.21.15.45:0/3466334455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015b9e0 0x7f76a00cc410 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f76a0070e60 tx=0x7f769001c450 comp rx=0 tx=0).stop 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- 172.21.15.45:0/3466334455 shutdown_connections 2024-04-25T10:26:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 --2- 172.21.15.45:0/3466334455 >> v2:172.21.15.117:3300/0 conn(0x7f76a00d1c30 0x7f76a00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 --2- 172.21.15.45:0/3466334455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015b9e0 0x7f76a00cc410 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- 172.21.15.45:0/3466334455 >> 172.21.15.45:0/3466334455 conn(0x7f76a009a9d0 msgr2=0x7f76a009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- 172.21.15.45:0/3466334455 shutdown_connections 2024-04-25T10:26:17.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- 172.21.15.45:0/3466334455 wait complete. 2024-04-25T10:26:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 Processor -- start 2024-04-25T10:26:17.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- start start 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00d1c30 0x7f76a01571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a015b9e0 0x7f76a0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 0x7f76a006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00c6630 con 0x7f76a015bdc0 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00c64b0 con 0x7f76a00d1c30 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f76a5000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00c67b0 con 0x7f76a015b9e0 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 0x7f76a006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 0x7f76a006bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37634/0 (socket says 172.21.15.45:37634) 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769e800640 1 -- 172.21.15.45:0/3090666937 learned_addr learned my addr 172.21.15.45:0/3090666937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.907+0000 7f769de00640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00d1c30 0x7f76a01571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7695400640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a015b9e0 0x7f76a0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f769e800640 1 -- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a015b9e0 msgr2=0x7f76a0159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f769e800640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a015b9e0 0x7f76a0159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f769e800640 1 -- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00d1c30 msgr2=0x7f76a01571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:17.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f769e800640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00d1c30 0x7f76a01571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f769e800640 1 -- 172.21.15.45:0/3090666937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00ca1e0 con 0x7f76a015bdc0 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f769e800640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 0x7f76a006bce0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f769800cf20 tx=0x7f769800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769801a070 con 0x7f76a015bdc0 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800d840 con 0x7f76a015bdc0 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800dc30 con 0x7f76a015bdc0 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a00ca4c0 con 0x7f76a015bdc0 2024-04-25T10:26:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a00caa80 con 0x7f76a015bdc0 2024-04-25T10:26:17.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a004ef30 con 0x7f76a015bdc0 2024-04-25T10:26:17.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f769801d070 con 0x7f76a015bdc0 2024-04-25T10:26:17.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7696a00640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f767c066f90 0x7f767c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:17.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.911+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76980858e0 con 0x7f76a015bdc0 2024-04-25T10:26:17.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.915+0000 7f769de00640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f767c066f90 0x7f767c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:17.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.915+0000 7f769de00640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f767c066f90 0x7f767c069450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f76900ad000 tx=0x7f7690008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:17.915+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76980561f0 con 0x7f76a015bdc0 2024-04-25T10:26:18.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.083+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76a00cdbc0 con 0x7f767c066f90 2024-04-25T10:26:18.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.087+0000 7f7696a00640 1 -- 172.21.15.45:0/3090666937 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+245874 (secure 0 0 0) 0x7f76a00cdbc0 con 0x7f767c066f90 2024-04-25T10:26:18.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:18.095 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:26:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f767c066f90 msgr2=0x7f767c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f767c066f90 0x7f767c069450 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f76900ad000 tx=0x7f7690008040 comp rx=0 tx=0).stop 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 msgr2=0x7f76a006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 0x7f76a006bce0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f769800cf20 tx=0x7f769800a7d0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 shutdown_connections 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f767c066f90 0x7f767c069450 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a015bdc0 0x7f76a006bce0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a015b9e0 0x7f76a0159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 --2- 172.21.15.45:0/3090666937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a00d1c30 0x7f76a01571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 >> 172.21.15.45:0/3090666937 conn(0x7f76a009a9d0 msgr2=0x7f76a00ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 shutdown_connections 2024-04-25T10:26:18.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.091+0000 7f76a5000640 1 -- 172.21.15.45:0/3090666937 wait complete. 2024-04-25T10:26:18.115 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T10:26:16.989392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417176,"kb_used_data":12252,"kb_used_omap":12,"kb_used_meta":404787,"kb_avail":754557544,"statfs":{"total":773094113280,"available":772666925056,"internally_reserved":0,"allocated":12546048,"data_stored":6089797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":414502448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042706"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960603+0000","last_change":"2024-04-25T10:26:12.960603+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960603+0000","last_undegraded":"2024-04-25T10:26:12.960603+0000","last_fullsized":"2024-04-25T10:26:12.960603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958621+0000","last_change":"2024-04-25T10:26:12.958621+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958621+0000","last_undegraded":"2024-04-25T10:26:12.958621+0000","last_fullsized":"2024-04-25T10:26:12.958621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957509+0000","last_change":"2024-04-25T10:26:12.957509+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957509+0000","last_undegraded":"2024-04-25T10:26:12.957509+0000","last_fullsized":"2024-04-25T10:26:12.957509+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960968+0000","last_change":"2024-04-25T10:26:12.960968+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960968+0000","last_undegraded":"2024-04-25T10:26:12.960968+0000","last_fullsized":"2024-04-25T10:26:12.960968+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960557+0000","last_change":"2024-04-25T10:26:12.960557+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960557+0000","last_undegraded":"2024-04-25T10:26:12.960557+0000","last_fullsized":"2024-04-25T10:26:12.960557+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957902+0000","last_change":"2024-04-25T10:26:12.957902+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957902+0000","last_undegraded":"2024-04-25T10:26:12.957902+0000","last_fullsized":"2024-04-25T10:26:12.957902+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955820+0000","last_change":"2024-04-25T10:26:12.955820+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955820+0000","last_undegraded":"2024-04-25T10:26:12.955820+0000","last_fullsized":"2024-04-25T10:26:12.955820+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960167+0000","last_change":"2024-04-25T10:26:12.960167+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960167+0000","last_undegraded":"2024-04-25T10:26:12.960167+0000","last_fullsized":"2024-04-25T10:26:12.960167+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963662+0000","last_change":"2024-04-25T10:26:12.963662+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963662+0000","last_undegraded":"2024-04-25T10:26:12.963662+0000","last_fullsized":"2024-04-25T10:26:12.963662+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958648+0000","last_change":"2024-04-25T10:26:12.958648+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958648+0000","last_undegraded":"2024-04-25T10:26:12.958648+0000","last_fullsized":"2024-04-25T10:26:12.958648+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960992+0000","last_change":"2024-04-25T10:26:12.960992+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960992+0000","last_undegraded":"2024-04-25T10:26:12.960992+0000","last_fullsized":"2024-04-25T10:26:12.960992+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.954373+0000","last_change":"2024-04-25T10:26:12.954373+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.954373+0000","last_undegraded":"2024-04-25T10:26:12.954373+0000","last_fullsized":"2024-04-25T10:26:12.954373+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962360+0000","last_change":"2024-04-25T10:26:12.962360+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962360+0000","last_undegraded":"2024-04-25T10:26:12.962360+0000","last_fullsized":"2024-04-25T10:26:12.962360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960133+0000","last_change":"2024-04-25T10:26:12.960133+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960133+0000","last_undegraded":"2024-04-25T10:26:12.960133+0000","last_fullsized":"2024-04-25T10:26:12.960133+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963701+0000","last_change":"2024-04-25T10:26:12.963701+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963701+0000","last_undegraded":"2024-04-25T10:26:12.963701+0000","last_fullsized":"2024-04-25T10:26:12.963701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955316+0000","last_change":"2024-04-25T10:26:12.955316+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955316+0000","last_undegraded":"2024-04-25T10:26:12.955316+0000","last_fullsized":"2024-04-25T10:26:12.955316+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956443+0000","last_change":"2024-04-25T10:26:12.956443+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956443+0000","last_undegraded":"2024-04-25T10:26:12.956443+0000","last_fullsized":"2024-04-25T10:26:12.956443+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955246+0000","last_change":"2024-04-25T10:26:12.955246+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955246+0000","last_undegraded":"2024-04-25T10:26:12.955246+0000","last_fullsized":"2024-04-25T10:26:12.955246+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955483+0000","last_change":"2024-04-25T10:26:12.955483+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955483+0000","last_undegraded":"2024-04-25T10:26:12.955483+0000","last_fullsized":"2024-04-25T10:26:12.955483+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956466+0000","last_change":"2024-04-25T10:26:12.956466+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956466+0000","last_undegraded":"2024-04-25T10:26:12.956466+0000","last_fullsized":"2024-04-25T10:26:12.956466+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960106+0000","last_change":"2024-04-25T10:26:12.960106+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960106+0000","last_undegraded":"2024-04-25T10:26:12.960106+0000","last_fullsized":"2024-04-25T10:26:12.960106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961108+0000","last_change":"2024-04-25T10:26:12.961108+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961108+0000","last_undegraded":"2024-04-25T10:26:12.961108+0000","last_fullsized":"2024-04-25T10:26:12.961108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955944+0000","last_change":"2024-04-25T10:26:12.955944+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955944+0000","last_undegraded":"2024-04-25T10:26:12.955944+0000","last_fullsized":"2024-04-25T10:26:12.955944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961486+0000","last_change":"2024-04-25T10:26:12.961486+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961486+0000","last_undegraded":"2024-04-25T10:26:12.961486+0000","last_fullsized":"2024-04-25T10:26:12.961486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959667+0000","last_change":"2024-04-25T10:26:12.959667+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959667+0000","last_undegraded":"2024-04-25T10:26:12.959667+0000","last_fullsized":"2024-04-25T10:26:12.959667+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963326+0000","last_change":"2024-04-25T10:26:12.963326+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963326+0000","last_undegraded":"2024-04-25T10:26:12.963326+0000","last_fullsized":"2024-04-25T10:26:12.963326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955697+0000","last_change":"2024-04-25T10:26:12.955697+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955697+0000","last_undegraded":"2024-04-25T10:26:12.955697+0000","last_fullsized":"2024-04-25T10:26:12.955697+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958589+0000","last_change":"2024-04-25T10:26:12.958589+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958589+0000","last_undegraded":"2024-04-25T10:26:12.958589+0000","last_fullsized":"2024-04-25T10:26:12.958589+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963268+0000","last_change":"2024-04-25T10:26:12.963268+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963268+0000","last_undegraded":"2024-04-25T10:26:12.963268+0000","last_fullsized":"2024-04-25T10:26:12.963268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959445+0000","last_change":"2024-04-25T10:26:12.959445+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959445+0000","last_undegraded":"2024-04-25T10:26:12.959445+0000","last_fullsized":"2024-04-25T10:26:12.959445+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.974161+0000","last_change":"2024-04-25T10:26:12.973421+0000","last_active":"2024-04-25T10:26:12.974161+0000","last_peered":"2024-04-25T10:26:12.974161+0000","last_clean":"2024-04-25T10:26:12.974161+0000","last_became_active":"2024-04-25T10:26:12.973108+0000","last_became_peered":"2024-04-25T10:26:12.973108+0000","last_unstale":"2024-04-25T10:26:12.974161+0000","last_undegraded":"2024-04-25T10:26:12.974161+0000","last_fullsized":"2024-04-25T10:26:12.974161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.116774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962804+0000","last_change":"2024-04-25T10:26:12.962804+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962804+0000","last_undegraded":"2024-04-25T10:26:12.962804+0000","last_fullsized":"2024-04-25T10:26:12.962804+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971928+0000","last_change":"2024-04-25T10:26:12.965663+0000","last_active":"2024-04-25T10:26:12.971928+0000","last_peered":"2024-04-25T10:26:12.971928+0000","last_clean":"2024-04-25T10:26:12.971928+0000","last_became_active":"2024-04-25T10:26:12.965358+0000","last_became_peered":"2024-04-25T10:26:12.965358+0000","last_unstale":"2024-04-25T10:26:12.971928+0000","last_undegraded":"2024-04-25T10:26:12.971928+0000","last_fullsized":"2024-04-25T10:26:12.971928+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:53.575119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959310+0000","last_change":"2024-04-25T10:26:12.959310+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959310+0000","last_undegraded":"2024-04-25T10:26:12.959310+0000","last_fullsized":"2024-04-25T10:26:12.959310+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950769+0000","last_change":"2024-04-25T10:26:11.950769+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950769+0000","last_undegraded":"2024-04-25T10:26:11.950769+0000","last_fullsized":"2024-04-25T10:26:11.950769+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957091+0000","last_change":"2024-04-25T10:26:12.957091+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957091+0000","last_undegraded":"2024-04-25T10:26:12.957091+0000","last_fullsized":"2024-04-25T10:26:12.957091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.951276+0000","last_change":"2024-04-25T10:26:11.951276+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.951276+0000","last_undegraded":"2024-04-25T10:26:11.951276+0000","last_fullsized":"2024-04-25T10:26:11.951276+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971383+0000","last_change":"2024-04-25T10:26:12.969937+0000","last_active":"2024-04-25T10:26:12.971383+0000","last_peered":"2024-04-25T10:26:12.971383+0000","last_clean":"2024-04-25T10:26:12.971383+0000","last_became_active":"2024-04-25T10:26:12.969666+0000","last_became_peered":"2024-04-25T10:26:12.969666+0000","last_unstale":"2024-04-25T10:26:12.971383+0000","last_undegraded":"2024-04-25T10:26:12.971383+0000","last_fullsized":"2024-04-25T10:26:12.971383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:45.691111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956910+0000","last_change":"2024-04-25T10:26:12.956910+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956910+0000","last_undegraded":"2024-04-25T10:26:12.956910+0000","last_fullsized":"2024-04-25T10:26:12.956910+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.969780+0000","last_change":"2024-04-25T10:26:12.968858+0000","last_active":"2024-04-25T10:26:12.969780+0000","last_peered":"2024-04-25T10:26:12.969780+0000","last_clean":"2024-04-25T10:26:12.969780+0000","last_became_active":"2024-04-25T10:26:12.968309+0000","last_became_peered":"2024-04-25T10:26:12.968309+0000","last_unstale":"2024-04-25T10:26:12.969780+0000","last_undegraded":"2024-04-25T10:26:12.969780+0000","last_fullsized":"2024-04-25T10:26:12.969780+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:38.622906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961042+0000","last_change":"2024-04-25T10:26:12.961042+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961042+0000","last_undegraded":"2024-04-25T10:26:12.961042+0000","last_fullsized":"2024-04-25T10:26:12.961042+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970364+0000","last_change":"2024-04-25T10:26:12.968507+0000","last_active":"2024-04-25T10:26:12.970364+0000","last_peered":"2024-04-25T10:26:12.970364+0000","last_clean":"2024-04-25T10:26:12.970364+0000","last_became_active":"2024-04-25T10:26:12.967779+0000","last_became_peered":"2024-04-25T10:26:12.967779+0000","last_unstale":"2024-04-25T10:26:12.970364+0000","last_undegraded":"2024-04-25T10:26:12.970364+0000","last_fullsized":"2024-04-25T10:26:12.970364+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.385060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957397+0000","last_change":"2024-04-25T10:26:12.957397+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957397+0000","last_undegraded":"2024-04-25T10:26:12.957397+0000","last_fullsized":"2024-04-25T10:26:12.957397+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977221+0000","last_change":"2024-04-25T10:26:12.974852+0000","last_active":"2024-04-25T10:26:12.977221+0000","last_peered":"2024-04-25T10:26:12.977221+0000","last_clean":"2024-04-25T10:26:12.977221+0000","last_became_active":"2024-04-25T10:26:12.974599+0000","last_became_peered":"2024-04-25T10:26:12.974599+0000","last_unstale":"2024-04-25T10:26:12.977221+0000","last_undegraded":"2024-04-25T10:26:12.977221+0000","last_fullsized":"2024-04-25T10:26:12.977221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:17.313609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.953984+0000","last_change":"2024-04-25T10:26:12.953984+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.953984+0000","last_undegraded":"2024-04-25T10:26:12.953984+0000","last_fullsized":"2024-04-25T10:26:12.953984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950616+0000","last_change":"2024-04-25T10:26:11.950616+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950616+0000","last_undegraded":"2024-04-25T10:26:11.950616+0000","last_fullsized":"2024-04-25T10:26:11.950616+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.953982+0000","last_change":"2024-04-25T10:26:12.953982+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.953982+0000","last_undegraded":"2024-04-25T10:26:12.953982+0000","last_fullsized":"2024-04-25T10:26:12.953982+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.978516+0000","last_change":"2024-04-25T10:26:12.974260+0000","last_active":"2024-04-25T10:26:12.978516+0000","last_peered":"2024-04-25T10:26:12.978516+0000","last_clean":"2024-04-25T10:26:12.978516+0000","last_became_active":"2024-04-25T10:26:12.973967+0000","last_became_peered":"2024-04-25T10:26:12.973967+0000","last_unstale":"2024-04-25T10:26:12.978516+0000","last_undegraded":"2024-04-25T10:26:12.978516+0000","last_fullsized":"2024-04-25T10:26:12.978516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:53.892616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958876+0000","last_change":"2024-04-25T10:26:12.958876+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958876+0000","last_undegraded":"2024-04-25T10:26:12.958876+0000","last_fullsized":"2024-04-25T10:26:12.958876+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.980535+0000","last_change":"2024-04-25T10:26:12.979707+0000","last_active":"2024-04-25T10:26:12.980535+0000","last_peered":"2024-04-25T10:26:12.980535+0000","last_clean":"2024-04-25T10:26:12.980535+0000","last_became_active":"2024-04-25T10:26:12.979419+0000","last_became_peered":"2024-04-25T10:26:12.979419+0000","last_unstale":"2024-04-25T10:26:12.980535+0000","last_undegraded":"2024-04-25T10:26:12.980535+0000","last_fullsized":"2024-04-25T10:26:12.980535+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:54.363182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955903+0000","last_change":"2024-04-25T10:26:12.955903+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955903+0000","last_undegraded":"2024-04-25T10:26:12.955903+0000","last_fullsized":"2024-04-25T10:26:12.955903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950904+0000","last_change":"2024-04-25T10:26:11.950904+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950904+0000","last_undegraded":"2024-04-25T10:26:11.950904+0000","last_fullsized":"2024-04-25T10:26:11.950904+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960737+0000","last_change":"2024-04-25T10:26:12.960737+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960737+0000","last_undegraded":"2024-04-25T10:26:12.960737+0000","last_fullsized":"2024-04-25T10:26:12.960737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977415+0000","last_change":"2024-04-25T10:26:12.974720+0000","last_active":"2024-04-25T10:26:12.977415+0000","last_peered":"2024-04-25T10:26:12.977415+0000","last_clean":"2024-04-25T10:26:12.977415+0000","last_became_active":"2024-04-25T10:26:12.974006+0000","last_became_peered":"2024-04-25T10:26:12.974006+0000","last_unstale":"2024-04-25T10:26:12.977415+0000","last_undegraded":"2024-04-25T10:26:12.977415+0000","last_fullsized":"2024-04-25T10:26:12.977415+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:06.748603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971459+0000","last_change":"2024-04-25T10:26:12.970303+0000","last_active":"2024-04-25T10:26:12.971459+0000","last_peered":"2024-04-25T10:26:12.971459+0000","last_clean":"2024-04-25T10:26:12.971459+0000","last_became_active":"2024-04-25T10:26:12.969076+0000","last_became_peered":"2024-04-25T10:26:12.969076+0000","last_unstale":"2024-04-25T10:26:12.971459+0000","last_undegraded":"2024-04-25T10:26:12.971459+0000","last_fullsized":"2024-04-25T10:26:12.971459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973497+0000","last_change":"2024-04-25T10:26:12.970708+0000","last_active":"2024-04-25T10:26:12.973497+0000","last_peered":"2024-04-25T10:26:12.973497+0000","last_clean":"2024-04-25T10:26:12.973497+0000","last_became_active":"2024-04-25T10:26:12.970360+0000","last_became_peered":"2024-04-25T10:26:12.970360+0000","last_unstale":"2024-04-25T10:26:12.973497+0000","last_undegraded":"2024-04-25T10:26:12.973497+0000","last_fullsized":"2024-04-25T10:26:12.973497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:50.722488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958521+0000","last_change":"2024-04-25T10:26:12.958521+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958521+0000","last_undegraded":"2024-04-25T10:26:12.958521+0000","last_fullsized":"2024-04-25T10:26:12.958521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977416+0000","last_change":"2024-04-25T10:26:12.972335+0000","last_active":"2024-04-25T10:26:12.977416+0000","last_peered":"2024-04-25T10:26:12.977416+0000","last_clean":"2024-04-25T10:26:12.977416+0000","last_became_active":"2024-04-25T10:26:12.972062+0000","last_became_peered":"2024-04-25T10:26:12.972062+0000","last_unstale":"2024-04-25T10:26:12.977416+0000","last_undegraded":"2024-04-25T10:26:12.977416+0000","last_fullsized":"2024-04-25T10:26:12.977416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.527361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958201+0000","last_change":"2024-04-25T10:26:12.958201+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958201+0000","last_undegraded":"2024-04-25T10:26:12.958201+0000","last_fullsized":"2024-04-25T10:26:12.958201+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977158+0000","last_change":"2024-04-25T10:26:12.974012+0000","last_active":"2024-04-25T10:26:12.977158+0000","last_peered":"2024-04-25T10:26:12.977158+0000","last_clean":"2024-04-25T10:26:12.977158+0000","last_became_active":"2024-04-25T10:26:12.973735+0000","last_became_peered":"2024-04-25T10:26:12.973735+0000","last_unstale":"2024-04-25T10:26:12.977158+0000","last_undegraded":"2024-04-25T10:26:12.977158+0000","last_fullsized":"2024-04-25T10:26:12.977158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.821837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955301+0000","last_change":"2024-04-25T10:26:12.955301+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955301+0000","last_undegraded":"2024-04-25T10:26:12.955301+0000","last_fullsized":"2024-04-25T10:26:12.955301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973705+0000","last_change":"2024-04-25T10:26:12.972502+0000","last_active":"2024-04-25T10:26:12.973705+0000","last_peered":"2024-04-25T10:26:12.973705+0000","last_clean":"2024-04-25T10:26:12.973705+0000","last_became_active":"2024-04-25T10:26:12.972135+0000","last_became_peered":"2024-04-25T10:26:12.972135+0000","last_unstale":"2024-04-25T10:26:12.973705+0000","last_undegraded":"2024-04-25T10:26:12.973705+0000","last_fullsized":"2024-04-25T10:26:12.973705+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:14.890002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963858+0000","last_change":"2024-04-25T10:26:12.963858+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963858+0000","last_undegraded":"2024-04-25T10:26:12.963858+0000","last_fullsized":"2024-04-25T10:26:12.963858+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.975964+0000","last_change":"2024-04-25T10:26:12.975104+0000","last_active":"2024-04-25T10:26:12.975964+0000","last_peered":"2024-04-25T10:26:12.975964+0000","last_clean":"2024-04-25T10:26:12.975964+0000","last_became_active":"2024-04-25T10:26:12.974542+0000","last_became_peered":"2024-04-25T10:26:12.974542+0000","last_unstale":"2024-04-25T10:26:12.975964+0000","last_undegraded":"2024-04-25T10:26:12.975964+0000","last_fullsized":"2024-04-25T10:26:12.975964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:43.021754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955308+0000","last_change":"2024-04-25T10:26:12.955308+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955308+0000","last_undegraded":"2024-04-25T10:26:12.955308+0000","last_fullsized":"2024-04-25T10:26:12.955308+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977436+0000","last_change":"2024-04-25T10:26:12.974951+0000","last_active":"2024-04-25T10:26:12.977436+0000","last_peered":"2024-04-25T10:26:12.977436+0000","last_clean":"2024-04-25T10:26:12.977436+0000","last_became_active":"2024-04-25T10:26:12.974439+0000","last_became_peered":"2024-04-25T10:26:12.974439+0000","last_unstale":"2024-04-25T10:26:12.977436+0000","last_undegraded":"2024-04-25T10:26:12.977436+0000","last_fullsized":"2024-04-25T10:26:12.977436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:16.500633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958165+0000","last_change":"2024-04-25T10:26:12.958165+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958165+0000","last_undegraded":"2024-04-25T10:26:12.958165+0000","last_fullsized":"2024-04-25T10:26:12.958165+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973079+0000","last_change":"2024-04-25T10:26:12.971625+0000","last_active":"2024-04-25T10:26:12.973079+0000","last_peered":"2024-04-25T10:26:12.973079+0000","last_clean":"2024-04-25T10:26:12.973079+0000","last_became_active":"2024-04-25T10:26:12.971364+0000","last_became_peered":"2024-04-25T10:26:12.971364+0000","last_unstale":"2024-04-25T10:26:12.973079+0000","last_undegraded":"2024-04-25T10:26:12.973079+0000","last_fullsized":"2024-04-25T10:26:12.973079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:13.635613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963797+0000","last_change":"2024-04-25T10:26:12.963797+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963797+0000","last_undegraded":"2024-04-25T10:26:12.963797+0000","last_fullsized":"2024-04-25T10:26:12.963797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949756+0000","last_change":"2024-04-25T10:26:11.949756+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949756+0000","last_undegraded":"2024-04-25T10:26:11.949756+0000","last_fullsized":"2024-04-25T10:26:11.949756+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963044+0000","last_change":"2024-04-25T10:26:12.963044+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963044+0000","last_undegraded":"2024-04-25T10:26:12.963044+0000","last_fullsized":"2024-04-25T10:26:12.963044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949650+0000","last_change":"2024-04-25T10:26:11.949650+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949650+0000","last_undegraded":"2024-04-25T10:26:11.949650+0000","last_fullsized":"2024-04-25T10:26:11.949650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962374+0000","last_change":"2024-04-25T10:26:12.962374+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962374+0000","last_undegraded":"2024-04-25T10:26:12.962374+0000","last_fullsized":"2024-04-25T10:26:12.962374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949647+0000","last_change":"2024-04-25T10:26:11.949647+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949647+0000","last_undegraded":"2024-04-25T10:26:11.949647+0000","last_fullsized":"2024-04-25T10:26:11.949647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962208+0000","last_change":"2024-04-25T10:26:12.962208+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962208+0000","last_undegraded":"2024-04-25T10:26:12.962208+0000","last_fullsized":"2024-04-25T10:26:12.962208+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973683+0000","last_change":"2024-04-25T10:26:12.972745+0000","last_active":"2024-04-25T10:26:12.973683+0000","last_peered":"2024-04-25T10:26:12.973683+0000","last_clean":"2024-04-25T10:26:12.973683+0000","last_became_active":"2024-04-25T10:26:12.972443+0000","last_became_peered":"2024-04-25T10:26:12.972443+0000","last_unstale":"2024-04-25T10:26:12.973683+0000","last_undegraded":"2024-04-25T10:26:12.973683+0000","last_fullsized":"2024-04-25T10:26:12.973683+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:57.380293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960977+0000","last_change":"2024-04-25T10:26:12.960977+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960977+0000","last_undegraded":"2024-04-25T10:26:12.960977+0000","last_fullsized":"2024-04-25T10:26:12.960977+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949642+0000","last_change":"2024-04-25T10:26:11.949642+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949642+0000","last_undegraded":"2024-04-25T10:26:11.949642+0000","last_fullsized":"2024-04-25T10:26:11.949642+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963427+0000","last_change":"2024-04-25T10:26:12.963427+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963427+0000","last_undegraded":"2024-04-25T10:26:12.963427+0000","last_fullsized":"2024-04-25T10:26:12.963427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950177+0000","last_change":"2024-04-25T10:26:11.950177+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950177+0000","last_undegraded":"2024-04-25T10:26:11.950177+0000","last_fullsized":"2024-04-25T10:26:11.950177+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.954750+0000","last_change":"2024-04-25T10:26:12.954750+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.954750+0000","last_undegraded":"2024-04-25T10:26:12.954750+0000","last_fullsized":"2024-04-25T10:26:12.954750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.969031+0000","last_change":"2024-04-25T10:26:12.965357+0000","last_active":"2024-04-25T10:26:12.969031+0000","last_peered":"2024-04-25T10:26:12.969031+0000","last_clean":"2024-04-25T10:26:12.969031+0000","last_became_active":"2024-04-25T10:26:12.964963+0000","last_became_peered":"2024-04-25T10:26:12.964963+0000","last_unstale":"2024-04-25T10:26:12.969031+0000","last_undegraded":"2024-04-25T10:26:12.969031+0000","last_fullsized":"2024-04-25T10:26:12.969031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:17.853096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961968+0000","last_change":"2024-04-25T10:26:12.961968+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961968+0000","last_undegraded":"2024-04-25T10:26:12.961968+0000","last_fullsized":"2024-04-25T10:26:12.961968+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970394+0000","last_change":"2024-04-25T10:26:12.968833+0000","last_active":"2024-04-25T10:26:12.970394+0000","last_peered":"2024-04-25T10:26:12.970394+0000","last_clean":"2024-04-25T10:26:12.970394+0000","last_became_active":"2024-04-25T10:26:12.968544+0000","last_became_peered":"2024-04-25T10:26:12.968544+0000","last_unstale":"2024-04-25T10:26:12.970394+0000","last_undegraded":"2024-04-25T10:26:12.970394+0000","last_fullsized":"2024-04-25T10:26:12.970394+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961084+0000","last_change":"2024-04-25T10:26:12.961084+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961084+0000","last_undegraded":"2024-04-25T10:26:12.961084+0000","last_fullsized":"2024-04-25T10:26:12.961084+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.972809+0000","last_change":"2024-04-25T10:26:12.970872+0000","last_active":"2024-04-25T10:26:12.972809+0000","last_peered":"2024-04-25T10:26:12.972809+0000","last_clean":"2024-04-25T10:26:12.972809+0000","last_became_active":"2024-04-25T10:26:12.970594+0000","last_became_peered":"2024-04-25T10:26:12.970594+0000","last_unstale":"2024-04-25T10:26:12.972809+0000","last_undegraded":"2024-04-25T10:26:12.972809+0000","last_fullsized":"2024-04-25T10:26:12.972809+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.206365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957844+0000","last_change":"2024-04-25T10:26:12.957844+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957844+0000","last_undegraded":"2024-04-25T10:26:12.957844+0000","last_fullsized":"2024-04-25T10:26:12.957844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973677+0000","last_change":"2024-04-25T10:26:12.969936+0000","last_active":"2024-04-25T10:26:12.973677+0000","last_peered":"2024-04-25T10:26:12.973677+0000","last_clean":"2024-04-25T10:26:12.973677+0000","last_became_active":"2024-04-25T10:26:12.969633+0000","last_became_peered":"2024-04-25T10:26:12.969633+0000","last_unstale":"2024-04-25T10:26:12.973677+0000","last_undegraded":"2024-04-25T10:26:12.973677+0000","last_fullsized":"2024-04-25T10:26:12.973677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970691+0000","last_change":"2024-04-25T10:26:12.968805+0000","last_active":"2024-04-25T10:26:12.970691+0000","last_peered":"2024-04-25T10:26:12.970691+0000","last_clean":"2024-04-25T10:26:12.970691+0000","last_became_active":"2024-04-25T10:26:12.968534+0000","last_became_peered":"2024-04-25T10:26:12.968534+0000","last_unstale":"2024-04-25T10:26:12.970691+0000","last_undegraded":"2024-04-25T10:26:12.970691+0000","last_fullsized":"2024-04-25T10:26:12.970691+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:27.777514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":347,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.958158+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:26:12.958158+0000","last_peered":"2024-04-25T10:26:12.958158+0000","last_clean":"2024-04-25T10:26:12.958158+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:26:12.958158+0000","last_undegraded":"2024-04-25T10:26:12.958158+0000","last_fullsized":"2024-04-25T10:26:12.958158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50616,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321224,"statfs":{"total":96636764160,"available":96584933376,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509545,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50624,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321216,"statfs":{"total":96636764160,"available":96584925184,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2015232,"data_stored":1204785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2015232,"data_stored":1204785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28416,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343424,"statfs":{"total":96636764160,"available":96607666176,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542247,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58924,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312916,"statfs":{"total":96636764160,"available":96576425984,"internally_reserved":0,"allocated":1409024,"data_stored":608142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:18.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:18.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc633200640 1 Processor -- start 2024-04-25T10:26:18.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc633200640 1 -- start start 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc633200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc633200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c0cd120 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc633200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c0cd400 con 0x7fc62c0ce700 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc633200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0d2600 con 0x7fc62c15b9b0 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc632800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc62c0b5fb0 0x7fc62c0cfa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60840/0 (socket says 172.21.15.45:60840) 2024-04-25T10:26:18.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc632800640 1 -- 172.21.15.45:0/1974919167 learned_addr learned my addr 172.21.15.45:0/1974919167 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc631e00640 1 --2- 172.21.15.45:0/1974919167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc630a00640 1 -- 172.21.15.45:0/1974919167 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129959253 0 0) 0x7fc62c0cd120 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc630a00640 1 -- 172.21.15.45:0/1974919167 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc614003530 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc631e00640 1 -- 172.21.15.45:0/1974919167 >> v1:172.21.15.117:6789/0 conn(0x7fc62c0ce700 legacy=0x7fc62c0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc630a00640 1 -- 172.21.15.45:0/1974919167 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863786466 0 0) 0x7fc62c0cd400 con 0x7fc62c0ce700 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc631e00640 1 -- 172.21.15.45:0/1974919167 >> v1:172.21.15.45:6789/0 conn(0x7fc62c0b5fb0 legacy=0x7fc62c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.299+0000 7fc631e00640 1 -- 172.21.15.45:0/1974919167 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c0b47a0 con 0x7fc62c15b9b0 2024-04-25T10:26:18.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc631e00640 1 --2- 172.21.15.45:0/1974919167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c0ce1c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc6200097f0 tx=0x7fc62001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a77ded17dabf9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc630a00640 1 -- 172.21.15.45:0/1974919167 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b5070 con 0x7fc62c15b9b0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc630a00640 1 -- 172.21.15.45:0/1974919167 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200a89d0 con 0x7fc62c15b9b0 2024-04-25T10:26:18.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc630a00640 1 -- 172.21.15.45:0/1974919167 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200a8da0 con 0x7fc62c15b9b0 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 -- 172.21.15.45:0/1974919167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 msgr2=0x7fc62c0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 --2- 172.21.15.45:0/1974919167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c0ce1c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc6200097f0 tx=0x7fc62001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 -- 172.21.15.45:0/1974919167 shutdown_connections 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 --2- 172.21.15.45:0/1974919167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c0ce1c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 -- 172.21.15.45:0/1974919167 >> 172.21.15.45:0/1974919167 conn(0x7fc62c0d17b0 msgr2=0x7fc62c0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 -- 172.21.15.45:0/1974919167 shutdown_connections 2024-04-25T10:26:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 -- 172.21.15.45:0/1974919167 wait complete. 2024-04-25T10:26:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.303+0000 7fc633200640 1 Processor -- start 2024-04-25T10:26:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- start start 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 0x7fc62c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc62c0ce700 0x7fc62c1f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c1f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c15bf10 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c15bd90 con 0x7fc62c0ce700 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0ccd30 con 0x7fc62c15b9b0 2024-04-25T10:26:18.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 0x7fc62c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 0x7fc62c0c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37640/0 (socket says 172.21.15.45:37640) 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 -- 172.21.15.45:0/2627157773 learned_addr learned my addr 172.21.15.45:0/2627157773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631400640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc62c0ce700 0x7fc62c1f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc632800640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c1f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 -- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 msgr2=0x7fc62c1f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c1f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 -- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc62c0ce700 msgr2=0x7fc62c1f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc62c0ce700 0x7fc62c1f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 -- 172.21.15.45:0/2627157773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c1f5140 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc631e00640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 0x7fc62c0c7e80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc6200097f0 tx=0x7fc62001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b5040 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200b0440 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6200b07f0 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c1f5420 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c0c9b00 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.307+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c075120 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.311+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6200c1020 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.311+0000 7fc626000640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc610066f90 0x7fc610069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.311+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc620121e00 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.311+0000 7fc631400640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc610066f90 0x7fc610069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.311+0000 7fc631400640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc610066f90 0x7fc610069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc62c09f850 tx=0x7fc61c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.315+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc620018830 con 0x7fc62c0b5fb0 2024-04-25T10:26:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.475+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc62c0c98d0 con 0x7fc610066f90 2024-04-25T10:26:18.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.479+0000 7fc626000640 1 -- 172.21.15.45:0/2627157773 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+245874 (secure 0 0 0) 0x7fc62c0c98d0 con 0x7fc610066f90 2024-04-25T10:26:18.484 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:18.486 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc610066f90 msgr2=0x7fc610069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc610066f90 0x7fc610069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc62c09f850 tx=0x7fc61c009040 comp rx=0 tx=0).stop 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 msgr2=0x7fc62c0c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 0x7fc62c0c7e80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc6200097f0 tx=0x7fc62001ba70 comp rx=0 tx=0).stop 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 shutdown_connections 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc610066f90 0x7fc610069450 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc62c15b9b0 0x7fc62c1f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc62c0ce700 0x7fc62c1f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 --2- 172.21.15.45:0/2627157773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc62c0b5fb0 0x7fc62c0c7e80 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 >> 172.21.15.45:0/2627157773 conn(0x7fc62c0d17b0 msgr2=0x7fc62c06e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 shutdown_connections 2024-04-25T10:26:18.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.483+0000 7fc633200640 1 -- 172.21.15.45:0/2627157773 wait complete. 2024-04-25T10:26:18.505 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T10:26:16.989392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417176,"kb_used_data":12252,"kb_used_omap":12,"kb_used_meta":404787,"kb_avail":754557544,"statfs":{"total":773094113280,"available":772666925056,"internally_reserved":0,"allocated":12546048,"data_stored":6089797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":414502448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042706"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960603+0000","last_change":"2024-04-25T10:26:12.960603+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960603+0000","last_undegraded":"2024-04-25T10:26:12.960603+0000","last_fullsized":"2024-04-25T10:26:12.960603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958621+0000","last_change":"2024-04-25T10:26:12.958621+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958621+0000","last_undegraded":"2024-04-25T10:26:12.958621+0000","last_fullsized":"2024-04-25T10:26:12.958621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957509+0000","last_change":"2024-04-25T10:26:12.957509+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957509+0000","last_undegraded":"2024-04-25T10:26:12.957509+0000","last_fullsized":"2024-04-25T10:26:12.957509+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960968+0000","last_change":"2024-04-25T10:26:12.960968+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960968+0000","last_undegraded":"2024-04-25T10:26:12.960968+0000","last_fullsized":"2024-04-25T10:26:12.960968+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960557+0000","last_change":"2024-04-25T10:26:12.960557+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960557+0000","last_undegraded":"2024-04-25T10:26:12.960557+0000","last_fullsized":"2024-04-25T10:26:12.960557+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957902+0000","last_change":"2024-04-25T10:26:12.957902+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957902+0000","last_undegraded":"2024-04-25T10:26:12.957902+0000","last_fullsized":"2024-04-25T10:26:12.957902+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955820+0000","last_change":"2024-04-25T10:26:12.955820+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955820+0000","last_undegraded":"2024-04-25T10:26:12.955820+0000","last_fullsized":"2024-04-25T10:26:12.955820+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960167+0000","last_change":"2024-04-25T10:26:12.960167+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960167+0000","last_undegraded":"2024-04-25T10:26:12.960167+0000","last_fullsized":"2024-04-25T10:26:12.960167+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963662+0000","last_change":"2024-04-25T10:26:12.963662+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963662+0000","last_undegraded":"2024-04-25T10:26:12.963662+0000","last_fullsized":"2024-04-25T10:26:12.963662+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958648+0000","last_change":"2024-04-25T10:26:12.958648+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958648+0000","last_undegraded":"2024-04-25T10:26:12.958648+0000","last_fullsized":"2024-04-25T10:26:12.958648+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960992+0000","last_change":"2024-04-25T10:26:12.960992+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960992+0000","last_undegraded":"2024-04-25T10:26:12.960992+0000","last_fullsized":"2024-04-25T10:26:12.960992+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.954373+0000","last_change":"2024-04-25T10:26:12.954373+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.954373+0000","last_undegraded":"2024-04-25T10:26:12.954373+0000","last_fullsized":"2024-04-25T10:26:12.954373+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962360+0000","last_change":"2024-04-25T10:26:12.962360+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962360+0000","last_undegraded":"2024-04-25T10:26:12.962360+0000","last_fullsized":"2024-04-25T10:26:12.962360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960133+0000","last_change":"2024-04-25T10:26:12.960133+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960133+0000","last_undegraded":"2024-04-25T10:26:12.960133+0000","last_fullsized":"2024-04-25T10:26:12.960133+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963701+0000","last_change":"2024-04-25T10:26:12.963701+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963701+0000","last_undegraded":"2024-04-25T10:26:12.963701+0000","last_fullsized":"2024-04-25T10:26:12.963701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955316+0000","last_change":"2024-04-25T10:26:12.955316+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955316+0000","last_undegraded":"2024-04-25T10:26:12.955316+0000","last_fullsized":"2024-04-25T10:26:12.955316+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956443+0000","last_change":"2024-04-25T10:26:12.956443+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956443+0000","last_undegraded":"2024-04-25T10:26:12.956443+0000","last_fullsized":"2024-04-25T10:26:12.956443+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955246+0000","last_change":"2024-04-25T10:26:12.955246+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955246+0000","last_undegraded":"2024-04-25T10:26:12.955246+0000","last_fullsized":"2024-04-25T10:26:12.955246+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955483+0000","last_change":"2024-04-25T10:26:12.955483+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955483+0000","last_undegraded":"2024-04-25T10:26:12.955483+0000","last_fullsized":"2024-04-25T10:26:12.955483+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956466+0000","last_change":"2024-04-25T10:26:12.956466+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956466+0000","last_undegraded":"2024-04-25T10:26:12.956466+0000","last_fullsized":"2024-04-25T10:26:12.956466+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960106+0000","last_change":"2024-04-25T10:26:12.960106+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960106+0000","last_undegraded":"2024-04-25T10:26:12.960106+0000","last_fullsized":"2024-04-25T10:26:12.960106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961108+0000","last_change":"2024-04-25T10:26:12.961108+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961108+0000","last_undegraded":"2024-04-25T10:26:12.961108+0000","last_fullsized":"2024-04-25T10:26:12.961108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955944+0000","last_change":"2024-04-25T10:26:12.955944+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955944+0000","last_undegraded":"2024-04-25T10:26:12.955944+0000","last_fullsized":"2024-04-25T10:26:12.955944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961486+0000","last_change":"2024-04-25T10:26:12.961486+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961486+0000","last_undegraded":"2024-04-25T10:26:12.961486+0000","last_fullsized":"2024-04-25T10:26:12.961486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959667+0000","last_change":"2024-04-25T10:26:12.959667+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959667+0000","last_undegraded":"2024-04-25T10:26:12.959667+0000","last_fullsized":"2024-04-25T10:26:12.959667+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963326+0000","last_change":"2024-04-25T10:26:12.963326+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963326+0000","last_undegraded":"2024-04-25T10:26:12.963326+0000","last_fullsized":"2024-04-25T10:26:12.963326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955697+0000","last_change":"2024-04-25T10:26:12.955697+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955697+0000","last_undegraded":"2024-04-25T10:26:12.955697+0000","last_fullsized":"2024-04-25T10:26:12.955697+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958589+0000","last_change":"2024-04-25T10:26:12.958589+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958589+0000","last_undegraded":"2024-04-25T10:26:12.958589+0000","last_fullsized":"2024-04-25T10:26:12.958589+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963268+0000","last_change":"2024-04-25T10:26:12.963268+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963268+0000","last_undegraded":"2024-04-25T10:26:12.963268+0000","last_fullsized":"2024-04-25T10:26:12.963268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959445+0000","last_change":"2024-04-25T10:26:12.959445+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959445+0000","last_undegraded":"2024-04-25T10:26:12.959445+0000","last_fullsized":"2024-04-25T10:26:12.959445+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.974161+0000","last_change":"2024-04-25T10:26:12.973421+0000","last_active":"2024-04-25T10:26:12.974161+0000","last_peered":"2024-04-25T10:26:12.974161+0000","last_clean":"2024-04-25T10:26:12.974161+0000","last_became_active":"2024-04-25T10:26:12.973108+0000","last_became_peered":"2024-04-25T10:26:12.973108+0000","last_unstale":"2024-04-25T10:26:12.974161+0000","last_undegraded":"2024-04-25T10:26:12.974161+0000","last_fullsized":"2024-04-25T10:26:12.974161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.116774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962804+0000","last_change":"2024-04-25T10:26:12.962804+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962804+0000","last_undegraded":"2024-04-25T10:26:12.962804+0000","last_fullsized":"2024-04-25T10:26:12.962804+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971928+0000","last_change":"2024-04-25T10:26:12.965663+0000","last_active":"2024-04-25T10:26:12.971928+0000","last_peered":"2024-04-25T10:26:12.971928+0000","last_clean":"2024-04-25T10:26:12.971928+0000","last_became_active":"2024-04-25T10:26:12.965358+0000","last_became_peered":"2024-04-25T10:26:12.965358+0000","last_unstale":"2024-04-25T10:26:12.971928+0000","last_undegraded":"2024-04-25T10:26:12.971928+0000","last_fullsized":"2024-04-25T10:26:12.971928+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:53.575119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959310+0000","last_change":"2024-04-25T10:26:12.959310+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959310+0000","last_undegraded":"2024-04-25T10:26:12.959310+0000","last_fullsized":"2024-04-25T10:26:12.959310+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950769+0000","last_change":"2024-04-25T10:26:11.950769+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950769+0000","last_undegraded":"2024-04-25T10:26:11.950769+0000","last_fullsized":"2024-04-25T10:26:11.950769+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957091+0000","last_change":"2024-04-25T10:26:12.957091+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957091+0000","last_undegraded":"2024-04-25T10:26:12.957091+0000","last_fullsized":"2024-04-25T10:26:12.957091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.951276+0000","last_change":"2024-04-25T10:26:11.951276+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.951276+0000","last_undegraded":"2024-04-25T10:26:11.951276+0000","last_fullsized":"2024-04-25T10:26:11.951276+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971383+0000","last_change":"2024-04-25T10:26:12.969937+0000","last_active":"2024-04-25T10:26:12.971383+0000","last_peered":"2024-04-25T10:26:12.971383+0000","last_clean":"2024-04-25T10:26:12.971383+0000","last_became_active":"2024-04-25T10:26:12.969666+0000","last_became_peered":"2024-04-25T10:26:12.969666+0000","last_unstale":"2024-04-25T10:26:12.971383+0000","last_undegraded":"2024-04-25T10:26:12.971383+0000","last_fullsized":"2024-04-25T10:26:12.971383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:45.691111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956910+0000","last_change":"2024-04-25T10:26:12.956910+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956910+0000","last_undegraded":"2024-04-25T10:26:12.956910+0000","last_fullsized":"2024-04-25T10:26:12.956910+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.969780+0000","last_change":"2024-04-25T10:26:12.968858+0000","last_active":"2024-04-25T10:26:12.969780+0000","last_peered":"2024-04-25T10:26:12.969780+0000","last_clean":"2024-04-25T10:26:12.969780+0000","last_became_active":"2024-04-25T10:26:12.968309+0000","last_became_peered":"2024-04-25T10:26:12.968309+0000","last_unstale":"2024-04-25T10:26:12.969780+0000","last_undegraded":"2024-04-25T10:26:12.969780+0000","last_fullsized":"2024-04-25T10:26:12.969780+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:38.622906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961042+0000","last_change":"2024-04-25T10:26:12.961042+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961042+0000","last_undegraded":"2024-04-25T10:26:12.961042+0000","last_fullsized":"2024-04-25T10:26:12.961042+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970364+0000","last_change":"2024-04-25T10:26:12.968507+0000","last_active":"2024-04-25T10:26:12.970364+0000","last_peered":"2024-04-25T10:26:12.970364+0000","last_clean":"2024-04-25T10:26:12.970364+0000","last_became_active":"2024-04-25T10:26:12.967779+0000","last_became_peered":"2024-04-25T10:26:12.967779+0000","last_unstale":"2024-04-25T10:26:12.970364+0000","last_undegraded":"2024-04-25T10:26:12.970364+0000","last_fullsized":"2024-04-25T10:26:12.970364+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.385060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957397+0000","last_change":"2024-04-25T10:26:12.957397+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957397+0000","last_undegraded":"2024-04-25T10:26:12.957397+0000","last_fullsized":"2024-04-25T10:26:12.957397+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977221+0000","last_change":"2024-04-25T10:26:12.974852+0000","last_active":"2024-04-25T10:26:12.977221+0000","last_peered":"2024-04-25T10:26:12.977221+0000","last_clean":"2024-04-25T10:26:12.977221+0000","last_became_active":"2024-04-25T10:26:12.974599+0000","last_became_peered":"2024-04-25T10:26:12.974599+0000","last_unstale":"2024-04-25T10:26:12.977221+0000","last_undegraded":"2024-04-25T10:26:12.977221+0000","last_fullsized":"2024-04-25T10:26:12.977221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:17.313609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.953984+0000","last_change":"2024-04-25T10:26:12.953984+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.953984+0000","last_undegraded":"2024-04-25T10:26:12.953984+0000","last_fullsized":"2024-04-25T10:26:12.953984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950616+0000","last_change":"2024-04-25T10:26:11.950616+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950616+0000","last_undegraded":"2024-04-25T10:26:11.950616+0000","last_fullsized":"2024-04-25T10:26:11.950616+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.953982+0000","last_change":"2024-04-25T10:26:12.953982+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.953982+0000","last_undegraded":"2024-04-25T10:26:12.953982+0000","last_fullsized":"2024-04-25T10:26:12.953982+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.978516+0000","last_change":"2024-04-25T10:26:12.974260+0000","last_active":"2024-04-25T10:26:12.978516+0000","last_peered":"2024-04-25T10:26:12.978516+0000","last_clean":"2024-04-25T10:26:12.978516+0000","last_became_active":"2024-04-25T10:26:12.973967+0000","last_became_peered":"2024-04-25T10:26:12.973967+0000","last_unstale":"2024-04-25T10:26:12.978516+0000","last_undegraded":"2024-04-25T10:26:12.978516+0000","last_fullsized":"2024-04-25T10:26:12.978516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:53.892616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958876+0000","last_change":"2024-04-25T10:26:12.958876+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958876+0000","last_undegraded":"2024-04-25T10:26:12.958876+0000","last_fullsized":"2024-04-25T10:26:12.958876+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.980535+0000","last_change":"2024-04-25T10:26:12.979707+0000","last_active":"2024-04-25T10:26:12.980535+0000","last_peered":"2024-04-25T10:26:12.980535+0000","last_clean":"2024-04-25T10:26:12.980535+0000","last_became_active":"2024-04-25T10:26:12.979419+0000","last_became_peered":"2024-04-25T10:26:12.979419+0000","last_unstale":"2024-04-25T10:26:12.980535+0000","last_undegraded":"2024-04-25T10:26:12.980535+0000","last_fullsized":"2024-04-25T10:26:12.980535+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:54.363182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955903+0000","last_change":"2024-04-25T10:26:12.955903+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955903+0000","last_undegraded":"2024-04-25T10:26:12.955903+0000","last_fullsized":"2024-04-25T10:26:12.955903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950904+0000","last_change":"2024-04-25T10:26:11.950904+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950904+0000","last_undegraded":"2024-04-25T10:26:11.950904+0000","last_fullsized":"2024-04-25T10:26:11.950904+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960737+0000","last_change":"2024-04-25T10:26:12.960737+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960737+0000","last_undegraded":"2024-04-25T10:26:12.960737+0000","last_fullsized":"2024-04-25T10:26:12.960737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977415+0000","last_change":"2024-04-25T10:26:12.974720+0000","last_active":"2024-04-25T10:26:12.977415+0000","last_peered":"2024-04-25T10:26:12.977415+0000","last_clean":"2024-04-25T10:26:12.977415+0000","last_became_active":"2024-04-25T10:26:12.974006+0000","last_became_peered":"2024-04-25T10:26:12.974006+0000","last_unstale":"2024-04-25T10:26:12.977415+0000","last_undegraded":"2024-04-25T10:26:12.977415+0000","last_fullsized":"2024-04-25T10:26:12.977415+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:06.748603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971459+0000","last_change":"2024-04-25T10:26:12.970303+0000","last_active":"2024-04-25T10:26:12.971459+0000","last_peered":"2024-04-25T10:26:12.971459+0000","last_clean":"2024-04-25T10:26:12.971459+0000","last_became_active":"2024-04-25T10:26:12.969076+0000","last_became_peered":"2024-04-25T10:26:12.969076+0000","last_unstale":"2024-04-25T10:26:12.971459+0000","last_undegraded":"2024-04-25T10:26:12.971459+0000","last_fullsized":"2024-04-25T10:26:12.971459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973497+0000","last_change":"2024-04-25T10:26:12.970708+0000","last_active":"2024-04-25T10:26:12.973497+0000","last_peered":"2024-04-25T10:26:12.973497+0000","last_clean":"2024-04-25T10:26:12.973497+0000","last_became_active":"2024-04-25T10:26:12.970360+0000","last_became_peered":"2024-04-25T10:26:12.970360+0000","last_unstale":"2024-04-25T10:26:12.973497+0000","last_undegraded":"2024-04-25T10:26:12.973497+0000","last_fullsized":"2024-04-25T10:26:12.973497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:50.722488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958521+0000","last_change":"2024-04-25T10:26:12.958521+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958521+0000","last_undegraded":"2024-04-25T10:26:12.958521+0000","last_fullsized":"2024-04-25T10:26:12.958521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977416+0000","last_change":"2024-04-25T10:26:12.972335+0000","last_active":"2024-04-25T10:26:12.977416+0000","last_peered":"2024-04-25T10:26:12.977416+0000","last_clean":"2024-04-25T10:26:12.977416+0000","last_became_active":"2024-04-25T10:26:12.972062+0000","last_became_peered":"2024-04-25T10:26:12.972062+0000","last_unstale":"2024-04-25T10:26:12.977416+0000","last_undegraded":"2024-04-25T10:26:12.977416+0000","last_fullsized":"2024-04-25T10:26:12.977416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.527361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958201+0000","last_change":"2024-04-25T10:26:12.958201+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958201+0000","last_undegraded":"2024-04-25T10:26:12.958201+0000","last_fullsized":"2024-04-25T10:26:12.958201+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977158+0000","last_change":"2024-04-25T10:26:12.974012+0000","last_active":"2024-04-25T10:26:12.977158+0000","last_peered":"2024-04-25T10:26:12.977158+0000","last_clean":"2024-04-25T10:26:12.977158+0000","last_became_active":"2024-04-25T10:26:12.973735+0000","last_became_peered":"2024-04-25T10:26:12.973735+0000","last_unstale":"2024-04-25T10:26:12.977158+0000","last_undegraded":"2024-04-25T10:26:12.977158+0000","last_fullsized":"2024-04-25T10:26:12.977158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.821837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955301+0000","last_change":"2024-04-25T10:26:12.955301+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955301+0000","last_undegraded":"2024-04-25T10:26:12.955301+0000","last_fullsized":"2024-04-25T10:26:12.955301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973705+0000","last_change":"2024-04-25T10:26:12.972502+0000","last_active":"2024-04-25T10:26:12.973705+0000","last_peered":"2024-04-25T10:26:12.973705+0000","last_clean":"2024-04-25T10:26:12.973705+0000","last_became_active":"2024-04-25T10:26:12.972135+0000","last_became_peered":"2024-04-25T10:26:12.972135+0000","last_unstale":"2024-04-25T10:26:12.973705+0000","last_undegraded":"2024-04-25T10:26:12.973705+0000","last_fullsized":"2024-04-25T10:26:12.973705+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:14.890002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963858+0000","last_change":"2024-04-25T10:26:12.963858+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963858+0000","last_undegraded":"2024-04-25T10:26:12.963858+0000","last_fullsized":"2024-04-25T10:26:12.963858+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.975964+0000","last_change":"2024-04-25T10:26:12.975104+0000","last_active":"2024-04-25T10:26:12.975964+0000","last_peered":"2024-04-25T10:26:12.975964+0000","last_clean":"2024-04-25T10:26:12.975964+0000","last_became_active":"2024-04-25T10:26:12.974542+0000","last_became_peered":"2024-04-25T10:26:12.974542+0000","last_unstale":"2024-04-25T10:26:12.975964+0000","last_undegraded":"2024-04-25T10:26:12.975964+0000","last_fullsized":"2024-04-25T10:26:12.975964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:43.021754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955308+0000","last_change":"2024-04-25T10:26:12.955308+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955308+0000","last_undegraded":"2024-04-25T10:26:12.955308+0000","last_fullsized":"2024-04-25T10:26:12.955308+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977436+0000","last_change":"2024-04-25T10:26:12.974951+0000","last_active":"2024-04-25T10:26:12.977436+0000","last_peered":"2024-04-25T10:26:12.977436+0000","last_clean":"2024-04-25T10:26:12.977436+0000","last_became_active":"2024-04-25T10:26:12.974439+0000","last_became_peered":"2024-04-25T10:26:12.974439+0000","last_unstale":"2024-04-25T10:26:12.977436+0000","last_undegraded":"2024-04-25T10:26:12.977436+0000","last_fullsized":"2024-04-25T10:26:12.977436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:16.500633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958165+0000","last_change":"2024-04-25T10:26:12.958165+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958165+0000","last_undegraded":"2024-04-25T10:26:12.958165+0000","last_fullsized":"2024-04-25T10:26:12.958165+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973079+0000","last_change":"2024-04-25T10:26:12.971625+0000","last_active":"2024-04-25T10:26:12.973079+0000","last_peered":"2024-04-25T10:26:12.973079+0000","last_clean":"2024-04-25T10:26:12.973079+0000","last_became_active":"2024-04-25T10:26:12.971364+0000","last_became_peered":"2024-04-25T10:26:12.971364+0000","last_unstale":"2024-04-25T10:26:12.973079+0000","last_undegraded":"2024-04-25T10:26:12.973079+0000","last_fullsized":"2024-04-25T10:26:12.973079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:13.635613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963797+0000","last_change":"2024-04-25T10:26:12.963797+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963797+0000","last_undegraded":"2024-04-25T10:26:12.963797+0000","last_fullsized":"2024-04-25T10:26:12.963797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949756+0000","last_change":"2024-04-25T10:26:11.949756+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949756+0000","last_undegraded":"2024-04-25T10:26:11.949756+0000","last_fullsized":"2024-04-25T10:26:11.949756+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963044+0000","last_change":"2024-04-25T10:26:12.963044+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963044+0000","last_undegraded":"2024-04-25T10:26:12.963044+0000","last_fullsized":"2024-04-25T10:26:12.963044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949650+0000","last_change":"2024-04-25T10:26:11.949650+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949650+0000","last_undegraded":"2024-04-25T10:26:11.949650+0000","last_fullsized":"2024-04-25T10:26:11.949650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962374+0000","last_change":"2024-04-25T10:26:12.962374+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962374+0000","last_undegraded":"2024-04-25T10:26:12.962374+0000","last_fullsized":"2024-04-25T10:26:12.962374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949647+0000","last_change":"2024-04-25T10:26:11.949647+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949647+0000","last_undegraded":"2024-04-25T10:26:11.949647+0000","last_fullsized":"2024-04-25T10:26:11.949647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962208+0000","last_change":"2024-04-25T10:26:12.962208+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962208+0000","last_undegraded":"2024-04-25T10:26:12.962208+0000","last_fullsized":"2024-04-25T10:26:12.962208+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973683+0000","last_change":"2024-04-25T10:26:12.972745+0000","last_active":"2024-04-25T10:26:12.973683+0000","last_peered":"2024-04-25T10:26:12.973683+0000","last_clean":"2024-04-25T10:26:12.973683+0000","last_became_active":"2024-04-25T10:26:12.972443+0000","last_became_peered":"2024-04-25T10:26:12.972443+0000","last_unstale":"2024-04-25T10:26:12.973683+0000","last_undegraded":"2024-04-25T10:26:12.973683+0000","last_fullsized":"2024-04-25T10:26:12.973683+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:57.380293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960977+0000","last_change":"2024-04-25T10:26:12.960977+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960977+0000","last_undegraded":"2024-04-25T10:26:12.960977+0000","last_fullsized":"2024-04-25T10:26:12.960977+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949642+0000","last_change":"2024-04-25T10:26:11.949642+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949642+0000","last_undegraded":"2024-04-25T10:26:11.949642+0000","last_fullsized":"2024-04-25T10:26:11.949642+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963427+0000","last_change":"2024-04-25T10:26:12.963427+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963427+0000","last_undegraded":"2024-04-25T10:26:12.963427+0000","last_fullsized":"2024-04-25T10:26:12.963427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950177+0000","last_change":"2024-04-25T10:26:11.950177+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950177+0000","last_undegraded":"2024-04-25T10:26:11.950177+0000","last_fullsized":"2024-04-25T10:26:11.950177+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.954750+0000","last_change":"2024-04-25T10:26:12.954750+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.954750+0000","last_undegraded":"2024-04-25T10:26:12.954750+0000","last_fullsized":"2024-04-25T10:26:12.954750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.969031+0000","last_change":"2024-04-25T10:26:12.965357+0000","last_active":"2024-04-25T10:26:12.969031+0000","last_peered":"2024-04-25T10:26:12.969031+0000","last_clean":"2024-04-25T10:26:12.969031+0000","last_became_active":"2024-04-25T10:26:12.964963+0000","last_became_peered":"2024-04-25T10:26:12.964963+0000","last_unstale":"2024-04-25T10:26:12.969031+0000","last_undegraded":"2024-04-25T10:26:12.969031+0000","last_fullsized":"2024-04-25T10:26:12.969031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:17.853096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961968+0000","last_change":"2024-04-25T10:26:12.961968+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961968+0000","last_undegraded":"2024-04-25T10:26:12.961968+0000","last_fullsized":"2024-04-25T10:26:12.961968+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970394+0000","last_change":"2024-04-25T10:26:12.968833+0000","last_active":"2024-04-25T10:26:12.970394+0000","last_peered":"2024-04-25T10:26:12.970394+0000","last_clean":"2024-04-25T10:26:12.970394+0000","last_became_active":"2024-04-25T10:26:12.968544+0000","last_became_peered":"2024-04-25T10:26:12.968544+0000","last_unstale":"2024-04-25T10:26:12.970394+0000","last_undegraded":"2024-04-25T10:26:12.970394+0000","last_fullsized":"2024-04-25T10:26:12.970394+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961084+0000","last_change":"2024-04-25T10:26:12.961084+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961084+0000","last_undegraded":"2024-04-25T10:26:12.961084+0000","last_fullsized":"2024-04-25T10:26:12.961084+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.972809+0000","last_change":"2024-04-25T10:26:12.970872+0000","last_active":"2024-04-25T10:26:12.972809+0000","last_peered":"2024-04-25T10:26:12.972809+0000","last_clean":"2024-04-25T10:26:12.972809+0000","last_became_active":"2024-04-25T10:26:12.970594+0000","last_became_peered":"2024-04-25T10:26:12.970594+0000","last_unstale":"2024-04-25T10:26:12.972809+0000","last_undegraded":"2024-04-25T10:26:12.972809+0000","last_fullsized":"2024-04-25T10:26:12.972809+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.206365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957844+0000","last_change":"2024-04-25T10:26:12.957844+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957844+0000","last_undegraded":"2024-04-25T10:26:12.957844+0000","last_fullsized":"2024-04-25T10:26:12.957844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973677+0000","last_change":"2024-04-25T10:26:12.969936+0000","last_active":"2024-04-25T10:26:12.973677+0000","last_peered":"2024-04-25T10:26:12.973677+0000","last_clean":"2024-04-25T10:26:12.973677+0000","last_became_active":"2024-04-25T10:26:12.969633+0000","last_became_peered":"2024-04-25T10:26:12.969633+0000","last_unstale":"2024-04-25T10:26:12.973677+0000","last_undegraded":"2024-04-25T10:26:12.973677+0000","last_fullsized":"2024-04-25T10:26:12.973677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970691+0000","last_change":"2024-04-25T10:26:12.968805+0000","last_active":"2024-04-25T10:26:12.970691+0000","last_peered":"2024-04-25T10:26:12.970691+0000","last_clean":"2024-04-25T10:26:12.970691+0000","last_became_active":"2024-04-25T10:26:12.968534+0000","last_became_peered":"2024-04-25T10:26:12.968534+0000","last_unstale":"2024-04-25T10:26:12.970691+0000","last_undegraded":"2024-04-25T10:26:12.970691+0000","last_fullsized":"2024-04-25T10:26:12.970691+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:27.777514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":347,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.958158+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:26:12.958158+0000","last_peered":"2024-04-25T10:26:12.958158+0000","last_clean":"2024-04-25T10:26:12.958158+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:26:12.958158+0000","last_undegraded":"2024-04-25T10:26:12.958158+0000","last_fullsized":"2024-04-25T10:26:12.958158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50616,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321224,"statfs":{"total":96636764160,"available":96584933376,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509545,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50624,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321216,"statfs":{"total":96636764160,"available":96584925184,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2015232,"data_stored":1204785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2015232,"data_stored":1204785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28416,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343424,"statfs":{"total":96636764160,"available":96607666176,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542247,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58924,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312916,"statfs":{"total":96636764160,"available":96576425984,"internally_reserved":0,"allocated":1409024,"data_stored":608142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:18.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 Processor -- start 2024-04-25T10:26:18.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 -- start start 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb200ccb20 0x7fdb200d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb2015bdc0 0x7fdb200d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb200ccfb0 con 0x7fdb2015b9e0 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb200cd290 con 0x7fdb2015bdc0 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb25800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200cd570 con 0x7fdb200ccb20 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb2015bdc0 0x7fdb200d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb2015bdc0 0x7fdb200d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37656/0 (socket says 172.21.15.45:37656) 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 -- 172.21.15.45:0/4074992075 learned_addr learned my addr 172.21.15.45:0/4074992075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1e000640 1 --2- 172.21.15.45:0/4074992075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb200ccb20 0x7fdb200d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 -- 172.21.15.45:0/4074992075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb200ccb20 msgr2=0x7fdb200d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 --2- 172.21.15.45:0/4074992075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb200ccb20 0x7fdb200d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 -- 172.21.15.45:0/4074992075 >> v1:172.21.15.117:6789/0 conn(0x7fdb2015b9e0 legacy=0x7fdb200cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1f400640 1 -- 172.21.15.45:0/4074992075 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200cd850 con 0x7fdb2015bdc0 2024-04-25T10:26:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.687+0000 7fdb1d600640 1 -- 172.21.15.45:0/4074992075 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225235732 0 0) 0x7fdb200ccfb0 con 0x7fdb2015b9e0 2024-04-25T10:26:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb1f400640 1 --2- 172.21.15.45:0/4074992075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015bdc0 0x7fdb200d1a60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009e50 tx=0x7fdb1801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5a616fbdfd8b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb1d600640 1 -- 172.21.15.45:0/4074992075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180a5700 con 0x7fdb2015bdc0 2024-04-25T10:26:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb1d600640 1 -- 172.21.15.45:0/4074992075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb180a58a0 con 0x7fdb2015bdc0 2024-04-25T10:26:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb1d600640 1 -- 172.21.15.45:0/4074992075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180a5c70 con 0x7fdb2015bdc0 2024-04-25T10:26:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 -- 172.21.15.45:0/4074992075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015bdc0 msgr2=0x7fdb200d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 --2- 172.21.15.45:0/4074992075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015bdc0 0x7fdb200d1a60 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009e50 tx=0x7fdb1801ca30 comp rx=0 tx=0).stop 2024-04-25T10:26:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 -- 172.21.15.45:0/4074992075 shutdown_connections 2024-04-25T10:26:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 --2- 172.21.15.45:0/4074992075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015bdc0 0x7fdb200d1a60 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 --2- 172.21.15.45:0/4074992075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb200ccb20 0x7fdb200d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 -- 172.21.15.45:0/4074992075 >> 172.21.15.45:0/4074992075 conn(0x7fdb2009a9d0 msgr2=0x7fdb2009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 -- 172.21.15.45:0/4074992075 shutdown_connections 2024-04-25T10:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 -- 172.21.15.45:0/4074992075 wait complete. 2024-04-25T10:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 Processor -- start 2024-04-25T10:26:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.691+0000 7fdb25800640 1 -- start start 2024-04-25T10:26:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 0x7fdb201fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2015bdc0 0x7fdb201fde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb201fe360 0x7fdb201fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200c65b0 con 0x7fdb2015b9e0 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200c6430 con 0x7fdb2015bdc0 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200c6730 con 0x7fdb201fe360 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 0x7fdb201fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 0x7fdb201fd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37672/0 (socket says 172.21.15.45:37672) 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 -- 172.21.15.45:0/989825606 learned_addr learned my addr 172.21.15.45:0/989825606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1e000640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2015bdc0 0x7fdb201fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1f400640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb201fe360 0x7fdb201fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 -- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb201fe360 msgr2=0x7fdb201fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb201fe360 0x7fdb201fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 -- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2015bdc0 msgr2=0x7fdb201fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2015bdc0 0x7fdb201fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 -- 172.21.15.45:0/989825606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb201fc640 con 0x7fdb2015b9e0 2024-04-25T10:26:18.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1e000640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2015bdc0 0x7fdb201fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb1ea00640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 0x7fdb201fd6e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000c500 tx=0x7fdb1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10017070 con 0x7fdb2015b9e0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb10003600 con 0x7fdb2015b9e0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb100039f0 con 0x7fdb2015b9e0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb201fc920 con 0x7fdb2015b9e0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb201fcee0 con 0x7fdb2015b9e0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.695+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb201fe770 con 0x7fdb2015b9e0 2024-04-25T10:26:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.699+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb1001a070 con 0x7fdb2015b9e0 2024-04-25T10:26:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.699+0000 7fdb16a00640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb00066f60 0x7fdb00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.699+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb10084440 con 0x7fdb2015b9e0 2024-04-25T10:26:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.699+0000 7fdb1e000640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb00066f60 0x7fdb00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.699+0000 7fdb1e000640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb00066f60 0x7fdb00069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdb2006ff30 tx=0x7fdb0c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.703+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb201fe770 con 0x7fdb2015b9e0 2024-04-25T10:26:18.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.859+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb2004ef30 con 0x7fdb00066f60 2024-04-25T10:26:18.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.867+0000 7fdb16a00640 1 -- 172.21.15.45:0/989825606 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+245874 (secure 0 0 0) 0x7fdb2004ef30 con 0x7fdb00066f60 2024-04-25T10:26:18.870 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:18.872 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb00066f60 msgr2=0x7fdb00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb00066f60 0x7fdb00069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdb2006ff30 tx=0x7fdb0c009190 comp rx=0 tx=0).stop 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 msgr2=0x7fdb201fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 0x7fdb201fd6e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000c500 tx=0x7fdb1000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 shutdown_connections 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb00066f60 0x7fdb00069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb201fe360 0x7fdb201fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb2015bdc0 0x7fdb201fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 --2- 172.21.15.45:0/989825606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb2015b9e0 0x7fdb201fd6e0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:18.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 >> 172.21.15.45:0/989825606 conn(0x7fdb2009a9d0 msgr2=0x7fdb2006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:18.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 shutdown_connections 2024-04-25T10:26:18.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:18.871+0000 7fdb25800640 1 -- 172.21.15.45:0/989825606 wait complete. 2024-04-25T10:26:18.892 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T10:26:16.989392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":180,"acting":180,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417176,"kb_used_data":12252,"kb_used_omap":12,"kb_used_meta":404787,"kb_avail":754557544,"statfs":{"total":773094113280,"available":772666925056,"internally_reserved":0,"allocated":12546048,"data_stored":6089797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":414502448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042706"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960603+0000","last_change":"2024-04-25T10:26:12.960603+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960603+0000","last_undegraded":"2024-04-25T10:26:12.960603+0000","last_fullsized":"2024-04-25T10:26:12.960603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958621+0000","last_change":"2024-04-25T10:26:12.958621+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958621+0000","last_undegraded":"2024-04-25T10:26:12.958621+0000","last_fullsized":"2024-04-25T10:26:12.958621+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957509+0000","last_change":"2024-04-25T10:26:12.957509+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957509+0000","last_undegraded":"2024-04-25T10:26:12.957509+0000","last_fullsized":"2024-04-25T10:26:12.957509+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960968+0000","last_change":"2024-04-25T10:26:12.960968+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960968+0000","last_undegraded":"2024-04-25T10:26:12.960968+0000","last_fullsized":"2024-04-25T10:26:12.960968+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960557+0000","last_change":"2024-04-25T10:26:12.960557+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960557+0000","last_undegraded":"2024-04-25T10:26:12.960557+0000","last_fullsized":"2024-04-25T10:26:12.960557+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957902+0000","last_change":"2024-04-25T10:26:12.957902+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957902+0000","last_undegraded":"2024-04-25T10:26:12.957902+0000","last_fullsized":"2024-04-25T10:26:12.957902+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955820+0000","last_change":"2024-04-25T10:26:12.955820+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955820+0000","last_undegraded":"2024-04-25T10:26:12.955820+0000","last_fullsized":"2024-04-25T10:26:12.955820+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960167+0000","last_change":"2024-04-25T10:26:12.960167+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960167+0000","last_undegraded":"2024-04-25T10:26:12.960167+0000","last_fullsized":"2024-04-25T10:26:12.960167+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963662+0000","last_change":"2024-04-25T10:26:12.963662+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963662+0000","last_undegraded":"2024-04-25T10:26:12.963662+0000","last_fullsized":"2024-04-25T10:26:12.963662+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958648+0000","last_change":"2024-04-25T10:26:12.958648+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958648+0000","last_undegraded":"2024-04-25T10:26:12.958648+0000","last_fullsized":"2024-04-25T10:26:12.958648+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960992+0000","last_change":"2024-04-25T10:26:12.960992+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960992+0000","last_undegraded":"2024-04-25T10:26:12.960992+0000","last_fullsized":"2024-04-25T10:26:12.960992+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.954373+0000","last_change":"2024-04-25T10:26:12.954373+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.954373+0000","last_undegraded":"2024-04-25T10:26:12.954373+0000","last_fullsized":"2024-04-25T10:26:12.954373+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962360+0000","last_change":"2024-04-25T10:26:12.962360+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962360+0000","last_undegraded":"2024-04-25T10:26:12.962360+0000","last_fullsized":"2024-04-25T10:26:12.962360+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960133+0000","last_change":"2024-04-25T10:26:12.960133+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960133+0000","last_undegraded":"2024-04-25T10:26:12.960133+0000","last_fullsized":"2024-04-25T10:26:12.960133+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963701+0000","last_change":"2024-04-25T10:26:12.963701+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963701+0000","last_undegraded":"2024-04-25T10:26:12.963701+0000","last_fullsized":"2024-04-25T10:26:12.963701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955316+0000","last_change":"2024-04-25T10:26:12.955316+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955316+0000","last_undegraded":"2024-04-25T10:26:12.955316+0000","last_fullsized":"2024-04-25T10:26:12.955316+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956443+0000","last_change":"2024-04-25T10:26:12.956443+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956443+0000","last_undegraded":"2024-04-25T10:26:12.956443+0000","last_fullsized":"2024-04-25T10:26:12.956443+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955246+0000","last_change":"2024-04-25T10:26:12.955246+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955246+0000","last_undegraded":"2024-04-25T10:26:12.955246+0000","last_fullsized":"2024-04-25T10:26:12.955246+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955483+0000","last_change":"2024-04-25T10:26:12.955483+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955483+0000","last_undegraded":"2024-04-25T10:26:12.955483+0000","last_fullsized":"2024-04-25T10:26:12.955483+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956466+0000","last_change":"2024-04-25T10:26:12.956466+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956466+0000","last_undegraded":"2024-04-25T10:26:12.956466+0000","last_fullsized":"2024-04-25T10:26:12.956466+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960106+0000","last_change":"2024-04-25T10:26:12.960106+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960106+0000","last_undegraded":"2024-04-25T10:26:12.960106+0000","last_fullsized":"2024-04-25T10:26:12.960106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961108+0000","last_change":"2024-04-25T10:26:12.961108+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961108+0000","last_undegraded":"2024-04-25T10:26:12.961108+0000","last_fullsized":"2024-04-25T10:26:12.961108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955944+0000","last_change":"2024-04-25T10:26:12.955944+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955944+0000","last_undegraded":"2024-04-25T10:26:12.955944+0000","last_fullsized":"2024-04-25T10:26:12.955944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961486+0000","last_change":"2024-04-25T10:26:12.961486+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961486+0000","last_undegraded":"2024-04-25T10:26:12.961486+0000","last_fullsized":"2024-04-25T10:26:12.961486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959667+0000","last_change":"2024-04-25T10:26:12.959667+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959667+0000","last_undegraded":"2024-04-25T10:26:12.959667+0000","last_fullsized":"2024-04-25T10:26:12.959667+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963326+0000","last_change":"2024-04-25T10:26:12.963326+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963326+0000","last_undegraded":"2024-04-25T10:26:12.963326+0000","last_fullsized":"2024-04-25T10:26:12.963326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955697+0000","last_change":"2024-04-25T10:26:12.955697+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955697+0000","last_undegraded":"2024-04-25T10:26:12.955697+0000","last_fullsized":"2024-04-25T10:26:12.955697+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958589+0000","last_change":"2024-04-25T10:26:12.958589+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958589+0000","last_undegraded":"2024-04-25T10:26:12.958589+0000","last_fullsized":"2024-04-25T10:26:12.958589+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963268+0000","last_change":"2024-04-25T10:26:12.963268+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963268+0000","last_undegraded":"2024-04-25T10:26:12.963268+0000","last_fullsized":"2024-04-25T10:26:12.963268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959445+0000","last_change":"2024-04-25T10:26:12.959445+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959445+0000","last_undegraded":"2024-04-25T10:26:12.959445+0000","last_fullsized":"2024-04-25T10:26:12.959445+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.974161+0000","last_change":"2024-04-25T10:26:12.973421+0000","last_active":"2024-04-25T10:26:12.974161+0000","last_peered":"2024-04-25T10:26:12.974161+0000","last_clean":"2024-04-25T10:26:12.974161+0000","last_became_active":"2024-04-25T10:26:12.973108+0000","last_became_peered":"2024-04-25T10:26:12.973108+0000","last_unstale":"2024-04-25T10:26:12.974161+0000","last_undegraded":"2024-04-25T10:26:12.974161+0000","last_fullsized":"2024-04-25T10:26:12.974161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.116774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962804+0000","last_change":"2024-04-25T10:26:12.962804+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962804+0000","last_undegraded":"2024-04-25T10:26:12.962804+0000","last_fullsized":"2024-04-25T10:26:12.962804+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971928+0000","last_change":"2024-04-25T10:26:12.965663+0000","last_active":"2024-04-25T10:26:12.971928+0000","last_peered":"2024-04-25T10:26:12.971928+0000","last_clean":"2024-04-25T10:26:12.971928+0000","last_became_active":"2024-04-25T10:26:12.965358+0000","last_became_peered":"2024-04-25T10:26:12.965358+0000","last_unstale":"2024-04-25T10:26:12.971928+0000","last_undegraded":"2024-04-25T10:26:12.971928+0000","last_fullsized":"2024-04-25T10:26:12.971928+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:53.575119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.959310+0000","last_change":"2024-04-25T10:26:12.959310+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.959310+0000","last_undegraded":"2024-04-25T10:26:12.959310+0000","last_fullsized":"2024-04-25T10:26:12.959310+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950769+0000","last_change":"2024-04-25T10:26:11.950769+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950769+0000","last_undegraded":"2024-04-25T10:26:11.950769+0000","last_fullsized":"2024-04-25T10:26:11.950769+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957091+0000","last_change":"2024-04-25T10:26:12.957091+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957091+0000","last_undegraded":"2024-04-25T10:26:12.957091+0000","last_fullsized":"2024-04-25T10:26:12.957091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.951276+0000","last_change":"2024-04-25T10:26:11.951276+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.951276+0000","last_undegraded":"2024-04-25T10:26:11.951276+0000","last_fullsized":"2024-04-25T10:26:11.951276+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971383+0000","last_change":"2024-04-25T10:26:12.969937+0000","last_active":"2024-04-25T10:26:12.971383+0000","last_peered":"2024-04-25T10:26:12.971383+0000","last_clean":"2024-04-25T10:26:12.971383+0000","last_became_active":"2024-04-25T10:26:12.969666+0000","last_became_peered":"2024-04-25T10:26:12.969666+0000","last_unstale":"2024-04-25T10:26:12.971383+0000","last_undegraded":"2024-04-25T10:26:12.971383+0000","last_fullsized":"2024-04-25T10:26:12.971383+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:45.691111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.956910+0000","last_change":"2024-04-25T10:26:12.956910+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.956910+0000","last_undegraded":"2024-04-25T10:26:12.956910+0000","last_fullsized":"2024-04-25T10:26:12.956910+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.969780+0000","last_change":"2024-04-25T10:26:12.968858+0000","last_active":"2024-04-25T10:26:12.969780+0000","last_peered":"2024-04-25T10:26:12.969780+0000","last_clean":"2024-04-25T10:26:12.969780+0000","last_became_active":"2024-04-25T10:26:12.968309+0000","last_became_peered":"2024-04-25T10:26:12.968309+0000","last_unstale":"2024-04-25T10:26:12.969780+0000","last_undegraded":"2024-04-25T10:26:12.969780+0000","last_fullsized":"2024-04-25T10:26:12.969780+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:38.622906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961042+0000","last_change":"2024-04-25T10:26:12.961042+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961042+0000","last_undegraded":"2024-04-25T10:26:12.961042+0000","last_fullsized":"2024-04-25T10:26:12.961042+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970364+0000","last_change":"2024-04-25T10:26:12.968507+0000","last_active":"2024-04-25T10:26:12.970364+0000","last_peered":"2024-04-25T10:26:12.970364+0000","last_clean":"2024-04-25T10:26:12.970364+0000","last_became_active":"2024-04-25T10:26:12.967779+0000","last_became_peered":"2024-04-25T10:26:12.967779+0000","last_unstale":"2024-04-25T10:26:12.970364+0000","last_undegraded":"2024-04-25T10:26:12.970364+0000","last_fullsized":"2024-04-25T10:26:12.970364+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.385060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957397+0000","last_change":"2024-04-25T10:26:12.957397+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957397+0000","last_undegraded":"2024-04-25T10:26:12.957397+0000","last_fullsized":"2024-04-25T10:26:12.957397+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977221+0000","last_change":"2024-04-25T10:26:12.974852+0000","last_active":"2024-04-25T10:26:12.977221+0000","last_peered":"2024-04-25T10:26:12.977221+0000","last_clean":"2024-04-25T10:26:12.977221+0000","last_became_active":"2024-04-25T10:26:12.974599+0000","last_became_peered":"2024-04-25T10:26:12.974599+0000","last_unstale":"2024-04-25T10:26:12.977221+0000","last_undegraded":"2024-04-25T10:26:12.977221+0000","last_fullsized":"2024-04-25T10:26:12.977221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:17.313609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.953984+0000","last_change":"2024-04-25T10:26:12.953984+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.953984+0000","last_undegraded":"2024-04-25T10:26:12.953984+0000","last_fullsized":"2024-04-25T10:26:12.953984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950616+0000","last_change":"2024-04-25T10:26:11.950616+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950616+0000","last_undegraded":"2024-04-25T10:26:11.950616+0000","last_fullsized":"2024-04-25T10:26:11.950616+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.953982+0000","last_change":"2024-04-25T10:26:12.953982+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.953982+0000","last_undegraded":"2024-04-25T10:26:12.953982+0000","last_fullsized":"2024-04-25T10:26:12.953982+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.978516+0000","last_change":"2024-04-25T10:26:12.974260+0000","last_active":"2024-04-25T10:26:12.978516+0000","last_peered":"2024-04-25T10:26:12.978516+0000","last_clean":"2024-04-25T10:26:12.978516+0000","last_became_active":"2024-04-25T10:26:12.973967+0000","last_became_peered":"2024-04-25T10:26:12.973967+0000","last_unstale":"2024-04-25T10:26:12.978516+0000","last_undegraded":"2024-04-25T10:26:12.978516+0000","last_fullsized":"2024-04-25T10:26:12.978516+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:53.892616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958876+0000","last_change":"2024-04-25T10:26:12.958876+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958876+0000","last_undegraded":"2024-04-25T10:26:12.958876+0000","last_fullsized":"2024-04-25T10:26:12.958876+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.980535+0000","last_change":"2024-04-25T10:26:12.979707+0000","last_active":"2024-04-25T10:26:12.980535+0000","last_peered":"2024-04-25T10:26:12.980535+0000","last_clean":"2024-04-25T10:26:12.980535+0000","last_became_active":"2024-04-25T10:26:12.979419+0000","last_became_peered":"2024-04-25T10:26:12.979419+0000","last_unstale":"2024-04-25T10:26:12.980535+0000","last_undegraded":"2024-04-25T10:26:12.980535+0000","last_fullsized":"2024-04-25T10:26:12.980535+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:54.363182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955903+0000","last_change":"2024-04-25T10:26:12.955903+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955903+0000","last_undegraded":"2024-04-25T10:26:12.955903+0000","last_fullsized":"2024-04-25T10:26:12.955903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950904+0000","last_change":"2024-04-25T10:26:11.950904+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950904+0000","last_undegraded":"2024-04-25T10:26:11.950904+0000","last_fullsized":"2024-04-25T10:26:11.950904+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960737+0000","last_change":"2024-04-25T10:26:12.960737+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960737+0000","last_undegraded":"2024-04-25T10:26:12.960737+0000","last_fullsized":"2024-04-25T10:26:12.960737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977415+0000","last_change":"2024-04-25T10:26:12.974720+0000","last_active":"2024-04-25T10:26:12.977415+0000","last_peered":"2024-04-25T10:26:12.977415+0000","last_clean":"2024-04-25T10:26:12.977415+0000","last_became_active":"2024-04-25T10:26:12.974006+0000","last_became_peered":"2024-04-25T10:26:12.974006+0000","last_unstale":"2024-04-25T10:26:12.977415+0000","last_undegraded":"2024-04-25T10:26:12.977415+0000","last_fullsized":"2024-04-25T10:26:12.977415+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:06.748603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.971459+0000","last_change":"2024-04-25T10:26:12.970303+0000","last_active":"2024-04-25T10:26:12.971459+0000","last_peered":"2024-04-25T10:26:12.971459+0000","last_clean":"2024-04-25T10:26:12.971459+0000","last_became_active":"2024-04-25T10:26:12.969076+0000","last_became_peered":"2024-04-25T10:26:12.969076+0000","last_unstale":"2024-04-25T10:26:12.971459+0000","last_undegraded":"2024-04-25T10:26:12.971459+0000","last_fullsized":"2024-04-25T10:26:12.971459+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973497+0000","last_change":"2024-04-25T10:26:12.970708+0000","last_active":"2024-04-25T10:26:12.973497+0000","last_peered":"2024-04-25T10:26:12.973497+0000","last_clean":"2024-04-25T10:26:12.973497+0000","last_became_active":"2024-04-25T10:26:12.970360+0000","last_became_peered":"2024-04-25T10:26:12.970360+0000","last_unstale":"2024-04-25T10:26:12.973497+0000","last_undegraded":"2024-04-25T10:26:12.973497+0000","last_fullsized":"2024-04-25T10:26:12.973497+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:50.722488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958521+0000","last_change":"2024-04-25T10:26:12.958521+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958521+0000","last_undegraded":"2024-04-25T10:26:12.958521+0000","last_fullsized":"2024-04-25T10:26:12.958521+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977416+0000","last_change":"2024-04-25T10:26:12.972335+0000","last_active":"2024-04-25T10:26:12.977416+0000","last_peered":"2024-04-25T10:26:12.977416+0000","last_clean":"2024-04-25T10:26:12.977416+0000","last_became_active":"2024-04-25T10:26:12.972062+0000","last_became_peered":"2024-04-25T10:26:12.972062+0000","last_unstale":"2024-04-25T10:26:12.977416+0000","last_undegraded":"2024-04-25T10:26:12.977416+0000","last_fullsized":"2024-04-25T10:26:12.977416+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.527361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958201+0000","last_change":"2024-04-25T10:26:12.958201+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958201+0000","last_undegraded":"2024-04-25T10:26:12.958201+0000","last_fullsized":"2024-04-25T10:26:12.958201+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977158+0000","last_change":"2024-04-25T10:26:12.974012+0000","last_active":"2024-04-25T10:26:12.977158+0000","last_peered":"2024-04-25T10:26:12.977158+0000","last_clean":"2024-04-25T10:26:12.977158+0000","last_became_active":"2024-04-25T10:26:12.973735+0000","last_became_peered":"2024-04-25T10:26:12.973735+0000","last_unstale":"2024-04-25T10:26:12.977158+0000","last_undegraded":"2024-04-25T10:26:12.977158+0000","last_fullsized":"2024-04-25T10:26:12.977158+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.821837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955301+0000","last_change":"2024-04-25T10:26:12.955301+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955301+0000","last_undegraded":"2024-04-25T10:26:12.955301+0000","last_fullsized":"2024-04-25T10:26:12.955301+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973705+0000","last_change":"2024-04-25T10:26:12.972502+0000","last_active":"2024-04-25T10:26:12.973705+0000","last_peered":"2024-04-25T10:26:12.973705+0000","last_clean":"2024-04-25T10:26:12.973705+0000","last_became_active":"2024-04-25T10:26:12.972135+0000","last_became_peered":"2024-04-25T10:26:12.972135+0000","last_unstale":"2024-04-25T10:26:12.973705+0000","last_undegraded":"2024-04-25T10:26:12.973705+0000","last_fullsized":"2024-04-25T10:26:12.973705+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:14.890002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963858+0000","last_change":"2024-04-25T10:26:12.963858+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963858+0000","last_undegraded":"2024-04-25T10:26:12.963858+0000","last_fullsized":"2024-04-25T10:26:12.963858+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.975964+0000","last_change":"2024-04-25T10:26:12.975104+0000","last_active":"2024-04-25T10:26:12.975964+0000","last_peered":"2024-04-25T10:26:12.975964+0000","last_clean":"2024-04-25T10:26:12.975964+0000","last_became_active":"2024-04-25T10:26:12.974542+0000","last_became_peered":"2024-04-25T10:26:12.974542+0000","last_unstale":"2024-04-25T10:26:12.975964+0000","last_undegraded":"2024-04-25T10:26:12.975964+0000","last_fullsized":"2024-04-25T10:26:12.975964+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:43.021754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.955308+0000","last_change":"2024-04-25T10:26:12.955308+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.955308+0000","last_undegraded":"2024-04-25T10:26:12.955308+0000","last_fullsized":"2024-04-25T10:26:12.955308+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.977436+0000","last_change":"2024-04-25T10:26:12.974951+0000","last_active":"2024-04-25T10:26:12.977436+0000","last_peered":"2024-04-25T10:26:12.977436+0000","last_clean":"2024-04-25T10:26:12.977436+0000","last_became_active":"2024-04-25T10:26:12.974439+0000","last_became_peered":"2024-04-25T10:26:12.974439+0000","last_unstale":"2024-04-25T10:26:12.977436+0000","last_undegraded":"2024-04-25T10:26:12.977436+0000","last_fullsized":"2024-04-25T10:26:12.977436+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:16.500633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.958165+0000","last_change":"2024-04-25T10:26:12.958165+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.958165+0000","last_undegraded":"2024-04-25T10:26:12.958165+0000","last_fullsized":"2024-04-25T10:26:12.958165+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973079+0000","last_change":"2024-04-25T10:26:12.971625+0000","last_active":"2024-04-25T10:26:12.973079+0000","last_peered":"2024-04-25T10:26:12.973079+0000","last_clean":"2024-04-25T10:26:12.973079+0000","last_became_active":"2024-04-25T10:26:12.971364+0000","last_became_peered":"2024-04-25T10:26:12.971364+0000","last_unstale":"2024-04-25T10:26:12.973079+0000","last_undegraded":"2024-04-25T10:26:12.973079+0000","last_fullsized":"2024-04-25T10:26:12.973079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:13.635613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963797+0000","last_change":"2024-04-25T10:26:12.963797+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963797+0000","last_undegraded":"2024-04-25T10:26:12.963797+0000","last_fullsized":"2024-04-25T10:26:12.963797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949756+0000","last_change":"2024-04-25T10:26:11.949756+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949756+0000","last_undegraded":"2024-04-25T10:26:11.949756+0000","last_fullsized":"2024-04-25T10:26:11.949756+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963044+0000","last_change":"2024-04-25T10:26:12.963044+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963044+0000","last_undegraded":"2024-04-25T10:26:12.963044+0000","last_fullsized":"2024-04-25T10:26:12.963044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949650+0000","last_change":"2024-04-25T10:26:11.949650+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949650+0000","last_undegraded":"2024-04-25T10:26:11.949650+0000","last_fullsized":"2024-04-25T10:26:11.949650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962374+0000","last_change":"2024-04-25T10:26:12.962374+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962374+0000","last_undegraded":"2024-04-25T10:26:12.962374+0000","last_fullsized":"2024-04-25T10:26:12.962374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949647+0000","last_change":"2024-04-25T10:26:11.949647+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949647+0000","last_undegraded":"2024-04-25T10:26:11.949647+0000","last_fullsized":"2024-04-25T10:26:11.949647+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.962208+0000","last_change":"2024-04-25T10:26:12.962208+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.962208+0000","last_undegraded":"2024-04-25T10:26:12.962208+0000","last_fullsized":"2024-04-25T10:26:12.962208+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973683+0000","last_change":"2024-04-25T10:26:12.972745+0000","last_active":"2024-04-25T10:26:12.973683+0000","last_peered":"2024-04-25T10:26:12.973683+0000","last_clean":"2024-04-25T10:26:12.973683+0000","last_became_active":"2024-04-25T10:26:12.972443+0000","last_became_peered":"2024-04-25T10:26:12.972443+0000","last_unstale":"2024-04-25T10:26:12.973683+0000","last_undegraded":"2024-04-25T10:26:12.973683+0000","last_fullsized":"2024-04-25T10:26:12.973683+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:57.380293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.960977+0000","last_change":"2024-04-25T10:26:12.960977+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.960977+0000","last_undegraded":"2024-04-25T10:26:12.960977+0000","last_fullsized":"2024-04-25T10:26:12.960977+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.949642+0000","last_change":"2024-04-25T10:26:11.949642+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.949642+0000","last_undegraded":"2024-04-25T10:26:11.949642+0000","last_fullsized":"2024-04-25T10:26:11.949642+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.963427+0000","last_change":"2024-04-25T10:26:12.963427+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.963427+0000","last_undegraded":"2024-04-25T10:26:12.963427+0000","last_fullsized":"2024-04-25T10:26:12.963427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":114,"state":"creating+peering","last_fresh":"2024-04-25T10:26:11.950177+0000","last_change":"2024-04-25T10:26:11.950177+0000","last_active":"2024-04-25T10:26:11.937740+0000","last_peered":"2024-04-25T10:26:11.937740+0000","last_clean":"2024-04-25T10:26:11.937740+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:11.950177+0000","last_undegraded":"2024-04-25T10:26:11.950177+0000","last_fullsized":"2024-04-25T10:26:11.950177+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.954750+0000","last_change":"2024-04-25T10:26:12.954750+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.954750+0000","last_undegraded":"2024-04-25T10:26:12.954750+0000","last_fullsized":"2024-04-25T10:26:12.954750+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.969031+0000","last_change":"2024-04-25T10:26:12.965357+0000","last_active":"2024-04-25T10:26:12.969031+0000","last_peered":"2024-04-25T10:26:12.969031+0000","last_clean":"2024-04-25T10:26:12.969031+0000","last_became_active":"2024-04-25T10:26:12.964963+0000","last_became_peered":"2024-04-25T10:26:12.964963+0000","last_unstale":"2024-04-25T10:26:12.969031+0000","last_undegraded":"2024-04-25T10:26:12.969031+0000","last_fullsized":"2024-04-25T10:26:12.969031+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:17.853096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961968+0000","last_change":"2024-04-25T10:26:12.961968+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961968+0000","last_undegraded":"2024-04-25T10:26:12.961968+0000","last_fullsized":"2024-04-25T10:26:12.961968+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970394+0000","last_change":"2024-04-25T10:26:12.968833+0000","last_active":"2024-04-25T10:26:12.970394+0000","last_peered":"2024-04-25T10:26:12.970394+0000","last_clean":"2024-04-25T10:26:12.970394+0000","last_became_active":"2024-04-25T10:26:12.968544+0000","last_became_peered":"2024-04-25T10:26:12.968544+0000","last_unstale":"2024-04-25T10:26:12.970394+0000","last_undegraded":"2024-04-25T10:26:12.970394+0000","last_fullsized":"2024-04-25T10:26:12.970394+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.961084+0000","last_change":"2024-04-25T10:26:12.961084+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.961084+0000","last_undegraded":"2024-04-25T10:26:12.961084+0000","last_fullsized":"2024-04-25T10:26:12.961084+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.972809+0000","last_change":"2024-04-25T10:26:12.970872+0000","last_active":"2024-04-25T10:26:12.972809+0000","last_peered":"2024-04-25T10:26:12.972809+0000","last_clean":"2024-04-25T10:26:12.972809+0000","last_became_active":"2024-04-25T10:26:12.970594+0000","last_became_peered":"2024-04-25T10:26:12.970594+0000","last_unstale":"2024-04-25T10:26:12.972809+0000","last_undegraded":"2024-04-25T10:26:12.972809+0000","last_fullsized":"2024-04-25T10:26:12.972809+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.206365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":115,"state":"creating+peering","last_fresh":"2024-04-25T10:26:12.957844+0000","last_change":"2024-04-25T10:26:12.957844+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.957844+0000","last_undegraded":"2024-04-25T10:26:12.957844+0000","last_fullsized":"2024-04-25T10:26:12.957844+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.973677+0000","last_change":"2024-04-25T10:26:12.969936+0000","last_active":"2024-04-25T10:26:12.973677+0000","last_peered":"2024-04-25T10:26:12.973677+0000","last_clean":"2024-04-25T10:26:12.973677+0000","last_became_active":"2024-04-25T10:26:12.969633+0000","last_became_peered":"2024-04-25T10:26:12.969633+0000","last_unstale":"2024-04-25T10:26:12.973677+0000","last_undegraded":"2024-04-25T10:26:12.973677+0000","last_fullsized":"2024-04-25T10:26:12.973677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:26:12.944055+0000","last_change":"2024-04-25T10:26:12.944055+0000","last_active":"2024-04-25T10:26:12.944055+0000","last_peered":"2024-04-25T10:26:12.944055+0000","last_clean":"2024-04-25T10:26:12.944055+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:26:12.944055+0000","last_undegraded":"2024-04-25T10:26:12.944055+0000","last_fullsized":"2024-04-25T10:26:12.944055+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.970691+0000","last_change":"2024-04-25T10:26:12.968805+0000","last_active":"2024-04-25T10:26:12.970691+0000","last_peered":"2024-04-25T10:26:12.970691+0000","last_clean":"2024-04-25T10:26:12.970691+0000","last_became_active":"2024-04-25T10:26:12.968534+0000","last_became_peered":"2024-04-25T10:26:12.968534+0000","last_unstale":"2024-04-25T10:26:12.970691+0000","last_undegraded":"2024-04-25T10:26:12.970691+0000","last_fullsized":"2024-04-25T10:26:12.970691+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:27.777514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":347,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-25T10:26:12.958158+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:26:12.958158+0000","last_peered":"2024-04-25T10:26:12.958158+0000","last_clean":"2024-04-25T10:26:12.958158+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:26:12.958158+0000","last_undegraded":"2024-04-25T10:26:12.958158+0000","last_fullsized":"2024-04-25T10:26:12.958158+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50616,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321224,"statfs":{"total":96636764160,"available":96584933376,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509545,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50624,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321216,"statfs":{"total":96636764160,"available":96584925184,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2015232,"data_stored":1204785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2015232,"data_stored":1204785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28416,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343424,"statfs":{"total":96636764160,"available":96607666176,"internally_reserved":0,"allocated":1421312,"data_stored":614417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542247,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58924,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312916,"statfs":{"total":96636764160,"available":96576425984,"internally_reserved":0,"allocated":1409024,"data_stored":608142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:18.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960603+0000', 'last_change': '2024-04-25T10:26:12.960603+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960603+0000', 'last_undegraded': '2024-04-25T10:26:12.960603+0000', 'last_fullsized': '2024-04-25T10:26:12.960603+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958621+0000', 'last_change': '2024-04-25T10:26:12.958621+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958621+0000', 'last_undegraded': '2024-04-25T10:26:12.958621+0000', 'last_fullsized': '2024-04-25T10:26:12.958621+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.957509+0000', 'last_change': '2024-04-25T10:26:12.957509+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.957509+0000', 'last_undegraded': '2024-04-25T10:26:12.957509+0000', 'last_fullsized': '2024-04-25T10:26:12.957509+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2024-04-25T10:26:18.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960968+0000', 'last_change': '2024-04-25T10:26:12.960968+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960968+0000', 'last_undegraded': '2024-04-25T10:26:12.960968+0000', 'last_fullsized': '2024-04-25T10:26:12.960968+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960557+0000', 'last_change': '2024-04-25T10:26:12.960557+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960557+0000', 'last_undegraded': '2024-04-25T10:26:12.960557+0000', 'last_fullsized': '2024-04-25T10:26:12.960557+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.957902+0000', 'last_change': '2024-04-25T10:26:12.957902+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.957902+0000', 'last_undegraded': '2024-04-25T10:26:12.957902+0000', 'last_fullsized': '2024-04-25T10:26:12.957902+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955820+0000', 'last_change': '2024-04-25T10:26:12.955820+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955820+0000', 'last_undegraded': '2024-04-25T10:26:12.955820+0000', 'last_fullsized': '2024-04-25T10:26:12.955820+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2024-04-25T10:26:18.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960167+0000', 'last_change': '2024-04-25T10:26:12.960167+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960167+0000', 'last_undegraded': '2024-04-25T10:26:12.960167+0000', 'last_fullsized': '2024-04-25T10:26:12.960167+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963662+0000', 'last_change': '2024-04-25T10:26:12.963662+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963662+0000', 'last_undegraded': '2024-04-25T10:26:12.963662+0000', 'last_fullsized': '2024-04-25T10:26:12.963662+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958648+0000', 'last_change': '2024-04-25T10:26:12.958648+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958648+0000', 'last_undegraded': '2024-04-25T10:26:12.958648+0000', 'last_fullsized': '2024-04-25T10:26:12.958648+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960992+0000', 'last_change': '2024-04-25T10:26:12.960992+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960992+0000', 'last_undegraded': '2024-04-25T10:26:12.960992+0000', 'last_fullsized': '2024-04-25T10:26:12.960992+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.954373+0000', 'last_change': '2024-04-25T10:26:12.954373+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.954373+0000', 'last_undegraded': '2024-04-25T10:26:12.954373+0000', 'last_fullsized': '2024-04-25T10:26:12.954373+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2024-04-25T10:26:18.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.962360+0000', 'last_change': '2024-04-25T10:26:12.962360+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.962360+0000', 'last_undegraded': '2024-04-25T10:26:12.962360+0000', 'last_fullsized': '2024-04-25T10:26:12.962360+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960133+0000', 'last_change': '2024-04-25T10:26:12.960133+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960133+0000', 'last_undegraded': '2024-04-25T10:26:12.960133+0000', 'last_fullsized': '2024-04-25T10:26:12.960133+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963701+0000', 'last_change': '2024-04-25T10:26:12.963701+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963701+0000', 'last_undegraded': '2024-04-25T10:26:12.963701+0000', 'last_fullsized': '2024-04-25T10:26:12.963701+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955316+0000', 'last_change': '2024-04-25T10:26:12.955316+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955316+0000', 'last_undegraded': '2024-04-25T10:26:12.955316+0000', 'last_fullsized': '2024-04-25T10:26:12.955316+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.956443+0000', 'last_change': '2024-04-25T10:26:12.956443+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.956443+0000', 'last_undegraded': '2024-04-25T10:26:12.956443+0000', 'last_fullsized': '2024-04-25T10:26:12.956443+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2024-04-25T10:26:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955246+0000', 'last_change': '2024-04-25T10:26:12.955246+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955246+0000', 'last_undegraded': '2024-04-25T10:26:12.955246+0000', 'last_fullsized': '2024-04-25T10:26:12.955246+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955483+0000', 'last_change': '2024-04-25T10:26:12.955483+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955483+0000', 'last_undegraded': '2024-04-25T10:26:12.955483+0000', 'last_fullsized': '2024-04-25T10:26:12.955483+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.956466+0000', 'last_change': '2024-04-25T10:26:12.956466+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.956466+0000', 'last_undegraded': '2024-04-25T10:26:12.956466+0000', 'last_fullsized': '2024-04-25T10:26:12.956466+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960106+0000', 'last_change': '2024-04-25T10:26:12.960106+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960106+0000', 'last_undegraded': '2024-04-25T10:26:12.960106+0000', 'last_fullsized': '2024-04-25T10:26:12.960106+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.961108+0000', 'last_change': '2024-04-25T10:26:12.961108+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.961108+0000', 'last_undegraded': '2024-04-25T10:26:12.961108+0000', 'last_fullsized': '2024-04-25T10:26:12.961108+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2024-04-25T10:26:18.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955944+0000', 'last_change': '2024-04-25T10:26:12.955944+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955944+0000', 'last_undegraded': '2024-04-25T10:26:12.955944+0000', 'last_fullsized': '2024-04-25T10:26:12.955944+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.961486+0000', 'last_change': '2024-04-25T10:26:12.961486+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.961486+0000', 'last_undegraded': '2024-04-25T10:26:12.961486+0000', 'last_fullsized': '2024-04-25T10:26:12.961486+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2024-04-25T10:26:18.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.959667+0000', 'last_change': '2024-04-25T10:26:12.959667+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.959667+0000', 'last_undegraded': '2024-04-25T10:26:12.959667+0000', 'last_fullsized': '2024-04-25T10:26:12.959667+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963326+0000', 'last_change': '2024-04-25T10:26:12.963326+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963326+0000', 'last_undegraded': '2024-04-25T10:26:12.963326+0000', 'last_fullsized': '2024-04-25T10:26:12.963326+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955697+0000', 'last_change': '2024-04-25T10:26:12.955697+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955697+0000', 'last_undegraded': '2024-04-25T10:26:12.955697+0000', 'last_fullsized': '2024-04-25T10:26:12.955697+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958589+0000', 'last_change': '2024-04-25T10:26:12.958589+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958589+0000', 'last_undegraded': '2024-04-25T10:26:12.958589+0000', 'last_fullsized': '2024-04-25T10:26:12.958589+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963268+0000', 'last_change': '2024-04-25T10:26:12.963268+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963268+0000', 'last_undegraded': '2024-04-25T10:26:12.963268+0000', 'last_fullsized': '2024-04-25T10:26:12.963268+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.959445+0000', 'last_change': '2024-04-25T10:26:12.959445+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.959445+0000', 'last_undegraded': '2024-04-25T10:26:12.959445+0000', 'last_fullsized': '2024-04-25T10:26:12.959445+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.962804+0000', 'last_change': '2024-04-25T10:26:12.962804+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.962804+0000', 'last_undegraded': '2024-04-25T10:26:12.962804+0000', 'last_fullsized': '2024-04-25T10:26:12.962804+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.959310+0000', 'last_change': '2024-04-25T10:26:12.959310+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.959310+0000', 'last_undegraded': '2024-04-25T10:26:12.959310+0000', 'last_fullsized': '2024-04-25T10:26:12.959310+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.950769+0000', 'last_change': '2024-04-25T10:26:11.950769+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.950769+0000', 'last_undegraded': '2024-04-25T10:26:11.950769+0000', 'last_fullsized': '2024-04-25T10:26:11.950769+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.957091+0000', 'last_change': '2024-04-25T10:26:12.957091+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.957091+0000', 'last_undegraded': '2024-04-25T10:26:12.957091+0000', 'last_fullsized': '2024-04-25T10:26:12.957091+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-04-25T10:26:18.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.951276+0000', 'last_change': '2024-04-25T10:26:11.951276+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.951276+0000', 'last_undegraded': '2024-04-25T10:26:11.951276+0000', 'last_fullsized': '2024-04-25T10:26:11.951276+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.956910+0000', 'last_change': '2024-04-25T10:26:12.956910+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.956910+0000', 'last_undegraded': '2024-04-25T10:26:12.956910+0000', 'last_fullsized': '2024-04-25T10:26:12.956910+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.961042+0000', 'last_change': '2024-04-25T10:26:12.961042+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.961042+0000', 'last_undegraded': '2024-04-25T10:26:12.961042+0000', 'last_fullsized': '2024-04-25T10:26:12.961042+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.957397+0000', 'last_change': '2024-04-25T10:26:12.957397+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.957397+0000', 'last_undegraded': '2024-04-25T10:26:12.957397+0000', 'last_fullsized': '2024-04-25T10:26:12.957397+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2024-04-25T10:26:18.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.953984+0000', 'last_change': '2024-04-25T10:26:12.953984+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.953984+0000', 'last_undegraded': '2024-04-25T10:26:12.953984+0000', 'last_fullsized': '2024-04-25T10:26:12.953984+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.950616+0000', 'last_change': '2024-04-25T10:26:11.950616+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.950616+0000', 'last_undegraded': '2024-04-25T10:26:11.950616+0000', 'last_fullsized': '2024-04-25T10:26:11.950616+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.953982+0000', 'last_change': '2024-04-25T10:26:12.953982+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.953982+0000', 'last_undegraded': '2024-04-25T10:26:12.953982+0000', 'last_fullsized': '2024-04-25T10:26:12.953982+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958876+0000', 'last_change': '2024-04-25T10:26:12.958876+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958876+0000', 'last_undegraded': '2024-04-25T10:26:12.958876+0000', 'last_fullsized': '2024-04-25T10:26:12.958876+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955903+0000', 'last_change': '2024-04-25T10:26:12.955903+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955903+0000', 'last_undegraded': '2024-04-25T10:26:12.955903+0000', 'last_fullsized': '2024-04-25T10:26:12.955903+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.950904+0000', 'last_change': '2024-04-25T10:26:11.950904+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.950904+0000', 'last_undegraded': '2024-04-25T10:26:11.950904+0000', 'last_fullsized': '2024-04-25T10:26:11.950904+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2024-04-25T10:26:18.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960737+0000', 'last_change': '2024-04-25T10:26:12.960737+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960737+0000', 'last_undegraded': '2024-04-25T10:26:12.960737+0000', 'last_fullsized': '2024-04-25T10:26:12.960737+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958521+0000', 'last_change': '2024-04-25T10:26:12.958521+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958521+0000', 'last_undegraded': '2024-04-25T10:26:12.958521+0000', 'last_fullsized': '2024-04-25T10:26:12.958521+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958201+0000', 'last_change': '2024-04-25T10:26:12.958201+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958201+0000', 'last_undegraded': '2024-04-25T10:26:12.958201+0000', 'last_fullsized': '2024-04-25T10:26:12.958201+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955301+0000', 'last_change': '2024-04-25T10:26:12.955301+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955301+0000', 'last_undegraded': '2024-04-25T10:26:12.955301+0000', 'last_fullsized': '2024-04-25T10:26:12.955301+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:26:18.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963858+0000', 'last_change': '2024-04-25T10:26:12.963858+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963858+0000', 'last_undegraded': '2024-04-25T10:26:12.963858+0000', 'last_fullsized': '2024-04-25T10:26:12.963858+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.955308+0000', 'last_change': '2024-04-25T10:26:12.955308+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.955308+0000', 'last_undegraded': '2024-04-25T10:26:12.955308+0000', 'last_fullsized': '2024-04-25T10:26:12.955308+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.958165+0000', 'last_change': '2024-04-25T10:26:12.958165+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.958165+0000', 'last_undegraded': '2024-04-25T10:26:12.958165+0000', 'last_fullsized': '2024-04-25T10:26:12.958165+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963797+0000', 'last_change': '2024-04-25T10:26:12.963797+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963797+0000', 'last_undegraded': '2024-04-25T10:26:12.963797+0000', 'last_fullsized': '2024-04-25T10:26:12.963797+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.949756+0000', 'last_change': '2024-04-25T10:26:11.949756+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.949756+0000', 'last_undegraded': '2024-04-25T10:26:11.949756+0000', 'last_fullsized': '2024-04-25T10:26:11.949756+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2024-04-25T10:26:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963044+0000', 'last_change': '2024-04-25T10:26:12.963044+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963044+0000', 'last_undegraded': '2024-04-25T10:26:12.963044+0000', 'last_fullsized': '2024-04-25T10:26:12.963044+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.949650+0000', 'last_change': '2024-04-25T10:26:11.949650+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.949650+0000', 'last_undegraded': '2024-04-25T10:26:11.949650+0000', 'last_fullsized': '2024-04-25T10:26:11.949650+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.962374+0000', 'last_change': '2024-04-25T10:26:12.962374+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.962374+0000', 'last_undegraded': '2024-04-25T10:26:12.962374+0000', 'last_fullsized': '2024-04-25T10:26:12.962374+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.949647+0000', 'last_change': '2024-04-25T10:26:11.949647+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.949647+0000', 'last_undegraded': '2024-04-25T10:26:11.949647+0000', 'last_fullsized': '2024-04-25T10:26:11.949647+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.962208+0000', 'last_change': '2024-04-25T10:26:12.962208+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.962208+0000', 'last_undegraded': '2024-04-25T10:26:12.962208+0000', 'last_fullsized': '2024-04-25T10:26:12.962208+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.960977+0000', 'last_change': '2024-04-25T10:26:12.960977+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.960977+0000', 'last_undegraded': '2024-04-25T10:26:12.960977+0000', 'last_fullsized': '2024-04-25T10:26:12.960977+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-04-25T10:26:18.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.949642+0000', 'last_change': '2024-04-25T10:26:11.949642+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.949642+0000', 'last_undegraded': '2024-04-25T10:26:11.949642+0000', 'last_fullsized': '2024-04-25T10:26:11.949642+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.963427+0000', 'last_change': '2024-04-25T10:26:12.963427+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.963427+0000', 'last_undegraded': '2024-04-25T10:26:12.963427+0000', 'last_fullsized': '2024-04-25T10:26:12.963427+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 114, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:11.950177+0000', 'last_change': '2024-04-25T10:26:11.950177+0000', 'last_active': '2024-04-25T10:26:11.937740+0000', 'last_peered': '2024-04-25T10:26:11.937740+0000', 'last_clean': '2024-04-25T10:26:11.937740+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:11.950177+0000', 'last_undegraded': '2024-04-25T10:26:11.950177+0000', 'last_fullsized': '2024-04-25T10:26:11.950177+0000', 'mapping_epoch': 114, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 114, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:11.937740+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.954750+0000', 'last_change': '2024-04-25T10:26:12.954750+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.954750+0000', 'last_undegraded': '2024-04-25T10:26:12.954750+0000', 'last_fullsized': '2024-04-25T10:26:12.954750+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.961968+0000', 'last_change': '2024-04-25T10:26:12.961968+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.961968+0000', 'last_undegraded': '2024-04-25T10:26:12.961968+0000', 'last_fullsized': '2024-04-25T10:26:12.961968+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.961084+0000', 'last_change': '2024-04-25T10:26:12.961084+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.961084+0000', 'last_undegraded': '2024-04-25T10:26:12.961084+0000', 'last_fullsized': '2024-04-25T10:26:12.961084+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:26:18.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2024-04-25T10:26:18.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 115, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:26:12.957844+0000', 'last_change': '2024-04-25T10:26:12.957844+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.957844+0000', 'last_undegraded': '2024-04-25T10:26:12.957844+0000', 'last_fullsized': '2024-04-25T10:26:12.957844+0000', 'mapping_epoch': 115, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 115, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:26:18.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2024-04-25T10:26:18.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:26:12.944055+0000', 'last_change': '2024-04-25T10:26:12.944055+0000', 'last_active': '2024-04-25T10:26:12.944055+0000', 'last_peered': '2024-04-25T10:26:12.944055+0000', 'last_clean': '2024-04-25T10:26:12.944055+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:26:12.944055+0000', 'last_undegraded': '2024-04-25T10:26:12.944055+0000', 'last_fullsized': '2024-04-25T10:26:12.944055+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'last_clean_scrub_stamp': '2024-04-25T10:26:12.944055+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:26:18.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:19.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 Processor -- start 2024-04-25T10:26:19.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 -- start start 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c9c15cbc0 0x7f6c9c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c0d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6c9c0669a0 0x7f6c9c066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c067350 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c071260 con 0x7f6c9c0669a0 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca2400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0d1060 con 0x7f6c9c0d04f0 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c9c15cbc0 0x7f6c9c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6c9c15cbc0 0x7f6c9c071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37688/0 (socket says 172.21.15.45:37688) 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 -- 172.21.15.45:0/3079180242 learned_addr learned my addr 172.21.15.45:0/3079180242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6c93e00640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c0d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1a00640 1 --2- 172.21.15.45:0/3079180242 >> v2:172.21.15.117:3300/0 conn(0x7f6c9c0669a0 0x7f6c9c066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 -- 172.21.15.45:0/3079180242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 msgr2=0x7f6c9c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c0d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 -- 172.21.15.45:0/3079180242 >> v2:172.21.15.117:3300/0 conn(0x7f6c9c0669a0 msgr2=0x7f6c9c066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:19.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 --2- 172.21.15.45:0/3079180242 >> v2:172.21.15.117:3300/0 conn(0x7f6c9c0669a0 0x7f6c9c066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1000640 1 -- 172.21.15.45:0/3079180242 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0d1340 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6c93e00640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c0d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:19.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.079+0000 7f6ca1a00640 1 --2- 172.21.15.45:0/3079180242 >> v2:172.21.15.117:3300/0 conn(0x7f6c9c0669a0 0x7f6c9c066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:19.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca1000640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c071cb0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c152ff0 tx=0x7f6c8c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6cb7bbcd73a763 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6c9be00640 1 -- 172.21.15.45:0/3079180242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0b1070 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6c9be00640 1 -- 172.21.15.45:0/3079180242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0ac630 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6c9be00640 1 -- 172.21.15.45:0/3079180242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0a8050 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 -- 172.21.15.45:0/3079180242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 msgr2=0x7f6c9c071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c071cb0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c152ff0 tx=0x7f6c8c01c520 comp rx=0 tx=0).stop 2024-04-25T10:26:19.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 -- 172.21.15.45:0/3079180242 shutdown_connections 2024-04-25T10:26:19.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 --2- 172.21.15.45:0/3079180242 >> v2:172.21.15.117:3300/0 conn(0x7f6c9c0669a0 0x7f6c9c066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c0d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 --2- 172.21.15.45:0/3079180242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c071cb0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 -- 172.21.15.45:0/3079180242 >> 172.21.15.45:0/3079180242 conn(0x7f6c9c0b37d0 msgr2=0x7f6c9c0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 -- 172.21.15.45:0/3079180242 shutdown_connections 2024-04-25T10:26:19.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.083+0000 7f6ca2400640 1 -- 172.21.15.45:0/3079180242 wait complete. 2024-04-25T10:26:19.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 Processor -- start 2024-04-25T10:26:19.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- start start 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c9c0669a0 0x7f6c9c0ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c14a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c06db40 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0721f0 con 0x7f6c9c0669a0 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c06dcc0 con 0x7f6c9c0d04f0 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37698/0 (socket says 172.21.15.45:37698) 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 -- 172.21.15.45:0/1162501283 learned_addr learned my addr 172.21.15.45:0/1162501283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 -- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 msgr2=0x7f6c9c14a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6c93e00640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c14a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1000640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c9c0669a0 0x7f6c9c0ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c14a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 -- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c9c0669a0 msgr2=0x7f6c9c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c9c0669a0 0x7f6c9c0ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 -- 172.21.15.45:0/1162501283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0cecd0 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1000640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c9c0669a0 0x7f6c9c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6c93e00640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c14a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca1a00640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c159b10 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400cef0 tx=0x7f6c9400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9401a070 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9400d840 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c151af0 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400dc30 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c1504b0 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.087+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c075120 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.091+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c9401d070 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.091+0000 7f6c99600640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6c78066f60 0x7f6c78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.091+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c940857d0 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.091+0000 7f6ca1000640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6c78066f60 0x7f6c78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.091+0000 7f6ca1000640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6c78066f60 0x7f6c78069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0ae000 tx=0x7f6c8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.095+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c94056870 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.315+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6c9c0cce90 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6c99600640 1 -- 172.21.15.45:0/1162501283 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1563 (secure 0 0 0) 0x7f6c94056170 con 0x7f6c9c15cbc0 2024-04-25T10:26:19.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:19.321 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":523,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":117,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":427454464,"bytes_avail":772666658816,"bytes_total":773094113280,"write_bytes_sec":2375,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:25:52.983902+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6c78066f60 msgr2=0x7f6c78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6c78066f60 0x7f6c78069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0ae000 tx=0x7f6c8c008040 comp rx=0 tx=0).stop 2024-04-25T10:26:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 msgr2=0x7f6c9c159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c159b10 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400cef0 tx=0x7f6c9400a710 comp rx=0 tx=0).stop 2024-04-25T10:26:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 shutdown_connections 2024-04-25T10:26:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6c78066f60 0x7f6c78069420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c9c15cbc0 0x7f6c9c159b10 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c9c0d04f0 0x7f6c9c14a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 --2- 172.21.15.45:0/1162501283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c9c0669a0 0x7f6c9c0ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 >> 172.21.15.45:0/1162501283 conn(0x7f6c9c0b37d0 msgr2=0x7f6c9c15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 shutdown_connections 2024-04-25T10:26:19.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.319+0000 7f6ca2400640 1 -- 172.21.15.45:0/1162501283 wait complete. 2024-04-25T10:26:19.341 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:26:19.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 Processor -- start 2024-04-25T10:26:19.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 -- start start 2024-04-25T10:26:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c0cda10 0x7f437c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f437c0cf930 0x7f437c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437c0b2770 con 0x7f437c15bc40 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437c06c670 con 0x7f437c0cf930 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.479+0000 7f4381c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437c06c950 con 0x7f437c0cda10 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437aa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f437c15bc40 0x7f437c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60844/0 (socket says 172.21.15.45:60844) 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437aa00640 1 -- 172.21.15.45:0/335069203 learned_addr learned my addr 172.21.15.45:0/335069203 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f4373e00640 1 --2- 172.21.15.45:0/335069203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c0cda10 0x7f437c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437b400640 1 --2- 172.21.15.45:0/335069203 >> v2:172.21.15.117:3300/0 conn(0x7f437c0cf930 0x7f437c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437a000640 1 -- 172.21.15.45:0/335069203 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470694486 0 0) 0x7f437c0b2770 con 0x7f437c15bc40 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437a000640 1 -- 172.21.15.45:0/335069203 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4360003530 con 0x7f437c15bc40 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f4373e00640 1 -- 172.21.15.45:0/335069203 >> v2:172.21.15.117:3300/0 conn(0x7f437c0cf930 msgr2=0x7f437c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f4373e00640 1 --2- 172.21.15.45:0/335069203 >> v2:172.21.15.117:3300/0 conn(0x7f437c0cf930 0x7f437c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f4373e00640 1 -- 172.21.15.45:0/335069203 >> v1:172.21.15.45:6789/0 conn(0x7f437c15bc40 legacy=0x7f437c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f4373e00640 1 -- 172.21.15.45:0/335069203 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c06cc30 con 0x7f437c0cda10 2024-04-25T10:26:19.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437b400640 1 --2- 172.21.15.45:0/335069203 >> v2:172.21.15.117:3300/0 conn(0x7f437c0cf930 0x7f437c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f4373e00640 1 --2- 172.21.15.45:0/335069203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c0cda10 0x7f437c06c090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43640097c0 tx=0x7f436401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373ae11b9836e066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.483+0000 7f437a000640 1 -- 172.21.15.45:0/335069203 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640b0070 con 0x7f437c0cda10 2024-04-25T10:26:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f437a000640 1 -- 172.21.15.45:0/335069203 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640ab600 con 0x7f437c0cda10 2024-04-25T10:26:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f437a000640 1 -- 172.21.15.45:0/335069203 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640ab9d0 con 0x7f437c0cda10 2024-04-25T10:26:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- 172.21.15.45:0/335069203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c0cda10 msgr2=0x7f437c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 --2- 172.21.15.45:0/335069203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c0cda10 0x7f437c06c090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43640097c0 tx=0x7f436401c410 comp rx=0 tx=0).stop 2024-04-25T10:26:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- 172.21.15.45:0/335069203 shutdown_connections 2024-04-25T10:26:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 --2- 172.21.15.45:0/335069203 >> v2:172.21.15.117:3300/0 conn(0x7f437c0cf930 0x7f437c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 --2- 172.21.15.45:0/335069203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c0cda10 0x7f437c06c090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- 172.21.15.45:0/335069203 >> 172.21.15.45:0/335069203 conn(0x7f437c066b70 msgr2=0x7f437c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- 172.21.15.45:0/335069203 shutdown_connections 2024-04-25T10:26:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- 172.21.15.45:0/335069203 wait complete. 2024-04-25T10:26:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 Processor -- start 2024-04-25T10:26:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- start start 2024-04-25T10:26:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c0cda10 0x7f437c14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 0x7f437c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c15bc40 0x7f437c071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0b3850 con 0x7f437c0cf930 2024-04-25T10:26:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0b36d0 con 0x7f437c0cda10 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4381c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0b39d0 con 0x7f437c15bc40 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 0x7f437c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 0x7f437c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37714/0 (socket says 172.21.15.45:37714) 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 -- 172.21.15.45:0/1195753387 learned_addr learned my addr 172.21.15.45:0/1195753387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f437b400640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c15bc40 0x7f437c071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 -- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c15bc40 msgr2=0x7f437c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c15bc40 0x7f437c071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 -- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c0cda10 msgr2=0x7f437c14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c0cda10 0x7f437c14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.487+0000 7f4373e00640 1 -- 172.21.15.45:0/1195753387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c070bf0 con 0x7f437c0cf930 2024-04-25T10:26:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4373e00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 0x7f437c070310 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f436401a040 tx=0x7f43640191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640b8020 con 0x7f437c0cf930 2024-04-25T10:26:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640b4610 con 0x7f437c0cf930 2024-04-25T10:26:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437c070ed0 con 0x7f437c0cf930 2024-04-25T10:26:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640b0040 con 0x7f437c0cf930 2024-04-25T10:26:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437c071490 con 0x7f437c0cf930 2024-04-25T10:26:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437c04ef30 con 0x7f437c0cf930 2024-04-25T10:26:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43640bb020 con 0x7f437c0cf930 2024-04-25T10:26:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4372a00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4358066f60 0x7f4358069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.491+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f436411ec50 con 0x7f437c0cf930 2024-04-25T10:26:19.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.495+0000 7f437aa00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4358066f60 0x7f4358069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.495+0000 7f437aa00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4358066f60 0x7f4358069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f436c007fd0 tx=0x7f436c007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.495+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43640c2030 con 0x7f437c0cf930 2024-04-25T10:26:19.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.655+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f437c06d960 con 0x7f4358066f60 2024-04-25T10:26:19.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.663+0000 7f4372a00640 1 -- 172.21.15.45:0/1195753387 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252479 (secure 0 0 0) 0x7f437c06d960 con 0x7f4358066f60 2024-04-25T10:26:19.667 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:19.668 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4358066f60 msgr2=0x7f4358069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4358066f60 0x7f4358069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f436c007fd0 tx=0x7f436c007900 comp rx=0 tx=0).stop 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 msgr2=0x7f437c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 0x7f437c070310 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f436401a040 tx=0x7f43640191b0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 shutdown_connections 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4358066f60 0x7f4358069420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f437c15bc40 0x7f437c071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437c0cf930 0x7f437c070310 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 --2- 172.21.15.45:0/1195753387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f437c0cda10 0x7f437c14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 >> 172.21.15.45:0/1195753387 conn(0x7f437c066b70 msgr2=0x7f437c1580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 shutdown_connections 2024-04-25T10:26:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:19.667+0000 7f4381c00640 1 -- 172.21.15.45:0/1195753387 wait complete. 2024-04-25T10:26:19.687 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-04-25T10:26:18.989840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417436,"kb_used_data":12520,"kb_used_omap":12,"kb_used_meta":404787,"kb_avail":754557284,"statfs":{"total":773094113280,"available":772666658816,"internally_reserved":0,"allocated":12820480,"data_stored":6205492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":414502448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.035002"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649975+0000","last_change":"2024-04-25T10:26:14.005197+0000","last_active":"2024-04-25T10:26:15.649975+0000","last_peered":"2024-04-25T10:26:15.649975+0000","last_clean":"2024-04-25T10:26:15.649975+0000","last_became_active":"2024-04-25T10:26:14.004832+0000","last_became_peered":"2024-04-25T10:26:14.004832+0000","last_unstale":"2024-04-25T10:26:15.649975+0000","last_undegraded":"2024-04-25T10:26:15.649975+0000","last_fullsized":"2024-04-25T10:26:15.649975+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:53.953051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648603+0000","last_change":"2024-04-25T10:26:13.986851+0000","last_active":"2024-04-25T10:26:15.648603+0000","last_peered":"2024-04-25T10:26:15.648603+0000","last_clean":"2024-04-25T10:26:15.648603+0000","last_became_active":"2024-04-25T10:26:13.986540+0000","last_became_peered":"2024-04-25T10:26:13.986540+0000","last_unstale":"2024-04-25T10:26:15.648603+0000","last_undegraded":"2024-04-25T10:26:15.648603+0000","last_fullsized":"2024-04-25T10:26:15.648603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:52.176540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648152+0000","last_change":"2024-04-25T10:26:14.007315+0000","last_active":"2024-04-25T10:26:15.648152+0000","last_peered":"2024-04-25T10:26:15.648152+0000","last_clean":"2024-04-25T10:26:15.648152+0000","last_became_active":"2024-04-25T10:26:14.007043+0000","last_became_peered":"2024-04-25T10:26:14.007043+0000","last_unstale":"2024-04-25T10:26:15.648152+0000","last_undegraded":"2024-04-25T10:26:15.648152+0000","last_fullsized":"2024-04-25T10:26:15.648152+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:54.738625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985211+0000","last_change":"2024-04-25T10:26:13.987496+0000","last_active":"2024-04-25T10:26:14.985211+0000","last_peered":"2024-04-25T10:26:14.985211+0000","last_clean":"2024-04-25T10:26:14.985211+0000","last_became_active":"2024-04-25T10:26:13.987243+0000","last_became_peered":"2024-04-25T10:26:13.987243+0000","last_unstale":"2024-04-25T10:26:14.985211+0000","last_undegraded":"2024-04-25T10:26:14.985211+0000","last_fullsized":"2024-04-25T10:26:14.985211+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:23.888743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650455+0000","last_change":"2024-04-25T10:26:14.002763+0000","last_active":"2024-04-25T10:26:15.650455+0000","last_peered":"2024-04-25T10:26:15.650455+0000","last_clean":"2024-04-25T10:26:15.650455+0000","last_became_active":"2024-04-25T10:26:14.000386+0000","last_became_peered":"2024-04-25T10:26:14.000386+0000","last_unstale":"2024-04-25T10:26:15.650455+0000","last_undegraded":"2024-04-25T10:26:15.650455+0000","last_fullsized":"2024-04-25T10:26:15.650455+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:39.629222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985852+0000","last_change":"2024-04-25T10:26:13.981550+0000","last_active":"2024-04-25T10:26:14.985852+0000","last_peered":"2024-04-25T10:26:14.985852+0000","last_clean":"2024-04-25T10:26:14.985852+0000","last_became_active":"2024-04-25T10:26:13.980971+0000","last_became_peered":"2024-04-25T10:26:13.980971+0000","last_unstale":"2024-04-25T10:26:14.985852+0000","last_undegraded":"2024-04-25T10:26:14.985852+0000","last_fullsized":"2024-04-25T10:26:14.985852+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:00.289419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982224+0000","last_change":"2024-04-25T10:26:13.991938+0000","last_active":"2024-04-25T10:26:14.982224+0000","last_peered":"2024-04-25T10:26:14.982224+0000","last_clean":"2024-04-25T10:26:14.982224+0000","last_became_active":"2024-04-25T10:26:13.991261+0000","last_became_peered":"2024-04-25T10:26:13.991261+0000","last_unstale":"2024-04-25T10:26:14.982224+0000","last_undegraded":"2024-04-25T10:26:14.982224+0000","last_fullsized":"2024-04-25T10:26:14.982224+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:18.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982276+0000","last_change":"2024-04-25T10:26:14.001718+0000","last_active":"2024-04-25T10:26:14.982276+0000","last_peered":"2024-04-25T10:26:14.982276+0000","last_clean":"2024-04-25T10:26:14.982276+0000","last_became_active":"2024-04-25T10:26:13.999412+0000","last_became_peered":"2024-04-25T10:26:13.999412+0000","last_unstale":"2024-04-25T10:26:14.982276+0000","last_undegraded":"2024-04-25T10:26:14.982276+0000","last_fullsized":"2024-04-25T10:26:14.982276+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:38.264923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649990+0000","last_change":"2024-04-25T10:26:14.006956+0000","last_active":"2024-04-25T10:26:15.649990+0000","last_peered":"2024-04-25T10:26:15.649990+0000","last_clean":"2024-04-25T10:26:15.649990+0000","last_became_active":"2024-04-25T10:26:14.006640+0000","last_became_peered":"2024-04-25T10:26:14.006640+0000","last_unstale":"2024-04-25T10:26:15.649990+0000","last_undegraded":"2024-04-25T10:26:15.649990+0000","last_fullsized":"2024-04-25T10:26:15.649990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:57.589244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650998+0000","last_change":"2024-04-25T10:26:13.999830+0000","last_active":"2024-04-25T10:26:15.650998+0000","last_peered":"2024-04-25T10:26:15.650998+0000","last_clean":"2024-04-25T10:26:15.650998+0000","last_became_active":"2024-04-25T10:26:13.999525+0000","last_became_peered":"2024-04-25T10:26:13.999525+0000","last_unstale":"2024-04-25T10:26:15.650998+0000","last_undegraded":"2024-04-25T10:26:15.650998+0000","last_fullsized":"2024-04-25T10:26:15.650998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:41.212681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983017+0000","last_change":"2024-04-25T10:26:13.995853+0000","last_active":"2024-04-25T10:26:14.983017+0000","last_peered":"2024-04-25T10:26:14.983017+0000","last_clean":"2024-04-25T10:26:14.983017+0000","last_became_active":"2024-04-25T10:26:13.995365+0000","last_became_peered":"2024-04-25T10:26:13.995365+0000","last_unstale":"2024-04-25T10:26:14.983017+0000","last_undegraded":"2024-04-25T10:26:14.983017+0000","last_fullsized":"2024-04-25T10:26:14.983017+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.730661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650613+0000","last_change":"2024-04-25T10:26:13.999775+0000","last_active":"2024-04-25T10:26:15.650613+0000","last_peered":"2024-04-25T10:26:15.650613+0000","last_clean":"2024-04-25T10:26:15.650613+0000","last_became_active":"2024-04-25T10:26:13.999471+0000","last_became_peered":"2024-04-25T10:26:13.999471+0000","last_unstale":"2024-04-25T10:26:15.650613+0000","last_undegraded":"2024-04-25T10:26:15.650613+0000","last_fullsized":"2024-04-25T10:26:15.650613+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:51.728803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648495+0000","last_change":"2024-04-25T10:26:14.004568+0000","last_active":"2024-04-25T10:26:15.648495+0000","last_peered":"2024-04-25T10:26:15.648495+0000","last_clean":"2024-04-25T10:26:15.648495+0000","last_became_active":"2024-04-25T10:26:14.003306+0000","last_became_peered":"2024-04-25T10:26:14.003306+0000","last_unstale":"2024-04-25T10:26:15.648495+0000","last_undegraded":"2024-04-25T10:26:15.648495+0000","last_fullsized":"2024-04-25T10:26:15.648495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:57.589244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152826+0000","last_change":"2024-04-25T10:26:13.994705+0000","last_active":"2024-04-25T10:26:15.152826+0000","last_peered":"2024-04-25T10:26:15.152826+0000","last_clean":"2024-04-25T10:26:15.152826+0000","last_became_active":"2024-04-25T10:26:13.994412+0000","last_became_peered":"2024-04-25T10:26:13.994412+0000","last_unstale":"2024-04-25T10:26:15.152826+0000","last_undegraded":"2024-04-25T10:26:15.152826+0000","last_fullsized":"2024-04-25T10:26:15.152826+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:42.676705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649545+0000","last_change":"2024-04-25T10:26:14.001323+0000","last_active":"2024-04-25T10:26:15.649545+0000","last_peered":"2024-04-25T10:26:15.649545+0000","last_clean":"2024-04-25T10:26:15.649545+0000","last_became_active":"2024-04-25T10:26:14.000341+0000","last_became_peered":"2024-04-25T10:26:14.000341+0000","last_unstale":"2024-04-25T10:26:15.649545+0000","last_undegraded":"2024-04-25T10:26:15.649545+0000","last_fullsized":"2024-04-25T10:26:15.649545+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:52.176540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649424+0000","last_change":"2024-04-25T10:26:14.004901+0000","last_active":"2024-04-25T10:26:15.649424+0000","last_peered":"2024-04-25T10:26:15.649424+0000","last_clean":"2024-04-25T10:26:15.649424+0000","last_became_active":"2024-04-25T10:26:14.000646+0000","last_became_peered":"2024-04-25T10:26:14.000646+0000","last_unstale":"2024-04-25T10:26:15.649424+0000","last_undegraded":"2024-04-25T10:26:15.649424+0000","last_fullsized":"2024-04-25T10:26:15.649424+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:53.202091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648365+0000","last_change":"2024-04-25T10:26:14.001919+0000","last_active":"2024-04-25T10:26:15.648365+0000","last_peered":"2024-04-25T10:26:15.648365+0000","last_clean":"2024-04-25T10:26:15.648365+0000","last_became_active":"2024-04-25T10:26:14.001643+0000","last_became_peered":"2024-04-25T10:26:14.001643+0000","last_unstale":"2024-04-25T10:26:15.648365+0000","last_undegraded":"2024-04-25T10:26:15.648365+0000","last_fullsized":"2024-04-25T10:26:15.648365+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:46.548053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983099+0000","last_change":"2024-04-25T10:26:13.999078+0000","last_active":"2024-04-25T10:26:14.983099+0000","last_peered":"2024-04-25T10:26:14.983099+0000","last_clean":"2024-04-25T10:26:14.983099+0000","last_became_active":"2024-04-25T10:26:13.998778+0000","last_became_peered":"2024-04-25T10:26:13.998778+0000","last_unstale":"2024-04-25T10:26:14.983099+0000","last_undegraded":"2024-04-25T10:26:14.983099+0000","last_fullsized":"2024-04-25T10:26:14.983099+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:37.186955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648507+0000","last_change":"2024-04-25T10:26:14.006940+0000","last_active":"2024-04-25T10:26:15.648507+0000","last_peered":"2024-04-25T10:26:15.648507+0000","last_clean":"2024-04-25T10:26:15.648507+0000","last_became_active":"2024-04-25T10:26:14.006195+0000","last_became_peered":"2024-04-25T10:26:14.006195+0000","last_unstale":"2024-04-25T10:26:15.648507+0000","last_undegraded":"2024-04-25T10:26:15.648507+0000","last_fullsized":"2024-04-25T10:26:15.648507+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:21.573428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647976+0000","last_change":"2024-04-25T10:26:13.998498+0000","last_active":"2024-04-25T10:26:15.647976+0000","last_peered":"2024-04-25T10:26:15.647976+0000","last_clean":"2024-04-25T10:26:15.647976+0000","last_became_active":"2024-04-25T10:26:13.998242+0000","last_became_peered":"2024-04-25T10:26:13.998242+0000","last_unstale":"2024-04-25T10:26:15.647976+0000","last_undegraded":"2024-04-25T10:26:15.647976+0000","last_fullsized":"2024-04-25T10:26:15.647976+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:58.884962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982816+0000","last_change":"2024-04-25T10:26:13.999686+0000","last_active":"2024-04-25T10:26:14.982816+0000","last_peered":"2024-04-25T10:26:14.982816+0000","last_clean":"2024-04-25T10:26:14.982816+0000","last_became_active":"2024-04-25T10:26:13.999340+0000","last_became_peered":"2024-04-25T10:26:13.999340+0000","last_unstale":"2024-04-25T10:26:14.982816+0000","last_undegraded":"2024-04-25T10:26:14.982816+0000","last_fullsized":"2024-04-25T10:26:14.982816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:42.097549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985973+0000","last_change":"2024-04-25T10:26:13.984408+0000","last_active":"2024-04-25T10:26:14.985973+0000","last_peered":"2024-04-25T10:26:14.985973+0000","last_clean":"2024-04-25T10:26:14.985973+0000","last_became_active":"2024-04-25T10:26:13.984146+0000","last_became_peered":"2024-04-25T10:26:13.984146+0000","last_unstale":"2024-04-25T10:26:14.985973+0000","last_undegraded":"2024-04-25T10:26:14.985973+0000","last_fullsized":"2024-04-25T10:26:14.985973+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:38.406611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152906+0000","last_change":"2024-04-25T10:26:13.998347+0000","last_active":"2024-04-25T10:26:15.152906+0000","last_peered":"2024-04-25T10:26:15.152906+0000","last_clean":"2024-04-25T10:26:15.152906+0000","last_became_active":"2024-04-25T10:26:13.997933+0000","last_became_peered":"2024-04-25T10:26:13.997933+0000","last_unstale":"2024-04-25T10:26:15.152906+0000","last_undegraded":"2024-04-25T10:26:15.152906+0000","last_fullsized":"2024-04-25T10:26:15.152906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:50.526006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984718+0000","last_change":"2024-04-25T10:26:13.983856+0000","last_active":"2024-04-25T10:26:14.984718+0000","last_peered":"2024-04-25T10:26:14.984718+0000","last_clean":"2024-04-25T10:26:14.984718+0000","last_became_active":"2024-04-25T10:26:13.983578+0000","last_became_peered":"2024-04-25T10:26:13.983578+0000","last_unstale":"2024-04-25T10:26:14.984718+0000","last_undegraded":"2024-04-25T10:26:14.984718+0000","last_fullsized":"2024-04-25T10:26:14.984718+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:39.025850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983093+0000","last_change":"2024-04-25T10:26:13.998840+0000","last_active":"2024-04-25T10:26:14.983093+0000","last_peered":"2024-04-25T10:26:14.983093+0000","last_clean":"2024-04-25T10:26:14.983093+0000","last_became_active":"2024-04-25T10:26:13.998134+0000","last_became_peered":"2024-04-25T10:26:13.998134+0000","last_unstale":"2024-04-25T10:26:14.983093+0000","last_undegraded":"2024-04-25T10:26:14.983093+0000","last_fullsized":"2024-04-25T10:26:14.983093+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986701+0000","last_change":"2024-04-25T10:26:13.987446+0000","last_active":"2024-04-25T10:26:14.986701+0000","last_peered":"2024-04-25T10:26:14.986701+0000","last_clean":"2024-04-25T10:26:14.986701+0000","last_became_active":"2024-04-25T10:26:13.986508+0000","last_became_peered":"2024-04-25T10:26:13.986508+0000","last_unstale":"2024-04-25T10:26:14.986701+0000","last_undegraded":"2024-04-25T10:26:14.986701+0000","last_fullsized":"2024-04-25T10:26:14.986701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:55.173312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985498+0000","last_change":"2024-04-25T10:26:13.990128+0000","last_active":"2024-04-25T10:26:14.985498+0000","last_peered":"2024-04-25T10:26:14.985498+0000","last_clean":"2024-04-25T10:26:14.985498+0000","last_became_active":"2024-04-25T10:26:13.989787+0000","last_became_peered":"2024-04-25T10:26:13.989787+0000","last_unstale":"2024-04-25T10:26:14.985498+0000","last_undegraded":"2024-04-25T10:26:14.985498+0000","last_fullsized":"2024-04-25T10:26:14.985498+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:29.129511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986374+0000","last_change":"2024-04-25T10:26:13.981675+0000","last_active":"2024-04-25T10:26:14.986374+0000","last_peered":"2024-04-25T10:26:14.986374+0000","last_clean":"2024-04-25T10:26:14.986374+0000","last_became_active":"2024-04-25T10:26:13.981362+0000","last_became_peered":"2024-04-25T10:26:13.981362+0000","last_unstale":"2024-04-25T10:26:14.986374+0000","last_undegraded":"2024-04-25T10:26:14.986374+0000","last_fullsized":"2024-04-25T10:26:14.986374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:49.074238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153217+0000","last_change":"2024-04-25T10:26:13.990284+0000","last_active":"2024-04-25T10:26:15.153217+0000","last_peered":"2024-04-25T10:26:15.153217+0000","last_clean":"2024-04-25T10:26:15.153217+0000","last_became_active":"2024-04-25T10:26:13.989862+0000","last_became_peered":"2024-04-25T10:26:13.989862+0000","last_unstale":"2024-04-25T10:26:15.153217+0000","last_undegraded":"2024-04-25T10:26:15.153217+0000","last_fullsized":"2024-04-25T10:26:15.153217+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:29.655266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982619+0000","last_change":"2024-04-25T10:26:13.981364+0000","last_active":"2024-04-25T10:26:14.982619+0000","last_peered":"2024-04-25T10:26:14.982619+0000","last_clean":"2024-04-25T10:26:14.982619+0000","last_became_active":"2024-04-25T10:26:13.981057+0000","last_became_peered":"2024-04-25T10:26:13.981057+0000","last_unstale":"2024-04-25T10:26:14.982619+0000","last_undegraded":"2024-04-25T10:26:14.982619+0000","last_fullsized":"2024-04-25T10:26:14.982619+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:53.702981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650126+0000","last_change":"2024-04-25T10:26:13.997324+0000","last_active":"2024-04-25T10:26:15.650126+0000","last_peered":"2024-04-25T10:26:15.650126+0000","last_clean":"2024-04-25T10:26:15.650126+0000","last_became_active":"2024-04-25T10:26:13.994454+0000","last_became_peered":"2024-04-25T10:26:13.994454+0000","last_unstale":"2024-04-25T10:26:15.650126+0000","last_undegraded":"2024-04-25T10:26:15.650126+0000","last_fullsized":"2024-04-25T10:26:15.650126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:21.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649606+0000","last_change":"2024-04-25T10:26:14.005110+0000","last_active":"2024-04-25T10:26:15.649606+0000","last_peered":"2024-04-25T10:26:15.649606+0000","last_clean":"2024-04-25T10:26:15.649606+0000","last_became_active":"2024-04-25T10:26:14.000955+0000","last_became_peered":"2024-04-25T10:26:14.000955+0000","last_unstale":"2024-04-25T10:26:15.649606+0000","last_undegraded":"2024-04-25T10:26:15.649606+0000","last_fullsized":"2024-04-25T10:26:15.649606+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:54.738625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649983+0000","last_change":"2024-04-25T10:26:13.997490+0000","last_active":"2024-04-25T10:26:15.649983+0000","last_peered":"2024-04-25T10:26:15.649983+0000","last_clean":"2024-04-25T10:26:15.649983+0000","last_became_active":"2024-04-25T10:26:13.997153+0000","last_became_peered":"2024-04-25T10:26:13.997153+0000","last_unstale":"2024-04-25T10:26:15.649983+0000","last_undegraded":"2024-04-25T10:26:15.649983+0000","last_fullsized":"2024-04-25T10:26:15.649983+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:58.035839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"116'4","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153175+0000","last_change":"2024-04-25T10:26:12.973421+0000","last_active":"2024-04-25T10:26:15.153175+0000","last_peered":"2024-04-25T10:26:15.153175+0000","last_clean":"2024-04-25T10:26:15.153175+0000","last_became_active":"2024-04-25T10:26:12.973108+0000","last_became_peered":"2024-04-25T10:26:12.973108+0000","last_unstale":"2024-04-25T10:26:15.153175+0000","last_undegraded":"2024-04-25T10:26:15.153175+0000","last_fullsized":"2024-04-25T10:26:15.153175+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:35.304001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153148+0000","last_change":"2024-04-25T10:26:14.005677+0000","last_active":"2024-04-25T10:26:15.153148+0000","last_peered":"2024-04-25T10:26:15.153148+0000","last_clean":"2024-04-25T10:26:15.153148+0000","last_became_active":"2024-04-25T10:26:14.005373+0000","last_became_peered":"2024-04-25T10:26:14.005373+0000","last_unstale":"2024-04-25T10:26:15.153148+0000","last_undegraded":"2024-04-25T10:26:15.153148+0000","last_fullsized":"2024-04-25T10:26:15.153148+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:29.194434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648710+0000","last_change":"2024-04-25T10:26:12.965663+0000","last_active":"2024-04-25T10:26:15.648710+0000","last_peered":"2024-04-25T10:26:15.648710+0000","last_clean":"2024-04-25T10:26:15.648710+0000","last_became_active":"2024-04-25T10:26:12.965358+0000","last_became_peered":"2024-04-25T10:26:12.965358+0000","last_unstale":"2024-04-25T10:26:15.648710+0000","last_undegraded":"2024-04-25T10:26:15.648710+0000","last_fullsized":"2024-04-25T10:26:15.648710+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.527361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986627+0000","last_change":"2024-04-25T10:26:13.990794+0000","last_active":"2024-04-25T10:26:14.986627+0000","last_peered":"2024-04-25T10:26:14.986627+0000","last_clean":"2024-04-25T10:26:14.986627+0000","last_became_active":"2024-04-25T10:26:13.990456+0000","last_became_peered":"2024-04-25T10:26:13.990456+0000","last_unstale":"2024-04-25T10:26:14.986627+0000","last_undegraded":"2024-04-25T10:26:14.986627+0000","last_fullsized":"2024-04-25T10:26:14.986627+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:18.806868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985375+0000","last_change":"2024-04-25T10:26:12.978335+0000","last_active":"2024-04-25T10:26:14.985375+0000","last_peered":"2024-04-25T10:26:14.985375+0000","last_clean":"2024-04-25T10:26:14.985375+0000","last_became_active":"2024-04-25T10:26:12.978084+0000","last_became_peered":"2024-04-25T10:26:12.978084+0000","last_unstale":"2024-04-25T10:26:14.985375+0000","last_undegraded":"2024-04-25T10:26:14.985375+0000","last_fullsized":"2024-04-25T10:26:14.985375+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:06.204252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648153+0000","last_change":"2024-04-25T10:26:14.007186+0000","last_active":"2024-04-25T10:26:15.648153+0000","last_peered":"2024-04-25T10:26:15.648153+0000","last_clean":"2024-04-25T10:26:15.648153+0000","last_became_active":"2024-04-25T10:26:14.006656+0000","last_became_peered":"2024-04-25T10:26:14.006656+0000","last_unstale":"2024-04-25T10:26:15.648153+0000","last_undegraded":"2024-04-25T10:26:15.648153+0000","last_fullsized":"2024-04-25T10:26:15.648153+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:53.202091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985141+0000","last_change":"2024-04-25T10:26:12.976081+0000","last_active":"2024-04-25T10:26:14.985141+0000","last_peered":"2024-04-25T10:26:14.985141+0000","last_clean":"2024-04-25T10:26:14.985141+0000","last_became_active":"2024-04-25T10:26:12.974108+0000","last_became_peered":"2024-04-25T10:26:12.974108+0000","last_unstale":"2024-04-25T10:26:14.985141+0000","last_undegraded":"2024-04-25T10:26:14.985141+0000","last_fullsized":"2024-04-25T10:26:14.985141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:44.197112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984990+0000","last_change":"2024-04-25T10:26:13.990888+0000","last_active":"2024-04-25T10:26:14.984990+0000","last_peered":"2024-04-25T10:26:14.984990+0000","last_clean":"2024-04-25T10:26:14.984990+0000","last_became_active":"2024-04-25T10:26:13.990631+0000","last_became_peered":"2024-04-25T10:26:13.990631+0000","last_unstale":"2024-04-25T10:26:14.984990+0000","last_undegraded":"2024-04-25T10:26:14.984990+0000","last_fullsized":"2024-04-25T10:26:14.984990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:11.417723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647878+0000","last_change":"2024-04-25T10:26:12.969937+0000","last_active":"2024-04-25T10:26:15.647878+0000","last_peered":"2024-04-25T10:26:15.647878+0000","last_clean":"2024-04-25T10:26:15.647878+0000","last_became_active":"2024-04-25T10:26:12.969666+0000","last_became_peered":"2024-04-25T10:26:12.969666+0000","last_unstale":"2024-04-25T10:26:15.647878+0000","last_undegraded":"2024-04-25T10:26:15.647878+0000","last_fullsized":"2024-04-25T10:26:15.647878+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:23.925941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982622+0000","last_change":"2024-04-25T10:26:14.004973+0000","last_active":"2024-04-25T10:26:14.982622+0000","last_peered":"2024-04-25T10:26:14.982622+0000","last_clean":"2024-04-25T10:26:14.982622+0000","last_became_active":"2024-04-25T10:26:14.004702+0000","last_became_peered":"2024-04-25T10:26:14.004702+0000","last_unstale":"2024-04-25T10:26:14.982622+0000","last_undegraded":"2024-04-25T10:26:14.982622+0000","last_fullsized":"2024-04-25T10:26:14.982622+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:46.640917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649467+0000","last_change":"2024-04-25T10:26:12.968858+0000","last_active":"2024-04-25T10:26:15.649467+0000","last_peered":"2024-04-25T10:26:15.649467+0000","last_clean":"2024-04-25T10:26:15.649467+0000","last_became_active":"2024-04-25T10:26:12.968309+0000","last_became_peered":"2024-04-25T10:26:12.968309+0000","last_unstale":"2024-04-25T10:26:15.649467+0000","last_undegraded":"2024-04-25T10:26:15.649467+0000","last_fullsized":"2024-04-25T10:26:15.649467+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:41.670389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649771+0000","last_change":"2024-04-25T10:26:14.005318+0000","last_active":"2024-04-25T10:26:15.649771+0000","last_peered":"2024-04-25T10:26:15.649771+0000","last_clean":"2024-04-25T10:26:15.649771+0000","last_became_active":"2024-04-25T10:26:14.004635+0000","last_became_peered":"2024-04-25T10:26:14.004635+0000","last_unstale":"2024-04-25T10:26:15.649771+0000","last_undegraded":"2024-04-25T10:26:15.649771+0000","last_fullsized":"2024-04-25T10:26:15.649771+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:46.548053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982342+0000","last_change":"2024-04-25T10:26:12.968507+0000","last_active":"2024-04-25T10:26:14.982342+0000","last_peered":"2024-04-25T10:26:14.982342+0000","last_clean":"2024-04-25T10:26:14.982342+0000","last_became_active":"2024-04-25T10:26:12.967779+0000","last_became_peered":"2024-04-25T10:26:12.967779+0000","last_unstale":"2024-04-25T10:26:14.982342+0000","last_undegraded":"2024-04-25T10:26:14.982342+0000","last_fullsized":"2024-04-25T10:26:14.982342+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:21.393537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982224+0000","last_change":"2024-04-25T10:26:13.990544+0000","last_active":"2024-04-25T10:26:14.982224+0000","last_peered":"2024-04-25T10:26:14.982224+0000","last_clean":"2024-04-25T10:26:14.982224+0000","last_became_active":"2024-04-25T10:26:13.990245+0000","last_became_peered":"2024-04-25T10:26:13.990245+0000","last_unstale":"2024-04-25T10:26:14.982224+0000","last_undegraded":"2024-04-25T10:26:14.982224+0000","last_fullsized":"2024-04-25T10:26:14.982224+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:42.229525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649869+0000","last_change":"2024-04-25T10:26:12.974852+0000","last_active":"2024-04-25T10:26:15.649869+0000","last_peered":"2024-04-25T10:26:15.649869+0000","last_clean":"2024-04-25T10:26:15.649869+0000","last_became_active":"2024-04-25T10:26:12.974599+0000","last_became_peered":"2024-04-25T10:26:12.974599+0000","last_unstale":"2024-04-25T10:26:15.649869+0000","last_undegraded":"2024-04-25T10:26:15.649869+0000","last_fullsized":"2024-04-25T10:26:15.649869+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:23.925941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648639+0000","last_change":"2024-04-25T10:26:13.999919+0000","last_active":"2024-04-25T10:26:15.648639+0000","last_peered":"2024-04-25T10:26:15.648639+0000","last_clean":"2024-04-25T10:26:15.648639+0000","last_became_active":"2024-04-25T10:26:13.999552+0000","last_became_peered":"2024-04-25T10:26:13.999552+0000","last_unstale":"2024-04-25T10:26:15.648639+0000","last_undegraded":"2024-04-25T10:26:15.648639+0000","last_fullsized":"2024-04-25T10:26:15.648639+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:21.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986116+0000","last_change":"2024-04-25T10:26:12.978849+0000","last_active":"2024-04-25T10:26:14.986116+0000","last_peered":"2024-04-25T10:26:14.986116+0000","last_clean":"2024-04-25T10:26:14.986116+0000","last_became_active":"2024-04-25T10:26:12.978452+0000","last_became_peered":"2024-04-25T10:26:12.978452+0000","last_unstale":"2024-04-25T10:26:14.986116+0000","last_undegraded":"2024-04-25T10:26:14.986116+0000","last_fullsized":"2024-04-25T10:26:14.986116+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:28.709682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648911+0000","last_change":"2024-04-25T10:26:14.004253+0000","last_active":"2024-04-25T10:26:15.648911+0000","last_peered":"2024-04-25T10:26:15.648911+0000","last_clean":"2024-04-25T10:26:15.648911+0000","last_became_active":"2024-04-25T10:26:14.003964+0000","last_became_peered":"2024-04-25T10:26:14.003964+0000","last_unstale":"2024-04-25T10:26:15.648911+0000","last_undegraded":"2024-04-25T10:26:15.648911+0000","last_fullsized":"2024-04-25T10:26:15.648911+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:03.157961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650719+0000","last_change":"2024-04-25T10:26:12.974260+0000","last_active":"2024-04-25T10:26:15.650719+0000","last_peered":"2024-04-25T10:26:15.650719+0000","last_clean":"2024-04-25T10:26:15.650719+0000","last_became_active":"2024-04-25T10:26:12.973967+0000","last_became_peered":"2024-04-25T10:26:12.973967+0000","last_unstale":"2024-04-25T10:26:15.650719+0000","last_undegraded":"2024-04-25T10:26:15.650719+0000","last_fullsized":"2024-04-25T10:26:15.650719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:14.890002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986599+0000","last_change":"2024-04-25T10:26:13.978742+0000","last_active":"2024-04-25T10:26:14.986599+0000","last_peered":"2024-04-25T10:26:14.986599+0000","last_clean":"2024-04-25T10:26:14.986599+0000","last_became_active":"2024-04-25T10:26:13.978427+0000","last_became_peered":"2024-04-25T10:26:13.978427+0000","last_unstale":"2024-04-25T10:26:14.986599+0000","last_undegraded":"2024-04-25T10:26:14.986599+0000","last_fullsized":"2024-04-25T10:26:14.986599+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.104485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"116'3","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650636+0000","last_change":"2024-04-25T10:26:12.979707+0000","last_active":"2024-04-25T10:26:15.650636+0000","last_peered":"2024-04-25T10:26:15.650636+0000","last_clean":"2024-04-25T10:26:15.650636+0000","last_became_active":"2024-04-25T10:26:12.979419+0000","last_became_peered":"2024-04-25T10:26:12.979419+0000","last_unstale":"2024-04-25T10:26:15.650636+0000","last_undegraded":"2024-04-25T10:26:15.650636+0000","last_fullsized":"2024-04-25T10:26:15.650636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.116774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983049+0000","last_change":"2024-04-25T10:26:13.995636+0000","last_active":"2024-04-25T10:26:14.983049+0000","last_peered":"2024-04-25T10:26:14.983049+0000","last_clean":"2024-04-25T10:26:14.983049+0000","last_became_active":"2024-04-25T10:26:13.994939+0000","last_became_peered":"2024-04-25T10:26:13.994939+0000","last_unstale":"2024-04-25T10:26:14.983049+0000","last_undegraded":"2024-04-25T10:26:14.983049+0000","last_fullsized":"2024-04-25T10:26:14.983049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:54.463095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"116'6","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985266+0000","last_change":"2024-04-25T10:26:12.974365+0000","last_active":"2024-04-25T10:26:14.985266+0000","last_peered":"2024-04-25T10:26:14.985266+0000","last_clean":"2024-04-25T10:26:14.985266+0000","last_became_active":"2024-04-25T10:26:12.973427+0000","last_became_peered":"2024-04-25T10:26:12.973427+0000","last_unstale":"2024-04-25T10:26:14.985266+0000","last_undegraded":"2024-04-25T10:26:14.985266+0000","last_fullsized":"2024-04-25T10:26:14.985266+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:06.072227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650024+0000","last_change":"2024-04-25T10:26:13.999800+0000","last_active":"2024-04-25T10:26:15.650024+0000","last_peered":"2024-04-25T10:26:15.650024+0000","last_clean":"2024-04-25T10:26:15.650024+0000","last_became_active":"2024-04-25T10:26:13.999506+0000","last_became_peered":"2024-04-25T10:26:13.999506+0000","last_unstale":"2024-04-25T10:26:15.650024+0000","last_undegraded":"2024-04-25T10:26:15.650024+0000","last_fullsized":"2024-04-25T10:26:15.650024+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:28.280963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650411+0000","last_change":"2024-04-25T10:26:12.974720+0000","last_active":"2024-04-25T10:26:15.650411+0000","last_peered":"2024-04-25T10:26:15.650411+0000","last_clean":"2024-04-25T10:26:15.650411+0000","last_became_active":"2024-04-25T10:26:12.974006+0000","last_became_peered":"2024-04-25T10:26:12.974006+0000","last_unstale":"2024-04-25T10:26:15.650411+0000","last_undegraded":"2024-04-25T10:26:15.650411+0000","last_fullsized":"2024-04-25T10:26:15.650411+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:45.691111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985040+0000","last_change":"2024-04-25T10:26:13.994164+0000","last_active":"2024-04-25T10:26:14.985040+0000","last_peered":"2024-04-25T10:26:14.985040+0000","last_clean":"2024-04-25T10:26:14.985040+0000","last_became_active":"2024-04-25T10:26:13.993915+0000","last_became_peered":"2024-04-25T10:26:13.993915+0000","last_unstale":"2024-04-25T10:26:14.985040+0000","last_undegraded":"2024-04-25T10:26:14.985040+0000","last_fullsized":"2024-04-25T10:26:14.985040+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.294953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647768+0000","last_change":"2024-04-25T10:26:12.970303+0000","last_active":"2024-04-25T10:26:15.647768+0000","last_peered":"2024-04-25T10:26:15.647768+0000","last_clean":"2024-04-25T10:26:15.647768+0000","last_became_active":"2024-04-25T10:26:12.969076+0000","last_became_peered":"2024-04-25T10:26:12.969076+0000","last_unstale":"2024-04-25T10:26:15.647768+0000","last_undegraded":"2024-04-25T10:26:15.647768+0000","last_fullsized":"2024-04-25T10:26:15.647768+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985187+0000","last_change":"2024-04-25T10:26:13.991851+0000","last_active":"2024-04-25T10:26:14.985187+0000","last_peered":"2024-04-25T10:26:14.985187+0000","last_clean":"2024-04-25T10:26:14.985187+0000","last_became_active":"2024-04-25T10:26:13.991588+0000","last_became_peered":"2024-04-25T10:26:13.991588+0000","last_unstale":"2024-04-25T10:26:14.985187+0000","last_undegraded":"2024-04-25T10:26:14.985187+0000","last_fullsized":"2024-04-25T10:26:14.985187+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:19.981211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649347+0000","last_change":"2024-04-25T10:26:12.970708+0000","last_active":"2024-04-25T10:26:15.649347+0000","last_peered":"2024-04-25T10:26:15.649347+0000","last_clean":"2024-04-25T10:26:15.649347+0000","last_became_active":"2024-04-25T10:26:12.970360+0000","last_became_peered":"2024-04-25T10:26:12.970360+0000","last_unstale":"2024-04-25T10:26:15.649347+0000","last_undegraded":"2024-04-25T10:26:15.649347+0000","last_fullsized":"2024-04-25T10:26:15.649347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:49.519690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649898+0000","last_change":"2024-04-25T10:26:13.994489+0000","last_active":"2024-04-25T10:26:15.649898+0000","last_peered":"2024-04-25T10:26:15.649898+0000","last_clean":"2024-04-25T10:26:15.649898+0000","last_became_active":"2024-04-25T10:26:13.994104+0000","last_became_peered":"2024-04-25T10:26:13.994104+0000","last_unstale":"2024-04-25T10:26:15.649898+0000","last_undegraded":"2024-04-25T10:26:15.649898+0000","last_fullsized":"2024-04-25T10:26:15.649898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:27.574489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986494+0000","last_change":"2024-04-25T10:26:12.972335+0000","last_active":"2024-04-25T10:26:14.986494+0000","last_peered":"2024-04-25T10:26:14.986494+0000","last_clean":"2024-04-25T10:26:14.986494+0000","last_became_active":"2024-04-25T10:26:12.972062+0000","last_became_peered":"2024-04-25T10:26:12.972062+0000","last_unstale":"2024-04-25T10:26:14.986494+0000","last_undegraded":"2024-04-25T10:26:14.986494+0000","last_fullsized":"2024-04-25T10:26:14.986494+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:52.946736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650136+0000","last_change":"2024-04-25T10:26:13.997811+0000","last_active":"2024-04-25T10:26:15.650136+0000","last_peered":"2024-04-25T10:26:15.650136+0000","last_clean":"2024-04-25T10:26:15.650136+0000","last_became_active":"2024-04-25T10:26:13.997537+0000","last_became_peered":"2024-04-25T10:26:13.997537+0000","last_unstale":"2024-04-25T10:26:15.650136+0000","last_undegraded":"2024-04-25T10:26:15.650136+0000","last_fullsized":"2024-04-25T10:26:15.650136+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:03.157961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650828+0000","last_change":"2024-04-25T10:26:12.974012+0000","last_active":"2024-04-25T10:26:15.650828+0000","last_peered":"2024-04-25T10:26:15.650828+0000","last_clean":"2024-04-25T10:26:15.650828+0000","last_became_active":"2024-04-25T10:26:12.973735+0000","last_became_peered":"2024-04-25T10:26:12.973735+0000","last_unstale":"2024-04-25T10:26:15.650828+0000","last_undegraded":"2024-04-25T10:26:15.650828+0000","last_fullsized":"2024-04-25T10:26:15.650828+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982858+0000","last_change":"2024-04-25T10:26:13.988908+0000","last_active":"2024-04-25T10:26:14.982858+0000","last_peered":"2024-04-25T10:26:14.982858+0000","last_clean":"2024-04-25T10:26:14.982858+0000","last_became_active":"2024-04-25T10:26:13.988614+0000","last_became_peered":"2024-04-25T10:26:13.988614+0000","last_unstale":"2024-04-25T10:26:14.982858+0000","last_undegraded":"2024-04-25T10:26:14.982858+0000","last_fullsized":"2024-04-25T10:26:14.982858+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:34.598708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153197+0000","last_change":"2024-04-25T10:26:12.972502+0000","last_active":"2024-04-25T10:26:15.153197+0000","last_peered":"2024-04-25T10:26:15.153197+0000","last_clean":"2024-04-25T10:26:15.153197+0000","last_became_active":"2024-04-25T10:26:12.972135+0000","last_became_peered":"2024-04-25T10:26:12.972135+0000","last_unstale":"2024-04-25T10:26:15.153197+0000","last_undegraded":"2024-04-25T10:26:15.153197+0000","last_fullsized":"2024-04-25T10:26:15.153197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.952494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153092+0000","last_change":"2024-04-25T10:26:13.996926+0000","last_active":"2024-04-25T10:26:15.153092+0000","last_peered":"2024-04-25T10:26:15.153092+0000","last_clean":"2024-04-25T10:26:15.153092+0000","last_became_active":"2024-04-25T10:26:13.996624+0000","last_became_peered":"2024-04-25T10:26:13.996624+0000","last_unstale":"2024-04-25T10:26:15.153092+0000","last_undegraded":"2024-04-25T10:26:15.153092+0000","last_fullsized":"2024-04-25T10:26:15.153092+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:55.901875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152833+0000","last_change":"2024-04-25T10:26:12.975104+0000","last_active":"2024-04-25T10:26:15.152833+0000","last_peered":"2024-04-25T10:26:15.152833+0000","last_clean":"2024-04-25T10:26:15.152833+0000","last_became_active":"2024-04-25T10:26:12.974542+0000","last_became_peered":"2024-04-25T10:26:12.974542+0000","last_unstale":"2024-04-25T10:26:15.152833+0000","last_undegraded":"2024-04-25T10:26:15.152833+0000","last_fullsized":"2024-04-25T10:26:15.152833+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.161208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647644+0000","last_change":"2024-04-25T10:26:13.997176+0000","last_active":"2024-04-25T10:26:15.647644+0000","last_peered":"2024-04-25T10:26:15.647644+0000","last_clean":"2024-04-25T10:26:15.647644+0000","last_became_active":"2024-04-25T10:26:13.996632+0000","last_became_peered":"2024-04-25T10:26:13.996632+0000","last_unstale":"2024-04-25T10:26:15.647644+0000","last_undegraded":"2024-04-25T10:26:15.647644+0000","last_fullsized":"2024-04-25T10:26:15.647644+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.310317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650595+0000","last_change":"2024-04-25T10:26:12.974951+0000","last_active":"2024-04-25T10:26:15.650595+0000","last_peered":"2024-04-25T10:26:15.650595+0000","last_clean":"2024-04-25T10:26:15.650595+0000","last_became_active":"2024-04-25T10:26:12.974439+0000","last_became_peered":"2024-04-25T10:26:12.974439+0000","last_unstale":"2024-04-25T10:26:15.650595+0000","last_undegraded":"2024-04-25T10:26:15.650595+0000","last_fullsized":"2024-04-25T10:26:15.650595+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649896+0000","last_change":"2024-04-25T10:26:14.001676+0000","last_active":"2024-04-25T10:26:15.649896+0000","last_peered":"2024-04-25T10:26:15.649896+0000","last_clean":"2024-04-25T10:26:15.649896+0000","last_became_active":"2024-04-25T10:26:14.000901+0000","last_became_peered":"2024-04-25T10:26:14.000901+0000","last_unstale":"2024-04-25T10:26:15.649896+0000","last_undegraded":"2024-04-25T10:26:15.649896+0000","last_fullsized":"2024-04-25T10:26:15.649896+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:38.117447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649702+0000","last_change":"2024-04-25T10:26:12.971625+0000","last_active":"2024-04-25T10:26:15.649702+0000","last_peered":"2024-04-25T10:26:15.649702+0000","last_clean":"2024-04-25T10:26:15.649702+0000","last_became_active":"2024-04-25T10:26:12.971364+0000","last_became_peered":"2024-04-25T10:26:12.971364+0000","last_unstale":"2024-04-25T10:26:15.649702+0000","last_undegraded":"2024-04-25T10:26:15.649702+0000","last_fullsized":"2024-04-25T10:26:15.649702+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:23.625246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152776+0000","last_change":"2024-04-25T10:26:13.999968+0000","last_active":"2024-04-25T10:26:15.152776+0000","last_peered":"2024-04-25T10:26:15.152776+0000","last_clean":"2024-04-25T10:26:15.152776+0000","last_became_active":"2024-04-25T10:26:13.999689+0000","last_became_peered":"2024-04-25T10:26:13.999689+0000","last_unstale":"2024-04-25T10:26:15.152776+0000","last_undegraded":"2024-04-25T10:26:15.152776+0000","last_fullsized":"2024-04-25T10:26:15.152776+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:14.641929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984863+0000","last_change":"2024-04-25T10:26:12.973236+0000","last_active":"2024-04-25T10:26:14.984863+0000","last_peered":"2024-04-25T10:26:14.984863+0000","last_clean":"2024-04-25T10:26:14.984863+0000","last_became_active":"2024-04-25T10:26:12.972387+0000","last_became_peered":"2024-04-25T10:26:12.972387+0000","last_unstale":"2024-04-25T10:26:14.984863+0000","last_undegraded":"2024-04-25T10:26:14.984863+0000","last_fullsized":"2024-04-25T10:26:14.984863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:43.977648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649781+0000","last_change":"2024-04-25T10:26:13.997607+0000","last_active":"2024-04-25T10:26:15.649781+0000","last_peered":"2024-04-25T10:26:15.649781+0000","last_clean":"2024-04-25T10:26:15.649781+0000","last_became_active":"2024-04-25T10:26:13.997331+0000","last_became_peered":"2024-04-25T10:26:13.997331+0000","last_unstale":"2024-04-25T10:26:15.649781+0000","last_undegraded":"2024-04-25T10:26:15.649781+0000","last_fullsized":"2024-04-25T10:26:15.649781+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:21.573428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984822+0000","last_change":"2024-04-25T10:26:12.970548+0000","last_active":"2024-04-25T10:26:14.984822+0000","last_peered":"2024-04-25T10:26:14.984822+0000","last_clean":"2024-04-25T10:26:14.984822+0000","last_became_active":"2024-04-25T10:26:12.970247+0000","last_became_peered":"2024-04-25T10:26:12.970247+0000","last_unstale":"2024-04-25T10:26:14.984822+0000","last_undegraded":"2024-04-25T10:26:14.984822+0000","last_fullsized":"2024-04-25T10:26:14.984822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.774633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153074+0000","last_change":"2024-04-25T10:26:13.998614+0000","last_active":"2024-04-25T10:26:15.153074+0000","last_peered":"2024-04-25T10:26:15.153074+0000","last_clean":"2024-04-25T10:26:15.153074+0000","last_became_active":"2024-04-25T10:26:13.998313+0000","last_became_peered":"2024-04-25T10:26:13.998313+0000","last_unstale":"2024-04-25T10:26:15.153074+0000","last_undegraded":"2024-04-25T10:26:15.153074+0000","last_fullsized":"2024-04-25T10:26:15.153074+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:34.370206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986171+0000","last_change":"2024-04-25T10:26:12.978347+0000","last_active":"2024-04-25T10:26:14.986171+0000","last_peered":"2024-04-25T10:26:14.986171+0000","last_clean":"2024-04-25T10:26:14.986171+0000","last_became_active":"2024-04-25T10:26:12.977701+0000","last_became_peered":"2024-04-25T10:26:12.977701+0000","last_unstale":"2024-04-25T10:26:14.986171+0000","last_undegraded":"2024-04-25T10:26:14.986171+0000","last_fullsized":"2024-04-25T10:26:14.986171+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:39.355250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986517+0000","last_change":"2024-04-25T10:26:13.984186+0000","last_active":"2024-04-25T10:26:14.986517+0000","last_peered":"2024-04-25T10:26:14.986517+0000","last_clean":"2024-04-25T10:26:14.986517+0000","last_became_active":"2024-04-25T10:26:13.983911+0000","last_became_peered":"2024-04-25T10:26:13.983911+0000","last_unstale":"2024-04-25T10:26:14.986517+0000","last_undegraded":"2024-04-25T10:26:14.986517+0000","last_fullsized":"2024-04-25T10:26:14.986517+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.558024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"117'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983099+0000","last_change":"2024-04-25T10:26:12.972745+0000","last_active":"2024-04-25T10:26:14.983099+0000","last_peered":"2024-04-25T10:26:14.983099+0000","last_clean":"2024-04-25T10:26:14.983099+0000","last_became_active":"2024-04-25T10:26:12.972443+0000","last_became_peered":"2024-04-25T10:26:12.972443+0000","last_unstale":"2024-04-25T10:26:14.983099+0000","last_undegraded":"2024-04-25T10:26:14.983099+0000","last_fullsized":"2024-04-25T10:26:14.983099+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:03.914613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649797+0000","last_change":"2024-04-25T10:26:14.001199+0000","last_active":"2024-04-25T10:26:15.649797+0000","last_peered":"2024-04-25T10:26:15.649797+0000","last_clean":"2024-04-25T10:26:15.649797+0000","last_became_active":"2024-04-25T10:26:14.000919+0000","last_became_peered":"2024-04-25T10:26:14.000919+0000","last_unstale":"2024-04-25T10:26:15.649797+0000","last_undegraded":"2024-04-25T10:26:15.649797+0000","last_fullsized":"2024-04-25T10:26:15.649797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:01.962864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985492+0000","last_change":"2024-04-25T10:26:12.978046+0000","last_active":"2024-04-25T10:26:14.985492+0000","last_peered":"2024-04-25T10:26:14.985492+0000","last_clean":"2024-04-25T10:26:14.985492+0000","last_became_active":"2024-04-25T10:26:12.977677+0000","last_became_peered":"2024-04-25T10:26:12.977677+0000","last_unstale":"2024-04-25T10:26:14.985492+0000","last_undegraded":"2024-04-25T10:26:14.985492+0000","last_fullsized":"2024-04-25T10:26:14.985492+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:44.914361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153139+0000","last_change":"2024-04-25T10:26:14.010212+0000","last_active":"2024-04-25T10:26:15.153139+0000","last_peered":"2024-04-25T10:26:15.153139+0000","last_clean":"2024-04-25T10:26:15.153139+0000","last_became_active":"2024-04-25T10:26:14.009871+0000","last_became_peered":"2024-04-25T10:26:14.009871+0000","last_unstale":"2024-04-25T10:26:15.153139+0000","last_undegraded":"2024-04-25T10:26:15.153139+0000","last_fullsized":"2024-04-25T10:26:15.153139+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:16.795629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985277+0000","last_change":"2024-04-25T10:26:12.975861+0000","last_active":"2024-04-25T10:26:14.985277+0000","last_peered":"2024-04-25T10:26:14.985277+0000","last_clean":"2024-04-25T10:26:14.985277+0000","last_became_active":"2024-04-25T10:26:12.973756+0000","last_became_peered":"2024-04-25T10:26:12.973756+0000","last_unstale":"2024-04-25T10:26:14.985277+0000","last_undegraded":"2024-04-25T10:26:14.985277+0000","last_fullsized":"2024-04-25T10:26:14.985277+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:26.815973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647679+0000","last_change":"2024-04-25T10:26:13.997323+0000","last_active":"2024-04-25T10:26:15.647679+0000","last_peered":"2024-04-25T10:26:15.647679+0000","last_clean":"2024-04-25T10:26:15.647679+0000","last_became_active":"2024-04-25T10:26:13.997023+0000","last_became_peered":"2024-04-25T10:26:13.997023+0000","last_unstale":"2024-04-25T10:26:15.647679+0000","last_undegraded":"2024-04-25T10:26:15.647679+0000","last_fullsized":"2024-04-25T10:26:15.647679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:18.512773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982768+0000","last_change":"2024-04-25T10:26:12.965357+0000","last_active":"2024-04-25T10:26:14.982768+0000","last_peered":"2024-04-25T10:26:14.982768+0000","last_clean":"2024-04-25T10:26:14.982768+0000","last_became_active":"2024-04-25T10:26:12.964963+0000","last_became_peered":"2024-04-25T10:26:12.964963+0000","last_unstale":"2024-04-25T10:26:14.982768+0000","last_undegraded":"2024-04-25T10:26:14.982768+0000","last_fullsized":"2024-04-25T10:26:14.982768+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:56.157663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152954+0000","last_change":"2024-04-25T10:26:14.003694+0000","last_active":"2024-04-25T10:26:15.152954+0000","last_peered":"2024-04-25T10:26:15.152954+0000","last_clean":"2024-04-25T10:26:15.152954+0000","last_became_active":"2024-04-25T10:26:14.002353+0000","last_became_peered":"2024-04-25T10:26:14.002353+0000","last_unstale":"2024-04-25T10:26:15.152954+0000","last_undegraded":"2024-04-25T10:26:15.152954+0000","last_fullsized":"2024-04-25T10:26:15.152954+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:24.631561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152900+0000","last_change":"2024-04-25T10:26:12.968833+0000","last_active":"2024-04-25T10:26:15.152900+0000","last_peered":"2024-04-25T10:26:15.152900+0000","last_clean":"2024-04-25T10:26:15.152900+0000","last_became_active":"2024-04-25T10:26:12.968544+0000","last_became_peered":"2024-04-25T10:26:12.968544+0000","last_unstale":"2024-04-25T10:26:15.152900+0000","last_undegraded":"2024-04-25T10:26:15.152900+0000","last_fullsized":"2024-04-25T10:26:15.152900+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:08.010695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152855+0000","last_change":"2024-04-25T10:26:13.999839+0000","last_active":"2024-04-25T10:26:15.152855+0000","last_peered":"2024-04-25T10:26:15.152855+0000","last_clean":"2024-04-25T10:26:15.152855+0000","last_became_active":"2024-04-25T10:26:13.997251+0000","last_became_peered":"2024-04-25T10:26:13.997251+0000","last_unstale":"2024-04-25T10:26:15.152855+0000","last_undegraded":"2024-04-25T10:26:15.152855+0000","last_fullsized":"2024-04-25T10:26:15.152855+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:39.559343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650192+0000","last_change":"2024-04-25T10:26:12.970872+0000","last_active":"2024-04-25T10:26:15.650192+0000","last_peered":"2024-04-25T10:26:15.650192+0000","last_clean":"2024-04-25T10:26:15.650192+0000","last_became_active":"2024-04-25T10:26:12.970594+0000","last_became_peered":"2024-04-25T10:26:12.970594+0000","last_unstale":"2024-04-25T10:26:15.650192+0000","last_undegraded":"2024-04-25T10:26:15.650192+0000","last_fullsized":"2024-04-25T10:26:15.650192+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:38.553027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985727+0000","last_change":"2024-04-25T10:26:13.995296+0000","last_active":"2024-04-25T10:26:14.985727+0000","last_peered":"2024-04-25T10:26:14.985727+0000","last_clean":"2024-04-25T10:26:14.985727+0000","last_became_active":"2024-04-25T10:26:13.995024+0000","last_became_peered":"2024-04-25T10:26:13.995024+0000","last_unstale":"2024-04-25T10:26:14.985727+0000","last_undegraded":"2024-04-25T10:26:14.985727+0000","last_fullsized":"2024-04-25T10:26:14.985727+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:10.289490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985634+0000","last_change":"2024-04-25T10:26:12.969936+0000","last_active":"2024-04-25T10:26:14.985634+0000","last_peered":"2024-04-25T10:26:14.985634+0000","last_clean":"2024-04-25T10:26:14.985634+0000","last_became_active":"2024-04-25T10:26:12.969633+0000","last_became_peered":"2024-04-25T10:26:12.969633+0000","last_unstale":"2024-04-25T10:26:14.985634+0000","last_undegraded":"2024-04-25T10:26:14.985634+0000","last_fullsized":"2024-04-25T10:26:14.985634+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:26.568174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984766+0000","last_change":"2024-04-25T10:26:13.983478+0000","last_active":"2024-04-25T10:26:14.984766+0000","last_peered":"2024-04-25T10:26:14.984766+0000","last_clean":"2024-04-25T10:26:14.984766+0000","last_became_active":"2024-04-25T10:26:13.983191+0000","last_became_peered":"2024-04-25T10:26:13.983191+0000","last_unstale":"2024-04-25T10:26:14.984766+0000","last_undegraded":"2024-04-25T10:26:14.984766+0000","last_fullsized":"2024-04-25T10:26:14.984766+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:21.951679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648161+0000","last_change":"2024-04-25T10:26:12.968805+0000","last_active":"2024-04-25T10:26:15.648161+0000","last_peered":"2024-04-25T10:26:15.648161+0000","last_clean":"2024-04-25T10:26:15.648161+0000","last_became_active":"2024-04-25T10:26:12.968534+0000","last_became_peered":"2024-04-25T10:26:12.968534+0000","last_unstale":"2024-04-25T10:26:15.648161+0000","last_undegraded":"2024-04-25T10:26:15.648161+0000","last_fullsized":"2024-04-25T10:26:15.648161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":351,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647738+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:26:15.647738+0000","last_peered":"2024-04-25T10:26:15.647738+0000","last_clean":"2024-04-25T10:26:15.647738+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:26:15.647738+0000","last_undegraded":"2024-04-25T10:26:15.647738+0000","last_fullsized":"2024-04-25T10:26:15.647738+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1466368,"data_stored":629228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50644,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321196,"statfs":{"total":96636764160,"available":96584904704,"internally_reserved":0,"allocated":1445888,"data_stored":627492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509545,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2048000,"data_stored":1217952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58964,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312876,"statfs":{"total":96636764160,"available":96576385024,"internally_reserved":0,"allocated":1445888,"data_stored":627492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58972,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312868,"statfs":{"total":96636764160,"available":96576376832,"internally_reserved":0,"allocated":1458176,"data_stored":628130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55380,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316460,"statfs":{"total":96636764160,"available":96580055040,"internally_reserved":0,"allocated":2039808,"data_stored":1217860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28444,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343396,"statfs":{"total":96636764160,"available":96607637504,"internally_reserved":0,"allocated":1454080,"data_stored":628144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542248,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58980,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312860,"statfs":{"total":96636764160,"available":96576368640,"internally_reserved":0,"allocated":1462272,"data_stored":629194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:19.938 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:26:19.935+0000 7faab1c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:26:22.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:26:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 Processor -- start 2024-04-25T10:26:22.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 -- start start 2024-04-25T10:26:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f015db40 0x7f54f006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f00b5010 0x7f54f015df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54f006b2c0 0x7f54f006b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f006bc70 con 0x7f54f00b5010 2024-04-25T10:26:22.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f015e460 con 0x7f54f006b2c0 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54efe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b5780 con 0x7f54f015db40 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54e6000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f00b5010 0x7f54f015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54e6000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54f00b5010 0x7f54f015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37726/0 (socket says 172.21.15.45:37726) 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.839+0000 7f54e6000640 1 -- 172.21.15.45:0/1212070906 learned_addr learned my addr 172.21.15.45:0/1212070906 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54ef400640 1 --2- 172.21.15.45:0/1212070906 >> v2:172.21.15.117:3300/0 conn(0x7f54f006b2c0 0x7f54f006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54eea00640 1 --2- 172.21.15.45:0/1212070906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f015db40 0x7f54f006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54e6000640 1 -- 172.21.15.45:0/1212070906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f015db40 msgr2=0x7f54f006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54e6000640 1 --2- 172.21.15.45:0/1212070906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f015db40 0x7f54f006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54e6000640 1 -- 172.21.15.45:0/1212070906 >> v2:172.21.15.117:3300/0 conn(0x7f54f006b2c0 msgr2=0x7f54f006b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54e6000640 1 --2- 172.21.15.45:0/1212070906 >> v2:172.21.15.117:3300/0 conn(0x7f54f006b2c0 0x7f54f006b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54e6000640 1 -- 172.21.15.45:0/1212070906 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00b5a60 con 0x7f54f00b5010 2024-04-25T10:26:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54e6000640 1 --2- 172.21.15.45:0/1212070906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f00b5010 0x7f54f015df20 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f54f006dbd0 tx=0x7f54d801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158aa3d9abec42c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54ee000640 1 -- 172.21.15.45:0/1212070906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d80b5070 con 0x7f54f00b5010 2024-04-25T10:26:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54ee000640 1 -- 172.21.15.45:0/1212070906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d80a89d0 con 0x7f54f00b5010 2024-04-25T10:26:22.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54ee000640 1 -- 172.21.15.45:0/1212070906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d80a8da0 con 0x7f54f00b5010 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54efe00640 1 -- 172.21.15.45:0/1212070906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f00b5010 msgr2=0x7f54f015df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.843+0000 7f54efe00640 1 --2- 172.21.15.45:0/1212070906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f00b5010 0x7f54f015df20 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f54f006dbd0 tx=0x7f54d801ed10 comp rx=0 tx=0).stop 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- 172.21.15.45:0/1212070906 shutdown_connections 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 --2- 172.21.15.45:0/1212070906 >> v2:172.21.15.117:3300/0 conn(0x7f54f006b2c0 0x7f54f006b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 --2- 172.21.15.45:0/1212070906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f00b5010 0x7f54f015df20 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 --2- 172.21.15.45:0/1212070906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f015db40 0x7f54f006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- 172.21.15.45:0/1212070906 >> 172.21.15.45:0/1212070906 conn(0x7f54f00d17b0 msgr2=0x7f54f00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- 172.21.15.45:0/1212070906 shutdown_connections 2024-04-25T10:26:22.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- 172.21.15.45:0/1212070906 wait complete. 2024-04-25T10:26:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 Processor -- start 2024-04-25T10:26:22.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- start start 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 0x7f54f01f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f00b5010 0x7f54f009b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54f015db40 0x7f54f009bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b6300 con 0x7f54f006b2c0 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b6020 con 0x7f54f015db40 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54efe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00b5450 con 0x7f54f00b5010 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 0x7f54f01f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 0x7f54f01f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37740/0 (socket says 172.21.15.45:37740) 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 -- 172.21.15.45:0/2399331952 learned_addr learned my addr 172.21.15.45:0/2399331952 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:22.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54ef400640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54f015db40 0x7f54f009bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54e6000640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f00b5010 0x7f54f009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 -- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f00b5010 msgr2=0x7f54f009b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f00b5010 0x7f54f009b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 -- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54f015db40 msgr2=0x7f54f009bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54f015db40 0x7f54f009bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 -- 172.21.15.45:0/2399331952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f009e3f0 con 0x7f54f006b2c0 2024-04-25T10:26:22.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54eea00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 0x7f54f01f4550 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f54e000e870 tx=0x7f54e000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0017070 con 0x7f54f006b2c0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e000c940 con 0x7f54f006b2c0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.847+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e000ccf0 con 0x7f54f006b2c0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f009e6d0 con 0x7f54f006b2c0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f009ec90 con 0x7f54f006b2c0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f0075120 con 0x7f54f006b2c0 2024-04-25T10:26:22.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54e001a070 con 0x7f54f006b2c0 2024-04-25T10:26:22.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54e7e00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54cc066f60 0x7f54cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:22.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54e00851a0 con 0x7f54f006b2c0 2024-04-25T10:26:22.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54e6000640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54cc066f60 0x7f54cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:22.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.851+0000 7f54e6000640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54cc066f60 0x7f54cc069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f54d80b2000 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:22.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:22.855+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54e0055b50 con 0x7f54f006b2c0 2024-04-25T10:26:23.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.015+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54f006f470 con 0x7f54cc066f60 2024-04-25T10:26:23.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54e7e00640 1 -- 172.21.15.45:0/2399331952 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252467 (secure 0 0 0) 0x7f54f006f470 con 0x7f54cc066f60 2024-04-25T10:26:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:23.028 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54cc066f60 msgr2=0x7f54cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54cc066f60 0x7f54cc069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f54d80b2000 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 msgr2=0x7f54f01f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 0x7f54f01f4550 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f54e000e870 tx=0x7f54e000ec70 comp rx=0 tx=0).stop 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 shutdown_connections 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54cc066f60 0x7f54cc069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54f015db40 0x7f54f009bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54f00b5010 0x7f54f009b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 --2- 172.21.15.45:0/2399331952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54f006b2c0 0x7f54f01f4550 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 >> 172.21.15.45:0/2399331952 conn(0x7f54f00d17b0 msgr2=0x7f54f00d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 shutdown_connections 2024-04-25T10:26:23.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.023+0000 7f54efe00640 1 -- 172.21.15.45:0/2399331952 wait complete. 2024-04-25T10:26:23.047 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-25T10:26:22.990987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417436,"kb_used_data":12520,"kb_used_omap":12,"kb_used_meta":404787,"kb_avail":754557284,"statfs":{"total":773094113280,"available":772666658816,"internally_reserved":0,"allocated":12820480,"data_stored":6205492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":414502448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.016897"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649975+0000","last_change":"2024-04-25T10:26:14.005197+0000","last_active":"2024-04-25T10:26:15.649975+0000","last_peered":"2024-04-25T10:26:15.649975+0000","last_clean":"2024-04-25T10:26:15.649975+0000","last_became_active":"2024-04-25T10:26:14.004832+0000","last_became_peered":"2024-04-25T10:26:14.004832+0000","last_unstale":"2024-04-25T10:26:15.649975+0000","last_undegraded":"2024-04-25T10:26:15.649975+0000","last_fullsized":"2024-04-25T10:26:15.649975+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:53.953051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648603+0000","last_change":"2024-04-25T10:26:13.986851+0000","last_active":"2024-04-25T10:26:15.648603+0000","last_peered":"2024-04-25T10:26:15.648603+0000","last_clean":"2024-04-25T10:26:15.648603+0000","last_became_active":"2024-04-25T10:26:13.986540+0000","last_became_peered":"2024-04-25T10:26:13.986540+0000","last_unstale":"2024-04-25T10:26:15.648603+0000","last_undegraded":"2024-04-25T10:26:15.648603+0000","last_fullsized":"2024-04-25T10:26:15.648603+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:52.176540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648152+0000","last_change":"2024-04-25T10:26:14.007315+0000","last_active":"2024-04-25T10:26:15.648152+0000","last_peered":"2024-04-25T10:26:15.648152+0000","last_clean":"2024-04-25T10:26:15.648152+0000","last_became_active":"2024-04-25T10:26:14.007043+0000","last_became_peered":"2024-04-25T10:26:14.007043+0000","last_unstale":"2024-04-25T10:26:15.648152+0000","last_undegraded":"2024-04-25T10:26:15.648152+0000","last_fullsized":"2024-04-25T10:26:15.648152+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:54.738625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985211+0000","last_change":"2024-04-25T10:26:13.987496+0000","last_active":"2024-04-25T10:26:14.985211+0000","last_peered":"2024-04-25T10:26:14.985211+0000","last_clean":"2024-04-25T10:26:14.985211+0000","last_became_active":"2024-04-25T10:26:13.987243+0000","last_became_peered":"2024-04-25T10:26:13.987243+0000","last_unstale":"2024-04-25T10:26:14.985211+0000","last_undegraded":"2024-04-25T10:26:14.985211+0000","last_fullsized":"2024-04-25T10:26:14.985211+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:23.888743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650455+0000","last_change":"2024-04-25T10:26:14.002763+0000","last_active":"2024-04-25T10:26:15.650455+0000","last_peered":"2024-04-25T10:26:15.650455+0000","last_clean":"2024-04-25T10:26:15.650455+0000","last_became_active":"2024-04-25T10:26:14.000386+0000","last_became_peered":"2024-04-25T10:26:14.000386+0000","last_unstale":"2024-04-25T10:26:15.650455+0000","last_undegraded":"2024-04-25T10:26:15.650455+0000","last_fullsized":"2024-04-25T10:26:15.650455+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:39.629222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985852+0000","last_change":"2024-04-25T10:26:13.981550+0000","last_active":"2024-04-25T10:26:14.985852+0000","last_peered":"2024-04-25T10:26:14.985852+0000","last_clean":"2024-04-25T10:26:14.985852+0000","last_became_active":"2024-04-25T10:26:13.980971+0000","last_became_peered":"2024-04-25T10:26:13.980971+0000","last_unstale":"2024-04-25T10:26:14.985852+0000","last_undegraded":"2024-04-25T10:26:14.985852+0000","last_fullsized":"2024-04-25T10:26:14.985852+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:00.289419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982224+0000","last_change":"2024-04-25T10:26:13.991938+0000","last_active":"2024-04-25T10:26:14.982224+0000","last_peered":"2024-04-25T10:26:14.982224+0000","last_clean":"2024-04-25T10:26:14.982224+0000","last_became_active":"2024-04-25T10:26:13.991261+0000","last_became_peered":"2024-04-25T10:26:13.991261+0000","last_unstale":"2024-04-25T10:26:14.982224+0000","last_undegraded":"2024-04-25T10:26:14.982224+0000","last_fullsized":"2024-04-25T10:26:14.982224+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:18.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982276+0000","last_change":"2024-04-25T10:26:14.001718+0000","last_active":"2024-04-25T10:26:14.982276+0000","last_peered":"2024-04-25T10:26:14.982276+0000","last_clean":"2024-04-25T10:26:14.982276+0000","last_became_active":"2024-04-25T10:26:13.999412+0000","last_became_peered":"2024-04-25T10:26:13.999412+0000","last_unstale":"2024-04-25T10:26:14.982276+0000","last_undegraded":"2024-04-25T10:26:14.982276+0000","last_fullsized":"2024-04-25T10:26:14.982276+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:38.264923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649990+0000","last_change":"2024-04-25T10:26:14.006956+0000","last_active":"2024-04-25T10:26:15.649990+0000","last_peered":"2024-04-25T10:26:15.649990+0000","last_clean":"2024-04-25T10:26:15.649990+0000","last_became_active":"2024-04-25T10:26:14.006640+0000","last_became_peered":"2024-04-25T10:26:14.006640+0000","last_unstale":"2024-04-25T10:26:15.649990+0000","last_undegraded":"2024-04-25T10:26:15.649990+0000","last_fullsized":"2024-04-25T10:26:15.649990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:57.589244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650998+0000","last_change":"2024-04-25T10:26:13.999830+0000","last_active":"2024-04-25T10:26:15.650998+0000","last_peered":"2024-04-25T10:26:15.650998+0000","last_clean":"2024-04-25T10:26:15.650998+0000","last_became_active":"2024-04-25T10:26:13.999525+0000","last_became_peered":"2024-04-25T10:26:13.999525+0000","last_unstale":"2024-04-25T10:26:15.650998+0000","last_undegraded":"2024-04-25T10:26:15.650998+0000","last_fullsized":"2024-04-25T10:26:15.650998+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:41.212681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983017+0000","last_change":"2024-04-25T10:26:13.995853+0000","last_active":"2024-04-25T10:26:14.983017+0000","last_peered":"2024-04-25T10:26:14.983017+0000","last_clean":"2024-04-25T10:26:14.983017+0000","last_became_active":"2024-04-25T10:26:13.995365+0000","last_became_peered":"2024-04-25T10:26:13.995365+0000","last_unstale":"2024-04-25T10:26:14.983017+0000","last_undegraded":"2024-04-25T10:26:14.983017+0000","last_fullsized":"2024-04-25T10:26:14.983017+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.730661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650613+0000","last_change":"2024-04-25T10:26:13.999775+0000","last_active":"2024-04-25T10:26:15.650613+0000","last_peered":"2024-04-25T10:26:15.650613+0000","last_clean":"2024-04-25T10:26:15.650613+0000","last_became_active":"2024-04-25T10:26:13.999471+0000","last_became_peered":"2024-04-25T10:26:13.999471+0000","last_unstale":"2024-04-25T10:26:15.650613+0000","last_undegraded":"2024-04-25T10:26:15.650613+0000","last_fullsized":"2024-04-25T10:26:15.650613+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:51.728803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648495+0000","last_change":"2024-04-25T10:26:14.004568+0000","last_active":"2024-04-25T10:26:15.648495+0000","last_peered":"2024-04-25T10:26:15.648495+0000","last_clean":"2024-04-25T10:26:15.648495+0000","last_became_active":"2024-04-25T10:26:14.003306+0000","last_became_peered":"2024-04-25T10:26:14.003306+0000","last_unstale":"2024-04-25T10:26:15.648495+0000","last_undegraded":"2024-04-25T10:26:15.648495+0000","last_fullsized":"2024-04-25T10:26:15.648495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:57.589244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152826+0000","last_change":"2024-04-25T10:26:13.994705+0000","last_active":"2024-04-25T10:26:15.152826+0000","last_peered":"2024-04-25T10:26:15.152826+0000","last_clean":"2024-04-25T10:26:15.152826+0000","last_became_active":"2024-04-25T10:26:13.994412+0000","last_became_peered":"2024-04-25T10:26:13.994412+0000","last_unstale":"2024-04-25T10:26:15.152826+0000","last_undegraded":"2024-04-25T10:26:15.152826+0000","last_fullsized":"2024-04-25T10:26:15.152826+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:42.676705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649545+0000","last_change":"2024-04-25T10:26:14.001323+0000","last_active":"2024-04-25T10:26:15.649545+0000","last_peered":"2024-04-25T10:26:15.649545+0000","last_clean":"2024-04-25T10:26:15.649545+0000","last_became_active":"2024-04-25T10:26:14.000341+0000","last_became_peered":"2024-04-25T10:26:14.000341+0000","last_unstale":"2024-04-25T10:26:15.649545+0000","last_undegraded":"2024-04-25T10:26:15.649545+0000","last_fullsized":"2024-04-25T10:26:15.649545+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:52.176540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649424+0000","last_change":"2024-04-25T10:26:14.004901+0000","last_active":"2024-04-25T10:26:15.649424+0000","last_peered":"2024-04-25T10:26:15.649424+0000","last_clean":"2024-04-25T10:26:15.649424+0000","last_became_active":"2024-04-25T10:26:14.000646+0000","last_became_peered":"2024-04-25T10:26:14.000646+0000","last_unstale":"2024-04-25T10:26:15.649424+0000","last_undegraded":"2024-04-25T10:26:15.649424+0000","last_fullsized":"2024-04-25T10:26:15.649424+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:53.202091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648365+0000","last_change":"2024-04-25T10:26:14.001919+0000","last_active":"2024-04-25T10:26:15.648365+0000","last_peered":"2024-04-25T10:26:15.648365+0000","last_clean":"2024-04-25T10:26:15.648365+0000","last_became_active":"2024-04-25T10:26:14.001643+0000","last_became_peered":"2024-04-25T10:26:14.001643+0000","last_unstale":"2024-04-25T10:26:15.648365+0000","last_undegraded":"2024-04-25T10:26:15.648365+0000","last_fullsized":"2024-04-25T10:26:15.648365+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:46.548053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983099+0000","last_change":"2024-04-25T10:26:13.999078+0000","last_active":"2024-04-25T10:26:14.983099+0000","last_peered":"2024-04-25T10:26:14.983099+0000","last_clean":"2024-04-25T10:26:14.983099+0000","last_became_active":"2024-04-25T10:26:13.998778+0000","last_became_peered":"2024-04-25T10:26:13.998778+0000","last_unstale":"2024-04-25T10:26:14.983099+0000","last_undegraded":"2024-04-25T10:26:14.983099+0000","last_fullsized":"2024-04-25T10:26:14.983099+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:37.186955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648507+0000","last_change":"2024-04-25T10:26:14.006940+0000","last_active":"2024-04-25T10:26:15.648507+0000","last_peered":"2024-04-25T10:26:15.648507+0000","last_clean":"2024-04-25T10:26:15.648507+0000","last_became_active":"2024-04-25T10:26:14.006195+0000","last_became_peered":"2024-04-25T10:26:14.006195+0000","last_unstale":"2024-04-25T10:26:15.648507+0000","last_undegraded":"2024-04-25T10:26:15.648507+0000","last_fullsized":"2024-04-25T10:26:15.648507+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:21.573428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647976+0000","last_change":"2024-04-25T10:26:13.998498+0000","last_active":"2024-04-25T10:26:15.647976+0000","last_peered":"2024-04-25T10:26:15.647976+0000","last_clean":"2024-04-25T10:26:15.647976+0000","last_became_active":"2024-04-25T10:26:13.998242+0000","last_became_peered":"2024-04-25T10:26:13.998242+0000","last_unstale":"2024-04-25T10:26:15.647976+0000","last_undegraded":"2024-04-25T10:26:15.647976+0000","last_fullsized":"2024-04-25T10:26:15.647976+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:58.884962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982816+0000","last_change":"2024-04-25T10:26:13.999686+0000","last_active":"2024-04-25T10:26:14.982816+0000","last_peered":"2024-04-25T10:26:14.982816+0000","last_clean":"2024-04-25T10:26:14.982816+0000","last_became_active":"2024-04-25T10:26:13.999340+0000","last_became_peered":"2024-04-25T10:26:13.999340+0000","last_unstale":"2024-04-25T10:26:14.982816+0000","last_undegraded":"2024-04-25T10:26:14.982816+0000","last_fullsized":"2024-04-25T10:26:14.982816+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:42.097549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985973+0000","last_change":"2024-04-25T10:26:13.984408+0000","last_active":"2024-04-25T10:26:14.985973+0000","last_peered":"2024-04-25T10:26:14.985973+0000","last_clean":"2024-04-25T10:26:14.985973+0000","last_became_active":"2024-04-25T10:26:13.984146+0000","last_became_peered":"2024-04-25T10:26:13.984146+0000","last_unstale":"2024-04-25T10:26:14.985973+0000","last_undegraded":"2024-04-25T10:26:14.985973+0000","last_fullsized":"2024-04-25T10:26:14.985973+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:38.406611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152906+0000","last_change":"2024-04-25T10:26:13.998347+0000","last_active":"2024-04-25T10:26:15.152906+0000","last_peered":"2024-04-25T10:26:15.152906+0000","last_clean":"2024-04-25T10:26:15.152906+0000","last_became_active":"2024-04-25T10:26:13.997933+0000","last_became_peered":"2024-04-25T10:26:13.997933+0000","last_unstale":"2024-04-25T10:26:15.152906+0000","last_undegraded":"2024-04-25T10:26:15.152906+0000","last_fullsized":"2024-04-25T10:26:15.152906+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:50.526006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984718+0000","last_change":"2024-04-25T10:26:13.983856+0000","last_active":"2024-04-25T10:26:14.984718+0000","last_peered":"2024-04-25T10:26:14.984718+0000","last_clean":"2024-04-25T10:26:14.984718+0000","last_became_active":"2024-04-25T10:26:13.983578+0000","last_became_peered":"2024-04-25T10:26:13.983578+0000","last_unstale":"2024-04-25T10:26:14.984718+0000","last_undegraded":"2024-04-25T10:26:14.984718+0000","last_fullsized":"2024-04-25T10:26:14.984718+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:39.025850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983093+0000","last_change":"2024-04-25T10:26:13.998840+0000","last_active":"2024-04-25T10:26:14.983093+0000","last_peered":"2024-04-25T10:26:14.983093+0000","last_clean":"2024-04-25T10:26:14.983093+0000","last_became_active":"2024-04-25T10:26:13.998134+0000","last_became_peered":"2024-04-25T10:26:13.998134+0000","last_unstale":"2024-04-25T10:26:14.983093+0000","last_undegraded":"2024-04-25T10:26:14.983093+0000","last_fullsized":"2024-04-25T10:26:14.983093+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.767357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986701+0000","last_change":"2024-04-25T10:26:13.987446+0000","last_active":"2024-04-25T10:26:14.986701+0000","last_peered":"2024-04-25T10:26:14.986701+0000","last_clean":"2024-04-25T10:26:14.986701+0000","last_became_active":"2024-04-25T10:26:13.986508+0000","last_became_peered":"2024-04-25T10:26:13.986508+0000","last_unstale":"2024-04-25T10:26:14.986701+0000","last_undegraded":"2024-04-25T10:26:14.986701+0000","last_fullsized":"2024-04-25T10:26:14.986701+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:55.173312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985498+0000","last_change":"2024-04-25T10:26:13.990128+0000","last_active":"2024-04-25T10:26:14.985498+0000","last_peered":"2024-04-25T10:26:14.985498+0000","last_clean":"2024-04-25T10:26:14.985498+0000","last_became_active":"2024-04-25T10:26:13.989787+0000","last_became_peered":"2024-04-25T10:26:13.989787+0000","last_unstale":"2024-04-25T10:26:14.985498+0000","last_undegraded":"2024-04-25T10:26:14.985498+0000","last_fullsized":"2024-04-25T10:26:14.985498+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:29.129511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986374+0000","last_change":"2024-04-25T10:26:13.981675+0000","last_active":"2024-04-25T10:26:14.986374+0000","last_peered":"2024-04-25T10:26:14.986374+0000","last_clean":"2024-04-25T10:26:14.986374+0000","last_became_active":"2024-04-25T10:26:13.981362+0000","last_became_peered":"2024-04-25T10:26:13.981362+0000","last_unstale":"2024-04-25T10:26:14.986374+0000","last_undegraded":"2024-04-25T10:26:14.986374+0000","last_fullsized":"2024-04-25T10:26:14.986374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:49.074238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153217+0000","last_change":"2024-04-25T10:26:13.990284+0000","last_active":"2024-04-25T10:26:15.153217+0000","last_peered":"2024-04-25T10:26:15.153217+0000","last_clean":"2024-04-25T10:26:15.153217+0000","last_became_active":"2024-04-25T10:26:13.989862+0000","last_became_peered":"2024-04-25T10:26:13.989862+0000","last_unstale":"2024-04-25T10:26:15.153217+0000","last_undegraded":"2024-04-25T10:26:15.153217+0000","last_fullsized":"2024-04-25T10:26:15.153217+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:29.655266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982619+0000","last_change":"2024-04-25T10:26:13.981364+0000","last_active":"2024-04-25T10:26:14.982619+0000","last_peered":"2024-04-25T10:26:14.982619+0000","last_clean":"2024-04-25T10:26:14.982619+0000","last_became_active":"2024-04-25T10:26:13.981057+0000","last_became_peered":"2024-04-25T10:26:13.981057+0000","last_unstale":"2024-04-25T10:26:14.982619+0000","last_undegraded":"2024-04-25T10:26:14.982619+0000","last_fullsized":"2024-04-25T10:26:14.982619+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:53.702981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650126+0000","last_change":"2024-04-25T10:26:13.997324+0000","last_active":"2024-04-25T10:26:15.650126+0000","last_peered":"2024-04-25T10:26:15.650126+0000","last_clean":"2024-04-25T10:26:15.650126+0000","last_became_active":"2024-04-25T10:26:13.994454+0000","last_became_peered":"2024-04-25T10:26:13.994454+0000","last_unstale":"2024-04-25T10:26:15.650126+0000","last_undegraded":"2024-04-25T10:26:15.650126+0000","last_fullsized":"2024-04-25T10:26:15.650126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:21.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649606+0000","last_change":"2024-04-25T10:26:14.005110+0000","last_active":"2024-04-25T10:26:15.649606+0000","last_peered":"2024-04-25T10:26:15.649606+0000","last_clean":"2024-04-25T10:26:15.649606+0000","last_became_active":"2024-04-25T10:26:14.000955+0000","last_became_peered":"2024-04-25T10:26:14.000955+0000","last_unstale":"2024-04-25T10:26:15.649606+0000","last_undegraded":"2024-04-25T10:26:15.649606+0000","last_fullsized":"2024-04-25T10:26:15.649606+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:54.738625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649983+0000","last_change":"2024-04-25T10:26:13.997490+0000","last_active":"2024-04-25T10:26:15.649983+0000","last_peered":"2024-04-25T10:26:15.649983+0000","last_clean":"2024-04-25T10:26:15.649983+0000","last_became_active":"2024-04-25T10:26:13.997153+0000","last_became_peered":"2024-04-25T10:26:13.997153+0000","last_unstale":"2024-04-25T10:26:15.649983+0000","last_undegraded":"2024-04-25T10:26:15.649983+0000","last_fullsized":"2024-04-25T10:26:15.649983+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:58.035839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"116'4","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153175+0000","last_change":"2024-04-25T10:26:12.973421+0000","last_active":"2024-04-25T10:26:15.153175+0000","last_peered":"2024-04-25T10:26:15.153175+0000","last_clean":"2024-04-25T10:26:15.153175+0000","last_became_active":"2024-04-25T10:26:12.973108+0000","last_became_peered":"2024-04-25T10:26:12.973108+0000","last_unstale":"2024-04-25T10:26:15.153175+0000","last_undegraded":"2024-04-25T10:26:15.153175+0000","last_fullsized":"2024-04-25T10:26:15.153175+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:35.304001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153148+0000","last_change":"2024-04-25T10:26:14.005677+0000","last_active":"2024-04-25T10:26:15.153148+0000","last_peered":"2024-04-25T10:26:15.153148+0000","last_clean":"2024-04-25T10:26:15.153148+0000","last_became_active":"2024-04-25T10:26:14.005373+0000","last_became_peered":"2024-04-25T10:26:14.005373+0000","last_unstale":"2024-04-25T10:26:15.153148+0000","last_undegraded":"2024-04-25T10:26:15.153148+0000","last_fullsized":"2024-04-25T10:26:15.153148+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:29.194434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648710+0000","last_change":"2024-04-25T10:26:12.965663+0000","last_active":"2024-04-25T10:26:15.648710+0000","last_peered":"2024-04-25T10:26:15.648710+0000","last_clean":"2024-04-25T10:26:15.648710+0000","last_became_active":"2024-04-25T10:26:12.965358+0000","last_became_peered":"2024-04-25T10:26:12.965358+0000","last_unstale":"2024-04-25T10:26:15.648710+0000","last_undegraded":"2024-04-25T10:26:15.648710+0000","last_fullsized":"2024-04-25T10:26:15.648710+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.527361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986627+0000","last_change":"2024-04-25T10:26:13.990794+0000","last_active":"2024-04-25T10:26:14.986627+0000","last_peered":"2024-04-25T10:26:14.986627+0000","last_clean":"2024-04-25T10:26:14.986627+0000","last_became_active":"2024-04-25T10:26:13.990456+0000","last_became_peered":"2024-04-25T10:26:13.990456+0000","last_unstale":"2024-04-25T10:26:14.986627+0000","last_undegraded":"2024-04-25T10:26:14.986627+0000","last_fullsized":"2024-04-25T10:26:14.986627+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:18.806868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985375+0000","last_change":"2024-04-25T10:26:12.978335+0000","last_active":"2024-04-25T10:26:14.985375+0000","last_peered":"2024-04-25T10:26:14.985375+0000","last_clean":"2024-04-25T10:26:14.985375+0000","last_became_active":"2024-04-25T10:26:12.978084+0000","last_became_peered":"2024-04-25T10:26:12.978084+0000","last_unstale":"2024-04-25T10:26:14.985375+0000","last_undegraded":"2024-04-25T10:26:14.985375+0000","last_fullsized":"2024-04-25T10:26:14.985375+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:06.204252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648153+0000","last_change":"2024-04-25T10:26:14.007186+0000","last_active":"2024-04-25T10:26:15.648153+0000","last_peered":"2024-04-25T10:26:15.648153+0000","last_clean":"2024-04-25T10:26:15.648153+0000","last_became_active":"2024-04-25T10:26:14.006656+0000","last_became_peered":"2024-04-25T10:26:14.006656+0000","last_unstale":"2024-04-25T10:26:15.648153+0000","last_undegraded":"2024-04-25T10:26:15.648153+0000","last_fullsized":"2024-04-25T10:26:15.648153+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:53.202091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985141+0000","last_change":"2024-04-25T10:26:12.976081+0000","last_active":"2024-04-25T10:26:14.985141+0000","last_peered":"2024-04-25T10:26:14.985141+0000","last_clean":"2024-04-25T10:26:14.985141+0000","last_became_active":"2024-04-25T10:26:12.974108+0000","last_became_peered":"2024-04-25T10:26:12.974108+0000","last_unstale":"2024-04-25T10:26:14.985141+0000","last_undegraded":"2024-04-25T10:26:14.985141+0000","last_fullsized":"2024-04-25T10:26:14.985141+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:44.197112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984990+0000","last_change":"2024-04-25T10:26:13.990888+0000","last_active":"2024-04-25T10:26:14.984990+0000","last_peered":"2024-04-25T10:26:14.984990+0000","last_clean":"2024-04-25T10:26:14.984990+0000","last_became_active":"2024-04-25T10:26:13.990631+0000","last_became_peered":"2024-04-25T10:26:13.990631+0000","last_unstale":"2024-04-25T10:26:14.984990+0000","last_undegraded":"2024-04-25T10:26:14.984990+0000","last_fullsized":"2024-04-25T10:26:14.984990+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:11.417723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647878+0000","last_change":"2024-04-25T10:26:12.969937+0000","last_active":"2024-04-25T10:26:15.647878+0000","last_peered":"2024-04-25T10:26:15.647878+0000","last_clean":"2024-04-25T10:26:15.647878+0000","last_became_active":"2024-04-25T10:26:12.969666+0000","last_became_peered":"2024-04-25T10:26:12.969666+0000","last_unstale":"2024-04-25T10:26:15.647878+0000","last_undegraded":"2024-04-25T10:26:15.647878+0000","last_fullsized":"2024-04-25T10:26:15.647878+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:23.925941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982622+0000","last_change":"2024-04-25T10:26:14.004973+0000","last_active":"2024-04-25T10:26:14.982622+0000","last_peered":"2024-04-25T10:26:14.982622+0000","last_clean":"2024-04-25T10:26:14.982622+0000","last_became_active":"2024-04-25T10:26:14.004702+0000","last_became_peered":"2024-04-25T10:26:14.004702+0000","last_unstale":"2024-04-25T10:26:14.982622+0000","last_undegraded":"2024-04-25T10:26:14.982622+0000","last_fullsized":"2024-04-25T10:26:14.982622+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:46.640917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649467+0000","last_change":"2024-04-25T10:26:12.968858+0000","last_active":"2024-04-25T10:26:15.649467+0000","last_peered":"2024-04-25T10:26:15.649467+0000","last_clean":"2024-04-25T10:26:15.649467+0000","last_became_active":"2024-04-25T10:26:12.968309+0000","last_became_peered":"2024-04-25T10:26:12.968309+0000","last_unstale":"2024-04-25T10:26:15.649467+0000","last_undegraded":"2024-04-25T10:26:15.649467+0000","last_fullsized":"2024-04-25T10:26:15.649467+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:41.670389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649771+0000","last_change":"2024-04-25T10:26:14.005318+0000","last_active":"2024-04-25T10:26:15.649771+0000","last_peered":"2024-04-25T10:26:15.649771+0000","last_clean":"2024-04-25T10:26:15.649771+0000","last_became_active":"2024-04-25T10:26:14.004635+0000","last_became_peered":"2024-04-25T10:26:14.004635+0000","last_unstale":"2024-04-25T10:26:15.649771+0000","last_undegraded":"2024-04-25T10:26:15.649771+0000","last_fullsized":"2024-04-25T10:26:15.649771+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:46.548053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982342+0000","last_change":"2024-04-25T10:26:12.968507+0000","last_active":"2024-04-25T10:26:14.982342+0000","last_peered":"2024-04-25T10:26:14.982342+0000","last_clean":"2024-04-25T10:26:14.982342+0000","last_became_active":"2024-04-25T10:26:12.967779+0000","last_became_peered":"2024-04-25T10:26:12.967779+0000","last_unstale":"2024-04-25T10:26:14.982342+0000","last_undegraded":"2024-04-25T10:26:14.982342+0000","last_fullsized":"2024-04-25T10:26:14.982342+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:21.393537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982224+0000","last_change":"2024-04-25T10:26:13.990544+0000","last_active":"2024-04-25T10:26:14.982224+0000","last_peered":"2024-04-25T10:26:14.982224+0000","last_clean":"2024-04-25T10:26:14.982224+0000","last_became_active":"2024-04-25T10:26:13.990245+0000","last_became_peered":"2024-04-25T10:26:13.990245+0000","last_unstale":"2024-04-25T10:26:14.982224+0000","last_undegraded":"2024-04-25T10:26:14.982224+0000","last_fullsized":"2024-04-25T10:26:14.982224+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:42.229525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649869+0000","last_change":"2024-04-25T10:26:12.974852+0000","last_active":"2024-04-25T10:26:15.649869+0000","last_peered":"2024-04-25T10:26:15.649869+0000","last_clean":"2024-04-25T10:26:15.649869+0000","last_became_active":"2024-04-25T10:26:12.974599+0000","last_became_peered":"2024-04-25T10:26:12.974599+0000","last_unstale":"2024-04-25T10:26:15.649869+0000","last_undegraded":"2024-04-25T10:26:15.649869+0000","last_fullsized":"2024-04-25T10:26:15.649869+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:23.925941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648639+0000","last_change":"2024-04-25T10:26:13.999919+0000","last_active":"2024-04-25T10:26:15.648639+0000","last_peered":"2024-04-25T10:26:15.648639+0000","last_clean":"2024-04-25T10:26:15.648639+0000","last_became_active":"2024-04-25T10:26:13.999552+0000","last_became_peered":"2024-04-25T10:26:13.999552+0000","last_unstale":"2024-04-25T10:26:15.648639+0000","last_undegraded":"2024-04-25T10:26:15.648639+0000","last_fullsized":"2024-04-25T10:26:15.648639+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:21.402101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986116+0000","last_change":"2024-04-25T10:26:12.978849+0000","last_active":"2024-04-25T10:26:14.986116+0000","last_peered":"2024-04-25T10:26:14.986116+0000","last_clean":"2024-04-25T10:26:14.986116+0000","last_became_active":"2024-04-25T10:26:12.978452+0000","last_became_peered":"2024-04-25T10:26:12.978452+0000","last_unstale":"2024-04-25T10:26:14.986116+0000","last_undegraded":"2024-04-25T10:26:14.986116+0000","last_fullsized":"2024-04-25T10:26:14.986116+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:28.709682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648911+0000","last_change":"2024-04-25T10:26:14.004253+0000","last_active":"2024-04-25T10:26:15.648911+0000","last_peered":"2024-04-25T10:26:15.648911+0000","last_clean":"2024-04-25T10:26:15.648911+0000","last_became_active":"2024-04-25T10:26:14.003964+0000","last_became_peered":"2024-04-25T10:26:14.003964+0000","last_unstale":"2024-04-25T10:26:15.648911+0000","last_undegraded":"2024-04-25T10:26:15.648911+0000","last_fullsized":"2024-04-25T10:26:15.648911+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:03.157961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650719+0000","last_change":"2024-04-25T10:26:12.974260+0000","last_active":"2024-04-25T10:26:15.650719+0000","last_peered":"2024-04-25T10:26:15.650719+0000","last_clean":"2024-04-25T10:26:15.650719+0000","last_became_active":"2024-04-25T10:26:12.973967+0000","last_became_peered":"2024-04-25T10:26:12.973967+0000","last_unstale":"2024-04-25T10:26:15.650719+0000","last_undegraded":"2024-04-25T10:26:15.650719+0000","last_fullsized":"2024-04-25T10:26:15.650719+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:14.890002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986599+0000","last_change":"2024-04-25T10:26:13.978742+0000","last_active":"2024-04-25T10:26:14.986599+0000","last_peered":"2024-04-25T10:26:14.986599+0000","last_clean":"2024-04-25T10:26:14.986599+0000","last_became_active":"2024-04-25T10:26:13.978427+0000","last_became_peered":"2024-04-25T10:26:13.978427+0000","last_unstale":"2024-04-25T10:26:14.986599+0000","last_undegraded":"2024-04-25T10:26:14.986599+0000","last_fullsized":"2024-04-25T10:26:14.986599+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.104485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"116'3","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650636+0000","last_change":"2024-04-25T10:26:12.979707+0000","last_active":"2024-04-25T10:26:15.650636+0000","last_peered":"2024-04-25T10:26:15.650636+0000","last_clean":"2024-04-25T10:26:15.650636+0000","last_became_active":"2024-04-25T10:26:12.979419+0000","last_became_peered":"2024-04-25T10:26:12.979419+0000","last_unstale":"2024-04-25T10:26:15.650636+0000","last_undegraded":"2024-04-25T10:26:15.650636+0000","last_fullsized":"2024-04-25T10:26:15.650636+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.116774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983049+0000","last_change":"2024-04-25T10:26:13.995636+0000","last_active":"2024-04-25T10:26:14.983049+0000","last_peered":"2024-04-25T10:26:14.983049+0000","last_clean":"2024-04-25T10:26:14.983049+0000","last_became_active":"2024-04-25T10:26:13.994939+0000","last_became_peered":"2024-04-25T10:26:13.994939+0000","last_unstale":"2024-04-25T10:26:14.983049+0000","last_undegraded":"2024-04-25T10:26:14.983049+0000","last_fullsized":"2024-04-25T10:26:14.983049+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:54.463095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"116'6","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985266+0000","last_change":"2024-04-25T10:26:12.974365+0000","last_active":"2024-04-25T10:26:14.985266+0000","last_peered":"2024-04-25T10:26:14.985266+0000","last_clean":"2024-04-25T10:26:14.985266+0000","last_became_active":"2024-04-25T10:26:12.973427+0000","last_became_peered":"2024-04-25T10:26:12.973427+0000","last_unstale":"2024-04-25T10:26:14.985266+0000","last_undegraded":"2024-04-25T10:26:14.985266+0000","last_fullsized":"2024-04-25T10:26:14.985266+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:06.072227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650024+0000","last_change":"2024-04-25T10:26:13.999800+0000","last_active":"2024-04-25T10:26:15.650024+0000","last_peered":"2024-04-25T10:26:15.650024+0000","last_clean":"2024-04-25T10:26:15.650024+0000","last_became_active":"2024-04-25T10:26:13.999506+0000","last_became_peered":"2024-04-25T10:26:13.999506+0000","last_unstale":"2024-04-25T10:26:15.650024+0000","last_undegraded":"2024-04-25T10:26:15.650024+0000","last_fullsized":"2024-04-25T10:26:15.650024+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:28.280963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650411+0000","last_change":"2024-04-25T10:26:12.974720+0000","last_active":"2024-04-25T10:26:15.650411+0000","last_peered":"2024-04-25T10:26:15.650411+0000","last_clean":"2024-04-25T10:26:15.650411+0000","last_became_active":"2024-04-25T10:26:12.974006+0000","last_became_peered":"2024-04-25T10:26:12.974006+0000","last_unstale":"2024-04-25T10:26:15.650411+0000","last_undegraded":"2024-04-25T10:26:15.650411+0000","last_fullsized":"2024-04-25T10:26:15.650411+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:45.691111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985040+0000","last_change":"2024-04-25T10:26:13.994164+0000","last_active":"2024-04-25T10:26:14.985040+0000","last_peered":"2024-04-25T10:26:14.985040+0000","last_clean":"2024-04-25T10:26:14.985040+0000","last_became_active":"2024-04-25T10:26:13.993915+0000","last_became_peered":"2024-04-25T10:26:13.993915+0000","last_unstale":"2024-04-25T10:26:14.985040+0000","last_undegraded":"2024-04-25T10:26:14.985040+0000","last_fullsized":"2024-04-25T10:26:14.985040+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.294953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647768+0000","last_change":"2024-04-25T10:26:12.970303+0000","last_active":"2024-04-25T10:26:15.647768+0000","last_peered":"2024-04-25T10:26:15.647768+0000","last_clean":"2024-04-25T10:26:15.647768+0000","last_became_active":"2024-04-25T10:26:12.969076+0000","last_became_peered":"2024-04-25T10:26:12.969076+0000","last_unstale":"2024-04-25T10:26:15.647768+0000","last_undegraded":"2024-04-25T10:26:15.647768+0000","last_fullsized":"2024-04-25T10:26:15.647768+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985187+0000","last_change":"2024-04-25T10:26:13.991851+0000","last_active":"2024-04-25T10:26:14.985187+0000","last_peered":"2024-04-25T10:26:14.985187+0000","last_clean":"2024-04-25T10:26:14.985187+0000","last_became_active":"2024-04-25T10:26:13.991588+0000","last_became_peered":"2024-04-25T10:26:13.991588+0000","last_unstale":"2024-04-25T10:26:14.985187+0000","last_undegraded":"2024-04-25T10:26:14.985187+0000","last_fullsized":"2024-04-25T10:26:14.985187+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:19.981211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"116'8","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649347+0000","last_change":"2024-04-25T10:26:12.970708+0000","last_active":"2024-04-25T10:26:15.649347+0000","last_peered":"2024-04-25T10:26:15.649347+0000","last_clean":"2024-04-25T10:26:15.649347+0000","last_became_active":"2024-04-25T10:26:12.970360+0000","last_became_peered":"2024-04-25T10:26:12.970360+0000","last_unstale":"2024-04-25T10:26:15.649347+0000","last_undegraded":"2024-04-25T10:26:15.649347+0000","last_fullsized":"2024-04-25T10:26:15.649347+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:49.519690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649898+0000","last_change":"2024-04-25T10:26:13.994489+0000","last_active":"2024-04-25T10:26:15.649898+0000","last_peered":"2024-04-25T10:26:15.649898+0000","last_clean":"2024-04-25T10:26:15.649898+0000","last_became_active":"2024-04-25T10:26:13.994104+0000","last_became_peered":"2024-04-25T10:26:13.994104+0000","last_unstale":"2024-04-25T10:26:15.649898+0000","last_undegraded":"2024-04-25T10:26:15.649898+0000","last_fullsized":"2024-04-25T10:26:15.649898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:27.574489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986494+0000","last_change":"2024-04-25T10:26:12.972335+0000","last_active":"2024-04-25T10:26:14.986494+0000","last_peered":"2024-04-25T10:26:14.986494+0000","last_clean":"2024-04-25T10:26:14.986494+0000","last_became_active":"2024-04-25T10:26:12.972062+0000","last_became_peered":"2024-04-25T10:26:12.972062+0000","last_unstale":"2024-04-25T10:26:14.986494+0000","last_undegraded":"2024-04-25T10:26:14.986494+0000","last_fullsized":"2024-04-25T10:26:14.986494+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:52.946736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650136+0000","last_change":"2024-04-25T10:26:13.997811+0000","last_active":"2024-04-25T10:26:15.650136+0000","last_peered":"2024-04-25T10:26:15.650136+0000","last_clean":"2024-04-25T10:26:15.650136+0000","last_became_active":"2024-04-25T10:26:13.997537+0000","last_became_peered":"2024-04-25T10:26:13.997537+0000","last_unstale":"2024-04-25T10:26:15.650136+0000","last_undegraded":"2024-04-25T10:26:15.650136+0000","last_fullsized":"2024-04-25T10:26:15.650136+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:03.157961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650828+0000","last_change":"2024-04-25T10:26:12.974012+0000","last_active":"2024-04-25T10:26:15.650828+0000","last_peered":"2024-04-25T10:26:15.650828+0000","last_clean":"2024-04-25T10:26:15.650828+0000","last_became_active":"2024-04-25T10:26:12.973735+0000","last_became_peered":"2024-04-25T10:26:12.973735+0000","last_unstale":"2024-04-25T10:26:15.650828+0000","last_undegraded":"2024-04-25T10:26:15.650828+0000","last_fullsized":"2024-04-25T10:26:15.650828+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:38.772989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982858+0000","last_change":"2024-04-25T10:26:13.988908+0000","last_active":"2024-04-25T10:26:14.982858+0000","last_peered":"2024-04-25T10:26:14.982858+0000","last_clean":"2024-04-25T10:26:14.982858+0000","last_became_active":"2024-04-25T10:26:13.988614+0000","last_became_peered":"2024-04-25T10:26:13.988614+0000","last_unstale":"2024-04-25T10:26:14.982858+0000","last_undegraded":"2024-04-25T10:26:14.982858+0000","last_fullsized":"2024-04-25T10:26:14.982858+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:34.598708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"116'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153197+0000","last_change":"2024-04-25T10:26:12.972502+0000","last_active":"2024-04-25T10:26:15.153197+0000","last_peered":"2024-04-25T10:26:15.153197+0000","last_clean":"2024-04-25T10:26:15.153197+0000","last_became_active":"2024-04-25T10:26:12.972135+0000","last_became_peered":"2024-04-25T10:26:12.972135+0000","last_unstale":"2024-04-25T10:26:15.153197+0000","last_undegraded":"2024-04-25T10:26:15.153197+0000","last_fullsized":"2024-04-25T10:26:15.153197+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:41.952494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153092+0000","last_change":"2024-04-25T10:26:13.996926+0000","last_active":"2024-04-25T10:26:15.153092+0000","last_peered":"2024-04-25T10:26:15.153092+0000","last_clean":"2024-04-25T10:26:15.153092+0000","last_became_active":"2024-04-25T10:26:13.996624+0000","last_became_peered":"2024-04-25T10:26:13.996624+0000","last_unstale":"2024-04-25T10:26:15.153092+0000","last_undegraded":"2024-04-25T10:26:15.153092+0000","last_fullsized":"2024-04-25T10:26:15.153092+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:55.901875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152833+0000","last_change":"2024-04-25T10:26:12.975104+0000","last_active":"2024-04-25T10:26:15.152833+0000","last_peered":"2024-04-25T10:26:15.152833+0000","last_clean":"2024-04-25T10:26:15.152833+0000","last_became_active":"2024-04-25T10:26:12.974542+0000","last_became_peered":"2024-04-25T10:26:12.974542+0000","last_unstale":"2024-04-25T10:26:15.152833+0000","last_undegraded":"2024-04-25T10:26:15.152833+0000","last_fullsized":"2024-04-25T10:26:15.152833+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:10.161208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647644+0000","last_change":"2024-04-25T10:26:13.997176+0000","last_active":"2024-04-25T10:26:15.647644+0000","last_peered":"2024-04-25T10:26:15.647644+0000","last_clean":"2024-04-25T10:26:15.647644+0000","last_became_active":"2024-04-25T10:26:13.996632+0000","last_became_peered":"2024-04-25T10:26:13.996632+0000","last_unstale":"2024-04-25T10:26:15.647644+0000","last_undegraded":"2024-04-25T10:26:15.647644+0000","last_fullsized":"2024-04-25T10:26:15.647644+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.310317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650595+0000","last_change":"2024-04-25T10:26:12.974951+0000","last_active":"2024-04-25T10:26:15.650595+0000","last_peered":"2024-04-25T10:26:15.650595+0000","last_clean":"2024-04-25T10:26:15.650595+0000","last_became_active":"2024-04-25T10:26:12.974439+0000","last_became_peered":"2024-04-25T10:26:12.974439+0000","last_unstale":"2024-04-25T10:26:15.650595+0000","last_undegraded":"2024-04-25T10:26:15.650595+0000","last_fullsized":"2024-04-25T10:26:15.650595+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.324740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649896+0000","last_change":"2024-04-25T10:26:14.001676+0000","last_active":"2024-04-25T10:26:15.649896+0000","last_peered":"2024-04-25T10:26:15.649896+0000","last_clean":"2024-04-25T10:26:15.649896+0000","last_became_active":"2024-04-25T10:26:14.000901+0000","last_became_peered":"2024-04-25T10:26:14.000901+0000","last_unstale":"2024-04-25T10:26:15.649896+0000","last_undegraded":"2024-04-25T10:26:15.649896+0000","last_fullsized":"2024-04-25T10:26:15.649896+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:38.117447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649702+0000","last_change":"2024-04-25T10:26:12.971625+0000","last_active":"2024-04-25T10:26:15.649702+0000","last_peered":"2024-04-25T10:26:15.649702+0000","last_clean":"2024-04-25T10:26:15.649702+0000","last_became_active":"2024-04-25T10:26:12.971364+0000","last_became_peered":"2024-04-25T10:26:12.971364+0000","last_unstale":"2024-04-25T10:26:15.649702+0000","last_undegraded":"2024-04-25T10:26:15.649702+0000","last_fullsized":"2024-04-25T10:26:15.649702+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:23.625246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152776+0000","last_change":"2024-04-25T10:26:13.999968+0000","last_active":"2024-04-25T10:26:15.152776+0000","last_peered":"2024-04-25T10:26:15.152776+0000","last_clean":"2024-04-25T10:26:15.152776+0000","last_became_active":"2024-04-25T10:26:13.999689+0000","last_became_peered":"2024-04-25T10:26:13.999689+0000","last_unstale":"2024-04-25T10:26:15.152776+0000","last_undegraded":"2024-04-25T10:26:15.152776+0000","last_fullsized":"2024-04-25T10:26:15.152776+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:14.641929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984863+0000","last_change":"2024-04-25T10:26:12.973236+0000","last_active":"2024-04-25T10:26:14.984863+0000","last_peered":"2024-04-25T10:26:14.984863+0000","last_clean":"2024-04-25T10:26:14.984863+0000","last_became_active":"2024-04-25T10:26:12.972387+0000","last_became_peered":"2024-04-25T10:26:12.972387+0000","last_unstale":"2024-04-25T10:26:14.984863+0000","last_undegraded":"2024-04-25T10:26:14.984863+0000","last_fullsized":"2024-04-25T10:26:14.984863+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:43.977648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649781+0000","last_change":"2024-04-25T10:26:13.997607+0000","last_active":"2024-04-25T10:26:15.649781+0000","last_peered":"2024-04-25T10:26:15.649781+0000","last_clean":"2024-04-25T10:26:15.649781+0000","last_became_active":"2024-04-25T10:26:13.997331+0000","last_became_peered":"2024-04-25T10:26:13.997331+0000","last_unstale":"2024-04-25T10:26:15.649781+0000","last_undegraded":"2024-04-25T10:26:15.649781+0000","last_fullsized":"2024-04-25T10:26:15.649781+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:21.573428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984822+0000","last_change":"2024-04-25T10:26:12.970548+0000","last_active":"2024-04-25T10:26:14.984822+0000","last_peered":"2024-04-25T10:26:14.984822+0000","last_clean":"2024-04-25T10:26:14.984822+0000","last_became_active":"2024-04-25T10:26:12.970247+0000","last_became_peered":"2024-04-25T10:26:12.970247+0000","last_unstale":"2024-04-25T10:26:14.984822+0000","last_undegraded":"2024-04-25T10:26:14.984822+0000","last_fullsized":"2024-04-25T10:26:14.984822+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.774633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153074+0000","last_change":"2024-04-25T10:26:13.998614+0000","last_active":"2024-04-25T10:26:15.153074+0000","last_peered":"2024-04-25T10:26:15.153074+0000","last_clean":"2024-04-25T10:26:15.153074+0000","last_became_active":"2024-04-25T10:26:13.998313+0000","last_became_peered":"2024-04-25T10:26:13.998313+0000","last_unstale":"2024-04-25T10:26:15.153074+0000","last_undegraded":"2024-04-25T10:26:15.153074+0000","last_fullsized":"2024-04-25T10:26:15.153074+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:34.370206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"116'2","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986171+0000","last_change":"2024-04-25T10:26:12.978347+0000","last_active":"2024-04-25T10:26:14.986171+0000","last_peered":"2024-04-25T10:26:14.986171+0000","last_clean":"2024-04-25T10:26:14.986171+0000","last_became_active":"2024-04-25T10:26:12.977701+0000","last_became_peered":"2024-04-25T10:26:12.977701+0000","last_unstale":"2024-04-25T10:26:14.986171+0000","last_undegraded":"2024-04-25T10:26:14.986171+0000","last_fullsized":"2024-04-25T10:26:14.986171+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:39.355250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.986517+0000","last_change":"2024-04-25T10:26:13.984186+0000","last_active":"2024-04-25T10:26:14.986517+0000","last_peered":"2024-04-25T10:26:14.986517+0000","last_clean":"2024-04-25T10:26:14.986517+0000","last_became_active":"2024-04-25T10:26:13.983911+0000","last_became_peered":"2024-04-25T10:26:13.983911+0000","last_unstale":"2024-04-25T10:26:14.986517+0000","last_undegraded":"2024-04-25T10:26:14.986517+0000","last_fullsized":"2024-04-25T10:26:14.986517+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.558024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"117'1","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.983099+0000","last_change":"2024-04-25T10:26:12.972745+0000","last_active":"2024-04-25T10:26:14.983099+0000","last_peered":"2024-04-25T10:26:14.983099+0000","last_clean":"2024-04-25T10:26:14.983099+0000","last_became_active":"2024-04-25T10:26:12.972443+0000","last_became_peered":"2024-04-25T10:26:12.972443+0000","last_unstale":"2024-04-25T10:26:14.983099+0000","last_undegraded":"2024-04-25T10:26:14.983099+0000","last_fullsized":"2024-04-25T10:26:14.983099+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:03.914613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.649797+0000","last_change":"2024-04-25T10:26:14.001199+0000","last_active":"2024-04-25T10:26:15.649797+0000","last_peered":"2024-04-25T10:26:15.649797+0000","last_clean":"2024-04-25T10:26:15.649797+0000","last_became_active":"2024-04-25T10:26:14.000919+0000","last_became_peered":"2024-04-25T10:26:14.000919+0000","last_unstale":"2024-04-25T10:26:15.649797+0000","last_undegraded":"2024-04-25T10:26:15.649797+0000","last_fullsized":"2024-04-25T10:26:15.649797+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:01.962864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985492+0000","last_change":"2024-04-25T10:26:12.978046+0000","last_active":"2024-04-25T10:26:14.985492+0000","last_peered":"2024-04-25T10:26:14.985492+0000","last_clean":"2024-04-25T10:26:14.985492+0000","last_became_active":"2024-04-25T10:26:12.977677+0000","last_became_peered":"2024-04-25T10:26:12.977677+0000","last_unstale":"2024-04-25T10:26:14.985492+0000","last_undegraded":"2024-04-25T10:26:14.985492+0000","last_fullsized":"2024-04-25T10:26:14.985492+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:44.914361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.153139+0000","last_change":"2024-04-25T10:26:14.010212+0000","last_active":"2024-04-25T10:26:15.153139+0000","last_peered":"2024-04-25T10:26:15.153139+0000","last_clean":"2024-04-25T10:26:15.153139+0000","last_became_active":"2024-04-25T10:26:14.009871+0000","last_became_peered":"2024-04-25T10:26:14.009871+0000","last_unstale":"2024-04-25T10:26:15.153139+0000","last_undegraded":"2024-04-25T10:26:15.153139+0000","last_fullsized":"2024-04-25T10:26:15.153139+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:16.795629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985277+0000","last_change":"2024-04-25T10:26:12.975861+0000","last_active":"2024-04-25T10:26:14.985277+0000","last_peered":"2024-04-25T10:26:14.985277+0000","last_clean":"2024-04-25T10:26:14.985277+0000","last_became_active":"2024-04-25T10:26:12.973756+0000","last_became_peered":"2024-04-25T10:26:12.973756+0000","last_unstale":"2024-04-25T10:26:14.985277+0000","last_undegraded":"2024-04-25T10:26:14.985277+0000","last_fullsized":"2024-04-25T10:26:14.985277+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:26.815973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647679+0000","last_change":"2024-04-25T10:26:13.997323+0000","last_active":"2024-04-25T10:26:15.647679+0000","last_peered":"2024-04-25T10:26:15.647679+0000","last_clean":"2024-04-25T10:26:15.647679+0000","last_became_active":"2024-04-25T10:26:13.997023+0000","last_became_peered":"2024-04-25T10:26:13.997023+0000","last_unstale":"2024-04-25T10:26:15.647679+0000","last_undegraded":"2024-04-25T10:26:15.647679+0000","last_fullsized":"2024-04-25T10:26:15.647679+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:18.512773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.982768+0000","last_change":"2024-04-25T10:26:12.965357+0000","last_active":"2024-04-25T10:26:14.982768+0000","last_peered":"2024-04-25T10:26:14.982768+0000","last_clean":"2024-04-25T10:26:14.982768+0000","last_became_active":"2024-04-25T10:26:12.964963+0000","last_became_peered":"2024-04-25T10:26:12.964963+0000","last_unstale":"2024-04-25T10:26:14.982768+0000","last_undegraded":"2024-04-25T10:26:14.982768+0000","last_fullsized":"2024-04-25T10:26:14.982768+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:56.157663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152954+0000","last_change":"2024-04-25T10:26:14.003694+0000","last_active":"2024-04-25T10:26:15.152954+0000","last_peered":"2024-04-25T10:26:15.152954+0000","last_clean":"2024-04-25T10:26:15.152954+0000","last_became_active":"2024-04-25T10:26:14.002353+0000","last_became_peered":"2024-04-25T10:26:14.002353+0000","last_unstale":"2024-04-25T10:26:15.152954+0000","last_undegraded":"2024-04-25T10:26:15.152954+0000","last_fullsized":"2024-04-25T10:26:15.152954+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:24.631561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152900+0000","last_change":"2024-04-25T10:26:12.968833+0000","last_active":"2024-04-25T10:26:15.152900+0000","last_peered":"2024-04-25T10:26:15.152900+0000","last_clean":"2024-04-25T10:26:15.152900+0000","last_became_active":"2024-04-25T10:26:12.968544+0000","last_became_peered":"2024-04-25T10:26:12.968544+0000","last_unstale":"2024-04-25T10:26:15.152900+0000","last_undegraded":"2024-04-25T10:26:15.152900+0000","last_fullsized":"2024-04-25T10:26:15.152900+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:08.010695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.152855+0000","last_change":"2024-04-25T10:26:13.999839+0000","last_active":"2024-04-25T10:26:15.152855+0000","last_peered":"2024-04-25T10:26:15.152855+0000","last_clean":"2024-04-25T10:26:15.152855+0000","last_became_active":"2024-04-25T10:26:13.997251+0000","last_became_peered":"2024-04-25T10:26:13.997251+0000","last_unstale":"2024-04-25T10:26:15.152855+0000","last_undegraded":"2024-04-25T10:26:15.152855+0000","last_fullsized":"2024-04-25T10:26:15.152855+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:39.559343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.650192+0000","last_change":"2024-04-25T10:26:12.970872+0000","last_active":"2024-04-25T10:26:15.650192+0000","last_peered":"2024-04-25T10:26:15.650192+0000","last_clean":"2024-04-25T10:26:15.650192+0000","last_became_active":"2024-04-25T10:26:12.970594+0000","last_became_peered":"2024-04-25T10:26:12.970594+0000","last_unstale":"2024-04-25T10:26:15.650192+0000","last_undegraded":"2024-04-25T10:26:15.650192+0000","last_fullsized":"2024-04-25T10:26:15.650192+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:38.553027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985727+0000","last_change":"2024-04-25T10:26:13.995296+0000","last_active":"2024-04-25T10:26:14.985727+0000","last_peered":"2024-04-25T10:26:14.985727+0000","last_clean":"2024-04-25T10:26:14.985727+0000","last_became_active":"2024-04-25T10:26:13.995024+0000","last_became_peered":"2024-04-25T10:26:13.995024+0000","last_unstale":"2024-04-25T10:26:14.985727+0000","last_undegraded":"2024-04-25T10:26:14.985727+0000","last_fullsized":"2024-04-25T10:26:14.985727+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:10.289490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.985634+0000","last_change":"2024-04-25T10:26:12.969936+0000","last_active":"2024-04-25T10:26:14.985634+0000","last_peered":"2024-04-25T10:26:14.985634+0000","last_clean":"2024-04-25T10:26:14.985634+0000","last_became_active":"2024-04-25T10:26:12.969633+0000","last_became_peered":"2024-04-25T10:26:12.969633+0000","last_unstale":"2024-04-25T10:26:14.985634+0000","last_undegraded":"2024-04-25T10:26:14.985634+0000","last_fullsized":"2024-04-25T10:26:14.985634+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:26.568174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:14.984766+0000","last_change":"2024-04-25T10:26:13.983478+0000","last_active":"2024-04-25T10:26:14.984766+0000","last_peered":"2024-04-25T10:26:14.984766+0000","last_clean":"2024-04-25T10:26:14.984766+0000","last_became_active":"2024-04-25T10:26:13.983191+0000","last_became_peered":"2024-04-25T10:26:13.983191+0000","last_unstale":"2024-04-25T10:26:14.984766+0000","last_undegraded":"2024-04-25T10:26:14.984766+0000","last_fullsized":"2024-04-25T10:26:14.984766+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:12.944055+0000","last_clean_scrub_stamp":"2024-04-25T10:26:12.944055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:21.951679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.648161+0000","last_change":"2024-04-25T10:26:12.968805+0000","last_active":"2024-04-25T10:26:15.648161+0000","last_peered":"2024-04-25T10:26:15.648161+0000","last_clean":"2024-04-25T10:26:15.648161+0000","last_became_active":"2024-04-25T10:26:12.968534+0000","last_became_peered":"2024-04-25T10:26:12.968534+0000","last_unstale":"2024-04-25T10:26:15.648161+0000","last_undegraded":"2024-04-25T10:26:15.648161+0000","last_fullsized":"2024-04-25T10:26:15.648161+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:26:11.937740+0000","last_clean_scrub_stamp":"2024-04-25T10:26:11.937740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":351,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T10:26:15.647738+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:26:15.647738+0000","last_peered":"2024-04-25T10:26:15.647738+0000","last_clean":"2024-04-25T10:26:15.647738+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:26:15.647738+0000","last_undegraded":"2024-04-25T10:26:15.647738+0000","last_fullsized":"2024-04-25T10:26:15.647738+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50660,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321180,"statfs":{"total":96636764160,"available":96584888320,"internally_reserved":0,"allocated":1466368,"data_stored":629228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50644,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321196,"statfs":{"total":96636764160,"available":96584904704,"internally_reserved":0,"allocated":1445888,"data_stored":627492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509545,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2048000,"data_stored":1217952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58964,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312876,"statfs":{"total":96636764160,"available":96576385024,"internally_reserved":0,"allocated":1445888,"data_stored":627492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58972,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312868,"statfs":{"total":96636764160,"available":96576376832,"internally_reserved":0,"allocated":1458176,"data_stored":628130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55380,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316460,"statfs":{"total":96636764160,"available":96580055040,"internally_reserved":0,"allocated":2039808,"data_stored":1217860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28444,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343396,"statfs":{"total":96636764160,"available":96607637504,"internally_reserved":0,"allocated":1454080,"data_stored":628144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542249,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58980,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312860,"statfs":{"total":96636764160,"available":96576368640,"internally_reserved":0,"allocated":1462272,"data_stored":629194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:26:23.050 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:26:23.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.235+0000 7f2ce3800640 1 Processor -- start 2024-04-25T10:26:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.235+0000 7f2ce3800640 1 -- start start 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2ce3800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2cdc0d0070 0x7f2cdc0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2ce3800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc15e6e0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2ce3800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc15e9c0 con 0x7f2cdc154f70 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2ce3800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc15eca0 con 0x7f2cdc0d0070 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2ce1800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2cdc0b39d0 0x7f2cdc0ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60854/0 (socket says 172.21.15.45:60854) 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2ce1800640 1 -- 172.21.15.45:0/1040991539 learned_addr learned my addr 172.21.15.45:0/1040991539 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2cd3e00640 1 --2- 172.21.15.45:0/1040991539 >> v2:172.21.15.117:3300/0 conn(0x7f2cdc0d0070 0x7f2cdc0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2cdbe00640 1 -- 172.21.15.45:0/1040991539 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986266078 0 0) 0x7f2cdc15e6e0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2cdbe00640 1 -- 172.21.15.45:0/1040991539 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc0003530 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2cd3e00640 1 -- 172.21.15.45:0/1040991539 >> v1:172.21.15.117:6789/0 conn(0x7f2cdc154f70 legacy=0x7f2cdc0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2cd3e00640 1 -- 172.21.15.45:0/1040991539 >> v1:172.21.15.45:6789/0 conn(0x7f2cdc0b39d0 legacy=0x7f2cdc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.239+0000 7f2cd3e00640 1 -- 172.21.15.45:0/1040991539 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc15ef80 con 0x7f2cdc0d0070 2024-04-25T10:26:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2cd3e00640 1 --2- 172.21.15.45:0/1040991539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc0b3490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40097c0 tx=0x7f2cc401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d8d1f49223ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2cdbe00640 1 -- 172.21.15.45:0/1040991539 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40b0070 con 0x7f2cdc0d0070 2024-04-25T10:26:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2cdbe00640 1 -- 172.21.15.45:0/1040991539 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40ab600 con 0x7f2cdc0d0070 2024-04-25T10:26:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2cdbe00640 1 -- 172.21.15.45:0/1040991539 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40ab9d0 con 0x7f2cdc0d0070 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 -- 172.21.15.45:0/1040991539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 msgr2=0x7f2cdc0b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 --2- 172.21.15.45:0/1040991539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc0b3490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40097c0 tx=0x7f2cc401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 -- 172.21.15.45:0/1040991539 shutdown_connections 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 --2- 172.21.15.45:0/1040991539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc0b3490 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 -- 172.21.15.45:0/1040991539 >> 172.21.15.45:0/1040991539 conn(0x7f2cdc066b70 msgr2=0x7f2cdc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 -- 172.21.15.45:0/1040991539 shutdown_connections 2024-04-25T10:26:23.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 -- 172.21.15.45:0/1040991539 wait complete. 2024-04-25T10:26:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 Processor -- start 2024-04-25T10:26:23.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.243+0000 7f2ce3800640 1 -- start start 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 0x7f2cdc14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cdc154f70 0x7f2cdc152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc168a70 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1688f0 con 0x7f2cdc0d0070 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc168bf0 con 0x7f2cdc154f70 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 0x7f2cdc14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 0x7f2cdc14eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37750/0 (socket says 172.21.15.45:37750) 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 -- 172.21.15.45:0/870867573 learned_addr learned my addr 172.21.15.45:0/870867573 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2cd3e00640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce1800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cdc154f70 0x7f2cdc152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 -- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cdc154f70 msgr2=0x7f2cdc152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cdc154f70 0x7f2cdc152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 -- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 msgr2=0x7f2cdc159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 -- 172.21.15.45:0/870867573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc0cf090 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce0e00640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 0x7f2cdc14eb80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c420 tx=0x7f2ccc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0052d0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc005470 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc007260 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1588e0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc158ea0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc04ef30 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.247+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ccc007400 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.251+0000 7f2cd9600640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cb8066f60 0x7f2cb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.251+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ccc0844c0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.251+0000 7f2cd3e00640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cb8066f60 0x7f2cb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.251+0000 7f2cd3e00640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cb8066f60 0x7f2cb8069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40ad000 tx=0x7f2cc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.255+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ccc054e70 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.471+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2cdc070cf0 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2cd9600640 1 -- 172.21.15.45:0/870867573 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2ccc058f10 con 0x7f2cdc0b39d0 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:26:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:26:23.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cb8066f60 msgr2=0x7f2cb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cb8066f60 0x7f2cb8069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40ad000 tx=0x7f2cc4008040 comp rx=0 tx=0).stop 2024-04-25T10:26:23.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 msgr2=0x7f2cdc14eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 0x7f2cdc14eb80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c420 tx=0x7f2ccc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 shutdown_connections 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2cb8066f60 0x7f2cb8069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cdc154f70 0x7f2cdc152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cdc0d0070 0x7f2cdc159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 --2- 172.21.15.45:0/870867573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cdc0b39d0 0x7f2cdc14eb80 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 >> 172.21.15.45:0/870867573 conn(0x7f2cdc066b70 msgr2=0x7f2cdc0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 shutdown_connections 2024-04-25T10:26:23.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.475+0000 7f2ce3800640 1 -- 172.21.15.45:0/870867573 wait complete. 2024-04-25T10:26:23.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.635+0000 7ff76e200640 1 Processor -- start 2024-04-25T10:26:23.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.635+0000 7ff76e200640 1 -- start start 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.635+0000 7ff76e200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff768158260 0x7ff76815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.635+0000 7ff76e200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680b5c20 con 0x7ff768066fe0 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.635+0000 7ff76e200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680af6e0 con 0x7ff768066c00 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff76e200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680af9c0 con 0x7ff768158260 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff75fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff768066fe0 0x7ff768157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60866/0 (socket says 172.21.15.45:60866) 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff75fe00640 1 -- 172.21.15.45:0/3366164772 learned_addr learned my addr 172.21.15.45:0/3366164772 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff767e00640 1 --2- 172.21.15.45:0/3366164772 >> v2:172.21.15.117:3300/0 conn(0x7ff768158260 0x7ff76815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820067114 0 0) 0x7ff7680b5c20 con 0x7ff768066fe0 2024-04-25T10:26:23.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c003530 con 0x7ff768066fe0 2024-04-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455808581 0 0) 0x7ff7680af6e0 con 0x7ff768066c00 2024-04-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7680b5c20 con 0x7ff768066c00 2024-04-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff767e00640 1 -- 172.21.15.45:0/3366164772 >> v1:172.21.15.117:6789/0 conn(0x7ff768066c00 legacy=0x7ff76806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff767e00640 1 -- 172.21.15.45:0/3366164772 >> v1:172.21.15.45:6789/0 conn(0x7ff768066fe0 legacy=0x7ff768157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4234606274 0 0) 0x7ff74c003530 con 0x7ff768066fe0 2024-04-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.639+0000 7ff767e00640 1 -- 172.21.15.45:0/3366164772 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680afca0 con 0x7ff768158260 2024-04-25T10:26:23.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff767e00640 1 --2- 172.21.15.45:0/3366164772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff76815e110 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff76001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52217c2ee070211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600b2070 con 0x7ff768158260 2024-04-25T10:26:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7600a55e0 con 0x7ff768158260 2024-04-25T10:26:23.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff766a00640 1 -- 172.21.15.45:0/3366164772 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750001680 con 0x7ff768158260 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- 172.21.15.45:0/3366164772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 msgr2=0x7ff76815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 --2- 172.21.15.45:0/3366164772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff76815e110 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d90 tx=0x7ff76001c890 comp rx=0 tx=0).stop 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- 172.21.15.45:0/3366164772 shutdown_connections 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 --2- 172.21.15.45:0/3366164772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff76815e110 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- 172.21.15.45:0/3366164772 >> 172.21.15.45:0/3366164772 conn(0x7ff7680b24f0 msgr2=0x7ff7680b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- 172.21.15.45:0/3366164772 shutdown_connections 2024-04-25T10:26:23.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- 172.21.15.45:0/3366164772 wait complete. 2024-04-25T10:26:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 Processor -- start 2024-04-25T10:26:23.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- start start 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 0x7ff768151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff768066fe0 0x7ff76813d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff768150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76816bb00 con 0x7ff768066c00 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680af270 con 0x7ff768158260 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff76e200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76816bc80 con 0x7ff768066fe0 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.643+0000 7ff767400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 0x7ff768151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 0x7ff768151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37752/0 (socket says 172.21.15.45:37752) 2024-04-25T10:26:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 -- 172.21.15.45:0/185133019 learned_addr learned my addr 172.21.15.45:0/185133019 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767e00640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff768150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75fe00640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff768066fe0 0x7ff76813d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 -- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff768066fe0 msgr2=0x7ff76813d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff768066fe0 0x7ff76813d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 -- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 msgr2=0x7ff768150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff768150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 -- 172.21.15.45:0/185133019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768150b00 con 0x7ff768066c00 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767e00640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff768150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75fe00640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff768066fe0 0x7ff76813d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff767400640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 0x7ff768151ba0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c490 tx=0x7ff75800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580052d0 con 0x7ff768066c00 2024-04-25T10:26:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758005470 con 0x7ff768066c00 2024-04-25T10:26:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff768150de0 con 0x7ff768066c00 2024-04-25T10:26:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758012870 con 0x7ff768066c00 2024-04-25T10:26:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7681513a0 con 0x7ff768066c00 2024-04-25T10:26:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76804ef30 con 0x7ff768066c00 2024-04-25T10:26:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff758012a10 con 0x7ff768066c00 2024-04-25T10:26:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75f400640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff744066f60 0x7ff744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.647+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff758084d10 con 0x7ff768066c00 2024-04-25T10:26:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.651+0000 7ff75fe00640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff744066f60 0x7ff744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.651+0000 7ff75fe00640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff744066f60 0x7ff744069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff7500017f0 tx=0x7ff750009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.651+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff75808a020 con 0x7ff768066c00 2024-04-25T10:26:23.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.851+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff76806f030 con 0x7ff768066c00 2024-04-25T10:26:23.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.851+0000 7ff75f400640 1 -- 172.21.15.45:0/185133019 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff758055c80 con 0x7ff768066c00 2024-04-25T10:26:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:23.855 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:23.857 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:23.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff744066f60 msgr2=0x7ff744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff744066f60 0x7ff744069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff7500017f0 tx=0x7ff750009040 comp rx=0 tx=0).stop 2024-04-25T10:26:23.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 msgr2=0x7ff768151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:23.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 0x7ff768151ba0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c490 tx=0x7ff75800c950 comp rx=0 tx=0).stop 2024-04-25T10:26:23.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 shutdown_connections 2024-04-25T10:26:23.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff744066f60 0x7ff744069420 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff768158260 0x7ff768150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff768066fe0 0x7ff76813d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 --2- 172.21.15.45:0/185133019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff768066c00 0x7ff768151ba0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:23.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 >> 172.21.15.45:0/185133019 conn(0x7ff7680b24f0 msgr2=0x7ff76815e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:23.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 shutdown_connections 2024-04-25T10:26:23.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:23.855+0000 7ff76e200640 1 -- 172.21.15.45:0/185133019 wait complete. 2024-04-25T10:26:24.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:26:13.356392+0000', 'modified': '2024-04-25T10:26:16.392149+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8373': {'gid': 8373, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6834/1838662791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6832', 'nonce': 1838662791}, {'type': 'v1', 'addr': '172.21.15.45:6834', 'nonce': 1838662791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:26:24.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:26:24.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:26:24.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 Processor -- start 2024-04-25T10:26:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 -- start start 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 0x7ff7300cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff73006aea0 0x7ff73006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7300ccaa0 con 0x7ff7300d0180 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73015e340 con 0x7ff73006aea0 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff735000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300ccd80 con 0x7ff73015df60 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff725400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff7300d0180 0x7ff7300d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60868/0 (socket says 172.21.15.45:60868) 2024-04-25T10:26:24.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff725400640 1 -- 172.21.15.45:0/4150259682 learned_addr learned my addr 172.21.15.45:0/4150259682 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72e800640 1 --2- 172.21.15.45:0/4150259682 >> v2:172.21.15.117:3300/0 conn(0x7ff73006aea0 0x7ff73006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72de00640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 0x7ff7300cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758479547 0 0) 0x7ff7300ccaa0 con 0x7ff7300d0180 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff714003530 con 0x7ff7300d0180 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281991993 0 0) 0x7ff714003530 con 0x7ff7300d0180 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 msgr2=0x7ff7300cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 0x7ff7300cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 >> v2:172.21.15.117:3300/0 conn(0x7ff73006aea0 msgr2=0x7ff73006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 --2- 172.21.15.45:0/4150259682 >> v2:172.21.15.117:3300/0 conn(0x7ff73006aea0 0x7ff73006b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300cd060 con 0x7ff7300d0180 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7ff718001a00 con 0x7ff7300d0180 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 >> v1:172.21.15.45:6789/0 conn(0x7ff7300d0180 legacy=0x7ff7300d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 0x7ff7140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714003f40 0x7ff714004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff7140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300ccd80 con 0x7ff714004d50 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71400b9d0 con 0x7ff714003f40 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71400bba0 con 0x7ff73015df60 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.423+0000 7ff72e800640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff7140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72de00640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 0x7ff7140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff725400640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714003f40 0x7ff714004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 -- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 msgr2=0x7ff7140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff73015df60 0x7ff7140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 -- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714003f40 msgr2=0x7ff714004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff714003f40 0x7ff714004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 -- 172.21.15.45:0/4150259682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff714003530 con 0x7ff714004d50 2024-04-25T10:26:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff7140051d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff728009b10 tx=0x7ff72801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412f9ebab5fe9adf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7280b2070 con 0x7ff714004d50 2024-04-25T10:26:24.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7280a5590 con 0x7ff714004d50 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72d400640 1 -- 172.21.15.45:0/4150259682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7280096b0 con 0x7ff714004d50 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff735000640 1 -- 172.21.15.45:0/4150259682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 msgr2=0x7ff7140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff735000640 1 --2- 172.21.15.45:0/4150259682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff7140051d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff728009b10 tx=0x7ff72801cab0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.427+0000 7ff72e800640 1 -- 172.21.15.45:0/4150259682 reap_dead start 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- 172.21.15.45:0/4150259682 shutdown_connections 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- 172.21.15.45:0/4150259682 >> 172.21.15.45:0/4150259682 conn(0x7ff7300d1910 msgr2=0x7ff7300d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- 172.21.15.45:0/4150259682 shutdown_connections 2024-04-25T10:26:24.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- 172.21.15.45:0/4150259682 wait complete. 2024-04-25T10:26:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 Processor -- start 2024-04-25T10:26:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- start start 2024-04-25T10:26:24.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff73014f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730151e60 0x7ff730152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7301529b0 0x7ff7300cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300ceb50 con 0x7ff714004d50 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300cd620 con 0x7ff730151e60 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300cecd0 con 0x7ff7301529b0 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff73014f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff73014f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37770/0 (socket says 172.21.15.45:37770) 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 -- 172.21.15.45:0/356103409 learned_addr learned my addr 172.21.15.45:0/356103409 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff725400640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730151e60 0x7ff730152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72e800640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7301529b0 0x7ff7300cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 -- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7301529b0 msgr2=0x7ff7300cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7301529b0 0x7ff7300cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 -- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730151e60 msgr2=0x7ff730152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730151e60 0x7ff730152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 -- 172.21.15.45:0/356103409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300c5420 con 0x7ff714004d50 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff72de00640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff73014f570 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff720011a00 tx=0x7ff720011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff726a00640 1 -- 172.21.15.45:0/356103409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000faf0 con 0x7ff714004d50 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff726a00640 1 -- 172.21.15.45:0/356103409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72000fc90 con 0x7ff714004d50 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300c56c0 con 0x7ff714004d50 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff726a00640 1 -- 172.21.15.45:0/356103409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720014440 con 0x7ff714004d50 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.431+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7300c5c80 con 0x7ff714004d50 2024-04-25T10:26:24.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff726a00640 1 -- 172.21.15.45:0/356103409 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff720012040 con 0x7ff714004d50 2024-04-25T10:26:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff726a00640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff70c066f90 0x7ff70c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff726a00640 1 -- 172.21.15.45:0/356103409 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff720085650 con 0x7ff714004d50 2024-04-25T10:26:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff725400640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff70c066f90 0x7ff70c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff738417000 1 Processor -- start 2024-04-25T10:26:24.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff738417000 1 -- start start 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.435+0000 7ff725400640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff70c066f90 0x7ff70c069450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff7300b2c80 tx=0x7ff71800a0e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f7920 0x55c5090f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 0x55c5090ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508f955f0 con 0x55c5090f8240 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508efcda0 con 0x55c5090f7920 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c508e85150 con 0x55c5090adf30 2024-04-25T10:26:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 0x55c5090ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 0x55c5090ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37776/0 (socket says 172.21.15.45:37776) 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 -- 172.21.15.45:0/1391344171 learned_addr learned my addr 172.21.15.45:0/1391344171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff72de00640 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f7920 0x55c5090f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff72e800640 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 -- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 msgr2=0x55c5090ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 -- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f7920 msgr2=0x55c5090f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f7920 0x55c5090f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 -- 172.21.15.45:0/1391344171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c508e9eca0 con 0x55c5090f8240 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff72e800640 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff725400640 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 0x55c5090ad9f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff718014c80 tx=0x7ff718017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d39e9a235668a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff71ea00640 1 -- 172.21.15.45:0/1391344171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71801b070 con 0x55c5090f8240 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff71ea00640 1 -- 172.21.15.45:0/1391344171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718028070 con 0x55c5090f8240 2024-04-25T10:26:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff71ea00640 1 -- 172.21.15.45:0/1391344171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718023630 con 0x55c5090f8240 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- 172.21.15.45:0/1391344171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 msgr2=0x55c5090ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 0x55c5090ad9f0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff718014c80 tx=0x7ff718017000 comp rx=0 tx=0).stop 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- 172.21.15.45:0/1391344171 shutdown_connections 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f8240 0x55c5090ad9f0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 --2- 172.21.15.45:0/1391344171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f7920 0x55c5090f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- 172.21.15.45:0/1391344171 >> 172.21.15.45:0/1391344171 conn(0x55c5090b42a0 msgr2=0x55c5090ac2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- 172.21.15.45:0/1391344171 shutdown_connections 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- 172.21.15.45:0/1391344171 wait complete. 2024-04-25T10:26:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 -- start start 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.439+0000 7ff738417000 1 Processor -- start 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5090ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c5090b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508eb7990 con 0x55c5090f7920 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508ef8e40 con 0x55c5090f8240 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c508e88900 con 0x55c5090adf30 2024-04-25T10:26:24.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 10 client.0 _ll_register_callbacks cb 0x55c508fb01c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5090ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5090ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37780/0 (socket says 172.21.15.45:37780) 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 -- 172.21.15.45:0/694494393 learned_addr learned my addr 172.21.15.45:0/694494393 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff72de00640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff72e800640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c5090b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 -- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 msgr2=0x55c5090fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 -- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 msgr2=0x55c5090b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c5090b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 -- 172.21.15.45:0/694494393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c508eb5630 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff725400640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5090ff1c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff718002240 tx=0x7ff718001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa3934539a8466b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 1 -- 172.21.15.45:0/694494393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71802d680 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 1 -- 172.21.15.45:0/694494393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff718033800 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff718028070 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 1 -- 172.21.15.45:0/694494393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff718033a40 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 10 client.8541 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55c508e853a0 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 1 -- 172.21.15.45:0/694494393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718028070 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 1 -- 172.21.15.45:0/694494393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff718033be0 con 0x55c5090f7920 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 10 client.8541 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 10 client.8541 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 20 client.8541 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c5090b3160 0x55c509108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 4 client.8541 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c508e98a50 con 0x55c5090b3160 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff72de00640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c5090b3160 0x55c509108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff72de00640 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c5090b3160 0x55c509108ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5829d1bce26bb8aa server_cookie=826a472415224db9 in_seq=0 out_seq=0 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 10 client.8541 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 1 -- 172.21.15.45:0/694494393 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c508e98a50 con 0x55c5090b3160 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff713400640 10 client.8541 handle_command_reply: tid=0 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.443+0000 7ff738417000 1 client.8541 shutdown 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 msgr2=0x55c5090ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5090ff1c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff718002240 tx=0x7ff718001dd0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 shutdown_connections 2024-04-25T10:26:24.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c5090b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c5090b3160 0x55c509108ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5090ff1c0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 --2- 172.21.15.45:0/694494393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090adf30 0x55c5090fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff713400640 0 client.8541 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 >> 172.21.15.45:0/694494393 conn(0x55c5090b42a0 msgr2=0x55c5090ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 shutdown_connections 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 1 -- 172.21.15.45:0/694494393 wait complete. 2024-04-25T10:26:24.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.447+0000 7ff738417000 20 client.8541 trim_cache size 0 max 16384 2024-04-25T10:26:24.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 Processor -- start 2024-04-25T10:26:24.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- start start 2024-04-25T10:26:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090adf30 0x55c509153190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5091536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f8240 0x55c509153c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7301ff350 con 0x55c5090f7920 2024-04-25T10:26:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7301fe780 con 0x55c5090adf30 2024-04-25T10:26:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7301fed80 con 0x55c5090f8240 2024-04-25T10:26:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5091536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5091536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37786/0 (socket says 172.21.15.45:37786) 2024-04-25T10:26:24.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 -- 172.21.15.45:0/1931725543 learned_addr learned my addr 172.21.15.45:0/1931725543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 -- 172.21.15.45:0/1931725543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f8240 msgr2=0x55c509153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f8240 0x55c509153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 -- 172.21.15.45:0/1931725543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090adf30 msgr2=0x55c509153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090adf30 0x55c509153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 -- 172.21.15.45:0/1931725543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5090f8070 con 0x55c5090f7920 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff725400640 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5091536d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff718017fd0 tx=0x7ff718031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6afc92b630108c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff71e000640 1 -- 172.21.15.45:0/1931725543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71802d8c0 con 0x55c5090f7920 2024-04-25T10:26:24.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff71e000640 1 -- 172.21.15.45:0/1931725543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718028070 con 0x55c5090f7920 2024-04-25T10:26:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff71e000640 1 -- 172.21.15.45:0/1931725543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71802d480 con 0x55c5090f7920 2024-04-25T10:26:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- 172.21.15.45:0/1931725543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 msgr2=0x55c5091536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5091536d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff718017fd0 tx=0x7ff718031040 comp rx=0 tx=0).stop 2024-04-25T10:26:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- 172.21.15.45:0/1931725543 shutdown_connections 2024-04-25T10:26:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f8240 0x55c509153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090f7920 0x55c5091536d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 --2- 172.21.15.45:0/1931725543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090adf30 0x55c509153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- 172.21.15.45:0/1931725543 >> 172.21.15.45:0/1931725543 conn(0x55c5090b42a0 msgr2=0x55c5090b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- 172.21.15.45:0/1931725543 shutdown_connections 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- 172.21.15.45:0/1931725543 wait complete. 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 -- start start 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.463+0000 7ff738417000 1 Processor -- start 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 0x55c5091591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f7920 0x55c50915b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c50915dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508f013d0 con 0x55c5090adf30 2024-04-25T10:26:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c508fb8d50 con 0x55c5090f8240 2024-04-25T10:26:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c508e97980 con 0x55c5090f7920 2024-04-25T10:26:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 10 client.0 _ll_register_callbacks cb 0x7ff718007af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 0x55c5091591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 0x55c5091591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37790/0 (socket says 172.21.15.45:37790) 2024-04-25T10:26:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 -- 172.21.15.45:0/1717299449 learned_addr learned my addr 172.21.15.45:0/1717299449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff725400640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f7920 0x55c50915b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72e800640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c50915dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 -- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f7920 msgr2=0x55c50915b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f7920 0x55c50915b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 -- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 msgr2=0x55c50915dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c50915dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 -- 172.21.15.45:0/1717299449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c508f7e600 con 0x55c5090adf30 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff72de00640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 0x55c5091591e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff720052470 tx=0x7ff720085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f792ad1a9efe693e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 1 -- 172.21.15.45:0/1717299449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720081070 con 0x55c5090adf30 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 1 -- 172.21.15.45:0/1717299449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff720052680 con 0x55c5090adf30 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff718028070 con 0x55c5090adf30 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 1 -- 172.21.15.45:0/1717299449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff72007d050 con 0x55c5090adf30 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 10 client.8547 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55c508e919b0 con 0x55c5090adf30 2024-04-25T10:26:24.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 1 -- 172.21.15.45:0/1717299449 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718028070 con 0x55c5090adf30 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 1 -- 172.21.15.45:0/1717299449 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff72006dce0 con 0x55c5090adf30 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 10 client.8547 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 10 client.8547 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 20 client.8547 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c50915e1c0 0x55c5091605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 4 client.8547 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c508f0a2e0 con 0x55c50915e1c0 2024-04-25T10:26:24.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff725400640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c50915e1c0 0x55c5091605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff725400640 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c50915e1c0 0x55c5091605b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f871fd9425be35d server_cookie=83ac33e32e567b39 in_seq=0 out_seq=0 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 10 client.8547 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 1 -- 172.21.15.45:0/1717299449 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c508f0a2e0 con 0x55c50915e1c0 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 10 client.8547 handle_command_reply: tid=0 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 client.8547 shutdown 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 msgr2=0x55c5091591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 0x55c5091591e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff720052470 tx=0x7ff720085040 comp rx=0 tx=0).stop 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 shutdown_connections 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5090f8240 0x55c50915dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c5090f7920 0x55c50915b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c50915e1c0 0x55c5091605b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 --2- 172.21.15.45:0/1717299449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5090adf30 0x55c5091591e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff713400640 0 client.8547 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:24.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 >> 172.21.15.45:0/1717299449 conn(0x55c5090b42a0 msgr2=0x55c5090ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 shutdown_connections 2024-04-25T10:26:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 1 -- 172.21.15.45:0/1717299449 wait complete. 2024-04-25T10:26:24.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.467+0000 7ff738417000 20 client.8547 trim_cache size 0 max 16384 2024-04-25T10:26:24.486 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:26:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:26:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8373, 2024-04-25T10:26:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:26:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:26:24.487 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:26:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.502594575, 2024-04-25T10:26:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 82, 2024-04-25T10:26:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 117, 2024-04-25T10:26:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 117, 2024-04-25T10:26:24.488 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 15.337384588999999 2024-04-25T10:26:24.488 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff70c066f90 msgr2=0x7ff70c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff70c066f90 0x7ff70c069450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff7300b2c80 tx=0x7ff71800a0e0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 msgr2=0x7ff73014f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff73014f570 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff720011a00 tx=0x7ff720011ec0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 shutdown_connections 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff70c066f90 0x7ff70c069450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff7301529b0 0x7ff7300cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730151e60 0x7ff730152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 --2- 172.21.15.45:0/356103409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff714004d50 0x7ff73014f570 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 >> 172.21.15.45:0/356103409 conn(0x7ff7300d1910 msgr2=0x7ff7300d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 shutdown_connections 2024-04-25T10:26:24.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.471+0000 7ff735000640 1 -- 172.21.15.45:0/356103409 wait complete. 2024-04-25T10:26:24.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:26:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.631+0000 7f72b3200640 1 Processor -- start 2024-04-25T10:26:24.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b3200640 1 -- start start 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 0x7f72b4070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b3200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72b40b2260 0x7f72b40b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b3200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b415bf50 con 0x7f72b40711f0 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b3200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b4151d10 con 0x7f72b40b2260 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40b3980 con 0x7f72b4151930 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72a9400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f72b40711f0 0x7f72b40b1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60876/0 (socket says 172.21.15.45:60876) 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72a9400640 1 -- 172.21.15.45:0/2093117897 learned_addr learned my addr 172.21.15.45:0/2093117897 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1e00640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 0x7f72b4070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 --2- 172.21.15.45:0/2093117897 >> v2:172.21.15.117:3300/0 conn(0x7f72b40b2260 0x7f72b40b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856342028 0 0) 0x7f72b415bf50 con 0x7f72b40711f0 2024-04-25T10:26:24.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7298003530 con 0x7f72b40711f0 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326736804 0 0) 0x7f7298003530 con 0x7f72b40711f0 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 msgr2=0x7f72b4070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 0x7f72b4070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 >> v2:172.21.15.117:3300/0 conn(0x7f72b40b2260 msgr2=0x7f72b40b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 --2- 172.21.15.45:0/2093117897 >> v2:172.21.15.117:3300/0 conn(0x7f72b40b2260 0x7f72b40b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b415b0a0 con 0x7f72b40711f0 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f729c001800 con 0x7f72b40711f0 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 >> v1:172.21.15.45:6789/0 conn(0x7f72b40711f0 legacy=0x7f72b40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 0x7f72980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298003f40 0x7f7298004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7298004d50 0x7f72980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4151d10 con 0x7f7298004d50 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40b3980 con 0x7f7298003f40 2024-04-25T10:26:24.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729800ba20 con 0x7f72b4151930 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7298004d50 0x7f72980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b1e00640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 0x7f72980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72a9400640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298003f40 0x7f7298004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 -- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 msgr2=0x7f72980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b4151930 0x7f72980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 -- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298003f40 msgr2=0x7f7298004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298003f40 0x7f7298004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72b2800640 1 -- 172.21.15.45:0/2093117897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298003530 con 0x7f7298004d50 2024-04-25T10:26:24.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.635+0000 7f72a9400640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298003f40 0x7f7298004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b2800640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7298004d50 0x7f72980051d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009fc0 tx=0x7f72ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bef67561526af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0b2070 con 0x7f7298004d50 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0a5c90 con 0x7f7298004d50 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b1400640 1 -- 172.21.15.45:0/2093117897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac009c10 con 0x7f7298004d50 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b3200640 1 -- 172.21.15.45:0/2093117897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7298004d50 msgr2=0x7f72980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b3200640 1 --2- 172.21.15.45:0/2093117897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7298004d50 0x7f72980051d0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009fc0 tx=0x7f72ac01cc80 comp rx=0 tx=0).stop 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b2800640 1 -- 172.21.15.45:0/2093117897 reap_dead start 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b3200640 1 -- 172.21.15.45:0/2093117897 shutdown_connections 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b3200640 1 -- 172.21.15.45:0/2093117897 >> 172.21.15.45:0/2093117897 conn(0x7f72b406ac90 msgr2=0x7f72b406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b3200640 1 -- 172.21.15.45:0/2093117897 shutdown_connections 2024-04-25T10:26:24.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.639+0000 7f72b3200640 1 -- 172.21.15.45:0/2093117897 wait complete. 2024-04-25T10:26:24.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 Processor -- start 2024-04-25T10:26:24.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- start start 2024-04-25T10:26:24.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298004d50 0x7f72b4159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b415da60 0x7f72b415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 0x7f72b4146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4071b00 con 0x7f72b4146740 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40b3c60 con 0x7f7298004d50 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b4071de0 con 0x7f72b415da60 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 0x7f72b4146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 0x7f72b4146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37812/0 (socket says 172.21.15.45:37812) 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 -- 172.21.15.45:0/3625079984 learned_addr learned my addr 172.21.15.45:0/3625079984 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 -- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b415da60 msgr2=0x7f72b415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b415da60 0x7f72b415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 -- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298004d50 msgr2=0x7f72b4159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b1e00640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298004d50 0x7f72b4159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298004d50 0x7f72b4159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 -- 172.21.15.45:0/3625079984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b415e520 con 0x7f72b4146740 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b1e00640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298004d50 0x7f72b4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:24.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b2800640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 0x7f72b4146bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0b0040 tx=0x7f72ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0bd070 con 0x7f72b4146740 2024-04-25T10:26:24.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0a5a90 con 0x7f72b4146740 2024-04-25T10:26:24.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b40cd300 con 0x7f72b4146740 2024-04-25T10:26:24.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0b2040 con 0x7f72b4146740 2024-04-25T10:26:24.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b40cd8c0 con 0x7f72b4146740 2024-04-25T10:26:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.643+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b4074520 con 0x7f72b4146740 2024-04-25T10:26:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.647+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72ac0c0070 con 0x7f72b4146740 2024-04-25T10:26:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.647+0000 7f72aaa00640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7290066f90 0x7f7290069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.647+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72ac11f110 con 0x7f72b4146740 2024-04-25T10:26:24.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.647+0000 7f72b1e00640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7290066f90 0x7f7290069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:24.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.647+0000 7f72b1e00640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7290066f90 0x7f7290069450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009ad0 tx=0x7f72a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:24.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.651+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72ac0169d0 con 0x7f72b4146740 2024-04-25T10:26:24.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.847+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72b40b3530 con 0x7f72b4146740 2024-04-25T10:26:24.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72aaa00640 1 -- 172.21.15.45:0/3625079984 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f72ac0efa30 con 0x7f72b4146740 2024-04-25T10:26:24.852 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7290066f90 msgr2=0x7f7290069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7290066f90 0x7f7290069450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009ad0 tx=0x7f72a4008040 comp rx=0 tx=0).stop 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 msgr2=0x7f72b4146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 0x7f72b4146bc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0b0040 tx=0x7f72ac009750 comp rx=0 tx=0).stop 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 shutdown_connections 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7290066f90 0x7f7290069450 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72b4146740 0x7f72b4146bc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72b415da60 0x7f72b415dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 --2- 172.21.15.45:0/3625079984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7298004d50 0x7f72b4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:24.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 >> 172.21.15.45:0/3625079984 conn(0x7f72b406ac90 msgr2=0x7f72b41521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 shutdown_connections 2024-04-25T10:26:24.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:24.851+0000 7f72b3200640 1 -- 172.21.15.45:0/3625079984 wait complete. 2024-04-25T10:26:24.873 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:26:24.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:26:24.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:24.875 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:24.884 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41955sec preferred_lft 41955sec 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:26:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.886 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:26:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:26:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:26:24.887 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:26:24.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:24.888 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:26:24.953 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:26:24.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:24.956 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:26:25.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.014 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:26:25.014 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:25.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:26:25.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:26:25.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:26:25.014 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:25.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.110 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:26:25.119 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:26:25.119 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:26:25.119 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:26:25.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:26:25.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:26:25.183 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:25.235 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:26:25.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:26:25.235 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:25.287 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:26:25.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:26:25.377 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:26:25.377 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:26:25.391 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:26:25.391 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:26:25.391 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:26:25.391 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:26:25.391 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:26:25.395 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:26:25.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:26:25.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.416 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:26:25.481 INFO:teuthology.orchestra.run.smithi045.stdout:client8556 2024-04-25T10:26:25.481 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:26:25.481 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:26:25.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:25.532 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:25.532 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:25.532 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:26:25.532 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:25.532 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:25.533 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:25.533 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170228 s, 61.6 MB/s 2024-04-25T10:26:25.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.637 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163093 s, 64.3 MB/s 2024-04-25T10:26:25.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.656 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176228 s, 59.5 MB/s 2024-04-25T10:26:25.673 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.674 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160219 s, 65.4 MB/s 2024-04-25T10:26:25.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172389 s, 60.8 MB/s 2024-04-25T10:26:25.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.711 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163223 s, 64.2 MB/s 2024-04-25T10:26:25.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159934 s, 65.6 MB/s 2024-04-25T10:26:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156663 s, 66.9 MB/s 2024-04-25T10:26:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.764 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161806 s, 64.8 MB/s 2024-04-25T10:26:25.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.780 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142395 s, 73.6 MB/s 2024-04-25T10:26:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137139 s, 76.5 MB/s 2024-04-25T10:26:25.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.813 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159715 s, 65.7 MB/s 2024-04-25T10:26:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158888 s, 66.0 MB/s 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.847 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149565 s, 70.1 MB/s 2024-04-25T10:26:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141066 s, 74.3 MB/s 2024-04-25T10:26:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.879 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133782 s, 78.4 MB/s 2024-04-25T10:26:25.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.897 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166088 s, 63.1 MB/s 2024-04-25T10:26:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178159 s, 58.9 MB/s 2024-04-25T10:26:25.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133048 s, 78.8 MB/s 2024-04-25T10:26:25.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147144 s, 71.3 MB/s 2024-04-25T10:26:25.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.966 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162542 s, 64.5 MB/s 2024-04-25T10:26:25.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014769 s, 71.0 MB/s 2024-04-25T10:26:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:25.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144584 s, 72.5 MB/s 2024-04-25T10:26:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.017 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156617 s, 67.0 MB/s 2024-04-25T10:26:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157438 s, 66.6 MB/s 2024-04-25T10:26:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.055 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189003 s, 55.5 MB/s 2024-04-25T10:26:26.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.071 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144842 s, 72.4 MB/s 2024-04-25T10:26:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167622 s, 62.6 MB/s 2024-04-25T10:26:26.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132052 s, 79.4 MB/s 2024-04-25T10:26:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161168 s, 65.1 MB/s 2024-04-25T10:26:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142214 s, 73.7 MB/s 2024-04-25T10:26:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156916 s, 66.8 MB/s 2024-04-25T10:26:26.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:26.159 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.231 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176587 s, 59.4 MB/s 2024-04-25T10:26:26.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196853 s, 53.3 MB/s 2024-04-25T10:26:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016466 s, 63.7 MB/s 2024-04-25T10:26:26.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.287 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142793 s, 73.4 MB/s 2024-04-25T10:26:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158565 s, 66.1 MB/s 2024-04-25T10:26:26.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.323 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165562 s, 63.3 MB/s 2024-04-25T10:26:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139436 s, 75.2 MB/s 2024-04-25T10:26:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172761 s, 60.7 MB/s 2024-04-25T10:26:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145234 s, 72.2 MB/s 2024-04-25T10:26:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163754 s, 64.0 MB/s 2024-04-25T10:26:26.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.409 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152833 s, 68.6 MB/s 2024-04-25T10:26:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122762 s, 85.4 MB/s 2024-04-25T10:26:26.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013392 s, 78.3 MB/s 2024-04-25T10:26:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018931 s, 55.4 MB/s 2024-04-25T10:26:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.474 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133127 s, 78.8 MB/s 2024-04-25T10:26:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.490 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143675 s, 73.0 MB/s 2024-04-25T10:26:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141593 s, 74.1 MB/s 2024-04-25T10:26:26.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142445 s, 73.6 MB/s 2024-04-25T10:26:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190579 s, 55.0 MB/s 2024-04-25T10:26:26.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.561 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149534 s, 70.1 MB/s 2024-04-25T10:26:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132145 s, 79.4 MB/s 2024-04-25T10:26:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.591 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140526 s, 74.6 MB/s 2024-04-25T10:26:26.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138868 s, 75.5 MB/s 2024-04-25T10:26:26.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154063 s, 68.1 MB/s 2024-04-25T10:26:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137806 s, 76.1 MB/s 2024-04-25T10:26:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.659 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183851 s, 57.0 MB/s 2024-04-25T10:26:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137042 s, 76.5 MB/s 2024-04-25T10:26:26.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.691 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140098 s, 74.8 MB/s 2024-04-25T10:26:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174462 s, 60.1 MB/s 2024-04-25T10:26:26.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.726 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013716 s, 76.4 MB/s 2024-04-25T10:26:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140334 s, 74.7 MB/s 2024-04-25T10:26:26.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130025 s, 80.6 MB/s 2024-04-25T10:26:26.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:26.758 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:26.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193173 s, 54.3 MB/s 2024-04-25T10:26:26.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.847 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132294 s, 79.3 MB/s 2024-04-25T10:26:26.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180546 s, 58.1 MB/s 2024-04-25T10:26:26.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136101 s, 77.0 MB/s 2024-04-25T10:26:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131576 s, 79.7 MB/s 2024-04-25T10:26:26.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157184 s, 66.7 MB/s 2024-04-25T10:26:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.936 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018829 s, 55.7 MB/s 2024-04-25T10:26:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.951 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135424 s, 77.4 MB/s 2024-04-25T10:26:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154032 s, 68.1 MB/s 2024-04-25T10:26:26.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:26.985 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:26.985 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150329 s, 69.8 MB/s 2024-04-25T10:26:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167096 s, 62.8 MB/s 2024-04-25T10:26:27.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.020 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149746 s, 70.0 MB/s 2024-04-25T10:26:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.041 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188903 s, 55.5 MB/s 2024-04-25T10:26:27.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.057 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.057 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143505 s, 73.1 MB/s 2024-04-25T10:26:27.072 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.072 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135947 s, 77.1 MB/s 2024-04-25T10:26:27.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.089 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150451 s, 69.7 MB/s 2024-04-25T10:26:27.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139133 s, 75.4 MB/s 2024-04-25T10:26:27.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135595 s, 77.3 MB/s 2024-04-25T10:26:27.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150021 s, 69.9 MB/s 2024-04-25T10:26:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141485 s, 74.1 MB/s 2024-04-25T10:26:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136302 s, 76.9 MB/s 2024-04-25T10:26:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.185 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137936 s, 76.0 MB/s 2024-04-25T10:26:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142881 s, 73.4 MB/s 2024-04-25T10:26:27.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182031 s, 57.6 MB/s 2024-04-25T10:26:27.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.237 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014962 s, 70.1 MB/s 2024-04-25T10:26:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.254 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148938 s, 70.4 MB/s 2024-04-25T10:26:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.270 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140598 s, 74.6 MB/s 2024-04-25T10:26:27.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.285 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134942 s, 77.7 MB/s 2024-04-25T10:26:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.302 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148383 s, 70.7 MB/s 2024-04-25T10:26:27.317 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.317 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.317 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138976 s, 75.5 MB/s 2024-04-25T10:26:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.335 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156259 s, 67.1 MB/s 2024-04-25T10:26:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136522 s, 76.8 MB/s 2024-04-25T10:26:27.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:27.352 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133892 s, 78.3 MB/s 2024-04-25T10:26:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138706 s, 75.6 MB/s 2024-04-25T10:26:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.458 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184461 s, 56.8 MB/s 2024-04-25T10:26:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148474 s, 70.6 MB/s 2024-04-25T10:26:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125873 s, 83.3 MB/s 2024-04-25T10:26:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154269 s, 68.0 MB/s 2024-04-25T10:26:27.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015479 s, 67.7 MB/s 2024-04-25T10:26:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138159 s, 75.9 MB/s 2024-04-25T10:26:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.557 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156317 s, 67.1 MB/s 2024-04-25T10:26:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.574 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151364 s, 69.3 MB/s 2024-04-25T10:26:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137194 s, 76.4 MB/s 2024-04-25T10:26:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.604 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134866 s, 77.7 MB/s 2024-04-25T10:26:27.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.620 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136983 s, 76.5 MB/s 2024-04-25T10:26:27.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135567 s, 77.3 MB/s 2024-04-25T10:26:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144102 s, 72.8 MB/s 2024-04-25T10:26:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01337 s, 78.4 MB/s 2024-04-25T10:26:27.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153325 s, 68.4 MB/s 2024-04-25T10:26:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142353 s, 73.7 MB/s 2024-04-25T10:26:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130598 s, 80.3 MB/s 2024-04-25T10:26:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138077 s, 75.9 MB/s 2024-04-25T10:26:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147696 s, 71.0 MB/s 2024-04-25T10:26:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145482 s, 72.1 MB/s 2024-04-25T10:26:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.779 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145879 s, 71.9 MB/s 2024-04-25T10:26:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.797 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156627 s, 66.9 MB/s 2024-04-25T10:26:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013656 s, 76.8 MB/s 2024-04-25T10:26:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013798 s, 76.0 MB/s 2024-04-25T10:26:27.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.843 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133192 s, 78.7 MB/s 2024-04-25T10:26:27.858 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.858 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138871 s, 75.5 MB/s 2024-04-25T10:26:27.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.873 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012933 s, 81.1 MB/s 2024-04-25T10:26:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.890 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143416 s, 73.1 MB/s 2024-04-25T10:26:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180231 s, 58.2 MB/s 2024-04-25T10:26:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:27.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132553 s, 79.1 MB/s 2024-04-25T10:26:27.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:27.926 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178334 s, 58.8 MB/s 2024-04-25T10:26:28.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.020 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153411 s, 68.4 MB/s 2024-04-25T10:26:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156144 s, 67.2 MB/s 2024-04-25T10:26:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137247 s, 76.4 MB/s 2024-04-25T10:26:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.068 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138716 s, 75.6 MB/s 2024-04-25T10:26:28.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123983 s, 84.6 MB/s 2024-04-25T10:26:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147454 s, 71.1 MB/s 2024-04-25T10:26:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188174 s, 55.7 MB/s 2024-04-25T10:26:28.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.136 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141953 s, 73.9 MB/s 2024-04-25T10:26:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182639 s, 57.4 MB/s 2024-04-25T10:26:28.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136662 s, 76.7 MB/s 2024-04-25T10:26:28.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.188 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147117 s, 71.3 MB/s 2024-04-25T10:26:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.203 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135578 s, 77.3 MB/s 2024-04-25T10:26:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.219 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014032 s, 74.7 MB/s 2024-04-25T10:26:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142801 s, 73.4 MB/s 2024-04-25T10:26:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.256 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187114 s, 56.0 MB/s 2024-04-25T10:26:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.273 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151935 s, 69.0 MB/s 2024-04-25T10:26:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.289 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137821 s, 76.1 MB/s 2024-04-25T10:26:28.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135547 s, 77.4 MB/s 2024-04-25T10:26:28.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197942 s, 53.0 MB/s 2024-04-25T10:26:28.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.341 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139192 s, 75.3 MB/s 2024-04-25T10:26:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139745 s, 75.0 MB/s 2024-04-25T10:26:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157029 s, 66.8 MB/s 2024-04-25T10:26:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150438 s, 69.7 MB/s 2024-04-25T10:26:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014216 s, 73.8 MB/s 2024-04-25T10:26:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147731 s, 71.0 MB/s 2024-04-25T10:26:28.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.443 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016964 s, 61.8 MB/s 2024-04-25T10:26:28.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.463 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178328 s, 58.8 MB/s 2024-04-25T10:26:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129987 s, 80.7 MB/s 2024-04-25T10:26:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015492 s, 67.7 MB/s 2024-04-25T10:26:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184238 s, 56.9 MB/s 2024-04-25T10:26:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139752 s, 75.0 MB/s 2024-04-25T10:26:28.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:28.532 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:28.532 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:28.532 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:26:28.532 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:28.533 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:28.533 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:28.533 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138302 s, 75.8 MB/s 2024-04-25T10:26:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154614 s, 67.8 MB/s 2024-04-25T10:26:28.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139125 s, 75.4 MB/s 2024-04-25T10:26:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013702 s, 76.5 MB/s 2024-04-25T10:26:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187805 s, 55.8 MB/s 2024-04-25T10:26:28.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151866 s, 69.0 MB/s 2024-04-25T10:26:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015116 s, 69.4 MB/s 2024-04-25T10:26:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135849 s, 77.2 MB/s 2024-04-25T10:26:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01442 s, 72.7 MB/s 2024-04-25T10:26:28.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.757 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142469 s, 73.6 MB/s 2024-04-25T10:26:28.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013758 s, 76.2 MB/s 2024-04-25T10:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134953 s, 77.7 MB/s 2024-04-25T10:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192583 s, 54.4 MB/s 2024-04-25T10:26:28.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145118 s, 72.3 MB/s 2024-04-25T10:26:28.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.844 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156708 s, 66.9 MB/s 2024-04-25T10:26:28.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.859 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149407 s, 70.2 MB/s 2024-04-25T10:26:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191924 s, 54.6 MB/s 2024-04-25T10:26:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144152 s, 72.7 MB/s 2024-04-25T10:26:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.913 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140923 s, 74.4 MB/s 2024-04-25T10:26:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.928 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136988 s, 76.5 MB/s 2024-04-25T10:26:28.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.944 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145057 s, 72.3 MB/s 2024-04-25T10:26:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157432 s, 66.6 MB/s 2024-04-25T10:26:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183909 s, 57.0 MB/s 2024-04-25T10:26:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149293 s, 70.2 MB/s 2024-04-25T10:26:29.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131325 s, 79.8 MB/s 2024-04-25T10:26:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150122 s, 69.8 MB/s 2024-04-25T10:26:29.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189152 s, 55.4 MB/s 2024-04-25T10:26:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.066 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131414 s, 79.8 MB/s 2024-04-25T10:26:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150787 s, 69.5 MB/s 2024-04-25T10:26:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146115 s, 71.8 MB/s 2024-04-25T10:26:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143493 s, 73.1 MB/s 2024-04-25T10:26:29.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.131 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139175 s, 75.3 MB/s 2024-04-25T10:26:29.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:29.132 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:29.133 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:29.133 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:26:29.133 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:29.133 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:29.133 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:29.133 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140972 s, 74.4 MB/s 2024-04-25T10:26:29.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144265 s, 72.7 MB/s 2024-04-25T10:26:29.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.237 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014178 s, 74.0 MB/s 2024-04-25T10:26:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.251 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134212 s, 78.1 MB/s 2024-04-25T10:26:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132613 s, 79.1 MB/s 2024-04-25T10:26:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.283 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148715 s, 70.5 MB/s 2024-04-25T10:26:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145687 s, 72.0 MB/s 2024-04-25T10:26:29.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.315 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142976 s, 73.3 MB/s 2024-04-25T10:26:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147193 s, 71.2 MB/s 2024-04-25T10:26:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.347 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142139 s, 73.8 MB/s 2024-04-25T10:26:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014608 s, 71.8 MB/s 2024-04-25T10:26:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143343 s, 73.2 MB/s 2024-04-25T10:26:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.400 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182834 s, 57.4 MB/s 2024-04-25T10:26:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015539 s, 67.5 MB/s 2024-04-25T10:26:29.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136802 s, 76.6 MB/s 2024-04-25T10:26:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.449 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143313 s, 73.2 MB/s 2024-04-25T10:26:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147317 s, 71.2 MB/s 2024-04-25T10:26:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.481 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136331 s, 76.9 MB/s 2024-04-25T10:26:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01497 s, 70.0 MB/s 2024-04-25T10:26:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136092 s, 77.0 MB/s 2024-04-25T10:26:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175296 s, 59.8 MB/s 2024-04-25T10:26:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014139 s, 74.2 MB/s 2024-04-25T10:26:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131659 s, 79.6 MB/s 2024-04-25T10:26:29.571 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:29.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138502 s, 75.7 MB/s 2024-04-25T10:26:29.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160643 s, 65.3 MB/s 2024-04-25T10:26:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.611 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130971 s, 80.1 MB/s 2024-04-25T10:26:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148144 s, 70.8 MB/s 2024-04-25T10:26:29.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192728 s, 54.4 MB/s 2024-04-25T10:26:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014633 s, 71.7 MB/s 2024-04-25T10:26:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148263 s, 70.7 MB/s 2024-04-25T10:26:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191156 s, 54.9 MB/s 2024-04-25T10:26:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138608 s, 75.7 MB/s 2024-04-25T10:26:29.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:29.720 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:29.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.790 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136943 s, 76.6 MB/s 2024-04-25T10:26:29.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.808 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156967 s, 66.8 MB/s 2024-04-25T10:26:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190915 s, 54.9 MB/s 2024-04-25T10:26:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148453 s, 70.6 MB/s 2024-04-25T10:26:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128019 s, 81.9 MB/s 2024-04-25T10:26:29.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.876 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014484 s, 72.4 MB/s 2024-04-25T10:26:29.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.892 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138263 s, 75.8 MB/s 2024-04-25T10:26:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152879 s, 68.6 MB/s 2024-04-25T10:26:29.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013648 s, 76.8 MB/s 2024-04-25T10:26:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138066 s, 75.9 MB/s 2024-04-25T10:26:29.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177908 s, 58.9 MB/s 2024-04-25T10:26:29.975 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.975 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.975 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137252 s, 76.4 MB/s 2024-04-25T10:26:29.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:29.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:29.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020924 s, 50.1 MB/s 2024-04-25T10:26:30.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175649 s, 59.7 MB/s 2024-04-25T10:26:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141297 s, 74.2 MB/s 2024-04-25T10:26:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150751 s, 69.6 MB/s 2024-04-25T10:26:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182979 s, 57.3 MB/s 2024-04-25T10:26:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141758 s, 74.0 MB/s 2024-04-25T10:26:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135888 s, 77.2 MB/s 2024-04-25T10:26:30.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.119 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014908 s, 70.3 MB/s 2024-04-25T10:26:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152651 s, 68.7 MB/s 2024-04-25T10:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182209 s, 57.5 MB/s 2024-04-25T10:26:30.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.172 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147694 s, 71.0 MB/s 2024-04-25T10:26:30.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.188 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140521 s, 74.6 MB/s 2024-04-25T10:26:30.207 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.207 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.208 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177841 s, 59.0 MB/s 2024-04-25T10:26:30.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.225 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153007 s, 68.5 MB/s 2024-04-25T10:26:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138879 s, 75.5 MB/s 2024-04-25T10:26:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189103 s, 55.4 MB/s 2024-04-25T10:26:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149704 s, 70.0 MB/s 2024-04-25T10:26:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133675 s, 78.4 MB/s 2024-04-25T10:26:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182235 s, 57.5 MB/s 2024-04-25T10:26:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136577 s, 76.8 MB/s 2024-04-25T10:26:30.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:30.330 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:30.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.399 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139613 s, 75.1 MB/s 2024-04-25T10:26:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128276 s, 81.7 MB/s 2024-04-25T10:26:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148608 s, 70.6 MB/s 2024-04-25T10:26:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184667 s, 56.8 MB/s 2024-04-25T10:26:30.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133059 s, 78.8 MB/s 2024-04-25T10:26:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188584 s, 55.6 MB/s 2024-04-25T10:26:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137193 s, 76.4 MB/s 2024-04-25T10:26:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137426 s, 76.3 MB/s 2024-04-25T10:26:30.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187064 s, 56.1 MB/s 2024-04-25T10:26:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.554 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014372 s, 73.0 MB/s 2024-04-25T10:26:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.571 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152434 s, 68.8 MB/s 2024-04-25T10:26:30.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.589 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163849 s, 64.0 MB/s 2024-04-25T10:26:30.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.603 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124057 s, 84.5 MB/s 2024-04-25T10:26:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019021 s, 55.1 MB/s 2024-04-25T10:26:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.641 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150344 s, 69.7 MB/s 2024-04-25T10:26:30.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.657 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144077 s, 72.8 MB/s 2024-04-25T10:26:30.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.674 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151822 s, 69.1 MB/s 2024-04-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173407 s, 60.5 MB/s 2024-04-25T10:26:30.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014477 s, 72.4 MB/s 2024-04-25T10:26:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149589 s, 70.1 MB/s 2024-04-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184866 s, 56.7 MB/s 2024-04-25T10:26:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187895 s, 55.8 MB/s 2024-04-25T10:26:30.784 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.784 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.784 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149033 s, 70.4 MB/s 2024-04-25T10:26:30.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170795 s, 61.4 MB/s 2024-04-25T10:26:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133546 s, 78.5 MB/s 2024-04-25T10:26:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133673 s, 78.4 MB/s 2024-04-25T10:26:30.848 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.848 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.848 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129975 s, 80.7 MB/s 2024-04-25T10:26:30.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.865 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148647 s, 70.5 MB/s 2024-04-25T10:26:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137946 s, 76.0 MB/s 2024-04-25T10:26:30.895 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.895 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.895 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013366 s, 78.5 MB/s 2024-04-25T10:26:30.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.912 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143815 s, 72.9 MB/s 2024-04-25T10:26:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:30.927 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136464 s, 76.8 MB/s 2024-04-25T10:26:30.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:30.929 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:30.929 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:30.929 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:26:30.929 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:30.929 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:30.929 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:30.930 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:31.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.001 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.001 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154062 s, 68.1 MB/s 2024-04-25T10:26:31.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.022 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182898 s, 57.3 MB/s 2024-04-25T10:26:31.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138948 s, 75.5 MB/s 2024-04-25T10:26:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153937 s, 68.1 MB/s 2024-04-25T10:26:31.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141576 s, 74.1 MB/s 2024-04-25T10:26:31.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187838 s, 55.8 MB/s 2024-04-25T10:26:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179418 s, 58.4 MB/s 2024-04-25T10:26:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134827 s, 77.8 MB/s 2024-04-25T10:26:31.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.142 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140679 s, 74.5 MB/s 2024-04-25T10:26:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144225 s, 72.7 MB/s 2024-04-25T10:26:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141874 s, 73.9 MB/s 2024-04-25T10:26:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132794 s, 79.0 MB/s 2024-04-25T10:26:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151796 s, 69.1 MB/s 2024-04-25T10:26:31.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177277 s, 59.1 MB/s 2024-04-25T10:26:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.244 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166171 s, 63.1 MB/s 2024-04-25T10:26:31.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015083 s, 69.5 MB/s 2024-04-25T10:26:31.277 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.277 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.277 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144085 s, 72.8 MB/s 2024-04-25T10:26:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015794 s, 66.4 MB/s 2024-04-25T10:26:31.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145049 s, 72.3 MB/s 2024-04-25T10:26:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181281 s, 57.8 MB/s 2024-04-25T10:26:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148748 s, 70.5 MB/s 2024-04-25T10:26:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.363 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014015 s, 74.8 MB/s 2024-04-25T10:26:31.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.380 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145903 s, 71.9 MB/s 2024-04-25T10:26:31.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013102 s, 80.0 MB/s 2024-04-25T10:26:31.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.410 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140222 s, 74.8 MB/s 2024-04-25T10:26:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.425 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126086 s, 83.2 MB/s 2024-04-25T10:26:31.440 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.441 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138075 s, 75.9 MB/s 2024-04-25T10:26:31.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.457 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142428 s, 73.6 MB/s 2024-04-25T10:26:31.473 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.473 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.473 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148836 s, 70.5 MB/s 2024-04-25T10:26:31.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.490 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014931 s, 70.2 MB/s 2024-04-25T10:26:31.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138565 s, 75.7 MB/s 2024-04-25T10:26:31.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142333 s, 73.7 MB/s 2024-04-25T10:26:31.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:31.523 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.596 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146807 s, 71.4 MB/s 2024-04-25T10:26:31.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144461 s, 72.6 MB/s 2024-04-25T10:26:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.629 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150863 s, 69.5 MB/s 2024-04-25T10:26:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150201 s, 69.8 MB/s 2024-04-25T10:26:31.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.661 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142509 s, 73.6 MB/s 2024-04-25T10:26:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133931 s, 78.3 MB/s 2024-04-25T10:26:31.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148032 s, 70.8 MB/s 2024-04-25T10:26:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134843 s, 77.8 MB/s 2024-04-25T10:26:31.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144358 s, 72.6 MB/s 2024-04-25T10:26:31.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142129 s, 73.8 MB/s 2024-04-25T10:26:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137497 s, 76.3 MB/s 2024-04-25T10:26:31.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137939 s, 76.0 MB/s 2024-04-25T10:26:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129056 s, 81.2 MB/s 2024-04-25T10:26:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139114 s, 75.4 MB/s 2024-04-25T10:26:31.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149226 s, 70.3 MB/s 2024-04-25T10:26:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135548 s, 77.4 MB/s 2024-04-25T10:26:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.850 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132359 s, 79.2 MB/s 2024-04-25T10:26:31.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155334 s, 67.5 MB/s 2024-04-25T10:26:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136791 s, 76.7 MB/s 2024-04-25T10:26:31.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.900 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152252 s, 68.9 MB/s 2024-04-25T10:26:31.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148975 s, 70.4 MB/s 2024-04-25T10:26:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013527 s, 77.5 MB/s 2024-04-25T10:26:31.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013279 s, 79.0 MB/s 2024-04-25T10:26:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.962 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132111 s, 79.4 MB/s 2024-04-25T10:26:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142731 s, 73.5 MB/s 2024-04-25T10:26:31.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:31.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:31.995 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148944 s, 70.4 MB/s 2024-04-25T10:26:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153991 s, 68.1 MB/s 2024-04-25T10:26:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129568 s, 80.9 MB/s 2024-04-25T10:26:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153442 s, 68.3 MB/s 2024-04-25T10:26:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144647 s, 72.5 MB/s 2024-04-25T10:26:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.081 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181993 s, 57.6 MB/s 2024-04-25T10:26:32.097 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154536 s, 67.9 MB/s 2024-04-25T10:26:32.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:32.099 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:32.099 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:32.099 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:26:32.099 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:32.099 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:32.099 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:32.100 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.170 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133193 s, 78.7 MB/s 2024-04-25T10:26:32.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147556 s, 71.1 MB/s 2024-04-25T10:26:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.202 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135101 s, 77.6 MB/s 2024-04-25T10:26:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.218 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140578 s, 74.6 MB/s 2024-04-25T10:26:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128372 s, 81.7 MB/s 2024-04-25T10:26:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.249 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145513 s, 72.1 MB/s 2024-04-25T10:26:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134576 s, 77.9 MB/s 2024-04-25T10:26:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135141 s, 77.6 MB/s 2024-04-25T10:26:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.296 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144279 s, 72.7 MB/s 2024-04-25T10:26:32.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138432 s, 75.7 MB/s 2024-04-25T10:26:32.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138234 s, 75.9 MB/s 2024-04-25T10:26:32.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142864 s, 73.4 MB/s 2024-04-25T10:26:32.359 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.359 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.359 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141812 s, 73.9 MB/s 2024-04-25T10:26:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151773 s, 69.1 MB/s 2024-04-25T10:26:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134835 s, 77.8 MB/s 2024-04-25T10:26:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130973 s, 80.1 MB/s 2024-04-25T10:26:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151268 s, 69.3 MB/s 2024-04-25T10:26:32.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.440 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148525 s, 70.6 MB/s 2024-04-25T10:26:32.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.456 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144686 s, 72.5 MB/s 2024-04-25T10:26:32.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.472 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140657 s, 74.5 MB/s 2024-04-25T10:26:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151943 s, 69.0 MB/s 2024-04-25T10:26:32.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.505 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134163 s, 78.2 MB/s 2024-04-25T10:26:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.520 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144538 s, 72.5 MB/s 2024-04-25T10:26:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140097 s, 74.8 MB/s 2024-04-25T10:26:32.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013653 s, 76.8 MB/s 2024-04-25T10:26:32.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150806 s, 69.5 MB/s 2024-04-25T10:26:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.585 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158056 s, 66.3 MB/s 2024-04-25T10:26:32.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.601 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141718 s, 74.0 MB/s 2024-04-25T10:26:32.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.616 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128396 s, 81.7 MB/s 2024-04-25T10:26:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.631 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137982 s, 76.0 MB/s 2024-04-25T10:26:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134235 s, 78.1 MB/s 2024-04-25T10:26:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015105 s, 69.4 MB/s 2024-04-25T10:26:32.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:32.665 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:32.665 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:32.665 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:26:32.665 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:32.666 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:32.666 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:32.666 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:32.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.735 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138595 s, 75.7 MB/s 2024-04-25T10:26:32.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.752 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152741 s, 68.7 MB/s 2024-04-25T10:26:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.768 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147134 s, 71.3 MB/s 2024-04-25T10:26:32.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.784 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.784 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133706 s, 78.4 MB/s 2024-04-25T10:26:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.800 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146278 s, 71.7 MB/s 2024-04-25T10:26:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128855 s, 81.4 MB/s 2024-04-25T10:26:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.830 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134779 s, 77.8 MB/s 2024-04-25T10:26:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131383 s, 79.8 MB/s 2024-04-25T10:26:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013583 s, 77.2 MB/s 2024-04-25T10:26:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.876 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139828 s, 75.0 MB/s 2024-04-25T10:26:32.891 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137895 s, 76.0 MB/s 2024-04-25T10:26:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137358 s, 76.3 MB/s 2024-04-25T10:26:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129601 s, 80.9 MB/s 2024-04-25T10:26:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.938 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138675 s, 75.6 MB/s 2024-04-25T10:26:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147605 s, 71.0 MB/s 2024-04-25T10:26:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013537 s, 77.5 MB/s 2024-04-25T10:26:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143314 s, 73.2 MB/s 2024-04-25T10:26:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139627 s, 75.1 MB/s 2024-04-25T10:26:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.017 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137005 s, 76.5 MB/s 2024-04-25T10:26:33.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144496 s, 72.6 MB/s 2024-04-25T10:26:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154964 s, 67.7 MB/s 2024-04-25T10:26:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156921 s, 66.8 MB/s 2024-04-25T10:26:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145195 s, 72.2 MB/s 2024-04-25T10:26:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.101 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147753 s, 71.0 MB/s 2024-04-25T10:26:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154065 s, 68.1 MB/s 2024-04-25T10:26:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.135 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150314 s, 69.8 MB/s 2024-04-25T10:26:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137247 s, 76.4 MB/s 2024-04-25T10:26:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127126 s, 82.5 MB/s 2024-04-25T10:26:33.181 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.181 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.181 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133547 s, 78.5 MB/s 2024-04-25T10:26:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142184 s, 73.7 MB/s 2024-04-25T10:26:33.214 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.214 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.214 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153803 s, 68.2 MB/s 2024-04-25T10:26:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135872 s, 77.2 MB/s 2024-04-25T10:26:33.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:33.231 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:33.231 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:33.231 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:26:33.231 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:33.231 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:33.231 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:33.232 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135646 s, 77.3 MB/s 2024-04-25T10:26:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142085 s, 73.8 MB/s 2024-04-25T10:26:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159145 s, 65.9 MB/s 2024-04-25T10:26:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.353 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154749 s, 67.8 MB/s 2024-04-25T10:26:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135368 s, 77.5 MB/s 2024-04-25T10:26:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.386 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158145 s, 66.3 MB/s 2024-04-25T10:26:33.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184803 s, 56.7 MB/s 2024-04-25T10:26:33.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014114 s, 74.3 MB/s 2024-04-25T10:26:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139425 s, 75.2 MB/s 2024-04-25T10:26:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.455 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152255 s, 68.9 MB/s 2024-04-25T10:26:33.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.476 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185158 s, 56.6 MB/s 2024-04-25T10:26:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.490 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128067 s, 81.9 MB/s 2024-04-25T10:26:33.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139592 s, 75.1 MB/s 2024-04-25T10:26:33.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139489 s, 75.2 MB/s 2024-04-25T10:26:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147202 s, 71.2 MB/s 2024-04-25T10:26:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180024 s, 58.2 MB/s 2024-04-25T10:26:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178015 s, 58.9 MB/s 2024-04-25T10:26:33.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.594 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145349 s, 72.1 MB/s 2024-04-25T10:26:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.610 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139302 s, 75.3 MB/s 2024-04-25T10:26:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131967 s, 79.5 MB/s 2024-04-25T10:26:33.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131757 s, 79.6 MB/s 2024-04-25T10:26:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153335 s, 68.4 MB/s 2024-04-25T10:26:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135361 s, 77.5 MB/s 2024-04-25T10:26:33.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.689 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153066 s, 68.5 MB/s 2024-04-25T10:26:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.706 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153377 s, 68.4 MB/s 2024-04-25T10:26:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174601 s, 60.1 MB/s 2024-04-25T10:26:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.743 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152071 s, 69.0 MB/s 2024-04-25T10:26:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135408 s, 77.4 MB/s 2024-04-25T10:26:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185252 s, 56.6 MB/s 2024-04-25T10:26:33.794 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.794 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139828 s, 75.0 MB/s 2024-04-25T10:26:33.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.815 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189895 s, 55.2 MB/s 2024-04-25T10:26:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.833 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151152 s, 69.4 MB/s 2024-04-25T10:26:33.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:33.834 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139854 s, 75.0 MB/s 2024-04-25T10:26:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139459 s, 75.2 MB/s 2024-04-25T10:26:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014631 s, 71.7 MB/s 2024-04-25T10:26:33.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131231 s, 79.9 MB/s 2024-04-25T10:26:33.968 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.969 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129439 s, 81.0 MB/s 2024-04-25T10:26:33.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:33.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:33.988 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174204 s, 60.2 MB/s 2024-04-25T10:26:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155476 s, 67.4 MB/s 2024-04-25T10:26:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.022 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148309 s, 70.7 MB/s 2024-04-25T10:26:34.036 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131227 s, 79.9 MB/s 2024-04-25T10:26:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143609 s, 73.0 MB/s 2024-04-25T10:26:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153102 s, 68.5 MB/s 2024-04-25T10:26:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130337 s, 80.5 MB/s 2024-04-25T10:26:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.100 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130288 s, 80.5 MB/s 2024-04-25T10:26:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015139 s, 69.3 MB/s 2024-04-25T10:26:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135719 s, 77.3 MB/s 2024-04-25T10:26:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190326 s, 55.1 MB/s 2024-04-25T10:26:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154061 s, 68.1 MB/s 2024-04-25T10:26:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178054 s, 58.9 MB/s 2024-04-25T10:26:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163164 s, 64.3 MB/s 2024-04-25T10:26:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147933 s, 70.9 MB/s 2024-04-25T10:26:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143537 s, 73.1 MB/s 2024-04-25T10:26:34.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137187 s, 76.4 MB/s 2024-04-25T10:26:34.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.273 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135645 s, 77.3 MB/s 2024-04-25T10:26:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.289 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138 s, 76.0 MB/s 2024-04-25T10:26:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151949 s, 69.0 MB/s 2024-04-25T10:26:34.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191266 s, 54.8 MB/s 2024-04-25T10:26:34.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015337 s, 68.4 MB/s 2024-04-25T10:26:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139478 s, 75.2 MB/s 2024-04-25T10:26:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.379 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177858 s, 59.0 MB/s 2024-04-25T10:26:34.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.394 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130188 s, 80.5 MB/s 2024-04-25T10:26:34.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.409 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132264 s, 79.3 MB/s 2024-04-25T10:26:34.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.429 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018 s, 58.3 MB/s 2024-04-25T10:26:34.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:34.430 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:34.430 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:34.430 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:26:34.430 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:34.430 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:34.430 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:34.431 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172878 s, 60.7 MB/s 2024-04-25T10:26:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143853 s, 72.9 MB/s 2024-04-25T10:26:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138856 s, 75.5 MB/s 2024-04-25T10:26:34.557 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.557 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.557 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175772 s, 59.7 MB/s 2024-04-25T10:26:34.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.572 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129108 s, 81.2 MB/s 2024-04-25T10:26:34.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.587 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136978 s, 76.6 MB/s 2024-04-25T10:26:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140983 s, 74.4 MB/s 2024-04-25T10:26:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143616 s, 73.0 MB/s 2024-04-25T10:26:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151284 s, 69.3 MB/s 2024-04-25T10:26:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137147 s, 76.5 MB/s 2024-04-25T10:26:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.668 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147305 s, 71.2 MB/s 2024-04-25T10:26:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177758 s, 59.0 MB/s 2024-04-25T10:26:34.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.705 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015778 s, 66.5 MB/s 2024-04-25T10:26:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.723 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156511 s, 67.0 MB/s 2024-04-25T10:26:34.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.738 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136054 s, 77.1 MB/s 2024-04-25T10:26:34.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.755 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149424 s, 70.2 MB/s 2024-04-25T10:26:34.770 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.771 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140356 s, 74.7 MB/s 2024-04-25T10:26:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156773 s, 66.9 MB/s 2024-04-25T10:26:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014436 s, 72.6 MB/s 2024-04-25T10:26:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148346 s, 70.7 MB/s 2024-04-25T10:26:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144743 s, 72.4 MB/s 2024-04-25T10:26:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.853 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143252 s, 73.2 MB/s 2024-04-25T10:26:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.870 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149121 s, 70.3 MB/s 2024-04-25T10:26:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.886 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142627 s, 73.5 MB/s 2024-04-25T10:26:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144737 s, 72.4 MB/s 2024-04-25T10:26:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132477 s, 79.2 MB/s 2024-04-25T10:26:34.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150586 s, 69.6 MB/s 2024-04-25T10:26:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142756 s, 73.5 MB/s 2024-04-25T10:26:34.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.965 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.965 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134469 s, 78.0 MB/s 2024-04-25T10:26:34.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.982 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150729 s, 69.6 MB/s 2024-04-25T10:26:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:34.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140947 s, 74.4 MB/s 2024-04-25T10:26:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148784 s, 70.5 MB/s 2024-04-25T10:26:35.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:35.016 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:35.017 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:35.017 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:26:35.017 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:35.017 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:35.017 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:35.017 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148023 s, 70.8 MB/s 2024-04-25T10:26:35.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150928 s, 69.5 MB/s 2024-04-25T10:26:35.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152022 s, 69.0 MB/s 2024-04-25T10:26:35.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013186 s, 79.5 MB/s 2024-04-25T10:26:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01514 s, 69.3 MB/s 2024-04-25T10:26:35.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147757 s, 71.0 MB/s 2024-04-25T10:26:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169813 s, 61.7 MB/s 2024-04-25T10:26:35.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145466 s, 72.1 MB/s 2024-04-25T10:26:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149242 s, 70.3 MB/s 2024-04-25T10:26:35.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153647 s, 68.2 MB/s 2024-04-25T10:26:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.260 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017975 s, 58.3 MB/s 2024-04-25T10:26:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013118 s, 79.9 MB/s 2024-04-25T10:26:35.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151657 s, 69.1 MB/s 2024-04-25T10:26:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147828 s, 70.9 MB/s 2024-04-25T10:26:35.324 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.324 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.324 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144934 s, 72.3 MB/s 2024-04-25T10:26:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147665 s, 71.0 MB/s 2024-04-25T10:26:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189685 s, 55.3 MB/s 2024-04-25T10:26:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.379 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156373 s, 67.1 MB/s 2024-04-25T10:26:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.394 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133375 s, 78.6 MB/s 2024-04-25T10:26:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.411 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137074 s, 76.5 MB/s 2024-04-25T10:26:35.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.426 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139472 s, 75.2 MB/s 2024-04-25T10:26:35.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.443 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149036 s, 70.4 MB/s 2024-04-25T10:26:35.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139395 s, 75.2 MB/s 2024-04-25T10:26:35.473 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.474 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134198 s, 78.1 MB/s 2024-04-25T10:26:35.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012807 s, 81.9 MB/s 2024-04-25T10:26:35.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.504 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135129 s, 77.6 MB/s 2024-04-25T10:26:35.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.519 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137656 s, 76.2 MB/s 2024-04-25T10:26:35.540 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.540 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.541 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194163 s, 54.0 MB/s 2024-04-25T10:26:35.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138047 s, 76.0 MB/s 2024-04-25T10:26:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.573 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149989 s, 69.9 MB/s 2024-04-25T10:26:35.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150748 s, 69.6 MB/s 2024-04-25T10:26:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015363 s, 68.3 MB/s 2024-04-25T10:26:35.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:35.608 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:35.608 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:35.608 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:26:35.608 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:35.609 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:35.609 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:35.609 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.679 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139101 s, 75.4 MB/s 2024-04-25T10:26:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.694 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136183 s, 77.0 MB/s 2024-04-25T10:26:35.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.710 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.710 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139599 s, 75.1 MB/s 2024-04-25T10:26:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.726 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147647 s, 71.0 MB/s 2024-04-25T10:26:35.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189695 s, 55.3 MB/s 2024-04-25T10:26:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.764 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146854 s, 71.4 MB/s 2024-04-25T10:26:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.780 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150258 s, 69.8 MB/s 2024-04-25T10:26:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.796 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139054 s, 75.4 MB/s 2024-04-25T10:26:35.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.811 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013832 s, 75.8 MB/s 2024-04-25T10:26:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.829 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150953 s, 69.5 MB/s 2024-04-25T10:26:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145331 s, 72.2 MB/s 2024-04-25T10:26:35.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.865 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183324 s, 57.2 MB/s 2024-04-25T10:26:35.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137248 s, 76.4 MB/s 2024-04-25T10:26:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142863 s, 73.4 MB/s 2024-04-25T10:26:35.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181065 s, 57.9 MB/s 2024-04-25T10:26:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144264 s, 72.7 MB/s 2024-04-25T10:26:35.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150982 s, 69.5 MB/s 2024-04-25T10:26:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.966 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152684 s, 68.7 MB/s 2024-04-25T10:26:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:35.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180958 s, 57.9 MB/s 2024-04-25T10:26:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.002 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146007 s, 71.8 MB/s 2024-04-25T10:26:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138427 s, 75.7 MB/s 2024-04-25T10:26:36.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177782 s, 59.0 MB/s 2024-04-25T10:26:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147643 s, 71.0 MB/s 2024-04-25T10:26:36.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133183 s, 78.7 MB/s 2024-04-25T10:26:36.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149026 s, 70.4 MB/s 2024-04-25T10:26:36.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174949 s, 59.9 MB/s 2024-04-25T10:26:36.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136312 s, 76.9 MB/s 2024-04-25T10:26:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183973 s, 57.0 MB/s 2024-04-25T10:26:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141779 s, 74.0 MB/s 2024-04-25T10:26:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148866 s, 70.4 MB/s 2024-04-25T10:26:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013169 s, 79.6 MB/s 2024-04-25T10:26:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155378 s, 67.5 MB/s 2024-04-25T10:26:36.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:36.207 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142406 s, 73.6 MB/s 2024-04-25T10:26:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015091 s, 69.5 MB/s 2024-04-25T10:26:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134943 s, 77.7 MB/s 2024-04-25T10:26:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146112 s, 71.8 MB/s 2024-04-25T10:26:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124686 s, 84.1 MB/s 2024-04-25T10:26:36.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151543 s, 69.2 MB/s 2024-04-25T10:26:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143723 s, 73.0 MB/s 2024-04-25T10:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140638 s, 74.6 MB/s 2024-04-25T10:26:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138495 s, 75.7 MB/s 2024-04-25T10:26:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140052 s, 74.9 MB/s 2024-04-25T10:26:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171107 s, 61.3 MB/s 2024-04-25T10:26:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142518 s, 73.6 MB/s 2024-04-25T10:26:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.473 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135188 s, 77.6 MB/s 2024-04-25T10:26:36.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.490 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150288 s, 69.8 MB/s 2024-04-25T10:26:36.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110566 s, 94.8 MB/s 2024-04-25T10:26:36.516 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.517 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128534 s, 81.6 MB/s 2024-04-25T10:26:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151443 s, 69.2 MB/s 2024-04-25T10:26:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147112 s, 71.3 MB/s 2024-04-25T10:26:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.565 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132473 s, 79.2 MB/s 2024-04-25T10:26:36.581 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.581 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.581 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138756 s, 75.6 MB/s 2024-04-25T10:26:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141548 s, 74.1 MB/s 2024-04-25T10:26:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151664 s, 69.1 MB/s 2024-04-25T10:26:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129754 s, 80.8 MB/s 2024-04-25T10:26:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014986 s, 70.0 MB/s 2024-04-25T10:26:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155748 s, 67.3 MB/s 2024-04-25T10:26:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.679 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149221 s, 70.3 MB/s 2024-04-25T10:26:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.698 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164431 s, 63.8 MB/s 2024-04-25T10:26:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.712 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129204 s, 81.2 MB/s 2024-04-25T10:26:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015289 s, 68.6 MB/s 2024-04-25T10:26:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147003 s, 71.3 MB/s 2024-04-25T10:26:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151488 s, 69.2 MB/s 2024-04-25T10:26:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.779 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140845 s, 74.4 MB/s 2024-04-25T10:26:36.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:36.780 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:36.780 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:36.780 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:26:36.780 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:36.780 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:36.781 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:36.781 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:36.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.852 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150607 s, 69.6 MB/s 2024-04-25T10:26:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.868 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014644 s, 71.6 MB/s 2024-04-25T10:26:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147064 s, 71.3 MB/s 2024-04-25T10:26:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.901 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146459 s, 71.6 MB/s 2024-04-25T10:26:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135406 s, 77.4 MB/s 2024-04-25T10:26:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136011 s, 77.1 MB/s 2024-04-25T10:26:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139253 s, 75.3 MB/s 2024-04-25T10:26:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.964 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148229 s, 70.7 MB/s 2024-04-25T10:26:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.980 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145786 s, 71.9 MB/s 2024-04-25T10:26:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:36.997 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148808 s, 70.5 MB/s 2024-04-25T10:26:37.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136142 s, 77.0 MB/s 2024-04-25T10:26:37.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.029 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014176 s, 74.0 MB/s 2024-04-25T10:26:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137427 s, 76.3 MB/s 2024-04-25T10:26:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131399 s, 79.8 MB/s 2024-04-25T10:26:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149739 s, 70.0 MB/s 2024-04-25T10:26:37.092 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.093 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149025 s, 70.4 MB/s 2024-04-25T10:26:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.109 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.110 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150695 s, 69.6 MB/s 2024-04-25T10:26:37.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.126 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144928 s, 72.4 MB/s 2024-04-25T10:26:37.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.142 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142467 s, 73.6 MB/s 2024-04-25T10:26:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146035 s, 71.8 MB/s 2024-04-25T10:26:37.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145482 s, 72.1 MB/s 2024-04-25T10:26:37.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.191 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142191 s, 73.7 MB/s 2024-04-25T10:26:37.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133823 s, 78.4 MB/s 2024-04-25T10:26:37.221 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.221 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.221 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134605 s, 77.9 MB/s 2024-04-25T10:26:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136871 s, 76.6 MB/s 2024-04-25T10:26:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137923 s, 76.0 MB/s 2024-04-25T10:26:37.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147848 s, 70.9 MB/s 2024-04-25T10:26:37.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.284 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134242 s, 78.1 MB/s 2024-04-25T10:26:37.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180479 s, 58.1 MB/s 2024-04-25T10:26:37.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.320 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140524 s, 74.6 MB/s 2024-04-25T10:26:37.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.340 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188284 s, 55.7 MB/s 2024-04-25T10:26:37.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159833 s, 65.6 MB/s 2024-04-25T10:26:37.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:37.359 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:37.359 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:37.359 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:26:37.359 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:37.359 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:37.360 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:37.360 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.435 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131542 s, 79.7 MB/s 2024-04-25T10:26:37.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.450 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141377 s, 74.2 MB/s 2024-04-25T10:26:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180053 s, 58.2 MB/s 2024-04-25T10:26:37.485 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140048 s, 74.9 MB/s 2024-04-25T10:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195228 s, 53.7 MB/s 2024-04-25T10:26:37.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150543 s, 69.7 MB/s 2024-04-25T10:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138033 s, 76.0 MB/s 2024-04-25T10:26:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135331 s, 77.5 MB/s 2024-04-25T10:26:37.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133954 s, 78.3 MB/s 2024-04-25T10:26:37.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186068 s, 56.4 MB/s 2024-04-25T10:26:37.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151455 s, 69.2 MB/s 2024-04-25T10:26:37.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148103 s, 70.8 MB/s 2024-04-25T10:26:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145447 s, 72.1 MB/s 2024-04-25T10:26:37.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.655 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013561 s, 77.3 MB/s 2024-04-25T10:26:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137459 s, 76.3 MB/s 2024-04-25T10:26:37.688 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.688 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.688 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151486 s, 69.2 MB/s 2024-04-25T10:26:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.708 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186308 s, 56.3 MB/s 2024-04-25T10:26:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138579 s, 75.7 MB/s 2024-04-25T10:26:37.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.745 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185727 s, 56.5 MB/s 2024-04-25T10:26:37.760 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.761 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141257 s, 74.2 MB/s 2024-04-25T10:26:37.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.781 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180755 s, 58.0 MB/s 2024-04-25T10:26:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147755 s, 71.0 MB/s 2024-04-25T10:26:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.813 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144345 s, 72.6 MB/s 2024-04-25T10:26:37.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.829 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139567 s, 75.1 MB/s 2024-04-25T10:26:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.850 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196012 s, 53.5 MB/s 2024-04-25T10:26:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148938 s, 70.4 MB/s 2024-04-25T10:26:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135216 s, 77.5 MB/s 2024-04-25T10:26:37.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.897 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133579 s, 78.5 MB/s 2024-04-25T10:26:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141079 s, 74.3 MB/s 2024-04-25T10:26:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140701 s, 74.5 MB/s 2024-04-25T10:26:37.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181074 s, 57.9 MB/s 2024-04-25T10:26:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133579 s, 78.5 MB/s 2024-04-25T10:26:37.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:37.965 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:37.965 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:37.965 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:26:37.965 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:37.965 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:37.965 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:37.966 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.036 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148983 s, 70.4 MB/s 2024-04-25T10:26:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.055 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173488 s, 60.4 MB/s 2024-04-25T10:26:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133553 s, 78.5 MB/s 2024-04-25T10:26:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135481 s, 77.4 MB/s 2024-04-25T10:26:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.101 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136534 s, 76.8 MB/s 2024-04-25T10:26:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147513 s, 71.1 MB/s 2024-04-25T10:26:38.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.134 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153832 s, 68.2 MB/s 2024-04-25T10:26:38.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175623 s, 59.7 MB/s 2024-04-25T10:26:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.170 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137715 s, 76.1 MB/s 2024-04-25T10:26:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176533 s, 59.4 MB/s 2024-04-25T10:26:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150255 s, 69.8 MB/s 2024-04-25T10:26:38.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136144 s, 77.0 MB/s 2024-04-25T10:26:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.237 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145008 s, 72.3 MB/s 2024-04-25T10:26:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135496 s, 77.4 MB/s 2024-04-25T10:26:38.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.267 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133519 s, 78.5 MB/s 2024-04-25T10:26:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130797 s, 80.2 MB/s 2024-04-25T10:26:38.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154642 s, 67.8 MB/s 2024-04-25T10:26:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.319 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017827 s, 58.8 MB/s 2024-04-25T10:26:38.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.335 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136789 s, 76.7 MB/s 2024-04-25T10:26:38.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.351 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143875 s, 72.9 MB/s 2024-04-25T10:26:38.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.367 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139647 s, 75.1 MB/s 2024-04-25T10:26:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136943 s, 76.6 MB/s 2024-04-25T10:26:38.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139096 s, 75.4 MB/s 2024-04-25T10:26:38.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137778 s, 76.1 MB/s 2024-04-25T10:26:38.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.435 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193783 s, 54.1 MB/s 2024-04-25T10:26:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.450 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138318 s, 75.8 MB/s 2024-04-25T10:26:38.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.468 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153438 s, 68.3 MB/s 2024-04-25T10:26:38.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.484 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143049 s, 73.3 MB/s 2024-04-25T10:26:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134453 s, 78.0 MB/s 2024-04-25T10:26:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141975 s, 73.9 MB/s 2024-04-25T10:26:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.530 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138895 s, 75.5 MB/s 2024-04-25T10:26:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138912 s, 75.5 MB/s 2024-04-25T10:26:38.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:38.548 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.619 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136618 s, 76.8 MB/s 2024-04-25T10:26:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147561 s, 71.1 MB/s 2024-04-25T10:26:38.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.651 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138598 s, 75.7 MB/s 2024-04-25T10:26:38.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149505 s, 70.1 MB/s 2024-04-25T10:26:38.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141394 s, 74.2 MB/s 2024-04-25T10:26:38.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.701 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.701 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153034 s, 68.5 MB/s 2024-04-25T10:26:38.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.716 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137788 s, 76.1 MB/s 2024-04-25T10:26:38.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133976 s, 78.3 MB/s 2024-04-25T10:26:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140599 s, 74.6 MB/s 2024-04-25T10:26:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.764 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151603 s, 69.2 MB/s 2024-04-25T10:26:38.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.780 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014207 s, 73.8 MB/s 2024-04-25T10:26:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137618 s, 76.2 MB/s 2024-04-25T10:26:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.811 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135802 s, 77.2 MB/s 2024-04-25T10:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014266 s, 73.5 MB/s 2024-04-25T10:26:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157685 s, 66.5 MB/s 2024-04-25T10:26:38.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.859 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133746 s, 78.4 MB/s 2024-04-25T10:26:38.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.874 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129298 s, 81.1 MB/s 2024-04-25T10:26:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132957 s, 78.9 MB/s 2024-04-25T10:26:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132945 s, 78.9 MB/s 2024-04-25T10:26:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178786 s, 58.6 MB/s 2024-04-25T10:26:38.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142064 s, 73.8 MB/s 2024-04-25T10:26:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.957 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150555 s, 69.6 MB/s 2024-04-25T10:26:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183841 s, 57.0 MB/s 2024-04-25T10:26:38.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:38.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:38.995 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155564 s, 67.4 MB/s 2024-04-25T10:26:39.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182121 s, 57.6 MB/s 2024-04-25T10:26:39.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.028 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115434 s, 90.8 MB/s 2024-04-25T10:26:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014098 s, 74.4 MB/s 2024-04-25T10:26:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136941 s, 76.6 MB/s 2024-04-25T10:26:39.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.077 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158192 s, 66.3 MB/s 2024-04-25T10:26:39.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.092 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.092 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132456 s, 79.2 MB/s 2024-04-25T10:26:39.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135436 s, 77.4 MB/s 2024-04-25T10:26:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135879 s, 77.2 MB/s 2024-04-25T10:26:39.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:39.124 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:39.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206643 s, 50.7 MB/s 2024-04-25T10:26:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015055 s, 69.6 MB/s 2024-04-25T10:26:39.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01894 s, 55.4 MB/s 2024-04-25T10:26:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160127 s, 65.5 MB/s 2024-04-25T10:26:39.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138272 s, 75.8 MB/s 2024-04-25T10:26:39.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.290 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.290 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132406 s, 79.2 MB/s 2024-04-25T10:26:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.305 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137655 s, 76.2 MB/s 2024-04-25T10:26:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141513 s, 74.1 MB/s 2024-04-25T10:26:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131722 s, 79.6 MB/s 2024-04-25T10:26:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137355 s, 76.3 MB/s 2024-04-25T10:26:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147044 s, 71.3 MB/s 2024-04-25T10:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013912 s, 75.4 MB/s 2024-04-25T10:26:39.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152792 s, 68.6 MB/s 2024-04-25T10:26:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187196 s, 56.0 MB/s 2024-04-25T10:26:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014369 s, 73.0 MB/s 2024-04-25T10:26:39.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016469 s, 63.7 MB/s 2024-04-25T10:26:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173115 s, 60.6 MB/s 2024-04-25T10:26:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.492 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.493 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015637 s, 67.1 MB/s 2024-04-25T10:26:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.508 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137609 s, 76.2 MB/s 2024-04-25T10:26:39.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145469 s, 72.1 MB/s 2024-04-25T10:26:39.544 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.544 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.544 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184267 s, 56.9 MB/s 2024-04-25T10:26:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.561 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01463 s, 71.7 MB/s 2024-04-25T10:26:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175939 s, 59.6 MB/s 2024-04-25T10:26:39.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.596 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140796 s, 74.5 MB/s 2024-04-25T10:26:39.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137836 s, 76.1 MB/s 2024-04-25T10:26:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185006 s, 56.7 MB/s 2024-04-25T10:26:39.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129689 s, 80.9 MB/s 2024-04-25T10:26:39.664 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.664 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.664 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151933 s, 69.0 MB/s 2024-04-25T10:26:39.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145397 s, 72.1 MB/s 2024-04-25T10:26:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.695 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129269 s, 81.1 MB/s 2024-04-25T10:26:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185562 s, 56.5 MB/s 2024-04-25T10:26:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142627 s, 73.5 MB/s 2024-04-25T10:26:39.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:39.733 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154561 s, 67.8 MB/s 2024-04-25T10:26:39.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142796 s, 73.4 MB/s 2024-04-25T10:26:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129043 s, 81.3 MB/s 2024-04-25T10:26:39.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143741 s, 72.9 MB/s 2024-04-25T10:26:39.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150213 s, 69.8 MB/s 2024-04-25T10:26:39.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142734 s, 73.5 MB/s 2024-04-25T10:26:39.897 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.898 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123184 s, 85.1 MB/s 2024-04-25T10:26:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179524 s, 58.4 MB/s 2024-04-25T10:26:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.933 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135842 s, 77.2 MB/s 2024-04-25T10:26:39.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.950 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153387 s, 68.4 MB/s 2024-04-25T10:26:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.966 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141232 s, 74.2 MB/s 2024-04-25T10:26:39.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.982 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144985 s, 72.3 MB/s 2024-04-25T10:26:39.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:39.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:39.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148444 s, 70.6 MB/s 2024-04-25T10:26:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154007 s, 68.1 MB/s 2024-04-25T10:26:40.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.032 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146732 s, 71.5 MB/s 2024-04-25T10:26:40.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129728 s, 80.8 MB/s 2024-04-25T10:26:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.062 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137367 s, 76.3 MB/s 2024-04-25T10:26:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.078 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140865 s, 74.4 MB/s 2024-04-25T10:26:40.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.095 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.095 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149703 s, 70.0 MB/s 2024-04-25T10:26:40.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.111 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137006 s, 76.5 MB/s 2024-04-25T10:26:40.127 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.127 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.127 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150746 s, 69.6 MB/s 2024-04-25T10:26:40.143 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.143 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.143 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135146 s, 77.6 MB/s 2024-04-25T10:26:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013845 s, 75.7 MB/s 2024-04-25T10:26:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155334 s, 67.5 MB/s 2024-04-25T10:26:40.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.196 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184227 s, 56.9 MB/s 2024-04-25T10:26:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150064 s, 69.9 MB/s 2024-04-25T10:26:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134718 s, 77.8 MB/s 2024-04-25T10:26:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134687 s, 77.9 MB/s 2024-04-25T10:26:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013619 s, 77.0 MB/s 2024-04-25T10:26:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013296 s, 78.9 MB/s 2024-04-25T10:26:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165983 s, 63.2 MB/s 2024-04-25T10:26:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.309 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151089 s, 69.4 MB/s 2024-04-25T10:26:40.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:40.311 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189797 s, 55.2 MB/s 2024-04-25T10:26:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112451 s, 93.2 MB/s 2024-04-25T10:26:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.417 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144272 s, 72.7 MB/s 2024-04-25T10:26:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014314 s, 73.3 MB/s 2024-04-25T10:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135431 s, 77.4 MB/s 2024-04-25T10:26:40.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152216 s, 68.9 MB/s 2024-04-25T10:26:40.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.481 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01337 s, 78.4 MB/s 2024-04-25T10:26:40.496 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.496 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.496 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137373 s, 76.3 MB/s 2024-04-25T10:26:40.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.513 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.513 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146435 s, 71.6 MB/s 2024-04-25T10:26:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.528 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137285 s, 76.4 MB/s 2024-04-25T10:26:40.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.545 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015354 s, 68.3 MB/s 2024-04-25T10:26:40.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.565 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018094 s, 58.0 MB/s 2024-04-25T10:26:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.582 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152771 s, 68.6 MB/s 2024-04-25T10:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147977 s, 70.9 MB/s 2024-04-25T10:26:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178479 s, 58.8 MB/s 2024-04-25T10:26:40.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.635 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.635 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146273 s, 71.7 MB/s 2024-04-25T10:26:40.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129891 s, 80.7 MB/s 2024-04-25T10:26:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134293 s, 78.1 MB/s 2024-04-25T10:26:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197095 s, 53.2 MB/s 2024-04-25T10:26:40.701 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132954 s, 78.9 MB/s 2024-04-25T10:26:40.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.722 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186084 s, 56.3 MB/s 2024-04-25T10:26:40.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.738 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013598 s, 77.1 MB/s 2024-04-25T10:26:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136774 s, 76.7 MB/s 2024-04-25T10:26:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135538 s, 77.4 MB/s 2024-04-25T10:26:40.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.785 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146153 s, 71.7 MB/s 2024-04-25T10:26:40.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.800 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133938 s, 78.3 MB/s 2024-04-25T10:26:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.817 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149689 s, 70.1 MB/s 2024-04-25T10:26:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127061 s, 82.5 MB/s 2024-04-25T10:26:40.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.847 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014166 s, 74.0 MB/s 2024-04-25T10:26:40.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.862 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131156 s, 79.9 MB/s 2024-04-25T10:26:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152616 s, 68.7 MB/s 2024-04-25T10:26:40.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150281 s, 69.8 MB/s 2024-04-25T10:26:40.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:40.898 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.967 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144578 s, 72.5 MB/s 2024-04-25T10:26:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:40.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138725 s, 75.6 MB/s 2024-04-25T10:26:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.003 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189406 s, 55.4 MB/s 2024-04-25T10:26:41.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130292 s, 80.5 MB/s 2024-04-25T10:26:41.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141944 s, 73.9 MB/s 2024-04-25T10:26:41.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.050 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138832 s, 75.5 MB/s 2024-04-25T10:26:41.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.066 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.066 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145596 s, 72.0 MB/s 2024-04-25T10:26:41.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146043 s, 71.8 MB/s 2024-04-25T10:26:41.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146714 s, 71.5 MB/s 2024-04-25T10:26:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144147 s, 72.7 MB/s 2024-04-25T10:26:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133919 s, 78.3 MB/s 2024-04-25T10:26:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149844 s, 70.0 MB/s 2024-04-25T10:26:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013741 s, 76.3 MB/s 2024-04-25T10:26:41.177 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.178 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132017 s, 79.4 MB/s 2024-04-25T10:26:41.195 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.195 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.195 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151618 s, 69.2 MB/s 2024-04-25T10:26:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142491 s, 73.6 MB/s 2024-04-25T10:26:41.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132076 s, 79.4 MB/s 2024-04-25T10:26:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141791 s, 74.0 MB/s 2024-04-25T10:26:41.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147436 s, 71.1 MB/s 2024-04-25T10:26:41.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145058 s, 72.3 MB/s 2024-04-25T10:26:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013632 s, 76.9 MB/s 2024-04-25T10:26:41.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.306 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145065 s, 72.3 MB/s 2024-04-25T10:26:41.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.322 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013605 s, 77.1 MB/s 2024-04-25T10:26:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.337 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136466 s, 76.8 MB/s 2024-04-25T10:26:41.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013154 s, 79.7 MB/s 2024-04-25T10:26:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134534 s, 77.9 MB/s 2024-04-25T10:26:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139593 s, 75.1 MB/s 2024-04-25T10:26:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144912 s, 72.4 MB/s 2024-04-25T10:26:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167876 s, 62.5 MB/s 2024-04-25T10:26:41.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132991 s, 78.8 MB/s 2024-04-25T10:26:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134251 s, 78.1 MB/s 2024-04-25T10:26:41.464 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014609 s, 71.8 MB/s 2024-04-25T10:26:41.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:41.466 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:41.466 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:41.466 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:26:41.466 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:41.466 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:41.467 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:41.467 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147127 s, 71.3 MB/s 2024-04-25T10:26:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138539 s, 75.7 MB/s 2024-04-25T10:26:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131687 s, 79.6 MB/s 2024-04-25T10:26:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135101 s, 77.6 MB/s 2024-04-25T10:26:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154412 s, 67.9 MB/s 2024-04-25T10:26:41.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.619 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150043 s, 69.9 MB/s 2024-04-25T10:26:41.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151037 s, 69.4 MB/s 2024-04-25T10:26:41.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.655 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177174 s, 59.2 MB/s 2024-04-25T10:26:41.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132883 s, 78.9 MB/s 2024-04-25T10:26:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131139 s, 80.0 MB/s 2024-04-25T10:26:41.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132133 s, 79.4 MB/s 2024-04-25T10:26:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130579 s, 80.3 MB/s 2024-04-25T10:26:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143043 s, 73.3 MB/s 2024-04-25T10:26:41.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138711 s, 75.6 MB/s 2024-04-25T10:26:41.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.762 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013112 s, 80.0 MB/s 2024-04-25T10:26:41.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.777 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01309 s, 80.1 MB/s 2024-04-25T10:26:41.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.797 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180984 s, 57.9 MB/s 2024-04-25T10:26:41.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.813 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014672 s, 71.5 MB/s 2024-04-25T10:26:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155 s, 67.7 MB/s 2024-04-25T10:26:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118274 s, 88.7 MB/s 2024-04-25T10:26:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.859 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129341 s, 81.1 MB/s 2024-04-25T10:26:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132852 s, 78.9 MB/s 2024-04-25T10:26:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136424 s, 76.9 MB/s 2024-04-25T10:26:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134934 s, 77.7 MB/s 2024-04-25T10:26:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015064 s, 69.6 MB/s 2024-04-25T10:26:41.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.937 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137138 s, 76.5 MB/s 2024-04-25T10:26:41.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147883 s, 70.9 MB/s 2024-04-25T10:26:41.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137204 s, 76.4 MB/s 2024-04-25T10:26:41.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154751 s, 67.8 MB/s 2024-04-25T10:26:42.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.007 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186352 s, 56.3 MB/s 2024-04-25T10:26:42.024 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.024 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.024 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147631 s, 71.0 MB/s 2024-04-25T10:26:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181558 s, 57.8 MB/s 2024-04-25T10:26:42.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:42.046 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:42.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.114 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129835 s, 80.8 MB/s 2024-04-25T10:26:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.129 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131678 s, 79.6 MB/s 2024-04-25T10:26:42.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.145 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143976 s, 72.8 MB/s 2024-04-25T10:26:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134783 s, 77.8 MB/s 2024-04-25T10:26:42.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012349 s, 84.9 MB/s 2024-04-25T10:26:42.194 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.195 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.195 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182077 s, 57.6 MB/s 2024-04-25T10:26:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141012 s, 74.4 MB/s 2024-04-25T10:26:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135799 s, 77.2 MB/s 2024-04-25T10:26:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138971 s, 75.5 MB/s 2024-04-25T10:26:42.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144268 s, 72.7 MB/s 2024-04-25T10:26:42.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.273 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134716 s, 77.8 MB/s 2024-04-25T10:26:42.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.288 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130885 s, 80.1 MB/s 2024-04-25T10:26:42.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018357 s, 57.1 MB/s 2024-04-25T10:26:42.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.323 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133853 s, 78.3 MB/s 2024-04-25T10:26:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183075 s, 57.3 MB/s 2024-04-25T10:26:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185957 s, 56.4 MB/s 2024-04-25T10:26:42.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016155 s, 64.9 MB/s 2024-04-25T10:26:42.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.402 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.402 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178412 s, 58.8 MB/s 2024-04-25T10:26:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140902 s, 74.4 MB/s 2024-04-25T10:26:42.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.432 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128613 s, 81.5 MB/s 2024-04-25T10:26:42.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139214 s, 75.3 MB/s 2024-04-25T10:26:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.464 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139206 s, 75.3 MB/s 2024-04-25T10:26:42.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.478 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131115 s, 80.0 MB/s 2024-04-25T10:26:42.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184878 s, 56.7 MB/s 2024-04-25T10:26:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.514 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134493 s, 78.0 MB/s 2024-04-25T10:26:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191222 s, 54.8 MB/s 2024-04-25T10:26:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135404 s, 77.4 MB/s 2024-04-25T10:26:42.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.567 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146389 s, 71.6 MB/s 2024-04-25T10:26:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143671 s, 73.0 MB/s 2024-04-25T10:26:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145416 s, 72.1 MB/s 2024-04-25T10:26:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130671 s, 80.2 MB/s 2024-04-25T10:26:42.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.633 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174419 s, 60.1 MB/s 2024-04-25T10:26:42.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:42.635 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:42.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132802 s, 79.0 MB/s 2024-04-25T10:26:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162162 s, 64.7 MB/s 2024-04-25T10:26:42.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.736 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133886 s, 78.3 MB/s 2024-04-25T10:26:42.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.757 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186193 s, 56.3 MB/s 2024-04-25T10:26:42.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.771 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124897 s, 84.0 MB/s 2024-04-25T10:26:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190679 s, 55.0 MB/s 2024-04-25T10:26:42.807 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.807 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.807 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130924 s, 80.1 MB/s 2024-04-25T10:26:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.824 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146201 s, 71.7 MB/s 2024-04-25T10:26:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014458 s, 72.5 MB/s 2024-04-25T10:26:42.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.855 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140922 s, 74.4 MB/s 2024-04-25T10:26:42.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175855 s, 59.6 MB/s 2024-04-25T10:26:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135098 s, 77.6 MB/s 2024-04-25T10:26:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172592 s, 60.8 MB/s 2024-04-25T10:26:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136903 s, 76.6 MB/s 2024-04-25T10:26:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139842 s, 75.0 MB/s 2024-04-25T10:26:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01351 s, 77.6 MB/s 2024-04-25T10:26:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153415 s, 68.3 MB/s 2024-04-25T10:26:42.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:42.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:42.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140099 s, 74.8 MB/s 2024-04-25T10:26:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141676 s, 74.0 MB/s 2024-04-25T10:26:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.022 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154362 s, 67.9 MB/s 2024-04-25T10:26:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139175 s, 75.3 MB/s 2024-04-25T10:26:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134222 s, 78.1 MB/s 2024-04-25T10:26:43.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175868 s, 59.6 MB/s 2024-04-25T10:26:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135826 s, 77.2 MB/s 2024-04-25T10:26:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018047 s, 58.1 MB/s 2024-04-25T10:26:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144941 s, 72.3 MB/s 2024-04-25T10:26:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128271 s, 81.7 MB/s 2024-04-25T10:26:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158936 s, 66.0 MB/s 2024-04-25T10:26:43.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01818 s, 57.7 MB/s 2024-04-25T10:26:43.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151774 s, 69.1 MB/s 2024-04-25T10:26:43.209 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.209 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.209 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142346 s, 73.7 MB/s 2024-04-25T10:26:43.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142357 s, 73.7 MB/s 2024-04-25T10:26:43.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:43.227 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144731 s, 72.4 MB/s 2024-04-25T10:26:43.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.315 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179818 s, 58.3 MB/s 2024-04-25T10:26:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136975 s, 76.6 MB/s 2024-04-25T10:26:43.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133966 s, 78.3 MB/s 2024-04-25T10:26:43.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.731 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382857 s, 2.7 MB/s 2024-04-25T10:26:43.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.751 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185803 s, 56.4 MB/s 2024-04-25T10:26:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.774 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205503 s, 51.0 MB/s 2024-04-25T10:26:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.796 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019977 s, 52.5 MB/s 2024-04-25T10:26:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.813 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151955 s, 69.0 MB/s 2024-04-25T10:26:43.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.828 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134262 s, 78.1 MB/s 2024-04-25T10:26:43.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.843 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133029 s, 78.8 MB/s 2024-04-25T10:26:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.858 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128331 s, 81.7 MB/s 2024-04-25T10:26:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.873 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133154 s, 78.7 MB/s 2024-04-25T10:26:43.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138635 s, 75.6 MB/s 2024-04-25T10:26:43.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013667 s, 76.7 MB/s 2024-04-25T10:26:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189308 s, 55.4 MB/s 2024-04-25T10:26:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135759 s, 77.2 MB/s 2024-04-25T10:26:43.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.958 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.958 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154539 s, 67.9 MB/s 2024-04-25T10:26:43.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.973 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135672 s, 77.3 MB/s 2024-04-25T10:26:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:43.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141836 s, 73.9 MB/s 2024-04-25T10:26:44.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139505 s, 75.2 MB/s 2024-04-25T10:26:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128838 s, 81.4 MB/s 2024-04-25T10:26:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154199 s, 68.0 MB/s 2024-04-25T10:26:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135855 s, 77.2 MB/s 2024-04-25T10:26:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187598 s, 55.9 MB/s 2024-04-25T10:26:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.090 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152194 s, 68.9 MB/s 2024-04-25T10:26:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013613 s, 77.0 MB/s 2024-04-25T10:26:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138964 s, 75.5 MB/s 2024-04-25T10:26:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144315 s, 72.7 MB/s 2024-04-25T10:26:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169804 s, 61.8 MB/s 2024-04-25T10:26:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174357 s, 60.1 MB/s 2024-04-25T10:26:44.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152667 s, 68.7 MB/s 2024-04-25T10:26:44.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:44.195 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:26:44.195 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:26:44.195 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:26:44.195 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:26:44.196 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:26:44.196 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:26:44.196 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:26:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.262 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.263 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013987 s, 75.0 MB/s 2024-04-25T10:26:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138675 s, 75.6 MB/s 2024-04-25T10:26:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121829 s, 86.1 MB/s 2024-04-25T10:26:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013742 s, 76.3 MB/s 2024-04-25T10:26:44.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.323 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132516 s, 79.1 MB/s 2024-04-25T10:26:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.340 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153413 s, 68.3 MB/s 2024-04-25T10:26:44.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015112 s, 69.4 MB/s 2024-04-25T10:26:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145236 s, 72.2 MB/s 2024-04-25T10:26:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015415 s, 68.0 MB/s 2024-04-25T10:26:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015413 s, 68.0 MB/s 2024-04-25T10:26:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014035 s, 74.7 MB/s 2024-04-25T10:26:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133949 s, 78.3 MB/s 2024-04-25T10:26:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144896 s, 72.4 MB/s 2024-04-25T10:26:44.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.471 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147757 s, 71.0 MB/s 2024-04-25T10:26:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158615 s, 66.1 MB/s 2024-04-25T10:26:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150923 s, 69.5 MB/s 2024-04-25T10:26:44.523 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.523 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.523 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151922 s, 69.0 MB/s 2024-04-25T10:26:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018501 s, 56.7 MB/s 2024-04-25T10:26:44.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133963 s, 78.3 MB/s 2024-04-25T10:26:44.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172124 s, 60.9 MB/s 2024-04-25T10:26:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151541 s, 69.2 MB/s 2024-04-25T10:26:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150392 s, 69.7 MB/s 2024-04-25T10:26:44.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152641 s, 68.7 MB/s 2024-04-25T10:26:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148486 s, 70.6 MB/s 2024-04-25T10:26:44.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.661 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152754 s, 68.6 MB/s 2024-04-25T10:26:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.678 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150085 s, 69.9 MB/s 2024-04-25T10:26:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110979 s, 94.5 MB/s 2024-04-25T10:26:44.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.708 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153312 s, 68.4 MB/s 2024-04-25T10:26:44.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.726 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.726 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159739 s, 65.6 MB/s 2024-04-25T10:26:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.741 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131377 s, 79.8 MB/s 2024-04-25T10:26:44.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129155 s, 81.2 MB/s 2024-04-25T10:26:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:26:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:26:44.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015047 s, 69.7 MB/s 2024-04-25T10:26:44.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:44.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:26:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.954+0000 7f2a04200640 1 Processor -- start 2024-04-25T10:26:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a04200640 1 -- start start 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a04200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29fc15bd10 0x7f29fc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a04200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15c0f0 0x7f29fc0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a04200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29fc06c0e0 con 0x7f29fc0cf9e0 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a04200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29fc15c6a0 con 0x7f29fc15bd10 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a04200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc15c980 con 0x7f29fc15c0f0 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29fc15bd10 0x7f29fc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29fc15bd10 0x7f29fc0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56944/0 (socket says 172.21.15.45:56944) 2024-04-25T10:26:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 -- 172.21.15.45:0/185712623 learned_addr learned my addr 172.21.15.45:0/185712623 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 -- 172.21.15.45:0/185712623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15c0f0 msgr2=0x7f29fc0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a00e00640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15c0f0 0x7f29fc0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15c0f0 0x7f29fc0b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 -- 172.21.15.45:0/185712623 >> v1:172.21.15.117:6789/0 conn(0x7f29fc0cf9e0 legacy=0x7f29fc06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a01800640 1 -- 172.21.15.45:0/185712623 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc15cc60 con 0x7f29fc15bd10 2024-04-25T10:26:44.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.958+0000 7f2a00e00640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15c0f0 0x7f29fc0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:44.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a01800640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15bd10 0x7f29fc0b4610 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009870 tx=0x7f29f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d766bffdd83d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:44.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f29fbe00640 1 -- 172.21.15.45:0/185712623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f00aa430 con 0x7f29fc15bd10 2024-04-25T10:26:44.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f29fbe00640 1 -- 172.21.15.45:0/185712623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f00af070 con 0x7f29fc15bd10 2024-04-25T10:26:44.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f29fbe00640 1 -- 172.21.15.45:0/185712623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f00aa7f0 con 0x7f29fc15bd10 2024-04-25T10:26:44.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 -- 172.21.15.45:0/185712623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15bd10 msgr2=0x7f29fc0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15bd10 0x7f29fc0b4610 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009870 tx=0x7f29f001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 -- 172.21.15.45:0/185712623 shutdown_connections 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15c0f0 0x7f29fc0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 --2- 172.21.15.45:0/185712623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15bd10 0x7f29fc0b4610 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 -- 172.21.15.45:0/185712623 >> 172.21.15.45:0/185712623 conn(0x7f29fc066b70 msgr2=0x7f29fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 -- 172.21.15.45:0/185712623 shutdown_connections 2024-04-25T10:26:44.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 -- 172.21.15.45:0/185712623 wait complete. 2024-04-25T10:26:44.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 Processor -- start 2024-04-25T10:26:44.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.962+0000 7f2a04200640 1 -- start start 2024-04-25T10:26:44.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29fc0cf9e0 0x7f29fc0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15bd10 0x7f29fc1fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 0x7f29fc1fad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0b2ea0 con 0x7f29fc15c0f0 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0b2d20 con 0x7f29fc0cf9e0 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc1692d0 con 0x7f29fc15bd10 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 0x7f29fc1fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 0x7f29fc1fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56960/0 (socket says 172.21.15.45:56960) 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a01800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29fc0cf9e0 0x7f29fc0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 -- 172.21.15.45:0/1774657080 learned_addr learned my addr 172.21.15.45:0/1774657080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:44.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a00e00640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15bd10 0x7f29fc1fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 -- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15bd10 msgr2=0x7f29fc1fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15bd10 0x7f29fc1fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 -- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29fc0cf9e0 msgr2=0x7f29fc0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29fc0cf9e0 0x7f29fc0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 -- 172.21.15.45:0/1774657080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc070f10 con 0x7f29fc15c0f0 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a02200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 0x7f29fc1fad10 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c7b0 tx=0x7f29f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f29f9600640 1 -- 172.21.15.45:0/1774657080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004b70 con 0x7f29fc15c0f0 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f29f9600640 1 -- 172.21.15.45:0/1774657080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4004d10 con 0x7f29fc15c0f0 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f29f9600640 1 -- 172.21.15.45:0/1774657080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f400aa90 con 0x7f29fc15c0f0 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc0711f0 con 0x7f29fc15c0f0 2024-04-25T10:26:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.966+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc0717b0 con 0x7f29fc15c0f0 2024-04-25T10:26:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f29f9600640 1 -- 172.21.15.45:0/1774657080 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29f400ac30 con 0x7f29fc15c0f0 2024-04-25T10:26:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f29f9600640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29dc066f60 0x7f29dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f29f9600640 1 -- 172.21.15.45:0/1774657080 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f29f4085480 con 0x7f29fc15c0f0 2024-04-25T10:26:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 Processor -- start 2024-04-25T10:26:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a01800640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29dc066f60 0x7f29dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 -- start start 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a01800640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29dc066f60 0x7f29dc069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29f00ae000 tx=0x7f29f00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cf3f0 0x55c11c8cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cfd10 0x55c11c885fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c886900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11c6d3d50 con 0x55c11c886520 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11c65be40 con 0x55c11c8cfd10 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11c6cfdf0 con 0x55c11c8cf3f0 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a01800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cf3f0 0x55c11c8cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a02200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c886900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a00e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cfd10 0x55c11c885fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a01800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cf3f0 0x55c11c8cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57458/0 (socket says 172.21.15.45:57458) 2024-04-25T10:26:44.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a01800640 1 -- 172.21.15.45:0/242543790 learned_addr learned my addr 172.21.15.45:0/242543790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:44.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a02200640 1 -- 172.21.15.45:0/242543790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cf3f0 msgr2=0x55c11c8cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:44.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a02200640 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cf3f0 0x55c11c8cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a02200640 1 -- 172.21.15.45:0/242543790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cfd10 msgr2=0x55c11c885fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:44.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a02200640 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cfd10 0x55c11c885fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.970+0000 7f2a02200640 1 -- 172.21.15.45:0/242543790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11c670d90 con 0x55c11c886520 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a02200640 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c886900 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c780 tx=0x7f29f4055750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93415a4682188e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f29ee000640 1 -- 172.21.15.45:0/242543790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f406d030 con 0x55c11c886520 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f29ee000640 1 -- 172.21.15.45:0/242543790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f405e450 con 0x55c11c886520 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f29ee000640 1 -- 172.21.15.45:0/242543790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40540f0 con 0x55c11c886520 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- 172.21.15.45:0/242543790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 msgr2=0x55c11c886900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c886900 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c780 tx=0x7f29f4055750 comp rx=0 tx=0).stop 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- 172.21.15.45:0/242543790 shutdown_connections 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c886900 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cfd10 0x55c11c885fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- 172.21.15.45:0/242543790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cf3f0 0x55c11c8cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- 172.21.15.45:0/242543790 >> 172.21.15.45:0/242543790 conn(0x55c11c8824d0 msgr2=0x55c11c8848c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- 172.21.15.45:0/242543790 shutdown_connections 2024-04-25T10:26:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- 172.21.15.45:0/242543790 wait complete. 2024-04-25T10:26:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- start start 2024-04-25T10:26:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 Processor -- start 2024-04-25T10:26:44.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c88b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c8d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cfd10 0x55c11c8d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11c679610 con 0x55c11c886520 2024-04-25T10:26:44.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11c68e9a0 con 0x55c11c8cf3f0 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11c6d8380 con 0x55c11c8cfd10 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a07728000 10 client.0 _ll_register_callbacks cb 0x55c11c7880d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c88b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c88b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56984/0 (socket says 172.21.15.45:56984) 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 -- 172.21.15.45:0/1646014635 learned_addr learned my addr 172.21.15.45:0/1646014635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a00e00640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c8d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a02200640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cfd10 0x55c11c8d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 -- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cfd10 msgr2=0x55c11c8d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cfd10 0x55c11c8d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 -- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 msgr2=0x55c11c8d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c8d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 -- 172.21.15.45:0/1646014635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c11c6ced40 con 0x55c11c886520 2024-04-25T10:26:44.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f2a01800640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c88b4c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f29f001c530 tx=0x7f29f001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19ead02ea6a03c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.974+0000 7f29e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 1 -- 172.21.15.45:0/1646014635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f00cd060 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 1 -- 172.21.15.45:0/1646014635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f29f00bf8e0 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29f405e450 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 1 -- 172.21.15.45:0/1646014635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f00bfae0 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 10 client.8568 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55c11c756f00 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 1 -- 172.21.15.45:0/1646014635 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f405e450 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 1 -- 172.21.15.45:0/1646014635 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29f00a9070 con 0x55c11c886520 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 10 client.8568 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 10 client.8568 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 20 client.8568 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8d5cf0 0x55c11c8d6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 4 client.8568 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c11c66fa20 con 0x55c11c8d5cf0 2024-04-25T10:26:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a00e00640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8d5cf0 0x55c11c8d6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 10 client.8568 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:44.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a00e00640 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8d5cf0 0x55c11c8d6100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a91a27e94410727 server_cookie=321218253b96e200 in_seq=0 out_seq=0 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 1 -- 172.21.15.45:0/1646014635 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c11c66fa20 con 0x55c11c8d5cf0 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 10 client.8568 handle_command_reply: tid=0 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 client.8568 shutdown 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 msgr2=0x55c11c88b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c88b4c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f29f001c530 tx=0x7f29f001ad30 comp rx=0 tx=0).stop 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 shutdown_connections 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c8cfd10 0x55c11c8d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c8d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8d5cf0 0x55c11c8d6100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 --2- 172.21.15.45:0/1646014635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c886520 0x55c11c88b4c0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f29e2a00640 0 client.8568 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 >> 172.21.15.45:0/1646014635 conn(0x55c11c8824d0 msgr2=0x55c11c885150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 shutdown_connections 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 1 -- 172.21.15.45:0/1646014635 wait complete. 2024-04-25T10:26:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.978+0000 7f2a07728000 20 client.8568 trim_cache size 0 max 16384 2024-04-25T10:26:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 Processor -- start 2024-04-25T10:26:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 -- start start 2024-04-25T10:26:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8858d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c885e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c8d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc2035c0 con 0x55c11c8cfd10 2024-04-25T10:26:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0b1020 con 0x55c11c8cf3f0 2024-04-25T10:26:44.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0b0420 con 0x55c11c886520 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c885e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a02200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c8d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a01800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8858d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c885e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43364/0 (socket says 172.21.15.45:43364) 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 -- 172.21.15.45:0/130699718 learned_addr learned my addr 172.21.15.45:0/130699718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 -- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 msgr2=0x55c11c8858d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8858d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 -- 172.21.15.45:0/130699718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 msgr2=0x55c11c8d8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c8d8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a00e00640 1 -- 172.21.15.45:0/130699718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11c8cfb40 con 0x55c11c8cf3f0 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.994+0000 7f2a02200640 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c8d8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:45.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a00e00640 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c885e10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f29e8006fd0 tx=0x7f29e8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b37f3b32fe14fb0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f29ed600640 1 -- 172.21.15.45:0/130699718 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8015b50 con 0x55c11c8cf3f0 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f29ed600640 1 -- 172.21.15.45:0/130699718 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8021030 con 0x55c11c8cf3f0 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f29ed600640 1 -- 172.21.15.45:0/130699718 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e801de20 con 0x55c11c8cf3f0 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 msgr2=0x55c11c885e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c885e10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f29e8006fd0 tx=0x7f29e8008040 comp rx=0 tx=0).stop 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- 172.21.15.45:0/130699718 shutdown_connections 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c8d8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c885e10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- 172.21.15.45:0/130699718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8858d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- 172.21.15.45:0/130699718 >> 172.21.15.45:0/130699718 conn(0x55c11c8824d0 msgr2=0x55c11c88bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- 172.21.15.45:0/130699718 shutdown_connections 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- 172.21.15.45:0/130699718 wait complete. 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- start start 2024-04-25T10:26:45.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 Processor -- start 2024-04-25T10:26:45.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c92edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c92f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11c68adc0 con 0x55c11c8cfd10 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0b0a20 con 0x55c11c8cf3f0 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11c6d3060 con 0x55c11c886520 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a07728000 10 client.0 _ll_register_callbacks cb 0x7f29f400cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a02200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c92f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a00e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c92edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a02200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c92f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57000/0 (socket says 172.21.15.45:57000) 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8db700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57472/0 (socket says 172.21.15.45:57472) 2024-04-25T10:26:45.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a02200640 1 -- 172.21.15.45:0/4239426420 learned_addr learned my addr 172.21.15.45:0/4239426420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 -- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 msgr2=0x55c11c92edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c92edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 -- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 msgr2=0x55c11c92f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c92f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a01800640 1 -- 172.21.15.45:0/4239426420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c11c72d330 con 0x55c11c886520 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:44.998+0000 7f2a02200640 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c92f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a01800640 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8db700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29f0006aa0 tx=0x7f29f0008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77057c21b61d0a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 1 -- 172.21.15.45:0/4239426420 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f00aab00 con 0x55c11c886520 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29e8021030 con 0x55c11c886520 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 1 -- 172.21.15.45:0/4239426420 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f29f00bf7e0 con 0x55c11c886520 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 1 -- 172.21.15.45:0/4239426420 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f00e1450 con 0x55c11c886520 2024-04-25T10:26:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 10 client.7160 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55c11c6a05f0 con 0x55c11c886520 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 1 -- 172.21.15.45:0/4239426420 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8021030 con 0x55c11c886520 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 1 -- 172.21.15.45:0/4239426420 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29f00a9040 con 0x55c11c886520 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 10 client.7160 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 10 client.7160 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 20 client.7160 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8ecae0 0x55c11c938a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 4 client.7160 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55c11c6e1290 con 0x55c11c8ecae0 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a00e00640 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8ecae0 0x55c11c938a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f2a00e00640 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8ecae0 0x55c11c938a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26a8aa8f7c44a3c server_cookie=b2eb42361ae58a53 in_seq=0 out_seq=0 2024-04-25T10:26:45.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.002+0000 7f29e2a00640 10 client.7160 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f29e2a00640 1 -- 172.21.15.45:0/4239426420 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c11c6e1290 con 0x55c11c8ecae0 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f29e2a00640 10 client.7160 handle_command_reply: tid=0 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 client.7160 shutdown 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 msgr2=0x55c11c8db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8db700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29f0006aa0 tx=0x7f29f0008040 comp rx=0 tx=0).stop 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 shutdown_connections 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55c11c8ecae0 0x55c11c938a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c11c8cfd10 0x55c11c92f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c11c8cf3f0 0x55c11c92edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 --2- 172.21.15.45:0/4239426420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c11c886520 0x55c11c8db700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f29e2a00640 0 client.7160 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 >> 172.21.15.45:0/4239426420 conn(0x55c11c8824d0 msgr2=0x55c11c8848c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 shutdown_connections 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 1 -- 172.21.15.45:0/4239426420 wait complete. 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a07728000 20 client.7160 trim_cache size 0 max 16384 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "466a5c25-25e0-42e7-9ace-a8d436fc2d04", 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:26:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29dc066f60 msgr2=0x7f29dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29dc066f60 0x7f29dc069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29f00ae000 tx=0x7f29f00a7000 comp rx=0 tx=0).stop 2024-04-25T10:26:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 msgr2=0x7f29fc1fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 0x7f29fc1fad10 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c7b0 tx=0x7f29f400cc70 comp rx=0 tx=0).stop 2024-04-25T10:26:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 shutdown_connections 2024-04-25T10:26:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29dc066f60 0x7f29dc069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29fc15c0f0 0x7f29fc1fad10 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29fc15bd10 0x7f29fc1fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 --2- 172.21.15.45:0/1774657080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29fc0cf9e0 0x7f29fc0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 >> 172.21.15.45:0/1774657080 conn(0x7f29fc066b70 msgr2=0x7f29fc15e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 shutdown_connections 2024-04-25T10:26:45.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.006+0000 7f2a04200640 1 -- 172.21.15.45:0/1774657080 wait complete. 2024-04-25T10:26:45.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:45.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T10:26:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 Processor -- start 2024-04-25T10:26:45.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 -- start start 2024-04-25T10:26:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6240aeda0 0x7fe624071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe624072020 0x7fe624070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe624159920 con 0x7fe624159540 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62415c2f0 con 0x7fe624072020 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe629200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62415c5d0 con 0x7fe6240aeda0 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.210+0000 7fe622800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe624072020 0x7fe624070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe621e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe624159540 0x7fe6240ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57810/0 (socket says 172.21.15.45:57810) 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe621e00640 1 -- 172.21.15.45:0/660166979 learned_addr learned my addr 172.21.15.45:0/660166979 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe61be00640 1 --2- 172.21.15.45:0/660166979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6240aeda0 0x7fe624071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe622800640 1 -- 172.21.15.45:0/660166979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6240aeda0 msgr2=0x7fe624071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe622800640 1 --2- 172.21.15.45:0/660166979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6240aeda0 0x7fe624071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe622800640 1 -- 172.21.15.45:0/660166979 >> v1:172.21.15.45:6789/0 conn(0x7fe624159540 legacy=0x7fe6240ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe622800640 1 -- 172.21.15.45:0/660166979 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62415c8b0 con 0x7fe624072020 2024-04-25T10:26:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe621400640 1 -- 172.21.15.45:0/660166979 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294097079 0 0) 0x7fe624159920 con 0x7fe624159540 2024-04-25T10:26:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe622800640 1 --2- 172.21.15.45:0/660166979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe624072020 0x7fe624070350 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009d90 tx=0x7fe61c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e378f12ea7a630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe621400640 1 -- 172.21.15.45:0/660166979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0b2070 con 0x7fe624072020 2024-04-25T10:26:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe621400640 1 -- 172.21.15.45:0/660166979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c0a5a20 con 0x7fe624072020 2024-04-25T10:26:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.214+0000 7fe621400640 1 -- 172.21.15.45:0/660166979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0a5df0 con 0x7fe624072020 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- 172.21.15.45:0/660166979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe624072020 msgr2=0x7fe624070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 --2- 172.21.15.45:0/660166979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe624072020 0x7fe624070350 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009d90 tx=0x7fe61c01c940 comp rx=0 tx=0).stop 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- 172.21.15.45:0/660166979 shutdown_connections 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 --2- 172.21.15.45:0/660166979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe624072020 0x7fe624070350 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 --2- 172.21.15.45:0/660166979 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6240aeda0 0x7fe624071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- 172.21.15.45:0/660166979 >> 172.21.15.45:0/660166979 conn(0x7fe6240669a0 msgr2=0x7fe624066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- 172.21.15.45:0/660166979 shutdown_connections 2024-04-25T10:26:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- 172.21.15.45:0/660166979 wait complete. 2024-04-25T10:26:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 Processor -- start 2024-04-25T10:26:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- start start 2024-04-25T10:26:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe624072020 0x7fe6240b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 0x7fe6240b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe624159540 0x7fe624144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240710a0 con 0x7fe6240aeda0 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62406c0d0 con 0x7fe624072020 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe629200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe624071220 con 0x7fe624159540 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 0x7fe6240b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 0x7fe6240b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57026/0 (socket says 172.21.15.45:57026) 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 -- 172.21.15.45:0/673802021 learned_addr learned my addr 172.21.15.45:0/673802021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe622800640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe624159540 0x7fe624144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe621e00640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe624072020 0x7fe6240b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 -- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe624159540 msgr2=0x7fe624144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe624159540 0x7fe624144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 -- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe624072020 msgr2=0x7fe6240b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe624072020 0x7fe6240b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 -- 172.21.15.45:0/673802021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6240b43e0 con 0x7fe6240aeda0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61be00640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 0x7fe6240b3b90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e960 tx=0x7fe60c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.218+0000 7fe61a000640 1 -- 172.21.15.45:0/673802021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c017070 con 0x7fe6240aeda0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe61a000640 1 -- 172.21.15.45:0/673802021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c00ca00 con 0x7fe6240aeda0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe61a000640 1 -- 172.21.15.45:0/673802021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00cdb0 con 0x7fe6240aeda0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6240b46c0 con 0x7fe6240aeda0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6240b2930 con 0x7fe6240aeda0 2024-04-25T10:26:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe61a000640 1 -- 172.21.15.45:0/673802021 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe60c01a070 con 0x7fe6240aeda0 2024-04-25T10:26:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe61a000640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe61a000640 1 -- 172.21.15.45:0/673802021 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe60c0850c0 con 0x7fe6240aeda0 2024-04-25T10:26:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe621e00640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.222+0000 7fe62c753000 1 Processor -- start 2024-04-25T10:26:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe621e00640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe614000f70 tx=0x7fe614007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- start start 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c997e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c952850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c953170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614c8351f0 con 0x55614c998360 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614c79be90 con 0x55614c997a40 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55614c723f50 con 0x55614c952d90 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c952850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c952850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57040/0 (socket says 172.21.15.45:57040) 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 -- 172.21.15.45:0/2242056372 learned_addr learned my addr 172.21.15.45:0/2242056372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe622800640 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c953170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe621e00640 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c997e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 -- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 msgr2=0x55614c953170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c953170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 -- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 msgr2=0x55614c997e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c997e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 -- 172.21.15.45:0/2242056372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55614c7738b0 con 0x55614c998360 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe622800640 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c953170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe61be00640 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c952850 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe60c055910 tx=0x7fe60c055980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d878f5627baf32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe612a00640 1 -- 172.21.15.45:0/2242056372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c021040 con 0x55614c998360 2024-04-25T10:26:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe612a00640 1 -- 172.21.15.45:0/2242056372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c053c80 con 0x55614c998360 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe612a00640 1 -- 172.21.15.45:0/2242056372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c054dd0 con 0x55614c998360 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- 172.21.15.45:0/2242056372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 msgr2=0x55614c952850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c952850 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe60c055910 tx=0x7fe60c055980 comp rx=0 tx=0).stop 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- 172.21.15.45:0/2242056372 shutdown_connections 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c953170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c952850 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 --2- 172.21.15.45:0/2242056372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c997e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- 172.21.15.45:0/2242056372 >> 172.21.15.45:0/2242056372 conn(0x55614c94ed40 msgr2=0x55614c951130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- 172.21.15.45:0/2242056372 shutdown_connections 2024-04-25T10:26:45.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- 172.21.15.45:0/2242056372 wait complete. 2024-04-25T10:26:45.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 -- start start 2024-04-25T10:26:45.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.226+0000 7fe62c753000 1 Processor -- start 2024-04-25T10:26:45.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c99f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c99f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c958e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614c756540 con 0x55614c998360 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614c797f30 con 0x55614c952d90 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55614c77a010 con 0x55614c997a40 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 10 client.0 _ll_register_callbacks cb 0x55614c850180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe622800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c958e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe621e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c99f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c99f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe621e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c99f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43394/0 (socket says 172.21.15.45:43394) 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe621e00640 1 -- 172.21.15.45:0/4239238961 learned_addr learned my addr 172.21.15.45:0/4239238961 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 -- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 msgr2=0x55614c99f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c99f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 -- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 msgr2=0x55614c958e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c958e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 -- 172.21.15.45:0/4239238961 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55614c7241a0 con 0x55614c997a40 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe61be00640 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c99f700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe60c055680 tx=0x7fe60c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e13ea3e42dd0a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 1 -- 172.21.15.45:0/4239238961 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c082070 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 1 -- 172.21.15.45:0/4239238961 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe60c06d1d0 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe60c053c80 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 1 -- 172.21.15.45:0/4239238961 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe60c021040 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 10 client.7172 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55614c81e840 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 1 -- 172.21.15.45:0/4239238961 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c053c80 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 1 -- 172.21.15.45:0/4239238961 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe60c024060 con 0x55614c997a40 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 10 client.7172 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 10 client.7172 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 20 client.7172 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c959c00 0x55614c959fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 4 client.7172 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55614c737be0 con 0x55614c959c00 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe621e00640 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c959c00 0x55614c959fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe621e00640 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c959c00 0x55614c959fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0aed82f0b2d4d02 server_cookie=58914ac7f3455742 in_seq=0 out_seq=0 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 10 client.7172 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 1 -- 172.21.15.45:0/4239238961 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55614c737be0 con 0x55614c959c00 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe607400640 10 client.7172 handle_command_reply: tid=0 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.230+0000 7fe62c753000 1 client.7172 shutdown 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 msgr2=0x55614c99f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c99f700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe60c055680 tx=0x7fe60c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 shutdown_connections 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c959c00 0x55614c959fe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c958e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe607400640 0 client.7172 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c99f700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 --2- 172.21.15.45:0/4239238961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c99f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 >> 172.21.15.45:0/4239238961 conn(0x55614c94ed40 msgr2=0x55614c9519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 shutdown_connections 2024-04-25T10:26:45.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 1 -- 172.21.15.45:0/4239238961 wait complete. 2024-04-25T10:26:45.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.234+0000 7fe62c753000 20 client.7172 trim_cache size 0 max 16384 2024-04-25T10:26:45.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.246+0000 7fe62c753000 1 Processor -- start 2024-04-25T10:26:45.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.246+0000 7fe62c753000 1 -- start start 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.246+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c9f4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c9f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c9f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624202a30 con 0x55614c998360 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624157450 con 0x55614c952d90 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240b1010 con 0x55614c997a40 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c9f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c9f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57060/0 (socket says 172.21.15.45:57060) 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe621e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c9f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 -- 172.21.15.45:0/3600222737 learned_addr learned my addr 172.21.15.45:0/3600222737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe61be00640 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c9f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 -- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 msgr2=0x55614c9f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c9f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 -- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 msgr2=0x55614c9f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c9f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 -- 172.21.15.45:0/3600222737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55614c7f5a10 con 0x55614c998360 2024-04-25T10:26:45.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe621e00640 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c9f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:45.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe622800640 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c9f4d10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0b0040 tx=0x7fe61c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1086b47bb580012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe612000640 1 -- 172.21.15.45:0/3600222737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0b2040 con 0x55614c998360 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe612000640 1 -- 172.21.15.45:0/3600222737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c0a59c0 con 0x55614c998360 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe612000640 1 -- 172.21.15.45:0/3600222737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0a5db0 con 0x55614c998360 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- 172.21.15.45:0/3600222737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 msgr2=0x55614c9f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c9f4d10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0b0040 tx=0x7fe61c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- 172.21.15.45:0/3600222737 shutdown_connections 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c9f4d10 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c997a40 0x55614c9f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- 172.21.15.45:0/3600222737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c952d90 0x55614c9f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- 172.21.15.45:0/3600222737 >> 172.21.15.45:0/3600222737 conn(0x55614c94ed40 msgr2=0x55614c9961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- 172.21.15.45:0/3600222737 shutdown_connections 2024-04-25T10:26:45.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- 172.21.15.45:0/3600222737 wait complete. 2024-04-25T10:26:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 -- start start 2024-04-25T10:26:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 Processor -- start 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c9f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c9f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.250+0000 7fe62c753000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c99ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55614c75f720 con 0x55614c998360 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624156e50 con 0x55614c997a40 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55614c73f2d0 con 0x55614c952d90 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 10 client.0 _ll_register_callbacks cb 0x55614c75ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c99ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe621e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c9f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe61be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c9f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c99ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57070/0 (socket says 172.21.15.45:57070) 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 -- 172.21.15.45:0/2453173845 learned_addr learned my addr 172.21.15.45:0/2453173845 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 -- 172.21.15.45:0/2453173845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 msgr2=0x55614c9f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c9f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 -- 172.21.15.45:0/2453173845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 msgr2=0x55614c9f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c9f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 -- 172.21.15.45:0/2453173845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55614c730b00 con 0x55614c998360 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe622800640 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c99ced0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0ba040 tx=0x7fe61c01b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9119258fcf7fec01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 1 -- 172.21.15.45:0/2453173845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c0b2040 con 0x55614c998360 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe61c0a59c0 con 0x55614c998360 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 1 -- 172.21.15.45:0/2453173845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe61c0be040 con 0x55614c998360 2024-04-25T10:26:45.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 1 -- 172.21.15.45:0/2453173845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe61c0d4050 con 0x55614c998360 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 10 client.8595 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55614c742f80 con 0x55614c998360 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 1 -- 172.21.15.45:0/2453173845 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c0a59c0 con 0x55614c998360 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 1 -- 172.21.15.45:0/2453173845 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe61c0d7020 con 0x55614c998360 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 10 client.8595 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 10 client.8595 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 20 client.8595 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c99d8c0 0x55614c99dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 4 client.8595 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55614c7a93d0 con 0x55614c99d8c0 2024-04-25T10:26:45.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe621e00640 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c99d8c0 0x55614c99dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe621e00640 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c99d8c0 0x55614c99dcd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62a158d11341964 server_cookie=be17f6e236461527 in_seq=0 out_seq=0 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 10 client.8595 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 1 -- 172.21.15.45:0/2453173845 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55614c7a93d0 con 0x55614c99d8c0 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 10 client.8595 handle_command_reply: tid=0 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 client.8595 shutdown 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 msgr2=0x55614c99ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c99ced0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0ba040 tx=0x7fe61c01b180 comp rx=0 tx=0).stop 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 shutdown_connections 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55614c99d8c0 0x55614c99dcd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55614c998360 0x55614c99ced0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55614c997a40 0x55614c9f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 --2- 172.21.15.45:0/2453173845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55614c952d90 0x55614c9f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe607400640 0 client.8595 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 >> 172.21.15.45:0/2453173845 conn(0x55614c94ed40 msgr2=0x55614c996a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 shutdown_connections 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 1 -- 172.21.15.45:0/2453173845 wait complete. 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe62c753000 20 client.8595 trim_cache size 0 max 16384 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:26:45.264 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:45.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.254+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 msgr2=0x7fe600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe614000f70 tx=0x7fe614007b60 comp rx=0 tx=0).stop 2024-04-25T10:26:45.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 msgr2=0x7fe6240b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 0x7fe6240b3b90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e960 tx=0x7fe60c00ee20 comp rx=0 tx=0).stop 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 shutdown_connections 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe624159540 0x7fe624144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6240aeda0 0x7fe6240b3b90 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 --2- 172.21.15.45:0/673802021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe624072020 0x7fe6240b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 >> 172.21.15.45:0/673802021 conn(0x7fe6240669a0 msgr2=0x7fe624066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 shutdown_connections 2024-04-25T10:26:45.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.258+0000 7fe629200640 1 -- 172.21.15.45:0/673802021 wait complete. 2024-04-25T10:26:45.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:45.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:26:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 Processor -- start 2024-04-25T10:26:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 -- start start 2024-04-25T10:26:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd080b3480 0x7fbd0815e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd080cfb50 0x7fbd08070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd08071bc0 0x7fbd08072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0815bb90 con 0x7fbd080cfb50 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd080ad840 con 0x7fbd080b3480 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0815e710 con 0x7fbd08071bc0 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd080cfb50 0x7fbd08070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd080cfb50 0x7fbd08070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57082/0 (socket says 172.21.15.45:57082) 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 -- 172.21.15.45:0/692617385 learned_addr learned my addr 172.21.15.45:0/692617385 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0d400640 1 --2- 172.21.15.45:0/692617385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd08071bc0 0x7fbd08072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/692617385 >> v2:172.21.15.117:3300/0 conn(0x7fbd080b3480 0x7fbd0815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 -- 172.21.15.45:0/692617385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd08071bc0 msgr2=0x7fbd08072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 --2- 172.21.15.45:0/692617385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd08071bc0 0x7fbd08072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 -- 172.21.15.45:0/692617385 >> v2:172.21.15.117:3300/0 conn(0x7fbd080b3480 msgr2=0x7fbd0815e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 --2- 172.21.15.45:0/692617385 >> v2:172.21.15.117:3300/0 conn(0x7fbd080b3480 0x7fbd0815e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.418+0000 7fbd07e00640 1 -- 172.21.15.45:0/692617385 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0815e9f0 con 0x7fbd080cfb50 2024-04-25T10:26:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd07e00640 1 --2- 172.21.15.45:0/692617385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd080cfb50 0x7fbd08070350 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fbd0806e5e0 tx=0x7fbcf401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe0da91beda600d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd07400640 1 -- 172.21.15.45:0/692617385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40b0070 con 0x7fbd080cfb50 2024-04-25T10:26:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd07400640 1 -- 172.21.15.45:0/692617385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40ab600 con 0x7fbd080cfb50 2024-04-25T10:26:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd07400640 1 -- 172.21.15.45:0/692617385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40ab9d0 con 0x7fbd080cfb50 2024-04-25T10:26:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 -- 172.21.15.45:0/692617385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd080cfb50 msgr2=0x7fbd08070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 --2- 172.21.15.45:0/692617385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd080cfb50 0x7fbd08070350 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fbd0806e5e0 tx=0x7fbcf401c520 comp rx=0 tx=0).stop 2024-04-25T10:26:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 -- 172.21.15.45:0/692617385 shutdown_connections 2024-04-25T10:26:45.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 --2- 172.21.15.45:0/692617385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd08071bc0 0x7fbd08072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 --2- 172.21.15.45:0/692617385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd080cfb50 0x7fbd08070350 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 --2- 172.21.15.45:0/692617385 >> v2:172.21.15.117:3300/0 conn(0x7fbd080b3480 0x7fbd0815e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 -- 172.21.15.45:0/692617385 >> 172.21.15.45:0/692617385 conn(0x7fbd080669a0 msgr2=0x7fbd08066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 -- 172.21.15.45:0/692617385 shutdown_connections 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 -- 172.21.15.45:0/692617385 wait complete. 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.422+0000 7fbd0f400640 1 Processor -- start 2024-04-25T10:26:45.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 -- start start 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 0x7fbd08159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd080b3480 0x7fbd080afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd080cfb50 0x7fbd081530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08070a10 con 0x7fbd08071bc0 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08070890 con 0x7fbd080cfb50 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080ae530 con 0x7fbd080b3480 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 0x7fbd08159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 0x7fbd08159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57092/0 (socket says 172.21.15.45:57092) 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 -- 172.21.15.45:0/271129749 learned_addr learned my addr 172.21.15.45:0/271129749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd07e00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd080b3480 0x7fbd080afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 -- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd080b3480 msgr2=0x7fbd080afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd080b3480 0x7fbd080afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0d400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd080cfb50 0x7fbd081530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 -- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd080cfb50 msgr2=0x7fbd081530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd080cfb50 0x7fbd081530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 -- 172.21.15.45:0/271129749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd080b0c20 con 0x7fbd08071bc0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd07e00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd080b3480 0x7fbd080afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0d400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd080cfb50 0x7fbd081530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 0x7fbd08159c30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00e930 tx=0x7fbcfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd04c00640 1 -- 172.21.15.45:0/271129749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc017070 con 0x7fbd08071bc0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd04c00640 1 -- 172.21.15.45:0/271129749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc00ca00 con 0x7fbd08071bc0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd04c00640 1 -- 172.21.15.45:0/271129749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00cdb0 con 0x7fbd08071bc0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd080b0660 con 0x7fbd08071bc0 2024-04-25T10:26:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.426+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd081528c0 con 0x7fbd08071bc0 2024-04-25T10:26:45.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd04c00640 1 -- 172.21.15.45:0/271129749 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbcfc01a070 con 0x7fbd08071bc0 2024-04-25T10:26:45.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd04c00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbce8066f60 0x7fbce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd04c00640 1 -- 172.21.15.45:0/271129749 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbcfc085510 con 0x7fbd08071bc0 2024-04-25T10:26:45.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbce8066f60 0x7fbce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 Processor -- start 2024-04-25T10:26:45.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 -- start start 2024-04-25T10:26:45.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbce8066f60 0x7fbce8069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ad000 tx=0x7fbcf40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c24db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445bdab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445bdb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a445a27ff0 con 0x55a445c252f0 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4459b0150 con 0x55a445c249d0 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a445a24090 con 0x55a445bdb0a0 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445bdab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c24db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445bdab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57096/0 (socket says 172.21.15.45:57096) 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 -- 172.21.15.45:0/1257124464 learned_addr learned my addr 172.21.15.45:0/1257124464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd0d400640 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445bdb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 -- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 msgr2=0x55a445bdb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445bdb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 -- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 msgr2=0x55a445c24db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c24db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 -- 172.21.15.45:0/1257124464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a445bbe0d0 con 0x55a445c252f0 2024-04-25T10:26:45.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbd07e00640 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445bdab60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4097db0 tx=0x7fbcf4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f177e7962bd84a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbcf9600640 1 -- 172.21.15.45:0/1257124464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40b7450 con 0x55a445c252f0 2024-04-25T10:26:45.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbcf9600640 1 -- 172.21.15.45:0/1257124464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40c6070 con 0x55a445c252f0 2024-04-25T10:26:45.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.430+0000 7fbcf9600640 1 -- 172.21.15.45:0/1257124464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40b77f0 con 0x55a445c252f0 2024-04-25T10:26:45.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- 172.21.15.45:0/1257124464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 msgr2=0x55a445bdab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445bdab60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4097db0 tx=0x7fbcf4004740 comp rx=0 tx=0).stop 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- 172.21.15.45:0/1257124464 shutdown_connections 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445bdb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445bdab60 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1257124464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c24db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- 172.21.15.45:0/1257124464 >> 172.21.15.45:0/1257124464 conn(0x55a445bd7020 msgr2=0x55a445bd9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- 172.21.15.45:0/1257124464 shutdown_connections 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- 172.21.15.45:0/1257124464 wait complete. 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- start start 2024-04-25T10:26:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 Processor -- start 2024-04-25T10:26:45.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c09d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445c0a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4459ec250 con 0x55a445c252f0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4459e2690 con 0x55a445c249d0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4459b0a40 con 0x55a445bdb0a0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 10 client.0 _ll_register_callbacks cb 0x55a445bc5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445c0a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445c0a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57098/0 (socket says 172.21.15.45:57098) 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 -- 172.21.15.45:0/2847345607 learned_addr learned my addr 172.21.15.45:0/2847345607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd07e00640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c09d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 -- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 msgr2=0x55a445c097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 -- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 msgr2=0x55a445c09d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c09d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 -- 172.21.15.45:0/2847345607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a445ad51c0 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd0d400640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445c0a270 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007da0 tx=0x7fbd00012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8906c8ac14a85dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbcee000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbcee000640 1 -- 172.21.15.45:0/2847345607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0001c020 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbcee000640 1 -- 172.21.15.45:0/2847345607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbd00017440 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcf40c6070 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.434+0000 7fbcee000640 1 -- 172.21.15.45:0/2847345607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd00026050 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 10 client.8607 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a445ad1170 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbcee000640 1 -- 172.21.15.45:0/2847345607 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40c6070 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbcee000640 1 -- 172.21.15.45:0/2847345607 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd00029020 con 0x55a445c252f0 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 10 client.8607 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 10 client.8607 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:45.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 20 client.8607 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c34c80 0x55a445c37070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 4 client.8607 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4459c3c60 con 0x55a445c34c80 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c34c80 0x55a445c37070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbcee000640 10 client.8607 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c34c80 0x55a445c37070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b33738e11036b60 server_cookie=3a9b678476e57937 in_seq=0 out_seq=0 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbcee000640 1 -- 172.21.15.45:0/2847345607 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a4459c3c60 con 0x55a445c34c80 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbcee000640 10 client.8607 handle_command_reply: tid=0 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 client.8607 shutdown 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 msgr2=0x55a445c0a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445c0a270 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbd00007da0 tx=0x7fbd00012590 comp rx=0 tx=0).stop 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 shutdown_connections 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c34c80 0x55a445c37070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c252f0 0x55a445c0a270 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c09d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 --2- 172.21.15.45:0/2847345607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbcee000640 0 client.8607 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 >> 172.21.15.45:0/2847345607 conn(0x55a445bd7020 msgr2=0x55a445bd9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 shutdown_connections 2024-04-25T10:26:45.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 1 -- 172.21.15.45:0/2847345607 wait complete. 2024-04-25T10:26:45.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.438+0000 7fbd128a2000 20 client.8607 trim_cache size 0 max 16384 2024-04-25T10:26:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 Processor -- start 2024-04-25T10:26:45.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- start start 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 0x55a445c810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c81630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445c252f0 0x55a445c81b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08202e40 con 0x55a445bdb0a0 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08202050 con 0x55a445c249d0 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08148360 con 0x55a445c252f0 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 0x55a445c810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 0x55a445c810f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57114/0 (socket says 172.21.15.45:57114) 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 -- 172.21.15.45:0/3148616715 learned_addr learned my addr 172.21.15.45:0/3148616715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd07e00640 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c81630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0d400640 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445c252f0 0x55a445c81b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 -- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445c252f0 msgr2=0x55a445c81b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445c252f0 0x55a445c81b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 -- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 msgr2=0x55a445c81630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c81630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 -- 172.21.15.45:0/3148616715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a445a6ede0 con 0x55a445bdb0a0 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd07e00640 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c81630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 0x55a445c810f0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc055d90 tx=0x7fbcfc085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc3ee6773a542f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbcf8c00640 1 -- 172.21.15.45:0/3148616715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc021040 con 0x55a445bdb0a0 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbcf8c00640 1 -- 172.21.15.45:0/3148616715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc083a70 con 0x55a445bdb0a0 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbcf8c00640 1 -- 172.21.15.45:0/3148616715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc05c0b0 con 0x55a445bdb0a0 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- 172.21.15.45:0/3148616715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 msgr2=0x55a445c810f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 0x55a445c810f0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc055d90 tx=0x7fbcfc085040 comp rx=0 tx=0).stop 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- 172.21.15.45:0/3148616715 shutdown_connections 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445c252f0 0x55a445c81b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c249d0 0x55a445c81630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 --2- 172.21.15.45:0/3148616715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445bdb0a0 0x55a445c810f0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- 172.21.15.45:0/3148616715 >> 172.21.15.45:0/3148616715 conn(0x55a445bd7020 msgr2=0x55a445c31fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- 172.21.15.45:0/3148616715 shutdown_connections 2024-04-25T10:26:45.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.454+0000 7fbd128a2000 1 -- 172.21.15.45:0/3148616715 wait complete. 2024-04-25T10:26:45.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- start start 2024-04-25T10:26:45.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 Processor -- start 2024-04-25T10:26:45.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c37380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c249d0 0x55a445c378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c252f0 0x55a445c37e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4459c2e00 con 0x55a445c249d0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08148960 con 0x55a445c252f0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a445a27300 con 0x55a445bdb0a0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 10 client.0 _ll_register_callbacks cb 0x55a445ba6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c249d0 0x55a445c378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c37380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c252f0 0x55a445c37e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c37380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57594/0 (socket says 172.21.15.45:57594) 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 -- 172.21.15.45:0/1922065441 learned_addr learned my addr 172.21.15.45:0/1922065441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 -- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c252f0 msgr2=0x55a445c37e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c252f0 0x55a445c37e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 -- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c249d0 msgr2=0x55a445c378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c249d0 0x55a445c378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 -- 172.21.15.45:0/1922065441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4459c98e0 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd0ca00640 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c37380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc085480 tx=0x7fbd00009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f37b7d1192b5e772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbcee000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbcee000640 1 -- 172.21.15.45:0/1922065441 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc021050 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcfc083a70 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbcee000640 1 -- 172.21.15.45:0/1922065441 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbcfc052470 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbcee000640 1 -- 172.21.15.45:0/1922065441 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcfc063050 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 10 client.7193 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a445a818f0 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbcee000640 1 -- 172.21.15.45:0/1922065441 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc083a70 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbcee000640 1 -- 172.21.15.45:0/1922065441 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbcfc066080 con 0x55a445bdb0a0 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 10 client.7193 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 10 client.7193 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:45.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 20 client.7193 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c38760 0x55a445c8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 4 client.7193 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a445a35530 con 0x55a445c38760 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.458+0000 7fbd07e00640 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c38760 0x55a445c8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbcee000640 10 client.7193 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd07e00640 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c38760 0x55a445c8cf60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec80c6efc309d475 server_cookie=13ff991f933db7c5 in_seq=0 out_seq=0 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbcee000640 1 -- 172.21.15.45:0/1922065441 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55a445a35530 con 0x55a445c38760 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbcee000640 10 client.7193 handle_command_reply: tid=0 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 client.7193 shutdown 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 msgr2=0x55a445c37380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c37380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc085480 tx=0x7fbd00009750 comp rx=0 tx=0).stop 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 shutdown_connections 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a445c252f0 0x55a445c37e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55a445c38760 0x55a445c8cf60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a445c249d0 0x55a445c378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 --2- 172.21.15.45:0/1922065441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a445bdb0a0 0x55a445c37380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 >> 172.21.15.45:0/1922065441 conn(0x55a445bd7020 msgr2=0x55a445c230d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbcee000640 0 client.7193 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 shutdown_connections 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 1 -- 172.21.15.45:0/1922065441 wait complete. 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd128a2000 20 client.7193 trim_cache size 0 max 16384 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:45.468 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: "466a5c25-25e0-42e7-9ace-a8d436fc2d04": { 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "466a5c25-25e0-42e7-9ace-a8d436fc2d04", 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:26:45.469 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbce8066f60 msgr2=0x7fbce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbce8066f60 0x7fbce8069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ad000 tx=0x7fbcf40a6000 comp rx=0 tx=0).stop 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 msgr2=0x7fbd08159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 0x7fbd08159c30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00e930 tx=0x7fbcfc00edf0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 shutdown_connections 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbce8066f60 0x7fbce8069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd080cfb50 0x7fbd081530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd080b3480 0x7fbd080afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 --2- 172.21.15.45:0/271129749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd08071bc0 0x7fbd08159c30 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 >> 172.21.15.45:0/271129749 conn(0x7fbd080669a0 msgr2=0x7fbd08158140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 shutdown_connections 2024-04-25T10:26:45.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.462+0000 7fbd0f400640 1 -- 172.21.15.45:0/271129749 wait complete. 2024-04-25T10:26:45.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 Processor -- start 2024-04-25T10:26:45.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 -- start start 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77e80d0040 0x7f77e815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f77e815b740 0x7f77e80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e8071050 con 0x7f77e81550b0 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e815d040 con 0x7f77e80d0040 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77eec00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e80712f0 con 0x7f77e815b740 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77e7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f77e81550b0 0x7f77e80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57814/0 (socket says 172.21.15.45:57814) 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77e7e00640 1 -- 172.21.15.45:0/3754267717 learned_addr learned my addr 172.21.15.45:0/3754267717 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77df400640 1 --2- 172.21.15.45:0/3754267717 >> v2:172.21.15.45:3300/0 conn(0x7f77e80d0040 0x7f77e815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77ecc00640 1 --2- 172.21.15.45:0/3754267717 >> v2:172.21.15.117:3300/0 conn(0x7f77e815b740 0x7f77e80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77e7400640 1 -- 172.21.15.45:0/3754267717 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224701719 0 0) 0x7f77e8071050 con 0x7f77e81550b0 2024-04-25T10:26:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77e7400640 1 -- 172.21.15.45:0/3754267717 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77cc003530 con 0x7f77e81550b0 2024-04-25T10:26:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77df400640 1 -- 172.21.15.45:0/3754267717 >> v2:172.21.15.117:3300/0 conn(0x7f77e815b740 msgr2=0x7f77e80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77df400640 1 --2- 172.21.15.45:0/3754267717 >> v2:172.21.15.117:3300/0 conn(0x7f77e815b740 0x7f77e80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77df400640 1 -- 172.21.15.45:0/3754267717 >> v1:172.21.15.45:6789/0 conn(0x7f77e81550b0 legacy=0x7f77e80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.622+0000 7f77df400640 1 -- 172.21.15.45:0/3754267717 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80715d0 con 0x7f77e80d0040 2024-04-25T10:26:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77df400640 1 --2- 172.21.15.45:0/3754267717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80d0040 0x7f77e815b200 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f77e80d04d0 tx=0x7f77d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8439fc3a64d6e9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77e7400640 1 -- 172.21.15.45:0/3754267717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00ab650 con 0x7f77e80d0040 2024-04-25T10:26:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77e7400640 1 -- 172.21.15.45:0/3754267717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d00b0070 con 0x7f77e80d0040 2024-04-25T10:26:45.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77e7400640 1 -- 172.21.15.45:0/3754267717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00aba10 con 0x7f77e80d0040 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 -- 172.21.15.45:0/3754267717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80d0040 msgr2=0x7f77e815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 --2- 172.21.15.45:0/3754267717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80d0040 0x7f77e815b200 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f77e80d04d0 tx=0x7f77d001c450 comp rx=0 tx=0).stop 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 -- 172.21.15.45:0/3754267717 shutdown_connections 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 --2- 172.21.15.45:0/3754267717 >> v2:172.21.15.117:3300/0 conn(0x7f77e815b740 0x7f77e80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 --2- 172.21.15.45:0/3754267717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80d0040 0x7f77e815b200 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 -- 172.21.15.45:0/3754267717 >> 172.21.15.45:0/3754267717 conn(0x7f77e8066b70 msgr2=0x7f77e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 -- 172.21.15.45:0/3754267717 shutdown_connections 2024-04-25T10:26:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.626+0000 7f77eec00640 1 -- 172.21.15.45:0/3754267717 wait complete. 2024-04-25T10:26:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 Processor -- start 2024-04-25T10:26:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 -- start start 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77e80d0040 0x7f77e8152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 0x7f77e80cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e815b740 0x7f77e80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815bcd0 con 0x7f77e81550b0 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815bb50 con 0x7f77e815b740 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815c030 con 0x7f77e80d0040 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 0x7f77e80cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 0x7f77e80cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57158/0 (socket says 172.21.15.45:57158) 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 -- 172.21.15.45:0/56365122 learned_addr learned my addr 172.21.15.45:0/56365122 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77e7e00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77e80d0040 0x7f77e8152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77ecc00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e815b740 0x7f77e80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 -- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77e80d0040 msgr2=0x7f77e8152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77e80d0040 0x7f77e8152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 -- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e815b740 msgr2=0x7f77e80ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e815b740 0x7f77e80ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 -- 172.21.15.45:0/56365122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e815dda0 con 0x7f77e81550b0 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77ecc00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e815b740 0x7f77e80ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:45.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77df400640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 0x7f77e80cc7f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f77d0019060 tx=0x7f77d001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00bf610 con 0x7f77e81550b0 2024-04-25T10:26:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d00bf7d0 con 0x7f77e81550b0 2024-04-25T10:26:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d00bfbc0 con 0x7f77e81550b0 2024-04-25T10:26:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e815a540 con 0x7f77e81550b0 2024-04-25T10:26:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.630+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e80b1b60 con 0x7f77e81550b0 2024-04-25T10:26:45.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.634+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e804ef30 con 0x7f77e81550b0 2024-04-25T10:26:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.634+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77d00bd020 con 0x7f77e81550b0 2024-04-25T10:26:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.634+0000 7f77e4c00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77c4066f60 0x7f77c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:45.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.634+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f77d011fad0 con 0x7f77e81550b0 2024-04-25T10:26:45.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.634+0000 7f77e7e00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77c4066f60 0x7f77c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:45.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.634+0000 7f77e7e00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77c4066f60 0x7f77c4069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005890 tx=0x7f77d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.638+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77d0016070 con 0x7f77e81550b0 2024-04-25T10:26:45.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.834+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e815a9f0 con 0x7f77e81550b0 2024-04-25T10:26:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.834+0000 7f77e4c00640 1 -- 172.21.15.45:0/56365122 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f77d00f0c30 con 0x7f77e81550b0 2024-04-25T10:26:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:45.839 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77c4066f60 msgr2=0x7f77c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77c4066f60 0x7f77c4069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005890 tx=0x7f77d8009040 comp rx=0 tx=0).stop 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 msgr2=0x7f77e80cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 0x7f77e80cc7f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f77d0019060 tx=0x7f77d001ac90 comp rx=0 tx=0).stop 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 shutdown_connections 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77c4066f60 0x7f77c4069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e815b740 0x7f77e80ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e81550b0 0x7f77e80cc7f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 --2- 172.21.15.45:0/56365122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77e80d0040 0x7f77e8152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:45.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 >> 172.21.15.45:0/56365122 conn(0x7f77e8066b70 msgr2=0x7f77e8066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 shutdown_connections 2024-04-25T10:26:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.838+0000 7f77eec00640 1 -- 172.21.15.45:0/56365122 wait complete. 2024-04-25T10:26:45.860 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:45.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.998+0000 7fb0f8400640 1 Processor -- start 2024-04-25T10:26:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:45.998+0000 7fb0f8400640 1 -- start start 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f8400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0f00d07b0 0x7fb0f006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f8400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00ccaa0 con 0x7fb0f00d0350 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f8400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00ccd80 con 0x7fb0f015df60 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f8400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f015e340 con 0x7fb0f00d07b0 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f5000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb0f00d0350 0x7fb0f006aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57816/0 (socket says 172.21.15.45:57816) 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f5000640 1 -- 172.21.15.45:0/3737925256 learned_addr learned my addr 172.21.15.45:0/3737925256 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f6400640 1 --2- 172.21.15.45:0/3737925256 >> v2:172.21.15.117:3300/0 conn(0x7fb0f00d07b0 0x7fb0f006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f6400640 1 -- 172.21.15.45:0/3737925256 >> v1:172.21.15.117:6789/0 conn(0x7fb0f015df60 legacy=0x7fb0f00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0efe00640 1 -- 172.21.15.45:0/3737925256 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070467627 0 0) 0x7fb0f00ccaa0 con 0x7fb0f00d0350 2024-04-25T10:26:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f6400640 1 -- 172.21.15.45:0/3737925256 >> v1:172.21.15.45:6789/0 conn(0x7fb0f00d0350 legacy=0x7fb0f006aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f6400640 1 -- 172.21.15.45:0/3737925256 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00cd060 con 0x7fb0f00d07b0 2024-04-25T10:26:46.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.002+0000 7fb0f6400640 1 --2- 172.21.15.45:0/3737925256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f00d07b0 0x7fb0f006b5b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009d90 tx=0x7fb0e801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c9e87a2c54fabb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:46.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0efe00640 1 -- 172.21.15.45:0/3737925256 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80b2070 con 0x7fb0f00d07b0 2024-04-25T10:26:46.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0efe00640 1 -- 172.21.15.45:0/3737925256 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e80a5720 con 0x7fb0f00d07b0 2024-04-25T10:26:46.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0efe00640 1 -- 172.21.15.45:0/3737925256 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80a5af0 con 0x7fb0f00d07b0 2024-04-25T10:26:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- 172.21.15.45:0/3737925256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f00d07b0 msgr2=0x7fb0f006b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 --2- 172.21.15.45:0/3737925256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f00d07b0 0x7fb0f006b5b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009d90 tx=0x7fb0e801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- 172.21.15.45:0/3737925256 shutdown_connections 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 --2- 172.21.15.45:0/3737925256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f00d07b0 0x7fb0f006b5b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- 172.21.15.45:0/3737925256 >> 172.21.15.45:0/3737925256 conn(0x7fb0f00d1910 msgr2=0x7fb0f00d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- 172.21.15.45:0/3737925256 shutdown_connections 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- 172.21.15.45:0/3737925256 wait complete. 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 Processor -- start 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- start start 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d0350 0x7fb0f014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 0x7fb0f00b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f015df60 0x7fb0f014c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00d0ff0 con 0x7fb0f00d07b0 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00d0e70 con 0x7fb0f015df60 2024-04-25T10:26:46.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f8400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f016ba30 con 0x7fb0f00d0350 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 0x7fb0f00b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 0x7fb0f00b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57168/0 (socket says 172.21.15.45:57168) 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.006+0000 7fb0f5000640 1 -- 172.21.15.45:0/2435281021 learned_addr learned my addr 172.21.15.45:0/2435281021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5a00640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d0350 0x7fb0f014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f6400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f015df60 0x7fb0f014c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5000640 1 -- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d0350 msgr2=0x7fb0f014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5000640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d0350 0x7fb0f014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5000640 1 -- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f015df60 msgr2=0x7fb0f014c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5000640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f015df60 0x7fb0f014c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5000640 1 -- 172.21.15.45:0/2435281021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f0153330 con 0x7fb0f00d07b0 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f5000640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 0x7fb0f00b5a00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c4a0 tx=0x7fb0dc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc018070 con 0x7fb0f00d07b0 2024-04-25T10:26:46.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc0074e0 con 0x7fb0f00d07b0 2024-04-25T10:26:46.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc004a30 con 0x7fb0f00d07b0 2024-04-25T10:26:46.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f0153610 con 0x7fb0f00d07b0 2024-04-25T10:26:46.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f0153bd0 con 0x7fb0f00d07b0 2024-04-25T10:26:46.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f004ef30 con 0x7fb0f00d07b0 2024-04-25T10:26:46.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0dc004bd0 con 0x7fb0f00d07b0 2024-04-25T10:26:46.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0ed600640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:46.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.010+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb0dc084340 con 0x7fb0f00d07b0 2024-04-25T10:26:46.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.014+0000 7fb0f5a00640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:46.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.014+0000 7fb0f5a00640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4007fd0 tx=0x7fb0e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.018+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0dc054cb0 con 0x7fb0f00d07b0 2024-04-25T10:26:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.234+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb0f00cc070 con 0x7fb0f00d07b0 2024-04-25T10:26:46.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.234+0000 7fb0ed600640 1 -- 172.21.15.45:0/2435281021 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1749 (secure 0 0 0) 0x7fb0dc058d50 con 0x7fb0f00d07b0 2024-04-25T10:26:46.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:46.238 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":550,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":998,"data_bytes":1023544360,"bytes_used":2428411904,"bytes_avail":770665701376,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358887,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":69615589,"read_op_per_sec":0,"write_op_per_sec":69},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:25:52.983902+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 msgr2=0x7fb0d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4007fd0 tx=0x7fb0e4009040 comp rx=0 tx=0).stop 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 msgr2=0x7fb0f00b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 0x7fb0f00b5a00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c4a0 tx=0x7fb0dc00c960 comp rx=0 tx=0).stop 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 shutdown_connections 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f015df60 0x7fb0f014c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f00d07b0 0x7fb0f00b5a00 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 --2- 172.21.15.45:0/2435281021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f00d0350 0x7fb0f014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 >> 172.21.15.45:0/2435281021 conn(0x7fb0f00d1910 msgr2=0x7fb0f006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 shutdown_connections 2024-04-25T10:26:46.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:46.238+0000 7fb0f8400640 1 -- 172.21.15.45:0/2435281021 wait complete. 2024-04-25T10:26:46.258 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:26:47.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:47.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.406+0000 7f5ac6400640 1 Processor -- start 2024-04-25T10:26:47.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5ac6400640 1 -- start start 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5ac6400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5ac00b0f70 0x7f5ac00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5ac6400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ac00cd2e0 0x7f5ac00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5ac6400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac015f3b0 con 0x7f5ac00cfd90 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5ac6400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac015ef20 con 0x7f5ac00cd2e0 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5ac6400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac00ca810 con 0x7f5ac00b0f70 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ac00cd2e0 0x7f5ac00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ac00cd2e0 0x7f5ac00cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57172/0 (socket says 172.21.15.45:57172) 2024-04-25T10:26:47.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 -- 172.21.15.45:0/1266351294 learned_addr learned my addr 172.21.15.45:0/1266351294 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:47.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abf400640 1 --2- 172.21.15.45:0/1266351294 >> v2:172.21.15.117:3300/0 conn(0x7f5ac00b0f70 0x7f5ac00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 -- 172.21.15.45:0/1266351294 >> v2:172.21.15.117:3300/0 conn(0x7f5ac00b0f70 msgr2=0x7f5ac00b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:47.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 --2- 172.21.15.45:0/1266351294 >> v2:172.21.15.117:3300/0 conn(0x7f5ac00b0f70 0x7f5ac00b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 -- 172.21.15.45:0/1266351294 >> v1:172.21.15.117:6789/0 conn(0x7f5ac00cfd90 legacy=0x7f5ac00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:47.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.410+0000 7f5abfe00640 1 -- 172.21.15.45:0/1266351294 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00b1890 con 0x7f5ac00cd2e0 2024-04-25T10:26:47.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5abfe00640 1 --2- 172.21.15.45:0/1266351294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00cd2e0 0x7f5ac00cba20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0071230 tx=0x7f5ab801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805c4eb9d6f9fd54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5abea00640 1 -- 172.21.15.45:0/1266351294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80a5670 con 0x7f5ac00cd2e0 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5abea00640 1 -- 172.21.15.45:0/1266351294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab80a5810 con 0x7f5ac00cd2e0 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5abea00640 1 -- 172.21.15.45:0/1266351294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80a5be0 con 0x7f5ac00cd2e0 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 -- 172.21.15.45:0/1266351294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00cd2e0 msgr2=0x7f5ac00cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 --2- 172.21.15.45:0/1266351294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00cd2e0 0x7f5ac00cba20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0071230 tx=0x7f5ab801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 -- 172.21.15.45:0/1266351294 shutdown_connections 2024-04-25T10:26:47.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 --2- 172.21.15.45:0/1266351294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00cd2e0 0x7f5ac00cba20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 --2- 172.21.15.45:0/1266351294 >> v2:172.21.15.117:3300/0 conn(0x7f5ac00b0f70 0x7f5ac00b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 -- 172.21.15.45:0/1266351294 >> 172.21.15.45:0/1266351294 conn(0x7f5ac00b1d50 msgr2=0x7f5ac00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 -- 172.21.15.45:0/1266351294 shutdown_connections 2024-04-25T10:26:47.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 -- 172.21.15.45:0/1266351294 wait complete. 2024-04-25T10:26:47.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 Processor -- start 2024-04-25T10:26:47.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.414+0000 7f5ac6400640 1 -- start start 2024-04-25T10:26:47.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 0x7f5ac00b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac00cd2e0 0x7f5ac0156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac00cfd90 0x7f5ac0160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac0161b50 con 0x7f5ac00b0f70 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac01619d0 con 0x7f5ac00cfd90 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac0161cd0 con 0x7f5ac00cd2e0 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 0x7f5ac00b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 0x7f5ac00b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57178/0 (socket says 172.21.15.45:57178) 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 -- 172.21.15.45:0/484751476 learned_addr learned my addr 172.21.15.45:0/484751476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abfe00640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac00cfd90 0x7f5ac0160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ab6a00640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac00cd2e0 0x7f5ac0156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 -- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac00cd2e0 msgr2=0x7f5ac0156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac00cd2e0 0x7f5ac0156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 -- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac00cfd90 msgr2=0x7f5ac0160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac00cfd90 0x7f5ac0160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 -- 172.21.15.45:0/484751476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac00b5710 con 0x7f5ac00b0f70 2024-04-25T10:26:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5abf400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 0x7f5ac00b4b50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000d800 tx=0x7f5ab000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000b6c0 con 0x7f5ac00b0f70 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab0019070 con 0x7f5ac00b0f70 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000ba60 con 0x7f5ac00b0f70 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac00b5150 con 0x7f5ac00b0f70 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac00c5510 con 0x7f5ac00b0f70 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac004ef30 con 0x7f5ac00b0f70 2024-04-25T10:26:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.418+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ab000bc00 con 0x7f5ac00b0f70 2024-04-25T10:26:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.422+0000 7f5ab7e00640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.422+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5ab0085470 con 0x7f5ac00b0f70 2024-04-25T10:26:47.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.422+0000 7f5ab6a00640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.422+0000 7f5ab6a00640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a9c066f60 0x7f5a9c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0161450 tx=0x7f5aa8009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.426+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ab0055d50 con 0x7f5ac00b0f70 2024-04-25T10:26:47.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.618+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ac00cda20 con 0x7f5ac00b0f70 2024-04-25T10:26:47.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ab7e00640 1 -- 172.21.15.45:0/484751476 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f5ab0059df0 con 0x7f5ac00b0f70 2024-04-25T10:26:47.624 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:47.624 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:47.624 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a9c066f60 msgr2=0x7f5a9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a9c066f60 0x7f5a9c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0161450 tx=0x7f5aa8009060 comp rx=0 tx=0).stop 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 msgr2=0x7f5ac00b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 0x7f5ac00b4b50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000d800 tx=0x7f5ab000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 shutdown_connections 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac00cfd90 0x7f5ac0160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ac00cd2e0 0x7f5ac0156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 --2- 172.21.15.45:0/484751476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac00b0f70 0x7f5ac00b4b50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 >> 172.21.15.45:0/484751476 conn(0x7f5ac00b1d50 msgr2=0x7f5ac006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 shutdown_connections 2024-04-25T10:26:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.622+0000 7f5ac6400640 1 -- 172.21.15.45:0/484751476 wait complete. 2024-04-25T10:26:47.647 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:47.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.782+0000 7f0a4d000640 1 Processor -- start 2024-04-25T10:26:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.782+0000 7f0a4d000640 1 -- start start 2024-04-25T10:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a4d000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a4815c080 0x7f0a480669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a4d000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a480672f0 con 0x7f0a48066ee0 2024-04-25T10:26:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a4d000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a48156530 con 0x7f0a48156150 2024-04-25T10:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a4d000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4815d220 con 0x7f0a4815c080 2024-04-25T10:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a46800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a48066ee0 0x7f0a480af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57830/0 (socket says 172.21.15.45:57830) 2024-04-25T10:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a46800640 1 -- 172.21.15.45:0/1458277822 learned_addr learned my addr 172.21.15.45:0/1458277822 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a3d400640 1 --2- 172.21.15.45:0/1458277822 >> v2:172.21.15.117:3300/0 conn(0x7f0a4815c080 0x7f0a480669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a45400640 1 -- 172.21.15.45:0/1458277822 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399040331 0 0) 0x7f0a400014a0 con 0x7f0a48066ee0 2024-04-25T10:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a45400640 1 -- 172.21.15.45:0/1458277822 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a2c003530 con 0x7f0a48066ee0 2024-04-25T10:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a3d400640 1 -- 172.21.15.45:0/1458277822 >> v1:172.21.15.117:6789/0 conn(0x7f0a48156150 legacy=0x7f0a48071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a3d400640 1 -- 172.21.15.45:0/1458277822 >> v1:172.21.15.45:6789/0 conn(0x7f0a48066ee0 legacy=0x7f0a480af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.786+0000 7f0a3d400640 1 -- 172.21.15.45:0/1458277822 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4815c640 con 0x7f0a4815c080 2024-04-25T10:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a3d400640 1 --2- 172.21.15.45:0/1458277822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a4815c080 0x7f0a480669a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0a300097c0 tx=0x7f0a3001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4b8b75a0372598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a45400640 1 -- 172.21.15.45:0/1458277822 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300a7070 con 0x7f0a4815c080 2024-04-25T10:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a45400640 1 -- 172.21.15.45:0/1458277822 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a300b0070 con 0x7f0a4815c080 2024-04-25T10:26:47.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a45400640 1 -- 172.21.15.45:0/1458277822 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300ab890 con 0x7f0a4815c080 2024-04-25T10:26:47.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 -- 172.21.15.45:0/1458277822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a4815c080 msgr2=0x7f0a480669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 --2- 172.21.15.45:0/1458277822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a4815c080 0x7f0a480669a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0a300097c0 tx=0x7f0a3001c470 comp rx=0 tx=0).stop 2024-04-25T10:26:47.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 -- 172.21.15.45:0/1458277822 shutdown_connections 2024-04-25T10:26:47.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 --2- 172.21.15.45:0/1458277822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a4815c080 0x7f0a480669a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 -- 172.21.15.45:0/1458277822 >> 172.21.15.45:0/1458277822 conn(0x7f0a4806ac90 msgr2=0x7f0a4806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 -- 172.21.15.45:0/1458277822 shutdown_connections 2024-04-25T10:26:47.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 -- 172.21.15.45:0/1458277822 wait complete. 2024-04-25T10:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 Processor -- start 2024-04-25T10:26:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.790+0000 7f0a4d000640 1 -- start start 2024-04-25T10:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a48066ee0 0x7f0a48154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 0x7f0a481537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a4815c080 0x7f0a480b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48169830 con 0x7f0a48156150 2024-04-25T10:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481696b0 con 0x7f0a48066ee0 2024-04-25T10:26:47.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481699b0 con 0x7f0a4815c080 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 0x7f0a481537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 0x7f0a481537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57188/0 (socket says 172.21.15.45:57188) 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 -- 172.21.15.45:0/4217028259 learned_addr learned my addr 172.21.15.45:0/4217028259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a45e00640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a48066ee0 0x7f0a48154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a46800640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a4815c080 0x7f0a480b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 -- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a4815c080 msgr2=0x7f0a480b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a4815c080 0x7f0a480b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 -- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a48066ee0 msgr2=0x7f0a48154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a48066ee0 0x7f0a48154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 -- 172.21.15.45:0/4217028259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48155800 con 0x7f0a48156150 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a45e00640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a48066ee0 0x7f0a48154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a46800640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a4815c080 0x7f0a480b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3d400640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 0x7f0a481537c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0a3001a0a0 tx=0x7f0a30019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:47.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300abc30 con 0x7f0a48156150 2024-04-25T10:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a300abdd0 con 0x7f0a48156150 2024-04-25T10:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48154ee0 con 0x7f0a48156150 2024-04-25T10:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300bf690 con 0x7f0a48156150 2024-04-25T10:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a48155490 con 0x7f0a48156150 2024-04-25T10:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4804ef30 con 0x7f0a48156150 2024-04-25T10:26:47.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.794+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a300c7070 con 0x7f0a48156150 2024-04-25T10:26:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.798+0000 7f0a3ea00640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a24066f60 0x7f0a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.798+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0a3011f9d0 con 0x7f0a48156150 2024-04-25T10:26:47.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.798+0000 7f0a45e00640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a24066f60 0x7f0a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:47.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.798+0000 7f0a45e00640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a24066f60 0x7f0a24069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0a380016f0 tx=0x7f0a38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:47.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:47.802+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a30016090 con 0x7f0a48156150 2024-04-25T10:26:48.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.018+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0a4815e190 con 0x7f0a48156150 2024-04-25T10:26:48.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.018+0000 7f0a3ea00640 1 -- 172.21.15.45:0/4217028259 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1749 (secure 0 0 0) 0x7f0a300f0ab0 con 0x7f0a48156150 2024-04-25T10:26:48.022 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:48.022 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":552,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":998,"data_bytes":1023544360,"bytes_used":2428411904,"bytes_avail":770665701376,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358830,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":32839348,"read_op_per_sec":0,"write_op_per_sec":32},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:25:52.983902+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a24066f60 msgr2=0x7f0a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a24066f60 0x7f0a24069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0a380016f0 tx=0x7f0a38009040 comp rx=0 tx=0).stop 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 msgr2=0x7f0a481537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 0x7f0a481537c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f0a3001a0a0 tx=0x7f0a30019210 comp rx=0 tx=0).stop 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 shutdown_connections 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a24066f60 0x7f0a24069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a4815c080 0x7f0a480b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:48.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a48156150 0x7f0a481537c0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 --2- 172.21.15.45:0/4217028259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a48066ee0 0x7f0a48154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 >> 172.21.15.45:0/4217028259 conn(0x7f0a4806ac90 msgr2=0x7f0a480b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 shutdown_connections 2024-04-25T10:26:48.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:48.022+0000 7f0a4d000640 1 -- 172.21.15.45:0/4217028259 wait complete. 2024-04-25T10:26:48.043 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:26:49.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 Processor -- start 2024-04-25T10:26:49.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 -- start start 2024-04-25T10:26:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c157f40 0x7f639c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f639c0af380 0x7f639c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f639c066ee0 0x7f639c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c067350 con 0x7f639c0af380 2024-04-25T10:26:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c0ad840 con 0x7f639c066ee0 2024-04-25T10:26:49.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a3000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c06e7a0 con 0x7f639c157f40 2024-04-25T10:26:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f639c0af380 0x7f639c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f639c0af380 0x7f639c0669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57192/0 (socket says 172.21.15.45:57192) 2024-04-25T10:26:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 -- 172.21.15.45:0/3573143574 learned_addr learned my addr 172.21.15.45:0/3573143574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1c00640 1 --2- 172.21.15.45:0/3573143574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c157f40 0x7f639c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a2600640 1 --2- 172.21.15.45:0/3573143574 >> v2:172.21.15.117:3300/0 conn(0x7f639c066ee0 0x7f639c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 -- 172.21.15.45:0/3573143574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c157f40 msgr2=0x7f639c0aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 --2- 172.21.15.45:0/3573143574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c157f40 0x7f639c0aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 -- 172.21.15.45:0/3573143574 >> v2:172.21.15.117:3300/0 conn(0x7f639c066ee0 msgr2=0x7f639c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 --2- 172.21.15.45:0/3573143574 >> v2:172.21.15.117:3300/0 conn(0x7f639c066ee0 0x7f639c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.198+0000 7f63a1200640 1 -- 172.21.15.45:0/3573143574 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0af9b0 con 0x7f639c0af380 2024-04-25T10:26:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a1200640 1 --2- 172.21.15.45:0/3573143574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c0669a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f639c0aed00 tx=0x7f638801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64382e6bb82d3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f639be00640 1 -- 172.21.15.45:0/3573143574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880b0070 con 0x7f639c0af380 2024-04-25T10:26:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f639be00640 1 -- 172.21.15.45:0/3573143574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63880ab600 con 0x7f639c0af380 2024-04-25T10:26:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f639be00640 1 -- 172.21.15.45:0/3573143574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880ab9d0 con 0x7f639c0af380 2024-04-25T10:26:49.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 -- 172.21.15.45:0/3573143574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 msgr2=0x7f639c0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 --2- 172.21.15.45:0/3573143574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c0669a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f639c0aed00 tx=0x7f638801c260 comp rx=0 tx=0).stop 2024-04-25T10:26:49.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 -- 172.21.15.45:0/3573143574 shutdown_connections 2024-04-25T10:26:49.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 --2- 172.21.15.45:0/3573143574 >> v2:172.21.15.117:3300/0 conn(0x7f639c066ee0 0x7f639c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 --2- 172.21.15.45:0/3573143574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c0669a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 --2- 172.21.15.45:0/3573143574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c157f40 0x7f639c0aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 -- 172.21.15.45:0/3573143574 >> 172.21.15.45:0/3573143574 conn(0x7f639c074310 msgr2=0x7f639c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 -- 172.21.15.45:0/3573143574 shutdown_connections 2024-04-25T10:26:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.202+0000 7f63a3000640 1 -- 172.21.15.45:0/3573143574 wait complete. 2024-04-25T10:26:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 Processor -- start 2024-04-25T10:26:49.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 -- start start 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c066ee0 0x7f639c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f639c157f40 0x7f639c151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c15eee0 con 0x7f639c0af380 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c15e9c0 con 0x7f639c157f40 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c158320 con 0x7f639c066ee0 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57200/0 (socket says 172.21.15.45:57200) 2024-04-25T10:26:49.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 -- 172.21.15.45:0/412628894 learned_addr learned my addr 172.21.15.45:0/412628894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1c00640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c066ee0 0x7f639c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a2600640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f639c157f40 0x7f639c151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 -- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c066ee0 msgr2=0x7f639c152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c066ee0 0x7f639c152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 -- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f639c157f40 msgr2=0x7f639c151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f639c157f40 0x7f639c151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 -- 172.21.15.45:0/412628894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c1529b0 con 0x7f639c0af380 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1c00640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c066ee0 0x7f639c152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a2600640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f639c157f40 0x7f639c151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a1200640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c161aa0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f638801a910 tx=0x7f6388019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880ba070 con 0x7f639c0af380 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63880b4420 con 0x7f639c0af380 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880b0040 con 0x7f639c0af380 2024-04-25T10:26:49.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c152c90 con 0x7f639c0af380 2024-04-25T10:26:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.206+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c153250 con 0x7f639c0af380 2024-04-25T10:26:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.210+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c075120 con 0x7f639c0af380 2024-04-25T10:26:49.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.210+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63880bd070 con 0x7f639c0af380 2024-04-25T10:26:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.210+0000 7f6399600640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f637c066f60 0x7f637c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.210+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f638811e270 con 0x7f639c0af380 2024-04-25T10:26:49.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.210+0000 7f63a1c00640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f637c066f60 0x7f637c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.210+0000 7f63a1c00640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f637c066f60 0x7f637c069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f63900099c0 tx=0x7f6390008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.214+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6388016110 con 0x7f639c0af380 2024-04-25T10:26:49.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.438+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f639c0aefe0 con 0x7f639c0af380 2024-04-25T10:26:49.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.438+0000 7f6399600640 1 -- 172.21.15.45:0/412628894 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f63880ef1a0 con 0x7f639c0af380 2024-04-25T10:26:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:49.442 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:49.443 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f637c066f60 msgr2=0x7f637c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f637c066f60 0x7f637c069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f63900099c0 tx=0x7f6390008040 comp rx=0 tx=0).stop 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 msgr2=0x7f639c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c161aa0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f638801a910 tx=0x7f6388019000 comp rx=0 tx=0).stop 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 shutdown_connections 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f637c066f60 0x7f637c069420 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f639c157f40 0x7f639c151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f639c0af380 0x7f639c161aa0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 --2- 172.21.15.45:0/412628894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f639c066ee0 0x7f639c152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 >> 172.21.15.45:0/412628894 conn(0x7f639c074310 msgr2=0x7f639c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 shutdown_connections 2024-04-25T10:26:49.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.442+0000 7f63a3000640 1 -- 172.21.15.45:0/412628894 wait complete. 2024-04-25T10:26:49.465 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:49.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:49.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc68ae00640 1 Processor -- start 2024-04-25T10:26:49.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc68ae00640 1 -- start start 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc68ae00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc6840b15b0 0x7fc684168990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc68ae00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6840b1af0 con 0x7fc68415e050 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc68ae00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6840b1dd0 con 0x7fc6840d1820 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc68ae00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840b20b0 con 0x7fc6840b15b0 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc67b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc68415e050 0x7fc6840b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57844/0 (socket says 172.21.15.45:57844) 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc67b400640 1 -- 172.21.15.45:0/3955543778 learned_addr learned my addr 172.21.15.45:0/3955543778 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:49.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.602+0000 7fc688e00640 1 --2- 172.21.15.45:0/3955543778 >> v2:172.21.15.117:3300/0 conn(0x7fc6840b15b0 0x7fc684168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735517534 0 0) 0x7fc6840b1af0 con 0x7fc68415e050 2024-04-25T10:26:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc668003530 con 0x7fc68415e050 2024-04-25T10:26:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718690482 0 0) 0x7fc6840b1dd0 con 0x7fc6840d1820 2024-04-25T10:26:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6840b1af0 con 0x7fc6840d1820 2024-04-25T10:26:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc688e00640 1 -- 172.21.15.45:0/3955543778 >> v1:172.21.15.117:6789/0 conn(0x7fc6840d1820 legacy=0x7fc6840d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc688e00640 1 -- 172.21.15.45:0/3955543778 >> v1:172.21.15.45:6789/0 conn(0x7fc68415e050 legacy=0x7fc6840b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc688e00640 1 -- 172.21.15.45:0/3955543778 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840b2390 con 0x7fc6840b15b0 2024-04-25T10:26:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc688e00640 1 --2- 172.21.15.45:0/3955543778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6840b15b0 0x7fc684168990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0097c0 tx=0x7fc67c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2702f1bf488f4986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0b0070 con 0x7fc6840b15b0 2024-04-25T10:26:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67c0ab600 con 0x7fc6840b15b0 2024-04-25T10:26:49.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc683400640 1 -- 172.21.15.45:0/3955543778 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c0ab9d0 con 0x7fc6840b15b0 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc68ae00640 1 -- 172.21.15.45:0/3955543778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6840b15b0 msgr2=0x7fc684168990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.606+0000 7fc68ae00640 1 --2- 172.21.15.45:0/3955543778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6840b15b0 0x7fc684168990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0097c0 tx=0x7fc67c01c410 comp rx=0 tx=0).stop 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- 172.21.15.45:0/3955543778 shutdown_connections 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 --2- 172.21.15.45:0/3955543778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6840b15b0 0x7fc684168990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- 172.21.15.45:0/3955543778 >> 172.21.15.45:0/3955543778 conn(0x7fc68409a9d0 msgr2=0x7fc68409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- 172.21.15.45:0/3955543778 shutdown_connections 2024-04-25T10:26:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- 172.21.15.45:0/3955543778 wait complete. 2024-04-25T10:26:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 Processor -- start 2024-04-25T10:26:49.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- start start 2024-04-25T10:26:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6840b15b0 0x7fc6840c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 0x7fc684150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68415e050 0x7fc6840c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68416bda0 con 0x7fc6840d1820 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68416bc20 con 0x7fc68415e050 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68416bf20 con 0x7fc6840b15b0 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 0x7fc684150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 0x7fc684150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57202/0 (socket says 172.21.15.45:57202) 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 -- 172.21.15.45:0/2836908242 learned_addr learned my addr 172.21.15.45:0/2836908242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc683e00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6840b15b0 0x7fc6840c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc688e00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68415e050 0x7fc6840c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 -- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6840b15b0 msgr2=0x7fc6840c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6840b15b0 0x7fc6840c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 -- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68415e050 msgr2=0x7fc6840c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68415e050 0x7fc6840c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 -- 172.21.15.45:0/2836908242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6841596d0 con 0x7fc6840d1820 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc688e00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68415e050 0x7fc6840c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc683e00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6840b15b0 0x7fc6840c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc67b400640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 0x7fc684150fa0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc66c005890 tx=0x7fc66c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c017070 con 0x7fc6840d1820 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c005cf0 con 0x7fc6840d1820 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c007270 con 0x7fc6840d1820 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.610+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc684159970 con 0x7fc6840d1820 2024-04-25T10:26:49.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc684159f30 con 0x7fc6840d1820 2024-04-25T10:26:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68404ef30 con 0x7fc6840d1820 2024-04-25T10:26:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc66c01a070 con 0x7fc6840d1820 2024-04-25T10:26:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc680c00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc660066f60 0x7fc660069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc66c084430 con 0x7fc6840d1820 2024-04-25T10:26:49.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc683e00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc660066f60 0x7fc660069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:49.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.614+0000 7fc683e00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc660066f60 0x7fc660069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc674001570 tx=0x7fc674009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:49.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.618+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc66c054c80 con 0x7fc6840d1820 2024-04-25T10:26:49.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.838+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc68406b480 con 0x7fc6840d1820 2024-04-25T10:26:49.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.838+0000 7fc680c00640 1 -- 172.21.15.45:0/2836908242 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1750 (secure 0 0 0) 0x7fc66c058d20 con 0x7fc6840d1820 2024-04-25T10:26:49.843 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:49.843 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076546407,"bytes_used":2583810048,"bytes_avail":770510303232,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358836,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":39466486,"read_op_per_sec":0,"write_op_per_sec":40},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:25:52.983902+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc660066f60 msgr2=0x7fc660069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc660066f60 0x7fc660069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc674001570 tx=0x7fc674009040 comp rx=0 tx=0).stop 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 msgr2=0x7fc684150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 0x7fc684150fa0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc66c005890 tx=0x7fc66c00c980 comp rx=0 tx=0).stop 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 shutdown_connections 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc660066f60 0x7fc660069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc68415e050 0x7fc6840c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6840d1820 0x7fc684150fa0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 --2- 172.21.15.45:0/2836908242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6840b15b0 0x7fc6840c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:49.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 >> 172.21.15.45:0/2836908242 conn(0x7fc68409a9d0 msgr2=0x7fc68415d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:49.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 shutdown_connections 2024-04-25T10:26:49.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:49.842+0000 7fc68ae00640 1 -- 172.21.15.45:0/2836908242 wait complete. 2024-04-25T10:26:49.863 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:26:50.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa6200640 1 Processor -- start 2024-04-25T10:26:51.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa6200640 1 -- start start 2024-04-25T10:26:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa6200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa6200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa00b5200 con 0x7f5fa0071120 2024-04-25T10:26:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa6200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa00b38b0 con 0x7f5fa00b3360 2024-04-25T10:26:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa6200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa015ef10 con 0x7f5fa00cf9b0 2024-04-25T10:26:51.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa5800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5fa0071120 0x7f5fa01540c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57858/0 (socket says 172.21.15.45:57858) 2024-04-25T10:26:51.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.014+0000 7f5fa5800640 1 -- 172.21.15.45:0/4201673289 learned_addr learned my addr 172.21.15.45:0/4201673289 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:51.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5fa4e00640 1 --2- 172.21.15.45:0/4201673289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208188278 0 0) 0x7f5fa00b5200 con 0x7f5fa0071120 2024-04-25T10:26:51.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f84003530 con 0x7f5fa0071120 2024-04-25T10:26:51.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183730809 0 0) 0x7f5fa00b38b0 con 0x7f5fa00b3360 2024-04-25T10:26:51.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa00b5200 con 0x7f5fa00b3360 2024-04-25T10:26:51.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5fa4e00640 1 -- 172.21.15.45:0/4201673289 >> v1:172.21.15.117:6789/0 conn(0x7f5fa00b3360 legacy=0x7f5fa0070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5fa4e00640 1 -- 172.21.15.45:0/4201673289 >> v1:172.21.15.45:6789/0 conn(0x7f5fa0071120 legacy=0x7f5fa01540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5fa4e00640 1 -- 172.21.15.45:0/4201673289 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa00b3b90 con 0x7f5fa00cf9b0 2024-04-25T10:26:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5fa4e00640 1 --2- 172.21.15.45:0/4201673289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa015e9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f900097f0 tx=0x7f5f9001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e42b7f6a36d5590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900ab5f0 con 0x7f5fa00cf9b0 2024-04-25T10:26:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f900b0070 con 0x7f5fa00cf9b0 2024-04-25T10:26:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.018+0000 7f5f9fe00640 1 -- 172.21.15.45:0/4201673289 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900ab990 con 0x7f5fa00cf9b0 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- 172.21.15.45:0/4201673289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 msgr2=0x7f5fa015e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 --2- 172.21.15.45:0/4201673289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa015e9d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f900097f0 tx=0x7f5f9001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- 172.21.15.45:0/4201673289 shutdown_connections 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 --2- 172.21.15.45:0/4201673289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa015e9d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- 172.21.15.45:0/4201673289 >> 172.21.15.45:0/4201673289 conn(0x7f5fa00cfe80 msgr2=0x7f5fa00d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- 172.21.15.45:0/4201673289 shutdown_connections 2024-04-25T10:26:51.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- 172.21.15.45:0/4201673289 wait complete. 2024-04-25T10:26:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 Processor -- start 2024-04-25T10:26:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- start start 2024-04-25T10:26:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 0x7f5fa0147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fa00b3360 0x7f5fa014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa0150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa016bcd0 con 0x7f5fa0071120 2024-04-25T10:26:51.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa016bb50 con 0x7f5fa00b3360 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa6200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa016be50 con 0x7f5fa00cf9b0 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 0x7f5fa0147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 0x7f5fa0147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57206/0 (socket says 172.21.15.45:57206) 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 -- 172.21.15.45:0/2059501527 learned_addr learned my addr 172.21.15.45:0/2059501527 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5f97e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fa00b3360 0x7f5fa014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa5800640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa0150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 -- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 msgr2=0x7f5fa0150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa0150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 -- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fa00b3360 msgr2=0x7f5fa014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fa00b3360 0x7f5fa014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5fa4e00640 1 -- 172.21.15.45:0/2059501527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa006e170 con 0x7f5fa0071120 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.022+0000 7f5f97e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fa00b3360 0x7f5fa014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5fa4e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 0x7f5fa0147250 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5f9001a110 tx=0x7f5f90019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900b4a40 con 0x7f5fa0071120 2024-04-25T10:26:51.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f900b4be0 con 0x7f5fa0071120 2024-04-25T10:26:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa006e450 con 0x7f5fa0071120 2024-04-25T10:26:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900be3f0 con 0x7f5fa0071120 2024-04-25T10:26:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa0153040 con 0x7f5fa0071120 2024-04-25T10:26:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa0075120 con 0x7f5fa0071120 2024-04-25T10:26:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f900be590 con 0x7f5fa0071120 2024-04-25T10:26:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f9d600640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f7c066f60 0x7f5f7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5f9011efa0 con 0x7f5fa0071120 2024-04-25T10:26:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.026+0000 7f5f97e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f7c066f60 0x7f5f7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.030+0000 7f5f97e00640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f7c066f60 0x7f5f7c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5fa014d640 tx=0x7f5f88009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:51.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.030+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f900160d0 con 0x7f5fa0071120 2024-04-25T10:26:51.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fa00740e0 con 0x7f5fa0071120 2024-04-25T10:26:51.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5f9d600640 1 -- 172.21.15.45:0/2059501527 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f5f900f0080 con 0x7f5fa0071120 2024-04-25T10:26:51.233 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:51.233 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:51.234 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f7c066f60 msgr2=0x7f5f7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f7c066f60 0x7f5f7c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5fa014d640 tx=0x7f5f88009040 comp rx=0 tx=0).stop 2024-04-25T10:26:51.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 msgr2=0x7f5fa0147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 0x7f5fa0147250 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5f9001a110 tx=0x7f5f90019280 comp rx=0 tx=0).stop 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 shutdown_connections 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f7c066f60 0x7f5f7c069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fa00cf9b0 0x7f5fa0150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fa00b3360 0x7f5fa014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 --2- 172.21.15.45:0/2059501527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fa0071120 0x7f5fa0147250 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 >> 172.21.15.45:0/2059501527 conn(0x7f5fa00cfe80 msgr2=0x7f5fa00b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 shutdown_connections 2024-04-25T10:26:51.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.230+0000 7f5fa6200640 1 -- 172.21.15.45:0/2059501527 wait complete. 2024-04-25T10:26:51.255 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:51.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:51.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.390+0000 7f7897800640 1 Processor -- start 2024-04-25T10:26:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7897800640 1 -- start start 2024-04-25T10:26:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7897800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f78900cf930 0x7f78900cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7897800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78900ae020 0x7f78900ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7897800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900d0250 con 0x7f7890070c20 2024-04-25T10:26:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7897800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789015bb90 con 0x7f78900ae020 2024-04-25T10:26:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7897800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900ad840 con 0x7f78900cf930 2024-04-25T10:26:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78900ae020 0x7f78900ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78900ae020 0x7f78900ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57220/0 (socket says 172.21.15.45:57220) 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 -- 172.21.15.45:0/1321844357 learned_addr learned my addr 172.21.15.45:0/1321844357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7894e00640 1 --2- 172.21.15.45:0/1321844357 >> v2:172.21.15.117:3300/0 conn(0x7f78900cf930 0x7f78900cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 -- 172.21.15.45:0/1321844357 >> v2:172.21.15.117:3300/0 conn(0x7f78900cf930 msgr2=0x7f78900cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 --2- 172.21.15.45:0/1321844357 >> v2:172.21.15.117:3300/0 conn(0x7f78900cf930 0x7f78900cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 -- 172.21.15.45:0/1321844357 >> v1:172.21.15.117:6789/0 conn(0x7f7890070c20 legacy=0x7f789015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.394+0000 7f7895800640 1 -- 172.21.15.45:0/1321844357 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789015e9f0 con 0x7f78900ae020 2024-04-25T10:26:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7895800640 1 --2- 172.21.15.45:0/1321844357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78900ae020 0x7f78900ae4b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f78900aec50 tx=0x7f788801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0ee85cc9393bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f788fe00640 1 -- 172.21.15.45:0/1321844357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880a5880 con 0x7f78900ae020 2024-04-25T10:26:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f788fe00640 1 -- 172.21.15.45:0/1321844357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78880a5a20 con 0x7f78900ae020 2024-04-25T10:26:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f788fe00640 1 -- 172.21.15.45:0/1321844357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880a5df0 con 0x7f78900ae020 2024-04-25T10:26:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 -- 172.21.15.45:0/1321844357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78900ae020 msgr2=0x7f78900ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 --2- 172.21.15.45:0/1321844357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78900ae020 0x7f78900ae4b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f78900aec50 tx=0x7f788801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 -- 172.21.15.45:0/1321844357 shutdown_connections 2024-04-25T10:26:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 --2- 172.21.15.45:0/1321844357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78900ae020 0x7f78900ae4b0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 --2- 172.21.15.45:0/1321844357 >> v2:172.21.15.117:3300/0 conn(0x7f78900cf930 0x7f78900cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 -- 172.21.15.45:0/1321844357 >> 172.21.15.45:0/1321844357 conn(0x7f7890066b70 msgr2=0x7f7890066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 -- 172.21.15.45:0/1321844357 shutdown_connections 2024-04-25T10:26:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 -- 172.21.15.45:0/1321844357 wait complete. 2024-04-25T10:26:51.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 Processor -- start 2024-04-25T10:26:51.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.398+0000 7f7897800640 1 -- start start 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 0x7f7890153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78900ae020 0x7f7890156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78900cf930 0x7f78901f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789016a8e0 con 0x7f7890070c20 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789006e130 con 0x7f78900cf930 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789016aa60 con 0x7f78900ae020 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 0x7f7890153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 0x7f7890153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57226/0 (socket says 172.21.15.45:57226) 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 -- 172.21.15.45:0/4141619123 learned_addr learned my addr 172.21.15.45:0/4141619123 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7887e00640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78900ae020 0x7f7890156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7895800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78900cf930 0x7f78901f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 -- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78900ae020 msgr2=0x7f7890156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78900ae020 0x7f7890156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 -- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78900cf930 msgr2=0x7f78901f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78900cf930 0x7f78901f9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 -- 172.21.15.45:0/4141619123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890155b90 con 0x7f7890070c20 2024-04-25T10:26:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7894e00640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 0x7f7890153e10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f788000e8a0 tx=0x7f788000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880017070 con 0x7f7890070c20 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788000c940 con 0x7f7890070c20 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788000ccf0 con 0x7f7890070c20 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78901fab60 con 0x7f7890070c20 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78901561c0 con 0x7f7890070c20 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.402+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789004ef30 con 0x7f7890070c20 2024-04-25T10:26:51.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.406+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f788001a070 con 0x7f7890070c20 2024-04-25T10:26:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.406+0000 7f788d600640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f786c066f60 0x7f786c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:51.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.406+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7880085290 con 0x7f7890070c20 2024-04-25T10:26:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.406+0000 7f7887e00640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f786c066f60 0x7f786c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:51.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.406+0000 7f7887e00640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f786c066f60 0x7f786c069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f78900ae890 tx=0x7f7878009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:51.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.410+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7880055b70 con 0x7f7890070c20 2024-04-25T10:26:51.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7890066f50 con 0x7f7890070c20 2024-04-25T10:26:51.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f788d600640 1 -- 172.21.15.45:0/4141619123 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1750 (secure 0 0 0) 0x7f7880059c10 con 0x7f7890070c20 2024-04-25T10:26:51.628 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:51.628 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":555,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076546407,"bytes_used":2583810048,"bytes_avail":770510303232,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2358836,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":39466486,"read_op_per_sec":0,"write_op_per_sec":40},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:25:52.983902+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:51.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f786c066f60 msgr2=0x7f786c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f786c066f60 0x7f786c069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f78900ae890 tx=0x7f7878009040 comp rx=0 tx=0).stop 2024-04-25T10:26:51.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 msgr2=0x7f7890153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:51.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 0x7f7890153e10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f788000e8a0 tx=0x7f788000eca0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 shutdown_connections 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f786c066f60 0x7f786c069420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78900cf930 0x7f78901f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78900ae020 0x7f7890156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 --2- 172.21.15.45:0/4141619123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7890070c20 0x7f7890153e10 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 >> 172.21.15.45:0/4141619123 conn(0x7f7890066b70 msgr2=0x7f78900b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 shutdown_connections 2024-04-25T10:26:51.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:51.626+0000 7f7897800640 1 -- 172.21.15.45:0/4141619123 wait complete. 2024-04-25T10:26:51.650 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:26:52.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 Processor -- start 2024-04-25T10:26:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 -- start start 2024-04-25T10:26:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff61c0cda10 0x7ff61c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff61c0cf930 0x7ff61c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0b2770 con 0x7ff61c15bc40 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c06c670 con 0x7ff61c0cf930 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff621a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c06c950 con 0x7ff61c0cda10 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff61c0cf930 0x7ff61c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff61c0cf930 0x7ff61c0cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57232/0 (socket says 172.21.15.45:57232) 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 -- 172.21.15.45:0/2192398822 learned_addr learned my addr 172.21.15.45:0/2192398822 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:52.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff612000640 1 --2- 172.21.15.45:0/2192398822 >> v2:172.21.15.117:3300/0 conn(0x7ff61c0cda10 0x7ff61c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 -- 172.21.15.45:0/2192398822 >> v2:172.21.15.117:3300/0 conn(0x7ff61c0cda10 msgr2=0x7ff61c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 --2- 172.21.15.45:0/2192398822 >> v2:172.21.15.117:3300/0 conn(0x7ff61c0cda10 0x7ff61c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 -- 172.21.15.45:0/2192398822 >> v1:172.21.15.117:6789/0 conn(0x7ff61c15bc40 legacy=0x7ff61c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.802+0000 7ff61b400640 1 -- 172.21.15.45:0/2192398822 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c06cc30 con 0x7ff61c0cf930 2024-04-25T10:26:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff61b400640 1 --2- 172.21.15.45:0/2192398822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cf930 0x7ff61c0cfdc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0726f0 tx=0x7ff61401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d38a6352f6e2e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff61a000640 1 -- 172.21.15.45:0/2192398822 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140b0070 con 0x7ff61c0cf930 2024-04-25T10:26:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff61a000640 1 -- 172.21.15.45:0/2192398822 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6140ab600 con 0x7ff61c0cf930 2024-04-25T10:26:52.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff61a000640 1 -- 172.21.15.45:0/2192398822 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140ab9d0 con 0x7ff61c0cf930 2024-04-25T10:26:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 -- 172.21.15.45:0/2192398822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cf930 msgr2=0x7ff61c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 --2- 172.21.15.45:0/2192398822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cf930 0x7ff61c0cfdc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0726f0 tx=0x7ff61401c490 comp rx=0 tx=0).stop 2024-04-25T10:26:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 -- 172.21.15.45:0/2192398822 shutdown_connections 2024-04-25T10:26:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 --2- 172.21.15.45:0/2192398822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cf930 0x7ff61c0cfdc0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 --2- 172.21.15.45:0/2192398822 >> v2:172.21.15.117:3300/0 conn(0x7ff61c0cda10 0x7ff61c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:52.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 -- 172.21.15.45:0/2192398822 >> 172.21.15.45:0/2192398822 conn(0x7ff61c066b70 msgr2=0x7ff61c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:52.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 -- 172.21.15.45:0/2192398822 shutdown_connections 2024-04-25T10:26:52.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.806+0000 7ff621a00640 1 -- 172.21.15.45:0/2192398822 wait complete. 2024-04-25T10:26:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 Processor -- start 2024-04-25T10:26:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 -- start start 2024-04-25T10:26:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 0x7ff61c14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61c0cf930 0x7ff61c15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c15bc40 0x7ff61c15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b3850 con 0x7ff61c0cda10 2024-04-25T10:26:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b36d0 con 0x7ff61c15bc40 2024-04-25T10:26:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c0b39d0 con 0x7ff61c0cf930 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 0x7ff61c14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 0x7ff61c14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57244/0 (socket says 172.21.15.45:57244) 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 -- 172.21.15.45:0/3245828882 learned_addr learned my addr 172.21.15.45:0/3245828882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff612000640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61c0cf930 0x7ff61c15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61b400640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c15bc40 0x7ff61c15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 -- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61c0cf930 msgr2=0x7ff61c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61c0cf930 0x7ff61c15a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 -- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c15bc40 msgr2=0x7ff61c15ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c15bc40 0x7ff61c15ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 -- 172.21.15.45:0/3245828882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c15c2a0 con 0x7ff61c0cda10 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61b400640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c15bc40 0x7ff61c15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff61aa00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 0x7ff61c14c250 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c730 tx=0x7ff60c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:52.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c016020 con 0x7ff61c0cda10 2024-04-25T10:26:52.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c006d60 con 0x7ff61c0cda10 2024-04-25T10:26:52.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c15c580 con 0x7ff61c0cda10 2024-04-25T10:26:52.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c003bc0 con 0x7ff61c0cda10 2024-04-25T10:26:52.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.810+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c15cb40 con 0x7ff61c0cda10 2024-04-25T10:26:52.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.814+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c04ef30 con 0x7ff61c0cda10 2024-04-25T10:26:52.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.814+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff60c00a850 con 0x7ff61c0cda10 2024-04-25T10:26:52.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.814+0000 7ff613400640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:52.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.814+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff60c084c80 con 0x7ff61c0cda10 2024-04-25T10:26:52.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.814+0000 7ff612000640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:52.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.818+0000 7ff612000640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0cc080 tx=0x7ff604009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:52.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:52.818+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff60c020020 con 0x7ff61c0cda10 2024-04-25T10:26:53.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff61c15cf60 con 0x7ff61c0cda10 2024-04-25T10:26:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff613400640 1 -- 172.21.15.45:0/3245828882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff60c055c40 con 0x7ff61c0cda10 2024-04-25T10:26:53.021 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:53.021 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 msgr2=0x7ff5f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0cc080 tx=0x7ff604009190 comp rx=0 tx=0).stop 2024-04-25T10:26:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 msgr2=0x7ff61c14c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 0x7ff61c14c250 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c730 tx=0x7ff60c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 shutdown_connections 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f8066f60 0x7ff5f8069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff61c15bc40 0x7ff61c15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61c0cf930 0x7ff61c15a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 --2- 172.21.15.45:0/3245828882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61c0cda10 0x7ff61c14c250 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 >> 172.21.15.45:0/3245828882 conn(0x7ff61c066b70 msgr2=0x7ff61c154e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 shutdown_connections 2024-04-25T10:26:53.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.018+0000 7ff621a00640 1 -- 172.21.15.45:0/3245828882 wait complete. 2024-04-25T10:26:53.042 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:53.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:53.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 Processor -- start 2024-04-25T10:26:53.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 -- start start 2024-04-25T10:26:53.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80180cf930 0x7f80180cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f801815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80180d0250 con 0x7f80180ae020 2024-04-25T10:26:53.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801815bb90 con 0x7f80180cf930 2024-04-25T10:26:53.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80180ad840 con 0x7f8018070c20 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801d000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f80180ae020 0x7f80180b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57866/0 (socket says 172.21.15.45:57866) 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f801d000640 1 -- 172.21.15.45:0/1356851697 learned_addr learned my addr 172.21.15.45:0/1356851697 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f8017e00640 1 --2- 172.21.15.45:0/1356851697 >> v2:172.21.15.117:3300/0 conn(0x7f80180cf930 0x7f80180cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f800f400640 1 --2- 172.21.15.45:0/1356851697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f801815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f8017400640 1 -- 172.21.15.45:0/1356851697 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131641890 0 0) 0x7f80180d0250 con 0x7f80180ae020 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f8017400640 1 -- 172.21.15.45:0/1356851697 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ffc003530 con 0x7f80180ae020 2024-04-25T10:26:53.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f800f400640 1 -- 172.21.15.45:0/1356851697 >> v2:172.21.15.117:3300/0 conn(0x7f80180cf930 msgr2=0x7f80180cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f800f400640 1 --2- 172.21.15.45:0/1356851697 >> v2:172.21.15.117:3300/0 conn(0x7f80180cf930 0x7f80180cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f800f400640 1 -- 172.21.15.45:0/1356851697 >> v1:172.21.15.45:6789/0 conn(0x7f80180ae020 legacy=0x7f80180b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f800f400640 1 -- 172.21.15.45:0/1356851697 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801815e9f0 con 0x7f8018070c20 2024-04-25T10:26:53.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.182+0000 7f8017e00640 1 --2- 172.21.15.45:0/1356851697 >> v2:172.21.15.117:3300/0 conn(0x7f80180cf930 0x7f80180cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:53.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f800f400640 1 --2- 172.21.15.45:0/1356851697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f801815e170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80000097c0 tx=0x7f800001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=342659a0b2f40455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f8017400640 1 -- 172.21.15.45:0/1356851697 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a7070 con 0x7f8018070c20 2024-04-25T10:26:53.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f8017400640 1 -- 172.21.15.45:0/1356851697 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000b0070 con 0x7f8018070c20 2024-04-25T10:26:53.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f8017400640 1 -- 172.21.15.45:0/1356851697 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000ab870 con 0x7f8018070c20 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 -- 172.21.15.45:0/1356851697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 msgr2=0x7f801815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 --2- 172.21.15.45:0/1356851697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f801815e170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80000097c0 tx=0x7f800001c470 comp rx=0 tx=0).stop 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 -- 172.21.15.45:0/1356851697 shutdown_connections 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 --2- 172.21.15.45:0/1356851697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f801815e170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 --2- 172.21.15.45:0/1356851697 >> v2:172.21.15.117:3300/0 conn(0x7f80180cf930 0x7f80180cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 -- 172.21.15.45:0/1356851697 >> 172.21.15.45:0/1356851697 conn(0x7f8018066b70 msgr2=0x7f8018066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 -- 172.21.15.45:0/1356851697 shutdown_connections 2024-04-25T10:26:53.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.186+0000 7f801f000640 1 -- 172.21.15.45:0/1356851697 wait complete. 2024-04-25T10:26:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 Processor -- start 2024-04-25T10:26:53.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 -- start start 2024-04-25T10:26:53.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f8018157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80180ae020 0x7f8018154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 0x7f80181546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801806e130 con 0x7f80180cf930 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180ae840 con 0x7f80180ae020 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801815e6b0 con 0x7f8018070c20 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 0x7f80181546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 0x7f80181546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57260/0 (socket says 172.21.15.45:57260) 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 -- 172.21.15.45:0/2130519122 learned_addr learned my addr 172.21.15.45:0/2130519122 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f800f400640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80180ae020 0x7f8018154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f8017e00640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f8018157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 -- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 msgr2=0x7f8018157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f8018157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 -- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80180ae020 msgr2=0x7f8018154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80180ae020 0x7f8018154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 -- 172.21.15.45:0/2130519122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018156170 con 0x7f80180cf930 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801d000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 0x7f80181546a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f801000cb40 tx=0x7f801000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010017070 con 0x7f80180cf930 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801000aca0 con 0x7f80180cf930 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80181585b0 con 0x7f80180cf930 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010012620 con 0x7f80180cf930 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.190+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018158b70 con 0x7f80180cf930 2024-04-25T10:26:53.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.194+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801804ef30 con 0x7f80180cf930 2024-04-25T10:26:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.194+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f801001a070 con 0x7f80180cf930 2024-04-25T10:26:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.194+0000 7f8014c00640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ff4066f60 0x7f7ff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.194+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8010084e70 con 0x7f80180cf930 2024-04-25T10:26:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.194+0000 7f8017e00640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ff4066f60 0x7f7ff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.194+0000 7f8017e00640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ff4066f60 0x7f7ff4069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8008009580 tx=0x7f8008009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.198+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8010055750 con 0x7f80180cf930 2024-04-25T10:26:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.418+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f80180b4810 con 0x7f80180cf930 2024-04-25T10:26:53.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.418+0000 7f8014c00640 1 -- 172.21.15.45:0/2130519122 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1806 (secure 0 0 0) 0x7f80100597f0 con 0x7f80180cf930 2024-04-25T10:26:53.423 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:53.423 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076546407,"bytes_used":2583810048,"bytes_avail":770510303232,"bytes_total":773094113280,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2279761,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":38143461,"read_op_per_sec":0,"write_op_per_sec":38},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:26:50.997164+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ff4066f60 msgr2=0x7f7ff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ff4066f60 0x7f7ff4069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8008009580 tx=0x7f8008009190 comp rx=0 tx=0).stop 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 msgr2=0x7f80181546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 0x7f80181546a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f801000cb40 tx=0x7f801000a060 comp rx=0 tx=0).stop 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 shutdown_connections 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ff4066f60 0x7f7ff4069420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80180cf930 0x7f80181546a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80180ae020 0x7f8018154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 --2- 172.21.15.45:0/2130519122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8018070c20 0x7f8018157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 >> 172.21.15.45:0/2130519122 conn(0x7f8018066b70 msgr2=0x7f80180b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 shutdown_connections 2024-04-25T10:26:53.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.422+0000 7f801f000640 1 -- 172.21.15.45:0/2130519122 wait complete. 2024-04-25T10:26:53.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:26:53.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:53.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T10:26:53.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.582+0000 7ff2cec00640 1 Processor -- start 2024-04-25T10:26:53.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.582+0000 7ff2cec00640 1 -- start start 2024-04-25T10:26:53.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2cec00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2c80d0d00 0x7ff2c80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2cec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c806cf40 0x7ff2c806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2cec00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c80cdab0 con 0x7ff2c80cc200 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2cec00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c806d8f0 con 0x7ff2c80d0d00 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2cec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80cd210 con 0x7ff2c806cf40 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2c80d0d00 0x7ff2c80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c806cf40 0x7ff2c806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2c80d0d00 0x7ff2c80ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43666/0 (socket says 172.21.15.45:43666) 2024-04-25T10:26:53.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c806cf40 0x7ff2c806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57746/0 (socket says 172.21.15.45:57746) 2024-04-25T10:26:53.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 -- 172.21.15.45:0/344582590 learned_addr learned my addr 172.21.15.45:0/344582590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2bfe00640 1 --1- 172.21.15.45:0/344582590 >> v1:172.21.15.117:6789/0 conn(0x7ff2c80cc200 0x7ff2c8162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:51904/0 (socket says 172.21.15.45:51904) 2024-04-25T10:26:53.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7400640 1 -- 172.21.15.45:0/344582590 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551960563 0 0) 0x7ff2c80cdab0 con 0x7ff2c80cc200 2024-04-25T10:26:53.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 -- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c806cf40 msgr2=0x7ff2c806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c806cf40 0x7ff2c806d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 -- 172.21.15.45:0/344582590 >> v1:172.21.15.117:6789/0 conn(0x7ff2c80cc200 legacy=0x7ff2c8162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.586+0000 7ff2c7e00640 1 -- 172.21.15.45:0/344582590 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80cd4f0 con 0x7ff2c80d0d00 2024-04-25T10:26:53.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80ccc30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009870 tx=0x7ff2b801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e5bff021a17c815 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2c7400640 1 -- 172.21.15.45:0/344582590 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80b1070 con 0x7ff2c80d0d00 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2c7400640 1 -- 172.21.15.45:0/344582590 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b80ac620 con 0x7ff2c80d0d00 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2c7400640 1 -- 172.21.15.45:0/344582590 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80ac9f0 con 0x7ff2c80d0d00 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c80d0d00 msgr2=0x7ff2c80ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 --2- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80ccc30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009870 tx=0x7ff2b801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- 172.21.15.45:0/344582590 shutdown_connections 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 --2- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c806cf40 0x7ff2c806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 --2- 172.21.15.45:0/344582590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80ccc30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- 172.21.15.45:0/344582590 >> 172.21.15.45:0/344582590 conn(0x7ff2c8072290 msgr2=0x7ff2c80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- 172.21.15.45:0/344582590 shutdown_connections 2024-04-25T10:26:53.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- 172.21.15.45:0/344582590 wait complete. 2024-04-25T10:26:53.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 Processor -- start 2024-04-25T10:26:53.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- start start 2024-04-25T10:26:53.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c806cf40 0x7ff2c80c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c80cc200 0x7ff2c80aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8162a10 con 0x7ff2c80d0d00 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8162890 con 0x7ff2c806cf40 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.590+0000 7ff2cec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815e040 con 0x7ff2c80cc200 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57268/0 (socket says 172.21.15.45:57268) 2024-04-25T10:26:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3067762835 learned_addr learned my addr 172.21.15.45:0/3067762835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c806cf40 0x7ff2c80c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c80cc200 0x7ff2c80aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c80cc200 msgr2=0x7ff2c80aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c80cc200 0x7ff2c80aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c806cf40 msgr2=0x7ff2c80c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c806cf40 0x7ff2c80c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3067762835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8099460 con 0x7ff2c80d0d00 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c806cf40 0x7ff2c80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c80cc200 0x7ff2c80aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80af110 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009dd0 tx=0x7ff2c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2c4c00640 1 -- 172.21.15.45:0/3067762835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0018020 con 0x7ff2c80d0d00 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2c4c00640 1 -- 172.21.15.45:0/3067762835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0014480 con 0x7ff2c80d0d00 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c8160020 con 0x7ff2c80d0d00 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2c4c00640 1 -- 172.21.15.45:0/3067762835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0014870 con 0x7ff2c80d0d00 2024-04-25T10:26:53.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c8159f90 con 0x7ff2c80d0d00 2024-04-25T10:26:53.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.594+0000 7ff2c4c00640 1 -- 172.21.15.45:0/3067762835 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2c001b020 con 0x7ff2c80d0d00 2024-04-25T10:26:53.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2c4c00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff2a4066f60 0x7ff2a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2c4c00640 1 -- 172.21.15.45:0/3067762835 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff2c0085bb0 con 0x7ff2c80d0d00 2024-04-25T10:26:53.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 Processor -- start 2024-04-25T10:26:53.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff2a4066f60 0x7ff2a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- start start 2024-04-25T10:26:53.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff2a4066f60 0x7ff2a4069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ae000 tx=0x7ff2b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabaa40 0x555bceabae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 0x555bcea76170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bce9581f0 con 0x555bcea75d90 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bce8bee90 con 0x555bceabaa40 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bce846f50 con 0x555bceabb360 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 0x555bcea76170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 0x555bcea76170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57276/0 (socket says 172.21.15.45:57276) 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3217621293 learned_addr learned my addr 172.21.15.45:0/3217621293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabaa40 0x555bceabae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 msgr2=0x555bcea75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabaa40 msgr2=0x555bceabae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabaa40 0x555bceabae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3217621293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bce8968b0 con 0x555bcea75d90 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 0x555bcea76170 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff2c005e650 tx=0x7ff2c00565b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e635b8c50917f638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2bcc00640 1 -- 172.21.15.45:0/3217621293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0082070 con 0x555bcea75d90 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2bcc00640 1 -- 172.21.15.45:0/3217621293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0052ae0 con 0x555bcea75d90 2024-04-25T10:26:53.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2bcc00640 1 -- 172.21.15.45:0/3217621293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c005a3b0 con 0x555bcea75d90 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- 172.21.15.45:0/3217621293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 msgr2=0x555bcea76170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 0x555bcea76170 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff2c005e650 tx=0x7ff2c00565b0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- 172.21.15.45:0/3217621293 shutdown_connections 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bcea75d90 0x555bcea76170 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3217621293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabaa40 0x555bceabae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- 172.21.15.45:0/3217621293 >> 172.21.15.45:0/3217621293 conn(0x555bcea71d40 msgr2=0x555bcea74130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- 172.21.15.45:0/3217621293 shutdown_connections 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.598+0000 7ff2d2144000 1 -- 172.21.15.45:0/3217621293 wait complete. 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- start start 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 Processor -- start 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bceac21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceac2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bce879540 con 0x555bceabaa40 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bce8baf30 con 0x555bcea75d90 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bce88b7c0 con 0x555bceabb360 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 10 client.0 _ll_register_callbacks cb 0x555bce973180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceac2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceac2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57284/0 (socket says 172.21.15.45:57284) 2024-04-25T10:26:53.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 -- 172.21.15.45:0/3290048986 learned_addr learned my addr 172.21.15.45:0/3290048986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bceac21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea7bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 -- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 msgr2=0x555bcea7bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea7bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 -- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 msgr2=0x555bceac21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bceac21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 -- 172.21.15.45:0/3290048986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bce8471a0 con 0x555bceabaa40 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceac2700 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005f80 tx=0x7ff2b0001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbf5cb0f1bfbc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2b5600640 1 -- 172.21.15.45:0/3290048986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0019060 con 0x555bceabaa40 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2b5600640 1 -- 172.21.15.45:0/3290048986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff2b00114c0 con 0x555bceabaa40 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2c0052ae0 con 0x555bceabaa40 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2b5600640 1 -- 172.21.15.45:0/3290048986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b001f050 con 0x555bceabaa40 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 10 client.8685 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:53.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x555bce941840 con 0x555bceabaa40 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2b5600640 1 -- 172.21.15.45:0/3290048986 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0052ae0 con 0x555bceabaa40 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2b5600640 1 -- 172.21.15.45:0/3290048986 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff2b0022020 con 0x555bceabaa40 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 10 client.8685 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 10 client.8685 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 20 client.8685 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bcea7ca80 0x555bcea7ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 4 client.8685 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bce85abe0 con 0x555bcea7ca80 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.602+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bcea7ca80 0x555bcea7ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bcea7ca80 0x555bcea7ce60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ede905d506358a server_cookie=78093019e646f8e3 in_seq=0 out_seq=0 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2b5600640 10 client.8685 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2b5600640 1 -- 172.21.15.45:0/3290048986 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bce85abe0 con 0x555bcea7ca80 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2b5600640 10 client.8685 handle_command_reply: tid=0 2024-04-25T10:26:53.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 client.8685 shutdown 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 msgr2=0x555bceac2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceac2700 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005f80 tx=0x7ff2b0001690 comp rx=0 tx=0).stop 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 shutdown_connections 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabb360 0x555bcea7bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bcea7ca80 0x555bcea7ce60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceac2700 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3290048986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bceac21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 >> 172.21.15.45:0/3290048986 conn(0x555bcea71d40 msgr2=0x555bcea749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2b5600640 0 client.8685 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 shutdown_connections 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 1 -- 172.21.15.45:0/3290048986 wait complete. 2024-04-25T10:26:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.606+0000 7ff2d2144000 20 client.8685 trim_cache size 0 max 16384 2024-04-25T10:26:53.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 Processor -- start 2024-04-25T10:26:53.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- start start 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bcea75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabaa40 0x555bcea75680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 0x555bceab8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8202da0 con 0x555bceabb360 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8202160 con 0x555bcea75d90 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815c470 con 0x555bceabaa40 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 0x555bceab8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 0x555bceab8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57294/0 (socket says 172.21.15.45:57294) 2024-04-25T10:26:53.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3372280410 learned_addr learned my addr 172.21.15.45:0/3372280410 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabaa40 0x555bcea75680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bcea75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabaa40 msgr2=0x555bcea75680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabaa40 0x555bcea75680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 msgr2=0x555bcea75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bcea75140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 -- 172.21.15.45:0/3372280410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bce918a10 con 0x555bceabb360 2024-04-25T10:26:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 0x555bceab8c60 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff2c005a780 tx=0x7ff2c0085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ce28a0a4654797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2b7e00640 1 -- 172.21.15.45:0/3372280410 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0082050 con 0x555bceabb360 2024-04-25T10:26:53.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2b7e00640 1 -- 172.21.15.45:0/3372280410 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0083040 con 0x555bceabb360 2024-04-25T10:26:53.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2b7e00640 1 -- 172.21.15.45:0/3372280410 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0074420 con 0x555bceabb360 2024-04-25T10:26:53.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- 172.21.15.45:0/3372280410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 msgr2=0x555bceab8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 0x555bceab8c60 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff2c005a780 tx=0x7ff2c0085000 comp rx=0 tx=0).stop 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- 172.21.15.45:0/3372280410 shutdown_connections 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabb360 0x555bceab8c60 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bceabaa40 0x555bcea75680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 --2- 172.21.15.45:0/3372280410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bcea75d90 0x555bcea75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- 172.21.15.45:0/3372280410 >> 172.21.15.45:0/3372280410 conn(0x555bcea71d40 msgr2=0x555bceac88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- 172.21.15.45:0/3372280410 shutdown_connections 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- 172.21.15.45:0/3372280410 wait complete. 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 -- start start 2024-04-25T10:26:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.622+0000 7ff2d2144000 1 Processor -- start 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bcea75d90 0x555bceb19ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceabed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabb360 0x555bceabf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bce97bd10 con 0x555bceabaa40 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815ca70 con 0x555bceabb360 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bce882720 con 0x555bcea75d90 2024-04-25T10:26:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 10 client.0 _ll_register_callbacks cb 0x555bce881a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceabed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceabed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57302/0 (socket says 172.21.15.45:57302) 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 -- 172.21.15.45:0/1375170360 learned_addr learned my addr 172.21.15.45:0/1375170360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2ccc00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabb360 0x555bceabf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bcea75d90 0x555bceb19ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 -- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bcea75d90 msgr2=0x555bceb19ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bcea75d90 0x555bceb19ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 -- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabb360 msgr2=0x555bceabf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabb360 0x555bceabf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 -- 172.21.15.45:0/1375170360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bce853b00 con 0x555bceabaa40 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2bfe00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceabed40 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0007e40 tx=0x7ff2b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eabea54969d05c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 1 -- 172.21.15.45:0/1375170360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0001b90 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 1 -- 172.21.15.45:0/1375170360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff2b0007880 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2c0083040 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 1 -- 172.21.15.45:0/1375170360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b0011850 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 10 client.8691 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x555bce865f80 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 1 -- 172.21.15.45:0/1375170360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0083040 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 1 -- 172.21.15.45:0/1375170360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff2b0034050 con 0x555bceabaa40 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 10 client.8691 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 10 client.8691 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 20 client.8691 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bceabf7c0 0x555bceabfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 4 client.8691 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:26:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x555bce8cc3d0 con 0x555bceabf7c0 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bceabf7c0 0x555bceabfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2c7e00640 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bceabf7c0 0x555bceabfba0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedac00a7b84a9ad server_cookie=9b2f9174333113e7 in_seq=0 out_seq=0 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 10 client.8691 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 1 -- 172.21.15.45:0/1375170360 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x555bce8cc3d0 con 0x555bceabf7c0 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 10 client.8691 handle_command_reply: tid=0 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 client.8691 shutdown 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 msgr2=0x555bceabed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceabed40 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0007e40 tx=0x7ff2b0012000 comp rx=0 tx=0).stop 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 shutdown_connections 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bceabb360 0x555bceabf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x555bceabf7c0 0x555bceabfba0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bceabaa40 0x555bceabed40 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 --2- 172.21.15.45:0/1375170360 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555bcea75d90 0x555bceb19ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2b5600640 0 client.8691 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 >> 172.21.15.45:0/1375170360 conn(0x555bcea71d40 msgr2=0x555bcea74130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 shutdown_connections 2024-04-25T10:26:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 1 -- 172.21.15.45:0/1375170360 wait complete. 2024-04-25T10:26:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.626+0000 7ff2d2144000 20 client.8691 trim_cache size 0 max 16384 2024-04-25T10:26:53.636 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:53.636 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:26:53.636 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff2a4066f60 msgr2=0x7ff2a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff2a4066f60 0x7ff2a4069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ae000 tx=0x7ff2b8008040 comp rx=0 tx=0).stop 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 msgr2=0x7ff2c80af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80af110 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009dd0 tx=0x7ff2c000c490 comp rx=0 tx=0).stop 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 shutdown_connections 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff2a4066f60 0x7ff2a4069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2c80d0d00 0x7ff2c80af110 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2c80cc200 0x7ff2c80aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 --2- 172.21.15.45:0/3067762835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2c806cf40 0x7ff2c80c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 >> 172.21.15.45:0/3067762835 conn(0x7ff2c8072290 msgr2=0x7ff2c815eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 shutdown_connections 2024-04-25T10:26:53.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.630+0000 7ff2cec00640 1 -- 172.21.15.45:0/3067762835 wait complete. 2024-04-25T10:26:53.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:53.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:26:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 Processor -- start 2024-04-25T10:26:53.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 -- start start 2024-04-25T10:26:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f434c15dcc0 0x7f434c162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 0x7f434c0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c15f0b0 con 0x7f434c0b5480 2024-04-25T10:26:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434c161800 con 0x7f434c15dcc0 2024-04-25T10:26:53.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f4352400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0b3b20 con 0x7f434c0670b0 2024-04-25T10:26:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 0x7f434c0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f434c15dcc0 0x7f434c162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434be00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 0x7f434c0b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57810/0 (socket says 172.21.15.45:57810) 2024-04-25T10:26:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434be00640 1 -- 172.21.15.45:0/2791570477 learned_addr learned my addr 172.21.15.45:0/2791570477 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434b400640 1 -- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 msgr2=0x7f434c0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434b400640 1 --2- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 0x7f434c0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434b400640 1 -- 172.21.15.45:0/2791570477 >> v1:172.21.15.117:6789/0 conn(0x7f434c0b5480 legacy=0x7f434c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434b400640 1 -- 172.21.15.45:0/2791570477 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c0b3e00 con 0x7f434c15dcc0 2024-04-25T10:26:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.794+0000 7f434be00640 1 --2- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 0x7f434c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:53.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f434b400640 1 --2- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c15dcc0 0x7f434c162250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f433c0097c0 tx=0x7f433c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff2b5ec0f42154b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f434aa00640 1 -- 172.21.15.45:0/2791570477 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0a7070 con 0x7f434c15dcc0 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f434aa00640 1 -- 172.21.15.45:0/2791570477 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c0b0070 con 0x7f434c15dcc0 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f434aa00640 1 -- 172.21.15.45:0/2791570477 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0ab890 con 0x7f434c15dcc0 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 -- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c15dcc0 msgr2=0x7f434c162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 --2- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c15dcc0 0x7f434c162250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f433c0097c0 tx=0x7f433c01c470 comp rx=0 tx=0).stop 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 -- 172.21.15.45:0/2791570477 shutdown_connections 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 --2- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c0670b0 0x7f434c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 --2- 172.21.15.45:0/2791570477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c15dcc0 0x7f434c162250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 -- 172.21.15.45:0/2791570477 >> 172.21.15.45:0/2791570477 conn(0x7f434c06ac90 msgr2=0x7f434c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 -- 172.21.15.45:0/2791570477 shutdown_connections 2024-04-25T10:26:53.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.798+0000 7f4352400640 1 -- 172.21.15.45:0/2791570477 wait complete. 2024-04-25T10:26:53.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 Processor -- start 2024-04-25T10:26:53.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 -- start start 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 0x7f434c155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c0b5480 0x7f434c1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c15dcc0 0x7f434c0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0d0ec0 con 0x7f434c0670b0 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0d0d40 con 0x7f434c0b5480 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0d1040 con 0x7f434c15dcc0 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 0x7f434c155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 0x7f434c155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57308/0 (socket says 172.21.15.45:57308) 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 -- 172.21.15.45:0/2989445532 learned_addr learned my addr 172.21.15.45:0/2989445532 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4342a00640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c0b5480 0x7f434c1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434be00640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c15dcc0 0x7f434c0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 -- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c15dcc0 msgr2=0x7f434c0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c15dcc0 0x7f434c0ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 -- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c0b5480 msgr2=0x7f434c1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c0b5480 0x7f434c1491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 -- 172.21.15.45:0/2989445532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c0ceb30 con 0x7f434c0670b0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f434b400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 0x7f434c155f90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f433c0ad040 tx=0x7f433c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4343e00640 1 -- 172.21.15.45:0/2989445532 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0bf610 con 0x7f434c0670b0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4343e00640 1 -- 172.21.15.45:0/2989445532 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c0bf7d0 con 0x7f434c0670b0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4343e00640 1 -- 172.21.15.45:0/2989445532 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0bfbc0 con 0x7f434c0670b0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c0cee10 con 0x7f434c0670b0 2024-04-25T10:26:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.802+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c0cf3d0 con 0x7f434c0670b0 2024-04-25T10:26:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.806+0000 7f4343e00640 1 -- 172.21.15.45:0/2989445532 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f433c0bd020 con 0x7f434c0670b0 2024-04-25T10:26:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.806+0000 7f4343e00640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4328066f60 0x7f4328069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.806+0000 7f4343e00640 1 -- 172.21.15.45:0/2989445532 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f433c11f470 con 0x7f434c0670b0 2024-04-25T10:26:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.806+0000 7f4355889000 1 Processor -- start 2024-04-25T10:26:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.806+0000 7f4355889000 1 -- start start 2024-04-25T10:26:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.806+0000 7f4342a00640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4328066f60 0x7f4328069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e12db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1dc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1dc9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1c15ff0 con 0x55bfd1e129d0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1b9e150 con 0x55bfd1dc90a0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1c12090 con 0x55bfd1e132f0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e12db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4342a00640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4328066f60 0x7f4328069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f434c06b4c0 tx=0x7f4334009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e12db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57314/0 (socket says 172.21.15.45:57314) 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 -- 172.21.15.45:0/1166176784 learned_addr learned my addr 172.21.15.45:0/1166176784 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4342a00640 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1dc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434be00640 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1dc9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 -- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 msgr2=0x55bfd1dc8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1dc8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 -- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 msgr2=0x55bfd1dc9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1dc9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 -- 172.21.15.45:0/1166176784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd1dac0d0 con 0x55bfd1e129d0 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434b400640 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e12db0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f433c0efd70 tx=0x7f433c0efce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385ba941b21c7030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f433be00640 1 -- 172.21.15.45:0/1166176784 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0c2170 con 0x55bfd1e129d0 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f433be00640 1 -- 172.21.15.45:0/1166176784 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c0b0070 con 0x55bfd1e129d0 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f433be00640 1 -- 172.21.15.45:0/1166176784 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0f3e80 con 0x55bfd1e129d0 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- 172.21.15.45:0/1166176784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 msgr2=0x55bfd1e12db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e12db0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f433c0efd70 tx=0x7f433c0efce0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- 172.21.15.45:0/1166176784 shutdown_connections 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1dc9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1dc8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- 172.21.15.45:0/1166176784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e12db0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- 172.21.15.45:0/1166176784 >> 172.21.15.45:0/1166176784 conn(0x55bfd1dc5020 msgr2=0x55bfd1dc7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- 172.21.15.45:0/1166176784 shutdown_connections 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- 172.21.15.45:0/1166176784 wait complete. 2024-04-25T10:26:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- start start 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 Processor -- start 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1df77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e129d0 0x55bfd1df7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 0x55bfd1df8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1bda250 con 0x55bfd1e132f0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1bd0690 con 0x55bfd1dc90a0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1b9ea40 con 0x55bfd1e129d0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f4355889000 10 client.0 _ll_register_callbacks cb 0x55bfd1db3e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.810+0000 7f434be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 0x55bfd1df8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 0x55bfd1df8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57320/0 (socket says 172.21.15.45:57320) 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 -- 172.21.15.45:0/2616098107 learned_addr learned my addr 172.21.15.45:0/2616098107 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434b400640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1df77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4342a00640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e129d0 0x55bfd1df7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 -- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e129d0 msgr2=0x55bfd1df7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e129d0 0x55bfd1df7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 -- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 msgr2=0x55bfd1df77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1df77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 -- 172.21.15.45:0/2616098107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfd1cc31c0 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434be00640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 0x55bfd1df8270 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f43440079e0 tx=0x7f4344007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe11d48390ead0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 1 -- 172.21.15.45:0/2616098107 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434401c070 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 1 -- 172.21.15.45:0/2616098107 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4344013b70 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f433c0b0070 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 1 -- 172.21.15.45:0/2616098107 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4344027050 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 10 client.8700 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55bfd1cbf170 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 1 -- 172.21.15.45:0/2616098107 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c0b0070 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 1 -- 172.21.15.45:0/2616098107 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f434402a020 con 0x55bfd1e132f0 2024-04-25T10:26:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 10 client.8700 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 10 client.8700 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 20 client.8700 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e22c80 0x55bfd1e25070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 4 client.8700 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfd1bb1c60 con 0x55bfd1e22c80 2024-04-25T10:26:53.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434b400640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e22c80 0x55bfd1e25070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f434b400640 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e22c80 0x55bfd1e25070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3516b994f9899dd3 server_cookie=73194b6036e3e9b4 in_seq=0 out_seq=0 2024-04-25T10:26:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 10 client.8700 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 1 -- 172.21.15.45:0/2616098107 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfd1bb1c60 con 0x55bfd1e22c80 2024-04-25T10:26:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 10 client.8700 handle_command_reply: tid=0 2024-04-25T10:26:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 client.8700 shutdown 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 msgr2=0x55bfd1df8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 0x55bfd1df8270 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f43440079e0 tx=0x7f4344007ea0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 shutdown_connections 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e22c80 0x55bfd1e25070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e132f0 0x55bfd1df8270 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e129d0 0x55bfd1df7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 --2- 172.21.15.45:0/2616098107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1df77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4338c00640 0 client.8700 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 >> 172.21.15.45:0/2616098107 conn(0x55bfd1dc5020 msgr2=0x55bfd1dc7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 shutdown_connections 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 1 -- 172.21.15.45:0/2616098107 wait complete. 2024-04-25T10:26:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.814+0000 7f4355889000 20 client.8700 trim_cache size 0 max 16384 2024-04-25T10:26:53.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 Processor -- start 2024-04-25T10:26:53.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 -- start start 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e6eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1e129d0 0x55bfd1e6f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c202c90 con 0x55bfd1dc90a0 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c201f90 con 0x55bfd1e129d0 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c151bb0 con 0x55bfd1e132f0 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f434b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e6eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f434b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e6eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57336/0 (socket says 172.21.15.45:57336) 2024-04-25T10:26:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f434b400640 1 -- 172.21.15.45:0/3221052768 learned_addr learned my addr 172.21.15.45:0/3221052768 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f4342a00640 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1e129d0 0x55bfd1e6f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.830+0000 7f434be00640 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 -- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 msgr2=0x55bfd1e6f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e6f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 -- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1e129d0 msgr2=0x55bfd1e6f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1e129d0 0x55bfd1e6f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 -- 172.21.15.45:0/3221052768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfd1c5cde0 con 0x55bfd1dc90a0 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1e129d0 0x55bfd1e6f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e6eee0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f433c01c950 tx=0x7f433c0c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ee8990155c2c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f433b400640 1 -- 172.21.15.45:0/3221052768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c10f020 con 0x55bfd1dc90a0 2024-04-25T10:26:53.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f433b400640 1 -- 172.21.15.45:0/3221052768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c10c070 con 0x55bfd1dc90a0 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f433b400640 1 -- 172.21.15.45:0/3221052768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c0ee0e0 con 0x55bfd1dc90a0 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- 172.21.15.45:0/3221052768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 msgr2=0x55bfd1e6eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e6eee0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f433c01c950 tx=0x7f433c0c3000 comp rx=0 tx=0).stop 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- 172.21.15.45:0/3221052768 shutdown_connections 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e6f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1e129d0 0x55bfd1e6f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- 172.21.15.45:0/3221052768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e6eee0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- 172.21.15.45:0/3221052768 >> 172.21.15.45:0/3221052768 conn(0x55bfd1dc5020 msgr2=0x55bfd1e1ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- 172.21.15.45:0/3221052768 shutdown_connections 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- 172.21.15.45:0/3221052768 wait complete. 2024-04-25T10:26:53.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- start start 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 Processor -- start 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e72b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e24fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1bb0e00 con 0x55bfd1e129d0 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c155050 con 0x55bfd1dc90a0 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfd1c15300 con 0x55bfd1e132f0 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4355889000 10 client.0 _ll_register_callbacks cb 0x55bfd1d94040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e73080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e73080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57342/0 (socket says 172.21.15.45:57342) 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 -- 172.21.15.45:0/3327359117 learned_addr learned my addr 172.21.15.45:0/3327359117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:53.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434b400640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e72b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f434be00640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e24fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 -- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 msgr2=0x55bfd1e24fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e24fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 -- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 msgr2=0x55bfd1e72b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e72b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.834+0000 7f4342a00640 1 -- 172.21.15.45:0/3327359117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfd1bb78e0 con 0x55bfd1e129d0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4342a00640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e73080 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f4334015760 tx=0x7f4334000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40955d4d99ece9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 1 -- 172.21.15.45:0/3327359117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334028070 con 0x55bfd1e129d0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 1 -- 172.21.15.45:0/3327359117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4334010040 con 0x55bfd1e129d0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f433c10c070 con 0x55bfd1e129d0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 1 -- 172.21.15.45:0/3327359117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4334033050 con 0x55bfd1e129d0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 10 client.8706 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55bfd1c6f8f0 con 0x55bfd1e129d0 2024-04-25T10:26:53.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 1 -- 172.21.15.45:0/3327359117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c10c070 con 0x55bfd1e129d0 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 1 -- 172.21.15.45:0/3327359117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4334036020 con 0x55bfd1e129d0 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 10 client.8706 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 10 client.8706 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 20 client.8706 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e25d20 0x55bfd1e26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 4 client.8706 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bfd1c23530 con 0x55bfd1e25d20 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f434b400640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e25d20 0x55bfd1e26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f434b400640 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e25d20 0x55bfd1e26130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204605882912d5b7 server_cookie=a9d48d075b9e5610 in_seq=0 out_seq=0 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 10 client.8706 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 1 -- 172.21.15.45:0/3327359117 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55bfd1c23530 con 0x55bfd1e25d20 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 10 client.8706 handle_command_reply: tid=0 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 client.8706 shutdown 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 msgr2=0x55bfd1e73080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e73080 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f4334015760 tx=0x7f4334000bd0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 shutdown_connections 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55bfd1e132f0 0x55bfd1e24fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x55bfd1e25d20 0x55bfd1e26130 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bfd1e129d0 0x55bfd1e73080 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 --2- 172.21.15.45:0/3327359117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bfd1dc90a0 0x55bfd1e72b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4338c00640 0 client.8706 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 >> 172.21.15.45:0/3327359117 conn(0x55bfd1dc5020 msgr2=0x55bfd1e110d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 shutdown_connections 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 1 -- 172.21.15.45:0/3327359117 wait complete. 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4355889000 20 client.8706 trim_cache size 0 max 16384 2024-04-25T10:26:53.845 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:53.846 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:26:53.846 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:26:53.846 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:53.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4328066f60 msgr2=0x7f4328069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4328066f60 0x7f4328069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f434c06b4c0 tx=0x7f4334009040 comp rx=0 tx=0).stop 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 msgr2=0x7f434c155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 0x7f434c155f90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f433c0ad040 tx=0x7f433c01acb0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 shutdown_connections 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4328066f60 0x7f4328069420 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f434c15dcc0 0x7f434c0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f434c0b5480 0x7f434c1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 --2- 172.21.15.45:0/2989445532 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f434c0670b0 0x7f434c155f90 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 >> 172.21.15.45:0/2989445532 conn(0x7f434c06ac90 msgr2=0x7f434c149ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 shutdown_connections 2024-04-25T10:26:53.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:53.838+0000 7f4352400640 1 -- 172.21.15.45:0/2989445532 wait complete. 2024-04-25T10:26:53.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:54.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 Processor -- start 2024-04-25T10:26:54.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 -- start start 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e9415db40 0x7f1e9406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e9406b2c0 0x7f1e9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940b5780 con 0x7f1e940b5010 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940b5a60 con 0x7f1e9406b2c0 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e9a000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940b5d40 con 0x7f1e9415db40 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e93e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1e940b5010 0x7f1e9415df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57870/0 (socket says 172.21.15.45:57870) 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e93e00640 1 -- 172.21.15.45:0/3165742828 learned_addr learned my addr 172.21.15.45:0/3165742828 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e99600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1e9406b2c0 0x7f1e9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e9415db40 0x7f1e9406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e98c00640 1 -- 172.21.15.45:0/3165742828 >> v2:172.21.15.45:3300/0 conn(0x7f1e9406b2c0 msgr2=0x7f1e9406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e98c00640 1 --2- 172.21.15.45:0/3165742828 >> v2:172.21.15.45:3300/0 conn(0x7f1e9406b2c0 0x7f1e9406b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e98c00640 1 -- 172.21.15.45:0/3165742828 >> v1:172.21.15.45:6789/0 conn(0x7f1e940b5010 legacy=0x7f1e9415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e93400640 1 -- 172.21.15.45:0/3165742828 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110507277 0 0) 0x7f1e940b5780 con 0x7f1e940b5010 2024-04-25T10:26:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e98c00640 1 -- 172.21.15.45:0/3165742828 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940b6020 con 0x7f1e9415db40 2024-04-25T10:26:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.002+0000 7f1e99600640 1 --2- 172.21.15.45:0/3165742828 >> v2:172.21.15.45:3300/0 conn(0x7f1e9406b2c0 0x7f1e9406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:54.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e98c00640 1 --2- 172.21.15.45:0/3165742828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e9415db40 0x7f1e9406ac90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e880097c0 tx=0x7f1e8801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5368f348d10adde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e93400640 1 -- 172.21.15.45:0/3165742828 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880b5070 con 0x7f1e9415db40 2024-04-25T10:26:54.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e93400640 1 -- 172.21.15.45:0/3165742828 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e880a8990 con 0x7f1e9415db40 2024-04-25T10:26:54.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e93400640 1 -- 172.21.15.45:0/3165742828 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880a8d60 con 0x7f1e9415db40 2024-04-25T10:26:54.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 -- 172.21.15.45:0/3165742828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e9415db40 msgr2=0x7f1e9406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 --2- 172.21.15.45:0/3165742828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e9415db40 0x7f1e9406ac90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e880097c0 tx=0x7f1e8801ec90 comp rx=0 tx=0).stop 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 -- 172.21.15.45:0/3165742828 shutdown_connections 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 --2- 172.21.15.45:0/3165742828 >> v2:172.21.15.45:3300/0 conn(0x7f1e9406b2c0 0x7f1e9406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 --2- 172.21.15.45:0/3165742828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e9415db40 0x7f1e9406ac90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 -- 172.21.15.45:0/3165742828 >> 172.21.15.45:0/3165742828 conn(0x7f1e940d17b0 msgr2=0x7f1e940d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 -- 172.21.15.45:0/3165742828 shutdown_connections 2024-04-25T10:26:54.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.006+0000 7f1e9a000640 1 -- 172.21.15.45:0/3165742828 wait complete. 2024-04-25T10:26:54.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 Processor -- start 2024-04-25T10:26:54.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- start start 2024-04-25T10:26:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e9406b2c0 0x7f1e941f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e940b5010 0x7f1e9409b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 0x7f1e941e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940b6820 con 0x7f1e9415db40 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940b5450 con 0x7f1e9406b2c0 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940b69a0 con 0x7f1e940b5010 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 0x7f1e941e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 0x7f1e941e4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57362/0 (socket says 172.21.15.45:57362) 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 -- 172.21.15.45:0/2584444661 learned_addr learned my addr 172.21.15.45:0/2584444661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e98c00640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e9406b2c0 0x7f1e941f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e93e00640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e940b5010 0x7f1e9409b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 -- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e940b5010 msgr2=0x7f1e9409b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e940b5010 0x7f1e9409b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 -- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e9406b2c0 msgr2=0x7f1e941f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e9406b2c0 0x7f1e941f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 -- 172.21.15.45:0/2584444661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9409bf80 con 0x7f1e9415db40 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e99600640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 0x7f1e941e4500 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00efc0 tx=0x7f1e8c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c018030 con 0x7f1e9415db40 2024-04-25T10:26:54.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c00cde0 con 0x7f1e9415db40 2024-04-25T10:26:54.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9409c260 con 0x7f1e9415db40 2024-04-25T10:26:54.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c003bc0 con 0x7f1e9415db40 2024-04-25T10:26:54.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9409c820 con 0x7f1e9415db40 2024-04-25T10:26:54.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.010+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e94075120 con 0x7f1e9415db40 2024-04-25T10:26:54.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.014+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e8c01b070 con 0x7f1e9415db40 2024-04-25T10:26:54.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.014+0000 7f1e90c00640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e74066f60 0x7f1e74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.014+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1e8c085990 con 0x7f1e9415db40 2024-04-25T10:26:54.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.014+0000 7f1e98c00640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e74066f60 0x7f1e74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.014+0000 7f1e98c00640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e74066f60 0x7f1e74069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1e880b2000 tx=0x7f1e880aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.018+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e8c010040 con 0x7f1e9415db40 2024-04-25T10:26:54.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.214+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e9406c020 con 0x7f1e9415db40 2024-04-25T10:26:54.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.214+0000 7f1e90c00640 1 -- 172.21.15.45:0/2584444661 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f1e8c0569f0 con 0x7f1e9415db40 2024-04-25T10:26:54.218 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:54.218 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:54.219 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:54.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e74066f60 msgr2=0x7f1e74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e74066f60 0x7f1e74069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1e880b2000 tx=0x7f1e880aa070 comp rx=0 tx=0).stop 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 msgr2=0x7f1e941e4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 0x7f1e941e4500 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00efc0 tx=0x7f1e8c009f90 comp rx=0 tx=0).stop 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 shutdown_connections 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1e74066f60 0x7f1e74069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e9415db40 0x7f1e941e4500 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e940b5010 0x7f1e9409b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 --2- 172.21.15.45:0/2584444661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e9406b2c0 0x7f1e941f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 >> 172.21.15.45:0/2584444661 conn(0x7f1e940d17b0 msgr2=0x7f1e940d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 shutdown_connections 2024-04-25T10:26:54.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.218+0000 7f1e9a000640 1 -- 172.21.15.45:0/2584444661 wait complete. 2024-04-25T10:26:54.239 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:54.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.374+0000 7f9957c00640 1 Processor -- start 2024-04-25T10:26:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.374+0000 7f9957c00640 1 -- start start 2024-04-25T10:26:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9957c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99500d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9957c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f995015e050 0x7f99500b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9957c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99500b1870 con 0x7f99500b13e0 2024-04-25T10:26:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9957c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500b1af0 con 0x7f995015e050 2024-04-25T10:26:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9957c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99500b1dd0 con 0x7f99500d1820 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99500d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f994fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f995015e050 0x7f99500b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99500d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:57868/0 (socket says 172.21.15.45:57868) 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 -- 172.21.15.45:0/958460852 learned_addr learned my addr 172.21.15.45:0/958460852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 -- 172.21.15.45:0/958460852 >> v2:172.21.15.117:3300/0 conn(0x7f995015e050 msgr2=0x7f99500b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 --2- 172.21.15.45:0/958460852 >> v2:172.21.15.117:3300/0 conn(0x7f995015e050 0x7f99500b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 -- 172.21.15.45:0/958460852 >> v1:172.21.15.117:6789/0 conn(0x7f99500b13e0 legacy=0x7f99500ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f9955200640 1 -- 172.21.15.45:0/958460852 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500b20b0 con 0x7f99500d1820 2024-04-25T10:26:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.378+0000 7f994fe00640 1 --2- 172.21.15.45:0/958460852 >> v2:172.21.15.117:3300/0 conn(0x7f995015e050 0x7f99500b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:26:54.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9955200640 1 --2- 172.21.15.45:0/958460852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99500d1c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9944009870 tx=0x7f994401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a9e92c3571916d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f994f400640 1 -- 172.21.15.45:0/958460852 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99440a9620 con 0x7f99500d1820 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f994f400640 1 -- 172.21.15.45:0/958460852 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99440a97c0 con 0x7f99500d1820 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f994f400640 1 -- 172.21.15.45:0/958460852 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99440a9b90 con 0x7f99500d1820 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 -- 172.21.15.45:0/958460852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 msgr2=0x7f99500d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 --2- 172.21.15.45:0/958460852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99500d1c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9944009870 tx=0x7f994401c560 comp rx=0 tx=0).stop 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 -- 172.21.15.45:0/958460852 shutdown_connections 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 --2- 172.21.15.45:0/958460852 >> v2:172.21.15.117:3300/0 conn(0x7f995015e050 0x7f99500b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 --2- 172.21.15.45:0/958460852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99500d1c00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 -- 172.21.15.45:0/958460852 >> 172.21.15.45:0/958460852 conn(0x7f995009a9d0 msgr2=0x7f995009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 -- 172.21.15.45:0/958460852 shutdown_connections 2024-04-25T10:26:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 -- 172.21.15.45:0/958460852 wait complete. 2024-04-25T10:26:54.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 Processor -- start 2024-04-25T10:26:54.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.382+0000 7f9957c00640 1 -- start start 2024-04-25T10:26:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500b13e0 0x7f9950152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99501515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 0x7f995015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995016ab20 con 0x7f995015e050 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995016a9a0 con 0x7f99500b13e0 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995016aca0 con 0x7f99500d1820 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 0x7f995015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 0x7f995015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57368/0 (socket says 172.21.15.45:57368) 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 -- 172.21.15.45:0/1186927588 learned_addr learned my addr 172.21.15.45:0/1186927588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f994fe00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99501515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955200640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500b13e0 0x7f9950152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 -- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 msgr2=0x7f99501515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99501515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 -- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500b13e0 msgr2=0x7f9950152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500b13e0 0x7f9950152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 -- 172.21.15.45:0/1186927588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99501547a0 con 0x7f995015e050 2024-04-25T10:26:54.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955200640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500b13e0 0x7f9950152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:54.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9955c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 0x7f995015bc20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f994800c7a0 tx=0x7f994800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948003b80 con 0x7f995015e050 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9948003d20 con 0x7f995015e050 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994800aa90 con 0x7f995015e050 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9950154a80 con 0x7f995015e050 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9950155040 con 0x7f995015e050 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995004ef30 con 0x7f995015e050 2024-04-25T10:26:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.386+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f994800ac30 con 0x7f995015e050 2024-04-25T10:26:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.390+0000 7f994cc00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9930066f60 0x7f9930069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.390+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9948084e70 con 0x7f995015e050 2024-04-25T10:26:54.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.390+0000 7f9955200640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9930066f60 0x7f9930069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.390+0000 7f9955200640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9930066f60 0x7f9930069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f99440ae000 tx=0x7f99440b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.394+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99500b1870 con 0x7f995015e050 2024-04-25T10:26:54.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.610+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9950161480 con 0x7f995015e050 2024-04-25T10:26:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f994cc00640 1 -- 172.21.15.45:0/1186927588 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1806 (secure 0 0 0) 0x7f9948055ed0 con 0x7f995015e050 2024-04-25T10:26:54.616 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:54.616 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":558,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076617488,"bytes_used":2583875584,"bytes_avail":770510237696,"bytes_total":773094113280,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1886953,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":28584494,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:26:50.997164+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9930066f60 msgr2=0x7f9930069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9930066f60 0x7f9930069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f99440ae000 tx=0x7f99440b3040 comp rx=0 tx=0).stop 2024-04-25T10:26:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 msgr2=0x7f995015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 0x7f995015bc20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f994800c7a0 tx=0x7f994800cc60 comp rx=0 tx=0).stop 2024-04-25T10:26:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 shutdown_connections 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9930066f60 0x7f9930069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f995015e050 0x7f995015bc20 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99500d1820 0x7f99501515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 --2- 172.21.15.45:0/1186927588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99500b13e0 0x7f9950152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 >> 172.21.15.45:0/1186927588 conn(0x7f995009a9d0 msgr2=0x7f995006b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 shutdown_connections 2024-04-25T10:26:54.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.614+0000 7f9957c00640 1 -- 172.21.15.45:0/1186927588 wait complete. 2024-04-25T10:26:54.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:26:54.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:26:54.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T10:26:54.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 Processor -- start 2024-04-25T10:26:54.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 -- start start 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f3c159540 0x7f3f3c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0721f0 0x7f3f3c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c159920 con 0x7f3f3c0aebd0 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c15c2f0 con 0x7f3f3c159540 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f44000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c15c5d0 con 0x7f3f3c0721f0 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f3c159540 0x7f3f3c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f3c159540 0x7f3f3c0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57372/0 (socket says 172.21.15.45:57372) 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 -- 172.21.15.45:0/2463502468 learned_addr learned my addr 172.21.15.45:0/2463502468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f42000640 1 --2- 172.21.15.45:0/2463502468 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0721f0 0x7f3f3c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 -- 172.21.15.45:0/2463502468 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0721f0 msgr2=0x7f3f3c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 --2- 172.21.15.45:0/2463502468 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0721f0 0x7f3f3c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 -- 172.21.15.45:0/2463502468 >> v1:172.21.15.117:6789/0 conn(0x7f3f3c0aebd0 legacy=0x7f3f3c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.778+0000 7f3f41600640 1 -- 172.21.15.45:0/2463502468 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c15c8b0 con 0x7f3f3c159540 2024-04-25T10:26:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f41600640 1 --2- 172.21.15.45:0/2463502468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c159540 0x7f3f3c0ae690 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0cd200 tx=0x7f3f3001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed96e77ee6427a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f3be00640 1 -- 172.21.15.45:0/2463502468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300ab5f0 con 0x7f3f3c159540 2024-04-25T10:26:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f3be00640 1 -- 172.21.15.45:0/2463502468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300b0070 con 0x7f3f3c159540 2024-04-25T10:26:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f3be00640 1 -- 172.21.15.45:0/2463502468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300ab9b0 con 0x7f3f3c159540 2024-04-25T10:26:54.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 -- 172.21.15.45:0/2463502468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c159540 msgr2=0x7f3f3c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 --2- 172.21.15.45:0/2463502468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c159540 0x7f3f3c0ae690 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0cd200 tx=0x7f3f3001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 -- 172.21.15.45:0/2463502468 shutdown_connections 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 --2- 172.21.15.45:0/2463502468 >> v2:172.21.15.117:3300/0 conn(0x7f3f3c0721f0 0x7f3f3c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 --2- 172.21.15.45:0/2463502468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c159540 0x7f3f3c0ae690 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 -- 172.21.15.45:0/2463502468 >> 172.21.15.45:0/2463502468 conn(0x7f3f3c0669a0 msgr2=0x7f3f3c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 -- 172.21.15.45:0/2463502468 shutdown_connections 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.782+0000 7f3f44000640 1 -- 172.21.15.45:0/2463502468 wait complete. 2024-04-25T10:26:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 Processor -- start 2024-04-25T10:26:54.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 -- start start 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 0x7f3f3c1403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c0aebd0 0x7f3f3c0ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c159540 0x7f3f3c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c071220 con 0x7f3f3c0721f0 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0710a0 con 0x7f3f3c0aebd0 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0713a0 con 0x7f3f3c159540 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 0x7f3f3c1403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 0x7f3f3c1403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57378/0 (socket says 172.21.15.45:57378) 2024-04-25T10:26:54.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 -- 172.21.15.45:0/3280855943 learned_addr learned my addr 172.21.15.45:0/3280855943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f42000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c159540 0x7f3f3c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f40c00640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c0aebd0 0x7f3f3c0ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 -- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c159540 msgr2=0x7f3f3c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c159540 0x7f3f3c157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 -- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c0aebd0 msgr2=0x7f3f3c0ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c0aebd0 0x7f3f3c0ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 -- 172.21.15.45:0/3280855943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c1408f0 con 0x7f3f3c0721f0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f41600640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 0x7f3f3c1403b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ad040 tx=0x7f3f3001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f39600640 1 -- 172.21.15.45:0/3280855943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300abe60 con 0x7f3f3c0721f0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f39600640 1 -- 172.21.15.45:0/3280855943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300be5f0 con 0x7f3f3c0721f0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f39600640 1 -- 172.21.15.45:0/3280855943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300be9c0 con 0x7f3f3c0721f0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c155170 con 0x7f3f3c0721f0 2024-04-25T10:26:54.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.786+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c155730 con 0x7f3f3c0721f0 2024-04-25T10:26:54.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f39600640 1 -- 172.21.15.45:0/3280855943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f300b43f0 con 0x7f3f3c0721f0 2024-04-25T10:26:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f39600640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f1c066f60 0x7f3f1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f39600640 1 -- 172.21.15.45:0/3280855943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3f3011f420 con 0x7f3f3c0721f0 2024-04-25T10:26:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f40c00640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f1c066f60 0x7f3f1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f474be000 1 Processor -- start 2024-04-25T10:26:54.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f474be000 1 -- start start 2024-04-25T10:26:54.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.790+0000 7f3f40c00640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f1c066f60 0x7f3f1c069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c153760 tx=0x7f3f28007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130465210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x55613041af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x55613041b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556130263ff0 con 0x556130465750 2024-04-25T10:26:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561301ec150 con 0x55613041b4b0 2024-04-25T10:26:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556130260090 con 0x556130464e30 2024-04-25T10:26:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x55613041af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x55613041af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57386/0 (socket says 172.21.15.45:57386) 2024-04-25T10:26:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 -- 172.21.15.45:0/1903163807 learned_addr learned my addr 172.21.15.45:0/1903163807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f41600640 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130465210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f42000640 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x55613041b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 -- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 msgr2=0x556130465210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130465210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 -- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 msgr2=0x55613041b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x55613041b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 -- 172.21.15.45:0/1903163807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561303fa0d0 con 0x556130465750 2024-04-25T10:26:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f40c00640 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x55613041af70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800fb40 tx=0x7f3f28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8941288a04d47b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f2e000640 1 -- 172.21.15.45:0/1903163807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2801b050 con 0x556130465750 2024-04-25T10:26:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f2e000640 1 -- 172.21.15.45:0/1903163807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28025070 con 0x556130465750 2024-04-25T10:26:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f2e000640 1 -- 172.21.15.45:0/1903163807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28014de0 con 0x556130465750 2024-04-25T10:26:54.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- 172.21.15.45:0/1903163807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 msgr2=0x55613041af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x55613041af70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800fb40 tx=0x7f3f28012000 comp rx=0 tx=0).stop 2024-04-25T10:26:54.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- 172.21.15.45:0/1903163807 shutdown_connections 2024-04-25T10:26:54.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x55613041b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x55613041af70 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- 172.21.15.45:0/1903163807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130465210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- 172.21.15.45:0/1903163807 >> 172.21.15.45:0/1903163807 conn(0x556130413020 msgr2=0x556130415440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- 172.21.15.45:0/1903163807 shutdown_connections 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- 172.21.15.45:0/1903163807 wait complete. 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- start start 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 Processor -- start 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 0x556130421360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556130268620 con 0x556130464e30 2024-04-25T10:26:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55613021e690 con 0x55613041b4b0 2024-04-25T10:26:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561301eca40 con 0x556130465750 2024-04-25T10:26:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.794+0000 7f3f474be000 10 client.0 _ll_register_callbacks cb 0x556130401e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 0x556130421360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 0x556130421360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57396/0 (socket says 172.21.15.45:57396) 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f41600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 -- 172.21.15.45:0/95625524 learned_addr learned my addr 172.21.15.45:0/95625524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f42000640 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 -- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 msgr2=0x5561304218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 -- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 msgr2=0x5561304647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 -- 172.21.15.45:0/95625524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561303111c0 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f40c00640 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 0x556130421360 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f3f28014820 tx=0x7f3f28003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7408c34a067adc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 1 -- 172.21.15.45:0/95625524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28020a80 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 1 -- 172.21.15.45:0/95625524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3f28020c20 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f28025070 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 1 -- 172.21.15.45:0/95625524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f2801b020 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 10 client.8730 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55613030d170 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 1 -- 172.21.15.45:0/95625524 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28025070 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 1 -- 172.21.15.45:0/95625524 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3f28033430 con 0x556130464e30 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 10 client.8730 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 10 client.8730 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:54.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 20 client.8730 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304225e0 0x5561304770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 4 client.8730 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561301ffc60 con 0x5561304225e0 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f41600640 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304225e0 0x5561304770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 10 client.8730 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f41600640 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304225e0 0x5561304770f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1060b839ba3aff92 server_cookie=bfca391eebb231e1 in_seq=0 out_seq=0 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 1 -- 172.21.15.45:0/95625524 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561301ffc60 con 0x5561304225e0 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 10 client.8730 handle_command_reply: tid=0 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 client.8730 shutdown 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 msgr2=0x556130421360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 0x556130421360 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f3f28014820 tx=0x7f3f28003700 comp rx=0 tx=0).stop 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 shutdown_connections 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304225e0 0x5561304770f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130464e30 0x556130421360 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 --2- 172.21.15.45:0/95625524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f22a00640 0 client.8730 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 >> 172.21.15.45:0/95625524 conn(0x556130413020 msgr2=0x55613041c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 shutdown_connections 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 1 -- 172.21.15.45:0/95625524 wait complete. 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.798+0000 7f3f474be000 20 client.8730 trim_cache size 0 max 16384 2024-04-25T10:26:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 Processor -- start 2024-04-25T10:26:54.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 -- start start 2024-04-25T10:26:54.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 0x556130443ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556130464e30 0x5561304722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c15b160 con 0x55613041b4b0 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0b46d0 con 0x556130464e30 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0b3ad0 con 0x556130465750 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 0x556130443ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 0x556130443ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57402/0 (socket says 172.21.15.45:57402) 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 -- 172.21.15.45:0/2826165067 learned_addr learned my addr 172.21.15.45:0/2826165067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f40c00640 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556130464e30 0x5561304722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f42000640 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 -- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 msgr2=0x5561304727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 -- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556130464e30 msgr2=0x5561304722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556130464e30 0x5561304722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f41600640 1 -- 172.21.15.45:0/2826165067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561302aade0 con 0x55613041b4b0 2024-04-25T10:26:54.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.814+0000 7f3f42000640 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:54.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f41600640 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 0x556130443ef0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3f3001c9c0 tx=0x7f3f3011f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9711d9c7cdd76bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f2d600640 1 -- 172.21.15.45:0/2826165067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3011d980 con 0x55613041b4b0 2024-04-25T10:26:54.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f2d600640 1 -- 172.21.15.45:0/2826165067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300b0040 con 0x55613041b4b0 2024-04-25T10:26:54.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f2d600640 1 -- 172.21.15.45:0/2826165067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300ee120 con 0x55613041b4b0 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- 172.21.15.45:0/2826165067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 msgr2=0x556130443ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 0x556130443ef0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3f3001c9c0 tx=0x7f3f3011f040 comp rx=0 tx=0).stop 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- 172.21.15.45:0/2826165067 shutdown_connections 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130465750 0x5561304727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556130464e30 0x5561304722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- 172.21.15.45:0/2826165067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55613041b4b0 0x556130443ef0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- 172.21.15.45:0/2826165067 >> 172.21.15.45:0/2826165067 conn(0x556130413020 msgr2=0x556130471b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- 172.21.15.45:0/2826165067 shutdown_connections 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- 172.21.15.45:0/2826165067 wait complete. 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- start start 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 Processor -- start 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130477a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x556130477f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561301fee00 con 0x556130465750 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0b40d0 con 0x55613041b4b0 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556130263300 con 0x556130464e30 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 10 client.0 _ll_register_callbacks cb 0x5561303e2040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x556130477f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x556130477f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57414/0 (socket says 172.21.15.45:57414) 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 -- 172.21.15.45:0/1584081474 learned_addr learned my addr 172.21.15.45:0/1584081474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f41600640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f40c00640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130477a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 -- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 msgr2=0x556130477a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130477a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 -- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 msgr2=0x5561304774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 -- 172.21.15.45:0/1584081474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561302058e0 con 0x556130465750 2024-04-25T10:26:54.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f42000640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x556130477f60 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3f34004350 tx=0x7f3f34004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebac98c6f215c267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f22a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f22a00640 1 -- 172.21.15.45:0/1584081474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3401c020 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f22a00640 1 -- 172.21.15.45:0/1584081474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3f34007a20 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.818+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f300b0040 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 1 -- 172.21.15.45:0/1584081474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f34026050 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 10 client.8736 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5561302bd8f0 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 1 -- 172.21.15.45:0/1584081474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300b0040 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 1 -- 172.21.15.45:0/1584081474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3f34029020 con 0x556130465750 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 10 client.8736 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 10 client.8736 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 20 client.8736 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304cafd0 0x5561304cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 4 client.8736 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:26:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x556130271530 con 0x5561304cafd0 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f41600640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304cafd0 0x5561304cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f41600640 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304cafd0 0x5561304cd3c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f867079486d0e63 server_cookie=f9bdd399542a892e in_seq=0 out_seq=0 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 10 client.8736 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 1 -- 172.21.15.45:0/1584081474 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x556130271530 con 0x5561304cafd0 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 10 client.8736 handle_command_reply: tid=0 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 client.8736 shutdown 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 msgr2=0x556130477f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x556130477f60 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3f34004350 tx=0x7f3f34004810 comp rx=0 tx=0).stop 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 shutdown_connections 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x5561304cafd0 0x5561304cd3c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556130465750 0x556130477f60 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556130464e30 0x556130477a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 --2- 172.21.15.45:0/1584081474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55613041b4b0 0x5561304774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f22a00640 0 client.8736 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 >> 172.21.15.45:0/1584081474 conn(0x556130413020 msgr2=0x556130472420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 shutdown_connections 2024-04-25T10:26:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 1 -- 172.21.15.45:0/1584081474 wait complete. 2024-04-25T10:26:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f474be000 20 client.8736 trim_cache size 0 max 16384 2024-04-25T10:26:54.829 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:54.829 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:26:54.829 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f1c066f60 msgr2=0x7f3f1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f1c066f60 0x7f3f1c069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c153760 tx=0x7f3f28007e40 comp rx=0 tx=0).stop 2024-04-25T10:26:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 msgr2=0x7f3f3c1403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 0x7f3f3c1403b0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ad040 tx=0x7f3f3001ad20 comp rx=0 tx=0).stop 2024-04-25T10:26:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 shutdown_connections 2024-04-25T10:26:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f1c066f60 0x7f3f1c069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f3c159540 0x7f3f3c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3c0aebd0 0x7f3f3c0ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 --2- 172.21.15.45:0/3280855943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3c0721f0 0x7f3f3c1403b0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 >> 172.21.15.45:0/3280855943 conn(0x7f3f3c0669a0 msgr2=0x7f3f3c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 shutdown_connections 2024-04-25T10:26:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.822+0000 7f3f44000640 1 -- 172.21.15.45:0/3280855943 wait complete. 2024-04-25T10:26:54.845 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:26:54.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:26:54.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 Processor -- start 2024-04-25T10:26:54.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 -- start start 2024-04-25T10:26:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7100ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7100aebd0 0x7ff710071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff710072020 0x7ff71015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7100af030 con 0x7ff7100aebd0 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff710159920 con 0x7ff710072020 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff715c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71015c2f0 con 0x7ff710159540 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7100aebd0 0x7ff710071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7100aebd0 0x7ff710071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48830/0 (socket says 172.21.15.45:48830) 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 -- 172.21.15.45:0/2577032175 learned_addr learned my addr 172.21.15.45:0/2577032175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70ea00640 1 --2- 172.21.15.45:0/2577032175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7100ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70f400640 1 --2- 172.21.15.45:0/2577032175 >> v2:172.21.15.117:3300/0 conn(0x7ff710072020 0x7ff71015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 -- 172.21.15.45:0/2577032175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 msgr2=0x7ff7100ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 --2- 172.21.15.45:0/2577032175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7100ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 -- 172.21.15.45:0/2577032175 >> v2:172.21.15.117:3300/0 conn(0x7ff710072020 msgr2=0x7ff71015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 --2- 172.21.15.45:0/2577032175 >> v2:172.21.15.117:3300/0 conn(0x7ff710072020 0x7ff71015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.986+0000 7ff70e000640 1 -- 172.21.15.45:0/2577032175 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71015c5d0 con 0x7ff7100aebd0 2024-04-25T10:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff70e000640 1 --2- 172.21.15.45:0/2577032175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710071ae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff7100af250 tx=0x7ff6fc01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b53011e1f91ae78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff70d600640 1 -- 172.21.15.45:0/2577032175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0b0070 con 0x7ff7100aebd0 2024-04-25T10:26:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff70d600640 1 -- 172.21.15.45:0/2577032175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0ab600 con 0x7ff7100aebd0 2024-04-25T10:26:54.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff70d600640 1 -- 172.21.15.45:0/2577032175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0ab9d0 con 0x7ff7100aebd0 2024-04-25T10:26:54.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 -- 172.21.15.45:0/2577032175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 msgr2=0x7ff710071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 --2- 172.21.15.45:0/2577032175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710071ae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff7100af250 tx=0x7ff6fc01c230 comp rx=0 tx=0).stop 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 -- 172.21.15.45:0/2577032175 shutdown_connections 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 --2- 172.21.15.45:0/2577032175 >> v2:172.21.15.117:3300/0 conn(0x7ff710072020 0x7ff71015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 --2- 172.21.15.45:0/2577032175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710071ae0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 --2- 172.21.15.45:0/2577032175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7100ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 -- 172.21.15.45:0/2577032175 >> 172.21.15.45:0/2577032175 conn(0x7ff7100669a0 msgr2=0x7ff710066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 -- 172.21.15.45:0/2577032175 shutdown_connections 2024-04-25T10:26:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.990+0000 7ff715c00640 1 -- 172.21.15.45:0/2577032175 wait complete. 2024-04-25T10:26:54.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 Processor -- start 2024-04-25T10:26:54.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 -- start start 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff710072020 0x7ff710150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7101f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710072430 con 0x7ff7100aebd0 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71006c0d0 con 0x7ff710072020 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff710070350 con 0x7ff710159540 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48834/0 (socket says 172.21.15.45:48834) 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 -- 172.21.15.45:0/3882987799 learned_addr learned my addr 172.21.15.45:0/3882987799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70ea00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff710072020 0x7ff710150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70f400640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7101f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 -- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 msgr2=0x7ff7101f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7101f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 -- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff710072020 msgr2=0x7ff710150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff710072020 0x7ff710150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 -- 172.21.15.45:0/3882987799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71015baf0 con 0x7ff7100aebd0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70f400640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7101f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff70e000640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710159b30 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc01a8e0 tx=0x7ff6fc018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff706a00640 1 -- 172.21.15.45:0/3882987799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0ba070 con 0x7ff7100aebd0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff706a00640 1 -- 172.21.15.45:0/3882987799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0b4420 con 0x7ff7100aebd0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff706a00640 1 -- 172.21.15.45:0/3882987799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0b0040 con 0x7ff7100aebd0 2024-04-25T10:26:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7101588f0 con 0x7ff7100aebd0 2024-04-25T10:26:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.994+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71015a580 con 0x7ff7100aebd0 2024-04-25T10:26:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff706a00640 1 -- 172.21.15.45:0/3882987799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6fc0bd070 con 0x7ff7100aebd0 2024-04-25T10:26:55.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff706a00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6f0066f60 0x7ff6f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff706a00640 1 -- 172.21.15.45:0/3882987799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6fc11e2b0 con 0x7ff7100aebd0 2024-04-25T10:26:55.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6f0066f60 0x7ff6f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 Processor -- start 2024-04-25T10:26:55.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 -- start start 2024-04-25T10:26:55.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6f0066f60 0x7ff6f0069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff7100b1cf0 tx=0x7ff700008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf677210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf62a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf62ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf475ff0 con 0x558faf676e30 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf3fe150 con 0x558faf62a980 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558faf472090 con 0x558faf677750 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf677210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf677210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48840/0 (socket says 172.21.15.45:48840) 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 -- 172.21.15.45:0/1509172649 learned_addr learned my addr 172.21.15.45:0/1509172649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70e000640 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf62a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70f400640 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf62ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 -- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 msgr2=0x558faf62a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf62a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 -- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 msgr2=0x558faf62ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf62ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:54.998+0000 7ff70ea00640 1 -- 172.21.15.45:0/1509172649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558faf60c0d0 con 0x558faf676e30 2024-04-25T10:26:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70ea00640 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf677210 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ff700011c60 tx=0x7ff700014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead382a71d02fc48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff6fb400640 1 -- 172.21.15.45:0/1509172649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700011c90 con 0x558faf676e30 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff6fb400640 1 -- 172.21.15.45:0/1509172649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff700026070 con 0x558faf676e30 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff6fb400640 1 -- 172.21.15.45:0/1509172649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700021440 con 0x558faf676e30 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- 172.21.15.45:0/1509172649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 msgr2=0x558faf677210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf677210 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ff700011c60 tx=0x7ff700014000 comp rx=0 tx=0).stop 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- 172.21.15.45:0/1509172649 shutdown_connections 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf62ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf62a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- 172.21.15.45:0/1509172649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf677210 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- 172.21.15.45:0/1509172649 >> 172.21.15.45:0/1509172649 conn(0x558faf6579c0 msgr2=0x558faf659db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- 172.21.15.45:0/1509172649 shutdown_connections 2024-04-25T10:26:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- 172.21.15.45:0/1509172649 wait complete. 2024-04-25T10:26:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- start start 2024-04-25T10:26:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 Processor -- start 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf62a980 0x558faf65d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf676e30 0x558faf65d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf65dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf43a250 con 0x558faf677750 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf430690 con 0x558faf676e30 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558faf3fea40 con 0x558faf62a980 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff719075000 10 client.0 _ll_register_callbacks cb 0x558faf613e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf65dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf65dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48852/0 (socket says 172.21.15.45:48852) 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 -- 172.21.15.45:0/409804719 learned_addr learned my addr 172.21.15.45:0/409804719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70e000640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf676e30 0x558faf65d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70ea00640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf62a980 0x558faf65d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 -- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf62a980 msgr2=0x558faf65d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf62a980 0x558faf65d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 -- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf676e30 msgr2=0x558faf65d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf676e30 0x558faf65d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.002+0000 7ff70f400640 1 -- 172.21.15.45:0/409804719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558faf5231c0 con 0x558faf677750 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff70f400640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf65dba0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff708007da0 tx=0x7ff7080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa1db3de5fb6d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 1 -- 172.21.15.45:0/409804719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70801b020 con 0x558faf677750 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 1 -- 172.21.15.45:0/409804719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff708011e40 con 0x558faf677750 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff700026070 con 0x558faf677750 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 1 -- 172.21.15.45:0/409804719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff708025050 con 0x558faf677750 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 10 client.8748 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x558faf51f170 con 0x558faf677750 2024-04-25T10:26:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 1 -- 172.21.15.45:0/409804719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff700026070 con 0x558faf677750 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 1 -- 172.21.15.45:0/409804719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff708028020 con 0x558faf677750 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 10 client.8748 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 10 client.8748 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 20 client.8748 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf686ed0 0x558faf6892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 4 client.8748 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558faf411c60 con 0x558faf686ed0 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff70ea00640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf686ed0 0x558faf6892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff70ea00640 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf686ed0 0x558faf6892c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fad3a1e0cdcb72 server_cookie=5f74f31ff0390fc3 in_seq=0 out_seq=0 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 10 client.8748 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 1 -- 172.21.15.45:0/409804719 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558faf411c60 con 0x558faf686ed0 2024-04-25T10:26:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 10 client.8748 handle_command_reply: tid=0 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 client.8748 shutdown 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 msgr2=0x558faf65dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf65dba0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7ff708007da0 tx=0x7ff7080115a0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 shutdown_connections 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf686ed0 0x558faf6892c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf65dba0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf676e30 0x558faf65d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 --2- 172.21.15.45:0/409804719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf62a980 0x558faf65d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 >> 172.21.15.45:0/409804719 conn(0x558faf6579c0 msgr2=0x558faf659db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff6efe00640 0 client.8748 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 shutdown_connections 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 1 -- 172.21.15.45:0/409804719 wait complete. 2024-04-25T10:26:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.006+0000 7ff719075000 20 client.8748 trim_cache size 0 max 16384 2024-04-25T10:26:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 Processor -- start 2024-04-25T10:26:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 -- start start 2024-04-25T10:26:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf686ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf676e30 0x558faf687410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf687950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7101fe8f0 con 0x558faf677750 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100b4400 con 0x558faf62a980 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100b0fd0 con 0x558faf676e30 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf687950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf687950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48856/0 (socket says 172.21.15.45:48856) 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 -- 172.21.15.45:0/2059465691 learned_addr learned my addr 172.21.15.45:0/2059465691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70e000640 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf676e30 0x558faf687410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70ea00640 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf686ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 -- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf676e30 msgr2=0x558faf687410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf676e30 0x558faf687410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 -- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 msgr2=0x558faf686ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf686ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 -- 172.21.15.45:0/2059465691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558faf4bcde0 con 0x558faf677750 2024-04-25T10:26:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70ea00640 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf686ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.022+0000 7ff70f400640 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf687950 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff70800b010 tx=0x7ff708017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74de0df569566996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff6faa00640 1 -- 172.21.15.45:0/2059465691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70801b070 con 0x558faf677750 2024-04-25T10:26:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff6faa00640 1 -- 172.21.15.45:0/2059465691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708009410 con 0x558faf677750 2024-04-25T10:26:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff6faa00640 1 -- 172.21.15.45:0/2059465691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080090c0 con 0x558faf677750 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- 172.21.15.45:0/2059465691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 msgr2=0x558faf687950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf687950 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ff70800b010 tx=0x7ff708017000 comp rx=0 tx=0).stop 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- 172.21.15.45:0/2059465691 shutdown_connections 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf677750 0x558faf687950 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf676e30 0x558faf687410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- 172.21.15.45:0/2059465691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf686ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- 172.21.15.45:0/2059465691 >> 172.21.15.45:0/2059465691 conn(0x558faf6579c0 msgr2=0x558faf684420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- 172.21.15.45:0/2059465691 shutdown_connections 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- 172.21.15.45:0/2059465691 wait complete. 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- start start 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 Processor -- start 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf68af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf692d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf693280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf410e00 con 0x558faf676e30 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100b3e00 con 0x558faf62a980 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558faf475300 con 0x558faf677750 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 10 client.0 _ll_register_callbacks cb 0x558faf5f4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf692d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf692d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48864/0 (socket says 172.21.15.45:48864) 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 -- 172.21.15.45:0/3716984476 learned_addr learned my addr 172.21.15.45:0/3716984476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70ea00640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf68af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70f400640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf693280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 -- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 msgr2=0x558faf693280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf693280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 -- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 msgr2=0x558faf68af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf68af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 -- 172.21.15.45:0/3716984476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558faf4178e0 con 0x558faf676e30 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70f400640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf693280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff70e000640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf692d40 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0eea70 tx=0x7ff6fc11e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7191dcf7f83e846d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff6efe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff6efe00640 1 -- 172.21.15.45:0/3716984476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0f2c30 con 0x558faf676e30 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff6efe00640 1 -- 172.21.15.45:0/3716984476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6fc0f6d60 con 0x558faf676e30 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.026+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff708009410 con 0x558faf676e30 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 1 -- 172.21.15.45:0/3716984476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6fc0f71f0 con 0x558faf676e30 2024-04-25T10:26:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 10 client.8754 fetch_fsmap learned FSMap version 82 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x558faf4cf8f0 con 0x558faf676e30 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 1 -- 172.21.15.45:0/3716984476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708009410 con 0x558faf676e30 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 1 -- 172.21.15.45:0/3716984476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6fc0f3b50 con 0x558faf676e30 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 10 client.8754 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 10 client.8754 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 20 client.8754 populate_metadata read hostname 'smithi045' 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf6d69b0 0x558faf6d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 4 client.8754 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 --> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558faf483530 con 0x558faf6d69b0 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff70ea00640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf6d69b0 0x558faf6d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff70ea00640 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf6d69b0 0x558faf6d6dc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8961cc39c1c8cd95 server_cookie=12eba0460b207b4b in_seq=0 out_seq=0 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 10 client.8754 ms_handle_connect on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 1 -- 172.21.15.45:0/3716984476 <== mds.0 v2:172.21.15.45:6832/1838662791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558faf483530 con 0x558faf6d69b0 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 10 client.8754 handle_command_reply: tid=0 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 client.8754 shutdown 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 msgr2=0x558faf692d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf692d40 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0eea70 tx=0x7ff6fc11e120 comp rx=0 tx=0).stop 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 shutdown_connections 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558faf677750 0x558faf693280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:6832/1838662791,v1:172.21.15.45:6834/1838662791] conn(0x558faf6d69b0 0x558faf6d6dc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558faf676e30 0x558faf692d40 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 --2- 172.21.15.45:0/3716984476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faf62a980 0x558faf68af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff6efe00640 0 client.8754 ms_handle_reset on v2:172.21.15.45:6832/1838662791 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 >> 172.21.15.45:0/3716984476 conn(0x558faf6579c0 msgr2=0x558faf629430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 shutdown_connections 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 1 -- 172.21.15.45:0/3716984476 wait complete. 2024-04-25T10:26:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff719075000 20 client.8754 trim_cache size 0 max 16384 2024-04-25T10:26:55.040 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:26:55.040 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:26:55.040 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:26:55.040 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6f0066f60 msgr2=0x7ff6f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6f0066f60 0x7ff6f0069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff7100b1cf0 tx=0x7ff700008040 comp rx=0 tx=0).stop 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 msgr2=0x7ff710159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710159b30 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc01a8e0 tx=0x7ff6fc018fd0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 shutdown_connections 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff6f0066f60 0x7ff6f0069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff710159540 0x7ff7101f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7100aebd0 0x7ff710159b30 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 --2- 172.21.15.45:0/3882987799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff710072020 0x7ff710150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 >> 172.21.15.45:0/3882987799 conn(0x7ff7100669a0 msgr2=0x7ff710066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 shutdown_connections 2024-04-25T10:26:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.030+0000 7ff715c00640 1 -- 172.21.15.45:0/3882987799 wait complete. 2024-04-25T10:26:55.053 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:26:55.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.190+0000 7f58bb400640 1 Processor -- start 2024-04-25T10:26:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.190+0000 7f58bb400640 1 -- start start 2024-04-25T10:26:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58bb400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d0040 0x7f58b415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58bb400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58b415b740 0x7f58b40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58bb400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b4071050 con 0x7f58b41550b0 2024-04-25T10:26:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58bb400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b415d040 con 0x7f58b415b740 2024-04-25T10:26:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58bb400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40712f0 con 0x7f58b40d0040 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58b415b740 0x7f58b40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58b415b740 0x7f58b40b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37478/0 (socket says 172.21.15.45:37478) 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 -- 172.21.15.45:0/1827034904 learned_addr learned my addr 172.21.15.45:0/1827034904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b8a00640 1 --1- 172.21.15.45:0/1827034904 >> v1:172.21.15.117:6789/0 conn(0x7f58b41550b0 0x7f58b40cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:54040/0 (socket says 172.21.15.45:54040) 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58abe00640 1 --2- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d0040 0x7f58b415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 -- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d0040 msgr2=0x7f58b415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 --2- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d0040 0x7f58b415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 -- 172.21.15.45:0/1827034904 >> v1:172.21.15.117:6789/0 conn(0x7f58b41550b0 legacy=0x7f58b40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.194+0000 7f58b9400640 1 -- 172.21.15.45:0/1827034904 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40715d0 con 0x7f58b415b740 2024-04-25T10:26:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58b9400640 1 --2- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b415b740 0x7f58b40b3260 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633a1e086c2c7456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58b3e00640 1 -- 172.21.15.45:0/1827034904 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0a58d0 con 0x7f58b415b740 2024-04-25T10:26:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58b3e00640 1 -- 172.21.15.45:0/1827034904 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac0a5a70 con 0x7f58b415b740 2024-04-25T10:26:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58b3e00640 1 -- 172.21.15.45:0/1827034904 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0a5e40 con 0x7f58b415b740 2024-04-25T10:26:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 -- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b415b740 msgr2=0x7f58b40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 --2- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b415b740 0x7f58b40b3260 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009e50 tx=0x7f58ac01ca00 comp rx=0 tx=0).stop 2024-04-25T10:26:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 -- 172.21.15.45:0/1827034904 shutdown_connections 2024-04-25T10:26:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 --2- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b415b740 0x7f58b40b3260 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 --2- 172.21.15.45:0/1827034904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b40d0040 0x7f58b415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 -- 172.21.15.45:0/1827034904 >> 172.21.15.45:0/1827034904 conn(0x7f58b4066b70 msgr2=0x7f58b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 -- 172.21.15.45:0/1827034904 shutdown_connections 2024-04-25T10:26:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 -- 172.21.15.45:0/1827034904 wait complete. 2024-04-25T10:26:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 Processor -- start 2024-04-25T10:26:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.198+0000 7f58bb400640 1 -- start start 2024-04-25T10:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b40d0040 0x7f58b414b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b41550b0 0x7f58b4155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 0x7f58b41579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b415bd70 con 0x7f58b415b740 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b415bbf0 con 0x7f58b40d0040 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b415c030 con 0x7f58b41550b0 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 0x7f58b41579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 0x7f58b41579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48874/0 (socket says 172.21.15.45:48874) 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 -- 172.21.15.45:0/1568344114 learned_addr learned my addr 172.21.15.45:0/1568344114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b8a00640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b40d0040 0x7f58b414b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58abe00640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b41550b0 0x7f58b4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 -- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b41550b0 msgr2=0x7f58b4155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b41550b0 0x7f58b4155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 -- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b40d0040 msgr2=0x7f58b414b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b40d0040 0x7f58b414b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 -- 172.21.15.45:0/1568344114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4152e50 con 0x7f58b415b740 2024-04-25T10:26:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58abe00640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b41550b0 0x7f58b4155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b9400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 0x7f58b41579c0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0af040 tx=0x7f58ac0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0bc020 con 0x7f58b415b740 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac007df0 con 0x7f58b415b740 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0a58d0 con 0x7f58b415b740 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b4153130 con 0x7f58b415b740 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b41536f0 con 0x7f58b415b740 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b404ef30 con 0x7f58b415b740 2024-04-25T10:26:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.202+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58ac0bf020 con 0x7f58b415b740 2024-04-25T10:26:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.206+0000 7f58b1600640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5890066f90 0x7f5890069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.206+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f58ac120080 con 0x7f58b415b740 2024-04-25T10:26:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.206+0000 7f58b8a00640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5890066f90 0x7f5890069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.206+0000 7f58b8a00640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5890066f90 0x7f5890069450 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000f70 tx=0x7f58a4007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.210+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58ac016620 con 0x7f58b415b740 2024-04-25T10:26:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.402+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58b40cc430 con 0x7f58b415b740 2024-04-25T10:26:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.402+0000 7f58b1600640 1 -- 172.21.15.45:0/1568344114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f58ac0f1040 con 0x7f58b415b740 2024-04-25T10:26:55.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:55.407 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5890066f90 msgr2=0x7f5890069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5890066f90 0x7f5890069450 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000f70 tx=0x7f58a4007e90 comp rx=0 tx=0).stop 2024-04-25T10:26:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 msgr2=0x7f58b41579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 0x7f58b41579c0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0af040 tx=0x7f58ac0197a0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 shutdown_connections 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5890066f90 0x7f5890069450 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58b415b740 0x7f58b41579c0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58b41550b0 0x7f58b4155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 --2- 172.21.15.45:0/1568344114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b40d0040 0x7f58b414b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 >> 172.21.15.45:0/1568344114 conn(0x7f58b4066b70 msgr2=0x7f58b406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 shutdown_connections 2024-04-25T10:26:55.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.406+0000 7f58bb400640 1 -- 172.21.15.45:0/1568344114 wait complete. 2024-04-25T10:26:55.428 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:55.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 Processor -- start 2024-04-25T10:26:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 -- start start 2024-04-25T10:26:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fda740ce8a0 0x7fda7406aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda740b2af0 0x7fda7406b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740d0950 con 0x7fda7415de60 2024-04-25T10:26:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740d0c30 con 0x7fda740ce8a0 2024-04-25T10:26:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda7b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda740d0f10 con 0x7fda740b2af0 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda78a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fda7415de60 0x7fda740ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:54042/0 (socket says 172.21.15.45:54042) 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda6be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fda740ce8a0 0x7fda7406aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda78a00640 1 -- 172.21.15.45:0/2118476114 learned_addr learned my addr 172.21.15.45:0/2118476114 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda79400640 1 --2- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda740b2af0 0x7fda7406b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda73e00640 1 -- 172.21.15.45:0/2118476114 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765183784 0 0) 0x7fda740d0950 con 0x7fda7415de60 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda73e00640 1 -- 172.21.15.45:0/2118476114 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda58003530 con 0x7fda7415de60 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda6be00640 1 -- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda740b2af0 msgr2=0x7fda7406b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda6be00640 1 --2- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda740b2af0 0x7fda7406b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda6be00640 1 -- 172.21.15.45:0/2118476114 >> v1:172.21.15.117:6789/0 conn(0x7fda7415de60 legacy=0x7fda740ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.566+0000 7fda6be00640 1 -- 172.21.15.45:0/2118476114 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740d11f0 con 0x7fda740ce8a0 2024-04-25T10:26:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda6be00640 1 --2- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740ce8a0 0x7fda7406aea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0097c0 tx=0x7fda5c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26d3bb139227e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda73e00640 1 -- 172.21.15.45:0/2118476114 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0b0070 con 0x7fda740ce8a0 2024-04-25T10:26:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda73e00640 1 -- 172.21.15.45:0/2118476114 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c0ab600 con 0x7fda740ce8a0 2024-04-25T10:26:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda73e00640 1 -- 172.21.15.45:0/2118476114 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0ab9d0 con 0x7fda740ce8a0 2024-04-25T10:26:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 -- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740ce8a0 msgr2=0x7fda7406aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 --2- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740ce8a0 0x7fda7406aea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0097c0 tx=0x7fda5c01c410 comp rx=0 tx=0).stop 2024-04-25T10:26:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 -- 172.21.15.45:0/2118476114 shutdown_connections 2024-04-25T10:26:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 --2- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda740b2af0 0x7fda7406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 --2- 172.21.15.45:0/2118476114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740ce8a0 0x7fda7406aea0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 -- 172.21.15.45:0/2118476114 >> 172.21.15.45:0/2118476114 conn(0x7fda740d1910 msgr2=0x7fda740d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 -- 172.21.15.45:0/2118476114 shutdown_connections 2024-04-25T10:26:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.570+0000 7fda7b400640 1 -- 172.21.15.45:0/2118476114 wait complete. 2024-04-25T10:26:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 Processor -- start 2024-04-25T10:26:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 -- start start 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740b2af0 0x7fda740b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 0x7fda74156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda7415de60 0x7fda74156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7416afe0 con 0x7fda740ce8a0 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7416ae60 con 0x7fda740b2af0 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7416b160 con 0x7fda7415de60 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 0x7fda74156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 0x7fda74156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48876/0 (socket says 172.21.15.45:48876) 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 -- 172.21.15.45:0/2893459579 learned_addr learned my addr 172.21.15.45:0/2893459579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda79400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda7415de60 0x7fda74156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda78a00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740b2af0 0x7fda740b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 -- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda7415de60 msgr2=0x7fda74156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda7415de60 0x7fda74156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 -- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740b2af0 msgr2=0x7fda740b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740b2af0 0x7fda740b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 -- 172.21.15.45:0/2893459579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740ca480 con 0x7fda740ce8a0 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda78a00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740b2af0 0x7fda740b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda6be00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 0x7fda74156930 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0ae040 tx=0x7fda5c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0b8020 con 0x7fda740ce8a0 2024-04-25T10:26:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c0b4410 con 0x7fda740ce8a0 2024-04-25T10:26:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0b0040 con 0x7fda740ce8a0 2024-04-25T10:26:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda740cb8f0 con 0x7fda740ce8a0 2024-04-25T10:26:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.574+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda740cbe70 con 0x7fda740ce8a0 2024-04-25T10:26:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.578+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7404ef30 con 0x7fda740ce8a0 2024-04-25T10:26:55.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.578+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda5c0bb020 con 0x7fda740ce8a0 2024-04-25T10:26:55.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.578+0000 7fda71600640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda50066f60 0x7fda50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:55.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.578+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fda5c11eb10 con 0x7fda740ce8a0 2024-04-25T10:26:55.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.578+0000 7fda78a00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda50066f60 0x7fda50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:55.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.578+0000 7fda78a00640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda50066f60 0x7fda50069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fda640011f0 tx=0x7fda64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:55.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.582+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda5c0c2030 con 0x7fda740ce8a0 2024-04-25T10:26:55.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.802+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fda740cdab0 con 0x7fda740ce8a0 2024-04-25T10:26:55.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.802+0000 7fda71600640 1 -- 172.21.15.45:0/2893459579 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1773 (secure 0 0 0) 0x7fda5c016090 con 0x7fda740ce8a0 2024-04-25T10:26:55.807 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:55.807 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":559,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076617488,"bytes_used":2583937024,"bytes_avail":770510176256,"bytes_total":773094113280,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1673678,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":25353971,"read_op_per_sec":0,"write_op_per_sec":26},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:26:54.998609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda50066f60 msgr2=0x7fda50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda50066f60 0x7fda50069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fda640011f0 tx=0x7fda64009040 comp rx=0 tx=0).stop 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 msgr2=0x7fda74156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 0x7fda74156930 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0ae040 tx=0x7fda5c0191b0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 shutdown_connections 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fda50066f60 0x7fda50069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda7415de60 0x7fda74156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda740ce8a0 0x7fda74156930 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 --2- 172.21.15.45:0/2893459579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda740b2af0 0x7fda740b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 >> 172.21.15.45:0/2893459579 conn(0x7fda740d1910 msgr2=0x7fda740b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 shutdown_connections 2024-04-25T10:26:55.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:55.806+0000 7fda7b400640 1 -- 172.21.15.45:0/2893459579 wait complete. 2024-04-25T10:26:55.828 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:26:56.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:56.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.978+0000 7f1a2a000640 1 Processor -- start 2024-04-25T10:26:56.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.978+0000 7f1a2a000640 1 -- start start 2024-04-25T10:26:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a2a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a2415b910 0x7f1a2415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a2a000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2415bd60 con 0x7f1a241550b0 2024-04-25T10:26:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a2a000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a24071050 con 0x7f1a240d0040 2024-04-25T10:26:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a2a000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a240712f0 con 0x7f1a2415b910 2024-04-25T10:26:56.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a23400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1a241550b0 0x7f1a240cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45044/0 (socket says 172.21.15.45:45044) 2024-04-25T10:26:56.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a23e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1a2415b910 0x7f1a2415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a23400640 1 -- 172.21.15.45:0/3362188981 learned_addr learned my addr 172.21.15.45:0/3362188981 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:56.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a23e00640 1 -- 172.21.15.45:0/3362188981 >> v1:172.21.15.117:6789/0 conn(0x7f1a240d0040 legacy=0x7f1a2415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:56.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a23e00640 1 -- 172.21.15.45:0/3362188981 >> v1:172.21.15.45:6789/0 conn(0x7f1a241550b0 legacy=0x7f1a240cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a23e00640 1 -- 172.21.15.45:0/3362188981 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240715d0 con 0x7f1a2415b910 2024-04-25T10:26:56.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.982+0000 7f1a22a00640 1 -- 172.21.15.45:0/3362188981 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534200888 0 0) 0x7f1a2415bd60 con 0x7f1a241550b0 2024-04-25T10:26:56.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a23e00640 1 --2- 172.21.15.45:0/3362188981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a2415c030 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009d90 tx=0x7f1a1c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b857e860455e516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a22a00640 1 -- 172.21.15.45:0/3362188981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0b2070 con 0x7f1a2415b910 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a22a00640 1 -- 172.21.15.45:0/3362188981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c0a5a00 con 0x7f1a2415b910 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a22a00640 1 -- 172.21.15.45:0/3362188981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0a5dd0 con 0x7f1a2415b910 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 -- 172.21.15.45:0/3362188981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 msgr2=0x7f1a2415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 --2- 172.21.15.45:0/3362188981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a2415c030 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009d90 tx=0x7f1a1c01c940 comp rx=0 tx=0).stop 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 -- 172.21.15.45:0/3362188981 shutdown_connections 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 --2- 172.21.15.45:0/3362188981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a2415c030 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 -- 172.21.15.45:0/3362188981 >> 172.21.15.45:0/3362188981 conn(0x7f1a24066b70 msgr2=0x7f1a24066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 -- 172.21.15.45:0/3362188981 shutdown_connections 2024-04-25T10:26:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 -- 172.21.15.45:0/3362188981 wait complete. 2024-04-25T10:26:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 Processor -- start 2024-04-25T10:26:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.986+0000 7f1a2a000640 1 -- start start 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a240d0040 0x7f1a241411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a241550b0 0x7f1a241f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a240b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2415cf00 con 0x7f1a2415b910 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2415cd80 con 0x7f1a241550b0 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2415d080 con 0x7f1a240d0040 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a240b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a240b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48894/0 (socket says 172.21.15.45:48894) 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 -- 172.21.15.45:0/2104229455 learned_addr learned my addr 172.21.15.45:0/2104229455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a1aa00640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a241550b0 0x7f1a241f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23400640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a240d0040 0x7f1a241411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 -- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a240d0040 msgr2=0x7f1a241411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a240d0040 0x7f1a241411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 -- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a241550b0 msgr2=0x7f1a241f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a241550b0 0x7f1a241f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 -- 172.21.15.45:0/2104229455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a241583c0 con 0x7f1a2415b910 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a1aa00640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a241550b0 0x7f1a241f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a23e00640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a240b0520 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c01a570 tx=0x7f1a1c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0b2040 con 0x7f1a2415b910 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c0bf070 con 0x7f1a2415b910 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0adad0 con 0x7f1a2415b910 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240ccf70 con 0x7f1a2415b910 2024-04-25T10:26:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a240cd530 con 0x7f1a2415b910 2024-04-25T10:26:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2404ef30 con 0x7f1a2415b910 2024-04-25T10:26:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.990+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a1c0adc70 con 0x7f1a2415b910 2024-04-25T10:26:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.994+0000 7f1a1be00640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a00066f60 0x7f1a00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.994+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1a1c1204b0 con 0x7f1a2415b910 2024-04-25T10:26:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.994+0000 7f1a23400640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a00066f60 0x7f1a00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.994+0000 7f1a23400640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a00066f60 0x7f1a00069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1a14000f70 tx=0x7f1a14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:56.998+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a241583c0 con 0x7f1a2415b910 2024-04-25T10:26:57.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a240ae7d0 con 0x7f1a2415b910 2024-04-25T10:26:57.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a1be00640 1 -- 172.21.15.45:0/2104229455 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f1a1c016560 con 0x7f1a2415b910 2024-04-25T10:26:57.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:57.197 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:57.197 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a00066f60 msgr2=0x7f1a00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a00066f60 0x7f1a00069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1a14000f70 tx=0x7f1a14009040 comp rx=0 tx=0).stop 2024-04-25T10:26:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 msgr2=0x7f1a240b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a240b0520 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c01a570 tx=0x7f1a1c0196e0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 shutdown_connections 2024-04-25T10:26:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a00066f60 0x7f1a00069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2415b910 0x7f1a240b0520 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a241550b0 0x7f1a241f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 --2- 172.21.15.45:0/2104229455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a240d0040 0x7f1a241411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 >> 172.21.15.45:0/2104229455 conn(0x7f1a24066b70 msgr2=0x7f1a240ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 shutdown_connections 2024-04-25T10:26:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.194+0000 7f1a2a000640 1 -- 172.21.15.45:0/2104229455 wait complete. 2024-04-25T10:26:57.217 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:57.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f7866a00640 1 Processor -- start 2024-04-25T10:26:57.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f7866a00640 1 -- start start 2024-04-25T10:26:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f7866a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f786015b910 0x7f786015c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f7866a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786015bd60 con 0x7f78601550b0 2024-04-25T10:26:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f7866a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7860071050 con 0x7f78600d0040 2024-04-25T10:26:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f7866a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78600712f0 con 0x7f786015b910 2024-04-25T10:26:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f785fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f78601550b0 0x7f78600cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45054/0 (socket says 172.21.15.45:45054) 2024-04-25T10:26:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.354+0000 7f785fe00640 1 -- 172.21.15.45:0/1586389676 learned_addr learned my addr 172.21.15.45:0/1586389676 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7864a00640 1 --2- 172.21.15.45:0/1586389676 >> v2:172.21.15.117:3300/0 conn(0x7f786015b910 0x7f786015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115758191 0 0) 0x7f786015bd60 con 0x7f78601550b0 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7844003530 con 0x7f78601550b0 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042260842 0 0) 0x7f7844003530 con 0x7f78601550b0 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 >> v2:172.21.15.117:3300/0 conn(0x7f786015b910 msgr2=0x7f786015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 --2- 172.21.15.45:0/1586389676 >> v2:172.21.15.117:3300/0 conn(0x7f786015b910 0x7f786015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 >> v1:172.21.15.117:6789/0 conn(0x7f78600d0040 legacy=0x7f786015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600715d0 con 0x7f78601550b0 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f7850001680 con 0x7f78601550b0 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 >> v1:172.21.15.45:6789/0 conn(0x7f78601550b0 legacy=0x7f78600cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78440039c0 0x7f7844003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7844004810 0x7f7844004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78440051d0 0x7f784400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600712f0 con 0x7f7844004810 2024-04-25T10:26:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784400c070 con 0x7f78440039c0 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784400e480 con 0x7f78440051d0 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7857400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7844004810 0x7f7844004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f785fe00640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78440039c0 0x7f7844003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7864a00640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78440051d0 0x7f784400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7857400640 1 -- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78440051d0 msgr2=0x7f784400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7857400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78440051d0 0x7f784400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7857400640 1 -- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78440039c0 msgr2=0x7f7844003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7857400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78440039c0 0x7f7844003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7857400640 1 -- 172.21.15.45:0/1586389676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7844003530 con 0x7f7844004810 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7864a00640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78440051d0 0x7f784400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.358+0000 7f7864a00640 1 -- 172.21.15.45:0/1586389676 reap_dead start 2024-04-25T10:26:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7857400640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7844004810 0x7f7844004c90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7848001780 tx=0x7f784801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d963aeeb5956820d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78480013a0 con 0x7f7844004810 2024-04-25T10:26:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78480052f0 con 0x7f7844004810 2024-04-25T10:26:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f785f400640 1 -- 172.21.15.45:0/1586389676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848003840 con 0x7f7844004810 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 -- 172.21.15.45:0/1586389676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7844004810 msgr2=0x7f7844004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7844004810 0x7f7844004c90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7848001780 tx=0x7f784801c400 comp rx=0 tx=0).stop 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 -- 172.21.15.45:0/1586389676 shutdown_connections 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 --2- 172.21.15.45:0/1586389676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7844004810 0x7f7844004c90 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 -- 172.21.15.45:0/1586389676 >> 172.21.15.45:0/1586389676 conn(0x7f7860066b70 msgr2=0x7f7860066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 -- 172.21.15.45:0/1586389676 shutdown_connections 2024-04-25T10:26:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 -- 172.21.15.45:0/1586389676 wait complete. 2024-04-25T10:26:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 Processor -- start 2024-04-25T10:26:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.362+0000 7f7866a00640 1 -- start start 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7844004810 0x7f7860153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 0x7f78600ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78600add80 0x7f786015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600720b0 con 0x7f7860153dd0 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860071b90 con 0x7f7844004810 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786015b200 con 0x7f78600add80 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 0x7f78600ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 0x7f78600ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48908/0 (socket says 172.21.15.45:48908) 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 -- 172.21.15.45:0/3088049670 learned_addr learned my addr 172.21.15.45:0/3088049670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f785fe00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7844004810 0x7f7860153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7864a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78600add80 0x7f786015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 -- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78600add80 msgr2=0x7f786015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78600add80 0x7f786015a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 -- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7844004810 msgr2=0x7f7860153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7844004810 0x7f7860153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 -- 172.21.15.45:0/3088049670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78601f72b0 con 0x7f7860153dd0 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f785fe00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7844004810 0x7f7860153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7857400640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 0x7f78600ad840 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f784801c230 tx=0x7f7844005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848007a40 con 0x7f7860153dd0 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78480ba070 con 0x7f7860153dd0 2024-04-25T10:26:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78601f7590 con 0x7f7860153dd0 2024-04-25T10:26:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848007270 con 0x7f7860153dd0 2024-04-25T10:26:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78601f7b50 con 0x7f7860153dd0 2024-04-25T10:26:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786004ef30 con 0x7f7860153dd0 2024-04-25T10:26:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.366+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78480052f0 con 0x7f7860153dd0 2024-04-25T10:26:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.370+0000 7f785cc00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f783c066f90 0x7f783c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.370+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f784811f400 con 0x7f7860153dd0 2024-04-25T10:26:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.370+0000 7f785fe00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f783c066f90 0x7f783c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:57.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.370+0000 7f785fe00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f783c066f90 0x7f783c069450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7850007d10 tx=0x7f7850009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:57.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.374+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7848016110 con 0x7f7860153dd0 2024-04-25T10:26:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.590+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f78600ae160 con 0x7f7860153dd0 2024-04-25T10:26:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.590+0000 7f785cc00640 1 -- 172.21.15.45:0/3088049670 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1773 (secure 0 0 0) 0x7f78480f04e0 con 0x7f7860153dd0 2024-04-25T10:26:57.595 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:57.595 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":561,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076617488,"bytes_used":2583937024,"bytes_avail":770510176256,"bytes_total":773094113280,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":1673678,"recovering_keys_per_sec":0,"num_objects_recovered":18,"num_bytes_recovered":18874368,"num_keys_recovered":0,"write_bytes_sec":25353971,"read_op_per_sec":0,"write_op_per_sec":26},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:26:54.998609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f783c066f90 msgr2=0x7f783c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f783c066f90 0x7f783c069450 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7850007d10 tx=0x7f7850009040 comp rx=0 tx=0).stop 2024-04-25T10:26:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 msgr2=0x7f78600ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 0x7f78600ad840 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f784801c230 tx=0x7f7844005840 comp rx=0 tx=0).stop 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 shutdown_connections 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f783c066f90 0x7f783c069450 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78600add80 0x7f786015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7860153dd0 0x7f78600ad840 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 --2- 172.21.15.45:0/3088049670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7844004810 0x7f7860153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 >> 172.21.15.45:0/3088049670 conn(0x7f7860066b70 msgr2=0x7f786006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 shutdown_connections 2024-04-25T10:26:57.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:57.594+0000 7f7866a00640 1 -- 172.21.15.45:0/3088049670 wait complete. 2024-04-25T10:26:57.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:26:58.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:26:58.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f809d800640 1 Processor -- start 2024-04-25T10:26:58.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f809d800640 1 -- start start 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f809d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80980cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f809d800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980ccfb0 con 0x7f80980d1c30 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f809d800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980cd290 con 0x7f809815bdc0 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f809d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80980cd570 con 0x7f809815b9e0 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f8097400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f80980d1c30 0x7f80980c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45064/0 (socket says 172.21.15.45:45064) 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f8097400640 1 -- 172.21.15.45:0/1627115033 learned_addr learned my addr 172.21.15.45:0/1627115033 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:26:58.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.766+0000 7f8096a00640 1 --2- 172.21.15.45:0/1627115033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80980cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:58.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096a00640 1 -- 172.21.15.45:0/1627115033 >> v1:172.21.15.117:6789/0 conn(0x7f809815bdc0 legacy=0x7f80980d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096a00640 1 -- 172.21.15.45:0/1627115033 >> v1:172.21.15.45:6789/0 conn(0x7f80980d1c30 legacy=0x7f80980c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096a00640 1 -- 172.21.15.45:0/1627115033 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980cd850 con 0x7f809815b9e0 2024-04-25T10:26:58.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096a00640 1 --2- 172.21.15.45:0/1627115033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80980cc410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80880097f0 tx=0x7f808801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e852651151a4006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:58.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096000640 1 -- 172.21.15.45:0/1627115033 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880b0070 con 0x7f809815b9e0 2024-04-25T10:26:58.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096000640 1 -- 172.21.15.45:0/1627115033 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80880ab600 con 0x7f809815b9e0 2024-04-25T10:26:58.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.770+0000 7f8096000640 1 -- 172.21.15.45:0/1627115033 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80880ab9d0 con 0x7f809815b9e0 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- 172.21.15.45:0/1627115033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 msgr2=0x7f80980cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 --2- 172.21.15.45:0/1627115033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80980cc410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80880097f0 tx=0x7f808801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- 172.21.15.45:0/1627115033 shutdown_connections 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 --2- 172.21.15.45:0/1627115033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80980cc410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- 172.21.15.45:0/1627115033 >> 172.21.15.45:0/1627115033 conn(0x7f809809a9d0 msgr2=0x7f809809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- 172.21.15.45:0/1627115033 shutdown_connections 2024-04-25T10:26:58.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- 172.21.15.45:0/1627115033 wait complete. 2024-04-25T10:26:58.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 Processor -- start 2024-04-25T10:26:58.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- start start 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80980d1c30 0x7f809814e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80981579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 0x7f8098157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80980ccd60 con 0x7f809815bdc0 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80980c6c00 con 0x7f80980d1c30 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f809d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8098169840 con 0x7f809815b9e0 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 0x7f8098157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 0x7f8098157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48918/0 (socket says 172.21.15.45:48918) 2024-04-25T10:26:58.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 -- 172.21.15.45:0/2678205194 learned_addr learned my addr 172.21.15.45:0/2678205194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8096a00640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80980d1c30 0x7f809814e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f808e000640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80981579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 -- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 msgr2=0x7f80981579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80981579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 -- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80980d1c30 msgr2=0x7f809814e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80980d1c30 0x7f809814e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.774+0000 7f8097400640 1 -- 172.21.15.45:0/2678205194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980b6560 con 0x7f809815bdc0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f8097400640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 0x7f8098157f30 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f809000cbc0 tx=0x7f809000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090017070 con 0x7f809815bdc0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000ad40 con 0x7f809815bdc0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090012600 con 0x7f809815bdc0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80980b6800 con 0x7f809815bdc0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80980b70a0 con 0x7f809815bdc0 2024-04-25T10:26:58.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80980b77f0 con 0x7f809815bdc0 2024-04-25T10:26:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f809001a070 con 0x7f809815bdc0 2024-04-25T10:26:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f808f400640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8074066f90 0x7f8074069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8090084d10 con 0x7f809815bdc0 2024-04-25T10:26:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.778+0000 7f8096a00640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8074066f90 0x7f8074069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.782+0000 7f8096a00640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8074066f90 0x7f8074069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f80880ad000 tx=0x7f8088008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:58.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.782+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f809008a020 con 0x7f809815bdc0 2024-04-25T10:26:58.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f809804ef30 con 0x7f809815bdc0 2024-04-25T10:26:58.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f808f400640 1 -- 172.21.15.45:0/2678205194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f80900555f0 con 0x7f809815bdc0 2024-04-25T10:26:58.986 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:58.986 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:26:58.986 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:26:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8074066f90 msgr2=0x7f8074069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8074066f90 0x7f8074069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f80880ad000 tx=0x7f8088008040 comp rx=0 tx=0).stop 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 msgr2=0x7f8098157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 0x7f8098157f30 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f809000cbc0 tx=0x7f809000a430 comp rx=0 tx=0).stop 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 shutdown_connections 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8074066f90 0x7f8074069450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809815bdc0 0x7f8098157f30 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f809815b9e0 0x7f80981579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 --2- 172.21.15.45:0/2678205194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80980d1c30 0x7f809814e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 >> 172.21.15.45:0/2678205194 conn(0x7f809809a9d0 msgr2=0x7f80980c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 shutdown_connections 2024-04-25T10:26:58.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:58.982+0000 7f809d800640 1 -- 172.21.15.45:0/2678205194 wait complete. 2024-04-25T10:26:59.007 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:26:59.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:26:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 Processor -- start 2024-04-25T10:26:59.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 -- start start 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1b80cfe70 0x7fc1b815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1b815b740 0x7fc1b806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b8071050 con 0x7fc1b815b740 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b80712f0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80715d0 con 0x7fc1b81550b0 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1b815b740 0x7fc1b806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1b815b740 0x7fc1b806d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48930/0 (socket says 172.21.15.45:48930) 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 -- 172.21.15.45:0/847960891 learned_addr learned my addr 172.21.15.45:0/847960891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:59.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bce00640 1 --2- 172.21.15.45:0/847960891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1b7e00640 1 --2- 172.21.15.45:0/847960891 >> v2:172.21.15.117:3300/0 conn(0x7fc1b80cfe70 0x7fc1b815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 -- 172.21.15.45:0/847960891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 msgr2=0x7fc1b80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 --2- 172.21.15.45:0/847960891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 -- 172.21.15.45:0/847960891 >> v2:172.21.15.117:3300/0 conn(0x7fc1b80cfe70 msgr2=0x7fc1b815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:26:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 --2- 172.21.15.45:0/847960891 >> v2:172.21.15.117:3300/0 conn(0x7fc1b80cfe70 0x7fc1b815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.146+0000 7fc1bd800640 1 -- 172.21.15.45:0/847960891 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80718b0 con 0x7fc1b815b740 2024-04-25T10:26:59.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bd800640 1 --2- 172.21.15.45:0/847960891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b815b740 0x7fc1b806d1a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc1b806e290 tx=0x7fc1b001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f389e3a1f561e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1b7400640 1 -- 172.21.15.45:0/847960891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00ac020 con 0x7fc1b815b740 2024-04-25T10:26:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1b7400640 1 -- 172.21.15.45:0/847960891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b00a6440 con 0x7fc1b815b740 2024-04-25T10:26:59.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1b7400640 1 -- 172.21.15.45:0/847960891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00a6810 con 0x7fc1b815b740 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 -- 172.21.15.45:0/847960891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b815b740 msgr2=0x7fc1b806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 --2- 172.21.15.45:0/847960891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b815b740 0x7fc1b806d1a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc1b806e290 tx=0x7fc1b001c730 comp rx=0 tx=0).stop 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 -- 172.21.15.45:0/847960891 shutdown_connections 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 --2- 172.21.15.45:0/847960891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b815b740 0x7fc1b806d1a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 --2- 172.21.15.45:0/847960891 >> v2:172.21.15.117:3300/0 conn(0x7fc1b80cfe70 0x7fc1b815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 --2- 172.21.15.45:0/847960891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 -- 172.21.15.45:0/847960891 >> 172.21.15.45:0/847960891 conn(0x7fc1b8066b70 msgr2=0x7fc1b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 -- 172.21.15.45:0/847960891 shutdown_connections 2024-04-25T10:26:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 -- 172.21.15.45:0/847960891 wait complete. 2024-04-25T10:26:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 Processor -- start 2024-04-25T10:26:59.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.150+0000 7fc1bf800640 1 -- start start 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 0x7fc1b814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b815b740 0x7fc1b814e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80b33e0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80b3260 con 0x7fc1b815b740 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b80b3560 con 0x7fc1b81550b0 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 0x7fc1b814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 0x7fc1b814f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48942/0 (socket says 172.21.15.45:48942) 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 -- 172.21.15.45:0/3118993552 learned_addr learned my addr 172.21.15.45:0/3118993552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1b7e00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bd800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b815b740 0x7fc1b814e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 -- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 msgr2=0x7fc1b814dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b814dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 -- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b815b740 msgr2=0x7fc1b814e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b815b740 0x7fc1b814e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 -- 172.21.15.45:0/3118993552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b81513e0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bd800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b815b740 0x7fc1b814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bce00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 0x7fc1b814f1e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00d830 tx=0x7fc1ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00b6c0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac019070 con 0x7fc1b80cfe70 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b80ce580 con 0x7fc1b80cfe70 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00ba60 con 0x7fc1b80cfe70 2024-04-25T10:26:59.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b8146f40 con 0x7fc1b80cfe70 2024-04-25T10:26:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.154+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b804ef30 con 0x7fc1b80cfe70 2024-04-25T10:26:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.158+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1ac00bc00 con 0x7fc1b80cfe70 2024-04-25T10:26:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.158+0000 7fc1b4c00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc198066f60 0x7fc198069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:26:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.158+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc1ac0854a0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.158+0000 7fc1b7e00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc198066f60 0x7fc198069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:26:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.158+0000 7fc1b7e00640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc198066f60 0x7fc198069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc1b815a1a0 tx=0x7fc1a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:26:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.162+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1ac0563d0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc1b80ceaa0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1b4c00640 1 -- 172.21.15.45:0/3118993552 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1596 (secure 0 0 0) 0x7fc1ac05c0d0 con 0x7fc1b80cfe70 2024-04-25T10:26:59.390 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:26:59.390 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":563,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076617488,"bytes_used":2583937024,"bytes_avail":770510176256,"bytes_total":773094113280,"write_bytes_sec":4423854,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:26:54.998609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:26:59.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc198066f60 msgr2=0x7fc198069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:59.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc198066f60 0x7fc198069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc1b815a1a0 tx=0x7fc1a4008040 comp rx=0 tx=0).stop 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 msgr2=0x7fc1b814f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 0x7fc1b814f1e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00d830 tx=0x7fc1ac00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 shutdown_connections 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc198066f60 0x7fc198069420 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b815b740 0x7fc1b814e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b81550b0 0x7fc1b814dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 --2- 172.21.15.45:0/3118993552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b80cfe70 0x7fc1b814f1e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 >> 172.21.15.45:0/3118993552 conn(0x7fc1b8066b70 msgr2=0x7fc1b80b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.386+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 shutdown_connections 2024-04-25T10:26:59.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:26:59.390+0000 7fc1bf800640 1 -- 172.21.15.45:0/3118993552 wait complete. 2024-04-25T10:26:59.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:26:59.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:26:59.636 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:00.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 Processor -- start 2024-04-25T10:27:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 -- start start 2024-04-25T10:27:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fba3006eec0 0x7fba3006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba300ad850 0x7fba300adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3015e0c0 con 0x7fba3015dce0 2024-04-25T10:27:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300674d0 con 0x7fba300ad850 2024-04-25T10:27:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba34e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300671f0 con 0x7fba3006eec0 2024-04-25T10:27:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fba300ad850 0x7fba300adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2de00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fba3015dce0 0x7fba300669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45068/0 (socket says 172.21.15.45:45068) 2024-04-25T10:27:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2de00640 1 -- 172.21.15.45:0/3320208725 learned_addr learned my addr 172.21.15.45:0/3320208725 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2d400640 1 --2- 172.21.15.45:0/3320208725 >> v2:172.21.15.117:3300/0 conn(0x7fba3006eec0 0x7fba3006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2e800640 1 -- 172.21.15.45:0/3320208725 >> v2:172.21.15.117:3300/0 conn(0x7fba3006eec0 msgr2=0x7fba3006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2ca00640 1 -- 172.21.15.45:0/3320208725 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251553912 0 0) 0x7fba3015e0c0 con 0x7fba3015dce0 2024-04-25T10:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2e800640 1 --2- 172.21.15.45:0/3320208725 >> v2:172.21.15.117:3300/0 conn(0x7fba3006eec0 0x7fba3006df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2e800640 1 -- 172.21.15.45:0/3320208725 >> v1:172.21.15.45:6789/0 conn(0x7fba3015dce0 legacy=0x7fba300669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.562+0000 7fba2e800640 1 -- 172.21.15.45:0/3320208725 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3006d4b0 con 0x7fba300ad850 2024-04-25T10:27:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba2e800640 1 --2- 172.21.15.45:0/3320208725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba300adcb0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fba300d0850 tx=0x7fba2801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ade53e8296fa38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba2ca00640 1 -- 172.21.15.45:0/3320208725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba280b2070 con 0x7fba300ad850 2024-04-25T10:27:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba2ca00640 1 -- 172.21.15.45:0/3320208725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba280a57a0 con 0x7fba300ad850 2024-04-25T10:27:00.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba2ca00640 1 -- 172.21.15.45:0/3320208725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba280a5b70 con 0x7fba300ad850 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 -- 172.21.15.45:0/3320208725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 msgr2=0x7fba300adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 --2- 172.21.15.45:0/3320208725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba300adcb0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fba300d0850 tx=0x7fba2801c810 comp rx=0 tx=0).stop 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 -- 172.21.15.45:0/3320208725 shutdown_connections 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 --2- 172.21.15.45:0/3320208725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba300adcb0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 --2- 172.21.15.45:0/3320208725 >> v2:172.21.15.117:3300/0 conn(0x7fba3006eec0 0x7fba3006df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 -- 172.21.15.45:0/3320208725 >> 172.21.15.45:0/3320208725 conn(0x7fba300b2410 msgr2=0x7fba300b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 -- 172.21.15.45:0/3320208725 shutdown_connections 2024-04-25T10:27:00.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 -- 172.21.15.45:0/3320208725 wait complete. 2024-04-25T10:27:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 Processor -- start 2024-04-25T10:27:00.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.566+0000 7fba34e00640 1 -- start start 2024-04-25T10:27:00.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba3006eec0 0x7fba301508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba301580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba3015dce0 0x7fba30158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300d0320 con 0x7fba300ad850 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300d04a0 con 0x7fba3015dce0 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3016abc0 con 0x7fba3006eec0 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba301580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba301580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48974/0 (socket says 172.21.15.45:48974) 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 -- 172.21.15.45:0/2114189481 learned_addr learned my addr 172.21.15.45:0/2114189481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2e800640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba3015dce0 0x7fba30158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2de00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba3006eec0 0x7fba301508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 -- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba3006eec0 msgr2=0x7fba301508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba3006eec0 0x7fba301508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 -- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba3015dce0 msgr2=0x7fba30158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba3015dce0 0x7fba30158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 -- 172.21.15.45:0/2114189481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30152b20 con 0x7fba300ad850 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2e800640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba3015dce0 0x7fba30158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba2d400640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba301580d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00d5f0 tx=0x7fba1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0125f0 con 0x7fba300ad850 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c019070 con 0x7fba300ad850 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0129b0 con 0x7fba300ad850 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba30152e00 con 0x7fba300ad850 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba301533c0 con 0x7fba300ad850 2024-04-25T10:27:00.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.570+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3004ef30 con 0x7fba300ad850 2024-04-25T10:27:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.574+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba1c012b50 con 0x7fba300ad850 2024-04-25T10:27:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.574+0000 7fba26000640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba10066f60 0x7fba10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.574+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fba1c0854c0 con 0x7fba300ad850 2024-04-25T10:27:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.574+0000 7fba2de00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba10066f60 0x7fba10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.574+0000 7fba2de00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba10066f60 0x7fba10069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fba20000f70 tx=0x7fba20007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.578+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba1c056480 con 0x7fba300ad850 2024-04-25T10:27:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.770+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba3006eca0 con 0x7fba300ad850 2024-04-25T10:27:00.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.770+0000 7fba26000640 1 -- 172.21.15.45:0/2114189481 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fba1c055d80 con 0x7fba300ad850 2024-04-25T10:27:00.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:00.776 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:27:00.776 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba10066f60 msgr2=0x7fba10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba10066f60 0x7fba10069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fba20000f70 tx=0x7fba20007c10 comp rx=0 tx=0).stop 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 msgr2=0x7fba301580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba301580d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00d5f0 tx=0x7fba1c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 shutdown_connections 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba10066f60 0x7fba10069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba3015dce0 0x7fba30158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba300ad850 0x7fba301580d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 --2- 172.21.15.45:0/2114189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba3006eec0 0x7fba301508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 >> 172.21.15.45:0/2114189481 conn(0x7fba300b2410 msgr2=0x7fba30071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 shutdown_connections 2024-04-25T10:27:00.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.774+0000 7fba34e00640 1 -- 172.21.15.45:0/2114189481 wait complete. 2024-04-25T10:27:00.797 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:27:00.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:27:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 Processor -- start 2024-04-25T10:27:00.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 -- start start 2024-04-25T10:27:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 0x7f491c0d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c0d0b20 con 0x7f491c0cdb40 2024-04-25T10:27:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491c15e340 con 0x7f491c0d0180 2024-04-25T10:27:00.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4923e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0ccaa0 con 0x7f491c15df60 2024-04-25T10:27:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4920a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 0x7f491c0d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4921400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4920a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 0x7f491c0d05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37568/0 (socket says 172.21.15.45:37568) 2024-04-25T10:27:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4920a00640 1 -- 172.21.15.45:0/2240861261 learned_addr learned my addr 172.21.15.45:0/2240861261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:00.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.934+0000 7f4921e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f491c0cdb40 0x7f491c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:54102/0 (socket says 172.21.15.45:54102) 2024-04-25T10:27:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f4921400640 1 -- 172.21.15.45:0/2240861261 >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 msgr2=0x7f491c0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f4921400640 1 --2- 172.21.15.45:0/2240861261 >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 0x7f491c0d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f4921400640 1 -- 172.21.15.45:0/2240861261 >> v1:172.21.15.117:6789/0 conn(0x7f491c0cdb40 legacy=0x7f491c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f491be00640 1 -- 172.21.15.45:0/2240861261 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135329219 0 0) 0x7f491c0d0b20 con 0x7f491c0cdb40 2024-04-25T10:27:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f4921400640 1 -- 172.21.15.45:0/2240861261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0ccd80 con 0x7f491c15df60 2024-04-25T10:27:00.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f4920a00640 1 --2- 172.21.15.45:0/2240861261 >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 0x7f491c0d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f4921400640 1 --2- 172.21.15.45:0/2240861261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c0cfc40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49100097c0 tx=0x7f491001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e5546dc280128e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:00.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f491be00640 1 -- 172.21.15.45:0/2240861261 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100b0070 con 0x7f491c15df60 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f491be00640 1 -- 172.21.15.45:0/2240861261 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49100ab600 con 0x7f491c15df60 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.938+0000 7f491be00640 1 -- 172.21.15.45:0/2240861261 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100ab9d0 con 0x7f491c15df60 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- 172.21.15.45:0/2240861261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 msgr2=0x7f491c0cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 --2- 172.21.15.45:0/2240861261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c0cfc40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49100097c0 tx=0x7f491001c410 comp rx=0 tx=0).stop 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- 172.21.15.45:0/2240861261 shutdown_connections 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 --2- 172.21.15.45:0/2240861261 >> v2:172.21.15.117:3300/0 conn(0x7f491c0d0180 0x7f491c0d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 --2- 172.21.15.45:0/2240861261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c0cfc40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- 172.21.15.45:0/2240861261 >> 172.21.15.45:0/2240861261 conn(0x7f491c0d1910 msgr2=0x7f491c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- 172.21.15.45:0/2240861261 shutdown_connections 2024-04-25T10:27:00.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- 172.21.15.45:0/2240861261 wait complete. 2024-04-25T10:27:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 Processor -- start 2024-04-25T10:27:00.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- start start 2024-04-25T10:27:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 0x7f491c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f491c0d0180 0x7f491c1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0d0ff0 con 0x7f491c0cdb40 2024-04-25T10:27:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0d0e70 con 0x7f491c0d0180 2024-04-25T10:27:00.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4923e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0cead0 con 0x7f491c15df60 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 0x7f491c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 0x7f491c06f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48978/0 (socket says 172.21.15.45:48978) 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 -- 172.21.15.45:0/3390106844 learned_addr learned my addr 172.21.15.45:0/3390106844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4920a00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f491c0d0180 0x7f491c1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 -- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 msgr2=0x7f491c155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 -- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f491c0d0180 msgr2=0x7f491c1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f491c0d0180 0x7f491c1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 -- 172.21.15.45:0/3390106844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0cb7d0 con 0x7f491c0cdb40 2024-04-25T10:27:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.942+0000 7f4921400640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 0x7f491c06f1d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f491001a040 tx=0x7f491001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100b0050 con 0x7f491c0cdb40 2024-04-25T10:27:00.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49100bc070 con 0x7f491c0cdb40 2024-04-25T10:27:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49100b5830 con 0x7f491c0cdb40 2024-04-25T10:27:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c15fab0 con 0x7f491c0cdb40 2024-04-25T10:27:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c160070 con 0x7f491c0cdb40 2024-04-25T10:27:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c04ef30 con 0x7f491c0cdb40 2024-04-25T10:27:00.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49100b59d0 con 0x7f491c0cdb40 2024-04-25T10:27:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4919600640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48fc066f60 0x7f48fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f491011f7d0 con 0x7f491c0cdb40 2024-04-25T10:27:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4920a00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48fc066f60 0x7f48fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:00.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.946+0000 7f4920a00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48fc066f60 0x7f48fc069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f491c15f690 tx=0x7f4908008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:00.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:00.950+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4910016090 con 0x7f491c0cdb40 2024-04-25T10:27:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.174+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f491c15aaf0 con 0x7f491c0cdb40 2024-04-25T10:27:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.174+0000 7f4919600640 1 -- 172.21.15.45:0/3390106844 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1596 (secure 0 0 0) 0x7f49100f0700 con 0x7f491c0cdb40 2024-04-25T10:27:01.179 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:01.179 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":565,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076621713,"bytes_used":2583945216,"bytes_avail":770510168064,"bytes_total":773094113280,"write_bytes_sec":4424320,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:26:54.998609+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:27:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48fc066f60 msgr2=0x7f48fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48fc066f60 0x7f48fc069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f491c15f690 tx=0x7f4908008040 comp rx=0 tx=0).stop 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 msgr2=0x7f491c06f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 0x7f491c06f1d0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f491001a040 tx=0x7f491001ac50 comp rx=0 tx=0).stop 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 shutdown_connections 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f48fc066f60 0x7f48fc069420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f491c15df60 0x7f491c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f491c0d0180 0x7f491c1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 --2- 172.21.15.45:0/3390106844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f491c0cdb40 0x7f491c06f1d0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 >> 172.21.15.45:0/3390106844 conn(0x7f491c0d1910 msgr2=0x7f491c0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 shutdown_connections 2024-04-25T10:27:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:01.178+0000 7f4923e00640 1 -- 172.21.15.45:0/3390106844 wait complete. 2024-04-25T10:27:01.200 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T10:27:02.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ede000640 1 Processor -- start 2024-04-25T10:27:02.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ede000640 1 -- start start 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ede000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0ed81550b0 0x7f0ed80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ede000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed8071050 con 0x7f0ed80cfe70 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ede000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed815d040 con 0x7f0ed815b910 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ede000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed815bd80 con 0x7f0ed81550b0 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ecea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ed80cfe70 0x7f0ed815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45080/0 (socket says 172.21.15.45:45080) 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ecea00640 1 -- 172.21.15.45:0/629835962 learned_addr learned my addr 172.21.15.45:0/629835962 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ed7400640 1 --2- 172.21.15.45:0/629835962 >> v2:172.21.15.117:3300/0 conn(0x7f0ed81550b0 0x7f0ed80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.350+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255354128 0 0) 0x7f0ed8071050 con 0x7f0ed80cfe70 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc005740 con 0x7f0ed80cfe70 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223025406 0 0) 0x7f0ebc005740 con 0x7f0ed80cfe70 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 >> v2:172.21.15.117:3300/0 conn(0x7f0ed81550b0 msgr2=0x7f0ed80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 --2- 172.21.15.45:0/629835962 >> v2:172.21.15.117:3300/0 conn(0x7f0ed81550b0 0x7f0ed80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 >> v1:172.21.15.117:6789/0 conn(0x7f0ed815b910 legacy=0x7f0ed80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80712f0 con 0x7f0ed80cfe70 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f0ec0001680 con 0x7f0ed80cfe70 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7400640 1 --2- 172.21.15.45:0/629835962 >> v2:172.21.15.117:3300/0 conn(0x7f0ed81550b0 0x7f0ed80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 >> v1:172.21.15.45:6789/0 conn(0x7f0ed80cfe70 legacy=0x7f0ed815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ebc005bd0 0x7f0ebc005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ebc006a20 0x7f0ebc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ebc00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed815bd80 con 0x7f0ebc0073e0 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc00e280 con 0x7f0ebc005bd0 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc00e480 con 0x7f0ebc006a20 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ebc00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7400640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ebc005bd0 0x7f0ebc005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ecea00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ebc006a20 0x7f0ebc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 -- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ebc006a20 msgr2=0x7f0ebc006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ebc006a20 0x7f0ebc006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 -- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ebc005bd0 msgr2=0x7f0ebc005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ebc005bd0 0x7f0ebc005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 -- 172.21.15.45:0/629835962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc005740 con 0x7f0ebc0073e0 2024-04-25T10:27:02.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.354+0000 7f0ed7e00640 1 -- 172.21.15.45:0/629835962 reap_dead start 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed7e00640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ebc00dbe0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0001e70 tx=0x7f0ed001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485274fc99b45745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0001a90 con 0x7f0ebc0073e0 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0003ba0 con 0x7f0ebc0073e0 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed6a00640 1 -- 172.21.15.45:0/629835962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0005020 con 0x7f0ebc0073e0 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- 172.21.15.45:0/629835962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 msgr2=0x7f0ebc00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ebc00dbe0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0001e70 tx=0x7f0ed001c780 comp rx=0 tx=0).stop 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- 172.21.15.45:0/629835962 shutdown_connections 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 --2- 172.21.15.45:0/629835962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ebc00dbe0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- 172.21.15.45:0/629835962 >> 172.21.15.45:0/629835962 conn(0x7f0ed8066b70 msgr2=0x7f0ed8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- 172.21.15.45:0/629835962 shutdown_connections 2024-04-25T10:27:02.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- 172.21.15.45:0/629835962 wait complete. 2024-04-25T10:27:02.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 Processor -- start 2024-04-25T10:27:02.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- start start 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ed8149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed81e5580 0x7f0ed80ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ed81540c0 0x7f0ed80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8071b90 con 0x7f0ebc0073e0 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80718b0 con 0x7f0ed81e5580 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ede000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80720b0 con 0x7f0ed81540c0 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ed8149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ed8149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48996/0 (socket says 172.21.15.45:48996) 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.358+0000 7f0ed7400640 1 -- 172.21.15.45:0/1519970572 learned_addr learned my addr 172.21.15.45:0/1519970572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7e00640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ed81540c0 0x7f0ed80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecea00640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed81e5580 0x7f0ed80ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7400640 1 -- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ed81540c0 msgr2=0x7f0ed80ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7400640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ed81540c0 0x7f0ed80ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7400640 1 -- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed81e5580 msgr2=0x7f0ed80ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7400640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed81e5580 0x7f0ed80ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7400640 1 -- 172.21.15.45:0/1519970572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8155530 con 0x7f0ebc0073e0 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecea00640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed81e5580 0x7f0ed80ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ed7400640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ed8149140 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8011a00 tx=0x7f0ec8011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800fb90 con 0x7f0ebc0073e0 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec800fd30 con 0x7f0ebc0073e0 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004260 con 0x7f0ebc0073e0 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed81e5ad0 con 0x7f0ebc0073e0 2024-04-25T10:27:02.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81e6090 con 0x7f0ebc0073e0 2024-04-25T10:27:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed804ef30 con 0x7f0ebc0073e0 2024-04-25T10:27:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ec8012040 con 0x7f0ebc0073e0 2024-04-25T10:27:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecfe00640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0eb4066f60 0x7f0eb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.362+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0ec80853c0 con 0x7f0ebc0073e0 2024-04-25T10:27:02.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.366+0000 7f0ecea00640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0eb4066f60 0x7f0eb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.366+0000 7f0ecea00640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0eb4066f60 0x7f0eb4069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0ed806d350 tx=0x7f0ec0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:02.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.366+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ec8019030 con 0x7f0ebc0073e0 2024-04-25T10:27:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.566+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed806d380 con 0x7f0ebc0073e0 2024-04-25T10:27:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.566+0000 7f0ecfe00640 1 -- 172.21.15.45:0/1519970572 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f0ec8055d30 con 0x7f0ebc0073e0 2024-04-25T10:27:02.569 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:02.570 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:27:02.570 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.566+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0eb4066f60 msgr2=0x7f0eb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.566+0000 7f0ede000640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0eb4066f60 0x7f0eb4069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0ed806d350 tx=0x7f0ec0009040 comp rx=0 tx=0).stop 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.566+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 msgr2=0x7f0ed8149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.566+0000 7f0ede000640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ed8149140 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8011a00 tx=0x7f0ec8011ec0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 shutdown_connections 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0eb4066f60 0x7f0eb4069420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ed81540c0 0x7f0ed80ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ed81e5580 0x7f0ed80ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 --2- 172.21.15.45:0/1519970572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ebc0073e0 0x7f0ed8149140 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 >> 172.21.15.45:0/1519970572 conn(0x7f0ed8066b70 msgr2=0x7f0ed806d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 shutdown_connections 2024-04-25T10:27:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.570+0000 7f0ede000640 1 -- 172.21.15.45:0/1519970572 wait complete. 2024-04-25T10:27:02.590 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:27:02.590 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:27:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.726+0000 7f41bb600640 1 Processor -- start 2024-04-25T10:27:02.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.726+0000 7f41bb600640 1 -- start start 2024-04-25T10:27:02.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.726+0000 7f41bb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b40d0040 0x7f41b406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.726+0000 7f41bb600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b40b2770 con 0x7f41b415bc40 2024-04-25T10:27:02.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.726+0000 7f41bb600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b406c670 con 0x7f41b40cda10 2024-04-25T10:27:02.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.726+0000 7f41bb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b406c950 con 0x7f41b40d0040 2024-04-25T10:27:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b8c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f41b415bc40 0x7f41b40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45094/0 (socket says 172.21.15.45:45094) 2024-04-25T10:27:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b8c00640 1 -- 172.21.15.45:0/2326009453 learned_addr learned my addr 172.21.15.45:0/2326009453 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b9600640 1 --2- 172.21.15.45:0/2326009453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b40d0040 0x7f41b406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312138336 0 0) 0x7f41b40b2770 con 0x7f41b415bc40 2024-04-25T10:27:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f419c003530 con 0x7f41b415bc40 2024-04-25T10:27:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537583301 0 0) 0x7f41b406c670 con 0x7f41b40cda10 2024-04-25T10:27:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b40b2770 con 0x7f41b40cda10 2024-04-25T10:27:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b9600640 1 -- 172.21.15.45:0/2326009453 >> v1:172.21.15.117:6789/0 conn(0x7f41b40cda10 legacy=0x7f41b40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b9600640 1 -- 172.21.15.45:0/2326009453 >> v1:172.21.15.45:6789/0 conn(0x7f41b415bc40 legacy=0x7f41b40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b9600640 1 -- 172.21.15.45:0/2326009453 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b406cc30 con 0x7f41b40d0040 2024-04-25T10:27:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b9600640 1 --2- 172.21.15.45:0/2326009453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b40d0040 0x7f41b406c090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009dc0 tx=0x7f41ac01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7097323feaa2178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.730+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0b2070 con 0x7f41b40d0040 2024-04-25T10:27:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac0a58d0 con 0x7f41b40d0040 2024-04-25T10:27:02.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b3400640 1 -- 172.21.15.45:0/2326009453 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a5c80 con 0x7f41b40d0040 2024-04-25T10:27:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- 172.21.15.45:0/2326009453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b40d0040 msgr2=0x7f41b406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 --2- 172.21.15.45:0/2326009453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b40d0040 0x7f41b406c090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009dc0 tx=0x7f41ac01c900 comp rx=0 tx=0).stop 2024-04-25T10:27:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- 172.21.15.45:0/2326009453 shutdown_connections 2024-04-25T10:27:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 --2- 172.21.15.45:0/2326009453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b40d0040 0x7f41b406c090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- 172.21.15.45:0/2326009453 >> 172.21.15.45:0/2326009453 conn(0x7f41b4066b70 msgr2=0x7f41b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- 172.21.15.45:0/2326009453 shutdown_connections 2024-04-25T10:27:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- 172.21.15.45:0/2326009453 wait complete. 2024-04-25T10:27:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 Processor -- start 2024-04-25T10:27:02.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- start start 2024-04-25T10:27:02.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 0x7f41b4152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b40d0040 0x7f41b415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b415bc40 0x7f41b41f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b4169980 con 0x7f41b40cda10 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b4169800 con 0x7f41b40d0040 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41bb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b4169b00 con 0x7f41b415bc40 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 0x7f41b4152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 0x7f41b4152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49002/0 (socket says 172.21.15.45:49002) 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b8c00640 1 -- 172.21.15.45:0/1302888583 learned_addr learned my addr 172.21.15.45:0/1302888583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b3e00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b40d0040 0x7f41b415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b9600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b415bc40 0x7f41b41f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.734+0000 7f41b8c00640 1 -- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b415bc40 msgr2=0x7f41b41f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b8c00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b415bc40 0x7f41b41f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b8c00640 1 -- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b40d0040 msgr2=0x7f41b415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b8c00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b40d0040 0x7f41b415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b8c00640 1 -- 172.21.15.45:0/1302888583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b40704a0 con 0x7f41b40cda10 2024-04-25T10:27:02.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b3e00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b40d0040 0x7f41b415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:02.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b9600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b415bc40 0x7f41b41f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:02.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b8c00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 0x7f41b4152790 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c4d0 tx=0x7f41a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:02.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8017070 con 0x7f41b40cda10 2024-04-25T10:27:02.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a8005cf0 con 0x7f41b40cda10 2024-04-25T10:27:02.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b4070760 con 0x7f41b40cda10 2024-04-25T10:27:02.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80072b0 con 0x7f41b40cda10 2024-04-25T10:27:02.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b4070d20 con 0x7f41b40cda10 2024-04-25T10:27:02.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b404ef30 con 0x7f41b40cda10 2024-04-25T10:27:02.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41a801a070 con 0x7f41b40cda10 2024-04-25T10:27:02.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b0c00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4194066f60 0x7f4194069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:02.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f41a80844a0 con 0x7f41b40cda10 2024-04-25T10:27:02.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.738+0000 7f41b3e00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4194066f60 0x7f4194069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:02.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.742+0000 7f41b3e00640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4194066f60 0x7f4194069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f41b406c790 tx=0x7f41a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:02.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.742+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41a8055460 con 0x7f41b40cda10 2024-04-25T10:27:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f41b40aed60 con 0x7f41b40cda10 2024-04-25T10:27:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41b0c00640 1 -- 172.21.15.45:0/1302888583 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1570 (secure 0 0 0) 0x7f41a8054d60 con 0x7f41b40cda10 2024-04-25T10:27:02.966 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:02.966 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":567,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076621713,"bytes_used":2583945216,"bytes_avail":770510168064,"bytes_total":773094113280,"write_bytes_sec":6739,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8373}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:27:01.000137+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:27:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4194066f60 msgr2=0x7f4194069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4194066f60 0x7f4194069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f41b406c790 tx=0x7f41a0009040 comp rx=0 tx=0).stop 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 msgr2=0x7f41b4152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 0x7f41b4152790 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c4d0 tx=0x7f41a800c990 comp rx=0 tx=0).stop 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 shutdown_connections 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4194066f60 0x7f4194069420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41b415bc40 0x7f41b41f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41b40d0040 0x7f41b415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 --2- 172.21.15.45:0/1302888583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41b40cda10 0x7f41b4152790 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 >> 172.21.15.45:0/1302888583 conn(0x7f41b4066b70 msgr2=0x7f41b40af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.962+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 shutdown_connections 2024-04-25T10:27:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:02.966+0000 7f41bb600640 1 -- 172.21.15.45:0/1302888583 wait complete. 2024-04-25T10:27:02.986 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:27:02.987 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:27:02.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:27:03.098 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:27:03.190 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:27:03.190 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:27:03.190 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:03.200 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:27:03.200 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:03.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffac5000640 1 Processor -- start 2024-04-25T10:27:03.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffac5000640 1 -- start start 2024-04-25T10:27:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffac5000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffac00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffac5000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac0071050 con 0x7ffac015b910 2024-04-25T10:27:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffac5000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac015d040 con 0x7ffac00cfe70 2024-04-25T10:27:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffac5000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac015bda0 con 0x7ffac01550b0 2024-04-25T10:27:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabe800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffac015b910 0x7ffac00b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45100/0 (socket says 172.21.15.45:45100) 2024-04-25T10:27:03.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabe800640 1 -- 172.21.15.45:0/570367151 learned_addr learned my addr 172.21.15.45:0/570367151 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffac00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939981573 0 0) 0x7ffac0071050 con 0x7ffac015b910 2024-04-25T10:27:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa4003530 con 0x7ffac015b910 2024-04-25T10:27:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822813107 0 0) 0x7ffaa4003530 con 0x7ffac015b910 2024-04-25T10:27:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 msgr2=0x7ffac00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffac00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 >> v1:172.21.15.117:6789/0 conn(0x7ffac00cfe70 legacy=0x7ffac015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00712f0 con 0x7ffac015b910 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7ffab8001a00 con 0x7ffac015b910 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 >> v1:172.21.15.45:6789/0 conn(0x7ffac015b910 legacy=0x7ffac00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaa40039f0 0x7ffaa4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffaa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa4004d50 0x7ffaa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa400ba80 con 0x7ffaa40039f0 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa400bc30 con 0x7ffaa4004d50 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa400be30 con 0x7ffac01550b0 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaa40039f0 0x7ffaa4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabd400640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffaa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabe800640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa4004d50 0x7ffaa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 -- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 msgr2=0x7ffaa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffaa4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 -- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa4004d50 msgr2=0x7ffaa40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa4004d50 0x7ffaa40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabde00640 1 -- 172.21.15.45:0/570367151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4003530 con 0x7ffaa40039f0 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.438+0000 7ffabd400640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac01550b0 0x7ffaa4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffabde00640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaa40039f0 0x7ffaa4003e10 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009ab0 tx=0x7ffab001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c7abb8ec9aabe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:03.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00096b0 con 0x7ffaa40039f0 2024-04-25T10:27:03.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab00a9430 con 0x7ffaa40039f0 2024-04-25T10:27:03.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffabca00640 1 -- 172.21.15.45:0/570367151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00a9820 con 0x7ffaa40039f0 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffac5000640 1 -- 172.21.15.45:0/570367151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaa40039f0 msgr2=0x7ffaa4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffac5000640 1 --2- 172.21.15.45:0/570367151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaa40039f0 0x7ffaa4003e10 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009ab0 tx=0x7ffab001ca50 comp rx=0 tx=0).stop 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffabe800640 1 -- 172.21.15.45:0/570367151 reap_dead start 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffac5000640 1 -- 172.21.15.45:0/570367151 shutdown_connections 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffac5000640 1 -- 172.21.15.45:0/570367151 >> 172.21.15.45:0/570367151 conn(0x7ffac0066b70 msgr2=0x7ffac0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffac5000640 1 -- 172.21.15.45:0/570367151 shutdown_connections 2024-04-25T10:27:03.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.442+0000 7ffac5000640 1 -- 172.21.15.45:0/570367151 wait complete. 2024-04-25T10:27:03.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 Processor -- start 2024-04-25T10:27:03.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 -- start start 2024-04-25T10:27:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa40039f0 0x7ffac01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac015e3f0 0x7ffac0153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 0x7ffac00b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac01664b0 con 0x7ffac0157ec0 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00720b0 con 0x7ffaa40039f0 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0166630 con 0x7ffac015e3f0 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 0x7ffac00b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 0x7ffac00b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49012/0 (socket says 172.21.15.45:49012) 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 -- 172.21.15.45:0/2411176519 learned_addr learned my addr 172.21.15.45:0/2411176519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabd400640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac015e3f0 0x7ffac0153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabde00640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa40039f0 0x7ffac01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 -- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac015e3f0 msgr2=0x7ffac0153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac015e3f0 0x7ffac0153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 -- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa40039f0 msgr2=0x7ffac01565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa40039f0 0x7ffac01565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 -- 172.21.15.45:0/2411176519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac015e960 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffabe800640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 0x7ffac00b03f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ffac006e380 tx=0x7ffab8007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800ead0 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab800ec90 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800ee80 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac015ec40 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.446+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac0158340 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.450+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac004ef30 con 0x7ffac0157ec0 2024-04-25T10:27:03.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.450+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffab801a020 con 0x7ffac0157ec0 2024-04-25T10:27:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.450+0000 7ffab6000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffaa0066f60 0x7ffaa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.450+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ffab8084a60 con 0x7ffac0157ec0 2024-04-25T10:27:03.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.450+0000 7ffabde00640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffaa0066f60 0x7ffaa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.450+0000 7ffabde00640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffaa0066f60 0x7ffaa0069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffab00ae000 tx=0x7ffab00b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:03.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.454+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffab8021020 con 0x7ffac0157ec0 2024-04-25T10:27:03.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.614+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffac0155f10 con 0x7ffac0157ec0 2024-04-25T10:27:03.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffab6000640 1 -- 172.21.15.45:0/2411176519 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16769 (secure 0 0 0) 0x7ffab8055340 con 0x7ffac0157ec0 2024-04-25T10:27:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:03.620 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:43.714023+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.21","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffaa0066f60 msgr2=0x7ffaa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffaa0066f60 0x7ffaa0069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ffab00ae000 tx=0x7ffab00b3040 comp rx=0 tx=0).stop 2024-04-25T10:27:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 msgr2=0x7ffac00b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 0x7ffac00b03f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ffac006e380 tx=0x7ffab8007bc0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 shutdown_connections 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffaa0066f60 0x7ffaa0069420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffac0157ec0 0x7ffac00b03f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffac015e3f0 0x7ffac0153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 --2- 172.21.15.45:0/2411176519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffaa40039f0 0x7ffac01565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 >> 172.21.15.45:0/2411176519 conn(0x7ffac0066b70 msgr2=0x7ffac00b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 shutdown_connections 2024-04-25T10:27:03.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.618+0000 7ffac5000640 1 -- 172.21.15.45:0/2411176519 wait complete. 2024-04-25T10:27:03.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:26:11.768328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:26:12.345255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:27:03.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:27:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.782+0000 7fe3f0000640 1 Processor -- start 2024-04-25T10:27:03.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.782+0000 7fe3f0000640 1 -- start start 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3f0000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3e815bb70 0x7fe3e806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3f0000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3e8072220 0x7fe3e806ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3f0000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e806f250 con 0x7fe3e8071e40 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3f0000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e815f720 con 0x7fe3e8072220 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3f0000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e815f9c0 con 0x7fe3e815bb70 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3e8072220 0x7fe3e806ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3e8072220 0x7fe3e806ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49024/0 (socket says 172.21.15.45:49024) 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 -- 172.21.15.45:0/2057395740 learned_addr learned my addr 172.21.15.45:0/2057395740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:03.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ecc00640 1 --2- 172.21.15.45:0/2057395740 >> v2:172.21.15.117:3300/0 conn(0x7fe3e815bb70 0x7fe3e806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 -- 172.21.15.45:0/2057395740 >> v2:172.21.15.117:3300/0 conn(0x7fe3e815bb70 msgr2=0x7fe3e806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:03.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 --2- 172.21.15.45:0/2057395740 >> v2:172.21.15.117:3300/0 conn(0x7fe3e815bb70 0x7fe3e806e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 -- 172.21.15.45:0/2057395740 >> v1:172.21.15.117:6789/0 conn(0x7fe3e8071e40 legacy=0x7fe3e815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 -- 172.21.15.45:0/2057395740 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e815fca0 con 0x7fe3e8072220 2024-04-25T10:27:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.786+0000 7fe3ee000640 1 --2- 172.21.15.45:0/2057395740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e806ed10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80ce640 tx=0x7fe3e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e5a06ef779821f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3e7e00640 1 -- 172.21.15.45:0/2057395740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00a57f0 con 0x7fe3e8072220 2024-04-25T10:27:03.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3e7e00640 1 -- 172.21.15.45:0/2057395740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e00a5990 con 0x7fe3e8072220 2024-04-25T10:27:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3e7e00640 1 -- 172.21.15.45:0/2057395740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00a5d60 con 0x7fe3e8072220 2024-04-25T10:27:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- 172.21.15.45:0/2057395740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 msgr2=0x7fe3e806ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 --2- 172.21.15.45:0/2057395740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e806ed10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80ce640 tx=0x7fe3e001c930 comp rx=0 tx=0).stop 2024-04-25T10:27:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- 172.21.15.45:0/2057395740 shutdown_connections 2024-04-25T10:27:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 --2- 172.21.15.45:0/2057395740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e806ed10 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 --2- 172.21.15.45:0/2057395740 >> v2:172.21.15.117:3300/0 conn(0x7fe3e815bb70 0x7fe3e806e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- 172.21.15.45:0/2057395740 >> 172.21.15.45:0/2057395740 conn(0x7fe3e806ac90 msgr2=0x7fe3e806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:03.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- 172.21.15.45:0/2057395740 shutdown_connections 2024-04-25T10:27:03.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- 172.21.15.45:0/2057395740 wait complete. 2024-04-25T10:27:03.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 Processor -- start 2024-04-25T10:27:03.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- start start 2024-04-25T10:27:03.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e8071e40 0x7fe3e8148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e81657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e815bb70 0x7fe3e8165d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80cf870 con 0x7fe3e8072220 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80cf6f0 con 0x7fe3e815bb70 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3f0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80cf9f0 con 0x7fe3e8071e40 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.790+0000 7fe3ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e81657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e81657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49030/0 (socket says 172.21.15.45:49030) 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ee000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e815bb70 0x7fe3e8165d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 -- 172.21.15.45:0/1013090182 learned_addr learned my addr 172.21.15.45:0/1013090182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ed600640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e8071e40 0x7fe3e8148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 -- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e8071e40 msgr2=0x7fe3e8148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e8071e40 0x7fe3e8148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 -- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e815bb70 msgr2=0x7fe3e8165d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e815bb70 0x7fe3e8165d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 -- 172.21.15.45:0/1013090182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e814ac70 con 0x7fe3e8072220 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ee000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e815bb70 0x7fe3e8165d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3ecc00640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e81657c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e870 tx=0x7fe3d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4017070 con 0x7fe3e8072220 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d400c940 con 0x7fe3e8072220 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400ccf0 con 0x7fe3e8072220 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e814af50 con 0x7fe3e8072220 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e814b510 con 0x7fe3e8072220 2024-04-25T10:27:03.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e804ef30 con 0x7fe3e8072220 2024-04-25T10:27:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3d401a070 con 0x7fe3e8072220 2024-04-25T10:27:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.794+0000 7fe3e5600640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe3c8066f60 0x7fe3c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:03.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.798+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe3d40852f0 con 0x7fe3e8072220 2024-04-25T10:27:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.798+0000 7fe3ed600640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe3c8066f60 0x7fe3c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:03.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.798+0000 7fe3ed600640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe3c8066f60 0x7fe3c8069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001810 tx=0x7fe3dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:03.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.798+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3d4055bd0 con 0x7fe3e8072220 2024-04-25T10:27:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3e8071790 con 0x7fe3e8072220 2024-04-25T10:27:03.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3e5600640 1 -- 172.21.15.45:0/1013090182 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3d4059c70 con 0x7fe3e8072220 2024-04-25T10:27:03.968 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:27:03.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe3c8066f60 msgr2=0x7fe3c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe3c8066f60 0x7fe3c8069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001810 tx=0x7fe3dc009040 comp rx=0 tx=0).stop 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 msgr2=0x7fe3e81657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e81657c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400e870 tx=0x7fe3d400ec70 comp rx=0 tx=0).stop 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 shutdown_connections 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe3c8066f60 0x7fe3c8069420 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e815bb70 0x7fe3e8165d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3e8072220 0x7fe3e81657c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 --2- 172.21.15.45:0/1013090182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e8071e40 0x7fe3e8148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 >> 172.21.15.45:0/1013090182 conn(0x7fe3e806ac90 msgr2=0x7fe3e8164dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 shutdown_connections 2024-04-25T10:27:03.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:03.966+0000 7fe3f0000640 1 -- 172.21.15.45:0/1013090182 wait complete. 2024-04-25T10:27:03.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:27:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 Processor -- start 2024-04-25T10:27:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 -- start start 2024-04-25T10:27:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864159540 0x7f78640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78640aebd0 0x7f7864071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7864071360 con 0x7f7864072020 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640af030 con 0x7f78640aebd0 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f786bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7864159920 con 0x7f7864159540 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f7863e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78640aebd0 0x7f7864071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f7863e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f78640aebd0 0x7f7864071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49046/0 (socket says 172.21.15.45:49046) 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f7863e00640 1 -- 172.21.15.45:0/3327501686 learned_addr learned my addr 172.21.15.45:0/3327501686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.130+0000 7f7869200640 1 --2- 172.21.15.45:0/3327501686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864159540 0x7f78640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863e00640 1 -- 172.21.15.45:0/3327501686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864159540 msgr2=0x7f78640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863e00640 1 --2- 172.21.15.45:0/3327501686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864159540 0x7f78640ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863e00640 1 -- 172.21.15.45:0/3327501686 >> v1:172.21.15.117:6789/0 conn(0x7f7864072020 legacy=0x7f786415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:04.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863e00640 1 -- 172.21.15.45:0/3327501686 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786415c2f0 con 0x7f78640aebd0 2024-04-25T10:27:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863e00640 1 --2- 172.21.15.45:0/3327501686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f7864071ae0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7850009870 tx=0x7f785001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8360827c5cad7e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863400640 1 -- 172.21.15.45:0/3327501686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500ab650 con 0x7f78640aebd0 2024-04-25T10:27:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863400640 1 -- 172.21.15.45:0/3327501686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78500b0070 con 0x7f78640aebd0 2024-04-25T10:27:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f7863400640 1 -- 172.21.15.45:0/3327501686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500aba10 con 0x7f78640aebd0 2024-04-25T10:27:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f786bc00640 1 -- 172.21.15.45:0/3327501686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 msgr2=0x7f7864071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f786bc00640 1 --2- 172.21.15.45:0/3327501686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f7864071ae0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7850009870 tx=0x7f785001c530 comp rx=0 tx=0).stop 2024-04-25T10:27:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f786bc00640 1 -- 172.21.15.45:0/3327501686 shutdown_connections 2024-04-25T10:27:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.134+0000 7f786bc00640 1 --2- 172.21.15.45:0/3327501686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f7864071ae0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 --2- 172.21.15.45:0/3327501686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864159540 0x7f78640ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- 172.21.15.45:0/3327501686 >> 172.21.15.45:0/3327501686 conn(0x7f78640669a0 msgr2=0x7f7864066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- 172.21.15.45:0/3327501686 shutdown_connections 2024-04-25T10:27:04.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- 172.21.15.45:0/3327501686 wait complete. 2024-04-25T10:27:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 Processor -- start 2024-04-25T10:27:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- start start 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864072020 0x7f786413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f78641f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7864159540 0x7f786415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786406c0d0 con 0x7f78640aebd0 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786415bfa0 con 0x7f7864159540 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f786bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7864072430 con 0x7f7864072020 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f7863e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f78641f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f7869200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864072020 0x7f786413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f7863e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f78641f7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49050/0 (socket says 172.21.15.45:49050) 2024-04-25T10:27:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f7863e00640 1 -- 172.21.15.45:0/998281686 learned_addr learned my addr 172.21.15.45:0/998281686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.138+0000 7f7869c00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7864159540 0x7f786415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7863e00640 1 -- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864072020 msgr2=0x7f786413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7863e00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864072020 0x7f786413d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7863e00640 1 -- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7864159540 msgr2=0x7f786415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7863e00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7864159540 0x7f786415afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7863e00640 1 -- 172.21.15.45:0/998281686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864150a00 con 0x7f78640aebd0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7863e00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f78641f7b30 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f78500ad040 tx=0x7f785001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500be5f0 con 0x7f78640aebd0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78500be790 con 0x7f78640aebd0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500beb80 con 0x7f78640aebd0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7864150ce0 con 0x7f78640aebd0 2024-04-25T10:27:04.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786414dd30 con 0x7f78640aebd0 2024-04-25T10:27:04.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786404ef30 con 0x7f78640aebd0 2024-04-25T10:27:04.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.142+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78500b43f0 con 0x7f78640aebd0 2024-04-25T10:27:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.146+0000 7f7860c00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7844066f60 0x7f7844069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.146+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f785011f950 con 0x7f78640aebd0 2024-04-25T10:27:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.146+0000 7f7869200640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7844066f60 0x7f7844069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.146+0000 7f7869200640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7844066f60 0x7f7844069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f78580079c0 tx=0x7f7858008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.150+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7850016150 con 0x7f78640aebd0 2024-04-25T10:27:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7864166700 con 0x7f78640aebd0 2024-04-25T10:27:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f7860c00640 1 -- 172.21.15.45:0/998281686 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78500f0230 con 0x7f78640aebd0 2024-04-25T10:27:04.313 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7844066f60 msgr2=0x7f7844069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7844066f60 0x7f7844069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f78580079c0 tx=0x7f7858008040 comp rx=0 tx=0).stop 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 msgr2=0x7f78641f7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f78641f7b30 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f78500ad040 tx=0x7f785001ad70 comp rx=0 tx=0).stop 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 shutdown_connections 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7844066f60 0x7f7844069420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7864159540 0x7f786415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78640aebd0 0x7f78641f7b30 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 --2- 172.21.15.45:0/998281686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7864072020 0x7f786413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 >> 172.21.15.45:0/998281686 conn(0x7f78640669a0 msgr2=0x7f78640cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 shutdown_connections 2024-04-25T10:27:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.310+0000 7f786bc00640 1 -- 172.21.15.45:0/998281686 wait complete. 2024-04-25T10:27:04.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:27:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 Processor -- start 2024-04-25T10:27:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 -- start start 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a90070350 0x7f6a9015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a900b31c0 0x7f6a900b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9015b080 con 0x7f6a900aef80 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9015adb0 con 0x7f6a90070350 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a96200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9015eb60 con 0x7f6a900b31c0 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a8ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a90070350 0x7f6a9015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a8ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a90070350 0x7f6a9015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49060/0 (socket says 172.21.15.45:49060) 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.474+0000 7f6a8ea00640 1 -- 172.21.15.45:0/2344122233 learned_addr learned my addr 172.21.15.45:0/2344122233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8fe00640 1 --2- 172.21.15.45:0/2344122233 >> v2:172.21.15.117:3300/0 conn(0x7f6a900b31c0 0x7f6a900b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8ea00640 1 -- 172.21.15.45:0/2344122233 >> v2:172.21.15.117:3300/0 conn(0x7f6a900b31c0 msgr2=0x7f6a900b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8ea00640 1 --2- 172.21.15.45:0/2344122233 >> v2:172.21.15.117:3300/0 conn(0x7f6a900b31c0 0x7f6a900b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8ea00640 1 -- 172.21.15.45:0/2344122233 >> v1:172.21.15.117:6789/0 conn(0x7f6a900aef80 legacy=0x7f6a9015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8ea00640 1 -- 172.21.15.45:0/2344122233 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9015ee40 con 0x7f6a90070350 2024-04-25T10:27:04.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8ea00640 1 --2- 172.21.15.45:0/2344122233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a90070350 0x7f6a9015e480 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6a900d0750 tx=0x7f6a7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaaa5694ff9802a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8e000640 1 -- 172.21.15.45:0/2344122233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0ab430 con 0x7f6a90070350 2024-04-25T10:27:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8e000640 1 -- 172.21.15.45:0/2344122233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c0b0070 con 0x7f6a90070350 2024-04-25T10:27:04.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a8e000640 1 -- 172.21.15.45:0/2344122233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0ab7f0 con 0x7f6a90070350 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a96200640 1 -- 172.21.15.45:0/2344122233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a90070350 msgr2=0x7f6a9015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a96200640 1 --2- 172.21.15.45:0/2344122233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a90070350 0x7f6a9015e480 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6a900d0750 tx=0x7f6a7c01c520 comp rx=0 tx=0).stop 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.478+0000 7f6a96200640 1 -- 172.21.15.45:0/2344122233 shutdown_connections 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 --2- 172.21.15.45:0/2344122233 >> v2:172.21.15.117:3300/0 conn(0x7f6a900b31c0 0x7f6a900b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 --2- 172.21.15.45:0/2344122233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a90070350 0x7f6a9015e480 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- 172.21.15.45:0/2344122233 >> 172.21.15.45:0/2344122233 conn(0x7f6a900669a0 msgr2=0x7f6a90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- 172.21.15.45:0/2344122233 shutdown_connections 2024-04-25T10:27:04.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- 172.21.15.45:0/2344122233 wait complete. 2024-04-25T10:27:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 Processor -- start 2024-04-25T10:27:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- start start 2024-04-25T10:27:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a90070350 0x7f6a9014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a900aef80 0x7f6a9014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 0x7f6a9013d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901679a0 con 0x7f6a900b31c0 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a90167820 con 0x7f6a900aef80 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a90167b20 con 0x7f6a90070350 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 0x7f6a9013d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 0x7f6a9013d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49070/0 (socket says 172.21.15.45:49070) 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 -- 172.21.15.45:0/1806610346 learned_addr learned my addr 172.21.15.45:0/1806610346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 -- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a90070350 msgr2=0x7f6a9014eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a90070350 0x7f6a9014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8ea00640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a900aef80 0x7f6a9014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 -- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a900aef80 msgr2=0x7f6a9014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a900aef80 0x7f6a9014bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 -- 172.21.15.45:0/1806610346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9014c220 con 0x7f6a900b31c0 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a8fe00640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 0x7f6a9013d280 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6a88007e60 tx=0x7f6a8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88014420 con 0x7f6a900b31c0 2024-04-25T10:27:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a88019070 con 0x7f6a900b31c0 2024-04-25T10:27:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88014820 con 0x7f6a900b31c0 2024-04-25T10:27:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.482+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9013d7c0 con 0x7f6a900b31c0 2024-04-25T10:27:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a90152750 con 0x7f6a900b31c0 2024-04-25T10:27:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9004ef30 con 0x7f6a900b31c0 2024-04-25T10:27:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a880149c0 con 0x7f6a900b31c0 2024-04-25T10:27:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a87400640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a70066f60 0x7f6a70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6a88085960 con 0x7f6a900b31c0 2024-04-25T10:27:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a8f400640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a70066f60 0x7f6a70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.486+0000 7f6a8f400640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a70066f60 0x7f6a70069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6a80000f70 tx=0x7f6a800079e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.490+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a88056890 con 0x7f6a900b31c0 2024-04-25T10:27:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.650+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a900b2fa0 con 0x7f6a900b31c0 2024-04-25T10:27:04.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.650+0000 7f6a87400640 1 -- 172.21.15.45:0/1806610346 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a88056190 con 0x7f6a900b31c0 2024-04-25T10:27:04.655 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a70066f60 msgr2=0x7f6a70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a70066f60 0x7f6a70069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6a80000f70 tx=0x7f6a800079e0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 msgr2=0x7f6a9013d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 0x7f6a9013d280 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6a88007e60 tx=0x7f6a8800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 shutdown_connections 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a70066f60 0x7f6a70069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a900b31c0 0x7f6a9013d280 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a900aef80 0x7f6a9014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 --2- 172.21.15.45:0/1806610346 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a90070350 0x7f6a9014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 >> 172.21.15.45:0/1806610346 conn(0x7f6a900669a0 msgr2=0x7f6a90154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 shutdown_connections 2024-04-25T10:27:04.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.654+0000 7f6a96200640 1 -- 172.21.15.45:0/1806610346 wait complete. 2024-04-25T10:27:04.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:27:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 Processor -- start 2024-04-25T10:27:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 -- start start 2024-04-25T10:27:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01f80d0040 0x7f01f815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f815b740 0x7f01f80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f8071050 con 0x7f01f81550b0 2024-04-25T10:27:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f815d040 con 0x7f01f80d0040 2024-04-25T10:27:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f0200000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f80712f0 con 0x7f01f815b740 2024-04-25T10:27:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01f80d0040 0x7f01f815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fe000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f815b740 0x7f01f80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01f80d0040 0x7f01f815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37690/0 (socket says 172.21.15.45:37690) 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 -- 172.21.15.45:0/1523161906 learned_addr learned my addr 172.21.15.45:0/1523161906 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 -- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f815b740 msgr2=0x7f01f80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 --2- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f815b740 0x7f01f80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 -- 172.21.15.45:0/1523161906 >> v1:172.21.15.117:6789/0 conn(0x7f01f81550b0 legacy=0x7f01f80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.818+0000 7f01fcc00640 1 -- 172.21.15.45:0/1523161906 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f80715d0 con 0x7f01f80d0040 2024-04-25T10:27:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f01fcc00640 1 --2- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f815b200 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009870 tx=0x7f01e401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d7f2c95daab7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f01f7e00640 1 -- 172.21.15.45:0/1523161906 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40a95f0 con 0x7f01f80d0040 2024-04-25T10:27:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f01f7e00640 1 -- 172.21.15.45:0/1523161906 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40a9790 con 0x7f01f80d0040 2024-04-25T10:27:04.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f01f7e00640 1 -- 172.21.15.45:0/1523161906 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40a9b60 con 0x7f01f80d0040 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 -- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 msgr2=0x7f01f815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 --2- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f815b200 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009870 tx=0x7f01e401c530 comp rx=0 tx=0).stop 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 -- 172.21.15.45:0/1523161906 shutdown_connections 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 --2- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f815b740 0x7f01f80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 --2- 172.21.15.45:0/1523161906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f815b200 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 -- 172.21.15.45:0/1523161906 >> 172.21.15.45:0/1523161906 conn(0x7f01f8066b70 msgr2=0x7f01f8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 -- 172.21.15.45:0/1523161906 shutdown_connections 2024-04-25T10:27:04.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.822+0000 7f0200000640 1 -- 172.21.15.45:0/1523161906 wait complete. 2024-04-25T10:27:04.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 Processor -- start 2024-04-25T10:27:04.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 -- start start 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f814c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f81550b0 0x7f01f80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 0x7f01f8157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815bcd0 con 0x7f01f815b740 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815bb50 con 0x7f01f80d0040 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815c030 con 0x7f01f81550b0 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 0x7f01f8157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 0x7f01f8157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49080/0 (socket says 172.21.15.45:49080) 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 -- 172.21.15.45:0/2587071368 learned_addr learned my addr 172.21.15.45:0/2587071368 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fd600640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f814c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fcc00640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f81550b0 0x7f01f80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 -- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f81550b0 msgr2=0x7f01f80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f81550b0 0x7f01f80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 -- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 msgr2=0x7f01f814c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f814c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 -- 172.21.15.45:0/2587071368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f814ef10 con 0x7f01f815b740 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01fe000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 0x7f01f8157200 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009d30 tx=0x7f01f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0018020 con 0x7f01f815b740 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f0014480 con 0x7f01f815b740 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0014870 con 0x7f01f815b740 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f81557e0 con 0x7f01f815b740 2024-04-25T10:27:04.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.826+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f8156080 con 0x7f01f815b740 2024-04-25T10:27:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.830+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f8155bd0 con 0x7f01f815b740 2024-04-25T10:27:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.830+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01f001b020 con 0x7f01f815b740 2024-04-25T10:27:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.830+0000 7f01f5600640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01d8066f90 0x7f01d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:04.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.830+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01f00858e0 con 0x7f01f815b740 2024-04-25T10:27:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.830+0000 7f01fd600640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01d8066f90 0x7f01d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:04.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.830+0000 7f01fd600640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01d8066f90 0x7f01d8069450 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f01ec001810 tx=0x7f01ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:04.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:04.834+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01f0022070 con 0x7f01f815b740 2024-04-25T10:27:05.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.006+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f01f81506c0 con 0x7f01f815b740 2024-04-25T10:27:05.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f01f5600640 1 -- 172.21.15.45:0/2587071368 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) v1 ==== 80+0+0 (secure 0 0 0) 0x7f01f0010040 con 0x7f01f815b740 2024-04-25T10:27:05.012 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01d8066f90 msgr2=0x7f01d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01d8066f90 0x7f01d8069450 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f01ec001810 tx=0x7f01ec009040 comp rx=0 tx=0).stop 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 msgr2=0x7f01f8157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 0x7f01f8157200 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009d30 tx=0x7f01f000c490 comp rx=0 tx=0).stop 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 shutdown_connections 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01d8066f90 0x7f01d8069450 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01f815b740 0x7f01f8157200 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01f81550b0 0x7f01f80adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 --2- 172.21.15.45:0/2587071368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01f80d0040 0x7f01f814c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 >> 172.21.15.45:0/2587071368 conn(0x7f01f8066b70 msgr2=0x7f01f806c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:05.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 shutdown_connections 2024-04-25T10:27:05.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.010+0000 7f0200000640 1 -- 172.21.15.45:0/2587071368 wait complete. 2024-04-25T10:27:05.032 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:27:05.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:05.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:27:05.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:27:05.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:05.734 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:27:05.743 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:27:05.743 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:05.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:05.753 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:27:05.753 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:05.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:05.796 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:27:05.796 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:05.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:05.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 Processor -- start 2024-04-25T10:27:05.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 -- start start 2024-04-25T10:27:05.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 0x7f2b500cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b500b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b50071050 con 0x7f2b500cfe70 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5015bda0 con 0x7f2b501550b0 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b56000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015d040 con 0x7f2b5015b910 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 0x7f2b500cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b500b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 0x7f2b500cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57498/0 (socket says 172.21.15.45:57498) 2024-04-25T10:27:05.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4f400640 1 -- 172.21.15.45:0/2035653451 learned_addr learned my addr 172.21.15.45:0/2035653451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4fe00640 1 -- 172.21.15.45:0/2035653451 >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 msgr2=0x7f2b500cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4fe00640 1 --2- 172.21.15.45:0/2035653451 >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 0x7f2b500cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4fe00640 1 -- 172.21.15.45:0/2035653451 >> v1:172.21.15.117:6789/0 conn(0x7f2b500cfe70 legacy=0x7f2b5015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4fe00640 1 -- 172.21.15.45:0/2035653451 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500712f0 con 0x7f2b5015b910 2024-04-25T10:27:05.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.982+0000 7f2b4f400640 1 --2- 172.21.15.45:0/2035653451 >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 0x7f2b500cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b4fe00640 1 --2- 172.21.15.45:0/2035653451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b500b3260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009dc0 tx=0x7f2b4801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5f7f8327de45f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:05.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b4e000640 1 -- 172.21.15.45:0/2035653451 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480b2070 con 0x7f2b5015b910 2024-04-25T10:27:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b4e000640 1 -- 172.21.15.45:0/2035653451 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b480a5900 con 0x7f2b5015b910 2024-04-25T10:27:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b4e000640 1 -- 172.21.15.45:0/2035653451 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480a5cd0 con 0x7f2b5015b910 2024-04-25T10:27:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 -- 172.21.15.45:0/2035653451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 msgr2=0x7f2b500b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:05.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 --2- 172.21.15.45:0/2035653451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b500b3260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009dc0 tx=0x7f2b4801c950 comp rx=0 tx=0).stop 2024-04-25T10:27:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 -- 172.21.15.45:0/2035653451 shutdown_connections 2024-04-25T10:27:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 --2- 172.21.15.45:0/2035653451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b500b3260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 --2- 172.21.15.45:0/2035653451 >> v2:172.21.15.117:3300/0 conn(0x7f2b501550b0 0x7f2b500cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 -- 172.21.15.45:0/2035653451 >> 172.21.15.45:0/2035653451 conn(0x7f2b50066b70 msgr2=0x7f2b50066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 -- 172.21.15.45:0/2035653451 shutdown_connections 2024-04-25T10:27:05.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.986+0000 7f2b56000640 1 -- 172.21.15.45:0/2035653451 wait complete. 2024-04-25T10:27:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 Processor -- start 2024-04-25T10:27:05.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 -- start start 2024-04-25T10:27:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 0x7f2b5014e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b501550b0 0x7f2b500cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b5014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:05.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015c030 con 0x7f2b500cfe70 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500720b0 con 0x7f2b501550b0 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015c1b0 con 0x7f2b5015b910 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 0x7f2b5014e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 0x7f2b5014e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40808/0 (socket says 172.21.15.45:40808) 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 -- 172.21.15.45:0/4122821551 learned_addr learned my addr 172.21.15.45:0/4122821551 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4ea00640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b501550b0 0x7f2b500cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:05.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4fe00640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b5014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 -- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 msgr2=0x7f2b5014d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b5014d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 -- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b501550b0 msgr2=0x7f2b500cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:05.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b501550b0 0x7f2b500cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 -- 172.21.15.45:0/4122821551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5014efd0 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b4f400640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 0x7f2b5014e8d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000ea00 tx=0x7f2b4000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40017070 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4000ca00 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000cdb0 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b50155630 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.990+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b50155bf0 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.994+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5004ef30 con 0x7f2b500cfe70 2024-04-25T10:27:05.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.994+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b4001a070 con 0x7f2b500cfe70 2024-04-25T10:27:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.994+0000 7f2b47400640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b30066f60 0x7f2b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.994+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2b400852b0 con 0x7f2b500cfe70 2024-04-25T10:27:05.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.994+0000 7f2b4ea00640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b30066f60 0x7f2b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:05.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.994+0000 7f2b4ea00640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b30066f60 0x7f2b30069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b500cea00 tx=0x7f2b3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:05.998+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b40055c20 con 0x7f2b500cfe70 2024-04-25T10:27:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.194+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b5014dc60 con 0x7f2b500cfe70 2024-04-25T10:27:06.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.194+0000 7f2b47400640 1 -- 172.21.15.45:0/4122821551 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f2b40059cc0 con 0x7f2b500cfe70 2024-04-25T10:27:06.198 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:06.198 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:27:06.199 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:27:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b30066f60 msgr2=0x7f2b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b30066f60 0x7f2b30069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2b500cea00 tx=0x7f2b3c009040 comp rx=0 tx=0).stop 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 msgr2=0x7f2b5014e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 0x7f2b5014e8d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000ea00 tx=0x7f2b4000eec0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 shutdown_connections 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b30066f60 0x7f2b30069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b5015b910 0x7f2b5014d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b501550b0 0x7f2b500cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 --2- 172.21.15.45:0/4122821551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b500cfe70 0x7f2b5014e8d0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 >> 172.21.15.45:0/4122821551 conn(0x7f2b50066b70 msgr2=0x7f2b500b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:06.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 shutdown_connections 2024-04-25T10:27:06.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.198+0000 7f2b56000640 1 -- 172.21.15.45:0/4122821551 wait complete. 2024-04-25T10:27:06.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 Processor -- start 2024-04-25T10:27:06.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 -- start start 2024-04-25T10:27:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c0d0b20 con 0x7f378c0cdb40 2024-04-25T10:27:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c15e340 con 0x7f378c15df60 2024-04-25T10:27:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3793a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0ccaa0 con 0x7f378c0d0180 2024-04-25T10:27:06.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3791a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f378c0cdb40 0x7f378c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36774/0 (socket says 172.21.15.45:36774) 2024-04-25T10:27:06.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.358+0000 7f3791a00640 1 -- 172.21.15.45:0/3568092689 learned_addr learned my addr 172.21.15.45:0/3568092689 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:06.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3783e00640 1 --2- 172.21.15.45:0/3568092689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3791000640 1 --2- 172.21.15.45:0/3568092689 >> v2:172.21.15.117:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f378be00640 1 -- 172.21.15.45:0/3568092689 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239105516 0 0) 0x7f378c0d0b20 con 0x7f378c0cdb40 2024-04-25T10:27:06.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f378be00640 1 -- 172.21.15.45:0/3568092689 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3770003530 con 0x7f378c0cdb40 2024-04-25T10:27:06.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3783e00640 1 -- 172.21.15.45:0/3568092689 >> v2:172.21.15.117:3300/0 conn(0x7f378c15df60 msgr2=0x7f378c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:06.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3783e00640 1 --2- 172.21.15.45:0/3568092689 >> v2:172.21.15.117:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3783e00640 1 -- 172.21.15.45:0/3568092689 >> v1:172.21.15.45:6789/0 conn(0x7f378c0cdb40 legacy=0x7f378c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3783e00640 1 -- 172.21.15.45:0/3568092689 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0ccd80 con 0x7f378c0d0180 2024-04-25T10:27:06.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3791000640 1 --2- 172.21.15.45:0/3568092689 >> v2:172.21.15.117:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:06.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f3783e00640 1 --2- 172.21.15.45:0/3568092689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0d0180 0x7f378c0d05e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37740097c0 tx=0x7f377401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08d06bc933919dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f378be00640 1 -- 172.21.15.45:0/3568092689 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740b0070 con 0x7f378c0d0180 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f378be00640 1 -- 172.21.15.45:0/3568092689 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740ab600 con 0x7f378c0d0180 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.362+0000 7f378be00640 1 -- 172.21.15.45:0/3568092689 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740ab9b0 con 0x7f378c0d0180 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- 172.21.15.45:0/3568092689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0d0180 msgr2=0x7f378c0d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 --2- 172.21.15.45:0/3568092689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0d0180 0x7f378c0d05e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37740097c0 tx=0x7f377401c410 comp rx=0 tx=0).stop 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- 172.21.15.45:0/3568092689 shutdown_connections 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 --2- 172.21.15.45:0/3568092689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0d0180 0x7f378c0d05e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 --2- 172.21.15.45:0/3568092689 >> v2:172.21.15.117:3300/0 conn(0x7f378c15df60 0x7f378c0cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- 172.21.15.45:0/3568092689 >> 172.21.15.45:0/3568092689 conn(0x7f378c0d1910 msgr2=0x7f378c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- 172.21.15.45:0/3568092689 shutdown_connections 2024-04-25T10:27:06.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- 172.21.15.45:0/3568092689 wait complete. 2024-04-25T10:27:06.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 Processor -- start 2024-04-25T10:27:06.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- start start 2024-04-25T10:27:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0cdb40 0x7f378c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 0x7f378c155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378c15df60 0x7f378c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0d0ff0 con 0x7f378c0d0180 2024-04-25T10:27:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0d0e70 con 0x7f378c15df60 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3793a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0cead0 con 0x7f378c0cdb40 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 0x7f378c155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 0x7f378c155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40822/0 (socket says 172.21.15.45:40822) 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 -- 172.21.15.45:0/3493465256 learned_addr learned my addr 172.21.15.45:0/3493465256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3791a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378c15df60 0x7f378c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3791000640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0cdb40 0x7f378c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 -- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0cdb40 msgr2=0x7f378c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0cdb40 0x7f378c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 -- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378c15df60 msgr2=0x7f378c0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378c15df60 0x7f378c0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.366+0000 7f3783e00640 1 -- 172.21.15.45:0/3493465256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c06f9d0 con 0x7f378c0d0180 2024-04-25T10:27:06.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3783e00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 0x7f378c155720 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f377401a040 tx=0x7f37740191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740ba070 con 0x7f378c0d0180 2024-04-25T10:27:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37740ab9b0 con 0x7f378c0d0180 2024-04-25T10:27:06.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740b0040 con 0x7f378c0d0180 2024-04-25T10:27:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c06fcb0 con 0x7f378c0d0180 2024-04-25T10:27:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c15d6e0 con 0x7f378c0d0180 2024-04-25T10:27:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c04ef30 con 0x7f378c0d0180 2024-04-25T10:27:06.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37740ab480 con 0x7f378c0d0180 2024-04-25T10:27:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3789600640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3768066f90 0x7f3768069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f377411e200 con 0x7f378c0d0180 2024-04-25T10:27:06.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.370+0000 7f3791000640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3768066f90 0x7f3768069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.374+0000 7f3791000640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3768066f90 0x7f3768069450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f378c06eb80 tx=0x7f377c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.374+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3774016090 con 0x7f378c0d0180 2024-04-25T10:27:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.538+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f378c0ce2f0 con 0x7f378c0d0180 2024-04-25T10:27:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3789600640 1 -- 172.21.15.45:0/3493465256 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16769 (secure 0 0 0) 0x7f37740ef130 con 0x7f378c0d0180 2024-04-25T10:27:06.545 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:06.545 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:43.714023+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.21","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3768066f90 msgr2=0x7f3768069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3768066f90 0x7f3768069450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f378c06eb80 tx=0x7f377c008040 comp rx=0 tx=0).stop 2024-04-25T10:27:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 msgr2=0x7f378c155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 0x7f378c155720 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f377401a040 tx=0x7f37740191b0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 shutdown_connections 2024-04-25T10:27:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3768066f90 0x7f3768069450 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378c15df60 0x7f378c0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f378c0d0180 0x7f378c155720 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 --2- 172.21.15.45:0/3493465256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f378c0cdb40 0x7f378c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 >> 172.21.15.45:0/3493465256 conn(0x7f378c0d1910 msgr2=0x7f378c159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 shutdown_connections 2024-04-25T10:27:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.542+0000 7f3793a00640 1 -- 172.21.15.45:0/3493465256 wait complete. 2024-04-25T10:27:06.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:26:11.768328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:26:12.345255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:27:06.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:27:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.706+0000 7f73a8800640 1 Processor -- start 2024-04-25T10:27:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.706+0000 7f73a8800640 1 -- start start 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a8800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73a0159540 0x7f73a00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a8800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73a00721f0 0x7f73a0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a8800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a0159920 con 0x7f73a00aebd0 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a8800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a015c2f0 con 0x7f73a0159540 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a8800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a015c5d0 con 0x7f73a00721f0 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73a0159540 0x7f73a00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f73a00aebd0 0x7f73a0071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36782/0 (socket says 172.21.15.45:36782) 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5400640 1 -- 172.21.15.45:0/2590292213 learned_addr learned my addr 172.21.15.45:0/2590292213 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73a0159540 0x7f73a00ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40838/0 (socket says 172.21.15.45:40838) 2024-04-25T10:27:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a6800640 1 --2- 172.21.15.45:0/2590292213 >> v2:172.21.15.117:3300/0 conn(0x7f73a00721f0 0x7f73a0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5e00640 1 -- 172.21.15.45:0/2590292213 >> v2:172.21.15.117:3300/0 conn(0x7f73a00721f0 msgr2=0x7f73a0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5e00640 1 --2- 172.21.15.45:0/2590292213 >> v2:172.21.15.117:3300/0 conn(0x7f73a00721f0 0x7f73a0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5e00640 1 -- 172.21.15.45:0/2590292213 >> v1:172.21.15.45:6789/0 conn(0x7f73a00aebd0 legacy=0x7f73a0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a5e00640 1 -- 172.21.15.45:0/2590292213 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a015c8b0 con 0x7f73a0159540 2024-04-25T10:27:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.710+0000 7f73a6800640 1 --2- 172.21.15.45:0/2590292213 >> v2:172.21.15.117:3300/0 conn(0x7f73a00721f0 0x7f73a0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a5e00640 1 --2- 172.21.15.45:0/2590292213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a00ae690 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f73a00cd200 tx=0x7f739401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f56da4ae0d29e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a4a00640 1 -- 172.21.15.45:0/2590292213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940b1070 con 0x7f73a0159540 2024-04-25T10:27:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a4a00640 1 -- 172.21.15.45:0/2590292213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73940ac5f0 con 0x7f73a0159540 2024-04-25T10:27:06.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a4a00640 1 -- 172.21.15.45:0/2590292213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940ac9c0 con 0x7f73a0159540 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 -- 172.21.15.45:0/2590292213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 msgr2=0x7f73a00ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 --2- 172.21.15.45:0/2590292213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a00ae690 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f73a00cd200 tx=0x7f739401c480 comp rx=0 tx=0).stop 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 -- 172.21.15.45:0/2590292213 shutdown_connections 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 --2- 172.21.15.45:0/2590292213 >> v2:172.21.15.117:3300/0 conn(0x7f73a00721f0 0x7f73a0070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 --2- 172.21.15.45:0/2590292213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a00ae690 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 -- 172.21.15.45:0/2590292213 >> 172.21.15.45:0/2590292213 conn(0x7f73a00669a0 msgr2=0x7f73a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 -- 172.21.15.45:0/2590292213 shutdown_connections 2024-04-25T10:27:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 -- 172.21.15.45:0/2590292213 wait complete. 2024-04-25T10:27:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 Processor -- start 2024-04-25T10:27:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 -- start start 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.714+0000 7f73a8800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a00721f0 0x7f73a014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a00aebd0 0x7f73a014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0071220 con 0x7f73a0159540 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a00710a0 con 0x7f73a00aebd0 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a00713a0 con 0x7f73a00721f0 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a015a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40848/0 (socket says 172.21.15.45:40848) 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 -- 172.21.15.45:0/899270896 learned_addr learned my addr 172.21.15.45:0/899270896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a5400640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a00aebd0 0x7f73a014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a5e00640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a00721f0 0x7f73a014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 -- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a00721f0 msgr2=0x7f73a014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a00721f0 0x7f73a014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 -- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a00aebd0 msgr2=0x7f73a014b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a00aebd0 0x7f73a014b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 -- 172.21.15.45:0/899270896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a014cd50 con 0x7f73a0159540 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a5e00640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a00721f0 0x7f73a014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a6800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a015a6e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f739c009fa0 tx=0x7f739c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c01a070 con 0x7f73a0159540 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c003bc0 con 0x7f73a0159540 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c004030 con 0x7f73a0159540 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a014d030 con 0x7f73a0159540 2024-04-25T10:27:06.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a00af6b0 con 0x7f73a0159540 2024-04-25T10:27:06.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a004ef30 con 0x7f73a0159540 2024-04-25T10:27:06.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.718+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f739c01d070 con 0x7f73a0159540 2024-04-25T10:27:06.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.722+0000 7f739a000640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7384066f60 0x7f7384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:06.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.722+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f739c0852e0 con 0x7f73a0159540 2024-04-25T10:27:06.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.722+0000 7f73a5e00640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7384066f60 0x7f7384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:06.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.722+0000 7f73a5e00640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7384066f60 0x7f7384069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f73940ae000 tx=0x7f7394008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:06.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.722+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f739c010070 con 0x7f73a0159540 2024-04-25T10:27:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.886+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73a00cb980 con 0x7f73a0159540 2024-04-25T10:27:06.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.886+0000 7f739a000640 1 -- 172.21.15.45:0/899270896 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f739c055c50 con 0x7f73a0159540 2024-04-25T10:27:06.894 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7384066f60 msgr2=0x7f7384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7384066f60 0x7f7384069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f73940ae000 tx=0x7f7394008040 comp rx=0 tx=0).stop 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 msgr2=0x7f73a015a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a015a6e0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f739c009fa0 tx=0x7f739c00d560 comp rx=0 tx=0).stop 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 shutdown_connections 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7384066f60 0x7f7384069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a0159540 0x7f73a015a6e0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73a00aebd0 0x7f73a014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 --2- 172.21.15.45:0/899270896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73a00721f0 0x7f73a014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 >> 172.21.15.45:0/899270896 conn(0x7f73a00669a0 msgr2=0x7f73a0071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 shutdown_connections 2024-04-25T10:27:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:06.890+0000 7f73a8800640 1 -- 172.21.15.45:0/899270896 wait complete. 2024-04-25T10:27:06.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:27:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 Processor -- start 2024-04-25T10:27:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 -- start start 2024-04-25T10:27:07.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b5c070350 0x7f1b5c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b5c0b31c0 0x7f1b5c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c15b080 con 0x7f1b5c0aef80 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c15adb0 con 0x7f1b5c070350 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b63000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c15eb60 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b5be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b5c0aef80 0x7f1b5c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36798/0 (socket says 172.21.15.45:36798) 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b5be00640 1 -- 172.21.15.45:0/3970304651 learned_addr learned my addr 172.21.15.45:0/3970304651 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b53400640 1 --2- 172.21.15.45:0/3970304651 >> v2:172.21.15.45:3300/0 conn(0x7f1b5c070350 0x7f1b5c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b5b400640 1 -- 172.21.15.45:0/3970304651 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257703391 0 0) 0x7f1b5c15b080 con 0x7f1b5c0aef80 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b61000640 1 --2- 172.21.15.45:0/3970304651 >> v2:172.21.15.117:3300/0 conn(0x7f1b5c0b31c0 0x7f1b5c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b5b400640 1 -- 172.21.15.45:0/3970304651 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b40003530 con 0x7f1b5c0aef80 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b53400640 1 -- 172.21.15.45:0/3970304651 >> v2:172.21.15.117:3300/0 conn(0x7f1b5c0b31c0 msgr2=0x7f1b5c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b53400640 1 --2- 172.21.15.45:0/3970304651 >> v2:172.21.15.117:3300/0 conn(0x7f1b5c0b31c0 0x7f1b5c0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b53400640 1 -- 172.21.15.45:0/3970304651 >> v1:172.21.15.45:6789/0 conn(0x7f1b5c0aef80 legacy=0x7f1b5c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.054+0000 7f1b53400640 1 -- 172.21.15.45:0/3970304651 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c15ee40 con 0x7f1b5c070350 2024-04-25T10:27:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b53400640 1 --2- 172.21.15.45:0/3970304651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c070350 0x7f1b5c15e480 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0d0750 tx=0x7f1b4401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ca61fc94a6d92a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b5b400640 1 -- 172.21.15.45:0/3970304651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440b1070 con 0x7f1b5c070350 2024-04-25T10:27:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b5b400640 1 -- 172.21.15.45:0/3970304651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440ac5f0 con 0x7f1b5c070350 2024-04-25T10:27:07.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b5b400640 1 -- 172.21.15.45:0/3970304651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440ac9c0 con 0x7f1b5c070350 2024-04-25T10:27:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 -- 172.21.15.45:0/3970304651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c070350 msgr2=0x7f1b5c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 --2- 172.21.15.45:0/3970304651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c070350 0x7f1b5c15e480 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0d0750 tx=0x7f1b4401c480 comp rx=0 tx=0).stop 2024-04-25T10:27:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 -- 172.21.15.45:0/3970304651 shutdown_connections 2024-04-25T10:27:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 --2- 172.21.15.45:0/3970304651 >> v2:172.21.15.117:3300/0 conn(0x7f1b5c0b31c0 0x7f1b5c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 --2- 172.21.15.45:0/3970304651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c070350 0x7f1b5c15e480 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 -- 172.21.15.45:0/3970304651 >> 172.21.15.45:0/3970304651 conn(0x7f1b5c0669a0 msgr2=0x7f1b5c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 -- 172.21.15.45:0/3970304651 shutdown_connections 2024-04-25T10:27:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.058+0000 7f1b63000640 1 -- 172.21.15.45:0/3970304651 wait complete. 2024-04-25T10:27:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 Processor -- start 2024-04-25T10:27:07.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- start start 2024-04-25T10:27:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c070350 0x7f1b5c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0aef80 0x7f1b5c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 0x7f1b5c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c1679a0 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c167820 con 0x7f1b5c070350 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c167b20 con 0x7f1b5c0aef80 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 0x7f1b5c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 0x7f1b5c0b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40854/0 (socket says 172.21.15.45:40854) 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 -- 172.21.15.45:0/3692888919 learned_addr learned my addr 172.21.15.45:0/3692888919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b5be00640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c070350 0x7f1b5c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b53400640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0aef80 0x7f1b5c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 -- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0aef80 msgr2=0x7f1b5c0b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0aef80 0x7f1b5c0b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 -- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c070350 msgr2=0x7f1b5c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c070350 0x7f1b5c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 -- 172.21.15.45:0/3692888919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c14f8f0 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b61000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 0x7f1b5c0b1f60 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007e60 tx=0x7f1b5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54014420 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b54019070 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54014820 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c0b24a0 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c14fe70 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.062+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c04ef30 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.066+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b540149c0 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.066+0000 7f1b58c00640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b38066f60 0x7f1b38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.066+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1b540858b0 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.066+0000 7f1b5be00640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b38066f60 0x7f1b38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.066+0000 7f1b5be00640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b38066f60 0x7f1b38069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007fd0 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.070+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b54056100 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b5c150590 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b58c00640 1 -- 172.21.15.45:0/3692888919 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1b5405a1a0 con 0x7f1b5c0b31c0 2024-04-25T10:27:07.233 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:27:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b38066f60 msgr2=0x7f1b38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b38066f60 0x7f1b38069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007fd0 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-04-25T10:27:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 msgr2=0x7f1b5c0b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 0x7f1b5c0b1f60 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f1b54007e60 tx=0x7f1b5400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 shutdown_connections 2024-04-25T10:27:07.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b38066f60 0x7f1b38069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b5c0b31c0 0x7f1b5c0b1f60 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b5c0aef80 0x7f1b5c0b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 --2- 172.21.15.45:0/3692888919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b5c070350 0x7f1b5c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 >> 172.21.15.45:0/3692888919 conn(0x7f1b5c0669a0 msgr2=0x7f1b5c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 shutdown_connections 2024-04-25T10:27:07.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.230+0000 7f1b63000640 1 -- 172.21.15.45:0/3692888919 wait complete. 2024-04-25T10:27:07.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:27:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.394+0000 7f52fb600640 1 Processor -- start 2024-04-25T10:27:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.394+0000 7f52fb600640 1 -- start start 2024-04-25T10:27:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52fb600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f52f40cd9a0 0x7f52f40d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52fb600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52f406bd10 0x7f52f4161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52fb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40d0ba0 0x7f52f4161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52fb600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f406b880 con 0x7f52f406bd10 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52fb600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f406ac90 con 0x7f52f40cd9a0 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52fb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f406af50 con 0x7f52f40d0ba0 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52f406bd10 0x7f52f4161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52f406bd10 0x7f52f4161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40868/0 (socket says 172.21.15.45:40868) 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 -- 172.21.15.45:0/3453348852 learned_addr learned my addr 172.21.15.45:0/3453348852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52f9600640 1 --2- 172.21.15.45:0/3453348852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40d0ba0 0x7f52f4161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52f8c00640 1 --2- 172.21.15.45:0/3453348852 >> v2:172.21.15.117:3300/0 conn(0x7f52f40cd9a0 0x7f52f40d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 -- 172.21.15.45:0/3453348852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40d0ba0 msgr2=0x7f52f4161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 --2- 172.21.15.45:0/3453348852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40d0ba0 0x7f52f4161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 -- 172.21.15.45:0/3453348852 >> v2:172.21.15.117:3300/0 conn(0x7f52f40cd9a0 msgr2=0x7f52f40d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 --2- 172.21.15.45:0/3453348852 >> v2:172.21.15.117:3300/0 conn(0x7f52f40cd9a0 0x7f52f40d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.398+0000 7f52ebe00640 1 -- 172.21.15.45:0/3453348852 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f406b230 con 0x7f52f406bd10 2024-04-25T10:27:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52ebe00640 1 --2- 172.21.15.45:0/3453348852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f406bd10 0x7f52f4161910 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f52f40cf4a0 tx=0x7f52dc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d42bcc1fc064dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52f3e00640 1 -- 172.21.15.45:0/3453348852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc0b5070 con 0x7f52f406bd10 2024-04-25T10:27:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52f3e00640 1 -- 172.21.15.45:0/3453348852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc0a89d0 con 0x7f52f406bd10 2024-04-25T10:27:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52f3e00640 1 -- 172.21.15.45:0/3453348852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc0a8da0 con 0x7f52f406bd10 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 -- 172.21.15.45:0/3453348852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f406bd10 msgr2=0x7f52f4161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 --2- 172.21.15.45:0/3453348852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f406bd10 0x7f52f4161910 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f52f40cf4a0 tx=0x7f52dc01ed10 comp rx=0 tx=0).stop 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 -- 172.21.15.45:0/3453348852 shutdown_connections 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 --2- 172.21.15.45:0/3453348852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40d0ba0 0x7f52f4161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 --2- 172.21.15.45:0/3453348852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f406bd10 0x7f52f4161910 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 --2- 172.21.15.45:0/3453348852 >> v2:172.21.15.117:3300/0 conn(0x7f52f40cd9a0 0x7f52f40d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 -- 172.21.15.45:0/3453348852 >> 172.21.15.45:0/3453348852 conn(0x7f52f40d19a0 msgr2=0x7f52f40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 -- 172.21.15.45:0/3453348852 shutdown_connections 2024-04-25T10:27:07.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 -- 172.21.15.45:0/3453348852 wait complete. 2024-04-25T10:27:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 Processor -- start 2024-04-25T10:27:07.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.402+0000 7f52fb600640 1 -- start start 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52f406bd10 0x7f52f40ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40cd9a0 0x7f52f40b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 0x7f52f40cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4162660 con 0x7f52f40d0ba0 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40cdd80 con 0x7f52f406bd10 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f415d610 con 0x7f52f40cd9a0 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 0x7f52f40cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 0x7f52f40cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40874/0 (socket says 172.21.15.45:40874) 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 -- 172.21.15.45:0/1092029611 learned_addr learned my addr 172.21.15.45:0/1092029611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52ebe00640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40cd9a0 0x7f52f40b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f8c00640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52f406bd10 0x7f52f40ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 -- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40cd9a0 msgr2=0x7f52f40b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40cd9a0 0x7f52f40b6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 -- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52f406bd10 msgr2=0x7f52f40ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52f406bd10 0x7f52f40ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 -- 172.21.15.45:0/1092029611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40ca830 con 0x7f52f40d0ba0 2024-04-25T10:27:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f9600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 0x7f52f40cab10 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009d30 tx=0x7f52ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec018020 con 0x7f52f40d0ba0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec014480 con 0x7f52f40d0ba0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f41f42e0 con 0x7f52f40d0ba0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec014870 con 0x7f52f40d0ba0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f40b4b30 con 0x7f52f40d0ba0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f404ef30 con 0x7f52f40d0ba0 2024-04-25T10:27:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.406+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52ec01b020 con 0x7f52f40d0ba0 2024-04-25T10:27:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.410+0000 7f52f1600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52d0066f60 0x7f52d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.410+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52ec0859e0 con 0x7f52f40d0ba0 2024-04-25T10:27:07.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.410+0000 7f52f8c00640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52d0066f60 0x7f52d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.410+0000 7f52f8c00640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52d0066f60 0x7f52d0069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009ae0 tx=0x7f52e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.414+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52ec022070 con 0x7f52f40d0ba0 2024-04-25T10:27:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.574+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52f40b7920 con 0x7f52f40d0ba0 2024-04-25T10:27:07.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52f1600640 1 -- 172.21.15.45:0/1092029611 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52ec010040 con 0x7f52f40d0ba0 2024-04-25T10:27:07.580 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52d0066f60 msgr2=0x7f52d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52d0066f60 0x7f52d0069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009ae0 tx=0x7f52e4009190 comp rx=0 tx=0).stop 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 msgr2=0x7f52f40cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 0x7f52f40cab10 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009d30 tx=0x7f52ec00c490 comp rx=0 tx=0).stop 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 shutdown_connections 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f52d0066f60 0x7f52d0069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52f40d0ba0 0x7f52f40cab10 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52f40cd9a0 0x7f52f40b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 --2- 172.21.15.45:0/1092029611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52f406bd10 0x7f52f40ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 >> 172.21.15.45:0/1092029611 conn(0x7f52f40d19a0 msgr2=0x7f52f40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:07.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 shutdown_connections 2024-04-25T10:27:07.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.578+0000 7f52fb600640 1 -- 172.21.15.45:0/1092029611 wait complete. 2024-04-25T10:27:07.601 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:07.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.738+0000 7f2909600640 1 Processor -- start 2024-04-25T10:27:07.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.738+0000 7f2909600640 1 -- start start 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2909600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040d0040 0x7f290415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2909600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2904071050 con 0x7f290415b740 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2909600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290415d040 con 0x7f29041550b0 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2909600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040712f0 con 0x7f29040d0040 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2903400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f290415b740 0x7f29040b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36810/0 (socket says 172.21.15.45:36810) 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2903400640 1 -- 172.21.15.45:0/1759921108 learned_addr learned my addr 172.21.15.45:0/1759921108 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:07.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f28fa000640 1 --2- 172.21.15.45:0/1759921108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040d0040 0x7f290415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2902000640 1 -- 172.21.15.45:0/1759921108 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701928826 0 0) 0x7f2904071050 con 0x7f290415b740 2024-04-25T10:27:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2902000640 1 -- 172.21.15.45:0/1759921108 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28e8005740 con 0x7f290415b740 2024-04-25T10:27:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f28fa000640 1 -- 172.21.15.45:0/1759921108 >> v1:172.21.15.117:6789/0 conn(0x7f29041550b0 legacy=0x7f29040cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f28fa000640 1 -- 172.21.15.45:0/1759921108 >> v1:172.21.15.45:6789/0 conn(0x7f290415b740 legacy=0x7f29040b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f2902000640 1 -- 172.21.15.45:0/1759921108 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386015156 0 0) 0x7f290415d040 con 0x7f29041550b0 2024-04-25T10:27:07.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.742+0000 7f28fa000640 1 -- 172.21.15.45:0/1759921108 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040715d0 con 0x7f29040d0040 2024-04-25T10:27:07.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f28fa000640 1 --2- 172.21.15.45:0/1759921108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040d0040 0x7f290415b200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097f0 tx=0x7f28f401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614dab105efe4ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2902000640 1 -- 172.21.15.45:0/1759921108 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40b1070 con 0x7f29040d0040 2024-04-25T10:27:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2902000640 1 -- 172.21.15.45:0/1759921108 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f40ac410 con 0x7f29040d0040 2024-04-25T10:27:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2902000640 1 -- 172.21.15.45:0/1759921108 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40ac7e0 con 0x7f29040d0040 2024-04-25T10:27:07.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- 172.21.15.45:0/1759921108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040d0040 msgr2=0x7f290415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 --2- 172.21.15.45:0/1759921108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040d0040 0x7f290415b200 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097f0 tx=0x7f28f401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- 172.21.15.45:0/1759921108 shutdown_connections 2024-04-25T10:27:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 --2- 172.21.15.45:0/1759921108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040d0040 0x7f290415b200 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- 172.21.15.45:0/1759921108 >> 172.21.15.45:0/1759921108 conn(0x7f2904066b70 msgr2=0x7f2904066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- 172.21.15.45:0/1759921108 shutdown_connections 2024-04-25T10:27:07.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- 172.21.15.45:0/1759921108 wait complete. 2024-04-25T10:27:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 Processor -- start 2024-04-25T10:27:07.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- start start 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 0x7f2904157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29041550b0 0x7f2904159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290415b740 0x7f2904158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290415bcf0 con 0x7f29040d0040 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290415bb70 con 0x7f290415b740 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2909600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290406c090 con 0x7f29041550b0 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2902a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 0x7f2904157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2902a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 0x7f2904157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40886/0 (socket says 172.21.15.45:40886) 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.746+0000 7f2902a00640 1 -- 172.21.15.45:0/1104021422 learned_addr learned my addr 172.21.15.45:0/1104021422 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2903400640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290415b740 0x7f2904158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f28fa000640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29041550b0 0x7f2904159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2902a00640 1 -- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29041550b0 msgr2=0x7f2904159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2902a00640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29041550b0 0x7f2904159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2902a00640 1 -- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290415b740 msgr2=0x7f2904158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2902a00640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290415b740 0x7f2904158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2902a00640 1 -- 172.21.15.45:0/1104021422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29041560c0 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2902a00640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 0x7f2904157f80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00c480 tx=0x7f28ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec005480 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec003600 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0039f0 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2904159ce0 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2904156380 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290404ef30 con 0x7f29040d0040 2024-04-25T10:27:07.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28ec003b90 con 0x7f29040d0040 2024-04-25T10:27:07.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.750+0000 7f28fb400640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28e0066f90 0x7f28e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:07.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.754+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f28ec084b30 con 0x7f29040d0040 2024-04-25T10:27:07.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.754+0000 7f28fa000640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28e0066f90 0x7f28e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:07.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.754+0000 7f28fa000640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28e0066f90 0x7f28e0069450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f28f40ae000 tx=0x7f28f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:07.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.758+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28ec055540 con 0x7f29040d0040 2024-04-25T10:27:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f290406d190 con 0x7f29040d0040 2024-04-25T10:27:07.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f28fb400640 1 -- 172.21.15.45:0/1104021422 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f28ec0595e0 con 0x7f29040d0040 2024-04-25T10:27:07.961 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:27:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:27:07.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28e0066f90 msgr2=0x7f28e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28e0066f90 0x7f28e0069450 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f28f40ae000 tx=0x7f28f4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 msgr2=0x7f2904157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 0x7f2904157f80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00c480 tx=0x7f28ec00c940 comp rx=0 tx=0).stop 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 shutdown_connections 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28e0066f90 0x7f28e0069450 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f290415b740 0x7f2904158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29041550b0 0x7f2904159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 --2- 172.21.15.45:0/1104021422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29040d0040 0x7f2904157f80 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 >> 172.21.15.45:0/1104021422 conn(0x7f2904066b70 msgr2=0x7f29040b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 shutdown_connections 2024-04-25T10:27:07.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:07.958+0000 7f2909600640 1 -- 172.21.15.45:0/1104021422 wait complete. 2024-04-25T10:27:07.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:08.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 Processor -- start 2024-04-25T10:27:08.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 -- start start 2024-04-25T10:27:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63e80d0040 0x7f63e815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e815b740 0x7f63e80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e8071050 con 0x7f63e81550b0 2024-04-25T10:27:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e815d040 con 0x7f63e80d0040 2024-04-25T10:27:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63ed600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e80712f0 con 0x7f63e815b740 2024-04-25T10:27:08.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63e6a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f63e81550b0 0x7f63e80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36814/0 (socket says 172.21.15.45:36814) 2024-04-25T10:27:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63e6a00640 1 -- 172.21.15.45:0/1132428044 learned_addr learned my addr 172.21.15.45:0/1132428044 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.118+0000 7f63de000640 1 --2- 172.21.15.45:0/1132428044 >> v2:172.21.15.45:3300/0 conn(0x7f63e80d0040 0x7f63e815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63e7400640 1 --2- 172.21.15.45:0/1132428044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e815b740 0x7f63e80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63e6000640 1 -- 172.21.15.45:0/1132428044 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263907920 0 0) 0x7f63e8071050 con 0x7f63e81550b0 2024-04-25T10:27:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63e6000640 1 -- 172.21.15.45:0/1132428044 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63cc003530 con 0x7f63e81550b0 2024-04-25T10:27:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63de000640 1 -- 172.21.15.45:0/1132428044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e815b740 msgr2=0x7f63e80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63de000640 1 --2- 172.21.15.45:0/1132428044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e815b740 0x7f63e80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63de000640 1 -- 172.21.15.45:0/1132428044 >> v1:172.21.15.45:6789/0 conn(0x7f63e81550b0 legacy=0x7f63e80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63de000640 1 -- 172.21.15.45:0/1132428044 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e80715d0 con 0x7f63e80d0040 2024-04-25T10:27:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63de000640 1 --2- 172.21.15.45:0/1132428044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e80d0040 0x7f63e815b200 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f63d00097c0 tx=0x7f63d001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c03d21f2da25100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63e6000640 1 -- 172.21.15.45:0/1132428044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d00a7070 con 0x7f63e80d0040 2024-04-25T10:27:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63e6000640 1 -- 172.21.15.45:0/1132428044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d00b0070 con 0x7f63e80d0040 2024-04-25T10:27:08.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.122+0000 7f63e6000640 1 -- 172.21.15.45:0/1132428044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d00ab890 con 0x7f63e80d0040 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- 172.21.15.45:0/1132428044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e80d0040 msgr2=0x7f63e815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 --2- 172.21.15.45:0/1132428044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e80d0040 0x7f63e815b200 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f63d00097c0 tx=0x7f63d001c470 comp rx=0 tx=0).stop 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- 172.21.15.45:0/1132428044 shutdown_connections 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 --2- 172.21.15.45:0/1132428044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e815b740 0x7f63e80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 --2- 172.21.15.45:0/1132428044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e80d0040 0x7f63e815b200 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- 172.21.15.45:0/1132428044 >> 172.21.15.45:0/1132428044 conn(0x7f63e8066b70 msgr2=0x7f63e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- 172.21.15.45:0/1132428044 shutdown_connections 2024-04-25T10:27:08.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- 172.21.15.45:0/1132428044 wait complete. 2024-04-25T10:27:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 Processor -- start 2024-04-25T10:27:08.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- start start 2024-04-25T10:27:08.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63e80d0040 0x7f63e814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e81550b0 0x7f63e8156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 0x7f63e814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e815bcd0 con 0x7f63e815b740 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e815bb50 con 0x7f63e80d0040 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63ed600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e815c030 con 0x7f63e81550b0 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 0x7f63e814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 0x7f63e814fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40904/0 (socket says 172.21.15.45:40904) 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 -- 172.21.15.45:0/582579361 learned_addr learned my addr 172.21.15.45:0/582579361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63de000640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e81550b0 0x7f63e8156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e6a00640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63e80d0040 0x7f63e814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 -- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e81550b0 msgr2=0x7f63e8156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e81550b0 0x7f63e8156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 -- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63e80d0040 msgr2=0x7f63e814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63e80d0040 0x7f63e814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 -- 172.21.15.45:0/582579361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e8158d70 con 0x7f63e815b740 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e6a00640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63e80d0040 0x7f63e814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.126+0000 7f63e7400640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 0x7f63e814fa10 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009d30 tx=0x7f63e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0018020 con 0x7f63e815b740 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e0014480 con 0x7f63e815b740 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e8159050 con 0x7f63e815b740 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0014870 con 0x7f63e815b740 2024-04-25T10:27:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e8159610 con 0x7f63e815b740 2024-04-25T10:27:08.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e804ef30 con 0x7f63e815b740 2024-04-25T10:27:08.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63e001b020 con 0x7f63e815b740 2024-04-25T10:27:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63df400640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63c4066f60 0x7f63c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.130+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f63e0085af0 con 0x7f63e815b740 2024-04-25T10:27:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.134+0000 7f63e6a00640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63c4066f60 0x7f63c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.134+0000 7f63e6a00640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63c4066f60 0x7f63c4069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005890 tx=0x7f63d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.134+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63e0022070 con 0x7f63e815b740 2024-04-25T10:27:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.298+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63e80cc220 con 0x7f63e815b740 2024-04-25T10:27:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.298+0000 7f63df400640 1 -- 172.21.15.45:0/582579361 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16769 (secure 0 0 0) 0x7f63e0010040 con 0x7f63e815b740 2024-04-25T10:27:08.303 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:08.303 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:43.714023+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.21","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63c4066f60 msgr2=0x7f63c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63c4066f60 0x7f63c4069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005890 tx=0x7f63d8009040 comp rx=0 tx=0).stop 2024-04-25T10:27:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 msgr2=0x7f63e814fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 0x7f63e814fa10 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009d30 tx=0x7f63e000c490 comp rx=0 tx=0).stop 2024-04-25T10:27:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 shutdown_connections 2024-04-25T10:27:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f63c4066f60 0x7f63c4069420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63e815b740 0x7f63e814fa10 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63e81550b0 0x7f63e8156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 --2- 172.21.15.45:0/582579361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63e80d0040 0x7f63e814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 >> 172.21.15.45:0/582579361 conn(0x7f63e8066b70 msgr2=0x7f63e80b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 shutdown_connections 2024-04-25T10:27:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.302+0000 7f63ed600640 1 -- 172.21.15.45:0/582579361 wait complete. 2024-04-25T10:27:08.324 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:27:08.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd529600640 1 Processor -- start 2024-04-25T10:27:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd529600640 1 -- start start 2024-04-25T10:27:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd529600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd524066c00 0x7fd52415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd529600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240b5c20 con 0x7fd5240b5560 2024-04-25T10:27:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd529600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240af6e0 con 0x7fd524066fe0 2024-04-25T10:27:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd529600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240b59d0 con 0x7fd524066c00 2024-04-25T10:27:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.462+0000 7fd523400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd5240b5560 0x7fd524157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36820/0 (socket says 172.21.15.45:36820) 2024-04-25T10:27:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd523400640 1 -- 172.21.15.45:0/769433299 learned_addr learned my addr 172.21.15.45:0/769433299 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:08.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522a00640 1 --2- 172.21.15.45:0/769433299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd524066c00 0x7fd52415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939883243 0 0) 0x7fd5240b5c20 con 0x7fd5240b5560 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd508003530 con 0x7fd5240b5560 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110777391 0 0) 0x7fd5240af6e0 con 0x7fd524066fe0 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5240b5c20 con 0x7fd524066fe0 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522a00640 1 -- 172.21.15.45:0/769433299 >> v1:172.21.15.117:6789/0 conn(0x7fd524066fe0 legacy=0x7fd52406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522a00640 1 -- 172.21.15.45:0/769433299 >> v1:172.21.15.45:6789/0 conn(0x7fd5240b5560 legacy=0x7fd524157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522a00640 1 -- 172.21.15.45:0/769433299 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240af9c0 con 0x7fd524066c00 2024-04-25T10:27:08.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522a00640 1 --2- 172.21.15.45:0/769433299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd524066c00 0x7fd52415e110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b5b50 tx=0x7fd51401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a63f9be29b4c46b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140aa610 con 0x7fd524066c00 2024-04-25T10:27:08.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5140af070 con 0x7fd524066c00 2024-04-25T10:27:08.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.466+0000 7fd522000640 1 -- 172.21.15.45:0/769433299 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140aa9d0 con 0x7fd524066c00 2024-04-25T10:27:08.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- 172.21.15.45:0/769433299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd524066c00 msgr2=0x7fd52415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 --2- 172.21.15.45:0/769433299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd524066c00 0x7fd52415e110 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b5b50 tx=0x7fd51401c490 comp rx=0 tx=0).stop 2024-04-25T10:27:08.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- 172.21.15.45:0/769433299 shutdown_connections 2024-04-25T10:27:08.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 --2- 172.21.15.45:0/769433299 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd524066c00 0x7fd52415e110 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- 172.21.15.45:0/769433299 >> 172.21.15.45:0/769433299 conn(0x7fd5240b24f0 msgr2=0x7fd5240b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- 172.21.15.45:0/769433299 shutdown_connections 2024-04-25T10:27:08.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- 172.21.15.45:0/769433299 wait complete. 2024-04-25T10:27:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 Processor -- start 2024-04-25T10:27:08.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- start start 2024-04-25T10:27:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 0x7fd52415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd524066fe0 0x7fd52414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5240b5560 0x7fd5241501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52416bb00 con 0x7fd524066c00 2024-04-25T10:27:08.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240af2f0 con 0x7fd524066fe0 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd529600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52416bc80 con 0x7fd5240b5560 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd522a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 0x7fd52415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd522a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 0x7fd52415c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40918/0 (socket says 172.21.15.45:40918) 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd522a00640 1 -- 172.21.15.45:0/2143262641 learned_addr learned my addr 172.21.15.45:0/2143262641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd51be00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd524066fe0 0x7fd52414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.470+0000 7fd523400640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5240b5560 0x7fd5241501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd522a00640 1 -- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5240b5560 msgr2=0x7fd5241501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd522a00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5240b5560 0x7fd5241501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd522a00640 1 -- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd524066fe0 msgr2=0x7fd52414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd522a00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd524066fe0 0x7fd52414f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd522a00640 1 -- 172.21.15.45:0/2143262641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52415ce00 con 0x7fd524066c00 2024-04-25T10:27:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd522a00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 0x7fd52415c700 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd5140ac040 tx=0x7fd51401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140be410 con 0x7fd524066c00 2024-04-25T10:27:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5140be5d0 con 0x7fd524066c00 2024-04-25T10:27:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140be9c0 con 0x7fd524066c00 2024-04-25T10:27:08.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52415d0e0 con 0x7fd524066c00 2024-04-25T10:27:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52415d6a0 con 0x7fd524066c00 2024-04-25T10:27:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52404ef30 con 0x7fd524066c00 2024-04-25T10:27:08.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5140bc020 con 0x7fd524066c00 2024-04-25T10:27:08.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51aa00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd500066f60 0x7fd500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd51411f180 con 0x7fd524066c00 2024-04-25T10:27:08.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.474+0000 7fd51be00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd500066f60 0x7fd500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.478+0000 7fd51be00640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd500066f60 0x7fd500069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd50c007fd0 tx=0x7fd50c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.478+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5140c5030 con 0x7fd524066c00 2024-04-25T10:27:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5240720c0 con 0x7fd524066c00 2024-04-25T10:27:08.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd51aa00640 1 -- 172.21.15.45:0/2143262641 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fd5140a7070 con 0x7fd524066c00 2024-04-25T10:27:08.681 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:08.681 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:27:08.681 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd500066f60 msgr2=0x7fd500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd500066f60 0x7fd500069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd50c007fd0 tx=0x7fd50c009040 comp rx=0 tx=0).stop 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 msgr2=0x7fd52415c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 0x7fd52415c700 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd5140ac040 tx=0x7fd51401acd0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 shutdown_connections 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd500066f60 0x7fd500069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5240b5560 0x7fd5241501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd524066fe0 0x7fd52414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 --2- 172.21.15.45:0/2143262641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd524066c00 0x7fd52415c700 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 >> 172.21.15.45:0/2143262641 conn(0x7fd5240b24f0 msgr2=0x7fd52406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 shutdown_connections 2024-04-25T10:27:08.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.678+0000 7fd529600640 1 -- 172.21.15.45:0/2143262641 wait complete. 2024-04-25T10:27:08.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:08.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.838+0000 7fa338800640 1 Processor -- start 2024-04-25T10:27:08.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.838+0000 7fa338800640 1 -- start start 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa338800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa3300b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa338800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3300cfd90 0x7fa3300ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa338800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33015f3b0 con 0x7fa3300cd110 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa338800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33015ef20 con 0x7fa3300cfd90 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa338800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300ca810 con 0x7fa3300b0f70 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3300cfd90 0x7fa3300ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa336800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa3300cd110 0x7fa3300cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36828/0 (socket says 172.21.15.45:36828) 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3300cfd90 0x7fa3300ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40930/0 (socket says 172.21.15.45:40930) 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa336800640 1 -- 172.21.15.45:0/1665215902 learned_addr learned my addr 172.21.15.45:0/1665215902 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:08.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335e00640 1 --2- 172.21.15.45:0/1665215902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa3300b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335400640 1 -- 172.21.15.45:0/1665215902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 msgr2=0x7fa3300b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:08.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335400640 1 --2- 172.21.15.45:0/1665215902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa3300b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335400640 1 -- 172.21.15.45:0/1665215902 >> v1:172.21.15.45:6789/0 conn(0x7fa3300cd110 legacy=0x7fa3300cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa335400640 1 -- 172.21.15.45:0/1665215902 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300b1890 con 0x7fa3300cfd90 2024-04-25T10:27:08.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.842+0000 7fa334a00640 1 -- 172.21.15.45:0/1665215902 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949470159 0 0) 0x7fa33015f3b0 con 0x7fa3300cd110 2024-04-25T10:27:08.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa335400640 1 --2- 172.21.15.45:0/1665215902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cfd90 0x7fa3300ccbd0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097c0 tx=0x7fa32001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b081399727259d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa334a00640 1 -- 172.21.15.45:0/1665215902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b0070 con 0x7fa3300cfd90 2024-04-25T10:27:08.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa334a00640 1 -- 172.21.15.45:0/1665215902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200ab600 con 0x7fa3300cfd90 2024-04-25T10:27:08.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa334a00640 1 -- 172.21.15.45:0/1665215902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200ab9d0 con 0x7fa3300cfd90 2024-04-25T10:27:08.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 -- 172.21.15.45:0/1665215902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cfd90 msgr2=0x7fa3300ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 --2- 172.21.15.45:0/1665215902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cfd90 0x7fa3300ccbd0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097c0 tx=0x7fa32001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 -- 172.21.15.45:0/1665215902 shutdown_connections 2024-04-25T10:27:08.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 --2- 172.21.15.45:0/1665215902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cfd90 0x7fa3300ccbd0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 --2- 172.21.15.45:0/1665215902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa3300b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 -- 172.21.15.45:0/1665215902 >> 172.21.15.45:0/1665215902 conn(0x7fa3300b1d50 msgr2=0x7fa3300b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:08.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 -- 172.21.15.45:0/1665215902 shutdown_connections 2024-04-25T10:27:08.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 -- 172.21.15.45:0/1665215902 wait complete. 2024-04-25T10:27:08.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 Processor -- start 2024-04-25T10:27:08.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.846+0000 7fa338800640 1 -- start start 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa33009d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 0x7fa3300ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3300cfd90 0x7fa3300aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301619d0 con 0x7fa3300cd110 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3300cd540 con 0x7fa3300cfd90 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa330161b50 con 0x7fa3300b0f70 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 0x7fa3300ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 0x7fa3300ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40940/0 (socket says 172.21.15.45:40940) 2024-04-25T10:27:08.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 -- 172.21.15.45:0/1382533303 learned_addr learned my addr 172.21.15.45:0/1382533303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335e00640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa33009d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 -- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 msgr2=0x7fa33009d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa336800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3300cfd90 0x7fa3300aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa33009d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 -- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3300cfd90 msgr2=0x7fa3300aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3300cfd90 0x7fa3300aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 -- 172.21.15.45:0/1382533303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300af600 con 0x7fa3300cd110 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335e00640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa33009d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa336800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3300cfd90 0x7fa3300aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa335400640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 0x7fa3300ae7a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa32001a0d0 tx=0x7fa3300cc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b8020 con 0x7fa3300cd110 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200b4410 con 0x7fa3300cd110 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b0040 con 0x7fa3300cd110 2024-04-25T10:27:08.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3300af8e0 con 0x7fa3300cd110 2024-04-25T10:27:08.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33009dd00 con 0x7fa3300cd110 2024-04-25T10:27:08.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33004ef30 con 0x7fa3300cd110 2024-04-25T10:27:08.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.850+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3200bb020 con 0x7fa3300cd110 2024-04-25T10:27:08.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.854+0000 7fa32a000640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa314066f60 0x7fa314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:08.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.854+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa32011ec50 con 0x7fa3300cd110 2024-04-25T10:27:08.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.854+0000 7fa335e00640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa314066f60 0x7fa314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:08.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.854+0000 7fa335e00640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa314066f60 0x7fa314069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa3300b4d20 tx=0x7fa324008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:08.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:08.858+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3200c2030 con 0x7fa3300cd110 2024-04-25T10:27:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.054+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3300cda20 con 0x7fa3300cd110 2024-04-25T10:27:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.054+0000 7fa32a000640 1 -- 172.21.15.45:0/1382533303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa320016090 con 0x7fa3300cd110 2024-04-25T10:27:09.059 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:26:13.356392+0000","modified":"2024-04-25T10:26:16.392149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6834/1838662791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1838662791},{"type":"v1","addr":"172.21.15.45:6834","nonce":1838662791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:27:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 82 2024-04-25T10:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa314066f60 msgr2=0x7fa314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa314066f60 0x7fa314069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa3300b4d20 tx=0x7fa324008040 comp rx=0 tx=0).stop 2024-04-25T10:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 msgr2=0x7fa3300ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 0x7fa3300ae7a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa32001a0d0 tx=0x7fa3300cc940 comp rx=0 tx=0).stop 2024-04-25T10:27:09.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 shutdown_connections 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa314066f60 0x7fa314069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3300cfd90 0x7fa3300aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3300cd110 0x7fa3300ae7a0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 --2- 172.21.15.45:0/1382533303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3300b0f70 0x7fa33009d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 >> 172.21.15.45:0/1382533303 conn(0x7fa3300b1d50 msgr2=0x7fa3300b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 shutdown_connections 2024-04-25T10:27:09.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.058+0000 7fa338800640 1 -- 172.21.15.45:0/1382533303 wait complete. 2024-04-25T10:27:09.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.218+0000 7f457ba00640 1 Processor -- start 2024-04-25T10:27:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.218+0000 7f457ba00640 1 -- start start 2024-04-25T10:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f457ba00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45740d0350 0x7f45740d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f457ba00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45740ccaa0 con 0x7f457415df60 2024-04-25T10:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f457ba00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45740ccd80 con 0x7f457406aea0 2024-04-25T10:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f457ba00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457415e340 con 0x7f45740d0350 2024-04-25T10:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f457415df60 0x7f45740cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36832/0 (socket says 172.21.15.45:36832) 2024-04-25T10:27:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579000640 1 -- 172.21.15.45:0/1873803603 learned_addr learned my addr 172.21.15.45:0/1873803603 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f456be00640 1 --2- 172.21.15.45:0/1873803603 >> v2:172.21.15.117:3300/0 conn(0x7f45740d0350 0x7f45740d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866896945 0 0) 0x7f45740ccaa0 con 0x7f457415df60 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4558003530 con 0x7f457415df60 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2156130024 0 0) 0x7f4558003530 con 0x7f457415df60 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 >> v2:172.21.15.117:3300/0 conn(0x7f45740d0350 msgr2=0x7f45740d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 --2- 172.21.15.45:0/1873803603 >> v2:172.21.15.117:3300/0 conn(0x7f45740d0350 0x7f45740d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 >> v1:172.21.15.117:6789/0 conn(0x7f457406aea0 legacy=0x7f457406b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45740cd060 con 0x7f457415df60 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f4564001680 con 0x7f457415df60 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 >> v1:172.21.15.45:6789/0 conn(0x7f457415df60 legacy=0x7f45740cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45580039c0 0x7f4558003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45580043c0 0x7f4558004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4558005130 0x7f455800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455800c040 con 0x7f4558005130 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455800c1f0 con 0x7f45580043c0 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455800c3f0 con 0x7f45580039c0 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579a00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4558005130 0x7f455800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579000640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45580039c0 0x7f4558003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f456be00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45580043c0 0x7f4558004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579a00640 1 -- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45580039c0 msgr2=0x7f4558003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579a00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45580039c0 0x7f4558003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579a00640 1 -- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45580043c0 msgr2=0x7f4558004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579a00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45580043c0 0x7f4558004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f4579a00640 1 -- 172.21.15.45:0/1873803603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558003530 con 0x7f4558005130 2024-04-25T10:27:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.222+0000 7f456be00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45580043c0 0x7f4558004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f4579a00640 1 -- 172.21.15.45:0/1873803603 reap_dead start 2024-04-25T10:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f4579a00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4558005130 0x7f455800b9d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f456c001f60 tx=0x7f456c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10487fa1ce451d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c001b60 con 0x7f4558005130 2024-04-25T10:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457406c1d0 con 0x7f4558005130 2024-04-25T10:27:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f4573e00640 1 -- 172.21.15.45:0/1873803603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c003da0 con 0x7f4558005130 2024-04-25T10:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 -- 172.21.15.45:0/1873803603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4558005130 msgr2=0x7f455800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4558005130 0x7f455800b9d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f456c001f60 tx=0x7f456c01c880 comp rx=0 tx=0).stop 2024-04-25T10:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 -- 172.21.15.45:0/1873803603 shutdown_connections 2024-04-25T10:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 --2- 172.21.15.45:0/1873803603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4558005130 0x7f455800b9d0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 -- 172.21.15.45:0/1873803603 >> 172.21.15.45:0/1873803603 conn(0x7f45740d1910 msgr2=0x7f45740d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 -- 172.21.15.45:0/1873803603 shutdown_connections 2024-04-25T10:27:09.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 -- 172.21.15.45:0/1873803603 wait complete. 2024-04-25T10:27:09.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 Processor -- start 2024-04-25T10:27:09.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.226+0000 7f457ba00640 1 -- start start 2024-04-25T10:27:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4558005130 0x7f457414b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 0x7f45740c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45740c8f20 0x7f45740c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740d0730 con 0x7f45740c79e0 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740d1080 con 0x7f45740c8f20 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45740d08b0 con 0x7f4558005130 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 0x7f45740c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 0x7f45740c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40964/0 (socket says 172.21.15.45:40964) 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 -- 172.21.15.45:0/3641287683 learned_addr learned my addr 172.21.15.45:0/3641287683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 -- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4558005130 msgr2=0x7f457414b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4558005130 0x7f457414b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 -- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45740c8f20 msgr2=0x7f45740c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45740c8f20 0x7f45740c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 -- 172.21.15.45:0/3641287683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45740c7ee0 con 0x7f45740c79e0 2024-04-25T10:27:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f456be00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 0x7f45740c89e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f455c009160 tx=0x7f455c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c004220 con 0x7f45740c79e0 2024-04-25T10:27:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c0043c0 con 0x7f45740c79e0 2024-04-25T10:27:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c015890 con 0x7f45740c79e0 2024-04-25T10:27:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45740ca8b0 con 0x7f45740c79e0 2024-04-25T10:27:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45740cae70 con 0x7f45740c79e0 2024-04-25T10:27:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457404ef30 con 0x7f45740c79e0 2024-04-25T10:27:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.230+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f455c015a30 con 0x7f45740c79e0 2024-04-25T10:27:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.234+0000 7f4571600640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4550066fe0 0x7f45500694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.234+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f455c0855d0 con 0x7f45740c79e0 2024-04-25T10:27:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.234+0000 7f4579000640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4550066fe0 0x7f45500694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.234+0000 7f4579000640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4550066fe0 0x7f45500694a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4564000f70 tx=0x7f4564000ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.238+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f455c019030 con 0x7f45740c79e0 2024-04-25T10:27:09.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.398+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45740cdb40 con 0x7f45740c79e0 2024-04-25T10:27:09.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f4571600640 1 -- 172.21.15.45:0/3641287683 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16769 (secure 0 0 0) 0x7f455c056580 con 0x7f45740c79e0 2024-04-25T10:27:09.405 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:09.405 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:26:43.714023+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:26:11.768328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T10:26:12.345255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.21","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4550066fe0 msgr2=0x7f45500694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4550066fe0 0x7f45500694a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4564000f70 tx=0x7f4564000ee0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 msgr2=0x7f45740c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 0x7f45740c89e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f455c009160 tx=0x7f455c011a60 comp rx=0 tx=0).stop 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 shutdown_connections 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4550066fe0 0x7f45500694a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45740c8f20 0x7f45740c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45740c79e0 0x7f45740c89e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 --2- 172.21.15.45:0/3641287683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4558005130 0x7f457414b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 >> 172.21.15.45:0/3641287683 conn(0x7f45740d1910 msgr2=0x7f457406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:09.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 shutdown_connections 2024-04-25T10:27:09.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.402+0000 7f457ba00640 1 -- 172.21.15.45:0/3641287683 wait complete. 2024-04-25T10:27:09.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:27:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 Processor -- start 2024-04-25T10:27:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 -- start start 2024-04-25T10:27:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31201550b0 0x7f31200cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31200cfe70 0x7f312015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f312015b740 0x7f312006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3120071050 con 0x7f312015b740 2024-04-25T10:27:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31200712f0 con 0x7f31200cfe70 2024-04-25T10:27:09.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3127600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31200715d0 con 0x7f31201550b0 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f312015b740 0x7f312006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f312015b740 0x7f312006d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40972/0 (socket says 172.21.15.45:40972) 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 -- 172.21.15.45:0/4081934015 learned_addr learned my addr 172.21.15.45:0/4081934015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3124c00640 1 --2- 172.21.15.45:0/4081934015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31201550b0 0x7f31200cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f311fe00640 1 --2- 172.21.15.45:0/4081934015 >> v2:172.21.15.117:3300/0 conn(0x7f31200cfe70 0x7f312015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 -- 172.21.15.45:0/4081934015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31201550b0 msgr2=0x7f31200cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 --2- 172.21.15.45:0/4081934015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31201550b0 0x7f31200cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 -- 172.21.15.45:0/4081934015 >> v2:172.21.15.117:3300/0 conn(0x7f31200cfe70 msgr2=0x7f312015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 --2- 172.21.15.45:0/4081934015 >> v2:172.21.15.117:3300/0 conn(0x7f31200cfe70 0x7f312015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f3125600640 1 -- 172.21.15.45:0/4081934015 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31200718b0 con 0x7f312015b740 2024-04-25T10:27:09.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.566+0000 7f311fe00640 1 --2- 172.21.15.45:0/4081934015 >> v2:172.21.15.117:3300/0 conn(0x7f31200cfe70 0x7f312015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:09.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3125600640 1 --2- 172.21.15.45:0/4081934015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f312015b740 0x7f312006d1a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f312006e290 tx=0x7f311801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae9fdb6447d9b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f311f400640 1 -- 172.21.15.45:0/4081934015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180ac020 con 0x7f312015b740 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f311f400640 1 -- 172.21.15.45:0/4081934015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31180a6440 con 0x7f312015b740 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f311f400640 1 -- 172.21.15.45:0/4081934015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180a6810 con 0x7f312015b740 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 -- 172.21.15.45:0/4081934015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f312015b740 msgr2=0x7f312006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 --2- 172.21.15.45:0/4081934015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f312015b740 0x7f312006d1a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f312006e290 tx=0x7f311801c730 comp rx=0 tx=0).stop 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 -- 172.21.15.45:0/4081934015 shutdown_connections 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 --2- 172.21.15.45:0/4081934015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f312015b740 0x7f312006d1a0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 --2- 172.21.15.45:0/4081934015 >> v2:172.21.15.117:3300/0 conn(0x7f31200cfe70 0x7f312015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 --2- 172.21.15.45:0/4081934015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31201550b0 0x7f31200cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 -- 172.21.15.45:0/4081934015 >> 172.21.15.45:0/4081934015 conn(0x7f3120066b70 msgr2=0x7f3120066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 -- 172.21.15.45:0/4081934015 shutdown_connections 2024-04-25T10:27:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.570+0000 7f3127600640 1 -- 172.21.15.45:0/4081934015 wait complete. 2024-04-25T10:27:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 Processor -- start 2024-04-25T10:27:09.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 -- start start 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31200cfe70 0x7f312014c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 0x7f312014de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312015b740 0x7f312014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200b33e0 con 0x7f31201550b0 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200b3260 con 0x7f312015b740 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31200b3560 con 0x7f31200cfe70 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 0x7f312014de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 0x7f312014de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40974/0 (socket says 172.21.15.45:40974) 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 -- 172.21.15.45:0/3206741643 learned_addr learned my addr 172.21.15.45:0/3206741643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3124c00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31200cfe70 0x7f312014c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3125600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312015b740 0x7f312014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 -- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31200cfe70 msgr2=0x7f312014c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31200cfe70 0x7f312014c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 -- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312015b740 msgr2=0x7f312014c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312015b740 0x7f312014c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 -- 172.21.15.45:0/3206741643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31200ae7a0 con 0x7f31201550b0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311fe00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 0x7f312014de50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e810 tx=0x7f310c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c017070 con 0x7f31201550b0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c00c8d0 con 0x7f31201550b0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31200aea80 con 0x7f31201550b0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00f040 con 0x7f31201550b0 2024-04-25T10:27:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.574+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3120150c20 con 0x7f31201550b0 2024-04-25T10:27:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.578+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312004ef30 con 0x7f31201550b0 2024-04-25T10:27:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.578+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f310c01a070 con 0x7f31201550b0 2024-04-25T10:27:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.578+0000 7f311cc00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3100066f90 0x7f3100069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.578+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f310c084bc0 con 0x7f31201550b0 2024-04-25T10:27:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.578+0000 7f3124c00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3100066f90 0x7f3100069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.578+0000 7f3124c00640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3100066f90 0x7f3100069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3114009ae0 tx=0x7f3114009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.582+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f310c055530 con 0x7f31201550b0 2024-04-25T10:27:09.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.782+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f312006d470 con 0x7f31201550b0 2024-04-25T10:27:09.787 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:27:09.782+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:27:09.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.786+0000 7f311cc00640 1 -- 172.21.15.45:0/3206741643 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f310c0595d0 con 0x7f31201550b0 2024-04-25T10:27:09.789 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3100066f90 msgr2=0x7f3100069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3100066f90 0x7f3100069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3114009ae0 tx=0x7f3114009190 comp rx=0 tx=0).stop 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 msgr2=0x7f312014de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 0x7f312014de50 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e810 tx=0x7f310c00ecd0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 shutdown_connections 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3100066f90 0x7f3100069450 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f312015b740 0x7f312014c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31201550b0 0x7f312014de50 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 --2- 172.21.15.45:0/3206741643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31200cfe70 0x7f312014c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 >> 172.21.15.45:0/3206741643 conn(0x7f3120066b70 msgr2=0x7f3120158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 shutdown_connections 2024-04-25T10:27:09.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.790+0000 7f3127600640 1 -- 172.21.15.45:0/3206741643 wait complete. 2024-04-25T10:27:09.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:27:09.859 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:27:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4377200640 1 Processor -- start 2024-04-25T10:27:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4377200640 1 -- start start 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4377200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b0b60 0x7f437806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4377200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780b4960 con 0x7f43780b6390 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4377200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780b4c00 con 0x7f437815dca0 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4377200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437806cd20 con 0x7f43780b0b60 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4375e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f43780b6390 0x7f43780b6770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36860/0 (socket says 172.21.15.45:36860) 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.958+0000 7f4375e00640 1 -- 172.21.15.45:0/817093847 learned_addr learned my addr 172.21.15.45:0/817093847 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f436fe00640 1 --2- 172.21.15.45:0/817093847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b0b60 0x7f437806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961886206 0 0) 0x7f43780b4960 con 0x7f43780b6390 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f435c003530 con 0x7f43780b6390 2024-04-25T10:27:09.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33253731 0 0) 0x7f43780b4c00 con 0x7f437815dca0 2024-04-25T10:27:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43780b4960 con 0x7f437815dca0 2024-04-25T10:27:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f436fe00640 1 -- 172.21.15.45:0/817093847 >> v1:172.21.15.117:6789/0 conn(0x7f437815dca0 legacy=0x7f437806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f436fe00640 1 -- 172.21.15.45:0/817093847 >> v1:172.21.15.45:6789/0 conn(0x7f43780b6390 legacy=0x7f43780b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1118968412 0 0) 0x7f435c003530 con 0x7f43780b6390 2024-04-25T10:27:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f436fe00640 1 -- 172.21.15.45:0/817093847 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437815e2f0 con 0x7f43780b0b60 2024-04-25T10:27:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f436fe00640 1 --2- 172.21.15.45:0/817093847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b0b60 0x7f437806c740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43600097f0 tx=0x7f436001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a07c0213db97b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43600b0070 con 0x7f43780b0b60 2024-04-25T10:27:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43600ab400 con 0x7f43780b0b60 2024-04-25T10:27:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.962+0000 7f4375400640 1 -- 172.21.15.45:0/817093847 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368001680 con 0x7f43780b0b60 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- 172.21.15.45:0/817093847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b0b60 msgr2=0x7f437806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 --2- 172.21.15.45:0/817093847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b0b60 0x7f437806c740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43600097f0 tx=0x7f436001c400 comp rx=0 tx=0).stop 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- 172.21.15.45:0/817093847 shutdown_connections 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 --2- 172.21.15.45:0/817093847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b0b60 0x7f437806c740 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- 172.21.15.45:0/817093847 >> 172.21.15.45:0/817093847 conn(0x7f4378072480 msgr2=0x7f4378072890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- 172.21.15.45:0/817093847 shutdown_connections 2024-04-25T10:27:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- 172.21.15.45:0/817093847 wait complete. 2024-04-25T10:27:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 Processor -- start 2024-04-25T10:27:09.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- start start 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43780b0b60 0x7f43781454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b6390 0x7f43780b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 0x7f437814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437816c0a0 con 0x7f437815dca0 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437806b3a0 con 0x7f43780b0b60 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4377200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437816c220 con 0x7f43780b6390 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 0x7f437814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 0x7f437814aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40992/0 (socket says 172.21.15.45:40992) 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 -- 172.21.15.45:0/1654440446 learned_addr learned my addr 172.21.15.45:0/1654440446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f436fe00640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b6390 0x7f43780b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4375e00640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43780b0b60 0x7f43781454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 -- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b6390 msgr2=0x7f43780b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b6390 0x7f43780b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 -- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43780b0b60 msgr2=0x7f43781454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43780b0b60 0x7f43781454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4376800640 1 -- 172.21.15.45:0/1654440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780b1d80 con 0x7f437815dca0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.966+0000 7f4375e00640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43780b0b60 0x7f43781454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f4376800640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 0x7f437814aa60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f437000c790 tx=0x7f437000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370003b80 con 0x7f437815dca0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370003d20 con 0x7f437815dca0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000aa50 con 0x7f437815dca0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43780b2060 con 0x7f437815dca0 2024-04-25T10:27:09.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43780b2620 con 0x7f437815dca0 2024-04-25T10:27:09.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378075120 con 0x7f437815dca0 2024-04-25T10:27:09.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f437000abf0 con 0x7f437815dca0 2024-04-25T10:27:09.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f436e000640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4354066f60 0x7f4354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:09.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4370085020 con 0x7f437815dca0 2024-04-25T10:27:09.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.970+0000 7f4375e00640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4354066f60 0x7f4354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:09.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.974+0000 7f4375e00640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4354066f60 0x7f4354069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f43680011f0 tx=0x7f4368009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:09.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:09.974+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4370055fa0 con 0x7f437815dca0 2024-04-25T10:27:10.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.178+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f43780b6ca0 con 0x7f437815dca0 2024-04-25T10:27:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.822+0000 7f436e000640 1 -- 172.21.15.45:0/1654440446 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43700558a0 con 0x7f437815dca0 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4354066f60 msgr2=0x7f4354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4354066f60 0x7f4354069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f43680011f0 tx=0x7f4368009040 comp rx=0 tx=0).stop 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 msgr2=0x7f437814aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 0x7f437814aa60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f437000c790 tx=0x7f437000cc50 comp rx=0 tx=0).stop 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 shutdown_connections 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4354066f60 0x7f4354069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f437815dca0 0x7f437814aa60 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43780b6390 0x7f43780b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 --2- 172.21.15.45:0/1654440446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43780b0b60 0x7f43781454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 >> 172.21.15.45:0/1654440446 conn(0x7f4378072480 msgr2=0x7f43780d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 shutdown_connections 2024-04-25T10:27:10.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:10.826+0000 7f4377200640 1 -- 172.21.15.45:0/1654440446 wait complete. 2024-04-25T10:27:10.853 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:27:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28da000640 1 Processor -- start 2024-04-25T10:27:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28da000640 1 -- start start 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28da000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40d00f0 0x7f28d40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28da000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d4071f80 con 0x7f28d40cf040 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28da000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d415b120 con 0x7f28d415bc00 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28da000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d4072260 con 0x7f28d40d00f0 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d3400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28d40cf040 0x7f28d40cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36866/0 (socket says 172.21.15.45:36866) 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d3400640 1 -- 172.21.15.45:0/1187160256 learned_addr learned my addr 172.21.15.45:0/1187160256 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28cbe00640 1 --2- 172.21.15.45:0/1187160256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40d00f0 0x7f28d40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504584465 0 0) 0x7f28d4071f80 con 0x7f28d40cf040 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b8003530 con 0x7f28d40cf040 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495743069 0 0) 0x7f28d415b120 con 0x7f28d415bc00 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d4071f80 con 0x7f28d415bc00 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2925041429 0 0) 0x7f28b8003530 con 0x7f28d40cf040 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28cbe00640 1 -- 172.21.15.45:0/1187160256 >> v1:172.21.15.117:6789/0 conn(0x7f28d415bc00 legacy=0x7f28d40ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28cbe00640 1 -- 172.21.15.45:0/1187160256 >> v1:172.21.15.45:6789/0 conn(0x7f28d40cf040 legacy=0x7f28d40cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.002+0000 7f28cbe00640 1 -- 172.21.15.45:0/1187160256 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d415b400 con 0x7f28d40d00f0 2024-04-25T10:27:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28cbe00640 1 --2- 172.21.15.45:0/1187160256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40d00f0 0x7f28d40713a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0097f0 tx=0x7f28bc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a5ff43ab9bf9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0b0070 con 0x7f28d40d00f0 2024-04-25T10:27:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0ab400 con 0x7f28d40d00f0 2024-04-25T10:27:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28d2a00640 1 -- 172.21.15.45:0/1187160256 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4001680 con 0x7f28d40d00f0 2024-04-25T10:27:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 -- 172.21.15.45:0/1187160256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40d00f0 msgr2=0x7f28d40713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 --2- 172.21.15.45:0/1187160256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40d00f0 0x7f28d40713a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0097f0 tx=0x7f28bc01c400 comp rx=0 tx=0).stop 2024-04-25T10:27:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 -- 172.21.15.45:0/1187160256 shutdown_connections 2024-04-25T10:27:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 --2- 172.21.15.45:0/1187160256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40d00f0 0x7f28d40713a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 -- 172.21.15.45:0/1187160256 >> 172.21.15.45:0/1187160256 conn(0x7f28d4066b70 msgr2=0x7f28d4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 -- 172.21.15.45:0/1187160256 shutdown_connections 2024-04-25T10:27:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.006+0000 7f28da000640 1 -- 172.21.15.45:0/1187160256 wait complete. 2024-04-25T10:27:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 Processor -- start 2024-04-25T10:27:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- start start 2024-04-25T10:27:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40cf040 0x7f28d416a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28d40d00f0 0x7f28d4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 0x7f28d414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d406c3b0 con 0x7f28d415bc00 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40cf630 con 0x7f28d40d00f0 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d406c530 con 0x7f28d40cf040 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 0x7f28d414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 0x7f28d414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41008/0 (socket says 172.21.15.45:41008) 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 -- 172.21.15.45:0/61567295 learned_addr learned my addr 172.21.15.45:0/61567295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28cbe00640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28d40d00f0 0x7f28d4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3400640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40cf040 0x7f28d416a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 -- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40cf040 msgr2=0x7f28d416a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40cf040 0x7f28d416a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 -- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28d40d00f0 msgr2=0x7f28d4152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28d40d00f0 0x7f28d4152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 -- 172.21.15.45:0/61567295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d40ae9c0 con 0x7f28d415bc00 2024-04-25T10:27:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28cbe00640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28d40d00f0 0x7f28d4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28d3e00640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 0x7f28d414b530 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00c7a0 tx=0x7f28cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc003b80 con 0x7f28d415bc00 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc003d20 con 0x7f28d415bc00 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc00aa90 con 0x7f28d415bc00 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d414c550 con 0x7f28d415bc00 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d4168af0 con 0x7f28d415bc00 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.010+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d404ef30 con 0x7f28d415bc00 2024-04-25T10:27:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.014+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28cc00ac30 con 0x7f28d415bc00 2024-04-25T10:27:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.014+0000 7f28cb400640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28b0066f90 0x7f28b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.014+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f28cc0850e0 con 0x7f28d415bc00 2024-04-25T10:27:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.014+0000 7f28d3400640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28b0066f90 0x7f28b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:11.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.014+0000 7f28d3400640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28b0066f90 0x7f28b0069450 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005890 tx=0x7f28c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:11.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.018+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28cc055fc0 con 0x7f28d415bc00 2024-04-25T10:27:11.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.182+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28d40b2a10 con 0x7f28d415bc00 2024-04-25T10:27:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.830+0000 7f28cb400640 1 -- 172.21.15.45:0/61567295 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28cc0558c0 con 0x7f28d415bc00 2024-04-25T10:27:11.836 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:27:11.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28b0066f90 msgr2=0x7f28b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28b0066f90 0x7f28b0069450 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005890 tx=0x7f28c4009040 comp rx=0 tx=0).stop 2024-04-25T10:27:11.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 msgr2=0x7f28d414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:11.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 0x7f28d414b530 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00c7a0 tx=0x7f28cc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 shutdown_connections 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f28b0066f90 0x7f28b0069450 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28d415bc00 0x7f28d414b530 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28d40d00f0 0x7f28d4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 --2- 172.21.15.45:0/61567295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f28d40cf040 0x7f28d416a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 >> 172.21.15.45:0/61567295 conn(0x7f28d4066b70 msgr2=0x7f28d4066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 shutdown_connections 2024-04-25T10:27:11.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:11.838+0000 7f28da000640 1 -- 172.21.15.45:0/61567295 wait complete. 2024-04-25T10:27:11.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:27:12.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.014+0000 7f596a800640 1 Processor -- start 2024-04-25T10:27:12.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.014+0000 7f596a800640 1 -- start start 2024-04-25T10:27:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f596a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f596406b0d0 0x7f59640cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f596a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406bbe0 0x7f59640cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:12.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f596a800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640cf290 con 0x7f596406b800 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f596a800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59640cde90 con 0x7f596406b0d0 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f596a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640cfdb0 con 0x7f596406bbe0 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f596406b0d0 0x7f59640cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f596406b0d0 0x7f59640cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41016/0 (socket says 172.21.15.45:41016) 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 -- 172.21.15.45:0/3566273668 learned_addr learned my addr 172.21.15.45:0/3566273668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f5963e00640 1 --2- 172.21.15.45:0/3566273668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406bbe0 0x7f59640cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 -- 172.21.15.45:0/3566273668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406bbe0 msgr2=0x7f59640cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 --2- 172.21.15.45:0/3566273668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406bbe0 0x7f59640cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 -- 172.21.15.45:0/3566273668 >> v1:172.21.15.117:6789/0 conn(0x7f596406b800 legacy=0x7f59640ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:12.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 -- 172.21.15.45:0/3566273668 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640d0090 con 0x7f596406b0d0 2024-04-25T10:27:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.018+0000 7f595be00640 1 --2- 172.21.15.45:0/3566273668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f59640cd410 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f594c009870 tx=0x7f594c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917e886d4e65d8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:12.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f5962a00640 1 -- 172.21.15.45:0/3566273668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0b5070 con 0x7f596406b0d0 2024-04-25T10:27:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f5962a00640 1 -- 172.21.15.45:0/3566273668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c0a8a50 con 0x7f596406b0d0 2024-04-25T10:27:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f5962a00640 1 -- 172.21.15.45:0/3566273668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0a8e20 con 0x7f596406b0d0 2024-04-25T10:27:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- 172.21.15.45:0/3566273668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 msgr2=0x7f59640cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 --2- 172.21.15.45:0/3566273668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f59640cd410 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f594c009870 tx=0x7f594c01ed50 comp rx=0 tx=0).stop 2024-04-25T10:27:12.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- 172.21.15.45:0/3566273668 shutdown_connections 2024-04-25T10:27:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 --2- 172.21.15.45:0/3566273668 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406bbe0 0x7f59640cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 --2- 172.21.15.45:0/3566273668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f59640cd410 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- 172.21.15.45:0/3566273668 >> 172.21.15.45:0/3566273668 conn(0x7f59640d19d0 msgr2=0x7f59640d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- 172.21.15.45:0/3566273668 shutdown_connections 2024-04-25T10:27:12.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- 172.21.15.45:0/3566273668 wait complete. 2024-04-25T10:27:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 Processor -- start 2024-04-25T10:27:12.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- start start 2024-04-25T10:27:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f596415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406b800 0x7f59641f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406bbe0 0x7f59640ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:12.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640d0930 con 0x7f596406b0d0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640d0650 con 0x7f596406bbe0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f596a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640d0e50 con 0x7f596406b800 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f5963400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f596415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f5963400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f596415e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41032/0 (socket says 172.21.15.45:41032) 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.022+0000 7f5963400640 1 -- 172.21.15.45:0/834833713 learned_addr learned my addr 172.21.15.45:0/834833713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f595be00640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406b800 0x7f59641f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963e00640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406bbe0 0x7f59640ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963400640 1 -- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406b800 msgr2=0x7f59641f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963400640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406b800 0x7f59641f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963400640 1 -- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406bbe0 msgr2=0x7f59640ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963400640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406bbe0 0x7f59640ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963400640 1 -- 172.21.15.45:0/834833713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640cad80 con 0x7f596406b0d0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f5963400640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f596415e420 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f595400c450 tx=0x7f595400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:12.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954016020 con 0x7f596406b0d0 2024-04-25T10:27:12.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540052d0 con 0x7f596406b0d0 2024-04-25T10:27:12.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640caaa0 con 0x7f596406b0d0 2024-04-25T10:27:12.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954007260 con 0x7f596406b0d0 2024-04-25T10:27:12.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59640c8e20 con 0x7f596406b0d0 2024-04-25T10:27:12.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596404ef30 con 0x7f596406b0d0 2024-04-25T10:27:12.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.026+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5954007400 con 0x7f596406b0d0 2024-04-25T10:27:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.030+0000 7f595b400640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5940066f60 0x7f5940069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.030+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f5954084780 con 0x7f596406b0d0 2024-04-25T10:27:12.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.030+0000 7f595be00640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5940066f60 0x7f5940069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:12.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.030+0000 7f595be00640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5940066f60 0x7f5940069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f594c0b2000 tx=0x7f594c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:12.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.034+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5954020020 con 0x7f596406b0d0 2024-04-25T10:27:12.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.194+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59640b7950 con 0x7f596406b0d0 2024-04-25T10:27:12.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.834+0000 7f595b400640 1 -- 172.21.15.45:0/834833713 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v122) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5954005480 con 0x7f596406b0d0 2024-04-25T10:27:12.838 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:27:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5940066f60 msgr2=0x7f5940069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5940066f60 0x7f5940069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f594c0b2000 tx=0x7f594c009040 comp rx=0 tx=0).stop 2024-04-25T10:27:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 msgr2=0x7f596415e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f596415e420 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f595400c450 tx=0x7f595400c910 comp rx=0 tx=0).stop 2024-04-25T10:27:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 shutdown_connections 2024-04-25T10:27:12.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5940066f60 0x7f5940069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596406bbe0 0x7f59640ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596406b800 0x7f59641f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 --2- 172.21.15.45:0/834833713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596406b0d0 0x7f596415e420 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 >> 172.21.15.45:0/834833713 conn(0x7f59640d19d0 msgr2=0x7f59640ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 shutdown_connections 2024-04-25T10:27:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:12.838+0000 7f596a800640 1 -- 172.21.15.45:0/834833713 wait complete. 2024-04-25T10:27:12.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:27:13.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.018+0000 7fa4b0c00640 1 Processor -- start 2024-04-25T10:27:13.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.018+0000 7fa4b0c00640 1 -- start start 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4b0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac070350 0x7fa4ac15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4b0c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4ac0b31c0 0x7fa4ac0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4b0c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac15b080 con 0x7fa4ac0aef80 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4b0c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ac15adb0 con 0x7fa4ac0b31c0 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4b0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac15eb60 con 0x7fa4ac070350 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4a9e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa4ac0aef80 0x7fa4ac15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36880/0 (socket says 172.21.15.45:36880) 2024-04-25T10:27:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4a9e00640 1 -- 172.21.15.45:0/1093763352 learned_addr learned my addr 172.21.15.45:0/1093763352 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4aa800640 1 --2- 172.21.15.45:0/1093763352 >> v2:172.21.15.45:3300/0 conn(0x7fa4ac0b31c0 0x7fa4ac0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4a1400640 1 --2- 172.21.15.45:0/1093763352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac070350 0x7fa4ac15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4aa800640 1 -- 172.21.15.45:0/1093763352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac070350 msgr2=0x7fa4ac15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4aa800640 1 --2- 172.21.15.45:0/1093763352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac070350 0x7fa4ac15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4aa800640 1 -- 172.21.15.45:0/1093763352 >> v1:172.21.15.45:6789/0 conn(0x7fa4ac0aef80 legacy=0x7fa4ac15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4aa800640 1 -- 172.21.15.45:0/1093763352 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac15ee40 con 0x7fa4ac0b31c0 2024-04-25T10:27:13.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4aa800640 1 --2- 172.21.15.45:0/1093763352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0b31c0 0x7fa4ac0b3610 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009d90 tx=0x7fa4a401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6ef7a04aceace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4a9400640 1 -- 172.21.15.45:0/1093763352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40b2070 con 0x7fa4ac0b31c0 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.022+0000 7fa4a9400640 1 -- 172.21.15.45:0/1093763352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a40a5940 con 0x7fa4ac0b31c0 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a9400640 1 -- 172.21.15.45:0/1093763352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40a5d10 con 0x7fa4ac0b31c0 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- 172.21.15.45:0/1093763352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0b31c0 msgr2=0x7fa4ac0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/1093763352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0b31c0 0x7fa4ac0b3610 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009d90 tx=0x7fa4a401c970 comp rx=0 tx=0).stop 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- 172.21.15.45:0/1093763352 shutdown_connections 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/1093763352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0b31c0 0x7fa4ac0b3610 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/1093763352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac070350 0x7fa4ac15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- 172.21.15.45:0/1093763352 >> 172.21.15.45:0/1093763352 conn(0x7fa4ac0669a0 msgr2=0x7fa4ac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:13.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- 172.21.15.45:0/1093763352 shutdown_connections 2024-04-25T10:27:13.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- 172.21.15.45:0/1093763352 wait complete. 2024-04-25T10:27:13.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 Processor -- start 2024-04-25T10:27:13.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- start start 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4ac070350 0x7fa4ac0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 0x7fa4ac0af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac0b31c0 0x7fa4ac159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac1679a0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac167820 con 0x7fa4ac070350 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4b0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac167b20 con 0x7fa4ac0b31c0 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 0x7fa4ac0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 0x7fa4ac0af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41048/0 (socket says 172.21.15.45:41048) 2024-04-25T10:27:13.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 -- 172.21.15.45:0/986656233 learned_addr learned my addr 172.21.15.45:0/986656233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a9e00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4ac070350 0x7fa4ac0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4aa800640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac0b31c0 0x7fa4ac159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 -- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac0b31c0 msgr2=0x7fa4ac159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac0b31c0 0x7fa4ac159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 -- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4ac070350 msgr2=0x7fa4ac0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4ac070350 0x7fa4ac0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a1400640 1 -- 172.21.15.45:0/986656233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0b03d0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4aa800640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac0b31c0 0x7fa4ac159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.026+0000 7fa4a9e00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4ac070350 0x7fa4ac0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a1400640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 0x7fa4ac0af910 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa49400d800 tx=0x7fa49400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400b6c0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494019070 con 0x7fa4ac0aef80 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400ba60 con 0x7fa4ac0aef80 2024-04-25T10:27:13.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac0b06b0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0b0c70 con 0x7fa4ac0aef80 2024-04-25T10:27:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac04ef30 con 0x7fa4ac0aef80 2024-04-25T10:27:13.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa49400bc00 con 0x7fa4ac0aef80 2024-04-25T10:27:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a2a00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa488066f60 0x7fa488069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa494085140 con 0x7fa4ac0aef80 2024-04-25T10:27:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.030+0000 7fa4a9e00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa488066f60 0x7fa488069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.034+0000 7fa4a9e00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa488066f60 0x7fa488069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa49c000f70 tx=0x7fa49c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:13.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.034+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4940564e0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.198+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa4ac0b2fa0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4a2a00640 1 -- 172.21.15.45:0/986656233 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa494055de0 con 0x7fa4ac0aef80 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa488066f60 msgr2=0x7fa488069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa488066f60 0x7fa488069420 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa49c000f70 tx=0x7fa49c009040 comp rx=0 tx=0).stop 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 msgr2=0x7fa4ac0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 0x7fa4ac0af910 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa49400d800 tx=0x7fa49400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 shutdown_connections 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa488066f60 0x7fa488069420 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4ac0b31c0 0x7fa4ac159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ac0aef80 0x7fa4ac0af910 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 --2- 172.21.15.45:0/986656233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4ac070350 0x7fa4ac0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 >> 172.21.15.45:0/986656233 conn(0x7fa4ac0669a0 msgr2=0x7fa4ac15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:13.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 shutdown_connections 2024-04-25T10:27:13.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.202+0000 7fa4b0c00640 1 -- 172.21.15.45:0/986656233 wait complete. 2024-04-25T10:27:13.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:27:13.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 Processor -- start 2024-04-25T10:27:13.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 -- start start 2024-04-25T10:27:13.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f56b8146ed0 0x7f56b81472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56b80726b0 0x7f56b8071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b806c600 con 0x7f56b8071c80 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b8161980 con 0x7f56b8146ed0 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56bd000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b8072080 con 0x7f56b80726b0 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56b5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f56b8146ed0 0x7f56b81472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56b5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f56b8146ed0 0x7f56b81472b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41052/0 (socket says 172.21.15.45:41052) 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56b5e00640 1 -- 172.21.15.45:0/1047365237 learned_addr learned my addr 172.21.15.45:0/1047365237 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.366+0000 7f56afe00640 1 --2- 172.21.15.45:0/1047365237 >> v2:172.21.15.117:3300/0 conn(0x7f56b80726b0 0x7f56b8071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5e00640 1 -- 172.21.15.45:0/1047365237 >> v2:172.21.15.117:3300/0 conn(0x7f56b80726b0 msgr2=0x7f56b8071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5e00640 1 --2- 172.21.15.45:0/1047365237 >> v2:172.21.15.117:3300/0 conn(0x7f56b80726b0 0x7f56b8071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5e00640 1 -- 172.21.15.45:0/1047365237 >> v1:172.21.15.117:6789/0 conn(0x7f56b8071c80 legacy=0x7f56b806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5e00640 1 -- 172.21.15.45:0/1047365237 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8161bd0 con 0x7f56b8146ed0 2024-04-25T10:27:13.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5e00640 1 --2- 172.21.15.45:0/1047365237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b81472b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f56b806da70 tx=0x7f56a801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9573e85d842bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5400640 1 -- 172.21.15.45:0/1047365237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80aa650 con 0x7f56b8146ed0 2024-04-25T10:27:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5400640 1 -- 172.21.15.45:0/1047365237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80af070 con 0x7f56b8146ed0 2024-04-25T10:27:13.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56b5400640 1 -- 172.21.15.45:0/1047365237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80aaa10 con 0x7f56b8146ed0 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56bd000640 1 -- 172.21.15.45:0/1047365237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 msgr2=0x7f56b81472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56bd000640 1 --2- 172.21.15.45:0/1047365237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b81472b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f56b806da70 tx=0x7f56a801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56bd000640 1 -- 172.21.15.45:0/1047365237 shutdown_connections 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.370+0000 7f56bd000640 1 --2- 172.21.15.45:0/1047365237 >> v2:172.21.15.117:3300/0 conn(0x7f56b80726b0 0x7f56b8071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 --2- 172.21.15.45:0/1047365237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b81472b0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- 172.21.15.45:0/1047365237 >> 172.21.15.45:0/1047365237 conn(0x7f56b80669a0 msgr2=0x7f56b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- 172.21.15.45:0/1047365237 shutdown_connections 2024-04-25T10:27:13.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- 172.21.15.45:0/1047365237 wait complete. 2024-04-25T10:27:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 Processor -- start 2024-04-25T10:27:13.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- start start 2024-04-25T10:27:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b8071c80 0x7f56b8159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b80726b0 0x7f56b8158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b80b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8147bf0 con 0x7f56b8146ed0 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8147a70 con 0x7f56b8071c80 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56bd000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80ae850 con 0x7f56b80726b0 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b80b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b8071c80 0x7f56b8159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b80b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41066/0 (socket says 172.21.15.45:41066) 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 -- 172.21.15.45:0/3257668434 learned_addr learned my addr 172.21.15.45:0/3257668434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56afe00640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b80726b0 0x7f56b8158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 -- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b80726b0 msgr2=0x7f56b8158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b80726b0 0x7f56b8158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 -- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b8071c80 msgr2=0x7f56b8159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b8071c80 0x7f56b8159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 -- 172.21.15.45:0/3257668434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80b3fb0 con 0x7f56b8146ed0 2024-04-25T10:27:13.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b5e00640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b8071c80 0x7f56b8159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56b6800640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b80b35d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c770 tx=0x7f56b000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.374+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0003b80 con 0x7f56b8146ed0 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b0003d20 con 0x7f56b8146ed0 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000aa90 con 0x7f56b8146ed0 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b80b4290 con 0x7f56b8146ed0 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b815a3f0 con 0x7f56b8146ed0 2024-04-25T10:27:13.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b804ef30 con 0x7f56b8146ed0 2024-04-25T10:27:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56b000ac30 con 0x7f56b8146ed0 2024-04-25T10:27:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56ae000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5694066f60 0x7f5694069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f56b0084b30 con 0x7f56b8146ed0 2024-04-25T10:27:13.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56b5e00640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5694066f60 0x7f5694069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:13.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.378+0000 7f56b5e00640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5694066f60 0x7f5694069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f56a80ae000 tx=0x7f56a80a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:13.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.382+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56b00557e0 con 0x7f56b8146ed0 2024-04-25T10:27:13.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.598+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f56b8072490 con 0x7f56b8146ed0 2024-04-25T10:27:13.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56ae000640 1 -- 172.21.15.45:0/3257668434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v452) v1 ==== 166+0+0 (secure 0 0 0) 0x7f56b0059880 con 0x7f56b8146ed0 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5694066f60 msgr2=0x7f5694069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5694066f60 0x7f5694069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f56a80ae000 tx=0x7f56a80a7000 comp rx=0 tx=0).stop 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 msgr2=0x7f56b80b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b80b35d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c770 tx=0x7f56b000cc30 comp rx=0 tx=0).stop 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 shutdown_connections 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5694066f60 0x7f5694069420 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56b8146ed0 0x7f56b80b35d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b80726b0 0x7f56b8158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 --2- 172.21.15.45:0/3257668434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b8071c80 0x7f56b8159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:13.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 >> 172.21.15.45:0/3257668434 conn(0x7f56b80669a0 msgr2=0x7f56b813f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:13.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 shutdown_connections 2024-04-25T10:27:13.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:13.850+0000 7f56bd000640 1 -- 172.21.15.45:0/3257668434 wait complete. 2024-04-25T10:27:13.876 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:27:13.876 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:27:13.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:14.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa7400640 1 Processor -- start 2024-04-25T10:27:14.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa7400640 1 -- start start 2024-04-25T10:27:14.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcaa00d0350 0x7fcaa00d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa7400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa00ccaa0 con 0x7fcaa015df60 2024-04-25T10:27:14.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa7400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa00ccd80 con 0x7fcaa006aea0 2024-04-25T10:27:14.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa7400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa015e340 con 0x7fcaa00d0350 2024-04-25T10:27:14.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa4a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcaa015df60 0x7fcaa00cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36884/0 (socket says 172.21.15.45:36884) 2024-04-25T10:27:14.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fcaa4a00640 1 -- 172.21.15.45:0/2962037317 learned_addr learned my addr 172.21.15.45:0/2962037317 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:14.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fca9fe00640 1 --2- 172.21.15.45:0/2962037317 >> v2:172.21.15.117:3300/0 conn(0x7fcaa00d0350 0x7fcaa00d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.014+0000 7fca9f400640 1 -- 172.21.15.45:0/2962037317 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281103621 0 0) 0x7fcaa00ccaa0 con 0x7fcaa015df60 2024-04-25T10:27:14.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9f400640 1 -- 172.21.15.45:0/2962037317 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca88003530 con 0x7fcaa015df60 2024-04-25T10:27:14.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9f400640 1 -- 172.21.15.45:0/2962037317 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464779207 0 0) 0x7fca88003530 con 0x7fcaa015df60 2024-04-25T10:27:14.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9fe00640 1 -- 172.21.15.45:0/2962037317 >> v1:172.21.15.117:6789/0 conn(0x7fcaa006aea0 legacy=0x7fcaa006b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9fe00640 1 -- 172.21.15.45:0/2962037317 >> v1:172.21.15.45:6789/0 conn(0x7fcaa015df60 legacy=0x7fcaa00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9fe00640 1 -- 172.21.15.45:0/2962037317 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00cd060 con 0x7fcaa00d0350 2024-04-25T10:27:14.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9fe00640 1 --2- 172.21.15.45:0/2962037317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa00d0350 0x7fcaa00d07b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097c0 tx=0x7fca8c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3997c367cedd7270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9f400640 1 -- 172.21.15.45:0/2962037317 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0b0070 con 0x7fcaa00d0350 2024-04-25T10:27:14.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9f400640 1 -- 172.21.15.45:0/2962037317 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa006c1d0 con 0x7fcaa00d0350 2024-04-25T10:27:14.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.018+0000 7fca9f400640 1 -- 172.21.15.45:0/2962037317 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94001680 con 0x7fcaa00d0350 2024-04-25T10:27:14.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- 172.21.15.45:0/2962037317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa00d0350 msgr2=0x7fcaa00d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 --2- 172.21.15.45:0/2962037317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa00d0350 0x7fcaa00d07b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097c0 tx=0x7fca8c01c330 comp rx=0 tx=0).stop 2024-04-25T10:27:14.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- 172.21.15.45:0/2962037317 shutdown_connections 2024-04-25T10:27:14.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 --2- 172.21.15.45:0/2962037317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa00d0350 0x7fcaa00d07b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- 172.21.15.45:0/2962037317 >> 172.21.15.45:0/2962037317 conn(0x7fcaa00d1910 msgr2=0x7fcaa00d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:14.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- 172.21.15.45:0/2962037317 shutdown_connections 2024-04-25T10:27:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- 172.21.15.45:0/2962037317 wait complete. 2024-04-25T10:27:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 Processor -- start 2024-04-25T10:27:14.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- start start 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa006aea0 0x7fcaa0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 0x7fcaa01548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcaa015df60 0x7fcaa0158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00d0ff0 con 0x7fcaa00d0350 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00d0e70 con 0x7fcaa006aea0 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa7400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa016ba30 con 0x7fcaa015df60 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 0x7fcaa01548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 0x7fcaa01548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41078/0 (socket says 172.21.15.45:41078) 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 -- 172.21.15.45:0/1351635077 learned_addr learned my addr 172.21.15.45:0/1351635077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa4a00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa006aea0 0x7fcaa0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 -- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcaa015df60 msgr2=0x7fcaa0158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa5400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcaa015df60 0x7fcaa0158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcaa015df60 0x7fcaa0158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 -- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa006aea0 msgr2=0x7fcaa0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa006aea0 0x7fcaa0159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 -- 172.21.15.45:0/1351635077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00ca730 con 0x7fcaa00d0350 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa4a00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa006aea0 0x7fcaa0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fcaa5400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcaa015df60 0x7fcaa0158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.022+0000 7fca9fe00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 0x7fcaa01548f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ae040 tx=0x7fca8c01ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0b0050 con 0x7fcaa00d0350 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c0b1040 con 0x7fcaa00d0350 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa00caa10 con 0x7fcaa00d0350 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004020 con 0x7fcaa00d0350 2024-04-25T10:27:14.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa00cafd0 con 0x7fcaa00d0350 2024-04-25T10:27:14.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa004ef30 con 0x7fcaa00d0350 2024-04-25T10:27:14.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca8c0be020 con 0x7fcaa00d0350 2024-04-25T10:27:14.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fca9cc00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca80066f90 0x7fca80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fca8c11ed00 con 0x7fcaa00d0350 2024-04-25T10:27:14.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.026+0000 7fcaa4a00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca80066f90 0x7fca80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.030+0000 7fcaa4a00640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca80066f90 0x7fca80069450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fca94001390 tx=0x7fca94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.030+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca8c015fb0 con 0x7fcaa00d0350 2024-04-25T10:27:14.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.214+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaa0067f00 con 0x7fcaa00d0350 2024-04-25T10:27:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.214+0000 7fca9cc00640 1 -- 172.21.15.45:0/1351635077 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+13018 (secure 0 0 0) 0x7fca8c0bf370 con 0x7fcaa00d0350 2024-04-25T10:27:14.218 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:14.218 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:27:12.837039+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6834/1838662791":"2024-04-26T10:27:09.788700+0000","172.21.15.45:6832/1838662791":"2024-04-26T10:27:09.788700+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca80066f90 msgr2=0x7fca80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca80066f90 0x7fca80069450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fca94001390 tx=0x7fca94009040 comp rx=0 tx=0).stop 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 msgr2=0x7fcaa01548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 0x7fcaa01548f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ae040 tx=0x7fca8c01ab70 comp rx=0 tx=0).stop 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 shutdown_connections 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca80066f90 0x7fca80069450 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcaa015df60 0x7fcaa0158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaa00d0350 0x7fcaa01548f0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 --2- 172.21.15.45:0/1351635077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcaa006aea0 0x7fcaa0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 >> 172.21.15.45:0/1351635077 conn(0x7fcaa00d1910 msgr2=0x7fcaa00cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 shutdown_connections 2024-04-25T10:27:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.218+0000 7fcaa7400640 1 -- 172.21.15.45:0/1351635077 wait complete. 2024-04-25T10:27:14.241 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:27:14.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:27:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e7400640 1 Processor -- start 2024-04-25T10:27:14.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e7400640 1 -- start start 2024-04-25T10:27:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 0x7f03e006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e7400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e006b1d0 con 0x7f03e00d1fb0 2024-04-25T10:27:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e7400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e006b470 con 0x7f03e00cfb30 2024-04-25T10:27:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e7400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015e110 con 0x7f03e015dd30 2024-04-25T10:27:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e5400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f03e00d1fb0 0x7f03e00b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36898/0 (socket says 172.21.15.45:36898) 2024-04-25T10:27:14.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e5400640 1 -- 172.21.15.45:0/362523350 learned_addr learned my addr 172.21.15.45:0/362523350 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03e4a00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 0x7f03e006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316284167 0 0) 0x7f03e006b1d0 con 0x7f03e00d1fb0 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c4003530 con 0x7f03e00d1fb0 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570928948 0 0) 0x7f03e006b470 con 0x7f03e00cfb30 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e006b1d0 con 0x7f03e00cfb30 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284138243 0 0) 0x7f03c4003530 con 0x7f03e00d1fb0 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 msgr2=0x7f03e006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 0x7f03e006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 >> v1:172.21.15.117:6789/0 conn(0x7f03e00cfb30 legacy=0x7f03e00d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e006b750 con 0x7f03e00d1fb0 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f03d8001a00 con 0x7f03e00d1fb0 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 >> v1:172.21.15.45:6789/0 conn(0x7f03e00d1fb0 legacy=0x7f03e00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 0x7f03c4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.382+0000 7f03dfe00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c40040e0 0x7f03c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c4004d50 0x7f03c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c400ba80 con 0x7f03c40040e0 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c400bc30 con 0x7f03c4004d50 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c400be30 con 0x7f03e015dd30 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c40040e0 0x7f03c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03e4a00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 0x7f03c4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03e5400640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c4004d50 0x7f03c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 -- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 msgr2=0x7f03c4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e015dd30 0x7f03c4003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 -- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c4004d50 msgr2=0x7f03c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c4004d50 0x7f03c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 -- 172.21.15.45:0/362523350 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4003530 con 0x7f03c40040e0 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03d7e00640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c40040e0 0x7f03c4004810 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f03e00cff10 tx=0x7f03c801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165c43e52d6a696e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8001520 con 0x7f03c40040e0 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c80052f0 con 0x7f03c40040e0 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.386+0000 7f03dfe00640 1 -- 172.21.15.45:0/362523350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80a8930 con 0x7f03c40040e0 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- 172.21.15.45:0/362523350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c40040e0 msgr2=0x7f03c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 --2- 172.21.15.45:0/362523350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c40040e0 0x7f03c4004810 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f03e00cff10 tx=0x7f03c801ecd0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 -- 172.21.15.45:0/362523350 reap_dead start 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- 172.21.15.45:0/362523350 shutdown_connections 2024-04-25T10:27:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- 172.21.15.45:0/362523350 >> 172.21.15.45:0/362523350 conn(0x7f03e009a9d0 msgr2=0x7f03e009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:14.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- 172.21.15.45:0/362523350 shutdown_connections 2024-04-25T10:27:14.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- 172.21.15.45:0/362523350 wait complete. 2024-04-25T10:27:14.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 Processor -- start 2024-04-25T10:27:14.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- start start 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c40040e0 0x7f03e00c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e00c5520 0x7f03e0151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 0x7f03e00afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00ce220 con 0x7f03e00af6f0 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e006bd10 con 0x7f03c40040e0 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e7400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00ce3a0 con 0x7f03e00c5520 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 0x7f03e00afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 0x7f03e00afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41096/0 (socket says 172.21.15.45:41096) 2024-04-25T10:27:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 -- 172.21.15.45:0/266115274 learned_addr learned my addr 172.21.15.45:0/266115274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03d7e00640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e00c5520 0x7f03e0151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e4a00640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c40040e0 0x7f03e00c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 -- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e00c5520 msgr2=0x7f03e0151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e00c5520 0x7f03e0151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 -- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c40040e0 msgr2=0x7f03e00c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c40040e0 0x7f03e00c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.390+0000 7f03e5400640 1 -- 172.21.15.45:0/266115274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00b5ef0 con 0x7f03e00af6f0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03e5400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 0x7f03e00afba0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f03e00b11a0 tx=0x7f03d8007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800aad0 con 0x7f03e00af6f0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d8018070 con 0x7f03e00af6f0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80133f0 con 0x7f03e00af6f0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e00b61d0 con 0x7f03e00af6f0 2024-04-25T10:27:14.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e00b6790 con 0x7f03e00af6f0 2024-04-25T10:27:14.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e004ef30 con 0x7f03e00af6f0 2024-04-25T10:27:14.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03d8013590 con 0x7f03e00af6f0 2024-04-25T10:27:14.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03dd600640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03bc066f60 0x7f03bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.394+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03d8085080 con 0x7f03e00af6f0 2024-04-25T10:27:14.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.398+0000 7f03e4a00640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03bc066f60 0x7f03bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.398+0000 7f03e4a00640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03bc066f60 0x7f03bc069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009b00 tx=0x7f03d0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.398+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03d8056420 con 0x7f03e00af6f0 2024-04-25T10:27:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.562+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03e00b7920 con 0x7f03e00af6f0 2024-04-25T10:27:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.562+0000 7f03dd600640 1 -- 172.21.15.45:0/266115274 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03d8055d20 con 0x7f03e00af6f0 2024-04-25T10:27:14.567 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03bc066f60 msgr2=0x7f03bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03bc066f60 0x7f03bc069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009b00 tx=0x7f03d0009190 comp rx=0 tx=0).stop 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 msgr2=0x7f03e00afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 0x7f03e00afba0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f03e00b11a0 tx=0x7f03d8007b10 comp rx=0 tx=0).stop 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 shutdown_connections 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f03bc066f60 0x7f03bc069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e00af6f0 0x7f03e00afba0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03e00c5520 0x7f03e0151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 --2- 172.21.15.45:0/266115274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c40040e0 0x7f03e00c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 >> 172.21.15.45:0/266115274 conn(0x7f03e009a9d0 msgr2=0x7f03e00cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 shutdown_connections 2024-04-25T10:27:14.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.566+0000 7f03e7400640 1 -- 172.21.15.45:0/266115274 wait complete. 2024-04-25T10:27:14.588 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:27:14.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.738+0000 7f45f9400640 1 Processor -- start 2024-04-25T10:27:14.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f9400640 1 -- start start 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f9400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f9400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40ad840 con 0x7f45f4070c20 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f9400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f415e9f0 con 0x7f45f40cf930 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f9400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f415bb90 con 0x7f45f40ae020 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45ebe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45f4070c20 0x7f45f415e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36906/0 (socket says 172.21.15.45:36906) 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45ebe00640 1 -- 172.21.15.45:0/3227995595 learned_addr learned my addr 172.21.15.45:0/3227995595 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f3200640 1 --2- 172.21.15.45:0/3227995595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f1e00640 1 -- 172.21.15.45:0/3227995595 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168142248 0 0) 0x7f45f40ad840 con 0x7f45f4070c20 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f1e00640 1 -- 172.21.15.45:0/3227995595 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d8003530 con 0x7f45f4070c20 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f1e00640 1 -- 172.21.15.45:0/3227995595 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866563120 0 0) 0x7f45f415e9f0 con 0x7f45f40cf930 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f3200640 1 -- 172.21.15.45:0/3227995595 >> v1:172.21.15.117:6789/0 conn(0x7f45f40cf930 legacy=0x7f45f40cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f3200640 1 -- 172.21.15.45:0/3227995595 >> v1:172.21.15.45:6789/0 conn(0x7f45f4070c20 legacy=0x7f45f415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.742+0000 7f45f3200640 1 -- 172.21.15.45:0/3227995595 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f415ecd0 con 0x7f45f40ae020 2024-04-25T10:27:14.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f3200640 1 --2- 172.21.15.45:0/3227995595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40ae4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009dc0 tx=0x7f45ec01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e428869ab09bb454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f1e00640 1 -- 172.21.15.45:0/3227995595 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0b2070 con 0x7f45f40ae020 2024-04-25T10:27:14.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f1e00640 1 -- 172.21.15.45:0/3227995595 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0a5a20 con 0x7f45f40ae020 2024-04-25T10:27:14.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f1e00640 1 -- 172.21.15.45:0/3227995595 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0a5df0 con 0x7f45f40ae020 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 -- 172.21.15.45:0/3227995595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 msgr2=0x7f45f40ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 --2- 172.21.15.45:0/3227995595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40ae4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009dc0 tx=0x7f45ec01c960 comp rx=0 tx=0).stop 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 -- 172.21.15.45:0/3227995595 shutdown_connections 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 --2- 172.21.15.45:0/3227995595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40ae4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 -- 172.21.15.45:0/3227995595 >> 172.21.15.45:0/3227995595 conn(0x7f45f4066b70 msgr2=0x7f45f4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 -- 172.21.15.45:0/3227995595 shutdown_connections 2024-04-25T10:27:14.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 -- 172.21.15.45:0/3227995595 wait complete. 2024-04-25T10:27:14.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 Processor -- start 2024-04-25T10:27:14.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.746+0000 7f45f9400640 1 -- start start 2024-04-25T10:27:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f4070c20 0x7f45f415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 0x7f45f40ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f416ba70 con 0x7f45f40cf930 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f416b8f0 con 0x7f45f4070c20 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f416bbf0 con 0x7f45f40ae020 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 0x7f45f40ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 0x7f45f40ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41108/0 (socket says 172.21.15.45:41108) 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 -- 172.21.15.45:0/849829072 learned_addr learned my addr 172.21.15.45:0/849829072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45ebe00640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f2800640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f4070c20 0x7f45f415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 -- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 msgr2=0x7f45f40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 -- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f4070c20 msgr2=0x7f45f415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f4070c20 0x7f45f415a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 -- 172.21.15.45:0/849829072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4157980 con 0x7f45f40cf930 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f2800640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f4070c20 0x7f45f415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f3200640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 0x7f45f40ccd20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f45ec01ce40 tx=0x7f45ec019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0b2040 con 0x7f45f40cf930 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0bf070 con 0x7f45f40cf930 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f4155990 con 0x7f45f40cf930 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0b8690 con 0x7f45f40cf930 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4155f50 con 0x7f45f40cf930 2024-04-25T10:27:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.750+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f404ef30 con 0x7f45f40cf930 2024-04-25T10:27:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.754+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45ec0b8830 con 0x7f45f40cf930 2024-04-25T10:27:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.754+0000 7f45eaa00640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f45d0066f60 0x7f45d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.754+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f45ec11fd00 con 0x7f45f40cf930 2024-04-25T10:27:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.754+0000 7f45f2800640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f45d0066f60 0x7f45d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.754+0000 7f45f2800640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f45d0066f60 0x7f45d0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000f70 tx=0x7f45e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:14.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.758+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45ec016580 con 0x7f45f40cf930 2024-04-25T10:27:14.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:14.978+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f45f406b390 con 0x7f45f40cf930 2024-04-25T10:27:15.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.858+0000 7f45eaa00640 1 -- 172.21.15.45:0/849829072 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v454) v1 ==== 169+0+0 (secure 0 0 0) 0x7f45ec0f1090 con 0x7f45f40cf930 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f45d0066f60 msgr2=0x7f45d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f45d0066f60 0x7f45d0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000f70 tx=0x7f45e4009040 comp rx=0 tx=0).stop 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 msgr2=0x7f45f40ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 0x7f45f40ccd20 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f45ec01ce40 tx=0x7f45ec019700 comp rx=0 tx=0).stop 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 shutdown_connections 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f45d0066f60 0x7f45d0069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45f40cf930 0x7f45f40ccd20 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45f40ae020 0x7f45f40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 --2- 172.21.15.45:0/849829072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45f4070c20 0x7f45f415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 >> 172.21.15.45:0/849829072 conn(0x7f45f4066b70 msgr2=0x7f45f415f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 shutdown_connections 2024-04-25T10:27:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:15.862+0000 7f45f9400640 1 -- 172.21.15.45:0/849829072 wait complete. 2024-04-25T10:27:15.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:27:16.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 Processor -- start 2024-04-25T10:27:16.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 -- start start 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa10070350 0x7ffa1015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100b31c0 0x7ffa100b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1015b080 con 0x7ffa100aef80 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1015adb0 con 0x7ffa10070350 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.034+0000 7ffa15600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1015eb60 con 0x7ffa100b31c0 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa06000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffa10070350 0x7ffa1015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffa100aef80 0x7ffa1015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44274/0 (socket says 172.21.15.45:44274) 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0ea00640 1 -- 172.21.15.45:0/519294131 learned_addr learned my addr 172.21.15.45:0/519294131 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:16.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0f400640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100b31c0 0x7ffa100b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa06000640 1 -- 172.21.15.45:0/519294131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100b31c0 msgr2=0x7ffa100b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:16.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa06000640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100b31c0 0x7ffa100b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa06000640 1 -- 172.21.15.45:0/519294131 >> v1:172.21.15.45:6789/0 conn(0x7ffa100aef80 legacy=0x7ffa1015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa06000640 1 -- 172.21.15.45:0/519294131 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1015ee40 con 0x7ffa10070350 2024-04-25T10:27:16.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0e000640 1 -- 172.21.15.45:0/519294131 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136074732 0 0) 0x7ffa1015b080 con 0x7ffa100aef80 2024-04-25T10:27:16.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0f400640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100b31c0 0x7ffa100b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:16.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa06000640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10070350 0x7ffa1015e480 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80097c0 tx=0x7ff9f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9e41ff9f867cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:16.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0e000640 1 -- 172.21.15.45:0/519294131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80b1070 con 0x7ffa10070350 2024-04-25T10:27:16.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0e000640 1 -- 172.21.15.45:0/519294131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f80ac410 con 0x7ffa10070350 2024-04-25T10:27:16.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.038+0000 7ffa0e000640 1 -- 172.21.15.45:0/519294131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80ac7e0 con 0x7ffa10070350 2024-04-25T10:27:16.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- 172.21.15.45:0/519294131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10070350 msgr2=0x7ffa1015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10070350 0x7ffa1015e480 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80097c0 tx=0x7ff9f801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- 172.21.15.45:0/519294131 shutdown_connections 2024-04-25T10:27:16.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100b31c0 0x7ffa100b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 --2- 172.21.15.45:0/519294131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa10070350 0x7ffa1015e480 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- 172.21.15.45:0/519294131 >> 172.21.15.45:0/519294131 conn(0x7ffa100669a0 msgr2=0x7ffa10066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:16.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- 172.21.15.45:0/519294131 shutdown_connections 2024-04-25T10:27:16.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- 172.21.15.45:0/519294131 wait complete. 2024-04-25T10:27:16.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 Processor -- start 2024-04-25T10:27:16.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- start start 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa10070350 0x7ffa1015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100aef80 0x7ffa10157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 0x7ffa10146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1006c250 con 0x7ffa100b31c0 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1006c0d0 con 0x7ffa10070350 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa15600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1006c3d0 con 0x7ffa100aef80 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 0x7ffa10146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 0x7ffa10146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53732/0 (socket says 172.21.15.45:53732) 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 -- 172.21.15.45:0/1314671393 learned_addr learned my addr 172.21.15.45:0/1314671393 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa06000640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100aef80 0x7ffa10157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0ea00640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa10070350 0x7ffa1015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 -- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100aef80 msgr2=0x7ffa10157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100aef80 0x7ffa10157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 -- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa10070350 msgr2=0x7ffa1015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa10070350 0x7ffa1015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.042+0000 7ffa0f400640 1 -- 172.21.15.45:0/1314671393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10152560 con 0x7ffa100b31c0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa0f400640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 0x7ffa10146050 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ffa08007df0 tx=0x7ffa0800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0801a070 con 0x7ffa100b31c0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa080039a0 con 0x7ffa100b31c0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08003d90 con 0x7ffa100b31c0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa10155450 con 0x7ffa100b31c0 2024-04-25T10:27:16.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa10155c90 con 0x7ffa100b31c0 2024-04-25T10:27:16.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1004ef30 con 0x7ffa100b31c0 2024-04-25T10:27:16.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa0801d070 con 0x7ffa100b31c0 2024-04-25T10:27:16.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa07400640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa08084fe0 con 0x7ffa100b31c0 2024-04-25T10:27:16.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.046+0000 7ffa0ea00640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.050+0000 7ffa0ea00640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9ec066f60 0x7ff9ec069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffa00007fd0 tx=0x7ffa00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:16.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.050+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa10155450 con 0x7ffa100b31c0 2024-04-25T10:27:16.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.214+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ffa100b2fa0 con 0x7ffa100b31c0 2024-04-25T10:27:16.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.218+0000 7ffa07400640 1 -- 172.21.15.45:0/1314671393 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffa08010070 con 0x7ffa100b31c0 2024-04-25T10:27:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9ec066f60 msgr2=0x7ff9ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9ec066f60 0x7ff9ec069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffa00007fd0 tx=0x7ffa00009040 comp rx=0 tx=0).stop 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 msgr2=0x7ffa10146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 0x7ffa10146050 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ffa08007df0 tx=0x7ffa0800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 shutdown_connections 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa100b31c0 0x7ffa10146050 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa100aef80 0x7ffa10157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 --2- 172.21.15.45:0/1314671393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa10070350 0x7ffa1015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 >> 172.21.15.45:0/1314671393 conn(0x7ffa100669a0 msgr2=0x7ffa10066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 shutdown_connections 2024-04-25T10:27:16.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.222+0000 7ffa15600640 1 -- 172.21.15.45:0/1314671393 wait complete. 2024-04-25T10:27:16.246 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:27:16.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:27:16.362 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:27:16.454 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:27:16.454 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:27:16.454 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:16.463 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:16.463 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:27:16.463 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:16.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:16.473 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:27:16.473 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:16.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:27:16.516 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:27:16.516 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:16.527 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:27:16.528 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:27:16.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.694+0000 7f059a200640 1 Processor -- start 2024-04-25T10:27:16.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.694+0000 7f059a200640 1 -- start start 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f059a200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f059415bc40 0x7f059406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f059a200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05940cda10 0x7f05940b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f059a200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05940b2770 con 0x7f05940cf930 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f059a200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406c670 con 0x7f059415bc40 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f059a200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406c950 con 0x7f05940cda10 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f059415bc40 0x7f059406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f05940cf930 0x7f059415f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44276/0 (socket says 172.21.15.45:44276) 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593e00640 1 -- 172.21.15.45:0/2733260350 learned_addr learned my addr 172.21.15.45:0/2733260350 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f058aa00640 1 --2- 172.21.15.45:0/2733260350 >> v2:172.21.15.117:3300/0 conn(0x7f05940cda10 0x7f05940b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593400640 1 -- 172.21.15.45:0/2733260350 >> v2:172.21.15.117:3300/0 conn(0x7f05940cda10 msgr2=0x7f05940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593400640 1 --2- 172.21.15.45:0/2733260350 >> v2:172.21.15.117:3300/0 conn(0x7f05940cda10 0x7f05940b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593400640 1 -- 172.21.15.45:0/2733260350 >> v1:172.21.15.45:6789/0 conn(0x7f05940cf930 legacy=0x7f059415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f0593400640 1 -- 172.21.15.45:0/2733260350 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059406cc30 con 0x7f059415bc40 2024-04-25T10:27:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.698+0000 7f058aa00640 1 --2- 172.21.15.45:0/2733260350 >> v2:172.21.15.117:3300/0 conn(0x7f05940cda10 0x7f05940b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f0593400640 1 --2- 172.21.15.45:0/2733260350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f059415bc40 0x7f059406c090 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f05940726f0 tx=0x7f058401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5079c9ff95c9141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f0592a00640 1 -- 172.21.15.45:0/2733260350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05840b1070 con 0x7f059415bc40 2024-04-25T10:27:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f0592a00640 1 -- 172.21.15.45:0/2733260350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05840ac5f0 con 0x7f059415bc40 2024-04-25T10:27:16.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f0592a00640 1 -- 172.21.15.45:0/2733260350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05840ac9c0 con 0x7f059415bc40 2024-04-25T10:27:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 -- 172.21.15.45:0/2733260350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f059415bc40 msgr2=0x7f059406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 --2- 172.21.15.45:0/2733260350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f059415bc40 0x7f059406c090 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f05940726f0 tx=0x7f058401c480 comp rx=0 tx=0).stop 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 -- 172.21.15.45:0/2733260350 shutdown_connections 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 --2- 172.21.15.45:0/2733260350 >> v2:172.21.15.117:3300/0 conn(0x7f05940cda10 0x7f05940b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 --2- 172.21.15.45:0/2733260350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f059415bc40 0x7f059406c090 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 -- 172.21.15.45:0/2733260350 >> 172.21.15.45:0/2733260350 conn(0x7f0594066b70 msgr2=0x7f0594066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 -- 172.21.15.45:0/2733260350 shutdown_connections 2024-04-25T10:27:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 -- 172.21.15.45:0/2733260350 wait complete. 2024-04-25T10:27:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 Processor -- start 2024-04-25T10:27:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.702+0000 7f059a200640 1 -- start start 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05940cda10 0x7f05941561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 0x7f05941460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f059415bc40 0x7f05940af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594168b50 con 0x7f05940cf930 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05941689d0 con 0x7f059415bc40 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0594168cd0 con 0x7f05940cda10 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 0x7f05941460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 0x7f05941460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53742/0 (socket says 172.21.15.45:53742) 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 -- 172.21.15.45:0/1870517007 learned_addr learned my addr 172.21.15.45:0/1870517007 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 -- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05940cda10 msgr2=0x7f05941561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f0593400640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05940cda10 0x7f05941561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f0593e00640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f059415bc40 0x7f05940af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05940cda10 0x7f05941561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 -- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f059415bc40 msgr2=0x7f05940af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f059415bc40 0x7f05940af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 -- 172.21.15.45:0/1870517007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594156a00 con 0x7f05940cf930 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f0593400640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05940cda10 0x7f05941561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f0593e00640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f059415bc40 0x7f05940af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058aa00640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 0x7f05941460f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f057c00d800 tx=0x7f057c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c018020 con 0x7f05940cf930 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c0096c0 con 0x7f05940cf930 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c009ab0 con 0x7f05940cf930 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594156ce0 con 0x7f05940cf930 2024-04-25T10:27:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05941572a0 con 0x7f05940cf930 2024-04-25T10:27:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059404ef30 con 0x7f05940cf930 2024-04-25T10:27:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.706+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f057c01b020 con 0x7f05940cf930 2024-04-25T10:27:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.710+0000 7f058be00640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0570066f60 0x7f0570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.710+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f057c085340 con 0x7f05940cf930 2024-04-25T10:27:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.710+0000 7f0593400640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0570066f60 0x7f0570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.710+0000 7f0593400640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0570066f60 0x7f0570069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f05840ae000 tx=0x7f05840a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:16.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.714+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f057c022020 con 0x7f05940cf930 2024-04-25T10:27:16.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.906+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05940af1c0 con 0x7f05940cf930 2024-04-25T10:27:16.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.906+0000 7f058be00640 1 -- 172.21.15.45:0/1870517007 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f057c0565b0 con 0x7f05940cf930 2024-04-25T10:27:16.911 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:16.911 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7043,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/94806623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":94806623},{"type":"v1","addr":"172.21.15.117:6837","nonce":94806623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3691977780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3691977780},{"type":"v1","addr":"172.21.15.45:6835","nonce":3691977780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7879,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1427951733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1427951733},{"type":"v1","addr":"172.21.15.117:6835","nonce":1427951733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3861819528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3861819528},{"type":"v1","addr":"172.21.15.45:6834","nonce":3861819528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[]} 2024-04-25T10:27:16.911 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 84 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0570066f60 msgr2=0x7f0570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0570066f60 0x7f0570069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f05840ae000 tx=0x7f05840a7000 comp rx=0 tx=0).stop 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 msgr2=0x7f05941460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 0x7f05941460f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f057c00d800 tx=0x7f057c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 shutdown_connections 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0570066f60 0x7f0570069420 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f059415bc40 0x7f05940af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05940cf930 0x7f05941460f0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 --2- 172.21.15.45:0/1870517007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05940cda10 0x7f05941561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 >> 172.21.15.45:0/1870517007 conn(0x7f0594066b70 msgr2=0x7f0594158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 shutdown_connections 2024-04-25T10:27:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:16.910+0000 7f059a200640 1 -- 172.21.15.45:0/1870517007 wait complete. 2024-04-25T10:27:16.932 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:27:16.933 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:27:16.933 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:27:16.933 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:27:16.933 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:27:16.933 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:27:16.933 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:27:16.934 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:27:16.934 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:27:16.934 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:27:16.934 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:27:16.934 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:27:16.934 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:27:16.934 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:27:16.934 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:27:16.935 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:27:16.947 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:27:16.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:27:16.949 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:27:16.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:27:16.951 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:27:16.951 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:27:16.951 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:27:16.952 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:27:16.998 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:27:16.998 DEBUG:teuthology.parallel:result is None 2024-04-25T10:27:17.000 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:27:17.000 DEBUG:teuthology.parallel:result is None 2024-04-25T10:27:17.002 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:27:17.003 DEBUG:teuthology.parallel:result is None 2024-04-25T10:27:17.004 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:27:17.004 DEBUG:teuthology.parallel:result is None 2024-04-25T10:27:17.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:27:17.086 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:27:17.088 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:27:17.094 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:27:17.094 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:27:17.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f250d400640 1 Processor -- start 2024-04-25T10:27:17.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f250d400640 1 -- start start 2024-04-25T10:27:17.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f250d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25080b15b0 0x7f2508168990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:17.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f250d400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080b1af0 con 0x7f25080d1820 2024-04-25T10:27:17.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f250d400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080b1dd0 con 0x7f250815e050 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f250d400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25080b20b0 con 0x7f25080b15b0 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2507200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25080b15b0 0x7f2508168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2507200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25080b15b0 0x7f2508168990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53772/0 (socket says 172.21.15.45:53772) 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2507200640 1 -- 172.21.15.45:0/681636210 learned_addr learned my addr 172.21.15.45:0/681636210 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2507200640 1 -- 172.21.15.45:0/681636210 >> v1:172.21.15.117:6789/0 conn(0x7f250815e050 legacy=0x7f25080b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2507200640 1 -- 172.21.15.45:0/681636210 >> v1:172.21.15.45:6789/0 conn(0x7f25080d1820 legacy=0x7f25080d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2505e00640 1 -- 172.21.15.45:0/681636210 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102622591 0 0) 0x7f25080b1af0 con 0x7f25080d1820 2024-04-25T10:27:17.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.150+0000 7f2507200640 1 -- 172.21.15.45:0/681636210 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080b2390 con 0x7f25080b15b0 2024-04-25T10:27:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f2507200640 1 --2- 172.21.15.45:0/681636210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f2508168990 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2500009e50 tx=0x7f250001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a95f45b8b9c8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f2505e00640 1 -- 172.21.15.45:0/681636210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000a5770 con 0x7f25080b15b0 2024-04-25T10:27:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f2505e00640 1 -- 172.21.15.45:0/681636210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000a5910 con 0x7f25080b15b0 2024-04-25T10:27:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f2505e00640 1 -- 172.21.15.45:0/681636210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000a5ce0 con 0x7f25080b15b0 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 -- 172.21.15.45:0/681636210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 msgr2=0x7f2508168990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 --2- 172.21.15.45:0/681636210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f2508168990 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2500009e50 tx=0x7f250001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 -- 172.21.15.45:0/681636210 shutdown_connections 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 --2- 172.21.15.45:0/681636210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f2508168990 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 -- 172.21.15.45:0/681636210 >> 172.21.15.45:0/681636210 conn(0x7f250809a9d0 msgr2=0x7f250809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 -- 172.21.15.45:0/681636210 shutdown_connections 2024-04-25T10:27:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.154+0000 7f250d400640 1 -- 172.21.15.45:0/681636210 wait complete. 2024-04-25T10:27:17.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 Processor -- start 2024-04-25T10:27:17.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 -- start start 2024-04-25T10:27:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f250814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25080d1820 0x7f250806f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f250815e050 0x7f25080b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250816bda0 con 0x7f25080b15b0 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250816bc20 con 0x7f25080d1820 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250816bf20 con 0x7f250815e050 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f250814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f250814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53774/0 (socket says 172.21.15.45:53774) 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 -- 172.21.15.45:0/1118405426 learned_addr learned my addr 172.21.15.45:0/1118405426 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 -- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f250815e050 msgr2=0x7f25080b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f250815e050 0x7f25080b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 -- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25080d1820 msgr2=0x7f250806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25080d1820 0x7f250806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 -- 172.21.15.45:0/1118405426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080b4a10 con 0x7f25080b15b0 2024-04-25T10:27:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f2506800640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f250814fce0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c440 tx=0x7f24f800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8005480 con 0x7f25080b15b0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f8003600 con 0x7f25080b15b0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80039f0 con 0x7f25080b15b0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2508155170 con 0x7f25080b15b0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.158+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250806ef30 con 0x7f25080b15b0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.162+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250804ef30 con 0x7f25080b15b0 2024-04-25T10:27:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.162+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24f8003b90 con 0x7f25080b15b0 2024-04-25T10:27:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.162+0000 7f24ff400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24e4066f60 0x7f24e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.162+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f24f8084940 con 0x7f25080b15b0 2024-04-25T10:27:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.162+0000 7f24fde00640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24e4066f60 0x7f24e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.162+0000 7f24fde00640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24e4066f60 0x7f24e4069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f24f0000f70 tx=0x7f24f00079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:17.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.166+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24f80555f0 con 0x7f25080b15b0 2024-04-25T10:27:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.342+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2508153320 con 0x7f25080b15b0 2024-04-25T10:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.874+0000 7f24ff400640 1 -- 172.21.15.45:0/1118405426 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v124) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24f8059690 con 0x7f25080b15b0 2024-04-25T10:27:17.878 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24e4066f60 msgr2=0x7f24e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24e4066f60 0x7f24e4069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f24f0000f70 tx=0x7f24f00079c0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 msgr2=0x7f250814fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f250814fce0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f24f800c440 tx=0x7f24f800c900 comp rx=0 tx=0).stop 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 shutdown_connections 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f24e4066f60 0x7f24e4069420 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f250815e050 0x7f25080b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25080d1820 0x7f250806f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 --2- 172.21.15.45:0/1118405426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25080b15b0 0x7f250814fce0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 >> 172.21.15.45:0/1118405426 conn(0x7f250809a9d0 msgr2=0x7f25080b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 shutdown_connections 2024-04-25T10:27:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:17.878+0000 7f250d400640 1 -- 172.21.15.45:0/1118405426 wait complete. 2024-04-25T10:27:17.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:27:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.042+0000 7fedb6c00640 1 Processor -- start 2024-04-25T10:27:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.042+0000 7fedb6c00640 1 -- start start 2024-04-25T10:27:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedb6c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fedb015a390 0x7fedb00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedb6c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fedb00cdd90 0x7fedb00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedb6c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb015a770 con 0x7fedb015eac0 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedb6c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb00ce550 con 0x7fedb00cdd90 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedb6c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb015e000 con 0x7fedb015a390 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fedb00cdd90 0x7fedb00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fedb00cdd90 0x7fedb00ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53776/0 (socket says 172.21.15.45:53776) 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 -- 172.21.15.45:0/1553338106 learned_addr learned my addr 172.21.15.45:0/1553338106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedb4c00640 1 --1- 172.21.15.45:0/1553338106 >> v1:172.21.15.45:6789/0 conn(0x7fedb015eac0 0x7fedb0072200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44286/0 (socket says 172.21.15.45:44286) 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7fedafe00640 1 --2- 172.21.15.45:0/1553338106 >> v2:172.21.15.117:3300/0 conn(0x7fedb015a390 0x7fedb00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 -- 172.21.15.45:0/1553338106 >> v2:172.21.15.117:3300/0 conn(0x7fedb015a390 msgr2=0x7fedb00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 --2- 172.21.15.45:0/1553338106 >> v2:172.21.15.117:3300/0 conn(0x7fedb015a390 0x7fedb00cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 -- 172.21.15.45:0/1553338106 >> v1:172.21.15.45:6789/0 conn(0x7fedb015eac0 legacy=0x7fedb0072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 -- 172.21.15.45:0/1553338106 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00ce270 con 0x7fedb00cdd90 2024-04-25T10:27:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.046+0000 7feda7e00640 1 --2- 172.21.15.45:0/1553338106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb00ccb40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fedb0067ac0 tx=0x7fed9801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32e742b50e3402f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedaf400640 1 -- 172.21.15.45:0/1553338106 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed980b1070 con 0x7fedb00cdd90 2024-04-25T10:27:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedaf400640 1 -- 172.21.15.45:0/1553338106 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed980ac410 con 0x7fedb00cdd90 2024-04-25T10:27:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedaf400640 1 -- 172.21.15.45:0/1553338106 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed980ac7e0 con 0x7fedb00cdd90 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- 172.21.15.45:0/1553338106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 msgr2=0x7fedb00ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1553338106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb00ccb40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fedb0067ac0 tx=0x7fed9801c520 comp rx=0 tx=0).stop 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- 172.21.15.45:0/1553338106 shutdown_connections 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1553338106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb00ccb40 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1553338106 >> v2:172.21.15.117:3300/0 conn(0x7fedb015a390 0x7fedb00cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- 172.21.15.45:0/1553338106 >> 172.21.15.45:0/1553338106 conn(0x7fedb00b1d50 msgr2=0x7fedb00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- 172.21.15.45:0/1553338106 shutdown_connections 2024-04-25T10:27:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- 172.21.15.45:0/1553338106 wait complete. 2024-04-25T10:27:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 Processor -- start 2024-04-25T10:27:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- start start 2024-04-25T10:27:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb0155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedb015a390 0x7fedb0099960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb015eac0 0x7fedb00b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0072910 con 0x7fedb00cdd90 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00cfed0 con 0x7fedb015eac0 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb6c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0072a90 con 0x7fedb015a390 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb0155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb0155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53788/0 (socket says 172.21.15.45:53788) 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedafe00640 1 -- 172.21.15.45:0/1182709521 learned_addr learned my addr 172.21.15.45:0/1182709521 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.050+0000 7fedb4c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb015eac0 0x7fedb00b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7feda7e00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedb015a390 0x7fedb0099960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedafe00640 1 -- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedb015a390 msgr2=0x7fedb0099960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedafe00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedb015a390 0x7fedb0099960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedafe00640 1 -- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb015eac0 msgr2=0x7fedb00b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedafe00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb015eac0 0x7fedb00b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedafe00640 1 -- 172.21.15.45:0/1182709521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00b5a20 con 0x7fedb00cdd90 2024-04-25T10:27:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedb4c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb015eac0 0x7fedb00b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7feda7e00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedb015a390 0x7fedb0099960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedafe00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb0155320 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7feda000d6d0 tx=0x7feda000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0015070 con 0x7fedb00cdd90 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda0007900 con 0x7fedb00cdd90 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda000b040 con 0x7fedb00cdd90 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb0155920 con 0x7fedb00cdd90 2024-04-25T10:27:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb00b4ce0 con 0x7fedb00cdd90 2024-04-25T10:27:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb004ef30 con 0x7fedb00cdd90 2024-04-25T10:27:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feda0018070 con 0x7fedb00cdd90 2024-04-25T10:27:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.054+0000 7fedacc00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed8c066f90 0x7fed8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.058+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feda0084f60 con 0x7fedb00cdd90 2024-04-25T10:27:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.058+0000 7feda7e00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed8c066f90 0x7fed8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.058+0000 7feda7e00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed8c066f90 0x7fed8c069450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fed980ae000 tx=0x7fed98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.062+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feda00562b0 con 0x7fedb00cdd90 2024-04-25T10:27:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.278+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fedb006c9f0 con 0x7fedb00cdd90 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.278+0000 7fedacc00640 1 -- 172.21.15.45:0/1182709521 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7feda0055bb0 con 0x7fedb00cdd90 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:27:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:27:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:27:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:27:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:27:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:27:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:27:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:27:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:27:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:27:18.296 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed8c066f90 msgr2=0x7fed8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed8c066f90 0x7fed8c069450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fed980ae000 tx=0x7fed98008040 comp rx=0 tx=0).stop 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 msgr2=0x7fedb0155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb0155320 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7feda000d6d0 tx=0x7feda000db90 comp rx=0 tx=0).stop 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 shutdown_connections 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed8c066f90 0x7fed8c069450 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedb015eac0 0x7fedb00b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedb015a390 0x7fedb0099960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 --2- 172.21.15.45:0/1182709521 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedb00cdd90 0x7fedb0155320 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 >> 172.21.15.45:0/1182709521 conn(0x7fedb00b1d50 msgr2=0x7fedb00af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:18.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 shutdown_connections 2024-04-25T10:27:18.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.282+0000 7fedb6c00640 1 -- 172.21.15.45:0/1182709521 wait complete. 2024-04-25T10:27:18.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:18.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 Processor -- start 2024-04-25T10:27:18.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 -- start start 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5ec80cf930 0x7f5ec806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ec80cef90 0x7f5ec80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80b4470 con 0x7f5ec8165bb0 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80b4cd0 con 0x7f5ec80cef90 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ed0400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec8071320 con 0x7f5ec80cf930 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ec80cef90 0x7f5ec80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ec80cef90 0x7f5ec80cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53790/0 (socket says 172.21.15.45:53790) 2024-04-25T10:27:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 -- 172.21.15.45:0/1461245202 learned_addr learned my addr 172.21.15.45:0/1461245202 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecda00640 1 --2- 172.21.15.45:0/1461245202 >> v2:172.21.15.117:3300/0 conn(0x7f5ec80cf930 0x7f5ec806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 -- 172.21.15.45:0/1461245202 >> v2:172.21.15.117:3300/0 conn(0x7f5ec80cf930 msgr2=0x7f5ec806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 --2- 172.21.15.45:0/1461245202 >> v2:172.21.15.117:3300/0 conn(0x7f5ec80cf930 0x7f5ec806ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 -- 172.21.15.45:0/1461245202 >> v1:172.21.15.117:6789/0 conn(0x7f5ec8165bb0 legacy=0x7f5ec8166040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.442+0000 7f5ecd000640 1 -- 172.21.15.45:0/1461245202 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80d0250 con 0x7f5ec80cef90 2024-04-25T10:27:18.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ecd000640 1 --2- 172.21.15.45:0/1461245202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec80cfd10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80d0720 tx=0x7f5eb401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a358730b7c1ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ec7e00640 1 -- 172.21.15.45:0/1461245202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40b0070 con 0x7f5ec80cef90 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ec7e00640 1 -- 172.21.15.45:0/1461245202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb40ab600 con 0x7f5ec80cef90 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ec7e00640 1 -- 172.21.15.45:0/1461245202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40ab9d0 con 0x7f5ec80cef90 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 -- 172.21.15.45:0/1461245202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 msgr2=0x7f5ec80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 --2- 172.21.15.45:0/1461245202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec80cfd10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80d0720 tx=0x7f5eb401c450 comp rx=0 tx=0).stop 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 -- 172.21.15.45:0/1461245202 shutdown_connections 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 --2- 172.21.15.45:0/1461245202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec80cfd10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 --2- 172.21.15.45:0/1461245202 >> v2:172.21.15.117:3300/0 conn(0x7f5ec80cf930 0x7f5ec806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 -- 172.21.15.45:0/1461245202 >> 172.21.15.45:0/1461245202 conn(0x7f5ec8066b70 msgr2=0x7f5ec8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 -- 172.21.15.45:0/1461245202 shutdown_connections 2024-04-25T10:27:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.446+0000 7f5ed0400640 1 -- 172.21.15.45:0/1461245202 wait complete. 2024-04-25T10:27:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 Processor -- start 2024-04-25T10:27:18.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 -- start start 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ec80cf930 0x7f5ec8154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8165bb0 0x7f5ec81fb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8071e40 con 0x7f5ec80cef90 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80cf410 con 0x7f5ec8165bb0 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8071fc0 con 0x7f5ec80cf930 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec815e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53806/0 (socket says 172.21.15.45:53806) 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 -- 172.21.15.45:0/278151700 learned_addr learned my addr 172.21.15.45:0/278151700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecd000640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ec80cf930 0x7f5ec8154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ece400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8165bb0 0x7f5ec81fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 -- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ec80cf930 msgr2=0x7f5ec8154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ec80cf930 0x7f5ec8154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 -- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8165bb0 msgr2=0x7f5ec81fb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8165bb0 0x7f5ec81fb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 -- 172.21.15.45:0/278151700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80cc730 con 0x7f5ec80cef90 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ecda00640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec815e1c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d830 tx=0x7f5ebc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00b6c0 con 0x7f5ec80cef90 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc019070 con 0x7f5ec80cef90 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00ba60 con 0x7f5ec80cef90 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec80cc9d0 con 0x7f5ec80cef90 2024-04-25T10:27:18.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.450+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec80ccf90 con 0x7f5ec80cef90 2024-04-25T10:27:18.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.454+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec804ef30 con 0x7f5ec80cef90 2024-04-25T10:27:18.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.454+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ebc01e460 con 0x7f5ec80cef90 2024-04-25T10:27:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.454+0000 7f5ec5600640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5ea8066f60 0x7f5ea8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.454+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ebc085460 con 0x7f5ec80cef90 2024-04-25T10:27:18.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.454+0000 7f5ecd000640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5ea8066f60 0x7f5ea8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.454+0000 7f5ecd000640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5ea8066f60 0x7f5ea8069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ec806dda0 tx=0x7f5eb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.458+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ebc056720 con 0x7f5ec80cef90 2024-04-25T10:27:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.618+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ec806d4f0 con 0x7f5ec80cef90 2024-04-25T10:27:18.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ec5600640 1 -- 172.21.15.45:0/278151700 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f5ebc056020 con 0x7f5ec80cef90 2024-04-25T10:27:18.624 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:18.624 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":126,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:27:17.885919+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:18.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5ea8066f60 msgr2=0x7f5ea8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5ea8066f60 0x7f5ea8069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ec806dda0 tx=0x7f5eb4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 msgr2=0x7f5ec815e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec815e1c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d830 tx=0x7f5ebc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 shutdown_connections 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5ea8066f60 0x7f5ea8069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8165bb0 0x7f5ec81fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ec80cf930 0x7f5ec8154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 --2- 172.21.15.45:0/278151700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec80cef90 0x7f5ec815e1c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 >> 172.21.15.45:0/278151700 conn(0x7f5ec8066b70 msgr2=0x7f5ec80aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 shutdown_connections 2024-04-25T10:27:18.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.622+0000 7f5ed0400640 1 -- 172.21.15.45:0/278151700 wait complete. 2024-04-25T10:27:18.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:27:18.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:27:18.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.786+0000 7f3d4a800640 1 Processor -- start 2024-04-25T10:27:18.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.786+0000 7f3d4a800640 1 -- start start 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d4a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d44070350 0x7f3d4415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d4a800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4415b080 con 0x7f3d440aef80 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d4a800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4415adb0 con 0x7f3d440b31c0 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d4a800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4415eb60 con 0x7f3d44070350 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d3be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d44070350 0x7f3d4415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d43400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d440aef80 0x7f3d4415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44290/0 (socket says 172.21.15.45:44290) 2024-04-25T10:27:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d43400640 1 -- 172.21.15.45:0/176622821 learned_addr learned my addr 172.21.15.45:0/176622821 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d3be00640 1 -- 172.21.15.45:0/176622821 >> v1:172.21.15.117:6789/0 conn(0x7f3d440b31c0 legacy=0x7f3d440cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d3be00640 1 -- 172.21.15.45:0/176622821 >> v1:172.21.15.45:6789/0 conn(0x7f3d440aef80 legacy=0x7f3d4415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d3be00640 1 -- 172.21.15.45:0/176622821 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4415ee40 con 0x7f3d44070350 2024-04-25T10:27:18.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d42a00640 1 -- 172.21.15.45:0/176622821 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077905741 0 0) 0x7f3d4415b080 con 0x7f3d440aef80 2024-04-25T10:27:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.790+0000 7f3d3be00640 1 --2- 172.21.15.45:0/176622821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d44070350 0x7f3d4415e480 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0097c0 tx=0x7f3d2c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1386e705ad36edc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d42a00640 1 -- 172.21.15.45:0/176622821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0ab680 con 0x7f3d44070350 2024-04-25T10:27:18.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d42a00640 1 -- 172.21.15.45:0/176622821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c0b0070 con 0x7f3d44070350 2024-04-25T10:27:18.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d42a00640 1 -- 172.21.15.45:0/176622821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0aba40 con 0x7f3d44070350 2024-04-25T10:27:18.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- 172.21.15.45:0/176622821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d44070350 msgr2=0x7f3d4415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 --2- 172.21.15.45:0/176622821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d44070350 0x7f3d4415e480 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0097c0 tx=0x7f3d2c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- 172.21.15.45:0/176622821 shutdown_connections 2024-04-25T10:27:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 --2- 172.21.15.45:0/176622821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d44070350 0x7f3d4415e480 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- 172.21.15.45:0/176622821 >> 172.21.15.45:0/176622821 conn(0x7f3d440669a0 msgr2=0x7f3d44066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- 172.21.15.45:0/176622821 shutdown_connections 2024-04-25T10:27:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- 172.21.15.45:0/176622821 wait complete. 2024-04-25T10:27:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 Processor -- start 2024-04-25T10:27:18.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- start start 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d44070350 0x7f3d44153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d440aef80 0x7f3d440af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 0x7f3d4414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440cfef0 con 0x7f3d440b31c0 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440cfd70 con 0x7f3d440aef80 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d4a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d440b35d0 con 0x7f3d44070350 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d43e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 0x7f3d4414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d43e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 0x7f3d4414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53822/0 (socket says 172.21.15.45:53822) 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d43e00640 1 -- 172.21.15.45:0/2270942575 learned_addr learned my addr 172.21.15.45:0/2270942575 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d43400640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d44070350 0x7f3d44153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.794+0000 7f3d3be00640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d440aef80 0x7f3d440af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43e00640 1 -- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d44070350 msgr2=0x7f3d44153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43e00640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d44070350 0x7f3d44153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43e00640 1 -- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d440aef80 msgr2=0x7f3d440af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43e00640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d440aef80 0x7f3d440af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43e00640 1 -- 172.21.15.45:0/2270942575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d44152de0 con 0x7f3d440b31c0 2024-04-25T10:27:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43e00640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 0x7f3d4414ef70 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c730 tx=0x7f3d3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c003b80 con 0x7f3d440b31c0 2024-04-25T10:27:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c003d20 con 0x7f3d440b31c0 2024-04-25T10:27:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c00aa30 con 0x7f3d440b31c0 2024-04-25T10:27:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4414f4b0 con 0x7f3d440b31c0 2024-04-25T10:27:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d440afc90 con 0x7f3d440b31c0 2024-04-25T10:27:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4404ef30 con 0x7f3d440b31c0 2024-04-25T10:27:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d3c00abd0 con 0x7f3d440b31c0 2024-04-25T10:27:18.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d3b400640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d20066f90 0x7f3d20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:18.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3d3c084a70 con 0x7f3d440b31c0 2024-04-25T10:27:18.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.798+0000 7f3d43400640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d20066f90 0x7f3d20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:18.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.802+0000 7f3d43400640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d20066f90 0x7f3d20069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3d34007a90 tx=0x7f3d340079e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.802+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d3c0556e0 con 0x7f3d440b31c0 2024-04-25T10:27:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d440b2fa0 con 0x7f3d440b31c0 2024-04-25T10:27:18.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d3b400640 1 -- 172.21.15.45:0/2270942575 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d3c059780 con 0x7f3d440b31c0 2024-04-25T10:27:18.974 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:27:18.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d20066f90 msgr2=0x7f3d20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d20066f90 0x7f3d20069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3d34007a90 tx=0x7f3d340079e0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 msgr2=0x7f3d4414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 0x7f3d4414ef70 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c730 tx=0x7f3d3c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 shutdown_connections 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d20066f90 0x7f3d20069450 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d440b31c0 0x7f3d4414ef70 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d440aef80 0x7f3d440af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 --2- 172.21.15.45:0/2270942575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d44070350 0x7f3d44153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.970+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 >> 172.21.15.45:0/2270942575 conn(0x7f3d440669a0 msgr2=0x7f3d4415bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.974+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 shutdown_connections 2024-04-25T10:27:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:18.974+0000 7f3d4a800640 1 -- 172.21.15.45:0/2270942575 wait complete. 2024-04-25T10:27:18.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:27:19.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.130+0000 7f9fc4a00640 1 Processor -- start 2024-04-25T10:27:19.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.130+0000 7f9fc4a00640 1 -- start start 2024-04-25T10:27:19.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fc4a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9fc01550b0 0x7f9fc00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fc4a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fc015b910 0x7f9fc00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fc4a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc0071050 con 0x7f9fc00cfe70 2024-04-25T10:27:19.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fc4a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc015bd80 con 0x7f9fc015b910 2024-04-25T10:27:19.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fc4a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc015d040 con 0x7f9fc01550b0 2024-04-25T10:27:19.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fc015b910 0x7f9fc00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9fc015b910 0x7f9fc00b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53824/0 (socket says 172.21.15.45:53824) 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 -- 172.21.15.45:0/2832454014 learned_addr learned my addr 172.21.15.45:0/2832454014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbde00640 1 --2- 172.21.15.45:0/2832454014 >> v2:172.21.15.117:3300/0 conn(0x7f9fc01550b0 0x7f9fc00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 -- 172.21.15.45:0/2832454014 >> v2:172.21.15.117:3300/0 conn(0x7f9fc01550b0 msgr2=0x7f9fc00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 --2- 172.21.15.45:0/2832454014 >> v2:172.21.15.117:3300/0 conn(0x7f9fc01550b0 0x7f9fc00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 -- 172.21.15.45:0/2832454014 >> v1:172.21.15.117:6789/0 conn(0x7f9fc00cfe70 legacy=0x7f9fc015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.134+0000 7f9fbe800640 1 -- 172.21.15.45:0/2832454014 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc00712f0 con 0x7f9fc015b910 2024-04-25T10:27:19.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fbe800640 1 --2- 172.21.15.45:0/2832454014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc015b910 0x7f9fc00b3260 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009e50 tx=0x7f9fb801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689e3bfc3670fe66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:19.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fbd400640 1 -- 172.21.15.45:0/2832454014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80a57a0 con 0x7f9fc015b910 2024-04-25T10:27:19.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fbd400640 1 -- 172.21.15.45:0/2832454014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb80a5940 con 0x7f9fc015b910 2024-04-25T10:27:19.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fbd400640 1 -- 172.21.15.45:0/2832454014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80a5d10 con 0x7f9fc015b910 2024-04-25T10:27:19.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2832454014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc015b910 msgr2=0x7f9fc00b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2832454014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc015b910 0x7f9fc00b3260 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009e50 tx=0x7f9fb801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2832454014 shutdown_connections 2024-04-25T10:27:19.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2832454014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc015b910 0x7f9fc00b3260 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2832454014 >> v2:172.21.15.117:3300/0 conn(0x7f9fc01550b0 0x7f9fc00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2832454014 >> 172.21.15.45:0/2832454014 conn(0x7f9fc0066b70 msgr2=0x7f9fc0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:19.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2832454014 shutdown_connections 2024-04-25T10:27:19.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2832454014 wait complete. 2024-04-25T10:27:19.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 Processor -- start 2024-04-25T10:27:19.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.138+0000 7f9fc4a00640 1 -- start start 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fc00cfe70 0x7f9fc0157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 0x7f9fc0148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc015b910 0x7f9fc00af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc015c030 con 0x7f9fc01550b0 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc00720b0 con 0x7f9fc015b910 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc015c1b0 con 0x7f9fc00cfe70 2024-04-25T10:27:19.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 0x7f9fc0148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 0x7f9fc0148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53834/0 (socket says 172.21.15.45:53834) 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 -- 172.21.15.45:0/2732910563 learned_addr learned my addr 172.21.15.45:0/2732910563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fbde00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fc00cfe70 0x7f9fc0157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fbe800640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc015b910 0x7f9fc00af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 -- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fc00cfe70 msgr2=0x7f9fc0157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fc00cfe70 0x7f9fc0157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 -- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc015b910 msgr2=0x7f9fc00af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc015b910 0x7f9fc00af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 -- 172.21.15.45:0/2732910563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc014dac0 con 0x7f9fc01550b0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb5400640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 0x7f9fc0148e80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c420 tx=0x7f9fa800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80052d0 con 0x7f9fc01550b0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8005470 con 0x7f9fc01550b0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8007260 con 0x7f9fc01550b0 2024-04-25T10:27:19.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc01493c0 con 0x7f9fc01550b0 2024-04-25T10:27:19.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc01579f0 con 0x7f9fc01550b0 2024-04-25T10:27:19.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc004ef30 con 0x7f9fc01550b0 2024-04-25T10:27:19.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.142+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fa8007400 con 0x7f9fc01550b0 2024-04-25T10:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.146+0000 7f9fb6a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f9c066f60 0x7f9f9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.146+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9fa8084130 con 0x7f9fc01550b0 2024-04-25T10:27:19.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.146+0000 7f9fbde00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f9c066f60 0x7f9f9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.146+0000 7f9fbde00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f9c066f60 0x7f9f9c069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0007a60 tx=0x7f9fb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:19.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.150+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fc01493c0 con 0x7f9fc01550b0 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.314+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9fc006b390 con 0x7f9fc01550b0 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.314+0000 7f9fb6a00640 1 -- 172.21.15.45:0/2732910563 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9fc014dac0 con 0x7f9fc01550b0 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:19.319 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f9c066f60 msgr2=0x7f9f9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f9c066f60 0x7f9f9c069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0007a60 tx=0x7f9fb0008040 comp rx=0 tx=0).stop 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 msgr2=0x7f9fc0148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 0x7f9fc0148e80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c420 tx=0x7f9fa800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 shutdown_connections 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f9c066f60 0x7f9f9c069420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc015b910 0x7f9fc00af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9fc01550b0 0x7f9fc0148e80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 --2- 172.21.15.45:0/2732910563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fc00cfe70 0x7f9fc0157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 >> 172.21.15.45:0/2732910563 conn(0x7f9fc0066b70 msgr2=0x7f9fc015d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 shutdown_connections 2024-04-25T10:27:19.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.318+0000 7f9fc4a00640 1 -- 172.21.15.45:0/2732910563 wait complete. 2024-04-25T10:27:19.341 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:27:19.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb80000640 1 Processor -- start 2024-04-25T10:27:19.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb80000640 1 -- start start 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb80000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb780b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb80000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb780ce360 con 0x7fbb7815f540 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb80000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb780ce640 con 0x7fbb780d0640 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb80000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780cf250 con 0x7fbb7815f160 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb7cc00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbb7815f540 0x7fbb780cff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44294/0 (socket says 172.21.15.45:44294) 2024-04-25T10:27:19.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb7cc00640 1 -- 172.21.15.45:0/1408231457 learned_addr learned my addr 172.21.15.45:0/1408231457 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.482+0000 7fbb7d600640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb780b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047779468 0 0) 0x7fbb780ce360 con 0x7fbb7815f540 2024-04-25T10:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb60003530 con 0x7fbb7815f540 2024-04-25T10:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731691713 0 0) 0x7fbb780ce640 con 0x7fbb780d0640 2024-04-25T10:27:19.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb780ce360 con 0x7fbb780d0640 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799447108 0 0) 0x7fbb60003530 con 0x7fbb7815f540 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 msgr2=0x7fbb780b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb780b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 >> v1:172.21.15.117:6789/0 conn(0x7fbb780d0640 legacy=0x7fbb780cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780ce920 con 0x7fbb7815f540 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fbb64001680 con 0x7fbb7815f540 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb780b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 >> v1:172.21.15.45:6789/0 conn(0x7fbb7815f540 legacy=0x7fbb780cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb60003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb60004d50 0x7fbb600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780cf250 con 0x7fbb60003b80 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6000ba80 con 0x7fbb60004d50 2024-04-25T10:27:19.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6000bc80 con 0x7fbb7815f160 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb60003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7cc00640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7e000640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb60004d50 0x7fbb600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 -- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 msgr2=0x7fbb60004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb7815f160 0x7fbb60004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 -- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb60004d50 msgr2=0x7fbb600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb60004d50 0x7fbb600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 -- 172.21.15.45:0/1408231457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60003530 con 0x7fbb60003b80 2024-04-25T10:27:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.486+0000 7fbb7d600640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb60003f80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009cb0 tx=0x7fbb6c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ed718a19be1d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0b2070 con 0x7fbb60003b80 2024-04-25T10:27:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c0a5790 con 0x7fbb60003b80 2024-04-25T10:27:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb77e00640 1 -- 172.21.15.45:0/1408231457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c009880 con 0x7fbb60003b80 2024-04-25T10:27:19.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- 172.21.15.45:0/1408231457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 msgr2=0x7fbb60003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 --2- 172.21.15.45:0/1408231457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb60003f80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009cb0 tx=0x7fbb6c01cbe0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb7e000640 1 -- 172.21.15.45:0/1408231457 reap_dead start 2024-04-25T10:27:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- 172.21.15.45:0/1408231457 shutdown_connections 2024-04-25T10:27:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- 172.21.15.45:0/1408231457 >> 172.21.15.45:0/1408231457 conn(0x7fbb780b34a0 msgr2=0x7fbb780b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- 172.21.15.45:0/1408231457 shutdown_connections 2024-04-25T10:27:19.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- 172.21.15.45:0/1408231457 wait complete. 2024-04-25T10:27:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 Processor -- start 2024-04-25T10:27:19.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- start start 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb78153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78155940 0x7fbb78155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb780cacc0 0x7fbb780cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78167d80 con 0x7fbb60003b80 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780cd750 con 0x7fbb78155940 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.490+0000 7fbb80000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78167f00 con 0x7fbb780cacc0 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb78153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb78153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53866/0 (socket says 172.21.15.45:53866) 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 -- 172.21.15.45:0/3274590689 learned_addr learned my addr 172.21.15.45:0/3274590689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7cc00640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78155940 0x7fbb78155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7e000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb780cacc0 0x7fbb780cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 -- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb780cacc0 msgr2=0x7fbb780cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb780cacc0 0x7fbb780cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 -- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78155940 msgr2=0x7fbb78155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78155940 0x7fbb78155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 -- 172.21.15.45:0/3274590689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb781fac60 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb7d600640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb78153520 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c01c650 tx=0x7fbb60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0bc020 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c009400 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb781faf00 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0b2040 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb781fb4c0 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7804ef30 con 0x7fbb60003b80 2024-04-25T10:27:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.494+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb6c004260 con 0x7fbb60003b80 2024-04-25T10:27:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.498+0000 7fbb75600640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.498+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb6c11ef30 con 0x7fbb60003b80 2024-04-25T10:27:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.498+0000 7fbb7cc00640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.498+0000 7fbb7cc00640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb780c86b0 tx=0x7fbb64007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:19.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.498+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb6c0c6020 con 0x7fbb60003b80 2024-04-25T10:27:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.666+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb780b3da0 con 0x7fbb60003b80 2024-04-25T10:27:19.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.878+0000 7fbb75600640 1 -- 172.21.15.45:0/3274590689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb6c016920 con 0x7fbb60003b80 2024-04-25T10:27:19.883 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 msgr2=0x7fbb58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb780c86b0 tx=0x7fbb64007040 comp rx=0 tx=0).stop 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 msgr2=0x7fbb78153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb78153520 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c01c650 tx=0x7fbb60005840 comp rx=0 tx=0).stop 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 shutdown_connections 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbb58066f90 0x7fbb58069450 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb780cacc0 0x7fbb780cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb78155940 0x7fbb78155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 --2- 172.21.15.45:0/3274590689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb60003b80 0x7fbb78153520 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:19.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 >> 172.21.15.45:0/3274590689 conn(0x7fbb780b34a0 msgr2=0x7fbb7806b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:19.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 shutdown_connections 2024-04-25T10:27:19.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:19.882+0000 7fbb80000640 1 -- 172.21.15.45:0/3274590689 wait complete. 2024-04-25T10:27:19.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:27:20.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.054+0000 7f98e9800640 1 Processor -- start 2024-04-25T10:27:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.054+0000 7f98e9800640 1 -- start start 2024-04-25T10:27:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e9800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e40d0070 0x7f98e40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e9800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e415e6e0 con 0x7f98e4154f70 2024-04-25T10:27:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e9800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e415e9c0 con 0x7f98e40b39d0 2024-04-25T10:27:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e9800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e415eca0 con 0x7f98e40d0070 2024-04-25T10:27:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98da000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e40d0070 0x7f98e40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:20.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f98e4154f70 0x7f98e40cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44300/0 (socket says 172.21.15.45:44300) 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e2a00640 1 -- 172.21.15.45:0/3469630043 learned_addr learned my addr 172.21.15.45:0/3469630043 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98da000640 1 -- 172.21.15.45:0/3469630043 >> v1:172.21.15.117:6789/0 conn(0x7f98e40b39d0 legacy=0x7f98e40ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98da000640 1 -- 172.21.15.45:0/3469630043 >> v1:172.21.15.45:6789/0 conn(0x7f98e4154f70 legacy=0x7f98e40cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98da000640 1 -- 172.21.15.45:0/3469630043 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e415ef80 con 0x7f98e40d0070 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e2000640 1 -- 172.21.15.45:0/3469630043 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615760921 0 0) 0x7f98e415e6e0 con 0x7f98e4154f70 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98da000640 1 --2- 172.21.15.45:0/3469630043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e40b3490 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097c0 tx=0x7f98cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7026a62c32364800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.058+0000 7f98e2000640 1 -- 172.21.15.45:0/3469630043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0b0070 con 0x7f98e40d0070 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e2000640 1 -- 172.21.15.45:0/3469630043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc0ab600 con 0x7f98e40d0070 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e2000640 1 -- 172.21.15.45:0/3469630043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0ab9d0 con 0x7f98e40d0070 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- 172.21.15.45:0/3469630043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 msgr2=0x7f98e40b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 --2- 172.21.15.45:0/3469630043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e40b3490 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097c0 tx=0x7f98cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- 172.21.15.45:0/3469630043 shutdown_connections 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 --2- 172.21.15.45:0/3469630043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e40b3490 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- 172.21.15.45:0/3469630043 >> 172.21.15.45:0/3469630043 conn(0x7f98e4066b70 msgr2=0x7f98e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- 172.21.15.45:0/3469630043 shutdown_connections 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- 172.21.15.45:0/3469630043 wait complete. 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 Processor -- start 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- start start 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e40b39d0 0x7f98e4153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e4152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98e4154f70 0x7f98e41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4168a70 con 0x7f98e40d0070 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41688f0 con 0x7f98e40b39d0 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e9800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4168bf0 con 0x7f98e4154f70 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e4152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e4152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53884/0 (socket says 172.21.15.45:53884) 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 -- 172.21.15.45:0/535463746 learned_addr learned my addr 172.21.15.45:0/535463746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:20.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e2a00640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e40b39d0 0x7f98e4153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98e3400640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98e4154f70 0x7f98e41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 -- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98e4154f70 msgr2=0x7f98e41588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98e4154f70 0x7f98e41588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 -- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e40b39d0 msgr2=0x7f98e4153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e40b39d0 0x7f98e4153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.062+0000 7f98da000640 1 -- 172.21.15.45:0/535463746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e41592c0 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98da000640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e4152810 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097c0 tx=0x7f98cc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0bc070 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc0b1040 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e41595a0 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0b0050 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e41522f0 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e404ef30 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.066+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98cc0bf070 con 0x7f98e40d0070 2024-04-25T10:27:20.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.070+0000 7f98db400640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98c0066f60 0x7f98c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.070+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f98cc11e9a0 con 0x7f98e40d0070 2024-04-25T10:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.070+0000 7f98e2a00640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98c0066f60 0x7f98c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.070+0000 7f98e2a00640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98c0066f60 0x7f98c0069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007fd0 tx=0x7f98d4007ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:20.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.074+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98cc016090 con 0x7f98e40d0070 2024-04-25T10:27:20.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.238+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f98e4159770 con 0x7f98e40d0070 2024-04-25T10:27:20.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.882+0000 7f98db400640 1 -- 172.21.15.45:0/535463746 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) v1 ==== 170+0+0 (secure 0 0 0) 0x7f98cc0efc00 con 0x7f98e40d0070 2024-04-25T10:27:20.886 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:27:20.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98c0066f60 msgr2=0x7f98c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98c0066f60 0x7f98c0069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f98d4007fd0 tx=0x7f98d4007ba0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 msgr2=0x7f98e4152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e4152810 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097c0 tx=0x7f98cc01ace0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 shutdown_connections 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f98c0066f60 0x7f98c0069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98e4154f70 0x7f98e41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e40d0070 0x7f98e4152810 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 --2- 172.21.15.45:0/535463746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e40b39d0 0x7f98e4153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 >> 172.21.15.45:0/535463746 conn(0x7f98e4066b70 msgr2=0x7f98e40afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 shutdown_connections 2024-04-25T10:27:20.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:20.886+0000 7f98e9800640 1 -- 172.21.15.45:0/535463746 wait complete. 2024-04-25T10:27:20.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:27:21.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94c0c00640 1 Processor -- start 2024-04-25T10:27:21.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94c0c00640 1 -- start start 2024-04-25T10:27:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94c0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94bc070cc0 0x7f94bc0cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94c0c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc0ce260 con 0x7f94bc0cdd50 2024-04-25T10:27:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94c0c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc0ce540 con 0x7f94bc0710a0 2024-04-25T10:27:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94c0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0cf840 con 0x7f94bc070cc0 2024-04-25T10:27:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94baa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94bc0cdd50 0x7f94bc15b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44310/0 (socket says 172.21.15.45:44310) 2024-04-25T10:27:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94baa00640 1 -- 172.21.15.45:0/4012605015 learned_addr learned my addr 172.21.15.45:0/4012605015 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:21.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94bc070cc0 0x7f94bc0cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030304333 0 0) 0x7f94bc0ce260 con 0x7f94bc0cdd50 2024-04-25T10:27:21.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a0003530 con 0x7f94bc0cdd50 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751988905 0 0) 0x7f94a0003530 con 0x7f94bc0cdd50 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94bc070cc0 msgr2=0x7f94bc0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94bc070cc0 0x7f94bc0cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 >> v1:172.21.15.117:6789/0 conn(0x7f94bc0710a0 legacy=0x7f94bc1407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0ce820 con 0x7f94bc0cdd50 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f94a8001680 con 0x7f94bc0cdd50 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 >> v1:172.21.15.45:6789/0 conn(0x7f94bc0cdd50 legacy=0x7f94bc15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94a0003a60 0x7f94a0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0004810 0x7f94a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a00051d0 0x7f94a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0cf840 con 0x7f94a0003a60 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a000c150 con 0x7f94a00051d0 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a000c350 con 0x7f94a0004810 2024-04-25T10:27:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94bc070cc0 0x7f94bc0cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94a0003a60 0x7f94a0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bbe00640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a00051d0 0x7f94a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94baa00640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0004810 0x7f94a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 -- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0004810 msgr2=0x7f94a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0004810 0x7f94a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 -- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a00051d0 msgr2=0x7f94a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a00051d0 0x7f94a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.070+0000 7f94bb400640 1 -- 172.21.15.45:0/4012605015 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0003530 con 0x7f94a0003a60 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94bb400640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94a0003a60 0x7f94a0003e80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0d0f10 tx=0x7f94ac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8df6739c2a0b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0ac020 con 0x7f94a0003a60 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac004030 con 0x7f94a0003a60 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94ba000640 1 -- 172.21.15.45:0/4012605015 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0097f0 con 0x7f94a0003a60 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94c0c00640 1 -- 172.21.15.45:0/4012605015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94a0003a60 msgr2=0x7f94a0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94c0c00640 1 --2- 172.21.15.45:0/4012605015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94a0003a60 0x7f94a0003e80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0d0f10 tx=0x7f94ac01cab0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94bbe00640 1 -- 172.21.15.45:0/4012605015 reap_dead start 2024-04-25T10:27:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94c0c00640 1 -- 172.21.15.45:0/4012605015 shutdown_connections 2024-04-25T10:27:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94c0c00640 1 -- 172.21.15.45:0/4012605015 >> 172.21.15.45:0/4012605015 conn(0x7f94bc06b8a0 msgr2=0x7f94bc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94c0c00640 1 -- 172.21.15.45:0/4012605015 shutdown_connections 2024-04-25T10:27:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.074+0000 7f94c0c00640 1 -- 172.21.15.45:0/4012605015 wait complete. 2024-04-25T10:27:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 Processor -- start 2024-04-25T10:27:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 -- start start 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0003a60 0x7f94bc1e6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc1f3d80 0x7f94bc06d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 0x7f94bc1f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0b3a60 con 0x7f94bc1f6560 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0b38e0 con 0x7f94bc1f3d80 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0cee70 con 0x7f94a0003a60 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 0x7f94bc1f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 0x7f94bc1f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53904/0 (socket says 172.21.15.45:53904) 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 -- 172.21.15.45:0/2529864263 learned_addr learned my addr 172.21.15.45:0/2529864263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bb400640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0003a60 0x7f94bc1e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94baa00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc1f3d80 0x7f94bc06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 -- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0003a60 msgr2=0x7f94bc1e6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0003a60 0x7f94bc1e6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 -- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc1f3d80 msgr2=0x7f94bc06d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc1f3d80 0x7f94bc06d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 -- 172.21.15.45:0/2529864263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc06dc50 con 0x7f94bc1f6560 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94bbe00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 0x7f94bc1f69f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cc710 tx=0x7f94b400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b401a070 con 0x7f94bc1f6560 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4006d70 con 0x7f94bc1f6560 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4003ba0 con 0x7f94bc1f6560 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc06ded0 con 0x7f94bc1f6560 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.078+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc06e490 con 0x7f94bc1f6560 2024-04-25T10:27:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.082+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc075120 con 0x7f94bc1f6560 2024-04-25T10:27:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.082+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94b4005070 con 0x7f94bc1f6560 2024-04-25T10:27:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.082+0000 7f94b3400640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f949c066f90 0x7f949c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.082+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f94b40844b0 con 0x7f94bc1f6560 2024-04-25T10:27:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.082+0000 7f94bb400640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f949c066f90 0x7f949c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.082+0000 7f94bb400640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f949c066f90 0x7f949c069450 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0aa000 tx=0x7f94ac0046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:21.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.086+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94bc06dc50 con 0x7f94bc1f6560 2024-04-25T10:27:21.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.286+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f94bc06bca0 con 0x7f94bc1f6560 2024-04-25T10:27:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.894+0000 7f94b3400640 1 -- 172.21.15.45:0/2529864263 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7f94b4054e30 con 0x7f94bc1f6560 2024-04-25T10:27:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T10:27:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.898+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f949c066f90 msgr2=0x7f949c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.898+0000 7f94c0c00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f949c066f90 0x7f949c069450 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0aa000 tx=0x7f94ac0046d0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.898+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 msgr2=0x7f94bc1f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.898+0000 7f94c0c00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 0x7f94bc1f69f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cc710 tx=0x7f94b400fcd0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 shutdown_connections 2024-04-25T10:27:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f949c066f90 0x7f949c069450 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94bc1f6560 0x7f94bc1f69f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc1f3d80 0x7f94bc06d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 --2- 172.21.15.45:0/2529864263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a0003a60 0x7f94bc1e6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 >> 172.21.15.45:0/2529864263 conn(0x7f94bc06b8a0 msgr2=0x7f94bc0b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 shutdown_connections 2024-04-25T10:27:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:21.902+0000 7f94c0c00640 1 -- 172.21.15.45:0/2529864263 wait complete. 2024-04-25T10:27:21.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 Processor -- start 2024-04-25T10:27:22.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 -- start start 2024-04-25T10:27:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c941550b0 0x7f2c940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c940cfe70 0x7f2c9415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c94071050 con 0x7f2c9415b740 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9415d040 con 0x7f2c941550b0 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c9ba00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940712f0 con 0x7f2c940cfe70 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c99000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c941550b0 0x7f2c940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c99000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2c941550b0 0x7f2c940cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53916/0 (socket says 172.21.15.45:53916) 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.090+0000 7f2c99000640 1 -- 172.21.15.45:0/1465397304 learned_addr learned my addr 172.21.15.45:0/1465397304 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c8be00640 1 --2- 172.21.15.45:0/1465397304 >> v2:172.21.15.117:3300/0 conn(0x7f2c940cfe70 0x7f2c9415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c99000640 1 -- 172.21.15.45:0/1465397304 >> v2:172.21.15.117:3300/0 conn(0x7f2c940cfe70 msgr2=0x7f2c9415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c99000640 1 --2- 172.21.15.45:0/1465397304 >> v2:172.21.15.117:3300/0 conn(0x7f2c940cfe70 0x7f2c9415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c99000640 1 -- 172.21.15.45:0/1465397304 >> v1:172.21.15.117:6789/0 conn(0x7f2c9415b740 legacy=0x7f2c9406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c99000640 1 -- 172.21.15.45:0/1465397304 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940715d0 con 0x7f2c941550b0 2024-04-25T10:27:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c8be00640 1 --2- 172.21.15.45:0/1465397304 >> v2:172.21.15.117:3300/0 conn(0x7f2c940cfe70 0x7f2c9415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:22.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c99000640 1 --2- 172.21.15.45:0/1465397304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c940cf930 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009870 tx=0x7f2c8401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59ee79970769f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c93e00640 1 -- 172.21.15.45:0/1465397304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840b0070 con 0x7f2c941550b0 2024-04-25T10:27:22.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c93e00640 1 -- 172.21.15.45:0/1465397304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c840ab600 con 0x7f2c941550b0 2024-04-25T10:27:22.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c93e00640 1 -- 172.21.15.45:0/1465397304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840ab9d0 con 0x7f2c941550b0 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1465397304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 msgr2=0x7f2c940cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.094+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1465397304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c940cf930 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009870 tx=0x7f2c8401c520 comp rx=0 tx=0).stop 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1465397304 shutdown_connections 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1465397304 >> v2:172.21.15.117:3300/0 conn(0x7f2c940cfe70 0x7f2c9415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1465397304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c940cf930 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1465397304 >> 172.21.15.45:0/1465397304 conn(0x7f2c94066b70 msgr2=0x7f2c94066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1465397304 shutdown_connections 2024-04-25T10:27:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1465397304 wait complete. 2024-04-25T10:27:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 Processor -- start 2024-04-25T10:27:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- start start 2024-04-25T10:27:22.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c940cfe70 0x7f2c94135a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c9414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c9415b740 0x7f2c9414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415bcd0 con 0x7f2c941550b0 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415bb50 con 0x7f2c940cfe70 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c9ba00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415c030 con 0x7f2c9415b740 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c9414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c9414a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53932/0 (socket says 172.21.15.45:53932) 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 -- 172.21.15.45:0/1119789059 learned_addr learned my addr 172.21.15.45:0/1119789059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c99000640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c940cfe70 0x7f2c94135a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c99a00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c9415b740 0x7f2c9414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 -- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c9415b740 msgr2=0x7f2c9414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c9415b740 0x7f2c9414a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 -- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c940cfe70 msgr2=0x7f2c94135a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c940cfe70 0x7f2c94135a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 -- 172.21.15.45:0/1119789059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94157bd0 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c99a00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c9415b740 0x7f2c9414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.098+0000 7f2c8be00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c9414a1c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e840 tx=0x7f2c7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c017070 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c00c870 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00f040 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c94157eb0 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c941577f0 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9404ef30 con 0x7f2c941550b0 2024-04-25T10:27:22.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c7c01a070 con 0x7f2c941550b0 2024-04-25T10:27:22.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c91600640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c70066f60 0x7f2c70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c7c084ba0 con 0x7f2c941550b0 2024-04-25T10:27:22.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.102+0000 7f2c99000640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c70066f60 0x7f2c70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.106+0000 7f2c99000640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c70066f60 0x7f2c70069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2c840ad000 tx=0x7f2c840a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.106+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c7c055550 con 0x7f2c941550b0 2024-04-25T10:27:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.270+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c9414fc60 con 0x7f2c941550b0 2024-04-25T10:27:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c91600640 1 -- 172.21.15.45:0/1119789059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f2c7c0595f0 con 0x7f2c941550b0 2024-04-25T10:27:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:22.276 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":129,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:27:21.290991+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c70066f60 msgr2=0x7f2c70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c70066f60 0x7f2c70069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2c840ad000 tx=0x7f2c840a6000 comp rx=0 tx=0).stop 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 msgr2=0x7f2c9414a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c9414a1c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e840 tx=0x7f2c7c00ed00 comp rx=0 tx=0).stop 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 shutdown_connections 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c70066f60 0x7f2c70069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c9415b740 0x7f2c9414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c941550b0 0x7f2c9414a1c0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 --2- 172.21.15.45:0/1119789059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c940cfe70 0x7f2c94135a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 >> 172.21.15.45:0/1119789059 conn(0x7f2c94066b70 msgr2=0x7f2c940b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 shutdown_connections 2024-04-25T10:27:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.274+0000 7f2c9ba00640 1 -- 172.21.15.45:0/1119789059 wait complete. 2024-04-25T10:27:22.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.434+0000 7f48b9c00640 1 Processor -- start 2024-04-25T10:27:22.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.434+0000 7f48b9c00640 1 -- start start 2024-04-25T10:27:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b9c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f48b406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b9c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b40b2770 con 0x7f48b40cf930 2024-04-25T10:27:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b9c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b406c670 con 0x7f48b415bc40 2024-04-25T10:27:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b9c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b406c950 con 0x7f48b40cda10 2024-04-25T10:27:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b3400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f48b40cf930 0x7f48b40b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44320/0 (socket says 172.21.15.45:44320) 2024-04-25T10:27:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b3400640 1 -- 172.21.15.45:0/1150968025 learned_addr learned my addr 172.21.15.45:0/1150968025 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2000640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f48b406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506001387 0 0) 0x7f48b40b2770 con 0x7f48b40cf930 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4898003530 con 0x7f48b40cf930 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153399901 0 0) 0x7f4898003530 con 0x7f48b40cf930 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 msgr2=0x7f48b406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f48b406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 >> v1:172.21.15.117:6789/0 conn(0x7f48b415bc40 legacy=0x7f48b40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b406cc30 con 0x7f48b40cf930 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2000640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f48b406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f48ac001680 con 0x7f48b40cf930 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 >> v1:172.21.15.45:6789/0 conn(0x7f48b40cf930 legacy=0x7f48b40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48980039f0 0x7f4898003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f4898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4898004d50 0x7f48980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b406c950 con 0x7f48980039f0 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489800ba20 con 0x7f4898004d50 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489800bc20 con 0x7f48b40cda10 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48980039f0 0x7f4898003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2000640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f4898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b3400640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4898004d50 0x7f48980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2a00640 1 -- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 msgr2=0x7f4898004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40cda10 0x7f4898004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2a00640 1 -- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4898004d50 msgr2=0x7f48980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4898004d50 0x7f48980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b2a00640 1 -- 172.21.15.45:0/1150968025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898003530 con 0x7f48980039f0 2024-04-25T10:27:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.438+0000 7f48b3400640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4898004d50 0x7f48980051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48980039f0 0x7f4898003dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f48a4002010 tx=0x7f48a401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e9fd06fa1e77ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4001c10 con 0x7f48980039f0 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a40a8660 con 0x7f48980039f0 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b1600640 1 -- 172.21.15.45:0/1150968025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40a8a50 con 0x7f48980039f0 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 -- 172.21.15.45:0/1150968025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48980039f0 msgr2=0x7f4898003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1150968025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48980039f0 0x7f4898003dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f48a4002010 tx=0x7f48a401c900 comp rx=0 tx=0).stop 2024-04-25T10:27:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b3400640 1 -- 172.21.15.45:0/1150968025 reap_dead start 2024-04-25T10:27:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 -- 172.21.15.45:0/1150968025 shutdown_connections 2024-04-25T10:27:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 -- 172.21.15.45:0/1150968025 >> 172.21.15.45:0/1150968025 conn(0x7f48b4066b70 msgr2=0x7f48b4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 -- 172.21.15.45:0/1150968025 shutdown_connections 2024-04-25T10:27:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 -- 172.21.15.45:0/1150968025 wait complete. 2024-04-25T10:27:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 Processor -- start 2024-04-25T10:27:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.442+0000 7f48b9c00640 1 -- start start 2024-04-25T10:27:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b41f73f0 0x7f48b4161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 0x7f48b40ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40ae680 0x7f48b415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4167b50 con 0x7f48b4155200 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b41679d0 con 0x7f48b41f73f0 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4167cd0 con 0x7f48b40ae680 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 0x7f48b40ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 0x7f48b40ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53948/0 (socket says 172.21.15.45:53948) 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 -- 172.21.15.45:0/1836303432 learned_addr learned my addr 172.21.15.45:0/1836303432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b41f73f0 0x7f48b4161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b3400640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40ae680 0x7f48b415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 -- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40ae680 msgr2=0x7f48b415f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40ae680 0x7f48b415f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 -- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b41f73f0 msgr2=0x7f48b4161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b41f73f0 0x7f48b4161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 -- 172.21.15.45:0/1836303432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4155780 con 0x7f48b4155200 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b2000640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 0x7f48b40ae140 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f48a0011a30 tx=0x7f48a0011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000faf0 con 0x7f48b4155200 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a000fc90 con 0x7f48b4155200 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b40af510 con 0x7f48b4155200 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0004030 con 0x7f48b4155200 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b40afad0 con 0x7f48b4155200 2024-04-25T10:27:22.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.446+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b40aea90 con 0x7f48b4155200 2024-04-25T10:27:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.450+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48a0012040 con 0x7f48b4155200 2024-04-25T10:27:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.450+0000 7f48aaa00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4894066f60 0x7f4894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.450+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48a0085270 con 0x7f48b4155200 2024-04-25T10:27:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.450+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4894066f60 0x7f4894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.450+0000 7f48b2a00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4894066f60 0x7f4894069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f48a40af000 tx=0x7f48a40b2040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.454+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48b40aea90 con 0x7f48b4155200 2024-04-25T10:27:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.614+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48b404ef30 con 0x7f48b4155200 2024-04-25T10:27:22.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48aaa00640 1 -- 172.21.15.45:0/1836303432 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f48a0019070 con 0x7f48b4155200 2024-04-25T10:27:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:22.620 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":129,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:27:21.290991+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4894066f60 msgr2=0x7f4894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4894066f60 0x7f4894069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f48a40af000 tx=0x7f48a40b2040 comp rx=0 tx=0).stop 2024-04-25T10:27:22.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 msgr2=0x7f48b40ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 0x7f48b40ae140 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f48a0011a30 tx=0x7f48a0011e30 comp rx=0 tx=0).stop 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 shutdown_connections 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4894066f60 0x7f4894069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f48b40ae680 0x7f48b415f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48b4155200 0x7f48b40ae140 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 --2- 172.21.15.45:0/1836303432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b41f73f0 0x7f48b4161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 >> 172.21.15.45:0/1836303432 conn(0x7f48b4066b70 msgr2=0x7f48b4167270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 shutdown_connections 2024-04-25T10:27:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.618+0000 7f48b9c00640 1 -- 172.21.15.45:0/1836303432 wait complete. 2024-04-25T10:27:22.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:27:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.782+0000 7fc25ca00640 1 Processor -- start 2024-04-25T10:27:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.782+0000 7fc25ca00640 1 -- start start 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc25ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc258071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc25ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc258072020 0x7fc258070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc25ca00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc258159920 con 0x7fc258159540 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc25ca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25815c2f0 con 0x7fc258072020 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc25ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25815c5d0 con 0x7fc2580aeda0 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc258159540 0x7fc2580ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44328/0 (socket says 172.21.15.45:44328) 2024-04-25T10:27:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 -- 172.21.15.45:0/1887171537 learned_addr learned my addr 172.21.15.45:0/1887171537 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:22.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc24fe00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc258071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc256800640 1 --2- 172.21.15.45:0/1887171537 >> v2:172.21.15.117:3300/0 conn(0x7fc258072020 0x7fc258070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293415165 0 0) 0x7fc258159920 con 0x7fc258159540 2024-04-25T10:27:22.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc23c003530 con 0x7fc258159540 2024-04-25T10:27:22.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2954989345 0 0) 0x7fc23c003530 con 0x7fc258159540 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 msgr2=0x7fc258071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc258071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 >> v2:172.21.15.117:3300/0 conn(0x7fc258072020 msgr2=0x7fc258070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 --2- 172.21.15.45:0/1887171537 >> v2:172.21.15.117:3300/0 conn(0x7fc258072020 0x7fc258070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25815c8b0 con 0x7fc258159540 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fc248001680 con 0x7fc258159540 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 >> v1:172.21.15.45:6789/0 conn(0x7fc258159540 legacy=0x7fc2580ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc23c0039c0 0x7fc23c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc23c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c004d50 0x7fc23c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25815c2f0 con 0x7fc23c0039c0 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25815c5d0 con 0x7fc23c004d50 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c00b9d0 con 0x7fc2580aeda0 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc23c0039c0 0x7fc23c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc24fe00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc23c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc256800640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c004d50 0x7fc23c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 -- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 msgr2=0x7fc23c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc23c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 -- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c004d50 msgr2=0x7fc23c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c004d50 0x7fc23c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc255e00640 1 -- 172.21.15.45:0/1887171537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c003530 con 0x7fc23c0039c0 2024-04-25T10:27:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.786+0000 7fc24fe00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2580aeda0 0x7fc23c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc255e00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc23c0039c0 0x7fc23c003de0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c190 tx=0x7fc24801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9aeeeea10ec5ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248007c90 con 0x7fc23c0039c0 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2480a9020 con 0x7fc23c0039c0 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc255400640 1 -- 172.21.15.45:0/1887171537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248005b00 con 0x7fc23c0039c0 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 -- 172.21.15.45:0/1887171537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc23c0039c0 msgr2=0x7fc23c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 --2- 172.21.15.45:0/1887171537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc23c0039c0 0x7fc23c003de0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c190 tx=0x7fc24801cb90 comp rx=0 tx=0).stop 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc256800640 1 -- 172.21.15.45:0/1887171537 reap_dead start 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 -- 172.21.15.45:0/1887171537 shutdown_connections 2024-04-25T10:27:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 -- 172.21.15.45:0/1887171537 >> 172.21.15.45:0/1887171537 conn(0x7fc2580669a0 msgr2=0x7fc258066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 -- 172.21.15.45:0/1887171537 shutdown_connections 2024-04-25T10:27:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 -- 172.21.15.45:0/1887171537 wait complete. 2024-04-25T10:27:22.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 Processor -- start 2024-04-25T10:27:22.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.790+0000 7fc25ca00640 1 -- start start 2024-04-25T10:27:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c0039c0 0x7fc258141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 0x7fc2581575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc258144000 0x7fc258144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580710a0 con 0x7fc258157160 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc258072430 con 0x7fc23c0039c0 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc258071220 con 0x7fc258144000 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 0x7fc2581575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 0x7fc2581575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53954/0 (socket says 172.21.15.45:53954) 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 -- 172.21.15.45:0/4215389743 learned_addr learned my addr 172.21.15.45:0/4215389743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc255e00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c0039c0 0x7fc258141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc256800640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc258144000 0x7fc258144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 -- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc258144000 msgr2=0x7fc258144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc258144000 0x7fc258144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 -- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c0039c0 msgr2=0x7fc258141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c0039c0 0x7fc258141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 -- 172.21.15.45:0/4215389743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc258144d30 con 0x7fc258157160 2024-04-25T10:27:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc256800640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc258144000 0x7fc258144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc255e00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c0039c0 0x7fc258141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24fe00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 0x7fc2581575e0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc240011620 tx=0x7fc240011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240004220 con 0x7fc258157160 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2400043c0 con 0x7fc258157160 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240015870 con 0x7fc258157160 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2580a9540 con 0x7fc258157160 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2580a9b00 con 0x7fc258157160 2024-04-25T10:27:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.794+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25804ef30 con 0x7fc258157160 2024-04-25T10:27:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.798+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc240015a10 con 0x7fc258157160 2024-04-25T10:27:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.798+0000 7fc24e000640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc234066f60 0x7fc234069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.798+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2400852e0 con 0x7fc258157160 2024-04-25T10:27:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.798+0000 7fc255e00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc234066f60 0x7fc234069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.798+0000 7fc255e00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc234066f60 0x7fc234069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc2480aa000 tx=0x7fc248000ff0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:22.802+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc240019030 con 0x7fc258157160 2024-04-25T10:27:23.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.002+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc2580cc660 con 0x7fc258157160 2024-04-25T10:27:23.882 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:27:23.878+0000 7f1aa0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:27:23.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.922+0000 7fc24e000640 1 -- 172.21.15.45:0/4215389743 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc240056250 con 0x7fc258157160 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc234066f60 msgr2=0x7fc234069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc234066f60 0x7fc234069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc2480aa000 tx=0x7fc248000ff0 comp rx=0 tx=0).stop 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 msgr2=0x7fc2581575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 0x7fc2581575e0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fc240011620 tx=0x7fc240011ae0 comp rx=0 tx=0).stop 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 shutdown_connections 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc234066f60 0x7fc234069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc258144000 0x7fc258144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc258157160 0x7fc2581575e0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 --2- 172.21.15.45:0/4215389743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc23c0039c0 0x7fc258141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:23.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 >> 172.21.15.45:0/4215389743 conn(0x7fc2580669a0 msgr2=0x7fc258066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 shutdown_connections 2024-04-25T10:27:23.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:23.926+0000 7fc25ca00640 1 -- 172.21.15.45:0/4215389743 wait complete. 2024-04-25T10:27:23.954 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:27:23.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:27:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.102+0000 7f0667e00640 1 Processor -- start 2024-04-25T10:27:24.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.102+0000 7f0667e00640 1 -- start start 2024-04-25T10:27:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.102+0000 7f0667e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0660162020 0x7f066015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.102+0000 7f0667e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066006ac90 con 0x7f066006bd10 2024-04-25T10:27:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0667e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066006af50 con 0x7f06600cd9a0 2024-04-25T10:27:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0667e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066006b880 con 0x7f0660162020 2024-04-25T10:27:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f066006bd10 0x7f0660161910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44342/0 (socket says 172.21.15.45:44342) 2024-04-25T10:27:24.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 -- 172.21.15.45:0/1007061603 learned_addr learned my addr 172.21.15.45:0/1007061603 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0665e00640 1 --2- 172.21.15.45:0/1007061603 >> v2:172.21.15.117:3300/0 conn(0x7f0660162020 0x7f066015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286909138 0 0) 0x7f066006ac90 con 0x7f066006bd10 2024-04-25T10:27:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0648003530 con 0x7f066006bd10 2024-04-25T10:27:24.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 65256183 0 0) 0x7f0648003530 con 0x7f066006bd10 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 >> v2:172.21.15.117:3300/0 conn(0x7f0660162020 msgr2=0x7f066015d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 --2- 172.21.15.45:0/1007061603 >> v2:172.21.15.117:3300/0 conn(0x7f0660162020 0x7f066015d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 >> v1:172.21.15.117:6789/0 conn(0x7f06600cd9a0 legacy=0x7f06600d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066006b230 con 0x7f066006bd10 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f064c001680 con 0x7f066006bd10 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 >> v1:172.21.15.45:6789/0 conn(0x7f066006bd10 legacy=0x7f0660161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480039c0 0x7f0648003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004810 0x7f0648004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480051d0 0x7f064800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066006b880 con 0x7f0648004810 2024-04-25T10:27:24.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064800c080 con 0x7f06480051d0 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064800c280 con 0x7f06480039c0 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004810 0x7f0648004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0665400640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480039c0 0x7f0648003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0665e00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480051d0 0x7f064800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 -- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480039c0 msgr2=0x7f0648003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480039c0 0x7f0648003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 -- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480051d0 msgr2=0x7f064800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480051d0 0x7f064800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0664a00640 1 -- 172.21.15.45:0/1007061603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648003530 con 0x7f0648004810 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0665e00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06480051d0 0x7f064800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0665400640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06480039c0 0x7f0648003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:24.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.106+0000 7f0665e00640 1 -- 172.21.15.45:0/1007061603 reap_dead start 2024-04-25T10:27:24.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0664a00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004810 0x7f0648004c90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c210 tx=0x7f064c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec0e415987a569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c007d40 con 0x7f0648004810 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c0a9070 con 0x7f0648004810 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f065fe00640 1 -- 172.21.15.45:0/1007061603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0a5e00 con 0x7f0648004810 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 -- 172.21.15.45:0/1007061603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004810 msgr2=0x7f0648004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004810 0x7f0648004c90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c210 tx=0x7f064c01cc60 comp rx=0 tx=0).stop 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 -- 172.21.15.45:0/1007061603 shutdown_connections 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 --2- 172.21.15.45:0/1007061603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0648004810 0x7f0648004c90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 -- 172.21.15.45:0/1007061603 >> 172.21.15.45:0/1007061603 conn(0x7f06600d19a0 msgr2=0x7f06600d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 -- 172.21.15.45:0/1007061603 shutdown_connections 2024-04-25T10:27:24.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 -- 172.21.15.45:0/1007061603 wait complete. 2024-04-25T10:27:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 Processor -- start 2024-04-25T10:27:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.110+0000 7f0667e00640 1 -- start start 2024-04-25T10:27:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0648004810 0x7f06601e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 0x7f0660150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06600d0070 0x7f06600d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0660162660 con 0x7f06600c8a70 2024-04-25T10:27:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600cdd80 con 0x7f06600d0070 2024-04-25T10:27:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0660161910 con 0x7f0648004810 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 0x7f0660150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 0x7f0660150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53980/0 (socket says 172.21.15.45:53980) 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 -- 172.21.15.45:0/3684704815 learned_addr learned my addr 172.21.15.45:0/3684704815 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0665e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06600d0070 0x7f06600d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0665400640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0648004810 0x7f06601e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 -- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0648004810 msgr2=0x7f06601e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0648004810 0x7f06601e8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 -- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06600d0070 msgr2=0x7f06600d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06600d0070 0x7f06600d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 -- 172.21.15.45:0/3684704815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600c8f70 con 0x7f06600c8a70 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0665400640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0648004810 0x7f06601e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0664a00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 0x7f0660150db0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c210 tx=0x7f0648005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0a5b70 con 0x7f06600c8a70 2024-04-25T10:27:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c0a5d10 con 0x7f06600c8a70 2024-04-25T10:27:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c007380 con 0x7f06600c8a70 2024-04-25T10:27:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06600c9250 con 0x7f06600c8a70 2024-04-25T10:27:24.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06600c9810 con 0x7f06600c8a70 2024-04-25T10:27:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066004ef30 con 0x7f06600c8a70 2024-04-25T10:27:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.114+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f064c007520 con 0x7f06600c8a70 2024-04-25T10:27:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.118+0000 7f065d600640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0640066f90 0x7f0640069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.118+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f064c11eb50 con 0x7f06600c8a70 2024-04-25T10:27:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.118+0000 7f0665400640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0640066f90 0x7f0640069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.118+0000 7f0665400640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0640066f90 0x7f0640069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0654007fd0 tx=0x7f0654009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:24.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.122+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f064c016970 con 0x7f06600c8a70 2024-04-25T10:27:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.318+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f066006ac90 con 0x7f06600c8a70 2024-04-25T10:27:24.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.922+0000 7f065d600640 1 -- 172.21.15.45:0/3684704815 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f064c0efac0 con 0x7f06600c8a70 2024-04-25T10:27:24.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0640066f90 msgr2=0x7f0640069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0640066f90 0x7f0640069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0654007fd0 tx=0x7f0654009040 comp rx=0 tx=0).stop 2024-04-25T10:27:24.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 msgr2=0x7f0660150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:24.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 0x7f0660150db0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c210 tx=0x7f0648005840 comp rx=0 tx=0).stop 2024-04-25T10:27:24.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 shutdown_connections 2024-04-25T10:27:24.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0640066f90 0x7f0640069450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06600d0070 0x7f06600d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06600c8a70 0x7f0660150db0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 --2- 172.21.15.45:0/3684704815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0648004810 0x7f06601e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 >> 172.21.15.45:0/3684704815 conn(0x7f06600d19a0 msgr2=0x7f066015fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 shutdown_connections 2024-04-25T10:27:24.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:24.926+0000 7f0667e00640 1 -- 172.21.15.45:0/3684704815 wait complete. 2024-04-25T10:27:24.953 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:27:24.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:25.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbd2600640 1 Processor -- start 2024-04-25T10:27:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbd2600640 1 -- start start 2024-04-25T10:27:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbd2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbcc0d1820 0x7fcbcc0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbd2600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc0b1af0 con 0x7fcbcc0b15b0 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbd2600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc0b1dd0 con 0x7fcbcc15e050 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbd2600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0b20b0 con 0x7fcbcc0d1820 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbcc0d1820 0x7fcbcc0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcb400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcbcc0d1820 0x7fcbcc0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44730/0 (socket says 172.21.15.45:44730) 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcb400640 1 -- 172.21.15.45:0/1840088163 learned_addr learned my addr 172.21.15.45:0/1840088163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcbe00640 1 --1- 172.21.15.45:0/1840088163 >> v1:172.21.15.45:6789/0 conn(0x7fcbcc0b15b0 0x7fcbcc0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51804/0 (socket says 172.21.15.45:51804) 2024-04-25T10:27:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcb400640 1 -- 172.21.15.45:0/1840088163 >> v1:172.21.15.117:6789/0 conn(0x7fcbcc15e050 legacy=0x7fcbcc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcb400640 1 -- 172.21.15.45:0/1840088163 >> v1:172.21.15.45:6789/0 conn(0x7fcbcc0b15b0 legacy=0x7fcbcc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcaa00640 1 -- 172.21.15.45:0/1840088163 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511446563 0 0) 0x7fcbcc0b1af0 con 0x7fcbcc0b15b0 2024-04-25T10:27:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.098+0000 7fcbcb400640 1 -- 172.21.15.45:0/1840088163 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0b2390 con 0x7fcbcc0d1820 2024-04-25T10:27:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbcb400640 1 --2- 172.21.15.45:0/1840088163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc0d1c00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009e50 tx=0x7fcbbc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d48eed8cacd53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbcaa00640 1 -- 172.21.15.45:0/1840088163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0a5920 con 0x7fcbcc0d1820 2024-04-25T10:27:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbcaa00640 1 -- 172.21.15.45:0/1840088163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc0a5ac0 con 0x7fcbcc0d1820 2024-04-25T10:27:25.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbcaa00640 1 -- 172.21.15.45:0/1840088163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0ad3f0 con 0x7fcbcc0d1820 2024-04-25T10:27:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 -- 172.21.15.45:0/1840088163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 msgr2=0x7fcbcc0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1840088163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc0d1c00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009e50 tx=0x7fcbbc01ca30 comp rx=0 tx=0).stop 2024-04-25T10:27:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 -- 172.21.15.45:0/1840088163 shutdown_connections 2024-04-25T10:27:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1840088163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc0d1c00 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 -- 172.21.15.45:0/1840088163 >> 172.21.15.45:0/1840088163 conn(0x7fcbcc09a9d0 msgr2=0x7fcbcc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 -- 172.21.15.45:0/1840088163 shutdown_connections 2024-04-25T10:27:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.102+0000 7fcbd2600640 1 -- 172.21.15.45:0/1840088163 wait complete. 2024-04-25T10:27:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 Processor -- start 2024-04-25T10:27:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- start start 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbcc0b15b0 0x7fcbcc15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbcc15e050 0x7fcbcc15a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0d0f20 con 0x7fcbcc0d1820 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0d0da0 con 0x7fcbcc0b15b0 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc16b9b0 con 0x7fcbcc15e050 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44746/0 (socket says 172.21.15.45:44746) 2024-04-25T10:27:25.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 -- 172.21.15.45:0/1865456685 learned_addr learned my addr 172.21.15.45:0/1865456685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbcb400640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbcc0b15b0 0x7fcbcc15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbcbe00640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbcc15e050 0x7fcbcc15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 -- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbcc15e050 msgr2=0x7fcbcc15a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbcc15e050 0x7fcbcc15a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 -- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbcc0b15b0 msgr2=0x7fcbcc15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbcc0b15b0 0x7fcbcc15cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 -- 172.21.15.45:0/1865456685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0b57f0 con 0x7fcbcc0d1820 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc2a00640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc160880 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c390 tx=0x7fcbb400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4017070 con 0x7fcbcc0d1820 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40052d0 con 0x7fcbcc0d1820 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc0b5230 con 0x7fcbcc0d1820 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4003820 con 0x7fcbcc0d1820 2024-04-25T10:27:25.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc151800 con 0x7fcbcc0d1820 2024-04-25T10:27:25.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.106+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc04ef30 con 0x7fcbcc0d1820 2024-04-25T10:27:25.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.110+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbb401a070 con 0x7fcbcc0d1820 2024-04-25T10:27:25.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.110+0000 7fcbc3e00640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcba8066f60 0x7fcba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.110+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbb4084390 con 0x7fcbcc0d1820 2024-04-25T10:27:25.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.110+0000 7fcbcb400640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcba8066f60 0x7fcba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.110+0000 7fcbcb400640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcba8066f60 0x7fcba8069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0ae000 tx=0x7fcbbc0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.114+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbb4005480 con 0x7fcbcc0d1820 2024-04-25T10:27:25.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.310+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbcc0b0800 con 0x7fcbcc0d1820 2024-04-25T10:27:25.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.310+0000 7fcbc3e00640 1 -- 172.21.15.45:0/1865456685 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fcbb4054cb0 con 0x7fcbcc0d1820 2024-04-25T10:27:25.314 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:25.315 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcba8066f60 msgr2=0x7fcba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcba8066f60 0x7fcba8069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc0ae000 tx=0x7fcbbc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 msgr2=0x7fcbcc160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc160880 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c390 tx=0x7fcbb400c850 comp rx=0 tx=0).stop 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 shutdown_connections 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcba8066f60 0x7fcba8069420 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbcc15e050 0x7fcbcc15a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbcc0d1820 0x7fcbcc160880 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 --2- 172.21.15.45:0/1865456685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbcc0b15b0 0x7fcbcc15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 >> 172.21.15.45:0/1865456685 conn(0x7fcbcc09a9d0 msgr2=0x7fcbcc06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:25.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 shutdown_connections 2024-04-25T10:27:25.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.314+0000 7fcbd2600640 1 -- 172.21.15.45:0/1865456685 wait complete. 2024-04-25T10:27:25.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:27:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.474+0000 7f59ea000640 1 Processor -- start 2024-04-25T10:27:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.474+0000 7f59ea000640 1 -- start start 2024-04-25T10:27:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.474+0000 7f59ea000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59e415bc40 0x7f59e406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59ea000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f59e40cf930 0x7f59e40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59ea000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e40b2770 con 0x7f59e40cda10 2024-04-25T10:27:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59ea000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e406c670 con 0x7f59e415bc40 2024-04-25T10:27:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59ea000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e406c950 con 0x7f59e40cf930 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59e415bc40 0x7f59e406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59e415bc40 0x7f59e406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44760/0 (socket says 172.21.15.45:44760) 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 -- 172.21.15.45:0/801661339 learned_addr learned my addr 172.21.15.45:0/801661339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3e00640 1 --2- 172.21.15.45:0/801661339 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cf930 0x7f59e40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 -- 172.21.15.45:0/801661339 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cf930 msgr2=0x7f59e40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 --2- 172.21.15.45:0/801661339 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cf930 0x7f59e40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 -- 172.21.15.45:0/801661339 >> v1:172.21.15.117:6789/0 conn(0x7f59e40cda10 legacy=0x7f59e40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 -- 172.21.15.45:0/801661339 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e406cc30 con 0x7f59e415bc40 2024-04-25T10:27:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3e00640 1 --2- 172.21.15.45:0/801661339 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cf930 0x7f59e40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e3400640 1 --2- 172.21.15.45:0/801661339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e415bc40 0x7f59e406c090 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f59e40ae4c0 tx=0x7f59d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7981bdd883aaff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e2a00640 1 -- 172.21.15.45:0/801661339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b0070 con 0x7f59e415bc40 2024-04-25T10:27:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e2a00640 1 -- 172.21.15.45:0/801661339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40ab600 con 0x7f59e415bc40 2024-04-25T10:27:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.478+0000 7f59e2a00640 1 -- 172.21.15.45:0/801661339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40ab9d0 con 0x7f59e415bc40 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- 172.21.15.45:0/801661339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e415bc40 msgr2=0x7f59e406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 --2- 172.21.15.45:0/801661339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e415bc40 0x7f59e406c090 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f59e40ae4c0 tx=0x7f59d401c450 comp rx=0 tx=0).stop 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- 172.21.15.45:0/801661339 shutdown_connections 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 --2- 172.21.15.45:0/801661339 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cf930 0x7f59e40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 --2- 172.21.15.45:0/801661339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e415bc40 0x7f59e406c090 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- 172.21.15.45:0/801661339 >> 172.21.15.45:0/801661339 conn(0x7f59e4066b70 msgr2=0x7f59e4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- 172.21.15.45:0/801661339 shutdown_connections 2024-04-25T10:27:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- 172.21.15.45:0/801661339 wait complete. 2024-04-25T10:27:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 Processor -- start 2024-04-25T10:27:25.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- start start 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 0x7f59e40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cf930 0x7f59e415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e415bc40 0x7f59e415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40b3850 con 0x7f59e40cda10 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40b36d0 con 0x7f59e40cf930 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59ea000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40b39d0 con 0x7f59e415bc40 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59e3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 0x7f59e40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59e3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 0x7f59e40b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44772/0 (socket says 172.21.15.45:44772) 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59e3400640 1 -- 172.21.15.45:0/3844733465 learned_addr learned my addr 172.21.15.45:0/3844733465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59dbe00640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cf930 0x7f59e415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.482+0000 7f59e3e00640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e415bc40 0x7f59e415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3400640 1 -- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e415bc40 msgr2=0x7f59e415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3400640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e415bc40 0x7f59e415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3400640 1 -- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cf930 msgr2=0x7f59e415db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3400640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cf930 0x7f59e415db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3400640 1 -- 172.21.15.45:0/3844733465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e415ec90 con 0x7f59e40cda10 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3e00640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e415bc40 0x7f59e415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59e3400640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 0x7f59e40b1f40 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ae040 tx=0x7f59d40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b0040 con 0x7f59e40cda10 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40b9070 con 0x7f59e40cda10 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e415ef70 con 0x7f59e40cda10 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b38a0 con 0x7f59e40cda10 2024-04-25T10:27:25.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e415e610 con 0x7f59e40cda10 2024-04-25T10:27:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e404ef30 con 0x7f59e40cda10 2024-04-25T10:27:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59d40b3a40 con 0x7f59e40cda10 2024-04-25T10:27:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59db400640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59d411edf0 con 0x7f59e40cda10 2024-04-25T10:27:25.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.486+0000 7f59dbe00640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.490+0000 7f59dbe00640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f59e41fa7f0 tx=0x7f59cc009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.490+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59d40160d0 con 0x7f59e40cda10 2024-04-25T10:27:25.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.658+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59e40cf4c0 con 0x7f59e40cda10 2024-04-25T10:27:25.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.658+0000 7f59db400640 1 -- 172.21.15.45:0/3844733465 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f59d40eff10 con 0x7f59e40cda10 2024-04-25T10:27:25.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:25.664 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":130,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:27:22.913376+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:27:25.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 msgr2=0x7f59c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f59e41fa7f0 tx=0x7f59cc009060 comp rx=0 tx=0).stop 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 msgr2=0x7f59e40b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 0x7f59e40b1f40 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ae040 tx=0x7f59d40191f0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 shutdown_connections 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e415bc40 0x7f59e415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cf930 0x7f59e415db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 --2- 172.21.15.45:0/3844733465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cda10 0x7f59e40b1f40 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 >> 172.21.15.45:0/3844733465 conn(0x7f59e4066b70 msgr2=0x7f59e41573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 shutdown_connections 2024-04-25T10:27:25.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.662+0000 7f59ea000640 1 -- 172.21.15.45:0/3844733465 wait complete. 2024-04-25T10:27:25.686 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:27:25.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:27:25.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 Processor -- start 2024-04-25T10:27:25.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 -- start start 2024-04-25T10:27:25.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc071f70 0x7f22dc0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22dc0b2fc0 0x7f22dc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc070ed0 con 0x7f22dc15a0e0 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc0711b0 con 0x7f22dc0b2fc0 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22e1e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc071490 con 0x7f22dc071f70 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22dc0b2fc0 0x7f22dc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22dc0b2fc0 0x7f22dc06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44780/0 (socket says 172.21.15.45:44780) 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 -- 172.21.15.45:0/4116933804 learned_addr learned my addr 172.21.15.45:0/4116933804 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22d3e00640 1 --2- 172.21.15.45:0/4116933804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc071f70 0x7f22dc0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 -- 172.21.15.45:0/4116933804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc071f70 msgr2=0x7f22dc0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 --2- 172.21.15.45:0/4116933804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc071f70 0x7f22dc0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 -- 172.21.15.45:0/4116933804 >> v1:172.21.15.117:6789/0 conn(0x7f22dc15a0e0 legacy=0x7f22dc15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:25.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.826+0000 7f22db400640 1 -- 172.21.15.45:0/4116933804 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc071770 con 0x7f22dc0b2fc0 2024-04-25T10:27:25.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22db400640 1 --2- 172.21.15.45:0/4116933804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc0b2fc0 0x7f22dc06c090 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009e50 tx=0x7f22d401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4511c5a40f48618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22da000640 1 -- 172.21.15.45:0/4116933804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40a5790 con 0x7f22dc0b2fc0 2024-04-25T10:27:25.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22da000640 1 -- 172.21.15.45:0/4116933804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40a5930 con 0x7f22dc0b2fc0 2024-04-25T10:27:25.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22da000640 1 -- 172.21.15.45:0/4116933804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40a5d00 con 0x7f22dc0b2fc0 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 -- 172.21.15.45:0/4116933804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc0b2fc0 msgr2=0x7f22dc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 --2- 172.21.15.45:0/4116933804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc0b2fc0 0x7f22dc06c090 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009e50 tx=0x7f22d401ca30 comp rx=0 tx=0).stop 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 -- 172.21.15.45:0/4116933804 shutdown_connections 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 --2- 172.21.15.45:0/4116933804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc0b2fc0 0x7f22dc06c090 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 --2- 172.21.15.45:0/4116933804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc071f70 0x7f22dc0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 -- 172.21.15.45:0/4116933804 >> 172.21.15.45:0/4116933804 conn(0x7f22dc066b70 msgr2=0x7f22dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 -- 172.21.15.45:0/4116933804 shutdown_connections 2024-04-25T10:27:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 -- 172.21.15.45:0/4116933804 wait complete. 2024-04-25T10:27:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.830+0000 7f22e1e00640 1 Processor -- start 2024-04-25T10:27:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- start start 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22dc071f70 0x7f22dc14ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc0b2fc0 0x7f22dc15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 0x7f22dc15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc072350 con 0x7f22dc15a0e0 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc070b50 con 0x7f22dc071f70 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0724d0 con 0x7f22dc0b2fc0 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 0x7f22dc15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 0x7f22dc15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44790/0 (socket says 172.21.15.45:44790) 2024-04-25T10:27:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 -- 172.21.15.45:0/2920105129 learned_addr learned my addr 172.21.15.45:0/2920105129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22daa00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22dc071f70 0x7f22dc14ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22d3e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc0b2fc0 0x7f22dc15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 -- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc0b2fc0 msgr2=0x7f22dc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc0b2fc0 0x7f22dc15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 -- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22dc071f70 msgr2=0x7f22dc14ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22dc071f70 0x7f22dc14ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 -- 172.21.15.45:0/2920105129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc155ef0 con 0x7f22dc15a0e0 2024-04-25T10:27:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22db400640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 0x7f22dc15b120 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f22d40af040 tx=0x7f22d40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b8670 con 0x7f22dc15a0e0 2024-04-25T10:27:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40be070 con 0x7f22dc15a0e0 2024-04-25T10:27:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc1561e0 con 0x7f22dc15a0e0 2024-04-25T10:27:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b8a70 con 0x7f22dc15a0e0 2024-04-25T10:27:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc14b340 con 0x7f22dc15a0e0 2024-04-25T10:27:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.834+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc04ef30 con 0x7f22dc15a0e0 2024-04-25T10:27:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.838+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22d40b8c10 con 0x7f22dc15a0e0 2024-04-25T10:27:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.838+0000 7f22d3400640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b8066f60 0x7f22b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.838+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22d4120350 con 0x7f22dc15a0e0 2024-04-25T10:27:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.838+0000 7f22daa00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b8066f60 0x7f22b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.838+0000 7f22daa00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b8066f60 0x7f22b8069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0015b0 tx=0x7f22cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:25.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:25.842+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22dc1561e0 con 0x7f22dc15a0e0 2024-04-25T10:27:26.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.006+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22dc0703e0 con 0x7f22b8066f60 2024-04-25T10:27:26.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.014+0000 7f22d3400640 1 -- 172.21.15.45:0/2920105129 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f22dc0703e0 con 0x7f22b8066f60 2024-04-25T10:27:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:26.021 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b8066f60 msgr2=0x7f22b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b8066f60 0x7f22b8069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0015b0 tx=0x7f22cc009040 comp rx=0 tx=0).stop 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 msgr2=0x7f22dc15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 0x7f22dc15b120 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f22d40af040 tx=0x7f22d40197d0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 shutdown_connections 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f22b8066f60 0x7f22b8069420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22dc15a0e0 0x7f22dc15b120 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22dc0b2fc0 0x7f22dc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 --2- 172.21.15.45:0/2920105129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22dc071f70 0x7f22dc14ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 >> 172.21.15.45:0/2920105129 conn(0x7f22dc066b70 msgr2=0x7f22dc0af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 shutdown_connections 2024-04-25T10:27:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.018+0000 7f22e1e00640 1 -- 172.21.15.45:0/2920105129 wait complete. 2024-04-25T10:27:26.040 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-04-25T10:27:25.005881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427060,"kb_used_data":13820,"kb_used_omap":12,"kb_used_meta":413107,"kb_avail":754547660,"statfs":{"total":773094113280,"available":772656803840,"internally_reserved":0,"allocated":14151680,"data_stored":6801824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":423022128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.115499"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925119+0000","last_change":"2024-04-25T10:27:21.924707+0000","last_active":"2024-04-25T10:27:22.925119+0000","last_peered":"2024-04-25T10:27:22.925119+0000","last_clean":"2024-04-25T10:27:22.925119+0000","last_became_active":"2024-04-25T10:27:21.920671+0000","last_became_peered":"2024-04-25T10:27:21.920671+0000","last_unstale":"2024-04-25T10:27:22.925119+0000","last_undegraded":"2024-04-25T10:27:22.925119+0000","last_fullsized":"2024-04-25T10:27:22.925119+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:21.650682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924180+0000","last_change":"2024-04-25T10:27:21.919709+0000","last_active":"2024-04-25T10:27:22.924180+0000","last_peered":"2024-04-25T10:27:22.924180+0000","last_clean":"2024-04-25T10:27:22.924180+0000","last_became_active":"2024-04-25T10:27:21.919404+0000","last_became_peered":"2024-04-25T10:27:21.919404+0000","last_unstale":"2024-04-25T10:27:22.924180+0000","last_undegraded":"2024-04-25T10:27:22.924180+0000","last_fullsized":"2024-04-25T10:27:22.924180+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.768193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069756+0000","last_change":"2024-04-25T10:27:21.927125+0000","last_active":"2024-04-25T10:27:23.069756+0000","last_peered":"2024-04-25T10:27:23.069756+0000","last_clean":"2024-04-25T10:27:23.069756+0000","last_became_active":"2024-04-25T10:27:21.926842+0000","last_became_peered":"2024-04-25T10:27:21.926842+0000","last_unstale":"2024-04-25T10:27:23.069756+0000","last_undegraded":"2024-04-25T10:27:23.069756+0000","last_fullsized":"2024-04-25T10:27:23.069756+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:16.053117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924704+0000","last_change":"2024-04-25T10:27:21.925048+0000","last_active":"2024-04-25T10:27:22.924704+0000","last_peered":"2024-04-25T10:27:22.924704+0000","last_clean":"2024-04-25T10:27:22.924704+0000","last_became_active":"2024-04-25T10:27:21.924753+0000","last_became_peered":"2024-04-25T10:27:21.924753+0000","last_unstale":"2024-04-25T10:27:22.924704+0000","last_undegraded":"2024-04-25T10:27:22.924704+0000","last_fullsized":"2024-04-25T10:27:22.924704+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:46.204788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069272+0000","last_change":"2024-04-25T10:27:21.935596+0000","last_active":"2024-04-25T10:27:23.069272+0000","last_peered":"2024-04-25T10:27:23.069272+0000","last_clean":"2024-04-25T10:27:23.069272+0000","last_became_active":"2024-04-25T10:27:21.935340+0000","last_became_peered":"2024-04-25T10:27:21.935340+0000","last_unstale":"2024-04-25T10:27:23.069272+0000","last_undegraded":"2024-04-25T10:27:23.069272+0000","last_fullsized":"2024-04-25T10:27:23.069272+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.566400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069576+0000","last_change":"2024-04-25T10:27:21.936190+0000","last_active":"2024-04-25T10:27:23.069576+0000","last_peered":"2024-04-25T10:27:23.069576+0000","last_clean":"2024-04-25T10:27:23.069576+0000","last_became_active":"2024-04-25T10:27:21.935921+0000","last_became_peered":"2024-04-25T10:27:21.935921+0000","last_unstale":"2024-04-25T10:27:23.069576+0000","last_undegraded":"2024-04-25T10:27:23.069576+0000","last_fullsized":"2024-04-25T10:27:23.069576+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:58.331241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069069+0000","last_change":"2024-04-25T10:27:21.934099+0000","last_active":"2024-04-25T10:27:23.069069+0000","last_peered":"2024-04-25T10:27:23.069069+0000","last_clean":"2024-04-25T10:27:23.069069+0000","last_became_active":"2024-04-25T10:27:21.933715+0000","last_became_peered":"2024-04-25T10:27:21.933715+0000","last_unstale":"2024-04-25T10:27:23.069069+0000","last_undegraded":"2024-04-25T10:27:23.069069+0000","last_fullsized":"2024-04-25T10:27:23.069069+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:17.685061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.068867+0000","last_change":"2024-04-25T10:27:21.921242+0000","last_active":"2024-04-25T10:27:23.068867+0000","last_peered":"2024-04-25T10:27:23.068867+0000","last_clean":"2024-04-25T10:27:23.068867+0000","last_became_active":"2024-04-25T10:27:21.920902+0000","last_became_peered":"2024-04-25T10:27:21.920902+0000","last_unstale":"2024-04-25T10:27:23.068867+0000","last_undegraded":"2024-04-25T10:27:23.068867+0000","last_fullsized":"2024-04-25T10:27:23.068867+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:12.860794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070561+0000","last_change":"2024-04-25T10:27:21.922501+0000","last_active":"2024-04-25T10:27:23.070561+0000","last_peered":"2024-04-25T10:27:23.070561+0000","last_clean":"2024-04-25T10:27:23.070561+0000","last_became_active":"2024-04-25T10:27:21.922169+0000","last_became_peered":"2024-04-25T10:27:21.922169+0000","last_unstale":"2024-04-25T10:27:23.070561+0000","last_undegraded":"2024-04-25T10:27:23.070561+0000","last_fullsized":"2024-04-25T10:27:23.070561+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:43.589479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924365+0000","last_change":"2024-04-25T10:27:21.920170+0000","last_active":"2024-04-25T10:27:22.924365+0000","last_peered":"2024-04-25T10:27:22.924365+0000","last_clean":"2024-04-25T10:27:22.924365+0000","last_became_active":"2024-04-25T10:27:21.919870+0000","last_became_peered":"2024-04-25T10:27:21.919870+0000","last_unstale":"2024-04-25T10:27:22.924365+0000","last_undegraded":"2024-04-25T10:27:22.924365+0000","last_fullsized":"2024-04-25T10:27:22.924365+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:12.779663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924951+0000","last_change":"2024-04-25T10:27:21.921617+0000","last_active":"2024-04-25T10:27:22.924951+0000","last_peered":"2024-04-25T10:27:22.924951+0000","last_clean":"2024-04-25T10:27:22.924951+0000","last_became_active":"2024-04-25T10:27:21.921338+0000","last_became_peered":"2024-04-25T10:27:21.921338+0000","last_unstale":"2024-04-25T10:27:22.924951+0000","last_undegraded":"2024-04-25T10:27:22.924951+0000","last_fullsized":"2024-04-25T10:27:22.924951+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:26.690677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070135+0000","last_change":"2024-04-25T10:27:21.934879+0000","last_active":"2024-04-25T10:27:23.070135+0000","last_peered":"2024-04-25T10:27:23.070135+0000","last_clean":"2024-04-25T10:27:23.070135+0000","last_became_active":"2024-04-25T10:27:21.932695+0000","last_became_peered":"2024-04-25T10:27:21.932695+0000","last_unstale":"2024-04-25T10:27:23.070135+0000","last_undegraded":"2024-04-25T10:27:23.070135+0000","last_fullsized":"2024-04-25T10:27:23.070135+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:17.685061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924367+0000","last_change":"2024-04-25T10:27:21.911281+0000","last_active":"2024-04-25T10:27:22.924367+0000","last_peered":"2024-04-25T10:27:22.924367+0000","last_clean":"2024-04-25T10:27:22.924367+0000","last_became_active":"2024-04-25T10:27:21.910975+0000","last_became_peered":"2024-04-25T10:27:21.910975+0000","last_unstale":"2024-04-25T10:27:22.924367+0000","last_undegraded":"2024-04-25T10:27:22.924367+0000","last_fullsized":"2024-04-25T10:27:22.924367+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:50.169390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070666+0000","last_change":"2024-04-25T10:27:21.925746+0000","last_active":"2024-04-25T10:27:23.070666+0000","last_peered":"2024-04-25T10:27:23.070666+0000","last_clean":"2024-04-25T10:27:23.070666+0000","last_became_active":"2024-04-25T10:27:21.925461+0000","last_became_peered":"2024-04-25T10:27:21.925461+0000","last_unstale":"2024-04-25T10:27:23.070666+0000","last_undegraded":"2024-04-25T10:27:23.070666+0000","last_fullsized":"2024-04-25T10:27:23.070666+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.726829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071087+0000","last_change":"2024-04-25T10:27:21.943419+0000","last_active":"2024-04-25T10:27:23.071087+0000","last_peered":"2024-04-25T10:27:23.071087+0000","last_clean":"2024-04-25T10:27:23.071087+0000","last_became_active":"2024-04-25T10:27:21.943088+0000","last_became_peered":"2024-04-25T10:27:21.943088+0000","last_unstale":"2024-04-25T10:27:23.071087+0000","last_undegraded":"2024-04-25T10:27:23.071087+0000","last_fullsized":"2024-04-25T10:27:23.071087+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:36.321999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070802+0000","last_change":"2024-04-25T10:27:21.932986+0000","last_active":"2024-04-25T10:27:23.070802+0000","last_peered":"2024-04-25T10:27:23.070802+0000","last_clean":"2024-04-25T10:27:23.070802+0000","last_became_active":"2024-04-25T10:27:21.932646+0000","last_became_peered":"2024-04-25T10:27:21.932646+0000","last_unstale":"2024-04-25T10:27:23.070802+0000","last_undegraded":"2024-04-25T10:27:23.070802+0000","last_fullsized":"2024-04-25T10:27:23.070802+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:15.528287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070722+0000","last_change":"2024-04-25T10:27:21.922417+0000","last_active":"2024-04-25T10:27:23.070722+0000","last_peered":"2024-04-25T10:27:23.070722+0000","last_clean":"2024-04-25T10:27:23.070722+0000","last_became_active":"2024-04-25T10:27:21.922104+0000","last_became_peered":"2024-04-25T10:27:21.922104+0000","last_unstale":"2024-04-25T10:27:23.070722+0000","last_undegraded":"2024-04-25T10:27:23.070722+0000","last_fullsized":"2024-04-25T10:27:23.070722+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:37.490557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071081+0000","last_change":"2024-04-25T10:27:21.930171+0000","last_active":"2024-04-25T10:27:23.071081+0000","last_peered":"2024-04-25T10:27:23.071081+0000","last_clean":"2024-04-25T10:27:23.071081+0000","last_became_active":"2024-04-25T10:27:21.929892+0000","last_became_peered":"2024-04-25T10:27:21.929892+0000","last_unstale":"2024-04-25T10:27:23.071081+0000","last_undegraded":"2024-04-25T10:27:23.071081+0000","last_fullsized":"2024-04-25T10:27:23.071081+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:25.295312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924145+0000","last_change":"2024-04-25T10:27:21.917768+0000","last_active":"2024-04-25T10:27:22.924145+0000","last_peered":"2024-04-25T10:27:22.924145+0000","last_clean":"2024-04-25T10:27:22.924145+0000","last_became_active":"2024-04-25T10:27:21.917458+0000","last_became_peered":"2024-04-25T10:27:21.917458+0000","last_unstale":"2024-04-25T10:27:22.924145+0000","last_undegraded":"2024-04-25T10:27:22.924145+0000","last_fullsized":"2024-04-25T10:27:22.924145+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:28.294242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070549+0000","last_change":"2024-04-25T10:27:21.931196+0000","last_active":"2024-04-25T10:27:23.070549+0000","last_peered":"2024-04-25T10:27:23.070549+0000","last_clean":"2024-04-25T10:27:23.070549+0000","last_became_active":"2024-04-25T10:27:21.928240+0000","last_became_peered":"2024-04-25T10:27:21.928240+0000","last_unstale":"2024-04-25T10:27:23.070549+0000","last_undegraded":"2024-04-25T10:27:23.070549+0000","last_fullsized":"2024-04-25T10:27:23.070549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:24.539328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070056+0000","last_change":"2024-04-25T10:27:21.943154+0000","last_active":"2024-04-25T10:27:23.070056+0000","last_peered":"2024-04-25T10:27:23.070056+0000","last_clean":"2024-04-25T10:27:23.070056+0000","last_became_active":"2024-04-25T10:27:21.942883+0000","last_became_peered":"2024-04-25T10:27:21.942883+0000","last_unstale":"2024-04-25T10:27:23.070056+0000","last_undegraded":"2024-04-25T10:27:23.070056+0000","last_fullsized":"2024-04-25T10:27:23.070056+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:12.860794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924949+0000","last_change":"2024-04-25T10:27:21.914986+0000","last_active":"2024-04-25T10:27:22.924949+0000","last_peered":"2024-04-25T10:27:22.924949+0000","last_clean":"2024-04-25T10:27:22.924949+0000","last_became_active":"2024-04-25T10:27:21.914616+0000","last_became_peered":"2024-04-25T10:27:21.914616+0000","last_unstale":"2024-04-25T10:27:22.924949+0000","last_undegraded":"2024-04-25T10:27:22.924949+0000","last_fullsized":"2024-04-25T10:27:22.924949+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:14.614850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921097+0000","last_change":"2024-04-25T10:27:21.931035+0000","last_active":"2024-04-25T10:27:22.921097+0000","last_peered":"2024-04-25T10:27:22.921097+0000","last_clean":"2024-04-25T10:27:22.921097+0000","last_became_active":"2024-04-25T10:27:21.919919+0000","last_became_peered":"2024-04-25T10:27:21.919919+0000","last_unstale":"2024-04-25T10:27:22.921097+0000","last_undegraded":"2024-04-25T10:27:22.921097+0000","last_fullsized":"2024-04-25T10:27:22.921097+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:19.661310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070656+0000","last_change":"2024-04-25T10:27:21.922189+0000","last_active":"2024-04-25T10:27:23.070656+0000","last_peered":"2024-04-25T10:27:23.070656+0000","last_clean":"2024-04-25T10:27:23.070656+0000","last_became_active":"2024-04-25T10:27:21.921736+0000","last_became_peered":"2024-04-25T10:27:21.921736+0000","last_unstale":"2024-04-25T10:27:23.070656+0000","last_undegraded":"2024-04-25T10:27:23.070656+0000","last_fullsized":"2024-04-25T10:27:23.070656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:54.684599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925117+0000","last_change":"2024-04-25T10:27:21.912698+0000","last_active":"2024-04-25T10:27:22.925117+0000","last_peered":"2024-04-25T10:27:22.925117+0000","last_clean":"2024-04-25T10:27:22.925117+0000","last_became_active":"2024-04-25T10:27:21.911861+0000","last_became_peered":"2024-04-25T10:27:21.911861+0000","last_unstale":"2024-04-25T10:27:22.925117+0000","last_undegraded":"2024-04-25T10:27:22.925117+0000","last_fullsized":"2024-04-25T10:27:22.925117+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:54.580782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924550+0000","last_change":"2024-04-25T10:27:21.931937+0000","last_active":"2024-04-25T10:27:22.924550+0000","last_peered":"2024-04-25T10:27:22.924550+0000","last_clean":"2024-04-25T10:27:22.924550+0000","last_became_active":"2024-04-25T10:27:21.929168+0000","last_became_peered":"2024-04-25T10:27:21.929168+0000","last_unstale":"2024-04-25T10:27:22.924550+0000","last_undegraded":"2024-04-25T10:27:22.924550+0000","last_fullsized":"2024-04-25T10:27:22.924550+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:31.735635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071657+0000","last_change":"2024-04-25T10:27:21.933358+0000","last_active":"2024-04-25T10:27:23.071657+0000","last_peered":"2024-04-25T10:27:23.071657+0000","last_clean":"2024-04-25T10:27:23.071657+0000","last_became_active":"2024-04-25T10:27:21.932898+0000","last_became_peered":"2024-04-25T10:27:21.932898+0000","last_unstale":"2024-04-25T10:27:23.071657+0000","last_undegraded":"2024-04-25T10:27:23.071657+0000","last_fullsized":"2024-04-25T10:27:23.071657+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:46.204788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921483+0000","last_change":"2024-04-25T10:27:21.928600+0000","last_active":"2024-04-25T10:27:22.921483+0000","last_peered":"2024-04-25T10:27:22.921483+0000","last_clean":"2024-04-25T10:27:22.921483+0000","last_became_active":"2024-04-25T10:27:21.928333+0000","last_became_peered":"2024-04-25T10:27:21.928333+0000","last_unstale":"2024-04-25T10:27:22.921483+0000","last_undegraded":"2024-04-25T10:27:22.921483+0000","last_fullsized":"2024-04-25T10:27:22.921483+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:57.014104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924527+0000","last_change":"2024-04-25T10:27:21.929531+0000","last_active":"2024-04-25T10:27:22.924527+0000","last_peered":"2024-04-25T10:27:22.924527+0000","last_clean":"2024-04-25T10:27:22.924527+0000","last_became_active":"2024-04-25T10:27:21.929207+0000","last_became_peered":"2024-04-25T10:27:21.929207+0000","last_unstale":"2024-04-25T10:27:22.924527+0000","last_undegraded":"2024-04-25T10:27:22.924527+0000","last_fullsized":"2024-04-25T10:27:22.924527+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:55.628413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924634+0000","last_change":"2024-04-25T10:27:21.917236+0000","last_active":"2024-04-25T10:27:22.924634+0000","last_peered":"2024-04-25T10:27:22.924634+0000","last_clean":"2024-04-25T10:27:22.924634+0000","last_became_active":"2024-04-25T10:27:21.916914+0000","last_became_peered":"2024-04-25T10:27:21.916914+0000","last_unstale":"2024-04-25T10:27:22.924634+0000","last_undegraded":"2024-04-25T10:27:22.924634+0000","last_fullsized":"2024-04-25T10:27:22.924634+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:04.174286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921122+0000","last_change":"2024-04-25T10:27:21.932110+0000","last_active":"2024-04-25T10:27:22.921122+0000","last_peered":"2024-04-25T10:27:22.921122+0000","last_clean":"2024-04-25T10:27:22.921122+0000","last_became_active":"2024-04-25T10:27:21.931817+0000","last_became_peered":"2024-04-25T10:27:21.931817+0000","last_unstale":"2024-04-25T10:27:22.921122+0000","last_undegraded":"2024-04-25T10:27:22.921122+0000","last_fullsized":"2024-04-25T10:27:22.921122+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.229355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069092+0000","last_change":"2024-04-25T10:27:21.924775+0000","last_active":"2024-04-25T10:27:23.069092+0000","last_peered":"2024-04-25T10:27:23.069092+0000","last_clean":"2024-04-25T10:27:23.069092+0000","last_became_active":"2024-04-25T10:27:21.923142+0000","last_became_peered":"2024-04-25T10:27:21.923142+0000","last_unstale":"2024-04-25T10:27:23.069092+0000","last_undegraded":"2024-04-25T10:27:23.069092+0000","last_fullsized":"2024-04-25T10:27:23.069092+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:04.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071634+0000","last_change":"2024-04-25T10:27:21.929201+0000","last_active":"2024-04-25T10:27:23.071634+0000","last_peered":"2024-04-25T10:27:23.071634+0000","last_clean":"2024-04-25T10:27:23.071634+0000","last_became_active":"2024-04-25T10:27:21.928773+0000","last_became_peered":"2024-04-25T10:27:21.928773+0000","last_unstale":"2024-04-25T10:27:23.071634+0000","last_undegraded":"2024-04-25T10:27:23.071634+0000","last_fullsized":"2024-04-25T10:27:23.071634+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:50.169390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070006+0000","last_change":"2024-04-25T10:27:20.905609+0000","last_active":"2024-04-25T10:27:23.070006+0000","last_peered":"2024-04-25T10:27:23.070006+0000","last_clean":"2024-04-25T10:27:23.070006+0000","last_became_active":"2024-04-25T10:27:20.905324+0000","last_became_peered":"2024-04-25T10:27:20.905324+0000","last_unstale":"2024-04-25T10:27:23.070006+0000","last_undegraded":"2024-04-25T10:27:23.070006+0000","last_fullsized":"2024-04-25T10:27:23.070006+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:17.334926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070765+0000","last_change":"2024-04-25T10:27:21.921853+0000","last_active":"2024-04-25T10:27:23.070765+0000","last_peered":"2024-04-25T10:27:23.070765+0000","last_clean":"2024-04-25T10:27:23.070765+0000","last_became_active":"2024-04-25T10:27:21.921520+0000","last_became_peered":"2024-04-25T10:27:21.921520+0000","last_unstale":"2024-04-25T10:27:23.070765+0000","last_undegraded":"2024-04-25T10:27:23.070765+0000","last_fullsized":"2024-04-25T10:27:23.070765+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:12.401369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070479+0000","last_change":"2024-04-25T10:27:20.915760+0000","last_active":"2024-04-25T10:27:23.070479+0000","last_peered":"2024-04-25T10:27:23.070479+0000","last_clean":"2024-04-25T10:27:23.070479+0000","last_became_active":"2024-04-25T10:27:20.915478+0000","last_became_peered":"2024-04-25T10:27:20.915478+0000","last_unstale":"2024-04-25T10:27:23.070479+0000","last_undegraded":"2024-04-25T10:27:23.070479+0000","last_fullsized":"2024-04-25T10:27:23.070479+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:24.380220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069580+0000","last_change":"2024-04-25T10:27:21.939718+0000","last_active":"2024-04-25T10:27:23.069580+0000","last_peered":"2024-04-25T10:27:23.069580+0000","last_clean":"2024-04-25T10:27:23.069580+0000","last_became_active":"2024-04-25T10:27:21.938772+0000","last_became_peered":"2024-04-25T10:27:21.938772+0000","last_unstale":"2024-04-25T10:27:23.069580+0000","last_undegraded":"2024-04-25T10:27:23.069580+0000","last_fullsized":"2024-04-25T10:27:23.069580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:48.408886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924924+0000","last_change":"2024-04-25T10:27:20.905337+0000","last_active":"2024-04-25T10:27:22.924924+0000","last_peered":"2024-04-25T10:27:22.924924+0000","last_clean":"2024-04-25T10:27:22.924924+0000","last_became_active":"2024-04-25T10:27:20.904489+0000","last_became_peered":"2024-04-25T10:27:20.904489+0000","last_unstale":"2024-04-25T10:27:22.924924+0000","last_undegraded":"2024-04-25T10:27:22.924924+0000","last_fullsized":"2024-04-25T10:27:22.924924+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:53.847466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070571+0000","last_change":"2024-04-25T10:27:21.937554+0000","last_active":"2024-04-25T10:27:23.070571+0000","last_peered":"2024-04-25T10:27:23.070571+0000","last_clean":"2024-04-25T10:27:23.070571+0000","last_became_active":"2024-04-25T10:27:21.937186+0000","last_became_peered":"2024-04-25T10:27:21.937186+0000","last_unstale":"2024-04-25T10:27:23.070571+0000","last_undegraded":"2024-04-25T10:27:23.070571+0000","last_fullsized":"2024-04-25T10:27:23.070571+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.726829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924372+0000","last_change":"2024-04-25T10:27:20.904016+0000","last_active":"2024-04-25T10:27:22.924372+0000","last_peered":"2024-04-25T10:27:22.924372+0000","last_clean":"2024-04-25T10:27:22.924372+0000","last_became_active":"2024-04-25T10:27:20.903655+0000","last_became_peered":"2024-04-25T10:27:20.903655+0000","last_unstale":"2024-04-25T10:27:22.924372+0000","last_undegraded":"2024-04-25T10:27:22.924372+0000","last_fullsized":"2024-04-25T10:27:22.924372+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:23.749370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070619+0000","last_change":"2024-04-25T10:27:21.936187+0000","last_active":"2024-04-25T10:27:23.070619+0000","last_peered":"2024-04-25T10:27:23.070619+0000","last_clean":"2024-04-25T10:27:23.070619+0000","last_became_active":"2024-04-25T10:27:21.935755+0000","last_became_peered":"2024-04-25T10:27:21.935755+0000","last_unstale":"2024-04-25T10:27:23.070619+0000","last_undegraded":"2024-04-25T10:27:23.070619+0000","last_fullsized":"2024-04-25T10:27:23.070619+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:08.584101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924638+0000","last_change":"2024-04-25T10:27:20.897883+0000","last_active":"2024-04-25T10:27:22.924638+0000","last_peered":"2024-04-25T10:27:22.924638+0000","last_clean":"2024-04-25T10:27:22.924638+0000","last_became_active":"2024-04-25T10:27:20.897384+0000","last_became_peered":"2024-04-25T10:27:20.897384+0000","last_unstale":"2024-04-25T10:27:22.924638+0000","last_undegraded":"2024-04-25T10:27:22.924638+0000","last_fullsized":"2024-04-25T10:27:22.924638+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:46.704911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070149+0000","last_change":"2024-04-25T10:27:21.935998+0000","last_active":"2024-04-25T10:27:23.070149+0000","last_peered":"2024-04-25T10:27:23.070149+0000","last_clean":"2024-04-25T10:27:23.070149+0000","last_became_active":"2024-04-25T10:27:21.935664+0000","last_became_peered":"2024-04-25T10:27:21.935664+0000","last_unstale":"2024-04-25T10:27:23.070149+0000","last_undegraded":"2024-04-25T10:27:23.070149+0000","last_fullsized":"2024-04-25T10:27:23.070149+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.566400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070674+0000","last_change":"2024-04-25T10:27:20.910870+0000","last_active":"2024-04-25T10:27:23.070674+0000","last_peered":"2024-04-25T10:27:23.070674+0000","last_clean":"2024-04-25T10:27:23.070674+0000","last_became_active":"2024-04-25T10:27:20.910556+0000","last_became_peered":"2024-04-25T10:27:20.910556+0000","last_unstale":"2024-04-25T10:27:23.070674+0000","last_undegraded":"2024-04-25T10:27:23.070674+0000","last_fullsized":"2024-04-25T10:27:23.070674+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.593678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069632+0000","last_change":"2024-04-25T10:27:21.932133+0000","last_active":"2024-04-25T10:27:23.069632+0000","last_peered":"2024-04-25T10:27:23.069632+0000","last_clean":"2024-04-25T10:27:23.069632+0000","last_became_active":"2024-04-25T10:27:21.931861+0000","last_became_peered":"2024-04-25T10:27:21.931861+0000","last_unstale":"2024-04-25T10:27:23.069632+0000","last_undegraded":"2024-04-25T10:27:23.069632+0000","last_fullsized":"2024-04-25T10:27:23.069632+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:06.326474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069878+0000","last_change":"2024-04-25T10:27:20.910610+0000","last_active":"2024-04-25T10:27:23.069878+0000","last_peered":"2024-04-25T10:27:23.069878+0000","last_clean":"2024-04-25T10:27:23.069878+0000","last_became_active":"2024-04-25T10:27:20.910321+0000","last_became_peered":"2024-04-25T10:27:20.910321+0000","last_unstale":"2024-04-25T10:27:23.069878+0000","last_undegraded":"2024-04-25T10:27:23.069878+0000","last_fullsized":"2024-04-25T10:27:23.069878+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:51.054444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924051+0000","last_change":"2024-04-25T10:27:21.910936+0000","last_active":"2024-04-25T10:27:22.924051+0000","last_peered":"2024-04-25T10:27:22.924051+0000","last_clean":"2024-04-25T10:27:22.924051+0000","last_became_active":"2024-04-25T10:27:21.910549+0000","last_became_peered":"2024-04-25T10:27:21.910549+0000","last_unstale":"2024-04-25T10:27:22.924051+0000","last_undegraded":"2024-04-25T10:27:22.924051+0000","last_fullsized":"2024-04-25T10:27:22.924051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.447889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924136+0000","last_change":"2024-04-25T10:27:20.906207+0000","last_active":"2024-04-25T10:27:22.924136+0000","last_peered":"2024-04-25T10:27:22.924136+0000","last_clean":"2024-04-25T10:27:22.924136+0000","last_became_active":"2024-04-25T10:27:20.905124+0000","last_became_peered":"2024-04-25T10:27:20.905124+0000","last_unstale":"2024-04-25T10:27:22.924136+0000","last_undegraded":"2024-04-25T10:27:22.924136+0000","last_fullsized":"2024-04-25T10:27:22.924136+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:02.890347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925243+0000","last_change":"2024-04-25T10:27:21.928121+0000","last_active":"2024-04-25T10:27:22.925243+0000","last_peered":"2024-04-25T10:27:22.925243+0000","last_clean":"2024-04-25T10:27:22.925243+0000","last_became_active":"2024-04-25T10:27:21.924777+0000","last_became_peered":"2024-04-25T10:27:21.924777+0000","last_unstale":"2024-04-25T10:27:22.925243+0000","last_undegraded":"2024-04-25T10:27:22.925243+0000","last_fullsized":"2024-04-25T10:27:22.925243+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:55.967733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070888+0000","last_change":"2024-04-25T10:27:20.912027+0000","last_active":"2024-04-25T10:27:23.070888+0000","last_peered":"2024-04-25T10:27:23.070888+0000","last_clean":"2024-04-25T10:27:23.070888+0000","last_became_active":"2024-04-25T10:27:20.911347+0000","last_became_peered":"2024-04-25T10:27:20.911347+0000","last_unstale":"2024-04-25T10:27:23.070888+0000","last_undegraded":"2024-04-25T10:27:23.070888+0000","last_fullsized":"2024-04-25T10:27:23.070888+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:53.678274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069691+0000","last_change":"2024-04-25T10:27:21.944788+0000","last_active":"2024-04-25T10:27:23.069691+0000","last_peered":"2024-04-25T10:27:23.069691+0000","last_clean":"2024-04-25T10:27:23.069691+0000","last_became_active":"2024-04-25T10:27:21.944498+0000","last_became_peered":"2024-04-25T10:27:21.944498+0000","last_unstale":"2024-04-25T10:27:23.069691+0000","last_undegraded":"2024-04-25T10:27:23.069691+0000","last_fullsized":"2024-04-25T10:27:23.069691+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:48.990957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071075+0000","last_change":"2024-04-25T10:27:20.911101+0000","last_active":"2024-04-25T10:27:23.071075+0000","last_peered":"2024-04-25T10:27:23.071075+0000","last_clean":"2024-04-25T10:27:23.071075+0000","last_became_active":"2024-04-25T10:27:20.910775+0000","last_became_peered":"2024-04-25T10:27:20.910775+0000","last_unstale":"2024-04-25T10:27:23.071075+0000","last_undegraded":"2024-04-25T10:27:23.071075+0000","last_fullsized":"2024-04-25T10:27:23.071075+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:17.223031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070280+0000","last_change":"2024-04-25T10:27:21.940855+0000","last_active":"2024-04-25T10:27:23.070280+0000","last_peered":"2024-04-25T10:27:23.070280+0000","last_clean":"2024-04-25T10:27:23.070280+0000","last_became_active":"2024-04-25T10:27:21.940557+0000","last_became_peered":"2024-04-25T10:27:21.940557+0000","last_unstale":"2024-04-25T10:27:23.070280+0000","last_undegraded":"2024-04-25T10:27:23.070280+0000","last_fullsized":"2024-04-25T10:27:23.070280+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:33.299050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069514+0000","last_change":"2024-04-25T10:27:20.906898+0000","last_active":"2024-04-25T10:27:23.069514+0000","last_peered":"2024-04-25T10:27:23.069514+0000","last_clean":"2024-04-25T10:27:23.069514+0000","last_became_active":"2024-04-25T10:27:20.906640+0000","last_became_peered":"2024-04-25T10:27:20.906640+0000","last_unstale":"2024-04-25T10:27:23.069514+0000","last_undegraded":"2024-04-25T10:27:23.069514+0000","last_fullsized":"2024-04-25T10:27:23.069514+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:25.792952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921268+0000","last_change":"2024-04-25T10:27:21.928237+0000","last_active":"2024-04-25T10:27:22.921268+0000","last_peered":"2024-04-25T10:27:22.921268+0000","last_clean":"2024-04-25T10:27:22.921268+0000","last_became_active":"2024-04-25T10:27:21.927959+0000","last_became_peered":"2024-04-25T10:27:21.927959+0000","last_unstale":"2024-04-25T10:27:22.921268+0000","last_undegraded":"2024-04-25T10:27:22.921268+0000","last_fullsized":"2024-04-25T10:27:22.921268+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:46.346477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"129'6","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921416+0000","last_change":"2024-04-25T10:27:20.912722+0000","last_active":"2024-04-25T10:27:22.921416+0000","last_peered":"2024-04-25T10:27:22.921416+0000","last_clean":"2024-04-25T10:27:22.921416+0000","last_became_active":"2024-04-25T10:27:20.912417+0000","last_became_peered":"2024-04-25T10:27:20.912417+0000","last_unstale":"2024-04-25T10:27:22.921416+0000","last_undegraded":"2024-04-25T10:27:22.921416+0000","last_fullsized":"2024-04-25T10:27:22.921416+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:04.969380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071245+0000","last_change":"2024-04-25T10:27:21.925360+0000","last_active":"2024-04-25T10:27:23.071245+0000","last_peered":"2024-04-25T10:27:23.071245+0000","last_clean":"2024-04-25T10:27:23.071245+0000","last_became_active":"2024-04-25T10:27:21.924166+0000","last_became_peered":"2024-04-25T10:27:21.924166+0000","last_unstale":"2024-04-25T10:27:23.071245+0000","last_undegraded":"2024-04-25T10:27:23.071245+0000","last_fullsized":"2024-04-25T10:27:23.071245+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:12.404090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071067+0000","last_change":"2024-04-25T10:27:20.913074+0000","last_active":"2024-04-25T10:27:23.071067+0000","last_peered":"2024-04-25T10:27:23.071067+0000","last_clean":"2024-04-25T10:27:23.071067+0000","last_became_active":"2024-04-25T10:27:20.912740+0000","last_became_peered":"2024-04-25T10:27:20.912740+0000","last_unstale":"2024-04-25T10:27:23.071067+0000","last_undegraded":"2024-04-25T10:27:23.071067+0000","last_fullsized":"2024-04-25T10:27:23.071067+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:40.146777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070762+0000","last_change":"2024-04-25T10:27:21.928923+0000","last_active":"2024-04-25T10:27:23.070762+0000","last_peered":"2024-04-25T10:27:23.070762+0000","last_clean":"2024-04-25T10:27:23.070762+0000","last_became_active":"2024-04-25T10:27:21.928646+0000","last_became_peered":"2024-04-25T10:27:21.928646+0000","last_unstale":"2024-04-25T10:27:23.070762+0000","last_undegraded":"2024-04-25T10:27:23.070762+0000","last_fullsized":"2024-04-25T10:27:23.070762+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:54.853790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924451+0000","last_change":"2024-04-25T10:27:20.912158+0000","last_active":"2024-04-25T10:27:22.924451+0000","last_peered":"2024-04-25T10:27:22.924451+0000","last_clean":"2024-04-25T10:27:22.924451+0000","last_became_active":"2024-04-25T10:27:20.911866+0000","last_became_peered":"2024-04-25T10:27:20.911866+0000","last_unstale":"2024-04-25T10:27:22.924451+0000","last_undegraded":"2024-04-25T10:27:22.924451+0000","last_fullsized":"2024-04-25T10:27:22.924451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.292726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069823+0000","last_change":"2024-04-25T10:27:21.940800+0000","last_active":"2024-04-25T10:27:23.069823+0000","last_peered":"2024-04-25T10:27:23.069823+0000","last_clean":"2024-04-25T10:27:23.069823+0000","last_became_active":"2024-04-25T10:27:21.940487+0000","last_became_peered":"2024-04-25T10:27:21.940487+0000","last_unstale":"2024-04-25T10:27:23.069823+0000","last_undegraded":"2024-04-25T10:27:23.069823+0000","last_fullsized":"2024-04-25T10:27:23.069823+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.515339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"129'8","reported_seq":22,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069730+0000","last_change":"2024-04-25T10:27:20.909036+0000","last_active":"2024-04-25T10:27:23.069730+0000","last_peered":"2024-04-25T10:27:23.069730+0000","last_clean":"2024-04-25T10:27:23.069730+0000","last_became_active":"2024-04-25T10:27:20.908753+0000","last_became_peered":"2024-04-25T10:27:20.908753+0000","last_unstale":"2024-04-25T10:27:23.069730+0000","last_undegraded":"2024-04-25T10:27:23.069730+0000","last_fullsized":"2024-04-25T10:27:23.069730+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:23.533004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069715+0000","last_change":"2024-04-25T10:27:21.924396+0000","last_active":"2024-04-25T10:27:23.069715+0000","last_peered":"2024-04-25T10:27:23.069715+0000","last_clean":"2024-04-25T10:27:23.069715+0000","last_became_active":"2024-04-25T10:27:21.924075+0000","last_became_peered":"2024-04-25T10:27:21.924075+0000","last_unstale":"2024-04-25T10:27:23.069715+0000","last_undegraded":"2024-04-25T10:27:23.069715+0000","last_fullsized":"2024-04-25T10:27:23.069715+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:33.299050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070930+0000","last_change":"2024-04-25T10:27:20.917529+0000","last_active":"2024-04-25T10:27:23.070930+0000","last_peered":"2024-04-25T10:27:23.070930+0000","last_clean":"2024-04-25T10:27:23.070930+0000","last_became_active":"2024-04-25T10:27:20.916954+0000","last_became_peered":"2024-04-25T10:27:20.916954+0000","last_unstale":"2024-04-25T10:27:23.070930+0000","last_undegraded":"2024-04-25T10:27:23.070930+0000","last_fullsized":"2024-04-25T10:27:23.070930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:07.577776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069835+0000","last_change":"2024-04-25T10:27:21.941636+0000","last_active":"2024-04-25T10:27:23.069835+0000","last_peered":"2024-04-25T10:27:23.069835+0000","last_clean":"2024-04-25T10:27:23.069835+0000","last_became_active":"2024-04-25T10:27:21.940314+0000","last_became_peered":"2024-04-25T10:27:21.940314+0000","last_unstale":"2024-04-25T10:27:23.069835+0000","last_undegraded":"2024-04-25T10:27:23.069835+0000","last_fullsized":"2024-04-25T10:27:23.069835+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.726829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924417+0000","last_change":"2024-04-25T10:27:20.899288+0000","last_active":"2024-04-25T10:27:22.924417+0000","last_peered":"2024-04-25T10:27:22.924417+0000","last_clean":"2024-04-25T10:27:22.924417+0000","last_became_active":"2024-04-25T10:27:20.899001+0000","last_became_peered":"2024-04-25T10:27:20.899001+0000","last_unstale":"2024-04-25T10:27:22.924417+0000","last_undegraded":"2024-04-25T10:27:22.924417+0000","last_fullsized":"2024-04-25T10:27:22.924417+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:47.984633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925316+0000","last_change":"2024-04-25T10:27:21.930710+0000","last_active":"2024-04-25T10:27:22.925316+0000","last_peered":"2024-04-25T10:27:22.925316+0000","last_clean":"2024-04-25T10:27:22.925316+0000","last_became_active":"2024-04-25T10:27:21.930289+0000","last_became_peered":"2024-04-25T10:27:21.930289+0000","last_unstale":"2024-04-25T10:27:22.925316+0000","last_undegraded":"2024-04-25T10:27:22.925316+0000","last_fullsized":"2024-04-25T10:27:22.925316+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:48.477977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070252+0000","last_change":"2024-04-25T10:27:20.909811+0000","last_active":"2024-04-25T10:27:23.070252+0000","last_peered":"2024-04-25T10:27:23.070252+0000","last_clean":"2024-04-25T10:27:23.070252+0000","last_became_active":"2024-04-25T10:27:20.909469+0000","last_became_peered":"2024-04-25T10:27:20.909469+0000","last_unstale":"2024-04-25T10:27:23.070252+0000","last_undegraded":"2024-04-25T10:27:23.070252+0000","last_fullsized":"2024-04-25T10:27:23.070252+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:31.841070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924982+0000","last_change":"2024-04-25T10:27:21.914642+0000","last_active":"2024-04-25T10:27:22.924982+0000","last_peered":"2024-04-25T10:27:22.924982+0000","last_clean":"2024-04-25T10:27:22.924982+0000","last_became_active":"2024-04-25T10:27:21.914343+0000","last_became_peered":"2024-04-25T10:27:21.914343+0000","last_unstale":"2024-04-25T10:27:22.924982+0000","last_undegraded":"2024-04-25T10:27:22.924982+0000","last_fullsized":"2024-04-25T10:27:22.924982+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069396+0000","last_change":"2024-04-25T10:27:20.905577+0000","last_active":"2024-04-25T10:27:23.069396+0000","last_peered":"2024-04-25T10:27:23.069396+0000","last_clean":"2024-04-25T10:27:23.069396+0000","last_became_active":"2024-04-25T10:27:20.905273+0000","last_became_peered":"2024-04-25T10:27:20.905273+0000","last_unstale":"2024-04-25T10:27:23.069396+0000","last_undegraded":"2024-04-25T10:27:23.069396+0000","last_fullsized":"2024-04-25T10:27:23.069396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:57.324917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924819+0000","last_change":"2024-04-25T10:27:21.916592+0000","last_active":"2024-04-25T10:27:22.924819+0000","last_peered":"2024-04-25T10:27:22.924819+0000","last_clean":"2024-04-25T10:27:22.924819+0000","last_became_active":"2024-04-25T10:27:21.914998+0000","last_became_peered":"2024-04-25T10:27:21.914998+0000","last_unstale":"2024-04-25T10:27:22.924819+0000","last_undegraded":"2024-04-25T10:27:22.924819+0000","last_fullsized":"2024-04-25T10:27:22.924819+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:43.455427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069530+0000","last_change":"2024-04-25T10:27:20.910440+0000","last_active":"2024-04-25T10:27:23.069530+0000","last_peered":"2024-04-25T10:27:23.069530+0000","last_clean":"2024-04-25T10:27:23.069530+0000","last_became_active":"2024-04-25T10:27:20.910175+0000","last_became_peered":"2024-04-25T10:27:20.910175+0000","last_unstale":"2024-04-25T10:27:23.069530+0000","last_undegraded":"2024-04-25T10:27:23.069530+0000","last_fullsized":"2024-04-25T10:27:23.069530+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:55.377671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921156+0000","last_change":"2024-04-25T10:27:21.921211+0000","last_active":"2024-04-25T10:27:22.921156+0000","last_peered":"2024-04-25T10:27:22.921156+0000","last_clean":"2024-04-25T10:27:22.921156+0000","last_became_active":"2024-04-25T10:27:21.920915+0000","last_became_peered":"2024-04-25T10:27:21.920915+0000","last_unstale":"2024-04-25T10:27:22.921156+0000","last_undegraded":"2024-04-25T10:27:22.921156+0000","last_fullsized":"2024-04-25T10:27:22.921156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:08.229284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070730+0000","last_change":"2024-04-25T10:27:20.914965+0000","last_active":"2024-04-25T10:27:23.070730+0000","last_peered":"2024-04-25T10:27:23.070730+0000","last_clean":"2024-04-25T10:27:23.070730+0000","last_became_active":"2024-04-25T10:27:20.914657+0000","last_became_peered":"2024-04-25T10:27:20.914657+0000","last_unstale":"2024-04-25T10:27:23.070730+0000","last_undegraded":"2024-04-25T10:27:23.070730+0000","last_fullsized":"2024-04-25T10:27:23.070730+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:51.886072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070927+0000","last_change":"2024-04-25T10:27:21.923200+0000","last_active":"2024-04-25T10:27:23.070927+0000","last_peered":"2024-04-25T10:27:23.070927+0000","last_clean":"2024-04-25T10:27:23.070927+0000","last_became_active":"2024-04-25T10:27:21.922921+0000","last_became_peered":"2024-04-25T10:27:21.922921+0000","last_unstale":"2024-04-25T10:27:23.070927+0000","last_undegraded":"2024-04-25T10:27:23.070927+0000","last_fullsized":"2024-04-25T10:27:23.070927+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:02.469210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070596+0000","last_change":"2024-04-25T10:27:20.916877+0000","last_active":"2024-04-25T10:27:23.070596+0000","last_peered":"2024-04-25T10:27:23.070596+0000","last_clean":"2024-04-25T10:27:23.070596+0000","last_became_active":"2024-04-25T10:27:20.916201+0000","last_became_peered":"2024-04-25T10:27:20.916201+0000","last_unstale":"2024-04-25T10:27:23.070596+0000","last_undegraded":"2024-04-25T10:27:23.070596+0000","last_fullsized":"2024-04-25T10:27:23.070596+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:17.334926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069987+0000","last_change":"2024-04-25T10:27:21.925842+0000","last_active":"2024-04-25T10:27:23.069987+0000","last_peered":"2024-04-25T10:27:23.069987+0000","last_clean":"2024-04-25T10:27:23.069987+0000","last_became_active":"2024-04-25T10:27:21.925583+0000","last_became_peered":"2024-04-25T10:27:21.925583+0000","last_unstale":"2024-04-25T10:27:23.069987+0000","last_undegraded":"2024-04-25T10:27:23.069987+0000","last_fullsized":"2024-04-25T10:27:23.069987+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:05.103843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.923856+0000","last_change":"2024-04-25T10:27:20.905362+0000","last_active":"2024-04-25T10:27:22.923856+0000","last_peered":"2024-04-25T10:27:22.923856+0000","last_clean":"2024-04-25T10:27:22.923856+0000","last_became_active":"2024-04-25T10:27:20.905080+0000","last_became_peered":"2024-04-25T10:27:20.905080+0000","last_unstale":"2024-04-25T10:27:22.923856+0000","last_undegraded":"2024-04-25T10:27:22.923856+0000","last_fullsized":"2024-04-25T10:27:22.923856+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:29.560075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921506+0000","last_change":"2024-04-25T10:27:21.927047+0000","last_active":"2024-04-25T10:27:22.921506+0000","last_peered":"2024-04-25T10:27:22.921506+0000","last_clean":"2024-04-25T10:27:22.921506+0000","last_became_active":"2024-04-25T10:27:21.926408+0000","last_became_peered":"2024-04-25T10:27:21.926408+0000","last_unstale":"2024-04-25T10:27:22.921506+0000","last_undegraded":"2024-04-25T10:27:22.921506+0000","last_fullsized":"2024-04-25T10:27:22.921506+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:36.497889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071237+0000","last_change":"2024-04-25T10:27:20.916973+0000","last_active":"2024-04-25T10:27:23.071237+0000","last_peered":"2024-04-25T10:27:23.071237+0000","last_clean":"2024-04-25T10:27:23.071237+0000","last_became_active":"2024-04-25T10:27:20.916669+0000","last_became_peered":"2024-04-25T10:27:20.916669+0000","last_unstale":"2024-04-25T10:27:23.071237+0000","last_undegraded":"2024-04-25T10:27:23.071237+0000","last_fullsized":"2024-04-25T10:27:23.071237+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:07.222960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925210+0000","last_change":"2024-04-25T10:27:21.934025+0000","last_active":"2024-04-25T10:27:22.925210+0000","last_peered":"2024-04-25T10:27:22.925210+0000","last_clean":"2024-04-25T10:27:22.925210+0000","last_became_active":"2024-04-25T10:27:21.933571+0000","last_became_peered":"2024-04-25T10:27:21.933571+0000","last_unstale":"2024-04-25T10:27:22.925210+0000","last_undegraded":"2024-04-25T10:27:22.925210+0000","last_fullsized":"2024-04-25T10:27:22.925210+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:40.290064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"130'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.910525+0000","last_change":"2024-04-25T10:27:20.915852+0000","last_active":"2024-04-25T10:27:23.910525+0000","last_peered":"2024-04-25T10:27:23.910525+0000","last_clean":"2024-04-25T10:27:23.910525+0000","last_became_active":"2024-04-25T10:27:20.914929+0000","last_became_peered":"2024-04-25T10:27:20.914929+0000","last_unstale":"2024-04-25T10:27:23.910525+0000","last_undegraded":"2024-04-25T10:27:23.910525+0000","last_fullsized":"2024-04-25T10:27:23.910525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:40.146777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071479+0000","last_change":"2024-04-25T10:27:21.921903+0000","last_active":"2024-04-25T10:27:23.071479+0000","last_peered":"2024-04-25T10:27:23.071479+0000","last_clean":"2024-04-25T10:27:23.071479+0000","last_became_active":"2024-04-25T10:27:21.921598+0000","last_became_peered":"2024-04-25T10:27:21.921598+0000","last_unstale":"2024-04-25T10:27:23.071479+0000","last_undegraded":"2024-04-25T10:27:23.071479+0000","last_fullsized":"2024-04-25T10:27:23.071479+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:19.183695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070948+0000","last_change":"2024-04-25T10:27:20.917080+0000","last_active":"2024-04-25T10:27:23.070948+0000","last_peered":"2024-04-25T10:27:23.070948+0000","last_clean":"2024-04-25T10:27:23.070948+0000","last_became_active":"2024-04-25T10:27:20.916805+0000","last_became_peered":"2024-04-25T10:27:20.916805+0000","last_unstale":"2024-04-25T10:27:23.070948+0000","last_undegraded":"2024-04-25T10:27:23.070948+0000","last_fullsized":"2024-04-25T10:27:23.070948+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:18.654986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070271+0000","last_change":"2024-04-25T10:27:21.926551+0000","last_active":"2024-04-25T10:27:23.070271+0000","last_peered":"2024-04-25T10:27:23.070271+0000","last_clean":"2024-04-25T10:27:23.070271+0000","last_became_active":"2024-04-25T10:27:21.925702+0000","last_became_peered":"2024-04-25T10:27:21.925702+0000","last_unstale":"2024-04-25T10:27:23.070271+0000","last_undegraded":"2024-04-25T10:27:23.070271+0000","last_fullsized":"2024-04-25T10:27:23.070271+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:12.794314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071531+0000","last_change":"2024-04-25T10:27:20.909373+0000","last_active":"2024-04-25T10:27:23.071531+0000","last_peered":"2024-04-25T10:27:23.071531+0000","last_clean":"2024-04-25T10:27:23.071531+0000","last_became_active":"2024-04-25T10:27:20.909035+0000","last_became_peered":"2024-04-25T10:27:20.909035+0000","last_unstale":"2024-04-25T10:27:23.071531+0000","last_undegraded":"2024-04-25T10:27:23.071531+0000","last_fullsized":"2024-04-25T10:27:23.071531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:25.792952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069962+0000","last_change":"2024-04-25T10:27:21.942269+0000","last_active":"2024-04-25T10:27:23.069962+0000","last_peered":"2024-04-25T10:27:23.069962+0000","last_clean":"2024-04-25T10:27:23.069962+0000","last_became_active":"2024-04-25T10:27:21.942005+0000","last_became_peered":"2024-04-25T10:27:21.942005+0000","last_unstale":"2024-04-25T10:27:23.069962+0000","last_undegraded":"2024-04-25T10:27:23.069962+0000","last_fullsized":"2024-04-25T10:27:23.069962+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:30.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925056+0000","last_change":"2024-04-25T10:27:20.905514+0000","last_active":"2024-04-25T10:27:22.925056+0000","last_peered":"2024-04-25T10:27:22.925056+0000","last_clean":"2024-04-25T10:27:22.925056+0000","last_became_active":"2024-04-25T10:27:20.904241+0000","last_became_peered":"2024-04-25T10:27:20.904241+0000","last_unstale":"2024-04-25T10:27:22.925056+0000","last_undegraded":"2024-04-25T10:27:22.925056+0000","last_fullsized":"2024-04-25T10:27:22.925056+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.747328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070178+0000","last_change":"2024-04-25T10:27:21.928249+0000","last_active":"2024-04-25T10:27:23.070178+0000","last_peered":"2024-04-25T10:27:23.070178+0000","last_clean":"2024-04-25T10:27:23.070178+0000","last_became_active":"2024-04-25T10:27:21.927993+0000","last_became_peered":"2024-04-25T10:27:21.927993+0000","last_unstale":"2024-04-25T10:27:23.070178+0000","last_undegraded":"2024-04-25T10:27:23.070178+0000","last_fullsized":"2024-04-25T10:27:23.070178+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:48.990957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069938+0000","last_change":"2024-04-25T10:27:20.911526+0000","last_active":"2024-04-25T10:27:23.069938+0000","last_peered":"2024-04-25T10:27:23.069938+0000","last_clean":"2024-04-25T10:27:23.069938+0000","last_became_active":"2024-04-25T10:27:20.911143+0000","last_became_peered":"2024-04-25T10:27:20.911143+0000","last_unstale":"2024-04-25T10:27:23.069938+0000","last_undegraded":"2024-04-25T10:27:23.069938+0000","last_fullsized":"2024-04-25T10:27:23.069938+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:27.384578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070826+0000","last_change":"2024-04-25T10:27:21.926832+0000","last_active":"2024-04-25T10:27:23.070826+0000","last_peered":"2024-04-25T10:27:23.070826+0000","last_clean":"2024-04-25T10:27:23.070826+0000","last_became_active":"2024-04-25T10:27:21.926296+0000","last_became_peered":"2024-04-25T10:27:21.926296+0000","last_unstale":"2024-04-25T10:27:23.070826+0000","last_undegraded":"2024-04-25T10:27:23.070826+0000","last_fullsized":"2024-04-25T10:27:23.070826+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:28.390902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924566+0000","last_change":"2024-04-25T10:27:20.912078+0000","last_active":"2024-04-25T10:27:22.924566+0000","last_peered":"2024-04-25T10:27:22.924566+0000","last_clean":"2024-04-25T10:27:22.924566+0000","last_became_active":"2024-04-25T10:27:20.911765+0000","last_became_peered":"2024-04-25T10:27:20.911765+0000","last_unstale":"2024-04-25T10:27:22.924566+0000","last_undegraded":"2024-04-25T10:27:22.924566+0000","last_fullsized":"2024-04-25T10:27:22.924566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:56.720505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070665+0000","last_change":"2024-04-25T10:27:21.936864+0000","last_active":"2024-04-25T10:27:23.070665+0000","last_peered":"2024-04-25T10:27:23.070665+0000","last_clean":"2024-04-25T10:27:23.070665+0000","last_became_active":"2024-04-25T10:27:21.936577+0000","last_became_peered":"2024-04-25T10:27:21.936577+0000","last_unstale":"2024-04-25T10:27:23.070665+0000","last_undegraded":"2024-04-25T10:27:23.070665+0000","last_fullsized":"2024-04-25T10:27:23.070665+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:33.299050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070910+0000","last_change":"2024-04-25T10:27:20.910002+0000","last_active":"2024-04-25T10:27:23.070910+0000","last_peered":"2024-04-25T10:27:23.070910+0000","last_clean":"2024-04-25T10:27:23.070910+0000","last_became_active":"2024-04-25T10:27:20.908507+0000","last_became_peered":"2024-04-25T10:27:20.908507+0000","last_unstale":"2024-04-25T10:27:23.070910+0000","last_undegraded":"2024-04-25T10:27:23.070910+0000","last_fullsized":"2024-04-25T10:27:23.070910+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:07.111031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069562+0000","last_change":"2024-04-25T10:27:21.923984+0000","last_active":"2024-04-25T10:27:23.069562+0000","last_peered":"2024-04-25T10:27:23.069562+0000","last_clean":"2024-04-25T10:27:23.069562+0000","last_became_active":"2024-04-25T10:27:21.923735+0000","last_became_peered":"2024-04-25T10:27:21.923735+0000","last_unstale":"2024-04-25T10:27:23.069562+0000","last_undegraded":"2024-04-25T10:27:23.069562+0000","last_fullsized":"2024-04-25T10:27:23.069562+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:41.294316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070271+0000","last_change":"2024-04-25T10:27:20.911796+0000","last_active":"2024-04-25T10:27:23.070271+0000","last_peered":"2024-04-25T10:27:23.070271+0000","last_clean":"2024-04-25T10:27:23.070271+0000","last_became_active":"2024-04-25T10:27:20.911490+0000","last_became_peered":"2024-04-25T10:27:20.911490+0000","last_unstale":"2024-04-25T10:27:23.070271+0000","last_undegraded":"2024-04-25T10:27:23.070271+0000","last_fullsized":"2024-04-25T10:27:23.070271+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:24.288988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":377,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.920880+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:27:22.920880+0000","last_peered":"2024-04-25T10:27:22.920880+0000","last_clean":"2024-04-25T10:27:22.920880+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:27:22.920880+0000","last_undegraded":"2024-04-25T10:27:22.920880+0000","last_fullsized":"2024-04-25T10:27:22.920880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50812,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321028,"statfs":{"total":96636764160,"available":96584732672,"internally_reserved":0,"allocated":1617920,"data_stored":702855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509559,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50804,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1613824,"data_stored":702821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2220032,"data_stored":1293931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59128,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312712,"statfs":{"total":96636764160,"available":96576217088,"internally_reserved":0,"allocated":1617920,"data_stored":702891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63296,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308544,"statfs":{"total":96636764160,"available":96571949056,"internally_reserved":0,"allocated":1622016,"data_stored":703493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55568,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316272,"statfs":{"total":96636764160,"available":96579862528,"internally_reserved":0,"allocated":2224128,"data_stored":1294823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1634304,"data_stored":704489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59120,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312720,"statfs":{"total":96636764160,"available":96576225280,"internally_reserved":0,"allocated":1601536,"data_stored":696521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:27:26.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:27:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.218+0000 7f58c8000640 1 Processor -- start 2024-04-25T10:27:26.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.218+0000 7f58c8000640 1 -- start start 2024-04-25T10:27:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c8000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c015df60 0x7f58c00cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c8000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c006aea0 0x7f58c006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c8000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c00ccaa0 con 0x7f58c00d0180 2024-04-25T10:27:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c8000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c015e340 con 0x7f58c015df60 2024-04-25T10:27:26.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c8000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00ccd80 con 0x7f58c006aea0 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c015df60 0x7f58c00cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f58c015df60 0x7f58c00cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44802/0 (socket says 172.21.15.45:44802) 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 -- 172.21.15.45:0/3523966219 learned_addr learned my addr 172.21.15.45:0/3523966219 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c4c00640 1 --1- 172.21.15.45:0/3523966219 >> v1:172.21.15.45:6789/0 conn(0x7f58c00d0180 0x7f58c00d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51816/0 (socket says 172.21.15.45:51816) 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c6000640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c006aea0 0x7f58c006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 -- 172.21.15.45:0/3523966219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c006aea0 msgr2=0x7f58c006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c006aea0 0x7f58c006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 -- 172.21.15.45:0/3523966219 >> v1:172.21.15.45:6789/0 conn(0x7f58c00d0180 legacy=0x7f58c00d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c5600640 1 -- 172.21.15.45:0/3523966219 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00cd060 con 0x7f58c015df60 2024-04-25T10:27:26.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.222+0000 7f58c6000640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c006aea0 0x7f58c006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:26.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c5600640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c015df60 0x7f58c00cfc40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1b71cc0b2b9da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58bfe00640 1 -- 172.21.15.45:0/3523966219 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40b1070 con 0x7f58c015df60 2024-04-25T10:27:26.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58bfe00640 1 -- 172.21.15.45:0/3523966219 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40ac620 con 0x7f58c015df60 2024-04-25T10:27:26.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58bfe00640 1 -- 172.21.15.45:0/3523966219 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ac9f0 con 0x7f58c015df60 2024-04-25T10:27:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 -- 172.21.15.45:0/3523966219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c015df60 msgr2=0x7f58c00cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c015df60 0x7f58c00cfc40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009870 tx=0x7f58b401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 -- 172.21.15.45:0/3523966219 shutdown_connections 2024-04-25T10:27:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c006aea0 0x7f58c006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 --2- 172.21.15.45:0/3523966219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c015df60 0x7f58c00cfc40 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 -- 172.21.15.45:0/3523966219 >> 172.21.15.45:0/3523966219 conn(0x7f58c00d1910 msgr2=0x7f58c00d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 -- 172.21.15.45:0/3523966219 shutdown_connections 2024-04-25T10:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 -- 172.21.15.45:0/3523966219 wait complete. 2024-04-25T10:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 Processor -- start 2024-04-25T10:27:26.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 -- start start 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c006aea0 0x7f58c014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.226+0000 7f58c8000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 0x7f58c0157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c015df60 0x7f58c0157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00d0ff0 con 0x7f58c00d0180 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00d0e70 con 0x7f58c006aea0 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c016aa20 con 0x7f58c015df60 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 0x7f58c0157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 0x7f58c0157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44810/0 (socket says 172.21.15.45:44810) 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 -- 172.21.15.45:0/3722520126 learned_addr learned my addr 172.21.15.45:0/3722520126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c5600640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c006aea0 0x7f58c014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c6000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c015df60 0x7f58c0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 -- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c015df60 msgr2=0x7f58c0157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c015df60 0x7f58c0157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 -- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c006aea0 msgr2=0x7f58c014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c006aea0 0x7f58c014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 -- 172.21.15.45:0/3722520126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c014e2f0 con 0x7f58c00d0180 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c4c00640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 0x7f58c0157210 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c420 tx=0x7f58ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0074e0 con 0x7f58c00d0180 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac007680 con 0x7f58c00d0180 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004a30 con 0x7f58c00d0180 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c014e5d0 con 0x7f58c00d0180 2024-04-25T10:27:26.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c014eb90 con 0x7f58c00d0180 2024-04-25T10:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c004ef30 con 0x7f58c00d0180 2024-04-25T10:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.230+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58ac01a020 con 0x7f58c00d0180 2024-04-25T10:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.234+0000 7f58bd600640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f58a0066f60 0x7f58a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.234+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58ac084890 con 0x7f58c00d0180 2024-04-25T10:27:26.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.234+0000 7f58c5600640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f58a0066f60 0x7f58a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.234+0000 7f58c5600640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f58a0066f60 0x7f58a0069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f58b40ae000 tx=0x7f58b40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.238+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58ac055890 con 0x7f58c00d0180 2024-04-25T10:27:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.394+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58c014e090 con 0x7f58a0066f60 2024-04-25T10:27:26.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.402+0000 7f58bd600640 1 -- 172.21.15.45:0/3722520126 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f58b40ac680 con 0x7f58a0066f60 2024-04-25T10:27:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:26.410 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f58a0066f60 msgr2=0x7f58a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f58a0066f60 0x7f58a0069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f58b40ae000 tx=0x7f58b40a7000 comp rx=0 tx=0).stop 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 msgr2=0x7f58c0157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 0x7f58c0157210 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c420 tx=0x7f58ac00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 shutdown_connections 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f58a0066f60 0x7f58a0069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58c015df60 0x7f58c0157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58c00d0180 0x7f58c0157210 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 --2- 172.21.15.45:0/3722520126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58c006aea0 0x7f58c014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 >> 172.21.15.45:0/3722520126 conn(0x7f58c00d1910 msgr2=0x7f58c0153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 shutdown_connections 2024-04-25T10:27:26.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.406+0000 7f58c8000640 1 -- 172.21.15.45:0/3722520126 wait complete. 2024-04-25T10:27:26.429 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-04-25T10:27:25.005881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427060,"kb_used_data":13820,"kb_used_omap":12,"kb_used_meta":413107,"kb_avail":754547660,"statfs":{"total":773094113280,"available":772656803840,"internally_reserved":0,"allocated":14151680,"data_stored":6801824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":423022128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.115499"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925119+0000","last_change":"2024-04-25T10:27:21.924707+0000","last_active":"2024-04-25T10:27:22.925119+0000","last_peered":"2024-04-25T10:27:22.925119+0000","last_clean":"2024-04-25T10:27:22.925119+0000","last_became_active":"2024-04-25T10:27:21.920671+0000","last_became_peered":"2024-04-25T10:27:21.920671+0000","last_unstale":"2024-04-25T10:27:22.925119+0000","last_undegraded":"2024-04-25T10:27:22.925119+0000","last_fullsized":"2024-04-25T10:27:22.925119+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:21.650682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924180+0000","last_change":"2024-04-25T10:27:21.919709+0000","last_active":"2024-04-25T10:27:22.924180+0000","last_peered":"2024-04-25T10:27:22.924180+0000","last_clean":"2024-04-25T10:27:22.924180+0000","last_became_active":"2024-04-25T10:27:21.919404+0000","last_became_peered":"2024-04-25T10:27:21.919404+0000","last_unstale":"2024-04-25T10:27:22.924180+0000","last_undegraded":"2024-04-25T10:27:22.924180+0000","last_fullsized":"2024-04-25T10:27:22.924180+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.768193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069756+0000","last_change":"2024-04-25T10:27:21.927125+0000","last_active":"2024-04-25T10:27:23.069756+0000","last_peered":"2024-04-25T10:27:23.069756+0000","last_clean":"2024-04-25T10:27:23.069756+0000","last_became_active":"2024-04-25T10:27:21.926842+0000","last_became_peered":"2024-04-25T10:27:21.926842+0000","last_unstale":"2024-04-25T10:27:23.069756+0000","last_undegraded":"2024-04-25T10:27:23.069756+0000","last_fullsized":"2024-04-25T10:27:23.069756+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:16.053117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924704+0000","last_change":"2024-04-25T10:27:21.925048+0000","last_active":"2024-04-25T10:27:22.924704+0000","last_peered":"2024-04-25T10:27:22.924704+0000","last_clean":"2024-04-25T10:27:22.924704+0000","last_became_active":"2024-04-25T10:27:21.924753+0000","last_became_peered":"2024-04-25T10:27:21.924753+0000","last_unstale":"2024-04-25T10:27:22.924704+0000","last_undegraded":"2024-04-25T10:27:22.924704+0000","last_fullsized":"2024-04-25T10:27:22.924704+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:46.204788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069272+0000","last_change":"2024-04-25T10:27:21.935596+0000","last_active":"2024-04-25T10:27:23.069272+0000","last_peered":"2024-04-25T10:27:23.069272+0000","last_clean":"2024-04-25T10:27:23.069272+0000","last_became_active":"2024-04-25T10:27:21.935340+0000","last_became_peered":"2024-04-25T10:27:21.935340+0000","last_unstale":"2024-04-25T10:27:23.069272+0000","last_undegraded":"2024-04-25T10:27:23.069272+0000","last_fullsized":"2024-04-25T10:27:23.069272+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.566400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069576+0000","last_change":"2024-04-25T10:27:21.936190+0000","last_active":"2024-04-25T10:27:23.069576+0000","last_peered":"2024-04-25T10:27:23.069576+0000","last_clean":"2024-04-25T10:27:23.069576+0000","last_became_active":"2024-04-25T10:27:21.935921+0000","last_became_peered":"2024-04-25T10:27:21.935921+0000","last_unstale":"2024-04-25T10:27:23.069576+0000","last_undegraded":"2024-04-25T10:27:23.069576+0000","last_fullsized":"2024-04-25T10:27:23.069576+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:58.331241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069069+0000","last_change":"2024-04-25T10:27:21.934099+0000","last_active":"2024-04-25T10:27:23.069069+0000","last_peered":"2024-04-25T10:27:23.069069+0000","last_clean":"2024-04-25T10:27:23.069069+0000","last_became_active":"2024-04-25T10:27:21.933715+0000","last_became_peered":"2024-04-25T10:27:21.933715+0000","last_unstale":"2024-04-25T10:27:23.069069+0000","last_undegraded":"2024-04-25T10:27:23.069069+0000","last_fullsized":"2024-04-25T10:27:23.069069+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:17.685061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.068867+0000","last_change":"2024-04-25T10:27:21.921242+0000","last_active":"2024-04-25T10:27:23.068867+0000","last_peered":"2024-04-25T10:27:23.068867+0000","last_clean":"2024-04-25T10:27:23.068867+0000","last_became_active":"2024-04-25T10:27:21.920902+0000","last_became_peered":"2024-04-25T10:27:21.920902+0000","last_unstale":"2024-04-25T10:27:23.068867+0000","last_undegraded":"2024-04-25T10:27:23.068867+0000","last_fullsized":"2024-04-25T10:27:23.068867+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:12.860794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070561+0000","last_change":"2024-04-25T10:27:21.922501+0000","last_active":"2024-04-25T10:27:23.070561+0000","last_peered":"2024-04-25T10:27:23.070561+0000","last_clean":"2024-04-25T10:27:23.070561+0000","last_became_active":"2024-04-25T10:27:21.922169+0000","last_became_peered":"2024-04-25T10:27:21.922169+0000","last_unstale":"2024-04-25T10:27:23.070561+0000","last_undegraded":"2024-04-25T10:27:23.070561+0000","last_fullsized":"2024-04-25T10:27:23.070561+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:43.589479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924365+0000","last_change":"2024-04-25T10:27:21.920170+0000","last_active":"2024-04-25T10:27:22.924365+0000","last_peered":"2024-04-25T10:27:22.924365+0000","last_clean":"2024-04-25T10:27:22.924365+0000","last_became_active":"2024-04-25T10:27:21.919870+0000","last_became_peered":"2024-04-25T10:27:21.919870+0000","last_unstale":"2024-04-25T10:27:22.924365+0000","last_undegraded":"2024-04-25T10:27:22.924365+0000","last_fullsized":"2024-04-25T10:27:22.924365+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:12.779663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924951+0000","last_change":"2024-04-25T10:27:21.921617+0000","last_active":"2024-04-25T10:27:22.924951+0000","last_peered":"2024-04-25T10:27:22.924951+0000","last_clean":"2024-04-25T10:27:22.924951+0000","last_became_active":"2024-04-25T10:27:21.921338+0000","last_became_peered":"2024-04-25T10:27:21.921338+0000","last_unstale":"2024-04-25T10:27:22.924951+0000","last_undegraded":"2024-04-25T10:27:22.924951+0000","last_fullsized":"2024-04-25T10:27:22.924951+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:26.690677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070135+0000","last_change":"2024-04-25T10:27:21.934879+0000","last_active":"2024-04-25T10:27:23.070135+0000","last_peered":"2024-04-25T10:27:23.070135+0000","last_clean":"2024-04-25T10:27:23.070135+0000","last_became_active":"2024-04-25T10:27:21.932695+0000","last_became_peered":"2024-04-25T10:27:21.932695+0000","last_unstale":"2024-04-25T10:27:23.070135+0000","last_undegraded":"2024-04-25T10:27:23.070135+0000","last_fullsized":"2024-04-25T10:27:23.070135+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:17.685061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924367+0000","last_change":"2024-04-25T10:27:21.911281+0000","last_active":"2024-04-25T10:27:22.924367+0000","last_peered":"2024-04-25T10:27:22.924367+0000","last_clean":"2024-04-25T10:27:22.924367+0000","last_became_active":"2024-04-25T10:27:21.910975+0000","last_became_peered":"2024-04-25T10:27:21.910975+0000","last_unstale":"2024-04-25T10:27:22.924367+0000","last_undegraded":"2024-04-25T10:27:22.924367+0000","last_fullsized":"2024-04-25T10:27:22.924367+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:50.169390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070666+0000","last_change":"2024-04-25T10:27:21.925746+0000","last_active":"2024-04-25T10:27:23.070666+0000","last_peered":"2024-04-25T10:27:23.070666+0000","last_clean":"2024-04-25T10:27:23.070666+0000","last_became_active":"2024-04-25T10:27:21.925461+0000","last_became_peered":"2024-04-25T10:27:21.925461+0000","last_unstale":"2024-04-25T10:27:23.070666+0000","last_undegraded":"2024-04-25T10:27:23.070666+0000","last_fullsized":"2024-04-25T10:27:23.070666+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.726829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071087+0000","last_change":"2024-04-25T10:27:21.943419+0000","last_active":"2024-04-25T10:27:23.071087+0000","last_peered":"2024-04-25T10:27:23.071087+0000","last_clean":"2024-04-25T10:27:23.071087+0000","last_became_active":"2024-04-25T10:27:21.943088+0000","last_became_peered":"2024-04-25T10:27:21.943088+0000","last_unstale":"2024-04-25T10:27:23.071087+0000","last_undegraded":"2024-04-25T10:27:23.071087+0000","last_fullsized":"2024-04-25T10:27:23.071087+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:36.321999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070802+0000","last_change":"2024-04-25T10:27:21.932986+0000","last_active":"2024-04-25T10:27:23.070802+0000","last_peered":"2024-04-25T10:27:23.070802+0000","last_clean":"2024-04-25T10:27:23.070802+0000","last_became_active":"2024-04-25T10:27:21.932646+0000","last_became_peered":"2024-04-25T10:27:21.932646+0000","last_unstale":"2024-04-25T10:27:23.070802+0000","last_undegraded":"2024-04-25T10:27:23.070802+0000","last_fullsized":"2024-04-25T10:27:23.070802+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:15.528287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070722+0000","last_change":"2024-04-25T10:27:21.922417+0000","last_active":"2024-04-25T10:27:23.070722+0000","last_peered":"2024-04-25T10:27:23.070722+0000","last_clean":"2024-04-25T10:27:23.070722+0000","last_became_active":"2024-04-25T10:27:21.922104+0000","last_became_peered":"2024-04-25T10:27:21.922104+0000","last_unstale":"2024-04-25T10:27:23.070722+0000","last_undegraded":"2024-04-25T10:27:23.070722+0000","last_fullsized":"2024-04-25T10:27:23.070722+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:37.490557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071081+0000","last_change":"2024-04-25T10:27:21.930171+0000","last_active":"2024-04-25T10:27:23.071081+0000","last_peered":"2024-04-25T10:27:23.071081+0000","last_clean":"2024-04-25T10:27:23.071081+0000","last_became_active":"2024-04-25T10:27:21.929892+0000","last_became_peered":"2024-04-25T10:27:21.929892+0000","last_unstale":"2024-04-25T10:27:23.071081+0000","last_undegraded":"2024-04-25T10:27:23.071081+0000","last_fullsized":"2024-04-25T10:27:23.071081+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:25.295312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924145+0000","last_change":"2024-04-25T10:27:21.917768+0000","last_active":"2024-04-25T10:27:22.924145+0000","last_peered":"2024-04-25T10:27:22.924145+0000","last_clean":"2024-04-25T10:27:22.924145+0000","last_became_active":"2024-04-25T10:27:21.917458+0000","last_became_peered":"2024-04-25T10:27:21.917458+0000","last_unstale":"2024-04-25T10:27:22.924145+0000","last_undegraded":"2024-04-25T10:27:22.924145+0000","last_fullsized":"2024-04-25T10:27:22.924145+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:28.294242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070549+0000","last_change":"2024-04-25T10:27:21.931196+0000","last_active":"2024-04-25T10:27:23.070549+0000","last_peered":"2024-04-25T10:27:23.070549+0000","last_clean":"2024-04-25T10:27:23.070549+0000","last_became_active":"2024-04-25T10:27:21.928240+0000","last_became_peered":"2024-04-25T10:27:21.928240+0000","last_unstale":"2024-04-25T10:27:23.070549+0000","last_undegraded":"2024-04-25T10:27:23.070549+0000","last_fullsized":"2024-04-25T10:27:23.070549+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:24.539328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070056+0000","last_change":"2024-04-25T10:27:21.943154+0000","last_active":"2024-04-25T10:27:23.070056+0000","last_peered":"2024-04-25T10:27:23.070056+0000","last_clean":"2024-04-25T10:27:23.070056+0000","last_became_active":"2024-04-25T10:27:21.942883+0000","last_became_peered":"2024-04-25T10:27:21.942883+0000","last_unstale":"2024-04-25T10:27:23.070056+0000","last_undegraded":"2024-04-25T10:27:23.070056+0000","last_fullsized":"2024-04-25T10:27:23.070056+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:12.860794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924949+0000","last_change":"2024-04-25T10:27:21.914986+0000","last_active":"2024-04-25T10:27:22.924949+0000","last_peered":"2024-04-25T10:27:22.924949+0000","last_clean":"2024-04-25T10:27:22.924949+0000","last_became_active":"2024-04-25T10:27:21.914616+0000","last_became_peered":"2024-04-25T10:27:21.914616+0000","last_unstale":"2024-04-25T10:27:22.924949+0000","last_undegraded":"2024-04-25T10:27:22.924949+0000","last_fullsized":"2024-04-25T10:27:22.924949+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:14.614850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921097+0000","last_change":"2024-04-25T10:27:21.931035+0000","last_active":"2024-04-25T10:27:22.921097+0000","last_peered":"2024-04-25T10:27:22.921097+0000","last_clean":"2024-04-25T10:27:22.921097+0000","last_became_active":"2024-04-25T10:27:21.919919+0000","last_became_peered":"2024-04-25T10:27:21.919919+0000","last_unstale":"2024-04-25T10:27:22.921097+0000","last_undegraded":"2024-04-25T10:27:22.921097+0000","last_fullsized":"2024-04-25T10:27:22.921097+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:19.661310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070656+0000","last_change":"2024-04-25T10:27:21.922189+0000","last_active":"2024-04-25T10:27:23.070656+0000","last_peered":"2024-04-25T10:27:23.070656+0000","last_clean":"2024-04-25T10:27:23.070656+0000","last_became_active":"2024-04-25T10:27:21.921736+0000","last_became_peered":"2024-04-25T10:27:21.921736+0000","last_unstale":"2024-04-25T10:27:23.070656+0000","last_undegraded":"2024-04-25T10:27:23.070656+0000","last_fullsized":"2024-04-25T10:27:23.070656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:54.684599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925117+0000","last_change":"2024-04-25T10:27:21.912698+0000","last_active":"2024-04-25T10:27:22.925117+0000","last_peered":"2024-04-25T10:27:22.925117+0000","last_clean":"2024-04-25T10:27:22.925117+0000","last_became_active":"2024-04-25T10:27:21.911861+0000","last_became_peered":"2024-04-25T10:27:21.911861+0000","last_unstale":"2024-04-25T10:27:22.925117+0000","last_undegraded":"2024-04-25T10:27:22.925117+0000","last_fullsized":"2024-04-25T10:27:22.925117+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:54.580782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924550+0000","last_change":"2024-04-25T10:27:21.931937+0000","last_active":"2024-04-25T10:27:22.924550+0000","last_peered":"2024-04-25T10:27:22.924550+0000","last_clean":"2024-04-25T10:27:22.924550+0000","last_became_active":"2024-04-25T10:27:21.929168+0000","last_became_peered":"2024-04-25T10:27:21.929168+0000","last_unstale":"2024-04-25T10:27:22.924550+0000","last_undegraded":"2024-04-25T10:27:22.924550+0000","last_fullsized":"2024-04-25T10:27:22.924550+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:31.735635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071657+0000","last_change":"2024-04-25T10:27:21.933358+0000","last_active":"2024-04-25T10:27:23.071657+0000","last_peered":"2024-04-25T10:27:23.071657+0000","last_clean":"2024-04-25T10:27:23.071657+0000","last_became_active":"2024-04-25T10:27:21.932898+0000","last_became_peered":"2024-04-25T10:27:21.932898+0000","last_unstale":"2024-04-25T10:27:23.071657+0000","last_undegraded":"2024-04-25T10:27:23.071657+0000","last_fullsized":"2024-04-25T10:27:23.071657+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:46.204788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921483+0000","last_change":"2024-04-25T10:27:21.928600+0000","last_active":"2024-04-25T10:27:22.921483+0000","last_peered":"2024-04-25T10:27:22.921483+0000","last_clean":"2024-04-25T10:27:22.921483+0000","last_became_active":"2024-04-25T10:27:21.928333+0000","last_became_peered":"2024-04-25T10:27:21.928333+0000","last_unstale":"2024-04-25T10:27:22.921483+0000","last_undegraded":"2024-04-25T10:27:22.921483+0000","last_fullsized":"2024-04-25T10:27:22.921483+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:57.014104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924527+0000","last_change":"2024-04-25T10:27:21.929531+0000","last_active":"2024-04-25T10:27:22.924527+0000","last_peered":"2024-04-25T10:27:22.924527+0000","last_clean":"2024-04-25T10:27:22.924527+0000","last_became_active":"2024-04-25T10:27:21.929207+0000","last_became_peered":"2024-04-25T10:27:21.929207+0000","last_unstale":"2024-04-25T10:27:22.924527+0000","last_undegraded":"2024-04-25T10:27:22.924527+0000","last_fullsized":"2024-04-25T10:27:22.924527+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:55.628413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924634+0000","last_change":"2024-04-25T10:27:21.917236+0000","last_active":"2024-04-25T10:27:22.924634+0000","last_peered":"2024-04-25T10:27:22.924634+0000","last_clean":"2024-04-25T10:27:22.924634+0000","last_became_active":"2024-04-25T10:27:21.916914+0000","last_became_peered":"2024-04-25T10:27:21.916914+0000","last_unstale":"2024-04-25T10:27:22.924634+0000","last_undegraded":"2024-04-25T10:27:22.924634+0000","last_fullsized":"2024-04-25T10:27:22.924634+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:04.174286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921122+0000","last_change":"2024-04-25T10:27:21.932110+0000","last_active":"2024-04-25T10:27:22.921122+0000","last_peered":"2024-04-25T10:27:22.921122+0000","last_clean":"2024-04-25T10:27:22.921122+0000","last_became_active":"2024-04-25T10:27:21.931817+0000","last_became_peered":"2024-04-25T10:27:21.931817+0000","last_unstale":"2024-04-25T10:27:22.921122+0000","last_undegraded":"2024-04-25T10:27:22.921122+0000","last_fullsized":"2024-04-25T10:27:22.921122+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:18.229355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069092+0000","last_change":"2024-04-25T10:27:21.924775+0000","last_active":"2024-04-25T10:27:23.069092+0000","last_peered":"2024-04-25T10:27:23.069092+0000","last_clean":"2024-04-25T10:27:23.069092+0000","last_became_active":"2024-04-25T10:27:21.923142+0000","last_became_peered":"2024-04-25T10:27:21.923142+0000","last_unstale":"2024-04-25T10:27:23.069092+0000","last_undegraded":"2024-04-25T10:27:23.069092+0000","last_fullsized":"2024-04-25T10:27:23.069092+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:04.298177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071634+0000","last_change":"2024-04-25T10:27:21.929201+0000","last_active":"2024-04-25T10:27:23.071634+0000","last_peered":"2024-04-25T10:27:23.071634+0000","last_clean":"2024-04-25T10:27:23.071634+0000","last_became_active":"2024-04-25T10:27:21.928773+0000","last_became_peered":"2024-04-25T10:27:21.928773+0000","last_unstale":"2024-04-25T10:27:23.071634+0000","last_undegraded":"2024-04-25T10:27:23.071634+0000","last_fullsized":"2024-04-25T10:27:23.071634+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:50.169390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070006+0000","last_change":"2024-04-25T10:27:20.905609+0000","last_active":"2024-04-25T10:27:23.070006+0000","last_peered":"2024-04-25T10:27:23.070006+0000","last_clean":"2024-04-25T10:27:23.070006+0000","last_became_active":"2024-04-25T10:27:20.905324+0000","last_became_peered":"2024-04-25T10:27:20.905324+0000","last_unstale":"2024-04-25T10:27:23.070006+0000","last_undegraded":"2024-04-25T10:27:23.070006+0000","last_fullsized":"2024-04-25T10:27:23.070006+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:17.334926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070765+0000","last_change":"2024-04-25T10:27:21.921853+0000","last_active":"2024-04-25T10:27:23.070765+0000","last_peered":"2024-04-25T10:27:23.070765+0000","last_clean":"2024-04-25T10:27:23.070765+0000","last_became_active":"2024-04-25T10:27:21.921520+0000","last_became_peered":"2024-04-25T10:27:21.921520+0000","last_unstale":"2024-04-25T10:27:23.070765+0000","last_undegraded":"2024-04-25T10:27:23.070765+0000","last_fullsized":"2024-04-25T10:27:23.070765+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:12.401369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070479+0000","last_change":"2024-04-25T10:27:20.915760+0000","last_active":"2024-04-25T10:27:23.070479+0000","last_peered":"2024-04-25T10:27:23.070479+0000","last_clean":"2024-04-25T10:27:23.070479+0000","last_became_active":"2024-04-25T10:27:20.915478+0000","last_became_peered":"2024-04-25T10:27:20.915478+0000","last_unstale":"2024-04-25T10:27:23.070479+0000","last_undegraded":"2024-04-25T10:27:23.070479+0000","last_fullsized":"2024-04-25T10:27:23.070479+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:24.380220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069580+0000","last_change":"2024-04-25T10:27:21.939718+0000","last_active":"2024-04-25T10:27:23.069580+0000","last_peered":"2024-04-25T10:27:23.069580+0000","last_clean":"2024-04-25T10:27:23.069580+0000","last_became_active":"2024-04-25T10:27:21.938772+0000","last_became_peered":"2024-04-25T10:27:21.938772+0000","last_unstale":"2024-04-25T10:27:23.069580+0000","last_undegraded":"2024-04-25T10:27:23.069580+0000","last_fullsized":"2024-04-25T10:27:23.069580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:48.408886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924924+0000","last_change":"2024-04-25T10:27:20.905337+0000","last_active":"2024-04-25T10:27:22.924924+0000","last_peered":"2024-04-25T10:27:22.924924+0000","last_clean":"2024-04-25T10:27:22.924924+0000","last_became_active":"2024-04-25T10:27:20.904489+0000","last_became_peered":"2024-04-25T10:27:20.904489+0000","last_unstale":"2024-04-25T10:27:22.924924+0000","last_undegraded":"2024-04-25T10:27:22.924924+0000","last_fullsized":"2024-04-25T10:27:22.924924+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:53.847466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070571+0000","last_change":"2024-04-25T10:27:21.937554+0000","last_active":"2024-04-25T10:27:23.070571+0000","last_peered":"2024-04-25T10:27:23.070571+0000","last_clean":"2024-04-25T10:27:23.070571+0000","last_became_active":"2024-04-25T10:27:21.937186+0000","last_became_peered":"2024-04-25T10:27:21.937186+0000","last_unstale":"2024-04-25T10:27:23.070571+0000","last_undegraded":"2024-04-25T10:27:23.070571+0000","last_fullsized":"2024-04-25T10:27:23.070571+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.726829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924372+0000","last_change":"2024-04-25T10:27:20.904016+0000","last_active":"2024-04-25T10:27:22.924372+0000","last_peered":"2024-04-25T10:27:22.924372+0000","last_clean":"2024-04-25T10:27:22.924372+0000","last_became_active":"2024-04-25T10:27:20.903655+0000","last_became_peered":"2024-04-25T10:27:20.903655+0000","last_unstale":"2024-04-25T10:27:22.924372+0000","last_undegraded":"2024-04-25T10:27:22.924372+0000","last_fullsized":"2024-04-25T10:27:22.924372+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:23.749370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070619+0000","last_change":"2024-04-25T10:27:21.936187+0000","last_active":"2024-04-25T10:27:23.070619+0000","last_peered":"2024-04-25T10:27:23.070619+0000","last_clean":"2024-04-25T10:27:23.070619+0000","last_became_active":"2024-04-25T10:27:21.935755+0000","last_became_peered":"2024-04-25T10:27:21.935755+0000","last_unstale":"2024-04-25T10:27:23.070619+0000","last_undegraded":"2024-04-25T10:27:23.070619+0000","last_fullsized":"2024-04-25T10:27:23.070619+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:08.584101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924638+0000","last_change":"2024-04-25T10:27:20.897883+0000","last_active":"2024-04-25T10:27:22.924638+0000","last_peered":"2024-04-25T10:27:22.924638+0000","last_clean":"2024-04-25T10:27:22.924638+0000","last_became_active":"2024-04-25T10:27:20.897384+0000","last_became_peered":"2024-04-25T10:27:20.897384+0000","last_unstale":"2024-04-25T10:27:22.924638+0000","last_undegraded":"2024-04-25T10:27:22.924638+0000","last_fullsized":"2024-04-25T10:27:22.924638+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:46.704911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070149+0000","last_change":"2024-04-25T10:27:21.935998+0000","last_active":"2024-04-25T10:27:23.070149+0000","last_peered":"2024-04-25T10:27:23.070149+0000","last_clean":"2024-04-25T10:27:23.070149+0000","last_became_active":"2024-04-25T10:27:21.935664+0000","last_became_peered":"2024-04-25T10:27:21.935664+0000","last_unstale":"2024-04-25T10:27:23.070149+0000","last_undegraded":"2024-04-25T10:27:23.070149+0000","last_fullsized":"2024-04-25T10:27:23.070149+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.566400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070674+0000","last_change":"2024-04-25T10:27:20.910870+0000","last_active":"2024-04-25T10:27:23.070674+0000","last_peered":"2024-04-25T10:27:23.070674+0000","last_clean":"2024-04-25T10:27:23.070674+0000","last_became_active":"2024-04-25T10:27:20.910556+0000","last_became_peered":"2024-04-25T10:27:20.910556+0000","last_unstale":"2024-04-25T10:27:23.070674+0000","last_undegraded":"2024-04-25T10:27:23.070674+0000","last_fullsized":"2024-04-25T10:27:23.070674+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.593678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069632+0000","last_change":"2024-04-25T10:27:21.932133+0000","last_active":"2024-04-25T10:27:23.069632+0000","last_peered":"2024-04-25T10:27:23.069632+0000","last_clean":"2024-04-25T10:27:23.069632+0000","last_became_active":"2024-04-25T10:27:21.931861+0000","last_became_peered":"2024-04-25T10:27:21.931861+0000","last_unstale":"2024-04-25T10:27:23.069632+0000","last_undegraded":"2024-04-25T10:27:23.069632+0000","last_fullsized":"2024-04-25T10:27:23.069632+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:06.326474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069878+0000","last_change":"2024-04-25T10:27:20.910610+0000","last_active":"2024-04-25T10:27:23.069878+0000","last_peered":"2024-04-25T10:27:23.069878+0000","last_clean":"2024-04-25T10:27:23.069878+0000","last_became_active":"2024-04-25T10:27:20.910321+0000","last_became_peered":"2024-04-25T10:27:20.910321+0000","last_unstale":"2024-04-25T10:27:23.069878+0000","last_undegraded":"2024-04-25T10:27:23.069878+0000","last_fullsized":"2024-04-25T10:27:23.069878+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:51.054444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924051+0000","last_change":"2024-04-25T10:27:21.910936+0000","last_active":"2024-04-25T10:27:22.924051+0000","last_peered":"2024-04-25T10:27:22.924051+0000","last_clean":"2024-04-25T10:27:22.924051+0000","last_became_active":"2024-04-25T10:27:21.910549+0000","last_became_peered":"2024-04-25T10:27:21.910549+0000","last_unstale":"2024-04-25T10:27:22.924051+0000","last_undegraded":"2024-04-25T10:27:22.924051+0000","last_fullsized":"2024-04-25T10:27:22.924051+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.447889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924136+0000","last_change":"2024-04-25T10:27:20.906207+0000","last_active":"2024-04-25T10:27:22.924136+0000","last_peered":"2024-04-25T10:27:22.924136+0000","last_clean":"2024-04-25T10:27:22.924136+0000","last_became_active":"2024-04-25T10:27:20.905124+0000","last_became_peered":"2024-04-25T10:27:20.905124+0000","last_unstale":"2024-04-25T10:27:22.924136+0000","last_undegraded":"2024-04-25T10:27:22.924136+0000","last_fullsized":"2024-04-25T10:27:22.924136+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:02.890347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925243+0000","last_change":"2024-04-25T10:27:21.928121+0000","last_active":"2024-04-25T10:27:22.925243+0000","last_peered":"2024-04-25T10:27:22.925243+0000","last_clean":"2024-04-25T10:27:22.925243+0000","last_became_active":"2024-04-25T10:27:21.924777+0000","last_became_peered":"2024-04-25T10:27:21.924777+0000","last_unstale":"2024-04-25T10:27:22.925243+0000","last_undegraded":"2024-04-25T10:27:22.925243+0000","last_fullsized":"2024-04-25T10:27:22.925243+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:55.967733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070888+0000","last_change":"2024-04-25T10:27:20.912027+0000","last_active":"2024-04-25T10:27:23.070888+0000","last_peered":"2024-04-25T10:27:23.070888+0000","last_clean":"2024-04-25T10:27:23.070888+0000","last_became_active":"2024-04-25T10:27:20.911347+0000","last_became_peered":"2024-04-25T10:27:20.911347+0000","last_unstale":"2024-04-25T10:27:23.070888+0000","last_undegraded":"2024-04-25T10:27:23.070888+0000","last_fullsized":"2024-04-25T10:27:23.070888+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:53.678274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069691+0000","last_change":"2024-04-25T10:27:21.944788+0000","last_active":"2024-04-25T10:27:23.069691+0000","last_peered":"2024-04-25T10:27:23.069691+0000","last_clean":"2024-04-25T10:27:23.069691+0000","last_became_active":"2024-04-25T10:27:21.944498+0000","last_became_peered":"2024-04-25T10:27:21.944498+0000","last_unstale":"2024-04-25T10:27:23.069691+0000","last_undegraded":"2024-04-25T10:27:23.069691+0000","last_fullsized":"2024-04-25T10:27:23.069691+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:48.990957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071075+0000","last_change":"2024-04-25T10:27:20.911101+0000","last_active":"2024-04-25T10:27:23.071075+0000","last_peered":"2024-04-25T10:27:23.071075+0000","last_clean":"2024-04-25T10:27:23.071075+0000","last_became_active":"2024-04-25T10:27:20.910775+0000","last_became_peered":"2024-04-25T10:27:20.910775+0000","last_unstale":"2024-04-25T10:27:23.071075+0000","last_undegraded":"2024-04-25T10:27:23.071075+0000","last_fullsized":"2024-04-25T10:27:23.071075+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:17.223031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070280+0000","last_change":"2024-04-25T10:27:21.940855+0000","last_active":"2024-04-25T10:27:23.070280+0000","last_peered":"2024-04-25T10:27:23.070280+0000","last_clean":"2024-04-25T10:27:23.070280+0000","last_became_active":"2024-04-25T10:27:21.940557+0000","last_became_peered":"2024-04-25T10:27:21.940557+0000","last_unstale":"2024-04-25T10:27:23.070280+0000","last_undegraded":"2024-04-25T10:27:23.070280+0000","last_fullsized":"2024-04-25T10:27:23.070280+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:33.299050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069514+0000","last_change":"2024-04-25T10:27:20.906898+0000","last_active":"2024-04-25T10:27:23.069514+0000","last_peered":"2024-04-25T10:27:23.069514+0000","last_clean":"2024-04-25T10:27:23.069514+0000","last_became_active":"2024-04-25T10:27:20.906640+0000","last_became_peered":"2024-04-25T10:27:20.906640+0000","last_unstale":"2024-04-25T10:27:23.069514+0000","last_undegraded":"2024-04-25T10:27:23.069514+0000","last_fullsized":"2024-04-25T10:27:23.069514+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:25.792952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921268+0000","last_change":"2024-04-25T10:27:21.928237+0000","last_active":"2024-04-25T10:27:22.921268+0000","last_peered":"2024-04-25T10:27:22.921268+0000","last_clean":"2024-04-25T10:27:22.921268+0000","last_became_active":"2024-04-25T10:27:21.927959+0000","last_became_peered":"2024-04-25T10:27:21.927959+0000","last_unstale":"2024-04-25T10:27:22.921268+0000","last_undegraded":"2024-04-25T10:27:22.921268+0000","last_fullsized":"2024-04-25T10:27:22.921268+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:46.346477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"129'6","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921416+0000","last_change":"2024-04-25T10:27:20.912722+0000","last_active":"2024-04-25T10:27:22.921416+0000","last_peered":"2024-04-25T10:27:22.921416+0000","last_clean":"2024-04-25T10:27:22.921416+0000","last_became_active":"2024-04-25T10:27:20.912417+0000","last_became_peered":"2024-04-25T10:27:20.912417+0000","last_unstale":"2024-04-25T10:27:22.921416+0000","last_undegraded":"2024-04-25T10:27:22.921416+0000","last_fullsized":"2024-04-25T10:27:22.921416+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:04.969380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071245+0000","last_change":"2024-04-25T10:27:21.925360+0000","last_active":"2024-04-25T10:27:23.071245+0000","last_peered":"2024-04-25T10:27:23.071245+0000","last_clean":"2024-04-25T10:27:23.071245+0000","last_became_active":"2024-04-25T10:27:21.924166+0000","last_became_peered":"2024-04-25T10:27:21.924166+0000","last_unstale":"2024-04-25T10:27:23.071245+0000","last_undegraded":"2024-04-25T10:27:23.071245+0000","last_fullsized":"2024-04-25T10:27:23.071245+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:12.404090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071067+0000","last_change":"2024-04-25T10:27:20.913074+0000","last_active":"2024-04-25T10:27:23.071067+0000","last_peered":"2024-04-25T10:27:23.071067+0000","last_clean":"2024-04-25T10:27:23.071067+0000","last_became_active":"2024-04-25T10:27:20.912740+0000","last_became_peered":"2024-04-25T10:27:20.912740+0000","last_unstale":"2024-04-25T10:27:23.071067+0000","last_undegraded":"2024-04-25T10:27:23.071067+0000","last_fullsized":"2024-04-25T10:27:23.071067+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:40.146777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070762+0000","last_change":"2024-04-25T10:27:21.928923+0000","last_active":"2024-04-25T10:27:23.070762+0000","last_peered":"2024-04-25T10:27:23.070762+0000","last_clean":"2024-04-25T10:27:23.070762+0000","last_became_active":"2024-04-25T10:27:21.928646+0000","last_became_peered":"2024-04-25T10:27:21.928646+0000","last_unstale":"2024-04-25T10:27:23.070762+0000","last_undegraded":"2024-04-25T10:27:23.070762+0000","last_fullsized":"2024-04-25T10:27:23.070762+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:54.853790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924451+0000","last_change":"2024-04-25T10:27:20.912158+0000","last_active":"2024-04-25T10:27:22.924451+0000","last_peered":"2024-04-25T10:27:22.924451+0000","last_clean":"2024-04-25T10:27:22.924451+0000","last_became_active":"2024-04-25T10:27:20.911866+0000","last_became_peered":"2024-04-25T10:27:20.911866+0000","last_unstale":"2024-04-25T10:27:22.924451+0000","last_undegraded":"2024-04-25T10:27:22.924451+0000","last_fullsized":"2024-04-25T10:27:22.924451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:32.292726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069823+0000","last_change":"2024-04-25T10:27:21.940800+0000","last_active":"2024-04-25T10:27:23.069823+0000","last_peered":"2024-04-25T10:27:23.069823+0000","last_clean":"2024-04-25T10:27:23.069823+0000","last_became_active":"2024-04-25T10:27:21.940487+0000","last_became_peered":"2024-04-25T10:27:21.940487+0000","last_unstale":"2024-04-25T10:27:23.069823+0000","last_undegraded":"2024-04-25T10:27:23.069823+0000","last_fullsized":"2024-04-25T10:27:23.069823+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.515339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"129'8","reported_seq":22,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069730+0000","last_change":"2024-04-25T10:27:20.909036+0000","last_active":"2024-04-25T10:27:23.069730+0000","last_peered":"2024-04-25T10:27:23.069730+0000","last_clean":"2024-04-25T10:27:23.069730+0000","last_became_active":"2024-04-25T10:27:20.908753+0000","last_became_peered":"2024-04-25T10:27:20.908753+0000","last_unstale":"2024-04-25T10:27:23.069730+0000","last_undegraded":"2024-04-25T10:27:23.069730+0000","last_fullsized":"2024-04-25T10:27:23.069730+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:23.533004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069715+0000","last_change":"2024-04-25T10:27:21.924396+0000","last_active":"2024-04-25T10:27:23.069715+0000","last_peered":"2024-04-25T10:27:23.069715+0000","last_clean":"2024-04-25T10:27:23.069715+0000","last_became_active":"2024-04-25T10:27:21.924075+0000","last_became_peered":"2024-04-25T10:27:21.924075+0000","last_unstale":"2024-04-25T10:27:23.069715+0000","last_undegraded":"2024-04-25T10:27:23.069715+0000","last_fullsized":"2024-04-25T10:27:23.069715+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:33.299050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070930+0000","last_change":"2024-04-25T10:27:20.917529+0000","last_active":"2024-04-25T10:27:23.070930+0000","last_peered":"2024-04-25T10:27:23.070930+0000","last_clean":"2024-04-25T10:27:23.070930+0000","last_became_active":"2024-04-25T10:27:20.916954+0000","last_became_peered":"2024-04-25T10:27:20.916954+0000","last_unstale":"2024-04-25T10:27:23.070930+0000","last_undegraded":"2024-04-25T10:27:23.070930+0000","last_fullsized":"2024-04-25T10:27:23.070930+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:07.577776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069835+0000","last_change":"2024-04-25T10:27:21.941636+0000","last_active":"2024-04-25T10:27:23.069835+0000","last_peered":"2024-04-25T10:27:23.069835+0000","last_clean":"2024-04-25T10:27:23.069835+0000","last_became_active":"2024-04-25T10:27:21.940314+0000","last_became_peered":"2024-04-25T10:27:21.940314+0000","last_unstale":"2024-04-25T10:27:23.069835+0000","last_undegraded":"2024-04-25T10:27:23.069835+0000","last_fullsized":"2024-04-25T10:27:23.069835+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:57.726829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924417+0000","last_change":"2024-04-25T10:27:20.899288+0000","last_active":"2024-04-25T10:27:22.924417+0000","last_peered":"2024-04-25T10:27:22.924417+0000","last_clean":"2024-04-25T10:27:22.924417+0000","last_became_active":"2024-04-25T10:27:20.899001+0000","last_became_peered":"2024-04-25T10:27:20.899001+0000","last_unstale":"2024-04-25T10:27:22.924417+0000","last_undegraded":"2024-04-25T10:27:22.924417+0000","last_fullsized":"2024-04-25T10:27:22.924417+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:47.984633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925316+0000","last_change":"2024-04-25T10:27:21.930710+0000","last_active":"2024-04-25T10:27:22.925316+0000","last_peered":"2024-04-25T10:27:22.925316+0000","last_clean":"2024-04-25T10:27:22.925316+0000","last_became_active":"2024-04-25T10:27:21.930289+0000","last_became_peered":"2024-04-25T10:27:21.930289+0000","last_unstale":"2024-04-25T10:27:22.925316+0000","last_undegraded":"2024-04-25T10:27:22.925316+0000","last_fullsized":"2024-04-25T10:27:22.925316+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:48.477977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070252+0000","last_change":"2024-04-25T10:27:20.909811+0000","last_active":"2024-04-25T10:27:23.070252+0000","last_peered":"2024-04-25T10:27:23.070252+0000","last_clean":"2024-04-25T10:27:23.070252+0000","last_became_active":"2024-04-25T10:27:20.909469+0000","last_became_peered":"2024-04-25T10:27:20.909469+0000","last_unstale":"2024-04-25T10:27:23.070252+0000","last_undegraded":"2024-04-25T10:27:23.070252+0000","last_fullsized":"2024-04-25T10:27:23.070252+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:31.841070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924982+0000","last_change":"2024-04-25T10:27:21.914642+0000","last_active":"2024-04-25T10:27:22.924982+0000","last_peered":"2024-04-25T10:27:22.924982+0000","last_clean":"2024-04-25T10:27:22.924982+0000","last_became_active":"2024-04-25T10:27:21.914343+0000","last_became_peered":"2024-04-25T10:27:21.914343+0000","last_unstale":"2024-04-25T10:27:22.924982+0000","last_undegraded":"2024-04-25T10:27:22.924982+0000","last_fullsized":"2024-04-25T10:27:22.924982+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069396+0000","last_change":"2024-04-25T10:27:20.905577+0000","last_active":"2024-04-25T10:27:23.069396+0000","last_peered":"2024-04-25T10:27:23.069396+0000","last_clean":"2024-04-25T10:27:23.069396+0000","last_became_active":"2024-04-25T10:27:20.905273+0000","last_became_peered":"2024-04-25T10:27:20.905273+0000","last_unstale":"2024-04-25T10:27:23.069396+0000","last_undegraded":"2024-04-25T10:27:23.069396+0000","last_fullsized":"2024-04-25T10:27:23.069396+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:57.324917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924819+0000","last_change":"2024-04-25T10:27:21.916592+0000","last_active":"2024-04-25T10:27:22.924819+0000","last_peered":"2024-04-25T10:27:22.924819+0000","last_clean":"2024-04-25T10:27:22.924819+0000","last_became_active":"2024-04-25T10:27:21.914998+0000","last_became_peered":"2024-04-25T10:27:21.914998+0000","last_unstale":"2024-04-25T10:27:22.924819+0000","last_undegraded":"2024-04-25T10:27:22.924819+0000","last_fullsized":"2024-04-25T10:27:22.924819+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:43.455427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069530+0000","last_change":"2024-04-25T10:27:20.910440+0000","last_active":"2024-04-25T10:27:23.069530+0000","last_peered":"2024-04-25T10:27:23.069530+0000","last_clean":"2024-04-25T10:27:23.069530+0000","last_became_active":"2024-04-25T10:27:20.910175+0000","last_became_peered":"2024-04-25T10:27:20.910175+0000","last_unstale":"2024-04-25T10:27:23.069530+0000","last_undegraded":"2024-04-25T10:27:23.069530+0000","last_fullsized":"2024-04-25T10:27:23.069530+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:55.377671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921156+0000","last_change":"2024-04-25T10:27:21.921211+0000","last_active":"2024-04-25T10:27:22.921156+0000","last_peered":"2024-04-25T10:27:22.921156+0000","last_clean":"2024-04-25T10:27:22.921156+0000","last_became_active":"2024-04-25T10:27:21.920915+0000","last_became_peered":"2024-04-25T10:27:21.920915+0000","last_unstale":"2024-04-25T10:27:22.921156+0000","last_undegraded":"2024-04-25T10:27:22.921156+0000","last_fullsized":"2024-04-25T10:27:22.921156+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:08.229284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070730+0000","last_change":"2024-04-25T10:27:20.914965+0000","last_active":"2024-04-25T10:27:23.070730+0000","last_peered":"2024-04-25T10:27:23.070730+0000","last_clean":"2024-04-25T10:27:23.070730+0000","last_became_active":"2024-04-25T10:27:20.914657+0000","last_became_peered":"2024-04-25T10:27:20.914657+0000","last_unstale":"2024-04-25T10:27:23.070730+0000","last_undegraded":"2024-04-25T10:27:23.070730+0000","last_fullsized":"2024-04-25T10:27:23.070730+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:51.886072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070927+0000","last_change":"2024-04-25T10:27:21.923200+0000","last_active":"2024-04-25T10:27:23.070927+0000","last_peered":"2024-04-25T10:27:23.070927+0000","last_clean":"2024-04-25T10:27:23.070927+0000","last_became_active":"2024-04-25T10:27:21.922921+0000","last_became_peered":"2024-04-25T10:27:21.922921+0000","last_unstale":"2024-04-25T10:27:23.070927+0000","last_undegraded":"2024-04-25T10:27:23.070927+0000","last_fullsized":"2024-04-25T10:27:23.070927+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:02.469210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070596+0000","last_change":"2024-04-25T10:27:20.916877+0000","last_active":"2024-04-25T10:27:23.070596+0000","last_peered":"2024-04-25T10:27:23.070596+0000","last_clean":"2024-04-25T10:27:23.070596+0000","last_became_active":"2024-04-25T10:27:20.916201+0000","last_became_peered":"2024-04-25T10:27:20.916201+0000","last_unstale":"2024-04-25T10:27:23.070596+0000","last_undegraded":"2024-04-25T10:27:23.070596+0000","last_fullsized":"2024-04-25T10:27:23.070596+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:17.334926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069987+0000","last_change":"2024-04-25T10:27:21.925842+0000","last_active":"2024-04-25T10:27:23.069987+0000","last_peered":"2024-04-25T10:27:23.069987+0000","last_clean":"2024-04-25T10:27:23.069987+0000","last_became_active":"2024-04-25T10:27:21.925583+0000","last_became_peered":"2024-04-25T10:27:21.925583+0000","last_unstale":"2024-04-25T10:27:23.069987+0000","last_undegraded":"2024-04-25T10:27:23.069987+0000","last_fullsized":"2024-04-25T10:27:23.069987+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:05.103843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.923856+0000","last_change":"2024-04-25T10:27:20.905362+0000","last_active":"2024-04-25T10:27:22.923856+0000","last_peered":"2024-04-25T10:27:22.923856+0000","last_clean":"2024-04-25T10:27:22.923856+0000","last_became_active":"2024-04-25T10:27:20.905080+0000","last_became_peered":"2024-04-25T10:27:20.905080+0000","last_unstale":"2024-04-25T10:27:22.923856+0000","last_undegraded":"2024-04-25T10:27:22.923856+0000","last_fullsized":"2024-04-25T10:27:22.923856+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:29.560075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.921506+0000","last_change":"2024-04-25T10:27:21.927047+0000","last_active":"2024-04-25T10:27:22.921506+0000","last_peered":"2024-04-25T10:27:22.921506+0000","last_clean":"2024-04-25T10:27:22.921506+0000","last_became_active":"2024-04-25T10:27:21.926408+0000","last_became_peered":"2024-04-25T10:27:21.926408+0000","last_unstale":"2024-04-25T10:27:22.921506+0000","last_undegraded":"2024-04-25T10:27:22.921506+0000","last_fullsized":"2024-04-25T10:27:22.921506+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:36.497889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071237+0000","last_change":"2024-04-25T10:27:20.916973+0000","last_active":"2024-04-25T10:27:23.071237+0000","last_peered":"2024-04-25T10:27:23.071237+0000","last_clean":"2024-04-25T10:27:23.071237+0000","last_became_active":"2024-04-25T10:27:20.916669+0000","last_became_peered":"2024-04-25T10:27:20.916669+0000","last_unstale":"2024-04-25T10:27:23.071237+0000","last_undegraded":"2024-04-25T10:27:23.071237+0000","last_fullsized":"2024-04-25T10:27:23.071237+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:07.222960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925210+0000","last_change":"2024-04-25T10:27:21.934025+0000","last_active":"2024-04-25T10:27:22.925210+0000","last_peered":"2024-04-25T10:27:22.925210+0000","last_clean":"2024-04-25T10:27:22.925210+0000","last_became_active":"2024-04-25T10:27:21.933571+0000","last_became_peered":"2024-04-25T10:27:21.933571+0000","last_unstale":"2024-04-25T10:27:22.925210+0000","last_undegraded":"2024-04-25T10:27:22.925210+0000","last_fullsized":"2024-04-25T10:27:22.925210+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:40.290064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"130'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.910525+0000","last_change":"2024-04-25T10:27:20.915852+0000","last_active":"2024-04-25T10:27:23.910525+0000","last_peered":"2024-04-25T10:27:23.910525+0000","last_clean":"2024-04-25T10:27:23.910525+0000","last_became_active":"2024-04-25T10:27:20.914929+0000","last_became_peered":"2024-04-25T10:27:20.914929+0000","last_unstale":"2024-04-25T10:27:23.910525+0000","last_undegraded":"2024-04-25T10:27:23.910525+0000","last_fullsized":"2024-04-25T10:27:23.910525+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:40.146777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071479+0000","last_change":"2024-04-25T10:27:21.921903+0000","last_active":"2024-04-25T10:27:23.071479+0000","last_peered":"2024-04-25T10:27:23.071479+0000","last_clean":"2024-04-25T10:27:23.071479+0000","last_became_active":"2024-04-25T10:27:21.921598+0000","last_became_peered":"2024-04-25T10:27:21.921598+0000","last_unstale":"2024-04-25T10:27:23.071479+0000","last_undegraded":"2024-04-25T10:27:23.071479+0000","last_fullsized":"2024-04-25T10:27:23.071479+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:19.183695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070948+0000","last_change":"2024-04-25T10:27:20.917080+0000","last_active":"2024-04-25T10:27:23.070948+0000","last_peered":"2024-04-25T10:27:23.070948+0000","last_clean":"2024-04-25T10:27:23.070948+0000","last_became_active":"2024-04-25T10:27:20.916805+0000","last_became_peered":"2024-04-25T10:27:20.916805+0000","last_unstale":"2024-04-25T10:27:23.070948+0000","last_undegraded":"2024-04-25T10:27:23.070948+0000","last_fullsized":"2024-04-25T10:27:23.070948+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:18.654986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070271+0000","last_change":"2024-04-25T10:27:21.926551+0000","last_active":"2024-04-25T10:27:23.070271+0000","last_peered":"2024-04-25T10:27:23.070271+0000","last_clean":"2024-04-25T10:27:23.070271+0000","last_became_active":"2024-04-25T10:27:21.925702+0000","last_became_peered":"2024-04-25T10:27:21.925702+0000","last_unstale":"2024-04-25T10:27:23.070271+0000","last_undegraded":"2024-04-25T10:27:23.070271+0000","last_fullsized":"2024-04-25T10:27:23.070271+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:12.794314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.071531+0000","last_change":"2024-04-25T10:27:20.909373+0000","last_active":"2024-04-25T10:27:23.071531+0000","last_peered":"2024-04-25T10:27:23.071531+0000","last_clean":"2024-04-25T10:27:23.071531+0000","last_became_active":"2024-04-25T10:27:20.909035+0000","last_became_peered":"2024-04-25T10:27:20.909035+0000","last_unstale":"2024-04-25T10:27:23.071531+0000","last_undegraded":"2024-04-25T10:27:23.071531+0000","last_fullsized":"2024-04-25T10:27:23.071531+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:25.792952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069962+0000","last_change":"2024-04-25T10:27:21.942269+0000","last_active":"2024-04-25T10:27:23.069962+0000","last_peered":"2024-04-25T10:27:23.069962+0000","last_clean":"2024-04-25T10:27:23.069962+0000","last_became_active":"2024-04-25T10:27:21.942005+0000","last_became_peered":"2024-04-25T10:27:21.942005+0000","last_unstale":"2024-04-25T10:27:23.069962+0000","last_undegraded":"2024-04-25T10:27:23.069962+0000","last_fullsized":"2024-04-25T10:27:23.069962+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:30.339717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.925056+0000","last_change":"2024-04-25T10:27:20.905514+0000","last_active":"2024-04-25T10:27:22.925056+0000","last_peered":"2024-04-25T10:27:22.925056+0000","last_clean":"2024-04-25T10:27:22.925056+0000","last_became_active":"2024-04-25T10:27:20.904241+0000","last_became_peered":"2024-04-25T10:27:20.904241+0000","last_unstale":"2024-04-25T10:27:22.925056+0000","last_undegraded":"2024-04-25T10:27:22.925056+0000","last_fullsized":"2024-04-25T10:27:22.925056+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:15.747328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070178+0000","last_change":"2024-04-25T10:27:21.928249+0000","last_active":"2024-04-25T10:27:23.070178+0000","last_peered":"2024-04-25T10:27:23.070178+0000","last_clean":"2024-04-25T10:27:23.070178+0000","last_became_active":"2024-04-25T10:27:21.927993+0000","last_became_peered":"2024-04-25T10:27:21.927993+0000","last_unstale":"2024-04-25T10:27:23.070178+0000","last_undegraded":"2024-04-25T10:27:23.070178+0000","last_fullsized":"2024-04-25T10:27:23.070178+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:48.990957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069938+0000","last_change":"2024-04-25T10:27:20.911526+0000","last_active":"2024-04-25T10:27:23.069938+0000","last_peered":"2024-04-25T10:27:23.069938+0000","last_clean":"2024-04-25T10:27:23.069938+0000","last_became_active":"2024-04-25T10:27:20.911143+0000","last_became_peered":"2024-04-25T10:27:20.911143+0000","last_unstale":"2024-04-25T10:27:23.069938+0000","last_undegraded":"2024-04-25T10:27:23.069938+0000","last_fullsized":"2024-04-25T10:27:23.069938+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:27.384578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070826+0000","last_change":"2024-04-25T10:27:21.926832+0000","last_active":"2024-04-25T10:27:23.070826+0000","last_peered":"2024-04-25T10:27:23.070826+0000","last_clean":"2024-04-25T10:27:23.070826+0000","last_became_active":"2024-04-25T10:27:21.926296+0000","last_became_peered":"2024-04-25T10:27:21.926296+0000","last_unstale":"2024-04-25T10:27:23.070826+0000","last_undegraded":"2024-04-25T10:27:23.070826+0000","last_fullsized":"2024-04-25T10:27:23.070826+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:28.390902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.924566+0000","last_change":"2024-04-25T10:27:20.912078+0000","last_active":"2024-04-25T10:27:22.924566+0000","last_peered":"2024-04-25T10:27:22.924566+0000","last_clean":"2024-04-25T10:27:22.924566+0000","last_became_active":"2024-04-25T10:27:20.911765+0000","last_became_peered":"2024-04-25T10:27:20.911765+0000","last_unstale":"2024-04-25T10:27:22.924566+0000","last_undegraded":"2024-04-25T10:27:22.924566+0000","last_fullsized":"2024-04-25T10:27:22.924566+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:56.720505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070665+0000","last_change":"2024-04-25T10:27:21.936864+0000","last_active":"2024-04-25T10:27:23.070665+0000","last_peered":"2024-04-25T10:27:23.070665+0000","last_clean":"2024-04-25T10:27:23.070665+0000","last_became_active":"2024-04-25T10:27:21.936577+0000","last_became_peered":"2024-04-25T10:27:21.936577+0000","last_unstale":"2024-04-25T10:27:23.070665+0000","last_undegraded":"2024-04-25T10:27:23.070665+0000","last_fullsized":"2024-04-25T10:27:23.070665+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:33.299050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070910+0000","last_change":"2024-04-25T10:27:20.910002+0000","last_active":"2024-04-25T10:27:23.070910+0000","last_peered":"2024-04-25T10:27:23.070910+0000","last_clean":"2024-04-25T10:27:23.070910+0000","last_became_active":"2024-04-25T10:27:20.908507+0000","last_became_peered":"2024-04-25T10:27:20.908507+0000","last_unstale":"2024-04-25T10:27:23.070910+0000","last_undegraded":"2024-04-25T10:27:23.070910+0000","last_fullsized":"2024-04-25T10:27:23.070910+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:07.111031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.069562+0000","last_change":"2024-04-25T10:27:21.923984+0000","last_active":"2024-04-25T10:27:23.069562+0000","last_peered":"2024-04-25T10:27:23.069562+0000","last_clean":"2024-04-25T10:27:23.069562+0000","last_became_active":"2024-04-25T10:27:21.923735+0000","last_became_peered":"2024-04-25T10:27:21.923735+0000","last_unstale":"2024-04-25T10:27:23.069562+0000","last_undegraded":"2024-04-25T10:27:23.069562+0000","last_fullsized":"2024-04-25T10:27:23.069562+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:20.883920+0000","last_clean_scrub_stamp":"2024-04-25T10:27:20.883920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:41.294316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:23.070271+0000","last_change":"2024-04-25T10:27:20.911796+0000","last_active":"2024-04-25T10:27:23.070271+0000","last_peered":"2024-04-25T10:27:23.070271+0000","last_clean":"2024-04-25T10:27:23.070271+0000","last_became_active":"2024-04-25T10:27:20.911490+0000","last_became_peered":"2024-04-25T10:27:20.911490+0000","last_unstale":"2024-04-25T10:27:23.070271+0000","last_undegraded":"2024-04-25T10:27:23.070271+0000","last_fullsized":"2024-04-25T10:27:23.070271+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:27:19.877596+0000","last_clean_scrub_stamp":"2024-04-25T10:27:19.877596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:24.288988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":377,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T10:27:22.920880+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:27:22.920880+0000","last_peered":"2024-04-25T10:27:22.920880+0000","last_clean":"2024-04-25T10:27:22.920880+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:27:22.920880+0000","last_undegraded":"2024-04-25T10:27:22.920880+0000","last_fullsized":"2024-04-25T10:27:22.920880+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50812,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321028,"statfs":{"total":96636764160,"available":96584732672,"internally_reserved":0,"allocated":1617920,"data_stored":702855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509559,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50804,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1613824,"data_stored":702821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509558,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2220032,"data_stored":1293931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59128,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312712,"statfs":{"total":96636764160,"available":96576217088,"internally_reserved":0,"allocated":1617920,"data_stored":702891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63296,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308544,"statfs":{"total":96636764160,"available":96571949056,"internally_reserved":0,"allocated":1622016,"data_stored":703493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55568,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316272,"statfs":{"total":96636764160,"available":96579862528,"internally_reserved":0,"allocated":2224128,"data_stored":1294823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1634304,"data_stored":704489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59120,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312720,"statfs":{"total":96636764160,"available":96576225280,"internally_reserved":0,"allocated":1601536,"data_stored":696521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:27:26.432 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:27:26.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:27:26.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.618+0000 7fd87d200640 1 Processor -- start 2024-04-25T10:27:26.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.618+0000 7fd87d200640 1 -- start start 2024-04-25T10:27:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd87d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd8780ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd87d200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8780ae720 0x7fd8780b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd87d200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd878071f50 con 0x7fd8780b4020 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd87d200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8781587a0 con 0x7fd8780ae720 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd87d200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780add50 con 0x7fd8781583c0 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8780ae720 0x7fd8780b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8780ae720 0x7fd8780b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44816/0 (socket says 172.21.15.45:44816) 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 -- 172.21.15.45:0/1345021622 learned_addr learned my addr 172.21.15.45:0/1345021622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd875e00640 1 --2- 172.21.15.45:0/1345021622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd8780ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 -- 172.21.15.45:0/1345021622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 msgr2=0x7fd8780ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 --2- 172.21.15.45:0/1345021622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd8780ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 -- 172.21.15.45:0/1345021622 >> v1:172.21.15.117:6789/0 conn(0x7fd8780b4020 legacy=0x7fd87806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 -- 172.21.15.45:0/1345021622 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878072230 con 0x7fd8780ae720 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd86d400640 1 --2- 172.21.15.45:0/1345021622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780b3ae0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd8780b2910 tx=0x7fd86001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a18472517a5da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.622+0000 7fd875400640 1 -- 172.21.15.45:0/1345021622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8600b0070 con 0x7fd8780ae720 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875400640 1 -- 172.21.15.45:0/1345021622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8600ab600 con 0x7fd8780ae720 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875400640 1 -- 172.21.15.45:0/1345021622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8600ab9d0 con 0x7fd8780ae720 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- 172.21.15.45:0/1345021622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 msgr2=0x7fd8780b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 --2- 172.21.15.45:0/1345021622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780b3ae0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fd8780b2910 tx=0x7fd86001c450 comp rx=0 tx=0).stop 2024-04-25T10:27:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- 172.21.15.45:0/1345021622 shutdown_connections 2024-04-25T10:27:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 --2- 172.21.15.45:0/1345021622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780b3ae0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 --2- 172.21.15.45:0/1345021622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd8780ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- 172.21.15.45:0/1345021622 >> 172.21.15.45:0/1345021622 conn(0x7fd87806b760 msgr2=0x7fd8780b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- 172.21.15.45:0/1345021622 shutdown_connections 2024-04-25T10:27:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- 172.21.15.45:0/1345021622 wait complete. 2024-04-25T10:27:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 Processor -- start 2024-04-25T10:27:26.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- start start 2024-04-25T10:27:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8780b4020 0x7fd87814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd878157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87806e8b0 con 0x7fd8780ae720 2024-04-25T10:27:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87806ea30 con 0x7fd8780b4020 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd87d200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87816a8d0 con 0x7fd8781583c0 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44818/0 (socket says 172.21.15.45:44818) 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 -- 172.21.15.45:0/3305805303 learned_addr learned my addr 172.21.15.45:0/3305805303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd86d400640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8780b4020 0x7fd87814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd876800640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd878157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 -- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 msgr2=0x7fd878157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd878157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 -- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8780b4020 msgr2=0x7fd87814d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8780b4020 0x7fd87814d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.626+0000 7fd875e00640 1 -- 172.21.15.45:0/3305805303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87814d9f0 con 0x7fd8780ae720 2024-04-25T10:27:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd875e00640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780cef20 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd868007ef0 tx=0x7fd86800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868014620 con 0x7fd8780ae720 2024-04-25T10:27:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868019070 con 0x7fd8780ae720 2024-04-25T10:27:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8780cde30 con 0x7fd8780ae720 2024-04-25T10:27:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868014a00 con 0x7fd8780ae720 2024-04-25T10:27:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8780ce720 con 0x7fd8780ae720 2024-04-25T10:27:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87804ef30 con 0x7fd8780ae720 2024-04-25T10:27:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd868014ba0 con 0x7fd8780ae720 2024-04-25T10:27:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86ea00640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd854066f90 0x7fd854069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8680857d0 con 0x7fd8780ae720 2024-04-25T10:27:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.630+0000 7fd86d400640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd854066f90 0x7fd854069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.634+0000 7fd86d400640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd854066f90 0x7fd854069450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8600ad000 tx=0x7fd860008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.634+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd868056180 con 0x7fd8780ae720 2024-04-25T10:27:26.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.854+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd87815e470 con 0x7fd8780ae720 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.854+0000 7fd86ea00640 1 -- 172.21.15.45:0/3305805303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd86805a220 con 0x7fd8780ae720 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:27:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:27:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd854066f90 msgr2=0x7fd854069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd854066f90 0x7fd854069450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8600ad000 tx=0x7fd860008040 comp rx=0 tx=0).stop 2024-04-25T10:27:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 msgr2=0x7fd8780cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780cef20 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fd868007ef0 tx=0x7fd86800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 shutdown_connections 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd854066f90 0x7fd854069450 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8781583c0 0x7fd878157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8780b4020 0x7fd87814d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 --2- 172.21.15.45:0/3305805303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8780ae720 0x7fd8780cef20 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 >> 172.21.15.45:0/3305805303 conn(0x7fd87806b760 msgr2=0x7fd87806e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 shutdown_connections 2024-04-25T10:27:26.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:26.858+0000 7fd87d200640 1 -- 172.21.15.45:0/3305805303 wait complete. 2024-04-25T10:27:26.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:27.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 Processor -- start 2024-04-25T10:27:27.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 -- start start 2024-04-25T10:27:27.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 0x7f23b8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23b815c0a0 0x7f23b80669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b815c640 con 0x7f23b8066ee0 2024-04-25T10:27:27.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b8067350 con 0x7f23b815c0a0 2024-04-25T10:27:27.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bf000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815d260 con 0x7f23b8156030 2024-04-25T10:27:27.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bd000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f23b8066ee0 0x7f23b80af490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51830/0 (socket says 172.21.15.45:51830) 2024-04-25T10:27:27.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bd000640 1 -- 172.21.15.45:0/3844881365 learned_addr learned my addr 172.21.15.45:0/3844881365 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:27.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 0x7f23b8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7400640 1 --2- 172.21.15.45:0/3844881365 >> v2:172.21.15.117:3300/0 conn(0x7f23b815c0a0 0x7f23b80669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88774799 0 0) 0x7f23b815c640 con 0x7f23b8066ee0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f239c003530 con 0x7f23b8066ee0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539874146 0 0) 0x7f239c003530 con 0x7f23b8066ee0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 msgr2=0x7f23b8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 0x7f23b8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 >> v2:172.21.15.117:3300/0 conn(0x7f23b815c0a0 msgr2=0x7f23b80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 --2- 172.21.15.45:0/3844881365 >> v2:172.21.15.117:3300/0 conn(0x7f23b815c0a0 0x7f23b80669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b815c920 con 0x7f23b8066ee0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f23b0001a00 con 0x7f23b8066ee0 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 >> v1:172.21.15.45:6789/0 conn(0x7f23b8066ee0 legacy=0x7f23b80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f239c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 0x7f239c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8067350 con 0x7f239c0039c0 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00b9d0 con 0x7f239c004d50 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c00bb50 con 0x7f23b8156030 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f239c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7400640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 0x7f239c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23bd000640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 -- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 msgr2=0x7f239c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b8156030 0x7f239c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 -- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c004d50 msgr2=0x7f239c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239c004d50 0x7f239c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.022+0000 7f23b7e00640 1 -- 172.21.15.45:0/3844881365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c003530 con 0x7f239c0039c0 2024-04-25T10:27:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23b7e00640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f239c003de0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009ae0 tx=0x7f23a801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b4eadc787c7d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80b2070 con 0x7f239c0039c0 2024-04-25T10:27:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a80a5610 con 0x7f239c0039c0 2024-04-25T10:27:27.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23b6a00640 1 -- 172.21.15.45:0/3844881365 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80096b0 con 0x7f239c0039c0 2024-04-25T10:27:27.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bf000640 1 -- 172.21.15.45:0/3844881365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 msgr2=0x7f239c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bf000640 1 --2- 172.21.15.45:0/3844881365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f239c003de0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009ae0 tx=0x7f23a801ca40 comp rx=0 tx=0).stop 2024-04-25T10:27:27.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bd000640 1 -- 172.21.15.45:0/3844881365 reap_dead start 2024-04-25T10:27:27.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bf000640 1 -- 172.21.15.45:0/3844881365 shutdown_connections 2024-04-25T10:27:27.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bf000640 1 -- 172.21.15.45:0/3844881365 >> 172.21.15.45:0/3844881365 conn(0x7f23b806ac90 msgr2=0x7f23b806b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bf000640 1 -- 172.21.15.45:0/3844881365 shutdown_connections 2024-04-25T10:27:27.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.026+0000 7f23bf000640 1 -- 172.21.15.45:0/3844881365 wait complete. 2024-04-25T10:27:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 Processor -- start 2024-04-25T10:27:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 -- start start 2024-04-25T10:27:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f23b81f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b814c220 0x7f23b815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b815b730 0x7f23b815fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80724a0 con 0x7f239c0039c0 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8072320 con 0x7f23b815b730 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815d6d0 con 0x7f23b814c220 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f23b81f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f23b81f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44832/0 (socket says 172.21.15.45:44832) 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 -- 172.21.15.45:0/1321504341 learned_addr learned my addr 172.21.15.45:0/1321504341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bd000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b815b730 0x7f23b815fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7400640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b814c220 0x7f23b815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 -- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b814c220 msgr2=0x7f23b815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b814c220 0x7f23b815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 -- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b815b730 msgr2=0x7f23b815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b815b730 0x7f23b815fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 -- 172.21.15.45:0/1321504341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b81f7320 con 0x7f239c0039c0 2024-04-25T10:27:27.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23b7e00640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f23b81f6de0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009ae0 tx=0x7f239c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80b2040 con 0x7f239c0039c0 2024-04-25T10:27:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800f000 con 0x7f239c0039c0 2024-04-25T10:27:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b80ae5d0 con 0x7f239c0039c0 2024-04-25T10:27:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004020 con 0x7f239c0039c0 2024-04-25T10:27:27.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.030+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b80aeb90 con 0x7f239c0039c0 2024-04-25T10:27:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.034+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b804ef30 con 0x7f239c0039c0 2024-04-25T10:27:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.034+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23a80c0030 con 0x7f239c0039c0 2024-04-25T10:27:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.034+0000 7f23afe00640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.034+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23a811ec00 con 0x7f239c0039c0 2024-04-25T10:27:27.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.034+0000 7f23b7400640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.034+0000 7f23b7400640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f23b80d17b0 tx=0x7f23a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.038+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23a8125080 con 0x7f239c0039c0 2024-04-25T10:27:27.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.234+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23b80ab7d0 con 0x7f239c0039c0 2024-04-25T10:27:27.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.234+0000 7f23afe00640 1 -- 172.21.15.45:0/1321504341 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f23a80c7060 con 0x7f239c0039c0 2024-04-25T10:27:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:27.238 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:27.239 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 msgr2=0x7f2398069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f23b80d17b0 tx=0x7f23a4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 msgr2=0x7f23b81f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f23b81f6de0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009ae0 tx=0x7f239c005840 comp rx=0 tx=0).stop 2024-04-25T10:27:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 shutdown_connections 2024-04-25T10:27:27.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2398066f60 0x7f2398069420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b815b730 0x7f23b815fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23b814c220 0x7f23b815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 --2- 172.21.15.45:0/1321504341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f239c0039c0 0x7f23b81f6de0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 >> 172.21.15.45:0/1321504341 conn(0x7f23b806ac90 msgr2=0x7f23b806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 shutdown_connections 2024-04-25T10:27:27.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.238+0000 7f23bf000640 1 -- 172.21.15.45:0/1321504341 wait complete. 2024-04-25T10:27:27.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:27:21.291341+0000', 'modified': '2024-04-25T10:27:24.321672+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9012': {'gid': 9012, 'name': 'c', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/1857937743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1857937743}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1857937743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:27:27.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:27:27.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:27:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.806+0000 7f608c400640 1 Processor -- start 2024-04-25T10:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.806+0000 7f608c400640 1 -- start start 2024-04-25T10:27:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608c400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066fe0 0x7f608415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608c400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60840b5560 0x7f6084157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608c400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60840b5c20 con 0x7f6084066c00 2024-04-25T10:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608c400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60840b59f0 con 0x7f60840b5560 2024-04-25T10:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608c400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60840af6e0 con 0x7f6084066fe0 2024-04-25T10:27:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6084066c00 0x7f608406ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:48656/0 (socket says 172.21.15.45:48656) 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089a00640 1 -- 172.21.15.45:0/2726002259 learned_addr learned my addr 172.21.15.45:0/2726002259 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608a400640 1 --2- 172.21.15.45:0/2726002259 >> v2:172.21.15.117:3300/0 conn(0x7f60840b5560 0x7f6084157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089000640 1 --2- 172.21.15.45:0/2726002259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066fe0 0x7f608415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089000640 1 -- 172.21.15.45:0/2726002259 >> v2:172.21.15.117:3300/0 conn(0x7f60840b5560 msgr2=0x7f6084157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089000640 1 --2- 172.21.15.45:0/2726002259 >> v2:172.21.15.117:3300/0 conn(0x7f60840b5560 0x7f6084157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089000640 1 -- 172.21.15.45:0/2726002259 >> v1:172.21.15.117:6789/0 conn(0x7f6084066c00 legacy=0x7f608406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6083e00640 1 -- 172.21.15.45:0/2726002259 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796856457 0 0) 0x7f60840b5c20 con 0x7f6084066c00 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f6089000640 1 -- 172.21.15.45:0/2726002259 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60840af9c0 con 0x7f6084066fe0 2024-04-25T10:27:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.810+0000 7f608a400640 1 --2- 172.21.15.45:0/2726002259 >> v2:172.21.15.117:3300/0 conn(0x7f60840b5560 0x7f6084157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:27.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f6089000640 1 --2- 172.21.15.45:0/2726002259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066fe0 0x7f608415e110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f60700097c0 tx=0x7f607001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=696babfd92242428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f6083e00640 1 -- 172.21.15.45:0/2726002259 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700a7070 con 0x7f6084066fe0 2024-04-25T10:27:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f6083e00640 1 -- 172.21.15.45:0/2726002259 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60700b0070 con 0x7f6084066fe0 2024-04-25T10:27:27.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f6083e00640 1 -- 172.21.15.45:0/2726002259 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700ab890 con 0x7f6084066fe0 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 -- 172.21.15.45:0/2726002259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066fe0 msgr2=0x7f608415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 --2- 172.21.15.45:0/2726002259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066fe0 0x7f608415e110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f60700097c0 tx=0x7f607001c470 comp rx=0 tx=0).stop 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 -- 172.21.15.45:0/2726002259 shutdown_connections 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 --2- 172.21.15.45:0/2726002259 >> v2:172.21.15.117:3300/0 conn(0x7f60840b5560 0x7f6084157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 --2- 172.21.15.45:0/2726002259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066fe0 0x7f608415e110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 -- 172.21.15.45:0/2726002259 >> 172.21.15.45:0/2726002259 conn(0x7f60840b24f0 msgr2=0x7f60840b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 -- 172.21.15.45:0/2726002259 shutdown_connections 2024-04-25T10:27:27.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 -- 172.21.15.45:0/2726002259 wait complete. 2024-04-25T10:27:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 Processor -- start 2024-04-25T10:27:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.814+0000 7f608c400640 1 -- start start 2024-04-25T10:27:27.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066c00 0x7f608415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6084066fe0 0x7f6084147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 0x7f6084160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60840af1e0 con 0x7f60840b5560 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60840af060 con 0x7f6084066fe0 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608406ac90 con 0x7f6084066c00 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 0x7f6084160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 0x7f6084160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44840/0 (socket says 172.21.15.45:44840) 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 -- 172.21.15.45:0/1369254711 learned_addr learned my addr 172.21.15.45:0/1369254711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6089000640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6084066fe0 0x7f6084147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 -- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066c00 msgr2=0x7f608415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6089a00640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066c00 0x7f608415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066c00 0x7f608415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 -- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6084066fe0 msgr2=0x7f6084147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6084066fe0 0x7f6084147ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 -- 172.21.15.45:0/1369254711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60841fb030 con 0x7f60840b5560 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6089000640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6084066fe0 0x7f6084147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6089a00640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066c00 0x7f608415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608a400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 0x7f6084160700 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f607c00e960 tx=0x7f607c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6081600640 1 -- 172.21.15.45:0/1369254711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c017070 con 0x7f60840b5560 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6081600640 1 -- 172.21.15.45:0/1369254711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c00ca00 con 0x7f60840b5560 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f6081600640 1 -- 172.21.15.45:0/1369254711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00ce10 con 0x7f60840b5560 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608415a6c0 con 0x7f60840b5560 2024-04-25T10:27:27.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.818+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6084148420 con 0x7f60840b5560 2024-04-25T10:27:27.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6081600640 1 -- 172.21.15.45:0/1369254711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f607c013470 con 0x7f60840b5560 2024-04-25T10:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6081600640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6064066f60 0x7f6064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6081600640 1 -- 172.21.15.45:0/1369254711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f607c084f30 con 0x7f60840b5560 2024-04-25T10:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6064066f60 0x7f6064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 Processor -- start 2024-04-25T10:27:27.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 -- start start 2024-04-25T10:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6064066f60 0x7f6064069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6078007fd0 tx=0x7f6078009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3bf5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3bab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3bac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f3a935f0 con 0x55c8f3bf5920 2024-04-25T10:27:27.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f39fada0 con 0x55c8f3babf30 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f3983150 con 0x55c8f3bf6240 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3bf5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3bf5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44856/0 (socket says 172.21.15.45:44856) 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 -- 172.21.15.45:0/1196899358 learned_addr learned my addr 172.21.15.45:0/1196899358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 -- 172.21.15.45:0/1196899358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 msgr2=0x55c8f3bab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3bab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 -- 172.21.15.45:0/1196899358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 msgr2=0x55c8f3bac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3bac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.822+0000 7f6089a00640 1 -- 172.21.15.45:0/1196899358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f399cca0 con 0x55c8f3bf5920 2024-04-25T10:27:27.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f6089a00640 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3bf5d00 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f607800def0 tx=0x7f6078010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8a631566b86875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f6076000640 1 -- 172.21.15.45:0/1196899358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078020450 con 0x55c8f3bf5920 2024-04-25T10:27:27.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f6076000640 1 -- 172.21.15.45:0/1196899358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60780205f0 con 0x55c8f3bf5920 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f6076000640 1 -- 172.21.15.45:0/1196899358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780209c0 con 0x55c8f3bf5920 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- 172.21.15.45:0/1196899358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 msgr2=0x55c8f3bf5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3bf5d00 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f607800def0 tx=0x7f6078010000 comp rx=0 tx=0).stop 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- 172.21.15.45:0/1196899358 shutdown_connections 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3bac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3bab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- 172.21.15.45:0/1196899358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3bf5d00 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- 172.21.15.45:0/1196899358 >> 172.21.15.45:0/1196899358 conn(0x55c8f3bb22a0 msgr2=0x55c8f3baa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- 172.21.15.45:0/1196899358 shutdown_connections 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- 172.21.15.45:0/1196899358 wait complete. 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- start start 2024-04-25T10:27:27.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 Processor -- start 2024-04-25T10:27:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3be2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf5920 0x55c8f3bafaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3baffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f39b5990 con 0x55c8f3bf6240 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f39f6e40 con 0x55c8f3babf30 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f3986900 con 0x55c8f3bf5920 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608f7e9000 10 client.0 _ll_register_callbacks cb 0x55c8f3aae1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3baffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3baffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44860/0 (socket says 172.21.15.45:44860) 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 -- 172.21.15.45:0/3227638906 learned_addr learned my addr 172.21.15.45:0/3227638906 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f6089a00640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3be2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f6089000640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf5920 0x55c8f3bafaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 -- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf5920 msgr2=0x55c8f3bafaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf5920 0x55c8f3bafaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 -- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 msgr2=0x55c8f3be2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3be2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.826+0000 7f608a400640 1 -- 172.21.15.45:0/3227638906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8f39b3630 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608a400640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3baffe0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f607c009fd0 tx=0x7f607c055fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05f4baf3f4a77ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 1 -- 172.21.15.45:0/3227638906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c06d020 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60780205f0 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 1 -- 172.21.15.45:0/3227638906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f607c054bf0 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 1 -- 172.21.15.45:0/3227638906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607c020040 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 10 client.9153 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55c8f39833a0 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 1 -- 172.21.15.45:0/3227638906 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60780205f0 con 0x55c8f3bf6240 2024-04-25T10:27:27.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 1 -- 172.21.15.45:0/3227638906 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f607c023060 con 0x55c8f3bf6240 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 10 client.9153 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 10 client.9153 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 20 client.9153 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c04b30 0x55c8f3c06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 4 client.9153 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8f3996a50 con 0x55c8f3c04b30 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f6089a00640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c04b30 0x55c8f3c06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 10 client.9153 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:27.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f6089a00640 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c04b30 0x55c8f3c06f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a0b4797ad00b30 server_cookie=e86e7975b819fd04 in_seq=0 out_seq=0 2024-04-25T10:27:27.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 1 -- 172.21.15.45:0/3227638906 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8f3996a50 con 0x55c8f3c04b30 2024-04-25T10:27:27.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 10 client.9153 handle_command_reply: tid=0 2024-04-25T10:27:27.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 client.9153 shutdown 2024-04-25T10:27:27.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 msgr2=0x55c8f3baffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3baffe0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f607c009fd0 tx=0x7f607c055fe0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 shutdown_connections 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3baffe0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf5920 0x55c8f3bafaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c04b30 0x55c8f3c06f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 --2- 172.21.15.45:0/3227638906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3be2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 >> 172.21.15.45:0/3227638906 conn(0x55c8f3bb22a0 msgr2=0x55c8f3baa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f606aa00640 0 client.9153 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 shutdown_connections 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 1 -- 172.21.15.45:0/3227638906 wait complete. 2024-04-25T10:27:27.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.830+0000 7f608f7e9000 20 client.9153 trim_cache size 0 max 16384 2024-04-25T10:27:27.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 Processor -- start 2024-04-25T10:27:27.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 -- start start 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3babf30 0x55c8f3c08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3bf5920 0x55c8f3c091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3c04f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084150f30 con 0x55c8f3bf6240 2024-04-25T10:27:27.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084159420 con 0x55c8f3bf5920 2024-04-25T10:27:27.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.846+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6084150930 con 0x55c8f3babf30 2024-04-25T10:27:27.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3c04f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3c04f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44868/0 (socket says 172.21.15.45:44868) 2024-04-25T10:27:27.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 -- 172.21.15.45:0/495077882 learned_addr learned my addr 172.21.15.45:0/495077882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:27.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089a00640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3babf30 0x55c8f3c08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 -- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3babf30 msgr2=0x55c8f3c08c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089000640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3bf5920 0x55c8f3c091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3babf30 0x55c8f3c08c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 -- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3bf5920 msgr2=0x55c8f3c091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3bf5920 0x55c8f3c091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 -- 172.21.15.45:0/495077882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f3bf6070 con 0x55c8f3bf6240 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089000640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3bf5920 0x55c8f3c091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089a00640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3babf30 0x55c8f3c08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3c04f10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f607c00e930 tx=0x7f607c059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37caff4801505f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6075600640 1 -- 172.21.15.45:0/495077882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c06d040 con 0x55c8f3bf6240 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6075600640 1 -- 172.21.15.45:0/495077882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c054bf0 con 0x55c8f3bf6240 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6075600640 1 -- 172.21.15.45:0/495077882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c0514f0 con 0x55c8f3bf6240 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- 172.21.15.45:0/495077882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 msgr2=0x55c8f3c04f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3c04f10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f607c00e930 tx=0x7f607c059000 comp rx=0 tx=0).stop 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- 172.21.15.45:0/495077882 shutdown_connections 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf6240 0x55c8f3c04f10 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3bf5920 0x55c8f3c091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- 172.21.15.45:0/495077882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3babf30 0x55c8f3c08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- 172.21.15.45:0/495077882 >> 172.21.15.45:0/495077882 conn(0x55c8f3bb22a0 msgr2=0x55c8f3bf40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- 172.21.15.45:0/495077882 shutdown_connections 2024-04-25T10:27:27.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- 172.21.15.45:0/495077882 wait complete. 2024-04-25T10:27:27.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- start start 2024-04-25T10:27:27.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 Processor -- start 2024-04-25T10:27:27.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3c08bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3c09110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3c56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f39ff3d0 con 0x55c8f3bf5920 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f3ab6d50 con 0x55c8f3babf30 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f3995980 con 0x55c8f3bf6240 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608f7e9000 10 client.0 _ll_register_callbacks cb 0x55c8f3a002c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3c09110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3c08bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3c09110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44874/0 (socket says 172.21.15.45:44874) 2024-04-25T10:27:27.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f6089000640 1 -- 172.21.15.45:0/2065385633 learned_addr learned my addr 172.21.15.45:0/2065385633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.850+0000 7f608a400640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3c56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089000640 1 -- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 msgr2=0x55c8f3c56960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089000640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3c56960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089000640 1 -- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 msgr2=0x55c8f3c08bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089000640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3c08bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089000640 1 -- 172.21.15.45:0/2065385633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8f3a7c600 con 0x55c8f3bf5920 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089a00640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3c08bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089000640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3c09110 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f607001c950 tx=0x7f607001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93e1f14f5e32dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 1 -- 172.21.15.45:0/2065385633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700c3060 con 0x55c8f3bf5920 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f607c054bf0 con 0x55c8f3bf5920 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 1 -- 172.21.15.45:0/2065385633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60700aba70 con 0x55c8f3bf5920 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 1 -- 172.21.15.45:0/2065385633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60700c9050 con 0x55c8f3bf5920 2024-04-25T10:27:27.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 10 client.9159 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55c8f398f9b0 con 0x55c8f3bf5920 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 1 -- 172.21.15.45:0/2065385633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c054bf0 con 0x55c8f3bf5920 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 1 -- 172.21.15.45:0/2065385633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f60700cc020 con 0x55c8f3bf5920 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 10 client.9159 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 10 client.9159 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 20 client.9159 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c56ea0 0x55c8f3c57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 4 client.9159 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8f3a082e0 con 0x55c8f3c56ea0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089a00640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c56ea0 0x55c8f3c57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f6089a00640 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c56ea0 0x55c8f3c57280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97be9ff4e8f70304 server_cookie=e5d10029e183065d in_seq=0 out_seq=0 2024-04-25T10:27:27.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 10 client.9159 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 1 -- 172.21.15.45:0/2065385633 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c8f3a082e0 con 0x55c8f3c56ea0 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 10 client.9159 handle_command_reply: tid=0 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 client.9159 shutdown 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 msgr2=0x55c8f3c09110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3c09110 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f607001c950 tx=0x7f607001acb0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 shutdown_connections 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c8f3bf6240 0x55c8f3c56960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8f3bf5920 0x55c8f3c09110 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x55c8f3c56ea0 0x55c8f3c57280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 --2- 172.21.15.45:0/2065385633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8f3babf30 0x55c8f3c08bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 >> 172.21.15.45:0/2065385633 conn(0x55c8f3bb22a0 msgr2=0x55c8f3bf47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f606aa00640 0 client.9159 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 shutdown_connections 2024-04-25T10:27:27.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 1 -- 172.21.15.45:0/2065385633 wait complete. 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608f7e9000 20 client.9159 trim_cache size 0 max 16384 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9012, 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:27:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.958146256, 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 91, 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 130, 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 130, 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.765669533000001 2024-04-25T10:27:27.864 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6064066f60 msgr2=0x7f6064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6064066f60 0x7f6064069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6078007fd0 tx=0x7f6078009040 comp rx=0 tx=0).stop 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 msgr2=0x7f6084160700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 0x7f6084160700 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f607c00e960 tx=0x7f607c00ee20 comp rx=0 tx=0).stop 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 shutdown_connections 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6064066f60 0x7f6064069420 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60840b5560 0x7f6084160700 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6084066fe0 0x7f6084147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 --2- 172.21.15.45:0/1369254711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6084066c00 0x7f608415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 >> 172.21.15.45:0/1369254711 conn(0x7f60840b24f0 msgr2=0x7f608406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 shutdown_connections 2024-04-25T10:27:27.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:27.854+0000 7f608c400640 1 -- 172.21.15.45:0/1369254711 wait complete. 2024-04-25T10:27:27.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:27:28.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.014+0000 7fd8c2e00640 1 Processor -- start 2024-04-25T10:27:28.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.014+0000 7fd8c2e00640 1 -- start start 2024-04-25T10:27:28.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c2e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8bc066c00 0x7fd8bc15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c2e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 0x7fd8bc157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c2e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc0b59b0 con 0x7fd8bc066fe0 2024-04-25T10:27:28.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c2e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b5c20 con 0x7fd8bc066c00 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c2e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0af6e0 con 0x7fd8bc0b5560 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8b3400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8bc066fe0 0x7fd8bc06ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51840/0 (socket says 172.21.15.45:51840) 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8b3400640 1 -- 172.21.15.45:0/1064199547 learned_addr learned my addr 172.21.15.45:0/1064199547 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/1064199547 >> v2:172.21.15.117:3300/0 conn(0x7fd8bc066c00 0x7fd8bc15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c0e00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 0x7fd8bc157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293532086 0 0) 0x7fd8bc0b59b0 con 0x7fd8bc066fe0 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a0003530 con 0x7fd8bc066fe0 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766384144 0 0) 0x7fd8a0003530 con 0x7fd8bc066fe0 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 msgr2=0x7fd8bc157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 0x7fd8bc157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 >> v2:172.21.15.117:3300/0 conn(0x7fd8bc066c00 msgr2=0x7fd8bc15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 --2- 172.21.15.45:0/1064199547 >> v2:172.21.15.117:3300/0 conn(0x7fd8bc066c00 0x7fd8bc15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0af9c0 con 0x7fd8bc066fe0 2024-04-25T10:27:28.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fd8a4001680 con 0x7fd8bc066fe0 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 >> v1:172.21.15.45:6789/0 conn(0x7fd8bc066fe0 legacy=0x7fd8bc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a00039c0 0x7fd8a0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 0x7fd8a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a0004d50 0x7fd8a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0b5c20 con 0x7fd8a00039c0 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0af6e0 con 0x7fd8a0004d50 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a000b9d0 con 0x7fd8bc0b5560 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a00039c0 0x7fd8a0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8b3400640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 0x7fd8a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8c0e00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a0004d50 0x7fd8a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 -- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 msgr2=0x7fd8a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b5560 0x7fd8a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 -- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a0004d50 msgr2=0x7fd8a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:28.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a0004d50 0x7fd8a00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.018+0000 7fd8bbe00640 1 -- 172.21.15.45:0/1064199547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0003530 con 0x7fd8a00039c0 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a00039c0 0x7fd8a0003de0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009ae0 tx=0x7fd8ac01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ea271b33b4d421 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0b2070 con 0x7fd8a00039c0 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0a5650 con 0x7fd8a00039c0 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8bb400640 1 -- 172.21.15.45:0/1064199547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0096b0 con 0x7fd8a00039c0 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c2e00640 1 -- 172.21.15.45:0/1064199547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a00039c0 msgr2=0x7fd8a0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/1064199547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a00039c0 0x7fd8a0003de0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009ae0 tx=0x7fd8ac01ca80 comp rx=0 tx=0).stop 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c0e00640 1 -- 172.21.15.45:0/1064199547 reap_dead start 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c2e00640 1 -- 172.21.15.45:0/1064199547 shutdown_connections 2024-04-25T10:27:28.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c2e00640 1 -- 172.21.15.45:0/1064199547 >> 172.21.15.45:0/1064199547 conn(0x7fd8bc0b24f0 msgr2=0x7fd8bc0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c2e00640 1 -- 172.21.15.45:0/1064199547 shutdown_connections 2024-04-25T10:27:28.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.022+0000 7fd8c2e00640 1 -- 172.21.15.45:0/1064199547 wait complete. 2024-04-25T10:27:28.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 Processor -- start 2024-04-25T10:27:28.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- start start 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a00039c0 0x7fd8bc153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 0x7fd8bc13eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b0b80 0x7fd8bc0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0ae850 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc158570 con 0x7fd8a00039c0 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc0ae9d0 con 0x7fd8bc0b0b80 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 0x7fd8bc13eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 0x7fd8bc13eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44890/0 (socket says 172.21.15.45:44890) 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 -- 172.21.15.45:0/3532476316 learned_addr learned my addr 172.21.15.45:0/3532476316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c0e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b0b80 0x7fd8bc0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a00039c0 0x7fd8bc153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 -- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b0b80 msgr2=0x7fd8bc0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b0b80 0x7fd8bc0b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 -- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a00039c0 msgr2=0x7fd8bc153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a00039c0 0x7fd8bc153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 -- 172.21.15.45:0/3532476316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc160220 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c0e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b0b80 0x7fd8bc0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b3400640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 0x7fd8bc13eee0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4007370 tx=0x7fd8a4007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400f960 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4018070 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400fd40 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc160500 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc160ac0 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.026+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc0b1540 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.030+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8a401d460 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.030+0000 7fd8b8c00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd898066f90 0x7fd898069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.030+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8a4085160 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.030+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd898066f90 0x7fd898069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:28.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.030+0000 7fd8bbe00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd898066f90 0x7fd898069450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0af000 tx=0x7fd8ac0046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:28.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.034+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8a4056160 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.230+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd8bc071c70 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.230+0000 7fd8b8c00640 1 -- 172.21.15.45:0/3532476316 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd8a4055a60 con 0x7fd8bc15f9d0 2024-04-25T10:27:28.235 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd898066f90 msgr2=0x7fd898069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd898066f90 0x7fd898069450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0af000 tx=0x7fd8ac0046d0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 msgr2=0x7fd8bc13eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 0x7fd8bc13eee0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4007370 tx=0x7fd8a4007830 comp rx=0 tx=0).stop 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 shutdown_connections 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd898066f90 0x7fd898069450 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8bc0b0b80 0x7fd8bc0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8bc15f9d0 0x7fd8bc13eee0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 --2- 172.21.15.45:0/3532476316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8a00039c0 0x7fd8bc153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 >> 172.21.15.45:0/3532476316 conn(0x7fd8bc0b24f0 msgr2=0x7fd8bc0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 shutdown_connections 2024-04-25T10:27:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:28.234+0000 7fd8c2e00640 1 -- 172.21.15.45:0/3532476316 wait complete. 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:27:28.256 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:27:28.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.257 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:27:28.266 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41892sec preferred_lft 41892sec 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:27:28.267 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.268 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:27:28.269 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:27:28.269 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:27:28.269 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:27:28.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.270 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:27:28.333 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:27:28.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.336 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:27:28.388 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:27:28.388 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:27:28.389 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:27:28.389 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:27:28.389 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:27:28.389 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:27:28.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.389 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:27:28.389 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:28.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:27:28.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:27:28.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:27:28.390 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:28.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.486 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:27:28.494 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:27:28.495 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:27:28.495 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:27:28.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:27:28.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:27:28.559 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:27:28.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:27:28.611 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:27:28.663 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:27:28.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:27:28.756 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:27:28.756 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:27:28.769 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:27:28.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:27:28.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:27:28.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:27:28.769 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:27:28.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:27:28.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:27:28.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:28.794 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:27:28.854 INFO:teuthology.orchestra.run.smithi045.stdout:client7562 2024-04-25T10:27:28.854 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:27:28.854 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:27:28.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 Processor -- start 2024-04-25T10:27:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 -- start start 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f59c815dce0 0x7f59c80669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59c806eec0 0x7f59c806df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c815e0c0 con 0x7f59c80ad850 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c80674d0 con 0x7f59c806eec0 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59ce800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c80671f0 con 0x7f59c815dce0 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59c806eec0 0x7f59c806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f59c80ad850 0x7f59c80adcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51852/0 (socket says 172.21.15.45:51852) 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c7e00640 1 -- 172.21.15.45:0/396101976 learned_addr learned my addr 172.21.15.45:0/396101976 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c7400640 1 --2- 172.21.15.45:0/396101976 >> v2:172.21.15.117:3300/0 conn(0x7f59c815dce0 0x7f59c80669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c6a00640 1 -- 172.21.15.45:0/396101976 >> v2:172.21.15.117:3300/0 conn(0x7f59c815dce0 msgr2=0x7f59c80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c6a00640 1 --2- 172.21.15.45:0/396101976 >> v2:172.21.15.117:3300/0 conn(0x7f59c815dce0 0x7f59c80669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c6a00640 1 -- 172.21.15.45:0/396101976 >> v1:172.21.15.45:6789/0 conn(0x7f59c80ad850 legacy=0x7f59c80adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c6a00640 1 -- 172.21.15.45:0/396101976 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c806d4b0 con 0x7f59c806eec0 2024-04-25T10:27:29.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.090+0000 7f59c6000640 1 -- 172.21.15.45:0/396101976 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252639918 0 0) 0x7f59c815e0c0 con 0x7f59c80ad850 2024-04-25T10:27:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59c6a00640 1 --2- 172.21.15.45:0/396101976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c806df90 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f59c8074d80 tx=0x7f59b401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f57e17567fe854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59c6000640 1 -- 172.21.15.45:0/396101976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40b0070 con 0x7f59c806eec0 2024-04-25T10:27:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59c6000640 1 -- 172.21.15.45:0/396101976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b40ab600 con 0x7f59c806eec0 2024-04-25T10:27:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59c6000640 1 -- 172.21.15.45:0/396101976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40ab9d0 con 0x7f59c806eec0 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 -- 172.21.15.45:0/396101976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 msgr2=0x7f59c806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 --2- 172.21.15.45:0/396101976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c806df90 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f59c8074d80 tx=0x7f59b401c360 comp rx=0 tx=0).stop 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 -- 172.21.15.45:0/396101976 shutdown_connections 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 --2- 172.21.15.45:0/396101976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c806df90 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 --2- 172.21.15.45:0/396101976 >> v2:172.21.15.117:3300/0 conn(0x7f59c815dce0 0x7f59c80669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 -- 172.21.15.45:0/396101976 >> 172.21.15.45:0/396101976 conn(0x7f59c80b2410 msgr2=0x7f59c80b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 -- 172.21.15.45:0/396101976 shutdown_connections 2024-04-25T10:27:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 -- 172.21.15.45:0/396101976 wait complete. 2024-04-25T10:27:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.094+0000 7f59ce800640 1 Processor -- start 2024-04-25T10:27:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- start start 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c8152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59c80ad850 0x7f59c8154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59c815dce0 0x7f59c815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80d0320 con 0x7f59c806eec0 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c80d04a0 con 0x7f59c80ad850 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c816abc0 con 0x7f59c815dce0 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c8152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c8152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44922/0 (socket says 172.21.15.45:44922) 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 -- 172.21.15.45:0/1329495985 learned_addr learned my addr 172.21.15.45:0/1329495985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c6a00640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59c80ad850 0x7f59c8154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7e00640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59c815dce0 0x7f59c815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 -- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59c815dce0 msgr2=0x7f59c815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59c815dce0 0x7f59c815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 -- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59c80ad850 msgr2=0x7f59c8154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59c80ad850 0x7f59c8154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 -- 172.21.15.45:0/1329495985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c8155230 con 0x7f59c806eec0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59c7400640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c8152ca0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f59b800e930 tx=0x7f59b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8017070 con 0x7f59c806eec0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b800ca00 con 0x7f59c806eec0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800cdb0 con 0x7f59c806eec0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c8155510 con 0x7f59c806eec0 2024-04-25T10:27:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c8155ad0 con 0x7f59c806eec0 2024-04-25T10:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.098+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c804ef30 con 0x7f59c806eec0 2024-04-25T10:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.102+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59b801a070 con 0x7f59c806eec0 2024-04-25T10:27:29.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.102+0000 7f59bf400640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59a8066f60 0x7f59a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.102+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59b80854d0 con 0x7f59c806eec0 2024-04-25T10:27:29.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.102+0000 7f59c6a00640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59a8066f60 0x7f59a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.102+0000 7f59c6a00640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59a8066f60 0x7f59a8069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f59b40ad000 tx=0x7f59b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.106+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59b8056500 con 0x7f59c806eec0 2024-04-25T10:27:29.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.306+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59c80af770 con 0x7f59c806eec0 2024-04-25T10:27:29.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.306+0000 7f59bf400640 1 -- 172.21.15.45:0/1329495985 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f59b8055e00 con 0x7f59c806eec0 2024-04-25T10:27:29.310 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:29.310 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:29.311 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:29.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59a8066f60 msgr2=0x7f59a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59a8066f60 0x7f59a8069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f59b40ad000 tx=0x7f59b4008040 comp rx=0 tx=0).stop 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 msgr2=0x7f59c8152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c8152ca0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f59b800e930 tx=0x7f59b800edf0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 shutdown_connections 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59a8066f60 0x7f59a8069420 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59c815dce0 0x7f59c815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59c80ad850 0x7f59c8154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 --2- 172.21.15.45:0/1329495985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59c806eec0 0x7f59c8152ca0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 >> 172.21.15.45:0/1329495985 conn(0x7f59c80b2410 msgr2=0x7f59c80adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 shutdown_connections 2024-04-25T10:27:29.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.310+0000 7f59ce800640 1 -- 172.21.15.45:0/1329495985 wait complete. 2024-04-25T10:27:29.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:29.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd70d800640 1 Processor -- start 2024-04-25T10:27:29.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd70d800640 1 -- start start 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd70d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7080d0040 0x7fd70815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd70d800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd708071050 con 0x7fd7081550b0 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd70d800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70815d040 con 0x7fd70815b740 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd70d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080712f0 con 0x7fd7080d0040 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd706a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd7081550b0 0x7fd7080cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51868/0 (socket says 172.21.15.45:51868) 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd706a00640 1 -- 172.21.15.45:0/4290450319 learned_addr learned my addr 172.21.15.45:0/4290450319 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd6fe000640 1 --2- 172.21.15.45:0/4290450319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7080d0040 0x7fd70815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126978924 0 0) 0x7fd708071050 con 0x7fd7081550b0 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ec003530 con 0x7fd7081550b0 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801402282 0 0) 0x7fd70815d040 con 0x7fd70815b740 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd708071050 con 0x7fd70815b740 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd6fe000640 1 -- 172.21.15.45:0/4290450319 >> v1:172.21.15.117:6789/0 conn(0x7fd70815b740 legacy=0x7fd7080b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd6fe000640 1 -- 172.21.15.45:0/4290450319 >> v1:172.21.15.45:6789/0 conn(0x7fd7081550b0 legacy=0x7fd7080cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.470+0000 7fd6fe000640 1 -- 172.21.15.45:0/4290450319 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080715d0 con 0x7fd7080d0040 2024-04-25T10:27:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd6fe000640 1 --2- 172.21.15.45:0/4290450319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7080d0040 0x7fd70815b200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00097f0 tx=0x7fd6f001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e51c125797a62de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00b1070 con 0x7fd7080d0040 2024-04-25T10:27:29.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00ac410 con 0x7fd7080d0040 2024-04-25T10:27:29.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd706000640 1 -- 172.21.15.45:0/4290450319 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00ac7c0 con 0x7fd7080d0040 2024-04-25T10:27:29.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 -- 172.21.15.45:0/4290450319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7080d0040 msgr2=0x7fd70815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 --2- 172.21.15.45:0/4290450319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7080d0040 0x7fd70815b200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00097f0 tx=0x7fd6f001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 -- 172.21.15.45:0/4290450319 shutdown_connections 2024-04-25T10:27:29.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 --2- 172.21.15.45:0/4290450319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7080d0040 0x7fd70815b200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 -- 172.21.15.45:0/4290450319 >> 172.21.15.45:0/4290450319 conn(0x7fd708066b70 msgr2=0x7fd708066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 -- 172.21.15.45:0/4290450319 shutdown_connections 2024-04-25T10:27:29.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.474+0000 7fd70d800640 1 -- 172.21.15.45:0/4290450319 wait complete. 2024-04-25T10:27:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 Processor -- start 2024-04-25T10:27:29.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 -- start start 2024-04-25T10:27:29.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7080d0040 0x7fd70814e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7081550b0 0x7fd708153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 0x7fd7080cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70815bcf0 con 0x7fd70815b740 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70815bb70 con 0x7fd7080d0040 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081684c0 con 0x7fd7081550b0 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 0x7fd7080cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 0x7fd7080cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44932/0 (socket says 172.21.15.45:44932) 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 -- 172.21.15.45:0/1202790424 learned_addr learned my addr 172.21.15.45:0/1202790424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd6fe000640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7081550b0 0x7fd708153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd706a00640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7080d0040 0x7fd70814e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 -- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7081550b0 msgr2=0x7fd708153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7081550b0 0x7fd708153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 -- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7080d0040 msgr2=0x7fd70814e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7080d0040 0x7fd70814e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 -- 172.21.15.45:0/1202790424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70814e880 con 0x7fd70815b740 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd707400640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 0x7fd7080cef00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c760 tx=0x7fd70000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700003b80 con 0x7fd70815b740 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700003d20 con 0x7fd70815b740 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000aa90 con 0x7fd70815b740 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd708158aa0 con 0x7fd70815b740 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.478+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708159060 con 0x7fd70815b740 2024-04-25T10:27:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.482+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70804ef30 con 0x7fd70815b740 2024-04-25T10:27:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.482+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd70000ac30 con 0x7fd70815b740 2024-04-25T10:27:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.482+0000 7fd6ff400640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6e4066f90 0x7fd6e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.482+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd700084f10 con 0x7fd70815b740 2024-04-25T10:27:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.482+0000 7fd706a00640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6e4066f90 0x7fd6e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.482+0000 7fd706a00640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6e4066f90 0x7fd6e4069450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80011f0 tx=0x7fd6f8007d40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.486+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7000557a0 con 0x7fd70815b740 2024-04-25T10:27:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd70815b290 con 0x7fd70815b740 2024-04-25T10:27:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd6ff400640 1 -- 172.21.15.45:0/1202790424 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd700059840 con 0x7fd70815b740 2024-04-25T10:27:29.685 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:29.685 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6e4066f90 msgr2=0x7fd6e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6e4066f90 0x7fd6e4069450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80011f0 tx=0x7fd6f8007d40 comp rx=0 tx=0).stop 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 msgr2=0x7fd7080cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 0x7fd7080cef00 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c760 tx=0x7fd70000cc20 comp rx=0 tx=0).stop 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 shutdown_connections 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6e4066f90 0x7fd6e4069450 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd70815b740 0x7fd7080cef00 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7081550b0 0x7fd708153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 --2- 172.21.15.45:0/1202790424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7080d0040 0x7fd70814e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 >> 172.21.15.45:0/1202790424 conn(0x7fd708066b70 msgr2=0x7fd7080b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 shutdown_connections 2024-04-25T10:27:29.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:29.682+0000 7fd70d800640 1 -- 172.21.15.45:0/1202790424 wait complete. 2024-04-25T10:27:29.703 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:29.707 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:29.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:29.709 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.796 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192416 s, 54.5 MB/s 2024-04-25T10:27:29.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207712 s, 50.5 MB/s 2024-04-25T10:27:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152561 s, 68.7 MB/s 2024-04-25T10:27:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174191 s, 60.2 MB/s 2024-04-25T10:27:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156809 s, 66.9 MB/s 2024-04-25T10:27:29.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146856 s, 71.4 MB/s 2024-04-25T10:27:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145586 s, 72.0 MB/s 2024-04-25T10:27:29.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160236 s, 65.4 MB/s 2024-04-25T10:27:29.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219448 s, 47.8 MB/s 2024-04-25T10:27:29.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.966 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167567 s, 62.6 MB/s 2024-04-25T10:27:29.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.981 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135523 s, 77.4 MB/s 2024-04-25T10:27:29.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:29.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:29.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159555 s, 65.7 MB/s 2024-04-25T10:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013699 s, 76.5 MB/s 2024-04-25T10:27:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165821 s, 63.2 MB/s 2024-04-25T10:27:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155313 s, 67.5 MB/s 2024-04-25T10:27:30.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.068 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159277 s, 65.8 MB/s 2024-04-25T10:27:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185014 s, 56.7 MB/s 2024-04-25T10:27:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148526 s, 70.6 MB/s 2024-04-25T10:27:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152959 s, 68.6 MB/s 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149225 s, 70.3 MB/s 2024-04-25T10:27:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014764 s, 71.0 MB/s 2024-04-25T10:27:30.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154318 s, 67.9 MB/s 2024-04-25T10:27:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168519 s, 62.2 MB/s 2024-04-25T10:27:30.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.210 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169381 s, 61.9 MB/s 2024-04-25T10:27:30.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.231 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184386 s, 56.9 MB/s 2024-04-25T10:27:30.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.250 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179811 s, 58.3 MB/s 2024-04-25T10:27:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.267 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143497 s, 73.1 MB/s 2024-04-25T10:27:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.282 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137644 s, 76.2 MB/s 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127867 s, 82.0 MB/s 2024-04-25T10:27:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146826 s, 71.4 MB/s 2024-04-25T10:27:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145859 s, 71.9 MB/s 2024-04-25T10:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129508 s, 81.0 MB/s 2024-04-25T10:27:30.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:30.346 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147417 s, 71.1 MB/s 2024-04-25T10:27:30.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127587 s, 82.2 MB/s 2024-04-25T10:27:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207308 s, 50.6 MB/s 2024-04-25T10:27:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139411 s, 75.2 MB/s 2024-04-25T10:27:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158367 s, 66.2 MB/s 2024-04-25T10:27:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136561 s, 76.8 MB/s 2024-04-25T10:27:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.519 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015182 s, 69.1 MB/s 2024-04-25T10:27:30.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147771 s, 71.0 MB/s 2024-04-25T10:27:30.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.550 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133082 s, 78.8 MB/s 2024-04-25T10:27:30.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153421 s, 68.3 MB/s 2024-04-25T10:27:30.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143804 s, 72.9 MB/s 2024-04-25T10:27:30.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.604 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188073 s, 55.8 MB/s 2024-04-25T10:27:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148078 s, 70.8 MB/s 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134224 s, 78.1 MB/s 2024-04-25T10:27:30.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144696 s, 72.5 MB/s 2024-04-25T10:27:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140127 s, 74.8 MB/s 2024-04-25T10:27:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140681 s, 74.5 MB/s 2024-04-25T10:27:30.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159593 s, 65.7 MB/s 2024-04-25T10:27:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.719 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151527 s, 69.2 MB/s 2024-04-25T10:27:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015839 s, 66.2 MB/s 2024-04-25T10:27:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.754 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162665 s, 64.5 MB/s 2024-04-25T10:27:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159781 s, 65.6 MB/s 2024-04-25T10:27:30.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131658 s, 79.6 MB/s 2024-04-25T10:27:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141654 s, 74.0 MB/s 2024-04-25T10:27:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153321 s, 68.4 MB/s 2024-04-25T10:27:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147197 s, 71.2 MB/s 2024-04-25T10:27:30.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.854 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150488 s, 69.7 MB/s 2024-04-25T10:27:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186115 s, 56.3 MB/s 2024-04-25T10:27:30.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135125 s, 77.6 MB/s 2024-04-25T10:27:30.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.906 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015082 s, 69.5 MB/s 2024-04-25T10:27:30.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153419 s, 68.3 MB/s 2024-04-25T10:27:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:30.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147194 s, 71.2 MB/s 2024-04-25T10:27:30.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:30.941 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:30.941 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:30.941 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:27:30.941 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:30.941 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:30.941 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:30.942 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:31.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156784 s, 66.9 MB/s 2024-04-25T10:27:31.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145313 s, 72.2 MB/s 2024-04-25T10:27:31.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133849 s, 78.3 MB/s 2024-04-25T10:27:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.063 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018082 s, 58.0 MB/s 2024-04-25T10:27:31.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.080 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.080 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150056 s, 69.9 MB/s 2024-04-25T10:27:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156006 s, 67.2 MB/s 2024-04-25T10:27:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173422 s, 60.5 MB/s 2024-04-25T10:27:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148884 s, 70.4 MB/s 2024-04-25T10:27:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142871 s, 73.4 MB/s 2024-04-25T10:27:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153856 s, 68.2 MB/s 2024-04-25T10:27:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.183 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141734 s, 74.0 MB/s 2024-04-25T10:27:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.203 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185783 s, 56.4 MB/s 2024-04-25T10:27:31.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145806 s, 71.9 MB/s 2024-04-25T10:27:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.236 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145071 s, 72.3 MB/s 2024-04-25T10:27:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137653 s, 76.2 MB/s 2024-04-25T10:27:31.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.267 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013928 s, 75.3 MB/s 2024-04-25T10:27:31.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.283 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144416 s, 72.6 MB/s 2024-04-25T10:27:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177775 s, 59.0 MB/s 2024-04-25T10:27:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.317 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129496 s, 81.0 MB/s 2024-04-25T10:27:31.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.334 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014507 s, 72.3 MB/s 2024-04-25T10:27:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152218 s, 68.9 MB/s 2024-04-25T10:27:31.370 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.370 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.370 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177776 s, 59.0 MB/s 2024-04-25T10:27:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.388 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015488 s, 67.7 MB/s 2024-04-25T10:27:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.405 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148947 s, 70.4 MB/s 2024-04-25T10:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133385 s, 78.6 MB/s 2024-04-25T10:27:31.436 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.437 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.437 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152915 s, 68.6 MB/s 2024-04-25T10:27:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.451 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129268 s, 81.1 MB/s 2024-04-25T10:27:31.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137864 s, 76.1 MB/s 2024-04-25T10:27:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.484 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153025 s, 68.5 MB/s 2024-04-25T10:27:31.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.501 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153024 s, 68.5 MB/s 2024-04-25T10:27:31.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184217 s, 56.9 MB/s 2024-04-25T10:27:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145321 s, 72.2 MB/s 2024-04-25T10:27:31.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:31.539 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:31.539 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:31.539 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:27:31.539 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:31.539 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:31.539 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:31.540 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191142 s, 54.9 MB/s 2024-04-25T10:27:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.631 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133792 s, 78.4 MB/s 2024-04-25T10:27:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152068 s, 69.0 MB/s 2024-04-25T10:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019497 s, 53.8 MB/s 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149013 s, 70.4 MB/s 2024-04-25T10:27:31.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144655 s, 72.5 MB/s 2024-04-25T10:27:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137697 s, 76.2 MB/s 2024-04-25T10:27:31.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.733 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133333 s, 78.6 MB/s 2024-04-25T10:27:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140494 s, 74.6 MB/s 2024-04-25T10:27:31.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.765 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142528 s, 73.6 MB/s 2024-04-25T10:27:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.781 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143977 s, 72.8 MB/s 2024-04-25T10:27:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153951 s, 68.1 MB/s 2024-04-25T10:27:31.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.815 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146633 s, 71.5 MB/s 2024-04-25T10:27:31.829 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.830 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128931 s, 81.3 MB/s 2024-04-25T10:27:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014672 s, 71.5 MB/s 2024-04-25T10:27:31.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.861 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133558 s, 78.5 MB/s 2024-04-25T10:27:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.877 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142378 s, 73.6 MB/s 2024-04-25T10:27:31.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.893 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135386 s, 77.5 MB/s 2024-04-25T10:27:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.910 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155462 s, 67.4 MB/s 2024-04-25T10:27:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191111 s, 54.9 MB/s 2024-04-25T10:27:31.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141225 s, 74.2 MB/s 2024-04-25T10:27:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.965 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155359 s, 67.5 MB/s 2024-04-25T10:27:31.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.981 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142282 s, 73.7 MB/s 2024-04-25T10:27:31.993 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:31.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:31.994 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114454 s, 91.6 MB/s 2024-04-25T10:27:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.009 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135142 s, 77.6 MB/s 2024-04-25T10:27:32.024 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.024 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.024 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013527 s, 77.5 MB/s 2024-04-25T10:27:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182342 s, 57.5 MB/s 2024-04-25T10:27:32.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013193 s, 79.5 MB/s 2024-04-25T10:27:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146874 s, 71.4 MB/s 2024-04-25T10:27:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136646 s, 76.7 MB/s 2024-04-25T10:27:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283948 s, 36.9 MB/s 2024-04-25T10:27:32.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137229 s, 76.4 MB/s 2024-04-25T10:27:32.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:32.139 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:32.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012998 s, 80.7 MB/s 2024-04-25T10:27:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148724 s, 70.5 MB/s 2024-04-25T10:27:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.239 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145572 s, 72.0 MB/s 2024-04-25T10:27:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.254 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137938 s, 76.0 MB/s 2024-04-25T10:27:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.270 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138054 s, 76.0 MB/s 2024-04-25T10:27:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014331 s, 73.2 MB/s 2024-04-25T10:27:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111274 s, 94.2 MB/s 2024-04-25T10:27:32.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01473 s, 71.2 MB/s 2024-04-25T10:27:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189257 s, 55.4 MB/s 2024-04-25T10:27:32.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145306 s, 72.2 MB/s 2024-04-25T10:27:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171698 s, 61.1 MB/s 2024-04-25T10:27:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130765 s, 80.2 MB/s 2024-04-25T10:27:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.399 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011639 s, 90.1 MB/s 2024-04-25T10:27:32.415 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.416 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143211 s, 73.2 MB/s 2024-04-25T10:27:32.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157866 s, 66.4 MB/s 2024-04-25T10:27:32.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.453 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179355 s, 58.5 MB/s 2024-04-25T10:27:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.470 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152755 s, 68.6 MB/s 2024-04-25T10:27:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.488 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158281 s, 66.2 MB/s 2024-04-25T10:27:32.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179583 s, 58.4 MB/s 2024-04-25T10:27:32.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133357 s, 78.6 MB/s 2024-04-25T10:27:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149544 s, 70.1 MB/s 2024-04-25T10:27:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149578 s, 70.1 MB/s 2024-04-25T10:27:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176238 s, 59.5 MB/s 2024-04-25T10:27:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013536 s, 77.5 MB/s 2024-04-25T10:27:32.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150762 s, 69.6 MB/s 2024-04-25T10:27:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158888 s, 66.0 MB/s 2024-04-25T10:27:32.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.646 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187859 s, 55.8 MB/s 2024-04-25T10:27:32.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.662 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146386 s, 71.6 MB/s 2024-04-25T10:27:32.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.679 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015145 s, 69.2 MB/s 2024-04-25T10:27:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180393 s, 58.1 MB/s 2024-04-25T10:27:32.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.716 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152892 s, 68.6 MB/s 2024-04-25T10:27:32.736 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.737 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185846 s, 56.4 MB/s 2024-04-25T10:27:32.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:32.738 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.815 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187168 s, 56.0 MB/s 2024-04-25T10:27:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.832 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148223 s, 70.7 MB/s 2024-04-25T10:27:32.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.853 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191276 s, 54.8 MB/s 2024-04-25T10:27:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172568 s, 60.8 MB/s 2024-04-25T10:27:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.885 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114043 s, 91.9 MB/s 2024-04-25T10:27:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146537 s, 71.6 MB/s 2024-04-25T10:27:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134325 s, 78.1 MB/s 2024-04-25T10:27:32.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155642 s, 67.4 MB/s 2024-04-25T10:27:32.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.949 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.949 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136311 s, 76.9 MB/s 2024-04-25T10:27:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190747 s, 55.0 MB/s 2024-04-25T10:27:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149308 s, 70.2 MB/s 2024-04-25T10:27:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.007 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187291 s, 56.0 MB/s 2024-04-25T10:27:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.023 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139821 s, 75.0 MB/s 2024-04-25T10:27:33.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.039 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014045 s, 74.7 MB/s 2024-04-25T10:27:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182797 s, 57.4 MB/s 2024-04-25T10:27:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143107 s, 73.3 MB/s 2024-04-25T10:27:33.093 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.094 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158472 s, 66.2 MB/s 2024-04-25T10:27:33.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132315 s, 79.2 MB/s 2024-04-25T10:27:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144149 s, 72.7 MB/s 2024-04-25T10:27:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.141 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148338 s, 70.7 MB/s 2024-04-25T10:27:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186828 s, 56.1 MB/s 2024-04-25T10:27:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013973 s, 75.0 MB/s 2024-04-25T10:27:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141057 s, 74.3 MB/s 2024-04-25T10:27:33.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.210 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150734 s, 69.6 MB/s 2024-04-25T10:27:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132699 s, 79.0 MB/s 2024-04-25T10:27:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140631 s, 74.6 MB/s 2024-04-25T10:27:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177187 s, 59.2 MB/s 2024-04-25T10:27:33.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.279 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146052 s, 71.8 MB/s 2024-04-25T10:27:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145179 s, 72.2 MB/s 2024-04-25T10:27:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182281 s, 57.5 MB/s 2024-04-25T10:27:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147323 s, 71.2 MB/s 2024-04-25T10:27:33.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.345 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137612 s, 76.2 MB/s 2024-04-25T10:27:33.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:33.347 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:33.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.420 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189577 s, 55.3 MB/s 2024-04-25T10:27:33.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.435 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142814 s, 73.4 MB/s 2024-04-25T10:27:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.451 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131996 s, 79.4 MB/s 2024-04-25T10:27:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140878 s, 74.4 MB/s 2024-04-25T10:27:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.483 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142634 s, 73.5 MB/s 2024-04-25T10:27:33.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129781 s, 80.8 MB/s 2024-04-25T10:27:33.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.519 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191798 s, 54.7 MB/s 2024-04-25T10:27:33.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136233 s, 77.0 MB/s 2024-04-25T10:27:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132994 s, 78.8 MB/s 2024-04-25T10:27:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.565 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014309 s, 73.3 MB/s 2024-04-25T10:27:33.580 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.580 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.581 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136463 s, 76.8 MB/s 2024-04-25T10:27:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.602 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195267 s, 53.7 MB/s 2024-04-25T10:27:33.617 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137677 s, 76.2 MB/s 2024-04-25T10:27:33.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.632 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127145 s, 82.5 MB/s 2024-04-25T10:27:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131061 s, 80.0 MB/s 2024-04-25T10:27:33.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143556 s, 73.0 MB/s 2024-04-25T10:27:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148642 s, 70.5 MB/s 2024-04-25T10:27:33.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.694 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121913 s, 86.0 MB/s 2024-04-25T10:27:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130028 s, 80.6 MB/s 2024-04-25T10:27:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141164 s, 74.3 MB/s 2024-04-25T10:27:33.740 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.740 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.740 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137069 s, 76.5 MB/s 2024-04-25T10:27:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137801 s, 76.1 MB/s 2024-04-25T10:27:33.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139547 s, 75.1 MB/s 2024-04-25T10:27:33.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132499 s, 79.1 MB/s 2024-04-25T10:27:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132725 s, 79.0 MB/s 2024-04-25T10:27:33.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.817 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130578 s, 80.3 MB/s 2024-04-25T10:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132493 s, 79.1 MB/s 2024-04-25T10:27:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128519 s, 81.6 MB/s 2024-04-25T10:27:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127519 s, 82.2 MB/s 2024-04-25T10:27:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149273 s, 70.2 MB/s 2024-04-25T10:27:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144307 s, 72.7 MB/s 2024-04-25T10:27:33.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.910 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137067 s, 76.5 MB/s 2024-04-25T10:27:33.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:33.911 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:33.911 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:33.912 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:27:33.912 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:33.912 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:33.912 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:33.912 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:33.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138615 s, 75.6 MB/s 2024-04-25T10:27:34.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.000 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152639 s, 68.7 MB/s 2024-04-25T10:27:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149277 s, 70.2 MB/s 2024-04-25T10:27:34.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.032 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013737 s, 76.3 MB/s 2024-04-25T10:27:34.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.048 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139868 s, 75.0 MB/s 2024-04-25T10:27:34.067 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183155 s, 57.3 MB/s 2024-04-25T10:27:34.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.083 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136069 s, 77.1 MB/s 2024-04-25T10:27:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013938 s, 75.2 MB/s 2024-04-25T10:27:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013868 s, 75.6 MB/s 2024-04-25T10:27:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.129 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01273 s, 82.4 MB/s 2024-04-25T10:27:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.144 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132235 s, 79.3 MB/s 2024-04-25T10:27:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131998 s, 79.4 MB/s 2024-04-25T10:27:34.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150291 s, 69.8 MB/s 2024-04-25T10:27:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.191 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135897 s, 77.2 MB/s 2024-04-25T10:27:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.212 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184483 s, 56.8 MB/s 2024-04-25T10:27:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144255 s, 72.7 MB/s 2024-04-25T10:27:34.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.248 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184607 s, 56.8 MB/s 2024-04-25T10:27:34.264 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.264 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.264 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144189 s, 72.7 MB/s 2024-04-25T10:27:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.279 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136853 s, 76.6 MB/s 2024-04-25T10:27:34.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.298 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156442 s, 67.0 MB/s 2024-04-25T10:27:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143038 s, 73.3 MB/s 2024-04-25T10:27:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142071 s, 73.8 MB/s 2024-04-25T10:27:34.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151128 s, 69.4 MB/s 2024-04-25T10:27:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.360 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125688 s, 83.4 MB/s 2024-04-25T10:27:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.376 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140969 s, 74.4 MB/s 2024-04-25T10:27:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179788 s, 58.3 MB/s 2024-04-25T10:27:34.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.411 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.411 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133165 s, 78.7 MB/s 2024-04-25T10:27:34.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178456 s, 58.8 MB/s 2024-04-25T10:27:34.446 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.446 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.446 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137103 s, 76.5 MB/s 2024-04-25T10:27:34.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.461 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133844 s, 78.3 MB/s 2024-04-25T10:27:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139631 s, 75.1 MB/s 2024-04-25T10:27:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.497 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182157 s, 57.6 MB/s 2024-04-25T10:27:34.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:34.498 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:34.498 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:34.498 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:27:34.499 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:34.499 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:34.499 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:34.499 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.569 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156179 s, 67.1 MB/s 2024-04-25T10:27:34.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.589 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181441 s, 57.8 MB/s 2024-04-25T10:27:34.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151478 s, 69.2 MB/s 2024-04-25T10:27:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.626 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185278 s, 56.6 MB/s 2024-04-25T10:27:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.642 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140006 s, 74.9 MB/s 2024-04-25T10:27:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.658 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140615 s, 74.6 MB/s 2024-04-25T10:27:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.674 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014786 s, 70.9 MB/s 2024-04-25T10:27:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.692 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154469 s, 67.9 MB/s 2024-04-25T10:27:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181689 s, 57.7 MB/s 2024-04-25T10:27:34.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153944 s, 68.1 MB/s 2024-04-25T10:27:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149957 s, 69.9 MB/s 2024-04-25T10:27:34.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.762 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146785 s, 71.4 MB/s 2024-04-25T10:27:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013757 s, 76.2 MB/s 2024-04-25T10:27:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.798 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018466 s, 56.8 MB/s 2024-04-25T10:27:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.815 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147318 s, 71.2 MB/s 2024-04-25T10:27:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137923 s, 76.0 MB/s 2024-04-25T10:27:34.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137289 s, 76.4 MB/s 2024-04-25T10:27:34.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.863 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015164 s, 69.1 MB/s 2024-04-25T10:27:34.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.880 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.880 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152552 s, 68.7 MB/s 2024-04-25T10:27:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145837 s, 71.9 MB/s 2024-04-25T10:27:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190307 s, 55.1 MB/s 2024-04-25T10:27:34.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.933 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014909 s, 70.3 MB/s 2024-04-25T10:27:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147294 s, 71.2 MB/s 2024-04-25T10:27:34.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.967 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158008 s, 66.4 MB/s 2024-04-25T10:27:34.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:34.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:34.988 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190354 s, 55.1 MB/s 2024-04-25T10:27:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140918 s, 74.4 MB/s 2024-04-25T10:27:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148431 s, 70.6 MB/s 2024-04-25T10:27:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144173 s, 72.7 MB/s 2024-04-25T10:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144721 s, 72.5 MB/s 2024-04-25T10:27:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143172 s, 73.2 MB/s 2024-04-25T10:27:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015222 s, 68.9 MB/s 2024-04-25T10:27:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140995 s, 74.4 MB/s 2024-04-25T10:27:35.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:35.104 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:35.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.179 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177059 s, 59.2 MB/s 2024-04-25T10:27:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146087 s, 71.8 MB/s 2024-04-25T10:27:35.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.215 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186121 s, 56.3 MB/s 2024-04-25T10:27:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136394 s, 76.9 MB/s 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.246 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135052 s, 77.6 MB/s 2024-04-25T10:27:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.263 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154716 s, 67.8 MB/s 2024-04-25T10:27:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133013 s, 78.8 MB/s 2024-04-25T10:27:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133354 s, 78.6 MB/s 2024-04-25T10:27:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148644 s, 70.5 MB/s 2024-04-25T10:27:35.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150061 s, 69.9 MB/s 2024-04-25T10:27:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185762 s, 56.4 MB/s 2024-04-25T10:27:35.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122602 s, 85.5 MB/s 2024-04-25T10:27:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.378 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147505 s, 71.1 MB/s 2024-04-25T10:27:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013178 s, 79.6 MB/s 2024-04-25T10:27:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185922 s, 56.4 MB/s 2024-04-25T10:27:35.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.428 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130929 s, 80.1 MB/s 2024-04-25T10:27:35.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.450 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192621 s, 54.4 MB/s 2024-04-25T10:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013973 s, 75.0 MB/s 2024-04-25T10:27:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.481 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137721 s, 76.1 MB/s 2024-04-25T10:27:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161637 s, 64.9 MB/s 2024-04-25T10:27:35.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145959 s, 71.8 MB/s 2024-04-25T10:27:35.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.531 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.531 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134117 s, 78.2 MB/s 2024-04-25T10:27:35.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.547 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146903 s, 71.4 MB/s 2024-04-25T10:27:35.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144024 s, 72.8 MB/s 2024-04-25T10:27:35.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176993 s, 59.2 MB/s 2024-04-25T10:27:35.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.596 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.596 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121115 s, 86.6 MB/s 2024-04-25T10:27:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140936 s, 74.4 MB/s 2024-04-25T10:27:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138791 s, 75.6 MB/s 2024-04-25T10:27:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191897 s, 54.6 MB/s 2024-04-25T10:27:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150367 s, 69.7 MB/s 2024-04-25T10:27:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170976 s, 61.3 MB/s 2024-04-25T10:27:35.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.701 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134285 s, 78.1 MB/s 2024-04-25T10:27:35.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:35.703 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:35.703 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:35.703 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:27:35.703 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:35.704 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:35.704 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:35.704 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013015 s, 80.6 MB/s 2024-04-25T10:27:35.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187018 s, 56.1 MB/s 2024-04-25T10:27:35.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.812 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192198 s, 54.6 MB/s 2024-04-25T10:27:35.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133437 s, 78.6 MB/s 2024-04-25T10:27:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.844 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149286 s, 70.2 MB/s 2024-04-25T10:27:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123419 s, 85.0 MB/s 2024-04-25T10:27:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140186 s, 74.8 MB/s 2024-04-25T10:27:35.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.890 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013576 s, 77.2 MB/s 2024-04-25T10:27:35.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.905 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132164 s, 79.3 MB/s 2024-04-25T10:27:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018697 s, 56.1 MB/s 2024-04-25T10:27:35.941 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.941 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.941 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144544 s, 72.5 MB/s 2024-04-25T10:27:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.963 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198746 s, 52.8 MB/s 2024-04-25T10:27:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146382 s, 71.6 MB/s 2024-04-25T10:27:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150106 s, 69.9 MB/s 2024-04-25T10:27:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187632 s, 55.9 MB/s 2024-04-25T10:27:36.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.035 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160901 s, 65.2 MB/s 2024-04-25T10:27:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017819 s, 58.8 MB/s 2024-04-25T10:27:36.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136784 s, 76.7 MB/s 2024-04-25T10:27:36.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.085 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.085 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137704 s, 76.1 MB/s 2024-04-25T10:27:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.102 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152534 s, 68.7 MB/s 2024-04-25T10:27:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162476 s, 64.5 MB/s 2024-04-25T10:27:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.141 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183616 s, 57.1 MB/s 2024-04-25T10:27:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133549 s, 78.5 MB/s 2024-04-25T10:27:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136535 s, 76.8 MB/s 2024-04-25T10:27:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136594 s, 76.8 MB/s 2024-04-25T10:27:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153276 s, 68.4 MB/s 2024-04-25T10:27:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183712 s, 57.1 MB/s 2024-04-25T10:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150932 s, 69.5 MB/s 2024-04-25T10:27:36.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.256 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014207 s, 73.8 MB/s 2024-04-25T10:27:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187093 s, 56.0 MB/s 2024-04-25T10:27:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130912 s, 80.1 MB/s 2024-04-25T10:27:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148024 s, 70.8 MB/s 2024-04-25T10:27:36.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:36.310 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:36.310 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:36.310 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:27:36.310 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:36.310 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:36.310 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:36.311 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181266 s, 57.8 MB/s 2024-04-25T10:27:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139096 s, 75.4 MB/s 2024-04-25T10:27:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181655 s, 57.7 MB/s 2024-04-25T10:27:36.436 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.436 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.436 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147651 s, 71.0 MB/s 2024-04-25T10:27:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151605 s, 69.2 MB/s 2024-04-25T10:27:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.468 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138972 s, 75.5 MB/s 2024-04-25T10:27:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164168 s, 63.9 MB/s 2024-04-25T10:27:36.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.503 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145321 s, 72.2 MB/s 2024-04-25T10:27:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.518 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132517 s, 79.1 MB/s 2024-04-25T10:27:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136712 s, 76.7 MB/s 2024-04-25T10:27:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136262 s, 77.0 MB/s 2024-04-25T10:27:36.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186093 s, 56.3 MB/s 2024-04-25T10:27:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138031 s, 76.0 MB/s 2024-04-25T10:27:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179808 s, 58.3 MB/s 2024-04-25T10:27:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137235 s, 76.4 MB/s 2024-04-25T10:27:36.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175944 s, 59.6 MB/s 2024-04-25T10:27:36.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.660 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183854 s, 57.0 MB/s 2024-04-25T10:27:36.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150738 s, 69.6 MB/s 2024-04-25T10:27:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.692 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135825 s, 77.2 MB/s 2024-04-25T10:27:36.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149188 s, 70.3 MB/s 2024-04-25T10:27:36.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173787 s, 60.3 MB/s 2024-04-25T10:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.742 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129483 s, 81.0 MB/s 2024-04-25T10:27:36.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134802 s, 77.8 MB/s 2024-04-25T10:27:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.774 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141338 s, 74.2 MB/s 2024-04-25T10:27:36.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.789 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129428 s, 81.0 MB/s 2024-04-25T10:27:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137531 s, 76.2 MB/s 2024-04-25T10:27:36.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.824 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180024 s, 58.2 MB/s 2024-04-25T10:27:36.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.839 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134288 s, 78.1 MB/s 2024-04-25T10:27:36.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.855 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141195 s, 74.3 MB/s 2024-04-25T10:27:36.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148377 s, 70.7 MB/s 2024-04-25T10:27:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182381 s, 57.5 MB/s 2024-04-25T10:27:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139236 s, 75.3 MB/s 2024-04-25T10:27:36.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:36.909 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.984 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:36.984 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018394 s, 57.0 MB/s 2024-04-25T10:27:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.000 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143473 s, 73.1 MB/s 2024-04-25T10:27:37.017 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162311 s, 64.6 MB/s 2024-04-25T10:27:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146372 s, 71.6 MB/s 2024-04-25T10:27:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177169 s, 59.2 MB/s 2024-04-25T10:27:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.070 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145131 s, 72.3 MB/s 2024-04-25T10:27:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165742 s, 63.3 MB/s 2024-04-25T10:27:37.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.108 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177221 s, 59.2 MB/s 2024-04-25T10:27:37.128 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.128 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.128 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180551 s, 58.1 MB/s 2024-04-25T10:27:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.145 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145401 s, 72.1 MB/s 2024-04-25T10:27:37.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.161 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143925 s, 72.9 MB/s 2024-04-25T10:27:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.176 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130762 s, 80.2 MB/s 2024-04-25T10:27:37.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.197 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186387 s, 56.3 MB/s 2024-04-25T10:27:37.211 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.212 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132286 s, 79.3 MB/s 2024-04-25T10:27:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135028 s, 77.7 MB/s 2024-04-25T10:27:37.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131818 s, 79.5 MB/s 2024-04-25T10:27:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142584 s, 73.5 MB/s 2024-04-25T10:27:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181561 s, 57.8 MB/s 2024-04-25T10:27:37.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.294 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147818 s, 70.9 MB/s 2024-04-25T10:27:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131204 s, 79.9 MB/s 2024-04-25T10:27:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.324 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01271 s, 82.5 MB/s 2024-04-25T10:27:37.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.340 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.340 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143217 s, 73.2 MB/s 2024-04-25T10:27:37.355 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.355 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.355 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134812 s, 77.8 MB/s 2024-04-25T10:27:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187067 s, 56.1 MB/s 2024-04-25T10:27:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147597 s, 71.0 MB/s 2024-04-25T10:27:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136262 s, 77.0 MB/s 2024-04-25T10:27:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137552 s, 76.2 MB/s 2024-04-25T10:27:37.439 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.440 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.440 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145287 s, 72.2 MB/s 2024-04-25T10:27:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.456 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147599 s, 71.0 MB/s 2024-04-25T10:27:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013782 s, 76.1 MB/s 2024-04-25T10:27:37.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140601 s, 74.6 MB/s 2024-04-25T10:27:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.503 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013786 s, 76.1 MB/s 2024-04-25T10:27:37.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:37.504 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:37.504 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:37.504 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:27:37.504 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:37.504 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:37.505 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:37.505 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144934 s, 72.3 MB/s 2024-04-25T10:27:37.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.594 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160036 s, 65.5 MB/s 2024-04-25T10:27:37.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.609 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135516 s, 77.4 MB/s 2024-04-25T10:27:37.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014716 s, 71.3 MB/s 2024-04-25T10:27:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187605 s, 55.9 MB/s 2024-04-25T10:27:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146957 s, 71.4 MB/s 2024-04-25T10:27:37.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182732 s, 57.4 MB/s 2024-04-25T10:27:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149516 s, 70.1 MB/s 2024-04-25T10:27:37.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.715 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137558 s, 76.2 MB/s 2024-04-25T10:27:37.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.730 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136226 s, 77.0 MB/s 2024-04-25T10:27:37.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152378 s, 68.8 MB/s 2024-04-25T10:27:37.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135517 s, 77.4 MB/s 2024-04-25T10:27:37.781 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.782 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149769 s, 70.0 MB/s 2024-04-25T10:27:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.800 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187922 s, 55.8 MB/s 2024-04-25T10:27:37.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.816 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013817 s, 75.9 MB/s 2024-04-25T10:27:37.830 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133564 s, 78.5 MB/s 2024-04-25T10:27:37.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.846 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131429 s, 79.8 MB/s 2024-04-25T10:27:37.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.862 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147843 s, 70.9 MB/s 2024-04-25T10:27:37.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.877 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133362 s, 78.6 MB/s 2024-04-25T10:27:37.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222915 s, 47.0 MB/s 2024-04-25T10:27:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201797 s, 52.0 MB/s 2024-04-25T10:27:37.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.945 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197366 s, 53.1 MB/s 2024-04-25T10:27:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.962 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147672 s, 71.0 MB/s 2024-04-25T10:27:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015226 s, 68.9 MB/s 2024-04-25T10:27:37.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:37.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:37.995 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146649 s, 71.5 MB/s 2024-04-25T10:27:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138266 s, 75.8 MB/s 2024-04-25T10:27:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148772 s, 70.5 MB/s 2024-04-25T10:27:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146598 s, 71.5 MB/s 2024-04-25T10:27:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134736 s, 77.8 MB/s 2024-04-25T10:27:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015053 s, 69.7 MB/s 2024-04-25T10:27:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.096 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183554 s, 57.1 MB/s 2024-04-25T10:27:38.112 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.113 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.113 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146883 s, 71.4 MB/s 2024-04-25T10:27:38.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:38.115 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.183 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136623 s, 76.7 MB/s 2024-04-25T10:27:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136259 s, 77.0 MB/s 2024-04-25T10:27:38.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129777 s, 80.8 MB/s 2024-04-25T10:27:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143037 s, 73.3 MB/s 2024-04-25T10:27:38.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.249 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018135 s, 57.8 MB/s 2024-04-25T10:27:38.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.265 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148634 s, 70.5 MB/s 2024-04-25T10:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.282 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152242 s, 68.9 MB/s 2024-04-25T10:27:38.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.298 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141244 s, 74.2 MB/s 2024-04-25T10:27:38.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179003 s, 58.6 MB/s 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.334 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144189 s, 72.7 MB/s 2024-04-25T10:27:38.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.349 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.349 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137102 s, 76.5 MB/s 2024-04-25T10:27:38.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.365 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.365 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145132 s, 72.2 MB/s 2024-04-25T10:27:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.381 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139623 s, 75.1 MB/s 2024-04-25T10:27:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.402 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189449 s, 55.3 MB/s 2024-04-25T10:27:38.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.418 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140655 s, 74.5 MB/s 2024-04-25T10:27:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137373 s, 76.3 MB/s 2024-04-25T10:27:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132205 s, 79.3 MB/s 2024-04-25T10:27:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153152 s, 68.5 MB/s 2024-04-25T10:27:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.480 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132678 s, 79.0 MB/s 2024-04-25T10:27:38.496 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.497 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144346 s, 72.6 MB/s 2024-04-25T10:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.512 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136316 s, 76.9 MB/s 2024-04-25T10:27:38.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.528 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140573 s, 74.6 MB/s 2024-04-25T10:27:38.548 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.548 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.548 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183522 s, 57.1 MB/s 2024-04-25T10:27:38.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132424 s, 79.2 MB/s 2024-04-25T10:27:38.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147438 s, 71.1 MB/s 2024-04-25T10:27:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.597 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140718 s, 74.5 MB/s 2024-04-25T10:27:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.610 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135144 s, 77.6 MB/s 2024-04-25T10:27:38.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.626 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138533 s, 75.7 MB/s 2024-04-25T10:27:38.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148075 s, 70.8 MB/s 2024-04-25T10:27:38.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.663 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.663 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181882 s, 57.7 MB/s 2024-04-25T10:27:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.679 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144564 s, 72.5 MB/s 2024-04-25T10:27:38.695 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.695 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.696 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146432 s, 71.6 MB/s 2024-04-25T10:27:38.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:38.697 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.768 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149218 s, 70.3 MB/s 2024-04-25T10:27:38.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.783 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136493 s, 76.8 MB/s 2024-04-25T10:27:38.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.798 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012643 s, 82.9 MB/s 2024-04-25T10:27:38.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014778 s, 71.0 MB/s 2024-04-25T10:27:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.828 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121986 s, 86.0 MB/s 2024-04-25T10:27:38.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.845 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145548 s, 72.0 MB/s 2024-04-25T10:27:38.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.859 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130042 s, 80.6 MB/s 2024-04-25T10:27:38.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.876 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.877 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015178 s, 69.1 MB/s 2024-04-25T10:27:38.891 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.892 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.892 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134275 s, 78.1 MB/s 2024-04-25T10:27:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135466 s, 77.4 MB/s 2024-04-25T10:27:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138774 s, 75.6 MB/s 2024-04-25T10:27:38.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.939 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149795 s, 70.0 MB/s 2024-04-25T10:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133013 s, 78.8 MB/s 2024-04-25T10:27:38.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131704 s, 79.6 MB/s 2024-04-25T10:27:38.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:38.990 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179513 s, 58.4 MB/s 2024-04-25T10:27:39.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.006 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149968 s, 69.9 MB/s 2024-04-25T10:27:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.024 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148998 s, 70.4 MB/s 2024-04-25T10:27:39.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.039 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014071 s, 74.5 MB/s 2024-04-25T10:27:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149945 s, 69.9 MB/s 2024-04-25T10:27:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.071 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130004 s, 80.7 MB/s 2024-04-25T10:27:39.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017848 s, 58.8 MB/s 2024-04-25T10:27:39.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.106 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133657 s, 78.5 MB/s 2024-04-25T10:27:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.123 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154067 s, 68.1 MB/s 2024-04-25T10:27:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139031 s, 75.4 MB/s 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176934 s, 59.3 MB/s 2024-04-25T10:27:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152447 s, 68.8 MB/s 2024-04-25T10:27:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.192 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151669 s, 69.1 MB/s 2024-04-25T10:27:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.208 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139388 s, 75.2 MB/s 2024-04-25T10:27:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154288 s, 68.0 MB/s 2024-04-25T10:27:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146493 s, 71.6 MB/s 2024-04-25T10:27:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151605 s, 69.2 MB/s 2024-04-25T10:27:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.273 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129323 s, 81.1 MB/s 2024-04-25T10:27:39.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:39.275 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137375 s, 76.3 MB/s 2024-04-25T10:27:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.367 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148626 s, 70.6 MB/s 2024-04-25T10:27:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131822 s, 79.5 MB/s 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141472 s, 74.1 MB/s 2024-04-25T10:27:39.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137271 s, 76.4 MB/s 2024-04-25T10:27:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152604 s, 68.7 MB/s 2024-04-25T10:27:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.447 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151255 s, 69.3 MB/s 2024-04-25T10:27:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143157 s, 73.2 MB/s 2024-04-25T10:27:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.480 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148026 s, 70.8 MB/s 2024-04-25T10:27:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.496 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014452 s, 72.6 MB/s 2024-04-25T10:27:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.513 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147048 s, 71.3 MB/s 2024-04-25T10:27:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134574 s, 77.9 MB/s 2024-04-25T10:27:39.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135364 s, 77.5 MB/s 2024-04-25T10:27:39.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.559 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138761 s, 75.6 MB/s 2024-04-25T10:27:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149562 s, 70.1 MB/s 2024-04-25T10:27:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129713 s, 80.8 MB/s 2024-04-25T10:27:39.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148113 s, 70.8 MB/s 2024-04-25T10:27:39.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.623 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146822 s, 71.4 MB/s 2024-04-25T10:27:39.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131848 s, 79.5 MB/s 2024-04-25T10:27:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134744 s, 77.8 MB/s 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148163 s, 70.8 MB/s 2024-04-25T10:27:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.686 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140469 s, 74.6 MB/s 2024-04-25T10:27:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149319 s, 70.2 MB/s 2024-04-25T10:27:39.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.719 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138516 s, 75.7 MB/s 2024-04-25T10:27:39.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.735 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150084 s, 69.9 MB/s 2024-04-25T10:27:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146264 s, 71.7 MB/s 2024-04-25T10:27:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146553 s, 71.5 MB/s 2024-04-25T10:27:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.785 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014724 s, 71.2 MB/s 2024-04-25T10:27:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153195 s, 68.4 MB/s 2024-04-25T10:27:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149025 s, 70.4 MB/s 2024-04-25T10:27:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145946 s, 71.8 MB/s 2024-04-25T10:27:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013552 s, 77.4 MB/s 2024-04-25T10:27:39.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:39.852 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:39.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.929 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146197 s, 71.7 MB/s 2024-04-25T10:27:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.944 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138497 s, 75.7 MB/s 2024-04-25T10:27:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.961 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154568 s, 67.8 MB/s 2024-04-25T10:27:39.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.982 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191049 s, 54.9 MB/s 2024-04-25T10:27:39.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:39.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:39.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134926 s, 77.7 MB/s 2024-04-25T10:27:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165026 s, 63.5 MB/s 2024-04-25T10:27:40.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190141 s, 55.1 MB/s 2024-04-25T10:27:40.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133872 s, 78.3 MB/s 2024-04-25T10:27:40.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154309 s, 68.0 MB/s 2024-04-25T10:27:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148277 s, 70.7 MB/s 2024-04-25T10:27:40.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.106 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183055 s, 57.3 MB/s 2024-04-25T10:27:40.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.123 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152665 s, 68.7 MB/s 2024-04-25T10:27:40.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132142 s, 79.4 MB/s 2024-04-25T10:27:40.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013777 s, 76.1 MB/s 2024-04-25T10:27:40.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.170 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139819 s, 75.0 MB/s 2024-04-25T10:27:40.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.185 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138136 s, 75.9 MB/s 2024-04-25T10:27:40.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136516 s, 76.8 MB/s 2024-04-25T10:27:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.218 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151254 s, 69.3 MB/s 2024-04-25T10:27:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151713 s, 69.1 MB/s 2024-04-25T10:27:40.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.249 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.249 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131439 s, 79.8 MB/s 2024-04-25T10:27:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.271 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195442 s, 53.7 MB/s 2024-04-25T10:27:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.288 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148401 s, 70.7 MB/s 2024-04-25T10:27:40.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188988 s, 55.5 MB/s 2024-04-25T10:27:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.324 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144161 s, 72.7 MB/s 2024-04-25T10:27:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178313 s, 58.8 MB/s 2024-04-25T10:27:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014679 s, 71.4 MB/s 2024-04-25T10:27:40.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.377 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148713 s, 70.5 MB/s 2024-04-25T10:27:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142565 s, 73.6 MB/s 2024-04-25T10:27:40.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187959 s, 55.8 MB/s 2024-04-25T10:27:40.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.430 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.430 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147029 s, 71.3 MB/s 2024-04-25T10:27:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.447 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150396 s, 69.7 MB/s 2024-04-25T10:27:40.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.463 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142463 s, 73.6 MB/s 2024-04-25T10:27:40.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:40.464 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:40.465 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:40.465 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:27:40.465 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:40.465 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:40.465 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:40.465 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:40.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184068 s, 57.0 MB/s 2024-04-25T10:27:40.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.560 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.560 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147232 s, 71.2 MB/s 2024-04-25T10:27:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.578 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159606 s, 65.7 MB/s 2024-04-25T10:27:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155098 s, 67.6 MB/s 2024-04-25T10:27:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151053 s, 69.4 MB/s 2024-04-25T10:27:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.628 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151004 s, 69.4 MB/s 2024-04-25T10:27:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.644 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135892 s, 77.2 MB/s 2024-04-25T10:27:40.659 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.660 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139833 s, 75.0 MB/s 2024-04-25T10:27:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.675 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135298 s, 77.5 MB/s 2024-04-25T10:27:40.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129562 s, 80.9 MB/s 2024-04-25T10:27:40.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.706 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136966 s, 76.6 MB/s 2024-04-25T10:27:40.720 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.720 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129061 s, 81.2 MB/s 2024-04-25T10:27:40.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.738 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153199 s, 68.4 MB/s 2024-04-25T10:27:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.753 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013699 s, 76.5 MB/s 2024-04-25T10:27:40.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.770 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147882 s, 70.9 MB/s 2024-04-25T10:27:40.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.786 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149094 s, 70.3 MB/s 2024-04-25T10:27:40.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145266 s, 72.2 MB/s 2024-04-25T10:27:40.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149782 s, 70.0 MB/s 2024-04-25T10:27:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144897 s, 72.4 MB/s 2024-04-25T10:27:40.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.854 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138924 s, 75.5 MB/s 2024-04-25T10:27:40.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135788 s, 77.2 MB/s 2024-04-25T10:27:40.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152145 s, 68.9 MB/s 2024-04-25T10:27:40.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.899 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013263 s, 79.1 MB/s 2024-04-25T10:27:40.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149313 s, 70.2 MB/s 2024-04-25T10:27:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.932 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149175 s, 70.3 MB/s 2024-04-25T10:27:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.947 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129798 s, 80.8 MB/s 2024-04-25T10:27:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144903 s, 72.4 MB/s 2024-04-25T10:27:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:40.980 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150674 s, 69.6 MB/s 2024-04-25T10:27:41.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.000 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.000 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017999 s, 58.3 MB/s 2024-04-25T10:27:41.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146793 s, 71.4 MB/s 2024-04-25T10:27:41.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.033 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145387 s, 72.1 MB/s 2024-04-25T10:27:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.049 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151849 s, 69.1 MB/s 2024-04-25T10:27:41.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:41.051 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131781 s, 79.6 MB/s 2024-04-25T10:27:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017955 s, 58.4 MB/s 2024-04-25T10:27:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149197 s, 70.3 MB/s 2024-04-25T10:27:41.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141184 s, 74.3 MB/s 2024-04-25T10:27:41.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.191 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144652 s, 72.5 MB/s 2024-04-25T10:27:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147704 s, 71.0 MB/s 2024-04-25T10:27:41.227 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182894 s, 57.3 MB/s 2024-04-25T10:27:41.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.244 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148984 s, 70.4 MB/s 2024-04-25T10:27:41.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149061 s, 70.3 MB/s 2024-04-25T10:27:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.276 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133913 s, 78.3 MB/s 2024-04-25T10:27:41.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168349 s, 62.3 MB/s 2024-04-25T10:27:41.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.310 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.310 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138204 s, 75.9 MB/s 2024-04-25T10:27:41.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149634 s, 70.1 MB/s 2024-04-25T10:27:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.347 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182626 s, 57.4 MB/s 2024-04-25T10:27:41.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.362 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137218 s, 76.4 MB/s 2024-04-25T10:27:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184178 s, 56.9 MB/s 2024-04-25T10:27:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148626 s, 70.6 MB/s 2024-04-25T10:27:41.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117469 s, 89.3 MB/s 2024-04-25T10:27:41.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184674 s, 56.8 MB/s 2024-04-25T10:27:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147051 s, 71.3 MB/s 2024-04-25T10:27:41.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.466 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150239 s, 69.8 MB/s 2024-04-25T10:27:41.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185198 s, 56.6 MB/s 2024-04-25T10:27:41.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.503 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149319 s, 70.2 MB/s 2024-04-25T10:27:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.520 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014539 s, 72.1 MB/s 2024-04-25T10:27:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143894 s, 72.9 MB/s 2024-04-25T10:27:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155343 s, 67.5 MB/s 2024-04-25T10:27:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145048 s, 72.3 MB/s 2024-04-25T10:27:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139758 s, 75.0 MB/s 2024-04-25T10:27:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141304 s, 74.2 MB/s 2024-04-25T10:27:41.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150007 s, 69.9 MB/s 2024-04-25T10:27:41.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.638 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186445 s, 56.2 MB/s 2024-04-25T10:27:41.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166793 s, 62.9 MB/s 2024-04-25T10:27:41.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:41.658 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:41.658 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:41.658 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:27:41.658 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:41.658 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:41.659 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:41.659 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:41.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.734 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172634 s, 60.7 MB/s 2024-04-25T10:27:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128571 s, 81.6 MB/s 2024-04-25T10:27:41.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.768 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.769 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183673 s, 57.1 MB/s 2024-04-25T10:27:41.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.785 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.785 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149477 s, 70.1 MB/s 2024-04-25T10:27:41.801 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.801 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.801 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139332 s, 75.3 MB/s 2024-04-25T10:27:41.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.816 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136228 s, 77.0 MB/s 2024-04-25T10:27:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138129 s, 75.9 MB/s 2024-04-25T10:27:41.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.853 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.853 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189501 s, 55.3 MB/s 2024-04-25T10:27:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135798 s, 77.2 MB/s 2024-04-25T10:27:41.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.885 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155709 s, 67.3 MB/s 2024-04-25T10:27:41.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.906 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188716 s, 55.6 MB/s 2024-04-25T10:27:41.921 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.921 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138475 s, 75.7 MB/s 2024-04-25T10:27:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177394 s, 59.1 MB/s 2024-04-25T10:27:41.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.957 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146171 s, 71.7 MB/s 2024-04-25T10:27:41.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.973 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.973 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013627 s, 76.9 MB/s 2024-04-25T10:27:41.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:41.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:41.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142638 s, 73.5 MB/s 2024-04-25T10:27:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.009 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186901 s, 56.1 MB/s 2024-04-25T10:27:42.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.026 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154953 s, 67.7 MB/s 2024-04-25T10:27:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.043 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147126 s, 71.3 MB/s 2024-04-25T10:27:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.064 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194887 s, 53.8 MB/s 2024-04-25T10:27:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01463 s, 71.7 MB/s 2024-04-25T10:27:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.096 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134149 s, 78.2 MB/s 2024-04-25T10:27:42.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225502 s, 46.5 MB/s 2024-04-25T10:27:42.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158291 s, 66.2 MB/s 2024-04-25T10:27:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015236 s, 68.8 MB/s 2024-04-25T10:27:42.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146091 s, 71.8 MB/s 2024-04-25T10:27:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130614 s, 80.3 MB/s 2024-04-25T10:27:42.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.200 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112135 s, 93.5 MB/s 2024-04-25T10:27:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134805 s, 77.8 MB/s 2024-04-25T10:27:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.231 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134934 s, 77.7 MB/s 2024-04-25T10:27:42.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.248 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156743 s, 66.9 MB/s 2024-04-25T10:27:42.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.263 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133121 s, 78.8 MB/s 2024-04-25T10:27:42.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:42.265 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135703 s, 77.3 MB/s 2024-04-25T10:27:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153369 s, 68.4 MB/s 2024-04-25T10:27:42.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.371 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180713 s, 58.0 MB/s 2024-04-25T10:27:42.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.388 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150445 s, 69.7 MB/s 2024-04-25T10:27:42.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.404 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141253 s, 74.2 MB/s 2024-04-25T10:27:42.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.421 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014577 s, 71.9 MB/s 2024-04-25T10:27:42.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136426 s, 76.9 MB/s 2024-04-25T10:27:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158297 s, 66.2 MB/s 2024-04-25T10:27:42.469 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.470 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144153 s, 72.7 MB/s 2024-04-25T10:27:42.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174538 s, 60.1 MB/s 2024-04-25T10:27:42.504 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.505 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136627 s, 76.7 MB/s 2024-04-25T10:27:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.525 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186695 s, 56.2 MB/s 2024-04-25T10:27:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.541 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147085 s, 71.3 MB/s 2024-04-25T10:27:42.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143871 s, 72.9 MB/s 2024-04-25T10:27:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.572 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127415 s, 82.3 MB/s 2024-04-25T10:27:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175887 s, 59.6 MB/s 2024-04-25T10:27:42.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135243 s, 77.5 MB/s 2024-04-25T10:27:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.626 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01787 s, 58.7 MB/s 2024-04-25T10:27:42.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146769 s, 71.4 MB/s 2024-04-25T10:27:42.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.658 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135193 s, 77.6 MB/s 2024-04-25T10:27:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.674 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138127 s, 75.9 MB/s 2024-04-25T10:27:42.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.694 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.694 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184977 s, 56.7 MB/s 2024-04-25T10:27:42.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132548 s, 79.1 MB/s 2024-04-25T10:27:42.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.725 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.725 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139411 s, 75.2 MB/s 2024-04-25T10:27:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128816 s, 81.4 MB/s 2024-04-25T10:27:42.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.756 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.756 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143225 s, 73.2 MB/s 2024-04-25T10:27:42.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145822 s, 71.9 MB/s 2024-04-25T10:27:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184711 s, 56.8 MB/s 2024-04-25T10:27:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.809 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148271 s, 70.7 MB/s 2024-04-25T10:27:42.823 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.824 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129097 s, 81.2 MB/s 2024-04-25T10:27:42.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.841 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015225 s, 68.9 MB/s 2024-04-25T10:27:42.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176353 s, 59.5 MB/s 2024-04-25T10:27:42.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:42.861 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:42.861 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:42.862 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:27:42.862 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:42.862 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:42.862 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:42.862 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:42.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133849 s, 78.3 MB/s 2024-04-25T10:27:42.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160031 s, 65.5 MB/s 2024-04-25T10:27:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128489 s, 81.6 MB/s 2024-04-25T10:27:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139791 s, 75.0 MB/s 2024-04-25T10:27:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:42.999 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014448 s, 72.6 MB/s 2024-04-25T10:27:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151249 s, 69.3 MB/s 2024-04-25T10:27:43.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.033 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.033 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151794 s, 69.1 MB/s 2024-04-25T10:27:43.049 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.050 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145416 s, 72.1 MB/s 2024-04-25T10:27:43.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018063 s, 58.1 MB/s 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146738 s, 71.5 MB/s 2024-04-25T10:27:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135269 s, 77.5 MB/s 2024-04-25T10:27:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017909 s, 58.6 MB/s 2024-04-25T10:27:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133803 s, 78.4 MB/s 2024-04-25T10:27:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.152 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.152 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134558 s, 77.9 MB/s 2024-04-25T10:27:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142841 s, 73.4 MB/s 2024-04-25T10:27:43.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.186 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160201 s, 65.5 MB/s 2024-04-25T10:27:43.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.202 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014241 s, 73.6 MB/s 2024-04-25T10:27:43.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.218 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146112 s, 71.8 MB/s 2024-04-25T10:27:43.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.238 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.238 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147692 s, 71.0 MB/s 2024-04-25T10:27:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132538 s, 79.1 MB/s 2024-04-25T10:27:43.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.265 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134198 s, 78.1 MB/s 2024-04-25T10:27:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.280 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136703 s, 76.7 MB/s 2024-04-25T10:27:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151013 s, 69.4 MB/s 2024-04-25T10:27:43.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.315 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158529 s, 66.1 MB/s 2024-04-25T10:27:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.330 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131642 s, 79.7 MB/s 2024-04-25T10:27:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153884 s, 68.1 MB/s 2024-04-25T10:27:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132244 s, 79.3 MB/s 2024-04-25T10:27:43.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.378 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138351 s, 75.8 MB/s 2024-04-25T10:27:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131972 s, 79.5 MB/s 2024-04-25T10:27:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014 s, 74.9 MB/s 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.428 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174448 s, 60.1 MB/s 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.445 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152527 s, 68.7 MB/s 2024-04-25T10:27:43.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:43.447 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:43.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.520 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189299 s, 55.4 MB/s 2024-04-25T10:27:43.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.536 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148477 s, 70.6 MB/s 2024-04-25T10:27:43.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141933 s, 73.9 MB/s 2024-04-25T10:27:43.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.568 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.568 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140797 s, 74.5 MB/s 2024-04-25T10:27:43.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133755 s, 78.4 MB/s 2024-04-25T10:27:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143217 s, 73.2 MB/s 2024-04-25T10:27:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195897 s, 53.5 MB/s 2024-04-25T10:27:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140461 s, 74.7 MB/s 2024-04-25T10:27:43.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.653 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146156 s, 71.7 MB/s 2024-04-25T10:27:43.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.668 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135532 s, 77.4 MB/s 2024-04-25T10:27:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121729 s, 86.1 MB/s 2024-04-25T10:27:43.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.704 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198989 s, 52.7 MB/s 2024-04-25T10:27:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128344 s, 81.7 MB/s 2024-04-25T10:27:43.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.734 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135831 s, 77.2 MB/s 2024-04-25T10:27:43.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138208 s, 75.9 MB/s 2024-04-25T10:27:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141006 s, 74.4 MB/s 2024-04-25T10:27:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193535 s, 54.2 MB/s 2024-04-25T10:27:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149504 s, 70.1 MB/s 2024-04-25T10:27:43.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133667 s, 78.4 MB/s 2024-04-25T10:27:43.838 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.839 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184367 s, 56.9 MB/s 2024-04-25T10:27:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.854 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138513 s, 75.7 MB/s 2024-04-25T10:27:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.870 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137471 s, 76.3 MB/s 2024-04-25T10:27:43.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136249 s, 77.0 MB/s 2024-04-25T10:27:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151333 s, 69.3 MB/s 2024-04-25T10:27:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.923 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187432 s, 55.9 MB/s 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190093 s, 55.2 MB/s 2024-04-25T10:27:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.961 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155291 s, 67.5 MB/s 2024-04-25T10:27:43.976 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.976 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.976 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013205 s, 79.4 MB/s 2024-04-25T10:27:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:43.993 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015463 s, 67.8 MB/s 2024-04-25T10:27:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185063 s, 56.7 MB/s 2024-04-25T10:27:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.030 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147885 s, 70.9 MB/s 2024-04-25T10:27:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.046 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141101 s, 74.3 MB/s 2024-04-25T10:27:44.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:44.048 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018692 s, 56.1 MB/s 2024-04-25T10:27:44.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.141 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154379 s, 67.9 MB/s 2024-04-25T10:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013939 s, 75.2 MB/s 2024-04-25T10:27:44.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153678 s, 68.2 MB/s 2024-04-25T10:27:44.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.189 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139409 s, 75.2 MB/s 2024-04-25T10:27:44.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137697 s, 76.2 MB/s 2024-04-25T10:27:44.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.225 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018144 s, 57.8 MB/s 2024-04-25T10:27:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.241 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152472 s, 68.8 MB/s 2024-04-25T10:27:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179184 s, 58.5 MB/s 2024-04-25T10:27:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149355 s, 70.2 MB/s 2024-04-25T10:27:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.293 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128234 s, 81.8 MB/s 2024-04-25T10:27:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137746 s, 76.1 MB/s 2024-04-25T10:27:44.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.323 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.323 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128044 s, 81.9 MB/s 2024-04-25T10:27:44.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013935 s, 75.2 MB/s 2024-04-25T10:27:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149584 s, 70.1 MB/s 2024-04-25T10:27:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149489 s, 70.1 MB/s 2024-04-25T10:27:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.388 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013793 s, 76.0 MB/s 2024-04-25T10:27:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.403 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138414 s, 75.8 MB/s 2024-04-25T10:27:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142537 s, 73.6 MB/s 2024-04-25T10:27:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131214 s, 79.9 MB/s 2024-04-25T10:27:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154066 s, 68.1 MB/s 2024-04-25T10:27:44.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.472 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018313 s, 57.3 MB/s 2024-04-25T10:27:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150234 s, 69.8 MB/s 2024-04-25T10:27:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.505 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144924 s, 72.4 MB/s 2024-04-25T10:27:44.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.525 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.525 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185947 s, 56.4 MB/s 2024-04-25T10:27:44.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.542 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137043 s, 76.5 MB/s 2024-04-25T10:27:44.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137191 s, 76.4 MB/s 2024-04-25T10:27:44.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.572 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138465 s, 75.7 MB/s 2024-04-25T10:27:44.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.587 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134968 s, 77.7 MB/s 2024-04-25T10:27:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134721 s, 77.8 MB/s 2024-04-25T10:27:44.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.619 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149187 s, 70.3 MB/s 2024-04-25T10:27:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182476 s, 57.5 MB/s 2024-04-25T10:27:44.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:44.641 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013155 s, 79.7 MB/s 2024-04-25T10:27:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187673 s, 55.9 MB/s 2024-04-25T10:27:44.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128646 s, 81.5 MB/s 2024-04-25T10:27:44.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.765 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.765 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138609 s, 75.7 MB/s 2024-04-25T10:27:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132816 s, 78.9 MB/s 2024-04-25T10:27:44.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.797 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.797 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156064 s, 67.2 MB/s 2024-04-25T10:27:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184193 s, 56.9 MB/s 2024-04-25T10:27:44.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.834 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147853 s, 70.9 MB/s 2024-04-25T10:27:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139472 s, 75.2 MB/s 2024-04-25T10:27:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.864 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131977 s, 79.5 MB/s 2024-04-25T10:27:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.881 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148535 s, 70.6 MB/s 2024-04-25T10:27:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135623 s, 77.3 MB/s 2024-04-25T10:27:44.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.913 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141205 s, 74.3 MB/s 2024-04-25T10:27:44.928 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.929 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139251 s, 75.3 MB/s 2024-04-25T10:27:44.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.945 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142519 s, 73.6 MB/s 2024-04-25T10:27:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129214 s, 81.2 MB/s 2024-04-25T10:27:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125761 s, 83.4 MB/s 2024-04-25T10:27:44.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:44.988 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118497 s, 88.5 MB/s 2024-04-25T10:27:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.004 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144625 s, 72.5 MB/s 2024-04-25T10:27:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152507 s, 68.8 MB/s 2024-04-25T10:27:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.041 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188132 s, 55.7 MB/s 2024-04-25T10:27:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146583 s, 71.5 MB/s 2024-04-25T10:27:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014668 s, 71.5 MB/s 2024-04-25T10:27:45.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.090 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137946 s, 76.0 MB/s 2024-04-25T10:27:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151547 s, 69.2 MB/s 2024-04-25T10:27:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146538 s, 71.6 MB/s 2024-04-25T10:27:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138499 s, 75.7 MB/s 2024-04-25T10:27:45.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01362 s, 77.0 MB/s 2024-04-25T10:27:45.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140459 s, 74.7 MB/s 2024-04-25T10:27:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140352 s, 74.7 MB/s 2024-04-25T10:27:45.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.203 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145164 s, 72.2 MB/s 2024-04-25T10:27:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.218 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013389 s, 78.3 MB/s 2024-04-25T10:27:45.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:45.219 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:45.219 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:45.219 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:27:45.219 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:45.220 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:45.220 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:45.220 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:45.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186134 s, 56.3 MB/s 2024-04-25T10:27:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014377 s, 72.9 MB/s 2024-04-25T10:27:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.332 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187352 s, 56.0 MB/s 2024-04-25T10:27:45.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.348 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.348 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014646 s, 71.6 MB/s 2024-04-25T10:27:45.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.366 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013856 s, 75.7 MB/s 2024-04-25T10:27:45.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.380 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134253 s, 78.1 MB/s 2024-04-25T10:27:45.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.396 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014668 s, 71.5 MB/s 2024-04-25T10:27:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170815 s, 61.4 MB/s 2024-04-25T10:27:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.431 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147788 s, 71.0 MB/s 2024-04-25T10:27:45.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189271 s, 55.4 MB/s 2024-04-25T10:27:45.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.469 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148379 s, 70.7 MB/s 2024-04-25T10:27:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131624 s, 79.7 MB/s 2024-04-25T10:27:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139539 s, 75.1 MB/s 2024-04-25T10:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.516 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146205 s, 71.7 MB/s 2024-04-25T10:27:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154986 s, 67.7 MB/s 2024-04-25T10:27:45.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012833 s, 81.7 MB/s 2024-04-25T10:27:45.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144385 s, 72.6 MB/s 2024-04-25T10:27:45.580 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.617 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.617 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148706 s, 70.5 MB/s 2024-04-25T10:27:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184354 s, 56.9 MB/s 2024-04-25T10:27:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151795 s, 69.1 MB/s 2024-04-25T10:27:45.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184161 s, 56.9 MB/s 2024-04-25T10:27:45.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.654 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.654 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140367 s, 74.7 MB/s 2024-04-25T10:27:45.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014353 s, 73.1 MB/s 2024-04-25T10:27:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.687 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150742 s, 69.6 MB/s 2024-04-25T10:27:45.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.703 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144523 s, 72.6 MB/s 2024-04-25T10:27:45.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.723 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180068 s, 58.2 MB/s 2024-04-25T10:27:45.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139297 s, 75.3 MB/s 2024-04-25T10:27:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.755 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149861 s, 70.0 MB/s 2024-04-25T10:27:45.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.773 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149284 s, 70.2 MB/s 2024-04-25T10:27:45.788 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145992 s, 71.8 MB/s 2024-04-25T10:27:45.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.809 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186832 s, 56.1 MB/s 2024-04-25T10:27:45.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.826 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150498 s, 69.7 MB/s 2024-04-25T10:27:45.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:45.827 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:45.827 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:45.827 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:27:45.828 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:45.828 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:45.828 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:45.828 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:45.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015107 s, 69.4 MB/s 2024-04-25T10:27:45.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198216 s, 52.9 MB/s 2024-04-25T10:27:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.943 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154496 s, 67.9 MB/s 2024-04-25T10:27:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.959 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143822 s, 72.9 MB/s 2024-04-25T10:27:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.976 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148097 s, 70.8 MB/s 2024-04-25T10:27:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:45.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:45.996 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179034 s, 58.6 MB/s 2024-04-25T10:27:46.011 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141939 s, 73.9 MB/s 2024-04-25T10:27:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136342 s, 76.9 MB/s 2024-04-25T10:27:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150022 s, 69.9 MB/s 2024-04-25T10:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143321 s, 73.2 MB/s 2024-04-25T10:27:46.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135438 s, 77.4 MB/s 2024-04-25T10:27:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139005 s, 75.4 MB/s 2024-04-25T10:27:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.112 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186091 s, 56.3 MB/s 2024-04-25T10:27:46.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.129 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.129 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153635 s, 68.3 MB/s 2024-04-25T10:27:46.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.144 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.144 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133922 s, 78.3 MB/s 2024-04-25T10:27:46.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.161 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.161 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153902 s, 68.1 MB/s 2024-04-25T10:27:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.178 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148075 s, 70.8 MB/s 2024-04-25T10:27:46.194 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.194 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148194 s, 70.8 MB/s 2024-04-25T10:27:46.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.215 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.215 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183334 s, 57.2 MB/s 2024-04-25T10:27:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151159 s, 69.4 MB/s 2024-04-25T10:27:46.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187747 s, 55.9 MB/s 2024-04-25T10:27:46.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.268 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141081 s, 74.3 MB/s 2024-04-25T10:27:46.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.284 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139533 s, 75.1 MB/s 2024-04-25T10:27:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138548 s, 75.7 MB/s 2024-04-25T10:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148461 s, 70.6 MB/s 2024-04-25T10:27:46.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.337 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186953 s, 56.1 MB/s 2024-04-25T10:27:46.353 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.353 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.353 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143495 s, 73.1 MB/s 2024-04-25T10:27:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.369 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145984 s, 71.8 MB/s 2024-04-25T10:27:46.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.384 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136712 s, 76.7 MB/s 2024-04-25T10:27:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150162 s, 69.8 MB/s 2024-04-25T10:27:46.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.418 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.418 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149529 s, 70.1 MB/s 2024-04-25T10:27:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135045 s, 77.6 MB/s 2024-04-25T10:27:46.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:46.434 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:46.435 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:46.435 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:27:46.435 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:46.435 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:46.435 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:46.435 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182811 s, 57.4 MB/s 2024-04-25T10:27:46.523 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.523 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.523 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143981 s, 72.8 MB/s 2024-04-25T10:27:46.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.539 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.539 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014011 s, 74.8 MB/s 2024-04-25T10:27:46.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172821 s, 60.7 MB/s 2024-04-25T10:27:46.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.578 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179878 s, 58.3 MB/s 2024-04-25T10:27:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134013 s, 78.2 MB/s 2024-04-25T10:27:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.609 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143098 s, 73.3 MB/s 2024-04-25T10:27:46.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013209 s, 79.4 MB/s 2024-04-25T10:27:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135846 s, 77.2 MB/s 2024-04-25T10:27:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135826 s, 77.2 MB/s 2024-04-25T10:27:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.672 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154367 s, 67.9 MB/s 2024-04-25T10:27:46.692 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184669 s, 56.8 MB/s 2024-04-25T10:27:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136922 s, 76.6 MB/s 2024-04-25T10:27:46.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192328 s, 54.5 MB/s 2024-04-25T10:27:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147165 s, 71.3 MB/s 2024-04-25T10:27:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145254 s, 72.2 MB/s 2024-04-25T10:27:46.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014841 s, 70.7 MB/s 2024-04-25T10:27:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186285 s, 56.3 MB/s 2024-04-25T10:27:46.813 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.814 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.814 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128676 s, 81.5 MB/s 2024-04-25T10:27:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154731 s, 67.8 MB/s 2024-04-25T10:27:46.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.845 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.846 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127936 s, 82.0 MB/s 2024-04-25T10:27:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.861 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136428 s, 76.9 MB/s 2024-04-25T10:27:46.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148477 s, 70.6 MB/s 2024-04-25T10:27:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.898 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186189 s, 56.3 MB/s 2024-04-25T10:27:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141367 s, 74.2 MB/s 2024-04-25T10:27:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132464 s, 79.2 MB/s 2024-04-25T10:27:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148576 s, 70.6 MB/s 2024-04-25T10:27:46.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.963 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145789 s, 71.9 MB/s 2024-04-25T10:27:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147979 s, 70.9 MB/s 2024-04-25T10:27:46.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:46.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:46.994 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135447 s, 77.4 MB/s 2024-04-25T10:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.015 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186309 s, 56.3 MB/s 2024-04-25T10:27:47.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140364 s, 74.7 MB/s 2024-04-25T10:27:47.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:47.032 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.104 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153419 s, 68.3 MB/s 2024-04-25T10:27:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148065 s, 70.8 MB/s 2024-04-25T10:27:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136419 s, 76.9 MB/s 2024-04-25T10:27:47.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159386 s, 65.8 MB/s 2024-04-25T10:27:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018422 s, 56.9 MB/s 2024-04-25T10:27:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.194 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166426 s, 63.0 MB/s 2024-04-25T10:27:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168823 s, 62.1 MB/s 2024-04-25T10:27:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.230 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154082 s, 68.1 MB/s 2024-04-25T10:27:47.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.246 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143899 s, 72.9 MB/s 2024-04-25T10:27:47.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.263 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015587 s, 67.3 MB/s 2024-04-25T10:27:47.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.280 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145717 s, 72.0 MB/s 2024-04-25T10:27:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138687 s, 75.6 MB/s 2024-04-25T10:27:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140969 s, 74.4 MB/s 2024-04-25T10:27:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134536 s, 77.9 MB/s 2024-04-25T10:27:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138254 s, 75.8 MB/s 2024-04-25T10:27:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148158 s, 70.8 MB/s 2024-04-25T10:27:47.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.375 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143445 s, 73.1 MB/s 2024-04-25T10:27:47.388 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.389 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118799 s, 88.3 MB/s 2024-04-25T10:27:47.405 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.405 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.405 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150776 s, 69.5 MB/s 2024-04-25T10:27:47.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143947 s, 72.8 MB/s 2024-04-25T10:27:47.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147706 s, 71.0 MB/s 2024-04-25T10:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01386 s, 75.7 MB/s 2024-04-25T10:27:47.469 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.469 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.469 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139897 s, 75.0 MB/s 2024-04-25T10:27:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154583 s, 67.8 MB/s 2024-04-25T10:27:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.503 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153051 s, 68.5 MB/s 2024-04-25T10:27:47.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.520 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139764 s, 75.0 MB/s 2024-04-25T10:27:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.534 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133979 s, 78.3 MB/s 2024-04-25T10:27:47.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150435 s, 69.7 MB/s 2024-04-25T10:27:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134412 s, 78.0 MB/s 2024-04-25T10:27:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.583 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142477 s, 73.6 MB/s 2024-04-25T10:27:47.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.603 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179483 s, 58.4 MB/s 2024-04-25T10:27:47.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189919 s, 55.2 MB/s 2024-04-25T10:27:47.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:47.625 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:47.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142153 s, 73.8 MB/s 2024-04-25T10:27:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018808 s, 55.8 MB/s 2024-04-25T10:27:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.736 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013401 s, 78.2 MB/s 2024-04-25T10:27:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146039 s, 71.8 MB/s 2024-04-25T10:27:47.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138358 s, 75.8 MB/s 2024-04-25T10:27:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192936 s, 54.3 MB/s 2024-04-25T10:27:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.805 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146826 s, 71.4 MB/s 2024-04-25T10:27:47.822 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.822 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.822 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143135 s, 73.3 MB/s 2024-04-25T10:27:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119426 s, 87.8 MB/s 2024-04-25T10:27:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013839 s, 75.8 MB/s 2024-04-25T10:27:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135298 s, 77.5 MB/s 2024-04-25T10:27:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130366 s, 80.4 MB/s 2024-04-25T10:27:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136505 s, 76.8 MB/s 2024-04-25T10:27:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131673 s, 79.6 MB/s 2024-04-25T10:27:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139501 s, 75.2 MB/s 2024-04-25T10:27:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138708 s, 75.6 MB/s 2024-04-25T10:27:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156195 s, 67.1 MB/s 2024-04-25T10:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149541 s, 70.1 MB/s 2024-04-25T10:27:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183258 s, 57.2 MB/s 2024-04-25T10:27:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132882 s, 78.9 MB/s 2024-04-25T10:27:48.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.030 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015077 s, 69.5 MB/s 2024-04-25T10:27:48.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.045 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132094 s, 79.4 MB/s 2024-04-25T10:27:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137932 s, 76.0 MB/s 2024-04-25T10:27:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130578 s, 80.3 MB/s 2024-04-25T10:27:48.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.089 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121994 s, 86.0 MB/s 2024-04-25T10:27:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149715 s, 70.0 MB/s 2024-04-25T10:27:48.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147702 s, 71.0 MB/s 2024-04-25T10:27:48.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.143 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.143 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185778 s, 56.4 MB/s 2024-04-25T10:27:48.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140184 s, 74.8 MB/s 2024-04-25T10:27:48.174 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.175 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148208 s, 70.8 MB/s 2024-04-25T10:27:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141316 s, 74.2 MB/s 2024-04-25T10:27:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135227 s, 77.5 MB/s 2024-04-25T10:27:48.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:48.208 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:27:48.208 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:27:48.208 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:27:48.209 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:27:48.209 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:27:48.209 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:27:48.209 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:27:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177345 s, 59.1 MB/s 2024-04-25T10:27:48.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186479 s, 56.2 MB/s 2024-04-25T10:27:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.320 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150384 s, 69.7 MB/s 2024-04-25T10:27:48.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142855 s, 73.4 MB/s 2024-04-25T10:27:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.351 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138354 s, 75.8 MB/s 2024-04-25T10:27:48.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.368 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.368 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146333 s, 71.7 MB/s 2024-04-25T10:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.383 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131676 s, 79.6 MB/s 2024-04-25T10:27:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.404 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190428 s, 55.1 MB/s 2024-04-25T10:27:48.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.420 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.420 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145038 s, 72.3 MB/s 2024-04-25T10:27:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.435 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132495 s, 79.1 MB/s 2024-04-25T10:27:48.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.452 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.452 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153178 s, 68.5 MB/s 2024-04-25T10:27:48.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.468 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135666 s, 77.3 MB/s 2024-04-25T10:27:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.488 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183395 s, 57.2 MB/s 2024-04-25T10:27:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174586 s, 60.1 MB/s 2024-04-25T10:27:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.524 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153372 s, 68.4 MB/s 2024-04-25T10:27:48.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.542 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015388 s, 68.1 MB/s 2024-04-25T10:27:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184522 s, 56.8 MB/s 2024-04-25T10:27:48.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133004 s, 78.8 MB/s 2024-04-25T10:27:48.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.592 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132616 s, 79.1 MB/s 2024-04-25T10:27:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133461 s, 78.6 MB/s 2024-04-25T10:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.622 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130563 s, 80.3 MB/s 2024-04-25T10:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.642 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174331 s, 60.1 MB/s 2024-04-25T10:27:48.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130558 s, 80.3 MB/s 2024-04-25T10:27:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129963 s, 80.7 MB/s 2024-04-25T10:27:48.688 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.688 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.688 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153502 s, 68.3 MB/s 2024-04-25T10:27:48.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.703 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130591 s, 80.3 MB/s 2024-04-25T10:27:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.722 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170038 s, 61.7 MB/s 2024-04-25T10:27:48.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152295 s, 68.9 MB/s 2024-04-25T10:27:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182526 s, 57.4 MB/s 2024-04-25T10:27:48.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.776 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146291 s, 71.7 MB/s 2024-04-25T10:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152945 s, 68.6 MB/s 2024-04-25T10:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:27:48.809 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148175 s, 70.8 MB/s 2024-04-25T10:27:48.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:48.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:27:48.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 Processor -- start 2024-04-25T10:27:48.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 -- start start 2024-04-25T10:27:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f848c0cf040 0x7f848c0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:48.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f848c15bc00 0x7f848c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c071f80 con 0x7f848c0713a0 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c072260 con 0x7f848c15bc00 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8491a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848c0b3700 con 0x7f848c0cf040 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f848c15bc00 0x7f848c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f848c15bc00 0x7f848c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37348/0 (socket says 172.21.15.45:37348) 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 -- 172.21.15.45:0/1299763003 learned_addr learned my addr 172.21.15.45:0/1299763003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f848aa00640 1 --2- 172.21.15.45:0/1299763003 >> v2:172.21.15.117:3300/0 conn(0x7f848c0cf040 0x7f848c0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 -- 172.21.15.45:0/1299763003 >> v2:172.21.15.117:3300/0 conn(0x7f848c0cf040 msgr2=0x7f848c0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 --2- 172.21.15.45:0/1299763003 >> v2:172.21.15.117:3300/0 conn(0x7f848c0cf040 0x7f848c0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 -- 172.21.15.45:0/1299763003 >> v1:172.21.15.117:6789/0 conn(0x7f848c0713a0 legacy=0x7f848c0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:48.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.993+0000 7f8483e00640 1 -- 172.21.15.45:0/1299763003 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c15b120 con 0x7f848c15bc00 2024-04-25T10:27:49.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8483e00640 1 --2- 172.21.15.45:0/1299763003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c15bc00 0x7f848c0cff20 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f848c0d05d0 tx=0x7f847401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee22d9732b5c7aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f848a000640 1 -- 172.21.15.45:0/1299763003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740ab610 con 0x7f848c15bc00 2024-04-25T10:27:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f848a000640 1 -- 172.21.15.45:0/1299763003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84740b0070 con 0x7f848c15bc00 2024-04-25T10:27:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f848a000640 1 -- 172.21.15.45:0/1299763003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740ab9d0 con 0x7f848c15bc00 2024-04-25T10:27:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 -- 172.21.15.45:0/1299763003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c15bc00 msgr2=0x7f848c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 --2- 172.21.15.45:0/1299763003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c15bc00 0x7f848c0cff20 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f848c0d05d0 tx=0x7f847401c520 comp rx=0 tx=0).stop 2024-04-25T10:27:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 -- 172.21.15.45:0/1299763003 shutdown_connections 2024-04-25T10:27:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 --2- 172.21.15.45:0/1299763003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c15bc00 0x7f848c0cff20 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 --2- 172.21.15.45:0/1299763003 >> v2:172.21.15.117:3300/0 conn(0x7f848c0cf040 0x7f848c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 -- 172.21.15.45:0/1299763003 >> 172.21.15.45:0/1299763003 conn(0x7f848c066b70 msgr2=0x7f848c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 -- 172.21.15.45:0/1299763003 shutdown_connections 2024-04-25T10:27:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:48.997+0000 7f8491a00640 1 -- 172.21.15.45:0/1299763003 wait complete. 2024-04-25T10:27:49.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 Processor -- start 2024-04-25T10:27:49.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 -- start start 2024-04-25T10:27:49.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c0713a0 0x7f848c0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 0x7f848c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f848c15bc00 0x7f848c14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0cf5a0 con 0x7f848c0cf040 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0cf420 con 0x7f848c0713a0 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c15e500 con 0x7f848c15bc00 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 0x7f848c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 0x7f848c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37356/0 (socket says 172.21.15.45:37356) 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 -- 172.21.15.45:0/842206529 learned_addr learned my addr 172.21.15.45:0/842206529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f848b400640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f848c15bc00 0x7f848c14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f848aa00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c0713a0 0x7f848c0cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 -- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f848c15bc00 msgr2=0x7f848c14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f848c15bc00 0x7f848c14a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 -- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c0713a0 msgr2=0x7f848c0cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c0713a0 0x7f848c0cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 -- 172.21.15.45:0/842206529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c153ce0 con 0x7f848c0cf040 2024-04-25T10:27:49.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8483e00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 0x7f848c153540 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f84740ad040 tx=0x7f847401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8482a00640 1 -- 172.21.15.45:0/842206529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740be3f0 con 0x7f848c0cf040 2024-04-25T10:27:49.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8482a00640 1 -- 172.21.15.45:0/842206529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84740be590 con 0x7f848c0cf040 2024-04-25T10:27:49.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8482a00640 1 -- 172.21.15.45:0/842206529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740be960 con 0x7f848c0cf040 2024-04-25T10:27:49.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c153fc0 con 0x7f848c0cf040 2024-04-25T10:27:49.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.001+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c152310 con 0x7f848c0cf040 2024-04-25T10:27:49.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8482a00640 1 -- 172.21.15.45:0/842206529 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84740b43f0 con 0x7f848c0cf040 2024-04-25T10:27:49.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8482a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8468066f90 0x7f8468069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8482a00640 1 -- 172.21.15.45:0/842206529 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f847411f250 con 0x7f848c0cf040 2024-04-25T10:27:49.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f848aa00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8468066f90 0x7f8468069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 Processor -- start 2024-04-25T10:27:49.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 -- start start 2024-04-25T10:27:49.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f848aa00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8468066f90 0x7f8468069450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f847c009b60 tx=0x7f847c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d72c76d50 con 0x557d72e72d10 2024-04-25T10:27:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d72bfee40 con 0x557d72e723f0 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d72c72df0 con 0x557d72e29520 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.005+0000 7f8483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e28fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37370/0 (socket says 172.21.15.45:37370) 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 -- 172.21.15.45:0/3445468418 learned_addr learned my addr 172.21.15.45:0/3445468418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848b400640 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 -- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 msgr2=0x557d72e29900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e29900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 -- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 msgr2=0x557d72e727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 -- 172.21.15.45:0/3445468418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d72c13d90 con 0x557d72e72d10 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848aa00640 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e28fe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f84740efb90 tx=0x7f84740192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff0ca8f736e79f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f847b400640 1 -- 172.21.15.45:0/3445468418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740c4070 con 0x557d72e72d10 2024-04-25T10:27:49.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f847b400640 1 -- 172.21.15.45:0/3445468418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84740b0050 con 0x557d72e72d10 2024-04-25T10:27:49.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f847b400640 1 -- 172.21.15.45:0/3445468418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84740efc00 con 0x557d72e72d10 2024-04-25T10:27:49.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- 172.21.15.45:0/3445468418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 msgr2=0x557d72e28fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e28fe0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f84740efb90 tx=0x7f84740192c0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- 172.21.15.45:0/3445468418 shutdown_connections 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e29900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e28fe0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- 172.21.15.45:0/3445468418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- 172.21.15.45:0/3445468418 >> 172.21.15.45:0/3445468418 conn(0x557d72e254d0 msgr2=0x557d72e278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- 172.21.15.45:0/3445468418 shutdown_connections 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- 172.21.15.45:0/3445468418 wait complete. 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- start start 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 Processor -- start 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e2e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e723f0 0x557d72e2e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d72c1c610 con 0x557d72e72d10 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d72c319a0 con 0x557d72e29520 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d72c7b380 con 0x557d72e723f0 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8494f04000 10 client.0 _ll_register_callbacks cb 0x557d72d2b0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e73300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37384/0 (socket says 172.21.15.45:37384) 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848aa00640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e2e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f848b400640 1 -- 172.21.15.45:0/3879352257 learned_addr learned my addr 172.21.15.45:0/3879352257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.009+0000 7f8483e00640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e723f0 0x557d72e2e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848b400640 1 -- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e723f0 msgr2=0x557d72e2e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848b400640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e723f0 0x557d72e2e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848b400640 1 -- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 msgr2=0x557d72e2e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848b400640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e2e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848b400640 1 -- 172.21.15.45:0/3879352257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d72c71d40 con 0x557d72e72d10 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8483e00640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e723f0 0x557d72e2e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848b400640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e73300 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8484012660 tx=0x7f8484012a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c55441bd2a928f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 1 -- 172.21.15.45:0/3879352257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484019020 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84740b0050 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 1 -- 172.21.15.45:0/3879352257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8484003b90 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 1 -- 172.21.15.45:0/3879352257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8484023050 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 10 client.9192 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x557d72cf9f00 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 1 -- 172.21.15.45:0/3879352257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84740b0050 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 1 -- 172.21.15.45:0/3879352257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8484026020 con 0x557d72e72d10 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 10 client.9192 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 10 client.9192 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 20 client.9192 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e73d10 0x557d72e836a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 4 client.9192 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:27:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d72c12a20 con 0x557d72e73d10 2024-04-25T10:27:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848aa00640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e73d10 0x557d72e836a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f848aa00640 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e73d10 0x557d72e836a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f97217217cdb511 server_cookie=bd9637ddac058fd5 in_seq=0 out_seq=0 2024-04-25T10:27:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 10 client.9192 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 1 -- 172.21.15.45:0/3879352257 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d72c12a20 con 0x557d72e73d10 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 10 client.9192 handle_command_reply: tid=0 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 client.9192 shutdown 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 msgr2=0x557d72e73300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e73300 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8484012660 tx=0x7f8484012a60 comp rx=0 tx=0).stop 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 shutdown_connections 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e72d10 0x557d72e73300 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e723f0 0x557d72e2e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e73d10 0x557d72e836a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 --2- 172.21.15.45:0/3879352257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e2e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 >> 172.21.15.45:0/3879352257 conn(0x557d72e254d0 msgr2=0x557d72e28150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f846fe00640 0 client.9192 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 shutdown_connections 2024-04-25T10:27:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 1 -- 172.21.15.45:0/3879352257 wait complete. 2024-04-25T10:27:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.013+0000 7f8494f04000 20 client.9192 trim_cache size 0 max 16384 2024-04-25T10:27:49.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 Processor -- start 2024-04-25T10:27:49.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- start start 2024-04-25T10:27:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e254d0 0x557d72e71200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e71740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 0x557d72e288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c203140 con 0x557d72e723f0 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c1fb120 con 0x557d72e29520 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c1fa520 con 0x557d72e254d0 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 0x557d72e288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 0x557d72e288d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37390/0 (socket says 172.21.15.45:37390) 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e254d0 0x557d72e71200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 -- 172.21.15.45:0/1700159333 learned_addr learned my addr 172.21.15.45:0/1700159333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8483e00640 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e71740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 -- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e254d0 msgr2=0x557d72e71200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e254d0 0x557d72e71200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 -- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 msgr2=0x557d72e71740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e71740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 -- 172.21.15.45:0/1700159333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d72cd0330 con 0x557d72e723f0 2024-04-25T10:27:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f848b400640 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 0x557d72e288d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8484000f90 tx=0x7f848400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f9d976f68375c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f847aa00640 1 -- 172.21.15.45:0/1700159333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484019070 con 0x557d72e723f0 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f847aa00640 1 -- 172.21.15.45:0/1700159333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484006b60 con 0x557d72e723f0 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f847aa00640 1 -- 172.21.15.45:0/1700159333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484005270 con 0x557d72e723f0 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- 172.21.15.45:0/1700159333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 msgr2=0x557d72e288d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 0x557d72e288d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8484000f90 tx=0x7f848400c000 comp rx=0 tx=0).stop 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- 172.21.15.45:0/1700159333 shutdown_connections 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e723f0 0x557d72e288d0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e29520 0x557d72e71740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 --2- 172.21.15.45:0/1700159333 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e254d0 0x557d72e71200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- 172.21.15.45:0/1700159333 >> 172.21.15.45:0/1700159333 conn(0x557d72e73460 msgr2=0x557d72e2e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- 172.21.15.45:0/1700159333 shutdown_connections 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- 172.21.15.45:0/1700159333 wait complete. 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.033+0000 7f8494f04000 1 -- start start 2024-04-25T10:27:49.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 Processor -- start 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 0x557d72e84f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e8f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d72c2ddc0 con 0x557d72e254d0 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c1fab20 con 0x557d72e723f0 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d72c76060 con 0x557d72e29520 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 10 client.0 _ll_register_callbacks cb 0x7f847c009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 0x557d72e84f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 0x557d72e84f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37402/0 (socket says 172.21.15.45:37402) 2024-04-25T10:27:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 -- 172.21.15.45:0/2181592734 learned_addr learned my addr 172.21.15.45:0/2181592734 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8483e00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848b400640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e8f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 -- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 msgr2=0x557d72e8f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e8f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 -- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 msgr2=0x557d72e8f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e8f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 -- 172.21.15.45:0/2181592734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d72c435f0 con 0x557d72e254d0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8483e00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f848aa00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 0x557d72e84f40 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f847c004560 tx=0x7f847c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3532344a117a88e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 1 -- 172.21.15.45:0/2181592734 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c02c6b0 con 0x557d72e254d0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 1 -- 172.21.15.45:0/2181592734 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f847c02c850 con 0x557d72e254d0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8484006b60 con 0x557d72e254d0 2024-04-25T10:27:49.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 1 -- 172.21.15.45:0/2181592734 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f847c03d050 con 0x557d72e254d0 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 10 client.9198 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x557d72c1dfe0 con 0x557d72e254d0 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 1 -- 172.21.15.45:0/2181592734 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484006b60 con 0x557d72e254d0 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 1 -- 172.21.15.45:0/2181592734 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f847c003620 con 0x557d72e254d0 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 10 client.9198 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 10 client.9198 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 20 client.9198 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e72d10 0x557d72ed31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 4 client.9198 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x557d72c84290 con 0x557d72e72d10 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8483e00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e72d10 0x557d72ed31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f8483e00640 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e72d10 0x557d72ed31f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a437b8c7fe3f01 server_cookie=e3afed3cd74444ac in_seq=0 out_seq=0 2024-04-25T10:27:49.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.037+0000 7f846fe00640 10 client.9198 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f846fe00640 1 -- 172.21.15.45:0/2181592734 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557d72c84290 con 0x557d72e72d10 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f846fe00640 10 client.9198 handle_command_reply: tid=0 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 client.9198 shutdown 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 msgr2=0x557d72e84f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 0x557d72e84f40 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f847c004560 tx=0x7f847c004740 comp rx=0 tx=0).stop 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 shutdown_connections 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x557d72e72d10 0x557d72ed31f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d72e723f0 0x557d72e8f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x557d72e29520 0x557d72e8f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 --2- 172.21.15.45:0/2181592734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557d72e254d0 0x557d72e84f40 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f846fe00640 0 client.9198 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 >> 172.21.15.45:0/2181592734 conn(0x557d72e73460 msgr2=0x557d72e278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 shutdown_connections 2024-04-25T10:27:49.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 1 -- 172.21.15.45:0/2181592734 wait complete. 2024-04-25T10:27:49.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8494f04000 20 client.9198 trim_cache size 0 max 16384 2024-04-25T10:27:49.049 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:27:49.050 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:27:49.050 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "3daaeeed-e6e1-410f-9c1b-0e731c3d4636", 2024-04-25T10:27:49.050 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:27:49.050 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8468066f90 msgr2=0x7f8468069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8468066f90 0x7f8468069450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f847c009b60 tx=0x7f847c009190 comp rx=0 tx=0).stop 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 msgr2=0x7f848c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 0x7f848c153540 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f84740ad040 tx=0x7f847401ad60 comp rx=0 tx=0).stop 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 shutdown_connections 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8468066f90 0x7f8468069450 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f848c15bc00 0x7f848c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f848c0cf040 0x7f848c153540 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 --2- 172.21.15.45:0/842206529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f848c0713a0 0x7f848c0cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 >> 172.21.15.45:0/842206529 conn(0x7f848c066b70 msgr2=0x7f848c15c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 shutdown_connections 2024-04-25T10:27:49.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.041+0000 7f8491a00640 1 -- 172.21.15.45:0/842206529 wait complete. 2024-04-25T10:27:49.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:49.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T10:27:49.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.241+0000 7faba4800640 1 Processor -- start 2024-04-25T10:27:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.241+0000 7faba4800640 1 -- start start 2024-04-25T10:27:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba4800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab9c0aef80 0x7fab9c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba4800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0b31c0 0x7fab9c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba4800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c15b080 con 0x7fab9c070350 2024-04-25T10:27:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba4800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c15adb0 con 0x7fab9c0aef80 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba4800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c15eb60 con 0x7fab9c0b31c0 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab9c0aef80 0x7fab9c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab9c0aef80 0x7fab9c15e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37412/0 (socket says 172.21.15.45:37412) 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 -- 172.21.15.45:0/3676645995 learned_addr learned my addr 172.21.15.45:0/3676645995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1400640 1 --1- 172.21.15.45:0/3676645995 >> v1:172.21.15.45:6789/0 conn(0x7fab9c070350 0x7fab9c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36422/0 (socket says 172.21.15.45:36422) 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba2800640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0b31c0 0x7fab9c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 -- 172.21.15.45:0/3676645995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0b31c0 msgr2=0x7fab9c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0b31c0 0x7fab9c0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 -- 172.21.15.45:0/3676645995 >> v1:172.21.15.45:6789/0 conn(0x7fab9c070350 legacy=0x7fab9c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 -- 172.21.15.45:0/3676645995 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c15ee40 con 0x7fab9c0aef80 2024-04-25T10:27:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba2800640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0b31c0 0x7fab9c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.245+0000 7faba1e00640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0aef80 0x7fab9c15e480 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fab90009870 tx=0x7fab9001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c2680d8771a35b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba0a00640 1 -- 172.21.15.45:0/3676645995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900a7070 con 0x7fab9c0aef80 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba0a00640 1 -- 172.21.15.45:0/3676645995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900b0070 con 0x7fab9c0aef80 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba0a00640 1 -- 172.21.15.45:0/3676645995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900ab890 con 0x7fab9c0aef80 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- 172.21.15.45:0/3676645995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0aef80 msgr2=0x7fab9c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0aef80 0x7fab9c15e480 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fab90009870 tx=0x7fab9001c560 comp rx=0 tx=0).stop 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- 172.21.15.45:0/3676645995 shutdown_connections 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0b31c0 0x7fab9c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 --2- 172.21.15.45:0/3676645995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0aef80 0x7fab9c15e480 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- 172.21.15.45:0/3676645995 >> 172.21.15.45:0/3676645995 conn(0x7fab9c0669a0 msgr2=0x7fab9c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- 172.21.15.45:0/3676645995 shutdown_connections 2024-04-25T10:27:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- 172.21.15.45:0/3676645995 wait complete. 2024-04-25T10:27:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 Processor -- start 2024-04-25T10:27:49.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- start start 2024-04-25T10:27:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab9c070350 0x7fab9c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0aef80 0x7fab9c1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 0x7fab9c153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c165990 con 0x7fab9c0b31c0 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c165810 con 0x7fab9c070350 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba4800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c165b10 con 0x7fab9c0aef80 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 0x7fab9c153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 0x7fab9c153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37414/0 (socket says 172.21.15.45:37414) 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba2800640 1 -- 172.21.15.45:0/772699566 learned_addr learned my addr 172.21.15.45:0/772699566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba1e00640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab9c070350 0x7fab9c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.249+0000 7faba1400640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0aef80 0x7fab9c1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba2800640 1 -- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0aef80 msgr2=0x7fab9c1530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba2800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0aef80 0x7fab9c1530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba2800640 1 -- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab9c070350 msgr2=0x7fab9c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba2800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab9c070350 0x7fab9c14d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba2800640 1 -- 172.21.15.45:0/772699566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c167720 con 0x7fab9c0b31c0 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba1e00640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab9c070350 0x7fab9c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba2800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 0x7fab9c153600 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fab9800cd50 tx=0x7fab9800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7fab96000640 1 -- 172.21.15.45:0/772699566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800f680 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7fab96000640 1 -- 172.21.15.45:0/772699566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9800f840 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7fab96000640 1 -- 172.21.15.45:0/772699566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800fc30 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c14f000 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c14f5c0 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7fab96000640 1 -- 172.21.15.45:0/772699566 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab9800e3e0 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7fab96000640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab80066f60 0x7fab80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.253+0000 7fab96000640 1 -- 172.21.15.45:0/772699566 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab98085d10 con 0x7fab9c0b31c0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 Processor -- start 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1e00640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab80066f60 0x7fab80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- start start 2024-04-25T10:27:49.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1e00640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab80066f60 0x7fab80069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fab900af000 tx=0x7fab900a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c7e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 0x56015c7a3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7a3d90 0x56015c7a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015c6861f0 con 0x56015c7e9360 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015c5ece90 con 0x56015c7e8a40 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56015c574f50 con 0x56015c7a3d90 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 0x56015c7a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 0x56015c7a3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37430/0 (socket says 172.21.15.45:37430) 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 -- 172.21.15.45:0/780773810 learned_addr learned my addr 172.21.15.45:0/780773810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1e00640 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c7e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba2800640 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7a3d90 0x56015c7a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 -- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7a3d90 msgr2=0x56015c7a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7a3d90 0x56015c7a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 -- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 msgr2=0x56015c7e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c7e8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 -- 172.21.15.45:0/780773810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56015c5c48b0 con 0x56015c7e9360 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba1400640 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 0x56015c7a3850 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fab8c007670 tx=0x7fab8c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640dd37dff3a4b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7fab8aa00640 1 -- 172.21.15.45:0/780773810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c01a070 con 0x56015c7e9360 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7fab8aa00640 1 -- 172.21.15.45:0/780773810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c001850 con 0x56015c7e9360 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7fab8aa00640 1 -- 172.21.15.45:0/780773810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c001c00 con 0x56015c7e9360 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- 172.21.15.45:0/780773810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 msgr2=0x56015c7a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 0x56015c7a3850 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fab8c007670 tx=0x7fab8c004bc0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- 172.21.15.45:0/780773810 shutdown_connections 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7a3d90 0x56015c7a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e9360 0x56015c7a3850 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 --2- 172.21.15.45:0/780773810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c7e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- 172.21.15.45:0/780773810 >> 172.21.15.45:0/780773810 conn(0x56015c79fd40 msgr2=0x56015c7a2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- 172.21.15.45:0/780773810 shutdown_connections 2024-04-25T10:27:49.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- 172.21.15.45:0/780773810 wait complete. 2024-04-25T10:27:49.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.257+0000 7faba7c4d000 1 -- start start 2024-04-25T10:27:49.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 Processor -- start 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c7f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e8a40 0x56015c7f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c7a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015c5a7540 con 0x56015c7a3d90 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015c5e8f30 con 0x56015c7e9360 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56015c78a5b0 con 0x56015c7e8a40 2024-04-25T10:27:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 10 client.0 _ll_register_callbacks cb 0x56015c6a1180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c7f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c7f01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37442/0 (socket says 172.21.15.45:37442) 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 -- 172.21.15.45:0/1477509872 learned_addr learned my addr 172.21.15.45:0/1477509872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba2800640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c7a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1400640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e8a40 0x56015c7f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 -- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e8a40 msgr2=0x56015c7f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e8a40 0x56015c7f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 -- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 msgr2=0x56015c7a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c7a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 -- 172.21.15.45:0/1477509872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56015c5751a0 con 0x56015c7a3d90 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba2800640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c7a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1e00640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c7f01c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fab9001a190 tx=0x7fab9001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff5e74812705d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 1 -- 172.21.15.45:0/1477509872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900ce060 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 1 -- 172.21.15.45:0/1477509872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab900c0c80 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab8c001850 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 1 -- 172.21.15.45:0/1477509872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab900b43f0 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 10 client.9213 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x56015c66f840 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 1 -- 172.21.15.45:0/1477509872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c001850 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 1 -- 172.21.15.45:0/1477509872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab9c0cfe70 con 0x56015c7a3d90 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 10 client.9213 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 10 client.9213 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 20 client.9213 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7aab20 0x56015c7aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 4 client.9213 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56015c588be0 con 0x56015c7aab20 2024-04-25T10:27:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1400640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7aab20 0x56015c7aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7faba1400640 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7aab20 0x56015c7aaf30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230630a4caaee446 server_cookie=bad4cfd6e7e1e5fb in_seq=0 out_seq=0 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.261+0000 7fab7f400640 10 client.9213 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7fab7f400640 1 -- 172.21.15.45:0/1477509872 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56015c588be0 con 0x56015c7aab20 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7fab7f400640 10 client.9213 handle_command_reply: tid=0 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 client.9213 shutdown 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 msgr2=0x56015c7f01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c7f01c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fab9001a190 tx=0x7fab9001ada0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 shutdown_connections 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7aab20 0x56015c7aaf30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c7a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e8a40 0x56015c7f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7fab7f400640 0 client.9213 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 --2- 172.21.15.45:0/1477509872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c7f01c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 >> 172.21.15.45:0/1477509872 conn(0x56015c79fd40 msgr2=0x56015c7a29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 shutdown_connections 2024-04-25T10:27:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 1 -- 172.21.15.45:0/1477509872 wait complete. 2024-04-25T10:27:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.265+0000 7faba7c4d000 20 client.9213 trim_cache size 0 max 16384 2024-04-25T10:27:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.277+0000 7faba7c4d000 1 Processor -- start 2024-04-25T10:27:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.277+0000 7faba7c4d000 1 -- start start 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c845080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c8455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e9360 0x56015c845b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c2030b0 con 0x56015c7a3d90 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c2022b0 con 0x56015c7e8a40 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c2016b0 con 0x56015c7e9360 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c845080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c845080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37458/0 (socket says 172.21.15.45:37458) 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 -- 172.21.15.45:0/2315963196 learned_addr learned my addr 172.21.15.45:0/2315963196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1400640 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c8455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba2800640 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e9360 0x56015c845b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 -- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e9360 msgr2=0x56015c845b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e9360 0x56015c845b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 -- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 msgr2=0x56015c8455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c8455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 -- 172.21.15.45:0/2315963196 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56015c7e9190 con 0x56015c7a3d90 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba1e00640 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c845080 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fab90019170 tx=0x7fab900c5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d41157b42d116c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7fab8a000640 1 -- 172.21.15.45:0/2315963196 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900c0410 con 0x56015c7a3d90 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7fab8a000640 1 -- 172.21.15.45:0/2315963196 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900b0040 con 0x56015c7a3d90 2024-04-25T10:27:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7fab8a000640 1 -- 172.21.15.45:0/2315963196 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900b47d0 con 0x56015c7a3d90 2024-04-25T10:27:49.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- 172.21.15.45:0/2315963196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 msgr2=0x56015c845080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c845080 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fab90019170 tx=0x7fab900c5000 comp rx=0 tx=0).stop 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- 172.21.15.45:0/2315963196 shutdown_connections 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7e9360 0x56015c845b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e8a40 0x56015c8455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 --2- 172.21.15.45:0/2315963196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7a3d90 0x56015c845080 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- 172.21.15.45:0/2315963196 >> 172.21.15.45:0/2315963196 conn(0x56015c79fd40 msgr2=0x56015c7e70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- 172.21.15.45:0/2315963196 shutdown_connections 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- 172.21.15.45:0/2315963196 wait complete. 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 -- start start 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.281+0000 7faba7c4d000 1 Processor -- start 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 0x56015c8494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c8499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7eded0 0x56015c7ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56015c5ec1a0 con 0x56015c7e8a40 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c201cb0 con 0x56015c7e9360 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56015c5a5df0 con 0x56015c7eded0 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 10 client.0 _ll_register_callbacks cb 0x56015c5afa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 0x56015c8494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c8499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 0x56015c8494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37474/0 (socket says 172.21.15.45:37474) 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 -- 172.21.15.45:0/846555763 learned_addr learned my addr 172.21.15.45:0/846555763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba2800640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7eded0 0x56015c7ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 -- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7eded0 msgr2=0x56015c7ee2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7eded0 0x56015c7ee2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 -- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 msgr2=0x56015c8499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c8499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 -- 172.21.15.45:0/846555763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56015c646a10 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1400640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c8499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1e00640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 0x56015c8494b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fab9001cad0 tx=0x7fab90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc930712ff285ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 1 -- 172.21.15.45:0/846555763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900ce040 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 1 -- 172.21.15.45:0/846555763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab900b4c30 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab900b0040 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 1 -- 172.21.15.45:0/846555763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab900b4e30 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 10 client.9219 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x56015c581b00 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 1 -- 172.21.15.45:0/846555763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab900b0040 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 1 -- 172.21.15.45:0/846555763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab900d6070 con 0x56015c7e8a40 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 10 client.9219 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 10 client.9219 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:27:49.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 20 client.9219 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:49.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7eeca0 0x56015c7ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 4 client.9219 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:27:49.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x56015c5fa3d0 con 0x56015c7eeca0 2024-04-25T10:27:49.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1400640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7eeca0 0x56015c7ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba1400640 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7eeca0 0x56015c7ef0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cf2b1484efa427 server_cookie=9cc4090f214ff954 in_seq=0 out_seq=0 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 10 client.9219 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 1 -- 172.21.15.45:0/846555763 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56015c5fa3d0 con 0x56015c7eeca0 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 10 client.9219 handle_command_reply: tid=0 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 client.9219 shutdown 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 msgr2=0x56015c8494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 0x56015c8494b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fab9001cad0 tx=0x7fab90008040 comp rx=0 tx=0).stop 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 shutdown_connections 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56015c7eded0 0x56015c7ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x56015c7eeca0 0x56015c7ef0b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56015c7e9360 0x56015c8499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 --2- 172.21.15.45:0/846555763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56015c7e8a40 0x56015c8494b0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7fab7f400640 0 client.9219 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 >> 172.21.15.45:0/846555763 conn(0x56015c79fd40 msgr2=0x56015c7e7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 shutdown_connections 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 1 -- 172.21.15.45:0/846555763 wait complete. 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.285+0000 7faba7c4d000 20 client.9219 trim_cache size 0 max 16384 2024-04-25T10:27:49.295 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:27:49.296 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-04-25T10:27:49.296 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:27:49.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab80066f60 msgr2=0x7fab80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab80066f60 0x7fab80069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fab900af000 tx=0x7fab900a7000 comp rx=0 tx=0).stop 2024-04-25T10:27:49.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 msgr2=0x7fab9c153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 0x7fab9c153600 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fab9800cd50 tx=0x7fab9800a650 comp rx=0 tx=0).stop 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 shutdown_connections 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab80066f60 0x7fab80069420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab9c0b31c0 0x7fab9c153600 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab9c0aef80 0x7fab9c1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 --2- 172.21.15.45:0/772699566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab9c070350 0x7fab9c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 >> 172.21.15.45:0/772699566 conn(0x7fab9c0669a0 msgr2=0x7fab9c15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 shutdown_connections 2024-04-25T10:27:49.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.289+0000 7faba4800640 1 -- 172.21.15.45:0/772699566 wait complete. 2024-04-25T10:27:49.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:27:49.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T10:27:49.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 Processor -- start 2024-04-25T10:27:49.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 -- start start 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c0aef80 0x7f524c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 0x7f524c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c15b080 con 0x7f524c070350 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c15adb0 con 0x7f524c0b31c0 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5253e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c15eb60 con 0x7f524c0aef80 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 0x7f524c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5250a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f524c070350 0x7f524c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:35592/0 (socket says 172.21.15.45:35592) 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 0x7f524c0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37482/0 (socket says 172.21.15.45:37482) 2024-04-25T10:27:49.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5250a00640 1 -- 172.21.15.45:0/3510819831 learned_addr learned my addr 172.21.15.45:0/3510819831 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251400640 1 --2- 172.21.15.45:0/3510819831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c0aef80 0x7f524c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251400640 1 -- 172.21.15.45:0/3510819831 >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 msgr2=0x7f524c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251400640 1 --2- 172.21.15.45:0/3510819831 >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 0x7f524c0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251400640 1 -- 172.21.15.45:0/3510819831 >> v1:172.21.15.117:6789/0 conn(0x7f524c070350 legacy=0x7f524c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f524be00640 1 -- 172.21.15.45:0/3510819831 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060257709 0 0) 0x7f524c15b080 con 0x7f524c070350 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251400640 1 -- 172.21.15.45:0/3510819831 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c15ee40 con 0x7f524c0aef80 2024-04-25T10:27:49.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.449+0000 7f5251e00640 1 --2- 172.21.15.45:0/3510819831 >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 0x7f524c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5251400640 1 --2- 172.21.15.45:0/3510819831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c0aef80 0x7f524c15e480 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f52400097c0 tx=0x7f524001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541f4b37548a816e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f524be00640 1 -- 172.21.15.45:0/3510819831 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400b1070 con 0x7f524c0aef80 2024-04-25T10:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f524be00640 1 -- 172.21.15.45:0/3510819831 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52400ac410 con 0x7f524c0aef80 2024-04-25T10:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f524be00640 1 -- 172.21.15.45:0/3510819831 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400ac7e0 con 0x7f524c0aef80 2024-04-25T10:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 -- 172.21.15.45:0/3510819831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c0aef80 msgr2=0x7f524c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 --2- 172.21.15.45:0/3510819831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c0aef80 0x7f524c15e480 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f52400097c0 tx=0x7f524001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 -- 172.21.15.45:0/3510819831 shutdown_connections 2024-04-25T10:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 --2- 172.21.15.45:0/3510819831 >> v2:172.21.15.45:3300/0 conn(0x7f524c0b31c0 0x7f524c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 --2- 172.21.15.45:0/3510819831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c0aef80 0x7f524c15e480 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 -- 172.21.15.45:0/3510819831 >> 172.21.15.45:0/3510819831 conn(0x7f524c0669a0 msgr2=0x7f524c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 -- 172.21.15.45:0/3510819831 shutdown_connections 2024-04-25T10:27:49.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.453+0000 7f5253e00640 1 -- 172.21.15.45:0/3510819831 wait complete. 2024-04-25T10:27:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 Processor -- start 2024-04-25T10:27:49.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 -- start start 2024-04-25T10:27:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c070350 0x7f524c1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 0x7f524c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f524c0b31c0 0x7f524c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c06c250 con 0x7f524c0aef80 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c06c0d0 con 0x7f524c0b31c0 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c06c3d0 con 0x7f524c070350 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 0x7f524c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 0x7f524c151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37484/0 (socket says 172.21.15.45:37484) 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 -- 172.21.15.45:0/557934643 learned_addr learned my addr 172.21.15.45:0/557934643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5251400640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c070350 0x7f524c1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5251e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f524c0b31c0 0x7f524c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 -- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c070350 msgr2=0x7f524c1f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c070350 0x7f524c1f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 -- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f524c0b31c0 msgr2=0x7f524c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f524c0b31c0 0x7f524c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 -- 172.21.15.45:0/557934643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c152440 con 0x7f524c0aef80 2024-04-25T10:27:49.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5250a00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 0x7f524c151aa0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f523800c520 tx=0x7f523800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5249600640 1 -- 172.21.15.45:0/557934643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238018070 con 0x7f524c0aef80 2024-04-25T10:27:49.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5249600640 1 -- 172.21.15.45:0/557934643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52380074e0 con 0x7f524c0aef80 2024-04-25T10:27:49.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c0af580 con 0x7f524c0aef80 2024-04-25T10:27:49.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5249600640 1 -- 172.21.15.45:0/557934643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380049f0 con 0x7f524c0aef80 2024-04-25T10:27:49.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.457+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c0afb40 con 0x7f524c0aef80 2024-04-25T10:27:49.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5249600640 1 -- 172.21.15.45:0/557934643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f523801b070 con 0x7f524c0aef80 2024-04-25T10:27:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5249600640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f522c066f60 0x7f522c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5249600640 1 -- 172.21.15.45:0/557934643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5238084390 con 0x7f524c0aef80 2024-04-25T10:27:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 Processor -- start 2024-04-25T10:27:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 -- start start 2024-04-25T10:27:49.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5251400640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f522c066f60 0x7f522c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5251400640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f522c066f60 0x7f522c069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f52400ae000 tx=0x7f5240008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738ea8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 0x563738e5eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e5f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738cabff0 con 0x563738ea92f0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738c34150 con 0x563738e5f0a0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.461+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563738ca8090 con 0x563738ea89d0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 0x563738e5eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 0x563738e5eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37486/0 (socket says 172.21.15.45:37486) 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 -- 172.21.15.45:0/468287054 learned_addr learned my addr 172.21.15.45:0/468287054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5251400640 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738ea8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5251e00640 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e5f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 -- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 msgr2=0x563738ea8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738ea8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 -- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 msgr2=0x563738e5f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e5f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 -- 172.21.15.45:0/468287054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563738e420d0 con 0x563738ea92f0 2024-04-25T10:27:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5250a00640 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 0x563738e5eb60 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f5238054c40 tx=0x7f5238058d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f9581b922c41e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f523e000640 1 -- 172.21.15.45:0/468287054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238052fb0 con 0x563738ea92f0 2024-04-25T10:27:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f523e000640 1 -- 172.21.15.45:0/468287054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52380840c0 con 0x563738ea92f0 2024-04-25T10:27:49.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f523e000640 1 -- 172.21.15.45:0/468287054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238068bc0 con 0x563738ea92f0 2024-04-25T10:27:49.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- 172.21.15.45:0/468287054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 msgr2=0x563738e5eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 0x563738e5eb60 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f5238054c40 tx=0x7f5238058d50 comp rx=0 tx=0).stop 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- 172.21.15.45:0/468287054 shutdown_connections 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e5f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea92f0 0x563738e5eb60 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- 172.21.15.45:0/468287054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738ea8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- 172.21.15.45:0/468287054 >> 172.21.15.45:0/468287054 conn(0x563738e5b020 msgr2=0x563738e5d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- 172.21.15.45:0/468287054 shutdown_connections 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- 172.21.15.45:0/468287054 wait complete. 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- start start 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 Processor -- start 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e8d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e8dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea92f0 0x563738e8e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738c70250 con 0x563738ea89d0 2024-04-25T10:27:49.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738c66690 con 0x563738e5f0a0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563738c34a40 con 0x563738ea92f0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.465+0000 7f5257204000 10 client.0 _ll_register_callbacks cb 0x563738e49e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e8d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e8dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e8d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56032/0 (socket says 172.21.15.45:56032) 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 -- 172.21.15.45:0/602494086 learned_addr learned my addr 172.21.15.45:0/602494086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251e00640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea92f0 0x563738e8e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 -- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea92f0 msgr2=0x563738e8e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea92f0 0x563738e8e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 -- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 msgr2=0x563738e8dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e8dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 -- 172.21.15.45:0/602494086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563738d591c0 con 0x563738e5f0a0 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5250a00640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e8dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:49.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5251400640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e8d7f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5240000c10 tx=0x7f524001ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2913cd3701e93fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:27:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 1 -- 172.21.15.45:0/602494086 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400c8020 con 0x563738e5f0a0 2024-04-25T10:27:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 1 -- 172.21.15.45:0/602494086 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52400b79d0 con 0x563738e5f0a0 2024-04-25T10:27:49.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52380840c0 con 0x563738e5f0a0 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 1 -- 172.21.15.45:0/602494086 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52400b1040 con 0x563738e5f0a0 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 10 client.8629 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x563738d55170 con 0x563738e5f0a0 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 1 -- 172.21.15.45:0/602494086 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52380840c0 con 0x563738e5f0a0 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 1 -- 172.21.15.45:0/602494086 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f524c0cfe70 con 0x563738e5f0a0 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 10 client.8629 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 10 client.8629 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 20 client.8629 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738eb8c80 0x563738ebb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 4 client.8629 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563738c47c60 con 0x563738eb8c80 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5250a00640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738eb8c80 0x563738ebb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5250a00640 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738eb8c80 0x563738ebb070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b588f0118c82d2c server_cookie=6b4d6fdd24a22331 in_seq=0 out_seq=0 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 10 client.8629 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 1 -- 172.21.15.45:0/602494086 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f5238081a90 con 0x563738eb8c80 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 10 client.8629 handle_command_reply: tid=0 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 client.8629 shutdown 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 msgr2=0x563738e8d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e8d7f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5240000c10 tx=0x7f524001ace0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 shutdown_connections 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea92f0 0x563738e8e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e8dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738eb8c80 0x563738ebb070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 --2- 172.21.15.45:0/602494086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738e5f0a0 0x563738e8d7f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 >> 172.21.15.45:0/602494086 conn(0x563738e5b020 msgr2=0x563738e5dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5232a00640 0 client.8629 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 shutdown_connections 2024-04-25T10:27:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.469+0000 7f5257204000 1 -- 172.21.15.45:0/602494086 wait complete. 2024-04-25T10:27:49.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.473+0000 7f5257204000 20 client.8629 trim_cache size 0 max 16384 2024-04-25T10:27:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 Processor -- start 2024-04-25T10:27:49.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 -- start start 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738e5f0a0 0x563738e5e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e5e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1feae0 con 0x563738ea89d0 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1fdd50 con 0x563738ea92f0 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.485+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b1b00 con 0x563738e5f0a0 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e5e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e5e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37514/0 (socket says 172.21.15.45:37514) 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 -- 172.21.15.45:0/2613545028 learned_addr learned my addr 172.21.15.45:0/2613545028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5251e00640 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5251400640 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738e5f0a0 0x563738e5e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 -- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738e5f0a0 msgr2=0x563738e5e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738e5f0a0 0x563738e5e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 -- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 msgr2=0x563738ec6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 -- 172.21.15.45:0/2613545028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563738cf2de0 con 0x563738ea89d0 2024-04-25T10:27:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e5e960 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f52380011f0 tx=0x7f5238056000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8589b08b8f26bf58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f523d600640 1 -- 172.21.15.45:0/2613545028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238052dd0 con 0x563738ea89d0 2024-04-25T10:27:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f523d600640 1 -- 172.21.15.45:0/2613545028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523807d070 con 0x563738ea89d0 2024-04-25T10:27:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f523d600640 1 -- 172.21.15.45:0/2613545028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238068bc0 con 0x563738ea89d0 2024-04-25T10:27:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- 172.21.15.45:0/2613545028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 msgr2=0x563738e5e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e5e960 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f52380011f0 tx=0x7f5238056000 comp rx=0 tx=0).stop 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- 172.21.15.45:0/2613545028 shutdown_connections 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738ea89d0 0x563738e5e960 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- 172.21.15.45:0/2613545028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738e5f0a0 0x563738e5e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- 172.21.15.45:0/2613545028 >> 172.21.15.45:0/2613545028 conn(0x563738e5b020 msgr2=0x563738ea70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- 172.21.15.45:0/2613545028 shutdown_connections 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- 172.21.15.45:0/2613545028 wait complete. 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- start start 2024-04-25T10:27:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 Processor -- start 2024-04-25T10:27:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 0x563738f0d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738f0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563738c46e00 con 0x563738e5f0a0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b2100 con 0x563738ea92f0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563738cab300 con 0x563738ea89d0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5257204000 10 client.0 _ll_register_callbacks cb 0x563738e2a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5251400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 0x563738f0d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5251400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 0x563738f0d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37522/0 (socket says 172.21.15.45:37522) 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5251400640 1 -- 172.21.15.45:0/1329479869 learned_addr learned my addr 172.21.15.45:0/1329479869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5251e00640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.489+0000 7f5250a00640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738f0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5251400640 1 -- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 msgr2=0x563738f0dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5251400640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738f0dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5251400640 1 -- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 msgr2=0x563738ec8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5251400640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5251400640 1 -- 172.21.15.45:0/1329479869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563738c4d8e0 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5251400640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 0x563738f0d7a0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f52400a4eb0 tx=0x7f5240009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4509d36dfc09b536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 1 -- 172.21.15.45:0/1329479869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400dc060 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 1 -- 172.21.15.45:0/1329479869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52400c3650 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f523807d070 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 1 -- 172.21.15.45:0/1329479869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52400e2050 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 10 client.9237 fetch_fsmap learned FSMap version 91 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x563738d058f0 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 1 -- 172.21.15.45:0/1329479869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523807d070 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 1 -- 172.21.15.45:0/1329479869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52400e5020 con 0x563738e5f0a0 2024-04-25T10:27:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 10 client.9237 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 10 client.9237 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 20 client.9237 populate_metadata read hostname 'smithi045' 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738ec99c0 0x563738ec9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 4 client.9237 mds_command: new command op to 9012 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 --> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563738cb9530 con 0x563738ec99c0 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5250a00640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738ec99c0 0x563738ec9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5250a00640 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738ec99c0 0x563738ec9dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3882fb095c532a83 server_cookie=95a68bf2fbc4324a in_seq=0 out_seq=0 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 10 client.9237 ms_handle_connect on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 1 -- 172.21.15.45:0/1329479869 <== mds.0 v2:172.21.15.45:6833/1857937743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x563738cb9530 con 0x563738ec99c0 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 10 client.9237 handle_command_reply: tid=0 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 client.9237 shutdown 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 msgr2=0x563738f0d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 0x563738f0d7a0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f52400a4eb0 tx=0x7f5240009190 comp rx=0 tx=0).stop 2024-04-25T10:27:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 shutdown_connections 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:6833/1857937743,v1:172.21.15.45:6835/1857937743] conn(0x563738ec99c0 0x563738ec9dd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563738ea92f0 0x563738ec8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563738ea89d0 0x563738f0dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 --2- 172.21.15.45:0/1329479869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563738e5f0a0 0x563738f0d7a0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 >> 172.21.15.45:0/1329479869 conn(0x563738e5b020 msgr2=0x563738ea7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5232a00640 0 client.9237 ms_handle_reset on v2:172.21.15.45:6833/1857937743 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 shutdown_connections 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 1 -- 172.21.15.45:0/1329479869 wait complete. 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5257204000 20 client.9237 trim_cache size 0 max 16384 2024-04-25T10:27:49.502 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: "3daaeeed-e6e1-410f-9c1b-0e731c3d4636": { 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "3daaeeed-e6e1-410f-9c1b-0e731c3d4636", 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:27:49.503 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:27:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f522c066f60 msgr2=0x7f522c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f522c066f60 0x7f522c069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f52400ae000 tx=0x7f5240008040 comp rx=0 tx=0).stop 2024-04-25T10:27:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 msgr2=0x7f524c151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 0x7f524c151aa0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f523800c520 tx=0x7f523800c920 comp rx=0 tx=0).stop 2024-04-25T10:27:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 shutdown_connections 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f522c066f60 0x7f522c069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f524c0b31c0 0x7f524c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f524c0aef80 0x7f524c151aa0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 --2- 172.21.15.45:0/557934643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f524c070350 0x7f524c1f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 >> 172.21.15.45:0/557934643 conn(0x7f524c0669a0 msgr2=0x7f524c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 shutdown_connections 2024-04-25T10:27:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.493+0000 7f5253e00640 1 -- 172.21.15.45:0/557934643 wait complete. 2024-04-25T10:27:49.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 Processor -- start 2024-04-25T10:27:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 -- start start 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8f01550b0 0x7fc8f00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f00cfe70 0x7fc8f015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f0071050 con 0x7fc8f015b740 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015d040 con 0x7fc8f01550b0 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.653+0000 7fc8f5e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00712f0 con 0x7fc8f00cfe70 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8f01550b0 0x7fc8f00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8f01550b0 0x7fc8f00cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37532/0 (socket says 172.21.15.45:37532) 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8ef400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc8f015b740 0x7fc8f006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36424/0 (socket says 172.21.15.45:36424) 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 -- 172.21.15.45:0/243090901 learned_addr learned my addr 172.21.15.45:0/243090901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8e7e00640 1 --2- 172.21.15.45:0/243090901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f00cfe70 0x7fc8f015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 -- 172.21.15.45:0/243090901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f00cfe70 msgr2=0x7fc8f015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:49.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 --2- 172.21.15.45:0/243090901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f00cfe70 0x7fc8f015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 -- 172.21.15.45:0/243090901 >> v1:172.21.15.45:6789/0 conn(0x7fc8f015b740 legacy=0x7fc8f006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 -- 172.21.15.45:0/243090901 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00715d0 con 0x7fc8f01550b0 2024-04-25T10:27:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8ee000640 1 -- 172.21.15.45:0/243090901 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285368701 0 0) 0x7fc8f0071050 con 0x7fc8f015b740 2024-04-25T10:27:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8eea00640 1 --2- 172.21.15.45:0/243090901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f01550b0 0x7fc8f00cf930 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000bb10 tx=0x7fc8e001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467d8eada04f558e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8ee000640 1 -- 172.21.15.45:0/243090901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b4070 con 0x7fc8f01550b0 2024-04-25T10:27:49.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8ee000640 1 -- 172.21.15.45:0/243090901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00a7b10 con 0x7fc8f01550b0 2024-04-25T10:27:49.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.657+0000 7fc8ee000640 1 -- 172.21.15.45:0/243090901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00af440 con 0x7fc8f01550b0 2024-04-25T10:27:49.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- 172.21.15.45:0/243090901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f01550b0 msgr2=0x7fc8f00cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/243090901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f01550b0 0x7fc8f00cf930 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000bb10 tx=0x7fc8e001ec30 comp rx=0 tx=0).stop 2024-04-25T10:27:49.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- 172.21.15.45:0/243090901 shutdown_connections 2024-04-25T10:27:49.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/243090901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f00cfe70 0x7fc8f015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/243090901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f01550b0 0x7fc8f00cf930 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- 172.21.15.45:0/243090901 >> 172.21.15.45:0/243090901 conn(0x7fc8f0066b70 msgr2=0x7fc8f0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- 172.21.15.45:0/243090901 shutdown_connections 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- 172.21.15.45:0/243090901 wait complete. 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 Processor -- start 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- start start 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f00cfe70 0x7fc8f0150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f01550b0 0x7fc8f014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 0x7fc8f01516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015bd10 con 0x7fc8f015b740 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015bb90 con 0x7fc8f00cfe70 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8f5e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015c030 con 0x7fc8f01550b0 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 0x7fc8f01516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 0x7fc8f01516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37546/0 (socket says 172.21.15.45:37546) 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 -- 172.21.15.45:0/3902365631 learned_addr learned my addr 172.21.15.45:0/3902365631 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8e7e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f01550b0 0x7fc8f014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8eea00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f00cfe70 0x7fc8f0150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 -- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f01550b0 msgr2=0x7fc8f014f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f01550b0 0x7fc8f014f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 -- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f00cfe70 msgr2=0x7fc8f0150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f00cfe70 0x7fc8f0150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.661+0000 7fc8ef400640 1 -- 172.21.15.45:0/3902365631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f014b4d0 con 0x7fc8f015b740 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8ef400640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 0x7fc8f01516b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800c700 tx=0x7fc8e800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8018070 con 0x7fc8f015b740 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e800ce10 con 0x7fc8f015b740 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800a860 con 0x7fc8f015b740 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f014f5e0 con 0x7fc8f015b740 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f01542e0 con 0x7fc8f015b740 2024-04-25T10:27:49.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f01540d0 con 0x7fc8f015b740 2024-04-25T10:27:49.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8e800aa00 con 0x7fc8f015b740 2024-04-25T10:27:49.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.665+0000 7fc8e7400640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc8cc066f60 0x7fc8cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:49.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.669+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8e80845f0 con 0x7fc8f015b740 2024-04-25T10:27:49.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.669+0000 7fc8eea00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc8cc066f60 0x7fc8cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:49.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.669+0000 7fc8eea00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc8cc066f60 0x7fc8cc069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00b1000 tx=0x7fc8e00a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:49.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.673+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8e8054f10 con 0x7fc8f015b740 2024-04-25T10:27:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.865+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8f004ef30 con 0x7fc8f015b740 2024-04-25T10:27:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8e7400640 1 -- 172.21.15.45:0/3902365631 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc8e8058fb0 con 0x7fc8f015b740 2024-04-25T10:27:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:49.872 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc8cc066f60 msgr2=0x7fc8cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc8cc066f60 0x7fc8cc069420 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00b1000 tx=0x7fc8e00a9070 comp rx=0 tx=0).stop 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 msgr2=0x7fc8f01516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 0x7fc8f01516b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800c700 tx=0x7fc8e800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 shutdown_connections 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc8cc066f60 0x7fc8cc069420 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f015b740 0x7fc8f01516b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8f01550b0 0x7fc8f014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 --2- 172.21.15.45:0/3902365631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8f00cfe70 0x7fc8f0150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 >> 172.21.15.45:0/3902365631 conn(0x7fc8f0066b70 msgr2=0x7fc8f00b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 shutdown_connections 2024-04-25T10:27:49.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:49.869+0000 7fc8f5e00640 1 -- 172.21.15.45:0/3902365631 wait complete. 2024-04-25T10:27:49.892 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:27:49.893 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:27:50.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.029+0000 7f803da00640 1 Processor -- start 2024-04-25T10:27:50.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.029+0000 7f803da00640 1 -- start start 2024-04-25T10:27:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f803da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80380d1820 0x7f80380d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f803da00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80380b15b0 0x7f80380cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f803da00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80380b1af0 con 0x7f803815e050 2024-04-25T10:27:50.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f803da00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380b1dd0 con 0x7f80380b15b0 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f803da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380b20b0 con 0x7f80380d1820 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f802e000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f803815e050 0x7f80380b0ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36436/0 (socket says 172.21.15.45:36436) 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f802e000640 1 -- 172.21.15.45:0/339421824 learned_addr learned my addr 172.21.15.45:0/339421824 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8037400640 1 --2- 172.21.15.45:0/339421824 >> v2:172.21.15.45:3300/0 conn(0x7f80380b15b0 0x7f80380cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8036a00640 1 --2- 172.21.15.45:0/339421824 >> v2:172.21.15.117:3300/0 conn(0x7f80380d1820 0x7f80380d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8036000640 1 -- 172.21.15.45:0/339421824 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067773775 0 0) 0x7f80380b1af0 con 0x7f803815e050 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8036000640 1 -- 172.21.15.45:0/339421824 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f801c003530 con 0x7f803815e050 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8037400640 1 -- 172.21.15.45:0/339421824 >> v2:172.21.15.117:3300/0 conn(0x7f80380d1820 msgr2=0x7f80380d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8037400640 1 --2- 172.21.15.45:0/339421824 >> v2:172.21.15.117:3300/0 conn(0x7f80380d1820 0x7f80380d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8037400640 1 -- 172.21.15.45:0/339421824 >> v1:172.21.15.45:6789/0 conn(0x7f803815e050 legacy=0x7f80380b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:50.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.033+0000 7f8037400640 1 -- 172.21.15.45:0/339421824 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380b2390 con 0x7f80380b15b0 2024-04-25T10:27:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f8037400640 1 --2- 172.21.15.45:0/339421824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380b15b0 0x7f80380cfc70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8038070e90 tx=0x7f803001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed473ad889b7919b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f8036000640 1 -- 172.21.15.45:0/339421824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300b2070 con 0x7f80380b15b0 2024-04-25T10:27:50.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f8036000640 1 -- 172.21.15.45:0/339421824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80300a5a10 con 0x7f80380b15b0 2024-04-25T10:27:50.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f8036000640 1 -- 172.21.15.45:0/339421824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300a5de0 con 0x7f80380b15b0 2024-04-25T10:27:50.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 -- 172.21.15.45:0/339421824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380b15b0 msgr2=0x7f80380cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 --2- 172.21.15.45:0/339421824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380b15b0 0x7f80380cfc70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8038070e90 tx=0x7f803001c930 comp rx=0 tx=0).stop 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 -- 172.21.15.45:0/339421824 shutdown_connections 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 --2- 172.21.15.45:0/339421824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380b15b0 0x7f80380cfc70 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 --2- 172.21.15.45:0/339421824 >> v2:172.21.15.117:3300/0 conn(0x7f80380d1820 0x7f80380d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 -- 172.21.15.45:0/339421824 >> 172.21.15.45:0/339421824 conn(0x7f803809a9d0 msgr2=0x7f803809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 -- 172.21.15.45:0/339421824 shutdown_connections 2024-04-25T10:27:50.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.037+0000 7f803da00640 1 -- 172.21.15.45:0/339421824 wait complete. 2024-04-25T10:27:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 Processor -- start 2024-04-25T10:27:50.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- start start 2024-04-25T10:27:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80380b15b0 0x7f803815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 0x7f80380af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f803815e050 0x7f803815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:50.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380d0d90 con 0x7f80380d1820 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380d0c10 con 0x7f80380b15b0 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380d0f10 con 0x7f803815e050 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 0x7f80380af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 0x7f80380af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37566/0 (socket says 172.21.15.45:37566) 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 -- 172.21.15.45:0/3042404296 learned_addr learned my addr 172.21.15.45:0/3042404296 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f8036a00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80380b15b0 0x7f803815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f8037400640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f803815e050 0x7f803815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 -- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f803815e050 msgr2=0x7f803815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f803815e050 0x7f803815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 -- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80380b15b0 msgr2=0x7f803815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80380b15b0 0x7f803815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 -- 172.21.15.45:0/3042404296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380b05b0 con 0x7f80380d1820 2024-04-25T10:27:50.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802e000640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 0x7f80380af110 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f802000c420 tx=0x7f802000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020017070 con 0x7f80380d1820 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020005d10 con 0x7f80380d1820 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8038159e40 con 0x7f80380d1820 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020007310 con 0x7f80380d1820 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803815a400 con 0x7f80380d1820 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.041+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803804ef30 con 0x7f80380d1820 2024-04-25T10:27:50.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.045+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f802001a070 con 0x7f80380d1820 2024-04-25T10:27:50.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.045+0000 7f802f400640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8014066f60 0x7f8014069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:50.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.045+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8020084390 con 0x7f80380d1820 2024-04-25T10:27:50.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.045+0000 7f8036a00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8014066f60 0x7f8014069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:50.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.045+0000 7f8036a00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8014066f60 0x7f8014069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ac0 tx=0x7f8028009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:50.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.049+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8020054e00 con 0x7f80380d1820 2024-04-25T10:27:50.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.269+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f80380b07b0 con 0x7f80380d1820 2024-04-25T10:27:50.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.269+0000 7f802f400640 1 -- 172.21.15.45:0/3042404296 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f8020058ea0 con 0x7f80380d1820 2024-04-25T10:27:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:50.276 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1040,"data_bytes":1068021709,"bytes_used":2543411200,"bytes_avail":770550702080,"bytes_total":773094113280,"write_bytes_sec":70311437,"read_op_per_sec":0,"write_op_per_sec":70},"fsmap":{"epoch":91,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9012}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:27:11.002813+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8014066f60 msgr2=0x7f8014069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8014066f60 0x7f8014069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ac0 tx=0x7f8028009190 comp rx=0 tx=0).stop 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 msgr2=0x7f80380af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 0x7f80380af110 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f802000c420 tx=0x7f802000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 shutdown_connections 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8014066f60 0x7f8014069420 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f803815e050 0x7f803815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80380d1820 0x7f80380af110 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 --2- 172.21.15.45:0/3042404296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80380b15b0 0x7f803815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:50.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 >> 172.21.15.45:0/3042404296 conn(0x7f803809a9d0 msgr2=0x7f8038159110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 shutdown_connections 2024-04-25T10:27:50.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:50.273+0000 7f803da00640 1 -- 172.21.15.45:0/3042404296 wait complete. 2024-04-25T10:27:50.296 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:27:51.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 Processor -- start 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 -- start start 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0ce6c0 0x7fa95c072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa95c06cb10 0x7fa95c0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa95c0cfba0 0x7fa95c08b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c0b4cb0 con 0x7fa95c06cb10 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c0b4470 con 0x7fa95c0cfba0 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa962c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0cffb0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.445+0000 7fa953400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa95c06cb10 0x7fa95c0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.449+0000 7fa953400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa95c06cb10 0x7fa95c0cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37574/0 (socket says 172.21.15.45:37574) 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.449+0000 7fa95be00640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0ce6c0 0x7fa95c072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.449+0000 7fa953400640 1 -- 172.21.15.45:0/2121817485 learned_addr learned my addr 172.21.15.45:0/2121817485 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.449+0000 7fa960c00640 1 --2- 172.21.15.45:0/2121817485 >> v2:172.21.15.117:3300/0 conn(0x7fa95c0cfba0 0x7fa95c08b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa953400640 1 -- 172.21.15.45:0/2121817485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0ce6c0 msgr2=0x7fa95c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa953400640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0ce6c0 0x7fa95c072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa953400640 1 -- 172.21.15.45:0/2121817485 >> v2:172.21.15.117:3300/0 conn(0x7fa95c0cfba0 msgr2=0x7fa95c08b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa953400640 1 --2- 172.21.15.45:0/2121817485 >> v2:172.21.15.117:3300/0 conn(0x7fa95c0cfba0 0x7fa95c08b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa953400640 1 -- 172.21.15.45:0/2121817485 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c0b46f0 con 0x7fa95c06cb10 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa960c00640 1 --2- 172.21.15.45:0/2121817485 >> v2:172.21.15.117:3300/0 conn(0x7fa95c0cfba0 0x7fa95c08b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:51.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.637+0000 7fa95be00640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0ce6c0 0x7fa95c072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:51.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa953400640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c06cb10 0x7fa95c0cf660 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa95c06db10 tx=0x7fa94401cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde5467d375b68c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa95b400640 1 -- 172.21.15.45:0/2121817485 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440ab070 con 0x7fa95c06cb10 2024-04-25T10:27:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa95b400640 1 -- 172.21.15.45:0/2121817485 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9440a5a40 con 0x7fa95c06cb10 2024-04-25T10:27:51.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa95b400640 1 -- 172.21.15.45:0/2121817485 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440a5e10 con 0x7fa95c06cb10 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 -- 172.21.15.45:0/2121817485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c06cb10 msgr2=0x7fa95c0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c06cb10 0x7fa95c0cf660 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa95c06db10 tx=0x7fa94401cac0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 -- 172.21.15.45:0/2121817485 shutdown_connections 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 --2- 172.21.15.45:0/2121817485 >> v2:172.21.15.117:3300/0 conn(0x7fa95c0cfba0 0x7fa95c08b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c06cb10 0x7fa95c0cf660 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 --2- 172.21.15.45:0/2121817485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0ce6c0 0x7fa95c072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 -- 172.21.15.45:0/2121817485 >> 172.21.15.45:0/2121817485 conn(0x7fa95c0669a0 msgr2=0x7fa95c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 -- 172.21.15.45:0/2121817485 shutdown_connections 2024-04-25T10:27:51.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.641+0000 7fa962c00640 1 -- 172.21.15.45:0/2121817485 wait complete. 2024-04-25T10:27:51.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 Processor -- start 2024-04-25T10:27:51.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 -- start start 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa95c06cb10 0x7fa95c08ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 0x7fa95c161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0cfba0 0x7fa95c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0b2dc0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0ceaa0 con 0x7fa95c06cb10 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa962c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c15ab20 con 0x7fa95c0cfba0 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 0x7fa95c161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 0x7fa95c161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37590/0 (socket says 172.21.15.45:37590) 2024-04-25T10:27:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 -- 172.21.15.45:0/1892511708 learned_addr learned my addr 172.21.15.45:0/1892511708 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa95be00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa95c06cb10 0x7fa95c08ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa960c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0cfba0 0x7fa95c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 -- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0cfba0 msgr2=0x7fa95c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0cfba0 0x7fa95c15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 -- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa95c06cb10 msgr2=0x7fa95c08ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa95c06cb10 0x7fa95c08ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 -- 172.21.15.45:0/1892511708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c159f50 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa960c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0cfba0 0x7fa95c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:51.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.645+0000 7fa953400640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 0x7fa95c161c20 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa9440b1040 tx=0x7fa94400ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:51.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440ab040 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9440be070 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440b9400 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c15a1f0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c15a7b0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c04ef30 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9440b95a0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.649+0000 7fa958c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa938066f90 0x7fa938069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.653+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9441200c0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.653+0000 7fa95be00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa938066f90 0x7fa938069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.653+0000 7fa95be00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa938066f90 0x7fa938069450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bfa0 tx=0x7fa94c00b710 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:51.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.657+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9440167d0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.853+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa95c0cd510 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.853+0000 7fa958c00640 1 -- 172.21.15.45:0/1892511708 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa9440f10c0 con 0x7fa95c0ce6c0 2024-04-25T10:27:51.857 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:51.858 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.853+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa938066f90 msgr2=0x7fa938069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.853+0000 7fa962c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa938066f90 0x7fa938069450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bfa0 tx=0x7fa94c00b710 comp rx=0 tx=0).stop 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.853+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 msgr2=0x7fa95c161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.853+0000 7fa962c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 0x7fa95c161c20 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fa9440b1040 tx=0x7fa94400ad10 comp rx=0 tx=0).stop 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 shutdown_connections 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa938066f90 0x7fa938069450 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa95c0cfba0 0x7fa95c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95c0ce6c0 0x7fa95c161c20 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 --2- 172.21.15.45:0/1892511708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa95c06cb10 0x7fa95c08ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 >> 172.21.15.45:0/1892511708 conn(0x7fa95c0669a0 msgr2=0x7fa95c0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 shutdown_connections 2024-04-25T10:27:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:51.857+0000 7fa962c00640 1 -- 172.21.15.45:0/1892511708 wait complete. 2024-04-25T10:27:51.879 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:27:51.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:27:52.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.013+0000 7f91c3200640 1 Processor -- start 2024-04-25T10:27:52.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c3200640 1 -- start start 2024-04-25T10:27:52.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 0x7f91bc15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c3200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f91bc06f100 0x7f91bc071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c3200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0cdc30 con 0x7f91bc0719d0 2024-04-25T10:27:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c3200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc071e20 con 0x7f91bc06f100 2024-04-25T10:27:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc15b5d0 con 0x7f91bc15a9c0 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c1200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f91bc0719d0 0x7f91bc15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36440/0 (socket says 172.21.15.45:36440) 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c1200640 1 -- 172.21.15.45:0/2879140313 learned_addr learned my addr 172.21.15.45:0/2879140313 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bbe00640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 0x7f91bc15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 --2- 172.21.15.45:0/2879140313 >> v2:172.21.15.117:3300/0 conn(0x7f91bc06f100 0x7f91bc071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786196918 0 0) 0x7f91bc0cdc30 con 0x7f91bc0719d0 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a0005740 con 0x7f91bc0719d0 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2391170564 0 0) 0x7f91a0005740 con 0x7f91bc0719d0 2024-04-25T10:27:52.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 msgr2=0x7f91bc15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 0x7f91bc15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 >> v2:172.21.15.117:3300/0 conn(0x7f91bc06f100 msgr2=0x7f91bc071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 --2- 172.21.15.45:0/2879140313 >> v2:172.21.15.117:3300/0 conn(0x7f91bc06f100 0x7f91bc071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc15b8b0 con 0x7f91bc0719d0 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f91b4001a00 con 0x7f91bc0719d0 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 --2- 172.21.15.45:0/2879140313 >> v2:172.21.15.117:3300/0 conn(0x7f91bc06f100 0x7f91bc071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 >> v1:172.21.15.45:6789/0 conn(0x7f91bc0719d0 legacy=0x7f91bc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a0005bd0 0x7f91a0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a0006a20 0x7f91a0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 0x7f91a00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc071e20 con 0x7f91a0006a20 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc15b5d0 con 0x7f91a0005bd0 2024-04-25T10:27:52.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a000dbe0 con 0x7f91bc15a9c0 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a0006a20 0x7f91a0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bbe00640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a0005bd0 0x7f91a0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91c1200640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 0x7f91a00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 -- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 msgr2=0x7f91a00073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91bc15a9c0 0x7f91a00073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 -- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a0005bd0 msgr2=0x7f91a0005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a0005bd0 0x7f91a0005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.017+0000 7f91bb400640 1 -- 172.21.15.45:0/2879140313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a0005740 con 0x7f91a0006a20 2024-04-25T10:27:52.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91bb400640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a0006a20 0x7f91a0006ea0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009cb0 tx=0x7f91a801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abb15bb56348df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:52.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a80b2070 con 0x7f91a0006a20 2024-04-25T10:27:52.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a80a5760 con 0x7f91a0006a20 2024-04-25T10:27:52.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91baa00640 1 -- 172.21.15.45:0/2879140313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8009880 con 0x7f91a0006a20 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c3200640 1 -- 172.21.15.45:0/2879140313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a0006a20 msgr2=0x7f91a0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c3200640 1 --2- 172.21.15.45:0/2879140313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a0006a20 0x7f91a0006ea0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009cb0 tx=0x7f91a801cc90 comp rx=0 tx=0).stop 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c1200640 1 -- 172.21.15.45:0/2879140313 reap_dead start 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c3200640 1 -- 172.21.15.45:0/2879140313 shutdown_connections 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c3200640 1 -- 172.21.15.45:0/2879140313 >> 172.21.15.45:0/2879140313 conn(0x7f91bc0cca80 msgr2=0x7f91bc0cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c3200640 1 -- 172.21.15.45:0/2879140313 shutdown_connections 2024-04-25T10:27:52.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.021+0000 7f91c3200640 1 -- 172.21.15.45:0/2879140313 wait complete. 2024-04-25T10:27:52.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 Processor -- start 2024-04-25T10:27:52.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- start start 2024-04-25T10:27:52.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a0006a20 0x7f91bc1f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 0x7f91bc1f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc15cce0 0x7f91bc15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc071160 con 0x7f91bc1f35e0 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc15be70 con 0x7f91bc15cce0 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0cf0f0 con 0x7f91a0006a20 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 0x7f91bc1f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 0x7f91bc1f0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37596/0 (socket says 172.21.15.45:37596) 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 -- 172.21.15.45:0/3397872865 learned_addr learned my addr 172.21.15.45:0/3397872865 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c1200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc15cce0 0x7f91bc15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bbe00640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a0006a20 0x7f91bc1f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 -- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a0006a20 msgr2=0x7f91bc1f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a0006a20 0x7f91bc1f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 -- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc15cce0 msgr2=0x7f91bc15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc15cce0 0x7f91bc15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 -- 172.21.15.45:0/3397872865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0adf30 con 0x7f91bc1f35e0 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bbe00640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a0006a20 0x7f91bc1f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91bb400640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 0x7f91bc1f0900 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f91a80b0040 tx=0x7f91a0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a80bd020 con 0x7f91bc1f35e0 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a8004240 con 0x7f91bc1f35e0 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a80b2040 con 0x7f91bc1f35e0 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc1f3b30 con 0x7f91bc1f35e0 2024-04-25T10:27:52.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc1f40f0 con 0x7f91bc1f35e0 2024-04-25T10:27:52.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.025+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc15d680 con 0x7f91bc1f35e0 2024-04-25T10:27:52.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.029+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91a8003620 con 0x7f91bc1f35e0 2024-04-25T10:27:52.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.029+0000 7f91b3e00640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:52.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.029+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91a811ee90 con 0x7f91bc1f35e0 2024-04-25T10:27:52.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.029+0000 7f91bbe00640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:52.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.029+0000 7f91bbe00640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f919c066f60 0x7f919c069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f91ac006d50 tx=0x7f91ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:52.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.033+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91bc0adf30 con 0x7f91bc1f35e0 2024-04-25T10:27:52.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.261+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f91bc04ef30 con 0x7f91bc1f35e0 2024-04-25T10:27:52.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91b3e00640 1 -- 172.21.15.45:0/3397872865 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f91a8120080 con 0x7f91bc1f35e0 2024-04-25T10:27:52.268 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:52.268 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":616,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1040,"data_bytes":1068021709,"bytes_used":2543411200,"bytes_avail":770550702080,"bytes_total":773094113280,"write_bytes_sec":47543284,"read_op_per_sec":0,"write_op_per_sec":47},"fsmap":{"epoch":91,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9012}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:27:11.002813+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f919c066f60 msgr2=0x7f919c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f919c066f60 0x7f919c069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f91ac006d50 tx=0x7f91ac008040 comp rx=0 tx=0).stop 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 msgr2=0x7f91bc1f0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 0x7f91bc1f0900 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f91a80b0040 tx=0x7f91a0007a50 comp rx=0 tx=0).stop 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 shutdown_connections 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f919c066f60 0x7f919c069420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91bc15cce0 0x7f91bc15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91bc1f35e0 0x7f91bc1f0900 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 --2- 172.21.15.45:0/3397872865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a0006a20 0x7f91bc1f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:52.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 >> 172.21.15.45:0/3397872865 conn(0x7f91bc0cca80 msgr2=0x7f91bc15c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:52.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 shutdown_connections 2024-04-25T10:27:52.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:52.265+0000 7f91c3200640 1 -- 172.21.15.45:0/3397872865 wait complete. 2024-04-25T10:27:52.289 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:27:53.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.437+0000 7f228c600640 1 Processor -- start 2024-04-25T10:27:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.437+0000 7f228c600640 1 -- start start 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228c600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22841550b0 0x7f22840cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228c600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22840cfe70 0x7f228415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228c600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2284071050 con 0x7f228415b740 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228c600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228415d040 con 0x7f22840cfe70 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228c600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22840712f0 con 0x7f22841550b0 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22840cfe70 0x7f228415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228a600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f228415b740 0x7f228406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36448/0 (socket says 172.21.15.45:36448) 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f22840cfe70 0x7f228415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37612/0 (socket says 172.21.15.45:37612) 2024-04-25T10:27:53.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f228a600640 1 -- 172.21.15.45:0/1506549857 learned_addr learned my addr 172.21.15.45:0/1506549857 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289c00640 1 --2- 172.21.15.45:0/1506549857 >> v2:172.21.15.117:3300/0 conn(0x7f22841550b0 0x7f22840cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289200640 1 -- 172.21.15.45:0/1506549857 >> v2:172.21.15.117:3300/0 conn(0x7f22841550b0 msgr2=0x7f22840cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289200640 1 --2- 172.21.15.45:0/1506549857 >> v2:172.21.15.117:3300/0 conn(0x7f22841550b0 0x7f22840cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289200640 1 -- 172.21.15.45:0/1506549857 >> v1:172.21.15.45:6789/0 conn(0x7f228415b740 legacy=0x7f228406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289200640 1 -- 172.21.15.45:0/1506549857 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840715d0 con 0x7f22840cfe70 2024-04-25T10:27:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.441+0000 7f2289c00640 1 --2- 172.21.15.45:0/1506549857 >> v2:172.21.15.117:3300/0 conn(0x7f22841550b0 0x7f22840cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f2289200640 1 --2- 172.21.15.45:0/1506549857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f228415b200 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f22700097c0 tx=0x7f227001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaae0480e29e0c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f2283e00640 1 -- 172.21.15.45:0/1506549857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700b0070 con 0x7f22840cfe70 2024-04-25T10:27:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f2283e00640 1 -- 172.21.15.45:0/1506549857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22700ab600 con 0x7f22840cfe70 2024-04-25T10:27:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f2283e00640 1 -- 172.21.15.45:0/1506549857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22700ab9d0 con 0x7f22840cfe70 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 -- 172.21.15.45:0/1506549857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 msgr2=0x7f228415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 --2- 172.21.15.45:0/1506549857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f228415b200 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f22700097c0 tx=0x7f227001c460 comp rx=0 tx=0).stop 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 -- 172.21.15.45:0/1506549857 shutdown_connections 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 --2- 172.21.15.45:0/1506549857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f228415b200 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 --2- 172.21.15.45:0/1506549857 >> v2:172.21.15.117:3300/0 conn(0x7f22841550b0 0x7f22840cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 -- 172.21.15.45:0/1506549857 >> 172.21.15.45:0/1506549857 conn(0x7f2284066b70 msgr2=0x7f2284066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 -- 172.21.15.45:0/1506549857 shutdown_connections 2024-04-25T10:27:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 -- 172.21.15.45:0/1506549857 wait complete. 2024-04-25T10:27:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 Processor -- start 2024-04-25T10:27:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.445+0000 7f228c600640 1 -- start start 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f22840b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22841550b0 0x7f22840b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228415b740 0x7f22840ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228415bcd0 con 0x7f22840cfe70 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228415bb50 con 0x7f22841550b0 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228415c030 con 0x7f228415b740 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f22840b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f22840b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37628/0 (socket says 172.21.15.45:37628) 2024-04-25T10:27:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 -- 172.21.15.45:0/1647735006 learned_addr learned my addr 172.21.15.45:0/1647735006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289200640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22841550b0 0x7f22840b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 -- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228415b740 msgr2=0x7f22840ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228a600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228415b740 0x7f22840ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228415b740 0x7f22840ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 -- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22841550b0 msgr2=0x7f22840b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22841550b0 0x7f22840b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 -- 172.21.15.45:0/1647735006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228414f630 con 0x7f22840cfe70 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228a600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228415b740 0x7f22840ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2289c00640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f22840b3050 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f227800e810 tx=0x7f227800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278017070 con 0x7f22840cfe70 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227800c8d0 con 0x7f22840cfe70 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800f040 con 0x7f22840cfe70 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228414f8f0 con 0x7f22840cfe70 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228414feb0 con 0x7f22840cfe70 2024-04-25T10:27:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22841fa830 con 0x7f22840cfe70 2024-04-25T10:27:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.449+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f227801a070 con 0x7f22840cfe70 2024-04-25T10:27:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.453+0000 7f2281600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2264066f60 0x7f2264069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.453+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2278084ca0 con 0x7f22840cfe70 2024-04-25T10:27:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.453+0000 7f2289200640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2264066f60 0x7f2264069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.453+0000 7f2289200640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2264066f60 0x7f2264069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22700ad000 tx=0x7f2270008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.457+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2278055c10 con 0x7f22840cfe70 2024-04-25T10:27:53.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.649+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f228404ef30 con 0x7f22840cfe70 2024-04-25T10:27:53.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.649+0000 7f2281600640 1 -- 172.21.15.45:0/1647735006 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2278055510 con 0x7f22840cfe70 2024-04-25T10:27:53.654 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:53.654 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:53.654 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:53.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2264066f60 msgr2=0x7f2264069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2264066f60 0x7f2264069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22700ad000 tx=0x7f2270008040 comp rx=0 tx=0).stop 2024-04-25T10:27:53.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 msgr2=0x7f22840b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f22840b3050 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f227800e810 tx=0x7f227800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 shutdown_connections 2024-04-25T10:27:53.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2264066f60 0x7f2264069420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228415b740 0x7f22840ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22841550b0 0x7f22840b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 --2- 172.21.15.45:0/1647735006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22840cfe70 0x7f22840b3050 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 >> 172.21.15.45:0/1647735006 conn(0x7f2284066b70 msgr2=0x7f228406c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:53.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 shutdown_connections 2024-04-25T10:27:53.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.653+0000 7f228c600640 1 -- 172.21.15.45:0/1647735006 wait complete. 2024-04-25T10:27:53.676 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:27:53.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:27:53.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 Processor -- start 2024-04-25T10:27:53.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 -- start start 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcad40afda0 0x7fcad406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad4071f50 0x7fcad40669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad406f200 con 0x7fcad40af9c0 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad406d380 con 0x7fcad40afda0 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcadb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad40ccc10 con 0x7fcad4071f50 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcad40afda0 0x7fcad406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcad40afda0 0x7fcad406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37638/0 (socket says 172.21.15.45:37638) 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 -- 172.21.15.45:0/857357646 learned_addr learned my addr 172.21.15.45:0/857357646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcad9600640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad4071f50 0x7fcad40669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 -- 172.21.15.45:0/857357646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad4071f50 msgr2=0x7fcad40669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad4071f50 0x7fcad40669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 -- 172.21.15.45:0/857357646 >> v1:172.21.15.117:6789/0 conn(0x7fcad40af9c0 legacy=0x7fcad40cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcacbe00640 1 -- 172.21.15.45:0/857357646 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad40ccef0 con 0x7fcad40afda0 2024-04-25T10:27:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.813+0000 7fcad9600640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad4071f50 0x7fcad40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcacbe00640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad406ddd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009870 tx=0x7fcabc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32bf2e9c805c136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcad3e00640 1 -- 172.21.15.45:0/857357646 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0b1070 con 0x7fcad40afda0 2024-04-25T10:27:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcad3e00640 1 -- 172.21.15.45:0/857357646 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcabc0ac410 con 0x7fcad40afda0 2024-04-25T10:27:53.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcad3e00640 1 -- 172.21.15.45:0/857357646 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0ac7e0 con 0x7fcad40afda0 2024-04-25T10:27:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 -- 172.21.15.45:0/857357646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 msgr2=0x7fcad406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad406ddd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009870 tx=0x7fcabc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 -- 172.21.15.45:0/857357646 shutdown_connections 2024-04-25T10:27:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad4071f50 0x7fcad40669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 --2- 172.21.15.45:0/857357646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad406ddd0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 -- 172.21.15.45:0/857357646 >> 172.21.15.45:0/857357646 conn(0x7fcad40b2410 msgr2=0x7fcad40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 -- 172.21.15.45:0/857357646 shutdown_connections 2024-04-25T10:27:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.817+0000 7fcadb600640 1 -- 172.21.15.45:0/857357646 wait complete. 2024-04-25T10:27:53.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 Processor -- start 2024-04-25T10:27:53.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- start start 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad4071f50 0x7fcad40ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad40af9c0 0x7fcad41549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad414de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4160460 con 0x7fcad40afda0 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad41602e0 con 0x7fcad4071f50 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad4154f20 con 0x7fcad40af9c0 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad414de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad414de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37654/0 (socket says 172.21.15.45:37654) 2024-04-25T10:27:53.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 -- 172.21.15.45:0/161897234 learned_addr learned my addr 172.21.15.45:0/161897234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad8c00640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad4071f50 0x7fcad40ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcacbe00640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad40af9c0 0x7fcad41549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 -- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad40af9c0 msgr2=0x7fcad41549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad40af9c0 0x7fcad41549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 -- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad4071f50 msgr2=0x7fcad40ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad4071f50 0x7fcad40ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 -- 172.21.15.45:0/161897234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad40adee0 con 0x7fcad40afda0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad9600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad414de70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fd0 tx=0x7fcacc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc01a070 con 0x7fcad40afda0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc003bc0 con 0x7fcad40afda0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc004030 con 0x7fcad40afda0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad40ae1c0 con 0x7fcad40afda0 2024-04-25T10:27:53.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad40ae780 con 0x7fcad40afda0 2024-04-25T10:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.821+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad404ef30 con 0x7fcad40afda0 2024-04-25T10:27:53.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.825+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcacc01d070 con 0x7fcad40afda0 2024-04-25T10:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.825+0000 7fcad1600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcab0066f60 0x7fcab0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.825+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcacc085330 con 0x7fcad40afda0 2024-04-25T10:27:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.825+0000 7fcad8c00640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcab0066f60 0x7fcab0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:53.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.825+0000 7fcad8c00640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcab0066f60 0x7fcab0069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcad40d1420 tx=0x7fcac4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:53.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:53.829+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcacc010070 con 0x7fcad40afda0 2024-04-25T10:27:54.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.053+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcad4071420 con 0x7fcad40afda0 2024-04-25T10:27:54.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.053+0000 7fcad1600640 1 -- 172.21.15.45:0/161897234 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7fcacc055c50 con 0x7fcad40afda0 2024-04-25T10:27:54.059 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:54.059 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":618,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076410317,"bytes_used":2555994112,"bytes_avail":770538119168,"bytes_total":773094113280,"write_bytes_sec":48242581,"read_op_per_sec":0,"write_op_per_sec":48},"fsmap":{"epoch":91,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9012}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:27:11.002813+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcab0066f60 msgr2=0x7fcab0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcab0066f60 0x7fcab0069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcad40d1420 tx=0x7fcac4009040 comp rx=0 tx=0).stop 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 msgr2=0x7fcad414de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad414de70 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fd0 tx=0x7fcacc00d560 comp rx=0 tx=0).stop 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 shutdown_connections 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcab0066f60 0x7fcab0069420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad40afda0 0x7fcad414de70 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcad40af9c0 0x7fcad41549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 --2- 172.21.15.45:0/161897234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad4071f50 0x7fcad40ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:54.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 >> 172.21.15.45:0/161897234 conn(0x7fcad40b2410 msgr2=0x7fcad406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 shutdown_connections 2024-04-25T10:27:54.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:54.057+0000 7fcadb600640 1 -- 172.21.15.45:0/161897234 wait complete. 2024-04-25T10:27:54.080 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:27:55.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdf800640 1 Processor -- start 2024-04-25T10:27:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdf800640 1 -- start start 2024-04-25T10:27:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdf800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3bd80af9c0 0x7f3bd806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdf800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd806f200 con 0x7f3bd80afda0 2024-04-25T10:27:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdf800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd80ccc10 con 0x7f3bd8071f50 2024-04-25T10:27:55.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdf800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd806d380 con 0x7f3bd80af9c0 2024-04-25T10:27:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bcfe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3bd80afda0 0x7f3bd80cc3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60358/0 (socket says 172.21.15.45:60358) 2024-04-25T10:27:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bcfe00640 1 -- 172.21.15.45:0/732795800 learned_addr learned my addr 172.21.15.45:0/732795800 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdce00640 1 --2- 172.21.15.45:0/732795800 >> v2:172.21.15.117:3300/0 conn(0x7f3bd80af9c0 0x7f3bd806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154054297 0 0) 0x7f3bc0001120 con 0x7f3bd80afda0 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bbc003530 con 0x7f3bd80afda0 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471197156 0 0) 0x7f3bd80ccc10 con 0x7f3bd8071f50 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc0001120 con 0x7f3bd8071f50 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635761265 0 0) 0x7f3bbc003530 con 0x7f3bd80afda0 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 >> v2:172.21.15.117:3300/0 conn(0x7f3bd80af9c0 msgr2=0x7f3bd806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 --2- 172.21.15.45:0/732795800 >> v2:172.21.15.117:3300/0 conn(0x7f3bd80af9c0 0x7f3bd806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 >> v1:172.21.15.117:6789/0 conn(0x7f3bd8071f50 legacy=0x7f3bd80669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80ccef0 con 0x7f3bd80afda0 2024-04-25T10:27:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bdce00640 1 --2- 172.21.15.45:0/732795800 >> v2:172.21.15.117:3300/0 conn(0x7f3bd80af9c0 0x7f3bd806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f3bc0001870 con 0x7f3bd80afda0 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.229+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 >> v1:172.21.15.45:6789/0 conn(0x7f3bd80afda0 legacy=0x7f3bd80cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bbc003b50 0x7f3bbc003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bbc004810 0x7f3bbc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc0051d0 0x7f3bbc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd806d380 con 0x7f3bbc003b50 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc00c100 con 0x7f3bbc0051d0 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc00c300 con 0x7f3bbc004810 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bbc003b50 0x7f3bbc003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdd800640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc0051d0 0x7f3bbc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bcfe00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bbc004810 0x7f3bbc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 -- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bbc004810 msgr2=0x7f3bbc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bbc004810 0x7f3bbc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 -- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc0051d0 msgr2=0x7f3bbc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc0051d0 0x7f3bbc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 -- 172.21.15.45:0/732795800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc003530 con 0x7f3bbc003b50 2024-04-25T10:27:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdd800640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc0051d0 0x7f3bbc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdd800640 1 -- 172.21.15.45:0/732795800 reap_dead start 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bdce00640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bbc003b50 0x7f3bbc003f50 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009cb0 tx=0x7f3bc801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfa923763441eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80b2070 con 0x7f3bbc003b50 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc80a58a0 con 0x7f3bbc003b50 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.233+0000 7f3bd7e00640 1 -- 172.21.15.45:0/732795800 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8009880 con 0x7f3bbc003b50 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- 172.21.15.45:0/732795800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bbc003b50 msgr2=0x7f3bbc003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bbc003b50 0x7f3bbc003f50 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009cb0 tx=0x7f3bc801cbe0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- 172.21.15.45:0/732795800 shutdown_connections 2024-04-25T10:27:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 --2- 172.21.15.45:0/732795800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bbc003b50 0x7f3bbc003f50 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- 172.21.15.45:0/732795800 >> 172.21.15.45:0/732795800 conn(0x7f3bd80b2410 msgr2=0x7f3bd80b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- 172.21.15.45:0/732795800 shutdown_connections 2024-04-25T10:27:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- 172.21.15.45:0/732795800 wait complete. 2024-04-25T10:27:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 Processor -- start 2024-04-25T10:27:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- start start 2024-04-25T10:27:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc003b50 0x7f3bd80b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 0x7f3bd81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd8156b10 0x7f3bd81f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80669a0 con 0x7f3bd80b4640 2024-04-25T10:27:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd806d8b0 con 0x7f3bbc003b50 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8066b20 con 0x7f3bd8156b10 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 0x7f3bd81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 0x7f3bd81565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37114/0 (socket says 172.21.15.45:37114) 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 -- 172.21.15.45:0/2112463824 learned_addr learned my addr 172.21.15.45:0/2112463824 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdd800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd8156b10 0x7f3bd81f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bdce00640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc003b50 0x7f3bd80b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 -- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd8156b10 msgr2=0x7f3bd81f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd8156b10 0x7f3bd81f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 -- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc003b50 msgr2=0x7f3bd80b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc003b50 0x7f3bd80b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.237+0000 7f3bcfe00640 1 -- 172.21.15.45:0/2112463824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80b55d0 con 0x7f3bd80b4640 2024-04-25T10:27:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bcfe00640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 0x7f3bd81565d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000e760 tx=0x7f3bc000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:55.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000c8d0 con 0x7f3bd80b4640 2024-04-25T10:27:55.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0018070 con 0x7f3bd80b4640 2024-04-25T10:27:55.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd80b35a0 con 0x7f3bd80b4640 2024-04-25T10:27:55.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000ccb0 con 0x7f3bd80b4640 2024-04-25T10:27:55.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd80b3b60 con 0x7f3bd80b4640 2024-04-25T10:27:55.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd804ef30 con 0x7f3bd80b4640 2024-04-25T10:27:55.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bc001d460 con 0x7f3bd80b4640 2024-04-25T10:27:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bd5600640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3bb4066f60 0x7f3bb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.241+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bc0084fd0 con 0x7f3bd80b4640 2024-04-25T10:27:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.245+0000 7f3bdce00640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3bb4066f60 0x7f3bb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.245+0000 7f3bdce00640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3bb4066f60 0x7f3bb4069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80af000 tx=0x7f3bc8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:55.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.245+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bd806f200 con 0x7f3bd80b4640 2024-04-25T10:27:55.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.441+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bd8071420 con 0x7f3bd80b4640 2024-04-25T10:27:55.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.441+0000 7f3bd5600640 1 -- 172.21.15.45:0/2112463824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3bc0055f40 con 0x7f3bd80b4640 2024-04-25T10:27:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:55.448 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3bb4066f60 msgr2=0x7f3bb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3bb4066f60 0x7f3bb4069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80af000 tx=0x7f3bc8008040 comp rx=0 tx=0).stop 2024-04-25T10:27:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 msgr2=0x7f3bd81565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 0x7f3bd81565d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000e760 tx=0x7f3bc000ec20 comp rx=0 tx=0).stop 2024-04-25T10:27:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 shutdown_connections 2024-04-25T10:27:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3bb4066f60 0x7f3bb4069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3bd8156b10 0x7f3bd81f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bd80b4640 0x7f3bd81565d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 --2- 172.21.15.45:0/2112463824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3bbc003b50 0x7f3bd80b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 >> 172.21.15.45:0/2112463824 conn(0x7f3bd80b2410 msgr2=0x7f3bd8161350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:55.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 shutdown_connections 2024-04-25T10:27:55.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.445+0000 7f3bdf800640 1 -- 172.21.15.45:0/2112463824 wait complete. 2024-04-25T10:27:55.468 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:27:55.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:27:55.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 Processor -- start 2024-04-25T10:27:55.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 -- start start 2024-04-25T10:27:55.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05101550b0 0x7f05100cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05100cfe70 0x7f051015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0510071050 con 0x7f051015b740 2024-04-25T10:27:55.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051015d040 con 0x7f05100cfe70 2024-04-25T10:27:55.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0516800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05100712f0 con 0x7f05101550b0 2024-04-25T10:27:55.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05100cfe70 0x7f051015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05100cfe70 0x7f051015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37124/0 (socket says 172.21.15.45:37124) 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 -- 172.21.15.45:0/34941416 learned_addr learned my addr 172.21.15.45:0/34941416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f050f400640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05101550b0 0x7f05100cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 -- 172.21.15.45:0/34941416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05101550b0 msgr2=0x7f05100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05101550b0 0x7f05100cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 -- 172.21.15.45:0/34941416 >> v1:172.21.15.117:6789/0 conn(0x7f051015b740 legacy=0x7f051006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f0507e00640 1 -- 172.21.15.45:0/34941416 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05100715d0 con 0x7f05100cfe70 2024-04-25T10:27:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.605+0000 7f050f400640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05101550b0 0x7f05100cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0507e00640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f051015b200 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009870 tx=0x7f04f801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e2e21a37891ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f050ea00640 1 -- 172.21.15.45:0/34941416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f80b0070 con 0x7f05100cfe70 2024-04-25T10:27:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f050ea00640 1 -- 172.21.15.45:0/34941416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f80ab600 con 0x7f05100cfe70 2024-04-25T10:27:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f050ea00640 1 -- 172.21.15.45:0/34941416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f80ab9d0 con 0x7f05100cfe70 2024-04-25T10:27:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 -- 172.21.15.45:0/34941416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 msgr2=0x7f051015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f051015b200 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009870 tx=0x7f04f801c560 comp rx=0 tx=0).stop 2024-04-25T10:27:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 -- 172.21.15.45:0/34941416 shutdown_connections 2024-04-25T10:27:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f051015b200 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 --2- 172.21.15.45:0/34941416 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05101550b0 0x7f05100cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 -- 172.21.15.45:0/34941416 >> 172.21.15.45:0/34941416 conn(0x7f0510066b70 msgr2=0x7f0510066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 -- 172.21.15.45:0/34941416 shutdown_connections 2024-04-25T10:27:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.609+0000 7f0516800640 1 -- 172.21.15.45:0/34941416 wait complete. 2024-04-25T10:27:55.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 Processor -- start 2024-04-25T10:27:55.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- start start 2024-04-25T10:27:55.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f0510151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05101550b0 0x7f051014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f051015b740 0x7f0510148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051015bcd0 con 0x7f05100cfe70 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051015bb50 con 0x7f05101550b0 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051015c030 con 0x7f051015b740 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f0510151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f0510151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37130/0 (socket says 172.21.15.45:37130) 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 -- 172.21.15.45:0/2301590298 learned_addr learned my addr 172.21.15.45:0/2301590298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0507e00640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05101550b0 0x7f051014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050fe00640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f051015b740 0x7f0510148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 -- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f051015b740 msgr2=0x7f0510148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f051015b740 0x7f0510148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 -- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05101550b0 msgr2=0x7f051014dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05101550b0 0x7f051014dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 -- 172.21.15.45:0/2301590298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051014e320 con 0x7f05100cfe70 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050fe00640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f051015b740 0x7f0510148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0507e00640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05101550b0 0x7f051014dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f050f400640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f0510151110 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f050000c7c0 tx=0x7f050000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0500007a30 con 0x7f05100cfe70 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0500007bd0 con 0x7f05100cfe70 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0510151840 con 0x7f05100cfe70 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05000113f0 con 0x7f05100cfe70 2024-04-25T10:27:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0510151e00 con 0x7f05100cfe70 2024-04-25T10:27:55.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.613+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051004ef30 con 0x7f05100cfe70 2024-04-25T10:27:55.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.617+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0500011590 con 0x7f05100cfe70 2024-04-25T10:27:55.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.617+0000 7f0507400640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f04ec066f90 0x7f04ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:55.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.617+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0500085290 con 0x7f05100cfe70 2024-04-25T10:27:55.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.617+0000 7f0507e00640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f04ec066f90 0x7f04ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:55.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.617+0000 7f0507e00640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f04ec066f90 0x7f04ec069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f04f80ad000 tx=0x7f04f80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.621+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0500015020 con 0x7f05100cfe70 2024-04-25T10:27:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.841+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f051006c7f0 con 0x7f05100cfe70 2024-04-25T10:27:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.841+0000 7f0507400640 1 -- 172.21.15.45:0/2301590298 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f0500056200 con 0x7f05100cfe70 2024-04-25T10:27:55.846 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:55.846 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":620,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":130,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076608120,"bytes_used":2607775744,"bytes_avail":770486337536,"bytes_total":773094113280,"write_bytes_sec":42318348,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":91,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9012}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T10:27:55.012950+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f04ec066f90 msgr2=0x7f04ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f04ec066f90 0x7f04ec069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f04f80ad000 tx=0x7f04f80a6000 comp rx=0 tx=0).stop 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 msgr2=0x7f0510151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f0510151110 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f050000c7c0 tx=0x7f050000cc80 comp rx=0 tx=0).stop 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 shutdown_connections 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f04ec066f90 0x7f04ec069450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f051015b740 0x7f0510148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05101550b0 0x7f051014dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 --2- 172.21.15.45:0/2301590298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05100cfe70 0x7f0510151110 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 >> 172.21.15.45:0/2301590298 conn(0x7f0510066b70 msgr2=0x7f051006cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 shutdown_connections 2024-04-25T10:27:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:55.845+0000 7f0516800640 1 -- 172.21.15.45:0/2301590298 wait complete. 2024-04-25T10:27:55.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:27:55.867 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:27:55.867 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:27:55.882 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:27:55.883 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'a', 'b'} 2024-04-25T10:27:55.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T10:27:56.076 INFO:teuthology.orchestra.run.smithi045.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T10:27:56.077 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T10:27:56.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.213+0000 7f004ca00640 1 Processor -- start 2024-04-25T10:27:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.213+0000 7f004ca00640 1 -- start start 2024-04-25T10:27:56.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f004ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f004815b9b0 0x7f00480ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:56.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f004ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480b5fb0 0x7f00480ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f004ca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480cd120 con 0x7f00480ce700 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f004ca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00480d2600 con 0x7f004815b9b0 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f004ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00480cd400 con 0x7f00480b5fb0 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f004815b9b0 0x7f00480ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f004815b9b0 0x7f00480ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37138/0 (socket says 172.21.15.45:37138) 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 -- 172.21.15.45:0/2555757091 learned_addr learned my addr 172.21.15.45:0/2555757091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047e00640 1 --2- 172.21.15.45:0/2555757091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480b5fb0 0x7f00480ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 -- 172.21.15.45:0/2555757091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480b5fb0 msgr2=0x7f00480ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 --2- 172.21.15.45:0/2555757091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480b5fb0 0x7f00480ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 -- 172.21.15.45:0/2555757091 >> v1:172.21.15.117:6789/0 conn(0x7f00480ce700 legacy=0x7f00480cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.217+0000 7f0047400640 1 -- 172.21.15.45:0/2555757091 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480b47a0 con 0x7f004815b9b0 2024-04-25T10:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f0047400640 1 --2- 172.21.15.45:0/2555757091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004815b9b0 0x7f00480ce1c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0038009870 tx=0x7f003801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c11ac54484c8360 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f0046a00640 1 -- 172.21.15.45:0/2555757091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380b5070 con 0x7f004815b9b0 2024-04-25T10:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f0046a00640 1 -- 172.21.15.45:0/2555757091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00380a8a50 con 0x7f004815b9b0 2024-04-25T10:27:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f0046a00640 1 -- 172.21.15.45:0/2555757091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380a8e20 con 0x7f004815b9b0 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- 172.21.15.45:0/2555757091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004815b9b0 msgr2=0x7f00480ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 --2- 172.21.15.45:0/2555757091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004815b9b0 0x7f00480ce1c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0038009870 tx=0x7f003801ed50 comp rx=0 tx=0).stop 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- 172.21.15.45:0/2555757091 shutdown_connections 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 --2- 172.21.15.45:0/2555757091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480b5fb0 0x7f00480ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 --2- 172.21.15.45:0/2555757091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f004815b9b0 0x7f00480ce1c0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- 172.21.15.45:0/2555757091 >> 172.21.15.45:0/2555757091 conn(0x7f00480d17b0 msgr2=0x7f00480d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- 172.21.15.45:0/2555757091 shutdown_connections 2024-04-25T10:27:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- 172.21.15.45:0/2555757091 wait complete. 2024-04-25T10:27:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 Processor -- start 2024-04-25T10:27:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- start start 2024-04-25T10:27:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 0x7f004806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480ce700 0x7f00480c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004815b9b0 0x7f004809bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004815bf10 con 0x7f00480b5fb0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004815bd90 con 0x7f004815b9b0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f004ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00480ccd30 con 0x7f00480ce700 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.221+0000 7f0047400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 0x7f004806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 0x7f004806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37140/0 (socket says 172.21.15.45:37140) 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 -- 172.21.15.45:0/4049375293 learned_addr learned my addr 172.21.15.45:0/4049375293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f003fe00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480ce700 0x7f00480c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047e00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004815b9b0 0x7f004809bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 -- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480ce700 msgr2=0x7f00480c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480ce700 0x7f00480c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 -- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004815b9b0 msgr2=0x7f004809bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004815b9b0 0x7f004809bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 -- 172.21.15.45:0/4049375293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004806b4c0 con 0x7f00480b5fb0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f0047400640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 0x7f004806ac90 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f003801d400 tx=0x7f003801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380c0070 con 0x7f00480b5fb0 2024-04-25T10:27:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00380ae550 con 0x7f00480b5fb0 2024-04-25T10:27:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380b5070 con 0x7f00480b5fb0 2024-04-25T10:27:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004806b7a0 con 0x7f00480b5fb0 2024-04-25T10:27:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004806b1d0 con 0x7f00480b5fb0 2024-04-25T10:27:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0048075120 con 0x7f00480b5fb0 2024-04-25T10:27:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.225+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00380b9070 con 0x7f00480b5fb0 2024-04-25T10:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.229+0000 7f003f400640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0024066f90 0x7f0024069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.229+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00381227b0 con 0x7f00480b5fb0 2024-04-25T10:27:56.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.229+0000 7f003fe00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0024066f90 0x7f0024069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:27:56.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.229+0000 7f003fe00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0024066f90 0x7f0024069450 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f00480c8710 tx=0x7f0030009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:27:56.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.233+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00380188b0 con 0x7f00480b5fb0 2024-04-25T10:27:56.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.429+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00480d1da0 con 0x7f00480b5fb0 2024-04-25T10:27:56.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.429+0000 7f003f400640 1 -- 172.21.15.45:0/4049375293 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f00380f37b0 con 0x7f00480b5fb0 2024-04-25T10:27:56.435 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:27:56.435 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:27:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:27:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0024066f90 msgr2=0x7f0024069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:56.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0024066f90 0x7f0024069450 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f00480c8710 tx=0x7f0030009040 comp rx=0 tx=0).stop 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 msgr2=0x7f004806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 0x7f004806ac90 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f003801d400 tx=0x7f003801baf0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 shutdown_connections 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0024066f90 0x7f0024069450 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f004815b9b0 0x7f004809bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00480ce700 0x7f00480c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 --2- 172.21.15.45:0/4049375293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00480b5fb0 0x7f004806ac90 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 >> 172.21.15.45:0/4049375293 conn(0x7f00480d17b0 msgr2=0x7f004806f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 shutdown_connections 2024-04-25T10:27:56.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:27:56.433+0000 7f004ca00640 1 -- 172.21.15.45:0/4049375293 wait complete. 2024-04-25T10:27:56.457 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:27:59.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:27:59.765 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:01.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 Processor -- start 2024-04-25T10:28:01.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 -- start start 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71f0157f40 0x7f71f00aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00af380 0x7f71f00669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f71f0066ee0 0x7f71f006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f0067350 con 0x7f71f0157f40 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f00ad840 con 0x7f71f0066ee0 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f7000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f006e7a0 con 0x7f71f00af380 2024-04-25T10:28:01.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71f0157f40 0x7f71f00aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71f0157f40 0x7f71f00aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37152/0 (socket says 172.21.15.45:37152) 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 -- 172.21.15.45:0/2642747954 learned_addr learned my addr 172.21.15.45:0/2642747954 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f6600640 1 --2- 172.21.15.45:0/2642747954 >> v2:172.21.15.117:3300/0 conn(0x7f71f0066ee0 0x7f71f006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5200640 1 --2- 172.21.15.45:0/2642747954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00af380 0x7f71f00669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 -- 172.21.15.45:0/2642747954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00af380 msgr2=0x7f71f00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 --2- 172.21.15.45:0/2642747954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00af380 0x7f71f00669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 -- 172.21.15.45:0/2642747954 >> v2:172.21.15.117:3300/0 conn(0x7f71f0066ee0 msgr2=0x7f71f006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 --2- 172.21.15.45:0/2642747954 >> v2:172.21.15.117:3300/0 conn(0x7f71f0066ee0 0x7f71f006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f5c00640 1 -- 172.21.15.45:0/2642747954 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00af9b0 con 0x7f71f0157f40 2024-04-25T10:28:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.609+0000 7f71f6600640 1 --2- 172.21.15.45:0/2642747954 >> v2:172.21.15.117:3300/0 conn(0x7f71f0066ee0 0x7f71f006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f5c00640 1 --2- 172.21.15.45:0/2642747954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00aee40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f71f00aed00 tx=0x7f71e401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6e90c6b80e187a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71efe00640 1 -- 172.21.15.45:0/2642747954 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40b0070 con 0x7f71f0157f40 2024-04-25T10:28:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71efe00640 1 -- 172.21.15.45:0/2642747954 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e40ab600 con 0x7f71f0157f40 2024-04-25T10:28:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71efe00640 1 -- 172.21.15.45:0/2642747954 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40ab9d0 con 0x7f71f0157f40 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 -- 172.21.15.45:0/2642747954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 msgr2=0x7f71f00aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 --2- 172.21.15.45:0/2642747954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00aee40 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f71f00aed00 tx=0x7f71e401c260 comp rx=0 tx=0).stop 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 -- 172.21.15.45:0/2642747954 shutdown_connections 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 --2- 172.21.15.45:0/2642747954 >> v2:172.21.15.117:3300/0 conn(0x7f71f0066ee0 0x7f71f006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 --2- 172.21.15.45:0/2642747954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f00af380 0x7f71f00669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 --2- 172.21.15.45:0/2642747954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00aee40 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 -- 172.21.15.45:0/2642747954 >> 172.21.15.45:0/2642747954 conn(0x7f71f0074310 msgr2=0x7f71f006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 -- 172.21.15.45:0/2642747954 shutdown_connections 2024-04-25T10:28:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.613+0000 7f71f7000640 1 -- 172.21.15.45:0/2642747954 wait complete. 2024-04-25T10:28:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 Processor -- start 2024-04-25T10:28:01.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- start start 2024-04-25T10:28:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f0066ee0 0x7f71f0151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00af380 0x7f71f0160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f015eee0 con 0x7f71f0157f40 2024-04-25T10:28:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f015e9c0 con 0x7f71f00af380 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0158320 con 0x7f71f0066ee0 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37166/0 (socket says 172.21.15.45:37166) 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 -- 172.21.15.45:0/1468793659 learned_addr learned my addr 172.21.15.45:0/1468793659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f5c00640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f0066ee0 0x7f71f0151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f5200640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00af380 0x7f71f0160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 -- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f0066ee0 msgr2=0x7f71f0151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f0066ee0 0x7f71f0151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 -- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00af380 msgr2=0x7f71f0160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00af380 0x7f71f0160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 -- 172.21.15.45:0/1468793659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0154150 con 0x7f71f0157f40 2024-04-25T10:28:01.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f6600640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00b4a80 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f71e8007e20 tx=0x7f71e800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e801a070 con 0x7f71f0157f40 2024-04-25T10:28:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e80039a0 con 0x7f71f0157f40 2024-04-25T10:28:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f0154430 con 0x7f71f0157f40 2024-04-25T10:28:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8003d70 con 0x7f71f0157f40 2024-04-25T10:28:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f01523b0 con 0x7f71f0157f40 2024-04-25T10:28:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.617+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f01525b0 con 0x7f71f0157f40 2024-04-25T10:28:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.621+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71e801d070 con 0x7f71f0157f40 2024-04-25T10:28:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.621+0000 7f71ed600640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f71d0066f60 0x7f71d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.621+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71e80852b0 con 0x7f71f0157f40 2024-04-25T10:28:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.621+0000 7f71f5c00640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f71d0066f60 0x7f71d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.621+0000 7f71f5c00640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f71d0066f60 0x7f71d0069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f71e40ad000 tx=0x7f71e40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.625+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71e8010070 con 0x7f71f0157f40 2024-04-25T10:28:01.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.817+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f0075120 con 0x7f71f0157f40 2024-04-25T10:28:01.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.817+0000 7f71ed600640 1 -- 172.21.15.45:0/1468793659 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f71e8055bd0 con 0x7f71f0157f40 2024-04-25T10:28:01.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:01.824 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:01.824 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:28:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f71d0066f60 msgr2=0x7f71d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f71d0066f60 0x7f71d0069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f71e40ad000 tx=0x7f71e40a6000 comp rx=0 tx=0).stop 2024-04-25T10:28:01.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 msgr2=0x7f71f00b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00b4a80 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f71e8007e20 tx=0x7f71e800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 shutdown_connections 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f71d0066f60 0x7f71d0069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71f0157f40 0x7f71f00b4a80 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71f00af380 0x7f71f0160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 --2- 172.21.15.45:0/1468793659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71f0066ee0 0x7f71f0151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 >> 172.21.15.45:0/1468793659 conn(0x7f71f0074310 msgr2=0x7f71f00b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 shutdown_connections 2024-04-25T10:28:01.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:01.821+0000 7f71f7000640 1 -- 172.21.15.45:0/1468793659 wait complete. 2024-04-25T10:28:01.845 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:28:06.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:06.997+0000 7f2264000640 1 Processor -- start 2024-04-25T10:28:07.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:06.997+0000 7f2264000640 1 -- start start 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2264000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f225c0cda10 0x7f225c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2264000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c0b2770 con 0x7f225c15bc40 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2264000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c06c670 con 0x7f225c0cf930 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2264000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225c06c950 con 0x7f225c0cda10 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2261600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f225c15bc40 0x7f225c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36778/0 (socket says 172.21.15.45:36778) 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2261600640 1 -- 172.21.15.45:0/1368870247 learned_addr learned my addr 172.21.15.45:0/1368870247 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2260c00640 1 --2- 172.21.15.45:0/1368870247 >> v2:172.21.15.117:3300/0 conn(0x7f225c0cda10 0x7f225c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f225be00640 1 -- 172.21.15.45:0/1368870247 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228037975 0 0) 0x7f225c0b2770 con 0x7f225c15bc40 2024-04-25T10:28:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f225be00640 1 -- 172.21.15.45:0/1368870247 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2244003530 con 0x7f225c15bc40 2024-04-25T10:28:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2260c00640 1 -- 172.21.15.45:0/1368870247 >> v1:172.21.15.117:6789/0 conn(0x7f225c0cf930 legacy=0x7f225c0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2260c00640 1 -- 172.21.15.45:0/1368870247 >> v1:172.21.15.45:6789/0 conn(0x7f225c15bc40 legacy=0x7f225c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.001+0000 7f2260c00640 1 -- 172.21.15.45:0/1368870247 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c06cc30 con 0x7f225c0cda10 2024-04-25T10:28:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2260c00640 1 --2- 172.21.15.45:0/1368870247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cda10 0x7f225c06c090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f22480097c0 tx=0x7f224801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af1676531b2ec42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f225be00640 1 -- 172.21.15.45:0/1368870247 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480b0070 con 0x7f225c0cda10 2024-04-25T10:28:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f225be00640 1 -- 172.21.15.45:0/1368870247 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22480ab600 con 0x7f225c0cda10 2024-04-25T10:28:07.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f225be00640 1 -- 172.21.15.45:0/1368870247 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480ab9b0 con 0x7f225c0cda10 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 -- 172.21.15.45:0/1368870247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cda10 msgr2=0x7f225c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 --2- 172.21.15.45:0/1368870247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cda10 0x7f225c06c090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f22480097c0 tx=0x7f224801c410 comp rx=0 tx=0).stop 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 -- 172.21.15.45:0/1368870247 shutdown_connections 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 --2- 172.21.15.45:0/1368870247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cda10 0x7f225c06c090 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 -- 172.21.15.45:0/1368870247 >> 172.21.15.45:0/1368870247 conn(0x7f225c066b70 msgr2=0x7f225c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 -- 172.21.15.45:0/1368870247 shutdown_connections 2024-04-25T10:28:07.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 -- 172.21.15.45:0/1368870247 wait complete. 2024-04-25T10:28:07.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 Processor -- start 2024-04-25T10:28:07.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.005+0000 7f2264000640 1 -- start start 2024-04-25T10:28:07.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 0x7f225c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:07.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cf930 0x7f225c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:07.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f225c15bc40 0x7f225c14c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:07.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c169d70 con 0x7f225c0cda10 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c169bf0 con 0x7f225c0cf930 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c169ef0 con 0x7f225c15bc40 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 0x7f225c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 0x7f225c14d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34300/0 (socket says 172.21.15.45:34300) 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 -- 172.21.15.45:0/584167952 learned_addr learned my addr 172.21.15.45:0/584167952 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2260c00640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cf930 0x7f225c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2262000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f225c15bc40 0x7f225c14c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 -- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f225c15bc40 msgr2=0x7f225c14c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f225c15bc40 0x7f225c14c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 -- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cf930 msgr2=0x7f225c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cf930 0x7f225c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 -- 172.21.15.45:0/584167952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c14dd50 con 0x7f225c0cda10 2024-04-25T10:28:07.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2262000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f225c15bc40 0x7f225c14c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2261600640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 0x7f225c14d810 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f225000c4d0 tx=0x7f225000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250017070 con 0x7f225c0cda10 2024-04-25T10:28:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2250005cf0 con 0x7f225c0cda10 2024-04-25T10:28:07.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250007270 con 0x7f225c0cda10 2024-04-25T10:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c1507a0 con 0x7f225c0cda10 2024-04-25T10:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c150d60 con 0x7f225c0cda10 2024-04-25T10:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.009+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225c04ef30 con 0x7f225c0cda10 2024-04-25T10:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.013+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f225001a070 con 0x7f225c0cda10 2024-04-25T10:28:07.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.013+0000 7f2259600640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f223c066f60 0x7f223c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.013+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2250084390 con 0x7f225c0cda10 2024-04-25T10:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.013+0000 7f2260c00640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f223c066f60 0x7f223c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:07.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.013+0000 7f2260c00640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f223c066f60 0x7f223c069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f22480ad000 tx=0x7f22480a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:07.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.017+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2250054d40 con 0x7f225c0cda10 2024-04-25T10:28:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.213+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f225c06d960 con 0x7f225c0cda10 2024-04-25T10:28:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2259600640 1 -- 172.21.15.45:0/584167952 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2250058de0 con 0x7f225c0cda10 2024-04-25T10:28:07.220 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:07.220 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:27:24.321672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9012":{"gid":9012,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1857937743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1857937743},{"type":"v1","addr":"172.21.15.45:6835","nonce":1857937743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 91 2024-04-25T10:28:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f223c066f60 msgr2=0x7f223c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f223c066f60 0x7f223c069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f22480ad000 tx=0x7f22480a6000 comp rx=0 tx=0).stop 2024-04-25T10:28:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 msgr2=0x7f225c14d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 0x7f225c14d810 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f225000c4d0 tx=0x7f225000c990 comp rx=0 tx=0).stop 2024-04-25T10:28:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 shutdown_connections 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f223c066f60 0x7f223c069420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f225c15bc40 0x7f225c14c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f225c0cf930 0x7f225c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 --2- 172.21.15.45:0/584167952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f225c0cda10 0x7f225c14d810 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 >> 172.21.15.45:0/584167952 conn(0x7f225c066b70 msgr2=0x7f225c15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 shutdown_connections 2024-04-25T10:28:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:07.217+0000 7f2264000640 1 -- 172.21.15.45:0/584167952 wait complete. 2024-04-25T10:28:07.241 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:28:12.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:12.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.389+0000 7f680f400640 1 Processor -- start 2024-04-25T10:28:12.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.389+0000 7f680f400640 1 -- start start 2024-04-25T10:28:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f68081550b0 0x7f68080cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f680815b910 0x7f68080b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680f400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6808071050 con 0x7f68080cfe70 2024-04-25T10:28:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680f400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680815bd80 con 0x7f680815b910 2024-04-25T10:28:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680f400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680815d040 con 0x7f68081550b0 2024-04-25T10:28:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f680815b910 0x7f68080b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f680815b910 0x7f68080b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34314/0 (socket says 172.21.15.45:34314) 2024-04-25T10:28:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 -- 172.21.15.45:0/3419230856 learned_addr learned my addr 172.21.15.45:0/3419230856 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680ca00640 1 --2- 172.21.15.45:0/3419230856 >> v2:172.21.15.117:3300/0 conn(0x7f68081550b0 0x7f68080cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 -- 172.21.15.45:0/3419230856 >> v2:172.21.15.117:3300/0 conn(0x7f68081550b0 msgr2=0x7f68080cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 --2- 172.21.15.45:0/3419230856 >> v2:172.21.15.117:3300/0 conn(0x7f68081550b0 0x7f68080cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 -- 172.21.15.45:0/3419230856 >> v1:172.21.15.117:6789/0 conn(0x7f68080cfe70 legacy=0x7f680815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.393+0000 7f680d400640 1 -- 172.21.15.45:0/3419230856 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080712f0 con 0x7f680815b910 2024-04-25T10:28:12.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680d400640 1 --2- 172.21.15.45:0/3419230856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68080b3260 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f6800009e50 tx=0x7f680001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce9c25dba0b1824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:12.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f6807e00640 1 -- 172.21.15.45:0/3419230856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000a57a0 con 0x7f680815b910 2024-04-25T10:28:12.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f6807e00640 1 -- 172.21.15.45:0/3419230856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68000a5940 con 0x7f680815b910 2024-04-25T10:28:12.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f6807e00640 1 -- 172.21.15.45:0/3419230856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000a5d10 con 0x7f680815b910 2024-04-25T10:28:12.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- 172.21.15.45:0/3419230856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 msgr2=0x7f68080b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 --2- 172.21.15.45:0/3419230856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68080b3260 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f6800009e50 tx=0x7f680001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- 172.21.15.45:0/3419230856 shutdown_connections 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 --2- 172.21.15.45:0/3419230856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68080b3260 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 --2- 172.21.15.45:0/3419230856 >> v2:172.21.15.117:3300/0 conn(0x7f68081550b0 0x7f68080cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- 172.21.15.45:0/3419230856 >> 172.21.15.45:0/3419230856 conn(0x7f6808066b70 msgr2=0x7f6808066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- 172.21.15.45:0/3419230856 shutdown_connections 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- 172.21.15.45:0/3419230856 wait complete. 2024-04-25T10:28:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 Processor -- start 2024-04-25T10:28:12.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- start start 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080cfe70 0x7f680814a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68081550b0 0x7f68080afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68081501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680815c030 con 0x7f680815b910 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68080720b0 con 0x7f68081550b0 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.397+0000 7f680f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680815c1b0 con 0x7f68080cfe70 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68081501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68081501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34326/0 (socket says 172.21.15.45:34326) 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 -- 172.21.15.45:0/586261807 learned_addr learned my addr 172.21.15.45:0/586261807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f67ffe00640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68081550b0 0x7f68080afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680ca00640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080cfe70 0x7f680814a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 -- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080cfe70 msgr2=0x7f680814a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080cfe70 0x7f680814a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 -- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68081550b0 msgr2=0x7f68080afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68081550b0 0x7f68080afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 -- 172.21.15.45:0/586261807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080b0410 con 0x7f680815b910 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680ca00640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080cfe70 0x7f680814a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680d400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68081501c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f6800019600 tx=0x7f6800019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000b8670 con 0x7f680815b910 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68000be070 con 0x7f680815b910 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68000b8a50 con 0x7f680815b910 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68080b06f0 con 0x7f680815b910 2024-04-25T10:28:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68080b0cb0 con 0x7f680815b910 2024-04-25T10:28:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680804ef30 con 0x7f680815b910 2024-04-25T10:28:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68000b8bf0 con 0x7f680815b910 2024-04-25T10:28:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.401+0000 7f6805600640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67e4066f60 0x7f67e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:12.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.405+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f68001202b0 con 0x7f680815b910 2024-04-25T10:28:12.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.405+0000 7f680ca00640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67e4066f60 0x7f67e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:12.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.405+0000 7f680ca00640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67e4066f60 0x7f67e4069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009840 tx=0x7f67f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.409+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68000165e0 con 0x7f680815b910 2024-04-25T10:28:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.601+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6808154870 con 0x7f680815b910 2024-04-25T10:28:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f6805600640 1 -- 172.21.15.45:0/586261807 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f68000f1370 con 0x7f680815b910 2024-04-25T10:28:12.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:12.608 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:10.781205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 93 2024-04-25T10:28:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67e4066f60 msgr2=0x7f67e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67e4066f60 0x7f67e4069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009840 tx=0x7f67f8008040 comp rx=0 tx=0).stop 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 msgr2=0x7f68081501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68081501c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f6800019600 tx=0x7f6800019790 comp rx=0 tx=0).stop 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 shutdown_connections 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67e4066f60 0x7f67e4069420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f680815b910 0x7f68081501c0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68081550b0 0x7f68080afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 --2- 172.21.15.45:0/586261807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f68080cfe70 0x7f680814a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 >> 172.21.15.45:0/586261807 conn(0x7f6808066b70 msgr2=0x7f680815db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 shutdown_connections 2024-04-25T10:28:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:12.605+0000 7f680f400640 1 -- 172.21.15.45:0/586261807 wait complete. 2024-04-25T10:28:12.629 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:28:17.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:17.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 Processor -- start 2024-04-25T10:28:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 -- start start 2024-04-25T10:28:17.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe62015df60 0x7fe6200cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62006aea0 0x7fe62006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6200ccaa0 con 0x7fe6200d0180 2024-04-25T10:28:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62015e340 con 0x7fe62015df60 2024-04-25T10:28:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe628400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6200ccd80 con 0x7fe62006aea0 2024-04-25T10:28:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe6200d0180 0x7fe6200d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51680/0 (socket says 172.21.15.45:51680) 2024-04-25T10:28:17.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625000640 1 -- 172.21.15.45:0/571110714 learned_addr learned my addr 172.21.15.45:0/571110714 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625a00640 1 --2- 172.21.15.45:0/571110714 >> v2:172.21.15.117:3300/0 conn(0x7fe62015df60 0x7fe6200cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe626400640 1 --2- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62006aea0 0x7fe62006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe61fe00640 1 -- 172.21.15.45:0/571110714 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862053205 0 0) 0x7fe6200ccaa0 con 0x7fe6200d0180 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe61fe00640 1 -- 172.21.15.45:0/571110714 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe608003530 con 0x7fe6200d0180 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625a00640 1 -- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62006aea0 msgr2=0x7fe62006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625a00640 1 --2- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62006aea0 0x7fe62006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625a00640 1 -- 172.21.15.45:0/571110714 >> v1:172.21.15.45:6789/0 conn(0x7fe6200d0180 legacy=0x7fe6200d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:17.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.781+0000 7fe625a00640 1 -- 172.21.15.45:0/571110714 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6200cd060 con 0x7fe62015df60 2024-04-25T10:28:17.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe625a00640 1 --2- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe6200cfc40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe6140097c0 tx=0x7fe61401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62683dd649c80f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:17.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe61fe00640 1 -- 172.21.15.45:0/571110714 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b0070 con 0x7fe62015df60 2024-04-25T10:28:17.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe61fe00640 1 -- 172.21.15.45:0/571110714 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6140ab600 con 0x7fe62015df60 2024-04-25T10:28:17.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe61fe00640 1 -- 172.21.15.45:0/571110714 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140ab9b0 con 0x7fe62015df60 2024-04-25T10:28:17.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 -- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 msgr2=0x7fe6200cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:17.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 --2- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe6200cfc40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe6140097c0 tx=0x7fe61401c410 comp rx=0 tx=0).stop 2024-04-25T10:28:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 -- 172.21.15.45:0/571110714 shutdown_connections 2024-04-25T10:28:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 --2- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62006aea0 0x7fe62006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 --2- 172.21.15.45:0/571110714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe6200cfc40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 -- 172.21.15.45:0/571110714 >> 172.21.15.45:0/571110714 conn(0x7fe6200d1910 msgr2=0x7fe6200d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 -- 172.21.15.45:0/571110714 shutdown_connections 2024-04-25T10:28:17.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.785+0000 7fe628400640 1 -- 172.21.15.45:0/571110714 wait complete. 2024-04-25T10:28:17.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 Processor -- start 2024-04-25T10:28:17.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 -- start start 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 0x7fe620154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6200d0180 0x7fe6201582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe620159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6200d0ff0 con 0x7fe62006aea0 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6200d0e70 con 0x7fe62015df60 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62016aa20 con 0x7fe6200d0180 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 0x7fe620154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 0x7fe620154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54170/0 (socket says 172.21.15.45:54170) 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 -- 172.21.15.45:0/2803325488 learned_addr learned my addr 172.21.15.45:0/2803325488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625000640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6200d0180 0x7fe6201582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe626400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe620159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:17.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 -- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6200d0180 msgr2=0x7fe6201582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6200d0180 0x7fe6201582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 -- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 msgr2=0x7fe620159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe620159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 -- 172.21.15.45:0/2803325488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620158b40 con 0x7fe62006aea0 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625000640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6200d0180 0x7fe6201582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe626400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe620159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:17.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe625a00640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 0x7fe620154130 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fe6140ae040 tx=0x7fe61401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b0050 con 0x7fe62006aea0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6140b1040 con 0x7fe62006aea0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614006e20 con 0x7fe62006aea0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe620158e20 con 0x7fe62006aea0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.789+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6200ca3b0 con 0x7fe62006aea0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.793+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62004ef30 con 0x7fe62006aea0 2024-04-25T10:28:17.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.793+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6140be030 con 0x7fe62006aea0 2024-04-25T10:28:17.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.793+0000 7fe61d600640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.793+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe61411ee30 con 0x7fe62006aea0 2024-04-25T10:28:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.793+0000 7fe625000640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:17.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.793+0000 7fe625000640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe6200cd9d0 tx=0x7fe60c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:17.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:17.797+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6140c5020 con 0x7fe62006aea0 2024-04-25T10:28:17.883 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-04-25T10:28:17.877+0000 7fdec2200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:28:18.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.001+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6200cba50 con 0x7fe62006aea0 2024-04-25T10:28:18.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.001+0000 7fe61d600640 1 -- 172.21.15.45:0/2803325488 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fe614016090 con 0x7fe62006aea0 2024-04-25T10:28:18.006 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:18.007 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:12.703341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:18.007 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2024-04-25T10:28:18.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 msgr2=0x7fe600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe6200cd9d0 tx=0x7fe60c009040 comp rx=0 tx=0).stop 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 msgr2=0x7fe620154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 0x7fe620154130 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fe6140ae040 tx=0x7fe61401ac50 comp rx=0 tx=0).stop 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 shutdown_connections 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe600066f60 0x7fe600069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62015df60 0x7fe620159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6200d0180 0x7fe6201582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 --2- 172.21.15.45:0/2803325488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe62006aea0 0x7fe620154130 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 >> 172.21.15.45:0/2803325488 conn(0x7fe6200d1910 msgr2=0x7fe62006fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 shutdown_connections 2024-04-25T10:28:18.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.005+0000 7fe628400640 1 -- 172.21.15.45:0/2803325488 wait complete. 2024-04-25T10:28:18.029 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T10:28:18.030 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.165+0000 7f59eb600640 1 Processor -- start 2024-04-25T10:28:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.165+0000 7f59eb600640 1 -- start start 2024-04-25T10:28:18.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59eb600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59e40b0f70 0x7f59e40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59eb600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f59e40cfd90 0x7f59e40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59eb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40cd110 0x7f59e40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59eb600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e415ef20 con 0x7f59e40b0f70 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59eb600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e415f3b0 con 0x7f59e40cfd90 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59eb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40ca810 con 0x7f59e40cd110 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59e40b0f70 0x7f59e40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f59e40b0f70 0x7f59e40b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54180/0 (socket says 172.21.15.45:54180) 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 -- 172.21.15.45:0/2139443533 learned_addr learned my addr 172.21.15.45:0/2139443533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e9600640 1 --2- 172.21.15.45:0/2139443533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40cd110 0x7f59e40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59dbe00640 1 --2- 172.21.15.45:0/2139443533 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cfd90 0x7f59e40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 -- 172.21.15.45:0/2139443533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40cd110 msgr2=0x7f59e40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 --2- 172.21.15.45:0/2139443533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40cd110 0x7f59e40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 -- 172.21.15.45:0/2139443533 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cfd90 msgr2=0x7f59e40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 --2- 172.21.15.45:0/2139443533 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cfd90 0x7f59e40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 -- 172.21.15.45:0/2139443533 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40b1890 con 0x7f59e40b0f70 2024-04-25T10:28:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59dbe00640 1 --2- 172.21.15.45:0/2139443533 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cfd90 0x7f59e40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.169+0000 7f59e8c00640 1 --2- 172.21.15.45:0/2139443533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40b0f70 0x7f59e40b1350 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f59e4071c10 tx=0x7f59d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52645cb0ac7277a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e3e00640 1 -- 172.21.15.45:0/2139443533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b0070 con 0x7f59e40b0f70 2024-04-25T10:28:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e3e00640 1 -- 172.21.15.45:0/2139443533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40ab600 con 0x7f59e40b0f70 2024-04-25T10:28:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e3e00640 1 -- 172.21.15.45:0/2139443533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40ab9d0 con 0x7f59e40b0f70 2024-04-25T10:28:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- 172.21.15.45:0/2139443533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40b0f70 msgr2=0x7f59e40b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- 172.21.15.45:0/2139443533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40b0f70 0x7f59e40b1350 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f59e4071c10 tx=0x7f59d401c520 comp rx=0 tx=0).stop 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- 172.21.15.45:0/2139443533 shutdown_connections 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- 172.21.15.45:0/2139443533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40cd110 0x7f59e40cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- 172.21.15.45:0/2139443533 >> v2:172.21.15.117:3300/0 conn(0x7f59e40cfd90 0x7f59e40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- 172.21.15.45:0/2139443533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40b0f70 0x7f59e40b1350 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- 172.21.15.45:0/2139443533 >> 172.21.15.45:0/2139443533 conn(0x7f59e40b1d50 msgr2=0x7f59e40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- 172.21.15.45:0/2139443533 shutdown_connections 2024-04-25T10:28:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- 172.21.15.45:0/2139443533 wait complete. 2024-04-25T10:28:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 Processor -- start 2024-04-25T10:28:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- start start 2024-04-25T10:28:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40b0f70 0x7f59e40ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cd110 0x7f59e40c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 0x7f59e40c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4072200 con 0x7f59e40cfd90 2024-04-25T10:28:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40cd520 con 0x7f59e40cd110 2024-04-25T10:28:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59eb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4072380 con 0x7f59e40b0f70 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 0x7f59e40c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 0x7f59e40c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54192/0 (socket says 172.21.15.45:54192) 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e9600640 1 -- 172.21.15.45:0/1725995397 learned_addr learned my addr 172.21.15.45:0/1725995397 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59dbe00640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cd110 0x7f59e40c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.173+0000 7f59e8c00640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40b0f70 0x7f59e40ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e9600640 1 -- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40b0f70 msgr2=0x7f59e40ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e9600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40b0f70 0x7f59e40ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e9600640 1 -- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cd110 msgr2=0x7f59e40c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e9600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cd110 0x7f59e40c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e9600640 1 -- 172.21.15.45:0/1725995397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40ca180 con 0x7f59e40cfd90 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e9600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 0x7f59e40c5f20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00ef00 tx=0x7f59dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc018020 con 0x7f59e40cfd90 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59dc014400 con 0x7f59e40cfd90 2024-04-25T10:28:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e40afd60 con 0x7f59e40cfd90 2024-04-25T10:28:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0147f0 con 0x7f59e40cfd90 2024-04-25T10:28:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e40c5a00 con 0x7f59e40cfd90 2024-04-25T10:28:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e40aff30 con 0x7f59e40cfd90 2024-04-25T10:28:18.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59dc01b020 con 0x7f59e40cfd90 2024-04-25T10:28:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e1600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f59dc085940 con 0x7f59e40cfd90 2024-04-25T10:28:18.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.177+0000 7f59e8c00640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.181+0000 7f59e8c00640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ad000 tx=0x7f59d40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:18.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.181+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59dc022030 con 0x7f59e40cfd90 2024-04-25T10:28:18.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.377+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e404ef30 con 0x7f59e40cfd90 2024-04-25T10:28:18.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.377+0000 7f59e1600640 1 -- 172.21.15.45:0/1725995397 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f59dc056200 con 0x7f59e40cfd90 2024-04-25T10:28:18.383 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:18.383 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:12.703341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:18.384 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2024-04-25T10:28:18.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 msgr2=0x7f59c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ad000 tx=0x7f59d40a6000 comp rx=0 tx=0).stop 2024-04-25T10:28:18.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 msgr2=0x7f59e40c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 0x7f59e40c5f20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00ef00 tx=0x7f59dc00b480 comp rx=0 tx=0).stop 2024-04-25T10:28:18.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 shutdown_connections 2024-04-25T10:28:18.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f59c0066f60 0x7f59c0069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e40cfd90 0x7f59e40c5f20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e40cd110 0x7f59e40c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 --2- 172.21.15.45:0/1725995397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59e40b0f70 0x7f59e40ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 >> 172.21.15.45:0/1725995397 conn(0x7f59e40b1d50 msgr2=0x7f59e406d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:18.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 shutdown_connections 2024-04-25T10:28:18.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.381+0000 7f59eb600640 1 -- 172.21.15.45:0/1725995397 wait complete. 2024-04-25T10:28:18.405 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:28:18.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:28:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19ebc00640 1 Processor -- start 2024-04-25T10:28:18.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19ebc00640 1 -- start start 2024-04-25T10:28:18.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19ebc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f19e40d2490 0x7f19e40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19ebc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e40d06a0 con 0x7f19e40ce830 2024-04-25T10:28:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19ebc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e40b4960 con 0x7f19e415bc00 2024-04-25T10:28:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19ebc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e40ceca0 con 0x7f19e40d2490 2024-04-25T10:28:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19e9c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f19e40ce830 0x7f19e40b5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51696/0 (socket says 172.21.15.45:51696) 2024-04-25T10:28:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19e9c00640 1 -- 172.21.15.45:0/873562551 learned_addr learned my addr 172.21.15.45:0/873562551 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:18.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19dbe00640 1 --2- 172.21.15.45:0/873562551 >> v2:172.21.15.117:3300/0 conn(0x7f19e40d2490 0x7f19e40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165438929 0 0) 0x7f19e40d06a0 con 0x7f19e40ce830 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c8003530 con 0x7f19e40ce830 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937032998 0 0) 0x7f19e40b4960 con 0x7f19e415bc00 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e40d06a0 con 0x7f19e415bc00 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19dbe00640 1 -- 172.21.15.45:0/873562551 >> v1:172.21.15.117:6789/0 conn(0x7f19e415bc00 legacy=0x7f19e40cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19dbe00640 1 -- 172.21.15.45:0/873562551 >> v1:172.21.15.45:6789/0 conn(0x7f19e40ce830 legacy=0x7f19e40b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.541+0000 7f19dbe00640 1 -- 172.21.15.45:0/873562551 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e40b4c40 con 0x7f19e40d2490 2024-04-25T10:28:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19dbe00640 1 --2- 172.21.15.45:0/873562551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40d2490 0x7f19e40ce2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097c0 tx=0x7f19cc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef29af3976d0e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0b0070 con 0x7f19e40d2490 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc0ab600 con 0x7f19e40d2490 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19e3e00640 1 -- 172.21.15.45:0/873562551 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0ab9d0 con 0x7f19e40d2490 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 -- 172.21.15.45:0/873562551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40d2490 msgr2=0x7f19e40ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 --2- 172.21.15.45:0/873562551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40d2490 0x7f19e40ce2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097c0 tx=0x7f19cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 -- 172.21.15.45:0/873562551 shutdown_connections 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 --2- 172.21.15.45:0/873562551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40d2490 0x7f19e40ce2f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 -- 172.21.15.45:0/873562551 >> 172.21.15.45:0/873562551 conn(0x7f19e40d1900 msgr2=0x7f19e40d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 -- 172.21.15.45:0/873562551 shutdown_connections 2024-04-25T10:28:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.545+0000 7f19ebc00640 1 -- 172.21.15.45:0/873562551 wait complete. 2024-04-25T10:28:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 Processor -- start 2024-04-25T10:28:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 -- start start 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40ce830 0x7f19e40c5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 0x7f19e4156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19e415bc00 0x7f19e4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40b6730 con 0x7f19e40d2490 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e40b65b0 con 0x7f19e40ce830 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e41698b0 con 0x7f19e415bc00 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 0x7f19e4156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 0x7f19e4156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54194/0 (socket says 172.21.15.45:54194) 2024-04-25T10:28:18.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 -- 172.21.15.45:0/107942900 learned_addr learned my addr 172.21.15.45:0/107942900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19e9200640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40ce830 0x7f19e40c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19e9c00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19e415bc00 0x7f19e4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 -- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19e415bc00 msgr2=0x7f19e4156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19e415bc00 0x7f19e4156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 -- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40ce830 msgr2=0x7f19e40c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40ce830 0x7f19e40c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 -- 172.21.15.45:0/107942900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e40c5c60 con 0x7f19e40d2490 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19e9c00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19e415bc00 0x7f19e4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19dbe00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 0x7f19e4156800 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0190b0 tx=0x7f19cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0b8020 con 0x7f19e40d2490 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc0b4410 con 0x7f19e40d2490 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0b0040 con 0x7f19e40d2490 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e40c5f40 con 0x7f19e40d2490 2024-04-25T10:28:18.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.549+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e40c6500 con 0x7f19e40d2490 2024-04-25T10:28:18.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.553+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e404ef30 con 0x7f19e40d2490 2024-04-25T10:28:18.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.553+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19cc0bb020 con 0x7f19e40d2490 2024-04-25T10:28:18.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.553+0000 7f19e1600640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19c0066f60 0x7f19c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:18.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.553+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f19cc11eb50 con 0x7f19e40d2490 2024-04-25T10:28:18.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.553+0000 7f19e9200640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19c0066f60 0x7f19c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:18.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.557+0000 7f19e9200640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19c0066f60 0x7f19c0069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f19d40011f0 tx=0x7f19d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:18.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.557+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19cc0c2030 con 0x7f19e40d2490 2024-04-25T10:28:18.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.781+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f19e40b7690 con 0x7f19e40d2490 2024-04-25T10:28:18.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.781+0000 7f19e1600640 1 -- 172.21.15.45:0/107942900 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1467 (secure 0 0 0) 0x7f19cc016090 con 0x7f19e40d2490 2024-04-25T10:28:18.787 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:18.787 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":642,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":131,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076944896,"bytes_used":2607898624,"bytes_avail":770486214656,"bytes_total":773094113280,"read_bytes_sec":236482,"write_bytes_sec":59683,"read_op_per_sec":7,"write_op_per_sec":29},"fsmap":{"epoch":95,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9009}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:28:11.017546+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19c0066f60 msgr2=0x7f19c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19c0066f60 0x7f19c0069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f19d40011f0 tx=0x7f19d4009040 comp rx=0 tx=0).stop 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 msgr2=0x7f19e4156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 0x7f19e4156800 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0190b0 tx=0x7f19cc019240 comp rx=0 tx=0).stop 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 shutdown_connections 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19c0066f60 0x7f19c0069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19e415bc00 0x7f19e4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19e40d2490 0x7f19e4156800 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 --2- 172.21.15.45:0/107942900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19e40ce830 0x7f19e40c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 >> 172.21.15.45:0/107942900 conn(0x7f19e40d1900 msgr2=0x7f19e40b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 shutdown_connections 2024-04-25T10:28:18.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:18.785+0000 7f19ebc00640 1 -- 172.21.15.45:0/107942900 wait complete. 2024-04-25T10:28:18.807 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:28:18.807 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:28:18.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:28:18.890 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:28:18.990 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:28:18.990 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:28:18.990 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:18.999 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:28:18.999 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:19.052 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:19.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.229+0000 7f1465400640 1 Processor -- start 2024-04-25T10:28:19.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.229+0000 7f1465400640 1 -- start start 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f1465400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f146015b910 0x7f146015c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f1465400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1460071050 con 0x7f14601550b0 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f1465400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14600712f0 con 0x7f14600d0040 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f1465400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146015bda0 con 0x7f146015b910 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f14601550b0 0x7f14600cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51704/0 (socket says 172.21.15.45:51704) 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145e800640 1 -- 172.21.15.45:0/1126281884 learned_addr learned my addr 172.21.15.45:0/1126281884 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:19.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145f200640 1 --2- 172.21.15.45:0/1126281884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f146015b910 0x7f146015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944729790 0 0) 0x7f1460071050 con 0x7f14601550b0 2024-04-25T10:28:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1444003530 con 0x7f14601550b0 2024-04-25T10:28:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685867641 0 0) 0x7f14600712f0 con 0x7f14600d0040 2024-04-25T10:28:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1460071050 con 0x7f14600d0040 2024-04-25T10:28:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145f200640 1 -- 172.21.15.45:0/1126281884 >> v1:172.21.15.117:6789/0 conn(0x7f14600d0040 legacy=0x7f146015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145f200640 1 -- 172.21.15.45:0/1126281884 >> v1:172.21.15.45:6789/0 conn(0x7f14601550b0 legacy=0x7f14600cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.233+0000 7f145f200640 1 -- 172.21.15.45:0/1126281884 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600715d0 con 0x7f146015b910 2024-04-25T10:28:19.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f145f200640 1 --2- 172.21.15.45:0/1126281884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f146015b910 0x7f146015c030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1458009dc0 tx=0x7f145801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d4fe6367be827a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580b2070 con 0x7f146015b910 2024-04-25T10:28:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580a5a20 con 0x7f146015b910 2024-04-25T10:28:19.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f145de00640 1 -- 172.21.15.45:0/1126281884 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580a5dd0 con 0x7f146015b910 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- 172.21.15.45:0/1126281884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f146015b910 msgr2=0x7f146015c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 --2- 172.21.15.45:0/1126281884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f146015b910 0x7f146015c030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1458009dc0 tx=0x7f145801c960 comp rx=0 tx=0).stop 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- 172.21.15.45:0/1126281884 shutdown_connections 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 --2- 172.21.15.45:0/1126281884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f146015b910 0x7f146015c030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- 172.21.15.45:0/1126281884 >> 172.21.15.45:0/1126281884 conn(0x7f1460066b70 msgr2=0x7f1460066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- 172.21.15.45:0/1126281884 shutdown_connections 2024-04-25T10:28:19.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- 172.21.15.45:0/1126281884 wait complete. 2024-04-25T10:28:19.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 Processor -- start 2024-04-25T10:28:19.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- start start 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14600d0040 0x7f146014e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14601550b0 0x7f1460151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 0x7f1460159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146015cf00 con 0x7f146015b910 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146015cd80 con 0x7f14600d0040 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.237+0000 7f1465400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146015d080 con 0x7f14601550b0 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 0x7f1460159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 0x7f1460159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54204/0 (socket says 172.21.15.45:54204) 2024-04-25T10:28:19.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 -- 172.21.15.45:0/2858383188 learned_addr learned my addr 172.21.15.45:0/2858383188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145e800640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14600d0040 0x7f146014e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1457e00640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14601550b0 0x7f1460151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 -- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14601550b0 msgr2=0x7f1460151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14601550b0 0x7f1460151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 -- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14600d0040 msgr2=0x7f146014e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14600d0040 0x7f146014e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 -- 172.21.15.45:0/2858383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146014ee00 con 0x7f146015b910 2024-04-25T10:28:19.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f145f200640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 0x7f1460159a50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1458019570 tx=0x7f1458019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580b2050 con 0x7f146015b910 2024-04-25T10:28:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580be070 con 0x7f146015b910 2024-04-25T10:28:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14600ce1d0 con 0x7f146015b910 2024-04-25T10:28:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458004040 con 0x7f146015b910 2024-04-25T10:28:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14600ce790 con 0x7f146015b910 2024-04-25T10:28:19.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146004ef30 con 0x7f146015b910 2024-04-25T10:28:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14580041e0 con 0x7f146015b910 2024-04-25T10:28:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1456a00640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f143c066f60 0x7f143c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.241+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1458120310 con 0x7f146015b910 2024-04-25T10:28:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.245+0000 7f145e800640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f143c066f60 0x7f143c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.245+0000 7f145e800640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f143c066f60 0x7f143c069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1450000f70 tx=0x7f1450009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.245+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1458016580 con 0x7f146015b910 2024-04-25T10:28:19.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.409+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14600671a0 con 0x7f146015b910 2024-04-25T10:28:19.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.409+0000 7f1456a00640 1 -- 172.21.15.45:0/2858383188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f14580f12b0 con 0x7f146015b910 2024-04-25T10:28:19.415 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:19.415 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:10.677542+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1857937743":"2024-04-26T10:28:10.677492+0000","172.21.15.45:6833/1857937743":"2024-04-26T10:28:10.677492+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f143c066f60 msgr2=0x7f143c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f143c066f60 0x7f143c069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1450000f70 tx=0x7f1450009040 comp rx=0 tx=0).stop 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 msgr2=0x7f1460159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 0x7f1460159a50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1458019570 tx=0x7f1458019700 comp rx=0 tx=0).stop 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 shutdown_connections 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f143c066f60 0x7f143c069420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f146015b910 0x7f1460159a50 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14601550b0 0x7f1460151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 --2- 172.21.15.45:0/2858383188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14600d0040 0x7f146014e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 >> 172.21.15.45:0/2858383188 conn(0x7f1460066b70 msgr2=0x7f14600ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:19.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 shutdown_connections 2024-04-25T10:28:19.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.413+0000 7f1465400640 1 -- 172.21.15.45:0/2858383188 wait complete. 2024-04-25T10:28:19.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:27:19.715814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:27:20.287107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:28:19.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:28:19.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 Processor -- start 2024-04-25T10:28:19.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 -- start start 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f295406bd10 0x7f2954161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2954161e50 0x7f295415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295406ac90 con 0x7f29540cd9a0 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295406b880 con 0x7f295406bd10 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f295a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295406af50 con 0x7f2954161e50 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f29540cd9a0 0x7f29540d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36224/0 (socket says 172.21.15.45:36224) 2024-04-25T10:28:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953400640 1 -- 172.21.15.45:0/328716325 learned_addr learned my addr 172.21.15.45:0/328716325 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:19.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953e00640 1 --2- 172.21.15.45:0/328716325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2954161e50 0x7f295415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f294aa00640 1 --2- 172.21.15.45:0/328716325 >> v2:172.21.15.117:3300/0 conn(0x7f295406bd10 0x7f2954161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2952a00640 1 -- 172.21.15.45:0/328716325 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087364480 0 0) 0x7f295406ac90 con 0x7f29540cd9a0 2024-04-25T10:28:19.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2952a00640 1 -- 172.21.15.45:0/328716325 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2938003530 con 0x7f29540cd9a0 2024-04-25T10:28:19.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953e00640 1 -- 172.21.15.45:0/328716325 >> v2:172.21.15.117:3300/0 conn(0x7f295406bd10 msgr2=0x7f2954161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:19.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953e00640 1 --2- 172.21.15.45:0/328716325 >> v2:172.21.15.117:3300/0 conn(0x7f295406bd10 0x7f2954161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953e00640 1 -- 172.21.15.45:0/328716325 >> v1:172.21.15.117:6789/0 conn(0x7f29540cd9a0 legacy=0x7f29540d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f2953e00640 1 -- 172.21.15.45:0/328716325 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295406b230 con 0x7f2954161e50 2024-04-25T10:28:19.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.577+0000 7f294aa00640 1 --2- 172.21.15.45:0/328716325 >> v2:172.21.15.117:3300/0 conn(0x7f295406bd10 0x7f2954161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f2953e00640 1 --2- 172.21.15.45:0/328716325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2954161e50 0x7f295415d610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f294c009d90 tx=0x7f294c01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e9c4edf75c0f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f2952a00640 1 -- 172.21.15.45:0/328716325 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0b4070 con 0x7f2954161e50 2024-04-25T10:28:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f2952a00640 1 -- 172.21.15.45:0/328716325 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c0af600 con 0x7f2954161e50 2024-04-25T10:28:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f2952a00640 1 -- 172.21.15.45:0/328716325 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0af9d0 con 0x7f2954161e50 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 -- 172.21.15.45:0/328716325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2954161e50 msgr2=0x7f295415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 --2- 172.21.15.45:0/328716325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2954161e50 0x7f295415d610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f294c009d90 tx=0x7f294c01f150 comp rx=0 tx=0).stop 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 -- 172.21.15.45:0/328716325 shutdown_connections 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 --2- 172.21.15.45:0/328716325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2954161e50 0x7f295415d610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 --2- 172.21.15.45:0/328716325 >> v2:172.21.15.117:3300/0 conn(0x7f295406bd10 0x7f2954161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 -- 172.21.15.45:0/328716325 >> 172.21.15.45:0/328716325 conn(0x7f29540d19a0 msgr2=0x7f29540d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 -- 172.21.15.45:0/328716325 shutdown_connections 2024-04-25T10:28:19.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.581+0000 7f295a800640 1 -- 172.21.15.45:0/328716325 wait complete. 2024-04-25T10:28:19.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 Processor -- start 2024-04-25T10:28:19.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 -- start start 2024-04-25T10:28:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f295406bd10 0x7f295409ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 0x7f2954152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954161e50 0x7f295409c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2954162660 con 0x7f29540cd9a0 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29540cdd80 con 0x7f2954161e50 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29540d0d70 con 0x7f295406bd10 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 0x7f2954152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 0x7f2954152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54208/0 (socket says 172.21.15.45:54208) 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 -- 172.21.15.45:0/1899160766 learned_addr learned my addr 172.21.15.45:0/1899160766 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f2953400640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f295406bd10 0x7f295409ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f2953e00640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954161e50 0x7f295409c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 -- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f295406bd10 msgr2=0x7f295409ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f295406bd10 0x7f295409ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 -- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954161e50 msgr2=0x7f295409c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954161e50 0x7f295409c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 -- 172.21.15.45:0/1899160766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295409f460 con 0x7f29540cd9a0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f2953e00640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954161e50 0x7f295409c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294aa00640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 0x7f2954152c80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9b0 tx=0x7f293c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c017070 con 0x7f29540cd9a0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c00ca00 con 0x7f29540cd9a0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00cdd0 con 0x7f29540cd9a0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295409f720 con 0x7f29540cd9a0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.585+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29540c78b0 con 0x7f29540cd9a0 2024-04-25T10:28:19.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.589+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295404ef30 con 0x7f29540cd9a0 2024-04-25T10:28:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.589+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f293c01a070 con 0x7f29540cd9a0 2024-04-25T10:28:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.589+0000 7f294be00640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2930066f60 0x7f2930069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.589+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f293c084f50 con 0x7f29540cd9a0 2024-04-25T10:28:19.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.589+0000 7f2953400640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2930066f60 0x7f2930069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.589+0000 7f2953400640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2930066f60 0x7f2930069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2944005890 tx=0x7f2944009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.593+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f293c055ef0 con 0x7f29540cd9a0 2024-04-25T10:28:19.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29540d0350 con 0x7f29540cd9a0 2024-04-25T10:28:19.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f294be00640 1 -- 172.21.15.45:0/1899160766 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f293c0557f0 con 0x7f29540cd9a0 2024-04-25T10:28:19.761 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:28:19.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2930066f60 msgr2=0x7f2930069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2930066f60 0x7f2930069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2944005890 tx=0x7f2944009040 comp rx=0 tx=0).stop 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 msgr2=0x7f2954152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 0x7f2954152c80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9b0 tx=0x7f293c00ee70 comp rx=0 tx=0).stop 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 shutdown_connections 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2930066f60 0x7f2930069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2954161e50 0x7f295409c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29540cd9a0 0x7f2954152c80 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 --2- 172.21.15.45:0/1899160766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f295406bd10 0x7f295409ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 >> 172.21.15.45:0/1899160766 conn(0x7f29540d19a0 msgr2=0x7f29540b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.757+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 shutdown_connections 2024-04-25T10:28:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.761+0000 7f295a800640 1 -- 172.21.15.45:0/1899160766 wait complete. 2024-04-25T10:28:19.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:28:19.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.921+0000 7f71c0e00640 1 Processor -- start 2024-04-25T10:28:19.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.921+0000 7f71c0e00640 1 -- start start 2024-04-25T10:28:19.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71c0e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71bc0cbc30 0x7f71bc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71c0e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f71bc0b44c0 0x7f71bc0b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71c0e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc0b2460 con 0x7f71bc15b750 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71c0e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc0b4920 con 0x7f71bc0cbc30 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71c0e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc0b2d70 con 0x7f71bc0b44c0 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71b2a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71bc0cbc30 0x7f71bc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71bb400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f71bc15b750 0x7f71bc0cd590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51706/0 (socket says 172.21.15.45:51706) 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71bb400640 1 -- 172.21.15.45:0/4008444407 learned_addr learned my addr 172.21.15.45:0/4008444407 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71bbe00640 1 --2- 172.21.15.45:0/4008444407 >> v2:172.21.15.117:3300/0 conn(0x7f71bc0b44c0 0x7f71bc0b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71b2a00640 1 -- 172.21.15.45:0/4008444407 >> v2:172.21.15.117:3300/0 conn(0x7f71bc0b44c0 msgr2=0x7f71bc0b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71b2a00640 1 --2- 172.21.15.45:0/4008444407 >> v2:172.21.15.117:3300/0 conn(0x7f71bc0b44c0 0x7f71bc0b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71b2a00640 1 -- 172.21.15.45:0/4008444407 >> v1:172.21.15.45:6789/0 conn(0x7f71bc15b750 legacy=0x7f71bc0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71b2a00640 1 -- 172.21.15.45:0/4008444407 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc070fb0 con 0x7f71bc0cbc30 2024-04-25T10:28:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.925+0000 7f71baa00640 1 -- 172.21.15.45:0/4008444407 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568563957 0 0) 0x7f71bc0b2460 con 0x7f71bc15b750 2024-04-25T10:28:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71b2a00640 1 --2- 172.21.15.45:0/4008444407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc0b3f80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b2ef0 tx=0x7f71a401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9cea6f0555f582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71baa00640 1 -- 172.21.15.45:0/4008444407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40b0070 con 0x7f71bc0cbc30 2024-04-25T10:28:19.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71baa00640 1 -- 172.21.15.45:0/4008444407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4004230 con 0x7f71bc0cbc30 2024-04-25T10:28:19.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71baa00640 1 -- 172.21.15.45:0/4008444407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003620 con 0x7f71bc0cbc30 2024-04-25T10:28:19.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- 172.21.15.45:0/4008444407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 msgr2=0x7f71bc0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 --2- 172.21.15.45:0/4008444407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc0b3f80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0b2ef0 tx=0x7f71a401c460 comp rx=0 tx=0).stop 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- 172.21.15.45:0/4008444407 shutdown_connections 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 --2- 172.21.15.45:0/4008444407 >> v2:172.21.15.117:3300/0 conn(0x7f71bc0b44c0 0x7f71bc0b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 --2- 172.21.15.45:0/4008444407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc0b3f80 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- 172.21.15.45:0/4008444407 >> 172.21.15.45:0/4008444407 conn(0x7f71bc0743f0 msgr2=0x7f71bc0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- 172.21.15.45:0/4008444407 shutdown_connections 2024-04-25T10:28:19.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- 172.21.15.45:0/4008444407 wait complete. 2024-04-25T10:28:19.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 Processor -- start 2024-04-25T10:28:19.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- start start 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71bc0b44c0 0x7f71bc0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc14c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71bc15b750 0x7f71bc14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0b3b50 con 0x7f71bc0cbc30 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0cdca0 con 0x7f71bc0b44c0 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.929+0000 7f71c0e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc15bb30 con 0x7f71bc15b750 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc14c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc14c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54222/0 (socket says 172.21.15.45:54222) 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 -- 172.21.15.45:0/2246573499 learned_addr learned my addr 172.21.15.45:0/2246573499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71bb400640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71bc0b44c0 0x7f71bc0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71bbe00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71bc15b750 0x7f71bc14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 -- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71bc15b750 msgr2=0x7f71bc14b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71bc15b750 0x7f71bc14b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 -- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71bc0b44c0 msgr2=0x7f71bc0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71bc0b44c0 0x7f71bc0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 -- 172.21.15.45:0/2246573499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc14ce20 con 0x7f71bc0cbc30 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71bbe00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71bc15b750 0x7f71bc14b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:19.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b2a00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc14c8e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f71a401a090 tx=0x7f71a4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40b9070 con 0x7f71bc0cbc30 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4003db0 con 0x7f71bc0cbc30 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40b0040 con 0x7f71bc0cbc30 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc14d0c0 con 0x7f71bc0cbc30 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc14d680 con 0x7f71bc0cbc30 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc075120 con 0x7f71bc0cbc30 2024-04-25T10:28:19.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71a40bc070 con 0x7f71bc0cbc30 2024-04-25T10:28:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b3e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7198066f60 0x7f7198069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.933+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f71a411f210 con 0x7f71bc0cbc30 2024-04-25T10:28:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.937+0000 7f71bb400640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7198066f60 0x7f7198069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:19.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.937+0000 7f71bb400640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7198066f60 0x7f7198069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f71ac000f70 tx=0x7f71ac007df0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:19.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:19.937+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71a4016050 con 0x7f71bc0cbc30 2024-04-25T10:28:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.101+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71bc071990 con 0x7f71bc0cbc30 2024-04-25T10:28:20.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.101+0000 7f71b3e00640 1 -- 172.21.15.45:0/2246573499 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7f71a40efc80 con 0x7f71bc0cbc30 2024-04-25T10:28:20.106 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.101+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7198066f60 msgr2=0x7f7198069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.101+0000 7f71c0e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7198066f60 0x7f7198069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f71ac000f70 tx=0x7f71ac007df0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.101+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 msgr2=0x7f71bc14c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.101+0000 7f71c0e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc14c8e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f71a401a090 tx=0x7f71a4019200 comp rx=0 tx=0).stop 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 shutdown_connections 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7198066f60 0x7f7198069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71bc15b750 0x7f71bc14b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71bc0cbc30 0x7f71bc14c8e0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 --2- 172.21.15.45:0/2246573499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71bc0b44c0 0x7f71bc0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 >> 172.21.15.45:0/2246573499 conn(0x7f71bc0743f0 msgr2=0x7f71bc154620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 shutdown_connections 2024-04-25T10:28:20.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.105+0000 7f71c0e00640 1 -- 172.21.15.45:0/2246573499 wait complete. 2024-04-25T10:28:20.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:28:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 Processor -- start 2024-04-25T10:28:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 -- start start 2024-04-25T10:28:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd41406bd10 0x7fd414161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd414161e50 0x7fd41415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41406ac90 con 0x7fd4140cd9a0 2024-04-25T10:28:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41406b880 con 0x7fd41406bd10 2024-04-25T10:28:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41406af50 con 0x7fd414161e50 2024-04-25T10:28:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd41406bd10 0x7fd414161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd41406bd10 0x7fd414161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54234/0 (socket says 172.21.15.45:54234) 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 -- 172.21.15.45:0/228803642 learned_addr learned my addr 172.21.15.45:0/228803642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd413e00640 1 --2- 172.21.15.45:0/228803642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd414161e50 0x7fd41415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 -- 172.21.15.45:0/228803642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd414161e50 msgr2=0x7fd41415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 --2- 172.21.15.45:0/228803642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd414161e50 0x7fd41415d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 -- 172.21.15.45:0/228803642 >> v1:172.21.15.45:6789/0 conn(0x7fd4140cd9a0 legacy=0x7fd4140d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 -- 172.21.15.45:0/228803642 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41406b230 con 0x7fd41406bd10 2024-04-25T10:28:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd40be00640 1 --2- 172.21.15.45:0/228803642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41406bd10 0x7fd414161910 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009870 tx=0x7fd3fc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdf215ca4950de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd412a00640 1 -- 172.21.15.45:0/228803642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0b5070 con 0x7fd41406bd10 2024-04-25T10:28:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd412a00640 1 -- 172.21.15.45:0/228803642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0a8a50 con 0x7fd41406bd10 2024-04-25T10:28:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd412a00640 1 -- 172.21.15.45:0/228803642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0a8e20 con 0x7fd41406bd10 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 -- 172.21.15.45:0/228803642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41406bd10 msgr2=0x7fd414161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.269+0000 7fd41a400640 1 --2- 172.21.15.45:0/228803642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41406bd10 0x7fd414161910 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009870 tx=0x7fd3fc01ed50 comp rx=0 tx=0).stop 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- 172.21.15.45:0/228803642 shutdown_connections 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 --2- 172.21.15.45:0/228803642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd414161e50 0x7fd41415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 --2- 172.21.15.45:0/228803642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd41406bd10 0x7fd414161910 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- 172.21.15.45:0/228803642 >> 172.21.15.45:0/228803642 conn(0x7fd4140d19a0 msgr2=0x7fd4140d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- 172.21.15.45:0/228803642 shutdown_connections 2024-04-25T10:28:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- 172.21.15.45:0/228803642 wait complete. 2024-04-25T10:28:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 Processor -- start 2024-04-25T10:28:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- start start 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41406bd10 0x7fd41415e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4140cd9a0 0x7fd4141579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 0x7fd4140c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd414162660 con 0x7fd414161e50 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140cdd80 con 0x7fd41406bd10 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd41a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140d0d70 con 0x7fd4140cd9a0 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 0x7fd4140c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 0x7fd4140c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54238/0 (socket says 172.21.15.45:54238) 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 -- 172.21.15.45:0/212976919 learned_addr learned my addr 172.21.15.45:0/212976919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd40be00640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4140cd9a0 0x7fd4141579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41406bd10 0x7fd41415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 -- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4140cd9a0 msgr2=0x7fd4141579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4140cd9a0 0x7fd4141579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 -- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41406bd10 msgr2=0x7fd41415e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41406bd10 0x7fd41415e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 -- 172.21.15.45:0/212976919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4141f48a0 con 0x7fd414161e50 2024-04-25T10:28:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd413e00640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 0x7fd4140c7030 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fd40c007e40 tx=0x7fd40c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.273+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c014620 con 0x7fd414161e50 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c019070 con 0x7fd414161e50 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c014a20 con 0x7fd414161e50 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4141f3ae0 con 0x7fd414161e50 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4141f40a0 con 0x7fd414161e50 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41404ef30 con 0x7fd414161e50 2024-04-25T10:28:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd40c014bc0 con 0x7fd414161e50 2024-04-25T10:28:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd40b400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3f0066f60 0x7fd3f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd40c0858a0 con 0x7fd414161e50 2024-04-25T10:28:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.277+0000 7fd413400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3f0066f60 0x7fd3f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.281+0000 7fd413400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3f0066f60 0x7fd3f0069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd404001810 tx=0x7fd404009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.281+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd40c056160 con 0x7fd414161e50 2024-04-25T10:28:20.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.441+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4140b7920 con 0x7fd414161e50 2024-04-25T10:28:20.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.441+0000 7fd40b400640 1 -- 172.21.15.45:0/212976919 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd40c05a200 con 0x7fd414161e50 2024-04-25T10:28:20.447 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:28:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3f0066f60 msgr2=0x7fd3f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3f0066f60 0x7fd3f0069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd404001810 tx=0x7fd404009040 comp rx=0 tx=0).stop 2024-04-25T10:28:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 msgr2=0x7fd4140c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 0x7fd4140c7030 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fd40c007e40 tx=0x7fd40c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 shutdown_connections 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd3f0066f60 0x7fd3f0069420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd414161e50 0x7fd4140c7030 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4140cd9a0 0x7fd4141579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 --2- 172.21.15.45:0/212976919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41406bd10 0x7fd41415e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 >> 172.21.15.45:0/212976919 conn(0x7fd4140d19a0 msgr2=0x7fd4140d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 shutdown_connections 2024-04-25T10:28:20.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.445+0000 7fd41a400640 1 -- 172.21.15.45:0/212976919 wait complete. 2024-04-25T10:28:20.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:28:20.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.605+0000 7f3cc1400640 1 Processor -- start 2024-04-25T10:28:20.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.605+0000 7f3cc1400640 1 -- start start 2024-04-25T10:28:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc070c20 0x7f3cbc15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cc1400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cbc0ad840 con 0x7f3cbc0ae020 2024-04-25T10:28:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cc1400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cbc15e9f0 con 0x7f3cbc0cf930 2024-04-25T10:28:20.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cc1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc15bb90 con 0x7f3cbc070c20 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cbb200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3cbc0ae020 0x7f3cbc0b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51714/0 (socket says 172.21.15.45:51714) 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cbb200640 1 -- 172.21.15.45:0/106472042 learned_addr learned my addr 172.21.15.45:0/106472042 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/106472042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc070c20 0x7f3cbc15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb9e00640 1 -- 172.21.15.45:0/106472042 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455816050 0 0) 0x7f3cbc0ad840 con 0x7f3cbc0ae020 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb9e00640 1 -- 172.21.15.45:0/106472042 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ca0003530 con 0x7f3cbc0ae020 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb9e00640 1 -- 172.21.15.45:0/106472042 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894266114 0 0) 0x7f3cbc15e9f0 con 0x7f3cbc0cf930 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb1e00640 1 -- 172.21.15.45:0/106472042 >> v1:172.21.15.117:6789/0 conn(0x7f3cbc0cf930 legacy=0x7f3cbc0cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb1e00640 1 -- 172.21.15.45:0/106472042 >> v1:172.21.15.45:6789/0 conn(0x7f3cbc0ae020 legacy=0x7f3cbc0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.613+0000 7f3cb1e00640 1 -- 172.21.15.45:0/106472042 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc15ecd0 con 0x7f3cbc070c20 2024-04-25T10:28:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/106472042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc070c20 0x7f3cbc15e170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40097f0 tx=0x7f3ca401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998731770f669a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb9e00640 1 -- 172.21.15.45:0/106472042 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40aa610 con 0x7f3cbc070c20 2024-04-25T10:28:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb9e00640 1 -- 172.21.15.45:0/106472042 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca40af070 con 0x7f3cbc070c20 2024-04-25T10:28:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb9e00640 1 -- 172.21.15.45:0/106472042 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40aa9b0 con 0x7f3cbc070c20 2024-04-25T10:28:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/106472042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc070c20 msgr2=0x7f3cbc15e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 --2- 172.21.15.45:0/106472042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc070c20 0x7f3cbc15e170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40097f0 tx=0x7f3ca401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/106472042 shutdown_connections 2024-04-25T10:28:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 --2- 172.21.15.45:0/106472042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc070c20 0x7f3cbc15e170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/106472042 >> 172.21.15.45:0/106472042 conn(0x7f3cbc066b70 msgr2=0x7f3cbc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/106472042 shutdown_connections 2024-04-25T10:28:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/106472042 wait complete. 2024-04-25T10:28:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 Processor -- start 2024-04-25T10:28:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- start start 2024-04-25T10:28:20.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc070c20 0x7f3cbc168e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 0x7f3cbc1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc0cf930 0x7f3cbc1fa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc15e6b0 con 0x7f3cbc0ae020 2024-04-25T10:28:20.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc0ae840 con 0x7f3cbc070c20 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc16c100 con 0x7f3cbc0cf930 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 0x7f3cbc1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 0x7f3cbc1572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54246/0 (socket says 172.21.15.45:54246) 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 -- 172.21.15.45:0/94746751 learned_addr learned my addr 172.21.15.45:0/94746751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cba800640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc070c20 0x7f3cbc168e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cbb200640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc0cf930 0x7f3cbc1fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 -- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc0cf930 msgr2=0x7f3cbc1fa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc0cf930 0x7f3cbc1fa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 -- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc070c20 msgr2=0x7f3cbc168e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc070c20 0x7f3cbc168e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 -- 172.21.15.45:0/94746751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc164610 con 0x7f3cbc0ae020 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 0x7f3cbc1572c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40ac040 tx=0x7f3ca4019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4003820 con 0x7f3cbc0ae020 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca40039c0 con 0x7f3cbc0ae020 2024-04-25T10:28:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4003d90 con 0x7f3cbc0ae020 2024-04-25T10:28:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc1648f0 con 0x7f3cbc0ae020 2024-04-25T10:28:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc164eb0 con 0x7f3cbc0ae020 2024-04-25T10:28:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cbc04ef30 con 0x7f3cbc0ae020 2024-04-25T10:28:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ca40c2060 con 0x7f3cbc0ae020 2024-04-25T10:28:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.617+0000 7f3cb3400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c98066f60 0x7f3c98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.621+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3ca411ee80 con 0x7f3cbc0ae020 2024-04-25T10:28:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.621+0000 7f3cba800640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c98066f60 0x7f3c98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.621+0000 7f3cba800640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c98066f60 0x7f3c98069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0012f0 tx=0x7f3cac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:20.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.621+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ca40a7070 con 0x7f3cbc0ae020 2024-04-25T10:28:20.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.797+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3cbc0673b0 con 0x7f3cbc0ae020 2024-04-25T10:28:20.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.797+0000 7f3cb3400640 1 -- 172.21.15.45:0/94746751 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v131) v1 ==== 80+0+122 (secure 0 0 0) 0x7f3ca40160d0 con 0x7f3cbc0ae020 2024-04-25T10:28:20.803 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6835/1857937743 2024-04-26T10:28:10.677492+0000 2024-04-25T10:28:20.803 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6833/1857937743 2024-04-26T10:28:10.677492+0000 2024-04-25T10:28:20.803 INFO:teuthology.orchestra.run.smithi045.stderr:listed 2 entries 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c98066f60 msgr2=0x7f3c98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c98066f60 0x7f3c98069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0012f0 tx=0x7f3cac009040 comp rx=0 tx=0).stop 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 msgr2=0x7f3cbc1572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 0x7f3cbc1572c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40ac040 tx=0x7f3ca4019250 comp rx=0 tx=0).stop 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 shutdown_connections 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c98066f60 0x7f3c98069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cbc0cf930 0x7f3cbc1fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cbc0ae020 0x7f3cbc1572c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 --2- 172.21.15.45:0/94746751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cbc070c20 0x7f3cbc168e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 >> 172.21.15.45:0/94746751 conn(0x7f3cbc066b70 msgr2=0x7f3cbc0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 shutdown_connections 2024-04-25T10:28:20.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:20.801+0000 7f3cc1400640 1 -- 172.21.15.45:0/94746751 wait complete. 2024-04-25T10:28:20.824 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:28:20.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:20.825 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:28:20.851 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:28:20.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:20.852 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:28:20.908 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:28:20.908 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:20.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:28:20.917 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:28:20.917 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:20.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:28:20.960 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:28:20.960 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:20.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:28:20.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:21.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.141+0000 7f6190a00640 1 Processor -- start 2024-04-25T10:28:21.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.141+0000 7f6190a00640 1 -- start start 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f618c0cfd90 0x7f618c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f618c0cd110 0x7f618c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c15f3b0 con 0x7f618c0b0f70 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c15ef20 con 0x7f618c0cd110 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0ca810 con 0x7f618c0cfd90 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6189e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f618c0b0f70 0x7f618c0b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51716/0 (socket says 172.21.15.45:51716) 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6189e00640 1 -- 172.21.15.45:0/3931478879 learned_addr learned my addr 172.21.15.45:0/3931478879 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f618a800640 1 --2- 172.21.15.45:0/3931478879 >> v2:172.21.15.45:3300/0 conn(0x7f618c0cd110 0x7f618c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6181400640 1 --2- 172.21.15.45:0/3931478879 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cfd90 0x7f618c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f618a800640 1 -- 172.21.15.45:0/3931478879 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cfd90 msgr2=0x7f618c0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f618a800640 1 --2- 172.21.15.45:0/3931478879 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cfd90 0x7f618c0ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f618a800640 1 -- 172.21.15.45:0/3931478879 >> v1:172.21.15.45:6789/0 conn(0x7f618c0b0f70 legacy=0x7f618c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f618a800640 1 -- 172.21.15.45:0/3931478879 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c15e5f0 con 0x7f618c0cd110 2024-04-25T10:28:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f618a800640 1 --2- 172.21.15.45:0/3931478879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0cba20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f618c071c10 tx=0x7f618401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d78bc790d5969fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6189400640 1 -- 172.21.15.45:0/3931478879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840b2070 con 0x7f618c0cd110 2024-04-25T10:28:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6189400640 1 -- 172.21.15.45:0/3931478879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840a57f0 con 0x7f618c0cd110 2024-04-25T10:28:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6189400640 1 -- 172.21.15.45:0/3931478879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840a5bc0 con 0x7f618c0cd110 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- 172.21.15.45:0/3931478879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 msgr2=0x7f618c0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- 172.21.15.45:0/3931478879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0cba20 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f618c071c10 tx=0x7f618401c930 comp rx=0 tx=0).stop 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- 172.21.15.45:0/3931478879 shutdown_connections 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- 172.21.15.45:0/3931478879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0cba20 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- 172.21.15.45:0/3931478879 >> v2:172.21.15.117:3300/0 conn(0x7f618c0cfd90 0x7f618c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- 172.21.15.45:0/3931478879 >> 172.21.15.45:0/3931478879 conn(0x7f618c0b1d50 msgr2=0x7f618c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- 172.21.15.45:0/3931478879 shutdown_connections 2024-04-25T10:28:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- 172.21.15.45:0/3931478879 wait complete. 2024-04-25T10:28:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 Processor -- start 2024-04-25T10:28:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- start start 2024-04-25T10:28:21.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0b0f70 0x7f618c155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c0cfd90 0x7f618c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c161b50 con 0x7f618c0cd110 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1619d0 con 0x7f618c0cfd90 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.145+0000 7f6190a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c161cd0 con 0x7f618c0b0f70 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54264/0 (socket says 172.21.15.45:54264) 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 -- 172.21.15.45:0/3346965811 learned_addr learned my addr 172.21.15.45:0/3346965811 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f618a800640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c0cfd90 0x7f618c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 -- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0b0f70 msgr2=0x7f618c155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6189e00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0b0f70 0x7f618c155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0b0f70 0x7f618c155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 -- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c0cfd90 msgr2=0x7f618c0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c0cfd90 0x7f618c0b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 -- 172.21.15.45:0/3346965811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0b5260 con 0x7f618c0cd110 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f618a800640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c0cfd90 0x7f618c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6189e00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0b0f70 0x7f618c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6181400640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0b4710 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f617400d620 tx=0x7f617400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617400b620 con 0x7f618c0cd110 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174019070 con 0x7f618c0cd110 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c0b5540 con 0x7f618c0cd110 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617400b9e0 con 0x7f618c0cd110 2024-04-25T10:28:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c0b5b00 con 0x7f618c0cd110 2024-04-25T10:28:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c0b5f80 con 0x7f618c0cd110 2024-04-25T10:28:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f617400bb80 con 0x7f618c0cd110 2024-04-25T10:28:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.149+0000 7f6182a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f60 0x7f6168069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.153+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6174085490 con 0x7f618c0cd110 2024-04-25T10:28:21.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.153+0000 7f6189e00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f60 0x7f6168069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.153+0000 7f6189e00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f60 0x7f6168069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f617c007fd0 tx=0x7f617c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.161+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6174056430 con 0x7f618c0cd110 2024-04-25T10:28:21.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.349+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f618c0cda20 con 0x7f618c0cd110 2024-04-25T10:28:21.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.349+0000 7f6182a00640 1 -- 172.21.15.45:0/3346965811 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f6174055d30 con 0x7f618c0cd110 2024-04-25T10:28:21.354 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:21.354 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:12.703341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:21.355 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f60 msgr2=0x7f6168069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f60 0x7f6168069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f617c007fd0 tx=0x7f617c009040 comp rx=0 tx=0).stop 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 msgr2=0x7f618c0b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0b4710 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f617400d620 tx=0x7f617400dae0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 shutdown_connections 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6168066f60 0x7f6168069420 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f618c0cfd90 0x7f618c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f618c0cd110 0x7f618c0b4710 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 --2- 172.21.15.45:0/3346965811 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f618c0b0f70 0x7f618c155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 >> 172.21.15.45:0/3346965811 conn(0x7f618c0b1d50 msgr2=0x7f618c06e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 shutdown_connections 2024-04-25T10:28:21.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.353+0000 7f6190a00640 1 -- 172.21.15.45:0/3346965811 wait complete. 2024-04-25T10:28:21.376 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:21.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff103e00640 1 Processor -- start 2024-04-25T10:28:21.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff103e00640 1 -- start start 2024-04-25T10:28:21.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff103e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0fc0d0040 0x7ff0fc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff103e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc0b2770 con 0x7ff0fc15bc40 2024-04-25T10:28:21.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff103e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc06c670 con 0x7ff0fc0cda10 2024-04-25T10:28:21.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff103e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc06c950 con 0x7ff0fc0d0040 2024-04-25T10:28:21.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff101400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff0fc15bc40 0x7ff0fc0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51718/0 (socket says 172.21.15.45:51718) 2024-04-25T10:28:21.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff101400640 1 -- 172.21.15.45:0/3333771760 learned_addr learned my addr 172.21.15.45:0/3333771760 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:21.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff101e00640 1 --2- 172.21.15.45:0/3333771760 >> v2:172.21.15.45:3300/0 conn(0x7ff0fc0d0040 0x7ff0fc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff0fbe00640 1 -- 172.21.15.45:0/3333771760 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74797076 0 0) 0x7ff0fc0b2770 con 0x7ff0fc15bc40 2024-04-25T10:28:21.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff0fbe00640 1 -- 172.21.15.45:0/3333771760 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e4003530 con 0x7ff0fc15bc40 2024-04-25T10:28:21.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff101e00640 1 -- 172.21.15.45:0/3333771760 >> v1:172.21.15.117:6789/0 conn(0x7ff0fc0cda10 legacy=0x7ff0fc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff101e00640 1 -- 172.21.15.45:0/3333771760 >> v1:172.21.15.45:6789/0 conn(0x7ff0fc15bc40 legacy=0x7ff0fc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.517+0000 7ff101e00640 1 -- 172.21.15.45:0/3333771760 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc06cc30 con 0x7ff0fc0d0040 2024-04-25T10:28:21.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff101e00640 1 --2- 172.21.15.45:0/3333771760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc06c090 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009d90 tx=0x7ff0f401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c074d6ad6bb86ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff0fbe00640 1 -- 172.21.15.45:0/3333771760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40b2070 con 0x7ff0fc0d0040 2024-04-25T10:28:21.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff0fbe00640 1 -- 172.21.15.45:0/3333771760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f40a58b0 con 0x7ff0fc0d0040 2024-04-25T10:28:21.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff0fbe00640 1 -- 172.21.15.45:0/3333771760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40a5c80 con 0x7ff0fc0d0040 2024-04-25T10:28:21.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 -- 172.21.15.45:0/3333771760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0d0040 msgr2=0x7ff0fc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 --2- 172.21.15.45:0/3333771760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc06c090 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009d90 tx=0x7ff0f401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 -- 172.21.15.45:0/3333771760 shutdown_connections 2024-04-25T10:28:21.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 --2- 172.21.15.45:0/3333771760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc06c090 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 -- 172.21.15.45:0/3333771760 >> 172.21.15.45:0/3333771760 conn(0x7ff0fc066b70 msgr2=0x7ff0fc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:21.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 -- 172.21.15.45:0/3333771760 shutdown_connections 2024-04-25T10:28:21.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.521+0000 7ff103e00640 1 -- 172.21.15.45:0/3333771760 wait complete. 2024-04-25T10:28:21.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 Processor -- start 2024-04-25T10:28:21.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 -- start start 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 0x7ff0fc153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0fc15bc40 0x7ff0fc0718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc169980 con 0x7ff0fc0cda10 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc169800 con 0x7ff0fc0d0040 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc169b00 con 0x7ff0fc15bc40 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 0x7ff0fc153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 0x7ff0fc153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54282/0 (socket says 172.21.15.45:54282) 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 -- 172.21.15.45:0/1181686659 learned_addr learned my addr 172.21.15.45:0/1181686659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0fc15bc40 0x7ff0fc0718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff100a00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 -- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0fc15bc40 msgr2=0x7ff0fc0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0fc15bc40 0x7ff0fc0718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 -- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0fc0d0040 msgr2=0x7ff0fc071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 -- 172.21.15.45:0/1181686659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc15c310 con 0x7ff0fc0cda10 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff100a00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0fc15bc40 0x7ff0fc0718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff101400640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 0x7ff0fc153e10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000c4d0 tx=0x7ff0f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0017070 con 0x7ff0fc0cda10 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f0005cf0 con 0x7ff0fc0cda10 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc15c5f0 con 0x7ff0fc0cda10 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0007270 con 0x7ff0fc0cda10 2024-04-25T10:28:21.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.525+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc15d0d0 con 0x7ff0fc0cda10 2024-04-25T10:28:21.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.529+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc1580b0 con 0x7ff0fc0cda10 2024-04-25T10:28:21.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.529+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0f001a070 con 0x7ff0fc0cda10 2024-04-25T10:28:21.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.529+0000 7ff0f9600640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0dc066f60 0x7ff0dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.529+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff0f0084480 con 0x7ff0fc0cda10 2024-04-25T10:28:21.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.529+0000 7ff100a00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0dc066f60 0x7ff0dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.529+0000 7ff100a00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0dc066f60 0x7ff0dc069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc1fa2d0 tx=0x7ff0e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.533+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0f0054d40 con 0x7ff0fc0cda10 2024-04-25T10:28:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.693+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0fc04ef30 con 0x7ff0fc0cda10 2024-04-25T10:28:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff0f9600640 1 -- 172.21.15.45:0/1181686659 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16744 (secure 0 0 0) 0x7ff0f0058de0 con 0x7ff0fc0cda10 2024-04-25T10:28:21.696 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:21.696 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:10.677542+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1857937743":"2024-04-26T10:28:10.677492+0000","172.21.15.45:6833/1857937743":"2024-04-26T10:28:10.677492+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0dc066f60 msgr2=0x7ff0dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0dc066f60 0x7ff0dc069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc1fa2d0 tx=0x7ff0e8009040 comp rx=0 tx=0).stop 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 msgr2=0x7ff0fc153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 0x7ff0fc153e10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000c4d0 tx=0x7ff0f000c990 comp rx=0 tx=0).stop 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 shutdown_connections 2024-04-25T10:28:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff0dc066f60 0x7ff0dc069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff0fc15bc40 0x7ff0fc0718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0fc0d0040 0x7ff0fc071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 --2- 172.21.15.45:0/1181686659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0fc0cda10 0x7ff0fc153e10 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 >> 172.21.15.45:0/1181686659 conn(0x7ff0fc066b70 msgr2=0x7ff0fc155090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 shutdown_connections 2024-04-25T10:28:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.697+0000 7ff103e00640 1 -- 172.21.15.45:0/1181686659 wait complete. 2024-04-25T10:28:21.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:27:19.715814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:27:20.287107+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:28:21.719 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:28:21.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.861+0000 7f1af5400640 1 Processor -- start 2024-04-25T10:28:21.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.861+0000 7f1af5400640 1 -- start start 2024-04-25T10:28:21.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1af5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af00cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1af5400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af00ad840 con 0x7f1af015e130 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1af5400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af015e710 con 0x7f1af00b3480 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1af5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00d0090 con 0x7f1af00722d0 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1ae5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1af015e130 0x7f1af0071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51720/0 (socket says 172.21.15.45:51720) 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1ae5e00640 1 -- 172.21.15.45:0/2837565108 learned_addr learned my addr 172.21.15.45:0/2837565108 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:21.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aef200640 1 --2- 172.21.15.45:0/2837565108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af00cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561003309 0 0) 0x7f1af00ad840 con 0x7f1af015e130 2024-04-25T10:28:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad4003530 con 0x7f1af015e130 2024-04-25T10:28:21.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453484404 0 0) 0x7f1af015e710 con 0x7f1af00b3480 2024-04-25T10:28:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af00ad840 con 0x7f1af00b3480 2024-04-25T10:28:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aef200640 1 -- 172.21.15.45:0/2837565108 >> v1:172.21.15.117:6789/0 conn(0x7f1af00b3480 legacy=0x7f1af0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aef200640 1 -- 172.21.15.45:0/2837565108 >> v1:172.21.15.45:6789/0 conn(0x7f1af015e130 legacy=0x7f1af0071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.865+0000 7f1aef200640 1 -- 172.21.15.45:0/2837565108 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af015e9f0 con 0x7f1af00722d0 2024-04-25T10:28:21.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1aef200640 1 --2- 172.21.15.45:0/2837565108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af00cfb50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009dc0 tx=0x7f1ae801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90353e6a6e56e9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80b2070 con 0x7f1af00722d0 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80a5850 con 0x7f1af00722d0 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1aede00640 1 -- 172.21.15.45:0/2837565108 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80a5c20 con 0x7f1af00722d0 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 -- 172.21.15.45:0/2837565108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 msgr2=0x7f1af00cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 --2- 172.21.15.45:0/2837565108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af00cfb50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009dc0 tx=0x7f1ae801c990 comp rx=0 tx=0).stop 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 -- 172.21.15.45:0/2837565108 shutdown_connections 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 --2- 172.21.15.45:0/2837565108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af00cfb50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 -- 172.21.15.45:0/2837565108 >> 172.21.15.45:0/2837565108 conn(0x7f1af00669a0 msgr2=0x7f1af0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 -- 172.21.15.45:0/2837565108 shutdown_connections 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 -- 172.21.15.45:0/2837565108 wait complete. 2024-04-25T10:28:21.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 Processor -- start 2024-04-25T10:28:21.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.869+0000 7f1af5400640 1 -- start start 2024-04-25T10:28:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af0152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00b3480 0x7f1af013a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 0x7f1af01605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af015bb90 con 0x7f1af015e130 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00ae800 con 0x7f1af00b3480 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af016b8a0 con 0x7f1af00722d0 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 0x7f1af01605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 0x7f1af01605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54288/0 (socket says 172.21.15.45:54288) 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 -- 172.21.15.45:0/2413065182 learned_addr learned my addr 172.21.15.45:0/2413065182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1ae5e00640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00b3480 0x7f1af013a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aee800640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af0152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 -- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 msgr2=0x7f1af0152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af0152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 -- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00b3480 msgr2=0x7f1af013a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00b3480 0x7f1af013a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 -- 172.21.15.45:0/2413065182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af013a600 con 0x7f1af015e130 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aee800640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af0152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1aef200640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 0x7f1af01605f0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80195a0 tx=0x7f1ae8019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80b2050 con 0x7f1af015e130 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80be070 con 0x7f1af015e130 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af014fb60 con 0x7f1af015e130 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80ad890 con 0x7f1af015e130 2024-04-25T10:28:21.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af014fe00 con 0x7f1af015e130 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af004ef30 con 0x7f1af015e130 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.873+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ae80ada30 con 0x7f1af015e130 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.877+0000 7f1ae7400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1acc066f60 0x7f1acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.877+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1ae8120110 con 0x7f1af015e130 2024-04-25T10:28:21.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.877+0000 7f1aee800640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1acc066f60 0x7f1acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:21.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.877+0000 7f1aee800640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1acc066f60 0x7f1acc069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0001740 tx=0x7f1ae0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:21.881+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ae8016580 con 0x7f1af015e130 2024-04-25T10:28:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.041+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1af0161cc0 con 0x7f1af015e130 2024-04-25T10:28:22.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.041+0000 7f1ae7400640 1 -- 172.21.15.45:0/2413065182 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ae80f1170 con 0x7f1af015e130 2024-04-25T10:28:22.043 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:28:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1acc066f60 msgr2=0x7f1acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1acc066f60 0x7f1acc069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0001740 tx=0x7f1ae0009040 comp rx=0 tx=0).stop 2024-04-25T10:28:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 msgr2=0x7f1af01605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 0x7f1af01605f0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80195a0 tx=0x7f1ae8019730 comp rx=0 tx=0).stop 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 shutdown_connections 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1acc066f60 0x7f1acc069420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1af015e130 0x7f1af01605f0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00b3480 0x7f1af013a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 --2- 172.21.15.45:0/2413065182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af00722d0 0x7f1af0152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 >> 172.21.15.45:0/2413065182 conn(0x7f1af00669a0 msgr2=0x7f1af006d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 shutdown_connections 2024-04-25T10:28:22.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.045+0000 7f1af5400640 1 -- 172.21.15.45:0/2413065182 wait complete. 2024-04-25T10:28:22.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:28:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 Processor -- start 2024-04-25T10:28:22.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 -- start start 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0aef80 0x7fab4c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab4c0b31c0 0x7fab4c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4c15b080 con 0x7fab4c070350 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c15adb0 con 0x7fab4c0b31c0 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.217+0000 7fab50e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c15eb60 con 0x7fab4c0aef80 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab4c0b31c0 0x7fab4c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab4c0b31c0 0x7fab4c0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54302/0 (socket says 172.21.15.45:54302) 2024-04-25T10:28:22.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 -- 172.21.15.45:0/1434339708 learned_addr learned my addr 172.21.15.45:0/1434339708 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab49e00640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0aef80 0x7fab4c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 -- 172.21.15.45:0/1434339708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0aef80 msgr2=0x7fab4c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0aef80 0x7fab4c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 -- 172.21.15.45:0/1434339708 >> v1:172.21.15.117:6789/0 conn(0x7fab4c070350 legacy=0x7fab4c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 -- 172.21.15.45:0/1434339708 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c15ee40 con 0x7fab4c0b31c0 2024-04-25T10:28:22.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab49e00640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0aef80 0x7fab4c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:22.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab4a800640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0b31c0 0x7fab4c0b3610 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fab44009e50 tx=0x7fab4401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4a443ad8654d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab49400640 1 -- 172.21.15.45:0/1434339708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab440a5780 con 0x7fab4c0b31c0 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab49400640 1 -- 172.21.15.45:0/1434339708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab440a5920 con 0x7fab4c0b31c0 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.221+0000 7fab49400640 1 -- 172.21.15.45:0/1434339708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab440a5cf0 con 0x7fab4c0b31c0 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- 172.21.15.45:0/1434339708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0b31c0 msgr2=0x7fab4c0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0b31c0 0x7fab4c0b3610 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fab44009e50 tx=0x7fab4401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- 172.21.15.45:0/1434339708 shutdown_connections 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0b31c0 0x7fab4c0b3610 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 --2- 172.21.15.45:0/1434339708 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0aef80 0x7fab4c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- 172.21.15.45:0/1434339708 >> 172.21.15.45:0/1434339708 conn(0x7fab4c0669a0 msgr2=0x7fab4c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- 172.21.15.45:0/1434339708 shutdown_connections 2024-04-25T10:28:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- 172.21.15.45:0/1434339708 wait complete. 2024-04-25T10:28:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 Processor -- start 2024-04-25T10:28:22.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- start start 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c070350 0x7fab4c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 0x7fab4c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0b31c0 0x7fab4c144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c1679a0 con 0x7fab4c0aef80 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c167820 con 0x7fab4c070350 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab50e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c167b20 con 0x7fab4c0b31c0 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 0x7fab4c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 0x7fab4c153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54316/0 (socket says 172.21.15.45:54316) 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 -- 172.21.15.45:0/1335724891 learned_addr learned my addr 172.21.15.45:0/1335724891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab49e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c070350 0x7fab4c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab4a800640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0b31c0 0x7fab4c144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 -- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0b31c0 msgr2=0x7fab4c144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0b31c0 0x7fab4c144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 -- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c070350 msgr2=0x7fab4c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c070350 0x7fab4c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.225+0000 7fab41400640 1 -- 172.21.15.45:0/1335724891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c151fe0 con 0x7fab4c0aef80 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab41400640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 0x7fab4c153980 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c450 tx=0x7fab3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34012830 con 0x7fab4c0aef80 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab34017070 con 0x7fab4c0aef80 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c1522c0 con 0x7fab4c0aef80 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34012c30 con 0x7fab4c0aef80 2024-04-25T10:28:22.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c152880 con 0x7fab4c0aef80 2024-04-25T10:28:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c04ef30 con 0x7fab4c0aef80 2024-04-25T10:28:22.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab3401c460 con 0x7fab4c0aef80 2024-04-25T10:28:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab42a00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab28066f60 0x7fab28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.229+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fab34084dd0 con 0x7fab4c0aef80 2024-04-25T10:28:22.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.233+0000 7fab49e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab28066f60 0x7fab28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.233+0000 7fab49e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab28066f60 0x7fab28069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fab4c15ccf0 tx=0x7fab3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.233+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab34055690 con 0x7fab4c0aef80 2024-04-25T10:28:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab4c0671e0 con 0x7fab4c0aef80 2024-04-25T10:28:22.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab42a00640 1 -- 172.21.15.45:0/1335724891 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab34059730 con 0x7fab4c0aef80 2024-04-25T10:28:22.398 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:28:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab28066f60 msgr2=0x7fab28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab28066f60 0x7fab28069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fab4c15ccf0 tx=0x7fab3c008040 comp rx=0 tx=0).stop 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 msgr2=0x7fab4c153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 0x7fab4c153980 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c450 tx=0x7fab3400c910 comp rx=0 tx=0).stop 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 shutdown_connections 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab28066f60 0x7fab28069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab4c0b31c0 0x7fab4c144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab4c0aef80 0x7fab4c153980 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 --2- 172.21.15.45:0/1335724891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c070350 0x7fab4c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.397+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 >> 172.21.15.45:0/1335724891 conn(0x7fab4c0669a0 msgr2=0x7fab4c0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.401+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 shutdown_connections 2024-04-25T10:28:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.401+0000 7fab50e00640 1 -- 172.21.15.45:0/1335724891 wait complete. 2024-04-25T10:28:22.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:28:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.561+0000 7f67fbe00640 1 Processor -- start 2024-04-25T10:28:22.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.561+0000 7f67fbe00640 1 -- start start 2024-04-25T10:28:22.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 0x7f67f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67fbe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f40b2fd0 con 0x7f67f415a0e0 2024-04-25T10:28:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67fbe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f40b32b0 con 0x7f67f415b150 2024-04-25T10:28:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67fbe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f415b5c0 con 0x7f67f40aea50 2024-04-25T10:28:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f9400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f67f415a0e0 0x7f67f40b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51726/0 (socket says 172.21.15.45:51726) 2024-04-25T10:28:22.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f9400640 1 -- 172.21.15.45:0/2854995039 learned_addr learned my addr 172.21.15.45:0/2854995039 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f9e00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 0x7f67f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646361510 0 0) 0x7f67f40b2fd0 con 0x7f67f415a0e0 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67dc003530 con 0x7f67f415a0e0 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778778641 0 0) 0x7f67dc003530 con 0x7f67f415a0e0 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 msgr2=0x7f67f40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 0x7f67f40705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 >> v1:172.21.15.117:6789/0 conn(0x7f67f415b150 legacy=0x7f67f406c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f40b3590 con 0x7f67f415a0e0 2024-04-25T10:28:22.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f67e8001680 con 0x7f67f415a0e0 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 >> v1:172.21.15.45:6789/0 conn(0x7f67f415a0e0 legacy=0x7f67f40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 0x7f67dc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67dc004d50 0x7f67dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00dd70 con 0x7f67dc003f70 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00def0 con 0x7f67dc004d50 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00e0f0 con 0x7f67f40aea50 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f8a00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f9400640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 0x7f67dc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f9e00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67dc004d50 0x7f67dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f8a00640 1 -- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 msgr2=0x7f67dc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f8a00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f40aea50 0x7f67dc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f8a00640 1 -- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67dc004d50 msgr2=0x7f67dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:22.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f8a00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67dc004d50 0x7f67dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.565+0000 7f67f8a00640 1 -- 172.21.15.45:0/2854995039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc003530 con 0x7f67dc003f70 2024-04-25T10:28:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67f8a00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67dc004810 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f67e0001780 tx=0x7f67e001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e926cecc4122b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00013a0 con 0x7f67dc003f70 2024-04-25T10:28:22.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e00052f0 con 0x7f67dc003f70 2024-04-25T10:28:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67f3e00640 1 -- 172.21.15.45:0/2854995039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003840 con 0x7f67dc003f70 2024-04-25T10:28:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67fbe00640 1 -- 172.21.15.45:0/2854995039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 msgr2=0x7f67dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67fbe00640 1 --2- 172.21.15.45:0/2854995039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67dc004810 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f67e0001780 tx=0x7f67e001c420 comp rx=0 tx=0).stop 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67f9e00640 1 -- 172.21.15.45:0/2854995039 reap_dead start 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67fbe00640 1 -- 172.21.15.45:0/2854995039 shutdown_connections 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67fbe00640 1 -- 172.21.15.45:0/2854995039 >> 172.21.15.45:0/2854995039 conn(0x7f67f40669a0 msgr2=0x7f67f4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67fbe00640 1 -- 172.21.15.45:0/2854995039 shutdown_connections 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.569+0000 7f67fbe00640 1 -- 172.21.15.45:0/2854995039 wait complete. 2024-04-25T10:28:22.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 Processor -- start 2024-04-25T10:28:22.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- start start 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67f4134870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f41f6450 0x7f67f4163030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67f41f68d0 0x7f67f4163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f406c7e0 con 0x7f67dc003f70 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40af1d0 con 0x7f67f41f68d0 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f406c960 con 0x7f67f41f6450 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67f4134870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67f4134870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54334/0 (socket says 172.21.15.45:54334) 2024-04-25T10:28:22.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 -- 172.21.15.45:0/1616546733 learned_addr learned my addr 172.21.15.45:0/1616546733 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9e00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67f41f68d0 0x7f67f4163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f8a00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f41f6450 0x7f67f4163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 -- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f41f6450 msgr2=0x7f67f4163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f41f6450 0x7f67f4163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 -- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67f41f68d0 msgr2=0x7f67f4163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67f41f68d0 0x7f67f4163570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 -- 172.21.15.45:0/1616546733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4134db0 con 0x7f67dc003f70 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9e00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67f41f68d0 0x7f67f4163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f8a00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f41f6450 0x7f67f4163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:22.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f9400640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67f4134870 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f67e8007290 tx=0x7f67e8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800f8e0 con 0x7f67dc003f70 2024-04-25T10:28:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8018070 con 0x7f67dc003f70 2024-04-25T10:28:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800fce0 con 0x7f67dc003f70 2024-04-25T10:28:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f40ce0f0 con 0x7f67dc003f70 2024-04-25T10:28:22.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f40ce6b0 con 0x7f67dc003f70 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.573+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f404ef30 con 0x7f67dc003f70 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.577+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67e801d460 con 0x7f67dc003f70 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.577+0000 7f67f1600640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67d4066f60 0x7f67d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.577+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f67e8085180 con 0x7f67dc003f70 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.577+0000 7f67f8a00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67d4066f60 0x7f67d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.577+0000 7f67f8a00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67d4066f60 0x7f67d4069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f67e00ac000 tx=0x7f67e00011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.581+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67f4134db0 con 0x7f67dc003f70 2024-04-25T10:28:22.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.757+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67f41543b0 con 0x7f67dc003f70 2024-04-25T10:28:22.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.757+0000 7f67f1600640 1 -- 172.21.15.45:0/1616546733 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67e8055a40 con 0x7f67dc003f70 2024-04-25T10:28:22.759 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:28:22.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67d4066f60 msgr2=0x7f67d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67d4066f60 0x7f67d4069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f67e00ac000 tx=0x7f67e00011a0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 msgr2=0x7f67f4134870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67f4134870 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f67e8007290 tx=0x7f67e8007750 comp rx=0 tx=0).stop 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 shutdown_connections 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67d4066f60 0x7f67d4069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67f41f68d0 0x7f67f4163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67f41f6450 0x7f67f4163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 --2- 172.21.15.45:0/1616546733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67dc003f70 0x7f67f4134870 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 >> 172.21.15.45:0/1616546733 conn(0x7f67f40669a0 msgr2=0x7f67f4150190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 shutdown_connections 2024-04-25T10:28:22.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.761+0000 7f67fbe00640 1 -- 172.21.15.45:0/1616546733 wait complete. 2024-04-25T10:28:22.780 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 Processor -- start 2024-04-25T10:28:22.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 -- start start 2024-04-25T10:28:22.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3a415e050 0x7fc3a40b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a40b13e0 0x7fc3a40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40b1850 con 0x7fc3a40d1820 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a40b1af0 con 0x7fc3a415e050 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3aa800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40b1dd0 con 0x7fc3a40b13e0 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3a415e050 0x7fc3a40b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc3a415e050 0x7fc3a40b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54342/0 (socket says 172.21.15.45:54342) 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 -- 172.21.15.45:0/4065928018 learned_addr learned my addr 172.21.15.45:0/4065928018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc3a3e00640 1 --2- 172.21.15.45:0/4065928018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a40b13e0 0x7fc3a40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 -- 172.21.15.45:0/4065928018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a40b13e0 msgr2=0x7fc3a40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 --2- 172.21.15.45:0/4065928018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a40b13e0 0x7fc3a40cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 -- 172.21.15.45:0/4065928018 >> v1:172.21.15.117:6789/0 conn(0x7fc3a40d1820 legacy=0x7fc3a40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.925+0000 7fc39aa00640 1 -- 172.21.15.45:0/4065928018 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40b20b0 con 0x7fc3a415e050 2024-04-25T10:28:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc39aa00640 1 --2- 172.21.15.45:0/4065928018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a415e050 0x7fc3a40b0ea0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009870 tx=0x7fc38c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c2bcf098fc59ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3a2a00640 1 -- 172.21.15.45:0/4065928018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0aa430 con 0x7fc3a415e050 2024-04-25T10:28:22.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3a2a00640 1 -- 172.21.15.45:0/4065928018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c0af070 con 0x7fc3a415e050 2024-04-25T10:28:22.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3a2a00640 1 -- 172.21.15.45:0/4065928018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0aa7f0 con 0x7fc3a415e050 2024-04-25T10:28:22.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 -- 172.21.15.45:0/4065928018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a415e050 msgr2=0x7fc3a40b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 --2- 172.21.15.45:0/4065928018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a415e050 0x7fc3a40b0ea0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009870 tx=0x7fc38c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 -- 172.21.15.45:0/4065928018 shutdown_connections 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 --2- 172.21.15.45:0/4065928018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a40b13e0 0x7fc3a40cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 --2- 172.21.15.45:0/4065928018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a415e050 0x7fc3a40b0ea0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 -- 172.21.15.45:0/4065928018 >> 172.21.15.45:0/4065928018 conn(0x7fc3a409a9d0 msgr2=0x7fc3a409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 -- 172.21.15.45:0/4065928018 shutdown_connections 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.929+0000 7fc3aa800640 1 -- 172.21.15.45:0/4065928018 wait complete. 2024-04-25T10:28:22.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 Processor -- start 2024-04-25T10:28:22.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- start start 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 0x7fc3a415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a40d1820 0x7fc3a414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a415e050 0x7fc3a40b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40d0dd0 con 0x7fc3a40b13e0 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40d0c50 con 0x7fc3a40d1820 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40d0f50 con 0x7fc3a415e050 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 0x7fc3a415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 0x7fc3a415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54356/0 (socket says 172.21.15.45:54356) 2024-04-25T10:28:22.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 -- 172.21.15.45:0/2089539622 learned_addr learned my addr 172.21.15.45:0/2089539622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc39aa00640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a40d1820 0x7fc3a414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3e00640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a415e050 0x7fc3a40b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 -- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a415e050 msgr2=0x7fc3a40b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a415e050 0x7fc3a40b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 -- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a40d1820 msgr2=0x7fc3a414f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a40d1820 0x7fc3a414f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 -- 172.21.15.45:0/2089539622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a414fe60 con 0x7fc3a40b13e0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3a3400640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 0x7fc3a415a160 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c400 tx=0x7fc39400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394017070 con 0x7fc3a40b13e0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394005310 con 0x7fc3a40b13e0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a40b5120 con 0x7fc3a40b13e0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940054c0 con 0x7fc3a40b13e0 2024-04-25T10:28:22.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a40b56e0 con 0x7fc3a40b13e0 2024-04-25T10:28:22.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.933+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a404ef30 con 0x7fc3a40b13e0 2024-04-25T10:28:22.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.937+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc39401a070 con 0x7fc3a40b13e0 2024-04-25T10:28:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.937+0000 7fc39be00640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc380066f60 0x7fc380069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.937+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc3940842b0 con 0x7fc3a40b13e0 2024-04-25T10:28:22.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.937+0000 7fc39aa00640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc380066f60 0x7fc380069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:22.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.937+0000 7fc39aa00640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc380066f60 0x7fc380069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0ae000 tx=0x7fc38c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:22.941+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc394054c00 con 0x7fc3a40b13e0 2024-04-25T10:28:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3a41565f0 con 0x7fc3a40b13e0 2024-04-25T10:28:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc39be00640 1 -- 172.21.15.45:0/2089539622 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fc394058ca0 con 0x7fc3a40b13e0 2024-04-25T10:28:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:23.141 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:12.703341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:23.141 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2024-04-25T10:28:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc380066f60 msgr2=0x7fc380069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc380066f60 0x7fc380069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0ae000 tx=0x7fc38c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:28:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 msgr2=0x7fc3a415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 0x7fc3a415a160 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc39400c400 tx=0x7fc39400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 shutdown_connections 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc380066f60 0x7fc380069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a415e050 0x7fc3a40b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a40d1820 0x7fc3a414f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 --2- 172.21.15.45:0/2089539622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3a40b13e0 0x7fc3a415a160 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 >> 172.21.15.45:0/2089539622 conn(0x7fc3a409a9d0 msgr2=0x7fc3a40d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 shutdown_connections 2024-04-25T10:28:23.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.141+0000 7fc3aa800640 1 -- 172.21.15.45:0/2089539622 wait complete. 2024-04-25T10:28:23.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.301+0000 7f6d94000640 1 Processor -- start 2024-04-25T10:28:23.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.301+0000 7f6d94000640 1 -- start start 2024-04-25T10:28:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d94000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 0x7f6d8c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d94000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0ccaa0 con 0x7f6d8c06aea0 2024-04-25T10:28:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d94000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0ccd80 con 0x7f6d8c0d0180 2024-04-25T10:28:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d94000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15e340 con 0x7f6d8c15df60 2024-04-25T10:28:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d92000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6d8c06aea0 0x7f6d8c06b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51732/0 (socket says 172.21.15.45:51732) 2024-04-25T10:28:23.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d92000640 1 -- 172.21.15.45:0/3786595738 learned_addr learned my addr 172.21.15.45:0/3786595738 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877019732 0 0) 0x7f6d8c0ccaa0 con 0x7f6d8c06aea0 2024-04-25T10:28:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74003530 con 0x7f6d8c06aea0 2024-04-25T10:28:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d91600640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 0x7f6d8c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997845849 0 0) 0x7f6d74003530 con 0x7f6d8c06aea0 2024-04-25T10:28:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 msgr2=0x7f6d8c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:23.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 0x7f6d8c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 >> v1:172.21.15.117:6789/0 conn(0x7f6d8c0d0180 legacy=0x7f6d8c0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0cd060 con 0x7f6d8c06aea0 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f6d84001a00 con 0x7f6d8c06aea0 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 >> v1:172.21.15.45:6789/0 conn(0x7f6d8c06aea0 legacy=0x7f6d8c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d740039f0 0x7f6d74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 0x7f6d74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400bad0 con 0x7f6d74004810 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400bc80 con 0x7f6d740039f0 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400be80 con 0x7f6d8c15df60 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d90c00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d91600640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d740039f0 0x7f6d74003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d92000640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 0x7f6d74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d90c00640 1 -- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 msgr2=0x7f6d74005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d90c00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c15df60 0x7f6d74005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d90c00640 1 -- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d740039f0 msgr2=0x7f6d74003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d90c00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d740039f0 0x7f6d74003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.305+0000 7f6d90c00640 1 -- 172.21.15.45:0/3786595738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74003530 con 0x7f6d74004810 2024-04-25T10:28:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d90c00640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d74004c90 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f6d78001780 tx=0x7f6d7801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff4c0d0db6c0807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780013a0 con 0x7f6d74004810 2024-04-25T10:28:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d780052f0 con 0x7f6d74004810 2024-04-25T10:28:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d8be00640 1 -- 172.21.15.45:0/3786595738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78003840 con 0x7f6d74004810 2024-04-25T10:28:23.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 -- 172.21.15.45:0/3786595738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 msgr2=0x7f6d74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 --2- 172.21.15.45:0/3786595738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d74004c90 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f6d78001780 tx=0x7f6d7801c420 comp rx=0 tx=0).stop 2024-04-25T10:28:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d92000640 1 -- 172.21.15.45:0/3786595738 reap_dead start 2024-04-25T10:28:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 -- 172.21.15.45:0/3786595738 shutdown_connections 2024-04-25T10:28:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 -- 172.21.15.45:0/3786595738 >> 172.21.15.45:0/3786595738 conn(0x7f6d8c0d1910 msgr2=0x7f6d8c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 -- 172.21.15.45:0/3786595738 shutdown_connections 2024-04-25T10:28:23.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 -- 172.21.15.45:0/3786595738 wait complete. 2024-04-25T10:28:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 Processor -- start 2024-04-25T10:28:23.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.309+0000 7f6d94000640 1 -- start start 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d8c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c160030 0x7f6d8c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d8c1609f0 0x7f6d8c0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0d0ff0 con 0x7f6d74004810 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0d0e70 con 0x7f6d8c1609f0 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c169a20 con 0x7f6d8c160030 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d8c0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d8c0c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54376/0 (socket says 172.21.15.45:54376) 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 -- 172.21.15.45:0/740006480 learned_addr learned my addr 172.21.15.45:0/740006480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:23.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d92000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d8c1609f0 0x7f6d8c0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d90c00640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c160030 0x7f6d8c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 -- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c160030 msgr2=0x7f6d8c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c160030 0x7f6d8c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 -- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d8c1609f0 msgr2=0x7f6d8c0c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d8c1609f0 0x7f6d8c0c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 -- 172.21.15.45:0/740006480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c15e3e0 con 0x7f6d74004810 2024-04-25T10:28:23.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d91600640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d8c0c7f70 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011960 tx=0x7f6d80011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000faf0 con 0x7f6d74004810 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000fc90 con 0x7f6d74004810 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80014440 con 0x7f6d74004810 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c15e6c0 con 0x7f6d74004810 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0c9280 con 0x7f6d74004810 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.313+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c04ef30 con 0x7f6d74004810 2024-04-25T10:28:23.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.317+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d80012040 con 0x7f6d74004810 2024-04-25T10:28:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.317+0000 7f6d89600640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d6c066f60 0x7f6d6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.317+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d80085730 con 0x7f6d74004810 2024-04-25T10:28:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.317+0000 7f6d90c00640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d6c066f60 0x7f6d6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.317+0000 7f6d90c00640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d6c066f60 0x7f6d6c069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6d78001750 tx=0x7f6d780011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:23.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.321+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d80019030 con 0x7f6d74004810 2024-04-25T10:28:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.481+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d8c06b4d0 con 0x7f6d74004810 2024-04-25T10:28:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d89600640 1 -- 172.21.15.45:0/740006480 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f6d800566d0 con 0x7f6d74004810 2024-04-25T10:28:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:23.484 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:10.677542+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1857937743":"2024-04-26T10:28:10.677492+0000","172.21.15.45:6833/1857937743":"2024-04-26T10:28:10.677492+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d6c066f60 msgr2=0x7f6d6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d6c066f60 0x7f6d6c069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6d78001750 tx=0x7f6d780011a0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 msgr2=0x7f6d8c0c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d8c0c7f70 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6d80011960 tx=0x7f6d80011e20 comp rx=0 tx=0).stop 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 shutdown_connections 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d6c066f60 0x7f6d6c069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d8c1609f0 0x7f6d8c0c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d8c160030 0x7f6d8c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 --2- 172.21.15.45:0/740006480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d74004810 0x7f6d8c0c7f70 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 >> 172.21.15.45:0/740006480 conn(0x7f6d8c0d1910 msgr2=0x7f6d8c0b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 shutdown_connections 2024-04-25T10:28:23.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.485+0000 7f6d94000640 1 -- 172.21.15.45:0/740006480 wait complete. 2024-04-25T10:28:23.506 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:28:23.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.645+0000 7f4492e00640 1 Processor -- start 2024-04-25T10:28:23.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.645+0000 7f4492e00640 1 -- start start 2024-04-25T10:28:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4492e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f448c066c00 0x7f448c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4492e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f448c066fe0 0x7f448c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4492e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c0b5c20 con 0x7f448c06ed30 2024-04-25T10:28:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4492e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c0af6e0 con 0x7f448c066c00 2024-04-25T10:28:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4492e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c0af9c0 con 0x7f448c066fe0 2024-04-25T10:28:23.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f448be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f448c066c00 0x7f448c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4490e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f448c06ed30 0x7f448c157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51736/0 (socket says 172.21.15.45:51736) 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4490e00640 1 -- 172.21.15.45:0/3897850875 learned_addr learned my addr 172.21.15.45:0/3897850875 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f4483e00640 1 --2- 172.21.15.45:0/3897850875 >> v2:172.21.15.117:3300/0 conn(0x7f448c066fe0 0x7f448c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f448be00640 1 -- 172.21.15.45:0/3897850875 >> v2:172.21.15.117:3300/0 conn(0x7f448c066fe0 msgr2=0x7f448c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f448be00640 1 --2- 172.21.15.45:0/3897850875 >> v2:172.21.15.117:3300/0 conn(0x7f448c066fe0 0x7f448c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f448be00640 1 -- 172.21.15.45:0/3897850875 >> v1:172.21.15.45:6789/0 conn(0x7f448c06ed30 legacy=0x7f448c157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.649+0000 7f448be00640 1 -- 172.21.15.45:0/3897850875 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c0afca0 con 0x7f448c066c00 2024-04-25T10:28:23.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f448be00640 1 --2- 172.21.15.45:0/3897850875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066c00 0x7f448c15e110 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f448c06e850 tx=0x7f447c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cf8c22d8b6dfb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f448b400640 1 -- 172.21.15.45:0/3897850875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0a7070 con 0x7f448c066c00 2024-04-25T10:28:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f448b400640 1 -- 172.21.15.45:0/3897850875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0b0070 con 0x7f448c066c00 2024-04-25T10:28:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f448b400640 1 -- 172.21.15.45:0/3897850875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0ab890 con 0x7f448c066c00 2024-04-25T10:28:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 -- 172.21.15.45:0/3897850875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066c00 msgr2=0x7f448c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 --2- 172.21.15.45:0/3897850875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066c00 0x7f448c15e110 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f448c06e850 tx=0x7f447c01c450 comp rx=0 tx=0).stop 2024-04-25T10:28:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 -- 172.21.15.45:0/3897850875 shutdown_connections 2024-04-25T10:28:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 --2- 172.21.15.45:0/3897850875 >> v2:172.21.15.117:3300/0 conn(0x7f448c066fe0 0x7f448c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 --2- 172.21.15.45:0/3897850875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066c00 0x7f448c15e110 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 -- 172.21.15.45:0/3897850875 >> 172.21.15.45:0/3897850875 conn(0x7f448c0b24f0 msgr2=0x7f448c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 -- 172.21.15.45:0/3897850875 shutdown_connections 2024-04-25T10:28:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 -- 172.21.15.45:0/3897850875 wait complete. 2024-04-25T10:28:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 Processor -- start 2024-04-25T10:28:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.653+0000 7f4492e00640 1 -- start start 2024-04-25T10:28:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c066c00 0x7f448c14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 0x7f448c14c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f448c06ed30 0x7f448c14ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06ac90 con 0x7f448c066fe0 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0af270 con 0x7f448c06ed30 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06ae10 con 0x7f448c066c00 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 0x7f448c14c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 0x7f448c14c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54406/0 (socket says 172.21.15.45:54406) 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 -- 172.21.15.45:0/4162869900 learned_addr learned my addr 172.21.15.45:0/4162869900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f448be00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c066c00 0x7f448c14a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4490e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f448c06ed30 0x7f448c14ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 -- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c066c00 msgr2=0x7f448c14a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c066c00 0x7f448c14a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 -- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f448c06ed30 msgr2=0x7f448c14ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f448c06ed30 0x7f448c14ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 -- 172.21.15.45:0/4162869900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c14b240 con 0x7f448c066fe0 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f448be00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c066c00 0x7f448c14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4490e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f448c06ed30 0x7f448c14ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4483e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 0x7f448c14c8c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f447400d800 tx=0x7f447400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447400b6c0 con 0x7f448c066fe0 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474019070 con 0x7f448c066fe0 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447400ba60 con 0x7f448c066fe0 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c14b520 con 0x7f448c066fe0 2024-04-25T10:28:23.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c14c250 con 0x7f448c066fe0 2024-04-25T10:28:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c04ef30 con 0x7f448c066fe0 2024-04-25T10:28:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.657+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f447400bc00 con 0x7f448c066fe0 2024-04-25T10:28:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.661+0000 7f4488c00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468066f60 0x7f4468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.661+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f44740855d0 con 0x7f448c066fe0 2024-04-25T10:28:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.661+0000 7f448be00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468066f60 0x7f4468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.661+0000 7f448be00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468066f60 0x7f4468069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f447c0af000 tx=0x7f447c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.665+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44740564e0 con 0x7f448c066fe0 2024-04-25T10:28:23.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f448c14c6a0 con 0x7f448c066fe0 2024-04-25T10:28:23.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4488c00640 1 -- 172.21.15.45:0/4162869900 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f4474083e20 con 0x7f448c066fe0 2024-04-25T10:28:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:23.861 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:12.703341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:23.861 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468066f60 msgr2=0x7f4468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468066f60 0x7f4468069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f447c0af000 tx=0x7f447c008040 comp rx=0 tx=0).stop 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 msgr2=0x7f448c14c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 0x7f448c14c8c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f447400d800 tx=0x7f447400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 shutdown_connections 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468066f60 0x7f4468069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f448c06ed30 0x7f448c14ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c066fe0 0x7f448c14c8c0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 --2- 172.21.15.45:0/4162869900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c066c00 0x7f448c14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 >> 172.21.15.45:0/4162869900 conn(0x7f448c0b24f0 msgr2=0x7f448c0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:23.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 shutdown_connections 2024-04-25T10:28:23.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:23.861+0000 7f4492e00640 1 -- 172.21.15.45:0/4162869900 wait complete. 2024-04-25T10:28:23.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.021+0000 7f8d40800640 1 Processor -- start 2024-04-25T10:28:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.021+0000 7f8d40800640 1 -- start start 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d40800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d381550b0 0x7f8d380cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d40800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d3815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d40800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8d3815b740 0x7f8d3806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d40800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d38071050 con 0x7f8d381550b0 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d40800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d380712f0 con 0x7f8d3815b740 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d40800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380715d0 con 0x7f8d380cfe70 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d381550b0 0x7f8d380cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d381550b0 0x7f8d380cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54410/0 (socket says 172.21.15.45:54410) 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 -- 172.21.15.45:0/1551807927 learned_addr learned my addr 172.21.15.45:0/1551807927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3e800640 1 --2- 172.21.15.45:0/1551807927 >> v2:172.21.15.117:3300/0 conn(0x7f8d3815b740 0x7f8d3806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3d400640 1 --2- 172.21.15.45:0/1551807927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d3815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 -- 172.21.15.45:0/1551807927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 msgr2=0x7f8d3815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 --2- 172.21.15.45:0/1551807927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d3815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 -- 172.21.15.45:0/1551807927 >> v2:172.21.15.117:3300/0 conn(0x7f8d3815b740 msgr2=0x7f8d3806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 --2- 172.21.15.45:0/1551807927 >> v2:172.21.15.117:3300/0 conn(0x7f8d3815b740 0x7f8d3806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 -- 172.21.15.45:0/1551807927 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380718b0 con 0x7f8d381550b0 2024-04-25T10:28:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3e800640 1 --2- 172.21.15.45:0/1551807927 >> v2:172.21.15.117:3300/0 conn(0x7f8d3815b740 0x7f8d3806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3de00640 1 --2- 172.21.15.45:0/1551807927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d381550b0 0x7f8d380cf930 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8d380d04d0 tx=0x7f8d2c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bf37a036bd8fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3ca00640 1 -- 172.21.15.45:0/1551807927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0b0070 con 0x7f8d381550b0 2024-04-25T10:28:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3ca00640 1 -- 172.21.15.45:0/1551807927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c0ab600 con 0x7f8d381550b0 2024-04-25T10:28:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.025+0000 7f8d3ca00640 1 -- 172.21.15.45:0/1551807927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0ab9d0 con 0x7f8d381550b0 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- 172.21.15.45:0/1551807927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d381550b0 msgr2=0x7f8d380cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- 172.21.15.45:0/1551807927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d381550b0 0x7f8d380cf930 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f8d380d04d0 tx=0x7f8d2c01c260 comp rx=0 tx=0).stop 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- 172.21.15.45:0/1551807927 shutdown_connections 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- 172.21.15.45:0/1551807927 >> v2:172.21.15.117:3300/0 conn(0x7f8d3815b740 0x7f8d3806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- 172.21.15.45:0/1551807927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d3815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- 172.21.15.45:0/1551807927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d381550b0 0x7f8d380cf930 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- 172.21.15.45:0/1551807927 >> 172.21.15.45:0/1551807927 conn(0x7f8d38066b70 msgr2=0x7f8d38066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- 172.21.15.45:0/1551807927 shutdown_connections 2024-04-25T10:28:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- 172.21.15.45:0/1551807927 wait complete. 2024-04-25T10:28:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 Processor -- start 2024-04-25T10:28:24.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- start start 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d38140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d381550b0 0x7f8d38159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 0x7f8d3814ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380b3260 con 0x7f8d3815b740 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3815bd60 con 0x7f8d381550b0 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d40800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380b33e0 con 0x7f8d380cfe70 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 0x7f8d3814ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 0x7f8d3814ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54412/0 (socket says 172.21.15.45:54412) 2024-04-25T10:28:24.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 -- 172.21.15.45:0/235339263 learned_addr learned my addr 172.21.15.45:0/235339263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3d400640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d381550b0 0x7f8d38159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3de00640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d38140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 -- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 msgr2=0x7f8d38140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d38140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 -- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d381550b0 msgr2=0x7f8d38159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d381550b0 0x7f8d38159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.029+0000 7f8d3e800640 1 -- 172.21.15.45:0/235339263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d381406d0 con 0x7f8d3815b740 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d3e800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 0x7f8d3814ed30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400cd50 tx=0x7f8d3400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400f680 con 0x7f8d3815b740 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3400f840 con 0x7f8d3815b740 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400fc30 con 0x7f8d3815b740 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3815a360 con 0x7f8d3815b740 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3815a920 con 0x7f8d3815b740 2024-04-25T10:28:24.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3804ef30 con 0x7f8d3815b740 2024-04-25T10:28:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d3400e3e0 con 0x7f8d3815b740 2024-04-25T10:28:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d32000640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d1c066f60 0x7f8d1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8d34085bf0 con 0x7f8d3815b740 2024-04-25T10:28:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.033+0000 7f8d3de00640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d1c066f60 0x7f8d1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.037+0000 7f8d3de00640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d1c066f60 0x7f8d1c069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0ad000 tx=0x7f8d2c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.037+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d34014030 con 0x7f8d3815b740 2024-04-25T10:28:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.229+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d380af580 con 0x7f8d3815b740 2024-04-25T10:28:24.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d32000640 1 -- 172.21.15.45:0/235339263 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f8d34056540 con 0x7f8d3815b740 2024-04-25T10:28:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:24.232 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:27:21.291341+0000","modified":"2024-04-25T10:28:12.703341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6834/2446430072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2446430072},{"type":"v1","addr":"172.21.15.45:6834","nonce":2446430072}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:28:24.233 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 95 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d1c066f60 msgr2=0x7f8d1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d1c066f60 0x7f8d1c069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0ad000 tx=0x7f8d2c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 msgr2=0x7f8d3814ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 0x7f8d3814ed30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400cd50 tx=0x7f8d3400a650 comp rx=0 tx=0).stop 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 shutdown_connections 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8d1c066f60 0x7f8d1c069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3815b740 0x7f8d3814ed30 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d381550b0 0x7f8d38159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 --2- 172.21.15.45:0/235339263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d380cfe70 0x7f8d38140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 >> 172.21.15.45:0/235339263 conn(0x7f8d38066b70 msgr2=0x7f8d38150130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 shutdown_connections 2024-04-25T10:28:24.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.233+0000 7f8d40800640 1 -- 172.21.15.45:0/235339263 wait complete. 2024-04-25T10:28:24.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfe4e00640 1 Processor -- start 2024-04-25T10:28:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfe4e00640 1 -- start start 2024-04-25T10:28:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfe4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcfe00d07b0 0x7fcfe006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfe4e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe00ccaa0 con 0x7fcfe015df60 2024-04-25T10:28:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfe4e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe00ccd80 con 0x7fcfe00d0350 2024-04-25T10:28:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfe4e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe015e340 con 0x7fcfe00d07b0 2024-04-25T10:28:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfde800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcfe00d07b0 0x7fcfe006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfdde00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcfe015df60 0x7fcfe00cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51738/0 (socket says 172.21.15.45:51738) 2024-04-25T10:28:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfdde00640 1 -- 172.21.15.45:0/333564647 learned_addr learned my addr 172.21.15.45:0/333564647 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfde800640 1 -- 172.21.15.45:0/333564647 >> v1:172.21.15.117:6789/0 conn(0x7fcfe00d0350 legacy=0x7fcfe006aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfde800640 1 -- 172.21.15.45:0/333564647 >> v1:172.21.15.45:6789/0 conn(0x7fcfe015df60 legacy=0x7fcfe00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.393+0000 7fcfde800640 1 -- 172.21.15.45:0/333564647 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00cd060 con 0x7fcfe00d07b0 2024-04-25T10:28:24.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfde800640 1 --2- 172.21.15.45:0/333564647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe00d07b0 0x7fcfe006b5b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009d90 tx=0x7fcfd801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ceb686692852ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfdd400640 1 -- 172.21.15.45:0/333564647 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80b2070 con 0x7fcfe00d07b0 2024-04-25T10:28:24.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfdd400640 1 -- 172.21.15.45:0/333564647 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80a5720 con 0x7fcfe00d07b0 2024-04-25T10:28:24.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfdd400640 1 -- 172.21.15.45:0/333564647 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80a5af0 con 0x7fcfe00d07b0 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 -- 172.21.15.45:0/333564647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe00d07b0 msgr2=0x7fcfe006b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/333564647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe00d07b0 0x7fcfe006b5b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009d90 tx=0x7fcfd801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 -- 172.21.15.45:0/333564647 shutdown_connections 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/333564647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe00d07b0 0x7fcfe006b5b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 -- 172.21.15.45:0/333564647 >> 172.21.15.45:0/333564647 conn(0x7fcfe00d1910 msgr2=0x7fcfe00d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 -- 172.21.15.45:0/333564647 shutdown_connections 2024-04-25T10:28:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.397+0000 7fcfe4e00640 1 -- 172.21.15.45:0/333564647 wait complete. 2024-04-25T10:28:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 Processor -- start 2024-04-25T10:28:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 -- start start 2024-04-25T10:28:24.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfe00d0350 0x7fcfe00c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfe00d07b0 0x7fcfe0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 0x7fcfe0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00d0ff0 con 0x7fcfe015df60 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00d0e70 con 0x7fcfe00d07b0 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe016ba30 con 0x7fcfe00d0350 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 0x7fcfe0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 0x7fcfe0157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54420/0 (socket says 172.21.15.45:54420) 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 -- 172.21.15.45:0/76612757 learned_addr learned my addr 172.21.15.45:0/76612757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfd5400640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfe00d07b0 0x7fcfe0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfdde00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfe00d0350 0x7fcfe00c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 -- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfe00d0350 msgr2=0x7fcfe00c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfe00d0350 0x7fcfe00c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 -- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfe00d07b0 msgr2=0x7fcfe0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfe00d07b0 0x7fcfe0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 -- 172.21.15.45:0/76612757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00ce590 con 0x7fcfe015df60 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfd5400640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfe00d07b0 0x7fcfe0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:24.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfde800640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 0x7fcfe0157ac0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80b0040 tx=0x7fcfd8019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80b2050 con 0x7fcfe015df60 2024-04-25T10:28:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80be070 con 0x7fcfe015df60 2024-04-25T10:28:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80ada50 con 0x7fcfe015df60 2024-04-25T10:28:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe00b5240 con 0x7fcfe015df60 2024-04-25T10:28:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.401+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0154cc0 con 0x7fcfe015df60 2024-04-25T10:28:24.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.405+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe004ef30 con 0x7fcfe015df60 2024-04-25T10:28:24.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.405+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfd80a55a0 con 0x7fcfe015df60 2024-04-25T10:28:24.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.405+0000 7fcfd6a00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcfbc066f60 0x7fcfbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.405+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcfd8120120 con 0x7fcfe015df60 2024-04-25T10:28:24.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.405+0000 7fcfdde00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcfbc066f60 0x7fcfbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.405+0000 7fcfdde00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcfbc066f60 0x7fcfbc069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0158150 tx=0x7fcfd0007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.409+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfd8016560 con 0x7fcfe015df60 2024-04-25T10:28:24.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.573+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfe00cbab0 con 0x7fcfe015df60 2024-04-25T10:28:24.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfd6a00640 1 -- 172.21.15.45:0/76612757 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16744 (secure 0 0 0) 0x7fcfd80f10c0 con 0x7fcfe015df60 2024-04-25T10:28:24.576 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:24.576 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:10.677542+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:27:19.715814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T10:27:20.287107+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1857937743":"2024-04-26T10:28:10.677492+0000","172.21.15.45:6833/1857937743":"2024-04-26T10:28:10.677492+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcfbc066f60 msgr2=0x7fcfbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcfbc066f60 0x7fcfbc069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0158150 tx=0x7fcfd0007e90 comp rx=0 tx=0).stop 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 msgr2=0x7fcfe0157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 0x7fcfe0157ac0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80b0040 tx=0x7fcfd8019680 comp rx=0 tx=0).stop 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 shutdown_connections 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcfbc066f60 0x7fcfbc069420 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfe015df60 0x7fcfe0157ac0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfe00d07b0 0x7fcfe0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 --2- 172.21.15.45:0/76612757 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfe00d0350 0x7fcfe00c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 >> 172.21.15.45:0/76612757 conn(0x7fcfe00d1910 msgr2=0x7fcfe00b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:24.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 shutdown_connections 2024-04-25T10:28:24.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.577+0000 7fcfe4e00640 1 -- 172.21.15.45:0/76612757 wait complete. 2024-04-25T10:28:24.597 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:28:24.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.737+0000 7f0a42c00640 1 Processor -- start 2024-04-25T10:28:24.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a42c00640 1 -- start start 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a42c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a42c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0d15e0 con 0x7f0a3c0ce9a0 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a42c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0d0140 con 0x7f0a3c15f190 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a42c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0cfef0 con 0x7f0a3c15f570 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a33400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0a3c0ce9a0 0x7f0a3c0cc5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51752/0 (socket says 172.21.15.45:51752) 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a33400640 1 -- 172.21.15.45:0/3501833501 learned_addr learned my addr 172.21.15.45:0/3501833501 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:24.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a40c00640 1 --2- 172.21.15.45:0/3501833501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961153327 0 0) 0x7f0a24001120 con 0x7f0a3c0ce9a0 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a20003530 con 0x7f0a3c0ce9a0 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082362422 0 0) 0x7f0a3c0d0140 con 0x7f0a3c15f190 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a24001120 con 0x7f0a3c15f190 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a40c00640 1 -- 172.21.15.45:0/3501833501 >> v1:172.21.15.117:6789/0 conn(0x7f0a3c15f190 legacy=0x7f0a3c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a40c00640 1 -- 172.21.15.45:0/3501833501 >> v1:172.21.15.45:6789/0 conn(0x7f0a3c0ce9a0 legacy=0x7f0a3c0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.741+0000 7f0a40c00640 1 -- 172.21.15.45:0/3501833501 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0d0420 con 0x7f0a3c15f570 2024-04-25T10:28:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a40c00640 1 --2- 172.21.15.45:0/3501833501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c0ccce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009dc0 tx=0x7f0a3401c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d0844c2dfae102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340b2070 con 0x7f0a3c15f570 2024-04-25T10:28:24.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a340a5810 con 0x7f0a3c15f570 2024-04-25T10:28:24.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a3b400640 1 -- 172.21.15.45:0/3501833501 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340a5be0 con 0x7f0a3c15f570 2024-04-25T10:28:24.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 -- 172.21.15.45:0/3501833501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 msgr2=0x7f0a3c0ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3501833501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c0ccce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009dc0 tx=0x7f0a3401c840 comp rx=0 tx=0).stop 2024-04-25T10:28:24.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 -- 172.21.15.45:0/3501833501 shutdown_connections 2024-04-25T10:28:24.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3501833501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c0ccce0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 -- 172.21.15.45:0/3501833501 >> 172.21.15.45:0/3501833501 conn(0x7f0a3c0d1910 msgr2=0x7f0a3c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:24.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 -- 172.21.15.45:0/3501833501 shutdown_connections 2024-04-25T10:28:24.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 -- 172.21.15.45:0/3501833501 wait complete. 2024-04-25T10:28:24.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 Processor -- start 2024-04-25T10:28:24.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.745+0000 7f0a42c00640 1 -- start start 2024-04-25T10:28:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a3c0ce9a0 0x7f0a3c150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 0x7f0a3c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c169be0 con 0x7f0a3c15f190 2024-04-25T10:28:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c169a60 con 0x7f0a3c0ce9a0 2024-04-25T10:28:24.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c169d60 con 0x7f0a3c15f570 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 0x7f0a3c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 0x7f0a3c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54422/0 (socket says 172.21.15.45:54422) 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 -- 172.21.15.45:0/3665284417 learned_addr learned my addr 172.21.15.45:0/3665284417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a3be00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a3c0ce9a0 0x7f0a3c150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a40c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 -- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 msgr2=0x7f0a3c06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 -- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a3c0ce9a0 msgr2=0x7f0a3c150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a3c0ce9a0 0x7f0a3c150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 -- 172.21.15.45:0/3665284417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0cae70 con 0x7f0a3c15f190 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a33400640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 0x7f0a3c15aa80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400c730 tx=0x7f0a2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24017070 con 0x7f0a3c15f190 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a24005ae0 con 0x7f0a3c15f190 2024-04-25T10:28:24.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c0ca5d0 con 0x7f0a3c15f190 2024-04-25T10:28:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24007070 con 0x7f0a3c15f190 2024-04-25T10:28:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c0cab90 con 0x7f0a3c15f190 2024-04-25T10:28:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c04ef30 con 0x7f0a3c15f190 2024-04-25T10:28:24.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.749+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a2401a070 con 0x7f0a3c15f190 2024-04-25T10:28:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.753+0000 7f0a38c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a18066f60 0x7f0a18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.753+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0a24084790 con 0x7f0a3c15f190 2024-04-25T10:28:24.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.753+0000 7f0a3be00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a18066f60 0x7f0a18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:24.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.753+0000 7f0a3be00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a18066f60 0x7f0a18069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0b7700 tx=0x7f0a2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.757+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a3c0d15e0 con 0x7f0a3c15f190 2024-04-25T10:28:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.953+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0a3c0b2c40 con 0x7f0a3c15f190 2024-04-25T10:28:24.954 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:28:24.953+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:28:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a38c00640 1 -- 172.21.15.45:0/3665284417 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0a240550f0 con 0x7f0a3c15f190 2024-04-25T10:28:24.956 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:28:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a18066f60 msgr2=0x7f0a18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a18066f60 0x7f0a18069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0b7700 tx=0x7f0a2c009040 comp rx=0 tx=0).stop 2024-04-25T10:28:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 msgr2=0x7f0a3c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 0x7f0a3c15aa80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400c730 tx=0x7f0a2400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 shutdown_connections 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0a18066f60 0x7f0a18069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a3c15f570 0x7f0a3c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a3c15f190 0x7f0a3c15aa80 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 --2- 172.21.15.45:0/3665284417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a3c0ce9a0 0x7f0a3c150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 >> 172.21.15.45:0/3665284417 conn(0x7f0a3c0d1910 msgr2=0x7f0a3c06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 shutdown_connections 2024-04-25T10:28:24.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:24.957+0000 7f0a42c00640 1 -- 172.21.15.45:0/3665284417 wait complete. 2024-04-25T10:28:24.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:28:25.028 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:28:25.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.125+0000 7fbf5ca00640 1 Processor -- start 2024-04-25T10:28:25.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.125+0000 7fbf5ca00640 1 -- start start 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf5ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf580aeda0 0x7fbf58071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf58072020 0x7fbf58070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf5ca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf58159920 con 0x7fbf58159540 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf5ca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5815c2f0 con 0x7fbf580aeda0 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5815c5d0 con 0x7fbf58072020 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf580aeda0 0x7fbf58071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbf580aeda0 0x7fbf58071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41586/0 (socket says 172.21.15.45:41586) 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 -- 172.21.15.45:0/2039191992 learned_addr learned my addr 172.21.15.45:0/2039191992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf56800640 1 --2- 172.21.15.45:0/2039191992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf58072020 0x7fbf58070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 -- 172.21.15.45:0/2039191992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf58072020 msgr2=0x7fbf58070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:25.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 --2- 172.21.15.45:0/2039191992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf58072020 0x7fbf58070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 -- 172.21.15.45:0/2039191992 >> v1:172.21.15.117:6789/0 conn(0x7fbf58159540 legacy=0x7fbf580ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:25.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.129+0000 7fbf55400640 1 -- 172.21.15.45:0/2039191992 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5815c8b0 con 0x7fbf580aeda0 2024-04-25T10:28:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf55400640 1 --2- 172.21.15.45:0/2039191992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf580aeda0 0x7fbf58071ae0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009870 tx=0x7fbf4401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864b6b5b69bf303f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf54a00640 1 -- 172.21.15.45:0/2039191992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf440a9610 con 0x7fbf580aeda0 2024-04-25T10:28:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf54a00640 1 -- 172.21.15.45:0/2039191992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf440a97b0 con 0x7fbf580aeda0 2024-04-25T10:28:25.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf54a00640 1 -- 172.21.15.45:0/2039191992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf440a9b80 con 0x7fbf580aeda0 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 -- 172.21.15.45:0/2039191992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf580aeda0 msgr2=0x7fbf58071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/2039191992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf580aeda0 0x7fbf58071ae0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009870 tx=0x7fbf4401c530 comp rx=0 tx=0).stop 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 -- 172.21.15.45:0/2039191992 shutdown_connections 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/2039191992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf58072020 0x7fbf58070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/2039191992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf580aeda0 0x7fbf58071ae0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 -- 172.21.15.45:0/2039191992 >> 172.21.15.45:0/2039191992 conn(0x7fbf580669a0 msgr2=0x7fbf58066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 -- 172.21.15.45:0/2039191992 shutdown_connections 2024-04-25T10:28:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 -- 172.21.15.45:0/2039191992 wait complete. 2024-04-25T10:28:25.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 Processor -- start 2024-04-25T10:28:25.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.133+0000 7fbf5ca00640 1 -- start start 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 0x7fbf581387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf580aeda0 0x7fbf581538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58159540 0x7fbf5813b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf580710a0 con 0x7fbf58072020 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58072430 con 0x7fbf58159540 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58071220 con 0x7fbf580aeda0 2024-04-25T10:28:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 0x7fbf581387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 0x7fbf581387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41602/0 (socket says 172.21.15.45:41602) 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 -- 172.21.15.45:0/3862489408 learned_addr learned my addr 172.21.15.45:0/3862489408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf56800640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58159540 0x7fbf5813b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55400640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf580aeda0 0x7fbf581538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 -- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf580aeda0 msgr2=0x7fbf581538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf580aeda0 0x7fbf581538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 -- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58159540 msgr2=0x7fbf5813b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58159540 0x7fbf5813b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 -- 172.21.15.45:0/3862489408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58159bb0 con 0x7fbf58072020 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf55e00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 0x7fbf581387d0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800c3c0 tx=0x7fbf4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48017070 con 0x7fbf58072020 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480052d0 con 0x7fbf58072020 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48003840 con 0x7fbf58072020 2024-04-25T10:28:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf58154310 con 0x7fbf58072020 2024-04-25T10:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf581545f0 con 0x7fbf58072020 2024-04-25T10:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5804ef30 con 0x7fbf58072020 2024-04-25T10:28:25.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.137+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf4801a070 con 0x7fbf58072020 2024-04-25T10:28:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.141+0000 7fbf4e000640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf38066f60 0x7fbf38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.141+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fbf48084360 con 0x7fbf58072020 2024-04-25T10:28:25.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.141+0000 7fbf55400640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf38066f60 0x7fbf38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:25.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.141+0000 7fbf55400640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf38066f60 0x7fbf38069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbf440ae000 tx=0x7fbf440b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.145+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf48005480 con 0x7fbf58072020 2024-04-25T10:28:25.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.337+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbf580cfc30 con 0x7fbf58072020 2024-04-25T10:28:25.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.961+0000 7fbf4e000640 1 -- 172.21.15.45:0/3862489408 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf48054c60 con 0x7fbf58072020 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf38066f60 msgr2=0x7fbf38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf38066f60 0x7fbf38069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbf440ae000 tx=0x7fbf440b3040 comp rx=0 tx=0).stop 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 msgr2=0x7fbf581387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 0x7fbf581387d0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800c3c0 tx=0x7fbf4800c880 comp rx=0 tx=0).stop 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 shutdown_connections 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbf38066f60 0x7fbf38069420 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf58159540 0x7fbf5813b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf580aeda0 0x7fbf581538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 --2- 172.21.15.45:0/3862489408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58072020 0x7fbf581387d0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:25.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 >> 172.21.15.45:0/3862489408 conn(0x7fbf580669a0 msgr2=0x7fbf5815fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:25.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 shutdown_connections 2024-04-25T10:28:25.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:25.965+0000 7fbf5ca00640 1 -- 172.21.15.45:0/3862489408 wait complete. 2024-04-25T10:28:25.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:28:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597fc00640 1 Processor -- start 2024-04-25T10:28:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597fc00640 1 -- start start 2024-04-25T10:28:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597fc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 0x7f59780cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597fc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5978071f80 con 0x7f597815bc00 2024-04-25T10:28:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597fc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597815b120 con 0x7f5978071ab0 2024-04-25T10:28:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597fc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5978072260 con 0x7f59780cf040 2024-04-25T10:28:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f596fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f597815bc00 0x7f59780713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38866/0 (socket says 172.21.15.45:38866) 2024-04-25T10:28:26.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f596fe00640 1 -- 172.21.15.45:0/4098647632 learned_addr learned my addr 172.21.15.45:0/4098647632 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f597d200640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 0x7f59780cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152417550 0 0) 0x7f5978071f80 con 0x7f597815bc00 2024-04-25T10:28:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f595c003530 con 0x7f597815bc00 2024-04-25T10:28:26.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532863173 0 0) 0x7f597815b120 con 0x7f5978071ab0 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5978071f80 con 0x7f5978071ab0 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919482940 0 0) 0x7f595c003530 con 0x7f597815bc00 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 msgr2=0x7f59780cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 0x7f59780cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 >> v1:172.21.15.117:6789/0 conn(0x7f5978071ab0 legacy=0x7f59780ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597815b400 con 0x7f597815bc00 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f5960001680 con 0x7f597815bc00 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 >> v1:172.21.15.45:6789/0 conn(0x7f597815bc00 legacy=0x7f59780713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f595c003b80 0x7f595c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 0x7f595c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.133+0000 7f5977e00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c004d50 0x7f595c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c00dc90 con 0x7f595c004d50 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c00de10 con 0x7f595c003b80 2024-04-25T10:28:26.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c00dfc0 con 0x7f59780cf040 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c004d50 0x7f595c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597d200640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f595c003b80 0x7f595c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f596fe00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 0x7f595c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 -- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 msgr2=0x7f595c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59780cf040 0x7f595c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 -- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f595c003b80 msgr2=0x7f595c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f595c003b80 0x7f595c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 -- 172.21.15.45:0/4098647632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c003530 con 0x7f595c004d50 2024-04-25T10:28:26.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f597dc00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c004d50 0x7f595c0051d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5978071eb0 tx=0x7f597001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa0242ba4ce5c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:26.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970001c20 con 0x7f595c004d50 2024-04-25T10:28:26.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970003ba0 con 0x7f595c004d50 2024-04-25T10:28:26.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.137+0000 7f5977e00640 1 -- 172.21.15.45:0/4098647632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700050d0 con 0x7f595c004d50 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- 172.21.15.45:0/4098647632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c004d50 msgr2=0x7f595c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 --2- 172.21.15.45:0/4098647632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c004d50 0x7f595c0051d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5978071eb0 tx=0x7f597001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 -- 172.21.15.45:0/4098647632 reap_dead start 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- 172.21.15.45:0/4098647632 shutdown_connections 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- 172.21.15.45:0/4098647632 >> 172.21.15.45:0/4098647632 conn(0x7f5978066b70 msgr2=0x7f5978066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- 172.21.15.45:0/4098647632 shutdown_connections 2024-04-25T10:28:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- 172.21.15.45:0/4098647632 wait complete. 2024-04-25T10:28:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 Processor -- start 2024-04-25T10:28:26.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- start start 2024-04-25T10:28:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595c004d50 0x7f5978152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5978146f50 0x7f5978135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 0x7f597806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597815aec0 con 0x7f597806d130 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780b3700 con 0x7f5978146f50 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597fc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780cf420 con 0x7f595c004d50 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 0x7f597806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597d200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595c004d50 0x7f5978152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 0x7f597806d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41610/0 (socket says 172.21.15.45:41610) 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 -- 172.21.15.45:0/2688719801 learned_addr learned my addr 172.21.15.45:0/2688719801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f596fe00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5978146f50 0x7f5978135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 -- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595c004d50 msgr2=0x7f5978152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595c004d50 0x7f5978152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 -- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5978146f50 msgr2=0x7f5978135440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5978146f50 0x7f5978135440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 -- 172.21.15.45:0/2688719801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5978153320 con 0x7f597806d130 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.141+0000 7f597dc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 0x7f597806d5b0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f5970005040 tx=0x7f595c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970001c00 con 0x7f597806d130 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59700bb070 con 0x7f597806d130 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970003d50 con 0x7f597806d130 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5978153600 con 0x7f597806d130 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978135e60 con 0x7f597806d130 2024-04-25T10:28:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597804ef30 con 0x7f597806d130 2024-04-25T10:28:26.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59700b5070 con 0x7f597806d130 2024-04-25T10:28:26.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f5975600640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5954066f60 0x7f5954069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:26.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f597011ecd0 con 0x7f597806d130 2024-04-25T10:28:26.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.145+0000 7f597d200640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5954066f60 0x7f5954069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:26.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.149+0000 7f597d200640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5954066f60 0x7f5954069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5968010c80 tx=0x7f5968012040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.149+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59700165f0 con 0x7f597806d130 2024-04-25T10:28:26.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.313+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59780b2a10 con 0x7f597806d130 2024-04-25T10:28:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.961+0000 7f5975600640 1 -- 172.21.15.45:0/2688719801 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7f59700efc20 con 0x7f597806d130 2024-04-25T10:28:26.963 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:28:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.965+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5954066f60 msgr2=0x7f5954069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.965+0000 7f597fc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5954066f60 0x7f5954069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5968010c80 tx=0x7f5968012040 comp rx=0 tx=0).stop 2024-04-25T10:28:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.965+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 msgr2=0x7f597806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:26.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.965+0000 7f597fc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 0x7f597806d5b0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f5970005040 tx=0x7f595c005840 comp rx=0 tx=0).stop 2024-04-25T10:28:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 shutdown_connections 2024-04-25T10:28:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5954066f60 0x7f5954069420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f597806d130 0x7f597806d5b0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5978146f50 0x7f5978135440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 --2- 172.21.15.45:0/2688719801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f595c004d50 0x7f5978152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:26.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 >> 172.21.15.45:0/2688719801 conn(0x7f5978066b70 msgr2=0x7f59780ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:26.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 shutdown_connections 2024-04-25T10:28:26.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:26.969+0000 7f597fc00640 1 -- 172.21.15.45:0/2688719801 wait complete. 2024-04-25T10:28:26.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:28:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.145+0000 7f60b2600640 1 Processor -- start 2024-04-25T10:28:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.145+0000 7f60b2600640 1 -- start start 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60b2600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60ac15f160 0x7f60ac0b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60b2600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60ac15f540 0x7f60ac0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60b2600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ac0ce360 con 0x7f60ac0cff30 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60b2600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac0cf250 con 0x7f60ac15f160 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60b2600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ac0cd840 con 0x7f60ac15f540 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60ac15f160 0x7f60ac0b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60ac15f160 0x7f60ac0b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41624/0 (socket says 172.21.15.45:41624) 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 -- 172.21.15.45:0/320426610 learned_addr learned my addr 172.21.15.45:0/320426610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60a2a00640 1 --2- 172.21.15.45:0/320426610 >> v2:172.21.15.117:3300/0 conn(0x7f60ac15f540 0x7f60ac0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 -- 172.21.15.45:0/320426610 >> v2:172.21.15.117:3300/0 conn(0x7f60ac15f540 msgr2=0x7f60ac0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 --2- 172.21.15.45:0/320426610 >> v2:172.21.15.117:3300/0 conn(0x7f60ac15f540 0x7f60ac0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 -- 172.21.15.45:0/320426610 >> v1:172.21.15.117:6789/0 conn(0x7f60ac0cff30 legacy=0x7f60ac0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.149+0000 7f60ab400640 1 -- 172.21.15.45:0/320426610 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0ce640 con 0x7f60ac15f160 2024-04-25T10:28:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60ab400640 1 --2- 172.21.15.45:0/320426610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac0b2900 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0b3380 tx=0x7f609c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4830dddde181e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60aaa00640 1 -- 172.21.15.45:0/320426610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0aa650 con 0x7f60ac15f160 2024-04-25T10:28:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60aaa00640 1 -- 172.21.15.45:0/320426610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c0af070 con 0x7f60ac15f160 2024-04-25T10:28:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60aaa00640 1 -- 172.21.15.45:0/320426610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0aaa10 con 0x7f60ac15f160 2024-04-25T10:28:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- 172.21.15.45:0/320426610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 msgr2=0x7f60ac0b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 --2- 172.21.15.45:0/320426610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac0b2900 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0b3380 tx=0x7f609c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- 172.21.15.45:0/320426610 shutdown_connections 2024-04-25T10:28:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 --2- 172.21.15.45:0/320426610 >> v2:172.21.15.117:3300/0 conn(0x7f60ac15f540 0x7f60ac0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 --2- 172.21.15.45:0/320426610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac0b2900 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- 172.21.15.45:0/320426610 >> 172.21.15.45:0/320426610 conn(0x7f60ac0b34a0 msgr2=0x7f60ac0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- 172.21.15.45:0/320426610 shutdown_connections 2024-04-25T10:28:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- 172.21.15.45:0/320426610 wait complete. 2024-04-25T10:28:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 Processor -- start 2024-04-25T10:28:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- start start 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ac0cff30 0x7f60ac0c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ac15f540 0x7f60ac155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0cd040 con 0x7f60ac15f160 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0b47a0 con 0x7f60ac0cff30 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.153+0000 7f60b2600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ac0cd1c0 con 0x7f60ac15f540 2024-04-25T10:28:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41626/0 (socket says 172.21.15.45:41626) 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60ab400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ac0cff30 0x7f60ac0c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 -- 172.21.15.45:0/2289310068 learned_addr learned my addr 172.21.15.45:0/2289310068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60abe00640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ac15f540 0x7f60ac155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 -- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ac15f540 msgr2=0x7f60ac155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ac15f540 0x7f60ac155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 -- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ac0cff30 msgr2=0x7f60ac0c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ac0cff30 0x7f60ac0c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 -- 172.21.15.45:0/2289310068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac156c60 con 0x7f60ac15f160 2024-04-25T10:28:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a2a00640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac154f10 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f609400d600 tx=0x7f609400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60940125f0 con 0x7f60ac15f160 2024-04-25T10:28:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6094019070 con 0x7f60ac15f160 2024-04-25T10:28:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ac156f40 con 0x7f60ac15f160 2024-04-25T10:28:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60940129d0 con 0x7f60ac15f160 2024-04-25T10:28:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ac0ca180 con 0x7f60ac15f160 2024-04-25T10:28:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ac04ef30 con 0x7f60ac15f160 2024-04-25T10:28:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.157+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6094012b70 con 0x7f60ac15f160 2024-04-25T10:28:27.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.161+0000 7f60a3e00640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6088066f60 0x7f6088069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:27.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.161+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f6094084ff0 con 0x7f60ac15f160 2024-04-25T10:28:27.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.161+0000 7f60ab400640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6088066f60 0x7f6088069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:27.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.161+0000 7f60ab400640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6088066f60 0x7f6088069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f609c0ae000 tx=0x7f609c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.165+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6094023020 con 0x7f60ac15f160 2024-04-25T10:28:27.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.333+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60ac06aee0 con 0x7f60ac15f160 2024-04-25T10:28:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.969+0000 7f60a3e00640 1 -- 172.21.15.45:0/2289310068 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6094055a70 con 0x7f60ac15f160 2024-04-25T10:28:27.971 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:28:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6088066f60 msgr2=0x7f6088069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6088066f60 0x7f6088069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f609c0ae000 tx=0x7f609c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 msgr2=0x7f60ac154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac154f10 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f609400d600 tx=0x7f609400dac0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 shutdown_connections 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6088066f60 0x7f6088069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60ac15f540 0x7f60ac155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60ac15f160 0x7f60ac154f10 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 --2- 172.21.15.45:0/2289310068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60ac0cff30 0x7f60ac0c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.973+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 >> 172.21.15.45:0/2289310068 conn(0x7f60ac0b34a0 msgr2=0x7f60ac0b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.977+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 shutdown_connections 2024-04-25T10:28:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:27.977+0000 7f60b2600640 1 -- 172.21.15.45:0/2289310068 wait complete. 2024-04-25T10:28:28.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:28:28.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.177+0000 7fee8bc00640 1 Processor -- start 2024-04-25T10:28:28.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee8bc00640 1 -- start start 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee8bc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee840cc200 0x7fee840ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee8bc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840cdab0 con 0x7fee840d0d00 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee8bc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8406d8f0 con 0x7fee840cc200 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee8bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee840cd210 con 0x7fee8406cf40 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee840cc200 0x7fee840ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee840cc200 0x7fee840ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41642/0 (socket says 172.21.15.45:41642) 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 -- 172.21.15.45:0/3936813001 learned_addr learned my addr 172.21.15.45:0/3936813001 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:28.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee89c00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 -- 172.21.15.45:0/3936813001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 msgr2=0x7fee8406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 -- 172.21.15.45:0/3936813001 >> v1:172.21.15.117:6789/0 conn(0x7fee840d0d00 legacy=0x7fee84162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee7be00640 1 -- 172.21.15.45:0/3936813001 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840cd4f0 con 0x7fee840cc200 2024-04-25T10:28:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.181+0000 7fee89c00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:28.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee7be00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee840ccc30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009870 tx=0x7fee6c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294802f94856f49e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee83e00640 1 -- 172.21.15.45:0/3936813001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0b1070 con 0x7fee840cc200 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee83e00640 1 -- 172.21.15.45:0/3936813001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c0ac620 con 0x7fee840cc200 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee83e00640 1 -- 172.21.15.45:0/3936813001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0ac9f0 con 0x7fee840cc200 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 -- 172.21.15.45:0/3936813001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 msgr2=0x7fee840ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee840ccc30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009870 tx=0x7fee6c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 -- 172.21.15.45:0/3936813001 shutdown_connections 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 --2- 172.21.15.45:0/3936813001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee840ccc30 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 -- 172.21.15.45:0/3936813001 >> 172.21.15.45:0/3936813001 conn(0x7fee84072290 msgr2=0x7fee840726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 -- 172.21.15.45:0/3936813001 shutdown_connections 2024-04-25T10:28:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 -- 172.21.15.45:0/3936813001 wait complete. 2024-04-25T10:28:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 Processor -- start 2024-04-25T10:28:28.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.185+0000 7fee8bc00640 1 -- start start 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee8415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee840d0d00 0x7fee84095610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84162a10 con 0x7fee840cc200 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84162890 con 0x7fee840d0d00 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8415e040 con 0x7fee8406cf40 2024-04-25T10:28:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee8415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee8415c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41656/0 (socket says 172.21.15.45:41656) 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 -- 172.21.15.45:0/2045511487 learned_addr learned my addr 172.21.15.45:0/2045511487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee89c00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee840d0d00 0x7fee84095610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee89200640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 -- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 msgr2=0x7fee8415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8415a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 -- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee840d0d00 msgr2=0x7fee84095610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee840d0d00 0x7fee84095610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 -- 172.21.15.45:0/2045511487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8415c5e0 con 0x7fee840cc200 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee89200640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee7be00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee8415c0a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0af040 tx=0x7fee6c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0bc070 con 0x7fee840cc200 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c0b7630 con 0x7fee840cc200 2024-04-25T10:28:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8415c880 con 0x7fee840cc200 2024-04-25T10:28:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0b1040 con 0x7fee840cc200 2024-04-25T10:28:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8415ce40 con 0x7fee840cc200 2024-04-25T10:28:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8404ef30 con 0x7fee840cc200 2024-04-25T10:28:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.189+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee6c0bf070 con 0x7fee840cc200 2024-04-25T10:28:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.193+0000 7fee81600640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee60066f60 0x7fee60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.193+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fee6c11e620 con 0x7fee840cc200 2024-04-25T10:28:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.193+0000 7fee89200640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee60066f60 0x7fee60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.193+0000 7fee89200640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee60066f60 0x7fee60069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fee74001670 tx=0x7fee74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.197+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee6c016150 con 0x7fee840cc200 2024-04-25T10:28:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.361+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fee84072ff0 con 0x7fee840cc200 2024-04-25T10:28:28.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee81600640 1 -- 172.21.15.45:0/2045511487 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fee6c0ef270 con 0x7fee840cc200 2024-04-25T10:28:28.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee60066f60 msgr2=0x7fee60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee60066f60 0x7fee60069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fee74001670 tx=0x7fee74009040 comp rx=0 tx=0).stop 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 msgr2=0x7fee8415c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee8415c0a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0af040 tx=0x7fee6c019270 comp rx=0 tx=0).stop 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 shutdown_connections 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee60066f60 0x7fee60069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee840d0d00 0x7fee84095610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee840cc200 0x7fee8415c0a0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 --2- 172.21.15.45:0/2045511487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee8406cf40 0x7fee8415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 >> 172.21.15.45:0/2045511487 conn(0x7fee84072290 msgr2=0x7fee840cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 shutdown_connections 2024-04-25T10:28:28.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.365+0000 7fee8bc00640 1 -- 172.21.15.45:0/2045511487 wait complete. 2024-04-25T10:28:28.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:28:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 Processor -- start 2024-04-25T10:28:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 -- start start 2024-04-25T10:28:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f339406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33940cf930 0x7f33940cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33940b2770 con 0x7f33940cda10 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339406c670 con 0x7f33940cf930 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.525+0000 7f3399800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339406c950 con 0x7f339415bc40 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f338be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f33940cda10 0x7f33940b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38870/0 (socket says 172.21.15.45:38870) 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f338be00640 1 -- 172.21.15.45:0/907015868 learned_addr learned my addr 172.21.15.45:0/907015868 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3393400640 1 --2- 172.21.15.45:0/907015868 >> v2:172.21.15.117:3300/0 conn(0x7f33940cf930 0x7f33940cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3392a00640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f339406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3392000640 1 -- 172.21.15.45:0/907015868 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292796219 0 0) 0x7f33940b2770 con 0x7f33940cda10 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3392000640 1 -- 172.21.15.45:0/907015868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3378003530 con 0x7f33940cda10 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3393400640 1 -- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 msgr2=0x7f339406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3393400640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f339406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3393400640 1 -- 172.21.15.45:0/907015868 >> v1:172.21.15.45:6789/0 conn(0x7f33940cda10 legacy=0x7f33940b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3393400640 1 -- 172.21.15.45:0/907015868 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339406cc30 con 0x7f33940cf930 2024-04-25T10:28:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3392a00640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f339406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.529+0000 7f3393400640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f33940cfde0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603b2acd1802f136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3392000640 1 -- 172.21.15.45:0/907015868 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c0b2070 con 0x7f33940cf930 2024-04-25T10:28:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3392000640 1 -- 172.21.15.45:0/907015868 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c0a57a0 con 0x7f33940cf930 2024-04-25T10:28:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3392000640 1 -- 172.21.15.45:0/907015868 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c0a5b70 con 0x7f33940cf930 2024-04-25T10:28:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 msgr2=0x7f33940cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f33940cfde0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- 172.21.15.45:0/907015868 shutdown_connections 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f33940cfde0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 --2- 172.21.15.45:0/907015868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f339406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- 172.21.15.45:0/907015868 >> 172.21.15.45:0/907015868 conn(0x7f3394066b70 msgr2=0x7f3394066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- 172.21.15.45:0/907015868 shutdown_connections 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- 172.21.15.45:0/907015868 wait complete. 2024-04-25T10:28:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 Processor -- start 2024-04-25T10:28:28.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- start start 2024-04-25T10:28:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 0x7f339415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f3394149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f33940af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940b3810 con 0x7f33940cda10 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33940b3690 con 0x7f33940cf930 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3399800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33940b3990 con 0x7f339415bc40 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3392a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 0x7f339415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3392a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 0x7f339415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41668/0 (socket says 172.21.15.45:41668) 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3392a00640 1 -- 172.21.15.45:0/1331642817 learned_addr learned my addr 172.21.15.45:0/1331642817 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f3393400640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f33940af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.533+0000 7f338be00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f3394149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3392a00640 1 -- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 msgr2=0x7f33940af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3392a00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f33940af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3392a00640 1 -- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 msgr2=0x7f3394149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3392a00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f3394149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3392a00640 1 -- 172.21.15.45:0/1331642817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339415c3a0 con 0x7f33940cda10 2024-04-25T10:28:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3392a00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 0x7f339415d440 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f338400ebe0 tx=0x7f338400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384017070 con 0x7f33940cda10 2024-04-25T10:28:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338400cc30 con 0x7f33940cda10 2024-04-25T10:28:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339415c680 con 0x7f33940cda10 2024-04-25T10:28:28.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384013470 con 0x7f33940cda10 2024-04-25T10:28:28.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339415cc40 con 0x7f33940cda10 2024-04-25T10:28:28.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339404ef30 con 0x7f33940cda10 2024-04-25T10:28:28.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f338401a070 con 0x7f33940cda10 2024-04-25T10:28:28.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338aa00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3370066f60 0x7f3370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:28.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3384085020 con 0x7f33940cda10 2024-04-25T10:28:28.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.537+0000 7f338be00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3370066f60 0x7f3370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:28.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.541+0000 7f338be00640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3370066f60 0x7f3370069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f337c007fd0 tx=0x7f337c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:28.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.541+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3384055c80 con 0x7f33940cda10 2024-04-25T10:28:28.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.757+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f339406d960 con 0x7f33940cda10 2024-04-25T10:28:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f338aa00640 1 -- 172.21.15.45:0/1331642817 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v509) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3384059d20 con 0x7f33940cda10 2024-04-25T10:28:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3370066f60 msgr2=0x7f3370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3370066f60 0x7f3370069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f337c007fd0 tx=0x7f337c009040 comp rx=0 tx=0).stop 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 msgr2=0x7f339415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 0x7f339415d440 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f338400ebe0 tx=0x7f338400c520 comp rx=0 tx=0).stop 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 shutdown_connections 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3370066f60 0x7f3370069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f339415bc40 0x7f33940af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33940cf930 0x7f3394149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 --2- 172.21.15.45:0/1331642817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940cda10 0x7f339415d440 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.973+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 >> 172.21.15.45:0/1331642817 conn(0x7f3394066b70 msgr2=0x7f3394159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.977+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 shutdown_connections 2024-04-25T10:28:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:28.977+0000 7f3399800640 1 -- 172.21.15.45:0/1331642817 wait complete. 2024-04-25T10:28:28.996 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:28:28.996 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:28:28.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 Processor -- start 2024-04-25T10:28:29.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 -- start start 2024-04-25T10:28:29.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd3780cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3780cfe70 0x7fd37815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378071050 con 0x7fd37815b740 2024-04-25T10:28:29.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37815d040 con 0x7fd3780cfe70 2024-04-25T10:28:29.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd37d000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3780712f0 con 0x7fd3781550b0 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd375e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd3780cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd376800640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd37815b740 0x7fd37806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:34866/0 (socket says 172.21.15.45:34866) 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd36d400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3780cfe70 0x7fd37815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.137+0000 7fd376800640 1 -- 172.21.15.45:0/3479218576 learned_addr learned my addr 172.21.15.45:0/3479218576 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375400640 1 -- 172.21.15.45:0/3479218576 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518451750 0 0) 0x7fd378071050 con 0x7fd37815b740 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375400640 1 -- 172.21.15.45:0/3479218576 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003530 con 0x7fd37815b740 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375e00640 1 -- 172.21.15.45:0/3479218576 >> v2:172.21.15.117:3300/0 conn(0x7fd3780cfe70 msgr2=0x7fd37815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375e00640 1 --2- 172.21.15.45:0/3479218576 >> v2:172.21.15.117:3300/0 conn(0x7fd3780cfe70 0x7fd37815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375e00640 1 -- 172.21.15.45:0/3479218576 >> v1:172.21.15.117:6789/0 conn(0x7fd37815b740 legacy=0x7fd37806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375e00640 1 -- 172.21.15.45:0/3479218576 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780715d0 con 0x7fd3781550b0 2024-04-25T10:28:29.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd36d400640 1 --2- 172.21.15.45:0/3479218576 >> v2:172.21.15.117:3300/0 conn(0x7fd3780cfe70 0x7fd37815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:29.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375e00640 1 --2- 172.21.15.45:0/3479218576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd3780cf930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd3680097c0 tx=0x7fd36801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a9747e299865abc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375400640 1 -- 172.21.15.45:0/3479218576 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680b0070 con 0x7fd3781550b0 2024-04-25T10:28:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375400640 1 -- 172.21.15.45:0/3479218576 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3680ab600 con 0x7fd3781550b0 2024-04-25T10:28:29.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.141+0000 7fd375400640 1 -- 172.21.15.45:0/3479218576 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680ab9d0 con 0x7fd3781550b0 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- 172.21.15.45:0/3479218576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 msgr2=0x7fd3780cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 --2- 172.21.15.45:0/3479218576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd3780cf930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd3680097c0 tx=0x7fd36801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- 172.21.15.45:0/3479218576 shutdown_connections 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 --2- 172.21.15.45:0/3479218576 >> v2:172.21.15.117:3300/0 conn(0x7fd3780cfe70 0x7fd37815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 --2- 172.21.15.45:0/3479218576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd3780cf930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- 172.21.15.45:0/3479218576 >> 172.21.15.45:0/3479218576 conn(0x7fd378066b70 msgr2=0x7fd378066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- 172.21.15.45:0/3479218576 shutdown_connections 2024-04-25T10:28:29.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- 172.21.15.45:0/3479218576 wait complete. 2024-04-25T10:28:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 Processor -- start 2024-04-25T10:28:29.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- start start 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 0x7fd378153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd378152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37815b740 0x7fd37814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815bcd0 con 0x7fd3780cfe70 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815bb50 con 0x7fd37815b740 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd37d000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815c030 con 0x7fd3781550b0 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 0x7fd378153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 0x7fd378153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41680/0 (socket says 172.21.15.45:41680) 2024-04-25T10:28:29.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 -- 172.21.15.45:0/1800898554 learned_addr learned my addr 172.21.15.45:0/1800898554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd376800640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37815b740 0x7fd37814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd36d400640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd378152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 -- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 msgr2=0x7fd378152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd378152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 -- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37815b740 msgr2=0x7fd37814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37815b740 0x7fd37814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.145+0000 7fd375e00640 1 -- 172.21.15.45:0/1800898554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378153500 con 0x7fd3780cfe70 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd375e00640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 0x7fd378153c90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd3680ae040 tx=0x7fd36801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680b0050 con 0x7fd3780cfe70 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3680bc070 con 0x7fd3780cfe70 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368006e20 con 0x7fd3780cfe70 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd378150ad0 con 0x7fd3780cfe70 2024-04-25T10:28:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378151090 con 0x7fd3780cfe70 2024-04-25T10:28:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37804ef30 con 0x7fd3780cfe70 2024-04-25T10:28:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3680b5870 con 0x7fd3780cfe70 2024-04-25T10:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36ea00640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd354066f60 0x7fd354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd36811f330 con 0x7fd3780cfe70 2024-04-25T10:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.149+0000 7fd36d400640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd354066f60 0x7fd354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.153+0000 7fd36d400640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd354066f60 0x7fd354069420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd360009580 tx=0x7fd360009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.153+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd368016090 con 0x7fd3780cfe70 2024-04-25T10:28:29.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.317+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3780cfb10 con 0x7fd3780cfe70 2024-04-25T10:28:29.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.317+0000 7fd36ea00640 1 -- 172.21.15.45:0/1800898554 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+13148 (secure 0 0 0) 0x7fd3680efef0 con 0x7fd3780cfe70 2024-04-25T10:28:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:29.319 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":134,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:27.965286+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6832/2446430072":"2024-04-26T10:28:24.955757+0000","172.21.15.45:6835/1857937743":"2024-04-26T10:28:10.677492+0000","172.21.15.45:6834/2446430072":"2024-04-26T10:28:24.955757+0000","172.21.15.45:6833/1857937743":"2024-04-26T10:28:10.677492+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd354066f60 msgr2=0x7fd354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd354066f60 0x7fd354069420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd360009580 tx=0x7fd360009190 comp rx=0 tx=0).stop 2024-04-25T10:28:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 msgr2=0x7fd378153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 0x7fd378153c90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fd3680ae040 tx=0x7fd36801ace0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 shutdown_connections 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd354066f60 0x7fd354069420 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37815b740 0x7fd37814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3781550b0 0x7fd378152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 --2- 172.21.15.45:0/1800898554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd3780cfe70 0x7fd378153c90 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 >> 172.21.15.45:0/1800898554 conn(0x7fd378066b70 msgr2=0x7fd3780ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 shutdown_connections 2024-04-25T10:28:29.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.321+0000 7fd37d000640 1 -- 172.21.15.45:0/1800898554 wait complete. 2024-04-25T10:28:29.340 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:28:29.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:28:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 Processor -- start 2024-04-25T10:28:29.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 -- start start 2024-04-25T10:28:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74980aef80 0x7f749815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74980b31c0 0x7f74980b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749815b080 con 0x7f7498070350 2024-04-25T10:28:29.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749815adb0 con 0x7f74980aef80 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f74a0600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749815eb60 con 0x7f74980b31c0 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74980aef80 0x7f749815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74980aef80 0x7f749815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41688/0 (socket says 172.21.15.45:41688) 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 -- 172.21.15.45:0/4055610160 learned_addr learned my addr 172.21.15.45:0/4055610160 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749d200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7498070350 0x7f749815b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38884/0 (socket says 172.21.15.45:38884) 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749e600640 1 --2- 172.21.15.45:0/4055610160 >> v2:172.21.15.117:3300/0 conn(0x7f74980b31c0 0x7f74980b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 -- 172.21.15.45:0/4055610160 >> v2:172.21.15.117:3300/0 conn(0x7f74980b31c0 msgr2=0x7f74980b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:29.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 --2- 172.21.15.45:0/4055610160 >> v2:172.21.15.117:3300/0 conn(0x7f74980b31c0 0x7f74980b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 -- 172.21.15.45:0/4055610160 >> v1:172.21.15.45:6789/0 conn(0x7f7498070350 legacy=0x7f749815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f749dc00640 1 -- 172.21.15.45:0/4055610160 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749815ee40 con 0x7f74980aef80 2024-04-25T10:28:29.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.485+0000 7f7497e00640 1 -- 172.21.15.45:0/4055610160 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676121212 0 0) 0x7f749815b080 con 0x7f7498070350 2024-04-25T10:28:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f749dc00640 1 --2- 172.21.15.45:0/4055610160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980aef80 0x7f749815e480 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f74980d0750 tx=0x7f748c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e98c07042ddb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f7497e00640 1 -- 172.21.15.45:0/4055610160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0b4070 con 0x7f74980aef80 2024-04-25T10:28:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f7497e00640 1 -- 172.21.15.45:0/4055610160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c0a7bd0 con 0x7f74980aef80 2024-04-25T10:28:29.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f7497e00640 1 -- 172.21.15.45:0/4055610160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0af440 con 0x7f74980aef80 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 -- 172.21.15.45:0/4055610160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980aef80 msgr2=0x7f749815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 --2- 172.21.15.45:0/4055610160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980aef80 0x7f749815e480 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f74980d0750 tx=0x7f748c01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 -- 172.21.15.45:0/4055610160 shutdown_connections 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 --2- 172.21.15.45:0/4055610160 >> v2:172.21.15.117:3300/0 conn(0x7f74980b31c0 0x7f74980b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 --2- 172.21.15.45:0/4055610160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980aef80 0x7f749815e480 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 -- 172.21.15.45:0/4055610160 >> 172.21.15.45:0/4055610160 conn(0x7f74980669a0 msgr2=0x7f7498066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 -- 172.21.15.45:0/4055610160 shutdown_connections 2024-04-25T10:28:29.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 -- 172.21.15.45:0/4055610160 wait complete. 2024-04-25T10:28:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 Processor -- start 2024-04-25T10:28:29.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.489+0000 7f74a0600640 1 -- start start 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7498070350 0x7f74980ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74980aef80 0x7f7498151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 0x7f74981518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749806c250 con 0x7f74980b31c0 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749806c0d0 con 0x7f74980aef80 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749806c3d0 con 0x7f7498070350 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 0x7f74981518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 0x7f74981518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41704/0 (socket says 172.21.15.45:41704) 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 -- 172.21.15.45:0/810533983 learned_addr learned my addr 172.21.15.45:0/810533983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749d200640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74980aef80 0x7f7498151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749dc00640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7498070350 0x7f74980ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 -- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7498070350 msgr2=0x7f74980ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7498070350 0x7f74980ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 -- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74980aef80 msgr2=0x7f7498151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74980aef80 0x7f7498151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 -- 172.21.15.45:0/810533983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980a9430 con 0x7f74980b31c0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f749e600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 0x7f74981518d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7490007e60 tx=0x7f749000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490014420 con 0x7f74980b31c0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7490019070 con 0x7f74980b31c0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7498157150 con 0x7f74980b31c0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490014820 con 0x7f74980b31c0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7498157710 con 0x7f74980b31c0 2024-04-25T10:28:29.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.493+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749804ef30 con 0x7f74980b31c0 2024-04-25T10:28:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.497+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74900149c0 con 0x7f74980b31c0 2024-04-25T10:28:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.497+0000 7f7495600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7478066f60 0x7f7478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.497+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7490085540 con 0x7f74980b31c0 2024-04-25T10:28:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.497+0000 7f749dc00640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7478066f60 0x7f7478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.497+0000 7f749dc00640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7478066f60 0x7f7478069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f748c0b1000 tx=0x7f748c0a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.501+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7490056100 con 0x7f74980b31c0 2024-04-25T10:28:29.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74980b2fa0 con 0x7f74980b31c0 2024-04-25T10:28:29.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f7495600640 1 -- 172.21.15.45:0/810533983 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f749005a1a0 con 0x7f74980b31c0 2024-04-25T10:28:29.661 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:28:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7478066f60 msgr2=0x7f7478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7478066f60 0x7f7478069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f748c0b1000 tx=0x7f748c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:28:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 msgr2=0x7f74981518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 0x7f74981518d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7490007e60 tx=0x7f749000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 shutdown_connections 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7478066f60 0x7f7478069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74980b31c0 0x7f74981518d0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74980aef80 0x7f7498151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 --2- 172.21.15.45:0/810533983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7498070350 0x7f74980ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 >> 172.21.15.45:0/810533983 conn(0x7f74980669a0 msgr2=0x7f74980b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 shutdown_connections 2024-04-25T10:28:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.661+0000 7f74a0600640 1 -- 172.21.15.45:0/810533983 wait complete. 2024-04-25T10:28:29.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:28:29.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:29.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.821+0000 7f904c000640 1 Processor -- start 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.821+0000 7f904c000640 1 -- start start 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904c000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 0x7f9044166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904c000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440ccfb0 con 0x7f904415b9e0 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904c000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440cd290 con 0x7f90440ccb20 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904c000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440cd570 con 0x7f90440c6a20 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9049600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f904415b9e0 0x7f90440cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38896/0 (socket says 172.21.15.45:38896) 2024-04-25T10:28:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9049600640 1 -- 172.21.15.45:0/1352484608 learned_addr learned my addr 172.21.15.45:0/1352484608 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026537440 0 0) 0x7f90440ccfb0 con 0x7f904415b9e0 2024-04-25T10:28:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 0x7f9044166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f902c003530 con 0x7f904415b9e0 2024-04-25T10:28:29.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088246247 0 0) 0x7f902c003530 con 0x7f904415b9e0 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 msgr2=0x7f9044166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 0x7f9044166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 >> v1:172.21.15.117:6789/0 conn(0x7f90440ccb20 legacy=0x7f90440d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440cd850 con 0x7f904415b9e0 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f90380016a0 con 0x7f904415b9e0 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 >> v1:172.21.15.45:6789/0 conn(0x7f904415b9e0 legacy=0x7f90440cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f902c0039f0 0x7f902c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 0x7f902c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f902c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440cd570 con 0x7f902c004d50 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c00ba00 con 0x7f902c0039f0 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c00bc00 con 0x7f90440c6a20 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9049600640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f902c0039f0 0x7f902c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f902c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f9048c00640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 0x7f902c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 -- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 msgr2=0x7f902c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6a20 0x7f902c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 -- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f902c0039f0 msgr2=0x7f902c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f902c0039f0 0x7f902c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.825+0000 7f904a000640 1 -- 172.21.15.45:0/1352484608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c003530 con 0x7f902c004d50 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904a000640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f902c0051d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f903c009fc0 tx=0x7f903c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366d5b1a313ba04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c0b2070 con 0x7f902c004d50 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c0a5b90 con 0x7f902c004d50 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f9043e00640 1 -- 172.21.15.45:0/1352484608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c009c10 con 0x7f902c004d50 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904c000640 1 -- 172.21.15.45:0/1352484608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 msgr2=0x7f902c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904c000640 1 --2- 172.21.15.45:0/1352484608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f902c0051d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f903c009fc0 tx=0x7f903c01cc80 comp rx=0 tx=0).stop 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904a000640 1 -- 172.21.15.45:0/1352484608 reap_dead start 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904c000640 1 -- 172.21.15.45:0/1352484608 shutdown_connections 2024-04-25T10:28:29.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904c000640 1 -- 172.21.15.45:0/1352484608 >> 172.21.15.45:0/1352484608 conn(0x7f904409a9d0 msgr2=0x7f904409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904c000640 1 -- 172.21.15.45:0/1352484608 shutdown_connections 2024-04-25T10:28:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.829+0000 7f904c000640 1 -- 172.21.15.45:0/1352484608 wait complete. 2024-04-25T10:28:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 Processor -- start 2024-04-25T10:28:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 -- start start 2024-04-25T10:28:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f90441580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90440cae10 0x7f90440cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6ee0 0x7f90440c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440d1c30 con 0x7f902c004d50 2024-04-25T10:28:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440cf740 con 0x7f90440cae10 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440d1db0 con 0x7f90440c6ee0 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f90441580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f90441580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41718/0 (socket says 172.21.15.45:41718) 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 -- 172.21.15.45:0/3607782571 learned_addr learned my addr 172.21.15.45:0/3607782571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904a000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6ee0 0x7f90440c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9048c00640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90440cae10 0x7f90440cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 -- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6ee0 msgr2=0x7f90440c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6ee0 0x7f90440c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 -- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90440cae10 msgr2=0x7f90440cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90440cae10 0x7f90440cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 -- 172.21.15.45:0/3607782571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440b57d0 con 0x7f902c004d50 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9048c00640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90440cae10 0x7f90440cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9049600640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f90441580a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f9038012830 tx=0x7f9038012cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90380108c0 con 0x7f902c004d50 2024-04-25T10:28:29.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903801b070 con 0x7f902c004d50 2024-04-25T10:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038010cc0 con 0x7f902c004d50 2024-04-25T10:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440b5ab0 con 0x7f902c004d50 2024-04-25T10:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.833+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440b6070 con 0x7f902c004d50 2024-04-25T10:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.837+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904404ef30 con 0x7f902c004d50 2024-04-25T10:28:29.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.837+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9038016480 con 0x7f902c004d50 2024-04-25T10:28:29.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.837+0000 7f9041600640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9024066f60 0x7f9024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:29.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.837+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9038088220 con 0x7f902c004d50 2024-04-25T10:28:29.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.837+0000 7f9048c00640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9024066f60 0x7f9024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:29.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.837+0000 7f9048c00640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9024066f60 0x7f9024069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f90440c6210 tx=0x7f9030009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:29.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:29.841+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90380594c0 con 0x7f902c004d50 2024-04-25T10:28:30.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.061+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f90440d0a60 con 0x7f902c004d50 2024-04-25T10:28:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.981+0000 7f9041600640 1 -- 172.21.15.45:0/3607782571 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v511) v1 ==== 162+0+0 (secure 0 0 0) 0x7f9038058dc0 con 0x7f902c004d50 2024-04-25T10:28:30.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9024066f60 msgr2=0x7f9024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9024066f60 0x7f9024069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f90440c6210 tx=0x7f9030009040 comp rx=0 tx=0).stop 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 msgr2=0x7f90441580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f90441580a0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f9038012830 tx=0x7f9038012cf0 comp rx=0 tx=0).stop 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 shutdown_connections 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9024066f60 0x7f9024069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90440c6ee0 0x7f90440c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90440cae10 0x7f90440cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 --2- 172.21.15.45:0/3607782571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f902c004d50 0x7f90441580a0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 >> 172.21.15.45:0/3607782571 conn(0x7f904409a9d0 msgr2=0x7f90440cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 shutdown_connections 2024-04-25T10:28:30.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:30.985+0000 7f904c000640 1 -- 172.21.15.45:0/3607782571 wait complete. 2024-04-25T10:28:31.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:28:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 Processor -- start 2024-04-25T10:28:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 -- start start 2024-04-25T10:28:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34159540 0x7fee340ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee340aebd0 0x7fee34071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee34071360 con 0x7fee34072020 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340af030 con 0x7fee340aebd0 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.157+0000 7fee39000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34159920 con 0x7fee34159540 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee2be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee340aebd0 0x7fee34071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee32800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fee34072020 0x7fee3415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38898/0 (socket says 172.21.15.45:38898) 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee32800640 1 -- 172.21.15.45:0/1501716367 learned_addr learned my addr 172.21.15.45:0/1501716367 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee31e00640 1 --2- 172.21.15.45:0/1501716367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34159540 0x7fee340ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee31400640 1 -- 172.21.15.45:0/1501716367 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413819196 0 0) 0x7fee34071360 con 0x7fee34072020 2024-04-25T10:28:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee2be00640 1 -- 172.21.15.45:0/1501716367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34159540 msgr2=0x7fee340ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee2be00640 1 --2- 172.21.15.45:0/1501716367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34159540 0x7fee340ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee2be00640 1 -- 172.21.15.45:0/1501716367 >> v1:172.21.15.45:6789/0 conn(0x7fee34072020 legacy=0x7fee3415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee2be00640 1 -- 172.21.15.45:0/1501716367 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3415c2f0 con 0x7fee340aebd0 2024-04-25T10:28:31.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee2be00640 1 --2- 172.21.15.45:0/1501716367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee340aebd0 0x7fee34071ae0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0097c0 tx=0x7fee1c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76658fd8b1f9616b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee31400640 1 -- 172.21.15.45:0/1501716367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0b2070 con 0x7fee340aebd0 2024-04-25T10:28:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee31400640 1 -- 172.21.15.45:0/1501716367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c0ad640 con 0x7fee340aebd0 2024-04-25T10:28:31.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.161+0000 7fee31400640 1 -- 172.21.15.45:0/1501716367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0ada10 con 0x7fee340aebd0 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- 172.21.15.45:0/1501716367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee340aebd0 msgr2=0x7fee34071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 --2- 172.21.15.45:0/1501716367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee340aebd0 0x7fee34071ae0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0097c0 tx=0x7fee1c01c470 comp rx=0 tx=0).stop 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- 172.21.15.45:0/1501716367 shutdown_connections 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 --2- 172.21.15.45:0/1501716367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee340aebd0 0x7fee34071ae0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 --2- 172.21.15.45:0/1501716367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34159540 0x7fee340ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- 172.21.15.45:0/1501716367 >> 172.21.15.45:0/1501716367 conn(0x7fee340669a0 msgr2=0x7fee34066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- 172.21.15.45:0/1501716367 shutdown_connections 2024-04-25T10:28:31.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- 172.21.15.45:0/1501716367 wait complete. 2024-04-25T10:28:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 Processor -- start 2024-04-25T10:28:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- start start 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34072020 0x7fee341400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee340aebd0 0x7fee34151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 0x7fee340af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406c0d0 con 0x7fee34159540 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3415bfa0 con 0x7fee340aebd0 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee39000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34072430 con 0x7fee34072020 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 0x7fee340af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 0x7fee340af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41734/0 (socket says 172.21.15.45:41734) 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 -- 172.21.15.45:0/1094616609 learned_addr learned my addr 172.21.15.45:0/1094616609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee2be00640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee340aebd0 0x7fee34151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee31e00640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34072020 0x7fee341400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 -- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34072020 msgr2=0x7fee341400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34072020 0x7fee341400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 -- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee340aebd0 msgr2=0x7fee34151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee340aebd0 0x7fee34151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 -- 172.21.15.45:0/1094616609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34155e30 con 0x7fee34159540 2024-04-25T10:28:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.165+0000 7fee32800640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 0x7fee340af280 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c7a0 tx=0x7fee2c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c018070 con 0x7fee34159540 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00a640 con 0x7fee34159540 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00aab0 con 0x7fee34159540 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee34156110 con 0x7fee34159540 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee341517c0 con 0x7fee34159540 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3404ef30 con 0x7fee34159540 2024-04-25T10:28:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee2c00ac50 con 0x7fee34159540 2024-04-25T10:28:31.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee2a000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee10066f60 0x7fee10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fee2c0843d0 con 0x7fee34159540 2024-04-25T10:28:31.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.169+0000 7fee31e00640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee10066f60 0x7fee10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.173+0000 7fee31e00640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee10066f60 0x7fee10069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fee24009ac0 tx=0x7fee24009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.173+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee2c054f90 con 0x7fee34159540 2024-04-25T10:28:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.341+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fee34166700 con 0x7fee34159540 2024-04-25T10:28:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee2a000640 1 -- 172.21.15.45:0/1094616609 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fee2c059030 con 0x7fee34159540 2024-04-25T10:28:31.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee10066f60 msgr2=0x7fee10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee10066f60 0x7fee10069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fee24009ac0 tx=0x7fee24009190 comp rx=0 tx=0).stop 2024-04-25T10:28:31.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 msgr2=0x7fee340af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 0x7fee340af280 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c7a0 tx=0x7fee2c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 shutdown_connections 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fee10066f60 0x7fee10069420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee34159540 0x7fee340af280 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee340aebd0 0x7fee34151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 --2- 172.21.15.45:0/1094616609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee34072020 0x7fee341400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 >> 172.21.15.45:0/1094616609 conn(0x7fee340669a0 msgr2=0x7fee34070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 shutdown_connections 2024-04-25T10:28:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.345+0000 7fee39000640 1 -- 172.21.15.45:0/1094616609 wait complete. 2024-04-25T10:28:31.365 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:28:31.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:28:31.482 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:28:31.573 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:28:31.574 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:28:31.574 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:31.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:28:31.583 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:28:31.583 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:31.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:28:31.592 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:28:31.592 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:31.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:28:31.631 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:28:31.631 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:31.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:28:31.643 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:28:31.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:31.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.813+0000 7fe67bc00640 1 Processor -- start 2024-04-25T10:28:31.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.813+0000 7fe67bc00640 1 -- start start 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe67bc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6740b0f70 0x7fe6740b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe67bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740cd2e0 0x7fe6740cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe67bc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67415f3b0 con 0x7fe6740cfd90 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe67bc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67415ef20 con 0x7fe6740b0f70 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe67bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6740ca810 con 0x7fe6740cd2e0 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe66be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe6740cfd90 0x7fe6740ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38902/0 (socket says 172.21.15.45:38902) 2024-04-25T10:28:31.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe66be00640 1 -- 172.21.15.45:0/531177093 learned_addr learned my addr 172.21.15.45:0/531177093 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679200640 1 --2- 172.21.15.45:0/531177093 >> v2:172.21.15.117:3300/0 conn(0x7fe6740b0f70 0x7fe6740b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679c00640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740cd2e0 0x7fe6740cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe673e00640 1 -- 172.21.15.45:0/531177093 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000903498 0 0) 0x7fe67415f3b0 con 0x7fe6740cfd90 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe673e00640 1 -- 172.21.15.45:0/531177093 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe658003530 con 0x7fe6740cfd90 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679200640 1 -- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740cd2e0 msgr2=0x7fe6740cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679200640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740cd2e0 0x7fe6740cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe673e00640 1 -- 172.21.15.45:0/531177093 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3143039095 0 0) 0x7fe658003530 con 0x7fe6740cfd90 2024-04-25T10:28:31.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679200640 1 -- 172.21.15.45:0/531177093 >> v1:172.21.15.45:6789/0 conn(0x7fe6740cfd90 legacy=0x7fe6740ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679200640 1 -- 172.21.15.45:0/531177093 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740b1890 con 0x7fe6740b0f70 2024-04-25T10:28:31.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.817+0000 7fe679c00640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740cd2e0 0x7fe6740cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:31.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe679200640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740b0f70 0x7fe6740b1350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe6640097c0 tx=0x7fe66401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ee677393da2ff99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:31.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe673e00640 1 -- 172.21.15.45:0/531177093 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640b0070 con 0x7fe6740b0f70 2024-04-25T10:28:31.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe673e00640 1 -- 172.21.15.45:0/531177093 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6640ab400 con 0x7fe6740b0f70 2024-04-25T10:28:31.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe673e00640 1 -- 172.21.15.45:0/531177093 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c001680 con 0x7fe6740b0f70 2024-04-25T10:28:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 -- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740b0f70 msgr2=0x7fe6740b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740b0f70 0x7fe6740b1350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe6640097c0 tx=0x7fe66401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 -- 172.21.15.45:0/531177093 shutdown_connections 2024-04-25T10:28:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740cd2e0 0x7fe6740cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 --2- 172.21.15.45:0/531177093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740b0f70 0x7fe6740b1350 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 -- 172.21.15.45:0/531177093 >> 172.21.15.45:0/531177093 conn(0x7fe6740b1d50 msgr2=0x7fe6740b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:31.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 -- 172.21.15.45:0/531177093 shutdown_connections 2024-04-25T10:28:31.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 -- 172.21.15.45:0/531177093 wait complete. 2024-04-25T10:28:31.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 Processor -- start 2024-04-25T10:28:31.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.821+0000 7fe67bc00640 1 -- start start 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740b0f70 0x7fe67409dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740cd2e0 0x7fe6740ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 0x7fe674157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674161b50 con 0x7fe6740cfd90 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6741619d0 con 0x7fe6740cd2e0 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe674161cd0 con 0x7fe6740b0f70 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 0x7fe674157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 0x7fe674157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41736/0 (socket says 172.21.15.45:41736) 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 -- 172.21.15.45:0/185895895 learned_addr learned my addr 172.21.15.45:0/185895895 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe66be00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740cd2e0 0x7fe6740ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 -- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740b0f70 msgr2=0x7fe67409dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679200640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740b0f70 0x7fe67409dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740b0f70 0x7fe67409dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 -- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740cd2e0 msgr2=0x7fe6740ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740cd2e0 0x7fe6740ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 -- 172.21.15.45:0/185895895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740cbb70 con 0x7fe6740cfd90 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe66be00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740cd2e0 0x7fe6740ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe679c00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 0x7fe674157ca0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00efc0 tx=0x7fe66c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c018030 con 0x7fe6740cfd90 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c00cde0 con 0x7fe6740cfd90 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c003bc0 con 0x7fe6740cfd90 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6740b41c0 con 0x7fe6740cfd90 2024-04-25T10:28:31.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6740c5890 con 0x7fe6740cfd90 2024-04-25T10:28:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6740b46e0 con 0x7fe6740cfd90 2024-04-25T10:28:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.825+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe66c01b070 con 0x7fe6740cfd90 2024-04-25T10:28:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.829+0000 7fe671600640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe650066f90 0x7fe650069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:31.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.829+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe66c0857c0 con 0x7fe6740cfd90 2024-04-25T10:28:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.829+0000 7fe679200640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe650066f90 0x7fe650069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:31.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.829+0000 7fe679200640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe650066f90 0x7fe650069450 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe6640ad000 tx=0x7fe664008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:31.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:31.833+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe66c022070 con 0x7fe6740cfd90 2024-04-25T10:28:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.029+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6740c5680 con 0x7fe6740cfd90 2024-04-25T10:28:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.029+0000 7fe671600640 1 -- 172.21.15.45:0/185895895 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+2911 (secure 0 0 0) 0x7fe66c010040 con 0x7fe6740cfd90 2024-04-25T10:28:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:32.030 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":97,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2295227120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2295227120},{"type":"v1","addr":"172.21.15.117:6836","nonce":2295227120}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/494338063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":494338063},{"type":"v1","addr":"172.21.15.117:6837","nonce":494338063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9453,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1497366821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1497366821},{"type":"v1","addr":"172.21.15.45:6833","nonce":1497366821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[]} 2024-04-25T10:28:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 97 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe650066f90 msgr2=0x7fe650069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe650066f90 0x7fe650069450 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe6640ad000 tx=0x7fe664008040 comp rx=0 tx=0).stop 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 msgr2=0x7fe674157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 0x7fe674157ca0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00efc0 tx=0x7fe66c009f90 comp rx=0 tx=0).stop 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 shutdown_connections 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe650066f90 0x7fe650069450 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6740cfd90 0x7fe674157ca0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6740cd2e0 0x7fe6740ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 --2- 172.21.15.45:0/185895895 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6740b0f70 0x7fe67409dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 >> 172.21.15.45:0/185895895 conn(0x7fe6740b1d50 msgr2=0x7fe67415fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 shutdown_connections 2024-04-25T10:28:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.033+0000 7fe67bc00640 1 -- 172.21.15.45:0/185895895 wait complete. 2024-04-25T10:28:32.052 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:28:32.052 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:28:32.052 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:28:32.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:32.053 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:28:32.053 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:28:32.053 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:28:32.053 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:28:32.054 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:28:32.054 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:32.054 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:28:32.054 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:28:32.054 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:28:32.054 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:28:32.057 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:28:32.057 DEBUG:teuthology.parallel:result is None 2024-04-25T10:28:32.064 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:28:32.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:28:32.067 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:28:32.067 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:28:32.068 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:28:32.068 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:28:32.110 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:28:32.110 DEBUG:teuthology.parallel:result is None 2024-04-25T10:28:32.117 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:28:32.117 DEBUG:teuthology.parallel:result is None 2024-04-25T10:28:32.119 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:28:32.119 DEBUG:teuthology.parallel:result is None 2024-04-25T10:28:32.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:28:32.145 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:28:32.196 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:28:32.208 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:28:32.208 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:28:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f9280800640 1 Processor -- start 2024-04-25T10:28:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f9280800640 1 -- start start 2024-04-25T10:28:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f9280800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f927806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f9280800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92780b2770 con 0x7f92780cf930 2024-04-25T10:28:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f9280800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927806c670 con 0x7f927815bc40 2024-04-25T10:28:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f9280800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927806c950 con 0x7f92780cda10 2024-04-25T10:28:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f927e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f92780cf930 0x7f92780b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38924/0 (socket says 172.21.15.45:38924) 2024-04-25T10:28:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.265+0000 7f927e800640 1 -- 172.21.15.45:0/938041773 learned_addr learned my addr 172.21.15.45:0/938041773 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927d400640 1 --2- 172.21.15.45:0/938041773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f927806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345986511 0 0) 0x7f92780b2770 con 0x7f92780cf930 2024-04-25T10:28:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9260003530 con 0x7f92780cf930 2024-04-25T10:28:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897916957 0 0) 0x7f927806c670 con 0x7f927815bc40 2024-04-25T10:28:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92780b2770 con 0x7f927815bc40 2024-04-25T10:28:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2430653994 0 0) 0x7f9260003530 con 0x7f92780cf930 2024-04-25T10:28:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927d400640 1 -- 172.21.15.45:0/938041773 >> v1:172.21.15.117:6789/0 conn(0x7f927815bc40 legacy=0x7f92780b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927d400640 1 -- 172.21.15.45:0/938041773 >> v1:172.21.15.45:6789/0 conn(0x7f92780cf930 legacy=0x7f92780b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927d400640 1 -- 172.21.15.45:0/938041773 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927806cc30 con 0x7f92780cda10 2024-04-25T10:28:32.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:28:32.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:28:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927d400640 1 --2- 172.21.15.45:0/938041773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f927806c090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f92680097f0 tx=0x7f926801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc28db054c29e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92680b0070 con 0x7f92780cda10 2024-04-25T10:28:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92680ab400 con 0x7f92780cda10 2024-04-25T10:28:32.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.269+0000 7f927ca00640 1 -- 172.21.15.45:0/938041773 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274001a00 con 0x7f92780cda10 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- 172.21.15.45:0/938041773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 msgr2=0x7f927806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 --2- 172.21.15.45:0/938041773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f927806c090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f92680097f0 tx=0x7f926801c370 comp rx=0 tx=0).stop 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- 172.21.15.45:0/938041773 shutdown_connections 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 --2- 172.21.15.45:0/938041773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f927806c090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- 172.21.15.45:0/938041773 >> 172.21.15.45:0/938041773 conn(0x7f9278066b70 msgr2=0x7f9278066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- 172.21.15.45:0/938041773 shutdown_connections 2024-04-25T10:28:32.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- 172.21.15.45:0/938041773 wait complete. 2024-04-25T10:28:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 Processor -- start 2024-04-25T10:28:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- start start 2024-04-25T10:28:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f92780afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780cf930 0x7f92781525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 0x7f92780ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9278169d70 con 0x7f927815bc40 2024-04-25T10:28:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9278169bf0 con 0x7f92780cf930 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f9280800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9278169ef0 con 0x7f92780cda10 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 0x7f92780ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 0x7f92780ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41782/0 (socket says 172.21.15.45:41782) 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927e800640 1 -- 172.21.15.45:0/1211206546 learned_addr learned my addr 172.21.15.45:0/1211206546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927d400640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780cf930 0x7f92781525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927de00640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f92780afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927e800640 1 -- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 msgr2=0x7f92780afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927e800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f92780afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.273+0000 7f927e800640 1 -- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780cf930 msgr2=0x7f92781525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:32.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f927e800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780cf930 0x7f92781525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f927e800640 1 -- 172.21.15.45:0/1211206546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92780ae340 con 0x7f927815bc40 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f927d400640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780cf930 0x7f92781525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f927e800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 0x7f92780ade00 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9274001f40 tx=0x7f927400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274017070 con 0x7f927815bc40 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9274012400 con 0x7f927815bc40 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92780b03d0 con 0x7f927815bc40 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274012830 con 0x7f927815bc40 2024-04-25T10:28:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92780ac580 con 0x7f927815bc40 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927804ef30 con 0x7f927815bc40 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f927401a070 con 0x7f927815bc40 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9272000640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f925c066f60 0x7f925c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9274084cd0 con 0x7f927815bc40 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.277+0000 7f927de00640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f925c066f60 0x7f925c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.281+0000 7f927de00640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f925c066f60 0x7f925c069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f926c0011f0 tx=0x7f926c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.281+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9274055920 con 0x7f927815bc40 2024-04-25T10:28:32.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.457+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f927806d960 con 0x7f927815bc40 2024-04-25T10:28:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:32.997+0000 7f9272000640 1 -- 172.21.15.45:0/1211206546 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7f92740599c0 con 0x7f927815bc40 2024-04-25T10:28:32.999 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f925c066f60 msgr2=0x7f925c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f925c066f60 0x7f925c069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f926c0011f0 tx=0x7f926c009040 comp rx=0 tx=0).stop 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 msgr2=0x7f92780ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 0x7f92780ade00 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9274001f40 tx=0x7f927400a430 comp rx=0 tx=0).stop 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 shutdown_connections 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f925c066f60 0x7f925c069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927815bc40 0x7f92780ade00 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780cf930 0x7f92781525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 --2- 172.21.15.45:0/1211206546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92780cda10 0x7f92780afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 >> 172.21.15.45:0/1211206546 conn(0x7f9278066b70 msgr2=0x7f9278155cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 shutdown_connections 2024-04-25T10:28:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.001+0000 7f9280800640 1 -- 172.21.15.45:0/1211206546 wait complete. 2024-04-25T10:28:33.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:28:33.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.213+0000 7fd0bf400640 1 Processor -- start 2024-04-25T10:28:33.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.213+0000 7fd0bf400640 1 -- start start 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cda10 0x7fd0b806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bf400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b80cf930 0x7fd0b80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bf400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b80b2770 con 0x7fd0b815bc40 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bf400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b806c670 con 0x7fd0b80cf930 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bf400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b806c950 con 0x7fd0b80cda10 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b80cf930 0x7fd0b80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0b80cf930 0x7fd0b80cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41786/0 (socket says 172.21.15.45:41786) 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 -- 172.21.15.45:0/3599732810 learned_addr learned my addr 172.21.15.45:0/3599732810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:33.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0afe00640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cda10 0x7fd0b806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 -- 172.21.15.45:0/3599732810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cda10 msgr2=0x7fd0b806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cda10 0x7fd0b806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 -- 172.21.15.45:0/3599732810 >> v1:172.21.15.117:6789/0 conn(0x7fd0b815bc40 legacy=0x7fd0b80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 -- 172.21.15.45:0/3599732810 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b806cc30 con 0x7fd0b80cf930 2024-04-25T10:28:33.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0afe00640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cda10 0x7fd0b806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:33.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0bd400640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cf930 0x7fd0b80cfde0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009e50 tx=0x7fd0b001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318245907941c35a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.217+0000 7fd0b7e00640 1 -- 172.21.15.45:0/3599732810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00a5660 con 0x7fd0b80cf930 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0b7e00640 1 -- 172.21.15.45:0/3599732810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b00a5800 con 0x7fd0b80cf930 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0b7e00640 1 -- 172.21.15.45:0/3599732810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00a5bd0 con 0x7fd0b80cf930 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- 172.21.15.45:0/3599732810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cf930 msgr2=0x7fd0b80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cf930 0x7fd0b80cfde0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009e50 tx=0x7fd0b001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- 172.21.15.45:0/3599732810 shutdown_connections 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cf930 0x7fd0b80cfde0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3599732810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cda10 0x7fd0b806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- 172.21.15.45:0/3599732810 >> 172.21.15.45:0/3599732810 conn(0x7fd0b8066b70 msgr2=0x7fd0b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- 172.21.15.45:0/3599732810 shutdown_connections 2024-04-25T10:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- 172.21.15.45:0/3599732810 wait complete. 2024-04-25T10:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 Processor -- start 2024-04-25T10:28:33.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- start start 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 0x7fd0b8071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cf930 0x7fd0b81512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b815bc40 0x7fd0b80ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80b3850 con 0x7fd0b80cda10 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80b36d0 con 0x7fd0b815bc40 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bf400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80b39d0 con 0x7fd0b80cf930 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 0x7fd0b8071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 0x7fd0b8071f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41788/0 (socket says 172.21.15.45:41788) 2024-04-25T10:28:33.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 -- 172.21.15.45:0/3492661336 learned_addr learned my addr 172.21.15.45:0/3492661336 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0afe00640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cf930 0x7fd0b81512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bd400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b815bc40 0x7fd0b80ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 -- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cf930 msgr2=0x7fd0b81512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cf930 0x7fd0b81512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 -- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b815bc40 msgr2=0x7fd0b80ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b815bc40 0x7fd0b80ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.221+0000 7fd0bca00640 1 -- 172.21.15.45:0/3492661336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8070ef0 con 0x7fd0b80cda10 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0bca00640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 0x7fd0b8071f50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c3a0 tx=0x7fd0a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a80052d0 con 0x7fd0b80cda10 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a8005470 con 0x7fd0b80cda10 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8007260 con 0x7fd0b80cda10 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b8071190 con 0x7fd0b80cda10 2024-04-25T10:28:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b8071a30 con 0x7fd0b80cda10 2024-04-25T10:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b804ef30 con 0x7fd0b80cda10 2024-04-25T10:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0a8007400 con 0x7fd0b80cda10 2024-04-25T10:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0b5600640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd094066f60 0x7fd094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0a80840d0 con 0x7fd0b80cda10 2024-04-25T10:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.225+0000 7fd0afe00640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd094066f60 0x7fd094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.229+0000 7fd0afe00640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd094066f60 0x7fd094069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80cc5b0 tx=0x7fd0a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.229+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0a8054dd0 con 0x7fd0b80cda10 2024-04-25T10:28:33.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.449+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd0b80aed20 con 0x7fd0b80cda10 2024-04-25T10:28:33.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.449+0000 7fd0b5600640 1 -- 172.21.15.45:0/3492661336 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd0a8058e70 con 0x7fd0b80cda10 2024-04-25T10:28:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:28:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:28:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:28:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:28:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:28:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:28:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:28:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:28:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:28:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:28:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:28:33.465 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd094066f60 msgr2=0x7fd094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd094066f60 0x7fd094069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80cc5b0 tx=0x7fd0a0008040 comp rx=0 tx=0).stop 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 msgr2=0x7fd0b8071f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 0x7fd0b8071f50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c3a0 tx=0x7fd0a800c860 comp rx=0 tx=0).stop 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 shutdown_connections 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd094066f60 0x7fd094069420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b815bc40 0x7fd0b80ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b80cf930 0x7fd0b81512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3492661336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b80cda10 0x7fd0b8071f50 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 >> 172.21.15.45:0/3492661336 conn(0x7fd0b8066b70 msgr2=0x7fd0b8155170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 shutdown_connections 2024-04-25T10:28:33.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.453+0000 7fd0bf400640 1 -- 172.21.15.45:0/3492661336 wait complete. 2024-04-25T10:28:33.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 Processor -- start 2024-04-25T10:28:33.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 -- start start 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdcdc146ed0 0x7fdcdc1472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdcdc0726b0 0x7fdcdc071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc06c600 con 0x7fdcdc071c80 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc161980 con 0x7fdcdc0726b0 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce3e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc072080 con 0x7fdcdc146ed0 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdcdc0726b0 0x7fdcdc071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdcdc0726b0 0x7fdcdc071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41796/0 (socket says 172.21.15.45:41796) 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 -- 172.21.15.45:0/730867615 learned_addr learned my addr 172.21.15.45:0/730867615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce1400640 1 --2- 172.21.15.45:0/730867615 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc146ed0 0x7fdcdc1472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 -- 172.21.15.45:0/730867615 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc146ed0 msgr2=0x7fdcdc1472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 --2- 172.21.15.45:0/730867615 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc146ed0 0x7fdcdc1472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 -- 172.21.15.45:0/730867615 >> v1:172.21.15.117:6789/0 conn(0x7fdcdc071c80 legacy=0x7fdcdc06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.613+0000 7fdce0a00640 1 -- 172.21.15.45:0/730867615 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc161bd0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce0a00640 1 --2- 172.21.15.45:0/730867615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc071740 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0d0c80 tx=0x7fdcc801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c826c9ebb4a8aec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdcdbe00640 1 -- 172.21.15.45:0/730867615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80aa650 con 0x7fdcdc0726b0 2024-04-25T10:28:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdcdbe00640 1 -- 172.21.15.45:0/730867615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc80af070 con 0x7fdcdc0726b0 2024-04-25T10:28:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdcdbe00640 1 -- 172.21.15.45:0/730867615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80aaa10 con 0x7fdcdc0726b0 2024-04-25T10:28:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 -- 172.21.15.45:0/730867615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 msgr2=0x7fdcdc071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 --2- 172.21.15.45:0/730867615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc071740 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0d0c80 tx=0x7fdcc801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 -- 172.21.15.45:0/730867615 shutdown_connections 2024-04-25T10:28:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 --2- 172.21.15.45:0/730867615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc071740 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 --2- 172.21.15.45:0/730867615 >> v2:172.21.15.117:3300/0 conn(0x7fdcdc146ed0 0x7fdcdc1472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 -- 172.21.15.45:0/730867615 >> 172.21.15.45:0/730867615 conn(0x7fdcdc0669a0 msgr2=0x7fdcdc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 -- 172.21.15.45:0/730867615 shutdown_connections 2024-04-25T10:28:33.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.617+0000 7fdce3e00640 1 -- 172.21.15.45:0/730867615 wait complete. 2024-04-25T10:28:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 Processor -- start 2024-04-25T10:28:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 -- start start 2024-04-25T10:28:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc071c80 0x7fdcdc140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc146ed0 0x7fdcdc142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc147bf0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc147a70 con 0x7fdcdc146ed0 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0ae850 con 0x7fdcdc071c80 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc15ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41802/0 (socket says 172.21.15.45:41802) 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 -- 172.21.15.45:0/4157450795 learned_addr learned my addr 172.21.15.45:0/4157450795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce1400640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc071c80 0x7fdcdc140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce1e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc146ed0 0x7fdcdc142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 -- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc071c80 msgr2=0x7fdcdc140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc071c80 0x7fdcdc140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 -- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc146ed0 msgr2=0x7fdcdc142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc146ed0 0x7fdcdc142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 -- 172.21.15.45:0/4157450795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc13f0c0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce1e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc146ed0 0x7fdcdc142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:33.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce0a00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc15ce70 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8019100 tx=0x7fdcc801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80bd5f0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc80a6040 con 0x7fdcdc0726b0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80bd990 con 0x7fdcdc0726b0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.621+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc13f3a0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc13f960 con 0x7fdcdc0726b0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc04ef30 con 0x7fdcdc0726b0 2024-04-25T10:28:33.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcc80b33f0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdcd9600640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcbc066f60 0x7fdcbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdcc811f0f0 con 0x7fdcdc0726b0 2024-04-25T10:28:33.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdce1400640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcbc066f60 0x7fdcbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.625+0000 7fdce1400640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcbc066f60 0x7fdcbc069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc13ea90 tx=0x7fdcd0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.629+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcc80c3060 con 0x7fdcdc0726b0 2024-04-25T10:28:33.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.789+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcdc072490 con 0x7fdcdc0726b0 2024-04-25T10:28:33.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdcd9600640 1 -- 172.21.15.45:0/4157450795 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fdcc8016110 con 0x7fdcdc0726b0 2024-04-25T10:28:33.792 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:33.792 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":137,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:33.003914+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:33.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcbc066f60 msgr2=0x7fdcbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcbc066f60 0x7fdcbc069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc13ea90 tx=0x7fdcd0009190 comp rx=0 tx=0).stop 2024-04-25T10:28:33.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 msgr2=0x7fdcdc15ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc15ce70 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8019100 tx=0x7fdcc801ad30 comp rx=0 tx=0).stop 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 shutdown_connections 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcbc066f60 0x7fdcbc069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc146ed0 0x7fdcdc142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcdc0726b0 0x7fdcdc15ce70 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 --2- 172.21.15.45:0/4157450795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcdc071c80 0x7fdcdc140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 >> 172.21.15.45:0/4157450795 conn(0x7fdcdc0669a0 msgr2=0x7fdcdc13d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 shutdown_connections 2024-04-25T10:28:33.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.793+0000 7fdce3e00640 1 -- 172.21.15.45:0/4157450795 wait complete. 2024-04-25T10:28:33.814 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:28:33.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:28:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.957+0000 7f01d4c00640 1 Processor -- start 2024-04-25T10:28:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.957+0000 7f01d4c00640 1 -- start start 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.957+0000 7f01d4c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01d0162020 0x7f01d015d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.957+0000 7f01d4c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d006ac90 con 0x7f01d006bd10 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.957+0000 7f01d4c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d006af50 con 0x7f01d00cd9a0 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.957+0000 7f01d4c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d006b880 con 0x7f01d0162020 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01c5400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01d006bd10 0x7f01d0161910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38928/0 (socket says 172.21.15.45:38928) 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01c5400640 1 -- 172.21.15.45:0/628037758 learned_addr learned my addr 172.21.15.45:0/628037758 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01ce800640 1 --2- 172.21.15.45:0/628037758 >> v2:172.21.15.117:3300/0 conn(0x7f01d0162020 0x7f01d015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01cd400640 1 -- 172.21.15.45:0/628037758 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549598790 0 0) 0x7f01d006ac90 con 0x7f01d006bd10 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01cd400640 1 -- 172.21.15.45:0/628037758 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b4003530 con 0x7f01d006bd10 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01ce800640 1 -- 172.21.15.45:0/628037758 >> v1:172.21.15.117:6789/0 conn(0x7f01d00cd9a0 legacy=0x7f01d00d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01ce800640 1 -- 172.21.15.45:0/628037758 >> v1:172.21.15.45:6789/0 conn(0x7f01d006bd10 legacy=0x7f01d0161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01ce800640 1 -- 172.21.15.45:0/628037758 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d006b230 con 0x7f01d0162020 2024-04-25T10:28:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01ce800640 1 --2- 172.21.15.45:0/628037758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d015d610 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009d90 tx=0x7f01c801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84f29f6095e717a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01cd400640 1 -- 172.21.15.45:0/628037758 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80b4070 con 0x7f01d0162020 2024-04-25T10:28:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01cd400640 1 -- 172.21.15.45:0/628037758 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c80af600 con 0x7f01d0162020 2024-04-25T10:28:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.961+0000 7f01cd400640 1 -- 172.21.15.45:0/628037758 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80af9b0 con 0x7f01d0162020 2024-04-25T10:28:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- 172.21.15.45:0/628037758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 msgr2=0x7f01d015d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 --2- 172.21.15.45:0/628037758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d015d610 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009d90 tx=0x7f01c801f150 comp rx=0 tx=0).stop 2024-04-25T10:28:33.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- 172.21.15.45:0/628037758 shutdown_connections 2024-04-25T10:28:33.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 --2- 172.21.15.45:0/628037758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d015d610 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- 172.21.15.45:0/628037758 >> 172.21.15.45:0/628037758 conn(0x7f01d00d19a0 msgr2=0x7f01d00d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:33.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- 172.21.15.45:0/628037758 shutdown_connections 2024-04-25T10:28:33.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- 172.21.15.45:0/628037758 wait complete. 2024-04-25T10:28:33.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 Processor -- start 2024-04-25T10:28:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- start start 2024-04-25T10:28:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01d006bd10 0x7f01d014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 0x7f01d014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d00c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0162660 con 0x7f01d00cd9a0 2024-04-25T10:28:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d00cdd80 con 0x7f01d0162020 2024-04-25T10:28:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01d4c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d00d0d70 con 0x7f01d006bd10 2024-04-25T10:28:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01c5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 0x7f01d014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01cde00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01d006bd10 0x7f01d014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01ce800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d00c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01c5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 0x7f01d014cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41810/0 (socket says 172.21.15.45:41810) 2024-04-25T10:28:33.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01cde00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01d006bd10 0x7f01d014d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43682/0 (socket says 172.21.15.45:43682) 2024-04-25T10:28:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.965+0000 7f01c5400640 1 -- 172.21.15.45:0/1468424425 learned_addr learned my addr 172.21.15.45:0/1468424425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c5400640 1 -- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01d006bd10 msgr2=0x7f01d014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c5400640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01d006bd10 0x7f01d014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c5400640 1 -- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 msgr2=0x7f01d00c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c5400640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d00c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:33.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c5400640 1 -- 172.21.15.45:0/1468424425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d014de10 con 0x7f01d00cd9a0 2024-04-25T10:28:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c5400640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 0x7f01d014cde0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f01b800c420 tx=0x7f01b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8017070 con 0x7f01d00cd9a0 2024-04-25T10:28:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b8005cf0 con 0x7f01d00cd9a0 2024-04-25T10:28:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d00b4f50 con 0x7f01d00cd9a0 2024-04-25T10:28:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80072d0 con 0x7f01d00cd9a0 2024-04-25T10:28:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d00b54d0 con 0x7f01d00cd9a0 2024-04-25T10:28:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d004ef30 con 0x7f01d00cd9a0 2024-04-25T10:28:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.969+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01b801a070 con 0x7f01d00cd9a0 2024-04-25T10:28:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.973+0000 7f01c6a00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.973+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f01b8083fc0 con 0x7f01d00cd9a0 2024-04-25T10:28:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.973+0000 7f01cde00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.973+0000 7f01cde00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f01c0007fd0 tx=0x7f01c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:33.977+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01b8054cc0 con 0x7f01d00cd9a0 2024-04-25T10:28:34.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.137+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01d0161630 con 0x7f01d00cd9a0 2024-04-25T10:28:34.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.137+0000 7f01c6a00640 1 -- 172.21.15.45:0/1468424425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01b8058d60 con 0x7f01d00cd9a0 2024-04-25T10:28:34.139 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 msgr2=0x7f01ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f01c0007fd0 tx=0x7f01c0009040 comp rx=0 tx=0).stop 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 msgr2=0x7f01d014cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 0x7f01d014cde0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f01b800c420 tx=0x7f01b800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 shutdown_connections 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d0162020 0x7f01d00c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01d00cd9a0 0x7f01d014cde0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 --2- 172.21.15.45:0/1468424425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01d006bd10 0x7f01d014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 >> 172.21.15.45:0/1468424425 conn(0x7f01d00d19a0 msgr2=0x7f01d006f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 shutdown_connections 2024-04-25T10:28:34.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.141+0000 7f01d4c00640 1 -- 172.21.15.45:0/1468424425 wait complete. 2024-04-25T10:28:34.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:28:34.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.301+0000 7f0fee800640 1 Processor -- start 2024-04-25T10:28:34.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.301+0000 7f0fee800640 1 -- start start 2024-04-25T10:28:34.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fee800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0fe80d0040 0x7f0fe815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fee800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe8071050 con 0x7f0fe81550b0 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fee800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe815d040 con 0x7f0fe815b740 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fee800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe80712f0 con 0x7f0fe80d0040 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fe7400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0fe81550b0 0x7f0fe80cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38934/0 (socket says 172.21.15.45:38934) 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fe7400640 1 -- 172.21.15.45:0/3491510631 learned_addr learned my addr 172.21.15.45:0/3491510631 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fdfe00640 1 --2- 172.21.15.45:0/3491510631 >> v2:172.21.15.45:3300/0 conn(0x7f0fe80d0040 0x7f0fe815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fdfe00640 1 -- 172.21.15.45:0/3491510631 >> v1:172.21.15.117:6789/0 conn(0x7f0fe815b740 legacy=0x7f0fe80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fdfe00640 1 -- 172.21.15.45:0/3491510631 >> v1:172.21.15.45:6789/0 conn(0x7f0fe81550b0 legacy=0x7f0fe80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fdfe00640 1 -- 172.21.15.45:0/3491510631 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80715d0 con 0x7f0fe80d0040 2024-04-25T10:28:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.305+0000 7f0fdfe00640 1 --2- 172.21.15.45:0/3491510631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe80d0040 0x7f0fe815b200 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00097c0 tx=0x7f0fd001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7019aa7209a5562f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:34.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe6a00640 1 -- 172.21.15.45:0/3491510631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00b0070 con 0x7f0fe80d0040 2024-04-25T10:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe6a00640 1 -- 172.21.15.45:0/3491510631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd00ab600 con 0x7f0fe80d0040 2024-04-25T10:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe6a00640 1 -- 172.21.15.45:0/3491510631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00ab9d0 con 0x7f0fe80d0040 2024-04-25T10:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- 172.21.15.45:0/3491510631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe80d0040 msgr2=0x7f0fe815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 --2- 172.21.15.45:0/3491510631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe80d0040 0x7f0fe815b200 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00097c0 tx=0x7f0fd001c470 comp rx=0 tx=0).stop 2024-04-25T10:28:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- 172.21.15.45:0/3491510631 shutdown_connections 2024-04-25T10:28:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 --2- 172.21.15.45:0/3491510631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe80d0040 0x7f0fe815b200 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- 172.21.15.45:0/3491510631 >> 172.21.15.45:0/3491510631 conn(0x7f0fe8066b70 msgr2=0x7f0fe8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- 172.21.15.45:0/3491510631 shutdown_connections 2024-04-25T10:28:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- 172.21.15.45:0/3491510631 wait complete. 2024-04-25T10:28:34.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 Processor -- start 2024-04-25T10:28:34.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- start start 2024-04-25T10:28:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80d0040 0x7f0fe8153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe81550b0 0x7f0fe81fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 0x7f0fe8157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe815bcd0 con 0x7f0fe815b740 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe815bb50 con 0x7f0fe81550b0 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fee800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe81684c0 con 0x7f0fe80d0040 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 0x7f0fe8157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 0x7f0fe8157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41830/0 (socket says 172.21.15.45:41830) 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe7e00640 1 -- 172.21.15.45:0/23518953 learned_addr learned my addr 172.21.15.45:0/23518953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fe7400640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80d0040 0x7f0fe8153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.309+0000 7f0fdfe00640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe81550b0 0x7f0fe81fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7e00640 1 -- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80d0040 msgr2=0x7f0fe8153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7e00640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80d0040 0x7f0fe8153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7e00640 1 -- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe81550b0 msgr2=0x7f0fe81fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7e00640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe81550b0 0x7f0fe81fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7e00640 1 -- 172.21.15.45:0/23518953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe81fa930 con 0x7f0fe815b740 2024-04-25T10:28:34.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7e00640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 0x7f0fe8157b30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000c730 tx=0x7f0fe000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0003b80 con 0x7f0fe815b740 2024-04-25T10:28:34.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0003d20 con 0x7f0fe815b740 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000aa30 con 0x7f0fe815b740 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe81fac10 con 0x7f0fe815b740 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe8155b90 con 0x7f0fe815b740 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe804ef30 con 0x7f0fe815b740 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fe000abd0 con 0x7f0fe815b740 2024-04-25T10:28:34.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fdf400640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fc4066fe0 0x7f0fc40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0fe00849e0 con 0x7f0fe815b740 2024-04-25T10:28:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.313+0000 7f0fe7400640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fc4066fe0 0x7f0fc40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.317+0000 7f0fe7400640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fc4066fe0 0x7f0fc40694a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8005890 tx=0x7f0fd8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:34.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.317+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0fe00556d0 con 0x7f0fe815b740 2024-04-25T10:28:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.481+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0fe80b3730 con 0x7f0fe815b740 2024-04-25T10:28:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.481+0000 7f0fdf400640 1 -- 172.21.15.45:0/23518953 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0fe0059770 con 0x7f0fe815b740 2024-04-25T10:28:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:34.483 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:28:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:28:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:28:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:28:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:28:34.484 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:28:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fc4066fe0 msgr2=0x7f0fc40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fc4066fe0 0x7f0fc40694a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8005890 tx=0x7f0fd8009040 comp rx=0 tx=0).stop 2024-04-25T10:28:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 msgr2=0x7f0fe8157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 0x7f0fe8157b30 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000c730 tx=0x7f0fe000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 shutdown_connections 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0fc4066fe0 0x7f0fc40694a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fe815b740 0x7f0fe8157b30 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe81550b0 0x7f0fe81fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 --2- 172.21.15.45:0/23518953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe80d0040 0x7f0fe8153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 >> 172.21.15.45:0/23518953 conn(0x7f0fe8066b70 msgr2=0x7f0fe8153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 shutdown_connections 2024-04-25T10:28:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.485+0000 7f0fee800640 1 -- 172.21.15.45:0/23518953 wait complete. 2024-04-25T10:28:34.505 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:28:34.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:28:34.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 Processor -- start 2024-04-25T10:28:34.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 -- start start 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8e80cf040 0x7fb8e80cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e80cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e8071f80 con 0x7fb8e80713a0 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e8072260 con 0x7fb8e80cf040 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8ed000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e80b3700 con 0x7fb8e815bc00 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8e5e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8e80cf040 0x7fb8e80cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e80cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8e6800640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb8e80713a0 0x7fb8e80ae480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:34940/0 (socket says 172.21.15.45:34940) 2024-04-25T10:28:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8e5e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb8e80cf040 0x7fb8e80cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49230/0 (socket says 172.21.15.45:49230) 2024-04-25T10:28:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e80cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:43700/0 (socket says 172.21.15.45:43700) 2024-04-25T10:28:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.649+0000 7fb8e6800640 1 -- 172.21.15.45:0/1370043973 learned_addr learned my addr 172.21.15.45:0/1370043973 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5e00640 1 -- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 msgr2=0x7fb8e80cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5e00640 1 --2- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e80cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5e00640 1 -- 172.21.15.45:0/1370043973 >> v1:172.21.15.117:6789/0 conn(0x7fb8e80713a0 legacy=0x7fb8e80ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5e00640 1 -- 172.21.15.45:0/1370043973 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e815b120 con 0x7fb8e80cf040 2024-04-25T10:28:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5e00640 1 --2- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80cf040 0x7fb8e80cf9e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80097c0 tx=0x7fb8d801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6e3f7a625cd762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5400640 1 -- 172.21.15.45:0/1370043973 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80b2070 con 0x7fb8e80cf040 2024-04-25T10:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5400640 1 -- 172.21.15.45:0/1370043973 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d80ad640 con 0x7fb8e80cf040 2024-04-25T10:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8e5400640 1 -- 172.21.15.45:0/1370043973 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80ada10 con 0x7fb8e80cf040 2024-04-25T10:28:34.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8ed000640 1 -- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80cf040 msgr2=0x7fb8e80cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.653+0000 7fb8ed000640 1 --2- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80cf040 0x7fb8e80cf9e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80097c0 tx=0x7fb8d801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- 172.21.15.45:0/1370043973 shutdown_connections 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 --2- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e80cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 --2- 172.21.15.45:0/1370043973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80cf040 0x7fb8e80cf9e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- 172.21.15.45:0/1370043973 >> 172.21.15.45:0/1370043973 conn(0x7fb8e8066b70 msgr2=0x7fb8e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- 172.21.15.45:0/1370043973 shutdown_connections 2024-04-25T10:28:34.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- 172.21.15.45:0/1370043973 wait complete. 2024-04-25T10:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 Processor -- start 2024-04-25T10:28:34.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- start start 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80713a0 0x7fb8e8140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 0x7fb8e815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e80cf5a0 con 0x7fb8e80cf040 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e80cf420 con 0x7fb8e80713a0 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8ed000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e815e500 con 0x7fb8e815bc00 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 0x7fb8e815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 0x7fb8e815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41846/0 (socket says 172.21.15.45:41846) 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 -- 172.21.15.45:0/3074765239 learned_addr learned my addr 172.21.15.45:0/3074765239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8e5e00640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80713a0 0x7fb8e8140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8e6800640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 -- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 msgr2=0x7fb8e815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 -- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80713a0 msgr2=0x7fb8e8140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80713a0 0x7fb8e8140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 -- 172.21.15.45:0/3074765239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8150a80 con 0x7fb8e80cf040 2024-04-25T10:28:34.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dd400640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 0x7fb8e815c090 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e930 tx=0x7fb8d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.657+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0017070 con 0x7fb8e80cf040 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d000ca00 con 0x7fb8e80cf040 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d000cdb0 con 0x7fb8e80cf040 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e8150d60 con 0x7fb8e80cf040 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8151320 con 0x7fb8e80cf040 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e804ef30 con 0x7fb8e80cf040 2024-04-25T10:28:34.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8d001a070 con 0x7fb8e80cf040 2024-04-25T10:28:34.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8dea00640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8c4066f60 0x7fb8c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:34.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb8d0084ff0 con 0x7fb8e80cf040 2024-04-25T10:28:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.661+0000 7fb8e5e00640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8c4066f60 0x7fb8c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:34.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.665+0000 7fb8e5e00640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8c4066f60 0x7fb8c4069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80af000 tx=0x7fb8d8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:34.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.665+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8d0055dd0 con 0x7fb8e80cf040 2024-04-25T10:28:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:34.829+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb8e8070c30 con 0x7fb8e80cf040 2024-04-25T10:28:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.005+0000 7fb8dea00640 1 -- 172.21.15.45:0/3074765239 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v138) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb8d0059e70 con 0x7fb8e80cf040 2024-04-25T10:28:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8c4066f60 msgr2=0x7fb8c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8c4066f60 0x7fb8c4069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80af000 tx=0x7fb8d8008040 comp rx=0 tx=0).stop 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 msgr2=0x7fb8e815c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 0x7fb8e815c090 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e930 tx=0x7fb8d000edf0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 shutdown_connections 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8c4066f60 0x7fb8c4069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8e815bc00 0x7fb8e815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8e80cf040 0x7fb8e815c090 secure :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000e930 tx=0x7fb8d000edf0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 --2- 172.21.15.45:0/3074765239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8e80713a0 0x7fb8e8140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 >> 172.21.15.45:0/3074765239 conn(0x7fb8e8066b70 msgr2=0x7fb8e815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 shutdown_connections 2024-04-25T10:28:35.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.009+0000 7fb8ed000640 1 -- 172.21.15.45:0/3074765239 wait complete. 2024-04-25T10:28:35.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:28:35.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea60e00640 1 Processor -- start 2024-04-25T10:28:35.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea60e00640 1 -- start start 2024-04-25T10:28:35.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea60e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fea5c0b3ba0 0x7fea5c0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:35.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea60e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c15e6e0 con 0x7fea5c154f70 2024-04-25T10:28:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea60e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c15e9c0 con 0x7fea5c0d0070 2024-04-25T10:28:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea60e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5c15eca0 con 0x7fea5c0b3ba0 2024-04-25T10:28:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea59e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fea5c154f70 0x7fea5c0cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52920/0 (socket says 172.21.15.45:52920) 2024-04-25T10:28:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea59e00640 1 -- 172.21.15.45:0/3424338716 learned_addr learned my addr 172.21.15.45:0/3424338716 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea5a800640 1 --2- 172.21.15.45:0/3424338716 >> v2:172.21.15.117:3300/0 conn(0x7fea5c0b3ba0 0x7fea5c0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308555243 0 0) 0x7fea5c15e6e0 con 0x7fea5c154f70 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea40005740 con 0x7fea5c154f70 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751533235 0 0) 0x7fea5c15e9c0 con 0x7fea5c0d0070 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea5c15e6e0 con 0x7fea5c0d0070 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea5a800640 1 -- 172.21.15.45:0/3424338716 >> v1:172.21.15.117:6789/0 conn(0x7fea5c0d0070 legacy=0x7fea5c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea5a800640 1 -- 172.21.15.45:0/3424338716 >> v1:172.21.15.45:6789/0 conn(0x7fea5c154f70 legacy=0x7fea5c0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101286107 0 0) 0x7fea40005740 con 0x7fea5c154f70 2024-04-25T10:28:35.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.185+0000 7fea5a800640 1 -- 172.21.15.45:0/3424338716 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c15ef80 con 0x7fea5c0b3ba0 2024-04-25T10:28:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea5a800640 1 --2- 172.21.15.45:0/3424338716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c0ae480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fea54009d90 tx=0x7fea5401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe643147ce31ede4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c001680 con 0x7fea5c0b3ba0 2024-04-25T10:28:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea540a64e0 con 0x7fea5c0b3ba0 2024-04-25T10:28:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea58a00640 1 -- 172.21.15.45:0/3424338716 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540a68b0 con 0x7fea5c0b3ba0 2024-04-25T10:28:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 -- 172.21.15.45:0/3424338716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 msgr2=0x7fea5c0ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 --2- 172.21.15.45:0/3424338716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c0ae480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fea54009d90 tx=0x7fea5401c890 comp rx=0 tx=0).stop 2024-04-25T10:28:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 -- 172.21.15.45:0/3424338716 shutdown_connections 2024-04-25T10:28:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 --2- 172.21.15.45:0/3424338716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c0ae480 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 -- 172.21.15.45:0/3424338716 >> 172.21.15.45:0/3424338716 conn(0x7fea5c066b70 msgr2=0x7fea5c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 -- 172.21.15.45:0/3424338716 shutdown_connections 2024-04-25T10:28:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.189+0000 7fea60e00640 1 -- 172.21.15.45:0/3424338716 wait complete. 2024-04-25T10:28:35.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 Processor -- start 2024-04-25T10:28:35.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- start start 2024-04-25T10:28:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 0x7fea5c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea5c154f70 0x7fea5c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c06c4b0 con 0x7fea5c0d0070 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c06c330 con 0x7fea5c0b3ba0 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c06c630 con 0x7fea5c154f70 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 0x7fea5c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 0x7fea5c157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57948/0 (socket says 172.21.15.45:57948) 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 -- 172.21.15.45:0/589429402 learned_addr learned my addr 172.21.15.45:0/589429402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea5a800640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea5c154f70 0x7fea5c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 -- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea5c154f70 msgr2=0x7fea5c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea5c154f70 0x7fea5c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 -- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 msgr2=0x7fea5c14b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c14b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 -- 172.21.15.45:0/589429402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c14b840 con 0x7fea5c0d0070 2024-04-25T10:28:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea59400640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 0x7fea5c157c70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fea4800c490 tx=0x7fea4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea480052d0 con 0x7fea5c0d0070 2024-04-25T10:28:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea48005470 con 0x7fea5c0d0070 2024-04-25T10:28:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c0b0e20 con 0x7fea5c0d0070 2024-04-25T10:28:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48012870 con 0x7fea5c0d0070 2024-04-25T10:28:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c156ba0 con 0x7fea5c0d0070 2024-04-25T10:28:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.193+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c04ef30 con 0x7fea5c0d0070 2024-04-25T10:28:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.197+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea48012a10 con 0x7fea5c0d0070 2024-04-25T10:28:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.197+0000 7fea52000640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fea3c066f60 0x7fea3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.197+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fea48084a10 con 0x7fea5c0d0070 2024-04-25T10:28:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.197+0000 7fea59e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fea3c066f60 0x7fea3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:35.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.197+0000 7fea59e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fea3c066f60 0x7fea3c069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0011f0 tx=0x7fea4c0016a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:35.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.201+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea48055510 con 0x7fea5c0d0070 2024-04-25T10:28:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:35.369+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea5c070cf0 con 0x7fea5c0d0070 2024-04-25T10:28:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.017+0000 7fea52000640 1 -- 172.21.15.45:0/589429402 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v139) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea480595b0 con 0x7fea5c0d0070 2024-04-25T10:28:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fea3c066f60 msgr2=0x7fea3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fea3c066f60 0x7fea3c069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0011f0 tx=0x7fea4c0016a0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 msgr2=0x7fea5c157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 0x7fea5c157c70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fea4800c490 tx=0x7fea4800c950 comp rx=0 tx=0).stop 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 shutdown_connections 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fea3c066f60 0x7fea3c069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea5c154f70 0x7fea5c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea5c0d0070 0x7fea5c157c70 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 --2- 172.21.15.45:0/589429402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5c0b3ba0 0x7fea5c14b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 >> 172.21.15.45:0/589429402 conn(0x7fea5c066b70 msgr2=0x7fea5c1594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 shutdown_connections 2024-04-25T10:28:36.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.021+0000 7fea60e00640 1 -- 172.21.15.45:0/589429402 wait complete. 2024-04-25T10:28:36.062 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:28:36.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 Processor -- start 2024-04-25T10:28:36.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 -- start start 2024-04-25T10:28:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5a640aebd0 0x7f5a64071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a64071360 con 0x7f5a64072020 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640af030 con 0x7f5a640aebd0 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.209+0000 7f5a69e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64159920 con 0x7f5a64159540 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a5a000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5a640aebd0 0x7f5a64071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a62a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a63400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5a64072020 0x7f5a6415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:58122/0 (socket says 172.21.15.45:58122) 2024-04-25T10:28:36.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a63400640 1 -- 172.21.15.45:0/2716582466 learned_addr learned my addr 172.21.15.45:0/2716582466 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:36.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a5a000640 1 -- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 msgr2=0x7f5a640ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:36.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a5a000640 1 --2- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a5a000640 1 -- 172.21.15.45:0/2716582466 >> v1:172.21.15.117:6789/0 conn(0x7f5a64072020 legacy=0x7f5a6415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:36.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a5a000640 1 -- 172.21.15.45:0/2716582466 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6415c2f0 con 0x7f5a640aebd0 2024-04-25T10:28:36.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a62000640 1 -- 172.21.15.45:0/2716582466 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375707627 0 0) 0x7f5a64071360 con 0x7f5a64072020 2024-04-25T10:28:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.213+0000 7f5a5a000640 1 --2- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a64071ae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0097c0 tx=0x7f5a4c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb8f0c590ac615b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a62000640 1 -- 172.21.15.45:0/2716582466 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0b2070 con 0x7f5a640aebd0 2024-04-25T10:28:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a62000640 1 -- 172.21.15.45:0/2716582466 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c0ad640 con 0x7f5a640aebd0 2024-04-25T10:28:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a62000640 1 -- 172.21.15.45:0/2716582466 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0ada10 con 0x7f5a640aebd0 2024-04-25T10:28:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 msgr2=0x7f5a64071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 --2- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a64071ae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0097c0 tx=0x7f5a4c01c470 comp rx=0 tx=0).stop 2024-04-25T10:28:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- 172.21.15.45:0/2716582466 shutdown_connections 2024-04-25T10:28:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 --2- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a64071ae0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 --2- 172.21.15.45:0/2716582466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- 172.21.15.45:0/2716582466 >> 172.21.15.45:0/2716582466 conn(0x7f5a640669a0 msgr2=0x7f5a64066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- 172.21.15.45:0/2716582466 shutdown_connections 2024-04-25T10:28:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- 172.21.15.45:0/2716582466 wait complete. 2024-04-25T10:28:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 Processor -- start 2024-04-25T10:28:36.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- start start 2024-04-25T10:28:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 0x7f5a6414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a640cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6406c0d0 con 0x7f5a64072020 2024-04-25T10:28:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415bfa0 con 0x7f5a640aebd0 2024-04-25T10:28:36.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a69e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64072430 con 0x7f5a64159540 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a62a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 0x7f5a6414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a62a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 0x7f5a6414f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57950/0 (socket says 172.21.15.45:57950) 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.217+0000 7f5a62a00640 1 -- 172.21.15.45:0/482092764 learned_addr learned my addr 172.21.15.45:0/482092764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a5a000640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a640cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a63400640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a62a00640 1 -- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 msgr2=0x7f5a640cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a62a00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a62a00640 1 -- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 msgr2=0x7f5a640cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a62a00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a640cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a62a00640 1 -- 172.21.15.45:0/482092764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640b4770 con 0x7f5a64072020 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a63400640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a62a00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 0x7f5a6414f6d0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400efc0 tx=0x7f5a5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54018020 con 0x7f5a64072020 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54014480 con 0x7f5a64072020 2024-04-25T10:28:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a64150bf0 con 0x7f5a64072020 2024-04-25T10:28:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54014850 con 0x7f5a64072020 2024-04-25T10:28:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a641511b0 con 0x7f5a64072020 2024-04-25T10:28:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6404ef30 con 0x7f5a64072020 2024-04-25T10:28:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a5401b020 con 0x7f5a64072020 2024-04-25T10:28:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.221+0000 7f5a5be00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a40066f60 0x7f5a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.225+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5a54086290 con 0x7f5a64072020 2024-04-25T10:28:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.225+0000 7f5a5a000640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a40066f60 0x7f5a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.225+0000 7f5a5a000640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a40066f60 0x7f5a40069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0af000 tx=0x7f5a4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.225+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a54022070 con 0x7f5a64072020 2024-04-25T10:28:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:36.421+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5a640cf470 con 0x7f5a64072020 2024-04-25T10:28:37.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.021+0000 7f5a5be00640 1 -- 172.21.15.45:0/482092764 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v100) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5a54010040 con 0x7f5a64072020 2024-04-25T10:28:37.021 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T10:28:37.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a40066f60 msgr2=0x7f5a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a40066f60 0x7f5a40069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0af000 tx=0x7f5a4c008040 comp rx=0 tx=0).stop 2024-04-25T10:28:37.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 msgr2=0x7f5a6414f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 0x7f5a6414f6d0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400efc0 tx=0x7f5a5400c490 comp rx=0 tx=0).stop 2024-04-25T10:28:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 shutdown_connections 2024-04-25T10:28:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5a40066f60 0x7f5a40069420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a64159540 0x7f5a640cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a640aebd0 0x7f5a640cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 --2- 172.21.15.45:0/482092764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a64072020 0x7f5a6414f6d0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 >> 172.21.15.45:0/482092764 conn(0x7f5a640669a0 msgr2=0x7f5a64066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:37.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.025+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 shutdown_connections 2024-04-25T10:28:37.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.029+0000 7f5a69e00640 1 -- 172.21.15.45:0/482092764 wait complete. 2024-04-25T10:28:37.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 Processor -- start 2024-04-25T10:28:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 -- start start 2024-04-25T10:28:37.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f999406af30 0x7f99940b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f999406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99940cec70 con 0x7f99940b0b80 2024-04-25T10:28:37.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99940d27b0 con 0x7f999406af30 2024-04-25T10:28:37.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b26d0 con 0x7f99940aed20 2024-04-25T10:28:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f9999c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f999406af30 0x7f99940b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999a600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f999406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f9999c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f999406af30 0x7f99940b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37410/0 (socket says 172.21.15.45:37410) 2024-04-25T10:28:37.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f9999c00640 1 -- 172.21.15.45:0/3666021424 learned_addr learned my addr 172.21.15.45:0/3666021424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.225+0000 7f999a600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f999406b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:53784/0 (socket says 172.21.15.45:53784) 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9999c00640 1 -- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 msgr2=0x7f999406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9999c00640 1 --2- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f999406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9999c00640 1 -- 172.21.15.45:0/3666021424 >> v1:172.21.15.117:6789/0 conn(0x7f99940b0b80 legacy=0x7f99940ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9999c00640 1 -- 172.21.15.45:0/3666021424 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940b1da0 con 0x7f999406af30 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9993e00640 1 -- 172.21.15.45:0/3666021424 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398721519 0 0) 0x7f99940cec70 con 0x7f99940b0b80 2024-04-25T10:28:37.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999a600640 1 --2- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f999406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:37.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9999c00640 1 --2- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f999406af30 0x7f99940b2b60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9988009870 tx=0x7f998801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c797c140d9c53b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9993e00640 1 -- 172.21.15.45:0/3666021424 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880aa600 con 0x7f999406af30 2024-04-25T10:28:37.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9993e00640 1 -- 172.21.15.45:0/3666021424 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880af070 con 0x7f999406af30 2024-04-25T10:28:37.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f9993e00640 1 -- 172.21.15.45:0/3666021424 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880aa9c0 con 0x7f999406af30 2024-04-25T10:28:37.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 -- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f999406af30 msgr2=0x7f99940b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 --2- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f999406af30 0x7f99940b2b60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9988009870 tx=0x7f998801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 -- 172.21.15.45:0/3666021424 shutdown_connections 2024-04-25T10:28:37.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 --2- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f999406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 --2- 172.21.15.45:0/3666021424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f999406af30 0x7f99940b2b60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 -- 172.21.15.45:0/3666021424 >> 172.21.15.45:0/3666021424 conn(0x7f99940d1900 msgr2=0x7f99940d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:37.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 -- 172.21.15.45:0/3666021424 shutdown_connections 2024-04-25T10:28:37.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.229+0000 7f999c600640 1 -- 172.21.15.45:0/3666021424 wait complete. 2024-04-25T10:28:37.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 Processor -- start 2024-04-25T10:28:37.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 -- start start 2024-04-25T10:28:37.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 0x7f99940c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f99941530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99940b0b80 0x7f99940c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b3220 con 0x7f999406af30 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b30a0 con 0x7f99940b0b80 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940cfbd0 con 0x7f99940aed20 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 0x7f99940c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 0x7f99940c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57960/0 (socket says 172.21.15.45:57960) 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 -- 172.21.15.45:0/1501866954 learned_addr learned my addr 172.21.15.45:0/1501866954 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999200640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f99941530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999a600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99940b0b80 0x7f99940c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 -- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 msgr2=0x7f99941530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f99941530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 -- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99940b0b80 msgr2=0x7f99940c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99940b0b80 0x7f99940c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 -- 172.21.15.45:0/1501866954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99941559e0 con 0x7f999406af30 2024-04-25T10:28:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999a600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99940b0b80 0x7f99940c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999200640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f99941530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:37.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9999c00640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 0x7f99940c5360 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f99880ac040 tx=0x7f998801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880bd5f0 con 0x7f999406af30 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880aae50 con 0x7f999406af30 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880bd9f0 con 0x7f999406af30 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.233+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9994156fb0 con 0x7f999406af30 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9994157570 con 0x7f999406af30 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999404ef30 con 0x7f999406af30 2024-04-25T10:28:37.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99880a9070 con 0x7f999406af30 2024-04-25T10:28:37.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f9991600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9974066f60 0x7f9974069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f998811eff0 con 0x7f999406af30 2024-04-25T10:28:37.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f9999200640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9974066f60 0x7f9974069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.237+0000 7f9999200640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9974066f60 0x7f9974069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9994159330 tx=0x7f9980009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.241+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9988016150 con 0x7f999406af30 2024-04-25T10:28:37.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.409+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99940cf6b0 con 0x7f999406af30 2024-04-25T10:28:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.409+0000 7f9991600640 1 -- 172.21.15.45:0/1501866954 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f99880ef970 con 0x7f999406af30 2024-04-25T10:28:37.410 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:37.410 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":140,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:36.424782+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9974066f60 msgr2=0x7f9974069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9974066f60 0x7f9974069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9994159330 tx=0x7f9980009040 comp rx=0 tx=0).stop 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 msgr2=0x7f99940c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 0x7f99940c5360 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f99880ac040 tx=0x7f998801ace0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 shutdown_connections 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9974066f60 0x7f9974069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99940b0b80 0x7f99940c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99940aed20 0x7f99941530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 --2- 172.21.15.45:0/1501866954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f999406af30 0x7f99940c5360 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 >> 172.21.15.45:0/1501866954 conn(0x7f99940d1900 msgr2=0x7f999406f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:37.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 shutdown_connections 2024-04-25T10:28:37.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.413+0000 7f999c600640 1 -- 172.21.15.45:0/1501866954 wait complete. 2024-04-25T10:28:37.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:37.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 Processor -- start 2024-04-25T10:28:37.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 -- start start 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09b0066c00 0x7f09b015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b0066fe0 0x7f09b00b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00b5c20 con 0x7f09b006ed30 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00af6e0 con 0x7f09b0066c00 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09b5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00af9c0 con 0x7f09b0066fe0 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09ae800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f09b0066c00 0x7f09b015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09af200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f09b006ed30 0x7f09b0157b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52922/0 (socket says 172.21.15.45:52922) 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09af200640 1 -- 172.21.15.45:0/555771505 learned_addr learned my addr 172.21.15.45:0/555771505 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:37.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09a5e00640 1 --2- 172.21.15.45:0/555771505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b0066fe0 0x7f09b00b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09ae800640 1 -- 172.21.15.45:0/555771505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b0066fe0 msgr2=0x7f09b00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:37.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09ae800640 1 --2- 172.21.15.45:0/555771505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b0066fe0 0x7f09b00b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09ae800640 1 -- 172.21.15.45:0/555771505 >> v1:172.21.15.45:6789/0 conn(0x7f09b006ed30 legacy=0x7f09b0157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09ae800640 1 -- 172.21.15.45:0/555771505 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00afca0 con 0x7f09b0066c00 2024-04-25T10:28:37.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.573+0000 7f09ade00640 1 -- 172.21.15.45:0/555771505 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061613852 0 0) 0x7f09b00b5c20 con 0x7f09b006ed30 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09ae800640 1 --2- 172.21.15.45:0/555771505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066c00 0x7f09b015e110 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f09a00097c0 tx=0x7f09a001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223b60d4c7b76156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09ade00640 1 -- 172.21.15.45:0/555771505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00aa5f0 con 0x7f09b0066c00 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09ade00640 1 -- 172.21.15.45:0/555771505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a00af070 con 0x7f09b0066c00 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09ade00640 1 -- 172.21.15.45:0/555771505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00aa9b0 con 0x7f09b0066c00 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 -- 172.21.15.45:0/555771505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066c00 msgr2=0x7f09b015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 --2- 172.21.15.45:0/555771505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066c00 0x7f09b015e110 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f09a00097c0 tx=0x7f09a001c470 comp rx=0 tx=0).stop 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 -- 172.21.15.45:0/555771505 shutdown_connections 2024-04-25T10:28:37.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 --2- 172.21.15.45:0/555771505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b0066fe0 0x7f09b00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 --2- 172.21.15.45:0/555771505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066c00 0x7f09b015e110 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 -- 172.21.15.45:0/555771505 >> 172.21.15.45:0/555771505 conn(0x7f09b00b24f0 msgr2=0x7f09b00b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:37.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 -- 172.21.15.45:0/555771505 shutdown_connections 2024-04-25T10:28:37.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.577+0000 7f09b5400640 1 -- 172.21.15.45:0/555771505 wait complete. 2024-04-25T10:28:37.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 Processor -- start 2024-04-25T10:28:37.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 -- start start 2024-04-25T10:28:37.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09b0066c00 0x7f09b015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 0x7f09b014c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b006ed30 0x7f09b0158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b006ac90 con 0x7f09b0066fe0 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00af270 con 0x7f09b0066c00 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b006ae10 con 0x7f09b006ed30 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 0x7f09b014c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 0x7f09b014c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57986/0 (socket says 172.21.15.45:57986) 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 -- 172.21.15.45:0/744614619 learned_addr learned my addr 172.21.15.45:0/744614619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09af200640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b006ed30 0x7f09b0158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09ae800640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09b0066c00 0x7f09b015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 -- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b006ed30 msgr2=0x7f09b0158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b006ed30 0x7f09b0158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 -- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09b0066c00 msgr2=0x7f09b015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09b0066c00 0x7f09b015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 -- 172.21.15.45:0/744614619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0159100 con 0x7f09b0066fe0 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a5e00640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 0x7f09b014c700 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f099800e930 tx=0x7f099800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998017070 con 0x7f09b0066fe0 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099800ca00 con 0x7f09b0066fe0 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800cdb0 con 0x7f09b0066fe0 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b01593e0 con 0x7f09b0066fe0 2024-04-25T10:28:37.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.581+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b01599a0 con 0x7f09b0066fe0 2024-04-25T10:28:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.585+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b004ef30 con 0x7f09b0066fe0 2024-04-25T10:28:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.585+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f099801a070 con 0x7f09b0066fe0 2024-04-25T10:28:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.585+0000 7f09a7400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f098c066f60 0x7f098c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.585+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0998085100 con 0x7f09b0066fe0 2024-04-25T10:28:37.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.585+0000 7f09ae800640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f098c066f60 0x7f098c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.585+0000 7f09ae800640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f098c066f60 0x7f098c069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f09a00ae000 tx=0x7f09a00a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.589+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0998056130 con 0x7f09b0066fe0 2024-04-25T10:28:37.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.753+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09b0071c70 con 0x7f09b0066fe0 2024-04-25T10:28:37.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.753+0000 7f09a7400640 1 -- 172.21.15.45:0/744614619 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f0998055a30 con 0x7f09b0066fe0 2024-04-25T10:28:37.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:37.754 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":140,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:36.424782+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f098c066f60 msgr2=0x7f098c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f098c066f60 0x7f098c069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f09a00ae000 tx=0x7f09a00a7000 comp rx=0 tx=0).stop 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 msgr2=0x7f09b014c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 0x7f09b014c700 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f099800e930 tx=0x7f099800edf0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 shutdown_connections 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f098c066f60 0x7f098c069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09b006ed30 0x7f09b0158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09b0066fe0 0x7f09b014c700 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 --2- 172.21.15.45:0/744614619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09b0066c00 0x7f09b015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 >> 172.21.15.45:0/744614619 conn(0x7f09b00b24f0 msgr2=0x7f09b006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:37.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 shutdown_connections 2024-04-25T10:28:37.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.757+0000 7f09b5400640 1 -- 172.21.15.45:0/744614619 wait complete. 2024-04-25T10:28:37.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:28:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 Processor -- start 2024-04-25T10:28:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 -- start start 2024-04-25T10:28:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb84c15e130 0x7fb84c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb84c071bc0 0x7fb84c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c0ad840 con 0x7fb84c0b3480 2024-04-25T10:28:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c072570 con 0x7fb84c15e130 2024-04-25T10:28:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb854800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c0d0090 con 0x7fb84c071bc0 2024-04-25T10:28:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb84c15e130 0x7fb84c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb84c15e130 0x7fb84c0cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57994/0 (socket says 172.21.15.45:57994) 2024-04-25T10:28:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 -- 172.21.15.45:0/2020592146 learned_addr learned my addr 172.21.15.45:0/2020592146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb852800640 1 --2- 172.21.15.45:0/2020592146 >> v2:172.21.15.117:3300/0 conn(0x7fb84c071bc0 0x7fb84c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 -- 172.21.15.45:0/2020592146 >> v2:172.21.15.117:3300/0 conn(0x7fb84c071bc0 msgr2=0x7fb84c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 --2- 172.21.15.45:0/2020592146 >> v2:172.21.15.117:3300/0 conn(0x7fb84c071bc0 0x7fb84c072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 -- 172.21.15.45:0/2020592146 >> v1:172.21.15.117:6789/0 conn(0x7fb84c0b3480 legacy=0x7fb84c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.921+0000 7fb851400640 1 -- 172.21.15.45:0/2020592146 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c15e710 con 0x7fb84c15e130 2024-04-25T10:28:37.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb851400640 1 --2- 172.21.15.45:0/2020592146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c15e130 0x7fb84c0cfb50 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0b4c40 tx=0x7fb83c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb2f6e7961788e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb850a00640 1 -- 172.21.15.45:0/2020592146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0b0070 con 0x7fb84c15e130 2024-04-25T10:28:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb850a00640 1 -- 172.21.15.45:0/2020592146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c0ab600 con 0x7fb84c15e130 2024-04-25T10:28:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb850a00640 1 -- 172.21.15.45:0/2020592146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0ab9d0 con 0x7fb84c15e130 2024-04-25T10:28:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 -- 172.21.15.45:0/2020592146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c15e130 msgr2=0x7fb84c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 --2- 172.21.15.45:0/2020592146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c15e130 0x7fb84c0cfb50 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0b4c40 tx=0x7fb83c01c4c0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 -- 172.21.15.45:0/2020592146 shutdown_connections 2024-04-25T10:28:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 --2- 172.21.15.45:0/2020592146 >> v2:172.21.15.117:3300/0 conn(0x7fb84c071bc0 0x7fb84c072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 --2- 172.21.15.45:0/2020592146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c15e130 0x7fb84c0cfb50 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 -- 172.21.15.45:0/2020592146 >> 172.21.15.45:0/2020592146 conn(0x7fb84c0669a0 msgr2=0x7fb84c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 -- 172.21.15.45:0/2020592146 shutdown_connections 2024-04-25T10:28:37.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.925+0000 7fb854800640 1 -- 172.21.15.45:0/2020592146 wait complete. 2024-04-25T10:28:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 Processor -- start 2024-04-25T10:28:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 -- start start 2024-04-25T10:28:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb84c071bc0 0x7fb84c149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 0x7fb84c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb84c15e130 0x7fb84c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0ae530 con 0x7fb84c0b3480 2024-04-25T10:28:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c15bb90 con 0x7fb84c15e130 2024-04-25T10:28:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0ae6b0 con 0x7fb84c071bc0 2024-04-25T10:28:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 0x7fb84c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 0x7fb84c156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58010/0 (socket says 172.21.15.45:58010) 2024-04-25T10:28:37.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 -- 172.21.15.45:0/706319558 learned_addr learned my addr 172.21.15.45:0/706319558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:37.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb852800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb84c15e130 0x7fb84c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851e00640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb84c071bc0 0x7fb84c149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 -- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb84c071bc0 msgr2=0x7fb84c149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb84c071bc0 0x7fb84c149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 -- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb84c15e130 msgr2=0x7fb84c159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:37.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb84c15e130 0x7fb84c159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:37.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 -- 172.21.15.45:0/706319558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c157d50 con 0x7fb84c0b3480 2024-04-25T10:28:37.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb851400640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 0x7fb84c156b20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0ae040 tx=0x7fb83c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0b0050 con 0x7fb84c0b3480 2024-04-25T10:28:37.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c0bc070 con 0x7fb84c0b3480 2024-04-25T10:28:37.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0b5850 con 0x7fb84c0b3480 2024-04-25T10:28:37.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c1574b0 con 0x7fb84c0b3480 2024-04-25T10:28:37.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.929+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c157a70 con 0x7fb84c0b3480 2024-04-25T10:28:37.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.933+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c04ef30 con 0x7fb84c0b3480 2024-04-25T10:28:37.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.933+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb83c0b59f0 con 0x7fb84c0b3480 2024-04-25T10:28:37.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.933+0000 7fb846000640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb830066fb0 0x7fb830069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:37.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.933+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb83c11f5b0 con 0x7fb84c0b3480 2024-04-25T10:28:37.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.933+0000 7fb851e00640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb830066fb0 0x7fb830069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:37.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.933+0000 7fb851e00640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb830066fb0 0x7fb830069470 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb840007fd0 tx=0x7fb840007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:37.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:37.937+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb83c0160b0 con 0x7fb84c0b3480 2024-04-25T10:28:38.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:38.141+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb84c0722f0 con 0x7fb84c0b3480 2024-04-25T10:28:38.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:28:38.990 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:28:39.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb846000640 1 -- 172.21.15.45:0/706319558 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v103) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb83c0effe0 con 0x7fb84c0b3480 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb830066fb0 msgr2=0x7fb830069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb830066fb0 0x7fb830069470 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb840007fd0 tx=0x7fb840007e40 comp rx=0 tx=0).stop 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 msgr2=0x7fb84c156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 0x7fb84c156b20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0ae040 tx=0x7fb83c019260 comp rx=0 tx=0).stop 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 shutdown_connections 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb830066fb0 0x7fb830069470 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb84c15e130 0x7fb84c159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb84c0b3480 0x7fb84c156b20 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 --2- 172.21.15.45:0/706319558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb84c071bc0 0x7fb84c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 >> 172.21.15.45:0/706319558 conn(0x7fb84c0669a0 msgr2=0x7fb84c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:39.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 shutdown_connections 2024-04-25T10:28:39.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.053+0000 7fb854800640 1 -- 172.21.15.45:0/706319558 wait complete. 2024-04-25T10:28:39.079 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:28:39.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:28:39.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 Processor -- start 2024-04-25T10:28:39.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 -- start start 2024-04-25T10:28:39.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0e8c0cd9a0 0x7f0e8c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0e8c162020 0x7f0e8c15d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c06ac90 con 0x7f0e8c06bd10 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c06b880 con 0x7f0e8c0cd9a0 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e91a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c06af50 con 0x7f0e8c162020 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0e8c0cd9a0 0x7f0e8c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0e8c0cd9a0 0x7f0e8c0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58014/0 (socket says 172.21.15.45:58014) 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 -- 172.21.15.45:0/2083158081 learned_addr learned my addr 172.21.15.45:0/2083158081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8b400640 1 --2- 172.21.15.45:0/2083158081 >> v2:172.21.15.117:3300/0 conn(0x7f0e8c162020 0x7f0e8c15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 -- 172.21.15.45:0/2083158081 >> v2:172.21.15.117:3300/0 conn(0x7f0e8c162020 msgr2=0x7f0e8c15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 --2- 172.21.15.45:0/2083158081 >> v2:172.21.15.117:3300/0 conn(0x7f0e8c162020 0x7f0e8c15d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 -- 172.21.15.45:0/2083158081 >> v1:172.21.15.117:6789/0 conn(0x7f0e8c06bd10 legacy=0x7f0e8c161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:39.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.229+0000 7f0e8aa00640 1 -- 172.21.15.45:0/2083158081 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c06b230 con 0x7f0e8c0cd9a0 2024-04-25T10:28:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e8aa00640 1 --2- 172.21.15.45:0/2083158081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c0cd9a0 0x7f0e8c0d0660 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c1629b0 tx=0x7f0e7c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1147cc943c7173ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e8a000640 1 -- 172.21.15.45:0/2083158081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b5070 con 0x7f0e8c0cd9a0 2024-04-25T10:28:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e8a000640 1 -- 172.21.15.45:0/2083158081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c0a8a10 con 0x7f0e8c0cd9a0 2024-04-25T10:28:39.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e8a000640 1 -- 172.21.15.45:0/2083158081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0a8de0 con 0x7f0e8c0cd9a0 2024-04-25T10:28:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 -- 172.21.15.45:0/2083158081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c0cd9a0 msgr2=0x7f0e8c0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 --2- 172.21.15.45:0/2083158081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c0cd9a0 0x7f0e8c0d0660 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c1629b0 tx=0x7f0e7c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:28:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 -- 172.21.15.45:0/2083158081 shutdown_connections 2024-04-25T10:28:39.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 --2- 172.21.15.45:0/2083158081 >> v2:172.21.15.117:3300/0 conn(0x7f0e8c162020 0x7f0e8c15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 --2- 172.21.15.45:0/2083158081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c0cd9a0 0x7f0e8c0d0660 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 -- 172.21.15.45:0/2083158081 >> 172.21.15.45:0/2083158081 conn(0x7f0e8c0d19a0 msgr2=0x7f0e8c0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 -- 172.21.15.45:0/2083158081 shutdown_connections 2024-04-25T10:28:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 -- 172.21.15.45:0/2083158081 wait complete. 2024-04-25T10:28:39.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 Processor -- start 2024-04-25T10:28:39.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.233+0000 7f0e91a00640 1 -- start start 2024-04-25T10:28:39.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 0x7f0e8c0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:39.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e8c0cd9a0 0x7f0e8c14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:39.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e8c162020 0x7f0e8c0b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:39.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c162660 con 0x7f0e8c06bd10 2024-04-25T10:28:39.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0cdd80 con 0x7f0e8c162020 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0d0ba0 con 0x7f0e8c0cd9a0 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 0x7f0e8c0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 0x7f0e8c0c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58024/0 (socket says 172.21.15.45:58024) 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 -- 172.21.15.45:0/663366563 learned_addr learned my addr 172.21.15.45:0/663366563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8b400640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e8c162020 0x7f0e8c0b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e83e00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e8c0cd9a0 0x7f0e8c14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 -- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e8c0cd9a0 msgr2=0x7f0e8c14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e8c0cd9a0 0x7f0e8c14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 -- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e8c162020 msgr2=0x7f0e8c0b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e8c162020 0x7f0e8c0b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 -- 172.21.15.45:0/663366563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0ca320 con 0x7f0e8c06bd10 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8b400640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e8c162020 0x7f0e8c0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:39.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e8aa00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 0x7f0e8c0c9de0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01b920 tx=0x7f0e7c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0b5070 con 0x7f0e8c06bd10 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c0b0b40 con 0x7f0e8c06bd10 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c0ca5c0 con 0x7f0e8c06bd10 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0bb410 con 0x7f0e8c06bd10 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c0cab80 con 0x7f0e8c06bd10 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c04ef30 con 0x7f0e8c06bd10 2024-04-25T10:28:39.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.237+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e7c0c2060 con 0x7f0e8c06bd10 2024-04-25T10:28:39.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.241+0000 7f0e82a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e68066fb0 0x7f0e68069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:39.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.241+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e7c122170 con 0x7f0e8c06bd10 2024-04-25T10:28:39.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.241+0000 7f0e83e00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e68066fb0 0x7f0e68069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:39.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.241+0000 7f0e83e00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e68066fb0 0x7f0e68069470 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c1e7bf0 tx=0x7f0e74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:39.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.245+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e7c018870 con 0x7f0e8c06bd10 2024-04-25T10:28:39.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:39.441+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0e8c0b7920 con 0x7f0e8c06bd10 2024-04-25T10:28:40.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.057+0000 7f0e82a00640 1 -- 172.21.15.45:0/663366563 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v104) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0e7c0f3160 con 0x7f0e8c06bd10 2024-04-25T10:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e68066fb0 msgr2=0x7f0e68069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e68066fb0 0x7f0e68069470 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c1e7bf0 tx=0x7f0e74009040 comp rx=0 tx=0).stop 2024-04-25T10:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 msgr2=0x7f0e8c0c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 0x7f0e8c0c9de0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01b920 tx=0x7f0e7c01bab0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 shutdown_connections 2024-04-25T10:28:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e68066fb0 0x7f0e68069470 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e8c162020 0x7f0e8c0b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e8c0cd9a0 0x7f0e8c14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 --2- 172.21.15.45:0/663366563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e8c06bd10 0x7f0e8c0c9de0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 >> 172.21.15.45:0/663366563 conn(0x7f0e8c0d19a0 msgr2=0x7f0e8c15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:40.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 shutdown_connections 2024-04-25T10:28:40.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.061+0000 7f0e91a00640 1 -- 172.21.15.45:0/663366563 wait complete. 2024-04-25T10:28:40.084 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:28:40.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 Processor -- start 2024-04-25T10:28:40.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 -- start start 2024-04-25T10:28:40.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8bc415a0e0 0x7f8bc415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc40b3190 0x7f8bc406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc415b6e0 con 0x7f8bc4071f70 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc4070ed0 con 0x7f8bc415a0e0 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.229+0000 7f8bca200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40711b0 con 0x7f8bc40b3190 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bbbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8bc4071f70 0x7f8bc40b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52926/0 (socket says 172.21.15.45:52926) 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bbbe00640 1 -- 172.21.15.45:0/2636452344 learned_addr learned my addr 172.21.15.45:0/2636452344 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972528158 0 0) 0x7f8bac001120 con 0x7f8bc4071f70 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 --2- 172.21.15.45:0/2636452344 >> v2:172.21.15.117:3300/0 conn(0x7f8bc415a0e0 0x7f8bc415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3e00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc40b3190 0x7f8bc406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba8003530 con 0x7f8bc4071f70 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 162964661 0 0) 0x7f8ba8003530 con 0x7f8bc4071f70 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc40b3190 msgr2=0x7f8bc406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc40b3190 0x7f8bc406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 >> v2:172.21.15.117:3300/0 conn(0x7f8bc415a0e0 msgr2=0x7f8bc415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 --2- 172.21.15.45:0/2636452344 >> v2:172.21.15.117:3300/0 conn(0x7f8bc415a0e0 0x7f8bc415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4071490 con 0x7f8bc4071f70 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f8bac001850 con 0x7f8bc4071f70 2024-04-25T10:28:40.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 --2- 172.21.15.45:0/2636452344 >> v2:172.21.15.117:3300/0 conn(0x7f8bc415a0e0 0x7f8bc415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 >> v1:172.21.15.45:6789/0 conn(0x7f8bc4071f70 legacy=0x7f8bc40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba80039f0 0x7f8ba8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba8004810 0x7f8ba8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4070ed0 con 0x7f8ba80039f0 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40711b0 con 0x7f8ba8004810 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba800c070 con 0x7f8ba80051d0 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3e00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc40b3190 0x7f8bc406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba80039f0 0x7f8ba8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bbbe00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba8004810 0x7f8ba8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3e00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 -- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ba80051d0 msgr2=0x7f8ba800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 -- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba8004810 msgr2=0x7f8ba8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba8004810 0x7f8ba8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3400640 1 -- 172.21.15.45:0/2636452344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8003530 con 0x7f8ba80039f0 2024-04-25T10:28:40.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.233+0000 7f8bc3e00640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ba80051d0 0x7f8ba800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bc3400640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba80039f0 0x7f8ba8003e10 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40098d0 tx=0x7f8bb401c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073fbd9bb6ad1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400f040 con 0x7f8ba80039f0 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40b0600 con 0x7f8ba80039f0 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bc2a00640 1 -- 172.21.15.45:0/2636452344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40b09b0 con 0x7f8ba80039f0 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- 172.21.15.45:0/2636452344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba80039f0 msgr2=0x7f8ba8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 --2- 172.21.15.45:0/2636452344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba80039f0 0x7f8ba8003e10 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40098d0 tx=0x7f8bb401c620 comp rx=0 tx=0).stop 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bc3e00640 1 -- 172.21.15.45:0/2636452344 reap_dead start 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- 172.21.15.45:0/2636452344 shutdown_connections 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- 172.21.15.45:0/2636452344 >> 172.21.15.45:0/2636452344 conn(0x7f8bc4066b70 msgr2=0x7f8bc4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- 172.21.15.45:0/2636452344 shutdown_connections 2024-04-25T10:28:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- 172.21.15.45:0/2636452344 wait complete. 2024-04-25T10:28:40.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 Processor -- start 2024-04-25T10:28:40.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- start start 2024-04-25T10:28:40.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba80039f0 0x7f8bc40b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 0x7f8bc4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc4144de0 0x7f8bc4145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4070b50 con 0x7f8bc4155740 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415a4c0 con 0x7f8ba80039f0 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.237+0000 7f8bca200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4072370 con 0x7f8bc4144de0 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 0x7f8bc4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 0x7f8bc4155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58050/0 (socket says 172.21.15.45:58050) 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 -- 172.21.15.45:0/1181391978 learned_addr learned my addr 172.21.15.45:0/1181391978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bc3400640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba80039f0 0x7f8bc40b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bc3e00640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc4144de0 0x7f8bc4145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 -- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc4144de0 msgr2=0x7f8bc4145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc4144de0 0x7f8bc4145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 -- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba80039f0 msgr2=0x7f8bc40b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba80039f0 0x7f8bc40b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 -- 172.21.15.45:0/1181391978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc41457a0 con 0x7f8bc4155740 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bc3400640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba80039f0 0x7f8bc40b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbbe00640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 0x7f8bc4155bc0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00e7e0 tx=0x7f8bac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00ee90 con 0x7f8bc4155740 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac018070 con 0x7f8bc4155740 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00caf0 con 0x7f8bc4155740 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc41fa280 con 0x7f8bc4155740 2024-04-25T10:28:40.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc41fa840 con 0x7f8bc4155740 2024-04-25T10:28:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc404ef30 con 0x7f8bc4155740 2024-04-25T10:28:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bac00cc90 con 0x7f8bc4155740 2024-04-25T10:28:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbb400640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ba0067130 0x7f8ba00695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.241+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bac084f50 con 0x7f8bc4155740 2024-04-25T10:28:40.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.245+0000 7f8bc3400640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ba0067130 0x7f8ba00695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.245+0000 7f8bc3400640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ba0067130 0x7f8ba00695f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40af000 tx=0x7f8bb40046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.245+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bc41457a0 con 0x7f8bc4155740 2024-04-25T10:28:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.441+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bc40703e0 con 0x7f8bc4155740 2024-04-25T10:28:40.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.441+0000 7f8bbb400640 1 -- 172.21.15.45:0/1181391978 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4945 (secure 0 0 0) 0x7f8bc415b6e0 con 0x7f8bc4155740 2024-04-25T10:28:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:40.443 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8893,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:28:39.444741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:28:40.443 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 104 2024-04-25T10:28:40.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ba0067130 msgr2=0x7f8ba00695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ba0067130 0x7f8ba00695f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40af000 tx=0x7f8bb40046d0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 msgr2=0x7f8bc4155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 0x7f8bc4155bc0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00e7e0 tx=0x7f8bac00eca0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 shutdown_connections 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8ba0067130 0x7f8ba00695f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bc4144de0 0x7f8bc4145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bc4155740 0x7f8bc4155bc0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 --2- 172.21.15.45:0/1181391978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ba80039f0 0x7f8bc40b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 >> 172.21.15.45:0/1181391978 conn(0x7f8bc4066b70 msgr2=0x7f8bc41567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 shutdown_connections 2024-04-25T10:28:40.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.445+0000 7f8bca200640 1 -- 172.21.15.45:0/1181391978 wait complete. 2024-04-25T10:28:40.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:28:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151f400640 1 Processor -- start 2024-04-25T10:28:40.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151f400640 1 -- start start 2024-04-25T10:28:40.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b3190 0x7f151806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151f400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1518070ed0 con 0x7f1518071f70 2024-04-25T10:28:40.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151f400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15180711b0 con 0x7f151815a0e0 2024-04-25T10:28:40.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1518071490 con 0x7f15180b3190 2024-04-25T10:28:40.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f150fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1518071f70 0x7f15180b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52942/0 (socket says 172.21.15.45:52942) 2024-04-25T10:28:40.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f150fe00640 1 -- 172.21.15.45:0/4064668486 learned_addr learned my addr 172.21.15.45:0/4064668486 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:40.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151d400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b3190 0x7f151806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656614702 0 0) 0x7f1518070ed0 con 0x7f1518071f70 2024-04-25T10:28:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14fc003530 con 0x7f1518071f70 2024-04-25T10:28:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785758641 0 0) 0x7f15180711b0 con 0x7f151815a0e0 2024-04-25T10:28:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1518070ed0 con 0x7f151815a0e0 2024-04-25T10:28:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2367157441 0 0) 0x7f14fc003530 con 0x7f1518071f70 2024-04-25T10:28:40.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b3190 msgr2=0x7f151806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b3190 0x7f151806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 >> v1:172.21.15.117:6789/0 conn(0x7f151815a0e0 legacy=0x7f151815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518071770 con 0x7f1518071f70 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f1500001680 con 0x7f1518071f70 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 >> v1:172.21.15.45:6789/0 conn(0x7f1518071f70 legacy=0x7f15180b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc003b80 0x7f14fc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14fc004810 0x7f14fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14fc0051d0 0x7f14fc00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1518071490 con 0x7f14fc0051d0 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc00e200 con 0x7f14fc004810 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc00e400 con 0x7f14fc003b80 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.605+0000 7f151d400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15180b3190 0x7f151806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:40.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151ca00640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc003b80 0x7f14fc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14fc0051d0 0x7f14fc00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f150fe00640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14fc004810 0x7f14fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 -- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc003b80 msgr2=0x7f14fc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc003b80 0x7f14fc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 -- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14fc004810 msgr2=0x7f14fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14fc004810 0x7f14fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 -- 172.21.15.45:0/4064668486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc003530 con 0x7f14fc0051d0 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f151d400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14fc0051d0 0x7f14fc00dbe0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f151000e1f0 tx=0x7f151001cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7375c6a4a4dff69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510009d40 con 0x7f14fc0051d0 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15100a5d70 con 0x7f14fc0051d0 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.609+0000 7f1517e00640 1 -- 172.21.15.45:0/4064668486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510003bc0 con 0x7f14fc0051d0 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- 172.21.15.45:0/4064668486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14fc0051d0 msgr2=0x7f14fc00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 --2- 172.21.15.45:0/4064668486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14fc0051d0 0x7f14fc00dbe0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f151000e1f0 tx=0x7f151001cd20 comp rx=0 tx=0).stop 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151d400640 1 -- 172.21.15.45:0/4064668486 reap_dead start 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- 172.21.15.45:0/4064668486 shutdown_connections 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- 172.21.15.45:0/4064668486 >> 172.21.15.45:0/4064668486 conn(0x7f1518066b70 msgr2=0x7f1518066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:40.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- 172.21.15.45:0/4064668486 shutdown_connections 2024-04-25T10:28:40.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- 172.21.15.45:0/4064668486 wait complete. 2024-04-25T10:28:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 Processor -- start 2024-04-25T10:28:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- start start 2024-04-25T10:28:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc0051d0 0x7f151815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 0x7f151814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15181ef7b0 0x7f151814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151815a4c0 con 0x7f15181ef350 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180af030 con 0x7f15181ef7b0 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1518070b50 con 0x7f14fc0051d0 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 0x7f151814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 0x7f151814aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58066/0 (socket says 172.21.15.45:58066) 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 -- 172.21.15.45:0/4059480153 learned_addr learned my addr 172.21.15.45:0/4059480153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151ca00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc0051d0 0x7f151815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151d400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15181ef7b0 0x7f151814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 -- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc0051d0 msgr2=0x7f151815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc0051d0 0x7f151815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 -- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15181ef7b0 msgr2=0x7f151814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15181ef7b0 0x7f151814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 -- 172.21.15.45:0/4059480153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518148d00 con 0x7f15181ef350 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f151ca00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc0051d0 0x7f151815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.613+0000 7f150fe00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 0x7f151814aa70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1500012540 tx=0x7f1500012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500014810 con 0x7f15181ef350 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150001b070 con 0x7f15181ef350 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500014bd0 con 0x7f15181ef350 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1518148fe0 con 0x7f15181ef350 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15181495a0 con 0x7f15181ef350 2024-04-25T10:28:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151804ef30 con 0x7f15181ef350 2024-04-25T10:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1500016480 con 0x7f15181ef350 2024-04-25T10:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f1515600640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f14f4066f90 0x7f14f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1500088500 con 0x7f15181ef350 2024-04-25T10:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.617+0000 7f151ca00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f14f4066f90 0x7f14f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.621+0000 7f151ca00640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f14f4066f90 0x7f14f4069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1508007fd0 tx=0x7f1508009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.621+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1500059480 con 0x7f15181ef350 2024-04-25T10:28:40.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.785+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15181597a0 con 0x7f15181ef350 2024-04-25T10:28:40.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f1515600640 1 -- 172.21.15.45:0/4059480153 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f1500058d80 con 0x7f15181ef350 2024-04-25T10:28:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:40.789 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":141,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:38.022285+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:28:40.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f14f4066f90 msgr2=0x7f14f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f14f4066f90 0x7f14f4069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1508007fd0 tx=0x7f1508009040 comp rx=0 tx=0).stop 2024-04-25T10:28:40.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 msgr2=0x7f151814aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 0x7f151814aa70 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f1500012540 tx=0x7f1500012a00 comp rx=0 tx=0).stop 2024-04-25T10:28:40.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 shutdown_connections 2024-04-25T10:28:40.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f14f4066f90 0x7f14f4069450 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15181ef7b0 0x7f151814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15181ef350 0x7f151814aa70 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 --2- 172.21.15.45:0/4059480153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14fc0051d0 0x7f151815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 >> 172.21.15.45:0/4059480153 conn(0x7f1518066b70 msgr2=0x7f15180ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:40.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 shutdown_connections 2024-04-25T10:28:40.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.789+0000 7f151f400640 1 -- 172.21.15.45:0/4059480153 wait complete. 2024-04-25T10:28:40.811 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:28:40.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:28:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 Processor -- start 2024-04-25T10:28:40.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 -- start start 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01cc0af9c0 0x7f01cc06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01cc071f50 0x7f01cc0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc06d380 con 0x7f01cc071f50 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc06f200 con 0x7f01cc0af9c0 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0ccc10 con 0x7f01cc0afda0 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01cc071f50 0x7f01cc0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f01cc071f50 0x7f01cc0669a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58078/0 (socket says 172.21.15.45:58078) 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 -- 172.21.15.45:0/3550613672 learned_addr learned my addr 172.21.15.45:0/3550613672 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01cbe00640 1 --2- 172.21.15.45:0/3550613672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1200640 1 --2- 172.21.15.45:0/3550613672 >> v2:172.21.15.117:3300/0 conn(0x7f01cc0af9c0 0x7f01cc06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 -- 172.21.15.45:0/3550613672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 msgr2=0x7f01cc0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 --2- 172.21.15.45:0/3550613672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc0cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 -- 172.21.15.45:0/3550613672 >> v2:172.21.15.117:3300/0 conn(0x7f01cc0af9c0 msgr2=0x7f01cc06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 --2- 172.21.15.45:0/3550613672 >> v2:172.21.15.117:3300/0 conn(0x7f01cc0af9c0 0x7f01cc06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.953+0000 7f01d1c00640 1 -- 172.21.15.45:0/3550613672 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0ccef0 con 0x7f01cc071f50 2024-04-25T10:28:40.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d1c00640 1 --2- 172.21.15.45:0/3550613672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc071f50 0x7f01cc0669a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0ce370 tx=0x7f01c401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d82afcb87bca54e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01cb400640 1 -- 172.21.15.45:0/3550613672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c40a5630 con 0x7f01cc071f50 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01cb400640 1 -- 172.21.15.45:0/3550613672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c40a57d0 con 0x7f01cc071f50 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01cb400640 1 -- 172.21.15.45:0/3550613672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c40a5ba0 con 0x7f01cc071f50 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 -- 172.21.15.45:0/3550613672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc071f50 msgr2=0x7f01cc0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 --2- 172.21.15.45:0/3550613672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc071f50 0x7f01cc0669a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0ce370 tx=0x7f01c401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 -- 172.21.15.45:0/3550613672 shutdown_connections 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 --2- 172.21.15.45:0/3550613672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc071f50 0x7f01cc0669a0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 --2- 172.21.15.45:0/3550613672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 --2- 172.21.15.45:0/3550613672 >> v2:172.21.15.117:3300/0 conn(0x7f01cc0af9c0 0x7f01cc06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 -- 172.21.15.45:0/3550613672 >> 172.21.15.45:0/3550613672 conn(0x7f01cc0b2410 msgr2=0x7f01cc0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 -- 172.21.15.45:0/3550613672 shutdown_connections 2024-04-25T10:28:40.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.957+0000 7f01d3c00640 1 -- 172.21.15.45:0/3550613672 wait complete. 2024-04-25T10:28:40.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 Processor -- start 2024-04-25T10:28:40.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 -- start start 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01cc071f50 0x7f01cc153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 0x7f01cc0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc160250 con 0x7f01cc0af9c0 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc1600d0 con 0x7f01cc071f50 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc1603d0 con 0x7f01cc0afda0 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 0x7f01cc0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 0x7f01cc0c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58084/0 (socket says 172.21.15.45:58084) 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 -- 172.21.15.45:0/1764470408 learned_addr learned my addr 172.21.15.45:0/1764470408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d1c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d1200640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01cc071f50 0x7f01cc153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 -- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 msgr2=0x7f01cc15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 -- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01cc071f50 msgr2=0x7f01cc153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01cc071f50 0x7f01cc153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 -- 172.21.15.45:0/1764470408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc15ceb0 con 0x7f01cc0af9c0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d1c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d1200640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01cc071f50 0x7f01cc153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01cbe00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 0x7f01cc0c5e30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f01b800e870 tx=0x7f01b800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8017070 con 0x7f01cc0af9c0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b800c940 con 0x7f01cc0af9c0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800ccf0 con 0x7f01cc0af9c0 2024-04-25T10:28:40.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc0c5350 con 0x7f01cc0af9c0 2024-04-25T10:28:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.961+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc0c5910 con 0x7f01cc0af9c0 2024-04-25T10:28:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.965+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc04ef30 con 0x7f01cc0af9c0 2024-04-25T10:28:40.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.965+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01b801a070 con 0x7f01cc0af9c0 2024-04-25T10:28:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.965+0000 7f01c8c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.965+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01b80852e0 con 0x7f01cc0af9c0 2024-04-25T10:28:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.965+0000 7f01d1200640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:40.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.965+0000 7f01d1200640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009ae0 tx=0x7f01c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:40.969+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01b8056250 con 0x7f01cc0af9c0 2024-04-25T10:28:41.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.129+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01cc15d190 con 0x7f01ac066f60 2024-04-25T10:28:41.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.137+0000 7f01c8c00640 1 -- 172.21.15.45:0/1764470408 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252438 (secure 0 0 0) 0x7f01cc15d190 con 0x7f01ac066f60 2024-04-25T10:28:41.140 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:41.142 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:28:41.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 msgr2=0x7f01ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009ae0 tx=0x7f01c0008040 comp rx=0 tx=0).stop 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 msgr2=0x7f01cc0c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 0x7f01cc0c5e30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f01b800e870 tx=0x7f01b800ec70 comp rx=0 tx=0).stop 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 shutdown_connections 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f01ac066f60 0x7f01ac069420 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01cc0afda0 0x7f01cc15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01cc0af9c0 0x7f01cc0c5e30 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 --2- 172.21.15.45:0/1764470408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01cc071f50 0x7f01cc153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 >> 172.21.15.45:0/1764470408 conn(0x7f01cc0b2410 msgr2=0x7f01cc06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 shutdown_connections 2024-04-25T10:28:41.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.141+0000 7f01d3c00640 1 -- 172.21.15.45:0/1764470408 wait complete. 2024-04-25T10:28:41.160 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-25T10:28:41.023605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":437816,"kb_used_data":14860,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754536904,"statfs":{"total":773094113280,"available":772645789696,"internally_reserved":0,"allocated":15216640,"data_stored":7284737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001229"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050382+0000","last_change":"2024-04-25T10:28:37.045763+0000","last_active":"2024-04-25T10:28:37.050382+0000","last_peered":"2024-04-25T10:28:37.050382+0000","last_clean":"2024-04-25T10:28:37.050382+0000","last_became_active":"2024-04-25T10:28:37.045491+0000","last_became_peered":"2024-04-25T10:28:37.045491+0000","last_unstale":"2024-04-25T10:28:37.050382+0000","last_undegraded":"2024-04-25T10:28:37.050382+0000","last_fullsized":"2024-04-25T10:28:37.050382+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:50.509447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056390+0000","last_change":"2024-04-25T10:28:37.047130+0000","last_active":"2024-04-25T10:28:37.056390+0000","last_peered":"2024-04-25T10:28:37.056390+0000","last_clean":"2024-04-25T10:28:37.056390+0000","last_became_active":"2024-04-25T10:28:37.046805+0000","last_became_peered":"2024-04-25T10:28:37.046805+0000","last_unstale":"2024-04-25T10:28:37.056390+0000","last_undegraded":"2024-04-25T10:28:37.056390+0000","last_fullsized":"2024-04-25T10:28:37.056390+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:16.796196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050051+0000","last_change":"2024-04-25T10:28:37.038693+0000","last_active":"2024-04-25T10:28:37.050051+0000","last_peered":"2024-04-25T10:28:37.050051+0000","last_clean":"2024-04-25T10:28:37.050051+0000","last_became_active":"2024-04-25T10:28:37.038379+0000","last_became_peered":"2024-04-25T10:28:37.038379+0000","last_unstale":"2024-04-25T10:28:37.050051+0000","last_undegraded":"2024-04-25T10:28:37.050051+0000","last_fullsized":"2024-04-25T10:28:37.050051+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:03.963295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065614+0000","last_change":"2024-04-25T10:28:37.059436+0000","last_active":"2024-04-25T10:28:37.065614+0000","last_peered":"2024-04-25T10:28:37.065614+0000","last_clean":"2024-04-25T10:28:37.065614+0000","last_became_active":"2024-04-25T10:28:37.059176+0000","last_became_peered":"2024-04-25T10:28:37.059176+0000","last_unstale":"2024-04-25T10:28:37.065614+0000","last_undegraded":"2024-04-25T10:28:37.065614+0000","last_fullsized":"2024-04-25T10:28:37.065614+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:40.512214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029547+0000","last_change":"2024-04-25T10:28:37.050541+0000","last_active":"2024-04-25T10:28:38.029547+0000","last_peered":"2024-04-25T10:28:38.029547+0000","last_clean":"2024-04-25T10:28:38.029547+0000","last_became_active":"2024-04-25T10:28:37.047862+0000","last_became_peered":"2024-04-25T10:28:37.047862+0000","last_unstale":"2024-04-25T10:28:38.029547+0000","last_undegraded":"2024-04-25T10:28:38.029547+0000","last_fullsized":"2024-04-25T10:28:38.029547+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:34.309365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036048+0000","last_change":"2024-04-25T10:28:37.039418+0000","last_active":"2024-04-25T10:28:38.036048+0000","last_peered":"2024-04-25T10:28:38.036048+0000","last_clean":"2024-04-25T10:28:38.036048+0000","last_became_active":"2024-04-25T10:28:37.039121+0000","last_became_peered":"2024-04-25T10:28:37.039121+0000","last_unstale":"2024-04-25T10:28:38.036048+0000","last_undegraded":"2024-04-25T10:28:38.036048+0000","last_fullsized":"2024-04-25T10:28:38.036048+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:04.755093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.037641+0000","last_change":"2024-04-25T10:28:37.041979+0000","last_active":"2024-04-25T10:28:38.037641+0000","last_peered":"2024-04-25T10:28:38.037641+0000","last_clean":"2024-04-25T10:28:38.037641+0000","last_became_active":"2024-04-25T10:28:37.041682+0000","last_became_peered":"2024-04-25T10:28:37.041682+0000","last_unstale":"2024-04-25T10:28:38.037641+0000","last_undegraded":"2024-04-25T10:28:38.037641+0000","last_fullsized":"2024-04-25T10:28:38.037641+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:09.796346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059460+0000","last_change":"2024-04-25T10:28:37.058215+0000","last_active":"2024-04-25T10:28:37.059460+0000","last_peered":"2024-04-25T10:28:37.059460+0000","last_clean":"2024-04-25T10:28:37.059460+0000","last_became_active":"2024-04-25T10:28:37.057870+0000","last_became_peered":"2024-04-25T10:28:37.057870+0000","last_unstale":"2024-04-25T10:28:37.059460+0000","last_undegraded":"2024-04-25T10:28:37.059460+0000","last_fullsized":"2024-04-25T10:28:37.059460+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:17.890811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036432+0000","last_change":"2024-04-25T10:28:37.043363+0000","last_active":"2024-04-25T10:28:38.036432+0000","last_peered":"2024-04-25T10:28:38.036432+0000","last_clean":"2024-04-25T10:28:38.036432+0000","last_became_active":"2024-04-25T10:28:37.043021+0000","last_became_peered":"2024-04-25T10:28:37.043021+0000","last_unstale":"2024-04-25T10:28:38.036432+0000","last_undegraded":"2024-04-25T10:28:38.036432+0000","last_fullsized":"2024-04-25T10:28:38.036432+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:32.005095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056469+0000","last_change":"2024-04-25T10:28:37.054867+0000","last_active":"2024-04-25T10:28:37.056469+0000","last_peered":"2024-04-25T10:28:37.056469+0000","last_clean":"2024-04-25T10:28:37.056469+0000","last_became_active":"2024-04-25T10:28:37.054181+0000","last_became_peered":"2024-04-25T10:28:37.054181+0000","last_unstale":"2024-04-25T10:28:37.056469+0000","last_undegraded":"2024-04-25T10:28:37.056469+0000","last_fullsized":"2024-04-25T10:28:37.056469+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:44.572625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036496+0000","last_change":"2024-04-25T10:28:37.055355+0000","last_active":"2024-04-25T10:28:38.036496+0000","last_peered":"2024-04-25T10:28:38.036496+0000","last_clean":"2024-04-25T10:28:38.036496+0000","last_became_active":"2024-04-25T10:28:37.055077+0000","last_became_peered":"2024-04-25T10:28:37.055077+0000","last_unstale":"2024-04-25T10:28:38.036496+0000","last_undegraded":"2024-04-25T10:28:38.036496+0000","last_fullsized":"2024-04-25T10:28:38.036496+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:16.231448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272510+0000","last_change":"2024-04-25T10:28:37.061622+0000","last_active":"2024-04-25T10:28:38.272510+0000","last_peered":"2024-04-25T10:28:38.272510+0000","last_clean":"2024-04-25T10:28:38.272510+0000","last_became_active":"2024-04-25T10:28:37.061346+0000","last_became_peered":"2024-04-25T10:28:37.061346+0000","last_unstale":"2024-04-25T10:28:38.272510+0000","last_undegraded":"2024-04-25T10:28:38.272510+0000","last_fullsized":"2024-04-25T10:28:38.272510+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:07.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029688+0000","last_change":"2024-04-25T10:28:37.050801+0000","last_active":"2024-04-25T10:28:38.029688+0000","last_peered":"2024-04-25T10:28:38.029688+0000","last_clean":"2024-04-25T10:28:38.029688+0000","last_became_active":"2024-04-25T10:28:37.050339+0000","last_became_peered":"2024-04-25T10:28:37.050339+0000","last_unstale":"2024-04-25T10:28:38.029688+0000","last_undegraded":"2024-04-25T10:28:38.029688+0000","last_fullsized":"2024-04-25T10:28:38.029688+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.816347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272359+0000","last_change":"2024-04-25T10:28:37.059553+0000","last_active":"2024-04-25T10:28:38.272359+0000","last_peered":"2024-04-25T10:28:38.272359+0000","last_clean":"2024-04-25T10:28:38.272359+0000","last_became_active":"2024-04-25T10:28:37.059237+0000","last_became_peered":"2024-04-25T10:28:37.059237+0000","last_unstale":"2024-04-25T10:28:38.272359+0000","last_undegraded":"2024-04-25T10:28:38.272359+0000","last_fullsized":"2024-04-25T10:28:38.272359+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:42.356192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029393+0000","last_change":"2024-04-25T10:28:37.046334+0000","last_active":"2024-04-25T10:28:38.029393+0000","last_peered":"2024-04-25T10:28:38.029393+0000","last_clean":"2024-04-25T10:28:38.029393+0000","last_became_active":"2024-04-25T10:28:37.046048+0000","last_became_peered":"2024-04-25T10:28:37.046048+0000","last_unstale":"2024-04-25T10:28:38.029393+0000","last_undegraded":"2024-04-25T10:28:38.029393+0000","last_fullsized":"2024-04-25T10:28:38.029393+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:27.529759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.037653+0000","last_change":"2024-04-25T10:28:37.040302+0000","last_active":"2024-04-25T10:28:38.037653+0000","last_peered":"2024-04-25T10:28:38.037653+0000","last_clean":"2024-04-25T10:28:38.037653+0000","last_became_active":"2024-04-25T10:28:37.040009+0000","last_became_peered":"2024-04-25T10:28:37.040009+0000","last_unstale":"2024-04-25T10:28:38.037653+0000","last_undegraded":"2024-04-25T10:28:38.037653+0000","last_fullsized":"2024-04-25T10:28:38.037653+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:32.005095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050218+0000","last_change":"2024-04-25T10:28:37.046488+0000","last_active":"2024-04-25T10:28:37.050218+0000","last_peered":"2024-04-25T10:28:37.050218+0000","last_clean":"2024-04-25T10:28:37.050218+0000","last_became_active":"2024-04-25T10:28:37.046262+0000","last_became_peered":"2024-04-25T10:28:37.046262+0000","last_unstale":"2024-04-25T10:28:37.050218+0000","last_undegraded":"2024-04-25T10:28:37.050218+0000","last_fullsized":"2024-04-25T10:28:37.050218+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:31.531043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029689+0000","last_change":"2024-04-25T10:28:37.050641+0000","last_active":"2024-04-25T10:28:38.029689+0000","last_peered":"2024-04-25T10:28:38.029689+0000","last_clean":"2024-04-25T10:28:38.029689+0000","last_became_active":"2024-04-25T10:28:37.046820+0000","last_became_peered":"2024-04-25T10:28:37.046820+0000","last_unstale":"2024-04-25T10:28:38.029689+0000","last_undegraded":"2024-04-25T10:28:38.029689+0000","last_fullsized":"2024-04-25T10:28:38.029689+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:01.330458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.045365+0000","last_change":"2024-04-25T10:28:37.039223+0000","last_active":"2024-04-25T10:28:37.045365+0000","last_peered":"2024-04-25T10:28:37.045365+0000","last_clean":"2024-04-25T10:28:37.045365+0000","last_became_active":"2024-04-25T10:28:37.038899+0000","last_became_peered":"2024-04-25T10:28:37.038899+0000","last_unstale":"2024-04-25T10:28:37.045365+0000","last_undegraded":"2024-04-25T10:28:37.045365+0000","last_fullsized":"2024-04-25T10:28:37.045365+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:50.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.043700+0000","last_change":"2024-04-25T10:28:37.039979+0000","last_active":"2024-04-25T10:28:37.043700+0000","last_peered":"2024-04-25T10:28:37.043700+0000","last_clean":"2024-04-25T10:28:37.043700+0000","last_became_active":"2024-04-25T10:28:37.039549+0000","last_became_peered":"2024-04-25T10:28:37.039549+0000","last_unstale":"2024-04-25T10:28:37.043700+0000","last_undegraded":"2024-04-25T10:28:37.043700+0000","last_fullsized":"2024-04-25T10:28:37.043700+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:12.852310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050367+0000","last_change":"2024-04-25T10:28:37.046716+0000","last_active":"2024-04-25T10:28:37.050367+0000","last_peered":"2024-04-25T10:28:37.050367+0000","last_clean":"2024-04-25T10:28:37.050367+0000","last_became_active":"2024-04-25T10:28:37.044255+0000","last_became_peered":"2024-04-25T10:28:37.044255+0000","last_unstale":"2024-04-25T10:28:37.050367+0000","last_undegraded":"2024-04-25T10:28:37.050367+0000","last_fullsized":"2024-04-25T10:28:37.050367+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:58.158610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272371+0000","last_change":"2024-04-25T10:28:37.053702+0000","last_active":"2024-04-25T10:28:38.272371+0000","last_peered":"2024-04-25T10:28:38.272371+0000","last_clean":"2024-04-25T10:28:38.272371+0000","last_became_active":"2024-04-25T10:28:37.050180+0000","last_became_peered":"2024-04-25T10:28:37.050180+0000","last_unstale":"2024-04-25T10:28:38.272371+0000","last_undegraded":"2024-04-25T10:28:38.272371+0000","last_fullsized":"2024-04-25T10:28:38.272371+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.857540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056831+0000","last_change":"2024-04-25T10:28:37.046320+0000","last_active":"2024-04-25T10:28:37.056831+0000","last_peered":"2024-04-25T10:28:37.056831+0000","last_clean":"2024-04-25T10:28:37.056831+0000","last_became_active":"2024-04-25T10:28:37.045963+0000","last_became_peered":"2024-04-25T10:28:37.045963+0000","last_unstale":"2024-04-25T10:28:37.056831+0000","last_undegraded":"2024-04-25T10:28:37.056831+0000","last_fullsized":"2024-04-25T10:28:37.056831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.427512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029266+0000","last_change":"2024-04-25T10:28:37.061184+0000","last_active":"2024-04-25T10:28:38.029266+0000","last_peered":"2024-04-25T10:28:38.029266+0000","last_clean":"2024-04-25T10:28:38.029266+0000","last_became_active":"2024-04-25T10:28:37.060868+0000","last_became_peered":"2024-04-25T10:28:37.060868+0000","last_unstale":"2024-04-25T10:28:38.029266+0000","last_undegraded":"2024-04-25T10:28:38.029266+0000","last_fullsized":"2024-04-25T10:28:38.029266+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:17.594880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.043845+0000","last_change":"2024-04-25T10:28:37.042825+0000","last_active":"2024-04-25T10:28:37.043845+0000","last_peered":"2024-04-25T10:28:37.043845+0000","last_clean":"2024-04-25T10:28:37.043845+0000","last_became_active":"2024-04-25T10:28:37.042498+0000","last_became_peered":"2024-04-25T10:28:37.042498+0000","last_unstale":"2024-04-25T10:28:37.043845+0000","last_undegraded":"2024-04-25T10:28:37.043845+0000","last_fullsized":"2024-04-25T10:28:37.043845+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.768516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.035564+0000","last_change":"2024-04-25T10:28:37.034896+0000","last_active":"2024-04-25T10:28:37.035564+0000","last_peered":"2024-04-25T10:28:37.035564+0000","last_clean":"2024-04-25T10:28:37.035564+0000","last_became_active":"2024-04-25T10:28:37.034604+0000","last_became_peered":"2024-04-25T10:28:37.034604+0000","last_unstale":"2024-04-25T10:28:37.035564+0000","last_undegraded":"2024-04-25T10:28:37.035564+0000","last_fullsized":"2024-04-25T10:28:37.035564+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.085548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.055620+0000","last_change":"2024-04-25T10:28:37.047140+0000","last_active":"2024-04-25T10:28:37.055620+0000","last_peered":"2024-04-25T10:28:37.055620+0000","last_clean":"2024-04-25T10:28:37.055620+0000","last_became_active":"2024-04-25T10:28:37.046776+0000","last_became_peered":"2024-04-25T10:28:37.046776+0000","last_unstale":"2024-04-25T10:28:37.055620+0000","last_undegraded":"2024-04-25T10:28:37.055620+0000","last_fullsized":"2024-04-25T10:28:37.055620+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:17.528629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.048285+0000","last_change":"2024-04-25T10:28:37.045655+0000","last_active":"2024-04-25T10:28:37.048285+0000","last_peered":"2024-04-25T10:28:37.048285+0000","last_clean":"2024-04-25T10:28:37.048285+0000","last_became_active":"2024-04-25T10:28:37.044237+0000","last_became_peered":"2024-04-25T10:28:37.044237+0000","last_unstale":"2024-04-25T10:28:37.048285+0000","last_undegraded":"2024-04-25T10:28:37.048285+0000","last_fullsized":"2024-04-25T10:28:37.048285+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:05.163084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.055489+0000","last_change":"2024-04-25T10:28:37.053506+0000","last_active":"2024-04-25T10:28:37.055489+0000","last_peered":"2024-04-25T10:28:37.055489+0000","last_clean":"2024-04-25T10:28:37.055489+0000","last_became_active":"2024-04-25T10:28:37.053165+0000","last_became_peered":"2024-04-25T10:28:37.053165+0000","last_unstale":"2024-04-25T10:28:37.055489+0000","last_undegraded":"2024-04-25T10:28:37.055489+0000","last_fullsized":"2024-04-25T10:28:37.055489+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:40.446367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272541+0000","last_change":"2024-04-25T10:28:37.051038+0000","last_active":"2024-04-25T10:28:38.272541+0000","last_peered":"2024-04-25T10:28:38.272541+0000","last_clean":"2024-04-25T10:28:38.272541+0000","last_became_active":"2024-04-25T10:28:37.049843+0000","last_became_peered":"2024-04-25T10:28:37.049843+0000","last_unstale":"2024-04-25T10:28:38.272541+0000","last_undegraded":"2024-04-25T10:28:38.272541+0000","last_fullsized":"2024-04-25T10:28:38.272541+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:11.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057866+0000","last_change":"2024-04-25T10:28:37.055942+0000","last_active":"2024-04-25T10:28:37.057866+0000","last_peered":"2024-04-25T10:28:37.057866+0000","last_clean":"2024-04-25T10:28:37.057866+0000","last_became_active":"2024-04-25T10:28:37.055262+0000","last_became_peered":"2024-04-25T10:28:37.055262+0000","last_unstale":"2024-04-25T10:28:37.057866+0000","last_undegraded":"2024-04-25T10:28:37.057866+0000","last_fullsized":"2024-04-25T10:28:37.057866+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:47.485924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065519+0000","last_change":"2024-04-25T10:28:37.065019+0000","last_active":"2024-04-25T10:28:37.065519+0000","last_peered":"2024-04-25T10:28:37.065519+0000","last_clean":"2024-04-25T10:28:37.065519+0000","last_became_active":"2024-04-25T10:28:37.064774+0000","last_became_peered":"2024-04-25T10:28:37.064774+0000","last_unstale":"2024-04-25T10:28:37.065519+0000","last_undegraded":"2024-04-25T10:28:37.065519+0000","last_fullsized":"2024-04-25T10:28:37.065519+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:20.901336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029765+0000","last_change":"2024-04-25T10:28:37.050342+0000","last_active":"2024-04-25T10:28:38.029765+0000","last_peered":"2024-04-25T10:28:38.029765+0000","last_clean":"2024-04-25T10:28:38.029765+0000","last_became_active":"2024-04-25T10:28:37.046502+0000","last_became_peered":"2024-04-25T10:28:37.046502+0000","last_unstale":"2024-04-25T10:28:38.029765+0000","last_undegraded":"2024-04-25T10:28:38.029765+0000","last_fullsized":"2024-04-25T10:28:38.029765+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:09.706452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"140'4","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.058914+0000","last_change":"2024-04-25T10:28:36.034838+0000","last_active":"2024-04-25T10:28:37.058914+0000","last_peered":"2024-04-25T10:28:37.058914+0000","last_clean":"2024-04-25T10:28:37.058914+0000","last_became_active":"2024-04-25T10:28:36.034548+0000","last_became_peered":"2024-04-25T10:28:36.034548+0000","last_unstale":"2024-04-25T10:28:37.058914+0000","last_undegraded":"2024-04-25T10:28:37.058914+0000","last_fullsized":"2024-04-25T10:28:37.058914+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:12.769209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038266+0000","last_change":"2024-04-25T10:28:37.032722+0000","last_active":"2024-04-25T10:28:38.038266+0000","last_peered":"2024-04-25T10:28:38.038266+0000","last_clean":"2024-04-25T10:28:38.038266+0000","last_became_active":"2024-04-25T10:28:37.032406+0000","last_became_peered":"2024-04-25T10:28:37.032406+0000","last_unstale":"2024-04-25T10:28:38.038266+0000","last_undegraded":"2024-04-25T10:28:38.038266+0000","last_fullsized":"2024-04-25T10:28:38.038266+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:16.896821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036461+0000","last_change":"2024-04-25T10:28:36.047885+0000","last_active":"2024-04-25T10:28:38.036461+0000","last_peered":"2024-04-25T10:28:38.036461+0000","last_clean":"2024-04-25T10:28:38.036461+0000","last_became_active":"2024-04-25T10:28:36.047545+0000","last_became_peered":"2024-04-25T10:28:36.047545+0000","last_unstale":"2024-04-25T10:28:38.036461+0000","last_undegraded":"2024-04-25T10:28:38.036461+0000","last_fullsized":"2024-04-25T10:28:38.036461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:25.424854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036325+0000","last_change":"2024-04-25T10:28:37.057740+0000","last_active":"2024-04-25T10:28:38.036325+0000","last_peered":"2024-04-25T10:28:38.036325+0000","last_clean":"2024-04-25T10:28:38.036325+0000","last_became_active":"2024-04-25T10:28:37.057406+0000","last_became_peered":"2024-04-25T10:28:37.057406+0000","last_unstale":"2024-04-25T10:28:38.036325+0000","last_undegraded":"2024-04-25T10:28:38.036325+0000","last_fullsized":"2024-04-25T10:28:38.036325+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:09.796346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.062065+0000","last_change":"2024-04-25T10:28:36.041843+0000","last_active":"2024-04-25T10:28:37.062065+0000","last_peered":"2024-04-25T10:28:37.062065+0000","last_clean":"2024-04-25T10:28:37.062065+0000","last_became_active":"2024-04-25T10:28:36.041561+0000","last_became_peered":"2024-04-25T10:28:36.041561+0000","last_unstale":"2024-04-25T10:28:37.062065+0000","last_undegraded":"2024-04-25T10:28:37.062065+0000","last_fullsized":"2024-04-25T10:28:37.062065+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:58.499656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065609+0000","last_change":"2024-04-25T10:28:37.062772+0000","last_active":"2024-04-25T10:28:37.065609+0000","last_peered":"2024-04-25T10:28:37.065609+0000","last_clean":"2024-04-25T10:28:37.065609+0000","last_became_active":"2024-04-25T10:28:37.061646+0000","last_became_peered":"2024-04-25T10:28:37.061646+0000","last_unstale":"2024-04-25T10:28:37.065609+0000","last_undegraded":"2024-04-25T10:28:37.065609+0000","last_fullsized":"2024-04-25T10:28:37.065609+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:53.028725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.060751+0000","last_change":"2024-04-25T10:28:36.043640+0000","last_active":"2024-04-25T10:28:37.060751+0000","last_peered":"2024-04-25T10:28:37.060751+0000","last_clean":"2024-04-25T10:28:37.060751+0000","last_became_active":"2024-04-25T10:28:36.043297+0000","last_became_peered":"2024-04-25T10:28:36.043297+0000","last_unstale":"2024-04-25T10:28:37.060751+0000","last_undegraded":"2024-04-25T10:28:37.060751+0000","last_fullsized":"2024-04-25T10:28:37.060751+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:47.674855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.048278+0000","last_change":"2024-04-25T10:28:37.046033+0000","last_active":"2024-04-25T10:28:37.048278+0000","last_peered":"2024-04-25T10:28:37.048278+0000","last_clean":"2024-04-25T10:28:37.048278+0000","last_became_active":"2024-04-25T10:28:37.045057+0000","last_became_peered":"2024-04-25T10:28:37.045057+0000","last_unstale":"2024-04-25T10:28:37.048278+0000","last_undegraded":"2024-04-25T10:28:37.048278+0000","last_fullsized":"2024-04-25T10:28:37.048278+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:57.664243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"140'1","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059925+0000","last_change":"2024-04-25T10:28:36.039014+0000","last_active":"2024-04-25T10:28:37.059925+0000","last_peered":"2024-04-25T10:28:37.059925+0000","last_clean":"2024-04-25T10:28:37.059925+0000","last_became_active":"2024-04-25T10:28:36.038323+0000","last_became_peered":"2024-04-25T10:28:36.038323+0000","last_unstale":"2024-04-25T10:28:37.059925+0000","last_undegraded":"2024-04-25T10:28:37.059925+0000","last_fullsized":"2024-04-25T10:28:37.059925+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:26.509117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056931+0000","last_change":"2024-04-25T10:28:37.049501+0000","last_active":"2024-04-25T10:28:37.056931+0000","last_peered":"2024-04-25T10:28:37.056931+0000","last_clean":"2024-04-25T10:28:37.056931+0000","last_became_active":"2024-04-25T10:28:37.048837+0000","last_became_peered":"2024-04-25T10:28:37.048837+0000","last_unstale":"2024-04-25T10:28:37.056931+0000","last_undegraded":"2024-04-25T10:28:37.056931+0000","last_fullsized":"2024-04-25T10:28:37.056931+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.857540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029419+0000","last_change":"2024-04-25T10:28:36.037922+0000","last_active":"2024-04-25T10:28:38.029419+0000","last_peered":"2024-04-25T10:28:38.029419+0000","last_clean":"2024-04-25T10:28:38.029419+0000","last_became_active":"2024-04-25T10:28:36.036888+0000","last_became_peered":"2024-04-25T10:28:36.036888+0000","last_unstale":"2024-04-25T10:28:38.029419+0000","last_undegraded":"2024-04-25T10:28:38.029419+0000","last_fullsized":"2024-04-25T10:28:38.029419+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:44.462729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.063306+0000","last_change":"2024-04-25T10:28:37.062761+0000","last_active":"2024-04-25T10:28:37.063306+0000","last_peered":"2024-04-25T10:28:37.063306+0000","last_clean":"2024-04-25T10:28:37.063306+0000","last_became_active":"2024-04-25T10:28:37.062314+0000","last_became_peered":"2024-04-25T10:28:37.062314+0000","last_unstale":"2024-04-25T10:28:37.063306+0000","last_undegraded":"2024-04-25T10:28:37.063306+0000","last_fullsized":"2024-04-25T10:28:37.063306+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:18.102053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.035949+0000","last_change":"2024-04-25T10:28:36.043430+0000","last_active":"2024-04-25T10:28:38.035949+0000","last_peered":"2024-04-25T10:28:38.035949+0000","last_clean":"2024-04-25T10:28:38.035949+0000","last_became_active":"2024-04-25T10:28:36.043159+0000","last_became_peered":"2024-04-25T10:28:36.043159+0000","last_unstale":"2024-04-25T10:28:38.035949+0000","last_undegraded":"2024-04-25T10:28:38.035949+0000","last_fullsized":"2024-04-25T10:28:38.035949+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:35.546777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.046181+0000","last_change":"2024-04-25T10:28:37.042364+0000","last_active":"2024-04-25T10:28:37.046181+0000","last_peered":"2024-04-25T10:28:37.046181+0000","last_clean":"2024-04-25T10:28:37.046181+0000","last_became_active":"2024-04-25T10:28:37.041860+0000","last_became_peered":"2024-04-25T10:28:37.041860+0000","last_unstale":"2024-04-25T10:28:37.046181+0000","last_undegraded":"2024-04-25T10:28:37.046181+0000","last_fullsized":"2024-04-25T10:28:37.046181+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:38.249660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.025120+0000","last_change":"2024-04-25T10:28:36.042822+0000","last_active":"2024-04-25T10:28:37.025120+0000","last_peered":"2024-04-25T10:28:37.025120+0000","last_clean":"2024-04-25T10:28:37.025120+0000","last_became_active":"2024-04-25T10:28:36.041356+0000","last_became_peered":"2024-04-25T10:28:36.041356+0000","last_unstale":"2024-04-25T10:28:37.025120+0000","last_undegraded":"2024-04-25T10:28:37.025120+0000","last_fullsized":"2024-04-25T10:28:37.025120+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.041869+0000","last_change":"2024-04-25T10:28:37.040166+0000","last_active":"2024-04-25T10:28:37.041869+0000","last_peered":"2024-04-25T10:28:37.041869+0000","last_clean":"2024-04-25T10:28:37.041869+0000","last_became_active":"2024-04-25T10:28:37.039898+0000","last_became_peered":"2024-04-25T10:28:37.039898+0000","last_unstale":"2024-04-25T10:28:37.041869+0000","last_undegraded":"2024-04-25T10:28:37.041869+0000","last_fullsized":"2024-04-25T10:28:37.041869+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:34.289939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036473+0000","last_change":"2024-04-25T10:28:36.048304+0000","last_active":"2024-04-25T10:28:38.036473+0000","last_peered":"2024-04-25T10:28:38.036473+0000","last_clean":"2024-04-25T10:28:38.036473+0000","last_became_active":"2024-04-25T10:28:36.046336+0000","last_became_peered":"2024-04-25T10:28:36.046336+0000","last_unstale":"2024-04-25T10:28:38.036473+0000","last_undegraded":"2024-04-25T10:28:38.036473+0000","last_fullsized":"2024-04-25T10:28:38.036473+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:41.353534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272606+0000","last_change":"2024-04-25T10:28:37.060509+0000","last_active":"2024-04-25T10:28:38.272606+0000","last_peered":"2024-04-25T10:28:38.272606+0000","last_clean":"2024-04-25T10:28:38.272606+0000","last_became_active":"2024-04-25T10:28:37.060221+0000","last_became_peered":"2024-04-25T10:28:37.060221+0000","last_unstale":"2024-04-25T10:28:38.272606+0000","last_undegraded":"2024-04-25T10:28:38.272606+0000","last_fullsized":"2024-04-25T10:28:38.272606+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:47.973064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.030026+0000","last_change":"2024-04-25T10:28:36.040280+0000","last_active":"2024-04-25T10:28:38.030026+0000","last_peered":"2024-04-25T10:28:38.030026+0000","last_clean":"2024-04-25T10:28:38.030026+0000","last_became_active":"2024-04-25T10:28:36.039414+0000","last_became_peered":"2024-04-25T10:28:36.039414+0000","last_unstale":"2024-04-25T10:28:38.030026+0000","last_undegraded":"2024-04-25T10:28:38.030026+0000","last_fullsized":"2024-04-25T10:28:38.030026+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:19.226855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038363+0000","last_change":"2024-04-25T10:28:37.036469+0000","last_active":"2024-04-25T10:28:38.038363+0000","last_peered":"2024-04-25T10:28:38.038363+0000","last_clean":"2024-04-25T10:28:38.038363+0000","last_became_active":"2024-04-25T10:28:37.036182+0000","last_became_peered":"2024-04-25T10:28:37.036182+0000","last_unstale":"2024-04-25T10:28:38.038363+0000","last_undegraded":"2024-04-25T10:28:38.038363+0000","last_fullsized":"2024-04-25T10:28:38.038363+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:05.155103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"140'6","reported_seq":20,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038038+0000","last_change":"2024-04-25T10:28:36.046708+0000","last_active":"2024-04-25T10:28:38.038038+0000","last_peered":"2024-04-25T10:28:38.038038+0000","last_clean":"2024-04-25T10:28:38.038038+0000","last_became_active":"2024-04-25T10:28:36.044743+0000","last_became_peered":"2024-04-25T10:28:36.044743+0000","last_unstale":"2024-04-25T10:28:38.038038+0000","last_undegraded":"2024-04-25T10:28:38.038038+0000","last_fullsized":"2024-04-25T10:28:38.038038+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.158020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057049+0000","last_change":"2024-04-25T10:28:37.047525+0000","last_active":"2024-04-25T10:28:37.057049+0000","last_peered":"2024-04-25T10:28:37.057049+0000","last_clean":"2024-04-25T10:28:37.057049+0000","last_became_active":"2024-04-25T10:28:37.046252+0000","last_became_peered":"2024-04-25T10:28:37.046252+0000","last_unstale":"2024-04-25T10:28:37.057049+0000","last_undegraded":"2024-04-25T10:28:37.057049+0000","last_fullsized":"2024-04-25T10:28:37.057049+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:42.356192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.063641+0000","last_change":"2024-04-25T10:28:36.042988+0000","last_active":"2024-04-25T10:28:37.063641+0000","last_peered":"2024-04-25T10:28:37.063641+0000","last_clean":"2024-04-25T10:28:37.063641+0000","last_became_active":"2024-04-25T10:28:36.042673+0000","last_became_peered":"2024-04-25T10:28:36.042673+0000","last_unstale":"2024-04-25T10:28:37.063641+0000","last_undegraded":"2024-04-25T10:28:37.063641+0000","last_fullsized":"2024-04-25T10:28:37.063641+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:24.905838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056171+0000","last_change":"2024-04-25T10:28:37.053506+0000","last_active":"2024-04-25T10:28:37.056171+0000","last_peered":"2024-04-25T10:28:37.056171+0000","last_clean":"2024-04-25T10:28:37.056171+0000","last_became_active":"2024-04-25T10:28:37.053247+0000","last_became_peered":"2024-04-25T10:28:37.053247+0000","last_unstale":"2024-04-25T10:28:37.056171+0000","last_undegraded":"2024-04-25T10:28:37.056171+0000","last_fullsized":"2024-04-25T10:28:37.056171+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:32.767324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272645+0000","last_change":"2024-04-25T10:28:36.047901+0000","last_active":"2024-04-25T10:28:38.272645+0000","last_peered":"2024-04-25T10:28:38.272645+0000","last_clean":"2024-04-25T10:28:38.272645+0000","last_became_active":"2024-04-25T10:28:36.046402+0000","last_became_peered":"2024-04-25T10:28:36.046402+0000","last_unstale":"2024-04-25T10:28:38.272645+0000","last_undegraded":"2024-04-25T10:28:38.272645+0000","last_fullsized":"2024-04-25T10:28:38.272645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:34.613987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065572+0000","last_change":"2024-04-25T10:28:37.061094+0000","last_active":"2024-04-25T10:28:37.065572+0000","last_peered":"2024-04-25T10:28:37.065572+0000","last_clean":"2024-04-25T10:28:37.065572+0000","last_became_active":"2024-04-25T10:28:37.059350+0000","last_became_peered":"2024-04-25T10:28:37.059350+0000","last_unstale":"2024-04-25T10:28:37.065572+0000","last_undegraded":"2024-04-25T10:28:37.065572+0000","last_fullsized":"2024-04-25T10:28:37.065572+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:47.973064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.034637+0000","last_change":"2024-04-25T10:28:36.049124+0000","last_active":"2024-04-25T10:28:37.034637+0000","last_peered":"2024-04-25T10:28:37.034637+0000","last_clean":"2024-04-25T10:28:37.034637+0000","last_became_active":"2024-04-25T10:28:36.048853+0000","last_became_peered":"2024-04-25T10:28:36.048853+0000","last_unstale":"2024-04-25T10:28:37.034637+0000","last_undegraded":"2024-04-25T10:28:37.034637+0000","last_fullsized":"2024-04-25T10:28:37.034637+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:04.292402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.046160+0000","last_change":"2024-04-25T10:28:37.040887+0000","last_active":"2024-04-25T10:28:37.046160+0000","last_peered":"2024-04-25T10:28:37.046160+0000","last_clean":"2024-04-25T10:28:37.046160+0000","last_became_active":"2024-04-25T10:28:37.040594+0000","last_became_peered":"2024-04-25T10:28:37.040594+0000","last_unstale":"2024-04-25T10:28:37.046160+0000","last_undegraded":"2024-04-25T10:28:37.046160+0000","last_fullsized":"2024-04-25T10:28:37.046160+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:09.586495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"140'8","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.068039+0000","last_change":"2024-04-25T10:28:36.046419+0000","last_active":"2024-04-25T10:28:37.068039+0000","last_peered":"2024-04-25T10:28:37.068039+0000","last_clean":"2024-04-25T10:28:37.068039+0000","last_became_active":"2024-04-25T10:28:36.042259+0000","last_became_peered":"2024-04-25T10:28:36.042259+0000","last_unstale":"2024-04-25T10:28:37.068039+0000","last_undegraded":"2024-04-25T10:28:37.068039+0000","last_fullsized":"2024-04-25T10:28:37.068039+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.792545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.030021+0000","last_change":"2024-04-25T10:28:37.062915+0000","last_active":"2024-04-25T10:28:38.030021+0000","last_peered":"2024-04-25T10:28:38.030021+0000","last_clean":"2024-04-25T10:28:38.030021+0000","last_became_active":"2024-04-25T10:28:37.062515+0000","last_became_peered":"2024-04-25T10:28:37.062515+0000","last_unstale":"2024-04-25T10:28:38.030021+0000","last_undegraded":"2024-04-25T10:28:38.030021+0000","last_fullsized":"2024-04-25T10:28:38.030021+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.768516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"140'1","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059918+0000","last_change":"2024-04-25T10:28:36.042594+0000","last_active":"2024-04-25T10:28:37.059918+0000","last_peered":"2024-04-25T10:28:37.059918+0000","last_clean":"2024-04-25T10:28:37.059918+0000","last_became_active":"2024-04-25T10:28:36.042093+0000","last_became_peered":"2024-04-25T10:28:36.042093+0000","last_unstale":"2024-04-25T10:28:37.059918+0000","last_undegraded":"2024-04-25T10:28:37.059918+0000","last_fullsized":"2024-04-25T10:28:37.059918+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:00.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057084+0000","last_change":"2024-04-25T10:28:37.054548+0000","last_active":"2024-04-25T10:28:37.057084+0000","last_peered":"2024-04-25T10:28:37.057084+0000","last_clean":"2024-04-25T10:28:37.057084+0000","last_became_active":"2024-04-25T10:28:37.054258+0000","last_became_peered":"2024-04-25T10:28:37.054258+0000","last_unstale":"2024-04-25T10:28:37.057084+0000","last_undegraded":"2024-04-25T10:28:37.057084+0000","last_fullsized":"2024-04-25T10:28:37.057084+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:57.832892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059060+0000","last_change":"2024-04-25T10:28:36.041736+0000","last_active":"2024-04-25T10:28:37.059060+0000","last_peered":"2024-04-25T10:28:37.059060+0000","last_clean":"2024-04-25T10:28:37.059060+0000","last_became_active":"2024-04-25T10:28:36.041309+0000","last_became_peered":"2024-04-25T10:28:36.041309+0000","last_unstale":"2024-04-25T10:28:37.059060+0000","last_undegraded":"2024-04-25T10:28:37.059060+0000","last_fullsized":"2024-04-25T10:28:37.059060+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:20.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029674+0000","last_change":"2024-04-25T10:28:37.063585+0000","last_active":"2024-04-25T10:28:38.029674+0000","last_peered":"2024-04-25T10:28:38.029674+0000","last_clean":"2024-04-25T10:28:38.029674+0000","last_became_active":"2024-04-25T10:28:37.063326+0000","last_became_peered":"2024-04-25T10:28:37.063326+0000","last_unstale":"2024-04-25T10:28:38.029674+0000","last_undegraded":"2024-04-25T10:28:38.029674+0000","last_fullsized":"2024-04-25T10:28:38.029674+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:05.295060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038363+0000","last_change":"2024-04-25T10:28:36.047972+0000","last_active":"2024-04-25T10:28:38.038363+0000","last_peered":"2024-04-25T10:28:38.038363+0000","last_clean":"2024-04-25T10:28:38.038363+0000","last_became_active":"2024-04-25T10:28:36.046813+0000","last_became_peered":"2024-04-25T10:28:36.046813+0000","last_unstale":"2024-04-25T10:28:38.038363+0000","last_undegraded":"2024-04-25T10:28:38.038363+0000","last_fullsized":"2024-04-25T10:28:38.038363+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.854882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036610+0000","last_change":"2024-04-25T10:28:37.053645+0000","last_active":"2024-04-25T10:28:38.036610+0000","last_peered":"2024-04-25T10:28:38.036610+0000","last_clean":"2024-04-25T10:28:38.036610+0000","last_became_active":"2024-04-25T10:28:37.053377+0000","last_became_peered":"2024-04-25T10:28:37.053377+0000","last_unstale":"2024-04-25T10:28:38.036610+0000","last_undegraded":"2024-04-25T10:28:38.036610+0000","last_fullsized":"2024-04-25T10:28:38.036610+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.888830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.036259+0000","last_change":"2024-04-25T10:28:36.048436+0000","last_active":"2024-04-25T10:28:37.036259+0000","last_peered":"2024-04-25T10:28:37.036259+0000","last_clean":"2024-04-25T10:28:37.036259+0000","last_became_active":"2024-04-25T10:28:36.048167+0000","last_became_peered":"2024-04-25T10:28:36.048167+0000","last_unstale":"2024-04-25T10:28:37.036259+0000","last_undegraded":"2024-04-25T10:28:37.036259+0000","last_fullsized":"2024-04-25T10:28:37.036259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:00.327800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272539+0000","last_change":"2024-04-25T10:28:37.051773+0000","last_active":"2024-04-25T10:28:38.272539+0000","last_peered":"2024-04-25T10:28:38.272539+0000","last_clean":"2024-04-25T10:28:38.272539+0000","last_became_active":"2024-04-25T10:28:37.051475+0000","last_became_peered":"2024-04-25T10:28:37.051475+0000","last_unstale":"2024-04-25T10:28:38.272539+0000","last_undegraded":"2024-04-25T10:28:38.272539+0000","last_fullsized":"2024-04-25T10:28:38.272539+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:40.512214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036211+0000","last_change":"2024-04-25T10:28:36.044071+0000","last_active":"2024-04-25T10:28:38.036211+0000","last_peered":"2024-04-25T10:28:38.036211+0000","last_clean":"2024-04-25T10:28:38.036211+0000","last_became_active":"2024-04-25T10:28:36.042998+0000","last_became_peered":"2024-04-25T10:28:36.042998+0000","last_unstale":"2024-04-25T10:28:38.036211+0000","last_undegraded":"2024-04-25T10:28:38.036211+0000","last_fullsized":"2024-04-25T10:28:38.036211+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.158020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057083+0000","last_change":"2024-04-25T10:28:37.054529+0000","last_active":"2024-04-25T10:28:37.057083+0000","last_peered":"2024-04-25T10:28:37.057083+0000","last_clean":"2024-04-25T10:28:37.057083+0000","last_became_active":"2024-04-25T10:28:37.054177+0000","last_became_peered":"2024-04-25T10:28:37.054177+0000","last_unstale":"2024-04-25T10:28:37.057083+0000","last_undegraded":"2024-04-25T10:28:37.057083+0000","last_fullsized":"2024-04-25T10:28:37.057083+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:00.252490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.040011+0000","last_change":"2024-04-25T10:28:36.041403+0000","last_active":"2024-04-25T10:28:37.040011+0000","last_peered":"2024-04-25T10:28:37.040011+0000","last_clean":"2024-04-25T10:28:37.040011+0000","last_became_active":"2024-04-25T10:28:36.039220+0000","last_became_peered":"2024-04-25T10:28:36.039220+0000","last_unstale":"2024-04-25T10:28:37.040011+0000","last_undegraded":"2024-04-25T10:28:37.040011+0000","last_fullsized":"2024-04-25T10:28:37.040011+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:42.602655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272360+0000","last_change":"2024-04-25T10:28:37.054100+0000","last_active":"2024-04-25T10:28:38.272360+0000","last_peered":"2024-04-25T10:28:38.272360+0000","last_clean":"2024-04-25T10:28:38.272360+0000","last_became_active":"2024-04-25T10:28:37.053789+0000","last_became_peered":"2024-04-25T10:28:37.053789+0000","last_unstale":"2024-04-25T10:28:38.272360+0000","last_undegraded":"2024-04-25T10:28:38.272360+0000","last_fullsized":"2024-04-25T10:28:38.272360+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.427512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.038061+0000","last_change":"2024-04-25T10:28:36.042221+0000","last_active":"2024-04-25T10:28:37.038061+0000","last_peered":"2024-04-25T10:28:37.038061+0000","last_clean":"2024-04-25T10:28:37.038061+0000","last_became_active":"2024-04-25T10:28:36.041822+0000","last_became_peered":"2024-04-25T10:28:36.041822+0000","last_unstale":"2024-04-25T10:28:37.038061+0000","last_undegraded":"2024-04-25T10:28:37.038061+0000","last_fullsized":"2024-04-25T10:28:37.038061+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:34.613987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057846+0000","last_change":"2024-04-25T10:28:37.055719+0000","last_active":"2024-04-25T10:28:37.057846+0000","last_peered":"2024-04-25T10:28:37.057846+0000","last_clean":"2024-04-25T10:28:37.057846+0000","last_became_active":"2024-04-25T10:28:37.055296+0000","last_became_peered":"2024-04-25T10:28:37.055296+0000","last_unstale":"2024-04-25T10:28:37.057846+0000","last_undegraded":"2024-04-25T10:28:37.057846+0000","last_fullsized":"2024-04-25T10:28:37.057846+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:14.985960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.037784+0000","last_change":"2024-04-25T10:28:36.041123+0000","last_active":"2024-04-25T10:28:37.037784+0000","last_peered":"2024-04-25T10:28:37.037784+0000","last_clean":"2024-04-25T10:28:37.037784+0000","last_became_active":"2024-04-25T10:28:36.040675+0000","last_became_peered":"2024-04-25T10:28:36.040675+0000","last_unstale":"2024-04-25T10:28:37.037784+0000","last_undegraded":"2024-04-25T10:28:37.037784+0000","last_fullsized":"2024-04-25T10:28:37.037784+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:56.444371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065069+0000","last_change":"2024-04-25T10:28:37.058096+0000","last_active":"2024-04-25T10:28:37.065069+0000","last_peered":"2024-04-25T10:28:37.065069+0000","last_clean":"2024-04-25T10:28:37.065069+0000","last_became_active":"2024-04-25T10:28:37.057793+0000","last_became_peered":"2024-04-25T10:28:37.057793+0000","last_unstale":"2024-04-25T10:28:37.065069+0000","last_undegraded":"2024-04-25T10:28:37.065069+0000","last_fullsized":"2024-04-25T10:28:37.065069+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:07.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057651+0000","last_change":"2024-04-25T10:28:36.043845+0000","last_active":"2024-04-25T10:28:37.057651+0000","last_peered":"2024-04-25T10:28:37.057651+0000","last_clean":"2024-04-25T10:28:37.057651+0000","last_became_active":"2024-04-25T10:28:36.043563+0000","last_became_peered":"2024-04-25T10:28:36.043563+0000","last_unstale":"2024-04-25T10:28:37.057651+0000","last_undegraded":"2024-04-25T10:28:37.057651+0000","last_fullsized":"2024-04-25T10:28:37.057651+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:35.546777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.064414+0000","last_change":"2024-04-25T10:28:37.058471+0000","last_active":"2024-04-25T10:28:37.064414+0000","last_peered":"2024-04-25T10:28:37.064414+0000","last_clean":"2024-04-25T10:28:37.064414+0000","last_became_active":"2024-04-25T10:28:37.057931+0000","last_became_peered":"2024-04-25T10:28:37.057931+0000","last_unstale":"2024-04-25T10:28:37.064414+0000","last_undegraded":"2024-04-25T10:28:37.064414+0000","last_fullsized":"2024-04-25T10:28:37.064414+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:11.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.038486+0000","last_change":"2024-04-25T10:28:36.048060+0000","last_active":"2024-04-25T10:28:37.038486+0000","last_peered":"2024-04-25T10:28:37.038486+0000","last_clean":"2024-04-25T10:28:37.038486+0000","last_became_active":"2024-04-25T10:28:36.047740+0000","last_became_peered":"2024-04-25T10:28:36.047740+0000","last_unstale":"2024-04-25T10:28:37.038486+0000","last_undegraded":"2024-04-25T10:28:37.038486+0000","last_fullsized":"2024-04-25T10:28:37.038486+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:08.703794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.043676+0000","last_change":"2024-04-25T10:28:37.039857+0000","last_active":"2024-04-25T10:28:37.043676+0000","last_peered":"2024-04-25T10:28:37.043676+0000","last_clean":"2024-04-25T10:28:37.043676+0000","last_became_active":"2024-04-25T10:28:37.039119+0000","last_became_peered":"2024-04-25T10:28:37.039119+0000","last_unstale":"2024-04-25T10:28:37.043676+0000","last_undegraded":"2024-04-25T10:28:37.043676+0000","last_fullsized":"2024-04-25T10:28:37.043676+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:15.425634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038304+0000","last_change":"2024-04-25T10:28:36.040806+0000","last_active":"2024-04-25T10:28:38.038304+0000","last_peered":"2024-04-25T10:28:38.038304+0000","last_clean":"2024-04-25T10:28:38.038304+0000","last_became_active":"2024-04-25T10:28:36.040479+0000","last_became_peered":"2024-04-25T10:28:36.040479+0000","last_unstale":"2024-04-25T10:28:38.038304+0000","last_undegraded":"2024-04-25T10:28:38.038304+0000","last_fullsized":"2024-04-25T10:28:38.038304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:41.353534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.046103+0000","last_change":"2024-04-25T10:28:37.041638+0000","last_active":"2024-04-25T10:28:37.046103+0000","last_peered":"2024-04-25T10:28:37.046103+0000","last_clean":"2024-04-25T10:28:37.046103+0000","last_became_active":"2024-04-25T10:28:37.041328+0000","last_became_peered":"2024-04-25T10:28:37.041328+0000","last_unstale":"2024-04-25T10:28:37.046103+0000","last_undegraded":"2024-04-25T10:28:37.046103+0000","last_fullsized":"2024-04-25T10:28:37.046103+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:09.489406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038170+0000","last_change":"2024-04-25T10:28:36.046917+0000","last_active":"2024-04-25T10:28:38.038170+0000","last_peered":"2024-04-25T10:28:38.038170+0000","last_clean":"2024-04-25T10:28:38.038170+0000","last_became_active":"2024-04-25T10:28:36.046439+0000","last_became_peered":"2024-04-25T10:28:36.046439+0000","last_unstale":"2024-04-25T10:28:38.038170+0000","last_undegraded":"2024-04-25T10:28:38.038170+0000","last_fullsized":"2024-04-25T10:28:38.038170+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:25.424854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.037925+0000","last_change":"2024-04-25T10:28:37.057867+0000","last_active":"2024-04-25T10:28:38.037925+0000","last_peered":"2024-04-25T10:28:38.037925+0000","last_clean":"2024-04-25T10:28:38.037925+0000","last_became_active":"2024-04-25T10:28:37.056874+0000","last_became_peered":"2024-04-25T10:28:37.056874+0000","last_unstale":"2024-04-25T10:28:38.037925+0000","last_undegraded":"2024-04-25T10:28:38.037925+0000","last_fullsized":"2024-04-25T10:28:38.037925+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:16.231448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.037105+0000","last_change":"2024-04-25T10:28:36.049313+0000","last_active":"2024-04-25T10:28:37.037105+0000","last_peered":"2024-04-25T10:28:37.037105+0000","last_clean":"2024-04-25T10:28:37.037105+0000","last_became_active":"2024-04-25T10:28:36.049048+0000","last_became_peered":"2024-04-25T10:28:36.049048+0000","last_unstale":"2024-04-25T10:28:37.037105+0000","last_undegraded":"2024-04-25T10:28:37.037105+0000","last_fullsized":"2024-04-25T10:28:37.037105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:40.813689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.049651+0000","last_change":"2024-04-25T10:28:37.042538+0000","last_active":"2024-04-25T10:28:37.049651+0000","last_peered":"2024-04-25T10:28:37.049651+0000","last_clean":"2024-04-25T10:28:37.049651+0000","last_became_active":"2024-04-25T10:28:37.042193+0000","last_became_peered":"2024-04-25T10:28:37.042193+0000","last_unstale":"2024-04-25T10:28:37.049651+0000","last_undegraded":"2024-04-25T10:28:37.049651+0000","last_fullsized":"2024-04-25T10:28:37.049651+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:15.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.026148+0000","last_change":"2024-04-25T10:28:36.037038+0000","last_active":"2024-04-25T10:28:37.026148+0000","last_peered":"2024-04-25T10:28:37.026148+0000","last_clean":"2024-04-25T10:28:37.026148+0000","last_became_active":"2024-04-25T10:28:36.036667+0000","last_became_peered":"2024-04-25T10:28:36.036667+0000","last_unstale":"2024-04-25T10:28:37.026148+0000","last_undegraded":"2024-04-25T10:28:37.026148+0000","last_fullsized":"2024-04-25T10:28:37.026148+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:57.447189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056472+0000","last_change":"2024-04-25T10:28:37.045119+0000","last_active":"2024-04-25T10:28:37.056472+0000","last_peered":"2024-04-25T10:28:37.056472+0000","last_clean":"2024-04-25T10:28:37.056472+0000","last_became_active":"2024-04-25T10:28:37.043122+0000","last_became_peered":"2024-04-25T10:28:37.043122+0000","last_unstale":"2024-04-25T10:28:37.056472+0000","last_undegraded":"2024-04-25T10:28:37.056472+0000","last_fullsized":"2024-04-25T10:28:37.056472+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:45.160678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272196+0000","last_change":"2024-04-25T10:28:36.038955+0000","last_active":"2024-04-25T10:28:38.272196+0000","last_peered":"2024-04-25T10:28:38.272196+0000","last_clean":"2024-04-25T10:28:38.272196+0000","last_became_active":"2024-04-25T10:28:36.038499+0000","last_became_peered":"2024-04-25T10:28:36.038499+0000","last_unstale":"2024-04-25T10:28:38.272196+0000","last_undegraded":"2024-04-25T10:28:38.272196+0000","last_fullsized":"2024-04-25T10:28:38.272196+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:56.444371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.035638+0000","last_change":"2024-04-25T10:28:37.030408+0000","last_active":"2024-04-25T10:28:37.035638+0000","last_peered":"2024-04-25T10:28:37.035638+0000","last_clean":"2024-04-25T10:28:37.035638+0000","last_became_active":"2024-04-25T10:28:37.030130+0000","last_became_peered":"2024-04-25T10:28:37.030130+0000","last_unstale":"2024-04-25T10:28:37.035638+0000","last_undegraded":"2024-04-25T10:28:37.035638+0000","last_fullsized":"2024-04-25T10:28:37.035638+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:54.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.035786+0000","last_change":"2024-04-25T10:28:36.043816+0000","last_active":"2024-04-25T10:28:38.035786+0000","last_peered":"2024-04-25T10:28:38.035786+0000","last_clean":"2024-04-25T10:28:38.035786+0000","last_became_active":"2024-04-25T10:28:36.043403+0000","last_became_peered":"2024-04-25T10:28:36.043403+0000","last_unstale":"2024-04-25T10:28:38.035786+0000","last_undegraded":"2024-04-25T10:28:38.035786+0000","last_fullsized":"2024-04-25T10:28:38.035786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:00.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038050+0000","last_change":"2024-04-25T10:28:37.048328+0000","last_active":"2024-04-25T10:28:38.038050+0000","last_peered":"2024-04-25T10:28:38.038050+0000","last_clean":"2024-04-25T10:28:38.038050+0000","last_became_active":"2024-04-25T10:28:37.047926+0000","last_became_peered":"2024-04-25T10:28:37.047926+0000","last_unstale":"2024-04-25T10:28:38.038050+0000","last_undegraded":"2024-04-25T10:28:38.038050+0000","last_fullsized":"2024-04-25T10:28:38.038050+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.888830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"140'3","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272650+0000","last_change":"2024-04-25T10:28:36.046438+0000","last_active":"2024-04-25T10:28:38.272650+0000","last_peered":"2024-04-25T10:28:38.272650+0000","last_clean":"2024-04-25T10:28:38.272650+0000","last_became_active":"2024-04-25T10:28:36.045084+0000","last_became_peered":"2024-04-25T10:28:36.045084+0000","last_unstale":"2024-04-25T10:28:38.272650+0000","last_undegraded":"2024-04-25T10:28:38.272650+0000","last_fullsized":"2024-04-25T10:28:38.272650+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:30.187545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":399,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029147+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:28:38.029147+0000","last_peered":"2024-04-25T10:28:38.029147+0000","last_clean":"2024-04-25T10:28:38.029147+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:28:38.029147+0000","last_undegraded":"2024-04-25T10:28:38.029147+0000","last_fullsized":"2024-04-25T10:28:38.029147+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50928,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320912,"statfs":{"total":96636764160,"available":96584613888,"internally_reserved":0,"allocated":1744896,"data_stored":761159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509574,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55104,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316736,"statfs":{"total":96636764160,"available":96580337664,"internally_reserved":0,"allocated":1757184,"data_stored":767459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2375680,"data_stored":1359633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509573,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59268,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312572,"statfs":{"total":96636764160,"available":96576073728,"internally_reserved":0,"allocated":1753088,"data_stored":761775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509573,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46776,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325064,"statfs":{"total":96636764160,"available":96588865536,"internally_reserved":0,"allocated":1748992,"data_stored":761181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509574,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64008,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307832,"statfs":{"total":96636764160,"available":96571219968,"internally_reserved":0,"allocated":2355200,"data_stored":1357897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509574,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":761249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542276,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46764,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325076,"statfs":{"total":96636764160,"available":96588877824,"internally_reserved":0,"allocated":1732608,"data_stored":754384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:28:41.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:28:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f44800640 1 Processor -- start 2024-04-25T10:28:41.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f44800640 1 -- start start 2024-04-25T10:28:41.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f44800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5f3c0d1820 0x7f5f3c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f44800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c0b1af0 con 0x7f5f3c0b15b0 2024-04-25T10:28:41.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f44800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c0b1dd0 con 0x7f5f3c15e050 2024-04-25T10:28:41.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f44800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0b20b0 con 0x7f5f3c0d1820 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f42800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5f3c0b15b0 0x7f5f3c0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52950/0 (socket says 172.21.15.45:52950) 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f42800640 1 -- 172.21.15.45:0/4274023849 learned_addr learned my addr 172.21.15.45:0/4274023849 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f41e00640 1 --2- 172.21.15.45:0/4274023849 >> v2:172.21.15.117:3300/0 conn(0x7f5f3c0d1820 0x7f5f3c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f40a00640 1 -- 172.21.15.45:0/4274023849 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802725026 0 0) 0x7f5f3c0b1af0 con 0x7f5f3c0b15b0 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f40a00640 1 -- 172.21.15.45:0/4274023849 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f24003530 con 0x7f5f3c0b15b0 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f41e00640 1 -- 172.21.15.45:0/4274023849 >> v1:172.21.15.117:6789/0 conn(0x7f5f3c15e050 legacy=0x7f5f3c0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f41e00640 1 -- 172.21.15.45:0/4274023849 >> v1:172.21.15.45:6789/0 conn(0x7f5f3c0b15b0 legacy=0x7f5f3c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.349+0000 7f5f41e00640 1 -- 172.21.15.45:0/4274023849 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0b2390 con 0x7f5f3c0d1820 2024-04-25T10:28:41.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f41e00640 1 --2- 172.21.15.45:0/4274023849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c0d1c00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5f300097c0 tx=0x7f5f3001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4923a919e3d88123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:41.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f40a00640 1 -- 172.21.15.45:0/4274023849 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300aa620 con 0x7f5f3c0d1820 2024-04-25T10:28:41.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f40a00640 1 -- 172.21.15.45:0/4274023849 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f300af070 con 0x7f5f3c0d1820 2024-04-25T10:28:41.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f40a00640 1 -- 172.21.15.45:0/4274023849 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300aa9e0 con 0x7f5f3c0d1820 2024-04-25T10:28:41.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 -- 172.21.15.45:0/4274023849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0d1820 msgr2=0x7f5f3c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 --2- 172.21.15.45:0/4274023849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c0d1c00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5f300097c0 tx=0x7f5f3001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 -- 172.21.15.45:0/4274023849 shutdown_connections 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 --2- 172.21.15.45:0/4274023849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c0d1c00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 -- 172.21.15.45:0/4274023849 >> 172.21.15.45:0/4274023849 conn(0x7f5f3c09a9d0 msgr2=0x7f5f3c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 -- 172.21.15.45:0/4274023849 shutdown_connections 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.353+0000 7f5f44800640 1 -- 172.21.15.45:0/4274023849 wait complete. 2024-04-25T10:28:41.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 Processor -- start 2024-04-25T10:28:41.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- start start 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0b15b0 0x7f5f3c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c15e050 0x7f5f3c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0d0f20 con 0x7f5f3c0d1820 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c0d0da0 con 0x7f5f3c0b15b0 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c16b9b0 con 0x7f5f3c15e050 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58092/0 (socket says 172.21.15.45:58092) 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 -- 172.21.15.45:0/962198236 learned_addr learned my addr 172.21.15.45:0/962198236 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:41.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41e00640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0b15b0 0x7f5f3c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f42800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c15e050 0x7f5f3c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 -- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c15e050 msgr2=0x7f5f3c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c15e050 0x7f5f3c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 -- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0b15b0 msgr2=0x7f5f3c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0b15b0 0x7f5f3c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 -- 172.21.15.45:0/962198236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c15ca20 con 0x7f5f3c0d1820 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41e00640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0b15b0 0x7f5f3c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f41400640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c152350 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c460 tx=0x7f5f2c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c016020 con 0x7f5f3c0d1820 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c0052d0 con 0x7f5f3c0d1820 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c15cd20 con 0x7f5f3c0d1820 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c007260 con 0x7f5f3c0d1820 2024-04-25T10:28:41.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c0b4780 con 0x7f5f3c0d1820 2024-04-25T10:28:41.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.357+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c0b4980 con 0x7f5f3c0d1820 2024-04-25T10:28:41.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.361+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f2c007400 con 0x7f5f3c0d1820 2024-04-25T10:28:41.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.361+0000 7f5f36000640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.361+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f2c084a60 con 0x7f5f3c0d1820 2024-04-25T10:28:41.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.361+0000 7f5f41e00640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.361+0000 7f5f41e00640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5f300ae000 tx=0x7f5f300a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.365+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f2c020020 con 0x7f5f3c0d1820 2024-04-25T10:28:41.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.525+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f3c04ef30 con 0x7f5f20066f60 2024-04-25T10:28:41.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f36000640 1 -- 172.21.15.45:0/962198236 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252438 (secure 0 0 0) 0x7f5f3c04ef30 con 0x7f5f20066f60 2024-04-25T10:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:41.535 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:28:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 msgr2=0x7f5f20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5f300ae000 tx=0x7f5f300a7000 comp rx=0 tx=0).stop 2024-04-25T10:28:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 msgr2=0x7f5f3c152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c152350 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00c460 tx=0x7f5f2c00c920 comp rx=0 tx=0).stop 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 shutdown_connections 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f20066f60 0x7f5f20069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f3c15e050 0x7f5f3c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f3c0d1820 0x7f5f3c152350 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 --2- 172.21.15.45:0/962198236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3c0b15b0 0x7f5f3c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 >> 172.21.15.45:0/962198236 conn(0x7f5f3c09a9d0 msgr2=0x7f5f3c155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 shutdown_connections 2024-04-25T10:28:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.533+0000 7f5f44800640 1 -- 172.21.15.45:0/962198236 wait complete. 2024-04-25T10:28:41.554 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-25T10:28:41.023605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":437816,"kb_used_data":14860,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754536904,"statfs":{"total":773094113280,"available":772645789696,"internally_reserved":0,"allocated":15216640,"data_stored":7284737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001229"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050382+0000","last_change":"2024-04-25T10:28:37.045763+0000","last_active":"2024-04-25T10:28:37.050382+0000","last_peered":"2024-04-25T10:28:37.050382+0000","last_clean":"2024-04-25T10:28:37.050382+0000","last_became_active":"2024-04-25T10:28:37.045491+0000","last_became_peered":"2024-04-25T10:28:37.045491+0000","last_unstale":"2024-04-25T10:28:37.050382+0000","last_undegraded":"2024-04-25T10:28:37.050382+0000","last_fullsized":"2024-04-25T10:28:37.050382+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:50.509447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056390+0000","last_change":"2024-04-25T10:28:37.047130+0000","last_active":"2024-04-25T10:28:37.056390+0000","last_peered":"2024-04-25T10:28:37.056390+0000","last_clean":"2024-04-25T10:28:37.056390+0000","last_became_active":"2024-04-25T10:28:37.046805+0000","last_became_peered":"2024-04-25T10:28:37.046805+0000","last_unstale":"2024-04-25T10:28:37.056390+0000","last_undegraded":"2024-04-25T10:28:37.056390+0000","last_fullsized":"2024-04-25T10:28:37.056390+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:16.796196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050051+0000","last_change":"2024-04-25T10:28:37.038693+0000","last_active":"2024-04-25T10:28:37.050051+0000","last_peered":"2024-04-25T10:28:37.050051+0000","last_clean":"2024-04-25T10:28:37.050051+0000","last_became_active":"2024-04-25T10:28:37.038379+0000","last_became_peered":"2024-04-25T10:28:37.038379+0000","last_unstale":"2024-04-25T10:28:37.050051+0000","last_undegraded":"2024-04-25T10:28:37.050051+0000","last_fullsized":"2024-04-25T10:28:37.050051+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:03.963295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065614+0000","last_change":"2024-04-25T10:28:37.059436+0000","last_active":"2024-04-25T10:28:37.065614+0000","last_peered":"2024-04-25T10:28:37.065614+0000","last_clean":"2024-04-25T10:28:37.065614+0000","last_became_active":"2024-04-25T10:28:37.059176+0000","last_became_peered":"2024-04-25T10:28:37.059176+0000","last_unstale":"2024-04-25T10:28:37.065614+0000","last_undegraded":"2024-04-25T10:28:37.065614+0000","last_fullsized":"2024-04-25T10:28:37.065614+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:40.512214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029547+0000","last_change":"2024-04-25T10:28:37.050541+0000","last_active":"2024-04-25T10:28:38.029547+0000","last_peered":"2024-04-25T10:28:38.029547+0000","last_clean":"2024-04-25T10:28:38.029547+0000","last_became_active":"2024-04-25T10:28:37.047862+0000","last_became_peered":"2024-04-25T10:28:37.047862+0000","last_unstale":"2024-04-25T10:28:38.029547+0000","last_undegraded":"2024-04-25T10:28:38.029547+0000","last_fullsized":"2024-04-25T10:28:38.029547+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:34.309365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036048+0000","last_change":"2024-04-25T10:28:37.039418+0000","last_active":"2024-04-25T10:28:38.036048+0000","last_peered":"2024-04-25T10:28:38.036048+0000","last_clean":"2024-04-25T10:28:38.036048+0000","last_became_active":"2024-04-25T10:28:37.039121+0000","last_became_peered":"2024-04-25T10:28:37.039121+0000","last_unstale":"2024-04-25T10:28:38.036048+0000","last_undegraded":"2024-04-25T10:28:38.036048+0000","last_fullsized":"2024-04-25T10:28:38.036048+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:04.755093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.037641+0000","last_change":"2024-04-25T10:28:37.041979+0000","last_active":"2024-04-25T10:28:38.037641+0000","last_peered":"2024-04-25T10:28:38.037641+0000","last_clean":"2024-04-25T10:28:38.037641+0000","last_became_active":"2024-04-25T10:28:37.041682+0000","last_became_peered":"2024-04-25T10:28:37.041682+0000","last_unstale":"2024-04-25T10:28:38.037641+0000","last_undegraded":"2024-04-25T10:28:38.037641+0000","last_fullsized":"2024-04-25T10:28:38.037641+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:09.796346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059460+0000","last_change":"2024-04-25T10:28:37.058215+0000","last_active":"2024-04-25T10:28:37.059460+0000","last_peered":"2024-04-25T10:28:37.059460+0000","last_clean":"2024-04-25T10:28:37.059460+0000","last_became_active":"2024-04-25T10:28:37.057870+0000","last_became_peered":"2024-04-25T10:28:37.057870+0000","last_unstale":"2024-04-25T10:28:37.059460+0000","last_undegraded":"2024-04-25T10:28:37.059460+0000","last_fullsized":"2024-04-25T10:28:37.059460+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:17.890811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036432+0000","last_change":"2024-04-25T10:28:37.043363+0000","last_active":"2024-04-25T10:28:38.036432+0000","last_peered":"2024-04-25T10:28:38.036432+0000","last_clean":"2024-04-25T10:28:38.036432+0000","last_became_active":"2024-04-25T10:28:37.043021+0000","last_became_peered":"2024-04-25T10:28:37.043021+0000","last_unstale":"2024-04-25T10:28:38.036432+0000","last_undegraded":"2024-04-25T10:28:38.036432+0000","last_fullsized":"2024-04-25T10:28:38.036432+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:32.005095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056469+0000","last_change":"2024-04-25T10:28:37.054867+0000","last_active":"2024-04-25T10:28:37.056469+0000","last_peered":"2024-04-25T10:28:37.056469+0000","last_clean":"2024-04-25T10:28:37.056469+0000","last_became_active":"2024-04-25T10:28:37.054181+0000","last_became_peered":"2024-04-25T10:28:37.054181+0000","last_unstale":"2024-04-25T10:28:37.056469+0000","last_undegraded":"2024-04-25T10:28:37.056469+0000","last_fullsized":"2024-04-25T10:28:37.056469+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:44.572625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036496+0000","last_change":"2024-04-25T10:28:37.055355+0000","last_active":"2024-04-25T10:28:38.036496+0000","last_peered":"2024-04-25T10:28:38.036496+0000","last_clean":"2024-04-25T10:28:38.036496+0000","last_became_active":"2024-04-25T10:28:37.055077+0000","last_became_peered":"2024-04-25T10:28:37.055077+0000","last_unstale":"2024-04-25T10:28:38.036496+0000","last_undegraded":"2024-04-25T10:28:38.036496+0000","last_fullsized":"2024-04-25T10:28:38.036496+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:16.231448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272510+0000","last_change":"2024-04-25T10:28:37.061622+0000","last_active":"2024-04-25T10:28:38.272510+0000","last_peered":"2024-04-25T10:28:38.272510+0000","last_clean":"2024-04-25T10:28:38.272510+0000","last_became_active":"2024-04-25T10:28:37.061346+0000","last_became_peered":"2024-04-25T10:28:37.061346+0000","last_unstale":"2024-04-25T10:28:38.272510+0000","last_undegraded":"2024-04-25T10:28:38.272510+0000","last_fullsized":"2024-04-25T10:28:38.272510+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:07.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029688+0000","last_change":"2024-04-25T10:28:37.050801+0000","last_active":"2024-04-25T10:28:38.029688+0000","last_peered":"2024-04-25T10:28:38.029688+0000","last_clean":"2024-04-25T10:28:38.029688+0000","last_became_active":"2024-04-25T10:28:37.050339+0000","last_became_peered":"2024-04-25T10:28:37.050339+0000","last_unstale":"2024-04-25T10:28:38.029688+0000","last_undegraded":"2024-04-25T10:28:38.029688+0000","last_fullsized":"2024-04-25T10:28:38.029688+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:41.816347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272359+0000","last_change":"2024-04-25T10:28:37.059553+0000","last_active":"2024-04-25T10:28:38.272359+0000","last_peered":"2024-04-25T10:28:38.272359+0000","last_clean":"2024-04-25T10:28:38.272359+0000","last_became_active":"2024-04-25T10:28:37.059237+0000","last_became_peered":"2024-04-25T10:28:37.059237+0000","last_unstale":"2024-04-25T10:28:38.272359+0000","last_undegraded":"2024-04-25T10:28:38.272359+0000","last_fullsized":"2024-04-25T10:28:38.272359+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:42.356192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029393+0000","last_change":"2024-04-25T10:28:37.046334+0000","last_active":"2024-04-25T10:28:38.029393+0000","last_peered":"2024-04-25T10:28:38.029393+0000","last_clean":"2024-04-25T10:28:38.029393+0000","last_became_active":"2024-04-25T10:28:37.046048+0000","last_became_peered":"2024-04-25T10:28:37.046048+0000","last_unstale":"2024-04-25T10:28:38.029393+0000","last_undegraded":"2024-04-25T10:28:38.029393+0000","last_fullsized":"2024-04-25T10:28:38.029393+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:27.529759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.037653+0000","last_change":"2024-04-25T10:28:37.040302+0000","last_active":"2024-04-25T10:28:38.037653+0000","last_peered":"2024-04-25T10:28:38.037653+0000","last_clean":"2024-04-25T10:28:38.037653+0000","last_became_active":"2024-04-25T10:28:37.040009+0000","last_became_peered":"2024-04-25T10:28:37.040009+0000","last_unstale":"2024-04-25T10:28:38.037653+0000","last_undegraded":"2024-04-25T10:28:38.037653+0000","last_fullsized":"2024-04-25T10:28:38.037653+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:32.005095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050218+0000","last_change":"2024-04-25T10:28:37.046488+0000","last_active":"2024-04-25T10:28:37.050218+0000","last_peered":"2024-04-25T10:28:37.050218+0000","last_clean":"2024-04-25T10:28:37.050218+0000","last_became_active":"2024-04-25T10:28:37.046262+0000","last_became_peered":"2024-04-25T10:28:37.046262+0000","last_unstale":"2024-04-25T10:28:37.050218+0000","last_undegraded":"2024-04-25T10:28:37.050218+0000","last_fullsized":"2024-04-25T10:28:37.050218+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:31.531043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029689+0000","last_change":"2024-04-25T10:28:37.050641+0000","last_active":"2024-04-25T10:28:38.029689+0000","last_peered":"2024-04-25T10:28:38.029689+0000","last_clean":"2024-04-25T10:28:38.029689+0000","last_became_active":"2024-04-25T10:28:37.046820+0000","last_became_peered":"2024-04-25T10:28:37.046820+0000","last_unstale":"2024-04-25T10:28:38.029689+0000","last_undegraded":"2024-04-25T10:28:38.029689+0000","last_fullsized":"2024-04-25T10:28:38.029689+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:01.330458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.045365+0000","last_change":"2024-04-25T10:28:37.039223+0000","last_active":"2024-04-25T10:28:37.045365+0000","last_peered":"2024-04-25T10:28:37.045365+0000","last_clean":"2024-04-25T10:28:37.045365+0000","last_became_active":"2024-04-25T10:28:37.038899+0000","last_became_peered":"2024-04-25T10:28:37.038899+0000","last_unstale":"2024-04-25T10:28:37.045365+0000","last_undegraded":"2024-04-25T10:28:37.045365+0000","last_fullsized":"2024-04-25T10:28:37.045365+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:50.083262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.043700+0000","last_change":"2024-04-25T10:28:37.039979+0000","last_active":"2024-04-25T10:28:37.043700+0000","last_peered":"2024-04-25T10:28:37.043700+0000","last_clean":"2024-04-25T10:28:37.043700+0000","last_became_active":"2024-04-25T10:28:37.039549+0000","last_became_peered":"2024-04-25T10:28:37.039549+0000","last_unstale":"2024-04-25T10:28:37.043700+0000","last_undegraded":"2024-04-25T10:28:37.043700+0000","last_fullsized":"2024-04-25T10:28:37.043700+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:12.852310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.050367+0000","last_change":"2024-04-25T10:28:37.046716+0000","last_active":"2024-04-25T10:28:37.050367+0000","last_peered":"2024-04-25T10:28:37.050367+0000","last_clean":"2024-04-25T10:28:37.050367+0000","last_became_active":"2024-04-25T10:28:37.044255+0000","last_became_peered":"2024-04-25T10:28:37.044255+0000","last_unstale":"2024-04-25T10:28:37.050367+0000","last_undegraded":"2024-04-25T10:28:37.050367+0000","last_fullsized":"2024-04-25T10:28:37.050367+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:58.158610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272371+0000","last_change":"2024-04-25T10:28:37.053702+0000","last_active":"2024-04-25T10:28:38.272371+0000","last_peered":"2024-04-25T10:28:38.272371+0000","last_clean":"2024-04-25T10:28:38.272371+0000","last_became_active":"2024-04-25T10:28:37.050180+0000","last_became_peered":"2024-04-25T10:28:37.050180+0000","last_unstale":"2024-04-25T10:28:38.272371+0000","last_undegraded":"2024-04-25T10:28:38.272371+0000","last_fullsized":"2024-04-25T10:28:38.272371+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.857540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056831+0000","last_change":"2024-04-25T10:28:37.046320+0000","last_active":"2024-04-25T10:28:37.056831+0000","last_peered":"2024-04-25T10:28:37.056831+0000","last_clean":"2024-04-25T10:28:37.056831+0000","last_became_active":"2024-04-25T10:28:37.045963+0000","last_became_peered":"2024-04-25T10:28:37.045963+0000","last_unstale":"2024-04-25T10:28:37.056831+0000","last_undegraded":"2024-04-25T10:28:37.056831+0000","last_fullsized":"2024-04-25T10:28:37.056831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.427512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029266+0000","last_change":"2024-04-25T10:28:37.061184+0000","last_active":"2024-04-25T10:28:38.029266+0000","last_peered":"2024-04-25T10:28:38.029266+0000","last_clean":"2024-04-25T10:28:38.029266+0000","last_became_active":"2024-04-25T10:28:37.060868+0000","last_became_peered":"2024-04-25T10:28:37.060868+0000","last_unstale":"2024-04-25T10:28:38.029266+0000","last_undegraded":"2024-04-25T10:28:38.029266+0000","last_fullsized":"2024-04-25T10:28:38.029266+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:17.594880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.043845+0000","last_change":"2024-04-25T10:28:37.042825+0000","last_active":"2024-04-25T10:28:37.043845+0000","last_peered":"2024-04-25T10:28:37.043845+0000","last_clean":"2024-04-25T10:28:37.043845+0000","last_became_active":"2024-04-25T10:28:37.042498+0000","last_became_peered":"2024-04-25T10:28:37.042498+0000","last_unstale":"2024-04-25T10:28:37.043845+0000","last_undegraded":"2024-04-25T10:28:37.043845+0000","last_fullsized":"2024-04-25T10:28:37.043845+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.768516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.035564+0000","last_change":"2024-04-25T10:28:37.034896+0000","last_active":"2024-04-25T10:28:37.035564+0000","last_peered":"2024-04-25T10:28:37.035564+0000","last_clean":"2024-04-25T10:28:37.035564+0000","last_became_active":"2024-04-25T10:28:37.034604+0000","last_became_peered":"2024-04-25T10:28:37.034604+0000","last_unstale":"2024-04-25T10:28:37.035564+0000","last_undegraded":"2024-04-25T10:28:37.035564+0000","last_fullsized":"2024-04-25T10:28:37.035564+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.085548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.055620+0000","last_change":"2024-04-25T10:28:37.047140+0000","last_active":"2024-04-25T10:28:37.055620+0000","last_peered":"2024-04-25T10:28:37.055620+0000","last_clean":"2024-04-25T10:28:37.055620+0000","last_became_active":"2024-04-25T10:28:37.046776+0000","last_became_peered":"2024-04-25T10:28:37.046776+0000","last_unstale":"2024-04-25T10:28:37.055620+0000","last_undegraded":"2024-04-25T10:28:37.055620+0000","last_fullsized":"2024-04-25T10:28:37.055620+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:17.528629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.048285+0000","last_change":"2024-04-25T10:28:37.045655+0000","last_active":"2024-04-25T10:28:37.048285+0000","last_peered":"2024-04-25T10:28:37.048285+0000","last_clean":"2024-04-25T10:28:37.048285+0000","last_became_active":"2024-04-25T10:28:37.044237+0000","last_became_peered":"2024-04-25T10:28:37.044237+0000","last_unstale":"2024-04-25T10:28:37.048285+0000","last_undegraded":"2024-04-25T10:28:37.048285+0000","last_fullsized":"2024-04-25T10:28:37.048285+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:05.163084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.055489+0000","last_change":"2024-04-25T10:28:37.053506+0000","last_active":"2024-04-25T10:28:37.055489+0000","last_peered":"2024-04-25T10:28:37.055489+0000","last_clean":"2024-04-25T10:28:37.055489+0000","last_became_active":"2024-04-25T10:28:37.053165+0000","last_became_peered":"2024-04-25T10:28:37.053165+0000","last_unstale":"2024-04-25T10:28:37.055489+0000","last_undegraded":"2024-04-25T10:28:37.055489+0000","last_fullsized":"2024-04-25T10:28:37.055489+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:40.446367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272541+0000","last_change":"2024-04-25T10:28:37.051038+0000","last_active":"2024-04-25T10:28:38.272541+0000","last_peered":"2024-04-25T10:28:38.272541+0000","last_clean":"2024-04-25T10:28:38.272541+0000","last_became_active":"2024-04-25T10:28:37.049843+0000","last_became_peered":"2024-04-25T10:28:37.049843+0000","last_unstale":"2024-04-25T10:28:38.272541+0000","last_undegraded":"2024-04-25T10:28:38.272541+0000","last_fullsized":"2024-04-25T10:28:38.272541+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:11.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057866+0000","last_change":"2024-04-25T10:28:37.055942+0000","last_active":"2024-04-25T10:28:37.057866+0000","last_peered":"2024-04-25T10:28:37.057866+0000","last_clean":"2024-04-25T10:28:37.057866+0000","last_became_active":"2024-04-25T10:28:37.055262+0000","last_became_peered":"2024-04-25T10:28:37.055262+0000","last_unstale":"2024-04-25T10:28:37.057866+0000","last_undegraded":"2024-04-25T10:28:37.057866+0000","last_fullsized":"2024-04-25T10:28:37.057866+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:47.485924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065519+0000","last_change":"2024-04-25T10:28:37.065019+0000","last_active":"2024-04-25T10:28:37.065519+0000","last_peered":"2024-04-25T10:28:37.065519+0000","last_clean":"2024-04-25T10:28:37.065519+0000","last_became_active":"2024-04-25T10:28:37.064774+0000","last_became_peered":"2024-04-25T10:28:37.064774+0000","last_unstale":"2024-04-25T10:28:37.065519+0000","last_undegraded":"2024-04-25T10:28:37.065519+0000","last_fullsized":"2024-04-25T10:28:37.065519+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:20.901336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029765+0000","last_change":"2024-04-25T10:28:37.050342+0000","last_active":"2024-04-25T10:28:38.029765+0000","last_peered":"2024-04-25T10:28:38.029765+0000","last_clean":"2024-04-25T10:28:38.029765+0000","last_became_active":"2024-04-25T10:28:37.046502+0000","last_became_peered":"2024-04-25T10:28:37.046502+0000","last_unstale":"2024-04-25T10:28:38.029765+0000","last_undegraded":"2024-04-25T10:28:38.029765+0000","last_fullsized":"2024-04-25T10:28:38.029765+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:09.706452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"140'4","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.058914+0000","last_change":"2024-04-25T10:28:36.034838+0000","last_active":"2024-04-25T10:28:37.058914+0000","last_peered":"2024-04-25T10:28:37.058914+0000","last_clean":"2024-04-25T10:28:37.058914+0000","last_became_active":"2024-04-25T10:28:36.034548+0000","last_became_peered":"2024-04-25T10:28:36.034548+0000","last_unstale":"2024-04-25T10:28:37.058914+0000","last_undegraded":"2024-04-25T10:28:37.058914+0000","last_fullsized":"2024-04-25T10:28:37.058914+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:12.769209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038266+0000","last_change":"2024-04-25T10:28:37.032722+0000","last_active":"2024-04-25T10:28:38.038266+0000","last_peered":"2024-04-25T10:28:38.038266+0000","last_clean":"2024-04-25T10:28:38.038266+0000","last_became_active":"2024-04-25T10:28:37.032406+0000","last_became_peered":"2024-04-25T10:28:37.032406+0000","last_unstale":"2024-04-25T10:28:38.038266+0000","last_undegraded":"2024-04-25T10:28:38.038266+0000","last_fullsized":"2024-04-25T10:28:38.038266+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:16.896821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036461+0000","last_change":"2024-04-25T10:28:36.047885+0000","last_active":"2024-04-25T10:28:38.036461+0000","last_peered":"2024-04-25T10:28:38.036461+0000","last_clean":"2024-04-25T10:28:38.036461+0000","last_became_active":"2024-04-25T10:28:36.047545+0000","last_became_peered":"2024-04-25T10:28:36.047545+0000","last_unstale":"2024-04-25T10:28:38.036461+0000","last_undegraded":"2024-04-25T10:28:38.036461+0000","last_fullsized":"2024-04-25T10:28:38.036461+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:25.424854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036325+0000","last_change":"2024-04-25T10:28:37.057740+0000","last_active":"2024-04-25T10:28:38.036325+0000","last_peered":"2024-04-25T10:28:38.036325+0000","last_clean":"2024-04-25T10:28:38.036325+0000","last_became_active":"2024-04-25T10:28:37.057406+0000","last_became_peered":"2024-04-25T10:28:37.057406+0000","last_unstale":"2024-04-25T10:28:38.036325+0000","last_undegraded":"2024-04-25T10:28:38.036325+0000","last_fullsized":"2024-04-25T10:28:38.036325+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:09.796346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.062065+0000","last_change":"2024-04-25T10:28:36.041843+0000","last_active":"2024-04-25T10:28:37.062065+0000","last_peered":"2024-04-25T10:28:37.062065+0000","last_clean":"2024-04-25T10:28:37.062065+0000","last_became_active":"2024-04-25T10:28:36.041561+0000","last_became_peered":"2024-04-25T10:28:36.041561+0000","last_unstale":"2024-04-25T10:28:37.062065+0000","last_undegraded":"2024-04-25T10:28:37.062065+0000","last_fullsized":"2024-04-25T10:28:37.062065+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:58.499656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065609+0000","last_change":"2024-04-25T10:28:37.062772+0000","last_active":"2024-04-25T10:28:37.065609+0000","last_peered":"2024-04-25T10:28:37.065609+0000","last_clean":"2024-04-25T10:28:37.065609+0000","last_became_active":"2024-04-25T10:28:37.061646+0000","last_became_peered":"2024-04-25T10:28:37.061646+0000","last_unstale":"2024-04-25T10:28:37.065609+0000","last_undegraded":"2024-04-25T10:28:37.065609+0000","last_fullsized":"2024-04-25T10:28:37.065609+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:53.028725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.060751+0000","last_change":"2024-04-25T10:28:36.043640+0000","last_active":"2024-04-25T10:28:37.060751+0000","last_peered":"2024-04-25T10:28:37.060751+0000","last_clean":"2024-04-25T10:28:37.060751+0000","last_became_active":"2024-04-25T10:28:36.043297+0000","last_became_peered":"2024-04-25T10:28:36.043297+0000","last_unstale":"2024-04-25T10:28:37.060751+0000","last_undegraded":"2024-04-25T10:28:37.060751+0000","last_fullsized":"2024-04-25T10:28:37.060751+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:47.674855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.048278+0000","last_change":"2024-04-25T10:28:37.046033+0000","last_active":"2024-04-25T10:28:37.048278+0000","last_peered":"2024-04-25T10:28:37.048278+0000","last_clean":"2024-04-25T10:28:37.048278+0000","last_became_active":"2024-04-25T10:28:37.045057+0000","last_became_peered":"2024-04-25T10:28:37.045057+0000","last_unstale":"2024-04-25T10:28:37.048278+0000","last_undegraded":"2024-04-25T10:28:37.048278+0000","last_fullsized":"2024-04-25T10:28:37.048278+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:57.664243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"140'1","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059925+0000","last_change":"2024-04-25T10:28:36.039014+0000","last_active":"2024-04-25T10:28:37.059925+0000","last_peered":"2024-04-25T10:28:37.059925+0000","last_clean":"2024-04-25T10:28:37.059925+0000","last_became_active":"2024-04-25T10:28:36.038323+0000","last_became_peered":"2024-04-25T10:28:36.038323+0000","last_unstale":"2024-04-25T10:28:37.059925+0000","last_undegraded":"2024-04-25T10:28:37.059925+0000","last_fullsized":"2024-04-25T10:28:37.059925+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:26.509117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056931+0000","last_change":"2024-04-25T10:28:37.049501+0000","last_active":"2024-04-25T10:28:37.056931+0000","last_peered":"2024-04-25T10:28:37.056931+0000","last_clean":"2024-04-25T10:28:37.056931+0000","last_became_active":"2024-04-25T10:28:37.048837+0000","last_became_peered":"2024-04-25T10:28:37.048837+0000","last_unstale":"2024-04-25T10:28:37.056931+0000","last_undegraded":"2024-04-25T10:28:37.056931+0000","last_fullsized":"2024-04-25T10:28:37.056931+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:18.857540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029419+0000","last_change":"2024-04-25T10:28:36.037922+0000","last_active":"2024-04-25T10:28:38.029419+0000","last_peered":"2024-04-25T10:28:38.029419+0000","last_clean":"2024-04-25T10:28:38.029419+0000","last_became_active":"2024-04-25T10:28:36.036888+0000","last_became_peered":"2024-04-25T10:28:36.036888+0000","last_unstale":"2024-04-25T10:28:38.029419+0000","last_undegraded":"2024-04-25T10:28:38.029419+0000","last_fullsized":"2024-04-25T10:28:38.029419+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:44.462729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.063306+0000","last_change":"2024-04-25T10:28:37.062761+0000","last_active":"2024-04-25T10:28:37.063306+0000","last_peered":"2024-04-25T10:28:37.063306+0000","last_clean":"2024-04-25T10:28:37.063306+0000","last_became_active":"2024-04-25T10:28:37.062314+0000","last_became_peered":"2024-04-25T10:28:37.062314+0000","last_unstale":"2024-04-25T10:28:37.063306+0000","last_undegraded":"2024-04-25T10:28:37.063306+0000","last_fullsized":"2024-04-25T10:28:37.063306+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:18.102053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.035949+0000","last_change":"2024-04-25T10:28:36.043430+0000","last_active":"2024-04-25T10:28:38.035949+0000","last_peered":"2024-04-25T10:28:38.035949+0000","last_clean":"2024-04-25T10:28:38.035949+0000","last_became_active":"2024-04-25T10:28:36.043159+0000","last_became_peered":"2024-04-25T10:28:36.043159+0000","last_unstale":"2024-04-25T10:28:38.035949+0000","last_undegraded":"2024-04-25T10:28:38.035949+0000","last_fullsized":"2024-04-25T10:28:38.035949+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:35.546777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.046181+0000","last_change":"2024-04-25T10:28:37.042364+0000","last_active":"2024-04-25T10:28:37.046181+0000","last_peered":"2024-04-25T10:28:37.046181+0000","last_clean":"2024-04-25T10:28:37.046181+0000","last_became_active":"2024-04-25T10:28:37.041860+0000","last_became_peered":"2024-04-25T10:28:37.041860+0000","last_unstale":"2024-04-25T10:28:37.046181+0000","last_undegraded":"2024-04-25T10:28:37.046181+0000","last_fullsized":"2024-04-25T10:28:37.046181+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:38.249660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.025120+0000","last_change":"2024-04-25T10:28:36.042822+0000","last_active":"2024-04-25T10:28:37.025120+0000","last_peered":"2024-04-25T10:28:37.025120+0000","last_clean":"2024-04-25T10:28:37.025120+0000","last_became_active":"2024-04-25T10:28:36.041356+0000","last_became_peered":"2024-04-25T10:28:36.041356+0000","last_unstale":"2024-04-25T10:28:37.025120+0000","last_undegraded":"2024-04-25T10:28:37.025120+0000","last_fullsized":"2024-04-25T10:28:37.025120+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.041869+0000","last_change":"2024-04-25T10:28:37.040166+0000","last_active":"2024-04-25T10:28:37.041869+0000","last_peered":"2024-04-25T10:28:37.041869+0000","last_clean":"2024-04-25T10:28:37.041869+0000","last_became_active":"2024-04-25T10:28:37.039898+0000","last_became_peered":"2024-04-25T10:28:37.039898+0000","last_unstale":"2024-04-25T10:28:37.041869+0000","last_undegraded":"2024-04-25T10:28:37.041869+0000","last_fullsized":"2024-04-25T10:28:37.041869+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:34.289939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036473+0000","last_change":"2024-04-25T10:28:36.048304+0000","last_active":"2024-04-25T10:28:38.036473+0000","last_peered":"2024-04-25T10:28:38.036473+0000","last_clean":"2024-04-25T10:28:38.036473+0000","last_became_active":"2024-04-25T10:28:36.046336+0000","last_became_peered":"2024-04-25T10:28:36.046336+0000","last_unstale":"2024-04-25T10:28:38.036473+0000","last_undegraded":"2024-04-25T10:28:38.036473+0000","last_fullsized":"2024-04-25T10:28:38.036473+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:41.353534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272606+0000","last_change":"2024-04-25T10:28:37.060509+0000","last_active":"2024-04-25T10:28:38.272606+0000","last_peered":"2024-04-25T10:28:38.272606+0000","last_clean":"2024-04-25T10:28:38.272606+0000","last_became_active":"2024-04-25T10:28:37.060221+0000","last_became_peered":"2024-04-25T10:28:37.060221+0000","last_unstale":"2024-04-25T10:28:38.272606+0000","last_undegraded":"2024-04-25T10:28:38.272606+0000","last_fullsized":"2024-04-25T10:28:38.272606+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:47.973064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.030026+0000","last_change":"2024-04-25T10:28:36.040280+0000","last_active":"2024-04-25T10:28:38.030026+0000","last_peered":"2024-04-25T10:28:38.030026+0000","last_clean":"2024-04-25T10:28:38.030026+0000","last_became_active":"2024-04-25T10:28:36.039414+0000","last_became_peered":"2024-04-25T10:28:36.039414+0000","last_unstale":"2024-04-25T10:28:38.030026+0000","last_undegraded":"2024-04-25T10:28:38.030026+0000","last_fullsized":"2024-04-25T10:28:38.030026+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:19.226855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038363+0000","last_change":"2024-04-25T10:28:37.036469+0000","last_active":"2024-04-25T10:28:38.038363+0000","last_peered":"2024-04-25T10:28:38.038363+0000","last_clean":"2024-04-25T10:28:38.038363+0000","last_became_active":"2024-04-25T10:28:37.036182+0000","last_became_peered":"2024-04-25T10:28:37.036182+0000","last_unstale":"2024-04-25T10:28:38.038363+0000","last_undegraded":"2024-04-25T10:28:38.038363+0000","last_fullsized":"2024-04-25T10:28:38.038363+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:05.155103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"140'6","reported_seq":20,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038038+0000","last_change":"2024-04-25T10:28:36.046708+0000","last_active":"2024-04-25T10:28:38.038038+0000","last_peered":"2024-04-25T10:28:38.038038+0000","last_clean":"2024-04-25T10:28:38.038038+0000","last_became_active":"2024-04-25T10:28:36.044743+0000","last_became_peered":"2024-04-25T10:28:36.044743+0000","last_unstale":"2024-04-25T10:28:38.038038+0000","last_undegraded":"2024-04-25T10:28:38.038038+0000","last_fullsized":"2024-04-25T10:28:38.038038+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.158020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057049+0000","last_change":"2024-04-25T10:28:37.047525+0000","last_active":"2024-04-25T10:28:37.057049+0000","last_peered":"2024-04-25T10:28:37.057049+0000","last_clean":"2024-04-25T10:28:37.057049+0000","last_became_active":"2024-04-25T10:28:37.046252+0000","last_became_peered":"2024-04-25T10:28:37.046252+0000","last_unstale":"2024-04-25T10:28:37.057049+0000","last_undegraded":"2024-04-25T10:28:37.057049+0000","last_fullsized":"2024-04-25T10:28:37.057049+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:42.356192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.063641+0000","last_change":"2024-04-25T10:28:36.042988+0000","last_active":"2024-04-25T10:28:37.063641+0000","last_peered":"2024-04-25T10:28:37.063641+0000","last_clean":"2024-04-25T10:28:37.063641+0000","last_became_active":"2024-04-25T10:28:36.042673+0000","last_became_peered":"2024-04-25T10:28:36.042673+0000","last_unstale":"2024-04-25T10:28:37.063641+0000","last_undegraded":"2024-04-25T10:28:37.063641+0000","last_fullsized":"2024-04-25T10:28:37.063641+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:24.905838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056171+0000","last_change":"2024-04-25T10:28:37.053506+0000","last_active":"2024-04-25T10:28:37.056171+0000","last_peered":"2024-04-25T10:28:37.056171+0000","last_clean":"2024-04-25T10:28:37.056171+0000","last_became_active":"2024-04-25T10:28:37.053247+0000","last_became_peered":"2024-04-25T10:28:37.053247+0000","last_unstale":"2024-04-25T10:28:37.056171+0000","last_undegraded":"2024-04-25T10:28:37.056171+0000","last_fullsized":"2024-04-25T10:28:37.056171+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:32.767324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272645+0000","last_change":"2024-04-25T10:28:36.047901+0000","last_active":"2024-04-25T10:28:38.272645+0000","last_peered":"2024-04-25T10:28:38.272645+0000","last_clean":"2024-04-25T10:28:38.272645+0000","last_became_active":"2024-04-25T10:28:36.046402+0000","last_became_peered":"2024-04-25T10:28:36.046402+0000","last_unstale":"2024-04-25T10:28:38.272645+0000","last_undegraded":"2024-04-25T10:28:38.272645+0000","last_fullsized":"2024-04-25T10:28:38.272645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:34.613987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065572+0000","last_change":"2024-04-25T10:28:37.061094+0000","last_active":"2024-04-25T10:28:37.065572+0000","last_peered":"2024-04-25T10:28:37.065572+0000","last_clean":"2024-04-25T10:28:37.065572+0000","last_became_active":"2024-04-25T10:28:37.059350+0000","last_became_peered":"2024-04-25T10:28:37.059350+0000","last_unstale":"2024-04-25T10:28:37.065572+0000","last_undegraded":"2024-04-25T10:28:37.065572+0000","last_fullsized":"2024-04-25T10:28:37.065572+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:47.973064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.034637+0000","last_change":"2024-04-25T10:28:36.049124+0000","last_active":"2024-04-25T10:28:37.034637+0000","last_peered":"2024-04-25T10:28:37.034637+0000","last_clean":"2024-04-25T10:28:37.034637+0000","last_became_active":"2024-04-25T10:28:36.048853+0000","last_became_peered":"2024-04-25T10:28:36.048853+0000","last_unstale":"2024-04-25T10:28:37.034637+0000","last_undegraded":"2024-04-25T10:28:37.034637+0000","last_fullsized":"2024-04-25T10:28:37.034637+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:04.292402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.046160+0000","last_change":"2024-04-25T10:28:37.040887+0000","last_active":"2024-04-25T10:28:37.046160+0000","last_peered":"2024-04-25T10:28:37.046160+0000","last_clean":"2024-04-25T10:28:37.046160+0000","last_became_active":"2024-04-25T10:28:37.040594+0000","last_became_peered":"2024-04-25T10:28:37.040594+0000","last_unstale":"2024-04-25T10:28:37.046160+0000","last_undegraded":"2024-04-25T10:28:37.046160+0000","last_fullsized":"2024-04-25T10:28:37.046160+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:09.586495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"140'8","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.068039+0000","last_change":"2024-04-25T10:28:36.046419+0000","last_active":"2024-04-25T10:28:37.068039+0000","last_peered":"2024-04-25T10:28:37.068039+0000","last_clean":"2024-04-25T10:28:37.068039+0000","last_became_active":"2024-04-25T10:28:36.042259+0000","last_became_peered":"2024-04-25T10:28:36.042259+0000","last_unstale":"2024-04-25T10:28:37.068039+0000","last_undegraded":"2024-04-25T10:28:37.068039+0000","last_fullsized":"2024-04-25T10:28:37.068039+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.792545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.030021+0000","last_change":"2024-04-25T10:28:37.062915+0000","last_active":"2024-04-25T10:28:38.030021+0000","last_peered":"2024-04-25T10:28:38.030021+0000","last_clean":"2024-04-25T10:28:38.030021+0000","last_became_active":"2024-04-25T10:28:37.062515+0000","last_became_peered":"2024-04-25T10:28:37.062515+0000","last_unstale":"2024-04-25T10:28:38.030021+0000","last_undegraded":"2024-04-25T10:28:38.030021+0000","last_fullsized":"2024-04-25T10:28:38.030021+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:16.768516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"140'1","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059918+0000","last_change":"2024-04-25T10:28:36.042594+0000","last_active":"2024-04-25T10:28:37.059918+0000","last_peered":"2024-04-25T10:28:37.059918+0000","last_clean":"2024-04-25T10:28:37.059918+0000","last_became_active":"2024-04-25T10:28:36.042093+0000","last_became_peered":"2024-04-25T10:28:36.042093+0000","last_unstale":"2024-04-25T10:28:37.059918+0000","last_undegraded":"2024-04-25T10:28:37.059918+0000","last_fullsized":"2024-04-25T10:28:37.059918+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:00.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057084+0000","last_change":"2024-04-25T10:28:37.054548+0000","last_active":"2024-04-25T10:28:37.057084+0000","last_peered":"2024-04-25T10:28:37.057084+0000","last_clean":"2024-04-25T10:28:37.057084+0000","last_became_active":"2024-04-25T10:28:37.054258+0000","last_became_peered":"2024-04-25T10:28:37.054258+0000","last_unstale":"2024-04-25T10:28:37.057084+0000","last_undegraded":"2024-04-25T10:28:37.057084+0000","last_fullsized":"2024-04-25T10:28:37.057084+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:57.832892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.059060+0000","last_change":"2024-04-25T10:28:36.041736+0000","last_active":"2024-04-25T10:28:37.059060+0000","last_peered":"2024-04-25T10:28:37.059060+0000","last_clean":"2024-04-25T10:28:37.059060+0000","last_became_active":"2024-04-25T10:28:36.041309+0000","last_became_peered":"2024-04-25T10:28:36.041309+0000","last_unstale":"2024-04-25T10:28:37.059060+0000","last_undegraded":"2024-04-25T10:28:37.059060+0000","last_fullsized":"2024-04-25T10:28:37.059060+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:20.692046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029674+0000","last_change":"2024-04-25T10:28:37.063585+0000","last_active":"2024-04-25T10:28:38.029674+0000","last_peered":"2024-04-25T10:28:38.029674+0000","last_clean":"2024-04-25T10:28:38.029674+0000","last_became_active":"2024-04-25T10:28:37.063326+0000","last_became_peered":"2024-04-25T10:28:37.063326+0000","last_unstale":"2024-04-25T10:28:38.029674+0000","last_undegraded":"2024-04-25T10:28:38.029674+0000","last_fullsized":"2024-04-25T10:28:38.029674+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:05.295060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"140'1","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038363+0000","last_change":"2024-04-25T10:28:36.047972+0000","last_active":"2024-04-25T10:28:38.038363+0000","last_peered":"2024-04-25T10:28:38.038363+0000","last_clean":"2024-04-25T10:28:38.038363+0000","last_became_active":"2024-04-25T10:28:36.046813+0000","last_became_peered":"2024-04-25T10:28:36.046813+0000","last_unstale":"2024-04-25T10:28:38.038363+0000","last_undegraded":"2024-04-25T10:28:38.038363+0000","last_fullsized":"2024-04-25T10:28:38.038363+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.854882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036610+0000","last_change":"2024-04-25T10:28:37.053645+0000","last_active":"2024-04-25T10:28:38.036610+0000","last_peered":"2024-04-25T10:28:38.036610+0000","last_clean":"2024-04-25T10:28:38.036610+0000","last_became_active":"2024-04-25T10:28:37.053377+0000","last_became_peered":"2024-04-25T10:28:37.053377+0000","last_unstale":"2024-04-25T10:28:38.036610+0000","last_undegraded":"2024-04-25T10:28:38.036610+0000","last_fullsized":"2024-04-25T10:28:38.036610+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.888830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.036259+0000","last_change":"2024-04-25T10:28:36.048436+0000","last_active":"2024-04-25T10:28:37.036259+0000","last_peered":"2024-04-25T10:28:37.036259+0000","last_clean":"2024-04-25T10:28:37.036259+0000","last_became_active":"2024-04-25T10:28:36.048167+0000","last_became_peered":"2024-04-25T10:28:36.048167+0000","last_unstale":"2024-04-25T10:28:37.036259+0000","last_undegraded":"2024-04-25T10:28:37.036259+0000","last_fullsized":"2024-04-25T10:28:37.036259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:00.327800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272539+0000","last_change":"2024-04-25T10:28:37.051773+0000","last_active":"2024-04-25T10:28:38.272539+0000","last_peered":"2024-04-25T10:28:38.272539+0000","last_clean":"2024-04-25T10:28:38.272539+0000","last_became_active":"2024-04-25T10:28:37.051475+0000","last_became_peered":"2024-04-25T10:28:37.051475+0000","last_unstale":"2024-04-25T10:28:38.272539+0000","last_undegraded":"2024-04-25T10:28:38.272539+0000","last_fullsized":"2024-04-25T10:28:38.272539+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:40.512214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"140'2","reported_seq":16,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.036211+0000","last_change":"2024-04-25T10:28:36.044071+0000","last_active":"2024-04-25T10:28:38.036211+0000","last_peered":"2024-04-25T10:28:38.036211+0000","last_clean":"2024-04-25T10:28:38.036211+0000","last_became_active":"2024-04-25T10:28:36.042998+0000","last_became_peered":"2024-04-25T10:28:36.042998+0000","last_unstale":"2024-04-25T10:28:38.036211+0000","last_undegraded":"2024-04-25T10:28:38.036211+0000","last_fullsized":"2024-04-25T10:28:38.036211+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.158020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057083+0000","last_change":"2024-04-25T10:28:37.054529+0000","last_active":"2024-04-25T10:28:37.057083+0000","last_peered":"2024-04-25T10:28:37.057083+0000","last_clean":"2024-04-25T10:28:37.057083+0000","last_became_active":"2024-04-25T10:28:37.054177+0000","last_became_peered":"2024-04-25T10:28:37.054177+0000","last_unstale":"2024-04-25T10:28:37.057083+0000","last_undegraded":"2024-04-25T10:28:37.057083+0000","last_fullsized":"2024-04-25T10:28:37.057083+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:00.252490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.040011+0000","last_change":"2024-04-25T10:28:36.041403+0000","last_active":"2024-04-25T10:28:37.040011+0000","last_peered":"2024-04-25T10:28:37.040011+0000","last_clean":"2024-04-25T10:28:37.040011+0000","last_became_active":"2024-04-25T10:28:36.039220+0000","last_became_peered":"2024-04-25T10:28:36.039220+0000","last_unstale":"2024-04-25T10:28:37.040011+0000","last_undegraded":"2024-04-25T10:28:37.040011+0000","last_fullsized":"2024-04-25T10:28:37.040011+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:42.602655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272360+0000","last_change":"2024-04-25T10:28:37.054100+0000","last_active":"2024-04-25T10:28:38.272360+0000","last_peered":"2024-04-25T10:28:38.272360+0000","last_clean":"2024-04-25T10:28:38.272360+0000","last_became_active":"2024-04-25T10:28:37.053789+0000","last_became_peered":"2024-04-25T10:28:37.053789+0000","last_unstale":"2024-04-25T10:28:38.272360+0000","last_undegraded":"2024-04-25T10:28:38.272360+0000","last_fullsized":"2024-04-25T10:28:38.272360+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:26.427512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.038061+0000","last_change":"2024-04-25T10:28:36.042221+0000","last_active":"2024-04-25T10:28:37.038061+0000","last_peered":"2024-04-25T10:28:37.038061+0000","last_clean":"2024-04-25T10:28:37.038061+0000","last_became_active":"2024-04-25T10:28:36.041822+0000","last_became_peered":"2024-04-25T10:28:36.041822+0000","last_unstale":"2024-04-25T10:28:37.038061+0000","last_undegraded":"2024-04-25T10:28:37.038061+0000","last_fullsized":"2024-04-25T10:28:37.038061+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:34.613987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057846+0000","last_change":"2024-04-25T10:28:37.055719+0000","last_active":"2024-04-25T10:28:37.057846+0000","last_peered":"2024-04-25T10:28:37.057846+0000","last_clean":"2024-04-25T10:28:37.057846+0000","last_became_active":"2024-04-25T10:28:37.055296+0000","last_became_peered":"2024-04-25T10:28:37.055296+0000","last_unstale":"2024-04-25T10:28:37.057846+0000","last_undegraded":"2024-04-25T10:28:37.057846+0000","last_fullsized":"2024-04-25T10:28:37.057846+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:14.985960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.037784+0000","last_change":"2024-04-25T10:28:36.041123+0000","last_active":"2024-04-25T10:28:37.037784+0000","last_peered":"2024-04-25T10:28:37.037784+0000","last_clean":"2024-04-25T10:28:37.037784+0000","last_became_active":"2024-04-25T10:28:36.040675+0000","last_became_peered":"2024-04-25T10:28:36.040675+0000","last_unstale":"2024-04-25T10:28:37.037784+0000","last_undegraded":"2024-04-25T10:28:37.037784+0000","last_fullsized":"2024-04-25T10:28:37.037784+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:56.444371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.065069+0000","last_change":"2024-04-25T10:28:37.058096+0000","last_active":"2024-04-25T10:28:37.065069+0000","last_peered":"2024-04-25T10:28:37.065069+0000","last_clean":"2024-04-25T10:28:37.065069+0000","last_became_active":"2024-04-25T10:28:37.057793+0000","last_became_peered":"2024-04-25T10:28:37.057793+0000","last_unstale":"2024-04-25T10:28:37.065069+0000","last_undegraded":"2024-04-25T10:28:37.065069+0000","last_fullsized":"2024-04-25T10:28:37.065069+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:07.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"140'2","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.057651+0000","last_change":"2024-04-25T10:28:36.043845+0000","last_active":"2024-04-25T10:28:37.057651+0000","last_peered":"2024-04-25T10:28:37.057651+0000","last_clean":"2024-04-25T10:28:37.057651+0000","last_became_active":"2024-04-25T10:28:36.043563+0000","last_became_peered":"2024-04-25T10:28:36.043563+0000","last_unstale":"2024-04-25T10:28:37.057651+0000","last_undegraded":"2024-04-25T10:28:37.057651+0000","last_fullsized":"2024-04-25T10:28:37.057651+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:35.546777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.064414+0000","last_change":"2024-04-25T10:28:37.058471+0000","last_active":"2024-04-25T10:28:37.064414+0000","last_peered":"2024-04-25T10:28:37.064414+0000","last_clean":"2024-04-25T10:28:37.064414+0000","last_became_active":"2024-04-25T10:28:37.057931+0000","last_became_peered":"2024-04-25T10:28:37.057931+0000","last_unstale":"2024-04-25T10:28:37.064414+0000","last_undegraded":"2024-04-25T10:28:37.064414+0000","last_fullsized":"2024-04-25T10:28:37.064414+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:11.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.038486+0000","last_change":"2024-04-25T10:28:36.048060+0000","last_active":"2024-04-25T10:28:37.038486+0000","last_peered":"2024-04-25T10:28:37.038486+0000","last_clean":"2024-04-25T10:28:37.038486+0000","last_became_active":"2024-04-25T10:28:36.047740+0000","last_became_peered":"2024-04-25T10:28:36.047740+0000","last_unstale":"2024-04-25T10:28:37.038486+0000","last_undegraded":"2024-04-25T10:28:37.038486+0000","last_fullsized":"2024-04-25T10:28:37.038486+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:08.703794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.043676+0000","last_change":"2024-04-25T10:28:37.039857+0000","last_active":"2024-04-25T10:28:37.043676+0000","last_peered":"2024-04-25T10:28:37.043676+0000","last_clean":"2024-04-25T10:28:37.043676+0000","last_became_active":"2024-04-25T10:28:37.039119+0000","last_became_peered":"2024-04-25T10:28:37.039119+0000","last_unstale":"2024-04-25T10:28:37.043676+0000","last_undegraded":"2024-04-25T10:28:37.043676+0000","last_fullsized":"2024-04-25T10:28:37.043676+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:15.425634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038304+0000","last_change":"2024-04-25T10:28:36.040806+0000","last_active":"2024-04-25T10:28:38.038304+0000","last_peered":"2024-04-25T10:28:38.038304+0000","last_clean":"2024-04-25T10:28:38.038304+0000","last_became_active":"2024-04-25T10:28:36.040479+0000","last_became_peered":"2024-04-25T10:28:36.040479+0000","last_unstale":"2024-04-25T10:28:38.038304+0000","last_undegraded":"2024-04-25T10:28:38.038304+0000","last_fullsized":"2024-04-25T10:28:38.038304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:41.353534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.046103+0000","last_change":"2024-04-25T10:28:37.041638+0000","last_active":"2024-04-25T10:28:37.046103+0000","last_peered":"2024-04-25T10:28:37.046103+0000","last_clean":"2024-04-25T10:28:37.046103+0000","last_became_active":"2024-04-25T10:28:37.041328+0000","last_became_peered":"2024-04-25T10:28:37.041328+0000","last_unstale":"2024-04-25T10:28:37.046103+0000","last_undegraded":"2024-04-25T10:28:37.046103+0000","last_fullsized":"2024-04-25T10:28:37.046103+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:09.489406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038170+0000","last_change":"2024-04-25T10:28:36.046917+0000","last_active":"2024-04-25T10:28:38.038170+0000","last_peered":"2024-04-25T10:28:38.038170+0000","last_clean":"2024-04-25T10:28:38.038170+0000","last_became_active":"2024-04-25T10:28:36.046439+0000","last_became_peered":"2024-04-25T10:28:36.046439+0000","last_unstale":"2024-04-25T10:28:38.038170+0000","last_undegraded":"2024-04-25T10:28:38.038170+0000","last_fullsized":"2024-04-25T10:28:38.038170+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:25.424854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.037925+0000","last_change":"2024-04-25T10:28:37.057867+0000","last_active":"2024-04-25T10:28:38.037925+0000","last_peered":"2024-04-25T10:28:38.037925+0000","last_clean":"2024-04-25T10:28:38.037925+0000","last_became_active":"2024-04-25T10:28:37.056874+0000","last_became_peered":"2024-04-25T10:28:37.056874+0000","last_unstale":"2024-04-25T10:28:38.037925+0000","last_undegraded":"2024-04-25T10:28:38.037925+0000","last_fullsized":"2024-04-25T10:28:38.037925+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:16.231448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.037105+0000","last_change":"2024-04-25T10:28:36.049313+0000","last_active":"2024-04-25T10:28:37.037105+0000","last_peered":"2024-04-25T10:28:37.037105+0000","last_clean":"2024-04-25T10:28:37.037105+0000","last_became_active":"2024-04-25T10:28:36.049048+0000","last_became_peered":"2024-04-25T10:28:36.049048+0000","last_unstale":"2024-04-25T10:28:37.037105+0000","last_undegraded":"2024-04-25T10:28:37.037105+0000","last_fullsized":"2024-04-25T10:28:37.037105+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:40.813689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.049651+0000","last_change":"2024-04-25T10:28:37.042538+0000","last_active":"2024-04-25T10:28:37.049651+0000","last_peered":"2024-04-25T10:28:37.049651+0000","last_clean":"2024-04-25T10:28:37.049651+0000","last_became_active":"2024-04-25T10:28:37.042193+0000","last_became_peered":"2024-04-25T10:28:37.042193+0000","last_unstale":"2024-04-25T10:28:37.049651+0000","last_undegraded":"2024-04-25T10:28:37.049651+0000","last_fullsized":"2024-04-25T10:28:37.049651+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:15.875228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.026148+0000","last_change":"2024-04-25T10:28:36.037038+0000","last_active":"2024-04-25T10:28:37.026148+0000","last_peered":"2024-04-25T10:28:37.026148+0000","last_clean":"2024-04-25T10:28:37.026148+0000","last_became_active":"2024-04-25T10:28:36.036667+0000","last_became_peered":"2024-04-25T10:28:36.036667+0000","last_unstale":"2024-04-25T10:28:37.026148+0000","last_undegraded":"2024-04-25T10:28:37.026148+0000","last_fullsized":"2024-04-25T10:28:37.026148+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:57.447189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.056472+0000","last_change":"2024-04-25T10:28:37.045119+0000","last_active":"2024-04-25T10:28:37.056472+0000","last_peered":"2024-04-25T10:28:37.056472+0000","last_clean":"2024-04-25T10:28:37.056472+0000","last_became_active":"2024-04-25T10:28:37.043122+0000","last_became_peered":"2024-04-25T10:28:37.043122+0000","last_unstale":"2024-04-25T10:28:37.056472+0000","last_undegraded":"2024-04-25T10:28:37.056472+0000","last_fullsized":"2024-04-25T10:28:37.056472+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:45.160678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272196+0000","last_change":"2024-04-25T10:28:36.038955+0000","last_active":"2024-04-25T10:28:38.272196+0000","last_peered":"2024-04-25T10:28:38.272196+0000","last_clean":"2024-04-25T10:28:38.272196+0000","last_became_active":"2024-04-25T10:28:36.038499+0000","last_became_peered":"2024-04-25T10:28:36.038499+0000","last_unstale":"2024-04-25T10:28:38.272196+0000","last_undegraded":"2024-04-25T10:28:38.272196+0000","last_fullsized":"2024-04-25T10:28:38.272196+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:56.444371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-25T10:28:37.035638+0000","last_change":"2024-04-25T10:28:37.030408+0000","last_active":"2024-04-25T10:28:37.035638+0000","last_peered":"2024-04-25T10:28:37.035638+0000","last_clean":"2024-04-25T10:28:37.035638+0000","last_became_active":"2024-04-25T10:28:37.030130+0000","last_became_peered":"2024-04-25T10:28:37.030130+0000","last_unstale":"2024-04-25T10:28:37.035638+0000","last_undegraded":"2024-04-25T10:28:37.035638+0000","last_fullsized":"2024-04-25T10:28:37.035638+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:54.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.035786+0000","last_change":"2024-04-25T10:28:36.043816+0000","last_active":"2024-04-25T10:28:38.035786+0000","last_peered":"2024-04-25T10:28:38.035786+0000","last_clean":"2024-04-25T10:28:38.035786+0000","last_became_active":"2024-04-25T10:28:36.043403+0000","last_became_peered":"2024-04-25T10:28:36.043403+0000","last_unstale":"2024-04-25T10:28:38.035786+0000","last_undegraded":"2024-04-25T10:28:38.035786+0000","last_fullsized":"2024-04-25T10:28:38.035786+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:00.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.038050+0000","last_change":"2024-04-25T10:28:37.048328+0000","last_active":"2024-04-25T10:28:38.038050+0000","last_peered":"2024-04-25T10:28:38.038050+0000","last_clean":"2024-04-25T10:28:38.038050+0000","last_became_active":"2024-04-25T10:28:37.047926+0000","last_became_peered":"2024-04-25T10:28:37.047926+0000","last_unstale":"2024-04-25T10:28:38.038050+0000","last_undegraded":"2024-04-25T10:28:38.038050+0000","last_fullsized":"2024-04-25T10:28:38.038050+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:36.009590+0000","last_clean_scrub_stamp":"2024-04-25T10:28:36.009590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:51.888830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"140'3","reported_seq":17,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.272650+0000","last_change":"2024-04-25T10:28:36.046438+0000","last_active":"2024-04-25T10:28:38.272650+0000","last_peered":"2024-04-25T10:28:38.272650+0000","last_clean":"2024-04-25T10:28:38.272650+0000","last_became_active":"2024-04-25T10:28:36.045084+0000","last_became_peered":"2024-04-25T10:28:36.045084+0000","last_unstale":"2024-04-25T10:28:38.272650+0000","last_undegraded":"2024-04-25T10:28:38.272650+0000","last_fullsized":"2024-04-25T10:28:38.272650+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:28:35.006932+0000","last_clean_scrub_stamp":"2024-04-25T10:28:35.006932+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:30.187545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":399,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T10:28:38.029147+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:28:38.029147+0000","last_peered":"2024-04-25T10:28:38.029147+0000","last_clean":"2024-04-25T10:28:38.029147+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:28:38.029147+0000","last_undegraded":"2024-04-25T10:28:38.029147+0000","last_fullsized":"2024-04-25T10:28:38.029147+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509573,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50928,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320912,"statfs":{"total":96636764160,"available":96584613888,"internally_reserved":0,"allocated":1744896,"data_stored":761159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509574,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55104,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316736,"statfs":{"total":96636764160,"available":96580337664,"internally_reserved":0,"allocated":1757184,"data_stored":767459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509573,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59868,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311972,"statfs":{"total":96636764160,"available":96575459328,"internally_reserved":0,"allocated":2375680,"data_stored":1359633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509573,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59268,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312572,"statfs":{"total":96636764160,"available":96576073728,"internally_reserved":0,"allocated":1753088,"data_stored":761775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509573,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46776,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325064,"statfs":{"total":96636764160,"available":96588865536,"internally_reserved":0,"allocated":1748992,"data_stored":761181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509574,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64008,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307832,"statfs":{"total":96636764160,"available":96571219968,"internally_reserved":0,"allocated":2355200,"data_stored":1357897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509574,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1748992,"data_stored":761249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542276,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46764,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325076,"statfs":{"total":96636764160,"available":96588877824,"internally_reserved":0,"allocated":1732608,"data_stored":754384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:28:41.558 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:28:41.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:28:41.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.744+0000 7f8299600640 1 Processor -- start 2024-04-25T10:28:41.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.744+0000 7f8299600640 1 -- start start 2024-04-25T10:28:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8299600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f829415bbb0 0x7f82940cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8299600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82940cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8299600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829406ca90 con 0x7f82940b4660 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8299600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829406d050 con 0x7f829415bbb0 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8299600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829406c830 con 0x7f82940cf960 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f829415bbb0 0x7f82940cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f828be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f82940b4660 0x7f829406c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52962/0 (socket says 172.21.15.45:52962) 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f828be00640 1 -- 172.21.15.45:0/3472375221 learned_addr learned my addr 172.21.15.45:0/3472375221 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292a00640 1 -- 172.21.15.45:0/3472375221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 msgr2=0x7f82940cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8293400640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82940cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292000640 1 -- 172.21.15.45:0/3472375221 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266490279 0 0) 0x7f829406ca90 con 0x7f82940b4660 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292a00640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82940cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292a00640 1 -- 172.21.15.45:0/3472375221 >> v1:172.21.15.45:6789/0 conn(0x7f82940b4660 legacy=0x7f829406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292a00640 1 -- 172.21.15.45:0/3472375221 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829406cd70 con 0x7f829415bbb0 2024-04-25T10:28:41.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8293400640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82940cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292a00640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f82940cda80 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f82840097c0 tx=0x7f828401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a078248e5d71ecd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292000640 1 -- 172.21.15.45:0/3472375221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82840b0070 con 0x7f829415bbb0 2024-04-25T10:28:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292000640 1 -- 172.21.15.45:0/3472375221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82840ab600 con 0x7f829415bbb0 2024-04-25T10:28:41.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.748+0000 7f8292000640 1 -- 172.21.15.45:0/3472375221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82840ab9d0 con 0x7f829415bbb0 2024-04-25T10:28:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- 172.21.15.45:0/3472375221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 msgr2=0x7f82940cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f82940cda80 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f82840097c0 tx=0x7f828401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- 172.21.15.45:0/3472375221 shutdown_connections 2024-04-25T10:28:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82940cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 --2- 172.21.15.45:0/3472375221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f82940cda80 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- 172.21.15.45:0/3472375221 >> 172.21.15.45:0/3472375221 conn(0x7f8294066b70 msgr2=0x7f8294066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:41.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- 172.21.15.45:0/3472375221 shutdown_connections 2024-04-25T10:28:41.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- 172.21.15.45:0/3472375221 wait complete. 2024-04-25T10:28:41.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 Processor -- start 2024-04-25T10:28:41.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- start start 2024-04-25T10:28:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940b4660 0x7f82940aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82941574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f8294159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940b3a10 con 0x7f829415bbb0 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940b3890 con 0x7f82940b4660 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8299600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940b3b90 con 0x7f82940cf960 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8293400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f8294159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8293400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f8294159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58118/0 (socket says 172.21.15.45:58118) 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8293400640 1 -- 172.21.15.45:0/3437932634 learned_addr learned my addr 172.21.15.45:0/3437932634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f8292a00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940b4660 0x7f82940aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.752+0000 7f828be00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82941574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8293400640 1 -- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 msgr2=0x7f82941574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8293400640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82941574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8293400640 1 -- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940b4660 msgr2=0x7f82940aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8293400640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940b4660 0x7f82940aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8293400640 1 -- 172.21.15.45:0/3437932634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940aa090 con 0x7f829415bbb0 2024-04-25T10:28:41.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828be00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82941574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8292a00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940b4660 0x7f82940aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8293400640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f8294159c50 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f828c00cd50 tx=0x7f828c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00f680 con 0x7f829415bbb0 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c00f840 con 0x7f829415bbb0 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00fc30 con 0x7f829415bbb0 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82940aa370 con 0x7f829415bbb0 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8294149040 con 0x7f829415bbb0 2024-04-25T10:28:41.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829404ef30 con 0x7f829415bbb0 2024-04-25T10:28:41.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f828c00e3e0 con 0x7f829415bbb0 2024-04-25T10:28:41.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828aa00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8270066f60 0x7f8270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:41.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f828c085e50 con 0x7f829415bbb0 2024-04-25T10:28:41.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.756+0000 7f8292a00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8270066f60 0x7f8270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:41.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.760+0000 7f8292a00640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8270066f60 0x7f8270069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82840ad000 tx=0x7f82840a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:41.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.760+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f828c014030 con 0x7f829415bbb0 2024-04-25T10:28:41.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.980+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f82940b1fa0 con 0x7f829415bbb0 2024-04-25T10:28:41.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f828aa00640 1 -- 172.21.15.45:0/3437932634 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f828c056dc0 con 0x7f829415bbb0 2024-04-25T10:28:41.984 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:28:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:28:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:28:41.984 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:28:41.985 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:28:41.985 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8270066f60 msgr2=0x7f8270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8270066f60 0x7f8270069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82840ad000 tx=0x7f82840a6000 comp rx=0 tx=0).stop 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 msgr2=0x7f8294159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f8294159c50 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f828c00cd50 tx=0x7f828c00a650 comp rx=0 tx=0).stop 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 shutdown_connections 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8270066f60 0x7f8270069420 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829415bbb0 0x7f8294159c50 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f82940cf960 0x7f82941574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 --2- 172.21.15.45:0/3437932634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82940b4660 0x7f82940aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:41.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 >> 172.21.15.45:0/3437932634 conn(0x7f8294066b70 msgr2=0x7f82940b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:41.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 shutdown_connections 2024-04-25T10:28:41.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:41.984+0000 7f8299600640 1 -- 172.21.15.45:0/3437932634 wait complete. 2024-04-25T10:28:42.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:28:42.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.148+0000 7f3fb3e00640 1 Processor -- start 2024-04-25T10:28:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.148+0000 7f3fb3e00640 1 -- start start 2024-04-25T10:28:42.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fac0d0350 0x7f3fac0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb3e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac0ccaa0 con 0x7f3fac15df60 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb3e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac0ccd80 con 0x7f3fac06aea0 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb3e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac15e340 con 0x7f3fac0d0350 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fac0d0350 0x7f3fac0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3fac0d0350 0x7f3fac0d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58128/0 (socket says 172.21.15.45:58128) 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1177141505 learned_addr learned my addr 172.21.15.45:0/1177141505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb1400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3fac15df60 0x7f3fac0cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52964/0 (socket says 172.21.15.45:52964) 2024-04-25T10:28:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1177141505 >> v1:172.21.15.117:6789/0 conn(0x7f3fac06aea0 legacy=0x7f3fac06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1177141505 >> v1:172.21.15.45:6789/0 conn(0x7f3fac15df60 legacy=0x7f3fac0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1177141505 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0cd060 con 0x7f3fac0d0350 2024-04-25T10:28:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fabe00640 1 -- 172.21.15.45:0/1177141505 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712689195 0 0) 0x7f3fac0ccaa0 con 0x7f3fac15df60 2024-04-25T10:28:42.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fb0a00640 1 --2- 172.21.15.45:0/1177141505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac0d07b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009870 tx=0x7f3f9801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5efedc06767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fabe00640 1 -- 172.21.15.45:0/1177141505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980b1070 con 0x7f3fac0d0350 2024-04-25T10:28:42.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fabe00640 1 -- 172.21.15.45:0/1177141505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac06c1d0 con 0x7f3fac0d0350 2024-04-25T10:28:42.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.152+0000 7f3fabe00640 1 -- 172.21.15.45:0/1177141505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980ac840 con 0x7f3fac0d0350 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1177141505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 msgr2=0x7f3fac0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1177141505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac0d07b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009870 tx=0x7f3f9801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1177141505 shutdown_connections 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1177141505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac0d07b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1177141505 >> 172.21.15.45:0/1177141505 conn(0x7f3fac0d1910 msgr2=0x7f3fac0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1177141505 shutdown_connections 2024-04-25T10:28:42.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1177141505 wait complete. 2024-04-25T10:28:42.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 Processor -- start 2024-04-25T10:28:42.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- start start 2024-04-25T10:28:42.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fac06aea0 0x7f3fac0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fac15df60 0x7f3fac0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0d0ff0 con 0x7f3fac0d0350 2024-04-25T10:28:42.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0d0e70 con 0x7f3fac15df60 2024-04-25T10:28:42.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb3e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac16ba30 con 0x7f3fac06aea0 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58144/0 (socket says 172.21.15.45:58144) 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1858505501 learned_addr learned my addr 172.21.15.45:0/1858505501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb1e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fac15df60 0x7f3fac0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb1400640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fac06aea0 0x7f3fac0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fac06aea0 msgr2=0x7f3fac0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fac06aea0 0x7f3fac0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fac15df60 msgr2=0x7f3fac0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fac15df60 0x7f3fac0c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb0a00640 1 -- 172.21.15.45:0/1858505501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0c9f30 con 0x7f3fac0d0350 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.156+0000 7f3fb1e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fac15df60 0x7f3fac0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:42.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fb0a00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac152b30 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3f980190e0 tx=0x7f3f98019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980b1040 con 0x7f3fac0d0350 2024-04-25T10:28:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f980ba070 con 0x7f3fac0d0350 2024-04-25T10:28:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac0ca210 con 0x7f3fac0d0350 2024-04-25T10:28:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980b5680 con 0x7f3fac0d0350 2024-04-25T10:28:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac0ca7d0 con 0x7f3fac0d0350 2024-04-25T10:28:42.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac04ef30 con 0x7f3fac0d0350 2024-04-25T10:28:42.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f980b5820 con 0x7f3fac0d0350 2024-04-25T10:28:42.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fa9600640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f8c066f60 0x7f3f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f9811ef30 con 0x7f3fac0d0350 2024-04-25T10:28:42.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.160+0000 7f3fb1400640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f8c066f60 0x7f3f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.164+0000 7f3fb1400640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f8c066f60 0x7f3f8c069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0000f70 tx=0x7f3fa0007d10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.164+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f98016150 con 0x7f3fac0d0350 2024-04-25T10:28:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.360+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fac0b2a00 con 0x7f3fac0d0350 2024-04-25T10:28:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.360+0000 7f3fa9600640 1 -- 172.21.15.45:0/1858505501 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4945 (secure 0 0 0) 0x7f3f980eff30 con 0x7f3fac0d0350 2024-04-25T10:28:42.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:28:42.363 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8893,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:28:39.444741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:28:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 104 2024-04-25T10:28:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f8c066f60 msgr2=0x7f3f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f8c066f60 0x7f3f8c069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0000f70 tx=0x7f3fa0007d10 comp rx=0 tx=0).stop 2024-04-25T10:28:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 msgr2=0x7f3fac152b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac152b30 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f3f980190e0 tx=0x7f3f98019270 comp rx=0 tx=0).stop 2024-04-25T10:28:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 shutdown_connections 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f8c066f60 0x7f3f8c069420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fac15df60 0x7f3fac0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fac0d0350 0x7f3fac152b30 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/1858505501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3fac06aea0 0x7f3fac0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 >> 172.21.15.45:0/1858505501 conn(0x7f3fac0d1910 msgr2=0x7f3fac159640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 shutdown_connections 2024-04-25T10:28:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.364+0000 7f3fb3e00640 1 -- 172.21.15.45:0/1858505501 wait complete. 2024-04-25T10:28:42.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:28:36.425124+0000', 'modified': '2024-04-25T10:28:39.444741+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9531': {'gid': 9531, 'name': 'a', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/903752757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 903752757}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 903752757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:28:42.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:28:42.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:28:42.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.932+0000 7f8c4fc00640 1 Processor -- start 2024-04-25T10:28:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.932+0000 7f8c4fc00640 1 -- start start 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4fc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480b0f70 0x7f8c480b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4fc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c480cfd90 0x7f8c480ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4fc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4815f3b0 con 0x7f8c480cd110 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4fc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4815ef20 con 0x7f8c480cfd90 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4fc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480ca810 con 0x7f8c480b0f70 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c3fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c480cfd90 0x7f8c480ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4dc00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8c480cd110 0x7f8c480cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52976/0 (socket says 172.21.15.45:52976) 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4dc00640 1 -- 172.21.15.45:0/2768340807 learned_addr learned my addr 172.21.15.45:0/2768340807 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:28:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c4d200640 1 --2- 172.21.15.45:0/2768340807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480b0f70 0x7f8c480b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c3fe00640 1 -- 172.21.15.45:0/2768340807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480b0f70 msgr2=0x7f8c480b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/2768340807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480b0f70 0x7f8c480b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c3fe00640 1 -- 172.21.15.45:0/2768340807 >> v1:172.21.15.45:6789/0 conn(0x7f8c480cd110 legacy=0x7f8c480cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c3fe00640 1 -- 172.21.15.45:0/2768340807 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480b1890 con 0x7f8c480cfd90 2024-04-25T10:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/2768340807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480cfd90 0x7f8c480ccbd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097c0 tx=0x7f8c3001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bb208787bd6abf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.936+0000 7f8c47e00640 1 -- 172.21.15.45:0/2768340807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300b0070 con 0x7f8c480cfd90 2024-04-25T10:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c47e00640 1 -- 172.21.15.45:0/2768340807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c300ab600 con 0x7f8c480cfd90 2024-04-25T10:28:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c47e00640 1 -- 172.21.15.45:0/2768340807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300ab9d0 con 0x7f8c480cfd90 2024-04-25T10:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- 172.21.15.45:0/2768340807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480cfd90 msgr2=0x7f8c480ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/2768340807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480cfd90 0x7f8c480ccbd0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097c0 tx=0x7f8c3001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- 172.21.15.45:0/2768340807 shutdown_connections 2024-04-25T10:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/2768340807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480cfd90 0x7f8c480ccbd0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/2768340807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480b0f70 0x7f8c480b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- 172.21.15.45:0/2768340807 >> 172.21.15.45:0/2768340807 conn(0x7f8c480b1d50 msgr2=0x7f8c480b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- 172.21.15.45:0/2768340807 shutdown_connections 2024-04-25T10:28:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- 172.21.15.45:0/2768340807 wait complete. 2024-04-25T10:28:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 Processor -- start 2024-04-25T10:28:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- start start 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 0x7f8c48155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c480cd110 0x7f8c480c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480cfd90 0x7f8c48155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481619d0 con 0x7f8c480b0f70 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480cd520 con 0x7f8c480cd110 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4fc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48161b50 con 0x7f8c480cfd90 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 0x7f8c48155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 0x7f8c48155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58166/0 (socket says 172.21.15.45:58166) 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 -- 172.21.15.45:0/326583586 learned_addr learned my addr 172.21.15.45:0/326583586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480cfd90 0x7f8c48155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c480cd110 0x7f8c480c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 -- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480cfd90 msgr2=0x7f8c48155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480cfd90 0x7f8c48155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 -- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c480cd110 msgr2=0x7f8c480c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c480cd110 0x7f8c480c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.940+0000 7f8c4d200640 1 -- 172.21.15.45:0/326583586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480c55f0 con 0x7f8c480b0f70 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c480cd110 0x7f8c480c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c4d200640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 0x7f8c48155020 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e930 tx=0x7f8c3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c45600640 1 -- 172.21.15.45:0/326583586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38017070 con 0x7f8c480b0f70 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c45600640 1 -- 172.21.15.45:0/326583586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3800ca00 con 0x7f8c480b0f70 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c45600640 1 -- 172.21.15.45:0/326583586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800cdb0 con 0x7f8c480b0f70 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c480c58d0 con 0x7f8c480b0f70 2024-04-25T10:28:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c480c5e90 con 0x7f8c480b0f70 2024-04-25T10:28:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.944+0000 7f8c45600640 1 -- 172.21.15.45:0/326583586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c3801a070 con 0x7f8c480b0f70 2024-04-25T10:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c45600640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c24066f60 0x7f8c24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c45600640 1 -- 172.21.15.45:0/326583586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c38085380 con 0x7f8c480b0f70 2024-04-25T10:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 Processor -- start 2024-04-25T10:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- start start 2024-04-25T10:28:42.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c24066f60 0x7f8c24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f47ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f431310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c24066f60 0x7f8c24069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c300ad000 tx=0x7f8c30008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a64f3185f0 con 0x55a64f47a920 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a64f27fda0 con 0x55a64f430f30 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a64f208150 con 0x55a64f47b240 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f47ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f47ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58174/0 (socket says 172.21.15.45:58174) 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 -- 172.21.15.45:0/2978592051 learned_addr learned my addr 172.21.15.45:0/2978592051 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f431310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 -- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 msgr2=0x55a64f4309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 -- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 msgr2=0x55a64f431310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f431310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 -- 172.21.15.45:0/2978592051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a64f221ca0 con 0x55a64f47a920 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c4d200640 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f47ad00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8c380097c0 tx=0x7f8c38052100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31f0254452b00ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c3d600640 1 -- 172.21.15.45:0/2978592051 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380540e0 con 0x55a64f47a920 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c3d600640 1 -- 172.21.15.45:0/2978592051 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3805ae50 con 0x55a64f47a920 2024-04-25T10:28:42.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c3d600640 1 -- 172.21.15.45:0/2978592051 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3805c0b0 con 0x55a64f47a920 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- 172.21.15.45:0/2978592051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 msgr2=0x55a64f47ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f47ad00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f8c380097c0 tx=0x7f8c38052100 comp rx=0 tx=0).stop 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- 172.21.15.45:0/2978592051 shutdown_connections 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f431310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/2978592051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f47ad00 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- 172.21.15.45:0/2978592051 >> 172.21.15.45:0/2978592051 conn(0x55a64f4372a0 msgr2=0x55a64f42f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- 172.21.15.45:0/2978592051 shutdown_connections 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.948+0000 7f8c52ff6000 1 -- 172.21.15.45:0/2978592051 wait complete. 2024-04-25T10:28:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- start start 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 Processor -- start 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 0x55a64f481c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f47a920 0x55a64f4821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f434aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a64f23a990 con 0x55a64f430f30 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a64f27be40 con 0x55a64f47a920 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a64f20b900 con 0x55a64f47b240 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 10 client.0 _ll_register_callbacks cb 0x55a64f3331c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 0x55a64f481c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 0x55a64f481c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58186/0 (socket says 172.21.15.45:58186) 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 -- 172.21.15.45:0/131951441 learned_addr learned my addr 172.21.15.45:0/131951441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f434aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f47a920 0x55a64f4821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 -- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 msgr2=0x55a64f434aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f434aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 -- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f47a920 msgr2=0x55a64f4821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f47a920 0x55a64f4821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 -- 172.21.15.45:0/131951441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a64f238630 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f47a920 0x55a64f4821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c4d200640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 0x55a64f481c80 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8c38083990 tx=0x7f8c38000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ec6f96ff733f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 1 -- 172.21.15.45:0/131951441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38073070 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 1 -- 172.21.15.45:0/131951441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c38055230 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c3805ae50 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 1 -- 172.21.15.45:0/131951441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c380541a0 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 10 client.9672 fetch_fsmap learned FSMap version 104 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55a64f2083a0 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 1 -- 172.21.15.45:0/131951441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3805ae50 con 0x55a64f430f30 2024-04-25T10:28:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 1 -- 172.21.15.45:0/131951441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c38051b30 con 0x55a64f430f30 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 10 client.9672 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 10 client.9672 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 20 client.9672 populate_metadata read hostname 'smithi045' 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f43baf0 0x55a64f4354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 4 client.9672 mds_command: new command op to 9531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 --> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a64f21ba50 con 0x55a64f43baf0 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f43baf0 0x55a64f4354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f43baf0 0x55a64f4354b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81843471970f8db4 server_cookie=b1e5b802f9937d89 in_seq=0 out_seq=0 2024-04-25T10:28:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.952+0000 7f8c36000640 10 client.9672 ms_handle_connect on v2:172.21.15.45:6834/903752757 2024-04-25T10:28:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c36000640 1 -- 172.21.15.45:0/131951441 <== mds.0 v2:172.21.15.45:6834/903752757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a64f21ba50 con 0x55a64f43baf0 2024-04-25T10:28:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c36000640 10 client.9672 handle_command_reply: tid=0 2024-04-25T10:28:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 client.9672 shutdown 2024-04-25T10:28:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 msgr2=0x55a64f481c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 0x55a64f481c80 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8c38083990 tx=0x7f8c38000bd0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 shutdown_connections 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f43baf0 0x55a64f4354b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f434aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f47a920 0x55a64f4821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/131951441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f430f30 0x55a64f481c80 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c36000640 0 client.9672 ms_handle_reset on v2:172.21.15.45:6834/903752757 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 >> 172.21.15.45:0/131951441 conn(0x55a64f4372a0 msgr2=0x55a64f42f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 shutdown_connections 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 1 -- 172.21.15.45:0/131951441 wait complete. 2024-04-25T10:28:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.956+0000 7f8c52ff6000 20 client.9672 trim_cache size 0 max 16384 2024-04-25T10:28:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 Processor -- start 2024-04-25T10:28:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- start start 2024-04-25T10:28:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47a920 0x55a64f4d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 0x55a64f4d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48202140 con 0x55a64f47b240 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48201570 con 0x55a64f430f30 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48201b40 con 0x55a64f47a920 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 0x55a64f4d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 0x55a64f4d73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58194/0 (socket says 172.21.15.45:58194) 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 -- 172.21.15.45:0/3392365477 learned_addr learned my addr 172.21.15.45:0/3392365477 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4d200640 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47a920 0x55a64f4d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 -- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47a920 msgr2=0x55a64f4d6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47a920 0x55a64f4d6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 -- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 msgr2=0x55a64f4d6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4d6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 -- 172.21.15.45:0/3392365477 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a64f47b070 con 0x55a64f47b240 2024-04-25T10:28:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 0x55a64f4d73c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001bf0 tx=0x7f8c400020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aede02efa7b0284a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c3cc00640 1 -- 172.21.15.45:0/3392365477 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4001a020 con 0x55a64f47b240 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c3cc00640 1 -- 172.21.15.45:0/3392365477 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c400076f0 con 0x55a64f47b240 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c3cc00640 1 -- 172.21.15.45:0/3392365477 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40007aa0 con 0x55a64f47b240 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3392365477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 msgr2=0x55a64f4d73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 0x55a64f4d73c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001bf0 tx=0x7f8c400020b0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3392365477 shutdown_connections 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47b240 0x55a64f4d73c0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47a920 0x55a64f4d6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3392365477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3392365477 >> 172.21.15.45:0/3392365477 conn(0x55a64f4372a0 msgr2=0x55a64f466f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3392365477 shutdown_connections 2024-04-25T10:28:42.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3392365477 wait complete. 2024-04-25T10:28:42.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.972+0000 7f8c52ff6000 1 -- start start 2024-04-25T10:28:42.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 Processor -- start 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f4de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a64f2843d0 con 0x55a64f47a920 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a64f33bd50 con 0x55a64f430f30 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a64f21a980 con 0x55a64f47b240 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 10 client.0 _ll_register_callbacks cb 0x7f8c38009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:28:42.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f4de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f4de720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58202/0 (socket says 172.21.15.45:58202) 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 -- 172.21.15.45:0/3689936648 learned_addr learned my addr 172.21.15.45:0/3689936648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c4d200640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 -- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 msgr2=0x55a64f4e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 -- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 msgr2=0x55a64f4dc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4dc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 -- 172.21.15.45:0/3689936648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a64f301600 con 0x55a64f47a920 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c4dc00640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c4d200640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c3fe00640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f4de720 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8c3001ab50 tx=0x7f8c30019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70e786cb1d21401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 1 -- 172.21.15.45:0/3689936648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300c83f0 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 1 -- 172.21.15.45:0/3689936648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c300c8590 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c400076f0 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 1 -- 172.21.15.45:0/3689936648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c300c8790 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 10 client.9678 fetch_fsmap learned FSMap version 104 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55a64f2149b0 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 1 -- 172.21.15.45:0/3689936648 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c400076f0 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 1 -- 172.21.15.45:0/3689936648 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c300c0040 con 0x55a64f47a920 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 10 client.9678 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 10 client.9678 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 20 client.9678 populate_metadata read hostname 'smithi045' 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f4e11b0 0x55a64f4e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 4 client.9678 mds_command: new command op to 9531 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:28:42.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 --> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a64f28d2e0 con 0x55a64f4e11b0 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c4d200640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f4e11b0 0x55a64f4e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c4d200640 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f4e11b0 0x55a64f4e35a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b488daee379ced0 server_cookie=96ca4298168d6e27 in_seq=0 out_seq=0 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 10 client.9678 ms_handle_connect on v2:172.21.15.45:6834/903752757 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 1 -- 172.21.15.45:0/3689936648 <== mds.0 v2:172.21.15.45:6834/903752757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55a64f28d2e0 con 0x55a64f4e11b0 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c36000640 10 client.9678 handle_command_reply: tid=0 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 client.9678 shutdown 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 msgr2=0x55a64f4de720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.976+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f4de720 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8c3001ab50 tx=0x7f8c30019240 comp rx=0 tx=0).stop 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 shutdown_connections 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x55a64f4e11b0 0x55a64f4e35a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55a64f47b240 0x55a64f4e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a64f47a920 0x55a64f4de720 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c36000640 0 client.9678 ms_handle_reset on v2:172.21.15.45:6834/903752757 2024-04-25T10:28:42.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 --2- 172.21.15.45:0/3689936648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a64f430f30 0x55a64f4dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 >> 172.21.15.45:0/3689936648 conn(0x55a64f4372a0 msgr2=0x55a64f4354b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 shutdown_connections 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 1 -- 172.21.15.45:0/3689936648 wait complete. 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c52ff6000 20 client.9678 trim_cache size 0 max 16384 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9531, 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:28:42.984 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:28:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9579298129999998, 2024-04-25T10:28:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 104, 2024-04-25T10:28:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 141, 2024-04-25T10:28:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 141, 2024-04-25T10:28:42.985 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 10.779931691 2024-04-25T10:28:42.985 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c24066f60 msgr2=0x7f8c24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c24066f60 0x7f8c24069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c300ad000 tx=0x7f8c30008040 comp rx=0 tx=0).stop 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 msgr2=0x7f8c48155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 0x7f8c48155020 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e930 tx=0x7f8c3800edf0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 shutdown_connections 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c24066f60 0x7f8c24069420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c480cfd90 0x7f8c48155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c480cd110 0x7f8c480c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 --2- 172.21.15.45:0/326583586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c480b0f70 0x7f8c48155020 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:42.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 >> 172.21.15.45:0/326583586 conn(0x7f8c480b1d50 msgr2=0x7f8c48160870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:42.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 shutdown_connections 2024-04-25T10:28:42.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:42.980+0000 7f8c4fc00640 1 -- 172.21.15.45:0/326583586 wait complete. 2024-04-25T10:28:42.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:28:43.002 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-04-25T10:28:43.000+0000 7f47f0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:28:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.140+0000 7fd2d2400640 1 Processor -- start 2024-04-25T10:28:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.140+0000 7fd2d2400640 1 -- start start 2024-04-25T10:28:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2d2400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cda10 0x7fd2cc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2d2400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2cc0cf930 0x7fd2cc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2d2400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0b2770 con 0x7fd2cc15bc40 2024-04-25T10:28:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2d2400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2cc06c670 con 0x7fd2cc0cf930 2024-04-25T10:28:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2d2400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc06c950 con 0x7fd2cc0cda10 2024-04-25T10:28:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cda10 0x7fd2cc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2cc0cf930 0x7fd2cc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2cc0cf930 0x7fd2cc0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37610/0 (socket says 172.21.15.45:37610) 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 -- 172.21.15.45:0/3463027084 learned_addr learned my addr 172.21.15.45:0/3463027084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 -- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cda10 msgr2=0x7fd2cc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 --2- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cda10 0x7fd2cc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 -- 172.21.15.45:0/3463027084 >> v1:172.21.15.117:6789/0 conn(0x7fd2cc15bc40 legacy=0x7fd2cc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2cbe00640 1 -- 172.21.15.45:0/3463027084 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc06cc30 con 0x7fd2cc0cf930 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2c2a00640 1 --2- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cda10 0x7fd2cc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:28:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.144+0000 7fd2caa00640 1 -- 172.21.15.45:0/3463027084 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891050863 0 0) 0x7fd2cc0b2770 con 0x7fd2cc15bc40 2024-04-25T10:28:43.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2cbe00640 1 --2- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cf930 0x7fd2cc0cfde0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009e50 tx=0x7fd2c401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2114d8273aa3159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:43.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2caa00640 1 -- 172.21.15.45:0/3463027084 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40a5660 con 0x7fd2cc0cf930 2024-04-25T10:28:43.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2caa00640 1 -- 172.21.15.45:0/3463027084 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c40a5800 con 0x7fd2cc0cf930 2024-04-25T10:28:43.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2caa00640 1 -- 172.21.15.45:0/3463027084 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40a5bd0 con 0x7fd2cc0cf930 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 -- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cf930 msgr2=0x7fd2cc0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 --2- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cf930 0x7fd2cc0cfde0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009e50 tx=0x7fd2c401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 -- 172.21.15.45:0/3463027084 shutdown_connections 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 --2- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cf930 0x7fd2cc0cfde0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 --2- 172.21.15.45:0/3463027084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cda10 0x7fd2cc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 -- 172.21.15.45:0/3463027084 >> 172.21.15.45:0/3463027084 conn(0x7fd2cc066b70 msgr2=0x7fd2cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:43.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 -- 172.21.15.45:0/3463027084 shutdown_connections 2024-04-25T10:28:43.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 -- 172.21.15.45:0/3463027084 wait complete. 2024-04-25T10:28:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 Processor -- start 2024-04-25T10:28:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.148+0000 7fd2d2400640 1 -- start start 2024-04-25T10:28:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cda10 0x7fd2cc15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cf930 0x7fd2cc151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 0x7fd2cc157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b3850 con 0x7fd2cc15bc40 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b36d0 con 0x7fd2cc0cda10 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0b39d0 con 0x7fd2cc0cf930 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cda10 0x7fd2cc15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 0x7fd2cc157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 0x7fd2cc157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58212/0 (socket says 172.21.15.45:58212) 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 -- 172.21.15.45:0/663852552 learned_addr learned my addr 172.21.15.45:0/663852552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2c2a00640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cf930 0x7fd2cc151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 -- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cf930 msgr2=0x7fd2cc151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cf930 0x7fd2cc151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 -- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cda10 msgr2=0x7fd2cc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cda10 0x7fd2cc15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 -- 172.21.15.45:0/663852552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc15c900 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2cbe00640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 0x7fd2cc157420 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fd2c401a5d0 tx=0x7fd2c4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40bc020 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c40b8880 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc15cbe0 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40a5660 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc15d1a0 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.152+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc04ef30 con 0x7fd2cc15bc40 2024-04-25T10:28:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.156+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2c40bf020 con 0x7fd2cc15bc40 2024-04-25T10:28:43.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.156+0000 7fd2c3e00640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2a8066f60 0x7fd2a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:28:43.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.156+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd2c411fef0 con 0x7fd2cc15bc40 2024-04-25T10:28:43.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.156+0000 7fd2cb400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2a8066f60 0x7fd2a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:28:43.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.156+0000 7fd2cb400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2a8066f60 0x7fd2a8069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc007fd0 tx=0x7fd2bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:28:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.160+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2c40c6030 con 0x7fd2cc15bc40 2024-04-25T10:28:43.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.356+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd2cc15a450 con 0x7fd2cc15bc40 2024-04-25T10:28:43.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.356+0000 7fd2c3e00640 1 -- 172.21.15.45:0/663852552 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd2c4016620 con 0x7fd2cc15bc40 2024-04-25T10:28:43.359 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:28:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2a8066f60 msgr2=0x7fd2a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2a8066f60 0x7fd2a8069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc007fd0 tx=0x7fd2bc009040 comp rx=0 tx=0).stop 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 msgr2=0x7fd2cc157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 0x7fd2cc157420 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fd2c401a5d0 tx=0x7fd2c4019740 comp rx=0 tx=0).stop 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 shutdown_connections 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd2a8066f60 0x7fd2a8069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2cc15bc40 0x7fd2cc157420 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2cc0cf930 0x7fd2cc151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 --2- 172.21.15.45:0/663852552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2cc0cda10 0x7fd2cc15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 >> 172.21.15.45:0/663852552 conn(0x7fd2cc066b70 msgr2=0x7fd2cc159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 shutdown_connections 2024-04-25T10:28:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:28:43.360+0000 7fd2d2400640 1 -- 172.21.15.45:0/663852552 wait complete. 2024-04-25T10:28:43.380 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:28:43.380 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:28:43.380 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:28:43.380 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:28:43.380 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:28:43.381 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:28:43.381 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:28:43.381 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:28:43.381 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:28:43.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.381 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:28:43.391 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41817sec preferred_lft 41817sec 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:28:43.392 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:28:43.393 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:28:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:28:43.394 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:28:43.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.394 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:28:43.457 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:28:43.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.459 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:28:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:28:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:28:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:28:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:28:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:28:43.513 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:28:43.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.514 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:28:43.514 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:43.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:28:43.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:28:43.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:28:43.514 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:43.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.606 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:28:43.615 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:28:43.615 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:28:43.615 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:28:43.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:28:43.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:28:43.679 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:28:43.735 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:28:43.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:28:43.736 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:28:43.787 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:28:43.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:28:43.880 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:28:43.880 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:28:43.898 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:28:43.898 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:28:43.898 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:28:43.898 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:28:43.898 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:28:43.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:28:43.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:28:43.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:43.922 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:28:43.980 INFO:teuthology.orchestra.run.smithi045.stdout:client9043 2024-04-25T10:28:43.980 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:28:43.980 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:28:44.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:44.031 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:44.032 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:44.032 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:28:44.032 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:44.032 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:44.032 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:44.032 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.120 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186961 s, 56.1 MB/s 2024-04-25T10:28:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168269 s, 62.3 MB/s 2024-04-25T10:28:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150901 s, 69.5 MB/s 2024-04-25T10:28:44.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.179 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213189 s, 49.2 MB/s 2024-04-25T10:28:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.197 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156077 s, 67.2 MB/s 2024-04-25T10:28:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.213 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142874 s, 73.4 MB/s 2024-04-25T10:28:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.230 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156883 s, 66.8 MB/s 2024-04-25T10:28:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.247 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149495 s, 70.1 MB/s 2024-04-25T10:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.266 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169815 s, 61.7 MB/s 2024-04-25T10:28:44.371 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104013 s, 10.1 MB/s 2024-04-25T10:28:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.390 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162917 s, 64.4 MB/s 2024-04-25T10:28:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149942 s, 69.9 MB/s 2024-04-25T10:28:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158535 s, 66.1 MB/s 2024-04-25T10:28:44.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.442 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162201 s, 64.6 MB/s 2024-04-25T10:28:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153539 s, 68.3 MB/s 2024-04-25T10:28:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.476 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145352 s, 72.1 MB/s 2024-04-25T10:28:44.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.497 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194119 s, 54.0 MB/s 2024-04-25T10:28:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134226 s, 78.1 MB/s 2024-04-25T10:28:44.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.528 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.528 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139844 s, 75.0 MB/s 2024-04-25T10:28:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.545 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153221 s, 68.4 MB/s 2024-04-25T10:28:44.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.566 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190263 s, 55.1 MB/s 2024-04-25T10:28:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.582 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014555 s, 72.0 MB/s 2024-04-25T10:28:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.599 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151845 s, 69.1 MB/s 2024-04-25T10:28:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.616 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150088 s, 69.9 MB/s 2024-04-25T10:28:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149181 s, 70.3 MB/s 2024-04-25T10:28:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.649 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143181 s, 73.2 MB/s 2024-04-25T10:28:44.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138292 s, 75.8 MB/s 2024-04-25T10:28:44.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.684 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170732 s, 61.4 MB/s 2024-04-25T10:28:44.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134263 s, 78.1 MB/s 2024-04-25T10:28:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.719 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016385 s, 64.0 MB/s 2024-04-25T10:28:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132807 s, 79.0 MB/s 2024-04-25T10:28:44.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150478 s, 69.7 MB/s 2024-04-25T10:28:44.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:44.751 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:44.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134891 s, 77.7 MB/s 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149821 s, 70.0 MB/s 2024-04-25T10:28:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131904 s, 79.5 MB/s 2024-04-25T10:28:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149101 s, 70.3 MB/s 2024-04-25T10:28:44.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154646 s, 67.8 MB/s 2024-04-25T10:28:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167008 s, 62.8 MB/s 2024-04-25T10:28:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151985 s, 69.0 MB/s 2024-04-25T10:28:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135604 s, 77.3 MB/s 2024-04-25T10:28:44.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.955 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184058 s, 57.0 MB/s 2024-04-25T10:28:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130601 s, 80.3 MB/s 2024-04-25T10:28:44.991 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:44.991 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:44.991 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186238 s, 56.3 MB/s 2024-04-25T10:28:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.008 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155805 s, 67.3 MB/s 2024-04-25T10:28:45.025 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.025 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.025 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149687 s, 70.1 MB/s 2024-04-25T10:28:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.040 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136525 s, 76.8 MB/s 2024-04-25T10:28:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170461 s, 61.5 MB/s 2024-04-25T10:28:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141135 s, 74.3 MB/s 2024-04-25T10:28:45.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.090 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132654 s, 79.0 MB/s 2024-04-25T10:28:45.109 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.110 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170975 s, 61.3 MB/s 2024-04-25T10:28:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.126 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149692 s, 70.0 MB/s 2024-04-25T10:28:45.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.145 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.145 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016987 s, 61.7 MB/s 2024-04-25T10:28:45.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.162 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.162 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150015 s, 69.9 MB/s 2024-04-25T10:28:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132572 s, 79.1 MB/s 2024-04-25T10:28:45.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.191 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.191 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127153 s, 82.5 MB/s 2024-04-25T10:28:45.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.208 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150581 s, 69.6 MB/s 2024-04-25T10:28:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138037 s, 76.0 MB/s 2024-04-25T10:28:45.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.241 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151926 s, 69.0 MB/s 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.256 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134447 s, 78.0 MB/s 2024-04-25T10:28:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.272 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142727 s, 73.5 MB/s 2024-04-25T10:28:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.287 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132829 s, 78.9 MB/s 2024-04-25T10:28:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.304 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014396 s, 72.8 MB/s 2024-04-25T10:28:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131986 s, 79.4 MB/s 2024-04-25T10:28:45.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151431 s, 69.2 MB/s 2024-04-25T10:28:45.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:45.337 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:45.407 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148226 s, 70.7 MB/s 2024-04-25T10:28:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136287 s, 76.9 MB/s 2024-04-25T10:28:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013097 s, 80.1 MB/s 2024-04-25T10:28:45.453 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.454 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138987 s, 75.4 MB/s 2024-04-25T10:28:45.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.471 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151626 s, 69.2 MB/s 2024-04-25T10:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013225 s, 79.3 MB/s 2024-04-25T10:28:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014319 s, 73.2 MB/s 2024-04-25T10:28:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.521 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171888 s, 61.0 MB/s 2024-04-25T10:28:45.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.536 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133115 s, 78.8 MB/s 2024-04-25T10:28:45.550 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.551 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.551 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126422 s, 82.9 MB/s 2024-04-25T10:28:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.566 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135308 s, 77.5 MB/s 2024-04-25T10:28:45.581 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.581 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.581 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138625 s, 75.6 MB/s 2024-04-25T10:28:45.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.598 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.598 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147339 s, 71.2 MB/s 2024-04-25T10:28:45.617 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176636 s, 59.4 MB/s 2024-04-25T10:28:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121315 s, 86.4 MB/s 2024-04-25T10:28:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.648 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148975 s, 70.4 MB/s 2024-04-25T10:28:45.664 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.664 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.664 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142264 s, 73.7 MB/s 2024-04-25T10:28:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135777 s, 77.2 MB/s 2024-04-25T10:28:45.696 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.697 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151019 s, 69.4 MB/s 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.713 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138462 s, 75.7 MB/s 2024-04-25T10:28:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.729 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139809 s, 75.0 MB/s 2024-04-25T10:28:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.744 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138833 s, 75.5 MB/s 2024-04-25T10:28:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126482 s, 82.9 MB/s 2024-04-25T10:28:45.775 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.775 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.775 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142886 s, 73.4 MB/s 2024-04-25T10:28:45.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152728 s, 68.7 MB/s 2024-04-25T10:28:45.807 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.807 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.807 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136057 s, 77.1 MB/s 2024-04-25T10:28:45.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.824 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149743 s, 70.0 MB/s 2024-04-25T10:28:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018774 s, 55.9 MB/s 2024-04-25T10:28:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142824 s, 73.4 MB/s 2024-04-25T10:28:45.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159988 s, 65.5 MB/s 2024-04-25T10:28:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192036 s, 54.6 MB/s 2024-04-25T10:28:45.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142254 s, 73.7 MB/s 2024-04-25T10:28:45.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:45.917 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134251 s, 78.1 MB/s 2024-04-25T10:28:46.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.003 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150142 s, 69.8 MB/s 2024-04-25T10:28:46.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.019 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139353 s, 75.2 MB/s 2024-04-25T10:28:46.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165959 s, 63.2 MB/s 2024-04-25T10:28:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.053 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140757 s, 74.5 MB/s 2024-04-25T10:28:46.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185402 s, 56.6 MB/s 2024-04-25T10:28:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.090 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154618 s, 67.8 MB/s 2024-04-25T10:28:46.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152519 s, 68.8 MB/s 2024-04-25T10:28:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.121 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120972 s, 86.7 MB/s 2024-04-25T10:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149927 s, 69.9 MB/s 2024-04-25T10:28:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135078 s, 77.6 MB/s 2024-04-25T10:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129989 s, 80.7 MB/s 2024-04-25T10:28:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.183 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127962 s, 81.9 MB/s 2024-04-25T10:28:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.203 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177562 s, 59.1 MB/s 2024-04-25T10:28:46.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.219 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.219 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149345 s, 70.2 MB/s 2024-04-25T10:28:46.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179033 s, 58.6 MB/s 2024-04-25T10:28:46.255 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.256 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.256 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014895 s, 70.4 MB/s 2024-04-25T10:28:46.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.270 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128811 s, 81.4 MB/s 2024-04-25T10:28:46.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.285 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.285 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127013 s, 82.6 MB/s 2024-04-25T10:28:46.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.301 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.301 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136138 s, 77.0 MB/s 2024-04-25T10:28:46.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.316 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131425 s, 79.8 MB/s 2024-04-25T10:28:46.331 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.332 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.332 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139175 s, 75.3 MB/s 2024-04-25T10:28:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.347 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141387 s, 74.2 MB/s 2024-04-25T10:28:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183269 s, 57.2 MB/s 2024-04-25T10:28:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131896 s, 79.5 MB/s 2024-04-25T10:28:46.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.399 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014536 s, 72.1 MB/s 2024-04-25T10:28:46.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135085 s, 77.6 MB/s 2024-04-25T10:28:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.431 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153867 s, 68.1 MB/s 2024-04-25T10:28:46.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.447 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139368 s, 75.2 MB/s 2024-04-25T10:28:46.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.462 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130858 s, 80.1 MB/s 2024-04-25T10:28:46.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135163 s, 77.6 MB/s 2024-04-25T10:28:46.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188174 s, 55.7 MB/s 2024-04-25T10:28:46.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:46.499 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:46.499 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:46.499 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:28:46.499 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:46.500 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:46.500 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:46.500 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:46.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.570 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.570 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132007 s, 79.4 MB/s 2024-04-25T10:28:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147578 s, 71.1 MB/s 2024-04-25T10:28:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174992 s, 59.9 MB/s 2024-04-25T10:28:46.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.625 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016871 s, 62.2 MB/s 2024-04-25T10:28:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018572 s, 56.5 MB/s 2024-04-25T10:28:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183184 s, 57.2 MB/s 2024-04-25T10:28:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142926 s, 73.4 MB/s 2024-04-25T10:28:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.697 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141296 s, 74.2 MB/s 2024-04-25T10:28:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138325 s, 75.8 MB/s 2024-04-25T10:28:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014 s, 74.9 MB/s 2024-04-25T10:28:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.744 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.744 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136112 s, 77.0 MB/s 2024-04-25T10:28:46.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.759 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013119 s, 79.9 MB/s 2024-04-25T10:28:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015424 s, 68.0 MB/s 2024-04-25T10:28:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.791 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135377 s, 77.5 MB/s 2024-04-25T10:28:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127309 s, 82.4 MB/s 2024-04-25T10:28:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179724 s, 58.3 MB/s 2024-04-25T10:28:46.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.841 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.841 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130998 s, 80.0 MB/s 2024-04-25T10:28:46.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.857 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144377 s, 72.6 MB/s 2024-04-25T10:28:46.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.874 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.874 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150543 s, 69.7 MB/s 2024-04-25T10:28:46.894 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.894 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.895 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187745 s, 55.9 MB/s 2024-04-25T10:28:46.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.910 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.911 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142068 s, 73.8 MB/s 2024-04-25T10:28:46.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127159 s, 82.5 MB/s 2024-04-25T10:28:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150223 s, 69.8 MB/s 2024-04-25T10:28:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.957 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131599 s, 79.7 MB/s 2024-04-25T10:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.972 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131352 s, 79.8 MB/s 2024-04-25T10:28:46.988 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:46.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:46.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148999 s, 70.4 MB/s 2024-04-25T10:28:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.005 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146513 s, 71.6 MB/s 2024-04-25T10:28:47.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013684 s, 76.6 MB/s 2024-04-25T10:28:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137779 s, 76.1 MB/s 2024-04-25T10:28:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141951 s, 73.9 MB/s 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134851 s, 77.8 MB/s 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132008 s, 79.4 MB/s 2024-04-25T10:28:47.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:47.117 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:47.117 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:47.117 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:28:47.118 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:47.118 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:47.118 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:47.118 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134865 s, 77.8 MB/s 2024-04-25T10:28:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138373 s, 75.8 MB/s 2024-04-25T10:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151401 s, 69.3 MB/s 2024-04-25T10:28:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.240 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142293 s, 73.7 MB/s 2024-04-25T10:28:47.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157391 s, 66.6 MB/s 2024-04-25T10:28:47.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143603 s, 73.0 MB/s 2024-04-25T10:28:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.291 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148313 s, 70.7 MB/s 2024-04-25T10:28:47.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.305 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129518 s, 81.0 MB/s 2024-04-25T10:28:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188612 s, 55.6 MB/s 2024-04-25T10:28:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.342 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142489 s, 73.6 MB/s 2024-04-25T10:28:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.363 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184902 s, 56.7 MB/s 2024-04-25T10:28:47.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.378 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132329 s, 79.2 MB/s 2024-04-25T10:28:47.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131303 s, 79.9 MB/s 2024-04-25T10:28:47.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013786 s, 76.1 MB/s 2024-04-25T10:28:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133204 s, 78.7 MB/s 2024-04-25T10:28:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185238 s, 56.6 MB/s 2024-04-25T10:28:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012727 s, 82.4 MB/s 2024-04-25T10:28:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.473 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129274 s, 81.1 MB/s 2024-04-25T10:28:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141757 s, 74.0 MB/s 2024-04-25T10:28:47.508 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.509 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179383 s, 58.5 MB/s 2024-04-25T10:28:47.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119524 s, 87.7 MB/s 2024-04-25T10:28:47.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187614 s, 55.9 MB/s 2024-04-25T10:28:47.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013314 s, 78.8 MB/s 2024-04-25T10:28:47.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.575 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.575 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150636 s, 69.6 MB/s 2024-04-25T10:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.588 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120329 s, 87.1 MB/s 2024-04-25T10:28:47.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.606 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.606 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151124 s, 69.4 MB/s 2024-04-25T10:28:47.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.622 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014765 s, 71.0 MB/s 2024-04-25T10:28:47.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.638 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.638 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144185 s, 72.7 MB/s 2024-04-25T10:28:47.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.653 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.653 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135155 s, 77.6 MB/s 2024-04-25T10:28:47.668 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.669 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.669 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136852 s, 76.6 MB/s 2024-04-25T10:28:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147409 s, 71.1 MB/s 2024-04-25T10:28:47.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.701 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.701 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134994 s, 77.7 MB/s 2024-04-25T10:28:47.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:47.702 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:47.702 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:47.702 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:28:47.702 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:47.703 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:47.703 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:47.703 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139364 s, 75.2 MB/s 2024-04-25T10:28:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.795 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185226 s, 56.6 MB/s 2024-04-25T10:28:47.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.812 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.812 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152022 s, 69.0 MB/s 2024-04-25T10:28:47.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.828 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.828 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013819 s, 75.9 MB/s 2024-04-25T10:28:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149046 s, 70.4 MB/s 2024-04-25T10:28:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182606 s, 57.4 MB/s 2024-04-25T10:28:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155032 s, 67.6 MB/s 2024-04-25T10:28:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189664 s, 55.3 MB/s 2024-04-25T10:28:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130881 s, 80.1 MB/s 2024-04-25T10:28:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141448 s, 74.1 MB/s 2024-04-25T10:28:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134441 s, 78.0 MB/s 2024-04-25T10:28:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01333 s, 78.7 MB/s 2024-04-25T10:28:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151139 s, 69.4 MB/s 2024-04-25T10:28:47.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:47.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:47.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144709 s, 72.5 MB/s 2024-04-25T10:28:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151144 s, 69.4 MB/s 2024-04-25T10:28:48.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.032 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158056 s, 66.3 MB/s 2024-04-25T10:28:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189777 s, 55.3 MB/s 2024-04-25T10:28:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137473 s, 76.3 MB/s 2024-04-25T10:28:48.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.084 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139865 s, 75.0 MB/s 2024-04-25T10:28:48.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.100 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.100 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141531 s, 74.1 MB/s 2024-04-25T10:28:48.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148151 s, 70.8 MB/s 2024-04-25T10:28:48.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187194 s, 56.0 MB/s 2024-04-25T10:28:48.153 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.154 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149854 s, 70.0 MB/s 2024-04-25T10:28:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130926 s, 80.1 MB/s 2024-04-25T10:28:48.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.186 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.186 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152044 s, 69.0 MB/s 2024-04-25T10:28:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153342 s, 68.4 MB/s 2024-04-25T10:28:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152676 s, 68.7 MB/s 2024-04-25T10:28:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147965 s, 70.9 MB/s 2024-04-25T10:28:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135389 s, 77.4 MB/s 2024-04-25T10:28:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132265 s, 79.3 MB/s 2024-04-25T10:28:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014953 s, 70.1 MB/s 2024-04-25T10:28:48.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.300 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.300 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142884 s, 73.4 MB/s 2024-04-25T10:28:48.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:48.302 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.375 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148952 s, 70.4 MB/s 2024-04-25T10:28:48.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.392 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.392 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151694 s, 69.1 MB/s 2024-04-25T10:28:48.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143953 s, 72.8 MB/s 2024-04-25T10:28:48.425 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.425 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.425 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148072 s, 70.8 MB/s 2024-04-25T10:28:48.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.441 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.441 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142606 s, 73.5 MB/s 2024-04-25T10:28:48.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.456 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.457 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137833 s, 76.1 MB/s 2024-04-25T10:28:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133791 s, 78.4 MB/s 2024-04-25T10:28:48.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130139 s, 80.6 MB/s 2024-04-25T10:28:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.503 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.504 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148881 s, 70.4 MB/s 2024-04-25T10:28:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.521 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154668 s, 67.8 MB/s 2024-04-25T10:28:48.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.536 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.536 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129825 s, 80.8 MB/s 2024-04-25T10:28:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150266 s, 69.8 MB/s 2024-04-25T10:28:48.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.572 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.572 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172152 s, 60.9 MB/s 2024-04-25T10:28:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.587 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132656 s, 79.0 MB/s 2024-04-25T10:28:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.603 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148601 s, 70.6 MB/s 2024-04-25T10:28:48.619 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.620 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140079 s, 74.9 MB/s 2024-04-25T10:28:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.636 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147727 s, 71.0 MB/s 2024-04-25T10:28:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.652 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146059 s, 71.8 MB/s 2024-04-25T10:28:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130384 s, 80.4 MB/s 2024-04-25T10:28:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142835 s, 73.4 MB/s 2024-04-25T10:28:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150697 s, 69.6 MB/s 2024-04-25T10:28:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.716 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145437 s, 72.1 MB/s 2024-04-25T10:28:48.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.733 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152504 s, 68.8 MB/s 2024-04-25T10:28:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148276 s, 70.7 MB/s 2024-04-25T10:28:48.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154967 s, 67.7 MB/s 2024-04-25T10:28:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.784 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149674 s, 70.1 MB/s 2024-04-25T10:28:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169677 s, 61.8 MB/s 2024-04-25T10:28:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.821 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149547 s, 70.1 MB/s 2024-04-25T10:28:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133586 s, 78.5 MB/s 2024-04-25T10:28:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.851 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143636 s, 73.0 MB/s 2024-04-25T10:28:48.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.868 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149125 s, 70.3 MB/s 2024-04-25T10:28:48.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.884 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141293 s, 74.2 MB/s 2024-04-25T10:28:48.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:48.886 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:48.886 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:48.886 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:28:48.886 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:48.887 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:48.887 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:48.887 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:48.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.955 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014479 s, 72.4 MB/s 2024-04-25T10:28:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138585 s, 75.7 MB/s 2024-04-25T10:28:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130818 s, 80.2 MB/s 2024-04-25T10:28:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.003 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151604 s, 69.2 MB/s 2024-04-25T10:28:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136993 s, 76.5 MB/s 2024-04-25T10:28:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.039 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186575 s, 56.2 MB/s 2024-04-25T10:28:49.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013761 s, 76.2 MB/s 2024-04-25T10:28:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.071 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147933 s, 70.9 MB/s 2024-04-25T10:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.088 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152162 s, 68.9 MB/s 2024-04-25T10:28:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143326 s, 73.2 MB/s 2024-04-25T10:28:49.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.119 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.119 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132595 s, 79.1 MB/s 2024-04-25T10:28:49.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.134 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.134 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013062 s, 80.3 MB/s 2024-04-25T10:28:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184647 s, 56.8 MB/s 2024-04-25T10:28:49.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.172 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153607 s, 68.3 MB/s 2024-04-25T10:28:49.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.188 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.188 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150333 s, 69.8 MB/s 2024-04-25T10:28:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141161 s, 74.3 MB/s 2024-04-25T10:28:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181853 s, 57.7 MB/s 2024-04-25T10:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150235 s, 69.8 MB/s 2024-04-25T10:28:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150038 s, 69.9 MB/s 2024-04-25T10:28:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143788 s, 72.9 MB/s 2024-04-25T10:28:49.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.290 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138749 s, 75.6 MB/s 2024-04-25T10:28:49.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.306 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146071 s, 71.8 MB/s 2024-04-25T10:28:49.325 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.326 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.326 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178672 s, 58.7 MB/s 2024-04-25T10:28:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.341 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136538 s, 76.8 MB/s 2024-04-25T10:28:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178543 s, 58.7 MB/s 2024-04-25T10:28:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.377 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146236 s, 71.7 MB/s 2024-04-25T10:28:49.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146474 s, 71.6 MB/s 2024-04-25T10:28:49.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017552 s, 59.7 MB/s 2024-04-25T10:28:49.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.428 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139499 s, 75.2 MB/s 2024-04-25T10:28:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.445 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148389 s, 70.7 MB/s 2024-04-25T10:28:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.461 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143623 s, 73.0 MB/s 2024-04-25T10:28:49.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.476 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136729 s, 76.7 MB/s 2024-04-25T10:28:49.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:49.478 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:49.478 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:49.478 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:28:49.478 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:49.478 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:49.479 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:49.479 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014514 s, 72.2 MB/s 2024-04-25T10:28:49.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.563 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135302 s, 77.5 MB/s 2024-04-25T10:28:49.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.578 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.578 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133363 s, 78.6 MB/s 2024-04-25T10:28:49.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148674 s, 70.5 MB/s 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.612 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159228 s, 65.9 MB/s 2024-04-25T10:28:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134264 s, 78.1 MB/s 2024-04-25T10:28:49.642 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130547 s, 80.3 MB/s 2024-04-25T10:28:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140024 s, 74.9 MB/s 2024-04-25T10:28:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.675 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143563 s, 73.0 MB/s 2024-04-25T10:28:49.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138566 s, 75.7 MB/s 2024-04-25T10:28:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132132 s, 79.4 MB/s 2024-04-25T10:28:49.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139789 s, 75.0 MB/s 2024-04-25T10:28:49.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156629 s, 66.9 MB/s 2024-04-25T10:28:49.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.753 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.754 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132696 s, 79.0 MB/s 2024-04-25T10:28:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.769 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140751 s, 74.5 MB/s 2024-04-25T10:28:49.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.786 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152397 s, 68.8 MB/s 2024-04-25T10:28:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138546 s, 75.7 MB/s 2024-04-25T10:28:49.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152987 s, 68.5 MB/s 2024-04-25T10:28:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147505 s, 71.1 MB/s 2024-04-25T10:28:49.851 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.852 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139241 s, 75.3 MB/s 2024-04-25T10:28:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012142 s, 86.4 MB/s 2024-04-25T10:28:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.882 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151287 s, 69.3 MB/s 2024-04-25T10:28:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152717 s, 68.7 MB/s 2024-04-25T10:28:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142817 s, 73.4 MB/s 2024-04-25T10:28:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015896 s, 66.0 MB/s 2024-04-25T10:28:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129188 s, 81.2 MB/s 2024-04-25T10:28:49.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.964 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138216 s, 75.9 MB/s 2024-04-25T10:28:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.978 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129764 s, 80.8 MB/s 2024-04-25T10:28:49.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:49.995 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:49.995 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147394 s, 71.1 MB/s 2024-04-25T10:28:50.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.013 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158845 s, 66.0 MB/s 2024-04-25T10:28:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.029 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141527 s, 74.1 MB/s 2024-04-25T10:28:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.045 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139049 s, 75.4 MB/s 2024-04-25T10:28:50.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:50.047 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145546 s, 72.0 MB/s 2024-04-25T10:28:50.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.131 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014048 s, 74.6 MB/s 2024-04-25T10:28:50.147 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.148 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150699 s, 69.6 MB/s 2024-04-25T10:28:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188717 s, 55.6 MB/s 2024-04-25T10:28:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137024 s, 76.5 MB/s 2024-04-25T10:28:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178606 s, 58.7 MB/s 2024-04-25T10:28:50.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146894 s, 71.4 MB/s 2024-04-25T10:28:50.235 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.236 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137823 s, 76.1 MB/s 2024-04-25T10:28:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.251 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134547 s, 77.9 MB/s 2024-04-25T10:28:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.267 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147935 s, 70.9 MB/s 2024-04-25T10:28:50.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.284 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144651 s, 72.5 MB/s 2024-04-25T10:28:50.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013859 s, 75.7 MB/s 2024-04-25T10:28:50.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.315 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.315 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139052 s, 75.4 MB/s 2024-04-25T10:28:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190077 s, 55.2 MB/s 2024-04-25T10:28:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147573 s, 71.1 MB/s 2024-04-25T10:28:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.372 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017448 s, 60.1 MB/s 2024-04-25T10:28:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173367 s, 60.5 MB/s 2024-04-25T10:28:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135431 s, 77.4 MB/s 2024-04-25T10:28:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.421 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127528 s, 82.2 MB/s 2024-04-25T10:28:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.438 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150818 s, 69.5 MB/s 2024-04-25T10:28:50.454 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.455 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147535 s, 71.1 MB/s 2024-04-25T10:28:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177777 s, 59.0 MB/s 2024-04-25T10:28:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.490 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141496 s, 74.1 MB/s 2024-04-25T10:28:50.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.507 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150252 s, 69.8 MB/s 2024-04-25T10:28:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130323 s, 80.5 MB/s 2024-04-25T10:28:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.538 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145828 s, 71.9 MB/s 2024-04-25T10:28:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.553 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132497 s, 79.1 MB/s 2024-04-25T10:28:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140509 s, 74.6 MB/s 2024-04-25T10:28:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152589 s, 68.7 MB/s 2024-04-25T10:28:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013646 s, 76.8 MB/s 2024-04-25T10:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147332 s, 71.2 MB/s 2024-04-25T10:28:50.633 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.634 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135041 s, 77.6 MB/s 2024-04-25T10:28:50.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:50.635 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:50.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.707 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147209 s, 71.2 MB/s 2024-04-25T10:28:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.723 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132443 s, 79.2 MB/s 2024-04-25T10:28:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152647 s, 68.7 MB/s 2024-04-25T10:28:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.755 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014064 s, 74.6 MB/s 2024-04-25T10:28:50.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.771 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142396 s, 73.6 MB/s 2024-04-25T10:28:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148532 s, 70.6 MB/s 2024-04-25T10:28:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146288 s, 71.7 MB/s 2024-04-25T10:28:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140161 s, 74.8 MB/s 2024-04-25T10:28:50.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151423 s, 69.2 MB/s 2024-04-25T10:28:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138441 s, 75.7 MB/s 2024-04-25T10:28:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.867 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134067 s, 78.2 MB/s 2024-04-25T10:28:50.884 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.885 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153552 s, 68.3 MB/s 2024-04-25T10:28:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.901 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139617 s, 75.1 MB/s 2024-04-25T10:28:50.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014928 s, 70.2 MB/s 2024-04-25T10:28:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181221 s, 57.9 MB/s 2024-04-25T10:28:50.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.952 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132978 s, 78.9 MB/s 2024-04-25T10:28:50.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.967 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.967 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133632 s, 78.5 MB/s 2024-04-25T10:28:50.982 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134723 s, 77.8 MB/s 2024-04-25T10:28:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129612 s, 80.9 MB/s 2024-04-25T10:28:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133918 s, 78.3 MB/s 2024-04-25T10:28:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.030 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150574 s, 69.6 MB/s 2024-04-25T10:28:51.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.045 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130131 s, 80.6 MB/s 2024-04-25T10:28:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148661 s, 70.5 MB/s 2024-04-25T10:28:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.077 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.078 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147232 s, 71.2 MB/s 2024-04-25T10:28:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.099 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198515 s, 52.8 MB/s 2024-04-25T10:28:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148678 s, 70.5 MB/s 2024-04-25T10:28:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014388 s, 72.9 MB/s 2024-04-25T10:28:51.152 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.152 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.152 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183868 s, 57.0 MB/s 2024-04-25T10:28:51.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.167 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013619 s, 77.0 MB/s 2024-04-25T10:28:51.182 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.183 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135012 s, 77.7 MB/s 2024-04-25T10:28:51.199 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.199 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.199 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148474 s, 70.6 MB/s 2024-04-25T10:28:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.216 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149889 s, 70.0 MB/s 2024-04-25T10:28:51.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:51.218 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.286 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013191 s, 79.5 MB/s 2024-04-25T10:28:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152566 s, 68.7 MB/s 2024-04-25T10:28:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132751 s, 79.0 MB/s 2024-04-25T10:28:51.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.335 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152445 s, 68.8 MB/s 2024-04-25T10:28:51.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.352 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.352 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154172 s, 68.0 MB/s 2024-04-25T10:28:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.369 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150122 s, 69.8 MB/s 2024-04-25T10:28:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.383 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.384 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123139 s, 85.2 MB/s 2024-04-25T10:28:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150842 s, 69.5 MB/s 2024-04-25T10:28:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145697 s, 72.0 MB/s 2024-04-25T10:28:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01491 s, 70.3 MB/s 2024-04-25T10:28:51.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131177 s, 79.9 MB/s 2024-04-25T10:28:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.465 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148814 s, 70.5 MB/s 2024-04-25T10:28:51.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.480 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.481 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134944 s, 77.7 MB/s 2024-04-25T10:28:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131793 s, 79.6 MB/s 2024-04-25T10:28:51.511 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.512 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145958 s, 71.8 MB/s 2024-04-25T10:28:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.527 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01357 s, 77.3 MB/s 2024-04-25T10:28:51.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.542 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132032 s, 79.4 MB/s 2024-04-25T10:28:51.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142505 s, 73.6 MB/s 2024-04-25T10:28:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144114 s, 72.8 MB/s 2024-04-25T10:28:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149313 s, 70.2 MB/s 2024-04-25T10:28:51.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.607 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143365 s, 73.1 MB/s 2024-04-25T10:28:51.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149305 s, 70.2 MB/s 2024-04-25T10:28:51.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147439 s, 71.1 MB/s 2024-04-25T10:28:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014201 s, 73.8 MB/s 2024-04-25T10:28:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151645 s, 69.1 MB/s 2024-04-25T10:28:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148885 s, 70.4 MB/s 2024-04-25T10:28:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.707 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156862 s, 66.8 MB/s 2024-04-25T10:28:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140191 s, 74.8 MB/s 2024-04-25T10:28:51.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.738 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.739 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136702 s, 76.7 MB/s 2024-04-25T10:28:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143002 s, 73.3 MB/s 2024-04-25T10:28:51.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.771 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.772 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149188 s, 70.3 MB/s 2024-04-25T10:28:51.786 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133484 s, 78.6 MB/s 2024-04-25T10:28:51.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:51.788 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:51.788 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:51.788 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:28:51.789 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:51.789 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:51.789 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:51.789 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173571 s, 60.4 MB/s 2024-04-25T10:28:51.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131911 s, 79.5 MB/s 2024-04-25T10:28:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143195 s, 73.2 MB/s 2024-04-25T10:28:51.906 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.907 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.907 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118082 s, 88.8 MB/s 2024-04-25T10:28:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172242 s, 60.9 MB/s 2024-04-25T10:28:51.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.943 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.943 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152487 s, 68.8 MB/s 2024-04-25T10:28:51.959 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.959 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.959 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149581 s, 70.1 MB/s 2024-04-25T10:28:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129938 s, 80.7 MB/s 2024-04-25T10:28:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:51.989 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126797 s, 82.7 MB/s 2024-04-25T10:28:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149878 s, 70.0 MB/s 2024-04-25T10:28:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136888 s, 76.6 MB/s 2024-04-25T10:28:52.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.037 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.037 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144141 s, 72.7 MB/s 2024-04-25T10:28:52.053 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.054 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142324 s, 73.7 MB/s 2024-04-25T10:28:52.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179316 s, 58.5 MB/s 2024-04-25T10:28:52.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156035 s, 67.2 MB/s 2024-04-25T10:28:52.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143835 s, 72.9 MB/s 2024-04-25T10:28:52.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132618 s, 79.1 MB/s 2024-04-25T10:28:52.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.142 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.142 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176571 s, 59.4 MB/s 2024-04-25T10:28:52.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.159 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152359 s, 68.8 MB/s 2024-04-25T10:28:52.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.179 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189808 s, 55.2 MB/s 2024-04-25T10:28:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119482 s, 87.8 MB/s 2024-04-25T10:28:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155119 s, 67.6 MB/s 2024-04-25T10:28:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.228 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015495 s, 67.7 MB/s 2024-04-25T10:28:52.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.245 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.245 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152774 s, 68.6 MB/s 2024-04-25T10:28:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.270 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.271 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231392 s, 45.3 MB/s 2024-04-25T10:28:52.294 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221549 s, 47.3 MB/s 2024-04-25T10:28:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.311 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147841 s, 70.9 MB/s 2024-04-25T10:28:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137212 s, 76.4 MB/s 2024-04-25T10:28:52.342 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134314 s, 78.1 MB/s 2024-04-25T10:28:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133635 s, 78.5 MB/s 2024-04-25T10:28:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141838 s, 73.9 MB/s 2024-04-25T10:28:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.389 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013405 s, 78.2 MB/s 2024-04-25T10:28:52.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:52.390 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:52.390 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:52.391 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:28:52.391 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:52.391 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:52.391 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:52.391 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:52.461 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.462 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128421 s, 81.7 MB/s 2024-04-25T10:28:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.479 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159357 s, 65.8 MB/s 2024-04-25T10:28:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129517 s, 81.0 MB/s 2024-04-25T10:28:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.509 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134486 s, 78.0 MB/s 2024-04-25T10:28:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.526 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.527 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151755 s, 69.1 MB/s 2024-04-25T10:28:52.542 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143118 s, 73.3 MB/s 2024-04-25T10:28:52.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137103 s, 76.5 MB/s 2024-04-25T10:28:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.575 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145418 s, 72.1 MB/s 2024-04-25T10:28:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.594 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177062 s, 59.2 MB/s 2024-04-25T10:28:52.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.610 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138325 s, 75.8 MB/s 2024-04-25T10:28:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125251 s, 83.7 MB/s 2024-04-25T10:28:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.640 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013612 s, 77.0 MB/s 2024-04-25T10:28:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.655 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133799 s, 78.4 MB/s 2024-04-25T10:28:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137282 s, 76.4 MB/s 2024-04-25T10:28:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128877 s, 81.4 MB/s 2024-04-25T10:28:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116192 s, 90.2 MB/s 2024-04-25T10:28:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137006 s, 76.5 MB/s 2024-04-25T10:28:52.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.730 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139245 s, 75.3 MB/s 2024-04-25T10:28:52.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.746 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141027 s, 74.4 MB/s 2024-04-25T10:28:52.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156488 s, 67.0 MB/s 2024-04-25T10:28:52.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.780 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.781 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156352 s, 67.1 MB/s 2024-04-25T10:28:52.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.796 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.796 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138337 s, 75.8 MB/s 2024-04-25T10:28:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.811 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136174 s, 77.0 MB/s 2024-04-25T10:28:52.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013642 s, 76.9 MB/s 2024-04-25T10:28:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014535 s, 72.1 MB/s 2024-04-25T10:28:52.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.860 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.860 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150359 s, 69.7 MB/s 2024-04-25T10:28:52.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.877 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.877 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153773 s, 68.2 MB/s 2024-04-25T10:28:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135063 s, 77.6 MB/s 2024-04-25T10:28:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142802 s, 73.4 MB/s 2024-04-25T10:28:52.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.925 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141768 s, 74.0 MB/s 2024-04-25T10:28:52.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.945 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184367 s, 56.9 MB/s 2024-04-25T10:28:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:52.962 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146604 s, 71.5 MB/s 2024-04-25T10:28:52.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:52.963 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.036 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149269 s, 70.2 MB/s 2024-04-25T10:28:53.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143853 s, 72.9 MB/s 2024-04-25T10:28:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.069 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147573 s, 71.1 MB/s 2024-04-25T10:28:53.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.084 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139692 s, 75.1 MB/s 2024-04-25T10:28:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154466 s, 67.9 MB/s 2024-04-25T10:28:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130437 s, 80.4 MB/s 2024-04-25T10:28:53.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130869 s, 80.1 MB/s 2024-04-25T10:28:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015112 s, 69.4 MB/s 2024-04-25T10:28:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.164 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142854 s, 73.4 MB/s 2024-04-25T10:28:53.182 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.183 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.183 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164272 s, 63.8 MB/s 2024-04-25T10:28:53.197 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.198 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131753 s, 79.6 MB/s 2024-04-25T10:28:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138363 s, 75.8 MB/s 2024-04-25T10:28:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134773 s, 77.8 MB/s 2024-04-25T10:28:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.249 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180167 s, 58.2 MB/s 2024-04-25T10:28:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144238 s, 72.7 MB/s 2024-04-25T10:28:53.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.280 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136201 s, 77.0 MB/s 2024-04-25T10:28:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130287 s, 80.5 MB/s 2024-04-25T10:28:53.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154908 s, 67.7 MB/s 2024-04-25T10:28:53.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148995 s, 70.4 MB/s 2024-04-25T10:28:53.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.345 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.345 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133709 s, 78.4 MB/s 2024-04-25T10:28:53.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182918 s, 57.3 MB/s 2024-04-25T10:28:53.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.382 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152294 s, 68.9 MB/s 2024-04-25T10:28:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.398 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145959 s, 71.8 MB/s 2024-04-25T10:28:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.414 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148393 s, 70.7 MB/s 2024-04-25T10:28:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149296 s, 70.2 MB/s 2024-04-25T10:28:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138807 s, 75.5 MB/s 2024-04-25T10:28:53.467 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.468 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189737 s, 55.3 MB/s 2024-04-25T10:28:53.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.484 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150118 s, 69.9 MB/s 2024-04-25T10:28:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.500 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136516 s, 76.8 MB/s 2024-04-25T10:28:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.517 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149034 s, 70.4 MB/s 2024-04-25T10:28:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.535 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168743 s, 62.1 MB/s 2024-04-25T10:28:53.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.556 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.556 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185316 s, 56.6 MB/s 2024-04-25T10:28:53.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:53.557 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:53.557 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:53.558 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:28:53.558 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:53.558 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:53.558 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:53.558 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134852 s, 77.8 MB/s 2024-04-25T10:28:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013216 s, 79.3 MB/s 2024-04-25T10:28:53.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01879 s, 55.8 MB/s 2024-04-25T10:28:53.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.682 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.682 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186265 s, 56.3 MB/s 2024-04-25T10:28:53.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.697 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137204 s, 76.4 MB/s 2024-04-25T10:28:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162311 s, 64.6 MB/s 2024-04-25T10:28:53.731 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.732 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.732 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141948 s, 73.9 MB/s 2024-04-25T10:28:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.751 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018009 s, 58.2 MB/s 2024-04-25T10:28:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135676 s, 77.3 MB/s 2024-04-25T10:28:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.787 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183496 s, 57.1 MB/s 2024-04-25T10:28:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133381 s, 78.6 MB/s 2024-04-25T10:28:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143289 s, 73.2 MB/s 2024-04-25T10:28:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157008 s, 66.8 MB/s 2024-04-25T10:28:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019005 s, 55.2 MB/s 2024-04-25T10:28:53.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.872 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138257 s, 75.8 MB/s 2024-04-25T10:28:53.888 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147296 s, 71.2 MB/s 2024-04-25T10:28:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013059 s, 80.3 MB/s 2024-04-25T10:28:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182683 s, 57.4 MB/s 2024-04-25T10:28:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142213 s, 73.7 MB/s 2024-04-25T10:28:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127304 s, 82.4 MB/s 2024-04-25T10:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137489 s, 76.3 MB/s 2024-04-25T10:28:53.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:53.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:53.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149659 s, 70.1 MB/s 2024-04-25T10:28:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188424 s, 55.6 MB/s 2024-04-25T10:28:54.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.023 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.023 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137233 s, 76.4 MB/s 2024-04-25T10:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.043 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186787 s, 56.1 MB/s 2024-04-25T10:28:54.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.059 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013888 s, 75.5 MB/s 2024-04-25T10:28:54.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.074 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.074 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137446 s, 76.3 MB/s 2024-04-25T10:28:54.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145481 s, 72.1 MB/s 2024-04-25T10:28:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.106 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134285 s, 78.1 MB/s 2024-04-25T10:28:54.121 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135771 s, 77.2 MB/s 2024-04-25T10:28:54.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.139 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.139 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150065 s, 69.9 MB/s 2024-04-25T10:28:54.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.156 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.156 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152208 s, 68.9 MB/s 2024-04-25T10:28:54.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:54.157 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.230 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173702 s, 60.4 MB/s 2024-04-25T10:28:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150129 s, 69.8 MB/s 2024-04-25T10:28:54.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.263 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.263 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143884 s, 72.9 MB/s 2024-04-25T10:28:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136315 s, 76.9 MB/s 2024-04-25T10:28:54.295 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.296 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.296 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015449 s, 67.9 MB/s 2024-04-25T10:28:54.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.313 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.313 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153723 s, 68.2 MB/s 2024-04-25T10:28:54.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145226 s, 72.2 MB/s 2024-04-25T10:28:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131479 s, 79.8 MB/s 2024-04-25T10:28:54.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.364 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.364 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177637 s, 59.0 MB/s 2024-04-25T10:28:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135596 s, 77.3 MB/s 2024-04-25T10:28:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134498 s, 78.0 MB/s 2024-04-25T10:28:54.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.409 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.409 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134561 s, 77.9 MB/s 2024-04-25T10:28:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.424 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127843 s, 82.0 MB/s 2024-04-25T10:28:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.445 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185605 s, 56.5 MB/s 2024-04-25T10:28:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138567 s, 75.7 MB/s 2024-04-25T10:28:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.477 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148924 s, 70.4 MB/s 2024-04-25T10:28:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183106 s, 57.3 MB/s 2024-04-25T10:28:54.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.514 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153203 s, 68.4 MB/s 2024-04-25T10:28:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.530 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135962 s, 77.1 MB/s 2024-04-25T10:28:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141584 s, 74.1 MB/s 2024-04-25T10:28:54.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.561 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137833 s, 76.1 MB/s 2024-04-25T10:28:54.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.578 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140524 s, 74.6 MB/s 2024-04-25T10:28:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181563 s, 57.8 MB/s 2024-04-25T10:28:54.613 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.613 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.613 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142876 s, 73.4 MB/s 2024-04-25T10:28:54.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152413 s, 68.8 MB/s 2024-04-25T10:28:54.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.651 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.651 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186937 s, 56.1 MB/s 2024-04-25T10:28:54.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.666 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.666 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137184 s, 76.4 MB/s 2024-04-25T10:28:54.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.683 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153104 s, 68.5 MB/s 2024-04-25T10:28:54.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.699 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.699 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014217 s, 73.8 MB/s 2024-04-25T10:28:54.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.714 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.715 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132734 s, 79.0 MB/s 2024-04-25T10:28:54.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.730 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.730 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136317 s, 76.9 MB/s 2024-04-25T10:28:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.749 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174532 s, 60.1 MB/s 2024-04-25T10:28:54.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:54.751 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:54.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.818 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.818 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013713 s, 76.5 MB/s 2024-04-25T10:28:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160579 s, 65.3 MB/s 2024-04-25T10:28:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.853 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145601 s, 72.0 MB/s 2024-04-25T10:28:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143051 s, 73.3 MB/s 2024-04-25T10:28:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.885 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014686 s, 71.4 MB/s 2024-04-25T10:28:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135073 s, 77.6 MB/s 2024-04-25T10:28:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143949 s, 72.8 MB/s 2024-04-25T10:28:54.936 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.936 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.936 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175469 s, 59.8 MB/s 2024-04-25T10:28:54.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.952 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.952 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137747 s, 76.1 MB/s 2024-04-25T10:28:54.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.966 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.967 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01318 s, 79.6 MB/s 2024-04-25T10:28:54.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.983 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.983 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144007 s, 72.8 MB/s 2024-04-25T10:28:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134029 s, 78.2 MB/s 2024-04-25T10:28:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147688 s, 71.0 MB/s 2024-04-25T10:28:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157433 s, 66.6 MB/s 2024-04-25T10:28:55.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.047 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132529 s, 79.1 MB/s 2024-04-25T10:28:55.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.064 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144554 s, 72.5 MB/s 2024-04-25T10:28:55.083 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.084 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.084 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179807 s, 58.3 MB/s 2024-04-25T10:28:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015302 s, 68.5 MB/s 2024-04-25T10:28:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134671 s, 77.9 MB/s 2024-04-25T10:28:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128406 s, 81.7 MB/s 2024-04-25T10:28:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185742 s, 56.5 MB/s 2024-04-25T10:28:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139082 s, 75.4 MB/s 2024-04-25T10:28:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155643 s, 67.4 MB/s 2024-04-25T10:28:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165938 s, 63.2 MB/s 2024-04-25T10:28:55.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.223 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.223 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186936 s, 56.1 MB/s 2024-04-25T10:28:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.239 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148374 s, 70.7 MB/s 2024-04-25T10:28:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018255 s, 57.4 MB/s 2024-04-25T10:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136382 s, 76.9 MB/s 2024-04-25T10:28:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.291 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137476 s, 76.3 MB/s 2024-04-25T10:28:55.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.308 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.308 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152678 s, 68.7 MB/s 2024-04-25T10:28:55.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.327 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.327 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175919 s, 59.6 MB/s 2024-04-25T10:28:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154033 s, 68.1 MB/s 2024-04-25T10:28:55.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:55.348 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130947 s, 80.1 MB/s 2024-04-25T10:28:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184769 s, 56.8 MB/s 2024-04-25T10:28:55.457 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.457 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.457 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130815 s, 80.2 MB/s 2024-04-25T10:28:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128371 s, 81.7 MB/s 2024-04-25T10:28:55.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.487 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130884 s, 80.1 MB/s 2024-04-25T10:28:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130967 s, 80.1 MB/s 2024-04-25T10:28:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.522 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182248 s, 57.5 MB/s 2024-04-25T10:28:55.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.537 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.537 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134703 s, 77.8 MB/s 2024-04-25T10:28:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133703 s, 78.4 MB/s 2024-04-25T10:28:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150489 s, 69.7 MB/s 2024-04-25T10:28:55.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.585 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.585 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141638 s, 74.0 MB/s 2024-04-25T10:28:55.600 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.600 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.600 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133486 s, 78.6 MB/s 2024-04-25T10:28:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.615 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135495 s, 77.4 MB/s 2024-04-25T10:28:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129141 s, 81.2 MB/s 2024-04-25T10:28:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185604 s, 56.5 MB/s 2024-04-25T10:28:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014429 s, 72.7 MB/s 2024-04-25T10:28:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167805 s, 62.5 MB/s 2024-04-25T10:28:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181699 s, 57.7 MB/s 2024-04-25T10:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142124 s, 73.8 MB/s 2024-04-25T10:28:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194304 s, 54.0 MB/s 2024-04-25T10:28:55.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.758 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.758 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135879 s, 77.2 MB/s 2024-04-25T10:28:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014146 s, 74.1 MB/s 2024-04-25T10:28:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149383 s, 70.2 MB/s 2024-04-25T10:28:55.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.806 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.806 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130626 s, 80.3 MB/s 2024-04-25T10:28:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136497 s, 76.8 MB/s 2024-04-25T10:28:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137611 s, 76.2 MB/s 2024-04-25T10:28:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133711 s, 78.4 MB/s 2024-04-25T10:28:55.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.868 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141031 s, 74.4 MB/s 2024-04-25T10:28:55.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.883 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.883 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132084 s, 79.4 MB/s 2024-04-25T10:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.900 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148909 s, 70.4 MB/s 2024-04-25T10:28:55.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.917 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147435 s, 71.1 MB/s 2024-04-25T10:28:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:55.934 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148955 s, 70.4 MB/s 2024-04-25T10:28:55.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:55.935 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:55.935 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:55.935 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:28:55.936 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:55.936 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:55.936 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:55.936 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:56.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.012 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.012 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188013 s, 55.8 MB/s 2024-04-25T10:28:56.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.027 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.027 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133796 s, 78.4 MB/s 2024-04-25T10:28:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167505 s, 62.6 MB/s 2024-04-25T10:28:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135628 s, 77.3 MB/s 2024-04-25T10:28:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.082 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185827 s, 56.4 MB/s 2024-04-25T10:28:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.098 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01488 s, 70.5 MB/s 2024-04-25T10:28:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149016 s, 70.4 MB/s 2024-04-25T10:28:56.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.132 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146056 s, 71.8 MB/s 2024-04-25T10:28:56.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.148 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.148 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149809 s, 70.0 MB/s 2024-04-25T10:28:56.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.164 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.164 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134357 s, 78.0 MB/s 2024-04-25T10:28:56.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.179 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.179 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138116 s, 75.9 MB/s 2024-04-25T10:28:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151356 s, 69.3 MB/s 2024-04-25T10:28:56.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.212 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.212 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137991 s, 76.0 MB/s 2024-04-25T10:28:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149601 s, 70.1 MB/s 2024-04-25T10:28:56.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.244 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.244 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135676 s, 77.3 MB/s 2024-04-25T10:28:56.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.260 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.260 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142081 s, 73.8 MB/s 2024-04-25T10:28:56.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.275 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.275 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138447 s, 75.7 MB/s 2024-04-25T10:28:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.291 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135144 s, 77.6 MB/s 2024-04-25T10:28:56.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132189 s, 79.3 MB/s 2024-04-25T10:28:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.322 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142039 s, 73.8 MB/s 2024-04-25T10:28:56.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.337 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133756 s, 78.4 MB/s 2024-04-25T10:28:56.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.357 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186301 s, 56.3 MB/s 2024-04-25T10:28:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.375 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154659 s, 67.8 MB/s 2024-04-25T10:28:56.395 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.395 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.395 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187302 s, 56.0 MB/s 2024-04-25T10:28:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153353 s, 68.4 MB/s 2024-04-25T10:28:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.429 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014767 s, 71.0 MB/s 2024-04-25T10:28:56.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.444 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.444 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133451 s, 78.6 MB/s 2024-04-25T10:28:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157957 s, 66.4 MB/s 2024-04-25T10:28:56.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.482 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.482 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185881 s, 56.4 MB/s 2024-04-25T10:28:56.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.497 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.497 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131429 s, 79.8 MB/s 2024-04-25T10:28:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133131 s, 78.8 MB/s 2024-04-25T10:28:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166472 s, 63.0 MB/s 2024-04-25T10:28:56.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:56.532 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:56.532 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:56.533 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:28:56.533 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:56.533 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:56.533 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:56.533 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:56.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015186 s, 69.0 MB/s 2024-04-25T10:28:56.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.624 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.624 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014248 s, 73.6 MB/s 2024-04-25T10:28:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131582 s, 79.7 MB/s 2024-04-25T10:28:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.655 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143052 s, 73.3 MB/s 2024-04-25T10:28:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.671 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142458 s, 73.6 MB/s 2024-04-25T10:28:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146423 s, 71.6 MB/s 2024-04-25T10:28:56.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.702 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134388 s, 78.0 MB/s 2024-04-25T10:28:56.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135194 s, 77.6 MB/s 2024-04-25T10:28:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134451 s, 78.0 MB/s 2024-04-25T10:28:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.754 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190996 s, 54.9 MB/s 2024-04-25T10:28:56.772 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.773 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.773 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168231 s, 62.3 MB/s 2024-04-25T10:28:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.789 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141399 s, 74.2 MB/s 2024-04-25T10:28:56.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.808 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.808 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176031 s, 59.6 MB/s 2024-04-25T10:28:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.824 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.825 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147655 s, 71.0 MB/s 2024-04-25T10:28:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126219 s, 83.1 MB/s 2024-04-25T10:28:56.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.854 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.854 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135013 s, 77.7 MB/s 2024-04-25T10:28:56.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.871 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.871 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154369 s, 67.9 MB/s 2024-04-25T10:28:56.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.887 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.887 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134177 s, 78.1 MB/s 2024-04-25T10:28:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013701 s, 76.5 MB/s 2024-04-25T10:28:56.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.919 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.919 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154347 s, 67.9 MB/s 2024-04-25T10:28:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.940 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187407 s, 56.0 MB/s 2024-04-25T10:28:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141365 s, 74.2 MB/s 2024-04-25T10:28:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.971 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134963 s, 77.7 MB/s 2024-04-25T10:28:56.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:56.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:56.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133746 s, 78.4 MB/s 2024-04-25T10:28:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.006 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177158 s, 59.2 MB/s 2024-04-25T10:28:57.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.020 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.020 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128375 s, 81.7 MB/s 2024-04-25T10:28:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185309 s, 56.6 MB/s 2024-04-25T10:28:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130782 s, 80.2 MB/s 2024-04-25T10:28:57.071 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.072 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.072 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139571 s, 75.1 MB/s 2024-04-25T10:28:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125773 s, 83.4 MB/s 2024-04-25T10:28:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.101 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135369 s, 77.5 MB/s 2024-04-25T10:28:57.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.118 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147329 s, 71.2 MB/s 2024-04-25T10:28:57.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:57.119 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:57.120 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:57.120 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:28:57.120 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:57.120 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:57.120 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:57.120 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129663 s, 80.9 MB/s 2024-04-25T10:28:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.205 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136045 s, 77.1 MB/s 2024-04-25T10:28:57.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.226 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.226 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185552 s, 56.5 MB/s 2024-04-25T10:28:57.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239117 s, 43.9 MB/s 2024-04-25T10:28:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213217 s, 49.2 MB/s 2024-04-25T10:28:57.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013799 s, 76.0 MB/s 2024-04-25T10:28:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181179 s, 57.9 MB/s 2024-04-25T10:28:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149382 s, 70.2 MB/s 2024-04-25T10:28:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131506 s, 79.7 MB/s 2024-04-25T10:28:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152219 s, 68.9 MB/s 2024-04-25T10:28:57.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.380 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.380 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179569 s, 58.4 MB/s 2024-04-25T10:28:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.396 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137948 s, 76.0 MB/s 2024-04-25T10:28:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185601 s, 56.5 MB/s 2024-04-25T10:28:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.432 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148052 s, 70.8 MB/s 2024-04-25T10:28:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.449 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142758 s, 73.5 MB/s 2024-04-25T10:28:57.464 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.464 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.464 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132371 s, 79.2 MB/s 2024-04-25T10:28:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.479 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132982 s, 78.9 MB/s 2024-04-25T10:28:57.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.494 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140763 s, 74.5 MB/s 2024-04-25T10:28:57.511 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.512 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.512 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153709 s, 68.2 MB/s 2024-04-25T10:28:57.532 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.533 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019253 s, 54.5 MB/s 2024-04-25T10:28:57.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.552 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172898 s, 60.6 MB/s 2024-04-25T10:28:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.569 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149505 s, 70.1 MB/s 2024-04-25T10:28:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131168 s, 79.9 MB/s 2024-04-25T10:28:57.600 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.601 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.601 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151341 s, 69.3 MB/s 2024-04-25T10:28:57.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.616 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134352 s, 78.0 MB/s 2024-04-25T10:28:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140092 s, 74.8 MB/s 2024-04-25T10:28:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.645 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118851 s, 88.2 MB/s 2024-04-25T10:28:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139732 s, 75.0 MB/s 2024-04-25T10:28:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137071 s, 76.5 MB/s 2024-04-25T10:28:57.696 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184401 s, 56.9 MB/s 2024-04-25T10:28:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132198 s, 79.3 MB/s 2024-04-25T10:28:57.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143197 s, 73.2 MB/s 2024-04-25T10:28:57.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:57.729 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:57.729 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:57.729 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:28:57.729 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:57.729 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:57.729 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:57.730 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.803 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145899 s, 71.9 MB/s 2024-04-25T10:28:57.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.820 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.820 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147425 s, 71.1 MB/s 2024-04-25T10:28:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.837 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016045 s, 65.4 MB/s 2024-04-25T10:28:57.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.855 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.855 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157301 s, 66.7 MB/s 2024-04-25T10:28:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.870 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131327 s, 79.8 MB/s 2024-04-25T10:28:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.885 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.886 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136179 s, 77.0 MB/s 2024-04-25T10:28:57.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.901 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136058 s, 77.1 MB/s 2024-04-25T10:28:57.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.915 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.915 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013051 s, 80.3 MB/s 2024-04-25T10:28:57.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.930 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.930 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128028 s, 81.9 MB/s 2024-04-25T10:28:57.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.945 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.945 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132662 s, 79.0 MB/s 2024-04-25T10:28:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.963 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.963 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156626 s, 66.9 MB/s 2024-04-25T10:28:57.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.979 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.979 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144234 s, 72.7 MB/s 2024-04-25T10:28:57.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:57.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:57.994 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134273 s, 78.1 MB/s 2024-04-25T10:28:58.010 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.010 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.011 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141805 s, 73.9 MB/s 2024-04-25T10:28:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.026 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013687 s, 76.6 MB/s 2024-04-25T10:28:58.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.042 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138207 s, 75.9 MB/s 2024-04-25T10:28:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.058 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.059 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152688 s, 68.7 MB/s 2024-04-25T10:28:58.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.073 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.073 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130122 s, 80.6 MB/s 2024-04-25T10:28:58.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.089 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.089 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141749 s, 74.0 MB/s 2024-04-25T10:28:58.106 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150039 s, 69.9 MB/s 2024-04-25T10:28:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150175 s, 69.8 MB/s 2024-04-25T10:28:58.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.140 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.140 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149104 s, 70.3 MB/s 2024-04-25T10:28:58.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134373 s, 78.0 MB/s 2024-04-25T10:28:58.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135259 s, 77.5 MB/s 2024-04-25T10:28:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.187 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144219 s, 72.7 MB/s 2024-04-25T10:28:58.203 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.204 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149659 s, 70.1 MB/s 2024-04-25T10:28:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144303 s, 72.7 MB/s 2024-04-25T10:28:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146372 s, 71.6 MB/s 2024-04-25T10:28:58.251 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.252 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135412 s, 77.4 MB/s 2024-04-25T10:28:58.268 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.269 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.269 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147882 s, 70.9 MB/s 2024-04-25T10:28:58.283 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.284 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133668 s, 78.4 MB/s 2024-04-25T10:28:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.303 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177195 s, 59.2 MB/s 2024-04-25T10:28:58.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:58.305 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:58.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013271 s, 79.0 MB/s 2024-04-25T10:28:58.390 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.391 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148064 s, 70.8 MB/s 2024-04-25T10:28:58.410 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.411 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.411 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182488 s, 57.5 MB/s 2024-04-25T10:28:58.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.426 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.426 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134111 s, 78.2 MB/s 2024-04-25T10:28:58.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.443 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.443 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149481 s, 70.1 MB/s 2024-04-25T10:28:58.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.459 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.459 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148278 s, 70.7 MB/s 2024-04-25T10:28:58.474 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.475 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135985 s, 77.1 MB/s 2024-04-25T10:28:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.489 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.490 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131121 s, 80.0 MB/s 2024-04-25T10:28:58.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.506 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.506 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142652 s, 73.5 MB/s 2024-04-25T10:28:58.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.521 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.521 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134827 s, 77.8 MB/s 2024-04-25T10:28:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.541 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180261 s, 58.2 MB/s 2024-04-25T10:28:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146362 s, 71.6 MB/s 2024-04-25T10:28:58.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.573 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.573 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132019 s, 79.4 MB/s 2024-04-25T10:28:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.589 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.590 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150283 s, 69.8 MB/s 2024-04-25T10:28:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.605 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135884 s, 77.2 MB/s 2024-04-25T10:28:58.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.622 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.622 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142827 s, 73.4 MB/s 2024-04-25T10:28:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139446 s, 75.2 MB/s 2024-04-25T10:28:58.657 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.658 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188105 s, 55.7 MB/s 2024-04-25T10:28:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.672 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012867 s, 81.5 MB/s 2024-04-25T10:28:58.686 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.687 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.687 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128897 s, 81.3 MB/s 2024-04-25T10:28:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155713 s, 67.3 MB/s 2024-04-25T10:28:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.719 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131805 s, 79.6 MB/s 2024-04-25T10:28:58.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.734 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.734 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135308 s, 77.5 MB/s 2024-04-25T10:28:58.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.752 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.752 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015267 s, 68.7 MB/s 2024-04-25T10:28:58.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136863 s, 76.6 MB/s 2024-04-25T10:28:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.783 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141081 s, 74.3 MB/s 2024-04-25T10:28:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176024 s, 59.6 MB/s 2024-04-25T10:28:58.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.819 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.819 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145145 s, 72.2 MB/s 2024-04-25T10:28:58.834 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.835 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141558 s, 74.1 MB/s 2024-04-25T10:28:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.850 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134642 s, 77.9 MB/s 2024-04-25T10:28:58.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.870 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.870 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179233 s, 58.5 MB/s 2024-04-25T10:28:58.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.886 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148016 s, 70.8 MB/s 2024-04-25T10:28:58.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:58.888 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154055 s, 68.1 MB/s 2024-04-25T10:28:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.981 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190656 s, 55.0 MB/s 2024-04-25T10:28:58.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:58.996 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:58.997 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142135 s, 73.8 MB/s 2024-04-25T10:28:59.013 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.014 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.014 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152251 s, 68.9 MB/s 2024-04-25T10:28:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.031 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152658 s, 68.7 MB/s 2024-04-25T10:28:59.047 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150836 s, 69.5 MB/s 2024-04-25T10:28:59.064 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.065 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.065 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015047 s, 69.7 MB/s 2024-04-25T10:28:59.081 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.082 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.082 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151101 s, 69.4 MB/s 2024-04-25T10:28:59.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.101 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.102 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177825 s, 59.0 MB/s 2024-04-25T10:28:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.118 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149711 s, 70.0 MB/s 2024-04-25T10:28:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.135 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149656 s, 70.1 MB/s 2024-04-25T10:28:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.151 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150889 s, 69.5 MB/s 2024-04-25T10:28:59.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.168 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.168 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150293 s, 69.8 MB/s 2024-04-25T10:28:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.184 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141725 s, 74.0 MB/s 2024-04-25T10:28:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151758 s, 69.1 MB/s 2024-04-25T10:28:59.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.216 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.217 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132843 s, 78.9 MB/s 2024-04-25T10:28:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137072 s, 76.5 MB/s 2024-04-25T10:28:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.248 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147935 s, 70.9 MB/s 2024-04-25T10:28:59.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.265 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.265 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147176 s, 71.2 MB/s 2024-04-25T10:28:59.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.284 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.284 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176645 s, 59.4 MB/s 2024-04-25T10:28:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.299 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013283 s, 78.9 MB/s 2024-04-25T10:28:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.314 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129791 s, 80.8 MB/s 2024-04-25T10:28:59.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.330 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.330 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136214 s, 77.0 MB/s 2024-04-25T10:28:59.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.346 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.346 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014524 s, 72.2 MB/s 2024-04-25T10:28:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.363 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150249 s, 69.8 MB/s 2024-04-25T10:28:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.378 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134246 s, 78.1 MB/s 2024-04-25T10:28:59.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.393 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.393 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129736 s, 80.8 MB/s 2024-04-25T10:28:59.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.408 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.408 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135332 s, 77.5 MB/s 2024-04-25T10:28:59.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.424 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.425 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143277 s, 73.2 MB/s 2024-04-25T10:28:59.440 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.440 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.440 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134303 s, 78.1 MB/s 2024-04-25T10:28:59.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.455 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135571 s, 77.3 MB/s 2024-04-25T10:28:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.472 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155804 s, 67.3 MB/s 2024-04-25T10:28:59.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:28:59.474 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:28:59.474 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:28:59.474 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:28:59.474 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:28:59.474 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:28:59.475 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:28:59.475 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:28:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.543 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146306 s, 71.7 MB/s 2024-04-25T10:28:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.559 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146811 s, 71.4 MB/s 2024-04-25T10:28:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154968 s, 67.7 MB/s 2024-04-25T10:28:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136396 s, 76.9 MB/s 2024-04-25T10:28:59.607 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139076 s, 75.4 MB/s 2024-04-25T10:28:59.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.623 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.623 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133978 s, 78.3 MB/s 2024-04-25T10:28:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179254 s, 58.5 MB/s 2024-04-25T10:28:59.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.658 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.658 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013273 s, 79.0 MB/s 2024-04-25T10:28:59.674 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.675 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.675 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147964 s, 70.9 MB/s 2024-04-25T10:28:59.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.690 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.690 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140541 s, 74.6 MB/s 2024-04-25T10:28:59.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.711 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.711 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187031 s, 56.1 MB/s 2024-04-25T10:28:59.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151199 s, 69.4 MB/s 2024-04-25T10:28:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133718 s, 78.4 MB/s 2024-04-25T10:28:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.759 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143963 s, 72.8 MB/s 2024-04-25T10:28:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138174 s, 75.9 MB/s 2024-04-25T10:28:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.792 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156717 s, 66.9 MB/s 2024-04-25T10:28:59.807 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.807 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.807 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131957 s, 79.5 MB/s 2024-04-25T10:28:59.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.827 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.827 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183174 s, 57.2 MB/s 2024-04-25T10:28:59.842 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132015 s, 79.4 MB/s 2024-04-25T10:28:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.857 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132101 s, 79.4 MB/s 2024-04-25T10:28:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.873 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142275 s, 73.7 MB/s 2024-04-25T10:28:59.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:59.886 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:28:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018098 s, 57.9 MB/s 2024-04-25T10:28:59.908 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.909 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.909 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138565 s, 75.7 MB/s 2024-04-25T10:28:59.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.924 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.924 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136298 s, 76.9 MB/s 2024-04-25T10:28:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013641 s, 76.9 MB/s 2024-04-25T10:28:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.957 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152344 s, 68.8 MB/s 2024-04-25T10:28:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134633 s, 77.9 MB/s 2024-04-25T10:28:59.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:28:59.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:28:59.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013542 s, 77.4 MB/s 2024-04-25T10:29:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.002 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135271 s, 77.5 MB/s 2024-04-25T10:29:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.018 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132043 s, 79.4 MB/s 2024-04-25T10:29:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.034 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014138 s, 74.2 MB/s 2024-04-25T10:29:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015972 s, 65.7 MB/s 2024-04-25T10:29:00.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:29:00.054 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:29:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.131 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183985 s, 57.0 MB/s 2024-04-25T10:29:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.146 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136903 s, 76.6 MB/s 2024-04-25T10:29:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.163 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147457 s, 71.1 MB/s 2024-04-25T10:29:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.178 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013427 s, 78.1 MB/s 2024-04-25T10:29:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.192 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118098 s, 88.8 MB/s 2024-04-25T10:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.208 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146462 s, 71.6 MB/s 2024-04-25T10:29:00.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.225 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154996 s, 67.7 MB/s 2024-04-25T10:29:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.246 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185983 s, 56.4 MB/s 2024-04-25T10:29:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134556 s, 77.9 MB/s 2024-04-25T10:29:00.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.276 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.276 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013447 s, 78.0 MB/s 2024-04-25T10:29:00.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.292 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.292 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139171 s, 75.3 MB/s 2024-04-25T10:29:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.307 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134008 s, 78.2 MB/s 2024-04-25T10:29:00.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.322 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.322 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131627 s, 79.7 MB/s 2024-04-25T10:29:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.335 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115827 s, 90.5 MB/s 2024-04-25T10:29:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.350 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134196 s, 78.1 MB/s 2024-04-25T10:29:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013544 s, 77.4 MB/s 2024-04-25T10:29:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132542 s, 79.1 MB/s 2024-04-25T10:29:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.396 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134073 s, 78.2 MB/s 2024-04-25T10:29:00.415 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.415 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.415 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169352 s, 61.9 MB/s 2024-04-25T10:29:00.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.431 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.431 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142634 s, 73.5 MB/s 2024-04-25T10:29:00.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.449 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.449 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159552 s, 65.7 MB/s 2024-04-25T10:29:00.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.466 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.466 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148171 s, 70.8 MB/s 2024-04-25T10:29:00.485 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.486 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184314 s, 56.9 MB/s 2024-04-25T10:29:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130945 s, 80.1 MB/s 2024-04-25T10:29:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.515 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128189 s, 81.8 MB/s 2024-04-25T10:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.530 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130945 s, 80.1 MB/s 2024-04-25T10:29:00.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.549 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.549 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171115 s, 61.3 MB/s 2024-04-25T10:29:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.565 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139932 s, 74.9 MB/s 2024-04-25T10:29:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.580 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135667 s, 77.3 MB/s 2024-04-25T10:29:00.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129855 s, 80.7 MB/s 2024-04-25T10:29:00.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.616 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.617 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187002 s, 56.1 MB/s 2024-04-25T10:29:00.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.630 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.631 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132484 s, 79.1 MB/s 2024-04-25T10:29:00.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:29:00.632 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:29:00.702 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136631 s, 76.7 MB/s 2024-04-25T10:29:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130707 s, 80.2 MB/s 2024-04-25T10:29:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155999 s, 67.2 MB/s 2024-04-25T10:29:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138312 s, 75.8 MB/s 2024-04-25T10:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143423 s, 73.1 MB/s 2024-04-25T10:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.782 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132406 s, 79.2 MB/s 2024-04-25T10:29:00.798 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.799 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149013 s, 70.4 MB/s 2024-04-25T10:29:00.815 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.816 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150248 s, 69.8 MB/s 2024-04-25T10:29:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.831 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133598 s, 78.5 MB/s 2024-04-25T10:29:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149142 s, 70.3 MB/s 2024-04-25T10:29:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135518 s, 77.4 MB/s 2024-04-25T10:29:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.878 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135353 s, 77.5 MB/s 2024-04-25T10:29:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157295 s, 66.7 MB/s 2024-04-25T10:29:00.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.912 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.912 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141782 s, 74.0 MB/s 2024-04-25T10:29:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136578 s, 76.8 MB/s 2024-04-25T10:29:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.942 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133106 s, 78.8 MB/s 2024-04-25T10:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.957 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133522 s, 78.5 MB/s 2024-04-25T10:29:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170251 s, 61.6 MB/s 2024-04-25T10:29:00.993 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:00.994 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:00.994 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154241 s, 68.0 MB/s 2024-04-25T10:29:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.009 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.010 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013684 s, 76.6 MB/s 2024-04-25T10:29:01.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.028 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.028 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170662 s, 61.4 MB/s 2024-04-25T10:29:01.043 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.044 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.044 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136228 s, 77.0 MB/s 2024-04-25T10:29:01.060 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.061 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.061 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149002 s, 70.4 MB/s 2024-04-25T10:29:01.075 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129221 s, 81.1 MB/s 2024-04-25T10:29:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134373 s, 78.0 MB/s 2024-04-25T10:29:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.108 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149623 s, 70.1 MB/s 2024-04-25T10:29:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138813 s, 75.5 MB/s 2024-04-25T10:29:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127601 s, 82.2 MB/s 2024-04-25T10:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176452 s, 59.4 MB/s 2024-04-25T10:29:01.172 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.173 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.173 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129373 s, 81.1 MB/s 2024-04-25T10:29:01.189 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.190 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.190 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151863 s, 69.0 MB/s 2024-04-25T10:29:01.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.206 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.206 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150715 s, 69.6 MB/s 2024-04-25T10:29:01.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:29:01.208 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:29:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.286 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142293 s, 73.7 MB/s 2024-04-25T10:29:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.298 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131777 s, 79.6 MB/s 2024-04-25T10:29:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126084 s, 83.2 MB/s 2024-04-25T10:29:01.328 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.329 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.329 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146751 s, 71.5 MB/s 2024-04-25T10:29:01.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.344 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.344 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142521 s, 73.6 MB/s 2024-04-25T10:29:01.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.361 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.361 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145964 s, 71.8 MB/s 2024-04-25T10:29:01.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.381 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.381 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187056 s, 56.1 MB/s 2024-04-25T10:29:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.397 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136512 s, 76.8 MB/s 2024-04-25T10:29:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.413 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014504 s, 72.3 MB/s 2024-04-25T10:29:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.428 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013524 s, 77.5 MB/s 2024-04-25T10:29:01.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.448 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.448 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177973 s, 58.9 MB/s 2024-04-25T10:29:01.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.463 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.463 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135137 s, 77.6 MB/s 2024-04-25T10:29:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.478 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135394 s, 77.4 MB/s 2024-04-25T10:29:01.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.495 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.495 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148262 s, 70.7 MB/s 2024-04-25T10:29:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.516 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.516 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184062 s, 57.0 MB/s 2024-04-25T10:29:01.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.530 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.531 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133124 s, 78.8 MB/s 2024-04-25T10:29:01.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.546 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128156 s, 81.8 MB/s 2024-04-25T10:29:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01428 s, 73.4 MB/s 2024-04-25T10:29:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140716 s, 74.5 MB/s 2024-04-25T10:29:01.593 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.594 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149153 s, 70.3 MB/s 2024-04-25T10:29:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.609 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135708 s, 77.3 MB/s 2024-04-25T10:29:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.625 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.626 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146467 s, 71.6 MB/s 2024-04-25T10:29:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133409 s, 78.6 MB/s 2024-04-25T10:29:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135153 s, 77.6 MB/s 2024-04-25T10:29:01.676 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188324 s, 55.7 MB/s 2024-04-25T10:29:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014833 s, 70.7 MB/s 2024-04-25T10:29:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.709 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142241 s, 73.7 MB/s 2024-04-25T10:29:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016153 s, 64.9 MB/s 2024-04-25T10:29:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.747 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184743 s, 56.8 MB/s 2024-04-25T10:29:01.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.763 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.763 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138631 s, 75.6 MB/s 2024-04-25T10:29:01.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.778 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.778 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013513 s, 77.6 MB/s 2024-04-25T10:29:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.794 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137454 s, 76.3 MB/s 2024-04-25T10:29:01.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:01.795 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:29:01.795 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:29:01.795 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:29:01.796 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:29:01.796 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:29:01.796 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:29:01.796 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:29:01.867 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.868 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147183 s, 71.2 MB/s 2024-04-25T10:29:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169267 s, 61.9 MB/s 2024-04-25T10:29:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132434 s, 79.2 MB/s 2024-04-25T10:29:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130368 s, 80.4 MB/s 2024-04-25T10:29:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162248 s, 64.6 MB/s 2024-04-25T10:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148858 s, 70.4 MB/s 2024-04-25T10:29:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175318 s, 59.8 MB/s 2024-04-25T10:29:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154012 s, 68.1 MB/s 2024-04-25T10:29:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.007 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179656 s, 58.4 MB/s 2024-04-25T10:29:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.025 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015438 s, 67.9 MB/s 2024-04-25T10:29:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.041 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151404 s, 69.3 MB/s 2024-04-25T10:29:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.056 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134105 s, 78.2 MB/s 2024-04-25T10:29:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.076 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179175 s, 58.5 MB/s 2024-04-25T10:29:02.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.091 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.091 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130527 s, 80.3 MB/s 2024-04-25T10:29:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139095 s, 75.4 MB/s 2024-04-25T10:29:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133799 s, 78.4 MB/s 2024-04-25T10:29:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133366 s, 78.6 MB/s 2024-04-25T10:29:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150038 s, 69.9 MB/s 2024-04-25T10:29:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144185 s, 72.7 MB/s 2024-04-25T10:29:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.185 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126338 s, 83.0 MB/s 2024-04-25T10:29:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.201 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137601 s, 76.2 MB/s 2024-04-25T10:29:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.222 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196904 s, 53.3 MB/s 2024-04-25T10:29:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.248 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235626 s, 44.5 MB/s 2024-04-25T10:29:02.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.274 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229868 s, 45.6 MB/s 2024-04-25T10:29:02.288 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133434 s, 78.6 MB/s 2024-04-25T10:29:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148649 s, 70.5 MB/s 2024-04-25T10:29:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135207 s, 77.6 MB/s 2024-04-25T10:29:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145864 s, 71.9 MB/s 2024-04-25T10:29:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.358 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187878 s, 55.8 MB/s 2024-04-25T10:29:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014769 s, 71.0 MB/s 2024-04-25T10:29:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.391 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152532 s, 68.7 MB/s 2024-04-25T10:29:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.406 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133247 s, 78.7 MB/s 2024-04-25T10:29:02.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> done 2024-04-25T10:29:02.408 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:29:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.484 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150608 s, 69.6 MB/s 2024-04-25T10:29:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.499 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138739 s, 75.6 MB/s 2024-04-25T10:29:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.514 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130812 s, 80.2 MB/s 2024-04-25T10:29:02.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.529 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.529 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133812 s, 78.4 MB/s 2024-04-25T10:29:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.545 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014021 s, 74.8 MB/s 2024-04-25T10:29:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.562 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150864 s, 69.5 MB/s 2024-04-25T10:29:02.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.579 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.579 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150153 s, 69.8 MB/s 2024-04-25T10:29:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.595 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142554 s, 73.6 MB/s 2024-04-25T10:29:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182958 s, 57.3 MB/s 2024-04-25T10:29:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.631 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.632 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145314 s, 72.2 MB/s 2024-04-25T10:29:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.647 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138001 s, 76.0 MB/s 2024-04-25T10:29:02.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.662 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.662 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012902 s, 81.3 MB/s 2024-04-25T10:29:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.677 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133385 s, 78.6 MB/s 2024-04-25T10:29:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138717 s, 75.6 MB/s 2024-04-25T10:29:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.706 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111305 s, 94.2 MB/s 2024-04-25T10:29:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132948 s, 78.9 MB/s 2024-04-25T10:29:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.737 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149268 s, 70.2 MB/s 2024-04-25T10:29:02.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.757 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.757 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178255 s, 58.8 MB/s 2024-04-25T10:29:02.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.774 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.774 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153012 s, 68.5 MB/s 2024-04-25T10:29:02.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.791 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147064 s, 71.3 MB/s 2024-04-25T10:29:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018711 s, 56.0 MB/s 2024-04-25T10:29:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.825 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118079 s, 88.8 MB/s 2024-04-25T10:29:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.840 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137646 s, 76.2 MB/s 2024-04-25T10:29:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.857 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149642 s, 70.1 MB/s 2024-04-25T10:29:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.872 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.873 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132964 s, 78.9 MB/s 2024-04-25T10:29:02.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.889 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.889 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151705 s, 69.1 MB/s 2024-04-25T10:29:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.905 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143841 s, 72.9 MB/s 2024-04-25T10:29:02.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.922 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.922 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149511 s, 70.1 MB/s 2024-04-25T10:29:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145713 s, 72.0 MB/s 2024-04-25T10:29:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142602 s, 73.5 MB/s 2024-04-25T10:29:02.969 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.970 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136073 s, 77.1 MB/s 2024-04-25T10:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T10:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T10:29:02.986 INFO:teuthology.orchestra.run.smithi045.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146156 s, 71.7 MB/s 2024-04-25T10:29:02.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T10:29:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 Processor -- start 2024-04-25T10:29:03.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 -- start start 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4cf815b150 0x7f4cf80aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cf806c0d0 0x7f4cf806c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80b2fd0 con 0x7f4cf815a0e0 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf815b5c0 con 0x7f4cf806c0d0 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cfcc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf80aef90 con 0x7f4cf815b150 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cf806c0d0 0x7f4cf806c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4cf815a0e0 0x7f4cf80b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40588/0 (socket says 172.21.15.45:40588) 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4cf806c0d0 0x7f4cf806c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35688/0 (socket says 172.21.15.45:35688) 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf5e00640 1 -- 172.21.15.45:0/2055487975 learned_addr learned my addr 172.21.15.45:0/2055487975 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4ced400640 1 --2- 172.21.15.45:0/2055487975 >> v2:172.21.15.117:3300/0 conn(0x7f4cf815b150 0x7f4cf80aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf6800640 1 -- 172.21.15.45:0/2055487975 >> v2:172.21.15.117:3300/0 conn(0x7f4cf815b150 msgr2=0x7f4cf80aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf6800640 1 --2- 172.21.15.45:0/2055487975 >> v2:172.21.15.117:3300/0 conn(0x7f4cf815b150 0x7f4cf80aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf6800640 1 -- 172.21.15.45:0/2055487975 >> v1:172.21.15.45:6789/0 conn(0x7f4cf815a0e0 legacy=0x7f4cf80b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:03.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.176+0000 7f4cf6800640 1 -- 172.21.15.45:0/2055487975 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80b32b0 con 0x7f4cf806c0d0 2024-04-25T10:29:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cf6800640 1 --2- 172.21.15.45:0/2055487975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf806c0d0 0x7f4cf806c500 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f4cf815dae0 tx=0x7f4cf001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d176aabc47e4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cf5400640 1 -- 172.21.15.45:0/2055487975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00b2070 con 0x7f4cf806c0d0 2024-04-25T10:29:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cf5400640 1 -- 172.21.15.45:0/2055487975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf00a5760 con 0x7f4cf806c0d0 2024-04-25T10:29:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cf5400640 1 -- 172.21.15.45:0/2055487975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00a5b30 con 0x7f4cf806c0d0 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 -- 172.21.15.45:0/2055487975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf806c0d0 msgr2=0x7f4cf806c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/2055487975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf806c0d0 0x7f4cf806c500 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f4cf815dae0 tx=0x7f4cf001c810 comp rx=0 tx=0).stop 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 -- 172.21.15.45:0/2055487975 shutdown_connections 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/2055487975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf806c0d0 0x7f4cf806c500 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/2055487975 >> v2:172.21.15.117:3300/0 conn(0x7f4cf815b150 0x7f4cf80aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 -- 172.21.15.45:0/2055487975 >> 172.21.15.45:0/2055487975 conn(0x7f4cf80669a0 msgr2=0x7f4cf8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 -- 172.21.15.45:0/2055487975 shutdown_connections 2024-04-25T10:29:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.180+0000 7f4cfcc00640 1 -- 172.21.15.45:0/2055487975 wait complete. 2024-04-25T10:29:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 Processor -- start 2024-04-25T10:29:03.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- start start 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cf806c0d0 0x7f4cf8146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 0x7f4cf80ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cf815b150 0x7f4cf8149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80705f0 con 0x7f4cf815a0e0 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80af1d0 con 0x7f4cf806c0d0 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8070770 con 0x7f4cf815b150 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 0x7f4cf80ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 0x7f4cf80ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35690/0 (socket says 172.21.15.45:35690) 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 -- 172.21.15.45:0/657630059 learned_addr learned my addr 172.21.15.45:0/657630059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cf5e00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cf806c0d0 0x7f4cf8146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cf6800640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cf815b150 0x7f4cf8149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 -- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cf815b150 msgr2=0x7f4cf8149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cf815b150 0x7f4cf8149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 -- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cf806c0d0 msgr2=0x7f4cf8146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cf806c0d0 0x7f4cf8146660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 -- 172.21.15.45:0/657630059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf814a240 con 0x7f4cf815a0e0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cf5e00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cf806c0d0 0x7f4cf8146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ced400640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 0x7f4cf80ce270 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e870 tx=0x7f4ce000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0017070 con 0x7f4cf815a0e0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce000c940 con 0x7f4cf815a0e0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000ccf0 con 0x7f4cf815a0e0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf814a4e0 con 0x7f4cf815a0e0 2024-04-25T10:29:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf8148a80 con 0x7f4cf815a0e0 2024-04-25T10:29:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.184+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf804ef30 con 0x7f4cf815a0e0 2024-04-25T10:29:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.188+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ce001a070 con 0x7f4cf815a0e0 2024-04-25T10:29:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.188+0000 7f4ceea00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4cd4066f90 0x7f4cd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.188+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4ce0084f10 con 0x7f4cf815a0e0 2024-04-25T10:29:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.188+0000 7f4cf5e00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4cd4066f90 0x7f4cd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.188+0000 7f4cf5e00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4cd4066f90 0x7f4cd4069450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8000f70 tx=0x7f4ce8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:03.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.192+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ce0055ed0 con 0x7f4cf815a0e0 2024-04-25T10:29:03.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:03.388+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f4cf80b46b0 con 0x7f4cf815a0e0 2024-04-25T10:29:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.108+0000 7f4ceea00640 1 -- 172.21.15.45:0/657630059 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v105) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4ce00557d0 con 0x7f4cf815a0e0 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4cd4066f90 msgr2=0x7f4cd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4cd4066f90 0x7f4cd4069450 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8000f70 tx=0x7f4ce8009040 comp rx=0 tx=0).stop 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 msgr2=0x7f4cf80ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 0x7f4cf80ce270 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e870 tx=0x7f4ce000ec70 comp rx=0 tx=0).stop 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 shutdown_connections 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4cd4066f90 0x7f4cd4069450 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cf815b150 0x7f4cf8149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cf815a0e0 0x7f4cf80ce270 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 --2- 172.21.15.45:0/657630059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cf806c0d0 0x7f4cf8146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 >> 172.21.15.45:0/657630059 conn(0x7f4cf80669a0 msgr2=0x7f4cf814ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 shutdown_connections 2024-04-25T10:29:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.112+0000 7f4cfcc00640 1 -- 172.21.15.45:0/657630059 wait complete. 2024-04-25T10:29:04.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:29:04.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 Processor -- start 2024-04-25T10:29:04.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 -- start start 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5f015dcc0 0x7fa5f0162250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5f00669a0 0x7fa5f0066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f0161800 con 0x7fa5f015dcc0 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00b3b20 con 0x7fa5f00669a0 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5f5000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00b3e00 con 0x7fa5f00b5480 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5f015dcc0 0x7fa5f0162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5f015dcc0 0x7fa5f0162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35694/0 (socket says 172.21.15.45:35694) 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 -- 172.21.15.45:0/2475851349 learned_addr learned my addr 172.21.15.45:0/2475851349 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:04.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ee800640 1 --2- 172.21.15.45:0/2475851349 >> v2:172.21.15.117:3300/0 conn(0x7fa5f00669a0 0x7fa5f0066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5e5400640 1 --2- 172.21.15.45:0/2475851349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 -- 172.21.15.45:0/2475851349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 msgr2=0x7fa5f00b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 --2- 172.21.15.45:0/2475851349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 -- 172.21.15.45:0/2475851349 >> v2:172.21.15.117:3300/0 conn(0x7fa5f00669a0 msgr2=0x7fa5f0066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 --2- 172.21.15.45:0/2475851349 >> v2:172.21.15.117:3300/0 conn(0x7fa5f00669a0 0x7fa5f0066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.284+0000 7fa5ede00640 1 -- 172.21.15.45:0/2475851349 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0161a50 con 0x7fa5f015dcc0 2024-04-25T10:29:04.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5ede00640 1 --2- 172.21.15.45:0/2475851349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f0162250 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00af410 tx=0x7fa5e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443d417212338d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:04.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5ed400640 1 -- 172.21.15.45:0/2475851349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00b0070 con 0x7fa5f015dcc0 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5ed400640 1 -- 172.21.15.45:0/2475851349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e00ab600 con 0x7fa5f015dcc0 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5ed400640 1 -- 172.21.15.45:0/2475851349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00ab9d0 con 0x7fa5f015dcc0 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 -- 172.21.15.45:0/2475851349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 msgr2=0x7fa5f0162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 --2- 172.21.15.45:0/2475851349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f0162250 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00af410 tx=0x7fa5e001c490 comp rx=0 tx=0).stop 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 -- 172.21.15.45:0/2475851349 shutdown_connections 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 --2- 172.21.15.45:0/2475851349 >> v2:172.21.15.117:3300/0 conn(0x7fa5f00669a0 0x7fa5f0066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 --2- 172.21.15.45:0/2475851349 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 --2- 172.21.15.45:0/2475851349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f0162250 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 -- 172.21.15.45:0/2475851349 >> 172.21.15.45:0/2475851349 conn(0x7fa5f006ac90 msgr2=0x7fa5f006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 -- 172.21.15.45:0/2475851349 shutdown_connections 2024-04-25T10:29:04.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.288+0000 7fa5f5000640 1 -- 172.21.15.45:0/2475851349 wait complete. 2024-04-25T10:29:04.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 Processor -- start 2024-04-25T10:29:04.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 -- start start 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f00669a0 0x7fa5f01492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00d10d0 con 0x7fa5f015dcc0 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00d0f50 con 0x7fa5f00669a0 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f015ed20 con 0x7fa5f00b5480 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f00adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35696/0 (socket says 172.21.15.45:35696) 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 -- 172.21.15.45:0/527704096 learned_addr learned my addr 172.21.15.45:0/527704096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ede00640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f00669a0 0x7fa5f01492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5e5400640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 -- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 msgr2=0x7fa5f00ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 -- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f00669a0 msgr2=0x7fa5f01492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f00669a0 0x7fa5f01492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 -- 172.21.15.45:0/527704096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f015f000 con 0x7fa5f015dcc0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5ee800640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f00adfc0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8007d50 tx=0x7fa5e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8014420 con 0x7fa5f015dcc0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e8019070 con 0x7fa5f015dcc0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8014820 con 0x7fa5f015dcc0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f015f2e0 con 0x7fa5f015dcc0 2024-04-25T10:29:04.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.292+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f015f5c0 con 0x7fa5f015dcc0 2024-04-25T10:29:04.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.296+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f004ef30 con 0x7fa5f015dcc0 2024-04-25T10:29:04.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.296+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5e80149c0 con 0x7fa5f015dcc0 2024-04-25T10:29:04.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.296+0000 7fa5e6a00640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5cc066f90 0x7fa5cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:04.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.296+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa5e80856a0 con 0x7fa5f015dcc0 2024-04-25T10:29:04.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.296+0000 7fa5ede00640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5cc066f90 0x7fa5cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:04.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.296+0000 7fa5ede00640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5cc066f90 0x7fa5cc069450 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00ad000 tx=0x7fa5e00a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:04.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.300+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5e8056010 con 0x7fa5f015dcc0 2024-04-25T10:29:04.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:04.496+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa5f0168970 con 0x7fa5f015dcc0 2024-04-25T10:29:05.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.116+0000 7fa5e6a00640 1 -- 172.21.15.45:0/527704096 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v106) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa5e805a0b0 con 0x7fa5f015dcc0 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5cc066f90 msgr2=0x7fa5cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5cc066f90 0x7fa5cc069450 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00ad000 tx=0x7fa5e00a6000 comp rx=0 tx=0).stop 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 msgr2=0x7fa5f00adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f00adfc0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8007d50 tx=0x7fa5e800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 shutdown_connections 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5cc066f90 0x7fa5cc069450 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5f015dcc0 0x7fa5f00adfc0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5f00b5480 0x7fa5f00ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 --2- 172.21.15.45:0/527704096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5f00669a0 0x7fa5f01492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 >> 172.21.15.45:0/527704096 conn(0x7fa5f006ac90 msgr2=0x7fa5f0149ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 shutdown_connections 2024-04-25T10:29:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.120+0000 7fa5f5000640 1 -- 172.21.15.45:0/527704096 wait complete. 2024-04-25T10:29:05.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bfa00640 1 Processor -- start 2024-04-25T10:29:05.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bfa00640 1 -- start start 2024-04-25T10:29:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bfa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50b815bc40 0x7f50b806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bfa00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b80b2770 con 0x7f50b80cda10 2024-04-25T10:29:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bfa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b806c670 con 0x7f50b80cf930 2024-04-25T10:29:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bfa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b806c950 con 0x7f50b815bc40 2024-04-25T10:29:05.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50afe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f50b80cda10 0x7f50b80b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40666/0 (socket says 172.21.15.45:40666) 2024-04-25T10:29:05.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50afe00640 1 -- 172.21.15.45:0/3831172930 learned_addr learned my addr 172.21.15.45:0/3831172930 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181207149 0 0) 0x7f50b80b2770 con 0x7f50b80cda10 2024-04-25T10:29:05.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50bd000640 1 --2- 172.21.15.45:0/3831172930 >> v2:172.21.15.117:3300/0 conn(0x7f50b815bc40 0x7f50b806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.284+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f509c003530 con 0x7f50b80cda10 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3943077659 0 0) 0x7f509c003530 con 0x7f50b80cda10 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 >> v2:172.21.15.117:3300/0 conn(0x7f50b815bc40 msgr2=0x7f50b806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 --2- 172.21.15.45:0/3831172930 >> v2:172.21.15.117:3300/0 conn(0x7f50b815bc40 0x7f50b806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 >> v1:172.21.15.117:6789/0 conn(0x7f50b80cf930 legacy=0x7f50b80b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b806cc30 con 0x7f50b80cda10 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f50a0001680 con 0x7f50b80cda10 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 >> v1:172.21.15.45:6789/0 conn(0x7f50b80cda10 legacy=0x7f50b80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c003a10 0x7f509c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f509c004810 0x7f509c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c005220 0x7f509c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b806c950 con 0x7f509c004810 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c00c170 con 0x7f509c003a10 2024-04-25T10:29:05.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c00c370 con 0x7f509c005220 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c003a10 0x7f509c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50afe00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f509c004810 0x7f509c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bda00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c005220 0x7f509c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 -- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c005220 msgr2=0x7f509c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c005220 0x7f509c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 -- 172.21.15.45:0/3831172930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f509c004810 msgr2=0x7f509c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f509c004810 0x7f509c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 -- 172.21.15.45:0/3831172930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c003530 con 0x7f509c003a10 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50afe00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f509c004810 0x7f509c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:05.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bda00640 1 -- 172.21.15.45:0/3831172930 reap_dead start 2024-04-25T10:29:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.288+0000 7f50bd000640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c003a10 0x7f509c003e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009ab0 tx=0x7f50a801ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4544ed8ecb7de457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80096b0 con 0x7f509c003a10 2024-04-25T10:29:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a80a6450 con 0x7f509c003a10 2024-04-25T10:29:05.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50b7e00640 1 -- 172.21.15.45:0/3831172930 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80a6840 con 0x7f509c003a10 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c003a10 msgr2=0x7f509c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c003a10 0x7f509c003e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009ab0 tx=0x7f50a801ca70 comp rx=0 tx=0).stop 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- 172.21.15.45:0/3831172930 shutdown_connections 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 --2- 172.21.15.45:0/3831172930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c003a10 0x7f509c003e30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- 172.21.15.45:0/3831172930 >> 172.21.15.45:0/3831172930 conn(0x7f50b8066b70 msgr2=0x7f50b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- 172.21.15.45:0/3831172930 shutdown_connections 2024-04-25T10:29:05.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- 172.21.15.45:0/3831172930 wait complete. 2024-04-25T10:29:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 Processor -- start 2024-04-25T10:29:05.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- start start 2024-04-25T10:29:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c003a10 0x7f50b80ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b81f6bc0 0x7f50b8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 0x7f50b8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80d00d0 con 0x7f50b8157cd0 2024-04-25T10:29:05.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80cff50 con 0x7f50b81f6bc0 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bfa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80d0250 con 0x7f509c003a10 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 0x7f50b8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 0x7f50b8158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51776/0 (socket says 172.21.15.45:51776) 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 -- 172.21.15.45:0/4060297842 learned_addr learned my addr 172.21.15.45:0/4060297842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bd000640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c003a10 0x7f50b80ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50afe00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b81f6bc0 0x7f50b8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 -- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c003a10 msgr2=0x7f50b80ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c003a10 0x7f50b80ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 -- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b81f6bc0 msgr2=0x7f50b8153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.292+0000 7f50bda00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b81f6bc0 0x7f50b8153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50bda00640 1 -- 172.21.15.45:0/4060297842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b80aaee0 con 0x7f50b8157cd0 2024-04-25T10:29:05.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50bda00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 0x7f50b8158150 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f50b80702a0 tx=0x7f50b000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b000fe80 con 0x7f50b8157cd0 2024-04-25T10:29:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0018070 con 0x7f50b8157cd0 2024-04-25T10:29:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0003dd0 con 0x7f50b8157cd0 2024-04-25T10:29:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b81f70c0 con 0x7f50b8157cd0 2024-04-25T10:29:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b81f7680 con 0x7f50b8157cd0 2024-04-25T10:29:05.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b8158690 con 0x7f50b8157cd0 2024-04-25T10:29:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50b00050d0 con 0x7f50b8157cd0 2024-04-25T10:29:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50b5600640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:05.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f50b0084dc0 con 0x7f50b8157cd0 2024-04-25T10:29:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.296+0000 7f50bd000640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:05.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.300+0000 7f50bd000640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50a80ad000 tx=0x7f50a80b2040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:05.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.300+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50b00556a0 con 0x7f50b8157cd0 2024-04-25T10:29:05.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.496+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50b804ef30 con 0x7f50b8157cd0 2024-04-25T10:29:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.496+0000 7f50b5600640 1 -- 172.21.15.45:0/4060297842 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4952 (secure 0 0 0) 0x7f50b0059740 con 0x7f50b8157cd0 2024-04-25T10:29:05.498 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:05.498 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":107,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:29:05.122046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8893":{"gid":8893,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:29:05.498 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 107 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 msgr2=0x7f5094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50a80ad000 tx=0x7f50a80b2040 comp rx=0 tx=0).stop 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 msgr2=0x7f50b8158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 0x7f50b8158150 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f50b80702a0 tx=0x7f50b000fc90 comp rx=0 tx=0).stop 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 shutdown_connections 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8157cd0 0x7f50b8158150 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b81f6bc0 0x7f50b8153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 --2- 172.21.15.45:0/4060297842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f509c003a10 0x7f50b80ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 >> 172.21.15.45:0/4060297842 conn(0x7f50b8066b70 msgr2=0x7f50b80b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 shutdown_connections 2024-04-25T10:29:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:05.500+0000 7f50bfa00640 1 -- 172.21.15.45:0/4060297842 wait complete. 2024-04-25T10:29:05.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:28:36.425124+0000', 'modified': '2024-04-25T10:29:05.122046+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8893': {'gid': 8893, 'name': 'd', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6836/2453939747', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2453939747}, {'type': 'v1', 'addr': '172.21.15.117:6836', 'nonce': 2453939747}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9531': {'gid': 9531, 'name': 'a', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/903752757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 903752757}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 903752757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:29:05.922 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:29:05.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09a00640 1 Processor -- start 2024-04-25T10:29:06.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09a00640 1 -- start start 2024-04-25T10:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9b0406b9d0 0x7f9b040b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040b5780 con 0x7f9b0415db40 2024-04-25T10:29:06.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040b5a60 con 0x7f9b0406ac90 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b040b5550 con 0x7f9b0406b9d0 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9b0415db40 0x7f9b0415df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40682/0 (socket says 172.21.15.45:40682) 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03e00640 1 -- 172.21.15.45:0/4249796005 learned_addr learned my addr 172.21.15.45:0/4249796005 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09000640 1 --2- 172.21.15.45:0/4249796005 >> v2:172.21.15.117:3300/0 conn(0x7f9b0406b9d0 0x7f9b040b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838812442 0 0) 0x7f9b040b5780 con 0x7f9b0415db40 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003530 con 0x7f9b0415db40 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2050429718 0 0) 0x7f9ae8003530 con 0x7f9b0415db40 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 >> v2:172.21.15.117:3300/0 conn(0x7f9b0406b9d0 msgr2=0x7f9b040b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 --2- 172.21.15.45:0/4249796005 >> v2:172.21.15.117:3300/0 conn(0x7f9b0406b9d0 0x7f9b040b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 >> v1:172.21.15.117:6789/0 conn(0x7f9b0406ac90 legacy=0x7f9b0406b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040b5d40 con 0x7f9b0415db40 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f9af4001680 con 0x7f9b0415db40 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 >> v1:172.21.15.45:6789/0 conn(0x7f9b0415db40 legacy=0x7f9b0415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae80039c0 0x7f9ae8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae80051d0 0x7f9ae800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800c050 con 0x7f9ae8004810 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800c200 con 0x7f9ae80039c0 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800c400 con 0x7f9ae80051d0 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9afb400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b03e00640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae80039c0 0x7f9ae8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.068+0000 7f9b09000640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae80051d0 0x7f9ae800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9afb400640 1 -- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae80051d0 msgr2=0x7f9ae800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9afb400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae80051d0 0x7f9ae800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9afb400640 1 -- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae80039c0 msgr2=0x7f9ae8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9afb400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae80039c0 0x7f9ae8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9afb400640 1 -- 172.21.15.45:0/4249796005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8003530 con 0x7f9ae8004810 2024-04-25T10:29:06.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09000640 1 -- 172.21.15.45:0/4249796005 reap_dead start 2024-04-25T10:29:06.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9afb400640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001780 tx=0x7f9aec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b76c99c39baea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0013a0 con 0x7f9ae8004810 2024-04-25T10:29:06.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec0052f0 con 0x7f9ae8004810 2024-04-25T10:29:06.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b03400640 1 -- 172.21.15.45:0/4249796005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec003840 con 0x7f9ae8004810 2024-04-25T10:29:06.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 -- 172.21.15.45:0/4249796005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae8004810 msgr2=0x7f9ae8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001780 tx=0x7f9aec01c400 comp rx=0 tx=0).stop 2024-04-25T10:29:06.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 -- 172.21.15.45:0/4249796005 shutdown_connections 2024-04-25T10:29:06.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 --2- 172.21.15.45:0/4249796005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae8004810 0x7f9ae8004c90 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 -- 172.21.15.45:0/4249796005 >> 172.21.15.45:0/4249796005 conn(0x7f9b040d17b0 msgr2=0x7f9b040d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 -- 172.21.15.45:0/4249796005 shutdown_connections 2024-04-25T10:29:06.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.072+0000 7f9b09a00640 1 -- 172.21.15.45:0/4249796005 wait complete. 2024-04-25T10:29:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 Processor -- start 2024-04-25T10:29:06.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 -- start start 2024-04-25T10:29:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae8004810 0x7f9b041f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 0x7f9b0409b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b0409eeb0 0x7f9b040c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040b5010 con 0x7f9b040ce2b0 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040b6300 con 0x7f9b0409eeb0 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040b5190 con 0x7f9ae8004810 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 0x7f9b0409b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 0x7f9b0409b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51790/0 (socket says 172.21.15.45:51790) 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 -- 172.21.15.45:0/2908496176 learned_addr learned my addr 172.21.15.45:0/2908496176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09000640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b0409eeb0 0x7f9b040c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae8004810 0x7f9b041f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 -- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae8004810 msgr2=0x7f9b041f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae8004810 0x7f9b041f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 -- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b0409eeb0 msgr2=0x7f9b040c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b0409eeb0 0x7f9b040c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 -- 172.21.15.45:0/2908496176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b041f3ee0 con 0x7f9b040ce2b0 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09000640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b0409eeb0 0x7f9b040c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae8004810 0x7f9b041f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9afb400640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 0x7f9b0409b610 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f9aec01c230 tx=0x7f9ae8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b00c00640 1 -- 172.21.15.45:0/2908496176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec007a40 con 0x7f9b040ce2b0 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b00c00640 1 -- 172.21.15.45:0/2908496176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec0ba070 con 0x7f9b040ce2b0 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b00c00640 1 -- 172.21.15.45:0/2908496176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec007250 con 0x7f9b040ce2b0 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.076+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b041f41c0 con 0x7f9b040ce2b0 2024-04-25T10:29:06.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.080+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b040cbf10 con 0x7f9b040ce2b0 2024-04-25T10:29:06.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.080+0000 7f9b00c00640 1 -- 172.21.15.45:0/2908496176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9aec0052f0 con 0x7f9b040ce2b0 2024-04-25T10:29:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.080+0000 7f9b00c00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ae0066f90 0x7f9ae0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.080+0000 7f9b00c00640 1 -- 172.21.15.45:0/2908496176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9aec11f330 con 0x7f9b040ce2b0 2024-04-25T10:29:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.080+0000 7f9b0e488000 1 Processor -- start 2024-04-25T10:29:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ae0066f90 0x7f9ae0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- start start 2024-04-25T10:29:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ae0066f90 0x7f9ae0069450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9b041f4950 tx=0x7f9af4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x55983044cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55983044d240 0x5598304029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559830402f30 0x559830403310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598302ea5f0 con 0x55983044c920 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559830251da0 con 0x559830402f30 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598301da150 con 0x55983044d240 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x55983044cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x55983044cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51800/0 (socket says 172.21.15.45:51800) 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b09000640 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559830402f30 0x559830403310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 -- 172.21.15.45:0/2270176992 learned_addr learned my addr 172.21.15.45:0/2270176992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9afb400640 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55983044d240 0x5598304029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 -- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55983044d240 msgr2=0x5598304029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55983044d240 0x5598304029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 -- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559830402f30 msgr2=0x559830403310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559830402f30 0x559830403310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 -- 172.21.15.45:0/2270176992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598301f3ca0 con 0x55983044c920 2024-04-25T10:29:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b03e00640 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x55983044cd00 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9af4007fd0 tx=0x7f9af4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ec8e5c193da57d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9af8c00640 1 -- 172.21.15.45:0/2270176992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4021450 con 0x55983044c920 2024-04-25T10:29:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9af8c00640 1 -- 172.21.15.45:0/2270176992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4026070 con 0x55983044c920 2024-04-25T10:29:06.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9af8c00640 1 -- 172.21.15.45:0/2270176992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40217f0 con 0x55983044c920 2024-04-25T10:29:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- 172.21.15.45:0/2270176992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 msgr2=0x55983044cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x55983044cd00 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9af4007fd0 tx=0x7f9af4014000 comp rx=0 tx=0).stop 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- 172.21.15.45:0/2270176992 shutdown_connections 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559830402f30 0x559830403310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55983044d240 0x5598304029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 --2- 172.21.15.45:0/2270176992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x55983044cd00 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- 172.21.15.45:0/2270176992 >> 172.21.15.45:0/2270176992 conn(0x5598304092a0 msgr2=0x5598304012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- 172.21.15.45:0/2270176992 shutdown_connections 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- 172.21.15.45:0/2270176992 wait complete. 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 -- start start 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.084+0000 7f9b0e488000 1 Processor -- start 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x559830453cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044c920 0x559830454230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 0x559830406aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983020c990 con 0x55983044d240 2024-04-25T10:29:06.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983024de40 con 0x55983044c920 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598301dd900 con 0x559830402f30 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 10 client.0 _ll_register_callbacks cb 0x5598303051c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 0x559830406aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 0x559830406aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51810/0 (socket says 172.21.15.45:51810) 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 -- 172.21.15.45:0/4227752627 learned_addr learned my addr 172.21.15.45:0/4227752627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b03e00640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x559830453cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9afb400640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044c920 0x559830454230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 -- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 msgr2=0x559830453cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x559830453cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 -- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044c920 msgr2=0x559830454230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044c920 0x559830454230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 -- 172.21.15.45:0/4227752627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55983020a630 con 0x55983044d240 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b09000640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 0x559830406aa0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f9b04067950 tx=0x7f9afc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c162558138d42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 1 -- 172.21.15.45:0/4227752627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0178e0 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 1 -- 172.21.15.45:0/4227752627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9afc017a80 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9af4026070 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 1 -- 172.21.15.45:0/4227752627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9afc028050 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 10 client.9714 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5598301da3a0 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 1 -- 172.21.15.45:0/4227752627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4026070 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 1 -- 172.21.15.45:0/4227752627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f9afc02b020 con 0x55983044d240 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 10 client.9714 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 10 client.9714 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 20 client.9714 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408160 0x55983045dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 4 client.9714 mds_command: new command op to 9531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 --> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598301eda50 con 0x559830408160 2024-04-25T10:29:06.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b03e00640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408160 0x55983045dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9b03e00640 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408160 0x55983045dd50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2eb266a8ca56598 server_cookie=411c099c2b07eb2 in_seq=0 out_seq=0 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.088+0000 7f9af1600640 10 client.9714 ms_handle_connect on v2:172.21.15.45:6834/903752757 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9af1600640 1 -- 172.21.15.45:0/4227752627 <== mds.0 v2:172.21.15.45:6834/903752757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598301eda50 con 0x559830408160 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9af1600640 10 client.9714 handle_command_reply: tid=0 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 client.9714 shutdown 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 msgr2=0x559830406aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 0x559830406aa0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f9b04067950 tx=0x7f9afc0043c0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 shutdown_connections 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408160 0x55983045dd50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044d240 0x559830406aa0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044c920 0x559830454230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9af1600640 0 client.9714 ms_handle_reset on v2:172.21.15.45:6834/903752757 2024-04-25T10:29:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4227752627 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x559830453cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 >> 172.21.15.45:0/4227752627 conn(0x5598304092a0 msgr2=0x5598304019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 shutdown_connections 2024-04-25T10:29:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 1 -- 172.21.15.45:0/4227752627 wait complete. 2024-04-25T10:29:06.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.092+0000 7f9b0e488000 20 client.9714 trim_cache size 0 max 16384 2024-04-25T10:29:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 Processor -- start 2024-04-25T10:29:06.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- start start 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304a8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x5598304a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x5598304a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04200640 con 0x55983044c920 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041f7920 con 0x55983044d240 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041ff280 con 0x559830402f30 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x5598304a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x5598304a8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51812/0 (socket says 172.21.15.45:51812) 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 -- 172.21.15.45:0/3331076361 learned_addr learned my addr 172.21.15.45:0/3331076361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b03e00640 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b09000640 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x5598304a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 -- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 msgr2=0x5598304a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304a8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 -- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 msgr2=0x5598304a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x5598304a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 -- 172.21.15.45:0/3331076361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983044d070 con 0x55983044c920 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9afb400640 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x5598304a8740 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f9aec11d8b0 tx=0x7f9aec0ad040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45059013cc56ed88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9af3e00640 1 -- 172.21.15.45:0/3331076361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0efc90 con 0x55983044c920 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9af3e00640 1 -- 172.21.15.45:0/3331076361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec0f3de0 con 0x55983044c920 2024-04-25T10:29:06.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9af3e00640 1 -- 172.21.15.45:0/3331076361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0f83a0 con 0x55983044c920 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- 172.21.15.45:0/3331076361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 msgr2=0x5598304a8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x5598304a8740 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f9aec11d8b0 tx=0x7f9aec0ad040 comp rx=0 tx=0).stop 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- 172.21.15.45:0/3331076361 shutdown_connections 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x5598304a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x5598304a8740 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 --2- 172.21.15.45:0/3331076361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- 172.21.15.45:0/3331076361 >> 172.21.15.45:0/3331076361 conn(0x5598304092a0 msgr2=0x559830406220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- 172.21.15.45:0/3331076361 shutdown_connections 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- 172.21.15.45:0/3331076361 wait complete. 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.108+0000 7f9b0e488000 1 -- start start 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 Processor -- start 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x559830459110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x559830491dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598302563d0 con 0x55983044c920 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983030dd50 con 0x55983044d240 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598301ec980 con 0x559830402f30 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 10 client.0 _ll_register_callbacks cb 0x7f9af40077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x559830459110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x559830459110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51822/0 (socket says 172.21.15.45:51822) 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 -- 172.21.15.45:0/4123122790 learned_addr learned my addr 172.21.15.45:0/4123122790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b09000640 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x559830491dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 -- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 msgr2=0x5598304914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 -- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 msgr2=0x559830491dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x559830491dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 -- 172.21.15.45:0/4123122790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598302d3600 con 0x55983044c920 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9afb400640 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x559830459110 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f9aec11d900 tx=0x7f9aec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ac74f321b63a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 1 -- 172.21.15.45:0/4123122790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0f83a0 con 0x55983044c920 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 1 -- 172.21.15.45:0/4123122790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9aec0edf90 con 0x55983044c920 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9aec0f3de0 con 0x55983044c920 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 1 -- 172.21.15.45:0/4123122790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aec10b070 con 0x55983044c920 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 10 client.9720 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5598301e69b0 con 0x55983044c920 2024-04-25T10:29:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 1 -- 172.21.15.45:0/4123122790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec0f3de0 con 0x55983044c920 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 1 -- 172.21.15.45:0/4123122790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f9aec0efef0 con 0x55983044c920 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 10 client.9720 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 10 client.9720 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 20 client.9720 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408e80 0x559830492310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 4 client.9720 mds_command: new command op to 9531 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 --> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55983025f2e0 con 0x559830408e80 2024-04-25T10:29:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b03e00640 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408e80 0x559830492310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b03e00640 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408e80 0x559830492310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=753a39362e1d2dbf server_cookie=4956e724ab67cfe0 in_seq=0 out_seq=0 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 10 client.9720 ms_handle_connect on v2:172.21.15.45:6834/903752757 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 1 -- 172.21.15.45:0/4123122790 <== mds.0 v2:172.21.15.45:6834/903752757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55983025f2e0 con 0x559830408e80 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9af1600640 10 client.9720 handle_command_reply: tid=0 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.112+0000 7f9b0e488000 1 client.9720 shutdown 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 msgr2=0x559830459110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x559830459110 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f9aec11d900 tx=0x7f9aec01c400 comp rx=0 tx=0).stop 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 shutdown_connections 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:6834/903752757,v1:172.21.15.45:6835/903752757] conn(0x559830408e80 0x559830492310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55983044d240 0x559830491dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55983044c920 0x559830459110 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 --2- 172.21.15.45:0/4123122790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559830402f30 0x5598304914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9af1600640 0 client.9720 ms_handle_reset on v2:172.21.15.45:6834/903752757 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 >> 172.21.15.45:0/4123122790 conn(0x5598304092a0 msgr2=0x5598304547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 shutdown_connections 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 1 -- 172.21.15.45:0/4123122790 wait complete. 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b0e488000 20 client.9720 trim_cache size 0 max 16384 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9531, 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 29.094022808999998, 2024-04-25T10:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 107, 2024-04-25T10:29:06.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 143, 2024-04-25T10:29:06.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 143, 2024-04-25T10:29:06.121 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 33.916022239999997 2024-04-25T10:29:06.121 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:29:06.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ae0066f90 msgr2=0x7f9ae0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ae0066f90 0x7f9ae0069450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9b041f4950 tx=0x7f9af4009040 comp rx=0 tx=0).stop 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 msgr2=0x7f9b0409b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 0x7f9b0409b610 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f9aec01c230 tx=0x7f9ae8005840 comp rx=0 tx=0).stop 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 shutdown_connections 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ae0066f90 0x7f9ae0069450 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b0409eeb0 0x7f9b040c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b040ce2b0 0x7f9b0409b610 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 --2- 172.21.15.45:0/2908496176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae8004810 0x7f9b041f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 >> 172.21.15.45:0/2908496176 conn(0x7f9b040d17b0 msgr2=0x7f9b040b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 shutdown_connections 2024-04-25T10:29:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.116+0000 7f9b09a00640 1 -- 172.21.15.45:0/2908496176 wait complete. 2024-04-25T10:29:06.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub start abs_test_path recursive 2024-04-25T10:29:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 Processor -- start 2024-04-25T10:29:06.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 -- start start 2024-04-25T10:29:06.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fba640d00f0 0x7fba640713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba6415bc00 0x7fba640718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba64071f80 con 0x7fba640cf040 2024-04-25T10:29:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba64072260 con 0x7fba640d00f0 2024-04-25T10:29:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba6ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640b3700 con 0x7fba6415bc00 2024-04-25T10:29:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba63e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fba640cf040 0x7fba640cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40694/0 (socket says 172.21.15.45:40694) 2024-04-25T10:29:06.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba63e00640 1 -- 172.21.15.45:0/1191486027 learned_addr learned my addr 172.21.15.45:0/1191486027 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba68c00640 1 --2- 172.21.15.45:0/1191486027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba6415bc00 0x7fba640718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba5be00640 1 --2- 172.21.15.45:0/1191486027 >> v2:172.21.15.117:3300/0 conn(0x7fba640d00f0 0x7fba640713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba63400640 1 -- 172.21.15.45:0/1191486027 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949873106 0 0) 0x7fba64071f80 con 0x7fba640cf040 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba63400640 1 -- 172.21.15.45:0/1191486027 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba48003530 con 0x7fba640cf040 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba68c00640 1 -- 172.21.15.45:0/1191486027 >> v2:172.21.15.117:3300/0 conn(0x7fba640d00f0 msgr2=0x7fba640713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba68c00640 1 --2- 172.21.15.45:0/1191486027 >> v2:172.21.15.117:3300/0 conn(0x7fba640d00f0 0x7fba640713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba68c00640 1 -- 172.21.15.45:0/1191486027 >> v1:172.21.15.45:6789/0 conn(0x7fba640cf040 legacy=0x7fba640cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba63400640 1 -- 172.21.15.45:0/1191486027 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2804404946 0 0) 0x7fba48003530 con 0x7fba640cf040 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba68c00640 1 -- 172.21.15.45:0/1191486027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6415b120 con 0x7fba6415bc00 2024-04-25T10:29:06.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.280+0000 7fba5be00640 1 --2- 172.21.15.45:0/1191486027 >> v2:172.21.15.117:3300/0 conn(0x7fba640d00f0 0x7fba640713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:06.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba68c00640 1 --2- 172.21.15.45:0/1191486027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba6415bc00 0x7fba640718e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009d90 tx=0x7fba5c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31376222113a799c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba63400640 1 -- 172.21.15.45:0/1191486027 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0b2070 con 0x7fba6415bc00 2024-04-25T10:29:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba63400640 1 -- 172.21.15.45:0/1191486027 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0a5580 con 0x7fba6415bc00 2024-04-25T10:29:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba63400640 1 -- 172.21.15.45:0/1191486027 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba540016a0 con 0x7fba6415bc00 2024-04-25T10:29:06.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 -- 172.21.15.45:0/1191486027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba6415bc00 msgr2=0x7fba640718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 --2- 172.21.15.45:0/1191486027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba6415bc00 0x7fba640718e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009d90 tx=0x7fba5c01c890 comp rx=0 tx=0).stop 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 -- 172.21.15.45:0/1191486027 shutdown_connections 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 --2- 172.21.15.45:0/1191486027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba6415bc00 0x7fba640718e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 --2- 172.21.15.45:0/1191486027 >> v2:172.21.15.117:3300/0 conn(0x7fba640d00f0 0x7fba640713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 -- 172.21.15.45:0/1191486027 >> 172.21.15.45:0/1191486027 conn(0x7fba64066b70 msgr2=0x7fba64066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 -- 172.21.15.45:0/1191486027 shutdown_connections 2024-04-25T10:29:06.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.284+0000 7fba6ac00640 1 -- 172.21.15.45:0/1191486027 wait complete. 2024-04-25T10:29:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 Processor -- start 2024-04-25T10:29:06.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 -- start start 2024-04-25T10:29:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba640cf040 0x7fba64154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba640d00f0 0x7fba641489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba6415bc00 0x7fba641521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640aec90 con 0x7fba640d00f0 2024-04-25T10:29:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640cf630 con 0x7fba6415bc00 2024-04-25T10:29:06.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640aee10 con 0x7fba640cf040 2024-04-25T10:29:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba640d00f0 0x7fba641489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba640d00f0 0x7fba641489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51834/0 (socket says 172.21.15.45:51834) 2024-04-25T10:29:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 -- 172.21.15.45:0/483368796 learned_addr learned my addr 172.21.15.45:0/483368796 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba63e00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba640cf040 0x7fba64154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba68c00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba6415bc00 0x7fba641521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 -- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba640cf040 msgr2=0x7fba64154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba640cf040 0x7fba64154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 -- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba6415bc00 msgr2=0x7fba641521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba6415bc00 0x7fba641521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 -- 172.21.15.45:0/483368796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640cca00 con 0x7fba640d00f0 2024-04-25T10:29:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba5be00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fba640d00f0 0x7fba641489d0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00ea00 tx=0x7fba4c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba60c00640 1 -- 172.21.15.45:0/483368796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c017070 con 0x7fba640d00f0 2024-04-25T10:29:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba60c00640 1 -- 172.21.15.45:0/483368796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00ca60 con 0x7fba640d00f0 2024-04-25T10:29:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba60c00640 1 -- 172.21.15.45:0/483368796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00ce50 con 0x7fba640d00f0 2024-04-25T10:29:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 -- 172.21.15.45:0/483368796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640ccce0 con 0x7fba640d00f0 2024-04-25T10:29:06.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.288+0000 7fba6ac00640 1 -- 172.21.15.45:0/483368796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba640cd2a0 con 0x7fba640d00f0 2024-04-25T10:29:06.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.292+0000 7fba60c00640 1 -- 172.21.15.45:0/483368796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba4c01a070 con 0x7fba640d00f0 2024-04-25T10:29:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.292+0000 7fba60c00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba40066f60 0x7fba40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.292+0000 7fba60c00640 1 -- 172.21.15.45:0/483368796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fba4c084f30 con 0x7fba640d00f0 2024-04-25T10:29:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.292+0000 7fba6e06b000 1 Processor -- start 2024-04-25T10:29:06.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.292+0000 7fba63e00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba40066f60 0x7fba40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.292+0000 7fba6e06b000 1 -- start start 2024-04-25T10:29:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba63e00640 1 --2- 172.21.15.45:0/483368796 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fba40066f60 0x7fba40069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fba54000f70 tx=0x7fba54000ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f7aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f2e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f2ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d7ac30 con 0x5614d7f2e620 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d038c0 con 0x5614d7f7a6b0 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d76cd0 con 0x5614d7f7afd0 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f2ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f2ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51842/0 (socket says 172.21.15.45:51842) 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 -- 172.21.15.45:0/4102969150 learned_addr learned my addr 172.21.15.45:0/4102969150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba63e00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f7aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba5be00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f2e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 -- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 msgr2=0x5614d7f2e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f2e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 -- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 msgr2=0x5614d7f7aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f7aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 -- 172.21.15.45:0/4102969150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d7e03130 con 0x5614d7f2e620 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba5be00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f2e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba63e00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f7aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba68c00640 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f2ea00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009d90 tx=0x7fba5c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7052fe1af008489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba58c00640 1 -- 172.21.15.45:0/4102969150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0bf070 con 0x5614d7f2e620 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba58c00640 1 -- 172.21.15.45:0/4102969150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0a57a0 con 0x5614d7f2e620 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba58c00640 1 -- 172.21.15.45:0/4102969150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0b2040 con 0x5614d7f2e620 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- 172.21.15.45:0/4102969150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 msgr2=0x5614d7f2ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f2ea00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009d90 tx=0x7fba5c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- 172.21.15.45:0/4102969150 shutdown_connections 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f2ea00 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f2e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4102969150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f7aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- 172.21.15.45:0/4102969150 >> 172.21.15.45:0/4102969150 conn(0x5614d7e14c80 msgr2=0x5614d7f59420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- 172.21.15.45:0/4102969150 shutdown_connections 2024-04-25T10:29:06.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- 172.21.15.45:0/4102969150 wait complete. 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 -- start start 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.296+0000 7fba6e06b000 1 Processor -- start 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f59e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f79af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f7a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d23ad0 con 0x5614d7f2e620 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d35fa0 con 0x5614d7f7a6b0 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d46840 con 0x5614d7f7afd0 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 10 client.0 _ll_register_callbacks cb 0x5614d7f17b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f59e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f59e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51844/0 (socket says 172.21.15.45:51844) 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 -- 172.21.15.45:0/2642206050 learned_addr learned my addr 172.21.15.45:0/2642206050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 -- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 msgr2=0x5614d7f7a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba5be00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f79af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba68c00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f7a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f7a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 -- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 msgr2=0x5614d7f79af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f79af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 -- 172.21.15.45:0/2642206050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614d7dc0f90 con 0x5614d7f2e620 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba68c00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f7a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba5be00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f79af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba63e00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f59e50 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fba54015710 tx=0x7fba54001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca5211e07afb4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 1 -- 172.21.15.45:0/2642206050 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54012570 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 1 -- 172.21.15.45:0/2642206050 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fba54010040 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba5c0a57a0 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 1 -- 172.21.15.45:0/2642206050 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba54012710 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 10 client.9732 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5614d7d3f630 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 1 -- 172.21.15.45:0/2642206050 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0a57a0 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 1 -- 172.21.15.45:0/2642206050 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fba5402b050 con 0x5614d7f2e620 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 10 client.9732 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 10 client.9732 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 20 client.9732 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7f5f260 0x5614d7f8c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 4 client.9732 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614d7d16f80 con 0x5614d7f5f260 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba5be00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7f5f260 0x5614d7f8c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba5be00640 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7f5f260 0x5614d7f8c970 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=dc0806c035c9e755 server_cookie=38d8bc3056f3f521 in_seq=0 out_seq=0 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 10 client.9732 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 1 -- 172.21.15.45:0/2642206050 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614d7d16f80 con 0x5614d7f5f260 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba51600640 10 client.9732 handle_command_reply: tid=0 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.300+0000 7fba6e06b000 1 client.9732 shutdown 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 msgr2=0x5614d7f59e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f59e50 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fba54015710 tx=0x7fba54001260 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 shutdown_connections 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7f5f260 0x5614d7f8c970 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f7afd0 0x5614d7f7a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f79af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba51600640 0 client.9732 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 --2- 172.21.15.45:0/2642206050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f2e620 0x5614d7f59e50 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 >> 172.21.15.45:0/2642206050 conn(0x5614d7e14c80 msgr2=0x5614d7f293d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 shutdown_connections 2024-04-25T10:29:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 1 -- 172.21.15.45:0/2642206050 wait complete. 2024-04-25T10:29:06.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.304+0000 7fba6e06b000 20 client.9732 trim_cache size 0 max 16384 2024-04-25T10:29:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.316+0000 7fba6e06b000 1 Processor -- start 2024-04-25T10:29:06.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.316+0000 7fba6e06b000 1 -- start start 2024-04-25T10:29:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7f29380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f28430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 0x5614d7f28970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641fe1c0 con 0x5614d7f7afd0 2024-04-25T10:29:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641fe7f0 con 0x5614d7f7a6b0 2024-04-25T10:29:06.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640af2a0 con 0x5614d7f2e620 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 0x5614d7f28970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 0x5614d7f28970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51850/0 (socket says 172.21.15.45:51850) 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 -- 172.21.15.45:0/4008676339 learned_addr learned my addr 172.21.15.45:0/4008676339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 -- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 msgr2=0x5614d7f29380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7f29380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 -- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 msgr2=0x5614d7f28430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba5be00640 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f28430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f28430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 -- 172.21.15.45:0/4008676339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614d7d03b10 con 0x5614d7f7afd0 2024-04-25T10:29:06.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba68c00640 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 0x5614d7f28970 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fba5c004b00 tx=0x7fba5c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc2803aadd7b098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba53e00640 1 -- 172.21.15.45:0/4008676339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0b2050 con 0x5614d7f7afd0 2024-04-25T10:29:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba53e00640 1 -- 172.21.15.45:0/4008676339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0a5840 con 0x5614d7f7afd0 2024-04-25T10:29:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba53e00640 1 -- 172.21.15.45:0/4008676339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0bf040 con 0x5614d7f7afd0 2024-04-25T10:29:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- 172.21.15.45:0/4008676339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 msgr2=0x5614d7f28970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 0x5614d7f28970 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fba5c004b00 tx=0x7fba5c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- 172.21.15.45:0/4008676339 shutdown_connections 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7afd0 0x5614d7f28970 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7a6b0 0x5614d7f28430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- 172.21.15.45:0/4008676339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7f29380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- 172.21.15.45:0/4008676339 >> 172.21.15.45:0/4008676339 conn(0x5614d7e14c80 msgr2=0x5614d7f5e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- 172.21.15.45:0/4008676339 shutdown_connections 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- 172.21.15.45:0/4008676339 wait complete. 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- start start 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 Processor -- start 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7fdb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 0x5614d7fdbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7afd0 0x5614d7fd7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d16830 con 0x5614d7f7a6b0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640af8a0 con 0x5614d7f7afd0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.320+0000 7fba6e06b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614d7d32ab0 con 0x5614d7f2e620 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 10 client.0 _ll_register_callbacks cb 0x7fba48003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 0x5614d7fdbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 0x5614d7fdbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51860/0 (socket says 172.21.15.45:51860) 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 -- 172.21.15.45:0/1467332530 learned_addr learned my addr 172.21.15.45:0/1467332530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba63e00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7fdb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba68c00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7afd0 0x5614d7fd7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 -- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 msgr2=0x5614d7fdb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7fdb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 -- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7afd0 msgr2=0x5614d7fd7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7afd0 0x5614d7fd7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 -- 172.21.15.45:0/1467332530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614d7dd3cf0 con 0x5614d7f7a6b0 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba68c00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7afd0 0x5614d7fd7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba5be00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 0x5614d7fdbd70 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fba4c055840 tx=0x7fba4c055770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed80336141956bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 1 -- 172.21.15.45:0/1467332530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c021040 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 1 -- 172.21.15.45:0/1467332530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fba4c06d1d0 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba5c0a5840 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 1 -- 172.21.15.45:0/1467332530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba4c0830a0 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 10 client.9738 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5614d7b60a30 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 1 -- 172.21.15.45:0/1467332530 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c0a5840 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 1 -- 172.21.15.45:0/1467332530 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fba4c024060 con 0x5614d7f7a6b0 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 10 client.9738 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 10 client.9738 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 20 client.9738 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7fd8860 0x5614d7fd8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 4 client.9738 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:29:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}) v1 -- 0x5614d7d88170 con 0x5614d7fd8860 2024-04-25T10:29:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba63e00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7fd8860 0x5614d7fd8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba63e00640 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7fd8860 0x5614d7fd8c40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=6f455a69a751d960 server_cookie=507f67b8f6e8ffa5 in_seq=0 out_seq=0 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 10 client.9738 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 1 -- 172.21.15.45:0/1467332530 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x5614d7d88170 con 0x5614d7fd8860 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 10 client.9738 handle_command_reply: tid=0 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 client.9738 shutdown 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 msgr2=0x5614d7fdbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 0x5614d7fdbd70 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fba4c055840 tx=0x7fba4c055770 comp rx=0 tx=0).stop 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 shutdown_connections 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x5614d7fd8860 0x5614d7fd8c40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5614d7f7afd0 0x5614d7fd7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5614d7f7a6b0 0x5614d7fdbd70 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 --2- 172.21.15.45:0/1467332530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5614d7f2e620 0x5614d7fdb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba51600640 0 client.9738 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 >> 172.21.15.45:0/1467332530 conn(0x5614d7e14c80 msgr2=0x5614d7f5d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 shutdown_connections 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 1 -- 172.21.15.45:0/1467332530 wait complete. 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.324+0000 7fba6e06b000 20 client.9738 trim_cache size 0 max 16384 2024-04-25T10:29:06.331 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: MDS is not active 2024-04-25T10:29:06.333 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:29:06.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub pause 2024-04-25T10:29:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 Processor -- start 2024-04-25T10:29:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 -- start start 2024-04-25T10:29:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c50159640 0x7f5c500ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c500aebd0 0x7f5c50071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c50071360 con 0x7f5c50072020 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500af030 con 0x7f5c50159640 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c54c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5015c2f0 con 0x7f5c500aebd0 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c50159640 0x7f5c500ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c50159640 0x7f5c500ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51864/0 (socket says 172.21.15.45:51864) 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 -- 172.21.15.45:0/2007718941 learned_addr learned my addr 172.21.15.45:0/2007718941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c47e00640 1 --2- 172.21.15.45:0/2007718941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c500aebd0 0x7f5c50071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 -- 172.21.15.45:0/2007718941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c500aebd0 msgr2=0x7f5c50071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 --2- 172.21.15.45:0/2007718941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c500aebd0 0x7f5c50071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 -- 172.21.15.45:0/2007718941 >> v1:172.21.15.117:6789/0 conn(0x7f5c50072020 legacy=0x7f5c5015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.476+0000 7f5c4de00640 1 -- 172.21.15.45:0/2007718941 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5015c5d0 con 0x7f5c50159640 2024-04-25T10:29:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c4de00640 1 --2- 172.21.15.45:0/2007718941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c50159640 0x7f5c500ae690 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009870 tx=0x7f5c4001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc8f56a27aedca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c4d400640 1 -- 172.21.15.45:0/2007718941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400b0070 con 0x7f5c50159640 2024-04-25T10:29:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c4d400640 1 -- 172.21.15.45:0/2007718941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c400ab600 con 0x7f5c50159640 2024-04-25T10:29:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c4d400640 1 -- 172.21.15.45:0/2007718941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400ab9d0 con 0x7f5c50159640 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 -- 172.21.15.45:0/2007718941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c50159640 msgr2=0x7f5c500ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 --2- 172.21.15.45:0/2007718941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c50159640 0x7f5c500ae690 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009870 tx=0x7f5c4001c560 comp rx=0 tx=0).stop 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 -- 172.21.15.45:0/2007718941 shutdown_connections 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 --2- 172.21.15.45:0/2007718941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c500aebd0 0x7f5c50071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 --2- 172.21.15.45:0/2007718941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c50159640 0x7f5c500ae690 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 -- 172.21.15.45:0/2007718941 >> 172.21.15.45:0/2007718941 conn(0x7f5c500669a0 msgr2=0x7f5c50066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 -- 172.21.15.45:0/2007718941 shutdown_connections 2024-04-25T10:29:06.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.480+0000 7f5c54c00640 1 -- 172.21.15.45:0/2007718941 wait complete. 2024-04-25T10:29:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 Processor -- start 2024-04-25T10:29:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 -- start start 2024-04-25T10:29:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c50072020 0x7f5c501440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c500aebd0 0x7f5c500abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c50159640 0x7f5c501536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50072430 con 0x7f5c500aebd0 2024-04-25T10:29:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5006c0d0 con 0x7f5c50159640 2024-04-25T10:29:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50168930 con 0x7f5c50072020 2024-04-25T10:29:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c500aebd0 0x7f5c500abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c500aebd0 0x7f5c500abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51876/0 (socket says 172.21.15.45:51876) 2024-04-25T10:29:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 -- 172.21.15.45:0/3686342952 learned_addr learned my addr 172.21.15.45:0/3686342952 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c4de00640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c50072020 0x7f5c501440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c4e800640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c50159640 0x7f5c501536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 -- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c50072020 msgr2=0x7f5c501440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c50072020 0x7f5c501440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 -- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c50159640 msgr2=0x7f5c501536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c50159640 0x7f5c501536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 -- 172.21.15.45:0/3686342952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50154850 con 0x7f5c500aebd0 2024-04-25T10:29:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c47e00640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c500aebd0 0x7f5c500abd10 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e930 tx=0x7f5c3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c46000640 1 -- 172.21.15.45:0/3686342952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38017070 con 0x7f5c500aebd0 2024-04-25T10:29:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c46000640 1 -- 172.21.15.45:0/3686342952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800ca00 con 0x7f5c500aebd0 2024-04-25T10:29:06.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c46000640 1 -- 172.21.15.45:0/3686342952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800cdb0 con 0x7f5c500aebd0 2024-04-25T10:29:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 -- 172.21.15.45:0/3686342952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c50154290 con 0x7f5c500aebd0 2024-04-25T10:29:06.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.484+0000 7f5c54c00640 1 -- 172.21.15.45:0/3686342952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c501558a0 con 0x7f5c500aebd0 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c46000640 1 -- 172.21.15.45:0/3686342952 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c3801a070 con 0x7f5c500aebd0 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c46000640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c2c066f60 0x7f5c2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c46000640 1 -- 172.21.15.45:0/3686342952 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5c38085330 con 0x7f5c500aebd0 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 Processor -- start 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 -- start start 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c4de00640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c2c066f60 0x7f5c2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb66fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb6254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb625dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfb50cad0 con 0x55abfb6259e0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfb473f80 con 0x55abfb6700d0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abfb3fbee0 con 0x55abfb66f7b0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.488+0000 7f5c4de00640 1 --2- 172.21.15.45:0/3686342952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5c2c066f60 0x7f5c2c069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5c400ad000 tx=0x7f5c400a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb625dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c47e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb6254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb625dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51884/0 (socket says 172.21.15.45:51884) 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 -- 172.21.15.45:0/651499548 learned_addr learned my addr 172.21.15.45:0/651499548 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4de00640 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb66fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 -- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 msgr2=0x55abfb66fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb66fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 -- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 msgr2=0x55abfb6254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb6254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 -- 172.21.15.45:0/651499548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abfb42c930 con 0x55abfb6259e0 2024-04-25T10:29:06.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4de00640 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb66fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c4e800640 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb625dc0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001f50 tx=0x7f5c480016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6da9e703a492d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c3ea00640 1 -- 172.21.15.45:0/651499548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800fdf0 con 0x55abfb6259e0 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c3ea00640 1 -- 172.21.15.45:0/651499548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48018070 con 0x55abfb6259e0 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c3ea00640 1 -- 172.21.15.45:0/651499548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48003b80 con 0x55abfb6259e0 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- 172.21.15.45:0/651499548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 msgr2=0x55abfb625dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb625dc0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001f50 tx=0x7f5c480016b0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- 172.21.15.45:0/651499548 shutdown_connections 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb625dc0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb6254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- 172.21.15.45:0/651499548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb66fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- 172.21.15.45:0/651499548 >> 172.21.15.45:0/651499548 conn(0x55abfb62bf60 msgr2=0x55abfb623d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- 172.21.15.45:0/651499548 shutdown_connections 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- 172.21.15.45:0/651499548 wait complete. 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- start start 2024-04-25T10:29:06.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 Processor -- start 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb66f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb62aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb62b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfb42e390 con 0x55abfb6259e0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfb470020 con 0x55abfb6700d0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abfb470ff0 con 0x55abfb66f7b0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.492+0000 7f5c580ad000 10 client.0 _ll_register_callbacks cb 0x55abfb5276a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb66f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb66f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51894/0 (socket says 172.21.15.45:51894) 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 -- 172.21.15.45:0/784363844 learned_addr learned my addr 172.21.15.45:0/784363844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c47e00640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb62aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4e800640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb62b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 -- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 msgr2=0x55abfb62aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb62aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 -- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 msgr2=0x55abfb62b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb62b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 -- 172.21.15.45:0/784363844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abfb3fc130 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c4de00640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb66f200 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f5c400ae040 tx=0x7f5c4001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87867d7aff188d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 1 -- 172.21.15.45:0/784363844 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400c7070 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 1 -- 172.21.15.45:0/784363844 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5c400abd70 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c48018070 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 1 -- 172.21.15.45:0/784363844 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c400b0040 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 10 client.9750 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55abfb42eaa0 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 1 -- 172.21.15.45:0/784363844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48018070 con 0x55abfb6259e0 2024-04-25T10:29:06.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 1 -- 172.21.15.45:0/784363844 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f5c400bf040 con 0x55abfb6259e0 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 10 client.9750 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 10 client.9750 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 20 client.9750 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb631ff0 0x55abfb681860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 4 client.9750 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abfb40fa30 con 0x55abfb631ff0 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c47e00640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb631ff0 0x55abfb681860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c47e00640 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb631ff0 0x55abfb681860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=8cf9d68218fee3ba server_cookie=5930a226ab0a365c in_seq=0 out_seq=0 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 10 client.9750 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 1 -- 172.21.15.45:0/784363844 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abfb40fa30 con 0x55abfb631ff0 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 10 client.9750 handle_command_reply: tid=0 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 client.9750 shutdown 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 msgr2=0x55abfb66f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb66f200 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f5c400ae040 tx=0x7f5c4001ada0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 shutdown_connections 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb631ff0 0x55abfb681860 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb62b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb62aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 --2- 172.21.15.45:0/784363844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb66f200 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c33400640 0 client.9750 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.496+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 >> 172.21.15.45:0/784363844 conn(0x55abfb62bf60 msgr2=0x55abfb624490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.500+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 shutdown_connections 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.500+0000 7f5c580ad000 1 -- 172.21.15.45:0/784363844 wait complete. 2024-04-25T10:29:06.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.500+0000 7f5c580ad000 20 client.9750 trim_cache size 0 max 16384 2024-04-25T10:29:06.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 Processor -- start 2024-04-25T10:29:06.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 -- start start 2024-04-25T10:29:06.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb683bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb680860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb680da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50200840 con 0x55abfb6259e0 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50200db0 con 0x55abfb6700d0 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501567a0 con 0x55abfb66f7b0 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c4de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb683bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c4de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb683bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51902/0 (socket says 172.21.15.45:51902) 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c4de00640 1 -- 172.21.15.45:0/1793424459 learned_addr learned my addr 172.21.15.45:0/1793424459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c4e800640 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb680da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.512+0000 7f5c47e00640 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb680860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 msgr2=0x55abfb680860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb680860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 msgr2=0x55abfb680da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb680da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/1793424459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abfb4f5af0 con 0x55abfb6259e0 2024-04-25T10:29:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb683bf0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f5c4001c5a0 tx=0x7f5c400c1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c937136158fb759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c3e000640 1 -- 172.21.15.45:0/1793424459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400c7030 con 0x55abfb6259e0 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c3e000640 1 -- 172.21.15.45:0/1793424459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c400abd70 con 0x55abfb6259e0 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c3e000640 1 -- 172.21.15.45:0/1793424459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400c1850 con 0x55abfb6259e0 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- 172.21.15.45:0/1793424459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 msgr2=0x55abfb683bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb683bf0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f5c4001c5a0 tx=0x7f5c400c1000 comp rx=0 tx=0).stop 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- 172.21.15.45:0/1793424459 shutdown_connections 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb6700d0 0x55abfb680da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb66f7b0 0x55abfb680860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- 172.21.15.45:0/1793424459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb683bf0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- 172.21.15.45:0/1793424459 >> 172.21.15.45:0/1793424459 conn(0x55abfb62bf60 msgr2=0x55abfb630770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- 172.21.15.45:0/1793424459 shutdown_connections 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- 172.21.15.45:0/1793424459 wait complete. 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- start start 2024-04-25T10:29:06.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 Processor -- start 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb6cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb66f7b0 0x55abfb6ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb6700d0 0x55abfb6ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abfb473290 con 0x55abfb6259e0 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501f8730 con 0x55abfb66f7b0 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abfb4401f0 con 0x55abfb6700d0 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c580ad000 10 client.0 _ll_register_callbacks cb 0x55abfb47a450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb6cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb6cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51912/0 (socket says 172.21.15.45:51912) 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/3723605671 learned_addr learned my addr 172.21.15.45:0/3723605671 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c47e00640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb66f7b0 0x55abfb6ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4e800640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb6700d0 0x55abfb6ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb6700d0 msgr2=0x55abfb6ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb6700d0 0x55abfb6ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb66f7b0 msgr2=0x55abfb6ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb66f7b0 0x55abfb6ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4de00640 1 -- 172.21.15.45:0/3723605671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abfb4cbd50 con 0x55abfb6259e0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.516+0000 7f5c4e800640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb6700d0 0x55abfb6ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c4de00640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb6cd380 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5c4001ca40 tx=0x7f5c40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bc33dfaef3136b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 1 -- 172.21.15.45:0/3723605671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400c7040 con 0x55abfb6259e0 2024-04-25T10:29:06.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c400abd70 con 0x55abfb6259e0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 1 -- 172.21.15.45:0/3723605671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5c400d6440 con 0x55abfb6259e0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 1 -- 172.21.15.45:0/3723605671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c400c19a0 con 0x55abfb6259e0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 10 client.9756 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55abfb41ae90 con 0x55abfb6259e0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 1 -- 172.21.15.45:0/3723605671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c400abd70 con 0x55abfb6259e0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 1 -- 172.21.15.45:0/3723605671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f5c400d4070 con 0x55abfb6259e0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 10 client.9756 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 10 client.9756 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 20 client.9756 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb6cee10 0x55abfb6d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 4 client.9756 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55abfb4814c0 con 0x55abfb6cee10 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c47e00640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb6cee10 0x55abfb6d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c47e00640 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb6cee10 0x55abfb6d0500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=868d280e204f139b server_cookie=95e07653af719590 in_seq=0 out_seq=0 2024-04-25T10:29:06.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 10 client.9756 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 1 -- 172.21.15.45:0/3723605671 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x55abfb4814c0 con 0x55abfb6cee10 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 10 client.9756 handle_command_reply: tid=0 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 client.9756 shutdown 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 msgr2=0x55abfb6cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb6cd380 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5c4001ca40 tx=0x7f5c40008040 comp rx=0 tx=0).stop 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 shutdown_connections 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55abfb6cee10 0x55abfb6d0500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55abfb6700d0 0x55abfb6ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55abfb66f7b0 0x55abfb6ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 --2- 172.21.15.45:0/3723605671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abfb6259e0 0x55abfb6cd380 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 >> 172.21.15.45:0/3723605671 conn(0x55abfb62bf60 msgr2=0x55abfb631000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c33400640 0 client.9756 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 shutdown_connections 2024-04-25T10:29:06.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 1 -- 172.21.15.45:0/3723605671 wait complete. 2024-04-25T10:29:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.520+0000 7f5c580ad000 20 client.9756 trim_cache size 0 max 16384 2024-04-25T10:29:06.526 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: MDS is not active 2024-04-25T10:29:06.528 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:29:06.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub abort 2024-04-25T10:29:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.668+0000 7fe4d8400640 1 Processor -- start 2024-04-25T10:29:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.668+0000 7fe4d8400640 1 -- start start 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d8400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4d006d650 0x7fe4d00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d8400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d00cdab0 con 0x7fe4d00d0d00 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d8400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d00cd210 con 0x7fe4d00cc200 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d8400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d00cd4f0 con 0x7fe4d006d650 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d5a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe4d00d0d00 0x7fe4d0162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40700/0 (socket says 172.21.15.45:40700) 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d5a00640 1 -- 172.21.15.45:0/1258418818 learned_addr learned my addr 172.21.15.45:0/1258418818 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d6400640 1 --2- 172.21.15.45:0/1258418818 >> v2:172.21.15.45:3300/0 conn(0x7fe4d006d650 0x7fe4d00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d6400640 1 -- 172.21.15.45:0/1258418818 >> v1:172.21.15.117:6789/0 conn(0x7fe4d00cc200 legacy=0x7fe4d006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d6400640 1 -- 172.21.15.45:0/1258418818 >> v1:172.21.15.45:6789/0 conn(0x7fe4d00d0d00 legacy=0x7fe4d0162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4d6400640 1 -- 172.21.15.45:0/1258418818 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00cd7d0 con 0x7fe4d006d650 2024-04-25T10:29:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.672+0000 7fe4cfe00640 1 -- 172.21.15.45:0/1258418818 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101910751 0 0) 0x7fe4d00cdab0 con 0x7fe4d00d0d00 2024-04-25T10:29:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d6400640 1 --2- 172.21.15.45:0/1258418818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d006d650 0x7fe4d00ccc30 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009d90 tx=0x7fe4c801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412b68470ce9c2f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4cfe00640 1 -- 172.21.15.45:0/1258418818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b2070 con 0x7fe4d006d650 2024-04-25T10:29:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4cfe00640 1 -- 172.21.15.45:0/1258418818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80a5870 con 0x7fe4d006d650 2024-04-25T10:29:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4cfe00640 1 -- 172.21.15.45:0/1258418818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a5c40 con 0x7fe4d006d650 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- 172.21.15.45:0/1258418818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d006d650 msgr2=0x7fe4d00ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 --2- 172.21.15.45:0/1258418818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d006d650 0x7fe4d00ccc30 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009d90 tx=0x7fe4c801c8a0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- 172.21.15.45:0/1258418818 shutdown_connections 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 --2- 172.21.15.45:0/1258418818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d006d650 0x7fe4d00ccc30 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- 172.21.15.45:0/1258418818 >> 172.21.15.45:0/1258418818 conn(0x7fe4d0072290 msgr2=0x7fe4d00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- 172.21.15.45:0/1258418818 shutdown_connections 2024-04-25T10:29:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- 172.21.15.45:0/1258418818 wait complete. 2024-04-25T10:29:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 Processor -- start 2024-04-25T10:29:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- start start 2024-04-25T10:29:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d006d650 0x7fe4d00c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d00cc200 0x7fe4d00963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d00d0d00 0x7fe4d00968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d016c000 con 0x7fe4d00cc200 2024-04-25T10:29:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d016be80 con 0x7fe4d00d0d00 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d8400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d016c180 con 0x7fe4d006d650 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d00cc200 0x7fe4d00963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d00cc200 0x7fe4d00963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51936/0 (socket says 172.21.15.45:51936) 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.676+0000 7fe4d5000640 1 -- 172.21.15.45:0/3682088455 learned_addr learned my addr 172.21.15.45:0/3682088455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d006d650 0x7fe4d00c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d6400640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d00d0d00 0x7fe4d00968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5000640 1 -- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d006d650 msgr2=0x7fe4d00c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5000640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4d006d650 0x7fe4d00c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5000640 1 -- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d00d0d00 msgr2=0x7fe4d00968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5000640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d00d0d00 0x7fe4d00968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5000640 1 -- 172.21.15.45:0/3682088455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0161300 con 0x7fe4d00cc200 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d6400640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4d00d0d00 0x7fe4d00968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d5000640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d00cc200 0x7fe4d00963a0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c450 tx=0x7fe4bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4cd600640 1 -- 172.21.15.45:0/3682088455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc012830 con 0x7fe4d00cc200 2024-04-25T10:29:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4cd600640 1 -- 172.21.15.45:0/3682088455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc017070 con 0x7fe4d00cc200 2024-04-25T10:29:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d8400640 1 -- 172.21.15.45:0/3682088455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d00c3c80 con 0x7fe4d00cc200 2024-04-25T10:29:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4cd600640 1 -- 172.21.15.45:0/3682088455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc012c30 con 0x7fe4d00cc200 2024-04-25T10:29:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4d8400640 1 -- 172.21.15.45:0/3682088455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d00c6b60 con 0x7fe4d00cc200 2024-04-25T10:29:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.680+0000 7fe4cd600640 1 -- 172.21.15.45:0/3682088455 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4bc01c460 con 0x7fe4d00cc200 2024-04-25T10:29:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4cd600640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4b0066f60 0x7fe4b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4cd600640 1 -- 172.21.15.45:0/3682088455 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4bc084f50 con 0x7fe4d00cc200 2024-04-25T10:29:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4b0066f60 0x7fe4b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 Processor -- start 2024-04-25T10:29:06.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 -- start start 2024-04-25T10:29:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/3682088455 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe4b0066f60 0x7fe4b0069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4000f70 tx=0x7fe4c4007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3620b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db35d64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db34bdad0 con 0x555db35d69e0 2024-04-25T10:29:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db3424f80 con 0x555db36207b0 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db33acee0 con 0x555db36210d0 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35d6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51942/0 (socket says 172.21.15.45:51942) 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 -- 172.21.15.45:0/911550680 learned_addr learned my addr 172.21.15.45:0/911550680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d5000640 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db35d64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3620b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 -- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 msgr2=0x555db35d64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db35d64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 -- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 msgr2=0x555db3620b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3620b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 -- 172.21.15.45:0/911550680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db33dd930 con 0x555db35d69e0 2024-04-25T10:29:06.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4d6400640 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35d6dc0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80194b0 tx=0x7fe4c8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b46d03b3f081856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4c2000640 1 -- 172.21.15.45:0/911550680 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b2040 con 0x555db35d69e0 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4c2000640 1 -- 172.21.15.45:0/911550680 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80a5ab0 con 0x555db35d69e0 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4c2000640 1 -- 172.21.15.45:0/911550680 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b9460 con 0x555db35d69e0 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 -- 172.21.15.45:0/911550680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 msgr2=0x555db35d6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.684+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35d6dc0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80194b0 tx=0x7fe4c8004a50 comp rx=0 tx=0).stop 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- 172.21.15.45:0/911550680 shutdown_connections 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35d6dc0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db35d64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/911550680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3620b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- 172.21.15.45:0/911550680 >> 172.21.15.45:0/911550680 conn(0x555db35dcf60 msgr2=0x555db35d4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- 172.21.15.45:0/911550680 shutdown_connections 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- 172.21.15.45:0/911550680 wait complete. 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- start start 2024-04-25T10:29:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 Processor -- start 2024-04-25T10:29:06.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db35e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db35e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db33df390 con 0x555db36210d0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db3421020 con 0x555db36207b0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db3421ff0 con 0x555db35d69e0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 10 client.0 _ll_register_callbacks cb 0x555db34d86a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db35e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db35e23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51952/0 (socket says 172.21.15.45:51952) 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d5a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db35e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 -- 172.21.15.45:0/325923842 learned_addr learned my addr 172.21.15.45:0/325923842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d5000640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 -- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 msgr2=0x555db35e1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db35e1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 -- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 msgr2=0x555db35e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 -- 172.21.15.45:0/325923842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db33ad130 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d5000640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4d6400640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db35e23d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80194b0 tx=0x7fe4c8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f954b46e6ad6929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4b6a00640 1 -- 172.21.15.45:0/325923842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b2040 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c80a5ab0 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4b6a00640 1 -- 172.21.15.45:0/325923842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4c8003ba0 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4b6a00640 1 -- 172.21.15.45:0/325923842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c80ce050 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 10 client.9771 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x555db33dfaa0 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.688+0000 7fe4b6a00640 1 -- 172.21.15.45:0/325923842 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80a5ab0 con 0x555db36210d0 2024-04-25T10:29:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4b6a00640 1 -- 172.21.15.45:0/325923842 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fe4c80d1020 con 0x555db36210d0 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 10 client.9771 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 10 client.9771 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 20 client.9771 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db362fa50 0x555db3631e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 4 client.9771 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555db33c0a30 con 0x555db362fa50 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db362fa50 0x555db3631e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db362fa50 0x555db3631e40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=7c18a3b27ae570a2 server_cookie=f49304336da95e65 in_seq=0 out_seq=0 2024-04-25T10:29:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4b6a00640 10 client.9771 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4b6a00640 1 -- 172.21.15.45:0/325923842 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555db33c0a30 con 0x555db362fa50 2024-04-25T10:29:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4b6a00640 10 client.9771 handle_command_reply: tid=0 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 client.9771 shutdown 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 msgr2=0x555db35e23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db35e23d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80194b0 tx=0x7fe4c8019640 comp rx=0 tx=0).stop 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 shutdown_connections 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db362fa50 0x555db3631e40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db35e23d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/325923842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db35e1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 >> 172.21.15.45:0/325923842 conn(0x555db35dcf60 msgr2=0x555db35d5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4b6a00640 0 client.9771 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 shutdown_connections 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 1 -- 172.21.15.45:0/325923842 wait complete. 2024-04-25T10:29:06.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.692+0000 7fe4db7dd000 20 client.9771 trim_cache size 0 max 16384 2024-04-25T10:29:06.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 Processor -- start 2024-04-25T10:29:06.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- start start 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db3630a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0202be0 con 0x555db35d69e0 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d02013d0 con 0x555db36207b0 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0201850 con 0x555db36210d0 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35dbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51960/0 (socket says 172.21.15.45:51960) 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 -- 172.21.15.45:0/1910933572 learned_addr learned my addr 172.21.15.45:0/1910933572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d6400640 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db3630a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5000640 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 -- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 msgr2=0x555db3630a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db3630a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 -- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 msgr2=0x555db35dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 -- 172.21.15.45:0/1910933572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db3620f00 con 0x555db35d69e0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5000640 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35dbb50 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4005f70 tx=0x7fe4c4009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c121077fba7e1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4c1600640 1 -- 172.21.15.45:0/1910933572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4027e60 con 0x555db35d69e0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4c1600640 1 -- 172.21.15.45:0/1910933572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c40270b0 con 0x555db35d69e0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4c1600640 1 -- 172.21.15.45:0/1910933572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4027260 con 0x555db35d69e0 2024-04-25T10:29:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- 172.21.15.45:0/1910933572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 msgr2=0x555db35dbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35dbb50 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4005f70 tx=0x7fe4c4009030 comp rx=0 tx=0).stop 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- 172.21.15.45:0/1910933572 shutdown_connections 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db36210d0 0x555db3630a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db35dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/1910933572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db35d69e0 0x555db35dbb50 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- 172.21.15.45:0/1910933572 >> 172.21.15.45:0/1910933572 conn(0x555db35dcf60 msgr2=0x555db361f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- 172.21.15.45:0/1910933572 shutdown_connections 2024-04-25T10:29:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.708+0000 7fe4db7dd000 1 -- 172.21.15.45:0/1910933572 wait complete. 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- start start 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 Processor -- start 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db3633ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3663f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db36644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db33cba50 con 0x555db36210d0 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db34e1230 con 0x555db36207b0 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x555db35dff30 con 0x555db35d69e0 2024-04-25T10:29:06.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 10 client.0 _ll_register_callbacks cb 0x555db342b450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db36644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d5a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db3633ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db36644a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51964/0 (socket says 172.21.15.45:51964) 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 -- 172.21.15.45:0/2235745235 learned_addr learned my addr 172.21.15.45:0/2235745235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d5000640 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3663f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 -- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 msgr2=0x555db3633ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db3633ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 -- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 msgr2=0x555db3663f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3663f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 -- 172.21.15.45:0/2235745235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db34a6af0 con 0x555db36210d0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d6400640 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db36644a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80071c0 tx=0x7fe4c8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81593df40e8214c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4b6a00640 1 -- 172.21.15.45:0/2235745235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80b2040 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4b6a00640 1 -- 172.21.15.45:0/2235745235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4c80a5350 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c40270b0 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4b6a00640 1 -- 172.21.15.45:0/2235745235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c80039a0 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 10 client.9777 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x555db347cd50 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4b6a00640 1 -- 172.21.15.45:0/2235745235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c40270b0 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4b6a00640 1 -- 172.21.15.45:0/2235745235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fe4c80d1040 con 0x555db36210d0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 10 client.9777 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 10 client.9777 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 20 client.9777 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db3664d60 0x555db3665140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 4 client.9777 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x555db34324c0 con 0x555db3664d60 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.712+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db3664d60 0x555db3665140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4d5a00640 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db3664d60 0x555db3665140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=ca72abd8b318d63b server_cookie=350117efe191750b in_seq=0 out_seq=0 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4b6a00640 10 client.9777 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4b6a00640 1 -- 172.21.15.45:0/2235745235 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x555db34324c0 con 0x555db3664d60 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4b6a00640 10 client.9777 handle_command_reply: tid=0 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 client.9777 shutdown 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 msgr2=0x555db36644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db36644a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80071c0 tx=0x7fe4c8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 shutdown_connections 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x555db3664d60 0x555db3665140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555db36210d0 0x555db36644a0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555db36207b0 0x555db3663f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4b6a00640 0 client.9777 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 --2- 172.21.15.45:0/2235745235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x555db35d69e0 0x555db3633ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 >> 172.21.15.45:0/2235745235 conn(0x555db35dcf60 msgr2=0x555db35dc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 shutdown_connections 2024-04-25T10:29:06.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 1 -- 172.21.15.45:0/2235745235 wait complete. 2024-04-25T10:29:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.716+0000 7fe4db7dd000 20 client.9777 trim_cache size 0 max 16384 2024-04-25T10:29:06.721 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: MDS is not active 2024-04-25T10:29:06.723 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:29:06.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d scrub resume 2024-04-25T10:29:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.864+0000 7fcd39c00640 1 Processor -- start 2024-04-25T10:29:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.864+0000 7fcd39c00640 1 -- start start 2024-04-25T10:29:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd39c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd3406bd10 0x7fcd34161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd39c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd34161e50 0x7fcd3415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd39c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3406ac90 con 0x7fcd340cd9a0 2024-04-25T10:29:06.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd39c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3406b880 con 0x7fcd34161e50 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd39c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3406af50 con 0x7fcd3406bd10 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd34161e50 0x7fcd3415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd34161e50 0x7fcd3415d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51980/0 (socket says 172.21.15.45:51980) 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 -- 172.21.15.45:0/1374369866 learned_addr learned my addr 172.21.15.45:0/1374369866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd2a000640 1 --2- 172.21.15.45:0/1374369866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd3406bd10 0x7fcd34161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 -- 172.21.15.45:0/1374369866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd3406bd10 msgr2=0x7fcd34161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 --2- 172.21.15.45:0/1374369866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd3406bd10 0x7fcd34161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 -- 172.21.15.45:0/1374369866 >> v1:172.21.15.117:6789/0 conn(0x7fcd340cd9a0 legacy=0x7fcd340d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 -- 172.21.15.45:0/1374369866 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3406b230 con 0x7fcd34161e50 2024-04-25T10:29:06.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.868+0000 7fcd33400640 1 --2- 172.21.15.45:0/1374369866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd3415d610 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009870 tx=0x7fcd2c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cc0c0fab28bb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd32000640 1 -- 172.21.15.45:0/1374369866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b5070 con 0x7fcd34161e50 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd32000640 1 -- 172.21.15.45:0/1374369866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0a8a50 con 0x7fcd34161e50 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd32000640 1 -- 172.21.15.45:0/1374369866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0a8e20 con 0x7fcd34161e50 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- 172.21.15.45:0/1374369866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 msgr2=0x7fcd3415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 --2- 172.21.15.45:0/1374369866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd3415d610 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009870 tx=0x7fcd2c01ed50 comp rx=0 tx=0).stop 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- 172.21.15.45:0/1374369866 shutdown_connections 2024-04-25T10:29:06.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 --2- 172.21.15.45:0/1374369866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd3415d610 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 --2- 172.21.15.45:0/1374369866 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd3406bd10 0x7fcd34161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- 172.21.15.45:0/1374369866 >> 172.21.15.45:0/1374369866 conn(0x7fcd340d19a0 msgr2=0x7fcd340d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- 172.21.15.45:0/1374369866 shutdown_connections 2024-04-25T10:29:06.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- 172.21.15.45:0/1374369866 wait complete. 2024-04-25T10:29:06.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 Processor -- start 2024-04-25T10:29:06.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- start start 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd3406bd10 0x7fcd3415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd340cd9a0 0x7fcd340cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd340cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34162660 con 0x7fcd34161e50 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340cdd80 con 0x7fcd3406bd10 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd39c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340d0d70 con 0x7fcd340cd9a0 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd340cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd340cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51996/0 (socket says 172.21.15.45:51996) 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.872+0000 7fcd33400640 1 -- 172.21.15.45:0/59092338 learned_addr learned my addr 172.21.15.45:0/59092338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd2a000640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd340cd9a0 0x7fcd340cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd32a00640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd3406bd10 0x7fcd3415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd33400640 1 -- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd340cd9a0 msgr2=0x7fcd340cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd33400640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd340cd9a0 0x7fcd340cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd33400640 1 -- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd3406bd10 msgr2=0x7fcd3415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd33400640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd3406bd10 0x7fcd3415ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd33400640 1 -- 172.21.15.45:0/59092338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd341f39d0 con 0x7fcd34161e50 2024-04-25T10:29:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd33400640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd34161e50 0x7fcd340cb6a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c01c980 tx=0x7fcd2c01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd2b400640 1 -- 172.21.15.45:0/59092338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0c0070 con 0x7fcd34161e50 2024-04-25T10:29:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd2b400640 1 -- 172.21.15.45:0/59092338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0b0ba0 con 0x7fcd34161e50 2024-04-25T10:29:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd39c00640 1 -- 172.21.15.45:0/59092338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd341f3cb0 con 0x7fcd34161e50 2024-04-25T10:29:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd2b400640 1 -- 172.21.15.45:0/59092338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b5040 con 0x7fcd34161e50 2024-04-25T10:29:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd39c00640 1 -- 172.21.15.45:0/59092338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3415d390 con 0x7fcd34161e50 2024-04-25T10:29:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.876+0000 7fcd2b400640 1 -- 172.21.15.45:0/59092338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd2c0c3070 con 0x7fcd34161e50 2024-04-25T10:29:06.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2b400640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd10066fb0 0x7fcd10069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2b400640 1 -- 172.21.15.45:0/59092338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcd2c122480 con 0x7fcd34161e50 2024-04-25T10:29:06.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd32a00640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd10066fb0 0x7fcd10069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 Processor -- start 2024-04-25T10:29:06.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- start start 2024-04-25T10:29:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd32a00640 1 --2- 172.21.15.45:0/59092338 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd10066fb0 0x7fcd10069470 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcd340caf50 tx=0x7fcd24009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f2624e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f25e0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f24c21f0 con 0x55e4f2625360 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2428e90 con 0x55e4f2624a40 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f23b0f50 con 0x55e4f25dfd90 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25df850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52006/0 (socket says 172.21.15.45:52006) 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 -- 172.21.15.45:0/1299906606 learned_addr learned my addr 172.21.15.45:0/1299906606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd32a00640 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f2624e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 -- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 msgr2=0x55e4f25e0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd33400640 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f25e0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f25e0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 -- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 msgr2=0x55e4f2624e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f2624e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 -- 172.21.15.45:0/1299906606 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f24008b0 con 0x55e4f2625360 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd32a00640 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f2624e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd2a000640 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25df850 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c007900 tx=0x7fcd1c007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe77baa0d3202cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd23400640 1 -- 172.21.15.45:0/1299906606 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c018070 con 0x55e4f2625360 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd23400640 1 -- 172.21.15.45:0/1299906606 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c010b00 con 0x55e4f2625360 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd23400640 1 -- 172.21.15.45:0/1299906606 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c014430 con 0x55e4f2625360 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/1299906606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 msgr2=0x55e4f25df850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25df850 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c007900 tx=0x7fcd1c007dc0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/1299906606 shutdown_connections 2024-04-25T10:29:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f25e0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25df850 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/1299906606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f2624e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/1299906606 >> 172.21.15.45:0/1299906606 conn(0x55e4f25dbd40 msgr2=0x55e4f25de130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/1299906606 shutdown_connections 2024-04-25T10:29:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.880+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/1299906606 wait complete. 2024-04-25T10:29:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- start start 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 Processor -- start 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f262c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f262c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f23e3540 con 0x55e4f2625360 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2424f30 con 0x55e4f2624a40 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2407010 con 0x55e4f25dfd90 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 10 client.0 _ll_register_callbacks cb 0x55e4f24dd180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25e5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52020/0 (socket says 172.21.15.45:52020) 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 -- 172.21.15.45:0/3351888808 learned_addr learned my addr 172.21.15.45:0/3351888808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd2a000640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f262c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd32a00640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f262c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 -- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 msgr2=0x55e4f262c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f262c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 -- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 msgr2=0x55e4f262c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f262c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 -- 172.21.15.45:0/3351888808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4f23b11a0 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd33400640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25e5d80 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c01e7c0 tx=0x7fcd2c0f29e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7005f11b6c1b864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd17e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd17e00640 1 -- 172.21.15.45:0/3351888808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0fb440 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd17e00640 1 -- 172.21.15.45:0/3351888808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcd2c0ef450 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd1c010b00 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd17e00640 1 -- 172.21.15.45:0/3351888808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2c1220b0 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 10 client.9789 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55e4f24ab840 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd17e00640 1 -- 172.21.15.45:0/3351888808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c010b00 con 0x55e4f2625360 2024-04-25T10:29:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd17e00640 1 -- 172.21.15.45:0/3351888808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fcd2c0fafb0 con 0x55e4f2625360 2024-04-25T10:29:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 10 client.9789 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 10 client.9789 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 20 client.9789 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f25e6b20 0x55e4f25e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 4 client.9789 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4f23c4be0 con 0x55e4f25e6b20 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.884+0000 7fcd32a00640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f25e6b20 0x55e4f25e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd32a00640 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f25e6b20 0x55e4f25e6f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=4796ba33d2db7d6a server_cookie=a8099a7d7cd79a55 in_seq=0 out_seq=0 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd17e00640 10 client.9789 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd17e00640 1 -- 172.21.15.45:0/3351888808 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4f23c4be0 con 0x55e4f25e6b20 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd17e00640 10 client.9789 handle_command_reply: tid=0 2024-04-25T10:29:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 client.9789 shutdown 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 msgr2=0x55e4f25e5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25e5d80 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c01e7c0 tx=0x7fcd2c0f29e0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 shutdown_connections 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f25e6b20 0x55e4f25e6f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2625360 0x55e4f25e5d80 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2624a40 0x55e4f262c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd17e00640 0 client.9789 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/3351888808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f262c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 >> 172.21.15.45:0/3351888808 conn(0x55e4f25dbd40 msgr2=0x55e4f25de9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 shutdown_connections 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/3351888808 wait complete. 2024-04-25T10:29:06.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.888+0000 7fcd3d0a5000 20 client.9789 trim_cache size 0 max 16384 2024-04-25T10:29:06.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.900+0000 7fcd3d0a5000 1 Processor -- start 2024-04-25T10:29:06.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- start start 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f25dfd90 0x55e4f25df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f25de130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f2625360 0x55e4f25de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341ff850 con 0x55e4f2624a40 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341ffe70 con 0x55e4f25dfd90 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340d0840 con 0x55e4f2625360 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f25de130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f25de130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52022/0 (socket says 172.21.15.45:52022) 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 -- 172.21.15.45:0/2760363501 learned_addr learned my addr 172.21.15.45:0/2760363501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd32a00640 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f25dfd90 0x55e4f25df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd33400640 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f2625360 0x55e4f25de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 -- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f2625360 msgr2=0x55e4f25de670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f2625360 0x55e4f25de670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 -- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f25dfd90 msgr2=0x55e4f25df4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f25dfd90 0x55e4f25df4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 -- 172.21.15.45:0/2760363501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f2482a10 con 0x55e4f2624a40 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd32a00640 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f25dfd90 0x55e4f25df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd2a000640 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f25de130 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c007e00 tx=0x7fcd1c009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef11f8cb4da6247 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd22a00640 1 -- 172.21.15.45:0/2760363501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c018040 con 0x55e4f2624a40 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd22a00640 1 -- 172.21.15.45:0/2760363501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c010dc0 con 0x55e4f2624a40 2024-04-25T10:29:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd22a00640 1 -- 172.21.15.45:0/2760363501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c004240 con 0x55e4f2624a40 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2760363501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 msgr2=0x55e4f25de130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f25de130 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c007e00 tx=0x7fcd1c009000 comp rx=0 tx=0).stop 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2760363501 shutdown_connections 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f2625360 0x55e4f25de670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f25de130 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2760363501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f25dfd90 0x55e4f25df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2760363501 >> 172.21.15.45:0/2760363501 conn(0x55e4f25dbd40 msgr2=0x55e4f26328a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2760363501 shutdown_connections 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2760363501 wait complete. 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 -- start start 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.904+0000 7fcd3d0a5000 1 Processor -- start 2024-04-25T10:29:06.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f2685090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f26855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2625360 0x55e4f26296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f23ec720 con 0x55e4f2624a40 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd341f7790 con 0x55e4f2625360 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f25c65b0 con 0x55e4f25dfd90 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 10 client.0 _ll_register_callbacks cb 0x55e4f23eba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f26855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f26855d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52030/0 (socket says 172.21.15.45:52030) 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 -- 172.21.15.45:0/2704278208 learned_addr learned my addr 172.21.15.45:0/2704278208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd32a00640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f2685090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd33400640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2625360 0x55e4f26296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 -- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 msgr2=0x55e4f2685090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f2685090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 -- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2625360 msgr2=0x55e4f26296b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2625360 0x55e4f26296b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 -- 172.21.15.45:0/2704278208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4f23bdb00 con 0x55e4f2624a40 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd2a000640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f26855d0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c029040 tx=0x7fcd1c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dedaf0647454bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 1 -- 172.21.15.45:0/2704278208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c035060 con 0x55e4f2624a40 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 1 -- 172.21.15.45:0/2704278208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcd1c003be0 con 0x55e4f2624a40 2024-04-25T10:29:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd1c010dc0 con 0x55e4f2624a40 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 1 -- 172.21.15.45:0/2704278208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd1c03b050 con 0x55e4f2624a40 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 10 client.9795 fetch_fsmap learned FSMap version 107 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55e4f23cff80 con 0x55e4f2624a40 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 1 -- 172.21.15.45:0/2704278208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c010dc0 con 0x55e4f2624a40 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 1 -- 172.21.15.45:0/2704278208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fcd1c03e020 con 0x55e4f2624a40 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 10 client.9795 fetch_fsmap finished waiting for FSMap version 107 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 10 client.9795 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 20 client.9795 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:06.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f262a010 0x55e4f262a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 4 client.9795 mds_command: new command op to 8893 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 --> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55e4f24363d0 con 0x55e4f262a010 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd32a00640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f262a010 0x55e4f262a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd32a00640 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f262a010 0x55e4f262a3f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.8893 client_cookie=56360f299563fd86 server_cookie=c4a2e6e7b3428f9b in_seq=0 out_seq=0 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 10 client.9795 ms_handle_connect on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 1 -- 172.21.15.45:0/2704278208 <== mds.8893 v2:172.21.15.117:6834/2453939747 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x55e4f24363d0 con 0x55e4f262a010 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 10 client.9795 handle_command_reply: tid=0 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 client.9795 shutdown 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 msgr2=0x55e4f26855d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f26855d0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c029040 tx=0x7fcd1c004740 comp rx=0 tx=0).stop 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 shutdown_connections 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:6834/2453939747,v1:172.21.15.117:6836/2453939747] conn(0x55e4f262a010 0x55e4f262a3f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4f2625360 0x55e4f26296b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e4f2624a40 0x55e4f26855d0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd17e00640 0 client.9795 ms_handle_reset on v2:172.21.15.117:6834/2453939747 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 --2- 172.21.15.45:0/2704278208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e4f25dfd90 0x55e4f2685090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 >> 172.21.15.45:0/2704278208 conn(0x55e4f25dbd40 msgr2=0x55e4f26230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.908+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 shutdown_connections 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.912+0000 7fcd3d0a5000 1 -- 172.21.15.45:0/2704278208 wait complete. 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:06.912+0000 7fcd3d0a5000 20 client.9795 trim_cache size 0 max 16384 2024-04-25T10:29:06.915 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: MDS is not active 2024-04-25T10:29:06.918 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:29:06.918 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:29:06.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:29:07.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:29:07.122 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:29:07.122 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:29:07.122 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:07.131 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:29:07.131 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:07.183 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:07.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.368+0000 7f3878e00640 1 Processor -- start 2024-04-25T10:29:07.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.368+0000 7f3878e00640 1 -- start start 2024-04-25T10:29:07.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3878e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 0x7f38740b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3878e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3874074690 con 0x7f38740b0840 2024-04-25T10:29:07.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3878e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38740afd60 con 0x7f387415bb20 2024-04-25T10:29:07.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3878e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740afb10 con 0x7f3874074160 2024-04-25T10:29:07.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3872800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f38740b0840 0x7f38740cbb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40714/0 (socket says 172.21.15.45:40714) 2024-04-25T10:29:07.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3872800640 1 -- 172.21.15.45:0/861372169 learned_addr learned my addr 172.21.15.45:0/861372169 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:07.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 0x7f38740b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720933033 0 0) 0x7f3874074690 con 0x7f38740b0840 2024-04-25T10:29:07.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3858003530 con 0x7f38740b0840 2024-04-25T10:29:07.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246734653 0 0) 0x7f38740afd60 con 0x7f387415bb20 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3874074690 con 0x7f387415bb20 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484140691 0 0) 0x7f3858003530 con 0x7f38740b0840 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 msgr2=0x7f38740b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 0x7f38740b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 >> v1:172.21.15.117:6789/0 conn(0x7f387415bb20 legacy=0x7f3874072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740b0040 con 0x7f38740b0840 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f386c001680 con 0x7f38740b0840 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 >> v1:172.21.15.45:6789/0 conn(0x7f38740b0840 legacy=0x7f38740cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3858003b80 0x7f3858003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3858004810 0x7f3858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 0x7f3858005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740afb10 con 0x7f3858004810 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385800bb00 con 0x7f3858003b80 2024-04-25T10:29:07.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385800bd00 con 0x7f3874074160 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3858004810 0x7f3858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3871e00640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3858003b80 0x7f3858003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3872800640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 0x7f3858005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 -- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 msgr2=0x7f3858005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3874074160 0x7f3858005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 -- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3858003b80 msgr2=0x7f3858003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3858003b80 0x7f3858003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.372+0000 7f3869400640 1 -- 172.21.15.45:0/861372169 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858003530 con 0x7f3858004810 2024-04-25T10:29:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3869400640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3858004810 0x7f3858004c90 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ae0 tx=0x7f385c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508fab554e35a073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0096b0 con 0x7f3858004810 2024-04-25T10:29:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c0a64a0 con 0x7f3858004810 2024-04-25T10:29:07.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3871400640 1 -- 172.21.15.45:0/861372169 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0a6890 con 0x7f3858004810 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3878e00640 1 -- 172.21.15.45:0/861372169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3858004810 msgr2=0x7f3858004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3878e00640 1 --2- 172.21.15.45:0/861372169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3858004810 0x7f3858004c90 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ae0 tx=0x7f385c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3872800640 1 -- 172.21.15.45:0/861372169 reap_dead start 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3878e00640 1 -- 172.21.15.45:0/861372169 shutdown_connections 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3878e00640 1 -- 172.21.15.45:0/861372169 >> 172.21.15.45:0/861372169 conn(0x7f38740b1d50 msgr2=0x7f38740b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3878e00640 1 -- 172.21.15.45:0/861372169 shutdown_connections 2024-04-25T10:29:07.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.376+0000 7f3878e00640 1 -- 172.21.15.45:0/861372169 wait complete. 2024-04-25T10:29:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 Processor -- start 2024-04-25T10:29:07.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- start start 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3858004810 0x7f3874099770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 0x7f38740aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38740c3690 0x7f38740af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740b11b0 con 0x7f38740c65e0 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740b1030 con 0x7f38740c3690 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740b1330 con 0x7f3858004810 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 0x7f38740aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 0x7f38740aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52052/0 (socket says 172.21.15.45:52052) 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 -- 172.21.15.45:0/1569867224 learned_addr learned my addr 172.21.15.45:0/1569867224 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3872800640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38740c3690 0x7f38740af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3871e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3858004810 0x7f3874099770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 -- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3858004810 msgr2=0x7f3874099770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3858004810 0x7f3874099770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 -- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38740c3690 msgr2=0x7f38740af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38740c3690 0x7f38740af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 -- 172.21.15.45:0/1569867224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740c3f40 con 0x7f38740c65e0 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3872800640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38740c3690 0x7f38740af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3869400640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 0x7f38740aeb30 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ae0 tx=0x7f3858005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00f000 con 0x7f38740c65e0 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c0a6bd0 con 0x7f38740c65e0 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c004020 con 0x7f38740c65e0 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38740c4220 con 0x7f38740c65e0 2024-04-25T10:29:07.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38740c47e0 con 0x7f38740c65e0 2024-04-25T10:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.380+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387404ef30 con 0x7f38740c65e0 2024-04-25T10:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.384+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f385c0041c0 con 0x7f38740c65e0 2024-04-25T10:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.384+0000 7f386aa00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3850066f60 0x7f3850069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.384+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f385c11efb0 con 0x7f38740c65e0 2024-04-25T10:29:07.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.384+0000 7f3871e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3850066f60 0x7f3850069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.384+0000 7f3871e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3850066f60 0x7f3850069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3864000f90 tx=0x7f3864009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.388+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f385c0ab020 con 0x7f38740c65e0 2024-04-25T10:29:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.552+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3874071970 con 0x7f38740c65e0 2024-04-25T10:29:07.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.552+0000 7f386aa00640 1 -- 172.21.15.45:0/1569867224 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f385c016780 con 0x7f38740c65e0 2024-04-25T10:29:07.554 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:07.555 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":143,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:44.069863+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":2,"to":5}]},{"pgid":"21.2f","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3850066f60 msgr2=0x7f3850069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3850066f60 0x7f3850069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3864000f90 tx=0x7f3864009040 comp rx=0 tx=0).stop 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 msgr2=0x7f38740aeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 0x7f38740aeb30 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ae0 tx=0x7f3858005840 comp rx=0 tx=0).stop 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 shutdown_connections 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3850066f60 0x7f3850069420 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38740c3690 0x7f38740af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38740c65e0 0x7f38740aeb30 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 --2- 172.21.15.45:0/1569867224 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3858004810 0x7f3874099770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 >> 172.21.15.45:0/1569867224 conn(0x7f38740b1d50 msgr2=0x7f3874160e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 shutdown_connections 2024-04-25T10:29:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.556+0000 7f3878e00640 1 -- 172.21.15.45:0/1569867224 wait complete. 2024-04-25T10:29:07.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:28:34.874797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:28:35.415884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:07.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.720+0000 7f0c56800640 1 Processor -- start 2024-04-25T10:29:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.720+0000 7f0c56800640 1 -- start start 2024-04-25T10:29:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.720+0000 7f0c56800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c500d0d00 0x7f0c500ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.720+0000 7f0c56800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c500cdab0 con 0x7f0c5006cf40 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.720+0000 7f0c56800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c500cd210 con 0x7f0c500cc200 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.720+0000 7f0c56800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c500cd4f0 con 0x7f0c500d0d00 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c500d0d00 0x7f0c500ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0c500d0d00 0x7f0c500ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52058/0 (socket says 172.21.15.45:52058) 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 -- 172.21.15.45:0/1731795898 learned_addr learned my addr 172.21.15.45:0/1731795898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0c5006cf40 0x7f0c5015d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40716/0 (socket says 172.21.15.45:40716) 2024-04-25T10:29:07.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 -- 172.21.15.45:0/1731795898 >> v1:172.21.15.117:6789/0 conn(0x7f0c500cc200 legacy=0x7f0c50162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 -- 172.21.15.45:0/1731795898 >> v1:172.21.15.45:6789/0 conn(0x7f0c5006cf40 legacy=0x7f0c5015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 -- 172.21.15.45:0/1731795898 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c500cd7d0 con 0x7f0c500d0d00 2024-04-25T10:29:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4f400640 1 --2- 172.21.15.45:0/1731795898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c500ccc30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009870 tx=0x7f0c4001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef375102f475bb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4ea00640 1 -- 172.21.15.45:0/1731795898 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c400b0070 con 0x7f0c500d0d00 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4ea00640 1 -- 172.21.15.45:0/1731795898 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c400ab600 con 0x7f0c500d0d00 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.724+0000 7f0c4ea00640 1 -- 172.21.15.45:0/1731795898 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c400ab9d0 con 0x7f0c500d0d00 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- 172.21.15.45:0/1731795898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 msgr2=0x7f0c500ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 --2- 172.21.15.45:0/1731795898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c500ccc30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009870 tx=0x7f0c4001c4d0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- 172.21.15.45:0/1731795898 shutdown_connections 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 --2- 172.21.15.45:0/1731795898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c500ccc30 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- 172.21.15.45:0/1731795898 >> 172.21.15.45:0/1731795898 conn(0x7f0c50072290 msgr2=0x7f0c500726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- 172.21.15.45:0/1731795898 shutdown_connections 2024-04-25T10:29:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- 172.21.15.45:0/1731795898 wait complete. 2024-04-25T10:29:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 Processor -- start 2024-04-25T10:29:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- start start 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5006cf40 0x7f0c50150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c500cc200 0x7f0c50155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c501563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c50162a10 con 0x7f0c500d0d00 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c50162890 con 0x7f0c500cc200 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c56800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5016c690 con 0x7f0c5006cf40 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c501563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c501563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52066/0 (socket says 172.21.15.45:52066) 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 -- 172.21.15.45:0/1028101753 learned_addr learned my addr 172.21.15.45:0/1028101753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4f400640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5006cf40 0x7f0c50150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c47e00640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c500cc200 0x7f0c50155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 -- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5006cf40 msgr2=0x7f0c50150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5006cf40 0x7f0c50150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 -- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c500cc200 msgr2=0x7f0c50155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c500cc200 0x7f0c50155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.728+0000 7f0c4fe00640 1 -- 172.21.15.45:0/1028101753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c50153d40 con 0x7f0c500d0d00 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c4fe00640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c501563f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c700 tx=0x7f0c4800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48017070 con 0x7f0c500d0d00 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4800ce10 con 0x7f0c500d0d00 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800a7e0 con 0x7f0c500d0d00 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5015b640 con 0x7f0c500d0d00 2024-04-25T10:29:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5015b920 con 0x7f0c500d0d00 2024-04-25T10:29:07.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5004ef30 con 0x7f0c500d0d00 2024-04-25T10:29:07.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c4801a070 con 0x7f0c500d0d00 2024-04-25T10:29:07.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c47400640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c2c066f60 0x7f0c2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:07.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0c48084550 con 0x7f0c500d0d00 2024-04-25T10:29:07.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.732+0000 7f0c4f400640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c2c066f60 0x7f0c2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:07.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.736+0000 7f0c4f400640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c2c066f60 0x7f0c2c069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c400ad000 tx=0x7f0c40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:07.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.736+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c48054e30 con 0x7f0c500d0d00 2024-04-25T10:29:07.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c50072ff0 con 0x7f0c500d0d00 2024-04-25T10:29:07.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c47400640 1 -- 172.21.15.45:0/1028101753 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c48058ed0 con 0x7f0c500d0d00 2024-04-25T10:29:07.909 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c2c066f60 msgr2=0x7f0c2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c2c066f60 0x7f0c2c069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c400ad000 tx=0x7f0c40008040 comp rx=0 tx=0).stop 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 msgr2=0x7f0c501563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c501563f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800c700 tx=0x7f0c4800cbc0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 shutdown_connections 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0c2c066f60 0x7f0c2c069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c500d0d00 0x7f0c501563f0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c500cc200 0x7f0c50155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 --2- 172.21.15.45:0/1028101753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c5006cf40 0x7f0c50150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 >> 172.21.15.45:0/1028101753 conn(0x7f0c50072290 msgr2=0x7f0c500b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 shutdown_connections 2024-04-25T10:29:07.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:07.908+0000 7f0c56800640 1 -- 172.21.15.45:0/1028101753 wait complete. 2024-04-25T10:29:07.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 Processor -- start 2024-04-25T10:29:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 -- start start 2024-04-25T10:29:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06b40aeda0 0x7f06b4071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b4072020 0x7f06b4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b4159920 con 0x7f06b4159540 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b415c2f0 con 0x7f06b40aeda0 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06bb000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b415c5d0 con 0x7f06b4072020 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06b40aeda0 0x7f06b4071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06b40aeda0 0x7f06b4071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52080/0 (socket says 172.21.15.45:52080) 2024-04-25T10:29:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 -- 172.21.15.45:0/1860731326 learned_addr learned my addr 172.21.15.45:0/1860731326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06b9000640 1 --2- 172.21.15.45:0/1860731326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b4072020 0x7f06b4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 -- 172.21.15.45:0/1860731326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b4072020 msgr2=0x7f06b4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 --2- 172.21.15.45:0/1860731326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b4072020 0x7f06b4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 -- 172.21.15.45:0/1860731326 >> v1:172.21.15.45:6789/0 conn(0x7f06b4159540 legacy=0x7f06b40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.076+0000 7f06ab400640 1 -- 172.21.15.45:0/1860731326 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b415c8b0 con 0x7f06b40aeda0 2024-04-25T10:29:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06ab400640 1 --2- 172.21.15.45:0/1860731326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b40aeda0 0x7f06b4071ae0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f069c009870 tx=0x7f069c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7b9886aec79d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06b3400640 1 -- 172.21.15.45:0/1860731326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0a9610 con 0x7f06b40aeda0 2024-04-25T10:29:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06b3400640 1 -- 172.21.15.45:0/1860731326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c0a97b0 con 0x7f06b40aeda0 2024-04-25T10:29:08.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06b3400640 1 -- 172.21.15.45:0/1860731326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0a9b80 con 0x7f06b40aeda0 2024-04-25T10:29:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 -- 172.21.15.45:0/1860731326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b40aeda0 msgr2=0x7f06b4071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 --2- 172.21.15.45:0/1860731326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b40aeda0 0x7f06b4071ae0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f069c009870 tx=0x7f069c01c530 comp rx=0 tx=0).stop 2024-04-25T10:29:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 -- 172.21.15.45:0/1860731326 shutdown_connections 2024-04-25T10:29:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 --2- 172.21.15.45:0/1860731326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b4072020 0x7f06b4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 --2- 172.21.15.45:0/1860731326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b40aeda0 0x7f06b4071ae0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 -- 172.21.15.45:0/1860731326 >> 172.21.15.45:0/1860731326 conn(0x7f06b40669a0 msgr2=0x7f06b4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 -- 172.21.15.45:0/1860731326 shutdown_connections 2024-04-25T10:29:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.080+0000 7f06bb000640 1 -- 172.21.15.45:0/1860731326 wait complete. 2024-04-25T10:29:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 Processor -- start 2024-04-25T10:29:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- start start 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 0x7f06b414af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b40aeda0 0x7f06b414ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4159540 0x7f06b414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40710a0 con 0x7f06b4072020 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4072430 con 0x7f06b4159540 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b4071220 con 0x7f06b40aeda0 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 0x7f06b414af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 0x7f06b414af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52086/0 (socket says 172.21.15.45:52086) 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 -- 172.21.15.45:0/3413233629 learned_addr learned my addr 172.21.15.45:0/3413233629 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06ab400640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b40aeda0 0x7f06b414ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b9000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4159540 0x7f06b414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 -- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b40aeda0 msgr2=0x7f06b414ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b40aeda0 0x7f06b414ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 -- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4159540 msgr2=0x7f06b414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4159540 0x7f06b414a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 -- 172.21.15.45:0/3413233629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b414ac20 con 0x7f06b4072020 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b3e00640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 0x7f06b414af00 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f06a400c370 tx=0x7f06a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a40052d0 con 0x7f06b4072020 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a4005470 con 0x7f06b4072020 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4007260 con 0x7f06b4072020 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b40ae340 con 0x7f06b4072020 2024-04-25T10:29:08.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b414f450 con 0x7f06b4072020 2024-04-25T10:29:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.084+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b404ef30 con 0x7f06b4072020 2024-04-25T10:29:08.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.088+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06a4007400 con 0x7f06b4072020 2024-04-25T10:29:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.088+0000 7f06b0c00640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0690066f60 0x7f0690069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.088+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f06a40844e0 con 0x7f06b4072020 2024-04-25T10:29:08.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.088+0000 7f06ab400640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0690066f60 0x7f0690069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.088+0000 7f06ab400640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0690066f60 0x7f0690069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ae000 tx=0x7f069c0b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.092+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06a40554a0 con 0x7f06b4072020 2024-04-25T10:29:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06b40cf470 con 0x7f06b4072020 2024-04-25T10:29:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06b0c00640 1 -- 172.21.15.45:0/3413233629 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06a4054da0 con 0x7f06b4072020 2024-04-25T10:29:08.256 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:29:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0690066f60 msgr2=0x7f0690069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0690066f60 0x7f0690069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f069c0ae000 tx=0x7f069c0b3040 comp rx=0 tx=0).stop 2024-04-25T10:29:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 msgr2=0x7f06b414af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 0x7f06b414af00 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f06a400c370 tx=0x7f06a400c830 comp rx=0 tx=0).stop 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 shutdown_connections 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0690066f60 0x7f0690069420 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4159540 0x7f06b414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06b40aeda0 0x7f06b414ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 --2- 172.21.15.45:0/3413233629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06b4072020 0x7f06b414af00 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 >> 172.21.15.45:0/3413233629 conn(0x7f06b40669a0 msgr2=0x7f06b40cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 shutdown_connections 2024-04-25T10:29:08.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.256+0000 7f06bb000640 1 -- 172.21.15.45:0/3413233629 wait complete. 2024-04-25T10:29:08.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 Processor -- start 2024-04-25T10:29:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 -- start start 2024-04-25T10:29:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 0x7f36900ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f36900aebd0 0x7f3690071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3690071360 con 0x7f3690072020 2024-04-25T10:29:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36900af030 con 0x7f36900aebd0 2024-04-25T10:29:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3697e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3690159920 con 0x7f3690159540 2024-04-25T10:29:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3695e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3690072020 0x7f369015f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40726/0 (socket says 172.21.15.45:40726) 2024-04-25T10:29:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3695e00640 1 -- 172.21.15.45:0/1512757204 learned_addr learned my addr 172.21.15.45:0/1512757204 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3695400640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 0x7f36900ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f3694a00640 1 --2- 172.21.15.45:0/1512757204 >> v2:172.21.15.117:3300/0 conn(0x7f36900aebd0 0x7f3690071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316364043 0 0) 0x7f3690071360 con 0x7f3690072020 2024-04-25T10:29:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.420+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3678003530 con 0x7f3690072020 2024-04-25T10:29:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900855927 0 0) 0x7f3678003530 con 0x7f3690072020 2024-04-25T10:29:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 msgr2=0x7f36900ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 0x7f36900ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 >> v2:172.21.15.117:3300/0 conn(0x7f36900aebd0 msgr2=0x7f3690071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 --2- 172.21.15.45:0/1512757204 >> v2:172.21.15.117:3300/0 conn(0x7f36900aebd0 0x7f3690071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369015c2f0 con 0x7f3690072020 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f3688001a00 con 0x7f3690072020 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 >> v1:172.21.15.45:6789/0 conn(0x7f3690072020 legacy=0x7f369015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f3678003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 0x7f3678004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3678004d50 0x7f36780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3690159920 con 0x7f36780039c0 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367800b9d0 con 0x7f3678004d50 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367800bb50 con 0x7f3690159540 2024-04-25T10:29:08.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f3678003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3694a00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 0x7f3678004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695e00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3678004d50 0x7f36780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 -- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 msgr2=0x7f3678004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690159540 0x7f3678004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 -- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3678004d50 msgr2=0x7f36780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3678004d50 0x7f36780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 -- 172.21.15.45:0/1512757204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678003530 con 0x7f36780039c0 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.424+0000 7f3695400640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f3678003de0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3684009ae0 tx=0x7f368401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120d04fd61517df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840b2070 con 0x7f36780039c0 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36840a5520 con 0x7f36780039c0 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f368fe00640 1 -- 172.21.15.45:0/1512757204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840096b0 con 0x7f36780039c0 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- 172.21.15.45:0/1512757204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 msgr2=0x7f3678003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 --2- 172.21.15.45:0/1512757204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f3678003de0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3684009ae0 tx=0x7f368401ca40 comp rx=0 tx=0).stop 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3695e00640 1 -- 172.21.15.45:0/1512757204 reap_dead start 2024-04-25T10:29:08.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- 172.21.15.45:0/1512757204 shutdown_connections 2024-04-25T10:29:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- 172.21.15.45:0/1512757204 >> 172.21.15.45:0/1512757204 conn(0x7f36900669a0 msgr2=0x7f3690066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- 172.21.15.45:0/1512757204 shutdown_connections 2024-04-25T10:29:08.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- 172.21.15.45:0/1512757204 wait complete. 2024-04-25T10:29:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 Processor -- start 2024-04-25T10:29:08.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- start start 2024-04-25T10:29:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f36900b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690155380 0x7f369014a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3690157d20 0x7f36900ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369006c0d0 con 0x7f36780039c0 2024-04-25T10:29:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369015bfa0 con 0x7f3690157d20 2024-04-25T10:29:08.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3697e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3690072430 con 0x7f3690155380 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3695400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f36900b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3695400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f36900b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52108/0 (socket says 172.21.15.45:52108) 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3695400640 1 -- 172.21.15.45:0/1521295497 learned_addr learned my addr 172.21.15.45:0/1521295497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3694a00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690155380 0x7f369014a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.428+0000 7f3695e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3690157d20 0x7f36900ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3695400640 1 -- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690155380 msgr2=0x7f369014a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3695400640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690155380 0x7f369014a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3695400640 1 -- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3690157d20 msgr2=0x7f36900ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3695400640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3690157d20 0x7f36900ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3695400640 1 -- 172.21.15.45:0/1521295497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3690157560 con 0x7f36780039c0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3695400640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f36900b1cd0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f36840b0040 tx=0x7f3678005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840bc030 con 0x7f36780039c0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368400f000 con 0x7f36780039c0 2024-04-25T10:29:08.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3690155860 con 0x7f36780039c0 2024-04-25T10:29:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840b2040 con 0x7f36780039c0 2024-04-25T10:29:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3690157800 con 0x7f36780039c0 2024-04-25T10:29:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369004ef30 con 0x7f36780039c0 2024-04-25T10:29:08.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36840bf070 con 0x7f36780039c0 2024-04-25T10:29:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f368d600640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3670066f60 0x7f3670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.432+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f368411ec10 con 0x7f36780039c0 2024-04-25T10:29:08.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.436+0000 7f3694a00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3670066f60 0x7f3670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.436+0000 7f3694a00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3670066f60 0x7f3670069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f36900cc570 tx=0x7f367c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.436+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36840c7020 con 0x7f36780039c0 2024-04-25T10:29:08.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.596+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f369006d7f0 con 0x7f36780039c0 2024-04-25T10:29:08.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.596+0000 7f368d600640 1 -- 172.21.15.45:0/1521295497 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3684016780 con 0x7f36780039c0 2024-04-25T10:29:08.599 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:29:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3670066f60 msgr2=0x7f3670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3670066f60 0x7f3670069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f36900cc570 tx=0x7f367c009190 comp rx=0 tx=0).stop 2024-04-25T10:29:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 msgr2=0x7f36900b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f36900b1cd0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f36840b0040 tx=0x7f3678005840 comp rx=0 tx=0).stop 2024-04-25T10:29:08.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 shutdown_connections 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3670066f60 0x7f3670069420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3690157d20 0x7f36900ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3690155380 0x7f369014a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 --2- 172.21.15.45:0/1521295497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36780039c0 0x7f36900b1cd0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 >> 172.21.15.45:0/1521295497 conn(0x7f36900669a0 msgr2=0x7f36900cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 shutdown_connections 2024-04-25T10:29:08.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.600+0000 7f3697e00640 1 -- 172.21.15.45:0/1521295497 wait complete. 2024-04-25T10:29:08.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:29:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 Processor -- start 2024-04-25T10:29:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 -- start start 2024-04-25T10:29:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb400cf4e0 0x7feb400cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb4006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb400cc2c0 con 0x7feb4015a9f0 2024-04-25T10:29:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb400cc5a0 con 0x7feb400cf4e0 2024-04-25T10:29:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb45600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400cc880 con 0x7feb400cbab0 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb400cf4e0 0x7feb400cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb400cf4e0 0x7feb400cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52120/0 (socket says 172.21.15.45:52120) 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 -- 172.21.15.45:0/4208339441 learned_addr learned my addr 172.21.15.45:0/4208339441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb3f400640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb4006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 -- 172.21.15.45:0/4208339441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 msgr2=0x7feb4006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb4006d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 -- 172.21.15.45:0/4208339441 >> v1:172.21.15.117:6789/0 conn(0x7feb4015a9f0 legacy=0x7feb400cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb37e00640 1 -- 172.21.15.45:0/4208339441 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400ccb60 con 0x7feb400cf4e0 2024-04-25T10:29:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.760+0000 7feb3f400640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb4006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb37e00640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb400cf4e0 0x7feb400cb570 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7feb28009870 tx=0x7feb2801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1da81861978120 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb3e000640 1 -- 172.21.15.45:0/4208339441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280ab640 con 0x7feb400cf4e0 2024-04-25T10:29:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb3e000640 1 -- 172.21.15.45:0/4208339441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb280b0070 con 0x7feb400cf4e0 2024-04-25T10:29:08.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb3e000640 1 -- 172.21.15.45:0/4208339441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280aba00 con 0x7feb400cf4e0 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 -- 172.21.15.45:0/4208339441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb400cf4e0 msgr2=0x7feb400cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb400cf4e0 0x7feb400cb570 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7feb28009870 tx=0x7feb2801c530 comp rx=0 tx=0).stop 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 -- 172.21.15.45:0/4208339441 shutdown_connections 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb4006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 --2- 172.21.15.45:0/4208339441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb400cf4e0 0x7feb400cb570 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 -- 172.21.15.45:0/4208339441 >> 172.21.15.45:0/4208339441 conn(0x7feb400b2410 msgr2=0x7feb400b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 -- 172.21.15.45:0/4208339441 shutdown_connections 2024-04-25T10:29:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.764+0000 7feb45600640 1 -- 172.21.15.45:0/4208339441 wait complete. 2024-04-25T10:29:08.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 Processor -- start 2024-04-25T10:29:08.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 -- start start 2024-04-25T10:29:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb400c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb400cf4e0 0x7feb400c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 0x7feb400c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4006e330 con 0x7feb4015a9f0 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4006e1b0 con 0x7feb400cf4e0 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4006e4b0 con 0x7feb400cbab0 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 0x7feb400c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 0x7feb400c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52124/0 (socket says 172.21.15.45:52124) 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 -- 172.21.15.45:0/1022075077 learned_addr learned my addr 172.21.15.45:0/1022075077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3ea00640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb400c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb37e00640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb400cf4e0 0x7feb400c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 -- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 msgr2=0x7feb400c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb400c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 -- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb400cf4e0 msgr2=0x7feb400c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb400cf4e0 0x7feb400c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 -- 172.21.15.45:0/1022075077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400b63d0 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb37e00640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb400cf4e0 0x7feb400c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb3f400640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 0x7feb400c8ad0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7feb38009fd0 tx=0x7feb3800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3801a070 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38003bc0 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38004030 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400b3710 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.768+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb400c7370 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.772+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4004ef30 con 0x7feb4015a9f0 2024-04-25T10:29:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.772+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb3801d070 con 0x7feb4015a9f0 2024-04-25T10:29:08.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.772+0000 7feb36a00640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb1c066f60 0x7feb1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.772+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb380852e0 con 0x7feb4015a9f0 2024-04-25T10:29:08.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.772+0000 7feb3ea00640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb1c066f60 0x7feb1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.772+0000 7feb3ea00640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb1c066f60 0x7feb1c069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7feb30007fd0 tx=0x7feb30009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.776+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb38010070 con 0x7feb4015a9f0 2024-04-25T10:29:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.948+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb400c5000 con 0x7feb4015a9f0 2024-04-25T10:29:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.948+0000 7feb36a00640 1 -- 172.21.15.45:0/1022075077 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v143) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb38055c50 con 0x7feb4015a9f0 2024-04-25T10:29:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb1c066f60 msgr2=0x7feb1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb1c066f60 0x7feb1c069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7feb30007fd0 tx=0x7feb30009040 comp rx=0 tx=0).stop 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 msgr2=0x7feb400c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 0x7feb400c8ad0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7feb38009fd0 tx=0x7feb3800d560 comp rx=0 tx=0).stop 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 shutdown_connections 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb1c066f60 0x7feb1c069420 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb4015a9f0 0x7feb400c8ad0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb400cf4e0 0x7feb400c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 --2- 172.21.15.45:0/1022075077 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb400cbab0 0x7feb400c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 >> 172.21.15.45:0/1022075077 conn(0x7feb400b2410 msgr2=0x7feb400b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:08.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 shutdown_connections 2024-04-25T10:29:08.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:08.952+0000 7feb45600640 1 -- 172.21.15.45:0/1022075077 wait complete. 2024-04-25T10:29:08.973 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:29:08.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:08.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:09.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:29:09.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:09.725 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:29:09.734 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:29:09.734 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:09.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:29:09.743 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:29:09.743 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:09.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:29:09.787 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:29:09.788 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:09.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:29:09.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:09.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.972+0000 7f9f9f400640 1 Processor -- start 2024-04-25T10:29:09.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.972+0000 7f9f9f400640 1 -- start start 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f98066c00 0x7f9f9815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980b5560 0x7f9f98157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9f400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f980b59b0 con 0x7f9f98066fe0 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9f400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980b5c20 con 0x7f9f98066c00 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980af6e0 con 0x7f9f980b5560 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f98066c00 0x7f9f9815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f98066c00 0x7f9f9815e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52140/0 (socket says 172.21.15.45:52140) 2024-04-25T10:29:09.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2769876062 learned_addr learned my addr 172.21.15.45:0/2769876062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2769876062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980b5560 msgr2=0x7f9f98157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 --2- 172.21.15.45:0/2769876062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980b5560 0x7f9f98157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2769876062 >> v1:172.21.15.117:6789/0 conn(0x7f9f98066fe0 legacy=0x7f9f9806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:09.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.976+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2769876062 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980af9c0 con 0x7f9f98066c00 2024-04-25T10:29:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9ca00640 1 --2- 172.21.15.45:0/2769876062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f9815e110 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009870 tx=0x7f9f8801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54547cd71c4f783f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f97e00640 1 -- 172.21.15.45:0/2769876062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880aa610 con 0x7f9f98066c00 2024-04-25T10:29:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f97e00640 1 -- 172.21.15.45:0/2769876062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f880af070 con 0x7f9f98066c00 2024-04-25T10:29:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f97e00640 1 -- 172.21.15.45:0/2769876062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880aa9d0 con 0x7f9f98066c00 2024-04-25T10:29:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- 172.21.15.45:0/2769876062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 msgr2=0x7f9f9815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2769876062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f9815e110 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009870 tx=0x7f9f8801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- 172.21.15.45:0/2769876062 shutdown_connections 2024-04-25T10:29:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2769876062 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f980b5560 0x7f9f98157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2769876062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f9815e110 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- 172.21.15.45:0/2769876062 >> 172.21.15.45:0/2769876062 conn(0x7f9f980b24f0 msgr2=0x7f9f980b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- 172.21.15.45:0/2769876062 shutdown_connections 2024-04-25T10:29:09.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- 172.21.15.45:0/2769876062 wait complete. 2024-04-25T10:29:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 Processor -- start 2024-04-25T10:29:09.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- start start 2024-04-25T10:29:09.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f98147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f98066fe0 0x7f9f98147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f980b5560 0x7f9f9814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980af1e0 con 0x7f9f98066c00 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980af060 con 0x7f9f980b5560 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9806ac90 con 0x7f9f98066fe0 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f98147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f98147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52152/0 (socket says 172.21.15.45:52152) 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.980+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2053340743 learned_addr learned my addr 172.21.15.45:0/2053340743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f8fe00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f98066fe0 0x7f9f98147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9d400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f980b5560 0x7f9f9814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f98066fe0 msgr2=0x7f9f98147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9ca00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f98066fe0 0x7f9f98147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f980b5560 msgr2=0x7f9f9814cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9ca00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f980b5560 0x7f9f9814cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:09.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9ca00640 1 -- 172.21.15.45:0/2053340743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9814b370 con 0x7f9f98066c00 2024-04-25T10:29:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9d400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f980b5560 0x7f9f9814cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f8fe00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f98066fe0 0x7f9f98147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9ca00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f98147b50 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f9f8801aba0 tx=0x7f9f8801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880b3800 con 0x7f9f98066c00 2024-04-25T10:29:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f880b39c0 con 0x7f9f98066c00 2024-04-25T10:29:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9814b650 con 0x7f9f98066c00 2024-04-25T10:29:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880b3db0 con 0x7f9f98066c00 2024-04-25T10:29:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f98160e80 con 0x7f9f98066c00 2024-04-25T10:29:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9804ef30 con 0x7f9f98066c00 2024-04-25T10:29:09.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f880c6390 con 0x7f9f98066c00 2024-04-25T10:29:09.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f95600640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f74066f60 0x7f9f74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:09.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.984+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9f8811f9e0 con 0x7f9f98066c00 2024-04-25T10:29:09.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.988+0000 7f9f8fe00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f74066f60 0x7f9f74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:09.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.988+0000 7f9f8fe00640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f74066f60 0x7f9f74069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9f98068040 tx=0x7f9f80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:09.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:09.988+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f880c3060 con 0x7f9f98066c00 2024-04-25T10:29:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f9806bd50 con 0x7f9f98066c00 2024-04-25T10:29:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f95600640 1 -- 172.21.15.45:0/2053340743 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4952 (secure 0 0 0) 0x7f9f880a7070 con 0x7f9f98066c00 2024-04-25T10:29:10.197 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:10.197 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":107,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:29:05.122046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8893":{"gid":8893,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:29:10.197 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 107 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f74066f60 msgr2=0x7f9f74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f74066f60 0x7f9f74069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9f98068040 tx=0x7f9f80009040 comp rx=0 tx=0).stop 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 msgr2=0x7f9f98147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f98147b50 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f9f8801aba0 tx=0x7f9f8801ad30 comp rx=0 tx=0).stop 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 shutdown_connections 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f74066f60 0x7f9f74069420 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f980b5560 0x7f9f9814cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f98066fe0 0x7f9f98147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 --2- 172.21.15.45:0/2053340743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f98066c00 0x7f9f98147b50 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 >> 172.21.15.45:0/2053340743 conn(0x7f9f980b24f0 msgr2=0x7f9f9815e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 shutdown_connections 2024-04-25T10:29:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.196+0000 7f9f9f400640 1 -- 172.21.15.45:0/2053340743 wait complete. 2024-04-25T10:29:10.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:10.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 Processor -- start 2024-04-25T10:29:10.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 -- start start 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd71c15b9e0 0x7fd71c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd71c0d1520 0x7fd71c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0ccfb0 con 0x7fd71c0d1520 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0cd290 con 0x7fd71c15b9e0 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd721000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0cd570 con 0x7fd71c15bdc0 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd71c0d1520 0x7fd71c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd71c0d1520 0x7fd71c0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52162/0 (socket says 172.21.15.45:52162) 2024-04-25T10:29:10.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 -- 172.21.15.45:0/442917539 learned_addr learned my addr 172.21.15.45:0/442917539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd713e00640 1 --2- 172.21.15.45:0/442917539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd719e00640 1 --2- 172.21.15.45:0/442917539 >> v2:172.21.15.117:3300/0 conn(0x7fd71c15b9e0 0x7fd71c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 -- 172.21.15.45:0/442917539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 msgr2=0x7fd71c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 --2- 172.21.15.45:0/442917539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 -- 172.21.15.45:0/442917539 >> v2:172.21.15.117:3300/0 conn(0x7fd71c15b9e0 msgr2=0x7fd71c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 --2- 172.21.15.45:0/442917539 >> v2:172.21.15.117:3300/0 conn(0x7fd71c15b9e0 0x7fd71c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.360+0000 7fd71a800640 1 -- 172.21.15.45:0/442917539 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0cd850 con 0x7fd71c0d1520 2024-04-25T10:29:10.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd71a800640 1 --2- 172.21.15.45:0/442917539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c0d1520 0x7fd71c0d1970 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd71c070da0 tx=0x7fd71401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513c4f1157e6297c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:10.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd719400640 1 -- 172.21.15.45:0/442917539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140a5670 con 0x7fd71c0d1520 2024-04-25T10:29:10.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd719400640 1 -- 172.21.15.45:0/442917539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7140a5810 con 0x7fd71c0d1520 2024-04-25T10:29:10.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd719400640 1 -- 172.21.15.45:0/442917539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140a5be0 con 0x7fd71c0d1520 2024-04-25T10:29:10.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 -- 172.21.15.45:0/442917539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c0d1520 msgr2=0x7fd71c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 --2- 172.21.15.45:0/442917539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c0d1520 0x7fd71c0d1970 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd71c070da0 tx=0x7fd71401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 -- 172.21.15.45:0/442917539 shutdown_connections 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 --2- 172.21.15.45:0/442917539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c0d1520 0x7fd71c0d1970 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 --2- 172.21.15.45:0/442917539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 --2- 172.21.15.45:0/442917539 >> v2:172.21.15.117:3300/0 conn(0x7fd71c15b9e0 0x7fd71c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 -- 172.21.15.45:0/442917539 >> 172.21.15.45:0/442917539 conn(0x7fd71c09a9d0 msgr2=0x7fd71c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 -- 172.21.15.45:0/442917539 shutdown_connections 2024-04-25T10:29:10.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.364+0000 7fd721000640 1 -- 172.21.15.45:0/442917539 wait complete. 2024-04-25T10:29:10.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 Processor -- start 2024-04-25T10:29:10.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 -- start start 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71c0d1520 0x7fd71c155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 0x7fd71c0c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0cebd0 con 0x7fd71c15b9e0 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0cea50 con 0x7fd71c0d1520 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0ced50 con 0x7fd71c15bdc0 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 0x7fd71c0c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 0x7fd71c0c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52166/0 (socket says 172.21.15.45:52166) 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 -- 172.21.15.45:0/2408745399 learned_addr learned my addr 172.21.15.45:0/2408745399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd719e00640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71c0d1520 0x7fd71c155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd71a800640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 -- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 msgr2=0x7fd71c0c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 -- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71c0d1520 msgr2=0x7fd71c155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71c0d1520 0x7fd71c155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 -- 172.21.15.45:0/2408745399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c06be10 con 0x7fd71c15b9e0 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd71a800640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:10.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd713e00640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 0x7fd71c0c5910 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd70400d5f0 tx=0x7fd70400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040125f0 con 0x7fd71c15b9e0 2024-04-25T10:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd704019070 con 0x7fd71c15b9e0 2024-04-25T10:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040129b0 con 0x7fd71c15b9e0 2024-04-25T10:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c156710 con 0x7fd71c15b9e0 2024-04-25T10:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.368+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c156c90 con 0x7fd71c15b9e0 2024-04-25T10:29:10.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.372+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c04ef30 con 0x7fd71c15b9e0 2024-04-25T10:29:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.372+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd704012b50 con 0x7fd71c15b9e0 2024-04-25T10:29:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.372+0000 7fd712000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6f8066f60 0x7fd6f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.372+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7040853f0 con 0x7fd71c15b9e0 2024-04-25T10:29:10.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.372+0000 7fd719e00640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6f8066f60 0x7fd6f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.372+0000 7fd719e00640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6f8066f60 0x7fd6f8069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00b9d0 tx=0x7fd70c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:10.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.376+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7040563b0 con 0x7fd71c15b9e0 2024-04-25T10:29:10.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.536+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd71c156350 con 0x7fd71c15b9e0 2024-04-25T10:29:10.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd712000640 1 -- 172.21.15.45:0/2408745399 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16705 (secure 0 0 0) 0x7fd704055cb0 con 0x7fd71c15b9e0 2024-04-25T10:29:10.540 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:10.540 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":143,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:44.069863+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":2,"to":5}]},{"pgid":"21.2f","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6f8066f60 msgr2=0x7fd6f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6f8066f60 0x7fd6f8069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00b9d0 tx=0x7fd70c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 msgr2=0x7fd71c0c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 0x7fd71c0c5910 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd70400d5f0 tx=0x7fd70400dab0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 shutdown_connections 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd6f8066f60 0x7fd6f8069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd71c15bdc0 0x7fd71c0c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd71c15b9e0 0x7fd71c0c5910 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 --2- 172.21.15.45:0/2408745399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd71c0d1520 0x7fd71c155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 >> 172.21.15.45:0/2408745399 conn(0x7fd71c09a9d0 msgr2=0x7fd71c0d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:10.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 shutdown_connections 2024-04-25T10:29:10.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.540+0000 7fd721000640 1 -- 172.21.15.45:0/2408745399 wait complete. 2024-04-25T10:29:10.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:28:34.874797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:28:35.415884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:10.562 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.704+0000 7fa776e00640 1 Processor -- start 2024-04-25T10:29:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.704+0000 7fa776e00640 1 -- start start 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa776e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700cfb30 0x7fa7700d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa776e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77006b470 con 0x7fa7700ce220 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa776e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77006b750 con 0x7fa77015dd30 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa776e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77015e110 con 0x7fa7700cfb30 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa774e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa7700ce220 0x7fa7700b1930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40734/0 (socket says 172.21.15.45:40734) 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa774e00640 1 -- 172.21.15.45:0/3491946673 learned_addr learned my addr 172.21.15.45:0/3491946673 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa767e00640 1 --2- 172.21.15.45:0/3491946673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700cfb30 0x7fa7700d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805039931 0 0) 0x7fa77006b470 con 0x7fa7700ce220 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa754003530 con 0x7fa7700ce220 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062357354 0 0) 0x7fa77006b750 con 0x7fa77015dd30 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa77006b470 con 0x7fa77015dd30 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa767e00640 1 -- 172.21.15.45:0/3491946673 >> v1:172.21.15.117:6789/0 conn(0x7fa77015dd30 legacy=0x7fa77006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa767e00640 1 -- 172.21.15.45:0/3491946673 >> v1:172.21.15.45:6789/0 conn(0x7fa7700ce220 legacy=0x7fa7700b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.708+0000 7fa767e00640 1 -- 172.21.15.45:0/3491946673 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77006ba30 con 0x7fa7700cfb30 2024-04-25T10:29:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa767e00640 1 --2- 172.21.15.45:0/3491946673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700cfb30 0x7fa7700d18a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097f0 tx=0x7fa75801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b76289806ef3a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580b0070 con 0x7fa7700cfb30 2024-04-25T10:29:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7580ab600 con 0x7fa7700cfb30 2024-04-25T10:29:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa76f400640 1 -- 172.21.15.45:0/3491946673 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580ab9d0 con 0x7fa7700cfb30 2024-04-25T10:29:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- 172.21.15.45:0/3491946673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700cfb30 msgr2=0x7fa7700d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 --2- 172.21.15.45:0/3491946673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700cfb30 0x7fa7700d18a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097f0 tx=0x7fa75801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- 172.21.15.45:0/3491946673 shutdown_connections 2024-04-25T10:29:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 --2- 172.21.15.45:0/3491946673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700cfb30 0x7fa7700d18a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- 172.21.15.45:0/3491946673 >> 172.21.15.45:0/3491946673 conn(0x7fa77009a9d0 msgr2=0x7fa77009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- 172.21.15.45:0/3491946673 shutdown_connections 2024-04-25T10:29:10.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- 172.21.15.45:0/3491946673 wait complete. 2024-04-25T10:29:10.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 Processor -- start 2024-04-25T10:29:10.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- start start 2024-04-25T10:29:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700ce220 0x7fa7700cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700cfb30 0x7fa7700c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 0x7fa7700c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77016bf00 con 0x7fa77015dd30 2024-04-25T10:29:10.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77016bd80 con 0x7fa7700cfb30 2024-04-25T10:29:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa776e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77016c080 con 0x7fa7700ce220 2024-04-25T10:29:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.712+0000 7fa774e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 0x7fa7700c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 0x7fa7700c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52168/0 (socket says 172.21.15.45:52168) 2024-04-25T10:29:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 -- 172.21.15.45:0/4085990389 learned_addr learned my addr 172.21.15.45:0/4085990389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa76fe00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700ce220 0x7fa7700cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa767e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700cfb30 0x7fa7700c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 -- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700ce220 msgr2=0x7fa7700cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700ce220 0x7fa7700cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 -- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700cfb30 msgr2=0x7fa7700c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700cfb30 0x7fa7700c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 -- 172.21.15.45:0/4085990389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700b5500 con 0x7fa77015dd30 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa774e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 0x7fa7700c92c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa76800cb70 tx=0x7fa76800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768017070 con 0x7fa77015dd30 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800ace0 con 0x7fa77015dd30 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768012620 con 0x7fa77015dd30 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700cb730 con 0x7fa77015dd30 2024-04-25T10:29:10.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700b57a0 con 0x7fa77015dd30 2024-04-25T10:29:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77004ef30 con 0x7fa77015dd30 2024-04-25T10:29:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.716+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa76801a070 con 0x7fa77015dd30 2024-04-25T10:29:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.720+0000 7fa76cc00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa74c066f60 0x7fa74c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.720+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa768084d00 con 0x7fa77015dd30 2024-04-25T10:29:10.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.720+0000 7fa76fe00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa74c066f60 0x7fa74c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:10.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.720+0000 7fa76fe00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa74c066f60 0x7fa74c069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa770169b30 tx=0x7fa760009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:10.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.724+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa768055710 con 0x7fa77015dd30 2024-04-25T10:29:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa770068190 con 0x7fa77015dd30 2024-04-25T10:29:10.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa76cc00640 1 -- 172.21.15.45:0/4085990389 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7680597b0 con 0x7fa77015dd30 2024-04-25T10:29:10.888 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:29:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa74c066f60 msgr2=0x7fa74c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa74c066f60 0x7fa74c069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa770169b30 tx=0x7fa760009040 comp rx=0 tx=0).stop 2024-04-25T10:29:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 msgr2=0x7fa7700c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 0x7fa7700c92c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa76800cb70 tx=0x7fa76800a430 comp rx=0 tx=0).stop 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 shutdown_connections 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa74c066f60 0x7fa74c069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77015dd30 0x7fa7700c92c0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7700cfb30 0x7fa7700c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 --2- 172.21.15.45:0/4085990389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7700ce220 0x7fa7700cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 >> 172.21.15.45:0/4085990389 conn(0x7fa77009a9d0 msgr2=0x7fa7700b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 shutdown_connections 2024-04-25T10:29:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:10.888+0000 7fa776e00640 1 -- 172.21.15.45:0/4085990389 wait complete. 2024-04-25T10:29:10.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:29:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.052+0000 7f6f95000640 1 Processor -- start 2024-04-25T10:29:11.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.052+0000 7f6f95000640 1 -- start start 2024-04-25T10:29:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f95000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f900aef80 0x7f6f9015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f95000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f90070350 0x7f6f9015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f95000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f9015bac0 0x7f6f900b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f95000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9015b080 con 0x7f6f9015bac0 2024-04-25T10:29:11.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f95000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9015adb0 con 0x7f6f90070350 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f95000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9015eb60 con 0x7f6f900aef80 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f9015bac0 0x7f6f900b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6f9015bac0 0x7f6f900b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52184/0 (socket says 172.21.15.45:52184) 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 -- 172.21.15.45:0/2242397805 learned_addr learned my addr 172.21.15.45:0/2242397805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8de00640 1 --2- 172.21.15.45:0/2242397805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f900aef80 0x7f6f9015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f87e00640 1 --2- 172.21.15.45:0/2242397805 >> v2:172.21.15.117:3300/0 conn(0x7f6f90070350 0x7f6f9015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 -- 172.21.15.45:0/2242397805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f900aef80 msgr2=0x7f6f9015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 --2- 172.21.15.45:0/2242397805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f900aef80 0x7f6f9015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 -- 172.21.15.45:0/2242397805 >> v2:172.21.15.117:3300/0 conn(0x7f6f90070350 msgr2=0x7f6f9015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 --2- 172.21.15.45:0/2242397805 >> v2:172.21.15.117:3300/0 conn(0x7f6f90070350 0x7f6f9015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 -- 172.21.15.45:0/2242397805 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9015ee40 con 0x7f6f9015bac0 2024-04-25T10:29:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.056+0000 7f6f8e800640 1 --2- 172.21.15.45:0/2242397805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f9015bac0 0x7f6f900b31c0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f6f9006e2a0 tx=0x7f6f8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc74e63e4b9f5811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f8d400640 1 -- 172.21.15.45:0/2242397805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880a5780 con 0x7f6f9015bac0 2024-04-25T10:29:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f8d400640 1 -- 172.21.15.45:0/2242397805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f880a5920 con 0x7f6f9015bac0 2024-04-25T10:29:11.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f8d400640 1 -- 172.21.15.45:0/2242397805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880a5cf0 con 0x7f6f9015bac0 2024-04-25T10:29:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- 172.21.15.45:0/2242397805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f9015bac0 msgr2=0x7f6f900b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- 172.21.15.45:0/2242397805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f9015bac0 0x7f6f900b31c0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f6f9006e2a0 tx=0x7f6f8801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- 172.21.15.45:0/2242397805 shutdown_connections 2024-04-25T10:29:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- 172.21.15.45:0/2242397805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f9015bac0 0x7f6f900b31c0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- 172.21.15.45:0/2242397805 >> v2:172.21.15.117:3300/0 conn(0x7f6f90070350 0x7f6f9015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- 172.21.15.45:0/2242397805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f900aef80 0x7f6f9015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- 172.21.15.45:0/2242397805 >> 172.21.15.45:0/2242397805 conn(0x7f6f900669a0 msgr2=0x7f6f90066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- 172.21.15.45:0/2242397805 shutdown_connections 2024-04-25T10:29:11.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- 172.21.15.45:0/2242397805 wait complete. 2024-04-25T10:29:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 Processor -- start 2024-04-25T10:29:11.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- start start 2024-04-25T10:29:11.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f90070350 0x7f6f90149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 0x7f6f90158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f9015bac0 0x7f6f900ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9006d5b0 con 0x7f6f900aef80 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9006d430 con 0x7f6f90070350 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.060+0000 7f6f95000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9006d730 con 0x7f6f9015bac0 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 0x7f6f90158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 0x7f6f90158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52200/0 (socket says 172.21.15.45:52200) 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 -- 172.21.15.45:0/3333664965 learned_addr learned my addr 172.21.15.45:0/3333664965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f8de00640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f90070350 0x7f6f90149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f8e800640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f9015bac0 0x7f6f900ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 -- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f9015bac0 msgr2=0x7f6f900ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f9015bac0 0x7f6f900ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 -- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f90070350 msgr2=0x7f6f90149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f90070350 0x7f6f90149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 -- 172.21.15.45:0/3333664965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9014fc70 con 0x7f6f900aef80 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f87e00640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 0x7f6f90158fc0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800d800 tx=0x7f6f7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78018020 con 0x7f6f900aef80 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f780096a0 con 0x7f6f900aef80 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9014b210 con 0x7f6f900aef80 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78009a50 con 0x7f6f900aef80 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9014ff10 con 0x7f6f900aef80 2024-04-25T10:29:11.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9004ef30 con 0x7f6f900aef80 2024-04-25T10:29:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f7801b020 con 0x7f6f900aef80 2024-04-25T10:29:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.064+0000 7f6f86000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6f6c066f60 0x7f6f6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.068+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f78085530 con 0x7f6f900aef80 2024-04-25T10:29:11.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.068+0000 7f6f8de00640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6f6c066f60 0x7f6f6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.068+0000 7f6f8de00640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6f6c066f60 0x7f6f6c069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6f80007a60 tx=0x7f6f80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.072+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f78022020 con 0x7f6f900aef80 2024-04-25T10:29:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.232+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f90072330 con 0x7f6f900aef80 2024-04-25T10:29:11.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.232+0000 7f6f86000640 1 -- 172.21.15.45:0/3333664965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f78055e10 con 0x7f6f900aef80 2024-04-25T10:29:11.234 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:29:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6f6c066f60 msgr2=0x7f6f6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6f6c066f60 0x7f6f6c069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6f80007a60 tx=0x7f6f80008040 comp rx=0 tx=0).stop 2024-04-25T10:29:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 msgr2=0x7f6f90158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 0x7f6f90158fc0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800d800 tx=0x7f6f7800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 shutdown_connections 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6f6c066f60 0x7f6f6c069420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f9015bac0 0x7f6f900ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f900aef80 0x7f6f90158fc0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 --2- 172.21.15.45:0/3333664965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f90070350 0x7f6f90149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 >> 172.21.15.45:0/3333664965 conn(0x7f6f900669a0 msgr2=0x7f6f900cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 shutdown_connections 2024-04-25T10:29:11.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.236+0000 7f6f95000640 1 -- 172.21.15.45:0/3333664965 wait complete. 2024-04-25T10:29:11.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:29:11.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.396+0000 7f2cc4600640 1 Processor -- start 2024-04-25T10:29:11.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.396+0000 7f2cc4600640 1 -- start start 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc4600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2cbc159540 0x7f2cbc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc4600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cbc0aebd0 0x7f2cbc071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc4600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc071360 con 0x7f2cbc072020 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc4600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc159920 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc4600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc15c2f0 con 0x7f2cbc159540 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cbc0aebd0 0x7f2cbc071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cbc0aebd0 0x7f2cbc071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52208/0 (socket says 172.21.15.45:52208) 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 -- 172.21.15.45:0/1341583890 learned_addr learned my addr 172.21.15.45:0/1341583890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1c00640 1 --2- 172.21.15.45:0/1341583890 >> v2:172.21.15.117:3300/0 conn(0x7f2cbc159540 0x7f2cbc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 -- 172.21.15.45:0/1341583890 >> v2:172.21.15.117:3300/0 conn(0x7f2cbc159540 msgr2=0x7f2cbc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 --2- 172.21.15.45:0/1341583890 >> v2:172.21.15.117:3300/0 conn(0x7f2cbc159540 0x7f2cbc0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 -- 172.21.15.45:0/1341583890 >> v1:172.21.15.117:6789/0 conn(0x7f2cbc072020 legacy=0x7f2cbc15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.400+0000 7f2cc1200640 1 -- 172.21.15.45:0/1341583890 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc15c5d0 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc1200640 1 --2- 172.21.15.45:0/1341583890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc071ae0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0b4c20 tx=0x7f2ca801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca92957e10e210b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cbbe00640 1 -- 172.21.15.45:0/1341583890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80b0070 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cbbe00640 1 -- 172.21.15.45:0/1341583890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca80ab600 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cbbe00640 1 -- 172.21.15.45:0/1341583890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80ab9d0 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 -- 172.21.15.45:0/1341583890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 msgr2=0x7f2cbc071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 --2- 172.21.15.45:0/1341583890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc071ae0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0b4c20 tx=0x7f2ca801c520 comp rx=0 tx=0).stop 2024-04-25T10:29:11.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 -- 172.21.15.45:0/1341583890 shutdown_connections 2024-04-25T10:29:11.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 --2- 172.21.15.45:0/1341583890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc071ae0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 --2- 172.21.15.45:0/1341583890 >> v2:172.21.15.117:3300/0 conn(0x7f2cbc159540 0x7f2cbc0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 -- 172.21.15.45:0/1341583890 >> 172.21.15.45:0/1341583890 conn(0x7f2cbc0669a0 msgr2=0x7f2cbc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 -- 172.21.15.45:0/1341583890 shutdown_connections 2024-04-25T10:29:11.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 -- 172.21.15.45:0/1341583890 wait complete. 2024-04-25T10:29:11.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 Processor -- start 2024-04-25T10:29:11.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.404+0000 7f2cc4600640 1 -- start start 2024-04-25T10:29:11.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cbc072020 0x7f2cbc14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc13d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc159540 0x7f2cbc159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc072430 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc06c0d0 con 0x7f2cbc159540 2024-04-25T10:29:11.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc168930 con 0x7f2cbc072020 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc13d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc13d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52212/0 (socket says 172.21.15.45:52212) 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 -- 172.21.15.45:0/220333658 learned_addr learned my addr 172.21.15.45:0/220333658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1c00640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cbc072020 0x7f2cbc14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc2600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc159540 0x7f2cbc159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 -- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cbc072020 msgr2=0x7f2cbc14e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cbc072020 0x7f2cbc14e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 -- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc159540 msgr2=0x7f2cbc159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc159540 0x7f2cbc159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 -- 172.21.15.45:0/220333658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc15a230 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1c00640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cbc072020 0x7f2cbc14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc1200640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc13d160 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c10 tx=0x7f2ca80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80b0040 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca80ba070 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80b4610 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc15a510 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc15aad0 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc04ef30 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.408+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ca80b47b0 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.412+0000 7f2cb9600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c9c066fb0 0x7f2c9c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.412+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2ca811ed10 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.412+0000 7f2cc1c00640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c9c066fb0 0x7f2c9c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.412+0000 7f2cc1c00640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c9c066fb0 0x7f2c9c069470 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b9f0 tx=0x7f2cb0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.416+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ca8124020 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.576+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2cbc15eed0 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.576+0000 7f2cb9600640 1 -- 172.21.15.45:0/220333658 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ca8016110 con 0x7f2cbc0aebd0 2024-04-25T10:29:11.579 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:29:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c9c066fb0 msgr2=0x7f2c9c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c9c066fb0 0x7f2c9c069470 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b9f0 tx=0x7f2cb0008040 comp rx=0 tx=0).stop 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 msgr2=0x7f2cbc13d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc13d160 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c10 tx=0x7f2ca80192c0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 shutdown_connections 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c9c066fb0 0x7f2c9c069470 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cbc159540 0x7f2cbc159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cbc0aebd0 0x7f2cbc13d160 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 --2- 172.21.15.45:0/220333658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cbc072020 0x7f2cbc14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 >> 172.21.15.45:0/220333658 conn(0x7f2cbc0669a0 msgr2=0x7f2cbc06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 shutdown_connections 2024-04-25T10:29:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.580+0000 7f2cc4600640 1 -- 172.21.15.45:0/220333658 wait complete. 2024-04-25T10:29:11.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:11.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 Processor -- start 2024-04-25T10:29:11.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 -- start start 2024-04-25T10:29:11.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1601550b0 0x7fc1600cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1600cfe70 0x7fc16015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc160071050 con 0x7fc16015b740 2024-04-25T10:29:11.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16015d040 con 0x7fc1601550b0 2024-04-25T10:29:11.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc167e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1600712f0 con 0x7fc1600cfe70 2024-04-25T10:29:11.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1601550b0 0x7fc1600cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1601550b0 0x7fc1600cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52228/0 (socket says 172.21.15.45:52228) 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 -- 172.21.15.45:0/660363472 learned_addr learned my addr 172.21.15.45:0/660363472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc164a00640 1 --2- 172.21.15.45:0/660363472 >> v2:172.21.15.117:3300/0 conn(0x7fc1600cfe70 0x7fc16015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 -- 172.21.15.45:0/660363472 >> v2:172.21.15.117:3300/0 conn(0x7fc1600cfe70 msgr2=0x7fc16015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 --2- 172.21.15.45:0/660363472 >> v2:172.21.15.117:3300/0 conn(0x7fc1600cfe70 0x7fc16015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 -- 172.21.15.45:0/660363472 >> v1:172.21.15.117:6789/0 conn(0x7fc16015b740 legacy=0x7fc16006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:11.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.740+0000 7fc165400640 1 -- 172.21.15.45:0/660363472 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1600715d0 con 0x7fc1601550b0 2024-04-25T10:29:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc165400640 1 --2- 172.21.15.45:0/660363472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1601550b0 0x7fc1600cf930 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0a5e26472d4c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc15fe00640 1 -- 172.21.15.45:0/660363472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540b0070 con 0x7fc1601550b0 2024-04-25T10:29:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc15fe00640 1 -- 172.21.15.45:0/660363472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1540ab600 con 0x7fc1601550b0 2024-04-25T10:29:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc15fe00640 1 -- 172.21.15.45:0/660363472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1540ab9d0 con 0x7fc1601550b0 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 -- 172.21.15.45:0/660363472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1601550b0 msgr2=0x7fc1600cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 --2- 172.21.15.45:0/660363472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1601550b0 0x7fc1600cf930 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc154009870 tx=0x7fc15401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 -- 172.21.15.45:0/660363472 shutdown_connections 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 --2- 172.21.15.45:0/660363472 >> v2:172.21.15.117:3300/0 conn(0x7fc1600cfe70 0x7fc16015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 --2- 172.21.15.45:0/660363472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1601550b0 0x7fc1600cf930 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 -- 172.21.15.45:0/660363472 >> 172.21.15.45:0/660363472 conn(0x7fc160066b70 msgr2=0x7fc160066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 -- 172.21.15.45:0/660363472 shutdown_connections 2024-04-25T10:29:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.744+0000 7fc167e00640 1 -- 172.21.15.45:0/660363472 wait complete. 2024-04-25T10:29:11.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 Processor -- start 2024-04-25T10:29:11.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- start start 2024-04-25T10:29:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1600cfe70 0x7fc16015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1601550b0 0x7fc160153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 0x7fc16013c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015bd30 con 0x7fc16015b740 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015bbb0 con 0x7fc1600cfe70 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16015c030 con 0x7fc1601550b0 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 0x7fc16013c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 0x7fc16013c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52230/0 (socket says 172.21.15.45:52230) 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc164a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1601550b0 0x7fc160153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 -- 172.21.15.45:0/1058191618 learned_addr learned my addr 172.21.15.45:0/1058191618 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165400640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1600cfe70 0x7fc16015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 -- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1601550b0 msgr2=0x7fc160153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1601550b0 0x7fc160153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 -- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1600cfe70 msgr2=0x7fc16015a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1600cfe70 0x7fc16015a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 -- 172.21.15.45:0/1058191618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16013c690 con 0x7fc16015b740 2024-04-25T10:29:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc165e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 0x7fc16013c150 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fc15800b9a0 tx=0x7fc15800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158019070 con 0x7fc16015b740 2024-04-25T10:29:11.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15800cba0 con 0x7fc16015b740 2024-04-25T10:29:11.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc160154750 con 0x7fc16015b740 2024-04-25T10:29:11.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158014440 con 0x7fc16015b740 2024-04-25T10:29:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1600af430 con 0x7fc16015b740 2024-04-25T10:29:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.748+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1600aef40 con 0x7fc16015b740 2024-04-25T10:29:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.752+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc15801c070 con 0x7fc16015b740 2024-04-25T10:29:11.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.752+0000 7fc15d600640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.752+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc158084cd0 con 0x7fc16015b740 2024-04-25T10:29:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.752+0000 7fc165400640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:11.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.752+0000 7fc165400640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc140066f60 0x7fc140069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc1540ad000 tx=0x7fc1540a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.756+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1580555b0 con 0x7fc16015b740 2024-04-25T10:29:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.952+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc16004ef30 con 0x7fc16015b740 2024-04-25T10:29:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.952+0000 7fc15d600640 1 -- 172.21.15.45:0/1058191618 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4952 (secure 0 0 0) 0x7fc158059650 con 0x7fc16015b740 2024-04-25T10:29:11.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:11.955 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":107,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:29:05.122046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8893":{"gid":8893,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:29:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 107 2024-04-25T10:29:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc140066f60 msgr2=0x7fc140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc140066f60 0x7fc140069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc1540ad000 tx=0x7fc1540a6000 comp rx=0 tx=0).stop 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 msgr2=0x7fc16013c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 0x7fc16013c150 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fc15800b9a0 tx=0x7fc15800be60 comp rx=0 tx=0).stop 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 shutdown_connections 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc140066f60 0x7fc140069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc16015b740 0x7fc16013c150 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1601550b0 0x7fc160153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 --2- 172.21.15.45:0/1058191618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1600cfe70 0x7fc16015a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 >> 172.21.15.45:0/1058191618 conn(0x7fc160066b70 msgr2=0x7fc16014e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 shutdown_connections 2024-04-25T10:29:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:11.956+0000 7fc167e00640 1 -- 172.21.15.45:0/1058191618 wait complete. 2024-04-25T10:29:11.978 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.116+0000 7f445ee00640 1 Processor -- start 2024-04-25T10:29:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.116+0000 7f445ee00640 1 -- start start 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44581550b0 0x7f44580cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f445815b740 0x7f445806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4458071050 con 0x7f44581550b0 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44580712f0 con 0x7f445815b740 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44580715d0 con 0x7f44580cfe70 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44581550b0 0x7f44580cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f44581550b0 0x7f44580cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52246/0 (socket says 172.21.15.45:52246) 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 -- 172.21.15.45:0/4250535572 learned_addr learned my addr 172.21.15.45:0/4250535572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f444f400640 1 --2- 172.21.15.45:0/4250535572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f445ce00640 1 --2- 172.21.15.45:0/4250535572 >> v2:172.21.15.117:3300/0 conn(0x7f445815b740 0x7f445806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 -- 172.21.15.45:0/4250535572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 msgr2=0x7f445815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 --2- 172.21.15.45:0/4250535572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 -- 172.21.15.45:0/4250535572 >> v2:172.21.15.117:3300/0 conn(0x7f445815b740 msgr2=0x7f445806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 --2- 172.21.15.45:0/4250535572 >> v2:172.21.15.117:3300/0 conn(0x7f445815b740 0x7f445806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.120+0000 7f4457e00640 1 -- 172.21.15.45:0/4250535572 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580718b0 con 0x7f44581550b0 2024-04-25T10:29:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f4457e00640 1 --2- 172.21.15.45:0/4250535572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f44580cf930 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f44580d04d0 tx=0x7f444801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7f90b7edf0bc58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f4457400640 1 -- 172.21.15.45:0/4250535572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480ac020 con 0x7f44581550b0 2024-04-25T10:29:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f4457400640 1 -- 172.21.15.45:0/4250535572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44480a6440 con 0x7f44581550b0 2024-04-25T10:29:12.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f4457400640 1 -- 172.21.15.45:0/4250535572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480a6810 con 0x7f44581550b0 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 -- 172.21.15.45:0/4250535572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 msgr2=0x7f44580cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 --2- 172.21.15.45:0/4250535572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f44580cf930 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f44580d04d0 tx=0x7f444801c730 comp rx=0 tx=0).stop 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 -- 172.21.15.45:0/4250535572 shutdown_connections 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 --2- 172.21.15.45:0/4250535572 >> v2:172.21.15.117:3300/0 conn(0x7f445815b740 0x7f445806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 --2- 172.21.15.45:0/4250535572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 --2- 172.21.15.45:0/4250535572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f44580cf930 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 -- 172.21.15.45:0/4250535572 >> 172.21.15.45:0/4250535572 conn(0x7f4458066b70 msgr2=0x7f4458066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 -- 172.21.15.45:0/4250535572 shutdown_connections 2024-04-25T10:29:12.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 -- 172.21.15.45:0/4250535572 wait complete. 2024-04-25T10:29:12.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 Processor -- start 2024-04-25T10:29:12.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.124+0000 7f445ee00640 1 -- start start 2024-04-25T10:29:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f445814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f445815b740 0x7f4458157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580b3260 con 0x7f44581550b0 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445815bd60 con 0x7f445815b740 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44580b33e0 con 0x7f44580cfe70 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f445814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f445814a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52262/0 (socket says 172.21.15.45:52262) 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 -- 172.21.15.45:0/628836262 learned_addr learned my addr 172.21.15.45:0/628836262 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f4457e00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ce00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f445815b740 0x7f4458157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 -- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 msgr2=0x7f445814bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445814bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 -- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f445815b740 msgr2=0x7f4458157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f445815b740 0x7f4458157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 -- 172.21.15.45:0/628836262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4458158a00 con 0x7f44581550b0 2024-04-25T10:29:12.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f444f400640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f445814a1c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f444000e960 tx=0x7f444000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f44581550b0 2024-04-25T10:29:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000ca00 con 0x7f44581550b0 2024-04-25T10:29:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000cdb0 con 0x7f44581550b0 2024-04-25T10:29:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445814c710 con 0x7f44581550b0 2024-04-25T10:29:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445814ccd0 con 0x7f44581550b0 2024-04-25T10:29:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445804ef30 con 0x7f44581550b0 2024-04-25T10:29:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.128+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f444001a070 con 0x7f44581550b0 2024-04-25T10:29:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.132+0000 7f4454c00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4434066f60 0x7f4434069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.132+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4440085130 con 0x7f44581550b0 2024-04-25T10:29:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.132+0000 7f4457e00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4434066f60 0x7f4434069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.132+0000 7f4457e00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4434066f60 0x7f4434069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f44480ad000 tx=0x7f44480a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.136+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4440056060 con 0x7f44581550b0 2024-04-25T10:29:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.300+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f445815b390 con 0x7f44581550b0 2024-04-25T10:29:12.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.300+0000 7f4454c00640 1 -- 172.21.15.45:0/628836262 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f4440055960 con 0x7f44581550b0 2024-04-25T10:29:12.302 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:12.302 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":143,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:44.069863+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":2,"to":5}]},{"pgid":"21.2f","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4434066f60 msgr2=0x7f4434069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4434066f60 0x7f4434069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f44480ad000 tx=0x7f44480a7000 comp rx=0 tx=0).stop 2024-04-25T10:29:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 msgr2=0x7f445814a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f445814a1c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f444000e960 tx=0x7f444000ee20 comp rx=0 tx=0).stop 2024-04-25T10:29:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 shutdown_connections 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4434066f60 0x7f4434069420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f445815b740 0x7f4458157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44581550b0 0x7f445814a1c0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 --2- 172.21.15.45:0/628836262 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44580cfe70 0x7f445814bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 >> 172.21.15.45:0/628836262 conn(0x7f4458066b70 msgr2=0x7f445814f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 shutdown_connections 2024-04-25T10:29:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.304+0000 7f445ee00640 1 -- 172.21.15.45:0/628836262 wait complete. 2024-04-25T10:29:12.323 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:29:12.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 Processor -- start 2024-04-25T10:29:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 -- start start 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c0ccb20 0x7fb86c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb86c15bdc0 0x7fb86c0d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c0ccfb0 con 0x7fb86c15b9e0 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c0cd290 con 0x7fb86c15bdc0 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb872000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0cd570 con 0x7fb86c0ccb20 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb86b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb86c15b9e0 0x7fb86c0cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40738/0 (socket says 172.21.15.45:40738) 2024-04-25T10:29:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb86b400640 1 -- 172.21.15.45:0/2817937214 learned_addr learned my addr 172.21.15.45:0/2817937214 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb862a00640 1 --2- 172.21.15.45:0/2817937214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c0ccb20 0x7fb86c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb86be00640 1 --2- 172.21.15.45:0/2817937214 >> v2:172.21.15.117:3300/0 conn(0x7fb86c15bdc0 0x7fb86c0d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb86aa00640 1 -- 172.21.15.45:0/2817937214 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881066200 0 0) 0x7fb86c0ccfb0 con 0x7fb86c15b9e0 2024-04-25T10:29:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb86aa00640 1 -- 172.21.15.45:0/2817937214 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb850003530 con 0x7fb86c15b9e0 2024-04-25T10:29:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb862a00640 1 -- 172.21.15.45:0/2817937214 >> v2:172.21.15.117:3300/0 conn(0x7fb86c15bdc0 msgr2=0x7fb86c0d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb86aa00640 1 -- 172.21.15.45:0/2817937214 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649411764 0 0) 0x7fb850003530 con 0x7fb86c15b9e0 2024-04-25T10:29:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb862a00640 1 --2- 172.21.15.45:0/2817937214 >> v2:172.21.15.117:3300/0 conn(0x7fb86c15bdc0 0x7fb86c0d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb862a00640 1 -- 172.21.15.45:0/2817937214 >> v1:172.21.15.45:6789/0 conn(0x7fb86c15b9e0 legacy=0x7fb86c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.464+0000 7fb862a00640 1 -- 172.21.15.45:0/2817937214 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c0cd850 con 0x7fb86c0ccb20 2024-04-25T10:29:12.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb862a00640 1 --2- 172.21.15.45:0/2817937214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c0ccb20 0x7fb86c0d1520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8540097c0 tx=0x7fb85401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca0e86cfd7658e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb86aa00640 1 -- 172.21.15.45:0/2817937214 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8540b0070 con 0x7fb86c0ccb20 2024-04-25T10:29:12.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb86aa00640 1 -- 172.21.15.45:0/2817937214 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8540ab400 con 0x7fb86c0ccb20 2024-04-25T10:29:12.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb86aa00640 1 -- 172.21.15.45:0/2817937214 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c001680 con 0x7fb86c0ccb20 2024-04-25T10:29:12.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 -- 172.21.15.45:0/2817937214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c0ccb20 msgr2=0x7fb86c0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 --2- 172.21.15.45:0/2817937214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c0ccb20 0x7fb86c0d1520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb8540097c0 tx=0x7fb85401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 -- 172.21.15.45:0/2817937214 shutdown_connections 2024-04-25T10:29:12.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 --2- 172.21.15.45:0/2817937214 >> v2:172.21.15.117:3300/0 conn(0x7fb86c15bdc0 0x7fb86c0d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 --2- 172.21.15.45:0/2817937214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c0ccb20 0x7fb86c0d1520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 -- 172.21.15.45:0/2817937214 >> 172.21.15.45:0/2817937214 conn(0x7fb86c09a9d0 msgr2=0x7fb86c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 -- 172.21.15.45:0/2817937214 shutdown_connections 2024-04-25T10:29:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.468+0000 7fb872000640 1 -- 172.21.15.45:0/2817937214 wait complete. 2024-04-25T10:29:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 Processor -- start 2024-04-25T10:29:12.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 -- start start 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c15b9e0 0x7fb86c1531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c15bdc0 0x7fb86c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 0x7fb86c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c6630 con 0x7fb86c153730 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c64b0 con 0x7fb86c15b9e0 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c67b0 con 0x7fb86c15bdc0 2024-04-25T10:29:12.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 0x7fb86c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 0x7fb86c15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52270/0 (socket says 172.21.15.45:52270) 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 -- 172.21.15.45:0/2573084313 learned_addr learned my addr 172.21.15.45:0/2573084313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb862a00640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c15bdc0 0x7fb86c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86b400640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c15b9e0 0x7fb86c1531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 -- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c15bdc0 msgr2=0x7fb86c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c15bdc0 0x7fb86c15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 -- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c15b9e0 msgr2=0x7fb86c1531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c15b9e0 0x7fb86c1531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 -- 172.21.15.45:0/2573084313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c1596e0 con 0x7fb86c153730 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb86be00640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 0x7fb86c15c820 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb864009d30 tx=0x7fb86400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864018020 con 0x7fb86c153730 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864014480 con 0x7fb86c153730 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c1599c0 con 0x7fb86c153730 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864014870 con 0x7fb86c153730 2024-04-25T10:29:12.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.472+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c0c4ea0 con 0x7fb86c153730 2024-04-25T10:29:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.476+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c04ef30 con 0x7fb86c153730 2024-04-25T10:29:12.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.476+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb86401b020 con 0x7fb86c153730 2024-04-25T10:29:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.476+0000 7fb863e00640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb848066f60 0x7fb848069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.476+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb864085a70 con 0x7fb86c153730 2024-04-25T10:29:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.476+0000 7fb86b400640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb848066f60 0x7fb848069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.480+0000 7fb86b400640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb848066f60 0x7fb848069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0011f0 tx=0x7fb85c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.480+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb864022070 con 0x7fb86c153730 2024-04-25T10:29:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb86c159d20 con 0x7fb86c153730 2024-04-25T10:29:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb863e00640 1 -- 172.21.15.45:0/2573084313 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4952 (secure 0 0 0) 0x7fb864085e20 con 0x7fb86c153730 2024-04-25T10:29:12.702 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:12.702 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":107,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:29:05.122046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8893":{"gid":8893,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:29:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 107 2024-04-25T10:29:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb848066f60 msgr2=0x7fb848069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb848066f60 0x7fb848069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0011f0 tx=0x7fb85c009040 comp rx=0 tx=0).stop 2024-04-25T10:29:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 msgr2=0x7fb86c15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 0x7fb86c15c820 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb864009d30 tx=0x7fb86400c490 comp rx=0 tx=0).stop 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 shutdown_connections 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb848066f60 0x7fb848069420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb86c153730 0x7fb86c15c820 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb86c15bdc0 0x7fb86c15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 --2- 172.21.15.45:0/2573084313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c15b9e0 0x7fb86c1531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 >> 172.21.15.45:0/2573084313 conn(0x7fb86c09a9d0 msgr2=0x7fb86c06e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.680+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 shutdown_connections 2024-04-25T10:29:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.684+0000 7fb872000640 1 -- 172.21.15.45:0/2573084313 wait complete. 2024-04-25T10:29:12.705 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 Processor -- start 2024-04-25T10:29:12.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 -- start start 2024-04-25T10:29:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c06df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb54c0af1b0 0x7fb54c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb54c0af630 0x7fb54c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c15f100 con 0x7fb54c0af1b0 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c06e4a0 con 0x7fb54c0af630 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb552200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c15e4f0 con 0x7fb54c1580d0 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb54c0af1b0 0x7fb54c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb54c0af1b0 0x7fb54c0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52274/0 (socket says 172.21.15.45:52274) 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 -- 172.21.15.45:0/1272788267 learned_addr learned my addr 172.21.15.45:0/1272788267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb551800640 1 --2- 172.21.15.45:0/1272788267 >> v2:172.21.15.117:3300/0 conn(0x7fb54c0af630 0x7fb54c0d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb550e00640 1 --2- 172.21.15.45:0/1272788267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c06df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 -- 172.21.15.45:0/1272788267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 msgr2=0x7fb54c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 --2- 172.21.15.45:0/1272788267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c06df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 -- 172.21.15.45:0/1272788267 >> v2:172.21.15.117:3300/0 conn(0x7fb54c0af630 msgr2=0x7fb54c0d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:12.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 --2- 172.21.15.45:0/1272788267 >> v2:172.21.15.117:3300/0 conn(0x7fb54c0af630 0x7fb54c0d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.848+0000 7fb543e00640 1 -- 172.21.15.45:0/1272788267 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c15e7d0 con 0x7fb54c0af1b0 2024-04-25T10:29:12.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb543e00640 1 --2- 172.21.15.45:0/1272788267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af1b0 0x7fb54c0cfda0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fb54c071700 tx=0x7fb53401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f13256b27afe9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb54be00640 1 -- 172.21.15.45:0/1272788267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340ab620 con 0x7fb54c0af1b0 2024-04-25T10:29:12.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb54be00640 1 -- 172.21.15.45:0/1272788267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340b0070 con 0x7fb54c0af1b0 2024-04-25T10:29:12.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb54be00640 1 -- 172.21.15.45:0/1272788267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340ab9e0 con 0x7fb54c0af1b0 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 -- 172.21.15.45:0/1272788267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af1b0 msgr2=0x7fb54c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 --2- 172.21.15.45:0/1272788267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af1b0 0x7fb54c0cfda0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fb54c071700 tx=0x7fb53401c460 comp rx=0 tx=0).stop 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 -- 172.21.15.45:0/1272788267 shutdown_connections 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 --2- 172.21.15.45:0/1272788267 >> v2:172.21.15.117:3300/0 conn(0x7fb54c0af630 0x7fb54c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 --2- 172.21.15.45:0/1272788267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af1b0 0x7fb54c0cfda0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 --2- 172.21.15.45:0/1272788267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c06df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 -- 172.21.15.45:0/1272788267 >> 172.21.15.45:0/1272788267 conn(0x7fb54c06b4e0 msgr2=0x7fb54c0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 -- 172.21.15.45:0/1272788267 shutdown_connections 2024-04-25T10:29:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.852+0000 7fb552200640 1 -- 172.21.15.45:0/1272788267 wait complete. 2024-04-25T10:29:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 Processor -- start 2024-04-25T10:29:12.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 -- start start 2024-04-25T10:29:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb54c0af1b0 0x7fb54c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 0x7fb54c14bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c15e260 con 0x7fb54c0af630 2024-04-25T10:29:12.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c15e0e0 con 0x7fb54c0af1b0 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c162230 con 0x7fb54c1580d0 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 0x7fb54c14bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 0x7fb54c14bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52286/0 (socket says 172.21.15.45:52286) 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 -- 172.21.15.45:0/2183002515 learned_addr learned my addr 172.21.15.45:0/2183002515 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb550e00640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb54c0af1b0 0x7fb54c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb551800640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 -- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 msgr2=0x7fb54c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 -- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb54c0af1b0 msgr2=0x7fb54c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:12.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb54c0af1b0 0x7fb54c1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 -- 172.21.15.45:0/2183002515 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c0aff60 con 0x7fb54c0af630 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb543e00640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 0x7fb54c14bc00 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fb5340ad040 tx=0x7fb53401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340abdd0 con 0x7fb54c0af630 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340be3f0 con 0x7fb54c0af630 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340be7e0 con 0x7fb54c0af630 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54c14f310 con 0x7fb54c0af630 2024-04-25T10:29:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.856+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54c1607d0 con 0x7fb54c0af630 2024-04-25T10:29:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.860+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54c075120 con 0x7fb54c0af630 2024-04-25T10:29:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.860+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5340b43f0 con 0x7fb54c0af630 2024-04-25T10:29:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.860+0000 7fb549600640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb528066f60 0x7fb528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.860+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb53411f080 con 0x7fb54c0af630 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.860+0000 7fb550e00640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb528066f60 0x7fb528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.860+0000 7fb550e00640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb528066f60 0x7fb528069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb53c007a60 tx=0x7fb53c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:12.864+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb534016110 con 0x7fb54c0af630 2024-04-25T10:29:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb54c0cc5b0 con 0x7fb54c0af630 2024-04-25T10:29:13.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb549600640 1 -- 172.21.15.45:0/2183002515 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4952 (secure 0 0 0) 0x7fb5340ef8d0 con 0x7fb54c0af630 2024-04-25T10:29:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:13.061 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":107,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":107,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:28:36.425124+0000","modified":"2024-04-25T10:29:05.122046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8893":{"gid":8893,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6836/2453939747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2453939747},{"type":"v1","addr":"172.21.15.117:6836","nonce":2453939747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9531":{"gid":9531,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/903752757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":903752757},{"type":"v1","addr":"172.21.15.45:6835","nonce":903752757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:29:13.061 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 107 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb528066f60 msgr2=0x7fb528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb528066f60 0x7fb528069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb53c007a60 tx=0x7fb53c008040 comp rx=0 tx=0).stop 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 msgr2=0x7fb54c14bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 0x7fb54c14bc00 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fb5340ad040 tx=0x7fb53401aca0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 shutdown_connections 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb528066f60 0x7fb528069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb54c1580d0 0x7fb54c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb54c0af630 0x7fb54c14bc00 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 --2- 172.21.15.45:0/2183002515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb54c0af1b0 0x7fb54c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 >> 172.21.15.45:0/2183002515 conn(0x7fb54c06b4e0 msgr2=0x7fb54c0b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 shutdown_connections 2024-04-25T10:29:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.060+0000 7fb552200640 1 -- 172.21.15.45:0/2183002515 wait complete. 2024-04-25T10:29:13.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:13.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 Processor -- start 2024-04-25T10:29:13.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 -- start start 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f336815dc10 0x7f33680d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f336815dff0 0x7f336806b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33680cbff0 con 0x7f33680d09e0 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33680b45a0 con 0x7f336815dc10 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3367e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33680cbda0 con 0x7f336815dff0 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f336815dc10 0x7f33680d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f336815dc10 0x7f33680d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52300/0 (socket says 172.21.15.45:52300) 2024-04-25T10:29:13.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 -- 172.21.15.45:0/1026494632 learned_addr learned my addr 172.21.15.45:0/1026494632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 -- 172.21.15.45:0/1026494632 >> v2:172.21.15.117:3300/0 conn(0x7f336815dff0 msgr2=0x7f336806b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 --2- 172.21.15.45:0/1026494632 >> v2:172.21.15.117:3300/0 conn(0x7f336815dff0 0x7f336806b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 -- 172.21.15.45:0/1026494632 >> v1:172.21.15.45:6789/0 conn(0x7f33680d09e0 legacy=0x7f33680ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.224+0000 7f3366a00640 1 -- 172.21.15.45:0/1026494632 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33680cc2d0 con 0x7f336815dc10 2024-04-25T10:29:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3366a00640 1 --2- 172.21.15.45:0/1026494632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33680d04a0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3368072070 tx=0x7f335801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d055782f7538726b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3366000640 1 -- 172.21.15.45:0/1026494632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580ab600 con 0x7f336815dc10 2024-04-25T10:29:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3366000640 1 -- 172.21.15.45:0/1026494632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33580b0070 con 0x7f336815dc10 2024-04-25T10:29:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3366000640 1 -- 172.21.15.45:0/1026494632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580ab9c0 con 0x7f336815dc10 2024-04-25T10:29:13.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 -- 172.21.15.45:0/1026494632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 msgr2=0x7f33680d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 --2- 172.21.15.45:0/1026494632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33680d04a0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3368072070 tx=0x7f335801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 -- 172.21.15.45:0/1026494632 shutdown_connections 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 --2- 172.21.15.45:0/1026494632 >> v2:172.21.15.117:3300/0 conn(0x7f336815dff0 0x7f336806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 --2- 172.21.15.45:0/1026494632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33680d04a0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 -- 172.21.15.45:0/1026494632 >> 172.21.15.45:0/1026494632 conn(0x7f33680d1750 msgr2=0x7f33680d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 -- 172.21.15.45:0/1026494632 shutdown_connections 2024-04-25T10:29:13.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 -- 172.21.15.45:0/1026494632 wait complete. 2024-04-25T10:29:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 Processor -- start 2024-04-25T10:29:13.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.228+0000 7f3367e00640 1 -- start start 2024-04-25T10:29:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33680d09e0 0x7f33680c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33681f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f336815dff0 0x7f336809edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336806b1e0 con 0x7f336815dc10 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336806b060 con 0x7f336815dff0 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336816a960 con 0x7f33680d09e0 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33681f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33681f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52308/0 (socket says 172.21.15.45:52308) 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 -- 172.21.15.45:0/1883377731 learned_addr learned my addr 172.21.15.45:0/1883377731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367400640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f336815dff0 0x7f336809edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3366a00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33680d09e0 0x7f33680c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 -- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33680d09e0 msgr2=0x7f33680c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33680d09e0 0x7f33680c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 -- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f336815dff0 msgr2=0x7f336809edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f336815dff0 0x7f336809edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 -- 172.21.15.45:0/1883377731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33681f7ed0 con 0x7f336815dc10 2024-04-25T10:29:13.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335fe00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33681f7990 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f335000b440 tx=0x7f335000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350015020 con 0x7f336815dc10 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33500118b0 con 0x7f336815dc10 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350011ca0 con 0x7f336815dc10 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336809f330 con 0x7f336815dc10 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33681f6eb0 con 0x7f336815dc10 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3368075120 con 0x7f336815dc10 2024-04-25T10:29:13.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.232+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3350018020 con 0x7f336815dc10 2024-04-25T10:29:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.236+0000 7f335ea00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3344066f60 0x7f3344069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.236+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3350085210 con 0x7f336815dc10 2024-04-25T10:29:13.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.236+0000 7f3366a00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3344066f60 0x7f3344069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.236+0000 7f3366a00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3344066f60 0x7f3344069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f33580af000 tx=0x7f33580a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.240+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f335001f020 con 0x7f336815dc10 2024-04-25T10:29:13.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.404+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f336815ea20 con 0x7f336815dc10 2024-04-25T10:29:13.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.404+0000 7f335ea00640 1 -- 172.21.15.45:0/1883377731 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f335000d040 con 0x7f336815dc10 2024-04-25T10:29:13.406 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:13.407 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":143,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:28:44.069863+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:28:34.874797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T10:28:35.415884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":2,"to":5}]},{"pgid":"21.2f","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:13.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3344066f60 msgr2=0x7f3344069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3344066f60 0x7f3344069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f33580af000 tx=0x7f33580a7000 comp rx=0 tx=0).stop 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 msgr2=0x7f33681f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33681f7990 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f335000b440 tx=0x7f335000b900 comp rx=0 tx=0).stop 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 shutdown_connections 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3344066f60 0x7f3344069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f336815dff0 0x7f336809edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336815dc10 0x7f33681f7990 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 --2- 172.21.15.45:0/1883377731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33680d09e0 0x7f33680c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 >> 172.21.15.45:0/1883377731 conn(0x7f33680d1750 msgr2=0x7f33680b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 shutdown_connections 2024-04-25T10:29:13.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.408+0000 7f3367e00640 1 -- 172.21.15.45:0/1883377731 wait complete. 2024-04-25T10:29:13.428 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:29:13.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.568+0000 7f0ea9000640 1 Processor -- start 2024-04-25T10:29:13.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.568+0000 7f0ea9000640 1 -- start start 2024-04-25T10:29:13.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.568+0000 7f0ea9000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0ea40d0040 0x7f0ea406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.568+0000 7f0ea9000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea40b2770 con 0x7f0ea40cda10 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.568+0000 7f0ea9000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea406c670 con 0x7f0ea415bc40 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.568+0000 7f0ea9000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea406c950 con 0x7f0ea40d0040 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ea40cda10 0x7f0ea40cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40768/0 (socket says 172.21.15.45:40768) 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 -- 172.21.15.45:0/1180318960 learned_addr learned my addr 172.21.15.45:0/1180318960 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea2800640 1 --2- 172.21.15.45:0/1180318960 >> v2:172.21.15.117:3300/0 conn(0x7f0ea40d0040 0x7f0ea406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543580677 0 0) 0x7f0ea40b2770 con 0x7f0ea40cda10 2024-04-25T10:29:13.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e88003530 con 0x7f0ea40cda10 2024-04-25T10:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051378059 0 0) 0x7f0ea406c670 con 0x7f0ea415bc40 2024-04-25T10:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea40b2770 con 0x7f0ea415bc40 2024-04-25T10:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122954728 0 0) 0x7f0e88003530 con 0x7f0ea40cda10 2024-04-25T10:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 >> v2:172.21.15.117:3300/0 conn(0x7f0ea40d0040 msgr2=0x7f0ea406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 --2- 172.21.15.45:0/1180318960 >> v2:172.21.15.117:3300/0 conn(0x7f0ea40d0040 0x7f0ea406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 >> v1:172.21.15.117:6789/0 conn(0x7f0ea415bc40 legacy=0x7f0ea40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea406cc30 con 0x7f0ea40cda10 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f0e8c001680 con 0x7f0ea40cda10 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 >> v1:172.21.15.45:6789/0 conn(0x7f0ea40cda10 legacy=0x7f0ea40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88003b50 0x7f0e88003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88004810 0x7f0e88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e880051d0 0x7f0e8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea406c950 con 0x7f0e88004810 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8800c100 con 0x7f0e880051d0 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8800c300 con 0x7f0e88003b50 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88004810 0x7f0e88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea1e00640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88003b50 0x7f0e88003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea2800640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e880051d0 0x7f0e8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 -- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88003b50 msgr2=0x7f0e88003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88003b50 0x7f0e88003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 -- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e880051d0 msgr2=0x7f0e8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e880051d0 0x7f0e8800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0e99400640 1 -- 172.21.15.45:0/1180318960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88003530 con 0x7f0e88004810 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea2800640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e880051d0 0x7f0e8800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.572+0000 7f0ea2800640 1 -- 172.21.15.45:0/1180318960 reap_dead start 2024-04-25T10:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0e99400640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88004810 0x7f0e88004c90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c160 tx=0x7f0e8c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d74dfa3b26d6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c007cb0 con 0x7f0e88004810 2024-04-25T10:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0a9020 con 0x7f0e88004810 2024-04-25T10:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea1400640 1 -- 172.21.15.45:0/1180318960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c005ae0 con 0x7f0e88004810 2024-04-25T10:29:13.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 -- 172.21.15.45:0/1180318960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88004810 msgr2=0x7f0e88004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88004810 0x7f0e88004c90 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c160 tx=0x7f0e8c01cbe0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 -- 172.21.15.45:0/1180318960 shutdown_connections 2024-04-25T10:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 --2- 172.21.15.45:0/1180318960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88004810 0x7f0e88004c90 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 -- 172.21.15.45:0/1180318960 >> 172.21.15.45:0/1180318960 conn(0x7f0ea4066b70 msgr2=0x7f0ea4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 -- 172.21.15.45:0/1180318960 shutdown_connections 2024-04-25T10:29:13.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 -- 172.21.15.45:0/1180318960 wait complete. 2024-04-25T10:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 Processor -- start 2024-04-25T10:29:13.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.576+0000 7f0ea9000640 1 -- start start 2024-04-25T10:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88004810 0x7f0ea4155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 0x7f0ea41e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea40ad840 0x7f0ea40adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40b49a0 con 0x7f0ea41e7da0 2024-04-25T10:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40b4820 con 0x7f0ea40ad840 2024-04-25T10:29:13.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40b4b20 con 0x7f0e88004810 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 0x7f0ea41e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 0x7f0ea41e8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52332/0 (socket says 172.21.15.45:52332) 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 -- 172.21.15.45:0/3858793128 learned_addr learned my addr 172.21.15.45:0/3858793128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea1e00640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88004810 0x7f0ea4155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea2800640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea40ad840 0x7f0ea40adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 -- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88004810 msgr2=0x7f0ea4155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88004810 0x7f0ea4155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 -- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea40ad840 msgr2=0x7f0ea40adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea40ad840 0x7f0ea40adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 -- 172.21.15.45:0/3858793128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4155990 con 0x7f0ea41e7da0 2024-04-25T10:29:13.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e99400640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 0x7f0ea41e8220 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c001900 tx=0x7f0e88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c004c00 con 0x7f0ea41e7da0 2024-04-25T10:29:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0b2070 con 0x7f0ea41e7da0 2024-04-25T10:29:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea41eb990 con 0x7f0ea41e7da0 2024-04-25T10:29:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0070d0 con 0x7f0ea41e7da0 2024-04-25T10:29:13.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea41ebf50 con 0x7f0ea41e7da0 2024-04-25T10:29:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea404ef30 con 0x7f0ea41e7da0 2024-04-25T10:29:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.580+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e8c007270 con 0x7f0ea41e7da0 2024-04-25T10:29:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.584+0000 7f0e9aa00640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e80066f60 0x7f0e80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.584+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e8c11e280 con 0x7f0ea41e7da0 2024-04-25T10:29:13.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.584+0000 7f0ea1e00640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e80066f60 0x7f0e80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.584+0000 7f0ea1e00640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e80066f60 0x7f0e80069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c10 tx=0x7f0e94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.588+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e8c0168f0 con 0x7f0ea41e7da0 2024-04-25T10:29:13.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.792+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0ea406d960 con 0x7f0ea41e7da0 2024-04-25T10:29:13.796 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:29:13.796+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:29:13.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.796+0000 7f0e9aa00640 1 -- 172.21.15.45:0/3858793128 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e8c0eebf0 con 0x7f0ea41e7da0 2024-04-25T10:29:13.798 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:29:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e80066f60 msgr2=0x7f0e80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e80066f60 0x7f0e80069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c10 tx=0x7f0e94009040 comp rx=0 tx=0).stop 2024-04-25T10:29:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 msgr2=0x7f0ea41e8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 0x7f0ea41e8220 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c001900 tx=0x7f0e88005840 comp rx=0 tx=0).stop 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 shutdown_connections 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0e80066f60 0x7f0e80069420 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ea40ad840 0x7f0ea40adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41e7da0 0x7f0ea41e8220 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 --2- 172.21.15.45:0/3858793128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e88004810 0x7f0ea4155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 >> 172.21.15.45:0/3858793128 conn(0x7f0ea4066b70 msgr2=0x7f0ea4159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.800+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 shutdown_connections 2024-04-25T10:29:13.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.804+0000 7f0ea9000640 1 -- 172.21.15.45:0/3858793128 wait complete. 2024-04-25T10:29:13.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:29:13.867 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:29:13.870 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:29:13.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 Processor -- start 2024-04-25T10:29:13.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 -- start start 2024-04-25T10:29:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f56f80cc200 0x7f56f80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56f806cf40 0x7f56f806d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80cdab0 con 0x7f56f80d0d00 2024-04-25T10:29:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f806d8b0 con 0x7f56f80cc200 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56fda00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80cd210 con 0x7f56f806cf40 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56efe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f56f80cc200 0x7f56f80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56f6a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f56f80d0d00 0x7f56f8162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:55622/0 (socket says 172.21.15.45:55622) 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56f6a00640 1 -- 172.21.15.45:0/1793782160 learned_addr learned my addr 172.21.15.45:0/1793782160 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56f7400640 1 --2- 172.21.15.45:0/1793782160 >> v2:172.21.15.117:3300/0 conn(0x7f56f806cf40 0x7f56f806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56f6000640 1 -- 172.21.15.45:0/1793782160 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126085172 0 0) 0x7f56f80cdab0 con 0x7f56f80d0d00 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56f6000640 1 -- 172.21.15.45:0/1793782160 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56dc003530 con 0x7f56f80d0d00 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.972+0000 7f56efe00640 1 -- 172.21.15.45:0/1793782160 >> v2:172.21.15.117:3300/0 conn(0x7f56f806cf40 msgr2=0x7f56f806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56efe00640 1 --2- 172.21.15.45:0/1793782160 >> v2:172.21.15.117:3300/0 conn(0x7f56f806cf40 0x7f56f806d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56efe00640 1 -- 172.21.15.45:0/1793782160 >> v1:172.21.15.117:6789/0 conn(0x7f56f80d0d00 legacy=0x7f56f8162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56efe00640 1 -- 172.21.15.45:0/1793782160 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80cd4f0 con 0x7f56f80cc200 2024-04-25T10:29:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56efe00640 1 --2- 172.21.15.45:0/1793782160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f80cc200 0x7f56f80ccc30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f56f806f8f0 tx=0x7f56e001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d675bf8c3fb50b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56f6000640 1 -- 172.21.15.45:0/1793782160 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00b0070 con 0x7f56f80cc200 2024-04-25T10:29:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56f6000640 1 -- 172.21.15.45:0/1793782160 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e00ab600 con 0x7f56f80cc200 2024-04-25T10:29:13.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56f6000640 1 -- 172.21.15.45:0/1793782160 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00ab9d0 con 0x7f56f80cc200 2024-04-25T10:29:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 -- 172.21.15.45:0/1793782160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f80cc200 msgr2=0x7f56f80ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 --2- 172.21.15.45:0/1793782160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f80cc200 0x7f56f80ccc30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f56f806f8f0 tx=0x7f56e001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 -- 172.21.15.45:0/1793782160 shutdown_connections 2024-04-25T10:29:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 --2- 172.21.15.45:0/1793782160 >> v2:172.21.15.117:3300/0 conn(0x7f56f806cf40 0x7f56f806d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 --2- 172.21.15.45:0/1793782160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f80cc200 0x7f56f80ccc30 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 -- 172.21.15.45:0/1793782160 >> 172.21.15.45:0/1793782160 conn(0x7f56f8072290 msgr2=0x7f56f80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 -- 172.21.15.45:0/1793782160 shutdown_connections 2024-04-25T10:29:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.976+0000 7f56fda00640 1 -- 172.21.15.45:0/1793782160 wait complete. 2024-04-25T10:29:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 Processor -- start 2024-04-25T10:29:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 -- start start 2024-04-25T10:29:13.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 0x7f56f8098de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56f80cc200 0x7f56f81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56f80d0d00 0x7f56f81547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8162a10 con 0x7f56f806cf40 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8162890 con 0x7f56f80d0d00 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f815e040 con 0x7f56f80cc200 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 0x7f56f8098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 0x7f56f8098de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52376/0 (socket says 172.21.15.45:52376) 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 -- 172.21.15.45:0/15728706 learned_addr learned my addr 172.21.15.45:0/15728706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56efe00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56f80cc200 0x7f56f81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f7400640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56f80d0d00 0x7f56f81547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 -- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56f80cc200 msgr2=0x7f56f81542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56f80cc200 0x7f56f81542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 -- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56f80d0d00 msgr2=0x7f56f81547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56f80d0d00 0x7f56f81547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 -- 172.21.15.45:0/15728706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8099620 con 0x7f56f806cf40 2024-04-25T10:29:13.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56f6a00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 0x7f56f8098de0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c3c0 tx=0x7f56e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8017070 con 0x7f56f806cf40 2024-04-25T10:29:13.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8005cf0 con 0x7f56f806cf40 2024-04-25T10:29:13.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80072f0 con 0x7f56f806cf40 2024-04-25T10:29:13.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f8099900 con 0x7f56f806cf40 2024-04-25T10:29:13.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.980+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f8159410 con 0x7f56f806cf40 2024-04-25T10:29:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.984+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f804ef30 con 0x7f56f806cf40 2024-04-25T10:29:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.984+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56e801a070 con 0x7f56f806cf40 2024-04-25T10:29:13.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.984+0000 7f56eea00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f56d4066f60 0x7f56d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:13.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.984+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f56e8084410 con 0x7f56f806cf40 2024-04-25T10:29:13.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.984+0000 7f56efe00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f56d4066f60 0x7f56d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:13.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.984+0000 7f56efe00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f56d4066f60 0x7f56d4069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f56e00ad000 tx=0x7f56e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:13.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:13.988+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56e8054d30 con 0x7f56f806cf40 2024-04-25T10:29:14.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.184+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56f80ae540 con 0x7f56f806cf40 2024-04-25T10:29:14.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56eea00640 1 -- 172.21.15.45:0/15728706 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v109) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56e8058dd0 con 0x7f56f806cf40 2024-04-25T10:29:14.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f56d4066f60 msgr2=0x7f56d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f56d4066f60 0x7f56d4069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f56e00ad000 tx=0x7f56e0008040 comp rx=0 tx=0).stop 2024-04-25T10:29:14.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 msgr2=0x7f56f8098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 0x7f56f8098de0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c3c0 tx=0x7f56e800c880 comp rx=0 tx=0).stop 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 shutdown_connections 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f56d4066f60 0x7f56d4069420 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56f80d0d00 0x7f56f81547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56f80cc200 0x7f56f81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 --2- 172.21.15.45:0/15728706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f806cf40 0x7f56f8098de0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 >> 172.21.15.45:0/15728706 conn(0x7f56f8072290 msgr2=0x7f56f80cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 shutdown_connections 2024-04-25T10:29:14.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:14.800+0000 7f56fda00640 1 -- 172.21.15.45:0/15728706 wait complete. 2024-04-25T10:29:14.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:29:15.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16bd200640 1 Processor -- start 2024-04-25T10:29:15.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16bd200640 1 -- start start 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16bd200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16b8071ab0 0x7f16b80ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16bd200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b8071f80 con 0x7f16b80cf040 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16bd200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b815b120 con 0x7f16b80d00f0 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16bd200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b8072260 con 0x7f16b8071ab0 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16b6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16b8071ab0 0x7f16b80ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16b6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16b8071ab0 0x7f16b80ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33836/0 (socket says 172.21.15.45:33836) 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16b5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f16b80cf040 0x7f16b80cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54902/0 (socket says 172.21.15.45:54902) 2024-04-25T10:29:15.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.012+0000 7f16b6800640 1 -- 172.21.15.45:0/3674181452 learned_addr learned my addr 172.21.15.45:0/3674181452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b6800640 1 -- 172.21.15.45:0/3674181452 >> v1:172.21.15.117:6789/0 conn(0x7f16b80d00f0 legacy=0x7f16b80713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b6800640 1 -- 172.21.15.45:0/3674181452 >> v1:172.21.15.45:6789/0 conn(0x7f16b80cf040 legacy=0x7f16b80cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b6800640 1 -- 172.21.15.45:0/3674181452 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b815b400 con 0x7f16b8071ab0 2024-04-25T10:29:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b6800640 1 --2- 172.21.15.45:0/3674181452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b80ae480 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009e50 tx=0x7f16b001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50f8fb08d0a458e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b5400640 1 -- 172.21.15.45:0/3674181452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b00a5700 con 0x7f16b8071ab0 2024-04-25T10:29:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b5400640 1 -- 172.21.15.45:0/3674181452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b00a58a0 con 0x7f16b8071ab0 2024-04-25T10:29:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16b5400640 1 -- 172.21.15.45:0/3674181452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b00a5c70 con 0x7f16b8071ab0 2024-04-25T10:29:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16bd200640 1 -- 172.21.15.45:0/3674181452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 msgr2=0x7f16b80ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16bd200640 1 --2- 172.21.15.45:0/3674181452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b80ae480 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009e50 tx=0x7f16b001ca30 comp rx=0 tx=0).stop 2024-04-25T10:29:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16bd200640 1 -- 172.21.15.45:0/3674181452 shutdown_connections 2024-04-25T10:29:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16bd200640 1 --2- 172.21.15.45:0/3674181452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b80ae480 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16bd200640 1 -- 172.21.15.45:0/3674181452 >> 172.21.15.45:0/3674181452 conn(0x7f16b8066b70 msgr2=0x7f16b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.016+0000 7f16bd200640 1 -- 172.21.15.45:0/3674181452 shutdown_connections 2024-04-25T10:29:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- 172.21.15.45:0/3674181452 wait complete. 2024-04-25T10:29:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 Processor -- start 2024-04-25T10:29:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- start start 2024-04-25T10:29:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b80cf040 0x7f16b81534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b80d00f0 0x7f16b8165800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b806c0e0 con 0x7f16b8071ab0 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80cf630 con 0x7f16b80cf040 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b806c260 con 0x7f16b80d00f0 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b81513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33850/0 (socket says 172.21.15.45:33850) 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 -- 172.21.15.45:0/4169477089 learned_addr learned my addr 172.21.15.45:0/4169477089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16afe00640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b80cf040 0x7f16b81534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b6800640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b80d00f0 0x7f16b8165800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 -- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b80d00f0 msgr2=0x7f16b8165800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b80d00f0 0x7f16b8165800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 -- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b80cf040 msgr2=0x7f16b81534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b80cf040 0x7f16b81534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 -- 172.21.15.45:0/4169477089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b81502b0 con 0x7f16b8071ab0 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b6800640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b80d00f0 0x7f16b8165800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16b5e00640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b81513c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f16a800c490 tx=0x7f16a800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8018070 con 0x7f16b8071ab0 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8007480 con 0x7f16b8071ab0 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b8150600 con 0x7f16b8071ab0 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80047f0 con 0x7f16b8071ab0 2024-04-25T10:29:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.020+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8150bc0 con 0x7f16b8071ab0 2024-04-25T10:29:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.024+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b804ef30 con 0x7f16b8071ab0 2024-04-25T10:29:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.024+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16a801b070 con 0x7f16b8071ab0 2024-04-25T10:29:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.024+0000 7f16ae000640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1694066f60 0x7f1694069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.024+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f16a8084320 con 0x7f16b8071ab0 2024-04-25T10:29:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.024+0000 7f16afe00640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1694066f60 0x7f1694069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.028+0000 7f16afe00640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1694066f60 0x7f1694069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f16b80ae4a0 tx=0x7f16a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.028+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16a8054bb0 con 0x7f16b8071ab0 2024-04-25T10:29:15.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:15.196+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16b80b2a10 con 0x7f16b8071ab0 2024-04-25T10:29:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.156+0000 7f16ae000640 1 -- 172.21.15.45:0/4169477089 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v145) v1 ==== 181+0+0 (secure 0 0 0) 0x7f16a8058c50 con 0x7f16b8071ab0 2024-04-25T10:29:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1694066f60 msgr2=0x7f1694069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1694066f60 0x7f1694069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f16b80ae4a0 tx=0x7f16a0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 msgr2=0x7f16b81513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b81513c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f16a800c490 tx=0x7f16a800c950 comp rx=0 tx=0).stop 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 shutdown_connections 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1694066f60 0x7f1694069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16b80d00f0 0x7f16b8165800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b80cf040 0x7f16b81534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 --2- 172.21.15.45:0/4169477089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b8071ab0 0x7f16b81513c0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 >> 172.21.15.45:0/4169477089 conn(0x7f16b8066b70 msgr2=0x7f16b80cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:16.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 shutdown_connections 2024-04-25T10:29:16.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.160+0000 7f16bd200640 1 -- 172.21.15.45:0/4169477089 wait complete. 2024-04-25T10:29:16.185 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:29:16.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.332+0000 7f7e5ac00640 1 Processor -- start 2024-04-25T10:29:16.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.332+0000 7f7e5ac00640 1 -- start start 2024-04-25T10:29:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e5ac00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e540cf040 0x7f7e540cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e5ac00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e54071ab0 0x7f7e540cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:16.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e5ac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e54071f80 con 0x7f7e5415bc00 2024-04-25T10:29:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e5ac00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e54072260 con 0x7f7e54071ab0 2024-04-25T10:29:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e5ac00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e540b3700 con 0x7f7e540cf040 2024-04-25T10:29:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e58c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e54071ab0 0x7f7e540cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e4be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7e5415bc00 0x7f7e540713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54918/0 (socket says 172.21.15.45:54918) 2024-04-25T10:29:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e58c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e54071ab0 0x7f7e540cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33856/0 (socket says 172.21.15.45:33856) 2024-04-25T10:29:16.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e4be00640 1 -- 172.21.15.45:0/441978751 learned_addr learned my addr 172.21.15.45:0/441978751 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e53e00640 1 --2- 172.21.15.45:0/441978751 >> v2:172.21.15.117:3300/0 conn(0x7f7e540cf040 0x7f7e540cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e58c00640 1 -- 172.21.15.45:0/441978751 >> v2:172.21.15.117:3300/0 conn(0x7f7e540cf040 msgr2=0x7f7e540cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e58c00640 1 --2- 172.21.15.45:0/441978751 >> v2:172.21.15.117:3300/0 conn(0x7f7e540cf040 0x7f7e540cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e58c00640 1 -- 172.21.15.45:0/441978751 >> v1:172.21.15.45:6789/0 conn(0x7f7e5415bc00 legacy=0x7f7e540713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e53400640 1 -- 172.21.15.45:0/441978751 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224705653 0 0) 0x7f7e54071f80 con 0x7f7e5415bc00 2024-04-25T10:29:16.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.336+0000 7f7e58c00640 1 -- 172.21.15.45:0/441978751 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5415b120 con 0x7f7e54071ab0 2024-04-25T10:29:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e58c00640 1 --2- 172.21.15.45:0/441978751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e54071ab0 0x7f7e540cff20 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7e540d05d0 tx=0x7f7e4c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a5a45b02ffd414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e53400640 1 -- 172.21.15.45:0/441978751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0b2070 con 0x7f7e54071ab0 2024-04-25T10:29:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e53400640 1 -- 172.21.15.45:0/441978751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c0a58e0 con 0x7f7e54071ab0 2024-04-25T10:29:16.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e53400640 1 -- 172.21.15.45:0/441978751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0a5cb0 con 0x7f7e54071ab0 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 -- 172.21.15.45:0/441978751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e54071ab0 msgr2=0x7f7e540cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/441978751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e54071ab0 0x7f7e540cff20 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7e540d05d0 tx=0x7f7e4c01c930 comp rx=0 tx=0).stop 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 -- 172.21.15.45:0/441978751 shutdown_connections 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/441978751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e54071ab0 0x7f7e540cff20 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/441978751 >> v2:172.21.15.117:3300/0 conn(0x7f7e540cf040 0x7f7e540cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 -- 172.21.15.45:0/441978751 >> 172.21.15.45:0/441978751 conn(0x7f7e54066b70 msgr2=0x7f7e54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:16.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 -- 172.21.15.45:0/441978751 shutdown_connections 2024-04-25T10:29:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 -- 172.21.15.45:0/441978751 wait complete. 2024-04-25T10:29:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 Processor -- start 2024-04-25T10:29:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.340+0000 7f7e5ac00640 1 -- start start 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e54071ab0 0x7f7e5406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 0x7f7e54153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5415bc00 0x7f7e54153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540aee10 con 0x7f7e540cf040 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540aec90 con 0x7f7e5415bc00 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540aef90 con 0x7f7e54071ab0 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 0x7f7e54153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 0x7f7e54153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33858/0 (socket says 172.21.15.45:33858) 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 -- 172.21.15.45:0/2236610065 learned_addr learned my addr 172.21.15.45:0/2236610065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:16.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e58c00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5415bc00 0x7f7e54153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e53e00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e54071ab0 0x7f7e5406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 -- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e54071ab0 msgr2=0x7f7e5406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e54071ab0 0x7f7e5406c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 -- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5415bc00 msgr2=0x7f7e54153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5415bc00 0x7f7e54153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 -- 172.21.15.45:0/2236610065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e541543d0 con 0x7f7e540cf040 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e53e00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e54071ab0 0x7f7e5406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e4be00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 0x7f7e54153660 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00c520 tx=0x7f7e3c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c018070 con 0x7f7e540cf040 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c007580 con 0x7f7e540cf040 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c004a30 con 0x7f7e540cf040 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e541546b0 con 0x7f7e540cf040 2024-04-25T10:29:16.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e54154c70 con 0x7f7e540cf040 2024-04-25T10:29:16.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5404ef30 con 0x7f7e540cf040 2024-04-25T10:29:16.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.344+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e3c004bd0 con 0x7f7e540cf040 2024-04-25T10:29:16.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.348+0000 7f7e50c00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e30066f60 0x7f7e30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.348+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f7e3c084130 con 0x7f7e540cf040 2024-04-25T10:29:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.348+0000 7f7e53e00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e30066f60 0x7f7e30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.348+0000 7f7e53e00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e30066f60 0x7f7e30069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7e44007a70 tx=0x7f7e44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.352+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e3c04bcb0 con 0x7f7e540cf040 2024-04-25T10:29:16.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:16.512+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7e540b2a10 con 0x7f7e540cf040 2024-04-25T10:29:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.160+0000 7f7e50c00640 1 -- 172.21.15.45:0/2236610065 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v146) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7e3c054ca0 con 0x7f7e540cf040 2024-04-25T10:29:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:29:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e30066f60 msgr2=0x7f7e30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e30066f60 0x7f7e30069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7e44007a70 tx=0x7f7e44008040 comp rx=0 tx=0).stop 2024-04-25T10:29:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 msgr2=0x7f7e54153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 0x7f7e54153660 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00c520 tx=0x7f7e3c00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 shutdown_connections 2024-04-25T10:29:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7e30066f60 0x7f7e30069420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5415bc00 0x7f7e54153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e540cf040 0x7f7e54153660 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 --2- 172.21.15.45:0/2236610065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e54071ab0 0x7f7e5406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 >> 172.21.15.45:0/2236610065 conn(0x7f7e54066b70 msgr2=0x7f7e5415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 shutdown_connections 2024-04-25T10:29:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.164+0000 7f7e5ac00640 1 -- 172.21.15.45:0/2236610065 wait complete. 2024-04-25T10:29:17.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:29:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.340+0000 7fb8d9600640 1 Processor -- start 2024-04-25T10:29:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.340+0000 7fb8d9600640 1 -- start start 2024-04-25T10:29:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d406bd10 0x7fb8d4161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8d4161e50 0x7fb8d415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d9600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d406ac90 con 0x7fb8d40cd9a0 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d9600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d406b880 con 0x7fb8d4161e50 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d9600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d406af50 con 0x7fb8d406bd10 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8d4161e50 0x7fb8d415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8d4161e50 0x7fb8d415d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33870/0 (socket says 172.21.15.45:33870) 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb8d40cd9a0 0x7fb8d40d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54926/0 (socket says 172.21.15.45:54926) 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 -- 172.21.15.45:0/3580816338 learned_addr learned my addr 172.21.15.45:0/3580816338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8ca000640 1 --2- 172.21.15.45:0/3580816338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d406bd10 0x7fb8d4161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 -- 172.21.15.45:0/3580816338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d406bd10 msgr2=0x7fb8d4161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 --2- 172.21.15.45:0/3580816338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d406bd10 0x7fb8d4161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 -- 172.21.15.45:0/3580816338 >> v1:172.21.15.45:6789/0 conn(0x7fb8d40cd9a0 legacy=0x7fb8d40d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.344+0000 7fb8d3400640 1 -- 172.21.15.45:0/3580816338 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d406b230 con 0x7fb8d4161e50 2024-04-25T10:29:17.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d3400640 1 --2- 172.21.15.45:0/3580816338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d4161e50 0x7fb8d415d610 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009e50 tx=0x7fb8cc01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f15c3447f38dd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d2000640 1 -- 172.21.15.45:0/3580816338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0b4070 con 0x7fb8d4161e50 2024-04-25T10:29:17.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d2000640 1 -- 172.21.15.45:0/3580816338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc0af400 con 0x7fb8d4161e50 2024-04-25T10:29:17.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d2000640 1 -- 172.21.15.45:0/3580816338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0af7d0 con 0x7fb8d4161e50 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 -- 172.21.15.45:0/3580816338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d4161e50 msgr2=0x7fb8d415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 --2- 172.21.15.45:0/3580816338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d4161e50 0x7fb8d415d610 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009e50 tx=0x7fb8cc01f210 comp rx=0 tx=0).stop 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 -- 172.21.15.45:0/3580816338 shutdown_connections 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 --2- 172.21.15.45:0/3580816338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d4161e50 0x7fb8d415d610 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 --2- 172.21.15.45:0/3580816338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d406bd10 0x7fb8d4161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 -- 172.21.15.45:0/3580816338 >> 172.21.15.45:0/3580816338 conn(0x7fb8d40d19a0 msgr2=0x7fb8d40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 -- 172.21.15.45:0/3580816338 shutdown_connections 2024-04-25T10:29:17.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 -- 172.21.15.45:0/3580816338 wait complete. 2024-04-25T10:29:17.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 Processor -- start 2024-04-25T10:29:17.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.348+0000 7fb8d9600640 1 -- start start 2024-04-25T10:29:17.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 0x7fb8d415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d40cd9a0 0x7fb8d40cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8d4161e50 0x7fb8d40cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4162660 con 0x7fb8d406bd10 2024-04-25T10:29:17.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40cdd80 con 0x7fb8d4161e50 2024-04-25T10:29:17.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40d0d70 con 0x7fb8d40cd9a0 2024-04-25T10:29:17.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 0x7fb8d415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 0x7fb8d415ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33884/0 (socket says 172.21.15.45:33884) 2024-04-25T10:29:17.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 -- 172.21.15.45:0/2614034125 learned_addr learned my addr 172.21.15.45:0/2614034125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:17.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d3400640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8d4161e50 0x7fb8d40cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8ca000640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d40cd9a0 0x7fb8d40cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 -- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d40cd9a0 msgr2=0x7fb8d40cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d40cd9a0 0x7fb8d40cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 -- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8d4161e50 msgr2=0x7fb8d40cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8d4161e50 0x7fb8d40cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 -- 172.21.15.45:0/2614034125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d41f39d0 con 0x7fb8d406bd10 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d2a00640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 0x7fb8d415ce50 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c420 tx=0x7fb8c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40074e0 con 0x7fb8d406bd10 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c4007680 con 0x7fb8d406bd10 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d41f3cb0 con 0x7fb8d406bd10 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4004a30 con 0x7fb8d406bd10 2024-04-25T10:29:17.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d415d390 con 0x7fb8d406bd10 2024-04-25T10:29:17.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d404ef30 con 0x7fb8d406bd10 2024-04-25T10:29:17.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.352+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb8c401a020 con 0x7fb8d406bd10 2024-04-25T10:29:17.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.356+0000 7fb8cb400640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8b0066f60 0x7fb8b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.356+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8c4084580 con 0x7fb8d406bd10 2024-04-25T10:29:17.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.356+0000 7fb8ca000640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8b0066f60 0x7fb8b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.356+0000 7fb8ca000640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8b0066f60 0x7fb8b0069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00b930 tx=0x7fb8bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.360+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8c4055890 con 0x7fb8d406bd10 2024-04-25T10:29:17.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.544+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb8d40ceef0 con 0x7fb8d406bd10 2024-04-25T10:29:17.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.548+0000 7fb8cb400640 1 -- 172.21.15.45:0/2614034125 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8c4055190 con 0x7fb8d406bd10 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8b0066f60 msgr2=0x7fb8b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8b0066f60 0x7fb8b0069420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00b930 tx=0x7fb8bc008040 comp rx=0 tx=0).stop 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 msgr2=0x7fb8d415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 0x7fb8d415ce50 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c420 tx=0x7fb8c400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 shutdown_connections 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb8b0066f60 0x7fb8b0069420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8d4161e50 0x7fb8d40cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8d40cd9a0 0x7fb8d40cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 --2- 172.21.15.45:0/2614034125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8d406bd10 0x7fb8d415ce50 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 >> 172.21.15.45:0/2614034125 conn(0x7fb8d40d19a0 msgr2=0x7fb8d40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 shutdown_connections 2024-04-25T10:29:17.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.552+0000 7fb8d9600640 1 -- 172.21.15.45:0/2614034125 wait complete. 2024-04-25T10:29:17.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:29:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f130d400640 1 Processor -- start 2024-04-25T10:29:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f130d400640 1 -- start start 2024-04-25T10:29:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f130d400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f1308166820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f130d400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13080ccfb0 con 0x7f130815b9e0 2024-04-25T10:29:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f130d400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13080cd290 con 0x7f13080ccb20 2024-04-25T10:29:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f130d400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13080cd570 con 0x7f13080c6a20 2024-04-25T10:29:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1306800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f130815b9e0 0x7f13080cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54936/0 (socket says 172.21.15.45:54936) 2024-04-25T10:29:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1306800640 1 -- 172.21.15.45:0/2430452782 learned_addr learned my addr 172.21.15.45:0/2430452782 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1307200640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f1308166820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38806077 0 0) 0x7f13080ccfb0 con 0x7f130815b9e0 2024-04-25T10:29:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ec003530 con 0x7f130815b9e0 2024-04-25T10:29:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308424229 0 0) 0x7f13080cd290 con 0x7f13080ccb20 2024-04-25T10:29:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13080ccfb0 con 0x7f13080ccb20 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 890592325 0 0) 0x7f12ec003530 con 0x7f130815b9e0 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 msgr2=0x7f1308166820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f1308166820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 >> v1:172.21.15.117:6789/0 conn(0x7f13080ccb20 legacy=0x7f13080d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080cd850 con 0x7f130815b9e0 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f12f80016a0 con 0x7f130815b9e0 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1307200640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f1308166820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 >> v1:172.21.15.45:6789/0 conn(0x7f130815b9e0 legacy=0x7f13080cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f12ec003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f12ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d50 0x7f12ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13080cd570 con 0x7f12ec003b80 2024-04-25T10:29:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00ba80 con 0x7f12ec004d50 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.712+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec00bc30 con 0x7f13080c6a20 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f12ec003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f12fde00640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f12ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1307200640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d50 0x7f12ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 -- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 msgr2=0x7f12ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f12ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 -- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d50 msgr2=0x7f12ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12ec004d50 0x7f12ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 -- 172.21.15.45:0/2430452782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec003530 con 0x7f12ec003b80 2024-04-25T10:29:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f12fde00640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13080c6a20 0x7f12ec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1306800640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f12ec003f80 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f12f800c210 tx=0x7f12f801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f668db34e27f4b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8007cf0 con 0x7f12ec003b80 2024-04-25T10:29:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f80a9020 con 0x7f12ec003b80 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1305e00640 1 -- 172.21.15.45:0/2430452782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8005b00 con 0x7f12ec003b80 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f130d400640 1 -- 172.21.15.45:0/2430452782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 msgr2=0x7f12ec003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f130d400640 1 --2- 172.21.15.45:0/2430452782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f12ec003f80 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f12f800c210 tx=0x7f12f801cba0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.716+0000 7f1307200640 1 -- 172.21.15.45:0/2430452782 reap_dead start 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- 172.21.15.45:0/2430452782 shutdown_connections 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- 172.21.15.45:0/2430452782 >> 172.21.15.45:0/2430452782 conn(0x7f130809a9d0 msgr2=0x7f130809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- 172.21.15.45:0/2430452782 shutdown_connections 2024-04-25T10:29:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- 172.21.15.45:0/2430452782 wait complete. 2024-04-25T10:29:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 Processor -- start 2024-04-25T10:29:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- start start 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f130815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1308158c00 0x7f130814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1308155f00 0x7f1308156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13081673c0 con 0x7f12ec003b80 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308167240 con 0x7f1308155f00 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f130d400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1308167540 con 0x7f1308158c00 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f130815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f130815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33904/0 (socket says 172.21.15.45:33904) 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 -- 172.21.15.45:0/2434575585 learned_addr learned my addr 172.21.15.45:0/2434575585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f12fde00640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1308158c00 0x7f130814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1307200640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1308155f00 0x7f1308156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 -- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1308158c00 msgr2=0x7f130814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1308158c00 0x7f130814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 -- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1308155f00 msgr2=0x7f1308156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1308155f00 0x7f1308156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 -- 172.21.15.45:0/2434575585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308159060 con 0x7f12ec003b80 2024-04-25T10:29:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1307200640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1308155f00 0x7f1308156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:17.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f1306800640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f130815c260 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f12f80ab040 tx=0x7f12ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.720+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8007cf0 con 0x7f12ec003b80 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f80070d0 con 0x7f12ec003b80 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8001c30 con 0x7f12ec003b80 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130815c960 con 0x7f12ec003b80 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130815cf20 con 0x7f12ec003b80 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130804ef30 con 0x7f12ec003b80 2024-04-25T10:29:17.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12f8007480 con 0x7f12ec003b80 2024-04-25T10:29:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12ff400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12e4066f90 0x7f12e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f12f811dfb0 con 0x7f12ec003b80 2024-04-25T10:29:17.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12fde00640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12e4066f90 0x7f12e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:17.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.724+0000 7f12fde00640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12e4066f90 0x7f12e4069450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f13081589d0 tx=0x7f12f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:17.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.728+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12f80b4070 con 0x7f12ec003b80 2024-04-25T10:29:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:17.948+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f130815ef20 con 0x7f12ec003b80 2024-04-25T10:29:18.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f12ff400640 1 -- 172.21.15.45:0/2434575585 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v547) v1 ==== 159+0+0 (secure 0 0 0) 0x7f12f8016910 con 0x7f12ec003b80 2024-04-25T10:29:18.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12e4066f90 msgr2=0x7f12e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12e4066f90 0x7f12e4069450 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f13081589d0 tx=0x7f12f0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:18.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 msgr2=0x7f130815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f130815c260 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f12f80ab040 tx=0x7f12ec005840 comp rx=0 tx=0).stop 2024-04-25T10:29:18.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 shutdown_connections 2024-04-25T10:29:18.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f12e4066f90 0x7f12e4069450 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1308155f00 0x7f1308156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1308158c00 0x7f130814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 --2- 172.21.15.45:0/2434575585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12ec003b80 0x7f130815c260 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 >> 172.21.15.45:0/2434575585 conn(0x7f130809a9d0 msgr2=0x7f130815e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 shutdown_connections 2024-04-25T10:29:18.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.164+0000 7f130d400640 1 -- 172.21.15.45:0/2434575585 wait complete. 2024-04-25T10:29:18.188 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:29:18.188 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:29:18.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.348+0000 7ff48dc00640 1 Processor -- start 2024-04-25T10:29:18.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.348+0000 7ff48dc00640 1 -- start start 2024-04-25T10:29:18.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff48dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 0x7ff48004faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff48dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 0x7ff48004ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff487e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 0x7ff48004faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff487e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 0x7ff48004faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:49160/0 (socket says 172.21.15.45:49160) 2024-04-25T10:29:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff487400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 0x7ff48004ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff487400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 0x7ff48004ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33912/0 (socket says 172.21.15.45:33912) 2024-04-25T10:29:18.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff48d200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff4800e2d10 0x7ff4800e31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54942/0 (socket says 172.21.15.45:54942) 2024-04-25T10:29:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff487e00640 1 -- 172.21.15.45:0/971066223 learned_addr learned my addr 172.21.15.45:0/971066223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff48dc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff480054120 con 0x7ff4800e2d10 2024-04-25T10:29:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.352+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4800df690 con 0x7ff480033d00 2024-04-25T10:29:18.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff480054400 con 0x7ff4800df2b0 2024-04-25T10:29:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff486a00640 1 -- 172.21.15.45:0/971066223 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253595793 0 0) 0x7ff480054120 con 0x7ff4800e2d10 2024-04-25T10:29:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff486a00640 1 -- 172.21.15.45:0/971066223 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003450 con 0x7ff4800e2d10 2024-04-25T10:29:18.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff487e00640 1 -- 172.21.15.45:0/971066223 >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 msgr2=0x7ff48004ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff487e00640 1 --2- 172.21.15.45:0/971066223 >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 0x7ff48004ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff487e00640 1 -- 172.21.15.45:0/971066223 >> v1:172.21.15.45:6789/0 conn(0x7ff4800e2d10 legacy=0x7ff4800e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff487e00640 1 -- 172.21.15.45:0/971066223 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800546e0 con 0x7ff4800df2b0 2024-04-25T10:29:18.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.356+0000 7ff487400640 1 --2- 172.21.15.45:0/971066223 >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 0x7ff48004ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff487e00640 1 --2- 172.21.15.45:0/971066223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 0x7ff48004faa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005ad0 tx=0x7ff47c01cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f9ebd3c7ec62f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff486a00640 1 -- 172.21.15.45:0/971066223 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0b2070 con 0x7ff4800df2b0 2024-04-25T10:29:18.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff486a00640 1 -- 172.21.15.45:0/971066223 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c0a5900 con 0x7ff4800df2b0 2024-04-25T10:29:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff486a00640 1 -- 172.21.15.45:0/971066223 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48804f450 con 0x7ff4800df2b0 2024-04-25T10:29:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 msgr2=0x7ff48004faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 --2- 172.21.15.45:0/971066223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 0x7ff48004faa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff47c005ad0 tx=0x7ff47c01cc60 comp rx=0 tx=0).stop 2024-04-25T10:29:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 shutdown_connections 2024-04-25T10:29:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 --2- 172.21.15.45:0/971066223 >> v2:172.21.15.45:3300/0 conn(0x7ff480033d00 0x7ff48004ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 --2- 172.21.15.45:0/971066223 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800df2b0 0x7ff48004faa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 >> 172.21.15.45:0/971066223 conn(0x7ff480009fc0 msgr2=0x7ff48000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 shutdown_connections 2024-04-25T10:29:18.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.360+0000 7ff48dc00640 1 -- 172.21.15.45:0/971066223 wait complete. 2024-04-25T10:29:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 Processor -- start 2024-04-25T10:29:18.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 -- start start 2024-04-25T10:29:18.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff480033d00 0x7ff4800e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 0x7ff4800d7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800e2d10 0x7ff4800dd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48004f7e0 con 0x7ff4800df2b0 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48004f660 con 0x7ff480033d00 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800e38b0 con 0x7ff4800e2d10 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 0x7ff4800d7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 0x7ff4800d7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33926/0 (socket says 172.21.15.45:33926) 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 -- 172.21.15.45:0/1699557954 learned_addr learned my addr 172.21.15.45:0/1699557954 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487e00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff480033d00 0x7ff4800e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 -- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800e2d10 msgr2=0x7ff4800dd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48d200640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800e2d10 0x7ff4800dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800e2d10 0x7ff4800dd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 -- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff480033d00 msgr2=0x7ff4800e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff480033d00 0x7ff4800e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 -- 172.21.15.45:0/1699557954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800dfbc0 con 0x7ff4800df2b0 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487e00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff480033d00 0x7ff4800e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48d200640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800e2d10 0x7ff4800dd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff487400640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 0x7ff4800d7e90 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff47400ebe0 tx=0x7ff47400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474017070 con 0x7ff4800df2b0 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47400cc30 con 0x7ff4800df2b0 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4800e1800 con 0x7ff4800df2b0 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474013470 con 0x7ff4800df2b0 2024-04-25T10:29:18.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.364+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4800e1dc0 con 0x7ff4800df2b0 2024-04-25T10:29:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.368+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4800089e0 con 0x7ff4800df2b0 2024-04-25T10:29:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.368+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff47401a070 con 0x7ff4800df2b0 2024-04-25T10:29:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.368+0000 7ff47be00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff468066f60 0x7ff468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.368+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff474084cf0 con 0x7ff4800df2b0 2024-04-25T10:29:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.368+0000 7ff487e00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff468066f60 0x7ff468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.368+0000 7ff487e00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff468066f60 0x7ff468069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0af000 tx=0x7ff47c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.372+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff474055f60 con 0x7ff4800df2b0 2024-04-25T10:29:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.532+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4800053a0 con 0x7ff4800df2b0 2024-04-25T10:29:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff47be00640 1 -- 172.21.15.45:0/1699557954 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+13016 (secure 0 0 0) 0x7ff474055860 con 0x7ff4800df2b0 2024-04-25T10:29:18.536 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:18.536 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:17.158919+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/903752757":"2024-04-26T10:29:13.797402+0000","172.21.15.45:6834/903752757":"2024-04-26T10:29:13.797402+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff468066f60 msgr2=0x7ff468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff468066f60 0x7ff468069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0af000 tx=0x7ff47c0a7070 comp rx=0 tx=0).stop 2024-04-25T10:29:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 msgr2=0x7ff4800d7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 0x7ff4800d7e90 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff47400ebe0 tx=0x7ff47400c520 comp rx=0 tx=0).stop 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 shutdown_connections 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff468066f60 0x7ff468069420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4800e2d10 0x7ff4800dd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4800df2b0 0x7ff4800d7e90 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 --2- 172.21.15.45:0/1699557954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff480033d00 0x7ff4800e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 >> 172.21.15.45:0/1699557954 conn(0x7ff480009fc0 msgr2=0x7ff480035d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 shutdown_connections 2024-04-25T10:29:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.536+0000 7ff48dc00640 1 -- 172.21.15.45:0/1699557954 wait complete. 2024-04-25T10:29:18.557 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:18.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 Processor -- start 2024-04-25T10:29:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 -- start start 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35180aef80 0x7f351815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3518070350 0x7f351815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351815b080 con 0x7f351815bac0 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351815adb0 con 0x7f35180aef80 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f351e400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351815eb60 con 0x7f3518070350 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35180aef80 0x7f351815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35180aef80 0x7f351815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33940/0 (socket says 172.21.15.45:33940) 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 -- 172.21.15.45:0/2767603549 learned_addr learned my addr 172.21.15.45:0/2767603549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f350fe00640 1 --2- 172.21.15.45:0/2767603549 >> v2:172.21.15.117:3300/0 conn(0x7f3518070350 0x7f351815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 -- 172.21.15.45:0/2767603549 >> v2:172.21.15.117:3300/0 conn(0x7f3518070350 msgr2=0x7f351815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 --2- 172.21.15.45:0/2767603549 >> v2:172.21.15.117:3300/0 conn(0x7f3518070350 0x7f351815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 -- 172.21.15.45:0/2767603549 >> v1:172.21.15.117:6789/0 conn(0x7f351815bac0 legacy=0x7f35180b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f3517400640 1 -- 172.21.15.45:0/2767603549 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351815ee40 con 0x7f35180aef80 2024-04-25T10:29:18.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.700+0000 7f350fe00640 1 --2- 172.21.15.45:0/2767603549 >> v2:172.21.15.117:3300/0 conn(0x7f3518070350 0x7f351815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:18.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f3517400640 1 --2- 172.21.15.45:0/2767603549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180aef80 0x7f351815e480 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f351806e390 tx=0x7f350801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbe2a1ff4894a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f3516a00640 1 -- 172.21.15.45:0/2767603549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35080aa620 con 0x7f35180aef80 2024-04-25T10:29:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f3516a00640 1 -- 172.21.15.45:0/2767603549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35080af070 con 0x7f35180aef80 2024-04-25T10:29:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f3516a00640 1 -- 172.21.15.45:0/2767603549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35080aa9e0 con 0x7f35180aef80 2024-04-25T10:29:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 -- 172.21.15.45:0/2767603549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180aef80 msgr2=0x7f351815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 --2- 172.21.15.45:0/2767603549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180aef80 0x7f351815e480 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f351806e390 tx=0x7f350801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 -- 172.21.15.45:0/2767603549 shutdown_connections 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 --2- 172.21.15.45:0/2767603549 >> v2:172.21.15.117:3300/0 conn(0x7f3518070350 0x7f351815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 --2- 172.21.15.45:0/2767603549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35180aef80 0x7f351815e480 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 -- 172.21.15.45:0/2767603549 >> 172.21.15.45:0/2767603549 conn(0x7f35180669a0 msgr2=0x7f3518066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 -- 172.21.15.45:0/2767603549 shutdown_connections 2024-04-25T10:29:18.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.704+0000 7f351e400640 1 -- 172.21.15.45:0/2767603549 wait complete. 2024-04-25T10:29:18.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 Processor -- start 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 -- start start 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3518070350 0x7f3518158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35180aef80 0x7f3518161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 0x7f3518164c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35181679a0 con 0x7f351815bac0 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518167820 con 0x7f3518070350 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3518167b20 con 0x7f35180aef80 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 0x7f3518164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 0x7f3518164c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33950/0 (socket says 172.21.15.45:33950) 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 -- 172.21.15.45:0/2230384863 learned_addr learned my addr 172.21.15.45:0/2230384863 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3518070350 0x7f3518158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f350fe00640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35180aef80 0x7f3518161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 -- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35180aef80 msgr2=0x7f3518161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35180aef80 0x7f3518161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 -- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3518070350 msgr2=0x7f3518158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3518070350 0x7f3518158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 -- 172.21.15.45:0/2230384863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351815f900 con 0x7f351815bac0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f350fe00640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35180aef80 0x7f3518161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f3517e00640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 0x7f3518164c30 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f351000b970 tx=0x7f351000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510019070 con 0x7f351815bac0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000cb40 con 0x7f351815bac0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510014440 con 0x7f351815bac0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351815fbe0 con 0x7f351815bac0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.708+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35181601a0 con 0x7f351815bac0 2024-04-25T10:29:18.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.712+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3518160570 con 0x7f351815bac0 2024-04-25T10:29:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.712+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f351001c070 con 0x7f351815bac0 2024-04-25T10:29:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.712+0000 7f350f400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34f4066f60 0x7f34f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.712+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3510084920 con 0x7f351815bac0 2024-04-25T10:29:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.712+0000 7f3517400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34f4066f60 0x7f34f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:18.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.712+0000 7f3517400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34f4066f60 0x7f34f4069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f35080ae000 tx=0x7f35080a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:18.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.716+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3510055540 con 0x7f351815bac0 2024-04-25T10:29:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f351804ef30 con 0x7f351815bac0 2024-04-25T10:29:18.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f350f400640 1 -- 172.21.15.45:0/2230384863 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35100595e0 con 0x7f351815bac0 2024-04-25T10:29:18.881 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:29:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34f4066f60 msgr2=0x7f34f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34f4066f60 0x7f34f4069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f35080ae000 tx=0x7f35080a7000 comp rx=0 tx=0).stop 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 msgr2=0x7f3518164c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 0x7f3518164c30 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f351000b970 tx=0x7f351000be30 comp rx=0 tx=0).stop 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 shutdown_connections 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f34f4066f60 0x7f34f4069420 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f351815bac0 0x7f3518164c30 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35180aef80 0x7f3518161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 --2- 172.21.15.45:0/2230384863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3518070350 0x7f3518158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.880+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 >> 172.21.15.45:0/2230384863 conn(0x7f35180669a0 msgr2=0x7f351806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.884+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 shutdown_connections 2024-04-25T10:29:18.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:18.884+0000 7f351e400640 1 -- 172.21.15.45:0/2230384863 wait complete. 2024-04-25T10:29:18.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:29:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 Processor -- start 2024-04-25T10:29:19.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 -- start start 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe140cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe140cfe70 0x7fbe1415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe14071050 con 0x7fbe1415b740 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1415d040 con 0x7fbe140cfe70 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.044+0000 7fbe19400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140712f0 con 0x7fbe141550b0 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe13200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbe1415b740 0x7fbe1406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54952/0 (socket says 172.21.15.45:54952) 2024-04-25T10:29:19.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe13200640 1 -- 172.21.15.45:0/1612440752 learned_addr learned my addr 172.21.15.45:0/1612440752 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe09e00640 1 --2- 172.21.15.45:0/1612440752 >> v2:172.21.15.117:3300/0 conn(0x7fbe140cfe70 0x7fbe1415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe12800640 1 --2- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe140cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe11e00640 1 -- 172.21.15.45:0/1612440752 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764575704 0 0) 0x7fbe14071050 con 0x7fbe1415b740 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe11e00640 1 -- 172.21.15.45:0/1612440752 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf8003530 con 0x7fbe1415b740 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe11e00640 1 -- 172.21.15.45:0/1612440752 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440927150 0 0) 0x7fbdf8003530 con 0x7fbe1415b740 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe09e00640 1 -- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 msgr2=0x7fbe140cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe09e00640 1 --2- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe140cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe09e00640 1 -- 172.21.15.45:0/1612440752 >> v1:172.21.15.45:6789/0 conn(0x7fbe1415b740 legacy=0x7fbe1406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe09e00640 1 -- 172.21.15.45:0/1612440752 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe140715d0 con 0x7fbe140cfe70 2024-04-25T10:29:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.048+0000 7fbe09e00640 1 --2- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe1415b200 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0097c0 tx=0x7fbdfc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c0c3cb43ec2fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe11e00640 1 -- 172.21.15.45:0/1612440752 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0b0070 con 0x7fbe140cfe70 2024-04-25T10:29:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe11e00640 1 -- 172.21.15.45:0/1612440752 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc0ab400 con 0x7fbe140cfe70 2024-04-25T10:29:19.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe11e00640 1 -- 172.21.15.45:0/1612440752 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c001680 con 0x7fbe140cfe70 2024-04-25T10:29:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 msgr2=0x7fbe1415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 --2- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe1415b200 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0097c0 tx=0x7fbdfc01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- 172.21.15.45:0/1612440752 shutdown_connections 2024-04-25T10:29:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 --2- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe1415b200 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 --2- 172.21.15.45:0/1612440752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe140cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- 172.21.15.45:0/1612440752 >> 172.21.15.45:0/1612440752 conn(0x7fbe14066b70 msgr2=0x7fbe14066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- 172.21.15.45:0/1612440752 shutdown_connections 2024-04-25T10:29:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- 172.21.15.45:0/1612440752 wait complete. 2024-04-25T10:29:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 Processor -- start 2024-04-25T10:29:19.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- start start 2024-04-25T10:29:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe140af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe14156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 0x7fbe14157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1415bd50 con 0x7fbe1415b740 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1415bbd0 con 0x7fbe140cfe70 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe19400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1415c030 con 0x7fbe141550b0 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe13200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 0x7fbe14157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe13200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 0x7fbe14157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33960/0 (socket says 172.21.15.45:33960) 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe13200640 1 -- 172.21.15.45:0/1291242921 learned_addr learned my addr 172.21.15.45:0/1291242921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe12800640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe140af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.052+0000 7fbe09e00640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe14156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe13200640 1 -- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 msgr2=0x7fbe14156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe13200640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe14156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe13200640 1 -- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 msgr2=0x7fbe140af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe13200640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe140af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:19.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe13200640 1 -- 172.21.15.45:0/1291242921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14157d70 con 0x7fbe1415b740 2024-04-25T10:29:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe09e00640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe14156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:19.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe13200640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 0x7fbe14157510 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c5d0 tx=0x7fbe0c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c005cf0 con 0x7fbe1415b740 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c007070 con 0x7fbe1415b740 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c007460 con 0x7fbe1415b740 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe14158050 con 0x7fbe1415b740 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe14158e10 con 0x7fbe1415b740 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe14158220 con 0x7fbe1415b740 2024-04-25T10:29:19.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe0c007600 con 0x7fbe1415b740 2024-04-25T10:29:19.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.056+0000 7fbe0b400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdf0066f60 0x7fbdf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:19.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.060+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbe0c084b10 con 0x7fbe1415b740 2024-04-25T10:29:19.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.060+0000 7fbe12800640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdf0066f60 0x7fbdf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:19.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.060+0000 7fbe12800640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdf0066f60 0x7fbdf0069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbe040113e0 tx=0x7fbe0400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:19.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.064+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe0c055e10 con 0x7fbe1415b740 2024-04-25T10:29:19.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:19.284+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7fbe14157ab0 con 0x7fbe1415b740 2024-04-25T10:29:20.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe0b400640 1 -- 172.21.15.45:0/1291242921 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v549) v1 ==== 143+0+0 (secure 0 0 0) 0x7fbe0c055710 con 0x7fbe1415b740 2024-04-25T10:29:20.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdf0066f60 msgr2=0x7fbdf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdf0066f60 0x7fbdf0069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbe040113e0 tx=0x7fbe0400b040 comp rx=0 tx=0).stop 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 msgr2=0x7fbe14157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 0x7fbe14157510 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00c5d0 tx=0x7fbe0c00ca90 comp rx=0 tx=0).stop 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 shutdown_connections 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbdf0066f60 0x7fbdf0069420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1415b740 0x7fbe14157510 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe141550b0 0x7fbe14156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 --2- 172.21.15.45:0/1291242921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe140cfe70 0x7fbe140af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 >> 172.21.15.45:0/1291242921 conn(0x7fbe14066b70 msgr2=0x7fbe1406c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 shutdown_connections 2024-04-25T10:29:20.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.172+0000 7fbe19400640 1 -- 172.21.15.45:0/1291242921 wait complete. 2024-04-25T10:29:20.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:29:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.344+0000 7f29c9600640 1 Processor -- start 2024-04-25T10:29:20.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.344+0000 7f29c9600640 1 -- start start 2024-04-25T10:29:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c9600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29c40722d0 0x7f29c40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c9600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40ad840 con 0x7f29c415e130 2024-04-25T10:29:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c9600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c415e710 con 0x7f29c40b3480 2024-04-25T10:29:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c9600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40d0090 con 0x7f29c40722d0 2024-04-25T10:29:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29ba000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f29c415e130 0x7f29c4071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54958/0 (socket says 172.21.15.45:54958) 2024-04-25T10:29:20.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29ba000640 1 -- 172.21.15.45:0/3901470676 learned_addr learned my addr 172.21.15.45:0/3901470676 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c3400640 1 --2- 172.21.15.45:0/3901470676 >> v2:172.21.15.117:3300/0 conn(0x7f29c40722d0 0x7f29c40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c2000640 1 -- 172.21.15.45:0/3901470676 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599072600 0 0) 0x7f29c40ad840 con 0x7f29c415e130 2024-04-25T10:29:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c2000640 1 -- 172.21.15.45:0/3901470676 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a8003530 con 0x7f29c415e130 2024-04-25T10:29:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c3400640 1 -- 172.21.15.45:0/3901470676 >> v1:172.21.15.117:6789/0 conn(0x7f29c40b3480 legacy=0x7f29c4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c3400640 1 -- 172.21.15.45:0/3901470676 >> v1:172.21.15.45:6789/0 conn(0x7f29c415e130 legacy=0x7f29c4071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.348+0000 7f29c3400640 1 -- 172.21.15.45:0/3901470676 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c415e9f0 con 0x7f29c40722d0 2024-04-25T10:29:20.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c3400640 1 --2- 172.21.15.45:0/3901470676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c40cfb50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0097c0 tx=0x7f29bc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfcd1ba2e671380c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c2000640 1 -- 172.21.15.45:0/3901470676 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0b0070 con 0x7f29c40722d0 2024-04-25T10:29:20.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c2000640 1 -- 172.21.15.45:0/3901470676 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc0ab600 con 0x7f29c40722d0 2024-04-25T10:29:20.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c2000640 1 -- 172.21.15.45:0/3901470676 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0ab9d0 con 0x7f29c40722d0 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 -- 172.21.15.45:0/3901470676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 msgr2=0x7f29c40cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 --2- 172.21.15.45:0/3901470676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c40cfb50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f29bc0097c0 tx=0x7f29bc01c410 comp rx=0 tx=0).stop 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 -- 172.21.15.45:0/3901470676 shutdown_connections 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 --2- 172.21.15.45:0/3901470676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c40cfb50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 -- 172.21.15.45:0/3901470676 >> 172.21.15.45:0/3901470676 conn(0x7f29c40669a0 msgr2=0x7f29c4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 -- 172.21.15.45:0/3901470676 shutdown_connections 2024-04-25T10:29:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 -- 172.21.15.45:0/3901470676 wait complete. 2024-04-25T10:29:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 Processor -- start 2024-04-25T10:29:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.352+0000 7f29c9600640 1 -- start start 2024-04-25T10:29:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c41614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 0x7f29c4149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c415e130 0x7f29c414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c415bb90 con 0x7f29c40b3480 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40ae7c0 con 0x7f29c40722d0 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c416b8a0 con 0x7f29c415e130 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 0x7f29c4149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 0x7f29c4149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33970/0 (socket says 172.21.15.45:33970) 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 -- 172.21.15.45:0/4092244365 learned_addr learned my addr 172.21.15.45:0/4092244365 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c2a00640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c41614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c3400640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c415e130 0x7f29c414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 -- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c415e130 msgr2=0x7f29c414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c415e130 0x7f29c414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 -- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 msgr2=0x7f29c41614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c41614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 -- 172.21.15.45:0/4092244365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4161a10 con 0x7f29c40b3480 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c2a00640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c41614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29ba000640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 0x7f29c4149d30 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c450 tx=0x7f29ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac017070 con 0x7f29c40b3480 2024-04-25T10:29:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac005cf0 con 0x7f29c40b3480 2024-04-25T10:29:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac007270 con 0x7f29c40b3480 2024-04-25T10:29:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c414e150 con 0x7f29c40b3480 2024-04-25T10:29:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c414e710 con 0x7f29c40b3480 2024-04-25T10:29:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c4159ef0 con 0x7f29c40b3480 2024-04-25T10:29:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.356+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29ac01a070 con 0x7f29c40b3480 2024-04-25T10:29:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.360+0000 7f29bb400640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29a0066f60 0x7f29a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.360+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29ac083e60 con 0x7f29c40b3480 2024-04-25T10:29:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.360+0000 7f29c2a00640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29a0066f60 0x7f29a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.360+0000 7f29c2a00640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29a0066f60 0x7f29a0069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f29c40b1000 tx=0x7f29b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.364+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29c4161a10 con 0x7f29c40b3480 2024-04-25T10:29:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.524+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f29c404ef30 con 0x7f29c40b3480 2024-04-25T10:29:20.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.528+0000 7f29bb400640 1 -- 172.21.15.45:0/4092244365 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f29ac054b90 con 0x7f29c40b3480 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.528+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29a0066f60 msgr2=0x7f29a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29a0066f60 0x7f29a0069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f29c40b1000 tx=0x7f29b4009040 comp rx=0 tx=0).stop 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 msgr2=0x7f29c4149d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 0x7f29c4149d30 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c450 tx=0x7f29ac00c910 comp rx=0 tx=0).stop 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 shutdown_connections 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29a0066f60 0x7f29a0069420 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c415e130 0x7f29c414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29c40b3480 0x7f29c4149d30 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 --2- 172.21.15.45:0/4092244365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c40722d0 0x7f29c41614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 >> 172.21.15.45:0/4092244365 conn(0x7f29c40669a0 msgr2=0x7f29c41539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 shutdown_connections 2024-04-25T10:29:20.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.532+0000 7f29c9600640 1 -- 172.21.15.45:0/4092244365 wait complete. 2024-04-25T10:29:20.551 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:29:20.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:29:20.654 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:29:20.714 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:29:20.714 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:29:20.714 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:20.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:29:20.723 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:29:20.723 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:20.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:29:20.732 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:29:20.732 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:20.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:29:20.775 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:29:20.775 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:20.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:29:20.784 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:29:20.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56c0e00640 1 Processor -- start 2024-04-25T10:29:20.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56c0e00640 1 -- start start 2024-04-25T10:29:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56c0e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56c0e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc15bff0 con 0x7f56bc0d0270 2024-04-25T10:29:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56c0e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56bc0cd550 con 0x7f56bc15bc10 2024-04-25T10:29:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56c0e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b0da0 con 0x7f56bc0cf760 2024-04-25T10:29:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56bbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f56bc0d0270 0x7f56bc0b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54970/0 (socket says 172.21.15.45:54970) 2024-04-25T10:29:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56bbe00640 1 -- 172.21.15.45:0/2717950465 learned_addr learned my addr 172.21.15.45:0/2717950465 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56baa00640 1 --2- 172.21.15.45:0/2717950465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516515640 0 0) 0x7f56bc15bff0 con 0x7f56bc0d0270 2024-04-25T10:29:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003530 con 0x7f56bc0d0270 2024-04-25T10:29:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399506430 0 0) 0x7f56bc0cd550 con 0x7f56bc15bc10 2024-04-25T10:29:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc15bff0 con 0x7f56bc15bc10 2024-04-25T10:29:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56baa00640 1 -- 172.21.15.45:0/2717950465 >> v1:172.21.15.117:6789/0 conn(0x7f56bc15bc10 legacy=0x7f56bc0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56baa00640 1 -- 172.21.15.45:0/2717950465 >> v1:172.21.15.45:6789/0 conn(0x7f56bc0d0270 legacy=0x7f56bc0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.960+0000 7f56baa00640 1 -- 172.21.15.45:0/2717950465 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0b1080 con 0x7f56bc0cf760 2024-04-25T10:29:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56baa00640 1 --2- 172.21.15.45:0/2717950465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cfd30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56a80097f0 tx=0x7f56a801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce40a3adb2747cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80ab5f0 con 0x7f56bc0cf760 2024-04-25T10:29:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80b0070 con 0x7f56bc0cf760 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56ba000640 1 -- 172.21.15.45:0/2717950465 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80ab9b0 con 0x7f56bc0cf760 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 -- 172.21.15.45:0/2717950465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 msgr2=0x7f56bc0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 --2- 172.21.15.45:0/2717950465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cfd30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56a80097f0 tx=0x7f56a801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 -- 172.21.15.45:0/2717950465 shutdown_connections 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 --2- 172.21.15.45:0/2717950465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cfd30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 -- 172.21.15.45:0/2717950465 >> 172.21.15.45:0/2717950465 conn(0x7f56bc0d1750 msgr2=0x7f56bc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 -- 172.21.15.45:0/2717950465 shutdown_connections 2024-04-25T10:29:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.964+0000 7f56c0e00640 1 -- 172.21.15.45:0/2717950465 wait complete. 2024-04-25T10:29:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 Processor -- start 2024-04-25T10:29:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- start start 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0d0270 0x7f56bc15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 0x7f56bc140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b2b30 con 0x7f56bc15bc10 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0b29b0 con 0x7f56bc0d0270 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc0d0680 con 0x7f56bc0cf760 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 0x7f56bc140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 0x7f56bc140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33972/0 (socket says 172.21.15.45:33972) 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 -- 172.21.15.45:0/1132213780 learned_addr learned my addr 172.21.15.45:0/1132213780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56baa00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0d0270 0x7f56bc15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bb400640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 -- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 msgr2=0x7f56bc0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 -- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0d0270 msgr2=0x7f56bc15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0d0270 0x7f56bc15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 -- 172.21.15.45:0/1132213780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc1412f0 con 0x7f56bc15bc10 2024-04-25T10:29:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56bbe00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 0x7f56bc140740 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f56b400caf0 tx=0x7f56b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4017070 con 0x7f56bc15bc10 2024-04-25T10:29:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b400aca0 con 0x7f56bc15bc10 2024-04-25T10:29:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4012600 con 0x7f56bc15bc10 2024-04-25T10:29:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc0c9c10 con 0x7f56bc15bc10 2024-04-25T10:29:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc0ca1d0 con 0x7f56bc15bc10 2024-04-25T10:29:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.968+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc075120 con 0x7f56bc15bc10 2024-04-25T10:29:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.972+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56b401a070 con 0x7f56bc15bc10 2024-04-25T10:29:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.972+0000 7f56b3400640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f569c066f60 0x7f569c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.972+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f56b4084980 con 0x7f56bc15bc10 2024-04-25T10:29:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.972+0000 7f56bb400640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f569c066f60 0x7f569c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.972+0000 7f56bb400640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f569c066f60 0x7f569c069420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f56ac007fd0 tx=0x7f56ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:20.976+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f56b4055630 con 0x7f56bc15bc10 2024-04-25T10:29:21.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.172+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56bc0cb1f0 con 0x7f56bc15bc10 2024-04-25T10:29:21.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56b3400640 1 -- 172.21.15.45:0/1132213780 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+3699 (secure 0 0 0) 0x7f56b40596d0 con 0x7f56bc15bc10 2024-04-25T10:29:21.175 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:21.176 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":109,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1548325426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1548325426},{"type":"v1","addr":"172.21.15.45:6833","nonce":1548325426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/771029508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":771029508},{"type":"v1","addr":"172.21.15.117:6837","nonce":771029508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8078,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/262699796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":262699796},{"type":"v1","addr":"172.21.15.117:6836","nonce":262699796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/746557039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":746557039},{"type":"v1","addr":"172.21.15.45:6835","nonce":746557039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[]} 2024-04-25T10:29:21.176 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 109 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f569c066f60 msgr2=0x7f569c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f569c066f60 0x7f569c069420 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f56ac007fd0 tx=0x7f56ac009040 comp rx=0 tx=0).stop 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 msgr2=0x7f56bc140740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 0x7f56bc140740 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f56b400caf0 tx=0x7f56b400a060 comp rx=0 tx=0).stop 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 shutdown_connections 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f569c066f60 0x7f569c069420 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56bc15bc10 0x7f56bc140740 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56bc0d0270 0x7f56bc15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 --2- 172.21.15.45:0/1132213780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56bc0cf760 0x7f56bc0cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 >> 172.21.15.45:0/1132213780 conn(0x7f56bc0d1750 msgr2=0x7f56bc0afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 shutdown_connections 2024-04-25T10:29:21.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.176+0000 7f56c0e00640 1 -- 172.21.15.45:0/1132213780 wait complete. 2024-04-25T10:29:21.196 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:29:21.197 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:29:21.197 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:29:21.197 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:21.197 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:29:21.197 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:29:21.197 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:29:21.198 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:21.198 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:29:21.198 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:29:21.198 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:29:21.198 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:21.198 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:29:21.198 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:29:21.199 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:29:21.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:29:21.209 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:29:21.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:29:21.210 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:29:21.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:29:21.213 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:29:21.213 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:29:21.214 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:29:21.215 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:29:21.254 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:29:21.254 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.255 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:29:21.256 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.257 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:29:21.257 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.259 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:29:21.259 DEBUG:teuthology.parallel:result is None 2024-04-25T10:29:21.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:29:21.342 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:29:21.343 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:29:21.347 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:29:21.347 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:29:21.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.404+0000 7f20d0400640 1 Processor -- start 2024-04-25T10:29:21.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.404+0000 7f20d0400640 1 -- start start 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20d0400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20c815b9e0 0x7f20c80cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20d0400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80ccfb0 con 0x7f20c80d1c30 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20d0400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c80cd290 con 0x7f20c815bdc0 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20d0400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c80cd570 con 0x7f20c815b9e0 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20ce400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f20c80d1c30 0x7f20c80c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55006/0 (socket says 172.21.15.45:55006) 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20ce400640 1 -- 172.21.15.45:0/2343442981 learned_addr learned my addr 172.21.15.45:0/2343442981 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20cda00640 1 --2- 172.21.15.45:0/2343442981 >> v2:172.21.15.117:3300/0 conn(0x7f20c815b9e0 0x7f20c80cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20c7e00640 1 -- 172.21.15.45:0/2343442981 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922524495 0 0) 0x7f20c80ccfb0 con 0x7f20c80d1c30 2024-04-25T10:29:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20c7e00640 1 -- 172.21.15.45:0/2343442981 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20b0003530 con 0x7f20c80d1c30 2024-04-25T10:29:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20cda00640 1 -- 172.21.15.45:0/2343442981 >> v1:172.21.15.117:6789/0 conn(0x7f20c815bdc0 legacy=0x7f20c80d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20cda00640 1 -- 172.21.15.45:0/2343442981 >> v1:172.21.15.45:6789/0 conn(0x7f20c80d1c30 legacy=0x7f20c80c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.408+0000 7f20cda00640 1 -- 172.21.15.45:0/2343442981 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80cd850 con 0x7f20c815b9e0 2024-04-25T10:29:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20cda00640 1 --2- 172.21.15.45:0/2343442981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80cc410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0097c0 tx=0x7f20bc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cea77e5d723c6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20c7e00640 1 -- 172.21.15.45:0/2343442981 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0b0070 con 0x7f20c815b9e0 2024-04-25T10:29:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20c7e00640 1 -- 172.21.15.45:0/2343442981 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc0ab600 con 0x7f20c815b9e0 2024-04-25T10:29:21.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20c7e00640 1 -- 172.21.15.45:0/2343442981 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0ab9d0 con 0x7f20c815b9e0 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 -- 172.21.15.45:0/2343442981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 msgr2=0x7f20c80cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 --2- 172.21.15.45:0/2343442981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80cc410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0097c0 tx=0x7f20bc01c410 comp rx=0 tx=0).stop 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 -- 172.21.15.45:0/2343442981 shutdown_connections 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 --2- 172.21.15.45:0/2343442981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80cc410 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 -- 172.21.15.45:0/2343442981 >> 172.21.15.45:0/2343442981 conn(0x7f20c809a9d0 msgr2=0x7f20c809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 -- 172.21.15.45:0/2343442981 shutdown_connections 2024-04-25T10:29:21.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 -- 172.21.15.45:0/2343442981 wait complete. 2024-04-25T10:29:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 Processor -- start 2024-04-25T10:29:21.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.412+0000 7f20d0400640 1 -- start start 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 0x7f20c80cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c815bdc0 0x7f20c80b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80c6d40 con 0x7f20c80d1c30 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80c6bc0 con 0x7f20c815b9e0 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8169840 con 0x7f20c815bdc0 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 0x7f20c80cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 0x7f20c80cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34008/0 (socket says 172.21.15.45:34008) 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 -- 172.21.15.45:0/891113111 learned_addr learned my addr 172.21.15.45:0/891113111 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20ce400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c815bdc0 0x7f20c80b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cd000640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 -- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c815bdc0 msgr2=0x7f20c80b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c815bdc0 0x7f20c80b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 -- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 msgr2=0x7f20c80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 -- 172.21.15.45:0/891113111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80b6940 con 0x7f20c80d1c30 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cd000640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20ce400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c815bdc0 0x7f20c80b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:21.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20cda00640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 0x7f20c80cb090 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f20bc019020 tx=0x7f20bc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0b8020 con 0x7f20c80d1c30 2024-04-25T10:29:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc0b4410 con 0x7f20c80d1c30 2024-04-25T10:29:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0b0040 con 0x7f20c80d1c30 2024-04-25T10:29:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c80c97f0 con 0x7f20c80d1c30 2024-04-25T10:29:21.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c80c9db0 con 0x7f20c80d1c30 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c804ef30 con 0x7f20c80d1c30 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.416+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20bc0bb020 con 0x7f20c80d1c30 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.420+0000 7f20c5600640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20a8066f90 0x7f20a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.420+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f20bc11e7e0 con 0x7f20c80d1c30 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.420+0000 7f20cd000640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20a8066f90 0x7f20a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:21.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.420+0000 7f20cd000640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20a8066f90 0x7f20a8069450 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f20c80c7960 tx=0x7f20b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:21.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.424+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20bc0c2030 con 0x7f20c80d1c30 2024-04-25T10:29:21.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:21.600+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f20c80c9ff0 con 0x7f20c80d1c30 2024-04-25T10:29:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.548+0000 7f20c5600640 1 -- 172.21.15.45:0/891113111 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v151) v1 ==== 97+0+0 (secure 0 0 0) 0x7f20bc016090 con 0x7f20c80d1c30 2024-04-25T10:29:22.551 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20a8066f90 msgr2=0x7f20a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20a8066f90 0x7f20a8069450 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f20c80c7960 tx=0x7f20b4009040 comp rx=0 tx=0).stop 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 msgr2=0x7f20c80cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 0x7f20c80cb090 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f20bc019020 tx=0x7f20bc0191b0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 shutdown_connections 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20a8066f90 0x7f20a8069450 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c815bdc0 0x7f20c80b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c815b9e0 0x7f20c80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 --2- 172.21.15.45:0/891113111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20c80d1c30 0x7f20c80cb090 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 >> 172.21.15.45:0/891113111 conn(0x7f20c809a9d0 msgr2=0x7f20c8154a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 shutdown_connections 2024-04-25T10:29:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.552+0000 7f20d0400640 1 -- 172.21.15.45:0/891113111 wait complete. 2024-04-25T10:29:22.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:29:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.720+0000 7f600ee00640 1 Processor -- start 2024-04-25T10:29:22.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.720+0000 7f600ee00640 1 -- start start 2024-04-25T10:29:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60081550b0 0x7f60080cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f600815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ee00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6008071050 con 0x7f600815b740 2024-04-25T10:29:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600815d040 con 0x7f60081550b0 2024-04-25T10:29:22.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60080712f0 con 0x7f60080cfe70 2024-04-25T10:29:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60081550b0 0x7f60080cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ce00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f600815b740 0x7f600806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:54236/0 (socket says 172.21.15.45:54236) 2024-04-25T10:29:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60081550b0 0x7f60080cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45060/0 (socket says 172.21.15.45:45060) 2024-04-25T10:29:22.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f600ce00640 1 -- 172.21.15.45:0/2195909399 learned_addr learned my addr 172.21.15.45:0/2195909399 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007400640 1 --2- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f600815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007e00640 1 -- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 msgr2=0x7f600815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007e00640 1 --2- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f600815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007e00640 1 -- 172.21.15.45:0/2195909399 >> v1:172.21.15.117:6789/0 conn(0x7f600815b740 legacy=0x7f600806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.724+0000 7f6007e00640 1 -- 172.21.15.45:0/2195909399 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080715d0 con 0x7f60081550b0 2024-04-25T10:29:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f6007e00640 1 --2- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f60080cf930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80097c0 tx=0x7f5ff801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1192bb7b2a5a763b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f6006a00640 1 -- 172.21.15.45:0/2195909399 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80b0070 con 0x7f60081550b0 2024-04-25T10:29:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f6006a00640 1 -- 172.21.15.45:0/2195909399 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff80ab600 con 0x7f60081550b0 2024-04-25T10:29:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f6006a00640 1 -- 172.21.15.45:0/2195909399 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80ab9d0 con 0x7f60081550b0 2024-04-25T10:29:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 -- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 msgr2=0x7f60080cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 --2- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f60080cf930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80097c0 tx=0x7f5ff801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 -- 172.21.15.45:0/2195909399 shutdown_connections 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 --2- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f600815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 --2- 172.21.15.45:0/2195909399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f60080cf930 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 -- 172.21.15.45:0/2195909399 >> 172.21.15.45:0/2195909399 conn(0x7f6008066b70 msgr2=0x7f6008066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 -- 172.21.15.45:0/2195909399 shutdown_connections 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 -- 172.21.15.45:0/2195909399 wait complete. 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 Processor -- start 2024-04-25T10:29:22.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.728+0000 7f600ee00640 1 -- start start 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f6008157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f6008158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 0x7f6008155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600815bcd0 con 0x7f600815b740 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600815bb50 con 0x7f60081550b0 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600815c030 con 0x7f60080cfe70 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 0x7f6008155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 0x7f6008155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34024/0 (socket says 172.21.15.45:34024) 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 -- 172.21.15.45:0/878030571 learned_addr learned my addr 172.21.15.45:0/878030571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:22.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f6007e00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f6008157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f6007400640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f6008158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 -- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 msgr2=0x7f6008157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f6008157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 -- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 msgr2=0x7f6008158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f6008158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 -- 172.21.15.45:0/878030571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008159450 con 0x7f600815b740 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ce00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 0x7f6008155960 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f600000ca80 tx=0x7f600000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000017070 con 0x7f600815b740 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600000ac30 con 0x7f600815b740 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008159730 con 0x7f600815b740 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000012600 con 0x7f600815b740 2024-04-25T10:29:22.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008159ce0 con 0x7f600815b740 2024-04-25T10:29:22.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600814b920 con 0x7f600815b740 2024-04-25T10:29:22.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.732+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f600001a070 con 0x7f600815b740 2024-04-25T10:29:22.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.736+0000 7f5fffe00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe8066f60 0x7f5fe8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:22.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.736+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f60000849c0 con 0x7f600815b740 2024-04-25T10:29:22.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.736+0000 7f6007e00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe8066f60 0x7f5fe8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:22.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.736+0000 7f6007e00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe8066f60 0x7f5fe8069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80ad000 tx=0x7f5ff80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:22.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.740+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f600000ae20 con 0x7f600815b740 2024-04-25T10:29:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f600804ef30 con 0x7f600815b740 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f5fffe00640 1 -- 172.21.15.45:0/878030571 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6000055640 con 0x7f600815b740 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:29:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:29:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:29:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:29:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:29:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:29:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:29:22.973 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe8066f60 msgr2=0x7f5fe8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe8066f60 0x7f5fe8069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80ad000 tx=0x7f5ff80a6000 comp rx=0 tx=0).stop 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 msgr2=0x7f6008155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 0x7f6008155960 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f600000ca80 tx=0x7f600000cf40 comp rx=0 tx=0).stop 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 shutdown_connections 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe8066f60 0x7f5fe8069420 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600815b740 0x7f6008155960 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60081550b0 0x7f6008158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 --2- 172.21.15.45:0/878030571 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60080cfe70 0x7f6008157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 >> 172.21.15.45:0/878030571 conn(0x7f6008066b70 msgr2=0x7f600815c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 shutdown_connections 2024-04-25T10:29:22.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:22.960+0000 7f600ee00640 1 -- 172.21.15.45:0/878030571 wait complete. 2024-04-25T10:29:22.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.120+0000 7f0628000640 1 Processor -- start 2024-04-25T10:29:23.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.120+0000 7f0628000640 1 -- start start 2024-04-25T10:29:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0628000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0620070c20 0x7f062015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0628000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06200ae020 0x7f06200ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0628000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06200ad840 con 0x7f06200cf930 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0628000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062015bb90 con 0x7f06200ae020 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0628000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062015e9f0 con 0x7f0620070c20 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0626000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06200ae020 0x7f06200ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0625600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06200cf930 0x7f06200cfd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55012/0 (socket says 172.21.15.45:55012) 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0625600640 1 -- 172.21.15.45:0/123963620 learned_addr learned my addr 172.21.15.45:0/123963620 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0624c00640 1 --2- 172.21.15.45:0/123963620 >> v2:172.21.15.117:3300/0 conn(0x7f0620070c20 0x7f062015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0626000640 1 -- 172.21.15.45:0/123963620 >> v2:172.21.15.117:3300/0 conn(0x7f0620070c20 msgr2=0x7f062015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0626000640 1 --2- 172.21.15.45:0/123963620 >> v2:172.21.15.117:3300/0 conn(0x7f0620070c20 0x7f062015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0626000640 1 -- 172.21.15.45:0/123963620 >> v1:172.21.15.45:6789/0 conn(0x7f06200cf930 legacy=0x7f06200cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f0626000640 1 -- 172.21.15.45:0/123963620 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062015ecd0 con 0x7f06200ae020 2024-04-25T10:29:23.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.124+0000 7f061fe00640 1 -- 172.21.15.45:0/123963620 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155306596 0 0) 0x7f06200ad840 con 0x7f06200cf930 2024-04-25T10:29:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0626000640 1 --2- 172.21.15.45:0/123963620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f06200ae4b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f06200d0630 tx=0x7f061801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21546c2655bf5a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f061fe00640 1 -- 172.21.15.45:0/123963620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180b2070 con 0x7f06200ae020 2024-04-25T10:29:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f061fe00640 1 -- 172.21.15.45:0/123963620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06180a59c0 con 0x7f06200ae020 2024-04-25T10:29:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f061fe00640 1 -- 172.21.15.45:0/123963620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180a5d90 con 0x7f06200ae020 2024-04-25T10:29:23.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- 172.21.15.45:0/123963620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 msgr2=0x7f06200ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 --2- 172.21.15.45:0/123963620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f06200ae4b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f06200d0630 tx=0x7f061801c900 comp rx=0 tx=0).stop 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- 172.21.15.45:0/123963620 shutdown_connections 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 --2- 172.21.15.45:0/123963620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f06200ae4b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 --2- 172.21.15.45:0/123963620 >> v2:172.21.15.117:3300/0 conn(0x7f0620070c20 0x7f062015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- 172.21.15.45:0/123963620 >> 172.21.15.45:0/123963620 conn(0x7f0620066b70 msgr2=0x7f0620066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- 172.21.15.45:0/123963620 shutdown_connections 2024-04-25T10:29:23.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- 172.21.15.45:0/123963620 wait complete. 2024-04-25T10:29:23.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 Processor -- start 2024-04-25T10:29:23.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- start start 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0620070c20 0x7f06200cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f0620155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06200cf930 0x7f06200cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062016a8e0 con 0x7f06200ae020 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062015e6b0 con 0x7f06200cf930 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0628000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062016aa60 con 0x7f0620070c20 2024-04-25T10:29:23.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.128+0000 7f0624c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f0620155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f0620155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34046/0 (socket says 172.21.15.45:34046) 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 -- 172.21.15.45:0/1791526847 learned_addr learned my addr 172.21.15.45:0/1791526847 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0626000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06200cf930 0x7f06200cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 -- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0620070c20 msgr2=0x7f06200cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0620070c20 0x7f06200cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 -- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06200cf930 msgr2=0x7f06200cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06200cf930 0x7f06200cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 -- 172.21.15.45:0/1791526847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06200ce320 con 0x7f06200ae020 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0624c00640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f0620155730 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f060c00e870 tx=0x7f060c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c017070 con 0x7f06200ae020 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c00c940 con 0x7f06200ae020 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06200ce600 con 0x7f06200ae020 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c00ccf0 con 0x7f06200ae020 2024-04-25T10:29:23.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06200cebc0 con 0x7f06200ae020 2024-04-25T10:29:23.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062004ef30 con 0x7f06200ae020 2024-04-25T10:29:23.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.132+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f060c01a070 con 0x7f06200ae020 2024-04-25T10:29:23.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.136+0000 7f061d600640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0600066fb0 0x7f0600069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.136+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f060c084ee0 con 0x7f06200ae020 2024-04-25T10:29:23.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.136+0000 7f0625600640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0600066fb0 0x7f0600069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.136+0000 7f0625600640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0600066fb0 0x7f0600069470 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f06140011f0 tx=0x7f0614007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.140+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f060c0562b0 con 0x7f06200ae020 2024-04-25T10:29:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.300+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0620160de0 con 0x7f06200ae020 2024-04-25T10:29:23.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.300+0000 7f061d600640 1 -- 172.21.15.45:0/1791526847 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f060c055bb0 con 0x7f06200ae020 2024-04-25T10:29:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:23.303 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":151,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:22.550216+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0600066fb0 msgr2=0x7f0600069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0600066fb0 0x7f0600069470 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f06140011f0 tx=0x7f0614007930 comp rx=0 tx=0).stop 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 msgr2=0x7f0620155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f0620155730 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f060c00e870 tx=0x7f060c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 shutdown_connections 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0600066fb0 0x7f0600069470 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06200cf930 0x7f06200cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06200ae020 0x7f0620155730 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 --2- 172.21.15.45:0/1791526847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0620070c20 0x7f06200cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 >> 172.21.15.45:0/1791526847 conn(0x7f0620066b70 msgr2=0x7f062015f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 shutdown_connections 2024-04-25T10:29:23.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.304+0000 7f0628000640 1 -- 172.21.15.45:0/1791526847 wait complete. 2024-04-25T10:29:23.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:29:23.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:29:23.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.468+0000 7f2d86e00640 1 Processor -- start 2024-04-25T10:29:23.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.468+0000 7f2d86e00640 1 -- start start 2024-04-25T10:29:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.468+0000 7f2d86e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2d800b38d0 0x7f2d8015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.468+0000 7f2d86e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8015b080 con 0x7f2d80070350 2024-04-25T10:29:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.468+0000 7f2d86e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8015adb0 con 0x7f2d800aef80 2024-04-25T10:29:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.468+0000 7f2d86e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8015eb60 con 0x7f2d800b38d0 2024-04-25T10:29:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d77400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2d80070350 0x7f2d800b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55028/0 (socket says 172.21.15.45:55028) 2024-04-25T10:29:23.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d77400640 1 -- 172.21.15.45:0/3420953557 learned_addr learned my addr 172.21.15.45:0/3420953557 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 --2- 172.21.15.45:0/3420953557 >> v2:172.21.15.117:3300/0 conn(0x7f2d800b38d0 0x7f2d8015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049328148 0 0) 0x7f2d8015b080 con 0x7f2d80070350 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d64005740 con 0x7f2d80070350 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4151846045 0 0) 0x7f2d64005740 con 0x7f2d80070350 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 >> v2:172.21.15.117:3300/0 conn(0x7f2d800b38d0 msgr2=0x7f2d8015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 --2- 172.21.15.45:0/3420953557 >> v2:172.21.15.117:3300/0 conn(0x7f2d800b38d0 0x7f2d8015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 >> v1:172.21.15.117:6789/0 conn(0x7f2d800aef80 legacy=0x7f2d8015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8015ee40 con 0x7f2d80070350 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f2d68001680 con 0x7f2d80070350 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 >> v1:172.21.15.45:6789/0 conn(0x7f2d80070350 legacy=0x7f2d800b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d64005bd0 0x7f2d64005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d64006a20 0x7f2d64006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d640073e0 0x7f2d6400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8015eb60 con 0x7f2d640073e0 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6400e280 con 0x7f2d64005bd0 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6400e480 con 0x7f2d64006a20 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d640073e0 0x7f2d6400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d7fe00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d64005bd0 0x7f2d64005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d77400640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d64006a20 0x7f2d64006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 -- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d64006a20 msgr2=0x7f2d64006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d64006a20 0x7f2d64006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 -- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d64005bd0 msgr2=0x7f2d64005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d64005bd0 0x7f2d64005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d84e00640 1 -- 172.21.15.45:0/3420953557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64005740 con 0x7f2d640073e0 2024-04-25T10:29:23.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.472+0000 7f2d77400640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d64006a20 0x7f2d64006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d84e00640 1 -- 172.21.15.45:0/3420953557 reap_dead start 2024-04-25T10:29:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d84e00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d640073e0 0x7f2d6400dbe0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009f90 tx=0x7f2d7801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca2105e553bd516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78009c10 con 0x7f2d640073e0 2024-04-25T10:29:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d780a5c90 con 0x7f2d640073e0 2024-04-25T10:29:23.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d7f400640 1 -- 172.21.15.45:0/3420953557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003be0 con 0x7f2d640073e0 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 -- 172.21.15.45:0/3420953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d640073e0 msgr2=0x7f2d6400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d640073e0 0x7f2d6400dbe0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009f90 tx=0x7f2d7801cc80 comp rx=0 tx=0).stop 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 -- 172.21.15.45:0/3420953557 shutdown_connections 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 --2- 172.21.15.45:0/3420953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d640073e0 0x7f2d6400dbe0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 -- 172.21.15.45:0/3420953557 >> 172.21.15.45:0/3420953557 conn(0x7f2d800669a0 msgr2=0x7f2d80066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 -- 172.21.15.45:0/3420953557 shutdown_connections 2024-04-25T10:29:23.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 -- 172.21.15.45:0/3420953557 wait complete. 2024-04-25T10:29:23.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 Processor -- start 2024-04-25T10:29:23.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.476+0000 7f2d86e00640 1 -- start start 2024-04-25T10:29:23.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d640073e0 0x7f2d8014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d80157fe0 0x7f2d80158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 0x7f2d80154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8015e480 con 0x7f2d801547c0 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80071a00 con 0x7f2d80157fe0 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8015e600 con 0x7f2d640073e0 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 0x7f2d80154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 0x7f2d80154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34066/0 (socket says 172.21.15.45:34066) 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 -- 172.21.15.45:0/1350356353 learned_addr learned my addr 172.21.15.45:0/1350356353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d77400640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d80157fe0 0x7f2d80158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d7fe00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d640073e0 0x7f2d8014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 -- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d640073e0 msgr2=0x7f2d8014b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d640073e0 0x7f2d8014b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 -- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d80157fe0 msgr2=0x7f2d80158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d80157fe0 0x7f2d80158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 -- 172.21.15.45:0/1350356353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d801f3d20 con 0x7f2d801547c0 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d84e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 0x7f2d80154c40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f2d7801c6f0 tx=0x7f2d78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780a5a90 con 0x7f2d801547c0 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d780a6e50 con 0x7f2d801547c0 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780039a0 con 0x7f2d801547c0 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d801f4000 con 0x7f2d801547c0 2024-04-25T10:29:23.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d801f45c0 con 0x7f2d801547c0 2024-04-25T10:29:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8004ef30 con 0x7f2d801547c0 2024-04-25T10:29:23.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.480+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d780bf020 con 0x7f2d801547c0 2024-04-25T10:29:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.484+0000 7f2d7cc00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d5c066f90 0x7f2d5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.484+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d7811fd90 con 0x7f2d801547c0 2024-04-25T10:29:23.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.484+0000 7f2d7fe00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d5c066f90 0x7f2d5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.484+0000 7f2d7fe00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d5c066f90 0x7f2d5c069450 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c10 tx=0x7f2d70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.488+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d780c6020 con 0x7f2d801547c0 2024-04-25T10:29:23.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.648+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d800cd450 con 0x7f2d801547c0 2024-04-25T10:29:23.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.648+0000 7f2d7cc00640 1 -- 172.21.15.45:0/1350356353 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d78016990 con 0x7f2d801547c0 2024-04-25T10:29:23.651 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:29:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d5c066f90 msgr2=0x7f2d5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d5c066f90 0x7f2d5c069450 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000c10 tx=0x7f2d70009040 comp rx=0 tx=0).stop 2024-04-25T10:29:23.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 msgr2=0x7f2d80154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 0x7f2d80154c40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f2d7801c6f0 tx=0x7f2d78009750 comp rx=0 tx=0).stop 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 shutdown_connections 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2d5c066f90 0x7f2d5c069450 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d801547c0 0x7f2d80154c40 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d80157fe0 0x7f2d80158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 --2- 172.21.15.45:0/1350356353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d640073e0 0x7f2d8014b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 >> 172.21.15.45:0/1350356353 conn(0x7f2d800669a0 msgr2=0x7f2d80066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 shutdown_connections 2024-04-25T10:29:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.652+0000 7f2d86e00640 1 -- 172.21.15.45:0/1350356353 wait complete. 2024-04-25T10:29:23.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:29:23.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.812+0000 7fb05a200640 1 Processor -- start 2024-04-25T10:29:23.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.812+0000 7fb05a200640 1 -- start start 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb05a200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb05415bc40 0x7fb05406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb05a200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0540cf930 0x7fb0540cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb05a200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540b2770 con 0x7fb0540cda10 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb05a200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05406c670 con 0x7fb05415bc40 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb05a200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05406c950 con 0x7fb0540cf930 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb05415bc40 0x7fb05406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb05415bc40 0x7fb05406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34080/0 (socket says 172.21.15.45:34080) 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 -- 172.21.15.45:0/1766459053 learned_addr learned my addr 172.21.15.45:0/1766459053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053e00640 1 --2- 172.21.15.45:0/1766459053 >> v2:172.21.15.117:3300/0 conn(0x7fb0540cf930 0x7fb0540cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 -- 172.21.15.45:0/1766459053 >> v2:172.21.15.117:3300/0 conn(0x7fb0540cf930 msgr2=0x7fb0540cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 --2- 172.21.15.45:0/1766459053 >> v2:172.21.15.117:3300/0 conn(0x7fb0540cf930 0x7fb0540cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 -- 172.21.15.45:0/1766459053 >> v1:172.21.15.117:6789/0 conn(0x7fb0540cda10 legacy=0x7fb0540b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053400640 1 -- 172.21.15.45:0/1766459053 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05406cc30 con 0x7fb05415bc40 2024-04-25T10:29:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.816+0000 7fb053e00640 1 --2- 172.21.15.45:0/1766459053 >> v2:172.21.15.117:3300/0 conn(0x7fb0540cf930 0x7fb0540cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb053400640 1 --2- 172.21.15.45:0/1766459053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb05415bc40 0x7fb05406c090 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fb0540ae4c0 tx=0x7fb04401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e23ff0661e9102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb052a00640 1 -- 172.21.15.45:0/1766459053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440b0070 con 0x7fb05415bc40 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb052a00640 1 -- 172.21.15.45:0/1766459053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0440ab600 con 0x7fb05415bc40 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb052a00640 1 -- 172.21.15.45:0/1766459053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440ab9d0 con 0x7fb05415bc40 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- 172.21.15.45:0/1766459053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb05415bc40 msgr2=0x7fb05406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 --2- 172.21.15.45:0/1766459053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb05415bc40 0x7fb05406c090 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fb0540ae4c0 tx=0x7fb04401c490 comp rx=0 tx=0).stop 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- 172.21.15.45:0/1766459053 shutdown_connections 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 --2- 172.21.15.45:0/1766459053 >> v2:172.21.15.117:3300/0 conn(0x7fb0540cf930 0x7fb0540cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 --2- 172.21.15.45:0/1766459053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb05415bc40 0x7fb05406c090 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- 172.21.15.45:0/1766459053 >> 172.21.15.45:0/1766459053 conn(0x7fb054066b70 msgr2=0x7fb054066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- 172.21.15.45:0/1766459053 shutdown_connections 2024-04-25T10:29:23.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- 172.21.15.45:0/1766459053 wait complete. 2024-04-25T10:29:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 Processor -- start 2024-04-25T10:29:23.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- start start 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 0x7fb054156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0540cf930 0x7fb054156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb05415bc40 0x7fb0540af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.820+0000 7fb05a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b3850 con 0x7fb0540cda10 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb05a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b36d0 con 0x7fb05415bc40 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb05a200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b39d0 con 0x7fb0540cf930 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 0x7fb054156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 0x7fb054156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34096/0 (socket says 172.21.15.45:34096) 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 -- 172.21.15.45:0/2856894259 learned_addr learned my addr 172.21.15.45:0/2856894259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:23.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053e00640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb05415bc40 0x7fb0540af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb04aa00640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0540cf930 0x7fb054156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 -- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0540cf930 msgr2=0x7fb054156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0540cf930 0x7fb054156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 -- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb05415bc40 msgr2=0x7fb0540af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb05415bc40 0x7fb0540af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 -- 172.21.15.45:0/2856894259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540b0e30 con 0x7fb0540cda10 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb053400640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 0x7fb054156700 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb0440ae040 tx=0x7fb044019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440b0040 con 0x7fb0540cda10 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0440b9070 con 0x7fb0540cda10 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440b48a0 con 0x7fb0540cda10 2024-04-25T10:29:23.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540b10d0 con 0x7fb0540cda10 2024-04-25T10:29:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540b1690 con 0x7fb0540cda10 2024-04-25T10:29:23.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05404ef30 con 0x7fb0540cda10 2024-04-25T10:29:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.824+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0440b4a40 con 0x7fb0540cda10 2024-04-25T10:29:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.828+0000 7fb04be00640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb030066f60 0x7fb030069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.828+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb04411eaf0 con 0x7fb0540cda10 2024-04-25T10:29:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.828+0000 7fb04aa00640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb030066f60 0x7fb030069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.828+0000 7fb04aa00640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb030066f60 0x7fb030069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb0541fa5e0 tx=0x7fb03c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:23.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.832+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb044016110 con 0x7fb0540cda10 2024-04-25T10:29:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb05406d960 con 0x7fb0540cda10 2024-04-25T10:29:23.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb04be00640 1 -- 172.21.15.45:0/2856894259 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb0440efed0 con 0x7fb0540cda10 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:23.997 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb030066f60 msgr2=0x7fb030069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb030066f60 0x7fb030069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb0541fa5e0 tx=0x7fb03c009040 comp rx=0 tx=0).stop 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 msgr2=0x7fb054156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 0x7fb054156700 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb0440ae040 tx=0x7fb044019230 comp rx=0 tx=0).stop 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 shutdown_connections 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb030066f60 0x7fb030069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb05415bc40 0x7fb0540af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0540cf930 0x7fb054156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 --2- 172.21.15.45:0/2856894259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0540cda10 0x7fb054156700 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 >> 172.21.15.45:0/2856894259 conn(0x7fb054066b70 msgr2=0x7fb0540add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 shutdown_connections 2024-04-25T10:29:23.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:23.996+0000 7fb05a200640 1 -- 172.21.15.45:0/2856894259 wait complete. 2024-04-25T10:29:24.018 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:29:24.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:29:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.160+0000 7fc25b200640 1 Processor -- start 2024-04-25T10:29:24.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.160+0000 7fc25b200640 1 -- start start 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc25b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 0x7fc2540b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc25b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc2540cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc25b200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25406c0e0 con 0x7fc25415bd10 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc25b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2540d0390 con 0x7fc25415c0f0 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc25b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25415c6a0 con 0x7fc2540cf9e0 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc24be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 0x7fc2540b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc259200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc2540cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc24be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 0x7fc2540b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45146/0 (socket says 172.21.15.45:45146) 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc24be00640 1 -- 172.21.15.45:0/2595432695 learned_addr learned my addr 172.21.15.45:0/2595432695 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:24.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc253e00640 1 --1- 172.21.15.45:0/2595432695 >> v1:172.21.15.117:6789/0 conn(0x7fc25415bd10 0x7fc2540b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:54266/0 (socket says 172.21.15.45:54266) 2024-04-25T10:29:24.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc259200640 1 -- 172.21.15.45:0/2595432695 >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 msgr2=0x7fc2540b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:24.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc259200640 1 --2- 172.21.15.45:0/2595432695 >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 0x7fc2540b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc259200640 1 -- 172.21.15.45:0/2595432695 >> v1:172.21.15.117:6789/0 conn(0x7fc25415bd10 legacy=0x7fc2540b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc259200640 1 -- 172.21.15.45:0/2595432695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25415c980 con 0x7fc2540cf9e0 2024-04-25T10:29:24.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.164+0000 7fc24be00640 1 --2- 172.21.15.45:0/2595432695 >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 0x7fc2540b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:24.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc259200640 1 --2- 172.21.15.45:0/2595432695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc2540cfe50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009d90 tx=0x7fc24c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44a8d4903d946ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc253400640 1 -- 172.21.15.45:0/2595432695 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0b2070 con 0x7fc2540cf9e0 2024-04-25T10:29:24.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc253400640 1 -- 172.21.15.45:0/2595432695 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c0a5920 con 0x7fc2540cf9e0 2024-04-25T10:29:24.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc253400640 1 -- 172.21.15.45:0/2595432695 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0a5cf0 con 0x7fc2540cf9e0 2024-04-25T10:29:24.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 -- 172.21.15.45:0/2595432695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 msgr2=0x7fc2540cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 --2- 172.21.15.45:0/2595432695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc2540cfe50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009d90 tx=0x7fc24c01c970 comp rx=0 tx=0).stop 2024-04-25T10:29:24.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 -- 172.21.15.45:0/2595432695 shutdown_connections 2024-04-25T10:29:24.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 --2- 172.21.15.45:0/2595432695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc2540cfe50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 --2- 172.21.15.45:0/2595432695 >> v2:172.21.15.117:3300/0 conn(0x7fc25415c0f0 0x7fc2540b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 -- 172.21.15.45:0/2595432695 >> 172.21.15.45:0/2595432695 conn(0x7fc254066b70 msgr2=0x7fc254066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:24.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 -- 172.21.15.45:0/2595432695 shutdown_connections 2024-04-25T10:29:24.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 -- 172.21.15.45:0/2595432695 wait complete. 2024-04-25T10:29:24.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 Processor -- start 2024-04-25T10:29:24.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.168+0000 7fc25b200640 1 -- start start 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc25414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 0x7fc25414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc25415c0f0 0x7fc25414d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25406cf50 con 0x7fc25415bd10 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25406cdd0 con 0x7fc25415c0f0 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc254168ac0 con 0x7fc2540cf9e0 2024-04-25T10:29:24.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 0x7fc25414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 0x7fc25414d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34108/0 (socket says 172.21.15.45:34108) 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 -- 172.21.15.45:0/1018299041 learned_addr learned my addr 172.21.15.45:0/1018299041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc259200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc25415c0f0 0x7fc25414d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc253e00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc25414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 -- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 msgr2=0x7fc25414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc25414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 -- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc25415c0f0 msgr2=0x7fc25414d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:24.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc25415c0f0 0x7fc25414d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 -- 172.21.15.45:0/1018299041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540cd3c0 con 0x7fc25415bd10 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc24be00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 0x7fc25414d090 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00e960 tx=0x7fc23c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c017070 con 0x7fc25415bd10 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c00ca00 con 0x7fc25415bd10 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc254070360 con 0x7fc25415bd10 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00cdb0 con 0x7fc25415bd10 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25415df30 con 0x7fc25415bd10 2024-04-25T10:29:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.172+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25404ef30 con 0x7fc25415bd10 2024-04-25T10:29:24.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.176+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc23c01a070 con 0x7fc25415bd10 2024-04-25T10:29:24.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.176+0000 7fc250c00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc230066f60 0x7fc230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:24.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.176+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc23c084c20 con 0x7fc25415bd10 2024-04-25T10:29:24.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.176+0000 7fc253e00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc230066f60 0x7fc230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:24.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.176+0000 7fc253e00640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc230066f60 0x7fc230069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc2540b4510 tx=0x7fc244007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:24.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.180+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc23c055810 con 0x7fc25415bd10 2024-04-25T10:29:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:24.340+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc2540cc3a0 con 0x7fc25415bd10 2024-04-25T10:29:25.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.192+0000 7fc250c00640 1 -- 172.21.15.45:0/1018299041 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v152) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc23c0598b0 con 0x7fc25415bd10 2024-04-25T10:29:25.195 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:29:25.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc230066f60 msgr2=0x7fc230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc230066f60 0x7fc230069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc2540b4510 tx=0x7fc244007e90 comp rx=0 tx=0).stop 2024-04-25T10:29:25.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 msgr2=0x7fc25414d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 0x7fc25414d090 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00e960 tx=0x7fc23c00ee20 comp rx=0 tx=0).stop 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 shutdown_connections 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc230066f60 0x7fc230069420 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc25415c0f0 0x7fc25414d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc25415bd10 0x7fc25414d090 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 --2- 172.21.15.45:0/1018299041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2540cf9e0 0x7fc25414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 >> 172.21.15.45:0/1018299041 conn(0x7fc254066b70 msgr2=0x7fc2540b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.196+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 shutdown_connections 2024-04-25T10:29:25.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.200+0000 7fc25b200640 1 -- 172.21.15.45:0/1018299041 wait complete. 2024-04-25T10:29:25.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:29:25.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc17a200640 1 Processor -- start 2024-04-25T10:29:25.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc17a200640 1 -- start start 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc17a200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1740d18f0 0x7fc17406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc17a200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740d0170 con 0x7fc1740cfbb0 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc17a200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1740d03e0 con 0x7fc1740d1510 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc17a200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1740d06c0 con 0x7fc1740d18f0 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc179800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1740d18f0 0x7fc17406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc16be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc1740cfbb0 0x7fc1740cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60906/0 (socket says 172.21.15.45:60906) 2024-04-25T10:29:25.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc16be00640 1 -- 172.21.15.45:0/2005064500 learned_addr learned my addr 172.21.15.45:0/2005064500 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:25.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc179800640 1 -- 172.21.15.45:0/2005064500 >> v1:172.21.15.117:6789/0 conn(0x7fc1740d1510 legacy=0x7fc1740b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc179800640 1 -- 172.21.15.45:0/2005064500 >> v1:172.21.15.45:6789/0 conn(0x7fc1740cfbb0 legacy=0x7fc1740cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc179800640 1 -- 172.21.15.45:0/2005064500 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17406b5f0 con 0x7fc1740d18f0 2024-04-25T10:29:25.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.380+0000 7fc173e00640 1 -- 172.21.15.45:0/2005064500 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052481882 0 0) 0x7fc1740d0170 con 0x7fc1740cfbb0 2024-04-25T10:29:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc179800640 1 --2- 172.21.15.45:0/2005064500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17406bb10 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009d90 tx=0x7fc16c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80d4bf8c1bbcf15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc173e00640 1 -- 172.21.15.45:0/2005064500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0b2070 con 0x7fc1740d18f0 2024-04-25T10:29:25.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc173e00640 1 -- 172.21.15.45:0/2005064500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c0a5a50 con 0x7fc1740d18f0 2024-04-25T10:29:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc173e00640 1 -- 172.21.15.45:0/2005064500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0a5e20 con 0x7fc1740d18f0 2024-04-25T10:29:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 -- 172.21.15.45:0/2005064500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 msgr2=0x7fc17406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 --2- 172.21.15.45:0/2005064500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17406bb10 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009d90 tx=0x7fc16c01c970 comp rx=0 tx=0).stop 2024-04-25T10:29:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 -- 172.21.15.45:0/2005064500 shutdown_connections 2024-04-25T10:29:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 --2- 172.21.15.45:0/2005064500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17406bb10 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 -- 172.21.15.45:0/2005064500 >> 172.21.15.45:0/2005064500 conn(0x7fc174072820 msgr2=0x7fc17409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 -- 172.21.15.45:0/2005064500 shutdown_connections 2024-04-25T10:29:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 -- 172.21.15.45:0/2005064500 wait complete. 2024-04-25T10:29:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 Processor -- start 2024-04-25T10:29:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.384+0000 7fc17a200640 1 -- start start 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1740cfbb0 0x7fc17415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1740d1510 0x7fc17415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17415d450 con 0x7fc1740d18f0 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17415d5d0 con 0x7fc1740d1510 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17415d750 con 0x7fc1740cfbb0 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56720/0 (socket says 172.21.15.45:56720) 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 -- 172.21.15.45:0/499643558 learned_addr learned my addr 172.21.15.45:0/499643558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc178e00640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1740cfbb0 0x7fc17415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc16be00640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1740d1510 0x7fc17415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 -- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1740cfbb0 msgr2=0x7fc17415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1740cfbb0 0x7fc17415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 -- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1740d1510 msgr2=0x7fc17415a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1740d1510 0x7fc17415a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 -- 172.21.15.45:0/499643558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc174153200 con 0x7fc1740d18f0 2024-04-25T10:29:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc179800640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17415c090 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0b0040 tx=0x7fc16c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0b2040 con 0x7fc1740d18f0 2024-04-25T10:29:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c0bf070 con 0x7fc1740d18f0 2024-04-25T10:29:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c0b8470 con 0x7fc1740d18f0 2024-04-25T10:29:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1740c60b0 con 0x7fc1740d18f0 2024-04-25T10:29:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1740c6670 con 0x7fc1740d18f0 2024-04-25T10:29:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.388+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc174075120 con 0x7fc1740d18f0 2024-04-25T10:29:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.392+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc16c0b8610 con 0x7fc1740d18f0 2024-04-25T10:29:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.392+0000 7fc171600640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc150066f90 0x7fc150069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.392+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc16c11ff50 con 0x7fc1740d18f0 2024-04-25T10:29:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.392+0000 7fc178e00640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc150066f90 0x7fc150069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:25.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.392+0000 7fc178e00640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc150066f90 0x7fc150069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc164000f70 tx=0x7fc164009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.396+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc16c0c8020 con 0x7fc1740d18f0 2024-04-25T10:29:25.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:25.560+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc1740d2690 con 0x7fc1740d18f0 2024-04-25T10:29:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.196+0000 7fc171600640 1 -- 172.21.15.45:0/499643558 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v153) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc16c016560 con 0x7fc1740d18f0 2024-04-25T10:29:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc150066f90 msgr2=0x7fc150069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc150066f90 0x7fc150069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc164000f70 tx=0x7fc164009040 comp rx=0 tx=0).stop 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 msgr2=0x7fc17415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17415c090 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0b0040 tx=0x7fc16c019710 comp rx=0 tx=0).stop 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 shutdown_connections 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc150066f90 0x7fc150069450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1740d18f0 0x7fc17415c090 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1740d1510 0x7fc17415a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 --2- 172.21.15.45:0/499643558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1740cfbb0 0x7fc17415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 >> 172.21.15.45:0/499643558 conn(0x7fc174072820 msgr2=0x7fc1740aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 shutdown_connections 2024-04-25T10:29:26.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.200+0000 7fc17a200640 1 -- 172.21.15.45:0/499643558 wait complete. 2024-04-25T10:29:26.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:29:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.384+0000 7f50bf800640 1 Processor -- start 2024-04-25T10:29:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.384+0000 7f50bf800640 1 -- start start 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b8159540 0x7f50b80ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bf800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50b80aebd0 0x7f50b8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bf800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50b8072020 0x7f50b815f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bf800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b80af030 con 0x7f50b8072020 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bf800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b8159920 con 0x7f50b80aebd0 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b815c2f0 con 0x7f50b8159540 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50b8072020 0x7f50b815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50b8072020 0x7f50b815f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56736/0 (socket says 172.21.15.45:56736) 2024-04-25T10:29:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 -- 172.21.15.45:0/2101087139 learned_addr learned my addr 172.21.15.45:0/2101087139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50afe00640 1 --2- 172.21.15.45:0/2101087139 >> v2:172.21.15.117:3300/0 conn(0x7f50b80aebd0 0x7f50b8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bce00640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b8159540 0x7f50b80ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 -- 172.21.15.45:0/2101087139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b8159540 msgr2=0x7f50b80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b8159540 0x7f50b80ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 -- 172.21.15.45:0/2101087139 >> v2:172.21.15.117:3300/0 conn(0x7f50b80aebd0 msgr2=0x7f50b8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 --2- 172.21.15.45:0/2101087139 >> v2:172.21.15.117:3300/0 conn(0x7f50b80aebd0 0x7f50b8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 -- 172.21.15.45:0/2101087139 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b815c5d0 con 0x7f50b8072020 2024-04-25T10:29:26.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bce00640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b8159540 0x7f50b80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.388+0000 7f50bd800640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8072020 0x7f50b815f590 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f50b80b4b30 tx=0x7f50b001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787360595ba96a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50b7e00640 1 -- 172.21.15.45:0/2101087139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00a5580 con 0x7f50b8072020 2024-04-25T10:29:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50b7e00640 1 -- 172.21.15.45:0/2101087139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00a5720 con 0x7f50b8072020 2024-04-25T10:29:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50b7e00640 1 -- 172.21.15.45:0/2101087139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00a5af0 con 0x7f50b8072020 2024-04-25T10:29:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- 172.21.15.45:0/2101087139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8072020 msgr2=0x7f50b815f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8072020 0x7f50b815f590 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f50b80b4b30 tx=0x7f50b001c700 comp rx=0 tx=0).stop 2024-04-25T10:29:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- 172.21.15.45:0/2101087139 shutdown_connections 2024-04-25T10:29:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8072020 0x7f50b815f590 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- 172.21.15.45:0/2101087139 >> v2:172.21.15.117:3300/0 conn(0x7f50b80aebd0 0x7f50b8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- 172.21.15.45:0/2101087139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b8159540 0x7f50b80ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- 172.21.15.45:0/2101087139 >> 172.21.15.45:0/2101087139 conn(0x7f50b80669a0 msgr2=0x7f50b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- 172.21.15.45:0/2101087139 shutdown_connections 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- 172.21.15.45:0/2101087139 wait complete. 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 Processor -- start 2024-04-25T10:29:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- start start 2024-04-25T10:29:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b8072020 0x7f50b8153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b80aebd0 0x7f50b814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 0x7f50b814d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b806c0d0 con 0x7f50b8159540 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8072450 con 0x7f50b8072020 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bf800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8070350 con 0x7f50b80aebd0 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bd800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 0x7f50b814d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bd800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 0x7f50b814d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56746/0 (socket says 172.21.15.45:56746) 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.392+0000 7f50bd800640 1 -- 172.21.15.45:0/2914392152 learned_addr learned my addr 172.21.15.45:0/2914392152 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bce00640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b8072020 0x7f50b8153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50afe00640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b80aebd0 0x7f50b814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bd800640 1 -- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b80aebd0 msgr2=0x7f50b814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bd800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b80aebd0 0x7f50b814cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bd800640 1 -- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b8072020 msgr2=0x7f50b8153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bd800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b8072020 0x7f50b8153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bd800640 1 -- 172.21.15.45:0/2914392152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8153800 con 0x7f50b8159540 2024-04-25T10:29:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50afe00640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b80aebd0 0x7f50b814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bd800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 0x7f50b814d440 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f50b00af040 tx=0x7f50b00194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0009510 con 0x7f50b8159540 2024-04-25T10:29:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00be070 con 0x7f50b8159540 2024-04-25T10:29:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00b88b0 con 0x7f50b8159540 2024-04-25T10:29:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b8153ae0 con 0x7f50b8159540 2024-04-25T10:29:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b814f170 con 0x7f50b8159540 2024-04-25T10:29:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b804ef30 con 0x7f50b8159540 2024-04-25T10:29:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50b00b8a50 con 0x7f50b8159540 2024-04-25T10:29:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50b5600640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.396+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f50b0120ce0 con 0x7f50b8159540 2024-04-25T10:29:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.400+0000 7f50bce00640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:26.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.400+0000 7f50bce00640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006fd0 tx=0x7f50a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:26.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.400+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50b00165e0 con 0x7f50b8159540 2024-04-25T10:29:26.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:26.600+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f50b8166700 con 0x7f50b8159540 2024-04-25T10:29:27.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.220+0000 7f50b5600640 1 -- 172.21.15.45:0/2914392152 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v111) v1 ==== 174+0+0 (secure 0 0 0) 0x7f50b00f1f10 con 0x7f50b8159540 2024-04-25T10:29:27.224 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 22 and data pool 23 2024-04-25T10:29:27.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.228+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 msgr2=0x7f5094069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.228+0000 7f50bf800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006fd0 tx=0x7f50a8008040 comp rx=0 tx=0).stop 2024-04-25T10:29:27.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.228+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 msgr2=0x7f50b814d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.228+0000 7f50bf800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 0x7f50b814d440 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f50b00af040 tx=0x7f50b00194a0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 shutdown_connections 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5094066f60 0x7f5094069420 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50b8159540 0x7f50b814d440 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b80aebd0 0x7f50b814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 --2- 172.21.15.45:0/2914392152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b8072020 0x7f50b8153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 >> 172.21.15.45:0/2914392152 conn(0x7f50b80669a0 msgr2=0x7f50b8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 shutdown_connections 2024-04-25T10:29:27.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.232+0000 7f50bf800640 1 -- 172.21.15.45:0/2914392152 wait complete. 2024-04-25T10:29:27.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.408+0000 7f3f9c600640 1 Processor -- start 2024-04-25T10:29:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f9c600640 1 -- start start 2024-04-25T10:29:27.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f9c600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f9415b9e0 0x7f3f940cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f9c600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940ccfb0 con 0x7f3f940d1c30 2024-04-25T10:29:27.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f9c600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940cd290 con 0x7f3f9415bdc0 2024-04-25T10:29:27.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f9c600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f940cd570 con 0x7f3f9415b9e0 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f99c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f9415b9e0 0x7f3f940cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f99c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f9415b9e0 0x7f3f940cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56750/0 (socket says 172.21.15.45:56750) 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f9a600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f940d1c30 0x7f3f940c5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60914/0 (socket says 172.21.15.45:60914) 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f99c00640 1 -- 172.21.15.45:0/2317886468 learned_addr learned my addr 172.21.15.45:0/2317886468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f99c00640 1 -- 172.21.15.45:0/2317886468 >> v1:172.21.15.117:6789/0 conn(0x7f3f9415bdc0 legacy=0x7f3f940d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f99c00640 1 -- 172.21.15.45:0/2317886468 >> v1:172.21.15.45:6789/0 conn(0x7f3f940d1c30 legacy=0x7f3f940c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.412+0000 7f3f99c00640 1 -- 172.21.15.45:0/2317886468 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940cd850 con 0x7f3f9415b9e0 2024-04-25T10:29:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f99c00640 1 --2- 172.21.15.45:0/2317886468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9415b9e0 0x7f3f940cc410 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009870 tx=0x7f3f8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d528f25391ffe42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f93e00640 1 -- 172.21.15.45:0/2317886468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880b0070 con 0x7f3f9415b9e0 2024-04-25T10:29:27.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f93e00640 1 -- 172.21.15.45:0/2317886468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f880ab600 con 0x7f3f9415b9e0 2024-04-25T10:29:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f93e00640 1 -- 172.21.15.45:0/2317886468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880ab9d0 con 0x7f3f9415b9e0 2024-04-25T10:29:27.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 -- 172.21.15.45:0/2317886468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9415b9e0 msgr2=0x7f3f940cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 --2- 172.21.15.45:0/2317886468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9415b9e0 0x7f3f940cc410 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009870 tx=0x7f3f8801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 -- 172.21.15.45:0/2317886468 shutdown_connections 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 --2- 172.21.15.45:0/2317886468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9415b9e0 0x7f3f940cc410 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 -- 172.21.15.45:0/2317886468 >> 172.21.15.45:0/2317886468 conn(0x7f3f9409a9d0 msgr2=0x7f3f9409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 -- 172.21.15.45:0/2317886468 shutdown_connections 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 -- 172.21.15.45:0/2317886468 wait complete. 2024-04-25T10:29:27.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 Processor -- start 2024-04-25T10:29:27.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.416+0000 7f3f9c600640 1 -- start start 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 0x7f3f9414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f9415b9e0 0x7f3f9415de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9415bdc0 0x7f3f940b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940c6d40 con 0x7f3f940d1c30 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940c6bc0 con 0x7f3f9415bdc0 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94169840 con 0x7f3f9415b9e0 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 0x7f3f9414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 0x7f3f9414f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56752/0 (socket says 172.21.15.45:56752) 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 -- 172.21.15.45:0/1854863235 learned_addr learned my addr 172.21.15.45:0/1854863235 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9a600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9415bdc0 0x7f3f940b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99200640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f9415b9e0 0x7f3f9415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 -- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f9415b9e0 msgr2=0x7f3f9415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f9415b9e0 0x7f3f9415de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 -- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9415bdc0 msgr2=0x7f3f940b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9415bdc0 0x7f3f940b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 -- 172.21.15.45:0/1854863235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940b5100 con 0x7f3f940d1c30 2024-04-25T10:29:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f99c00640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 0x7f3f9414f9b0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ae040 tx=0x7f3f88019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880bb070 con 0x7f3f940d1c30 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f880b6630 con 0x7f3f940d1c30 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9415e750 con 0x7f3f940d1c30 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880b0040 con 0x7f3f940d1c30 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f940b5680 con 0x7f3f940d1c30 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9404ef30 con 0x7f3f940d1c30 2024-04-25T10:29:27.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.420+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f880be070 con 0x7f3f940d1c30 2024-04-25T10:29:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.424+0000 7f3f91600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f74066f60 0x7f3f74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.424+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f8811e7d0 con 0x7f3f940d1c30 2024-04-25T10:29:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.424+0000 7f3f99200640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f74066f60 0x7f3f74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.424+0000 7f3f99200640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f74066f60 0x7f3f74069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3f94157340 tx=0x7f3f80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.428+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f88016150 con 0x7f3f940d1c30 2024-04-25T10:29:27.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.588+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f940c4f30 con 0x7f3f940d1c30 2024-04-25T10:29:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f91600640 1 -- 172.21.15.45:0/1854863235 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f3f880ef740 con 0x7f3f940d1c30 2024-04-25T10:29:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:27.592 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":154,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:26.602812+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f74066f60 msgr2=0x7f3f74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f74066f60 0x7f3f74069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3f94157340 tx=0x7f3f80009040 comp rx=0 tx=0).stop 2024-04-25T10:29:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 msgr2=0x7f3f9414f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 0x7f3f9414f9b0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f3f880ae040 tx=0x7f3f88019270 comp rx=0 tx=0).stop 2024-04-25T10:29:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 shutdown_connections 2024-04-25T10:29:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f74066f60 0x7f3f74069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9415bdc0 0x7f3f940b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f9415b9e0 0x7f3f9415de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 --2- 172.21.15.45:0/1854863235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f940d1c30 0x7f3f9414f9b0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 >> 172.21.15.45:0/1854863235 conn(0x7f3f9409a9d0 msgr2=0x7f3f940c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 shutdown_connections 2024-04-25T10:29:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.592+0000 7f3f9c600640 1 -- 172.21.15.45:0/1854863235 wait complete. 2024-04-25T10:29:27.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:27.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.752+0000 7f1cd7200640 1 Processor -- start 2024-04-25T10:29:27.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.752+0000 7f1cd7200640 1 -- start start 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd7200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cd815db40 0x7f1cd806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd7200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd815df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd7200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd815e460 con 0x7f1cd806b2c0 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd7200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80b5780 con 0x7f1cd815db40 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd7200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80b5a60 con 0x7f1cd80b5010 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cd815db40 0x7f1cd806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd6800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1cd806b2c0 0x7f1cd80b6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60924/0 (socket says 172.21.15.45:60924) 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1cd815db40 0x7f1cd806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56766/0 (socket says 172.21.15.45:56766) 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd6800640 1 -- 172.21.15.45:0/4000739103 learned_addr learned my addr 172.21.15.45:0/4000739103 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1ccd400640 1 --2- 172.21.15.45:0/4000739103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 -- 172.21.15.45:0/4000739103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 msgr2=0x7f1cd815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 --2- 172.21.15.45:0/4000739103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd815df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 -- 172.21.15.45:0/4000739103 >> v1:172.21.15.45:6789/0 conn(0x7f1cd806b2c0 legacy=0x7f1cd80b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 -- 172.21.15.45:0/4000739103 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd80b5d40 con 0x7f1cd815db40 2024-04-25T10:29:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5e00640 1 --2- 172.21.15.45:0/4000739103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd806ac90 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097c0 tx=0x7f1cc801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790078b2b7e8411b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5400640 1 -- 172.21.15.45:0/4000739103 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80b5070 con 0x7f1cd815db40 2024-04-25T10:29:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5400640 1 -- 172.21.15.45:0/4000739103 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc80a8990 con 0x7f1cd815db40 2024-04-25T10:29:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.756+0000 7f1cd5400640 1 -- 172.21.15.45:0/4000739103 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80a8d60 con 0x7f1cd815db40 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- 172.21.15.45:0/4000739103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 msgr2=0x7f1cd806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 --2- 172.21.15.45:0/4000739103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd806ac90 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097c0 tx=0x7f1cc801ec90 comp rx=0 tx=0).stop 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- 172.21.15.45:0/4000739103 shutdown_connections 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 --2- 172.21.15.45:0/4000739103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 --2- 172.21.15.45:0/4000739103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd806ac90 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- 172.21.15.45:0/4000739103 >> 172.21.15.45:0/4000739103 conn(0x7f1cd80d17b0 msgr2=0x7f1cd80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- 172.21.15.45:0/4000739103 shutdown_connections 2024-04-25T10:29:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- 172.21.15.45:0/4000739103 wait complete. 2024-04-25T10:29:27.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 Processor -- start 2024-04-25T10:29:27.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- start start 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd806b2c0 0x7f1cd8155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd80c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd81e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80b5450 con 0x7f1cd815db40 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80b6300 con 0x7f1cd806b2c0 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd7200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd806b730 con 0x7f1cd80b5010 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd81e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd81e79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56768/0 (socket says 172.21.15.45:56768) 2024-04-25T10:29:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 -- 172.21.15.45:0/1399537264 learned_addr learned my addr 172.21.15.45:0/1399537264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1ccd400640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd80c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd5e00640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd806b2c0 0x7f1cd8155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 -- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 msgr2=0x7f1cd80c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd80c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 -- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd806b2c0 msgr2=0x7f1cd8155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd806b2c0 0x7f1cd8155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd6800640 1 -- 172.21.15.45:0/1399537264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8156210 con 0x7f1cd815db40 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.760+0000 7f1cd5e00640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd806b2c0 0x7f1cd8155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1cd6800640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd81e79d0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000cbc0 tx=0x7f1cd000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0017070 con 0x7f1cd815db40 2024-04-25T10:29:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000ad40 con 0x7f1cd815db40 2024-04-25T10:29:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0012600 con 0x7f1cd815db40 2024-04-25T10:29:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd80cd2c0 con 0x7f1cd815db40 2024-04-25T10:29:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd80cd880 con 0x7f1cd815db40 2024-04-25T10:29:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd8075120 con 0x7f1cd815db40 2024-04-25T10:29:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cd001a070 con 0x7f1cd815db40 2024-04-25T10:29:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1ccea00640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1cb4066f90 0x7f1cb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cd0085100 con 0x7f1cd815db40 2024-04-25T10:29:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.764+0000 7f1cd5e00640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1cb4066f90 0x7f1cb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.768+0000 7f1cd5e00640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1cb4066f90 0x7f1cb4069450 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80b2000 tx=0x7f1cc8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.768+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cd0055aa0 con 0x7f1cd815db40 2024-04-25T10:29:27.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.932+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cd80b29a0 con 0x7f1cd815db40 2024-04-25T10:29:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.932+0000 7f1ccea00640 1 -- 172.21.15.45:0/1399537264 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f1cd0059b40 con 0x7f1cd815db40 2024-04-25T10:29:27.934 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:27.934 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":154,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:26.602812+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:27.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1cb4066f90 msgr2=0x7f1cb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1cb4066f90 0x7f1cb4069450 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80b2000 tx=0x7f1cc8008040 comp rx=0 tx=0).stop 2024-04-25T10:29:27.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 msgr2=0x7f1cd81e79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd81e79d0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000cbc0 tx=0x7f1cd000a430 comp rx=0 tx=0).stop 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 shutdown_connections 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1cb4066f90 0x7f1cb4069450 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cd815db40 0x7f1cd81e79d0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd80b5010 0x7f1cd80c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 --2- 172.21.15.45:0/1399537264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd806b2c0 0x7f1cd8155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 >> 172.21.15.45:0/1399537264 conn(0x7f1cd80d17b0 msgr2=0x7f1cd806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 shutdown_connections 2024-04-25T10:29:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:27.936+0000 7f1cd7200640 1 -- 172.21.15.45:0/1399537264 wait complete. 2024-04-25T10:29:27.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:29:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 Processor -- start 2024-04-25T10:29:28.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 -- start start 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a40cd9a0 0x7f52a40d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52a4162020 0x7f52a415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a406ac90 con 0x7f52a406bd10 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a406b880 con 0x7f52a4162020 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52ac000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a406af50 con 0x7f52a40cd9a0 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52aa000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52a4162020 0x7f52a415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52a8c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f52a406bd10 0x7f52a4161910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60940/0 (socket says 172.21.15.45:60940) 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52a8c00640 1 -- 172.21.15.45:0/4064527714 learned_addr learned my addr 172.21.15.45:0/4064527714 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:28.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52aa000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52a4162020 0x7f52a415d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56782/0 (socket says 172.21.15.45:56782) 2024-04-25T10:29:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52a9600640 1 --2- 172.21.15.45:0/4064527714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a40cd9a0 0x7f52a40d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52aa000640 1 -- 172.21.15.45:0/4064527714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a40cd9a0 msgr2=0x7f52a40d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52aa000640 1 --2- 172.21.15.45:0/4064527714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a40cd9a0 0x7f52a40d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52aa000640 1 -- 172.21.15.45:0/4064527714 >> v1:172.21.15.45:6789/0 conn(0x7f52a406bd10 legacy=0x7f52a4161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52aa000640 1 -- 172.21.15.45:0/4064527714 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a406b230 con 0x7f52a4162020 2024-04-25T10:29:28.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.100+0000 7f52a3e00640 1 -- 172.21.15.45:0/4064527714 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074664095 0 0) 0x7f52a406ac90 con 0x7f52a406bd10 2024-04-25T10:29:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52aa000640 1 --2- 172.21.15.45:0/4064527714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a415d610 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f529c009d90 tx=0x7f529c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b27c9cc95cbf93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52a3e00640 1 -- 172.21.15.45:0/4064527714 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0b4070 con 0x7f52a4162020 2024-04-25T10:29:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52a3e00640 1 -- 172.21.15.45:0/4064527714 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c0af600 con 0x7f52a4162020 2024-04-25T10:29:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52a3e00640 1 -- 172.21.15.45:0/4064527714 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0af9d0 con 0x7f52a4162020 2024-04-25T10:29:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 -- 172.21.15.45:0/4064527714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 msgr2=0x7f52a415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 --2- 172.21.15.45:0/4064527714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a415d610 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f529c009d90 tx=0x7f529c01f150 comp rx=0 tx=0).stop 2024-04-25T10:29:28.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 -- 172.21.15.45:0/4064527714 shutdown_connections 2024-04-25T10:29:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 --2- 172.21.15.45:0/4064527714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a415d610 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 --2- 172.21.15.45:0/4064527714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a40cd9a0 0x7f52a40d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 -- 172.21.15.45:0/4064527714 >> 172.21.15.45:0/4064527714 conn(0x7f52a40d19a0 msgr2=0x7f52a40d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 -- 172.21.15.45:0/4064527714 shutdown_connections 2024-04-25T10:29:28.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.104+0000 7f52ac000640 1 -- 172.21.15.45:0/4064527714 wait complete. 2024-04-25T10:29:28.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 Processor -- start 2024-04-25T10:29:28.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- start start 2024-04-25T10:29:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a406bd10 0x7f52a40c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a40cd9a0 0x7f52a4152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a41f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4162660 con 0x7f52a4162020 2024-04-25T10:29:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40cdd80 con 0x7f52a40cd9a0 2024-04-25T10:29:28.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40d0ba0 con 0x7f52a406bd10 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a41f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a41f3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56792/0 (socket says 172.21.15.45:56792) 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 -- 172.21.15.45:0/1279800022 learned_addr learned my addr 172.21.15.45:0/1279800022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52a8c00640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a40cd9a0 0x7f52a4152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52a9600640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a406bd10 0x7f52a40c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 -- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a406bd10 msgr2=0x7f52a40c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a406bd10 0x7f52a40c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 -- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a40cd9a0 msgr2=0x7f52a4152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a40cd9a0 0x7f52a4152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 -- 172.21.15.45:0/1279800022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40c9df0 con 0x7f52a4162020 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52a8c00640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a40cd9a0 0x7f52a4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52aa000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a41f3b90 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f529c009d90 tx=0x7f529c01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c0b4040 con 0x7f52a4162020 2024-04-25T10:29:28.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c0039a0 con 0x7f52a4162020 2024-04-25T10:29:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c009450 con 0x7f52a4162020 2024-04-25T10:29:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40ca0d0 con 0x7f52a4162020 2024-04-25T10:29:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a40ca690 con 0x7f52a4162020 2024-04-25T10:29:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.108+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a404ef30 con 0x7f52a4162020 2024-04-25T10:29:28.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.112+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f529c0c4060 con 0x7f52a4162020 2024-04-25T10:29:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.112+0000 7f52a1600640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5284066f90 0x7f5284069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:28.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.112+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f529c123010 con 0x7f52a4162020 2024-04-25T10:29:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.112+0000 7f52a9600640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5284066f90 0x7f5284069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:28.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.112+0000 7f52a9600640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5284066f90 0x7f5284069450 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f529800b9d0 tx=0x7f5298008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.116+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f529c018cb0 con 0x7f52a4162020 2024-04-25T10:29:28.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:28.316+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f52a40b7920 con 0x7f52a4162020 2024-04-25T10:29:29.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.236+0000 7f52a1600640 1 -- 172.21.15.45:0/1279800022 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v114) v1 ==== 118+0+0 (secure 0 0 0) 0x7f529c0f41b0 con 0x7f52a4162020 2024-04-25T10:29:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5284066f90 msgr2=0x7f5284069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5284066f90 0x7f5284069450 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f529800b9d0 tx=0x7f5298008040 comp rx=0 tx=0).stop 2024-04-25T10:29:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 msgr2=0x7f52a41f3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a41f3b90 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f529c009d90 tx=0x7f529c01bef0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 shutdown_connections 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5284066f90 0x7f5284069450 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52a4162020 0x7f52a41f3b90 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52a40cd9a0 0x7f52a4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 --2- 172.21.15.45:0/1279800022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52a406bd10 0x7f52a40c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 >> 172.21.15.45:0/1279800022 conn(0x7f52a40d19a0 msgr2=0x7f52a4160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 shutdown_connections 2024-04-25T10:29:29.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.240+0000 7f52ac000640 1 -- 172.21.15.45:0/1279800022 wait complete. 2024-04-25T10:29:29.265 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:29:29.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:29:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.412+0000 7f9c44600640 1 Processor -- start 2024-04-25T10:29:29.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.412+0000 7f9c44600640 1 -- start start 2024-04-25T10:29:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c44600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c159540 0x7f9c3c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c44600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 0x7f9c3c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c44600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c159920 con 0x7f9c3c0aebd0 2024-04-25T10:29:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c44600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0af030 con 0x7f9c3c0721f0 2024-04-25T10:29:29.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c44600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c15c2f0 con 0x7f9c3c159540 2024-04-25T10:29:29.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c42600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 0x7f9c3c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c159540 0x7f9c3c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41200640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9c3c0aebd0 0x7f9c3c071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:46188/0 (socket says 172.21.15.45:46188) 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c42600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 0x7f9c3c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:48144/0 (socket says 172.21.15.45:48144) 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41200640 1 -- 172.21.15.45:0/611991374 learned_addr learned my addr 172.21.15.45:0/611991374 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c3be00640 1 -- 172.21.15.45:0/611991374 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634391768 0 0) 0x7f9c3c159920 con 0x7f9c3c0aebd0 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c3be00640 1 -- 172.21.15.45:0/611991374 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c24003530 con 0x7f9c3c0aebd0 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41c00640 1 -- 172.21.15.45:0/611991374 >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 msgr2=0x7f9c3c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41c00640 1 --2- 172.21.15.45:0/611991374 >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 0x7f9c3c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41c00640 1 -- 172.21.15.45:0/611991374 >> v1:172.21.15.117:6789/0 conn(0x7f9c3c0aebd0 legacy=0x7f9c3c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c41c00640 1 -- 172.21.15.45:0/611991374 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c15c5d0 con 0x7f9c3c159540 2024-04-25T10:29:29.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.416+0000 7f9c42600640 1 --2- 172.21.15.45:0/611991374 >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 0x7f9c3c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:29.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c41c00640 1 --2- 172.21.15.45:0/611991374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c159540 0x7f9c3c0ae690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097c0 tx=0x7f9c3001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62751e7b64756dca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:29.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c3be00640 1 -- 172.21.15.45:0/611991374 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300a7070 con 0x7f9c3c159540 2024-04-25T10:29:29.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c3be00640 1 -- 172.21.15.45:0/611991374 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300b0070 con 0x7f9c3c159540 2024-04-25T10:29:29.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c3be00640 1 -- 172.21.15.45:0/611991374 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300ab890 con 0x7f9c3c159540 2024-04-25T10:29:29.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- 172.21.15.45:0/611991374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c159540 msgr2=0x7f9c3c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 --2- 172.21.15.45:0/611991374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c159540 0x7f9c3c0ae690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097c0 tx=0x7f9c3001c470 comp rx=0 tx=0).stop 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- 172.21.15.45:0/611991374 shutdown_connections 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 --2- 172.21.15.45:0/611991374 >> v2:172.21.15.117:3300/0 conn(0x7f9c3c0721f0 0x7f9c3c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 --2- 172.21.15.45:0/611991374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c159540 0x7f9c3c0ae690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- 172.21.15.45:0/611991374 >> 172.21.15.45:0/611991374 conn(0x7f9c3c0669a0 msgr2=0x7f9c3c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- 172.21.15.45:0/611991374 shutdown_connections 2024-04-25T10:29:29.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- 172.21.15.45:0/611991374 wait complete. 2024-04-25T10:29:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 Processor -- start 2024-04-25T10:29:29.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- start start 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c0721f0 0x7f9c3c14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c3c0aebd0 0x7f9c3c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 0x7f9c3c0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0710a0 con 0x7f9c3c159540 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c06c0d0 con 0x7f9c3c0aebd0 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.420+0000 7f9c44600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c071220 con 0x7f9c3c0721f0 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 0x7f9c3c0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 0x7f9c3c0b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56796/0 (socket says 172.21.15.45:56796) 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 -- 172.21.15.45:0/3813496869 learned_addr learned my addr 172.21.15.45:0/3813496869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c41c00640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c0721f0 0x7f9c3c14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:29.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c41200640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c3c0aebd0 0x7f9c3c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 -- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c0721f0 msgr2=0x7f9c3c14d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c0721f0 0x7f9c3c14d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 -- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c3c0aebd0 msgr2=0x7f9c3c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c3c0aebd0 0x7f9c3c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 -- 172.21.15.45:0/3813496869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c15b4c0 con 0x7f9c3c159540 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c41200640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c3c0aebd0 0x7f9c3c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c41c00640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c0721f0 0x7f9c3c14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c42600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 0x7f9c3c0b3c80 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400efc0 tx=0x7f9c34009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34018030 con 0x7f9c3c159540 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3400cde0 con 0x7f9c3c159540 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0b2be0 con 0x7f9c3c159540 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34003bc0 con 0x7f9c3c159540 2024-04-25T10:29:29.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c0b31a0 con 0x7f9c3c159540 2024-04-25T10:29:29.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c04ef30 con 0x7f9c3c159540 2024-04-25T10:29:29.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.424+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c3401b070 con 0x7f9c3c159540 2024-04-25T10:29:29.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.428+0000 7f9c39600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9c1c066f60 0x7f9c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:29.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.428+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c34085af0 con 0x7f9c3c159540 2024-04-25T10:29:29.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.428+0000 7f9c41c00640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9c1c066f60 0x7f9c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:29.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.428+0000 7f9c41c00640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9c1c066f60 0x7f9c1c069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9c300af000 tx=0x7f9c300a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:29.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.428+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c34022070 con 0x7f9c3c159540 2024-04-25T10:29:29.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:29.624+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9c3c0cf470 con 0x7f9c3c159540 2024-04-25T10:29:29.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:29.946 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:29:30.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c39600640 1 -- 172.21.15.45:0/3813496869 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v115) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c34010040 con 0x7f9c3c159540 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9c1c066f60 msgr2=0x7f9c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9c1c066f60 0x7f9c1c069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9c300af000 tx=0x7f9c300a7000 comp rx=0 tx=0).stop 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 msgr2=0x7f9c3c0b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 0x7f9c3c0b3c80 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400efc0 tx=0x7f9c34009f90 comp rx=0 tx=0).stop 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 shutdown_connections 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9c1c066f60 0x7f9c1c069420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c3c159540 0x7f9c3c0b3c80 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c3c0aebd0 0x7f9c3c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 --2- 172.21.15.45:0/3813496869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c3c0721f0 0x7f9c3c14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 >> 172.21.15.45:0/3813496869 conn(0x7f9c3c0669a0 msgr2=0x7f9c3c0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 shutdown_connections 2024-04-25T10:29:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.240+0000 7f9c44600640 1 -- 172.21.15.45:0/3813496869 wait complete. 2024-04-25T10:29:30.265 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:29:30.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.404+0000 7f9658000640 1 Processor -- start 2024-04-25T10:29:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.404+0000 7f9658000640 1 -- start start 2024-04-25T10:29:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9658000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f965015a0e0 0x7f965015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9658000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9650071f70 0x7f96500b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9658000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965015b6e0 con 0x7f96500b2fc0 2024-04-25T10:29:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9658000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9650070ed0 con 0x7f9650071f70 2024-04-25T10:29:30.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9658000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96500711b0 con 0x7f965015a0e0 2024-04-25T10:29:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9654c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9650071f70 0x7f96500b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9656000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f96500b2fc0 0x7f965006c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60948/0 (socket says 172.21.15.45:60948) 2024-04-25T10:29:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9654c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9650071f70 0x7f96500b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56810/0 (socket says 172.21.15.45:56810) 2024-04-25T10:29:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9656000640 1 -- 172.21.15.45:0/634982047 learned_addr learned my addr 172.21.15.45:0/634982047 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9655600640 1 --2- 172.21.15.45:0/634982047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f965015a0e0 0x7f965015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9654c00640 1 -- 172.21.15.45:0/634982047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f965015a0e0 msgr2=0x7f965015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9654c00640 1 --2- 172.21.15.45:0/634982047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f965015a0e0 0x7f965015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9654c00640 1 -- 172.21.15.45:0/634982047 >> v1:172.21.15.45:6789/0 conn(0x7f96500b2fc0 legacy=0x7f965006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f9654c00640 1 -- 172.21.15.45:0/634982047 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650071490 con 0x7f9650071f70 2024-04-25T10:29:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.408+0000 7f964fe00640 1 -- 172.21.15.45:0/634982047 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164223595 0 0) 0x7f96480014a0 con 0x7f96500b2fc0 2024-04-25T10:29:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9654c00640 1 --2- 172.21.15.45:0/634982047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650071f70 0x7f96500b2a80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f963c0097c0 tx=0x7f963c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2147f712f23c8848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f964fe00640 1 -- 172.21.15.45:0/634982047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0b0070 con 0x7f9650071f70 2024-04-25T10:29:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f964fe00640 1 -- 172.21.15.45:0/634982047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c0ab600 con 0x7f9650071f70 2024-04-25T10:29:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f964fe00640 1 -- 172.21.15.45:0/634982047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0ab9d0 con 0x7f9650071f70 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 -- 172.21.15.45:0/634982047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650071f70 msgr2=0x7f96500b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 --2- 172.21.15.45:0/634982047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650071f70 0x7f96500b2a80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f963c0097c0 tx=0x7f963c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 -- 172.21.15.45:0/634982047 shutdown_connections 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 --2- 172.21.15.45:0/634982047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9650071f70 0x7f96500b2a80 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 --2- 172.21.15.45:0/634982047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f965015a0e0 0x7f965015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 -- 172.21.15.45:0/634982047 >> 172.21.15.45:0/634982047 conn(0x7f9650066b70 msgr2=0x7f9650066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 -- 172.21.15.45:0/634982047 shutdown_connections 2024-04-25T10:29:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 -- 172.21.15.45:0/634982047 wait complete. 2024-04-25T10:29:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 Processor -- start 2024-04-25T10:29:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.412+0000 7f9658000640 1 -- start start 2024-04-25T10:29:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650071f70 0x7f96501581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96500b2fc0 0x7f965014a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 0x7f965014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965015a4c0 con 0x7f965015a0e0 2024-04-25T10:29:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96500af030 con 0x7f9650071f70 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9650070b50 con 0x7f96500b2fc0 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 0x7f965014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 0x7f965014fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56816/0 (socket says 172.21.15.45:56816) 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 -- 172.21.15.45:0/3030204296 learned_addr learned my addr 172.21.15.45:0/3030204296 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9655600640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650071f70 0x7f96501581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9654c00640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96500b2fc0 0x7f965014a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 -- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96500b2fc0 msgr2=0x7f965014a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96500b2fc0 0x7f965014a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 -- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650071f70 msgr2=0x7f96501581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650071f70 0x7f96501581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 -- 172.21.15.45:0/3030204296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650158700 con 0x7f965015a0e0 2024-04-25T10:29:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9656000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 0x7f965014fba0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f964800c7f0 tx=0x7f964800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648018070 con 0x7f965015a0e0 2024-04-25T10:29:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800a640 con 0x7f965015a0e0 2024-04-25T10:29:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800aa30 con 0x7f965015a0e0 2024-04-25T10:29:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96501590f0 con 0x7f965015a0e0 2024-04-25T10:29:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501576e0 con 0x7f965015a0e0 2024-04-25T10:29:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965004ef30 con 0x7f965015a0e0 2024-04-25T10:29:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.416+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f964801b070 con 0x7f965015a0e0 2024-04-25T10:29:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.420+0000 7f964d600640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9630066f60 0x7f9630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.420+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9648084660 con 0x7f965015a0e0 2024-04-25T10:29:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.420+0000 7f9655600640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9630066f60 0x7f9630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.420+0000 7f9655600640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9630066f60 0x7f9630069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f96440079c0 tx=0x7f9644008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.424+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f965015b6e0 con 0x7f965015a0e0 2024-04-25T10:29:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96500703e0 con 0x7f965015a0e0 2024-04-25T10:29:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f964d600640 1 -- 172.21.15.45:0/3030204296 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f9648054f80 con 0x7f965015a0e0 2024-04-25T10:29:30.621 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:30.621 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:26.603160+0000","modified":"2024-04-25T10:29:29.629298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9996":{"gid":9996,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2482982951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2482982951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2482982951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:29:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-04-25T10:29:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9630066f60 msgr2=0x7f9630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9630066f60 0x7f9630069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f96440079c0 tx=0x7f9644008040 comp rx=0 tx=0).stop 2024-04-25T10:29:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 msgr2=0x7f965014fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 0x7f965014fba0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f964800c7f0 tx=0x7f964800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 shutdown_connections 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9630066f60 0x7f9630069420 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f965015a0e0 0x7f965014fba0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96500b2fc0 0x7f965014a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 --2- 172.21.15.45:0/3030204296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9650071f70 0x7f96501581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 >> 172.21.15.45:0/3030204296 conn(0x7f9650066b70 msgr2=0x7f965015bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 shutdown_connections 2024-04-25T10:29:30.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.620+0000 7f9658000640 1 -- 172.21.15.45:0/3030204296 wait complete. 2024-04-25T10:29:30.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:29:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.780+0000 7f3599c00640 1 Processor -- start 2024-04-25T10:29:30.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.780+0000 7f3599c00640 1 -- start start 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3599c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f359415b910 0x7f359415c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3599c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359415bd60 con 0x7f35941550b0 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3599c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3594071050 con 0x7f35940d0040 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3599c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35940712f0 con 0x7f359415b910 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3592a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f35941550b0 0x7f35940cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60950/0 (socket says 172.21.15.45:60950) 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3592a00640 1 -- 172.21.15.45:0/584070719 learned_addr learned my addr 172.21.15.45:0/584070719 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:30.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3593400640 1 --2- 172.21.15.45:0/584070719 >> v2:172.21.15.117:3300/0 conn(0x7f359415b910 0x7f359415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3592000640 1 -- 172.21.15.45:0/584070719 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060927674 0 0) 0x7f359415bd60 con 0x7f35941550b0 2024-04-25T10:29:30.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3592000640 1 -- 172.21.15.45:0/584070719 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578003530 con 0x7f35941550b0 2024-04-25T10:29:30.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3593400640 1 -- 172.21.15.45:0/584070719 >> v1:172.21.15.117:6789/0 conn(0x7f35940d0040 legacy=0x7f359415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3593400640 1 -- 172.21.15.45:0/584070719 >> v1:172.21.15.45:6789/0 conn(0x7f35941550b0 legacy=0x7f35940cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.784+0000 7f3593400640 1 -- 172.21.15.45:0/584070719 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940715d0 con 0x7f359415b910 2024-04-25T10:29:30.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3593400640 1 --2- 172.21.15.45:0/584070719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f359415c030 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f358c009d90 tx=0x7f358c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312b8d737ff9b39f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3592000640 1 -- 172.21.15.45:0/584070719 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0b2070 con 0x7f359415b910 2024-04-25T10:29:30.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3592000640 1 -- 172.21.15.45:0/584070719 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c0a5a00 con 0x7f359415b910 2024-04-25T10:29:30.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3592000640 1 -- 172.21.15.45:0/584070719 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0a5dd0 con 0x7f359415b910 2024-04-25T10:29:30.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 -- 172.21.15.45:0/584070719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 msgr2=0x7f359415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 --2- 172.21.15.45:0/584070719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f359415c030 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f358c009d90 tx=0x7f358c01c940 comp rx=0 tx=0).stop 2024-04-25T10:29:30.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 -- 172.21.15.45:0/584070719 shutdown_connections 2024-04-25T10:29:30.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 --2- 172.21.15.45:0/584070719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f359415c030 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 -- 172.21.15.45:0/584070719 >> 172.21.15.45:0/584070719 conn(0x7f3594066b70 msgr2=0x7f3594066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 -- 172.21.15.45:0/584070719 shutdown_connections 2024-04-25T10:29:30.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 -- 172.21.15.45:0/584070719 wait complete. 2024-04-25T10:29:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 Processor -- start 2024-04-25T10:29:30.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.788+0000 7f3599c00640 1 -- start start 2024-04-25T10:29:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35940d0040 0x7f35940ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 0x7f359414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f35940cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359415cf00 con 0x7f35941550b0 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359415cd80 con 0x7f359415b910 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359415d080 con 0x7f35940d0040 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 0x7f359414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 0x7f359414b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56828/0 (socket says 172.21.15.45:56828) 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 -- 172.21.15.45:0/3479061472 learned_addr learned my addr 172.21.15.45:0/3479061472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3593400640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f35940cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3592a00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35940d0040 0x7f35940ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 -- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35940d0040 msgr2=0x7f35940ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35940d0040 0x7f35940ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 -- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 msgr2=0x7f35940cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f35940cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 -- 172.21.15.45:0/3479061472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359414b7a0 con 0x7f35941550b0 2024-04-25T10:29:30.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3593400640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f35940cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358be00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 0x7f359414b260 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f357c00c370 tx=0x7f357c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c0052d0 con 0x7f35941550b0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c005470 con 0x7f35941550b0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35940cd4a0 con 0x7f35941550b0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c007260 con 0x7f35941550b0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35940cda60 con 0x7f35941550b0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.792+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359404ef30 con 0x7f35941550b0 2024-04-25T10:29:30.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.796+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f357c007400 con 0x7f35941550b0 2024-04-25T10:29:30.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.796+0000 7f358aa00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3570066f60 0x7f3570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:30.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.796+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f357c084bd0 con 0x7f35941550b0 2024-04-25T10:29:30.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.796+0000 7f3592a00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3570066f60 0x7f3570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:30.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.796+0000 7f3592a00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3570066f60 0x7f3570069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f35840011f0 tx=0x7f3584009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.800+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f357c055d50 con 0x7f35941550b0 2024-04-25T10:29:30.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.964+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35940cfc80 con 0x7f35941550b0 2024-04-25T10:29:30.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f358aa00640 1 -- 172.21.15.45:0/3479061472 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f357c055650 con 0x7f35941550b0 2024-04-25T10:29:30.968 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:30.968 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":155,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:28.223091+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:29:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3570066f60 msgr2=0x7f3570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3570066f60 0x7f3570069420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f35840011f0 tx=0x7f3584009040 comp rx=0 tx=0).stop 2024-04-25T10:29:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 msgr2=0x7f359414b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:30.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 0x7f359414b260 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f357c00c370 tx=0x7f357c00c830 comp rx=0 tx=0).stop 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 shutdown_connections 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3570066f60 0x7f3570069420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f359415b910 0x7f35940cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35941550b0 0x7f359414b260 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 --2- 172.21.15.45:0/3479061472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35940d0040 0x7f35940ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 >> 172.21.15.45:0/3479061472 conn(0x7f3594066b70 msgr2=0x7f359406ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 shutdown_connections 2024-04-25T10:29:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:30.968+0000 7f3599c00640 1 -- 172.21.15.45:0/3479061472 wait complete. 2024-04-25T10:29:30.990 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:29:30.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:31.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 Processor -- start 2024-04-25T10:29:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 -- start start 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f92cc0cf6f0 0x7f92cc0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92cc0b1310 con 0x7f92cc0d0430 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0ae5a0 con 0x7f92cc0cf6f0 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92d1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc15c1b0 con 0x7f92cc15bdd0 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92cb200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f92cc0d0430 0x7f92cc0cc520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60962/0 (socket says 172.21.15.45:60962) 2024-04-25T10:29:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92cb200640 1 -- 172.21.15.45:0/3062193843 learned_addr learned my addr 172.21.15.45:0/3062193843 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c3e00640 1 --2- 172.21.15.45:0/3062193843 >> v2:172.21.15.117:3300/0 conn(0x7f92cc0cf6f0 0x7f92cc0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92ca800640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660578070 0 0) 0x7f92cc0b1310 con 0x7f92cc0d0430 2024-04-25T10:29:31.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92b0003530 con 0x7f92cc0d0430 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034839130 0 0) 0x7f92b0003530 con 0x7f92cc0d0430 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 msgr2=0x7f92cc0d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 >> v2:172.21.15.117:3300/0 conn(0x7f92cc0cf6f0 msgr2=0x7f92cc0cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 --2- 172.21.15.45:0/3062193843 >> v2:172.21.15.117:3300/0 conn(0x7f92cc0cf6f0 0x7f92cc0cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0b15f0 con 0x7f92cc0d0430 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f92c4001a00 con 0x7f92cc0d0430 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 >> v1:172.21.15.45:6789/0 conn(0x7f92cc0d0430 legacy=0x7f92cc0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0004d50 0x7f92b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc15c1b0 con 0x7f92b0004d50 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b000b9d0 con 0x7f92b00039c0 2024-04-25T10:29:31.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b000bb50 con 0x7f92cc15bdd0 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.132+0000 7f92cb200640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0004d50 0x7f92b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92c3e00640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 -- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 msgr2=0x7f92b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 -- 172.21.15.45:0/3062193843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0004d50 msgr2=0x7f92b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0004d50 0x7f92b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 -- 172.21.15.45:0/3062193843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0003530 con 0x7f92b00039c0 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92c3e00640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc15bdd0 0x7f92b0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:31.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92cb200640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92b0004d50 0x7f92b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92ca800640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92b0003de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009ae0 tx=0x7f92bc01ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f2e5f537e68777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0b2070 con 0x7f92b00039c0 2024-04-25T10:29:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc0a5660 con 0x7f92b00039c0 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.136+0000 7f92c9e00640 1 -- 172.21.15.45:0/3062193843 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0096b0 con 0x7f92b00039c0 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 msgr2=0x7f92b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 --2- 172.21.15.45:0/3062193843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92b0003de0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009ae0 tx=0x7f92bc01ca70 comp rx=0 tx=0).stop 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92cb200640 1 -- 172.21.15.45:0/3062193843 reap_dead start 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- 172.21.15.45:0/3062193843 shutdown_connections 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- 172.21.15.45:0/3062193843 >> 172.21.15.45:0/3062193843 conn(0x7f92cc0d1910 msgr2=0x7f92cc0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- 172.21.15.45:0/3062193843 shutdown_connections 2024-04-25T10:29:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- 172.21.15.45:0/3062193843 wait complete. 2024-04-25T10:29:31.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 Processor -- start 2024-04-25T10:29:31.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- start start 2024-04-25T10:29:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92cc1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 0x7f92cc1f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc1f4bb0 0x7f92cc1f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0cfbd0 con 0x7f92cc1f4730 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0b1c40 con 0x7f92b00039c0 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92d1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0cd860 con 0x7f92cc1f4bb0 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 0x7f92cc1f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 0x7f92cc1f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56842/0 (socket says 172.21.15.45:56842) 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 -- 172.21.15.45:0/207975261 learned_addr learned my addr 172.21.15.45:0/207975261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92ca800640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92cc1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92cb200640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc1f4bb0 0x7f92cc1f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 -- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc1f4bb0 msgr2=0x7f92cc1f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc1f4bb0 0x7f92cc1f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 -- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 msgr2=0x7f92cc1f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92cc1f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 -- 172.21.15.45:0/207975261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc1f5eb0 con 0x7f92cc1f4730 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92cb200640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc1f4bb0 0x7f92cc1f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.140+0000 7f92c3e00640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 0x7f92cc1f5040 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f92b4011ca0 tx=0x7f92b400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4011e50 con 0x7f92cc1f4730 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4004020 con 0x7f92cc1f4730 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4004410 con 0x7f92cc1f4730 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc1f30a0 con 0x7f92cc1f4730 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc0c8c30 con 0x7f92cc1f4730 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc04ef30 con 0x7f92cc1f4730 2024-04-25T10:29:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92b4012040 con 0x7f92cc1f4730 2024-04-25T10:29:31.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92c2a00640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f60 0x7f92a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92b40854e0 con 0x7f92cc1f4730 2024-04-25T10:29:31.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.144+0000 7f92ca800640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f60 0x7f92a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.148+0000 7f92ca800640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f60 0x7f92a8069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0af000 tx=0x7f92bc0046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.148+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92b4019030 con 0x7f92cc1f4730 2024-04-25T10:29:31.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.312+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92cc0b3dd0 con 0x7f92a8066f60 2024-04-25T10:29:31.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.320+0000 7f92c2a00640 1 -- 172.21.15.45:0/207975261 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252474 (secure 0 0 0) 0x7f92cc0b3dd0 con 0x7f92a8066f60 2024-04-25T10:29:31.323 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f60 msgr2=0x7f92a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f60 0x7f92a8069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0af000 tx=0x7f92bc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 msgr2=0x7f92cc1f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 0x7f92cc1f5040 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f92b4011ca0 tx=0x7f92b400f610 comp rx=0 tx=0).stop 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 shutdown_connections 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f60 0x7f92a8069420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92cc1f4bb0 0x7f92cc1f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92cc1f4730 0x7f92cc1f5040 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 --2- 172.21.15.45:0/207975261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92b00039c0 0x7f92cc1f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 >> 172.21.15.45:0/207975261 conn(0x7f92cc0d1910 msgr2=0x7f92cc0b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:31.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 shutdown_connections 2024-04-25T10:29:31.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.324+0000 7f92d1400640 1 -- 172.21.15.45:0/207975261 wait complete. 2024-04-25T10:29:31.343 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-04-25T10:29:31.035111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432320,"kb_used_data":16320,"kb_used_omap":12,"kb_used_meta":415859,"kb_avail":754542400,"statfs":{"total":773094113280,"available":772651417600,"internally_reserved":0,"allocated":16711680,"data_stored":7951203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":425840176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.835035"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237520+0000","last_change":"2024-04-25T10:29:27.248081+0000","last_active":"2024-04-25T10:29:28.237520+0000","last_peered":"2024-04-25T10:29:28.237520+0000","last_clean":"2024-04-25T10:29:28.237520+0000","last_became_active":"2024-04-25T10:29:27.247768+0000","last_became_peered":"2024-04-25T10:29:27.247768+0000","last_unstale":"2024-04-25T10:29:28.237520+0000","last_undegraded":"2024-04-25T10:29:28.237520+0000","last_fullsized":"2024-04-25T10:29:28.237520+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:16.073626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237545+0000","last_change":"2024-04-25T10:29:27.248087+0000","last_active":"2024-04-25T10:29:28.237545+0000","last_peered":"2024-04-25T10:29:28.237545+0000","last_clean":"2024-04-25T10:29:28.237545+0000","last_became_active":"2024-04-25T10:29:27.247814+0000","last_became_peered":"2024-04-25T10:29:27.247814+0000","last_unstale":"2024-04-25T10:29:28.237545+0000","last_undegraded":"2024-04-25T10:29:28.237545+0000","last_fullsized":"2024-04-25T10:29:28.237545+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:26.351411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237219+0000","last_change":"2024-04-25T10:29:27.231026+0000","last_active":"2024-04-25T10:29:28.237219+0000","last_peered":"2024-04-25T10:29:28.237219+0000","last_clean":"2024-04-25T10:29:28.237219+0000","last_became_active":"2024-04-25T10:29:27.230711+0000","last_became_peered":"2024-04-25T10:29:27.230711+0000","last_unstale":"2024-04-25T10:29:28.237219+0000","last_undegraded":"2024-04-25T10:29:28.237219+0000","last_fullsized":"2024-04-25T10:29:28.237219+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:46.545736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.244966+0000","last_change":"2024-04-25T10:29:27.242022+0000","last_active":"2024-04-25T10:29:27.244966+0000","last_peered":"2024-04-25T10:29:27.244966+0000","last_clean":"2024-04-25T10:29:27.244966+0000","last_became_active":"2024-04-25T10:29:27.240586+0000","last_became_peered":"2024-04-25T10:29:27.240586+0000","last_unstale":"2024-04-25T10:29:27.244966+0000","last_undegraded":"2024-04-25T10:29:27.244966+0000","last_fullsized":"2024-04-25T10:29:27.244966+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:38.516858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233051+0000","last_change":"2024-04-25T10:29:27.243305+0000","last_active":"2024-04-25T10:29:28.233051+0000","last_peered":"2024-04-25T10:29:28.233051+0000","last_clean":"2024-04-25T10:29:28.233051+0000","last_became_active":"2024-04-25T10:29:27.243002+0000","last_became_peered":"2024-04-25T10:29:27.243002+0000","last_unstale":"2024-04-25T10:29:28.233051+0000","last_undegraded":"2024-04-25T10:29:28.233051+0000","last_fullsized":"2024-04-25T10:29:28.233051+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:43.215658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315707+0000","last_change":"2024-04-25T10:29:27.228343+0000","last_active":"2024-04-25T10:29:28.315707+0000","last_peered":"2024-04-25T10:29:28.315707+0000","last_clean":"2024-04-25T10:29:28.315707+0000","last_became_active":"2024-04-25T10:29:27.228030+0000","last_became_peered":"2024-04-25T10:29:27.228030+0000","last_unstale":"2024-04-25T10:29:28.315707+0000","last_undegraded":"2024-04-25T10:29:28.315707+0000","last_fullsized":"2024-04-25T10:29:28.315707+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:41.083508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316387+0000","last_change":"2024-04-25T10:29:27.249589+0000","last_active":"2024-04-25T10:29:28.316387+0000","last_peered":"2024-04-25T10:29:28.316387+0000","last_clean":"2024-04-25T10:29:28.316387+0000","last_became_active":"2024-04-25T10:29:27.249282+0000","last_became_peered":"2024-04-25T10:29:27.249282+0000","last_unstale":"2024-04-25T10:29:28.316387+0000","last_undegraded":"2024-04-25T10:29:28.316387+0000","last_fullsized":"2024-04-25T10:29:28.316387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:38.864445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312494+0000","last_change":"2024-04-25T10:29:27.241481+0000","last_active":"2024-04-25T10:29:28.312494+0000","last_peered":"2024-04-25T10:29:28.312494+0000","last_clean":"2024-04-25T10:29:28.312494+0000","last_became_active":"2024-04-25T10:29:27.240946+0000","last_became_peered":"2024-04-25T10:29:27.240946+0000","last_unstale":"2024-04-25T10:29:28.312494+0000","last_undegraded":"2024-04-25T10:29:28.312494+0000","last_fullsized":"2024-04-25T10:29:28.312494+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:41.124396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312941+0000","last_change":"2024-04-25T10:29:27.245541+0000","last_active":"2024-04-25T10:29:28.312941+0000","last_peered":"2024-04-25T10:29:28.312941+0000","last_clean":"2024-04-25T10:29:28.312941+0000","last_became_active":"2024-04-25T10:29:27.245124+0000","last_became_peered":"2024-04-25T10:29:27.245124+0000","last_unstale":"2024-04-25T10:29:28.312941+0000","last_undegraded":"2024-04-25T10:29:28.312941+0000","last_fullsized":"2024-04-25T10:29:28.312941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:26.418130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312878+0000","last_change":"2024-04-25T10:29:27.240287+0000","last_active":"2024-04-25T10:29:28.312878+0000","last_peered":"2024-04-25T10:29:28.312878+0000","last_clean":"2024-04-25T10:29:28.312878+0000","last_became_active":"2024-04-25T10:29:27.239984+0000","last_became_peered":"2024-04-25T10:29:27.239984+0000","last_unstale":"2024-04-25T10:29:28.312878+0000","last_undegraded":"2024-04-25T10:29:28.312878+0000","last_fullsized":"2024-04-25T10:29:28.312878+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:23.732931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233383+0000","last_change":"2024-04-25T10:29:27.233684+0000","last_active":"2024-04-25T10:29:28.233383+0000","last_peered":"2024-04-25T10:29:28.233383+0000","last_clean":"2024-04-25T10:29:28.233383+0000","last_became_active":"2024-04-25T10:29:27.233326+0000","last_became_peered":"2024-04-25T10:29:27.233326+0000","last_unstale":"2024-04-25T10:29:28.233383+0000","last_undegraded":"2024-04-25T10:29:28.233383+0000","last_fullsized":"2024-04-25T10:29:28.233383+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.942026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236894+0000","last_change":"2024-04-25T10:29:27.233900+0000","last_active":"2024-04-25T10:29:28.236894+0000","last_peered":"2024-04-25T10:29:28.236894+0000","last_clean":"2024-04-25T10:29:28.236894+0000","last_became_active":"2024-04-25T10:29:27.233637+0000","last_became_peered":"2024-04-25T10:29:27.233637+0000","last_unstale":"2024-04-25T10:29:28.236894+0000","last_undegraded":"2024-04-25T10:29:28.236894+0000","last_fullsized":"2024-04-25T10:29:28.236894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:26.940558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236465+0000","last_change":"2024-04-25T10:29:27.222766+0000","last_active":"2024-04-25T10:29:28.236465+0000","last_peered":"2024-04-25T10:29:28.236465+0000","last_clean":"2024-04-25T10:29:28.236465+0000","last_became_active":"2024-04-25T10:29:27.222497+0000","last_became_peered":"2024-04-25T10:29:27.222497+0000","last_unstale":"2024-04-25T10:29:28.236465+0000","last_undegraded":"2024-04-25T10:29:28.236465+0000","last_fullsized":"2024-04-25T10:29:28.236465+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:31.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312494+0000","last_change":"2024-04-25T10:29:27.248269+0000","last_active":"2024-04-25T10:29:28.312494+0000","last_peered":"2024-04-25T10:29:28.312494+0000","last_clean":"2024-04-25T10:29:28.312494+0000","last_became_active":"2024-04-25T10:29:27.247993+0000","last_became_peered":"2024-04-25T10:29:27.247993+0000","last_unstale":"2024-04-25T10:29:28.312494+0000","last_undegraded":"2024-04-25T10:29:28.312494+0000","last_fullsized":"2024-04-25T10:29:28.312494+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:17.450769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312503+0000","last_change":"2024-04-25T10:29:27.239149+0000","last_active":"2024-04-25T10:29:28.312503+0000","last_peered":"2024-04-25T10:29:28.312503+0000","last_clean":"2024-04-25T10:29:28.312503+0000","last_became_active":"2024-04-25T10:29:27.238865+0000","last_became_peered":"2024-04-25T10:29:27.238865+0000","last_unstale":"2024-04-25T10:29:28.312503+0000","last_undegraded":"2024-04-25T10:29:28.312503+0000","last_fullsized":"2024-04-25T10:29:28.312503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:49.946175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315254+0000","last_change":"2024-04-25T10:29:27.235646+0000","last_active":"2024-04-25T10:29:28.315254+0000","last_peered":"2024-04-25T10:29:28.315254+0000","last_clean":"2024-04-25T10:29:28.315254+0000","last_became_active":"2024-04-25T10:29:27.234069+0000","last_became_peered":"2024-04-25T10:29:27.234069+0000","last_unstale":"2024-04-25T10:29:28.315254+0000","last_undegraded":"2024-04-25T10:29:28.315254+0000","last_fullsized":"2024-04-25T10:29:28.315254+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316514+0000","last_change":"2024-04-25T10:29:27.226984+0000","last_active":"2024-04-25T10:29:28.316514+0000","last_peered":"2024-04-25T10:29:28.316514+0000","last_clean":"2024-04-25T10:29:28.316514+0000","last_became_active":"2024-04-25T10:29:27.226687+0000","last_became_peered":"2024-04-25T10:29:27.226687+0000","last_unstale":"2024-04-25T10:29:28.316514+0000","last_undegraded":"2024-04-25T10:29:28.316514+0000","last_fullsized":"2024-04-25T10:29:28.316514+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:30.100235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.248005+0000","last_change":"2024-04-25T10:29:27.243411+0000","last_active":"2024-04-25T10:29:27.248005+0000","last_peered":"2024-04-25T10:29:27.248005+0000","last_clean":"2024-04-25T10:29:27.248005+0000","last_became_active":"2024-04-25T10:29:27.242969+0000","last_became_peered":"2024-04-25T10:29:27.242969+0000","last_unstale":"2024-04-25T10:29:27.248005+0000","last_undegraded":"2024-04-25T10:29:27.248005+0000","last_fullsized":"2024-04-25T10:29:27.248005+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:03.199889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316281+0000","last_change":"2024-04-25T10:29:27.239540+0000","last_active":"2024-04-25T10:29:28.316281+0000","last_peered":"2024-04-25T10:29:28.316281+0000","last_clean":"2024-04-25T10:29:28.316281+0000","last_became_active":"2024-04-25T10:29:27.239269+0000","last_became_peered":"2024-04-25T10:29:27.239269+0000","last_unstale":"2024-04-25T10:29:28.316281+0000","last_undegraded":"2024-04-25T10:29:28.316281+0000","last_fullsized":"2024-04-25T10:29:28.316281+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.802262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236806+0000","last_change":"2024-04-25T10:29:27.224361+0000","last_active":"2024-04-25T10:29:28.236806+0000","last_peered":"2024-04-25T10:29:28.236806+0000","last_clean":"2024-04-25T10:29:28.236806+0000","last_became_active":"2024-04-25T10:29:27.224079+0000","last_became_peered":"2024-04-25T10:29:27.224079+0000","last_unstale":"2024-04-25T10:29:28.236806+0000","last_undegraded":"2024-04-25T10:29:28.236806+0000","last_fullsized":"2024-04-25T10:29:28.236806+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:46.578402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315715+0000","last_change":"2024-04-25T10:29:27.235569+0000","last_active":"2024-04-25T10:29:28.315715+0000","last_peered":"2024-04-25T10:29:28.315715+0000","last_clean":"2024-04-25T10:29:28.315715+0000","last_became_active":"2024-04-25T10:29:27.235290+0000","last_became_peered":"2024-04-25T10:29:27.235290+0000","last_unstale":"2024-04-25T10:29:28.315715+0000","last_undegraded":"2024-04-25T10:29:28.315715+0000","last_fullsized":"2024-04-25T10:29:28.315715+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:58.737894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232931+0000","last_change":"2024-04-25T10:29:27.226345+0000","last_active":"2024-04-25T10:29:28.232931+0000","last_peered":"2024-04-25T10:29:28.232931+0000","last_clean":"2024-04-25T10:29:28.232931+0000","last_became_active":"2024-04-25T10:29:27.226067+0000","last_became_peered":"2024-04-25T10:29:27.226067+0000","last_unstale":"2024-04-25T10:29:28.232931+0000","last_undegraded":"2024-04-25T10:29:28.232931+0000","last_fullsized":"2024-04-25T10:29:28.232931+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:38.159997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.239736+0000","last_change":"2024-04-25T10:29:27.224474+0000","last_active":"2024-04-25T10:29:28.239736+0000","last_peered":"2024-04-25T10:29:28.239736+0000","last_clean":"2024-04-25T10:29:28.239736+0000","last_became_active":"2024-04-25T10:29:27.224194+0000","last_became_peered":"2024-04-25T10:29:27.224194+0000","last_unstale":"2024-04-25T10:29:28.239736+0000","last_undegraded":"2024-04-25T10:29:28.239736+0000","last_fullsized":"2024-04-25T10:29:28.239736+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232747+0000","last_change":"2024-04-25T10:29:27.246389+0000","last_active":"2024-04-25T10:29:28.232747+0000","last_peered":"2024-04-25T10:29:28.232747+0000","last_clean":"2024-04-25T10:29:28.232747+0000","last_became_active":"2024-04-25T10:29:27.240755+0000","last_became_peered":"2024-04-25T10:29:27.240755+0000","last_unstale":"2024-04-25T10:29:28.232747+0000","last_undegraded":"2024-04-25T10:29:28.232747+0000","last_fullsized":"2024-04-25T10:29:28.232747+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:30.699147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233414+0000","last_change":"2024-04-25T10:29:27.247107+0000","last_active":"2024-04-25T10:29:28.233414+0000","last_peered":"2024-04-25T10:29:28.233414+0000","last_clean":"2024-04-25T10:29:28.233414+0000","last_became_active":"2024-04-25T10:29:27.246803+0000","last_became_peered":"2024-04-25T10:29:27.246803+0000","last_unstale":"2024-04-25T10:29:28.233414+0000","last_undegraded":"2024-04-25T10:29:28.233414+0000","last_fullsized":"2024-04-25T10:29:28.233414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:59.983279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240490+0000","last_change":"2024-04-25T10:29:27.232793+0000","last_active":"2024-04-25T10:29:28.240490+0000","last_peered":"2024-04-25T10:29:28.240490+0000","last_clean":"2024-04-25T10:29:28.240490+0000","last_became_active":"2024-04-25T10:29:27.232340+0000","last_became_peered":"2024-04-25T10:29:27.232340+0000","last_unstale":"2024-04-25T10:29:28.240490+0000","last_undegraded":"2024-04-25T10:29:28.240490+0000","last_fullsized":"2024-04-25T10:29:28.240490+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:16.095428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233496+0000","last_change":"2024-04-25T10:29:27.233105+0000","last_active":"2024-04-25T10:29:28.233496+0000","last_peered":"2024-04-25T10:29:28.233496+0000","last_clean":"2024-04-25T10:29:28.233496+0000","last_became_active":"2024-04-25T10:29:27.232016+0000","last_became_peered":"2024-04-25T10:29:27.232016+0000","last_unstale":"2024-04-25T10:29:28.233496+0000","last_undegraded":"2024-04-25T10:29:28.233496+0000","last_fullsized":"2024-04-25T10:29:28.233496+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:22.192028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315662+0000","last_change":"2024-04-25T10:29:27.238198+0000","last_active":"2024-04-25T10:29:28.315662+0000","last_peered":"2024-04-25T10:29:28.315662+0000","last_clean":"2024-04-25T10:29:28.315662+0000","last_became_active":"2024-04-25T10:29:27.237918+0000","last_became_peered":"2024-04-25T10:29:27.237918+0000","last_unstale":"2024-04-25T10:29:28.315662+0000","last_undegraded":"2024-04-25T10:29:28.315662+0000","last_fullsized":"2024-04-25T10:29:28.315662+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.436622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236952+0000","last_change":"2024-04-25T10:29:27.225093+0000","last_active":"2024-04-25T10:29:28.236952+0000","last_peered":"2024-04-25T10:29:28.236952+0000","last_clean":"2024-04-25T10:29:28.236952+0000","last_became_active":"2024-04-25T10:29:27.224813+0000","last_became_peered":"2024-04-25T10:29:27.224813+0000","last_unstale":"2024-04-25T10:29:28.236952+0000","last_undegraded":"2024-04-25T10:29:28.236952+0000","last_fullsized":"2024-04-25T10:29:28.236952+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:01.145754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312581+0000","last_change":"2024-04-25T10:29:27.249124+0000","last_active":"2024-04-25T10:29:28.312581+0000","last_peered":"2024-04-25T10:29:28.312581+0000","last_clean":"2024-04-25T10:29:28.312581+0000","last_became_active":"2024-04-25T10:29:27.246171+0000","last_became_peered":"2024-04-25T10:29:27.246171+0000","last_unstale":"2024-04-25T10:29:28.312581+0000","last_undegraded":"2024-04-25T10:29:28.312581+0000","last_fullsized":"2024-04-25T10:29:28.312581+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:46.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236665+0000","last_change":"2024-04-25T10:29:27.239619+0000","last_active":"2024-04-25T10:29:28.236665+0000","last_peered":"2024-04-25T10:29:28.236665+0000","last_clean":"2024-04-25T10:29:28.236665+0000","last_became_active":"2024-04-25T10:29:27.239303+0000","last_became_peered":"2024-04-25T10:29:27.239303+0000","last_unstale":"2024-04-25T10:29:28.236665+0000","last_undegraded":"2024-04-25T10:29:28.236665+0000","last_fullsized":"2024-04-25T10:29:28.236665+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:21.717976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315690+0000","last_change":"2024-04-25T10:29:27.241931+0000","last_active":"2024-04-25T10:29:28.315690+0000","last_peered":"2024-04-25T10:29:28.315690+0000","last_clean":"2024-04-25T10:29:28.315690+0000","last_became_active":"2024-04-25T10:29:27.241469+0000","last_became_peered":"2024-04-25T10:29:27.241469+0000","last_unstale":"2024-04-25T10:29:28.315690+0000","last_undegraded":"2024-04-25T10:29:28.315690+0000","last_fullsized":"2024-04-25T10:29:28.315690+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:56.497804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.247984+0000","last_change":"2024-04-25T10:29:27.243207+0000","last_active":"2024-04-25T10:29:27.247984+0000","last_peered":"2024-04-25T10:29:27.247984+0000","last_clean":"2024-04-25T10:29:27.247984+0000","last_became_active":"2024-04-25T10:29:27.242627+0000","last_became_peered":"2024-04-25T10:29:27.242627+0000","last_unstale":"2024-04-25T10:29:27.247984+0000","last_undegraded":"2024-04-25T10:29:27.247984+0000","last_fullsized":"2024-04-25T10:29:27.247984+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:10.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"154'4","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237997+0000","last_change":"2024-04-25T10:29:26.222270+0000","last_active":"2024-04-25T10:29:28.237997+0000","last_peered":"2024-04-25T10:29:28.237997+0000","last_clean":"2024-04-25T10:29:28.237997+0000","last_became_active":"2024-04-25T10:29:26.221733+0000","last_became_peered":"2024-04-25T10:29:26.221733+0000","last_unstale":"2024-04-25T10:29:28.237997+0000","last_undegraded":"2024-04-25T10:29:28.237997+0000","last_fullsized":"2024-04-25T10:29:28.237997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:38.604579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237979+0000","last_change":"2024-04-25T10:29:27.227435+0000","last_active":"2024-04-25T10:29:28.237979+0000","last_peered":"2024-04-25T10:29:28.237979+0000","last_clean":"2024-04-25T10:29:28.237979+0000","last_became_active":"2024-04-25T10:29:27.226826+0000","last_became_peered":"2024-04-25T10:29:27.226826+0000","last_unstale":"2024-04-25T10:29:28.237979+0000","last_undegraded":"2024-04-25T10:29:28.237979+0000","last_fullsized":"2024-04-25T10:29:28.237979+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:10.295388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312909+0000","last_change":"2024-04-25T10:29:26.217703+0000","last_active":"2024-04-25T10:29:28.312909+0000","last_peered":"2024-04-25T10:29:28.312909+0000","last_clean":"2024-04-25T10:29:28.312909+0000","last_became_active":"2024-04-25T10:29:26.217441+0000","last_became_peered":"2024-04-25T10:29:26.217441+0000","last_unstale":"2024-04-25T10:29:28.312909+0000","last_undegraded":"2024-04-25T10:29:28.312909+0000","last_fullsized":"2024-04-25T10:29:28.312909+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.283285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315732+0000","last_change":"2024-04-25T10:29:27.230550+0000","last_active":"2024-04-25T10:29:28.315732+0000","last_peered":"2024-04-25T10:29:28.315732+0000","last_clean":"2024-04-25T10:29:28.315732+0000","last_became_active":"2024-04-25T10:29:27.229291+0000","last_became_peered":"2024-04-25T10:29:27.229291+0000","last_unstale":"2024-04-25T10:29:28.315732+0000","last_undegraded":"2024-04-25T10:29:28.315732+0000","last_fullsized":"2024-04-25T10:29:28.315732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:20.949126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315858+0000","last_change":"2024-04-25T10:29:26.223811+0000","last_active":"2024-04-25T10:29:28.315858+0000","last_peered":"2024-04-25T10:29:28.315858+0000","last_clean":"2024-04-25T10:29:28.315858+0000","last_became_active":"2024-04-25T10:29:26.223494+0000","last_became_peered":"2024-04-25T10:29:26.223494+0000","last_unstale":"2024-04-25T10:29:28.315858+0000","last_undegraded":"2024-04-25T10:29:28.315858+0000","last_fullsized":"2024-04-25T10:29:28.315858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:05.410789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312910+0000","last_change":"2024-04-25T10:29:27.240278+0000","last_active":"2024-04-25T10:29:28.312910+0000","last_peered":"2024-04-25T10:29:28.312910+0000","last_clean":"2024-04-25T10:29:28.312910+0000","last_became_active":"2024-04-25T10:29:27.239965+0000","last_became_peered":"2024-04-25T10:29:27.239965+0000","last_unstale":"2024-04-25T10:29:28.312910+0000","last_undegraded":"2024-04-25T10:29:28.312910+0000","last_fullsized":"2024-04-25T10:29:28.312910+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:54.139015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240124+0000","last_change":"2024-04-25T10:29:26.212077+0000","last_active":"2024-04-25T10:29:28.240124+0000","last_peered":"2024-04-25T10:29:28.240124+0000","last_clean":"2024-04-25T10:29:28.240124+0000","last_became_active":"2024-04-25T10:29:26.211732+0000","last_became_peered":"2024-04-25T10:29:26.211732+0000","last_unstale":"2024-04-25T10:29:28.240124+0000","last_undegraded":"2024-04-25T10:29:28.240124+0000","last_fullsized":"2024-04-25T10:29:28.240124+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:53.131423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.245975+0000","last_change":"2024-04-25T10:29:27.241241+0000","last_active":"2024-04-25T10:29:27.245975+0000","last_peered":"2024-04-25T10:29:27.245975+0000","last_clean":"2024-04-25T10:29:27.245975+0000","last_became_active":"2024-04-25T10:29:27.240978+0000","last_became_peered":"2024-04-25T10:29:27.240978+0000","last_unstale":"2024-04-25T10:29:27.245975+0000","last_undegraded":"2024-04-25T10:29:27.245975+0000","last_fullsized":"2024-04-25T10:29:27.245975+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:22.860530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237058+0000","last_change":"2024-04-25T10:29:26.219489+0000","last_active":"2024-04-25T10:29:28.237058+0000","last_peered":"2024-04-25T10:29:28.237058+0000","last_clean":"2024-04-25T10:29:28.237058+0000","last_became_active":"2024-04-25T10:29:26.218116+0000","last_became_peered":"2024-04-25T10:29:26.218116+0000","last_unstale":"2024-04-25T10:29:28.237058+0000","last_undegraded":"2024-04-25T10:29:28.237058+0000","last_fullsized":"2024-04-25T10:29:28.237058+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:34.018654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315743+0000","last_change":"2024-04-25T10:29:27.245342+0000","last_active":"2024-04-25T10:29:28.315743+0000","last_peered":"2024-04-25T10:29:28.315743+0000","last_clean":"2024-04-25T10:29:28.315743+0000","last_became_active":"2024-04-25T10:29:27.244863+0000","last_became_peered":"2024-04-25T10:29:27.244863+0000","last_unstale":"2024-04-25T10:29:28.315743+0000","last_undegraded":"2024-04-25T10:29:28.315743+0000","last_fullsized":"2024-04-25T10:29:28.315743+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:16.387951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.254141+0000","last_change":"2024-04-25T10:29:26.228653+0000","last_active":"2024-04-25T10:29:27.254141+0000","last_peered":"2024-04-25T10:29:27.254141+0000","last_clean":"2024-04-25T10:29:27.254141+0000","last_became_active":"2024-04-25T10:29:26.228453+0000","last_became_peered":"2024-04-25T10:29:26.228453+0000","last_unstale":"2024-04-25T10:29:27.254141+0000","last_undegraded":"2024-04-25T10:29:27.254141+0000","last_fullsized":"2024-04-25T10:29:27.254141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:44.983715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.239364+0000","last_change":"2024-04-25T10:29:27.224009+0000","last_active":"2024-04-25T10:29:28.239364+0000","last_peered":"2024-04-25T10:29:28.239364+0000","last_clean":"2024-04-25T10:29:28.239364+0000","last_became_active":"2024-04-25T10:29:27.223721+0000","last_became_peered":"2024-04-25T10:29:27.223721+0000","last_unstale":"2024-04-25T10:29:28.239364+0000","last_undegraded":"2024-04-25T10:29:28.239364+0000","last_fullsized":"2024-04-25T10:29:28.239364+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:18.195645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236691+0000","last_change":"2024-04-25T10:29:26.217124+0000","last_active":"2024-04-25T10:29:28.236691+0000","last_peered":"2024-04-25T10:29:28.236691+0000","last_clean":"2024-04-25T10:29:28.236691+0000","last_became_active":"2024-04-25T10:29:26.216321+0000","last_became_peered":"2024-04-25T10:29:26.216321+0000","last_unstale":"2024-04-25T10:29:28.236691+0000","last_undegraded":"2024-04-25T10:29:28.236691+0000","last_fullsized":"2024-04-25T10:29:28.236691+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:53.131423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.246095+0000","last_change":"2024-04-25T10:29:27.242267+0000","last_active":"2024-04-25T10:29:27.246095+0000","last_peered":"2024-04-25T10:29:27.246095+0000","last_clean":"2024-04-25T10:29:27.246095+0000","last_became_active":"2024-04-25T10:29:27.241072+0000","last_became_peered":"2024-04-25T10:29:27.241072+0000","last_unstale":"2024-04-25T10:29:27.246095+0000","last_undegraded":"2024-04-25T10:29:27.246095+0000","last_fullsized":"2024-04-25T10:29:27.246095+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.656561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.239475+0000","last_change":"2024-04-25T10:29:26.217253+0000","last_active":"2024-04-25T10:29:28.239475+0000","last_peered":"2024-04-25T10:29:28.239475+0000","last_clean":"2024-04-25T10:29:28.239475+0000","last_became_active":"2024-04-25T10:29:26.216959+0000","last_became_peered":"2024-04-25T10:29:26.216959+0000","last_unstale":"2024-04-25T10:29:28.239475+0000","last_undegraded":"2024-04-25T10:29:28.239475+0000","last_fullsized":"2024-04-25T10:29:28.239475+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.938583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315634+0000","last_change":"2024-04-25T10:29:27.231661+0000","last_active":"2024-04-25T10:29:28.315634+0000","last_peered":"2024-04-25T10:29:28.315634+0000","last_clean":"2024-04-25T10:29:28.315634+0000","last_became_active":"2024-04-25T10:29:27.231371+0000","last_became_peered":"2024-04-25T10:29:27.231371+0000","last_unstale":"2024-04-25T10:29:28.315634+0000","last_undegraded":"2024-04-25T10:29:28.315634+0000","last_fullsized":"2024-04-25T10:29:28.315634+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:23.026428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316070+0000","last_change":"2024-04-25T10:29:26.227440+0000","last_active":"2024-04-25T10:29:28.316070+0000","last_peered":"2024-04-25T10:29:28.316070+0000","last_clean":"2024-04-25T10:29:28.316070+0000","last_became_active":"2024-04-25T10:29:26.227087+0000","last_became_peered":"2024-04-25T10:29:26.227087+0000","last_unstale":"2024-04-25T10:29:28.316070+0000","last_undegraded":"2024-04-25T10:29:28.316070+0000","last_fullsized":"2024-04-25T10:29:28.316070+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:41.068171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315809+0000","last_change":"2024-04-25T10:29:27.235142+0000","last_active":"2024-04-25T10:29:28.315809+0000","last_peered":"2024-04-25T10:29:28.315809+0000","last_clean":"2024-04-25T10:29:28.315809+0000","last_became_active":"2024-04-25T10:29:27.233457+0000","last_became_peered":"2024-04-25T10:29:27.233457+0000","last_unstale":"2024-04-25T10:29:28.315809+0000","last_undegraded":"2024-04-25T10:29:28.315809+0000","last_fullsized":"2024-04-25T10:29:28.315809+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.088588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"154'3","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240227+0000","last_change":"2024-04-25T10:29:26.216000+0000","last_active":"2024-04-25T10:29:28.240227+0000","last_peered":"2024-04-25T10:29:28.240227+0000","last_clean":"2024-04-25T10:29:28.240227+0000","last_became_active":"2024-04-25T10:29:26.215762+0000","last_became_peered":"2024-04-25T10:29:26.215762+0000","last_unstale":"2024-04-25T10:29:28.240227+0000","last_undegraded":"2024-04-25T10:29:28.240227+0000","last_fullsized":"2024-04-25T10:29:28.240227+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:57.135919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237213+0000","last_change":"2024-04-25T10:29:27.249555+0000","last_active":"2024-04-25T10:29:28.237213+0000","last_peered":"2024-04-25T10:29:28.237213+0000","last_clean":"2024-04-25T10:29:28.237213+0000","last_became_active":"2024-04-25T10:29:27.247104+0000","last_became_peered":"2024-04-25T10:29:27.247104+0000","last_unstale":"2024-04-25T10:29:28.237213+0000","last_undegraded":"2024-04-25T10:29:28.237213+0000","last_fullsized":"2024-04-25T10:29:28.237213+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:35.097185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"154'6","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233240+0000","last_change":"2024-04-25T10:29:26.213436+0000","last_active":"2024-04-25T10:29:28.233240+0000","last_peered":"2024-04-25T10:29:28.233240+0000","last_clean":"2024-04-25T10:29:28.233240+0000","last_became_active":"2024-04-25T10:29:26.213134+0000","last_became_peered":"2024-04-25T10:29:26.213134+0000","last_unstale":"2024-04-25T10:29:28.233240+0000","last_undegraded":"2024-04-25T10:29:28.233240+0000","last_fullsized":"2024-04-25T10:29:28.233240+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:05.054569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315408+0000","last_change":"2024-04-25T10:29:27.227667+0000","last_active":"2024-04-25T10:29:28.315408+0000","last_peered":"2024-04-25T10:29:28.315408+0000","last_clean":"2024-04-25T10:29:28.315408+0000","last_became_active":"2024-04-25T10:29:27.227411+0000","last_became_peered":"2024-04-25T10:29:27.227411+0000","last_unstale":"2024-04-25T10:29:28.315408+0000","last_undegraded":"2024-04-25T10:29:28.315408+0000","last_fullsized":"2024-04-25T10:29:28.315408+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:55.563643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.250668+0000","last_change":"2024-04-25T10:29:26.221865+0000","last_active":"2024-04-25T10:29:27.250668+0000","last_peered":"2024-04-25T10:29:27.250668+0000","last_clean":"2024-04-25T10:29:27.250668+0000","last_became_active":"2024-04-25T10:29:26.221570+0000","last_became_peered":"2024-04-25T10:29:26.221570+0000","last_unstale":"2024-04-25T10:29:27.250668+0000","last_undegraded":"2024-04-25T10:29:27.250668+0000","last_fullsized":"2024-04-25T10:29:27.250668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:38.606592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240054+0000","last_change":"2024-04-25T10:29:27.226960+0000","last_active":"2024-04-25T10:29:28.240054+0000","last_peered":"2024-04-25T10:29:28.240054+0000","last_clean":"2024-04-25T10:29:28.240054+0000","last_became_active":"2024-04-25T10:29:27.226633+0000","last_became_peered":"2024-04-25T10:29:27.226633+0000","last_unstale":"2024-04-25T10:29:28.240054+0000","last_undegraded":"2024-04-25T10:29:28.240054+0000","last_fullsized":"2024-04-25T10:29:28.240054+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:45.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315841+0000","last_change":"2024-04-25T10:29:26.218770+0000","last_active":"2024-04-25T10:29:28.315841+0000","last_peered":"2024-04-25T10:29:28.315841+0000","last_clean":"2024-04-25T10:29:28.315841+0000","last_became_active":"2024-04-25T10:29:26.218502+0000","last_became_peered":"2024-04-25T10:29:26.218502+0000","last_unstale":"2024-04-25T10:29:28.315841+0000","last_undegraded":"2024-04-25T10:29:28.315841+0000","last_fullsized":"2024-04-25T10:29:28.315841+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:21.184436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316023+0000","last_change":"2024-04-25T10:29:27.244813+0000","last_active":"2024-04-25T10:29:28.316023+0000","last_peered":"2024-04-25T10:29:28.316023+0000","last_clean":"2024-04-25T10:29:28.316023+0000","last_became_active":"2024-04-25T10:29:27.244549+0000","last_became_peered":"2024-04-25T10:29:27.244549+0000","last_unstale":"2024-04-25T10:29:28.316023+0000","last_undegraded":"2024-04-25T10:29:28.316023+0000","last_fullsized":"2024-04-25T10:29:28.316023+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:59.277739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316485+0000","last_change":"2024-04-25T10:29:26.218251+0000","last_active":"2024-04-25T10:29:28.316485+0000","last_peered":"2024-04-25T10:29:28.316485+0000","last_clean":"2024-04-25T10:29:28.316485+0000","last_became_active":"2024-04-25T10:29:26.217800+0000","last_became_peered":"2024-04-25T10:29:26.217800+0000","last_unstale":"2024-04-25T10:29:28.316485+0000","last_undegraded":"2024-04-25T10:29:28.316485+0000","last_fullsized":"2024-04-25T10:29:28.316485+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:29.092643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312503+0000","last_change":"2024-04-25T10:29:27.239441+0000","last_active":"2024-04-25T10:29:28.312503+0000","last_peered":"2024-04-25T10:29:28.312503+0000","last_clean":"2024-04-25T10:29:28.312503+0000","last_became_active":"2024-04-25T10:29:27.238516+0000","last_became_peered":"2024-04-25T10:29:27.238516+0000","last_unstale":"2024-04-25T10:29:28.312503+0000","last_undegraded":"2024-04-25T10:29:28.312503+0000","last_fullsized":"2024-04-25T10:29:28.312503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:58.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"154'8","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232758+0000","last_change":"2024-04-25T10:29:26.219265+0000","last_active":"2024-04-25T10:29:28.232758+0000","last_peered":"2024-04-25T10:29:28.232758+0000","last_clean":"2024-04-25T10:29:28.232758+0000","last_became_active":"2024-04-25T10:29:26.218998+0000","last_became_peered":"2024-04-25T10:29:26.218998+0000","last_unstale":"2024-04-25T10:29:28.232758+0000","last_undegraded":"2024-04-25T10:29:28.232758+0000","last_fullsized":"2024-04-25T10:29:28.232758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:23.469280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316095+0000","last_change":"2024-04-25T10:29:27.229323+0000","last_active":"2024-04-25T10:29:28.316095+0000","last_peered":"2024-04-25T10:29:28.316095+0000","last_clean":"2024-04-25T10:29:28.316095+0000","last_became_active":"2024-04-25T10:29:27.228385+0000","last_became_peered":"2024-04-25T10:29:27.228385+0000","last_unstale":"2024-04-25T10:29:28.316095+0000","last_undegraded":"2024-04-25T10:29:28.316095+0000","last_fullsized":"2024-04-25T10:29:28.316095+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233581+0000","last_change":"2024-04-25T10:29:26.215028+0000","last_active":"2024-04-25T10:29:28.233581+0000","last_peered":"2024-04-25T10:29:28.233581+0000","last_clean":"2024-04-25T10:29:28.233581+0000","last_became_active":"2024-04-25T10:29:26.214731+0000","last_became_peered":"2024-04-25T10:29:26.214731+0000","last_unstale":"2024-04-25T10:29:28.233581+0000","last_undegraded":"2024-04-25T10:29:28.233581+0000","last_fullsized":"2024-04-25T10:29:28.233581+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:27.429001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.248054+0000","last_change":"2024-04-25T10:29:27.244058+0000","last_active":"2024-04-25T10:29:27.248054+0000","last_peered":"2024-04-25T10:29:27.248054+0000","last_clean":"2024-04-25T10:29:27.248054+0000","last_became_active":"2024-04-25T10:29:27.243654+0000","last_became_peered":"2024-04-25T10:29:27.243654+0000","last_unstale":"2024-04-25T10:29:27.248054+0000","last_undegraded":"2024-04-25T10:29:27.248054+0000","last_fullsized":"2024-04-25T10:29:27.248054+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:51.922293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312964+0000","last_change":"2024-04-25T10:29:26.218430+0000","last_active":"2024-04-25T10:29:28.312964+0000","last_peered":"2024-04-25T10:29:28.312964+0000","last_clean":"2024-04-25T10:29:28.312964+0000","last_became_active":"2024-04-25T10:29:26.217159+0000","last_became_peered":"2024-04-25T10:29:26.217159+0000","last_unstale":"2024-04-25T10:29:28.312964+0000","last_undegraded":"2024-04-25T10:29:28.312964+0000","last_fullsized":"2024-04-25T10:29:28.312964+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:57.730302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.244990+0000","last_change":"2024-04-25T10:29:27.242205+0000","last_active":"2024-04-25T10:29:27.244990+0000","last_peered":"2024-04-25T10:29:27.244990+0000","last_clean":"2024-04-25T10:29:27.244990+0000","last_became_active":"2024-04-25T10:29:27.240476+0000","last_became_peered":"2024-04-25T10:29:27.240476+0000","last_unstale":"2024-04-25T10:29:27.244990+0000","last_undegraded":"2024-04-25T10:29:27.244990+0000","last_fullsized":"2024-04-25T10:29:27.244990+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:02.795407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233585+0000","last_change":"2024-04-25T10:29:26.214982+0000","last_active":"2024-04-25T10:29:28.233585+0000","last_peered":"2024-04-25T10:29:28.233585+0000","last_clean":"2024-04-25T10:29:28.233585+0000","last_became_active":"2024-04-25T10:29:26.214462+0000","last_became_peered":"2024-04-25T10:29:26.214462+0000","last_unstale":"2024-04-25T10:29:28.233585+0000","last_undegraded":"2024-04-25T10:29:28.233585+0000","last_fullsized":"2024-04-25T10:29:28.233585+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:25.728776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240177+0000","last_change":"2024-04-25T10:29:27.232404+0000","last_active":"2024-04-25T10:29:28.240177+0000","last_peered":"2024-04-25T10:29:28.240177+0000","last_clean":"2024-04-25T10:29:28.240177+0000","last_became_active":"2024-04-25T10:29:27.232136+0000","last_became_peered":"2024-04-25T10:29:27.232136+0000","last_unstale":"2024-04-25T10:29:28.240177+0000","last_undegraded":"2024-04-25T10:29:28.240177+0000","last_fullsized":"2024-04-25T10:29:28.240177+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.802262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237016+0000","last_change":"2024-04-25T10:29:26.223035+0000","last_active":"2024-04-25T10:29:28.237016+0000","last_peered":"2024-04-25T10:29:28.237016+0000","last_clean":"2024-04-25T10:29:28.237016+0000","last_became_active":"2024-04-25T10:29:26.222148+0000","last_became_peered":"2024-04-25T10:29:26.222148+0000","last_unstale":"2024-04-25T10:29:28.237016+0000","last_undegraded":"2024-04-25T10:29:28.237016+0000","last_fullsized":"2024-04-25T10:29:28.237016+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:41.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236772+0000","last_change":"2024-04-25T10:29:27.240797+0000","last_active":"2024-04-25T10:29:28.236772+0000","last_peered":"2024-04-25T10:29:28.236772+0000","last_clean":"2024-04-25T10:29:28.236772+0000","last_became_active":"2024-04-25T10:29:27.240436+0000","last_became_peered":"2024-04-25T10:29:27.240436+0000","last_unstale":"2024-04-25T10:29:28.236772+0000","last_undegraded":"2024-04-25T10:29:28.236772+0000","last_fullsized":"2024-04-25T10:29:28.236772+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:48.345543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240204+0000","last_change":"2024-04-25T10:29:26.225575+0000","last_active":"2024-04-25T10:29:28.240204+0000","last_peered":"2024-04-25T10:29:28.240204+0000","last_clean":"2024-04-25T10:29:28.240204+0000","last_became_active":"2024-04-25T10:29:26.225279+0000","last_became_peered":"2024-04-25T10:29:26.225279+0000","last_unstale":"2024-04-25T10:29:28.240204+0000","last_undegraded":"2024-04-25T10:29:28.240204+0000","last_fullsized":"2024-04-25T10:29:28.240204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:45.188700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240406+0000","last_change":"2024-04-25T10:29:27.242547+0000","last_active":"2024-04-25T10:29:28.240406+0000","last_peered":"2024-04-25T10:29:28.240406+0000","last_clean":"2024-04-25T10:29:28.240406+0000","last_became_active":"2024-04-25T10:29:27.242054+0000","last_became_peered":"2024-04-25T10:29:27.242054+0000","last_unstale":"2024-04-25T10:29:28.240406+0000","last_undegraded":"2024-04-25T10:29:28.240406+0000","last_fullsized":"2024-04-25T10:29:28.240406+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:38.864445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236374+0000","last_change":"2024-04-25T10:29:26.218404+0000","last_active":"2024-04-25T10:29:28.236374+0000","last_peered":"2024-04-25T10:29:28.236374+0000","last_clean":"2024-04-25T10:29:28.236374+0000","last_became_active":"2024-04-25T10:29:26.218125+0000","last_became_peered":"2024-04-25T10:29:26.218125+0000","last_unstale":"2024-04-25T10:29:28.236374+0000","last_undegraded":"2024-04-25T10:29:28.236374+0000","last_fullsized":"2024-04-25T10:29:28.236374+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:32.559695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236342+0000","last_change":"2024-04-25T10:29:27.233890+0000","last_active":"2024-04-25T10:29:28.236342+0000","last_peered":"2024-04-25T10:29:28.236342+0000","last_clean":"2024-04-25T10:29:28.236342+0000","last_became_active":"2024-04-25T10:29:27.233636+0000","last_became_peered":"2024-04-25T10:29:27.233636+0000","last_unstale":"2024-04-25T10:29:28.236342+0000","last_undegraded":"2024-04-25T10:29:28.236342+0000","last_fullsized":"2024-04-25T10:29:28.236342+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.441851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.236083+0000","last_change":"2024-04-25T10:29:26.225860+0000","last_active":"2024-04-25T10:29:27.236083+0000","last_peered":"2024-04-25T10:29:27.236083+0000","last_clean":"2024-04-25T10:29:27.236083+0000","last_became_active":"2024-04-25T10:29:26.224139+0000","last_became_peered":"2024-04-25T10:29:26.224139+0000","last_unstale":"2024-04-25T10:29:27.236083+0000","last_undegraded":"2024-04-25T10:29:27.236083+0000","last_fullsized":"2024-04-25T10:29:27.236083+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:17.718892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316250+0000","last_change":"2024-04-25T10:29:27.234499+0000","last_active":"2024-04-25T10:29:28.316250+0000","last_peered":"2024-04-25T10:29:28.316250+0000","last_clean":"2024-04-25T10:29:28.316250+0000","last_became_active":"2024-04-25T10:29:27.233878+0000","last_became_peered":"2024-04-25T10:29:27.233878+0000","last_unstale":"2024-04-25T10:29:28.316250+0000","last_undegraded":"2024-04-25T10:29:28.316250+0000","last_fullsized":"2024-04-25T10:29:28.316250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:45.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236348+0000","last_change":"2024-04-25T10:29:26.223768+0000","last_active":"2024-04-25T10:29:28.236348+0000","last_peered":"2024-04-25T10:29:28.236348+0000","last_clean":"2024-04-25T10:29:28.236348+0000","last_became_active":"2024-04-25T10:29:26.223294+0000","last_became_peered":"2024-04-25T10:29:26.223294+0000","last_unstale":"2024-04-25T10:29:28.236348+0000","last_undegraded":"2024-04-25T10:29:28.236348+0000","last_fullsized":"2024-04-25T10:29:28.236348+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.282561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240532+0000","last_change":"2024-04-25T10:29:27.232600+0000","last_active":"2024-04-25T10:29:28.240532+0000","last_peered":"2024-04-25T10:29:28.240532+0000","last_clean":"2024-04-25T10:29:28.240532+0000","last_became_active":"2024-04-25T10:29:27.232009+0000","last_became_peered":"2024-04-25T10:29:27.232009+0000","last_unstale":"2024-04-25T10:29:28.240532+0000","last_undegraded":"2024-04-25T10:29:28.240532+0000","last_fullsized":"2024-04-25T10:29:28.240532+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:49.689247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316475+0000","last_change":"2024-04-25T10:29:26.224020+0000","last_active":"2024-04-25T10:29:28.316475+0000","last_peered":"2024-04-25T10:29:28.316475+0000","last_clean":"2024-04-25T10:29:28.316475+0000","last_became_active":"2024-04-25T10:29:26.223709+0000","last_became_peered":"2024-04-25T10:29:26.223709+0000","last_unstale":"2024-04-25T10:29:28.316475+0000","last_undegraded":"2024-04-25T10:29:28.316475+0000","last_fullsized":"2024-04-25T10:29:28.316475+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:06.076162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.245934+0000","last_change":"2024-04-25T10:29:27.239292+0000","last_active":"2024-04-25T10:29:27.245934+0000","last_peered":"2024-04-25T10:29:27.245934+0000","last_clean":"2024-04-25T10:29:27.245934+0000","last_became_active":"2024-04-25T10:29:27.238797+0000","last_became_peered":"2024-04-25T10:29:27.238797+0000","last_unstale":"2024-04-25T10:29:27.245934+0000","last_undegraded":"2024-04-25T10:29:27.245934+0000","last_fullsized":"2024-04-25T10:29:27.245934+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:03.085959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.238336+0000","last_change":"2024-04-25T10:29:26.222859+0000","last_active":"2024-04-25T10:29:28.238336+0000","last_peered":"2024-04-25T10:29:28.238336+0000","last_clean":"2024-04-25T10:29:28.238336+0000","last_became_active":"2024-04-25T10:29:26.222585+0000","last_became_peered":"2024-04-25T10:29:26.222585+0000","last_unstale":"2024-04-25T10:29:28.238336+0000","last_undegraded":"2024-04-25T10:29:28.238336+0000","last_fullsized":"2024-04-25T10:29:28.238336+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:21.561672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.247041+0000","last_change":"2024-04-25T10:29:27.242834+0000","last_active":"2024-04-25T10:29:27.247041+0000","last_peered":"2024-04-25T10:29:27.247041+0000","last_clean":"2024-04-25T10:29:27.247041+0000","last_became_active":"2024-04-25T10:29:27.241231+0000","last_became_peered":"2024-04-25T10:29:27.241231+0000","last_unstale":"2024-04-25T10:29:27.247041+0000","last_undegraded":"2024-04-25T10:29:27.247041+0000","last_fullsized":"2024-04-25T10:29:27.247041+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:51.681540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237398+0000","last_change":"2024-04-25T10:29:26.221049+0000","last_active":"2024-04-25T10:29:28.237398+0000","last_peered":"2024-04-25T10:29:28.237398+0000","last_clean":"2024-04-25T10:29:28.237398+0000","last_became_active":"2024-04-25T10:29:26.218977+0000","last_became_peered":"2024-04-25T10:29:26.218977+0000","last_unstale":"2024-04-25T10:29:28.237398+0000","last_undegraded":"2024-04-25T10:29:28.237398+0000","last_fullsized":"2024-04-25T10:29:28.237398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:22.018836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.251199+0000","last_change":"2024-04-25T10:29:27.249715+0000","last_active":"2024-04-25T10:29:27.251199+0000","last_peered":"2024-04-25T10:29:27.251199+0000","last_clean":"2024-04-25T10:29:27.251199+0000","last_became_active":"2024-04-25T10:29:27.249449+0000","last_became_peered":"2024-04-25T10:29:27.249449+0000","last_unstale":"2024-04-25T10:29:27.251199+0000","last_undegraded":"2024-04-25T10:29:27.251199+0000","last_fullsized":"2024-04-25T10:29:27.251199+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:30.310975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240263+0000","last_change":"2024-04-25T10:29:26.216306+0000","last_active":"2024-04-25T10:29:28.240263+0000","last_peered":"2024-04-25T10:29:28.240263+0000","last_clean":"2024-04-25T10:29:28.240263+0000","last_became_active":"2024-04-25T10:29:26.214802+0000","last_became_peered":"2024-04-25T10:29:26.214802+0000","last_unstale":"2024-04-25T10:29:28.240263+0000","last_undegraded":"2024-04-25T10:29:28.240263+0000","last_fullsized":"2024-04-25T10:29:28.240263+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:22.725339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315710+0000","last_change":"2024-04-25T10:29:27.241706+0000","last_active":"2024-04-25T10:29:28.315710+0000","last_peered":"2024-04-25T10:29:28.315710+0000","last_clean":"2024-04-25T10:29:28.315710+0000","last_became_active":"2024-04-25T10:29:27.241412+0000","last_became_peered":"2024-04-25T10:29:27.241412+0000","last_unstale":"2024-04-25T10:29:28.315710+0000","last_undegraded":"2024-04-25T10:29:28.315710+0000","last_fullsized":"2024-04-25T10:29:28.315710+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:51.290877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233171+0000","last_change":"2024-04-25T10:29:26.219570+0000","last_active":"2024-04-25T10:29:28.233171+0000","last_peered":"2024-04-25T10:29:28.233171+0000","last_clean":"2024-04-25T10:29:28.233171+0000","last_became_active":"2024-04-25T10:29:26.219115+0000","last_became_peered":"2024-04-25T10:29:26.219115+0000","last_unstale":"2024-04-25T10:29:28.233171+0000","last_undegraded":"2024-04-25T10:29:28.233171+0000","last_fullsized":"2024-04-25T10:29:28.233171+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:58.765836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236949+0000","last_change":"2024-04-25T10:29:27.232984+0000","last_active":"2024-04-25T10:29:28.236949+0000","last_peered":"2024-04-25T10:29:28.236949+0000","last_clean":"2024-04-25T10:29:28.236949+0000","last_became_active":"2024-04-25T10:29:27.232567+0000","last_became_peered":"2024-04-25T10:29:27.232567+0000","last_unstale":"2024-04-25T10:29:28.236949+0000","last_undegraded":"2024-04-25T10:29:28.236949+0000","last_fullsized":"2024-04-25T10:29:28.236949+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:34.759558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315268+0000","last_change":"2024-04-25T10:29:26.226836+0000","last_active":"2024-04-25T10:29:28.315268+0000","last_peered":"2024-04-25T10:29:28.315268+0000","last_clean":"2024-04-25T10:29:28.315268+0000","last_became_active":"2024-04-25T10:29:26.226573+0000","last_became_peered":"2024-04-25T10:29:26.226573+0000","last_unstale":"2024-04-25T10:29:28.315268+0000","last_undegraded":"2024-04-25T10:29:28.315268+0000","last_fullsized":"2024-04-25T10:29:28.315268+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:58.270147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315300+0000","last_change":"2024-04-25T10:29:27.243036+0000","last_active":"2024-04-25T10:29:28.315300+0000","last_peered":"2024-04-25T10:29:28.315300+0000","last_clean":"2024-04-25T10:29:28.315300+0000","last_became_active":"2024-04-25T10:29:27.242736+0000","last_became_peered":"2024-04-25T10:29:27.242736+0000","last_unstale":"2024-04-25T10:29:28.315300+0000","last_undegraded":"2024-04-25T10:29:28.315300+0000","last_fullsized":"2024-04-25T10:29:28.315300+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:35.026246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236743+0000","last_change":"2024-04-25T10:29:26.221470+0000","last_active":"2024-04-25T10:29:28.236743+0000","last_peered":"2024-04-25T10:29:28.236743+0000","last_clean":"2024-04-25T10:29:28.236743+0000","last_became_active":"2024-04-25T10:29:26.219461+0000","last_became_peered":"2024-04-25T10:29:26.219461+0000","last_unstale":"2024-04-25T10:29:28.236743+0000","last_undegraded":"2024-04-25T10:29:28.236743+0000","last_fullsized":"2024-04-25T10:29:28.236743+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:25.410538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236603+0000","last_change":"2024-04-25T10:29:27.232875+0000","last_active":"2024-04-25T10:29:28.236603+0000","last_peered":"2024-04-25T10:29:28.236603+0000","last_clean":"2024-04-25T10:29:28.236603+0000","last_became_active":"2024-04-25T10:29:27.232569+0000","last_became_peered":"2024-04-25T10:29:27.232569+0000","last_unstale":"2024-04-25T10:29:28.236603+0000","last_undegraded":"2024-04-25T10:29:28.236603+0000","last_fullsized":"2024-04-25T10:29:28.236603+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:54.150228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236509+0000","last_change":"2024-04-25T10:29:26.217515+0000","last_active":"2024-04-25T10:29:28.236509+0000","last_peered":"2024-04-25T10:29:28.236509+0000","last_clean":"2024-04-25T10:29:28.236509+0000","last_became_active":"2024-04-25T10:29:26.216855+0000","last_became_peered":"2024-04-25T10:29:26.216855+0000","last_unstale":"2024-04-25T10:29:28.236509+0000","last_undegraded":"2024-04-25T10:29:28.236509+0000","last_fullsized":"2024-04-25T10:29:28.236509+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:22.725339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315961+0000","last_change":"2024-04-25T10:29:27.245408+0000","last_active":"2024-04-25T10:29:28.315961+0000","last_peered":"2024-04-25T10:29:28.315961+0000","last_clean":"2024-04-25T10:29:28.315961+0000","last_became_active":"2024-04-25T10:29:27.244939+0000","last_became_peered":"2024-04-25T10:29:27.244939+0000","last_unstale":"2024-04-25T10:29:28.315961+0000","last_undegraded":"2024-04-25T10:29:28.315961+0000","last_fullsized":"2024-04-25T10:29:28.315961+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:18.195645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233513+0000","last_change":"2024-04-25T10:29:26.215139+0000","last_active":"2024-04-25T10:29:28.233513+0000","last_peered":"2024-04-25T10:29:28.233513+0000","last_clean":"2024-04-25T10:29:28.233513+0000","last_became_active":"2024-04-25T10:29:26.214832+0000","last_became_peered":"2024-04-25T10:29:26.214832+0000","last_unstale":"2024-04-25T10:29:28.233513+0000","last_undegraded":"2024-04-25T10:29:28.233513+0000","last_fullsized":"2024-04-25T10:29:28.233513+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.286005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":426,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232803+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:29:28.232803+0000","last_peered":"2024-04-25T10:29:28.232803+0000","last_clean":"2024-04-25T10:29:28.232803+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:29:28.232803+0000","last_undegraded":"2024-04-25T10:29:28.232803+0000","last_fullsized":"2024-04-25T10:29:28.232803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55280,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316560,"statfs":{"total":96636764160,"available":96580157440,"internally_reserved":0,"allocated":1937408,"data_stored":848132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509584,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55276,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316564,"statfs":{"total":96636764160,"available":96580161536,"internally_reserved":0,"allocated":1929216,"data_stored":841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60036,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2539520,"data_stored":1439172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46984,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324856,"statfs":{"total":96636764160,"available":96588652544,"internally_reserved":0,"allocated":1953792,"data_stored":849766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46964,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324876,"statfs":{"total":96636764160,"available":96588673024,"internally_reserved":0,"allocated":1941504,"data_stored":848714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47572,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324268,"statfs":{"total":96636764160,"available":96588050432,"internally_reserved":0,"allocated":2555904,"data_stored":1440294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55288,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316552,"statfs":{"total":96636764160,"available":96580149248,"internally_reserved":0,"allocated":1941504,"data_stored":848202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":1912832,"data_stored":835057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:31.349 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:29:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.528+0000 7fca5b800640 1 Processor -- start 2024-04-25T10:29:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca5b800640 1 -- start start 2024-04-25T10:29:31.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca5b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540cfd90 0x7fca540ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca5b800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fca540cd110 0x7fca540cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca5b800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5415f3b0 con 0x7fca540b0f70 2024-04-25T10:29:31.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca5b800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5415ef20 con 0x7fca540cd110 2024-04-25T10:29:31.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca5b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca540ca810 con 0x7fca540cfd90 2024-04-25T10:29:31.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca58e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fca540b0f70 0x7fca540b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60964/0 (socket says 172.21.15.45:60964) 2024-04-25T10:29:31.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca58e00640 1 -- 172.21.15.45:0/1962862684 learned_addr learned my addr 172.21.15.45:0/1962862684 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca4be00640 1 --2- 172.21.15.45:0/1962862684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540cfd90 0x7fca540ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca59800640 1 --2- 172.21.15.45:0/1962862684 >> v2:172.21.15.117:3300/0 conn(0x7fca540cd110 0x7fca540cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca53e00640 1 -- 172.21.15.45:0/1962862684 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194079809 0 0) 0x7fca5415f3b0 con 0x7fca540b0f70 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca53e00640 1 -- 172.21.15.45:0/1962862684 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca38003530 con 0x7fca540b0f70 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca4be00640 1 -- 172.21.15.45:0/1962862684 >> v2:172.21.15.117:3300/0 conn(0x7fca540cd110 msgr2=0x7fca540cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca4be00640 1 --2- 172.21.15.45:0/1962862684 >> v2:172.21.15.117:3300/0 conn(0x7fca540cd110 0x7fca540cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca4be00640 1 -- 172.21.15.45:0/1962862684 >> v1:172.21.15.45:6789/0 conn(0x7fca540b0f70 legacy=0x7fca540b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca4be00640 1 -- 172.21.15.45:0/1962862684 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5415e5f0 con 0x7fca540cfd90 2024-04-25T10:29:31.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.532+0000 7fca59800640 1 --2- 172.21.15.45:0/1962862684 >> v2:172.21.15.117:3300/0 conn(0x7fca540cd110 0x7fca540cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:29:31.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca4be00640 1 --2- 172.21.15.45:0/1962862684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540cfd90 0x7fca540ccbd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0097c0 tx=0x7fca3c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dad7670f7c0c1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca53e00640 1 -- 172.21.15.45:0/1962862684 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0b0070 con 0x7fca540cfd90 2024-04-25T10:29:31.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca53e00640 1 -- 172.21.15.45:0/1962862684 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c0ab600 con 0x7fca540cfd90 2024-04-25T10:29:31.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca53e00640 1 -- 172.21.15.45:0/1962862684 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0ab9d0 con 0x7fca540cfd90 2024-04-25T10:29:31.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 -- 172.21.15.45:0/1962862684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540cfd90 msgr2=0x7fca540ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 --2- 172.21.15.45:0/1962862684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540cfd90 0x7fca540ccbd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0097c0 tx=0x7fca3c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 -- 172.21.15.45:0/1962862684 shutdown_connections 2024-04-25T10:29:31.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 --2- 172.21.15.45:0/1962862684 >> v2:172.21.15.117:3300/0 conn(0x7fca540cd110 0x7fca540cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 --2- 172.21.15.45:0/1962862684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540cfd90 0x7fca540ccbd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 -- 172.21.15.45:0/1962862684 >> 172.21.15.45:0/1962862684 conn(0x7fca540b1d50 msgr2=0x7fca540b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 -- 172.21.15.45:0/1962862684 shutdown_connections 2024-04-25T10:29:31.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 -- 172.21.15.45:0/1962862684 wait complete. 2024-04-25T10:29:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 Processor -- start 2024-04-25T10:29:31.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.536+0000 7fca5b800640 1 -- start start 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540b0f70 0x7fca540b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca540cd110 0x7fca54156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 0x7fca540b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca54161b50 con 0x7fca540cfd90 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca541619d0 con 0x7fca540cd110 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca54161cd0 con 0x7fca540b0f70 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 0x7fca540b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 0x7fca540b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56854/0 (socket says 172.21.15.45:56854) 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 -- 172.21.15.45:0/3080238581 learned_addr learned my addr 172.21.15.45:0/3080238581 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca58e00640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540b0f70 0x7fca540b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca4be00640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca540cd110 0x7fca54156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 -- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540b0f70 msgr2=0x7fca540b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540b0f70 0x7fca540b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 -- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca540cd110 msgr2=0x7fca54156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca540cd110 0x7fca54156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 -- 172.21.15.45:0/3080238581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540afea0 con 0x7fca540cfd90 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca59800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 0x7fca540b5990 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009f00 tx=0x7fca4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c018030 con 0x7fca540cfd90 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c00ce50 con 0x7fca540cfd90 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca540b0180 con 0x7fca540cfd90 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c003bc0 con 0x7fca540cfd90 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca54154070 con 0x7fca540cfd90 2024-04-25T10:29:31.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5404ef30 con 0x7fca540cfd90 2024-04-25T10:29:31.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.540+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca4c01b070 con 0x7fca540cfd90 2024-04-25T10:29:31.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.544+0000 7fca51600640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca30066f60 0x7fca30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.544+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca4c085b10 con 0x7fca540cfd90 2024-04-25T10:29:31.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.544+0000 7fca58e00640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca30066f60 0x7fca30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.544+0000 7fca58e00640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca30066f60 0x7fca30069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fca44005890 tx=0x7fca44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.548+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca4c022070 con 0x7fca540cfd90 2024-04-25T10:29:31.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.704+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca540cda20 con 0x7fca30066f60 2024-04-25T10:29:31.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca51600640 1 -- 172.21.15.45:0/3080238581 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252474 (secure 0 0 0) 0x7fca540cda20 con 0x7fca30066f60 2024-04-25T10:29:31.714 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:31.716 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca30066f60 msgr2=0x7fca30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca30066f60 0x7fca30069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fca44005890 tx=0x7fca44009040 comp rx=0 tx=0).stop 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 msgr2=0x7fca540b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 0x7fca540b5990 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009f00 tx=0x7fca4c00c490 comp rx=0 tx=0).stop 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 shutdown_connections 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca30066f60 0x7fca30069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca540cfd90 0x7fca540b5990 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca540cd110 0x7fca54156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 --2- 172.21.15.45:0/3080238581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca540b0f70 0x7fca540b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 >> 172.21.15.45:0/3080238581 conn(0x7fca540b1d50 msgr2=0x7fca5406df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.712+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 shutdown_connections 2024-04-25T10:29:31.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.716+0000 7fca5b800640 1 -- 172.21.15.45:0/3080238581 wait complete. 2024-04-25T10:29:31.735 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-04-25T10:29:31.035111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432320,"kb_used_data":16320,"kb_used_omap":12,"kb_used_meta":415859,"kb_avail":754542400,"statfs":{"total":773094113280,"available":772651417600,"internally_reserved":0,"allocated":16711680,"data_stored":7951203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":425840176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.835035"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237520+0000","last_change":"2024-04-25T10:29:27.248081+0000","last_active":"2024-04-25T10:29:28.237520+0000","last_peered":"2024-04-25T10:29:28.237520+0000","last_clean":"2024-04-25T10:29:28.237520+0000","last_became_active":"2024-04-25T10:29:27.247768+0000","last_became_peered":"2024-04-25T10:29:27.247768+0000","last_unstale":"2024-04-25T10:29:28.237520+0000","last_undegraded":"2024-04-25T10:29:28.237520+0000","last_fullsized":"2024-04-25T10:29:28.237520+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:16.073626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237545+0000","last_change":"2024-04-25T10:29:27.248087+0000","last_active":"2024-04-25T10:29:28.237545+0000","last_peered":"2024-04-25T10:29:28.237545+0000","last_clean":"2024-04-25T10:29:28.237545+0000","last_became_active":"2024-04-25T10:29:27.247814+0000","last_became_peered":"2024-04-25T10:29:27.247814+0000","last_unstale":"2024-04-25T10:29:28.237545+0000","last_undegraded":"2024-04-25T10:29:28.237545+0000","last_fullsized":"2024-04-25T10:29:28.237545+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:26.351411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237219+0000","last_change":"2024-04-25T10:29:27.231026+0000","last_active":"2024-04-25T10:29:28.237219+0000","last_peered":"2024-04-25T10:29:28.237219+0000","last_clean":"2024-04-25T10:29:28.237219+0000","last_became_active":"2024-04-25T10:29:27.230711+0000","last_became_peered":"2024-04-25T10:29:27.230711+0000","last_unstale":"2024-04-25T10:29:28.237219+0000","last_undegraded":"2024-04-25T10:29:28.237219+0000","last_fullsized":"2024-04-25T10:29:28.237219+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:46.545736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.244966+0000","last_change":"2024-04-25T10:29:27.242022+0000","last_active":"2024-04-25T10:29:27.244966+0000","last_peered":"2024-04-25T10:29:27.244966+0000","last_clean":"2024-04-25T10:29:27.244966+0000","last_became_active":"2024-04-25T10:29:27.240586+0000","last_became_peered":"2024-04-25T10:29:27.240586+0000","last_unstale":"2024-04-25T10:29:27.244966+0000","last_undegraded":"2024-04-25T10:29:27.244966+0000","last_fullsized":"2024-04-25T10:29:27.244966+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:38.516858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233051+0000","last_change":"2024-04-25T10:29:27.243305+0000","last_active":"2024-04-25T10:29:28.233051+0000","last_peered":"2024-04-25T10:29:28.233051+0000","last_clean":"2024-04-25T10:29:28.233051+0000","last_became_active":"2024-04-25T10:29:27.243002+0000","last_became_peered":"2024-04-25T10:29:27.243002+0000","last_unstale":"2024-04-25T10:29:28.233051+0000","last_undegraded":"2024-04-25T10:29:28.233051+0000","last_fullsized":"2024-04-25T10:29:28.233051+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:43.215658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315707+0000","last_change":"2024-04-25T10:29:27.228343+0000","last_active":"2024-04-25T10:29:28.315707+0000","last_peered":"2024-04-25T10:29:28.315707+0000","last_clean":"2024-04-25T10:29:28.315707+0000","last_became_active":"2024-04-25T10:29:27.228030+0000","last_became_peered":"2024-04-25T10:29:27.228030+0000","last_unstale":"2024-04-25T10:29:28.315707+0000","last_undegraded":"2024-04-25T10:29:28.315707+0000","last_fullsized":"2024-04-25T10:29:28.315707+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:41.083508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316387+0000","last_change":"2024-04-25T10:29:27.249589+0000","last_active":"2024-04-25T10:29:28.316387+0000","last_peered":"2024-04-25T10:29:28.316387+0000","last_clean":"2024-04-25T10:29:28.316387+0000","last_became_active":"2024-04-25T10:29:27.249282+0000","last_became_peered":"2024-04-25T10:29:27.249282+0000","last_unstale":"2024-04-25T10:29:28.316387+0000","last_undegraded":"2024-04-25T10:29:28.316387+0000","last_fullsized":"2024-04-25T10:29:28.316387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:38.864445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312494+0000","last_change":"2024-04-25T10:29:27.241481+0000","last_active":"2024-04-25T10:29:28.312494+0000","last_peered":"2024-04-25T10:29:28.312494+0000","last_clean":"2024-04-25T10:29:28.312494+0000","last_became_active":"2024-04-25T10:29:27.240946+0000","last_became_peered":"2024-04-25T10:29:27.240946+0000","last_unstale":"2024-04-25T10:29:28.312494+0000","last_undegraded":"2024-04-25T10:29:28.312494+0000","last_fullsized":"2024-04-25T10:29:28.312494+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:41.124396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312941+0000","last_change":"2024-04-25T10:29:27.245541+0000","last_active":"2024-04-25T10:29:28.312941+0000","last_peered":"2024-04-25T10:29:28.312941+0000","last_clean":"2024-04-25T10:29:28.312941+0000","last_became_active":"2024-04-25T10:29:27.245124+0000","last_became_peered":"2024-04-25T10:29:27.245124+0000","last_unstale":"2024-04-25T10:29:28.312941+0000","last_undegraded":"2024-04-25T10:29:28.312941+0000","last_fullsized":"2024-04-25T10:29:28.312941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:26.418130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312878+0000","last_change":"2024-04-25T10:29:27.240287+0000","last_active":"2024-04-25T10:29:28.312878+0000","last_peered":"2024-04-25T10:29:28.312878+0000","last_clean":"2024-04-25T10:29:28.312878+0000","last_became_active":"2024-04-25T10:29:27.239984+0000","last_became_peered":"2024-04-25T10:29:27.239984+0000","last_unstale":"2024-04-25T10:29:28.312878+0000","last_undegraded":"2024-04-25T10:29:28.312878+0000","last_fullsized":"2024-04-25T10:29:28.312878+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:23.732931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233383+0000","last_change":"2024-04-25T10:29:27.233684+0000","last_active":"2024-04-25T10:29:28.233383+0000","last_peered":"2024-04-25T10:29:28.233383+0000","last_clean":"2024-04-25T10:29:28.233383+0000","last_became_active":"2024-04-25T10:29:27.233326+0000","last_became_peered":"2024-04-25T10:29:27.233326+0000","last_unstale":"2024-04-25T10:29:28.233383+0000","last_undegraded":"2024-04-25T10:29:28.233383+0000","last_fullsized":"2024-04-25T10:29:28.233383+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.942026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236894+0000","last_change":"2024-04-25T10:29:27.233900+0000","last_active":"2024-04-25T10:29:28.236894+0000","last_peered":"2024-04-25T10:29:28.236894+0000","last_clean":"2024-04-25T10:29:28.236894+0000","last_became_active":"2024-04-25T10:29:27.233637+0000","last_became_peered":"2024-04-25T10:29:27.233637+0000","last_unstale":"2024-04-25T10:29:28.236894+0000","last_undegraded":"2024-04-25T10:29:28.236894+0000","last_fullsized":"2024-04-25T10:29:28.236894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:26.940558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236465+0000","last_change":"2024-04-25T10:29:27.222766+0000","last_active":"2024-04-25T10:29:28.236465+0000","last_peered":"2024-04-25T10:29:28.236465+0000","last_clean":"2024-04-25T10:29:28.236465+0000","last_became_active":"2024-04-25T10:29:27.222497+0000","last_became_peered":"2024-04-25T10:29:27.222497+0000","last_unstale":"2024-04-25T10:29:28.236465+0000","last_undegraded":"2024-04-25T10:29:28.236465+0000","last_fullsized":"2024-04-25T10:29:28.236465+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:31.624814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312494+0000","last_change":"2024-04-25T10:29:27.248269+0000","last_active":"2024-04-25T10:29:28.312494+0000","last_peered":"2024-04-25T10:29:28.312494+0000","last_clean":"2024-04-25T10:29:28.312494+0000","last_became_active":"2024-04-25T10:29:27.247993+0000","last_became_peered":"2024-04-25T10:29:27.247993+0000","last_unstale":"2024-04-25T10:29:28.312494+0000","last_undegraded":"2024-04-25T10:29:28.312494+0000","last_fullsized":"2024-04-25T10:29:28.312494+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:17.450769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312503+0000","last_change":"2024-04-25T10:29:27.239149+0000","last_active":"2024-04-25T10:29:28.312503+0000","last_peered":"2024-04-25T10:29:28.312503+0000","last_clean":"2024-04-25T10:29:28.312503+0000","last_became_active":"2024-04-25T10:29:27.238865+0000","last_became_peered":"2024-04-25T10:29:27.238865+0000","last_unstale":"2024-04-25T10:29:28.312503+0000","last_undegraded":"2024-04-25T10:29:28.312503+0000","last_fullsized":"2024-04-25T10:29:28.312503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:49.946175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315254+0000","last_change":"2024-04-25T10:29:27.235646+0000","last_active":"2024-04-25T10:29:28.315254+0000","last_peered":"2024-04-25T10:29:28.315254+0000","last_clean":"2024-04-25T10:29:28.315254+0000","last_became_active":"2024-04-25T10:29:27.234069+0000","last_became_peered":"2024-04-25T10:29:27.234069+0000","last_unstale":"2024-04-25T10:29:28.315254+0000","last_undegraded":"2024-04-25T10:29:28.315254+0000","last_fullsized":"2024-04-25T10:29:28.315254+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:39.612171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316514+0000","last_change":"2024-04-25T10:29:27.226984+0000","last_active":"2024-04-25T10:29:28.316514+0000","last_peered":"2024-04-25T10:29:28.316514+0000","last_clean":"2024-04-25T10:29:28.316514+0000","last_became_active":"2024-04-25T10:29:27.226687+0000","last_became_peered":"2024-04-25T10:29:27.226687+0000","last_unstale":"2024-04-25T10:29:28.316514+0000","last_undegraded":"2024-04-25T10:29:28.316514+0000","last_fullsized":"2024-04-25T10:29:28.316514+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:30.100235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.248005+0000","last_change":"2024-04-25T10:29:27.243411+0000","last_active":"2024-04-25T10:29:27.248005+0000","last_peered":"2024-04-25T10:29:27.248005+0000","last_clean":"2024-04-25T10:29:27.248005+0000","last_became_active":"2024-04-25T10:29:27.242969+0000","last_became_peered":"2024-04-25T10:29:27.242969+0000","last_unstale":"2024-04-25T10:29:27.248005+0000","last_undegraded":"2024-04-25T10:29:27.248005+0000","last_fullsized":"2024-04-25T10:29:27.248005+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:03.199889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316281+0000","last_change":"2024-04-25T10:29:27.239540+0000","last_active":"2024-04-25T10:29:28.316281+0000","last_peered":"2024-04-25T10:29:28.316281+0000","last_clean":"2024-04-25T10:29:28.316281+0000","last_became_active":"2024-04-25T10:29:27.239269+0000","last_became_peered":"2024-04-25T10:29:27.239269+0000","last_unstale":"2024-04-25T10:29:28.316281+0000","last_undegraded":"2024-04-25T10:29:28.316281+0000","last_fullsized":"2024-04-25T10:29:28.316281+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.802262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236806+0000","last_change":"2024-04-25T10:29:27.224361+0000","last_active":"2024-04-25T10:29:28.236806+0000","last_peered":"2024-04-25T10:29:28.236806+0000","last_clean":"2024-04-25T10:29:28.236806+0000","last_became_active":"2024-04-25T10:29:27.224079+0000","last_became_peered":"2024-04-25T10:29:27.224079+0000","last_unstale":"2024-04-25T10:29:28.236806+0000","last_undegraded":"2024-04-25T10:29:28.236806+0000","last_fullsized":"2024-04-25T10:29:28.236806+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:46.578402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315715+0000","last_change":"2024-04-25T10:29:27.235569+0000","last_active":"2024-04-25T10:29:28.315715+0000","last_peered":"2024-04-25T10:29:28.315715+0000","last_clean":"2024-04-25T10:29:28.315715+0000","last_became_active":"2024-04-25T10:29:27.235290+0000","last_became_peered":"2024-04-25T10:29:27.235290+0000","last_unstale":"2024-04-25T10:29:28.315715+0000","last_undegraded":"2024-04-25T10:29:28.315715+0000","last_fullsized":"2024-04-25T10:29:28.315715+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:58.737894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232931+0000","last_change":"2024-04-25T10:29:27.226345+0000","last_active":"2024-04-25T10:29:28.232931+0000","last_peered":"2024-04-25T10:29:28.232931+0000","last_clean":"2024-04-25T10:29:28.232931+0000","last_became_active":"2024-04-25T10:29:27.226067+0000","last_became_peered":"2024-04-25T10:29:27.226067+0000","last_unstale":"2024-04-25T10:29:28.232931+0000","last_undegraded":"2024-04-25T10:29:28.232931+0000","last_fullsized":"2024-04-25T10:29:28.232931+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:38.159997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.239736+0000","last_change":"2024-04-25T10:29:27.224474+0000","last_active":"2024-04-25T10:29:28.239736+0000","last_peered":"2024-04-25T10:29:28.239736+0000","last_clean":"2024-04-25T10:29:28.239736+0000","last_became_active":"2024-04-25T10:29:27.224194+0000","last_became_peered":"2024-04-25T10:29:27.224194+0000","last_unstale":"2024-04-25T10:29:28.239736+0000","last_undegraded":"2024-04-25T10:29:28.239736+0000","last_fullsized":"2024-04-25T10:29:28.239736+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232747+0000","last_change":"2024-04-25T10:29:27.246389+0000","last_active":"2024-04-25T10:29:28.232747+0000","last_peered":"2024-04-25T10:29:28.232747+0000","last_clean":"2024-04-25T10:29:28.232747+0000","last_became_active":"2024-04-25T10:29:27.240755+0000","last_became_peered":"2024-04-25T10:29:27.240755+0000","last_unstale":"2024-04-25T10:29:28.232747+0000","last_undegraded":"2024-04-25T10:29:28.232747+0000","last_fullsized":"2024-04-25T10:29:28.232747+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:30.699147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233414+0000","last_change":"2024-04-25T10:29:27.247107+0000","last_active":"2024-04-25T10:29:28.233414+0000","last_peered":"2024-04-25T10:29:28.233414+0000","last_clean":"2024-04-25T10:29:28.233414+0000","last_became_active":"2024-04-25T10:29:27.246803+0000","last_became_peered":"2024-04-25T10:29:27.246803+0000","last_unstale":"2024-04-25T10:29:28.233414+0000","last_undegraded":"2024-04-25T10:29:28.233414+0000","last_fullsized":"2024-04-25T10:29:28.233414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:59.983279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240490+0000","last_change":"2024-04-25T10:29:27.232793+0000","last_active":"2024-04-25T10:29:28.240490+0000","last_peered":"2024-04-25T10:29:28.240490+0000","last_clean":"2024-04-25T10:29:28.240490+0000","last_became_active":"2024-04-25T10:29:27.232340+0000","last_became_peered":"2024-04-25T10:29:27.232340+0000","last_unstale":"2024-04-25T10:29:28.240490+0000","last_undegraded":"2024-04-25T10:29:28.240490+0000","last_fullsized":"2024-04-25T10:29:28.240490+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:16.095428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233496+0000","last_change":"2024-04-25T10:29:27.233105+0000","last_active":"2024-04-25T10:29:28.233496+0000","last_peered":"2024-04-25T10:29:28.233496+0000","last_clean":"2024-04-25T10:29:28.233496+0000","last_became_active":"2024-04-25T10:29:27.232016+0000","last_became_peered":"2024-04-25T10:29:27.232016+0000","last_unstale":"2024-04-25T10:29:28.233496+0000","last_undegraded":"2024-04-25T10:29:28.233496+0000","last_fullsized":"2024-04-25T10:29:28.233496+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:22.192028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315662+0000","last_change":"2024-04-25T10:29:27.238198+0000","last_active":"2024-04-25T10:29:28.315662+0000","last_peered":"2024-04-25T10:29:28.315662+0000","last_clean":"2024-04-25T10:29:28.315662+0000","last_became_active":"2024-04-25T10:29:27.237918+0000","last_became_peered":"2024-04-25T10:29:27.237918+0000","last_unstale":"2024-04-25T10:29:28.315662+0000","last_undegraded":"2024-04-25T10:29:28.315662+0000","last_fullsized":"2024-04-25T10:29:28.315662+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.436622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236952+0000","last_change":"2024-04-25T10:29:27.225093+0000","last_active":"2024-04-25T10:29:28.236952+0000","last_peered":"2024-04-25T10:29:28.236952+0000","last_clean":"2024-04-25T10:29:28.236952+0000","last_became_active":"2024-04-25T10:29:27.224813+0000","last_became_peered":"2024-04-25T10:29:27.224813+0000","last_unstale":"2024-04-25T10:29:28.236952+0000","last_undegraded":"2024-04-25T10:29:28.236952+0000","last_fullsized":"2024-04-25T10:29:28.236952+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:01.145754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312581+0000","last_change":"2024-04-25T10:29:27.249124+0000","last_active":"2024-04-25T10:29:28.312581+0000","last_peered":"2024-04-25T10:29:28.312581+0000","last_clean":"2024-04-25T10:29:28.312581+0000","last_became_active":"2024-04-25T10:29:27.246171+0000","last_became_peered":"2024-04-25T10:29:27.246171+0000","last_unstale":"2024-04-25T10:29:28.312581+0000","last_undegraded":"2024-04-25T10:29:28.312581+0000","last_fullsized":"2024-04-25T10:29:28.312581+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:46.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236665+0000","last_change":"2024-04-25T10:29:27.239619+0000","last_active":"2024-04-25T10:29:28.236665+0000","last_peered":"2024-04-25T10:29:28.236665+0000","last_clean":"2024-04-25T10:29:28.236665+0000","last_became_active":"2024-04-25T10:29:27.239303+0000","last_became_peered":"2024-04-25T10:29:27.239303+0000","last_unstale":"2024-04-25T10:29:28.236665+0000","last_undegraded":"2024-04-25T10:29:28.236665+0000","last_fullsized":"2024-04-25T10:29:28.236665+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:21.717976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315690+0000","last_change":"2024-04-25T10:29:27.241931+0000","last_active":"2024-04-25T10:29:28.315690+0000","last_peered":"2024-04-25T10:29:28.315690+0000","last_clean":"2024-04-25T10:29:28.315690+0000","last_became_active":"2024-04-25T10:29:27.241469+0000","last_became_peered":"2024-04-25T10:29:27.241469+0000","last_unstale":"2024-04-25T10:29:28.315690+0000","last_undegraded":"2024-04-25T10:29:28.315690+0000","last_fullsized":"2024-04-25T10:29:28.315690+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:56.497804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.247984+0000","last_change":"2024-04-25T10:29:27.243207+0000","last_active":"2024-04-25T10:29:27.247984+0000","last_peered":"2024-04-25T10:29:27.247984+0000","last_clean":"2024-04-25T10:29:27.247984+0000","last_became_active":"2024-04-25T10:29:27.242627+0000","last_became_peered":"2024-04-25T10:29:27.242627+0000","last_unstale":"2024-04-25T10:29:27.247984+0000","last_undegraded":"2024-04-25T10:29:27.247984+0000","last_fullsized":"2024-04-25T10:29:27.247984+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:10.243157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"154'4","reported_seq":18,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237997+0000","last_change":"2024-04-25T10:29:26.222270+0000","last_active":"2024-04-25T10:29:28.237997+0000","last_peered":"2024-04-25T10:29:28.237997+0000","last_clean":"2024-04-25T10:29:28.237997+0000","last_became_active":"2024-04-25T10:29:26.221733+0000","last_became_peered":"2024-04-25T10:29:26.221733+0000","last_unstale":"2024-04-25T10:29:28.237997+0000","last_undegraded":"2024-04-25T10:29:28.237997+0000","last_fullsized":"2024-04-25T10:29:28.237997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:38.604579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237979+0000","last_change":"2024-04-25T10:29:27.227435+0000","last_active":"2024-04-25T10:29:28.237979+0000","last_peered":"2024-04-25T10:29:28.237979+0000","last_clean":"2024-04-25T10:29:28.237979+0000","last_became_active":"2024-04-25T10:29:27.226826+0000","last_became_peered":"2024-04-25T10:29:27.226826+0000","last_unstale":"2024-04-25T10:29:28.237979+0000","last_undegraded":"2024-04-25T10:29:28.237979+0000","last_fullsized":"2024-04-25T10:29:28.237979+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:10.295388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312909+0000","last_change":"2024-04-25T10:29:26.217703+0000","last_active":"2024-04-25T10:29:28.312909+0000","last_peered":"2024-04-25T10:29:28.312909+0000","last_clean":"2024-04-25T10:29:28.312909+0000","last_became_active":"2024-04-25T10:29:26.217441+0000","last_became_peered":"2024-04-25T10:29:26.217441+0000","last_unstale":"2024-04-25T10:29:28.312909+0000","last_undegraded":"2024-04-25T10:29:28.312909+0000","last_fullsized":"2024-04-25T10:29:28.312909+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.283285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315732+0000","last_change":"2024-04-25T10:29:27.230550+0000","last_active":"2024-04-25T10:29:28.315732+0000","last_peered":"2024-04-25T10:29:28.315732+0000","last_clean":"2024-04-25T10:29:28.315732+0000","last_became_active":"2024-04-25T10:29:27.229291+0000","last_became_peered":"2024-04-25T10:29:27.229291+0000","last_unstale":"2024-04-25T10:29:28.315732+0000","last_undegraded":"2024-04-25T10:29:28.315732+0000","last_fullsized":"2024-04-25T10:29:28.315732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:20.949126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315858+0000","last_change":"2024-04-25T10:29:26.223811+0000","last_active":"2024-04-25T10:29:28.315858+0000","last_peered":"2024-04-25T10:29:28.315858+0000","last_clean":"2024-04-25T10:29:28.315858+0000","last_became_active":"2024-04-25T10:29:26.223494+0000","last_became_peered":"2024-04-25T10:29:26.223494+0000","last_unstale":"2024-04-25T10:29:28.315858+0000","last_undegraded":"2024-04-25T10:29:28.315858+0000","last_fullsized":"2024-04-25T10:29:28.315858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:05.410789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312910+0000","last_change":"2024-04-25T10:29:27.240278+0000","last_active":"2024-04-25T10:29:28.312910+0000","last_peered":"2024-04-25T10:29:28.312910+0000","last_clean":"2024-04-25T10:29:28.312910+0000","last_became_active":"2024-04-25T10:29:27.239965+0000","last_became_peered":"2024-04-25T10:29:27.239965+0000","last_unstale":"2024-04-25T10:29:28.312910+0000","last_undegraded":"2024-04-25T10:29:28.312910+0000","last_fullsized":"2024-04-25T10:29:28.312910+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:54.139015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240124+0000","last_change":"2024-04-25T10:29:26.212077+0000","last_active":"2024-04-25T10:29:28.240124+0000","last_peered":"2024-04-25T10:29:28.240124+0000","last_clean":"2024-04-25T10:29:28.240124+0000","last_became_active":"2024-04-25T10:29:26.211732+0000","last_became_peered":"2024-04-25T10:29:26.211732+0000","last_unstale":"2024-04-25T10:29:28.240124+0000","last_undegraded":"2024-04-25T10:29:28.240124+0000","last_fullsized":"2024-04-25T10:29:28.240124+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:53.131423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.245975+0000","last_change":"2024-04-25T10:29:27.241241+0000","last_active":"2024-04-25T10:29:27.245975+0000","last_peered":"2024-04-25T10:29:27.245975+0000","last_clean":"2024-04-25T10:29:27.245975+0000","last_became_active":"2024-04-25T10:29:27.240978+0000","last_became_peered":"2024-04-25T10:29:27.240978+0000","last_unstale":"2024-04-25T10:29:27.245975+0000","last_undegraded":"2024-04-25T10:29:27.245975+0000","last_fullsized":"2024-04-25T10:29:27.245975+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:22.860530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237058+0000","last_change":"2024-04-25T10:29:26.219489+0000","last_active":"2024-04-25T10:29:28.237058+0000","last_peered":"2024-04-25T10:29:28.237058+0000","last_clean":"2024-04-25T10:29:28.237058+0000","last_became_active":"2024-04-25T10:29:26.218116+0000","last_became_peered":"2024-04-25T10:29:26.218116+0000","last_unstale":"2024-04-25T10:29:28.237058+0000","last_undegraded":"2024-04-25T10:29:28.237058+0000","last_fullsized":"2024-04-25T10:29:28.237058+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:34.018654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315743+0000","last_change":"2024-04-25T10:29:27.245342+0000","last_active":"2024-04-25T10:29:28.315743+0000","last_peered":"2024-04-25T10:29:28.315743+0000","last_clean":"2024-04-25T10:29:28.315743+0000","last_became_active":"2024-04-25T10:29:27.244863+0000","last_became_peered":"2024-04-25T10:29:27.244863+0000","last_unstale":"2024-04-25T10:29:28.315743+0000","last_undegraded":"2024-04-25T10:29:28.315743+0000","last_fullsized":"2024-04-25T10:29:28.315743+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:16.387951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.254141+0000","last_change":"2024-04-25T10:29:26.228653+0000","last_active":"2024-04-25T10:29:27.254141+0000","last_peered":"2024-04-25T10:29:27.254141+0000","last_clean":"2024-04-25T10:29:27.254141+0000","last_became_active":"2024-04-25T10:29:26.228453+0000","last_became_peered":"2024-04-25T10:29:26.228453+0000","last_unstale":"2024-04-25T10:29:27.254141+0000","last_undegraded":"2024-04-25T10:29:27.254141+0000","last_fullsized":"2024-04-25T10:29:27.254141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:44.983715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.239364+0000","last_change":"2024-04-25T10:29:27.224009+0000","last_active":"2024-04-25T10:29:28.239364+0000","last_peered":"2024-04-25T10:29:28.239364+0000","last_clean":"2024-04-25T10:29:28.239364+0000","last_became_active":"2024-04-25T10:29:27.223721+0000","last_became_peered":"2024-04-25T10:29:27.223721+0000","last_unstale":"2024-04-25T10:29:28.239364+0000","last_undegraded":"2024-04-25T10:29:28.239364+0000","last_fullsized":"2024-04-25T10:29:28.239364+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:18.195645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236691+0000","last_change":"2024-04-25T10:29:26.217124+0000","last_active":"2024-04-25T10:29:28.236691+0000","last_peered":"2024-04-25T10:29:28.236691+0000","last_clean":"2024-04-25T10:29:28.236691+0000","last_became_active":"2024-04-25T10:29:26.216321+0000","last_became_peered":"2024-04-25T10:29:26.216321+0000","last_unstale":"2024-04-25T10:29:28.236691+0000","last_undegraded":"2024-04-25T10:29:28.236691+0000","last_fullsized":"2024-04-25T10:29:28.236691+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:53.131423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.246095+0000","last_change":"2024-04-25T10:29:27.242267+0000","last_active":"2024-04-25T10:29:27.246095+0000","last_peered":"2024-04-25T10:29:27.246095+0000","last_clean":"2024-04-25T10:29:27.246095+0000","last_became_active":"2024-04-25T10:29:27.241072+0000","last_became_peered":"2024-04-25T10:29:27.241072+0000","last_unstale":"2024-04-25T10:29:27.246095+0000","last_undegraded":"2024-04-25T10:29:27.246095+0000","last_fullsized":"2024-04-25T10:29:27.246095+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.656561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.239475+0000","last_change":"2024-04-25T10:29:26.217253+0000","last_active":"2024-04-25T10:29:28.239475+0000","last_peered":"2024-04-25T10:29:28.239475+0000","last_clean":"2024-04-25T10:29:28.239475+0000","last_became_active":"2024-04-25T10:29:26.216959+0000","last_became_peered":"2024-04-25T10:29:26.216959+0000","last_unstale":"2024-04-25T10:29:28.239475+0000","last_undegraded":"2024-04-25T10:29:28.239475+0000","last_fullsized":"2024-04-25T10:29:28.239475+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.938583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315634+0000","last_change":"2024-04-25T10:29:27.231661+0000","last_active":"2024-04-25T10:29:28.315634+0000","last_peered":"2024-04-25T10:29:28.315634+0000","last_clean":"2024-04-25T10:29:28.315634+0000","last_became_active":"2024-04-25T10:29:27.231371+0000","last_became_peered":"2024-04-25T10:29:27.231371+0000","last_unstale":"2024-04-25T10:29:28.315634+0000","last_undegraded":"2024-04-25T10:29:28.315634+0000","last_fullsized":"2024-04-25T10:29:28.315634+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:23.026428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316070+0000","last_change":"2024-04-25T10:29:26.227440+0000","last_active":"2024-04-25T10:29:28.316070+0000","last_peered":"2024-04-25T10:29:28.316070+0000","last_clean":"2024-04-25T10:29:28.316070+0000","last_became_active":"2024-04-25T10:29:26.227087+0000","last_became_peered":"2024-04-25T10:29:26.227087+0000","last_unstale":"2024-04-25T10:29:28.316070+0000","last_undegraded":"2024-04-25T10:29:28.316070+0000","last_fullsized":"2024-04-25T10:29:28.316070+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:41.068171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315809+0000","last_change":"2024-04-25T10:29:27.235142+0000","last_active":"2024-04-25T10:29:28.315809+0000","last_peered":"2024-04-25T10:29:28.315809+0000","last_clean":"2024-04-25T10:29:28.315809+0000","last_became_active":"2024-04-25T10:29:27.233457+0000","last_became_peered":"2024-04-25T10:29:27.233457+0000","last_unstale":"2024-04-25T10:29:28.315809+0000","last_undegraded":"2024-04-25T10:29:28.315809+0000","last_fullsized":"2024-04-25T10:29:28.315809+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.088588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"154'3","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240227+0000","last_change":"2024-04-25T10:29:26.216000+0000","last_active":"2024-04-25T10:29:28.240227+0000","last_peered":"2024-04-25T10:29:28.240227+0000","last_clean":"2024-04-25T10:29:28.240227+0000","last_became_active":"2024-04-25T10:29:26.215762+0000","last_became_peered":"2024-04-25T10:29:26.215762+0000","last_unstale":"2024-04-25T10:29:28.240227+0000","last_undegraded":"2024-04-25T10:29:28.240227+0000","last_fullsized":"2024-04-25T10:29:28.240227+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:57.135919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237213+0000","last_change":"2024-04-25T10:29:27.249555+0000","last_active":"2024-04-25T10:29:28.237213+0000","last_peered":"2024-04-25T10:29:28.237213+0000","last_clean":"2024-04-25T10:29:28.237213+0000","last_became_active":"2024-04-25T10:29:27.247104+0000","last_became_peered":"2024-04-25T10:29:27.247104+0000","last_unstale":"2024-04-25T10:29:28.237213+0000","last_undegraded":"2024-04-25T10:29:28.237213+0000","last_fullsized":"2024-04-25T10:29:28.237213+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:35.097185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"154'6","reported_seq":20,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233240+0000","last_change":"2024-04-25T10:29:26.213436+0000","last_active":"2024-04-25T10:29:28.233240+0000","last_peered":"2024-04-25T10:29:28.233240+0000","last_clean":"2024-04-25T10:29:28.233240+0000","last_became_active":"2024-04-25T10:29:26.213134+0000","last_became_peered":"2024-04-25T10:29:26.213134+0000","last_unstale":"2024-04-25T10:29:28.233240+0000","last_undegraded":"2024-04-25T10:29:28.233240+0000","last_fullsized":"2024-04-25T10:29:28.233240+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:05.054569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315408+0000","last_change":"2024-04-25T10:29:27.227667+0000","last_active":"2024-04-25T10:29:28.315408+0000","last_peered":"2024-04-25T10:29:28.315408+0000","last_clean":"2024-04-25T10:29:28.315408+0000","last_became_active":"2024-04-25T10:29:27.227411+0000","last_became_peered":"2024-04-25T10:29:27.227411+0000","last_unstale":"2024-04-25T10:29:28.315408+0000","last_undegraded":"2024-04-25T10:29:28.315408+0000","last_fullsized":"2024-04-25T10:29:28.315408+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:55.563643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.250668+0000","last_change":"2024-04-25T10:29:26.221865+0000","last_active":"2024-04-25T10:29:27.250668+0000","last_peered":"2024-04-25T10:29:27.250668+0000","last_clean":"2024-04-25T10:29:27.250668+0000","last_became_active":"2024-04-25T10:29:26.221570+0000","last_became_peered":"2024-04-25T10:29:26.221570+0000","last_unstale":"2024-04-25T10:29:27.250668+0000","last_undegraded":"2024-04-25T10:29:27.250668+0000","last_fullsized":"2024-04-25T10:29:27.250668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:38.606592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240054+0000","last_change":"2024-04-25T10:29:27.226960+0000","last_active":"2024-04-25T10:29:28.240054+0000","last_peered":"2024-04-25T10:29:28.240054+0000","last_clean":"2024-04-25T10:29:28.240054+0000","last_became_active":"2024-04-25T10:29:27.226633+0000","last_became_peered":"2024-04-25T10:29:27.226633+0000","last_unstale":"2024-04-25T10:29:28.240054+0000","last_undegraded":"2024-04-25T10:29:28.240054+0000","last_fullsized":"2024-04-25T10:29:28.240054+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:45.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315841+0000","last_change":"2024-04-25T10:29:26.218770+0000","last_active":"2024-04-25T10:29:28.315841+0000","last_peered":"2024-04-25T10:29:28.315841+0000","last_clean":"2024-04-25T10:29:28.315841+0000","last_became_active":"2024-04-25T10:29:26.218502+0000","last_became_peered":"2024-04-25T10:29:26.218502+0000","last_unstale":"2024-04-25T10:29:28.315841+0000","last_undegraded":"2024-04-25T10:29:28.315841+0000","last_fullsized":"2024-04-25T10:29:28.315841+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:21.184436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316023+0000","last_change":"2024-04-25T10:29:27.244813+0000","last_active":"2024-04-25T10:29:28.316023+0000","last_peered":"2024-04-25T10:29:28.316023+0000","last_clean":"2024-04-25T10:29:28.316023+0000","last_became_active":"2024-04-25T10:29:27.244549+0000","last_became_peered":"2024-04-25T10:29:27.244549+0000","last_unstale":"2024-04-25T10:29:28.316023+0000","last_undegraded":"2024-04-25T10:29:28.316023+0000","last_fullsized":"2024-04-25T10:29:28.316023+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:59.277739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316485+0000","last_change":"2024-04-25T10:29:26.218251+0000","last_active":"2024-04-25T10:29:28.316485+0000","last_peered":"2024-04-25T10:29:28.316485+0000","last_clean":"2024-04-25T10:29:28.316485+0000","last_became_active":"2024-04-25T10:29:26.217800+0000","last_became_peered":"2024-04-25T10:29:26.217800+0000","last_unstale":"2024-04-25T10:29:28.316485+0000","last_undegraded":"2024-04-25T10:29:28.316485+0000","last_fullsized":"2024-04-25T10:29:28.316485+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:29.092643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312503+0000","last_change":"2024-04-25T10:29:27.239441+0000","last_active":"2024-04-25T10:29:28.312503+0000","last_peered":"2024-04-25T10:29:28.312503+0000","last_clean":"2024-04-25T10:29:28.312503+0000","last_became_active":"2024-04-25T10:29:27.238516+0000","last_became_peered":"2024-04-25T10:29:27.238516+0000","last_unstale":"2024-04-25T10:29:28.312503+0000","last_undegraded":"2024-04-25T10:29:28.312503+0000","last_fullsized":"2024-04-25T10:29:28.312503+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:58.143511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"154'8","reported_seq":22,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232758+0000","last_change":"2024-04-25T10:29:26.219265+0000","last_active":"2024-04-25T10:29:28.232758+0000","last_peered":"2024-04-25T10:29:28.232758+0000","last_clean":"2024-04-25T10:29:28.232758+0000","last_became_active":"2024-04-25T10:29:26.218998+0000","last_became_peered":"2024-04-25T10:29:26.218998+0000","last_unstale":"2024-04-25T10:29:28.232758+0000","last_undegraded":"2024-04-25T10:29:28.232758+0000","last_fullsized":"2024-04-25T10:29:28.232758+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:23.469280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316095+0000","last_change":"2024-04-25T10:29:27.229323+0000","last_active":"2024-04-25T10:29:28.316095+0000","last_peered":"2024-04-25T10:29:28.316095+0000","last_clean":"2024-04-25T10:29:28.316095+0000","last_became_active":"2024-04-25T10:29:27.228385+0000","last_became_peered":"2024-04-25T10:29:27.228385+0000","last_unstale":"2024-04-25T10:29:28.316095+0000","last_undegraded":"2024-04-25T10:29:28.316095+0000","last_fullsized":"2024-04-25T10:29:28.316095+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:47.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233581+0000","last_change":"2024-04-25T10:29:26.215028+0000","last_active":"2024-04-25T10:29:28.233581+0000","last_peered":"2024-04-25T10:29:28.233581+0000","last_clean":"2024-04-25T10:29:28.233581+0000","last_became_active":"2024-04-25T10:29:26.214731+0000","last_became_peered":"2024-04-25T10:29:26.214731+0000","last_unstale":"2024-04-25T10:29:28.233581+0000","last_undegraded":"2024-04-25T10:29:28.233581+0000","last_fullsized":"2024-04-25T10:29:28.233581+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:27.429001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.248054+0000","last_change":"2024-04-25T10:29:27.244058+0000","last_active":"2024-04-25T10:29:27.248054+0000","last_peered":"2024-04-25T10:29:27.248054+0000","last_clean":"2024-04-25T10:29:27.248054+0000","last_became_active":"2024-04-25T10:29:27.243654+0000","last_became_peered":"2024-04-25T10:29:27.243654+0000","last_unstale":"2024-04-25T10:29:27.248054+0000","last_undegraded":"2024-04-25T10:29:27.248054+0000","last_fullsized":"2024-04-25T10:29:27.248054+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:51.922293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.312964+0000","last_change":"2024-04-25T10:29:26.218430+0000","last_active":"2024-04-25T10:29:28.312964+0000","last_peered":"2024-04-25T10:29:28.312964+0000","last_clean":"2024-04-25T10:29:28.312964+0000","last_became_active":"2024-04-25T10:29:26.217159+0000","last_became_peered":"2024-04-25T10:29:26.217159+0000","last_unstale":"2024-04-25T10:29:28.312964+0000","last_undegraded":"2024-04-25T10:29:28.312964+0000","last_fullsized":"2024-04-25T10:29:28.312964+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:57.730302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.244990+0000","last_change":"2024-04-25T10:29:27.242205+0000","last_active":"2024-04-25T10:29:27.244990+0000","last_peered":"2024-04-25T10:29:27.244990+0000","last_clean":"2024-04-25T10:29:27.244990+0000","last_became_active":"2024-04-25T10:29:27.240476+0000","last_became_peered":"2024-04-25T10:29:27.240476+0000","last_unstale":"2024-04-25T10:29:27.244990+0000","last_undegraded":"2024-04-25T10:29:27.244990+0000","last_fullsized":"2024-04-25T10:29:27.244990+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:02.795407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"154'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233585+0000","last_change":"2024-04-25T10:29:26.214982+0000","last_active":"2024-04-25T10:29:28.233585+0000","last_peered":"2024-04-25T10:29:28.233585+0000","last_clean":"2024-04-25T10:29:28.233585+0000","last_became_active":"2024-04-25T10:29:26.214462+0000","last_became_peered":"2024-04-25T10:29:26.214462+0000","last_unstale":"2024-04-25T10:29:28.233585+0000","last_undegraded":"2024-04-25T10:29:28.233585+0000","last_fullsized":"2024-04-25T10:29:28.233585+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:25.728776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240177+0000","last_change":"2024-04-25T10:29:27.232404+0000","last_active":"2024-04-25T10:29:28.240177+0000","last_peered":"2024-04-25T10:29:28.240177+0000","last_clean":"2024-04-25T10:29:28.240177+0000","last_became_active":"2024-04-25T10:29:27.232136+0000","last_became_peered":"2024-04-25T10:29:27.232136+0000","last_unstale":"2024-04-25T10:29:28.240177+0000","last_undegraded":"2024-04-25T10:29:28.240177+0000","last_fullsized":"2024-04-25T10:29:28.240177+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:34.802262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237016+0000","last_change":"2024-04-25T10:29:26.223035+0000","last_active":"2024-04-25T10:29:28.237016+0000","last_peered":"2024-04-25T10:29:28.237016+0000","last_clean":"2024-04-25T10:29:28.237016+0000","last_became_active":"2024-04-25T10:29:26.222148+0000","last_became_peered":"2024-04-25T10:29:26.222148+0000","last_unstale":"2024-04-25T10:29:28.237016+0000","last_undegraded":"2024-04-25T10:29:28.237016+0000","last_fullsized":"2024-04-25T10:29:28.237016+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:41.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236772+0000","last_change":"2024-04-25T10:29:27.240797+0000","last_active":"2024-04-25T10:29:28.236772+0000","last_peered":"2024-04-25T10:29:28.236772+0000","last_clean":"2024-04-25T10:29:28.236772+0000","last_became_active":"2024-04-25T10:29:27.240436+0000","last_became_peered":"2024-04-25T10:29:27.240436+0000","last_unstale":"2024-04-25T10:29:28.236772+0000","last_undegraded":"2024-04-25T10:29:28.236772+0000","last_fullsized":"2024-04-25T10:29:28.236772+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:48.345543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240204+0000","last_change":"2024-04-25T10:29:26.225575+0000","last_active":"2024-04-25T10:29:28.240204+0000","last_peered":"2024-04-25T10:29:28.240204+0000","last_clean":"2024-04-25T10:29:28.240204+0000","last_became_active":"2024-04-25T10:29:26.225279+0000","last_became_peered":"2024-04-25T10:29:26.225279+0000","last_unstale":"2024-04-25T10:29:28.240204+0000","last_undegraded":"2024-04-25T10:29:28.240204+0000","last_fullsized":"2024-04-25T10:29:28.240204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:45.188700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240406+0000","last_change":"2024-04-25T10:29:27.242547+0000","last_active":"2024-04-25T10:29:28.240406+0000","last_peered":"2024-04-25T10:29:28.240406+0000","last_clean":"2024-04-25T10:29:28.240406+0000","last_became_active":"2024-04-25T10:29:27.242054+0000","last_became_peered":"2024-04-25T10:29:27.242054+0000","last_unstale":"2024-04-25T10:29:28.240406+0000","last_undegraded":"2024-04-25T10:29:28.240406+0000","last_fullsized":"2024-04-25T10:29:28.240406+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:38.864445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236374+0000","last_change":"2024-04-25T10:29:26.218404+0000","last_active":"2024-04-25T10:29:28.236374+0000","last_peered":"2024-04-25T10:29:28.236374+0000","last_clean":"2024-04-25T10:29:28.236374+0000","last_became_active":"2024-04-25T10:29:26.218125+0000","last_became_peered":"2024-04-25T10:29:26.218125+0000","last_unstale":"2024-04-25T10:29:28.236374+0000","last_undegraded":"2024-04-25T10:29:28.236374+0000","last_fullsized":"2024-04-25T10:29:28.236374+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:32.559695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236342+0000","last_change":"2024-04-25T10:29:27.233890+0000","last_active":"2024-04-25T10:29:28.236342+0000","last_peered":"2024-04-25T10:29:28.236342+0000","last_clean":"2024-04-25T10:29:28.236342+0000","last_became_active":"2024-04-25T10:29:27.233636+0000","last_became_peered":"2024-04-25T10:29:27.233636+0000","last_unstale":"2024-04-25T10:29:28.236342+0000","last_undegraded":"2024-04-25T10:29:28.236342+0000","last_fullsized":"2024-04-25T10:29:28.236342+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.441851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.236083+0000","last_change":"2024-04-25T10:29:26.225860+0000","last_active":"2024-04-25T10:29:27.236083+0000","last_peered":"2024-04-25T10:29:27.236083+0000","last_clean":"2024-04-25T10:29:27.236083+0000","last_became_active":"2024-04-25T10:29:26.224139+0000","last_became_peered":"2024-04-25T10:29:26.224139+0000","last_unstale":"2024-04-25T10:29:27.236083+0000","last_undegraded":"2024-04-25T10:29:27.236083+0000","last_fullsized":"2024-04-25T10:29:27.236083+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:17.718892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316250+0000","last_change":"2024-04-25T10:29:27.234499+0000","last_active":"2024-04-25T10:29:28.316250+0000","last_peered":"2024-04-25T10:29:28.316250+0000","last_clean":"2024-04-25T10:29:28.316250+0000","last_became_active":"2024-04-25T10:29:27.233878+0000","last_became_peered":"2024-04-25T10:29:27.233878+0000","last_unstale":"2024-04-25T10:29:28.316250+0000","last_undegraded":"2024-04-25T10:29:28.316250+0000","last_fullsized":"2024-04-25T10:29:28.316250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:45.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236348+0000","last_change":"2024-04-25T10:29:26.223768+0000","last_active":"2024-04-25T10:29:28.236348+0000","last_peered":"2024-04-25T10:29:28.236348+0000","last_clean":"2024-04-25T10:29:28.236348+0000","last_became_active":"2024-04-25T10:29:26.223294+0000","last_became_peered":"2024-04-25T10:29:26.223294+0000","last_unstale":"2024-04-25T10:29:28.236348+0000","last_undegraded":"2024-04-25T10:29:28.236348+0000","last_fullsized":"2024-04-25T10:29:28.236348+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:39.282561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240532+0000","last_change":"2024-04-25T10:29:27.232600+0000","last_active":"2024-04-25T10:29:28.240532+0000","last_peered":"2024-04-25T10:29:28.240532+0000","last_clean":"2024-04-25T10:29:28.240532+0000","last_became_active":"2024-04-25T10:29:27.232009+0000","last_became_peered":"2024-04-25T10:29:27.232009+0000","last_unstale":"2024-04-25T10:29:28.240532+0000","last_undegraded":"2024-04-25T10:29:28.240532+0000","last_fullsized":"2024-04-25T10:29:28.240532+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:49.689247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"154'2","reported_seq":16,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.316475+0000","last_change":"2024-04-25T10:29:26.224020+0000","last_active":"2024-04-25T10:29:28.316475+0000","last_peered":"2024-04-25T10:29:28.316475+0000","last_clean":"2024-04-25T10:29:28.316475+0000","last_became_active":"2024-04-25T10:29:26.223709+0000","last_became_peered":"2024-04-25T10:29:26.223709+0000","last_unstale":"2024-04-25T10:29:28.316475+0000","last_undegraded":"2024-04-25T10:29:28.316475+0000","last_fullsized":"2024-04-25T10:29:28.316475+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:06.076162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.245934+0000","last_change":"2024-04-25T10:29:27.239292+0000","last_active":"2024-04-25T10:29:27.245934+0000","last_peered":"2024-04-25T10:29:27.245934+0000","last_clean":"2024-04-25T10:29:27.245934+0000","last_became_active":"2024-04-25T10:29:27.238797+0000","last_became_peered":"2024-04-25T10:29:27.238797+0000","last_unstale":"2024-04-25T10:29:27.245934+0000","last_undegraded":"2024-04-25T10:29:27.245934+0000","last_fullsized":"2024-04-25T10:29:27.245934+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:03.085959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"155'1","reported_seq":15,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.238336+0000","last_change":"2024-04-25T10:29:26.222859+0000","last_active":"2024-04-25T10:29:28.238336+0000","last_peered":"2024-04-25T10:29:28.238336+0000","last_clean":"2024-04-25T10:29:28.238336+0000","last_became_active":"2024-04-25T10:29:26.222585+0000","last_became_peered":"2024-04-25T10:29:26.222585+0000","last_unstale":"2024-04-25T10:29:28.238336+0000","last_undegraded":"2024-04-25T10:29:28.238336+0000","last_fullsized":"2024-04-25T10:29:28.238336+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:21.561672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.247041+0000","last_change":"2024-04-25T10:29:27.242834+0000","last_active":"2024-04-25T10:29:27.247041+0000","last_peered":"2024-04-25T10:29:27.247041+0000","last_clean":"2024-04-25T10:29:27.247041+0000","last_became_active":"2024-04-25T10:29:27.241231+0000","last_became_peered":"2024-04-25T10:29:27.241231+0000","last_unstale":"2024-04-25T10:29:27.247041+0000","last_undegraded":"2024-04-25T10:29:27.247041+0000","last_fullsized":"2024-04-25T10:29:27.247041+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:51.681540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.237398+0000","last_change":"2024-04-25T10:29:26.221049+0000","last_active":"2024-04-25T10:29:28.237398+0000","last_peered":"2024-04-25T10:29:28.237398+0000","last_clean":"2024-04-25T10:29:28.237398+0000","last_became_active":"2024-04-25T10:29:26.218977+0000","last_became_peered":"2024-04-25T10:29:26.218977+0000","last_unstale":"2024-04-25T10:29:28.237398+0000","last_undegraded":"2024-04-25T10:29:28.237398+0000","last_fullsized":"2024-04-25T10:29:28.237398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:22.018836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T10:29:27.251199+0000","last_change":"2024-04-25T10:29:27.249715+0000","last_active":"2024-04-25T10:29:27.251199+0000","last_peered":"2024-04-25T10:29:27.251199+0000","last_clean":"2024-04-25T10:29:27.251199+0000","last_became_active":"2024-04-25T10:29:27.249449+0000","last_became_peered":"2024-04-25T10:29:27.249449+0000","last_unstale":"2024-04-25T10:29:27.251199+0000","last_undegraded":"2024-04-25T10:29:27.251199+0000","last_fullsized":"2024-04-25T10:29:27.251199+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:30.310975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.240263+0000","last_change":"2024-04-25T10:29:26.216306+0000","last_active":"2024-04-25T10:29:28.240263+0000","last_peered":"2024-04-25T10:29:28.240263+0000","last_clean":"2024-04-25T10:29:28.240263+0000","last_became_active":"2024-04-25T10:29:26.214802+0000","last_became_peered":"2024-04-25T10:29:26.214802+0000","last_unstale":"2024-04-25T10:29:28.240263+0000","last_undegraded":"2024-04-25T10:29:28.240263+0000","last_fullsized":"2024-04-25T10:29:28.240263+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:22.725339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315710+0000","last_change":"2024-04-25T10:29:27.241706+0000","last_active":"2024-04-25T10:29:28.315710+0000","last_peered":"2024-04-25T10:29:28.315710+0000","last_clean":"2024-04-25T10:29:28.315710+0000","last_became_active":"2024-04-25T10:29:27.241412+0000","last_became_peered":"2024-04-25T10:29:27.241412+0000","last_unstale":"2024-04-25T10:29:28.315710+0000","last_undegraded":"2024-04-25T10:29:28.315710+0000","last_fullsized":"2024-04-25T10:29:28.315710+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:51.290877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233171+0000","last_change":"2024-04-25T10:29:26.219570+0000","last_active":"2024-04-25T10:29:28.233171+0000","last_peered":"2024-04-25T10:29:28.233171+0000","last_clean":"2024-04-25T10:29:28.233171+0000","last_became_active":"2024-04-25T10:29:26.219115+0000","last_became_peered":"2024-04-25T10:29:26.219115+0000","last_unstale":"2024-04-25T10:29:28.233171+0000","last_undegraded":"2024-04-25T10:29:28.233171+0000","last_fullsized":"2024-04-25T10:29:28.233171+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:58.765836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236949+0000","last_change":"2024-04-25T10:29:27.232984+0000","last_active":"2024-04-25T10:29:28.236949+0000","last_peered":"2024-04-25T10:29:28.236949+0000","last_clean":"2024-04-25T10:29:28.236949+0000","last_became_active":"2024-04-25T10:29:27.232567+0000","last_became_peered":"2024-04-25T10:29:27.232567+0000","last_unstale":"2024-04-25T10:29:28.236949+0000","last_undegraded":"2024-04-25T10:29:28.236949+0000","last_fullsized":"2024-04-25T10:29:28.236949+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:34.759558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315268+0000","last_change":"2024-04-25T10:29:26.226836+0000","last_active":"2024-04-25T10:29:28.315268+0000","last_peered":"2024-04-25T10:29:28.315268+0000","last_clean":"2024-04-25T10:29:28.315268+0000","last_became_active":"2024-04-25T10:29:26.226573+0000","last_became_peered":"2024-04-25T10:29:26.226573+0000","last_unstale":"2024-04-25T10:29:28.315268+0000","last_undegraded":"2024-04-25T10:29:28.315268+0000","last_fullsized":"2024-04-25T10:29:28.315268+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:58.270147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315300+0000","last_change":"2024-04-25T10:29:27.243036+0000","last_active":"2024-04-25T10:29:28.315300+0000","last_peered":"2024-04-25T10:29:28.315300+0000","last_clean":"2024-04-25T10:29:28.315300+0000","last_became_active":"2024-04-25T10:29:27.242736+0000","last_became_peered":"2024-04-25T10:29:27.242736+0000","last_unstale":"2024-04-25T10:29:28.315300+0000","last_undegraded":"2024-04-25T10:29:28.315300+0000","last_fullsized":"2024-04-25T10:29:28.315300+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:35.026246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236743+0000","last_change":"2024-04-25T10:29:26.221470+0000","last_active":"2024-04-25T10:29:28.236743+0000","last_peered":"2024-04-25T10:29:28.236743+0000","last_clean":"2024-04-25T10:29:28.236743+0000","last_became_active":"2024-04-25T10:29:26.219461+0000","last_became_peered":"2024-04-25T10:29:26.219461+0000","last_unstale":"2024-04-25T10:29:28.236743+0000","last_undegraded":"2024-04-25T10:29:28.236743+0000","last_fullsized":"2024-04-25T10:29:28.236743+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:25.410538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236603+0000","last_change":"2024-04-25T10:29:27.232875+0000","last_active":"2024-04-25T10:29:28.236603+0000","last_peered":"2024-04-25T10:29:28.236603+0000","last_clean":"2024-04-25T10:29:28.236603+0000","last_became_active":"2024-04-25T10:29:27.232569+0000","last_became_peered":"2024-04-25T10:29:27.232569+0000","last_unstale":"2024-04-25T10:29:28.236603+0000","last_undegraded":"2024-04-25T10:29:28.236603+0000","last_fullsized":"2024-04-25T10:29:28.236603+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:54.150228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.236509+0000","last_change":"2024-04-25T10:29:26.217515+0000","last_active":"2024-04-25T10:29:28.236509+0000","last_peered":"2024-04-25T10:29:28.236509+0000","last_clean":"2024-04-25T10:29:28.236509+0000","last_became_active":"2024-04-25T10:29:26.216855+0000","last_became_peered":"2024-04-25T10:29:26.216855+0000","last_unstale":"2024-04-25T10:29:28.236509+0000","last_undegraded":"2024-04-25T10:29:28.236509+0000","last_fullsized":"2024-04-25T10:29:28.236509+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:22.725339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.315961+0000","last_change":"2024-04-25T10:29:27.245408+0000","last_active":"2024-04-25T10:29:28.315961+0000","last_peered":"2024-04-25T10:29:28.315961+0000","last_clean":"2024-04-25T10:29:28.315961+0000","last_became_active":"2024-04-25T10:29:27.244939+0000","last_became_peered":"2024-04-25T10:29:27.244939+0000","last_unstale":"2024-04-25T10:29:28.315961+0000","last_undegraded":"2024-04-25T10:29:28.315961+0000","last_fullsized":"2024-04-25T10:29:28.315961+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:26.196523+0000","last_clean_scrub_stamp":"2024-04-25T10:29:26.196523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:18.195645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.233513+0000","last_change":"2024-04-25T10:29:26.215139+0000","last_active":"2024-04-25T10:29:28.233513+0000","last_peered":"2024-04-25T10:29:28.233513+0000","last_clean":"2024-04-25T10:29:28.233513+0000","last_became_active":"2024-04-25T10:29:26.214832+0000","last_became_peered":"2024-04-25T10:29:26.214832+0000","last_unstale":"2024-04-25T10:29:28.233513+0000","last_undegraded":"2024-04-25T10:29:28.233513+0000","last_fullsized":"2024-04-25T10:29:28.233513+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:29:25.188931+0000","last_clean_scrub_stamp":"2024-04-25T10:29:25.188931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:50.286005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":426,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T10:29:28.232803+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:29:28.232803+0000","last_peered":"2024-04-25T10:29:28.232803+0000","last_clean":"2024-04-25T10:29:28.232803+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:29:28.232803+0000","last_undegraded":"2024-04-25T10:29:28.232803+0000","last_fullsized":"2024-04-25T10:29:28.232803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55280,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316560,"statfs":{"total":96636764160,"available":96580157440,"internally_reserved":0,"allocated":1937408,"data_stored":848132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509584,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55276,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316564,"statfs":{"total":96636764160,"available":96580161536,"internally_reserved":0,"allocated":1929216,"data_stored":841866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509583,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60036,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2539520,"data_stored":1439172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46984,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324856,"statfs":{"total":96636764160,"available":96588652544,"internally_reserved":0,"allocated":1953792,"data_stored":849766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509583,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46964,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324876,"statfs":{"total":96636764160,"available":96588673024,"internally_reserved":0,"allocated":1941504,"data_stored":848714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509584,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47572,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324268,"statfs":{"total":96636764160,"available":96588050432,"internally_reserved":0,"allocated":2555904,"data_stored":1440294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55288,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316552,"statfs":{"total":96636764160,"available":96580149248,"internally_reserved":0,"allocated":1941504,"data_stored":848202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64920,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306920,"statfs":{"total":96636764160,"available":96570286080,"internally_reserved":0,"allocated":1912832,"data_stored":835057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:29:31.739 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:29:31.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:29:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 Processor -- start 2024-04-25T10:29:31.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 -- start start 2024-04-25T10:29:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 0x7fd0cc0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd0cc0b39d0 0x7fd0cc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc15e6e0 con 0x7fd0cc154f70 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc15e9c0 con 0x7fd0cc0b39d0 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.924+0000 7fd0d3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc15eca0 con 0x7fd0cc0d0070 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd0cc154f70 0x7fd0cc0cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60976/0 (socket says 172.21.15.45:60976) 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1200640 1 -- 172.21.15.45:0/3547792885 learned_addr learned my addr 172.21.15.45:0/3547792885 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cbe00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 0x7fd0cc0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 --2- 172.21.15.45:0/3547792885 >> v2:172.21.15.117:3300/0 conn(0x7fd0cc0b39d0 0x7fd0cc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569850721 0 0) 0x7fd0cc15e6e0 con 0x7fd0cc154f70 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b4003530 con 0x7fd0cc154f70 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 525744984 0 0) 0x7fd0b4003530 con 0x7fd0cc154f70 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 msgr2=0x7fd0cc0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 0x7fd0cc0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 >> v2:172.21.15.117:3300/0 conn(0x7fd0cc0b39d0 msgr2=0x7fd0cc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 --2- 172.21.15.45:0/3547792885 >> v2:172.21.15.117:3300/0 conn(0x7fd0cc0b39d0 0x7fd0cc0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc15ef80 con 0x7fd0cc154f70 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fd0c0001680 con 0x7fd0cc154f70 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 >> v1:172.21.15.45:6789/0 conn(0x7fd0cc154f70 legacy=0x7fd0cc0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40039c0 0x7fd0b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 0x7fd0b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b4004d50 0x7fd0b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc15e9c0 con 0x7fd0b4004d50 2024-04-25T10:29:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc15eca0 con 0x7fd0b40039c0 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b400b9d0 con 0x7fd0cc0d0070 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b4004d50 0x7fd0b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0cbe00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 0x7fd0b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1200640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40039c0 0x7fd0b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 -- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 msgr2=0x7fd0b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc0d0070 0x7fd0b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 -- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40039c0 msgr2=0x7fd0b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40039c0 0x7fd0b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.928+0000 7fd0d1c00640 1 -- 172.21.15.45:0/3547792885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4003530 con 0x7fd0b4004d50 2024-04-25T10:29:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d1c00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b4004d50 0x7fd0b40051d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e170 tx=0x7fd0c401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4150091b55ccbc65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40b2070 con 0x7fd0b4004d50 2024-04-25T10:29:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c40a5cb0 con 0x7fd0b4004d50 2024-04-25T10:29:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0cb400640 1 -- 172.21.15.45:0/3547792885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4009c10 con 0x7fd0b4004d50 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 -- 172.21.15.45:0/3547792885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b4004d50 msgr2=0x7fd0b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/3547792885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0b4004d50 0x7fd0b40051d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400e170 tx=0x7fd0c401cca0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d1c00640 1 -- 172.21.15.45:0/3547792885 reap_dead start 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 -- 172.21.15.45:0/3547792885 shutdown_connections 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 -- 172.21.15.45:0/3547792885 >> 172.21.15.45:0/3547792885 conn(0x7fd0cc066b70 msgr2=0x7fd0cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 -- 172.21.15.45:0/3547792885 shutdown_connections 2024-04-25T10:29:31.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 -- 172.21.15.45:0/3547792885 wait complete. 2024-04-25T10:29:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 Processor -- start 2024-04-25T10:29:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.932+0000 7fd0d3c00640 1 -- start start 2024-04-25T10:29:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b4004d50 0x7fd0cc149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 0x7fd0cc14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc158f90 0x7fd0cc159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b3490 con 0x7fd0cc137030 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc155350 con 0x7fd0b4004d50 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b3610 con 0x7fd0cc158f90 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 0x7fd0cc14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 0x7fd0cc14a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56876/0 (socket says 172.21.15.45:56876) 2024-04-25T10:29:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 -- 172.21.15.45:0/1851101519 learned_addr learned my addr 172.21.15.45:0/1851101519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d1200640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b4004d50 0x7fd0cc149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d1c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc158f90 0x7fd0cc159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 -- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc158f90 msgr2=0x7fd0cc159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc158f90 0x7fd0cc159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 -- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b4004d50 msgr2=0x7fd0cc149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:31.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b4004d50 0x7fd0cc149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 -- 172.21.15.45:0/1851101519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc14aee0 con 0x7fd0cc137030 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0cbe00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 0x7fd0cc14a690 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8011a30 tx=0x7fd0b8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800fb50 con 0x7fd0cc137030 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b800fcf0 con 0x7fd0cc137030 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800fea0 con 0x7fd0cc137030 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc14b1c0 con 0x7fd0cc137030 2024-04-25T10:29:31.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc14dab0 con 0x7fd0cc137030 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc04ef30 con 0x7fd0cc137030 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.936+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0b8012040 con 0x7fd0cc137030 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.940+0000 7fd0c8c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0ac066f60 0x7fd0ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.940+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0b80856a0 con 0x7fd0cc137030 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.940+0000 7fd0d1200640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0ac066f60 0x7fd0ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.940+0000 7fd0d1200640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0ac066f60 0x7fd0ac069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0000c10 tx=0x7fd0c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:31.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:31.944+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0b8019030 con 0x7fd0cc137030 2024-04-25T10:29:32.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.164+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd0cc1666c0 con 0x7fd0cc137030 2024-04-25T10:29:32.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.164+0000 7fd0c8c00640 1 -- 172.21.15.45:0/1851101519 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd0b8056710 con 0x7fd0cc137030 2024-04-25T10:29:32.166 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:29:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:29:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:29:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:29:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:29:32.167 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0ac066f60 msgr2=0x7fd0ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0ac066f60 0x7fd0ac069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0000c10 tx=0x7fd0c0009040 comp rx=0 tx=0).stop 2024-04-25T10:29:32.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 msgr2=0x7fd0cc14a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 0x7fd0cc14a690 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8011a30 tx=0x7fd0b8011ef0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 shutdown_connections 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0ac066f60 0x7fd0ac069420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc158f90 0x7fd0cc159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0cc137030 0x7fd0cc14a690 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 --2- 172.21.15.45:0/1851101519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b4004d50 0x7fd0cc149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 >> 172.21.15.45:0/1851101519 conn(0x7fd0cc066b70 msgr2=0x7fd0cc0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 shutdown_connections 2024-04-25T10:29:32.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.168+0000 7fd0d3c00640 1 -- 172.21.15.45:0/1851101519 wait complete. 2024-04-25T10:29:32.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:29:32.191 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-04-25T10:29:32.188+0000 7fd531400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:29:32.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 Processor -- start 2024-04-25T10:29:32.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 -- start start 2024-04-25T10:29:32.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52001550b0 0x7f52000cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f520015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5200071050 con 0x7f520015b740 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520015d040 con 0x7f52001550b0 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5208000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52000712f0 con 0x7f52000cfe70 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52001550b0 0x7f52000cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f52001550b0 0x7f52000cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56890/0 (socket says 172.21.15.45:56890) 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 -- 172.21.15.45:0/3973177590 learned_addr learned my addr 172.21.15.45:0/3973177590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5206000640 1 --1- 172.21.15.45:0/3973177590 >> v1:172.21.15.45:6789/0 conn(0x7f520015b740 0x7f520006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60986/0 (socket says 172.21.15.45:60986) 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5204c00640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f520015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 -- 172.21.15.45:0/3973177590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 msgr2=0x7f520015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f520015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 -- 172.21.15.45:0/3973177590 >> v1:172.21.15.45:6789/0 conn(0x7f520015b740 legacy=0x7f520006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5205600640 1 -- 172.21.15.45:0/3973177590 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52000715d0 con 0x7f52001550b0 2024-04-25T10:29:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f51ffe00640 1 -- 172.21.15.45:0/3973177590 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073771254 0 0) 0x7f5200071050 con 0x7f520015b740 2024-04-25T10:29:32.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.328+0000 7f5204c00640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f520015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5205600640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f52000cf930 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009870 tx=0x7f51f401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d55dd2a6b28155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f51ffe00640 1 -- 172.21.15.45:0/3973177590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b0070 con 0x7f52001550b0 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f51ffe00640 1 -- 172.21.15.45:0/3973177590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40ab600 con 0x7f52001550b0 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f51ffe00640 1 -- 172.21.15.45:0/3973177590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40ab9d0 con 0x7f52001550b0 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 -- 172.21.15.45:0/3973177590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 msgr2=0x7f52000cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f52000cf930 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009870 tx=0x7f51f401c560 comp rx=0 tx=0).stop 2024-04-25T10:29:32.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 -- 172.21.15.45:0/3973177590 shutdown_connections 2024-04-25T10:29:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f520015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 --2- 172.21.15.45:0/3973177590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f52000cf930 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 -- 172.21.15.45:0/3973177590 >> 172.21.15.45:0/3973177590 conn(0x7f5200066b70 msgr2=0x7f5200066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 -- 172.21.15.45:0/3973177590 shutdown_connections 2024-04-25T10:29:32.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.332+0000 7f5208000640 1 -- 172.21.15.45:0/3973177590 wait complete. 2024-04-25T10:29:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 Processor -- start 2024-04-25T10:29:32.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- start start 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f5200144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f520006c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520015b740 0x7f520006c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520015bd10 con 0x7f52001550b0 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520015bb90 con 0x7f520015b740 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520015c030 con 0x7f52000cfe70 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f520006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f520006c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56900/0 (socket says 172.21.15.45:56900) 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 -- 172.21.15.45:0/3809504199 learned_addr learned my addr 172.21.15.45:0/3809504199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5205600640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f5200144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5206000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520015b740 0x7f520006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 -- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 msgr2=0x7f5200144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f5200144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 -- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520015b740 msgr2=0x7f520006c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520015b740 0x7f520006c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 -- 172.21.15.45:0/3809504199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52000ab940 con 0x7f52001550b0 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5205600640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f5200144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5204c00640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f520006c320 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c7c0 tx=0x7f51ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007a30 con 0x7f52001550b0 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec007bd0 con 0x7f52001550b0 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52001fa400 con 0x7f52001550b0 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0113f0 con 0x7f52001550b0 2024-04-25T10:29:32.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52001fa9c0 con 0x7f52001550b0 2024-04-25T10:29:32.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.336+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520004ef30 con 0x7f52001550b0 2024-04-25T10:29:32.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.340+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51ec011590 con 0x7f52001550b0 2024-04-25T10:29:32.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.340+0000 7f51fd600640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f51e0066f60 0x7f51e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:32.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.340+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51ec0851f0 con 0x7f52001550b0 2024-04-25T10:29:32.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.340+0000 7f5205600640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f51e0066f60 0x7f51e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:32.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.340+0000 7f5205600640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f51e0066f60 0x7f51e0069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f51f40ad000 tx=0x7f51f40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:32.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.344+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51ec015020 con 0x7f52001550b0 2024-04-25T10:29:32.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f520015b290 con 0x7f52001550b0 2024-04-25T10:29:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f51fd600640 1 -- 172.21.15.45:0/3809504199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f51ec0561f0 con 0x7f52001550b0 2024-04-25T10:29:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:32.540 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:26.603160+0000","modified":"2024-04-25T10:29:29.629298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9996":{"gid":9996,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2482982951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2482982951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2482982951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:29:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f51e0066f60 msgr2=0x7f51e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f51e0066f60 0x7f51e0069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f51f40ad000 tx=0x7f51f40a6000 comp rx=0 tx=0).stop 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 msgr2=0x7f520006c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f520006c320 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c7c0 tx=0x7f51ec00cc80 comp rx=0 tx=0).stop 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 shutdown_connections 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f51e0066f60 0x7f51e0069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f520015b740 0x7f520006c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52001550b0 0x7f520006c320 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 --2- 172.21.15.45:0/3809504199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52000cfe70 0x7f5200144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 >> 172.21.15.45:0/3809504199 conn(0x7f5200066b70 msgr2=0x7f52000b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 shutdown_connections 2024-04-25T10:29:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:32.540+0000 7f5208000640 1 -- 172.21.15.45:0/3809504199 wait complete. 2024-04-25T10:29:32.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 115, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:29:26.603160+0000', 'modified': '2024-04-25T10:29:29.629298+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9996': {'gid': 9996, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/2482982951', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2482982951}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2482982951}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:29:32.965 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:29:32.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:29:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f286b800640 1 Processor -- start 2024-04-25T10:29:33.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f286b800640 1 -- start start 2024-04-25T10:29:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f286b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286406bd20 0x7f28640cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f286b800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28640cc060 con 0x7f286415ddc0 2024-04-25T10:29:33.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f286b800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28640cc340 con 0x7f286415e1a0 2024-04-25T10:29:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f286b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28640cf2f0 con 0x7f286406bd20 2024-04-25T10:29:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f2868e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f286415ddc0 0x7f286406b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:32768/0 (socket says 172.21.15.45:32768) 2024-04-25T10:29:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f2868e00640 1 -- 172.21.15.45:0/4179655130 learned_addr learned my addr 172.21.15.45:0/4179655130 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:33.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f285be00640 1 --2- 172.21.15.45:0/4179655130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286406bd20 0x7f28640cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065076182 0 0) 0x7f28640cc060 con 0x7f286415ddc0 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2848003530 con 0x7f286415ddc0 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454849553 0 0) 0x7f28640cc340 con 0x7f286415e1a0 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28640cc060 con 0x7f286415e1a0 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f285be00640 1 -- 172.21.15.45:0/4179655130 >> v1:172.21.15.117:6789/0 conn(0x7f286415e1a0 legacy=0x7f28640d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f285be00640 1 -- 172.21.15.45:0/4179655130 >> v1:172.21.15.45:6789/0 conn(0x7f286415ddc0 legacy=0x7f286406b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.112+0000 7f285be00640 1 -- 172.21.15.45:0/4179655130 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28640cc620 con 0x7f286406bd20 2024-04-25T10:29:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f285be00640 1 --2- 172.21.15.45:0/4179655130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286406bd20 0x7f28640cfce0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f284c0097f0 tx=0x7f284c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3acac9bec3dbcde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c0b0070 con 0x7f286406bd20 2024-04-25T10:29:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c0ab600 con 0x7f286406bd20 2024-04-25T10:29:33.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f2863e00640 1 -- 172.21.15.45:0/4179655130 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c0ab9b0 con 0x7f286406bd20 2024-04-25T10:29:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 -- 172.21.15.45:0/4179655130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286406bd20 msgr2=0x7f28640cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 --2- 172.21.15.45:0/4179655130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286406bd20 0x7f28640cfce0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f284c0097f0 tx=0x7f284c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 -- 172.21.15.45:0/4179655130 shutdown_connections 2024-04-25T10:29:33.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 --2- 172.21.15.45:0/4179655130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286406bd20 0x7f28640cfce0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 -- 172.21.15.45:0/4179655130 >> 172.21.15.45:0/4179655130 conn(0x7f28640d1970 msgr2=0x7f28640d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 -- 172.21.15.45:0/4179655130 shutdown_connections 2024-04-25T10:29:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.116+0000 7f286b800640 1 -- 172.21.15.45:0/4179655130 wait complete. 2024-04-25T10:29:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 Processor -- start 2024-04-25T10:29:33.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 -- start start 2024-04-25T10:29:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f286406bd20 0x7f28640b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 0x7f286414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286415e1a0 0x7f286415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286416bd30 con 0x7f286415ddc0 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640cde30 con 0x7f286406bd20 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286416beb0 con 0x7f286415e1a0 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 0x7f286414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 0x7f286414cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56904/0 (socket says 172.21.15.45:56904) 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 -- 172.21.15.45:0/4155859161 learned_addr learned my addr 172.21.15.45:0/4155859161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f2868e00640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f286406bd20 0x7f28640b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f2869800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286415e1a0 0x7f286415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 -- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286415e1a0 msgr2=0x7f286415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286415e1a0 0x7f286415ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 -- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f286406bd20 msgr2=0x7f28640b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f286406bd20 0x7f28640b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 -- 172.21.15.45:0/4155859161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28640b7330 con 0x7f286415ddc0 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f2869800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286415e1a0 0x7f286415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f285be00640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 0x7f286414cab0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f284c0190f0 tx=0x7f284c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f2861600640 1 -- 172.21.15.45:0/4155859161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c0b0040 con 0x7f286415ddc0 2024-04-25T10:29:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f2861600640 1 -- 172.21.15.45:0/4155859161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c0ab400 con 0x7f286415ddc0 2024-04-25T10:29:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286415f890 con 0x7f286415ddc0 2024-04-25T10:29:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f2861600640 1 -- 172.21.15.45:0/4155859161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c006e20 con 0x7f286415ddc0 2024-04-25T10:29:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.120+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286415d520 con 0x7f286415ddc0 2024-04-25T10:29:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f2861600640 1 -- 172.21.15.45:0/4155859161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f284c0bb030 con 0x7f286415ddc0 2024-04-25T10:29:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f2861600640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2840066f60 0x7f2840069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f2861600640 1 -- 172.21.15.45:0/4155859161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f284c11e1d0 con 0x7f286415ddc0 2024-04-25T10:29:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f2868e00640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2840066f60 0x7f2840069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f286ed0e000 1 Processor -- start 2024-04-25T10:29:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f286ed0e000 1 -- start start 2024-04-25T10:29:33.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.124+0000 7f2868e00640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2840066f60 0x7f2840069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2854007fd0 tx=0x7f2854007d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06228d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061de9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d061df310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d060c65f0 con 0x559d061def30 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0602dda0 con 0x559d06228920 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d05fb6150 con 0x559d06229240 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d061df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d061df310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56914/0 (socket says 172.21.15.45:56914) 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 -- 172.21.15.45:0/2513477461 learned_addr learned my addr 172.21.15.45:0/2513477461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f285be00640 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061de9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2868e00640 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06228d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 -- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 msgr2=0x559d061de9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061de9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 -- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 msgr2=0x559d06228d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06228d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 -- 172.21.15.45:0/2513477461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d05fcfca0 con 0x559d061def30 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2869800640 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d061df310 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f285c001e40 tx=0x7f285c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2613f9953a7e0e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2859600640 1 -- 172.21.15.45:0/2513477461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c018070 con 0x559d061def30 2024-04-25T10:29:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2859600640 1 -- 172.21.15.45:0/2513477461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00de30 con 0x559d061def30 2024-04-25T10:29:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f2859600640 1 -- 172.21.15.45:0/2513477461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c003ba0 con 0x559d061def30 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- 172.21.15.45:0/2513477461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 msgr2=0x559d061df310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d061df310 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f285c001e40 tx=0x7f285c00d520 comp rx=0 tx=0).stop 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- 172.21.15.45:0/2513477461 shutdown_connections 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d061df310 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061de9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2513477461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06228d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- 172.21.15.45:0/2513477461 >> 172.21.15.45:0/2513477461 conn(0x559d061e52a0 msgr2=0x559d061dd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- 172.21.15.45:0/2513477461 shutdown_connections 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- 172.21.15.45:0/2513477461 wait complete. 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 -- start start 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 Processor -- start 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0622fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.128+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06230310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d05fe8990 con 0x559d061def30 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d06029e40 con 0x559d06228920 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d05fb9900 con 0x559d06229240 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 10 client.0 _ll_register_callbacks cb 0x559d060e11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0622fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0622fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56918/0 (socket says 172.21.15.45:56918) 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 -- 172.21.15.45:0/2609680390 learned_addr learned my addr 172.21.15.45:0/2609680390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:33.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f285be00640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06230310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2869800640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 -- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 msgr2=0x559d061e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 -- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 msgr2=0x559d06230310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06230310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 -- 172.21.15.45:0/2609680390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d05fe6630 con 0x559d061def30 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2868e00640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0622fdd0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f28540146d0 tx=0x7f2854001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b2df6a960bd623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 1 -- 172.21.15.45:0/2609680390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854025020 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 1 -- 172.21.15.45:0/2609680390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2854013680 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f285c00de30 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 1 -- 172.21.15.45:0/2609680390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f285402f050 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 10 client.10134 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x559d05fb63a0 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 1 -- 172.21.15.45:0/2609680390 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c00de30 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 1 -- 172.21.15.45:0/2609680390 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2854032020 con 0x559d061def30 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 10 client.10134 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 10 client.10134 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 20 client.10134 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d061e4160 0x559d06239e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 4 client.10134 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:33.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d05fc9a50 con 0x559d061e4160 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f285be00640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d061e4160 0x559d06239e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 10 client.10134 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f285be00640 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d061e4160 0x559d06239e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64279c8d38c294f server_cookie=bfdaba5d629a8983 in_seq=0 out_seq=0 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 1 -- 172.21.15.45:0/2609680390 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d05fc9a50 con 0x559d061e4160 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 10 client.10134 handle_command_reply: tid=0 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 client.10134 shutdown 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 msgr2=0x559d0622fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0622fdd0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f28540146d0 tx=0x7f2854001160 comp rx=0 tx=0).stop 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 shutdown_connections 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d061e4160 0x559d06239e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06229240 0x559d061e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d06230310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f2852000640 0 client.10134 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 --2- 172.21.15.45:0/2609680390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0622fdd0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.132+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 >> 172.21.15.45:0/2609680390 conn(0x559d061e52a0 msgr2=0x559d061dd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.136+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 shutdown_connections 2024-04-25T10:29:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.136+0000 7f286ed0e000 1 -- 172.21.15.45:0/2609680390 wait complete. 2024-04-25T10:29:33.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.136+0000 7f286ed0e000 20 client.10134 trim_cache size 0 max 16384 2024-04-25T10:29:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 Processor -- start 2024-04-25T10:29:33.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- start start 2024-04-25T10:29:33.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d061def30 0x559d061e2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d061e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 0x559d06232a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28641fae60 con 0x559d06229240 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640b52c0 con 0x559d06228920 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28641fa860 con 0x559d061def30 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 0x559d06232a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 0x559d06232a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56926/0 (socket says 172.21.15.45:56926) 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 -- 172.21.15.45:0/3008193643 learned_addr learned my addr 172.21.15.45:0/3008193643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2868e00640 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d061def30 0x559d061e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f285be00640 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d061e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 -- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d061def30 msgr2=0x559d061e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d061def30 0x559d061e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 -- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 msgr2=0x559d061e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d061e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 -- 172.21.15.45:0/3008193643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d06229070 con 0x559d06229240 2024-04-25T10:29:33.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2869800640 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 0x559d06232a00 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d560 tx=0x7f285c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8b9f6e30f213dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2858c00640 1 -- 172.21.15.45:0/3008193643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c023070 con 0x559d06229240 2024-04-25T10:29:33.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2858c00640 1 -- 172.21.15.45:0/3008193643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c019040 con 0x559d06229240 2024-04-25T10:29:33.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f2858c00640 1 -- 172.21.15.45:0/3008193643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c018040 con 0x559d06229240 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- 172.21.15.45:0/3008193643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 msgr2=0x559d06232a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 0x559d06232a00 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d560 tx=0x7f285c00b000 comp rx=0 tx=0).stop 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- 172.21.15.45:0/3008193643 shutdown_connections 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d06229240 0x559d06232a00 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06228920 0x559d061e2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 --2- 172.21.15.45:0/3008193643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d061def30 0x559d061e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- 172.21.15.45:0/3008193643 >> 172.21.15.45:0/3008193643 conn(0x559d061e52a0 msgr2=0x559d06215170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- 172.21.15.45:0/3008193643 shutdown_connections 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- 172.21.15.45:0/3008193643 wait complete. 2024-04-25T10:29:33.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 -- start start 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.152+0000 7f286ed0e000 1 Processor -- start 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0626cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06228920 0x559d0626d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06229240 0x559d0626d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d060323d0 con 0x559d061def30 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d060e9d50 con 0x559d06229240 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559d05fc8980 con 0x559d06228920 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 10 client.0 _ll_register_callbacks cb 0x7f285c0016e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0626cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0626cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56940/0 (socket says 172.21.15.45:56940) 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 -- 172.21.15.45:0/714800923 learned_addr learned my addr 172.21.15.45:0/714800923 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f285be00640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06228920 0x559d0626d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2869800640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06229240 0x559d0626d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 -- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06228920 msgr2=0x559d0626d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06228920 0x559d0626d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 -- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06229240 msgr2=0x559d0626d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06229240 0x559d0626d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 -- 172.21.15.45:0/714800923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d060af600 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2868e00640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0626cac0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f28540069e0 tx=0x7f2854021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e6384ded5c89b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 1 -- 172.21.15.45:0/714800923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854025040 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 1 -- 172.21.15.45:0/714800923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2854010040 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f285c019040 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 1 -- 172.21.15.45:0/714800923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2854045050 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 10 client.10140 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x559d05fc29b0 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 1 -- 172.21.15.45:0/714800923 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c019040 con 0x559d061def30 2024-04-25T10:29:33.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 1 -- 172.21.15.45:0/714800923 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2854048020 con 0x559d061def30 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 10 client.10140 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 10 client.10140 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 20 client.10140 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d0626de00 0x559d06235780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 4 client.10140 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559d0603b2e0 con 0x559d0626de00 2024-04-25T10:29:33.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f285be00640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d0626de00 0x559d06235780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f285be00640 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d0626de00 0x559d06235780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1451c5473796f3aa server_cookie=f24b583217ecdf1f in_seq=0 out_seq=0 2024-04-25T10:29:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 10 client.10140 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 1 -- 172.21.15.45:0/714800923 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x559d0603b2e0 con 0x559d0626de00 2024-04-25T10:29:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 10 client.10140 handle_command_reply: tid=0 2024-04-25T10:29:33.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 client.10140 shutdown 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 msgr2=0x559d0626cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0626cac0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f28540069e0 tx=0x7f2854021000 comp rx=0 tx=0).stop 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 shutdown_connections 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x559d0626de00 0x559d06235780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d06229240 0x559d0626d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559d06228920 0x559d0626d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 --2- 172.21.15.45:0/714800923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d061def30 0x559d0626cac0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f2852000640 0 client.10140 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 >> 172.21.15.45:0/714800923 conn(0x559d061e52a0 msgr2=0x559d061e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.156+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 shutdown_connections 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286ed0e000 1 -- 172.21.15.45:0/714800923 wait complete. 2024-04-25T10:29:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286ed0e000 20 client.10140 trim_cache size 0 max 16384 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 9996, 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9352759070000003, 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 115, 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 155, 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 155, 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.811972984000001 2024-04-25T10:29:33.164 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:29:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2840066f60 msgr2=0x7f2840069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2840066f60 0x7f2840069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2854007fd0 tx=0x7f2854007d80 comp rx=0 tx=0).stop 2024-04-25T10:29:33.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 msgr2=0x7f286414cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 0x7f286414cab0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f284c0190f0 tx=0x7f284c019280 comp rx=0 tx=0).stop 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 shutdown_connections 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2840066f60 0x7f2840069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f286415e1a0 0x7f286415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f286415ddc0 0x7f286414cab0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 --2- 172.21.15.45:0/4155859161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f286406bd20 0x7f28640b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 >> 172.21.15.45:0/4155859161 conn(0x7f28640d1970 msgr2=0x7f28640cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 shutdown_connections 2024-04-25T10:29:33.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.160+0000 7f286b800640 1 -- 172.21.15.45:0/4155859161 wait complete. 2024-04-25T10:29:33.179 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:29:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 Processor -- start 2024-04-25T10:29:33.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 -- start start 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f015c15bc40 0x7f015c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f015c0cda10 0x7f015c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0b2770 con 0x7f015c0cf930 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c06c670 con 0x7f015c0cda10 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0161800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c06c950 con 0x7f015c15bc40 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f015c0cda10 0x7f015c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f015c0cf930 0x7f015c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:32776/0 (socket says 172.21.15.45:32776) 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015a000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f015c0cda10 0x7f015c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56954/0 (socket says 172.21.15.45:56954) 2024-04-25T10:29:33.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015b400640 1 -- 172.21.15.45:0/3574497843 learned_addr learned my addr 172.21.15.45:0/3574497843 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015a000640 1 -- 172.21.15.45:0/3574497843 >> v2:172.21.15.117:3300/0 conn(0x7f015c15bc40 msgr2=0x7f015c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015aa00640 1 --2- 172.21.15.45:0/3574497843 >> v2:172.21.15.117:3300/0 conn(0x7f015c15bc40 0x7f015c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015a000640 1 --2- 172.21.15.45:0/3574497843 >> v2:172.21.15.117:3300/0 conn(0x7f015c15bc40 0x7f015c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015a000640 1 -- 172.21.15.45:0/3574497843 >> v1:172.21.15.45:6789/0 conn(0x7f015c0cf930 legacy=0x7f015c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015a000640 1 -- 172.21.15.45:0/3574497843 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c06cc30 con 0x7f015c0cda10 2024-04-25T10:29:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f0159600640 1 -- 172.21.15.45:0/3574497843 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796848174 0 0) 0x7f015c0b2770 con 0x7f015c0cf930 2024-04-25T10:29:33.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.320+0000 7f015aa00640 1 --2- 172.21.15.45:0/3574497843 >> v2:172.21.15.117:3300/0 conn(0x7f015c15bc40 0x7f015c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f015a000640 1 --2- 172.21.15.45:0/3574497843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0b4610 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f015c0d0800 tx=0x7f014801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f220b508bd804a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0159600640 1 -- 172.21.15.45:0/3574497843 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480b1070 con 0x7f015c0cda10 2024-04-25T10:29:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0159600640 1 -- 172.21.15.45:0/3574497843 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01480ac5f0 con 0x7f015c0cda10 2024-04-25T10:29:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0159600640 1 -- 172.21.15.45:0/3574497843 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480ac9c0 con 0x7f015c0cda10 2024-04-25T10:29:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 -- 172.21.15.45:0/3574497843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 msgr2=0x7f015c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 --2- 172.21.15.45:0/3574497843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0b4610 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f015c0d0800 tx=0x7f014801c480 comp rx=0 tx=0).stop 2024-04-25T10:29:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 -- 172.21.15.45:0/3574497843 shutdown_connections 2024-04-25T10:29:33.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 --2- 172.21.15.45:0/3574497843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0b4610 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 --2- 172.21.15.45:0/3574497843 >> v2:172.21.15.117:3300/0 conn(0x7f015c15bc40 0x7f015c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 -- 172.21.15.45:0/3574497843 >> 172.21.15.45:0/3574497843 conn(0x7f015c066b70 msgr2=0x7f015c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 -- 172.21.15.45:0/3574497843 shutdown_connections 2024-04-25T10:29:33.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 -- 172.21.15.45:0/3574497843 wait complete. 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 Processor -- start 2024-04-25T10:29:33.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.324+0000 7f0161800640 1 -- start start 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c0cf930 0x7f015c146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c15bc40 0x7f015c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c168b50 con 0x7f015c0cda10 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1689d0 con 0x7f015c15bc40 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c168cd0 con 0x7f015c0cf930 2024-04-25T10:29:33.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56960/0 (socket says 172.21.15.45:56960) 2024-04-25T10:29:33.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 -- 172.21.15.45:0/2060208238 learned_addr learned my addr 172.21.15.45:0/2060208238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015a000640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c0cf930 0x7f015c146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 -- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c0cf930 msgr2=0x7f015c146af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c0cf930 0x7f015c146af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 -- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c15bc40 msgr2=0x7f015c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015b400640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c15bc40 0x7f015c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c15bc40 0x7f015c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 -- 172.21.15.45:0/2060208238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c147030 con 0x7f015c0cda10 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015a000640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c0cf930 0x7f015c146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:33.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015b400640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c15bc40 0x7f015c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f015aa00640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0cbba0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f014c00c7c0 tx=0x7f014c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c007a30 con 0x7f015c0cda10 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c007bd0 con 0x7f015c0cda10 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c0113f0 con 0x7f015c0cda10 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c1f6f10 con 0x7f015c0cda10 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c1f74d0 con 0x7f015c0cda10 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.328+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c04ef30 con 0x7f015c0cda10 2024-04-25T10:29:33.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.332+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f014c011590 con 0x7f015c0cda10 2024-04-25T10:29:33.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.332+0000 7f0152a00640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f013c066f60 0x7f013c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:33.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.332+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f014c085240 con 0x7f015c0cda10 2024-04-25T10:29:33.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.332+0000 7f015a000640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f013c066f60 0x7f013c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:33.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.332+0000 7f015a000640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f013c066f60 0x7f013c069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f01480ae000 tx=0x7f01480a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:33.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.336+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f015c147030 con 0x7f015c0cda10 2024-04-25T10:29:33.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.532+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f015c0af1c0 con 0x7f015c0cda10 2024-04-25T10:29:33.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.532+0000 7f0152a00640 1 -- 172.21.15.45:0/2060208238 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v115) v1 ==== 53+0+73 (secure 0 0 0) 0x7f014c015020 con 0x7f015c0cda10 2024-04-25T10:29:33.534 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:29:33.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f013c066f60 msgr2=0x7f013c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f013c066f60 0x7f013c069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f01480ae000 tx=0x7f01480a7000 comp rx=0 tx=0).stop 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 msgr2=0x7f015c0cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0cbba0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f014c00c7c0 tx=0x7f014c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 shutdown_connections 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f013c066f60 0x7f013c069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c15bc40 0x7f015c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c0cf930 0x7f015c146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 --2- 172.21.15.45:0/2060208238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015c0cda10 0x7f015c0cbba0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 >> 172.21.15.45:0/2060208238 conn(0x7f015c066b70 msgr2=0x7f015c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 shutdown_connections 2024-04-25T10:29:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:33.536+0000 7f0161800640 1 -- 172.21.15.45:0/2060208238 wait complete. 2024-04-25T10:29:33.556 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:29:33.556 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:29:33.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:29:33.556 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:29:33.557 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:33.557 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:29:33.557 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:29:33.557 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:29:33.557 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:29:33.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.557 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:29:33.567 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:29:33.568 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41766sec preferred_lft 41766sec 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:29:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:29:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:29:33.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.571 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:29:33.629 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:29:33.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.632 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:29:33.688 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:29:33.689 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:29:33.689 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:29:33.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:29:33.689 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:29:33.689 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:29:33.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.689 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:29:33.689 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:29:33.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:29:33.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:29:33.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:29:33.690 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:29:33.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.780 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:29:33.790 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:29:33.790 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:29:33.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:29:33.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:29:33.851 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:33.851 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:33.903 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:29:33.903 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:29:33.904 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:33.958 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:29:33.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:33.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:29:34.049 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:29:34.049 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:29:34.060 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:29:34.060 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:29:34.060 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:29:34.060 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:29:34.060 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:29:34.063 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:29:34.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:29:34.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.085 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:29:34.147 INFO:teuthology.orchestra.run.smithi045.stdout:client10152 2024-04-25T10:29:34.148 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:29:34.148 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:29:34.203 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:29:34.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:29:34.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 Processor -- start 2024-04-25T10:29:34.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 -- start start 2024-04-25T10:29:34.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5cc0cd9a0 0x7fa5cc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5cc162020 0x7fa5cc15d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc06ac90 con 0x7fa5cc06bd10 2024-04-25T10:29:34.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc06b880 con 0x7fa5cc162020 2024-04-25T10:29:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.388+0000 7fa5d2000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc06af50 con 0x7fa5cc0cd9a0 2024-04-25T10:29:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5cc162020 0x7fa5cc15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5cc162020 0x7fa5cc15d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56972/0 (socket says 172.21.15.45:56972) 2024-04-25T10:29:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1346895244 learned_addr learned my addr 172.21.15.45:0/1346895244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cb400640 1 --2- 172.21.15.45:0/1346895244 >> v2:172.21.15.117:3300/0 conn(0x7fa5cc0cd9a0 0x7fa5cc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1346895244 >> v2:172.21.15.117:3300/0 conn(0x7fa5cc0cd9a0 msgr2=0x7fa5cc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/1346895244 >> v2:172.21.15.117:3300/0 conn(0x7fa5cc0cd9a0 0x7fa5cc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1346895244 >> v1:172.21.15.117:6789/0 conn(0x7fa5cc06bd10 legacy=0x7fa5cc161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1346895244 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc06b230 con 0x7fa5cc162020 2024-04-25T10:29:34.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/1346895244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc15d610 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0cf4a0 tx=0x7fa5c401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50e6b32b708dca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5caa00640 1 -- 172.21.15.45:0/1346895244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40b4070 con 0x7fa5cc162020 2024-04-25T10:29:34.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5caa00640 1 -- 172.21.15.45:0/1346895244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40af400 con 0x7fa5cc162020 2024-04-25T10:29:34.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.392+0000 7fa5caa00640 1 -- 172.21.15.45:0/1346895244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40af7d0 con 0x7fa5cc162020 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- 172.21.15.45:0/1346895244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 msgr2=0x7fa5cc15d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 --2- 172.21.15.45:0/1346895244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc15d610 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0cf4a0 tx=0x7fa5c401f1e0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- 172.21.15.45:0/1346895244 shutdown_connections 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 --2- 172.21.15.45:0/1346895244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc15d610 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 --2- 172.21.15.45:0/1346895244 >> v2:172.21.15.117:3300/0 conn(0x7fa5cc0cd9a0 0x7fa5cc0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- 172.21.15.45:0/1346895244 >> 172.21.15.45:0/1346895244 conn(0x7fa5cc0d19a0 msgr2=0x7fa5cc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- 172.21.15.45:0/1346895244 shutdown_connections 2024-04-25T10:29:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- 172.21.15.45:0/1346895244 wait complete. 2024-04-25T10:29:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 Processor -- start 2024-04-25T10:29:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- start start 2024-04-25T10:29:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5cc06bd10 0x7fa5cc14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc0cd9a0 0x7fa5cc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc1e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc162660 con 0x7fa5cc162020 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0cdd80 con 0x7fa5cc0cd9a0 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5d2000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0d0ba0 con 0x7fa5cc06bd10 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc1e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5c2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc0cd9a0 0x7fa5cc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc1e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56984/0 (socket says 172.21.15.45:56984) 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2158500236 learned_addr learned my addr 172.21.15.45:0/2158500236 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cb400640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5cc06bd10 0x7fa5cc14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5cc06bd10 msgr2=0x7fa5cc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5cc06bd10 0x7fa5cc14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc0cd9a0 msgr2=0x7fa5cc0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc0cd9a0 0x7fa5cc0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2158500236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc160cc0 con 0x7fa5cc162020 2024-04-25T10:29:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.396+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc1e7be0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40b2040 tx=0x7fa5c401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5c3e00640 1 -- 172.21.15.45:0/2158500236 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40b4040 con 0x7fa5cc162020 2024-04-25T10:29:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5c3e00640 1 -- 172.21.15.45:0/2158500236 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40c0070 con 0x7fa5cc162020 2024-04-25T10:29:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc150350 con 0x7fa5cc162020 2024-04-25T10:29:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5c3e00640 1 -- 172.21.15.45:0/2158500236 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4009510 con 0x7fa5cc162020 2024-04-25T10:29:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc0c7500 con 0x7fa5cc162020 2024-04-25T10:29:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5c3e00640 1 -- 172.21.15.45:0/2158500236 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5c40c5460 con 0x7fa5cc162020 2024-04-25T10:29:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5c3e00640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5a8066f90 0x7fa5a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5c3e00640 1 -- 172.21.15.45:0/2158500236 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5c4122ff0 con 0x7fa5cc162020 2024-04-25T10:29:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5cb400640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5a8066f90 0x7fa5a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5d54f0000 1 Processor -- start 2024-04-25T10:29:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.400+0000 7fa5d54f0000 1 -- start start 2024-04-25T10:29:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cb400640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5a8066f90 0x7fa5a8069450 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0d21b0 tx=0x7fa5bc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b845560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b845aa0 0x563a3b7fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 0x563a3b7fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3b648f20 con 0x563a3b7fb600 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3b5d1000 con 0x563a3b845180 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a3b644fc0 con 0x563a3b845aa0 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 0x563a3b7fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b845560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 0x563a3b7fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56994/0 (socket says 172.21.15.45:56994) 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2567945459 learned_addr learned my addr 172.21.15.45:0/2567945459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b845aa0 0x563a3b7fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b845aa0 msgr2=0x563a3b7fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b845aa0 0x563a3b7fb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 msgr2=0x563a3b845560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b845560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 -- 172.21.15.45:0/2567945459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a3b624ba0 con 0x563a3b7fb600 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 0x563a3b7fb9e0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40f3970 tx=0x7fa5c40f38e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7d4a82655b4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5bbe00640 1 -- 172.21.15.45:0/2567945459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40f7a80 con 0x563a3b7fb600 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5bbe00640 1 -- 172.21.15.45:0/2567945459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40fc040 con 0x563a3b7fb600 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5bbe00640 1 -- 172.21.15.45:0/2567945459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40f0050 con 0x563a3b7fb600 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2567945459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 msgr2=0x563a3b7fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 0x563a3b7fb9e0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40f3970 tx=0x7fa5c40f38e0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2567945459 shutdown_connections 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b7fb600 0x563a3b7fb9e0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b845aa0 0x563a3b7fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2567945459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b845560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2567945459 >> 172.21.15.45:0/2567945459 conn(0x563a3b801760 msgr2=0x563a3b7f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2567945459 shutdown_connections 2024-04-25T10:29:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2567945459 wait complete. 2024-04-25T10:29:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 -- start start 2024-04-25T10:29:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.404+0000 7fa5d54f0000 1 Processor -- start 2024-04-25T10:29:34.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b800c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845aa0 0x563a3b833450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3b7069c0 con 0x563a3b845180 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3b6035d0 con 0x563a3b845aa0 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a3b648f20 con 0x563a3b7fb600 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 10 client.0 _ll_register_callbacks cb 0x563a3b7e6a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b800c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b800c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57000/0 (socket says 172.21.15.45:57000) 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 -- 172.21.15.45:0/4041832482 learned_addr learned my addr 172.21.15.45:0/4041832482 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5cb400640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845aa0 0x563a3b833450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 -- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 msgr2=0x563a3b8006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 -- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845aa0 msgr2=0x563a3b833450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845aa0 0x563a3b833450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 -- 172.21.15.45:0/4041832482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a3b68f5a0 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b800c20 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4001e60 tx=0x7fa5b4012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40bd1ec97da8054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 1 -- 172.21.15.45:0/4041832482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4005320 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 1 -- 172.21.15.45:0/4041832482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5b4007030 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5c40fc040 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 1 -- 172.21.15.45:0/4041832482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b4007230 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 10 client.10164 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x563a3b6202b0 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 1 -- 172.21.15.45:0/4041832482 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40fc040 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 1 -- 172.21.15.45:0/4041832482 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa5b40073d0 con 0x563a3b845180 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 10 client.10164 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 10 client.10164 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 20 client.10164 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b833e60 0x563a3b857440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 4 client.10164 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a3b5e4c70 con 0x563a3b833e60 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5cb400640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b833e60 0x563a3b857440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5cb400640 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b833e60 0x563a3b857440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99df69864d96bdc2 server_cookie=b0e9f348db2170c3 in_seq=0 out_seq=0 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 10 client.10164 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 1 -- 172.21.15.45:0/4041832482 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a3b5e4c70 con 0x563a3b833e60 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 10 client.10164 handle_command_reply: tid=0 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 client.10164 shutdown 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 msgr2=0x563a3b800c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b800c20 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4001e60 tx=0x7fa5b4012750 comp rx=0 tx=0).stop 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 shutdown_connections 2024-04-25T10:29:34.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b833e60 0x563a3b857440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845aa0 0x563a3b833450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b800c20 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5b8c00640 0 client.10164 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/4041832482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.408+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 >> 172.21.15.45:0/4041832482 conn(0x563a3b801760 msgr2=0x563a3b7fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.412+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 shutdown_connections 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.412+0000 7fa5d54f0000 1 -- 172.21.15.45:0/4041832482 wait complete. 2024-04-25T10:29:34.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.412+0000 7fa5d54f0000 20 client.10164 trim_cache size 0 max 16384 2024-04-25T10:29:34.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 Processor -- start 2024-04-25T10:29:34.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- start start 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b801760 0x563a3b859680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b7fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b7fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1ffeb0 con 0x563a3b845180 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0c9b20 con 0x563a3b801760 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0c5f90 con 0x563a3b7fb600 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b7fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b7fa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57010/0 (socket says 172.21.15.45:57010) 2024-04-25T10:29:34.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1043421520 learned_addr learned my addr 172.21.15.45:0/1043421520 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cb400640 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b801760 0x563a3b859680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b7fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 msgr2=0x563a3b7fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b7fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b801760 msgr2=0x563a3b859680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b801760 0x563a3b859680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 -- 172.21.15.45:0/1043421520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a3b5d1250 con 0x563a3b845180 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b7fa660 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa5c409bb20 tx=0x7fa5c40c7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab6e60e109a76c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5bb400640 1 -- 172.21.15.45:0/1043421520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40f7a80 con 0x563a3b845180 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5bb400640 1 -- 172.21.15.45:0/1043421520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40f1ce0 con 0x563a3b845180 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5bb400640 1 -- 172.21.15.45:0/1043421520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40f2e30 con 0x563a3b845180 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- 172.21.15.45:0/1043421520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 msgr2=0x563a3b7fa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b7fa660 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa5c409bb20 tx=0x7fa5c40c7000 comp rx=0 tx=0).stop 2024-04-25T10:29:34.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- 172.21.15.45:0/1043421520 shutdown_connections 2024-04-25T10:29:34.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b845180 0x563a3b7fa660 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b7fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/1043421520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b801760 0x563a3b859680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- 172.21.15.45:0/1043421520 >> 172.21.15.45:0/1043421520 conn(0x563a3b8335b0 msgr2=0x563a3b844960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- 172.21.15.45:0/1043421520 shutdown_connections 2024-04-25T10:29:34.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- 172.21.15.45:0/1043421520 wait complete. 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 -- start start 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.424+0000 7fa5d54f0000 1 Processor -- start 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 0x563a3b8a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b857cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0c9520 con 0x563a3b801760 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a3b852fb0 con 0x563a3b845180 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a3b601e80 con 0x563a3b7fb600 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 10 client.0 _ll_register_callbacks cb 0x563a3b7c7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 0x563a3b8a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 0x563a3b8a5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57020/0 (socket says 172.21.15.45:57020) 2024-04-25T10:29:34.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 -- 172.21.15.45:0/2875768585 learned_addr learned my addr 172.21.15.45:0/2875768585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5cb400640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5cbe00640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b857cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 -- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 msgr2=0x563a3b8a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 -- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 msgr2=0x563a3b857cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b857cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 -- 172.21.15.45:0/2875768585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a3b5ddb90 con 0x563a3b801760 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5c2a00640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 0x563a3b8a5c20 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40011b0 tx=0x7fa5b4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931fc0a7c6c60b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 1 -- 172.21.15.45:0/2875768585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4006b00 con 0x563a3b801760 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 1 -- 172.21.15.45:0/2875768585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5b4006ca0 con 0x563a3b801760 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5c40f1ce0 con 0x563a3b801760 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 1 -- 172.21.15.45:0/2875768585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b4038050 con 0x563a3b801760 2024-04-25T10:29:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 10 client.10170 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x563a3b5f0010 con 0x563a3b801760 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 1 -- 172.21.15.45:0/2875768585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40f1ce0 con 0x563a3b801760 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 1 -- 172.21.15.45:0/2875768585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa5b403b020 con 0x563a3b801760 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 10 client.10170 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 10 client.10170 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 20 client.10170 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b845aa0 0x563a3b858bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 4 client.10170 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a3b656460 con 0x563a3b845aa0 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5cb400640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b845aa0 0x563a3b858bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5cb400640 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b845aa0 0x563a3b858bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2957145f02992e2a server_cookie=777f4c1c2ec0588f in_seq=0 out_seq=0 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 10 client.10170 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 1 -- 172.21.15.45:0/2875768585 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a3b656460 con 0x563a3b845aa0 2024-04-25T10:29:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 10 client.10170 handle_command_reply: tid=0 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 client.10170 shutdown 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 msgr2=0x563a3b8a5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 0x563a3b8a5c20 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40011b0 tx=0x7fa5b4007f90 comp rx=0 tx=0).stop 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 shutdown_connections 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x563a3b845aa0 0x563a3b858bf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a3b845180 0x563a3b857cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a3b801760 0x563a3b8a5c20 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5b8c00640 0 client.10170 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 --2- 172.21.15.45:0/2875768585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x563a3b7fb600 0x563a3b8a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 >> 172.21.15.45:0/2875768585 conn(0x563a3b8335b0 msgr2=0x563a3b852770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 shutdown_connections 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 1 -- 172.21.15.45:0/2875768585 wait complete. 2024-04-25T10:29:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.428+0000 7fa5d54f0000 20 client.10170 trim_cache size 0 max 16384 2024-04-25T10:29:34.436 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5a8066f90 msgr2=0x7fa5a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5a8066f90 0x7fa5a8069450 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0d21b0 tx=0x7fa5bc009190 comp rx=0 tx=0).stop 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 msgr2=0x7fa5cc1e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc1e7be0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40b2040 tx=0x7fa5c401bf80 comp rx=0 tx=0).stop 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 shutdown_connections 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5a8066f90 0x7fa5a8069450 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc162020 0x7fa5cc1e7be0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc0cd9a0 0x7fa5cc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 --2- 172.21.15.45:0/2158500236 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5cc06bd10 0x7fa5cc14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 >> 172.21.15.45:0/2158500236 conn(0x7fa5cc0d19a0 msgr2=0x7fa5cc0cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 shutdown_connections 2024-04-25T10:29:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.432+0000 7fa5d2000640 1 -- 172.21.15.45:0/2158500236 wait complete. 2024-04-25T10:29:34.450 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:29:34.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.588+0000 7f338d600640 1 Processor -- start 2024-04-25T10:29:34.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.588+0000 7f338d600640 1 -- start start 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f338d600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33880cda10 0x7f338806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f338d600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33880cf930 0x7f33880cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f338d600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33880b2770 con 0x7f338815bc40 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f338d600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338806c670 con 0x7f33880cda10 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f338d600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338806c950 con 0x7f33880cf930 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f3386a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f338815bc40 0x7f33880b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:32790/0 (socket says 172.21.15.45:32790) 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f3386a00640 1 -- 172.21.15.45:0/624281335 learned_addr learned my addr 172.21.15.45:0/624281335 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f337e000640 1 --2- 172.21.15.45:0/624281335 >> v2:172.21.15.45:3300/0 conn(0x7f33880cda10 0x7f338806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f3387400640 1 --2- 172.21.15.45:0/624281335 >> v2:172.21.15.117:3300/0 conn(0x7f33880cf930 0x7f33880cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f3386000640 1 -- 172.21.15.45:0/624281335 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242566513 0 0) 0x7f33880b2770 con 0x7f338815bc40 2024-04-25T10:29:34.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f3386000640 1 -- 172.21.15.45:0/624281335 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f336c003530 con 0x7f338815bc40 2024-04-25T10:29:34.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f337e000640 1 -- 172.21.15.45:0/624281335 >> v2:172.21.15.117:3300/0 conn(0x7f33880cf930 msgr2=0x7f33880cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f337e000640 1 --2- 172.21.15.45:0/624281335 >> v2:172.21.15.117:3300/0 conn(0x7f33880cf930 0x7f33880cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f337e000640 1 -- 172.21.15.45:0/624281335 >> v1:172.21.15.45:6789/0 conn(0x7f338815bc40 legacy=0x7f33880b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.592+0000 7f337e000640 1 -- 172.21.15.45:0/624281335 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338806cc30 con 0x7f33880cda10 2024-04-25T10:29:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f337e000640 1 --2- 172.21.15.45:0/624281335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cda10 0x7f338806c090 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f33880ae4c0 tx=0x7f337001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e12510f90c805b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f3386000640 1 -- 172.21.15.45:0/624281335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700b0070 con 0x7f33880cda10 2024-04-25T10:29:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f3386000640 1 -- 172.21.15.45:0/624281335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33700ab600 con 0x7f33880cda10 2024-04-25T10:29:34.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f3386000640 1 -- 172.21.15.45:0/624281335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700ab9d0 con 0x7f33880cda10 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 -- 172.21.15.45:0/624281335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cda10 msgr2=0x7f338806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 --2- 172.21.15.45:0/624281335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cda10 0x7f338806c090 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f33880ae4c0 tx=0x7f337001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 -- 172.21.15.45:0/624281335 shutdown_connections 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 --2- 172.21.15.45:0/624281335 >> v2:172.21.15.117:3300/0 conn(0x7f33880cf930 0x7f33880cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 --2- 172.21.15.45:0/624281335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cda10 0x7f338806c090 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 -- 172.21.15.45:0/624281335 >> 172.21.15.45:0/624281335 conn(0x7f3388066b70 msgr2=0x7f3388066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 -- 172.21.15.45:0/624281335 shutdown_connections 2024-04-25T10:29:34.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 -- 172.21.15.45:0/624281335 wait complete. 2024-04-25T10:29:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 Processor -- start 2024-04-25T10:29:34.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 -- start start 2024-04-25T10:29:34.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.596+0000 7f338d600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33880cda10 0x7f338813e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 0x7f3388151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f338815bc40 0x7f33880712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33880b3850 con 0x7f33880cf930 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33880b36d0 con 0x7f338815bc40 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33880b39d0 con 0x7f33880cda10 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 0x7f3388151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 0x7f3388151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57034/0 (socket says 172.21.15.45:57034) 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 -- 172.21.15.45:0/4211992673 learned_addr learned my addr 172.21.15.45:0/4211992673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f3387400640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f338815bc40 0x7f33880712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f3386a00640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33880cda10 0x7f338813e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 -- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33880cda10 msgr2=0x7f338813e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33880cda10 0x7f338813e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 -- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f338815bc40 msgr2=0x7f33880712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f338815bc40 0x7f33880712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 -- 172.21.15.45:0/4211992673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388070520 con 0x7f33880cf930 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f3387400640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f338815bc40 0x7f33880712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f3386a00640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33880cda10 0x7f338813e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337e000640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 0x7f3388151820 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f33700ae040 tx=0x7f3370019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700b0040 con 0x7f33880cf930 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33700b9070 con 0x7f33880cf930 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3388070800 con 0x7f33880cf930 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700b4870 con 0x7f33880cf930 2024-04-25T10:29:34.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3388070dc0 con 0x7f33880cf930 2024-04-25T10:29:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338804ef30 con 0x7f33880cf930 2024-04-25T10:29:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.600+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33700ab4c0 con 0x7f33880cf930 2024-04-25T10:29:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.604+0000 7f337f400640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3364066f60 0x7f3364069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.604+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f337011ed40 con 0x7f33880cf930 2024-04-25T10:29:34.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.604+0000 7f3386a00640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3364066f60 0x7f3364069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.604+0000 7f3386a00640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3364066f60 0x7f3364069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3378005890 tx=0x7f3378009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.608+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3370016070 con 0x7f33880cf930 2024-04-25T10:29:34.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.804+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f338806d960 con 0x7f33880cf930 2024-04-25T10:29:34.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f337f400640 1 -- 172.21.15.45:0/4211992673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f33700efd40 con 0x7f33880cf930 2024-04-25T10:29:34.808 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:29:34.808 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3364066f60 msgr2=0x7f3364069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3364066f60 0x7f3364069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3378005890 tx=0x7f3378009040 comp rx=0 tx=0).stop 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 msgr2=0x7f3388151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 0x7f3388151820 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f33700ae040 tx=0x7f3370019190 comp rx=0 tx=0).stop 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 shutdown_connections 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3364066f60 0x7f3364069420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f338815bc40 0x7f33880712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33880cf930 0x7f3388151820 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 --2- 172.21.15.45:0/4211992673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33880cda10 0x7f338813e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 >> 172.21.15.45:0/4211992673 conn(0x7f3388066b70 msgr2=0x7f33880ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 shutdown_connections 2024-04-25T10:29:34.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.808+0000 7f338d600640 1 -- 172.21.15.45:0/4211992673 wait complete. 2024-04-25T10:29:34.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:34.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start '~mdsdir' recursive 2024-04-25T10:29:34.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.972+0000 7f6bc3800640 1 Processor -- start 2024-04-25T10:29:34.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.972+0000 7f6bc3800640 1 -- start start 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc3800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc3800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc071f80 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc3800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc15b120 con 0x7f6bbc0cf040 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc3800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc072260 con 0x7f6bbc071ab0 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bb3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6bbc0d00f0 0x7f6bbc0713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33414/0 (socket says 172.21.15.45:33414) 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bb3e00640 1 -- 172.21.15.45:0/3005735233 learned_addr learned my addr 172.21.15.45:0/3005735233 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:29:34.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc1800640 1 --2- 172.21.15.45:0/3005735233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bbbe00640 1 -- 172.21.15.45:0/3005735233 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275393720 0 0) 0x7f6bbc071f80 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bbbe00640 1 -- 172.21.15.45:0/3005735233 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba0003530 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bbbe00640 1 -- 172.21.15.45:0/3005735233 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364041614 0 0) 0x7f6bbc15b120 con 0x7f6bbc0cf040 2024-04-25T10:29:34.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc1800640 1 -- 172.21.15.45:0/3005735233 >> v1:172.21.15.117:6789/0 conn(0x7f6bbc0cf040 legacy=0x7f6bbc0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc1800640 1 -- 172.21.15.45:0/3005735233 >> v1:172.21.15.45:6789/0 conn(0x7f6bbc0d00f0 legacy=0x7f6bbc0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.976+0000 7f6bc1800640 1 -- 172.21.15.45:0/3005735233 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc15b400 con 0x7f6bbc071ab0 2024-04-25T10:29:34.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc1800640 1 --2- 172.21.15.45:0/3005735233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc0ae480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009dc0 tx=0x7f6bb401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd163454456b69c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bbbe00640 1 -- 172.21.15.45:0/3005735233 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40b2070 con 0x7f6bbc071ab0 2024-04-25T10:29:34.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bbbe00640 1 -- 172.21.15.45:0/3005735233 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb40a5850 con 0x7f6bbc071ab0 2024-04-25T10:29:34.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bbbe00640 1 -- 172.21.15.45:0/3005735233 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40a5c00 con 0x7f6bbc071ab0 2024-04-25T10:29:34.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 -- 172.21.15.45:0/3005735233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 msgr2=0x7f6bbc0ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 --2- 172.21.15.45:0/3005735233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc0ae480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009dc0 tx=0x7f6bb401c990 comp rx=0 tx=0).stop 2024-04-25T10:29:34.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 -- 172.21.15.45:0/3005735233 shutdown_connections 2024-04-25T10:29:34.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 --2- 172.21.15.45:0/3005735233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc0ae480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 -- 172.21.15.45:0/3005735233 >> 172.21.15.45:0/3005735233 conn(0x7f6bbc066b70 msgr2=0x7f6bbc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 -- 172.21.15.45:0/3005735233 shutdown_connections 2024-04-25T10:29:34.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 -- 172.21.15.45:0/3005735233 wait complete. 2024-04-25T10:29:34.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 Processor -- start 2024-04-25T10:29:34.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.980+0000 7f6bc3800640 1 -- start start 2024-04-25T10:29:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bbc0cf040 0x7f6bbc0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 0x7f6bbc14a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc168870 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0cf630 con 0x7f6bbc0cf040 2024-04-25T10:29:34.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1689f0 con 0x7f6bbc071ab0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 0x7f6bbc14a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 0x7f6bbc14a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40180/0 (socket says 172.21.15.45:40180) 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 -- 172.21.15.45:0/4025868310 learned_addr learned my addr 172.21.15.45:0/4025868310 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bbc0cf040 0x7f6bbc0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 -- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 msgr2=0x7f6bbc137140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc137140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 -- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bbc0cf040 msgr2=0x7f6bbc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bbc0cf040 0x7f6bbc0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 -- 172.21.15.45:0/4025868310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0ccb00 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc1800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 0x7f6bbc14a3c0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f6bb401a5c0 tx=0x7f6bb4019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bb9600640 1 -- 172.21.15.45:0/4025868310 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40b2040 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bb9600640 1 -- 172.21.15.45:0/4025868310 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb40bf070 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bb9600640 1 -- 172.21.15.45:0/4025868310 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40b8690 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc0ccde0 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.984+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc0cd3a0 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb9600640 1 -- 172.21.15.45:0/4025868310 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bb40b8830 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb9600640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b98066f60 0x7f6b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb9600640 1 -- 172.21.15.45:0/4025868310 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bb411ffb0 con 0x7f6bbc0d00f0 2024-04-25T10:29:34.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b98066f60 0x7f6b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 Processor -- start 2024-04-25T10:29:34.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 -- start start 2024-04-25T10:29:34.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b98066f60 0x7f6b98069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6bac000f70 tx=0x7f6bac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940b94650 0x558940b94a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:34.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589409e2dd0 con 0x558940bdf1e0 2024-04-25T10:29:34.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894096b010 con 0x558940bde8c0 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589409dee70 con 0x558940b94650 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40188/0 (socket says 172.21.15.45:40188) 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 -- 172.21.15.45:0/3897189481 learned_addr learned my addr 172.21.15.45:0/3897189481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bc1800640 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940b94650 0x558940b94a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 -- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940b94650 msgr2=0x558940b94a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940b94650 0x558940b94a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 -- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 msgr2=0x558940bdeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 -- 172.21.15.45:0/3897189481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894099b9c0 con 0x558940bdf1e0 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b94110 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc070530 tx=0x7f6ba4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef22673ebaca9afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb1600640 1 -- 172.21.15.45:0/3897189481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4005ad0 con 0x558940bdf1e0 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb1600640 1 -- 172.21.15.45:0/3897189481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4018070 con 0x558940bdf1e0 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.988+0000 7f6bb1600640 1 -- 172.21.15.45:0/3897189481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba400f870 con 0x558940bdf1e0 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3897189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 msgr2=0x558940b94110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b94110 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc070530 tx=0x7f6ba4000bd0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3897189481 shutdown_connections 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940b94650 0x558940b94a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b94110 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3897189481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3897189481 >> 172.21.15.45:0/3897189481 conn(0x558940a7c870 msgr2=0x558940b929f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3897189481 shutdown_connections 2024-04-25T10:29:34.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3897189481 wait complete. 2024-04-25T10:29:35.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- start start 2024-04-25T10:29:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 Processor -- start 2024-04-25T10:29:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940b94650 0x558940b98f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bde8c0 0x558940b994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894099d420 con 0x558940bdf1e0 2024-04-25T10:29:35.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558940a65470 con 0x558940b94650 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558940b9dc20 con 0x558940bde8c0 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 10 client.0 _ll_register_callbacks cb 0x558940b7ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940b94650 0x558940b98f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bb3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bde8c0 0x558940b994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940b94650 0x558940b98f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46484/0 (socket says 172.21.15.45:46484) 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc0e00640 1 -- 172.21.15.45:0/2436193029 learned_addr learned my addr 172.21.15.45:0/2436193029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 -- 172.21.15.45:0/2436193029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bde8c0 msgr2=0x558940b994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bde8c0 0x558940b994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 -- 172.21.15.45:0/2436193029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940b94650 msgr2=0x558940b98f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940b94650 0x558940b98f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 -- 172.21.15.45:0/2436193029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55894096b260 con 0x558940bdf1e0 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc1800640 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b999e0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4098370 tx=0x7f6bb40f0380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6d461b2278c273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6baa000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6baa000640 1 -- 172.21.15.45:0/2436193029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40f1de0 con 0x558940bdf1e0 2024-04-25T10:29:35.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6baa000640 1 -- 172.21.15.45:0/2436193029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bb40f8bc0 con 0x558940bdf1e0 2024-04-25T10:29:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.992+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba4018070 con 0x558940bdf1e0 2024-04-25T10:29:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 1 -- 172.21.15.45:0/2436193029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bb40f8dc0 con 0x558940bdf1e0 2024-04-25T10:29:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 10 client.10191 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x55894099db30 con 0x558940bdf1e0 2024-04-25T10:29:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 1 -- 172.21.15.45:0/2436193029 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4018070 con 0x558940bdf1e0 2024-04-25T10:29:35.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 1 -- 172.21.15.45:0/2436193029 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6bb40efdb0 con 0x558940bdf1e0 2024-04-25T10:29:35.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 10 client.10191 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:35.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 10 client.10191 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:29:35.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 20 client.10191 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:35.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940b9a3f0 0x558940bef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 4 client.10191 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:35.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55894097ec80 con 0x558940b9a3f0 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940b9a3f0 0x558940bef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940b9a3f0 0x558940bef960 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346bc40fcc7b95b2 server_cookie=bba7db35a3b4cd4a in_seq=0 out_seq=0 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 10 client.10191 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 1 -- 172.21.15.45:0/2436193029 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55894097ec80 con 0x558940b9a3f0 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 10 client.10191 handle_command_reply: tid=0 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 client.10191 shutdown 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 msgr2=0x558940b999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b999e0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4098370 tx=0x7f6bb40f0380 comp rx=0 tx=0).stop 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 shutdown_connections 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940b9a3f0 0x558940bef960 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940bdf1e0 0x558940b999e0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bde8c0 0x558940b994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/2436193029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940b94650 0x558940b98f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6baa000640 0 client.10191 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 >> 172.21.15.45:0/2436193029 conn(0x558940a7c870 msgr2=0x558940b929f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 shutdown_connections 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/2436193029 wait complete. 2024-04-25T10:29:35.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:34.996+0000 7f6bc6bcb000 20 client.10191 trim_cache size 0 max 16384 2024-04-25T10:29:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 Processor -- start 2024-04-25T10:29:35.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 -- start start 2024-04-25T10:29:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940bdd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940bfa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc200e00 con 0x558940b94650 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc146940 con 0x558940bde8c0 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc145d40 con 0x558940bdf1e0 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940bdd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940bdd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40214/0 (socket says 172.21.15.45:40214) 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 -- 172.21.15.45:0/682044408 learned_addr learned my addr 172.21.15.45:0/682044408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc1800640 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940bfa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 -- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 msgr2=0x558940bfa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940bfa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 -- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 msgr2=0x558940bdde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 -- 172.21.15.45:0/682044408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558940a686d0 con 0x558940b94650 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc1800640 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940bfa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.012+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940bdd910 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f6bac001ec0 tx=0x7f6bac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790808cf18d63d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bb0c00640 1 -- 172.21.15.45:0/682044408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac014450 con 0x558940b94650 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bb0c00640 1 -- 172.21.15.45:0/682044408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac02c030 con 0x558940b94650 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bb0c00640 1 -- 172.21.15.45:0/682044408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0147f0 con 0x558940b94650 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/682044408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 msgr2=0x558940bdd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940bdd910 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f6bac001ec0 tx=0x7f6bac012000 comp rx=0 tx=0).stop 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/682044408 shutdown_connections 2024-04-25T10:29:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940bfa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940bdde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/682044408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940bdd910 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/682044408 >> 172.21.15.45:0/682044408 conn(0x558940a7c870 msgr2=0x558940bdd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/682044408 shutdown_connections 2024-04-25T10:29:35.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/682044408 wait complete. 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- start start 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 Processor -- start 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940c3f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940c3ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940c40470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x558940989ae0 con 0x558940b94650 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc146340 con 0x558940bde8c0 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x558940a45000 con 0x558940bdf1e0 2024-04-25T10:29:35.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 10 client.0 _ll_register_callbacks cb 0x55894097f200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940c3f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940c3f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40226/0 (socket says 172.21.15.45:40226) 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 -- 172.21.15.45:0/3941258512 learned_addr learned my addr 172.21.15.45:0/3941258512 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940c3ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc1800640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940c40470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 -- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 msgr2=0x558940c40470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940c40470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 -- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 msgr2=0x558940c3ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940c3ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 -- 172.21.15.45:0/3941258512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558940a65900 con 0x558940b94650 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc0e00640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940c3f9f0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0011f0 tx=0x7f6bac001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3287d8601f6fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6baa000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6baa000640 1 -- 172.21.15.45:0/3941258512 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac014710 con 0x558940b94650 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6baa000640 1 -- 172.21.15.45:0/3941258512 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bac03bae0 con 0x558940b94650 2024-04-25T10:29:35.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.016+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bac02c030 con 0x558940b94650 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 1 -- 172.21.15.45:0/3941258512 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bac03bce0 con 0x558940b94650 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 10 client.10197 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x558940989f20 con 0x558940b94650 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 1 -- 172.21.15.45:0/3941258512 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac02c030 con 0x558940b94650 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 1 -- 172.21.15.45:0/3941258512 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6bac020040 con 0x558940b94650 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 10 client.10197 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 10 client.10197 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 20 client.10197 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940c40d70 0x558940c4b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 4 client.10197 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x5589409f0310 con 0x558940c40d70 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940c40d70 0x558940c4b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 10 client.10197 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bb3e00640 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940c40d70 0x558940c4b3b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec4ea0ac8b208a4 server_cookie=dcd4f7f2d0747840 in_seq=0 out_seq=0 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 1 -- 172.21.15.45:0/3941258512 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5589409f0310 con 0x558940c40d70 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 10 client.10197 handle_command_reply: tid=0 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 client.10197 shutdown 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 msgr2=0x558940c3f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940c3f9f0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0011f0 tx=0x7f6bac001090 comp rx=0 tx=0).stop 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 shutdown_connections 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x558940c40d70 0x558940c4b3b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x558940bdf1e0 0x558940c40470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6baa000640 0 client.10197 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558940bde8c0 0x558940c3ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 --2- 172.21.15.45:0/3941258512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558940b94650 0x558940c3f9f0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 >> 172.21.15.45:0/3941258512 conn(0x558940a7c870 msgr2=0x558940bdd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 shutdown_connections 2024-04-25T10:29:35.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 1 -- 172.21.15.45:0/3941258512 wait complete. 2024-04-25T10:29:35.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc6bcb000 20 client.10197 trim_cache size 0 max 16384 2024-04-25T10:29:35.027 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:29:35.027 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:29:35.027 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "fe6ca11f-d308-47fd-99df-7b47929bc1dd", 2024-04-25T10:29:35.027 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:29:35.027 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b98066f60 msgr2=0x7f6b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b98066f60 0x7f6b98069420 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6bac000f70 tx=0x7f6bac009040 comp rx=0 tx=0).stop 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 msgr2=0x7f6bbc14a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 0x7f6bbc14a3c0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f6bb401a5c0 tx=0x7f6bb4019730 comp rx=0 tx=0).stop 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 shutdown_connections 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6b98066f60 0x7f6b98069420 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bbc0d00f0 0x7f6bbc14a3c0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bbc0cf040 0x7f6bbc0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 --2- 172.21.15.45:0/4025868310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6bbc071ab0 0x7f6bbc137140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 >> 172.21.15.45:0/4025868310 conn(0x7f6bbc066b70 msgr2=0x7f6bbc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 shutdown_connections 2024-04-25T10:29:35.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.020+0000 7f6bc3800640 1 -- 172.21.15.45:0/4025868310 wait complete. 2024-04-25T10:29:35.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:29:35.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:29:35.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 Processor -- start 2024-04-25T10:29:35.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 -- start start 2024-04-25T10:29:35.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5f80d0d00 0x7fb5f80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f80cc200 0x7fb5f8162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb5f81626c0 0x7fb5f806cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80cdab0 con 0x7fb5f80d0d00 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80cd210 con 0x7fb5f81626c0 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80cd4f0 con 0x7fb5f80cc200 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5f80d0d00 0x7fb5f80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5f80d0d00 0x7fb5f80ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40240/0 (socket says 172.21.15.45:40240) 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 -- 172.21.15.45:0/3750262562 learned_addr learned my addr 172.21.15.45:0/3750262562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5fce00640 1 --2- 172.21.15.45:0/3750262562 >> v2:172.21.15.117:3300/0 conn(0x7fb5f81626c0 0x7fb5f806cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5ef400640 1 --2- 172.21.15.45:0/3750262562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f80cc200 0x7fb5f8162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 -- 172.21.15.45:0/3750262562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f80cc200 msgr2=0x7fb5f8162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/3750262562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f80cc200 0x7fb5f8162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 -- 172.21.15.45:0/3750262562 >> v2:172.21.15.117:3300/0 conn(0x7fb5f81626c0 msgr2=0x7fb5f806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/3750262562 >> v2:172.21.15.117:3300/0 conn(0x7fb5f81626c0 0x7fb5f806cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.184+0000 7fb5f7e00640 1 -- 172.21.15.45:0/3750262562 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80cd7d0 con 0x7fb5f80d0d00 2024-04-25T10:29:35.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/3750262562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80ccc30 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8070510 tx=0x7fb5e801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c7bf33ec9692b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5f7400640 1 -- 172.21.15.45:0/3750262562 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80b0070 con 0x7fb5f80d0d00 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5f7400640 1 -- 172.21.15.45:0/3750262562 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80ab600 con 0x7fb5f80d0d00 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5f7400640 1 -- 172.21.15.45:0/3750262562 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80ab9d0 con 0x7fb5f80d0d00 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 -- 172.21.15.45:0/3750262562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80d0d00 msgr2=0x7fb5f80ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 --2- 172.21.15.45:0/3750262562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80ccc30 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8070510 tx=0x7fb5e801c490 comp rx=0 tx=0).stop 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 -- 172.21.15.45:0/3750262562 shutdown_connections 2024-04-25T10:29:35.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 --2- 172.21.15.45:0/3750262562 >> v2:172.21.15.117:3300/0 conn(0x7fb5f81626c0 0x7fb5f806cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 --2- 172.21.15.45:0/3750262562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f80cc200 0x7fb5f8162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 --2- 172.21.15.45:0/3750262562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80ccc30 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 -- 172.21.15.45:0/3750262562 >> 172.21.15.45:0/3750262562 conn(0x7fb5f8072290 msgr2=0x7fb5f80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 -- 172.21.15.45:0/3750262562 shutdown_connections 2024-04-25T10:29:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 -- 172.21.15.45:0/3750262562 wait complete. 2024-04-25T10:29:35.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 Processor -- start 2024-04-25T10:29:35.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.188+0000 7fb5fee00640 1 -- start start 2024-04-25T10:29:35.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 0x7fb5f80997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f81626c0 0x7fb5f80c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f806ed00 con 0x7fb5f80cc200 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f806eb80 con 0x7fb5f80d0d00 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f806ee80 con 0x7fb5f81626c0 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 0x7fb5f80997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 0x7fb5f80997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40242/0 (socket says 172.21.15.45:40242) 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2023678736 learned_addr learned my addr 172.21.15.45:0/2023678736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fce00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f81626c0 0x7fb5f80c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5ef400640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f81626c0 msgr2=0x7fb5f80c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f81626c0 0x7fb5f80c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f80d0d00 msgr2=0x7fb5f80c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2023678736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f815bd30 con 0x7fb5f80cc200 2024-04-25T10:29:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 0x7fb5f80997e0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80097c0 tx=0x7fb5e801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f4c00640 1 -- 172.21.15.45:0/2023678736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80ba020 con 0x7fb5f80cc200 2024-04-25T10:29:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f4c00640 1 -- 172.21.15.45:0/2023678736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80b6640 con 0x7fb5f80cc200 2024-04-25T10:29:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5f4c00640 1 -- 172.21.15.45:0/2023678736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80b0040 con 0x7fb5f80cc200 2024-04-25T10:29:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f80c4d00 con 0x7fb5f80cc200 2024-04-25T10:29:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.192+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f80c52c0 con 0x7fb5f80cc200 2024-04-25T10:29:35.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5f4c00640 1 -- 172.21.15.45:0/2023678736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5e80bd020 con 0x7fb5f80cc200 2024-04-25T10:29:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5f4c00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5d4066f60 0x7fb5d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5f4c00640 1 -- 172.21.15.45:0/2023678736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5e811eff0 con 0x7fb5f80cc200 2024-04-25T10:29:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5d4066f60 0x7fb5d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 Processor -- start 2024-04-25T10:29:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 -- start start 2024-04-25T10:29:35.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5d4066f60 0x7fb5d4069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb5f81fa8a0 tx=0x7fb5e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee79adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee750b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee7510a0 0x5556ee751480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ee59dff0 con 0x5556ee79b2f0 2024-04-25T10:29:35.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ee526150 con 0x5556ee79a9d0 2024-04-25T10:29:35.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556ee59a090 con 0x5556ee7510a0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee750b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee750b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40256/0 (socket says 172.21.15.45:40256) 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 -- 172.21.15.45:0/1432443597 learned_addr learned my addr 172.21.15.45:0/1432443597 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee79adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5fce00640 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee7510a0 0x5556ee751480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 -- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee7510a0 msgr2=0x5556ee751480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee7510a0 0x5556ee751480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 -- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 msgr2=0x5556ee79adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee79adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 -- 172.21.15.45:0/1432443597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ee7340d0 con 0x5556ee79b2f0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.196+0000 7fb5ef400640 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee750b60 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000b820 tx=0x7fb5e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fcc6482a89e90e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5ecc00640 1 -- 172.21.15.45:0/1432443597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e001c050 con 0x5556ee79b2f0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5ecc00640 1 -- 172.21.15.45:0/1432443597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e0026070 con 0x5556ee79b2f0 2024-04-25T10:29:35.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5ecc00640 1 -- 172.21.15.45:0/1432443597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0016c00 con 0x5556ee79b2f0 2024-04-25T10:29:35.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- 172.21.15.45:0/1432443597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 msgr2=0x5556ee750b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee750b60 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000b820 tx=0x7fb5e0013000 comp rx=0 tx=0).stop 2024-04-25T10:29:35.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- 172.21.15.45:0/1432443597 shutdown_connections 2024-04-25T10:29:35.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee7510a0 0x5556ee751480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee750b60 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- 172.21.15.45:0/1432443597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee79adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- 172.21.15.45:0/1432443597 >> 172.21.15.45:0/1432443597 conn(0x5556ee74d020 msgr2=0x5556ee74f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- 172.21.15.45:0/1432443597 shutdown_connections 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- 172.21.15.45:0/1432443597 wait complete. 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- start start 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 Processor -- start 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 0x5556ee77f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee77fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee780270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ee562250 con 0x5556ee7510a0 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ee558690 con 0x5556ee79a9d0 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556ee526a40 con 0x5556ee79b2f0 2024-04-25T10:29:35.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb602356000 10 client.0 _ll_register_callbacks cb 0x5556ee73be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 0x5556ee77f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 0x5556ee77f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40262/0 (socket says 172.21.15.45:40262) 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2274019457 learned_addr learned my addr 172.21.15.45:0/2274019457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5ef400640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee77fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5fce00640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee780270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 msgr2=0x5556ee780270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee780270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 msgr2=0x5556ee77fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee77fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 -- 172.21.15.45:0/2274019457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556ee64b1c0 con 0x5556ee7510a0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 0x5556ee77f7f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80abfd0 tx=0x7fb5e80ef500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ec039f74160bd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5e5600640 1 -- 172.21.15.45:0/2274019457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80f7cb0 con 0x5556ee7510a0 2024-04-25T10:29:35.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.200+0000 7fb5e5600640 1 -- 172.21.15.45:0/2274019457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5e80f7e50 con 0x5556ee7510a0 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5e0026070 con 0x5556ee7510a0 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 1 -- 172.21.15.45:0/2274019457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e811d260 con 0x5556ee7510a0 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 10 client.10209 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x5556ee647170 con 0x5556ee7510a0 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 1 -- 172.21.15.45:0/2274019457 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e0026070 con 0x5556ee7510a0 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 1 -- 172.21.15.45:0/2274019457 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb5e80b3030 con 0x5556ee7510a0 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 10 client.10209 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 10 client.10209 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 20 client.10209 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7aac80 0x5556ee7ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 4 client.10209 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556ee539c60 con 0x5556ee7aac80 2024-04-25T10:29:35.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5ef400640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7aac80 0x5556ee7ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 10 client.10209 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5ef400640 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7aac80 0x5556ee7ad070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb2ea606fa875a4 server_cookie=30456de0fb4c9ada in_seq=0 out_seq=0 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 1 -- 172.21.15.45:0/2274019457 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556ee539c60 con 0x5556ee7aac80 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 10 client.10209 handle_command_reply: tid=0 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 client.10209 shutdown 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 msgr2=0x5556ee77f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 0x5556ee77f7f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80abfd0 tx=0x7fb5e80ef500 comp rx=0 tx=0).stop 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 shutdown_connections 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7aac80 0x5556ee7ad070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee780270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee79a9d0 0x5556ee77fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 --2- 172.21.15.45:0/2274019457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee7510a0 0x5556ee77f7f0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb5e5600640 0 client.10209 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 >> 172.21.15.45:0/2274019457 conn(0x5556ee74d020 msgr2=0x5556ee74fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 shutdown_connections 2024-04-25T10:29:35.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 1 -- 172.21.15.45:0/2274019457 wait complete. 2024-04-25T10:29:35.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.204+0000 7fb602356000 20 client.10209 trim_cache size 0 max 16384 2024-04-25T10:29:35.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 Processor -- start 2024-04-25T10:29:35.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 -- start start 2024-04-25T10:29:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79a9d0 0x5556ee7f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee7f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f82031a0 con 0x5556ee79b2f0 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8202360 con 0x5556ee7510a0 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8201760 con 0x5556ee79a9d0 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee7f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee7f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40272/0 (socket says 172.21.15.45:40272) 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 -- 172.21.15.45:0/847341978 learned_addr learned my addr 172.21.15.45:0/847341978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5ef400640 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79a9d0 0x5556ee7f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 -- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79a9d0 msgr2=0x5556ee7f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79a9d0 0x5556ee7f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 -- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 msgr2=0x5556ee7f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 -- 172.21.15.45:0/847341978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ee5e4de0 con 0x5556ee79b2f0 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.220+0000 7fb5fce00640 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee7f7b70 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0003c20 tx=0x7fb5f0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0b260d9bb54823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5e7e00640 1 -- 172.21.15.45:0/847341978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f001c070 con 0x5556ee79b2f0 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5e7e00640 1 -- 172.21.15.45:0/847341978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0018450 con 0x5556ee79b2f0 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5e7e00640 1 -- 172.21.15.45:0/847341978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0018840 con 0x5556ee79b2f0 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- 172.21.15.45:0/847341978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 msgr2=0x5556ee7f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee7f7b70 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0003c20 tx=0x7fb5f0009820 comp rx=0 tx=0).stop 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- 172.21.15.45:0/847341978 shutdown_connections 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79b2f0 0x5556ee7f7b70 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79a9d0 0x5556ee7f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- 172.21.15.45:0/847341978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- 172.21.15.45:0/847341978 >> 172.21.15.45:0/847341978 conn(0x5556ee74d020 msgr2=0x5556ee7a7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- 172.21.15.45:0/847341978 shutdown_connections 2024-04-25T10:29:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- 172.21.15.45:0/847341978 wait complete. 2024-04-25T10:29:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- start start 2024-04-25T10:29:35.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 Processor -- start 2024-04-25T10:29:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 0x5556ee7ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee7ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ee538e00 con 0x5556ee79a9d0 2024-04-25T10:29:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8201d60 con 0x5556ee7510a0 2024-04-25T10:29:35.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556ee59d300 con 0x5556ee79b2f0 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 10 client.0 _ll_register_callbacks cb 0x5556ee71c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 0x5556ee7ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 0x5556ee7ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40276/0 (socket says 172.21.15.45:40276) 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 -- 172.21.15.45:0/1064333400 learned_addr learned my addr 172.21.15.45:0/1064333400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5fce00640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee7ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 -- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 msgr2=0x5556ee7ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee7ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 -- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 msgr2=0x5556ee7ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 -- 172.21.15.45:0/1064333400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556ee53f8e0 con 0x5556ee79a9d0 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:35.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5fce00640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee7ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5ef400640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 0x5556ee7ad8c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0023040 tx=0x7fb5e0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec6f458d48eed32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5e5600640 1 -- 172.21.15.45:0/1064333400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e001c040 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb5e5600640 1 -- 172.21.15.45:0/1064333400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5e0034b70 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.224+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5f0018450 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 1 -- 172.21.15.45:0/1064333400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e0034d70 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 10 client.10215 fetch_fsmap learned FSMap version 115 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x5556ee5f78f0 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 1 -- 172.21.15.45:0/1064333400 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0018450 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 1 -- 172.21.15.45:0/1064333400 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb5e00059a0 con 0x5556ee79a9d0 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 10 client.10215 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 10 client.10215 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:29:35.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 20 client.10215 populate_metadata read hostname 'smithi045' 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7ae9c0 0x5556ee802f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 4 client.10215 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556ee5ab530 con 0x5556ee7ae9c0 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7ae9c0 0x5556ee802f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 10 client.10215 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5f7e00640 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7ae9c0 0x5556ee802f60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85411699d3636919 server_cookie=c1b55367c81b699a in_seq=0 out_seq=0 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 1 -- 172.21.15.45:0/1064333400 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x5556ee5ab530 con 0x5556ee7ae9c0 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 10 client.10215 handle_command_reply: tid=0 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 client.10215 shutdown 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 msgr2=0x5556ee7ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 0x5556ee7ad8c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0023040 tx=0x7fb5e0004740 comp rx=0 tx=0).stop 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 shutdown_connections 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5556ee7ae9c0 0x5556ee802f60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5556ee79b2f0 0x5556ee7ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556ee79a9d0 0x5556ee7ad8c0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 --2- 172.21.15.45:0/1064333400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5556ee7510a0 0x5556ee7ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5e5600640 0 client.10215 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 >> 172.21.15.45:0/1064333400 conn(0x5556ee74d020 msgr2=0x5556ee7990d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 shutdown_connections 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 1 -- 172.21.15.45:0/1064333400 wait complete. 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb602356000 20 client.10215 trim_cache size 0 max 16384 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:29:35.233 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "fe6ca11f-d308-47fd-99df-7b47929bc1dd": { 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "~mds0", 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fe6ca11f-d308-47fd-99df-7b47929bc1dd", 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:29:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5d4066f60 msgr2=0x7fb5d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5d4066f60 0x7fb5d4069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb5f81fa8a0 tx=0x7fb5e0008040 comp rx=0 tx=0).stop 2024-04-25T10:29:35.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 msgr2=0x7fb5f80997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 0x7fb5f80997e0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80097c0 tx=0x7fb5e801acd0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 shutdown_connections 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5d4066f60 0x7fb5d4069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5f81626c0 0x7fb5f80c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5f80d0d00 0x7fb5f80c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 --2- 172.21.15.45:0/2023678736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5f80cc200 0x7fb5f80997e0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 >> 172.21.15.45:0/2023678736 conn(0x7fb5f8072290 msgr2=0x7fb5f8160660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 shutdown_connections 2024-04-25T10:29:35.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:29:35.228+0000 7fb5fee00640 1 -- 172.21.15.45:0/2023678736 wait complete. 2024-04-25T10:29:35.251 INFO:tasks.cephfs.filesystem:scrub status for tag:fe6ca11f-d308-47fd-99df-7b47929bc1dd - {'path': '~mds0', 'tag': 'fe6ca11f-d308-47fd-99df-7b47929bc1dd', 'options': 'recursive'} 2024-04-25T10:29:59.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:00.001 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:05.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:05.253 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:30:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 Processor -- start 2024-04-25T10:30:05.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 -- start start 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c0aeda0 0x7f1c4c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c4c072020 0x7f1c4c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c159920 con 0x7f1c4c159540 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c15c2f0 con 0x7f1c4c072020 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c54600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c15c5d0 con 0x7f1c4c0aeda0 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c52600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c4c072020 0x7f1c4c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c51c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1c4c159540 0x7f1c4c0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56324/0 (socket says 172.21.15.45:56324) 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c51c00640 1 -- 172.21.15.45:0/1227343433 learned_addr learned my addr 172.21.15.45:0/1227343433 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c51200640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c0aeda0 0x7f1c4c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c52600640 1 -- 172.21.15.45:0/1227343433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c0aeda0 msgr2=0x7f1c4c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c52600640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c0aeda0 0x7f1c4c071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c52600640 1 -- 172.21.15.45:0/1227343433 >> v1:172.21.15.45:6789/0 conn(0x7f1c4c159540 legacy=0x7f1c4c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c52600640 1 -- 172.21.15.45:0/1227343433 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c15c8b0 con 0x7f1c4c072020 2024-04-25T10:30:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.407+0000 7f1c51200640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c0aeda0 0x7f1c4c071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c52600640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c070350 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009d90 tx=0x7f1c4401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8214ea584e2da86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c4be00640 1 -- 172.21.15.45:0/1227343433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440b2070 con 0x7f1c4c072020 2024-04-25T10:30:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c4be00640 1 -- 172.21.15.45:0/1227343433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c440a5a20 con 0x7f1c4c072020 2024-04-25T10:30:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c4be00640 1 -- 172.21.15.45:0/1227343433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440a5df0 con 0x7f1c4c072020 2024-04-25T10:30:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 -- 172.21.15.45:0/1227343433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 msgr2=0x7f1c4c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c070350 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009d90 tx=0x7f1c4401c940 comp rx=0 tx=0).stop 2024-04-25T10:30:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 -- 172.21.15.45:0/1227343433 shutdown_connections 2024-04-25T10:30:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c070350 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 --2- 172.21.15.45:0/1227343433 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c0aeda0 0x7f1c4c071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 -- 172.21.15.45:0/1227343433 >> 172.21.15.45:0/1227343433 conn(0x7f1c4c0669a0 msgr2=0x7f1c4c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 -- 172.21.15.45:0/1227343433 shutdown_connections 2024-04-25T10:30:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.411+0000 7f1c54600640 1 -- 172.21.15.45:0/1227343433 wait complete. 2024-04-25T10:30:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 Processor -- start 2024-04-25T10:30:05.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 -- start start 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c4c0aeda0 0x7f1c4c14f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c159540 0x7f1c4c14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0710a0 con 0x7f1c4c072020 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c06c0d0 con 0x7f1c4c0aeda0 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c071220 con 0x7f1c4c159540 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c14e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39074/0 (socket says 172.21.15.45:39074) 2024-04-25T10:30:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 -- 172.21.15.45:0/774109179 learned_addr learned my addr 172.21.15.45:0/774109179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c52600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c159540 0x7f1c4c14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51200640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c4c0aeda0 0x7f1c4c14f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 -- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c159540 msgr2=0x7f1c4c14f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c159540 0x7f1c4c14f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 -- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c4c0aeda0 msgr2=0x7f1c4c14f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c4c0aeda0 0x7f1c4c14f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 -- 172.21.15.45:0/774109179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c153760 con 0x7f1c4c072020 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c52600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c159540 0x7f1c4c14f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c51c00640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c14e2e0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c420 tx=0x7f1c4000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c49600640 1 -- 172.21.15.45:0/774109179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40017070 con 0x7f1c4c072020 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c49600640 1 -- 172.21.15.45:0/774109179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c40005cf0 con 0x7f1c4c072020 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c153a40 con 0x7f1c4c072020 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c49600640 1 -- 172.21.15.45:0/774109179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400072f0 con 0x7f1c4c072020 2024-04-25T10:30:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.415+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c154000 con 0x7f1c4c072020 2024-04-25T10:30:05.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c49600640 1 -- 172.21.15.45:0/774109179 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c4001a070 con 0x7f1c4c072020 2024-04-25T10:30:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c49600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1c2c066f60 0x7f1c2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c49600640 1 -- 172.21.15.45:0/774109179 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1c40084420 con 0x7f1c4c072020 2024-04-25T10:30:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 Processor -- start 2024-04-25T10:30:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1c2c066f60 0x7f1c2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 -- start start 2024-04-25T10:30:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1c2c066f60 0x7f1c2c069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c151aa0 tx=0x7f1c38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d667b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6676eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d6676f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d665bbff0 con 0x562d667b92f0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d66544150 con 0x562d6676f0a0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d665b8090 con 0x562d667b89d0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6676eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6676eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39078/0 (socket says 172.21.15.45:39078) 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 -- 172.21.15.45:0/1783926538 learned_addr learned my addr 172.21.15.45:0/1783926538 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51c00640 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d667b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c52600640 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d6676f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 -- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 msgr2=0x562d667b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d667b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 -- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 msgr2=0x562d6676f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d6676f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.419+0000 7f1c51200640 1 -- 172.21.15.45:0/1783926538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d667520d0 con 0x562d667b92f0 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c51200640 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6676eb60 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f1c38010970 tx=0x7f1c38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae66151ea65d76b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c3e000640 1 -- 172.21.15.45:0/1783926538 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3801c050 con 0x562d667b92f0 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c3e000640 1 -- 172.21.15.45:0/1783926538 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38026070 con 0x562d667b92f0 2024-04-25T10:30:05.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c3e000640 1 -- 172.21.15.45:0/1783926538 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800f040 con 0x562d667b92f0 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- 172.21.15.45:0/1783926538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 msgr2=0x562d6676eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6676eb60 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f1c38010970 tx=0x7f1c38013000 comp rx=0 tx=0).stop 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- 172.21.15.45:0/1783926538 shutdown_connections 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d6676f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6676eb60 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/1783926538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d667b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- 172.21.15.45:0/1783926538 >> 172.21.15.45:0/1783926538 conn(0x562d6676b020 msgr2=0x562d6676d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- 172.21.15.45:0/1783926538 shutdown_connections 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- 172.21.15.45:0/1783926538 wait complete. 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- start start 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 Processor -- start 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d6676f0a0 0x562d6679d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d667b89d0 0x562d6679dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6679e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d66580250 con 0x562d667b92f0 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d66576690 con 0x562d667b89d0 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d66544a40 con 0x562d6676f0a0 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c57a9c000 10 client.0 _ll_register_callbacks cb 0x562d66759e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6679e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c51c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d6676f0a0 0x562d6679d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6679e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39080/0 (socket says 172.21.15.45:39080) 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 -- 172.21.15.45:0/3639750711 learned_addr learned my addr 172.21.15.45:0/3639750711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c51200640 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d667b89d0 0x562d6679dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 -- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d6676f0a0 msgr2=0x562d6679d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d6676f0a0 0x562d6679d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 -- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d667b89d0 msgr2=0x562d6679dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d667b89d0 0x562d6679dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.423+0000 7f1c52600640 1 -- 172.21.15.45:0/3639750711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d666691c0 con 0x562d667b92f0 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c52600640 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6679e270 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009d90 tx=0x7f1c4401b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d16400a0b4e8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 1 -- 172.21.15.45:0/3639750711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440b2040 con 0x562d667b92f0 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 1 -- 172.21.15.45:0/3639750711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1c440ad640 con 0x562d667b92f0 2024-04-25T10:30:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c38026070 con 0x562d667b92f0 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 1 -- 172.21.15.45:0/3639750711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c440cb050 con 0x562d667b92f0 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 10 client.10230 fetch_fsmap learned FSMap version 115 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x562d66665170 con 0x562d667b92f0 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 1 -- 172.21.15.45:0/3639750711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38026070 con 0x562d667b92f0 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 1 -- 172.21.15.45:0/3639750711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c440ce020 con 0x562d667b92f0 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 10 client.10230 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 10 client.10230 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 20 client.10230 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667c8c80 0x562d667cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 4 client.10230 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d66557c60 con 0x562d667c8c80 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c51c00640 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667c8c80 0x562d667cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 10 client.10230 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c51c00640 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667c8c80 0x562d667cb070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3ca1dae65d4bad server_cookie=6bae125933b68f6c in_seq=0 out_seq=0 2024-04-25T10:30:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 1 -- 172.21.15.45:0/3639750711 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d66557c60 con 0x562d667c8c80 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 10 client.10230 handle_command_reply: tid=0 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 client.10230 shutdown 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 msgr2=0x562d6679e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6679e270 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009d90 tx=0x7f1c4401b180 comp rx=0 tx=0).stop 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 shutdown_connections 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667c8c80 0x562d667cb070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d6679e270 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d667b89d0 0x562d6679dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3639750711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d6676f0a0 0x562d6679d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c32a00640 0 client.10230 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 >> 172.21.15.45:0/3639750711 conn(0x562d6676b020 msgr2=0x562d6676dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 shutdown_connections 2024-04-25T10:30:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3639750711 wait complete. 2024-04-25T10:30:05.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.427+0000 7f1c57a9c000 20 client.10230 trim_cache size 0 max 16384 2024-04-25T10:30:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 Processor -- start 2024-04-25T10:30:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 -- start start 2024-04-25T10:30:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d668150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d66815630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d66815b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b4560 con 0x562d667b92f0 2024-04-25T10:30:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b09b0 con 0x562d6676f0a0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c156480 con 0x562d667b89d0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d66815b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d66815b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39084/0 (socket says 172.21.15.45:39084) 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 -- 172.21.15.45:0/2970179761 learned_addr learned my addr 172.21.15.45:0/2970179761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c51c00640 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d668150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c51200640 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d66815630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 -- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 msgr2=0x562d66815630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d66815630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 -- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 msgr2=0x562d668150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d668150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c52600640 1 -- 172.21.15.45:0/2970179761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d66602de0 con 0x562d667b92f0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.443+0000 7f1c51c00640 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d668150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c52600640 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d66815b70 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c090 tx=0x7f1c440ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02eb1f89295015d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c3d600640 1 -- 172.21.15.45:0/2970179761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440b2070 con 0x562d667b92f0 2024-04-25T10:30:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c3d600640 1 -- 172.21.15.45:0/2970179761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c440ad4c0 con 0x562d667b92f0 2024-04-25T10:30:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c3d600640 1 -- 172.21.15.45:0/2970179761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440a5dc0 con 0x562d667b92f0 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- 172.21.15.45:0/2970179761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 msgr2=0x562d66815b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d66815b70 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c090 tx=0x7f1c440ba000 comp rx=0 tx=0).stop 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- 172.21.15.45:0/2970179761 shutdown_connections 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b92f0 0x562d66815b70 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b89d0 0x562d66815630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/2970179761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d668150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- 172.21.15.45:0/2970179761 >> 172.21.15.45:0/2970179761 conn(0x562d6676b020 msgr2=0x562d667c5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- 172.21.15.45:0/2970179761 shutdown_connections 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- 172.21.15.45:0/2970179761 wait complete. 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- start start 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 Processor -- start 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d667cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 0x562d667cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b92f0 0x562d667cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d66556e00 con 0x562d667b89d0 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b03b0 con 0x562d6676f0a0 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d665bb300 con 0x562d667b92f0 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c57a9c000 10 client.0 _ll_register_callbacks cb 0x562d6673a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 0x562d667cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 0x562d667cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39100/0 (socket says 172.21.15.45:39100) 2024-04-25T10:30:05.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d667cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51200640 1 -- 172.21.15.45:0/3670258466 learned_addr learned my addr 172.21.15.45:0/3670258466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c52600640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b92f0 0x562d667cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 -- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b92f0 msgr2=0x562d667cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b92f0 0x562d667cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 -- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 msgr2=0x562d667cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 0x562d667cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 -- 172.21.15.45:0/3670258466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d6655d8e0 con 0x562d6676f0a0 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51200640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 0x562d667cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c51c00640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d667cb380 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1c40082920 tx=0x7f1c4001e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f23c0700bcff616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.447+0000 7f1c32a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:30:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 1 -- 172.21.15.45:0/3670258466 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c400840b0 con 0x562d6676f0a0 2024-04-25T10:30:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 1 -- 172.21.15.45:0/3670258466 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1c40051370 con 0x562d6676f0a0 2024-04-25T10:30:05.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c440ad4c0 con 0x562d6676f0a0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 1 -- 172.21.15.45:0/3670258466 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c40007490 con 0x562d6676f0a0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 10 client.9535 fetch_fsmap learned FSMap version 115 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x562d666158f0 con 0x562d6676f0a0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 1 -- 172.21.15.45:0/3670258466 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c440ad4c0 con 0x562d6676f0a0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 1 -- 172.21.15.45:0/3670258466 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c40054150 con 0x562d6676f0a0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 10 client.9535 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 10 client.9535 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 20 client.9535 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667ccad0 0x562d66820f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 4 client.9535 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d665c9530 con 0x562d667ccad0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c51200640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667ccad0 0x562d66820f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c51200640 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667ccad0 0x562d66820f60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362a9d01f6aa2370 server_cookie=fba89b94065d890e in_seq=0 out_seq=0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 10 client.9535 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 1 -- 172.21.15.45:0/3670258466 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562d665c9530 con 0x562d667ccad0 2024-04-25T10:30:05.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 10 client.9535 handle_command_reply: tid=0 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 client.9535 shutdown 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 msgr2=0x562d667cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d667cb380 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1c40082920 tx=0x7f1c4001e040 comp rx=0 tx=0).stop 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 shutdown_connections 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x562d667ccad0 0x562d66820f60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562d667b92f0 0x562d667cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d667b89d0 0x562d667cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 --2- 172.21.15.45:0/3670258466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562d6676f0a0 0x562d667cb380 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c32a00640 0 client.9535 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 >> 172.21.15.45:0/3670258466 conn(0x562d6676b020 msgr2=0x562d667b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 shutdown_connections 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 1 -- 172.21.15.45:0/3670258466 wait complete. 2024-04-25T10:30:05.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c57a9c000 20 client.9535 trim_cache size 0 max 16384 2024-04-25T10:30:05.458 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:30:05.458 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:30:05.458 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:30:05.458 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1c2c066f60 msgr2=0x7f1c2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1c2c066f60 0x7f1c2c069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c151aa0 tx=0x7f1c38008040 comp rx=0 tx=0).stop 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 msgr2=0x7f1c4c14e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c14e2e0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c420 tx=0x7f1c4000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 shutdown_connections 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1c2c066f60 0x7f1c2c069420 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c4c159540 0x7f1c4c14f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c4c0aeda0 0x7f1c4c14f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 --2- 172.21.15.45:0/774109179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c4c072020 0x7f1c4c14e2e0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 >> 172.21.15.45:0/774109179 conn(0x7f1c4c0669a0 msgr2=0x7f1c4c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 shutdown_connections 2024-04-25T10:30:05.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.451+0000 7f1c54600640 1 -- 172.21.15.45:0/774109179 wait complete. 2024-04-25T10:30:05.473 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:30:05.473 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:30:05.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:30:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 Processor -- start 2024-04-25T10:30:05.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 -- start start 2024-04-25T10:30:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d80aeda0 0x7f65d8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f65d8072020 0x7f65d8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d8159920 con 0x7f65d8159540 2024-04-25T10:30:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d815c2f0 con 0x7f65d8072020 2024-04-25T10:30:05.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65dda00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d815c5d0 con 0x7f65d80aeda0 2024-04-25T10:30:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d7400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f65d8072020 0x7f65d8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65cfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d80aeda0 0x7f65d8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d6a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f65d8159540 0x7f65d80ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:35202/0 (socket says 172.21.15.45:35202) 2024-04-25T10:30:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d6a00640 1 -- 172.21.15.45:0/2407337438 learned_addr learned my addr 172.21.15.45:0/2407337438 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d6000640 1 -- 172.21.15.45:0/2407337438 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355426126 0 0) 0x7f65d8159920 con 0x7f65d8159540 2024-04-25T10:30:05.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d7400640 1 -- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d80aeda0 msgr2=0x7f65d8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d7400640 1 --2- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d80aeda0 0x7f65d8071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d7400640 1 -- 172.21.15.45:0/2407337438 >> v1:172.21.15.117:6789/0 conn(0x7f65d8159540 legacy=0x7f65d80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.615+0000 7f65d7400640 1 -- 172.21.15.45:0/2407337438 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d815c8b0 con 0x7f65d8072020 2024-04-25T10:30:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65d7400640 1 --2- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8072020 0x7f65d8070350 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009d90 tx=0x7f65d001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69294b4487a2224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65d6000640 1 -- 172.21.15.45:0/2407337438 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00b2070 con 0x7f65d8072020 2024-04-25T10:30:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65d6000640 1 -- 172.21.15.45:0/2407337438 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d00a5a20 con 0x7f65d8072020 2024-04-25T10:30:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65d6000640 1 -- 172.21.15.45:0/2407337438 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00a5df0 con 0x7f65d8072020 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 -- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8072020 msgr2=0x7f65d8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 --2- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8072020 0x7f65d8070350 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009d90 tx=0x7f65d001c940 comp rx=0 tx=0).stop 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 -- 172.21.15.45:0/2407337438 shutdown_connections 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 --2- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8072020 0x7f65d8070350 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 --2- 172.21.15.45:0/2407337438 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d80aeda0 0x7f65d8071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 -- 172.21.15.45:0/2407337438 >> 172.21.15.45:0/2407337438 conn(0x7f65d80669a0 msgr2=0x7f65d8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 -- 172.21.15.45:0/2407337438 shutdown_connections 2024-04-25T10:30:05.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.619+0000 7f65dda00640 1 -- 172.21.15.45:0/2407337438 wait complete. 2024-04-25T10:30:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 Processor -- start 2024-04-25T10:30:05.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 -- start start 2024-04-25T10:30:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d8072020 0x7f65d81faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 0x7f65d81451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8159540 0x7f65d80b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80710a0 con 0x7f65d80aeda0 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d806c0d0 con 0x7f65d8159540 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8071220 con 0x7f65d8072020 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 0x7f65d81451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 0x7f65d81451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39102/0 (socket says 172.21.15.45:39102) 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 -- 172.21.15.45:0/2492148937 learned_addr learned my addr 172.21.15.45:0/2492148937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65d7400640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8159540 0x7f65d80b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d8072020 0x7f65d81faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 -- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d8072020 msgr2=0x7f65d81faaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d8072020 0x7f65d81faaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 -- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8159540 msgr2=0x7f65d80b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8159540 0x7f65d80b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 -- 172.21.15.45:0/2492148937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80b3290 con 0x7f65d80aeda0 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65d7400640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8159540 0x7f65d80b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 0x7f65d81451e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f65c000e930 tx=0x7f65c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cea00640 1 -- 172.21.15.45:0/2492148937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0017070 con 0x7f65d80aeda0 2024-04-25T10:30:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cea00640 1 -- 172.21.15.45:0/2492148937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000ca00 con 0x7f65d80aeda0 2024-04-25T10:30:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d80b3570 con 0x7f65d80aeda0 2024-04-25T10:30:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65cea00640 1 -- 172.21.15.45:0/2492148937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000cdb0 con 0x7f65d80aeda0 2024-04-25T10:30:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.623+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d80b3b30 con 0x7f65d80aeda0 2024-04-25T10:30:05.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65cea00640 1 -- 172.21.15.45:0/2492148937 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65c001a070 con 0x7f65d80aeda0 2024-04-25T10:30:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65cea00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65cea00640 1 -- 172.21.15.45:0/2492148937 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f65c00854f0 con 0x7f65d80aeda0 2024-04-25T10:30:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 Processor -- start 2024-04-25T10:30:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 -- start start 2024-04-25T10:30:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b4066f60 0x7f65b4069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000f70 tx=0x7f65c8007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 0x5582d2cb4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4aa0 0x5582d2c6a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d2ab7f20 con 0x5582d2cb4180 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d2a40000 con 0x5582d2c6a600 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.627+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582d2ab3fc0 con 0x5582d2cb4aa0 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 0x5582d2cb4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 0x5582d2cb4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39116/0 (socket says 172.21.15.45:39116) 2024-04-25T10:30:05.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 -- 172.21.15.45:0/3015316992 learned_addr learned my addr 172.21.15.45:0/3015316992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d7400640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65cfe00640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4aa0 0x5582d2c6a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 -- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4aa0 msgr2=0x5582d2c6a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4aa0 0x5582d2c6a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 -- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 msgr2=0x5582d2c6a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c6a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 -- 172.21.15.45:0/3015316992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582d2a93ba0 con 0x5582d2cb4180 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d7400640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c6a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65cfe00640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4aa0 0x5582d2c6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 0x5582d2cb4560 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f65c8005d10 tx=0x7f65c8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d350e4fd7d466e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65c7400640 1 -- 172.21.15.45:0/3015316992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8015110 con 0x5582d2cb4180 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65c7400640 1 -- 172.21.15.45:0/3015316992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80159c0 con 0x5582d2cb4180 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65c7400640 1 -- 172.21.15.45:0/3015316992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8015d70 con 0x5582d2cb4180 2024-04-25T10:30:05.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- 172.21.15.45:0/3015316992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 msgr2=0x5582d2cb4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 0x5582d2cb4560 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f65c8005d10 tx=0x7f65c8012000 comp rx=0 tx=0).stop 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- 172.21.15.45:0/3015316992 shutdown_connections 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c6a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4aa0 0x5582d2c6a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- 172.21.15.45:0/3015316992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4180 0x5582d2cb4560 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- 172.21.15.45:0/3015316992 >> 172.21.15.45:0/3015316992 conn(0x5582d2c70760 msgr2=0x5582d2c689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- 172.21.15.45:0/3015316992 shutdown_connections 2024-04-25T10:30:05.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- 172.21.15.45:0/3015316992 wait complete. 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- start start 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 Processor -- start 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2c6a600 0x5582d2c6f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2cb4180 0x5582d2c6fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 0x5582d2ca2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d2b759c0 con 0x5582d2cb4aa0 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d2a725d0 con 0x5582d2cb4180 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582d2a74080 con 0x5582d2c6a600 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65e0e69000 10 client.0 _ll_register_callbacks cb 0x5582d2c55a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 0x5582d2ca2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 0x5582d2ca2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39118/0 (socket says 172.21.15.45:39118) 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d7400640 1 -- 172.21.15.45:0/2085380940 learned_addr learned my addr 172.21.15.45:0/2085380940 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65cfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2cb4180 0x5582d2c6fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.631+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2c6a600 0x5582d2c6f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d7400640 1 -- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2c6a600 msgr2=0x5582d2c6f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d7400640 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2c6a600 0x5582d2c6f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d7400640 1 -- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2cb4180 msgr2=0x5582d2c6fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d7400640 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2cb4180 0x5582d2c6fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d7400640 1 -- 172.21.15.45:0/2085380940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582d2afe5a0 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d7400640 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 0x5582d2ca2450 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f65d00b0040 tx=0x7f65d001b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477624981ce1abd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 1 -- 172.21.15.45:0/2085380940 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00b2040 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 1 -- 172.21.15.45:0/2085380940 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f65d00ad440 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65c80159c0 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 1 -- 172.21.15.45:0/2085380940 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65d00ca050 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 10 client.10245 fetch_fsmap learned FSMap version 115 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x5582d2a8f2b0 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 1 -- 172.21.15.45:0/2085380940 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80159c0 con 0x5582d2cb4aa0 2024-04-25T10:30:05.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 1 -- 172.21.15.45:0/2085380940 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65d00cd020 con 0x5582d2cb4aa0 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 10 client.10245 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 10 client.10245 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 20 client.10245 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2ca2e60 0x5582d2cc6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 4 client.10245 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582d2a53c70 con 0x5582d2ca2e60 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2ca2e60 0x5582d2cc6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2ca2e60 0x5582d2cc6440 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f269763a09ffe74 server_cookie=e3591aaed06cbd47 in_seq=0 out_seq=0 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 10 client.10245 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 1 -- 172.21.15.45:0/2085380940 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582d2a53c70 con 0x5582d2ca2e60 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 10 client.10245 handle_command_reply: tid=0 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 client.10245 shutdown 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 msgr2=0x5582d2ca2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 0x5582d2ca2450 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f65d00b0040 tx=0x7f65d001b180 comp rx=0 tx=0).stop 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 shutdown_connections 2024-04-25T10:30:05.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2ca2e60 0x5582d2cc6440 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2cb4aa0 0x5582d2ca2450 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2cb4180 0x5582d2c6fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2085380940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2c6a600 0x5582d2c6f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65bbe00640 0 client.10245 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 >> 172.21.15.45:0/2085380940 conn(0x5582d2c70760 msgr2=0x5582d2c690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 shutdown_connections 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 1 -- 172.21.15.45:0/2085380940 wait complete. 2024-04-25T10:30:05.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.635+0000 7f65e0e69000 20 client.10245 trim_cache size 0 max 16384 2024-04-25T10:30:05.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.647+0000 7f65e0e69000 1 Processor -- start 2024-04-25T10:30:05.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.647+0000 7f65e0e69000 1 -- start start 2024-04-25T10:30:05.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 0x5582d2cc8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2c69660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8202a70 con 0x5582d2c70760 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8201c40 con 0x5582d2c6a600 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8201090 con 0x5582d2cb4180 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 0x5582d2cc8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 0x5582d2cc8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39124/0 (socket says 172.21.15.45:39124) 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 -- 172.21.15.45:0/2770808271 learned_addr learned my addr 172.21.15.45:0/2770808271 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d7400640 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2c69660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 -- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 msgr2=0x5582d2c69660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2c69660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 -- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 msgr2=0x5582d2c69120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c69120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 -- 172.21.15.45:0/2770808271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582d2a40250 con 0x5582d2c70760 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 0x5582d2cc8680 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f65c8014000 tx=0x7f65c800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2baaeed317c9c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65c6a00640 1 -- 172.21.15.45:0/2770808271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8015c00 con 0x5582d2c70760 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65c6a00640 1 -- 172.21.15.45:0/2770808271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8015da0 con 0x5582d2c70760 2024-04-25T10:30:05.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65c6a00640 1 -- 172.21.15.45:0/2770808271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c802e260 con 0x5582d2c70760 2024-04-25T10:30:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- 172.21.15.45:0/2770808271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 msgr2=0x5582d2cc8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 0x5582d2cc8680 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f65c8014000 tx=0x7f65c800a000 comp rx=0 tx=0).stop 2024-04-25T10:30:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- 172.21.15.45:0/2770808271 shutdown_connections 2024-04-25T10:30:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2c69660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c6a600 0x5582d2c69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2770808271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c70760 0x5582d2cc8680 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- 172.21.15.45:0/2770808271 >> 172.21.15.45:0/2770808271 conn(0x5582d2ca25b0 msgr2=0x5582d2cb3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- 172.21.15.45:0/2770808271 shutdown_connections 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- 172.21.15.45:0/2770808271 wait complete. 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- start start 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 Processor -- start 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 0x5582d2d146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c70760 0x5582d2d14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2cc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d2a632f0 con 0x5582d2c6a600 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8201640 con 0x5582d2c70760 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582d2ab4f90 con 0x5582d2cb4180 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.651+0000 7f65e0e69000 10 client.0 _ll_register_callbacks cb 0x5582d2c36600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:05.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 0x5582d2d146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 0x5582d2d146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39130/0 (socket says 172.21.15.45:39130) 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d7400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2cc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 -- 172.21.15.45:0/2317176735 learned_addr learned my addr 172.21.15.45:0/2317176735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c70760 0x5582d2d14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 -- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 msgr2=0x5582d2cc6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2cc6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 -- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c70760 msgr2=0x5582d2d14c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c70760 0x5582d2d14c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 -- 172.21.15.45:0/2317176735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582d2a4cb90 con 0x5582d2c6a600 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65d6a00640 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 0x5582d2d146e0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x5582d2abdf00 tx=0x7f65c8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779a4ec708ec5d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 1 -- 172.21.15.45:0/2317176735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c803d060 con 0x5582d2c6a600 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 1 -- 172.21.15.45:0/2317176735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f65c802e020 con 0x5582d2c6a600 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65c8015da0 con 0x5582d2c6a600 2024-04-25T10:30:05.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 1 -- 172.21.15.45:0/2317176735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c8043050 con 0x5582d2c6a600 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 10 client.10251 fetch_fsmap learned FSMap version 115 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) v3 -- 0x5582d2a5f010 con 0x5582d2c6a600 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 1 -- 172.21.15.45:0/2317176735 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8015da0 con 0x5582d2c6a600 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 1 -- 172.21.15.45:0/2317176735 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 115) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65c8046020 con 0x5582d2c6a600 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 10 client.10251 fetch_fsmap finished waiting for FSMap version 115 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 10 client.10251 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.a 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 20 client.10251 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2cb4aa0 0x5582d2cc76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 4 client.10251 mds_command: new command op to 9996 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 --> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582d2ac5460 con 0x5582d2cb4aa0 2024-04-25T10:30:05.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2cb4aa0 0x5582d2cc76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65cfe00640 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2cb4aa0 0x5582d2cc76b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed5d5e57644f82e server_cookie=d6eda2fabb1afa82 in_seq=0 out_seq=0 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 10 client.10251 ms_handle_connect on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 1 -- 172.21.15.45:0/2317176735 <== mds.0 v2:172.21.15.45:6833/2482982951 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582d2ac5460 con 0x5582d2cb4aa0 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 10 client.10251 handle_command_reply: tid=0 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 client.10251 shutdown 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 msgr2=0x5582d2d146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 0x5582d2d146e0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x5582d2abdf00 tx=0x7f65c8000ee0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 shutdown_connections 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:6833/2482982951,v1:172.21.15.45:6835/2482982951] conn(0x5582d2cb4aa0 0x5582d2cc76b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5582d2cb4180 0x5582d2cc6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582d2c70760 0x5582d2d14c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 --2- 172.21.15.45:0/2317176735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582d2c6a600 0x5582d2d146e0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65bbe00640 0 client.10251 ms_handle_reset on v2:172.21.15.45:6833/2482982951 2024-04-25T10:30:05.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 >> 172.21.15.45:0/2317176735 conn(0x5582d2ca25b0 msgr2=0x5582d2cc1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 shutdown_connections 2024-04-25T10:30:05.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 1 -- 172.21.15.45:0/2317176735 wait complete. 2024-04-25T10:30:05.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65e0e69000 20 client.10251 trim_cache size 0 max 16384 2024-04-25T10:30:05.663 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b4066f60 msgr2=0x7f65b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65dda00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b4066f60 0x7f65b4069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000f70 tx=0x7f65c8007b60 comp rx=0 tx=0).stop 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 msgr2=0x7f65d81451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65dda00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 0x7f65d81451e0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f65c000e930 tx=0x7f65c000edf0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.655+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 shutdown_connections 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d8159540 0x7f65d80b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d80aeda0 0x7f65d81451e0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 --2- 172.21.15.45:0/2492148937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d8072020 0x7f65d81faaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 >> 172.21.15.45:0/2492148937 conn(0x7f65d80669a0 msgr2=0x7f65d815fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 shutdown_connections 2024-04-25T10:30:05.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.659+0000 7f65dda00640 1 -- 172.21.15.45:0/2492148937 wait complete. 2024-04-25T10:30:05.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:30:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.815+0000 7f9f83800640 1 Processor -- start 2024-04-25T10:30:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.815+0000 7f9f83800640 1 -- start start 2024-04-25T10:30:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f83800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9f7c15b9e0 0x7f9f7c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f83800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f7c15bdc0 0x7f9f7c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f83800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7c0ccfb0 con 0x7f9f7c0d1520 2024-04-25T10:30:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f83800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c0cd290 con 0x7f9f7c15bdc0 2024-04-25T10:30:05.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f83800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c0cd570 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f7c15bdc0 0x7f9f7c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f7c15bdc0 0x7f9f7c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39140/0 (socket says 172.21.15.45:39140) 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 -- 172.21.15.45:0/2950682359 learned_addr learned my addr 172.21.15.45:0/2950682359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f80e00640 1 --2- 172.21.15.45:0/2950682359 >> v2:172.21.15.117:3300/0 conn(0x7f9f7c15b9e0 0x7f9f7c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 -- 172.21.15.45:0/2950682359 >> v2:172.21.15.117:3300/0 conn(0x7f9f7c15b9e0 msgr2=0x7f9f7c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 --2- 172.21.15.45:0/2950682359 >> v2:172.21.15.117:3300/0 conn(0x7f9f7c15b9e0 0x7f9f7c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 -- 172.21.15.45:0/2950682359 >> v1:172.21.15.117:6789/0 conn(0x7f9f7c0d1520 legacy=0x7f9f7c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f7be00640 1 -- 172.21.15.45:0/2950682359 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0cd850 con 0x7f9f7c15bdc0 2024-04-25T10:30:05.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.819+0000 7f9f80e00640 1 --2- 172.21.15.45:0/2950682359 >> v2:172.21.15.117:3300/0 conn(0x7f9f7c15b9e0 0x7f9f7c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f7be00640 1 --2- 172.21.15.45:0/2950682359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15bdc0 0x7f9f7c0cc950 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c070da0 tx=0x7f9f6801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa45c3058afa02f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f7b400640 1 -- 172.21.15.45:0/2950682359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680b0070 con 0x7f9f7c15bdc0 2024-04-25T10:30:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f7b400640 1 -- 172.21.15.45:0/2950682359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f680ab600 con 0x7f9f7c15bdc0 2024-04-25T10:30:05.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f7b400640 1 -- 172.21.15.45:0/2950682359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680ab9d0 con 0x7f9f7c15bdc0 2024-04-25T10:30:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- 172.21.15.45:0/2950682359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15bdc0 msgr2=0x7f9f7c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 --2- 172.21.15.45:0/2950682359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15bdc0 0x7f9f7c0cc950 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c070da0 tx=0x7f9f6801c490 comp rx=0 tx=0).stop 2024-04-25T10:30:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- 172.21.15.45:0/2950682359 shutdown_connections 2024-04-25T10:30:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 --2- 172.21.15.45:0/2950682359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15bdc0 0x7f9f7c0cc950 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 --2- 172.21.15.45:0/2950682359 >> v2:172.21.15.117:3300/0 conn(0x7f9f7c15b9e0 0x7f9f7c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- 172.21.15.45:0/2950682359 >> 172.21.15.45:0/2950682359 conn(0x7f9f7c09a9d0 msgr2=0x7f9f7c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- 172.21.15.45:0/2950682359 shutdown_connections 2024-04-25T10:30:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- 172.21.15.45:0/2950682359 wait complete. 2024-04-25T10:30:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 Processor -- start 2024-04-25T10:30:05.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- start start 2024-04-25T10:30:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c0d1520 0x7f9f7c155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 0x7f9f7c1f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c15bdc0 0x7f9f7c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0c6630 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0c64b0 con 0x7f9f7c0d1520 2024-04-25T10:30:05.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.823+0000 7f9f83800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0c67b0 con 0x7f9f7c15bdc0 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 0x7f9f7c1f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 0x7f9f7c1f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39154/0 (socket says 172.21.15.45:39154) 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 -- 172.21.15.45:0/597340722 learned_addr learned my addr 172.21.15.45:0/597340722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f80e00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c0d1520 0x7f9f7c155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f81800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c15bdc0 0x7f9f7c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 -- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c15bdc0 msgr2=0x7f9f7c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c15bdc0 0x7f9f7c155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 -- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c0d1520 msgr2=0x7f9f7c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c0d1520 0x7f9f7c155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 -- 172.21.15.45:0/597340722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0b6090 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f7be00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 0x7f9f7c1f6680 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f9f680190a0 tx=0x7f9f68019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680b9070 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f680b4420 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680b0040 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c0cb8c0 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c0c9be0 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7c04ef30 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.827+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f680bc070 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.831+0000 7f9f78c00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f5c066f90 0x7f9f5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:05.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.831+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9f6811dfa0 con 0x7f9f7c15b9e0 2024-04-25T10:30:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.831+0000 7f9f80e00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f5c066f90 0x7f9f5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:05.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.831+0000 7f9f80e00640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f5c066f90 0x7f9f5c069450 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0c8280 tx=0x7f9f70009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:05.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:05.835+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f68016110 con 0x7f9f7c15b9e0 2024-04-25T10:30:06.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9f7c0c7fd0 con 0x7f9f7c15b9e0 2024-04-25T10:30:06.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f78c00640 1 -- 172.21.15.45:0/597340722 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9f680ee8a0 con 0x7f9f7c15b9e0 2024-04-25T10:30:06.033 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:06.033 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f5c066f90 msgr2=0x7f9f5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f5c066f90 0x7f9f5c069450 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0c8280 tx=0x7f9f70009190 comp rx=0 tx=0).stop 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 msgr2=0x7f9f7c1f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 0x7f9f7c1f6680 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f9f680190a0 tx=0x7f9f68019230 comp rx=0 tx=0).stop 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 shutdown_connections 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9f5c066f90 0x7f9f5c069450 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c15bdc0 0x7f9f7c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f7c15b9e0 0x7f9f7c1f6680 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 --2- 172.21.15.45:0/597340722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c0d1520 0x7f9f7c155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 >> 172.21.15.45:0/597340722 conn(0x7f9f7c09a9d0 msgr2=0x7f9f7c0c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 shutdown_connections 2024-04-25T10:30:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.031+0000 7f9f83800640 1 -- 172.21.15.45:0/597340722 wait complete. 2024-04-25T10:30:06.055 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:30:06.055 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:30:06.102 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:30:06.157 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:30:06.158 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:06.158 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:06.171 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:06.171 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:06.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:06.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.407+0000 7f3208200640 1 Processor -- start 2024-04-25T10:30:06.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.407+0000 7f3208200640 1 -- start start 2024-04-25T10:30:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3208200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f32000d0040 0x7f320015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3208200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3200071050 con 0x7f32001550b0 2024-04-25T10:30:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3208200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320015d040 con 0x7f320015b740 2024-04-25T10:30:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3208200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32000712f0 con 0x7f32000d0040 2024-04-25T10:30:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3205800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32001550b0 0x7f32000cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56328/0 (socket says 172.21.15.45:56328) 2024-04-25T10:30:06.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3205800640 1 -- 172.21.15.45:0/2934016025 learned_addr learned my addr 172.21.15.45:0/2934016025 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766122425 0 0) 0x7f3200071050 con 0x7f32001550b0 2024-04-25T10:30:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3204e00640 1 --2- 172.21.15.45:0/2934016025 >> v2:172.21.15.117:3300/0 conn(0x7f32000d0040 0x7f320015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e8005740 con 0x7f32001550b0 2024-04-25T10:30:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2784663897 0 0) 0x7f31e8005740 con 0x7f32001550b0 2024-04-25T10:30:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 >> v2:172.21.15.117:3300/0 conn(0x7f32000d0040 msgr2=0x7f320015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:06.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 --2- 172.21.15.45:0/2934016025 >> v2:172.21.15.117:3300/0 conn(0x7f32000d0040 0x7f320015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 >> v1:172.21.15.117:6789/0 conn(0x7f320015b740 legacy=0x7f32000b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000715d0 con 0x7f32001550b0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f31f4001680 con 0x7f32001550b0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 >> v1:172.21.15.45:6789/0 conn(0x7f32001550b0 legacy=0x7f32000cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e8005bd0 0x7f31e8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e8006a20 0x7f31e8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31e80073e0 0x7f31e800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32000712f0 con 0x7f31e80073e0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e800e230 con 0x7f31e8006a20 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e800e430 con 0x7f31e8005bd0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3206200640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31e80073e0 0x7f31e800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3204e00640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e8006a20 0x7f31e8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3205800640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e8005bd0 0x7f31e8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3206200640 1 -- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e8005bd0 msgr2=0x7f31e8005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3206200640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e8005bd0 0x7f31e8005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3206200640 1 -- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e8006a20 msgr2=0x7f31e8006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3206200640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31e8006a20 0x7f31e8006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.411+0000 7f3206200640 1 -- 172.21.15.45:0/2934016025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8005740 con 0x7f31e80073e0 2024-04-25T10:30:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3206200640 1 -- 172.21.15.45:0/2934016025 reap_dead start 2024-04-25T10:30:06.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3206200640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31e80073e0 0x7f31e800dbe0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f31f8001f60 tx=0x7f31f801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f2339d406ef5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8001b60 con 0x7f31e80073e0 2024-04-25T10:30:06.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f80a6670 con 0x7f31e80073e0 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f31ffe00640 1 -- 172.21.15.45:0/2934016025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f80a6a40 con 0x7f31e80073e0 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 -- 172.21.15.45:0/2934016025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31e80073e0 msgr2=0x7f31e800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31e80073e0 0x7f31e800dbe0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f31f8001f60 tx=0x7f31f801c880 comp rx=0 tx=0).stop 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 -- 172.21.15.45:0/2934016025 shutdown_connections 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 --2- 172.21.15.45:0/2934016025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31e80073e0 0x7f31e800dbe0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 -- 172.21.15.45:0/2934016025 >> 172.21.15.45:0/2934016025 conn(0x7f3200066b70 msgr2=0x7f3200066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 -- 172.21.15.45:0/2934016025 shutdown_connections 2024-04-25T10:30:06.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.415+0000 7f3208200640 1 -- 172.21.15.45:0/2934016025 wait complete. 2024-04-25T10:30:06.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 Processor -- start 2024-04-25T10:30:06.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- start start 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80073e0 0x7f32000ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32001534e0 0x7f32000af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 0x7f32001f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32000720b0 con 0x7f32000afea0 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200071b90 con 0x7f32001534e0 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320015bb90 con 0x7f31e80073e0 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 0x7f32001f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 0x7f32001f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39166/0 (socket says 172.21.15.45:39166) 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 -- 172.21.15.45:0/1510846453 learned_addr learned my addr 172.21.15.45:0/1510846453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3204e00640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32001534e0 0x7f32000af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3205800640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80073e0 0x7f32000ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 -- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80073e0 msgr2=0x7f32000ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80073e0 0x7f32000ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 -- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32001534e0 msgr2=0x7f32000af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32001534e0 0x7f32000af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 -- 172.21.15.45:0/1510846453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200159d80 con 0x7f32000afea0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3206200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 0x7f32001f23d0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007fd0 tx=0x7f31f8019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8001b40 con 0x7f32000afea0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f80b6070 con 0x7f32000afea0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320015a060 con 0x7f32000afea0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8003d90 con 0x7f32000afea0 2024-04-25T10:30:06.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320015a620 con 0x7f32000afea0 2024-04-25T10:30:06.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.419+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320004ef30 con 0x7f32000afea0 2024-04-25T10:30:06.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.423+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31f8005070 con 0x7f32000afea0 2024-04-25T10:30:06.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.423+0000 7f31fd600640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31e0066f90 0x7f31e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.423+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f31f811eb70 con 0x7f32000afea0 2024-04-25T10:30:06.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.423+0000 7f3205800640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31e0066f90 0x7f31e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.423+0000 7f3205800640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31e0066f90 0x7f31e0069450 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f31f4001060 tx=0x7f31f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.427+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f31f8016590 con 0x7f32000afea0 2024-04-25T10:30:06.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f320015ad10 con 0x7f32000afea0 2024-04-25T10:30:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f31fd600640 1 -- 172.21.15.45:0/1510846453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16717 (secure 0 0 0) 0x7f31f80efac0 con 0x7f32000afea0 2024-04-25T10:30:06.593 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:06.593 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":157,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:44.707588+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31e0066f90 msgr2=0x7f31e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31e0066f90 0x7f31e0069450 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f31f4001060 tx=0x7f31f4009040 comp rx=0 tx=0).stop 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 msgr2=0x7f32001f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 0x7f32001f23d0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007fd0 tx=0x7f31f8019620 comp rx=0 tx=0).stop 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 shutdown_connections 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f31e0066f90 0x7f31e0069450 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32000afea0 0x7f32001f23d0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32001534e0 0x7f32000af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 --2- 172.21.15.45:0/1510846453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31e80073e0 0x7f32000ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 >> 172.21.15.45:0/1510846453 conn(0x7f3200066b70 msgr2=0x7f320006d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 shutdown_connections 2024-04-25T10:30:06.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.591+0000 7f3208200640 1 -- 172.21.15.45:0/1510846453 wait complete. 2024-04-25T10:30:06.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:24.390187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:25.607844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:06.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 Processor -- start 2024-04-25T10:30:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 -- start start 2024-04-25T10:30:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c070350 0x7fec5c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec5c0b31c0 0x7fec5c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec5c15b080 con 0x7fec5c0aef80 2024-04-25T10:30:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5c15adb0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec61800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c15eb60 con 0x7fec5c070350 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec5c0b31c0 0x7fec5c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c070350 0x7fec5c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fec5c0b31c0 0x7fec5c0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44228/0 (socket says 172.21.15.45:44228) 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 -- 172.21.15.45:0/1467890833 learned_addr learned my addr 172.21.15.45:0/1467890833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 -- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c070350 msgr2=0x7fec5c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 --2- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c070350 0x7fec5c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 -- 172.21.15.45:0/1467890833 >> v1:172.21.15.117:6789/0 conn(0x7fec5c0aef80 legacy=0x7fec5c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.759+0000 7fec5b400640 1 -- 172.21.15.45:0/1467890833 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c15ee40 con 0x7fec5c0b31c0 2024-04-25T10:30:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec5b400640 1 --2- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0b31c0 0x7fec5c0b3610 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fec54009e50 tx=0x7fec5401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0accc4d7ea6a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec59600640 1 -- 172.21.15.45:0/1467890833 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540a5810 con 0x7fec5c0b31c0 2024-04-25T10:30:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec59600640 1 -- 172.21.15.45:0/1467890833 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec540a59b0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec59600640 1 -- 172.21.15.45:0/1467890833 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540a5d80 con 0x7fec5c0b31c0 2024-04-25T10:30:06.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 -- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0b31c0 msgr2=0x7fec5c0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 --2- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0b31c0 0x7fec5c0b3610 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fec54009e50 tx=0x7fec5401ca30 comp rx=0 tx=0).stop 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 -- 172.21.15.45:0/1467890833 shutdown_connections 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 --2- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c0b31c0 0x7fec5c0b3610 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 --2- 172.21.15.45:0/1467890833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c070350 0x7fec5c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 -- 172.21.15.45:0/1467890833 >> 172.21.15.45:0/1467890833 conn(0x7fec5c0669a0 msgr2=0x7fec5c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 -- 172.21.15.45:0/1467890833 shutdown_connections 2024-04-25T10:30:06.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.763+0000 7fec61800640 1 -- 172.21.15.45:0/1467890833 wait complete. 2024-04-25T10:30:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 Processor -- start 2024-04-25T10:30:06.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 -- start start 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c070350 0x7fec5c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c0aef80 0x7fec5c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 0x7fec5c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c1679a0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c167820 con 0x7fec5c070350 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c167b20 con 0x7fec5c0aef80 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 0x7fec5c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 0x7fec5c155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39174/0 (socket says 172.21.15.45:39174) 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 -- 172.21.15.45:0/654746437 learned_addr learned my addr 172.21.15.45:0/654746437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:06.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5aa00640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c070350 0x7fec5c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5a000640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c0aef80 0x7fec5c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 -- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c0aef80 msgr2=0x7fec5c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c0aef80 0x7fec5c1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 -- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c070350 msgr2=0x7fec5c06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c070350 0x7fec5c06d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 -- 172.21.15.45:0/654746437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c0af850 con 0x7fec5c0b31c0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec5b400640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 0x7fec5c155e00 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fec540af040 tx=0x7fec540197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540b8670 con 0x7fec5c0b31c0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec540be070 con 0x7fec5c0b31c0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c0afb30 con 0x7fec5c0b31c0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540b8a70 con 0x7fec5c0b31c0 2024-04-25T10:30:06.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.767+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c0b00f0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.771+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5c0b04d0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.771+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec540b8c10 con 0x7fec5c0b31c0 2024-04-25T10:30:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.775+0000 7fec52a00640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec3c066f60 0x7fec3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.775+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fec54120a40 con 0x7fec5c0b31c0 2024-04-25T10:30:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.775+0000 7fec5aa00640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec3c066f60 0x7fec3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.775+0000 7fec5aa00640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec3c066f60 0x7fec3c069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0718e0 tx=0x7fec4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:06.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.775+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec5c0b04d0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec5c0b10c0 con 0x7fec5c0b31c0 2024-04-25T10:30:06.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec52a00640 1 -- 172.21.15.45:0/654746437 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec54016620 con 0x7fec5c0b31c0 2024-04-25T10:30:06.940 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec3c066f60 msgr2=0x7fec3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec3c066f60 0x7fec3c069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0718e0 tx=0x7fec4c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 msgr2=0x7fec5c155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 0x7fec5c155e00 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fec540af040 tx=0x7fec540197d0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 shutdown_connections 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fec3c066f60 0x7fec3c069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0b31c0 0x7fec5c155e00 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fec5c0aef80 0x7fec5c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 --2- 172.21.15.45:0/654746437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec5c070350 0x7fec5c06d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 >> 172.21.15.45:0/654746437 conn(0x7fec5c0669a0 msgr2=0x7fec5c0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 shutdown_connections 2024-04-25T10:30:06.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:06.939+0000 7fec61800640 1 -- 172.21.15.45:0/654746437 wait complete. 2024-04-25T10:30:06.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.103+0000 7fdfcb600640 1 Processor -- start 2024-04-25T10:30:07.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.103+0000 7fdfcb600640 1 -- start start 2024-04-25T10:30:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfcb600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdfc4159540 0x7fdfc40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc40721f0 0x7fdfc4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfcb600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc4159920 con 0x7fdfc40aebd0 2024-04-25T10:30:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfcb600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc415c2f0 con 0x7fdfc4159540 2024-04-25T10:30:07.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfcb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc415c5d0 con 0x7fdfc40721f0 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdfc4159540 0x7fdfc40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc3e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fdfc40aebd0 0x7fdfc4071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:35230/0 (socket says 172.21.15.45:35230) 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc40721f0 0x7fdfc4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdfc4159540 0x7fdfc40ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44248/0 (socket says 172.21.15.45:44248) 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc3e00640 1 -- 172.21.15.45:0/3578619840 learned_addr learned my addr 172.21.15.45:0/3578619840 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc8c00640 1 -- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc40721f0 msgr2=0x7fdfc4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc8c00640 1 --2- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc40721f0 0x7fdfc4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc8c00640 1 -- 172.21.15.45:0/3578619840 >> v1:172.21.15.117:6789/0 conn(0x7fdfc40aebd0 legacy=0x7fdfc4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.107+0000 7fdfc8c00640 1 -- 172.21.15.45:0/3578619840 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc415c8b0 con 0x7fdfc4159540 2024-04-25T10:30:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfc8c00640 1 --2- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc4159540 0x7fdfc40ae690 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097c0 tx=0x7fdfb801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c710b5df97e94dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfc3400640 1 -- 172.21.15.45:0/3578619840 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80b1070 con 0x7fdfc4159540 2024-04-25T10:30:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfc3400640 1 -- 172.21.15.45:0/3578619840 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80ac410 con 0x7fdfc4159540 2024-04-25T10:30:07.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfc3400640 1 -- 172.21.15.45:0/3578619840 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80ac7e0 con 0x7fdfc4159540 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 -- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc4159540 msgr2=0x7fdfc40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 --2- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc4159540 0x7fdfc40ae690 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097c0 tx=0x7fdfb801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 -- 172.21.15.45:0/3578619840 shutdown_connections 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 --2- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc40721f0 0x7fdfc4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 --2- 172.21.15.45:0/3578619840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc4159540 0x7fdfc40ae690 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 -- 172.21.15.45:0/3578619840 >> 172.21.15.45:0/3578619840 conn(0x7fdfc40669a0 msgr2=0x7fdfc4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 -- 172.21.15.45:0/3578619840 shutdown_connections 2024-04-25T10:30:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 -- 172.21.15.45:0/3578619840 wait complete. 2024-04-25T10:30:07.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 Processor -- start 2024-04-25T10:30:07.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.111+0000 7fdfcb600640 1 -- start start 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 0x7fdfc413f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc40aebd0 0x7fdfc4150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc4159540 0x7fdfc41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4071220 con 0x7fdfc40721f0 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40710a0 con 0x7fdfc40aebd0 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40713a0 con 0x7fdfc4159540 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 0x7fdfc413f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 0x7fdfc413f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39176/0 (socket says 172.21.15.45:39176) 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 -- 172.21.15.45:0/96077179 learned_addr learned my addr 172.21.15.45:0/96077179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc3e00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc40aebd0 0x7fdfc4150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc9600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc4159540 0x7fdfc41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 -- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc4159540 msgr2=0x7fdfc41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc4159540 0x7fdfc41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 -- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc40aebd0 msgr2=0x7fdfc4150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc40aebd0 0x7fdfc4150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 -- 172.21.15.45:0/96077179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4150bc0 con 0x7fdfc40721f0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc8c00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 0x7fdfc413f3a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097c0 tx=0x7fdfb8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80b1040 con 0x7fdfc40721f0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb80bb070 con 0x7fdfc40721f0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80b5410 con 0x7fdfc40721f0 2024-04-25T10:30:07.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc4150ea0 con 0x7fdfc40721f0 2024-04-25T10:30:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc4151460 con 0x7fdfc40721f0 2024-04-25T10:30:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc404ef30 con 0x7fdfc40721f0 2024-04-25T10:30:07.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.115+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfb80b55b0 con 0x7fdfc40721f0 2024-04-25T10:30:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.119+0000 7fdfc0c00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfa4066f60 0x7fdfa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.119+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdfb811eca0 con 0x7fdfc40721f0 2024-04-25T10:30:07.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.119+0000 7fdfc3e00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfa4066f60 0x7fdfa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.119+0000 7fdfc3e00640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfa4066f60 0x7fdfa4069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdfc415b4b0 tx=0x7fdfb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.123+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfb8016090 con 0x7fdfc40721f0 2024-04-25T10:30:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfc40cfc30 con 0x7fdfc40721f0 2024-04-25T10:30:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfc0c00640 1 -- 172.21.15.45:0/96077179 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfb80efb60 con 0x7fdfc40721f0 2024-04-25T10:30:07.288 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:30:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfa4066f60 msgr2=0x7fdfa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfa4066f60 0x7fdfa4069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdfc415b4b0 tx=0x7fdfb0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 msgr2=0x7fdfc413f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 0x7fdfc413f3a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097c0 tx=0x7fdfb8019240 comp rx=0 tx=0).stop 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 shutdown_connections 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfa4066f60 0x7fdfa4069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfc4159540 0x7fdfc41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfc40aebd0 0x7fdfc4150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 --2- 172.21.15.45:0/96077179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc40721f0 0x7fdfc413f3a0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 >> 172.21.15.45:0/96077179 conn(0x7fdfc40669a0 msgr2=0x7fdfc415e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 shutdown_connections 2024-04-25T10:30:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.287+0000 7fdfcb600640 1 -- 172.21.15.45:0/96077179 wait complete. 2024-04-25T10:30:07.311 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.451+0000 7f95d6c00640 1 Processor -- start 2024-04-25T10:30:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.451+0000 7f95d6c00640 1 -- start start 2024-04-25T10:30:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d6c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95d00cdb60 0x7f95d006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d6c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d00cdf40 con 0x7f95d006bcb0 2024-04-25T10:30:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d6c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d00b1230 con 0x7f95d00cbdf0 2024-04-25T10:30:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d6c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d006b3e0 con 0x7f95d00cdb60 2024-04-25T10:30:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d5800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f95d00cdb60 0x7f95d006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d6200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f95d006bcb0 0x7f95d00b22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56342/0 (socket says 172.21.15.45:56342) 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d6200640 1 -- 172.21.15.45:0/4274988217 learned_addr learned my addr 172.21.15.45:0/4274988217 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95cfe00640 1 -- 172.21.15.45:0/4274988217 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803572769 0 0) 0x7f95d00cdf40 con 0x7f95d006bcb0 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95cfe00640 1 -- 172.21.15.45:0/4274988217 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b8003530 con 0x7f95d006bcb0 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d5800640 1 -- 172.21.15.45:0/4274988217 >> v1:172.21.15.117:6789/0 conn(0x7f95d00cbdf0 legacy=0x7f95d006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d5800640 1 -- 172.21.15.45:0/4274988217 >> v1:172.21.15.45:6789/0 conn(0x7f95d006bcb0 legacy=0x7f95d00b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.455+0000 7f95d5800640 1 -- 172.21.15.45:0/4274988217 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00b1510 con 0x7f95d00cdb60 2024-04-25T10:30:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d5800640 1 --2- 172.21.15.45:0/4274988217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d006aea0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097c0 tx=0x7f95c401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608ffa00968e30f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95cfe00640 1 -- 172.21.15.45:0/4274988217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40a7070 con 0x7f95d00cdb60 2024-04-25T10:30:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95cfe00640 1 -- 172.21.15.45:0/4274988217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c40b0070 con 0x7f95d00cdb60 2024-04-25T10:30:07.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95cfe00640 1 -- 172.21.15.45:0/4274988217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40ab890 con 0x7f95d00cdb60 2024-04-25T10:30:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- 172.21.15.45:0/4274988217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 msgr2=0x7f95d006aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 --2- 172.21.15.45:0/4274988217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d006aea0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097c0 tx=0x7f95c401c470 comp rx=0 tx=0).stop 2024-04-25T10:30:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- 172.21.15.45:0/4274988217 shutdown_connections 2024-04-25T10:30:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 --2- 172.21.15.45:0/4274988217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d006aea0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- 172.21.15.45:0/4274988217 >> 172.21.15.45:0/4274988217 conn(0x7f95d00d17a0 msgr2=0x7f95d00d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- 172.21.15.45:0/4274988217 shutdown_connections 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- 172.21.15.45:0/4274988217 wait complete. 2024-04-25T10:30:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 Processor -- start 2024-04-25T10:30:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- start start 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d006bcb0 0x7f95d00cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00cbdf0 0x7f95d00cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d00c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d015d590 con 0x7f95d00cdb60 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d015d410 con 0x7f95d006bcb0 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d015d710 con 0x7f95d00cbdf0 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d00c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d00c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39184/0 (socket says 172.21.15.45:39184) 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.459+0000 7f95d6200640 1 -- 172.21.15.45:0/168635653 learned_addr learned my addr 172.21.15.45:0/168635653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d5800640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d006bcb0 0x7f95d00cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d4e00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00cbdf0 0x7f95d00cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6200640 1 -- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00cbdf0 msgr2=0x7f95d00cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6200640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00cbdf0 0x7f95d00cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6200640 1 -- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d006bcb0 msgr2=0x7f95d00cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6200640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d006bcb0 0x7f95d00cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6200640 1 -- 172.21.15.45:0/168635653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00b5fc0 con 0x7f95d00cdb60 2024-04-25T10:30:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6200640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d00c8970 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f95c800cb70 tx=0x7f95c800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8017070 con 0x7f95d00cdb60 2024-04-25T10:30:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800ad40 con 0x7f95d00cdb60 2024-04-25T10:30:07.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8012620 con 0x7f95d00cdb60 2024-04-25T10:30:07.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d00b62a0 con 0x7f95d00cdb60 2024-04-25T10:30:07.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00b6860 con 0x7f95d00cdb60 2024-04-25T10:30:07.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d0075120 con 0x7f95d00cdb60 2024-04-25T10:30:07.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95c801a070 con 0x7f95d00cdb60 2024-04-25T10:30:07.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95cd600640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95b0066f90 0x7f95b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.463+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f95c8084e50 con 0x7f95d00cdb60 2024-04-25T10:30:07.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.467+0000 7f95d5800640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95b0066f90 0x7f95b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.467+0000 7f95d5800640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95b0066f90 0x7f95b0069450 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f95c40af000 tx=0x7f95c40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.467+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95c8055750 con 0x7f95d00cdb60 2024-04-25T10:30:07.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95d00b7620 con 0x7f95d00cdb60 2024-04-25T10:30:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95cd600640 1 -- 172.21.15.45:0/168635653 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95c80597f0 con 0x7f95d00cdb60 2024-04-25T10:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95b0066f90 msgr2=0x7f95b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95b0066f90 0x7f95b0069450 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f95c40af000 tx=0x7f95c40a7000 comp rx=0 tx=0).stop 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 msgr2=0x7f95d00c8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d00c8970 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f95c800cb70 tx=0x7f95c800a430 comp rx=0 tx=0).stop 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 shutdown_connections 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f95b0066f90 0x7f95b0069450 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95d00cdb60 0x7f95d00c8970 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f95d00cbdf0 0x7f95d00cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 --2- 172.21.15.45:0/168635653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95d006bcb0 0x7f95d00cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 >> 172.21.15.45:0/168635653 conn(0x7f95d00d17a0 msgr2=0x7f95d00d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 shutdown_connections 2024-04-25T10:30:07.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.631+0000 7f95d6c00640 1 -- 172.21.15.45:0/168635653 wait complete. 2024-04-25T10:30:07.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:30:07.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 Processor -- start 2024-04-25T10:30:07.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 -- start start 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ddc15bc40 0x7f4ddc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ddc0cf930 0x7f4ddc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc0b2770 con 0x7f4ddc0cda10 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc06c670 con 0x7f4ddc15bc40 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de4000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc06c950 con 0x7f4ddc0cf930 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de1600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4ddc15bc40 0x7f4ddc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de0c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4ddc0cda10 0x7f4ddc0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56350/0 (socket says 172.21.15.45:56350) 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de0c00640 1 -- 172.21.15.45:0/985962489 learned_addr learned my addr 172.21.15.45:0/985962489 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:07.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de2000640 1 --2- 172.21.15.45:0/985962489 >> v2:172.21.15.117:3300/0 conn(0x7f4ddc0cf930 0x7f4ddc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de1600640 1 -- 172.21.15.45:0/985962489 >> v2:172.21.15.117:3300/0 conn(0x7f4ddc0cf930 msgr2=0x7f4ddc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de1600640 1 --2- 172.21.15.45:0/985962489 >> v2:172.21.15.117:3300/0 conn(0x7f4ddc0cf930 0x7f4ddc0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de1600640 1 -- 172.21.15.45:0/985962489 >> v1:172.21.15.45:6789/0 conn(0x7f4ddc0cda10 legacy=0x7f4ddc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4ddbe00640 1 -- 172.21.15.45:0/985962489 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726445923 0 0) 0x7f4ddc0b2770 con 0x7f4ddc0cda10 2024-04-25T10:30:07.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.795+0000 7f4de1600640 1 -- 172.21.15.45:0/985962489 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc06cc30 con 0x7f4ddc15bc40 2024-04-25T10:30:07.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de1600640 1 --2- 172.21.15.45:0/985962489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc15bc40 0x7f4ddc06c090 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0ae4c0 tx=0x7f4dd001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5472fa873f711ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4ddbe00640 1 -- 172.21.15.45:0/985962489 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00b0070 con 0x7f4ddc15bc40 2024-04-25T10:30:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4ddbe00640 1 -- 172.21.15.45:0/985962489 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd00ab600 con 0x7f4ddc15bc40 2024-04-25T10:30:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4ddbe00640 1 -- 172.21.15.45:0/985962489 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00ab9d0 con 0x7f4ddc15bc40 2024-04-25T10:30:07.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 -- 172.21.15.45:0/985962489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc15bc40 msgr2=0x7f4ddc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 --2- 172.21.15.45:0/985962489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc15bc40 0x7f4ddc06c090 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0ae4c0 tx=0x7f4dd001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 -- 172.21.15.45:0/985962489 shutdown_connections 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 --2- 172.21.15.45:0/985962489 >> v2:172.21.15.117:3300/0 conn(0x7f4ddc0cf930 0x7f4ddc0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 --2- 172.21.15.45:0/985962489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc15bc40 0x7f4ddc06c090 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 -- 172.21.15.45:0/985962489 >> 172.21.15.45:0/985962489 conn(0x7f4ddc066b70 msgr2=0x7f4ddc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 -- 172.21.15.45:0/985962489 shutdown_connections 2024-04-25T10:30:07.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 -- 172.21.15.45:0/985962489 wait complete. 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.799+0000 7f4de4000640 1 Processor -- start 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- start start 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 0x7f4ddc1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc0cf930 0x7f4ddc14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc15bc40 0x7f4ddc152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b3850 con 0x7f4ddc0cda10 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b36d0 con 0x7f4ddc0cf930 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b39d0 con 0x7f4ddc15bc40 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 0x7f4ddc1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 0x7f4ddc1f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39196/0 (socket says 172.21.15.45:39196) 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 -- 172.21.15.45:0/752033137 learned_addr learned my addr 172.21.15.45:0/752033137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de2000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc15bc40 0x7f4ddc152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de0c00640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc0cf930 0x7f4ddc14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 -- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc15bc40 msgr2=0x7f4ddc152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc15bc40 0x7f4ddc152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 -- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc0cf930 msgr2=0x7f4ddc14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc0cf930 0x7f4ddc14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 -- 172.21.15.45:0/752033137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc1f70d0 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de1600640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 0x7f4ddc1f6a80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00ae040 tx=0x7f4dd0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00b8020 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd00b4630 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00b0040 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc1f73b0 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc1fa0f0 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.803+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc04ef30 con 0x7f4ddc0cda10 2024-04-25T10:30:07.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.807+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4dd00bb020 con 0x7f4ddc0cda10 2024-04-25T10:30:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.807+0000 7f4dd9600640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4dbc066f90 0x7f4dbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.807+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4dd011eba0 con 0x7f4ddc0cda10 2024-04-25T10:30:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.807+0000 7f4de0c00640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4dbc066f90 0x7f4dbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.807+0000 7f4de0c00640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4dbc066f90 0x7f4dbc069450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc153250 tx=0x7f4dc8007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.811+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4dd00c2030 con 0x7f4ddc0cda10 2024-04-25T10:30:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.983+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ddc06d960 con 0x7f4ddc0cda10 2024-04-25T10:30:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.983+0000 7f4dd9600640 1 -- 172.21.15.45:0/752033137 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v157) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4dd0016070 con 0x7f4ddc0cda10 2024-04-25T10:30:07.987 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4dbc066f90 msgr2=0x7f4dbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4dbc066f90 0x7f4dbc069450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc153250 tx=0x7f4dc8007c10 comp rx=0 tx=0).stop 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 msgr2=0x7f4ddc1f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 0x7f4ddc1f6a80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00ae040 tx=0x7f4dd0019190 comp rx=0 tx=0).stop 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 shutdown_connections 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4dbc066f90 0x7f4dbc069450 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ddc15bc40 0x7f4ddc152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ddc0cf930 0x7f4ddc14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 --2- 172.21.15.45:0/752033137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ddc0cda10 0x7f4ddc1f6a80 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:07.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 >> 172.21.15.45:0/752033137 conn(0x7f4ddc066b70 msgr2=0x7f4ddc0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:07.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 shutdown_connections 2024-04-25T10:30:07.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:07.987+0000 7f4de4000640 1 -- 172.21.15.45:0/752033137 wait complete. 2024-04-25T10:30:08.011 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:30:08.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:08.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:08.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:30:08.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:08.037 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:30:08.092 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:30:08.092 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:08.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:30:08.102 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:08.102 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:08.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:30:08.148 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:30:08.148 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:08.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:30:08.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:08.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 Processor -- start 2024-04-25T10:30:08.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 -- start start 2024-04-25T10:30:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6b00ccb20 0x7fd6b00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b00d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b00ccfb0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b00cd290 con 0x7fd6b00ccb20 2024-04-25T10:30:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6b5200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00cd570 con 0x7fd6b015bdc0 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6a7e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6b00ccb20 0x7fd6b00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b00d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ade00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd6b015b9e0 0x7fd6b00cc410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:35252/0 (socket says 172.21.15.45:35252) 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ade00640 1 -- 172.21.15.45:0/3833279772 learned_addr learned my addr 172.21.15.45:0/3833279772 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ae800640 1 -- 172.21.15.45:0/3833279772 >> v2:172.21.15.117:3300/0 conn(0x7fd6b00ccb20 msgr2=0x7fd6b00d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ae800640 1 --2- 172.21.15.45:0/3833279772 >> v2:172.21.15.117:3300/0 conn(0x7fd6b00ccb20 0x7fd6b00d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ae800640 1 -- 172.21.15.45:0/3833279772 >> v1:172.21.15.117:6789/0 conn(0x7fd6b015b9e0 legacy=0x7fd6b00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.335+0000 7fd6ae800640 1 -- 172.21.15.45:0/3833279772 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00cd850 con 0x7fd6b015bdc0 2024-04-25T10:30:08.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6ae800640 1 --2- 172.21.15.45:0/3833279772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b00d1a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009d90 tx=0x7fd6a801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c7d4065544107b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6ad400640 1 -- 172.21.15.45:0/3833279772 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80b2070 con 0x7fd6b015bdc0 2024-04-25T10:30:08.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6ad400640 1 -- 172.21.15.45:0/3833279772 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80a5830 con 0x7fd6b015bdc0 2024-04-25T10:30:08.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6ad400640 1 -- 172.21.15.45:0/3833279772 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80a5c00 con 0x7fd6b015bdc0 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 -- 172.21.15.45:0/3833279772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 msgr2=0x7fd6b00d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 --2- 172.21.15.45:0/3833279772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b00d1a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009d90 tx=0x7fd6a801c970 comp rx=0 tx=0).stop 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 -- 172.21.15.45:0/3833279772 shutdown_connections 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 --2- 172.21.15.45:0/3833279772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b00d1a60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 --2- 172.21.15.45:0/3833279772 >> v2:172.21.15.117:3300/0 conn(0x7fd6b00ccb20 0x7fd6b00d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 -- 172.21.15.45:0/3833279772 >> 172.21.15.45:0/3833279772 conn(0x7fd6b009a9d0 msgr2=0x7fd6b009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 -- 172.21.15.45:0/3833279772 shutdown_connections 2024-04-25T10:30:08.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.339+0000 7fd6b5200640 1 -- 172.21.15.45:0/3833279772 wait complete. 2024-04-25T10:30:08.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 Processor -- start 2024-04-25T10:30:08.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- start start 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 0x7fd6b014a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b0153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b014a990 0x7fd6b01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00c65b0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00c6430 con 0x7fd6b014a990 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00c6730 con 0x7fd6b015bdc0 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 0x7fd6b014a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 0x7fd6b014a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39212/0 (socket says 172.21.15.45:39212) 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 -- 172.21.15.45:0/254476045 learned_addr learned my addr 172.21.15.45:0/254476045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6a7e00640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b0153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ae800640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b014a990 0x7fd6b01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 -- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 msgr2=0x7fd6b0153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b0153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 -- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b014a990 msgr2=0x7fd6b01536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b014a990 0x7fd6b01536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 -- 172.21.15.45:0/254476045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b006c7e0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6ade00640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 0x7fd6b014a130 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c420 tx=0x7fd6a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0017070 con 0x7fd6b015b9e0 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a0005cf0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b006cac0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00072f0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01660e0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.343+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b004ef30 con 0x7fd6b015b9e0 2024-04-25T10:30:08.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.347+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6a001a070 con 0x7fd6b015b9e0 2024-04-25T10:30:08.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.347+0000 7fd6a6000640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.347+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd6a00843e0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.347+0000 7fd6a7e00640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.347+0000 7fd6a7e00640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0158e80 tx=0x7fd698009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.351+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6a0054ce0 con 0x7fd6b015b9e0 2024-04-25T10:30:08.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6b00c7c40 con 0x7fd6b015b9e0 2024-04-25T10:30:08.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6a6000640 1 -- 172.21.15.45:0/254476045 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fd6a0058d80 con 0x7fd6b015b9e0 2024-04-25T10:30:08.549 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:08.549 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:26.603160+0000","modified":"2024-04-25T10:29:29.629298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9996":{"gid":9996,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2482982951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2482982951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2482982951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:30:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 msgr2=0x7fd68c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0158e80 tx=0x7fd698009190 comp rx=0 tx=0).stop 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 msgr2=0x7fd6b014a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 0x7fd6b014a130 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c420 tx=0x7fd6a000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 shutdown_connections 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd68c066f60 0x7fd68c069420 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6b014a990 0x7fd6b01536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6b015bdc0 0x7fd6b0153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 --2- 172.21.15.45:0/254476045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b015b9e0 0x7fd6b014a130 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 >> 172.21.15.45:0/254476045 conn(0x7fd6b009a9d0 msgr2=0x7fd6b006ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 shutdown_connections 2024-04-25T10:30:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.547+0000 7fd6b5200640 1 -- 172.21.15.45:0/254476045 wait complete. 2024-04-25T10:30:08.570 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:08.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 Processor -- start 2024-04-25T10:30:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 -- start start 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8c7c0b47d0 0x7f8c7c15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c7c0ccf40 0x7f8c7c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c0cd8a0 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c0b3730 con 0x7f8c7c0ccf40 2024-04-25T10:30:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c81200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c0b3a10 con 0x7f8c7c0b47d0 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c7c0ccf40 0x7f8c7c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c7c0ccf40 0x7f8c7c0cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39226/0 (socket says 172.21.15.45:39226) 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 -- 172.21.15.45:0/187903838 learned_addr learned my addr 172.21.15.45:0/187903838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c73e00640 1 --2- 172.21.15.45:0/187903838 >> v2:172.21.15.117:3300/0 conn(0x7f8c7c0b47d0 0x7f8c7c15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 -- 172.21.15.45:0/187903838 >> v2:172.21.15.117:3300/0 conn(0x7f8c7c0b47d0 msgr2=0x7f8c7c15cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 --2- 172.21.15.45:0/187903838 >> v2:172.21.15.117:3300/0 conn(0x7f8c7c0b47d0 0x7f8c7c15cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 -- 172.21.15.45:0/187903838 >> v1:172.21.15.117:6789/0 conn(0x7f8c7c15c6d0 legacy=0x7f8c7c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.711+0000 7f8c7be00640 1 -- 172.21.15.45:0/187903838 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0b3cf0 con 0x7f8c7c0ccf40 2024-04-25T10:30:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c7be00640 1 --2- 172.21.15.45:0/187903838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c0cd360 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0cfab0 tx=0x7f8c7401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9eac3243fcaf97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c7aa00640 1 -- 172.21.15.45:0/187903838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740a5880 con 0x7f8c7c0ccf40 2024-04-25T10:30:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c7aa00640 1 -- 172.21.15.45:0/187903838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c740a5a20 con 0x7f8c7c0ccf40 2024-04-25T10:30:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c7aa00640 1 -- 172.21.15.45:0/187903838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740a5df0 con 0x7f8c7c0ccf40 2024-04-25T10:30:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 -- 172.21.15.45:0/187903838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c0ccf40 msgr2=0x7f8c7c0cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 --2- 172.21.15.45:0/187903838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c0cd360 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0cfab0 tx=0x7f8c7401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 -- 172.21.15.45:0/187903838 shutdown_connections 2024-04-25T10:30:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 --2- 172.21.15.45:0/187903838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c0cd360 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 --2- 172.21.15.45:0/187903838 >> v2:172.21.15.117:3300/0 conn(0x7f8c7c0b47d0 0x7f8c7c15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 -- 172.21.15.45:0/187903838 >> 172.21.15.45:0/187903838 conn(0x7f8c7c0669a0 msgr2=0x7f8c7c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 -- 172.21.15.45:0/187903838 shutdown_connections 2024-04-25T10:30:08.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 -- 172.21.15.45:0/187903838 wait complete. 2024-04-25T10:30:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 Processor -- start 2024-04-25T10:30:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.715+0000 7f8c81200640 1 -- start start 2024-04-25T10:30:08.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c7c0b47d0 0x7f8c7c160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 0x7f8c7c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c06d040 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c06cec0 con 0x7f8c7c0ccf40 2024-04-25T10:30:08.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c06d1c0 con 0x7f8c7c0b47d0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 0x7f8c7c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 0x7f8c7c15a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39240/0 (socket says 172.21.15.45:39240) 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 -- 172.21.15.45:0/543797467 learned_addr learned my addr 172.21.15.45:0/543797467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c73e00640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7b400640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c7c0b47d0 0x7f8c7c160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 -- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c7c0b47d0 msgr2=0x7f8c7c160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c7c0b47d0 0x7f8c7c160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 -- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7c0ccf40 msgr2=0x7f8c7c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 -- 172.21.15.45:0/543797467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c14eac0 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7b400640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c7c0b47d0 0x7f8c7c160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c7be00640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 0x7f8c7c15a9f0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f8c740af040 tx=0x7f8c74019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740bc020 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c740b8670 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c14ee50 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c740b8a60 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c14f130 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c166130 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.719+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c740bf020 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.723+0000 7f8c73400640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c58066f60 0x7f8c58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:08.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.723+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8c74120470 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.723+0000 7f8c7b400640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c58066f60 0x7f8c58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:08.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.723+0000 7f8c7b400640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c58066f60 0x7f8c58069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0b1d40 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:08.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.727+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c740c6030 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.891+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c7c075120 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.891+0000 7f8c73400640 1 -- 172.21.15.45:0/543797467 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16717 (secure 0 0 0) 0x7f8c740165e0 con 0x7f8c7c15c6d0 2024-04-25T10:30:08.895 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:08.895 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":157,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:44.707588+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c58066f60 msgr2=0x7f8c58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c58066f60 0x7f8c58069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0b1d40 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 msgr2=0x7f8c7c15a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 0x7f8c7c15a9f0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f8c740af040 tx=0x7f8c74019760 comp rx=0 tx=0).stop 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 shutdown_connections 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8c58066f60 0x7f8c58069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c7c15c6d0 0x7f8c7c15a9f0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c7c0ccf40 0x7f8c7c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 --2- 172.21.15.45:0/543797467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c7c0b47d0 0x7f8c7c160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:08.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 >> 172.21.15.45:0/543797467 conn(0x7f8c7c0669a0 msgr2=0x7f8c7c152120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:08.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 shutdown_connections 2024-04-25T10:30:08.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:08.895+0000 7f8c81200640 1 -- 172.21.15.45:0/543797467 wait complete. 2024-04-25T10:30:08.918 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:29:24.390187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:29:25.607844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:08.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:09.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.059+0000 7f13e0000640 1 Processor -- start 2024-04-25T10:30:09.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.059+0000 7f13e0000640 1 -- start start 2024-04-25T10:30:09.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13e0000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13d815b150 0x7f13d80aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13e0000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13d806c0d0 0x7f13d806c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13e0000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80b2fd0 con 0x7f13d815a0e0 2024-04-25T10:30:09.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13e0000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d815b5c0 con 0x7f13d815b150 2024-04-25T10:30:09.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13e0000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d80aef90 con 0x7f13d806c0d0 2024-04-25T10:30:09.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dcc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f13d815b150 0x7f13d80aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dd600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f13d815a0e0 0x7f13d80b3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56352/0 (socket says 172.21.15.45:56352) 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dd600640 1 -- 172.21.15.45:0/3365579662 learned_addr learned my addr 172.21.15.45:0/3365579662 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dcc00640 1 -- 172.21.15.45:0/3365579662 >> v2:172.21.15.117:3300/0 conn(0x7f13d806c0d0 msgr2=0x7f13d806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dcc00640 1 --2- 172.21.15.45:0/3365579662 >> v2:172.21.15.117:3300/0 conn(0x7f13d806c0d0 0x7f13d806c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dcc00640 1 -- 172.21.15.45:0/3365579662 >> v1:172.21.15.45:6789/0 conn(0x7f13d815a0e0 legacy=0x7f13d80b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13dcc00640 1 -- 172.21.15.45:0/3365579662 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80b32b0 con 0x7f13d815b150 2024-04-25T10:30:09.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.063+0000 7f13d7e00640 1 -- 172.21.15.45:0/3365579662 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169975679 0 0) 0x7f13d80b2fd0 con 0x7f13d815a0e0 2024-04-25T10:30:09.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13dcc00640 1 --2- 172.21.15.45:0/3365579662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815b150 0x7f13d80aea50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f13d80cc120 tx=0x7f13c401d730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad6203bb467243f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13d7e00640 1 -- 172.21.15.45:0/3365579662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c400b040 con 0x7f13d815b150 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13d7e00640 1 -- 172.21.15.45:0/3365579662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c40a76a0 con 0x7f13d815b150 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13d7e00640 1 -- 172.21.15.45:0/3365579662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c40a7a70 con 0x7f13d815b150 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- 172.21.15.45:0/3365579662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815b150 msgr2=0x7f13d80aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 --2- 172.21.15.45:0/3365579662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815b150 0x7f13d80aea50 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f13d80cc120 tx=0x7f13c401d730 comp rx=0 tx=0).stop 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- 172.21.15.45:0/3365579662 shutdown_connections 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 --2- 172.21.15.45:0/3365579662 >> v2:172.21.15.117:3300/0 conn(0x7f13d806c0d0 0x7f13d806c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 --2- 172.21.15.45:0/3365579662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815b150 0x7f13d80aea50 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- 172.21.15.45:0/3365579662 >> 172.21.15.45:0/3365579662 conn(0x7f13d80669a0 msgr2=0x7f13d8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- 172.21.15.45:0/3365579662 shutdown_connections 2024-04-25T10:30:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- 172.21.15.45:0/3365579662 wait complete. 2024-04-25T10:30:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 Processor -- start 2024-04-25T10:30:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- start start 2024-04-25T10:30:09.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d806c0d0 0x7f13d81521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 0x7f13d814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d815b150 0x7f13d8146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80705f0 con 0x7f13d815a0e0 2024-04-25T10:30:09.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80af1d0 con 0x7f13d806c0d0 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.067+0000 7f13e0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8070770 con 0x7f13d815b150 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 0x7f13d814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 0x7f13d814ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39266/0 (socket says 172.21.15.45:39266) 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 -- 172.21.15.45:0/3474024718 learned_addr learned my addr 172.21.15.45:0/3474024718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dd600640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d806c0d0 0x7f13d81521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13de000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d815b150 0x7f13d8146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 -- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d815b150 msgr2=0x7f13d8146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d815b150 0x7f13d8146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 -- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d806c0d0 msgr2=0x7f13d81521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d806c0d0 0x7f13d81521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 -- 172.21.15.45:0/3474024718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8152a10 con 0x7f13d815a0e0 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13dcc00640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 0x7f13d814ff30 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f13c401b360 tx=0x7f13c401a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c40a8890 con 0x7f13d815a0e0 2024-04-25T10:30:09.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c40a8a30 con 0x7f13d815a0e0 2024-04-25T10:30:09.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8152cf0 con 0x7f13d815a0e0 2024-04-25T10:30:09.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c40a8de0 con 0x7f13d815a0e0 2024-04-25T10:30:09.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81532b0 con 0x7f13d815a0e0 2024-04-25T10:30:09.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d804ef30 con 0x7f13d815a0e0 2024-04-25T10:30:09.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.071+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13c40a7520 con 0x7f13d815a0e0 2024-04-25T10:30:09.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.075+0000 7f13d5600640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13b8066f60 0x7f13b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.075+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f13c411f220 con 0x7f13d815a0e0 2024-04-25T10:30:09.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.075+0000 7f13dd600640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13b8066f60 0x7f13b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.075+0000 7f13dd600640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13b8066f60 0x7f13b8069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f13cc000f70 tx=0x7f13cc007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.079+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13c40be060 con 0x7f13d815a0e0 2024-04-25T10:30:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13d806e1f0 con 0x7f13d815a0e0 2024-04-25T10:30:09.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13d5600640 1 -- 172.21.15.45:0/3474024718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13c40ad030 con 0x7f13d815a0e0 2024-04-25T10:30:09.241 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:30:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13b8066f60 msgr2=0x7f13b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13b8066f60 0x7f13b8069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f13cc000f70 tx=0x7f13cc007b60 comp rx=0 tx=0).stop 2024-04-25T10:30:09.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 msgr2=0x7f13d814ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 0x7f13d814ff30 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f13c401b360 tx=0x7f13c401a4d0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 shutdown_connections 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f13b8066f60 0x7f13b8069420 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d815b150 0x7f13d8146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13d815a0e0 0x7f13d814ff30 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 --2- 172.21.15.45:0/3474024718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d806c0d0 0x7f13d81521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 >> 172.21.15.45:0/3474024718 conn(0x7f13d80669a0 msgr2=0x7f13d80ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.239+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 shutdown_connections 2024-04-25T10:30:09.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.243+0000 7f13e0000640 1 -- 172.21.15.45:0/3474024718 wait complete. 2024-04-25T10:30:09.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:30:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 Processor -- start 2024-04-25T10:30:09.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 -- start start 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f55000aeda0 0x7f5500071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5500072020 0x7f5500070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5500159920 con 0x7f5500159540 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550015c2f0 con 0x7f5500072020 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f5505800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550015c5d0 con 0x7f55000aeda0 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5500072020 0x7f5500070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5500072020 0x7f5500070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39268/0 (socket says 172.21.15.45:39268) 2024-04-25T10:30:09.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 -- 172.21.15.45:0/2065485936 learned_addr learned my addr 172.21.15.45:0/2065485936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54f6000640 1 --2- 172.21.15.45:0/2065485936 >> v2:172.21.15.117:3300/0 conn(0x7f55000aeda0 0x7f5500071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 -- 172.21.15.45:0/2065485936 >> v2:172.21.15.117:3300/0 conn(0x7f55000aeda0 msgr2=0x7f5500071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 --2- 172.21.15.45:0/2065485936 >> v2:172.21.15.117:3300/0 conn(0x7f55000aeda0 0x7f5500071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 -- 172.21.15.45:0/2065485936 >> v1:172.21.15.117:6789/0 conn(0x7f5500159540 legacy=0x7f55000ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.407+0000 7f54ff400640 1 -- 172.21.15.45:0/2065485936 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550015c8b0 con 0x7f5500072020 2024-04-25T10:30:09.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f54ff400640 1 --2- 172.21.15.45:0/2065485936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500072020 0x7f5500070350 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f55000cd510 tx=0x7f54f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c775691dbcbdd81c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f54fe000640 1 -- 172.21.15.45:0/2065485936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80aa430 con 0x7f5500072020 2024-04-25T10:30:09.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f54fe000640 1 -- 172.21.15.45:0/2065485936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f80af070 con 0x7f5500072020 2024-04-25T10:30:09.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f54fe000640 1 -- 172.21.15.45:0/2065485936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80aa7f0 con 0x7f5500072020 2024-04-25T10:30:09.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 -- 172.21.15.45:0/2065485936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500072020 msgr2=0x7f5500070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 --2- 172.21.15.45:0/2065485936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500072020 0x7f5500070350 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f55000cd510 tx=0x7f54f801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 -- 172.21.15.45:0/2065485936 shutdown_connections 2024-04-25T10:30:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 --2- 172.21.15.45:0/2065485936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5500072020 0x7f5500070350 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 --2- 172.21.15.45:0/2065485936 >> v2:172.21.15.117:3300/0 conn(0x7f55000aeda0 0x7f5500071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 -- 172.21.15.45:0/2065485936 >> 172.21.15.45:0/2065485936 conn(0x7f55000669a0 msgr2=0x7f5500066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 -- 172.21.15.45:0/2065485936 shutdown_connections 2024-04-25T10:30:09.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 -- 172.21.15.45:0/2065485936 wait complete. 2024-04-25T10:30:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.411+0000 7f5505800640 1 Processor -- start 2024-04-25T10:30:09.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- start start 2024-04-25T10:30:09.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5500072020 0x7f550015b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 0x7f550013dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5500159540 0x7f5500156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000710a0 con 0x7f55000aeda0 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550006c0d0 con 0x7f5500072020 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5500071220 con 0x7f5500159540 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 0x7f550013dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 0x7f550013dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39272/0 (socket says 172.21.15.45:39272) 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 -- 172.21.15.45:0/3970549596 learned_addr learned my addr 172.21.15.45:0/3970549596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54fea00640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5500072020 0x7f550015b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 -- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5500159540 msgr2=0x7f5500156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54ff400640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5500159540 0x7f5500156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5500159540 0x7f5500156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 -- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5500072020 msgr2=0x7f550015b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5500072020 0x7f550015b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 -- 172.21.15.45:0/3970549596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55000b29f0 con 0x7f55000aeda0 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54fea00640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5500072020 0x7f550015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54ff400640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5500159540 0x7f5500156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f6000640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 0x7f550013dfd0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f54e800d7d0 tx=0x7f54e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800b6c0 con 0x7f55000aeda0 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8019070 con 0x7f55000aeda0 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800ba80 con 0x7f55000aeda0 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55000b2cb0 con 0x7f55000aeda0 2024-04-25T10:30:09.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55000b3270 con 0x7f55000aeda0 2024-04-25T10:30:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.415+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550004ef30 con 0x7f55000aeda0 2024-04-25T10:30:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.419+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54e800bc20 con 0x7f55000aeda0 2024-04-25T10:30:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.419+0000 7f54f7400640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54dc066f90 0x7f54dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.419+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f54e8085540 con 0x7f55000aeda0 2024-04-25T10:30:09.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.419+0000 7f54fea00640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54dc066f90 0x7f54dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.419+0000 7f54fea00640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54dc066f90 0x7f54dc069450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f54f0007fd0 tx=0x7f54f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.423+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54e8056400 con 0x7f55000aeda0 2024-04-25T10:30:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.583+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5500166700 con 0x7f55000aeda0 2024-04-25T10:30:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.583+0000 7f54f7400640 1 -- 172.21.15.45:0/3970549596 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54e8055d00 con 0x7f55000aeda0 2024-04-25T10:30:09.587 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54dc066f90 msgr2=0x7f54dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54dc066f90 0x7f54dc069450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f54f0007fd0 tx=0x7f54f0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 msgr2=0x7f550013dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 0x7f550013dfd0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f54e800d7d0 tx=0x7f54e800dc90 comp rx=0 tx=0).stop 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 shutdown_connections 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54dc066f90 0x7f54dc069450 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5500159540 0x7f5500156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55000aeda0 0x7f550013dfd0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 --2- 172.21.15.45:0/3970549596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5500072020 0x7f550015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 >> 172.21.15.45:0/3970549596 conn(0x7f55000669a0 msgr2=0x7f5500066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 shutdown_connections 2024-04-25T10:30:09.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.587+0000 7f5505800640 1 -- 172.21.15.45:0/3970549596 wait complete. 2024-04-25T10:30:09.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:30:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 Processor -- start 2024-04-25T10:30:09.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 -- start start 2024-04-25T10:30:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750159540 0x7fa7500ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7500aebd0 0x7fa750071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa750072020 0x7fa75015f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7500af030 con 0x7fa750072020 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa750159920 con 0x7fa7500aebd0 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa757200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75015c2f0 con 0x7fa750159540 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa750072020 0x7fa75015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa750072020 0x7fa75015f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39276/0 (socket says 172.21.15.45:39276) 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 -- 172.21.15.45:0/4202675124 learned_addr learned my addr 172.21.15.45:0/4202675124 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa74fe00640 1 --2- 172.21.15.45:0/4202675124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750159540 0x7fa7500ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa74f400640 1 --2- 172.21.15.45:0/4202675124 >> v2:172.21.15.117:3300/0 conn(0x7fa7500aebd0 0x7fa750071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 -- 172.21.15.45:0/4202675124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750159540 msgr2=0x7fa7500ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 --2- 172.21.15.45:0/4202675124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750159540 0x7fa7500ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 -- 172.21.15.45:0/4202675124 >> v2:172.21.15.117:3300/0 conn(0x7fa7500aebd0 msgr2=0x7fa750071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 --2- 172.21.15.45:0/4202675124 >> v2:172.21.15.117:3300/0 conn(0x7fa7500aebd0 0x7fa750071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.751+0000 7fa755200640 1 -- 172.21.15.45:0/4202675124 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75015c5d0 con 0x7fa750072020 2024-04-25T10:30:09.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa755200640 1 --2- 172.21.15.45:0/4202675124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750072020 0x7fa75015f590 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fa7500b4b30 tx=0x7fa74801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3a118915c3d26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa74ea00640 1 -- 172.21.15.45:0/4202675124 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480a5580 con 0x7fa750072020 2024-04-25T10:30:09.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa74ea00640 1 -- 172.21.15.45:0/4202675124 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7480a5720 con 0x7fa750072020 2024-04-25T10:30:09.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa74ea00640 1 -- 172.21.15.45:0/4202675124 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480a5af0 con 0x7fa750072020 2024-04-25T10:30:09.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 -- 172.21.15.45:0/4202675124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750072020 msgr2=0x7fa75015f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 --2- 172.21.15.45:0/4202675124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750072020 0x7fa75015f590 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fa7500b4b30 tx=0x7fa74801c700 comp rx=0 tx=0).stop 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 -- 172.21.15.45:0/4202675124 shutdown_connections 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 --2- 172.21.15.45:0/4202675124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750072020 0x7fa75015f590 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 --2- 172.21.15.45:0/4202675124 >> v2:172.21.15.117:3300/0 conn(0x7fa7500aebd0 0x7fa750071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 --2- 172.21.15.45:0/4202675124 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750159540 0x7fa7500ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 -- 172.21.15.45:0/4202675124 >> 172.21.15.45:0/4202675124 conn(0x7fa7500669a0 msgr2=0x7fa750066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 -- 172.21.15.45:0/4202675124 shutdown_connections 2024-04-25T10:30:09.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 -- 172.21.15.45:0/4202675124 wait complete. 2024-04-25T10:30:09.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 Processor -- start 2024-04-25T10:30:09.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.755+0000 7fa757200640 1 -- start start 2024-04-25T10:30:09.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750072020 0x7fa75014dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7500aebd0 0x7fa75015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 0x7fa750159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75006c0d0 con 0x7fa750159540 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa750072450 con 0x7fa7500aebd0 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa750070350 con 0x7fa750072020 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 0x7fa750159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 0x7fa750159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39278/0 (socket says 172.21.15.45:39278) 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 -- 172.21.15.45:0/2327458584 learned_addr learned my addr 172.21.15.45:0/2327458584 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa74fe00640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750072020 0x7fa75014dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa74f400640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7500aebd0 0x7fa75015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 -- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750072020 msgr2=0x7fa75014dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750072020 0x7fa75014dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 -- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7500aebd0 msgr2=0x7fa75015e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7500aebd0 0x7fa75015e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 -- 172.21.15.45:0/2327458584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750156c60 con 0x7fa750159540 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa755200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 0x7fa750159b30 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fa7480af040 tx=0x7fa7480194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748009510 con 0x7fa750159540 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7480be070 con 0x7fa750159540 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75014e9f0 con 0x7fa750159540 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480b88b0 con 0x7fa750159540 2024-04-25T10:30:09.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa750156f20 con 0x7fa750159540 2024-04-25T10:30:09.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.759+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75004ef30 con 0x7fa750159540 2024-04-25T10:30:09.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.763+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7480b8a50 con 0x7fa750159540 2024-04-25T10:30:09.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.763+0000 7fa747e00640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa730066f60 0x7fa730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:09.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.763+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa7481202f0 con 0x7fa750159540 2024-04-25T10:30:09.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.763+0000 7fa74fe00640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa730066f60 0x7fa730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:09.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.763+0000 7fa74fe00640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa730066f60 0x7fa730069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa740007940 tx=0x7fa740008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:09.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.767+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7480165e0 con 0x7fa750159540 2024-04-25T10:30:09.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.927+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7500cf470 con 0x7fa750159540 2024-04-25T10:30:09.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.927+0000 7fa747e00640 1 -- 172.21.15.45:0/2327458584 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7480f0dd0 con 0x7fa750159540 2024-04-25T10:30:09.930 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa730066f60 msgr2=0x7fa730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa730066f60 0x7fa730069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa740007940 tx=0x7fa740008040 comp rx=0 tx=0).stop 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 msgr2=0x7fa750159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 0x7fa750159b30 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fa7480af040 tx=0x7fa7480194a0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 shutdown_connections 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa730066f60 0x7fa730069420 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa750159540 0x7fa750159b30 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7500aebd0 0x7fa75015e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 --2- 172.21.15.45:0/2327458584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa750072020 0x7fa75014dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 >> 172.21.15.45:0/2327458584 conn(0x7fa7500669a0 msgr2=0x7fa7500cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:09.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 shutdown_connections 2024-04-25T10:30:09.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:09.931+0000 7fa757200640 1 -- 172.21.15.45:0/2327458584 wait complete. 2024-04-25T10:30:09.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 Processor -- start 2024-04-25T10:30:10.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 -- start start 2024-04-25T10:30:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f228c15bc40 0x7f228c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cda10 0x7f228c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c0b2770 con 0x7f228c0cf930 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c06c670 con 0x7f228c15bc40 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2294200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c06c950 con 0x7f228c0cda10 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2291800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f228c15bc40 0x7f228c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2292200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f228c0cf930 0x7f228c15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56362/0 (socket says 172.21.15.45:56362) 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2292200640 1 -- 172.21.15.45:0/2611131804 learned_addr learned my addr 172.21.15.45:0/2611131804 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f2290e00640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cda10 0x7f228c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.091+0000 7f228be00640 1 -- 172.21.15.45:0/2611131804 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470588095 0 0) 0x7f228c0b2770 con 0x7f228c0cf930 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f228be00640 1 -- 172.21.15.45:0/2611131804 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2274003530 con 0x7f228c0cf930 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2291800640 1 -- 172.21.15.45:0/2611131804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cda10 msgr2=0x7f228c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2291800640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cda10 0x7f228c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2291800640 1 -- 172.21.15.45:0/2611131804 >> v1:172.21.15.45:6789/0 conn(0x7f228c0cf930 legacy=0x7f228c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2291800640 1 -- 172.21.15.45:0/2611131804 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c06cc30 con 0x7f228c15bc40 2024-04-25T10:30:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2290e00640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cda10 0x7f228c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2291800640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c06c090 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f22800097c0 tx=0x7f228001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4705583043ad5d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f228be00640 1 -- 172.21.15.45:0/2611131804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800b1070 con 0x7f228c15bc40 2024-04-25T10:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f228be00640 1 -- 172.21.15.45:0/2611131804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22800ac410 con 0x7f228c15bc40 2024-04-25T10:30:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f228be00640 1 -- 172.21.15.45:0/2611131804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800ac7c0 con 0x7f228c15bc40 2024-04-25T10:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 -- 172.21.15.45:0/2611131804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 msgr2=0x7f228c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c06c090 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f22800097c0 tx=0x7f228001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 -- 172.21.15.45:0/2611131804 shutdown_connections 2024-04-25T10:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cda10 0x7f228c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 --2- 172.21.15.45:0/2611131804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c06c090 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 -- 172.21.15.45:0/2611131804 >> 172.21.15.45:0/2611131804 conn(0x7f228c066b70 msgr2=0x7f228c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 -- 172.21.15.45:0/2611131804 shutdown_connections 2024-04-25T10:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.095+0000 7f2294200640 1 -- 172.21.15.45:0/2611131804 wait complete. 2024-04-25T10:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 Processor -- start 2024-04-25T10:30:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 -- start start 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c0cda10 0x7f228c148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cf930 0x7f228c1526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c168b50 con 0x7f228c15bc40 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1689d0 con 0x7f228c0cda10 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c168cd0 con 0x7f228c0cf930 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39298/0 (socket says 172.21.15.45:39298) 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 -- 172.21.15.45:0/3695902079 learned_addr learned my addr 172.21.15.45:0/3695902079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2291800640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c0cda10 0x7f228c148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2290e00640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cf930 0x7f228c1526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 -- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cf930 msgr2=0x7f228c1526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cf930 0x7f228c1526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 -- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c0cda10 msgr2=0x7f228c148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c0cda10 0x7f228c148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 -- 172.21.15.45:0/3695902079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c1492f0 con 0x7f228c15bc40 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2291800640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c0cda10 0x7f228c148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2292200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c152bf0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f228400caa0 tx=0x7f228400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284017070 con 0x7f228c15bc40 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228400ac00 con 0x7f228c15bc40 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c148770 con 0x7f228c15bc40 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284012400 con 0x7f228c15bc40 2024-04-25T10:30:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.099+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c148d30 con 0x7f228c15bc40 2024-04-25T10:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.103+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c04ef30 con 0x7f228c15bc40 2024-04-25T10:30:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.103+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f228401a070 con 0x7f228c15bc40 2024-04-25T10:30:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.103+0000 7f2289600640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f226c066f60 0x7f226c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.103+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2284084ea0 con 0x7f228c15bc40 2024-04-25T10:30:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.103+0000 7f2291800640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f226c066f60 0x7f226c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.107+0000 7f2291800640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f226c066f60 0x7f226c069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f22800ae000 tx=0x7f22800a6040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.107+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2284055df0 con 0x7f228c15bc40 2024-04-25T10:30:10.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f228c06d960 con 0x7f228c15bc40 2024-04-25T10:30:10.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2289600640 1 -- 172.21.15.45:0/3695902079 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f22840556f0 con 0x7f228c15bc40 2024-04-25T10:30:10.305 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:10.305 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:26.603160+0000","modified":"2024-04-25T10:29:29.629298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9996":{"gid":9996,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2482982951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2482982951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2482982951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:30:10.306 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-04-25T10:30:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f226c066f60 msgr2=0x7f226c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f226c066f60 0x7f226c069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f22800ae000 tx=0x7f22800a6040 comp rx=0 tx=0).stop 2024-04-25T10:30:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 msgr2=0x7f228c152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c152bf0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f228400caa0 tx=0x7f228400cf60 comp rx=0 tx=0).stop 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 shutdown_connections 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f226c066f60 0x7f226c069420 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f228c15bc40 0x7f228c152bf0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228c0cf930 0x7f228c1526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 --2- 172.21.15.45:0/3695902079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f228c0cda10 0x7f228c148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.303+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 >> 172.21.15.45:0/3695902079 conn(0x7f228c066b70 msgr2=0x7f228c158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.307+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 shutdown_connections 2024-04-25T10:30:10.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.307+0000 7f2294200640 1 -- 172.21.15.45:0/3695902079 wait complete. 2024-04-25T10:30:10.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 Processor -- start 2024-04-25T10:30:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 -- start start 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb67c066c00 0x7fb67c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb67c0b5560 0x7fb67c157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0b5970 con 0x7fb67c066fe0 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0b5c20 con 0x7fb67c066c00 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb681e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0af6e0 con 0x7fb67c0b5560 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb67c066c00 0x7fb67c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb67c066c00 0x7fb67c15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39302/0 (socket says 172.21.15.45:39302) 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 -- 172.21.15.45:0/287429223 learned_addr learned my addr 172.21.15.45:0/287429223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67b400640 1 --2- 172.21.15.45:0/287429223 >> v2:172.21.15.117:3300/0 conn(0x7fb67c0b5560 0x7fb67c157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 -- 172.21.15.45:0/287429223 >> v2:172.21.15.117:3300/0 conn(0x7fb67c0b5560 msgr2=0x7fb67c157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 --2- 172.21.15.45:0/287429223 >> v2:172.21.15.117:3300/0 conn(0x7fb67c0b5560 0x7fb67c157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 -- 172.21.15.45:0/287429223 >> v1:172.21.15.117:6789/0 conn(0x7fb67c066fe0 legacy=0x7fb67c06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.467+0000 7fb67aa00640 1 -- 172.21.15.45:0/287429223 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0af9c0 con 0x7fb67c066c00 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb67aa00640 1 --2- 172.21.15.45:0/287429223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c15e110 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0b3360 tx=0x7fb66c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd7b27b16b0e491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb67a000640 1 -- 172.21.15.45:0/287429223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0aa650 con 0x7fb67c066c00 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb67a000640 1 -- 172.21.15.45:0/287429223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c0af070 con 0x7fb67c066c00 2024-04-25T10:30:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb67a000640 1 -- 172.21.15.45:0/287429223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0aaa10 con 0x7fb67c066c00 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 -- 172.21.15.45:0/287429223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 msgr2=0x7fb67c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 --2- 172.21.15.45:0/287429223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c15e110 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0b3360 tx=0x7fb66c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 -- 172.21.15.45:0/287429223 shutdown_connections 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 --2- 172.21.15.45:0/287429223 >> v2:172.21.15.117:3300/0 conn(0x7fb67c0b5560 0x7fb67c157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 --2- 172.21.15.45:0/287429223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c15e110 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 -- 172.21.15.45:0/287429223 >> 172.21.15.45:0/287429223 conn(0x7fb67c0b24f0 msgr2=0x7fb67c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 -- 172.21.15.45:0/287429223 shutdown_connections 2024-04-25T10:30:10.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.471+0000 7fb681e00640 1 -- 172.21.15.45:0/287429223 wait complete. 2024-04-25T10:30:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 Processor -- start 2024-04-25T10:30:10.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- start start 2024-04-25T10:30:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c13a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c066fe0 0x7fb67c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb67c0b5560 0x7fb67c15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0af1e0 con 0x7fb67c066c00 2024-04-25T10:30:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0af060 con 0x7fb67c066fe0 2024-04-25T10:30:10.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c06ac90 con 0x7fb67c0b5560 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c13a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c13a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39310/0 (socket says 172.21.15.45:39310) 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 -- 172.21.15.45:0/208584737 learned_addr learned my addr 172.21.15.45:0/208584737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb672000640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c066fe0 0x7fb67c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67b400640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb67c0b5560 0x7fb67c15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 -- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb67c0b5560 msgr2=0x7fb67c15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb67c0b5560 0x7fb67c15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 -- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c066fe0 msgr2=0x7fb67c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c066fe0 0x7fb67c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 -- 172.21.15.45:0/208584737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c160bd0 con 0x7fb67c066c00 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb672000640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c066fe0 0x7fb67c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb67aa00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c13a1e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0ac040 tx=0x7fb66c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0bd5f0 con 0x7fb67c066c00 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c0a6040 con 0x7fb67c066c00 2024-04-25T10:30:10.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0bd9b0 con 0x7fb67c066c00 2024-04-25T10:30:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c1594d0 con 0x7fb67c066c00 2024-04-25T10:30:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c13a720 con 0x7fb67c066c00 2024-04-25T10:30:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.475+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c04ef30 con 0x7fb67c066c00 2024-04-25T10:30:10.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.479+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb66c0b33f0 con 0x7fb67c066c00 2024-04-25T10:30:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.479+0000 7fb673e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb658066f60 0x7fb658069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.479+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb66c11f400 con 0x7fb67c066c00 2024-04-25T10:30:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.479+0000 7fb672000640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb658066f60 0x7fb658069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.479+0000 7fb672000640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb658066f60 0x7fb658069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb67c1fb4e0 tx=0x7fb664009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.483+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb66c016110 con 0x7fb67c066c00 2024-04-25T10:30:10.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.643+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb67c071c70 con 0x7fb67c066c00 2024-04-25T10:30:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb673e00640 1 -- 172.21.15.45:0/208584737 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fb66c0efd90 con 0x7fb67c066c00 2024-04-25T10:30:10.648 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:10.648 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":157,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:44.707588+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:10.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb658066f60 msgr2=0x7fb658069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb658066f60 0x7fb658069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb67c1fb4e0 tx=0x7fb664009040 comp rx=0 tx=0).stop 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 msgr2=0x7fb67c13a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c13a1e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0ac040 tx=0x7fb66c01ad30 comp rx=0 tx=0).stop 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 shutdown_connections 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb658066f60 0x7fb658069420 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb67c0b5560 0x7fb67c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c066fe0 0x7fb67c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 --2- 172.21.15.45:0/208584737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c066c00 0x7fb67c13a1e0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 >> 172.21.15.45:0/208584737 conn(0x7fb67c0b24f0 msgr2=0x7fb67c0b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 shutdown_connections 2024-04-25T10:30:10.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.647+0000 7fb681e00640 1 -- 172.21.15.45:0/208584737 wait complete. 2024-04-25T10:30:10.670 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:30:10.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f0702200640 1 Processor -- start 2024-04-25T10:30:10.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f0702200640 1 -- start start 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f0702200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f06fc156150 0x7f06fc071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f0702200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc156530 con 0x7f06fc15c080 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f0702200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc15d220 con 0x7f06fc0670b0 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f0702200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc15c640 con 0x7f06fc156150 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06faa00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06fc15c080 0x7f06fc0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56364/0 (socket says 172.21.15.45:56364) 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06faa00640 1 -- 172.21.15.45:0/1124802521 learned_addr learned my addr 172.21.15.45:0/1124802521 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:10.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fb400640 1 --2- 172.21.15.45:0/1124802521 >> v2:172.21.15.117:3300/0 conn(0x7f06fc156150 0x7f06fc071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620721972 0 0) 0x7f06fc156530 con 0x7f06fc15c080 2024-04-25T10:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e0003530 con 0x7f06fc15c080 2024-04-25T10:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018021735 0 0) 0x7f06fc15d220 con 0x7f06fc0670b0 2024-04-25T10:30:10.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06fc156530 con 0x7f06fc0670b0 2024-04-25T10:30:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fb400640 1 -- 172.21.15.45:0/1124802521 >> v1:172.21.15.117:6789/0 conn(0x7f06fc0670b0 legacy=0x7f06fc0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fb400640 1 -- 172.21.15.45:0/1124802521 >> v1:172.21.15.45:6789/0 conn(0x7f06fc15c080 legacy=0x7f06fc0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.811+0000 7f06fb400640 1 -- 172.21.15.45:0/1124802521 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc15c920 con 0x7f06fc156150 2024-04-25T10:30:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f06fb400640 1 --2- 172.21.15.45:0/1124802521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc156150 0x7f06fc071f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0097c0 tx=0x7f06ec01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dabf47c51d084076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0a7070 con 0x7f06fc156150 2024-04-25T10:30:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec0b0070 con 0x7f06fc156150 2024-04-25T10:30:10.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f06fa000640 1 -- 172.21.15.45:0/1124802521 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0ab890 con 0x7f06fc156150 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 -- 172.21.15.45:0/1124802521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc156150 msgr2=0x7f06fc071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 --2- 172.21.15.45:0/1124802521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc156150 0x7f06fc071f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0097c0 tx=0x7f06ec01c470 comp rx=0 tx=0).stop 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 -- 172.21.15.45:0/1124802521 shutdown_connections 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 --2- 172.21.15.45:0/1124802521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc156150 0x7f06fc071f30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 -- 172.21.15.45:0/1124802521 >> 172.21.15.45:0/1124802521 conn(0x7f06fc06ac90 msgr2=0x7f06fc06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 -- 172.21.15.45:0/1124802521 shutdown_connections 2024-04-25T10:30:10.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.815+0000 7f0702200640 1 -- 172.21.15.45:0/1124802521 wait complete. 2024-04-25T10:30:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 Processor -- start 2024-04-25T10:30:10.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- start start 2024-04-25T10:30:10.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc0670b0 0x7f06fc154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 0x7f06fc150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc15c080 0x7f06fc0aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc169830 con 0x7f06fc156150 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1696b0 con 0x7f06fc0670b0 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1699b0 con 0x7f06fc15c080 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 0x7f06fc150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 0x7f06fc150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39322/0 (socket says 172.21.15.45:39322) 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 -- 172.21.15.45:0/901387114 learned_addr learned my addr 172.21.15.45:0/901387114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06fb400640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc0670b0 0x7f06fc154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06fbe00640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc15c080 0x7f06fc0aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 -- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc15c080 msgr2=0x7f06fc0aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc15c080 0x7f06fc0aac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 -- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc0670b0 msgr2=0x7f06fc154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc0670b0 0x7f06fc154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 -- 172.21.15.45:0/901387114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc1518b0 con 0x7f06fc156150 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06fbe00640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc15c080 0x7f06fc0aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06faa00640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 0x7f06fc150720 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f06e800c4d0 tx=0x7f06e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8017070 con 0x7f06fc156150 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8005cf0 con 0x7f06fc156150 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc154e10 con 0x7f06fc156150 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8007290 con 0x7f06fc156150 2024-04-25T10:30:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc0aa120 con 0x7f06fc156150 2024-04-25T10:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.819+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc04ef30 con 0x7f06fc156150 2024-04-25T10:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.823+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06e801a070 con 0x7f06fc156150 2024-04-25T10:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.823+0000 7f06f3400640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06dc066f90 0x7f06dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:10.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.823+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f06e8084440 con 0x7f06fc156150 2024-04-25T10:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.823+0000 7f06fb400640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06dc066f90 0x7f06dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:10.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.823+0000 7f06fb400640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06dc066f90 0x7f06dc069450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0af000 tx=0x7f06ec0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:10.827+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06e8055420 con 0x7f06fc156150 2024-04-25T10:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06fc06c3f0 con 0x7f06fc156150 2024-04-25T10:30:11.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f06f3400640 1 -- 172.21.15.45:0/901387114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f06e8054d20 con 0x7f06fc156150 2024-04-25T10:30:11.025 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:11.025 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:26.603160+0000","modified":"2024-04-25T10:29:29.629298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9996":{"gid":9996,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2482982951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2482982951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2482982951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:30:11.025 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-04-25T10:30:11.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06dc066f90 msgr2=0x7f06dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06dc066f90 0x7f06dc069450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0af000 tx=0x7f06ec0a7000 comp rx=0 tx=0).stop 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 msgr2=0x7f06fc150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 0x7f06fc150720 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f06e800c4d0 tx=0x7f06e800c990 comp rx=0 tx=0).stop 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 shutdown_connections 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f06dc066f90 0x7f06dc069450 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f06fc15c080 0x7f06fc0aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06fc156150 0x7f06fc150720 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 --2- 172.21.15.45:0/901387114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06fc0670b0 0x7f06fc154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 >> 172.21.15.45:0/901387114 conn(0x7f06fc06ac90 msgr2=0x7f06fc15d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 shutdown_connections 2024-04-25T10:30:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.023+0000 7f0702200640 1 -- 172.21.15.45:0/901387114 wait complete. 2024-04-25T10:30:11.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:11.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 Processor -- start 2024-04-25T10:30:11.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 -- start start 2024-04-25T10:30:11.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0f415dc10 0x7fb0f40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0f406bdb0 0x7fb0f40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f40cbff0 con 0x7fb0f415dff0 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40b45a0 con 0x7fb0f406bdb0 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40cbda0 con 0x7fb0f415dc10 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0f406bdb0 0x7fb0f40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0ebe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb0f415dff0 0x7fb0f406b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56372/0 (socket says 172.21.15.45:56372) 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb0f406bdb0 0x7fb0f40ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39332/0 (socket says 172.21.15.45:39332) 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0ebe00640 1 -- 172.21.15.45:0/1060864824 learned_addr learned my addr 172.21.15.45:0/1060864824 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f2a00640 1 --2- 172.21.15.45:0/1060864824 >> v2:172.21.15.117:3300/0 conn(0x7fb0f415dc10 0x7fb0f40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3400640 1 -- 172.21.15.45:0/1060864824 >> v2:172.21.15.117:3300/0 conn(0x7fb0f415dc10 msgr2=0x7fb0f40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3400640 1 --2- 172.21.15.45:0/1060864824 >> v2:172.21.15.117:3300/0 conn(0x7fb0f415dc10 0x7fb0f40d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3400640 1 -- 172.21.15.45:0/1060864824 >> v1:172.21.15.45:6789/0 conn(0x7fb0f415dff0 legacy=0x7fb0f406b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f2000640 1 -- 172.21.15.45:0/1060864824 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245429966 0 0) 0x7fb0f40cbff0 con 0x7fb0f415dff0 2024-04-25T10:30:11.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.187+0000 7fb0f3400640 1 -- 172.21.15.45:0/1060864824 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40cc2d0 con 0x7fb0f406bdb0 2024-04-25T10:30:11.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3400640 1 --2- 172.21.15.45:0/1060864824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f406bdb0 0x7fb0f40ccdb0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b4270 tx=0x7fb0ec01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1713859ea9055649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f2000640 1 -- 172.21.15.45:0/1060864824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0b2070 con 0x7fb0f406bdb0 2024-04-25T10:30:11.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f2000640 1 -- 172.21.15.45:0/1060864824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec0a59c0 con 0x7fb0f406bdb0 2024-04-25T10:30:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f2000640 1 -- 172.21.15.45:0/1060864824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0a5d90 con 0x7fb0f406bdb0 2024-04-25T10:30:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 -- 172.21.15.45:0/1060864824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f406bdb0 msgr2=0x7fb0f40ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/1060864824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f406bdb0 0x7fb0f40ccdb0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40b4270 tx=0x7fb0ec01c900 comp rx=0 tx=0).stop 2024-04-25T10:30:11.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 -- 172.21.15.45:0/1060864824 shutdown_connections 2024-04-25T10:30:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/1060864824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f406bdb0 0x7fb0f40ccdb0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/1060864824 >> v2:172.21.15.117:3300/0 conn(0x7fb0f415dc10 0x7fb0f40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 -- 172.21.15.45:0/1060864824 >> 172.21.15.45:0/1060864824 conn(0x7fb0f40d1750 msgr2=0x7fb0f40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 -- 172.21.15.45:0/1060864824 shutdown_connections 2024-04-25T10:30:11.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 -- 172.21.15.45:0/1060864824 wait complete. 2024-04-25T10:30:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 Processor -- start 2024-04-25T10:30:11.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.191+0000 7fb0f3e00640 1 -- start start 2024-04-25T10:30:11.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f406bdb0 0x7fb0f41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f415dc10 0x7fb0f414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 0x7fb0f4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40cdc80 con 0x7fb0f415dff0 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40cdb00 con 0x7fb0f406bdb0 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40cde00 con 0x7fb0f415dc10 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 0x7fb0f4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 0x7fb0f4154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39340/0 (socket says 172.21.15.45:39340) 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 -- 172.21.15.45:0/3982851952 learned_addr learned my addr 172.21.15.45:0/3982851952 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f2a00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f406bdb0 0x7fb0f41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 -- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f415dc10 msgr2=0x7fb0f414c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0ebe00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f415dc10 0x7fb0f414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f415dc10 0x7fb0f414c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 -- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f406bdb0 msgr2=0x7fb0f41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f406bdb0 0x7fb0f41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 -- 172.21.15.45:0/3982851952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40c8a70 con 0x7fb0f415dff0 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f2a00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f406bdb0 0x7fb0f41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0ebe00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f415dc10 0x7fb0f414c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3400640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 0x7fb0f4154550 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec019510 tx=0x7fb0ec0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0b2050 con 0x7fb0f415dff0 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec0be070 con 0x7fb0f415dff0 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0ada70 con 0x7fb0f415dff0 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f40c8d50 con 0x7fb0f415dff0 2024-04-25T10:30:11.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f414cc80 con 0x7fb0f415dff0 2024-04-25T10:30:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.195+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4075120 con 0x7fb0f415dff0 2024-04-25T10:30:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.199+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0ec0adc10 con 0x7fb0f415dff0 2024-04-25T10:30:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.199+0000 7fb0eaa00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.199+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb0ec120340 con 0x7fb0f415dff0 2024-04-25T10:30:11.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.199+0000 7fb0f2a00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.199+0000 7fb0f2a00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40097a0 tx=0x7fb0e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.203+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0ec016520 con 0x7fb0f415dff0 2024-04-25T10:30:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0f4068650 con 0x7fb0f415dff0 2024-04-25T10:30:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0eaa00640 1 -- 172.21.15.45:0/3982851952 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fb0ec0ab340 con 0x7fb0f415dff0 2024-04-25T10:30:11.401 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:11.401 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:29:26.603160+0000","modified":"2024-04-25T10:29:29.629298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9996":{"gid":9996,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/2482982951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2482982951},{"type":"v1","addr":"172.21.15.45:6835","nonce":2482982951}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:30:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-04-25T10:30:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 msgr2=0x7fb0d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40097a0 tx=0x7fb0e4008040 comp rx=0 tx=0).stop 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 msgr2=0x7fb0f4154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 0x7fb0f4154550 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec019510 tx=0x7fb0ec0196a0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 shutdown_connections 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb0d0066f60 0x7fb0d0069420 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0f415dff0 0x7fb0f4154550 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0f415dc10 0x7fb0f414c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 --2- 172.21.15.45:0/3982851952 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0f406bdb0 0x7fb0f41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 >> 172.21.15.45:0/3982851952 conn(0x7fb0f40d1750 msgr2=0x7fb0f40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 shutdown_connections 2024-04-25T10:30:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.399+0000 7fb0f3e00640 1 -- 172.21.15.45:0/3982851952 wait complete. 2024-04-25T10:30:11.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:11.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 Processor -- start 2024-04-25T10:30:11.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 -- start start 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fe815b910 0x7f4fe80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe8071050 con 0x7f4fe80cfe70 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe815bda0 con 0x7f4fe815b910 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fef600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe815d040 con 0x7f4fe81550b0 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fed600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fe815b910 0x7f4fe80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fe7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4fe80cfe70 0x7f4fe815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56380/0 (socket says 172.21.15.45:56380) 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.563+0000 7f4fe7e00640 1 -- 172.21.15.45:0/2134917201 learned_addr learned my addr 172.21.15.45:0/2134917201 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:11.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fecc00640 1 --2- 172.21.15.45:0/2134917201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fed600640 1 -- 172.21.15.45:0/2134917201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 msgr2=0x7f4fe80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fed600640 1 --2- 172.21.15.45:0/2134917201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fed600640 1 -- 172.21.15.45:0/2134917201 >> v1:172.21.15.45:6789/0 conn(0x7f4fe80cfe70 legacy=0x7f4fe815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fed600640 1 -- 172.21.15.45:0/2134917201 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80712f0 con 0x7f4fe815b910 2024-04-25T10:30:11.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fed600640 1 --2- 172.21.15.45:0/2134917201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe815b910 0x7f4fe80b3260 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009d90 tx=0x7f4fe001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80932c9a7a6001f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fe7400640 1 -- 172.21.15.45:0/2134917201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00b2070 con 0x7f4fe815b910 2024-04-25T10:30:11.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fe7400640 1 -- 172.21.15.45:0/2134917201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00a5920 con 0x7f4fe815b910 2024-04-25T10:30:11.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fe7400640 1 -- 172.21.15.45:0/2134917201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00a5cf0 con 0x7f4fe815b910 2024-04-25T10:30:11.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fef600640 1 -- 172.21.15.45:0/2134917201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe815b910 msgr2=0x7f4fe80b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.567+0000 7f4fef600640 1 --2- 172.21.15.45:0/2134917201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe815b910 0x7f4fe80b3260 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009d90 tx=0x7f4fe001c970 comp rx=0 tx=0).stop 2024-04-25T10:30:11.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- 172.21.15.45:0/2134917201 shutdown_connections 2024-04-25T10:30:11.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 --2- 172.21.15.45:0/2134917201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe815b910 0x7f4fe80b3260 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 --2- 172.21.15.45:0/2134917201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- 172.21.15.45:0/2134917201 >> 172.21.15.45:0/2134917201 conn(0x7f4fe8066b70 msgr2=0x7f4fe8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- 172.21.15.45:0/2134917201 shutdown_connections 2024-04-25T10:30:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- 172.21.15.45:0/2134917201 wait complete. 2024-04-25T10:30:11.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 Processor -- start 2024-04-25T10:30:11.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- start start 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 0x7f4fe8157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe815b910 0x7f4fe814ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe815c030 con 0x7f4fe80cfe70 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80720b0 con 0x7f4fe815b910 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe815c1b0 con 0x7f4fe81550b0 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 0x7f4fe8157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 0x7f4fe8157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39354/0 (socket says 172.21.15.45:39354) 2024-04-25T10:30:11.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 -- 172.21.15.45:0/2024752818 learned_addr learned my addr 172.21.15.45:0/2024752818 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 -- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 msgr2=0x7f4fe814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 -- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe815b910 msgr2=0x7f4fe814ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe815b910 0x7f4fe814ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 -- 172.21.15.45:0/2024752818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe814cbd0 con 0x7f4fe80cfe70 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fecc00640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 0x7f4fe8157290 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00d800 tx=0x7f4fdc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00b6c0 con 0x7f4fe80cfe70 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc019070 con 0x7f4fe80cfe70 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc00ba60 con 0x7f4fe80cfe70 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe8157a20 con 0x7f4fe80cfe70 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.571+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe8157fe0 con 0x7f4fe80cfe70 2024-04-25T10:30:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.575+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe8158320 con 0x7f4fe80cfe70 2024-04-25T10:30:11.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.575+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fdc00bc00 con 0x7f4fe80cfe70 2024-04-25T10:30:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.575+0000 7f4fe4c00640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fc8066f90 0x7f4fc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.575+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4fdc0854c0 con 0x7f4fe80cfe70 2024-04-25T10:30:11.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.575+0000 7f4fe7e00640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fc8066f90 0x7f4fc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.579+0000 7f4fe7e00640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fc8066f90 0x7f4fc8069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000f70 tx=0x7f4fd4007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.579+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fe8158320 con 0x7f4fe80cfe70 2024-04-25T10:30:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.743+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe804ef30 con 0x7f4fe80cfe70 2024-04-25T10:30:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.743+0000 7f4fe4c00640 1 -- 172.21.15.45:0/2024752818 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16717 (secure 0 0 0) 0x7f4fdc0564a0 con 0x7f4fe80cfe70 2024-04-25T10:30:11.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:11.747 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":157,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:29:44.707588+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:29:24.390187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T10:29:25.607844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.29","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fc8066f90 msgr2=0x7f4fc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fc8066f90 0x7f4fc8069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000f70 tx=0x7f4fd4007a90 comp rx=0 tx=0).stop 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 msgr2=0x7f4fe8157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 0x7f4fe8157290 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc00d800 tx=0x7f4fdc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 shutdown_connections 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4fc8066f90 0x7f4fc8069450 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe815b910 0x7f4fe814ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fe81550b0 0x7f4fe814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 --2- 172.21.15.45:0/2024752818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fe80cfe70 0x7f4fe8157290 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 >> 172.21.15.45:0/2024752818 conn(0x7f4fe8066b70 msgr2=0x7f4fe806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 shutdown_connections 2024-04-25T10:30:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.747+0000 7f4fef600640 1 -- 172.21.15.45:0/2024752818 wait complete. 2024-04-25T10:30:11.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd93ce00640 1 Processor -- start 2024-04-25T10:30:11.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd93ce00640 1 -- start start 2024-04-25T10:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd93ce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 0x7fd9380b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd93ce00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93806be90 con 0x7fd9380b1f50 2024-04-25T10:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd93ce00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93806b8d0 con 0x7fd9380b23c0 2024-04-25T10:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd93ce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380d0b50 con 0x7fd9380cde90 2024-04-25T10:30:11.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd937400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd9380b1f50 0x7fd93806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56382/0 (socket says 172.21.15.45:56382) 2024-04-25T10:30:11.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd937400640 1 -- 172.21.15.45:0/100862747 learned_addr learned my addr 172.21.15.45:0/100862747 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370284044 0 0) 0x7fd928001120 con 0x7fd9380b1f50 2024-04-25T10:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd937e00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 0x7fd9380b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd91c003530 con 0x7fd9380b1f50 2024-04-25T10:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766063291 0 0) 0x7fd91c003530 con 0x7fd9380b1f50 2024-04-25T10:30:11.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 msgr2=0x7fd9380b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 0x7fd9380b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 >> v1:172.21.15.117:6789/0 conn(0x7fd9380b23c0 legacy=0x7fd9380cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93806bbb0 con 0x7fd9380b1f50 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fd928001850 con 0x7fd9380b1f50 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 >> v1:172.21.15.45:6789/0 conn(0x7fd9380b1f50 legacy=0x7fd93806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 0x7fd91c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd91c004020 0x7fd91c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd91c004da0 0x7fd91c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c00bc80 con 0x7fd91c004020 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c00be30 con 0x7fd91c004da0 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c00c030 con 0x7fd9380cde90 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd92fe00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd91c004020 0x7fd91c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd937400640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 0x7fd91c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd937e00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd91c004da0 0x7fd91c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd92fe00640 1 -- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 msgr2=0x7fd91c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd92fe00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9380cde90 0x7fd91c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd92fe00640 1 -- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd91c004da0 msgr2=0x7fd91c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd92fe00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd91c004da0 0x7fd91c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.911+0000 7fd92fe00640 1 -- 172.21.15.45:0/100862747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c003530 con 0x7fd91c004020 2024-04-25T10:30:11.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd92fe00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd91c004020 0x7fd91c004810 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd920001750 tx=0x7fd92001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e23d142ea83109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200013a0 con 0x7fd91c004020 2024-04-25T10:30:11.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9200052f0 con 0x7fd91c004020 2024-04-25T10:30:11.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd936a00640 1 -- 172.21.15.45:0/100862747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920003840 con 0x7fd91c004020 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd93ce00640 1 -- 172.21.15.45:0/100862747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd91c004020 msgr2=0x7fd91c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd93ce00640 1 --2- 172.21.15.45:0/100862747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd91c004020 0x7fd91c004810 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd920001750 tx=0x7fd92001c3e0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd937e00640 1 -- 172.21.15.45:0/100862747 reap_dead start 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd93ce00640 1 -- 172.21.15.45:0/100862747 shutdown_connections 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd93ce00640 1 -- 172.21.15.45:0/100862747 >> 172.21.15.45:0/100862747 conn(0x7fd9380b32b0 msgr2=0x7fd9380b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd93ce00640 1 -- 172.21.15.45:0/100862747 shutdown_connections 2024-04-25T10:30:11.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.915+0000 7fd93ce00640 1 -- 172.21.15.45:0/100862747 wait complete. 2024-04-25T10:30:11.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 Processor -- start 2024-04-25T10:30:11.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 -- start start 2024-04-25T10:30:11.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd91c004020 0x7fd938155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd93815bbb0 0x7fd93815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 0x7fd9380afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380b41e0 con 0x7fd9380af610 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380d04d0 con 0x7fd93815bbb0 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd938169a10 con 0x7fd91c004020 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 0x7fd9380afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 0x7fd9380afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39374/0 (socket says 172.21.15.45:39374) 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 -- 172.21.15.45:0/3626682850 learned_addr learned my addr 172.21.15.45:0/3626682850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937400640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd91c004020 0x7fd938155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd92fe00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd93815bbb0 0x7fd93815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 -- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd91c004020 msgr2=0x7fd938155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd91c004020 0x7fd938155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 -- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd93815bbb0 msgr2=0x7fd93815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd93815bbb0 0x7fd93815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 -- 172.21.15.45:0/3626682850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380ae590 con 0x7fd9380af610 2024-04-25T10:30:11.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd937e00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 0x7fd9380afa90 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fd930007df0 tx=0x7fd930010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930010d90 con 0x7fd9380af610 2024-04-25T10:30:11.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93001c070 con 0x7fd9380af610 2024-04-25T10:30:11.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930017670 con 0x7fd9380af610 2024-04-25T10:30:11.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.919+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9380ae870 con 0x7fd9380af610 2024-04-25T10:30:11.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.923+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9380aee30 con 0x7fd9380af610 2024-04-25T10:30:11.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.923+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd938075120 con 0x7fd9380af610 2024-04-25T10:30:11.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.923+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd930017810 con 0x7fd9380af610 2024-04-25T10:30:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.923+0000 7fd92f400640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd914066f60 0x7fd914069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.923+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd930086b40 con 0x7fd9380af610 2024-04-25T10:30:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.923+0000 7fd937400640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd914066f60 0x7fd914069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:11.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.927+0000 7fd937400640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd914066f60 0x7fd914069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd928000c10 tx=0x7fd928009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:11.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:11.927+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd930026020 con 0x7fd9380af610 2024-04-25T10:30:12.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.123+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd9380b7890 con 0x7fd9380af610 2024-04-25T10:30:12.127 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:30:12.123+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:30:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.127+0000 7fd92f400640 1 -- 172.21.15.45:0/3626682850 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v116) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd930057b10 con 0x7fd9380af610 2024-04-25T10:30:12.129 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:30:12.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.127+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd914066f60 msgr2=0x7fd914069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.127+0000 7fd93ce00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd914066f60 0x7fd914069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd928000c10 tx=0x7fd928009040 comp rx=0 tx=0).stop 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.127+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 msgr2=0x7fd9380afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.127+0000 7fd93ce00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 0x7fd9380afa90 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fd930007df0 tx=0x7fd930010530 comp rx=0 tx=0).stop 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 shutdown_connections 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd914066f60 0x7fd914069420 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9380af610 0x7fd9380afa90 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd93815bbb0 0x7fd93815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 --2- 172.21.15.45:0/3626682850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd91c004020 0x7fd938155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 >> 172.21.15.45:0/3626682850 conn(0x7fd9380b32b0 msgr2=0x7fd93815f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 shutdown_connections 2024-04-25T10:30:12.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.131+0000 7fd93ce00640 1 -- 172.21.15.45:0/3626682850 wait complete. 2024-04-25T10:30:12.138 INFO:tasks.ceph.mds.a.smithi045.stderr: -3363> 2024-04-25T10:29:32.188+0000 7fd531400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:12.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:30:12.187 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:30:12.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 Processor -- start 2024-04-25T10:30:12.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 -- start start 2024-04-25T10:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e3006cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5e3015e2e0 0x7f5e300cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e300cf9d0 0x7f5e300712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3006de90 con 0x7f5e300cf9d0 2024-04-25T10:30:12.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3015aba0 con 0x7f5e3015e2e0 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e35200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3006d5f0 con 0x7f5e3015a7c0 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e2fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e300cf9d0 0x7f5e300712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e2fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e300cf9d0 0x7f5e300712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39410/0 (socket says 172.21.15.45:39410) 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.295+0000 7f5e2fe00640 1 -- 172.21.15.45:0/1040988094 learned_addr learned my addr 172.21.15.45:0/1040988094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e26a00640 1 --2- 172.21.15.45:0/1040988094 >> v2:172.21.15.117:3300/0 conn(0x7f5e3015e2e0 0x7f5e300cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2f400640 1 --2- 172.21.15.45:0/1040988094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e3006cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2fe00640 1 -- 172.21.15.45:0/1040988094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 msgr2=0x7f5e3006cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2fe00640 1 --2- 172.21.15.45:0/1040988094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e3006cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2fe00640 1 -- 172.21.15.45:0/1040988094 >> v2:172.21.15.117:3300/0 conn(0x7f5e3015e2e0 msgr2=0x7f5e300cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2fe00640 1 --2- 172.21.15.45:0/1040988094 >> v2:172.21.15.117:3300/0 conn(0x7f5e3015e2e0 0x7f5e300cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2fe00640 1 -- 172.21.15.45:0/1040988094 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3006d8d0 con 0x7f5e300cf9d0 2024-04-25T10:30:12.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2fe00640 1 --2- 172.21.15.45:0/1040988094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e300cf9d0 0x7f5e300712a0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f5e30067800 tx=0x7f5e2801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc02bd937fd0b94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:12.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2ea00640 1 -- 172.21.15.45:0/1040988094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280b0070 con 0x7f5e300cf9d0 2024-04-25T10:30:12.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2ea00640 1 -- 172.21.15.45:0/1040988094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e280ab600 con 0x7f5e300cf9d0 2024-04-25T10:30:12.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e2ea00640 1 -- 172.21.15.45:0/1040988094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280ab9d0 con 0x7f5e300cf9d0 2024-04-25T10:30:12.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 -- 172.21.15.45:0/1040988094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e300cf9d0 msgr2=0x7f5e300712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 --2- 172.21.15.45:0/1040988094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e300cf9d0 0x7f5e300712a0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f5e30067800 tx=0x7f5e2801c1d0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 -- 172.21.15.45:0/1040988094 shutdown_connections 2024-04-25T10:30:12.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 --2- 172.21.15.45:0/1040988094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e300cf9d0 0x7f5e300712a0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 --2- 172.21.15.45:0/1040988094 >> v2:172.21.15.117:3300/0 conn(0x7f5e3015e2e0 0x7f5e300cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 --2- 172.21.15.45:0/1040988094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e3006cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.299+0000 7f5e35200640 1 -- 172.21.15.45:0/1040988094 >> 172.21.15.45:0/1040988094 conn(0x7f5e300669a0 msgr2=0x7f5e30066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 -- 172.21.15.45:0/1040988094 shutdown_connections 2024-04-25T10:30:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 -- 172.21.15.45:0/1040988094 wait complete. 2024-04-25T10:30:12.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 Processor -- start 2024-04-25T10:30:12.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 -- start start 2024-04-25T10:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e300cf9d0 0x7f5e30159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e30146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 0x7f5e301590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300cc5e0 con 0x7f5e3015e2e0 2024-04-25T10:30:12.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300cc460 con 0x7f5e300cf9d0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e35200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300b2b20 con 0x7f5e3015a7c0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 0x7f5e301590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 0x7f5e301590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39416/0 (socket says 172.21.15.45:39416) 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 -- 172.21.15.45:0/3461765795 learned_addr learned my addr 172.21.15.45:0/3461765795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e26a00640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e30146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2f400640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e300cf9d0 0x7f5e30159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 -- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 msgr2=0x7f5e30146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e30146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 -- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e300cf9d0 msgr2=0x7f5e30159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e300cf9d0 0x7f5e30159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 -- 172.21.15.45:0/3461765795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e301474f0 con 0x7f5e3015e2e0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2f400640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e300cf9d0 0x7f5e30159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e2fe00640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 0x7f5e301590c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ae040 tx=0x7f5e28018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:12.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.303+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280b0040 con 0x7f5e3015e2e0 2024-04-25T10:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e280b9070 con 0x7f5e3015e2e0 2024-04-25T10:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e30159600 con 0x7f5e3015e2e0 2024-04-25T10:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280b4670 con 0x7f5e3015e2e0 2024-04-25T10:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3015b250 con 0x7f5e3015e2e0 2024-04-25T10:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e30075120 con 0x7f5e3015e2e0 2024-04-25T10:30:12.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e280b4810 con 0x7f5e3015e2e0 2024-04-25T10:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e27e00640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e0c066f60 0x7f5e0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f5e2811f430 con 0x7f5e3015e2e0 2024-04-25T10:30:12.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.307+0000 7f5e2f400640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e0c066f60 0x7f5e0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:12.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.311+0000 7f5e2f400640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e0c066f60 0x7f5e0c069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003fd0 tx=0x7f5e200039e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:12.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.311+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e28016110 con 0x7f5e3015e2e0 2024-04-25T10:30:12.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:12.507+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5e300afb80 con 0x7f5e3015e2e0 2024-04-25T10:30:13.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.127+0000 7f5e27e00640 1 -- 172.21.15.45:0/3461765795 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v117) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5e280f0460 con 0x7f5e3015e2e0 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e0c066f60 msgr2=0x7f5e0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e0c066f60 0x7f5e0c069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5e20003fd0 tx=0x7f5e200039e0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 msgr2=0x7f5e301590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 0x7f5e301590c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ae040 tx=0x7f5e28018f70 comp rx=0 tx=0).stop 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 shutdown_connections 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5e0c066f60 0x7f5e0c069420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e3015e2e0 0x7f5e301590c0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e3015a7c0 0x7f5e30146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 --2- 172.21.15.45:0/3461765795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e300cf9d0 0x7f5e30159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 >> 172.21.15.45:0/3461765795 conn(0x7f5e300669a0 msgr2=0x7f5e300b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 shutdown_connections 2024-04-25T10:30:13.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.131+0000 7f5e35200640 1 -- 172.21.15.45:0/3461765795 wait complete. 2024-04-25T10:30:13.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:30:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 Processor -- start 2024-04-25T10:30:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 -- start start 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f26b806bd30 0x7f26b80b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26b80ceb20 0x7f26b80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b806c2d0 con 0x7f26b80ce030 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80cbe00 con 0x7f26b806bd30 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26bca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b806c890 con 0x7f26b80ceb20 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f26b806bd30 0x7f26b80b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f26b806bd30 0x7f26b80b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39418/0 (socket says 172.21.15.45:39418) 2024-04-25T10:30:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 -- 172.21.15.45:0/2128762321 learned_addr learned my addr 172.21.15.45:0/2128762321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26b7e00640 1 --2- 172.21.15.45:0/2128762321 >> v2:172.21.15.117:3300/0 conn(0x7f26b80ceb20 0x7f26b80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 -- 172.21.15.45:0/2128762321 >> v2:172.21.15.117:3300/0 conn(0x7f26b80ceb20 msgr2=0x7f26b80b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 --2- 172.21.15.45:0/2128762321 >> v2:172.21.15.117:3300/0 conn(0x7f26b80ceb20 0x7f26b80b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 -- 172.21.15.45:0/2128762321 >> v1:172.21.15.117:6789/0 conn(0x7f26b80ce030 legacy=0x7f26b80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.347+0000 7f26afe00640 1 -- 172.21.15.45:0/2128762321 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b806c5b0 con 0x7f26b806bd30 2024-04-25T10:30:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26afe00640 1 --2- 172.21.15.45:0/2128762321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b80b1910 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f26b809b080 tx=0x7f26a001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c6cfbb33c7c44c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26b6a00640 1 -- 172.21.15.45:0/2128762321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00b0070 con 0x7f26b806bd30 2024-04-25T10:30:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26b6a00640 1 -- 172.21.15.45:0/2128762321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a00ab600 con 0x7f26b806bd30 2024-04-25T10:30:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26b6a00640 1 -- 172.21.15.45:0/2128762321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00ab9d0 con 0x7f26b806bd30 2024-04-25T10:30:13.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 -- 172.21.15.45:0/2128762321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 msgr2=0x7f26b80b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 --2- 172.21.15.45:0/2128762321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b80b1910 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f26b809b080 tx=0x7f26a001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 -- 172.21.15.45:0/2128762321 shutdown_connections 2024-04-25T10:30:13.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 --2- 172.21.15.45:0/2128762321 >> v2:172.21.15.117:3300/0 conn(0x7f26b80ceb20 0x7f26b80b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 --2- 172.21.15.45:0/2128762321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b80b1910 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 -- 172.21.15.45:0/2128762321 >> 172.21.15.45:0/2128762321 conn(0x7f26b809a9d0 msgr2=0x7f26b80d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:13.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 -- 172.21.15.45:0/2128762321 shutdown_connections 2024-04-25T10:30:13.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 -- 172.21.15.45:0/2128762321 wait complete. 2024-04-25T10:30:13.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 Processor -- start 2024-04-25T10:30:13.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.351+0000 7f26bca00640 1 -- start start 2024-04-25T10:30:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26b80ce030 0x7f26b806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26b80ceb20 0x7f26b806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80cdc30 con 0x7f26b806bd30 2024-04-25T10:30:13.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80cdab0 con 0x7f26b80ceb20 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b816a8a0 con 0x7f26b80ce030 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39434/0 (socket says 172.21.15.45:39434) 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 -- 172.21.15.45:0/3988687253 learned_addr learned my addr 172.21.15.45:0/3988687253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7e00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26b80ceb20 0x7f26b806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26afe00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26b80ce030 0x7f26b806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 -- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26b80ce030 msgr2=0x7f26b806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26b80ce030 0x7f26b806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 -- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26b80ceb20 msgr2=0x7f26b806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26b80ceb20 0x7f26b806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:13.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 -- 172.21.15.45:0/3988687253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80b4350 con 0x7f26b806bd30 2024-04-25T10:30:13.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26b7400640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b8159620 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c430 tx=0x7f26a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8011790 con 0x7f26b806bd30 2024-04-25T10:30:13.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8017070 con 0x7f26b806bd30 2024-04-25T10:30:13.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8011bb0 con 0x7f26b806bd30 2024-04-25T10:30:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b80b4630 con 0x7f26b806bd30 2024-04-25T10:30:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b80b4bf0 con 0x7f26b806bd30 2024-04-25T10:30:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b806e4e0 con 0x7f26b806bd30 2024-04-25T10:30:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.355+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26a8012420 con 0x7f26b806bd30 2024-04-25T10:30:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.359+0000 7f26af400640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2694066f60 0x7f2694069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.359+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f26a8084a90 con 0x7f26b806bd30 2024-04-25T10:30:13.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.359+0000 7f26afe00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2694066f60 0x7f2694069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:13.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.359+0000 7f26afe00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2694066f60 0x7f2694069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f26a00ad000 tx=0x7f26a00a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.363+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26a8055360 con 0x7f26b806bd30 2024-04-25T10:30:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:13.523+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f26b80687c0 con 0x7f26b806bd30 2024-04-25T10:30:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.131+0000 7f26af400640 1 -- 172.21.15.45:0/3988687253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v159) v1 ==== 181+0+0 (secure 0 0 0) 0x7f26a8059400 con 0x7f26b806bd30 2024-04-25T10:30:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:30:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2694066f60 msgr2=0x7f2694069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2694066f60 0x7f2694069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f26a00ad000 tx=0x7f26a00a6000 comp rx=0 tx=0).stop 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 msgr2=0x7f26b8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b8159620 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c430 tx=0x7f26a800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 shutdown_connections 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2694066f60 0x7f2694069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26b80ceb20 0x7f26b806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26b80ce030 0x7f26b806e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 --2- 172.21.15.45:0/3988687253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26b806bd30 0x7f26b8159620 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 >> 172.21.15.45:0/3988687253 conn(0x7f26b809a9d0 msgr2=0x7f26b80d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 shutdown_connections 2024-04-25T10:30:14.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.135+0000 7f26bca00640 1 -- 172.21.15.45:0/3988687253 wait complete. 2024-04-25T10:30:14.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:30:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 Processor -- start 2024-04-25T10:30:14.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 -- start start 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ad00cf040 0x7f1ad00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ad015bc00 0x7f1ad00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00713a0 0x7f1ad0071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad0072260 con 0x7f1ad00cf040 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00b3700 con 0x7f1ad015bc00 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ad6600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad0071d40 con 0x7f1ad00713a0 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ad00cf040 0x7f1ad00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1ad00cf040 0x7f1ad00cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39446/0 (socket says 172.21.15.45:39446) 2024-04-25T10:30:14.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 -- 172.21.15.45:0/1757798186 learned_addr learned my addr 172.21.15.45:0/1757798186 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/1757798186 >> v2:172.21.15.117:3300/0 conn(0x7f1ad015bc00 0x7f1ad00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acfe00640 1 --2- 172.21.15.45:0/1757798186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00713a0 0x7f1ad0071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 -- 172.21.15.45:0/1757798186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00713a0 msgr2=0x7f1ad0071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 --2- 172.21.15.45:0/1757798186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00713a0 0x7f1ad0071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 -- 172.21.15.45:0/1757798186 >> v2:172.21.15.117:3300/0 conn(0x7f1ad015bc00 msgr2=0x7f1ad00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 --2- 172.21.15.45:0/1757798186 >> v2:172.21.15.117:3300/0 conn(0x7f1ad015bc00 0x7f1ad00cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.307+0000 7f1acf400640 1 -- 172.21.15.45:0/1757798186 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0071f80 con 0x7f1ad00cf040 2024-04-25T10:30:14.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1acf400640 1 --2- 172.21.15.45:0/1757798186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad00cf040 0x7f1ad00cf9e0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f1ad006e840 tx=0x7f1ac001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bee7c245c78992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1acea00640 1 -- 172.21.15.45:0/1757798186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b1070 con 0x7f1ad00cf040 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1acea00640 1 -- 172.21.15.45:0/1757798186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00ac630 con 0x7f1ad00cf040 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1acea00640 1 -- 172.21.15.45:0/1757798186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a8050 con 0x7f1ad00cf040 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 -- 172.21.15.45:0/1757798186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad00cf040 msgr2=0x7f1ad00cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 --2- 172.21.15.45:0/1757798186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad00cf040 0x7f1ad00cf9e0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f1ad006e840 tx=0x7f1ac001c520 comp rx=0 tx=0).stop 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 -- 172.21.15.45:0/1757798186 shutdown_connections 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 --2- 172.21.15.45:0/1757798186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00713a0 0x7f1ad0071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 --2- 172.21.15.45:0/1757798186 >> v2:172.21.15.117:3300/0 conn(0x7f1ad015bc00 0x7f1ad00cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 --2- 172.21.15.45:0/1757798186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad00cf040 0x7f1ad00cf9e0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 -- 172.21.15.45:0/1757798186 >> 172.21.15.45:0/1757798186 conn(0x7f1ad0066b70 msgr2=0x7f1ad0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:14.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 -- 172.21.15.45:0/1757798186 shutdown_connections 2024-04-25T10:30:14.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.311+0000 7f1ad6600640 1 -- 172.21.15.45:0/1757798186 wait complete. 2024-04-25T10:30:14.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 Processor -- start 2024-04-25T10:30:14.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- start start 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad00713a0 0x7f1ad014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00cf040 0x7f1ad01f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 0x7f1ad00cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00cf5a0 con 0x7f1ad015bc00 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00cf420 con 0x7f1ad00713a0 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00aec90 con 0x7f1ad00cf040 2024-04-25T10:30:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 0x7f1ad00cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 0x7f1ad00cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39454/0 (socket says 172.21.15.45:39454) 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 -- 172.21.15.45:0/376162482 learned_addr learned my addr 172.21.15.45:0/376162482 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acf400640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad00713a0 0x7f1ad014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ac6a00640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00cf040 0x7f1ad01f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 -- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00cf040 msgr2=0x7f1ad01f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00cf040 0x7f1ad01f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 -- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad00713a0 msgr2=0x7f1ad014de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad00713a0 0x7f1ad014de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 -- 172.21.15.45:0/376162482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00b4c70 con 0x7f1ad015bc00 2024-04-25T10:30:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1acfe00640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 0x7f1ad00cd800 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800ef00 tx=0x7f1ac800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8018020 con 0x7f1ad015bc00 2024-04-25T10:30:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac800bd20 con 0x7f1ad015bc00 2024-04-25T10:30:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad01f71a0 con 0x7f1ad015bc00 2024-04-25T10:30:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8014640 con 0x7f1ad015bc00 2024-04-25T10:30:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad01526d0 con 0x7f1ad015bc00 2024-04-25T10:30:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.315+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad004ef30 con 0x7f1ad015bc00 2024-04-25T10:30:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.319+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ac801b020 con 0x7f1ad015bc00 2024-04-25T10:30:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.319+0000 7f1ac7e00640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aac066f60 0x7f1aac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:14.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.319+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f1ac80853e0 con 0x7f1ad015bc00 2024-04-25T10:30:14.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.319+0000 7f1acf400640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aac066f60 0x7f1aac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:14.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.319+0000 7f1acf400640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aac066f60 0x7f1aac069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00ae000 tx=0x7f1ac0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.323+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ac8022070 con 0x7f1ad015bc00 2024-04-25T10:30:14.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:14.487+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ad00b2a10 con 0x7f1ad015bc00 2024-04-25T10:30:15.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.131+0000 7f1ac7e00640 1 -- 172.21.15.45:0/376162482 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v160) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1ac8055e40 con 0x7f1ad015bc00 2024-04-25T10:30:15.135 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:30:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.135+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aac066f60 msgr2=0x7f1aac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.135+0000 7f1ad6600640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aac066f60 0x7f1aac069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00ae000 tx=0x7f1ac0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.135+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 msgr2=0x7f1ad00cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.135+0000 7f1ad6600640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 0x7f1ad00cd800 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800ef00 tx=0x7f1ac800b480 comp rx=0 tx=0).stop 2024-04-25T10:30:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 shutdown_connections 2024-04-25T10:30:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1aac066f60 0x7f1aac069420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ad015bc00 0x7f1ad00cd800 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ad00cf040 0x7f1ad01f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 --2- 172.21.15.45:0/376162482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad00713a0 0x7f1ad014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 >> 172.21.15.45:0/376162482 conn(0x7f1ad0066b70 msgr2=0x7f1ad01510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 shutdown_connections 2024-04-25T10:30:15.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.139+0000 7f1ad6600640 1 -- 172.21.15.45:0/376162482 wait complete. 2024-04-25T10:30:15.167 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:30:15.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.311+0000 7fb5e3a00640 1 Processor -- start 2024-04-25T10:30:15.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.311+0000 7fb5e3a00640 1 -- start start 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e3a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb5dc159540 0x7fb5dc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e3a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5dc0721f0 0x7fb5dc070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e3a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc159920 con 0x7fb5dc0aebd0 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e3a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5dc15c2f0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e3a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5dc15c5d0 con 0x7fb5dc159540 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5dc0721f0 0x7fb5dc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb5dc0721f0 0x7fb5dc070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36750/0 (socket says 172.21.15.45:36750) 2024-04-25T10:30:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 -- 172.21.15.45:0/1477299519 learned_addr learned my addr 172.21.15.45:0/1477299519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1000640 1 --2- 172.21.15.45:0/1477299519 >> v2:172.21.15.117:3300/0 conn(0x7fb5dc159540 0x7fb5dc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 -- 172.21.15.45:0/1477299519 >> v2:172.21.15.117:3300/0 conn(0x7fb5dc159540 msgr2=0x7fb5dc0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 --2- 172.21.15.45:0/1477299519 >> v2:172.21.15.117:3300/0 conn(0x7fb5dc159540 0x7fb5dc0ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 -- 172.21.15.45:0/1477299519 >> v1:172.21.15.45:6789/0 conn(0x7fb5dc0aebd0 legacy=0x7fb5dc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 -- 172.21.15.45:0/1477299519 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc15c8b0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.315+0000 7fb5e1a00640 1 --2- 172.21.15.45:0/1477299519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc070350 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0b4c20 tx=0x7fb5d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d736e291cbd1a173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5dbe00640 1 -- 172.21.15.45:0/1477299519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40a57d0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5dbe00640 1 -- 172.21.15.45:0/1477299519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d40a5970 con 0x7fb5dc0721f0 2024-04-25T10:30:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5dbe00640 1 -- 172.21.15.45:0/1477299519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d40a5d40 con 0x7fb5dc0721f0 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- 172.21.15.45:0/1477299519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 msgr2=0x7fb5dc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/1477299519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc070350 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc0b4c20 tx=0x7fb5d401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- 172.21.15.45:0/1477299519 shutdown_connections 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/1477299519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc070350 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/1477299519 >> v2:172.21.15.117:3300/0 conn(0x7fb5dc159540 0x7fb5dc0ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- 172.21.15.45:0/1477299519 >> 172.21.15.45:0/1477299519 conn(0x7fb5dc0669a0 msgr2=0x7fb5dc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- 172.21.15.45:0/1477299519 shutdown_connections 2024-04-25T10:30:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- 172.21.15.45:0/1477299519 wait complete. 2024-04-25T10:30:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 Processor -- start 2024-04-25T10:30:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- start start 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc166190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5dc0aebd0 0x7fb5dc1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5dc159540 0x7fb5dc152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc071220 con 0x7fb5dc0721f0 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0710a0 con 0x7fb5dc159540 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e3a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5dc0713a0 con 0x7fb5dc0aebd0 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc166190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36752/0 (socket says 172.21.15.45:36752) 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.319+0000 7fb5e1000640 1 -- 172.21.15.45:0/2742244905 learned_addr learned my addr 172.21.15.45:0/2742244905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5dc159540 0x7fb5dc152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d3e00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5dc0aebd0 0x7fb5dc1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1000640 1 -- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5dc0aebd0 msgr2=0x7fb5dc1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1000640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5dc0aebd0 0x7fb5dc1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1000640 1 -- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5dc159540 msgr2=0x7fb5dc152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1000640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5dc159540 0x7fb5dc152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1000640 1 -- 172.21.15.45:0/2742244905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc154130 con 0x7fb5dc0721f0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d3e00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5dc0aebd0 0x7fb5dc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e1000640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc166190 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00d5f0 tx=0x7fb5cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0125f0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc019070 con 0x7fb5dc0721f0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc154410 con 0x7fb5dc0721f0 2024-04-25T10:30:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0129b0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc1549d0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5dc04ef30 con 0x7fb5dc0721f0 2024-04-25T10:30:15.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5cc012b50 con 0x7fb5dc0721f0 2024-04-25T10:30:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d9600640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5b8066f60 0x7fb5b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.323+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb5cc085030 con 0x7fb5dc0721f0 2024-04-25T10:30:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.327+0000 7fb5d3e00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5b8066f60 0x7fb5b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.327+0000 7fb5d3e00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5b8066f60 0x7fb5b8069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc155320 tx=0x7fb5c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.327+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5cc0563b0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.495+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5dc06d7f0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.499+0000 7fb5d9600640 1 -- 172.21.15.45:0/2742244905 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5cc055cb0 con 0x7fb5dc0721f0 2024-04-25T10:30:15.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5b8066f60 msgr2=0x7fb5b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5b8066f60 0x7fb5b8069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc155320 tx=0x7fb5c4009040 comp rx=0 tx=0).stop 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 msgr2=0x7fb5dc166190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc166190 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00d5f0 tx=0x7fb5cc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 shutdown_connections 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb5b8066f60 0x7fb5b8069420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5dc159540 0x7fb5dc152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5dc0aebd0 0x7fb5dc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 --2- 172.21.15.45:0/2742244905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5dc0721f0 0x7fb5dc166190 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 >> 172.21.15.45:0/2742244905 conn(0x7fb5dc0669a0 msgr2=0x7fb5dc15fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 shutdown_connections 2024-04-25T10:30:15.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.503+0000 7fb5e3a00640 1 -- 172.21.15.45:0/2742244905 wait complete. 2024-04-25T10:30:15.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:30:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.659+0000 7f3e59e00640 1 Processor -- start 2024-04-25T10:30:15.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e59e00640 1 -- start start 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e59e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e541550b0 0x7f3e540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e59e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e540cfe70 0x7f3e5415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e59e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54071050 con 0x7f3e5415b740 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e59e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5415d040 con 0x7f3e541550b0 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e59e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540712f0 con 0x7f3e540cfe70 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e541550b0 0x7f3e540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e541550b0 0x7f3e540cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36756/0 (socket says 172.21.15.45:36756) 2024-04-25T10:30:15.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 -- 172.21.15.45:0/810918571 learned_addr learned my addr 172.21.15.45:0/810918571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e4a000640 1 --2- 172.21.15.45:0/810918571 >> v2:172.21.15.117:3300/0 conn(0x7f3e540cfe70 0x7f3e5415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 -- 172.21.15.45:0/810918571 >> v2:172.21.15.117:3300/0 conn(0x7f3e540cfe70 msgr2=0x7f3e5415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 --2- 172.21.15.45:0/810918571 >> v2:172.21.15.117:3300/0 conn(0x7f3e540cfe70 0x7f3e5415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 -- 172.21.15.45:0/810918571 >> v1:172.21.15.117:6789/0 conn(0x7f3e5415b740 legacy=0x7f3e5406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.663+0000 7f3e52a00640 1 -- 172.21.15.45:0/810918571 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540715d0 con 0x7f3e541550b0 2024-04-25T10:30:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e52a00640 1 --2- 172.21.15.45:0/810918571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541550b0 0x7f3e540cf930 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009870 tx=0x7f3e4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ef3828b8e5a880 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e52000640 1 -- 172.21.15.45:0/810918571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b0070 con 0x7f3e541550b0 2024-04-25T10:30:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e52000640 1 -- 172.21.15.45:0/810918571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e440ab600 con 0x7f3e541550b0 2024-04-25T10:30:15.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e52000640 1 -- 172.21.15.45:0/810918571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440ab9d0 con 0x7f3e541550b0 2024-04-25T10:30:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 -- 172.21.15.45:0/810918571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541550b0 msgr2=0x7f3e540cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 --2- 172.21.15.45:0/810918571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541550b0 0x7f3e540cf930 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009870 tx=0x7f3e4401c520 comp rx=0 tx=0).stop 2024-04-25T10:30:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 -- 172.21.15.45:0/810918571 shutdown_connections 2024-04-25T10:30:15.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 --2- 172.21.15.45:0/810918571 >> v2:172.21.15.117:3300/0 conn(0x7f3e540cfe70 0x7f3e5415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 --2- 172.21.15.45:0/810918571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e541550b0 0x7f3e540cf930 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 -- 172.21.15.45:0/810918571 >> 172.21.15.45:0/810918571 conn(0x7f3e54066b70 msgr2=0x7f3e54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 -- 172.21.15.45:0/810918571 shutdown_connections 2024-04-25T10:30:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 -- 172.21.15.45:0/810918571 wait complete. 2024-04-25T10:30:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 Processor -- start 2024-04-25T10:30:15.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.667+0000 7f3e59e00640 1 -- start start 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e540cfe70 0x7f3e540cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e541550b0 0x7f3e54156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 0x7f3e5414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415bcd0 con 0x7f3e5415b740 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415bb50 con 0x7f3e541550b0 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415c030 con 0x7f3e540cfe70 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 0x7f3e5414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 0x7f3e5414a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36762/0 (socket says 172.21.15.45:36762) 2024-04-25T10:30:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 -- 172.21.15.45:0/1150871744 learned_addr learned my addr 172.21.15.45:0/1150871744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e4a000640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e541550b0 0x7f3e54156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e52a00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e540cfe70 0x7f3e540cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 -- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e540cfe70 msgr2=0x7f3e540cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e540cfe70 0x7f3e540cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 -- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e541550b0 msgr2=0x7f3e54156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e541550b0 0x7f3e54156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 -- 172.21.15.45:0/1150871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54157430 con 0x7f3e5415b740 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e53400640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 0x7f3e5414a250 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c760 tx=0x7f3e4c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c003b80 con 0x7f3e5415b740 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c003d20 con 0x7f3e5415b740 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e541568b0 con 0x7f3e5415b740 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00aa30 con 0x7f3e5415b740 2024-04-25T10:30:15.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e54156e70 con 0x7f3e5415b740 2024-04-25T10:30:15.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5404ef30 con 0x7f3e5415b740 2024-04-25T10:30:15.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.671+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e4c00abd0 con 0x7f3e5415b740 2024-04-25T10:30:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.675+0000 7f3e4be00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e30066f60 0x7f3e30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.675+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e4c084ac0 con 0x7f3e5415b740 2024-04-25T10:30:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.675+0000 7f3e52a00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e30066f60 0x7f3e30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:15.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.675+0000 7f3e52a00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e30066f60 0x7f3e30069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3e440ad000 tx=0x7f3e440a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.679+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e4c055770 con 0x7f3e5415b740 2024-04-25T10:30:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:15.899+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f3e54158aa0 con 0x7f3e5415b740 2024-04-25T10:30:16.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.135+0000 7f3e4be00640 1 -- 172.21.15.45:0/1150871744 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v591) v1 ==== 140+0+0 (secure 0 0 0) 0x7f3e4c059810 con 0x7f3e5415b740 2024-04-25T10:30:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.135+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e30066f60 msgr2=0x7f3e30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.135+0000 7f3e59e00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e30066f60 0x7f3e30069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3e440ad000 tx=0x7f3e440a6000 comp rx=0 tx=0).stop 2024-04-25T10:30:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.135+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 msgr2=0x7f3e5414a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.135+0000 7f3e59e00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 0x7f3e5414a250 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c760 tx=0x7f3e4c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 shutdown_connections 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e30066f60 0x7f3e30069420 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e5415b740 0x7f3e5414a250 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e541550b0 0x7f3e54156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 --2- 172.21.15.45:0/1150871744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e540cfe70 0x7f3e540cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 >> 172.21.15.45:0/1150871744 conn(0x7f3e54066b70 msgr2=0x7f3e5415d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 shutdown_connections 2024-04-25T10:30:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.139+0000 7f3e59e00640 1 -- 172.21.15.45:0/1150871744 wait complete. 2024-04-25T10:30:16.160 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:30:16.161 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:30:16.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.307+0000 7fe656c00640 1 Processor -- start 2024-04-25T10:30:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe656c00640 1 -- start start 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe656c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6500cda10 0x7fe65006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe656c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6500cf930 0x7fe6500cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe656c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6500b2770 con 0x7fe65015bc40 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe656c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65006c670 con 0x7fe6500cda10 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe656c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65006c950 con 0x7fe6500cf930 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6500cda10 0x7fe65006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6500cda10 0x7fe65006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36772/0 (socket says 172.21.15.45:36772) 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 -- 172.21.15.45:0/940969144 learned_addr learned my addr 172.21.15.45:0/940969144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe654c00640 1 --2- 172.21.15.45:0/940969144 >> v2:172.21.15.117:3300/0 conn(0x7fe6500cf930 0x7fe6500cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 -- 172.21.15.45:0/940969144 >> v2:172.21.15.117:3300/0 conn(0x7fe6500cf930 msgr2=0x7fe6500cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 --2- 172.21.15.45:0/940969144 >> v2:172.21.15.117:3300/0 conn(0x7fe6500cf930 0x7fe6500cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 -- 172.21.15.45:0/940969144 >> v1:172.21.15.45:6789/0 conn(0x7fe65015bc40 legacy=0x7fe6500b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe647400640 1 -- 172.21.15.45:0/940969144 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65006cc30 con 0x7fe6500cda10 2024-04-25T10:30:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.311+0000 7fe64f400640 1 -- 172.21.15.45:0/940969144 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642198458 0 0) 0x7fe6500b2770 con 0x7fe65015bc40 2024-04-25T10:30:16.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe647400640 1 --2- 172.21.15.45:0/940969144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cda10 0x7fe65006c090 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fe6500ae4c0 tx=0x7fe63801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63b7d3e7e4ceddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe64f400640 1 -- 172.21.15.45:0/940969144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b0070 con 0x7fe6500cda10 2024-04-25T10:30:16.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe64f400640 1 -- 172.21.15.45:0/940969144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6380ab600 con 0x7fe6500cda10 2024-04-25T10:30:16.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe64f400640 1 -- 172.21.15.45:0/940969144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380ab9d0 con 0x7fe6500cda10 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 -- 172.21.15.45:0/940969144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cda10 msgr2=0x7fe65006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 --2- 172.21.15.45:0/940969144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cda10 0x7fe65006c090 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fe6500ae4c0 tx=0x7fe63801c490 comp rx=0 tx=0).stop 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 -- 172.21.15.45:0/940969144 shutdown_connections 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 --2- 172.21.15.45:0/940969144 >> v2:172.21.15.117:3300/0 conn(0x7fe6500cf930 0x7fe6500cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 --2- 172.21.15.45:0/940969144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cda10 0x7fe65006c090 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 -- 172.21.15.45:0/940969144 >> 172.21.15.45:0/940969144 conn(0x7fe650066b70 msgr2=0x7fe650066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 -- 172.21.15.45:0/940969144 shutdown_connections 2024-04-25T10:30:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 -- 172.21.15.45:0/940969144 wait complete. 2024-04-25T10:30:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 Processor -- start 2024-04-25T10:30:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.315+0000 7fe656c00640 1 -- start start 2024-04-25T10:30:16.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6500cda10 0x7fe65015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 0x7fe650155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe65015bc40 0x7fe65015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6500b3850 con 0x7fe6500cf930 2024-04-25T10:30:16.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6500b36d0 con 0x7fe65015bc40 2024-04-25T10:30:16.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6500b39d0 con 0x7fe6500cda10 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 0x7fe650155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 0x7fe650155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36786/0 (socket says 172.21.15.45:36786) 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 -- 172.21.15.45:0/4162826974 learned_addr learned my addr 172.21.15.45:0/4162826974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe654c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe65015bc40 0x7fe65015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe64fe00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6500cda10 0x7fe65015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 -- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6500cda10 msgr2=0x7fe65015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6500cda10 0x7fe65015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 -- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe65015bc40 msgr2=0x7fe65015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe65015bc40 0x7fe65015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 -- 172.21.15.45:0/4162826974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65015dc20 con 0x7fe6500cf930 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe64fe00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6500cda10 0x7fe65015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe647400640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 0x7fe650155150 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fe6380190a0 tx=0x7fe638019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b0040 con 0x7fe6500cf930 2024-04-25T10:30:16.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6380b9070 con 0x7fe6500cf930 2024-04-25T10:30:16.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380b38a0 con 0x7fe6500cf930 2024-04-25T10:30:16.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65015df00 con 0x7fe6500cf930 2024-04-25T10:30:16.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65015e4c0 con 0x7fe6500cf930 2024-04-25T10:30:16.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.319+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65004ef30 con 0x7fe6500cf930 2024-04-25T10:30:16.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.323+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6380b3a40 con 0x7fe6500cf930 2024-04-25T10:30:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.323+0000 7fe64cc00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe62c066f60 0x7fe62c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.323+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe63811eb10 con 0x7fe6500cf930 2024-04-25T10:30:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.323+0000 7fe64fe00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe62c066f60 0x7fe62c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.323+0000 7fe64fe00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe62c066f60 0x7fe62c069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe640000f70 tx=0x7fe640009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.327+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe638016110 con 0x7fe6500cf930 2024-04-25T10:30:16.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.491+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe65006d960 con 0x7fe6500cf930 2024-04-25T10:30:16.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.491+0000 7fe64cc00640 1 -- 172.21.15.45:0/4162826974 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+13018 (secure 0 0 0) 0x7fe6380efea0 con 0x7fe6500cf930 2024-04-25T10:30:16.494 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:16.494 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":160,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:15.134645+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2482982951":"2024-04-26T10:30:12.128540+0000","172.21.15.45:6833/2482982951":"2024-04-26T10:30:12.128540+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe62c066f60 msgr2=0x7fe62c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe62c066f60 0x7fe62c069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe640000f70 tx=0x7fe640009040 comp rx=0 tx=0).stop 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 msgr2=0x7fe650155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 0x7fe650155150 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fe6380190a0 tx=0x7fe638019230 comp rx=0 tx=0).stop 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 shutdown_connections 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe62c066f60 0x7fe62c069420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe65015bc40 0x7fe65015d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6500cf930 0x7fe650155150 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 --2- 172.21.15.45:0/4162826974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6500cda10 0x7fe65015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 >> 172.21.15.45:0/4162826974 conn(0x7fe650066b70 msgr2=0x7fe650066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 shutdown_connections 2024-04-25T10:30:16.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.495+0000 7fe656c00640 1 -- 172.21.15.45:0/4162826974 wait complete. 2024-04-25T10:30:16.515 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:16.515 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:16.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.655+0000 7f2742c00640 1 Processor -- start 2024-04-25T10:30:16.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.655+0000 7f2742c00640 1 -- start start 2024-04-25T10:30:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2742c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0d0d00 0x7f273c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2742c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f273c0cc200 0x7f273c162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2742c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c0cdab0 con 0x7f273c1626c0 2024-04-25T10:30:16.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2742c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c0cd210 con 0x7f273c0cc200 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2742c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0cd4f0 con 0x7f273c0d0d00 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f273c0cc200 0x7f273c162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f273c0cc200 0x7f273c162180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36788/0 (socket says 172.21.15.45:36788) 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 -- 172.21.15.45:0/468585330 learned_addr learned my addr 172.21.15.45:0/468585330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f273be00640 1 --2- 172.21.15.45:0/468585330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0d0d00 0x7f273c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 -- 172.21.15.45:0/468585330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0d0d00 msgr2=0x7f273c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:16.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 --2- 172.21.15.45:0/468585330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0d0d00 0x7f273c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 -- 172.21.15.45:0/468585330 >> v1:172.21.15.117:6789/0 conn(0x7f273c1626c0 legacy=0x7f273c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.659+0000 7f2733400640 1 -- 172.21.15.45:0/468585330 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0cd7d0 con 0x7f273c0cc200 2024-04-25T10:30:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2733400640 1 --2- 172.21.15.45:0/468585330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c0cc200 0x7f273c162180 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f2724009870 tx=0x7f272401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ed2f89b5e4baa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f273b400640 1 -- 172.21.15.45:0/468585330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240b0070 con 0x7f273c0cc200 2024-04-25T10:30:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f273b400640 1 -- 172.21.15.45:0/468585330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27240ab600 con 0x7f273c0cc200 2024-04-25T10:30:16.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f273b400640 1 -- 172.21.15.45:0/468585330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240ab9d0 con 0x7f273c0cc200 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 -- 172.21.15.45:0/468585330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c0cc200 msgr2=0x7f273c162180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 --2- 172.21.15.45:0/468585330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c0cc200 0x7f273c162180 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f2724009870 tx=0x7f272401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 -- 172.21.15.45:0/468585330 shutdown_connections 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 --2- 172.21.15.45:0/468585330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c0cc200 0x7f273c162180 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 --2- 172.21.15.45:0/468585330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0d0d00 0x7f273c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 -- 172.21.15.45:0/468585330 >> 172.21.15.45:0/468585330 conn(0x7f273c072290 msgr2=0x7f273c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 -- 172.21.15.45:0/468585330 shutdown_connections 2024-04-25T10:30:16.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 -- 172.21.15.45:0/468585330 wait complete. 2024-04-25T10:30:16.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 Processor -- start 2024-04-25T10:30:16.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.663+0000 7f2742c00640 1 -- start start 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0cc200 0x7f273c154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f273c0d0d00 0x7f273c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 0x7f273c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15e040 con 0x7f273c1626c0 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c06d860 con 0x7f273c0d0d00 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15e1c0 con 0x7f273c0cc200 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 0x7f273c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 0x7f273c0b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36798/0 (socket says 172.21.15.45:36798) 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 -- 172.21.15.45:0/3590681697 learned_addr learned my addr 172.21.15.45:0/3590681697 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2733400640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f273c0d0d00 0x7f273c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f273be00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0cc200 0x7f273c154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 -- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0cc200 msgr2=0x7f273c154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0cc200 0x7f273c154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 -- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f273c0d0d00 msgr2=0x7f273c0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f273c0d0d00 0x7f273c0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 -- 172.21.15.45:0/3590681697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0c4ad0 con 0x7f273c1626c0 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2740c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 0x7f273c0b2340 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f273400c730 tx=0x7f273400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734003b80 con 0x7f273c1626c0 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2734003d20 con 0x7f273c1626c0 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c0c4db0 con 0x7f273c1626c0 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273400aa30 con 0x7f273c1626c0 2024-04-25T10:30:16.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c0c5370 con 0x7f273c1626c0 2024-04-25T10:30:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c04ef30 con 0x7f273c1626c0 2024-04-25T10:30:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.667+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f273400abd0 con 0x7f273c1626c0 2024-04-25T10:30:16.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.671+0000 7f2738c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2718066f60 0x7f2718069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.671+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2734084af0 con 0x7f273c1626c0 2024-04-25T10:30:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.671+0000 7f273be00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2718066f60 0x7f2718069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:16.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.671+0000 7f273be00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2718066f60 0x7f2718069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f273c09e350 tx=0x7f272c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:16.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.675+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2734055710 con 0x7f273c1626c0 2024-04-25T10:30:16.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.835+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f273c072ff0 con 0x7f273c1626c0 2024-04-25T10:30:16.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2738c00640 1 -- 172.21.15.45:0/3590681697 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27340597b0 con 0x7f273c1626c0 2024-04-25T10:30:16.839 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2718066f60 msgr2=0x7f2718069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2718066f60 0x7f2718069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f273c09e350 tx=0x7f272c009190 comp rx=0 tx=0).stop 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 msgr2=0x7f273c0b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 0x7f273c0b2340 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f273400c730 tx=0x7f273400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 shutdown_connections 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2718066f60 0x7f2718069420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f273c1626c0 0x7f273c0b2340 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f273c0d0d00 0x7f273c0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 --2- 172.21.15.45:0/3590681697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f273c0cc200 0x7f273c154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 >> 172.21.15.45:0/3590681697 conn(0x7f273c072290 msgr2=0x7f273c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:16.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 shutdown_connections 2024-04-25T10:30:16.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.839+0000 7f2742c00640 1 -- 172.21.15.45:0/3590681697 wait complete. 2024-04-25T10:30:16.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:30:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 Processor -- start 2024-04-25T10:30:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 -- start start 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3e781550b0 0x7f3e780cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e7815b910 0x7f3e780b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e78071050 con 0x7f3e780cfe70 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7815bd80 con 0x7f3e7815b910 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e80200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7815d040 con 0x7f3e781550b0 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e7815b910 0x7f3e780b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3e7815b910 0x7f3e780b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36814/0 (socket says 172.21.15.45:36814) 2024-04-25T10:30:17.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 -- 172.21.15.45:0/2331994472 learned_addr learned my addr 172.21.15.45:0/2331994472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7d800640 1 --2- 172.21.15.45:0/2331994472 >> v2:172.21.15.117:3300/0 conn(0x7f3e781550b0 0x7f3e780cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 -- 172.21.15.45:0/2331994472 >> v2:172.21.15.117:3300/0 conn(0x7f3e781550b0 msgr2=0x7f3e780cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 --2- 172.21.15.45:0/2331994472 >> v2:172.21.15.117:3300/0 conn(0x7f3e781550b0 0x7f3e780cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 -- 172.21.15.45:0/2331994472 >> v1:172.21.15.117:6789/0 conn(0x7f3e780cfe70 legacy=0x7f3e7815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7e200640 1 -- 172.21.15.45:0/2331994472 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780712f0 con 0x7f3e7815b910 2024-04-25T10:30:17.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:16.999+0000 7f3e7d800640 1 --2- 172.21.15.45:0/2331994472 >> v2:172.21.15.117:3300/0 conn(0x7f3e781550b0 0x7f3e780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e7e200640 1 --2- 172.21.15.45:0/2331994472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e780b3260 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009e50 tx=0x7f3e7001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbc3385bac64201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e77e00640 1 -- 172.21.15.45:0/2331994472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700a57a0 con 0x7f3e7815b910 2024-04-25T10:30:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e77e00640 1 -- 172.21.15.45:0/2331994472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e700a5940 con 0x7f3e7815b910 2024-04-25T10:30:17.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e77e00640 1 -- 172.21.15.45:0/2331994472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700a5d10 con 0x7f3e7815b910 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 -- 172.21.15.45:0/2331994472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 msgr2=0x7f3e780b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 --2- 172.21.15.45:0/2331994472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e780b3260 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009e50 tx=0x7f3e7001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 -- 172.21.15.45:0/2331994472 shutdown_connections 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 --2- 172.21.15.45:0/2331994472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e780b3260 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 --2- 172.21.15.45:0/2331994472 >> v2:172.21.15.117:3300/0 conn(0x7f3e781550b0 0x7f3e780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 -- 172.21.15.45:0/2331994472 >> 172.21.15.45:0/2331994472 conn(0x7f3e78066b70 msgr2=0x7f3e78066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 -- 172.21.15.45:0/2331994472 shutdown_connections 2024-04-25T10:30:17.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.003+0000 7f3e80200640 1 -- 172.21.15.45:0/2331994472 wait complete. 2024-04-25T10:30:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 Processor -- start 2024-04-25T10:30:17.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 -- start start 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e780cfe70 0x7f3e781f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e781550b0 0x7f3e781f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e781577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7815c030 con 0x7f3e7815b910 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e780720b0 con 0x7f3e781550b0 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7815c1b0 con 0x7f3e780cfe70 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e781577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e781577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36828/0 (socket says 172.21.15.45:36828) 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7d800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e780cfe70 0x7f3e781f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 -- 172.21.15.45:0/82713690 learned_addr learned my addr 172.21.15.45:0/82713690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:17.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7ce00640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e781550b0 0x7f3e781f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 -- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e780cfe70 msgr2=0x7f3e781f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e780cfe70 0x7f3e781f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 -- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e781550b0 msgr2=0x7f3e781f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e781550b0 0x7f3e781f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 -- 172.21.15.45:0/82713690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780cf1e0 con 0x7f3e7815b910 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e7e200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e781577e0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f3e700af040 tx=0x7f3e70019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700b8670 con 0x7f3e7815b910 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e700be070 con 0x7f3e7815b910 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e700b8a50 con 0x7f3e7815b910 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.007+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e781f72a0 con 0x7f3e7815b910 2024-04-25T10:30:17.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781f7860 con 0x7f3e7815b910 2024-04-25T10:30:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e781f7aa0 con 0x7f3e7815b910 2024-04-25T10:30:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e700b8bf0 con 0x7f3e7815b910 2024-04-25T10:30:17.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e75600640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e58066f60 0x7f3e58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e7011fe50 con 0x7f3e7815b910 2024-04-25T10:30:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e7d800640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e58066f60 0x7f3e58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:17.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.011+0000 7f3e7d800640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e58066f60 0x7f3e58069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3e781529b0 tx=0x7f3e6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:17.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.015+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e700165e0 con 0x7f3e7815b910 2024-04-25T10:30:17.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:17.235+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f3e7804ef30 con 0x7f3e7815b910 2024-04-25T10:30:18.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.143+0000 7f3e75600640 1 -- 172.21.15.45:0/82713690 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v593) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3e700f0ba0 con 0x7f3e7815b910 2024-04-25T10:30:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e58066f60 msgr2=0x7f3e58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e58066f60 0x7f3e58069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3e781529b0 tx=0x7f3e6c008040 comp rx=0 tx=0).stop 2024-04-25T10:30:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 msgr2=0x7f3e781577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e781577e0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f3e700af040 tx=0x7f3e70019790 comp rx=0 tx=0).stop 2024-04-25T10:30:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 shutdown_connections 2024-04-25T10:30:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3e58066f60 0x7f3e58069420 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e7815b910 0x7f3e781577e0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e781550b0 0x7f3e781f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 --2- 172.21.15.45:0/82713690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e780cfe70 0x7f3e781f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 >> 172.21.15.45:0/82713690 conn(0x7f3e78066b70 msgr2=0x7f3e7815d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 shutdown_connections 2024-04-25T10:30:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.147+0000 7f3e80200640 1 -- 172.21.15.45:0/82713690 wait complete. 2024-04-25T10:30:18.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:30:18.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 Processor -- start 2024-04-25T10:30:18.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 -- start start 2024-04-25T10:30:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740aeda0 0x7f1374071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1374072020 0x7f1374070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1374159920 con 0x7f1374159540 2024-04-25T10:30:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137415c2f0 con 0x7f1374072020 2024-04-25T10:30:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1378e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137415c5d0 con 0x7f13740aeda0 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1371e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1374159540 0x7f13740ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48700/0 (socket says 172.21.15.45:48700) 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1371e00640 1 -- 172.21.15.45:0/2099628785 learned_addr learned my addr 172.21.15.45:0/2099628785 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1372800640 1 --2- 172.21.15.45:0/2099628785 >> v2:172.21.15.45:3300/0 conn(0x7f1374072020 0x7f1374070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1369400640 1 --2- 172.21.15.45:0/2099628785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740aeda0 0x7f1374071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1371400640 1 -- 172.21.15.45:0/2099628785 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945926439 0 0) 0x7f1374159920 con 0x7f1374159540 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1372800640 1 -- 172.21.15.45:0/2099628785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740aeda0 msgr2=0x7f1374071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1372800640 1 --2- 172.21.15.45:0/2099628785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740aeda0 0x7f1374071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1372800640 1 -- 172.21.15.45:0/2099628785 >> v1:172.21.15.45:6789/0 conn(0x7f1374159540 legacy=0x7f13740ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.319+0000 7f1372800640 1 -- 172.21.15.45:0/2099628785 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137415c8b0 con 0x7f1374072020 2024-04-25T10:30:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1372800640 1 --2- 172.21.15.45:0/2099628785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374072020 0x7f1374070350 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f136c009d90 tx=0x7f136c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bd6f34ccaa9e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1371400640 1 -- 172.21.15.45:0/2099628785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0b2070 con 0x7f1374072020 2024-04-25T10:30:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1371400640 1 -- 172.21.15.45:0/2099628785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c0a5a20 con 0x7f1374072020 2024-04-25T10:30:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1371400640 1 -- 172.21.15.45:0/2099628785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0a5df0 con 0x7f1374072020 2024-04-25T10:30:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 -- 172.21.15.45:0/2099628785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374072020 msgr2=0x7f1374070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 --2- 172.21.15.45:0/2099628785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374072020 0x7f1374070350 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f136c009d90 tx=0x7f136c01c940 comp rx=0 tx=0).stop 2024-04-25T10:30:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 -- 172.21.15.45:0/2099628785 shutdown_connections 2024-04-25T10:30:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 --2- 172.21.15.45:0/2099628785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374072020 0x7f1374070350 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 --2- 172.21.15.45:0/2099628785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740aeda0 0x7f1374071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 -- 172.21.15.45:0/2099628785 >> 172.21.15.45:0/2099628785 conn(0x7f13740669a0 msgr2=0x7f1374066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 -- 172.21.15.45:0/2099628785 shutdown_connections 2024-04-25T10:30:18.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 -- 172.21.15.45:0/2099628785 wait complete. 2024-04-25T10:30:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 Processor -- start 2024-04-25T10:30:18.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.323+0000 7f1378e00640 1 -- start start 2024-04-25T10:30:18.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1374072020 0x7f1374154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740aeda0 0x7f1374150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 0x7f13740b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13740710a0 con 0x7f1374159540 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137406c0d0 con 0x7f13740aeda0 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1374071220 con 0x7f1374072020 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 0x7f13740b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 0x7f13740b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36848/0 (socket says 172.21.15.45:36848) 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 -- 172.21.15.45:0/2768371801 learned_addr learned my addr 172.21.15.45:0/2768371801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1371e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1374072020 0x7f1374154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1369400640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740aeda0 0x7f1374150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 -- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1374072020 msgr2=0x7f1374154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1374072020 0x7f1374154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 -- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740aeda0 msgr2=0x7f1374150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740aeda0 0x7f1374150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 -- 172.21.15.45:0/2768371801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13740b1bc0 con 0x7f1374159540 2024-04-25T10:30:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1372800640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 0x7f13740b12e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f136c01aff0 tx=0x7f136c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0b2040 con 0x7f1374159540 2024-04-25T10:30:18.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c0bf070 con 0x7f1374159540 2024-04-25T10:30:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0ad8c0 con 0x7f1374159540 2024-04-25T10:30:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13740b1ea0 con 0x7f1374159540 2024-04-25T10:30:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13741506d0 con 0x7f1374159540 2024-04-25T10:30:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.327+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137404ef30 con 0x7f1374159540 2024-04-25T10:30:18.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.331+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f136c0ada60 con 0x7f1374159540 2024-04-25T10:30:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.331+0000 7f136aa00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.331+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f136c11fe70 con 0x7f1374159540 2024-04-25T10:30:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.331+0000 7f1371e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.331+0000 7f1371e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1364007fd0 tx=0x7f1364007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.335+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f136c016560 con 0x7f1374159540 2024-04-25T10:30:18.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.495+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f13740cf470 con 0x7f1374159540 2024-04-25T10:30:18.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.499+0000 7f136aa00640 1 -- 172.21.15.45:0/2768371801 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f136c0f0c40 con 0x7f1374159540 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 msgr2=0x7f1350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1364007fd0 tx=0x7f1364007d60 comp rx=0 tx=0).stop 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 msgr2=0x7f13740b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 0x7f13740b12e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f136c01aff0 tx=0x7f136c0196e0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 shutdown_connections 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1374159540 0x7f13740b12e0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740aeda0 0x7f1374150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 --2- 172.21.15.45:0/2768371801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1374072020 0x7f1374154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 >> 172.21.15.45:0/2768371801 conn(0x7f13740669a0 msgr2=0x7f13740cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 shutdown_connections 2024-04-25T10:30:18.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.503+0000 7f1378e00640 1 -- 172.21.15.45:0/2768371801 wait complete. 2024-04-25T10:30:18.524 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:30:18.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:30:18.614 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:30:18.708 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:30:18.709 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:18.709 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:18.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:30:18.717 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:30:18.717 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:18.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:30:18.727 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:18.727 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:18.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:30:18.767 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:30:18.767 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:18.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:30:18.778 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:30:18.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:18.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.947+0000 7f33dce00640 1 Processor -- start 2024-04-25T10:30:18.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.947+0000 7f33dce00640 1 -- start start 2024-04-25T10:30:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33dce00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33d81550b0 0x7f33d80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33dce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33dce00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d8071050 con 0x7f33d815b740 2024-04-25T10:30:18.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33dce00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d815d040 con 0x7f33d81550b0 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33dce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80712f0 con 0x7f33d80cfe70 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33d81550b0 0x7f33d80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33d81550b0 0x7f33d80cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36862/0 (socket says 172.21.15.45:36862) 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 -- 172.21.15.45:0/3943018971 learned_addr learned my addr 172.21.15.45:0/3943018971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33cd400640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 -- 172.21.15.45:0/3943018971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 msgr2=0x7f33d815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 -- 172.21.15.45:0/3943018971 >> v1:172.21.15.117:6789/0 conn(0x7f33d815b740 legacy=0x7f33d806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33d5e00640 1 -- 172.21.15.45:0/3943018971 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80715d0 con 0x7f33d81550b0 2024-04-25T10:30:18.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.951+0000 7f33cd400640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:18.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33d5e00640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d80cf930 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2745aabc008239ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33d5400640 1 -- 172.21.15.45:0/3943018971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c80b0070 con 0x7f33d81550b0 2024-04-25T10:30:18.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33d5400640 1 -- 172.21.15.45:0/3943018971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c80ab600 con 0x7f33d81550b0 2024-04-25T10:30:18.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33d5400640 1 -- 172.21.15.45:0/3943018971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c80ab9d0 con 0x7f33d81550b0 2024-04-25T10:30:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- 172.21.15.45:0/3943018971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 msgr2=0x7f33d80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d80cf930 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c801c560 comp rx=0 tx=0).stop 2024-04-25T10:30:18.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- 172.21.15.45:0/3943018971 shutdown_connections 2024-04-25T10:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 --2- 172.21.15.45:0/3943018971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d80cf930 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- 172.21.15.45:0/3943018971 >> 172.21.15.45:0/3943018971 conn(0x7f33d8066b70 msgr2=0x7f33d8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- 172.21.15.45:0/3943018971 shutdown_connections 2024-04-25T10:30:18.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- 172.21.15.45:0/3943018971 wait complete. 2024-04-25T10:30:18.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 Processor -- start 2024-04-25T10:30:18.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- start start 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d81400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d815b740 0x7f33d8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d815bcd0 con 0x7f33d81550b0 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d815bb50 con 0x7f33d815b740 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.955+0000 7f33dce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d815c030 con 0x7f33d80cfe70 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d81400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d81400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36872/0 (socket says 172.21.15.45:36872) 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 -- 172.21.15.45:0/4158214500 learned_addr learned my addr 172.21.15.45:0/4158214500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33d5e00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33d6800640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d815b740 0x7f33d8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 -- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 msgr2=0x7f33d80ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d80ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 -- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d815b740 msgr2=0x7f33d8153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d815b740 0x7f33d8153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 -- 172.21.15.45:0/4158214500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80ce790 con 0x7f33d81550b0 2024-04-25T10:30:18.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cd400640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d81400b0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f33c000e840 tx=0x7f33c000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0017070 con 0x7f33d81550b0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d8155b60 con 0x7f33d81550b0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c000c8d0 con 0x7f33d81550b0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c000f040 con 0x7f33d81550b0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d8156120 con 0x7f33d81550b0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d804ef30 con 0x7f33d81550b0 2024-04-25T10:30:18.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.959+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33c001a070 con 0x7f33d81550b0 2024-04-25T10:30:18.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.963+0000 7f33cea00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b4066f90 0x7f33b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:18.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.963+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f33c0084880 con 0x7f33d81550b0 2024-04-25T10:30:18.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.963+0000 7f33d5e00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b4066f90 0x7f33b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:18.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.963+0000 7f33d5e00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b4066f90 0x7f33b4069450 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f33c80ad000 tx=0x7f33c80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:18.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:18.963+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33c00554c0 con 0x7f33d81550b0 2024-04-25T10:30:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.163+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33d80b35a0 con 0x7f33d81550b0 2024-04-25T10:30:19.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.163+0000 7f33cea00640 1 -- 172.21.15.45:0/4158214500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f33c0059560 con 0x7f33d81550b0 2024-04-25T10:30:19.166 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:19.166 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":117,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9322,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2319606458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2319606458},{"type":"v1","addr":"172.21.15.117:6836","nonce":2319606458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9325,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3886078917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3886078917},{"type":"v1","addr":"172.21.15.117:6837","nonce":3886078917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":9993,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/228608058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":228608058},{"type":"v1","addr":"172.21.15.45:6834","nonce":228608058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":10371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3829783883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3829783883},{"type":"v1","addr":"172.21.15.45:6835","nonce":3829783883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[]} 2024-04-25T10:30:19.166 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 117 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b4066f90 msgr2=0x7f33b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b4066f90 0x7f33b4069450 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f33c80ad000 tx=0x7f33c80a6000 comp rx=0 tx=0).stop 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 msgr2=0x7f33d81400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d81400b0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f33c000e840 tx=0x7f33c000ed00 comp rx=0 tx=0).stop 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 shutdown_connections 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f33b4066f90 0x7f33b4069450 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33d815b740 0x7f33d8153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33d81550b0 0x7f33d81400b0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 --2- 172.21.15.45:0/4158214500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33d80cfe70 0x7f33d80ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 >> 172.21.15.45:0/4158214500 conn(0x7f33d8066b70 msgr2=0x7f33d815aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 shutdown_connections 2024-04-25T10:30:19.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.167+0000 7f33dce00640 1 -- 172.21.15.45:0/4158214500 wait complete. 2024-04-25T10:30:19.187 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:30:19.188 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:30:19.188 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:30:19.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:19.188 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:30:19.188 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:30:19.188 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:30:19.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:19.189 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:30:19.189 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:30:19.189 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:30:19.189 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:19.189 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:30:19.189 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:30:19.189 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:30:19.189 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:30:19.201 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:30:19.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:30:19.202 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:30:19.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:30:19.204 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:30:19.204 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:30:19.205 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:30:19.205 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:30:19.249 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:30:19.249 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:19.251 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:30:19.251 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:19.253 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:30:19.253 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:19.254 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:30:19.255 DEBUG:teuthology.parallel:result is None 2024-04-25T10:30:19.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:30:19.337 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:30:19.337 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:30:19.344 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:30:19.344 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:30:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.399+0000 7fc219a00640 1 Processor -- start 2024-04-25T10:30:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.399+0000 7fc219a00640 1 -- start start 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc219a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc21415bc40 0x7fc21406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc219a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cda10 0x7fc2140b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc219a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc2140cf930 0x7fc2140cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc219a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2140b2770 con 0x7fc21415bc40 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc219a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21406c670 con 0x7fc2140cf930 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc219a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21406c950 con 0x7fc2140cda10 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc21415bc40 0x7fc21406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc21415bc40 0x7fc21406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36910/0 (socket says 172.21.15.45:36910) 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 -- 172.21.15.45:0/609514071 learned_addr learned my addr 172.21.15.45:0/609514071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212000640 1 --2- 172.21.15.45:0/609514071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cda10 0x7fc2140b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc213400640 1 --2- 172.21.15.45:0/609514071 >> v2:172.21.15.117:3300/0 conn(0x7fc2140cf930 0x7fc2140cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 -- 172.21.15.45:0/609514071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cda10 msgr2=0x7fc2140b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 --2- 172.21.15.45:0/609514071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cda10 0x7fc2140b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 -- 172.21.15.45:0/609514071 >> v2:172.21.15.117:3300/0 conn(0x7fc2140cf930 msgr2=0x7fc2140cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 --2- 172.21.15.45:0/609514071 >> v2:172.21.15.117:3300/0 conn(0x7fc2140cf930 0x7fc2140cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc212a00640 1 -- 172.21.15.45:0/609514071 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21406cc30 con 0x7fc21415bc40 2024-04-25T10:30:19.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.403+0000 7fc213400640 1 --2- 172.21.15.45:0/609514071 >> v2:172.21.15.117:3300/0 conn(0x7fc2140cf930 0x7fc2140cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:19.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:30:19.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:30:19.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:30:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc212a00640 1 --2- 172.21.15.45:0/609514071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21406c090 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fc2140b3f80 tx=0x7fc20401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfacfddc2e079a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc211600640 1 -- 172.21.15.45:0/609514071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2040b0070 con 0x7fc21415bc40 2024-04-25T10:30:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc211600640 1 -- 172.21.15.45:0/609514071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2040ab600 con 0x7fc21415bc40 2024-04-25T10:30:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc211600640 1 -- 172.21.15.45:0/609514071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2040ab9d0 con 0x7fc21415bc40 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 -- 172.21.15.45:0/609514071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 msgr2=0x7fc21406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 --2- 172.21.15.45:0/609514071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21406c090 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fc2140b3f80 tx=0x7fc20401c520 comp rx=0 tx=0).stop 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 -- 172.21.15.45:0/609514071 shutdown_connections 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 --2- 172.21.15.45:0/609514071 >> v2:172.21.15.117:3300/0 conn(0x7fc2140cf930 0x7fc2140cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 --2- 172.21.15.45:0/609514071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cda10 0x7fc2140b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 --2- 172.21.15.45:0/609514071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21406c090 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 -- 172.21.15.45:0/609514071 >> 172.21.15.45:0/609514071 conn(0x7fc214066b70 msgr2=0x7fc214066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 -- 172.21.15.45:0/609514071 shutdown_connections 2024-04-25T10:30:19.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 -- 172.21.15.45:0/609514071 wait complete. 2024-04-25T10:30:19.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 Processor -- start 2024-04-25T10:30:19.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.407+0000 7fc219a00640 1 -- start start 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2140cda10 0x7fc214156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cf930 0x7fc214157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214160230 con 0x7fc21415bc40 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2141600b0 con 0x7fc2140cda10 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2141603b0 con 0x7fc2140cf930 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21415d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36918/0 (socket says 172.21.15.45:36918) 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc212000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cf930 0x7fc214157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 -- 172.21.15.45:0/381161533 learned_addr learned my addr 172.21.15.45:0/381161533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc212a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2140cda10 0x7fc214156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 -- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cf930 msgr2=0x7fc214157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:19.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cf930 0x7fc214157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 -- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2140cda10 msgr2=0x7fc214156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:19.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2140cda10 0x7fc214156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:19.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 -- 172.21.15.45:0/381161533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2141558c0 con 0x7fc21415bc40 2024-04-25T10:30:19.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc213400640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21415d920 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00cd50 tx=0x7fc20c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00f680 con 0x7fc21415bc40 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00f840 con 0x7fc21415bc40 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc214155ba0 con 0x7fc21415bc40 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00fc30 con 0x7fc21415bc40 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc214156440 con 0x7fc21415bc40 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21404ef30 con 0x7fc21415bc40 2024-04-25T10:30:19.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.411+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc20c0213f0 con 0x7fc21415bc40 2024-04-25T10:30:19.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.415+0000 7fc20aa00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:19.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.415+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc20c085a80 con 0x7fc21415bc40 2024-04-25T10:30:19.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.415+0000 7fc212a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:19.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.415+0000 7fc212a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2040ad000 tx=0x7fc2040a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:19.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.419+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc20c087e20 con 0x7fc21415bc40 2024-04-25T10:30:19.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:19.595+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc2140af1c0 con 0x7fc21415bc40 2024-04-25T10:30:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.519+0000 7fc20aa00640 1 -- 172.21.15.45:0/381161533 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v165) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc20c014030 con 0x7fc21415bc40 2024-04-25T10:30:20.521 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:30:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 msgr2=0x7fc1f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc2040ad000 tx=0x7fc2040a6000 comp rx=0 tx=0).stop 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 msgr2=0x7fc21415d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21415d920 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00cd50 tx=0x7fc20c00a650 comp rx=0 tx=0).stop 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 shutdown_connections 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc21415bc40 0x7fc21415d920 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2140cf930 0x7fc214157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 --2- 172.21.15.45:0/381161533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2140cda10 0x7fc214156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 >> 172.21.15.45:0/381161533 conn(0x7fc214066b70 msgr2=0x7fc2140adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 shutdown_connections 2024-04-25T10:30:20.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.523+0000 7fc219a00640 1 -- 172.21.15.45:0/381161533 wait complete. 2024-04-25T10:30:20.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 Processor -- start 2024-04-25T10:30:20.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 -- start start 2024-04-25T10:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5fbc15a0e0 0x7f5fbc15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5fbc0b3190 0x7f5fbc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fbc15b6e0 con 0x7f5fbc071f70 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fbc070ed0 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fc1400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fbc0711b0 con 0x7f5fbc0b3190 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fba800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5fbc15a0e0 0x7f5fbc15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fba800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5fbc15a0e0 0x7f5fbc15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36932/0 (socket says 172.21.15.45:36932) 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fba800640 1 -- 172.21.15.45:0/4229249211 learned_addr learned my addr 172.21.15.45:0/4229249211 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fba800640 1 -- 172.21.15.45:0/4229249211 >> v2:172.21.15.117:3300/0 conn(0x7f5fbc0b3190 msgr2=0x7f5fbc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.695+0000 7f5fbb200640 1 --2- 172.21.15.45:0/4229249211 >> v2:172.21.15.117:3300/0 conn(0x7f5fbc0b3190 0x7f5fbc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fba800640 1 --2- 172.21.15.45:0/4229249211 >> v2:172.21.15.117:3300/0 conn(0x7f5fbc0b3190 0x7f5fbc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fba800640 1 -- 172.21.15.45:0/4229249211 >> v1:172.21.15.117:6789/0 conn(0x7f5fbc071f70 legacy=0x7f5fbc0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fba800640 1 -- 172.21.15.45:0/4229249211 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc071490 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fbb200640 1 --2- 172.21.15.45:0/4229249211 >> v2:172.21.15.117:3300/0 conn(0x7f5fbc0b3190 0x7f5fbc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fba800640 1 --2- 172.21.15.45:0/4229249211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc15b1a0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0d0670 tx=0x7f5fac01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e736ee0cf1b53610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fb9e00640 1 -- 172.21.15.45:0/4229249211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0b0070 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fb9e00640 1 -- 172.21.15.45:0/4229249211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac0a4e50 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fb9e00640 1 -- 172.21.15.45:0/4229249211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0ab820 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 -- 172.21.15.45:0/4229249211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 msgr2=0x7f5fbc15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 --2- 172.21.15.45:0/4229249211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc15b1a0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0d0670 tx=0x7f5fac01c240 comp rx=0 tx=0).stop 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 -- 172.21.15.45:0/4229249211 shutdown_connections 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 --2- 172.21.15.45:0/4229249211 >> v2:172.21.15.117:3300/0 conn(0x7f5fbc0b3190 0x7f5fbc06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 --2- 172.21.15.45:0/4229249211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc15b1a0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 -- 172.21.15.45:0/4229249211 >> 172.21.15.45:0/4229249211 conn(0x7f5fbc066b70 msgr2=0x7f5fbc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 -- 172.21.15.45:0/4229249211 shutdown_connections 2024-04-25T10:30:20.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.699+0000 7f5fc1400640 1 -- 172.21.15.45:0/4229249211 wait complete. 2024-04-25T10:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 Processor -- start 2024-04-25T10:30:20.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 -- start start 2024-04-25T10:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc071f70 0x7f5fbc14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fbc0b3190 0x7f5fbc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc070b50 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc15a4c0 con 0x7f5fbc071f70 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0723f0 con 0x7f5fbc0b3190 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc06d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36934/0 (socket says 172.21.15.45:36934) 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 -- 172.21.15.45:0/1403340188 learned_addr learned my addr 172.21.15.45:0/1403340188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fba800640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc071f70 0x7f5fbc14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fb3e00640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fbc0b3190 0x7f5fbc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 -- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fbc0b3190 msgr2=0x7f5fbc14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fbc0b3190 0x7f5fbc14e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 -- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc071f70 msgr2=0x7f5fbc14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc071f70 0x7f5fbc14ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 -- 172.21.15.45:0/1403340188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc1f6900 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fba800640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc071f70 0x7f5fbc14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fb3e00640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fbc0b3190 0x7f5fbc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:20.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fbb200640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc06d1a0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400cf20 tx=0x7f5fb400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb401a070 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb400d840 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb400dc30 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc14f1d0 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.703+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc14f790 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.707+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc04ef30 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.707+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fb401d070 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.707+0000 7f5fb2a00640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f98066f60 0x7f5f98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.707+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5fb4085500 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.707+0000 7f5fba800640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f98066f60 0x7f5f98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:20.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.711+0000 7f5fba800640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f98066f60 0x7f5f98069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0ad000 tx=0x7f5fac0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:20.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.711+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fb40568d0 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5fbc14b2c0 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fb2a00640 1 -- 172.21.15.45:0/1403340188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5fb40561d0 con 0x7f5fbc15a0e0 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:30:20.933 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.935 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.937 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.939 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:30:20.941 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.942 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:30:20.943 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:30:20.945 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:30:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:30:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:20.946 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:30:20.946 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f98066f60 msgr2=0x7f5f98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f98066f60 0x7f5f98069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0ad000 tx=0x7f5fac0a6000 comp rx=0 tx=0).stop 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 msgr2=0x7f5fbc06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc06d1a0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400cf20 tx=0x7f5fb400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 shutdown_connections 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5f98066f60 0x7f5f98069420 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fbc15a0e0 0x7f5fbc06d1a0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fbc0b3190 0x7f5fbc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 --2- 172.21.15.45:0/1403340188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc071f70 0x7f5fbc14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 >> 172.21.15.45:0/1403340188 conn(0x7f5fbc066b70 msgr2=0x7f5fbc15bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 shutdown_connections 2024-04-25T10:30:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:20.931+0000 7f5fc1400640 1 -- 172.21.15.45:0/1403340188 wait complete. 2024-04-25T10:30:20.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.095+0000 7fa618600640 1 Processor -- start 2024-04-25T10:30:21.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.095+0000 7fa618600640 1 -- start start 2024-04-25T10:30:21.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa618600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6100d0040 0x7fa61015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa618600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa610071050 con 0x7fa6101550b0 2024-04-25T10:30:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa618600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61015d040 con 0x7fa61015b740 2024-04-25T10:30:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa618600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100712f0 con 0x7fa6100d0040 2024-04-25T10:30:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa615c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa6101550b0 0x7fa6100cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48720/0 (socket says 172.21.15.45:48720) 2024-04-25T10:30:21.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa615c00640 1 -- 172.21.15.45:0/188758553 learned_addr learned my addr 172.21.15.45:0/188758553 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa615200640 1 --2- 172.21.15.45:0/188758553 >> v2:172.21.15.117:3300/0 conn(0x7fa6100d0040 0x7fa61015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194073990 0 0) 0x7fa610071050 con 0x7fa6101550b0 2024-04-25T10:30:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f8003530 con 0x7fa6101550b0 2024-04-25T10:30:21.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527464375 0 0) 0x7fa61015d040 con 0x7fa61015b740 2024-04-25T10:30:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa610071050 con 0x7fa61015b740 2024-04-25T10:30:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3810414457 0 0) 0x7fa5f8003530 con 0x7fa6101550b0 2024-04-25T10:30:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa615200640 1 -- 172.21.15.45:0/188758553 >> v1:172.21.15.117:6789/0 conn(0x7fa61015b740 legacy=0x7fa6100b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa615200640 1 -- 172.21.15.45:0/188758553 >> v1:172.21.15.45:6789/0 conn(0x7fa6101550b0 legacy=0x7fa6100cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.099+0000 7fa615200640 1 -- 172.21.15.45:0/188758553 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100715d0 con 0x7fa6100d0040 2024-04-25T10:30:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa615200640 1 --2- 172.21.15.45:0/188758553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6100d0040 0x7fa61015b200 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0097c0 tx=0x7fa5fc01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf4aafd1e3ba0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0b0070 con 0x7fa6100d0040 2024-04-25T10:30:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc0ab400 con 0x7fa6100d0040 2024-04-25T10:30:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa60fe00640 1 -- 172.21.15.45:0/188758553 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604001600 con 0x7fa6100d0040 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 -- 172.21.15.45:0/188758553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6100d0040 msgr2=0x7fa61015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 --2- 172.21.15.45:0/188758553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6100d0040 0x7fa61015b200 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0097c0 tx=0x7fa5fc01c390 comp rx=0 tx=0).stop 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 -- 172.21.15.45:0/188758553 shutdown_connections 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 --2- 172.21.15.45:0/188758553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6100d0040 0x7fa61015b200 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 -- 172.21.15.45:0/188758553 >> 172.21.15.45:0/188758553 conn(0x7fa610066b70 msgr2=0x7fa610066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 -- 172.21.15.45:0/188758553 shutdown_connections 2024-04-25T10:30:21.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 -- 172.21.15.45:0/188758553 wait complete. 2024-04-25T10:30:21.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 Processor -- start 2024-04-25T10:30:21.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.103+0000 7fa618600640 1 -- start start 2024-04-25T10:30:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 0x7fa610155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6101550b0 0x7fa6100b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61015b740 0x7fa61013cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015bcd0 con 0x7fa6100d0040 2024-04-25T10:30:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015bb50 con 0x7fa6101550b0 2024-04-25T10:30:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61006c090 con 0x7fa61015b740 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 0x7fa610155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 0x7fa610155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36950/0 (socket says 172.21.15.45:36950) 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 -- 172.21.15.45:0/931501242 learned_addr learned my addr 172.21.15.45:0/931501242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615200640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6101550b0 0x7fa6100b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa616600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61015b740 0x7fa61013cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 -- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61015b740 msgr2=0x7fa61013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61015b740 0x7fa61013cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 -- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6101550b0 msgr2=0x7fa6100b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6101550b0 0x7fa6100b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 -- 172.21.15.45:0/931501242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610157b50 con 0x7fa6100d0040 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615200640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6101550b0 0x7fa6100b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa615c00640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 0x7fa610155490 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c780 tx=0x7fa60400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604005ae0 con 0x7fa6100d0040 2024-04-25T10:30:21.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604005c80 con 0x7fa6100d0040 2024-04-25T10:30:21.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa610157e30 con 0x7fa6100d0040 2024-04-25T10:30:21.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040072b0 con 0x7fa6100d0040 2024-04-25T10:30:21.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100cc990 con 0x7fa6100d0040 2024-04-25T10:30:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61004ef30 con 0x7fa6100d0040 2024-04-25T10:30:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.107+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa604007450 con 0x7fa6100d0040 2024-04-25T10:30:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.111+0000 7fa60d600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.111+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa604084bc0 con 0x7fa6100d0040 2024-04-25T10:30:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.111+0000 7fa615200640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.111+0000 7fa615200640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0ad000 tx=0x7fa5fc0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.115+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa604055e80 con 0x7fa6100d0040 2024-04-25T10:30:21.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.275+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa61015ab70 con 0x7fa6100d0040 2024-04-25T10:30:21.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa60d600640 1 -- 172.21.15.45:0/931501242 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fa604055780 con 0x7fa6100d0040 2024-04-25T10:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":165,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:20.520507+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5f0066f60 msgr2=0x7fa5f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5f0066f60 0x7fa5f0069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0ad000 tx=0x7fa5fc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 msgr2=0x7fa610155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 0x7fa610155490 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c780 tx=0x7fa60400cc40 comp rx=0 tx=0).stop 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 shutdown_connections 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa5f0066f60 0x7fa5f0069420 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa61015b740 0x7fa61013cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6101550b0 0x7fa6100b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 --2- 172.21.15.45:0/931501242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6100d0040 0x7fa610155490 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 >> 172.21.15.45:0/931501242 conn(0x7fa610066b70 msgr2=0x7fa6100af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 shutdown_connections 2024-04-25T10:30:21.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.279+0000 7fa618600640 1 -- 172.21.15.45:0/931501242 wait complete. 2024-04-25T10:30:21.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:30:21.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:30:21.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 Processor -- start 2024-04-25T10:30:21.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 -- start start 2024-04-25T10:30:21.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1dc0159540 0x7f1dc00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1dc00aebd0 0x7f1dc0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc0071360 con 0x7f1dc0072020 2024-04-25T10:30:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc0159920 con 0x7f1dc00aebd0 2024-04-25T10:30:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dc5800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc015c2f0 con 0x7f1dc0159540 2024-04-25T10:30:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1dc00aebd0 0x7f1dc0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1dc00aebd0 0x7f1dc0071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36954/0 (socket says 172.21.15.45:36954) 2024-04-25T10:30:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 -- 172.21.15.45:0/1079100408 learned_addr learned my addr 172.21.15.45:0/1079100408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dbea00640 1 --2- 172.21.15.45:0/1079100408 >> v2:172.21.15.117:3300/0 conn(0x7f1dc0159540 0x7f1dc00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 -- 172.21.15.45:0/1079100408 >> v2:172.21.15.117:3300/0 conn(0x7f1dc0159540 msgr2=0x7f1dc00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 --2- 172.21.15.45:0/1079100408 >> v2:172.21.15.117:3300/0 conn(0x7f1dc0159540 0x7f1dc00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 -- 172.21.15.45:0/1079100408 >> v1:172.21.15.45:6789/0 conn(0x7f1dc0072020 legacy=0x7f1dc015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1db7e00640 1 -- 172.21.15.45:0/1079100408 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc015c5d0 con 0x7f1dc00aebd0 2024-04-25T10:30:21.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.447+0000 7f1dbe000640 1 -- 172.21.15.45:0/1079100408 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147193875 0 0) 0x7f1dc0071360 con 0x7f1dc0072020 2024-04-25T10:30:21.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1db7e00640 1 --2- 172.21.15.45:0/1079100408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc00aebd0 0x7f1dc0071ae0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00b4c20 tx=0x7f1da801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec53336e63622dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dbe000640 1 -- 172.21.15.45:0/1079100408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da80b0070 con 0x7f1dc00aebd0 2024-04-25T10:30:21.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dbe000640 1 -- 172.21.15.45:0/1079100408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da80ab600 con 0x7f1dc00aebd0 2024-04-25T10:30:21.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dbe000640 1 -- 172.21.15.45:0/1079100408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da80ab9d0 con 0x7f1dc00aebd0 2024-04-25T10:30:21.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 -- 172.21.15.45:0/1079100408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc00aebd0 msgr2=0x7f1dc0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 --2- 172.21.15.45:0/1079100408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc00aebd0 0x7f1dc0071ae0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00b4c20 tx=0x7f1da801c520 comp rx=0 tx=0).stop 2024-04-25T10:30:21.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 -- 172.21.15.45:0/1079100408 shutdown_connections 2024-04-25T10:30:21.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 --2- 172.21.15.45:0/1079100408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc00aebd0 0x7f1dc0071ae0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 --2- 172.21.15.45:0/1079100408 >> v2:172.21.15.117:3300/0 conn(0x7f1dc0159540 0x7f1dc00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 -- 172.21.15.45:0/1079100408 >> 172.21.15.45:0/1079100408 conn(0x7f1dc00669a0 msgr2=0x7f1dc0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 -- 172.21.15.45:0/1079100408 shutdown_connections 2024-04-25T10:30:21.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.451+0000 7f1dc5800640 1 -- 172.21.15.45:0/1079100408 wait complete. 2024-04-25T10:30:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 Processor -- start 2024-04-25T10:30:21.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- start start 2024-04-25T10:30:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 0x7f1dc00af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc00aebd0 0x7f1dc014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc0159540 0x7f1dc014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0072430 con 0x7f1dc0072020 2024-04-25T10:30:21.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc006c0d0 con 0x7f1dc0159540 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc0168930 con 0x7f1dc00aebd0 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 0x7f1dc00af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 0x7f1dc00af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36958/0 (socket says 172.21.15.45:36958) 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 -- 172.21.15.45:0/2725130342 learned_addr learned my addr 172.21.15.45:0/2725130342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbf400640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc0159540 0x7f1dc014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1db7e00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc00aebd0 0x7f1dc014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 -- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc00aebd0 msgr2=0x7f1dc014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc00aebd0 0x7f1dc014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 -- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc0159540 msgr2=0x7f1dc014f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc0159540 0x7f1dc014f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 -- 172.21.15.45:0/2725130342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc014ecd0 con 0x7f1dc0072020 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1db7e00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc00aebd0 0x7f1dc014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dbea00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 0x7f1dc00af280 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1db000d5f0 tx=0x7f1db000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00125f0 con 0x7f1dc0072020 2024-04-25T10:30:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db0019070 con 0x7f1dc0072020 2024-04-25T10:30:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc00b1e70 con 0x7f1dc0072020 2024-04-25T10:30:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00129b0 con 0x7f1dc0072020 2024-04-25T10:30:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc00b2150 con 0x7f1dc0072020 2024-04-25T10:30:21.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.455+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc004ef30 con 0x7f1dc0072020 2024-04-25T10:30:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.459+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1db0012b50 con 0x7f1dc0072020 2024-04-25T10:30:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.459+0000 7f1db6a00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1d9c066f90 0x7f1d9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.459+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1db0085110 con 0x7f1dc0072020 2024-04-25T10:30:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.459+0000 7f1db7e00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1d9c066f90 0x7f1d9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.459+0000 7f1db7e00640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1d9c066f90 0x7f1d9c069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1da80ad000 tx=0x7f1da8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.463+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1db0056350 con 0x7f1dc0072020 2024-04-25T10:30:21.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.623+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1dc00cf470 con 0x7f1dc0072020 2024-04-25T10:30:21.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1db6a00640 1 -- 172.21.15.45:0/2725130342 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v165) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1db0055c50 con 0x7f1dc0072020 2024-04-25T10:30:21.628 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1d9c066f90 msgr2=0x7f1d9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1d9c066f90 0x7f1d9c069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1da80ad000 tx=0x7f1da8008040 comp rx=0 tx=0).stop 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 msgr2=0x7f1dc00af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 0x7f1dc00af280 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1db000d5f0 tx=0x7f1db000dab0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 shutdown_connections 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1d9c066f90 0x7f1d9c069450 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1dc0159540 0x7f1dc014f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1dc00aebd0 0x7f1dc014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 --2- 172.21.15.45:0/2725130342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1dc0072020 0x7f1dc00af280 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 >> 172.21.15.45:0/2725130342 conn(0x7f1dc00669a0 msgr2=0x7f1dc0070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 shutdown_connections 2024-04-25T10:30:21.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.627+0000 7f1dc5800640 1 -- 172.21.15.45:0/2725130342 wait complete. 2024-04-25T10:30:21.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:30:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.787+0000 7f0b5f000640 1 Processor -- start 2024-04-25T10:30:21.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.787+0000 7f0b5f000640 1 -- start start 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.787+0000 7f0b5f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b580cf930 0x7f0b580cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.787+0000 7f0b5f000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b58070c20 0x7f0b5815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.787+0000 7f0b5f000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b580ae020 0x7f0b580ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.787+0000 7f0b5f000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5815bb90 con 0x7f0b58070c20 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b5f000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b580d0250 con 0x7f0b580ae020 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b5f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b580ad840 con 0x7f0b580cf930 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b58070c20 0x7f0b5815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b58070c20 0x7f0b5815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36960/0 (socket says 172.21.15.45:36960) 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4068945092 learned_addr learned my addr 172.21.15.45:0/4068945092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b5d000640 1 --2- 172.21.15.45:0/4068945092 >> v2:172.21.15.117:3300/0 conn(0x7f0b580ae020 0x7f0b580ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b57e00640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b580cf930 0x7f0b580cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4068945092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b580cf930 msgr2=0x7f0b580cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b580cf930 0x7f0b580cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4068945092 >> v2:172.21.15.117:3300/0 conn(0x7f0b580ae020 msgr2=0x7f0b580ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 --2- 172.21.15.45:0/4068945092 >> v2:172.21.15.117:3300/0 conn(0x7f0b580ae020 0x7f0b580ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4068945092 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5815e9f0 con 0x7f0b58070c20 2024-04-25T10:30:21.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b57e00640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b580cf930 0x7f0b580cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b4fe00640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b58070c20 0x7f0b5815e170 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f0b580d05c0 tx=0x7f0b4001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e10004360d095e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b57400640 1 -- 172.21.15.45:0/4068945092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b400aa610 con 0x7f0b58070c20 2024-04-25T10:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.791+0000 7f0b57400640 1 -- 172.21.15.45:0/4068945092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b400af070 con 0x7f0b58070c20 2024-04-25T10:30:21.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b57400640 1 -- 172.21.15.45:0/4068945092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b400aa9d0 con 0x7f0b58070c20 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- 172.21.15.45:0/4068945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b58070c20 msgr2=0x7f0b5815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b58070c20 0x7f0b5815e170 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f0b580d05c0 tx=0x7f0b4001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- 172.21.15.45:0/4068945092 shutdown_connections 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4068945092 >> v2:172.21.15.117:3300/0 conn(0x7f0b580ae020 0x7f0b580ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b58070c20 0x7f0b5815e170 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4068945092 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b580cf930 0x7f0b580cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- 172.21.15.45:0/4068945092 >> 172.21.15.45:0/4068945092 conn(0x7f0b58066b70 msgr2=0x7f0b58066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- 172.21.15.45:0/4068945092 shutdown_connections 2024-04-25T10:30:21.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- 172.21.15.45:0/4068945092 wait complete. 2024-04-25T10:30:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 Processor -- start 2024-04-25T10:30:21.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- start start 2024-04-25T10:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b58070c20 0x7f0b58151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 0x7f0b58156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b580cf930 0x7f0b580ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5815e6b0 con 0x7f0b580ae020 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5806e130 con 0x7f0b580cf930 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5f000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b58161790 con 0x7f0b58070c20 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 0x7f0b58156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 0x7f0b58156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36976/0 (socket says 172.21.15.45:36976) 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4081248187 learned_addr learned my addr 172.21.15.45:0/4081248187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b5d000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b580cf930 0x7f0b580ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.795+0000 7f0b57e00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b58070c20 0x7f0b58151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b58070c20 msgr2=0x7f0b58151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b4fe00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b58070c20 0x7f0b58151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b580cf930 msgr2=0x7f0b580ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b4fe00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b580cf930 0x7f0b580ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b4fe00640 1 -- 172.21.15.45:0/4081248187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58152800 con 0x7f0b580ae020 2024-04-25T10:30:21.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b4fe00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 0x7f0b58156a90 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f0b4001aba0 tx=0x7f0b4001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b400aada0 con 0x7f0b580ae020 2024-04-25T10:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b400bd3f0 con 0x7f0b580ae020 2024-04-25T10:30:21.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b58152ae0 con 0x7f0b580ae020 2024-04-25T10:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b400bd7a0 con 0x7f0b580ae020 2024-04-25T10:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b581530a0 con 0x7f0b580ae020 2024-04-25T10:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5804ef30 con 0x7f0b580ae020 2024-04-25T10:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b400b33f0 con 0x7f0b580ae020 2024-04-25T10:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b54c00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0b34066f60 0x7f0b34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:21.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b4011efc0 con 0x7f0b580ae020 2024-04-25T10:30:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.799+0000 7f0b57e00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0b34066f60 0x7f0b34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:21.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.803+0000 7f0b57e00640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0b34066f60 0x7f0b34069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0b581688a0 tx=0x7f0b48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:21.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.803+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b400c3060 con 0x7f0b580ae020 2024-04-25T10:30:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0b5815d7e0 con 0x7f0b580ae020 2024-04-25T10:30:21.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b54c00640 1 -- 172.21.15.45:0/4081248187 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v165) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0b400a7070 con 0x7f0b580ae020 2024-04-25T10:30:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:21.973 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:30:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:30:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:30:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:30:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:21.973 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0b34066f60 msgr2=0x7f0b34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0b34066f60 0x7f0b34069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0b581688a0 tx=0x7f0b48008040 comp rx=0 tx=0).stop 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 msgr2=0x7f0b58156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 0x7f0b58156a90 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f0b4001aba0 tx=0x7f0b4001ad30 comp rx=0 tx=0).stop 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 shutdown_connections 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0b34066f60 0x7f0b34069420 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b580cf930 0x7f0b580ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b580ae020 0x7f0b58156a90 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 --2- 172.21.15.45:0/4081248187 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b58070c20 0x7f0b58151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 >> 172.21.15.45:0/4081248187 conn(0x7f0b58066b70 msgr2=0x7f0b58160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 shutdown_connections 2024-04-25T10:30:21.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:21.971+0000 7f0b5f000640 1 -- 172.21.15.45:0/4081248187 wait complete. 2024-04-25T10:30:21.994 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:30:21.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:30:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.135+0000 7f18d2a00640 1 Processor -- start 2024-04-25T10:30:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.135+0000 7f18d2a00640 1 -- start start 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18d2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 0x7f18cc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18d2a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc071f80 con 0x7f18cc0cf040 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18d2a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18cc15b120 con 0x7f18cc0d00f0 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18d2a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc072260 con 0x7f18cc071ab0 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f18cc0cf040 0x7f18cc0cf9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48742/0 (socket says 172.21.15.45:48742) 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cbe00640 1 -- 172.21.15.45:0/255905817 learned_addr learned my addr 172.21.15.45:0/255905817 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18d0a00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 0x7f18cc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826321756 0 0) 0x7f18cc071f80 con 0x7f18cc0cf040 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b0003530 con 0x7f18cc0cf040 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101163312 0 0) 0x7f18cc15b120 con 0x7f18cc0d00f0 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18cc071f80 con 0x7f18cc0d00f0 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2776322850 0 0) 0x7f18b0003530 con 0x7f18cc0cf040 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 msgr2=0x7f18cc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 0x7f18cc0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 >> v1:172.21.15.117:6789/0 conn(0x7f18cc0d00f0 legacy=0x7f18cc0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc15b400 con 0x7f18cc0cf040 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f18bc001680 con 0x7f18cc0cf040 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 >> v1:172.21.15.45:6789/0 conn(0x7f18cc0cf040 legacy=0x7f18cc0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b0003b80 0x7f18b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 0x7f18b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b000dc90 con 0x7f18b0004810 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b000de10 con 0x7f18b0003b80 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b000dfe0 con 0x7f18cc071ab0 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18c3e00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18cbe00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b0003b80 0x7f18b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18d0a00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 0x7f18b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18c3e00640 1 -- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 msgr2=0x7f18b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18c3e00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc071ab0 0x7f18b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18c3e00640 1 -- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b0003b80 msgr2=0x7f18b0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18c3e00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18b0003b80 0x7f18b0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.139+0000 7f18c3e00640 1 -- 172.21.15.45:0/255905817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b0003530 con 0x7f18b0004810 2024-04-25T10:30:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18c3e00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f18cc070530 tx=0x7f18b401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36085ea4e6da856b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4001520 con 0x7f18b0004810 2024-04-25T10:30:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b40052f0 con 0x7f18b0004810 2024-04-25T10:30:22.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18cb400640 1 -- 172.21.15.45:0/255905817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b40a88d0 con 0x7f18b0004810 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 -- 172.21.15.45:0/255905817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 msgr2=0x7f18b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 --2- 172.21.15.45:0/255905817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f18cc070530 tx=0x7f18b401ec70 comp rx=0 tx=0).stop 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d0a00640 1 -- 172.21.15.45:0/255905817 reap_dead start 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 -- 172.21.15.45:0/255905817 shutdown_connections 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 -- 172.21.15.45:0/255905817 >> 172.21.15.45:0/255905817 conn(0x7f18cc066b70 msgr2=0x7f18cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 -- 172.21.15.45:0/255905817 shutdown_connections 2024-04-25T10:30:22.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 -- 172.21.15.45:0/255905817 wait complete. 2024-04-25T10:30:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 Processor -- start 2024-04-25T10:30:22.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.143+0000 7f18d2a00640 1 -- start start 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18cc13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc14f550 0x7f18cc14f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18cc153910 0x7f18cc153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc15aec0 con 0x7f18b0004810 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0b3700 con 0x7f18cc153910 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc0cf420 con 0x7f18cc14f550 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18cc13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18cc13f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37008/0 (socket says 172.21.15.45:37008) 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 -- 172.21.15.45:0/1037232513 learned_addr learned my addr 172.21.15.45:0/1037232513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 -- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc14f550 msgr2=0x7f18cc14f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc14f550 0x7f18cc14f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 -- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18cc153910 msgr2=0x7f18cc153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:22.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18cc153910 0x7f18cc153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 -- 172.21.15.45:0/1037232513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc14af80 con 0x7f18b0004810 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18cbe00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18cc13f4e0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f18bc007fb0 tx=0x7f18bc0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc007740 con 0x7f18b0004810 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc018070 con 0x7f18b0004810 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc13fa20 con 0x7f18b0004810 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00fb80 con 0x7f18b0004810 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc1548e0 con 0x7f18b0004810 2024-04-25T10:30:22.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc04ef30 con 0x7f18b0004810 2024-04-25T10:30:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.147+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18bc00fd20 con 0x7f18b0004810 2024-04-25T10:30:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.151+0000 7f18c8c00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18a8066f60 0x7f18a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:22.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.151+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f18bc084cd0 con 0x7f18b0004810 2024-04-25T10:30:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.151+0000 7f18c3e00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18a8066f60 0x7f18a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:22.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.151+0000 7f18c3e00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18a8066f60 0x7f18a8069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f18b40b0000 tx=0x7f18b40012e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:22.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.155+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18bc055f90 con 0x7f18b0004810 2024-04-25T10:30:22.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:22.319+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f18cc151880 con 0x7f18b0004810 2024-04-25T10:30:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.171+0000 7f18c8c00640 1 -- 172.21.15.45:0/1037232513 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v166) v1 ==== 138+0+0 (secure 0 0 0) 0x7f18bc055890 con 0x7f18b0004810 2024-04-25T10:30:23.172 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:30:23.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18a8066f60 msgr2=0x7f18a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18a8066f60 0x7f18a8069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f18b40b0000 tx=0x7f18b40012e0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 msgr2=0x7f18cc13f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18cc13f4e0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f18bc007fb0 tx=0x7f18bc0074f0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 shutdown_connections 2024-04-25T10:30:23.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f18a8066f60 0x7f18a8069420 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18cc153910 0x7f18cc153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18cc14f550 0x7f18cc14f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 --2- 172.21.15.45:0/1037232513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18b0004810 0x7f18cc13f4e0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 >> 172.21.15.45:0/1037232513 conn(0x7f18cc066b70 msgr2=0x7f18cc15a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 shutdown_connections 2024-04-25T10:30:23.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.175+0000 7f18d2a00640 1 -- 172.21.15.45:0/1037232513 wait complete. 2024-04-25T10:30:23.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 Processor -- start 2024-04-25T10:30:23.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 -- start start 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f354815dfb0 0x7f35480af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35480b1600 0x7f35480cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35480af850 con 0x7f35480b08f0 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35480b0460 con 0x7f354815dfb0 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f354ec00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35480afb30 con 0x7f35480b1600 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f354815dfb0 0x7f35480af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f354815dfb0 0x7f35480af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37012/0 (socket says 172.21.15.45:37012) 2024-04-25T10:30:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 -- 172.21.15.45:0/2828442386 learned_addr learned my addr 172.21.15.45:0/2828442386 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 -- 172.21.15.45:0/2828442386 >> v2:172.21.15.117:3300/0 conn(0x7f35480b1600 msgr2=0x7f35480cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 --2- 172.21.15.45:0/2828442386 >> v2:172.21.15.117:3300/0 conn(0x7f35480b1600 0x7f35480cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 -- 172.21.15.45:0/2828442386 >> v1:172.21.15.45:6789/0 conn(0x7f35480b08f0 legacy=0x7f35480b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.355+0000 7f3547e00640 1 -- 172.21.15.45:0/2828442386 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480afe10 con 0x7f354815dfb0 2024-04-25T10:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f3547e00640 1 --2- 172.21.15.45:0/2828442386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354815dfb0 0x7f35480af220 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f354806e580 tx=0x7f353801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251bc477ef94733 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f3547400640 1 -- 172.21.15.45:0/2828442386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380b5070 con 0x7f354815dfb0 2024-04-25T10:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f3547400640 1 -- 172.21.15.45:0/2828442386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35380a8a10 con 0x7f354815dfb0 2024-04-25T10:30:23.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f3547400640 1 -- 172.21.15.45:0/2828442386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380a8de0 con 0x7f354815dfb0 2024-04-25T10:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 -- 172.21.15.45:0/2828442386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354815dfb0 msgr2=0x7f35480af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 --2- 172.21.15.45:0/2828442386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354815dfb0 0x7f35480af220 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f354806e580 tx=0x7f353801ed10 comp rx=0 tx=0).stop 2024-04-25T10:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 -- 172.21.15.45:0/2828442386 shutdown_connections 2024-04-25T10:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 --2- 172.21.15.45:0/2828442386 >> v2:172.21.15.117:3300/0 conn(0x7f35480b1600 0x7f35480cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 --2- 172.21.15.45:0/2828442386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f354815dfb0 0x7f35480af220 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 -- 172.21.15.45:0/2828442386 >> 172.21.15.45:0/2828442386 conn(0x7f35480b3580 msgr2=0x7f35480b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 -- 172.21.15.45:0/2828442386 shutdown_connections 2024-04-25T10:30:23.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.359+0000 7f354ec00640 1 -- 172.21.15.45:0/2828442386 wait complete. 2024-04-25T10:30:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 Processor -- start 2024-04-25T10:30:23.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 -- start start 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 0x7f35480b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35480b1600 0x7f35480b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354815dfb0 0x7f3548152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354806bfd0 con 0x7f35480b08f0 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354806be50 con 0x7f354815dfb0 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35480b1a90 con 0x7f35480b1600 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 0x7f35480b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 0x7f35480b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37016/0 (socket says 172.21.15.45:37016) 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 -- 172.21.15.45:0/1873481696 learned_addr learned my addr 172.21.15.45:0/1873481696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f353fe00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35480b1600 0x7f35480b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354cc00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354815dfb0 0x7f3548152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 -- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35480b1600 msgr2=0x7f35480b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35480b1600 0x7f35480b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 -- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354815dfb0 msgr2=0x7f3548152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354815dfb0 0x7f3548152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 -- 172.21.15.45:0/1873481696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480b5680 con 0x7f35480b08f0 2024-04-25T10:30:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354cc00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354815dfb0 0x7f3548152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3547e00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 0x7f35480b6490 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f3538000c10 tx=0x7f353801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380bf070 con 0x7f35480b08f0 2024-04-25T10:30:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35380b0840 con 0x7f35480b08f0 2024-04-25T10:30:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35380b5040 con 0x7f35480b08f0 2024-04-25T10:30:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35480b50c0 con 0x7f35480b08f0 2024-04-25T10:30:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.363+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35481578a0 con 0x7f35480b08f0 2024-04-25T10:30:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.367+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354804ef30 con 0x7f35480b08f0 2024-04-25T10:30:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.367+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35380b0c10 con 0x7f35480b08f0 2024-04-25T10:30:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.367+0000 7f3544c00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3524066f60 0x7f3524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:23.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.367+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3538121910 con 0x7f35480b08f0 2024-04-25T10:30:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.367+0000 7f353fe00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3524066f60 0x7f3524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:23.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.367+0000 7f353fe00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3524066f60 0x7f3524069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f35481f8370 tx=0x7f3530009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:23.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.371+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3538018870 con 0x7f35480b08f0 2024-04-25T10:30:23.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:23.535+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f35480b29e0 con 0x7f35480b08f0 2024-04-25T10:30:24.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f3544c00640 1 -- 172.21.15.45:0/1873481696 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v167) v1 ==== 170+0+0 (secure 0 0 0) 0x7f35380f2410 con 0x7f35480b08f0 2024-04-25T10:30:24.176 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:30:24.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3524066f60 msgr2=0x7f3524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3524066f60 0x7f3524069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f35481f8370 tx=0x7f3530009040 comp rx=0 tx=0).stop 2024-04-25T10:30:24.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 msgr2=0x7f35480b6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 0x7f35480b6490 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f3538000c10 tx=0x7f353801bab0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 shutdown_connections 2024-04-25T10:30:24.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3524066f60 0x7f3524069420 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f354815dfb0 0x7f3548152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35480b1600 0x7f35480b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 --2- 172.21.15.45:0/1873481696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35480b08f0 0x7f35480b6490 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 >> 172.21.15.45:0/1873481696 conn(0x7f35480b3580 msgr2=0x7f35480b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:24.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 shutdown_connections 2024-04-25T10:30:24.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.175+0000 7f354ec00640 1 -- 172.21.15.45:0/1873481696 wait complete. 2024-04-25T10:30:24.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:30:24.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 Processor -- start 2024-04-25T10:30:24.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 -- start start 2024-04-25T10:30:24.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fb80aeda0 0x7f4fb8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8072020 0x7f4fb8070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb8159920 con 0x7f4fb8159540 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb815c2f0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fc0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815c5d0 con 0x7f4fb8072020 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbcc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fb80aeda0 0x7f4fb8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbd600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4fb8159540 0x7f4fb80ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48754/0 (socket says 172.21.15.45:48754) 2024-04-25T10:30:24.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbd600640 1 -- 172.21.15.45:0/1901678073 learned_addr learned my addr 172.21.15.45:0/1901678073 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbe000640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8072020 0x7f4fb8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbcc00640 1 -- 172.21.15.45:0/1901678073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8072020 msgr2=0x7f4fb8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbcc00640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8072020 0x7f4fb8070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbcc00640 1 -- 172.21.15.45:0/1901678073 >> v1:172.21.15.45:6789/0 conn(0x7f4fb8159540 legacy=0x7f4fb80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbcc00640 1 -- 172.21.15.45:0/1901678073 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb815c8b0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fbe000640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8072020 0x7f4fb8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:24.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.359+0000 7f4fb7e00640 1 -- 172.21.15.45:0/1901678073 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075823617 0 0) 0x7f4fb8159920 con 0x7f4fb8159540 2024-04-25T10:30:24.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fbcc00640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb8071ae0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097c0 tx=0x7f4fa401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b828ca761b70ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fb7e00640 1 -- 172.21.15.45:0/1901678073 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40aa5f0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fb7e00640 1 -- 172.21.15.45:0/1901678073 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa40af070 con 0x7f4fb80aeda0 2024-04-25T10:30:24.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fb7e00640 1 -- 172.21.15.45:0/1901678073 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40aa9b0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 -- 172.21.15.45:0/1901678073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 msgr2=0x7f4fb8071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb8071ae0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097c0 tx=0x7f4fa401c470 comp rx=0 tx=0).stop 2024-04-25T10:30:24.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 -- 172.21.15.45:0/1901678073 shutdown_connections 2024-04-25T10:30:24.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8072020 0x7f4fb8070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 --2- 172.21.15.45:0/1901678073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb8071ae0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 -- 172.21.15.45:0/1901678073 >> 172.21.15.45:0/1901678073 conn(0x7f4fb80669a0 msgr2=0x7f4fb8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:24.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 -- 172.21.15.45:0/1901678073 shutdown_connections 2024-04-25T10:30:24.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.363+0000 7f4fc0000640 1 -- 172.21.15.45:0/1901678073 wait complete. 2024-04-25T10:30:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 Processor -- start 2024-04-25T10:30:24.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 -- start start 2024-04-25T10:30:24.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fb8072020 0x7f4fb814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb813f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8159540 0x7f4fb8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80710a0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8072430 con 0x7f4fb8072020 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8071220 con 0x7f4fb8159540 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb813f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb813f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37040/0 (socket says 172.21.15.45:37040) 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 -- 172.21.15.45:0/3182399854 learned_addr learned my addr 172.21.15.45:0/3182399854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbd600640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fb8072020 0x7f4fb814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbe000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8159540 0x7f4fb8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 -- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8159540 msgr2=0x7f4fb8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8159540 0x7f4fb8152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 -- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fb8072020 msgr2=0x7f4fb814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fb8072020 0x7f4fb814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 -- 172.21.15.45:0/3182399854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8153720 con 0x7f4fb80aeda0 2024-04-25T10:30:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fbcc00640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb813f080 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097c0 tx=0x7f4fa401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40be610 con 0x7f4fb80aeda0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa40be7d0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40bebc0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb8153a00 con 0x7f4fb80aeda0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.367+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb8153fc0 con 0x7f4fb80aeda0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.371+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb80af860 con 0x7f4fb80aeda0 2024-04-25T10:30:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.371+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fa40bc020 con 0x7f4fb80aeda0 2024-04-25T10:30:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.371+0000 7f4fb5600640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4f98066f60 0x7f4f98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.371+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4fa411fc80 con 0x7f4fb80aeda0 2024-04-25T10:30:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.371+0000 7f4fbd600640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4f98066f60 0x7f4f98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.371+0000 7f4fbd600640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4f98066f60 0x7f4f98069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f70 tx=0x7f4fac007df0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:24.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.375+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fa4016090 con 0x7f4fb80aeda0 2024-04-25T10:30:24.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:24.571+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4fb804ef30 con 0x7f4fb80aeda0 2024-04-25T10:30:25.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.187+0000 7f4fb5600640 1 -- 172.21.15.45:0/3182399854 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v119) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4fa40f07a0 con 0x7f4fb80aeda0 2024-04-25T10:30:25.189 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 24 and data pool 25 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4f98066f60 msgr2=0x7f4f98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4f98066f60 0x7f4f98069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000f70 tx=0x7f4fac007df0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 msgr2=0x7f4fb813f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb813f080 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40097c0 tx=0x7f4fa401acb0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 shutdown_connections 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4f98066f60 0x7f4f98069420 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fb8159540 0x7f4fb8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fb80aeda0 0x7f4fb813f080 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 --2- 172.21.15.45:0/3182399854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fb8072020 0x7f4fb814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 >> 172.21.15.45:0/3182399854 conn(0x7f4fb80669a0 msgr2=0x7f4fb80cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 shutdown_connections 2024-04-25T10:30:25.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.191+0000 7f4fc0000640 1 -- 172.21.15.45:0/3182399854 wait complete. 2024-04-25T10:30:25.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 Processor -- start 2024-04-25T10:30:25.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 -- start start 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ab815e050 0x7f8ab80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80b1850 con 0x7f8ab80d1820 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab80b1af0 con 0x7f8ab815e050 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abf200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b1dd0 con 0x7f8ab80b13e0 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab80cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:40828/0 (socket says 172.21.15.45:40828) 2024-04-25T10:30:25.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 -- 172.21.15.45:0/2134147450 learned_addr learned my addr 172.21.15.45:0/2134147450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8aaf400640 1 --2- 172.21.15.45:0/2134147450 >> v2:172.21.15.117:3300/0 conn(0x7f8ab815e050 0x7f8ab80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 -- 172.21.15.45:0/2134147450 >> v2:172.21.15.117:3300/0 conn(0x7f8ab815e050 msgr2=0x7f8ab80b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 --2- 172.21.15.45:0/2134147450 >> v2:172.21.15.117:3300/0 conn(0x7f8ab815e050 0x7f8ab80b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 -- 172.21.15.45:0/2134147450 >> v1:172.21.15.117:6789/0 conn(0x7f8ab80d1820 legacy=0x7f8ab80d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8abd200640 1 -- 172.21.15.45:0/2134147450 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80b20b0 con 0x7f8ab80b13e0 2024-04-25T10:30:25.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.383+0000 7f8aaf400640 1 --2- 172.21.15.45:0/2134147450 >> v2:172.21.15.117:3300/0 conn(0x7f8ab815e050 0x7f8ab80b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:25.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abd200640 1 --2- 172.21.15.45:0/2134147450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab80cfc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009e50 tx=0x7f8ab001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7035f8880dc92aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8ab7400640 1 -- 172.21.15.45:0/2134147450 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00a5860 con 0x7f8ab80b13e0 2024-04-25T10:30:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8ab7400640 1 -- 172.21.15.45:0/2134147450 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab00a5a00 con 0x7f8ab80b13e0 2024-04-25T10:30:25.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8ab7400640 1 -- 172.21.15.45:0/2134147450 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00a5dd0 con 0x7f8ab80b13e0 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 -- 172.21.15.45:0/2134147450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 msgr2=0x7f8ab80cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 --2- 172.21.15.45:0/2134147450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab80cfc70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009e50 tx=0x7f8ab001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 -- 172.21.15.45:0/2134147450 shutdown_connections 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 --2- 172.21.15.45:0/2134147450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab80cfc70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 --2- 172.21.15.45:0/2134147450 >> v2:172.21.15.117:3300/0 conn(0x7f8ab815e050 0x7f8ab80b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 -- 172.21.15.45:0/2134147450 >> 172.21.15.45:0/2134147450 conn(0x7f8ab809a9d0 msgr2=0x7f8ab809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 -- 172.21.15.45:0/2134147450 shutdown_connections 2024-04-25T10:30:25.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.387+0000 7f8abf200640 1 -- 172.21.15.45:0/2134147450 wait complete. 2024-04-25T10:30:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 Processor -- start 2024-04-25T10:30:25.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 -- start start 2024-04-25T10:30:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 0x7f8ab8158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab815e050 0x7f8ab8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80d0d90 con 0x7f8ab80d1820 2024-04-25T10:30:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80d0c10 con 0x7f8ab815e050 2024-04-25T10:30:25.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80d0f10 con 0x7f8ab80b13e0 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 0x7f8ab8158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 0x7f8ab8158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55670/0 (socket says 172.21.15.45:55670) 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 -- 172.21.15.45:0/2796329941 learned_addr learned my addr 172.21.15.45:0/2796329941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8ab7e00640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abd200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab815e050 0x7f8ab8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 -- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 msgr2=0x7f8ab815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab815a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 -- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab815e050 msgr2=0x7f8ab8159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab815e050 0x7f8ab8159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 -- 172.21.15.45:0/2796329941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab815ab80 con 0x7f8ab80d1820 2024-04-25T10:30:25.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8aaf400640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 0x7f8ab8158e10 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000ebe0 tx=0x7f8aa000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0017070 con 0x7f8ab80d1820 2024-04-25T10:30:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa000cc30 con 0x7f8ab80d1820 2024-04-25T10:30:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0013470 con 0x7f8ab80d1820 2024-04-25T10:30:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab815bec0 con 0x7f8ab80d1820 2024-04-25T10:30:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.391+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab815b100 con 0x7f8ab80d1820 2024-04-25T10:30:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.395+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab804ef30 con 0x7f8ab80d1820 2024-04-25T10:30:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.395+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8aa001a070 con 0x7f8ab80d1820 2024-04-25T10:30:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.395+0000 7f8ab4c00640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a94066f90 0x7f8a94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.395+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aa0084f70 con 0x7f8ab80d1820 2024-04-25T10:30:25.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.395+0000 7f8ab7e00640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a94066f90 0x7f8a94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.399+0000 7f8ab7e00640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a94066f90 0x7f8a94069450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8ab81fa2d0 tx=0x7f8aa8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.399+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8aa0055890 con 0x7f8ab80d1820 2024-04-25T10:30:25.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.563+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ab80d1550 con 0x7f8ab80d1820 2024-04-25T10:30:25.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8ab4c00640 1 -- 172.21.15.45:0/2796329941 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16624 (secure 0 0 0) 0x7f8aa0059930 con 0x7f8ab80d1820 2024-04-25T10:30:25.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:25.568 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":168,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:24.574058+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a94066f90 msgr2=0x7f8a94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a94066f90 0x7f8a94069450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8ab81fa2d0 tx=0x7f8aa8009040 comp rx=0 tx=0).stop 2024-04-25T10:30:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 msgr2=0x7f8ab8158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 0x7f8ab8158e10 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000ebe0 tx=0x7f8aa000c520 comp rx=0 tx=0).stop 2024-04-25T10:30:25.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 shutdown_connections 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a94066f90 0x7f8a94069450 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab815e050 0x7f8ab8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ab80d1820 0x7f8ab8158e10 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 --2- 172.21.15.45:0/2796329941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ab80b13e0 0x7f8ab815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 >> 172.21.15.45:0/2796329941 conn(0x7f8ab809a9d0 msgr2=0x7f8ab80ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 shutdown_connections 2024-04-25T10:30:25.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.567+0000 7f8abf200640 1 -- 172.21.15.45:0/2796329941 wait complete. 2024-04-25T10:30:25.590 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:25.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 Processor -- start 2024-04-25T10:30:25.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 -- start start 2024-04-25T10:30:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9de81550b0 0x7f9de80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 0x7f9de80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de8071050 con 0x7f9de80cfe70 2024-04-25T10:30:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de815bda0 con 0x7f9de81550b0 2024-04-25T10:30:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9dee400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815d040 con 0x7f9de815b910 2024-04-25T10:30:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 0x7f9de80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9ddea00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9de80cfe70 0x7f9de815b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:44110/0 (socket says 172.21.15.45:44110) 2024-04-25T10:30:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 0x7f9de80b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:40840/0 (socket says 172.21.15.45:40840) 2024-04-25T10:30:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9ddea00640 1 -- 172.21.15.45:0/3508306076 learned_addr learned my addr 172.21.15.45:0/3508306076 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7400640 1 --2- 172.21.15.45:0/3508306076 >> v2:172.21.15.117:3300/0 conn(0x7f9de81550b0 0x7f9de80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de6a00640 1 -- 172.21.15.45:0/3508306076 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935527962 0 0) 0x7f9de8071050 con 0x7f9de80cfe70 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de6a00640 1 -- 172.21.15.45:0/3508306076 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dcc003530 con 0x7f9de80cfe70 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7e00640 1 -- 172.21.15.45:0/3508306076 >> v2:172.21.15.117:3300/0 conn(0x7f9de81550b0 msgr2=0x7f9de80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7e00640 1 --2- 172.21.15.45:0/3508306076 >> v2:172.21.15.117:3300/0 conn(0x7f9de81550b0 0x7f9de80cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7e00640 1 -- 172.21.15.45:0/3508306076 >> v1:172.21.15.117:6789/0 conn(0x7f9de80cfe70 legacy=0x7f9de815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7e00640 1 -- 172.21.15.45:0/3508306076 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80712f0 con 0x7f9de815b910 2024-04-25T10:30:25.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.731+0000 7f9de7400640 1 --2- 172.21.15.45:0/3508306076 >> v2:172.21.15.117:3300/0 conn(0x7f9de81550b0 0x7f9de80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:25.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9de7e00640 1 --2- 172.21.15.45:0/3508306076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 0x7f9de80b3260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9de000ba90 tx=0x7f9de001eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e564131284b3b848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9de6a00640 1 -- 172.21.15.45:0/3508306076 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b4070 con 0x7f9de815b910 2024-04-25T10:30:25.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9de6a00640 1 -- 172.21.15.45:0/3508306076 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00a7b30 con 0x7f9de815b910 2024-04-25T10:30:25.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9de6a00640 1 -- 172.21.15.45:0/3508306076 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00af440 con 0x7f9de815b910 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 -- 172.21.15.45:0/3508306076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 msgr2=0x7f9de80b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 --2- 172.21.15.45:0/3508306076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 0x7f9de80b3260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9de000ba90 tx=0x7f9de001eb70 comp rx=0 tx=0).stop 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 -- 172.21.15.45:0/3508306076 shutdown_connections 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 --2- 172.21.15.45:0/3508306076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de815b910 0x7f9de80b3260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 --2- 172.21.15.45:0/3508306076 >> v2:172.21.15.117:3300/0 conn(0x7f9de81550b0 0x7f9de80cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 -- 172.21.15.45:0/3508306076 >> 172.21.15.45:0/3508306076 conn(0x7f9de8066b70 msgr2=0x7f9de8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 -- 172.21.15.45:0/3508306076 shutdown_connections 2024-04-25T10:30:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.735+0000 7f9dee400640 1 -- 172.21.15.45:0/3508306076 wait complete. 2024-04-25T10:30:25.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 Processor -- start 2024-04-25T10:30:25.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 -- start start 2024-04-25T10:30:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80cfe70 0x7f9de8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 0x7f9de814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de815b910 0x7f9de81481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815c030 con 0x7f9de81550b0 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80720b0 con 0x7f9de815b910 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815c1b0 con 0x7f9de80cfe70 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 0x7f9de814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 0x7f9de814b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55674/0 (socket says 172.21.15.45:55674) 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 -- 172.21.15.45:0/1295177861 learned_addr learned my addr 172.21.15.45:0/1295177861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9de7e00640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de815b910 0x7f9de81481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9de7400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80cfe70 0x7f9de8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 -- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80cfe70 msgr2=0x7f9de8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80cfe70 0x7f9de8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 -- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de815b910 msgr2=0x7f9de81481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de815b910 0x7f9de81481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 -- 172.21.15.45:0/1295177861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de806dfe0 con 0x7f9de81550b0 2024-04-25T10:30:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddea00640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 0x7f9de814b0f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000cb70 tx=0x7f9dd000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0017070 con 0x7f9de81550b0 2024-04-25T10:30:25.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd000ad40 con 0x7f9de81550b0 2024-04-25T10:30:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0012620 con 0x7f9de81550b0 2024-04-25T10:30:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.739+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de8158f40 con 0x7f9de81550b0 2024-04-25T10:30:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de806d6b0 con 0x7f9de81550b0 2024-04-25T10:30:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de804ef30 con 0x7f9de81550b0 2024-04-25T10:30:25.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dd001a070 con 0x7f9de81550b0 2024-04-25T10:30:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9ddfe00640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc4066f90 0x7f9dc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd0084ca0 con 0x7f9de81550b0 2024-04-25T10:30:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9de7400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc4066f90 0x7f9dc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:25.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.743+0000 7f9de7400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc4066f90 0x7f9dc4069450 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9de81f5a00 tx=0x7f9dd8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.747+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dd00556f0 con 0x7f9de81550b0 2024-04-25T10:30:25.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9de80cf120 con 0x7f9de81550b0 2024-04-25T10:30:25.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9ddfe00640 1 -- 172.21.15.45:0/1295177861 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16624 (secure 0 0 0) 0x7f9dd0059790 con 0x7f9de81550b0 2024-04-25T10:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:25.913 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":168,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:24.574058+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc4066f90 msgr2=0x7f9dc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc4066f90 0x7f9dc4069450 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9de81f5a00 tx=0x7f9dd8009190 comp rx=0 tx=0).stop 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 msgr2=0x7f9de814b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 0x7f9de814b0f0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000cb70 tx=0x7f9dd000a430 comp rx=0 tx=0).stop 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 shutdown_connections 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc4066f90 0x7f9dc4069450 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de815b910 0x7f9de81481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de81550b0 0x7f9de814b0f0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 --2- 172.21.15.45:0/1295177861 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80cfe70 0x7f9de8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 >> 172.21.15.45:0/1295177861 conn(0x7f9de8066b70 msgr2=0x7f9de814f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 shutdown_connections 2024-04-25T10:30:25.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:25.911+0000 7f9dee400640 1 -- 172.21.15.45:0/1295177861 wait complete. 2024-04-25T10:30:25.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:30:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f0106200640 1 Processor -- start 2024-04-25T10:30:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f0106200640 1 -- start start 2024-04-25T10:30:26.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f0106200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01000b38d0 0x7f010015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f0106200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015b080 con 0x7f01000aef80 2024-04-25T10:30:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f0106200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015adb0 con 0x7f0100070350 2024-04-25T10:30:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f0106200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010015eb60 con 0x7f01000b38d0 2024-04-25T10:30:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f00ff400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01000aef80 0x7f010015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33602/0 (socket says 172.21.15.45:33602) 2024-04-25T10:30:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f00ff400640 1 -- 172.21.15.45:0/1948726603 learned_addr learned my addr 172.21.15.45:0/1948726603 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f00ffe00640 1 --2- 172.21.15.45:0/1948726603 >> v2:172.21.15.117:3300/0 conn(0x7f01000b38d0 0x7f010015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f00fea00640 1 -- 172.21.15.45:0/1948726603 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074801496 0 0) 0x7f010015b080 con 0x7f01000aef80 2024-04-25T10:30:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.079+0000 7f00fea00640 1 -- 172.21.15.45:0/1948726603 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e4005740 con 0x7f01000aef80 2024-04-25T10:30:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00fea00640 1 -- 172.21.15.45:0/1948726603 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758886150 0 0) 0x7f010015adb0 con 0x7f0100070350 2024-04-25T10:30:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00ffe00640 1 -- 172.21.15.45:0/1948726603 >> v1:172.21.15.117:6789/0 conn(0x7f0100070350 legacy=0x7f01000b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00ffe00640 1 -- 172.21.15.45:0/1948726603 >> v1:172.21.15.45:6789/0 conn(0x7f01000aef80 legacy=0x7f010015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00ffe00640 1 -- 172.21.15.45:0/1948726603 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010015ee40 con 0x7f01000b38d0 2024-04-25T10:30:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00ffe00640 1 --2- 172.21.15.45:0/1948726603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01000b38d0 0x7f010015e480 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009d90 tx=0x7f00f801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36ef92fe797974b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00fea00640 1 -- 172.21.15.45:0/1948726603 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80b2070 con 0x7f01000b38d0 2024-04-25T10:30:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00fea00640 1 -- 172.21.15.45:0/1948726603 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80a5a10 con 0x7f01000b38d0 2024-04-25T10:30:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.083+0000 7f00fea00640 1 -- 172.21.15.45:0/1948726603 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80a5dc0 con 0x7f01000b38d0 2024-04-25T10:30:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- 172.21.15.45:0/1948726603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01000b38d0 msgr2=0x7f010015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 --2- 172.21.15.45:0/1948726603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01000b38d0 0x7f010015e480 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009d90 tx=0x7f00f801c970 comp rx=0 tx=0).stop 2024-04-25T10:30:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- 172.21.15.45:0/1948726603 shutdown_connections 2024-04-25T10:30:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 --2- 172.21.15.45:0/1948726603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01000b38d0 0x7f010015e480 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- 172.21.15.45:0/1948726603 >> 172.21.15.45:0/1948726603 conn(0x7f01000669a0 msgr2=0x7f0100066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- 172.21.15.45:0/1948726603 shutdown_connections 2024-04-25T10:30:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- 172.21.15.45:0/1948726603 wait complete. 2024-04-25T10:30:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 Processor -- start 2024-04-25T10:30:26.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- start start 2024-04-25T10:30:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0100070350 0x7f01000b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 0x7f0100154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f010015f850 0x7f010015fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010006c250 con 0x7f01000aef80 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010006c0d0 con 0x7f0100070350 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010006c3d0 con 0x7f010015f850 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 0x7f0100154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 0x7f0100154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55684/0 (socket says 172.21.15.45:55684) 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 -- 172.21.15.45:0/4191503673 learned_addr learned my addr 172.21.15.45:0/4191503673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00ff400640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0100070350 0x7f01000b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00ffe00640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f010015f850 0x7f010015fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 -- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f010015f850 msgr2=0x7f010015fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f010015f850 0x7f010015fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 -- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0100070350 msgr2=0x7f01000b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0100070350 0x7f01000b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 -- 172.21.15.45:0/4191503673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000b0c60 con 0x7f01000aef80 2024-04-25T10:30:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00ff400640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0100070350 0x7f01000b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00ffe00640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f010015f850 0x7f010015fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7e00640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 0x7f0100154bf0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f00e800c500 tx=0x7f00e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8017070 con 0x7f01000aef80 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e8003600 con 0x7f01000aef80 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e80039f0 con 0x7f01000aef80 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.087+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01000b0f40 con 0x7f01000aef80 2024-04-25T10:30:26.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.091+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000b1500 con 0x7f01000aef80 2024-04-25T10:30:26.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.091+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010004ef30 con 0x7f01000aef80 2024-04-25T10:30:26.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.091+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00e801a070 con 0x7f01000aef80 2024-04-25T10:30:26.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.091+0000 7f00f7400640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f00dc066f60 0x7f00dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:26.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.091+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00e8084450 con 0x7f01000aef80 2024-04-25T10:30:26.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.091+0000 7f00ff400640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f00dc066f60 0x7f00dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:26.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.095+0000 7f00ff400640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f00dc066f60 0x7f00dc069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f00f0005890 tx=0x7f00f0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:26.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.095+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00e80555b0 con 0x7f01000aef80 2024-04-25T10:30:26.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:30:26.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:30:26.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:30:26.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:26.295+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f010015e480 con 0x7f01000aef80 2024-04-25T10:30:27.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.207+0000 7f00f7400640 1 -- 172.21.15.45:0/4191503673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v121) v1 ==== 118+0+0 (secure 0 0 0) 0x7f00e8054eb0 con 0x7f01000aef80 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f00dc066f60 msgr2=0x7f00dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f00dc066f60 0x7f00dc069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f00f0005890 tx=0x7f00f0009040 comp rx=0 tx=0).stop 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 msgr2=0x7f0100154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 0x7f0100154bf0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f00e800c500 tx=0x7f00e800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 shutdown_connections 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f00dc066f60 0x7f00dc069420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f010015f850 0x7f010015fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01000aef80 0x7f0100154bf0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 --2- 172.21.15.45:0/4191503673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0100070350 0x7f01000b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 >> 172.21.15.45:0/4191503673 conn(0x7f01000669a0 msgr2=0x7f0100066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:27.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 shutdown_connections 2024-04-25T10:30:27.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.211+0000 7f0106200640 1 -- 172.21.15.45:0/4191503673 wait complete. 2024-04-25T10:30:27.236 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:30:27.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:30:27.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 Processor -- start 2024-04-25T10:30:27.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 -- start start 2024-04-25T10:30:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73080d0d00 0x7f73080ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 0x7f730806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080cdab0 con 0x7f73080cc200 2024-04-25T10:30:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730806d8f0 con 0x7f73080d0d00 2024-04-25T10:30:27.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.383+0000 7f730ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080cd210 con 0x7f730806cf40 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 0x7f730806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 0x7f730806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:40860/0 (socket says 172.21.15.45:40860) 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 -- 172.21.15.45:0/3417631755 learned_addr learned my addr 172.21.15.45:0/3417631755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f72ffe00640 1 --1- 172.21.15.45:0/3417631755 >> v1:172.21.15.117:6789/0 conn(0x7f73080cc200 0x7f7308162180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:44136/0 (socket says 172.21.15.45:44136) 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f7307e00640 1 --2- 172.21.15.45:0/3417631755 >> v2:172.21.15.117:3300/0 conn(0x7f73080d0d00 0x7f73080ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 -- 172.21.15.45:0/3417631755 >> v2:172.21.15.117:3300/0 conn(0x7f73080d0d00 msgr2=0x7f73080ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 --2- 172.21.15.45:0/3417631755 >> v2:172.21.15.117:3300/0 conn(0x7f73080d0d00 0x7f73080ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 -- 172.21.15.45:0/3417631755 >> v1:172.21.15.117:6789/0 conn(0x7f73080cc200 legacy=0x7f7308162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 -- 172.21.15.45:0/3417631755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080cd4f0 con 0x7f730806cf40 2024-04-25T10:30:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f7307e00640 1 --2- 172.21.15.45:0/3417631755 >> v2:172.21.15.117:3300/0 conn(0x7f73080d0d00 0x7f73080ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:27.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.387+0000 7f730ca00640 1 --2- 172.21.15.45:0/3417631755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 0x7f730806d3b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7300009e50 tx=0x7f730001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f02b7bb5b07e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f7307400640 1 -- 172.21.15.45:0/3417631755 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000a5750 con 0x7f730806cf40 2024-04-25T10:30:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f7307400640 1 -- 172.21.15.45:0/3417631755 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73000a58f0 con 0x7f730806cf40 2024-04-25T10:30:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f7307400640 1 -- 172.21.15.45:0/3417631755 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000a5cc0 con 0x7f730806cf40 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- 172.21.15.45:0/3417631755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 msgr2=0x7f730806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 --2- 172.21.15.45:0/3417631755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 0x7f730806d3b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7300009e50 tx=0x7f730001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- 172.21.15.45:0/3417631755 shutdown_connections 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 --2- 172.21.15.45:0/3417631755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f730806cf40 0x7f730806d3b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 --2- 172.21.15.45:0/3417631755 >> v2:172.21.15.117:3300/0 conn(0x7f73080d0d00 0x7f73080ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- 172.21.15.45:0/3417631755 >> 172.21.15.45:0/3417631755 conn(0x7f7308072290 msgr2=0x7f73080726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- 172.21.15.45:0/3417631755 shutdown_connections 2024-04-25T10:30:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- 172.21.15.45:0/3417631755 wait complete. 2024-04-25T10:30:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 Processor -- start 2024-04-25T10:30:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- start start 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 0x7f7308154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73080cc200 0x7f730815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73080d0d00 0x7f73080cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308162a10 con 0x7f730806cf40 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7308162890 con 0x7f73080d0d00 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f730ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730815e040 con 0x7f73080cc200 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f7307e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 0x7f7308154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f7307e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 0x7f7308154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55698/0 (socket says 172.21.15.45:55698) 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.391+0000 7f7307e00640 1 -- 172.21.15.45:0/2247711065 learned_addr learned my addr 172.21.15.45:0/2247711065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7307e00640 1 -- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73080cc200 msgr2=0x7f730815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7307e00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73080cc200 0x7f730815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7307e00640 1 -- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73080d0d00 msgr2=0x7f73080cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7307e00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73080d0d00 0x7f73080cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7307e00640 1 -- 172.21.15.45:0/2247711065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308160b90 con 0x7f730806cf40 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7307e00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 0x7f7308154e50 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f72f800e980 tx=0x7f72f800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8017070 con 0x7f730806cf40 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f800ca00 con 0x7f730806cf40 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7308160e70 con 0x7f730806cf40 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800cdd0 con 0x7f730806cf40 2024-04-25T10:30:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7308161430 con 0x7f730806cf40 2024-04-25T10:30:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730804ef30 con 0x7f730806cf40 2024-04-25T10:30:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72f801a070 con 0x7f730806cf40 2024-04-25T10:30:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7304c00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72e4066f60 0x7f72e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:27.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.395+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72f80852e0 con 0x7f730806cf40 2024-04-25T10:30:27.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.399+0000 7f72ffe00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72e4066f60 0x7f72e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:27.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.399+0000 7f72ffe00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72e4066f60 0x7f72e4069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f72f0000f70 tx=0x7f72f0007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:27.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.399+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72f8056250 con 0x7f730806cf40 2024-04-25T10:30:27.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:27.599+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f73080ada20 con 0x7f730806cf40 2024-04-25T10:30:28.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.207+0000 7f7304c00640 1 -- 172.21.15.45:0/2247711065 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v122) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72f8055b50 con 0x7f730806cf40 2024-04-25T10:30:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72e4066f60 msgr2=0x7f72e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72e4066f60 0x7f72e4069420 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f72f0000f70 tx=0x7f72f0007a10 comp rx=0 tx=0).stop 2024-04-25T10:30:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 msgr2=0x7f7308154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 0x7f7308154e50 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f72f800e980 tx=0x7f72f800ee40 comp rx=0 tx=0).stop 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 shutdown_connections 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f72e4066f60 0x7f72e4069420 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73080d0d00 0x7f73080cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73080cc200 0x7f730815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 --2- 172.21.15.45:0/2247711065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f730806cf40 0x7f7308154e50 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 >> 172.21.15.45:0/2247711065 conn(0x7f7308072290 msgr2=0x7f73080adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 shutdown_connections 2024-04-25T10:30:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.211+0000 7f730ea00640 1 -- 172.21.15.45:0/2247711065 wait complete. 2024-04-25T10:30:28.237 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:30:28.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.379+0000 7f4a83600640 1 Processor -- start 2024-04-25T10:30:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.379+0000 7f4a83600640 1 -- start start 2024-04-25T10:30:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a83600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a7c066c00 0x7f4a7c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a83600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066fe0 0x7f4a7c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a83600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7c0b5c20 con 0x7f4a7c06ed30 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a83600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c0af6e0 con 0x7f4a7c066c00 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a83600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0af9c0 con 0x7f4a7c066fe0 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a7c066c00 0x7f4a7c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4a7c066c00 0x7f4a7c15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55708/0 (socket says 172.21.15.45:55708) 2024-04-25T10:30:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 -- 172.21.15.45:0/3655083806 learned_addr learned my addr 172.21.15.45:0/3655083806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a73e00640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066fe0 0x7f4a7c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 -- 172.21.15.45:0/3655083806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066fe0 msgr2=0x7f4a7c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066fe0 0x7f4a7c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 -- 172.21.15.45:0/3655083806 >> v1:172.21.15.117:6789/0 conn(0x7f4a7c06ed30 legacy=0x7f4a7c157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a80c00640 1 -- 172.21.15.45:0/3655083806 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0afca0 con 0x7f4a7c066c00 2024-04-25T10:30:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.383+0000 7f4a73e00640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066fe0 0x7f4a7c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a80c00640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c066c00 0x7f4a7c15e110 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009870 tx=0x7f4a6c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7e1f567c83d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a7be00640 1 -- 172.21.15.45:0/3655083806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0aa430 con 0x7f4a7c066c00 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a7be00640 1 -- 172.21.15.45:0/3655083806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c0af070 con 0x7f4a7c066c00 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a7be00640 1 -- 172.21.15.45:0/3655083806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0aa7f0 con 0x7f4a7c066c00 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- 172.21.15.45:0/3655083806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c066c00 msgr2=0x7f4a7c15e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c066c00 0x7f4a7c15e110 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009870 tx=0x7f4a6c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- 172.21.15.45:0/3655083806 shutdown_connections 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066fe0 0x7f4a7c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 --2- 172.21.15.45:0/3655083806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c066c00 0x7f4a7c15e110 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- 172.21.15.45:0/3655083806 >> 172.21.15.45:0/3655083806 conn(0x7f4a7c0b24f0 msgr2=0x7f4a7c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- 172.21.15.45:0/3655083806 shutdown_connections 2024-04-25T10:30:28.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- 172.21.15.45:0/3655083806 wait complete. 2024-04-25T10:30:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 Processor -- start 2024-04-25T10:30:28.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- start start 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066c00 0x7f4a7c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a7c066fe0 0x7f4a7c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c06ed30 0x7f4a7c14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c06ac90 con 0x7f4a7c06ed30 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0af270 con 0x7f4a7c066fe0 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.387+0000 7f4a83600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c06ae10 con 0x7f4a7c066c00 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c06ed30 0x7f4a7c14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a80c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066c00 0x7f4a7c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a7c066fe0 0x7f4a7c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a80c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066c00 0x7f4a7c14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:40896/0 (socket says 172.21.15.45:40896) 2024-04-25T10:30:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a80c00640 1 -- 172.21.15.45:0/4235132935 learned_addr learned my addr 172.21.15.45:0/4235132935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 -- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066c00 msgr2=0x7f4a7c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066c00 0x7f4a7c14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 -- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a7c066fe0 msgr2=0x7f4a7c0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a7c066fe0 0x7f4a7c0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 -- 172.21.15.45:0/4235132935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c14eef0 con 0x7f4a7c06ed30 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a81600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c06ed30 0x7f4a7c14e0d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400be00 tx=0x7f4a7400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74003980 con 0x7f4a7c06ed30 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a74003b20 con 0x7f4a7c06ed30 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74004020 con 0x7f4a7c06ed30 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7c14f1d0 con 0x7f4a7c06ed30 2024-04-25T10:30:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c14f790 con 0x7f4a7c06ed30 2024-04-25T10:30:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7c04ef30 con 0x7f4a7c06ed30 2024-04-25T10:30:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a74006d10 con 0x7f4a7c06ed30 2024-04-25T10:30:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a79600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a58066f60 0x7f4a58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.391+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a74084de0 con 0x7f4a7c06ed30 2024-04-25T10:30:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.395+0000 7f4a80c00640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a58066f60 0x7f4a58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.395+0000 7f4a80c00640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a58066f60 0x7f4a58069420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009870 tx=0x7f4a6c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.399+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a74055700 con 0x7f4a7c06ed30 2024-04-25T10:30:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.591+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a7c15ec60 con 0x7f4a7c06ed30 2024-04-25T10:30:28.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.591+0000 7f4a79600640 1 -- 172.21.15.45:0/4235132935 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7f4a740597a0 con 0x7f4a7c06ed30 2024-04-25T10:30:28.595 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:28.595 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:30:28.596 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:30:28.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a58066f60 msgr2=0x7f4a58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a58066f60 0x7f4a58069420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009870 tx=0x7f4a6c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:30:28.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c06ed30 msgr2=0x7f4a7c14e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c06ed30 0x7f4a7c14e0d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400be00 tx=0x7f4a7400bef0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 shutdown_connections 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4a58066f60 0x7f4a58069420 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a7c06ed30 0x7f4a7c14e0d0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a7c066fe0 0x7f4a7c0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 --2- 172.21.15.45:0/4235132935 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a7c066c00 0x7f4a7c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 >> 172.21.15.45:0/4235132935 conn(0x7f4a7c0b24f0 msgr2=0x7f4a7c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 shutdown_connections 2024-04-25T10:30:28.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.595+0000 7f4a83600640 1 -- 172.21.15.45:0/4235132935 wait complete. 2024-04-25T10:30:28.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:30:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.755+0000 7fd51f200640 1 Processor -- start 2024-04-25T10:30:28.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.755+0000 7fd51f200640 1 -- start start 2024-04-25T10:30:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200b1f50 0x7fd5200b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51f200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52006be90 con 0x7fd5200cd780 2024-04-25T10:30:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52006b8d0 con 0x7fd5200b3dd0 2024-04-25T10:30:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200d0b50 con 0x7fd5200b1f50 2024-04-25T10:30:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd5200cd780 0x7fd5200cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33616/0 (socket says 172.21.15.45:33616) 2024-04-25T10:30:28.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51e800640 1 -- 172.21.15.45:0/520533864 learned_addr learned my addr 172.21.15.45:0/520533864 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:28.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51de00640 1 --2- 172.21.15.45:0/520533864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200b1f50 0x7fd5200b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947067249 0 0) 0x7fd518001120 con 0x7fd5200cd780 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd504003530 con 0x7fd5200cd780 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40893654 0 0) 0x7fd52006b8d0 con 0x7fd5200b3dd0 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd518001120 con 0x7fd5200b3dd0 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51de00640 1 -- 172.21.15.45:0/520533864 >> v1:172.21.15.117:6789/0 conn(0x7fd5200b3dd0 legacy=0x7fd52006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941493140 0 0) 0x7fd504003530 con 0x7fd5200cd780 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51de00640 1 -- 172.21.15.45:0/520533864 >> v1:172.21.15.45:6789/0 conn(0x7fd5200cd780 legacy=0x7fd5200cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.759+0000 7fd51de00640 1 -- 172.21.15.45:0/520533864 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52006bbb0 con 0x7fd5200b1f50 2024-04-25T10:30:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51de00640 1 --2- 172.21.15.45:0/520533864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200b1f50 0x7fd5200b23c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd510009dc0 tx=0x7fd51001c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb08128311e76ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518001850 con 0x7fd5200b1f50 2024-04-25T10:30:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100a8440 con 0x7fd5200b1f50 2024-04-25T10:30:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51d400640 1 -- 172.21.15.45:0/520533864 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100a87f0 con 0x7fd5200b1f50 2024-04-25T10:30:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 -- 172.21.15.45:0/520533864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200b1f50 msgr2=0x7fd5200b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 --2- 172.21.15.45:0/520533864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200b1f50 0x7fd5200b23c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd510009dc0 tx=0x7fd51001c790 comp rx=0 tx=0).stop 2024-04-25T10:30:28.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 -- 172.21.15.45:0/520533864 shutdown_connections 2024-04-25T10:30:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 --2- 172.21.15.45:0/520533864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200b1f50 0x7fd5200b23c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 -- 172.21.15.45:0/520533864 >> 172.21.15.45:0/520533864 conn(0x7fd5200b32b0 msgr2=0x7fd5200b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 -- 172.21.15.45:0/520533864 shutdown_connections 2024-04-25T10:30:28.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 -- 172.21.15.45:0/520533864 wait complete. 2024-04-25T10:30:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 Processor -- start 2024-04-25T10:30:28.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.763+0000 7fd51f200640 1 -- start start 2024-04-25T10:30:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 0x7fd52015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5200b3dd0 0x7fd52015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200cd780 0x7fd5200b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52016c230 con 0x7fd5200b1f50 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200d0550 con 0x7fd5200b3dd0 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52016c3b0 con 0x7fd5200cd780 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 0x7fd52015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 0x7fd52015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55724/0 (socket says 172.21.15.45:55724) 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 -- 172.21.15.45:0/3719244867 learned_addr learned my addr 172.21.15.45:0/3719244867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51e800640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200cd780 0x7fd5200b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd515400640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5200b3dd0 0x7fd52015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 -- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200cd780 msgr2=0x7fd5200b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200cd780 0x7fd5200b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 -- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5200b3dd0 msgr2=0x7fd52015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5200b3dd0 0x7fd52015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 -- 172.21.15.45:0/3719244867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200b6b40 con 0x7fd5200b1f50 2024-04-25T10:30:28.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51de00640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 0x7fd52015a1f0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd51001cc70 tx=0x7fd510019530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100c0070 con 0x7fd5200b1f50 2024-04-25T10:30:28.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100bb670 con 0x7fd5200b1f50 2024-04-25T10:30:28.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510009450 con 0x7fd5200b1f50 2024-04-25T10:30:28.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd520153d80 con 0x7fd5200b1f50 2024-04-25T10:30:28.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52015c5d0 con 0x7fd5200b1f50 2024-04-25T10:30:28.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.767+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520075120 con 0x7fd5200b1f50 2024-04-25T10:30:28.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.771+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5100c3070 con 0x7fd5200b1f50 2024-04-25T10:30:28.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.771+0000 7fd516a00640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:28.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.771+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd51011fb00 con 0x7fd5200b1f50 2024-04-25T10:30:28.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.771+0000 7fd515400640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:28.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.771+0000 7fd515400640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd4fc066f60 0x7fd4fc069420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd52015d3f0 tx=0x7fd508009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:28.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.775+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5100ac030 con 0x7fd5200b1f50 2024-04-25T10:30:28.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.939+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5200d2370 con 0x7fd5200b1f50 2024-04-25T10:30:28.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.939+0000 7fd516a00640 1 -- 172.21.15.45:0/3719244867 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16611 (secure 0 0 0) 0x7fd5100164a0 con 0x7fd5200b1f50 2024-04-25T10:30:28.943 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:28.943 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":169,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:26.190418+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd4fc066f60 msgr2=0x7fd4fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd4fc066f60 0x7fd4fc069420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd52015d3f0 tx=0x7fd508009040 comp rx=0 tx=0).stop 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 msgr2=0x7fd52015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 0x7fd52015a1f0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd51001cc70 tx=0x7fd510019530 comp rx=0 tx=0).stop 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 shutdown_connections 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5200cd780 0x7fd5200b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5200b3dd0 0x7fd52015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 --2- 172.21.15.45:0/3719244867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5200b1f50 0x7fd52015a1f0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 >> 172.21.15.45:0/3719244867 conn(0x7fd5200b32b0 msgr2=0x7fd520068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 shutdown_connections 2024-04-25T10:30:28.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:28.943+0000 7fd51f200640 1 -- 172.21.15.45:0/3719244867 wait complete. 2024-04-25T10:30:28.965 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:30:28.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.103+0000 7f4493800640 1 Processor -- start 2024-04-25T10:30:29.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.103+0000 7f4493800640 1 -- start start 2024-04-25T10:30:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4493800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 0x7f448c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4493800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c0b2770 con 0x7f448c15bc40 2024-04-25T10:30:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4493800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c06c670 con 0x7f448c0cda10 2024-04-25T10:30:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4493800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06c950 con 0x7f448c0d0040 2024-04-25T10:30:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f448c15bc40 0x7f448c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33620/0 (socket says 172.21.15.45:33620) 2024-04-25T10:30:29.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 -- 172.21.15.45:0/1885466315 learned_addr learned my addr 172.21.15.45:0/1885466315 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:29.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4491800640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 0x7f448c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47783736 0 0) 0x7f448c0b2770 con 0x7f448c15bc40 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4470003530 con 0x7f448c15bc40 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189488418 0 0) 0x7f448c06c670 con 0x7f448c0cda10 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f448c0b2770 con 0x7f448c0cda10 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 700690873 0 0) 0x7f4470003530 con 0x7f448c15bc40 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 msgr2=0x7f448c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 0x7f448c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 >> v1:172.21.15.117:6789/0 conn(0x7f448c0cda10 legacy=0x7f448c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c06cc30 con 0x7f448c15bc40 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f447c001680 con 0x7f448c15bc40 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 >> v1:172.21.15.45:6789/0 conn(0x7f448c15bc40 legacy=0x7f448c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4470003b80 0x7f4470003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 0x7f4470004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470004d50 0x7f44700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c06c950 con 0x7f4470003b80 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447000bb00 con 0x7f4470004d50 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447000bd00 con 0x7f448c0d0040 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4470003b80 0x7f4470003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4483e00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 0x7f4470004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4491800640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470004d50 0x7f44700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 -- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 msgr2=0x7f4470004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c0d0040 0x7f4470004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 -- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470004d50 msgr2=0x7f44700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470004d50 0x7f44700051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.107+0000 7f4490e00640 1 -- 172.21.15.45:0/1885466315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470003530 con 0x7f4470003b80 2024-04-25T10:30:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4490e00640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4470003b80 0x7f4470003f80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f447c00c190 tx=0x7f447c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5491bb4b08e9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c007cd0 con 0x7f4470003b80 2024-04-25T10:30:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0a9020 con 0x7f4470003b80 2024-04-25T10:30:29.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f448be00640 1 -- 172.21.15.45:0/1885466315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c005ae0 con 0x7f4470003b80 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 -- 172.21.15.45:0/1885466315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4470003b80 msgr2=0x7f4470003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 --2- 172.21.15.45:0/1885466315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4470003b80 0x7f4470003f80 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f447c00c190 tx=0x7f447c01cc20 comp rx=0 tx=0).stop 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4491800640 1 -- 172.21.15.45:0/1885466315 reap_dead start 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 -- 172.21.15.45:0/1885466315 shutdown_connections 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 -- 172.21.15.45:0/1885466315 >> 172.21.15.45:0/1885466315 conn(0x7f448c066b70 msgr2=0x7f448c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 -- 172.21.15.45:0/1885466315 shutdown_connections 2024-04-25T10:30:29.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 -- 172.21.15.45:0/1885466315 wait complete. 2024-04-25T10:30:29.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 Processor -- start 2024-04-25T10:30:29.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.111+0000 7f4493800640 1 -- start start 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470003b80 0x7f448c15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 0x7f448c0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c152d10 0x7f448c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c167560 con 0x7f448c15c100 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c1673e0 con 0x7f4470003b80 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c1676e0 con 0x7f448c152d10 2024-04-25T10:30:29.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 0x7f448c0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 0x7f448c0b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55744/0 (socket says 172.21.15.45:55744) 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 -- 172.21.15.45:0/4031693945 learned_addr learned my addr 172.21.15.45:0/4031693945 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4490e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470003b80 0x7f448c15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4491800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c152d10 0x7f448c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 -- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c152d10 msgr2=0x7f448c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c152d10 0x7f448c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 -- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470003b80 msgr2=0x7f448c15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470003b80 0x7f448c15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 -- 172.21.15.45:0/4031693945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c1536e0 con 0x7f448c15c100 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4490e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470003b80 0x7f448c15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4483e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 0x7f448c0b16b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f447400e780 tx=0x7f447400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740050d0 con 0x7f448c15c100 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474018070 con 0x7f448c15c100 2024-04-25T10:30:29.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474005490 con 0x7f448c15c100 2024-04-25T10:30:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c0b0530 con 0x7f448c15c100 2024-04-25T10:30:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c0b07d0 con 0x7f448c15c100 2024-04-25T10:30:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c04ef30 con 0x7f448c15c100 2024-04-25T10:30:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.115+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f447400c6c0 con 0x7f448c15c100 2024-04-25T10:30:29.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.119+0000 7f4489600640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468067130 0x7f44680695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.119+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4474084ed0 con 0x7f448c15c100 2024-04-25T10:30:29.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.119+0000 7f4490e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468067130 0x7f44680695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.119+0000 7f4490e00640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468067130 0x7f44680695f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f447c0aa000 tx=0x7f447c0b0040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.123+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4474055e40 con 0x7f448c15c100 2024-04-25T10:30:29.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.279+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f448c0af1c0 con 0x7f4468067130 2024-04-25T10:30:29.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.287+0000 7f4489600640 1 -- 172.21.15.45:0/4031693945 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f447c0f9a60 con 0x7f4468067130 2024-04-25T10:30:29.292 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:29.293 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468067130 msgr2=0x7f44680695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468067130 0x7f44680695f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f447c0aa000 tx=0x7f447c0b0040 comp rx=0 tx=0).stop 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 msgr2=0x7f448c0b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 0x7f448c0b16b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f447400e780 tx=0x7f447400ec40 comp rx=0 tx=0).stop 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 shutdown_connections 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4468067130 0x7f44680695f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f448c152d10 0x7f448c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f448c15c100 0x7f448c0b16b0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 --2- 172.21.15.45:0/4031693945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4470003b80 0x7f448c15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 >> 172.21.15.45:0/4031693945 conn(0x7f448c066b70 msgr2=0x7f448c0ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 shutdown_connections 2024-04-25T10:30:29.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.291+0000 7f4493800640 1 -- 172.21.15.45:0/4031693945 wait complete. 2024-04-25T10:30:29.312 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-25T10:30:29.050056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":455900,"kb_used_data":17768,"kb_used_omap":12,"kb_used_meta":438003,"kb_avail":754518820,"statfs":{"total":773094113280,"available":772627271680,"internally_reserved":0,"allocated":18194432,"data_stored":8614636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448515632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.867934"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234611+0000","last_change":"2024-04-25T10:30:25.214742+0000","last_active":"2024-04-25T10:30:26.234611+0000","last_peered":"2024-04-25T10:30:26.234611+0000","last_clean":"2024-04-25T10:30:26.234611+0000","last_became_active":"2024-04-25T10:30:25.210635+0000","last_became_peered":"2024-04-25T10:30:25.210635+0000","last_unstale":"2024-04-25T10:30:26.234611+0000","last_undegraded":"2024-04-25T10:30:26.234611+0000","last_fullsized":"2024-04-25T10:30:26.234611+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:52.008231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021849+0000","last_change":"2024-04-25T10:30:25.203862+0000","last_active":"2024-04-25T10:30:27.021849+0000","last_peered":"2024-04-25T10:30:27.021849+0000","last_clean":"2024-04-25T10:30:27.021849+0000","last_became_active":"2024-04-25T10:30:25.202849+0000","last_became_peered":"2024-04-25T10:30:25.202849+0000","last_unstale":"2024-04-25T10:30:27.021849+0000","last_undegraded":"2024-04-25T10:30:27.021849+0000","last_fullsized":"2024-04-25T10:30:27.021849+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:14.072724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200804+0000","last_change":"2024-04-25T10:30:25.217676+0000","last_active":"2024-04-25T10:30:26.200804+0000","last_peered":"2024-04-25T10:30:26.200804+0000","last_clean":"2024-04-25T10:30:26.200804+0000","last_became_active":"2024-04-25T10:30:25.217093+0000","last_became_peered":"2024-04-25T10:30:25.217093+0000","last_unstale":"2024-04-25T10:30:26.200804+0000","last_undegraded":"2024-04-25T10:30:26.200804+0000","last_fullsized":"2024-04-25T10:30:26.200804+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:38.267448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021727+0000","last_change":"2024-04-25T10:30:25.223287+0000","last_active":"2024-04-25T10:30:27.021727+0000","last_peered":"2024-04-25T10:30:27.021727+0000","last_clean":"2024-04-25T10:30:27.021727+0000","last_became_active":"2024-04-25T10:30:25.222139+0000","last_became_peered":"2024-04-25T10:30:25.222139+0000","last_unstale":"2024-04-25T10:30:27.021727+0000","last_undegraded":"2024-04-25T10:30:27.021727+0000","last_fullsized":"2024-04-25T10:30:27.021727+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:43.771462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097499+0000","last_change":"2024-04-25T10:30:25.215431+0000","last_active":"2024-04-25T10:30:27.097499+0000","last_peered":"2024-04-25T10:30:27.097499+0000","last_clean":"2024-04-25T10:30:27.097499+0000","last_became_active":"2024-04-25T10:30:25.215075+0000","last_became_peered":"2024-04-25T10:30:25.215075+0000","last_unstale":"2024-04-25T10:30:27.097499+0000","last_undegraded":"2024-04-25T10:30:27.097499+0000","last_fullsized":"2024-04-25T10:30:27.097499+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:42.746015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096809+0000","last_change":"2024-04-25T10:30:25.213934+0000","last_active":"2024-04-25T10:30:27.096809+0000","last_peered":"2024-04-25T10:30:27.096809+0000","last_clean":"2024-04-25T10:30:27.096809+0000","last_became_active":"2024-04-25T10:30:25.213664+0000","last_became_peered":"2024-04-25T10:30:25.213664+0000","last_unstale":"2024-04-25T10:30:27.096809+0000","last_undegraded":"2024-04-25T10:30:27.096809+0000","last_fullsized":"2024-04-25T10:30:27.096809+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:53.201775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194264+0000","last_change":"2024-04-25T10:30:25.222788+0000","last_active":"2024-04-25T10:30:26.194264+0000","last_peered":"2024-04-25T10:30:26.194264+0000","last_clean":"2024-04-25T10:30:26.194264+0000","last_became_active":"2024-04-25T10:30:25.222498+0000","last_became_peered":"2024-04-25T10:30:25.222498+0000","last_unstale":"2024-04-25T10:30:26.194264+0000","last_undegraded":"2024-04-25T10:30:26.194264+0000","last_fullsized":"2024-04-25T10:30:26.194264+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:03.093782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096764+0000","last_change":"2024-04-25T10:30:25.220781+0000","last_active":"2024-04-25T10:30:27.096764+0000","last_peered":"2024-04-25T10:30:27.096764+0000","last_clean":"2024-04-25T10:30:27.096764+0000","last_became_active":"2024-04-25T10:30:25.220534+0000","last_became_peered":"2024-04-25T10:30:25.220534+0000","last_unstale":"2024-04-25T10:30:27.096764+0000","last_undegraded":"2024-04-25T10:30:27.096764+0000","last_fullsized":"2024-04-25T10:30:27.096764+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:13.160678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021887+0000","last_change":"2024-04-25T10:30:25.212488+0000","last_active":"2024-04-25T10:30:27.021887+0000","last_peered":"2024-04-25T10:30:27.021887+0000","last_clean":"2024-04-25T10:30:27.021887+0000","last_became_active":"2024-04-25T10:30:25.212177+0000","last_became_peered":"2024-04-25T10:30:25.212177+0000","last_unstale":"2024-04-25T10:30:27.021887+0000","last_undegraded":"2024-04-25T10:30:27.021887+0000","last_fullsized":"2024-04-25T10:30:27.021887+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:01.936095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200818+0000","last_change":"2024-04-25T10:30:25.213057+0000","last_active":"2024-04-25T10:30:26.200818+0000","last_peered":"2024-04-25T10:30:26.200818+0000","last_clean":"2024-04-25T10:30:26.200818+0000","last_became_active":"2024-04-25T10:30:25.212757+0000","last_became_peered":"2024-04-25T10:30:25.212757+0000","last_unstale":"2024-04-25T10:30:26.200818+0000","last_undegraded":"2024-04-25T10:30:26.200818+0000","last_fullsized":"2024-04-25T10:30:26.200818+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:42.746015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234467+0000","last_change":"2024-04-25T10:30:25.204387+0000","last_active":"2024-04-25T10:30:26.234467+0000","last_peered":"2024-04-25T10:30:26.234467+0000","last_clean":"2024-04-25T10:30:26.234467+0000","last_became_active":"2024-04-25T10:30:25.204063+0000","last_became_peered":"2024-04-25T10:30:25.204063+0000","last_unstale":"2024-04-25T10:30:26.234467+0000","last_undegraded":"2024-04-25T10:30:26.234467+0000","last_fullsized":"2024-04-25T10:30:26.234467+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:56.347500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094261+0000","last_change":"2024-04-25T10:30:25.221199+0000","last_active":"2024-04-25T10:30:27.094261+0000","last_peered":"2024-04-25T10:30:27.094261+0000","last_clean":"2024-04-25T10:30:27.094261+0000","last_became_active":"2024-04-25T10:30:25.220527+0000","last_became_peered":"2024-04-25T10:30:25.220527+0000","last_unstale":"2024-04-25T10:30:27.094261+0000","last_undegraded":"2024-04-25T10:30:27.094261+0000","last_fullsized":"2024-04-25T10:30:27.094261+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:10.809229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234272+0000","last_change":"2024-04-25T10:30:25.211536+0000","last_active":"2024-04-25T10:30:26.234272+0000","last_peered":"2024-04-25T10:30:26.234272+0000","last_clean":"2024-04-25T10:30:26.234272+0000","last_became_active":"2024-04-25T10:30:25.211254+0000","last_became_peered":"2024-04-25T10:30:25.211254+0000","last_unstale":"2024-04-25T10:30:26.234272+0000","last_undegraded":"2024-04-25T10:30:26.234272+0000","last_fullsized":"2024-04-25T10:30:26.234272+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:19.179379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021228+0000","last_change":"2024-04-25T10:30:25.212804+0000","last_active":"2024-04-25T10:30:27.021228+0000","last_peered":"2024-04-25T10:30:27.021228+0000","last_clean":"2024-04-25T10:30:27.021228+0000","last_became_active":"2024-04-25T10:30:25.212107+0000","last_became_peered":"2024-04-25T10:30:25.212107+0000","last_unstale":"2024-04-25T10:30:27.021228+0000","last_undegraded":"2024-04-25T10:30:27.021228+0000","last_fullsized":"2024-04-25T10:30:27.021228+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.413918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234259+0000","last_change":"2024-04-25T10:30:25.205188+0000","last_active":"2024-04-25T10:30:26.234259+0000","last_peered":"2024-04-25T10:30:26.234259+0000","last_clean":"2024-04-25T10:30:26.234259+0000","last_became_active":"2024-04-25T10:30:25.204921+0000","last_became_peered":"2024-04-25T10:30:25.204921+0000","last_unstale":"2024-04-25T10:30:26.234259+0000","last_undegraded":"2024-04-25T10:30:26.234259+0000","last_fullsized":"2024-04-25T10:30:26.234259+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:29.866914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.199760+0000","last_change":"2024-04-25T10:30:25.212298+0000","last_active":"2024-04-25T10:30:26.199760+0000","last_peered":"2024-04-25T10:30:26.199760+0000","last_clean":"2024-04-25T10:30:26.199760+0000","last_became_active":"2024-04-25T10:30:25.212026+0000","last_became_peered":"2024-04-25T10:30:25.212026+0000","last_unstale":"2024-04-25T10:30:26.199760+0000","last_undegraded":"2024-04-25T10:30:26.199760+0000","last_fullsized":"2024-04-25T10:30:26.199760+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:42.746015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097701+0000","last_change":"2024-04-25T10:30:25.217970+0000","last_active":"2024-04-25T10:30:27.097701+0000","last_peered":"2024-04-25T10:30:27.097701+0000","last_clean":"2024-04-25T10:30:27.097701+0000","last_became_active":"2024-04-25T10:30:25.217671+0000","last_became_peered":"2024-04-25T10:30:25.217671+0000","last_unstale":"2024-04-25T10:30:27.097701+0000","last_undegraded":"2024-04-25T10:30:27.097701+0000","last_fullsized":"2024-04-25T10:30:27.097701+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:11.094619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194756+0000","last_change":"2024-04-25T10:30:25.212332+0000","last_active":"2024-04-25T10:30:26.194756+0000","last_peered":"2024-04-25T10:30:26.194756+0000","last_clean":"2024-04-25T10:30:26.194756+0000","last_became_active":"2024-04-25T10:30:25.212029+0000","last_became_peered":"2024-04-25T10:30:25.212029+0000","last_unstale":"2024-04-25T10:30:26.194756+0000","last_undegraded":"2024-04-25T10:30:26.194756+0000","last_fullsized":"2024-04-25T10:30:26.194756+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:24.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200877+0000","last_change":"2024-04-25T10:30:25.227421+0000","last_active":"2024-04-25T10:30:26.200877+0000","last_peered":"2024-04-25T10:30:26.200877+0000","last_clean":"2024-04-25T10:30:26.200877+0000","last_became_active":"2024-04-25T10:30:25.227046+0000","last_became_peered":"2024-04-25T10:30:25.227046+0000","last_unstale":"2024-04-25T10:30:26.200877+0000","last_undegraded":"2024-04-25T10:30:26.200877+0000","last_fullsized":"2024-04-25T10:30:26.200877+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:20.546559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097752+0000","last_change":"2024-04-25T10:30:25.230009+0000","last_active":"2024-04-25T10:30:27.097752+0000","last_peered":"2024-04-25T10:30:27.097752+0000","last_clean":"2024-04-25T10:30:27.097752+0000","last_became_active":"2024-04-25T10:30:25.229688+0000","last_became_peered":"2024-04-25T10:30:25.229688+0000","last_unstale":"2024-04-25T10:30:27.097752+0000","last_undegraded":"2024-04-25T10:30:27.097752+0000","last_fullsized":"2024-04-25T10:30:27.097752+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.544583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200200+0000","last_change":"2024-04-25T10:30:25.212112+0000","last_active":"2024-04-25T10:30:26.200200+0000","last_peered":"2024-04-25T10:30:26.200200+0000","last_clean":"2024-04-25T10:30:26.200200+0000","last_became_active":"2024-04-25T10:30:25.211822+0000","last_became_peered":"2024-04-25T10:30:25.211822+0000","last_unstale":"2024-04-25T10:30:26.200200+0000","last_undegraded":"2024-04-25T10:30:26.200200+0000","last_fullsized":"2024-04-25T10:30:26.200200+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.397559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097321+0000","last_change":"2024-04-25T10:30:25.230212+0000","last_active":"2024-04-25T10:30:27.097321+0000","last_peered":"2024-04-25T10:30:27.097321+0000","last_clean":"2024-04-25T10:30:27.097321+0000","last_became_active":"2024-04-25T10:30:25.229914+0000","last_became_peered":"2024-04-25T10:30:25.229914+0000","last_unstale":"2024-04-25T10:30:27.097321+0000","last_undegraded":"2024-04-25T10:30:27.097321+0000","last_fullsized":"2024-04-25T10:30:27.097321+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:40.152211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.199748+0000","last_change":"2024-04-25T10:30:25.220723+0000","last_active":"2024-04-25T10:30:26.199748+0000","last_peered":"2024-04-25T10:30:26.199748+0000","last_clean":"2024-04-25T10:30:26.199748+0000","last_became_active":"2024-04-25T10:30:25.220413+0000","last_became_peered":"2024-04-25T10:30:25.220413+0000","last_unstale":"2024-04-25T10:30:26.199748+0000","last_undegraded":"2024-04-25T10:30:26.199748+0000","last_fullsized":"2024-04-25T10:30:26.199748+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.305758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096577+0000","last_change":"2024-04-25T10:30:25.226698+0000","last_active":"2024-04-25T10:30:27.096577+0000","last_peered":"2024-04-25T10:30:27.096577+0000","last_clean":"2024-04-25T10:30:27.096577+0000","last_became_active":"2024-04-25T10:30:25.224720+0000","last_became_peered":"2024-04-25T10:30:25.224720+0000","last_unstale":"2024-04-25T10:30:27.096577+0000","last_undegraded":"2024-04-25T10:30:27.096577+0000","last_fullsized":"2024-04-25T10:30:27.096577+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:10.809229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021875+0000","last_change":"2024-04-25T10:30:25.204903+0000","last_active":"2024-04-25T10:30:27.021875+0000","last_peered":"2024-04-25T10:30:27.021875+0000","last_clean":"2024-04-25T10:30:27.021875+0000","last_became_active":"2024-04-25T10:30:25.202766+0000","last_became_peered":"2024-04-25T10:30:25.202766+0000","last_unstale":"2024-04-25T10:30:27.021875+0000","last_undegraded":"2024-04-25T10:30:27.021875+0000","last_fullsized":"2024-04-25T10:30:27.021875+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:47.666542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094591+0000","last_change":"2024-04-25T10:30:25.214966+0000","last_active":"2024-04-25T10:30:27.094591+0000","last_peered":"2024-04-25T10:30:27.094591+0000","last_clean":"2024-04-25T10:30:27.094591+0000","last_became_active":"2024-04-25T10:30:25.214039+0000","last_became_peered":"2024-04-25T10:30:25.214039+0000","last_unstale":"2024-04-25T10:30:27.094591+0000","last_undegraded":"2024-04-25T10:30:27.094591+0000","last_fullsized":"2024-04-25T10:30:27.094591+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:16.659507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200992+0000","last_change":"2024-04-25T10:30:25.217859+0000","last_active":"2024-04-25T10:30:26.200992+0000","last_peered":"2024-04-25T10:30:26.200992+0000","last_clean":"2024-04-25T10:30:26.200992+0000","last_became_active":"2024-04-25T10:30:25.217591+0000","last_became_peered":"2024-04-25T10:30:25.217591+0000","last_unstale":"2024-04-25T10:30:26.200992+0000","last_undegraded":"2024-04-25T10:30:26.200992+0000","last_fullsized":"2024-04-25T10:30:26.200992+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:13.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201207+0000","last_change":"2024-04-25T10:30:25.225196+0000","last_active":"2024-04-25T10:30:26.201207+0000","last_peered":"2024-04-25T10:30:26.201207+0000","last_clean":"2024-04-25T10:30:26.201207+0000","last_became_active":"2024-04-25T10:30:25.224911+0000","last_became_peered":"2024-04-25T10:30:25.224911+0000","last_unstale":"2024-04-25T10:30:26.201207+0000","last_undegraded":"2024-04-25T10:30:26.201207+0000","last_fullsized":"2024-04-25T10:30:26.201207+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.544583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094172+0000","last_change":"2024-04-25T10:30:25.208393+0000","last_active":"2024-04-25T10:30:27.094172+0000","last_peered":"2024-04-25T10:30:27.094172+0000","last_clean":"2024-04-25T10:30:27.094172+0000","last_became_active":"2024-04-25T10:30:25.208099+0000","last_became_peered":"2024-04-25T10:30:25.208099+0000","last_unstale":"2024-04-25T10:30:27.094172+0000","last_undegraded":"2024-04-25T10:30:27.094172+0000","last_fullsized":"2024-04-25T10:30:27.094172+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:52.010205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094359+0000","last_change":"2024-04-25T10:30:25.210082+0000","last_active":"2024-04-25T10:30:27.094359+0000","last_peered":"2024-04-25T10:30:27.094359+0000","last_clean":"2024-04-25T10:30:27.094359+0000","last_became_active":"2024-04-25T10:30:25.209722+0000","last_became_peered":"2024-04-25T10:30:25.209722+0000","last_unstale":"2024-04-25T10:30:27.094359+0000","last_undegraded":"2024-04-25T10:30:27.094359+0000","last_fullsized":"2024-04-25T10:30:27.094359+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.053768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096957+0000","last_change":"2024-04-25T10:30:25.221732+0000","last_active":"2024-04-25T10:30:27.096957+0000","last_peered":"2024-04-25T10:30:27.096957+0000","last_clean":"2024-04-25T10:30:27.096957+0000","last_became_active":"2024-04-25T10:30:25.221431+0000","last_became_peered":"2024-04-25T10:30:25.221431+0000","last_unstale":"2024-04-25T10:30:27.096957+0000","last_undegraded":"2024-04-25T10:30:27.096957+0000","last_fullsized":"2024-04-25T10:30:27.096957+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:30.406873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096636+0000","last_change":"2024-04-25T10:30:25.220293+0000","last_active":"2024-04-25T10:30:27.096636+0000","last_peered":"2024-04-25T10:30:27.096636+0000","last_clean":"2024-04-25T10:30:27.096636+0000","last_became_active":"2024-04-25T10:30:25.218531+0000","last_became_peered":"2024-04-25T10:30:25.218531+0000","last_unstale":"2024-04-25T10:30:27.096636+0000","last_undegraded":"2024-04-25T10:30:27.096636+0000","last_fullsized":"2024-04-25T10:30:27.096636+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.053768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097272+0000","last_change":"2024-04-25T10:30:25.229071+0000","last_active":"2024-04-25T10:30:27.097272+0000","last_peered":"2024-04-25T10:30:27.097272+0000","last_clean":"2024-04-25T10:30:27.097272+0000","last_became_active":"2024-04-25T10:30:25.228802+0000","last_became_peered":"2024-04-25T10:30:25.228802+0000","last_unstale":"2024-04-25T10:30:27.097272+0000","last_undegraded":"2024-04-25T10:30:27.097272+0000","last_fullsized":"2024-04-25T10:30:27.097272+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.828621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"168'4","reported_seq":18,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201092+0000","last_change":"2024-04-25T10:30:24.193125+0000","last_active":"2024-04-25T10:30:26.201092+0000","last_peered":"2024-04-25T10:30:26.201092+0000","last_clean":"2024-04-25T10:30:26.201092+0000","last_became_active":"2024-04-25T10:30:24.192792+0000","last_became_peered":"2024-04-25T10:30:24.192792+0000","last_unstale":"2024-04-25T10:30:26.201092+0000","last_undegraded":"2024-04-25T10:30:26.201092+0000","last_fullsized":"2024-04-25T10:30:26.201092+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:12.906089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201366+0000","last_change":"2024-04-25T10:30:25.224829+0000","last_active":"2024-04-25T10:30:26.201366+0000","last_peered":"2024-04-25T10:30:26.201366+0000","last_clean":"2024-04-25T10:30:26.201366+0000","last_became_active":"2024-04-25T10:30:25.223655+0000","last_became_peered":"2024-04-25T10:30:25.223655+0000","last_unstale":"2024-04-25T10:30:26.201366+0000","last_undegraded":"2024-04-25T10:30:26.201366+0000","last_fullsized":"2024-04-25T10:30:26.201366+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:38.267448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094463+0000","last_change":"2024-04-25T10:30:24.203317+0000","last_active":"2024-04-25T10:30:27.094463+0000","last_peered":"2024-04-25T10:30:27.094463+0000","last_clean":"2024-04-25T10:30:27.094463+0000","last_became_active":"2024-04-25T10:30:24.202617+0000","last_became_peered":"2024-04-25T10:30:24.202617+0000","last_unstale":"2024-04-25T10:30:27.094463+0000","last_undegraded":"2024-04-25T10:30:27.094463+0000","last_fullsized":"2024-04-25T10:30:27.094463+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:43.317402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021636+0000","last_change":"2024-04-25T10:30:25.199054+0000","last_active":"2024-04-25T10:30:27.021636+0000","last_peered":"2024-04-25T10:30:27.021636+0000","last_clean":"2024-04-25T10:30:27.021636+0000","last_became_active":"2024-04-25T10:30:25.198763+0000","last_became_peered":"2024-04-25T10:30:25.198763+0000","last_unstale":"2024-04-25T10:30:27.021636+0000","last_undegraded":"2024-04-25T10:30:27.021636+0000","last_fullsized":"2024-04-25T10:30:27.021636+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:16.172941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097418+0000","last_change":"2024-04-25T10:30:24.204900+0000","last_active":"2024-04-25T10:30:27.097418+0000","last_peered":"2024-04-25T10:30:27.097418+0000","last_clean":"2024-04-25T10:30:27.097418+0000","last_became_active":"2024-04-25T10:30:24.204595+0000","last_became_peered":"2024-04-25T10:30:24.204595+0000","last_unstale":"2024-04-25T10:30:27.097418+0000","last_undegraded":"2024-04-25T10:30:27.097418+0000","last_fullsized":"2024-04-25T10:30:27.097418+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.511957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021682+0000","last_change":"2024-04-25T10:30:25.211817+0000","last_active":"2024-04-25T10:30:27.021682+0000","last_peered":"2024-04-25T10:30:27.021682+0000","last_clean":"2024-04-25T10:30:27.021682+0000","last_became_active":"2024-04-25T10:30:25.211576+0000","last_became_peered":"2024-04-25T10:30:25.211576+0000","last_unstale":"2024-04-25T10:30:27.021682+0000","last_undegraded":"2024-04-25T10:30:27.021682+0000","last_fullsized":"2024-04-25T10:30:27.021682+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094336+0000","last_change":"2024-04-25T10:30:24.204491+0000","last_active":"2024-04-25T10:30:27.094336+0000","last_peered":"2024-04-25T10:30:27.094336+0000","last_clean":"2024-04-25T10:30:27.094336+0000","last_became_active":"2024-04-25T10:30:24.203319+0000","last_became_peered":"2024-04-25T10:30:24.203319+0000","last_unstale":"2024-04-25T10:30:27.094336+0000","last_undegraded":"2024-04-25T10:30:27.094336+0000","last_fullsized":"2024-04-25T10:30:27.094336+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.039848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096981+0000","last_change":"2024-04-25T10:30:25.213178+0000","last_active":"2024-04-25T10:30:27.096981+0000","last_peered":"2024-04-25T10:30:27.096981+0000","last_clean":"2024-04-25T10:30:27.096981+0000","last_became_active":"2024-04-25T10:30:25.212906+0000","last_became_peered":"2024-04-25T10:30:25.212906+0000","last_unstale":"2024-04-25T10:30:27.096981+0000","last_undegraded":"2024-04-25T10:30:27.096981+0000","last_fullsized":"2024-04-25T10:30:27.096981+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:15.735860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097437+0000","last_change":"2024-04-25T10:30:24.205083+0000","last_active":"2024-04-25T10:30:27.097437+0000","last_peered":"2024-04-25T10:30:27.097437+0000","last_clean":"2024-04-25T10:30:27.097437+0000","last_became_active":"2024-04-25T10:30:24.204652+0000","last_became_peered":"2024-04-25T10:30:24.204652+0000","last_unstale":"2024-04-25T10:30:27.097437+0000","last_undegraded":"2024-04-25T10:30:27.097437+0000","last_fullsized":"2024-04-25T10:30:27.097437+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.684197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097367+0000","last_change":"2024-04-25T10:30:25.215934+0000","last_active":"2024-04-25T10:30:27.097367+0000","last_peered":"2024-04-25T10:30:27.097367+0000","last_clean":"2024-04-25T10:30:27.097367+0000","last_became_active":"2024-04-25T10:30:25.215683+0000","last_became_peered":"2024-04-25T10:30:25.215683+0000","last_unstale":"2024-04-25T10:30:27.097367+0000","last_undegraded":"2024-04-25T10:30:27.097367+0000","last_fullsized":"2024-04-25T10:30:27.097367+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:03.590877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097250+0000","last_change":"2024-04-25T10:30:24.204028+0000","last_active":"2024-04-25T10:30:27.097250+0000","last_peered":"2024-04-25T10:30:27.097250+0000","last_clean":"2024-04-25T10:30:27.097250+0000","last_became_active":"2024-04-25T10:30:24.203757+0000","last_became_peered":"2024-04-25T10:30:24.203757+0000","last_unstale":"2024-04-25T10:30:27.097250+0000","last_undegraded":"2024-04-25T10:30:27.097250+0000","last_fullsized":"2024-04-25T10:30:27.097250+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.039848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234252+0000","last_change":"2024-04-25T10:30:25.211466+0000","last_active":"2024-04-25T10:30:26.234252+0000","last_peered":"2024-04-25T10:30:26.234252+0000","last_clean":"2024-04-25T10:30:26.234252+0000","last_became_active":"2024-04-25T10:30:25.211171+0000","last_became_peered":"2024-04-25T10:30:25.211171+0000","last_unstale":"2024-04-25T10:30:26.234252+0000","last_undegraded":"2024-04-25T10:30:26.234252+0000","last_fullsized":"2024-04-25T10:30:26.234252+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:30.230476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194093+0000","last_change":"2024-04-25T10:30:24.197654+0000","last_active":"2024-04-25T10:30:26.194093+0000","last_peered":"2024-04-25T10:30:26.194093+0000","last_clean":"2024-04-25T10:30:26.194093+0000","last_became_active":"2024-04-25T10:30:24.197357+0000","last_became_peered":"2024-04-25T10:30:24.197357+0000","last_unstale":"2024-04-25T10:30:26.194093+0000","last_undegraded":"2024-04-25T10:30:26.194093+0000","last_fullsized":"2024-04-25T10:30:26.194093+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:37.256374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200942+0000","last_change":"2024-04-25T10:30:25.224500+0000","last_active":"2024-04-25T10:30:26.200942+0000","last_peered":"2024-04-25T10:30:26.200942+0000","last_clean":"2024-04-25T10:30:26.200942+0000","last_became_active":"2024-04-25T10:30:25.220464+0000","last_became_peered":"2024-04-25T10:30:25.220464+0000","last_unstale":"2024-04-25T10:30:26.200942+0000","last_undegraded":"2024-04-25T10:30:26.200942+0000","last_fullsized":"2024-04-25T10:30:26.200942+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:40.107598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234570+0000","last_change":"2024-04-25T10:30:24.205112+0000","last_active":"2024-04-25T10:30:26.234570+0000","last_peered":"2024-04-25T10:30:26.234570+0000","last_clean":"2024-04-25T10:30:26.234570+0000","last_became_active":"2024-04-25T10:30:24.203155+0000","last_became_peered":"2024-04-25T10:30:24.203155+0000","last_unstale":"2024-04-25T10:30:26.234570+0000","last_undegraded":"2024-04-25T10:30:26.234570+0000","last_fullsized":"2024-04-25T10:30:26.234570+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:40.325544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201369+0000","last_change":"2024-04-25T10:30:25.220632+0000","last_active":"2024-04-25T10:30:26.201369+0000","last_peered":"2024-04-25T10:30:26.201369+0000","last_clean":"2024-04-25T10:30:26.201369+0000","last_became_active":"2024-04-25T10:30:25.219956+0000","last_became_peered":"2024-04-25T10:30:25.219956+0000","last_unstale":"2024-04-25T10:30:26.201369+0000","last_undegraded":"2024-04-25T10:30:26.201369+0000","last_fullsized":"2024-04-25T10:30:26.201369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:20.546559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201390+0000","last_change":"2024-04-25T10:30:24.200093+0000","last_active":"2024-04-25T10:30:26.201390+0000","last_peered":"2024-04-25T10:30:26.201390+0000","last_clean":"2024-04-25T10:30:26.201390+0000","last_became_active":"2024-04-25T10:30:24.199728+0000","last_became_peered":"2024-04-25T10:30:24.199728+0000","last_unstale":"2024-04-25T10:30:26.201390+0000","last_undegraded":"2024-04-25T10:30:26.201390+0000","last_fullsized":"2024-04-25T10:30:26.201390+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.213666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094435+0000","last_change":"2024-04-25T10:30:25.207621+0000","last_active":"2024-04-25T10:30:27.094435+0000","last_peered":"2024-04-25T10:30:27.094435+0000","last_clean":"2024-04-25T10:30:27.094435+0000","last_became_active":"2024-04-25T10:30:25.207315+0000","last_became_peered":"2024-04-25T10:30:25.207315+0000","last_unstale":"2024-04-25T10:30:27.094435+0000","last_undegraded":"2024-04-25T10:30:27.094435+0000","last_fullsized":"2024-04-25T10:30:27.094435+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:30.406873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"168'3","reported_seq":17,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097669+0000","last_change":"2024-04-25T10:30:24.207259+0000","last_active":"2024-04-25T10:30:27.097669+0000","last_peered":"2024-04-25T10:30:27.097669+0000","last_clean":"2024-04-25T10:30:27.097669+0000","last_became_active":"2024-04-25T10:30:24.206935+0000","last_became_peered":"2024-04-25T10:30:24.206935+0000","last_unstale":"2024-04-25T10:30:27.097669+0000","last_undegraded":"2024-04-25T10:30:27.097669+0000","last_fullsized":"2024-04-25T10:30:27.097669+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:58.111975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200906+0000","last_change":"2024-04-25T10:30:25.216648+0000","last_active":"2024-04-25T10:30:26.200906+0000","last_peered":"2024-04-25T10:30:26.200906+0000","last_clean":"2024-04-25T10:30:26.200906+0000","last_became_active":"2024-04-25T10:30:25.216384+0000","last_became_peered":"2024-04-25T10:30:25.216384+0000","last_unstale":"2024-04-25T10:30:26.200906+0000","last_undegraded":"2024-04-25T10:30:26.200906+0000","last_fullsized":"2024-04-25T10:30:26.200906+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.397559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"168'6","reported_seq":20,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097707+0000","last_change":"2024-04-25T10:30:24.206758+0000","last_active":"2024-04-25T10:30:27.097707+0000","last_peered":"2024-04-25T10:30:27.097707+0000","last_clean":"2024-04-25T10:30:27.097707+0000","last_became_active":"2024-04-25T10:30:24.206434+0000","last_became_peered":"2024-04-25T10:30:24.206434+0000","last_unstale":"2024-04-25T10:30:27.097707+0000","last_undegraded":"2024-04-25T10:30:27.097707+0000","last_fullsized":"2024-04-25T10:30:27.097707+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:23.317632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200534+0000","last_change":"2024-04-25T10:30:25.213206+0000","last_active":"2024-04-25T10:30:26.200534+0000","last_peered":"2024-04-25T10:30:26.200534+0000","last_clean":"2024-04-25T10:30:26.200534+0000","last_became_active":"2024-04-25T10:30:25.212904+0000","last_became_peered":"2024-04-25T10:30:25.212904+0000","last_unstale":"2024-04-25T10:30:26.200534+0000","last_undegraded":"2024-04-25T10:30:26.200534+0000","last_fullsized":"2024-04-25T10:30:26.200534+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:03.590877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094210+0000","last_change":"2024-04-25T10:30:24.197901+0000","last_active":"2024-04-25T10:30:27.094210+0000","last_peered":"2024-04-25T10:30:27.094210+0000","last_clean":"2024-04-25T10:30:27.094210+0000","last_became_active":"2024-04-25T10:30:24.197591+0000","last_became_peered":"2024-04-25T10:30:24.197591+0000","last_unstale":"2024-04-25T10:30:27.094210+0000","last_undegraded":"2024-04-25T10:30:27.094210+0000","last_fullsized":"2024-04-25T10:30:27.094210+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:23.317632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194313+0000","last_change":"2024-04-25T10:30:25.224849+0000","last_active":"2024-04-25T10:30:26.194313+0000","last_peered":"2024-04-25T10:30:26.194313+0000","last_clean":"2024-04-25T10:30:26.194313+0000","last_became_active":"2024-04-25T10:30:25.224322+0000","last_became_peered":"2024-04-25T10:30:25.224322+0000","last_unstale":"2024-04-25T10:30:26.194313+0000","last_undegraded":"2024-04-25T10:30:26.194313+0000","last_fullsized":"2024-04-25T10:30:26.194313+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:29.602109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200601+0000","last_change":"2024-04-25T10:30:24.201781+0000","last_active":"2024-04-25T10:30:26.200601+0000","last_peered":"2024-04-25T10:30:26.200601+0000","last_clean":"2024-04-25T10:30:26.200601+0000","last_became_active":"2024-04-25T10:30:24.201433+0000","last_became_peered":"2024-04-25T10:30:24.201433+0000","last_unstale":"2024-04-25T10:30:26.200601+0000","last_undegraded":"2024-04-25T10:30:26.200601+0000","last_fullsized":"2024-04-25T10:30:26.200601+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:21.759031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094040+0000","last_change":"2024-04-25T10:30:25.222706+0000","last_active":"2024-04-25T10:30:27.094040+0000","last_peered":"2024-04-25T10:30:27.094040+0000","last_clean":"2024-04-25T10:30:27.094040+0000","last_became_active":"2024-04-25T10:30:25.222395+0000","last_became_peered":"2024-04-25T10:30:25.222395+0000","last_unstale":"2024-04-25T10:30:27.094040+0000","last_undegraded":"2024-04-25T10:30:27.094040+0000","last_fullsized":"2024-04-25T10:30:27.094040+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:16.054666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097485+0000","last_change":"2024-04-25T10:30:24.200714+0000","last_active":"2024-04-25T10:30:27.097485+0000","last_peered":"2024-04-25T10:30:27.097485+0000","last_clean":"2024-04-25T10:30:27.097485+0000","last_became_active":"2024-04-25T10:30:24.200363+0000","last_became_peered":"2024-04-25T10:30:24.200363+0000","last_unstale":"2024-04-25T10:30:27.097485+0000","last_undegraded":"2024-04-25T10:30:27.097485+0000","last_fullsized":"2024-04-25T10:30:27.097485+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:41.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097131+0000","last_change":"2024-04-25T10:30:25.214376+0000","last_active":"2024-04-25T10:30:27.097131+0000","last_peered":"2024-04-25T10:30:27.097131+0000","last_clean":"2024-04-25T10:30:27.097131+0000","last_became_active":"2024-04-25T10:30:25.212888+0000","last_became_peered":"2024-04-25T10:30:25.212888+0000","last_unstale":"2024-04-25T10:30:27.097131+0000","last_undegraded":"2024-04-25T10:30:27.097131+0000","last_fullsized":"2024-04-25T10:30:27.097131+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:16.659507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"168'8","reported_seq":22,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194171+0000","last_change":"2024-04-25T10:30:24.194903+0000","last_active":"2024-04-25T10:30:26.194171+0000","last_peered":"2024-04-25T10:30:26.194171+0000","last_clean":"2024-04-25T10:30:26.194171+0000","last_became_active":"2024-04-25T10:30:24.194639+0000","last_became_peered":"2024-04-25T10:30:24.194639+0000","last_unstale":"2024-04-25T10:30:26.194171+0000","last_undegraded":"2024-04-25T10:30:26.194171+0000","last_fullsized":"2024-04-25T10:30:26.194171+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:19.535485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201484+0000","last_change":"2024-04-25T10:30:25.221618+0000","last_active":"2024-04-25T10:30:26.201484+0000","last_peered":"2024-04-25T10:30:26.201484+0000","last_clean":"2024-04-25T10:30:26.201484+0000","last_became_active":"2024-04-25T10:30:25.221323+0000","last_became_peered":"2024-04-25T10:30:25.221323+0000","last_unstale":"2024-04-25T10:30:26.201484+0000","last_undegraded":"2024-04-25T10:30:26.201484+0000","last_fullsized":"2024-04-25T10:30:26.201484+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:40.107598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097661+0000","last_change":"2024-04-25T10:30:24.205238+0000","last_active":"2024-04-25T10:30:27.097661+0000","last_peered":"2024-04-25T10:30:27.097661+0000","last_clean":"2024-04-25T10:30:27.097661+0000","last_became_active":"2024-04-25T10:30:24.204933+0000","last_became_peered":"2024-04-25T10:30:24.204933+0000","last_unstale":"2024-04-25T10:30:27.097661+0000","last_undegraded":"2024-04-25T10:30:27.097661+0000","last_fullsized":"2024-04-25T10:30:27.097661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:31.725779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094653+0000","last_change":"2024-04-25T10:30:25.221109+0000","last_active":"2024-04-25T10:30:27.094653+0000","last_peered":"2024-04-25T10:30:27.094653+0000","last_clean":"2024-04-25T10:30:27.094653+0000","last_became_active":"2024-04-25T10:30:25.220818+0000","last_became_peered":"2024-04-25T10:30:25.220818+0000","last_unstale":"2024-04-25T10:30:27.094653+0000","last_undegraded":"2024-04-25T10:30:27.094653+0000","last_fullsized":"2024-04-25T10:30:27.094653+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.828621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097207+0000","last_change":"2024-04-25T10:30:24.202700+0000","last_active":"2024-04-25T10:30:27.097207+0000","last_peered":"2024-04-25T10:30:27.097207+0000","last_clean":"2024-04-25T10:30:27.097207+0000","last_became_active":"2024-04-25T10:30:24.202342+0000","last_became_peered":"2024-04-25T10:30:24.202342+0000","last_unstale":"2024-04-25T10:30:27.097207+0000","last_undegraded":"2024-04-25T10:30:27.097207+0000","last_fullsized":"2024-04-25T10:30:27.097207+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:59.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021852+0000","last_change":"2024-04-25T10:30:25.215252+0000","last_active":"2024-04-25T10:30:27.021852+0000","last_peered":"2024-04-25T10:30:27.021852+0000","last_clean":"2024-04-25T10:30:27.021852+0000","last_became_active":"2024-04-25T10:30:25.214572+0000","last_became_peered":"2024-04-25T10:30:25.214572+0000","last_unstale":"2024-04-25T10:30:27.021852+0000","last_undegraded":"2024-04-25T10:30:27.021852+0000","last_fullsized":"2024-04-25T10:30:27.021852+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.841741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234379+0000","last_change":"2024-04-25T10:30:24.196772+0000","last_active":"2024-04-25T10:30:26.234379+0000","last_peered":"2024-04-25T10:30:26.234379+0000","last_clean":"2024-04-25T10:30:26.234379+0000","last_became_active":"2024-04-25T10:30:24.196508+0000","last_became_peered":"2024-04-25T10:30:24.196508+0000","last_unstale":"2024-04-25T10:30:26.234379+0000","last_undegraded":"2024-04-25T10:30:26.234379+0000","last_fullsized":"2024-04-25T10:30:26.234379+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:33.469263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094385+0000","last_change":"2024-04-25T10:30:25.211205+0000","last_active":"2024-04-25T10:30:27.094385+0000","last_peered":"2024-04-25T10:30:27.094385+0000","last_clean":"2024-04-25T10:30:27.094385+0000","last_became_active":"2024-04-25T10:30:25.210920+0000","last_became_peered":"2024-04-25T10:30:25.210920+0000","last_unstale":"2024-04-25T10:30:27.094385+0000","last_undegraded":"2024-04-25T10:30:27.094385+0000","last_fullsized":"2024-04-25T10:30:27.094385+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:13.160678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234468+0000","last_change":"2024-04-25T10:30:24.199627+0000","last_active":"2024-04-25T10:30:26.234468+0000","last_peered":"2024-04-25T10:30:26.234468+0000","last_clean":"2024-04-25T10:30:26.234468+0000","last_became_active":"2024-04-25T10:30:24.199337+0000","last_became_peered":"2024-04-25T10:30:24.199337+0000","last_unstale":"2024-04-25T10:30:26.234468+0000","last_undegraded":"2024-04-25T10:30:26.234468+0000","last_fullsized":"2024-04-25T10:30:26.234468+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.574335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097073+0000","last_change":"2024-04-25T10:30:25.219214+0000","last_active":"2024-04-25T10:30:27.097073+0000","last_peered":"2024-04-25T10:30:27.097073+0000","last_clean":"2024-04-25T10:30:27.097073+0000","last_became_active":"2024-04-25T10:30:25.218700+0000","last_became_peered":"2024-04-25T10:30:25.218700+0000","last_unstale":"2024-04-25T10:30:27.097073+0000","last_undegraded":"2024-04-25T10:30:27.097073+0000","last_fullsized":"2024-04-25T10:30:27.097073+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:23.227421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097561+0000","last_change":"2024-04-25T10:30:24.207079+0000","last_active":"2024-04-25T10:30:27.097561+0000","last_peered":"2024-04-25T10:30:27.097561+0000","last_clean":"2024-04-25T10:30:27.097561+0000","last_became_active":"2024-04-25T10:30:24.206725+0000","last_became_peered":"2024-04-25T10:30:24.206725+0000","last_unstale":"2024-04-25T10:30:27.097561+0000","last_undegraded":"2024-04-25T10:30:27.097561+0000","last_fullsized":"2024-04-25T10:30:27.097561+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:45.311764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194459+0000","last_change":"2024-04-25T10:30:25.212733+0000","last_active":"2024-04-25T10:30:26.194459+0000","last_peered":"2024-04-25T10:30:26.194459+0000","last_clean":"2024-04-25T10:30:26.194459+0000","last_became_active":"2024-04-25T10:30:25.212459+0000","last_became_peered":"2024-04-25T10:30:25.212459+0000","last_unstale":"2024-04-25T10:30:26.194459+0000","last_undegraded":"2024-04-25T10:30:26.194459+0000","last_fullsized":"2024-04-25T10:30:26.194459+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:44.555697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097889+0000","last_change":"2024-04-25T10:30:24.194286+0000","last_active":"2024-04-25T10:30:27.097889+0000","last_peered":"2024-04-25T10:30:27.097889+0000","last_clean":"2024-04-25T10:30:27.097889+0000","last_became_active":"2024-04-25T10:30:24.194001+0000","last_became_peered":"2024-04-25T10:30:24.194001+0000","last_unstale":"2024-04-25T10:30:27.097889+0000","last_undegraded":"2024-04-25T10:30:27.097889+0000","last_fullsized":"2024-04-25T10:30:27.097889+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.946597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234262+0000","last_change":"2024-04-25T10:30:25.213983+0000","last_active":"2024-04-25T10:30:26.234262+0000","last_peered":"2024-04-25T10:30:26.234262+0000","last_clean":"2024-04-25T10:30:26.234262+0000","last_became_active":"2024-04-25T10:30:25.213712+0000","last_became_peered":"2024-04-25T10:30:25.213712+0000","last_unstale":"2024-04-25T10:30:26.234262+0000","last_undegraded":"2024-04-25T10:30:26.234262+0000","last_fullsized":"2024-04-25T10:30:26.234262+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:34.009571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096706+0000","last_change":"2024-04-25T10:30:24.201413+0000","last_active":"2024-04-25T10:30:27.096706+0000","last_peered":"2024-04-25T10:30:27.096706+0000","last_clean":"2024-04-25T10:30:27.096706+0000","last_became_active":"2024-04-25T10:30:24.201156+0000","last_became_peered":"2024-04-25T10:30:24.201156+0000","last_unstale":"2024-04-25T10:30:27.096706+0000","last_undegraded":"2024-04-25T10:30:27.096706+0000","last_fullsized":"2024-04-25T10:30:27.096706+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:15.045266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.199974+0000","last_change":"2024-04-25T10:30:25.209589+0000","last_active":"2024-04-25T10:30:26.199974+0000","last_peered":"2024-04-25T10:30:26.199974+0000","last_clean":"2024-04-25T10:30:26.199974+0000","last_became_active":"2024-04-25T10:30:25.209126+0000","last_became_peered":"2024-04-25T10:30:25.209126+0000","last_unstale":"2024-04-25T10:30:26.199974+0000","last_undegraded":"2024-04-25T10:30:26.199974+0000","last_fullsized":"2024-04-25T10:30:26.199974+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:40.152211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234130+0000","last_change":"2024-04-25T10:30:24.204401+0000","last_active":"2024-04-25T10:30:26.234130+0000","last_peered":"2024-04-25T10:30:26.234130+0000","last_clean":"2024-04-25T10:30:26.234130+0000","last_became_active":"2024-04-25T10:30:24.202760+0000","last_became_peered":"2024-04-25T10:30:24.202760+0000","last_unstale":"2024-04-25T10:30:26.234130+0000","last_undegraded":"2024-04-25T10:30:26.234130+0000","last_fullsized":"2024-04-25T10:30:26.234130+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:35.052141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201113+0000","last_change":"2024-04-25T10:30:25.209784+0000","last_active":"2024-04-25T10:30:26.201113+0000","last_peered":"2024-04-25T10:30:26.201113+0000","last_clean":"2024-04-25T10:30:26.201113+0000","last_became_active":"2024-04-25T10:30:25.209504+0000","last_became_peered":"2024-04-25T10:30:25.209504+0000","last_unstale":"2024-04-25T10:30:26.201113+0000","last_undegraded":"2024-04-25T10:30:26.201113+0000","last_fullsized":"2024-04-25T10:30:26.201113+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.447201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234620+0000","last_change":"2024-04-25T10:30:24.199165+0000","last_active":"2024-04-25T10:30:26.234620+0000","last_peered":"2024-04-25T10:30:26.234620+0000","last_clean":"2024-04-25T10:30:26.234620+0000","last_became_active":"2024-04-25T10:30:24.198889+0000","last_became_peered":"2024-04-25T10:30:24.198889+0000","last_unstale":"2024-04-25T10:30:26.234620+0000","last_undegraded":"2024-04-25T10:30:26.234620+0000","last_fullsized":"2024-04-25T10:30:26.234620+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:09.752276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201150+0000","last_change":"2024-04-25T10:30:25.214734+0000","last_active":"2024-04-25T10:30:26.201150+0000","last_peered":"2024-04-25T10:30:26.201150+0000","last_clean":"2024-04-25T10:30:26.201150+0000","last_became_active":"2024-04-25T10:30:25.214425+0000","last_became_peered":"2024-04-25T10:30:25.214425+0000","last_unstale":"2024-04-25T10:30:26.201150+0000","last_undegraded":"2024-04-25T10:30:26.201150+0000","last_fullsized":"2024-04-25T10:30:26.201150+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:00.456831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"169'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201222+0000","last_change":"2024-04-25T10:30:24.200959+0000","last_active":"2024-04-25T10:30:26.201222+0000","last_peered":"2024-04-25T10:30:26.201222+0000","last_clean":"2024-04-25T10:30:26.201222+0000","last_became_active":"2024-04-25T10:30:24.200611+0000","last_became_peered":"2024-04-25T10:30:24.200611+0000","last_unstale":"2024-04-25T10:30:26.201222+0000","last_undegraded":"2024-04-25T10:30:26.201222+0000","last_fullsized":"2024-04-25T10:30:26.201222+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.999131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200286+0000","last_change":"2024-04-25T10:30:25.225951+0000","last_active":"2024-04-25T10:30:26.200286+0000","last_peered":"2024-04-25T10:30:26.200286+0000","last_clean":"2024-04-25T10:30:26.200286+0000","last_became_active":"2024-04-25T10:30:25.225614+0000","last_became_peered":"2024-04-25T10:30:25.225614+0000","last_unstale":"2024-04-25T10:30:26.200286+0000","last_undegraded":"2024-04-25T10:30:26.200286+0000","last_fullsized":"2024-04-25T10:30:26.200286+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.544583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200943+0000","last_change":"2024-04-25T10:30:24.201053+0000","last_active":"2024-04-25T10:30:26.200943+0000","last_peered":"2024-04-25T10:30:26.200943+0000","last_clean":"2024-04-25T10:30:26.200943+0000","last_became_active":"2024-04-25T10:30:24.200242+0000","last_became_peered":"2024-04-25T10:30:24.200242+0000","last_unstale":"2024-04-25T10:30:26.200943+0000","last_undegraded":"2024-04-25T10:30:26.200943+0000","last_fullsized":"2024-04-25T10:30:26.200943+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:15.043592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097604+0000","last_change":"2024-04-25T10:30:25.222705+0000","last_active":"2024-04-25T10:30:27.097604+0000","last_peered":"2024-04-25T10:30:27.097604+0000","last_clean":"2024-04-25T10:30:27.097604+0000","last_became_active":"2024-04-25T10:30:25.222423+0000","last_became_peered":"2024-04-25T10:30:25.222423+0000","last_unstale":"2024-04-25T10:30:27.097604+0000","last_undegraded":"2024-04-25T10:30:27.097604+0000","last_fullsized":"2024-04-25T10:30:27.097604+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.397559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097805+0000","last_change":"2024-04-25T10:30:24.205709+0000","last_active":"2024-04-25T10:30:27.097805+0000","last_peered":"2024-04-25T10:30:27.097805+0000","last_clean":"2024-04-25T10:30:27.097805+0000","last_became_active":"2024-04-25T10:30:24.205428+0000","last_became_peered":"2024-04-25T10:30:24.205428+0000","last_unstale":"2024-04-25T10:30:27.097805+0000","last_undegraded":"2024-04-25T10:30:27.097805+0000","last_fullsized":"2024-04-25T10:30:27.097805+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:51.116449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021806+0000","last_change":"2024-04-25T10:30:25.213043+0000","last_active":"2024-04-25T10:30:27.021806+0000","last_peered":"2024-04-25T10:30:27.021806+0000","last_clean":"2024-04-25T10:30:27.021806+0000","last_became_active":"2024-04-25T10:30:25.212500+0000","last_became_peered":"2024-04-25T10:30:25.212500+0000","last_unstale":"2024-04-25T10:30:27.021806+0000","last_undegraded":"2024-04-25T10:30:27.021806+0000","last_fullsized":"2024-04-25T10:30:27.021806+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.779557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021271+0000","last_change":"2024-04-25T10:30:24.202686+0000","last_active":"2024-04-25T10:30:27.021271+0000","last_peered":"2024-04-25T10:30:27.021271+0000","last_clean":"2024-04-25T10:30:27.021271+0000","last_became_active":"2024-04-25T10:30:24.201970+0000","last_became_peered":"2024-04-25T10:30:24.201970+0000","last_unstale":"2024-04-25T10:30:27.021271+0000","last_undegraded":"2024-04-25T10:30:27.021271+0000","last_fullsized":"2024-04-25T10:30:27.021271+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:34.948942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234401+0000","last_change":"2024-04-25T10:30:25.208619+0000","last_active":"2024-04-25T10:30:26.234401+0000","last_peered":"2024-04-25T10:30:26.234401+0000","last_clean":"2024-04-25T10:30:26.234401+0000","last_became_active":"2024-04-25T10:30:25.208303+0000","last_became_peered":"2024-04-25T10:30:25.208303+0000","last_unstale":"2024-04-25T10:30:26.234401+0000","last_undegraded":"2024-04-25T10:30:26.234401+0000","last_fullsized":"2024-04-25T10:30:26.234401+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:46.810186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021353+0000","last_change":"2024-04-25T10:30:24.202866+0000","last_active":"2024-04-25T10:30:27.021353+0000","last_peered":"2024-04-25T10:30:27.021353+0000","last_clean":"2024-04-25T10:30:27.021353+0000","last_became_active":"2024-04-25T10:30:24.200965+0000","last_became_peered":"2024-04-25T10:30:24.200965+0000","last_unstale":"2024-04-25T10:30:27.021353+0000","last_undegraded":"2024-04-25T10:30:27.021353+0000","last_fullsized":"2024-04-25T10:30:27.021353+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:38.090617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021348+0000","last_change":"2024-04-25T10:30:25.211272+0000","last_active":"2024-04-25T10:30:27.021348+0000","last_peered":"2024-04-25T10:30:27.021348+0000","last_clean":"2024-04-25T10:30:27.021348+0000","last_became_active":"2024-04-25T10:30:25.210911+0000","last_became_peered":"2024-04-25T10:30:25.210911+0000","last_unstale":"2024-04-25T10:30:27.021348+0000","last_undegraded":"2024-04-25T10:30:27.021348+0000","last_fullsized":"2024-04-25T10:30:27.021348+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:21.065884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021199+0000","last_change":"2024-04-25T10:30:24.202691+0000","last_active":"2024-04-25T10:30:27.021199+0000","last_peered":"2024-04-25T10:30:27.021199+0000","last_clean":"2024-04-25T10:30:27.021199+0000","last_became_active":"2024-04-25T10:30:24.200994+0000","last_became_peered":"2024-04-25T10:30:24.200994+0000","last_unstale":"2024-04-25T10:30:27.021199+0000","last_undegraded":"2024-04-25T10:30:27.021199+0000","last_fullsized":"2024-04-25T10:30:27.021199+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.914386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097319+0000","last_change":"2024-04-25T10:30:25.213017+0000","last_active":"2024-04-25T10:30:27.097319+0000","last_peered":"2024-04-25T10:30:27.097319+0000","last_clean":"2024-04-25T10:30:27.097319+0000","last_became_active":"2024-04-25T10:30:25.212668+0000","last_became_peered":"2024-04-25T10:30:25.212668+0000","last_unstale":"2024-04-25T10:30:27.097319+0000","last_undegraded":"2024-04-25T10:30:27.097319+0000","last_fullsized":"2024-04-25T10:30:27.097319+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.305758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096902+0000","last_change":"2024-04-25T10:30:24.193826+0000","last_active":"2024-04-25T10:30:27.096902+0000","last_peered":"2024-04-25T10:30:27.096902+0000","last_clean":"2024-04-25T10:30:27.096902+0000","last_became_active":"2024-04-25T10:30:24.193546+0000","last_became_peered":"2024-04-25T10:30:24.193546+0000","last_unstale":"2024-04-25T10:30:27.096902+0000","last_undegraded":"2024-04-25T10:30:27.096902+0000","last_fullsized":"2024-04-25T10:30:27.096902+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:23.111042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096635+0000","last_change":"2024-04-25T10:30:25.217340+0000","last_active":"2024-04-25T10:30:27.096635+0000","last_peered":"2024-04-25T10:30:27.096635+0000","last_clean":"2024-04-25T10:30:27.096635+0000","last_became_active":"2024-04-25T10:30:25.217071+0000","last_became_peered":"2024-04-25T10:30:25.217071+0000","last_unstale":"2024-04-25T10:30:27.096635+0000","last_undegraded":"2024-04-25T10:30:27.096635+0000","last_fullsized":"2024-04-25T10:30:27.096635+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:11.242436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094066+0000","last_change":"2024-04-25T10:30:24.194259+0000","last_active":"2024-04-25T10:30:27.094066+0000","last_peered":"2024-04-25T10:30:27.094066+0000","last_clean":"2024-04-25T10:30:27.094066+0000","last_became_active":"2024-04-25T10:30:24.193996+0000","last_became_peered":"2024-04-25T10:30:24.193996+0000","last_unstale":"2024-04-25T10:30:27.094066+0000","last_undegraded":"2024-04-25T10:30:27.094066+0000","last_fullsized":"2024-04-25T10:30:27.094066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.511957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":454,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021210+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:30:27.021210+0000","last_peered":"2024-04-25T10:30:27.021210+0000","last_clean":"2024-04-25T10:30:27.021210+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:30:27.021210+0000","last_undegraded":"2024-04-25T10:30:27.021210+0000","last_fullsized":"2024-04-25T10:30:27.021210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55476,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316364,"statfs":{"total":96636764160,"available":96579956736,"internally_reserved":0,"allocated":2138112,"data_stored":930359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55468,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316372,"statfs":{"total":96636764160,"available":96579964928,"internally_reserved":0,"allocated":2121728,"data_stored":928725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60212,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311628,"statfs":{"total":96636764160,"available":96575107072,"internally_reserved":0,"allocated":2719744,"data_stored":1519607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47152,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324688,"statfs":{"total":96636764160,"available":96588480512,"internally_reserved":0,"allocated":2134016,"data_stored":930317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509595,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69288,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302552,"statfs":{"total":96636764160,"available":96565813248,"internally_reserved":0,"allocated":2121728,"data_stored":928705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47716,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324124,"statfs":{"total":96636764160,"available":96587902976,"internally_reserved":0,"allocated":2711552,"data_stored":1519003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55464,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316376,"statfs":{"total":96636764160,"available":96579969024,"internally_reserved":0,"allocated":2125824,"data_stored":929251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65124,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306716,"statfs":{"total":96636764160,"available":96570077184,"internally_reserved":0,"allocated":2121728,"data_stored":928669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:29.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:30:29.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 Processor -- start 2024-04-25T10:30:29.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 -- start start 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 0x7f7fc806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7fc806b2c0 0x7f7fc806b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc80b5780 con 0x7f7fc80b5010 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc80b5a60 con 0x7f7fc806b2c0 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fcce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80b5d40 con 0x7f7fc815db40 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fbfe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7fc80b5010 0x7f7fc815df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33636/0 (socket says 172.21.15.45:33636) 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fbfe00640 1 -- 172.21.15.45:0/1769961527 learned_addr learned my addr 172.21.15.45:0/1769961527 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 0x7f7fc806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/1769961527 >> v2:172.21.15.117:3300/0 conn(0x7f7fc806b2c0 0x7f7fc806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357973801 0 0) 0x7f7fc80b5780 con 0x7f7fc80b5010 2024-04-25T10:30:29.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fac003530 con 0x7f7fc80b5010 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350203894 0 0) 0x7f7fac003530 con 0x7f7fc80b5010 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 msgr2=0x7f7fc806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 0x7f7fc806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 >> v2:172.21.15.117:3300/0 conn(0x7f7fc806b2c0 msgr2=0x7f7fc806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 --2- 172.21.15.45:0/1769961527 >> v2:172.21.15.117:3300/0 conn(0x7f7fc806b2c0 0x7f7fc806b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80b6020 con 0x7f7fc80b5010 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f7fb0001680 con 0x7f7fc80b5010 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 >> v1:172.21.15.45:6789/0 conn(0x7f7fc80b5010 legacy=0x7f7fc815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fac0039c0 0x7f7fac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 0x7f7fac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac004d50 0x7f7fac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80b5a60 con 0x7f7fac0039c0 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80b5d40 con 0x7f7fac004d50 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac00ba90 con 0x7f7fc815db40 2024-04-25T10:30:29.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/1769961527 >> v2:172.21.15.117:3300/0 conn(0x7f7fc806b2c0 0x7f7fc806b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fac0039c0 0x7f7fac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fbfe00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 0x7f7fac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac004d50 0x7f7fac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 -- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 msgr2=0x7f7fac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc815db40 0x7f7fac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 -- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac004d50 msgr2=0x7f7fac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac004d50 0x7f7fac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7400640 1 -- 172.21.15.45:0/1769961527 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac003530 con 0x7f7fac0039c0 2024-04-25T10:30:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.499+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fac004d50 0x7f7fac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fc7400640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fac0039c0 0x7f7fac003de0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009ae0 tx=0x7f7fb801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c298f6ff869f7c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80b1070 con 0x7f7fac0039c0 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb80a4bb0 con 0x7f7fac0039c0 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fc6a00640 1 -- 172.21.15.45:0/1769961527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80096b0 con 0x7f7fac0039c0 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fcce00640 1 -- 172.21.15.45:0/1769961527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fac0039c0 msgr2=0x7f7fac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fcce00640 1 --2- 172.21.15.45:0/1769961527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fac0039c0 0x7f7fac003de0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009ae0 tx=0x7f7fb801c750 comp rx=0 tx=0).stop 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fc7e00640 1 -- 172.21.15.45:0/1769961527 reap_dead start 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fcce00640 1 -- 172.21.15.45:0/1769961527 shutdown_connections 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fcce00640 1 -- 172.21.15.45:0/1769961527 >> 172.21.15.45:0/1769961527 conn(0x7f7fc80d17b0 msgr2=0x7f7fc80d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fcce00640 1 -- 172.21.15.45:0/1769961527 shutdown_connections 2024-04-25T10:30:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.503+0000 7f7fcce00640 1 -- 172.21.15.45:0/1769961527 wait complete. 2024-04-25T10:30:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 Processor -- start 2024-04-25T10:30:29.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 -- start start 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fac0039c0 0x7f7fc81590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8155d10 0x7f7fc815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 0x7f7fc815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc806ac90 con 0x7f7fc815bb50 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80b5450 con 0x7f7fc8155d10 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc806ae10 con 0x7f7fac0039c0 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 0x7f7fc815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 0x7f7fc815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55768/0 (socket says 172.21.15.45:55768) 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 -- 172.21.15.45:0/699209473 learned_addr learned my addr 172.21.15.45:0/699209473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7400640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fac0039c0 0x7f7fc81590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fbfe00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8155d10 0x7f7fc815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 -- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fac0039c0 msgr2=0x7f7fc81590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fac0039c0 0x7f7fc81590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 -- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8155d10 msgr2=0x7f7fc815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8155d10 0x7f7fc815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 -- 172.21.15.45:0/699209473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc815a5f0 con 0x7f7fc815bb50 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fc7e00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 0x7f7fc815c000 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8068870 tx=0x7f7fc0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0014410 con 0x7f7fc815bb50 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0019070 con 0x7f7fc815bb50 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00147f0 con 0x7f7fc815bb50 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc815a8d0 con 0x7f7fc815bb50 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.507+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc8156660 con 0x7f7fc815bb50 2024-04-25T10:30:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.511+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc8075120 con 0x7f7fc815bb50 2024-04-25T10:30:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.511+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fc0014990 con 0x7f7fc815bb50 2024-04-25T10:30:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.511+0000 7f7fbf400640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7fa4066f90 0x7f7fa4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.511+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fc0085540 con 0x7f7fc815bb50 2024-04-25T10:30:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.511+0000 7f7fc7400640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7fa4066f90 0x7f7fa4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.511+0000 7f7fc7400640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7fa4066f90 0x7f7fa4069450 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80ae000 tx=0x7f7fb80046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.515+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fc0055e60 con 0x7f7fc815bb50 2024-04-25T10:30:29.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.675+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fc80b38e0 con 0x7f7fa4066f90 2024-04-25T10:30:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.683+0000 7f7fbf400640 1 -- 172.21.15.45:0/699209473 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f7fc80b38e0 con 0x7f7fa4066f90 2024-04-25T10:30:29.687 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7fa4066f90 msgr2=0x7f7fa4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7fa4066f90 0x7f7fa4069450 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80ae000 tx=0x7f7fb80046d0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 msgr2=0x7f7fc815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 0x7f7fc815c000 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8068870 tx=0x7f7fc0007700 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 shutdown_connections 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7fa4066f90 0x7f7fa4069450 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fc815bb50 0x7f7fc815c000 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc8155d10 0x7f7fc815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 --2- 172.21.15.45:0/699209473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fac0039c0 0x7f7fc81590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 >> 172.21.15.45:0/699209473 conn(0x7f7fc80d17b0 msgr2=0x7f7fc81e4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 shutdown_connections 2024-04-25T10:30:29.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.687+0000 7f7fcce00640 1 -- 172.21.15.45:0/699209473 wait complete. 2024-04-25T10:30:29.708 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2024-04-25T10:30:29.050056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":455900,"kb_used_data":17768,"kb_used_omap":12,"kb_used_meta":438003,"kb_avail":754518820,"statfs":{"total":773094113280,"available":772627271680,"internally_reserved":0,"allocated":18194432,"data_stored":8614636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448515632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.867934"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234611+0000","last_change":"2024-04-25T10:30:25.214742+0000","last_active":"2024-04-25T10:30:26.234611+0000","last_peered":"2024-04-25T10:30:26.234611+0000","last_clean":"2024-04-25T10:30:26.234611+0000","last_became_active":"2024-04-25T10:30:25.210635+0000","last_became_peered":"2024-04-25T10:30:25.210635+0000","last_unstale":"2024-04-25T10:30:26.234611+0000","last_undegraded":"2024-04-25T10:30:26.234611+0000","last_fullsized":"2024-04-25T10:30:26.234611+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:52.008231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021849+0000","last_change":"2024-04-25T10:30:25.203862+0000","last_active":"2024-04-25T10:30:27.021849+0000","last_peered":"2024-04-25T10:30:27.021849+0000","last_clean":"2024-04-25T10:30:27.021849+0000","last_became_active":"2024-04-25T10:30:25.202849+0000","last_became_peered":"2024-04-25T10:30:25.202849+0000","last_unstale":"2024-04-25T10:30:27.021849+0000","last_undegraded":"2024-04-25T10:30:27.021849+0000","last_fullsized":"2024-04-25T10:30:27.021849+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:14.072724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200804+0000","last_change":"2024-04-25T10:30:25.217676+0000","last_active":"2024-04-25T10:30:26.200804+0000","last_peered":"2024-04-25T10:30:26.200804+0000","last_clean":"2024-04-25T10:30:26.200804+0000","last_became_active":"2024-04-25T10:30:25.217093+0000","last_became_peered":"2024-04-25T10:30:25.217093+0000","last_unstale":"2024-04-25T10:30:26.200804+0000","last_undegraded":"2024-04-25T10:30:26.200804+0000","last_fullsized":"2024-04-25T10:30:26.200804+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:38.267448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021727+0000","last_change":"2024-04-25T10:30:25.223287+0000","last_active":"2024-04-25T10:30:27.021727+0000","last_peered":"2024-04-25T10:30:27.021727+0000","last_clean":"2024-04-25T10:30:27.021727+0000","last_became_active":"2024-04-25T10:30:25.222139+0000","last_became_peered":"2024-04-25T10:30:25.222139+0000","last_unstale":"2024-04-25T10:30:27.021727+0000","last_undegraded":"2024-04-25T10:30:27.021727+0000","last_fullsized":"2024-04-25T10:30:27.021727+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:43.771462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097499+0000","last_change":"2024-04-25T10:30:25.215431+0000","last_active":"2024-04-25T10:30:27.097499+0000","last_peered":"2024-04-25T10:30:27.097499+0000","last_clean":"2024-04-25T10:30:27.097499+0000","last_became_active":"2024-04-25T10:30:25.215075+0000","last_became_peered":"2024-04-25T10:30:25.215075+0000","last_unstale":"2024-04-25T10:30:27.097499+0000","last_undegraded":"2024-04-25T10:30:27.097499+0000","last_fullsized":"2024-04-25T10:30:27.097499+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:42.746015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096809+0000","last_change":"2024-04-25T10:30:25.213934+0000","last_active":"2024-04-25T10:30:27.096809+0000","last_peered":"2024-04-25T10:30:27.096809+0000","last_clean":"2024-04-25T10:30:27.096809+0000","last_became_active":"2024-04-25T10:30:25.213664+0000","last_became_peered":"2024-04-25T10:30:25.213664+0000","last_unstale":"2024-04-25T10:30:27.096809+0000","last_undegraded":"2024-04-25T10:30:27.096809+0000","last_fullsized":"2024-04-25T10:30:27.096809+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:53.201775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194264+0000","last_change":"2024-04-25T10:30:25.222788+0000","last_active":"2024-04-25T10:30:26.194264+0000","last_peered":"2024-04-25T10:30:26.194264+0000","last_clean":"2024-04-25T10:30:26.194264+0000","last_became_active":"2024-04-25T10:30:25.222498+0000","last_became_peered":"2024-04-25T10:30:25.222498+0000","last_unstale":"2024-04-25T10:30:26.194264+0000","last_undegraded":"2024-04-25T10:30:26.194264+0000","last_fullsized":"2024-04-25T10:30:26.194264+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:03.093782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096764+0000","last_change":"2024-04-25T10:30:25.220781+0000","last_active":"2024-04-25T10:30:27.096764+0000","last_peered":"2024-04-25T10:30:27.096764+0000","last_clean":"2024-04-25T10:30:27.096764+0000","last_became_active":"2024-04-25T10:30:25.220534+0000","last_became_peered":"2024-04-25T10:30:25.220534+0000","last_unstale":"2024-04-25T10:30:27.096764+0000","last_undegraded":"2024-04-25T10:30:27.096764+0000","last_fullsized":"2024-04-25T10:30:27.096764+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:13.160678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021887+0000","last_change":"2024-04-25T10:30:25.212488+0000","last_active":"2024-04-25T10:30:27.021887+0000","last_peered":"2024-04-25T10:30:27.021887+0000","last_clean":"2024-04-25T10:30:27.021887+0000","last_became_active":"2024-04-25T10:30:25.212177+0000","last_became_peered":"2024-04-25T10:30:25.212177+0000","last_unstale":"2024-04-25T10:30:27.021887+0000","last_undegraded":"2024-04-25T10:30:27.021887+0000","last_fullsized":"2024-04-25T10:30:27.021887+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:01.936095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200818+0000","last_change":"2024-04-25T10:30:25.213057+0000","last_active":"2024-04-25T10:30:26.200818+0000","last_peered":"2024-04-25T10:30:26.200818+0000","last_clean":"2024-04-25T10:30:26.200818+0000","last_became_active":"2024-04-25T10:30:25.212757+0000","last_became_peered":"2024-04-25T10:30:25.212757+0000","last_unstale":"2024-04-25T10:30:26.200818+0000","last_undegraded":"2024-04-25T10:30:26.200818+0000","last_fullsized":"2024-04-25T10:30:26.200818+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:42.746015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234467+0000","last_change":"2024-04-25T10:30:25.204387+0000","last_active":"2024-04-25T10:30:26.234467+0000","last_peered":"2024-04-25T10:30:26.234467+0000","last_clean":"2024-04-25T10:30:26.234467+0000","last_became_active":"2024-04-25T10:30:25.204063+0000","last_became_peered":"2024-04-25T10:30:25.204063+0000","last_unstale":"2024-04-25T10:30:26.234467+0000","last_undegraded":"2024-04-25T10:30:26.234467+0000","last_fullsized":"2024-04-25T10:30:26.234467+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:56.347500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094261+0000","last_change":"2024-04-25T10:30:25.221199+0000","last_active":"2024-04-25T10:30:27.094261+0000","last_peered":"2024-04-25T10:30:27.094261+0000","last_clean":"2024-04-25T10:30:27.094261+0000","last_became_active":"2024-04-25T10:30:25.220527+0000","last_became_peered":"2024-04-25T10:30:25.220527+0000","last_unstale":"2024-04-25T10:30:27.094261+0000","last_undegraded":"2024-04-25T10:30:27.094261+0000","last_fullsized":"2024-04-25T10:30:27.094261+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:10.809229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234272+0000","last_change":"2024-04-25T10:30:25.211536+0000","last_active":"2024-04-25T10:30:26.234272+0000","last_peered":"2024-04-25T10:30:26.234272+0000","last_clean":"2024-04-25T10:30:26.234272+0000","last_became_active":"2024-04-25T10:30:25.211254+0000","last_became_peered":"2024-04-25T10:30:25.211254+0000","last_unstale":"2024-04-25T10:30:26.234272+0000","last_undegraded":"2024-04-25T10:30:26.234272+0000","last_fullsized":"2024-04-25T10:30:26.234272+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:19.179379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021228+0000","last_change":"2024-04-25T10:30:25.212804+0000","last_active":"2024-04-25T10:30:27.021228+0000","last_peered":"2024-04-25T10:30:27.021228+0000","last_clean":"2024-04-25T10:30:27.021228+0000","last_became_active":"2024-04-25T10:30:25.212107+0000","last_became_peered":"2024-04-25T10:30:25.212107+0000","last_unstale":"2024-04-25T10:30:27.021228+0000","last_undegraded":"2024-04-25T10:30:27.021228+0000","last_fullsized":"2024-04-25T10:30:27.021228+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:53.413918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234259+0000","last_change":"2024-04-25T10:30:25.205188+0000","last_active":"2024-04-25T10:30:26.234259+0000","last_peered":"2024-04-25T10:30:26.234259+0000","last_clean":"2024-04-25T10:30:26.234259+0000","last_became_active":"2024-04-25T10:30:25.204921+0000","last_became_peered":"2024-04-25T10:30:25.204921+0000","last_unstale":"2024-04-25T10:30:26.234259+0000","last_undegraded":"2024-04-25T10:30:26.234259+0000","last_fullsized":"2024-04-25T10:30:26.234259+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:29.866914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.199760+0000","last_change":"2024-04-25T10:30:25.212298+0000","last_active":"2024-04-25T10:30:26.199760+0000","last_peered":"2024-04-25T10:30:26.199760+0000","last_clean":"2024-04-25T10:30:26.199760+0000","last_became_active":"2024-04-25T10:30:25.212026+0000","last_became_peered":"2024-04-25T10:30:25.212026+0000","last_unstale":"2024-04-25T10:30:26.199760+0000","last_undegraded":"2024-04-25T10:30:26.199760+0000","last_fullsized":"2024-04-25T10:30:26.199760+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:42.746015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097701+0000","last_change":"2024-04-25T10:30:25.217970+0000","last_active":"2024-04-25T10:30:27.097701+0000","last_peered":"2024-04-25T10:30:27.097701+0000","last_clean":"2024-04-25T10:30:27.097701+0000","last_became_active":"2024-04-25T10:30:25.217671+0000","last_became_peered":"2024-04-25T10:30:25.217671+0000","last_unstale":"2024-04-25T10:30:27.097701+0000","last_undegraded":"2024-04-25T10:30:27.097701+0000","last_fullsized":"2024-04-25T10:30:27.097701+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:11.094619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194756+0000","last_change":"2024-04-25T10:30:25.212332+0000","last_active":"2024-04-25T10:30:26.194756+0000","last_peered":"2024-04-25T10:30:26.194756+0000","last_clean":"2024-04-25T10:30:26.194756+0000","last_became_active":"2024-04-25T10:30:25.212029+0000","last_became_peered":"2024-04-25T10:30:25.212029+0000","last_unstale":"2024-04-25T10:30:26.194756+0000","last_undegraded":"2024-04-25T10:30:26.194756+0000","last_fullsized":"2024-04-25T10:30:26.194756+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:24.917853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200877+0000","last_change":"2024-04-25T10:30:25.227421+0000","last_active":"2024-04-25T10:30:26.200877+0000","last_peered":"2024-04-25T10:30:26.200877+0000","last_clean":"2024-04-25T10:30:26.200877+0000","last_became_active":"2024-04-25T10:30:25.227046+0000","last_became_peered":"2024-04-25T10:30:25.227046+0000","last_unstale":"2024-04-25T10:30:26.200877+0000","last_undegraded":"2024-04-25T10:30:26.200877+0000","last_fullsized":"2024-04-25T10:30:26.200877+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:20.546559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097752+0000","last_change":"2024-04-25T10:30:25.230009+0000","last_active":"2024-04-25T10:30:27.097752+0000","last_peered":"2024-04-25T10:30:27.097752+0000","last_clean":"2024-04-25T10:30:27.097752+0000","last_became_active":"2024-04-25T10:30:25.229688+0000","last_became_peered":"2024-04-25T10:30:25.229688+0000","last_unstale":"2024-04-25T10:30:27.097752+0000","last_undegraded":"2024-04-25T10:30:27.097752+0000","last_fullsized":"2024-04-25T10:30:27.097752+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.544583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200200+0000","last_change":"2024-04-25T10:30:25.212112+0000","last_active":"2024-04-25T10:30:26.200200+0000","last_peered":"2024-04-25T10:30:26.200200+0000","last_clean":"2024-04-25T10:30:26.200200+0000","last_became_active":"2024-04-25T10:30:25.211822+0000","last_became_peered":"2024-04-25T10:30:25.211822+0000","last_unstale":"2024-04-25T10:30:26.200200+0000","last_undegraded":"2024-04-25T10:30:26.200200+0000","last_fullsized":"2024-04-25T10:30:26.200200+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.397559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097321+0000","last_change":"2024-04-25T10:30:25.230212+0000","last_active":"2024-04-25T10:30:27.097321+0000","last_peered":"2024-04-25T10:30:27.097321+0000","last_clean":"2024-04-25T10:30:27.097321+0000","last_became_active":"2024-04-25T10:30:25.229914+0000","last_became_peered":"2024-04-25T10:30:25.229914+0000","last_unstale":"2024-04-25T10:30:27.097321+0000","last_undegraded":"2024-04-25T10:30:27.097321+0000","last_fullsized":"2024-04-25T10:30:27.097321+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:40.152211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.199748+0000","last_change":"2024-04-25T10:30:25.220723+0000","last_active":"2024-04-25T10:30:26.199748+0000","last_peered":"2024-04-25T10:30:26.199748+0000","last_clean":"2024-04-25T10:30:26.199748+0000","last_became_active":"2024-04-25T10:30:25.220413+0000","last_became_peered":"2024-04-25T10:30:25.220413+0000","last_unstale":"2024-04-25T10:30:26.199748+0000","last_undegraded":"2024-04-25T10:30:26.199748+0000","last_fullsized":"2024-04-25T10:30:26.199748+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.305758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096577+0000","last_change":"2024-04-25T10:30:25.226698+0000","last_active":"2024-04-25T10:30:27.096577+0000","last_peered":"2024-04-25T10:30:27.096577+0000","last_clean":"2024-04-25T10:30:27.096577+0000","last_became_active":"2024-04-25T10:30:25.224720+0000","last_became_peered":"2024-04-25T10:30:25.224720+0000","last_unstale":"2024-04-25T10:30:27.096577+0000","last_undegraded":"2024-04-25T10:30:27.096577+0000","last_fullsized":"2024-04-25T10:30:27.096577+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:10.809229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021875+0000","last_change":"2024-04-25T10:30:25.204903+0000","last_active":"2024-04-25T10:30:27.021875+0000","last_peered":"2024-04-25T10:30:27.021875+0000","last_clean":"2024-04-25T10:30:27.021875+0000","last_became_active":"2024-04-25T10:30:25.202766+0000","last_became_peered":"2024-04-25T10:30:25.202766+0000","last_unstale":"2024-04-25T10:30:27.021875+0000","last_undegraded":"2024-04-25T10:30:27.021875+0000","last_fullsized":"2024-04-25T10:30:27.021875+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:47.666542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094591+0000","last_change":"2024-04-25T10:30:25.214966+0000","last_active":"2024-04-25T10:30:27.094591+0000","last_peered":"2024-04-25T10:30:27.094591+0000","last_clean":"2024-04-25T10:30:27.094591+0000","last_became_active":"2024-04-25T10:30:25.214039+0000","last_became_peered":"2024-04-25T10:30:25.214039+0000","last_unstale":"2024-04-25T10:30:27.094591+0000","last_undegraded":"2024-04-25T10:30:27.094591+0000","last_fullsized":"2024-04-25T10:30:27.094591+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:16.659507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200992+0000","last_change":"2024-04-25T10:30:25.217859+0000","last_active":"2024-04-25T10:30:26.200992+0000","last_peered":"2024-04-25T10:30:26.200992+0000","last_clean":"2024-04-25T10:30:26.200992+0000","last_became_active":"2024-04-25T10:30:25.217591+0000","last_became_peered":"2024-04-25T10:30:25.217591+0000","last_unstale":"2024-04-25T10:30:26.200992+0000","last_undegraded":"2024-04-25T10:30:26.200992+0000","last_fullsized":"2024-04-25T10:30:26.200992+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:13.216699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201207+0000","last_change":"2024-04-25T10:30:25.225196+0000","last_active":"2024-04-25T10:30:26.201207+0000","last_peered":"2024-04-25T10:30:26.201207+0000","last_clean":"2024-04-25T10:30:26.201207+0000","last_became_active":"2024-04-25T10:30:25.224911+0000","last_became_peered":"2024-04-25T10:30:25.224911+0000","last_unstale":"2024-04-25T10:30:26.201207+0000","last_undegraded":"2024-04-25T10:30:26.201207+0000","last_fullsized":"2024-04-25T10:30:26.201207+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.544583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094172+0000","last_change":"2024-04-25T10:30:25.208393+0000","last_active":"2024-04-25T10:30:27.094172+0000","last_peered":"2024-04-25T10:30:27.094172+0000","last_clean":"2024-04-25T10:30:27.094172+0000","last_became_active":"2024-04-25T10:30:25.208099+0000","last_became_peered":"2024-04-25T10:30:25.208099+0000","last_unstale":"2024-04-25T10:30:27.094172+0000","last_undegraded":"2024-04-25T10:30:27.094172+0000","last_fullsized":"2024-04-25T10:30:27.094172+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:52.010205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094359+0000","last_change":"2024-04-25T10:30:25.210082+0000","last_active":"2024-04-25T10:30:27.094359+0000","last_peered":"2024-04-25T10:30:27.094359+0000","last_clean":"2024-04-25T10:30:27.094359+0000","last_became_active":"2024-04-25T10:30:25.209722+0000","last_became_peered":"2024-04-25T10:30:25.209722+0000","last_unstale":"2024-04-25T10:30:27.094359+0000","last_undegraded":"2024-04-25T10:30:27.094359+0000","last_fullsized":"2024-04-25T10:30:27.094359+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.053768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096957+0000","last_change":"2024-04-25T10:30:25.221732+0000","last_active":"2024-04-25T10:30:27.096957+0000","last_peered":"2024-04-25T10:30:27.096957+0000","last_clean":"2024-04-25T10:30:27.096957+0000","last_became_active":"2024-04-25T10:30:25.221431+0000","last_became_peered":"2024-04-25T10:30:25.221431+0000","last_unstale":"2024-04-25T10:30:27.096957+0000","last_undegraded":"2024-04-25T10:30:27.096957+0000","last_fullsized":"2024-04-25T10:30:27.096957+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:30.406873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096636+0000","last_change":"2024-04-25T10:30:25.220293+0000","last_active":"2024-04-25T10:30:27.096636+0000","last_peered":"2024-04-25T10:30:27.096636+0000","last_clean":"2024-04-25T10:30:27.096636+0000","last_became_active":"2024-04-25T10:30:25.218531+0000","last_became_peered":"2024-04-25T10:30:25.218531+0000","last_unstale":"2024-04-25T10:30:27.096636+0000","last_undegraded":"2024-04-25T10:30:27.096636+0000","last_fullsized":"2024-04-25T10:30:27.096636+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.053768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097272+0000","last_change":"2024-04-25T10:30:25.229071+0000","last_active":"2024-04-25T10:30:27.097272+0000","last_peered":"2024-04-25T10:30:27.097272+0000","last_clean":"2024-04-25T10:30:27.097272+0000","last_became_active":"2024-04-25T10:30:25.228802+0000","last_became_peered":"2024-04-25T10:30:25.228802+0000","last_unstale":"2024-04-25T10:30:27.097272+0000","last_undegraded":"2024-04-25T10:30:27.097272+0000","last_fullsized":"2024-04-25T10:30:27.097272+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.828621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"168'4","reported_seq":18,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201092+0000","last_change":"2024-04-25T10:30:24.193125+0000","last_active":"2024-04-25T10:30:26.201092+0000","last_peered":"2024-04-25T10:30:26.201092+0000","last_clean":"2024-04-25T10:30:26.201092+0000","last_became_active":"2024-04-25T10:30:24.192792+0000","last_became_peered":"2024-04-25T10:30:24.192792+0000","last_unstale":"2024-04-25T10:30:26.201092+0000","last_undegraded":"2024-04-25T10:30:26.201092+0000","last_fullsized":"2024-04-25T10:30:26.201092+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:12.906089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201366+0000","last_change":"2024-04-25T10:30:25.224829+0000","last_active":"2024-04-25T10:30:26.201366+0000","last_peered":"2024-04-25T10:30:26.201366+0000","last_clean":"2024-04-25T10:30:26.201366+0000","last_became_active":"2024-04-25T10:30:25.223655+0000","last_became_peered":"2024-04-25T10:30:25.223655+0000","last_unstale":"2024-04-25T10:30:26.201366+0000","last_undegraded":"2024-04-25T10:30:26.201366+0000","last_fullsized":"2024-04-25T10:30:26.201366+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:38.267448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094463+0000","last_change":"2024-04-25T10:30:24.203317+0000","last_active":"2024-04-25T10:30:27.094463+0000","last_peered":"2024-04-25T10:30:27.094463+0000","last_clean":"2024-04-25T10:30:27.094463+0000","last_became_active":"2024-04-25T10:30:24.202617+0000","last_became_peered":"2024-04-25T10:30:24.202617+0000","last_unstale":"2024-04-25T10:30:27.094463+0000","last_undegraded":"2024-04-25T10:30:27.094463+0000","last_fullsized":"2024-04-25T10:30:27.094463+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:43.317402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021636+0000","last_change":"2024-04-25T10:30:25.199054+0000","last_active":"2024-04-25T10:30:27.021636+0000","last_peered":"2024-04-25T10:30:27.021636+0000","last_clean":"2024-04-25T10:30:27.021636+0000","last_became_active":"2024-04-25T10:30:25.198763+0000","last_became_peered":"2024-04-25T10:30:25.198763+0000","last_unstale":"2024-04-25T10:30:27.021636+0000","last_undegraded":"2024-04-25T10:30:27.021636+0000","last_fullsized":"2024-04-25T10:30:27.021636+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:16.172941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097418+0000","last_change":"2024-04-25T10:30:24.204900+0000","last_active":"2024-04-25T10:30:27.097418+0000","last_peered":"2024-04-25T10:30:27.097418+0000","last_clean":"2024-04-25T10:30:27.097418+0000","last_became_active":"2024-04-25T10:30:24.204595+0000","last_became_peered":"2024-04-25T10:30:24.204595+0000","last_unstale":"2024-04-25T10:30:27.097418+0000","last_undegraded":"2024-04-25T10:30:27.097418+0000","last_fullsized":"2024-04-25T10:30:27.097418+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.511957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021682+0000","last_change":"2024-04-25T10:30:25.211817+0000","last_active":"2024-04-25T10:30:27.021682+0000","last_peered":"2024-04-25T10:30:27.021682+0000","last_clean":"2024-04-25T10:30:27.021682+0000","last_became_active":"2024-04-25T10:30:25.211576+0000","last_became_peered":"2024-04-25T10:30:25.211576+0000","last_unstale":"2024-04-25T10:30:27.021682+0000","last_undegraded":"2024-04-25T10:30:27.021682+0000","last_fullsized":"2024-04-25T10:30:27.021682+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.324312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094336+0000","last_change":"2024-04-25T10:30:24.204491+0000","last_active":"2024-04-25T10:30:27.094336+0000","last_peered":"2024-04-25T10:30:27.094336+0000","last_clean":"2024-04-25T10:30:27.094336+0000","last_became_active":"2024-04-25T10:30:24.203319+0000","last_became_peered":"2024-04-25T10:30:24.203319+0000","last_unstale":"2024-04-25T10:30:27.094336+0000","last_undegraded":"2024-04-25T10:30:27.094336+0000","last_fullsized":"2024-04-25T10:30:27.094336+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.039848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096981+0000","last_change":"2024-04-25T10:30:25.213178+0000","last_active":"2024-04-25T10:30:27.096981+0000","last_peered":"2024-04-25T10:30:27.096981+0000","last_clean":"2024-04-25T10:30:27.096981+0000","last_became_active":"2024-04-25T10:30:25.212906+0000","last_became_peered":"2024-04-25T10:30:25.212906+0000","last_unstale":"2024-04-25T10:30:27.096981+0000","last_undegraded":"2024-04-25T10:30:27.096981+0000","last_fullsized":"2024-04-25T10:30:27.096981+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:15.735860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097437+0000","last_change":"2024-04-25T10:30:24.205083+0000","last_active":"2024-04-25T10:30:27.097437+0000","last_peered":"2024-04-25T10:30:27.097437+0000","last_clean":"2024-04-25T10:30:27.097437+0000","last_became_active":"2024-04-25T10:30:24.204652+0000","last_became_peered":"2024-04-25T10:30:24.204652+0000","last_unstale":"2024-04-25T10:30:27.097437+0000","last_undegraded":"2024-04-25T10:30:27.097437+0000","last_fullsized":"2024-04-25T10:30:27.097437+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.684197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097367+0000","last_change":"2024-04-25T10:30:25.215934+0000","last_active":"2024-04-25T10:30:27.097367+0000","last_peered":"2024-04-25T10:30:27.097367+0000","last_clean":"2024-04-25T10:30:27.097367+0000","last_became_active":"2024-04-25T10:30:25.215683+0000","last_became_peered":"2024-04-25T10:30:25.215683+0000","last_unstale":"2024-04-25T10:30:27.097367+0000","last_undegraded":"2024-04-25T10:30:27.097367+0000","last_fullsized":"2024-04-25T10:30:27.097367+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:03.590877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097250+0000","last_change":"2024-04-25T10:30:24.204028+0000","last_active":"2024-04-25T10:30:27.097250+0000","last_peered":"2024-04-25T10:30:27.097250+0000","last_clean":"2024-04-25T10:30:27.097250+0000","last_became_active":"2024-04-25T10:30:24.203757+0000","last_became_peered":"2024-04-25T10:30:24.203757+0000","last_unstale":"2024-04-25T10:30:27.097250+0000","last_undegraded":"2024-04-25T10:30:27.097250+0000","last_fullsized":"2024-04-25T10:30:27.097250+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:13.039848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234252+0000","last_change":"2024-04-25T10:30:25.211466+0000","last_active":"2024-04-25T10:30:26.234252+0000","last_peered":"2024-04-25T10:30:26.234252+0000","last_clean":"2024-04-25T10:30:26.234252+0000","last_became_active":"2024-04-25T10:30:25.211171+0000","last_became_peered":"2024-04-25T10:30:25.211171+0000","last_unstale":"2024-04-25T10:30:26.234252+0000","last_undegraded":"2024-04-25T10:30:26.234252+0000","last_fullsized":"2024-04-25T10:30:26.234252+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:30.230476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194093+0000","last_change":"2024-04-25T10:30:24.197654+0000","last_active":"2024-04-25T10:30:26.194093+0000","last_peered":"2024-04-25T10:30:26.194093+0000","last_clean":"2024-04-25T10:30:26.194093+0000","last_became_active":"2024-04-25T10:30:24.197357+0000","last_became_peered":"2024-04-25T10:30:24.197357+0000","last_unstale":"2024-04-25T10:30:26.194093+0000","last_undegraded":"2024-04-25T10:30:26.194093+0000","last_fullsized":"2024-04-25T10:30:26.194093+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:37.256374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200942+0000","last_change":"2024-04-25T10:30:25.224500+0000","last_active":"2024-04-25T10:30:26.200942+0000","last_peered":"2024-04-25T10:30:26.200942+0000","last_clean":"2024-04-25T10:30:26.200942+0000","last_became_active":"2024-04-25T10:30:25.220464+0000","last_became_peered":"2024-04-25T10:30:25.220464+0000","last_unstale":"2024-04-25T10:30:26.200942+0000","last_undegraded":"2024-04-25T10:30:26.200942+0000","last_fullsized":"2024-04-25T10:30:26.200942+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:40.107598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234570+0000","last_change":"2024-04-25T10:30:24.205112+0000","last_active":"2024-04-25T10:30:26.234570+0000","last_peered":"2024-04-25T10:30:26.234570+0000","last_clean":"2024-04-25T10:30:26.234570+0000","last_became_active":"2024-04-25T10:30:24.203155+0000","last_became_peered":"2024-04-25T10:30:24.203155+0000","last_unstale":"2024-04-25T10:30:26.234570+0000","last_undegraded":"2024-04-25T10:30:26.234570+0000","last_fullsized":"2024-04-25T10:30:26.234570+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:40.325544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201369+0000","last_change":"2024-04-25T10:30:25.220632+0000","last_active":"2024-04-25T10:30:26.201369+0000","last_peered":"2024-04-25T10:30:26.201369+0000","last_clean":"2024-04-25T10:30:26.201369+0000","last_became_active":"2024-04-25T10:30:25.219956+0000","last_became_peered":"2024-04-25T10:30:25.219956+0000","last_unstale":"2024-04-25T10:30:26.201369+0000","last_undegraded":"2024-04-25T10:30:26.201369+0000","last_fullsized":"2024-04-25T10:30:26.201369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:20.546559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201390+0000","last_change":"2024-04-25T10:30:24.200093+0000","last_active":"2024-04-25T10:30:26.201390+0000","last_peered":"2024-04-25T10:30:26.201390+0000","last_clean":"2024-04-25T10:30:26.201390+0000","last_became_active":"2024-04-25T10:30:24.199728+0000","last_became_peered":"2024-04-25T10:30:24.199728+0000","last_unstale":"2024-04-25T10:30:26.201390+0000","last_undegraded":"2024-04-25T10:30:26.201390+0000","last_fullsized":"2024-04-25T10:30:26.201390+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.213666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094435+0000","last_change":"2024-04-25T10:30:25.207621+0000","last_active":"2024-04-25T10:30:27.094435+0000","last_peered":"2024-04-25T10:30:27.094435+0000","last_clean":"2024-04-25T10:30:27.094435+0000","last_became_active":"2024-04-25T10:30:25.207315+0000","last_became_peered":"2024-04-25T10:30:25.207315+0000","last_unstale":"2024-04-25T10:30:27.094435+0000","last_undegraded":"2024-04-25T10:30:27.094435+0000","last_fullsized":"2024-04-25T10:30:27.094435+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:30.406873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"168'3","reported_seq":17,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097669+0000","last_change":"2024-04-25T10:30:24.207259+0000","last_active":"2024-04-25T10:30:27.097669+0000","last_peered":"2024-04-25T10:30:27.097669+0000","last_clean":"2024-04-25T10:30:27.097669+0000","last_became_active":"2024-04-25T10:30:24.206935+0000","last_became_peered":"2024-04-25T10:30:24.206935+0000","last_unstale":"2024-04-25T10:30:27.097669+0000","last_undegraded":"2024-04-25T10:30:27.097669+0000","last_fullsized":"2024-04-25T10:30:27.097669+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:58.111975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200906+0000","last_change":"2024-04-25T10:30:25.216648+0000","last_active":"2024-04-25T10:30:26.200906+0000","last_peered":"2024-04-25T10:30:26.200906+0000","last_clean":"2024-04-25T10:30:26.200906+0000","last_became_active":"2024-04-25T10:30:25.216384+0000","last_became_peered":"2024-04-25T10:30:25.216384+0000","last_unstale":"2024-04-25T10:30:26.200906+0000","last_undegraded":"2024-04-25T10:30:26.200906+0000","last_fullsized":"2024-04-25T10:30:26.200906+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.397559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"168'6","reported_seq":20,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097707+0000","last_change":"2024-04-25T10:30:24.206758+0000","last_active":"2024-04-25T10:30:27.097707+0000","last_peered":"2024-04-25T10:30:27.097707+0000","last_clean":"2024-04-25T10:30:27.097707+0000","last_became_active":"2024-04-25T10:30:24.206434+0000","last_became_peered":"2024-04-25T10:30:24.206434+0000","last_unstale":"2024-04-25T10:30:27.097707+0000","last_undegraded":"2024-04-25T10:30:27.097707+0000","last_fullsized":"2024-04-25T10:30:27.097707+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:23.317632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200534+0000","last_change":"2024-04-25T10:30:25.213206+0000","last_active":"2024-04-25T10:30:26.200534+0000","last_peered":"2024-04-25T10:30:26.200534+0000","last_clean":"2024-04-25T10:30:26.200534+0000","last_became_active":"2024-04-25T10:30:25.212904+0000","last_became_peered":"2024-04-25T10:30:25.212904+0000","last_unstale":"2024-04-25T10:30:26.200534+0000","last_undegraded":"2024-04-25T10:30:26.200534+0000","last_fullsized":"2024-04-25T10:30:26.200534+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:03.590877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094210+0000","last_change":"2024-04-25T10:30:24.197901+0000","last_active":"2024-04-25T10:30:27.094210+0000","last_peered":"2024-04-25T10:30:27.094210+0000","last_clean":"2024-04-25T10:30:27.094210+0000","last_became_active":"2024-04-25T10:30:24.197591+0000","last_became_peered":"2024-04-25T10:30:24.197591+0000","last_unstale":"2024-04-25T10:30:27.094210+0000","last_undegraded":"2024-04-25T10:30:27.094210+0000","last_fullsized":"2024-04-25T10:30:27.094210+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:23.317632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194313+0000","last_change":"2024-04-25T10:30:25.224849+0000","last_active":"2024-04-25T10:30:26.194313+0000","last_peered":"2024-04-25T10:30:26.194313+0000","last_clean":"2024-04-25T10:30:26.194313+0000","last_became_active":"2024-04-25T10:30:25.224322+0000","last_became_peered":"2024-04-25T10:30:25.224322+0000","last_unstale":"2024-04-25T10:30:26.194313+0000","last_undegraded":"2024-04-25T10:30:26.194313+0000","last_fullsized":"2024-04-25T10:30:26.194313+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:29.602109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200601+0000","last_change":"2024-04-25T10:30:24.201781+0000","last_active":"2024-04-25T10:30:26.200601+0000","last_peered":"2024-04-25T10:30:26.200601+0000","last_clean":"2024-04-25T10:30:26.200601+0000","last_became_active":"2024-04-25T10:30:24.201433+0000","last_became_peered":"2024-04-25T10:30:24.201433+0000","last_unstale":"2024-04-25T10:30:26.200601+0000","last_undegraded":"2024-04-25T10:30:26.200601+0000","last_fullsized":"2024-04-25T10:30:26.200601+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:21.759031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094040+0000","last_change":"2024-04-25T10:30:25.222706+0000","last_active":"2024-04-25T10:30:27.094040+0000","last_peered":"2024-04-25T10:30:27.094040+0000","last_clean":"2024-04-25T10:30:27.094040+0000","last_became_active":"2024-04-25T10:30:25.222395+0000","last_became_peered":"2024-04-25T10:30:25.222395+0000","last_unstale":"2024-04-25T10:30:27.094040+0000","last_undegraded":"2024-04-25T10:30:27.094040+0000","last_fullsized":"2024-04-25T10:30:27.094040+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:16.054666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097485+0000","last_change":"2024-04-25T10:30:24.200714+0000","last_active":"2024-04-25T10:30:27.097485+0000","last_peered":"2024-04-25T10:30:27.097485+0000","last_clean":"2024-04-25T10:30:27.097485+0000","last_became_active":"2024-04-25T10:30:24.200363+0000","last_became_peered":"2024-04-25T10:30:24.200363+0000","last_unstale":"2024-04-25T10:30:27.097485+0000","last_undegraded":"2024-04-25T10:30:27.097485+0000","last_fullsized":"2024-04-25T10:30:27.097485+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:41.408072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097131+0000","last_change":"2024-04-25T10:30:25.214376+0000","last_active":"2024-04-25T10:30:27.097131+0000","last_peered":"2024-04-25T10:30:27.097131+0000","last_clean":"2024-04-25T10:30:27.097131+0000","last_became_active":"2024-04-25T10:30:25.212888+0000","last_became_peered":"2024-04-25T10:30:25.212888+0000","last_unstale":"2024-04-25T10:30:27.097131+0000","last_undegraded":"2024-04-25T10:30:27.097131+0000","last_fullsized":"2024-04-25T10:30:27.097131+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:16.659507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"168'8","reported_seq":22,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194171+0000","last_change":"2024-04-25T10:30:24.194903+0000","last_active":"2024-04-25T10:30:26.194171+0000","last_peered":"2024-04-25T10:30:26.194171+0000","last_clean":"2024-04-25T10:30:26.194171+0000","last_became_active":"2024-04-25T10:30:24.194639+0000","last_became_peered":"2024-04-25T10:30:24.194639+0000","last_unstale":"2024-04-25T10:30:26.194171+0000","last_undegraded":"2024-04-25T10:30:26.194171+0000","last_fullsized":"2024-04-25T10:30:26.194171+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:19.535485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201484+0000","last_change":"2024-04-25T10:30:25.221618+0000","last_active":"2024-04-25T10:30:26.201484+0000","last_peered":"2024-04-25T10:30:26.201484+0000","last_clean":"2024-04-25T10:30:26.201484+0000","last_became_active":"2024-04-25T10:30:25.221323+0000","last_became_peered":"2024-04-25T10:30:25.221323+0000","last_unstale":"2024-04-25T10:30:26.201484+0000","last_undegraded":"2024-04-25T10:30:26.201484+0000","last_fullsized":"2024-04-25T10:30:26.201484+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:40.107598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097661+0000","last_change":"2024-04-25T10:30:24.205238+0000","last_active":"2024-04-25T10:30:27.097661+0000","last_peered":"2024-04-25T10:30:27.097661+0000","last_clean":"2024-04-25T10:30:27.097661+0000","last_became_active":"2024-04-25T10:30:24.204933+0000","last_became_peered":"2024-04-25T10:30:24.204933+0000","last_unstale":"2024-04-25T10:30:27.097661+0000","last_undegraded":"2024-04-25T10:30:27.097661+0000","last_fullsized":"2024-04-25T10:30:27.097661+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:31.725779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094653+0000","last_change":"2024-04-25T10:30:25.221109+0000","last_active":"2024-04-25T10:30:27.094653+0000","last_peered":"2024-04-25T10:30:27.094653+0000","last_clean":"2024-04-25T10:30:27.094653+0000","last_became_active":"2024-04-25T10:30:25.220818+0000","last_became_peered":"2024-04-25T10:30:25.220818+0000","last_unstale":"2024-04-25T10:30:27.094653+0000","last_undegraded":"2024-04-25T10:30:27.094653+0000","last_fullsized":"2024-04-25T10:30:27.094653+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.828621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097207+0000","last_change":"2024-04-25T10:30:24.202700+0000","last_active":"2024-04-25T10:30:27.097207+0000","last_peered":"2024-04-25T10:30:27.097207+0000","last_clean":"2024-04-25T10:30:27.097207+0000","last_became_active":"2024-04-25T10:30:24.202342+0000","last_became_peered":"2024-04-25T10:30:24.202342+0000","last_unstale":"2024-04-25T10:30:27.097207+0000","last_undegraded":"2024-04-25T10:30:27.097207+0000","last_fullsized":"2024-04-25T10:30:27.097207+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:59.000941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021852+0000","last_change":"2024-04-25T10:30:25.215252+0000","last_active":"2024-04-25T10:30:27.021852+0000","last_peered":"2024-04-25T10:30:27.021852+0000","last_clean":"2024-04-25T10:30:27.021852+0000","last_became_active":"2024-04-25T10:30:25.214572+0000","last_became_peered":"2024-04-25T10:30:25.214572+0000","last_unstale":"2024-04-25T10:30:27.021852+0000","last_undegraded":"2024-04-25T10:30:27.021852+0000","last_fullsized":"2024-04-25T10:30:27.021852+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.841741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"168'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234379+0000","last_change":"2024-04-25T10:30:24.196772+0000","last_active":"2024-04-25T10:30:26.234379+0000","last_peered":"2024-04-25T10:30:26.234379+0000","last_clean":"2024-04-25T10:30:26.234379+0000","last_became_active":"2024-04-25T10:30:24.196508+0000","last_became_peered":"2024-04-25T10:30:24.196508+0000","last_unstale":"2024-04-25T10:30:26.234379+0000","last_undegraded":"2024-04-25T10:30:26.234379+0000","last_fullsized":"2024-04-25T10:30:26.234379+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:33.469263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094385+0000","last_change":"2024-04-25T10:30:25.211205+0000","last_active":"2024-04-25T10:30:27.094385+0000","last_peered":"2024-04-25T10:30:27.094385+0000","last_clean":"2024-04-25T10:30:27.094385+0000","last_became_active":"2024-04-25T10:30:25.210920+0000","last_became_peered":"2024-04-25T10:30:25.210920+0000","last_unstale":"2024-04-25T10:30:27.094385+0000","last_undegraded":"2024-04-25T10:30:27.094385+0000","last_fullsized":"2024-04-25T10:30:27.094385+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:13.160678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234468+0000","last_change":"2024-04-25T10:30:24.199627+0000","last_active":"2024-04-25T10:30:26.234468+0000","last_peered":"2024-04-25T10:30:26.234468+0000","last_clean":"2024-04-25T10:30:26.234468+0000","last_became_active":"2024-04-25T10:30:24.199337+0000","last_became_peered":"2024-04-25T10:30:24.199337+0000","last_unstale":"2024-04-25T10:30:26.234468+0000","last_undegraded":"2024-04-25T10:30:26.234468+0000","last_fullsized":"2024-04-25T10:30:26.234468+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.574335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097073+0000","last_change":"2024-04-25T10:30:25.219214+0000","last_active":"2024-04-25T10:30:27.097073+0000","last_peered":"2024-04-25T10:30:27.097073+0000","last_clean":"2024-04-25T10:30:27.097073+0000","last_became_active":"2024-04-25T10:30:25.218700+0000","last_became_peered":"2024-04-25T10:30:25.218700+0000","last_unstale":"2024-04-25T10:30:27.097073+0000","last_undegraded":"2024-04-25T10:30:27.097073+0000","last_fullsized":"2024-04-25T10:30:27.097073+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:23.227421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097561+0000","last_change":"2024-04-25T10:30:24.207079+0000","last_active":"2024-04-25T10:30:27.097561+0000","last_peered":"2024-04-25T10:30:27.097561+0000","last_clean":"2024-04-25T10:30:27.097561+0000","last_became_active":"2024-04-25T10:30:24.206725+0000","last_became_peered":"2024-04-25T10:30:24.206725+0000","last_unstale":"2024-04-25T10:30:27.097561+0000","last_undegraded":"2024-04-25T10:30:27.097561+0000","last_fullsized":"2024-04-25T10:30:27.097561+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:45.311764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.194459+0000","last_change":"2024-04-25T10:30:25.212733+0000","last_active":"2024-04-25T10:30:26.194459+0000","last_peered":"2024-04-25T10:30:26.194459+0000","last_clean":"2024-04-25T10:30:26.194459+0000","last_became_active":"2024-04-25T10:30:25.212459+0000","last_became_peered":"2024-04-25T10:30:25.212459+0000","last_unstale":"2024-04-25T10:30:26.194459+0000","last_undegraded":"2024-04-25T10:30:26.194459+0000","last_fullsized":"2024-04-25T10:30:26.194459+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:44.555697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097889+0000","last_change":"2024-04-25T10:30:24.194286+0000","last_active":"2024-04-25T10:30:27.097889+0000","last_peered":"2024-04-25T10:30:27.097889+0000","last_clean":"2024-04-25T10:30:27.097889+0000","last_became_active":"2024-04-25T10:30:24.194001+0000","last_became_peered":"2024-04-25T10:30:24.194001+0000","last_unstale":"2024-04-25T10:30:27.097889+0000","last_undegraded":"2024-04-25T10:30:27.097889+0000","last_fullsized":"2024-04-25T10:30:27.097889+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.946597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234262+0000","last_change":"2024-04-25T10:30:25.213983+0000","last_active":"2024-04-25T10:30:26.234262+0000","last_peered":"2024-04-25T10:30:26.234262+0000","last_clean":"2024-04-25T10:30:26.234262+0000","last_became_active":"2024-04-25T10:30:25.213712+0000","last_became_peered":"2024-04-25T10:30:25.213712+0000","last_unstale":"2024-04-25T10:30:26.234262+0000","last_undegraded":"2024-04-25T10:30:26.234262+0000","last_fullsized":"2024-04-25T10:30:26.234262+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:34.009571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096706+0000","last_change":"2024-04-25T10:30:24.201413+0000","last_active":"2024-04-25T10:30:27.096706+0000","last_peered":"2024-04-25T10:30:27.096706+0000","last_clean":"2024-04-25T10:30:27.096706+0000","last_became_active":"2024-04-25T10:30:24.201156+0000","last_became_peered":"2024-04-25T10:30:24.201156+0000","last_unstale":"2024-04-25T10:30:27.096706+0000","last_undegraded":"2024-04-25T10:30:27.096706+0000","last_fullsized":"2024-04-25T10:30:27.096706+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:15.045266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.199974+0000","last_change":"2024-04-25T10:30:25.209589+0000","last_active":"2024-04-25T10:30:26.199974+0000","last_peered":"2024-04-25T10:30:26.199974+0000","last_clean":"2024-04-25T10:30:26.199974+0000","last_became_active":"2024-04-25T10:30:25.209126+0000","last_became_peered":"2024-04-25T10:30:25.209126+0000","last_unstale":"2024-04-25T10:30:26.199974+0000","last_undegraded":"2024-04-25T10:30:26.199974+0000","last_fullsized":"2024-04-25T10:30:26.199974+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:40.152211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234130+0000","last_change":"2024-04-25T10:30:24.204401+0000","last_active":"2024-04-25T10:30:26.234130+0000","last_peered":"2024-04-25T10:30:26.234130+0000","last_clean":"2024-04-25T10:30:26.234130+0000","last_became_active":"2024-04-25T10:30:24.202760+0000","last_became_peered":"2024-04-25T10:30:24.202760+0000","last_unstale":"2024-04-25T10:30:26.234130+0000","last_undegraded":"2024-04-25T10:30:26.234130+0000","last_fullsized":"2024-04-25T10:30:26.234130+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:35.052141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201113+0000","last_change":"2024-04-25T10:30:25.209784+0000","last_active":"2024-04-25T10:30:26.201113+0000","last_peered":"2024-04-25T10:30:26.201113+0000","last_clean":"2024-04-25T10:30:26.201113+0000","last_became_active":"2024-04-25T10:30:25.209504+0000","last_became_peered":"2024-04-25T10:30:25.209504+0000","last_unstale":"2024-04-25T10:30:26.201113+0000","last_undegraded":"2024-04-25T10:30:26.201113+0000","last_fullsized":"2024-04-25T10:30:26.201113+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.447201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"168'2","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234620+0000","last_change":"2024-04-25T10:30:24.199165+0000","last_active":"2024-04-25T10:30:26.234620+0000","last_peered":"2024-04-25T10:30:26.234620+0000","last_clean":"2024-04-25T10:30:26.234620+0000","last_became_active":"2024-04-25T10:30:24.198889+0000","last_became_peered":"2024-04-25T10:30:24.198889+0000","last_unstale":"2024-04-25T10:30:26.234620+0000","last_undegraded":"2024-04-25T10:30:26.234620+0000","last_fullsized":"2024-04-25T10:30:26.234620+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:09.752276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201150+0000","last_change":"2024-04-25T10:30:25.214734+0000","last_active":"2024-04-25T10:30:26.201150+0000","last_peered":"2024-04-25T10:30:26.201150+0000","last_clean":"2024-04-25T10:30:26.201150+0000","last_became_active":"2024-04-25T10:30:25.214425+0000","last_became_peered":"2024-04-25T10:30:25.214425+0000","last_unstale":"2024-04-25T10:30:26.201150+0000","last_undegraded":"2024-04-25T10:30:26.201150+0000","last_fullsized":"2024-04-25T10:30:26.201150+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:00.456831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"169'1","reported_seq":15,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.201222+0000","last_change":"2024-04-25T10:30:24.200959+0000","last_active":"2024-04-25T10:30:26.201222+0000","last_peered":"2024-04-25T10:30:26.201222+0000","last_clean":"2024-04-25T10:30:26.201222+0000","last_became_active":"2024-04-25T10:30:24.200611+0000","last_became_peered":"2024-04-25T10:30:24.200611+0000","last_unstale":"2024-04-25T10:30:26.201222+0000","last_undegraded":"2024-04-25T10:30:26.201222+0000","last_fullsized":"2024-04-25T10:30:26.201222+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:50.999131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200286+0000","last_change":"2024-04-25T10:30:25.225951+0000","last_active":"2024-04-25T10:30:26.200286+0000","last_peered":"2024-04-25T10:30:26.200286+0000","last_clean":"2024-04-25T10:30:26.200286+0000","last_became_active":"2024-04-25T10:30:25.225614+0000","last_became_peered":"2024-04-25T10:30:25.225614+0000","last_unstale":"2024-04-25T10:30:26.200286+0000","last_undegraded":"2024-04-25T10:30:26.200286+0000","last_fullsized":"2024-04-25T10:30:26.200286+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.544583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.200943+0000","last_change":"2024-04-25T10:30:24.201053+0000","last_active":"2024-04-25T10:30:26.200943+0000","last_peered":"2024-04-25T10:30:26.200943+0000","last_clean":"2024-04-25T10:30:26.200943+0000","last_became_active":"2024-04-25T10:30:24.200242+0000","last_became_peered":"2024-04-25T10:30:24.200242+0000","last_unstale":"2024-04-25T10:30:26.200943+0000","last_undegraded":"2024-04-25T10:30:26.200943+0000","last_fullsized":"2024-04-25T10:30:26.200943+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:15.043592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097604+0000","last_change":"2024-04-25T10:30:25.222705+0000","last_active":"2024-04-25T10:30:27.097604+0000","last_peered":"2024-04-25T10:30:27.097604+0000","last_clean":"2024-04-25T10:30:27.097604+0000","last_became_active":"2024-04-25T10:30:25.222423+0000","last_became_peered":"2024-04-25T10:30:25.222423+0000","last_unstale":"2024-04-25T10:30:27.097604+0000","last_undegraded":"2024-04-25T10:30:27.097604+0000","last_fullsized":"2024-04-25T10:30:27.097604+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.397559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097805+0000","last_change":"2024-04-25T10:30:24.205709+0000","last_active":"2024-04-25T10:30:27.097805+0000","last_peered":"2024-04-25T10:30:27.097805+0000","last_clean":"2024-04-25T10:30:27.097805+0000","last_became_active":"2024-04-25T10:30:24.205428+0000","last_became_peered":"2024-04-25T10:30:24.205428+0000","last_unstale":"2024-04-25T10:30:27.097805+0000","last_undegraded":"2024-04-25T10:30:27.097805+0000","last_fullsized":"2024-04-25T10:30:27.097805+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:51.116449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021806+0000","last_change":"2024-04-25T10:30:25.213043+0000","last_active":"2024-04-25T10:30:27.021806+0000","last_peered":"2024-04-25T10:30:27.021806+0000","last_clean":"2024-04-25T10:30:27.021806+0000","last_became_active":"2024-04-25T10:30:25.212500+0000","last_became_peered":"2024-04-25T10:30:25.212500+0000","last_unstale":"2024-04-25T10:30:27.021806+0000","last_undegraded":"2024-04-25T10:30:27.021806+0000","last_fullsized":"2024-04-25T10:30:27.021806+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.779557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021271+0000","last_change":"2024-04-25T10:30:24.202686+0000","last_active":"2024-04-25T10:30:27.021271+0000","last_peered":"2024-04-25T10:30:27.021271+0000","last_clean":"2024-04-25T10:30:27.021271+0000","last_became_active":"2024-04-25T10:30:24.201970+0000","last_became_peered":"2024-04-25T10:30:24.201970+0000","last_unstale":"2024-04-25T10:30:27.021271+0000","last_undegraded":"2024-04-25T10:30:27.021271+0000","last_fullsized":"2024-04-25T10:30:27.021271+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:34.948942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:26.234401+0000","last_change":"2024-04-25T10:30:25.208619+0000","last_active":"2024-04-25T10:30:26.234401+0000","last_peered":"2024-04-25T10:30:26.234401+0000","last_clean":"2024-04-25T10:30:26.234401+0000","last_became_active":"2024-04-25T10:30:25.208303+0000","last_became_peered":"2024-04-25T10:30:25.208303+0000","last_unstale":"2024-04-25T10:30:26.234401+0000","last_undegraded":"2024-04-25T10:30:26.234401+0000","last_fullsized":"2024-04-25T10:30:26.234401+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:46.810186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021353+0000","last_change":"2024-04-25T10:30:24.202866+0000","last_active":"2024-04-25T10:30:27.021353+0000","last_peered":"2024-04-25T10:30:27.021353+0000","last_clean":"2024-04-25T10:30:27.021353+0000","last_became_active":"2024-04-25T10:30:24.200965+0000","last_became_peered":"2024-04-25T10:30:24.200965+0000","last_unstale":"2024-04-25T10:30:27.021353+0000","last_undegraded":"2024-04-25T10:30:27.021353+0000","last_fullsized":"2024-04-25T10:30:27.021353+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:38.090617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021348+0000","last_change":"2024-04-25T10:30:25.211272+0000","last_active":"2024-04-25T10:30:27.021348+0000","last_peered":"2024-04-25T10:30:27.021348+0000","last_clean":"2024-04-25T10:30:27.021348+0000","last_became_active":"2024-04-25T10:30:25.210911+0000","last_became_peered":"2024-04-25T10:30:25.210911+0000","last_unstale":"2024-04-25T10:30:27.021348+0000","last_undegraded":"2024-04-25T10:30:27.021348+0000","last_fullsized":"2024-04-25T10:30:27.021348+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:21.065884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021199+0000","last_change":"2024-04-25T10:30:24.202691+0000","last_active":"2024-04-25T10:30:27.021199+0000","last_peered":"2024-04-25T10:30:27.021199+0000","last_clean":"2024-04-25T10:30:27.021199+0000","last_became_active":"2024-04-25T10:30:24.200994+0000","last_became_peered":"2024-04-25T10:30:24.200994+0000","last_unstale":"2024-04-25T10:30:27.021199+0000","last_undegraded":"2024-04-25T10:30:27.021199+0000","last_fullsized":"2024-04-25T10:30:27.021199+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.914386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.097319+0000","last_change":"2024-04-25T10:30:25.213017+0000","last_active":"2024-04-25T10:30:27.097319+0000","last_peered":"2024-04-25T10:30:27.097319+0000","last_clean":"2024-04-25T10:30:27.097319+0000","last_became_active":"2024-04-25T10:30:25.212668+0000","last_became_peered":"2024-04-25T10:30:25.212668+0000","last_unstale":"2024-04-25T10:30:27.097319+0000","last_undegraded":"2024-04-25T10:30:27.097319+0000","last_fullsized":"2024-04-25T10:30:27.097319+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:20.305758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096902+0000","last_change":"2024-04-25T10:30:24.193826+0000","last_active":"2024-04-25T10:30:27.096902+0000","last_peered":"2024-04-25T10:30:27.096902+0000","last_clean":"2024-04-25T10:30:27.096902+0000","last_became_active":"2024-04-25T10:30:24.193546+0000","last_became_peered":"2024-04-25T10:30:24.193546+0000","last_unstale":"2024-04-25T10:30:27.096902+0000","last_undegraded":"2024-04-25T10:30:27.096902+0000","last_fullsized":"2024-04-25T10:30:27.096902+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:23.111042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.096635+0000","last_change":"2024-04-25T10:30:25.217340+0000","last_active":"2024-04-25T10:30:27.096635+0000","last_peered":"2024-04-25T10:30:27.096635+0000","last_clean":"2024-04-25T10:30:27.096635+0000","last_became_active":"2024-04-25T10:30:25.217071+0000","last_became_peered":"2024-04-25T10:30:25.217071+0000","last_unstale":"2024-04-25T10:30:27.096635+0000","last_undegraded":"2024-04-25T10:30:27.096635+0000","last_fullsized":"2024-04-25T10:30:27.096635+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:24.173818+0000","last_clean_scrub_stamp":"2024-04-25T10:30:24.173818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:11.242436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.094066+0000","last_change":"2024-04-25T10:30:24.194259+0000","last_active":"2024-04-25T10:30:27.094066+0000","last_peered":"2024-04-25T10:30:27.094066+0000","last_clean":"2024-04-25T10:30:27.094066+0000","last_became_active":"2024-04-25T10:30:24.193996+0000","last_became_peered":"2024-04-25T10:30:24.193996+0000","last_unstale":"2024-04-25T10:30:27.094066+0000","last_undegraded":"2024-04-25T10:30:27.094066+0000","last_fullsized":"2024-04-25T10:30:27.094066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:30:23.162744+0000","last_clean_scrub_stamp":"2024-04-25T10:30:23.162744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.511957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":454,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-04-25T10:30:27.021210+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:30:27.021210+0000","last_peered":"2024-04-25T10:30:27.021210+0000","last_clean":"2024-04-25T10:30:27.021210+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:30:27.021210+0000","last_undegraded":"2024-04-25T10:30:27.021210+0000","last_fullsized":"2024-04-25T10:30:27.021210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55476,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316364,"statfs":{"total":96636764160,"available":96579956736,"internally_reserved":0,"allocated":2138112,"data_stored":930359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55468,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316372,"statfs":{"total":96636764160,"available":96579964928,"internally_reserved":0,"allocated":2121728,"data_stored":928725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60212,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311628,"statfs":{"total":96636764160,"available":96575107072,"internally_reserved":0,"allocated":2719744,"data_stored":1519607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47152,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324688,"statfs":{"total":96636764160,"available":96588480512,"internally_reserved":0,"allocated":2134016,"data_stored":930317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509595,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69288,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302552,"statfs":{"total":96636764160,"available":96565813248,"internally_reserved":0,"allocated":2121728,"data_stored":928705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47716,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324124,"statfs":{"total":96636764160,"available":96587902976,"internally_reserved":0,"allocated":2711552,"data_stored":1519003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55464,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316376,"statfs":{"total":96636764160,"available":96579969024,"internally_reserved":0,"allocated":2125824,"data_stored":929251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65124,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306716,"statfs":{"total":96636764160,"available":96570077184,"internally_reserved":0,"allocated":2121728,"data_stored":928669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:30:29.712 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:30:29.712 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:30:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.895+0000 7f3cd3e00640 1 Processor -- start 2024-04-25T10:30:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.895+0000 7f3cd3e00640 1 -- start start 2024-04-25T10:30:29.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3cd40d09c0 0x7f3cd40cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40b0de0 con 0x7f3cd415dd00 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd406b330 con 0x7f3cd406aea0 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd40cbd70 con 0x7f3cd40d09c0 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3cd40d09c0 0x7f3cd40cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3cd415dd00 0x7f3cd406b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33650/0 (socket says 172.21.15.45:33650) 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd2a00640 1 -- 172.21.15.45:0/197963656 learned_addr learned my addr 172.21.15.45:0/197963656 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3400640 1 -- 172.21.15.45:0/197963656 >> v1:172.21.15.117:6789/0 conn(0x7f3cd406aea0 legacy=0x7f3cd40ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3400640 1 -- 172.21.15.45:0/197963656 >> v1:172.21.15.45:6789/0 conn(0x7f3cd415dd00 legacy=0x7f3cd406b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3400640 1 -- 172.21.15.45:0/197963656 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40cbfc0 con 0x7f3cd40d09c0 2024-04-25T10:30:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.899+0000 7f3cd3400640 1 --2- 172.21.15.45:0/197963656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd40d09c0 0x7f3cd40cd1b0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009d90 tx=0x7f3ccc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99968c46ef4f4a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd2000640 1 -- 172.21.15.45:0/197963656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0b4070 con 0x7f3cd40d09c0 2024-04-25T10:30:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd2000640 1 -- 172.21.15.45:0/197963656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc0af600 con 0x7f3cd40d09c0 2024-04-25T10:30:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd2000640 1 -- 172.21.15.45:0/197963656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0af9d0 con 0x7f3cd40d09c0 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- 172.21.15.45:0/197963656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd40d09c0 msgr2=0x7f3cd40cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/197963656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd40d09c0 0x7f3cd40cd1b0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009d90 tx=0x7f3ccc01f150 comp rx=0 tx=0).stop 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- 172.21.15.45:0/197963656 shutdown_connections 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/197963656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd40d09c0 0x7f3cd40cd1b0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- 172.21.15.45:0/197963656 >> 172.21.15.45:0/197963656 conn(0x7f3cd40d1720 msgr2=0x7f3cd40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- 172.21.15.45:0/197963656 shutdown_connections 2024-04-25T10:30:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- 172.21.15.45:0/197963656 wait complete. 2024-04-25T10:30:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 Processor -- start 2024-04-25T10:30:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- start start 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd40d09c0 0x7f3cd40b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 0x7f3cd40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40b5400 0x7f3cd40ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd415e0e0 con 0x7f3cd415dd00 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40cc580 con 0x7f3cd40b5400 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3cd3e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd406bd20 con 0x7f3cd40d09c0 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 0x7f3cd40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 0x7f3cd40b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55780/0 (socket says 172.21.15.45:55780) 2024-04-25T10:30:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.903+0000 7f3ccbe00640 1 -- 172.21.15.45:0/589138150 learned_addr learned my addr 172.21.15.45:0/589138150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd2a00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd40d09c0 0x7f3cd40b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccbe00640 1 -- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd40d09c0 msgr2=0x7f3cd40b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccbe00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd40d09c0 0x7f3cd40b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd3400640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40b5400 0x7f3cd40ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccbe00640 1 -- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40b5400 msgr2=0x7f3cd40ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccbe00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40b5400 0x7f3cd40ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccbe00640 1 -- 172.21.15.45:0/589138150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd415e400 con 0x7f3cd415dd00 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd2a00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd40d09c0 0x7f3cd40b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd3400640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40b5400 0x7f3cd40ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccbe00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 0x7f3cd40b4ec0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c420 tx=0x7f3cbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc0052d0 con 0x7f3cd415dd00 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc005470 con 0x7f3cd415dd00 2024-04-25T10:30:29.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc007260 con 0x7f3cd415dd00 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd415e6e0 con 0x7f3cd415dd00 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd40b73c0 con 0x7f3cd415dd00 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd4075120 con 0x7f3cd415dd00 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cbc007400 con 0x7f3cd415dd00 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccb400640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3cb0066f60 0x7f3cb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cbc0845d0 con 0x7f3cd415dd00 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.907+0000 7f3cd2a00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3cb0066f60 0x7f3cb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.911+0000 7f3cd2a00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3cb0066f60 0x7f3cb0069420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4005890 tx=0x7f3cc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:29.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:29.911+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3cbc055540 con 0x7f3cd415dd00 2024-04-25T10:30:30.063 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:30.067 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:30:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.135+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3cd40b2be0 con 0x7f3cd415dd00 2024-04-25T10:30:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.135+0000 7f3ccb400640 1 -- 172.21.15.45:0/589138150 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3cbc054e40 con 0x7f3cd415dd00 2024-04-25T10:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:30:30.139 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:30:30.140 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:30:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3cb0066f60 msgr2=0x7f3cb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3cb0066f60 0x7f3cb0069420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4005890 tx=0x7f3cc4009040 comp rx=0 tx=0).stop 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 msgr2=0x7f3cd40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 0x7f3cd40b4ec0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c420 tx=0x7f3cbc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 shutdown_connections 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3cb0066f60 0x7f3cb0069420 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cd40b5400 0x7f3cd40ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cd415dd00 0x7f3cd40b4ec0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 --2- 172.21.15.45:0/589138150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3cd40d09c0 0x7f3cd40b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 >> 172.21.15.45:0/589138150 conn(0x7f3cd40d1720 msgr2=0x7f3cd40d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 shutdown_connections 2024-04-25T10:30:30.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.139+0000 7f3cd3e00640 1 -- 172.21.15.45:0/589138150 wait complete. 2024-04-25T10:30:30.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:30.164 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:30:30.163+0000 7f8f4e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:30:30.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 Processor -- start 2024-04-25T10:30:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 -- start start 2024-04-25T10:30:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54ec15b9b0 0x7f54ec0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54ec0b5fb0 0x7f54ec0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec0cd120 con 0x7f54ec0ce700 2024-04-25T10:30:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0d2600 con 0x7f54ec15b9b0 2024-04-25T10:30:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ebe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0cd400 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eaa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54ec15b9b0 0x7f54ec0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54e2000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f54ec0ce700 0x7f54ec0cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33652/0 (socket says 172.21.15.45:33652) 2024-04-25T10:30:30.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54e2000640 1 -- 172.21.15.45:0/1164851080 learned_addr learned my addr 172.21.15.45:0/1164851080 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eaa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f54ec15b9b0 0x7f54ec0ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55796/0 (socket says 172.21.15.45:55796) 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eb400640 1 --2- 172.21.15.45:0/1164851080 >> v2:172.21.15.117:3300/0 conn(0x7f54ec0b5fb0 0x7f54ec0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eaa00640 1 -- 172.21.15.45:0/1164851080 >> v2:172.21.15.117:3300/0 conn(0x7f54ec0b5fb0 msgr2=0x7f54ec0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eaa00640 1 --2- 172.21.15.45:0/1164851080 >> v2:172.21.15.117:3300/0 conn(0x7f54ec0b5fb0 0x7f54ec0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eaa00640 1 -- 172.21.15.45:0/1164851080 >> v1:172.21.15.45:6789/0 conn(0x7f54ec0ce700 legacy=0x7f54ec0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54eaa00640 1 -- 172.21.15.45:0/1164851080 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0b47a0 con 0x7f54ec15b9b0 2024-04-25T10:30:30.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.347+0000 7f54ea000640 1 -- 172.21.15.45:0/1164851080 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941460242 0 0) 0x7f54ec0cd120 con 0x7f54ec0ce700 2024-04-25T10:30:30.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54eaa00640 1 --2- 172.21.15.45:0/1164851080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec15b9b0 0x7f54ec0ce1c0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0cbd90 tx=0x7f54dc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2ea94adc2006de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ea000640 1 -- 172.21.15.45:0/1164851080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0b0070 con 0x7f54ec15b9b0 2024-04-25T10:30:30.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ea000640 1 -- 172.21.15.45:0/1164851080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc0ab600 con 0x7f54ec15b9b0 2024-04-25T10:30:30.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ea000640 1 -- 172.21.15.45:0/1164851080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0ab9d0 con 0x7f54ec15b9b0 2024-04-25T10:30:30.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 -- 172.21.15.45:0/1164851080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec15b9b0 msgr2=0x7f54ec0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 --2- 172.21.15.45:0/1164851080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec15b9b0 0x7f54ec0ce1c0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0cbd90 tx=0x7f54dc01c480 comp rx=0 tx=0).stop 2024-04-25T10:30:30.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 -- 172.21.15.45:0/1164851080 shutdown_connections 2024-04-25T10:30:30.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 --2- 172.21.15.45:0/1164851080 >> v2:172.21.15.117:3300/0 conn(0x7f54ec0b5fb0 0x7f54ec0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 --2- 172.21.15.45:0/1164851080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec15b9b0 0x7f54ec0ce1c0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 -- 172.21.15.45:0/1164851080 >> 172.21.15.45:0/1164851080 conn(0x7f54ec0d17b0 msgr2=0x7f54ec0d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 -- 172.21.15.45:0/1164851080 shutdown_connections 2024-04-25T10:30:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 -- 172.21.15.45:0/1164851080 wait complete. 2024-04-25T10:30:30.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 Processor -- start 2024-04-25T10:30:30.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.351+0000 7f54ebe00640 1 -- start start 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 0x7f54ec14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec0ce700 0x7f54ec14ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b9b0 0x7f54ec157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec168760 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1685e0 con 0x7f54ec15b9b0 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec1688e0 con 0x7f54ec0ce700 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 0x7f54ec14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 0x7f54ec14ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55804/0 (socket says 172.21.15.45:55804) 2024-04-25T10:30:30.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 -- 172.21.15.45:0/121502928 learned_addr learned my addr 172.21.15.45:0/121502928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eb400640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b9b0 0x7f54ec157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54e2000640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec0ce700 0x7f54ec14ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 -- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec0ce700 msgr2=0x7f54ec14ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec0ce700 0x7f54ec14ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 -- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b9b0 msgr2=0x7f54ec157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b9b0 0x7f54ec157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 -- 172.21.15.45:0/121502928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec14c910 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54eaa00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 0x7f54ec14ebf0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0ae040 tx=0x7f54dc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0bc070 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc0b1040 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0b0050 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec14b590 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec14bb50 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec075120 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.355+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54dc0bf070 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.359+0000 7f54e3e00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54c8066f60 0x7f54c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:30.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.359+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54dc11eca0 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.359+0000 7f54e2000640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54c8066f60 0x7f54c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:30.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.359+0000 7f54e2000640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54c8066f60 0x7f54c8069420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f54ec159320 tx=0x7f54d4007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:30.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.363+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54ec14c910 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.555+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54ec0b3bb0 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.555+0000 7f54e3e00640 1 -- 172.21.15.45:0/121502928 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7f54dc016070 con 0x7f54ec0b5fb0 2024-04-25T10:30:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:30.558 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:30:30.558 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:30:30.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.555+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54c8066f60 msgr2=0x7f54c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.555+0000 7f54ebe00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54c8066f60 0x7f54c8069420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f54ec159320 tx=0x7f54d4007b60 comp rx=0 tx=0).stop 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 msgr2=0x7f54ec14ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 0x7f54ec14ebf0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0ae040 tx=0x7f54dc01acc0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 shutdown_connections 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f54c8066f60 0x7f54c8069420 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54ec15b9b0 0x7f54ec157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54ec0ce700 0x7f54ec14ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 --2- 172.21.15.45:0/121502928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54ec0b5fb0 0x7f54ec14ebf0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 >> 172.21.15.45:0/121502928 conn(0x7f54ec0d17b0 msgr2=0x7f54ec0cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 shutdown_connections 2024-04-25T10:30:30.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:30.559+0000 7f54ebe00640 1 -- 172.21.15.45:0/121502928 wait complete. 2024-04-25T10:30:30.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 122, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:30:24.574396+0000', 'modified': '2024-04-25T10:30:27.602192+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10467': {'gid': 10467, 'name': 'd', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6837/1582038965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6835', 'nonce': 1582038965}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1582038965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:30:30.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:30:30.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:30:31.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.123+0000 7f0197200640 1 Processor -- start 2024-04-25T10:30:31.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.123+0000 7f0197200640 1 -- start start 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0197200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900b0f70 0x7f01900b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0197200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01900cd2e0 0x7f01900cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0197200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019015f3b0 con 0x7f01900cfd90 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0197200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019015ef20 con 0x7f01900cd2e0 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0197200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900ca810 con 0x7f01900b0f70 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0187400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f01900cfd90 0x7f01900ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33666/0 (socket says 172.21.15.45:33666) 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0187400640 1 -- 172.21.15.45:0/1544732156 learned_addr learned my addr 172.21.15.45:0/1544732156 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0195200640 1 --2- 172.21.15.45:0/1544732156 >> v2:172.21.15.117:3300/0 conn(0x7f01900cd2e0 0x7f01900cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f018fe00640 1 --2- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900b0f70 0x7f01900b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f018f400640 1 -- 172.21.15.45:0/1544732156 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946378985 0 0) 0x7f019015f3b0 con 0x7f01900cfd90 2024-04-25T10:30:31.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f018f400640 1 -- 172.21.15.45:0/1544732156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0174003530 con 0x7f01900cfd90 2024-04-25T10:30:31.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0195200640 1 -- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900b0f70 msgr2=0x7f01900b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0195200640 1 --2- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900b0f70 0x7f01900b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0195200640 1 -- 172.21.15.45:0/1544732156 >> v1:172.21.15.45:6789/0 conn(0x7f01900cfd90 legacy=0x7f01900ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.127+0000 7f0195200640 1 -- 172.21.15.45:0/1544732156 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900b1890 con 0x7f01900cd2e0 2024-04-25T10:30:31.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0195200640 1 --2- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900cd2e0 0x7f01900cba20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0188009d90 tx=0x7f018801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be0a877c526db68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f018f400640 1 -- 172.21.15.45:0/1544732156 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880b2070 con 0x7f01900cd2e0 2024-04-25T10:30:31.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f018f400640 1 -- 172.21.15.45:0/1544732156 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880a5830 con 0x7f01900cd2e0 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f018f400640 1 -- 172.21.15.45:0/1544732156 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880a5c00 con 0x7f01900cd2e0 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 -- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900cd2e0 msgr2=0x7f01900cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 --2- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900cd2e0 0x7f01900cba20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0188009d90 tx=0x7f018801c970 comp rx=0 tx=0).stop 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 -- 172.21.15.45:0/1544732156 shutdown_connections 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 --2- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900cd2e0 0x7f01900cba20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 --2- 172.21.15.45:0/1544732156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900b0f70 0x7f01900b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 -- 172.21.15.45:0/1544732156 >> 172.21.15.45:0/1544732156 conn(0x7f01900b1d50 msgr2=0x7f01900b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 -- 172.21.15.45:0/1544732156 shutdown_connections 2024-04-25T10:30:31.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 -- 172.21.15.45:0/1544732156 wait complete. 2024-04-25T10:30:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 Processor -- start 2024-04-25T10:30:31.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.131+0000 7f0197200640 1 -- start start 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900b0f70 0x7f0190157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 0x7f01900b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900cfd90 0x7f01900c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190161b50 con 0x7f01900cd2e0 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01901619d0 con 0x7f01900b0f70 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0190161cd0 con 0x7f01900cfd90 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 0x7f01900b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 0x7f01900b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55820/0 (socket says 172.21.15.45:55820) 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 -- 172.21.15.45:0/2637817615 learned_addr learned my addr 172.21.15.45:0/2637817615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f018fe00640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900b0f70 0x7f0190157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0195200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900cfd90 0x7f01900c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 -- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900cfd90 msgr2=0x7f01900c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900cfd90 0x7f01900c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 -- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900b0f70 msgr2=0x7f0190157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900b0f70 0x7f0190157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 -- 172.21.15.45:0/2637817615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900c6160 con 0x7f01900cd2e0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0187400640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 0x7f01900b5460 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f017800c3c0 tx=0x7f017800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f018cc00640 1 -- 172.21.15.45:0/2637817615 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178017070 con 0x7f01900cd2e0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f018cc00640 1 -- 172.21.15.45:0/2637817615 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0178005d10 con 0x7f01900cd2e0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f018cc00640 1 -- 172.21.15.45:0/2637817615 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178007310 con 0x7f01900cd2e0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01900c6440 con 0x7f01900cd2e0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01900b59a0 con 0x7f01900cd2e0 2024-04-25T10:30:31.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.135+0000 7f018cc00640 1 -- 172.21.15.45:0/2637817615 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f017801a070 con 0x7f01900cd2e0 2024-04-25T10:30:31.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f018cc00640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f016c066f60 0x7f016c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f018cc00640 1 -- 172.21.15.45:0/2637817615 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0178084470 con 0x7f01900cd2e0 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f018fe00640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f016c066f60 0x7f016c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 Processor -- start 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 -- start start 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f018fe00640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f016c066f60 0x7f016c069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0180009af0 tx=0x7f0180008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc4920 0x55cd9acc4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc5240 0x55cd9ac7a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9ab625f0 con 0x55cd9ac7af30 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aac9da0 con 0x55cd9acc5240 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aa52150 con 0x55cd9acc4920 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55828/0 (socket says 172.21.15.45:55828) 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 -- 172.21.15.45:0/50583243 learned_addr learned my addr 172.21.15.45:0/50583243 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f018fe00640 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc4920 0x55cd9acc4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0187400640 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc5240 0x55cd9ac7a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 -- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc4920 msgr2=0x55cd9acc4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc4920 0x55cd9acc4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 -- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc5240 msgr2=0x55cd9ac7a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc5240 0x55cd9ac7a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 -- 172.21.15.45:0/50583243 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9aa6bca0 con 0x55cd9ac7af30 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0187400640 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc5240 0x55cd9ac7a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:31.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0195200640 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7b310 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f018801b020 tx=0x7f0188004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ed0241c0aac225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0184c00640 1 -- 172.21.15.45:0/50583243 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880b2040 con 0x55cd9ac7af30 2024-04-25T10:30:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0184c00640 1 -- 172.21.15.45:0/50583243 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880a5830 con 0x55cd9ac7af30 2024-04-25T10:30:31.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.139+0000 7f0184c00640 1 -- 172.21.15.45:0/50583243 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880a5c20 con 0x55cd9ac7af30 2024-04-25T10:30:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/50583243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 msgr2=0x55cd9ac7b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7b310 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f018801b020 tx=0x7f0188004ac0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/50583243 shutdown_connections 2024-04-25T10:30:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7b310 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc5240 0x55cd9ac7a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- 172.21.15.45:0/50583243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc4920 0x55cd9acc4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/50583243 >> 172.21.15.45:0/50583243 conn(0x55cd9ac812a0 msgr2=0x55cd9ac792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/50583243 shutdown_connections 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/50583243 wait complete. 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- start start 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 Processor -- start 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9ac7af30 0x55cd9accbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9accc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 0x55cd9ac7eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aa84990 con 0x55cd9acc5240 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aac5e40 con 0x55cd9acc4920 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aa55900 con 0x55cd9ac7af30 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 10 client.0 _ll_register_callbacks cb 0x55cd9ab7d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 0x55cd9ac7eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 0x55cd9ac7eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55830/0 (socket says 172.21.15.45:55830) 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 -- 172.21.15.45:0/147384899 learned_addr learned my addr 172.21.15.45:0/147384899 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f018fe00640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9ac7af30 0x55cd9accbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0187400640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9accc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 -- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9ac7af30 msgr2=0x55cd9accbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9ac7af30 0x55cd9accbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 -- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 msgr2=0x55cd9accc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9accc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 -- 172.21.15.45:0/147384899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9aa82630 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0187400640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9accc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f0195200640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 0x55cd9ac7eaa0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f018801c9b0 tx=0x7f018801b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca40e041ad6fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f017d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f017d600640 1 -- 172.21.15.45:0/147384899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880c6030 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01880a5830 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f017d600640 1 -- 172.21.15.45:0/147384899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01880a5630 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f017d600640 1 -- 172.21.15.45:0/147384899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01880d2050 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 10 client.10593 fetch_fsmap learned FSMap version 122 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x55cd9aa523a0 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.143+0000 7f017d600640 1 -- 172.21.15.45:0/147384899 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880a5830 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f017d600640 1 -- 172.21.15.45:0/147384899 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01880d5020 con 0x55cd9acc5240 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 10 client.10593 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:30:31.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 10 client.10593 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 20 client.10593 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ac85af0 0x55cd9ac7f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 4 client.10593 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd9aa65a50 con 0x55cd9ac85af0 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f018fe00640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ac85af0 0x55cd9ac7f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f018fe00640 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ac85af0 0x55cd9ac7f4b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb99cc85ce24ca0 server_cookie=6721c704189d0a3 in_seq=0 out_seq=0 2024-04-25T10:30:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f017d600640 10 client.10593 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f017d600640 1 -- 172.21.15.45:0/147384899 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd9aa65a50 con 0x55cd9ac85af0 2024-04-25T10:30:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f017d600640 10 client.10593 handle_command_reply: tid=0 2024-04-25T10:30:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 client.10593 shutdown 2024-04-25T10:30:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 msgr2=0x55cd9ac7eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 0x55cd9ac7eaa0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f018801c9b0 tx=0x7f018801b1b0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 shutdown_connections 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ac85af0 0x55cd9ac7f4b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc5240 0x55cd9ac7eaa0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9accc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 --2- 172.21.15.45:0/147384899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9ac7af30 0x55cd9accbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f017d600640 0 client.10593 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 >> 172.21.15.45:0/147384899 conn(0x55cd9ac812a0 msgr2=0x55cd9ac799e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 shutdown_connections 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 1 -- 172.21.15.45:0/147384899 wait complete. 2024-04-25T10:30:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.147+0000 7f019a790000 20 client.10593 trim_cache size 0 max 16384 2024-04-25T10:30:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 Processor -- start 2024-04-25T10:30:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 -- start start 2024-04-25T10:30:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9ac7e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190202020 con 0x55cd9ac7af30 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01901576c0 con 0x55cd9acc4920 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0190201a50 con 0x55cd9acc5240 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55838/0 (socket says 172.21.15.45:55838) 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 -- 172.21.15.45:0/3306103072 learned_addr learned my addr 172.21.15.45:0/3306103072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f0195200640 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f0187400640 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9ac7e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 -- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 msgr2=0x55cd9acce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 -- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 msgr2=0x55cd9ac7e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9ac7e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f018fe00640 1 -- 172.21.15.45:0/3306103072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9ab4b600 con 0x55cd9ac7af30 2024-04-25T10:30:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.163+0000 7f0195200640 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f018fe00640 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7e330 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x55cd9aad02f0 tx=0x7f0180014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5e689d20e06cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f017fe00640 1 -- 172.21.15.45:0/3306103072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180016920 con 0x55cd9ac7af30 2024-04-25T10:30:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f017fe00640 1 -- 172.21.15.45:0/3306103072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0180030070 con 0x55cd9ac7af30 2024-04-25T10:30:31.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f017fe00640 1 -- 172.21.15.45:0/3306103072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180016ac0 con 0x55cd9ac7af30 2024-04-25T10:30:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- 172.21.15.45:0/3306103072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 msgr2=0x55cd9ac7e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7e330 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x55cd9aad02f0 tx=0x7f0180014000 comp rx=0 tx=0).stop 2024-04-25T10:30:31.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- 172.21.15.45:0/3306103072 shutdown_connections 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9acc4920 0x55cd9ac7e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- 172.21.15.45:0/3306103072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9ac7af30 0x55cd9ac7e330 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- 172.21.15.45:0/3306103072 >> 172.21.15.45:0/3306103072 conn(0x55cd9ac812a0 msgr2=0x55cd9ac7fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- 172.21.15.45:0/3306103072 shutdown_connections 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- 172.21.15.45:0/3306103072 wait complete. 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- start start 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 Processor -- start 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9ac7af30 0x55cd9ad22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 0x55cd9acd13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aace3d0 con 0x55cd9acc4920 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9ab85d50 con 0x55cd9ac7af30 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9aa64980 con 0x55cd9acc5240 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 10 client.0 _ll_register_callbacks cb 0x7f01880b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 0x55cd9acd13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 0x55cd9acd13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55842/0 (socket says 172.21.15.45:55842) 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 -- 172.21.15.45:0/2947412542 learned_addr learned my addr 172.21.15.45:0/2947412542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f018fe00640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9ac7af30 0x55cd9ad22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0195200640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 -- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 msgr2=0x55cd9acd1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acd1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 -- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9ac7af30 msgr2=0x55cd9ad22180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9ac7af30 0x55cd9ad22180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 -- 172.21.15.45:0/2947412542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9aa5e9b0 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0195200640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f0187400640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 0x55cd9acd13f0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f0178054aa0 tx=0x7f0178000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5406302374aa39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f017d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f017d600640 1 -- 172.21.15.45:0/2947412542 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178053160 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f017d600640 1 -- 172.21.15.45:0/2947412542 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01780840b0 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.167+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0180030070 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 1 -- 172.21.15.45:0/2947412542 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0178075030 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 10 client.10599 fetch_fsmap learned FSMap version 122 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x55cd9aa710b0 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 1 -- 172.21.15.45:0/2947412542 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0180030070 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 1 -- 172.21.15.45:0/2947412542 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01780074b0 con 0x55cd9acc4920 2024-04-25T10:30:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 10 client.10599 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 10 client.10599 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 20 client.10599 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ad229b0 0x55cd9ad22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 4 client.10599 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cd9aad72e0 con 0x55cd9ad229b0 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f018fe00640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ad229b0 0x55cd9ad22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 10 client.10599 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f018fe00640 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ad229b0 0x55cd9ad22d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b1b1d9a76312b2 server_cookie=2f3e7ca821dab452 in_seq=0 out_seq=0 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 1 -- 172.21.15.45:0/2947412542 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55cd9aad72e0 con 0x55cd9ad229b0 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 10 client.10599 handle_command_reply: tid=0 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 client.10599 shutdown 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 msgr2=0x55cd9acd13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 0x55cd9acd13f0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f0178054aa0 tx=0x7f0178000ee0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 shutdown_connections 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55cd9ad229b0 0x55cd9ad22d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55cd9acc5240 0x55cd9acd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd9acc4920 0x55cd9acd13f0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f017d600640 0 client.10599 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 --2- 172.21.15.45:0/2947412542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd9ac7af30 0x55cd9ad22180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 >> 172.21.15.45:0/2947412542 conn(0x55cd9ac812a0 msgr2=0x55cd9ac80450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 shutdown_connections 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 1 -- 172.21.15.45:0/2947412542 wait complete. 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f019a790000 20 client.10599 trim_cache size 0 max 16384 2024-04-25T10:30:31.178 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10467, 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9834981559999996, 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 122, 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 169, 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 169, 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.824907523 2024-04-25T10:30:31.179 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:30:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f016c066f60 msgr2=0x7f016c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f016c066f60 0x7f016c069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0180009af0 tx=0x7f0180008040 comp rx=0 tx=0).stop 2024-04-25T10:30:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 msgr2=0x7f01900b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 0x7f01900b5460 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f017800c3c0 tx=0x7f017800c880 comp rx=0 tx=0).stop 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 shutdown_connections 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f016c066f60 0x7f016c069420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01900cfd90 0x7f01900c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01900cd2e0 0x7f01900b5460 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 --2- 172.21.15.45:0/2637817615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01900b0f70 0x7f0190157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 >> 172.21.15.45:0/2637817615 conn(0x7f01900b1d50 msgr2=0x7f0190072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 shutdown_connections 2024-04-25T10:30:31.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.171+0000 7f0197200640 1 -- 172.21.15.45:0/2637817615 wait complete. 2024-04-25T10:30:31.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:30:31.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 Processor -- start 2024-04-25T10:30:31.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 -- start start 2024-04-25T10:30:31.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5a80d1820 0x7fd5a80d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5a815e050 0x7fd5a80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a80b1850 con 0x7fd5a80d1820 2024-04-25T10:30:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a80b1af0 con 0x7fd5a815e050 2024-04-25T10:30:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5acc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80b1dd0 con 0x7fd5a80b13e0 2024-04-25T10:30:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5a80d1820 0x7fd5a80d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5a80d1820 0x7fd5a80d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55854/0 (socket says 172.21.15.45:55854) 2024-04-25T10:30:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 -- 172.21.15.45:0/3511184221 learned_addr learned my addr 172.21.15.45:0/3511184221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a6800640 1 --2- 172.21.15.45:0/3511184221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd59d400640 1 --2- 172.21.15.45:0/3511184221 >> v2:172.21.15.117:3300/0 conn(0x7fd5a815e050 0x7fd5a80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 -- 172.21.15.45:0/3511184221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 msgr2=0x7fd5a80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 --2- 172.21.15.45:0/3511184221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a80ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 -- 172.21.15.45:0/3511184221 >> v2:172.21.15.117:3300/0 conn(0x7fd5a815e050 msgr2=0x7fd5a80b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 --2- 172.21.15.45:0/3511184221 >> v2:172.21.15.117:3300/0 conn(0x7fd5a815e050 0x7fd5a80b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.335+0000 7fd5a5e00640 1 -- 172.21.15.45:0/3511184221 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80b20b0 con 0x7fd5a80d1820 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5a5e00640 1 --2- 172.21.15.45:0/3511184221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a80d1c00 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80c5400 tx=0x7fd59801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6b6ca3fba6a657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5a5400640 1 -- 172.21.15.45:0/3511184221 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980a55b0 con 0x7fd5a80d1820 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5a5400640 1 -- 172.21.15.45:0/3511184221 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5980a5750 con 0x7fd5a80d1820 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5a5400640 1 -- 172.21.15.45:0/3511184221 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980a5b20 con 0x7fd5a80d1820 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 -- 172.21.15.45:0/3511184221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 msgr2=0x7fd5a80d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 --2- 172.21.15.45:0/3511184221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a80d1c00 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80c5400 tx=0x7fd59801c730 comp rx=0 tx=0).stop 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 -- 172.21.15.45:0/3511184221 shutdown_connections 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 --2- 172.21.15.45:0/3511184221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 --2- 172.21.15.45:0/3511184221 >> v2:172.21.15.117:3300/0 conn(0x7fd5a815e050 0x7fd5a80b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 --2- 172.21.15.45:0/3511184221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a80d1c00 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 -- 172.21.15.45:0/3511184221 >> 172.21.15.45:0/3511184221 conn(0x7fd5a809a9d0 msgr2=0x7fd5a809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 -- 172.21.15.45:0/3511184221 shutdown_connections 2024-04-25T10:30:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.339+0000 7fd5acc00640 1 -- 172.21.15.45:0/3511184221 wait complete. 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 Processor -- start 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- start start 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a81528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a815e050 0x7fd5a8152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80cfdf0 con 0x7fd5a80d1820 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80cfc70 con 0x7fd5a815e050 2024-04-25T10:30:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80cff70 con 0x7fd5a80b13e0 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a81528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a81528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55860/0 (socket says 172.21.15.45:55860) 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 -- 172.21.15.45:0/440731381 learned_addr learned my addr 172.21.15.45:0/440731381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5a5e00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5a6800640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a815e050 0x7fd5a8152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 -- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 msgr2=0x7fd5a815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 -- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a815e050 msgr2=0x7fd5a8152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a815e050 0x7fd5a8152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 -- 172.21.15.45:0/440731381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80b7740 con 0x7fd5a80d1820 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59d400640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a81528a0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d600 tx=0x7fd59000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590004240 con 0x7fd5a80d1820 2024-04-25T10:30:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5900043e0 con 0x7fd5a80d1820 2024-04-25T10:30:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590003840 con 0x7fd5a80d1820 2024-04-25T10:30:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8153960 con 0x7fd5a80d1820 2024-04-25T10:30:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a8153f20 con 0x7fd5a80d1820 2024-04-25T10:30:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.343+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a804ef30 con 0x7fd5a80d1820 2024-04-25T10:30:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.347+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5900039e0 con 0x7fd5a80d1820 2024-04-25T10:30:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.347+0000 7fd59ea00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd584066f60 0x7fd584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.347+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5900850e0 con 0x7fd5a80d1820 2024-04-25T10:30:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.347+0000 7fd5a5e00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd584066f60 0x7fd584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.347+0000 7fd5a5e00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd584066f60 0x7fd584069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd5980ae000 tx=0x7fd5980a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:31.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.351+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd590055a90 con 0x7fd5a80d1820 2024-04-25T10:30:31.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.547+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd5a80d0a30 con 0x7fd5a80d1820 2024-04-25T10:30:31.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd59ea00640 1 -- 172.21.15.45:0/440731381 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v122) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd590059b30 con 0x7fd5a80d1820 2024-04-25T10:30:31.552 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:30:31.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd584066f60 msgr2=0x7fd584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd584066f60 0x7fd584069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd5980ae000 tx=0x7fd5980a7070 comp rx=0 tx=0).stop 2024-04-25T10:30:31.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 msgr2=0x7fd5a81528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:31.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a81528a0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d600 tx=0x7fd59000dac0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 shutdown_connections 2024-04-25T10:30:31.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd584066f60 0x7fd584069420 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5a815e050 0x7fd5a8152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5a80d1820 0x7fd5a81528a0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 --2- 172.21.15.45:0/440731381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5a80b13e0 0x7fd5a815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 >> 172.21.15.45:0/440731381 conn(0x7fd5a809a9d0 msgr2=0x7fd5a8156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 shutdown_connections 2024-04-25T10:30:31.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:31.551+0000 7fd5acc00640 1 -- 172.21.15.45:0/440731381 wait complete. 2024-04-25T10:30:31.573 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:30:31.573 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:30:31.573 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:30:31.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:30:31.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:31.574 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:30:31.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:30:31.574 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:30:31.574 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:31.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.574 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:30:31.584 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41708sec preferred_lft 41708sec 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:31.585 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:30:31.586 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:30:31.587 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.587 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:30:31.587 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:30:31.587 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:30:31.587 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:30:31.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.587 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:30:31.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.647 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:30:31.700 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:30:31.700 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:30:31.700 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:30:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:30:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:30:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:30:31.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.701 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:30:31.701 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:30:31.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:30:31.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:30:31.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:30:31.701 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:30:31.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.792 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:30:31.802 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:30:31.802 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:30:31.802 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:31.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.803 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:30:31.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:30:31.867 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:31.919 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:30:31.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:30:31.919 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:31.975 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:30:31.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:31.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:30:32.068 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:30:32.068 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:30:32.081 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:30:32.082 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:30:32.082 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:30:32.082 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:30:32.082 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:30:32.086 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:30:32.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:32.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:30:32.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:32.107 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:30:32.174 INFO:teuthology.orchestra.run.smithi045.stdout:client9877 2024-04-25T10:30:32.174 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:32.175 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:32.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:32.227 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:30:32.287 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:30:32.287 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:30:32.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:32.339 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:30:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2024-04-25T10:30:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2024-04-25T10:30:32.478 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0849856 s, 74.0 MB/s 2024-04-25T10:30:32.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:32.483 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-04-25T10:30:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-04-25T10:30:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-04-25T10:30:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = bitmap 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-04-25T10:30:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-04-25T10:30:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-04-25T10:30:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-04-25T10:30:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.534 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-04-25T10:30:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-04-25T10:30:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-04-25T10:30:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-04-25T10:30:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-04-25T10:30:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds cache max size = 1000 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds log max segments = 1 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = False 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:32.538 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:32.539 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:32.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-04-25T10:30:32.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-04-25T10:30:32.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:32.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore allocator = bitmap 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 1/20 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 1/20 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 4/10 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.546 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:32.547 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.548 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.549 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.550 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: mds cache max size = 1000 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2024-04-25T10:30:32.551 INFO:teuthology.orchestra.run.smithi117.stdout: mds log max segments = 1 2024-04-25T10:30:32.552 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-04-25T10:30:32.552 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = False 2024-04-25T10:30:32.552 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:30:32.552 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.552 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:32.602 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:30:32.642 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T10:30:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-04-25T10:30:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:30:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-04-25T10:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = bitmap 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-04-25T10:30:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-04-25T10:30:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-04-25T10:30:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: mds cache max size = 1000 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = False 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: mds log max segments = 1 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = False 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-04-25T10:30:32.655 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-04-25T10:30:32.656 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-04-25T10:30:32.657 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:30:32.658 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:30:32.659 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore allocator = bitmap 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-04-25T10:30:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 1/20 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 1/20 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 4/10 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.661 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-04-25T10:30:32.662 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-04-25T10:30:32.663 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.664 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:30:32.665 INFO:teuthology.orchestra.run.smithi117.stdout: mds cache max size = 1000 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = False 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: mds log max segments = 1 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = False 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:30:32.666 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> import json 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:30:32.668 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2024-04-25T10:30:32.669 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mode": 33204, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ino": 1099511627777, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_dev": 44, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_nlink": 1, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_uid": 1000, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_gid": 1271, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_size": 6291456, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_atime": 1714041032.3875804, 2024-04-25T10:30:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mtime": 1714041032.42358, 2024-04-25T10:30:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ctime": 1714041032.42358 2024-04-25T10:30:32.782 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:30:32.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:30:32.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:30:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd6400640 1 Processor -- start 2024-04-25T10:30:32.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd6400640 1 -- start start 2024-04-25T10:30:32.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd6400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fccd00670b0 0x7fccd00b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd6400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd015de30 con 0x7fccd015da50 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd6400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd00b59f0 con 0x7fccd00b16c0 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd6400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00b5cd0 con 0x7fccd00670b0 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd5000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fccd015da50 0x7fccd00669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33668/0 (socket says 172.21.15.45:33668) 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd5000640 1 -- 172.21.15.45:0/3246754344 learned_addr learned my addr 172.21.15.45:0/3246754344 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd5a00640 1 --2- 172.21.15.45:0/3246754344 >> v2:172.21.15.117:3300/0 conn(0x7fccd00670b0 0x7fccd00b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fcccfe00640 1 -- 172.21.15.45:0/3246754344 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017191656 0 0) 0x7fccd015de30 con 0x7fccd015da50 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fcccfe00640 1 -- 172.21.15.45:0/3246754344 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb4003530 con 0x7fccd015da50 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd5a00640 1 -- 172.21.15.45:0/3246754344 >> v1:172.21.15.117:6789/0 conn(0x7fccd00b16c0 legacy=0x7fccd00ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd5a00640 1 -- 172.21.15.45:0/3246754344 >> v1:172.21.15.45:6789/0 conn(0x7fccd015da50 legacy=0x7fccd00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.975+0000 7fccd5a00640 1 -- 172.21.15.45:0/3246754344 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00b5fb0 con 0x7fccd00670b0 2024-04-25T10:30:32.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd5a00640 1 --2- 172.21.15.45:0/3246754344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00670b0 0x7fccd00b1d10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009d90 tx=0x7fccc801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa5984442cc76c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fcccfe00640 1 -- 172.21.15.45:0/3246754344 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc80b2070 con 0x7fccd00670b0 2024-04-25T10:30:32.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fcccfe00640 1 -- 172.21.15.45:0/3246754344 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc80a5940 con 0x7fccd00670b0 2024-04-25T10:30:32.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fcccfe00640 1 -- 172.21.15.45:0/3246754344 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc80a5cf0 con 0x7fccd00670b0 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 -- 172.21.15.45:0/3246754344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00670b0 msgr2=0x7fccd00b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 --2- 172.21.15.45:0/3246754344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00670b0 0x7fccd00b1d10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009d90 tx=0x7fccc801c970 comp rx=0 tx=0).stop 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 -- 172.21.15.45:0/3246754344 shutdown_connections 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 --2- 172.21.15.45:0/3246754344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00670b0 0x7fccd00b1d10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 -- 172.21.15.45:0/3246754344 >> 172.21.15.45:0/3246754344 conn(0x7fccd00d0690 msgr2=0x7fccd00d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 -- 172.21.15.45:0/3246754344 shutdown_connections 2024-04-25T10:30:32.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.979+0000 7fccd6400640 1 -- 172.21.15.45:0/3246754344 wait complete. 2024-04-25T10:30:32.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 Processor -- start 2024-04-25T10:30:32.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- start start 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 0x7fccd0151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00b16c0 0x7fccd0152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd015da50 0x7fccd0152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00cf9a0 con 0x7fccd00670b0 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00cf820 con 0x7fccd00b16c0 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd016b610 con 0x7fccd015da50 2024-04-25T10:30:32.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 0x7fccd0151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 0x7fccd0151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55868/0 (socket says 172.21.15.45:55868) 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 -- 172.21.15.45:0/3977491725 learned_addr learned my addr 172.21.15.45:0/3977491725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5a00640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd015da50 0x7fccd0152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccc7e00640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00b16c0 0x7fccd0152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 -- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd015da50 msgr2=0x7fccd0152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd015da50 0x7fccd0152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 -- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00b16c0 msgr2=0x7fccd0152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00b16c0 0x7fccd0152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 -- 172.21.15.45:0/3977491725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd0158c10 con 0x7fccd00670b0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd5000640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 0x7fccd0151ae0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c420 tx=0x7fccc000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0017070 con 0x7fccd00670b0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc0005cf0 con 0x7fccd00670b0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00072d0 con 0x7fccd00670b0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd00b5650 con 0x7fccd00670b0 2024-04-25T10:30:32.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd0159900 con 0x7fccd00670b0 2024-04-25T10:30:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.983+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd0158de0 con 0x7fccd00670b0 2024-04-25T10:30:32.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.987+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fccc001a070 con 0x7fccd00670b0 2024-04-25T10:30:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.987+0000 7fcccd600640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fccac066f60 0x7fccac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.987+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccc00842c0 con 0x7fccd00670b0 2024-04-25T10:30:32.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.987+0000 7fccc7e00640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fccac066f60 0x7fccac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:32.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.987+0000 7fccc7e00640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fccac066f60 0x7fccac069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fccd0153700 tx=0x7fccb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:32.991+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fccc0054d30 con 0x7fccd00670b0 2024-04-25T10:30:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccd0153490 con 0x7fccd00670b0 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fcccd600640 1 -- 172.21.15.45:0/3977491725 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7fccc0058dd0 con 0x7fccd00670b0 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:30:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:30:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fccac066f60 msgr2=0x7fccac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fccac066f60 0x7fccac069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fccd0153700 tx=0x7fccb8009040 comp rx=0 tx=0).stop 2024-04-25T10:30:33.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 msgr2=0x7fccd0151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 0x7fccd0151ae0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c420 tx=0x7fccc000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 shutdown_connections 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fccac066f60 0x7fccac069420 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccd015da50 0x7fccd0152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd00b16c0 0x7fccd0152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 --2- 172.21.15.45:0/3977491725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccd00670b0 0x7fccd0151ae0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 >> 172.21.15.45:0/3977491725 conn(0x7fccd00d0690 msgr2=0x7fccd006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 shutdown_connections 2024-04-25T10:30:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.187+0000 7fccd6400640 1 -- 172.21.15.45:0/3977491725 wait complete. 2024-04-25T10:30:33.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:30:33.380 INFO:teuthology.orchestra.run.smithi117.stdout:{"message":"","return_code":0} 2024-04-25T10:30:33.380 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:30:33.381 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T10:30:33.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 Processor -- start 2024-04-25T10:30:33.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 -- start start 2024-04-25T10:30:33.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f09240820 0x561f092d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 0x561f09363ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f092d0a40 0x561f09364410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f092ce3d0 con 0x561f09240c00 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f0919d9f0 con 0x561f092d0a40 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f382c19a080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f092ce550 con 0x561f09240820 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 0x561f09363ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 0x561f09363ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55888/0 (socket says 172.21.15.45:55888) 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 -- 172.21.15.45:0/2129610902 learned_addr learned my addr 172.21.15.45:0/2129610902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3829400640 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f09240820 0x561f092d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3829e00640 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f092d0a40 0x561f09364410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 -- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f09240820 msgr2=0x561f092d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f09240820 0x561f092d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 -- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f092d0a40 msgr2=0x561f09364410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f092d0a40 0x561f09364410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.427+0000 7f3828a00640 1 -- 172.21.15.45:0/2129610902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f092d2b80 con 0x561f09240c00 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3828a00640 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 0x561f09363ed0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f38180ad040 tx=0x7f3818003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd887fd5a2e24577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3823400640 1 -- 172.21.15.45:0/2129610902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38180af040 con 0x561f09240c00 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3823400640 1 -- 172.21.15.45:0/2129610902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38180b9070 con 0x561f09240c00 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3823400640 1 -- 172.21.15.45:0/2129610902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38180aacd0 con 0x561f09240c00 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- 172.21.15.45:0/2129610902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 msgr2=0x561f09363ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 0x561f09363ed0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f38180ad040 tx=0x7f3818003030 comp rx=0 tx=0).stop 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- 172.21.15.45:0/2129610902 shutdown_connections 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f092d0a40 0x561f09364410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240c00 0x561f09363ed0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- 172.21.15.45:0/2129610902 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f09240820 0x561f092d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- 172.21.15.45:0/2129610902 >> 172.21.15.45:0/2129610902 conn(0x561f09221760 msgr2=0x561f092d6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- 172.21.15.45:0/2129610902 shutdown_connections 2024-04-25T10:30:33.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- 172.21.15.45:0/2129610902 wait complete. 2024-04-25T10:30:33.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 Processor -- start 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- start start 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240820 0x561f0935f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f09240c00 0x561f092d4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f092d0a40 0x561f092cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f092d2e20 con 0x561f09240820 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f092cd460 con 0x561f09240c00 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f382c19a080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f092d5fb0 con 0x561f092d0a40 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240820 0x561f0935f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240820 0x561f0935f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55894/0 (socket says 172.21.15.45:55894) 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 -- 172.21.15.45:0/2624511500 learned_addr learned my addr 172.21.15.45:0/2624511500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3828a00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f09240c00 0x561f092d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829e00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f092d0a40 0x561f092cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 -- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f092d0a40 msgr2=0x561f092cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x561f092d0a40 0x561f092cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 -- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f09240c00 msgr2=0x561f092d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f09240c00 0x561f092d4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3829400640 1 -- 172.21.15.45:0/2624511500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f0935fb60 con 0x561f09240820 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.431+0000 7f3828a00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f09240c00 0x561f092d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3829400640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240820 0x561f0935f530 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f381c008780 tx=0x7f381c008c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3822000640 1 -- 172.21.15.45:0/2624511500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c019070 con 0x561f09240820 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3822000640 1 -- 172.21.15.45:0/2624511500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c010ba0 con 0x561f09240820 2024-04-25T10:30:33.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561f0935fe00 con 0x561f09240820 2024-04-25T10:30:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3822000640 1 -- 172.21.15.45:0/2624511500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c015450 con 0x561f09240820 2024-04-25T10:30:33.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f09360340 con 0x561f09240820 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3822000640 1 -- 172.21.15.45:0/2624511500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f381c01c070 con 0x561f09240820 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3822000640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f380c065940 0x7f380c067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3822000640 1 -- 172.21.15.45:0/2624511500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f381c084730 con 0x561f09240820 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f3828a00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f380c065940 0x7f380c067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f382c19a080 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x561f09367ee0 0x561f093682c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.435+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 --> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] -- osd_op(unknown.0.0:1 25.7 25:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169) v9 -- 0x561f09368800 con 0x561f09367ee0 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f3829e00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x561f09367ee0 0x561f093682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f3828a00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f380c065940 0x7f380c067e00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3818019010 tx=0x7f38180ac040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f3829e00640 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x561f09367ee0 0x561f093682c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f3829e00640 1 -- 172.21.15.45:0/2624511500 <== osd.5 v2:172.21.15.117:6802/2444936005 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v169'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3824002630 con 0x561f09367ee0 2024-04-25T10:30:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x561f09367ee0 msgr2=0x561f093682c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x561f09367ee0 0x561f093682c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f380c065940 msgr2=0x7f380c067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f380c065940 0x7f380c067e00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3818019010 tx=0x7f38180ac040 comp rx=0 tx=0).stop 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240820 msgr2=0x561f0935f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 --2- 172.21.15.45:0/2624511500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f09240820 0x561f0935f530 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f381c008780 tx=0x7f381c008c40 comp rx=0 tx=0).stop 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f3829e00640 1 -- 172.21.15.45:0/2624511500 reap_dead start 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 shutdown_connections 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 >> 172.21.15.45:0/2624511500 conn(0x561f09221760 msgr2=0x561f092cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 shutdown_connections 2024-04-25T10:30:33.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.439+0000 7f382c19a080 1 -- 172.21.15.45:0/2624511500 wait complete. 2024-04-25T10:30:33.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:33.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start / recursive,repair 2024-04-25T10:30:33.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 Processor -- start 2024-04-25T10:30:33.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 -- start start 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f064c15bd10 0x7f064c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f064c0cf9e0 0x7f064c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c06c0e0 con 0x7f064c15c0f0 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c0b2ef0 con 0x7f064c15bd10 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f0651a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c0d0350 con 0x7f064c0cf9e0 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f064c15bd10 0x7f064c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f064c15bd10 0x7f064c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55900/0 (socket says 172.21.15.45:55900) 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 -- 172.21.15.45:0/2859127022 learned_addr learned my addr 172.21.15.45:0/2859127022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064b400640 1 --2- 172.21.15.45:0/2859127022 >> v2:172.21.15.117:3300/0 conn(0x7f064c0cf9e0 0x7f064c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 -- 172.21.15.45:0/2859127022 >> v2:172.21.15.117:3300/0 conn(0x7f064c0cf9e0 msgr2=0x7f064c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 --2- 172.21.15.45:0/2859127022 >> v2:172.21.15.117:3300/0 conn(0x7f064c0cf9e0 0x7f064c0cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 -- 172.21.15.45:0/2859127022 >> v1:172.21.15.117:6789/0 conn(0x7f064c15c0f0 legacy=0x7f064c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.631+0000 7f064aa00640 1 -- 172.21.15.45:0/2859127022 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c15c6a0 con 0x7f064c15bd10 2024-04-25T10:30:33.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f064aa00640 1 --2- 172.21.15.45:0/2859127022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0b4610 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f064c0702f0 tx=0x7f063c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c675aa81945d397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f064a000640 1 -- 172.21.15.45:0/2859127022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0aa650 con 0x7f064c15bd10 2024-04-25T10:30:33.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f064a000640 1 -- 172.21.15.45:0/2859127022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c0af070 con 0x7f064c15bd10 2024-04-25T10:30:33.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f064a000640 1 -- 172.21.15.45:0/2859127022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0aaa10 con 0x7f064c15bd10 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 -- 172.21.15.45:0/2859127022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 msgr2=0x7f064c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 --2- 172.21.15.45:0/2859127022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0b4610 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f064c0702f0 tx=0x7f063c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 -- 172.21.15.45:0/2859127022 shutdown_connections 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 --2- 172.21.15.45:0/2859127022 >> v2:172.21.15.117:3300/0 conn(0x7f064c0cf9e0 0x7f064c0cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 --2- 172.21.15.45:0/2859127022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0b4610 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 -- 172.21.15.45:0/2859127022 >> 172.21.15.45:0/2859127022 conn(0x7f064c066b70 msgr2=0x7f064c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 -- 172.21.15.45:0/2859127022 shutdown_connections 2024-04-25T10:30:33.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.635+0000 7f0651a00640 1 -- 172.21.15.45:0/2859127022 wait complete. 2024-04-25T10:30:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 Processor -- start 2024-04-25T10:30:33.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 -- start start 2024-04-25T10:30:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f064c0cf9e0 0x7f064c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064c15c0f0 0x7f064c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0b4cd0 con 0x7f064c15bd10 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0b4b50 con 0x7f064c0cf9e0 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c06cdf0 con 0x7f064c15c0f0 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55914/0 (socket says 172.21.15.45:55914) 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 -- 172.21.15.45:0/308432795 learned_addr learned my addr 172.21.15.45:0/308432795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f064b400640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064c15c0f0 0x7f064c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f064aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f064c0cf9e0 0x7f064c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 -- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064c15c0f0 msgr2=0x7f064c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064c15c0f0 0x7f064c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 -- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f064c0cf9e0 msgr2=0x7f064c071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f064c0cf9e0 0x7f064c071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 -- 172.21.15.45:0/308432795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0af890 con 0x7f064c15bd10 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0642000640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0ad840 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f063400c370 tx=0x7f063400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0643400640 1 -- 172.21.15.45:0/308432795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340052d0 con 0x7f064c15bd10 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0643400640 1 -- 172.21.15.45:0/308432795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0634005470 con 0x7f064c15bd10 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0643400640 1 -- 172.21.15.45:0/308432795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634007260 con 0x7f064c15bd10 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c159e10 con 0x7f064c15bd10 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.639+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c0af5b0 con 0x7f064c15bd10 2024-04-25T10:30:33.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0643400640 1 -- 172.21.15.45:0/308432795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0634007400 con 0x7f064c15bd10 2024-04-25T10:30:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0643400640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0628066f60 0x7f0628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0643400640 1 -- 172.21.15.45:0/308432795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06340844b0 con 0x7f064c15bd10 2024-04-25T10:30:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 Processor -- start 2024-04-25T10:30:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f064aa00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0628066f60 0x7f0628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 -- start start 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f064aa00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0628066f60 0x7f0628069420 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f063c0ae000 tx=0x7f063c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48dc90 0x55828e48e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e443d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e444290 0x55828e444670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828e32af40 con 0x55828e48e5b0 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828e292070 con 0x55828e444290 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55828e21a1a0 con 0x55828e48dc90 2024-04-25T10:30:33.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e443d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e443d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55930/0 (socket says 172.21.15.45:55930) 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 -- 172.21.15.45:0/128045084 learned_addr learned my addr 172.21.15.45:0/128045084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f064aa00640 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48dc90 0x55828e48e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f064b400640 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e444290 0x55828e444670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 -- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48dc90 msgr2=0x55828e48e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48dc90 0x55828e48e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 -- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e444290 msgr2=0x55828e444670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e444290 0x55828e444670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 -- 172.21.15.45:0/128045084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55828e218510 con 0x55828e48e5b0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f064b400640 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e444290 0x55828e444670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.643+0000 7f0642000640 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e443d50 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f063400a040 tx=0x7f0634000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a7113b6a1ec35d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f063b400640 1 -- 172.21.15.45:0/128045084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063405d430 con 0x55828e48e5b0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f063b400640 1 -- 172.21.15.45:0/128045084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063405b0c0 con 0x55828e48e5b0 2024-04-25T10:30:33.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f063b400640 1 -- 172.21.15.45:0/128045084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634051440 con 0x55828e48e5b0 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- 172.21.15.45:0/128045084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 msgr2=0x55828e443d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e443d50 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f063400a040 tx=0x7f0634000bd0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- 172.21.15.45:0/128045084 shutdown_connections 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e444290 0x55828e444670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e443d50 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- 172.21.15.45:0/128045084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48dc90 0x55828e48e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- 172.21.15.45:0/128045084 >> 172.21.15.45:0/128045084 conn(0x55828e440240 msgr2=0x55828e442630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- 172.21.15.45:0/128045084 shutdown_connections 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- 172.21.15.45:0/128045084 wait complete. 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- start start 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 Processor -- start 2024-04-25T10:30:33.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e44ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 0x55828e44f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48e5b0 0x55828e44f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828e24c6c0 con 0x55828e48dc90 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828e28e110 con 0x55828e48e5b0 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55828e348f20 con 0x55828e444290 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 10 client.0 _ll_register_callbacks cb 0x55828e42f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 0x55828e44f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 0x55828e44f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55942/0 (socket says 172.21.15.45:55942) 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 -- 172.21.15.45:0/275824753 learned_addr learned my addr 172.21.15.45:0/275824753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f064b400640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48e5b0 0x55828e44f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f064aa00640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e44ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 -- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 msgr2=0x55828e44ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e44ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 -- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48e5b0 msgr2=0x55828e44f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48e5b0 0x55828e44f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 -- 172.21.15.45:0/275824753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55828e21a3b0 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0642000640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 0x55828e44f370 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f063400cd30 tx=0x7f0634000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02ae0f4bd84d5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f062fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f062fe00640 1 -- 172.21.15.45:0/275824753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634053190 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f062fe00640 1 -- 172.21.15.45:0/275824753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0634054220 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.647+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063405b0c0 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 1 -- 172.21.15.45:0/275824753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06340842d0 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 10 client.10632 fetch_fsmap learned FSMap version 122 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x55828e313e00 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 1 -- 172.21.15.45:0/275824753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063405b0c0 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 1 -- 172.21.15.45:0/275824753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0634068ce0 con 0x55828e48dc90 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 10 client.10632 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 10 client.10632 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:30:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 20 client.10632 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e450180 0x55828e49f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 4 client.10632 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55828e22dd60 con 0x55828e450180 2024-04-25T10:30:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f064aa00640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e450180 0x55828e49f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f064aa00640 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e450180 0x55828e49f410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37260b08e8a58778 server_cookie=97c88fe6819ef35d in_seq=0 out_seq=0 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 10 client.10632 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 1 -- 172.21.15.45:0/275824753 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55828e22dd60 con 0x55828e450180 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 10 client.10632 handle_command_reply: tid=0 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 client.10632 shutdown 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 msgr2=0x55828e44f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 0x55828e44f370 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f063400cd30 tx=0x7f0634000f80 comp rx=0 tx=0).stop 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 shutdown_connections 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e450180 0x55828e49f410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48e5b0 0x55828e44f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48dc90 0x55828e44f370 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 --2- 172.21.15.45:0/275824753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e44ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f062fe00640 0 client.10632 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 >> 172.21.15.45:0/275824753 conn(0x55828e440240 msgr2=0x55828e442ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 shutdown_connections 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 1 -- 172.21.15.45:0/275824753 wait complete. 2024-04-25T10:30:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.651+0000 7f0654f23000 20 client.10632 trim_cache size 0 max 16384 2024-04-25T10:30:33.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 Processor -- start 2024-04-25T10:30:33.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 -- start start 2024-04-25T10:30:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e443640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e443b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e49f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c202d70 con 0x55828e48e5b0 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c201f10 con 0x55828e48dc90 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0714b0 con 0x55828e444290 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e49f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e443640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e49f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55958/0 (socket says 172.21.15.45:55958) 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064b400640 1 -- 172.21.15.45:0/3348963302 learned_addr learned my addr 172.21.15.45:0/3348963302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f0642000640 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e443b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 -- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 msgr2=0x55828e443b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e443b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 -- 172.21.15.45:0/3348963302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 msgr2=0x55828e49f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e49f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 -- 172.21.15.45:0/3348963302 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55828e2e9c50 con 0x55828e444290 2024-04-25T10:30:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064b400640 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e49f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.667+0000 7f064aa00640 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e443640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f063c0ab000 tx=0x7f063c004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616e6a42aa4c6ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f063aa00640 1 -- 172.21.15.45:0/3348963302 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0aaa80 con 0x55828e444290 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f063aa00640 1 -- 172.21.15.45:0/3348963302 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c0af040 con 0x55828e444290 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f063aa00640 1 -- 172.21.15.45:0/3348963302 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0aae20 con 0x55828e444290 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 msgr2=0x55828e443640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e443640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f063c0ab000 tx=0x7f063c004740 comp rx=0 tx=0).stop 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- 172.21.15.45:0/3348963302 shutdown_connections 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e48e5b0 0x55828e49f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e443b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- 172.21.15.45:0/3348963302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e444290 0x55828e443640 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- 172.21.15.45:0/3348963302 >> 172.21.15.45:0/3348963302 conn(0x55828e440240 msgr2=0x55828e448d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- 172.21.15.45:0/3348963302 shutdown_connections 2024-04-25T10:30:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- 172.21.15.45:0/3348963302 wait complete. 2024-04-25T10:30:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- start start 2024-04-25T10:30:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 Processor -- start 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 0x55828e4a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e4a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48e5b0 0x55828e4aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828e234bd0 con 0x55828e444290 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1f7170 con 0x55828e48dc90 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55828e4400c0 con 0x55828e48e5b0 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 10 client.0 _ll_register_callbacks cb 0x7f0644009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 0x55828e4a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 0x55828e4a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55970/0 (socket says 172.21.15.45:55970) 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 -- 172.21.15.45:0/1106799815 learned_addr learned my addr 172.21.15.45:0/1106799815 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0642000640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e4a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064b400640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48e5b0 0x55828e4aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 -- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48e5b0 msgr2=0x55828e4aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48e5b0 0x55828e4aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 -- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 msgr2=0x55828e4a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e4a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 -- 172.21.15.45:0/1106799815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55828e25a120 con 0x55828e444290 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0642000640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e4a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f064aa00640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 0x55828e4a14d0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f063c0097f0 tx=0x7f063c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b526dc4540150c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f062fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f062fe00640 1 -- 172.21.15.45:0/1106799815 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0aaa80 con 0x55828e444290 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f062fe00640 1 -- 172.21.15.45:0/1106799815 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f063c0aac20 con 0x55828e444290 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.671+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063c0af040 con 0x55828e444290 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 1 -- 172.21.15.45:0/1106799815 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f063c0aae20 con 0x55828e444290 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 10 client.10638 fetch_fsmap learned FSMap version 122 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x55828e239170 con 0x55828e444290 2024-04-25T10:30:33.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 1 -- 172.21.15.45:0/1106799815 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c0af040 con 0x55828e444290 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 1 -- 172.21.15.45:0/1106799815 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f063c0cd020 con 0x55828e444290 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 10 client.10638 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 10 client.10638 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 20 client.10638 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e4edd60 0x55828e4ee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 4 client.10638 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55828e29f5b0 con 0x55828e4edd60 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0642000640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e4edd60 0x55828e4ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0642000640 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e4edd60 0x55828e4ee170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e351c3828ccdb89 server_cookie=7ca96efb3cc68f51 in_seq=0 out_seq=0 2024-04-25T10:30:33.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 10 client.10638 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.680 INFO:tasks.ceph.mds.d.smithi117.stderr:2024-04-25T10:30:33.675+0000 7f8f49200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:30:32.419579+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={9877=0-12582912@1} caps={9877=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=9877 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55d096209b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(25)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 1 -- 172.21.15.45:0/1106799815 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55828e29f5b0 con 0x55828e4edd60 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 10 client.10638 handle_command_reply: tid=0 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 client.10638 shutdown 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 msgr2=0x55828e4a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 0x55828e4a14d0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f063c0097f0 tx=0x7f063c01ad30 comp rx=0 tx=0).stop 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 shutdown_connections 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x55828e4edd60 0x55828e4ee170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55828e48e5b0 0x55828e4aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55828e48dc90 0x55828e4a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 --2- 172.21.15.45:0/1106799815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55828e444290 0x55828e4a14d0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f062fe00640 0 client.10638 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 >> 172.21.15.45:0/1106799815 conn(0x55828e440240 msgr2=0x55828e449470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 shutdown_connections 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 1 -- 172.21.15.45:0/1106799815 wait complete. 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0654f23000 20 client.10638 trim_cache size 0 max 16384 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "dc359c70-7451-4b6b-9a9a-967b473250da", 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:30:33.682 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:30:33.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0628066f60 msgr2=0x7f0628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0628066f60 0x7f0628069420 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f063c0ae000 tx=0x7f063c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:30:33.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 msgr2=0x7f064c0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0ad840 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f063400c370 tx=0x7f063400c830 comp rx=0 tx=0).stop 2024-04-25T10:30:33.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 shutdown_connections 2024-04-25T10:30:33.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0628066f60 0x7f0628069420 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f064c15c0f0 0x7f064c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f064c15bd10 0x7f064c0ad840 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 --2- 172.21.15.45:0/308432795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f064c0cf9e0 0x7f064c071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 >> 172.21.15.45:0/308432795 conn(0x7f064c066b70 msgr2=0x7f064c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 shutdown_connections 2024-04-25T10:30:33.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.675+0000 7f0651a00640 1 -- 172.21.15.45:0/308432795 wait complete. 2024-04-25T10:30:33.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:30:33.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:30:33.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfc1800640 1 Processor -- start 2024-04-25T10:30:33.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfc1800640 1 -- start start 2024-04-25T10:30:33.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfc1800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcfbc162020 0x7fcfbc15d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfc1800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc06ac90 con 0x7fcfbc06bd10 2024-04-25T10:30:33.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfc1800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc06af50 con 0x7fcfbc0cd9a0 2024-04-25T10:30:33.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfc1800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc06b880 con 0x7fcfbc162020 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfb2000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcfbc06bd10 0x7fcfbc161910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33682/0 (socket says 172.21.15.45:33682) 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfb2000640 1 -- 172.21.15.45:0/603215859 learned_addr learned my addr 172.21.15.45:0/603215859 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfbb400640 1 --2- 172.21.15.45:0/603215859 >> v2:172.21.15.45:3300/0 conn(0x7fcfbc162020 0x7fcfbc15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfba000640 1 -- 172.21.15.45:0/603215859 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931598377 0 0) 0x7fcfbc06ac90 con 0x7fcfbc06bd10 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfba000640 1 -- 172.21.15.45:0/603215859 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa0003530 con 0x7fcfbc06bd10 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfbb400640 1 -- 172.21.15.45:0/603215859 >> v1:172.21.15.117:6789/0 conn(0x7fcfbc0cd9a0 legacy=0x7fcfbc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfbb400640 1 -- 172.21.15.45:0/603215859 >> v1:172.21.15.45:6789/0 conn(0x7fcfbc06bd10 legacy=0x7fcfbc161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.839+0000 7fcfbb400640 1 -- 172.21.15.45:0/603215859 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc06b230 con 0x7fcfbc162020 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfbb400640 1 --2- 172.21.15.45:0/603215859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc15d610 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40097c0 tx=0x7fcfb401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3fb2d898ef2bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfba000640 1 -- 172.21.15.45:0/603215859 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40b5070 con 0x7fcfbc162020 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfba000640 1 -- 172.21.15.45:0/603215859 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb40a8990 con 0x7fcfbc162020 2024-04-25T10:30:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfba000640 1 -- 172.21.15.45:0/603215859 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40a8d60 con 0x7fcfbc162020 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 -- 172.21.15.45:0/603215859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 msgr2=0x7fcfbc15d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 --2- 172.21.15.45:0/603215859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc15d610 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40097c0 tx=0x7fcfb401ec90 comp rx=0 tx=0).stop 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 -- 172.21.15.45:0/603215859 shutdown_connections 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 --2- 172.21.15.45:0/603215859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc15d610 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 -- 172.21.15.45:0/603215859 >> 172.21.15.45:0/603215859 conn(0x7fcfbc0d19a0 msgr2=0x7fcfbc0d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 -- 172.21.15.45:0/603215859 shutdown_connections 2024-04-25T10:30:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.843+0000 7fcfc1800640 1 -- 172.21.15.45:0/603215859 wait complete. 2024-04-25T10:30:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 Processor -- start 2024-04-25T10:30:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 -- start start 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfbc06bd10 0x7fcfbc1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfbc0cd9a0 0x7fcfbc1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc162660 con 0x7fcfbc162020 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0cdd80 con 0x7fcfbc0cd9a0 2024-04-25T10:30:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0d0d70 con 0x7fcfbc06bd10 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc0b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56002/0 (socket says 172.21.15.45:56002) 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 -- 172.21.15.45:0/4159236704 learned_addr learned my addr 172.21.15.45:0/4159236704 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfbc06bd10 0x7fcfbc1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfb2000640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfbc0cd9a0 0x7fcfbc1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 -- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfbc06bd10 msgr2=0x7fcfbc1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfbc06bd10 0x7fcfbc1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 -- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfbc0cd9a0 msgr2=0x7fcfbc1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfbc0cd9a0 0x7fcfbc1593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 -- 172.21.15.45:0/4159236704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc1e7620 con 0x7fcfbc162020 2024-04-25T10:30:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfbb400640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc0b5a80 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40097c0 tx=0x7fcfb401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfb3400640 1 -- 172.21.15.45:0/4159236704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40bf070 con 0x7fcfbc162020 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfb3400640 1 -- 172.21.15.45:0/4159236704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb40b0640 con 0x7fcfbc162020 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc1e7900 con 0x7fcfbc162020 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfb3400640 1 -- 172.21.15.45:0/4159236704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40b5040 con 0x7fcfbc162020 2024-04-25T10:30:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.847+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc159910 con 0x7fcfbc162020 2024-04-25T10:30:33.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfb3400640 1 -- 172.21.15.45:0/4159236704 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfb40c2070 con 0x7fcfbc162020 2024-04-25T10:30:33.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfb3400640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcf98066f60 0x7fcf98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfb3400640 1 -- 172.21.15.45:0/4159236704 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfb4121780 con 0x7fcfbc162020 2024-04-25T10:30:33.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcf98066f60 0x7fcf98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfc4ce4000 1 Processor -- start 2024-04-25T10:30:33.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfc4ce4000 1 -- start start 2024-04-25T10:30:33.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.851+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcf98066f60 0x7fcf98069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcfac001ea0 tx=0x7fcfac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e84fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 0x5576e84b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e82ffff0 con 0x5576e84fd2f0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e8288150 con 0x5576e84b30a0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e82fc090 con 0x5576e84fc9d0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 0x5576e84b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 0x5576e84b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56010/0 (socket says 172.21.15.45:56010) 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 -- 172.21.15.45:0/1544334473 learned_addr learned my addr 172.21.15.45:0/1544334473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e84fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfbb400640 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 -- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 msgr2=0x5576e84fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e84fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 -- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 msgr2=0x5576e84b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 -- 172.21.15.45:0/1544334473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e84960d0 con 0x5576e84fd2f0 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfbb400640 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfb2000640 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 0x5576e84b2b60 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4007fd0 tx=0x7fcfa4007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4468df63742082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfab400640 1 -- 172.21.15.45:0/1544334473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4001840 con 0x5576e84fd2f0 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfab400640 1 -- 172.21.15.45:0/1544334473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4018070 con 0x5576e84fd2f0 2024-04-25T10:30:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfab400640 1 -- 172.21.15.45:0/1544334473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4001be0 con 0x5576e84fd2f0 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1544334473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 msgr2=0x5576e84b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 0x5576e84b2b60 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4007fd0 tx=0x7fcfa4007410 comp rx=0 tx=0).stop 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1544334473 shutdown_connections 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fd2f0 0x5576e84b2b60 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1544334473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e84fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1544334473 >> 172.21.15.45:0/1544334473 conn(0x5576e84af020 msgr2=0x5576e84b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1544334473 shutdown_connections 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1544334473 wait complete. 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 -- start start 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.855+0000 7fcfc4ce4000 1 Processor -- start 2024-04-25T10:30:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fc9d0 0x5576e84e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e84e2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e82c4250 con 0x5576e84fc9d0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e82ba690 con 0x5576e84b30a0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e8288a40 con 0x5576e84fd2f0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 10 client.0 _ll_register_callbacks cb 0x5576e849de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfb2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fc9d0 0x5576e84e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e84e2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfb2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fc9d0 0x5576e84e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56026/0 (socket says 172.21.15.45:56026) 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfb2000640 1 -- 172.21.15.45:0/1677712744 learned_addr learned my addr 172.21.15.45:0/1677712744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 -- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 msgr2=0x5576e84e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84e17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 -- 172.21.15.45:0/1677712744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fc9d0 msgr2=0x5576e84e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fc9d0 0x5576e84e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 -- 172.21.15.45:0/1677712744 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576e83ad1c0 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbb400640 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e84e2270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40098b0 tx=0x7fcfb40f1d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76573478d510d79c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 1 -- 172.21.15.45:0/1677712744 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40fa500 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 1 -- 172.21.15.45:0/1677712744 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfb411acb0 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfa4018070 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 1 -- 172.21.15.45:0/1677712744 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfb411aeb0 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 10 client.8570 fetch_fsmap learned FSMap version 122 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x5576e83a9170 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 1 -- 172.21.15.45:0/1677712744 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4018070 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 1 -- 172.21.15.45:0/1677712744 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcfb40b8070 con 0x5576e84fd2f0 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 10 client.8570 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 10 client.8570 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:30:33.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 20 client.8570 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850cc80 0x5576e850f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 4 client.8570 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:30:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576e829bc60 con 0x5576e850cc80 2024-04-25T10:30:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850cc80 0x5576e850f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850cc80 0x5576e850f070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc27124481a7b733 server_cookie=a56a5d83be9c0960 in_seq=0 out_seq=0 2024-04-25T10:30:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.859+0000 7fcf9fe00640 10 client.8570 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcf9fe00640 1 -- 172.21.15.45:0/1677712744 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576e829bc60 con 0x5576e850cc80 2024-04-25T10:30:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcf9fe00640 10 client.8570 handle_command_reply: tid=0 2024-04-25T10:30:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 client.8570 shutdown 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 msgr2=0x5576e84e2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e84e2270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40098b0 tx=0x7fcfb40f1d30 comp rx=0 tx=0).stop 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 shutdown_connections 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850cc80 0x5576e850f070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e84e2270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84fc9d0 0x5576e84e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/1677712744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84b30a0 0x5576e84e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcf9fe00640 0 client.8570 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 >> 172.21.15.45:0/1677712744 conn(0x5576e84af020 msgr2=0x5576e84b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 shutdown_connections 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/1677712744 wait complete. 2024-04-25T10:30:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.863+0000 7fcfc4ce4000 20 client.8570 trim_cache size 0 max 16384 2024-04-25T10:30:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 Processor -- start 2024-04-25T10:30:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- start start 2024-04-25T10:30:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e8509e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e850a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fd2f0 0x5576e84b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc1ffdb0 con 0x5576e84b30a0 2024-04-25T10:30:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0c60d0 con 0x5576e84fd2f0 2024-04-25T10:30:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0c54d0 con 0x5576e84fc9d0 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e8509e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e8509e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56036/0 (socket says 172.21.15.45:56036) 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 -- 172.21.15.45:0/2082248801 learned_addr learned my addr 172.21.15.45:0/2082248801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfb2000640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e850a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbb400640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fd2f0 0x5576e84b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 -- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 msgr2=0x5576e850a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e850a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 -- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fd2f0 msgr2=0x5576e84b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fd2f0 0x5576e84b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 -- 172.21.15.45:0/2082248801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e8346de0 con 0x5576e84b30a0 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfb2000640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e850a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbb400640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fd2f0 0x5576e84b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e8509e40 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fcfac001750 tx=0x7fcfac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28855b3174375dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfaaa00640 1 -- 172.21.15.45:0/2082248801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac007880 con 0x5576e84b30a0 2024-04-25T10:30:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfaaa00640 1 -- 172.21.15.45:0/2082248801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac007a20 con 0x5576e84b30a0 2024-04-25T10:30:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfaaa00640 1 -- 172.21.15.45:0/2082248801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0225f0 con 0x5576e84b30a0 2024-04-25T10:30:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/2082248801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 msgr2=0x5576e8509e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e8509e40 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fcfac001750 tx=0x7fcfac012000 comp rx=0 tx=0).stop 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/2082248801 shutdown_connections 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fd2f0 0x5576e84b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fc9d0 0x5576e850a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/2082248801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e8509e40 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/2082248801 >> 172.21.15.45:0/2082248801 conn(0x5576e84af020 msgr2=0x5576e8509730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/2082248801 shutdown_connections 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/2082248801 wait complete. 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 -- start start 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.879+0000 7fcfc4ce4000 1 Processor -- start 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e855c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fc9d0 0x5576e855cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e850ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e829ae00 con 0x5576e84b30a0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0c5ad0 con 0x5576e84fc9d0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576e82ff300 con 0x5576e84fd2f0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 10 client.0 _ll_register_callbacks cb 0x5576e847e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e855c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e855c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56050/0 (socket says 172.21.15.45:56050) 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 -- 172.21.15.45:0/4038534926 learned_addr learned my addr 172.21.15.45:0/4038534926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfb2000640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fc9d0 0x5576e855cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbb400640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e850ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 -- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 msgr2=0x5576e850ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e850ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 -- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fc9d0 msgr2=0x5576e855cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fc9d0 0x5576e855cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 -- 172.21.15.45:0/4038534926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576e82a18e0 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfb2000640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fc9d0 0x5576e855cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbb400640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e850ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfbaa00640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e855c720 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00fa30 tx=0x7fcfac001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e697e49c649104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 1 -- 172.21.15.45:0/4038534926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac034070 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 1 -- 172.21.15.45:0/4038534926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfac0138c0 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfac007a20 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 1 -- 172.21.15.45:0/4038534926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfac03e050 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 10 client.10659 fetch_fsmap learned FSMap version 122 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x5576e83598f0 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 1 -- 172.21.15.45:0/4038534926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac007a20 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 1 -- 172.21.15.45:0/4038534926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcfac041020 con 0x5576e84b30a0 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 10 client.10659 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:30:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 10 client.10659 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 20 client.10659 populate_metadata read hostname 'smithi045' 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850f380 0x5576e850f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 4 client.10659 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5576e830d530 con 0x5576e850f380 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfb2000640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850f380 0x5576e850f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfb2000640 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850f380 0x5576e850f790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d0eb2e2ebf3878 server_cookie=56d9f570a39ae3b4 in_seq=0 out_seq=0 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 10 client.10659 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 1 -- 172.21.15.45:0/4038534926 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5576e830d530 con 0x5576e850f380 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 10 client.10659 handle_command_reply: tid=0 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 client.10659 shutdown 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 msgr2=0x5576e855c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e855c720 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00fa30 tx=0x7fcfac001260 comp rx=0 tx=0).stop 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 shutdown_connections 2024-04-25T10:30:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x5576e850f380 0x5576e850f790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5576e84fd2f0 0x5576e850ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576e84fc9d0 0x5576e855cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 --2- 172.21.15.45:0/4038534926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5576e84b30a0 0x5576e855c720 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcf9fe00640 0 client.10659 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 >> 172.21.15.45:0/4038534926 conn(0x5576e84af020 msgr2=0x5576e8509fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 shutdown_connections 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.883+0000 7fcfc4ce4000 1 -- 172.21.15.45:0/4038534926 wait complete. 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc4ce4000 20 client.10659 trim_cache size 0 max 16384 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:30:33.892 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: "dc359c70-7451-4b6b-9a9a-967b473250da": { 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "dc359c70-7451-4b6b-9a9a-967b473250da", 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:30:33.893 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcf98066f60 msgr2=0x7fcf98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcf98066f60 0x7fcf98069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcfac001ea0 tx=0x7fcfac009040 comp rx=0 tx=0).stop 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 msgr2=0x7fcfbc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc0b5a80 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40097c0 tx=0x7fcfb401ba30 comp rx=0 tx=0).stop 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 shutdown_connections 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcf98066f60 0x7fcf98069420 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc162020 0x7fcfbc0b5a80 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfbc0cd9a0 0x7fcfbc1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 --2- 172.21.15.45:0/4159236704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfbc06bd10 0x7fcfbc1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 >> 172.21.15.45:0/4159236704 conn(0x7fcfbc0d19a0 msgr2=0x7fcfbc0cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 shutdown_connections 2024-04-25T10:30:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:30:33.887+0000 7fcfc1800640 1 -- 172.21.15.45:0/4159236704 wait complete. 2024-04-25T10:30:33.906 INFO:tasks.cephfs.filesystem:scrub status for tag:dc359c70-7451-4b6b-9a9a-967b473250da - {'path': '/', 'tag': 'dc359c70-7451-4b6b-9a9a-967b473250da', 'options': 'recursive,repair'} 2024-04-25T10:31:00.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:00.126 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:03.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:03.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:31:04.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 Processor -- start 2024-04-25T10:31:04.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 -- start start 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20a00aebd0 0x7f20a0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a0071360 con 0x7f20a0072020 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a00af030 con 0x7f20a00aebd0 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f20a4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0159920 con 0x7f20a0159540 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f20a0072020 0x7f20a015f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48292/0 (socket says 172.21.15.45:48292) 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209e800640 1 -- 172.21.15.45:0/1572609165 learned_addr learned my addr 172.21.15.45:0/1572609165 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:04.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209de00640 1 --2- 172.21.15.45:0/1572609165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f2097e00640 1 --2- 172.21.15.45:0/1572609165 >> v2:172.21.15.117:3300/0 conn(0x7f20a00aebd0 0x7f20a0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209d400640 1 -- 172.21.15.45:0/1572609165 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820987447 0 0) 0x7f20a0071360 con 0x7f20a0072020 2024-04-25T10:31:04.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209d400640 1 -- 172.21.15.45:0/1572609165 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003530 con 0x7f20a0072020 2024-04-25T10:31:04.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209de00640 1 -- 172.21.15.45:0/1572609165 >> v2:172.21.15.117:3300/0 conn(0x7f20a00aebd0 msgr2=0x7f20a0071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209de00640 1 --2- 172.21.15.45:0/1572609165 >> v2:172.21.15.117:3300/0 conn(0x7f20a00aebd0 0x7f20a0071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209de00640 1 -- 172.21.15.45:0/1572609165 >> v1:172.21.15.45:6789/0 conn(0x7f20a0072020 legacy=0x7f20a015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f209de00640 1 -- 172.21.15.45:0/1572609165 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a015c2f0 con 0x7f20a0159540 2024-04-25T10:31:04.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.063+0000 7f2097e00640 1 --2- 172.21.15.45:0/1572609165 >> v2:172.21.15.117:3300/0 conn(0x7f20a00aebd0 0x7f20a0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:04.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f209de00640 1 --2- 172.21.15.45:0/1572609165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00ae690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f20900097c0 tx=0x7f209001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270650b90322abce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f209d400640 1 -- 172.21.15.45:0/1572609165 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900b2070 con 0x7f20a0159540 2024-04-25T10:31:04.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f209d400640 1 -- 172.21.15.45:0/1572609165 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900ad640 con 0x7f20a0159540 2024-04-25T10:31:04.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f209d400640 1 -- 172.21.15.45:0/1572609165 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900ada10 con 0x7f20a0159540 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 -- 172.21.15.45:0/1572609165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 msgr2=0x7f20a00ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 --2- 172.21.15.45:0/1572609165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00ae690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f20900097c0 tx=0x7f209001c470 comp rx=0 tx=0).stop 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 -- 172.21.15.45:0/1572609165 shutdown_connections 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 --2- 172.21.15.45:0/1572609165 >> v2:172.21.15.117:3300/0 conn(0x7f20a00aebd0 0x7f20a0071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 --2- 172.21.15.45:0/1572609165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00ae690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 -- 172.21.15.45:0/1572609165 >> 172.21.15.45:0/1572609165 conn(0x7f20a00669a0 msgr2=0x7f20a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 -- 172.21.15.45:0/1572609165 shutdown_connections 2024-04-25T10:31:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.067+0000 7f20a4e00640 1 -- 172.21.15.45:0/1572609165 wait complete. 2024-04-25T10:31:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 Processor -- start 2024-04-25T10:31:04.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 -- start start 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20a0072020 0x7f20a0140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 0x7f20a0150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a006c0d0 con 0x7f20a00aebd0 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a015bfa0 con 0x7f20a0072020 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0072430 con 0x7f20a0159540 2024-04-25T10:31:04.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 0x7f20a0150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 0x7f20a0150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49010/0 (socket says 172.21.15.45:49010) 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 -- 172.21.15.45:0/2230023963 learned_addr learned my addr 172.21.15.45:0/2230023963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f209e800640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f209de00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20a0072020 0x7f20a0140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 -- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 msgr2=0x7f20a00af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 -- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20a0072020 msgr2=0x7f20a0140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20a0072020 0x7f20a0140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 -- 172.21.15.45:0/2230023963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00b38e0 con 0x7f20a00aebd0 2024-04-25T10:31:04.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2097e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 0x7f20a0150fc0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f208800ebe0 tx=0x7f208800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2096000640 1 -- 172.21.15.45:0/2230023963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088017070 con 0x7f20a00aebd0 2024-04-25T10:31:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2096000640 1 -- 172.21.15.45:0/2230023963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800cc30 con 0x7f20a00aebd0 2024-04-25T10:31:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a00b3320 con 0x7f20a00aebd0 2024-04-25T10:31:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f2096000640 1 -- 172.21.15.45:0/2230023963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088013470 con 0x7f20a00aebd0 2024-04-25T10:31:04.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.071+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a0152cc0 con 0x7f20a00aebd0 2024-04-25T10:31:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f2096000640 1 -- 172.21.15.45:0/2230023963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f208801a070 con 0x7f20a00aebd0 2024-04-25T10:31:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f2096000640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f207c066f60 0x7f207c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f2096000640 1 -- 172.21.15.45:0/2230023963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2088084ff0 con 0x7f20a00aebd0 2024-04-25T10:31:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f209de00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f207c066f60 0x7f207c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f20a8321000 1 Processor -- start 2024-04-25T10:31:04.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f20a8321000 1 -- start start 2024-04-25T10:31:04.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.075+0000 7f209de00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f207c066f60 0x7f207c069420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f20900af000 tx=0x7f2090008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 0x56496afd0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd12f0 0x56496af86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496af870a0 0x56496af87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496add3ff0 con 0x56496afd09d0 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ad5c150 con 0x56496af870a0 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56496add0090 con 0x56496afd12f0 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 0x56496afd0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 0x56496afd0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49012/0 (socket says 172.21.15.45:49012) 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 -- 172.21.15.45:0/2997665853 learned_addr learned my addr 172.21.15.45:0/2997665853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209e800640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496af870a0 0x56496af87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f2097e00640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd12f0 0x56496af86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 -- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd12f0 msgr2=0x56496af86b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd12f0 0x56496af86b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 -- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496af870a0 msgr2=0x56496af87480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496af870a0 0x56496af87480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 -- 172.21.15.45:0/2997665853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56496af6a0d0 con 0x56496afd09d0 2024-04-25T10:31:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209e800640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496af870a0 0x56496af87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f2097e00640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd12f0 0x56496af86b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f209de00640 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 0x56496afd0db0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f20900097c0 tx=0x7f2090004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2294edf734d2d52b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f208ea00640 1 -- 172.21.15.45:0/2997665853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900c9070 con 0x56496afd09d0 2024-04-25T10:31:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f208ea00640 1 -- 172.21.15.45:0/2997665853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900b3040 con 0x56496afd09d0 2024-04-25T10:31:04.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f208ea00640 1 -- 172.21.15.45:0/2997665853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900ade60 con 0x56496afd09d0 2024-04-25T10:31:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- 172.21.15.45:0/2997665853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 msgr2=0x56496afd0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 0x56496afd0db0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f20900097c0 tx=0x7f2090004740 comp rx=0 tx=0).stop 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- 172.21.15.45:0/2997665853 shutdown_connections 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496af870a0 0x56496af87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd12f0 0x56496af86b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 --2- 172.21.15.45:0/2997665853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd09d0 0x56496afd0db0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- 172.21.15.45:0/2997665853 >> 172.21.15.45:0/2997665853 conn(0x56496af83020 msgr2=0x56496af85440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- 172.21.15.45:0/2997665853 shutdown_connections 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- 172.21.15.45:0/2997665853 wait complete. 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 -- start start 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.079+0000 7f20a8321000 1 Processor -- start 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496afb57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496afb5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496afb6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ad98250 con 0x56496af870a0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ad8e690 con 0x56496afd12f0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56496ad5ca40 con 0x56496afd09d0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 10 client.0 _ll_register_callbacks cb 0x56496af71e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496afb57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496afb57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49022/0 (socket says 172.21.15.45:49022) 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 -- 172.21.15.45:0/2385080383 learned_addr learned my addr 172.21.15.45:0/2385080383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 -- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 msgr2=0x56496afb5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496afb5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 -- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 msgr2=0x56496afb6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496afb6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 -- 172.21.15.45:0/2385080383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56496ae811c0 con 0x56496af870a0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f209de00640 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496afb57f0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f209001c950 tx=0x7f209001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1273c5171352123a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 1 -- 172.21.15.45:0/2385080383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900d4060 con 0x56496af870a0 2024-04-25T10:31:04.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 1 -- 172.21.15.45:0/2385080383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f20900b9a60 con 0x56496af870a0 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20900b3040 con 0x56496af870a0 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 1 -- 172.21.15.45:0/2385080383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20900da050 con 0x56496af870a0 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 10 client.10671 fetch_fsmap learned FSMap version 122 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x56496ae7d170 con 0x56496af870a0 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 1 -- 172.21.15.45:0/2385080383 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900b3040 con 0x56496af870a0 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 1 -- 172.21.15.45:0/2385080383 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20900dd020 con 0x56496af870a0 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 10 client.10671 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 10 client.10671 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 20 client.10671 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496afe0c80 0x56496afe3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 4 client.10671 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:04.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56496ad6fc60 con 0x56496afe0c80 2024-04-25T10:31:04.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2097e00640 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496afe0c80 0x56496afe3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2097e00640 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496afe0c80 0x56496afe3070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3673c8e4fda148 server_cookie=362395085b8d28ca in_seq=0 out_seq=0 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.083+0000 7f2083400640 10 client.10671 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f2083400640 1 -- 172.21.15.45:0/2385080383 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56496ad6fc60 con 0x56496afe0c80 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f2083400640 10 client.10671 handle_command_reply: tid=0 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 client.10671 shutdown 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 msgr2=0x56496afb57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496afb57f0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f209001c950 tx=0x7f209001acb0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 shutdown_connections 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496afe0c80 0x56496afe3070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496afb6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496afb5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f2083400640 0 client.10671 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 --2- 172.21.15.45:0/2385080383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496afb57f0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 >> 172.21.15.45:0/2385080383 conn(0x56496af83020 msgr2=0x56496af85ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 shutdown_connections 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 1 -- 172.21.15.45:0/2385080383 wait complete. 2024-04-25T10:31:04.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.087+0000 7f20a8321000 20 client.10671 trim_cache size 0 max 16384 2024-04-25T10:31:04.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.099+0000 7f20a8321000 1 Processor -- start 2024-04-25T10:31:04.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- start start 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496b02d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496b02d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496b02dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0202d90 con 0x56496af870a0 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00ae590 con 0x56496afd12f0 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00ad990 con 0x56496afd09d0 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496b02d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496b02d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49032/0 (socket says 172.21.15.45:49032) 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 -- 172.21.15.45:0/4275537483 learned_addr learned my addr 172.21.15.45:0/4275537483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209e800640 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496b02dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f2097e00640 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496b02d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 -- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 msgr2=0x56496b02d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496b02d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 -- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 msgr2=0x56496b02dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496b02dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 -- 172.21.15.45:0/4275537483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56496ae1ade0 con 0x56496af870a0 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209e800640 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496b02dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f209de00640 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496b02d300 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f2090019080 tx=0x7f20900ca000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4df4d6b0d2ce8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f208e000640 1 -- 172.21.15.45:0/4275537483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900c9040 con 0x56496af870a0 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f208e000640 1 -- 172.21.15.45:0/4275537483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900b3040 con 0x56496af870a0 2024-04-25T10:31:04.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f208e000640 1 -- 172.21.15.45:0/4275537483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900cf430 con 0x56496af870a0 2024-04-25T10:31:04.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- 172.21.15.45:0/4275537483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 msgr2=0x56496b02d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496b02d300 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f2090019080 tx=0x7f20900ca000 comp rx=0 tx=0).stop 2024-04-25T10:31:04.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- 172.21.15.45:0/4275537483 shutdown_connections 2024-04-25T10:31:04.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd12f0 0x56496b02dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496afd09d0 0x56496b02d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 --2- 172.21.15.45:0/4275537483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496af870a0 0x56496b02d300 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- 172.21.15.45:0/4275537483 >> 172.21.15.45:0/4275537483 conn(0x56496af83020 msgr2=0x56496afcf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- 172.21.15.45:0/4275537483 shutdown_connections 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- 172.21.15.45:0/4275537483 wait complete. 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 -- start start 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.103+0000 7f20a8321000 1 Processor -- start 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496af870a0 0x56496afe3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd09d0 0x56496afe38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 0x56496afe3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ad6ee00 con 0x56496afd12f0 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00adf90 con 0x56496afd09d0 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56496add3300 con 0x56496af870a0 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 10 client.0 _ll_register_callbacks cb 0x56496af52040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:04.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 0x56496afe3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 0x56496afe3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49040/0 (socket says 172.21.15.45:49040) 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 -- 172.21.15.45:0/712733788 learned_addr learned my addr 172.21.15.45:0/712733788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209de00640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496af870a0 0x56496afe3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2097e00640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd09d0 0x56496afe38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 -- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496af870a0 msgr2=0x56496afe3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496af870a0 0x56496afe3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 -- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd09d0 msgr2=0x56496afe38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd09d0 0x56496afe38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 -- 172.21.15.45:0/712733788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56496ad758e0 con 0x56496afd12f0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209e800640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 0x56496afe3e00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f2098007620 tx=0x7f2098007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da8ad11435416c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 1 -- 172.21.15.45:0/712733788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098019030 con 0x56496afd12f0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 1 -- 172.21.15.45:0/712733788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f20980119e0 con 0x56496afd12f0 2024-04-25T10:31:04.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20900b3040 con 0x56496afd12f0 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 1 -- 172.21.15.45:0/712733788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2098022050 con 0x56496afd12f0 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 10 client.10677 fetch_fsmap learned FSMap version 122 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x56496ae2d8f0 con 0x56496afd12f0 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 1 -- 172.21.15.45:0/712733788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900b3040 con 0x56496afd12f0 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 1 -- 172.21.15.45:0/712733788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 122) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2098025020 con 0x56496afd12f0 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 10 client.10677 fetch_fsmap finished waiting for FSMap version 122 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 10 client.10677 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.d 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 20 client.10677 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496b036f50 0x56496b039340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 4 client.10677 mds_command: new command op to 10467 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 --> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56496ade1530 con 0x56496b036f50 2024-04-25T10:31:04.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209de00640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496b036f50 0x56496b039340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f209de00640 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496b036f50 0x56496b039340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2998d4073ae3ea8 server_cookie=d744e7b67c80cb8e in_seq=0 out_seq=0 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 10 client.10677 ms_handle_connect on v2:172.21.15.117:6835/1582038965 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 1 -- 172.21.15.45:0/712733788 <== mds.0 v2:172.21.15.117:6835/1582038965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56496ade1530 con 0x56496b036f50 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 10 client.10677 handle_command_reply: tid=0 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 client.10677 shutdown 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 msgr2=0x56496afe3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 0x56496afe3e00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f2098007620 tx=0x7f2098007ae0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 shutdown_connections 2024-04-25T10:31:04.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:6835/1582038965,v1:172.21.15.117:6837/1582038965] conn(0x56496b036f50 0x56496b039340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56496afd12f0 0x56496afe3e00 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f2083400640 0 client.10677 ms_handle_reset on v2:172.21.15.117:6835/1582038965 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56496afd09d0 0x56496afe38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 --2- 172.21.15.45:0/712733788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56496af870a0 0x56496afe3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 >> 172.21.15.45:0/712733788 conn(0x56496af83020 msgr2=0x56496afcf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 shutdown_connections 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.107+0000 7f20a8321000 1 -- 172.21.15.45:0/712733788 wait complete. 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a8321000 20 client.10677 trim_cache size 0 max 16384 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:31:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:31:04.116 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:04.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f207c066f60 msgr2=0x7f207c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f207c066f60 0x7f207c069420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f20900af000 tx=0x7f2090008040 comp rx=0 tx=0).stop 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 msgr2=0x7f20a0150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 0x7f20a0150fc0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f208800ebe0 tx=0x7f208800c520 comp rx=0 tx=0).stop 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 shutdown_connections 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f207c066f60 0x7f207c069420 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20a0159540 0x7f20a00af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20a00aebd0 0x7f20a0150fc0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 --2- 172.21.15.45:0/2230023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20a0072020 0x7f20a0140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 >> 172.21.15.45:0/2230023963 conn(0x7f20a00669a0 msgr2=0x7f20a00cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 shutdown_connections 2024-04-25T10:31:04.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.111+0000 7f20a4e00640 1 -- 172.21.15.45:0/2230023963 wait complete. 2024-04-25T10:31:04.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> import json 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:31:04.131 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2024-04-25T10:31:04.132 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:31:04.205 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mode": 33204, 2024-04-25T10:31:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ino": 1099511627777, 2024-04-25T10:31:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "st_dev": 44, 2024-04-25T10:31:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "st_nlink": 1, 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "st_uid": 1000, 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "st_gid": 1271, 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "st_size": 6291456, 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "st_atime": 1714041032.3875804, 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "st_mtime": 1714041032.42358, 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "st_ctime": 1714041032.42358 2024-04-25T10:31:04.206 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:04.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:04.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.395+0000 7fa781e00640 1 Processor -- start 2024-04-25T10:31:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781e00640 1 -- start start 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa77c0b16c0 0x7fa77c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 0x7fa77c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c15de30 con 0x7fa77c15da50 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0b59f0 con 0x7fa77c0b16c0 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0b5cd0 con 0x7fa77c0670b0 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa780a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa77c15da50 0x7fa77c0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48300/0 (socket says 172.21.15.45:48300) 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa780a00640 1 -- 172.21.15.45:0/3165198785 learned_addr learned my addr 172.21.15.45:0/3165198785 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781400640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 0x7fa77c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 --2- 172.21.15.45:0/3165198785 >> v2:172.21.15.117:3300/0 conn(0x7fa77c0b16c0 0x7fa77c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216969200 0 0) 0x7fa77c15de30 con 0x7fa77c15da50 2024-04-25T10:31:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa764003530 con 0x7fa77c15da50 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 271805266 0 0) 0x7fa764003530 con 0x7fa77c15da50 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 msgr2=0x7fa77c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 0x7fa77c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 >> v2:172.21.15.117:3300/0 conn(0x7fa77c0b16c0 msgr2=0x7fa77c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 --2- 172.21.15.45:0/3165198785 >> v2:172.21.15.117:3300/0 conn(0x7fa77c0b16c0 0x7fa77c0b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0b5fb0 con 0x7fa77c15da50 2024-04-25T10:31:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fa7700016a0 con 0x7fa77c15da50 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 >> v1:172.21.15.45:6789/0 conn(0x7fa77c15da50 legacy=0x7fa77c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 0x7fa7640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa764003f40 0x7fa764004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764004d50 0x7fa7640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0b59f0 con 0x7fa764003f40 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0b5cd0 con 0x7fa764004d50 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76400ba20 con 0x7fa77c0670b0 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa764003f40 0x7fa764004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa780a00640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 0x7fa7640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa781400640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764004d50 0x7fa7640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 -- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 msgr2=0x7fa7640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c0670b0 0x7fa7640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 -- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764004d50 msgr2=0x7fa7640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764004d50 0x7fa7640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.399+0000 7fa77be00640 1 -- 172.21.15.45:0/3165198785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764003530 con 0x7fa764003f40 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa77be00640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa764003f40 0x7fa764004810 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fa768009b10 tx=0x7fa76801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21560afeaa800e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680b2070 con 0x7fa764003f40 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7680a5590 con 0x7fa764003f40 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa77b400640 1 -- 172.21.15.45:0/3165198785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680096b0 con 0x7fa764003f40 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781e00640 1 -- 172.21.15.45:0/3165198785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa764003f40 msgr2=0x7fa764004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781e00640 1 --2- 172.21.15.45:0/3165198785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa764003f40 0x7fa764004810 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fa768009b10 tx=0x7fa76801cab0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781400640 1 -- 172.21.15.45:0/3165198785 reap_dead start 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781e00640 1 -- 172.21.15.45:0/3165198785 shutdown_connections 2024-04-25T10:31:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781e00640 1 -- 172.21.15.45:0/3165198785 >> 172.21.15.45:0/3165198785 conn(0x7fa77c0d0690 msgr2=0x7fa77c0d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781e00640 1 -- 172.21.15.45:0/3165198785 shutdown_connections 2024-04-25T10:31:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.403+0000 7fa781e00640 1 -- 172.21.15.45:0/3165198785 wait complete. 2024-04-25T10:31:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 Processor -- start 2024-04-25T10:31:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- start start 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764003f40 0x7fa77c14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 0x7fa77c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c157100 0x7fa77c1f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0cf250 con 0x7fa77c0ca300 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0cf0d0 con 0x7fa764003f40 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0cf3d0 con 0x7fa77c157100 2024-04-25T10:31:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 0x7fa77c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 0x7fa77c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49044/0 (socket says 172.21.15.45:49044) 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 -- 172.21.15.45:0/3091504918 learned_addr learned my addr 172.21.15.45:0/3091504918 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa780a00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764003f40 0x7fa77c14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781400640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c157100 0x7fa77c1f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 -- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c157100 msgr2=0x7fa77c1f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c157100 0x7fa77c1f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 -- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764003f40 msgr2=0x7fa77c14a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764003f40 0x7fa77c14a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 -- 172.21.15.45:0/3091504918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c1575c0 con 0x7fa77c0ca300 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781400640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c157100 0x7fa77c1f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa77be00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 0x7fa77c156bc0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa76801c8e0 tx=0x7fa764005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680bc020 con 0x7fa77c0ca300 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7680a5d40 con 0x7fa77c0ca300 2024-04-25T10:31:04.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680b2040 con 0x7fa77c0ca300 2024-04-25T10:31:04.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c1578a0 con 0x7fa77c0ca300 2024-04-25T10:31:04.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c14a550 con 0x7fa77c0ca300 2024-04-25T10:31:04.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.407+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c075120 con 0x7fa77c0ca300 2024-04-25T10:31:04.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.411+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa768004260 con 0x7fa77c0ca300 2024-04-25T10:31:04.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.411+0000 7fa778c00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa75c066f60 0x7fa75c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.411+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa76811f380 con 0x7fa77c0ca300 2024-04-25T10:31:04.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.411+0000 7fa780a00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa75c066f60 0x7fa75c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.411+0000 7fa780a00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa75c066f60 0x7fa75c069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa77c1f9b70 tx=0x7fa770001e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.415+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7680c6020 con 0x7fa77c0ca300 2024-04-25T10:31:04.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa77c1541f0 con 0x7fa77c0ca300 2024-04-25T10:31:04.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa778c00640 1 -- 172.21.15.45:0/3091504918 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7fa768016800 con 0x7fa77c0ca300 2024-04-25T10:31:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:04.613 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:31:04.613 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:31:04.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa75c066f60 msgr2=0x7fa75c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa75c066f60 0x7fa75c069420 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa77c1f9b70 tx=0x7fa770001e80 comp rx=0 tx=0).stop 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 msgr2=0x7fa77c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 0x7fa77c156bc0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa76801c8e0 tx=0x7fa764005840 comp rx=0 tx=0).stop 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 shutdown_connections 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa75c066f60 0x7fa75c069420 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77c157100 0x7fa77c1f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77c0ca300 0x7fa77c156bc0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 --2- 172.21.15.45:0/3091504918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa764003f40 0x7fa77c14a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 >> 172.21.15.45:0/3091504918 conn(0x7fa77c0d0690 msgr2=0x7fa77c0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 shutdown_connections 2024-04-25T10:31:04.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.611+0000 7fa781e00640 1 -- 172.21.15.45:0/3091504918 wait complete. 2024-04-25T10:31:04.634 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:31:04.804 INFO:teuthology.orchestra.run.smithi117.stdout:{"message":"","return_code":0} 2024-04-25T10:31:04.804 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:31:04.805 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:31:04.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 Processor -- start 2024-04-25T10:31:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- start start 2024-04-25T10:31:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f56f3280 0x5583f586f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 0x5583f57dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f5738680 0x5583f57e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f57dea30 con 0x5583f57382a0 2024-04-25T10:31:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f56ae9f0 con 0x5583f56f3280 2024-04-25T10:31:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583f57debb0 con 0x5583f5738680 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 0x5583f57dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 0x5583f57dff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49066/0 (socket says 172.21.15.45:49066) 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 -- 172.21.15.45:0/693978083 learned_addr learned my addr 172.21.15.45:0/693978083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4007400640 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f56f3280 0x5583f586f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4007e00640 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f5738680 0x5583f57e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 -- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f5738680 msgr2=0x5583f57e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f5738680 0x5583f57e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 -- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f56f3280 msgr2=0x5583f586f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f56f3280 0x5583f586f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 -- 172.21.15.45:0/693978083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583f57e38d0 con 0x5583f57382a0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4006a00640 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 0x5583f57dff30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400a590 tx=0x7f3ff4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a33fdd68df8bec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4005600640 1 -- 172.21.15.45:0/693978083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40af040 con 0x5583f57382a0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4005600640 1 -- 172.21.15.45:0/693978083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff40b9070 con 0x5583f57382a0 2024-04-25T10:31:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f4005600640 1 -- 172.21.15.45:0/693978083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40aad50 con 0x5583f57382a0 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- 172.21.15.45:0/693978083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 msgr2=0x5583f57dff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 0x5583f57dff30 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400a590 tx=0x7f3ff4003030 comp rx=0 tx=0).stop 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- 172.21.15.45:0/693978083 shutdown_connections 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f5738680 0x5583f57e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f57382a0 0x5583f57dff30 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 --2- 172.21.15.45:0/693978083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f56f3280 0x5583f586f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- 172.21.15.45:0/693978083 >> 172.21.15.45:0/693978083 conn(0x5583f5732760 msgr2=0x5583f57333d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- 172.21.15.45:0/693978083 shutdown_connections 2024-04-25T10:31:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- 172.21.15.45:0/693978083 wait complete. 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 Processor -- start 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.851+0000 7f400a292080 1 -- start start 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f56f3280 0x5583f5730eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f57382a0 0x5583f57313f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f5738680 0x5583f572f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f57e8060 con 0x5583f5738680 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583f57e8560 con 0x5583f57382a0 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583f57e3630 con 0x5583f56f3280 2024-04-25T10:31:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f5738680 0x5583f572f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f56f3280 0x5583f5730eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f5738680 0x5583f572f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49074/0 (socket says 172.21.15.45:49074) 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 -- 172.21.15.45:0/2530871744 learned_addr learned my addr 172.21.15.45:0/2530871744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4006a00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f57382a0 0x5583f57313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 -- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f56f3280 msgr2=0x5583f5730eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5583f56f3280 0x5583f5730eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 -- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f57382a0 msgr2=0x5583f57313f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583f57382a0 0x5583f57313f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 -- 172.21.15.45:0/2530871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583f5878600 con 0x5583f5738680 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f4007e00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f5738680 0x5583f572f3b0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f400000c190 tx=0x7f400000c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f3fffe00640 1 -- 172.21.15.45:0/2530871744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000017020 con 0x5583f5738680 2024-04-25T10:31:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f3fffe00640 1 -- 172.21.15.45:0/2530871744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4000013460 con 0x5583f5738680 2024-04-25T10:31:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f3fffe00640 1 -- 172.21.15.45:0/2530871744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000013830 con 0x5583f5738680 2024-04-25T10:31:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5583f58788a0 con 0x5583f5738680 2024-04-25T10:31:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.855+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583f5878de0 con 0x5583f5738680 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f3fffe00640 1 -- 172.21.15.45:0/2530871744 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f400001a020 con 0x5583f5738680 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f3fffe00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3fe80659c0 0x7f3fe8067e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f3fffe00640 1 -- 172.21.15.45:0/2530871744 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4000083e30 con 0x5583f5738680 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x5583f5879520 0x5583f5879900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 --> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] -- osd_op(unknown.0.0:1 25.7 25:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171) v9 -- 0x5583f5879e40 con 0x5583f5879520 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f4007400640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3fe80659c0 0x7f3fe8067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f4006a00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x5583f5879520 0x5583f5879900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f4007400640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3fe80659c0 0x7f3fe8067e80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8011570 tx=0x7f3ff80017a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f4006a00640 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x5583f5879520 0x5583f5879900 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f4006a00640 1 -- 172.21.15.45:0/2530871744 <== osd.5 v2:172.21.15.117:6802/2444936005 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f3ff40b2040 con 0x5583f5879520 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x5583f5879520 msgr2=0x5583f5879900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x5583f5879520 0x5583f5879900 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3fe80659c0 msgr2=0x7f3fe8067e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3fe80659c0 0x7f3fe8067e80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8011570 tx=0x7f3ff80017a0 comp rx=0 tx=0).stop 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f5738680 msgr2=0x5583f572f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 --2- 172.21.15.45:0/2530871744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583f5738680 0x5583f572f3b0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f400000c190 tx=0x7f400000c650 comp rx=0 tx=0).stop 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f4007e00640 1 -- 172.21.15.45:0/2530871744 reap_dead start 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 shutdown_connections 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 >> 172.21.15.45:0/2530871744 conn(0x5583f5732760 msgr2=0x5583f57deee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 shutdown_connections 2024-04-25T10:31:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:04.859+0000 7f400a292080 1 -- 172.21.15.45:0/2530871744 wait complete. 2024-04-25T10:31:04.866 INFO:teuthology.orchestra.run.smithi045.stdout:Ysixmegssubdir  2024-04-25T10:31:04.866 DEBUG:teuthology.orchestra.run.smithi045:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: "ancestors": [ 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1099511627776, 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "sixmegs", 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 8 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:04.958 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1, 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "subdir", 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 11 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 25, 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: "old_pools": [] 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:04.959 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:04.962 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:31:04.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:05.105 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:05.169 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:31:05.169 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:05.170 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:05.179 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:05.179 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:05.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 Processor -- start 2024-04-25T10:31:05.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 -- start start 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47b00cdd90 0x7f47b00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47b015eac0 0x7f47b00cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b015a770 con 0x7f47b015a390 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00ce550 con 0x7f47b015eac0 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b8800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b015e000 con 0x7f47b00cdd90 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47b015eac0 0x7f47b00cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b5400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47b00cdd90 0x7f47b00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47b015eac0 0x7f47b00cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35156/0 (socket says 172.21.15.45:35156) 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b6800640 1 -- 172.21.15.45:0/3002336826 learned_addr learned my addr 172.21.15.45:0/3002336826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:05.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.419+0000 7f47b5e00640 1 --1- 172.21.15.45:0/3002336826 >> v1:172.21.15.117:6789/0 conn(0x7f47b015a390 0x7f47b00cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36460/0 (socket says 172.21.15.45:36460) 2024-04-25T10:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b6800640 1 -- 172.21.15.45:0/3002336826 >> v2:172.21.15.117:3300/0 conn(0x7f47b00cdd90 msgr2=0x7f47b00ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b6800640 1 --2- 172.21.15.45:0/3002336826 >> v2:172.21.15.117:3300/0 conn(0x7f47b00cdd90 0x7f47b00ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b6800640 1 -- 172.21.15.45:0/3002336826 >> v1:172.21.15.117:6789/0 conn(0x7f47b015a390 legacy=0x7f47b00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b6800640 1 -- 172.21.15.45:0/3002336826 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00ce270 con 0x7f47b015eac0 2024-04-25T10:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b6800640 1 --2- 172.21.15.45:0/3002336826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b015eac0 0x7f47b00cd080 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f47b00712e0 tx=0x7f47ac01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746833c7f8a00f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b4a00640 1 -- 172.21.15.45:0/3002336826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00b2d0 con 0x7f47b015eac0 2024-04-25T10:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b4a00640 1 -- 172.21.15.45:0/3002336826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac00b470 con 0x7f47b015eac0 2024-04-25T10:31:05.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b4a00640 1 -- 172.21.15.45:0/3002336826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac0a7b70 con 0x7f47b015eac0 2024-04-25T10:31:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b8800640 1 -- 172.21.15.45:0/3002336826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b015eac0 msgr2=0x7f47b00cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.423+0000 7f47b8800640 1 --2- 172.21.15.45:0/3002336826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b015eac0 0x7f47b00cd080 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f47b00712e0 tx=0x7f47ac01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- 172.21.15.45:0/3002336826 shutdown_connections 2024-04-25T10:31:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 --2- 172.21.15.45:0/3002336826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b015eac0 0x7f47b00cd080 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 --2- 172.21.15.45:0/3002336826 >> v2:172.21.15.117:3300/0 conn(0x7f47b00cdd90 0x7f47b00ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- 172.21.15.45:0/3002336826 >> 172.21.15.45:0/3002336826 conn(0x7f47b00b1d50 msgr2=0x7f47b00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- 172.21.15.45:0/3002336826 shutdown_connections 2024-04-25T10:31:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- 172.21.15.45:0/3002336826 wait complete. 2024-04-25T10:31:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 Processor -- start 2024-04-25T10:31:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- start start 2024-04-25T10:31:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 0x7f47b01f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47b015a390 0x7f47b00b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b015eac0 0x7f47b00b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00aff10 con 0x7f47b00cdd90 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00afd90 con 0x7f47b015eac0 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00b0090 con 0x7f47b015a390 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 0x7f47b01f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 0x7f47b01f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35164/0 (socket says 172.21.15.45:35164) 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 -- 172.21.15.45:0/1461318700 learned_addr learned my addr 172.21.15.45:0/1461318700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b6800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b015eac0 0x7f47b00b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5400640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47b015a390 0x7f47b00b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 -- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47b015a390 msgr2=0x7f47b00b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47b015a390 0x7f47b00b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 -- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b015eac0 msgr2=0x7f47b00b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b015eac0 0x7f47b00b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 -- 172.21.15.45:0/1461318700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0098a40 con 0x7f47b00cdd90 2024-04-25T10:31:05.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b5e00640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 0x7f47b01f6400 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c420 tx=0x7f47a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a40074e0 con 0x7f47b00cdd90 2024-04-25T10:31:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a4007680 con 0x7f47b00cdd90 2024-04-25T10:31:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4004a30 con 0x7f47b00cdd90 2024-04-25T10:31:05.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.427+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b0098d20 con 0x7f47b00cdd90 2024-04-25T10:31:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b00992e0 con 0x7f47b00cdd90 2024-04-25T10:31:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b004ef30 con 0x7f47b00cdd90 2024-04-25T10:31:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47a401a020 con 0x7f47b00cdd90 2024-04-25T10:31:05.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47aa000640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4794066f60 0x7f4794069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f47a4084840 con 0x7f47b00cdd90 2024-04-25T10:31:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47b5400640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4794066f60 0x7f4794069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.431+0000 7f47b5400640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4794066f60 0x7f4794069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f47b00cb710 tx=0x7f47a0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.435+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47a40550e0 con 0x7f47b00cdd90 2024-04-25T10:31:05.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.599+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47b006c9f0 con 0x7f47b00cdd90 2024-04-25T10:31:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47aa000640 1 -- 172.21.15.45:0/1461318700 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f47a4059180 con 0x7f47b00cdd90 2024-04-25T10:31:05.604 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:05.604 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":171,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:45.264415+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":7,"to":4}]},{"pgid":"25.20","mappings":[{"from":1,"to":4}]},{"pgid":"25.21","mappings":[{"from":7,"to":3}]},{"pgid":"25.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4794066f60 msgr2=0x7f4794069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4794066f60 0x7f4794069420 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f47b00cb710 tx=0x7f47a0009190 comp rx=0 tx=0).stop 2024-04-25T10:31:05.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 msgr2=0x7f47b01f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 0x7f47b01f6400 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f47a400c420 tx=0x7f47a400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 shutdown_connections 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4794066f60 0x7f4794069420 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47b015eac0 0x7f47b00b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47b015a390 0x7f47b00b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 --2- 172.21.15.45:0/1461318700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47b00cdd90 0x7f47b01f6400 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 >> 172.21.15.45:0/1461318700 conn(0x7f47b00b1d50 msgr2=0x7f47b0157240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 shutdown_connections 2024-04-25T10:31:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.603+0000 7f47b8800640 1 -- 172.21.15.45:0/1461318700 wait complete. 2024-04-25T10:31:05.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:22.368848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:23.583569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:05.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:05.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9df0200640 1 Processor -- start 2024-04-25T10:31:05.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9df0200640 1 -- start start 2024-04-25T10:31:05.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9df0200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9df0200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de8071360 con 0x7f9de80aeda0 2024-04-25T10:31:05.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9df0200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de8159920 con 0x7f9de8159540 2024-04-25T10:31:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9df0200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815c2f0 con 0x7f9de80721f0 2024-04-25T10:31:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9dece00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9de80aeda0 0x7f9de8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51346/0 (socket says 172.21.15.45:51346) 2024-04-25T10:31:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9dece00640 1 -- 172.21.15.45:0/3171496280 learned_addr learned my addr 172.21.15.45:0/3171496280 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:05.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9dee200640 1 --2- 172.21.15.45:0/3171496280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.767+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717780596 0 0) 0x7f9de8071360 con 0x7f9de80aeda0 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd0003530 con 0x7f9de80aeda0 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265000922 0 0) 0x7f9de8159920 con 0x7f9de8159540 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de8071360 con 0x7f9de8159540 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9dee200640 1 -- 172.21.15.45:0/3171496280 >> v1:172.21.15.117:6789/0 conn(0x7f9de8159540 legacy=0x7f9de80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9dee200640 1 -- 172.21.15.45:0/3171496280 >> v1:172.21.15.45:6789/0 conn(0x7f9de80aeda0 legacy=0x7f9de8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9dee200640 1 -- 172.21.15.45:0/3171496280 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de815c5d0 con 0x7f9de80721f0 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9dee200640 1 --2- 172.21.15.45:0/3171496280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8166920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009dc0 tx=0x7f9de001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b5fa2e75a03f856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b2070 con 0x7f9de80721f0 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00a5a20 con 0x7f9de80721f0 2024-04-25T10:31:05.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9de7e00640 1 -- 172.21.15.45:0/3171496280 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00a5df0 con 0x7f9de80721f0 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9df0200640 1 -- 172.21.15.45:0/3171496280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 msgr2=0x7f9de8166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9df0200640 1 --2- 172.21.15.45:0/3171496280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8166920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009dc0 tx=0x7f9de001c960 comp rx=0 tx=0).stop 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9df0200640 1 -- 172.21.15.45:0/3171496280 shutdown_connections 2024-04-25T10:31:05.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9df0200640 1 --2- 172.21.15.45:0/3171496280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8166920 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.771+0000 7f9df0200640 1 -- 172.21.15.45:0/3171496280 >> 172.21.15.45:0/3171496280 conn(0x7f9de80669a0 msgr2=0x7f9de8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- 172.21.15.45:0/3171496280 shutdown_connections 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- 172.21.15.45:0/3171496280 wait complete. 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 Processor -- start 2024-04-25T10:31:05.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- start start 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 0x7f9de8145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de8159540 0x7f9de8156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8169680 con 0x7f9de80aeda0 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de806c0d0 con 0x7f9de8159540 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8169800 con 0x7f9de80721f0 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 0x7f9de8145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 0x7f9de8145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35170/0 (socket says 172.21.15.45:35170) 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 -- 172.21.15.45:0/2053246451 learned_addr learned my addr 172.21.15.45:0/2053246451 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9ded800640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dee200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de8159540 0x7f9de8156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 -- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 msgr2=0x7f9de8152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 -- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de8159540 msgr2=0x7f9de8156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de8159540 0x7f9de8156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 -- 172.21.15.45:0/2053246451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80b4890 con 0x7f9de80aeda0 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9dece00640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 0x7f9de8145ee0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c500 tx=0x7f9dd400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4017070 con 0x7f9de80aeda0 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd4005cf0 con 0x7f9de80aeda0 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de80b29a0 con 0x7f9de80aeda0 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.775+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40072f0 con 0x7f9de80aeda0 2024-04-25T10:31:05.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.779+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de80b42d0 con 0x7f9de80aeda0 2024-04-25T10:31:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.779+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de804ef30 con 0x7f9de80aeda0 2024-04-25T10:31:05.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.779+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dd401a070 con 0x7f9de80aeda0 2024-04-25T10:31:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.779+0000 7f9de5600640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc8066f90 0x7f9dc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.779+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9dd40842f0 con 0x7f9de80aeda0 2024-04-25T10:31:05.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.779+0000 7f9ded800640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc8066f90 0x7f9dc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:05.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.783+0000 7f9ded800640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc8066f90 0x7f9dc8069450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc001720 tx=0x7f9ddc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.783+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dd4054cc0 con 0x7f9de80aeda0 2024-04-25T10:31:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9de8071b80 con 0x7f9de80aeda0 2024-04-25T10:31:05.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9de5600640 1 -- 172.21.15.45:0/2053246451 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9dd4058d60 con 0x7f9de80aeda0 2024-04-25T10:31:05.952 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc8066f90 msgr2=0x7f9dc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc8066f90 0x7f9dc8069450 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc001720 tx=0x7f9ddc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 msgr2=0x7f9de8145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 0x7f9de8145ee0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c500 tx=0x7f9dd400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 shutdown_connections 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9dc8066f90 0x7f9dc8069450 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9de8159540 0x7f9de8156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9de80aeda0 0x7f9de8145ee0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 --2- 172.21.15.45:0/2053246451 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9de80721f0 0x7f9de8152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:05.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 >> 172.21.15.45:0/2053246451 conn(0x7f9de80669a0 msgr2=0x7f9de806de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:05.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 shutdown_connections 2024-04-25T10:31:05.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:05.951+0000 7f9df0200640 1 -- 172.21.15.45:0/2053246451 wait complete. 2024-04-25T10:31:05.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa18800640 1 Processor -- start 2024-04-25T10:31:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa18800640 1 -- start start 2024-04-25T10:31:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa18800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faa1015ff20 0x7faa100d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa18800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa100cd900 con 0x7faa1015bef0 2024-04-25T10:31:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa18800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa100d0960 con 0x7faa1015bb10 2024-04-25T10:31:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa18800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa100cc470 con 0x7faa1015ff20 2024-04-25T10:31:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa15400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faa1015bef0 0x7faa1015f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51360/0 (socket says 172.21.15.45:51360) 2024-04-25T10:31:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa15400640 1 -- 172.21.15.45:0/176916124 learned_addr learned my addr 172.21.15.45:0/176916124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa16800640 1 --2- 172.21.15.45:0/176916124 >> v2:172.21.15.117:3300/0 conn(0x7faa1015ff20 0x7faa100d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa14a00640 1 -- 172.21.15.45:0/176916124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106106576 0 0) 0x7faa100cd900 con 0x7faa1015bef0 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa14a00640 1 -- 172.21.15.45:0/176916124 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f8003530 con 0x7faa1015bef0 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa16800640 1 -- 172.21.15.45:0/176916124 >> v1:172.21.15.117:6789/0 conn(0x7faa1015bb10 legacy=0x7faa100cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa14a00640 1 -- 172.21.15.45:0/176916124 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2599553893 0 0) 0x7fa9f8003530 con 0x7faa1015bef0 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa16800640 1 -- 172.21.15.45:0/176916124 >> v1:172.21.15.45:6789/0 conn(0x7faa1015bef0 legacy=0x7faa1015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.115+0000 7faa16800640 1 -- 172.21.15.45:0/176916124 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100d0680 con 0x7faa1015ff20 2024-04-25T10:31:06.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa16800640 1 --2- 172.21.15.45:0/176916124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015ff20 0x7faa100d24c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009d90 tx=0x7faa0c01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8e48d1a667a258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa14a00640 1 -- 172.21.15.45:0/176916124 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0b2070 con 0x7faa1015ff20 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa14a00640 1 -- 172.21.15.45:0/176916124 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c0a56a0 con 0x7faa1015ff20 2024-04-25T10:31:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa14a00640 1 -- 172.21.15.45:0/176916124 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00001680 con 0x7faa1015ff20 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 -- 172.21.15.45:0/176916124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015ff20 msgr2=0x7faa100d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 --2- 172.21.15.45:0/176916124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015ff20 0x7faa100d24c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009d90 tx=0x7faa0c01c860 comp rx=0 tx=0).stop 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 -- 172.21.15.45:0/176916124 shutdown_connections 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 --2- 172.21.15.45:0/176916124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015ff20 0x7faa100d24c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 -- 172.21.15.45:0/176916124 >> 172.21.15.45:0/176916124 conn(0x7faa100d19a0 msgr2=0x7faa100d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 -- 172.21.15.45:0/176916124 shutdown_connections 2024-04-25T10:31:06.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.119+0000 7faa18800640 1 -- 172.21.15.45:0/176916124 wait complete. 2024-04-25T10:31:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 Processor -- start 2024-04-25T10:31:06.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 -- start start 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1015bb10 0x7faa100b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015bef0 0x7faa1006be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 0x7faa1006c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa10169b60 con 0x7faa1015ff20 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100d0360 con 0x7faa1015bef0 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa10169ce0 con 0x7faa1015bb10 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 0x7faa1006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 0x7faa1006c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35178/0 (socket says 172.21.15.45:35178) 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 -- 172.21.15.45:0/2812079151 learned_addr learned my addr 172.21.15.45:0/2812079151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa15400640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015bef0 0x7faa1006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa15e00640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1015bb10 0x7faa100b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 -- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1015bb10 msgr2=0x7faa100b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1015bb10 0x7faa100b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 -- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015bef0 msgr2=0x7faa1006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015bef0 0x7faa1006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 -- 172.21.15.45:0/2812079151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1015f010 con 0x7faa1015ff20 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa15e00640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1015bb10 0x7faa100b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa16800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 0x7faa1006c390 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0b0040 tx=0x7faa0c019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0bd020 con 0x7faa1015ff20 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c0b3040 con 0x7faa1015ff20 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0b2050 con 0x7faa1015ff20 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.123+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1006adb0 con 0x7faa1015ff20 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1006b370 con 0x7faa1015ff20 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1004ef30 con 0x7faa1015ff20 2024-04-25T10:31:06.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa0c0ad650 con 0x7faa1015ff20 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa0a000640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa9f4066f60 0x7fa9f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faa0c11fb20 con 0x7faa1015ff20 2024-04-25T10:31:06.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa15e00640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa9f4066f60 0x7fa9f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.127+0000 7faa15e00640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa9f4066f60 0x7fa9f4069420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faa04000f70 tx=0x7faa04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.131+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa0c0c7020 con 0x7faa1015ff20 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.295+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa100d24c0 con 0x7faa1015ff20 2024-04-25T10:31:06.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.295+0000 7faa0a000640 1 -- 172.21.15.45:0/2812079151 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa0c016480 con 0x7faa1015ff20 2024-04-25T10:31:06.299 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa9f4066f60 msgr2=0x7fa9f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa9f4066f60 0x7fa9f4069420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7faa04000f70 tx=0x7faa04009040 comp rx=0 tx=0).stop 2024-04-25T10:31:06.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 msgr2=0x7faa1006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 0x7faa1006c390 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7faa0c0b0040 tx=0x7faa0c019600 comp rx=0 tx=0).stop 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 shutdown_connections 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa9f4066f60 0x7fa9f4069420 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1015ff20 0x7faa1006c390 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1015bef0 0x7faa1006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 --2- 172.21.15.45:0/2812079151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1015bb10 0x7faa100b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 >> 172.21.15.45:0/2812079151 conn(0x7faa100d19a0 msgr2=0x7faa100cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 shutdown_connections 2024-04-25T10:31:06.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.299+0000 7faa18800640 1 -- 172.21.15.45:0/2812079151 wait complete. 2024-04-25T10:31:06.322 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.463+0000 7fbfe9000640 1 Processor -- start 2024-04-25T10:31:06.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe9000640 1 -- start start 2024-04-25T10:31:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b1f50 0x7fbfe40b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe9000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe406be90 con 0x7fbfe40cd780 2024-04-25T10:31:06.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe9000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe406b8d0 con 0x7fbfe40b3dd0 2024-04-25T10:31:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40d0b50 con 0x7fbfe40b1f50 2024-04-25T10:31:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe3e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbfe40cd780 0x7fbfe40cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51374/0 (socket says 172.21.15.45:51374) 2024-04-25T10:31:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe3e00640 1 -- 172.21.15.45:0/55602139 learned_addr learned my addr 172.21.15.45:0/55602139 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:06.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe3400640 1 --2- 172.21.15.45:0/55602139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b1f50 0x7fbfe40b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368261213 0 0) 0x7fbfdc0014a0 con 0x7fbfe40cd780 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc8003530 con 0x7fbfe40cd780 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381165016 0 0) 0x7fbfe406b8d0 con 0x7fbfe40b3dd0 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfdc0014a0 con 0x7fbfe40b3dd0 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe3400640 1 -- 172.21.15.45:0/55602139 >> v1:172.21.15.117:6789/0 conn(0x7fbfe40b3dd0 legacy=0x7fbfe406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997291595 0 0) 0x7fbfc8003530 con 0x7fbfe40cd780 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe3400640 1 -- 172.21.15.45:0/55602139 >> v1:172.21.15.45:6789/0 conn(0x7fbfe40cd780 legacy=0x7fbfe40cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.467+0000 7fbfe3400640 1 -- 172.21.15.45:0/55602139 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe406bbb0 con 0x7fbfe40b1f50 2024-04-25T10:31:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe3400640 1 --2- 172.21.15.45:0/55602139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b1f50 0x7fbfe40b23c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40097f0 tx=0x7fbfd401c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bc2a0211e4891e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40b0070 con 0x7fbfe40b1f50 2024-04-25T10:31:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd40ab400 con 0x7fbfe40b1f50 2024-04-25T10:31:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe2a00640 1 -- 172.21.15.45:0/55602139 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc001bd0 con 0x7fbfe40b1f50 2024-04-25T10:31:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 -- 172.21.15.45:0/55602139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b1f50 msgr2=0x7fbfe40b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 --2- 172.21.15.45:0/55602139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b1f50 0x7fbfe40b23c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40097f0 tx=0x7fbfd401c2e0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 -- 172.21.15.45:0/55602139 shutdown_connections 2024-04-25T10:31:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 --2- 172.21.15.45:0/55602139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b1f50 0x7fbfe40b23c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 -- 172.21.15.45:0/55602139 >> 172.21.15.45:0/55602139 conn(0x7fbfe40b32b0 msgr2=0x7fbfe40b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 -- 172.21.15.45:0/55602139 shutdown_connections 2024-04-25T10:31:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 -- 172.21.15.45:0/55602139 wait complete. 2024-04-25T10:31:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.471+0000 7fbfe9000640 1 Processor -- start 2024-04-25T10:31:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- start start 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 0x7fbfe40b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b3dd0 0x7fbfe4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe40cd780 0x7fbfe4159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe416c230 con 0x7fbfe40b1f50 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40d0550 con 0x7fbfe40cd780 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe416c3b0 con 0x7fbfe40b3dd0 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 0x7fbfe40b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 0x7fbfe40b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35192/0 (socket says 172.21.15.45:35192) 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 -- 172.21.15.45:0/3431195271 learned_addr learned my addr 172.21.15.45:0/3431195271 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfdaa00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b3dd0 0x7fbfe4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3e00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe40cd780 0x7fbfe4159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 -- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b3dd0 msgr2=0x7fbfe4159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b3dd0 0x7fbfe4159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 -- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe40cd780 msgr2=0x7fbfe4159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe40cd780 0x7fbfe4159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 -- 172.21.15.45:0/3431195271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe40aef70 con 0x7fbfe40b1f50 2024-04-25T10:31:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3e00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe40cd780 0x7fbfe4159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfdaa00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b3dd0 0x7fbfe4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe3400640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 0x7fbfe40b03e0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40ae040 tx=0x7fbfd4019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40b0040 con 0x7fbfe40b1f50 2024-04-25T10:31:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd4003620 con 0x7fbfe40b1f50 2024-04-25T10:31:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd40039d0 con 0x7fbfe40b1f50 2024-04-25T10:31:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40b0920 con 0x7fbfe40b1f50 2024-04-25T10:31:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe40af4d0 con 0x7fbfe40b1f50 2024-04-25T10:31:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.475+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe4075120 con 0x7fbfe40b1f50 2024-04-25T10:31:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.479+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfd40bb030 con 0x7fbfe40b1f50 2024-04-25T10:31:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.479+0000 7fbfdbe00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfc0066f60 0x7fbfc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.479+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbfd411e4b0 con 0x7fbfe40b1f50 2024-04-25T10:31:06.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.479+0000 7fbfdaa00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfc0066f60 0x7fbfc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.479+0000 7fbfdaa00640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfc0066f60 0x7fbfc0069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbfe415d7b0 tx=0x7fbfcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.483+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfd40c2020 con 0x7fbfe40b1f50 2024-04-25T10:31:06.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.643+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfe40affb0 con 0x7fbfe40b1f50 2024-04-25T10:31:06.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.643+0000 7fbfdbe00640 1 -- 172.21.15.45:0/3431195271 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfe40aef70 con 0x7fbfe40b1f50 2024-04-25T10:31:06.647 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfc0066f60 msgr2=0x7fbfc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfc0066f60 0x7fbfc0069420 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbfe415d7b0 tx=0x7fbfcc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 msgr2=0x7fbfe40b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 0x7fbfe40b03e0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40ae040 tx=0x7fbfd4019080 comp rx=0 tx=0).stop 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 shutdown_connections 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbfc0066f60 0x7fbfc0069420 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfe40cd780 0x7fbfe4159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfe40b3dd0 0x7fbfe4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 --2- 172.21.15.45:0/3431195271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfe40b1f50 0x7fbfe40b03e0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 >> 172.21.15.45:0/3431195271 conn(0x7fbfe40b32b0 msgr2=0x7fbfe40cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 shutdown_connections 2024-04-25T10:31:06.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.647+0000 7fbfe9000640 1 -- 172.21.15.45:0/3431195271 wait complete. 2024-04-25T10:31:06.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:31:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 Processor -- start 2024-04-25T10:31:06.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 -- start start 2024-04-25T10:31:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 0x7f37180ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37180cd110 0x7f37180cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371815f3b0 con 0x7f37180b0f70 2024-04-25T10:31:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371815ef20 con 0x7f37180cd110 2024-04-25T10:31:06.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f371e400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37180ca810 con 0x7f37180cfd90 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f370ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 0x7f37180ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f3717e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37180cd110 0x7f37180cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f370ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 0x7f37180ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:41660/0 (socket says 172.21.15.45:41660) 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f370ea00640 1 -- 172.21.15.45:0/1465630040 learned_addr learned my addr 172.21.15.45:0/1465630040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f3717e00640 1 -- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 msgr2=0x7f37180ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f3717e00640 1 --2- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 0x7f37180ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f3717e00640 1 -- 172.21.15.45:0/1465630040 >> v1:172.21.15.117:6789/0 conn(0x7f37180b0f70 legacy=0x7f37180b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f3717e00640 1 -- 172.21.15.45:0/1465630040 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371815e5f0 con 0x7f37180cd110 2024-04-25T10:31:06.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f370ea00640 1 --2- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 0x7f37180ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:06.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.811+0000 7f3716a00640 1 -- 172.21.15.45:0/1465630040 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18840163 0 0) 0x7f371815f3b0 con 0x7f37180b0f70 2024-04-25T10:31:06.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f3717e00640 1 --2- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cd110 0x7f37180cba20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d90 tx=0x7f371001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb1c7b32e9818b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f3716a00640 1 -- 172.21.15.45:0/1465630040 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100b2070 con 0x7f37180cd110 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f3716a00640 1 -- 172.21.15.45:0/1465630040 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37100a5830 con 0x7f37180cd110 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f3716a00640 1 -- 172.21.15.45:0/1465630040 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100a5c00 con 0x7f37180cd110 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 -- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cd110 msgr2=0x7f37180cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 --2- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cd110 0x7f37180cba20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d90 tx=0x7f371001c970 comp rx=0 tx=0).stop 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 -- 172.21.15.45:0/1465630040 shutdown_connections 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 --2- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cd110 0x7f37180cba20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 --2- 172.21.15.45:0/1465630040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cfd90 0x7f37180ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 -- 172.21.15.45:0/1465630040 >> 172.21.15.45:0/1465630040 conn(0x7f37180b1d50 msgr2=0x7f37180b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 -- 172.21.15.45:0/1465630040 shutdown_connections 2024-04-25T10:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.815+0000 7f371e400640 1 -- 172.21.15.45:0/1465630040 wait complete. 2024-04-25T10:31:06.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 Processor -- start 2024-04-25T10:31:06.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 -- start start 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 0x7f37180aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cd110 0x7f3718095fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cfd90 0x7f3718155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3718161b50 con 0x7f37180b0f70 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37181619d0 con 0x7f37180cfd90 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3718161cd0 con 0x7f37180cd110 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 0x7f37180aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 0x7f37180aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35200/0 (socket says 172.21.15.45:35200) 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 -- 172.21.15.45:0/79169602 learned_addr learned my addr 172.21.15.45:0/79169602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717e00640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cfd90 0x7f3718155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f370ea00640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cd110 0x7f3718095fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 -- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cd110 msgr2=0x7f3718095fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cd110 0x7f3718095fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 -- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cfd90 msgr2=0x7f3718155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cfd90 0x7f3718155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 -- 172.21.15.45:0/79169602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180964f0 con 0x7f37180b0f70 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f3717400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 0x7f37180aea00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f370800c4d0 tx=0x7f370800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708017070 con 0x7f37180b0f70 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3708005480 con 0x7f37180b0f70 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37180af070 con 0x7f37180b0f70 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708003840 con 0x7f37180b0f70 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.819+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37180af910 con 0x7f37180b0f70 2024-04-25T10:31:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.823+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371804ef30 con 0x7f37180b0f70 2024-04-25T10:31:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.823+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f370801a070 con 0x7f37180b0f70 2024-04-25T10:31:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.823+0000 7f370fe00640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f60 0x7f36f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.823+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3708084380 con 0x7f37180b0f70 2024-04-25T10:31:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.823+0000 7f370ea00640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f60 0x7f36f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.823+0000 7f370ea00640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f60 0x7f36f4069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f37000096d0 tx=0x7f3700009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:06.827+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3708054df0 con 0x7f37180b0f70 2024-04-25T10:31:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37180cc9b0 con 0x7f37180b0f70 2024-04-25T10:31:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f370fe00640 1 -- 172.21.15.45:0/79169602 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v171) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3708058e90 con 0x7f37180b0f70 2024-04-25T10:31:07.005 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:31:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f60 msgr2=0x7f36f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f60 0x7f36f4069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f37000096d0 tx=0x7f3700009240 comp rx=0 tx=0).stop 2024-04-25T10:31:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 msgr2=0x7f37180aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 0x7f37180aea00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f370800c4d0 tx=0x7f370800c990 comp rx=0 tx=0).stop 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 shutdown_connections 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f36f4066f60 0x7f36f4069420 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37180cfd90 0x7f3718155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37180cd110 0x7f3718095fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 --2- 172.21.15.45:0/79169602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37180b0f70 0x7f37180aea00 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.003+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 >> 172.21.15.45:0/79169602 conn(0x7f37180b1d50 msgr2=0x7f3718072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.007+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 shutdown_connections 2024-04-25T10:31:07.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.007+0000 7f371e400640 1 -- 172.21.15.45:0/79169602 wait complete. 2024-04-25T10:31:07.026 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:31:07.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:07.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:31:07.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:07.090 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:31:07.104 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:07.104 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:07.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:07.113 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:07.113 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:07.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:07.155 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:07.155 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:07.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:07.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1fa800640 1 Processor -- start 2024-04-25T10:31:07.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1fa800640 1 -- start start 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415b9e0 0x7fc1f40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1fa800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f40ccfb0 con 0x7fc1f415bdc0 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1fa800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f40cd290 con 0x7fc1f40d1c30 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1fa800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f40cd570 con 0x7fc1f415b9e0 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc1f415bdc0 0x7fc1f40d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51390/0 (socket says 172.21.15.45:51390) 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f2a00640 1 -- 172.21.15.45:0/2588147837 learned_addr learned my addr 172.21.15.45:0/2588147837 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:07.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f3400640 1 --2- 172.21.15.45:0/2588147837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415b9e0 0x7fc1f40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523663399 0 0) 0x7fc1f40ccfb0 con 0x7fc1f415bdc0 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d8003530 con 0x7fc1f415bdc0 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340067607 0 0) 0x7fc1f40cd290 con 0x7fc1f40d1c30 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f40ccfb0 con 0x7fc1f40d1c30 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f3400640 1 -- 172.21.15.45:0/2588147837 >> v1:172.21.15.117:6789/0 conn(0x7fc1f40d1c30 legacy=0x7fc1f40c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f3400640 1 -- 172.21.15.45:0/2588147837 >> v1:172.21.15.45:6789/0 conn(0x7fc1f415bdc0 legacy=0x7fc1f40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.339+0000 7fc1f3400640 1 -- 172.21.15.45:0/2588147837 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f40cd850 con 0x7fc1f415b9e0 2024-04-25T10:31:07.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1f3400640 1 --2- 172.21.15.45:0/2588147837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415b9e0 0x7fc1f40cc410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40097f0 tx=0x7fc1e401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2886af44229f2286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40b0070 con 0x7fc1f415b9e0 2024-04-25T10:31:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e40ab600 con 0x7fc1f415b9e0 2024-04-25T10:31:07.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1f2000640 1 -- 172.21.15.45:0/2588147837 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40ab9d0 con 0x7fc1f415b9e0 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 -- 172.21.15.45:0/2588147837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415b9e0 msgr2=0x7fc1f40cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 --2- 172.21.15.45:0/2588147837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415b9e0 0x7fc1f40cc410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40097f0 tx=0x7fc1e401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 -- 172.21.15.45:0/2588147837 shutdown_connections 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 --2- 172.21.15.45:0/2588147837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415b9e0 0x7fc1f40cc410 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 -- 172.21.15.45:0/2588147837 >> 172.21.15.45:0/2588147837 conn(0x7fc1f409a9d0 msgr2=0x7fc1f409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 -- 172.21.15.45:0/2588147837 shutdown_connections 2024-04-25T10:31:07.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.343+0000 7fc1fa800640 1 -- 172.21.15.45:0/2588147837 wait complete. 2024-04-25T10:31:07.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 Processor -- start 2024-04-25T10:31:07.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- start start 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 0x7fc1f4151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1f415b9e0 0x7fc1f40ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415bdc0 0x7fc1f414cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f40ccd60 con 0x7fc1f40d1c30 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f40c6c00 con 0x7fc1f415b9e0 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f4169840 con 0x7fc1f415bdc0 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 0x7fc1f4151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 0x7fc1f4151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35202/0 (socket says 172.21.15.45:35202) 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 -- 172.21.15.45:0/1473794047 learned_addr learned my addr 172.21.15.45:0/1473794047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f2a00640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1f415b9e0 0x7fc1f40ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3e00640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415bdc0 0x7fc1f414cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 -- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415bdc0 msgr2=0x7fc1f414cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415bdc0 0x7fc1f414cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 -- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1f415b9e0 msgr2=0x7fc1f40ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1f415b9e0 0x7fc1f40ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 -- 172.21.15.45:0/1473794047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f4150e90 con 0x7fc1f40d1c30 2024-04-25T10:31:07.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1f3400640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 0x7fc1f4151130 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40190f0 tx=0x7fc1e401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40b0050 con 0x7fc1f40d1c30 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e40bc070 con 0x7fc1f40d1c30 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4006e20 con 0x7fc1f40d1c30 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f40ca770 con 0x7fc1f40d1c30 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f40cad30 con 0x7fc1f40d1c30 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.347+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f404ef30 con 0x7fc1f40d1c30 2024-04-25T10:31:07.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.351+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1e40b5870 con 0x7fc1f40d1c30 2024-04-25T10:31:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.351+0000 7fc1eb400640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1d4066f60 0x7fc1d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.351+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc1e411f750 con 0x7fc1f40d1c30 2024-04-25T10:31:07.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.351+0000 7fc1f2a00640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1d4066f60 0x7fc1d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.351+0000 7fc1f2a00640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1d4066f60 0x7fc1d4069420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4070330 tx=0x7fc1e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.355+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc1e40160d0 con 0x7fc1f40d1c30 2024-04-25T10:31:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.547+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1f406c960 con 0x7fc1f40d1c30 2024-04-25T10:31:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.547+0000 7fc1eb400640 1 -- 172.21.15.45:0/1473794047 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7fc1e40f06c0 con 0x7fc1f40d1c30 2024-04-25T10:31:07.551 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:07.551 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:31:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1d4066f60 msgr2=0x7fc1d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1d4066f60 0x7fc1d4069420 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4070330 tx=0x7fc1e0009040 comp rx=0 tx=0).stop 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 msgr2=0x7fc1f4151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 0x7fc1f4151130 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40190f0 tx=0x7fc1e401ad20 comp rx=0 tx=0).stop 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 shutdown_connections 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1d4066f60 0x7fc1d4069420 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1f415bdc0 0x7fc1f414cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1f415b9e0 0x7fc1f40ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 --2- 172.21.15.45:0/1473794047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1f40d1c30 0x7fc1f4151130 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 >> 172.21.15.45:0/1473794047 conn(0x7fc1f409a9d0 msgr2=0x7fc1f40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 shutdown_connections 2024-04-25T10:31:07.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.551+0000 7fc1fa800640 1 -- 172.21.15.45:0/1473794047 wait complete. 2024-04-25T10:31:07.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:07.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 Processor -- start 2024-04-25T10:31:07.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 -- start start 2024-04-25T10:31:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c3c15bc40 0x7f2c3c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 0x7f2c3c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3c0b2770 con 0x7f2c3c0cda10 2024-04-25T10:31:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c06c670 con 0x7f2c3c15bc40 2024-04-25T10:31:07.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c44400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c06c950 con 0x7f2c3c0cf930 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 0x7f2c3c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 0x7f2c3c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:41702/0 (socket says 172.21.15.45:41702) 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 -- 172.21.15.45:0/1919059622 learned_addr learned my addr 172.21.15.45:0/1919059622 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c41000640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2c3c0cda10 0x7f2c3c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36520/0 (socket says 172.21.15.45:36520) 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c41a00640 1 --2- 172.21.15.45:0/1919059622 >> v2:172.21.15.117:3300/0 conn(0x7f2c3c15bc40 0x7f2c3c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c3be00640 1 -- 172.21.15.45:0/1919059622 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855071908 0 0) 0x7f2c3c0b2770 con 0x7f2c3c0cda10 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c3be00640 1 -- 172.21.15.45:0/1919059622 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c24003530 con 0x7f2c3c0cda10 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 -- 172.21.15.45:0/1919059622 >> v2:172.21.15.117:3300/0 conn(0x7f2c3c15bc40 msgr2=0x7f2c3c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 --2- 172.21.15.45:0/1919059622 >> v2:172.21.15.117:3300/0 conn(0x7f2c3c15bc40 0x7f2c3c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 -- 172.21.15.45:0/1919059622 >> v1:172.21.15.117:6789/0 conn(0x7f2c3c0cda10 legacy=0x7f2c3c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.715+0000 7f2c42400640 1 -- 172.21.15.45:0/1919059622 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c06cc30 con 0x7f2c3c0cf930 2024-04-25T10:31:07.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c42400640 1 --2- 172.21.15.45:0/1919059622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 0x7f2c3c0cfde0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009e50 tx=0x7f2c3401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e0cc2eef776cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c3be00640 1 -- 172.21.15.45:0/1919059622 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340a5660 con 0x7f2c3c0cf930 2024-04-25T10:31:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c3be00640 1 -- 172.21.15.45:0/1919059622 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c340a5800 con 0x7f2c3c0cf930 2024-04-25T10:31:07.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c3be00640 1 -- 172.21.15.45:0/1919059622 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c340a5bd0 con 0x7f2c3c0cf930 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 -- 172.21.15.45:0/1919059622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 msgr2=0x7f2c3c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 --2- 172.21.15.45:0/1919059622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 0x7f2c3c0cfde0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009e50 tx=0x7f2c3401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 -- 172.21.15.45:0/1919059622 shutdown_connections 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 --2- 172.21.15.45:0/1919059622 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c0cf930 0x7f2c3c0cfde0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 --2- 172.21.15.45:0/1919059622 >> v2:172.21.15.117:3300/0 conn(0x7f2c3c15bc40 0x7f2c3c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 -- 172.21.15.45:0/1919059622 >> 172.21.15.45:0/1919059622 conn(0x7f2c3c066b70 msgr2=0x7f2c3c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 -- 172.21.15.45:0/1919059622 shutdown_connections 2024-04-25T10:31:07.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.719+0000 7f2c44400640 1 -- 172.21.15.45:0/1919059622 wait complete. 2024-04-25T10:31:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 Processor -- start 2024-04-25T10:31:07.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- start start 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c3c0cda10 0x7f2c3c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 0x7f2c3c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c15bc40 0x7f2c3c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0b3890 con 0x7f2c3c0cf930 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0b3710 con 0x7f2c3c0cda10 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0b3a10 con 0x7f2c3c15bc40 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 0x7f2c3c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 0x7f2c3c0b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35204/0 (socket says 172.21.15.45:35204) 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 -- 172.21.15.45:0/3181664609 learned_addr learned my addr 172.21.15.45:0/3181664609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:07.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c42400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c15bc40 0x7f2c3c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41a00640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c3c0cda10 0x7f2c3c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 -- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c15bc40 msgr2=0x7f2c3c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c15bc40 0x7f2c3c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 -- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c3c0cda10 msgr2=0x7f2c3c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c3c0cda10 0x7f2c3c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 -- 172.21.15.45:0/3181664609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c14bc00 con 0x7f2c3c0cf930 2024-04-25T10:31:07.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c41000640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 0x7f2c3c0b1630 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800c3c0 tx=0x7f2c2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280074e0 con 0x7f2c3c0cf930 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280047f0 con 0x7f2c3c0cf930 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280076a0 con 0x7f2c3c0cf930 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c14be70 con 0x7f2c3c0cf930 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c14c430 con 0x7f2c3c0cf930 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.723+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c04ef30 con 0x7f2c3c0cf930 2024-04-25T10:31:07.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.727+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c2801a020 con 0x7f2c3c0cf930 2024-04-25T10:31:07.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.727+0000 7f2c39600640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.727+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2c28084800 con 0x7f2c3c0cf930 2024-04-25T10:31:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.727+0000 7f2c41a00640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:07.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.727+0000 7f2c41a00640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c15ab30 tx=0x7f2c30009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:07.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.731+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c280550a0 con 0x7f2c3c0cf930 2024-04-25T10:31:07.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.895+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c3c0cf4e0 con 0x7f2c3c0cf930 2024-04-25T10:31:07.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c39600640 1 -- 172.21.15.45:0/3181664609 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f2c28059140 con 0x7f2c3c0cf930 2024-04-25T10:31:07.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:07.900 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":171,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:45.264415+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":7,"to":4}]},{"pgid":"25.20","mappings":[{"from":1,"to":4}]},{"pgid":"25.21","mappings":[{"from":7,"to":3}]},{"pgid":"25.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c1c066f60 msgr2=0x7f2c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c15ab30 tx=0x7f2c30009190 comp rx=0 tx=0).stop 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 msgr2=0x7f2c3c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 0x7f2c3c0b1630 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800c3c0 tx=0x7f2c2800c880 comp rx=0 tx=0).stop 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 shutdown_connections 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c3c15bc40 0x7f2c3c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2c3c0cf930 0x7f2c3c0b1630 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 --2- 172.21.15.45:0/3181664609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c3c0cda10 0x7f2c3c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 >> 172.21.15.45:0/3181664609 conn(0x7f2c3c066b70 msgr2=0x7f2c3c14e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 shutdown_connections 2024-04-25T10:31:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:07.899+0000 7f2c44400640 1 -- 172.21.15.45:0/3181664609 wait complete. 2024-04-25T10:31:07.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:30:22.368848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:30:23.583569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:07.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.063+0000 7f036b400640 1 Processor -- start 2024-04-25T10:31:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f036b400640 1 -- start start 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f036b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03640aef80 0x7f036415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f036b400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03640b31c0 0x7f03640b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f036b400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036415b080 con 0x7f0364070350 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f036b400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036415adb0 con 0x7f03640b31c0 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f036b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036415eb60 con 0x7f03640aef80 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03640b31c0 0x7f03640b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03640b31c0 0x7f03640b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35214/0 (socket says 172.21.15.45:35214) 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 -- 172.21.15.45:0/3860298661 learned_addr learned my addr 172.21.15.45:0/3860298661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0368a00640 1 --2- 172.21.15.45:0/3860298661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03640aef80 0x7f036415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 -- 172.21.15.45:0/3860298661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03640aef80 msgr2=0x7f036415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 --2- 172.21.15.45:0/3860298661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03640aef80 0x7f036415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 -- 172.21.15.45:0/3860298661 >> v1:172.21.15.117:6789/0 conn(0x7f0364070350 legacy=0x7f036415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.067+0000 7f0369400640 1 -- 172.21.15.45:0/3860298661 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036415ee40 con 0x7f03640b31c0 2024-04-25T10:31:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f0369400640 1 --2- 172.21.15.45:0/3860298661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03640b3610 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f035c009e50 tx=0x7f035c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a22976c41aff11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f0363e00640 1 -- 172.21.15.45:0/3860298661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0a5810 con 0x7f03640b31c0 2024-04-25T10:31:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f0363e00640 1 -- 172.21.15.45:0/3860298661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c0a59b0 con 0x7f03640b31c0 2024-04-25T10:31:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f0363e00640 1 -- 172.21.15.45:0/3860298661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0a5d80 con 0x7f03640b31c0 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 -- 172.21.15.45:0/3860298661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 msgr2=0x7f03640b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 --2- 172.21.15.45:0/3860298661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03640b3610 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f035c009e50 tx=0x7f035c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 -- 172.21.15.45:0/3860298661 shutdown_connections 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 --2- 172.21.15.45:0/3860298661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03640b3610 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 --2- 172.21.15.45:0/3860298661 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03640aef80 0x7f036415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 -- 172.21.15.45:0/3860298661 >> 172.21.15.45:0/3860298661 conn(0x7f03640669a0 msgr2=0x7f0364066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 -- 172.21.15.45:0/3860298661 shutdown_connections 2024-04-25T10:31:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 -- 172.21.15.45:0/3860298661 wait complete. 2024-04-25T10:31:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 Processor -- start 2024-04-25T10:31:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.071+0000 7f036b400640 1 -- start start 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0364070350 0x7f036414b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03640aef80 0x7f03640cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03641f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364165990 con 0x7f03640b31c0 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364165810 con 0x7f03640aef80 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0364165b10 con 0x7f0364070350 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03641f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03641f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35220/0 (socket says 172.21.15.45:35220) 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 -- 172.21.15.45:0/1007422754 learned_addr learned my addr 172.21.15.45:0/1007422754 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f035be00640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03640aef80 0x7f03640cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0368a00640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0364070350 0x7f036414b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 -- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0364070350 msgr2=0x7f036414b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0364070350 0x7f036414b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 -- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03640aef80 msgr2=0x7f03640cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03640aef80 0x7f03640cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 -- 172.21.15.45:0/1007422754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036414b9a0 con 0x7f03640b31c0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0369400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03641f4810 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f035c01a660 tx=0x7f035c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0b8640 con 0x7f03640b31c0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c0bf070 con 0x7f03640b31c0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03640ce370 con 0x7f03640b31c0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0b8a40 con 0x7f03640b31c0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03640af630 con 0x7f03640b31c0 2024-04-25T10:31:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036404ef30 con 0x7f03640b31c0 2024-04-25T10:31:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.075+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f035c0b8be0 con 0x7f03640b31c0 2024-04-25T10:31:08.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.079+0000 7f0361600640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0340066f60 0x7f0340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.079+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f035c11fc40 con 0x7f03640b31c0 2024-04-25T10:31:08.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.079+0000 7f0368a00640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0340066f60 0x7f0340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.079+0000 7f0368a00640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0340066f60 0x7f0340069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0354009b60 tx=0x7f0354009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.083+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f035c016620 con 0x7f03640b31c0 2024-04-25T10:31:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0364156000 con 0x7f03640b31c0 2024-04-25T10:31:08.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f0361600640 1 -- 172.21.15.45:0/1007422754 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f035c0a5980 con 0x7f03640b31c0 2024-04-25T10:31:08.245 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:08.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0340066f60 msgr2=0x7f0340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0340066f60 0x7f0340069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0354009b60 tx=0x7f0354009190 comp rx=0 tx=0).stop 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 msgr2=0x7f03641f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03641f4810 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f035c01a660 tx=0x7f035c0197d0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 shutdown_connections 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0340066f60 0x7f0340069420 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03640b31c0 0x7f03641f4810 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03640aef80 0x7f03640cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 --2- 172.21.15.45:0/1007422754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0364070350 0x7f036414b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 >> 172.21.15.45:0/1007422754 conn(0x7f03640669a0 msgr2=0x7f0364066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 shutdown_connections 2024-04-25T10:31:08.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.243+0000 7f036b400640 1 -- 172.21.15.45:0/1007422754 wait complete. 2024-04-25T10:31:08.265 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 Processor -- start 2024-04-25T10:31:08.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 -- start start 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f34159540 0x7f3f340ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f340721f0 0x7f3f34070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f34159920 con 0x7f3f340aebd0 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3415c2f0 con 0x7f3f34159540 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f3a400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3415c5d0 con 0x7f3f340721f0 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f33400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f34159540 0x7f3f340ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f2be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f340aebd0 0x7f3f34071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51404/0 (socket says 172.21.15.45:51404) 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.407+0000 7f3f2be00640 1 -- 172.21.15.45:0/4081866274 learned_addr learned my addr 172.21.15.45:0/4081866274 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:08.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f33e00640 1 --2- 172.21.15.45:0/4081866274 >> v2:172.21.15.117:3300/0 conn(0x7f3f340721f0 0x7f3f34070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f33400640 1 -- 172.21.15.45:0/4081866274 >> v2:172.21.15.117:3300/0 conn(0x7f3f340721f0 msgr2=0x7f3f34070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f33400640 1 --2- 172.21.15.45:0/4081866274 >> v2:172.21.15.117:3300/0 conn(0x7f3f340721f0 0x7f3f34070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f33400640 1 -- 172.21.15.45:0/4081866274 >> v1:172.21.15.45:6789/0 conn(0x7f3f340aebd0 legacy=0x7f3f34071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f33400640 1 -- 172.21.15.45:0/4081866274 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3415c8b0 con 0x7f3f34159540 2024-04-25T10:31:08.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f32a00640 1 -- 172.21.15.45:0/4081866274 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423220045 0 0) 0x7f3f34159920 con 0x7f3f340aebd0 2024-04-25T10:31:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f33400640 1 --2- 172.21.15.45:0/4081866274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34159540 0x7f3f340ae690 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f3f340cd200 tx=0x7f3f2401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef4466212a44592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f32a00640 1 -- 172.21.15.45:0/4081866274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240b1070 con 0x7f3f34159540 2024-04-25T10:31:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f32a00640 1 -- 172.21.15.45:0/4081866274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f240ac5f0 con 0x7f3f34159540 2024-04-25T10:31:08.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.411+0000 7f3f32a00640 1 -- 172.21.15.45:0/4081866274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240ac9c0 con 0x7f3f34159540 2024-04-25T10:31:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- 172.21.15.45:0/4081866274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34159540 msgr2=0x7f3f340ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 --2- 172.21.15.45:0/4081866274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34159540 0x7f3f340ae690 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f3f340cd200 tx=0x7f3f2401c480 comp rx=0 tx=0).stop 2024-04-25T10:31:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- 172.21.15.45:0/4081866274 shutdown_connections 2024-04-25T10:31:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 --2- 172.21.15.45:0/4081866274 >> v2:172.21.15.117:3300/0 conn(0x7f3f340721f0 0x7f3f34070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 --2- 172.21.15.45:0/4081866274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34159540 0x7f3f340ae690 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- 172.21.15.45:0/4081866274 >> 172.21.15.45:0/4081866274 conn(0x7f3f340669a0 msgr2=0x7f3f34066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- 172.21.15.45:0/4081866274 shutdown_connections 2024-04-25T10:31:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- 172.21.15.45:0/4081866274 wait complete. 2024-04-25T10:31:08.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 Processor -- start 2024-04-25T10:31:08.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- start start 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 0x7f3f3413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f340aebd0 0x7f3f341417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34159540 0x7f3f3415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f34071220 con 0x7f3f340721f0 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340710a0 con 0x7f3f34159540 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f3a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340713a0 con 0x7f3f340aebd0 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 0x7f3f3413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 0x7f3f3413eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35244/0 (socket says 172.21.15.45:35244) 2024-04-25T10:31:08.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 -- 172.21.15.45:0/911488632 learned_addr learned my addr 172.21.15.45:0/911488632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f2be00640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f340aebd0 0x7f3f341417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33e00640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34159540 0x7f3f3415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 -- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f340aebd0 msgr2=0x7f3f341417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f340aebd0 0x7f3f341417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 -- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34159540 msgr2=0x7f3f3415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34159540 0x7f3f3415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 -- 172.21.15.45:0/911488632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3413f420 con 0x7f3f340721f0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f2be00640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f340aebd0 0x7f3f341417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33e00640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34159540 0x7f3f3415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.415+0000 7f3f33400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 0x7f3f3413eee0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f3f24000c10 tx=0x7f3f24019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240bc070 con 0x7f3f340721f0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f24004030 con 0x7f3f340721f0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f240b1040 con 0x7f3f340721f0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f341420b0 con 0x7f3f340721f0 2024-04-25T10:31:08.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f34142670 con 0x7f3f340721f0 2024-04-25T10:31:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f340cc3b0 con 0x7f3f340721f0 2024-04-25T10:31:08.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f24003620 con 0x7f3f340721f0 2024-04-25T10:31:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2b400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f10066f60 0x7f3f10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3f2411e1b0 con 0x7f3f340721f0 2024-04-25T10:31:08.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.419+0000 7f3f2be00640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f10066f60 0x7f3f10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.423+0000 7f3f2be00640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f10066f60 0x7f3f10069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3f340aea60 tx=0x7f3f1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.423+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f240a9070 con 0x7f3f340721f0 2024-04-25T10:31:08.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.583+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f3404ef30 con 0x7f3f340721f0 2024-04-25T10:31:08.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.583+0000 7f3f2b400640 1 -- 172.21.15.45:0/911488632 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f24016070 con 0x7f3f340721f0 2024-04-25T10:31:08.587 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:31:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f10066f60 msgr2=0x7f3f10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f10066f60 0x7f3f10069420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3f340aea60 tx=0x7f3f1c009040 comp rx=0 tx=0).stop 2024-04-25T10:31:08.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 msgr2=0x7f3f3413eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 0x7f3f3413eee0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f3f24000c10 tx=0x7f3f24019220 comp rx=0 tx=0).stop 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 shutdown_connections 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f10066f60 0x7f3f10069420 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34159540 0x7f3f3415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f340aebd0 0x7f3f341417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 --2- 172.21.15.45:0/911488632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f340721f0 0x7f3f3413eee0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 >> 172.21.15.45:0/911488632 conn(0x7f3f340669a0 msgr2=0x7f3f340cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 shutdown_connections 2024-04-25T10:31:08.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.587+0000 7f3f3a400640 1 -- 172.21.15.45:0/911488632 wait complete. 2024-04-25T10:31:08.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:08.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef8200640 1 Processor -- start 2024-04-25T10:31:08.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef8200640 1 -- start start 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef8200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ef0066c00 0x7f6ef0161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef8200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef00b40c0 con 0x7f6ef0066fe0 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef8200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef00b5b00 con 0x7f6ef00b5560 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef8200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef0161d60 con 0x7f6ef0066c00 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef4e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6ef0066fe0 0x7f6ef0072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51410/0 (socket says 172.21.15.45:51410) 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef4e00640 1 -- 172.21.15.45:0/2393402636 learned_addr learned my addr 172.21.15.45:0/2393402636 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:08.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef5800640 1 --2- 172.21.15.45:0/2393402636 >> v2:172.21.15.117:3300/0 conn(0x7f6ef0066c00 0x7f6ef0161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6eefe00640 1 -- 172.21.15.45:0/2393402636 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711769429 0 0) 0x7f6ef00b40c0 con 0x7f6ef0066fe0 2024-04-25T10:31:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6eefe00640 1 -- 172.21.15.45:0/2393402636 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed8003530 con 0x7f6ef0066fe0 2024-04-25T10:31:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6eefe00640 1 -- 172.21.15.45:0/2393402636 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890170735 0 0) 0x7f6ef00b5b00 con 0x7f6ef00b5560 2024-04-25T10:31:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef5800640 1 -- 172.21.15.45:0/2393402636 >> v1:172.21.15.117:6789/0 conn(0x7f6ef00b5560 legacy=0x7f6ef00ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef5800640 1 -- 172.21.15.45:0/2393402636 >> v1:172.21.15.45:6789/0 conn(0x7f6ef0066fe0 legacy=0x7f6ef0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.751+0000 7f6ef5800640 1 -- 172.21.15.45:0/2393402636 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef00b3a00 con 0x7f6ef0066c00 2024-04-25T10:31:08.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef5800640 1 --2- 172.21.15.45:0/2393402636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0066c00 0x7f6ef0161820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40097c0 tx=0x7f6ee401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968a2d07b8fe8246 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6eefe00640 1 -- 172.21.15.45:0/2393402636 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40b0070 con 0x7f6ef0066c00 2024-04-25T10:31:08.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6eefe00640 1 -- 172.21.15.45:0/2393402636 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee40ab600 con 0x7f6ef0066c00 2024-04-25T10:31:08.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6eefe00640 1 -- 172.21.15.45:0/2393402636 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40ab9b0 con 0x7f6ef0066c00 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 -- 172.21.15.45:0/2393402636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0066c00 msgr2=0x7f6ef0161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 --2- 172.21.15.45:0/2393402636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0066c00 0x7f6ef0161820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40097c0 tx=0x7f6ee401c380 comp rx=0 tx=0).stop 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 -- 172.21.15.45:0/2393402636 shutdown_connections 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 --2- 172.21.15.45:0/2393402636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0066c00 0x7f6ef0161820 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 -- 172.21.15.45:0/2393402636 >> 172.21.15.45:0/2393402636 conn(0x7f6ef00b2560 msgr2=0x7f6ef00b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 -- 172.21.15.45:0/2393402636 shutdown_connections 2024-04-25T10:31:08.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.755+0000 7f6ef8200640 1 -- 172.21.15.45:0/2393402636 wait complete. 2024-04-25T10:31:08.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 Processor -- start 2024-04-25T10:31:08.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- start start 2024-04-25T10:31:08.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 0x7f6ef0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef0066fe0 0x7f6ef0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef00b5560 0x7f6ef0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef015e540 con 0x7f6ef0066c00 2024-04-25T10:31:08.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef01585a0 con 0x7f6ef00b5560 2024-04-25T10:31:08.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef015e6c0 con 0x7f6ef0066fe0 2024-04-25T10:31:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 0x7f6ef0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 0x7f6ef0152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35248/0 (socket says 172.21.15.45:35248) 2024-04-25T10:31:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 -- 172.21.15.45:0/1433129598 learned_addr learned my addr 172.21.15.45:0/1433129598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef6200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef00b5560 0x7f6ef0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef0066fe0 0x7f6ef0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 -- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef0066fe0 msgr2=0x7f6ef0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef0066fe0 0x7f6ef0155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 -- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef00b5560 msgr2=0x7f6ef0155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef00b5560 0x7f6ef0155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 -- 172.21.15.45:0/1433129598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef0150650 con 0x7f6ef0066c00 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef0066fe0 0x7f6ef0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef5800640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 0x7f6ef0152670 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40ae040 tx=0x7f6ee401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40b0050 con 0x7f6ef0066c00 2024-04-25T10:31:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee40b1040 con 0x7f6ef0066c00 2024-04-25T10:31:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef0155db0 con 0x7f6ef0066c00 2024-04-25T10:31:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40b5810 con 0x7f6ef0066c00 2024-04-25T10:31:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef0150bf0 con 0x7f6ef0066c00 2024-04-25T10:31:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.759+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef004ef30 con 0x7f6ef0066c00 2024-04-25T10:31:08.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.763+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ee40be030 con 0x7f6ef0066c00 2024-04-25T10:31:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.763+0000 7f6eed600640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ed0066f60 0x7f6ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.763+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6ee411eff0 con 0x7f6ef0066c00 2024-04-25T10:31:08.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.763+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ed0066f60 0x7f6ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:08.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.763+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ed0066f60 0x7f6ed0069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6ef01536d0 tx=0x7f6edc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:08.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.767+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ee40c5020 con 0x7f6ef0066c00 2024-04-25T10:31:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.927+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ef01521f0 con 0x7f6ef0066c00 2024-04-25T10:31:08.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.927+0000 7f6eed600640 1 -- 172.21.15.45:0/1433129598 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ee4016090 con 0x7f6ef0066c00 2024-04-25T10:31:08.929 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:31:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.927+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ed0066f60 msgr2=0x7f6ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.927+0000 7f6ef8200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ed0066f60 0x7f6ed0069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6ef01536d0 tx=0x7f6edc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.927+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 msgr2=0x7f6ef0152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.927+0000 7f6ef8200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 0x7f6ef0152670 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40ae040 tx=0x7f6ee401abc0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 shutdown_connections 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6ed0066f60 0x7f6ed0069420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef00b5560 0x7f6ef0155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef0066fe0 0x7f6ef0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 --2- 172.21.15.45:0/1433129598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0066c00 0x7f6ef0152670 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 >> 172.21.15.45:0/1433129598 conn(0x7f6ef00b2560 msgr2=0x7f6ef00adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 shutdown_connections 2024-04-25T10:31:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:08.931+0000 7f6ef8200640 1 -- 172.21.15.45:0/1433129598 wait complete. 2024-04-25T10:31:08.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.087+0000 7f7227400640 1 Processor -- start 2024-04-25T10:31:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7227400640 1 -- start start 2024-04-25T10:31:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7227400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f722015e050 0x7f72200b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7227400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72200b13e0 0x7f72200cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7227400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200b1870 con 0x7f72200d1820 2024-04-25T10:31:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7227400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72200b1af0 con 0x7f72200b13e0 2024-04-25T10:31:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7227400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200b1dd0 con 0x7f722015e050 2024-04-25T10:31:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7225400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72200b13e0 0x7f72200cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7224a00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f72200d1820 0x7f72200d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36532/0 (socket says 172.21.15.45:36532) 2024-04-25T10:31:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7224a00640 1 -- 172.21.15.45:0/2426895074 learned_addr learned my addr 172.21.15.45:0/2426895074 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f721fe00640 1 --2- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f722015e050 0x7f72200b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7225400640 1 -- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f722015e050 msgr2=0x7f72200b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7225400640 1 --2- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f722015e050 0x7f72200b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7225400640 1 -- 172.21.15.45:0/2426895074 >> v1:172.21.15.117:6789/0 conn(0x7f72200d1820 legacy=0x7f72200d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.091+0000 7f7225400640 1 -- 172.21.15.45:0/2426895074 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200b20b0 con 0x7f72200b13e0 2024-04-25T10:31:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7225400640 1 --2- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200b13e0 0x7f72200cfc70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7218009d90 tx=0x7f721801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20eaf54c3f717108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f721f400640 1 -- 172.21.15.45:0/2426895074 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180b2070 con 0x7f72200b13e0 2024-04-25T10:31:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f721f400640 1 -- 172.21.15.45:0/2426895074 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72180a59a0 con 0x7f72200b13e0 2024-04-25T10:31:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f721f400640 1 -- 172.21.15.45:0/2426895074 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180a5d70 con 0x7f72200b13e0 2024-04-25T10:31:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 -- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200b13e0 msgr2=0x7f72200cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 --2- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200b13e0 0x7f72200cfc70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7218009d90 tx=0x7f721801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 -- 172.21.15.45:0/2426895074 shutdown_connections 2024-04-25T10:31:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 --2- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200b13e0 0x7f72200cfc70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 --2- 172.21.15.45:0/2426895074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f722015e050 0x7f72200b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 -- 172.21.15.45:0/2426895074 >> 172.21.15.45:0/2426895074 conn(0x7f722009a9d0 msgr2=0x7f722009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 -- 172.21.15.45:0/2426895074 shutdown_connections 2024-04-25T10:31:09.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 -- 172.21.15.45:0/2426895074 wait complete. 2024-04-25T10:31:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.095+0000 7f7227400640 1 Processor -- start 2024-04-25T10:31:09.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- start start 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72200b13e0 0x7f7220151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200d1820 0x7f72200c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 0x7f72200aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200d0d90 con 0x7f722015e050 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200d0c10 con 0x7f72200d1820 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200d0f10 con 0x7f72200b13e0 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 0x7f72200aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 0x7f72200aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35258/0 (socket says 172.21.15.45:35258) 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 -- 172.21.15.45:0/2866485501 learned_addr learned my addr 172.21.15.45:0/2866485501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:09.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f721fe00640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200d1820 0x7f72200c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7224a00640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72200b13e0 0x7f7220151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 -- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72200b13e0 msgr2=0x7f7220151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72200b13e0 0x7f7220151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 -- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200d1820 msgr2=0x7f72200c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200d1820 0x7f72200c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 -- 172.21.15.45:0/2866485501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200cb810 con 0x7f722015e050 2024-04-25T10:31:09.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7225400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 0x7f72200aef70 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f72180b0040 tx=0x7f7218019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180b2050 con 0x7f722015e050 2024-04-25T10:31:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72180be070 con 0x7f722015e050 2024-04-25T10:31:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180ada90 con 0x7f722015e050 2024-04-25T10:31:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72200ca770 con 0x7f722015e050 2024-04-25T10:31:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72200cad30 con 0x7f722015e050 2024-04-25T10:31:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.099+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722004ef30 con 0x7f722015e050 2024-04-25T10:31:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.103+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72180adc30 con 0x7f722015e050 2024-04-25T10:31:09.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.103+0000 7f721cc00640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7200066f90 0x7f7200069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.103+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f72181200f0 con 0x7f722015e050 2024-04-25T10:31:09.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.103+0000 7f7224a00640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7200066f90 0x7f7200069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.103+0000 7f7224a00640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7200066f90 0x7f7200069450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7214000f70 tx=0x7f7214007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.107+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7218016560 con 0x7f722015e050 2024-04-25T10:31:09.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.303+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f722006fc40 con 0x7f722015e050 2024-04-25T10:31:09.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f721cc00640 1 -- 172.21.15.45:0/2866485501 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7f72180f0990 con 0x7f722015e050 2024-04-25T10:31:09.307 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:09.307 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:31:09.308 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:31:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7200066f90 msgr2=0x7f7200069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7200066f90 0x7f7200069450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7214000f70 tx=0x7f7214007e90 comp rx=0 tx=0).stop 2024-04-25T10:31:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 msgr2=0x7f72200aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 0x7f72200aef70 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f72180b0040 tx=0x7f7218019680 comp rx=0 tx=0).stop 2024-04-25T10:31:09.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 shutdown_connections 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7200066f90 0x7f7200069450 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f722015e050 0x7f72200aef70 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72200d1820 0x7f72200c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 --2- 172.21.15.45:0/2866485501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72200b13e0 0x7f7220151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 >> 172.21.15.45:0/2866485501 conn(0x7f722009a9d0 msgr2=0x7f7220154800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 shutdown_connections 2024-04-25T10:31:09.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.307+0000 7f7227400640 1 -- 172.21.15.45:0/2866485501 wait complete. 2024-04-25T10:31:09.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.467+0000 7f0be7600640 1 Processor -- start 2024-04-25T10:31:09.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.467+0000 7f0be7600640 1 -- start start 2024-04-25T10:31:09.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be7600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0be00cda10 0x7f0be006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be7600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0be00cf930 0x7f0be00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be7600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be00b2770 con 0x7f0be015bc40 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be7600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be006c670 con 0x7f0be00cda10 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be7600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be006c950 con 0x7f0be00cf930 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be4c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0be015bc40 0x7f0be00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51418/0 (socket says 172.21.15.45:51418) 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be4c00640 1 -- 172.21.15.45:0/1989765105 learned_addr learned my addr 172.21.15.45:0/1989765105 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdfe00640 1 --2- 172.21.15.45:0/1989765105 >> v2:172.21.15.45:3300/0 conn(0x7f0be00cda10 0x7f0be006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0be5600640 1 --2- 172.21.15.45:0/1989765105 >> v2:172.21.15.117:3300/0 conn(0x7f0be00cf930 0x7f0be00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdf400640 1 -- 172.21.15.45:0/1989765105 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96683115 0 0) 0x7f0be00b2770 con 0x7f0be015bc40 2024-04-25T10:31:09.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdf400640 1 -- 172.21.15.45:0/1989765105 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc8003530 con 0x7f0be015bc40 2024-04-25T10:31:09.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdfe00640 1 -- 172.21.15.45:0/1989765105 >> v2:172.21.15.117:3300/0 conn(0x7f0be00cf930 msgr2=0x7f0be00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdfe00640 1 --2- 172.21.15.45:0/1989765105 >> v2:172.21.15.117:3300/0 conn(0x7f0be00cf930 0x7f0be00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdfe00640 1 -- 172.21.15.45:0/1989765105 >> v1:172.21.15.45:6789/0 conn(0x7f0be015bc40 legacy=0x7f0be00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdfe00640 1 -- 172.21.15.45:0/1989765105 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be006cc30 con 0x7f0be00cda10 2024-04-25T10:31:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.471+0000 7f0bdfe00640 1 --2- 172.21.15.45:0/1989765105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be006c090 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f0be00ae4c0 tx=0x7f0bcc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13df0d11e22a533d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0bdf400640 1 -- 172.21.15.45:0/1989765105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0b0070 con 0x7f0be00cda10 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0bdf400640 1 -- 172.21.15.45:0/1989765105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc0ab600 con 0x7f0be00cda10 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0bdf400640 1 -- 172.21.15.45:0/1989765105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0ab9d0 con 0x7f0be00cda10 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- 172.21.15.45:0/1989765105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 msgr2=0x7f0be006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 --2- 172.21.15.45:0/1989765105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be006c090 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f0be00ae4c0 tx=0x7f0bcc01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- 172.21.15.45:0/1989765105 shutdown_connections 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 --2- 172.21.15.45:0/1989765105 >> v2:172.21.15.117:3300/0 conn(0x7f0be00cf930 0x7f0be00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 --2- 172.21.15.45:0/1989765105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be006c090 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- 172.21.15.45:0/1989765105 >> 172.21.15.45:0/1989765105 conn(0x7f0be0066b70 msgr2=0x7f0be0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- 172.21.15.45:0/1989765105 shutdown_connections 2024-04-25T10:31:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- 172.21.15.45:0/1989765105 wait complete. 2024-04-25T10:31:09.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 Processor -- start 2024-04-25T10:31:09.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- start start 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be0150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be00cf930 0x7f0be014a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be015bc40 0x7f0be01568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00b3850 con 0x7f0be00cda10 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00b36d0 con 0x7f0be015bc40 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be7600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00b39d0 con 0x7f0be00cf930 2024-04-25T10:31:09.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be0150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be0150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35272/0 (socket says 172.21.15.45:35272) 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.475+0000 7f0be4c00640 1 -- 172.21.15.45:0/3602370768 learned_addr learned my addr 172.21.15.45:0/3602370768 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be5600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be015bc40 0x7f0be01568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdfe00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be00cf930 0x7f0be014a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be4c00640 1 -- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be00cf930 msgr2=0x7f0be014a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be4c00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be00cf930 0x7f0be014a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be4c00640 1 -- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be015bc40 msgr2=0x7f0be01568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be4c00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be015bc40 0x7f0be01568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be4c00640 1 -- 172.21.15.45:0/3602370768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be00b08d0 con 0x7f0be00cda10 2024-04-25T10:31:09.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be4c00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be0150120 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400c3f0 tx=0x7f0bd400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4017070 con 0x7f0be00cda10 2024-04-25T10:31:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd4005cf0 con 0x7f0be00cda10 2024-04-25T10:31:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40072f0 con 0x7f0be00cda10 2024-04-25T10:31:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be0150660 con 0x7f0be00cda10 2024-04-25T10:31:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be0070720 con 0x7f0be00cda10 2024-04-25T10:31:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be01573e0 con 0x7f0be00cda10 2024-04-25T10:31:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bd401a070 con 0x7f0be00cda10 2024-04-25T10:31:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdcc00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bc0066f60 0x7f0bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.479+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0bd4084500 con 0x7f0be00cda10 2024-04-25T10:31:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.483+0000 7f0bdfe00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bc0066f60 0x7f0bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.483+0000 7f0bdfe00640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bc0066f60 0x7f0bc0069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0ad000 tx=0x7f0bcc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.483+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0be01573e0 con 0x7f0be00cda10 2024-04-25T10:31:09.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.647+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be00af1c0 con 0x7f0be00cda10 2024-04-25T10:31:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.647+0000 7f0bdcc00640 1 -- 172.21.15.45:0/3602370768 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f0bd4055480 con 0x7f0be00cda10 2024-04-25T10:31:09.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:09.650 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":171,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:45.264415+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":7,"to":4}]},{"pgid":"25.20","mappings":[{"from":1,"to":4}]},{"pgid":"25.21","mappings":[{"from":7,"to":3}]},{"pgid":"25.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:09.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bc0066f60 msgr2=0x7f0bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bc0066f60 0x7f0bc0069420 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0ad000 tx=0x7f0bcc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 msgr2=0x7f0be0150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be0150120 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400c3f0 tx=0x7f0bd400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 shutdown_connections 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0bc0066f60 0x7f0bc0069420 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0be015bc40 0x7f0be01568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0be00cf930 0x7f0be014a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 --2- 172.21.15.45:0/3602370768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0be00cda10 0x7f0be0150120 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 >> 172.21.15.45:0/3602370768 conn(0x7f0be0066b70 msgr2=0x7f0be00cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 shutdown_connections 2024-04-25T10:31:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.651+0000 7f0be7600640 1 -- 172.21.15.45:0/3602370768 wait complete. 2024-04-25T10:31:09.672 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:31:09.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.811+0000 7ffb98a00640 1 Processor -- start 2024-04-25T10:31:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.811+0000 7ffb98a00640 1 -- start start 2024-04-25T10:31:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb98a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 0x7ffb940cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb98a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9415f3b0 con 0x7ffb940b0f70 2024-04-25T10:31:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb98a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb940ca810 con 0x7ffb940cfd90 2024-04-25T10:31:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb98a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415ef20 con 0x7ffb940cd2e0 2024-04-25T10:31:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffb940b0f70 0x7ffb940b1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51420/0 (socket says 172.21.15.45:51420) 2024-04-25T10:31:09.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91e00640 1 -- 172.21.15.45:0/3482900407 learned_addr learned my addr 172.21.15.45:0/3482900407 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb92800640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 0x7ffb940cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224505389 0 0) 0x7ffb9415f3b0 con 0x7ffb940b0f70 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb78003530 con 0x7ffb940b0f70 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2994528218 0 0) 0x7ffb78003530 con 0x7ffb940b0f70 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 msgr2=0x7ffb940cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 0x7ffb940cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 >> v1:172.21.15.117:6789/0 conn(0x7ffb940cfd90 legacy=0x7ffb940ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9415e5f0 con 0x7ffb940b0f70 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7ffb84001680 con 0x7ffb940b0f70 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 >> v1:172.21.15.45:6789/0 conn(0x7ffb940b0f70 legacy=0x7ffb940b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780039f0 0x7ffb78003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 0x7ffb780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415ef20 con 0x7ffb78004810 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800ba80 con 0x7ffb780039f0 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800bc80 con 0x7ffb940cd2e0 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb89400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb91e00640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780039f0 0x7ffb78003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb92800640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 0x7ffb780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb89400640 1 -- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 msgr2=0x7ffb780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb89400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb940cd2e0 0x7ffb780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb89400640 1 -- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780039f0 msgr2=0x7ffb78003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb89400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb780039f0 0x7ffb78003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.815+0000 7ffb89400640 1 -- 172.21.15.45:0/3482900407 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78003530 con 0x7ffb78004810 2024-04-25T10:31:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb89400640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c001750 tx=0x7ffb7c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8b0dcd9c42d8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0013a0 con 0x7ffb78004810 2024-04-25T10:31:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0052f0 con 0x7ffb78004810 2024-04-25T10:31:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb91400640 1 -- 172.21.15.45:0/3482900407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003840 con 0x7ffb78004810 2024-04-25T10:31:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb98a00640 1 -- 172.21.15.45:0/3482900407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 msgr2=0x7ffb78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb98a00640 1 --2- 172.21.15.45:0/3482900407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c001750 tx=0x7ffb7c01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb92800640 1 -- 172.21.15.45:0/3482900407 reap_dead start 2024-04-25T10:31:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb98a00640 1 -- 172.21.15.45:0/3482900407 shutdown_connections 2024-04-25T10:31:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb98a00640 1 -- 172.21.15.45:0/3482900407 >> 172.21.15.45:0/3482900407 conn(0x7ffb940b1d50 msgr2=0x7ffb940b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb98a00640 1 -- 172.21.15.45:0/3482900407 shutdown_connections 2024-04-25T10:31:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.819+0000 7ffb98a00640 1 -- 172.21.15.45:0/3482900407 wait complete. 2024-04-25T10:31:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 Processor -- start 2024-04-25T10:31:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- start start 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb940c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb94158230 0x7ffb941563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb940c7bc0 0x7ffb940c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940cba20 con 0x7ffb78004810 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415ebb0 con 0x7ffb940c7bc0 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940cbba0 con 0x7ffb94158230 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb940c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb940c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35294/0 (socket says 172.21.15.45:35294) 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 -- 172.21.15.45:0/4064733034 learned_addr learned my addr 172.21.15.45:0/4064733034 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:09.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb92800640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb940c7bc0 0x7ffb940c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb89400640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb94158230 0x7ffb941563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 -- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb94158230 msgr2=0x7ffb941563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb94158230 0x7ffb941563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 -- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb940c7bc0 msgr2=0x7ffb940c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb940c7bc0 0x7ffb940c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 -- 172.21.15.45:0/4064733034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb940c51b0 con 0x7ffb78004810 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb91e00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb940c4c70 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7ffb840017f0 tx=0x7ffb840129a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8401c070 con 0x7ffb78004810 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84014750 con 0x7ffb78004810 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb941579c0 con 0x7ffb78004810 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84014b00 con 0x7ffb78004810 2024-04-25T10:31:09.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb940c73c0 con 0x7ffb78004810 2024-04-25T10:31:09.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.823+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9404ef30 con 0x7ffb78004810 2024-04-25T10:31:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.827+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb8401f070 con 0x7ffb78004810 2024-04-25T10:31:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.827+0000 7ffb8aa00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.827+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ffb84087aa0 con 0x7ffb78004810 2024-04-25T10:31:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.827+0000 7ffb89400640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.827+0000 7ffb89400640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb70066f60 0x7ffb70069420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ffb78004790 tx=0x7ffb7c0011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:09.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:09.831+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb840583d0 con 0x7ffb78004810 2024-04-25T10:31:10.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.023+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb940cda20 con 0x7ffb78004810 2024-04-25T10:31:10.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.023+0000 7ffb8aa00640 1 -- 172.21.15.45:0/4064733034 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7ffb8405c470 con 0x7ffb78004810 2024-04-25T10:31:10.027 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:10.027 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:31:10.027 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb70066f60 msgr2=0x7ffb70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb70066f60 0x7ffb70069420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ffb78004790 tx=0x7ffb7c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 msgr2=0x7ffb940c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb940c4c70 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7ffb840017f0 tx=0x7ffb840129a0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 shutdown_connections 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb940c7bc0 0x7ffb940c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb94158230 0x7ffb941563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 --2- 172.21.15.45:0/4064733034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb78004810 0x7ffb940c4c70 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 >> 172.21.15.45:0/4064733034 conn(0x7ffb940b1d50 msgr2=0x7ffb9406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 shutdown_connections 2024-04-25T10:31:10.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.027+0000 7ffb98a00640 1 -- 172.21.15.45:0/4064733034 wait complete. 2024-04-25T10:31:10.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:10.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 Processor -- start 2024-04-25T10:31:10.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 -- start start 2024-04-25T10:31:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c15dce0 0x7f934c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f934c0ad850 0x7f934c0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c06e6a0 con 0x7f934c06eec0 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c0674d0 con 0x7f934c0ad850 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9353e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c15e0c0 con 0x7f934c15dce0 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f934c0ad850 0x7f934c0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f934c0ad850 0x7f934c0adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35300/0 (socket says 172.21.15.45:35300) 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9350a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f934c06eec0 0x7f934c06df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51434/0 (socket says 172.21.15.45:51434) 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 -- 172.21.15.45:0/4016677549 learned_addr learned my addr 172.21.15.45:0/4016677549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351400640 1 --2- 172.21.15.45:0/4016677549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c15dce0 0x7f934c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 -- 172.21.15.45:0/4016677549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c15dce0 msgr2=0x7f934c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 --2- 172.21.15.45:0/4016677549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c15dce0 0x7f934c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 -- 172.21.15.45:0/4016677549 >> v1:172.21.15.45:6789/0 conn(0x7f934c06eec0 legacy=0x7f934c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f9351e00640 1 -- 172.21.15.45:0/4016677549 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0671f0 con 0x7f934c0ad850 2024-04-25T10:31:10.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.187+0000 7f934be00640 1 -- 172.21.15.45:0/4016677549 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379513711 0 0) 0x7f934c06e6a0 con 0x7f934c06eec0 2024-04-25T10:31:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9351e00640 1 --2- 172.21.15.45:0/4016677549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0ad850 0x7f934c0adca0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f9344009e50 tx=0x7f934401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2cef3a8f6b86b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f934be00640 1 -- 172.21.15.45:0/4016677549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440a5750 con 0x7f934c0ad850 2024-04-25T10:31:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f934be00640 1 -- 172.21.15.45:0/4016677549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93440a58f0 con 0x7f934c0ad850 2024-04-25T10:31:10.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f934be00640 1 -- 172.21.15.45:0/4016677549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440a5cc0 con 0x7f934c0ad850 2024-04-25T10:31:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 -- 172.21.15.45:0/4016677549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0ad850 msgr2=0x7f934c0adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 --2- 172.21.15.45:0/4016677549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0ad850 0x7f934c0adca0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f9344009e50 tx=0x7f934401c9a0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 -- 172.21.15.45:0/4016677549 shutdown_connections 2024-04-25T10:31:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 --2- 172.21.15.45:0/4016677549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0ad850 0x7f934c0adca0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 --2- 172.21.15.45:0/4016677549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c15dce0 0x7f934c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 -- 172.21.15.45:0/4016677549 >> 172.21.15.45:0/4016677549 conn(0x7f934c0b2410 msgr2=0x7f934c0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 -- 172.21.15.45:0/4016677549 shutdown_connections 2024-04-25T10:31:10.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.191+0000 7f9353e00640 1 -- 172.21.15.45:0/4016677549 wait complete. 2024-04-25T10:31:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 Processor -- start 2024-04-25T10:31:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 -- start start 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934c06eec0 0x7f934c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c0ad850 0x7f934c147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 0x7f934c0c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0d0320 con 0x7f934c15dce0 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c066ee0 con 0x7f934c06eec0 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0d04a0 con 0x7f934c0ad850 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 0x7f934c0c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 0x7f934c0c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35314/0 (socket says 172.21.15.45:35314) 2024-04-25T10:31:10.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 -- 172.21.15.45:0/497403453 learned_addr learned my addr 172.21.15.45:0/497403453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351400640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934c06eec0 0x7f934c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9350a00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c0ad850 0x7f934c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 -- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c0ad850 msgr2=0x7f934c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c0ad850 0x7f934c147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 -- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934c06eec0 msgr2=0x7f934c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934c06eec0 0x7f934c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 -- 172.21.15.45:0/497403453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c147c80 con 0x7f934c15dce0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9350a00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c0ad850 0x7f934c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9351e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 0x7f934c0c2f40 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f93440af040 tx=0x7f9344019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440bc020 con 0x7f934c15dce0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93440b8670 con 0x7f934c15dce0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c0c3480 con 0x7f934c15dce0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440b8a60 con 0x7f934c15dce0 2024-04-25T10:31:10.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.195+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c1481e0 con 0x7f934c15dce0 2024-04-25T10:31:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.199+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c04ef30 con 0x7f934c15dce0 2024-04-25T10:31:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.199+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93440bf020 con 0x7f934c15dce0 2024-04-25T10:31:10.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.199+0000 7f9349600640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f932c066f60 0x7f932c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.199+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f934411fff0 con 0x7f934c15dce0 2024-04-25T10:31:10.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.199+0000 7f9351400640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f932c066f60 0x7f932c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.199+0000 7f9351400640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f932c066f60 0x7f932c069420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ac0 tx=0x7f9340009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.203+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93440c6030 con 0x7f934c15dce0 2024-04-25T10:31:10.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.399+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934c071630 con 0x7f934c15dce0 2024-04-25T10:31:10.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.399+0000 7f9349600640 1 -- 172.21.15.45:0/497403453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4958 (secure 0 0 0) 0x7f934c147c80 con 0x7f934c15dce0 2024-04-25T10:31:10.505 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:10.505 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":122,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:30:24.574396+0000","modified":"2024-04-25T10:30:27.602192+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10467":{"gid":10467,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6837/1582038965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":1582038965},{"type":"v1","addr":"172.21.15.117:6837","nonce":1582038965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 122 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f932c066f60 msgr2=0x7f932c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f932c066f60 0x7f932c069420 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9340009ac0 tx=0x7f9340009190 comp rx=0 tx=0).stop 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 msgr2=0x7f934c0c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 0x7f934c0c2f40 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f93440af040 tx=0x7f9344019740 comp rx=0 tx=0).stop 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 shutdown_connections 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f932c066f60 0x7f932c069420 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c15dce0 0x7f934c0c2f40 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f934c0ad850 0x7f934c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 --2- 172.21.15.45:0/497403453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f934c06eec0 0x7f934c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 >> 172.21.15.45:0/497403453 conn(0x7f934c0b2410 msgr2=0x7f934c0ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 shutdown_connections 2024-04-25T10:31:10.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.403+0000 7f9353e00640 1 -- 172.21.15.45:0/497403453 wait complete. 2024-04-25T10:31:10.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 Processor -- start 2024-04-25T10:31:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 -- start start 2024-04-25T10:31:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24980b0f70 0x7f24980b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cd2e0 0x7f24980cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249815f3b0 con 0x7f24980cfd90 2024-04-25T10:31:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249815ef20 con 0x7f24980b0f70 2024-04-25T10:31:10.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f249da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24980ca810 con 0x7f24980cd2e0 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f2496a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24980b0f70 0x7f24980b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f2497400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cd2e0 0x7f24980cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f248fe00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f24980cfd90 0x7f24980ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36562/0 (socket says 172.21.15.45:36562) 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f248fe00640 1 -- 172.21.15.45:0/4094101272 learned_addr learned my addr 172.21.15.45:0/4094101272 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.651+0000 7f2496a00640 1 -- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cd2e0 msgr2=0x7f24980cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496a00640 1 --2- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cd2e0 0x7f24980cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496a00640 1 -- 172.21.15.45:0/4094101272 >> v1:172.21.15.117:6789/0 conn(0x7f24980cfd90 legacy=0x7f24980ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496000640 1 -- 172.21.15.45:0/4094101272 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880092161 0 0) 0x7f249815f3b0 con 0x7f24980cfd90 2024-04-25T10:31:10.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496a00640 1 -- 172.21.15.45:0/4094101272 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980b1890 con 0x7f24980b0f70 2024-04-25T10:31:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496a00640 1 --2- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980b0f70 0x7f24980b1350 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24880097c0 tx=0x7f248801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f92154d81210e01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496000640 1 -- 172.21.15.45:0/4094101272 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880b0070 con 0x7f24980b0f70 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496000640 1 -- 172.21.15.45:0/4094101272 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24880ab600 con 0x7f24980b0f70 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f2496000640 1 -- 172.21.15.45:0/4094101272 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880ab9d0 con 0x7f24980b0f70 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 -- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980b0f70 msgr2=0x7f24980b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 --2- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980b0f70 0x7f24980b1350 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f24880097c0 tx=0x7f248801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 -- 172.21.15.45:0/4094101272 shutdown_connections 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 --2- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cd2e0 0x7f24980cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 --2- 172.21.15.45:0/4094101272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980b0f70 0x7f24980b1350 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 -- 172.21.15.45:0/4094101272 >> 172.21.15.45:0/4094101272 conn(0x7f24980b1d50 msgr2=0x7f24980b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 -- 172.21.15.45:0/4094101272 shutdown_connections 2024-04-25T10:31:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.655+0000 7f249da00640 1 -- 172.21.15.45:0/4094101272 wait complete. 2024-04-25T10:31:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 Processor -- start 2024-04-25T10:31:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 -- start start 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 0x7f2498155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980cd2e0 0x7f2498072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cfd90 0x7f2498156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498161b50 con 0x7f24980b0f70 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981619d0 con 0x7f24980cd2e0 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2498161cd0 con 0x7f24980cfd90 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 0x7f2498155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 0x7f2498155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35318/0 (socket says 172.21.15.45:35318) 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 -- 172.21.15.45:0/991792566 learned_addr learned my addr 172.21.15.45:0/991792566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f248fe00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980cd2e0 0x7f2498072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2497400640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cfd90 0x7f2498156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 -- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cfd90 msgr2=0x7f2498156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cfd90 0x7f2498156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 -- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980cd2e0 msgr2=0x7f2498072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980cd2e0 0x7f2498072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 -- 172.21.15.45:0/991792566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498154cc0 con 0x7f24980b0f70 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f2496a00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 0x7f2498155610 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f24880097c0 tx=0x7f2488019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880ba070 con 0x7f24980b0f70 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24880b43f0 con 0x7f24980b0f70 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880b0040 con 0x7f24980b0f70 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24980ad160 con 0x7f24980b0f70 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.659+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24980ad400 con 0x7f24980b0f70 2024-04-25T10:31:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.663+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249804ef30 con 0x7f24980b0f70 2024-04-25T10:31:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.663+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24880b47c0 con 0x7f24980b0f70 2024-04-25T10:31:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.663+0000 7f248ea00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2474066f60 0x7f2474069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.663+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f248811e2c0 con 0x7f24980b0f70 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.663+0000 7f248fe00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2474066f60 0x7f2474069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.663+0000 7f248fe00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2474066f60 0x7f2474069420 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2498155b50 tx=0x7f2480007c10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:10.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.667+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2488016090 con 0x7f24980b0f70 2024-04-25T10:31:10.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.835+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24980ce450 con 0x7f24980b0f70 2024-04-25T10:31:10.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.835+0000 7f248ea00640 1 -- 172.21.15.45:0/991792566 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f24880ef240 con 0x7f24980b0f70 2024-04-25T10:31:10.838 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:10.838 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":171,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:30:45.264415+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:30:22.368848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-04-25T10:30:23.583569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.12","mappings":[{"from":7,"to":4}]},{"pgid":"25.20","mappings":[{"from":1,"to":4}]},{"pgid":"25.21","mappings":[{"from":7,"to":3}]},{"pgid":"25.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2474066f60 msgr2=0x7f2474069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2474066f60 0x7f2474069420 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2498155b50 tx=0x7f2480007c10 comp rx=0 tx=0).stop 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 msgr2=0x7f2498155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 0x7f2498155610 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f24880097c0 tx=0x7f2488019240 comp rx=0 tx=0).stop 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 shutdown_connections 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2474066f60 0x7f2474069420 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24980cfd90 0x7f2498156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24980cd2e0 0x7f2498072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 --2- 172.21.15.45:0/991792566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24980b0f70 0x7f2498155610 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 >> 172.21.15.45:0/991792566 conn(0x7f24980b1d50 msgr2=0x7f2498160200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 shutdown_connections 2024-04-25T10:31:10.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.839+0000 7f249da00640 1 -- 172.21.15.45:0/991792566 wait complete. 2024-04-25T10:31:10.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:31:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 Processor -- start 2024-04-25T10:31:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 -- start start 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3aa415a0e0 0x7f3aa415b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 0x7f3aa40b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa415b6e0 con 0x7f3aa40b2fc0 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa4070ed0 con 0x7f3aa415a0e0 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa9e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40711b0 con 0x7f3aa4071f70 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa3400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3aa40b2fc0 0x7f3aa406c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51448/0 (socket says 172.21.15.45:51448) 2024-04-25T10:31:11.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:10.999+0000 7f3aa3400640 1 -- 172.21.15.45:0/2465624608 learned_addr learned my addr 172.21.15.45:0/2465624608 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2a00640 1 --2- 172.21.15.45:0/2465624608 >> v2:172.21.15.117:3300/0 conn(0x7f3aa415a0e0 0x7f3aa415b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3a9a000640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 0x7f3aa40b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839248841 0 0) 0x7f3a9c0014a0 con 0x7f3aa40b2fc0 2024-04-25T10:31:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a88003450 con 0x7f3aa40b2fc0 2024-04-25T10:31:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 588079081 0 0) 0x7f3a88003450 con 0x7f3aa40b2fc0 2024-04-25T10:31:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 msgr2=0x7f3aa40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 0x7f3aa40b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 >> v2:172.21.15.117:3300/0 conn(0x7f3aa415a0e0 msgr2=0x7f3aa415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 --2- 172.21.15.45:0/2465624608 >> v2:172.21.15.117:3300/0 conn(0x7f3aa415a0e0 0x7f3aa415b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4071490 con 0x7f3aa40b2fc0 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f3a9c001bd0 con 0x7f3aa40b2fc0 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2a00640 1 --2- 172.21.15.45:0/2465624608 >> v2:172.21.15.117:3300/0 conn(0x7f3aa415a0e0 0x7f3aa415b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 >> v1:172.21.15.45:6789/0 conn(0x7f3aa40b2fc0 legacy=0x7f3aa406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a88003ab0 0x7f3a88003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 0x7f3a8800a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8800ae40 0x7f3a8800d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4070ed0 con 0x7f3a8800ae40 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40711b0 con 0x7f3a88003ab0 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8800db30 con 0x7f3aa4071f70 2024-04-25T10:31:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa3400640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8800ae40 0x7f3a8800d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3a9a000640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 0x7f3a8800a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa2a00640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a88003ab0 0x7f3a88003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa3400640 1 -- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 msgr2=0x7f3a8800a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa3400640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3aa4071f70 0x7f3a8800a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa3400640 1 -- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a88003ab0 msgr2=0x7f3a88003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa3400640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a88003ab0 0x7f3a88003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.003+0000 7f3aa3400640 1 -- 172.21.15.45:0/2465624608 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88003450 con 0x7f3a8800ae40 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa3400640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8800ae40 0x7f3a8800d2d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00aa70 tx=0x7f3a9c01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e40ba639e1fc33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00a690 con 0x7f3a8800ae40 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0a9070 con 0x7f3a8800ae40 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa2000640 1 -- 172.21.15.45:0/2465624608 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0ae8d0 con 0x7f3a8800ae40 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 -- 172.21.15.45:0/2465624608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8800ae40 msgr2=0x7f3a8800d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/2465624608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8800ae40 0x7f3a8800d2d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00aa70 tx=0x7f3a9c01d3c0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa3400640 1 -- 172.21.15.45:0/2465624608 reap_dead start 2024-04-25T10:31:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 -- 172.21.15.45:0/2465624608 shutdown_connections 2024-04-25T10:31:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 -- 172.21.15.45:0/2465624608 >> 172.21.15.45:0/2465624608 conn(0x7f3aa4066b70 msgr2=0x7f3aa4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 -- 172.21.15.45:0/2465624608 shutdown_connections 2024-04-25T10:31:11.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 -- 172.21.15.45:0/2465624608 wait complete. 2024-04-25T10:31:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 Processor -- start 2024-04-25T10:31:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 -- start start 2024-04-25T10:31:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.007+0000 7f3aa9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a8800ae40 0x7f3aa414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa41587d0 0x7f3aa4158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 0x7f3aa4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40af030 con 0x7f3aa41505e0 2024-04-25T10:31:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4071a50 con 0x7f3aa41587d0 2024-04-25T10:31:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa415a4c0 con 0x7f3a8800ae40 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 0x7f3aa4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 0x7f3aa4150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35340/0 (socket says 172.21.15.45:35340) 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 -- 172.21.15.45:0/60524586 learned_addr learned my addr 172.21.15.45:0/60524586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3a9a000640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa41587d0 0x7f3aa4158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa2a00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a8800ae40 0x7f3aa414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 -- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a8800ae40 msgr2=0x7f3aa414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a8800ae40 0x7f3aa414e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 -- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa41587d0 msgr2=0x7f3aa4158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa41587d0 0x7f3aa4158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 -- 172.21.15.45:0/60524586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4152a50 con 0x7f3aa41505e0 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3a9a000640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa41587d0 0x7f3aa4158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa3400640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 0x7f3aa4150a90 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00aa70 tx=0x7f3a88003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0b3070 con 0x7f3aa41505e0 2024-04-25T10:31:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0ae880 con 0x7f3aa41505e0 2024-04-25T10:31:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00a690 con 0x7f3aa41505e0 2024-04-25T10:31:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa414edf0 con 0x7f3aa41505e0 2024-04-25T10:31:11.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa4152fd0 con 0x7f3aa41505e0 2024-04-25T10:31:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa4159190 con 0x7f3aa41505e0 2024-04-25T10:31:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.011+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a9c0aec30 con 0x7f3aa41505e0 2024-04-25T10:31:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.015+0000 7f3a9be00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3a80066f90 0x7f3a80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.015+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3a9c11f490 con 0x7f3aa41505e0 2024-04-25T10:31:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.015+0000 7f3aa2a00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3a80066f90 0x7f3a80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.015+0000 7f3aa2a00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3a80066f90 0x7f3a80069450 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40cf080 tx=0x7f3a94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.019+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3aa4159190 con 0x7f3aa41505e0 2024-04-25T10:31:11.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.219+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3aa41502a0 con 0x7f3aa41505e0 2024-04-25T10:31:11.221 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:31:11.219+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:31:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.219+0000 7f3a9be00640 1 -- 172.21.15.45:0/60524586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v123) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3aa415b6e0 con 0x7f3aa41505e0 2024-04-25T10:31:11.223 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:31:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3a80066f90 msgr2=0x7f3a80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3a80066f90 0x7f3a80069450 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40cf080 tx=0x7f3a94008040 comp rx=0 tx=0).stop 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 msgr2=0x7f3aa4150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 0x7f3aa4150a90 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00aa70 tx=0x7f3a88003740 comp rx=0 tx=0).stop 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 shutdown_connections 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3a80066f90 0x7f3a80069450 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3aa41505e0 0x7f3aa4150a90 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3aa41587d0 0x7f3aa4158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 --2- 172.21.15.45:0/60524586 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a8800ae40 0x7f3aa414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 >> 172.21.15.45:0/60524586 conn(0x7f3aa4066b70 msgr2=0x7f3aa4155b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 shutdown_connections 2024-04-25T10:31:11.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.223+0000 7f3aa9e00640 1 -- 172.21.15.45:0/60524586 wait complete. 2024-04-25T10:31:11.238 INFO:tasks.ceph.mds.d.smithi117.stderr: -6217> 2024-04-25T10:30:30.163+0000 7f8f4e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:31:11.243 INFO:tasks.ceph.mds.d.smithi117.stderr: -3366> 2024-04-25T10:30:33.675+0000 7f8f49200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:30:32.419579+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={9877=0-12582912@1} caps={9877=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=9877 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55d096209b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(25)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:31:11.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:31:11.289 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:31:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.387+0000 7fdcee000640 1 Processor -- start 2024-04-25T10:31:11.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.387+0000 7fdcee000640 1 -- start start 2024-04-25T10:31:11.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdcee000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdce80aef80 0x7fdce815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdcee000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdce80b31c0 0x7fdce80b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdcee000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce815b080 con 0x7fdce8070350 2024-04-25T10:31:11.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdcee000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce815adb0 con 0x7fdce80aef80 2024-04-25T10:31:11.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdcee000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce815eb60 con 0x7fdce80b31c0 2024-04-25T10:31:11.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdce80aef80 0x7fdce815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdce80aef80 0x7fdce815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35354/0 (socket says 172.21.15.45:35354) 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 -- 172.21.15.45:0/1217654833 learned_addr learned my addr 172.21.15.45:0/1217654833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7e00640 1 --2- 172.21.15.45:0/1217654833 >> v2:172.21.15.117:3300/0 conn(0x7fdce80b31c0 0x7fdce80b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 -- 172.21.15.45:0/1217654833 >> v2:172.21.15.117:3300/0 conn(0x7fdce80b31c0 msgr2=0x7fdce80b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 --2- 172.21.15.45:0/1217654833 >> v2:172.21.15.117:3300/0 conn(0x7fdce80b31c0 0x7fdce80b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 -- 172.21.15.45:0/1217654833 >> v1:172.21.15.117:6789/0 conn(0x7fdce8070350 legacy=0x7fdce815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 -- 172.21.15.45:0/1217654833 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce815ee40 con 0x7fdce80aef80 2024-04-25T10:31:11.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7e00640 1 --2- 172.21.15.45:0/1217654833 >> v2:172.21.15.117:3300/0 conn(0x7fdce80b31c0 0x7fdce80b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:11.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.391+0000 7fdce7400640 1 --2- 172.21.15.45:0/1217654833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce80aef80 0x7fdce815e480 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fdce80d0750 tx=0x7fdcd801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb7a8621c41d79b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce6a00640 1 -- 172.21.15.45:0/1217654833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80ab430 con 0x7fdce80aef80 2024-04-25T10:31:11.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce6a00640 1 -- 172.21.15.45:0/1217654833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd80b0070 con 0x7fdce80aef80 2024-04-25T10:31:11.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce6a00640 1 -- 172.21.15.45:0/1217654833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80ab7f0 con 0x7fdce80aef80 2024-04-25T10:31:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- 172.21.15.45:0/1217654833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce80aef80 msgr2=0x7fdce815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 --2- 172.21.15.45:0/1217654833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce80aef80 0x7fdce815e480 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fdce80d0750 tx=0x7fdcd801c520 comp rx=0 tx=0).stop 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- 172.21.15.45:0/1217654833 shutdown_connections 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 --2- 172.21.15.45:0/1217654833 >> v2:172.21.15.117:3300/0 conn(0x7fdce80b31c0 0x7fdce80b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 --2- 172.21.15.45:0/1217654833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce80aef80 0x7fdce815e480 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- 172.21.15.45:0/1217654833 >> 172.21.15.45:0/1217654833 conn(0x7fdce80669a0 msgr2=0x7fdce8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- 172.21.15.45:0/1217654833 shutdown_connections 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- 172.21.15.45:0/1217654833 wait complete. 2024-04-25T10:31:11.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 Processor -- start 2024-04-25T10:31:11.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- start start 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 0x7fdce80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce80aef80 0x7fdce8159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce80b31c0 0x7fdce81543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce806c250 con 0x7fdce8070350 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce806c0d0 con 0x7fdce80aef80 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdcee000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce806c3d0 con 0x7fdce80b31c0 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 0x7fdce80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 0x7fdce80b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35370/0 (socket says 172.21.15.45:35370) 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce7400640 1 -- 172.21.15.45:0/2313322104 learned_addr learned my addr 172.21.15.45:0/2313322104 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.395+0000 7fdce7e00640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce80b31c0 0x7fdce81543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdea00640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce80aef80 0x7fdce8159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdce7400640 1 -- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce80b31c0 msgr2=0x7fdce81543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdce7400640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce80b31c0 0x7fdce81543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdce7400640 1 -- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce80aef80 msgr2=0x7fdce8159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdce7400640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce80aef80 0x7fdce8159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:11.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdce7400640 1 -- 172.21.15.45:0/2313322104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80b0b30 con 0x7fdce8070350 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdea00640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce80aef80 0x7fdce8159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdce7400640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 0x7fdce80b0430 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ad040 tx=0x7fdcd801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80be5f0 con 0x7fdce8070350 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd80be790 con 0x7fdce8070350 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80beb60 con 0x7fdce8070350 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce80b0e10 con 0x7fdce8070350 2024-04-25T10:31:11.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce80b13d0 con 0x7fdce8070350 2024-04-25T10:31:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce804ef30 con 0x7fdce8070350 2024-04-25T10:31:11.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcd80b43f0 con 0x7fdce8070350 2024-04-25T10:31:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdfe00640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcc4066f60 0x7fdcc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.399+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fdcd80a8030 con 0x7fdce8070350 2024-04-25T10:31:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.403+0000 7fdcdea00640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcc4066f60 0x7fdcc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:11.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.403+0000 7fdcdea00640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcc4066f60 0x7fdcc4069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdce8157980 tx=0x7fdcd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:11.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.403+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcd8016110 con 0x7fdce8070350 2024-04-25T10:31:11.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:11.599+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdce80b2fa0 con 0x7fdce8070350 2024-04-25T10:31:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.223+0000 7fdcdfe00640 1 -- 172.21.15.45:0/2313322104 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v124) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdcd80f08d0 con 0x7fdce8070350 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcc4066f60 msgr2=0x7fdcc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcc4066f60 0x7fdcc4069420 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdce8157980 tx=0x7fdcd0009040 comp rx=0 tx=0).stop 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 msgr2=0x7fdce80b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 0x7fdce80b0430 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ad040 tx=0x7fdcd801ad60 comp rx=0 tx=0).stop 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 shutdown_connections 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdcc4066f60 0x7fdcc4069420 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce80b31c0 0x7fdce81543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce80aef80 0x7fdce8159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 --2- 172.21.15.45:0/2313322104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdce8070350 0x7fdce80b0430 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 >> 172.21.15.45:0/2313322104 conn(0x7fdce80669a0 msgr2=0x7fdce81565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 shutdown_connections 2024-04-25T10:31:12.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.227+0000 7fdcee000640 1 -- 172.21.15.45:0/2313322104 wait complete. 2024-04-25T10:31:12.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:31:12.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 Processor -- start 2024-04-25T10:31:12.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 -- start start 2024-04-25T10:31:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc0cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5bc071ab0 0x7fa5bc0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc071f80 con 0x7fa5bc15bc00 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5bc072260 con 0x7fa5bc071ab0 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5c0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0b3700 con 0x7fa5bc0cf040 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5bc071ab0 0x7fa5bc0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5bc071ab0 0x7fa5bc0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35376/0 (socket says 172.21.15.45:35376) 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 -- 172.21.15.45:0/2019225651 learned_addr learned my addr 172.21.15.45:0/2019225651 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5b9e00640 1 --2- 172.21.15.45:0/2019225651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc0cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 -- 172.21.15.45:0/2019225651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 msgr2=0x7fa5bc0cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 --2- 172.21.15.45:0/2019225651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc0cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 -- 172.21.15.45:0/2019225651 >> v1:172.21.15.117:6789/0 conn(0x7fa5bc15bc00 legacy=0x7fa5bc0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.395+0000 7fa5ba800640 1 -- 172.21.15.45:0/2019225651 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc15b120 con 0x7fa5bc071ab0 2024-04-25T10:31:12.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5ba800640 1 --2- 172.21.15.45:0/2019225651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc0cff20 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009e50 tx=0x7fa5b401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884b4ef46f2e4786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5b8a00640 1 -- 172.21.15.45:0/2019225651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40a57f0 con 0x7fa5bc071ab0 2024-04-25T10:31:12.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5b8a00640 1 -- 172.21.15.45:0/2019225651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b40a5990 con 0x7fa5bc071ab0 2024-04-25T10:31:12.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5b8a00640 1 -- 172.21.15.45:0/2019225651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40a5d60 con 0x7fa5bc071ab0 2024-04-25T10:31:12.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 -- 172.21.15.45:0/2019225651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 msgr2=0x7fa5bc0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/2019225651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc0cff20 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009e50 tx=0x7fa5b401ca30 comp rx=0 tx=0).stop 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 -- 172.21.15.45:0/2019225651 shutdown_connections 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/2019225651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc0cff20 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/2019225651 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 -- 172.21.15.45:0/2019225651 >> 172.21.15.45:0/2019225651 conn(0x7fa5bc066b70 msgr2=0x7fa5bc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 -- 172.21.15.45:0/2019225651 shutdown_connections 2024-04-25T10:31:12.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.399+0000 7fa5c0a00640 1 -- 172.21.15.45:0/2019225651 wait complete. 2024-04-25T10:31:12.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 Processor -- start 2024-04-25T10:31:12.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- start start 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc15bc00 0x7fa5bc156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0aec90 con 0x7fa5bc071ab0 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0cf630 con 0x7fa5bc15bc00 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0aee10 con 0x7fa5bc0cf040 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35382/0 (socket says 172.21.15.45:35382) 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 -- 172.21.15.45:0/685247992 learned_addr learned my addr 172.21.15.45:0/685247992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5ba800640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc15bc00 0x7fa5bc156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9400640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 -- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 msgr2=0x7fa5bc150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 -- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc15bc00 msgr2=0x7fa5bc156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc15bc00 0x7fa5bc156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 -- 172.21.15.45:0/685247992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc157a00 con 0x7fa5bc071ab0 2024-04-25T10:31:12.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b9e00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc151270 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d800 tx=0x7fa5ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00b6c0 con 0x7fa5bc071ab0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac019070 con 0x7fa5bc071ab0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00ba60 con 0x7fa5bc071ab0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc158f70 con 0x7fa5bc071ab0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc157720 con 0x7fa5bc071ab0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.403+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc04ef30 con 0x7fa5bc071ab0 2024-04-25T10:31:12.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.407+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5ac00bc00 con 0x7fa5bc071ab0 2024-04-25T10:31:12.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.407+0000 7fa5b2000640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa59c066f90 0x7fa59c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:12.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.407+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fa5ac0854b0 con 0x7fa5bc071ab0 2024-04-25T10:31:12.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.407+0000 7fa5b9400640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa59c066f90 0x7fa59c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:12.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.407+0000 7fa5b9400640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa59c066f90 0x7fa59c069450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc166620 tx=0x7fa5a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:12.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.411+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5ac055d00 con 0x7fa5bc071ab0 2024-04-25T10:31:12.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:12.575+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5bc0b2a10 con 0x7fa5bc071ab0 2024-04-25T10:31:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5b2000640 1 -- 172.21.15.45:0/685247992 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v173) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa5ac059da0 con 0x7fa5bc071ab0 2024-04-25T10:31:13.359 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:31:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa59c066f90 msgr2=0x7fa59c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa59c066f90 0x7fa59c069450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc166620 tx=0x7fa5a8009040 comp rx=0 tx=0).stop 2024-04-25T10:31:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 msgr2=0x7fa5bc151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc151270 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d800 tx=0x7fa5ac00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 shutdown_connections 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa59c066f90 0x7fa59c069450 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5bc15bc00 0x7fa5bc156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5bc0cf040 0x7fa5bc150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 --2- 172.21.15.45:0/685247992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5bc071ab0 0x7fa5bc151270 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.359+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 >> 172.21.15.45:0/685247992 conn(0x7fa5bc066b70 msgr2=0x7fa5bc066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.363+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 shutdown_connections 2024-04-25T10:31:13.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.363+0000 7fa5c0a00640 1 -- 172.21.15.45:0/685247992 wait complete. 2024-04-25T10:31:13.387 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:31:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 Processor -- start 2024-04-25T10:31:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 -- start start 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0ad415bd10 0x7f0ad40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0ad40cf9e0 0x7f0ad40cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad406c0e0 con 0x7f0ad415c0f0 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad40b2ef0 con 0x7f0ad40cf9e0 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad9a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad40d0350 con 0x7f0ad415bd10 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad3400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0ad40cf9e0 0x7f0ad40cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0aca000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ad415c0f0 0x7f0ad40b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51450/0 (socket says 172.21.15.45:51450) 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0aca000640 1 -- 172.21.15.45:0/1677906255 learned_addr learned my addr 172.21.15.45:0/1677906255 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad2a00640 1 --2- 172.21.15.45:0/1677906255 >> v2:172.21.15.117:3300/0 conn(0x7f0ad415bd10 0x7f0ad40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad3400640 1 -- 172.21.15.45:0/1677906255 >> v2:172.21.15.117:3300/0 conn(0x7f0ad415bd10 msgr2=0x7f0ad40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad3400640 1 --2- 172.21.15.45:0/1677906255 >> v2:172.21.15.117:3300/0 conn(0x7f0ad415bd10 0x7f0ad40b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad3400640 1 -- 172.21.15.45:0/1677906255 >> v1:172.21.15.45:6789/0 conn(0x7f0ad415c0f0 legacy=0x7f0ad40b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad3400640 1 -- 172.21.15.45:0/1677906255 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad415c6a0 con 0x7f0ad40cf9e0 2024-04-25T10:31:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.535+0000 7f0ad2000640 1 -- 172.21.15.45:0/1677906255 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673953882 0 0) 0x7f0ad406c0e0 con 0x7f0ad415c0f0 2024-04-25T10:31:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad3400640 1 --2- 172.21.15.45:0/1677906255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad40cf9e0 0x7f0ad40cfe10 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4072810 tx=0x7f0acc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8f6b23ef48c461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad2000640 1 -- 172.21.15.45:0/1677906255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0ab430 con 0x7f0ad40cf9e0 2024-04-25T10:31:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad2000640 1 -- 172.21.15.45:0/1677906255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc0b0070 con 0x7f0ad40cf9e0 2024-04-25T10:31:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad2000640 1 -- 172.21.15.45:0/1677906255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0ab7f0 con 0x7f0ad40cf9e0 2024-04-25T10:31:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1677906255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad40cf9e0 msgr2=0x7f0ad40cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1677906255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad40cf9e0 0x7f0ad40cfe10 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4072810 tx=0x7f0acc01c430 comp rx=0 tx=0).stop 2024-04-25T10:31:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1677906255 shutdown_connections 2024-04-25T10:31:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1677906255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad40cf9e0 0x7f0ad40cfe10 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1677906255 >> v2:172.21.15.117:3300/0 conn(0x7f0ad415bd10 0x7f0ad40b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1677906255 >> 172.21.15.45:0/1677906255 conn(0x7f0ad4066b70 msgr2=0x7f0ad4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1677906255 shutdown_connections 2024-04-25T10:31:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.539+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1677906255 wait complete. 2024-04-25T10:31:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 Processor -- start 2024-04-25T10:31:13.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 -- start start 2024-04-25T10:31:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ad40cf9e0 0x7f0ad4157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 0x7f0ad41481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ad415c0f0 0x7f0ad41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40b4cd0 con 0x7f0ad415bd10 2024-04-25T10:31:13.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40b4b50 con 0x7f0ad415c0f0 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad406cdd0 con 0x7f0ad40cf9e0 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 0x7f0ad41481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 0x7f0ad41481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35404/0 (socket says 172.21.15.45:35404) 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 -- 172.21.15.45:0/1812180012 learned_addr learned my addr 172.21.15.45:0/1812180012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad2a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ad40cf9e0 0x7f0ad4157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad3400640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ad415c0f0 0x7f0ad41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 -- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ad40cf9e0 msgr2=0x7f0ad4157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ad40cf9e0 0x7f0ad4157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 -- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ad415c0f0 msgr2=0x7f0ad41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ad415c0f0 0x7f0ad41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 -- 172.21.15.45:0/1812180012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad4156c80 con 0x7f0ad415bd10 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad2a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ad40cf9e0 0x7f0ad4157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad3400640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ad415c0f0 0x7f0ad41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0aca000640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 0x7f0ad41481c0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c480 tx=0x7f0abc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc018070 con 0x7f0ad415bd10 2024-04-25T10:31:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc007500 con 0x7f0ad415bd10 2024-04-25T10:31:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc004a30 con 0x7f0ad415bd10 2024-04-25T10:31:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad4156f60 con 0x7f0ad415bd10 2024-04-25T10:31:13.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.543+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad4157520 con 0x7f0ad415bd10 2024-04-25T10:31:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.547+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad4157a10 con 0x7f0ad415bd10 2024-04-25T10:31:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.547+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0abc01b070 con 0x7f0ad415bd10 2024-04-25T10:31:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.547+0000 7f0acb400640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0ab0066f90 0x7f0ab0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.547+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f0abc0842f0 con 0x7f0ad415bd10 2024-04-25T10:31:13.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.547+0000 7f0ad2a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0ab0066f90 0x7f0ab0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:13.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.547+0000 7f0ad2a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0ab0066f90 0x7f0ab0069450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007330 tx=0x7f0ac400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.551+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0abc0553d0 con 0x7f0ad415bd10 2024-04-25T10:31:13.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:13.711+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ad4166890 con 0x7f0ad415bd10 2024-04-25T10:31:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.359+0000 7f0acb400640 1 -- 172.21.15.45:0/1812180012 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v174) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0abc054cd0 con 0x7f0ad415bd10 2024-04-25T10:31:14.362 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0ab0066f90 msgr2=0x7f0ab0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0ab0066f90 0x7f0ab0069450 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007330 tx=0x7f0ac400b040 comp rx=0 tx=0).stop 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 msgr2=0x7f0ad41481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 0x7f0ad41481c0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c480 tx=0x7f0abc00c940 comp rx=0 tx=0).stop 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 shutdown_connections 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0ab0066f90 0x7f0ab0069450 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ad415c0f0 0x7f0ad41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ad415bd10 0x7f0ad41481c0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 --2- 172.21.15.45:0/1812180012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ad40cf9e0 0x7f0ad4157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 >> 172.21.15.45:0/1812180012 conn(0x7f0ad4066b70 msgr2=0x7f0ad40ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 shutdown_connections 2024-04-25T10:31:14.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.363+0000 7f0ad9a00640 1 -- 172.21.15.45:0/1812180012 wait complete. 2024-04-25T10:31:14.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:31:14.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 Processor -- start 2024-04-25T10:31:14.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 -- start start 2024-04-25T10:31:14.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a2815d9b0 0x7f8a280cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a280669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2806d220 con 0x7f8a281620b0 2024-04-25T10:31:14.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a28162600 con 0x7f8a2815d9b0 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280cad00 con 0x7f8a2815dd90 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a2815d9b0 0x7f8a280cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a2815d9b0 0x7f8a280cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35406/0 (socket says 172.21.15.45:35406) 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 -- 172.21.15.45:0/352924599 learned_addr learned my addr 172.21.15.45:0/352924599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2d400640 1 --2- 172.21.15.45:0/352924599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a280669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 -- 172.21.15.45:0/352924599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 msgr2=0x7f8a280669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 --2- 172.21.15.45:0/352924599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a280669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 -- 172.21.15.45:0/352924599 >> v1:172.21.15.117:6789/0 conn(0x7f8a281620b0 legacy=0x7f8a2806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.539+0000 7f8a2de00640 1 -- 172.21.15.45:0/352924599 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280cafe0 con 0x7f8a2815d9b0 2024-04-25T10:31:14.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2de00640 1 --2- 172.21.15.45:0/352924599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a2815d9b0 0x7f8a280cc860 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009870 tx=0x7f8a1c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c956a3318910d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2ca00640 1 -- 172.21.15.45:0/352924599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a7070 con 0x7f8a2815d9b0 2024-04-25T10:31:14.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2ca00640 1 -- 172.21.15.45:0/352924599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0b0070 con 0x7f8a2815d9b0 2024-04-25T10:31:14.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2ca00640 1 -- 172.21.15.45:0/352924599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0ab890 con 0x7f8a2815d9b0 2024-04-25T10:31:14.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 -- 172.21.15.45:0/352924599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a2815d9b0 msgr2=0x7f8a280cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 --2- 172.21.15.45:0/352924599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a2815d9b0 0x7f8a280cc860 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009870 tx=0x7f8a1c01c560 comp rx=0 tx=0).stop 2024-04-25T10:31:14.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 -- 172.21.15.45:0/352924599 shutdown_connections 2024-04-25T10:31:14.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 --2- 172.21.15.45:0/352924599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a280669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 --2- 172.21.15.45:0/352924599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a2815d9b0 0x7f8a280cc860 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 -- 172.21.15.45:0/352924599 >> 172.21.15.45:0/352924599 conn(0x7f8a280d07e0 msgr2=0x7f8a280d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 -- 172.21.15.45:0/352924599 shutdown_connections 2024-04-25T10:31:14.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 -- 172.21.15.45:0/352924599 wait complete. 2024-04-25T10:31:14.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 Processor -- start 2024-04-25T10:31:14.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.543+0000 7f8a2f200640 1 -- start start 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a2815d9b0 0x7f8a28152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a2815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 0x7f8a28155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2816aca0 con 0x7f8a281620b0 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280672f0 con 0x7f8a2815d9b0 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2816ae20 con 0x7f8a2815dd90 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 0x7f8a28155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 0x7f8a28155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35422/0 (socket says 172.21.15.45:35422) 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 -- 172.21.15.45:0/1106123127 learned_addr learned my addr 172.21.15.45:0/1106123127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2d400640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a2815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2de00640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a2815d9b0 0x7f8a28152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 -- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 msgr2=0x7f8a2815b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a2815b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 -- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a2815d9b0 msgr2=0x7f8a28152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a2815d9b0 0x7f8a28152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 -- 172.21.15.45:0/1106123127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a28152fd0 con 0x7f8a281620b0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2d400640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a2815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2e800640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 0x7f8a28155530 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c760 tx=0x7f8a2400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a24003ba0 con 0x7f8a281620b0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a24003d40 con 0x7f8a281620b0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a281532b0 con 0x7f8a281620b0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2400aa90 con 0x7f8a281620b0 2024-04-25T10:31:14.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a28153870 con 0x7f8a281620b0 2024-04-25T10:31:14.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a28075120 con 0x7f8a281620b0 2024-04-25T10:31:14.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.547+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a2400ac30 con 0x7f8a281620b0 2024-04-25T10:31:14.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.551+0000 7f8a22000640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a0c066f60 0x7f8a0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.551+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a24084bd0 con 0x7f8a281620b0 2024-04-25T10:31:14.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.551+0000 7f8a2de00640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a0c066f60 0x7f8a0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.551+0000 7f8a2de00640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a0c066f60 0x7f8a0c069420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0af000 tx=0x7f8a1c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.555+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a24055ec0 con 0x7f8a281620b0 2024-04-25T10:31:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.719+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8a28071d60 con 0x7f8a281620b0 2024-04-25T10:31:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.723+0000 7f8a22000640 1 -- 172.21.15.45:0/1106123127 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a240557c0 con 0x7f8a281620b0 2024-04-25T10:31:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a0c066f60 msgr2=0x7f8a0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a0c066f60 0x7f8a0c069420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0af000 tx=0x7f8a1c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:31:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 msgr2=0x7f8a28155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 0x7f8a28155530 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c760 tx=0x7f8a2400cc20 comp rx=0 tx=0).stop 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 shutdown_connections 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8a0c066f60 0x7f8a0c069420 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a281620b0 0x7f8a28155530 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a2815dd90 0x7f8a2815b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 --2- 172.21.15.45:0/1106123127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a2815d9b0 0x7f8a28152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 >> 172.21.15.45:0/1106123127 conn(0x7f8a280d07e0 msgr2=0x7f8a28156d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 shutdown_connections 2024-04-25T10:31:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.727+0000 7f8a2f200640 1 -- 172.21.15.45:0/1106123127 wait complete. 2024-04-25T10:31:14.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:31:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99a00640 1 Processor -- start 2024-04-25T10:31:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99a00640 1 -- start start 2024-04-25T10:31:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7c9406d670 0x7f7c94161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c940b32a0 con 0x7f7c940b2260 2024-04-25T10:31:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c940af510 con 0x7f7c94154af0 2024-04-25T10:31:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c940b3580 con 0x7f7c9406d670 2024-04-25T10:31:14.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c8b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7c940b2260 0x7f7c94161600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41558/0 (socket says 172.21.15.45:41558) 2024-04-25T10:31:14.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c8b400640 1 -- 172.21.15.45:0/729262335 learned_addr learned my addr 172.21.15.45:0/729262335 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99000640 1 --2- 172.21.15.45:0/729262335 >> v2:172.21.15.117:3300/0 conn(0x7f7c9406d670 0x7f7c94161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810885332 0 0) 0x7f7c940b32a0 con 0x7f7c940b2260 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c78003530 con 0x7f7c940b2260 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634062774 0 0) 0x7f7c940af510 con 0x7f7c94154af0 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c940b32a0 con 0x7f7c94154af0 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99000640 1 -- 172.21.15.45:0/729262335 >> v1:172.21.15.117:6789/0 conn(0x7f7c94154af0 legacy=0x7f7c940b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99000640 1 -- 172.21.15.45:0/729262335 >> v1:172.21.15.45:6789/0 conn(0x7f7c940b2260 legacy=0x7f7c94161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.887+0000 7f7c99000640 1 -- 172.21.15.45:0/729262335 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c940af7f0 con 0x7f7c9406d670 2024-04-25T10:31:14.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99000640 1 --2- 172.21.15.45:0/729262335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c9406d670 0x7f7c94161d10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009d90 tx=0x7f7c8c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34eddfe98657ef96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0b2070 con 0x7f7c9406d670 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c0a5830 con 0x7f7c9406d670 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c93400640 1 -- 172.21.15.45:0/729262335 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0a5c00 con 0x7f7c9406d670 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 -- 172.21.15.45:0/729262335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c9406d670 msgr2=0x7f7c94161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 --2- 172.21.15.45:0/729262335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c9406d670 0x7f7c94161d10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009d90 tx=0x7f7c8c01c970 comp rx=0 tx=0).stop 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 -- 172.21.15.45:0/729262335 shutdown_connections 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 --2- 172.21.15.45:0/729262335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c9406d670 0x7f7c94161d10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 -- 172.21.15.45:0/729262335 >> 172.21.15.45:0/729262335 conn(0x7f7c9406ac90 msgr2=0x7f7c9406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:14.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 -- 172.21.15.45:0/729262335 shutdown_connections 2024-04-25T10:31:14.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.891+0000 7f7c99a00640 1 -- 172.21.15.45:0/729262335 wait complete. 2024-04-25T10:31:14.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 Processor -- start 2024-04-25T10:31:14.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 -- start start 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 0x7f7c940ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c940b2260 0x7f7c94148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c94154af0 0x7f7c9414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940d0760 con 0x7f7c9406d670 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940b4dc0 con 0x7f7c94154af0 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940d0bd0 con 0x7f7c940b2260 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 0x7f7c940ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 0x7f7c940ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32904/0 (socket says 172.21.15.45:32904) 2024-04-25T10:31:14.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 -- 172.21.15.45:0/2343262459 learned_addr learned my addr 172.21.15.45:0/2343262459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99000640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c94154af0 0x7f7c9414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c8b400640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c940b2260 0x7f7c94148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 -- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c940b2260 msgr2=0x7f7c94148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c940b2260 0x7f7c94148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 -- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c94154af0 msgr2=0x7f7c9414c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c94154af0 0x7f7c9414c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 -- 172.21.15.45:0/2343262459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94160b20 con 0x7f7c9406d670 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c8b400640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c940b2260 0x7f7c94148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c93e00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 0x7f7c940ce110 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c430 tx=0x7f7c8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84017070 con 0x7f7c9406d670 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c840052f0 con 0x7f7c9406d670 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c94160e00 con 0x7f7c9406d670 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84003840 con 0x7f7c9406d670 2024-04-25T10:31:14.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.895+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c940cd630 con 0x7f7c9406d670 2024-04-25T10:31:14.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.899+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c94075120 con 0x7f7c9406d670 2024-04-25T10:31:14.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.899+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c8401a070 con 0x7f7c9406d670 2024-04-25T10:31:14.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.899+0000 7f7c90c00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c70066f60 0x7f7c70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:14.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.899+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7c840841e0 con 0x7f7c9406d670 2024-04-25T10:31:14.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.899+0000 7f7c8b400640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c70066f60 0x7f7c70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:14.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.899+0000 7f7c8b400640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c70066f60 0x7f7c70069420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7c941fa7a0 tx=0x7f7c7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:14.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:14.903+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c84055450 con 0x7f7c9406d670 2024-04-25T10:31:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.127+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f7c940cc680 con 0x7f7c9406d670 2024-04-25T10:31:15.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.371+0000 7f7c90c00640 1 -- 172.21.15.45:0/2343262459 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v637) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7c84054d50 con 0x7f7c9406d670 2024-04-25T10:31:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c70066f60 msgr2=0x7f7c70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c70066f60 0x7f7c70069420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7c941fa7a0 tx=0x7f7c7c009040 comp rx=0 tx=0).stop 2024-04-25T10:31:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 msgr2=0x7f7c940ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 0x7f7c940ce110 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c430 tx=0x7f7c8400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 shutdown_connections 2024-04-25T10:31:15.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c70066f60 0x7f7c70069420 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c94154af0 0x7f7c9414c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c940b2260 0x7f7c94148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 --2- 172.21.15.45:0/2343262459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c9406d670 0x7f7c940ce110 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 >> 172.21.15.45:0/2343262459 conn(0x7f7c9406ac90 msgr2=0x7f7c9406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 shutdown_connections 2024-04-25T10:31:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.375+0000 7f7c99a00640 1 -- 172.21.15.45:0/2343262459 wait complete. 2024-04-25T10:31:15.397 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:31:15.397 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:31:15.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9f1400640 1 Processor -- start 2024-04-25T10:31:15.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9f1400640 1 -- start start 2024-04-25T10:31:15.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9f1400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe9ec15d9b0 0x7fe9ec15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9f1400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec15e400 con 0x7fe9ec0b38d0 2024-04-25T10:31:15.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9f1400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec0b0000 con 0x7fe9ec15eea0 2024-04-25T10:31:15.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9f1400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b02e0 con 0x7fe9ec15d9b0 2024-04-25T10:31:15.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e1e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe9ec0b38d0 0x7fe9ec0af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41570/0 (socket says 172.21.15.45:41570) 2024-04-25T10:31:15.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e1e00640 1 -- 172.21.15.45:0/413312332 learned_addr learned my addr 172.21.15.45:0/413312332 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9ea800640 1 --2- 172.21.15.45:0/413312332 >> v2:172.21.15.117:3300/0 conn(0x7fe9ec15d9b0 0x7fe9ec15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615165251 0 0) 0x7fe9ec15e400 con 0x7fe9ec0b38d0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d0003530 con 0x7fe9ec0b38d0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921620305 0 0) 0x7fe9ec0b0000 con 0x7fe9ec15eea0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ec15e400 con 0x7fe9ec15eea0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1080635374 0 0) 0x7fe9d0003530 con 0x7fe9ec0b38d0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 >> v2:172.21.15.117:3300/0 conn(0x7fe9ec15d9b0 msgr2=0x7fe9ec15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 --2- 172.21.15.45:0/413312332 >> v2:172.21.15.117:3300/0 conn(0x7fe9ec15d9b0 0x7fe9ec15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 >> v1:172.21.15.117:6789/0 conn(0x7fe9ec15eea0 legacy=0x7fe9ec0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec15e6a0 con 0x7fe9ec0b38d0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fe9d4001680 con 0x7fe9ec0b38d0 2024-04-25T10:31:15.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 >> v1:172.21.15.45:6789/0 conn(0x7fe9ec0b38d0 legacy=0x7fe9ec0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d0003ba0 0x7fe9d0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d0004810 0x7fe9d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b02e0 con 0x7fe9d0005220 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d000c150 con 0x7fe9d0004810 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d000c350 con 0x7fe9d0003ba0 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9ea800640 1 --2- 172.21.15.45:0/413312332 >> v2:172.21.15.117:3300/0 conn(0x7fe9ec15d9b0 0x7fe9ec15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.539+0000 7fe9eb200640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9e1e00640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d0004810 0x7fe9d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9ea800640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d0003ba0 0x7fe9d0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 -- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d0003ba0 msgr2=0x7fe9d0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d0003ba0 0x7fe9d0003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 -- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d0004810 msgr2=0x7fe9d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:15.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d0004810 0x7fe9d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 -- 172.21.15.45:0/413312332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0003530 con 0x7fe9d0005220 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 -- 172.21.15.45:0/413312332 reap_dead start 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9eb200640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9d000b9d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4001ff0 tx=0x7fe9e401f090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606f789eeaf3a47a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4001c40 con 0x7fe9d0005220 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e40a8cb0 con 0x7fe9d0005220 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9e9e00640 1 -- 172.21.15.45:0/413312332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40a8e60 con 0x7fe9d0005220 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9f1400640 1 -- 172.21.15.45:0/413312332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 msgr2=0x7fe9d000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9f1400640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9d000b9d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4001ff0 tx=0x7fe9e401f090 comp rx=0 tx=0).stop 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9f1400640 1 -- 172.21.15.45:0/413312332 shutdown_connections 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9f1400640 1 --2- 172.21.15.45:0/413312332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9d000b9d0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.543+0000 7fe9f1400640 1 -- 172.21.15.45:0/413312332 >> 172.21.15.45:0/413312332 conn(0x7fe9ec06ac90 msgr2=0x7fe9ec06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:15.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- 172.21.15.45:0/413312332 shutdown_connections 2024-04-25T10:31:15.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- 172.21.15.45:0/413312332 wait complete. 2024-04-25T10:31:15.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 Processor -- start 2024-04-25T10:31:15.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- start start 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9ec159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9ec161dd0 0x7fe9ec066c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9ec0ae890 0x7fe9ec0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec15df10 con 0x7fe9d0005220 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec15dd90 con 0x7fe9ec0ae890 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec15e090 con 0x7fe9ec161dd0 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9ec159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9ec159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32934/0 (socket says 172.21.15.45:32934) 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 -- 172.21.15.45:0/4220173348 learned_addr learned my addr 172.21.15.45:0/4220173348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9eb200640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9ec0ae890 0x7fe9ec0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9e1e00640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9ec161dd0 0x7fe9ec066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 -- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9ec161dd0 msgr2=0x7fe9ec066c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9ec161dd0 0x7fe9ec066c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 -- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9ec0ae890 msgr2=0x7fe9ec0aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9ec0ae890 0x7fe9ec0aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 -- 172.21.15.45:0/4220173348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec159fc0 con 0x7fe9d0005220 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9ea800640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9ec159770 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc011930 tx=0x7fe9dc011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00fb70 con 0x7fe9d0005220 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc00fd10 con 0x7fe9d0005220 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec15a2a0 con 0x7fe9d0005220 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc015660 con 0x7fe9d0005220 2024-04-25T10:31:15.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.547+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec15a860 con 0x7fe9d0005220 2024-04-25T10:31:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.551+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec04ef30 con 0x7fe9d0005220 2024-04-25T10:31:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.551+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9dc015800 con 0x7fe9d0005220 2024-04-25T10:31:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.551+0000 7fe9e3400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9c8066f90 0x7fe9c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.551+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe9dc0856c0 con 0x7fe9d0005220 2024-04-25T10:31:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.551+0000 7fe9e1e00640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9c8066f90 0x7fe9c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.551+0000 7fe9e1e00640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9c8066f90 0x7fe9c8069450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec06ee60 tx=0x7fe9d4000f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:15.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.555+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9dc019030 con 0x7fe9d0005220 2024-04-25T10:31:15.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.723+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9ec0719b0 con 0x7fe9d0005220 2024-04-25T10:31:15.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9e3400640 1 -- 172.21.15.45:0/4220173348 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+13020 (secure 0 0 0) 0x7fe9dc0562e0 con 0x7fe9d0005220 2024-04-25T10:31:15.728 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:15.728 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":174,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:14.361140+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/1582038965":"2024-04-26T10:31:11.222952+0000","172.21.15.117:6835/1582038965":"2024-04-26T10:31:11.222952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9c8066f90 msgr2=0x7fe9c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9c8066f90 0x7fe9c8069450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec06ee60 tx=0x7fe9d4000f90 comp rx=0 tx=0).stop 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 msgr2=0x7fe9ec159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9ec159770 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc011930 tx=0x7fe9dc011df0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 shutdown_connections 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9c8066f90 0x7fe9c8069450 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9ec0ae890 0x7fe9ec0aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9ec161dd0 0x7fe9ec066c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 --2- 172.21.15.45:0/4220173348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9d0005220 0x7fe9ec159770 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 >> 172.21.15.45:0/4220173348 conn(0x7fe9ec06ac90 msgr2=0x7fe9ec15c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 shutdown_connections 2024-04-25T10:31:15.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.727+0000 7fe9f1400640 1 -- 172.21.15.45:0/4220173348 wait complete. 2024-04-25T10:31:15.749 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:15.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:15.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.891+0000 7f83d1200640 1 Processor -- start 2024-04-25T10:31:15.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.891+0000 7f83d1200640 1 -- start start 2024-04-25T10:31:15.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.891+0000 7f83d1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f83cc159540 0x7f83cc0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.891+0000 7f83d1200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc0aebd0 0x7f83cc071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83d1200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83cc071360 con 0x7f83cc072020 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83d1200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc159920 con 0x7f83cc159540 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83d1200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc15c2f0 con 0x7f83cc0aebd0 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f83cc159540 0x7f83cc0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f83cc159540 0x7f83cc0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32938/0 (socket says 172.21.15.45:32938) 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 -- 172.21.15.45:0/3032553370 learned_addr learned my addr 172.21.15.45:0/3032553370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c1400640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc0aebd0 0x7f83cc071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 -- 172.21.15.45:0/3032553370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc0aebd0 msgr2=0x7f83cc071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc0aebd0 0x7f83cc071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 -- 172.21.15.45:0/3032553370 >> v1:172.21.15.117:6789/0 conn(0x7f83cc072020 legacy=0x7f83cc15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 -- 172.21.15.45:0/3032553370 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc15c5d0 con 0x7f83cc159540 2024-04-25T10:31:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c1400640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc0aebd0 0x7f83cc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9e00640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc159540 0x7f83cc0ae690 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0b4c20 tx=0x7f83bc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942d41197d674168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9400640 1 -- 172.21.15.45:0/3032553370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0b0070 con 0x7f83cc159540 2024-04-25T10:31:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9400640 1 -- 172.21.15.45:0/3032553370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc0ab600 con 0x7f83cc159540 2024-04-25T10:31:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.895+0000 7f83c9400640 1 -- 172.21.15.45:0/3032553370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0ab9d0 con 0x7f83cc159540 2024-04-25T10:31:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- 172.21.15.45:0/3032553370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc159540 msgr2=0x7f83cc0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc159540 0x7f83cc0ae690 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0b4c20 tx=0x7f83bc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- 172.21.15.45:0/3032553370 shutdown_connections 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc0aebd0 0x7f83cc071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 --2- 172.21.15.45:0/3032553370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc159540 0x7f83cc0ae690 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- 172.21.15.45:0/3032553370 >> 172.21.15.45:0/3032553370 conn(0x7f83cc0669a0 msgr2=0x7f83cc066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- 172.21.15.45:0/3032553370 shutdown_connections 2024-04-25T10:31:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- 172.21.15.45:0/3032553370 wait complete. 2024-04-25T10:31:15.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 Processor -- start 2024-04-25T10:31:15.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- start start 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83cc072020 0x7f83cc0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 0x7f83cc0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc159540 0x7f83cc154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc06c0d0 con 0x7f83cc0aebd0 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc15bfa0 con 0x7f83cc072020 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83d1200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc168930 con 0x7f83cc159540 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 0x7f83cc0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 0x7f83cc0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32944/0 (socket says 172.21.15.45:32944) 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 -- 172.21.15.45:0/2381118917 learned_addr learned my addr 172.21.15.45:0/2381118917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c9e00640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83cc072020 0x7f83cc0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83ca800640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc159540 0x7f83cc154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 -- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc159540 msgr2=0x7f83cc154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc159540 0x7f83cc154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 -- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83cc072020 msgr2=0x7f83cc0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83cc072020 0x7f83cc0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83c1400640 1 -- 172.21.15.45:0/2381118917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0b1050 con 0x7f83cc0aebd0 2024-04-25T10:31:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.899+0000 7f83ca800640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc159540 0x7f83cc154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c1400640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 0x7f83cc0b20f0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c7c0 tx=0x7f83b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4007a30 con 0x7f83cc0aebd0 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4007bd0 con 0x7f83cc0aebd0 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40113f0 con 0x7f83cc0aebd0 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc0b1330 con 0x7f83cc0aebd0 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc0b18f0 con 0x7f83cc0aebd0 2024-04-25T10:31:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc04ef30 con 0x7f83cc0aebd0 2024-04-25T10:31:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83b4011590 con 0x7f83cc0aebd0 2024-04-25T10:31:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c2a00640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83a8066f90 0x7f83a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83b4084e80 con 0x7f83cc0aebd0 2024-04-25T10:31:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.903+0000 7f83c9e00640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83a8066f90 0x7f83a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:15.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.907+0000 7f83c9e00640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83a8066f90 0x7f83a8069450 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0ad000 tx=0x7f83bc0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:15.907+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83b4015020 con 0x7f83cc0aebd0 2024-04-25T10:31:16.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.071+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83cc06d7f0 con 0x7f83cc0aebd0 2024-04-25T10:31:16.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.071+0000 7f83c2a00640 1 -- 172.21.15.45:0/2381118917 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83b4055b40 con 0x7f83cc0aebd0 2024-04-25T10:31:16.075 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83a8066f90 msgr2=0x7f83a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83a8066f90 0x7f83a8069450 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0ad000 tx=0x7f83bc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 msgr2=0x7f83cc0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 0x7f83cc0b20f0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c7c0 tx=0x7f83b400cc80 comp rx=0 tx=0).stop 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 shutdown_connections 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f83a8066f90 0x7f83a8069450 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83cc159540 0x7f83cc154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83cc0aebd0 0x7f83cc0b20f0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 --2- 172.21.15.45:0/2381118917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83cc072020 0x7f83cc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 >> 172.21.15.45:0/2381118917 conn(0x7f83cc0669a0 msgr2=0x7f83cc15d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:16.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 shutdown_connections 2024-04-25T10:31:16.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.075+0000 7f83d1200640 1 -- 172.21.15.45:0/2381118917 wait complete. 2024-04-25T10:31:16.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:31:16.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.235+0000 7f932ac00640 1 Processor -- start 2024-04-25T10:31:16.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f932ac00640 1 -- start start 2024-04-25T10:31:16.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f932ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:16.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f932ac00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f93240cf930 0x7f93240cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:16.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f932ac00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93240b2770 con 0x7f932415bc40 2024-04-25T10:31:16.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f932ac00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932406c670 con 0x7f93240cf930 2024-04-25T10:31:16.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f932ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932406c950 con 0x7f93240cda10 2024-04-25T10:31:16.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9323e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f932415bc40 0x7f93240b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41574/0 (socket says 172.21.15.45:41574) 2024-04-25T10:31:16.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9323e00640 1 -- 172.21.15.45:0/2877982647 learned_addr learned my addr 172.21.15.45:0/2877982647 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f931be00640 1 --2- 172.21.15.45:0/2877982647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9328c00640 1 --2- 172.21.15.45:0/2877982647 >> v2:172.21.15.117:3300/0 conn(0x7f93240cf930 0x7f93240cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9323400640 1 -- 172.21.15.45:0/2877982647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249674677 0 0) 0x7f93240b2770 con 0x7f932415bc40 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9323400640 1 -- 172.21.15.45:0/2877982647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9308003530 con 0x7f932415bc40 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f931be00640 1 -- 172.21.15.45:0/2877982647 >> v2:172.21.15.117:3300/0 conn(0x7f93240cf930 msgr2=0x7f93240cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f931be00640 1 --2- 172.21.15.45:0/2877982647 >> v2:172.21.15.117:3300/0 conn(0x7f93240cf930 0x7f93240cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9323400640 1 -- 172.21.15.45:0/2877982647 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996874133 0 0) 0x7f9308003530 con 0x7f932415bc40 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f931be00640 1 -- 172.21.15.45:0/2877982647 >> v1:172.21.15.45:6789/0 conn(0x7f932415bc40 legacy=0x7f93240b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f931be00640 1 -- 172.21.15.45:0/2877982647 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932406cc30 con 0x7f93240cda10 2024-04-25T10:31:16.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.239+0000 7f9328c00640 1 --2- 172.21.15.45:0/2877982647 >> v2:172.21.15.117:3300/0 conn(0x7f93240cf930 0x7f93240cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:16.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f931be00640 1 --2- 172.21.15.45:0/2877982647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932406c090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097c0 tx=0x7f930c01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=902454bec1e09b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:16.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f9323400640 1 -- 172.21.15.45:0/2877982647 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0b0070 con 0x7f93240cda10 2024-04-25T10:31:16.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f9323400640 1 -- 172.21.15.45:0/2877982647 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c0ab400 con 0x7f93240cda10 2024-04-25T10:31:16.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f9323400640 1 -- 172.21.15.45:0/2877982647 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314001680 con 0x7f93240cda10 2024-04-25T10:31:16.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 -- 172.21.15.45:0/2877982647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 msgr2=0x7f932406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:16.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 --2- 172.21.15.45:0/2877982647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932406c090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f930c0097c0 tx=0x7f930c01c330 comp rx=0 tx=0).stop 2024-04-25T10:31:16.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 -- 172.21.15.45:0/2877982647 shutdown_connections 2024-04-25T10:31:16.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 --2- 172.21.15.45:0/2877982647 >> v2:172.21.15.117:3300/0 conn(0x7f93240cf930 0x7f93240cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 --2- 172.21.15.45:0/2877982647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932406c090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 -- 172.21.15.45:0/2877982647 >> 172.21.15.45:0/2877982647 conn(0x7f9324066b70 msgr2=0x7f9324066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:16.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 -- 172.21.15.45:0/2877982647 shutdown_connections 2024-04-25T10:31:16.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.243+0000 7f932ac00640 1 -- 172.21.15.45:0/2877982647 wait complete. 2024-04-25T10:31:16.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 Processor -- start 2024-04-25T10:31:16.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- start start 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240cf930 0x7f932414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 0x7f932415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b3850 con 0x7f932415bc40 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b36d0 con 0x7f93240cf930 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b39d0 con 0x7f93240cda10 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 0x7f932415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 0x7f932415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32948/0 (socket says 172.21.15.45:32948) 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 -- 172.21.15.45:0/366883941 learned_addr learned my addr 172.21.15.45:0/366883941 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9323e00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f931be00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240cf930 0x7f932414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:16.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 -- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 msgr2=0x7f932414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 -- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240cf930 msgr2=0x7f932414a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240cf930 0x7f932414a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 -- 172.21.15.45:0/366883941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932415c6d0 con 0x7f932415bc40 2024-04-25T10:31:16.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9328c00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 0x7f932415c190 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f931c00efc0 tx=0x7f931c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c018030 con 0x7f932415bc40 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00cdf0 con 0x7f932415bc40 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c003bc0 con 0x7f932415bc40 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93241537f0 con 0x7f932415bc40 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9324153db0 con 0x7f932415bc40 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.247+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932404ef30 con 0x7f932415bc40 2024-04-25T10:31:16.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.251+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f931c01b070 con 0x7f932415bc40 2024-04-25T10:31:16.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.251+0000 7f9320c00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9300066fb0 0x7f9300069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.251+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f931c085540 con 0x7f932415bc40 2024-04-25T10:31:16.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.251+0000 7f9323e00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9300066fb0 0x7f9300069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:16.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.251+0000 7f9323e00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9300066fb0 0x7f9300069470 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f93140017f0 tx=0x7f9314001580 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:16.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.255+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f931c022070 con 0x7f932415bc40 2024-04-25T10:31:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:16.475+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f932406d960 con 0x7f932415bc40 2024-04-25T10:31:17.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.379+0000 7f9320c00640 1 -- 172.21.15.45:0/366883941 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v639) v1 ==== 150+0+0 (secure 0 0 0) 0x7f931c010040 con 0x7f932415bc40 2024-04-25T10:31:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9300066fb0 msgr2=0x7f9300069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9300066fb0 0x7f9300069470 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f93140017f0 tx=0x7f9314001580 comp rx=0 tx=0).stop 2024-04-25T10:31:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 msgr2=0x7f932415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 0x7f932415c190 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f931c00efc0 tx=0x7f931c00c490 comp rx=0 tx=0).stop 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 shutdown_connections 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9300066fb0 0x7f9300069470 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932415bc40 0x7f932415c190 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93240cf930 0x7f932414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 --2- 172.21.15.45:0/366883941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93240cda10 0x7f932414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 >> 172.21.15.45:0/366883941 conn(0x7f9324066b70 msgr2=0x7f93240ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 shutdown_connections 2024-04-25T10:31:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.383+0000 7f932ac00640 1 -- 172.21.15.45:0/366883941 wait complete. 2024-04-25T10:31:17.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:31:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43dfc00640 1 Processor -- start 2024-04-25T10:31:17.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43dfc00640 1 -- start start 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43dfc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43d80721f0 0x7f43d8166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43dfc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d8071360 con 0x7f43d80aeda0 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43dfc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d8159920 con 0x7f43d8159540 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43dfc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d815c2f0 con 0x7f43d80721f0 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43ddc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43d80721f0 0x7f43d8166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43ddc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43d80721f0 0x7f43d8166920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32964/0 (socket says 172.21.15.45:32964) 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43cfe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f43d80aeda0 0x7f43d8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41586/0 (socket says 172.21.15.45:41586) 2024-04-25T10:31:17.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43ddc00640 1 -- 172.21.15.45:0/3604253274 learned_addr learned my addr 172.21.15.45:0/3604253274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43ddc00640 1 -- 172.21.15.45:0/3604253274 >> v1:172.21.15.117:6789/0 conn(0x7f43d8159540 legacy=0x7f43d80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43ddc00640 1 -- 172.21.15.45:0/3604253274 >> v1:172.21.15.45:6789/0 conn(0x7f43d80aeda0 legacy=0x7f43d8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43ddc00640 1 -- 172.21.15.45:0/3604253274 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d815c5d0 con 0x7f43d80721f0 2024-04-25T10:31:17.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.555+0000 7f43d7e00640 1 -- 172.21.15.45:0/3604253274 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473701795 0 0) 0x7f43d8071360 con 0x7f43d80aeda0 2024-04-25T10:31:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43ddc00640 1 --2- 172.21.15.45:0/3604253274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d80721f0 0x7f43d8166920 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009e50 tx=0x7f43d001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe760665f367e2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43d7e00640 1 -- 172.21.15.45:0/3604253274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00a5880 con 0x7f43d80721f0 2024-04-25T10:31:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43d7e00640 1 -- 172.21.15.45:0/3604253274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d00a5a20 con 0x7f43d80721f0 2024-04-25T10:31:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43d7e00640 1 -- 172.21.15.45:0/3604253274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00a5df0 con 0x7f43d80721f0 2024-04-25T10:31:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 -- 172.21.15.45:0/3604253274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d80721f0 msgr2=0x7f43d8166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 --2- 172.21.15.45:0/3604253274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d80721f0 0x7f43d8166920 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009e50 tx=0x7f43d001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 -- 172.21.15.45:0/3604253274 shutdown_connections 2024-04-25T10:31:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 --2- 172.21.15.45:0/3604253274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d80721f0 0x7f43d8166920 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 -- 172.21.15.45:0/3604253274 >> 172.21.15.45:0/3604253274 conn(0x7f43d80669a0 msgr2=0x7f43d8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 -- 172.21.15.45:0/3604253274 shutdown_connections 2024-04-25T10:31:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.559+0000 7f43dfc00640 1 -- 172.21.15.45:0/3604253274 wait complete. 2024-04-25T10:31:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 Processor -- start 2024-04-25T10:31:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 -- start start 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d80721f0 0x7f43d815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d80aeda0 0x7f43d8143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 0x7f43d815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8169680 con 0x7f43d8159540 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d806c0d0 con 0x7f43d80721f0 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8169800 con 0x7f43d80aeda0 2024-04-25T10:31:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 0x7f43d815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 0x7f43d815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32972/0 (socket says 172.21.15.45:32972) 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 -- 172.21.15.45:0/2327949844 learned_addr learned my addr 172.21.15.45:0/2327949844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dd200640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d80721f0 0x7f43d815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43cfe00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d80aeda0 0x7f43d8143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 -- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d80aeda0 msgr2=0x7f43d8143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d80aeda0 0x7f43d8143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 -- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d80721f0 msgr2=0x7f43d815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d80721f0 0x7f43d815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 -- 172.21.15.45:0/2327949844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8153950 con 0x7f43d8159540 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43ddc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 0x7f43d815dc90 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f43d00af040 tx=0x7f43d0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00bc020 con 0x7f43d8159540 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d00b88a0 con 0x7f43d8159540 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d8153c30 con 0x7f43d8159540 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00b8c90 con 0x7f43d8159540 2024-04-25T10:31:17.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.563+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d81541f0 con 0x7f43d8159540 2024-04-25T10:31:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.567+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d804ef30 con 0x7f43d8159540 2024-04-25T10:31:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.567+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43d00bf020 con 0x7f43d8159540 2024-04-25T10:31:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.567+0000 7f43d5600640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43b4066f60 0x7f43b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.567+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43d0120170 con 0x7f43d8159540 2024-04-25T10:31:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.567+0000 7f43dd200640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43b4066f60 0x7f43b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.567+0000 7f43dd200640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43b4066f60 0x7f43b4069420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f43c800b810 tx=0x7f43c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.571+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43d00c6030 con 0x7f43d8159540 2024-04-25T10:31:17.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.735+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f43d815b9e0 con 0x7f43d8159540 2024-04-25T10:31:17.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43d5600640 1 -- 172.21.15.45:0/2327949844 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43d00165e0 con 0x7f43d8159540 2024-04-25T10:31:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43b4066f60 msgr2=0x7f43b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43b4066f60 0x7f43b4069420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f43c800b810 tx=0x7f43c8008040 comp rx=0 tx=0).stop 2024-04-25T10:31:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 msgr2=0x7f43d815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 0x7f43d815dc90 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f43d00af040 tx=0x7f43d0019760 comp rx=0 tx=0).stop 2024-04-25T10:31:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 shutdown_connections 2024-04-25T10:31:17.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f43b4066f60 0x7f43b4069420 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43d8159540 0x7f43d815dc90 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43d80aeda0 0x7f43d8143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 --2- 172.21.15.45:0/2327949844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43d80721f0 0x7f43d815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 >> 172.21.15.45:0/2327949844 conn(0x7f43d80669a0 msgr2=0x7f43d80cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 shutdown_connections 2024-04-25T10:31:17.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:17.739+0000 7f43dfc00640 1 -- 172.21.15.45:0/2327949844 wait complete. 2024-04-25T10:31:17.762 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:31:17.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:17.866 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:17.989 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:31:17.989 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:17.989 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:17.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:17.998 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:17.998 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:18.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:18.007 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:18.007 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:18.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:18.048 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:18.048 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:18.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:18.058 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:31:18.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.231+0000 7fc21d000640 1 Processor -- start 2024-04-25T10:31:18.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.231+0000 7fc21d000640 1 -- start start 2024-04-25T10:31:18.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc21d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2180b0f70 0x7fc2180b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc21d000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21815f3b0 con 0x7fc2180cd2e0 2024-04-25T10:31:18.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc21d000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180ca810 con 0x7fc2180cfd90 2024-04-25T10:31:18.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc21d000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815ef20 con 0x7fc2180b0f70 2024-04-25T10:31:18.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc216800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc2180cd2e0 0x7fc2180cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41596/0 (socket says 172.21.15.45:41596) 2024-04-25T10:31:18.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc216800640 1 -- 172.21.15.45:0/3634339629 learned_addr learned my addr 172.21.15.45:0/3634339629 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2180b0f70 0x7fc2180b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331369416 0 0) 0x7fc21815f3b0 con 0x7fc2180cd2e0 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1fc003530 con 0x7fc2180cd2e0 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3522990048 0 0) 0x7fc2180ca810 con 0x7fc2180cfd90 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc21815f3b0 con 0x7fc2180cfd90 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434075591 0 0) 0x7fc1fc003530 con 0x7fc2180cd2e0 2024-04-25T10:31:18.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2180b0f70 msgr2=0x7fc2180b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2180b0f70 0x7fc2180b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 >> v1:172.21.15.117:6789/0 conn(0x7fc2180cfd90 legacy=0x7fc2180ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180b1890 con 0x7fc2180cd2e0 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fc210001a00 con 0x7fc2180cd2e0 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 >> v1:172.21.15.45:6789/0 conn(0x7fc2180cd2e0 legacy=0x7fc2180cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc1fc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1fc004810 0x7fc1fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1fc0051d0 0x7fc1fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815ef20 con 0x7fc1fc003b80 2024-04-25T10:31:18.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc00c020 con 0x7fc1fc0051d0 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc00c220 con 0x7fc1fc004810 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2180b0f70 0x7fc2180b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc1fc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc216800640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1fc0051d0 0x7fc1fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc20fe00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1fc004810 0x7fc1fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 -- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1fc004810 msgr2=0x7fc1fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1fc004810 0x7fc1fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 -- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1fc0051d0 msgr2=0x7fc1fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1fc0051d0 0x7fc1fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.235+0000 7fc215e00640 1 -- 172.21.15.45:0/3634339629 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc003530 con 0x7fc1fc003b80 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc215e00640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc1fc003fa0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fc2180d12c0 tx=0x7fc2180cc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139623a37da55ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ac070 con 0x7fc1fc003b80 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208004030 con 0x7fc1fc003b80 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc215400640 1 -- 172.21.15.45:0/3634339629 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080097f0 con 0x7fc1fc003b80 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc21d000640 1 -- 172.21.15.45:0/3634339629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 msgr2=0x7fc1fc003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc21d000640 1 --2- 172.21.15.45:0/3634339629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc1fc003fa0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fc2180d12c0 tx=0x7fc2180cc940 comp rx=0 tx=0).stop 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc216800640 1 -- 172.21.15.45:0/3634339629 reap_dead start 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc21d000640 1 -- 172.21.15.45:0/3634339629 shutdown_connections 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc21d000640 1 -- 172.21.15.45:0/3634339629 >> 172.21.15.45:0/3634339629 conn(0x7fc2180b1d50 msgr2=0x7fc2180b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc21d000640 1 -- 172.21.15.45:0/3634339629 shutdown_connections 2024-04-25T10:31:18.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.239+0000 7fc21d000640 1 -- 172.21.15.45:0/3634339629 wait complete. 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 Processor -- start 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- start start 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc218168790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc218099b20 0x7fc218099290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218154b70 0x7fc218165d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218169ef0 con 0x7fc1fc003b80 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218169d70 con 0x7fc218154b70 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21816a070 con 0x7fc218099b20 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc218168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc218168790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32984/0 (socket says 172.21.15.45:32984) 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 -- 172.21.15.45:0/2297606853 learned_addr learned my addr 172.21.15.45:0/2297606853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc20fe00640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc218099b20 0x7fc218099290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc216800640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218154b70 0x7fc218165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 -- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc218099b20 msgr2=0x7fc218099290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc218099b20 0x7fc218099290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 -- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218154b70 msgr2=0x7fc218165d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218154b70 0x7fc218165d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 -- 172.21.15.45:0/2297606853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180997d0 con 0x7fc1fc003b80 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc215e00640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc218168790 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc20801c8e0 tx=0x7fc1fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b7020 con 0x7fc1fc003b80 2024-04-25T10:31:18.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208003820 con 0x7fc1fc003b80 2024-04-25T10:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21815e100 con 0x7fc1fc003b80 2024-04-25T10:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ac040 con 0x7fc1fc003b80 2024-04-25T10:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc218167f90 con 0x7fc1fc003b80 2024-04-25T10:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.243+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21804ef30 con 0x7fc1fc003b80 2024-04-25T10:31:18.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.247+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2080ba020 con 0x7fc1fc003b80 2024-04-25T10:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.247+0000 7fc20e000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.247+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc20811ecf0 con 0x7fc1fc003b80 2024-04-25T10:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.247+0000 7fc20fe00640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.247+0000 7fc20fe00640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc2180c6e00 tx=0x7fc200009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.251+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2080c1030 con 0x7fc1fc003b80 2024-04-25T10:31:18.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2180cda20 con 0x7fc1fc003b80 2024-04-25T10:31:18.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc20e000640 1 -- 172.21.15.45:0/2297606853 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+3711 (secure 0 0 0) 0x7fc2080167c0 con 0x7fc1fc003b80 2024-04-25T10:31:18.449 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:18.449 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":124,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/1557073206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1557073206},{"type":"v1","addr":"172.21.15.117:6836","nonce":1557073206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":8684,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2569564833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":2569564833},{"type":"v1","addr":"172.21.15.117:6837","nonce":2569564833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124},{"gid":9742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1150326686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1150326686},{"type":"v1","addr":"172.21.15.45:6834","nonce":1150326686}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":10461,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3694143545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3694143545},{"type":"v1","addr":"172.21.15.45:6835","nonce":3694143545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118}],"filesystems":[]} 2024-04-25T10:31:18.450 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 124 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 msgr2=0x7fc1f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc2180c6e00 tx=0x7fc200009040 comp rx=0 tx=0).stop 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 msgr2=0x7fc218168790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc218168790 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fc20801c8e0 tx=0x7fc1fc005840 comp rx=0 tx=0).stop 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 shutdown_connections 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc218154b70 0x7fc218165d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc218099b20 0x7fc218099290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 --2- 172.21.15.45:0/2297606853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1fc003b80 0x7fc218168790 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.447+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 >> 172.21.15.45:0/2297606853 conn(0x7fc2180b1d50 msgr2=0x7fc21806e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.451+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 shutdown_connections 2024-04-25T10:31:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.451+0000 7fc21d000640 1 -- 172.21.15.45:0/2297606853 wait complete. 2024-04-25T10:31:18.471 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:31:18.471 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:31:18.472 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:31:18.472 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:18.472 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:31:18.472 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:31:18.472 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:31:18.472 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:18.472 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:31:18.472 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:31:18.473 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:31:18.473 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:18.473 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:31:18.473 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:31:18.473 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:31:18.473 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:18.486 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:31:18.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:31:18.488 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:31:18.488 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:31:18.490 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:31:18.491 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:31:18.491 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:31:18.492 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:31:18.533 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:31:18.534 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:18.535 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:31:18.535 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:18.540 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:31:18.540 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:18.542 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:31:18.542 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:18.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:31:18.622 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:31:18.623 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:31:18.631 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:31:18.632 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.687+0000 7fa0d0200640 1 Processor -- start 2024-04-25T10:31:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.687+0000 7fa0d0200640 1 -- start start 2024-04-25T10:31:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0d0200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0c81550b0 0x7fa0c80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0d0200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0d0200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c8071050 con 0x7fa0c80cfe70 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0d0200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c815bda0 con 0x7fa0c81550b0 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0d0200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c815d040 con 0x7fa0c815b910 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0c81550b0 0x7fa0c80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0c81550b0 0x7fa0c80cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33036/0 (socket says 172.21.15.45:33036) 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 -- 172.21.15.45:0/2980423027 learned_addr learned my addr 172.21.15.45:0/2980423027 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0ce200640 1 --2- 172.21.15.45:0/2980423027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 -- 172.21.15.45:0/2980423027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 msgr2=0x7fa0c80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 --2- 172.21.15.45:0/2980423027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 -- 172.21.15.45:0/2980423027 >> v1:172.21.15.117:6789/0 conn(0x7fa0c80cfe70 legacy=0x7fa0c815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.691+0000 7fa0cd800640 1 -- 172.21.15.45:0/2980423027 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80712f0 con 0x7fa0c81550b0 2024-04-25T10:31:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0cd800640 1 --2- 172.21.15.45:0/2980423027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c81550b0 0x7fa0c80cf930 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098a0 tx=0x7fa0bc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9004b49a09f6c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2980423027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0ab430 con 0x7fa0c81550b0 2024-04-25T10:31:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2980423027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0b0070 con 0x7fa0c81550b0 2024-04-25T10:31:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2980423027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0ab7f0 con 0x7fa0c81550b0 2024-04-25T10:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- 172.21.15.45:0/2980423027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c81550b0 msgr2=0x7fa0c80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 --2- 172.21.15.45:0/2980423027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c81550b0 0x7fa0c80cf930 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098a0 tx=0x7fa0bc01c540 comp rx=0 tx=0).stop 2024-04-25T10:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- 172.21.15.45:0/2980423027 shutdown_connections 2024-04-25T10:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 --2- 172.21.15.45:0/2980423027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 --2- 172.21.15.45:0/2980423027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c81550b0 0x7fa0c80cf930 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- 172.21.15.45:0/2980423027 >> 172.21.15.45:0/2980423027 conn(0x7fa0c8066b70 msgr2=0x7fa0c8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- 172.21.15.45:0/2980423027 shutdown_connections 2024-04-25T10:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- 172.21.15.45:0/2980423027 wait complete. 2024-04-25T10:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 Processor -- start 2024-04-25T10:31:18.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- start start 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 0x7fa0c81495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81550b0 0x7fa0c81510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c8153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c815c030 con 0x7fa0c80cfe70 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c80720b0 con 0x7fa0c81550b0 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0d0200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c815c1b0 con 0x7fa0c815b910 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0cd800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 0x7fa0c81495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0cd800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 0x7fa0c81495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33042/0 (socket says 172.21.15.45:33042) 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.695+0000 7fa0cd800640 1 -- 172.21.15.45:0/1178944127 learned_addr learned my addr 172.21.15.45:0/1178944127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81550b0 0x7fa0c81510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cd800640 1 -- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 msgr2=0x7fa0c8153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0ce200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c8153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cd800640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c8153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cd800640 1 -- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81550b0 msgr2=0x7fa0c81510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cd800640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81550b0 0x7fa0c81510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cd800640 1 -- 172.21.15.45:0/1178944127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c814fbc0 con 0x7fa0c80cfe70 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81550b0 0x7fa0c81510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0ce200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c8153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cd800640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 0x7fa0c81495d0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0ad040 tx=0x7fa0bc0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0abe00 con 0x7fa0c80cfe70 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0bb070 con 0x7fa0c80cfe70 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c814fea0 con 0x7fa0c80cfe70 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0b4660 con 0x7fa0c80cfe70 2024-04-25T10:31:18.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c8150460 con 0x7fa0c80cfe70 2024-04-25T10:31:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c804ef30 con 0x7fa0c80cfe70 2024-04-25T10:31:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0bc0b4800 con 0x7fa0c80cfe70 2024-04-25T10:31:18.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0c5600640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0a8066f60 0x7fa0a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa0bc11eb60 con 0x7fa0c80cfe70 2024-04-25T10:31:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.699+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0a8066f60 0x7fa0a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:18.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.703+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0a8066f60 0x7fa0a8069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa0c81507e0 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:18.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.703+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0bc0c5020 con 0x7fa0c80cfe70 2024-04-25T10:31:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:18.883+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa0c815d1c0 con 0x7fa0c80cfe70 2024-04-25T10:31:19.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.395+0000 7fa0c5600640 1 -- 172.21.15.45:0/1178944127 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v176) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa0bc016130 con 0x7fa0c80cfe70 2024-04-25T10:31:19.399 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:31:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0a8066f60 msgr2=0x7fa0a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0a8066f60 0x7fa0a8069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa0c81507e0 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2024-04-25T10:31:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 msgr2=0x7fa0c81495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 0x7fa0c81495d0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0ad040 tx=0x7fa0bc0192e0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 shutdown_connections 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa0a8066f60 0x7fa0a8069420 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0c815b910 0x7fa0c8153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81550b0 0x7fa0c81510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 --2- 172.21.15.45:0/1178944127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c80cfe70 0x7fa0c81495d0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 >> 172.21.15.45:0/1178944127 conn(0x7fa0c8066b70 msgr2=0x7fa0c80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 shutdown_connections 2024-04-25T10:31:19.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.399+0000 7fa0d0200640 1 -- 172.21.15.45:0/1178944127 wait complete. 2024-04-25T10:31:19.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:31:19.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 Processor -- start 2024-04-25T10:31:19.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 -- start start 2024-04-25T10:31:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14241550b0 0x7f14240cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f142415b740 0x7f142406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1424071050 con 0x7f142415b740 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14240712f0 con 0x7f14241550b0 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.563+0000 7f142a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14240715d0 con 0x7f14240cfe70 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f142415b740 0x7f142406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f142415b740 0x7f142406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33052/0 (socket says 172.21.15.45:33052) 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 -- 172.21.15.45:0/3669828773 learned_addr learned my addr 172.21.15.45:0/3669828773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f141aa00640 1 --2- 172.21.15.45:0/3669828773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423400640 1 --2- 172.21.15.45:0/3669828773 >> v2:172.21.15.117:3300/0 conn(0x7f14241550b0 0x7f14240cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 -- 172.21.15.45:0/3669828773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 msgr2=0x7f142415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 --2- 172.21.15.45:0/3669828773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 -- 172.21.15.45:0/3669828773 >> v2:172.21.15.117:3300/0 conn(0x7f14241550b0 msgr2=0x7f14240cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 --2- 172.21.15.45:0/3669828773 >> v2:172.21.15.117:3300/0 conn(0x7f14241550b0 0x7f14240cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 -- 172.21.15.45:0/3669828773 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14240718b0 con 0x7f142415b740 2024-04-25T10:31:19.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423400640 1 --2- 172.21.15.45:0/3669828773 >> v2:172.21.15.117:3300/0 conn(0x7f14241550b0 0x7f14240cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:19.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1423e00640 1 --2- 172.21.15.45:0/3669828773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142415b740 0x7f142406d1a0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f1424067980 tx=0x7f141c01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c3d092bc5e449 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1422a00640 1 -- 172.21.15.45:0/3669828773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c00b2d0 con 0x7f142415b740 2024-04-25T10:31:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1422a00640 1 -- 172.21.15.45:0/3669828773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c00b470 con 0x7f142415b740 2024-04-25T10:31:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.567+0000 7f1422a00640 1 -- 172.21.15.45:0/3669828773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0a7860 con 0x7f142415b740 2024-04-25T10:31:19.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- 172.21.15.45:0/3669828773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142415b740 msgr2=0x7f142406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- 172.21.15.45:0/3669828773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142415b740 0x7f142406d1a0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f1424067980 tx=0x7f141c01e930 comp rx=0 tx=0).stop 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- 172.21.15.45:0/3669828773 shutdown_connections 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- 172.21.15.45:0/3669828773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142415b740 0x7f142406d1a0 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- 172.21.15.45:0/3669828773 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- 172.21.15.45:0/3669828773 >> v2:172.21.15.117:3300/0 conn(0x7f14241550b0 0x7f14240cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- 172.21.15.45:0/3669828773 >> 172.21.15.45:0/3669828773 conn(0x7f1424066b70 msgr2=0x7f1424066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- 172.21.15.45:0/3669828773 shutdown_connections 2024-04-25T10:31:19.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- 172.21.15.45:0/3669828773 wait complete. 2024-04-25T10:31:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 Processor -- start 2024-04-25T10:31:19.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- start start 2024-04-25T10:31:19.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 0x7f142406c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142415b740 0x7f142406cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14240b33e0 con 0x7f14241550b0 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14240b3260 con 0x7f142415b740 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f142a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14240b3560 con 0x7f14240cfe70 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 0x7f142406c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 0x7f142406c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33068/0 (socket says 172.21.15.45:33068) 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 -- 172.21.15.45:0/812663396 learned_addr learned my addr 172.21.15.45:0/812663396 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f1423e00640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142415b740 0x7f142406cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f1423400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 -- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 msgr2=0x7f142414b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142414b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 -- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142415b740 msgr2=0x7f142406cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142415b740 0x7f142406cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 -- 172.21.15.45:0/812663396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142414a780 con 0x7f14241550b0 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f1423e00640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142415b740 0x7f142406cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.571+0000 7f141aa00640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 0x7f142406c630 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f140c00d800 tx=0x7f140c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c00b6c0 con 0x7f14241550b0 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c019070 con 0x7f14241550b0 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c00ba60 con 0x7f14241550b0 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142414aa20 con 0x7f14241550b0 2024-04-25T10:31:19.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142414afe0 con 0x7f14241550b0 2024-04-25T10:31:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142404ef30 con 0x7f14241550b0 2024-04-25T10:31:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f140c00bc00 con 0x7f14241550b0 2024-04-25T10:31:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f141be00640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1400066f60 0x7f1400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f140c085190 con 0x7f14241550b0 2024-04-25T10:31:19.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.575+0000 7f1423400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1400066f60 0x7f1400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.579+0000 7f1423400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1400066f60 0x7f1400069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1414009830 tx=0x7f1414008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:19.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.579+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f140c0564e0 con 0x7f14241550b0 2024-04-25T10:31:19.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.795+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1424150030 con 0x7f14241550b0 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.795+0000 7f141be00640 1 -- 172.21.15.45:0/812663396 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f140c055de0 con 0x7f14241550b0 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.798 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.799 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.801 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.802 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:31:19.803 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:31:19.805 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:31:19.808 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:31:19.809 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:31:19.811 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:31:19.812 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:19.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:19.812 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:31:19.812 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1400066f60 msgr2=0x7f1400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1400066f60 0x7f1400069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f1414009830 tx=0x7f1414008040 comp rx=0 tx=0).stop 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 msgr2=0x7f142406c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 0x7f142406c630 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f140c00d800 tx=0x7f140c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 shutdown_connections 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1400066f60 0x7f1400069420 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142415b740 0x7f142406cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14241550b0 0x7f142406c630 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 --2- 172.21.15.45:0/812663396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14240cfe70 0x7f142414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 >> 172.21.15.45:0/812663396 conn(0x7f1424066b70 msgr2=0x7f142415daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 shutdown_connections 2024-04-25T10:31:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.799+0000 7f142a400640 1 -- 172.21.15.45:0/812663396 wait complete. 2024-04-25T10:31:19.819 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:19.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 Processor -- start 2024-04-25T10:31:19.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 -- start start 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc2c40d1820 0x7fc2c40d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2c415e050 0x7fc2c40b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40b1af0 con 0x7fc2c40b13e0 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40b1dd0 con 0x7fc2c415e050 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c8c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40b20b0 con 0x7fc2c40d1820 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c1400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2c415e050 0x7fc2c40b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c2800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc2c40b13e0 0x7fc2c40ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41598/0 (socket says 172.21.15.45:41598) 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c2800640 1 -- 172.21.15.45:0/190761012 learned_addr learned my addr 172.21.15.45:0/190761012 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:19.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c1e00640 1 --2- 172.21.15.45:0/190761012 >> v2:172.21.15.117:3300/0 conn(0x7fc2c40d1820 0x7fc2c40d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c0a00640 1 -- 172.21.15.45:0/190761012 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549462813 0 0) 0x7fc2c40b1af0 con 0x7fc2c40b13e0 2024-04-25T10:31:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c1400640 1 -- 172.21.15.45:0/190761012 >> v2:172.21.15.117:3300/0 conn(0x7fc2c40d1820 msgr2=0x7fc2c40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c1400640 1 --2- 172.21.15.45:0/190761012 >> v2:172.21.15.117:3300/0 conn(0x7fc2c40d1820 0x7fc2c40d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c1400640 1 -- 172.21.15.45:0/190761012 >> v1:172.21.15.45:6789/0 conn(0x7fc2c40b13e0 legacy=0x7fc2c40ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.959+0000 7fc2c1400640 1 -- 172.21.15.45:0/190761012 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40b2390 con 0x7fc2c415e050 2024-04-25T10:31:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c1400640 1 --2- 172.21.15.45:0/190761012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c40b0ea0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4070e90 tx=0x7fc2b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6762a5b0dd4ee8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c0a00640 1 -- 172.21.15.45:0/190761012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00aa600 con 0x7fc2c415e050 2024-04-25T10:31:19.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c0a00640 1 -- 172.21.15.45:0/190761012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b00af070 con 0x7fc2c415e050 2024-04-25T10:31:19.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c0a00640 1 -- 172.21.15.45:0/190761012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00aa9c0 con 0x7fc2c415e050 2024-04-25T10:31:19.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 -- 172.21.15.45:0/190761012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 msgr2=0x7fc2c40b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/190761012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c40b0ea0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4070e90 tx=0x7fc2b001c480 comp rx=0 tx=0).stop 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 -- 172.21.15.45:0/190761012 shutdown_connections 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/190761012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c40b0ea0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/190761012 >> v2:172.21.15.117:3300/0 conn(0x7fc2c40d1820 0x7fc2c40d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 -- 172.21.15.45:0/190761012 >> 172.21.15.45:0/190761012 conn(0x7fc2c409a9d0 msgr2=0x7fc2c409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 -- 172.21.15.45:0/190761012 shutdown_connections 2024-04-25T10:31:19.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 -- 172.21.15.45:0/190761012 wait complete. 2024-04-25T10:31:19.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 Processor -- start 2024-04-25T10:31:19.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.963+0000 7fc2c8c00640 1 -- start start 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2c40b13e0 0x7fc2c415e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2c40d1820 0x7fc2c4154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c409b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c416ab20 con 0x7fc2c415e050 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c416a9a0 con 0x7fc2c40b13e0 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c416aca0 con 0x7fc2c40d1820 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c409b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c409b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33078/0 (socket says 172.21.15.45:33078) 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 -- 172.21.15.45:0/860880382 learned_addr learned my addr 172.21.15.45:0/860880382 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c1e00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2c40b13e0 0x7fc2c415e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 -- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2c40d1820 msgr2=0x7fc2c4154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2c40d1820 0x7fc2c4154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 -- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2c40b13e0 msgr2=0x7fc2c415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2c40b13e0 0x7fc2c415e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 -- 172.21.15.45:0/860880382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c409d8c0 con 0x7fc2c415e050 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c1e00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2c40b13e0 0x7fc2c415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c2800640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c409b790 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c7a0 tx=0x7fc2bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc018070 con 0x7fc2c415e050 2024-04-25T10:31:19.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc00a5e0 con 0x7fc2c415e050 2024-04-25T10:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00aa50 con 0x7fc2c415e050 2024-04-25T10:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c409dba0 con 0x7fc2c415e050 2024-04-25T10:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c409e440 con 0x7fc2c415e050 2024-04-25T10:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.967+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c409df90 con 0x7fc2c415e050 2024-04-25T10:31:19.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.971+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2bc01b070 con 0x7fc2c415e050 2024-04-25T10:31:19.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.971+0000 7fc2ba000640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2a4066f60 0x7fc2a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:19.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.971+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc2bc084310 con 0x7fc2c415e050 2024-04-25T10:31:19.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.971+0000 7fc2c1e00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2a4066f60 0x7fc2a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:19.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.971+0000 7fc2c1e00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2a4066f60 0x7fc2a4069420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009ae0 tx=0x7fc2b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:19.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:19.975+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2bc055660 con 0x7fc2c415e050 2024-04-25T10:31:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.135+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2c404ef30 con 0x7fc2c415e050 2024-04-25T10:31:20.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2ba000640 1 -- 172.21.15.45:0/860880382 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fc2bc054f60 con 0x7fc2c415e050 2024-04-25T10:31:20.140 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:20.140 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":178,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:19.409386+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2a4066f60 msgr2=0x7fc2a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2a4066f60 0x7fc2a4069420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009ae0 tx=0x7fc2b4008040 comp rx=0 tx=0).stop 2024-04-25T10:31:20.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 msgr2=0x7fc2c409b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c409b790 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c7a0 tx=0x7fc2bc00cc60 comp rx=0 tx=0).stop 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 shutdown_connections 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc2a4066f60 0x7fc2a4069420 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2c415e050 0x7fc2c409b790 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2c40d1820 0x7fc2c4154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 --2- 172.21.15.45:0/860880382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2c40b13e0 0x7fc2c415e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 >> 172.21.15.45:0/860880382 conn(0x7fc2c409a9d0 msgr2=0x7fc2c40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 shutdown_connections 2024-04-25T10:31:20.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.139+0000 7fc2c8c00640 1 -- 172.21.15.45:0/860880382 wait complete. 2024-04-25T10:31:20.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:20.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:20.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25fd000640 1 Processor -- start 2024-04-25T10:31:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25fd000640 1 -- start start 2024-04-25T10:31:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25fd000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25f80b38d0 0x7f25f815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25fd000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f815b080 con 0x7f25f8070350 2024-04-25T10:31:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25fd000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f815adb0 con 0x7f25f80aef80 2024-04-25T10:31:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25fd000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f815eb60 con 0x7f25f80b38d0 2024-04-25T10:31:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25f6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25f80b38d0 0x7f25f815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25f6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f25f80b38d0 0x7f25f815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33092/0 (socket says 172.21.15.45:33092) 2024-04-25T10:31:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25f6800640 1 -- 172.21.15.45:0/3669080109 learned_addr learned my addr 172.21.15.45:0/3669080109 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25efe00640 1 --1- 172.21.15.45:0/3669080109 >> v1:172.21.15.45:6789/0 conn(0x7f25f8070350 0x7f25f80b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41612/0 (socket says 172.21.15.45:41612) 2024-04-25T10:31:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25f6800640 1 -- 172.21.15.45:0/3669080109 >> v1:172.21.15.117:6789/0 conn(0x7f25f80aef80 legacy=0x7f25f815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25f6800640 1 -- 172.21.15.45:0/3669080109 >> v1:172.21.15.45:6789/0 conn(0x7f25f8070350 legacy=0x7f25f80b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.303+0000 7f25f6800640 1 -- 172.21.15.45:0/3669080109 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f815ee40 con 0x7f25f80b38d0 2024-04-25T10:31:20.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25f6800640 1 --2- 172.21.15.45:0/3669080109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f80b38d0 0x7f25f815e480 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009e50 tx=0x7f25f001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d27bd0cc3429f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25f5400640 1 -- 172.21.15.45:0/3669080109 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f00a58e0 con 0x7f25f80b38d0 2024-04-25T10:31:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25f5400640 1 -- 172.21.15.45:0/3669080109 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f00a5a80 con 0x7f25f80b38d0 2024-04-25T10:31:20.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25f5400640 1 -- 172.21.15.45:0/3669080109 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f00a5e50 con 0x7f25f80b38d0 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 -- 172.21.15.45:0/3669080109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f80b38d0 msgr2=0x7f25f815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 --2- 172.21.15.45:0/3669080109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f80b38d0 0x7f25f815e480 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009e50 tx=0x7f25f001ca30 comp rx=0 tx=0).stop 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 -- 172.21.15.45:0/3669080109 shutdown_connections 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 --2- 172.21.15.45:0/3669080109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f80b38d0 0x7f25f815e480 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 -- 172.21.15.45:0/3669080109 >> 172.21.15.45:0/3669080109 conn(0x7f25f80669a0 msgr2=0x7f25f8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 -- 172.21.15.45:0/3669080109 shutdown_connections 2024-04-25T10:31:20.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.307+0000 7f25fd000640 1 -- 172.21.15.45:0/3669080109 wait complete. 2024-04-25T10:31:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 Processor -- start 2024-04-25T10:31:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 -- start start 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f8070350 0x7f25f8149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80aef80 0x7f25f8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 0x7f25f814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81689b0 con 0x7f25f806cdd0 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8168830 con 0x7f25f80aef80 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8168b30 con 0x7f25f8070350 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 0x7f25f814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 0x7f25f814a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33094/0 (socket says 172.21.15.45:33094) 2024-04-25T10:31:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 -- 172.21.15.45:0/2881618376 learned_addr learned my addr 172.21.15.45:0/2881618376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f5e00640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f8070350 0x7f25f8149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25efe00640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80aef80 0x7f25f8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 -- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f8070350 msgr2=0x7f25f8149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f8070350 0x7f25f8149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 -- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80aef80 msgr2=0x7f25f8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80aef80 0x7f25f8159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 -- 172.21.15.45:0/2881618376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8149b10 con 0x7f25f806cdd0 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f5e00640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f8070350 0x7f25f8149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25efe00640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80aef80 0x7f25f8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25f6800640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 0x7f25f814a8d0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f25f00af040 tx=0x7f25f00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f00bc020 con 0x7f25f806cdd0 2024-04-25T10:31:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f00b8470 con 0x7f25f806cdd0 2024-04-25T10:31:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f00a58e0 con 0x7f25f806cdd0 2024-04-25T10:31:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f8149df0 con 0x7f25f806cdd0 2024-04-25T10:31:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.311+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f814a3b0 con 0x7f25f806cdd0 2024-04-25T10:31:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.315+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f804ef30 con 0x7f25f806cdd0 2024-04-25T10:31:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.315+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25f00bf020 con 0x7f25f806cdd0 2024-04-25T10:31:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.315+0000 7f25ee000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d4066fe0 0x7f25d40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.315+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25f011fdb0 con 0x7f25f806cdd0 2024-04-25T10:31:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.315+0000 7f25f5e00640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d4066fe0 0x7f25d40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.315+0000 7f25f5e00640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d4066fe0 0x7f25d40694a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f25f80cd9a0 tx=0x7f25e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:20.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.319+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25f0124070 con 0x7f25f806cdd0 2024-04-25T10:31:20.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.483+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25f80b2fa0 con 0x7f25f806cdd0 2024-04-25T10:31:20.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.483+0000 7f25ee000640 1 -- 172.21.15.45:0/2881618376 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25f00c6020 con 0x7f25f806cdd0 2024-04-25T10:31:20.487 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d4066fe0 msgr2=0x7f25d40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d4066fe0 0x7f25d40694a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f25f80cd9a0 tx=0x7f25e8009040 comp rx=0 tx=0).stop 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 msgr2=0x7f25f814a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 0x7f25f814a8d0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f25f00af040 tx=0x7f25f00197d0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 shutdown_connections 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f25d4066fe0 0x7f25d40694a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25f806cdd0 0x7f25f814a8d0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25f80aef80 0x7f25f8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 --2- 172.21.15.45:0/2881618376 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f25f8070350 0x7f25f8149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 >> 172.21.15.45:0/2881618376 conn(0x7f25f80669a0 msgr2=0x7f25f80cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 shutdown_connections 2024-04-25T10:31:20.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.487+0000 7f25fd000640 1 -- 172.21.15.45:0/2881618376 wait complete. 2024-04-25T10:31:20.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:31:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.647+0000 7f3158a00640 1 Processor -- start 2024-04-25T10:31:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.647+0000 7f3158a00640 1 -- start start 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.647+0000 7f3158a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3154146ed0 0x7f31541472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.647+0000 7f3158a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31540726b0 0x7f3154071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.647+0000 7f3158a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315406c600 con 0x7f3154071c80 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3158a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3154161980 con 0x7f3154146ed0 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3158a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3154072080 con 0x7f31540726b0 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3154146ed0 0x7f31541472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3152800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3154071c80 0x7f315406c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41618/0 (socket says 172.21.15.45:41618) 2024-04-25T10:31:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3152800640 1 -- 172.21.15.45:0/3751674100 learned_addr learned my addr 172.21.15.45:0/3751674100 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f314be00640 1 --2- 172.21.15.45:0/3751674100 >> v2:172.21.15.117:3300/0 conn(0x7f31540726b0 0x7f3154071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151400640 1 -- 172.21.15.45:0/3751674100 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587106048 0 0) 0x7f315406c600 con 0x7f3154071c80 2024-04-25T10:31:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151400640 1 -- 172.21.15.45:0/3751674100 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3138003530 con 0x7f3154071c80 2024-04-25T10:31:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151e00640 1 -- 172.21.15.45:0/3751674100 >> v2:172.21.15.117:3300/0 conn(0x7f31540726b0 msgr2=0x7f3154071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151e00640 1 --2- 172.21.15.45:0/3751674100 >> v2:172.21.15.117:3300/0 conn(0x7f31540726b0 0x7f3154071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151e00640 1 -- 172.21.15.45:0/3751674100 >> v1:172.21.15.45:6789/0 conn(0x7f3154071c80 legacy=0x7f315406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151e00640 1 -- 172.21.15.45:0/3751674100 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3154161bd0 con 0x7f3154146ed0 2024-04-25T10:31:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f314be00640 1 --2- 172.21.15.45:0/3751674100 >> v2:172.21.15.117:3300/0 conn(0x7f31540726b0 0x7f3154071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151e00640 1 --2- 172.21.15.45:0/3751674100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f31541472b0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f315406da70 tx=0x7f314401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3366e7838437ee89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151400640 1 -- 172.21.15.45:0/3751674100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31440ab650 con 0x7f3154146ed0 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151400640 1 -- 172.21.15.45:0/3751674100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31440b0070 con 0x7f3154146ed0 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.651+0000 7f3151400640 1 -- 172.21.15.45:0/3751674100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31440aba10 con 0x7f3154146ed0 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- 172.21.15.45:0/3751674100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 msgr2=0x7f31541472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 --2- 172.21.15.45:0/3751674100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f31541472b0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f315406da70 tx=0x7f314401c450 comp rx=0 tx=0).stop 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- 172.21.15.45:0/3751674100 shutdown_connections 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 --2- 172.21.15.45:0/3751674100 >> v2:172.21.15.117:3300/0 conn(0x7f31540726b0 0x7f3154071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 --2- 172.21.15.45:0/3751674100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f31541472b0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- 172.21.15.45:0/3751674100 >> 172.21.15.45:0/3751674100 conn(0x7f31540669a0 msgr2=0x7f3154066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- 172.21.15.45:0/3751674100 shutdown_connections 2024-04-25T10:31:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- 172.21.15.45:0/3751674100 wait complete. 2024-04-25T10:31:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 Processor -- start 2024-04-25T10:31:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- start start 2024-04-25T10:31:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3154071c80 0x7f315415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31540726b0 0x7f3154140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f3154141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154147bf0 con 0x7f3154146ed0 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154147a70 con 0x7f31540726b0 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3158a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31540ae850 con 0x7f3154071c80 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3152800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f3154141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3152800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f3154141150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33112/0 (socket says 172.21.15.45:33112) 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3152800640 1 -- 172.21.15.45:0/2268641949 learned_addr learned my addr 172.21.15.45:0/2268641949 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f3151e00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3154071c80 0x7f315415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.655+0000 7f314be00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31540726b0 0x7f3154140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3152800640 1 -- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3154071c80 msgr2=0x7f315415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3152800640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3154071c80 0x7f315415bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3152800640 1 -- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31540726b0 msgr2=0x7f3154140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3152800640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31540726b0 0x7f3154140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3152800640 1 -- 172.21.15.45:0/2268641949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315406cf30 con 0x7f3154146ed0 2024-04-25T10:31:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3152800640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f3154141150 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f314c00cb40 tx=0x7f314c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c017070 con 0x7f3154146ed0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c00aca0 con 0x7f3154146ed0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c012620 con 0x7f3154146ed0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315415e620 con 0x7f3154146ed0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315415ce80 con 0x7f3154146ed0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3154144ed0 con 0x7f3154146ed0 2024-04-25T10:31:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f314c01a070 con 0x7f3154146ed0 2024-04-25T10:31:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f314a000640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3130066f90 0x7f3130069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.659+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f314c084a00 con 0x7f3154146ed0 2024-04-25T10:31:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.663+0000 7f3151e00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3130066f90 0x7f3130069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.663+0000 7f3151e00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3130066f90 0x7f3130069450 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f31440af000 tx=0x7f31440a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:20.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.663+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f314c055700 con 0x7f3154146ed0 2024-04-25T10:31:20.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.831+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f315404ef30 con 0x7f3154146ed0 2024-04-25T10:31:20.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.831+0000 7f314a000640 1 -- 172.21.15.45:0/2268641949 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v178) v1 ==== 84+0+65 (secure 0 0 0) 0x7f314c0597a0 con 0x7f3154146ed0 2024-04-25T10:31:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:20.834 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:31:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:31:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:31:20.835 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:20.835 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3130066f90 msgr2=0x7f3130069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3130066f90 0x7f3130069450 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f31440af000 tx=0x7f31440a7000 comp rx=0 tx=0).stop 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 msgr2=0x7f3154141150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f3154141150 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f314c00cb40 tx=0x7f314c00a060 comp rx=0 tx=0).stop 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 shutdown_connections 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3130066f90 0x7f3130069450 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3154146ed0 0x7f3154141150 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31540726b0 0x7f3154140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 --2- 172.21.15.45:0/2268641949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3154071c80 0x7f315415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 >> 172.21.15.45:0/2268641949 conn(0x7f31540669a0 msgr2=0x7f31540af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 shutdown_connections 2024-04-25T10:31:20.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.835+0000 7f3158a00640 1 -- 172.21.15.45:0/2268641949 wait complete. 2024-04-25T10:31:20.856 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:31:20.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:31:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa710e00640 1 Processor -- start 2024-04-25T10:31:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa710e00640 1 -- start start 2024-04-25T10:31:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa710e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0c5fe0 0x7fa70c0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa710e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0d15f0 con 0x7fa70c0ce1d0 2024-04-25T10:31:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa710e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0cd680 con 0x7fa70c0d2090 2024-04-25T10:31:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa710e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0ccde0 con 0x7fa70c0c5fe0 2024-04-25T10:31:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa70b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa70c0ce1d0 0x7fa70c0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41628/0 (socket says 172.21.15.45:41628) 2024-04-25T10:31:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa70b400640 1 -- 172.21.15.45:0/1778041649 learned_addr learned my addr 172.21.15.45:0/1778041649 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:20.998+0000 7fa70be00640 1 --2- 172.21.15.45:0/1778041649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0c5fe0 0x7fa70c0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991731143 0 0) 0x7fa70c0d15f0 con 0x7fa70c0ce1d0 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f0003530 con 0x7fa70c0ce1d0 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462029688 0 0) 0x7fa70c0cd680 con 0x7fa70c0d2090 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c0d15f0 con 0x7fa70c0d2090 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70be00640 1 -- 172.21.15.45:0/1778041649 >> v1:172.21.15.117:6789/0 conn(0x7fa70c0d2090 legacy=0x7fa70c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70be00640 1 -- 172.21.15.45:0/1778041649 >> v1:172.21.15.45:6789/0 conn(0x7fa70c0ce1d0 legacy=0x7fa70c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70be00640 1 -- 172.21.15.45:0/1778041649 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0cd0c0 con 0x7fa70c0c5fe0 2024-04-25T10:31:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70be00640 1 --2- 172.21.15.45:0/1778041649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0c5fe0 0x7fa70c0c6490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa704009dc0 tx=0x7fa70401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2876c6c2fa0bea2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b2070 con 0x7fa70c0c5fe0 2024-04-25T10:31:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040a5a20 con 0x7fa70c0c5fe0 2024-04-25T10:31:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa70aa00640 1 -- 172.21.15.45:0/1778041649 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040a5df0 con 0x7fa70c0c5fe0 2024-04-25T10:31:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa710e00640 1 -- 172.21.15.45:0/1778041649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0c5fe0 msgr2=0x7fa70c0c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.002+0000 7fa710e00640 1 --2- 172.21.15.45:0/1778041649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0c5fe0 0x7fa70c0c6490 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa704009dc0 tx=0x7fa70401c960 comp rx=0 tx=0).stop 2024-04-25T10:31:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- 172.21.15.45:0/1778041649 shutdown_connections 2024-04-25T10:31:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 --2- 172.21.15.45:0/1778041649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0c5fe0 0x7fa70c0c6490 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- 172.21.15.45:0/1778041649 >> 172.21.15.45:0/1778041649 conn(0x7fa70c09a9d0 msgr2=0x7fa70c0d17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- 172.21.15.45:0/1778041649 shutdown_connections 2024-04-25T10:31:21.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- 172.21.15.45:0/1778041649 wait complete. 2024-04-25T10:31:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 Processor -- start 2024-04-25T10:31:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- start start 2024-04-25T10:31:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c0c5fe0 0x7fa70c166dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0ce1d0 0x7fa70c149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 0x7fa70c14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0c86d0 con 0x7fa70c0d2090 2024-04-25T10:31:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0c8550 con 0x7fa70c0c5fe0 2024-04-25T10:31:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa710e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0d2470 con 0x7fa70c0ce1d0 2024-04-25T10:31:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 0x7fa70c14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 0x7fa70c14a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33116/0 (socket says 172.21.15.45:33116) 2024-04-25T10:31:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 -- 172.21.15.45:0/3296819585 learned_addr learned my addr 172.21.15.45:0/3296819585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70b400640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c0c5fe0 0x7fa70c166dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa702a00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0ce1d0 0x7fa70c149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 -- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0ce1d0 msgr2=0x7fa70c149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0ce1d0 0x7fa70c149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 -- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c0c5fe0 msgr2=0x7fa70c166dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c0c5fe0 0x7fa70c166dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 -- 172.21.15.45:0/3296819585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c152340 con 0x7fa70c0d2090 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70b400640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c0c5fe0 0x7fa70c166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.006+0000 7fa70be00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 0x7fa70c14a390 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fa70401a590 tx=0x7fa704019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b2050 con 0x7fa70c0d2090 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040be070 con 0x7fa70c0d2090 2024-04-25T10:31:21.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040adab0 con 0x7fa70c0d2090 2024-04-25T10:31:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c151e30 con 0x7fa70c0d2090 2024-04-25T10:31:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c1528e0 con 0x7fa70c0d2090 2024-04-25T10:31:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c075120 con 0x7fa70c0d2090 2024-04-25T10:31:21.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7040a58a0 con 0x7fa70c0d2090 2024-04-25T10:31:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa703e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6e8066f60 0x7fa6e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa7041200d0 con 0x7fa70c0d2090 2024-04-25T10:31:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.010+0000 7fa70b400640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6e8066f60 0x7fa6e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.014+0000 7fa70b400640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6e8066f60 0x7fa6e8069420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc000f70 tx=0x7fa6fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.014+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa704016580 con 0x7fa70c0d2090 2024-04-25T10:31:21.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.178+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa70c06e850 con 0x7fa70c0d2090 2024-04-25T10:31:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa703e00640 1 -- 172.21.15.45:0/3296819585 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v179) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa7040f1310 con 0x7fa70c0d2090 2024-04-25T10:31:21.408 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:31:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6e8066f60 msgr2=0x7fa6e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6e8066f60 0x7fa6e8069420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc000f70 tx=0x7fa6fc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 msgr2=0x7fa70c14a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 0x7fa70c14a390 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fa70401a590 tx=0x7fa704019700 comp rx=0 tx=0).stop 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 shutdown_connections 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa6e8066f60 0x7fa6e8069420 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa70c0d2090 0x7fa70c14a390 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa70c0ce1d0 0x7fa70c149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 --2- 172.21.15.45:0/3296819585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa70c0c5fe0 0x7fa70c166dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 >> 172.21.15.45:0/3296819585 conn(0x7fa70c09a9d0 msgr2=0x7fa70c0c63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 shutdown_connections 2024-04-25T10:31:21.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.406+0000 7fa710e00640 1 -- 172.21.15.45:0/3296819585 wait complete. 2024-04-25T10:31:21.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:31:21.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.578+0000 7fe8e3e00640 1 Processor -- start 2024-04-25T10:31:21.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.578+0000 7fe8e3e00640 1 -- start start 2024-04-25T10:31:21.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e3e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15bbb0 0x7fe8dc0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8dc0b4660 0x7fe8dc06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e3e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc06ca90 con 0x7fe8dc0cf960 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e3e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc06d050 con 0x7fe8dc0b4660 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e3e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc06cd70 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8dc0b4660 0x7fe8dc06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8dc0b4660 0x7fe8dc06c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33118/0 (socket says 172.21.15.45:33118) 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 -- 172.21.15.45:0/986141067 learned_addr learned my addr 172.21.15.45:0/986141067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e1400640 1 --2- 172.21.15.45:0/986141067 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15bbb0 0x7fe8dc0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 -- 172.21.15.45:0/986141067 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15bbb0 msgr2=0x7fe8dc0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 --2- 172.21.15.45:0/986141067 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15bbb0 0x7fe8dc0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 -- 172.21.15.45:0/986141067 >> v1:172.21.15.117:6789/0 conn(0x7fe8dc0cf960 legacy=0x7fe8dc0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.582+0000 7fe8e0a00640 1 -- 172.21.15.45:0/986141067 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0b2880 con 0x7fe8dc0b4660 2024-04-25T10:31:21.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e0a00640 1 --2- 172.21.15.45:0/986141067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc06c120 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06e8b0 tx=0x7fe8c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213bd59339969ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8dbe00640 1 -- 172.21.15.45:0/986141067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80ab610 con 0x7fe8dc0b4660 2024-04-25T10:31:21.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8dbe00640 1 -- 172.21.15.45:0/986141067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c80b0070 con 0x7fe8dc0b4660 2024-04-25T10:31:21.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8dbe00640 1 -- 172.21.15.45:0/986141067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80ab9d0 con 0x7fe8dc0b4660 2024-04-25T10:31:21.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 -- 172.21.15.45:0/986141067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc0b4660 msgr2=0x7fe8dc06c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/986141067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc06c120 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06e8b0 tx=0x7fe8c801c520 comp rx=0 tx=0).stop 2024-04-25T10:31:21.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 -- 172.21.15.45:0/986141067 shutdown_connections 2024-04-25T10:31:21.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/986141067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc06c120 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/986141067 >> v2:172.21.15.117:3300/0 conn(0x7fe8dc15bbb0 0x7fe8dc0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 -- 172.21.15.45:0/986141067 >> 172.21.15.45:0/986141067 conn(0x7fe8dc066b70 msgr2=0x7fe8dc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:21.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 -- 172.21.15.45:0/986141067 shutdown_connections 2024-04-25T10:31:21.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 -- 172.21.15.45:0/986141067 wait complete. 2024-04-25T10:31:21.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 Processor -- start 2024-04-25T10:31:21.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.586+0000 7fe8e3e00640 1 -- start start 2024-04-25T10:31:21.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc1f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0cf960 0x7fe8dc155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 0x7fe8dc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0ce140 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0cdfc0 con 0x7fe8dc0b4660 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc06c660 con 0x7fe8dc0cf960 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 0x7fe8dc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc1f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 0x7fe8dc0ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33120/0 (socket says 172.21.15.45:33120) 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 -- 172.21.15.45:0/2371327966 learned_addr learned my addr 172.21.15.45:0/2371327966 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e0a00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0cf960 0x7fe8dc155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 -- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0cf960 msgr2=0x7fe8dc155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0cf960 0x7fe8dc155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 -- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0b4660 msgr2=0x7fe8dc1f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc1f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 -- 172.21.15.45:0/2371327966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc1555b0 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e1e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 0x7fe8dc0ad840 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c770 tx=0x7fe8d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4003b80 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4003d20 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400aa90 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc15eaa0 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc1609b0 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc04ef30 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.590+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8d400ac30 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.594+0000 7fe8d9600640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8bc066f60 0x7fe8bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:21.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.594+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe8d4084ce0 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.594+0000 7fe8e1400640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8bc066f60 0x7fe8bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:21.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.594+0000 7fe8e1400640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8bc066f60 0x7fe8bc069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc15e3f0 tx=0x7fe8d0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:21.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.598+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8d40557e0 con 0x7fe8dc15bbb0 2024-04-25T10:31:21.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:21.762+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe8dc06d980 con 0x7fe8dc15bbb0 2024-04-25T10:31:22.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.414+0000 7fe8d9600640 1 -- 172.21.15.45:0/2371327966 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v180) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8d4059880 con 0x7fe8dc15bbb0 2024-04-25T10:31:22.417 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:31:22.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8bc066f60 msgr2=0x7fe8bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:22.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8bc066f60 0x7fe8bc069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc15e3f0 tx=0x7fe8d0009190 comp rx=0 tx=0).stop 2024-04-25T10:31:22.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 msgr2=0x7fe8dc0ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:22.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 0x7fe8dc0ad840 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c770 tx=0x7fe8d400cc30 comp rx=0 tx=0).stop 2024-04-25T10:31:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 shutdown_connections 2024-04-25T10:31:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8bc066f60 0x7fe8bc069420 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8dc15bbb0 0x7fe8dc0ad840 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8dc0cf960 0x7fe8dc155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 --2- 172.21.15.45:0/2371327966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8dc0b4660 0x7fe8dc1f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 >> 172.21.15.45:0/2371327966 conn(0x7fe8dc066b70 msgr2=0x7fe8dc15a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 shutdown_connections 2024-04-25T10:31:22.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.418+0000 7fe8e3e00640 1 -- 172.21.15.45:0/2371327966 wait complete. 2024-04-25T10:31:22.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:31:22.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 Processor -- start 2024-04-25T10:31:22.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 -- start start 2024-04-25T10:31:22.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65d4159540 0x7f65d40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d40aebd0 0x7f65d4071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d4071360 con 0x7f65d4072020 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d4159920 con 0x7f65d4159540 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.602+0000 7f65dae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d415c2f0 con 0x7f65d40aebd0 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65d4159540 0x7f65d40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65d4159540 0x7f65d40ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33132/0 (socket says 172.21.15.45:33132) 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 -- 172.21.15.45:0/1891131948 learned_addr learned my addr 172.21.15.45:0/1891131948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65cbe00640 1 --2- 172.21.15.45:0/1891131948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d40aebd0 0x7f65d4071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 -- 172.21.15.45:0/1891131948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d40aebd0 msgr2=0x7f65d4071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 --2- 172.21.15.45:0/1891131948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d40aebd0 0x7f65d4071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 -- 172.21.15.45:0/1891131948 >> v1:172.21.15.117:6789/0 conn(0x7f65d4072020 legacy=0x7f65d415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:22.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 -- 172.21.15.45:0/1891131948 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d415c5d0 con 0x7f65d4159540 2024-04-25T10:31:22.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3e00640 1 --2- 172.21.15.45:0/1891131948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d4159540 0x7f65d40ae690 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009870 tx=0x7f65c401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dbcea6f4498d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3400640 1 -- 172.21.15.45:0/1891131948 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c40b0070 con 0x7f65d4159540 2024-04-25T10:31:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3400640 1 -- 172.21.15.45:0/1891131948 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c40ab600 con 0x7f65d4159540 2024-04-25T10:31:22.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.606+0000 7f65d3400640 1 -- 172.21.15.45:0/1891131948 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c40ab9d0 con 0x7f65d4159540 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- 172.21.15.45:0/1891131948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d4159540 msgr2=0x7f65d40ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 --2- 172.21.15.45:0/1891131948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d4159540 0x7f65d40ae690 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009870 tx=0x7f65c401c560 comp rx=0 tx=0).stop 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- 172.21.15.45:0/1891131948 shutdown_connections 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 --2- 172.21.15.45:0/1891131948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d40aebd0 0x7f65d4071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 --2- 172.21.15.45:0/1891131948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d4159540 0x7f65d40ae690 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- 172.21.15.45:0/1891131948 >> 172.21.15.45:0/1891131948 conn(0x7f65d40669a0 msgr2=0x7f65d4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- 172.21.15.45:0/1891131948 shutdown_connections 2024-04-25T10:31:22.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- 172.21.15.45:0/1891131948 wait complete. 2024-04-25T10:31:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 Processor -- start 2024-04-25T10:31:22.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- start start 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d4072020 0x7f65d40b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 0x7f65d413e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d4159540 0x7f65d414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4072430 con 0x7f65d40aebd0 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d406c0d0 con 0x7f65d4072020 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65dae00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4168930 con 0x7f65d4159540 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 0x7f65d413e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 0x7f65d413e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33148/0 (socket says 172.21.15.45:33148) 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 -- 172.21.15.45:0/1272649423 learned_addr learned my addr 172.21.15.45:0/1272649423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:22.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65d3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d4072020 0x7f65d40b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65d8e00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d4159540 0x7f65d414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 -- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d4159540 msgr2=0x7f65d414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d4159540 0x7f65d414c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 -- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d4072020 msgr2=0x7f65d40b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d4072020 0x7f65d40b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.610+0000 7f65cbe00640 1 -- 172.21.15.45:0/1272649423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d413e590 con 0x7f65d40aebd0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65cbe00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 0x7f65d413e050 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00e960 tx=0x7f65bc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc017070 con 0x7f65d40aebd0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc00ca00 con 0x7f65d40aebd0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00cdb0 con 0x7f65d40aebd0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d414c760 con 0x7f65d40aebd0 2024-04-25T10:31:22.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d414cce0 con 0x7f65d40aebd0 2024-04-25T10:31:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d404ef30 con 0x7f65d40aebd0 2024-04-25T10:31:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65bc01a070 con 0x7f65d40aebd0 2024-04-25T10:31:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d0c00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b0066f60 0x7f65b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f65bc084f70 con 0x7f65d40aebd0 2024-04-25T10:31:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.614+0000 7f65d3e00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b0066f60 0x7f65b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:22.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.618+0000 7f65d3e00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b0066f60 0x7f65b0069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f65c40ad000 tx=0x7f65c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:22.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.622+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65d413e590 con 0x7f65d40aebd0 2024-04-25T10:31:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:22.818+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65d4166700 con 0x7f65d40aebd0 2024-04-25T10:31:23.434 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 26 and data pool 27 2024-04-25T10:31:23.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.430+0000 7f65d0c00640 1 -- 172.21.15.45:0/1272649423 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v127) v1 ==== 174+0+0 (secure 0 0 0) 0x7f65bc055980 con 0x7f65d40aebd0 2024-04-25T10:31:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b0066f60 msgr2=0x7f65b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b0066f60 0x7f65b0069420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f65c40ad000 tx=0x7f65c4008040 comp rx=0 tx=0).stop 2024-04-25T10:31:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 msgr2=0x7f65d413e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 0x7f65d413e050 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00e960 tx=0x7f65bc00ee20 comp rx=0 tx=0).stop 2024-04-25T10:31:23.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 shutdown_connections 2024-04-25T10:31:23.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65b0066f60 0x7f65b0069420 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65d4159540 0x7f65d414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65d40aebd0 0x7f65d413e050 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 --2- 172.21.15.45:0/1272649423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65d4072020 0x7f65d40b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.438+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 >> 172.21.15.45:0/1272649423 conn(0x7f65d40669a0 msgr2=0x7f65d4066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.442+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 shutdown_connections 2024-04-25T10:31:23.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.442+0000 7f65dae00640 1 -- 172.21.15.45:0/1272649423 wait complete. 2024-04-25T10:31:23.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:23.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 Processor -- start 2024-04-25T10:31:23.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 -- start start 2024-04-25T10:31:23.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd0d01550b0 0x7fd0d00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0d00cfe70 0x7fd0d015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d015b740 0x7fd0d006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d0071050 con 0x7fd0d00cfe70 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d00712f0 con 0x7fd0d01550b0 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d8800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00715d0 con 0x7fd0d015b740 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0d00cfe70 0x7fd0d015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0d00cfe70 0x7fd0d015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33160/0 (socket says 172.21.15.45:33160) 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 -- 172.21.15.45:0/1429341291 learned_addr learned my addr 172.21.15.45:0/1429341291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5e00640 1 --2- 172.21.15.45:0/1429341291 >> v2:172.21.15.117:3300/0 conn(0x7fd0d01550b0 0x7fd0d00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d6800640 1 --2- 172.21.15.45:0/1429341291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d015b740 0x7fd0d006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 -- 172.21.15.45:0/1429341291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d015b740 msgr2=0x7fd0d006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:23.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 --2- 172.21.15.45:0/1429341291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d015b740 0x7fd0d006d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 -- 172.21.15.45:0/1429341291 >> v2:172.21.15.117:3300/0 conn(0x7fd0d01550b0 msgr2=0x7fd0d00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 --2- 172.21.15.45:0/1429341291 >> v2:172.21.15.117:3300/0 conn(0x7fd0d01550b0 0x7fd0d00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.614+0000 7fd0d5400640 1 -- 172.21.15.45:0/1429341291 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00718b0 con 0x7fd0d00cfe70 2024-04-25T10:31:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d5400640 1 --2- 172.21.15.45:0/1429341291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d00cfe70 0x7fd0d015b200 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000bb10 tx=0x7fd0c001e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cd8424c0078e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d4a00640 1 -- 172.21.15.45:0/1429341291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b4070 con 0x7fd0d00cfe70 2024-04-25T10:31:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d4a00640 1 -- 172.21.15.45:0/1429341291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c00a7840 con 0x7fd0d00cfe70 2024-04-25T10:31:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d4a00640 1 -- 172.21.15.45:0/1429341291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00a7c10 con 0x7fd0d00cfe70 2024-04-25T10:31:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 -- 172.21.15.45:0/1429341291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d00cfe70 msgr2=0x7fd0d015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 --2- 172.21.15.45:0/1429341291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d00cfe70 0x7fd0d015b200 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000bb10 tx=0x7fd0c001e930 comp rx=0 tx=0).stop 2024-04-25T10:31:23.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 -- 172.21.15.45:0/1429341291 shutdown_connections 2024-04-25T10:31:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 --2- 172.21.15.45:0/1429341291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d015b740 0x7fd0d006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 --2- 172.21.15.45:0/1429341291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d00cfe70 0x7fd0d015b200 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 --2- 172.21.15.45:0/1429341291 >> v2:172.21.15.117:3300/0 conn(0x7fd0d01550b0 0x7fd0d00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 -- 172.21.15.45:0/1429341291 >> 172.21.15.45:0/1429341291 conn(0x7fd0d0066b70 msgr2=0x7fd0d0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 -- 172.21.15.45:0/1429341291 shutdown_connections 2024-04-25T10:31:23.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.618+0000 7fd0d8800640 1 -- 172.21.15.45:0/1429341291 wait complete. 2024-04-25T10:31:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 Processor -- start 2024-04-25T10:31:23.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- start start 2024-04-25T10:31:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d00cfe70 0x7fd0d0143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d01550b0 0x7fd0d0157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 0x7fd0d00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00b3260 con 0x7fd0d015b740 2024-04-25T10:31:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d015bd60 con 0x7fd0d01550b0 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00b33e0 con 0x7fd0d00cfe70 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 0x7fd0d00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d01550b0 0x7fd0d0157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 0x7fd0d00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33170/0 (socket says 172.21.15.45:33170) 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d6800640 1 -- 172.21.15.45:0/2435773106 learned_addr learned my addr 172.21.15.45:0/2435773106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d00cfe70 0x7fd0d0143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 -- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d00cfe70 msgr2=0x7fd0d0143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d00cfe70 0x7fd0d0143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 -- 172.21.15.45:0/2435773106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 msgr2=0x7fd0d00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 0x7fd0d00cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 -- 172.21.15.45:0/2435773106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0156ba0 con 0x7fd0d01550b0 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d6800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 0x7fd0d00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:23.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d5400640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d01550b0 0x7fd0d0157e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00ae000 tx=0x7fd0c001b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b4070 con 0x7fd0d01550b0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c00b6070 con 0x7fd0d01550b0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0156e80 con 0x7fd0d01550b0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00afc10 con 0x7fd0d01550b0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d0157440 con 0x7fd0d01550b0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.622+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d004ef30 con 0x7fd0d01550b0 2024-04-25T10:31:23.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.626+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0c00c53f0 con 0x7fd0d01550b0 2024-04-25T10:31:23.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.626+0000 7fd0ca000640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0b4066f60 0x7fd0b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.626+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0c011f1c0 con 0x7fd0d01550b0 2024-04-25T10:31:23.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.626+0000 7fd0d5e00640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0b4066f60 0x7fd0b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.626+0000 7fd0d5e00640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0b4066f60 0x7fd0b4069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0153d90 tx=0x7fd0c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.630+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0c00187e0 con 0x7fd0d01550b0 2024-04-25T10:31:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.790+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0d00af5f0 con 0x7fd0d01550b0 2024-04-25T10:31:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0ca000640 1 -- 172.21.15.45:0/2435773106 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+16616 (secure 0 0 0) 0x7fd0c00a7810 con 0x7fd0d01550b0 2024-04-25T10:31:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:23.797 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":181,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:22.823521+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0b4066f60 msgr2=0x7fd0b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0b4066f60 0x7fd0b4069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0153d90 tx=0x7fd0c4008040 comp rx=0 tx=0).stop 2024-04-25T10:31:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d01550b0 msgr2=0x7fd0d0157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d01550b0 0x7fd0d0157e70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00ae000 tx=0x7fd0c001b6d0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 shutdown_connections 2024-04-25T10:31:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd0b4066f60 0x7fd0b4069420 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0d015b740 0x7fd0d00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0d01550b0 0x7fd0d0157e70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 --2- 172.21.15.45:0/2435773106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0d00cfe70 0x7fd0d0143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 >> 172.21.15.45:0/2435773106 conn(0x7fd0d0066b70 msgr2=0x7fd0d0153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 shutdown_connections 2024-04-25T10:31:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.794+0000 7fd0d8800640 1 -- 172.21.15.45:0/2435773106 wait complete. 2024-04-25T10:31:23.819 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 Processor -- start 2024-04-25T10:31:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 -- start start 2024-04-25T10:31:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb3cc15bc40 0x7fb3cc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb3cc0cda10 0x7fb3cc0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc0b2770 con 0x7fb3cc0cf930 2024-04-25T10:31:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc06c670 con 0x7fb3cc15bc40 2024-04-25T10:31:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3d1200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc06c950 con 0x7fb3cc0cda10 2024-04-25T10:31:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3c9e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb3cc15bc40 0x7fb3cc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3ca800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb3cc0cf930 0x7fb3cc15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41636/0 (socket says 172.21.15.45:41636) 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3ca800640 1 -- 172.21.15.45:0/482223421 learned_addr learned my addr 172.21.15.45:0/482223421 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3c9400640 1 --2- 172.21.15.45:0/482223421 >> v2:172.21.15.117:3300/0 conn(0x7fb3cc0cda10 0x7fb3cc0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3c9e00640 1 -- 172.21.15.45:0/482223421 >> v2:172.21.15.117:3300/0 conn(0x7fb3cc0cda10 msgr2=0x7fb3cc0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3c9e00640 1 --2- 172.21.15.45:0/482223421 >> v2:172.21.15.117:3300/0 conn(0x7fb3cc0cda10 0x7fb3cc0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3c9e00640 1 -- 172.21.15.45:0/482223421 >> v1:172.21.15.45:6789/0 conn(0x7fb3cc0cf930 legacy=0x7fb3cc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.958+0000 7fb3c9e00640 1 -- 172.21.15.45:0/482223421 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc06cc30 con 0x7fb3cc15bc40 2024-04-25T10:31:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3c9e00640 1 --2- 172.21.15.45:0/482223421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc06c090 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0726f0 tx=0x7fb3bc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c26c7b9db73734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3c8a00640 1 -- 172.21.15.45:0/482223421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0b1070 con 0x7fb3cc15bc40 2024-04-25T10:31:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3c8a00640 1 -- 172.21.15.45:0/482223421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc0ac5f0 con 0x7fb3cc15bc40 2024-04-25T10:31:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3c8a00640 1 -- 172.21.15.45:0/482223421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc0ac9c0 con 0x7fb3cc15bc40 2024-04-25T10:31:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 -- 172.21.15.45:0/482223421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 msgr2=0x7fb3cc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 --2- 172.21.15.45:0/482223421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc06c090 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0726f0 tx=0x7fb3bc01c480 comp rx=0 tx=0).stop 2024-04-25T10:31:23.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 -- 172.21.15.45:0/482223421 shutdown_connections 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 --2- 172.21.15.45:0/482223421 >> v2:172.21.15.117:3300/0 conn(0x7fb3cc0cda10 0x7fb3cc0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 --2- 172.21.15.45:0/482223421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc06c090 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 -- 172.21.15.45:0/482223421 >> 172.21.15.45:0/482223421 conn(0x7fb3cc066b70 msgr2=0x7fb3cc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 -- 172.21.15.45:0/482223421 shutdown_connections 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 -- 172.21.15.45:0/482223421 wait complete. 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 Processor -- start 2024-04-25T10:31:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.962+0000 7fb3d1200640 1 -- start start 2024-04-25T10:31:23.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0cda10 0x7fb3cc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc0cf930 0x7fb3cc13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc168b50 con 0x7fb3cc15bc40 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc1689d0 con 0x7fb3cc0cda10 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc168cd0 con 0x7fb3cc0cf930 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc0b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33200/0 (socket says 172.21.15.45:33200) 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 -- 172.21.15.45:0/3375472711 learned_addr learned my addr 172.21.15.45:0/3375472711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3c9e00640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0cda10 0x7fb3cc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3c9400640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc0cf930 0x7fb3cc13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 -- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc0cf930 msgr2=0x7fb3cc13c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc0cf930 0x7fb3cc13c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 -- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0cda10 msgr2=0x7fb3cc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0cda10 0x7fb3cc15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 -- 172.21.15.45:0/3375472711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc0b1210 con 0x7fb3cc15bc40 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3ca800640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc0b0aa0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400cab0 tx=0x7fb3c400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4017070 con 0x7fb3cc15bc40 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c400ac30 con 0x7fb3cc15bc40 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4012400 con 0x7fb3cc15bc40 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc0b14f0 con 0x7fb3cc15bc40 2024-04-25T10:31:23.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc0b1ab0 con 0x7fb3cc15bc40 2024-04-25T10:31:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc04ef30 con 0x7fb3cc15bc40 2024-04-25T10:31:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.966+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3c401a070 con 0x7fb3cc15bc40 2024-04-25T10:31:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.970+0000 7fb3c2000640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3ac066f60 0x7fb3ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.970+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3c4084c90 con 0x7fb3cc15bc40 2024-04-25T10:31:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.970+0000 7fb3c9e00640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3ac066f60 0x7fb3ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:23.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.970+0000 7fb3c9e00640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3ac066f60 0x7fb3ac069420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0ae000 tx=0x7fb3bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:23.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:23.974+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3c40555b0 con 0x7fb3cc15bc40 2024-04-25T10:31:24.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.138+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3cc06d960 con 0x7fb3cc15bc40 2024-04-25T10:31:24.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.138+0000 7fb3c2000640 1 -- 172.21.15.45:0/3375472711 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+16616 (secure 0 0 0) 0x7fb3c4059650 con 0x7fb3cc15bc40 2024-04-25T10:31:24.142 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:24.143 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":181,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:22.823521+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:24.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3ac066f60 msgr2=0x7fb3ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3ac066f60 0x7fb3ac069420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0ae000 tx=0x7fb3bc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 msgr2=0x7fb3cc0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc0b0aa0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400cab0 tx=0x7fb3c400cf70 comp rx=0 tx=0).stop 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 shutdown_connections 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3ac066f60 0x7fb3ac069420 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc15bc40 0x7fb3cc0b0aa0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc0cf930 0x7fb3cc13c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 --2- 172.21.15.45:0/3375472711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0cda10 0x7fb3cc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 >> 172.21.15.45:0/3375472711 conn(0x7fb3cc066b70 msgr2=0x7fb3cc15a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 shutdown_connections 2024-04-25T10:31:24.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.142+0000 7fb3d1200640 1 -- 172.21.15.45:0/3375472711 wait complete. 2024-04-25T10:31:24.164 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:31:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3ee400640 1 Processor -- start 2024-04-25T10:31:24.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3ee400640 1 -- start start 2024-04-25T10:31:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3ee400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb3e80ae2d0 0x7fb3e80717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3ee400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80720c0 con 0x7fb3e8071ce0 2024-04-25T10:31:24.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3ee400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e8147a20 con 0x7fb3e8147640 2024-04-25T10:31:24.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3ee400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e806e290 con 0x7fb3e80ae2d0 2024-04-25T10:31:24.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e7e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb3e8071ce0 0x7fb3e80b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41648/0 (socket says 172.21.15.45:41648) 2024-04-25T10:31:24.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e7e00640 1 -- 172.21.15.45:0/2995698745 learned_addr learned my addr 172.21.15.45:0/2995698745 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 --2- 172.21.15.45:0/2995698745 >> v2:172.21.15.117:3300/0 conn(0x7fb3e80ae2d0 0x7fb3e80717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236616431 0 0) 0x7fb3e80720c0 con 0x7fb3e8071ce0 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3cc003530 con 0x7fb3e8071ce0 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945794443 0 0) 0x7fb3cc003530 con 0x7fb3e8071ce0 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 >> v2:172.21.15.117:3300/0 conn(0x7fb3e80ae2d0 msgr2=0x7fb3e80717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 --2- 172.21.15.45:0/2995698745 >> v2:172.21.15.117:3300/0 conn(0x7fb3e80ae2d0 0x7fb3e80717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 >> v1:172.21.15.117:6789/0 conn(0x7fb3e8147640 legacy=0x7fb3e806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e806e570 con 0x7fb3e8071ce0 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fb3e0001a00 con 0x7fb3e8071ce0 2024-04-25T10:31:24.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 >> v1:172.21.15.45:6789/0 conn(0x7fb3e8071ce0 legacy=0x7fb3e80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc003a10 0x7fb3cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc004810 0x7fb3cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0051d0 0x7fb3cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e806e290 con 0x7fb3cc004810 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc00c0d0 con 0x7fb3cc0051d0 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc00c2d0 con 0x7fb3cc003a10 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc004810 0x7fb3cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e7400640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc003a10 0x7fb3cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e7e00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0051d0 0x7fb3cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 -- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc003a10 msgr2=0x7fb3cc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc003a10 0x7fb3cc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 -- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0051d0 msgr2=0x7fb3cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc0051d0 0x7fb3cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3dfe00640 1 -- 172.21.15.45:0/2995698745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc003530 con 0x7fb3cc004810 2024-04-25T10:31:24.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e7e00640 1 -- 172.21.15.45:0/2995698745 reap_dead start 2024-04-25T10:31:24.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.306+0000 7fb3e7400640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc003a10 0x7fb3cc003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3dfe00640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc004810 0x7fb3cc004c90 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009ae0 tx=0x7fb3d001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23126f081225d796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00b2070 con 0x7fb3cc004810 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d00a5630 con 0x7fb3cc004810 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3e6a00640 1 -- 172.21.15.45:0/2995698745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00096b0 con 0x7fb3cc004810 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 -- 172.21.15.45:0/2995698745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc004810 msgr2=0x7fb3cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc004810 0x7fb3cc004c90 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009ae0 tx=0x7fb3d001ca40 comp rx=0 tx=0).stop 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 -- 172.21.15.45:0/2995698745 shutdown_connections 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 --2- 172.21.15.45:0/2995698745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc004810 0x7fb3cc004c90 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 -- 172.21.15.45:0/2995698745 >> 172.21.15.45:0/2995698745 conn(0x7fb3e80b21c0 msgr2=0x7fb3e80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 -- 172.21.15.45:0/2995698745 shutdown_connections 2024-04-25T10:31:24.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.310+0000 7fb3ee400640 1 -- 172.21.15.45:0/2995698745 wait complete. 2024-04-25T10:31:24.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 Processor -- start 2024-04-25T10:31:24.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- start start 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc004810 0x7fb3e813e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e815cd40 0x7fb3e81f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 0x7fb3e81f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e806ee10 con 0x7fb3e815d1c0 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e806eb30 con 0x7fb3e815cd40 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80ae760 con 0x7fb3cc004810 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 0x7fb3e81f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 0x7fb3e81f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33222/0 (socket says 172.21.15.45:33222) 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 -- 172.21.15.45:0/4088317191 learned_addr learned my addr 172.21.15.45:0/4088317191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc004810 0x7fb3e813e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3dfe00640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e815cd40 0x7fb3e81f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 -- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc004810 msgr2=0x7fb3e813e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc004810 0x7fb3e813e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 -- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e815cd40 msgr2=0x7fb3e81f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e815cd40 0x7fb3e81f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 -- 172.21.15.45:0/4088317191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e813ea40 con 0x7fb3e815d1c0 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3e7e00640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 0x7fb3e81f77f0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8073f10 tx=0x7fb3e0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0010040 con 0x7fb3e815d1c0 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e001a070 con 0x7fb3e815d1c0 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0011ea0 con 0x7fb3e815d1c0 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e813cf50 con 0x7fb3e815d1c0 2024-04-25T10:31:24.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e813d510 con 0x7fb3e815d1c0 2024-04-25T10:31:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.314+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e804ef30 con 0x7fb3e815d1c0 2024-04-25T10:31:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.318+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3e000fb40 con 0x7fb3e815d1c0 2024-04-25T10:31:24.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.318+0000 7fb3df400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3c4066f60 0x7fb3c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.318+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3e0087fa0 con 0x7fb3e815d1c0 2024-04-25T10:31:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.318+0000 7fb3e7400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3c4066f60 0x7fb3c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:24.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.318+0000 7fb3e7400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3c4066f60 0x7fb3c4069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8000c10 tx=0x7fb3d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:24.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.322+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3e0058f10 con 0x7fb3e815d1c0 2024-04-25T10:31:24.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:24.522+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb3e813ec70 con 0x7fb3e815d1c0 2024-04-25T10:31:25.401 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:31:25.402+0000 7f1c92000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:31:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.438+0000 7fb3df400640 1 -- 172.21.15.45:0/4088317191 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v130) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb3e0058810 con 0x7fb3e815d1c0 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3c4066f60 msgr2=0x7fb3c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3c4066f60 0x7fb3c4069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8000c10 tx=0x7fb3d8009040 comp rx=0 tx=0).stop 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 msgr2=0x7fb3e81f77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 0x7fb3e81f77f0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8073f10 tx=0x7fb3e0011b60 comp rx=0 tx=0).stop 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 shutdown_connections 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb3c4066f60 0x7fb3c4069420 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3e815d1c0 0x7fb3e81f77f0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3e815cd40 0x7fb3e81f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 --2- 172.21.15.45:0/4088317191 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3cc004810 0x7fb3e813e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 >> 172.21.15.45:0/4088317191 conn(0x7fb3e80b21c0 msgr2=0x7fb3e80b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 shutdown_connections 2024-04-25T10:31:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.442+0000 7fb3ee400640 1 -- 172.21.15.45:0/4088317191 wait complete. 2024-04-25T10:31:25.467 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:31:25.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:31:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.610+0000 7f29eb800640 1 Processor -- start 2024-04-25T10:31:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29eb800640 1 -- start start 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29eb800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29e415e130 0x7f29e40cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29eb800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29e4071bc0 0x7f29e4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29eb800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e40ad840 con 0x7f29e40b3480 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29eb800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e4072570 con 0x7f29e4071bc0 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29eb800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e40d0090 con 0x7f29e415e130 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29e4071bc0 0x7f29e4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29e4071bc0 0x7f29e4072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57536/0 (socket says 172.21.15.45:57536) 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 -- 172.21.15.45:0/3473783354 learned_addr learned my addr 172.21.15.45:0/3473783354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e3e00640 1 --2- 172.21.15.45:0/3473783354 >> v2:172.21.15.117:3300/0 conn(0x7f29e415e130 0x7f29e40cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 -- 172.21.15.45:0/3473783354 >> v2:172.21.15.117:3300/0 conn(0x7f29e415e130 msgr2=0x7f29e40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 --2- 172.21.15.45:0/3473783354 >> v2:172.21.15.117:3300/0 conn(0x7f29e415e130 0x7f29e40cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 -- 172.21.15.45:0/3473783354 >> v1:172.21.15.45:6789/0 conn(0x7f29e40b3480 legacy=0x7f29e4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.614+0000 7f29e9800640 1 -- 172.21.15.45:0/3473783354 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e415e710 con 0x7f29e4071bc0 2024-04-25T10:31:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29e9800640 1 --2- 172.21.15.45:0/3473783354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e4072030 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f29e40b4c40 tx=0x7f29dc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2270eb56bd11f121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29e3400640 1 -- 172.21.15.45:0/3473783354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0a56c0 con 0x7f29e4071bc0 2024-04-25T10:31:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29e3400640 1 -- 172.21.15.45:0/3473783354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc0a5860 con 0x7f29e4071bc0 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29e3400640 1 -- 172.21.15.45:0/3473783354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0a5c30 con 0x7f29e4071bc0 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 -- 172.21.15.45:0/3473783354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 msgr2=0x7f29e4072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 --2- 172.21.15.45:0/3473783354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e4072030 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f29e40b4c40 tx=0x7f29dc01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 -- 172.21.15.45:0/3473783354 shutdown_connections 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 --2- 172.21.15.45:0/3473783354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e4072030 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 --2- 172.21.15.45:0/3473783354 >> v2:172.21.15.117:3300/0 conn(0x7f29e415e130 0x7f29e40cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 -- 172.21.15.45:0/3473783354 >> 172.21.15.45:0/3473783354 conn(0x7f29e40669a0 msgr2=0x7f29e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 -- 172.21.15.45:0/3473783354 shutdown_connections 2024-04-25T10:31:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 -- 172.21.15.45:0/3473783354 wait complete. 2024-04-25T10:31:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 Processor -- start 2024-04-25T10:31:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.618+0000 7f29eb800640 1 -- start start 2024-04-25T10:31:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e40b3480 0x7f29e4157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29e415e130 0x7f29e413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40ae530 con 0x7f29e4071bc0 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e415bb90 con 0x7f29e40b3480 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40ae6b0 con 0x7f29e415e130 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e415ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57540/0 (socket says 172.21.15.45:57540) 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 -- 172.21.15.45:0/2460320297 learned_addr learned my addr 172.21.15.45:0/2460320297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e3e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e40b3480 0x7f29e4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e9800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29e415e130 0x7f29e413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 -- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29e415e130 msgr2=0x7f29e413bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29e415e130 0x7f29e413bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 -- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e40b3480 msgr2=0x7f29e4157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e40b3480 0x7f29e4157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 -- 172.21.15.45:0/2460320297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e41593c0 con 0x7f29e4071bc0 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e3e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e40b3480 0x7f29e4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e8e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e415ae90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c400 tx=0x7f29d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8017070 con 0x7f29e4071bc0 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8005310 con 0x7f29e4071bc0 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80054c0 con 0x7f29e4071bc0 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e41596a0 con 0x7f29e4071bc0 2024-04-25T10:31:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e4159c60 con 0x7f29e4071bc0 2024-04-25T10:31:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e404ef30 con 0x7f29e4071bc0 2024-04-25T10:31:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.622+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29d801a070 con 0x7f29e4071bc0 2024-04-25T10:31:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.626+0000 7f29e0c00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29c4066f60 0x7f29c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.626+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d8084370 con 0x7f29e4071bc0 2024-04-25T10:31:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.626+0000 7f29e3e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29c4066f60 0x7f29c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.626+0000 7f29e3e00640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29c4066f60 0x7f29c4069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f29e415aaf0 tx=0x7f29d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.630+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29d8054c90 con 0x7f29e4071bc0 2024-04-25T10:31:25.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:25.826+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f29e4158eb0 con 0x7f29e4071bc0 2024-04-25T10:31:26.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.438+0000 7f29e0c00640 1 -- 172.21.15.45:0/2460320297 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v131) v1 ==== 122+0+0 (secure 0 0 0) 0x7f29d8058d30 con 0x7f29e4071bc0 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29c4066f60 msgr2=0x7f29c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29c4066f60 0x7f29c4069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f29e415aaf0 tx=0x7f29d0008040 comp rx=0 tx=0).stop 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 msgr2=0x7f29e415ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e415ae90 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c400 tx=0x7f29d800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 shutdown_connections 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f29c4066f60 0x7f29c4069420 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29e415e130 0x7f29e413bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29e40b3480 0x7f29e4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 --2- 172.21.15.45:0/2460320297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e4071bc0 0x7f29e415ae90 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 >> 172.21.15.45:0/2460320297 conn(0x7f29e40669a0 msgr2=0x7f29e415d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 shutdown_connections 2024-04-25T10:31:26.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.442+0000 7f29eb800640 1 -- 172.21.15.45:0/2460320297 wait complete. 2024-04-25T10:31:26.467 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:31:26.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:26.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3459800640 1 Processor -- start 2024-04-25T10:31:26.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3459800640 1 -- start start 2024-04-25T10:31:26.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3459800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34540d0040 0x7f345415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3459800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3454071050 con 0x7f345415b740 2024-04-25T10:31:26.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3459800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345415d040 con 0x7f34541550b0 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3459800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34540712f0 con 0x7f34540d0040 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3453400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f345415b740 0x7f34540b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55668/0 (socket says 172.21.15.45:55668) 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3453400640 1 -- 172.21.15.45:0/3286690372 learned_addr learned my addr 172.21.15.45:0/3286690372 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f344be00640 1 --2- 172.21.15.45:0/3286690372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34540d0040 0x7f345415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3452000640 1 -- 172.21.15.45:0/3286690372 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188776007 0 0) 0x7f3454071050 con 0x7f345415b740 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3452000640 1 -- 172.21.15.45:0/3286690372 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3438003530 con 0x7f345415b740 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f344be00640 1 -- 172.21.15.45:0/3286690372 >> v1:172.21.15.117:6789/0 conn(0x7f34541550b0 legacy=0x7f34540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f3452000640 1 -- 172.21.15.45:0/3286690372 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008068743 0 0) 0x7f345415d040 con 0x7f34541550b0 2024-04-25T10:31:26.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f344be00640 1 -- 172.21.15.45:0/3286690372 >> v1:172.21.15.45:6789/0 conn(0x7f345415b740 legacy=0x7f34540b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.610+0000 7f344be00640 1 -- 172.21.15.45:0/3286690372 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540715d0 con 0x7f34540d0040 2024-04-25T10:31:26.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f344be00640 1 --2- 172.21.15.45:0/3286690372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34540d0040 0x7f345415b200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f343c0097f0 tx=0x7f343c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80dc5d8403bc411 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:26.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3452000640 1 -- 172.21.15.45:0/3286690372 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0b1070 con 0x7f34540d0040 2024-04-25T10:31:26.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3452000640 1 -- 172.21.15.45:0/3286690372 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0ac410 con 0x7f34540d0040 2024-04-25T10:31:26.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3452000640 1 -- 172.21.15.45:0/3286690372 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0ac7c0 con 0x7f34540d0040 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 -- 172.21.15.45:0/3286690372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34540d0040 msgr2=0x7f345415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 --2- 172.21.15.45:0/3286690372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34540d0040 0x7f345415b200 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f343c0097f0 tx=0x7f343c01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 -- 172.21.15.45:0/3286690372 shutdown_connections 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 --2- 172.21.15.45:0/3286690372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34540d0040 0x7f345415b200 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 -- 172.21.15.45:0/3286690372 >> 172.21.15.45:0/3286690372 conn(0x7f3454066b70 msgr2=0x7f3454066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 -- 172.21.15.45:0/3286690372 shutdown_connections 2024-04-25T10:31:26.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.614+0000 7f3459800640 1 -- 172.21.15.45:0/3286690372 wait complete. 2024-04-25T10:31:26.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 Processor -- start 2024-04-25T10:31:26.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- start start 2024-04-25T10:31:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 0x7f345414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34541550b0 0x7f3454146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f345415b740 0x7f34540cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345415bcf0 con 0x7f34540d0040 2024-04-25T10:31:26.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345415bb70 con 0x7f345415b740 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34541684c0 con 0x7f34541550b0 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 0x7f345414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 0x7f345414edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57552/0 (socket says 172.21.15.45:57552) 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 -- 172.21.15.45:0/1958288418 learned_addr learned my addr 172.21.15.45:0/1958288418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f344be00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34541550b0 0x7f3454146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3453400640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f345415b740 0x7f34540cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 -- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34541550b0 msgr2=0x7f3454146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34541550b0 0x7f3454146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 -- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f345415b740 msgr2=0x7f34540cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f345415b740 0x7f34540cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 -- 172.21.15.45:0/1958288418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540cdd90 con 0x7f34540d0040 2024-04-25T10:31:26.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3452a00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 0x7f345414edc0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f344400c500 tx=0x7f344400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444017070 con 0x7f34540d0040 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444003600 con 0x7f34540d0040 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440039f0 con 0x7f34540d0040 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540ce070 con 0x7f34540d0040 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34540ce630 con 0x7f34540d0040 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.618+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345404ef30 con 0x7f34540d0040 2024-04-25T10:31:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.622+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f344401a070 con 0x7f34540d0040 2024-04-25T10:31:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.622+0000 7f344aa00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3430066f60 0x7f3430069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.622+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3444084550 con 0x7f34540d0040 2024-04-25T10:31:26.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.622+0000 7f344be00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3430066f60 0x7f3430069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.622+0000 7f344be00640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3430066f60 0x7f3430069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f343c0ae000 tx=0x7f343c0a6040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.626+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3444055590 con 0x7f34540d0040 2024-04-25T10:31:26.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.826+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f345415c450 con 0x7f34540d0040 2024-04-25T10:31:26.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.826+0000 7f344aa00640 1 -- 172.21.15.45:0/1958288418 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f3444054e90 con 0x7f34540d0040 2024-04-25T10:31:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:26.832 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:22.823850+0000","modified":"2024-04-25T10:31:25.830407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10881":{"gid":10881,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1455178901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1455178901},{"type":"v1","addr":"172.21.15.45:6835","nonce":1455178901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:31:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 131 2024-04-25T10:31:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3430066f60 msgr2=0x7f3430069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3430066f60 0x7f3430069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f343c0ae000 tx=0x7f343c0a6040 comp rx=0 tx=0).stop 2024-04-25T10:31:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 msgr2=0x7f345414edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 0x7f345414edc0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f344400c500 tx=0x7f344400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 shutdown_connections 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3430066f60 0x7f3430069420 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f345415b740 0x7f34540cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34541550b0 0x7f3454146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 --2- 172.21.15.45:0/1958288418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34540d0040 0x7f345414edc0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 >> 172.21.15.45:0/1958288418 conn(0x7f3454066b70 msgr2=0x7f345406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 shutdown_connections 2024-04-25T10:31:26.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.830+0000 7f3459800640 1 -- 172.21.15.45:0/1958288418 wait complete. 2024-04-25T10:31:26.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:26.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 Processor -- start 2024-04-25T10:31:26.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 -- start start 2024-04-25T10:31:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99540669a0 0x7f9954066e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9954161600 0x7f9954161a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540ad840 con 0x7f9954157f40 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995406e7a0 con 0x7f99540669a0 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f995a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540af9b0 con 0x7f9954161600 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99540669a0 0x7f9954066e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99540669a0 0x7f9954066e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57556/0 (socket says 172.21.15.45:57556) 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 -- 172.21.15.45:0/4269714157 learned_addr learned my addr 172.21.15.45:0/4269714157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9959a00640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9954161600 0x7f9954161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 -- 172.21.15.45:0/4269714157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9954161600 msgr2=0x7f9954161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9954161600 0x7f9954161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 -- 172.21.15.45:0/4269714157 >> v1:172.21.15.117:6789/0 conn(0x7f9954157f40 legacy=0x7f99540aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9953e00640 1 -- 172.21.15.45:0/4269714157 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995415e120 con 0x7f99540669a0 2024-04-25T10:31:26.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.990+0000 7f9959a00640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9954161600 0x7f9954161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:26.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f9953e00640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99540669a0 0x7f9954066e50 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f9940009870 tx=0x7f994001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22774448fea636a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f9953400640 1 -- 172.21.15.45:0/4269714157 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400b0070 con 0x7f99540669a0 2024-04-25T10:31:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f9953400640 1 -- 172.21.15.45:0/4269714157 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99400ab600 con 0x7f99540669a0 2024-04-25T10:31:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f9953400640 1 -- 172.21.15.45:0/4269714157 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400ab9d0 con 0x7f99540669a0 2024-04-25T10:31:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 -- 172.21.15.45:0/4269714157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99540669a0 msgr2=0x7f9954066e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:26.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99540669a0 0x7f9954066e50 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f9940009870 tx=0x7f994001c340 comp rx=0 tx=0).stop 2024-04-25T10:31:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 -- 172.21.15.45:0/4269714157 shutdown_connections 2024-04-25T10:31:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9954161600 0x7f9954161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 --2- 172.21.15.45:0/4269714157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99540669a0 0x7f9954066e50 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 -- 172.21.15.45:0/4269714157 >> 172.21.15.45:0/4269714157 conn(0x7f9954074310 msgr2=0x7f995406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 -- 172.21.15.45:0/4269714157 shutdown_connections 2024-04-25T10:31:26.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.994+0000 7f995a400640 1 -- 172.21.15.45:0/4269714157 wait complete. 2024-04-25T10:31:26.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 Processor -- start 2024-04-25T10:31:26.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- start start 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99540669a0 0x7f9954142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 0x7f9954156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9954161600 0x7f9954156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99541623c0 con 0x7f9954157f40 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995415eee0 con 0x7f9954161600 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9954162540 con 0x7f99540669a0 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 0x7f9954156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 0x7f9954156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57568/0 (socket says 172.21.15.45:57568) 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 -- 172.21.15.45:0/2080957068 learned_addr learned my addr 172.21.15.45:0/2080957068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9959000640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99540669a0 0x7f9954142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9959a00640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9954161600 0x7f9954156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 -- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99540669a0 msgr2=0x7f9954142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99540669a0 0x7f9954142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 -- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9954161600 msgr2=0x7f9954156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9954161600 0x7f9954156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 -- 172.21.15.45:0/2080957068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954150340 con 0x7f9954157f40 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9953e00640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 0x7f9954156860 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f99400ae040 tx=0x7f99400190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400b0040 con 0x7f9954157f40 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99400b9070 con 0x7f9954157f40 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400b4880 con 0x7f9954157f40 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99541505e0 con 0x7f9954157f40 2024-04-25T10:31:27.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9954150ba0 con 0x7f9954157f40 2024-04-25T10:31:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:26.998+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9954154fe0 con 0x7f9954157f40 2024-04-25T10:31:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.002+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99400ab4c0 con 0x7f9954157f40 2024-04-25T10:31:27.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.002+0000 7f9950c00640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9934066f60 0x7f9934069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.002+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f994011f8d0 con 0x7f9954157f40 2024-04-25T10:31:27.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.002+0000 7f9959000640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9934066f60 0x7f9934069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.002+0000 7f9959000640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9934066f60 0x7f9934069420 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9948007fd0 tx=0x7f9948007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.006+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9954154fe0 con 0x7f9954157f40 2024-04-25T10:31:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.166+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99541613e0 con 0x7f9954157f40 2024-04-25T10:31:27.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f9950c00640 1 -- 172.21.15.45:0/2080957068 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f99540ad840 con 0x7f9954157f40 2024-04-25T10:31:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:27.171 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":182,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:24.430055+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9934066f60 msgr2=0x7f9934069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9934066f60 0x7f9934069420 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9948007fd0 tx=0x7f9948007e40 comp rx=0 tx=0).stop 2024-04-25T10:31:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 msgr2=0x7f9954156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 0x7f9954156860 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f99400ae040 tx=0x7f99400190e0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 shutdown_connections 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9934066f60 0x7f9934069420 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9954161600 0x7f9954156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9954157f40 0x7f9954156860 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 --2- 172.21.15.45:0/2080957068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99540669a0 0x7f9954142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 >> 172.21.15.45:0/2080957068 conn(0x7f9954074310 msgr2=0x7f99540b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 shutdown_connections 2024-04-25T10:31:27.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.170+0000 7f995a400640 1 -- 172.21.15.45:0/2080957068 wait complete. 2024-04-25T10:31:27.195 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:31:27.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 Processor -- start 2024-04-25T10:31:27.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 -- start start 2024-04-25T10:31:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc15c15a9f0 0x7fc15c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cf310 0x7fc15c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0cc2c0 con 0x7fc15c0cbab0 2024-04-25T10:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c0cf770 con 0x7fc15c15a9f0 2024-04-25T10:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc161a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0cc5a0 con 0x7fc15c0cf310 2024-04-25T10:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15aa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc15c15a9f0 0x7fc15c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc153e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cf310 0x7fc15c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15b400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc15c0cbab0 0x7fc15c071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36794/0 (socket says 172.21.15.45:36794) 2024-04-25T10:31:27.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15b400640 1 -- 172.21.15.45:0/3884602727 learned_addr learned my addr 172.21.15.45:0/3884602727 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15a000640 1 -- 172.21.15.45:0/3884602727 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440772870 0 0) 0x7fc15c0cc2c0 con 0x7fc15c0cbab0 2024-04-25T10:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15a000640 1 -- 172.21.15.45:0/3884602727 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc140003530 con 0x7fc15c0cbab0 2024-04-25T10:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15aa00640 1 -- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cf310 msgr2=0x7fc15c0cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15aa00640 1 --2- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cf310 0x7fc15c0cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15aa00640 1 -- 172.21.15.45:0/3884602727 >> v1:172.21.15.117:6789/0 conn(0x7fc15c0cbab0 legacy=0x7fc15c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.334+0000 7fc15aa00640 1 -- 172.21.15.45:0/3884602727 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0cc880 con 0x7fc15c15a9f0 2024-04-25T10:31:27.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc15aa00640 1 --2- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c0cedd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0097c0 tx=0x7fc14c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab7254749046c43a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc15a000640 1 -- 172.21.15.45:0/3884602727 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0a7070 con 0x7fc15c15a9f0 2024-04-25T10:31:27.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc15a000640 1 -- 172.21.15.45:0/3884602727 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c0b0070 con 0x7fc15c15a9f0 2024-04-25T10:31:27.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc15a000640 1 -- 172.21.15.45:0/3884602727 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0ab890 con 0x7fc15c15a9f0 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 -- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 msgr2=0x7fc15c0cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 --2- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c0cedd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0097c0 tx=0x7fc14c01c470 comp rx=0 tx=0).stop 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 -- 172.21.15.45:0/3884602727 shutdown_connections 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 --2- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cf310 0x7fc15c0cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 --2- 172.21.15.45:0/3884602727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c0cedd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 -- 172.21.15.45:0/3884602727 >> 172.21.15.45:0/3884602727 conn(0x7fc15c0b2410 msgr2=0x7fc15c0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 -- 172.21.15.45:0/3884602727 shutdown_connections 2024-04-25T10:31:27.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.338+0000 7fc161a00640 1 -- 172.21.15.45:0/3884602727 wait complete. 2024-04-25T10:31:27.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 Processor -- start 2024-04-25T10:31:27.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 -- start start 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cbab0 0x7fc15c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 0x7fc15c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0cc040 con 0x7fc15c0cf310 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0cbec0 con 0x7fc15c15a9f0 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0669a0 con 0x7fc15c0cbab0 2024-04-25T10:31:27.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 0x7fc15c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 0x7fc15c159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57574/0 (socket says 172.21.15.45:57574) 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 -- 172.21.15.45:0/2291957402 learned_addr learned my addr 172.21.15.45:0/2291957402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc15aa00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cbab0 0x7fc15c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc15b400640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 -- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cbab0 msgr2=0x7fc15c152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cbab0 0x7fc15c152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 -- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 msgr2=0x7fc15c144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 -- 172.21.15.45:0/2291957402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c159f90 con 0x7fc15c0cf310 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc15b400640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:27.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc153e00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 0x7fc15c159840 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fc14400e930 tx=0x7fc14400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc144017070 con 0x7fc15c0cf310 2024-04-25T10:31:27.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14400ca00 con 0x7fc15c0cf310 2024-04-25T10:31:27.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14400cdb0 con 0x7fc15c0cf310 2024-04-25T10:31:27.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c1676e0 con 0x7fc15c0cf310 2024-04-25T10:31:27.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.342+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c167ca0 con 0x7fc15c0cf310 2024-04-25T10:31:27.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.346+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c04ef30 con 0x7fc15c0cf310 2024-04-25T10:31:27.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.346+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc14401a070 con 0x7fc15c0cf310 2024-04-25T10:31:27.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.346+0000 7fc152a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc138066f60 0x7fc138069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.346+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc144085390 con 0x7fc15c0cf310 2024-04-25T10:31:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.346+0000 7fc15aa00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc138066f60 0x7fc138069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.346+0000 7fc15aa00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc138066f60 0x7fc138069420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0af000 tx=0x7fc14c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.350+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc144055e00 con 0x7fc15c0cf310 2024-04-25T10:31:27.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.510+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc15c0c73e0 con 0x7fc138066f60 2024-04-25T10:31:27.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.518+0000 7fc152a00640 1 -- 172.21.15.45:0/2291957402 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251989 (secure 0 0 0) 0x7fc15c0c73e0 con 0x7fc138066f60 2024-04-25T10:31:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:27.524 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:31:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc138066f60 msgr2=0x7fc138069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc138066f60 0x7fc138069420 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0af000 tx=0x7fc14c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:31:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 msgr2=0x7fc15c159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 0x7fc15c159840 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fc14400e930 tx=0x7fc14400edf0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 shutdown_connections 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc138066f60 0x7fc138069420 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc15c15a9f0 0x7fc15c144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc15c0cf310 0x7fc15c159840 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 --2- 172.21.15.45:0/2291957402 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc15c0cbab0 0x7fc15c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 >> 172.21.15.45:0/2291957402 conn(0x7fc15c0b2410 msgr2=0x7fc15c066cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 shutdown_connections 2024-04-25T10:31:27.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.522+0000 7fc161a00640 1 -- 172.21.15.45:0/2291957402 wait complete. 2024-04-25T10:31:27.542 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2024-04-25T10:31:27.064660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461240,"kb_used_data":18932,"kb_used_omap":12,"kb_used_meta":442163,"kb_avail":754513480,"statfs":{"total":773094113280,"available":772621803520,"internally_reserved":0,"allocated":19386368,"data_stored":9148229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":452775472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.646564"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.442975+0000","last_change":"2024-04-25T10:31:23.435470+0000","last_active":"2024-04-25T10:31:23.442975+0000","last_peered":"2024-04-25T10:31:23.442975+0000","last_clean":"2024-04-25T10:31:23.442975+0000","last_became_active":"2024-04-25T10:31:23.433761+0000","last_became_peered":"2024-04-25T10:31:23.433761+0000","last_unstale":"2024-04-25T10:31:23.442975+0000","last_undegraded":"2024-04-25T10:31:23.442975+0000","last_fullsized":"2024-04-25T10:31:23.442975+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456702+0000","last_change":"2024-04-25T10:31:23.449323+0000","last_active":"2024-04-25T10:31:23.456702+0000","last_peered":"2024-04-25T10:31:23.456702+0000","last_clean":"2024-04-25T10:31:23.456702+0000","last_became_active":"2024-04-25T10:31:23.448894+0000","last_became_peered":"2024-04-25T10:31:23.448894+0000","last_unstale":"2024-04-25T10:31:23.456702+0000","last_undegraded":"2024-04-25T10:31:23.456702+0000","last_fullsized":"2024-04-25T10:31:23.456702+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:51.236038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459271+0000","last_change":"2024-04-25T10:31:23.455956+0000","last_active":"2024-04-25T10:31:23.459271+0000","last_peered":"2024-04-25T10:31:23.459271+0000","last_clean":"2024-04-25T10:31:23.459271+0000","last_became_active":"2024-04-25T10:31:23.451989+0000","last_became_peered":"2024-04-25T10:31:23.451989+0000","last_unstale":"2024-04-25T10:31:23.459271+0000","last_undegraded":"2024-04-25T10:31:23.459271+0000","last_fullsized":"2024-04-25T10:31:23.459271+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.779040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451553+0000","last_change":"2024-04-25T10:31:23.441950+0000","last_active":"2024-04-25T10:31:23.451553+0000","last_peered":"2024-04-25T10:31:23.451553+0000","last_clean":"2024-04-25T10:31:23.451553+0000","last_became_active":"2024-04-25T10:31:23.439585+0000","last_became_peered":"2024-04-25T10:31:23.439585+0000","last_unstale":"2024-04-25T10:31:23.451553+0000","last_undegraded":"2024-04-25T10:31:23.451553+0000","last_fullsized":"2024-04-25T10:31:23.451553+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:40.982507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458086+0000","last_change":"2024-04-25T10:31:23.452097+0000","last_active":"2024-04-25T10:31:23.458086+0000","last_peered":"2024-04-25T10:31:23.458086+0000","last_clean":"2024-04-25T10:31:23.458086+0000","last_became_active":"2024-04-25T10:31:23.447134+0000","last_became_peered":"2024-04-25T10:31:23.447134+0000","last_unstale":"2024-04-25T10:31:23.458086+0000","last_undegraded":"2024-04-25T10:31:23.458086+0000","last_fullsized":"2024-04-25T10:31:23.458086+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443135+0000","last_change":"2024-04-25T10:31:23.436119+0000","last_active":"2024-04-25T10:31:23.443135+0000","last_peered":"2024-04-25T10:31:23.443135+0000","last_clean":"2024-04-25T10:31:23.443135+0000","last_became_active":"2024-04-25T10:31:23.435858+0000","last_became_peered":"2024-04-25T10:31:23.435858+0000","last_unstale":"2024-04-25T10:31:23.443135+0000","last_undegraded":"2024-04-25T10:31:23.443135+0000","last_fullsized":"2024-04-25T10:31:23.443135+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448609+0000","last_change":"2024-04-25T10:31:23.446258+0000","last_active":"2024-04-25T10:31:23.448609+0000","last_peered":"2024-04-25T10:31:23.448609+0000","last_clean":"2024-04-25T10:31:23.448609+0000","last_became_active":"2024-04-25T10:31:23.445639+0000","last_became_peered":"2024-04-25T10:31:23.445639+0000","last_unstale":"2024-04-25T10:31:23.448609+0000","last_undegraded":"2024-04-25T10:31:23.448609+0000","last_fullsized":"2024-04-25T10:31:23.448609+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.543524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426407+0000","last_change":"2024-04-25T10:31:22.426407+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426407+0000","last_undegraded":"2024-04-25T10:31:22.426407+0000","last_fullsized":"2024-04-25T10:31:22.426407+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.435022+0000","last_change":"2024-04-25T10:31:23.434142+0000","last_active":"2024-04-25T10:31:23.435022+0000","last_peered":"2024-04-25T10:31:23.435022+0000","last_clean":"2024-04-25T10:31:23.435022+0000","last_became_active":"2024-04-25T10:31:23.433096+0000","last_became_peered":"2024-04-25T10:31:23.433096+0000","last_unstale":"2024-04-25T10:31:23.435022+0000","last_undegraded":"2024-04-25T10:31:23.435022+0000","last_fullsized":"2024-04-25T10:31:23.435022+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.452401+0000","last_change":"2024-04-25T10:31:23.445620+0000","last_active":"2024-04-25T10:31:23.452401+0000","last_peered":"2024-04-25T10:31:23.452401+0000","last_clean":"2024-04-25T10:31:23.452401+0000","last_became_active":"2024-04-25T10:31:23.445303+0000","last_became_peered":"2024-04-25T10:31:23.445303+0000","last_unstale":"2024-04-25T10:31:23.452401+0000","last_undegraded":"2024-04-25T10:31:23.452401+0000","last_fullsized":"2024-04-25T10:31:23.452401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:47.724852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.444838+0000","last_change":"2024-04-25T10:31:23.438628+0000","last_active":"2024-04-25T10:31:23.444838+0000","last_peered":"2024-04-25T10:31:23.444838+0000","last_clean":"2024-04-25T10:31:23.444838+0000","last_became_active":"2024-04-25T10:31:23.438301+0000","last_became_peered":"2024-04-25T10:31:23.438301+0000","last_unstale":"2024-04-25T10:31:23.444838+0000","last_undegraded":"2024-04-25T10:31:23.444838+0000","last_fullsized":"2024-04-25T10:31:23.444838+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:31.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448367+0000","last_change":"2024-04-25T10:31:23.444805+0000","last_active":"2024-04-25T10:31:23.448367+0000","last_peered":"2024-04-25T10:31:23.448367+0000","last_clean":"2024-04-25T10:31:23.448367+0000","last_became_active":"2024-04-25T10:31:23.444534+0000","last_became_peered":"2024-04-25T10:31:23.444534+0000","last_unstale":"2024-04-25T10:31:23.448367+0000","last_undegraded":"2024-04-25T10:31:23.448367+0000","last_fullsized":"2024-04-25T10:31:23.448367+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:11.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450950+0000","last_change":"2024-04-25T10:31:23.450197+0000","last_active":"2024-04-25T10:31:23.450950+0000","last_peered":"2024-04-25T10:31:23.450950+0000","last_clean":"2024-04-25T10:31:23.450950+0000","last_became_active":"2024-04-25T10:31:23.449302+0000","last_became_peered":"2024-04-25T10:31:23.449302+0000","last_unstale":"2024-04-25T10:31:23.450950+0000","last_undegraded":"2024-04-25T10:31:23.450950+0000","last_fullsized":"2024-04-25T10:31:23.450950+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:18.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454241+0000","last_change":"2024-04-25T10:31:23.449629+0000","last_active":"2024-04-25T10:31:23.454241+0000","last_peered":"2024-04-25T10:31:23.454241+0000","last_clean":"2024-04-25T10:31:23.454241+0000","last_became_active":"2024-04-25T10:31:23.449364+0000","last_became_peered":"2024-04-25T10:31:23.449364+0000","last_unstale":"2024-04-25T10:31:23.454241+0000","last_undegraded":"2024-04-25T10:31:23.454241+0000","last_fullsized":"2024-04-25T10:31:23.454241+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.444572+0000","last_change":"2024-04-25T10:31:23.438502+0000","last_active":"2024-04-25T10:31:23.444572+0000","last_peered":"2024-04-25T10:31:23.444572+0000","last_clean":"2024-04-25T10:31:23.444572+0000","last_became_active":"2024-04-25T10:31:23.437827+0000","last_became_peered":"2024-04-25T10:31:23.437827+0000","last_unstale":"2024-04-25T10:31:23.444572+0000","last_undegraded":"2024-04-25T10:31:23.444572+0000","last_fullsized":"2024-04-25T10:31:23.444572+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:10.584331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443709+0000","last_change":"2024-04-25T10:31:23.435579+0000","last_active":"2024-04-25T10:31:23.443709+0000","last_peered":"2024-04-25T10:31:23.443709+0000","last_clean":"2024-04-25T10:31:23.443709+0000","last_became_active":"2024-04-25T10:31:23.432705+0000","last_became_peered":"2024-04-25T10:31:23.432705+0000","last_unstale":"2024-04-25T10:31:23.443709+0000","last_undegraded":"2024-04-25T10:31:23.443709+0000","last_fullsized":"2024-04-25T10:31:23.443709+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455317+0000","last_change":"2024-04-25T10:31:23.450098+0000","last_active":"2024-04-25T10:31:23.455317+0000","last_peered":"2024-04-25T10:31:23.455317+0000","last_clean":"2024-04-25T10:31:23.455317+0000","last_became_active":"2024-04-25T10:31:23.449711+0000","last_became_peered":"2024-04-25T10:31:23.449711+0000","last_unstale":"2024-04-25T10:31:23.455317+0000","last_undegraded":"2024-04-25T10:31:23.455317+0000","last_fullsized":"2024-04-25T10:31:23.455317+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:32.246063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460719+0000","last_change":"2024-04-25T10:31:23.459671+0000","last_active":"2024-04-25T10:31:23.460719+0000","last_peered":"2024-04-25T10:31:23.460719+0000","last_clean":"2024-04-25T10:31:23.460719+0000","last_became_active":"2024-04-25T10:31:23.459391+0000","last_became_peered":"2024-04-25T10:31:23.459391+0000","last_unstale":"2024-04-25T10:31:23.460719+0000","last_undegraded":"2024-04-25T10:31:23.460719+0000","last_fullsized":"2024-04-25T10:31:23.460719+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459294+0000","last_change":"2024-04-25T10:31:23.456172+0000","last_active":"2024-04-25T10:31:23.459294+0000","last_peered":"2024-04-25T10:31:23.459294+0000","last_clean":"2024-04-25T10:31:23.459294+0000","last_became_active":"2024-04-25T10:31:23.452838+0000","last_became_peered":"2024-04-25T10:31:23.452838+0000","last_unstale":"2024-04-25T10:31:23.459294+0000","last_undegraded":"2024-04-25T10:31:23.459294+0000","last_fullsized":"2024-04-25T10:31:23.459294+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455909+0000","last_change":"2024-04-25T10:31:23.447420+0000","last_active":"2024-04-25T10:31:23.455909+0000","last_peered":"2024-04-25T10:31:23.455909+0000","last_clean":"2024-04-25T10:31:23.455909+0000","last_became_active":"2024-04-25T10:31:23.447112+0000","last_became_peered":"2024-04-25T10:31:23.447112+0000","last_unstale":"2024-04-25T10:31:23.455909+0000","last_undegraded":"2024-04-25T10:31:23.455909+0000","last_fullsized":"2024-04-25T10:31:23.455909+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451740+0000","last_change":"2024-04-25T10:31:23.450445+0000","last_active":"2024-04-25T10:31:23.451740+0000","last_peered":"2024-04-25T10:31:23.451740+0000","last_clean":"2024-04-25T10:31:23.451740+0000","last_became_active":"2024-04-25T10:31:23.450181+0000","last_became_peered":"2024-04-25T10:31:23.450181+0000","last_unstale":"2024-04-25T10:31:23.451740+0000","last_undegraded":"2024-04-25T10:31:23.451740+0000","last_fullsized":"2024-04-25T10:31:23.451740+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:58.112849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451667+0000","last_change":"2024-04-25T10:31:23.446467+0000","last_active":"2024-04-25T10:31:23.451667+0000","last_peered":"2024-04-25T10:31:23.451667+0000","last_clean":"2024-04-25T10:31:23.451667+0000","last_became_active":"2024-04-25T10:31:23.446185+0000","last_became_peered":"2024-04-25T10:31:23.446185+0000","last_unstale":"2024-04-25T10:31:23.451667+0000","last_undegraded":"2024-04-25T10:31:23.451667+0000","last_fullsized":"2024-04-25T10:31:23.451667+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:36.155132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.441966+0000","last_change":"2024-04-25T10:31:23.432662+0000","last_active":"2024-04-25T10:31:23.441966+0000","last_peered":"2024-04-25T10:31:23.441966+0000","last_clean":"2024-04-25T10:31:23.441966+0000","last_became_active":"2024-04-25T10:31:23.432317+0000","last_became_peered":"2024-04-25T10:31:23.432317+0000","last_unstale":"2024-04-25T10:31:23.441966+0000","last_undegraded":"2024-04-25T10:31:23.441966+0000","last_fullsized":"2024-04-25T10:31:23.441966+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:18.542250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.433442+0000","last_change":"2024-04-25T10:31:23.431527+0000","last_active":"2024-04-25T10:31:23.433442+0000","last_peered":"2024-04-25T10:31:23.433442+0000","last_clean":"2024-04-25T10:31:23.433442+0000","last_became_active":"2024-04-25T10:31:23.431141+0000","last_became_peered":"2024-04-25T10:31:23.431141+0000","last_unstale":"2024-04-25T10:31:23.433442+0000","last_undegraded":"2024-04-25T10:31:23.433442+0000","last_fullsized":"2024-04-25T10:31:23.433442+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450741+0000","last_change":"2024-04-25T10:31:23.441994+0000","last_active":"2024-04-25T10:31:23.450741+0000","last_peered":"2024-04-25T10:31:23.450741+0000","last_clean":"2024-04-25T10:31:23.450741+0000","last_became_active":"2024-04-25T10:31:23.439445+0000","last_became_peered":"2024-04-25T10:31:23.439445+0000","last_unstale":"2024-04-25T10:31:23.450741+0000","last_undegraded":"2024-04-25T10:31:23.450741+0000","last_fullsized":"2024-04-25T10:31:23.450741+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:56.634051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426113+0000","last_change":"2024-04-25T10:31:22.426113+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426113+0000","last_undegraded":"2024-04-25T10:31:22.426113+0000","last_fullsized":"2024-04-25T10:31:22.426113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461073+0000","last_change":"2024-04-25T10:31:23.458263+0000","last_active":"2024-04-25T10:31:23.461073+0000","last_peered":"2024-04-25T10:31:23.461073+0000","last_clean":"2024-04-25T10:31:23.461073+0000","last_became_active":"2024-04-25T10:31:23.457924+0000","last_became_peered":"2024-04-25T10:31:23.457924+0000","last_unstale":"2024-04-25T10:31:23.461073+0000","last_undegraded":"2024-04-25T10:31:23.461073+0000","last_fullsized":"2024-04-25T10:31:23.461073+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.239183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.425919+0000","last_change":"2024-04-25T10:31:22.425919+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.425919+0000","last_undegraded":"2024-04-25T10:31:22.425919+0000","last_fullsized":"2024-04-25T10:31:22.425919+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450551+0000","last_change":"2024-04-25T10:31:23.435570+0000","last_active":"2024-04-25T10:31:23.450551+0000","last_peered":"2024-04-25T10:31:23.450551+0000","last_clean":"2024-04-25T10:31:23.450551+0000","last_became_active":"2024-04-25T10:31:23.435293+0000","last_became_peered":"2024-04-25T10:31:23.435293+0000","last_unstale":"2024-04-25T10:31:23.450551+0000","last_undegraded":"2024-04-25T10:31:23.450551+0000","last_fullsized":"2024-04-25T10:31:23.450551+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:58.849145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.442632+0000","last_change":"2024-04-25T10:31:23.438686+0000","last_active":"2024-04-25T10:31:23.442632+0000","last_peered":"2024-04-25T10:31:23.442632+0000","last_clean":"2024-04-25T10:31:23.442632+0000","last_became_active":"2024-04-25T10:31:23.432709+0000","last_became_peered":"2024-04-25T10:31:23.432709+0000","last_unstale":"2024-04-25T10:31:23.442632+0000","last_undegraded":"2024-04-25T10:31:23.442632+0000","last_fullsized":"2024-04-25T10:31:23.442632+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:01.827369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451828+0000","last_change":"2024-04-25T10:31:23.443235+0000","last_active":"2024-04-25T10:31:23.451828+0000","last_peered":"2024-04-25T10:31:23.451828+0000","last_clean":"2024-04-25T10:31:23.451828+0000","last_became_active":"2024-04-25T10:31:23.442960+0000","last_became_peered":"2024-04-25T10:31:23.442960+0000","last_unstale":"2024-04-25T10:31:23.451828+0000","last_undegraded":"2024-04-25T10:31:23.451828+0000","last_fullsized":"2024-04-25T10:31:23.451828+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:50.981294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.447933+0000","last_change":"2024-04-25T10:31:23.445902+0000","last_active":"2024-04-25T10:31:23.447933+0000","last_peered":"2024-04-25T10:31:23.447933+0000","last_clean":"2024-04-25T10:31:23.447933+0000","last_became_active":"2024-04-25T10:31:23.445617+0000","last_became_peered":"2024-04-25T10:31:23.445617+0000","last_unstale":"2024-04-25T10:31:23.447933+0000","last_undegraded":"2024-04-25T10:31:23.447933+0000","last_fullsized":"2024-04-25T10:31:23.447933+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458804+0000","last_change":"2024-04-25T10:31:23.456990+0000","last_active":"2024-04-25T10:31:23.458804+0000","last_peered":"2024-04-25T10:31:23.458804+0000","last_clean":"2024-04-25T10:31:23.458804+0000","last_became_active":"2024-04-25T10:31:23.451914+0000","last_became_peered":"2024-04-25T10:31:23.451914+0000","last_unstale":"2024-04-25T10:31:23.458804+0000","last_undegraded":"2024-04-25T10:31:23.458804+0000","last_fullsized":"2024-04-25T10:31:23.458804+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:44.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459198+0000","last_change":"2024-04-25T10:31:22.444200+0000","last_active":"2024-04-25T10:31:23.459198+0000","last_peered":"2024-04-25T10:31:23.459198+0000","last_clean":"2024-04-25T10:31:23.459198+0000","last_became_active":"2024-04-25T10:31:22.443896+0000","last_became_peered":"2024-04-25T10:31:22.443896+0000","last_unstale":"2024-04-25T10:31:23.459198+0000","last_undegraded":"2024-04-25T10:31:23.459198+0000","last_fullsized":"2024-04-25T10:31:23.459198+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:07.360121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450721+0000","last_change":"2024-04-25T10:31:23.438605+0000","last_active":"2024-04-25T10:31:23.450721+0000","last_peered":"2024-04-25T10:31:23.450721+0000","last_clean":"2024-04-25T10:31:23.450721+0000","last_became_active":"2024-04-25T10:31:23.437768+0000","last_became_peered":"2024-04-25T10:31:23.437768+0000","last_unstale":"2024-04-25T10:31:23.450721+0000","last_undegraded":"2024-04-25T10:31:23.450721+0000","last_fullsized":"2024-04-25T10:31:23.450721+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.388703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459747+0000","last_change":"2024-04-25T10:31:22.436396+0000","last_active":"2024-04-25T10:31:23.459747+0000","last_peered":"2024-04-25T10:31:23.459747+0000","last_clean":"2024-04-25T10:31:23.459747+0000","last_became_active":"2024-04-25T10:31:22.436121+0000","last_became_peered":"2024-04-25T10:31:22.436121+0000","last_unstale":"2024-04-25T10:31:23.459747+0000","last_undegraded":"2024-04-25T10:31:23.459747+0000","last_fullsized":"2024-04-25T10:31:23.459747+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:51.783124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459463+0000","last_change":"2024-04-25T10:31:23.457159+0000","last_active":"2024-04-25T10:31:23.459463+0000","last_peered":"2024-04-25T10:31:23.459463+0000","last_clean":"2024-04-25T10:31:23.459463+0000","last_became_active":"2024-04-25T10:31:23.456837+0000","last_became_peered":"2024-04-25T10:31:23.456837+0000","last_unstale":"2024-04-25T10:31:23.459463+0000","last_undegraded":"2024-04-25T10:31:23.459463+0000","last_fullsized":"2024-04-25T10:31:23.459463+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:08.821901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443385+0000","last_change":"2024-04-25T10:31:22.439739+0000","last_active":"2024-04-25T10:31:22.443385+0000","last_peered":"2024-04-25T10:31:22.443385+0000","last_clean":"2024-04-25T10:31:22.443385+0000","last_became_active":"2024-04-25T10:31:22.439422+0000","last_became_peered":"2024-04-25T10:31:22.439422+0000","last_unstale":"2024-04-25T10:31:22.443385+0000","last_undegraded":"2024-04-25T10:31:22.443385+0000","last_fullsized":"2024-04-25T10:31:22.443385+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:41.108746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459321+0000","last_change":"2024-04-25T10:31:23.456576+0000","last_active":"2024-04-25T10:31:23.459321+0000","last_peered":"2024-04-25T10:31:23.459321+0000","last_clean":"2024-04-25T10:31:23.459321+0000","last_became_active":"2024-04-25T10:31:23.455891+0000","last_became_peered":"2024-04-25T10:31:23.455891+0000","last_unstale":"2024-04-25T10:31:23.459321+0000","last_undegraded":"2024-04-25T10:31:23.459321+0000","last_fullsized":"2024-04-25T10:31:23.459321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461755+0000","last_change":"2024-04-25T10:31:22.433482+0000","last_active":"2024-04-25T10:31:23.461755+0000","last_peered":"2024-04-25T10:31:23.461755+0000","last_clean":"2024-04-25T10:31:23.461755+0000","last_became_active":"2024-04-25T10:31:22.433201+0000","last_became_peered":"2024-04-25T10:31:22.433201+0000","last_unstale":"2024-04-25T10:31:23.461755+0000","last_undegraded":"2024-04-25T10:31:23.461755+0000","last_fullsized":"2024-04-25T10:31:23.461755+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:10.387788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454868+0000","last_change":"2024-04-25T10:31:23.452485+0000","last_active":"2024-04-25T10:31:23.454868+0000","last_peered":"2024-04-25T10:31:23.454868+0000","last_clean":"2024-04-25T10:31:23.454868+0000","last_became_active":"2024-04-25T10:31:23.452223+0000","last_became_peered":"2024-04-25T10:31:23.452223+0000","last_unstale":"2024-04-25T10:31:23.454868+0000","last_undegraded":"2024-04-25T10:31:23.454868+0000","last_fullsized":"2024-04-25T10:31:23.454868+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:38.344090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461678+0000","last_change":"2024-04-25T10:31:22.445426+0000","last_active":"2024-04-25T10:31:23.461678+0000","last_peered":"2024-04-25T10:31:23.461678+0000","last_clean":"2024-04-25T10:31:23.461678+0000","last_became_active":"2024-04-25T10:31:22.445054+0000","last_became_peered":"2024-04-25T10:31:22.445054+0000","last_unstale":"2024-04-25T10:31:23.461678+0000","last_undegraded":"2024-04-25T10:31:23.461678+0000","last_fullsized":"2024-04-25T10:31:23.461678+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443685+0000","last_change":"2024-04-25T10:31:23.434832+0000","last_active":"2024-04-25T10:31:23.443685+0000","last_peered":"2024-04-25T10:31:23.443685+0000","last_clean":"2024-04-25T10:31:23.443685+0000","last_became_active":"2024-04-25T10:31:23.433275+0000","last_became_peered":"2024-04-25T10:31:23.433275+0000","last_unstale":"2024-04-25T10:31:23.443685+0000","last_undegraded":"2024-04-25T10:31:23.443685+0000","last_fullsized":"2024-04-25T10:31:23.443685+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443196+0000","last_change":"2024-04-25T10:31:22.440188+0000","last_active":"2024-04-25T10:31:22.443196+0000","last_peered":"2024-04-25T10:31:22.443196+0000","last_clean":"2024-04-25T10:31:22.443196+0000","last_became_active":"2024-04-25T10:31:22.438568+0000","last_became_peered":"2024-04-25T10:31:22.438568+0000","last_unstale":"2024-04-25T10:31:22.443196+0000","last_undegraded":"2024-04-25T10:31:22.443196+0000","last_fullsized":"2024-04-25T10:31:22.443196+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.008298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.452463+0000","last_change":"2024-04-25T10:31:23.445257+0000","last_active":"2024-04-25T10:31:23.452463+0000","last_peered":"2024-04-25T10:31:23.452463+0000","last_clean":"2024-04-25T10:31:23.452463+0000","last_became_active":"2024-04-25T10:31:23.444984+0000","last_became_peered":"2024-04-25T10:31:23.444984+0000","last_unstale":"2024-04-25T10:31:23.452463+0000","last_undegraded":"2024-04-25T10:31:23.452463+0000","last_fullsized":"2024-04-25T10:31:23.452463+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:08.999842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426663+0000","last_change":"2024-04-25T10:31:22.443402+0000","last_active":"2024-04-25T10:31:23.426663+0000","last_peered":"2024-04-25T10:31:23.426663+0000","last_clean":"2024-04-25T10:31:23.426663+0000","last_became_active":"2024-04-25T10:31:22.443034+0000","last_became_peered":"2024-04-25T10:31:22.443034+0000","last_unstale":"2024-04-25T10:31:23.426663+0000","last_undegraded":"2024-04-25T10:31:23.426663+0000","last_fullsized":"2024-04-25T10:31:23.426663+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:08.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451254+0000","last_change":"2024-04-25T10:31:23.432502+0000","last_active":"2024-04-25T10:31:23.451254+0000","last_peered":"2024-04-25T10:31:23.451254+0000","last_clean":"2024-04-25T10:31:23.451254+0000","last_became_active":"2024-04-25T10:31:23.430905+0000","last_became_peered":"2024-04-25T10:31:23.430905+0000","last_unstale":"2024-04-25T10:31:23.451254+0000","last_undegraded":"2024-04-25T10:31:23.451254+0000","last_fullsized":"2024-04-25T10:31:23.451254+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:24.065113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.439096+0000","last_change":"2024-04-25T10:31:22.446818+0000","last_active":"2024-04-25T10:31:23.439096+0000","last_peered":"2024-04-25T10:31:23.439096+0000","last_clean":"2024-04-25T10:31:23.439096+0000","last_became_active":"2024-04-25T10:31:22.446198+0000","last_became_peered":"2024-04-25T10:31:22.446198+0000","last_unstale":"2024-04-25T10:31:23.439096+0000","last_undegraded":"2024-04-25T10:31:23.439096+0000","last_fullsized":"2024-04-25T10:31:23.439096+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:27.331081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460947+0000","last_change":"2024-04-25T10:31:23.460404+0000","last_active":"2024-04-25T10:31:23.460947+0000","last_peered":"2024-04-25T10:31:23.460947+0000","last_clean":"2024-04-25T10:31:23.460947+0000","last_became_active":"2024-04-25T10:31:23.459692+0000","last_became_peered":"2024-04-25T10:31:23.459692+0000","last_unstale":"2024-04-25T10:31:23.460947+0000","last_undegraded":"2024-04-25T10:31:23.460947+0000","last_fullsized":"2024-04-25T10:31:23.460947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426137+0000","last_change":"2024-04-25T10:31:22.435228+0000","last_active":"2024-04-25T10:31:23.426137+0000","last_peered":"2024-04-25T10:31:23.426137+0000","last_clean":"2024-04-25T10:31:23.426137+0000","last_became_active":"2024-04-25T10:31:22.434923+0000","last_became_peered":"2024-04-25T10:31:22.434923+0000","last_unstale":"2024-04-25T10:31:23.426137+0000","last_undegraded":"2024-04-25T10:31:23.426137+0000","last_fullsized":"2024-04-25T10:31:23.426137+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:49.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.434391+0000","last_change":"2024-04-25T10:31:23.433349+0000","last_active":"2024-04-25T10:31:23.434391+0000","last_peered":"2024-04-25T10:31:23.434391+0000","last_clean":"2024-04-25T10:31:23.434391+0000","last_became_active":"2024-04-25T10:31:23.433057+0000","last_became_peered":"2024-04-25T10:31:23.433057+0000","last_unstale":"2024-04-25T10:31:23.434391+0000","last_undegraded":"2024-04-25T10:31:23.434391+0000","last_fullsized":"2024-04-25T10:31:23.434391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:08.821901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.444353+0000","last_change":"2024-04-25T10:31:22.442622+0000","last_active":"2024-04-25T10:31:22.444353+0000","last_peered":"2024-04-25T10:31:22.444353+0000","last_clean":"2024-04-25T10:31:22.444353+0000","last_became_active":"2024-04-25T10:31:22.441086+0000","last_became_peered":"2024-04-25T10:31:22.441086+0000","last_unstale":"2024-04-25T10:31:22.444353+0000","last_undegraded":"2024-04-25T10:31:22.444353+0000","last_fullsized":"2024-04-25T10:31:22.444353+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:42.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426939+0000","last_change":"2024-04-25T10:31:22.426939+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426939+0000","last_undegraded":"2024-04-25T10:31:22.426939+0000","last_fullsized":"2024-04-25T10:31:22.426939+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461514+0000","last_change":"2024-04-25T10:31:22.436482+0000","last_active":"2024-04-25T10:31:23.461514+0000","last_peered":"2024-04-25T10:31:23.461514+0000","last_clean":"2024-04-25T10:31:23.461514+0000","last_became_active":"2024-04-25T10:31:22.436192+0000","last_became_peered":"2024-04-25T10:31:22.436192+0000","last_unstale":"2024-04-25T10:31:23.461514+0000","last_undegraded":"2024-04-25T10:31:23.461514+0000","last_fullsized":"2024-04-25T10:31:23.461514+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454216+0000","last_change":"2024-04-25T10:31:23.448277+0000","last_active":"2024-04-25T10:31:23.454216+0000","last_peered":"2024-04-25T10:31:23.454216+0000","last_clean":"2024-04-25T10:31:23.454216+0000","last_became_active":"2024-04-25T10:31:23.447714+0000","last_became_peered":"2024-04-25T10:31:23.447714+0000","last_unstale":"2024-04-25T10:31:23.454216+0000","last_undegraded":"2024-04-25T10:31:23.454216+0000","last_fullsized":"2024-04-25T10:31:23.454216+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456921+0000","last_change":"2024-04-25T10:31:22.437143+0000","last_active":"2024-04-25T10:31:23.456921+0000","last_peered":"2024-04-25T10:31:23.456921+0000","last_clean":"2024-04-25T10:31:23.456921+0000","last_became_active":"2024-04-25T10:31:22.436870+0000","last_became_peered":"2024-04-25T10:31:22.436870+0000","last_unstale":"2024-04-25T10:31:23.456921+0000","last_undegraded":"2024-04-25T10:31:23.456921+0000","last_fullsized":"2024-04-25T10:31:23.456921+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458152+0000","last_change":"2024-04-25T10:31:23.452399+0000","last_active":"2024-04-25T10:31:23.458152+0000","last_peered":"2024-04-25T10:31:23.458152+0000","last_clean":"2024-04-25T10:31:23.458152+0000","last_became_active":"2024-04-25T10:31:23.451792+0000","last_became_peered":"2024-04-25T10:31:23.451792+0000","last_unstale":"2024-04-25T10:31:23.458152+0000","last_undegraded":"2024-04-25T10:31:23.458152+0000","last_fullsized":"2024-04-25T10:31:23.458152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.430252+0000","last_change":"2024-04-25T10:31:22.435264+0000","last_active":"2024-04-25T10:31:23.430252+0000","last_peered":"2024-04-25T10:31:23.430252+0000","last_clean":"2024-04-25T10:31:23.430252+0000","last_became_active":"2024-04-25T10:31:22.434517+0000","last_became_peered":"2024-04-25T10:31:22.434517+0000","last_unstale":"2024-04-25T10:31:23.430252+0000","last_undegraded":"2024-04-25T10:31:23.430252+0000","last_fullsized":"2024-04-25T10:31:23.430252+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458070+0000","last_change":"2024-04-25T10:31:23.452667+0000","last_active":"2024-04-25T10:31:23.458070+0000","last_peered":"2024-04-25T10:31:23.458070+0000","last_clean":"2024-04-25T10:31:23.458070+0000","last_became_active":"2024-04-25T10:31:23.452366+0000","last_became_peered":"2024-04-25T10:31:23.452366+0000","last_unstale":"2024-04-25T10:31:23.458070+0000","last_undegraded":"2024-04-25T10:31:23.458070+0000","last_fullsized":"2024-04-25T10:31:23.458070+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:37.926986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443232+0000","last_change":"2024-04-25T10:31:22.441210+0000","last_active":"2024-04-25T10:31:22.443232+0000","last_peered":"2024-04-25T10:31:22.443232+0000","last_clean":"2024-04-25T10:31:22.443232+0000","last_became_active":"2024-04-25T10:31:22.440936+0000","last_became_peered":"2024-04-25T10:31:22.440936+0000","last_unstale":"2024-04-25T10:31:22.443232+0000","last_undegraded":"2024-04-25T10:31:22.443232+0000","last_fullsized":"2024-04-25T10:31:22.443232+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:54.030915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448629+0000","last_change":"2024-04-25T10:31:23.446637+0000","last_active":"2024-04-25T10:31:23.448629+0000","last_peered":"2024-04-25T10:31:23.448629+0000","last_clean":"2024-04-25T10:31:23.448629+0000","last_became_active":"2024-04-25T10:31:23.445987+0000","last_became_peered":"2024-04-25T10:31:23.445987+0000","last_unstale":"2024-04-25T10:31:23.448629+0000","last_undegraded":"2024-04-25T10:31:23.448629+0000","last_fullsized":"2024-04-25T10:31:23.448629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:28.466968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.439829+0000","last_change":"2024-04-25T10:31:22.436186+0000","last_active":"2024-04-25T10:31:22.439829+0000","last_peered":"2024-04-25T10:31:22.439829+0000","last_clean":"2024-04-25T10:31:22.439829+0000","last_became_active":"2024-04-25T10:31:22.435837+0000","last_became_peered":"2024-04-25T10:31:22.435837+0000","last_unstale":"2024-04-25T10:31:22.439829+0000","last_undegraded":"2024-04-25T10:31:22.439829+0000","last_fullsized":"2024-04-25T10:31:22.439829+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:41.160467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458756+0000","last_change":"2024-04-25T10:31:23.455136+0000","last_active":"2024-04-25T10:31:23.458756+0000","last_peered":"2024-04-25T10:31:23.458756+0000","last_clean":"2024-04-25T10:31:23.458756+0000","last_became_active":"2024-04-25T10:31:23.454870+0000","last_became_peered":"2024-04-25T10:31:23.454870+0000","last_unstale":"2024-04-25T10:31:23.458756+0000","last_undegraded":"2024-04-25T10:31:23.458756+0000","last_fullsized":"2024-04-25T10:31:23.458756+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:39.353583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460494+0000","last_change":"2024-04-25T10:31:22.445257+0000","last_active":"2024-04-25T10:31:23.460494+0000","last_peered":"2024-04-25T10:31:23.460494+0000","last_clean":"2024-04-25T10:31:23.460494+0000","last_became_active":"2024-04-25T10:31:22.444918+0000","last_became_peered":"2024-04-25T10:31:22.444918+0000","last_unstale":"2024-04-25T10:31:23.460494+0000","last_undegraded":"2024-04-25T10:31:23.460494+0000","last_fullsized":"2024-04-25T10:31:23.460494+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455592+0000","last_change":"2024-04-25T10:31:23.453329+0000","last_active":"2024-04-25T10:31:23.455592+0000","last_peered":"2024-04-25T10:31:23.455592+0000","last_clean":"2024-04-25T10:31:23.455592+0000","last_became_active":"2024-04-25T10:31:23.452956+0000","last_became_peered":"2024-04-25T10:31:23.452956+0000","last_unstale":"2024-04-25T10:31:23.455592+0000","last_undegraded":"2024-04-25T10:31:23.455592+0000","last_fullsized":"2024-04-25T10:31:23.455592+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:11.453191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455285+0000","last_change":"2024-04-25T10:31:22.436617+0000","last_active":"2024-04-25T10:31:23.455285+0000","last_peered":"2024-04-25T10:31:23.455285+0000","last_clean":"2024-04-25T10:31:23.455285+0000","last_became_active":"2024-04-25T10:31:22.436313+0000","last_became_peered":"2024-04-25T10:31:22.436313+0000","last_unstale":"2024-04-25T10:31:23.455285+0000","last_undegraded":"2024-04-25T10:31:23.455285+0000","last_fullsized":"2024-04-25T10:31:23.455285+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458066+0000","last_change":"2024-04-25T10:31:23.455754+0000","last_active":"2024-04-25T10:31:23.458066+0000","last_peered":"2024-04-25T10:31:23.458066+0000","last_clean":"2024-04-25T10:31:23.458066+0000","last_became_active":"2024-04-25T10:31:23.447211+0000","last_became_peered":"2024-04-25T10:31:23.447211+0000","last_unstale":"2024-04-25T10:31:23.458066+0000","last_undegraded":"2024-04-25T10:31:23.458066+0000","last_fullsized":"2024-04-25T10:31:23.458066+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.783049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460720+0000","last_change":"2024-04-25T10:31:22.444016+0000","last_active":"2024-04-25T10:31:23.460720+0000","last_peered":"2024-04-25T10:31:23.460720+0000","last_clean":"2024-04-25T10:31:23.460720+0000","last_became_active":"2024-04-25T10:31:22.443711+0000","last_became_peered":"2024-04-25T10:31:22.443711+0000","last_unstale":"2024-04-25T10:31:23.460720+0000","last_undegraded":"2024-04-25T10:31:23.460720+0000","last_fullsized":"2024-04-25T10:31:23.460720+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:28.985863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.440052+0000","last_change":"2024-04-25T10:31:23.434350+0000","last_active":"2024-04-25T10:31:23.440052+0000","last_peered":"2024-04-25T10:31:23.440052+0000","last_clean":"2024-04-25T10:31:23.440052+0000","last_became_active":"2024-04-25T10:31:23.434029+0000","last_became_peered":"2024-04-25T10:31:23.434029+0000","last_unstale":"2024-04-25T10:31:23.440052+0000","last_undegraded":"2024-04-25T10:31:23.440052+0000","last_fullsized":"2024-04-25T10:31:23.440052+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431185+0000","last_change":"2024-04-25T10:31:22.438645+0000","last_active":"2024-04-25T10:31:23.431185+0000","last_peered":"2024-04-25T10:31:23.431185+0000","last_clean":"2024-04-25T10:31:23.431185+0000","last_became_active":"2024-04-25T10:31:22.438312+0000","last_became_peered":"2024-04-25T10:31:22.438312+0000","last_unstale":"2024-04-25T10:31:23.431185+0000","last_undegraded":"2024-04-25T10:31:23.431185+0000","last_fullsized":"2024-04-25T10:31:23.431185+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456346+0000","last_change":"2024-04-25T10:31:23.450857+0000","last_active":"2024-04-25T10:31:23.456346+0000","last_peered":"2024-04-25T10:31:23.456346+0000","last_clean":"2024-04-25T10:31:23.456346+0000","last_became_active":"2024-04-25T10:31:23.449482+0000","last_became_peered":"2024-04-25T10:31:23.449482+0000","last_unstale":"2024-04-25T10:31:23.456346+0000","last_undegraded":"2024-04-25T10:31:23.456346+0000","last_fullsized":"2024-04-25T10:31:23.456346+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460111+0000","last_change":"2024-04-25T10:31:22.439123+0000","last_active":"2024-04-25T10:31:23.460111+0000","last_peered":"2024-04-25T10:31:23.460111+0000","last_clean":"2024-04-25T10:31:23.460111+0000","last_became_active":"2024-04-25T10:31:22.438833+0000","last_became_peered":"2024-04-25T10:31:22.438833+0000","last_unstale":"2024-04-25T10:31:23.460111+0000","last_undegraded":"2024-04-25T10:31:23.460111+0000","last_fullsized":"2024-04-25T10:31:23.460111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.449153+0000","last_change":"2024-04-25T10:31:23.444511+0000","last_active":"2024-04-25T10:31:23.449153+0000","last_peered":"2024-04-25T10:31:23.449153+0000","last_clean":"2024-04-25T10:31:23.449153+0000","last_became_active":"2024-04-25T10:31:23.444242+0000","last_became_peered":"2024-04-25T10:31:23.444242+0000","last_unstale":"2024-04-25T10:31:23.449153+0000","last_undegraded":"2024-04-25T10:31:23.449153+0000","last_fullsized":"2024-04-25T10:31:23.449153+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:38.996519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.442493+0000","last_change":"2024-04-25T10:31:22.438630+0000","last_active":"2024-04-25T10:31:22.442493+0000","last_peered":"2024-04-25T10:31:22.442493+0000","last_clean":"2024-04-25T10:31:22.442493+0000","last_became_active":"2024-04-25T10:31:22.438311+0000","last_became_peered":"2024-04-25T10:31:22.438311+0000","last_unstale":"2024-04-25T10:31:22.442493+0000","last_undegraded":"2024-04-25T10:31:22.442493+0000","last_fullsized":"2024-04-25T10:31:22.442493+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:43.342262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.441051+0000","last_change":"2024-04-25T10:31:23.431648+0000","last_active":"2024-04-25T10:31:23.441051+0000","last_peered":"2024-04-25T10:31:23.441051+0000","last_clean":"2024-04-25T10:31:23.441051+0000","last_became_active":"2024-04-25T10:31:23.430571+0000","last_became_peered":"2024-04-25T10:31:23.430571+0000","last_unstale":"2024-04-25T10:31:23.441051+0000","last_undegraded":"2024-04-25T10:31:23.441051+0000","last_fullsized":"2024-04-25T10:31:23.441051+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:14.895999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.435164+0000","last_change":"2024-04-25T10:31:22.446969+0000","last_active":"2024-04-25T10:31:23.435164+0000","last_peered":"2024-04-25T10:31:23.435164+0000","last_clean":"2024-04-25T10:31:23.435164+0000","last_became_active":"2024-04-25T10:31:22.446445+0000","last_became_peered":"2024-04-25T10:31:22.446445+0000","last_unstale":"2024-04-25T10:31:23.435164+0000","last_undegraded":"2024-04-25T10:31:23.435164+0000","last_fullsized":"2024-04-25T10:31:23.435164+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443904+0000","last_change":"2024-04-25T10:31:23.442351+0000","last_active":"2024-04-25T10:31:23.443904+0000","last_peered":"2024-04-25T10:31:23.443904+0000","last_clean":"2024-04-25T10:31:23.443904+0000","last_became_active":"2024-04-25T10:31:23.442056+0000","last_became_peered":"2024-04-25T10:31:23.442056+0000","last_unstale":"2024-04-25T10:31:23.443904+0000","last_undegraded":"2024-04-25T10:31:23.443904+0000","last_fullsized":"2024-04-25T10:31:23.443904+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:29.781075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.444571+0000","last_change":"2024-04-25T10:31:22.443307+0000","last_active":"2024-04-25T10:31:22.444571+0000","last_peered":"2024-04-25T10:31:22.444571+0000","last_clean":"2024-04-25T10:31:22.444571+0000","last_became_active":"2024-04-25T10:31:22.442920+0000","last_became_peered":"2024-04-25T10:31:22.442920+0000","last_unstale":"2024-04-25T10:31:22.444571+0000","last_undegraded":"2024-04-25T10:31:22.444571+0000","last_fullsized":"2024-04-25T10:31:22.444571+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:38.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451144+0000","last_change":"2024-04-25T10:31:23.444361+0000","last_active":"2024-04-25T10:31:23.451144+0000","last_peered":"2024-04-25T10:31:23.451144+0000","last_clean":"2024-04-25T10:31:23.451144+0000","last_became_active":"2024-04-25T10:31:23.444080+0000","last_became_peered":"2024-04-25T10:31:23.444080+0000","last_unstale":"2024-04-25T10:31:23.451144+0000","last_undegraded":"2024-04-25T10:31:23.451144+0000","last_fullsized":"2024-04-25T10:31:23.451144+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.478405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.441407+0000","last_change":"2024-04-25T10:31:22.436880+0000","last_active":"2024-04-25T10:31:22.441407+0000","last_peered":"2024-04-25T10:31:22.441407+0000","last_clean":"2024-04-25T10:31:22.441407+0000","last_became_active":"2024-04-25T10:31:22.436570+0000","last_became_peered":"2024-04-25T10:31:22.436570+0000","last_unstale":"2024-04-25T10:31:22.441407+0000","last_undegraded":"2024-04-25T10:31:22.441407+0000","last_fullsized":"2024-04-25T10:31:22.441407+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:11.125776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.453381+0000","last_change":"2024-04-25T10:31:23.449985+0000","last_active":"2024-04-25T10:31:23.453381+0000","last_peered":"2024-04-25T10:31:23.453381+0000","last_clean":"2024-04-25T10:31:23.453381+0000","last_became_active":"2024-04-25T10:31:23.449345+0000","last_became_peered":"2024-04-25T10:31:23.449345+0000","last_unstale":"2024-04-25T10:31:23.453381+0000","last_undegraded":"2024-04-25T10:31:23.453381+0000","last_fullsized":"2024-04-25T10:31:23.453381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:28.103406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431065+0000","last_change":"2024-04-25T10:31:22.440950+0000","last_active":"2024-04-25T10:31:23.431065+0000","last_peered":"2024-04-25T10:31:23.431065+0000","last_clean":"2024-04-25T10:31:23.431065+0000","last_became_active":"2024-04-25T10:31:22.440664+0000","last_became_peered":"2024-04-25T10:31:22.440664+0000","last_unstale":"2024-04-25T10:31:23.431065+0000","last_undegraded":"2024-04-25T10:31:23.431065+0000","last_fullsized":"2024-04-25T10:31:23.431065+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:21.897192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456515+0000","last_change":"2024-04-25T10:31:23.448149+0000","last_active":"2024-04-25T10:31:23.456515+0000","last_peered":"2024-04-25T10:31:23.456515+0000","last_clean":"2024-04-25T10:31:23.456515+0000","last_became_active":"2024-04-25T10:31:23.445318+0000","last_became_peered":"2024-04-25T10:31:23.445318+0000","last_unstale":"2024-04-25T10:31:23.456515+0000","last_undegraded":"2024-04-25T10:31:23.456515+0000","last_fullsized":"2024-04-25T10:31:23.456515+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:53.854726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431621+0000","last_change":"2024-04-25T10:31:22.446060+0000","last_active":"2024-04-25T10:31:23.431621+0000","last_peered":"2024-04-25T10:31:23.431621+0000","last_clean":"2024-04-25T10:31:23.431621+0000","last_became_active":"2024-04-25T10:31:22.445706+0000","last_became_peered":"2024-04-25T10:31:22.445706+0000","last_unstale":"2024-04-25T10:31:23.431621+0000","last_undegraded":"2024-04-25T10:31:23.431621+0000","last_fullsized":"2024-04-25T10:31:23.431621+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:31.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458152+0000","last_change":"2024-04-25T10:31:23.453869+0000","last_active":"2024-04-25T10:31:23.458152+0000","last_peered":"2024-04-25T10:31:23.458152+0000","last_clean":"2024-04-25T10:31:23.458152+0000","last_became_active":"2024-04-25T10:31:23.453600+0000","last_became_peered":"2024-04-25T10:31:23.453600+0000","last_unstale":"2024-04-25T10:31:23.458152+0000","last_undegraded":"2024-04-25T10:31:23.458152+0000","last_fullsized":"2024-04-25T10:31:23.458152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:11.359233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431702+0000","last_change":"2024-04-25T10:31:22.445692+0000","last_active":"2024-04-25T10:31:23.431702+0000","last_peered":"2024-04-25T10:31:23.431702+0000","last_clean":"2024-04-25T10:31:23.431702+0000","last_became_active":"2024-04-25T10:31:22.445368+0000","last_became_peered":"2024-04-25T10:31:22.445368+0000","last_unstale":"2024-04-25T10:31:23.431702+0000","last_undegraded":"2024-04-25T10:31:23.431702+0000","last_fullsized":"2024-04-25T10:31:23.431702+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:07.812518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.438304+0000","last_change":"2024-04-25T10:31:23.434539+0000","last_active":"2024-04-25T10:31:23.438304+0000","last_peered":"2024-04-25T10:31:23.438304+0000","last_clean":"2024-04-25T10:31:23.438304+0000","last_became_active":"2024-04-25T10:31:23.434040+0000","last_became_peered":"2024-04-25T10:31:23.434040+0000","last_unstale":"2024-04-25T10:31:23.438304+0000","last_undegraded":"2024-04-25T10:31:23.438304+0000","last_fullsized":"2024-04-25T10:31:23.438304+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:08.999842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.427426+0000","last_change":"2024-04-25T10:31:22.431495+0000","last_active":"2024-04-25T10:31:23.427426+0000","last_peered":"2024-04-25T10:31:23.427426+0000","last_clean":"2024-04-25T10:31:23.427426+0000","last_became_active":"2024-04-25T10:31:22.431217+0000","last_became_peered":"2024-04-25T10:31:22.431217+0000","last_unstale":"2024-04-25T10:31:23.427426+0000","last_undegraded":"2024-04-25T10:31:23.427426+0000","last_fullsized":"2024-04-25T10:31:23.427426+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:27.633982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450575+0000","last_change":"2024-04-25T10:31:23.445933+0000","last_active":"2024-04-25T10:31:23.450575+0000","last_peered":"2024-04-25T10:31:23.450575+0000","last_clean":"2024-04-25T10:31:23.450575+0000","last_became_active":"2024-04-25T10:31:23.445660+0000","last_became_peered":"2024-04-25T10:31:23.445660+0000","last_unstale":"2024-04-25T10:31:23.450575+0000","last_undegraded":"2024-04-25T10:31:23.450575+0000","last_fullsized":"2024-04-25T10:31:23.450575+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:28.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.436951+0000","last_change":"2024-04-25T10:31:22.445594+0000","last_active":"2024-04-25T10:31:23.436951+0000","last_peered":"2024-04-25T10:31:23.436951+0000","last_clean":"2024-04-25T10:31:23.436951+0000","last_became_active":"2024-04-25T10:31:22.445288+0000","last_became_peered":"2024-04-25T10:31:22.445288+0000","last_unstale":"2024-04-25T10:31:23.436951+0000","last_undegraded":"2024-04-25T10:31:23.436951+0000","last_fullsized":"2024-04-25T10:31:23.436951+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:54.229800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.425826+0000","last_change":"2024-04-25T10:31:22.425826+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.425826+0000","last_undegraded":"2024-04-25T10:31:22.425826+0000","last_fullsized":"2024-04-25T10:31:22.425826+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.436066+0000","last_change":"2024-04-25T10:31:22.440617+0000","last_active":"2024-04-25T10:31:23.436066+0000","last_peered":"2024-04-25T10:31:23.436066+0000","last_clean":"2024-04-25T10:31:23.436066+0000","last_became_active":"2024-04-25T10:31:22.440345+0000","last_became_peered":"2024-04-25T10:31:22.440345+0000","last_unstale":"2024-04-25T10:31:23.436066+0000","last_undegraded":"2024-04-25T10:31:23.436066+0000","last_fullsized":"2024-04-25T10:31:23.436066+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:11.911547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443957+0000","last_change":"2024-04-25T10:31:23.443024+0000","last_active":"2024-04-25T10:31:23.443957+0000","last_peered":"2024-04-25T10:31:23.443957+0000","last_clean":"2024-04-25T10:31:23.443957+0000","last_became_active":"2024-04-25T10:31:23.442748+0000","last_became_peered":"2024-04-25T10:31:23.442748+0000","last_unstale":"2024-04-25T10:31:23.443957+0000","last_undegraded":"2024-04-25T10:31:23.443957+0000","last_fullsized":"2024-04-25T10:31:23.443957+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:36.503940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426863+0000","last_change":"2024-04-25T10:31:22.441054+0000","last_active":"2024-04-25T10:31:23.426863+0000","last_peered":"2024-04-25T10:31:23.426863+0000","last_clean":"2024-04-25T10:31:23.426863+0000","last_became_active":"2024-04-25T10:31:22.440642+0000","last_became_peered":"2024-04-25T10:31:22.440642+0000","last_unstale":"2024-04-25T10:31:23.426863+0000","last_undegraded":"2024-04-25T10:31:23.426863+0000","last_fullsized":"2024-04-25T10:31:23.426863+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451102+0000","last_change":"2024-04-25T10:31:23.446822+0000","last_active":"2024-04-25T10:31:23.451102+0000","last_peered":"2024-04-25T10:31:23.451102+0000","last_clean":"2024-04-25T10:31:23.451102+0000","last_became_active":"2024-04-25T10:31:23.446539+0000","last_became_peered":"2024-04-25T10:31:23.446539+0000","last_unstale":"2024-04-25T10:31:23.451102+0000","last_undegraded":"2024-04-25T10:31:23.451102+0000","last_fullsized":"2024-04-25T10:31:23.451102+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.995246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454545+0000","last_change":"2024-04-25T10:31:22.436501+0000","last_active":"2024-04-25T10:31:23.454545+0000","last_peered":"2024-04-25T10:31:23.454545+0000","last_clean":"2024-04-25T10:31:23.454545+0000","last_became_active":"2024-04-25T10:31:22.436167+0000","last_became_peered":"2024-04-25T10:31:22.436167+0000","last_unstale":"2024-04-25T10:31:23.454545+0000","last_undegraded":"2024-04-25T10:31:23.454545+0000","last_fullsized":"2024-04-25T10:31:23.454545+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.310017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":478,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426941+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:23.426941+0000","last_peered":"2024-04-25T10:31:23.426941+0000","last_clean":"2024-04-25T10:31:23.426941+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:23.426941+0000","last_undegraded":"2024-04-25T10:31:23.426941+0000","last_fullsized":"2024-04-25T10:31:23.426941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2277376,"data_stored":998047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55620,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316220,"statfs":{"total":96636764160,"available":96579809280,"internally_reserved":0,"allocated":2277376,"data_stored":998095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60372,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311468,"statfs":{"total":96636764160,"available":96574943232,"internally_reserved":0,"allocated":2883584,"data_stored":1589121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509606,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47284,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324556,"statfs":{"total":96636764160,"available":96588345344,"internally_reserved":0,"allocated":2260992,"data_stored":991250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69456,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302384,"statfs":{"total":96636764160,"available":96565641216,"internally_reserved":0,"allocated":2293760,"data_stored":999741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1588393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509607,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59780,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312060,"statfs":{"total":96636764160,"available":96575549440,"internally_reserved":0,"allocated":2277376,"data_stored":998607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65248,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306592,"statfs":{"total":96636764160,"available":96569950208,"internally_reserved":0,"allocated":2248704,"data_stored":984975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:27.547 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 Processor -- start 2024-04-25T10:31:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 -- start start 2024-04-25T10:31:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a0c0b38d0 0x7f7a0c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15eea0 0x7f7a0c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c15e400 con 0x7f7a0c15d9b0 2024-04-25T10:31:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0b02e0 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a12200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0afda0 con 0x7f7a0c15eea0 2024-04-25T10:31:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7a0c15d9b0 0x7f7a0c0af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55682/0 (socket says 172.21.15.45:55682) 2024-04-25T10:31:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0b400640 1 -- 172.21.15.45:0/1563372391 learned_addr learned my addr 172.21.15.45:0/1563372391 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0aa00640 1 --2- 172.21.15.45:0/1563372391 >> v2:172.21.15.45:3300/0 conn(0x7f7a0c0b38d0 0x7f7a0c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0be00640 1 --2- 172.21.15.45:0/1563372391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15eea0 0x7f7a0c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0a000640 1 -- 172.21.15.45:0/1563372391 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965719606 0 0) 0x7f7a0c15e400 con 0x7f7a0c15d9b0 2024-04-25T10:31:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0aa00640 1 -- 172.21.15.45:0/1563372391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15eea0 msgr2=0x7f7a0c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0aa00640 1 --2- 172.21.15.45:0/1563372391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15eea0 0x7f7a0c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0aa00640 1 -- 172.21.15.45:0/1563372391 >> v1:172.21.15.45:6789/0 conn(0x7f7a0c15d9b0 legacy=0x7f7a0c0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.726+0000 7f7a0aa00640 1 -- 172.21.15.45:0/1563372391 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0b0000 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a0aa00640 1 --2- 172.21.15.45:0/1563372391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15dd90 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097c0 tx=0x7f79f801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465a444087060ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a0a000640 1 -- 172.21.15.45:0/1563372391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80b0070 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a0a000640 1 -- 172.21.15.45:0/1563372391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f80ab600 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a0a000640 1 -- 172.21.15.45:0/1563372391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80ab9d0 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 -- 172.21.15.45:0/1563372391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 msgr2=0x7f7a0c15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 --2- 172.21.15.45:0/1563372391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15dd90 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097c0 tx=0x7f79f801c3b0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 -- 172.21.15.45:0/1563372391 shutdown_connections 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 --2- 172.21.15.45:0/1563372391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15eea0 0x7f7a0c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 --2- 172.21.15.45:0/1563372391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15dd90 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 -- 172.21.15.45:0/1563372391 >> 172.21.15.45:0/1563372391 conn(0x7f7a0c06ac90 msgr2=0x7f7a0c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 -- 172.21.15.45:0/1563372391 shutdown_connections 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 -- 172.21.15.45:0/1563372391 wait complete. 2024-04-25T10:31:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 Processor -- start 2024-04-25T10:31:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.730+0000 7f7a12200640 1 -- start start 2024-04-25T10:31:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15d9b0 0x7f7a0c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a0c15eea0 0x7f7a0c14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0d1120 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0d0fa0 con 0x7f7a0c15eea0 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0d12a0 con 0x7f7a0c15d9b0 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57598/0 (socket says 172.21.15.45:57598) 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 -- 172.21.15.45:0/1886123374 learned_addr learned my addr 172.21.15.45:0/1886123374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 -- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15d9b0 msgr2=0x7f7a0c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15d9b0 0x7f7a0c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 -- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a0c15eea0 msgr2=0x7f7a0c14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a0c15eea0 0x7f7a0c14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 -- 172.21.15.45:0/1886123374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0ce2d0 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a0b400640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15bc40 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0011f0 tx=0x7f79fc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc018070 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc0074e0 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc004a10 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c0ce5b0 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c0ceb70 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.734+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c04ef30 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.738+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79fc01b070 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.738+0000 7f7a03400640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79ec066f60 0x7f79ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.738+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79fc084360 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.738+0000 7f7a0aa00640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79ec066f60 0x7f79ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.738+0000 7f7a0aa00640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79ec066f60 0x7f79ec069420 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f79f80ad000 tx=0x7f79f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.742+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79fc055400 con 0x7f7a0c0b38d0 2024-04-25T10:31:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.898+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a0c0719b0 con 0x7f79ec066f60 2024-04-25T10:31:27.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.906+0000 7f7a03400640 1 -- 172.21.15.45:0/1886123374 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251989 (secure 0 0 0) 0x7f7a0c0719b0 con 0x7f79ec066f60 2024-04-25T10:31:27.911 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:27.912 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:31:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79ec066f60 msgr2=0x7f79ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79ec066f60 0x7f79ec069420 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f79f80ad000 tx=0x7f79f8008040 comp rx=0 tx=0).stop 2024-04-25T10:31:27.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 msgr2=0x7f7a0c15bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15bc40 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0011f0 tx=0x7f79fc00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 shutdown_connections 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f79ec066f60 0x7f79ec069420 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a0c15eea0 0x7f7a0c14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a0c15d9b0 0x7f7a0c0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 --2- 172.21.15.45:0/1886123374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a0c0b38d0 0x7f7a0c15bc40 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 >> 172.21.15.45:0/1886123374 conn(0x7f7a0c06ac90 msgr2=0x7f7a0c071e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 shutdown_connections 2024-04-25T10:31:27.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:27.910+0000 7f7a12200640 1 -- 172.21.15.45:0/1886123374 wait complete. 2024-04-25T10:31:27.931 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2024-04-25T10:31:27.064660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461240,"kb_used_data":18932,"kb_used_omap":12,"kb_used_meta":442163,"kb_avail":754513480,"statfs":{"total":773094113280,"available":772621803520,"internally_reserved":0,"allocated":19386368,"data_stored":9148229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":452775472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.646564"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.442975+0000","last_change":"2024-04-25T10:31:23.435470+0000","last_active":"2024-04-25T10:31:23.442975+0000","last_peered":"2024-04-25T10:31:23.442975+0000","last_clean":"2024-04-25T10:31:23.442975+0000","last_became_active":"2024-04-25T10:31:23.433761+0000","last_became_peered":"2024-04-25T10:31:23.433761+0000","last_unstale":"2024-04-25T10:31:23.442975+0000","last_undegraded":"2024-04-25T10:31:23.442975+0000","last_fullsized":"2024-04-25T10:31:23.442975+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456702+0000","last_change":"2024-04-25T10:31:23.449323+0000","last_active":"2024-04-25T10:31:23.456702+0000","last_peered":"2024-04-25T10:31:23.456702+0000","last_clean":"2024-04-25T10:31:23.456702+0000","last_became_active":"2024-04-25T10:31:23.448894+0000","last_became_peered":"2024-04-25T10:31:23.448894+0000","last_unstale":"2024-04-25T10:31:23.456702+0000","last_undegraded":"2024-04-25T10:31:23.456702+0000","last_fullsized":"2024-04-25T10:31:23.456702+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:51.236038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459271+0000","last_change":"2024-04-25T10:31:23.455956+0000","last_active":"2024-04-25T10:31:23.459271+0000","last_peered":"2024-04-25T10:31:23.459271+0000","last_clean":"2024-04-25T10:31:23.459271+0000","last_became_active":"2024-04-25T10:31:23.451989+0000","last_became_peered":"2024-04-25T10:31:23.451989+0000","last_unstale":"2024-04-25T10:31:23.459271+0000","last_undegraded":"2024-04-25T10:31:23.459271+0000","last_fullsized":"2024-04-25T10:31:23.459271+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.779040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451553+0000","last_change":"2024-04-25T10:31:23.441950+0000","last_active":"2024-04-25T10:31:23.451553+0000","last_peered":"2024-04-25T10:31:23.451553+0000","last_clean":"2024-04-25T10:31:23.451553+0000","last_became_active":"2024-04-25T10:31:23.439585+0000","last_became_peered":"2024-04-25T10:31:23.439585+0000","last_unstale":"2024-04-25T10:31:23.451553+0000","last_undegraded":"2024-04-25T10:31:23.451553+0000","last_fullsized":"2024-04-25T10:31:23.451553+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:40.982507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458086+0000","last_change":"2024-04-25T10:31:23.452097+0000","last_active":"2024-04-25T10:31:23.458086+0000","last_peered":"2024-04-25T10:31:23.458086+0000","last_clean":"2024-04-25T10:31:23.458086+0000","last_became_active":"2024-04-25T10:31:23.447134+0000","last_became_peered":"2024-04-25T10:31:23.447134+0000","last_unstale":"2024-04-25T10:31:23.458086+0000","last_undegraded":"2024-04-25T10:31:23.458086+0000","last_fullsized":"2024-04-25T10:31:23.458086+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443135+0000","last_change":"2024-04-25T10:31:23.436119+0000","last_active":"2024-04-25T10:31:23.443135+0000","last_peered":"2024-04-25T10:31:23.443135+0000","last_clean":"2024-04-25T10:31:23.443135+0000","last_became_active":"2024-04-25T10:31:23.435858+0000","last_became_peered":"2024-04-25T10:31:23.435858+0000","last_unstale":"2024-04-25T10:31:23.443135+0000","last_undegraded":"2024-04-25T10:31:23.443135+0000","last_fullsized":"2024-04-25T10:31:23.443135+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448609+0000","last_change":"2024-04-25T10:31:23.446258+0000","last_active":"2024-04-25T10:31:23.448609+0000","last_peered":"2024-04-25T10:31:23.448609+0000","last_clean":"2024-04-25T10:31:23.448609+0000","last_became_active":"2024-04-25T10:31:23.445639+0000","last_became_peered":"2024-04-25T10:31:23.445639+0000","last_unstale":"2024-04-25T10:31:23.448609+0000","last_undegraded":"2024-04-25T10:31:23.448609+0000","last_fullsized":"2024-04-25T10:31:23.448609+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.543524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426407+0000","last_change":"2024-04-25T10:31:22.426407+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426407+0000","last_undegraded":"2024-04-25T10:31:22.426407+0000","last_fullsized":"2024-04-25T10:31:22.426407+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.435022+0000","last_change":"2024-04-25T10:31:23.434142+0000","last_active":"2024-04-25T10:31:23.435022+0000","last_peered":"2024-04-25T10:31:23.435022+0000","last_clean":"2024-04-25T10:31:23.435022+0000","last_became_active":"2024-04-25T10:31:23.433096+0000","last_became_peered":"2024-04-25T10:31:23.433096+0000","last_unstale":"2024-04-25T10:31:23.435022+0000","last_undegraded":"2024-04-25T10:31:23.435022+0000","last_fullsized":"2024-04-25T10:31:23.435022+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.452401+0000","last_change":"2024-04-25T10:31:23.445620+0000","last_active":"2024-04-25T10:31:23.452401+0000","last_peered":"2024-04-25T10:31:23.452401+0000","last_clean":"2024-04-25T10:31:23.452401+0000","last_became_active":"2024-04-25T10:31:23.445303+0000","last_became_peered":"2024-04-25T10:31:23.445303+0000","last_unstale":"2024-04-25T10:31:23.452401+0000","last_undegraded":"2024-04-25T10:31:23.452401+0000","last_fullsized":"2024-04-25T10:31:23.452401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:47.724852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.444838+0000","last_change":"2024-04-25T10:31:23.438628+0000","last_active":"2024-04-25T10:31:23.444838+0000","last_peered":"2024-04-25T10:31:23.444838+0000","last_clean":"2024-04-25T10:31:23.444838+0000","last_became_active":"2024-04-25T10:31:23.438301+0000","last_became_peered":"2024-04-25T10:31:23.438301+0000","last_unstale":"2024-04-25T10:31:23.444838+0000","last_undegraded":"2024-04-25T10:31:23.444838+0000","last_fullsized":"2024-04-25T10:31:23.444838+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:31.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448367+0000","last_change":"2024-04-25T10:31:23.444805+0000","last_active":"2024-04-25T10:31:23.448367+0000","last_peered":"2024-04-25T10:31:23.448367+0000","last_clean":"2024-04-25T10:31:23.448367+0000","last_became_active":"2024-04-25T10:31:23.444534+0000","last_became_peered":"2024-04-25T10:31:23.444534+0000","last_unstale":"2024-04-25T10:31:23.448367+0000","last_undegraded":"2024-04-25T10:31:23.448367+0000","last_fullsized":"2024-04-25T10:31:23.448367+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:11.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450950+0000","last_change":"2024-04-25T10:31:23.450197+0000","last_active":"2024-04-25T10:31:23.450950+0000","last_peered":"2024-04-25T10:31:23.450950+0000","last_clean":"2024-04-25T10:31:23.450950+0000","last_became_active":"2024-04-25T10:31:23.449302+0000","last_became_peered":"2024-04-25T10:31:23.449302+0000","last_unstale":"2024-04-25T10:31:23.450950+0000","last_undegraded":"2024-04-25T10:31:23.450950+0000","last_fullsized":"2024-04-25T10:31:23.450950+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:18.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454241+0000","last_change":"2024-04-25T10:31:23.449629+0000","last_active":"2024-04-25T10:31:23.454241+0000","last_peered":"2024-04-25T10:31:23.454241+0000","last_clean":"2024-04-25T10:31:23.454241+0000","last_became_active":"2024-04-25T10:31:23.449364+0000","last_became_peered":"2024-04-25T10:31:23.449364+0000","last_unstale":"2024-04-25T10:31:23.454241+0000","last_undegraded":"2024-04-25T10:31:23.454241+0000","last_fullsized":"2024-04-25T10:31:23.454241+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.444572+0000","last_change":"2024-04-25T10:31:23.438502+0000","last_active":"2024-04-25T10:31:23.444572+0000","last_peered":"2024-04-25T10:31:23.444572+0000","last_clean":"2024-04-25T10:31:23.444572+0000","last_became_active":"2024-04-25T10:31:23.437827+0000","last_became_peered":"2024-04-25T10:31:23.437827+0000","last_unstale":"2024-04-25T10:31:23.444572+0000","last_undegraded":"2024-04-25T10:31:23.444572+0000","last_fullsized":"2024-04-25T10:31:23.444572+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:10.584331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443709+0000","last_change":"2024-04-25T10:31:23.435579+0000","last_active":"2024-04-25T10:31:23.443709+0000","last_peered":"2024-04-25T10:31:23.443709+0000","last_clean":"2024-04-25T10:31:23.443709+0000","last_became_active":"2024-04-25T10:31:23.432705+0000","last_became_peered":"2024-04-25T10:31:23.432705+0000","last_unstale":"2024-04-25T10:31:23.443709+0000","last_undegraded":"2024-04-25T10:31:23.443709+0000","last_fullsized":"2024-04-25T10:31:23.443709+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455317+0000","last_change":"2024-04-25T10:31:23.450098+0000","last_active":"2024-04-25T10:31:23.455317+0000","last_peered":"2024-04-25T10:31:23.455317+0000","last_clean":"2024-04-25T10:31:23.455317+0000","last_became_active":"2024-04-25T10:31:23.449711+0000","last_became_peered":"2024-04-25T10:31:23.449711+0000","last_unstale":"2024-04-25T10:31:23.455317+0000","last_undegraded":"2024-04-25T10:31:23.455317+0000","last_fullsized":"2024-04-25T10:31:23.455317+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:32.246063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460719+0000","last_change":"2024-04-25T10:31:23.459671+0000","last_active":"2024-04-25T10:31:23.460719+0000","last_peered":"2024-04-25T10:31:23.460719+0000","last_clean":"2024-04-25T10:31:23.460719+0000","last_became_active":"2024-04-25T10:31:23.459391+0000","last_became_peered":"2024-04-25T10:31:23.459391+0000","last_unstale":"2024-04-25T10:31:23.460719+0000","last_undegraded":"2024-04-25T10:31:23.460719+0000","last_fullsized":"2024-04-25T10:31:23.460719+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459294+0000","last_change":"2024-04-25T10:31:23.456172+0000","last_active":"2024-04-25T10:31:23.459294+0000","last_peered":"2024-04-25T10:31:23.459294+0000","last_clean":"2024-04-25T10:31:23.459294+0000","last_became_active":"2024-04-25T10:31:23.452838+0000","last_became_peered":"2024-04-25T10:31:23.452838+0000","last_unstale":"2024-04-25T10:31:23.459294+0000","last_undegraded":"2024-04-25T10:31:23.459294+0000","last_fullsized":"2024-04-25T10:31:23.459294+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455909+0000","last_change":"2024-04-25T10:31:23.447420+0000","last_active":"2024-04-25T10:31:23.455909+0000","last_peered":"2024-04-25T10:31:23.455909+0000","last_clean":"2024-04-25T10:31:23.455909+0000","last_became_active":"2024-04-25T10:31:23.447112+0000","last_became_peered":"2024-04-25T10:31:23.447112+0000","last_unstale":"2024-04-25T10:31:23.455909+0000","last_undegraded":"2024-04-25T10:31:23.455909+0000","last_fullsized":"2024-04-25T10:31:23.455909+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451740+0000","last_change":"2024-04-25T10:31:23.450445+0000","last_active":"2024-04-25T10:31:23.451740+0000","last_peered":"2024-04-25T10:31:23.451740+0000","last_clean":"2024-04-25T10:31:23.451740+0000","last_became_active":"2024-04-25T10:31:23.450181+0000","last_became_peered":"2024-04-25T10:31:23.450181+0000","last_unstale":"2024-04-25T10:31:23.451740+0000","last_undegraded":"2024-04-25T10:31:23.451740+0000","last_fullsized":"2024-04-25T10:31:23.451740+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:58.112849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451667+0000","last_change":"2024-04-25T10:31:23.446467+0000","last_active":"2024-04-25T10:31:23.451667+0000","last_peered":"2024-04-25T10:31:23.451667+0000","last_clean":"2024-04-25T10:31:23.451667+0000","last_became_active":"2024-04-25T10:31:23.446185+0000","last_became_peered":"2024-04-25T10:31:23.446185+0000","last_unstale":"2024-04-25T10:31:23.451667+0000","last_undegraded":"2024-04-25T10:31:23.451667+0000","last_fullsized":"2024-04-25T10:31:23.451667+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:36.155132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.441966+0000","last_change":"2024-04-25T10:31:23.432662+0000","last_active":"2024-04-25T10:31:23.441966+0000","last_peered":"2024-04-25T10:31:23.441966+0000","last_clean":"2024-04-25T10:31:23.441966+0000","last_became_active":"2024-04-25T10:31:23.432317+0000","last_became_peered":"2024-04-25T10:31:23.432317+0000","last_unstale":"2024-04-25T10:31:23.441966+0000","last_undegraded":"2024-04-25T10:31:23.441966+0000","last_fullsized":"2024-04-25T10:31:23.441966+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:18.542250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.433442+0000","last_change":"2024-04-25T10:31:23.431527+0000","last_active":"2024-04-25T10:31:23.433442+0000","last_peered":"2024-04-25T10:31:23.433442+0000","last_clean":"2024-04-25T10:31:23.433442+0000","last_became_active":"2024-04-25T10:31:23.431141+0000","last_became_peered":"2024-04-25T10:31:23.431141+0000","last_unstale":"2024-04-25T10:31:23.433442+0000","last_undegraded":"2024-04-25T10:31:23.433442+0000","last_fullsized":"2024-04-25T10:31:23.433442+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450741+0000","last_change":"2024-04-25T10:31:23.441994+0000","last_active":"2024-04-25T10:31:23.450741+0000","last_peered":"2024-04-25T10:31:23.450741+0000","last_clean":"2024-04-25T10:31:23.450741+0000","last_became_active":"2024-04-25T10:31:23.439445+0000","last_became_peered":"2024-04-25T10:31:23.439445+0000","last_unstale":"2024-04-25T10:31:23.450741+0000","last_undegraded":"2024-04-25T10:31:23.450741+0000","last_fullsized":"2024-04-25T10:31:23.450741+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:56.634051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426113+0000","last_change":"2024-04-25T10:31:22.426113+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426113+0000","last_undegraded":"2024-04-25T10:31:22.426113+0000","last_fullsized":"2024-04-25T10:31:22.426113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461073+0000","last_change":"2024-04-25T10:31:23.458263+0000","last_active":"2024-04-25T10:31:23.461073+0000","last_peered":"2024-04-25T10:31:23.461073+0000","last_clean":"2024-04-25T10:31:23.461073+0000","last_became_active":"2024-04-25T10:31:23.457924+0000","last_became_peered":"2024-04-25T10:31:23.457924+0000","last_unstale":"2024-04-25T10:31:23.461073+0000","last_undegraded":"2024-04-25T10:31:23.461073+0000","last_fullsized":"2024-04-25T10:31:23.461073+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.239183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.425919+0000","last_change":"2024-04-25T10:31:22.425919+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.425919+0000","last_undegraded":"2024-04-25T10:31:22.425919+0000","last_fullsized":"2024-04-25T10:31:22.425919+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450551+0000","last_change":"2024-04-25T10:31:23.435570+0000","last_active":"2024-04-25T10:31:23.450551+0000","last_peered":"2024-04-25T10:31:23.450551+0000","last_clean":"2024-04-25T10:31:23.450551+0000","last_became_active":"2024-04-25T10:31:23.435293+0000","last_became_peered":"2024-04-25T10:31:23.435293+0000","last_unstale":"2024-04-25T10:31:23.450551+0000","last_undegraded":"2024-04-25T10:31:23.450551+0000","last_fullsized":"2024-04-25T10:31:23.450551+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:58.849145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.442632+0000","last_change":"2024-04-25T10:31:23.438686+0000","last_active":"2024-04-25T10:31:23.442632+0000","last_peered":"2024-04-25T10:31:23.442632+0000","last_clean":"2024-04-25T10:31:23.442632+0000","last_became_active":"2024-04-25T10:31:23.432709+0000","last_became_peered":"2024-04-25T10:31:23.432709+0000","last_unstale":"2024-04-25T10:31:23.442632+0000","last_undegraded":"2024-04-25T10:31:23.442632+0000","last_fullsized":"2024-04-25T10:31:23.442632+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:01.827369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451828+0000","last_change":"2024-04-25T10:31:23.443235+0000","last_active":"2024-04-25T10:31:23.451828+0000","last_peered":"2024-04-25T10:31:23.451828+0000","last_clean":"2024-04-25T10:31:23.451828+0000","last_became_active":"2024-04-25T10:31:23.442960+0000","last_became_peered":"2024-04-25T10:31:23.442960+0000","last_unstale":"2024-04-25T10:31:23.451828+0000","last_undegraded":"2024-04-25T10:31:23.451828+0000","last_fullsized":"2024-04-25T10:31:23.451828+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:50.981294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.447933+0000","last_change":"2024-04-25T10:31:23.445902+0000","last_active":"2024-04-25T10:31:23.447933+0000","last_peered":"2024-04-25T10:31:23.447933+0000","last_clean":"2024-04-25T10:31:23.447933+0000","last_became_active":"2024-04-25T10:31:23.445617+0000","last_became_peered":"2024-04-25T10:31:23.445617+0000","last_unstale":"2024-04-25T10:31:23.447933+0000","last_undegraded":"2024-04-25T10:31:23.447933+0000","last_fullsized":"2024-04-25T10:31:23.447933+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458804+0000","last_change":"2024-04-25T10:31:23.456990+0000","last_active":"2024-04-25T10:31:23.458804+0000","last_peered":"2024-04-25T10:31:23.458804+0000","last_clean":"2024-04-25T10:31:23.458804+0000","last_became_active":"2024-04-25T10:31:23.451914+0000","last_became_peered":"2024-04-25T10:31:23.451914+0000","last_unstale":"2024-04-25T10:31:23.458804+0000","last_undegraded":"2024-04-25T10:31:23.458804+0000","last_fullsized":"2024-04-25T10:31:23.458804+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:44.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459198+0000","last_change":"2024-04-25T10:31:22.444200+0000","last_active":"2024-04-25T10:31:23.459198+0000","last_peered":"2024-04-25T10:31:23.459198+0000","last_clean":"2024-04-25T10:31:23.459198+0000","last_became_active":"2024-04-25T10:31:22.443896+0000","last_became_peered":"2024-04-25T10:31:22.443896+0000","last_unstale":"2024-04-25T10:31:23.459198+0000","last_undegraded":"2024-04-25T10:31:23.459198+0000","last_fullsized":"2024-04-25T10:31:23.459198+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:07.360121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450721+0000","last_change":"2024-04-25T10:31:23.438605+0000","last_active":"2024-04-25T10:31:23.450721+0000","last_peered":"2024-04-25T10:31:23.450721+0000","last_clean":"2024-04-25T10:31:23.450721+0000","last_became_active":"2024-04-25T10:31:23.437768+0000","last_became_peered":"2024-04-25T10:31:23.437768+0000","last_unstale":"2024-04-25T10:31:23.450721+0000","last_undegraded":"2024-04-25T10:31:23.450721+0000","last_fullsized":"2024-04-25T10:31:23.450721+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.388703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459747+0000","last_change":"2024-04-25T10:31:22.436396+0000","last_active":"2024-04-25T10:31:23.459747+0000","last_peered":"2024-04-25T10:31:23.459747+0000","last_clean":"2024-04-25T10:31:23.459747+0000","last_became_active":"2024-04-25T10:31:22.436121+0000","last_became_peered":"2024-04-25T10:31:22.436121+0000","last_unstale":"2024-04-25T10:31:23.459747+0000","last_undegraded":"2024-04-25T10:31:23.459747+0000","last_fullsized":"2024-04-25T10:31:23.459747+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:51.783124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459463+0000","last_change":"2024-04-25T10:31:23.457159+0000","last_active":"2024-04-25T10:31:23.459463+0000","last_peered":"2024-04-25T10:31:23.459463+0000","last_clean":"2024-04-25T10:31:23.459463+0000","last_became_active":"2024-04-25T10:31:23.456837+0000","last_became_peered":"2024-04-25T10:31:23.456837+0000","last_unstale":"2024-04-25T10:31:23.459463+0000","last_undegraded":"2024-04-25T10:31:23.459463+0000","last_fullsized":"2024-04-25T10:31:23.459463+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:08.821901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443385+0000","last_change":"2024-04-25T10:31:22.439739+0000","last_active":"2024-04-25T10:31:22.443385+0000","last_peered":"2024-04-25T10:31:22.443385+0000","last_clean":"2024-04-25T10:31:22.443385+0000","last_became_active":"2024-04-25T10:31:22.439422+0000","last_became_peered":"2024-04-25T10:31:22.439422+0000","last_unstale":"2024-04-25T10:31:22.443385+0000","last_undegraded":"2024-04-25T10:31:22.443385+0000","last_fullsized":"2024-04-25T10:31:22.443385+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:41.108746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459321+0000","last_change":"2024-04-25T10:31:23.456576+0000","last_active":"2024-04-25T10:31:23.459321+0000","last_peered":"2024-04-25T10:31:23.459321+0000","last_clean":"2024-04-25T10:31:23.459321+0000","last_became_active":"2024-04-25T10:31:23.455891+0000","last_became_peered":"2024-04-25T10:31:23.455891+0000","last_unstale":"2024-04-25T10:31:23.459321+0000","last_undegraded":"2024-04-25T10:31:23.459321+0000","last_fullsized":"2024-04-25T10:31:23.459321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461755+0000","last_change":"2024-04-25T10:31:22.433482+0000","last_active":"2024-04-25T10:31:23.461755+0000","last_peered":"2024-04-25T10:31:23.461755+0000","last_clean":"2024-04-25T10:31:23.461755+0000","last_became_active":"2024-04-25T10:31:22.433201+0000","last_became_peered":"2024-04-25T10:31:22.433201+0000","last_unstale":"2024-04-25T10:31:23.461755+0000","last_undegraded":"2024-04-25T10:31:23.461755+0000","last_fullsized":"2024-04-25T10:31:23.461755+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:10.387788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454868+0000","last_change":"2024-04-25T10:31:23.452485+0000","last_active":"2024-04-25T10:31:23.454868+0000","last_peered":"2024-04-25T10:31:23.454868+0000","last_clean":"2024-04-25T10:31:23.454868+0000","last_became_active":"2024-04-25T10:31:23.452223+0000","last_became_peered":"2024-04-25T10:31:23.452223+0000","last_unstale":"2024-04-25T10:31:23.454868+0000","last_undegraded":"2024-04-25T10:31:23.454868+0000","last_fullsized":"2024-04-25T10:31:23.454868+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:38.344090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461678+0000","last_change":"2024-04-25T10:31:22.445426+0000","last_active":"2024-04-25T10:31:23.461678+0000","last_peered":"2024-04-25T10:31:23.461678+0000","last_clean":"2024-04-25T10:31:23.461678+0000","last_became_active":"2024-04-25T10:31:22.445054+0000","last_became_peered":"2024-04-25T10:31:22.445054+0000","last_unstale":"2024-04-25T10:31:23.461678+0000","last_undegraded":"2024-04-25T10:31:23.461678+0000","last_fullsized":"2024-04-25T10:31:23.461678+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443685+0000","last_change":"2024-04-25T10:31:23.434832+0000","last_active":"2024-04-25T10:31:23.443685+0000","last_peered":"2024-04-25T10:31:23.443685+0000","last_clean":"2024-04-25T10:31:23.443685+0000","last_became_active":"2024-04-25T10:31:23.433275+0000","last_became_peered":"2024-04-25T10:31:23.433275+0000","last_unstale":"2024-04-25T10:31:23.443685+0000","last_undegraded":"2024-04-25T10:31:23.443685+0000","last_fullsized":"2024-04-25T10:31:23.443685+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443196+0000","last_change":"2024-04-25T10:31:22.440188+0000","last_active":"2024-04-25T10:31:22.443196+0000","last_peered":"2024-04-25T10:31:22.443196+0000","last_clean":"2024-04-25T10:31:22.443196+0000","last_became_active":"2024-04-25T10:31:22.438568+0000","last_became_peered":"2024-04-25T10:31:22.438568+0000","last_unstale":"2024-04-25T10:31:22.443196+0000","last_undegraded":"2024-04-25T10:31:22.443196+0000","last_fullsized":"2024-04-25T10:31:22.443196+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.008298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.452463+0000","last_change":"2024-04-25T10:31:23.445257+0000","last_active":"2024-04-25T10:31:23.452463+0000","last_peered":"2024-04-25T10:31:23.452463+0000","last_clean":"2024-04-25T10:31:23.452463+0000","last_became_active":"2024-04-25T10:31:23.444984+0000","last_became_peered":"2024-04-25T10:31:23.444984+0000","last_unstale":"2024-04-25T10:31:23.452463+0000","last_undegraded":"2024-04-25T10:31:23.452463+0000","last_fullsized":"2024-04-25T10:31:23.452463+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:08.999842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426663+0000","last_change":"2024-04-25T10:31:22.443402+0000","last_active":"2024-04-25T10:31:23.426663+0000","last_peered":"2024-04-25T10:31:23.426663+0000","last_clean":"2024-04-25T10:31:23.426663+0000","last_became_active":"2024-04-25T10:31:22.443034+0000","last_became_peered":"2024-04-25T10:31:22.443034+0000","last_unstale":"2024-04-25T10:31:23.426663+0000","last_undegraded":"2024-04-25T10:31:23.426663+0000","last_fullsized":"2024-04-25T10:31:23.426663+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:08.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451254+0000","last_change":"2024-04-25T10:31:23.432502+0000","last_active":"2024-04-25T10:31:23.451254+0000","last_peered":"2024-04-25T10:31:23.451254+0000","last_clean":"2024-04-25T10:31:23.451254+0000","last_became_active":"2024-04-25T10:31:23.430905+0000","last_became_peered":"2024-04-25T10:31:23.430905+0000","last_unstale":"2024-04-25T10:31:23.451254+0000","last_undegraded":"2024-04-25T10:31:23.451254+0000","last_fullsized":"2024-04-25T10:31:23.451254+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:24.065113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.439096+0000","last_change":"2024-04-25T10:31:22.446818+0000","last_active":"2024-04-25T10:31:23.439096+0000","last_peered":"2024-04-25T10:31:23.439096+0000","last_clean":"2024-04-25T10:31:23.439096+0000","last_became_active":"2024-04-25T10:31:22.446198+0000","last_became_peered":"2024-04-25T10:31:22.446198+0000","last_unstale":"2024-04-25T10:31:23.439096+0000","last_undegraded":"2024-04-25T10:31:23.439096+0000","last_fullsized":"2024-04-25T10:31:23.439096+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:27.331081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460947+0000","last_change":"2024-04-25T10:31:23.460404+0000","last_active":"2024-04-25T10:31:23.460947+0000","last_peered":"2024-04-25T10:31:23.460947+0000","last_clean":"2024-04-25T10:31:23.460947+0000","last_became_active":"2024-04-25T10:31:23.459692+0000","last_became_peered":"2024-04-25T10:31:23.459692+0000","last_unstale":"2024-04-25T10:31:23.460947+0000","last_undegraded":"2024-04-25T10:31:23.460947+0000","last_fullsized":"2024-04-25T10:31:23.460947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426137+0000","last_change":"2024-04-25T10:31:22.435228+0000","last_active":"2024-04-25T10:31:23.426137+0000","last_peered":"2024-04-25T10:31:23.426137+0000","last_clean":"2024-04-25T10:31:23.426137+0000","last_became_active":"2024-04-25T10:31:22.434923+0000","last_became_peered":"2024-04-25T10:31:22.434923+0000","last_unstale":"2024-04-25T10:31:23.426137+0000","last_undegraded":"2024-04-25T10:31:23.426137+0000","last_fullsized":"2024-04-25T10:31:23.426137+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:49.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.434391+0000","last_change":"2024-04-25T10:31:23.433349+0000","last_active":"2024-04-25T10:31:23.434391+0000","last_peered":"2024-04-25T10:31:23.434391+0000","last_clean":"2024-04-25T10:31:23.434391+0000","last_became_active":"2024-04-25T10:31:23.433057+0000","last_became_peered":"2024-04-25T10:31:23.433057+0000","last_unstale":"2024-04-25T10:31:23.434391+0000","last_undegraded":"2024-04-25T10:31:23.434391+0000","last_fullsized":"2024-04-25T10:31:23.434391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:08.821901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.444353+0000","last_change":"2024-04-25T10:31:22.442622+0000","last_active":"2024-04-25T10:31:22.444353+0000","last_peered":"2024-04-25T10:31:22.444353+0000","last_clean":"2024-04-25T10:31:22.444353+0000","last_became_active":"2024-04-25T10:31:22.441086+0000","last_became_peered":"2024-04-25T10:31:22.441086+0000","last_unstale":"2024-04-25T10:31:22.444353+0000","last_undegraded":"2024-04-25T10:31:22.444353+0000","last_fullsized":"2024-04-25T10:31:22.444353+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:42.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426939+0000","last_change":"2024-04-25T10:31:22.426939+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426939+0000","last_undegraded":"2024-04-25T10:31:22.426939+0000","last_fullsized":"2024-04-25T10:31:22.426939+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461514+0000","last_change":"2024-04-25T10:31:22.436482+0000","last_active":"2024-04-25T10:31:23.461514+0000","last_peered":"2024-04-25T10:31:23.461514+0000","last_clean":"2024-04-25T10:31:23.461514+0000","last_became_active":"2024-04-25T10:31:22.436192+0000","last_became_peered":"2024-04-25T10:31:22.436192+0000","last_unstale":"2024-04-25T10:31:23.461514+0000","last_undegraded":"2024-04-25T10:31:23.461514+0000","last_fullsized":"2024-04-25T10:31:23.461514+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454216+0000","last_change":"2024-04-25T10:31:23.448277+0000","last_active":"2024-04-25T10:31:23.454216+0000","last_peered":"2024-04-25T10:31:23.454216+0000","last_clean":"2024-04-25T10:31:23.454216+0000","last_became_active":"2024-04-25T10:31:23.447714+0000","last_became_peered":"2024-04-25T10:31:23.447714+0000","last_unstale":"2024-04-25T10:31:23.454216+0000","last_undegraded":"2024-04-25T10:31:23.454216+0000","last_fullsized":"2024-04-25T10:31:23.454216+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456921+0000","last_change":"2024-04-25T10:31:22.437143+0000","last_active":"2024-04-25T10:31:23.456921+0000","last_peered":"2024-04-25T10:31:23.456921+0000","last_clean":"2024-04-25T10:31:23.456921+0000","last_became_active":"2024-04-25T10:31:22.436870+0000","last_became_peered":"2024-04-25T10:31:22.436870+0000","last_unstale":"2024-04-25T10:31:23.456921+0000","last_undegraded":"2024-04-25T10:31:23.456921+0000","last_fullsized":"2024-04-25T10:31:23.456921+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458152+0000","last_change":"2024-04-25T10:31:23.452399+0000","last_active":"2024-04-25T10:31:23.458152+0000","last_peered":"2024-04-25T10:31:23.458152+0000","last_clean":"2024-04-25T10:31:23.458152+0000","last_became_active":"2024-04-25T10:31:23.451792+0000","last_became_peered":"2024-04-25T10:31:23.451792+0000","last_unstale":"2024-04-25T10:31:23.458152+0000","last_undegraded":"2024-04-25T10:31:23.458152+0000","last_fullsized":"2024-04-25T10:31:23.458152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.430252+0000","last_change":"2024-04-25T10:31:22.435264+0000","last_active":"2024-04-25T10:31:23.430252+0000","last_peered":"2024-04-25T10:31:23.430252+0000","last_clean":"2024-04-25T10:31:23.430252+0000","last_became_active":"2024-04-25T10:31:22.434517+0000","last_became_peered":"2024-04-25T10:31:22.434517+0000","last_unstale":"2024-04-25T10:31:23.430252+0000","last_undegraded":"2024-04-25T10:31:23.430252+0000","last_fullsized":"2024-04-25T10:31:23.430252+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458070+0000","last_change":"2024-04-25T10:31:23.452667+0000","last_active":"2024-04-25T10:31:23.458070+0000","last_peered":"2024-04-25T10:31:23.458070+0000","last_clean":"2024-04-25T10:31:23.458070+0000","last_became_active":"2024-04-25T10:31:23.452366+0000","last_became_peered":"2024-04-25T10:31:23.452366+0000","last_unstale":"2024-04-25T10:31:23.458070+0000","last_undegraded":"2024-04-25T10:31:23.458070+0000","last_fullsized":"2024-04-25T10:31:23.458070+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:37.926986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443232+0000","last_change":"2024-04-25T10:31:22.441210+0000","last_active":"2024-04-25T10:31:22.443232+0000","last_peered":"2024-04-25T10:31:22.443232+0000","last_clean":"2024-04-25T10:31:22.443232+0000","last_became_active":"2024-04-25T10:31:22.440936+0000","last_became_peered":"2024-04-25T10:31:22.440936+0000","last_unstale":"2024-04-25T10:31:22.443232+0000","last_undegraded":"2024-04-25T10:31:22.443232+0000","last_fullsized":"2024-04-25T10:31:22.443232+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:54.030915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448629+0000","last_change":"2024-04-25T10:31:23.446637+0000","last_active":"2024-04-25T10:31:23.448629+0000","last_peered":"2024-04-25T10:31:23.448629+0000","last_clean":"2024-04-25T10:31:23.448629+0000","last_became_active":"2024-04-25T10:31:23.445987+0000","last_became_peered":"2024-04-25T10:31:23.445987+0000","last_unstale":"2024-04-25T10:31:23.448629+0000","last_undegraded":"2024-04-25T10:31:23.448629+0000","last_fullsized":"2024-04-25T10:31:23.448629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:28.466968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.439829+0000","last_change":"2024-04-25T10:31:22.436186+0000","last_active":"2024-04-25T10:31:22.439829+0000","last_peered":"2024-04-25T10:31:22.439829+0000","last_clean":"2024-04-25T10:31:22.439829+0000","last_became_active":"2024-04-25T10:31:22.435837+0000","last_became_peered":"2024-04-25T10:31:22.435837+0000","last_unstale":"2024-04-25T10:31:22.439829+0000","last_undegraded":"2024-04-25T10:31:22.439829+0000","last_fullsized":"2024-04-25T10:31:22.439829+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:41.160467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458756+0000","last_change":"2024-04-25T10:31:23.455136+0000","last_active":"2024-04-25T10:31:23.458756+0000","last_peered":"2024-04-25T10:31:23.458756+0000","last_clean":"2024-04-25T10:31:23.458756+0000","last_became_active":"2024-04-25T10:31:23.454870+0000","last_became_peered":"2024-04-25T10:31:23.454870+0000","last_unstale":"2024-04-25T10:31:23.458756+0000","last_undegraded":"2024-04-25T10:31:23.458756+0000","last_fullsized":"2024-04-25T10:31:23.458756+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:39.353583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460494+0000","last_change":"2024-04-25T10:31:22.445257+0000","last_active":"2024-04-25T10:31:23.460494+0000","last_peered":"2024-04-25T10:31:23.460494+0000","last_clean":"2024-04-25T10:31:23.460494+0000","last_became_active":"2024-04-25T10:31:22.444918+0000","last_became_peered":"2024-04-25T10:31:22.444918+0000","last_unstale":"2024-04-25T10:31:23.460494+0000","last_undegraded":"2024-04-25T10:31:23.460494+0000","last_fullsized":"2024-04-25T10:31:23.460494+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455592+0000","last_change":"2024-04-25T10:31:23.453329+0000","last_active":"2024-04-25T10:31:23.455592+0000","last_peered":"2024-04-25T10:31:23.455592+0000","last_clean":"2024-04-25T10:31:23.455592+0000","last_became_active":"2024-04-25T10:31:23.452956+0000","last_became_peered":"2024-04-25T10:31:23.452956+0000","last_unstale":"2024-04-25T10:31:23.455592+0000","last_undegraded":"2024-04-25T10:31:23.455592+0000","last_fullsized":"2024-04-25T10:31:23.455592+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:11.453191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455285+0000","last_change":"2024-04-25T10:31:22.436617+0000","last_active":"2024-04-25T10:31:23.455285+0000","last_peered":"2024-04-25T10:31:23.455285+0000","last_clean":"2024-04-25T10:31:23.455285+0000","last_became_active":"2024-04-25T10:31:22.436313+0000","last_became_peered":"2024-04-25T10:31:22.436313+0000","last_unstale":"2024-04-25T10:31:23.455285+0000","last_undegraded":"2024-04-25T10:31:23.455285+0000","last_fullsized":"2024-04-25T10:31:23.455285+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458066+0000","last_change":"2024-04-25T10:31:23.455754+0000","last_active":"2024-04-25T10:31:23.458066+0000","last_peered":"2024-04-25T10:31:23.458066+0000","last_clean":"2024-04-25T10:31:23.458066+0000","last_became_active":"2024-04-25T10:31:23.447211+0000","last_became_peered":"2024-04-25T10:31:23.447211+0000","last_unstale":"2024-04-25T10:31:23.458066+0000","last_undegraded":"2024-04-25T10:31:23.458066+0000","last_fullsized":"2024-04-25T10:31:23.458066+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.783049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460720+0000","last_change":"2024-04-25T10:31:22.444016+0000","last_active":"2024-04-25T10:31:23.460720+0000","last_peered":"2024-04-25T10:31:23.460720+0000","last_clean":"2024-04-25T10:31:23.460720+0000","last_became_active":"2024-04-25T10:31:22.443711+0000","last_became_peered":"2024-04-25T10:31:22.443711+0000","last_unstale":"2024-04-25T10:31:23.460720+0000","last_undegraded":"2024-04-25T10:31:23.460720+0000","last_fullsized":"2024-04-25T10:31:23.460720+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:28.985863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.440052+0000","last_change":"2024-04-25T10:31:23.434350+0000","last_active":"2024-04-25T10:31:23.440052+0000","last_peered":"2024-04-25T10:31:23.440052+0000","last_clean":"2024-04-25T10:31:23.440052+0000","last_became_active":"2024-04-25T10:31:23.434029+0000","last_became_peered":"2024-04-25T10:31:23.434029+0000","last_unstale":"2024-04-25T10:31:23.440052+0000","last_undegraded":"2024-04-25T10:31:23.440052+0000","last_fullsized":"2024-04-25T10:31:23.440052+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431185+0000","last_change":"2024-04-25T10:31:22.438645+0000","last_active":"2024-04-25T10:31:23.431185+0000","last_peered":"2024-04-25T10:31:23.431185+0000","last_clean":"2024-04-25T10:31:23.431185+0000","last_became_active":"2024-04-25T10:31:22.438312+0000","last_became_peered":"2024-04-25T10:31:22.438312+0000","last_unstale":"2024-04-25T10:31:23.431185+0000","last_undegraded":"2024-04-25T10:31:23.431185+0000","last_fullsized":"2024-04-25T10:31:23.431185+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456346+0000","last_change":"2024-04-25T10:31:23.450857+0000","last_active":"2024-04-25T10:31:23.456346+0000","last_peered":"2024-04-25T10:31:23.456346+0000","last_clean":"2024-04-25T10:31:23.456346+0000","last_became_active":"2024-04-25T10:31:23.449482+0000","last_became_peered":"2024-04-25T10:31:23.449482+0000","last_unstale":"2024-04-25T10:31:23.456346+0000","last_undegraded":"2024-04-25T10:31:23.456346+0000","last_fullsized":"2024-04-25T10:31:23.456346+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460111+0000","last_change":"2024-04-25T10:31:22.439123+0000","last_active":"2024-04-25T10:31:23.460111+0000","last_peered":"2024-04-25T10:31:23.460111+0000","last_clean":"2024-04-25T10:31:23.460111+0000","last_became_active":"2024-04-25T10:31:22.438833+0000","last_became_peered":"2024-04-25T10:31:22.438833+0000","last_unstale":"2024-04-25T10:31:23.460111+0000","last_undegraded":"2024-04-25T10:31:23.460111+0000","last_fullsized":"2024-04-25T10:31:23.460111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.449153+0000","last_change":"2024-04-25T10:31:23.444511+0000","last_active":"2024-04-25T10:31:23.449153+0000","last_peered":"2024-04-25T10:31:23.449153+0000","last_clean":"2024-04-25T10:31:23.449153+0000","last_became_active":"2024-04-25T10:31:23.444242+0000","last_became_peered":"2024-04-25T10:31:23.444242+0000","last_unstale":"2024-04-25T10:31:23.449153+0000","last_undegraded":"2024-04-25T10:31:23.449153+0000","last_fullsized":"2024-04-25T10:31:23.449153+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:38.996519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.442493+0000","last_change":"2024-04-25T10:31:22.438630+0000","last_active":"2024-04-25T10:31:22.442493+0000","last_peered":"2024-04-25T10:31:22.442493+0000","last_clean":"2024-04-25T10:31:22.442493+0000","last_became_active":"2024-04-25T10:31:22.438311+0000","last_became_peered":"2024-04-25T10:31:22.438311+0000","last_unstale":"2024-04-25T10:31:22.442493+0000","last_undegraded":"2024-04-25T10:31:22.442493+0000","last_fullsized":"2024-04-25T10:31:22.442493+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:43.342262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.441051+0000","last_change":"2024-04-25T10:31:23.431648+0000","last_active":"2024-04-25T10:31:23.441051+0000","last_peered":"2024-04-25T10:31:23.441051+0000","last_clean":"2024-04-25T10:31:23.441051+0000","last_became_active":"2024-04-25T10:31:23.430571+0000","last_became_peered":"2024-04-25T10:31:23.430571+0000","last_unstale":"2024-04-25T10:31:23.441051+0000","last_undegraded":"2024-04-25T10:31:23.441051+0000","last_fullsized":"2024-04-25T10:31:23.441051+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:14.895999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.435164+0000","last_change":"2024-04-25T10:31:22.446969+0000","last_active":"2024-04-25T10:31:23.435164+0000","last_peered":"2024-04-25T10:31:23.435164+0000","last_clean":"2024-04-25T10:31:23.435164+0000","last_became_active":"2024-04-25T10:31:22.446445+0000","last_became_peered":"2024-04-25T10:31:22.446445+0000","last_unstale":"2024-04-25T10:31:23.435164+0000","last_undegraded":"2024-04-25T10:31:23.435164+0000","last_fullsized":"2024-04-25T10:31:23.435164+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443904+0000","last_change":"2024-04-25T10:31:23.442351+0000","last_active":"2024-04-25T10:31:23.443904+0000","last_peered":"2024-04-25T10:31:23.443904+0000","last_clean":"2024-04-25T10:31:23.443904+0000","last_became_active":"2024-04-25T10:31:23.442056+0000","last_became_peered":"2024-04-25T10:31:23.442056+0000","last_unstale":"2024-04-25T10:31:23.443904+0000","last_undegraded":"2024-04-25T10:31:23.443904+0000","last_fullsized":"2024-04-25T10:31:23.443904+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:29.781075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.444571+0000","last_change":"2024-04-25T10:31:22.443307+0000","last_active":"2024-04-25T10:31:22.444571+0000","last_peered":"2024-04-25T10:31:22.444571+0000","last_clean":"2024-04-25T10:31:22.444571+0000","last_became_active":"2024-04-25T10:31:22.442920+0000","last_became_peered":"2024-04-25T10:31:22.442920+0000","last_unstale":"2024-04-25T10:31:22.444571+0000","last_undegraded":"2024-04-25T10:31:22.444571+0000","last_fullsized":"2024-04-25T10:31:22.444571+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:38.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451144+0000","last_change":"2024-04-25T10:31:23.444361+0000","last_active":"2024-04-25T10:31:23.451144+0000","last_peered":"2024-04-25T10:31:23.451144+0000","last_clean":"2024-04-25T10:31:23.451144+0000","last_became_active":"2024-04-25T10:31:23.444080+0000","last_became_peered":"2024-04-25T10:31:23.444080+0000","last_unstale":"2024-04-25T10:31:23.451144+0000","last_undegraded":"2024-04-25T10:31:23.451144+0000","last_fullsized":"2024-04-25T10:31:23.451144+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.478405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.441407+0000","last_change":"2024-04-25T10:31:22.436880+0000","last_active":"2024-04-25T10:31:22.441407+0000","last_peered":"2024-04-25T10:31:22.441407+0000","last_clean":"2024-04-25T10:31:22.441407+0000","last_became_active":"2024-04-25T10:31:22.436570+0000","last_became_peered":"2024-04-25T10:31:22.436570+0000","last_unstale":"2024-04-25T10:31:22.441407+0000","last_undegraded":"2024-04-25T10:31:22.441407+0000","last_fullsized":"2024-04-25T10:31:22.441407+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:11.125776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.453381+0000","last_change":"2024-04-25T10:31:23.449985+0000","last_active":"2024-04-25T10:31:23.453381+0000","last_peered":"2024-04-25T10:31:23.453381+0000","last_clean":"2024-04-25T10:31:23.453381+0000","last_became_active":"2024-04-25T10:31:23.449345+0000","last_became_peered":"2024-04-25T10:31:23.449345+0000","last_unstale":"2024-04-25T10:31:23.453381+0000","last_undegraded":"2024-04-25T10:31:23.453381+0000","last_fullsized":"2024-04-25T10:31:23.453381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:28.103406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431065+0000","last_change":"2024-04-25T10:31:22.440950+0000","last_active":"2024-04-25T10:31:23.431065+0000","last_peered":"2024-04-25T10:31:23.431065+0000","last_clean":"2024-04-25T10:31:23.431065+0000","last_became_active":"2024-04-25T10:31:22.440664+0000","last_became_peered":"2024-04-25T10:31:22.440664+0000","last_unstale":"2024-04-25T10:31:23.431065+0000","last_undegraded":"2024-04-25T10:31:23.431065+0000","last_fullsized":"2024-04-25T10:31:23.431065+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:21.897192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456515+0000","last_change":"2024-04-25T10:31:23.448149+0000","last_active":"2024-04-25T10:31:23.456515+0000","last_peered":"2024-04-25T10:31:23.456515+0000","last_clean":"2024-04-25T10:31:23.456515+0000","last_became_active":"2024-04-25T10:31:23.445318+0000","last_became_peered":"2024-04-25T10:31:23.445318+0000","last_unstale":"2024-04-25T10:31:23.456515+0000","last_undegraded":"2024-04-25T10:31:23.456515+0000","last_fullsized":"2024-04-25T10:31:23.456515+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:53.854726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431621+0000","last_change":"2024-04-25T10:31:22.446060+0000","last_active":"2024-04-25T10:31:23.431621+0000","last_peered":"2024-04-25T10:31:23.431621+0000","last_clean":"2024-04-25T10:31:23.431621+0000","last_became_active":"2024-04-25T10:31:22.445706+0000","last_became_peered":"2024-04-25T10:31:22.445706+0000","last_unstale":"2024-04-25T10:31:23.431621+0000","last_undegraded":"2024-04-25T10:31:23.431621+0000","last_fullsized":"2024-04-25T10:31:23.431621+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:31.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458152+0000","last_change":"2024-04-25T10:31:23.453869+0000","last_active":"2024-04-25T10:31:23.458152+0000","last_peered":"2024-04-25T10:31:23.458152+0000","last_clean":"2024-04-25T10:31:23.458152+0000","last_became_active":"2024-04-25T10:31:23.453600+0000","last_became_peered":"2024-04-25T10:31:23.453600+0000","last_unstale":"2024-04-25T10:31:23.458152+0000","last_undegraded":"2024-04-25T10:31:23.458152+0000","last_fullsized":"2024-04-25T10:31:23.458152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:11.359233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431702+0000","last_change":"2024-04-25T10:31:22.445692+0000","last_active":"2024-04-25T10:31:23.431702+0000","last_peered":"2024-04-25T10:31:23.431702+0000","last_clean":"2024-04-25T10:31:23.431702+0000","last_became_active":"2024-04-25T10:31:22.445368+0000","last_became_peered":"2024-04-25T10:31:22.445368+0000","last_unstale":"2024-04-25T10:31:23.431702+0000","last_undegraded":"2024-04-25T10:31:23.431702+0000","last_fullsized":"2024-04-25T10:31:23.431702+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:07.812518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.438304+0000","last_change":"2024-04-25T10:31:23.434539+0000","last_active":"2024-04-25T10:31:23.438304+0000","last_peered":"2024-04-25T10:31:23.438304+0000","last_clean":"2024-04-25T10:31:23.438304+0000","last_became_active":"2024-04-25T10:31:23.434040+0000","last_became_peered":"2024-04-25T10:31:23.434040+0000","last_unstale":"2024-04-25T10:31:23.438304+0000","last_undegraded":"2024-04-25T10:31:23.438304+0000","last_fullsized":"2024-04-25T10:31:23.438304+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:08.999842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.427426+0000","last_change":"2024-04-25T10:31:22.431495+0000","last_active":"2024-04-25T10:31:23.427426+0000","last_peered":"2024-04-25T10:31:23.427426+0000","last_clean":"2024-04-25T10:31:23.427426+0000","last_became_active":"2024-04-25T10:31:22.431217+0000","last_became_peered":"2024-04-25T10:31:22.431217+0000","last_unstale":"2024-04-25T10:31:23.427426+0000","last_undegraded":"2024-04-25T10:31:23.427426+0000","last_fullsized":"2024-04-25T10:31:23.427426+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:27.633982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450575+0000","last_change":"2024-04-25T10:31:23.445933+0000","last_active":"2024-04-25T10:31:23.450575+0000","last_peered":"2024-04-25T10:31:23.450575+0000","last_clean":"2024-04-25T10:31:23.450575+0000","last_became_active":"2024-04-25T10:31:23.445660+0000","last_became_peered":"2024-04-25T10:31:23.445660+0000","last_unstale":"2024-04-25T10:31:23.450575+0000","last_undegraded":"2024-04-25T10:31:23.450575+0000","last_fullsized":"2024-04-25T10:31:23.450575+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:28.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.436951+0000","last_change":"2024-04-25T10:31:22.445594+0000","last_active":"2024-04-25T10:31:23.436951+0000","last_peered":"2024-04-25T10:31:23.436951+0000","last_clean":"2024-04-25T10:31:23.436951+0000","last_became_active":"2024-04-25T10:31:22.445288+0000","last_became_peered":"2024-04-25T10:31:22.445288+0000","last_unstale":"2024-04-25T10:31:23.436951+0000","last_undegraded":"2024-04-25T10:31:23.436951+0000","last_fullsized":"2024-04-25T10:31:23.436951+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:54.229800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.425826+0000","last_change":"2024-04-25T10:31:22.425826+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.425826+0000","last_undegraded":"2024-04-25T10:31:22.425826+0000","last_fullsized":"2024-04-25T10:31:22.425826+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.436066+0000","last_change":"2024-04-25T10:31:22.440617+0000","last_active":"2024-04-25T10:31:23.436066+0000","last_peered":"2024-04-25T10:31:23.436066+0000","last_clean":"2024-04-25T10:31:23.436066+0000","last_became_active":"2024-04-25T10:31:22.440345+0000","last_became_peered":"2024-04-25T10:31:22.440345+0000","last_unstale":"2024-04-25T10:31:23.436066+0000","last_undegraded":"2024-04-25T10:31:23.436066+0000","last_fullsized":"2024-04-25T10:31:23.436066+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:11.911547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443957+0000","last_change":"2024-04-25T10:31:23.443024+0000","last_active":"2024-04-25T10:31:23.443957+0000","last_peered":"2024-04-25T10:31:23.443957+0000","last_clean":"2024-04-25T10:31:23.443957+0000","last_became_active":"2024-04-25T10:31:23.442748+0000","last_became_peered":"2024-04-25T10:31:23.442748+0000","last_unstale":"2024-04-25T10:31:23.443957+0000","last_undegraded":"2024-04-25T10:31:23.443957+0000","last_fullsized":"2024-04-25T10:31:23.443957+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:36.503940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426863+0000","last_change":"2024-04-25T10:31:22.441054+0000","last_active":"2024-04-25T10:31:23.426863+0000","last_peered":"2024-04-25T10:31:23.426863+0000","last_clean":"2024-04-25T10:31:23.426863+0000","last_became_active":"2024-04-25T10:31:22.440642+0000","last_became_peered":"2024-04-25T10:31:22.440642+0000","last_unstale":"2024-04-25T10:31:23.426863+0000","last_undegraded":"2024-04-25T10:31:23.426863+0000","last_fullsized":"2024-04-25T10:31:23.426863+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451102+0000","last_change":"2024-04-25T10:31:23.446822+0000","last_active":"2024-04-25T10:31:23.451102+0000","last_peered":"2024-04-25T10:31:23.451102+0000","last_clean":"2024-04-25T10:31:23.451102+0000","last_became_active":"2024-04-25T10:31:23.446539+0000","last_became_peered":"2024-04-25T10:31:23.446539+0000","last_unstale":"2024-04-25T10:31:23.451102+0000","last_undegraded":"2024-04-25T10:31:23.451102+0000","last_fullsized":"2024-04-25T10:31:23.451102+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.995246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454545+0000","last_change":"2024-04-25T10:31:22.436501+0000","last_active":"2024-04-25T10:31:23.454545+0000","last_peered":"2024-04-25T10:31:23.454545+0000","last_clean":"2024-04-25T10:31:23.454545+0000","last_became_active":"2024-04-25T10:31:22.436167+0000","last_became_peered":"2024-04-25T10:31:22.436167+0000","last_unstale":"2024-04-25T10:31:23.454545+0000","last_undegraded":"2024-04-25T10:31:23.454545+0000","last_fullsized":"2024-04-25T10:31:23.454545+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.310017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":478,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426941+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:23.426941+0000","last_peered":"2024-04-25T10:31:23.426941+0000","last_clean":"2024-04-25T10:31:23.426941+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:23.426941+0000","last_undegraded":"2024-04-25T10:31:23.426941+0000","last_fullsized":"2024-04-25T10:31:23.426941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2277376,"data_stored":998047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55620,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316220,"statfs":{"total":96636764160,"available":96579809280,"internally_reserved":0,"allocated":2277376,"data_stored":998095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60372,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311468,"statfs":{"total":96636764160,"available":96574943232,"internally_reserved":0,"allocated":2883584,"data_stored":1589121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509606,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47284,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324556,"statfs":{"total":96636764160,"available":96588345344,"internally_reserved":0,"allocated":2260992,"data_stored":991250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69456,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302384,"statfs":{"total":96636764160,"available":96565641216,"internally_reserved":0,"allocated":2293760,"data_stored":999741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1588393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509607,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59780,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312060,"statfs":{"total":96636764160,"available":96575549440,"internally_reserved":0,"allocated":2277376,"data_stored":998607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65248,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306592,"statfs":{"total":96636764160,"available":96569950208,"internally_reserved":0,"allocated":2248704,"data_stored":984975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:27.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.110+0000 7f2f21600640 1 Processor -- start 2024-04-25T10:31:28.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f21600640 1 -- start start 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f21600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 0x7f2f1c0cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f21600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f1c06aea0 0x7f2f1c06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f21600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0ccaa0 con 0x7f2f1c0d0180 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f21600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c15e340 con 0x7f2f1c06aea0 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f21600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0ccd80 con 0x7f2f1c15df60 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f13e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2f1c0d0180 0x7f2f1c0d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55698/0 (socket says 172.21.15.45:55698) 2024-04-25T10:31:28.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f13e00640 1 -- 172.21.15.45:0/4253605487 learned_addr learned my addr 172.21.15.45:0/4253605487 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 0x7f2f1c0cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1b400640 1 --2- 172.21.15.45:0/4253605487 >> v2:172.21.15.117:3300/0 conn(0x7f2f1c06aea0 0x7f2f1c06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553491015 0 0) 0x7f2f1c0ccaa0 con 0x7f2f1c0d0180 2024-04-25T10:31:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f00003530 con 0x7f2f1c0d0180 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194940572 0 0) 0x7f2f00003530 con 0x7f2f1c0d0180 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 msgr2=0x7f2f1c0cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 0x7f2f1c0cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 >> v2:172.21.15.117:3300/0 conn(0x7f2f1c06aea0 msgr2=0x7f2f1c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 --2- 172.21.15.45:0/4253605487 >> v2:172.21.15.117:3300/0 conn(0x7f2f1c06aea0 0x7f2f1c06b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0cd060 con 0x7f2f1c0d0180 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f2f04001680 con 0x7f2f1c0d0180 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 >> v1:172.21.15.45:6789/0 conn(0x7f2f1c0d0180 legacy=0x7f2f1c0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000039c0 0x7f2f00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 0x7f2f00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00004d50 0x7f2f000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0ccd80 con 0x7f2f000039c0 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0000b9d0 con 0x7f2f00004d50 2024-04-25T10:31:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0000bb80 con 0x7f2f1c15df60 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000039c0 0x7f2f00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f13e00640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 0x7f2f00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1b400640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00004d50 0x7f2f000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 -- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 msgr2=0x7f2f00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15df60 0x7f2f00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 -- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00004d50 msgr2=0x7f2f000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00004d50 0x7f2f000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1aa00640 1 -- 172.21.15.45:0/4253605487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00003530 con 0x7f2f000039c0 2024-04-25T10:31:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.114+0000 7f2f1b400640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f00004d50 0x7f2f000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000039c0 0x7f2f00003de0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009ae0 tx=0x7f2f0c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3789527252e328dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0096b0 con 0x7f2f000039c0 2024-04-25T10:31:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c0a6450 con 0x7f2f000039c0 2024-04-25T10:31:28.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f1a000640 1 -- 172.21.15.45:0/4253605487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0a6840 con 0x7f2f000039c0 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f21600640 1 -- 172.21.15.45:0/4253605487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000039c0 msgr2=0x7f2f00003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f21600640 1 --2- 172.21.15.45:0/4253605487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f000039c0 0x7f2f00003de0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009ae0 tx=0x7f2f0c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f1b400640 1 -- 172.21.15.45:0/4253605487 reap_dead start 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f21600640 1 -- 172.21.15.45:0/4253605487 shutdown_connections 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f21600640 1 -- 172.21.15.45:0/4253605487 >> 172.21.15.45:0/4253605487 conn(0x7f2f1c0d1910 msgr2=0x7f2f1c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f21600640 1 -- 172.21.15.45:0/4253605487 shutdown_connections 2024-04-25T10:31:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.118+0000 7f2f21600640 1 -- 172.21.15.45:0/4253605487 wait complete. 2024-04-25T10:31:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 Processor -- start 2024-04-25T10:31:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 -- start start 2024-04-25T10:31:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000039c0 0x7f2f1c0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0c6950 0x7f2f1c0b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 0x7f2f1c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0d0ff0 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0d0e70 con 0x7f2f000039c0 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c168a10 con 0x7f2f1c0c6950 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 0x7f2f1c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 0x7f2f1c15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57606/0 (socket says 172.21.15.45:57606) 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 -- 172.21.15.45:0/314083614 learned_addr learned my addr 172.21.15.45:0/314083614 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000039c0 0x7f2f1c0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f13e00640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0c6950 0x7f2f1c0b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 -- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0c6950 msgr2=0x7f2f1c0b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0c6950 0x7f2f1c0b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 -- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000039c0 msgr2=0x7f2f1c0b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000039c0 0x7f2f1c0b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 -- 172.21.15.45:0/314083614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0c6ef0 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000039c0 0x7f2f1c0b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f1b400640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 0x7f2f1c15c9e0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c067740 tx=0x7f2f14010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14010d80 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f140039a0 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14003dd0 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0c71d0 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.122+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c0c7790 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.126+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c04ef30 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.126+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f1401e020 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.126+0000 7f2f12a00640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.126+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f14085790 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.126+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.126+0000 7f2f1aa00640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0ad000 tx=0x7f2f0c0b2040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.130+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f14025020 con 0x7f2f1c0c7f90 2024-04-25T10:31:28.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.290+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f1c06d500 con 0x7f2ef8066f60 2024-04-25T10:31:28.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.298+0000 7f2f12a00640 1 -- 172.21.15.45:0/314083614 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251989 (secure 0 0 0) 0x7f2f1c06d500 con 0x7f2ef8066f60 2024-04-25T10:31:28.302 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:28.304 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.298+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 msgr2=0x7f2ef8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.298+0000 7f2f21600640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0ad000 tx=0x7f2f0c0b2040 comp rx=0 tx=0).stop 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.298+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 msgr2=0x7f2f1c15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.298+0000 7f2f21600640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 0x7f2f1c15c9e0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c067740 tx=0x7f2f14010590 comp rx=0 tx=0).stop 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 shutdown_connections 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c0c7f90 0x7f2f1c15c9e0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0c6950 0x7f2f1c0b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 --2- 172.21.15.45:0/314083614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f000039c0 0x7f2f1c0b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 >> 172.21.15.45:0/314083614 conn(0x7f2f1c0d1910 msgr2=0x7f2f1c0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 shutdown_connections 2024-04-25T10:31:28.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.302+0000 7f2f21600640 1 -- 172.21.15.45:0/314083614 wait complete. 2024-04-25T10:31:28.323 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":585,"stamp":"2024-04-25T10:31:27.064660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592688,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":181,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461240,"kb_used_data":18932,"kb_used_omap":12,"kb_used_meta":442163,"kb_avail":754513480,"statfs":{"total":773094113280,"available":772621803520,"internally_reserved":0,"allocated":19386368,"data_stored":9148229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":452775472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.646564"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.442975+0000","last_change":"2024-04-25T10:31:23.435470+0000","last_active":"2024-04-25T10:31:23.442975+0000","last_peered":"2024-04-25T10:31:23.442975+0000","last_clean":"2024-04-25T10:31:23.442975+0000","last_became_active":"2024-04-25T10:31:23.433761+0000","last_became_peered":"2024-04-25T10:31:23.433761+0000","last_unstale":"2024-04-25T10:31:23.442975+0000","last_undegraded":"2024-04-25T10:31:23.442975+0000","last_fullsized":"2024-04-25T10:31:23.442975+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456702+0000","last_change":"2024-04-25T10:31:23.449323+0000","last_active":"2024-04-25T10:31:23.456702+0000","last_peered":"2024-04-25T10:31:23.456702+0000","last_clean":"2024-04-25T10:31:23.456702+0000","last_became_active":"2024-04-25T10:31:23.448894+0000","last_became_peered":"2024-04-25T10:31:23.448894+0000","last_unstale":"2024-04-25T10:31:23.456702+0000","last_undegraded":"2024-04-25T10:31:23.456702+0000","last_fullsized":"2024-04-25T10:31:23.456702+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:51.236038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459271+0000","last_change":"2024-04-25T10:31:23.455956+0000","last_active":"2024-04-25T10:31:23.459271+0000","last_peered":"2024-04-25T10:31:23.459271+0000","last_clean":"2024-04-25T10:31:23.459271+0000","last_became_active":"2024-04-25T10:31:23.451989+0000","last_became_peered":"2024-04-25T10:31:23.451989+0000","last_unstale":"2024-04-25T10:31:23.459271+0000","last_undegraded":"2024-04-25T10:31:23.459271+0000","last_fullsized":"2024-04-25T10:31:23.459271+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.779040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451553+0000","last_change":"2024-04-25T10:31:23.441950+0000","last_active":"2024-04-25T10:31:23.451553+0000","last_peered":"2024-04-25T10:31:23.451553+0000","last_clean":"2024-04-25T10:31:23.451553+0000","last_became_active":"2024-04-25T10:31:23.439585+0000","last_became_peered":"2024-04-25T10:31:23.439585+0000","last_unstale":"2024-04-25T10:31:23.451553+0000","last_undegraded":"2024-04-25T10:31:23.451553+0000","last_fullsized":"2024-04-25T10:31:23.451553+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:40.982507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458086+0000","last_change":"2024-04-25T10:31:23.452097+0000","last_active":"2024-04-25T10:31:23.458086+0000","last_peered":"2024-04-25T10:31:23.458086+0000","last_clean":"2024-04-25T10:31:23.458086+0000","last_became_active":"2024-04-25T10:31:23.447134+0000","last_became_peered":"2024-04-25T10:31:23.447134+0000","last_unstale":"2024-04-25T10:31:23.458086+0000","last_undegraded":"2024-04-25T10:31:23.458086+0000","last_fullsized":"2024-04-25T10:31:23.458086+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443135+0000","last_change":"2024-04-25T10:31:23.436119+0000","last_active":"2024-04-25T10:31:23.443135+0000","last_peered":"2024-04-25T10:31:23.443135+0000","last_clean":"2024-04-25T10:31:23.443135+0000","last_became_active":"2024-04-25T10:31:23.435858+0000","last_became_peered":"2024-04-25T10:31:23.435858+0000","last_unstale":"2024-04-25T10:31:23.443135+0000","last_undegraded":"2024-04-25T10:31:23.443135+0000","last_fullsized":"2024-04-25T10:31:23.443135+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448609+0000","last_change":"2024-04-25T10:31:23.446258+0000","last_active":"2024-04-25T10:31:23.448609+0000","last_peered":"2024-04-25T10:31:23.448609+0000","last_clean":"2024-04-25T10:31:23.448609+0000","last_became_active":"2024-04-25T10:31:23.445639+0000","last_became_peered":"2024-04-25T10:31:23.445639+0000","last_unstale":"2024-04-25T10:31:23.448609+0000","last_undegraded":"2024-04-25T10:31:23.448609+0000","last_fullsized":"2024-04-25T10:31:23.448609+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.543524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426407+0000","last_change":"2024-04-25T10:31:22.426407+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426407+0000","last_undegraded":"2024-04-25T10:31:22.426407+0000","last_fullsized":"2024-04-25T10:31:22.426407+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.435022+0000","last_change":"2024-04-25T10:31:23.434142+0000","last_active":"2024-04-25T10:31:23.435022+0000","last_peered":"2024-04-25T10:31:23.435022+0000","last_clean":"2024-04-25T10:31:23.435022+0000","last_became_active":"2024-04-25T10:31:23.433096+0000","last_became_peered":"2024-04-25T10:31:23.433096+0000","last_unstale":"2024-04-25T10:31:23.435022+0000","last_undegraded":"2024-04-25T10:31:23.435022+0000","last_fullsized":"2024-04-25T10:31:23.435022+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.452401+0000","last_change":"2024-04-25T10:31:23.445620+0000","last_active":"2024-04-25T10:31:23.452401+0000","last_peered":"2024-04-25T10:31:23.452401+0000","last_clean":"2024-04-25T10:31:23.452401+0000","last_became_active":"2024-04-25T10:31:23.445303+0000","last_became_peered":"2024-04-25T10:31:23.445303+0000","last_unstale":"2024-04-25T10:31:23.452401+0000","last_undegraded":"2024-04-25T10:31:23.452401+0000","last_fullsized":"2024-04-25T10:31:23.452401+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:47.724852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.444838+0000","last_change":"2024-04-25T10:31:23.438628+0000","last_active":"2024-04-25T10:31:23.444838+0000","last_peered":"2024-04-25T10:31:23.444838+0000","last_clean":"2024-04-25T10:31:23.444838+0000","last_became_active":"2024-04-25T10:31:23.438301+0000","last_became_peered":"2024-04-25T10:31:23.438301+0000","last_unstale":"2024-04-25T10:31:23.444838+0000","last_undegraded":"2024-04-25T10:31:23.444838+0000","last_fullsized":"2024-04-25T10:31:23.444838+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:31.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448367+0000","last_change":"2024-04-25T10:31:23.444805+0000","last_active":"2024-04-25T10:31:23.448367+0000","last_peered":"2024-04-25T10:31:23.448367+0000","last_clean":"2024-04-25T10:31:23.448367+0000","last_became_active":"2024-04-25T10:31:23.444534+0000","last_became_peered":"2024-04-25T10:31:23.444534+0000","last_unstale":"2024-04-25T10:31:23.448367+0000","last_undegraded":"2024-04-25T10:31:23.448367+0000","last_fullsized":"2024-04-25T10:31:23.448367+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:11.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450950+0000","last_change":"2024-04-25T10:31:23.450197+0000","last_active":"2024-04-25T10:31:23.450950+0000","last_peered":"2024-04-25T10:31:23.450950+0000","last_clean":"2024-04-25T10:31:23.450950+0000","last_became_active":"2024-04-25T10:31:23.449302+0000","last_became_peered":"2024-04-25T10:31:23.449302+0000","last_unstale":"2024-04-25T10:31:23.450950+0000","last_undegraded":"2024-04-25T10:31:23.450950+0000","last_fullsized":"2024-04-25T10:31:23.450950+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:18.783051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454241+0000","last_change":"2024-04-25T10:31:23.449629+0000","last_active":"2024-04-25T10:31:23.454241+0000","last_peered":"2024-04-25T10:31:23.454241+0000","last_clean":"2024-04-25T10:31:23.454241+0000","last_became_active":"2024-04-25T10:31:23.449364+0000","last_became_peered":"2024-04-25T10:31:23.449364+0000","last_unstale":"2024-04-25T10:31:23.454241+0000","last_undegraded":"2024-04-25T10:31:23.454241+0000","last_fullsized":"2024-04-25T10:31:23.454241+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.444572+0000","last_change":"2024-04-25T10:31:23.438502+0000","last_active":"2024-04-25T10:31:23.444572+0000","last_peered":"2024-04-25T10:31:23.444572+0000","last_clean":"2024-04-25T10:31:23.444572+0000","last_became_active":"2024-04-25T10:31:23.437827+0000","last_became_peered":"2024-04-25T10:31:23.437827+0000","last_unstale":"2024-04-25T10:31:23.444572+0000","last_undegraded":"2024-04-25T10:31:23.444572+0000","last_fullsized":"2024-04-25T10:31:23.444572+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:10.584331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443709+0000","last_change":"2024-04-25T10:31:23.435579+0000","last_active":"2024-04-25T10:31:23.443709+0000","last_peered":"2024-04-25T10:31:23.443709+0000","last_clean":"2024-04-25T10:31:23.443709+0000","last_became_active":"2024-04-25T10:31:23.432705+0000","last_became_peered":"2024-04-25T10:31:23.432705+0000","last_unstale":"2024-04-25T10:31:23.443709+0000","last_undegraded":"2024-04-25T10:31:23.443709+0000","last_fullsized":"2024-04-25T10:31:23.443709+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455317+0000","last_change":"2024-04-25T10:31:23.450098+0000","last_active":"2024-04-25T10:31:23.455317+0000","last_peered":"2024-04-25T10:31:23.455317+0000","last_clean":"2024-04-25T10:31:23.455317+0000","last_became_active":"2024-04-25T10:31:23.449711+0000","last_became_peered":"2024-04-25T10:31:23.449711+0000","last_unstale":"2024-04-25T10:31:23.455317+0000","last_undegraded":"2024-04-25T10:31:23.455317+0000","last_fullsized":"2024-04-25T10:31:23.455317+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:32.246063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460719+0000","last_change":"2024-04-25T10:31:23.459671+0000","last_active":"2024-04-25T10:31:23.460719+0000","last_peered":"2024-04-25T10:31:23.460719+0000","last_clean":"2024-04-25T10:31:23.460719+0000","last_became_active":"2024-04-25T10:31:23.459391+0000","last_became_peered":"2024-04-25T10:31:23.459391+0000","last_unstale":"2024-04-25T10:31:23.460719+0000","last_undegraded":"2024-04-25T10:31:23.460719+0000","last_fullsized":"2024-04-25T10:31:23.460719+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459294+0000","last_change":"2024-04-25T10:31:23.456172+0000","last_active":"2024-04-25T10:31:23.459294+0000","last_peered":"2024-04-25T10:31:23.459294+0000","last_clean":"2024-04-25T10:31:23.459294+0000","last_became_active":"2024-04-25T10:31:23.452838+0000","last_became_peered":"2024-04-25T10:31:23.452838+0000","last_unstale":"2024-04-25T10:31:23.459294+0000","last_undegraded":"2024-04-25T10:31:23.459294+0000","last_fullsized":"2024-04-25T10:31:23.459294+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455909+0000","last_change":"2024-04-25T10:31:23.447420+0000","last_active":"2024-04-25T10:31:23.455909+0000","last_peered":"2024-04-25T10:31:23.455909+0000","last_clean":"2024-04-25T10:31:23.455909+0000","last_became_active":"2024-04-25T10:31:23.447112+0000","last_became_peered":"2024-04-25T10:31:23.447112+0000","last_unstale":"2024-04-25T10:31:23.455909+0000","last_undegraded":"2024-04-25T10:31:23.455909+0000","last_fullsized":"2024-04-25T10:31:23.455909+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451740+0000","last_change":"2024-04-25T10:31:23.450445+0000","last_active":"2024-04-25T10:31:23.451740+0000","last_peered":"2024-04-25T10:31:23.451740+0000","last_clean":"2024-04-25T10:31:23.451740+0000","last_became_active":"2024-04-25T10:31:23.450181+0000","last_became_peered":"2024-04-25T10:31:23.450181+0000","last_unstale":"2024-04-25T10:31:23.451740+0000","last_undegraded":"2024-04-25T10:31:23.451740+0000","last_fullsized":"2024-04-25T10:31:23.451740+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:58.112849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451667+0000","last_change":"2024-04-25T10:31:23.446467+0000","last_active":"2024-04-25T10:31:23.451667+0000","last_peered":"2024-04-25T10:31:23.451667+0000","last_clean":"2024-04-25T10:31:23.451667+0000","last_became_active":"2024-04-25T10:31:23.446185+0000","last_became_peered":"2024-04-25T10:31:23.446185+0000","last_unstale":"2024-04-25T10:31:23.451667+0000","last_undegraded":"2024-04-25T10:31:23.451667+0000","last_fullsized":"2024-04-25T10:31:23.451667+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:36.155132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.441966+0000","last_change":"2024-04-25T10:31:23.432662+0000","last_active":"2024-04-25T10:31:23.441966+0000","last_peered":"2024-04-25T10:31:23.441966+0000","last_clean":"2024-04-25T10:31:23.441966+0000","last_became_active":"2024-04-25T10:31:23.432317+0000","last_became_peered":"2024-04-25T10:31:23.432317+0000","last_unstale":"2024-04-25T10:31:23.441966+0000","last_undegraded":"2024-04-25T10:31:23.441966+0000","last_fullsized":"2024-04-25T10:31:23.441966+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:18.542250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.433442+0000","last_change":"2024-04-25T10:31:23.431527+0000","last_active":"2024-04-25T10:31:23.433442+0000","last_peered":"2024-04-25T10:31:23.433442+0000","last_clean":"2024-04-25T10:31:23.433442+0000","last_became_active":"2024-04-25T10:31:23.431141+0000","last_became_peered":"2024-04-25T10:31:23.431141+0000","last_unstale":"2024-04-25T10:31:23.433442+0000","last_undegraded":"2024-04-25T10:31:23.433442+0000","last_fullsized":"2024-04-25T10:31:23.433442+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450741+0000","last_change":"2024-04-25T10:31:23.441994+0000","last_active":"2024-04-25T10:31:23.450741+0000","last_peered":"2024-04-25T10:31:23.450741+0000","last_clean":"2024-04-25T10:31:23.450741+0000","last_became_active":"2024-04-25T10:31:23.439445+0000","last_became_peered":"2024-04-25T10:31:23.439445+0000","last_unstale":"2024-04-25T10:31:23.450741+0000","last_undegraded":"2024-04-25T10:31:23.450741+0000","last_fullsized":"2024-04-25T10:31:23.450741+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:56.634051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426113+0000","last_change":"2024-04-25T10:31:22.426113+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426113+0000","last_undegraded":"2024-04-25T10:31:22.426113+0000","last_fullsized":"2024-04-25T10:31:22.426113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461073+0000","last_change":"2024-04-25T10:31:23.458263+0000","last_active":"2024-04-25T10:31:23.461073+0000","last_peered":"2024-04-25T10:31:23.461073+0000","last_clean":"2024-04-25T10:31:23.461073+0000","last_became_active":"2024-04-25T10:31:23.457924+0000","last_became_peered":"2024-04-25T10:31:23.457924+0000","last_unstale":"2024-04-25T10:31:23.461073+0000","last_undegraded":"2024-04-25T10:31:23.461073+0000","last_fullsized":"2024-04-25T10:31:23.461073+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:55.239183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.425919+0000","last_change":"2024-04-25T10:31:22.425919+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.425919+0000","last_undegraded":"2024-04-25T10:31:22.425919+0000","last_fullsized":"2024-04-25T10:31:22.425919+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450551+0000","last_change":"2024-04-25T10:31:23.435570+0000","last_active":"2024-04-25T10:31:23.450551+0000","last_peered":"2024-04-25T10:31:23.450551+0000","last_clean":"2024-04-25T10:31:23.450551+0000","last_became_active":"2024-04-25T10:31:23.435293+0000","last_became_peered":"2024-04-25T10:31:23.435293+0000","last_unstale":"2024-04-25T10:31:23.450551+0000","last_undegraded":"2024-04-25T10:31:23.450551+0000","last_fullsized":"2024-04-25T10:31:23.450551+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:58.849145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.442632+0000","last_change":"2024-04-25T10:31:23.438686+0000","last_active":"2024-04-25T10:31:23.442632+0000","last_peered":"2024-04-25T10:31:23.442632+0000","last_clean":"2024-04-25T10:31:23.442632+0000","last_became_active":"2024-04-25T10:31:23.432709+0000","last_became_peered":"2024-04-25T10:31:23.432709+0000","last_unstale":"2024-04-25T10:31:23.442632+0000","last_undegraded":"2024-04-25T10:31:23.442632+0000","last_fullsized":"2024-04-25T10:31:23.442632+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:01.827369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451828+0000","last_change":"2024-04-25T10:31:23.443235+0000","last_active":"2024-04-25T10:31:23.451828+0000","last_peered":"2024-04-25T10:31:23.451828+0000","last_clean":"2024-04-25T10:31:23.451828+0000","last_became_active":"2024-04-25T10:31:23.442960+0000","last_became_peered":"2024-04-25T10:31:23.442960+0000","last_unstale":"2024-04-25T10:31:23.451828+0000","last_undegraded":"2024-04-25T10:31:23.451828+0000","last_fullsized":"2024-04-25T10:31:23.451828+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:50.981294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.447933+0000","last_change":"2024-04-25T10:31:23.445902+0000","last_active":"2024-04-25T10:31:23.447933+0000","last_peered":"2024-04-25T10:31:23.447933+0000","last_clean":"2024-04-25T10:31:23.447933+0000","last_became_active":"2024-04-25T10:31:23.445617+0000","last_became_peered":"2024-04-25T10:31:23.445617+0000","last_unstale":"2024-04-25T10:31:23.447933+0000","last_undegraded":"2024-04-25T10:31:23.447933+0000","last_fullsized":"2024-04-25T10:31:23.447933+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458804+0000","last_change":"2024-04-25T10:31:23.456990+0000","last_active":"2024-04-25T10:31:23.458804+0000","last_peered":"2024-04-25T10:31:23.458804+0000","last_clean":"2024-04-25T10:31:23.458804+0000","last_became_active":"2024-04-25T10:31:23.451914+0000","last_became_peered":"2024-04-25T10:31:23.451914+0000","last_unstale":"2024-04-25T10:31:23.458804+0000","last_undegraded":"2024-04-25T10:31:23.458804+0000","last_fullsized":"2024-04-25T10:31:23.458804+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:44.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459198+0000","last_change":"2024-04-25T10:31:22.444200+0000","last_active":"2024-04-25T10:31:23.459198+0000","last_peered":"2024-04-25T10:31:23.459198+0000","last_clean":"2024-04-25T10:31:23.459198+0000","last_became_active":"2024-04-25T10:31:22.443896+0000","last_became_peered":"2024-04-25T10:31:22.443896+0000","last_unstale":"2024-04-25T10:31:23.459198+0000","last_undegraded":"2024-04-25T10:31:23.459198+0000","last_fullsized":"2024-04-25T10:31:23.459198+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:07.360121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450721+0000","last_change":"2024-04-25T10:31:23.438605+0000","last_active":"2024-04-25T10:31:23.450721+0000","last_peered":"2024-04-25T10:31:23.450721+0000","last_clean":"2024-04-25T10:31:23.450721+0000","last_became_active":"2024-04-25T10:31:23.437768+0000","last_became_peered":"2024-04-25T10:31:23.437768+0000","last_unstale":"2024-04-25T10:31:23.450721+0000","last_undegraded":"2024-04-25T10:31:23.450721+0000","last_fullsized":"2024-04-25T10:31:23.450721+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.388703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459747+0000","last_change":"2024-04-25T10:31:22.436396+0000","last_active":"2024-04-25T10:31:23.459747+0000","last_peered":"2024-04-25T10:31:23.459747+0000","last_clean":"2024-04-25T10:31:23.459747+0000","last_became_active":"2024-04-25T10:31:22.436121+0000","last_became_peered":"2024-04-25T10:31:22.436121+0000","last_unstale":"2024-04-25T10:31:23.459747+0000","last_undegraded":"2024-04-25T10:31:23.459747+0000","last_fullsized":"2024-04-25T10:31:23.459747+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:51.783124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459463+0000","last_change":"2024-04-25T10:31:23.457159+0000","last_active":"2024-04-25T10:31:23.459463+0000","last_peered":"2024-04-25T10:31:23.459463+0000","last_clean":"2024-04-25T10:31:23.459463+0000","last_became_active":"2024-04-25T10:31:23.456837+0000","last_became_peered":"2024-04-25T10:31:23.456837+0000","last_unstale":"2024-04-25T10:31:23.459463+0000","last_undegraded":"2024-04-25T10:31:23.459463+0000","last_fullsized":"2024-04-25T10:31:23.459463+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:08.821901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443385+0000","last_change":"2024-04-25T10:31:22.439739+0000","last_active":"2024-04-25T10:31:22.443385+0000","last_peered":"2024-04-25T10:31:22.443385+0000","last_clean":"2024-04-25T10:31:22.443385+0000","last_became_active":"2024-04-25T10:31:22.439422+0000","last_became_peered":"2024-04-25T10:31:22.439422+0000","last_unstale":"2024-04-25T10:31:22.443385+0000","last_undegraded":"2024-04-25T10:31:22.443385+0000","last_fullsized":"2024-04-25T10:31:22.443385+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:41.108746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.459321+0000","last_change":"2024-04-25T10:31:23.456576+0000","last_active":"2024-04-25T10:31:23.459321+0000","last_peered":"2024-04-25T10:31:23.459321+0000","last_clean":"2024-04-25T10:31:23.459321+0000","last_became_active":"2024-04-25T10:31:23.455891+0000","last_became_peered":"2024-04-25T10:31:23.455891+0000","last_unstale":"2024-04-25T10:31:23.459321+0000","last_undegraded":"2024-04-25T10:31:23.459321+0000","last_fullsized":"2024-04-25T10:31:23.459321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461755+0000","last_change":"2024-04-25T10:31:22.433482+0000","last_active":"2024-04-25T10:31:23.461755+0000","last_peered":"2024-04-25T10:31:23.461755+0000","last_clean":"2024-04-25T10:31:23.461755+0000","last_became_active":"2024-04-25T10:31:22.433201+0000","last_became_peered":"2024-04-25T10:31:22.433201+0000","last_unstale":"2024-04-25T10:31:23.461755+0000","last_undegraded":"2024-04-25T10:31:23.461755+0000","last_fullsized":"2024-04-25T10:31:23.461755+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:10.387788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454868+0000","last_change":"2024-04-25T10:31:23.452485+0000","last_active":"2024-04-25T10:31:23.454868+0000","last_peered":"2024-04-25T10:31:23.454868+0000","last_clean":"2024-04-25T10:31:23.454868+0000","last_became_active":"2024-04-25T10:31:23.452223+0000","last_became_peered":"2024-04-25T10:31:23.452223+0000","last_unstale":"2024-04-25T10:31:23.454868+0000","last_undegraded":"2024-04-25T10:31:23.454868+0000","last_fullsized":"2024-04-25T10:31:23.454868+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:38.344090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461678+0000","last_change":"2024-04-25T10:31:22.445426+0000","last_active":"2024-04-25T10:31:23.461678+0000","last_peered":"2024-04-25T10:31:23.461678+0000","last_clean":"2024-04-25T10:31:23.461678+0000","last_became_active":"2024-04-25T10:31:22.445054+0000","last_became_peered":"2024-04-25T10:31:22.445054+0000","last_unstale":"2024-04-25T10:31:23.461678+0000","last_undegraded":"2024-04-25T10:31:23.461678+0000","last_fullsized":"2024-04-25T10:31:23.461678+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443685+0000","last_change":"2024-04-25T10:31:23.434832+0000","last_active":"2024-04-25T10:31:23.443685+0000","last_peered":"2024-04-25T10:31:23.443685+0000","last_clean":"2024-04-25T10:31:23.443685+0000","last_became_active":"2024-04-25T10:31:23.433275+0000","last_became_peered":"2024-04-25T10:31:23.433275+0000","last_unstale":"2024-04-25T10:31:23.443685+0000","last_undegraded":"2024-04-25T10:31:23.443685+0000","last_fullsized":"2024-04-25T10:31:23.443685+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443196+0000","last_change":"2024-04-25T10:31:22.440188+0000","last_active":"2024-04-25T10:31:22.443196+0000","last_peered":"2024-04-25T10:31:22.443196+0000","last_clean":"2024-04-25T10:31:22.443196+0000","last_became_active":"2024-04-25T10:31:22.438568+0000","last_became_peered":"2024-04-25T10:31:22.438568+0000","last_unstale":"2024-04-25T10:31:22.443196+0000","last_undegraded":"2024-04-25T10:31:22.443196+0000","last_fullsized":"2024-04-25T10:31:22.443196+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:03.008298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.452463+0000","last_change":"2024-04-25T10:31:23.445257+0000","last_active":"2024-04-25T10:31:23.452463+0000","last_peered":"2024-04-25T10:31:23.452463+0000","last_clean":"2024-04-25T10:31:23.452463+0000","last_became_active":"2024-04-25T10:31:23.444984+0000","last_became_peered":"2024-04-25T10:31:23.444984+0000","last_unstale":"2024-04-25T10:31:23.452463+0000","last_undegraded":"2024-04-25T10:31:23.452463+0000","last_fullsized":"2024-04-25T10:31:23.452463+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:08.999842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426663+0000","last_change":"2024-04-25T10:31:22.443402+0000","last_active":"2024-04-25T10:31:23.426663+0000","last_peered":"2024-04-25T10:31:23.426663+0000","last_clean":"2024-04-25T10:31:23.426663+0000","last_became_active":"2024-04-25T10:31:22.443034+0000","last_became_peered":"2024-04-25T10:31:22.443034+0000","last_unstale":"2024-04-25T10:31:23.426663+0000","last_undegraded":"2024-04-25T10:31:23.426663+0000","last_fullsized":"2024-04-25T10:31:23.426663+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:08.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451254+0000","last_change":"2024-04-25T10:31:23.432502+0000","last_active":"2024-04-25T10:31:23.451254+0000","last_peered":"2024-04-25T10:31:23.451254+0000","last_clean":"2024-04-25T10:31:23.451254+0000","last_became_active":"2024-04-25T10:31:23.430905+0000","last_became_peered":"2024-04-25T10:31:23.430905+0000","last_unstale":"2024-04-25T10:31:23.451254+0000","last_undegraded":"2024-04-25T10:31:23.451254+0000","last_fullsized":"2024-04-25T10:31:23.451254+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:24.065113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.439096+0000","last_change":"2024-04-25T10:31:22.446818+0000","last_active":"2024-04-25T10:31:23.439096+0000","last_peered":"2024-04-25T10:31:23.439096+0000","last_clean":"2024-04-25T10:31:23.439096+0000","last_became_active":"2024-04-25T10:31:22.446198+0000","last_became_peered":"2024-04-25T10:31:22.446198+0000","last_unstale":"2024-04-25T10:31:23.439096+0000","last_undegraded":"2024-04-25T10:31:23.439096+0000","last_fullsized":"2024-04-25T10:31:23.439096+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:27.331081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460947+0000","last_change":"2024-04-25T10:31:23.460404+0000","last_active":"2024-04-25T10:31:23.460947+0000","last_peered":"2024-04-25T10:31:23.460947+0000","last_clean":"2024-04-25T10:31:23.460947+0000","last_became_active":"2024-04-25T10:31:23.459692+0000","last_became_peered":"2024-04-25T10:31:23.459692+0000","last_unstale":"2024-04-25T10:31:23.460947+0000","last_undegraded":"2024-04-25T10:31:23.460947+0000","last_fullsized":"2024-04-25T10:31:23.460947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426137+0000","last_change":"2024-04-25T10:31:22.435228+0000","last_active":"2024-04-25T10:31:23.426137+0000","last_peered":"2024-04-25T10:31:23.426137+0000","last_clean":"2024-04-25T10:31:23.426137+0000","last_became_active":"2024-04-25T10:31:22.434923+0000","last_became_peered":"2024-04-25T10:31:22.434923+0000","last_unstale":"2024-04-25T10:31:23.426137+0000","last_undegraded":"2024-04-25T10:31:23.426137+0000","last_fullsized":"2024-04-25T10:31:23.426137+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:49.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.434391+0000","last_change":"2024-04-25T10:31:23.433349+0000","last_active":"2024-04-25T10:31:23.434391+0000","last_peered":"2024-04-25T10:31:23.434391+0000","last_clean":"2024-04-25T10:31:23.434391+0000","last_became_active":"2024-04-25T10:31:23.433057+0000","last_became_peered":"2024-04-25T10:31:23.433057+0000","last_unstale":"2024-04-25T10:31:23.434391+0000","last_undegraded":"2024-04-25T10:31:23.434391+0000","last_fullsized":"2024-04-25T10:31:23.434391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:08.821901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.444353+0000","last_change":"2024-04-25T10:31:22.442622+0000","last_active":"2024-04-25T10:31:22.444353+0000","last_peered":"2024-04-25T10:31:22.444353+0000","last_clean":"2024-04-25T10:31:22.444353+0000","last_became_active":"2024-04-25T10:31:22.441086+0000","last_became_peered":"2024-04-25T10:31:22.441086+0000","last_unstale":"2024-04-25T10:31:22.444353+0000","last_undegraded":"2024-04-25T10:31:22.444353+0000","last_fullsized":"2024-04-25T10:31:22.444353+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:42.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.426939+0000","last_change":"2024-04-25T10:31:22.426939+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.426939+0000","last_undegraded":"2024-04-25T10:31:22.426939+0000","last_fullsized":"2024-04-25T10:31:22.426939+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.461514+0000","last_change":"2024-04-25T10:31:22.436482+0000","last_active":"2024-04-25T10:31:23.461514+0000","last_peered":"2024-04-25T10:31:23.461514+0000","last_clean":"2024-04-25T10:31:23.461514+0000","last_became_active":"2024-04-25T10:31:22.436192+0000","last_became_peered":"2024-04-25T10:31:22.436192+0000","last_unstale":"2024-04-25T10:31:23.461514+0000","last_undegraded":"2024-04-25T10:31:23.461514+0000","last_fullsized":"2024-04-25T10:31:23.461514+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454216+0000","last_change":"2024-04-25T10:31:23.448277+0000","last_active":"2024-04-25T10:31:23.454216+0000","last_peered":"2024-04-25T10:31:23.454216+0000","last_clean":"2024-04-25T10:31:23.454216+0000","last_became_active":"2024-04-25T10:31:23.447714+0000","last_became_peered":"2024-04-25T10:31:23.447714+0000","last_unstale":"2024-04-25T10:31:23.454216+0000","last_undegraded":"2024-04-25T10:31:23.454216+0000","last_fullsized":"2024-04-25T10:31:23.454216+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456921+0000","last_change":"2024-04-25T10:31:22.437143+0000","last_active":"2024-04-25T10:31:23.456921+0000","last_peered":"2024-04-25T10:31:23.456921+0000","last_clean":"2024-04-25T10:31:23.456921+0000","last_became_active":"2024-04-25T10:31:22.436870+0000","last_became_peered":"2024-04-25T10:31:22.436870+0000","last_unstale":"2024-04-25T10:31:23.456921+0000","last_undegraded":"2024-04-25T10:31:23.456921+0000","last_fullsized":"2024-04-25T10:31:23.456921+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458152+0000","last_change":"2024-04-25T10:31:23.452399+0000","last_active":"2024-04-25T10:31:23.458152+0000","last_peered":"2024-04-25T10:31:23.458152+0000","last_clean":"2024-04-25T10:31:23.458152+0000","last_became_active":"2024-04-25T10:31:23.451792+0000","last_became_peered":"2024-04-25T10:31:23.451792+0000","last_unstale":"2024-04-25T10:31:23.458152+0000","last_undegraded":"2024-04-25T10:31:23.458152+0000","last_fullsized":"2024-04-25T10:31:23.458152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.430252+0000","last_change":"2024-04-25T10:31:22.435264+0000","last_active":"2024-04-25T10:31:23.430252+0000","last_peered":"2024-04-25T10:31:23.430252+0000","last_clean":"2024-04-25T10:31:23.430252+0000","last_became_active":"2024-04-25T10:31:22.434517+0000","last_became_peered":"2024-04-25T10:31:22.434517+0000","last_unstale":"2024-04-25T10:31:23.430252+0000","last_undegraded":"2024-04-25T10:31:23.430252+0000","last_fullsized":"2024-04-25T10:31:23.430252+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458070+0000","last_change":"2024-04-25T10:31:23.452667+0000","last_active":"2024-04-25T10:31:23.458070+0000","last_peered":"2024-04-25T10:31:23.458070+0000","last_clean":"2024-04-25T10:31:23.458070+0000","last_became_active":"2024-04-25T10:31:23.452366+0000","last_became_peered":"2024-04-25T10:31:23.452366+0000","last_unstale":"2024-04-25T10:31:23.458070+0000","last_undegraded":"2024-04-25T10:31:23.458070+0000","last_fullsized":"2024-04-25T10:31:23.458070+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:37.926986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.443232+0000","last_change":"2024-04-25T10:31:22.441210+0000","last_active":"2024-04-25T10:31:22.443232+0000","last_peered":"2024-04-25T10:31:22.443232+0000","last_clean":"2024-04-25T10:31:22.443232+0000","last_became_active":"2024-04-25T10:31:22.440936+0000","last_became_peered":"2024-04-25T10:31:22.440936+0000","last_unstale":"2024-04-25T10:31:22.443232+0000","last_undegraded":"2024-04-25T10:31:22.443232+0000","last_fullsized":"2024-04-25T10:31:22.443232+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:54.030915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.448629+0000","last_change":"2024-04-25T10:31:23.446637+0000","last_active":"2024-04-25T10:31:23.448629+0000","last_peered":"2024-04-25T10:31:23.448629+0000","last_clean":"2024-04-25T10:31:23.448629+0000","last_became_active":"2024-04-25T10:31:23.445987+0000","last_became_peered":"2024-04-25T10:31:23.445987+0000","last_unstale":"2024-04-25T10:31:23.448629+0000","last_undegraded":"2024-04-25T10:31:23.448629+0000","last_fullsized":"2024-04-25T10:31:23.448629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:28.466968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.439829+0000","last_change":"2024-04-25T10:31:22.436186+0000","last_active":"2024-04-25T10:31:22.439829+0000","last_peered":"2024-04-25T10:31:22.439829+0000","last_clean":"2024-04-25T10:31:22.439829+0000","last_became_active":"2024-04-25T10:31:22.435837+0000","last_became_peered":"2024-04-25T10:31:22.435837+0000","last_unstale":"2024-04-25T10:31:22.439829+0000","last_undegraded":"2024-04-25T10:31:22.439829+0000","last_fullsized":"2024-04-25T10:31:22.439829+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:41.160467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458756+0000","last_change":"2024-04-25T10:31:23.455136+0000","last_active":"2024-04-25T10:31:23.458756+0000","last_peered":"2024-04-25T10:31:23.458756+0000","last_clean":"2024-04-25T10:31:23.458756+0000","last_became_active":"2024-04-25T10:31:23.454870+0000","last_became_peered":"2024-04-25T10:31:23.454870+0000","last_unstale":"2024-04-25T10:31:23.458756+0000","last_undegraded":"2024-04-25T10:31:23.458756+0000","last_fullsized":"2024-04-25T10:31:23.458756+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:39.353583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460494+0000","last_change":"2024-04-25T10:31:22.445257+0000","last_active":"2024-04-25T10:31:23.460494+0000","last_peered":"2024-04-25T10:31:23.460494+0000","last_clean":"2024-04-25T10:31:23.460494+0000","last_became_active":"2024-04-25T10:31:22.444918+0000","last_became_peered":"2024-04-25T10:31:22.444918+0000","last_unstale":"2024-04-25T10:31:23.460494+0000","last_undegraded":"2024-04-25T10:31:23.460494+0000","last_fullsized":"2024-04-25T10:31:23.460494+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455592+0000","last_change":"2024-04-25T10:31:23.453329+0000","last_active":"2024-04-25T10:31:23.455592+0000","last_peered":"2024-04-25T10:31:23.455592+0000","last_clean":"2024-04-25T10:31:23.455592+0000","last_became_active":"2024-04-25T10:31:23.452956+0000","last_became_peered":"2024-04-25T10:31:23.452956+0000","last_unstale":"2024-04-25T10:31:23.455592+0000","last_undegraded":"2024-04-25T10:31:23.455592+0000","last_fullsized":"2024-04-25T10:31:23.455592+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:11.453191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.455285+0000","last_change":"2024-04-25T10:31:22.436617+0000","last_active":"2024-04-25T10:31:23.455285+0000","last_peered":"2024-04-25T10:31:23.455285+0000","last_clean":"2024-04-25T10:31:23.455285+0000","last_became_active":"2024-04-25T10:31:22.436313+0000","last_became_peered":"2024-04-25T10:31:22.436313+0000","last_unstale":"2024-04-25T10:31:23.455285+0000","last_undegraded":"2024-04-25T10:31:23.455285+0000","last_fullsized":"2024-04-25T10:31:23.455285+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458066+0000","last_change":"2024-04-25T10:31:23.455754+0000","last_active":"2024-04-25T10:31:23.458066+0000","last_peered":"2024-04-25T10:31:23.458066+0000","last_clean":"2024-04-25T10:31:23.458066+0000","last_became_active":"2024-04-25T10:31:23.447211+0000","last_became_peered":"2024-04-25T10:31:23.447211+0000","last_unstale":"2024-04-25T10:31:23.458066+0000","last_undegraded":"2024-04-25T10:31:23.458066+0000","last_fullsized":"2024-04-25T10:31:23.458066+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.783049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460720+0000","last_change":"2024-04-25T10:31:22.444016+0000","last_active":"2024-04-25T10:31:23.460720+0000","last_peered":"2024-04-25T10:31:23.460720+0000","last_clean":"2024-04-25T10:31:23.460720+0000","last_became_active":"2024-04-25T10:31:22.443711+0000","last_became_peered":"2024-04-25T10:31:22.443711+0000","last_unstale":"2024-04-25T10:31:23.460720+0000","last_undegraded":"2024-04-25T10:31:23.460720+0000","last_fullsized":"2024-04-25T10:31:23.460720+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:28.985863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.440052+0000","last_change":"2024-04-25T10:31:23.434350+0000","last_active":"2024-04-25T10:31:23.440052+0000","last_peered":"2024-04-25T10:31:23.440052+0000","last_clean":"2024-04-25T10:31:23.440052+0000","last_became_active":"2024-04-25T10:31:23.434029+0000","last_became_peered":"2024-04-25T10:31:23.434029+0000","last_unstale":"2024-04-25T10:31:23.440052+0000","last_undegraded":"2024-04-25T10:31:23.440052+0000","last_fullsized":"2024-04-25T10:31:23.440052+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:39.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431185+0000","last_change":"2024-04-25T10:31:22.438645+0000","last_active":"2024-04-25T10:31:23.431185+0000","last_peered":"2024-04-25T10:31:23.431185+0000","last_clean":"2024-04-25T10:31:23.431185+0000","last_became_active":"2024-04-25T10:31:22.438312+0000","last_became_peered":"2024-04-25T10:31:22.438312+0000","last_unstale":"2024-04-25T10:31:23.431185+0000","last_undegraded":"2024-04-25T10:31:23.431185+0000","last_fullsized":"2024-04-25T10:31:23.431185+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456346+0000","last_change":"2024-04-25T10:31:23.450857+0000","last_active":"2024-04-25T10:31:23.456346+0000","last_peered":"2024-04-25T10:31:23.456346+0000","last_clean":"2024-04-25T10:31:23.456346+0000","last_became_active":"2024-04-25T10:31:23.449482+0000","last_became_peered":"2024-04-25T10:31:23.449482+0000","last_unstale":"2024-04-25T10:31:23.456346+0000","last_undegraded":"2024-04-25T10:31:23.456346+0000","last_fullsized":"2024-04-25T10:31:23.456346+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.460111+0000","last_change":"2024-04-25T10:31:22.439123+0000","last_active":"2024-04-25T10:31:23.460111+0000","last_peered":"2024-04-25T10:31:23.460111+0000","last_clean":"2024-04-25T10:31:23.460111+0000","last_became_active":"2024-04-25T10:31:22.438833+0000","last_became_peered":"2024-04-25T10:31:22.438833+0000","last_unstale":"2024-04-25T10:31:23.460111+0000","last_undegraded":"2024-04-25T10:31:23.460111+0000","last_fullsized":"2024-04-25T10:31:23.460111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.449153+0000","last_change":"2024-04-25T10:31:23.444511+0000","last_active":"2024-04-25T10:31:23.449153+0000","last_peered":"2024-04-25T10:31:23.449153+0000","last_clean":"2024-04-25T10:31:23.449153+0000","last_became_active":"2024-04-25T10:31:23.444242+0000","last_became_peered":"2024-04-25T10:31:23.444242+0000","last_unstale":"2024-04-25T10:31:23.449153+0000","last_undegraded":"2024-04-25T10:31:23.449153+0000","last_fullsized":"2024-04-25T10:31:23.449153+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:38.996519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.442493+0000","last_change":"2024-04-25T10:31:22.438630+0000","last_active":"2024-04-25T10:31:22.442493+0000","last_peered":"2024-04-25T10:31:22.442493+0000","last_clean":"2024-04-25T10:31:22.442493+0000","last_became_active":"2024-04-25T10:31:22.438311+0000","last_became_peered":"2024-04-25T10:31:22.438311+0000","last_unstale":"2024-04-25T10:31:22.442493+0000","last_undegraded":"2024-04-25T10:31:22.442493+0000","last_fullsized":"2024-04-25T10:31:22.442493+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:43.342262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.441051+0000","last_change":"2024-04-25T10:31:23.431648+0000","last_active":"2024-04-25T10:31:23.441051+0000","last_peered":"2024-04-25T10:31:23.441051+0000","last_clean":"2024-04-25T10:31:23.441051+0000","last_became_active":"2024-04-25T10:31:23.430571+0000","last_became_peered":"2024-04-25T10:31:23.430571+0000","last_unstale":"2024-04-25T10:31:23.441051+0000","last_undegraded":"2024-04-25T10:31:23.441051+0000","last_fullsized":"2024-04-25T10:31:23.441051+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:14.895999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.435164+0000","last_change":"2024-04-25T10:31:22.446969+0000","last_active":"2024-04-25T10:31:23.435164+0000","last_peered":"2024-04-25T10:31:23.435164+0000","last_clean":"2024-04-25T10:31:23.435164+0000","last_became_active":"2024-04-25T10:31:22.446445+0000","last_became_peered":"2024-04-25T10:31:22.446445+0000","last_unstale":"2024-04-25T10:31:23.435164+0000","last_undegraded":"2024-04-25T10:31:23.435164+0000","last_fullsized":"2024-04-25T10:31:23.435164+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443904+0000","last_change":"2024-04-25T10:31:23.442351+0000","last_active":"2024-04-25T10:31:23.443904+0000","last_peered":"2024-04-25T10:31:23.443904+0000","last_clean":"2024-04-25T10:31:23.443904+0000","last_became_active":"2024-04-25T10:31:23.442056+0000","last_became_peered":"2024-04-25T10:31:23.442056+0000","last_unstale":"2024-04-25T10:31:23.443904+0000","last_undegraded":"2024-04-25T10:31:23.443904+0000","last_fullsized":"2024-04-25T10:31:23.443904+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:29.781075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.444571+0000","last_change":"2024-04-25T10:31:22.443307+0000","last_active":"2024-04-25T10:31:22.444571+0000","last_peered":"2024-04-25T10:31:22.444571+0000","last_clean":"2024-04-25T10:31:22.444571+0000","last_became_active":"2024-04-25T10:31:22.442920+0000","last_became_peered":"2024-04-25T10:31:22.442920+0000","last_unstale":"2024-04-25T10:31:22.444571+0000","last_undegraded":"2024-04-25T10:31:22.444571+0000","last_fullsized":"2024-04-25T10:31:22.444571+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:38.846475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451144+0000","last_change":"2024-04-25T10:31:23.444361+0000","last_active":"2024-04-25T10:31:23.451144+0000","last_peered":"2024-04-25T10:31:23.451144+0000","last_clean":"2024-04-25T10:31:23.451144+0000","last_became_active":"2024-04-25T10:31:23.444080+0000","last_became_peered":"2024-04-25T10:31:23.444080+0000","last_unstale":"2024-04-25T10:31:23.451144+0000","last_undegraded":"2024-04-25T10:31:23.451144+0000","last_fullsized":"2024-04-25T10:31:23.451144+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.478405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":180,"state":"active+clean","last_fresh":"2024-04-25T10:31:22.441407+0000","last_change":"2024-04-25T10:31:22.436880+0000","last_active":"2024-04-25T10:31:22.441407+0000","last_peered":"2024-04-25T10:31:22.441407+0000","last_clean":"2024-04-25T10:31:22.441407+0000","last_became_active":"2024-04-25T10:31:22.436570+0000","last_became_peered":"2024-04-25T10:31:22.436570+0000","last_unstale":"2024-04-25T10:31:22.441407+0000","last_undegraded":"2024-04-25T10:31:22.441407+0000","last_fullsized":"2024-04-25T10:31:22.441407+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:11.125776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.453381+0000","last_change":"2024-04-25T10:31:23.449985+0000","last_active":"2024-04-25T10:31:23.453381+0000","last_peered":"2024-04-25T10:31:23.453381+0000","last_clean":"2024-04-25T10:31:23.453381+0000","last_became_active":"2024-04-25T10:31:23.449345+0000","last_became_peered":"2024-04-25T10:31:23.449345+0000","last_unstale":"2024-04-25T10:31:23.453381+0000","last_undegraded":"2024-04-25T10:31:23.453381+0000","last_fullsized":"2024-04-25T10:31:23.453381+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:28.103406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431065+0000","last_change":"2024-04-25T10:31:22.440950+0000","last_active":"2024-04-25T10:31:23.431065+0000","last_peered":"2024-04-25T10:31:23.431065+0000","last_clean":"2024-04-25T10:31:23.431065+0000","last_became_active":"2024-04-25T10:31:22.440664+0000","last_became_peered":"2024-04-25T10:31:22.440664+0000","last_unstale":"2024-04-25T10:31:23.431065+0000","last_undegraded":"2024-04-25T10:31:23.431065+0000","last_fullsized":"2024-04-25T10:31:23.431065+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:21.897192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.456515+0000","last_change":"2024-04-25T10:31:23.448149+0000","last_active":"2024-04-25T10:31:23.456515+0000","last_peered":"2024-04-25T10:31:23.456515+0000","last_clean":"2024-04-25T10:31:23.456515+0000","last_became_active":"2024-04-25T10:31:23.445318+0000","last_became_peered":"2024-04-25T10:31:23.445318+0000","last_unstale":"2024-04-25T10:31:23.456515+0000","last_undegraded":"2024-04-25T10:31:23.456515+0000","last_fullsized":"2024-04-25T10:31:23.456515+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:53.854726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431621+0000","last_change":"2024-04-25T10:31:22.446060+0000","last_active":"2024-04-25T10:31:23.431621+0000","last_peered":"2024-04-25T10:31:23.431621+0000","last_clean":"2024-04-25T10:31:23.431621+0000","last_became_active":"2024-04-25T10:31:22.445706+0000","last_became_peered":"2024-04-25T10:31:22.445706+0000","last_unstale":"2024-04-25T10:31:23.431621+0000","last_undegraded":"2024-04-25T10:31:23.431621+0000","last_fullsized":"2024-04-25T10:31:23.431621+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:31.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.458152+0000","last_change":"2024-04-25T10:31:23.453869+0000","last_active":"2024-04-25T10:31:23.458152+0000","last_peered":"2024-04-25T10:31:23.458152+0000","last_clean":"2024-04-25T10:31:23.458152+0000","last_became_active":"2024-04-25T10:31:23.453600+0000","last_became_peered":"2024-04-25T10:31:23.453600+0000","last_unstale":"2024-04-25T10:31:23.458152+0000","last_undegraded":"2024-04-25T10:31:23.458152+0000","last_fullsized":"2024-04-25T10:31:23.458152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:11.359233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.431702+0000","last_change":"2024-04-25T10:31:22.445692+0000","last_active":"2024-04-25T10:31:23.431702+0000","last_peered":"2024-04-25T10:31:23.431702+0000","last_clean":"2024-04-25T10:31:23.431702+0000","last_became_active":"2024-04-25T10:31:22.445368+0000","last_became_peered":"2024-04-25T10:31:22.445368+0000","last_unstale":"2024-04-25T10:31:23.431702+0000","last_undegraded":"2024-04-25T10:31:23.431702+0000","last_fullsized":"2024-04-25T10:31:23.431702+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:07.812518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.438304+0000","last_change":"2024-04-25T10:31:23.434539+0000","last_active":"2024-04-25T10:31:23.438304+0000","last_peered":"2024-04-25T10:31:23.438304+0000","last_clean":"2024-04-25T10:31:23.438304+0000","last_became_active":"2024-04-25T10:31:23.434040+0000","last_became_peered":"2024-04-25T10:31:23.434040+0000","last_unstale":"2024-04-25T10:31:23.438304+0000","last_undegraded":"2024-04-25T10:31:23.438304+0000","last_fullsized":"2024-04-25T10:31:23.438304+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:08.999842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.427426+0000","last_change":"2024-04-25T10:31:22.431495+0000","last_active":"2024-04-25T10:31:23.427426+0000","last_peered":"2024-04-25T10:31:23.427426+0000","last_clean":"2024-04-25T10:31:23.427426+0000","last_became_active":"2024-04-25T10:31:22.431217+0000","last_became_peered":"2024-04-25T10:31:22.431217+0000","last_unstale":"2024-04-25T10:31:23.427426+0000","last_undegraded":"2024-04-25T10:31:23.427426+0000","last_fullsized":"2024-04-25T10:31:23.427426+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:27.633982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.450575+0000","last_change":"2024-04-25T10:31:23.445933+0000","last_active":"2024-04-25T10:31:23.450575+0000","last_peered":"2024-04-25T10:31:23.450575+0000","last_clean":"2024-04-25T10:31:23.450575+0000","last_became_active":"2024-04-25T10:31:23.445660+0000","last_became_peered":"2024-04-25T10:31:23.445660+0000","last_unstale":"2024-04-25T10:31:23.450575+0000","last_undegraded":"2024-04-25T10:31:23.450575+0000","last_fullsized":"2024-04-25T10:31:23.450575+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:28.599670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.436951+0000","last_change":"2024-04-25T10:31:22.445594+0000","last_active":"2024-04-25T10:31:23.436951+0000","last_peered":"2024-04-25T10:31:23.436951+0000","last_clean":"2024-04-25T10:31:23.436951+0000","last_became_active":"2024-04-25T10:31:22.445288+0000","last_became_peered":"2024-04-25T10:31:22.445288+0000","last_unstale":"2024-04-25T10:31:23.436951+0000","last_undegraded":"2024-04-25T10:31:23.436951+0000","last_fullsized":"2024-04-25T10:31:23.436951+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:54.229800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":180,"state":"creating+peering","last_fresh":"2024-04-25T10:31:22.425826+0000","last_change":"2024-04-25T10:31:22.425826+0000","last_active":"2024-04-25T10:31:22.410310+0000","last_peered":"2024-04-25T10:31:22.410310+0000","last_clean":"2024-04-25T10:31:22.410310+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:31:22.425826+0000","last_undegraded":"2024-04-25T10:31:22.425826+0000","last_fullsized":"2024-04-25T10:31:22.425826+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.436066+0000","last_change":"2024-04-25T10:31:22.440617+0000","last_active":"2024-04-25T10:31:23.436066+0000","last_peered":"2024-04-25T10:31:23.436066+0000","last_clean":"2024-04-25T10:31:23.436066+0000","last_became_active":"2024-04-25T10:31:22.440345+0000","last_became_peered":"2024-04-25T10:31:22.440345+0000","last_unstale":"2024-04-25T10:31:23.436066+0000","last_undegraded":"2024-04-25T10:31:23.436066+0000","last_fullsized":"2024-04-25T10:31:23.436066+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:11.911547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.443957+0000","last_change":"2024-04-25T10:31:23.443024+0000","last_active":"2024-04-25T10:31:23.443957+0000","last_peered":"2024-04-25T10:31:23.443957+0000","last_clean":"2024-04-25T10:31:23.443957+0000","last_became_active":"2024-04-25T10:31:23.442748+0000","last_became_peered":"2024-04-25T10:31:23.442748+0000","last_unstale":"2024-04-25T10:31:23.443957+0000","last_undegraded":"2024-04-25T10:31:23.443957+0000","last_fullsized":"2024-04-25T10:31:23.443957+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:36.503940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426863+0000","last_change":"2024-04-25T10:31:22.441054+0000","last_active":"2024-04-25T10:31:23.426863+0000","last_peered":"2024-04-25T10:31:23.426863+0000","last_clean":"2024-04-25T10:31:23.426863+0000","last_became_active":"2024-04-25T10:31:22.440642+0000","last_became_peered":"2024-04-25T10:31:22.440642+0000","last_unstale":"2024-04-25T10:31:23.426863+0000","last_undegraded":"2024-04-25T10:31:23.426863+0000","last_fullsized":"2024-04-25T10:31:23.426863+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.451102+0000","last_change":"2024-04-25T10:31:23.446822+0000","last_active":"2024-04-25T10:31:23.451102+0000","last_peered":"2024-04-25T10:31:23.451102+0000","last_clean":"2024-04-25T10:31:23.451102+0000","last_became_active":"2024-04-25T10:31:23.446539+0000","last_became_peered":"2024-04-25T10:31:23.446539+0000","last_unstale":"2024-04-25T10:31:23.451102+0000","last_undegraded":"2024-04-25T10:31:23.451102+0000","last_fullsized":"2024-04-25T10:31:23.451102+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.995246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.454545+0000","last_change":"2024-04-25T10:31:22.436501+0000","last_active":"2024-04-25T10:31:23.454545+0000","last_peered":"2024-04-25T10:31:23.454545+0000","last_clean":"2024-04-25T10:31:23.454545+0000","last_became_active":"2024-04-25T10:31:22.436167+0000","last_became_peered":"2024-04-25T10:31:22.436167+0000","last_unstale":"2024-04-25T10:31:23.454545+0000","last_undegraded":"2024-04-25T10:31:23.454545+0000","last_fullsized":"2024-04-25T10:31:23.454545+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.310017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":478,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-04-25T10:31:23.426941+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:23.426941+0000","last_peered":"2024-04-25T10:31:23.426941+0000","last_clean":"2024-04-25T10:31:23.426941+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:23.426941+0000","last_undegraded":"2024-04-25T10:31:23.426941+0000","last_fullsized":"2024-04-25T10:31:23.426941+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2277376,"data_stored":998047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55620,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316220,"statfs":{"total":96636764160,"available":96579809280,"internally_reserved":0,"allocated":2277376,"data_stored":998095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509606,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60372,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311468,"statfs":{"total":96636764160,"available":96574943232,"internally_reserved":0,"allocated":2883584,"data_stored":1589121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509606,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47284,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324556,"statfs":{"total":96636764160,"available":96588345344,"internally_reserved":0,"allocated":2260992,"data_stored":991250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509606,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69456,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302384,"statfs":{"total":96636764160,"available":96565641216,"internally_reserved":0,"allocated":2293760,"data_stored":999741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2867200,"data_stored":1588393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509607,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59780,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312060,"statfs":{"total":96636764160,"available":96575549440,"internally_reserved":0,"allocated":2277376,"data_stored":998607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65248,"kb_used_data":2196,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306592,"statfs":{"total":96636764160,"available":96569950208,"internally_reserved":0,"allocated":2248704,"data_stored":984975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.38 is not active+clean 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 180, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:31:22.426407+0000', 'last_change': '2024-04-25T10:31:22.426407+0000', 'last_active': '2024-04-25T10:31:22.410310+0000', 'last_peered': '2024-04-25T10:31:22.410310+0000', 'last_clean': '2024-04-25T10:31:22.410310+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:31:22.426407+0000', 'last_undegraded': '2024-04-25T10:31:22.426407+0000', 'last_fullsized': '2024-04-25T10:31:22.426407+0000', 'mapping_epoch': 180, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 180, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_clean_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.26 is not active+clean 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 180, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:31:22.426113+0000', 'last_change': '2024-04-25T10:31:22.426113+0000', 'last_active': '2024-04-25T10:31:22.410310+0000', 'last_peered': '2024-04-25T10:31:22.410310+0000', 'last_clean': '2024-04-25T10:31:22.410310+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:31:22.426113+0000', 'last_undegraded': '2024-04-25T10:31:22.426113+0000', 'last_fullsized': '2024-04-25T10:31:22.426113+0000', 'mapping_epoch': 180, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 180, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_clean_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.24 is not active+clean 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 180, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:31:22.425919+0000', 'last_change': '2024-04-25T10:31:22.425919+0000', 'last_active': '2024-04-25T10:31:22.410310+0000', 'last_peered': '2024-04-25T10:31:22.410310+0000', 'last_clean': '2024-04-25T10:31:22.410310+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:31:22.425919+0000', 'last_undegraded': '2024-04-25T10:31:22.425919+0000', 'last_fullsized': '2024-04-25T10:31:22.425919+0000', 'mapping_epoch': 180, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 180, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_clean_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:31:28.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.15 is not active+clean 2024-04-25T10:31:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 180, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:31:22.426939+0000', 'last_change': '2024-04-25T10:31:22.426939+0000', 'last_active': '2024-04-25T10:31:22.410310+0000', 'last_peered': '2024-04-25T10:31:22.410310+0000', 'last_clean': '2024-04-25T10:31:22.410310+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:31:22.426939+0000', 'last_undegraded': '2024-04-25T10:31:22.426939+0000', 'last_fullsized': '2024-04-25T10:31:22.426939+0000', 'mapping_epoch': 180, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 180, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_clean_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:31:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.0 is not active+clean 2024-04-25T10:31:28.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 180, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:31:22.425826+0000', 'last_change': '2024-04-25T10:31:22.425826+0000', 'last_active': '2024-04-25T10:31:22.410310+0000', 'last_peered': '2024-04-25T10:31:22.410310+0000', 'last_clean': '2024-04-25T10:31:22.410310+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:31:22.425826+0000', 'last_undegraded': '2024-04-25T10:31:22.425826+0000', 'last_fullsized': '2024-04-25T10:31:22.425826+0000', 'mapping_epoch': 180, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 180, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'last_clean_scrub_stamp': '2024-04-25T10:31:22.410310+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:31:28.328 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:31:28.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 Processor -- start 2024-04-25T10:31:28.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 -- start start 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f780415b9e0 0x7f78040cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f780415bdc0 0x7f78040cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f78040d1520 0x7f78040d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78040ccfb0 con 0x7f780415bdc0 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78040cd290 con 0x7f78040d1520 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f780a600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78040cd570 con 0x7f780415b9e0 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f780415bdc0 0x7f78040cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f780415bdc0 0x7f78040cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57614/0 (socket says 172.21.15.45:57614) 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 -- 172.21.15.45:0/1297764222 learned_addr learned my addr 172.21.15.45:0/1297764222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f7803400640 1 --2- 172.21.15.45:0/1297764222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f780415b9e0 0x7f78040cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f7803e00640 1 --2- 172.21.15.45:0/1297764222 >> v2:172.21.15.117:3300/0 conn(0x7f78040d1520 0x7f78040d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 -- 172.21.15.45:0/1297764222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f780415b9e0 msgr2=0x7f78040cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 --2- 172.21.15.45:0/1297764222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f780415b9e0 0x7f78040cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 -- 172.21.15.45:0/1297764222 >> v2:172.21.15.117:3300/0 conn(0x7f78040d1520 msgr2=0x7f78040d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 --2- 172.21.15.45:0/1297764222 >> v2:172.21.15.117:3300/0 conn(0x7f78040d1520 0x7f78040d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.510+0000 7f77faa00640 1 -- 172.21.15.45:0/1297764222 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78040cd850 con 0x7f780415bdc0 2024-04-25T10:31:28.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f77faa00640 1 --2- 172.21.15.45:0/1297764222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415bdc0 0x7f78040cc950 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f78040b3180 tx=0x7f77ec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311f018e10c2d2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f7802a00640 1 -- 172.21.15.45:0/1297764222 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0b1070 con 0x7f780415bdc0 2024-04-25T10:31:28.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f7802a00640 1 -- 172.21.15.45:0/1297764222 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec0ac630 con 0x7f780415bdc0 2024-04-25T10:31:28.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f7802a00640 1 -- 172.21.15.45:0/1297764222 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0a8050 con 0x7f780415bdc0 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 -- 172.21.15.45:0/1297764222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415bdc0 msgr2=0x7f78040cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 --2- 172.21.15.45:0/1297764222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415bdc0 0x7f78040cc950 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f78040b3180 tx=0x7f77ec01c520 comp rx=0 tx=0).stop 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 -- 172.21.15.45:0/1297764222 shutdown_connections 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 --2- 172.21.15.45:0/1297764222 >> v2:172.21.15.117:3300/0 conn(0x7f78040d1520 0x7f78040d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 --2- 172.21.15.45:0/1297764222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415bdc0 0x7f78040cc950 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 --2- 172.21.15.45:0/1297764222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f780415b9e0 0x7f78040cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 -- 172.21.15.45:0/1297764222 >> 172.21.15.45:0/1297764222 conn(0x7f780409a9d0 msgr2=0x7f780409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 -- 172.21.15.45:0/1297764222 shutdown_connections 2024-04-25T10:31:28.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.514+0000 7f780a600640 1 -- 172.21.15.45:0/1297764222 wait complete. 2024-04-25T10:31:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 Processor -- start 2024-04-25T10:31:28.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- start start 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78040d1520 0x7f78040b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 0x7f7804150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f780415bdc0 0x7f78040b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78040cebd0 con 0x7f780415b9e0 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78040cea50 con 0x7f780415bdc0 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78040ced50 con 0x7f78040d1520 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 0x7f7804150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 0x7f7804150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57618/0 (socket says 172.21.15.45:57618) 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 -- 172.21.15.45:0/1259039574 learned_addr learned my addr 172.21.15.45:0/1259039574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:28.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f7803e00640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f780415bdc0 0x7f78040b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f7803400640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78040d1520 0x7f78040b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 -- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78040d1520 msgr2=0x7f78040b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78040d1520 0x7f78040b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 -- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f780415bdc0 msgr2=0x7f78040b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f780415bdc0 0x7f78040b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 -- 172.21.15.45:0/1259039574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78040b4d50 con 0x7f780415b9e0 2024-04-25T10:31:28.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77faa00640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 0x7f7804150f80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f77ec01abd0 tx=0x7f77ec0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0bc070 con 0x7f780415b9e0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec0ab5e0 con 0x7f780415b9e0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0b1070 con 0x7f780415b9e0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78040b5030 con 0x7f780415b9e0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78040b55f0 con 0x7f780415b9e0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.518+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780404ef30 con 0x7f780415b9e0 2024-04-25T10:31:28.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.522+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77ec0ab9d0 con 0x7f780415b9e0 2024-04-25T10:31:28.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.522+0000 7f77fbe00640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77e0066f60 0x7f77e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.522+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77ec11eb20 con 0x7f780415b9e0 2024-04-25T10:31:28.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.522+0000 7f7803400640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77e0066f60 0x7f77e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.522+0000 7f7803400640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77e0066f60 0x7f77e0069420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f78040b2930 tx=0x7f77f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.526+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77ec016110 con 0x7f780415b9e0 2024-04-25T10:31:28.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.742+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f78040b2db0 con 0x7f780415b9e0 2024-04-25T10:31:28.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f77fbe00640 1 -- 172.21.15.45:0/1259039574 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1524 (secure 0 0 0) 0x7f77ec0efa00 con 0x7f780415b9e0 2024-04-25T10:31:28.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:28.747 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"d7e12072-0435-4dda-862b-d1045153027e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":832,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":182,"num_osds":8,"num_up_osds":8,"osd_up_since":1714040268,"num_in_osds":8,"osd_in_since":1714040259,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":15,"data_bytes":592688,"bytes_used":472309760,"bytes_avail":772621803520,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2357,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":131,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"c","status":"up:active","gid":10881}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:30:13.046596+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:31:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77e0066f60 msgr2=0x7f77e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77e0066f60 0x7f77e0069420 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f78040b2930 tx=0x7f77f4008040 comp rx=0 tx=0).stop 2024-04-25T10:31:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 msgr2=0x7f7804150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 0x7f7804150f80 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f77ec01abd0 tx=0x7f77ec0192c0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 shutdown_connections 2024-04-25T10:31:28.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f77e0066f60 0x7f77e0069420 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f780415bdc0 0x7f78040b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780415b9e0 0x7f7804150f80 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 --2- 172.21.15.45:0/1259039574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78040d1520 0x7f78040b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 >> 172.21.15.45:0/1259039574 conn(0x7f780409a9d0 msgr2=0x7f78040c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 shutdown_connections 2024-04-25T10:31:28.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.746+0000 7f780a600640 1 -- 172.21.15.45:0/1259039574 wait complete. 2024-04-25T10:31:28.769 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:31:28.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:28.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.906+0000 7f3775000640 1 Processor -- start 2024-04-25T10:31:28.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.906+0000 7f3775000640 1 -- start start 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f3775000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f377015dce0 0x7f37700669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f3775000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37700ad850 0x7f37700adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f3775000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377006e6a0 con 0x7f377006eec0 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f3775000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37700674d0 con 0x7f37700ad850 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f3775000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377015e0c0 con 0x7f377015dce0 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37700ad850 0x7f37700adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f37700ad850 0x7f37700adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57624/0 (socket says 172.21.15.45:57624) 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 -- 172.21.15.45:0/280605544 learned_addr learned my addr 172.21.15.45:0/280605544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376de00640 1 --2- 172.21.15.45:0/280605544 >> v2:172.21.15.117:3300/0 conn(0x7f377015dce0 0x7f37700669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 -- 172.21.15.45:0/280605544 >> v2:172.21.15.117:3300/0 conn(0x7f377015dce0 msgr2=0x7f37700669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 --2- 172.21.15.45:0/280605544 >> v2:172.21.15.117:3300/0 conn(0x7f377015dce0 0x7f37700669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 -- 172.21.15.45:0/280605544 >> v1:172.21.15.117:6789/0 conn(0x7f377006eec0 legacy=0x7f377006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.910+0000 7f376e800640 1 -- 172.21.15.45:0/280605544 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37700671f0 con 0x7f37700ad850 2024-04-25T10:31:28.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376e800640 1 --2- 172.21.15.45:0/280605544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37700ad850 0x7f37700adcb0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f3770074d80 tx=0x7f376801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3f7b30fa667f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376d400640 1 -- 172.21.15.45:0/280605544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680b0070 con 0x7f37700ad850 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376d400640 1 -- 172.21.15.45:0/280605544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37680ab600 con 0x7f37700ad850 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376d400640 1 -- 172.21.15.45:0/280605544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680ab9d0 con 0x7f37700ad850 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- 172.21.15.45:0/280605544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37700ad850 msgr2=0x7f37700adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 --2- 172.21.15.45:0/280605544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37700ad850 0x7f37700adcb0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f3770074d80 tx=0x7f376801c400 comp rx=0 tx=0).stop 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- 172.21.15.45:0/280605544 shutdown_connections 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 --2- 172.21.15.45:0/280605544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37700ad850 0x7f37700adcb0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 --2- 172.21.15.45:0/280605544 >> v2:172.21.15.117:3300/0 conn(0x7f377015dce0 0x7f37700669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- 172.21.15.45:0/280605544 >> 172.21.15.45:0/280605544 conn(0x7f37700b2410 msgr2=0x7f37700b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- 172.21.15.45:0/280605544 shutdown_connections 2024-04-25T10:31:28.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- 172.21.15.45:0/280605544 wait complete. 2024-04-25T10:31:28.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 Processor -- start 2024-04-25T10:31:28.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- start start 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f377006eec0 0x7f3770151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37700ad850 0x7f3770150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 0x7f37701573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700d0320 con 0x7f377015dce0 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770066ee0 con 0x7f37700ad850 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3775000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37700d04a0 con 0x7f377006eec0 2024-04-25T10:31:28.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 0x7f37701573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 0x7f37701573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57636/0 (socket says 172.21.15.45:57636) 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f377006eec0 0x7f3770151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f376e800640 1 -- 172.21.15.45:0/1992390987 learned_addr learned my addr 172.21.15.45:0/1992390987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.914+0000 7f3765400640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37700ad850 0x7f3770150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376e800640 1 -- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f377006eec0 msgr2=0x7f3770151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376e800640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f377006eec0 0x7f3770151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376e800640 1 -- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37700ad850 msgr2=0x7f3770150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376e800640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37700ad850 0x7f3770150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376e800640 1 -- 172.21.15.45:0/1992390987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37701522e0 con 0x7f377015dce0 2024-04-25T10:31:28.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376e800640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 0x7f37701573d0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f37680ae040 tx=0x7f37680191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680b0040 con 0x7f377015dce0 2024-04-25T10:31:28.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37680b9070 con 0x7f377015dce0 2024-04-25T10:31:28.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3770152750 con 0x7f377015dce0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680b4640 con 0x7f377015dce0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37700c4cc0 con 0x7f377015dce0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377004ef30 con 0x7f377015dce0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37680b47e0 con 0x7f377015dce0 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3766a00640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f374c066f90 0x7f374c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:28.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f376811e8c0 con 0x7f377015dce0 2024-04-25T10:31:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.918+0000 7f376de00640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f374c066f90 0x7f374c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:28.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.922+0000 7f376de00640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f374c066f90 0x7f374c069450 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f377015f9f0 tx=0x7f376000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:28.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:28.922+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37701522e0 con 0x7f377015dce0 2024-04-25T10:31:29.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.082+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f377006eca0 con 0x7f374c066f90 2024-04-25T10:31:29.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.090+0000 7f3766a00640 1 -- 172.21.15.45:0/1992390987 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252441 (secure 0 0 0) 0x7f377006eca0 con 0x7f374c066f90 2024-04-25T10:31:29.095 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:29.097 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f374c066f90 msgr2=0x7f374c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f374c066f90 0x7f374c069450 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f377015f9f0 tx=0x7f376000b040 comp rx=0 tx=0).stop 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 msgr2=0x7f37701573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 0x7f37701573d0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f37680ae040 tx=0x7f37680191a0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 shutdown_connections 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f374c066f90 0x7f374c069450 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f377015dce0 0x7f37701573d0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37700ad850 0x7f3770150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 --2- 172.21.15.45:0/1992390987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f377006eec0 0x7f3770151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 >> 172.21.15.45:0/1992390987 conn(0x7f37700b2410 msgr2=0x7f37700ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 shutdown_connections 2024-04-25T10:31:29.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:29.094+0000 7f3775000640 1 -- 172.21.15.45:0/1992390987 wait complete. 2024-04-25T10:31:29.115 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":586,"stamp":"2024-04-25T10:31:29.065108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465536,"kb_used_data":19080,"kb_used_omap":12,"kb_used_meta":446323,"kb_avail":754509184,"statfs":{"total":773094113280,"available":772617404416,"internally_reserved":0,"allocated":19537920,"data_stored":9220156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457035312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.642814"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444539+0000","last_change":"2024-04-25T10:31:23.435470+0000","last_active":"2024-04-25T10:31:24.444539+0000","last_peered":"2024-04-25T10:31:24.444539+0000","last_clean":"2024-04-25T10:31:24.444539+0000","last_became_active":"2024-04-25T10:31:23.433761+0000","last_became_peered":"2024-04-25T10:31:23.433761+0000","last_unstale":"2024-04-25T10:31:24.444539+0000","last_undegraded":"2024-04-25T10:31:24.444539+0000","last_fullsized":"2024-04-25T10:31:24.444539+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:31.132959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443506+0000","last_change":"2024-04-25T10:31:23.449323+0000","last_active":"2024-04-25T10:31:24.443506+0000","last_peered":"2024-04-25T10:31:24.443506+0000","last_clean":"2024-04-25T10:31:24.443506+0000","last_became_active":"2024-04-25T10:31:23.448894+0000","last_became_peered":"2024-04-25T10:31:23.448894+0000","last_unstale":"2024-04-25T10:31:24.443506+0000","last_undegraded":"2024-04-25T10:31:24.443506+0000","last_fullsized":"2024-04-25T10:31:24.443506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:19.802540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895465+0000","last_change":"2024-04-25T10:31:23.455956+0000","last_active":"2024-04-25T10:31:24.895465+0000","last_peered":"2024-04-25T10:31:24.895465+0000","last_clean":"2024-04-25T10:31:24.895465+0000","last_became_active":"2024-04-25T10:31:23.451989+0000","last_became_peered":"2024-04-25T10:31:23.451989+0000","last_unstale":"2024-04-25T10:31:24.895465+0000","last_undegraded":"2024-04-25T10:31:24.895465+0000","last_fullsized":"2024-04-25T10:31:24.895465+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.543524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438472+0000","last_change":"2024-04-25T10:31:23.441950+0000","last_active":"2024-04-25T10:31:24.438472+0000","last_peered":"2024-04-25T10:31:24.438472+0000","last_clean":"2024-04-25T10:31:24.438472+0000","last_became_active":"2024-04-25T10:31:23.439585+0000","last_became_peered":"2024-04-25T10:31:23.439585+0000","last_unstale":"2024-04-25T10:31:24.438472+0000","last_undegraded":"2024-04-25T10:31:24.438472+0000","last_fullsized":"2024-04-25T10:31:24.438472+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:58.112849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895376+0000","last_change":"2024-04-25T10:31:23.452097+0000","last_active":"2024-04-25T10:31:24.895376+0000","last_peered":"2024-04-25T10:31:24.895376+0000","last_clean":"2024-04-25T10:31:24.895376+0000","last_became_active":"2024-04-25T10:31:23.447134+0000","last_became_peered":"2024-04-25T10:31:23.447134+0000","last_unstale":"2024-04-25T10:31:24.895376+0000","last_undegraded":"2024-04-25T10:31:24.895376+0000","last_fullsized":"2024-04-25T10:31:24.895376+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:38.996519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443453+0000","last_change":"2024-04-25T10:31:23.436119+0000","last_active":"2024-04-25T10:31:24.443453+0000","last_peered":"2024-04-25T10:31:24.443453+0000","last_clean":"2024-04-25T10:31:24.443453+0000","last_became_active":"2024-04-25T10:31:23.435858+0000","last_became_peered":"2024-04-25T10:31:23.435858+0000","last_unstale":"2024-04-25T10:31:24.443453+0000","last_undegraded":"2024-04-25T10:31:24.443453+0000","last_fullsized":"2024-04-25T10:31:24.443453+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:15.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442850+0000","last_change":"2024-04-25T10:31:23.446258+0000","last_active":"2024-04-25T10:31:24.442850+0000","last_peered":"2024-04-25T10:31:24.442850+0000","last_clean":"2024-04-25T10:31:24.442850+0000","last_became_active":"2024-04-25T10:31:23.445639+0000","last_became_peered":"2024-04-25T10:31:23.445639+0000","last_unstale":"2024-04-25T10:31:24.442850+0000","last_undegraded":"2024-04-25T10:31:24.442850+0000","last_fullsized":"2024-04-25T10:31:24.442850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:17.415871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442772+0000","last_change":"2024-04-25T10:31:23.443133+0000","last_active":"2024-04-25T10:31:24.442772+0000","last_peered":"2024-04-25T10:31:24.442772+0000","last_clean":"2024-04-25T10:31:24.442772+0000","last_became_active":"2024-04-25T10:31:23.442850+0000","last_became_peered":"2024-04-25T10:31:23.442850+0000","last_unstale":"2024-04-25T10:31:24.442772+0000","last_undegraded":"2024-04-25T10:31:24.442772+0000","last_fullsized":"2024-04-25T10:31:24.442772+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:32.074770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445889+0000","last_change":"2024-04-25T10:31:23.434142+0000","last_active":"2024-04-25T10:31:24.445889+0000","last_peered":"2024-04-25T10:31:24.445889+0000","last_clean":"2024-04-25T10:31:24.445889+0000","last_became_active":"2024-04-25T10:31:23.433096+0000","last_became_peered":"2024-04-25T10:31:23.433096+0000","last_unstale":"2024-04-25T10:31:24.445889+0000","last_undegraded":"2024-04-25T10:31:24.445889+0000","last_fullsized":"2024-04-25T10:31:24.445889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895762+0000","last_change":"2024-04-25T10:31:23.445620+0000","last_active":"2024-04-25T10:31:24.895762+0000","last_peered":"2024-04-25T10:31:24.895762+0000","last_clean":"2024-04-25T10:31:24.895762+0000","last_became_active":"2024-04-25T10:31:23.445303+0000","last_became_peered":"2024-04-25T10:31:23.445303+0000","last_unstale":"2024-04-25T10:31:24.895762+0000","last_undegraded":"2024-04-25T10:31:24.895762+0000","last_fullsized":"2024-04-25T10:31:24.895762+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:54.583992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443855+0000","last_change":"2024-04-25T10:31:23.438628+0000","last_active":"2024-04-25T10:31:24.443855+0000","last_peered":"2024-04-25T10:31:24.443855+0000","last_clean":"2024-04-25T10:31:24.443855+0000","last_became_active":"2024-04-25T10:31:23.438301+0000","last_became_peered":"2024-04-25T10:31:23.438301+0000","last_unstale":"2024-04-25T10:31:24.443855+0000","last_undegraded":"2024-04-25T10:31:24.443855+0000","last_fullsized":"2024-04-25T10:31:24.443855+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:56.852220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892458+0000","last_change":"2024-04-25T10:31:23.444805+0000","last_active":"2024-04-25T10:31:24.892458+0000","last_peered":"2024-04-25T10:31:24.892458+0000","last_clean":"2024-04-25T10:31:24.892458+0000","last_became_active":"2024-04-25T10:31:23.444534+0000","last_became_peered":"2024-04-25T10:31:23.444534+0000","last_unstale":"2024-04-25T10:31:24.892458+0000","last_undegraded":"2024-04-25T10:31:24.892458+0000","last_fullsized":"2024-04-25T10:31:24.892458+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.995246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437564+0000","last_change":"2024-04-25T10:31:23.450197+0000","last_active":"2024-04-25T10:31:24.437564+0000","last_peered":"2024-04-25T10:31:24.437564+0000","last_clean":"2024-04-25T10:31:24.437564+0000","last_became_active":"2024-04-25T10:31:23.449302+0000","last_became_peered":"2024-04-25T10:31:23.449302+0000","last_unstale":"2024-04-25T10:31:24.437564+0000","last_undegraded":"2024-04-25T10:31:24.437564+0000","last_fullsized":"2024-04-25T10:31:24.437564+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:47.993239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894464+0000","last_change":"2024-04-25T10:31:23.449629+0000","last_active":"2024-04-25T10:31:24.894464+0000","last_peered":"2024-04-25T10:31:24.894464+0000","last_clean":"2024-04-25T10:31:24.894464+0000","last_became_active":"2024-04-25T10:31:23.449364+0000","last_became_peered":"2024-04-25T10:31:23.449364+0000","last_unstale":"2024-04-25T10:31:24.894464+0000","last_undegraded":"2024-04-25T10:31:24.894464+0000","last_fullsized":"2024-04-25T10:31:24.894464+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.779040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443094+0000","last_change":"2024-04-25T10:31:23.438502+0000","last_active":"2024-04-25T10:31:24.443094+0000","last_peered":"2024-04-25T10:31:24.443094+0000","last_clean":"2024-04-25T10:31:24.443094+0000","last_became_active":"2024-04-25T10:31:23.437827+0000","last_became_peered":"2024-04-25T10:31:23.437827+0000","last_unstale":"2024-04-25T10:31:24.443094+0000","last_undegraded":"2024-04-25T10:31:24.443094+0000","last_fullsized":"2024-04-25T10:31:24.443094+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:45.046678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443113+0000","last_change":"2024-04-25T10:31:23.435579+0000","last_active":"2024-04-25T10:31:24.443113+0000","last_peered":"2024-04-25T10:31:24.443113+0000","last_clean":"2024-04-25T10:31:24.443113+0000","last_became_active":"2024-04-25T10:31:23.432705+0000","last_became_peered":"2024-04-25T10:31:23.432705+0000","last_unstale":"2024-04-25T10:31:24.443113+0000","last_undegraded":"2024-04-25T10:31:24.443113+0000","last_fullsized":"2024-04-25T10:31:24.443113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:54.583992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444431+0000","last_change":"2024-04-25T10:31:23.450098+0000","last_active":"2024-04-25T10:31:24.444431+0000","last_peered":"2024-04-25T10:31:24.444431+0000","last_clean":"2024-04-25T10:31:24.444431+0000","last_became_active":"2024-04-25T10:31:23.449711+0000","last_became_peered":"2024-04-25T10:31:23.449711+0000","last_unstale":"2024-04-25T10:31:24.444431+0000","last_undegraded":"2024-04-25T10:31:24.444431+0000","last_fullsized":"2024-04-25T10:31:24.444431+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:45.563742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895680+0000","last_change":"2024-04-25T10:31:23.459671+0000","last_active":"2024-04-25T10:31:24.895680+0000","last_peered":"2024-04-25T10:31:24.895680+0000","last_clean":"2024-04-25T10:31:24.895680+0000","last_became_active":"2024-04-25T10:31:23.459391+0000","last_became_peered":"2024-04-25T10:31:23.459391+0000","last_unstale":"2024-04-25T10:31:24.895680+0000","last_undegraded":"2024-04-25T10:31:24.895680+0000","last_fullsized":"2024-04-25T10:31:24.895680+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:45.046678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895368+0000","last_change":"2024-04-25T10:31:23.456172+0000","last_active":"2024-04-25T10:31:24.895368+0000","last_peered":"2024-04-25T10:31:24.895368+0000","last_clean":"2024-04-25T10:31:24.895368+0000","last_became_active":"2024-04-25T10:31:23.452838+0000","last_became_peered":"2024-04-25T10:31:23.452838+0000","last_unstale":"2024-04-25T10:31:24.895368+0000","last_undegraded":"2024-04-25T10:31:24.895368+0000","last_fullsized":"2024-04-25T10:31:24.895368+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895295+0000","last_change":"2024-04-25T10:31:23.447420+0000","last_active":"2024-04-25T10:31:24.895295+0000","last_peered":"2024-04-25T10:31:24.895295+0000","last_clean":"2024-04-25T10:31:24.895295+0000","last_became_active":"2024-04-25T10:31:23.447112+0000","last_became_peered":"2024-04-25T10:31:23.447112+0000","last_unstale":"2024-04-25T10:31:24.895295+0000","last_undegraded":"2024-04-25T10:31:24.895295+0000","last_fullsized":"2024-04-25T10:31:24.895295+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892448+0000","last_change":"2024-04-25T10:31:23.450445+0000","last_active":"2024-04-25T10:31:24.892448+0000","last_peered":"2024-04-25T10:31:24.892448+0000","last_clean":"2024-04-25T10:31:24.892448+0000","last_became_active":"2024-04-25T10:31:23.450181+0000","last_became_peered":"2024-04-25T10:31:23.450181+0000","last_unstale":"2024-04-25T10:31:24.892448+0000","last_undegraded":"2024-04-25T10:31:24.892448+0000","last_fullsized":"2024-04-25T10:31:24.892448+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:47.724852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443636+0000","last_change":"2024-04-25T10:31:23.446467+0000","last_active":"2024-04-25T10:31:24.443636+0000","last_peered":"2024-04-25T10:31:24.443636+0000","last_clean":"2024-04-25T10:31:24.443636+0000","last_became_active":"2024-04-25T10:31:23.446185+0000","last_became_peered":"2024-04-25T10:31:23.446185+0000","last_unstale":"2024-04-25T10:31:24.443636+0000","last_undegraded":"2024-04-25T10:31:24.443636+0000","last_fullsized":"2024-04-25T10:31:24.443636+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:55.953997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437433+0000","last_change":"2024-04-25T10:31:23.432662+0000","last_active":"2024-04-25T10:31:24.437433+0000","last_peered":"2024-04-25T10:31:24.437433+0000","last_clean":"2024-04-25T10:31:24.437433+0000","last_became_active":"2024-04-25T10:31:23.432317+0000","last_became_peered":"2024-04-25T10:31:23.432317+0000","last_unstale":"2024-04-25T10:31:24.437433+0000","last_undegraded":"2024-04-25T10:31:24.437433+0000","last_fullsized":"2024-04-25T10:31:24.437433+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:01.330274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445557+0000","last_change":"2024-04-25T10:31:23.431527+0000","last_active":"2024-04-25T10:31:24.445557+0000","last_peered":"2024-04-25T10:31:24.445557+0000","last_clean":"2024-04-25T10:31:24.445557+0000","last_became_active":"2024-04-25T10:31:23.431141+0000","last_became_peered":"2024-04-25T10:31:23.431141+0000","last_unstale":"2024-04-25T10:31:24.445557+0000","last_undegraded":"2024-04-25T10:31:24.445557+0000","last_fullsized":"2024-04-25T10:31:24.445557+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438011+0000","last_change":"2024-04-25T10:31:23.441994+0000","last_active":"2024-04-25T10:31:24.438011+0000","last_peered":"2024-04-25T10:31:24.438011+0000","last_clean":"2024-04-25T10:31:24.438011+0000","last_became_active":"2024-04-25T10:31:23.439445+0000","last_became_peered":"2024-04-25T10:31:23.439445+0000","last_unstale":"2024-04-25T10:31:24.438011+0000","last_undegraded":"2024-04-25T10:31:24.438011+0000","last_fullsized":"2024-04-25T10:31:24.438011+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:12.920929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445013+0000","last_change":"2024-04-25T10:31:23.433270+0000","last_active":"2024-04-25T10:31:24.445013+0000","last_peered":"2024-04-25T10:31:24.445013+0000","last_clean":"2024-04-25T10:31:24.445013+0000","last_became_active":"2024-04-25T10:31:23.432966+0000","last_became_peered":"2024-04-25T10:31:23.432966+0000","last_unstale":"2024-04-25T10:31:24.445013+0000","last_undegraded":"2024-04-25T10:31:24.445013+0000","last_fullsized":"2024-04-25T10:31:24.445013+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:20.117231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892410+0000","last_change":"2024-04-25T10:31:23.458263+0000","last_active":"2024-04-25T10:31:24.892410+0000","last_peered":"2024-04-25T10:31:24.892410+0000","last_clean":"2024-04-25T10:31:24.892410+0000","last_became_active":"2024-04-25T10:31:23.457924+0000","last_became_peered":"2024-04-25T10:31:23.457924+0000","last_unstale":"2024-04-25T10:31:24.892410+0000","last_undegraded":"2024-04-25T10:31:24.892410+0000","last_fullsized":"2024-04-25T10:31:24.892410+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:50.981294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444516+0000","last_change":"2024-04-25T10:31:23.436457+0000","last_active":"2024-04-25T10:31:24.444516+0000","last_peered":"2024-04-25T10:31:24.444516+0000","last_clean":"2024-04-25T10:31:24.444516+0000","last_became_active":"2024-04-25T10:31:23.436186+0000","last_became_peered":"2024-04-25T10:31:23.436186+0000","last_unstale":"2024-04-25T10:31:24.444516+0000","last_undegraded":"2024-04-25T10:31:24.444516+0000","last_fullsized":"2024-04-25T10:31:24.444516+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:18.766534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443415+0000","last_change":"2024-04-25T10:31:23.435570+0000","last_active":"2024-04-25T10:31:24.443415+0000","last_peered":"2024-04-25T10:31:24.443415+0000","last_clean":"2024-04-25T10:31:24.443415+0000","last_became_active":"2024-04-25T10:31:23.435293+0000","last_became_peered":"2024-04-25T10:31:23.435293+0000","last_unstale":"2024-04-25T10:31:24.443415+0000","last_undegraded":"2024-04-25T10:31:24.443415+0000","last_fullsized":"2024-04-25T10:31:24.443415+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:21.689139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437501+0000","last_change":"2024-04-25T10:31:23.438686+0000","last_active":"2024-04-25T10:31:24.437501+0000","last_peered":"2024-04-25T10:31:24.437501+0000","last_clean":"2024-04-25T10:31:24.437501+0000","last_became_active":"2024-04-25T10:31:23.432709+0000","last_became_peered":"2024-04-25T10:31:23.432709+0000","last_unstale":"2024-04-25T10:31:24.437501+0000","last_undegraded":"2024-04-25T10:31:24.437501+0000","last_fullsized":"2024-04-25T10:31:24.437501+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:27.838602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894984+0000","last_change":"2024-04-25T10:31:23.443235+0000","last_active":"2024-04-25T10:31:24.894984+0000","last_peered":"2024-04-25T10:31:24.894984+0000","last_clean":"2024-04-25T10:31:24.894984+0000","last_became_active":"2024-04-25T10:31:23.442960+0000","last_became_peered":"2024-04-25T10:31:23.442960+0000","last_unstale":"2024-04-25T10:31:24.894984+0000","last_undegraded":"2024-04-25T10:31:24.894984+0000","last_fullsized":"2024-04-25T10:31:24.894984+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:31.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894425+0000","last_change":"2024-04-25T10:31:23.445902+0000","last_active":"2024-04-25T10:31:24.894425+0000","last_peered":"2024-04-25T10:31:24.894425+0000","last_clean":"2024-04-25T10:31:24.894425+0000","last_became_active":"2024-04-25T10:31:23.445617+0000","last_became_peered":"2024-04-25T10:31:23.445617+0000","last_unstale":"2024-04-25T10:31:24.894425+0000","last_undegraded":"2024-04-25T10:31:24.894425+0000","last_fullsized":"2024-04-25T10:31:24.894425+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.783049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892730+0000","last_change":"2024-04-25T10:31:23.456990+0000","last_active":"2024-04-25T10:31:24.892730+0000","last_peered":"2024-04-25T10:31:24.892730+0000","last_clean":"2024-04-25T10:31:24.892730+0000","last_became_active":"2024-04-25T10:31:23.451914+0000","last_became_peered":"2024-04-25T10:31:23.451914+0000","last_unstale":"2024-04-25T10:31:24.892730+0000","last_undegraded":"2024-04-25T10:31:24.892730+0000","last_fullsized":"2024-04-25T10:31:24.892730+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892722+0000","last_change":"2024-04-25T10:31:22.444200+0000","last_active":"2024-04-25T10:31:24.892722+0000","last_peered":"2024-04-25T10:31:24.892722+0000","last_clean":"2024-04-25T10:31:24.892722+0000","last_became_active":"2024-04-25T10:31:22.443896+0000","last_became_peered":"2024-04-25T10:31:22.443896+0000","last_unstale":"2024-04-25T10:31:24.892722+0000","last_undegraded":"2024-04-25T10:31:24.892722+0000","last_fullsized":"2024-04-25T10:31:24.892722+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:07.360121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438041+0000","last_change":"2024-04-25T10:31:23.438605+0000","last_active":"2024-04-25T10:31:24.438041+0000","last_peered":"2024-04-25T10:31:24.438041+0000","last_clean":"2024-04-25T10:31:24.438041+0000","last_became_active":"2024-04-25T10:31:23.437768+0000","last_became_peered":"2024-04-25T10:31:23.437768+0000","last_unstale":"2024-04-25T10:31:24.438041+0000","last_undegraded":"2024-04-25T10:31:24.438041+0000","last_fullsized":"2024-04-25T10:31:24.438041+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:08.369504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443500+0000","last_change":"2024-04-25T10:31:22.436396+0000","last_active":"2024-04-25T10:31:24.443500+0000","last_peered":"2024-04-25T10:31:24.443500+0000","last_clean":"2024-04-25T10:31:24.443500+0000","last_became_active":"2024-04-25T10:31:22.436121+0000","last_became_peered":"2024-04-25T10:31:22.436121+0000","last_unstale":"2024-04-25T10:31:24.443500+0000","last_undegraded":"2024-04-25T10:31:24.443500+0000","last_fullsized":"2024-04-25T10:31:24.443500+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:51.783124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895375+0000","last_change":"2024-04-25T10:31:23.457159+0000","last_active":"2024-04-25T10:31:24.895375+0000","last_peered":"2024-04-25T10:31:24.895375+0000","last_clean":"2024-04-25T10:31:24.895375+0000","last_became_active":"2024-04-25T10:31:23.456837+0000","last_became_peered":"2024-04-25T10:31:23.456837+0000","last_unstale":"2024-04-25T10:31:24.895375+0000","last_undegraded":"2024-04-25T10:31:24.895375+0000","last_fullsized":"2024-04-25T10:31:24.895375+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:10.584331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444666+0000","last_change":"2024-04-25T10:31:22.439739+0000","last_active":"2024-04-25T10:31:24.444666+0000","last_peered":"2024-04-25T10:31:24.444666+0000","last_clean":"2024-04-25T10:31:24.444666+0000","last_became_active":"2024-04-25T10:31:22.439422+0000","last_became_peered":"2024-04-25T10:31:22.439422+0000","last_unstale":"2024-04-25T10:31:24.444666+0000","last_undegraded":"2024-04-25T10:31:24.444666+0000","last_fullsized":"2024-04-25T10:31:24.444666+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:31.763486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895686+0000","last_change":"2024-04-25T10:31:23.456576+0000","last_active":"2024-04-25T10:31:24.895686+0000","last_peered":"2024-04-25T10:31:24.895686+0000","last_clean":"2024-04-25T10:31:24.895686+0000","last_became_active":"2024-04-25T10:31:23.455891+0000","last_became_peered":"2024-04-25T10:31:23.455891+0000","last_unstale":"2024-04-25T10:31:24.895686+0000","last_undegraded":"2024-04-25T10:31:24.895686+0000","last_fullsized":"2024-04-25T10:31:24.895686+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:17.415871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437782+0000","last_change":"2024-04-25T10:31:22.433482+0000","last_active":"2024-04-25T10:31:24.437782+0000","last_peered":"2024-04-25T10:31:24.437782+0000","last_clean":"2024-04-25T10:31:24.437782+0000","last_became_active":"2024-04-25T10:31:22.433201+0000","last_became_peered":"2024-04-25T10:31:22.433201+0000","last_unstale":"2024-04-25T10:31:24.437782+0000","last_undegraded":"2024-04-25T10:31:24.437782+0000","last_fullsized":"2024-04-25T10:31:24.437782+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:10.387788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438054+0000","last_change":"2024-04-25T10:31:23.452485+0000","last_active":"2024-04-25T10:31:24.438054+0000","last_peered":"2024-04-25T10:31:24.438054+0000","last_clean":"2024-04-25T10:31:24.438054+0000","last_became_active":"2024-04-25T10:31:23.452223+0000","last_became_peered":"2024-04-25T10:31:23.452223+0000","last_unstale":"2024-04-25T10:31:24.438054+0000","last_undegraded":"2024-04-25T10:31:24.438054+0000","last_fullsized":"2024-04-25T10:31:24.438054+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.478405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443707+0000","last_change":"2024-04-25T10:31:22.445426+0000","last_active":"2024-04-25T10:31:24.443707+0000","last_peered":"2024-04-25T10:31:24.443707+0000","last_clean":"2024-04-25T10:31:24.443707+0000","last_became_active":"2024-04-25T10:31:22.445054+0000","last_became_peered":"2024-04-25T10:31:22.445054+0000","last_unstale":"2024-04-25T10:31:24.443707+0000","last_undegraded":"2024-04-25T10:31:24.443707+0000","last_fullsized":"2024-04-25T10:31:24.443707+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443271+0000","last_change":"2024-04-25T10:31:23.434832+0000","last_active":"2024-04-25T10:31:24.443271+0000","last_peered":"2024-04-25T10:31:24.443271+0000","last_clean":"2024-04-25T10:31:24.443271+0000","last_became_active":"2024-04-25T10:31:23.433275+0000","last_became_peered":"2024-04-25T10:31:23.433275+0000","last_unstale":"2024-04-25T10:31:24.443271+0000","last_undegraded":"2024-04-25T10:31:24.443271+0000","last_fullsized":"2024-04-25T10:31:24.443271+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:23.154975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443864+0000","last_change":"2024-04-25T10:31:22.440188+0000","last_active":"2024-04-25T10:31:24.443864+0000","last_peered":"2024-04-25T10:31:24.443864+0000","last_clean":"2024-04-25T10:31:24.443864+0000","last_became_active":"2024-04-25T10:31:22.438568+0000","last_became_peered":"2024-04-25T10:31:22.438568+0000","last_unstale":"2024-04-25T10:31:24.443864+0000","last_undegraded":"2024-04-25T10:31:24.443864+0000","last_fullsized":"2024-04-25T10:31:24.443864+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:15.814748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445486+0000","last_change":"2024-04-25T10:31:23.445257+0000","last_active":"2024-04-25T10:31:24.445486+0000","last_peered":"2024-04-25T10:31:24.445486+0000","last_clean":"2024-04-25T10:31:24.445486+0000","last_became_active":"2024-04-25T10:31:23.444984+0000","last_became_peered":"2024-04-25T10:31:23.444984+0000","last_unstale":"2024-04-25T10:31:24.445486+0000","last_undegraded":"2024-04-25T10:31:24.445486+0000","last_fullsized":"2024-04-25T10:31:24.445486+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437364+0000","last_change":"2024-04-25T10:31:22.443402+0000","last_active":"2024-04-25T10:31:24.437364+0000","last_peered":"2024-04-25T10:31:24.437364+0000","last_clean":"2024-04-25T10:31:24.437364+0000","last_became_active":"2024-04-25T10:31:22.443034+0000","last_became_peered":"2024-04-25T10:31:22.443034+0000","last_unstale":"2024-04-25T10:31:24.437364+0000","last_undegraded":"2024-04-25T10:31:24.437364+0000","last_fullsized":"2024-04-25T10:31:24.437364+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:08.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437882+0000","last_change":"2024-04-25T10:31:23.432502+0000","last_active":"2024-04-25T10:31:24.437882+0000","last_peered":"2024-04-25T10:31:24.437882+0000","last_clean":"2024-04-25T10:31:24.437882+0000","last_became_active":"2024-04-25T10:31:23.430905+0000","last_became_peered":"2024-04-25T10:31:23.430905+0000","last_unstale":"2024-04-25T10:31:24.437882+0000","last_undegraded":"2024-04-25T10:31:24.437882+0000","last_fullsized":"2024-04-25T10:31:24.437882+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:24.427463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895537+0000","last_change":"2024-04-25T10:31:22.446818+0000","last_active":"2024-04-25T10:31:24.895537+0000","last_peered":"2024-04-25T10:31:24.895537+0000","last_clean":"2024-04-25T10:31:24.895537+0000","last_became_active":"2024-04-25T10:31:22.446198+0000","last_became_peered":"2024-04-25T10:31:22.446198+0000","last_unstale":"2024-04-25T10:31:24.895537+0000","last_undegraded":"2024-04-25T10:31:24.895537+0000","last_fullsized":"2024-04-25T10:31:24.895537+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:27.331081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894626+0000","last_change":"2024-04-25T10:31:23.460404+0000","last_active":"2024-04-25T10:31:24.894626+0000","last_peered":"2024-04-25T10:31:24.894626+0000","last_clean":"2024-04-25T10:31:24.894626+0000","last_became_active":"2024-04-25T10:31:23.459692+0000","last_became_peered":"2024-04-25T10:31:23.459692+0000","last_unstale":"2024-04-25T10:31:24.894626+0000","last_undegraded":"2024-04-25T10:31:24.894626+0000","last_fullsized":"2024-04-25T10:31:24.894626+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437905+0000","last_change":"2024-04-25T10:31:22.435228+0000","last_active":"2024-04-25T10:31:24.437905+0000","last_peered":"2024-04-25T10:31:24.437905+0000","last_clean":"2024-04-25T10:31:24.437905+0000","last_became_active":"2024-04-25T10:31:22.434923+0000","last_became_peered":"2024-04-25T10:31:22.434923+0000","last_unstale":"2024-04-25T10:31:24.437905+0000","last_undegraded":"2024-04-25T10:31:24.437905+0000","last_fullsized":"2024-04-25T10:31:24.437905+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:49.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445664+0000","last_change":"2024-04-25T10:31:23.433349+0000","last_active":"2024-04-25T10:31:24.445664+0000","last_peered":"2024-04-25T10:31:24.445664+0000","last_clean":"2024-04-25T10:31:24.445664+0000","last_became_active":"2024-04-25T10:31:23.433057+0000","last_became_peered":"2024-04-25T10:31:23.433057+0000","last_unstale":"2024-04-25T10:31:24.445664+0000","last_undegraded":"2024-04-25T10:31:24.445664+0000","last_fullsized":"2024-04-25T10:31:24.445664+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"181'3","reported_seq":17,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444682+0000","last_change":"2024-04-25T10:31:22.442622+0000","last_active":"2024-04-25T10:31:24.444682+0000","last_peered":"2024-04-25T10:31:24.444682+0000","last_clean":"2024-04-25T10:31:24.444682+0000","last_became_active":"2024-04-25T10:31:22.441086+0000","last_became_peered":"2024-04-25T10:31:22.441086+0000","last_unstale":"2024-04-25T10:31:24.444682+0000","last_undegraded":"2024-04-25T10:31:24.444682+0000","last_fullsized":"2024-04-25T10:31:24.444682+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:21.946296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444495+0000","last_change":"2024-04-25T10:31:23.448227+0000","last_active":"2024-04-25T10:31:24.444495+0000","last_peered":"2024-04-25T10:31:24.444495+0000","last_clean":"2024-04-25T10:31:24.444495+0000","last_became_active":"2024-04-25T10:31:23.447906+0000","last_became_peered":"2024-04-25T10:31:23.447906+0000","last_unstale":"2024-04-25T10:31:24.444495+0000","last_undegraded":"2024-04-25T10:31:24.444495+0000","last_fullsized":"2024-04-25T10:31:24.444495+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:23.339336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":20,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445723+0000","last_change":"2024-04-25T10:31:22.436482+0000","last_active":"2024-04-25T10:31:24.445723+0000","last_peered":"2024-04-25T10:31:24.445723+0000","last_clean":"2024-04-25T10:31:24.445723+0000","last_became_active":"2024-04-25T10:31:22.436192+0000","last_became_peered":"2024-04-25T10:31:22.436192+0000","last_unstale":"2024-04-25T10:31:24.445723+0000","last_undegraded":"2024-04-25T10:31:24.445723+0000","last_fullsized":"2024-04-25T10:31:24.445723+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894556+0000","last_change":"2024-04-25T10:31:23.448277+0000","last_active":"2024-04-25T10:31:24.894556+0000","last_peered":"2024-04-25T10:31:24.894556+0000","last_clean":"2024-04-25T10:31:24.894556+0000","last_became_active":"2024-04-25T10:31:23.447714+0000","last_became_peered":"2024-04-25T10:31:23.447714+0000","last_unstale":"2024-04-25T10:31:24.894556+0000","last_undegraded":"2024-04-25T10:31:24.894556+0000","last_fullsized":"2024-04-25T10:31:24.894556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445556+0000","last_change":"2024-04-25T10:31:22.437143+0000","last_active":"2024-04-25T10:31:24.445556+0000","last_peered":"2024-04-25T10:31:24.445556+0000","last_clean":"2024-04-25T10:31:24.445556+0000","last_became_active":"2024-04-25T10:31:22.436870+0000","last_became_peered":"2024-04-25T10:31:22.436870+0000","last_unstale":"2024-04-25T10:31:24.445556+0000","last_undegraded":"2024-04-25T10:31:24.445556+0000","last_fullsized":"2024-04-25T10:31:24.445556+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895631+0000","last_change":"2024-04-25T10:31:23.452399+0000","last_active":"2024-04-25T10:31:24.895631+0000","last_peered":"2024-04-25T10:31:24.895631+0000","last_clean":"2024-04-25T10:31:24.895631+0000","last_became_active":"2024-04-25T10:31:23.451792+0000","last_became_peered":"2024-04-25T10:31:23.451792+0000","last_unstale":"2024-04-25T10:31:24.895631+0000","last_undegraded":"2024-04-25T10:31:24.895631+0000","last_fullsized":"2024-04-25T10:31:24.895631+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:32.246063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443413+0000","last_change":"2024-04-25T10:31:22.435264+0000","last_active":"2024-04-25T10:31:24.443413+0000","last_peered":"2024-04-25T10:31:24.443413+0000","last_clean":"2024-04-25T10:31:24.443413+0000","last_became_active":"2024-04-25T10:31:22.434517+0000","last_became_peered":"2024-04-25T10:31:22.434517+0000","last_unstale":"2024-04-25T10:31:24.443413+0000","last_undegraded":"2024-04-25T10:31:24.443413+0000","last_fullsized":"2024-04-25T10:31:24.443413+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894965+0000","last_change":"2024-04-25T10:31:23.452667+0000","last_active":"2024-04-25T10:31:24.894965+0000","last_peered":"2024-04-25T10:31:24.894965+0000","last_clean":"2024-04-25T10:31:24.894965+0000","last_became_active":"2024-04-25T10:31:23.452366+0000","last_became_peered":"2024-04-25T10:31:23.452366+0000","last_unstale":"2024-04-25T10:31:24.894965+0000","last_undegraded":"2024-04-25T10:31:24.894965+0000","last_fullsized":"2024-04-25T10:31:24.894965+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443896+0000","last_change":"2024-04-25T10:31:22.441210+0000","last_active":"2024-04-25T10:31:24.443896+0000","last_peered":"2024-04-25T10:31:24.443896+0000","last_clean":"2024-04-25T10:31:24.443896+0000","last_became_active":"2024-04-25T10:31:22.440936+0000","last_became_peered":"2024-04-25T10:31:22.440936+0000","last_unstale":"2024-04-25T10:31:24.443896+0000","last_undegraded":"2024-04-25T10:31:24.443896+0000","last_fullsized":"2024-04-25T10:31:24.443896+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:08.832902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443141+0000","last_change":"2024-04-25T10:31:23.446637+0000","last_active":"2024-04-25T10:31:24.443141+0000","last_peered":"2024-04-25T10:31:24.443141+0000","last_clean":"2024-04-25T10:31:24.443141+0000","last_became_active":"2024-04-25T10:31:23.445987+0000","last_became_peered":"2024-04-25T10:31:23.445987+0000","last_unstale":"2024-04-25T10:31:24.443141+0000","last_undegraded":"2024-04-25T10:31:24.443141+0000","last_fullsized":"2024-04-25T10:31:24.443141+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:50.244723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"181'8","reported_seq":22,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443609+0000","last_change":"2024-04-25T10:31:22.436186+0000","last_active":"2024-04-25T10:31:24.443609+0000","last_peered":"2024-04-25T10:31:24.443609+0000","last_clean":"2024-04-25T10:31:24.443609+0000","last_became_active":"2024-04-25T10:31:22.435837+0000","last_became_peered":"2024-04-25T10:31:22.435837+0000","last_unstale":"2024-04-25T10:31:24.443609+0000","last_undegraded":"2024-04-25T10:31:24.443609+0000","last_fullsized":"2024-04-25T10:31:24.443609+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:25.915093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892796+0000","last_change":"2024-04-25T10:31:23.455136+0000","last_active":"2024-04-25T10:31:24.892796+0000","last_peered":"2024-04-25T10:31:24.892796+0000","last_clean":"2024-04-25T10:31:24.892796+0000","last_became_active":"2024-04-25T10:31:23.454870+0000","last_became_peered":"2024-04-25T10:31:23.454870+0000","last_unstale":"2024-04-25T10:31:24.892796+0000","last_undegraded":"2024-04-25T10:31:24.892796+0000","last_fullsized":"2024-04-25T10:31:24.892796+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444711+0000","last_change":"2024-04-25T10:31:22.445257+0000","last_active":"2024-04-25T10:31:24.444711+0000","last_peered":"2024-04-25T10:31:24.444711+0000","last_clean":"2024-04-25T10:31:24.444711+0000","last_became_active":"2024-04-25T10:31:22.444918+0000","last_became_peered":"2024-04-25T10:31:22.444918+0000","last_unstale":"2024-04-25T10:31:24.444711+0000","last_undegraded":"2024-04-25T10:31:24.444711+0000","last_fullsized":"2024-04-25T10:31:24.444711+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438159+0000","last_change":"2024-04-25T10:31:23.453329+0000","last_active":"2024-04-25T10:31:24.438159+0000","last_peered":"2024-04-25T10:31:24.438159+0000","last_clean":"2024-04-25T10:31:24.438159+0000","last_became_active":"2024-04-25T10:31:23.452956+0000","last_became_peered":"2024-04-25T10:31:23.452956+0000","last_unstale":"2024-04-25T10:31:24.438159+0000","last_undegraded":"2024-04-25T10:31:24.438159+0000","last_fullsized":"2024-04-25T10:31:24.438159+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:11.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445770+0000","last_change":"2024-04-25T10:31:22.436617+0000","last_active":"2024-04-25T10:31:24.445770+0000","last_peered":"2024-04-25T10:31:24.445770+0000","last_clean":"2024-04-25T10:31:24.445770+0000","last_became_active":"2024-04-25T10:31:22.436313+0000","last_became_peered":"2024-04-25T10:31:22.436313+0000","last_unstale":"2024-04-25T10:31:24.445770+0000","last_undegraded":"2024-04-25T10:31:24.445770+0000","last_fullsized":"2024-04-25T10:31:24.445770+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895606+0000","last_change":"2024-04-25T10:31:23.455754+0000","last_active":"2024-04-25T10:31:24.895606+0000","last_peered":"2024-04-25T10:31:24.895606+0000","last_clean":"2024-04-25T10:31:24.895606+0000","last_became_active":"2024-04-25T10:31:23.447211+0000","last_became_peered":"2024-04-25T10:31:23.447211+0000","last_unstale":"2024-04-25T10:31:24.895606+0000","last_undegraded":"2024-04-25T10:31:24.895606+0000","last_fullsized":"2024-04-25T10:31:24.895606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:28.103406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894782+0000","last_change":"2024-04-25T10:31:22.444016+0000","last_active":"2024-04-25T10:31:24.894782+0000","last_peered":"2024-04-25T10:31:24.894782+0000","last_clean":"2024-04-25T10:31:24.894782+0000","last_became_active":"2024-04-25T10:31:22.443711+0000","last_became_peered":"2024-04-25T10:31:22.443711+0000","last_unstale":"2024-04-25T10:31:24.894782+0000","last_undegraded":"2024-04-25T10:31:24.894782+0000","last_fullsized":"2024-04-25T10:31:24.894782+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:28.985863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443324+0000","last_change":"2024-04-25T10:31:23.434350+0000","last_active":"2024-04-25T10:31:24.443324+0000","last_peered":"2024-04-25T10:31:24.443324+0000","last_clean":"2024-04-25T10:31:24.443324+0000","last_became_active":"2024-04-25T10:31:23.434029+0000","last_became_peered":"2024-04-25T10:31:23.434029+0000","last_unstale":"2024-04-25T10:31:24.443324+0000","last_undegraded":"2024-04-25T10:31:24.443324+0000","last_fullsized":"2024-04-25T10:31:24.443324+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:47.857879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442891+0000","last_change":"2024-04-25T10:31:22.438645+0000","last_active":"2024-04-25T10:31:24.442891+0000","last_peered":"2024-04-25T10:31:24.442891+0000","last_clean":"2024-04-25T10:31:24.442891+0000","last_became_active":"2024-04-25T10:31:22.438312+0000","last_became_peered":"2024-04-25T10:31:22.438312+0000","last_unstale":"2024-04-25T10:31:24.442891+0000","last_undegraded":"2024-04-25T10:31:24.442891+0000","last_fullsized":"2024-04-25T10:31:24.442891+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895617+0000","last_change":"2024-04-25T10:31:23.450857+0000","last_active":"2024-04-25T10:31:24.895617+0000","last_peered":"2024-04-25T10:31:24.895617+0000","last_clean":"2024-04-25T10:31:24.895617+0000","last_became_active":"2024-04-25T10:31:23.449482+0000","last_became_peered":"2024-04-25T10:31:23.449482+0000","last_unstale":"2024-04-25T10:31:24.895617+0000","last_undegraded":"2024-04-25T10:31:24.895617+0000","last_fullsized":"2024-04-25T10:31:24.895617+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:28.466968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894371+0000","last_change":"2024-04-25T10:31:22.439123+0000","last_active":"2024-04-25T10:31:24.894371+0000","last_peered":"2024-04-25T10:31:24.894371+0000","last_clean":"2024-04-25T10:31:24.894371+0000","last_became_active":"2024-04-25T10:31:22.438833+0000","last_became_peered":"2024-04-25T10:31:22.438833+0000","last_unstale":"2024-04-25T10:31:24.894371+0000","last_undegraded":"2024-04-25T10:31:24.894371+0000","last_fullsized":"2024-04-25T10:31:24.894371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443231+0000","last_change":"2024-04-25T10:31:23.444511+0000","last_active":"2024-04-25T10:31:24.443231+0000","last_peered":"2024-04-25T10:31:24.443231+0000","last_clean":"2024-04-25T10:31:24.443231+0000","last_became_active":"2024-04-25T10:31:23.444242+0000","last_became_peered":"2024-04-25T10:31:23.444242+0000","last_unstale":"2024-04-25T10:31:24.443231+0000","last_undegraded":"2024-04-25T10:31:24.443231+0000","last_fullsized":"2024-04-25T10:31:24.443231+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:50.976811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444346+0000","last_change":"2024-04-25T10:31:22.438630+0000","last_active":"2024-04-25T10:31:24.444346+0000","last_peered":"2024-04-25T10:31:24.444346+0000","last_clean":"2024-04-25T10:31:24.444346+0000","last_became_active":"2024-04-25T10:31:22.438311+0000","last_became_peered":"2024-04-25T10:31:22.438311+0000","last_unstale":"2024-04-25T10:31:24.444346+0000","last_undegraded":"2024-04-25T10:31:24.444346+0000","last_fullsized":"2024-04-25T10:31:24.444346+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.394944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437306+0000","last_change":"2024-04-25T10:31:23.431648+0000","last_active":"2024-04-25T10:31:24.437306+0000","last_peered":"2024-04-25T10:31:24.437306+0000","last_clean":"2024-04-25T10:31:24.437306+0000","last_became_active":"2024-04-25T10:31:23.430571+0000","last_became_peered":"2024-04-25T10:31:23.430571+0000","last_unstale":"2024-04-25T10:31:24.437306+0000","last_undegraded":"2024-04-25T10:31:24.437306+0000","last_fullsized":"2024-04-25T10:31:24.437306+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:58.693323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895010+0000","last_change":"2024-04-25T10:31:22.446969+0000","last_active":"2024-04-25T10:31:24.895010+0000","last_peered":"2024-04-25T10:31:24.895010+0000","last_clean":"2024-04-25T10:31:24.895010+0000","last_became_active":"2024-04-25T10:31:22.446445+0000","last_became_peered":"2024-04-25T10:31:22.446445+0000","last_unstale":"2024-04-25T10:31:24.895010+0000","last_undegraded":"2024-04-25T10:31:24.895010+0000","last_fullsized":"2024-04-25T10:31:24.895010+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437617+0000","last_change":"2024-04-25T10:31:23.442351+0000","last_active":"2024-04-25T10:31:24.437617+0000","last_peered":"2024-04-25T10:31:24.437617+0000","last_clean":"2024-04-25T10:31:24.437617+0000","last_became_active":"2024-04-25T10:31:23.442056+0000","last_became_peered":"2024-04-25T10:31:23.442056+0000","last_unstale":"2024-04-25T10:31:24.437617+0000","last_undegraded":"2024-04-25T10:31:24.437617+0000","last_fullsized":"2024-04-25T10:31:24.437617+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:42.792189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443727+0000","last_change":"2024-04-25T10:31:22.443307+0000","last_active":"2024-04-25T10:31:24.443727+0000","last_peered":"2024-04-25T10:31:24.443727+0000","last_clean":"2024-04-25T10:31:24.443727+0000","last_became_active":"2024-04-25T10:31:22.442920+0000","last_became_peered":"2024-04-25T10:31:22.442920+0000","last_unstale":"2024-04-25T10:31:24.443727+0000","last_undegraded":"2024-04-25T10:31:24.443727+0000","last_fullsized":"2024-04-25T10:31:24.443727+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:44.842685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892820+0000","last_change":"2024-04-25T10:31:23.444361+0000","last_active":"2024-04-25T10:31:24.892820+0000","last_peered":"2024-04-25T10:31:24.892820+0000","last_clean":"2024-04-25T10:31:24.892820+0000","last_became_active":"2024-04-25T10:31:23.444080+0000","last_became_peered":"2024-04-25T10:31:23.444080+0000","last_unstale":"2024-04-25T10:31:24.892820+0000","last_undegraded":"2024-04-25T10:31:24.892820+0000","last_fullsized":"2024-04-25T10:31:24.892820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444937+0000","last_change":"2024-04-25T10:31:22.436880+0000","last_active":"2024-04-25T10:31:24.444937+0000","last_peered":"2024-04-25T10:31:24.444937+0000","last_clean":"2024-04-25T10:31:24.444937+0000","last_became_active":"2024-04-25T10:31:22.436570+0000","last_became_peered":"2024-04-25T10:31:22.436570+0000","last_unstale":"2024-04-25T10:31:24.444937+0000","last_undegraded":"2024-04-25T10:31:24.444937+0000","last_fullsized":"2024-04-25T10:31:24.444937+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:24.968695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444606+0000","last_change":"2024-04-25T10:31:23.449985+0000","last_active":"2024-04-25T10:31:24.444606+0000","last_peered":"2024-04-25T10:31:24.444606+0000","last_clean":"2024-04-25T10:31:24.444606+0000","last_became_active":"2024-04-25T10:31:23.449345+0000","last_became_peered":"2024-04-25T10:31:23.449345+0000","last_unstale":"2024-04-25T10:31:24.444606+0000","last_undegraded":"2024-04-25T10:31:24.444606+0000","last_fullsized":"2024-04-25T10:31:24.444606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.368876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"182'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443663+0000","last_change":"2024-04-25T10:31:22.440950+0000","last_active":"2024-04-25T10:31:24.443663+0000","last_peered":"2024-04-25T10:31:24.443663+0000","last_clean":"2024-04-25T10:31:24.443663+0000","last_became_active":"2024-04-25T10:31:22.440664+0000","last_became_peered":"2024-04-25T10:31:22.440664+0000","last_unstale":"2024-04-25T10:31:24.443663+0000","last_undegraded":"2024-04-25T10:31:24.443663+0000","last_fullsized":"2024-04-25T10:31:24.443663+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:21.897192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443530+0000","last_change":"2024-04-25T10:31:23.448149+0000","last_active":"2024-04-25T10:31:24.443530+0000","last_peered":"2024-04-25T10:31:24.443530+0000","last_clean":"2024-04-25T10:31:24.443530+0000","last_became_active":"2024-04-25T10:31:23.445318+0000","last_became_peered":"2024-04-25T10:31:23.445318+0000","last_unstale":"2024-04-25T10:31:24.443530+0000","last_undegraded":"2024-04-25T10:31:24.443530+0000","last_fullsized":"2024-04-25T10:31:24.443530+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:14.838094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443730+0000","last_change":"2024-04-25T10:31:22.446060+0000","last_active":"2024-04-25T10:31:24.443730+0000","last_peered":"2024-04-25T10:31:24.443730+0000","last_clean":"2024-04-25T10:31:24.443730+0000","last_became_active":"2024-04-25T10:31:22.445706+0000","last_became_peered":"2024-04-25T10:31:22.445706+0000","last_unstale":"2024-04-25T10:31:24.443730+0000","last_undegraded":"2024-04-25T10:31:24.443730+0000","last_fullsized":"2024-04-25T10:31:24.443730+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:31.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892620+0000","last_change":"2024-04-25T10:31:23.453869+0000","last_active":"2024-04-25T10:31:24.892620+0000","last_peered":"2024-04-25T10:31:24.892620+0000","last_clean":"2024-04-25T10:31:24.892620+0000","last_became_active":"2024-04-25T10:31:23.453600+0000","last_became_peered":"2024-04-25T10:31:23.453600+0000","last_unstale":"2024-04-25T10:31:24.892620+0000","last_undegraded":"2024-04-25T10:31:24.892620+0000","last_fullsized":"2024-04-25T10:31:24.892620+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443689+0000","last_change":"2024-04-25T10:31:22.445692+0000","last_active":"2024-04-25T10:31:24.443689+0000","last_peered":"2024-04-25T10:31:24.443689+0000","last_clean":"2024-04-25T10:31:24.443689+0000","last_became_active":"2024-04-25T10:31:22.445368+0000","last_became_peered":"2024-04-25T10:31:22.445368+0000","last_unstale":"2024-04-25T10:31:24.443689+0000","last_undegraded":"2024-04-25T10:31:24.443689+0000","last_fullsized":"2024-04-25T10:31:24.443689+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:07.812518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443603+0000","last_change":"2024-04-25T10:31:23.434539+0000","last_active":"2024-04-25T10:31:24.443603+0000","last_peered":"2024-04-25T10:31:24.443603+0000","last_clean":"2024-04-25T10:31:24.443603+0000","last_became_active":"2024-04-25T10:31:23.434040+0000","last_became_peered":"2024-04-25T10:31:23.434040+0000","last_unstale":"2024-04-25T10:31:24.443603+0000","last_undegraded":"2024-04-25T10:31:24.443603+0000","last_fullsized":"2024-04-25T10:31:24.443603+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:58.561747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437773+0000","last_change":"2024-04-25T10:31:22.431495+0000","last_active":"2024-04-25T10:31:24.437773+0000","last_peered":"2024-04-25T10:31:24.437773+0000","last_clean":"2024-04-25T10:31:24.437773+0000","last_became_active":"2024-04-25T10:31:22.431217+0000","last_became_peered":"2024-04-25T10:31:22.431217+0000","last_unstale":"2024-04-25T10:31:24.437773+0000","last_undegraded":"2024-04-25T10:31:24.437773+0000","last_fullsized":"2024-04-25T10:31:24.437773+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:27.633982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443417+0000","last_change":"2024-04-25T10:31:23.445933+0000","last_active":"2024-04-25T10:31:24.443417+0000","last_peered":"2024-04-25T10:31:24.443417+0000","last_clean":"2024-04-25T10:31:24.443417+0000","last_became_active":"2024-04-25T10:31:23.445660+0000","last_became_peered":"2024-04-25T10:31:23.445660+0000","last_unstale":"2024-04-25T10:31:24.443417+0000","last_undegraded":"2024-04-25T10:31:24.443417+0000","last_fullsized":"2024-04-25T10:31:24.443417+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:09.073459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895310+0000","last_change":"2024-04-25T10:31:22.445594+0000","last_active":"2024-04-25T10:31:24.895310+0000","last_peered":"2024-04-25T10:31:24.895310+0000","last_clean":"2024-04-25T10:31:24.895310+0000","last_became_active":"2024-04-25T10:31:22.445288+0000","last_became_peered":"2024-04-25T10:31:22.445288+0000","last_unstale":"2024-04-25T10:31:24.895310+0000","last_undegraded":"2024-04-25T10:31:24.895310+0000","last_fullsized":"2024-04-25T10:31:24.895310+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:54.229800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442850+0000","last_change":"2024-04-25T10:31:23.445663+0000","last_active":"2024-04-25T10:31:24.442850+0000","last_peered":"2024-04-25T10:31:24.442850+0000","last_clean":"2024-04-25T10:31:24.442850+0000","last_became_active":"2024-04-25T10:31:23.445361+0000","last_became_peered":"2024-04-25T10:31:23.445361+0000","last_unstale":"2024-04-25T10:31:24.442850+0000","last_undegraded":"2024-04-25T10:31:24.442850+0000","last_fullsized":"2024-04-25T10:31:24.442850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:12.695852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892351+0000","last_change":"2024-04-25T10:31:22.440617+0000","last_active":"2024-04-25T10:31:24.892351+0000","last_peered":"2024-04-25T10:31:24.892351+0000","last_clean":"2024-04-25T10:31:24.892351+0000","last_became_active":"2024-04-25T10:31:22.440345+0000","last_became_peered":"2024-04-25T10:31:22.440345+0000","last_unstale":"2024-04-25T10:31:24.892351+0000","last_undegraded":"2024-04-25T10:31:24.892351+0000","last_fullsized":"2024-04-25T10:31:24.892351+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:11.911547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437625+0000","last_change":"2024-04-25T10:31:23.443024+0000","last_active":"2024-04-25T10:31:24.437625+0000","last_peered":"2024-04-25T10:31:24.437625+0000","last_clean":"2024-04-25T10:31:24.437625+0000","last_became_active":"2024-04-25T10:31:23.442748+0000","last_became_peered":"2024-04-25T10:31:23.442748+0000","last_unstale":"2024-04-25T10:31:24.437625+0000","last_undegraded":"2024-04-25T10:31:24.437625+0000","last_fullsized":"2024-04-25T10:31:24.437625+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.154345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437413+0000","last_change":"2024-04-25T10:31:22.441054+0000","last_active":"2024-04-25T10:31:24.437413+0000","last_peered":"2024-04-25T10:31:24.437413+0000","last_clean":"2024-04-25T10:31:24.437413+0000","last_became_active":"2024-04-25T10:31:22.440642+0000","last_became_peered":"2024-04-25T10:31:22.440642+0000","last_unstale":"2024-04-25T10:31:24.437413+0000","last_undegraded":"2024-04-25T10:31:24.437413+0000","last_fullsized":"2024-04-25T10:31:24.437413+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895180+0000","last_change":"2024-04-25T10:31:23.446822+0000","last_active":"2024-04-25T10:31:24.895180+0000","last_peered":"2024-04-25T10:31:24.895180+0000","last_clean":"2024-04-25T10:31:24.895180+0000","last_became_active":"2024-04-25T10:31:23.446539+0000","last_became_peered":"2024-04-25T10:31:23.446539+0000","last_unstale":"2024-04-25T10:31:24.895180+0000","last_undegraded":"2024-04-25T10:31:24.895180+0000","last_fullsized":"2024-04-25T10:31:24.895180+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443530+0000","last_change":"2024-04-25T10:31:22.436501+0000","last_active":"2024-04-25T10:31:24.443530+0000","last_peered":"2024-04-25T10:31:24.443530+0000","last_clean":"2024-04-25T10:31:24.443530+0000","last_became_active":"2024-04-25T10:31:22.436167+0000","last_became_peered":"2024-04-25T10:31:22.436167+0000","last_unstale":"2024-04-25T10:31:24.443530+0000","last_undegraded":"2024-04-25T10:31:24.443530+0000","last_fullsized":"2024-04-25T10:31:24.443530+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.310017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":480,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437480+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:24.437480+0000","last_peered":"2024-04-25T10:31:24.437480+0000","last_clean":"2024-04-25T10:31:24.437480+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:24.437480+0000","last_undegraded":"2024-04-25T10:31:24.437480+0000","last_fullsized":"2024-04-25T10:31:24.437480+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509607,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55628,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316212,"statfs":{"total":96636764160,"available":96579801088,"internally_reserved":0,"allocated":2289664,"data_stored":1004347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509608,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55624,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316216,"statfs":{"total":96636764160,"available":96579805184,"internally_reserved":0,"allocated":2289664,"data_stored":1004395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509607,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60376,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311464,"statfs":{"total":96636764160,"available":96574939136,"internally_reserved":0,"allocated":2895872,"data_stored":1595421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509607,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47316,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324524,"statfs":{"total":96636764160,"available":96588312576,"internally_reserved":0,"allocated":2297856,"data_stored":1005937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509607,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69472,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302368,"statfs":{"total":96636764160,"available":96565624832,"internally_reserved":0,"allocated":2306048,"data_stored":1006041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509608,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":2812,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":2879488,"data_stored":1594693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509608,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2289664,"data_stored":1004907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542310,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69452,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302388,"statfs":{"total":96636764160,"available":96565645312,"internally_reserved":0,"allocated":2289664,"data_stored":1004415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:30.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:30.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:31:32.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 Processor -- start 2024-04-25T10:31:32.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 -- start start 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ff01550b0 0x7f2ff00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ff00cfe70 0x7f2ff015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff0071050 con 0x7f2ff015b740 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff015d040 con 0x7f2ff00cfe70 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.270+0000 7f2ff8200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff00712f0 con 0x7f2ff01550b0 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff4e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2ff00cfe70 0x7f2ff015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff6200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ff015b740 0x7f2ff006d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55706/0 (socket says 172.21.15.45:55706) 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff6200640 1 -- 172.21.15.45:0/2843504028 learned_addr learned my addr 172.21.15.45:0/2843504028 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:32.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff5800640 1 --2- 172.21.15.45:0/2843504028 >> v2:172.21.15.117:3300/0 conn(0x7f2ff01550b0 0x7f2ff00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff4e00640 1 -- 172.21.15.45:0/2843504028 >> v2:172.21.15.117:3300/0 conn(0x7f2ff01550b0 msgr2=0x7f2ff00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff4e00640 1 --2- 172.21.15.45:0/2843504028 >> v2:172.21.15.117:3300/0 conn(0x7f2ff01550b0 0x7f2ff00cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2fefe00640 1 -- 172.21.15.45:0/2843504028 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155230391 0 0) 0x7f2ff0071050 con 0x7f2ff015b740 2024-04-25T10:31:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff4e00640 1 -- 172.21.15.45:0/2843504028 >> v1:172.21.15.45:6789/0 conn(0x7f2ff015b740 legacy=0x7f2ff006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff4e00640 1 -- 172.21.15.45:0/2843504028 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00715d0 con 0x7f2ff00cfe70 2024-04-25T10:31:32.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2ff4e00640 1 --2- 172.21.15.45:0/2843504028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff015b200 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0097c0 tx=0x7f2fdc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3563524c41d3a9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2fefe00640 1 -- 172.21.15.45:0/2843504028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0b0070 con 0x7f2ff00cfe70 2024-04-25T10:31:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2fefe00640 1 -- 172.21.15.45:0/2843504028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc0ab600 con 0x7f2ff00cfe70 2024-04-25T10:31:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.274+0000 7f2fefe00640 1 -- 172.21.15.45:0/2843504028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0ab9d0 con 0x7f2ff00cfe70 2024-04-25T10:31:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- 172.21.15.45:0/2843504028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 msgr2=0x7f2ff015b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 --2- 172.21.15.45:0/2843504028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff015b200 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0097c0 tx=0x7f2fdc01c460 comp rx=0 tx=0).stop 2024-04-25T10:31:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- 172.21.15.45:0/2843504028 shutdown_connections 2024-04-25T10:31:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 --2- 172.21.15.45:0/2843504028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff015b200 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 --2- 172.21.15.45:0/2843504028 >> v2:172.21.15.117:3300/0 conn(0x7f2ff01550b0 0x7f2ff00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- 172.21.15.45:0/2843504028 >> 172.21.15.45:0/2843504028 conn(0x7f2ff0066b70 msgr2=0x7f2ff0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- 172.21.15.45:0/2843504028 shutdown_connections 2024-04-25T10:31:32.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- 172.21.15.45:0/2843504028 wait complete. 2024-04-25T10:31:32.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 Processor -- start 2024-04-25T10:31:32.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- start start 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff01520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff01550b0 0x7f2ff014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff015b740 0x7f2ff0147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015bcf0 con 0x7f2ff00cfe70 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015bb70 con 0x7f2ff01550b0 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff8200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015c030 con 0x7f2ff015b740 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff01520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff01520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57656/0 (socket says 172.21.15.45:57656) 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 -- 172.21.15.45:0/1420217425 learned_addr learned my addr 172.21.15.45:0/1420217425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff4e00640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff01550b0 0x7f2ff014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 -- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff015b740 msgr2=0x7f2ff0147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff015b740 0x7f2ff0147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 -- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff01550b0 msgr2=0x7f2ff014d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff01550b0 0x7f2ff014d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.278+0000 7f2ff5800640 1 -- 172.21.15.45:0/1420217425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0150470 con 0x7f2ff00cfe70 2024-04-25T10:31:32.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2ff5800640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff01520b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e930 tx=0x7f2fe400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4017070 con 0x7f2ff00cfe70 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe400ca00 con 0x7f2ff00cfe70 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400cdb0 con 0x7f2ff00cfe70 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff014ded0 con 0x7f2ff00cfe70 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff0148b90 con 0x7f2ff00cfe70 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff004ef30 con 0x7f2ff00cfe70 2024-04-25T10:31:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fe401a070 con 0x7f2ff00cfe70 2024-04-25T10:31:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2fed600640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fd0066f60 0x7f2fd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fe4085100 con 0x7f2ff00cfe70 2024-04-25T10:31:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.282+0000 7f2ff4e00640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fd0066f60 0x7f2fd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.286+0000 7f2ff4e00640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fd0066f60 0x7f2fd0069420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0ad000 tx=0x7f2fdc0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.286+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fe4056130 con 0x7f2ff00cfe70 2024-04-25T10:31:32.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.446+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ff00adfc0 con 0x7f2fd0066f60 2024-04-25T10:31:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.454+0000 7f2fed600640 1 -- 172.21.15.45:0/1420217425 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252441 (secure 0 0 0) 0x7f2ff00adfc0 con 0x7f2fd0066f60 2024-04-25T10:31:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:32.461 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:31:32.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fd0066f60 msgr2=0x7f2fd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fd0066f60 0x7f2fd0069420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0ad000 tx=0x7f2fdc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 msgr2=0x7f2ff01520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff01520b0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e930 tx=0x7f2fe400edf0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 shutdown_connections 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2fd0066f60 0x7f2fd0069420 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ff015b740 0x7f2ff0147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff01550b0 0x7f2ff014d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 --2- 172.21.15.45:0/1420217425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00cfe70 0x7f2ff01520b0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 >> 172.21.15.45:0/1420217425 conn(0x7f2ff0066b70 msgr2=0x7f2ff00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 shutdown_connections 2024-04-25T10:31:32.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.458+0000 7f2ff8200640 1 -- 172.21.15.45:0/1420217425 wait complete. 2024-04-25T10:31:32.482 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":587,"stamp":"2024-04-25T10:31:31.065656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465536,"kb_used_data":19080,"kb_used_omap":12,"kb_used_meta":446323,"kb_avail":754509184,"statfs":{"total":773094113280,"available":772617404416,"internally_reserved":0,"allocated":19537920,"data_stored":9220156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457035312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001987"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444539+0000","last_change":"2024-04-25T10:31:23.435470+0000","last_active":"2024-04-25T10:31:24.444539+0000","last_peered":"2024-04-25T10:31:24.444539+0000","last_clean":"2024-04-25T10:31:24.444539+0000","last_became_active":"2024-04-25T10:31:23.433761+0000","last_became_peered":"2024-04-25T10:31:23.433761+0000","last_unstale":"2024-04-25T10:31:24.444539+0000","last_undegraded":"2024-04-25T10:31:24.444539+0000","last_fullsized":"2024-04-25T10:31:24.444539+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:31.132959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443506+0000","last_change":"2024-04-25T10:31:23.449323+0000","last_active":"2024-04-25T10:31:24.443506+0000","last_peered":"2024-04-25T10:31:24.443506+0000","last_clean":"2024-04-25T10:31:24.443506+0000","last_became_active":"2024-04-25T10:31:23.448894+0000","last_became_peered":"2024-04-25T10:31:23.448894+0000","last_unstale":"2024-04-25T10:31:24.443506+0000","last_undegraded":"2024-04-25T10:31:24.443506+0000","last_fullsized":"2024-04-25T10:31:24.443506+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:19.802540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895465+0000","last_change":"2024-04-25T10:31:23.455956+0000","last_active":"2024-04-25T10:31:24.895465+0000","last_peered":"2024-04-25T10:31:24.895465+0000","last_clean":"2024-04-25T10:31:24.895465+0000","last_became_active":"2024-04-25T10:31:23.451989+0000","last_became_peered":"2024-04-25T10:31:23.451989+0000","last_unstale":"2024-04-25T10:31:24.895465+0000","last_undegraded":"2024-04-25T10:31:24.895465+0000","last_fullsized":"2024-04-25T10:31:24.895465+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:56.543524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438472+0000","last_change":"2024-04-25T10:31:23.441950+0000","last_active":"2024-04-25T10:31:24.438472+0000","last_peered":"2024-04-25T10:31:24.438472+0000","last_clean":"2024-04-25T10:31:24.438472+0000","last_became_active":"2024-04-25T10:31:23.439585+0000","last_became_peered":"2024-04-25T10:31:23.439585+0000","last_unstale":"2024-04-25T10:31:24.438472+0000","last_undegraded":"2024-04-25T10:31:24.438472+0000","last_fullsized":"2024-04-25T10:31:24.438472+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:58.112849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895376+0000","last_change":"2024-04-25T10:31:23.452097+0000","last_active":"2024-04-25T10:31:24.895376+0000","last_peered":"2024-04-25T10:31:24.895376+0000","last_clean":"2024-04-25T10:31:24.895376+0000","last_became_active":"2024-04-25T10:31:23.447134+0000","last_became_peered":"2024-04-25T10:31:23.447134+0000","last_unstale":"2024-04-25T10:31:24.895376+0000","last_undegraded":"2024-04-25T10:31:24.895376+0000","last_fullsized":"2024-04-25T10:31:24.895376+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:38.996519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443453+0000","last_change":"2024-04-25T10:31:23.436119+0000","last_active":"2024-04-25T10:31:24.443453+0000","last_peered":"2024-04-25T10:31:24.443453+0000","last_clean":"2024-04-25T10:31:24.443453+0000","last_became_active":"2024-04-25T10:31:23.435858+0000","last_became_peered":"2024-04-25T10:31:23.435858+0000","last_unstale":"2024-04-25T10:31:24.443453+0000","last_undegraded":"2024-04-25T10:31:24.443453+0000","last_fullsized":"2024-04-25T10:31:24.443453+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:15.559285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442850+0000","last_change":"2024-04-25T10:31:23.446258+0000","last_active":"2024-04-25T10:31:24.442850+0000","last_peered":"2024-04-25T10:31:24.442850+0000","last_clean":"2024-04-25T10:31:24.442850+0000","last_became_active":"2024-04-25T10:31:23.445639+0000","last_became_peered":"2024-04-25T10:31:23.445639+0000","last_unstale":"2024-04-25T10:31:24.442850+0000","last_undegraded":"2024-04-25T10:31:24.442850+0000","last_fullsized":"2024-04-25T10:31:24.442850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:17.415871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442772+0000","last_change":"2024-04-25T10:31:23.443133+0000","last_active":"2024-04-25T10:31:24.442772+0000","last_peered":"2024-04-25T10:31:24.442772+0000","last_clean":"2024-04-25T10:31:24.442772+0000","last_became_active":"2024-04-25T10:31:23.442850+0000","last_became_peered":"2024-04-25T10:31:23.442850+0000","last_unstale":"2024-04-25T10:31:24.442772+0000","last_undegraded":"2024-04-25T10:31:24.442772+0000","last_fullsized":"2024-04-25T10:31:24.442772+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:32.074770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445889+0000","last_change":"2024-04-25T10:31:23.434142+0000","last_active":"2024-04-25T10:31:24.445889+0000","last_peered":"2024-04-25T10:31:24.445889+0000","last_clean":"2024-04-25T10:31:24.445889+0000","last_became_active":"2024-04-25T10:31:23.433096+0000","last_became_peered":"2024-04-25T10:31:23.433096+0000","last_unstale":"2024-04-25T10:31:24.445889+0000","last_undegraded":"2024-04-25T10:31:24.445889+0000","last_fullsized":"2024-04-25T10:31:24.445889+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895762+0000","last_change":"2024-04-25T10:31:23.445620+0000","last_active":"2024-04-25T10:31:24.895762+0000","last_peered":"2024-04-25T10:31:24.895762+0000","last_clean":"2024-04-25T10:31:24.895762+0000","last_became_active":"2024-04-25T10:31:23.445303+0000","last_became_peered":"2024-04-25T10:31:23.445303+0000","last_unstale":"2024-04-25T10:31:24.895762+0000","last_undegraded":"2024-04-25T10:31:24.895762+0000","last_fullsized":"2024-04-25T10:31:24.895762+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:54.583992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443855+0000","last_change":"2024-04-25T10:31:23.438628+0000","last_active":"2024-04-25T10:31:24.443855+0000","last_peered":"2024-04-25T10:31:24.443855+0000","last_clean":"2024-04-25T10:31:24.443855+0000","last_became_active":"2024-04-25T10:31:23.438301+0000","last_became_peered":"2024-04-25T10:31:23.438301+0000","last_unstale":"2024-04-25T10:31:24.443855+0000","last_undegraded":"2024-04-25T10:31:24.443855+0000","last_fullsized":"2024-04-25T10:31:24.443855+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:56.852220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892458+0000","last_change":"2024-04-25T10:31:23.444805+0000","last_active":"2024-04-25T10:31:24.892458+0000","last_peered":"2024-04-25T10:31:24.892458+0000","last_clean":"2024-04-25T10:31:24.892458+0000","last_became_active":"2024-04-25T10:31:23.444534+0000","last_became_peered":"2024-04-25T10:31:23.444534+0000","last_unstale":"2024-04-25T10:31:24.892458+0000","last_undegraded":"2024-04-25T10:31:24.892458+0000","last_fullsized":"2024-04-25T10:31:24.892458+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:29.995246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437564+0000","last_change":"2024-04-25T10:31:23.450197+0000","last_active":"2024-04-25T10:31:24.437564+0000","last_peered":"2024-04-25T10:31:24.437564+0000","last_clean":"2024-04-25T10:31:24.437564+0000","last_became_active":"2024-04-25T10:31:23.449302+0000","last_became_peered":"2024-04-25T10:31:23.449302+0000","last_unstale":"2024-04-25T10:31:24.437564+0000","last_undegraded":"2024-04-25T10:31:24.437564+0000","last_fullsized":"2024-04-25T10:31:24.437564+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:47.993239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894464+0000","last_change":"2024-04-25T10:31:23.449629+0000","last_active":"2024-04-25T10:31:24.894464+0000","last_peered":"2024-04-25T10:31:24.894464+0000","last_clean":"2024-04-25T10:31:24.894464+0000","last_became_active":"2024-04-25T10:31:23.449364+0000","last_became_peered":"2024-04-25T10:31:23.449364+0000","last_unstale":"2024-04-25T10:31:24.894464+0000","last_undegraded":"2024-04-25T10:31:24.894464+0000","last_fullsized":"2024-04-25T10:31:24.894464+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:19.779040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443094+0000","last_change":"2024-04-25T10:31:23.438502+0000","last_active":"2024-04-25T10:31:24.443094+0000","last_peered":"2024-04-25T10:31:24.443094+0000","last_clean":"2024-04-25T10:31:24.443094+0000","last_became_active":"2024-04-25T10:31:23.437827+0000","last_became_peered":"2024-04-25T10:31:23.437827+0000","last_unstale":"2024-04-25T10:31:24.443094+0000","last_undegraded":"2024-04-25T10:31:24.443094+0000","last_fullsized":"2024-04-25T10:31:24.443094+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:45.046678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443113+0000","last_change":"2024-04-25T10:31:23.435579+0000","last_active":"2024-04-25T10:31:24.443113+0000","last_peered":"2024-04-25T10:31:24.443113+0000","last_clean":"2024-04-25T10:31:24.443113+0000","last_became_active":"2024-04-25T10:31:23.432705+0000","last_became_peered":"2024-04-25T10:31:23.432705+0000","last_unstale":"2024-04-25T10:31:24.443113+0000","last_undegraded":"2024-04-25T10:31:24.443113+0000","last_fullsized":"2024-04-25T10:31:24.443113+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:54.583992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444431+0000","last_change":"2024-04-25T10:31:23.450098+0000","last_active":"2024-04-25T10:31:24.444431+0000","last_peered":"2024-04-25T10:31:24.444431+0000","last_clean":"2024-04-25T10:31:24.444431+0000","last_became_active":"2024-04-25T10:31:23.449711+0000","last_became_peered":"2024-04-25T10:31:23.449711+0000","last_unstale":"2024-04-25T10:31:24.444431+0000","last_undegraded":"2024-04-25T10:31:24.444431+0000","last_fullsized":"2024-04-25T10:31:24.444431+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:45.563742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895680+0000","last_change":"2024-04-25T10:31:23.459671+0000","last_active":"2024-04-25T10:31:24.895680+0000","last_peered":"2024-04-25T10:31:24.895680+0000","last_clean":"2024-04-25T10:31:24.895680+0000","last_became_active":"2024-04-25T10:31:23.459391+0000","last_became_peered":"2024-04-25T10:31:23.459391+0000","last_unstale":"2024-04-25T10:31:24.895680+0000","last_undegraded":"2024-04-25T10:31:24.895680+0000","last_fullsized":"2024-04-25T10:31:24.895680+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:45.046678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895368+0000","last_change":"2024-04-25T10:31:23.456172+0000","last_active":"2024-04-25T10:31:24.895368+0000","last_peered":"2024-04-25T10:31:24.895368+0000","last_clean":"2024-04-25T10:31:24.895368+0000","last_became_active":"2024-04-25T10:31:23.452838+0000","last_became_peered":"2024-04-25T10:31:23.452838+0000","last_unstale":"2024-04-25T10:31:24.895368+0000","last_undegraded":"2024-04-25T10:31:24.895368+0000","last_fullsized":"2024-04-25T10:31:24.895368+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.485900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895295+0000","last_change":"2024-04-25T10:31:23.447420+0000","last_active":"2024-04-25T10:31:24.895295+0000","last_peered":"2024-04-25T10:31:24.895295+0000","last_clean":"2024-04-25T10:31:24.895295+0000","last_became_active":"2024-04-25T10:31:23.447112+0000","last_became_peered":"2024-04-25T10:31:23.447112+0000","last_unstale":"2024-04-25T10:31:24.895295+0000","last_undegraded":"2024-04-25T10:31:24.895295+0000","last_fullsized":"2024-04-25T10:31:24.895295+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892448+0000","last_change":"2024-04-25T10:31:23.450445+0000","last_active":"2024-04-25T10:31:24.892448+0000","last_peered":"2024-04-25T10:31:24.892448+0000","last_clean":"2024-04-25T10:31:24.892448+0000","last_became_active":"2024-04-25T10:31:23.450181+0000","last_became_peered":"2024-04-25T10:31:23.450181+0000","last_unstale":"2024-04-25T10:31:24.892448+0000","last_undegraded":"2024-04-25T10:31:24.892448+0000","last_fullsized":"2024-04-25T10:31:24.892448+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:47.724852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443636+0000","last_change":"2024-04-25T10:31:23.446467+0000","last_active":"2024-04-25T10:31:24.443636+0000","last_peered":"2024-04-25T10:31:24.443636+0000","last_clean":"2024-04-25T10:31:24.443636+0000","last_became_active":"2024-04-25T10:31:23.446185+0000","last_became_peered":"2024-04-25T10:31:23.446185+0000","last_unstale":"2024-04-25T10:31:24.443636+0000","last_undegraded":"2024-04-25T10:31:24.443636+0000","last_fullsized":"2024-04-25T10:31:24.443636+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:55.953997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437433+0000","last_change":"2024-04-25T10:31:23.432662+0000","last_active":"2024-04-25T10:31:24.437433+0000","last_peered":"2024-04-25T10:31:24.437433+0000","last_clean":"2024-04-25T10:31:24.437433+0000","last_became_active":"2024-04-25T10:31:23.432317+0000","last_became_peered":"2024-04-25T10:31:23.432317+0000","last_unstale":"2024-04-25T10:31:24.437433+0000","last_undegraded":"2024-04-25T10:31:24.437433+0000","last_fullsized":"2024-04-25T10:31:24.437433+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:01.330274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445557+0000","last_change":"2024-04-25T10:31:23.431527+0000","last_active":"2024-04-25T10:31:24.445557+0000","last_peered":"2024-04-25T10:31:24.445557+0000","last_clean":"2024-04-25T10:31:24.445557+0000","last_became_active":"2024-04-25T10:31:23.431141+0000","last_became_peered":"2024-04-25T10:31:23.431141+0000","last_unstale":"2024-04-25T10:31:24.445557+0000","last_undegraded":"2024-04-25T10:31:24.445557+0000","last_fullsized":"2024-04-25T10:31:24.445557+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438011+0000","last_change":"2024-04-25T10:31:23.441994+0000","last_active":"2024-04-25T10:31:24.438011+0000","last_peered":"2024-04-25T10:31:24.438011+0000","last_clean":"2024-04-25T10:31:24.438011+0000","last_became_active":"2024-04-25T10:31:23.439445+0000","last_became_peered":"2024-04-25T10:31:23.439445+0000","last_unstale":"2024-04-25T10:31:24.438011+0000","last_undegraded":"2024-04-25T10:31:24.438011+0000","last_fullsized":"2024-04-25T10:31:24.438011+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:12.920929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445013+0000","last_change":"2024-04-25T10:31:23.433270+0000","last_active":"2024-04-25T10:31:24.445013+0000","last_peered":"2024-04-25T10:31:24.445013+0000","last_clean":"2024-04-25T10:31:24.445013+0000","last_became_active":"2024-04-25T10:31:23.432966+0000","last_became_peered":"2024-04-25T10:31:23.432966+0000","last_unstale":"2024-04-25T10:31:24.445013+0000","last_undegraded":"2024-04-25T10:31:24.445013+0000","last_fullsized":"2024-04-25T10:31:24.445013+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:20.117231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892410+0000","last_change":"2024-04-25T10:31:23.458263+0000","last_active":"2024-04-25T10:31:24.892410+0000","last_peered":"2024-04-25T10:31:24.892410+0000","last_clean":"2024-04-25T10:31:24.892410+0000","last_became_active":"2024-04-25T10:31:23.457924+0000","last_became_peered":"2024-04-25T10:31:23.457924+0000","last_unstale":"2024-04-25T10:31:24.892410+0000","last_undegraded":"2024-04-25T10:31:24.892410+0000","last_fullsized":"2024-04-25T10:31:24.892410+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:50.981294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444516+0000","last_change":"2024-04-25T10:31:23.436457+0000","last_active":"2024-04-25T10:31:24.444516+0000","last_peered":"2024-04-25T10:31:24.444516+0000","last_clean":"2024-04-25T10:31:24.444516+0000","last_became_active":"2024-04-25T10:31:23.436186+0000","last_became_peered":"2024-04-25T10:31:23.436186+0000","last_unstale":"2024-04-25T10:31:24.444516+0000","last_undegraded":"2024-04-25T10:31:24.444516+0000","last_fullsized":"2024-04-25T10:31:24.444516+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:18.766534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443415+0000","last_change":"2024-04-25T10:31:23.435570+0000","last_active":"2024-04-25T10:31:24.443415+0000","last_peered":"2024-04-25T10:31:24.443415+0000","last_clean":"2024-04-25T10:31:24.443415+0000","last_became_active":"2024-04-25T10:31:23.435293+0000","last_became_peered":"2024-04-25T10:31:23.435293+0000","last_unstale":"2024-04-25T10:31:24.443415+0000","last_undegraded":"2024-04-25T10:31:24.443415+0000","last_fullsized":"2024-04-25T10:31:24.443415+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:21.689139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437501+0000","last_change":"2024-04-25T10:31:23.438686+0000","last_active":"2024-04-25T10:31:24.437501+0000","last_peered":"2024-04-25T10:31:24.437501+0000","last_clean":"2024-04-25T10:31:24.437501+0000","last_became_active":"2024-04-25T10:31:23.432709+0000","last_became_peered":"2024-04-25T10:31:23.432709+0000","last_unstale":"2024-04-25T10:31:24.437501+0000","last_undegraded":"2024-04-25T10:31:24.437501+0000","last_fullsized":"2024-04-25T10:31:24.437501+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:27.838602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894984+0000","last_change":"2024-04-25T10:31:23.443235+0000","last_active":"2024-04-25T10:31:24.894984+0000","last_peered":"2024-04-25T10:31:24.894984+0000","last_clean":"2024-04-25T10:31:24.894984+0000","last_became_active":"2024-04-25T10:31:23.442960+0000","last_became_peered":"2024-04-25T10:31:23.442960+0000","last_unstale":"2024-04-25T10:31:24.894984+0000","last_undegraded":"2024-04-25T10:31:24.894984+0000","last_fullsized":"2024-04-25T10:31:24.894984+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:31.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894425+0000","last_change":"2024-04-25T10:31:23.445902+0000","last_active":"2024-04-25T10:31:24.894425+0000","last_peered":"2024-04-25T10:31:24.894425+0000","last_clean":"2024-04-25T10:31:24.894425+0000","last_became_active":"2024-04-25T10:31:23.445617+0000","last_became_peered":"2024-04-25T10:31:23.445617+0000","last_unstale":"2024-04-25T10:31:24.894425+0000","last_undegraded":"2024-04-25T10:31:24.894425+0000","last_fullsized":"2024-04-25T10:31:24.894425+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:16.783049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892730+0000","last_change":"2024-04-25T10:31:23.456990+0000","last_active":"2024-04-25T10:31:24.892730+0000","last_peered":"2024-04-25T10:31:24.892730+0000","last_clean":"2024-04-25T10:31:24.892730+0000","last_became_active":"2024-04-25T10:31:23.451914+0000","last_became_peered":"2024-04-25T10:31:23.451914+0000","last_unstale":"2024-04-25T10:31:24.892730+0000","last_undegraded":"2024-04-25T10:31:24.892730+0000","last_fullsized":"2024-04-25T10:31:24.892730+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:03.703265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"181'4","reported_seq":18,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892722+0000","last_change":"2024-04-25T10:31:22.444200+0000","last_active":"2024-04-25T10:31:24.892722+0000","last_peered":"2024-04-25T10:31:24.892722+0000","last_clean":"2024-04-25T10:31:24.892722+0000","last_became_active":"2024-04-25T10:31:22.443896+0000","last_became_peered":"2024-04-25T10:31:22.443896+0000","last_unstale":"2024-04-25T10:31:24.892722+0000","last_undegraded":"2024-04-25T10:31:24.892722+0000","last_fullsized":"2024-04-25T10:31:24.892722+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:07.360121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438041+0000","last_change":"2024-04-25T10:31:23.438605+0000","last_active":"2024-04-25T10:31:24.438041+0000","last_peered":"2024-04-25T10:31:24.438041+0000","last_clean":"2024-04-25T10:31:24.438041+0000","last_became_active":"2024-04-25T10:31:23.437768+0000","last_became_peered":"2024-04-25T10:31:23.437768+0000","last_unstale":"2024-04-25T10:31:24.438041+0000","last_undegraded":"2024-04-25T10:31:24.438041+0000","last_fullsized":"2024-04-25T10:31:24.438041+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:08.369504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443500+0000","last_change":"2024-04-25T10:31:22.436396+0000","last_active":"2024-04-25T10:31:24.443500+0000","last_peered":"2024-04-25T10:31:24.443500+0000","last_clean":"2024-04-25T10:31:24.443500+0000","last_became_active":"2024-04-25T10:31:22.436121+0000","last_became_peered":"2024-04-25T10:31:22.436121+0000","last_unstale":"2024-04-25T10:31:24.443500+0000","last_undegraded":"2024-04-25T10:31:24.443500+0000","last_fullsized":"2024-04-25T10:31:24.443500+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:51.783124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895375+0000","last_change":"2024-04-25T10:31:23.457159+0000","last_active":"2024-04-25T10:31:24.895375+0000","last_peered":"2024-04-25T10:31:24.895375+0000","last_clean":"2024-04-25T10:31:24.895375+0000","last_became_active":"2024-04-25T10:31:23.456837+0000","last_became_peered":"2024-04-25T10:31:23.456837+0000","last_unstale":"2024-04-25T10:31:24.895375+0000","last_undegraded":"2024-04-25T10:31:24.895375+0000","last_fullsized":"2024-04-25T10:31:24.895375+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:10.584331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444666+0000","last_change":"2024-04-25T10:31:22.439739+0000","last_active":"2024-04-25T10:31:24.444666+0000","last_peered":"2024-04-25T10:31:24.444666+0000","last_clean":"2024-04-25T10:31:24.444666+0000","last_became_active":"2024-04-25T10:31:22.439422+0000","last_became_peered":"2024-04-25T10:31:22.439422+0000","last_unstale":"2024-04-25T10:31:24.444666+0000","last_undegraded":"2024-04-25T10:31:24.444666+0000","last_fullsized":"2024-04-25T10:31:24.444666+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:31.763486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895686+0000","last_change":"2024-04-25T10:31:23.456576+0000","last_active":"2024-04-25T10:31:24.895686+0000","last_peered":"2024-04-25T10:31:24.895686+0000","last_clean":"2024-04-25T10:31:24.895686+0000","last_became_active":"2024-04-25T10:31:23.455891+0000","last_became_peered":"2024-04-25T10:31:23.455891+0000","last_unstale":"2024-04-25T10:31:24.895686+0000","last_undegraded":"2024-04-25T10:31:24.895686+0000","last_fullsized":"2024-04-25T10:31:24.895686+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:17.415871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437782+0000","last_change":"2024-04-25T10:31:22.433482+0000","last_active":"2024-04-25T10:31:24.437782+0000","last_peered":"2024-04-25T10:31:24.437782+0000","last_clean":"2024-04-25T10:31:24.437782+0000","last_became_active":"2024-04-25T10:31:22.433201+0000","last_became_peered":"2024-04-25T10:31:22.433201+0000","last_unstale":"2024-04-25T10:31:24.437782+0000","last_undegraded":"2024-04-25T10:31:24.437782+0000","last_fullsized":"2024-04-25T10:31:24.437782+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:10.387788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438054+0000","last_change":"2024-04-25T10:31:23.452485+0000","last_active":"2024-04-25T10:31:24.438054+0000","last_peered":"2024-04-25T10:31:24.438054+0000","last_clean":"2024-04-25T10:31:24.438054+0000","last_became_active":"2024-04-25T10:31:23.452223+0000","last_became_peered":"2024-04-25T10:31:23.452223+0000","last_unstale":"2024-04-25T10:31:24.438054+0000","last_undegraded":"2024-04-25T10:31:24.438054+0000","last_fullsized":"2024-04-25T10:31:24.438054+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:19.478405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443707+0000","last_change":"2024-04-25T10:31:22.445426+0000","last_active":"2024-04-25T10:31:24.443707+0000","last_peered":"2024-04-25T10:31:24.443707+0000","last_clean":"2024-04-25T10:31:24.443707+0000","last_became_active":"2024-04-25T10:31:22.445054+0000","last_became_peered":"2024-04-25T10:31:22.445054+0000","last_unstale":"2024-04-25T10:31:24.443707+0000","last_undegraded":"2024-04-25T10:31:24.443707+0000","last_fullsized":"2024-04-25T10:31:24.443707+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443271+0000","last_change":"2024-04-25T10:31:23.434832+0000","last_active":"2024-04-25T10:31:24.443271+0000","last_peered":"2024-04-25T10:31:24.443271+0000","last_clean":"2024-04-25T10:31:24.443271+0000","last_became_active":"2024-04-25T10:31:23.433275+0000","last_became_peered":"2024-04-25T10:31:23.433275+0000","last_unstale":"2024-04-25T10:31:24.443271+0000","last_undegraded":"2024-04-25T10:31:24.443271+0000","last_fullsized":"2024-04-25T10:31:24.443271+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:23.154975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443864+0000","last_change":"2024-04-25T10:31:22.440188+0000","last_active":"2024-04-25T10:31:24.443864+0000","last_peered":"2024-04-25T10:31:24.443864+0000","last_clean":"2024-04-25T10:31:24.443864+0000","last_became_active":"2024-04-25T10:31:22.438568+0000","last_became_peered":"2024-04-25T10:31:22.438568+0000","last_unstale":"2024-04-25T10:31:24.443864+0000","last_undegraded":"2024-04-25T10:31:24.443864+0000","last_fullsized":"2024-04-25T10:31:24.443864+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:15.814748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445486+0000","last_change":"2024-04-25T10:31:23.445257+0000","last_active":"2024-04-25T10:31:24.445486+0000","last_peered":"2024-04-25T10:31:24.445486+0000","last_clean":"2024-04-25T10:31:24.445486+0000","last_became_active":"2024-04-25T10:31:23.444984+0000","last_became_peered":"2024-04-25T10:31:23.444984+0000","last_unstale":"2024-04-25T10:31:24.445486+0000","last_undegraded":"2024-04-25T10:31:24.445486+0000","last_fullsized":"2024-04-25T10:31:24.445486+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437364+0000","last_change":"2024-04-25T10:31:22.443402+0000","last_active":"2024-04-25T10:31:24.437364+0000","last_peered":"2024-04-25T10:31:24.437364+0000","last_clean":"2024-04-25T10:31:24.437364+0000","last_became_active":"2024-04-25T10:31:22.443034+0000","last_became_peered":"2024-04-25T10:31:22.443034+0000","last_unstale":"2024-04-25T10:31:24.437364+0000","last_undegraded":"2024-04-25T10:31:24.437364+0000","last_fullsized":"2024-04-25T10:31:24.437364+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:08.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437882+0000","last_change":"2024-04-25T10:31:23.432502+0000","last_active":"2024-04-25T10:31:24.437882+0000","last_peered":"2024-04-25T10:31:24.437882+0000","last_clean":"2024-04-25T10:31:24.437882+0000","last_became_active":"2024-04-25T10:31:23.430905+0000","last_became_peered":"2024-04-25T10:31:23.430905+0000","last_unstale":"2024-04-25T10:31:24.437882+0000","last_undegraded":"2024-04-25T10:31:24.437882+0000","last_fullsized":"2024-04-25T10:31:24.437882+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:24.427463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895537+0000","last_change":"2024-04-25T10:31:22.446818+0000","last_active":"2024-04-25T10:31:24.895537+0000","last_peered":"2024-04-25T10:31:24.895537+0000","last_clean":"2024-04-25T10:31:24.895537+0000","last_became_active":"2024-04-25T10:31:22.446198+0000","last_became_peered":"2024-04-25T10:31:22.446198+0000","last_unstale":"2024-04-25T10:31:24.895537+0000","last_undegraded":"2024-04-25T10:31:24.895537+0000","last_fullsized":"2024-04-25T10:31:24.895537+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:27.331081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894626+0000","last_change":"2024-04-25T10:31:23.460404+0000","last_active":"2024-04-25T10:31:24.894626+0000","last_peered":"2024-04-25T10:31:24.894626+0000","last_clean":"2024-04-25T10:31:24.894626+0000","last_became_active":"2024-04-25T10:31:23.459692+0000","last_became_peered":"2024-04-25T10:31:23.459692+0000","last_unstale":"2024-04-25T10:31:24.894626+0000","last_undegraded":"2024-04-25T10:31:24.894626+0000","last_fullsized":"2024-04-25T10:31:24.894626+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:32.716829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437905+0000","last_change":"2024-04-25T10:31:22.435228+0000","last_active":"2024-04-25T10:31:24.437905+0000","last_peered":"2024-04-25T10:31:24.437905+0000","last_clean":"2024-04-25T10:31:24.437905+0000","last_became_active":"2024-04-25T10:31:22.434923+0000","last_became_peered":"2024-04-25T10:31:22.434923+0000","last_unstale":"2024-04-25T10:31:24.437905+0000","last_undegraded":"2024-04-25T10:31:24.437905+0000","last_fullsized":"2024-04-25T10:31:24.437905+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:49.237314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445664+0000","last_change":"2024-04-25T10:31:23.433349+0000","last_active":"2024-04-25T10:31:24.445664+0000","last_peered":"2024-04-25T10:31:24.445664+0000","last_clean":"2024-04-25T10:31:24.445664+0000","last_became_active":"2024-04-25T10:31:23.433057+0000","last_became_peered":"2024-04-25T10:31:23.433057+0000","last_unstale":"2024-04-25T10:31:24.445664+0000","last_undegraded":"2024-04-25T10:31:24.445664+0000","last_fullsized":"2024-04-25T10:31:24.445664+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.381721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"181'3","reported_seq":17,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444682+0000","last_change":"2024-04-25T10:31:22.442622+0000","last_active":"2024-04-25T10:31:24.444682+0000","last_peered":"2024-04-25T10:31:24.444682+0000","last_clean":"2024-04-25T10:31:24.444682+0000","last_became_active":"2024-04-25T10:31:22.441086+0000","last_became_peered":"2024-04-25T10:31:22.441086+0000","last_unstale":"2024-04-25T10:31:24.444682+0000","last_undegraded":"2024-04-25T10:31:24.444682+0000","last_fullsized":"2024-04-25T10:31:24.444682+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:21.946296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444495+0000","last_change":"2024-04-25T10:31:23.448227+0000","last_active":"2024-04-25T10:31:24.444495+0000","last_peered":"2024-04-25T10:31:24.444495+0000","last_clean":"2024-04-25T10:31:24.444495+0000","last_became_active":"2024-04-25T10:31:23.447906+0000","last_became_peered":"2024-04-25T10:31:23.447906+0000","last_unstale":"2024-04-25T10:31:24.444495+0000","last_undegraded":"2024-04-25T10:31:24.444495+0000","last_fullsized":"2024-04-25T10:31:24.444495+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:23.339336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"181'6","reported_seq":20,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445723+0000","last_change":"2024-04-25T10:31:22.436482+0000","last_active":"2024-04-25T10:31:24.445723+0000","last_peered":"2024-04-25T10:31:24.445723+0000","last_clean":"2024-04-25T10:31:24.445723+0000","last_became_active":"2024-04-25T10:31:22.436192+0000","last_became_peered":"2024-04-25T10:31:22.436192+0000","last_unstale":"2024-04-25T10:31:24.445723+0000","last_undegraded":"2024-04-25T10:31:24.445723+0000","last_fullsized":"2024-04-25T10:31:24.445723+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894556+0000","last_change":"2024-04-25T10:31:23.448277+0000","last_active":"2024-04-25T10:31:24.894556+0000","last_peered":"2024-04-25T10:31:24.894556+0000","last_clean":"2024-04-25T10:31:24.894556+0000","last_became_active":"2024-04-25T10:31:23.447714+0000","last_became_peered":"2024-04-25T10:31:23.447714+0000","last_unstale":"2024-04-25T10:31:24.894556+0000","last_undegraded":"2024-04-25T10:31:24.894556+0000","last_fullsized":"2024-04-25T10:31:24.894556+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:34.299707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445556+0000","last_change":"2024-04-25T10:31:22.437143+0000","last_active":"2024-04-25T10:31:24.445556+0000","last_peered":"2024-04-25T10:31:24.445556+0000","last_clean":"2024-04-25T10:31:24.445556+0000","last_became_active":"2024-04-25T10:31:22.436870+0000","last_became_peered":"2024-04-25T10:31:22.436870+0000","last_unstale":"2024-04-25T10:31:24.445556+0000","last_undegraded":"2024-04-25T10:31:24.445556+0000","last_fullsized":"2024-04-25T10:31:24.445556+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:15.773667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895631+0000","last_change":"2024-04-25T10:31:23.452399+0000","last_active":"2024-04-25T10:31:24.895631+0000","last_peered":"2024-04-25T10:31:24.895631+0000","last_clean":"2024-04-25T10:31:24.895631+0000","last_became_active":"2024-04-25T10:31:23.451792+0000","last_became_peered":"2024-04-25T10:31:23.451792+0000","last_unstale":"2024-04-25T10:31:24.895631+0000","last_undegraded":"2024-04-25T10:31:24.895631+0000","last_fullsized":"2024-04-25T10:31:24.895631+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:32.246063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443413+0000","last_change":"2024-04-25T10:31:22.435264+0000","last_active":"2024-04-25T10:31:24.443413+0000","last_peered":"2024-04-25T10:31:24.443413+0000","last_clean":"2024-04-25T10:31:24.443413+0000","last_became_active":"2024-04-25T10:31:22.434517+0000","last_became_peered":"2024-04-25T10:31:22.434517+0000","last_unstale":"2024-04-25T10:31:24.443413+0000","last_undegraded":"2024-04-25T10:31:24.443413+0000","last_fullsized":"2024-04-25T10:31:24.443413+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.176802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894965+0000","last_change":"2024-04-25T10:31:23.452667+0000","last_active":"2024-04-25T10:31:24.894965+0000","last_peered":"2024-04-25T10:31:24.894965+0000","last_clean":"2024-04-25T10:31:24.894965+0000","last_became_active":"2024-04-25T10:31:23.452366+0000","last_became_peered":"2024-04-25T10:31:23.452366+0000","last_unstale":"2024-04-25T10:31:24.894965+0000","last_undegraded":"2024-04-25T10:31:24.894965+0000","last_fullsized":"2024-04-25T10:31:24.894965+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:47.741845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443896+0000","last_change":"2024-04-25T10:31:22.441210+0000","last_active":"2024-04-25T10:31:24.443896+0000","last_peered":"2024-04-25T10:31:24.443896+0000","last_clean":"2024-04-25T10:31:24.443896+0000","last_became_active":"2024-04-25T10:31:22.440936+0000","last_became_peered":"2024-04-25T10:31:22.440936+0000","last_unstale":"2024-04-25T10:31:24.443896+0000","last_undegraded":"2024-04-25T10:31:24.443896+0000","last_fullsized":"2024-04-25T10:31:24.443896+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:08.832902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443141+0000","last_change":"2024-04-25T10:31:23.446637+0000","last_active":"2024-04-25T10:31:24.443141+0000","last_peered":"2024-04-25T10:31:24.443141+0000","last_clean":"2024-04-25T10:31:24.443141+0000","last_became_active":"2024-04-25T10:31:23.445987+0000","last_became_peered":"2024-04-25T10:31:23.445987+0000","last_unstale":"2024-04-25T10:31:24.443141+0000","last_undegraded":"2024-04-25T10:31:24.443141+0000","last_fullsized":"2024-04-25T10:31:24.443141+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:50.244723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"181'8","reported_seq":22,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443609+0000","last_change":"2024-04-25T10:31:22.436186+0000","last_active":"2024-04-25T10:31:24.443609+0000","last_peered":"2024-04-25T10:31:24.443609+0000","last_clean":"2024-04-25T10:31:24.443609+0000","last_became_active":"2024-04-25T10:31:22.435837+0000","last_became_peered":"2024-04-25T10:31:22.435837+0000","last_unstale":"2024-04-25T10:31:24.443609+0000","last_undegraded":"2024-04-25T10:31:24.443609+0000","last_fullsized":"2024-04-25T10:31:24.443609+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:25.915093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892796+0000","last_change":"2024-04-25T10:31:23.455136+0000","last_active":"2024-04-25T10:31:24.892796+0000","last_peered":"2024-04-25T10:31:24.892796+0000","last_clean":"2024-04-25T10:31:24.892796+0000","last_became_active":"2024-04-25T10:31:23.454870+0000","last_became_peered":"2024-04-25T10:31:23.454870+0000","last_unstale":"2024-04-25T10:31:24.892796+0000","last_undegraded":"2024-04-25T10:31:24.892796+0000","last_fullsized":"2024-04-25T10:31:24.892796+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:09.412305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444711+0000","last_change":"2024-04-25T10:31:22.445257+0000","last_active":"2024-04-25T10:31:24.444711+0000","last_peered":"2024-04-25T10:31:24.444711+0000","last_clean":"2024-04-25T10:31:24.444711+0000","last_became_active":"2024-04-25T10:31:22.444918+0000","last_became_peered":"2024-04-25T10:31:22.444918+0000","last_unstale":"2024-04-25T10:31:24.444711+0000","last_undegraded":"2024-04-25T10:31:24.444711+0000","last_fullsized":"2024-04-25T10:31:24.444711+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:33.290324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.438159+0000","last_change":"2024-04-25T10:31:23.453329+0000","last_active":"2024-04-25T10:31:24.438159+0000","last_peered":"2024-04-25T10:31:24.438159+0000","last_clean":"2024-04-25T10:31:24.438159+0000","last_became_active":"2024-04-25T10:31:23.452956+0000","last_became_peered":"2024-04-25T10:31:23.452956+0000","last_unstale":"2024-04-25T10:31:24.438159+0000","last_undegraded":"2024-04-25T10:31:24.438159+0000","last_fullsized":"2024-04-25T10:31:24.438159+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:11.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.445770+0000","last_change":"2024-04-25T10:31:22.436617+0000","last_active":"2024-04-25T10:31:24.445770+0000","last_peered":"2024-04-25T10:31:24.445770+0000","last_clean":"2024-04-25T10:31:24.445770+0000","last_became_active":"2024-04-25T10:31:22.436313+0000","last_became_peered":"2024-04-25T10:31:22.436313+0000","last_unstale":"2024-04-25T10:31:24.445770+0000","last_undegraded":"2024-04-25T10:31:24.445770+0000","last_fullsized":"2024-04-25T10:31:24.445770+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895606+0000","last_change":"2024-04-25T10:31:23.455754+0000","last_active":"2024-04-25T10:31:24.895606+0000","last_peered":"2024-04-25T10:31:24.895606+0000","last_clean":"2024-04-25T10:31:24.895606+0000","last_became_active":"2024-04-25T10:31:23.447211+0000","last_became_peered":"2024-04-25T10:31:23.447211+0000","last_unstale":"2024-04-25T10:31:24.895606+0000","last_undegraded":"2024-04-25T10:31:24.895606+0000","last_fullsized":"2024-04-25T10:31:24.895606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:28.103406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894782+0000","last_change":"2024-04-25T10:31:22.444016+0000","last_active":"2024-04-25T10:31:24.894782+0000","last_peered":"2024-04-25T10:31:24.894782+0000","last_clean":"2024-04-25T10:31:24.894782+0000","last_became_active":"2024-04-25T10:31:22.443711+0000","last_became_peered":"2024-04-25T10:31:22.443711+0000","last_unstale":"2024-04-25T10:31:24.894782+0000","last_undegraded":"2024-04-25T10:31:24.894782+0000","last_fullsized":"2024-04-25T10:31:24.894782+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:28.985863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443324+0000","last_change":"2024-04-25T10:31:23.434350+0000","last_active":"2024-04-25T10:31:24.443324+0000","last_peered":"2024-04-25T10:31:24.443324+0000","last_clean":"2024-04-25T10:31:24.443324+0000","last_became_active":"2024-04-25T10:31:23.434029+0000","last_became_peered":"2024-04-25T10:31:23.434029+0000","last_unstale":"2024-04-25T10:31:24.443324+0000","last_undegraded":"2024-04-25T10:31:24.443324+0000","last_fullsized":"2024-04-25T10:31:24.443324+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:47.857879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442891+0000","last_change":"2024-04-25T10:31:22.438645+0000","last_active":"2024-04-25T10:31:24.442891+0000","last_peered":"2024-04-25T10:31:24.442891+0000","last_clean":"2024-04-25T10:31:24.442891+0000","last_became_active":"2024-04-25T10:31:22.438312+0000","last_became_peered":"2024-04-25T10:31:22.438312+0000","last_unstale":"2024-04-25T10:31:24.442891+0000","last_undegraded":"2024-04-25T10:31:24.442891+0000","last_fullsized":"2024-04-25T10:31:24.442891+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:18.769658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895617+0000","last_change":"2024-04-25T10:31:23.450857+0000","last_active":"2024-04-25T10:31:24.895617+0000","last_peered":"2024-04-25T10:31:24.895617+0000","last_clean":"2024-04-25T10:31:24.895617+0000","last_became_active":"2024-04-25T10:31:23.449482+0000","last_became_peered":"2024-04-25T10:31:23.449482+0000","last_unstale":"2024-04-25T10:31:24.895617+0000","last_undegraded":"2024-04-25T10:31:24.895617+0000","last_fullsized":"2024-04-25T10:31:24.895617+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:28.466968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.894371+0000","last_change":"2024-04-25T10:31:22.439123+0000","last_active":"2024-04-25T10:31:24.894371+0000","last_peered":"2024-04-25T10:31:24.894371+0000","last_clean":"2024-04-25T10:31:24.894371+0000","last_became_active":"2024-04-25T10:31:22.438833+0000","last_became_peered":"2024-04-25T10:31:22.438833+0000","last_unstale":"2024-04-25T10:31:24.894371+0000","last_undegraded":"2024-04-25T10:31:24.894371+0000","last_fullsized":"2024-04-25T10:31:24.894371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:46.715469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443231+0000","last_change":"2024-04-25T10:31:23.444511+0000","last_active":"2024-04-25T10:31:24.443231+0000","last_peered":"2024-04-25T10:31:24.443231+0000","last_clean":"2024-04-25T10:31:24.443231+0000","last_became_active":"2024-04-25T10:31:23.444242+0000","last_became_peered":"2024-04-25T10:31:23.444242+0000","last_unstale":"2024-04-25T10:31:24.443231+0000","last_undegraded":"2024-04-25T10:31:24.443231+0000","last_fullsized":"2024-04-25T10:31:24.443231+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:50.976811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444346+0000","last_change":"2024-04-25T10:31:22.438630+0000","last_active":"2024-04-25T10:31:24.444346+0000","last_peered":"2024-04-25T10:31:24.444346+0000","last_clean":"2024-04-25T10:31:24.444346+0000","last_became_active":"2024-04-25T10:31:22.438311+0000","last_became_peered":"2024-04-25T10:31:22.438311+0000","last_unstale":"2024-04-25T10:31:24.444346+0000","last_undegraded":"2024-04-25T10:31:24.444346+0000","last_fullsized":"2024-04-25T10:31:24.444346+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.394944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437306+0000","last_change":"2024-04-25T10:31:23.431648+0000","last_active":"2024-04-25T10:31:24.437306+0000","last_peered":"2024-04-25T10:31:24.437306+0000","last_clean":"2024-04-25T10:31:24.437306+0000","last_became_active":"2024-04-25T10:31:23.430571+0000","last_became_peered":"2024-04-25T10:31:23.430571+0000","last_unstale":"2024-04-25T10:31:24.437306+0000","last_undegraded":"2024-04-25T10:31:24.437306+0000","last_fullsized":"2024-04-25T10:31:24.437306+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:58.693323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895010+0000","last_change":"2024-04-25T10:31:22.446969+0000","last_active":"2024-04-25T10:31:24.895010+0000","last_peered":"2024-04-25T10:31:24.895010+0000","last_clean":"2024-04-25T10:31:24.895010+0000","last_became_active":"2024-04-25T10:31:22.446445+0000","last_became_peered":"2024-04-25T10:31:22.446445+0000","last_unstale":"2024-04-25T10:31:24.895010+0000","last_undegraded":"2024-04-25T10:31:24.895010+0000","last_fullsized":"2024-04-25T10:31:24.895010+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.738856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437617+0000","last_change":"2024-04-25T10:31:23.442351+0000","last_active":"2024-04-25T10:31:24.437617+0000","last_peered":"2024-04-25T10:31:24.437617+0000","last_clean":"2024-04-25T10:31:24.437617+0000","last_became_active":"2024-04-25T10:31:23.442056+0000","last_became_peered":"2024-04-25T10:31:23.442056+0000","last_unstale":"2024-04-25T10:31:24.437617+0000","last_undegraded":"2024-04-25T10:31:24.437617+0000","last_fullsized":"2024-04-25T10:31:24.437617+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:42.792189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443727+0000","last_change":"2024-04-25T10:31:22.443307+0000","last_active":"2024-04-25T10:31:24.443727+0000","last_peered":"2024-04-25T10:31:24.443727+0000","last_clean":"2024-04-25T10:31:24.443727+0000","last_became_active":"2024-04-25T10:31:22.442920+0000","last_became_peered":"2024-04-25T10:31:22.442920+0000","last_unstale":"2024-04-25T10:31:24.443727+0000","last_undegraded":"2024-04-25T10:31:24.443727+0000","last_fullsized":"2024-04-25T10:31:24.443727+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:44.842685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892820+0000","last_change":"2024-04-25T10:31:23.444361+0000","last_active":"2024-04-25T10:31:24.892820+0000","last_peered":"2024-04-25T10:31:24.892820+0000","last_clean":"2024-04-25T10:31:24.892820+0000","last_became_active":"2024-04-25T10:31:23.444080+0000","last_became_peered":"2024-04-25T10:31:23.444080+0000","last_unstale":"2024-04-25T10:31:24.892820+0000","last_undegraded":"2024-04-25T10:31:24.892820+0000","last_fullsized":"2024-04-25T10:31:24.892820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:05.685411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"181'2","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444937+0000","last_change":"2024-04-25T10:31:22.436880+0000","last_active":"2024-04-25T10:31:24.444937+0000","last_peered":"2024-04-25T10:31:24.444937+0000","last_clean":"2024-04-25T10:31:24.444937+0000","last_became_active":"2024-04-25T10:31:22.436570+0000","last_became_peered":"2024-04-25T10:31:22.436570+0000","last_unstale":"2024-04-25T10:31:24.444937+0000","last_undegraded":"2024-04-25T10:31:24.444937+0000","last_fullsized":"2024-04-25T10:31:24.444937+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:24.968695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.444606+0000","last_change":"2024-04-25T10:31:23.449985+0000","last_active":"2024-04-25T10:31:24.444606+0000","last_peered":"2024-04-25T10:31:24.444606+0000","last_clean":"2024-04-25T10:31:24.444606+0000","last_became_active":"2024-04-25T10:31:23.449345+0000","last_became_peered":"2024-04-25T10:31:23.449345+0000","last_unstale":"2024-04-25T10:31:24.444606+0000","last_undegraded":"2024-04-25T10:31:24.444606+0000","last_fullsized":"2024-04-25T10:31:24.444606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:12.368876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"182'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443663+0000","last_change":"2024-04-25T10:31:22.440950+0000","last_active":"2024-04-25T10:31:24.443663+0000","last_peered":"2024-04-25T10:31:24.443663+0000","last_clean":"2024-04-25T10:31:24.443663+0000","last_became_active":"2024-04-25T10:31:22.440664+0000","last_became_peered":"2024-04-25T10:31:22.440664+0000","last_unstale":"2024-04-25T10:31:24.443663+0000","last_undegraded":"2024-04-25T10:31:24.443663+0000","last_fullsized":"2024-04-25T10:31:24.443663+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:21.897192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443530+0000","last_change":"2024-04-25T10:31:23.448149+0000","last_active":"2024-04-25T10:31:24.443530+0000","last_peered":"2024-04-25T10:31:24.443530+0000","last_clean":"2024-04-25T10:31:24.443530+0000","last_became_active":"2024-04-25T10:31:23.445318+0000","last_became_peered":"2024-04-25T10:31:23.445318+0000","last_unstale":"2024-04-25T10:31:24.443530+0000","last_undegraded":"2024-04-25T10:31:24.443530+0000","last_fullsized":"2024-04-25T10:31:24.443530+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:14.838094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443730+0000","last_change":"2024-04-25T10:31:22.446060+0000","last_active":"2024-04-25T10:31:24.443730+0000","last_peered":"2024-04-25T10:31:24.443730+0000","last_clean":"2024-04-25T10:31:24.443730+0000","last_became_active":"2024-04-25T10:31:22.445706+0000","last_became_peered":"2024-04-25T10:31:22.445706+0000","last_unstale":"2024-04-25T10:31:24.443730+0000","last_undegraded":"2024-04-25T10:31:24.443730+0000","last_fullsized":"2024-04-25T10:31:24.443730+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:31.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892620+0000","last_change":"2024-04-25T10:31:23.453869+0000","last_active":"2024-04-25T10:31:24.892620+0000","last_peered":"2024-04-25T10:31:24.892620+0000","last_clean":"2024-04-25T10:31:24.892620+0000","last_became_active":"2024-04-25T10:31:23.453600+0000","last_became_peered":"2024-04-25T10:31:23.453600+0000","last_unstale":"2024-04-25T10:31:24.892620+0000","last_undegraded":"2024-04-25T10:31:24.892620+0000","last_fullsized":"2024-04-25T10:31:24.892620+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:47.113234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443689+0000","last_change":"2024-04-25T10:31:22.445692+0000","last_active":"2024-04-25T10:31:24.443689+0000","last_peered":"2024-04-25T10:31:24.443689+0000","last_clean":"2024-04-25T10:31:24.443689+0000","last_became_active":"2024-04-25T10:31:22.445368+0000","last_became_peered":"2024-04-25T10:31:22.445368+0000","last_unstale":"2024-04-25T10:31:24.443689+0000","last_undegraded":"2024-04-25T10:31:24.443689+0000","last_fullsized":"2024-04-25T10:31:24.443689+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:07.812518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443603+0000","last_change":"2024-04-25T10:31:23.434539+0000","last_active":"2024-04-25T10:31:24.443603+0000","last_peered":"2024-04-25T10:31:24.443603+0000","last_clean":"2024-04-25T10:31:24.443603+0000","last_became_active":"2024-04-25T10:31:23.434040+0000","last_became_peered":"2024-04-25T10:31:23.434040+0000","last_unstale":"2024-04-25T10:31:24.443603+0000","last_undegraded":"2024-04-25T10:31:24.443603+0000","last_fullsized":"2024-04-25T10:31:24.443603+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:58.561747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437773+0000","last_change":"2024-04-25T10:31:22.431495+0000","last_active":"2024-04-25T10:31:24.437773+0000","last_peered":"2024-04-25T10:31:24.437773+0000","last_clean":"2024-04-25T10:31:24.437773+0000","last_became_active":"2024-04-25T10:31:22.431217+0000","last_became_peered":"2024-04-25T10:31:22.431217+0000","last_unstale":"2024-04-25T10:31:24.437773+0000","last_undegraded":"2024-04-25T10:31:24.437773+0000","last_fullsized":"2024-04-25T10:31:24.437773+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:27.633982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443417+0000","last_change":"2024-04-25T10:31:23.445933+0000","last_active":"2024-04-25T10:31:24.443417+0000","last_peered":"2024-04-25T10:31:24.443417+0000","last_clean":"2024-04-25T10:31:24.443417+0000","last_became_active":"2024-04-25T10:31:23.445660+0000","last_became_peered":"2024-04-25T10:31:23.445660+0000","last_unstale":"2024-04-25T10:31:24.443417+0000","last_undegraded":"2024-04-25T10:31:24.443417+0000","last_fullsized":"2024-04-25T10:31:24.443417+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:09.073459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895310+0000","last_change":"2024-04-25T10:31:22.445594+0000","last_active":"2024-04-25T10:31:24.895310+0000","last_peered":"2024-04-25T10:31:24.895310+0000","last_clean":"2024-04-25T10:31:24.895310+0000","last_became_active":"2024-04-25T10:31:22.445288+0000","last_became_peered":"2024-04-25T10:31:22.445288+0000","last_unstale":"2024-04-25T10:31:24.895310+0000","last_undegraded":"2024-04-25T10:31:24.895310+0000","last_fullsized":"2024-04-25T10:31:24.895310+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:54.229800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.442850+0000","last_change":"2024-04-25T10:31:23.445663+0000","last_active":"2024-04-25T10:31:24.442850+0000","last_peered":"2024-04-25T10:31:24.442850+0000","last_clean":"2024-04-25T10:31:24.442850+0000","last_became_active":"2024-04-25T10:31:23.445361+0000","last_became_peered":"2024-04-25T10:31:23.445361+0000","last_unstale":"2024-04-25T10:31:24.442850+0000","last_undegraded":"2024-04-25T10:31:24.442850+0000","last_fullsized":"2024-04-25T10:31:24.442850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:12.695852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.892351+0000","last_change":"2024-04-25T10:31:22.440617+0000","last_active":"2024-04-25T10:31:24.892351+0000","last_peered":"2024-04-25T10:31:24.892351+0000","last_clean":"2024-04-25T10:31:24.892351+0000","last_became_active":"2024-04-25T10:31:22.440345+0000","last_became_peered":"2024-04-25T10:31:22.440345+0000","last_unstale":"2024-04-25T10:31:24.892351+0000","last_undegraded":"2024-04-25T10:31:24.892351+0000","last_fullsized":"2024-04-25T10:31:24.892351+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:11.911547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437625+0000","last_change":"2024-04-25T10:31:23.443024+0000","last_active":"2024-04-25T10:31:24.437625+0000","last_peered":"2024-04-25T10:31:24.437625+0000","last_clean":"2024-04-25T10:31:24.437625+0000","last_became_active":"2024-04-25T10:31:23.442748+0000","last_became_peered":"2024-04-25T10:31:23.442748+0000","last_unstale":"2024-04-25T10:31:24.437625+0000","last_undegraded":"2024-04-25T10:31:24.437625+0000","last_fullsized":"2024-04-25T10:31:24.437625+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.154345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437413+0000","last_change":"2024-04-25T10:31:22.441054+0000","last_active":"2024-04-25T10:31:24.437413+0000","last_peered":"2024-04-25T10:31:24.437413+0000","last_clean":"2024-04-25T10:31:24.437413+0000","last_became_active":"2024-04-25T10:31:22.440642+0000","last_became_peered":"2024-04-25T10:31:22.440642+0000","last_unstale":"2024-04-25T10:31:24.437413+0000","last_undegraded":"2024-04-25T10:31:24.437413+0000","last_fullsized":"2024-04-25T10:31:24.437413+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.280878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.895180+0000","last_change":"2024-04-25T10:31:23.446822+0000","last_active":"2024-04-25T10:31:24.895180+0000","last_peered":"2024-04-25T10:31:24.895180+0000","last_clean":"2024-04-25T10:31:24.895180+0000","last_became_active":"2024-04-25T10:31:23.446539+0000","last_became_peered":"2024-04-25T10:31:23.446539+0000","last_unstale":"2024-04-25T10:31:24.895180+0000","last_undegraded":"2024-04-25T10:31:24.895180+0000","last_fullsized":"2024-04-25T10:31:24.895180+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:22.410310+0000","last_clean_scrub_stamp":"2024-04-25T10:31:22.410310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:45.001402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"181'1","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.443530+0000","last_change":"2024-04-25T10:31:22.436501+0000","last_active":"2024-04-25T10:31:24.443530+0000","last_peered":"2024-04-25T10:31:24.443530+0000","last_clean":"2024-04-25T10:31:24.443530+0000","last_became_active":"2024-04-25T10:31:22.436167+0000","last_became_peered":"2024-04-25T10:31:22.436167+0000","last_unstale":"2024-04-25T10:31:24.443530+0000","last_undegraded":"2024-04-25T10:31:24.443530+0000","last_fullsized":"2024-04-25T10:31:24.443530+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:21.400928+0000","last_clean_scrub_stamp":"2024-04-25T10:31:21.400928+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:25.310017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":480,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:31:24.437480+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:24.437480+0000","last_peered":"2024-04-25T10:31:24.437480+0000","last_clean":"2024-04-25T10:31:24.437480+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:24.437480+0000","last_undegraded":"2024-04-25T10:31:24.437480+0000","last_fullsized":"2024-04-25T10:31:24.437480+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509607,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55628,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316212,"statfs":{"total":96636764160,"available":96579801088,"internally_reserved":0,"allocated":2289664,"data_stored":1004347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509608,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55624,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316216,"statfs":{"total":96636764160,"available":96579805184,"internally_reserved":0,"allocated":2289664,"data_stored":1004395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509607,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60376,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311464,"statfs":{"total":96636764160,"available":96574939136,"internally_reserved":0,"allocated":2895872,"data_stored":1595421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509607,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47316,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324524,"statfs":{"total":96636764160,"available":96588312576,"internally_reserved":0,"allocated":2297856,"data_stored":1005937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509607,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69472,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302368,"statfs":{"total":96636764160,"available":96565624832,"internally_reserved":0,"allocated":2306048,"data_stored":1006041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509608,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":2812,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":2879488,"data_stored":1594693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509608,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59784,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312056,"statfs":{"total":96636764160,"available":96575545344,"internally_reserved":0,"allocated":2289664,"data_stored":1004907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542310,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69452,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302388,"statfs":{"total":96636764160,"available":96565645312,"internally_reserved":0,"allocated":2289664,"data_stored":1004415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:31:32.486 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:31:32.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:31:32.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.670+0000 7f1379600640 1 Processor -- start 2024-04-25T10:31:32.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.670+0000 7f1379600640 1 -- start start 2024-04-25T10:31:32.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1379600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13740af7c0 0x7f1374071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1379600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740670b0 0x7f13741618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1379600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137415e3c0 con 0x7f137415b800 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1379600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13740afc50 con 0x7f13740af7c0 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1379600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137415ecf0 con 0x7f13740670b0 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13740af7c0 0x7f1374071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1373400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740670b0 0x7f13741618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13740af7c0 0x7f1374071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:55576/0 (socket says 172.21.15.45:55576) 2024-04-25T10:31:32.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 -- 172.21.15.45:0/1208262625 learned_addr learned my addr 172.21.15.45:0/1208262625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:32.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f1372a00640 1 --1- 172.21.15.45:0/1208262625 >> v1:172.21.15.117:6789/0 conn(0x7f137415b800 0x7f13740669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:36808/0 (socket says 172.21.15.45:36808) 2024-04-25T10:31:32.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 -- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740670b0 msgr2=0x7f13741618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:32.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 --2- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740670b0 0x7f13741618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 -- 172.21.15.45:0/1208262625 >> v1:172.21.15.117:6789/0 conn(0x7f137415b800 legacy=0x7f13740669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.674+0000 7f136be00640 1 -- 172.21.15.45:0/1208262625 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137415e6a0 con 0x7f13740af7c0 2024-04-25T10:31:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f136be00640 1 --2- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740af7c0 0x7f1374071e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f135c009870 tx=0x7f135c01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a511e387df4b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1372000640 1 -- 172.21.15.45:0/1208262625 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c0a9610 con 0x7f13740af7c0 2024-04-25T10:31:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1372000640 1 -- 172.21.15.45:0/1208262625 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c0a97b0 con 0x7f13740af7c0 2024-04-25T10:31:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1372000640 1 -- 172.21.15.45:0/1208262625 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c0a9b80 con 0x7f13740af7c0 2024-04-25T10:31:32.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 -- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740af7c0 msgr2=0x7f1374071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 --2- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740af7c0 0x7f1374071e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f135c009870 tx=0x7f135c01c530 comp rx=0 tx=0).stop 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 -- 172.21.15.45:0/1208262625 shutdown_connections 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 --2- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740670b0 0x7f13741618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 --2- 172.21.15.45:0/1208262625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740af7c0 0x7f1374071e20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 -- 172.21.15.45:0/1208262625 >> 172.21.15.45:0/1208262625 conn(0x7f13740d0820 msgr2=0x7f13740d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 -- 172.21.15.45:0/1208262625 shutdown_connections 2024-04-25T10:31:32.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 -- 172.21.15.45:0/1208262625 wait complete. 2024-04-25T10:31:32.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 Processor -- start 2024-04-25T10:31:32.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.678+0000 7f1379600640 1 -- start start 2024-04-25T10:31:32.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740670b0 0x7f13741384b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 0x7f1374150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740cea00 0x7f13740cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137416ac40 con 0x7f137415b800 2024-04-25T10:31:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137415bbe0 con 0x7f13740670b0 2024-04-25T10:31:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137416adc0 con 0x7f13740cea00 2024-04-25T10:31:32.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 0x7f1374150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 0x7f1374150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57658/0 (socket says 172.21.15.45:57658) 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 -- 172.21.15.45:0/652268710 learned_addr learned my addr 172.21.15.45:0/652268710 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1372a00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740670b0 0x7f13741384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1373400640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740cea00 0x7f13740cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 -- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740cea00 msgr2=0x7f13740cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740cea00 0x7f13740cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 -- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740670b0 msgr2=0x7f13741384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740670b0 0x7f13741384b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 -- 172.21.15.45:0/652268710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374153cd0 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136be00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 0x7f1374150e20 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ab040 tx=0x7f135c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c0bc650 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c0bc7f0 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c0bcbe0 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1374152c30 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13741531f0 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137404ef30 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.682+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f135c0a9630 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.686+0000 7f136aa00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.686+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f135c11f390 con 0x7f137415b800 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.686+0000 7f1372a00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:32.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.686+0000 7f1372a00640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1364000f70 tx=0x7f1364007e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:32.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.690+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f135c0c3030 con 0x7f137415b800 2024-04-25T10:31:32.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.906+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f13740ca170 con 0x7f137415b800 2024-04-25T10:31:32.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.910+0000 7f136aa00640 1 -- 172.21.15.45:0/652268710 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f135c0ad020 con 0x7f137415b800 2024-04-25T10:31:32.913 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:31:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:31:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:31:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:31:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:31:32.914 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:31:32.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.910+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 msgr2=0x7f1350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.910+0000 7f1379600640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1364000f70 tx=0x7f1364007e90 comp rx=0 tx=0).stop 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.910+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 msgr2=0x7f1374150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 0x7f1374150e20 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ab040 tx=0x7f135c01ad70 comp rx=0 tx=0).stop 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 shutdown_connections 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1350066f60 0x7f1350069420 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13740cea00 0x7f13740cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f137415b800 0x7f1374150e20 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 --2- 172.21.15.45:0/652268710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13740670b0 0x7f13741384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 >> 172.21.15.45:0/652268710 conn(0x7f13740d0820 msgr2=0x7f137406d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 shutdown_connections 2024-04-25T10:31:32.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:32.914+0000 7f1379600640 1 -- 172.21.15.45:0/652268710 wait complete. 2024-04-25T10:31:32.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf45a00640 1 Processor -- start 2024-04-25T10:31:33.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf45a00640 1 -- start start 2024-04-25T10:31:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf45a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf4015bc10 0x7faf400cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf45a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4015bff0 con 0x7faf400ccd40 2024-04-25T10:31:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf45a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf400cd550 con 0x7faf400d0440 2024-04-25T10:31:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf45a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf400b0da0 con 0x7faf4015bc10 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf3fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf4015bc10 0x7faf400cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf37400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faf400ccd40 0x7faf400cfd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55710/0 (socket says 172.21.15.45:55710) 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf37400640 1 -- 172.21.15.45:0/3608602148 learned_addr learned my addr 172.21.15.45:0/3608602148 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf3fe00640 1 -- 172.21.15.45:0/3608602148 >> v1:172.21.15.117:6789/0 conn(0x7faf400d0440 legacy=0x7faf400b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf3fe00640 1 -- 172.21.15.45:0/3608602148 >> v1:172.21.15.45:6789/0 conn(0x7faf400ccd40 legacy=0x7faf400cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf3fe00640 1 -- 172.21.15.45:0/3608602148 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400b1080 con 0x7faf4015bc10 2024-04-25T10:31:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.074+0000 7faf3f400640 1 -- 172.21.15.45:0/3608602148 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844235968 0 0) 0x7faf4015bff0 con 0x7faf400ccd40 2024-04-25T10:31:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf3fe00640 1 --2- 172.21.15.45:0/3608602148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4015bc10 0x7faf400cf760 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7faf300097f0 tx=0x7faf3001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae1239f3a50b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf3f400640 1 -- 172.21.15.45:0/3608602148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300ab680 con 0x7faf4015bc10 2024-04-25T10:31:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf3f400640 1 -- 172.21.15.45:0/3608602148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300b0070 con 0x7faf4015bc10 2024-04-25T10:31:33.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf3f400640 1 -- 172.21.15.45:0/3608602148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300aba40 con 0x7faf4015bc10 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 -- 172.21.15.45:0/3608602148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4015bc10 msgr2=0x7faf400cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 --2- 172.21.15.45:0/3608602148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4015bc10 0x7faf400cf760 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7faf300097f0 tx=0x7faf3001c460 comp rx=0 tx=0).stop 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 -- 172.21.15.45:0/3608602148 shutdown_connections 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 --2- 172.21.15.45:0/3608602148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4015bc10 0x7faf400cf760 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 -- 172.21.15.45:0/3608602148 >> 172.21.15.45:0/3608602148 conn(0x7faf400d1750 msgr2=0x7faf400d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 -- 172.21.15.45:0/3608602148 shutdown_connections 2024-04-25T10:31:33.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 -- 172.21.15.45:0/3608602148 wait complete. 2024-04-25T10:31:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 Processor -- start 2024-04-25T10:31:33.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.078+0000 7faf45a00640 1 -- start start 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf400ccd40 0x7faf400cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 0x7faf40156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4015bc10 0x7faf401574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400b2bb0 con 0x7faf400d0440 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf400b2a30 con 0x7faf4015bc10 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf40169770 con 0x7faf400ccd40 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 0x7faf40156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 0x7faf40156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57680/0 (socket says 172.21.15.45:57680) 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 -- 172.21.15.45:0/3106148293 learned_addr learned my addr 172.21.15.45:0/3106148293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:33.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf3fe00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf400ccd40 0x7faf400cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45000640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4015bc10 0x7faf401574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 -- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf400ccd40 msgr2=0x7faf400cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf400ccd40 0x7faf400cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 -- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4015bc10 msgr2=0x7faf401574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4015bc10 0x7faf401574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 -- 172.21.15.45:0/3106148293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4015c370 con 0x7faf400d0440 2024-04-25T10:31:33.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45000640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4015bc10 0x7faf401574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf37400640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 0x7faf40156f60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c480 tx=0x7faf2800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28018070 con 0x7faf400d0440 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28007500 con 0x7faf400d0440 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4015c650 con 0x7faf400d0440 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28004a30 con 0x7faf400d0440 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4015cc10 con 0x7faf400d0440 2024-04-25T10:31:33.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf40075120 con 0x7faf400d0440 2024-04-25T10:31:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.082+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf2801b070 con 0x7faf400d0440 2024-04-25T10:31:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.086+0000 7faf3cc00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f60 0x7faf1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.086+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf28084400 con 0x7faf400d0440 2024-04-25T10:31:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.086+0000 7faf3fe00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f60 0x7faf1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.086+0000 7faf3fe00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f60 0x7faf1c069420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7faf300af000 tx=0x7faf300a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.090+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf28055400 con 0x7faf400d0440 2024-04-25T10:31:33.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.282+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf400b4060 con 0x7faf400d0440 2024-04-25T10:31:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.282+0000 7faf3cc00640 1 -- 172.21.15.45:0/3106148293 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4960 (secure 0 0 0) 0x7faf28054d00 con 0x7faf400d0440 2024-04-25T10:31:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:33.287 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:22.823850+0000","modified":"2024-04-25T10:31:25.830407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10881":{"gid":10881,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1455178901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1455178901},{"type":"v1","addr":"172.21.15.45:6835","nonce":1455178901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:31:33.287 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 131 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f60 msgr2=0x7faf1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f60 0x7faf1c069420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7faf300af000 tx=0x7faf300a7000 comp rx=0 tx=0).stop 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 msgr2=0x7faf40156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 0x7faf40156f60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c480 tx=0x7faf2800c940 comp rx=0 tx=0).stop 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 shutdown_connections 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f60 0x7faf1c069420 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4015bc10 0x7faf401574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400d0440 0x7faf40156f60 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 --2- 172.21.15.45:0/3106148293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf400ccd40 0x7faf400cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 >> 172.21.15.45:0/3106148293 conn(0x7faf400d1750 msgr2=0x7faf4006ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 shutdown_connections 2024-04-25T10:31:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.286+0000 7faf45a00640 1 -- 172.21.15.45:0/3106148293 wait complete. 2024-04-25T10:31:33.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:31:22.823850+0000', 'modified': '2024-04-25T10:31:25.830407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10881}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10881': {'gid': 10881, 'name': 'c', 'rank': 0, 'incarnation': 127, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/1455178901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1455178901}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1455178901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:31:33.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:31:33.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:31:33.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 Processor -- start 2024-04-25T10:31:33.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 -- start start 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b640cda10 0x7f1b6406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b640cf930 0x7f1b640cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b640b2770 con 0x7f1b6415bc40 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6406c670 con 0x7f1b640cda10 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6c000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6406c950 con 0x7f1b640cf930 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b69600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b6415bc40 0x7f1b640b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55724/0 (socket says 172.21.15.45:55724) 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b68c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b640cda10 0x7f1b6406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b69600640 1 -- 172.21.15.45:0/1121563376 learned_addr learned my addr 172.21.15.45:0/1121563376 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b6a000640 1 --2- 172.21.15.45:0/1121563376 >> v2:172.21.15.117:3300/0 conn(0x7f1b640cf930 0x7f1b640cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b63e00640 1 -- 172.21.15.45:0/1121563376 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305591630 0 0) 0x7f1b640b2770 con 0x7f1b6415bc40 2024-04-25T10:31:33.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b68c00640 1 -- 172.21.15.45:0/1121563376 >> v2:172.21.15.117:3300/0 conn(0x7f1b640cf930 msgr2=0x7f1b640cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b68c00640 1 --2- 172.21.15.45:0/1121563376 >> v2:172.21.15.117:3300/0 conn(0x7f1b640cf930 0x7f1b640cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b68c00640 1 -- 172.21.15.45:0/1121563376 >> v1:172.21.15.45:6789/0 conn(0x7f1b6415bc40 legacy=0x7f1b640b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.854+0000 7f1b68c00640 1 -- 172.21.15.45:0/1121563376 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6406cc30 con 0x7f1b640cda10 2024-04-25T10:31:33.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b68c00640 1 --2- 172.21.15.45:0/1121563376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6406c090 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f1b640ae4c0 tx=0x7f1b5001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457d3e872d3475e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b63e00640 1 -- 172.21.15.45:0/1121563376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b500b0070 con 0x7f1b640cda10 2024-04-25T10:31:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b63e00640 1 -- 172.21.15.45:0/1121563376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b500ab600 con 0x7f1b640cda10 2024-04-25T10:31:33.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b63e00640 1 -- 172.21.15.45:0/1121563376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b500ab9d0 con 0x7f1b640cda10 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 -- 172.21.15.45:0/1121563376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 msgr2=0x7f1b6406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 --2- 172.21.15.45:0/1121563376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6406c090 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f1b640ae4c0 tx=0x7f1b5001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 -- 172.21.15.45:0/1121563376 shutdown_connections 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 --2- 172.21.15.45:0/1121563376 >> v2:172.21.15.117:3300/0 conn(0x7f1b640cf930 0x7f1b640cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 --2- 172.21.15.45:0/1121563376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6406c090 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 -- 172.21.15.45:0/1121563376 >> 172.21.15.45:0/1121563376 conn(0x7f1b64066b70 msgr2=0x7f1b64066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 -- 172.21.15.45:0/1121563376 shutdown_connections 2024-04-25T10:31:33.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.858+0000 7f1b6c000640 1 -- 172.21.15.45:0/1121563376 wait complete. 2024-04-25T10:31:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 Processor -- start 2024-04-25T10:31:33.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 -- start start 2024-04-25T10:31:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b640cf930 0x7f1b64071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6415bc40 0x7f1b6415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b640b3850 con 0x7f1b640cda10 2024-04-25T10:31:33.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b640b36d0 con 0x7f1b6415bc40 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b640b39d0 con 0x7f1b640cf930 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6414a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57694/0 (socket says 172.21.15.45:57694) 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 -- 172.21.15.45:0/3234442573 learned_addr learned my addr 172.21.15.45:0/3234442573 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6a000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6415bc40 0x7f1b6415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b640cf930 0x7f1b64071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 -- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b640cf930 msgr2=0x7f1b64071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b640cf930 0x7f1b64071650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 -- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6415bc40 msgr2=0x7f1b6415ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6415bc40 0x7f1b6415ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 -- 172.21.15.45:0/3234442573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64070310 con 0x7f1b640cda10 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b640cf930 0x7f1b64071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:33.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b69600640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6414a5e0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c480 tx=0x7f1b5800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b61600640 1 -- 172.21.15.45:0/3234442573 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58018070 con 0x7f1b640cda10 2024-04-25T10:31:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b61600640 1 -- 172.21.15.45:0/3234442573 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58007500 con 0x7f1b640cda10 2024-04-25T10:31:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b640705b0 con 0x7f1b640cda10 2024-04-25T10:31:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b61600640 1 -- 172.21.15.45:0/3234442573 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58004a30 con 0x7f1b640cda10 2024-04-25T10:31:33.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.862+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b64070b70 con 0x7f1b640cda10 2024-04-25T10:31:33.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b61600640 1 -- 172.21.15.45:0/3234442573 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b5801b070 con 0x7f1b640cda10 2024-04-25T10:31:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b61600640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b44066f60 0x7f1b44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b61600640 1 -- 172.21.15.45:0/3234442573 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b58084350 con 0x7f1b640cda10 2024-04-25T10:31:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 Processor -- start 2024-04-25T10:31:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 -- start start 2024-04-25T10:31:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b44066f60 0x7f1b44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad51920 0x562a2ad51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad52240 0x562a2ad079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2ad08310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2abef5f0 con 0x562a2ad07f30 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2ab56da0 con 0x562a2ad51920 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a2aadf150 con 0x562a2ad52240 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2ad08310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2ad08310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57704/0 (socket says 172.21.15.45:57704) 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.866+0000 7f1b6a000640 1 -- 172.21.15.45:0/2865071233 learned_addr learned my addr 172.21.15.45:0/2865071233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b69600640 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad51920 0x562a2ad51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b44066f60 0x7f1b44069420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1b500ad000 tx=0x7f1b500a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b68c00640 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad52240 0x562a2ad079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 -- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad52240 msgr2=0x562a2ad079f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad52240 0x562a2ad079f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 -- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad51920 msgr2=0x562a2ad51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad51920 0x562a2ad51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 -- 172.21.15.45:0/2865071233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a2aaf8ca0 con 0x562a2ad07f30 2024-04-25T10:31:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2ad08310 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009f70 tx=0x7f1b5c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb22f073d928a0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b56000640 1 -- 172.21.15.45:0/2865071233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c015420 con 0x562a2ad07f30 2024-04-25T10:31:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b56000640 1 -- 172.21.15.45:0/2865071233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c01a070 con 0x562a2ad07f30 2024-04-25T10:31:33.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b56000640 1 -- 172.21.15.45:0/2865071233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0157c0 con 0x562a2ad07f30 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- 172.21.15.45:0/2865071233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 msgr2=0x562a2ad08310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2ad08310 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009f70 tx=0x7f1b5c003e30 comp rx=0 tx=0).stop 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- 172.21.15.45:0/2865071233 shutdown_connections 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2ad08310 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad52240 0x562a2ad079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- 172.21.15.45:0/2865071233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad51920 0x562a2ad51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- 172.21.15.45:0/2865071233 >> 172.21.15.45:0/2865071233 conn(0x562a2ad0e2a0 msgr2=0x562a2ad062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- 172.21.15.45:0/2865071233 shutdown_connections 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- 172.21.15.45:0/2865071233 wait complete. 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- start start 2024-04-25T10:31:33.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 Processor -- start 2024-04-25T10:31:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad07f30 0x562a2ad58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2ad59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 0x562a2ad0baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2ab11990 con 0x562a2ad52240 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2ab52e40 con 0x562a2ad07f30 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a2aae2900 con 0x562a2ad51920 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6f579000 10 client.0 _ll_register_callbacks cb 0x562a2ac0a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 0x562a2ad0baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 0x562a2ad0baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57716/0 (socket says 172.21.15.45:57716) 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b6a000640 1 -- 172.21.15.45:0/3889152095 learned_addr learned my addr 172.21.15.45:0/3889152095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2ad59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.870+0000 7f1b69600640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad07f30 0x562a2ad58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6a000640 1 -- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 msgr2=0x562a2ad59310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6a000640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2ad59310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6a000640 1 -- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad07f30 msgr2=0x562a2ad58dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6a000640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad07f30 0x562a2ad58dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6a000640 1 -- 172.21.15.45:0/3889152095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a2ab0f630 con 0x562a2ad52240 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2ad59310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6a000640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 0x562a2ad0baa0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c017040 tx=0x7f1b5c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d468920a68f900e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3889152095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c023070 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3889152095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b5c0159e0 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b5c01a070 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3889152095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b5c030050 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 10 client.11052 fetch_fsmap learned FSMap version 131 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x562a2aadf3a0 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3889152095 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c01a070 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3889152095 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b5c033020 con 0x562a2ad52240 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 10 client.11052 fetch_fsmap finished waiting for FSMap version 131 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 10 client.11052 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 20 client.11052 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad0d160 0x562a2ad62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 4 client.11052 mds_command: new command op to 10881 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 --> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a2aaf2a50 con 0x562a2ad0d160 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b69600640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad0d160 0x562a2ad62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b69600640 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad0d160 0x562a2ad62e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e052fc834f47e70 server_cookie=eac56ffb6563d427 in_seq=0 out_seq=0 2024-04-25T10:31:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 10 client.11052 ms_handle_connect on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3889152095 <== mds.0 v2:172.21.15.45:6833/1455178901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a2aaf2a50 con 0x562a2ad0d160 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 10 client.11052 handle_command_reply: tid=0 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 client.11052 shutdown 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 msgr2=0x562a2ad0baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 0x562a2ad0baa0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c017040 tx=0x7f1b5c004ac0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 shutdown_connections 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad0d160 0x562a2ad62e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad52240 0x562a2ad0baa0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2ad59310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3889152095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad07f30 0x562a2ad58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b4aa00640 0 client.11052 ms_handle_reset on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 >> 172.21.15.45:0/3889152095 conn(0x562a2ad0e2a0 msgr2=0x562a2ad069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 shutdown_connections 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 1 -- 172.21.15.45:0/3889152095 wait complete. 2024-04-25T10:31:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.874+0000 7f1b6f579000 20 client.11052 trim_cache size 0 max 16384 2024-04-25T10:31:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.890+0000 7f1b6f579000 1 Processor -- start 2024-04-25T10:31:33.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.890+0000 7f1b6f579000 1 -- start start 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.890+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2adad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2adad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2adadb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64158a30 con 0x562a2ad07f30 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64154de0 con 0x562a2ad52240 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64158430 con 0x562a2ad51920 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2adad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2adad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57726/0 (socket says 172.21.15.45:57726) 2024-04-25T10:31:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 -- 172.21.15.45:0/1162105432 learned_addr learned my addr 172.21.15.45:0/1162105432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 -- 172.21.15.45:0/1162105432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 msgr2=0x562a2adad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2adad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 -- 172.21.15.45:0/1162105432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 msgr2=0x562a2adadb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2adadb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 -- 172.21.15.45:0/1162105432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a2ad52070 con 0x562a2ad07f30 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2adad0d0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f1b58051160 tx=0x7f1b5801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a35371b2ead679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b55600640 1 -- 172.21.15.45:0/1162105432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580828f0 con 0x562a2ad07f30 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b55600640 1 -- 172.21.15.45:0/1162105432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58001070 con 0x562a2ad07f30 2024-04-25T10:31:33.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b55600640 1 -- 172.21.15.45:0/1162105432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58053000 con 0x562a2ad07f30 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- 172.21.15.45:0/1162105432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 msgr2=0x562a2adad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2adad0d0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f1b58051160 tx=0x7f1b5801f040 comp rx=0 tx=0).stop 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- 172.21.15.45:0/1162105432 shutdown_connections 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2adadb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad51920 0x562a2adad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- 172.21.15.45:0/1162105432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad07f30 0x562a2adad0d0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- 172.21.15.45:0/1162105432 >> 172.21.15.45:0/1162105432 conn(0x562a2ad0e2a0 msgr2=0x562a2ad0b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- 172.21.15.45:0/1162105432 shutdown_connections 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- 172.21.15.45:0/1162105432 wait complete. 2024-04-25T10:31:33.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- start start 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 Processor -- start 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad07f30 0x562a2ad5e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 0x562a2ad95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2ad95d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2ab5b3d0 con 0x562a2ad51920 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a2ac12d50 con 0x562a2ad52240 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a2aaf1980 con 0x562a2ad07f30 2024-04-25T10:31:33.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b6f579000 10 client.0 _ll_register_callbacks cb 0x7f1b5c009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b68c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 0x562a2ad95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b68c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 0x562a2ad95840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57728/0 (socket says 172.21.15.45:57728) 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b68c00640 1 -- 172.21.15.45:0/3372645367 learned_addr learned my addr 172.21.15.45:0/3372645367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.894+0000 7f1b69600640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad07f30 0x562a2ad5e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6a000640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2ad95d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b68c00640 1 -- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad07f30 msgr2=0x562a2ad5e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad07f30 0x562a2ad5e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b68c00640 1 -- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 msgr2=0x562a2ad95d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2ad95d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b68c00640 1 -- 172.21.15.45:0/3372645367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a2abd8600 con 0x562a2ad51920 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b68c00640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 0x562a2ad95840 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f1b50000c10 tx=0x7f1b50019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7386a3574a82bc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3372645367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b500c9020 con 0x562a2ad51920 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b58001070 con 0x562a2ad51920 2024-04-25T10:31:33.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3372645367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b50003b30 con 0x562a2ad51920 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3372645367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b500d3050 con 0x562a2ad51920 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 10 client.11058 fetch_fsmap learned FSMap version 131 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x562a2aaeb9b0 con 0x562a2ad51920 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3372645367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58001070 con 0x562a2ad51920 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3372645367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b500d6020 con 0x562a2ad51920 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 10 client.11058 fetch_fsmap finished waiting for FSMap version 131 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 10 client.11058 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 20 client.11058 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad96690 0x562a2ad96a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 4 client.11058 mds_command: new command op to 10881 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 --> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562a2ab642e0 con 0x562a2ad96690 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b69600640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad96690 0x562a2ad96a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b69600640 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad96690 0x562a2ad96a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9755b28a9bda4c server_cookie=4ab894260d2b983a in_seq=0 out_seq=0 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 10 client.11058 ms_handle_connect on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:33.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 1 -- 172.21.15.45:0/3372645367 <== mds.0 v2:172.21.15.45:6833/1455178901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x562a2ab642e0 con 0x562a2ad96690 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 10 client.11058 handle_command_reply: tid=0 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 client.11058 shutdown 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 msgr2=0x562a2ad95840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 0x562a2ad95840 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f1b50000c10 tx=0x7f1b50019190 comp rx=0 tx=0).stop 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 shutdown_connections 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x562a2ad96690 0x562a2ad96a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a2ad52240 0x562a2ad95d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a2ad51920 0x562a2ad95840 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b4aa00640 0 client.11058 ms_handle_reset on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 --2- 172.21.15.45:0/3372645367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x562a2ad07f30 0x562a2ad5e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 >> 172.21.15.45:0/3372645367 conn(0x562a2ad0e2a0 msgr2=0x562a2ad598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 shutdown_connections 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 1 -- 172.21.15.45:0/3372645367 wait complete. 2024-04-25T10:31:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6f579000 20 client.11058 trim_cache size 0 max 16384 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 10881, 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.471608979999999, 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 131, 2024-04-25T10:31:33.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 182, 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 182, 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 15.273982971000001 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b44066f60 msgr2=0x7f1b44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b44066f60 0x7f1b44069420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1b500ad000 tx=0x7f1b500a6000 comp rx=0 tx=0).stop 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 msgr2=0x7f1b6414a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:33.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6414a5e0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c480 tx=0x7f1b5800c940 comp rx=0 tx=0).stop 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 shutdown_connections 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1b44066f60 0x7f1b44069420 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b6415bc40 0x7f1b6415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b640cf930 0x7f1b64071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 --2- 172.21.15.45:0/3234442573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b640cda10 0x7f1b6414a5e0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 >> 172.21.15.45:0/3234442573 conn(0x7f1b64066b70 msgr2=0x7f1b640ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 shutdown_connections 2024-04-25T10:31:33.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:33.898+0000 7f1b6c000640 1 -- 172.21.15.45:0/3234442573 wait complete. 2024-04-25T10:31:33.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:31:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 Processor -- start 2024-04-25T10:31:34.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 -- start start 2024-04-25T10:31:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f416815bc40 0x7f416806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:34.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cda10 0x7f41680b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680b2770 con 0x7f41680cf930 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416806c670 con 0x7f416815bc40 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f416e600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416806c950 con 0x7f41680cda10 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f416815bc40 0x7f416806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f41680cf930 0x7f416815f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55732/0 (socket says 172.21.15.45:55732) 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167e00640 1 -- 172.21.15.45:0/2297443761 learned_addr learned my addr 172.21.15.45:0/2297443761 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f415ea00640 1 --2- 172.21.15.45:0/2297443761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cda10 0x7f41680b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167400640 1 -- 172.21.15.45:0/2297443761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cda10 msgr2=0x7f41680b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167400640 1 --2- 172.21.15.45:0/2297443761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cda10 0x7f41680b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167400640 1 -- 172.21.15.45:0/2297443761 >> v1:172.21.15.45:6789/0 conn(0x7f41680cf930 legacy=0x7f416815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:34.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.062+0000 7f4167400640 1 -- 172.21.15.45:0/2297443761 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416806cc30 con 0x7f416815bc40 2024-04-25T10:31:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f4167400640 1 --2- 172.21.15.45:0/2297443761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f416806c090 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f41580097c0 tx=0x7f415801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77b3b1b98e1f870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:34.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f4166a00640 1 -- 172.21.15.45:0/2297443761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41580b1070 con 0x7f416815bc40 2024-04-25T10:31:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f4166a00640 1 -- 172.21.15.45:0/2297443761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41580ac410 con 0x7f416815bc40 2024-04-25T10:31:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f4166a00640 1 -- 172.21.15.45:0/2297443761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41580ac7e0 con 0x7f416815bc40 2024-04-25T10:31:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 -- 172.21.15.45:0/2297443761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 msgr2=0x7f416806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:34.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 --2- 172.21.15.45:0/2297443761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f416806c090 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f41580097c0 tx=0x7f415801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 -- 172.21.15.45:0/2297443761 shutdown_connections 2024-04-25T10:31:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 --2- 172.21.15.45:0/2297443761 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cda10 0x7f41680b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 --2- 172.21.15.45:0/2297443761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f416806c090 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 -- 172.21.15.45:0/2297443761 >> 172.21.15.45:0/2297443761 conn(0x7f4168066b70 msgr2=0x7f4168066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 -- 172.21.15.45:0/2297443761 shutdown_connections 2024-04-25T10:31:34.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.066+0000 7f416e600640 1 -- 172.21.15.45:0/2297443761 wait complete. 2024-04-25T10:31:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 Processor -- start 2024-04-25T10:31:34.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- start start 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41680cda10 0x7f41680b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cf930 0x7f416814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f4168070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168168b50 con 0x7f416815bc40 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681689d0 con 0x7f41680cda10 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4168168cd0 con 0x7f41680cf930 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f4168070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f4168070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57742/0 (socket says 172.21.15.45:57742) 2024-04-25T10:31:34.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 -- 172.21.15.45:0/500261965 learned_addr learned my addr 172.21.15.45:0/500261965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f415ea00640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cf930 0x7f416814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167400640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41680cda10 0x7f41680b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 -- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cf930 msgr2=0x7f416814f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cf930 0x7f416814f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 -- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41680cda10 msgr2=0x7f41680b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41680cda10 0x7f41680b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 -- 172.21.15.45:0/500261965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416814fe20 con 0x7f416815bc40 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167400640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41680cda10 0x7f41680b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f4167e00640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f4168070f50 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f416000ca80 tx=0x7f416000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:34.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160017070 con 0x7f416815bc40 2024-04-25T10:31:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416000ac30 con 0x7f416815bc40 2024-04-25T10:31:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160012600 con 0x7f416815bc40 2024-04-25T10:31:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4168150100 con 0x7f416815bc40 2024-04-25T10:31:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41681506c0 con 0x7f416815bc40 2024-04-25T10:31:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.070+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416804ef30 con 0x7f416815bc40 2024-04-25T10:31:34.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.074+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f416001a070 con 0x7f416815bc40 2024-04-25T10:31:34.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.074+0000 7f415fe00640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4144066f90 0x7f4144069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:34.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.074+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4160084c90 con 0x7f416815bc40 2024-04-25T10:31:34.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.074+0000 7f4167400640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4144066f90 0x7f4144069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:34.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.074+0000 7f4167400640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4144066f90 0x7f4144069450 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f41580ae000 tx=0x7f4158008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:34.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.078+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f416000ae20 con 0x7f416815bc40 2024-04-25T10:31:34.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.274+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f416806d960 con 0x7f416815bc40 2024-04-25T10:31:34.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.274+0000 7f415fe00640 1 -- 172.21.15.45:0/500261965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v131) v1 ==== 53+0+73 (secure 0 0 0) 0x7f41600555b0 con 0x7f416815bc40 2024-04-25T10:31:34.279 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:31:34.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4144066f90 msgr2=0x7f4144069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4144066f90 0x7f4144069450 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f41580ae000 tx=0x7f4158008040 comp rx=0 tx=0).stop 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 msgr2=0x7f4168070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f4168070f50 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f416000ca80 tx=0x7f416000cf40 comp rx=0 tx=0).stop 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 shutdown_connections 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4144066f90 0x7f4144069450 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f416815bc40 0x7f4168070f50 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41680cf930 0x7f416814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 --2- 172.21.15.45:0/500261965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41680cda10 0x7f41680b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 >> 172.21.15.45:0/500261965 conn(0x7f4168066b70 msgr2=0x7f41680b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 shutdown_connections 2024-04-25T10:31:34.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:34.278+0000 7f416e600640 1 -- 172.21.15.45:0/500261965 wait complete. 2024-04-25T10:31:34.299 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:31:34.299 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:31:34.299 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:31:34.299 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:31:34.300 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:34.300 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:31:34.300 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:31:34.300 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:31:34.300 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:31:34.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.300 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:31:34.309 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:31:34.309 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:31:34.309 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:31:34.309 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.309 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41646sec preferred_lft 41646sec 2024-04-25T10:31:34.310 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:31:34.312 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:31:34.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.313 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:31:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:31:34.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.375 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:31:34.432 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:31:34.432 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:31:34.432 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:31:34.432 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:31:34.432 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:31:34.432 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:31:34.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.433 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:31:34.433 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:31:34.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:31:34.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:31:34.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:31:34.433 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:31:34.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.529 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:31:34.538 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:31:34.538 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:31:34.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:31:34.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:31:34.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:31:34.598 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:34.655 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:31:34.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:31:34.655 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:34.706 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:31:34.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:31:34.797 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:31:34.797 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:31:34.809 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:31:34.809 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:31:34.809 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:31:34.809 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:31:34.810 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:31:34.813 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:31:34.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:34.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:34.834 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:31:34.895 INFO:teuthology.orchestra.run.smithi045.stdout:client11070 2024-04-25T10:31:34.896 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:34.896 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:34.947 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:34.947 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> import uuid 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> n = 1000 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> create_hard_links = False 2024-04-25T10:31:34.999 DEBUG:teuthology.orchestra.run.smithi045:> if 0 > 0: 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> create_hard_links = True 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> dpath = os.path.dirname(path) 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> fnameprefix = os.path.basename(path) 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> for i in range(n): 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> f.write(f"{i}") 2024-04-25T10:31:35.000 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> f.flush() 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(f.fileno()) 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> os.unlink(fpath) 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(dirfd) 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> if create_hard_links: 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> for j in range(0): 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> if False: 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> os.fsync(dirfd) 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> finally: 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> os.close(dirfd) 2024-04-25T10:31:35.001 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-25T10:31:35.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:31:35.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:35.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T10:31:35.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 Processor -- start 2024-04-25T10:31:35.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 -- start start 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6180aef80 0x7ff61815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff618070350 0x7ff61815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61815bac0 0x7ff6180b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61815b080 con 0x7ff6180aef80 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61815adb0 con 0x7ff618070350 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff61ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61815eb60 con 0x7ff61815bac0 2024-04-25T10:31:35.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6180aef80 0x7ff61815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff6180aef80 0x7ff61815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58590/0 (socket says 172.21.15.45:58590) 2024-04-25T10:31:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 -- 172.21.15.45:0/3607833454 learned_addr learned my addr 172.21.15.45:0/3607833454 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 -- 172.21.15.45:0/3607833454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61815bac0 msgr2=0x7ff6180b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:35.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 --2- 172.21.15.45:0/3607833454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61815bac0 0x7ff6180b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 -- 172.21.15.45:0/3607833454 >> v2:172.21.15.117:3300/0 conn(0x7ff618070350 msgr2=0x7ff61815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff60fe00640 1 --2- 172.21.15.45:0/3607833454 >> v2:172.21.15.117:3300/0 conn(0x7ff618070350 0x7ff61815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 --2- 172.21.15.45:0/3607833454 >> v2:172.21.15.117:3300/0 conn(0x7ff618070350 0x7ff61815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff617e00640 1 -- 172.21.15.45:0/3607833454 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61815ee40 con 0x7ff6180aef80 2024-04-25T10:31:35.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.966+0000 7ff60fe00640 1 --2- 172.21.15.45:0/3607833454 >> v2:172.21.15.117:3300/0 conn(0x7ff618070350 0x7ff61815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff617e00640 1 --2- 172.21.15.45:0/3607833454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180aef80 0x7ff61815e480 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7ff61806e390 tx=0x7ff60801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04039b9b01d4db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff617400640 1 -- 172.21.15.45:0/3607833454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6080ab5f0 con 0x7ff6180aef80 2024-04-25T10:31:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff617400640 1 -- 172.21.15.45:0/3607833454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6080b0070 con 0x7ff6180aef80 2024-04-25T10:31:35.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff617400640 1 -- 172.21.15.45:0/3607833454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6080ab9b0 con 0x7ff6180aef80 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 -- 172.21.15.45:0/3607833454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180aef80 msgr2=0x7ff61815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 --2- 172.21.15.45:0/3607833454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180aef80 0x7ff61815e480 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7ff61806e390 tx=0x7ff60801c520 comp rx=0 tx=0).stop 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 -- 172.21.15.45:0/3607833454 shutdown_connections 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 --2- 172.21.15.45:0/3607833454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff61815bac0 0x7ff6180b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 --2- 172.21.15.45:0/3607833454 >> v2:172.21.15.117:3300/0 conn(0x7ff618070350 0x7ff61815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 --2- 172.21.15.45:0/3607833454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6180aef80 0x7ff61815e480 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 -- 172.21.15.45:0/3607833454 >> 172.21.15.45:0/3607833454 conn(0x7ff6180669a0 msgr2=0x7ff618066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 -- 172.21.15.45:0/3607833454 shutdown_connections 2024-04-25T10:31:35.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.970+0000 7ff61ea00640 1 -- 172.21.15.45:0/3607833454 wait complete. 2024-04-25T10:31:35.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 Processor -- start 2024-04-25T10:31:35.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 -- start start 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff618070350 0x7ff618156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6180aef80 0x7ff61814bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 0x7ff618158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806c460 con 0x7ff61815bac0 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806c2e0 con 0x7ff6180aef80 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61806c5e0 con 0x7ff618070350 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 0x7ff618158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 0x7ff618158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58600/0 (socket says 172.21.15.45:58600) 2024-04-25T10:31:35.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 -- 172.21.15.45:0/839337574 learned_addr learned my addr 172.21.15.45:0/839337574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff60fe00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6180aef80 0x7ff61814bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff617e00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff618070350 0x7ff618156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 -- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff618070350 msgr2=0x7ff618156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff618070350 0x7ff618156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 -- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6180aef80 msgr2=0x7ff61814bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6180aef80 0x7ff61814bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 -- 172.21.15.45:0/839337574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618157250 con 0x7ff61815bac0 2024-04-25T10:31:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ca00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 0x7ff618158f00 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b970 tx=0x7ff61000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff614c00640 1 -- 172.21.15.45:0/839337574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610019070 con 0x7ff61815bac0 2024-04-25T10:31:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff614c00640 1 -- 172.21.15.45:0/839337574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61000cb60 con 0x7ff61815bac0 2024-04-25T10:31:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff618157530 con 0x7ff61815bac0 2024-04-25T10:31:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff614c00640 1 -- 172.21.15.45:0/839337574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610014440 con 0x7ff61815bac0 2024-04-25T10:31:35.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.974+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61814c0a0 con 0x7ff61815bac0 2024-04-25T10:31:35.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff614c00640 1 -- 172.21.15.45:0/839337574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff61001c070 con 0x7ff61815bac0 2024-04-25T10:31:35.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff614c00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f4066f60 0x7ff5f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff614c00640 1 -- 172.21.15.45:0/839337574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff610084e60 con 0x7ff61815bac0 2024-04-25T10:31:35.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff617e00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f4066f60 0x7ff5f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 Processor -- start 2024-04-25T10:31:35.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 -- start start 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff617e00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f4066f60 0x7ff5f4069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff6080af000 tx=0x7ff6080a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbc9d0 0x55f352fbcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 0x55f352f72b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352f730a0 0x55f352f73480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f352dbfff0 con 0x55f352fbd2f0 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f352d48150 con 0x55f352f730a0 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f352dbc090 con 0x55f352fbc9d0 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 0x55f352f72b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 0x55f352f72b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58604/0 (socket says 172.21.15.45:58604) 2024-04-25T10:31:35.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff60fe00640 1 -- 172.21.15.45:0/331880474 learned_addr learned my addr 172.21.15.45:0/331880474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff617e00640 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbc9d0 0x55f352fbcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.978+0000 7ff61ca00640 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352f730a0 0x55f352f73480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 -- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbc9d0 msgr2=0x55f352fbcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbc9d0 0x55f352fbcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 -- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352f730a0 msgr2=0x55f352f73480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352f730a0 0x55f352f73480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 -- 172.21.15.45:0/331880474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f352f560d0 con 0x55f352fbd2f0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 0x55f352f72b60 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7ff600004660 tx=0x7ff60000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c28e2064f7c70fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60cc00640 1 -- 172.21.15.45:0/331880474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000f9c0 con 0x55f352fbd2f0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60cc00640 1 -- 172.21.15.45:0/331880474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000fb60 con 0x55f352fbd2f0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60cc00640 1 -- 172.21.15.45:0/331880474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600014480 con 0x55f352fbd2f0 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- 172.21.15.45:0/331880474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 msgr2=0x55f352f72b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:35.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 0x55f352f72b60 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7ff600004660 tx=0x7ff60000dd90 comp rx=0 tx=0).stop 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- 172.21.15.45:0/331880474 shutdown_connections 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352f730a0 0x55f352f73480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbd2f0 0x55f352f72b60 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- 172.21.15.45:0/331880474 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbc9d0 0x55f352fbcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- 172.21.15.45:0/331880474 >> 172.21.15.45:0/331880474 conn(0x55f352f6f020 msgr2=0x55f352f71440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- 172.21.15.45:0/331880474 shutdown_connections 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- 172.21.15.45:0/331880474 wait complete. 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- start start 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 Processor -- start 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f352fa17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f352fa1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fa2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f352d84250 con 0x55f352f730a0 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f352d7a690 con 0x55f352fbc9d0 2024-04-25T10:31:35.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f352d48a40 con 0x55f352fbd2f0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff621dc3000 10 client.0 _ll_register_callbacks cb 0x55f352f5de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f352fa17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f352fa17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58612/0 (socket says 172.21.15.45:58612) 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 -- 172.21.15.45:0/2095513916 learned_addr learned my addr 172.21.15.45:0/2095513916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff60fe00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f352fa1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff61ca00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fa2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 -- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 msgr2=0x55f352fa2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fa2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 -- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 msgr2=0x55f352fa1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f352fa1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.982+0000 7ff617e00640 1 -- 172.21.15.45:0/2095513916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f352e6d1c0 con 0x55f352f730a0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff617e00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f352fa17f0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7ff6080ad040 tx=0x7ff60801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a8acaf4b21c820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:35.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 1 -- 172.21.15.45:0/2095513916 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6080cd060 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 1 -- 172.21.15.45:0/2095513916 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6080bf8e0 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60000fb60 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 1 -- 172.21.15.45:0/2095513916 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6080d3050 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 10 client.11082 fetch_fsmap learned FSMap version 131 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x55f352e69170 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 1 -- 172.21.15.45:0/2095513916 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000fb60 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 1 -- 172.21.15.45:0/2095513916 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6080c7070 con 0x55f352f730a0 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 10 client.11082 fetch_fsmap finished waiting for FSMap version 131 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 10 client.11082 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 20 client.11082 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fccc80 0x55f352fcf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 4 client.11082 mds_command: new command op to 10881 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 --> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f352d5bc60 con 0x55f352fccc80 2024-04-25T10:31:35.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff60fe00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fccc80 0x55f352fcf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff60fe00640 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fccc80 0x55f352fcf070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd9389f1ec572db server_cookie=fde9c01782e163e1 in_seq=0 out_seq=0 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 10 client.11082 ms_handle_connect on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 1 -- 172.21.15.45:0/2095513916 <== mds.0 v2:172.21.15.45:6833/1455178901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f352d5bc60 con 0x55f352fccc80 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 10 client.11082 handle_command_reply: tid=0 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 client.11082 shutdown 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 msgr2=0x55f352fa17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f352fa17f0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7ff6080ad040 tx=0x7ff60801ad60 comp rx=0 tx=0).stop 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 shutdown_connections 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fccc80 0x55f352fcf070 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fa2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f352fa1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2095513916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f352fa17f0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff605600640 0 client.11082 ms_handle_reset on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:35.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 >> 172.21.15.45:0/2095513916 conn(0x55f352f6f020 msgr2=0x55f352f71ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:35.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 shutdown_connections 2024-04-25T10:31:35.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 1 -- 172.21.15.45:0/2095513916 wait complete. 2024-04-25T10:31:35.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:35.986+0000 7ff621dc3000 20 client.11082 trim_cache size 0 max 16384 2024-04-25T10:31:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 Processor -- start 2024-04-25T10:31:36.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 -- start start 2024-04-25T10:31:36.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352f730a0 0x55f352f71410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 0x55f352f71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbd2f0 0x55f352f71e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618203570 con 0x55f352fbc9d0 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618202840 con 0x55f352fbd2f0 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180b1af0 con 0x55f352f730a0 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 0x55f352f71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 0x55f352f71950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58620/0 (socket says 172.21.15.45:58620) 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 -- 172.21.15.45:0/230430524 learned_addr learned my addr 172.21.15.45:0/230430524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff617e00640 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352f730a0 0x55f352f71410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff61ca00640 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbd2f0 0x55f352f71e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 -- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352f730a0 msgr2=0x55f352f71410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352f730a0 0x55f352f71410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 -- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbd2f0 msgr2=0x55f352f71e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbd2f0 0x55f352f71e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.002+0000 7ff60fe00640 1 -- 172.21.15.45:0/230430524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f352e06de0 con 0x55f352fbc9d0 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff60fe00640 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 0x55f352f71950 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7ff600004630 tx=0x7ff600014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198c1dc995bf149a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff607e00640 1 -- 172.21.15.45:0/230430524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600030070 con 0x55f352fbc9d0 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff607e00640 1 -- 172.21.15.45:0/230430524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60002b0d0 con 0x55f352fbc9d0 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff607e00640 1 -- 172.21.15.45:0/230430524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60002b4a0 con 0x55f352fbc9d0 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- 172.21.15.45:0/230430524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 msgr2=0x55f352f71950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 0x55f352f71950 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7ff600004630 tx=0x7ff600014000 comp rx=0 tx=0).stop 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- 172.21.15.45:0/230430524 shutdown_connections 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbd2f0 0x55f352f71e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352fbc9d0 0x55f352f71950 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- 172.21.15.45:0/230430524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352f730a0 0x55f352f71410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- 172.21.15.45:0/230430524 >> 172.21.15.45:0/230430524 conn(0x55f352f6f020 msgr2=0x55f352fc9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- 172.21.15.45:0/230430524 shutdown_connections 2024-04-25T10:31:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- 172.21.15.45:0/230430524 wait complete. 2024-04-25T10:31:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- start start 2024-04-25T10:31:36.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 Processor -- start 2024-04-25T10:31:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f35301c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f35301ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fcea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f352d5ae00 con 0x55f352f730a0 2024-04-25T10:31:36.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180b20f0 con 0x55f352fbc9d0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f352dbf300 con 0x55f352fbd2f0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff621dc3000 10 client.0 _ll_register_callbacks cb 0x55f352f3e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f35301c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f35301c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58630/0 (socket says 172.21.15.45:58630) 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 -- 172.21.15.45:0/2756113238 learned_addr learned my addr 172.21.15.45:0/2756113238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff60fe00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f35301ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff61ca00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fcea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 -- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 msgr2=0x55f352fcea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fcea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 -- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 msgr2=0x55f35301ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f35301ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 -- 172.21.15.45:0/2756113238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f352d618e0 con 0x55f352f730a0 2024-04-25T10:31:36.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff617e00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f35301c510 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7ff608009840 tx=0x7ff608008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35d2252e17ed81b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.006+0000 7ff605600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 1 -- 172.21.15.45:0/2756113238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6080ba070 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 1 -- 172.21.15.45:0/2756113238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6080059a0 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60002b0d0 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 1 -- 172.21.15.45:0/2756113238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6080ab590 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 10 client.11088 fetch_fsmap learned FSMap version 131 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x55f352e198f0 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 1 -- 172.21.15.45:0/2756113238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60002b0d0 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 1 -- 172.21.15.45:0/2756113238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6080e9020 con 0x55f352f730a0 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 10 client.11088 fetch_fsmap finished waiting for FSMap version 131 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 10 client.11088 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 20 client.11088 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fcf7a0 0x55f352fcfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 4 client.11088 mds_command: new command op to 10881 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 --> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f352dcd530 con 0x55f352fcf7a0 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff60fe00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fcf7a0 0x55f352fcfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff60fe00640 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fcf7a0 0x55f352fcfbb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17294e8f55247d64 server_cookie=3ccdffbbfd69e203 in_seq=0 out_seq=0 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 10 client.11088 ms_handle_connect on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 1 -- 172.21.15.45:0/2756113238 <== mds.0 v2:172.21.15.45:6833/1455178901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f352dcd530 con 0x55f352fcf7a0 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 10 client.11088 handle_command_reply: tid=0 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 client.11088 shutdown 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 msgr2=0x55f35301c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f35301c510 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7ff608009840 tx=0x7ff608008040 comp rx=0 tx=0).stop 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 shutdown_connections 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x55f352fcf7a0 0x55f352fcfbb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55f352fbd2f0 0x55f352fcea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f352fbc9d0 0x55f35301ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff605600640 0 client.11088 ms_handle_reset on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 --2- 172.21.15.45:0/2756113238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f352f730a0 0x55f35301c510 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 >> 172.21.15.45:0/2756113238 conn(0x55f352f6f020 msgr2=0x55f352fc9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 shutdown_connections 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 1 -- 172.21.15.45:0/2756113238 wait complete. 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff621dc3000 20 client.11088 trim_cache size 0 max 16384 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:31:36.017 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f4066f60 msgr2=0x7ff5f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f4066f60 0x7ff5f4069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff6080af000 tx=0x7ff6080a7000 comp rx=0 tx=0).stop 2024-04-25T10:31:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 msgr2=0x7ff618158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 0x7ff618158f00 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b970 tx=0x7ff61000be30 comp rx=0 tx=0).stop 2024-04-25T10:31:36.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 shutdown_connections 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff5f4066f60 0x7ff5f4069420 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff61815bac0 0x7ff618158f00 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6180aef80 0x7ff61814bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 --2- 172.21.15.45:0/839337574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff618070350 0x7ff618156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 >> 172.21.15.45:0/839337574 conn(0x7ff6180669a0 msgr2=0x7ff6180b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 shutdown_connections 2024-04-25T10:31:36.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.010+0000 7ff61ea00640 1 -- 172.21.15.45:0/839337574 wait complete. 2024-04-25T10:31:36.032 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:31:36.032 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:31:36.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 damage ls 2024-04-25T10:31:36.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 Processor -- start 2024-04-25T10:31:36.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 -- start start 2024-04-25T10:31:36.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27780aef80 0x7f277815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2778070350 0x7f277815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277815b080 con 0x7f277815bac0 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277815adb0 con 0x7f2778070350 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277815eb60 con 0x7f27780aef80 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2778070350 0x7f277815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2778070350 0x7f277815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54134/0 (socket says 172.21.15.45:54134) 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277d000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27780aef80 0x7f277815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 -- 172.21.15.45:0/3132281472 learned_addr learned my addr 172.21.15.45:0/3132281472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f277da00640 1 --1- 172.21.15.45:0/3132281472 >> v1:172.21.15.117:6789/0 conn(0x7f277815bac0 0x7f27780b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:59994/0 (socket says 172.21.15.45:59994) 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 -- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27780aef80 msgr2=0x7f277815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 --2- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27780aef80 0x7f277815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 -- 172.21.15.45:0/3132281472 >> v1:172.21.15.117:6789/0 conn(0x7f277815bac0 legacy=0x7f27780b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.174+0000 7f276fe00640 1 -- 172.21.15.45:0/3132281472 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277815ee40 con 0x7f2778070350 2024-04-25T10:31:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f276fe00640 1 --2- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778070350 0x7f277815b580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f276000bb10 tx=0x7f276001ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5aecb0d9671f5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f2777e00640 1 -- 172.21.15.45:0/3132281472 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600b4070 con 0x7f2778070350 2024-04-25T10:31:36.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f2777e00640 1 -- 172.21.15.45:0/3132281472 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600a7ce0 con 0x7f2778070350 2024-04-25T10:31:36.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f2777e00640 1 -- 172.21.15.45:0/3132281472 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600af660 con 0x7f2778070350 2024-04-25T10:31:36.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 -- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778070350 msgr2=0x7f277815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 --2- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778070350 0x7f277815b580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f276000bb10 tx=0x7f276001ec30 comp rx=0 tx=0).stop 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 -- 172.21.15.45:0/3132281472 shutdown_connections 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 --2- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778070350 0x7f277815b580 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 --2- 172.21.15.45:0/3132281472 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27780aef80 0x7f277815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 -- 172.21.15.45:0/3132281472 >> 172.21.15.45:0/3132281472 conn(0x7f27780669a0 msgr2=0x7f2778066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 -- 172.21.15.45:0/3132281472 shutdown_connections 2024-04-25T10:31:36.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.178+0000 7f277fa00640 1 -- 172.21.15.45:0/3132281472 wait complete. 2024-04-25T10:31:36.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 Processor -- start 2024-04-25T10:31:36.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 -- start start 2024-04-25T10:31:36.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2778070350 0x7f277813f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27780aef80 0x7f27781520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 0x7f27780cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277806c0d0 con 0x7f277815bac0 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780b3ae0 con 0x7f27780aef80 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277806c250 con 0x7f2778070350 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 0x7f27780cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 0x7f27780cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58642/0 (socket says 172.21.15.45:58642) 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 -- 172.21.15.45:0/1121014634 learned_addr learned my addr 172.21.15.45:0/1121014634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277d000640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2778070350 0x7f277813f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f276fe00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27780aef80 0x7f27781520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 -- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2778070350 msgr2=0x7f277813f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2778070350 0x7f277813f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 -- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27780aef80 msgr2=0x7f27781520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27780aef80 0x7f27781520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 -- 172.21.15.45:0/1121014634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780af3c0 con 0x7f277815bac0 2024-04-25T10:31:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277da00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 0x7f27780cd990 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f277000c700 tx=0x7f277000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f2775600640 1 -- 172.21.15.45:0/1121014634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770017070 con 0x7f277815bac0 2024-04-25T10:31:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f2775600640 1 -- 172.21.15.45:0/1121014634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277000ce10 con 0x7f277815bac0 2024-04-25T10:31:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f2775600640 1 -- 172.21.15.45:0/1121014634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000a7e0 con 0x7f277815bac0 2024-04-25T10:31:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780af6a0 con 0x7f277815bac0 2024-04-25T10:31:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.182+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27780afc60 con 0x7f277815bac0 2024-04-25T10:31:36.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.186+0000 7f2775600640 1 -- 172.21.15.45:0/1121014634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f277001a070 con 0x7f277815bac0 2024-04-25T10:31:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.186+0000 7f2775600640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.186+0000 7f2775600640 1 -- 172.21.15.45:0/1121014634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2770084550 con 0x7f277815bac0 2024-04-25T10:31:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.186+0000 7f277d000640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.186+0000 7f2782de4000 1 Processor -- start 2024-04-25T10:31:36.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.186+0000 7f2782de4000 1 -- start start 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f277d000640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2754066f60 0x7f2754069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2768009b60 tx=0x7f2768009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7f1aef20 con 0x564c7f3abaa0 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7f137000 con 0x564c7f361600 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c7f1aafc0 con 0x564c7f3ab180 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58658/0 (socket says 172.21.15.45:58658) 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 -- 172.21.15.45:0/2120701704 learned_addr learned my addr 172.21.15.45:0/2120701704 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f277da00640 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f277d000640 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 -- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 msgr2=0x564c7f3ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 -- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 msgr2=0x564c7f3619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 -- 172.21.15.45:0/2120701704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7f18aba0 con 0x564c7f3abaa0 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f277da00640 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276fe00640 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3610c0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f276001ec70 tx=0x7f27600a9070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7e558a004b1c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276d600640 1 -- 172.21.15.45:0/2120701704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600b4040 con 0x564c7f3abaa0 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276d600640 1 -- 172.21.15.45:0/2120701704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600bf070 con 0x564c7f3abaa0 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f276d600640 1 -- 172.21.15.45:0/2120701704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600af840 con 0x564c7f3abaa0 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- 172.21.15.45:0/2120701704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 msgr2=0x564c7f3610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3610c0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f276001ec70 tx=0x7f27600a9070 comp rx=0 tx=0).stop 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- 172.21.15.45:0/2120701704 shutdown_connections 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3610c0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 --2- 172.21.15.45:0/2120701704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- 172.21.15.45:0/2120701704 >> 172.21.15.45:0/2120701704 conn(0x564c7f367760 msgr2=0x564c7f35f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- 172.21.15.45:0/2120701704 shutdown_connections 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- 172.21.15.45:0/2120701704 wait complete. 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 -- start start 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.190+0000 7f2782de4000 1 Processor -- start 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 0x564c7f3ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f3abaa0 0x564c7f3acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7f26c9c0 con 0x564c7f361600 2024-04-25T10:31:36.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7f1695d0 con 0x564c7f3abaa0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c7f135ac0 con 0x564c7f3ab180 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 10 client.0 _ll_register_callbacks cb 0x564c7f34ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 0x564c7f3ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 0x564c7f3ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58674/0 (socket says 172.21.15.45:58674) 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 -- 172.21.15.45:0/3762394528 learned_addr learned my addr 172.21.15.45:0/3762394528 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f276fe00640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277da00640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f3abaa0 0x564c7f3acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 -- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 msgr2=0x564c7f3ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 -- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f3abaa0 msgr2=0x564c7f3acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f3abaa0 0x564c7f3acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 -- 172.21.15.45:0/3762394528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c7f1f55a0 con 0x564c7f361600 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f277d000640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 0x564c7f3ac090 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f2768011fc0 tx=0x7f27680046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f92159fe14d8f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:36.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 1 -- 172.21.15.45:0/3762394528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768027020 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 1 -- 172.21.15.45:0/3762394528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2768023450 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27600bf070 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 1 -- 172.21.15.45:0/3762394528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2768023650 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 10 client.11097 fetch_fsmap learned FSMap version 131 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x564c7f1862b0 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 1 -- 172.21.15.45:0/3762394528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600bf070 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 1 -- 172.21.15.45:0/3762394528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f27680237f0 con 0x564c7f361600 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 10 client.11097 fetch_fsmap finished waiting for FSMap version 131 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 10 client.11097 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 20 client.11097 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f3beb10 0x564c7f3c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 4 client.11097 mds_command: new command op to 10881 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 --> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c7f14ac70 con 0x564c7f3beb10 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f276fe00640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f3beb10 0x564c7f3c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f276fe00640 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f3beb10 0x564c7f3c0f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cc415af47dd0d5 server_cookie=107d83a1fea44565 in_seq=0 out_seq=0 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 10 client.11097 ms_handle_connect on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 1 -- 172.21.15.45:0/3762394528 <== mds.0 v2:172.21.15.45:6833/1455178901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c7f14ac70 con 0x564c7f3beb10 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2766000640 10 client.11097 handle_command_reply: tid=0 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.194+0000 7f2782de4000 1 client.11097 shutdown 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 msgr2=0x564c7f3ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 0x564c7f3ac090 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f2768011fc0 tx=0x7f27680046d0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 shutdown_connections 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f3beb10 0x564c7f3c0f00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f3abaa0 0x564c7f3acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 --2- 172.21.15.45:0/3762394528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f361600 0x564c7f3ac090 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 >> 172.21.15.45:0/3762394528 conn(0x564c7f367760 msgr2=0x564c7f3600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2766000640 0 client.11097 ms_handle_reset on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 shutdown_connections 2024-04-25T10:31:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 1 -- 172.21.15.45:0/3762394528 wait complete. 2024-04-25T10:31:36.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.198+0000 7f2782de4000 20 client.11097 trim_cache size 0 max 16384 2024-04-25T10:31:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 Processor -- start 2024-04-25T10:31:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- start start 2024-04-25T10:31:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781fead0 con 0x564c7f3abaa0 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781fdc50 con 0x564c7f361600 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27781fd050 con 0x564c7f3ab180 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58686/0 (socket says 172.21.15.45:58686) 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 -- 172.21.15.45:0/1045837639 learned_addr learned my addr 172.21.15.45:0/1045837639 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f276fe00640 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277d000640 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 -- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 msgr2=0x564c7f3ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 -- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 msgr2=0x564c7f3b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 -- 172.21.15.45:0/1045837639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7f137250 con 0x564c7f3abaa0 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f277da00640 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3aee00 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f2770054db0 tx=0x7f277005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe6d5fd503e163e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f276cc00640 1 -- 172.21.15.45:0/1045837639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277006c070 con 0x564c7f3abaa0 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f276cc00640 1 -- 172.21.15.45:0/1045837639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277005b0b0 con 0x564c7f3abaa0 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f276cc00640 1 -- 172.21.15.45:0/1045837639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277005d040 con 0x564c7f3abaa0 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- 172.21.15.45:0/1045837639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 msgr2=0x564c7f3aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3aee00 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f2770054db0 tx=0x7f277005b040 comp rx=0 tx=0).stop 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- 172.21.15.45:0/1045837639 shutdown_connections 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f3aee00 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f3ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 --2- 172.21.15.45:0/1045837639 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f3b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- 172.21.15.45:0/1045837639 >> 172.21.15.45:0/1045837639 conn(0x564c7f367760 msgr2=0x564c7f366490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- 172.21.15.45:0/1045837639 shutdown_connections 2024-04-25T10:31:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.210+0000 7f2782de4000 1 -- 172.21.15.45:0/1045837639 wait complete. 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- start start 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 Processor -- start 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f410510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f412a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f414fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7f1b3550 con 0x564c7f3abaa0 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781fd650 con 0x564c7f361600 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c7f155c90 con 0x564c7f3ab180 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 10 client.0 _ll_register_callbacks cb 0x564c7f32d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f414fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f414fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58694/0 (socket says 172.21.15.45:58694) 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 -- 172.21.15.45:0/1865367240 learned_addr learned my addr 172.21.15.45:0/1865367240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277d000640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f410510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f276fe00640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f412a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 -- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 msgr2=0x564c7f412a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f412a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 -- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 msgr2=0x564c7f410510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f410510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 -- 172.21.15.45:0/1865367240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c7f143b90 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277da00640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f414fb0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f27700512d0 tx=0x7f2770001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4b73c5ca113bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2766000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2766000640 1 -- 172.21.15.45:0/1865367240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277006c040 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2766000640 1 -- 172.21.15.45:0/1865367240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27700840b0 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f277005b0b0 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2766000640 1 -- 172.21.15.45:0/1865367240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27700842b0 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 10 client.11103 fetch_fsmap learned FSMap version 131 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x564c7f156010 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2766000640 1 -- 172.21.15.45:0/1865367240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277005b0b0 con 0x564c7f3abaa0 2024-04-25T10:31:36.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2766000640 1 -- 172.21.15.45:0/1865367240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 131) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f277006cb50 con 0x564c7f3abaa0 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 10 client.11103 fetch_fsmap finished waiting for FSMap version 131 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 10 client.11103 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 20 client.11103 populate_metadata read hostname 'smithi045' 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f4154f0 0x564c7f4178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 4 client.11103 mds_command: new command op to 10881 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 --> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c7f1bc460 con 0x564c7f4154f0 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.214+0000 7f277d000640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f4154f0 0x564c7f4178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277d000640 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f4154f0 0x564c7f4178e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada7b5288430d686 server_cookie=e4750f27f09b960a in_seq=0 out_seq=0 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2766000640 10 client.11103 ms_handle_connect on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2766000640 1 -- 172.21.15.45:0/1865367240 <== mds.0 v2:172.21.15.45:6833/1455178901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c7f1bc460 con 0x564c7f4154f0 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2766000640 10 client.11103 handle_command_reply: tid=0 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 client.11103 shutdown 2024-04-25T10:31:36.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 msgr2=0x564c7f414fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f414fb0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f27700512d0 tx=0x7f2770001260 comp rx=0 tx=0).stop 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 shutdown_connections 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:6833/1455178901,v1:172.21.15.45:6835/1455178901] conn(0x564c7f4154f0 0x564c7f4178e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564c7f3abaa0 0x564c7f414fb0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x564c7f3ab180 0x564c7f412a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 --2- 172.21.15.45:0/1865367240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564c7f361600 0x564c7f410510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2766000640 0 client.11103 ms_handle_reset on v2:172.21.15.45:6833/1455178901 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 >> 172.21.15.45:0/1865367240 conn(0x564c7f367760 msgr2=0x564c7f366ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 shutdown_connections 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 1 -- 172.21.15.45:0/1865367240 wait complete. 2024-04-25T10:31:36.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f2782de4000 20 client.11103 trim_cache size 0 max 16384 2024-04-25T10:31:36.228 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-25T10:31:36.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2754066f60 msgr2=0x7f2754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2754066f60 0x7f2754069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2768009b60 tx=0x7f2768009190 comp rx=0 tx=0).stop 2024-04-25T10:31:36.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 msgr2=0x7f27780cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 0x7f27780cd990 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f277000c700 tx=0x7f277000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 shutdown_connections 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f277815bac0 0x7f27780cd990 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27780aef80 0x7f27781520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 --2- 172.21.15.45:0/1121014634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2778070350 0x7f277813f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 >> 172.21.15.45:0/1121014634 conn(0x7f27780669a0 msgr2=0x7f277815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 shutdown_connections 2024-04-25T10:31:36.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.218+0000 7f277fa00640 1 -- 172.21.15.45:0/1121014634 wait complete. 2024-04-25T10:31:36.240 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:31:36.240 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:36.290 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:36.353 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:31:36.353 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:36.353 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:36.362 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:36.362 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:36.411 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 Processor -- start 2024-04-25T10:31:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 -- start start 2024-04-25T10:31:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fab2415b9e0 0x7fab240cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab240d1c30 0x7fab240cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240ccfb0 con 0x7fab2415bdc0 2024-04-25T10:31:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240cd290 con 0x7fab240d1c30 2024-04-25T10:31:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2c400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240cd570 con 0x7fab2415b9e0 2024-04-25T10:31:36.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab240d1c30 0x7fab240cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fab240d1c30 0x7fab240cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58696/0 (socket says 172.21.15.45:58696) 2024-04-25T10:31:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 -- 172.21.15.45:0/1372720331 learned_addr learned my addr 172.21.15.45:0/1372720331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab29a00640 1 --2- 172.21.15.45:0/1372720331 >> v2:172.21.15.117:3300/0 conn(0x7fab2415b9e0 0x7fab240cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 -- 172.21.15.45:0/1372720331 >> v2:172.21.15.117:3300/0 conn(0x7fab2415b9e0 msgr2=0x7fab240cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 --2- 172.21.15.45:0/1372720331 >> v2:172.21.15.117:3300/0 conn(0x7fab2415b9e0 0x7fab240cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 -- 172.21.15.45:0/1372720331 >> v1:172.21.15.117:6789/0 conn(0x7fab2415bdc0 legacy=0x7fab240d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab2a400640 1 -- 172.21.15.45:0/1372720331 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240cd850 con 0x7fab240d1c30 2024-04-25T10:31:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.598+0000 7fab29a00640 1 --2- 172.21.15.45:0/1372720331 >> v2:172.21.15.117:3300/0 conn(0x7fab2415b9e0 0x7fab240cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2a400640 1 --2- 172.21.15.45:0/1372720331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab240d1c30 0x7fab240cc950 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fab24070da0 tx=0x7fab1c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6d694fde85b725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab23e00640 1 -- 172.21.15.45:0/1372720331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0a5690 con 0x7fab240d1c30 2024-04-25T10:31:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab23e00640 1 -- 172.21.15.45:0/1372720331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c0a5830 con 0x7fab240d1c30 2024-04-25T10:31:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab23e00640 1 -- 172.21.15.45:0/1372720331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0a5c00 con 0x7fab240d1c30 2024-04-25T10:31:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 -- 172.21.15.45:0/1372720331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab240d1c30 msgr2=0x7fab240cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 --2- 172.21.15.45:0/1372720331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab240d1c30 0x7fab240cc950 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fab24070da0 tx=0x7fab1c01c920 comp rx=0 tx=0).stop 2024-04-25T10:31:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 -- 172.21.15.45:0/1372720331 shutdown_connections 2024-04-25T10:31:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 --2- 172.21.15.45:0/1372720331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab240d1c30 0x7fab240cc950 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 --2- 172.21.15.45:0/1372720331 >> v2:172.21.15.117:3300/0 conn(0x7fab2415b9e0 0x7fab240cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 -- 172.21.15.45:0/1372720331 >> 172.21.15.45:0/1372720331 conn(0x7fab2409a9d0 msgr2=0x7fab2409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 -- 172.21.15.45:0/1372720331 shutdown_connections 2024-04-25T10:31:36.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.602+0000 7fab2c400640 1 -- 172.21.15.45:0/1372720331 wait complete. 2024-04-25T10:31:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 Processor -- start 2024-04-25T10:31:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- start start 2024-04-25T10:31:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab240d1c30 0x7fab2414dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 0x7fab2415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2415bdc0 0x7fab240ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240c6630 con 0x7fab2415b9e0 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240c64b0 con 0x7fab2415bdc0 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab240c67b0 con 0x7fab240d1c30 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 0x7fab2415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 0x7fab2415ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58700/0 (socket says 172.21.15.45:58700) 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 -- 172.21.15.45:0/3028925085 learned_addr learned my addr 172.21.15.45:0/3028925085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29a00640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab240d1c30 0x7fab2414dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2a400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2415bdc0 0x7fab240ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 -- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab240d1c30 msgr2=0x7fab2414dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab240d1c30 0x7fab2414dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 -- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2415bdc0 msgr2=0x7fab240ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2415bdc0 0x7fab240ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 -- 172.21.15.45:0/3028925085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240cb180 con 0x7fab2415b9e0 2024-04-25T10:31:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab29000640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 0x7fab2415ee00 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fab1000b440 tx=0x7fab1000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10015020 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab10011830 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10011c20 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240cb460 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab240cb740 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.606+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2404ef30 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.610+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab10018020 con 0x7fab2415b9e0 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.610+0000 7fab21600640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab04066f60 0x7fab04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.610+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab10084bd0 con 0x7fab2415b9e0 2024-04-25T10:31:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.610+0000 7fab29a00640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab04066f60 0x7fab04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.610+0000 7fab29a00640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab04066f60 0x7fab04069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fab2415a9a0 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.614+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab1001f030 con 0x7fab2415b9e0 2024-04-25T10:31:36.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.778+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab240caf60 con 0x7fab2415b9e0 2024-04-25T10:31:36.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.778+0000 7fab21600640 1 -- 172.21.15.45:0/3028925085 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16603 (secure 0 0 0) 0x7fab100554f0 con 0x7fab2415b9e0 2024-04-25T10:31:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:36.782 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":182,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:24.430055+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab04066f60 msgr2=0x7fab04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab04066f60 0x7fab04069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fab2415a9a0 tx=0x7fab18008040 comp rx=0 tx=0).stop 2024-04-25T10:31:36.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 msgr2=0x7fab2415ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 0x7fab2415ee00 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fab1000b440 tx=0x7fab1000b900 comp rx=0 tx=0).stop 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 shutdown_connections 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fab04066f60 0x7fab04069420 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2415bdc0 0x7fab240ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2415b9e0 0x7fab2415ee00 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 --2- 172.21.15.45:0/3028925085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab240d1c30 0x7fab2414dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 >> 172.21.15.45:0/3028925085 conn(0x7fab2409a9d0 msgr2=0x7fab2406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 shutdown_connections 2024-04-25T10:31:36.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.782+0000 7fab2c400640 1 -- 172.21.15.45:0/3028925085 wait complete. 2024-04-25T10:31:36.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:31:21.231280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:31:21.811204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:36.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 Processor -- start 2024-04-25T10:31:36.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 -- start start 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1f40aef80 0x7fa1f415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f4070350 0x7fa1f415b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1f415bac0 0x7fa1f40b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f415b080 con 0x7fa1f40aef80 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f415adb0 con 0x7fa1f415bac0 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f415eb60 con 0x7fa1f4070350 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1f40aef80 0x7fa1f415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1f40aef80 0x7fa1f415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58706/0 (socket says 172.21.15.45:58706) 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 -- 172.21.15.45:0/2108457802 learned_addr learned my addr 172.21.15.45:0/2108457802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f2800640 1 --2- 172.21.15.45:0/2108457802 >> v2:172.21.15.117:3300/0 conn(0x7fa1f415bac0 0x7fa1f40b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1e9400640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f4070350 0x7fa1f415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 -- 172.21.15.45:0/2108457802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f4070350 msgr2=0x7fa1f415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f4070350 0x7fa1f415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 -- 172.21.15.45:0/2108457802 >> v2:172.21.15.117:3300/0 conn(0x7fa1f415bac0 msgr2=0x7fa1f40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 --2- 172.21.15.45:0/2108457802 >> v2:172.21.15.117:3300/0 conn(0x7fa1f415bac0 0x7fa1f40b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.946+0000 7fa1f1e00640 1 -- 172.21.15.45:0/2108457802 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f415ee40 con 0x7fa1f40aef80 2024-04-25T10:31:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f2800640 1 --2- 172.21.15.45:0/2108457802 >> v2:172.21.15.117:3300/0 conn(0x7fa1f415bac0 0x7fa1f40b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:36.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1e9400640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f4070350 0x7fa1f415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:36.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f1e00640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f40aef80 0x7fa1f415e480 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40cc330 tx=0x7fa1e401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ee121af817cf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f1400640 1 -- 172.21.15.45:0/2108457802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40a5780 con 0x7fa1f40aef80 2024-04-25T10:31:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f1400640 1 -- 172.21.15.45:0/2108457802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e40a5920 con 0x7fa1f40aef80 2024-04-25T10:31:36.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f1400640 1 -- 172.21.15.45:0/2108457802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40a5cf0 con 0x7fa1f40aef80 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 -- 172.21.15.45:0/2108457802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f40aef80 msgr2=0x7fa1f415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f40aef80 0x7fa1f415e480 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40cc330 tx=0x7fa1e401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 -- 172.21.15.45:0/2108457802 shutdown_connections 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/2108457802 >> v2:172.21.15.117:3300/0 conn(0x7fa1f415bac0 0x7fa1f40b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f4070350 0x7fa1f415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/2108457802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f40aef80 0x7fa1f415e480 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 -- 172.21.15.45:0/2108457802 >> 172.21.15.45:0/2108457802 conn(0x7fa1f40669a0 msgr2=0x7fa1f4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 -- 172.21.15.45:0/2108457802 shutdown_connections 2024-04-25T10:31:36.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.950+0000 7fa1f8c00640 1 -- 172.21.15.45:0/2108457802 wait complete. 2024-04-25T10:31:36.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 Processor -- start 2024-04-25T10:31:36.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 -- start start 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 0x7fa1f41624d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f40aef80 0x7fa1f41fa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f415bac0 0x7fa1f41fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406d570 con 0x7fa1f4070350 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406d3f0 con 0x7fa1f40aef80 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406d6f0 con 0x7fa1f415bac0 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 0x7fa1f41624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 0x7fa1f41624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58720/0 (socket says 172.21.15.45:58720) 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 -- 172.21.15.45:0/1080577337 learned_addr learned my addr 172.21.15.45:0/1080577337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f2800640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f415bac0 0x7fa1f41fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1e9400640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f40aef80 0x7fa1f41fa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 -- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f415bac0 msgr2=0x7fa1f41fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f415bac0 0x7fa1f41fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 -- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f40aef80 msgr2=0x7fa1f41fa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f40aef80 0x7fa1f41fa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 -- 172.21.15.45:0/1080577337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4162a10 con 0x7fa1f4070350 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f1e00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 0x7fa1f41624d0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40af040 tx=0x7fa1e4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40b8670 con 0x7fa1f4070350 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e40be070 con 0x7fa1f4070350 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40b8a70 con 0x7fa1f4070350 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.954+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f4163910 con 0x7fa1f4070350 2024-04-25T10:31:36.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f4164f20 con 0x7fa1f4070350 2024-04-25T10:31:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f404ef30 con 0x7fa1f4070350 2024-04-25T10:31:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1e40b8c10 con 0x7fa1f4070350 2024-04-25T10:31:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1eaa00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa1d0066f60 0x7fa1d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:36.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1e4120190 con 0x7fa1f4070350 2024-04-25T10:31:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1e9400640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa1d0066f60 0x7fa1d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:36.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.958+0000 7fa1e9400640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa1d0066f60 0x7fa1d0069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4067080 tx=0x7fa1dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:36.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:36.962+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1e40165e0 con 0x7fa1f4070350 2024-04-25T10:31:37.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.126+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1f40722c0 con 0x7fa1f4070350 2024-04-25T10:31:37.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1eaa00640 1 -- 172.21.15.45:0/1080577337 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1e40f0ab0 con 0x7fa1f4070350 2024-04-25T10:31:37.131 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa1d0066f60 msgr2=0x7fa1d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa1d0066f60 0x7fa1d0069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4067080 tx=0x7fa1dc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 msgr2=0x7fa1f41624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 0x7fa1f41624d0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40af040 tx=0x7fa1e4019790 comp rx=0 tx=0).stop 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 shutdown_connections 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa1d0066f60 0x7fa1d0069420 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1f415bac0 0x7fa1f41fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f40aef80 0x7fa1f41fa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 --2- 172.21.15.45:0/1080577337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1f4070350 0x7fa1f41624d0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 >> 172.21.15.45:0/1080577337 conn(0x7fa1f40669a0 msgr2=0x7fa1f415d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 shutdown_connections 2024-04-25T10:31:37.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.130+0000 7fa1f8c00640 1 -- 172.21.15.45:0/1080577337 wait complete. 2024-04-25T10:31:37.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:37.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca9e00640 1 Processor -- start 2024-04-25T10:31:37.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca9e00640 1 -- start start 2024-04-25T10:31:37.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40b38d0 0x7f7ca415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca9e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca415b080 con 0x7f7ca40aef80 2024-04-25T10:31:37.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca9e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca415adb0 con 0x7f7ca4070350 2024-04-25T10:31:37.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca9e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca415eb60 con 0x7f7ca40b38d0 2024-04-25T10:31:37.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca2a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7ca40aef80 0x7f7ca415b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33988/0 (socket says 172.21.15.45:33988) 2024-04-25T10:31:37.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca2a00640 1 -- 172.21.15.45:0/1658039410 learned_addr learned my addr 172.21.15.45:0/1658039410 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:37.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.294+0000 7f7ca3400640 1 --2- 172.21.15.45:0/1658039410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40b38d0 0x7f7ca415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca2000640 1 -- 172.21.15.45:0/1658039410 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575520714 0 0) 0x7f7ca415b080 con 0x7f7ca40aef80 2024-04-25T10:31:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca2000640 1 -- 172.21.15.45:0/1658039410 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c88003530 con 0x7f7ca40aef80 2024-04-25T10:31:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca3400640 1 -- 172.21.15.45:0/1658039410 >> v1:172.21.15.117:6789/0 conn(0x7f7ca4070350 legacy=0x7f7ca40b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca2000640 1 -- 172.21.15.45:0/1658039410 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2848373125 0 0) 0x7f7c88003530 con 0x7f7ca40aef80 2024-04-25T10:31:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca3400640 1 -- 172.21.15.45:0/1658039410 >> v1:172.21.15.45:6789/0 conn(0x7f7ca40aef80 legacy=0x7f7ca415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca3400640 1 -- 172.21.15.45:0/1658039410 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca415ee40 con 0x7f7ca40b38d0 2024-04-25T10:31:37.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca3400640 1 --2- 172.21.15.45:0/1658039410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40b38d0 0x7f7ca415e480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009dc0 tx=0x7f7c9c01c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=990320e4d4bf19b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca2000640 1 -- 172.21.15.45:0/1658039410 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94001680 con 0x7f7ca40b38d0 2024-04-25T10:31:37.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca2000640 1 -- 172.21.15.45:0/1658039410 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c0a6510 con 0x7f7ca40b38d0 2024-04-25T10:31:37.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.298+0000 7f7ca2000640 1 -- 172.21.15.45:0/1658039410 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0a68c0 con 0x7f7ca40b38d0 2024-04-25T10:31:37.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- 172.21.15.45:0/1658039410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40b38d0 msgr2=0x7f7ca415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/1658039410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40b38d0 0x7f7ca415e480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009dc0 tx=0x7f7c9c01c760 comp rx=0 tx=0).stop 2024-04-25T10:31:37.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- 172.21.15.45:0/1658039410 shutdown_connections 2024-04-25T10:31:37.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/1658039410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40b38d0 0x7f7ca415e480 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- 172.21.15.45:0/1658039410 >> 172.21.15.45:0/1658039410 conn(0x7f7ca40669a0 msgr2=0x7f7ca4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- 172.21.15.45:0/1658039410 shutdown_connections 2024-04-25T10:31:37.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- 172.21.15.45:0/1658039410 wait complete. 2024-04-25T10:31:37.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 Processor -- start 2024-04-25T10:31:37.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- start start 2024-04-25T10:31:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ca4070350 0x7f7ca414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40aef80 0x7f7ca41500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 0x7f7ca4155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41689b0 con 0x7f7ca414b5f0 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4168830 con 0x7f7ca4070350 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca9e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4168b30 con 0x7f7ca40aef80 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 0x7f7ca4155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ca4070350 0x7f7ca414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 0x7f7ca4155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58722/0 (socket says 172.21.15.45:58722) 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 -- 172.21.15.45:0/4124604225 learned_addr learned my addr 172.21.15.45:0/4124604225 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:37.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7c9be00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40aef80 0x7f7ca41500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 -- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40aef80 msgr2=0x7f7ca41500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40aef80 0x7f7ca41500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 -- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ca4070350 msgr2=0x7f7ca414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ca4070350 0x7f7ca414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7ca3400640 1 -- 172.21.15.45:0/4124604225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4154040 con 0x7f7ca414b5f0 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.302+0000 7f7c9be00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40aef80 0x7f7ca41500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:37.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7ca3400640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 0x7f7ca4155280 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c01ae10 tx=0x7f7c9c019500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009450 con 0x7f7ca414b5f0 2024-04-25T10:31:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c0ba680 con 0x7f7ca414b5f0 2024-04-25T10:31:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca4154320 con 0x7f7ca414b5f0 2024-04-25T10:31:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0baa30 con 0x7f7ca414b5f0 2024-04-25T10:31:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca4151cb0 con 0x7f7ca414b5f0 2024-04-25T10:31:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca404ef30 con 0x7f7ca414b5f0 2024-04-25T10:31:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c9c0c2030 con 0x7f7ca414b5f0 2024-04-25T10:31:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7c9b400640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c80066f90 0x7f7c80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c9c11fc70 con 0x7f7ca414b5f0 2024-04-25T10:31:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.306+0000 7f7ca2a00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c80066f90 0x7f7c80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.310+0000 7f7ca2a00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c80066f90 0x7f7c80069450 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7c940017f0 tx=0x7f7c94001240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.310+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c9c0c9020 con 0x7f7ca414b5f0 2024-04-25T10:31:37.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ca40b2fa0 con 0x7f7ca414b5f0 2024-04-25T10:31:37.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7c9b400640 1 -- 172.21.15.45:0/4124604225 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c9c0164a0 con 0x7f7ca414b5f0 2024-04-25T10:31:37.476 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c80066f90 msgr2=0x7f7c80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c80066f90 0x7f7c80069450 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7c940017f0 tx=0x7f7c94001240 comp rx=0 tx=0).stop 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 msgr2=0x7f7ca4155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 0x7f7ca4155280 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c01ae10 tx=0x7f7c9c019500 comp rx=0 tx=0).stop 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 shutdown_connections 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7c80066f90 0x7f7c80069450 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca414b5f0 0x7f7ca4155280 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ca40aef80 0x7f7ca41500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 --2- 172.21.15.45:0/4124604225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ca4070350 0x7f7ca414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 >> 172.21.15.45:0/4124604225 conn(0x7f7ca40669a0 msgr2=0x7f7ca415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 shutdown_connections 2024-04-25T10:31:37.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.474+0000 7f7ca9e00640 1 -- 172.21.15.45:0/4124604225 wait complete. 2024-04-25T10:31:37.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f600c400640 1 Processor -- start 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f600c400640 1 -- start start 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f600c400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f600406b6a0 0x7f6004162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f600c400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6004162980 con 0x7f600406ba80 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f600c400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600406b1d0 con 0x7f60040b1300 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f600c400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600415bf60 con 0x7f600406b6a0 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f6009000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f600406ba80 0x7f60040d0a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33990/0 (socket says 172.21.15.45:33990) 2024-04-25T10:31:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f6009a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f600406b6a0 0x7f6004162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f6009000640 1 -- 172.21.15.45:0/2841453144 learned_addr learned my addr 172.21.15.45:0/2841453144 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f6009a00640 1 -- 172.21.15.45:0/2841453144 >> v1:172.21.15.117:6789/0 conn(0x7f60040b1300 legacy=0x7f60040b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f6009a00640 1 -- 172.21.15.45:0/2841453144 >> v1:172.21.15.45:6789/0 conn(0x7f600406ba80 legacy=0x7f60040d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.638+0000 7f6009a00640 1 -- 172.21.15.45:0/2841453144 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040cff70 con 0x7f600406b6a0 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f6009a00640 1 --2- 172.21.15.45:0/2841453144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004162440 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80097f0 tx=0x7f5ff801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd3c06ce6b3ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f6003e00640 1 -- 172.21.15.45:0/2841453144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80b0070 con 0x7f600406b6a0 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f6003e00640 1 -- 172.21.15.45:0/2841453144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff80ab600 con 0x7f600406b6a0 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f6003e00640 1 -- 172.21.15.45:0/2841453144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80ab9d0 con 0x7f600406b6a0 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 -- 172.21.15.45:0/2841453144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 msgr2=0x7f6004162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 --2- 172.21.15.45:0/2841453144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004162440 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80097f0 tx=0x7f5ff801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 -- 172.21.15.45:0/2841453144 shutdown_connections 2024-04-25T10:31:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 --2- 172.21.15.45:0/2841453144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004162440 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 -- 172.21.15.45:0/2841453144 >> 172.21.15.45:0/2841453144 conn(0x7f60040d19a0 msgr2=0x7f60040d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 -- 172.21.15.45:0/2841453144 shutdown_connections 2024-04-25T10:31:37.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.642+0000 7f600c400640 1 -- 172.21.15.45:0/2841453144 wait complete. 2024-04-25T10:31:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 Processor -- start 2024-04-25T10:31:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 -- start start 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f600406ba80 0x7f600414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60040b1300 0x7f60040c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600406be90 con 0x7f600406b6a0 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600406c010 con 0x7f600406ba80 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040b0b30 con 0x7f60040b1300 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58736/0 (socket says 172.21.15.45:58736) 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 -- 172.21.15.45:0/2233662850 learned_addr learned my addr 172.21.15.45:0/2233662850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009000640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f600406ba80 0x7f600414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600a400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60040b1300 0x7f60040c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 -- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60040b1300 msgr2=0x7f60040c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60040b1300 0x7f60040c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 -- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f600406ba80 msgr2=0x7f600414fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f600406ba80 0x7f600414fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 -- 172.21.15.45:0/2233662850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004153190 con 0x7f600406b6a0 2024-04-25T10:31:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6009a00640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004157e20 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80190b0 tx=0x7f5ff801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80b0050 con 0x7f600406b6a0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff80bc070 con 0x7f600406b6a0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80b5a70 con 0x7f600406b6a0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040c6150 con 0x7f600406b6a0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.646+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040c6710 con 0x7f600406b6a0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.650+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600404ef30 con 0x7f600406b6a0 2024-04-25T10:31:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.650+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ff80ab480 con 0x7f600406b6a0 2024-04-25T10:31:37.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.650+0000 7f6001600640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe4066f90 0x7f5fe4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:37.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.650+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ff811f850 con 0x7f600406b6a0 2024-04-25T10:31:37.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.650+0000 7f6009000640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe4066f90 0x7f5fe4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.650+0000 7f6009000640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe4066f90 0x7f5fe4069450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6004162280 tx=0x7f5ff0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.654+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ff8016090 con 0x7f600406b6a0 2024-04-25T10:31:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.834+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6004070d90 con 0x7f600406b6a0 2024-04-25T10:31:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.834+0000 7f6001600640 1 -- 172.21.15.45:0/2233662850 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v182) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ff80f0170 con 0x7f600406b6a0 2024-04-25T10:31:37.837 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe4066f90 msgr2=0x7f5fe4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe4066f90 0x7f5fe4069450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6004162280 tx=0x7f5ff0009040 comp rx=0 tx=0).stop 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 msgr2=0x7f6004157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004157e20 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80190b0 tx=0x7f5ff801ace0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 shutdown_connections 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5fe4066f90 0x7f5fe4069450 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60040b1300 0x7f60040c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f600406ba80 0x7f600414fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 --2- 172.21.15.45:0/2233662850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600406b6a0 0x7f6004157e20 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 >> 172.21.15.45:0/2233662850 conn(0x7f60040d19a0 msgr2=0x7f6004153c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 shutdown_connections 2024-04-25T10:31:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:37.838+0000 7f600c400640 1 -- 172.21.15.45:0/2233662850 wait complete. 2024-04-25T10:31:37.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:31:38.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f1855e00640 1 Processor -- start 2024-04-25T10:31:38.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f1855e00640 1 -- start start 2024-04-25T10:31:38.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f1855e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18500cfd90 0x7f18500ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f1855e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185015f3b0 con 0x7f18500cd110 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f1855e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18500ca810 con 0x7f18500b0f70 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f1855e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185015ef20 con 0x7f18500cfd90 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184e000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f18500cfd90 0x7f18500ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184f400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f18500cd110 0x7f18500cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34000/0 (socket says 172.21.15.45:34000) 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184f400640 1 -- 172.21.15.45:0/3809373371 learned_addr learned my addr 172.21.15.45:0/3809373371 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:38.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184d600640 1 -- 172.21.15.45:0/3809373371 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447065921 0 0) 0x7f185015f3b0 con 0x7f18500cd110 2024-04-25T10:31:38.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184e000640 1 -- 172.21.15.45:0/3809373371 >> v1:172.21.15.117:6789/0 conn(0x7f18500b0f70 legacy=0x7f18500b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184e000640 1 -- 172.21.15.45:0/3809373371 >> v1:172.21.15.45:6789/0 conn(0x7f18500cd110 legacy=0x7f18500cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.002+0000 7f184e000640 1 -- 172.21.15.45:0/3809373371 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185015e5f0 con 0x7f18500cfd90 2024-04-25T10:31:38.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f184e000640 1 --2- 172.21.15.45:0/3809373371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cfd90 0x7f18500ccbd0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f185015f0a0 tx=0x7f183c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb565521d49fa88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:38.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f184d600640 1 -- 172.21.15.45:0/3809373371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b0070 con 0x7f18500cfd90 2024-04-25T10:31:38.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f184d600640 1 -- 172.21.15.45:0/3809373371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c0ab600 con 0x7f18500cfd90 2024-04-25T10:31:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f184d600640 1 -- 172.21.15.45:0/3809373371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0ab9d0 con 0x7f18500cfd90 2024-04-25T10:31:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 -- 172.21.15.45:0/3809373371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cfd90 msgr2=0x7f18500ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 --2- 172.21.15.45:0/3809373371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cfd90 0x7f18500ccbd0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f185015f0a0 tx=0x7f183c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 -- 172.21.15.45:0/3809373371 shutdown_connections 2024-04-25T10:31:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 --2- 172.21.15.45:0/3809373371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cfd90 0x7f18500ccbd0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 -- 172.21.15.45:0/3809373371 >> 172.21.15.45:0/3809373371 conn(0x7f18500b1d50 msgr2=0x7f18500b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 -- 172.21.15.45:0/3809373371 shutdown_connections 2024-04-25T10:31:38.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 -- 172.21.15.45:0/3809373371 wait complete. 2024-04-25T10:31:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 Processor -- start 2024-04-25T10:31:38.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.006+0000 7f1855e00640 1 -- start start 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18500b0f70 0x7f18500b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 0x7f18500c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18500cfd90 0x7f1850099480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185016bf00 con 0x7f18500cd110 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185016bd80 con 0x7f18500cfd90 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185016c080 con 0x7f18500b0f70 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 0x7f18500c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 0x7f18500c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58758/0 (socket says 172.21.15.45:58758) 2024-04-25T10:31:38.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 -- 172.21.15.45:0/3960870815 learned_addr learned my addr 172.21.15.45:0/3960870815 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184ea00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18500b0f70 0x7f18500b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184f400640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18500cfd90 0x7f1850099480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 -- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18500b0f70 msgr2=0x7f18500b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18500b0f70 0x7f18500b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 -- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18500cfd90 msgr2=0x7f1850099480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18500cfd90 0x7f1850099480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 -- 172.21.15.45:0/3960870815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18500c9300 con 0x7f18500cd110 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184f400640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18500cfd90 0x7f1850099480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f184e000640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 0x7f18500c8920 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f183c0ae040 tx=0x7f18500cc940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b8020 con 0x7f18500cd110 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c0b4610 con 0x7f18500cd110 2024-04-25T10:31:38.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b0040 con 0x7f18500cd110 2024-04-25T10:31:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18500c95e0 con 0x7f18500cd110 2024-04-25T10:31:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18500c9ba0 con 0x7f18500cd110 2024-04-25T10:31:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.010+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185004ef30 con 0x7f18500cd110 2024-04-25T10:31:38.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.014+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f183c0bb020 con 0x7f18500cd110 2024-04-25T10:31:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.014+0000 7f1847400640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1830066f90 0x7f1830069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.014+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f183c11ec40 con 0x7f18500cd110 2024-04-25T10:31:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.014+0000 7f184ea00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1830066f90 0x7f1830069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.014+0000 7f184ea00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1830066f90 0x7f1830069450 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1840000f70 tx=0x7f1840009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:38.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.018+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f183c0c2030 con 0x7f18500cd110 2024-04-25T10:31:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18500cc9b0 con 0x7f18500cd110 2024-04-25T10:31:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1847400640 1 -- 172.21.15.45:0/3960870815 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v182) v1 ==== 80+0+0 (secure 0 0 0) 0x7f183c016090 con 0x7f18500cd110 2024-04-25T10:31:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1830066f90 msgr2=0x7f1830069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1830066f90 0x7f1830069450 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1840000f70 tx=0x7f1840009040 comp rx=0 tx=0).stop 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 msgr2=0x7f18500c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 0x7f18500c8920 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f183c0ae040 tx=0x7f18500cc940 comp rx=0 tx=0).stop 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 shutdown_connections 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1830066f90 0x7f1830069450 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18500cfd90 0x7f1850099480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18500cd110 0x7f18500c8920 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 --2- 172.21.15.45:0/3960870815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18500b0f70 0x7f18500b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 >> 172.21.15.45:0/3960870815 conn(0x7f18500b1d50 msgr2=0x7f185006e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:38.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 shutdown_connections 2024-04-25T10:31:38.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.194+0000 7f1855e00640 1 -- 172.21.15.45:0/3960870815 wait complete. 2024-04-25T10:31:38.218 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:31:38.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:38.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:31:38.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:31:38.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:31:38.719 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:31:38.728 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:38.729 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:38.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:38.738 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:38.738 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:38.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:38.779 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:38.779 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:38.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:38.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:38.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 Processor -- start 2024-04-25T10:31:38.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 -- start start 2024-04-25T10:31:38.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73481550b0 0x7f73480cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f73480cfe70 0x7f734815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f734815b740 0x7f734806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7348071050 con 0x7f73481550b0 2024-04-25T10:31:38.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73480712f0 con 0x7f73480cfe70 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f734d600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480715d0 con 0x7f734815b740 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73481550b0 0x7f73480cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f73481550b0 0x7f73480cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58764/0 (socket says 172.21.15.45:58764) 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 -- 172.21.15.45:0/152724183 learned_addr learned my addr 172.21.15.45:0/152724183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7347400640 1 --2- 172.21.15.45:0/152724183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f734815b740 0x7f734806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f733e000640 1 --2- 172.21.15.45:0/152724183 >> v2:172.21.15.117:3300/0 conn(0x7f73480cfe70 0x7f734815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 -- 172.21.15.45:0/152724183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f734815b740 msgr2=0x7f734806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 --2- 172.21.15.45:0/152724183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f734815b740 0x7f734806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 -- 172.21.15.45:0/152724183 >> v2:172.21.15.117:3300/0 conn(0x7f73480cfe70 msgr2=0x7f734815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 --2- 172.21.15.45:0/152724183 >> v2:172.21.15.117:3300/0 conn(0x7f73480cfe70 0x7f734815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.962+0000 7f7346a00640 1 -- 172.21.15.45:0/152724183 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480718b0 con 0x7f73481550b0 2024-04-25T10:31:38.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f7346a00640 1 --2- 172.21.15.45:0/152724183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73480cf930 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f7338009870 tx=0x7f733801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecae8b18bba5837e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:38.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f7346000640 1 -- 172.21.15.45:0/152724183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380b0070 con 0x7f73481550b0 2024-04-25T10:31:38.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f7346000640 1 -- 172.21.15.45:0/152724183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73380ab600 con 0x7f73481550b0 2024-04-25T10:31:38.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f7346000640 1 -- 172.21.15.45:0/152724183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380ab9d0 con 0x7f73481550b0 2024-04-25T10:31:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 -- 172.21.15.45:0/152724183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 msgr2=0x7f73480cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 --2- 172.21.15.45:0/152724183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73480cf930 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f7338009870 tx=0x7f733801c260 comp rx=0 tx=0).stop 2024-04-25T10:31:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 -- 172.21.15.45:0/152724183 shutdown_connections 2024-04-25T10:31:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 --2- 172.21.15.45:0/152724183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f734815b740 0x7f734806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 --2- 172.21.15.45:0/152724183 >> v2:172.21.15.117:3300/0 conn(0x7f73480cfe70 0x7f734815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 --2- 172.21.15.45:0/152724183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73480cf930 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 -- 172.21.15.45:0/152724183 >> 172.21.15.45:0/152724183 conn(0x7f7348066b70 msgr2=0x7f7348066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 -- 172.21.15.45:0/152724183 shutdown_connections 2024-04-25T10:31:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.966+0000 7f734d600640 1 -- 172.21.15.45:0/152724183 wait complete. 2024-04-25T10:31:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 Processor -- start 2024-04-25T10:31:38.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 -- start start 2024-04-25T10:31:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73480cfe70 0x7f73480ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73481f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734815b740 0x7f73480cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480b3260 con 0x7f73481550b0 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734815bd60 con 0x7f734815b740 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480b33e0 con 0x7f73480cfe70 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73481f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73481f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58776/0 (socket says 172.21.15.45:58776) 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 -- 172.21.15.45:0/1984733834 learned_addr learned my addr 172.21.15.45:0/1984733834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f7346a00640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73480cfe70 0x7f73480ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f7347400640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734815b740 0x7f73480cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 -- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73480cfe70 msgr2=0x7f73480ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73480cfe70 0x7f73480ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 -- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734815b740 msgr2=0x7f73480cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734815b740 0x7f73480cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 -- 172.21.15.45:0/1984733834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480cece0 con 0x7f73481550b0 2024-04-25T10:31:38.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733e000640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73481f5580 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f733000d800 tx=0x7f733000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000b6c0 con 0x7f73481550b0 2024-04-25T10:31:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330019070 con 0x7f73481550b0 2024-04-25T10:31:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73481f5ac0 con 0x7f73481550b0 2024-04-25T10:31:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000ba60 con 0x7f73481550b0 2024-04-25T10:31:38.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.970+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73481f8530 con 0x7f73481550b0 2024-04-25T10:31:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.974+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734804ef30 con 0x7f73481550b0 2024-04-25T10:31:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.974+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f733000bc00 con 0x7f73481550b0 2024-04-25T10:31:38.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.974+0000 7f733f400640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7324066f60 0x7f7324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.974+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7330085370 con 0x7f73481550b0 2024-04-25T10:31:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.974+0000 7f7346a00640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7324066f60 0x7f7324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:38.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.974+0000 7f7346a00640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7324066f60 0x7f7324069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f73380ad000 tx=0x7f73380a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:38.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:38.978+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73481f5ac0 con 0x7f73481550b0 2024-04-25T10:31:39.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.174+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f734814e7d0 con 0x7f73481550b0 2024-04-25T10:31:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.174+0000 7f733f400640 1 -- 172.21.15.45:0/1984733834 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f73480cece0 con 0x7f73481550b0 2024-04-25T10:31:39.178 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:39.178 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:22.823850+0000","modified":"2024-04-25T10:31:25.830407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10881":{"gid":10881,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1455178901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1455178901},{"type":"v1","addr":"172.21.15.45:6835","nonce":1455178901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:31:39.178 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 131 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7324066f60 msgr2=0x7f7324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7324066f60 0x7f7324069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f73380ad000 tx=0x7f73380a6000 comp rx=0 tx=0).stop 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 msgr2=0x7f73481f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73481f5580 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f733000d800 tx=0x7f733000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 shutdown_connections 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7324066f60 0x7f7324069420 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734815b740 0x7f73480cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73481550b0 0x7f73481f5580 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 --2- 172.21.15.45:0/1984733834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73480cfe70 0x7f73480ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 >> 172.21.15.45:0/1984733834 conn(0x7f7348066b70 msgr2=0x7f73480b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 shutdown_connections 2024-04-25T10:31:39.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.178+0000 7f734d600640 1 -- 172.21.15.45:0/1984733834 wait complete. 2024-04-25T10:31:39.199 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:39.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 Processor -- start 2024-04-25T10:31:39.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 -- start start 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d415df00 0x7fc7d40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7d40d0750 0x7fc7d40d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7d40d1ce0 0x7fc7d40b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d40d2490 con 0x7fc7d40d1ce0 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d40d2770 con 0x7fc7d40d0750 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40d2a50 con 0x7fc7d415df00 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7d40d1ce0 0x7fc7d40b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7d40d1ce0 0x7fc7d40b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58790/0 (socket says 172.21.15.45:58790) 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 -- 172.21.15.45:0/1699382025 learned_addr learned my addr 172.21.15.45:0/1699382025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:39.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7c9400640 1 --2- 172.21.15.45:0/1699382025 >> v2:172.21.15.117:3300/0 conn(0x7fc7d40d0750 0x7fc7d40d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d1e00640 1 --2- 172.21.15.45:0/1699382025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d415df00 0x7fc7d40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 -- 172.21.15.45:0/1699382025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d415df00 msgr2=0x7fc7d40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 --2- 172.21.15.45:0/1699382025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d415df00 0x7fc7d40d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 -- 172.21.15.45:0/1699382025 >> v2:172.21.15.117:3300/0 conn(0x7fc7d40d0750 msgr2=0x7fc7d40d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 --2- 172.21.15.45:0/1699382025 >> v2:172.21.15.117:3300/0 conn(0x7fc7d40d0750 0x7fc7d40d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.338+0000 7fc7d2800640 1 -- 172.21.15.45:0/1699382025 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40d2d30 con 0x7fc7d40d1ce0 2024-04-25T10:31:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d2800640 1 --2- 172.21.15.45:0/1699382025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d40d1ce0 0x7fc7d40b5610 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40b66e0 tx=0x7fc7cc01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef7bda7f9aea8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d1400640 1 -- 172.21.15.45:0/1699382025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0b0070 con 0x7fc7d40d1ce0 2024-04-25T10:31:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d1400640 1 -- 172.21.15.45:0/1699382025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc0ab600 con 0x7fc7d40d1ce0 2024-04-25T10:31:39.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d1400640 1 -- 172.21.15.45:0/1699382025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0ab9d0 con 0x7fc7d40d1ce0 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 -- 172.21.15.45:0/1699382025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d40d1ce0 msgr2=0x7fc7d40b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 --2- 172.21.15.45:0/1699382025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d40d1ce0 0x7fc7d40b5610 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40b66e0 tx=0x7fc7cc01cc90 comp rx=0 tx=0).stop 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 -- 172.21.15.45:0/1699382025 shutdown_connections 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 --2- 172.21.15.45:0/1699382025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d40d1ce0 0x7fc7d40b5610 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 --2- 172.21.15.45:0/1699382025 >> v2:172.21.15.117:3300/0 conn(0x7fc7d40d0750 0x7fc7d40d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 --2- 172.21.15.45:0/1699382025 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d415df00 0x7fc7d40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 -- 172.21.15.45:0/1699382025 >> 172.21.15.45:0/1699382025 conn(0x7fc7d406b290 msgr2=0x7fc7d40669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 -- 172.21.15.45:0/1699382025 shutdown_connections 2024-04-25T10:31:39.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.342+0000 7fc7d9000640 1 -- 172.21.15.45:0/1699382025 wait complete. 2024-04-25T10:31:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 Processor -- start 2024-04-25T10:31:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- start start 2024-04-25T10:31:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40d0750 0x7fc7d414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d40d1ce0 0x7fc7d414c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 0x7fc7d41fa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d415e460 con 0x7fc7d415df00 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d415e2e0 con 0x7fc7d40d0750 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40b71e0 con 0x7fc7d40d1ce0 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 0x7fc7d41fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7c9400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d40d1ce0 0x7fc7d414c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 0x7fc7d41fa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58794/0 (socket says 172.21.15.45:58794) 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 -- 172.21.15.45:0/3813156431 learned_addr learned my addr 172.21.15.45:0/3813156431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d1e00640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40d0750 0x7fc7d414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 -- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d40d1ce0 msgr2=0x7fc7d414c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d40d1ce0 0x7fc7d414c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 -- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40d0750 msgr2=0x7fc7d414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40d0750 0x7fc7d414fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 -- 172.21.15.45:0/3813156431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d414c560 con 0x7fc7d415df00 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7c9400640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d40d1ce0 0x7fc7d414c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d2800640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 0x7fc7d41fa430 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0ae040 tx=0x7fc7cc018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0b0040 con 0x7fc7d415df00 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc0b9070 con 0x7fc7d415df00 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d413b760 con 0x7fc7d415df00 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0b4670 con 0x7fc7d415df00 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d413bd20 con 0x7fc7d415df00 2024-04-25T10:31:39.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.346+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d404ef30 con 0x7fc7d415df00 2024-04-25T10:31:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.350+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7cc0b4810 con 0x7fc7d415df00 2024-04-25T10:31:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.350+0000 7fc7caa00640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc7b0066f60 0x7fc7b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.350+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7cc11e5f0 con 0x7fc7d415df00 2024-04-25T10:31:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.350+0000 7fc7d1e00640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc7b0066f60 0x7fc7b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.350+0000 7fc7d1e00640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc7b0066f60 0x7fc7b0069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40039f0 tx=0x7fc7c4009830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.354+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7cc0c2020 con 0x7fc7d415df00 2024-04-25T10:31:39.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.522+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7d41557d0 con 0x7fc7d415df00 2024-04-25T10:31:39.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.522+0000 7fc7caa00640 1 -- 172.21.15.45:0/3813156431 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16603 (secure 0 0 0) 0x7fc7d414c560 con 0x7fc7d415df00 2024-04-25T10:31:39.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:39.526 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":182,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:24.430055+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc7b0066f60 msgr2=0x7fc7b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc7b0066f60 0x7fc7b0069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40039f0 tx=0x7fc7c4009830 comp rx=0 tx=0).stop 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 msgr2=0x7fc7d41fa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 0x7fc7d41fa430 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0ae040 tx=0x7fc7cc018fd0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 shutdown_connections 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc7b0066f60 0x7fc7b0069420 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d415df00 0x7fc7d41fa430 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7d40d1ce0 0x7fc7d414c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 --2- 172.21.15.45:0/3813156431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40d0750 0x7fc7d414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 >> 172.21.15.45:0/3813156431 conn(0x7fc7d406b290 msgr2=0x7fc7d40b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 shutdown_connections 2024-04-25T10:31:39.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.526+0000 7fc7d9000640 1 -- 172.21.15.45:0/3813156431 wait complete. 2024-04-25T10:31:39.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:31:21.231280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:31:21.811204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:39.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:39.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d12200640 1 Processor -- start 2024-04-25T10:31:39.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d12200640 1 -- start start 2024-04-25T10:31:39.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d12200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c0721f0 0x7f7d0c166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d12200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c071360 con 0x7f7d0c159540 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d12200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c159920 con 0x7f7d0c0aeda0 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d12200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c15c2f0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7d0c159540 0x7f7d0c0ae690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34006/0 (socket says 172.21.15.45:34006) 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0b400640 1 -- 172.21.15.45:0/3954074044 learned_addr learned my addr 172.21.15.45:0/3954074044 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0be00640 1 --2- 172.21.15.45:0/3954074044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c0721f0 0x7f7d0c166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0aa00640 1 -- 172.21.15.45:0/3954074044 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308829934 0 0) 0x7f7d0c071360 con 0x7f7d0c159540 2024-04-25T10:31:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0aa00640 1 -- 172.21.15.45:0/3954074044 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf0003530 con 0x7f7d0c159540 2024-04-25T10:31:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0aa00640 1 -- 172.21.15.45:0/3954074044 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704060143 0 0) 0x7f7d0c159920 con 0x7f7d0c0aeda0 2024-04-25T10:31:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0be00640 1 -- 172.21.15.45:0/3954074044 >> v1:172.21.15.117:6789/0 conn(0x7f7d0c0aeda0 legacy=0x7f7d0c071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0be00640 1 -- 172.21.15.45:0/3954074044 >> v1:172.21.15.45:6789/0 conn(0x7f7d0c159540 legacy=0x7f7d0c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.690+0000 7f7d0be00640 1 -- 172.21.15.45:0/3954074044 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c15c5d0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d0be00640 1 --2- 172.21.15.45:0/3954074044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c0721f0 0x7f7d0c166920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009dc0 tx=0x7f7d0401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcad7f87d2ad26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d0aa00640 1 -- 172.21.15.45:0/3954074044 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d040b2070 con 0x7f7d0c0721f0 2024-04-25T10:31:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d0aa00640 1 -- 172.21.15.45:0/3954074044 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d040a5a20 con 0x7f7d0c0721f0 2024-04-25T10:31:39.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d0aa00640 1 -- 172.21.15.45:0/3954074044 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d040a5dd0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 -- 172.21.15.45:0/3954074044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c0721f0 msgr2=0x7f7d0c166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 --2- 172.21.15.45:0/3954074044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c0721f0 0x7f7d0c166920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009dc0 tx=0x7f7d0401c960 comp rx=0 tx=0).stop 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 -- 172.21.15.45:0/3954074044 shutdown_connections 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 --2- 172.21.15.45:0/3954074044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c0721f0 0x7f7d0c166920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 -- 172.21.15.45:0/3954074044 >> 172.21.15.45:0/3954074044 conn(0x7f7d0c0669a0 msgr2=0x7f7d0c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 -- 172.21.15.45:0/3954074044 shutdown_connections 2024-04-25T10:31:39.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 -- 172.21.15.45:0/3954074044 wait complete. 2024-04-25T10:31:39.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.694+0000 7f7d12200640 1 Processor -- start 2024-04-25T10:31:39.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- start start 2024-04-25T10:31:39.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 0x7f7d0c1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0c0aeda0 0x7f7d0c14d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c159540 0x7f7d0c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c169680 con 0x7f7d0c0721f0 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c06c0d0 con 0x7f7d0c0aeda0 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c169800 con 0x7f7d0c159540 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 0x7f7d0c1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 0x7f7d0c1588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58808/0 (socket says 172.21.15.45:58808) 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 -- 172.21.15.45:0/3624908261 learned_addr learned my addr 172.21.15.45:0/3624908261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d03e00640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0c0aeda0 0x7f7d0c14d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0be00640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c159540 0x7f7d0c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 -- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c159540 msgr2=0x7f7d0c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c159540 0x7f7d0c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 -- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0c0aeda0 msgr2=0x7f7d0c14d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0c0aeda0 0x7f7d0c14d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 -- 172.21.15.45:0/3624908261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c153690 con 0x7f7d0c0721f0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d0b400640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 0x7f7d0c1588d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00c400 tx=0x7f7cfc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc017070 con 0x7f7d0c0721f0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc005d10 con 0x7f7d0c0721f0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc007310 con 0x7f7d0c0721f0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c14eae0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c14f0a0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.698+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c04ef30 con 0x7f7d0c0721f0 2024-04-25T10:31:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.702+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cfc01a070 con 0x7f7d0c0721f0 2024-04-25T10:31:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.702+0000 7f7d03400640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce8066f60 0x7f7ce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.702+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cfc0843f0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.702+0000 7f7d03e00640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce8066f60 0x7f7ce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:39.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.702+0000 7f7d03e00640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce8066f60 0x7f7ce8069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c15e8f0 tx=0x7f7cf4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:39.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.706+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cfc054d10 con 0x7f7d0c0721f0 2024-04-25T10:31:39.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.866+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d0c06d7f0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.866+0000 7f7d03400640 1 -- 172.21.15.45:0/3624908261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cfc058db0 con 0x7f7d0c0721f0 2024-04-25T10:31:39.870 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce8066f60 msgr2=0x7f7ce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce8066f60 0x7f7ce8069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c15e8f0 tx=0x7f7cf4009040 comp rx=0 tx=0).stop 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 msgr2=0x7f7d0c1588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 0x7f7d0c1588d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00c400 tx=0x7f7cfc00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 shutdown_connections 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7ce8066f60 0x7f7ce8069420 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d0c159540 0x7f7d0c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d0c0aeda0 0x7f7d0c14d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 --2- 172.21.15.45:0/3624908261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d0c0721f0 0x7f7d0c1588d0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:39.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 >> 172.21.15.45:0/3624908261 conn(0x7f7d0c0669a0 msgr2=0x7f7d0c066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:39.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 shutdown_connections 2024-04-25T10:31:39.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:39.870+0000 7f7d12200640 1 -- 172.21.15.45:0/3624908261 wait complete. 2024-04-25T10:31:39.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:31:40.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.034+0000 7f308dc00640 1 Processor -- start 2024-04-25T10:31:40.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.034+0000 7f308dc00640 1 -- start start 2024-04-25T10:31:40.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f308dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066c00 0x7f3088161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f308dc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30880b5560 0x7f30880ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f308dc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30880b40c0 con 0x7f3088066fe0 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f308dc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3088161d60 con 0x7f30880b5560 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f308dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30880b5b00 con 0x7f3088066c00 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30880b5560 0x7f30880ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f30880b5560 0x7f30880ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58822/0 (socket says 172.21.15.45:58822) 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 -- 172.21.15.45:0/1419967726 learned_addr learned my addr 172.21.15.45:0/1419967726 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3086a00640 1 --2- 172.21.15.45:0/1419967726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066c00 0x7f3088161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 -- 172.21.15.45:0/1419967726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066c00 msgr2=0x7f3088161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 --2- 172.21.15.45:0/1419967726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066c00 0x7f3088161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 -- 172.21.15.45:0/1419967726 >> v1:172.21.15.45:6789/0 conn(0x7f3088066fe0 legacy=0x7f3088072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 -- 172.21.15.45:0/1419967726 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30880b3a00 con 0x7f30880b5560 2024-04-25T10:31:40.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3087400640 1 --2- 172.21.15.45:0/1419967726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30880ae850 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f3080009e50 tx=0x7f308001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1b70027e42786a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3086000640 1 -- 172.21.15.45:0/1419967726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800a56e0 con 0x7f30880b5560 2024-04-25T10:31:40.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3086000640 1 -- 172.21.15.45:0/1419967726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30800a5880 con 0x7f30880b5560 2024-04-25T10:31:40.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.038+0000 7f3086000640 1 -- 172.21.15.45:0/1419967726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800a5c50 con 0x7f30880b5560 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- 172.21.15.45:0/1419967726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 msgr2=0x7f30880ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 --2- 172.21.15.45:0/1419967726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30880ae850 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f3080009e50 tx=0x7f308001c910 comp rx=0 tx=0).stop 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- 172.21.15.45:0/1419967726 shutdown_connections 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 --2- 172.21.15.45:0/1419967726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30880ae850 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 --2- 172.21.15.45:0/1419967726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066c00 0x7f3088161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- 172.21.15.45:0/1419967726 >> 172.21.15.45:0/1419967726 conn(0x7f30880b2560 msgr2=0x7f30880b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- 172.21.15.45:0/1419967726 shutdown_connections 2024-04-25T10:31:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- 172.21.15.45:0/1419967726 wait complete. 2024-04-25T10:31:40.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 Processor -- start 2024-04-25T10:31:40.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- start start 2024-04-25T10:31:40.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3088066c00 0x7f308814b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066fe0 0x7f30881544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30881503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308815e540 con 0x7f30880b5560 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30881585a0 con 0x7f3088066c00 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f308dc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308815e6c0 con 0x7f3088066fe0 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f3087400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30881503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f3087400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30881503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58828/0 (socket says 172.21.15.45:58828) 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f3086a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3088066c00 0x7f308814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f3087400640 1 -- 172.21.15.45:0/1859912807 learned_addr learned my addr 172.21.15.45:0/1859912807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f307e000640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066fe0 0x7f30881544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f3087400640 1 -- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066fe0 msgr2=0x7f30881544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.042+0000 7f3087400640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066fe0 0x7f30881544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f3087400640 1 -- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3088066c00 msgr2=0x7f308814b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f3087400640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3088066c00 0x7f308814b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f3087400640 1 -- 172.21.15.45:0/1859912807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088153400 con 0x7f30880b5560 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f3086a00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3088066c00 0x7f308814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f3087400640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30881503d0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f308001a540 tx=0x7f30800196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800bc070 con 0x7f30880b5560 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30800b8660 con 0x7f30880b5560 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800b8a50 con 0x7f30880b5560 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3088153740 con 0x7f30880b5560 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3088153f80 con 0x7f30880b5560 2024-04-25T10:31:40.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308804ef30 con 0x7f30880b5560 2024-04-25T10:31:40.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30800b8bf0 con 0x7f30880b5560 2024-04-25T10:31:40.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f307f400640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3064066f60 0x7f3064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f308011fc60 con 0x7f30880b5560 2024-04-25T10:31:40.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.046+0000 7f3086a00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3064066f60 0x7f3064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.050+0000 7f3086a00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3064066f60 0x7f3064069420 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f30880d0e90 tx=0x7f3070008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.050+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3088153740 con 0x7f30880b5560 2024-04-25T10:31:40.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3088162490 con 0x7f30880b5560 2024-04-25T10:31:40.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f307f400640 1 -- 172.21.15.45:0/1859912807 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v182) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3080016620 con 0x7f30880b5560 2024-04-25T10:31:40.217 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3064066f60 msgr2=0x7f3064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3064066f60 0x7f3064069420 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f30880d0e90 tx=0x7f3070008040 comp rx=0 tx=0).stop 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 msgr2=0x7f30881503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30881503d0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f308001a540 tx=0x7f30800196b0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 shutdown_connections 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3064066f60 0x7f3064069420 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30880b5560 0x7f30881503d0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3088066fe0 0x7f30881544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 --2- 172.21.15.45:0/1859912807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3088066c00 0x7f308814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 >> 172.21.15.45:0/1859912807 conn(0x7f30880b2560 msgr2=0x7f30880b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:40.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 shutdown_connections 2024-04-25T10:31:40.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.214+0000 7f308dc00640 1 -- 172.21.15.45:0/1859912807 wait complete. 2024-04-25T10:31:40.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:31:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb279000640 1 Processor -- start 2024-04-25T10:31:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb279000640 1 -- start start 2024-04-25T10:31:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb279000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2740d1fb0 0x7fb274168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb279000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27406b470 con 0x7fb27415dd30 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb279000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27406b750 con 0x7fb2740cfb30 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb279000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27415e110 con 0x7fb2740d1fb0 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb271e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb27415dd30 0x7fb27406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34028/0 (socket says 172.21.15.45:34028) 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb271e00640 1 -- 172.21.15.45:0/3176738414 learned_addr learned my addr 172.21.15.45:0/3176738414 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb272800640 1 --2- 172.21.15.45:0/3176738414 >> v2:172.21.15.45:3300/0 conn(0x7fb2740d1fb0 0x7fb274168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb271400640 1 -- 172.21.15.45:0/3176738414 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404832661 0 0) 0x7fb27406b470 con 0x7fb27415dd30 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb271400640 1 -- 172.21.15.45:0/3176738414 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb258003530 con 0x7fb27415dd30 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb272800640 1 -- 172.21.15.45:0/3176738414 >> v1:172.21.15.117:6789/0 conn(0x7fb2740cfb30 legacy=0x7fb2740d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb272800640 1 -- 172.21.15.45:0/3176738414 >> v1:172.21.15.45:6789/0 conn(0x7fb27415dd30 legacy=0x7fb27406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.378+0000 7fb272800640 1 -- 172.21.15.45:0/3176738414 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27406ba30 con 0x7fb2740d1fb0 2024-04-25T10:31:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb272800640 1 --2- 172.21.15.45:0/3176738414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2740d1fb0 0x7fb274168b50 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009d90 tx=0x7fb26c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e469dfb33416ff2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb271400640 1 -- 172.21.15.45:0/3176738414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0b2070 con 0x7fb2740d1fb0 2024-04-25T10:31:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb271400640 1 -- 172.21.15.45:0/3176738414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0a5830 con 0x7fb2740d1fb0 2024-04-25T10:31:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb271400640 1 -- 172.21.15.45:0/3176738414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0a5c00 con 0x7fb2740d1fb0 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 -- 172.21.15.45:0/3176738414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2740d1fb0 msgr2=0x7fb274168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 --2- 172.21.15.45:0/3176738414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2740d1fb0 0x7fb274168b50 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009d90 tx=0x7fb26c01c970 comp rx=0 tx=0).stop 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 -- 172.21.15.45:0/3176738414 shutdown_connections 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 --2- 172.21.15.45:0/3176738414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2740d1fb0 0x7fb274168b50 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 -- 172.21.15.45:0/3176738414 >> 172.21.15.45:0/3176738414 conn(0x7fb27409a9d0 msgr2=0x7fb27409adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 -- 172.21.15.45:0/3176738414 shutdown_connections 2024-04-25T10:31:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.382+0000 7fb279000640 1 -- 172.21.15.45:0/3176738414 wait complete. 2024-04-25T10:31:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 Processor -- start 2024-04-25T10:31:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 -- start start 2024-04-25T10:31:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2740cfb30 0x7fb274151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2740d1fb0 0x7fb2740ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 0x7fb2740d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416bc40 con 0x7fb27415dd30 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416bac0 con 0x7fb2740d1fb0 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27416bdc0 con 0x7fb2740cfb30 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 0x7fb2740d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 0x7fb2740d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58848/0 (socket says 172.21.15.45:58848) 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 -- 172.21.15.45:0/1134194153 learned_addr learned my addr 172.21.15.45:0/1134194153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb271e00640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2740cfb30 0x7fb274151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb269400640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2740d1fb0 0x7fb2740ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 -- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2740cfb30 msgr2=0x7fb274151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2740cfb30 0x7fb274151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 -- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2740d1fb0 msgr2=0x7fb2740ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2740d1fb0 0x7fb2740ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 -- 172.21.15.45:0/1134194153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274154e20 con 0x7fb27415dd30 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb269400640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2740d1fb0 0x7fb2740ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb272800640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 0x7fb2740d04f0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01a5a0 tx=0x7fb26c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0b2050 con 0x7fb27415dd30 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0be070 con 0x7fb27415dd30 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c009450 con 0x7fb27415dd30 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb274155100 con 0x7fb27415dd30 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.386+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2741556c0 con 0x7fb27415dd30 2024-04-25T10:31:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.390+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27404ef30 con 0x7fb27415dd30 2024-04-25T10:31:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.390+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb26c0adc20 con 0x7fb27415dd30 2024-04-25T10:31:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.390+0000 7fb26aa00640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb250066f90 0x7fb250069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.390+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb26c11ff20 con 0x7fb27415dd30 2024-04-25T10:31:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.390+0000 7fb271e00640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb250066f90 0x7fb250069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.390+0000 7fb271e00640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb250066f90 0x7fb250069450 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb2640011f0 tx=0x7fb264009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.394+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb26c016560 con 0x7fb27415dd30 2024-04-25T10:31:40.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.554+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2740b0bf0 con 0x7fb27415dd30 2024-04-25T10:31:40.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb26aa00640 1 -- 172.21.15.45:0/1134194153 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v182) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb26c0ab340 con 0x7fb27415dd30 2024-04-25T10:31:40.560 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:31:40.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb250066f90 msgr2=0x7fb250069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb250066f90 0x7fb250069450 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb2640011f0 tx=0x7fb264009040 comp rx=0 tx=0).stop 2024-04-25T10:31:40.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 msgr2=0x7fb2740d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 0x7fb2740d04f0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01a5a0 tx=0x7fb26c019710 comp rx=0 tx=0).stop 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 shutdown_connections 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb250066f90 0x7fb250069450 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb27415dd30 0x7fb2740d04f0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2740d1fb0 0x7fb2740ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 --2- 172.21.15.45:0/1134194153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2740cfb30 0x7fb274151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 >> 172.21.15.45:0/1134194153 conn(0x7fb27409a9d0 msgr2=0x7fb27406f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 shutdown_connections 2024-04-25T10:31:40.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.558+0000 7fb279000640 1 -- 172.21.15.45:0/1134194153 wait complete. 2024-04-25T10:31:40.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:40.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 Processor -- start 2024-04-25T10:31:40.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 -- start start 2024-04-25T10:31:40.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f443815b9e0 0x7f44380cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d1c30 0x7f44380cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44380ccfb0 con 0x7f443815bdc0 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44380cd290 con 0x7f443815b9e0 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f443e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44380cd570 con 0x7f44380d1c30 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f443815b9e0 0x7f44380cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f443815b9e0 0x7f44380cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58854/0 (socket says 172.21.15.45:58854) 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 -- 172.21.15.45:0/1474846525 learned_addr learned my addr 172.21.15.45:0/1474846525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437e00640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d1c30 0x7f44380cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 -- 172.21.15.45:0/1474846525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d1c30 msgr2=0x7f44380cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d1c30 0x7f44380cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 -- 172.21.15.45:0/1474846525 >> v1:172.21.15.117:6789/0 conn(0x7f443815bdc0 legacy=0x7f44380d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437400640 1 -- 172.21.15.45:0/1474846525 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380cd850 con 0x7f443815b9e0 2024-04-25T10:31:40.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.718+0000 7f4437e00640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d1c30 0x7f44380cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:40.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f4437400640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815b9e0 0x7f44380cc410 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f4428009870 tx=0x7f442801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569160856a6b967c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f4436a00640 1 -- 172.21.15.45:0/1474846525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44280b2070 con 0x7f443815b9e0 2024-04-25T10:31:40.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f4436a00640 1 -- 172.21.15.45:0/1474846525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44280ad680 con 0x7f443815b9e0 2024-04-25T10:31:40.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f4436a00640 1 -- 172.21.15.45:0/1474846525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44280ada50 con 0x7f443815b9e0 2024-04-25T10:31:40.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 -- 172.21.15.45:0/1474846525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815b9e0 msgr2=0x7f44380cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815b9e0 0x7f44380cc410 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f4428009870 tx=0x7f442801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 -- 172.21.15.45:0/1474846525 shutdown_connections 2024-04-25T10:31:40.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44380d1c30 0x7f44380cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 --2- 172.21.15.45:0/1474846525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815b9e0 0x7f44380cc410 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 -- 172.21.15.45:0/1474846525 >> 172.21.15.45:0/1474846525 conn(0x7f443809a9d0 msgr2=0x7f443809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:40.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 -- 172.21.15.45:0/1474846525 shutdown_connections 2024-04-25T10:31:40.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.722+0000 7f443e800640 1 -- 172.21.15.45:0/1474846525 wait complete. 2024-04-25T10:31:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 Processor -- start 2024-04-25T10:31:40.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 -- start start 2024-04-25T10:31:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44380d1c30 0x7f443814bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443815b9e0 0x7f443814c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 0x7f443815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c65f0 con 0x7f443815bdc0 2024-04-25T10:31:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c6470 con 0x7f44380d1c30 2024-04-25T10:31:40.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44380c6770 con 0x7f443815b9e0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 0x7f443815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 0x7f443815dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58864/0 (socket says 172.21.15.45:58864) 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 -- 172.21.15.45:0/2956591705 learned_addr learned my addr 172.21.15.45:0/2956591705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437400640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44380d1c30 0x7f443814bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f442fe00640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443815b9e0 0x7f443814c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 -- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443815b9e0 msgr2=0x7f443814c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443815b9e0 0x7f443814c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 -- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44380d1c30 msgr2=0x7f443814bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44380d1c30 0x7f443814bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 -- 172.21.15.45:0/2956591705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44381f6e40 con 0x7f443815bdc0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f4437e00640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 0x7f443815dbe0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f4430009fa0 tx=0x7f443000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443001a070 con 0x7f443815bdc0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4430003bc0 con 0x7f443815bdc0 2024-04-25T10:31:40.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430004030 con 0x7f443815bdc0 2024-04-25T10:31:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443815e8d0 con 0x7f443815bdc0 2024-04-25T10:31:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.726+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443806ae30 con 0x7f443815bdc0 2024-04-25T10:31:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.730+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443815edf0 con 0x7f443815bdc0 2024-04-25T10:31:40.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.730+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f443001d070 con 0x7f443815bdc0 2024-04-25T10:31:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.730+0000 7f442f400640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.730+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44300853a0 con 0x7f443815bdc0 2024-04-25T10:31:40.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.730+0000 7f4437400640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:40.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.730+0000 7f4437400640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f44280af000 tx=0x7f44280a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:40.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.734+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f443815edf0 con 0x7f443815bdc0 2024-04-25T10:31:40.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.934+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f443804ef30 con 0x7f443815bdc0 2024-04-25T10:31:40.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.934+0000 7f442f400640 1 -- 172.21.15.45:0/2956591705 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f4430010070 con 0x7f443815bdc0 2024-04-25T10:31:40.938 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:40.938 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:22.823850+0000","modified":"2024-04-25T10:31:25.830407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10881":{"gid":10881,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1455178901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1455178901},{"type":"v1","addr":"172.21.15.45:6835","nonce":1455178901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:31:40.938 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 131 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 msgr2=0x7f4414069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f44280af000 tx=0x7f44280a7000 comp rx=0 tx=0).stop 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 msgr2=0x7f443815dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 0x7f443815dbe0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f4430009fa0 tx=0x7f443000d560 comp rx=0 tx=0).stop 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 shutdown_connections 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4414066f60 0x7f4414069420 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f443815bdc0 0x7f443815dbe0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f443815b9e0 0x7f443814c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 --2- 172.21.15.45:0/2956591705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44380d1c30 0x7f443814bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:40.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 >> 172.21.15.45:0/2956591705 conn(0x7f443809a9d0 msgr2=0x7f44380d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:40.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 shutdown_connections 2024-04-25T10:31:40.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:40.938+0000 7f443e800640 1 -- 172.21.15.45:0/2956591705 wait complete. 2024-04-25T10:31:40.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:41.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 Processor -- start 2024-04-25T10:31:41.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 -- start start 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae40d0040 0x7f9ae415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ae415b740 0x7f9ae40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae4071050 con 0x7f9ae41550b0 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae415d040 con 0x7f9ae415b740 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.098+0000 7f9aeaa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae40712f0 con 0x7f9ae40d0040 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ae415b740 0x7f9ae40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ae415b740 0x7f9ae40b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58870/0 (socket says 172.21.15.45:58870) 2024-04-25T10:31:41.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 -- 172.21.15.45:0/3411777964 learned_addr learned my addr 172.21.15.45:0/3411777964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9adb400640 1 --2- 172.21.15.45:0/3411777964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae40d0040 0x7f9ae415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 -- 172.21.15.45:0/3411777964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae40d0040 msgr2=0x7f9ae415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 --2- 172.21.15.45:0/3411777964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae40d0040 0x7f9ae415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 -- 172.21.15.45:0/3411777964 >> v1:172.21.15.117:6789/0 conn(0x7f9ae41550b0 legacy=0x7f9ae40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:41.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 -- 172.21.15.45:0/3411777964 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40715d0 con 0x7f9ae415b740 2024-04-25T10:31:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae8a00640 1 --2- 172.21.15.45:0/3411777964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae40b3260 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009e50 tx=0x7f9adc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d1150f843955af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae3400640 1 -- 172.21.15.45:0/3411777964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a5840 con 0x7f9ae415b740 2024-04-25T10:31:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae3400640 1 -- 172.21.15.45:0/3411777964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc0a59e0 con 0x7f9ae415b740 2024-04-25T10:31:41.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.102+0000 7f9ae3400640 1 -- 172.21.15.45:0/3411777964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a5db0 con 0x7f9ae415b740 2024-04-25T10:31:41.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- 172.21.15.45:0/3411777964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 msgr2=0x7f9ae40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/3411777964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae40b3260 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009e50 tx=0x7f9adc01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- 172.21.15.45:0/3411777964 shutdown_connections 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/3411777964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae40b3260 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/3411777964 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae40d0040 0x7f9ae415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- 172.21.15.45:0/3411777964 >> 172.21.15.45:0/3411777964 conn(0x7f9ae4066b70 msgr2=0x7f9ae4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- 172.21.15.45:0/3411777964 shutdown_connections 2024-04-25T10:31:41.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- 172.21.15.45:0/3411777964 wait complete. 2024-04-25T10:31:41.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 Processor -- start 2024-04-25T10:31:41.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- start start 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae40d0040 0x7f9ae406d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae41550b0 0x7f9ae4139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae41684c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae415bd70 con 0x7f9ae415b740 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae415bbf0 con 0x7f9ae40d0040 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9aeaa00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae415c030 con 0x7f9ae41550b0 2024-04-25T10:31:41.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae41684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae41684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58878/0 (socket says 172.21.15.45:58878) 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 -- 172.21.15.45:0/408196392 learned_addr learned my addr 172.21.15.45:0/408196392 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9adb400640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae41550b0 0x7f9ae4139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae3e00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae40d0040 0x7f9ae406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 -- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae41550b0 msgr2=0x7f9ae4139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae41550b0 0x7f9ae4139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 -- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae40d0040 msgr2=0x7f9ae406d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae40d0040 0x7f9ae406d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 -- 172.21.15.45:0/408196392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae415ae00 con 0x7f9ae415b740 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.106+0000 7f9ae8a00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae41684c0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0af040 tx=0x7f9adc019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0b8670 con 0x7f9ae415b740 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9adc0be070 con 0x7f9ae415b740 2024-04-25T10:31:41.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae4151ba0 con 0x7f9ae415b740 2024-04-25T10:31:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0b8a90 con 0x7f9ae415b740 2024-04-25T10:31:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae4152160 con 0x7f9ae415b740 2024-04-25T10:31:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae404ef30 con 0x7f9ae415b740 2024-04-25T10:31:41.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9adc0b8c30 con 0x7f9ae415b740 2024-04-25T10:31:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae0c00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ac0066f60 0x7f9ac0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9adc1200a0 con 0x7f9ae415b740 2024-04-25T10:31:41.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.110+0000 7f9ae3e00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ac0066f60 0x7f9ac0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.114+0000 7f9ae3e00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ac0066f60 0x7f9ac0069420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4007fd0 tx=0x7f9ad4007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.114+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9adc0165e0 con 0x7f9ae415b740 2024-04-25T10:31:41.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.282+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ae4068000 con 0x7f9ae415b740 2024-04-25T10:31:41.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.282+0000 7f9ae0c00640 1 -- 172.21.15.45:0/408196392 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f9adc0f0b80 con 0x7f9ae415b740 2024-04-25T10:31:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:41.287 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":182,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:24.430055+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ac0066f60 msgr2=0x7f9ac0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ac0066f60 0x7f9ac0069420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4007fd0 tx=0x7f9ad4007e40 comp rx=0 tx=0).stop 2024-04-25T10:31:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 msgr2=0x7f9ae41684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae41684c0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f9adc0af040 tx=0x7f9adc019760 comp rx=0 tx=0).stop 2024-04-25T10:31:41.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 shutdown_connections 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9ac0066f60 0x7f9ac0069420 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ae415b740 0x7f9ae41684c0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ae41550b0 0x7f9ae4139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 --2- 172.21.15.45:0/408196392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ae40d0040 0x7f9ae406d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 >> 172.21.15.45:0/408196392 conn(0x7f9ae4066b70 msgr2=0x7f9ae40cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 shutdown_connections 2024-04-25T10:31:41.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.286+0000 7f9aeaa00640 1 -- 172.21.15.45:0/408196392 wait complete. 2024-04-25T10:31:41.308 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:31:41.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 Processor -- start 2024-04-25T10:31:41.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 -- start start 2024-04-25T10:31:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1a415b910 0x7fa1a40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a4071050 con 0x7fa1a40cfe70 2024-04-25T10:31:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a415bda0 con 0x7fa1a415b910 2024-04-25T10:31:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1abe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a415d040 con 0x7fa1a41550b0 2024-04-25T10:31:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1a8a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1a40cfe70 0x7fa1a415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34032/0 (socket says 172.21.15.45:34032) 2024-04-25T10:31:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.446+0000 7fa1a8a00640 1 -- 172.21.15.45:0/3323381314 learned_addr learned my addr 172.21.15.45:0/3323381314 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:41.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9e00640 1 --2- 172.21.15.45:0/3323381314 >> v2:172.21.15.117:3300/0 conn(0x7fa1a415b910 0x7fa1a40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9400640 1 --2- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a3e00640 1 -- 172.21.15.45:0/3323381314 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960153883 0 0) 0x7fa1a4071050 con 0x7fa1a40cfe70 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a3e00640 1 -- 172.21.15.45:0/3323381314 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa18c003530 con 0x7fa1a40cfe70 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9e00640 1 -- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 msgr2=0x7fa1a40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9e00640 1 --2- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9e00640 1 -- 172.21.15.45:0/3323381314 >> v1:172.21.15.45:6789/0 conn(0x7fa1a40cfe70 legacy=0x7fa1a415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9e00640 1 -- 172.21.15.45:0/3323381314 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40712f0 con 0x7fa1a415b910 2024-04-25T10:31:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a9e00640 1 --2- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a415b910 0x7fa1a40b3260 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009d90 tx=0x7fa19c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94e31b03407b812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a3e00640 1 -- 172.21.15.45:0/3323381314 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0b2070 con 0x7fa1a415b910 2024-04-25T10:31:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a3e00640 1 -- 172.21.15.45:0/3323381314 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c0a5920 con 0x7fa1a415b910 2024-04-25T10:31:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.450+0000 7fa1a3e00640 1 -- 172.21.15.45:0/3323381314 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0a5cf0 con 0x7fa1a415b910 2024-04-25T10:31:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a415b910 msgr2=0x7fa1a40b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 --2- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a415b910 0x7fa1a40b3260 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009d90 tx=0x7fa19c01c970 comp rx=0 tx=0).stop 2024-04-25T10:31:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- 172.21.15.45:0/3323381314 shutdown_connections 2024-04-25T10:31:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 --2- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a415b910 0x7fa1a40b3260 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 --2- 172.21.15.45:0/3323381314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- 172.21.15.45:0/3323381314 >> 172.21.15.45:0/3323381314 conn(0x7fa1a4066b70 msgr2=0x7fa1a4066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- 172.21.15.45:0/3323381314 shutdown_connections 2024-04-25T10:31:41.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- 172.21.15.45:0/3323381314 wait complete. 2024-04-25T10:31:41.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 Processor -- start 2024-04-25T10:31:41.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- start start 2024-04-25T10:31:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a40cfe70 0x7fa1a415d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 0x7fa1a414ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a415c030 con 0x7fa1a415b910 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40720b0 con 0x7fa1a40cfe70 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1abe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a415c1b0 con 0x7fa1a41550b0 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 0x7fa1a414ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 0x7fa1a414ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58888/0 (socket says 172.21.15.45:58888) 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 -- 172.21.15.45:0/494488924 learned_addr learned my addr 172.21.15.45:0/494488924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a8a00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9400640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a40cfe70 0x7fa1a415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 -- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 msgr2=0x7fa1a40adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 -- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a40cfe70 msgr2=0x7fa1a415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a40cfe70 0x7fa1a415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9e00640 1 -- 172.21.15.45:0/494488924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40af480 con 0x7fa1a415b910 2024-04-25T10:31:41.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.454+0000 7fa1a9400640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a40cfe70 0x7fa1a415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a9e00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 0x7fa1a414ae40 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0b0040 tx=0x7fa19c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0bd020 con 0x7fa1a415b910 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c0b8470 con 0x7fa1a415b910 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a4159c60 con 0x7fa1a415b910 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0b2040 con 0x7fa1a415b910 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a415a220 con 0x7fa1a415b910 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a404ef30 con 0x7fa1a415b910 2024-04-25T10:31:41.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa19c0c0020 con 0x7fa1a415b910 2024-04-25T10:31:41.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a1600640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa184066f60 0x7fa184069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa19c11ffe0 con 0x7fa1a415b910 2024-04-25T10:31:41.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.458+0000 7fa1a9400640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa184066f60 0x7fa184069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.462+0000 7fa1a9400640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa184066f60 0x7fa184069420 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa198009bb0 tx=0x7fa198008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.462+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa19c0c7030 con 0x7fa1a415b910 2024-04-25T10:31:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.658+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1a40b3580 con 0x7fa1a415b910 2024-04-25T10:31:41.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.658+0000 7fa1a1600640 1 -- 172.21.15.45:0/494488924 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fa19c016560 con 0x7fa1a415b910 2024-04-25T10:31:41.662 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:41.662 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:22.823850+0000","modified":"2024-04-25T10:31:25.830407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10881":{"gid":10881,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1455178901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1455178901},{"type":"v1","addr":"172.21.15.45:6835","nonce":1455178901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:31:41.662 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 131 2024-04-25T10:31:41.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa184066f60 msgr2=0x7fa184069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa184066f60 0x7fa184069420 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa198009bb0 tx=0x7fa198008040 comp rx=0 tx=0).stop 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 msgr2=0x7fa1a414ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 0x7fa1a414ae40 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0b0040 tx=0x7fa19c019710 comp rx=0 tx=0).stop 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 shutdown_connections 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa184066f60 0x7fa184069420 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a415b910 0x7fa1a414ae40 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1a41550b0 0x7fa1a40adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 --2- 172.21.15.45:0/494488924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a40cfe70 0x7fa1a415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 >> 172.21.15.45:0/494488924 conn(0x7fa1a4066b70 msgr2=0x7fa1a414e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 shutdown_connections 2024-04-25T10:31:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.662+0000 7fa1abe00640 1 -- 172.21.15.45:0/494488924 wait complete. 2024-04-25T10:31:41.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:41.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 Processor -- start 2024-04-25T10:31:41.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 -- start start 2024-04-25T10:31:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99a8071f70 0x7f99a80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a80b2fc0 0x7f99a806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a8070ed0 con 0x7f99a815a0e0 2024-04-25T10:31:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a80711b0 con 0x7f99a8071f70 2024-04-25T10:31:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99ade00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8071490 con 0x7f99a80b2fc0 2024-04-25T10:31:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f99a815a0e0 0x7f99a815b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34038/0 (socket says 172.21.15.45:34038) 2024-04-25T10:31:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6a00640 1 -- 172.21.15.45:0/1288349001 learned_addr learned my addr 172.21.15.45:0/1288349001 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:41.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6000640 1 --2- 172.21.15.45:0/1288349001 >> v2:172.21.15.45:3300/0 conn(0x7f99a8071f70 0x7f99a80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a7400640 1 --2- 172.21.15.45:0/1288349001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a80b2fc0 0x7f99a806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a5600640 1 -- 172.21.15.45:0/1288349001 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966020045 0 0) 0x7f99a8070ed0 con 0x7f99a815a0e0 2024-04-25T10:31:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a5600640 1 -- 172.21.15.45:0/1288349001 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998c005740 con 0x7f99a815a0e0 2024-04-25T10:31:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6000640 1 -- 172.21.15.45:0/1288349001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a80b2fc0 msgr2=0x7f99a806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6000640 1 --2- 172.21.15.45:0/1288349001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a80b2fc0 0x7f99a806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6000640 1 -- 172.21.15.45:0/1288349001 >> v1:172.21.15.45:6789/0 conn(0x7f99a815a0e0 legacy=0x7f99a815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.822+0000 7f99a6000640 1 -- 172.21.15.45:0/1288349001 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8071770 con 0x7f99a8071f70 2024-04-25T10:31:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99a6000640 1 --2- 172.21.15.45:0/1288349001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a80b2a80 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f99940097f0 tx=0x7f999401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acee1f08d6b9522d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99a5600640 1 -- 172.21.15.45:0/1288349001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940b0070 con 0x7f99a8071f70 2024-04-25T10:31:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99a5600640 1 -- 172.21.15.45:0/1288349001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99940ab600 con 0x7f99a8071f70 2024-04-25T10:31:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99a5600640 1 -- 172.21.15.45:0/1288349001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940ab9b0 con 0x7f99a8071f70 2024-04-25T10:31:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 -- 172.21.15.45:0/1288349001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 msgr2=0x7f99a80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 --2- 172.21.15.45:0/1288349001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a80b2a80 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f99940097f0 tx=0x7f999401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 -- 172.21.15.45:0/1288349001 shutdown_connections 2024-04-25T10:31:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 --2- 172.21.15.45:0/1288349001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a80b2fc0 0x7f99a806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 --2- 172.21.15.45:0/1288349001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a80b2a80 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 -- 172.21.15.45:0/1288349001 >> 172.21.15.45:0/1288349001 conn(0x7f99a8066b70 msgr2=0x7f99a8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 -- 172.21.15.45:0/1288349001 shutdown_connections 2024-04-25T10:31:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 -- 172.21.15.45:0/1288349001 wait complete. 2024-04-25T10:31:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.826+0000 7f99ade00640 1 Processor -- start 2024-04-25T10:31:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- start start 2024-04-25T10:31:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a80b2fc0 0x7f99a814ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a815a0e0 0x7f99a814b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80723f0 con 0x7f99a8071f70 2024-04-25T10:31:41.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8070b50 con 0x7f99a80b2fc0 2024-04-25T10:31:41.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8072570 con 0x7f99a815a0e0 2024-04-25T10:31:41.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58900/0 (socket says 172.21.15.45:58900) 2024-04-25T10:31:41.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 -- 172.21.15.45:0/1575046591 learned_addr learned my addr 172.21.15.45:0/1575046591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:41.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6000640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a80b2fc0 0x7f99a814ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a7400640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a815a0e0 0x7f99a814b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 -- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a815a0e0 msgr2=0x7f99a814b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a815a0e0 0x7f99a814b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 -- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a80b2fc0 msgr2=0x7f99a814ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a80b2fc0 0x7f99a814ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 -- 172.21.15.45:0/1575046591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a815c740 con 0x7f99a8071f70 2024-04-25T10:31:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99a6a00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a815a910 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f999800c3c0 tx=0x7f999800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998017070 con 0x7f99a8071f70 2024-04-25T10:31:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998005cf0 con 0x7f99a8071f70 2024-04-25T10:31:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980072f0 con 0x7f99a8071f70 2024-04-25T10:31:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a815ca20 con 0x7f99a8071f70 2024-04-25T10:31:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a815cfe0 con 0x7f99a8071f70 2024-04-25T10:31:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.830+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a804ef30 con 0x7f99a8071f70 2024-04-25T10:31:41.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.834+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f999801a070 con 0x7f99a8071f70 2024-04-25T10:31:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.834+0000 7f999f400640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9988066f60 0x7f9988069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.834+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99980844d0 con 0x7f99a8071f70 2024-04-25T10:31:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.834+0000 7f99a6000640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9988066f60 0x7f9988069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:41.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.834+0000 7f99a6000640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9988066f60 0x7f9988069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f99940ad000 tx=0x7f99940a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:41.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:41.838+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9998055440 con 0x7f99a8071f70 2024-04-25T10:31:42.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a8156ea0 con 0x7f99a8071f70 2024-04-25T10:31:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f999f400640 1 -- 172.21.15.45:0/1575046591 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f9998054d40 con 0x7f99a8071f70 2024-04-25T10:31:42.037 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:42.037 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":131,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:22.823850+0000","modified":"2024-04-25T10:31:25.830407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10881":{"gid":10881,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1455178901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1455178901},{"type":"v1","addr":"172.21.15.45:6835","nonce":1455178901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:31:42.037 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 131 2024-04-25T10:31:42.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9988066f60 msgr2=0x7f9988069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9988066f60 0x7f9988069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f99940ad000 tx=0x7f99940a6000 comp rx=0 tx=0).stop 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 msgr2=0x7f99a815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a815a910 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f999800c3c0 tx=0x7f999800c880 comp rx=0 tx=0).stop 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 shutdown_connections 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9988066f60 0x7f9988069420 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99a815a0e0 0x7f99a814b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99a80b2fc0 0x7f99a814ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 --2- 172.21.15.45:0/1575046591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a8071f70 0x7f99a815a910 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 >> 172.21.15.45:0/1575046591 conn(0x7f99a8066b70 msgr2=0x7f99a81579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 shutdown_connections 2024-04-25T10:31:42.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.034+0000 7f99ade00640 1 -- 172.21.15.45:0/1575046591 wait complete. 2024-04-25T10:31:42.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:42.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 Processor -- start 2024-04-25T10:31:42.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 -- start start 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feda406ac90 0x7feda40b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda406b2c0 0x7feda406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda40b5780 con 0x7feda415db40 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda40b5550 con 0x7feda406ac90 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40b5a60 con 0x7feda406b2c0 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7fed9be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feda406ac90 0x7feda40b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda1e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feda415db40 0x7feda415df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34052/0 (socket says 172.21.15.45:34052) 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda1e00640 1 -- 172.21.15.45:0/4074972616 learned_addr learned my addr 172.21.15.45:0/4074972616 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:42.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7feda2800640 1 --2- 172.21.15.45:0/4074972616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda406b2c0 0x7feda406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7fed9be00640 1 -- 172.21.15.45:0/4074972616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda406b2c0 msgr2=0x7feda406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7fed9be00640 1 --2- 172.21.15.45:0/4074972616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda406b2c0 0x7feda406b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7fed9be00640 1 -- 172.21.15.45:0/4074972616 >> v1:172.21.15.45:6789/0 conn(0x7feda415db40 legacy=0x7feda415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.198+0000 7fed9be00640 1 -- 172.21.15.45:0/4074972616 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40b5d40 con 0x7feda406ac90 2024-04-25T10:31:42.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7fed9be00640 1 --2- 172.21.15.45:0/4074972616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda40b5010 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0097c0 tx=0x7fed8c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714827de2b43dc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda1400640 1 -- 172.21.15.45:0/4074972616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0b5070 con 0x7feda406ac90 2024-04-25T10:31:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda1400640 1 -- 172.21.15.45:0/4074972616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c0a8990 con 0x7feda406ac90 2024-04-25T10:31:42.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda1400640 1 -- 172.21.15.45:0/4074972616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0a8d60 con 0x7feda406ac90 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 -- 172.21.15.45:0/4074972616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 msgr2=0x7feda40b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 --2- 172.21.15.45:0/4074972616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda40b5010 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0097c0 tx=0x7fed8c01ec90 comp rx=0 tx=0).stop 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 -- 172.21.15.45:0/4074972616 shutdown_connections 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 --2- 172.21.15.45:0/4074972616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda406b2c0 0x7feda406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 --2- 172.21.15.45:0/4074972616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda40b5010 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 -- 172.21.15.45:0/4074972616 >> 172.21.15.45:0/4074972616 conn(0x7feda40d17b0 msgr2=0x7feda40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 -- 172.21.15.45:0/4074972616 shutdown_connections 2024-04-25T10:31:42.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 -- 172.21.15.45:0/4074972616 wait complete. 2024-04-25T10:31:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 Processor -- start 2024-04-25T10:31:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.202+0000 7feda3200640 1 -- start start 2024-04-25T10:31:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda41f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda406b2c0 0x7feda41f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda415db40 0x7feda41e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40b6820 con 0x7feda406ac90 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40b6300 con 0x7feda406b2c0 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40b69a0 con 0x7feda415db40 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda41f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda41f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58910/0 (socket says 172.21.15.45:58910) 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 -- 172.21.15.45:0/3786772662 learned_addr learned my addr 172.21.15.45:0/3786772662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7fed9be00640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda406b2c0 0x7feda41f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda2800640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda415db40 0x7feda41e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 -- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda415db40 msgr2=0x7feda41e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda415db40 0x7feda41e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 -- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda406b2c0 msgr2=0x7feda41f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda406b2c0 0x7feda41f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 -- 172.21.15.45:0/3786772662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda415df20 con 0x7feda406ac90 2024-04-25T10:31:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda1e00640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda41f48c0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c3a0 tx=0x7fed9400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94017070 con 0x7feda406ac90 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94005cf0 con 0x7feda406ac90 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940072f0 con 0x7feda406ac90 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda409d8a0 con 0x7feda406ac90 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda409db20 con 0x7feda406ac90 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda4075120 con 0x7feda406ac90 2024-04-25T10:31:42.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.206+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed9401a070 con 0x7feda406ac90 2024-04-25T10:31:42.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.210+0000 7fed9a000640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed80066f60 0x7fed80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.210+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed94084290 con 0x7feda406ac90 2024-04-25T10:31:42.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.210+0000 7fed9be00640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed80066f60 0x7fed80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.210+0000 7fed9be00640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed80066f60 0x7fed80069420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0b2000 tx=0x7fed8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.214+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feda409d8a0 con 0x7feda406ac90 2024-04-25T10:31:42.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.374+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feda406b8f0 con 0x7feda406ac90 2024-04-25T10:31:42.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.374+0000 7fed9a000640 1 -- 172.21.15.45:0/3786772662 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+16603 (secure 0 0 0) 0x7fed940810f0 con 0x7feda406ac90 2024-04-25T10:31:42.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:42.379 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":182,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:24.430055+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:21.231280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:21.811204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed80066f60 msgr2=0x7fed80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed80066f60 0x7fed80069420 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0b2000 tx=0x7fed8c008040 comp rx=0 tx=0).stop 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 msgr2=0x7feda41f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda41f48c0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c3a0 tx=0x7fed9400c860 comp rx=0 tx=0).stop 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 shutdown_connections 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed80066f60 0x7fed80069420 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feda415db40 0x7feda41e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda406b2c0 0x7feda41f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 --2- 172.21.15.45:0/3786772662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feda406ac90 0x7feda41f48c0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 >> 172.21.15.45:0/3786772662 conn(0x7feda40d17b0 msgr2=0x7feda40d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 shutdown_connections 2024-04-25T10:31:42.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.378+0000 7feda3200640 1 -- 172.21.15.45:0/3786772662 wait complete. 2024-04-25T10:31:42.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:31:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 Processor -- start 2024-04-25T10:31:42.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 -- start start 2024-04-25T10:31:42.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b940d0040 0x7f3b9415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3b9415b740 0x7f3b940b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b94071050 con 0x7f3b941550b0 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9415d040 con 0x7f3b940d0040 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b98e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b940712f0 con 0x7f3b9415b740 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b8be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b940d0040 0x7f3b9415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b91e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3b941550b0 0x7f3b940cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34064/0 (socket says 172.21.15.45:34064) 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b91e00640 1 -- 172.21.15.45:0/768750389 learned_addr learned my addr 172.21.15.45:0/768750389 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b8be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b940d0040 0x7f3b9415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58918/0 (socket says 172.21.15.45:58918) 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b92800640 1 --2- 172.21.15.45:0/768750389 >> v2:172.21.15.117:3300/0 conn(0x7f3b9415b740 0x7f3b940b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b8be00640 1 -- 172.21.15.45:0/768750389 >> v2:172.21.15.117:3300/0 conn(0x7f3b9415b740 msgr2=0x7f3b940b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b8be00640 1 --2- 172.21.15.45:0/768750389 >> v2:172.21.15.117:3300/0 conn(0x7f3b9415b740 0x7f3b940b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b8be00640 1 -- 172.21.15.45:0/768750389 >> v1:172.21.15.45:6789/0 conn(0x7f3b941550b0 legacy=0x7f3b940cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.542+0000 7f3b8be00640 1 -- 172.21.15.45:0/768750389 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b940715d0 con 0x7f3b940d0040 2024-04-25T10:31:42.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b8be00640 1 --2- 172.21.15.45:0/768750389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b940d0040 0x7f3b9415b200 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3b940d04d0 tx=0x7f3b7c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7631bae59d053852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b91400640 1 -- 172.21.15.45:0/768750389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0ab650 con 0x7f3b940d0040 2024-04-25T10:31:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b91400640 1 -- 172.21.15.45:0/768750389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0b0070 con 0x7f3b940d0040 2024-04-25T10:31:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b91400640 1 -- 172.21.15.45:0/768750389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0aba10 con 0x7f3b940d0040 2024-04-25T10:31:42.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 -- 172.21.15.45:0/768750389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b940d0040 msgr2=0x7f3b9415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 --2- 172.21.15.45:0/768750389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b940d0040 0x7f3b9415b200 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3b940d04d0 tx=0x7f3b7c01c450 comp rx=0 tx=0).stop 2024-04-25T10:31:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 -- 172.21.15.45:0/768750389 shutdown_connections 2024-04-25T10:31:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 --2- 172.21.15.45:0/768750389 >> v2:172.21.15.117:3300/0 conn(0x7f3b9415b740 0x7f3b940b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 --2- 172.21.15.45:0/768750389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b940d0040 0x7f3b9415b200 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 -- 172.21.15.45:0/768750389 >> 172.21.15.45:0/768750389 conn(0x7f3b94066b70 msgr2=0x7f3b94066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 -- 172.21.15.45:0/768750389 shutdown_connections 2024-04-25T10:31:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 -- 172.21.15.45:0/768750389 wait complete. 2024-04-25T10:31:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 Processor -- start 2024-04-25T10:31:42.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.546+0000 7f3b98e00640 1 -- start start 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b940d0040 0x7f3b9414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 0x7f3b9413a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9415b740 0x7f3b941481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9415bcd0 con 0x7f3b941550b0 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9415bb50 con 0x7f3b940d0040 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9415c030 con 0x7f3b9415b740 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 0x7f3b9413a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 0x7f3b9413a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58930/0 (socket says 172.21.15.45:58930) 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 -- 172.21.15.45:0/1218598679 learned_addr learned my addr 172.21.15.45:0/1218598679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b91e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b940d0040 0x7f3b9414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b92800640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9415b740 0x7f3b941481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 -- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9415b740 msgr2=0x7f3b941481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9415b740 0x7f3b941481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 -- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b940d0040 msgr2=0x7f3b9414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b940d0040 0x7f3b9414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 -- 172.21.15.45:0/1218598679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94148720 con 0x7f3b941550b0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8be00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 0x7f3b9413a050 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c019060 tx=0x7f3b7c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0bf610 con 0x7f3b941550b0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0bf7d0 con 0x7f3b941550b0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0bfbc0 con 0x7f3b941550b0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b941489a0 con 0x7f3b941550b0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b94148f60 con 0x7f3b941550b0 2024-04-25T10:31:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.550+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9404ef30 con 0x7f3b941550b0 2024-04-25T10:31:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.554+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b7c0bd020 con 0x7f3b941550b0 2024-04-25T10:31:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.554+0000 7f3b8a000640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b70066f60 0x7f3b70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.554+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b7c11f4a0 con 0x7f3b941550b0 2024-04-25T10:31:42.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.554+0000 7f3b91e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b70066f60 0x7f3b70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.554+0000 7f3b91e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b70066f60 0x7f3b70069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000f70 tx=0x7f3b84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.558+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b7c0c6070 con 0x7f3b941550b0 2024-04-25T10:31:42.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.754+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3b940cea40 con 0x7f3b941550b0 2024-04-25T10:31:42.756 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:31:42.754+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:31:42.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.754+0000 7f3b8a000640 1 -- 172.21.15.45:0/1218598679 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v132) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3b94148720 con 0x7f3b941550b0 2024-04-25T10:31:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:31:42.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b70066f60 msgr2=0x7f3b70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b70066f60 0x7f3b70069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000f70 tx=0x7f3b84009040 comp rx=0 tx=0).stop 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 msgr2=0x7f3b9413a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 0x7f3b9413a050 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c019060 tx=0x7f3b7c01ac90 comp rx=0 tx=0).stop 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 shutdown_connections 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3b70066f60 0x7f3b70069420 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3b9415b740 0x7f3b941481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b941550b0 0x7f3b9413a050 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 --2- 172.21.15.45:0/1218598679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b940d0040 0x7f3b9414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 >> 172.21.15.45:0/1218598679 conn(0x7f3b94066b70 msgr2=0x7f3b940b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 shutdown_connections 2024-04-25T10:31:42.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.758+0000 7f3b98e00640 1 -- 172.21.15.45:0/1218598679 wait complete. 2024-04-25T10:31:42.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:31:42.834 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:31:42.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 Processor -- start 2024-04-25T10:31:42.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 -- start start 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc87c15a0e0 0x7fc87c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0aea50 0x7fc87c0b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0b2fd0 con 0x7fc87c0b42d0 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c0af1d0 con 0x7fc87c15a0e0 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc883e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0b32b0 con 0x7fc87c0aea50 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc87c0b42d0 0x7fc87c06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34066/0 (socket says 172.21.15.45:34066) 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881e00640 1 -- 172.21.15.45:0/1254057971 learned_addr learned my addr 172.21.15.45:0/1254057971 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881400640 1 --2- 172.21.15.45:0/1254057971 >> v2:172.21.15.117:3300/0 conn(0x7fc87c15a0e0 0x7fc87c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc880a00640 1 --2- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0aea50 0x7fc87c0b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc87be00640 1 -- 172.21.15.45:0/1254057971 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358484092 0 0) 0x7fc87c0b2fd0 con 0x7fc87c0b42d0 2024-04-25T10:31:42.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc87be00640 1 -- 172.21.15.45:0/1254057971 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc864003530 con 0x7fc87c0b42d0 2024-04-25T10:31:42.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881400640 1 -- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0aea50 msgr2=0x7fc87c0b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881400640 1 --2- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0aea50 0x7fc87c0b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881400640 1 -- 172.21.15.45:0/1254057971 >> v1:172.21.15.45:6789/0 conn(0x7fc87c0b42d0 legacy=0x7fc87c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.930+0000 7fc881400640 1 -- 172.21.15.45:0/1254057971 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c0b3590 con 0x7fc87c15a0e0 2024-04-25T10:31:42.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc881400640 1 --2- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c15a0e0 0x7fc87c15b150 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc87000a960 tx=0x7fc87001d7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4b0edae04c828f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc87be00640 1 -- 172.21.15.45:0/1254057971 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8700ad070 con 0x7fc87c15a0e0 2024-04-25T10:31:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc87be00640 1 -- 172.21.15.45:0/1254057971 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8700a7640 con 0x7fc87c15a0e0 2024-04-25T10:31:42.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc87be00640 1 -- 172.21.15.45:0/1254057971 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000b040 con 0x7fc87c15a0e0 2024-04-25T10:31:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 -- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c15a0e0 msgr2=0x7fc87c15b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 --2- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c15a0e0 0x7fc87c15b150 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc87000a960 tx=0x7fc87001d7e0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 -- 172.21.15.45:0/1254057971 shutdown_connections 2024-04-25T10:31:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 --2- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0aea50 0x7fc87c0b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 --2- 172.21.15.45:0/1254057971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c15a0e0 0x7fc87c15b150 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 -- 172.21.15.45:0/1254057971 >> 172.21.15.45:0/1254057971 conn(0x7fc87c0669a0 msgr2=0x7fc87c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 -- 172.21.15.45:0/1254057971 shutdown_connections 2024-04-25T10:31:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.934+0000 7fc883e00640 1 -- 172.21.15.45:0/1254057971 wait complete. 2024-04-25T10:31:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 Processor -- start 2024-04-25T10:31:42.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 -- start start 2024-04-25T10:31:42.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c0aea50 0x7fc87c145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0b42d0 0x7fc87c145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 0x7fc87c145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c070770 con 0x7fc87c15a0e0 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0705f0 con 0x7fc87c0aea50 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c0708f0 con 0x7fc87c0b42d0 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 0x7fc87c145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 0x7fc87c145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58962/0 (socket says 172.21.15.45:58962) 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 -- 172.21.15.45:0/1396358560 learned_addr learned my addr 172.21.15.45:0/1396358560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:42.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 -- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0b42d0 msgr2=0x7fc87c145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0b42d0 0x7fc87c145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 -- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c0aea50 msgr2=0x7fc87c145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881400640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c0aea50 0x7fc87c145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c0aea50 0x7fc87c145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 -- 172.21.15.45:0/1396358560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c0b4020 con 0x7fc87c15a0e0 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881400640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c0aea50 0x7fc87c145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:42.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc881e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 0x7fc87c145db0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fc87400cb70 tx=0x7fc87400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874017070 con 0x7fc87c15a0e0 2024-04-25T10:31:42.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87400ace0 con 0x7fc87c15a0e0 2024-04-25T10:31:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c1497a0 con 0x7fc87c15a0e0 2024-04-25T10:31:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874012620 con 0x7fc87c15a0e0 2024-04-25T10:31:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.938+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c149d60 con 0x7fc87c15a0e0 2024-04-25T10:31:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.942+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c04ef30 con 0x7fc87c15a0e0 2024-04-25T10:31:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.942+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc87401a070 con 0x7fc87c15a0e0 2024-04-25T10:31:42.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.942+0000 7fc879600640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc85c066f60 0x7fc85c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.942+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc874084e60 con 0x7fc87c15a0e0 2024-04-25T10:31:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.942+0000 7fc881400640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc85c066f60 0x7fc85c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:42.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.942+0000 7fc881400640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc85c066f60 0x7fc85c069420 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc8700af000 tx=0x7fc87000c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:42.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:42.946+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc874055e10 con 0x7fc87c15a0e0 2024-04-25T10:31:43.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.146+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc87c06e1f0 con 0x7fc87c15a0e0 2024-04-25T10:31:43.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.758+0000 7fc879600640 1 -- 172.21.15.45:0/1396358560 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v133) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc874055710 con 0x7fc87c15a0e0 2024-04-25T10:31:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc85c066f60 msgr2=0x7fc85c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc85c066f60 0x7fc85c069420 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc8700af000 tx=0x7fc87000c040 comp rx=0 tx=0).stop 2024-04-25T10:31:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 msgr2=0x7fc87c145db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 0x7fc87c145db0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fc87400cb70 tx=0x7fc87400a430 comp rx=0 tx=0).stop 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 shutdown_connections 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc85c066f60 0x7fc85c069420 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc87c15a0e0 0x7fc87c145db0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc87c0b42d0 0x7fc87c145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 --2- 172.21.15.45:0/1396358560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc87c0aea50 0x7fc87c145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 >> 172.21.15.45:0/1396358560 conn(0x7fc87c0669a0 msgr2=0x7fc87c0ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 shutdown_connections 2024-04-25T10:31:43.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.762+0000 7fc883e00640 1 -- 172.21.15.45:0/1396358560 wait complete. 2024-04-25T10:31:43.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:31:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.934+0000 7f23af200640 1 Processor -- start 2024-04-25T10:31:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.934+0000 7f23af200640 1 -- start start 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23af200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f23a80ce030 0x7f23a80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23af200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23a806bd30 0x7f23a80ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23af200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a806c2d0 con 0x7f23a80cf6b0 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23af200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a80cbe00 con 0x7f23a806bd30 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23af200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a80cee90 con 0x7f23a80ce030 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23a806bd30 0x7f23a80ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f23a806bd30 0x7f23a80ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58976/0 (socket says 172.21.15.45:58976) 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 -- 172.21.15.45:0/659591498 learned_addr learned my addr 172.21.15.45:0/659591498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ae800640 1 --1- 172.21.15.45:0/659591498 >> v1:172.21.15.45:6789/0 conn(0x7f23a80cf6b0 0x7f23a80cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34076/0 (socket says 172.21.15.45:34076) 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ade00640 1 --2- 172.21.15.45:0/659591498 >> v2:172.21.15.117:3300/0 conn(0x7f23a80ce030 0x7f23a80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 -- 172.21.15.45:0/659591498 >> v2:172.21.15.117:3300/0 conn(0x7f23a80ce030 msgr2=0x7f23a80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 --2- 172.21.15.45:0/659591498 >> v2:172.21.15.117:3300/0 conn(0x7f23a80ce030 0x7f23a80ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 -- 172.21.15.45:0/659591498 >> v1:172.21.15.45:6789/0 conn(0x7f23a80cf6b0 legacy=0x7f23a80cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.938+0000 7f23ad400640 1 -- 172.21.15.45:0/659591498 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a806c5b0 con 0x7f23a806bd30 2024-04-25T10:31:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23ad400640 1 --2- 172.21.15.45:0/659591498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a806bd30 0x7f23a80ce950 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f23a80719d0 tx=0x7f239801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff6d2eeab49b8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23aca00640 1 -- 172.21.15.45:0/659591498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980b0070 con 0x7f23a806bd30 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23aca00640 1 -- 172.21.15.45:0/659591498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980ab600 con 0x7f23a806bd30 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23aca00640 1 -- 172.21.15.45:0/659591498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980ab9d0 con 0x7f23a806bd30 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 -- 172.21.15.45:0/659591498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a806bd30 msgr2=0x7f23a80ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 --2- 172.21.15.45:0/659591498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a806bd30 0x7f23a80ce950 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f23a80719d0 tx=0x7f239801c520 comp rx=0 tx=0).stop 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 -- 172.21.15.45:0/659591498 shutdown_connections 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 --2- 172.21.15.45:0/659591498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a806bd30 0x7f23a80ce950 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 --2- 172.21.15.45:0/659591498 >> v2:172.21.15.117:3300/0 conn(0x7f23a80ce030 0x7f23a80ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 -- 172.21.15.45:0/659591498 >> 172.21.15.45:0/659591498 conn(0x7f23a809a9d0 msgr2=0x7f23a80d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 -- 172.21.15.45:0/659591498 shutdown_connections 2024-04-25T10:31:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 -- 172.21.15.45:0/659591498 wait complete. 2024-04-25T10:31:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 Processor -- start 2024-04-25T10:31:43.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.942+0000 7f23af200640 1 -- start start 2024-04-25T10:31:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23a806bd30 0x7f23a815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23a80ce030 0x7f23a8157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 0x7f23a815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80b2aa0 con 0x7f23a80cf6b0 2024-04-25T10:31:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80b2920 con 0x7f23a806bd30 2024-04-25T10:31:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80cdab0 con 0x7f23a80ce030 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 0x7f23a815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 0x7f23a815fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58990/0 (socket says 172.21.15.45:58990) 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 -- 172.21.15.45:0/1733752448 learned_addr learned my addr 172.21.15.45:0/1733752448 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ade00640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23a806bd30 0x7f23a815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ad400640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23a80ce030 0x7f23a8157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 -- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23a80ce030 msgr2=0x7f23a8157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23a80ce030 0x7f23a8157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 -- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23a806bd30 msgr2=0x7f23a815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23a806bd30 0x7f23a815f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 -- 172.21.15.45:0/1733752448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a8158450 con 0x7f23a80cf6b0 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ad400640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23a80ce030 0x7f23a8157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23ae800640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 0x7f23a815fee0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c7a0 tx=0x7f23a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:43.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4017070 con 0x7f23a80cf6b0 2024-04-25T10:31:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a400a640 con 0x7f23a80cf6b0 2024-04-25T10:31:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400a9f0 con 0x7f23a80cf6b0 2024-04-25T10:31:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a81607e0 con 0x7f23a80cf6b0 2024-04-25T10:31:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a815ed80 con 0x7f23a80cf6b0 2024-04-25T10:31:43.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.946+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a8075120 con 0x7f23a80cf6b0 2024-04-25T10:31:43.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.950+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23a401a070 con 0x7f23a80cf6b0 2024-04-25T10:31:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.950+0000 7f23a2000640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f238c066f60 0x7f238c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.950+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f23a4084a80 con 0x7f23a80cf6b0 2024-04-25T10:31:43.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.950+0000 7f23ade00640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f238c066f60 0x7f238c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:43.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.954+0000 7f23ade00640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f238c066f60 0x7f238c069420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f239c0079a0 tx=0x7f239c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:43.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:43.954+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23a40553c0 con 0x7f23a80cf6b0 2024-04-25T10:31:44.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.118+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f23a815daa0 con 0x7f23a80cf6b0 2024-04-25T10:31:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.954+0000 7f23a2000640 1 -- 172.21.15.45:0/1733752448 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v184) v1 ==== 181+0+0 (secure 0 0 0) 0x7f23a4059460 con 0x7f23a80cf6b0 2024-04-25T10:31:44.956 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:31:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f238c066f60 msgr2=0x7f238c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f238c066f60 0x7f238c069420 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f239c0079a0 tx=0x7f239c008040 comp rx=0 tx=0).stop 2024-04-25T10:31:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 msgr2=0x7f23a815fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:44.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 0x7f23a815fee0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c7a0 tx=0x7f23a400cc60 comp rx=0 tx=0).stop 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 shutdown_connections 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f238c066f60 0x7f238c069420 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23a80cf6b0 0x7f23a815fee0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23a80ce030 0x7f23a8157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 --2- 172.21.15.45:0/1733752448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23a806bd30 0x7f23a815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 >> 172.21.15.45:0/1733752448 conn(0x7f23a809a9d0 msgr2=0x7f23a80d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:44.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 shutdown_connections 2024-04-25T10:31:44.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:44.958+0000 7f23af200640 1 -- 172.21.15.45:0/1733752448 wait complete. 2024-04-25T10:31:44.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:31:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.138+0000 7f8146000640 1 Processor -- start 2024-04-25T10:31:45.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.138+0000 7f8146000640 1 -- start start 2024-04-25T10:31:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8146000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f814015ddf0 0x7f81400d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8146000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400d2270 con 0x7f81400b97f0 2024-04-25T10:31:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8146000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400b7210 con 0x7f814015da10 2024-04-25T10:31:45.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8146000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b74e0 con 0x7f814015ddf0 2024-04-25T10:31:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8136a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f81400b97f0 0x7f81400b6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47436/0 (socket says 172.21.15.45:47436) 2024-04-25T10:31:45.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8136a00640 1 -- 172.21.15.45:0/69419760 learned_addr learned my addr 172.21.15.45:0/69419760 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:45.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813fe00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f814015ddf0 0x7f81400d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410399574 0 0) 0x7f81400d2270 con 0x7f81400b97f0 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8124003530 con 0x7f81400b97f0 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264409651 0 0) 0x7f81400b7210 con 0x7f814015da10 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81400d2270 con 0x7f814015da10 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488069612 0 0) 0x7f8124003530 con 0x7f81400b97f0 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f814015ddf0 msgr2=0x7f81400d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f814015ddf0 0x7f81400d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 >> v1:172.21.15.117:6789/0 conn(0x7f814015da10 legacy=0x7f81400d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400b77c0 con 0x7f81400b97f0 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f8128001680 con 0x7f81400b97f0 2024-04-25T10:31:45.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 >> v1:172.21.15.45:6789/0 conn(0x7f81400b97f0 legacy=0x7f81400b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f8124003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8124004810 0x7f8124004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81240051d0 0x7f812400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b74e0 con 0x7f8124003b80 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812400bf40 con 0x7f81240051d0 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812400c140 con 0x7f8124004810 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813fe00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f814015ddf0 0x7f81400d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813f400640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f8124003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f8136a00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8124004810 0x7f8124004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813fe00640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81240051d0 0x7f812400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813f400640 1 -- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8124004810 msgr2=0x7f8124004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:45.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813f400640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8124004810 0x7f8124004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813f400640 1 -- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81240051d0 msgr2=0x7f812400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:45.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813f400640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81240051d0 0x7f812400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.142+0000 7f813f400640 1 -- 172.21.15.45:0/69419760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124003530 con 0x7f8124003b80 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f813f400640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f8124003f80 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f8130001fc0 tx=0x7f813001f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c98fc3d6d4db772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130001c10 con 0x7f8124003b80 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81300a8c90 con 0x7f8124003b80 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f813ea00640 1 -- 172.21.15.45:0/69419760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130003ba0 con 0x7f8124003b80 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 -- 172.21.15.45:0/69419760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 msgr2=0x7f8124003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 --2- 172.21.15.45:0/69419760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f8124003f80 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f8130001fc0 tx=0x7f813001f070 comp rx=0 tx=0).stop 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f813fe00640 1 -- 172.21.15.45:0/69419760 reap_dead start 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 -- 172.21.15.45:0/69419760 shutdown_connections 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 -- 172.21.15.45:0/69419760 >> 172.21.15.45:0/69419760 conn(0x7f814006ae90 msgr2=0x7f814006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:45.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 -- 172.21.15.45:0/69419760 shutdown_connections 2024-04-25T10:31:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 -- 172.21.15.45:0/69419760 wait complete. 2024-04-25T10:31:45.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 Processor -- start 2024-04-25T10:31:45.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.146+0000 7f8146000640 1 -- start start 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f814014d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81401563e0 0x7f8140153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140153e50 0x7f81401542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b8060 con 0x7f8124003b80 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b7d80 con 0x7f8140153e50 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81400d0940 con 0x7f81401563e0 2024-04-25T10:31:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f814014d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f814014d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41930/0 (socket says 172.21.15.45:41930) 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 -- 172.21.15.45:0/1739125327 learned_addr learned my addr 172.21.15.45:0/1739125327 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 -- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81401563e0 msgr2=0x7f8140153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81401563e0 0x7f8140153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 -- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140153e50 msgr2=0x7f81401542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140153e50 0x7f81401542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 -- 172.21.15.45:0/1739125327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140154810 con 0x7f8124003b80 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f813f400640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f814014d310 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f8130005040 tx=0x7f8124005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130001bf0 con 0x7f8124003b80 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81300a8a90 con 0x7f8124003b80 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130006d80 con 0x7f8124003b80 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8140154ac0 con 0x7f8124003b80 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8140155080 con 0x7f8124003b80 2024-04-25T10:31:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.150+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814004ef30 con 0x7f8124003b80 2024-04-25T10:31:45.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.154+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81300050d0 con 0x7f8124003b80 2024-04-25T10:31:45.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.154+0000 7f8137e00640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:45.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.154+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f8130122660 con 0x7f8124003b80 2024-04-25T10:31:45.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.154+0000 7f8136a00640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:45.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.154+0000 7f8136a00640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f811c066f60 0x7f811c069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f81400ceef0 tx=0x7f8128000f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.158+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81300bc020 con 0x7f8124003b80 2024-04-25T10:31:45.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.326+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81400d4560 con 0x7f8124003b80 2024-04-25T10:31:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.962+0000 7f8137e00640 1 -- 172.21.15.45:0/1739125327 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v185) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8130018d80 con 0x7f8124003b80 2024-04-25T10:31:45.965 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:31:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f811c066f60 msgr2=0x7f811c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f811c066f60 0x7f811c069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f81400ceef0 tx=0x7f8128000f80 comp rx=0 tx=0).stop 2024-04-25T10:31:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 msgr2=0x7f814014d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:45.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f814014d310 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f8130005040 tx=0x7f8124005840 comp rx=0 tx=0).stop 2024-04-25T10:31:45.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 shutdown_connections 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8140153e50 0x7f81401542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81401563e0 0x7f8140153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 --2- 172.21.15.45:0/1739125327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8124003b80 0x7f814014d310 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 >> 172.21.15.45:0/1739125327 conn(0x7f814006ae90 msgr2=0x7f81400d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 shutdown_connections 2024-04-25T10:31:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:45.966+0000 7f8146000640 1 -- 172.21.15.45:0/1739125327 wait complete. 2024-04-25T10:31:46.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:31:46.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.142+0000 7f5293a00640 1 Processor -- start 2024-04-25T10:31:46.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.142+0000 7f5293a00640 1 -- start start 2024-04-25T10:31:46.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5293a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f528c0aef80 0x7f528c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5293a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f528c0b31c0 0x7f528c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5293a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c15b080 con 0x7f528c070350 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5293a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c15adb0 con 0x7f528c0aef80 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5293a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c15eb60 con 0x7f528c0b31c0 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5291000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f528c0aef80 0x7f528c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5283e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f528c070350 0x7f528c15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47440/0 (socket says 172.21.15.45:47440) 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5283e00640 1 -- 172.21.15.45:0/1412548616 learned_addr learned my addr 172.21.15.45:0/1412548616 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5291a00640 1 --2- 172.21.15.45:0/1412548616 >> v2:172.21.15.117:3300/0 conn(0x7f528c0b31c0 0x7f528c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5291000640 1 -- 172.21.15.45:0/1412548616 >> v2:172.21.15.117:3300/0 conn(0x7f528c0b31c0 msgr2=0x7f528c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5291000640 1 --2- 172.21.15.45:0/1412548616 >> v2:172.21.15.117:3300/0 conn(0x7f528c0b31c0 0x7f528c0b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5291000640 1 -- 172.21.15.45:0/1412548616 >> v1:172.21.15.45:6789/0 conn(0x7f528c070350 legacy=0x7f528c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f5291000640 1 -- 172.21.15.45:0/1412548616 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c15ee40 con 0x7f528c0aef80 2024-04-25T10:31:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.146+0000 7f528be00640 1 -- 172.21.15.45:0/1412548616 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555637443 0 0) 0x7f528c15b080 con 0x7f528c070350 2024-04-25T10:31:46.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5291000640 1 --2- 172.21.15.45:0/1412548616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c0aef80 0x7f528c15e480 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f528c0d0750 tx=0x7f527c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4d5ab677cb62d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:46.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f528be00640 1 -- 172.21.15.45:0/1412548616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c0b1070 con 0x7f528c0aef80 2024-04-25T10:31:46.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f528be00640 1 -- 172.21.15.45:0/1412548616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c0ac5f0 con 0x7f528c0aef80 2024-04-25T10:31:46.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f528be00640 1 -- 172.21.15.45:0/1412548616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c0ac9c0 con 0x7f528c0aef80 2024-04-25T10:31:46.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 -- 172.21.15.45:0/1412548616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c0aef80 msgr2=0x7f528c15e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 --2- 172.21.15.45:0/1412548616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c0aef80 0x7f528c15e480 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f528c0d0750 tx=0x7f527c01c480 comp rx=0 tx=0).stop 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 -- 172.21.15.45:0/1412548616 shutdown_connections 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 --2- 172.21.15.45:0/1412548616 >> v2:172.21.15.117:3300/0 conn(0x7f528c0b31c0 0x7f528c0b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 --2- 172.21.15.45:0/1412548616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c0aef80 0x7f528c15e480 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 -- 172.21.15.45:0/1412548616 >> 172.21.15.45:0/1412548616 conn(0x7f528c0669a0 msgr2=0x7f528c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 -- 172.21.15.45:0/1412548616 shutdown_connections 2024-04-25T10:31:46.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 -- 172.21.15.45:0/1412548616 wait complete. 2024-04-25T10:31:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 Processor -- start 2024-04-25T10:31:46.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.150+0000 7f5293a00640 1 -- start start 2024-04-25T10:31:46.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 0x7f528c150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c0aef80 0x7f528c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c0b31c0 0x7f528c0b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1679a0 con 0x7f528c070350 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c167820 con 0x7f528c0aef80 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c167b20 con 0x7f528c0b31c0 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 0x7f528c150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 0x7f528c150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41950/0 (socket says 172.21.15.45:41950) 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 -- 172.21.15.45:0/3859392599 learned_addr learned my addr 172.21.15.45:0/3859392599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5283e00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c0aef80 0x7f528c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c0b31c0 0x7f528c0b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 -- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c0b31c0 msgr2=0x7f528c0b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c0b31c0 0x7f528c0b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 -- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c0aef80 msgr2=0x7f528c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c0aef80 0x7f528c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 -- 172.21.15.45:0/3859392599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c1548e0 con 0x7f528c070350 2024-04-25T10:31:46.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5291000640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 0x7f528c150a00 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f527c01ab30 tx=0x7f527c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:46.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c0b1040 con 0x7f528c070350 2024-04-25T10:31:46.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c0bc070 con 0x7f528c070350 2024-04-25T10:31:46.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c0b1a10 con 0x7f528c070350 2024-04-25T10:31:46.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c0b5410 con 0x7f528c070350 2024-04-25T10:31:46.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c1540e0 con 0x7f528c070350 2024-04-25T10:31:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c153c10 con 0x7f528c070350 2024-04-25T10:31:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.154+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f527c0b55b0 con 0x7f528c070350 2024-04-25T10:31:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.158+0000 7f5289600640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5268066f60 0x7f5268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.158+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f527c11e910 con 0x7f528c070350 2024-04-25T10:31:46.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.158+0000 7f5283e00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5268066f60 0x7f5268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.158+0000 7f5283e00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5268066f60 0x7f5268069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f528c151b10 tx=0x7f5274007b60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.162+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f527c0a9070 con 0x7f528c070350 2024-04-25T10:31:46.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.330+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f528c04ef30 con 0x7f528c070350 2024-04-25T10:31:46.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5289600640 1 -- 172.21.15.45:0/3859392599 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f527c016070 con 0x7f528c070350 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5268066f60 msgr2=0x7f5268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5268066f60 0x7f5268069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f528c151b10 tx=0x7f5274007b60 comp rx=0 tx=0).stop 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 msgr2=0x7f528c150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 0x7f528c150a00 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f527c01ab30 tx=0x7f527c019220 comp rx=0 tx=0).stop 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 shutdown_connections 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5268066f60 0x7f5268069420 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c0b31c0 0x7f528c0b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c0aef80 0x7f528c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 --2- 172.21.15.45:0/3859392599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f528c070350 0x7f528c150a00 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 >> 172.21.15.45:0/3859392599 conn(0x7f528c0669a0 msgr2=0x7f528c0b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 shutdown_connections 2024-04-25T10:31:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.334+0000 7f5293a00640 1 -- 172.21.15.45:0/3859392599 wait complete. 2024-04-25T10:31:46.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:31:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 Processor -- start 2024-04-25T10:31:46.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 -- start start 2024-04-25T10:31:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f661415a780 0x7f661415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f66140cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f66140cfd30 0x7f6614071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661406de60 con 0x7f661415a780 2024-04-25T10:31:46.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661415f2f0 con 0x7f66140cfd30 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6618c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140d01c0 con 0x7f66140cf3b0 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f661415a780 0x7f661415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f661415a780 0x7f661415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41954/0 (socket says 172.21.15.45:41954) 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 -- 172.21.15.45:0/2257387752 learned_addr learned my addr 172.21.15.45:0/2257387752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613e00640 1 --2- 172.21.15.45:0/2257387752 >> v2:172.21.15.117:3300/0 conn(0x7f66140cfd30 0x7f6614071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6612a00640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f66140cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 -- 172.21.15.45:0/2257387752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 msgr2=0x7f66140cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f66140cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 -- 172.21.15.45:0/2257387752 >> v2:172.21.15.117:3300/0 conn(0x7f66140cfd30 msgr2=0x7f6614071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 --2- 172.21.15.45:0/2257387752 >> v2:172.21.15.117:3300/0 conn(0x7f66140cfd30 0x7f6614071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613400640 1 -- 172.21.15.45:0/2257387752 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661406cdc0 con 0x7f661415a780 2024-04-25T10:31:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6613e00640 1 --2- 172.21.15.45:0/2257387752 >> v2:172.21.15.117:3300/0 conn(0x7f66140cfd30 0x7f6614071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:46.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.502+0000 7f6612a00640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f66140cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6613400640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f661415edb0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f66140cc190 tx=0x7f660401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9468f4a09d1955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:46.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6612000640 1 -- 172.21.15.45:0/2257387752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040b0070 con 0x7f661415a780 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6612000640 1 -- 172.21.15.45:0/2257387752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66040ab600 con 0x7f661415a780 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6612000640 1 -- 172.21.15.45:0/2257387752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040ab9d0 con 0x7f661415a780 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 -- 172.21.15.45:0/2257387752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 msgr2=0x7f661415edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f661415edb0 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f66140cc190 tx=0x7f660401c520 comp rx=0 tx=0).stop 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 -- 172.21.15.45:0/2257387752 shutdown_connections 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 --2- 172.21.15.45:0/2257387752 >> v2:172.21.15.117:3300/0 conn(0x7f66140cfd30 0x7f6614071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f66140cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 --2- 172.21.15.45:0/2257387752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f661415edb0 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 -- 172.21.15.45:0/2257387752 >> 172.21.15.45:0/2257387752 conn(0x7f66140743f0 msgr2=0x7f66140669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:46.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 -- 172.21.15.45:0/2257387752 shutdown_connections 2024-04-25T10:31:46.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 -- 172.21.15.45:0/2257387752 wait complete. 2024-04-25T10:31:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 Processor -- start 2024-04-25T10:31:46.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.506+0000 7f6618c00640 1 -- start start 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f661415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66140cfd30 0x7f6614136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f66141521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140b2460 con 0x7f661415a780 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661415ab60 con 0x7f66140cfd30 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140b25e0 con 0x7f66140cf3b0 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f66141521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f66141521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41964/0 (socket says 172.21.15.45:41964) 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 -- 172.21.15.45:0/2342061253 learned_addr learned my addr 172.21.15.45:0/2342061253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:46.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6612a00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66140cfd30 0x7f6614136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613400640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f661415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 -- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 msgr2=0x7f661415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f661415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 -- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66140cfd30 msgr2=0x7f6614136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66140cfd30 0x7f6614136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 -- 172.21.15.45:0/2342061253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614154f70 con 0x7f661415a780 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6613e00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f66141521a0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f660c00cef0 tx=0x7f660c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c01a070 con 0x7f661415a780 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00d860 con 0x7f661415a780 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66141554c0 con 0x7f661415a780 2024-04-25T10:31:46.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00dc50 con 0x7f661415a780 2024-04-25T10:31:46.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661415cbf0 con 0x7f661415a780 2024-04-25T10:31:46.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.510+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6614075120 con 0x7f661415a780 2024-04-25T10:31:46.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.514+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f660c01d070 con 0x7f661415a780 2024-04-25T10:31:46.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.514+0000 7f660b400640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65f4066f60 0x7f65f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:46.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.514+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f660c085530 con 0x7f661415a780 2024-04-25T10:31:46.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.514+0000 7f6613400640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65f4066f60 0x7f65f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:46.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.514+0000 7f6613400640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65f4066f60 0x7f65f4069420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f66040ad000 tx=0x7f6604008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:46.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.518+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f660c0562b0 con 0x7f661415a780 2024-04-25T10:31:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.738+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f66140b0df0 con 0x7f661415a780 2024-04-25T10:31:46.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.962+0000 7f660b400640 1 -- 172.21.15.45:0/2342061253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v666) v1 ==== 147+0+0 (secure 0 0 0) 0x7f660c05a350 con 0x7f661415a780 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65f4066f60 msgr2=0x7f65f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65f4066f60 0x7f65f4069420 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f66040ad000 tx=0x7f6604008040 comp rx=0 tx=0).stop 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 msgr2=0x7f66141521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f66141521a0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f660c00cef0 tx=0x7f660c00a710 comp rx=0 tx=0).stop 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 shutdown_connections 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f65f4066f60 0x7f65f4069420 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f661415a780 0x7f66141521a0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66140cfd30 0x7f6614136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 --2- 172.21.15.45:0/2342061253 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66140cf3b0 0x7f661415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:46.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 >> 172.21.15.45:0/2342061253 conn(0x7f66140743f0 msgr2=0x7f66140cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:46.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 shutdown_connections 2024-04-25T10:31:46.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:46.966+0000 7f6618c00640 1 -- 172.21.15.45:0/2342061253 wait complete. 2024-04-25T10:31:46.990 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:31:46.990 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:31:46.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:47.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 Processor -- start 2024-04-25T10:31:47.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 -- start start 2024-04-25T10:31:47.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77b815a0e0 0x7f77b815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f77b8071f70 0x7f77b80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b80b2fc0 0x7f77b806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b815b6e0 con 0x7f77b815a0e0 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b8070ed0 con 0x7f77b8071f70 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.130+0000 7f77bdc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b80711b0 con 0x7f77b80b2fc0 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77b815a0e0 0x7f77b815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77b815a0e0 0x7f77b815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41974/0 (socket says 172.21.15.45:41974) 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 -- 172.21.15.45:0/3597029456 learned_addr learned my addr 172.21.15.45:0/3597029456 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77ae000640 1 --2- 172.21.15.45:0/3597029456 >> v2:172.21.15.117:3300/0 conn(0x7f77b8071f70 0x7f77b80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b7400640 1 --2- 172.21.15.45:0/3597029456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b80b2fc0 0x7f77b806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 -- 172.21.15.45:0/3597029456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b80b2fc0 msgr2=0x7f77b806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 --2- 172.21.15.45:0/3597029456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b80b2fc0 0x7f77b806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 -- 172.21.15.45:0/3597029456 >> v2:172.21.15.117:3300/0 conn(0x7f77b8071f70 msgr2=0x7f77b80b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:47.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 --2- 172.21.15.45:0/3597029456 >> v2:172.21.15.117:3300/0 conn(0x7f77b8071f70 0x7f77b80b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 -- 172.21.15.45:0/3597029456 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b8071490 con 0x7f77b815a0e0 2024-04-25T10:31:47.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6a00640 1 --2- 172.21.15.45:0/3597029456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b815a0e0 0x7f77b815b1a0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f77b806e5d0 tx=0x7f77a001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2492bb7d285bd1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6000640 1 -- 172.21.15.45:0/3597029456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00b4070 con 0x7f77b815a0e0 2024-04-25T10:31:47.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6000640 1 -- 172.21.15.45:0/3597029456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a00a7af0 con 0x7f77b815a0e0 2024-04-25T10:31:47.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.134+0000 7f77b6000640 1 -- 172.21.15.45:0/3597029456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00af440 con 0x7f77b815a0e0 2024-04-25T10:31:47.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- 172.21.15.45:0/3597029456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b815a0e0 msgr2=0x7f77b815b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- 172.21.15.45:0/3597029456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b815a0e0 0x7f77b815b1a0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f77b806e5d0 tx=0x7f77a001ebf0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- 172.21.15.45:0/3597029456 shutdown_connections 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- 172.21.15.45:0/3597029456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b80b2fc0 0x7f77b806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- 172.21.15.45:0/3597029456 >> v2:172.21.15.117:3300/0 conn(0x7f77b8071f70 0x7f77b80b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- 172.21.15.45:0/3597029456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b815a0e0 0x7f77b815b1a0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- 172.21.15.45:0/3597029456 >> 172.21.15.45:0/3597029456 conn(0x7f77b8066b70 msgr2=0x7f77b8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- 172.21.15.45:0/3597029456 shutdown_connections 2024-04-25T10:31:47.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- 172.21.15.45:0/3597029456 wait complete. 2024-04-25T10:31:47.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 Processor -- start 2024-04-25T10:31:47.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- start start 2024-04-25T10:31:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 0x7f77b80a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b80b2fc0 0x7f77b814c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b81f7810 0x7f77b80a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8070b50 con 0x7f77b8071f70 2024-04-25T10:31:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b815a4c0 con 0x7f77b80b2fc0 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77bdc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8072350 con 0x7f77b81f7810 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 0x7f77b80a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 0x7f77b80a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41986/0 (socket says 172.21.15.45:41986) 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 -- 172.21.15.45:0/4065970119 learned_addr learned my addr 172.21.15.45:0/4065970119 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77ae000640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b80b2fc0 0x7f77b814c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b7400640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b81f7810 0x7f77b80a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 -- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b81f7810 msgr2=0x7f77b80a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b81f7810 0x7f77b80a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 -- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b80b2fc0 msgr2=0x7f77b814c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b80b2fc0 0x7f77b814c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.138+0000 7f77b6a00640 1 -- 172.21.15.45:0/4065970119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b81f81d0 con 0x7f77b8071f70 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77b6a00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 0x7f77b80a9430 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b2c0 tx=0x7f77a001d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00bf070 con 0x7f77b8071f70 2024-04-25T10:31:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a00ba440 con 0x7f77b8071f70 2024-04-25T10:31:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00b4070 con 0x7f77b8071f70 2024-04-25T10:31:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b81f84b0 con 0x7f77b8071f70 2024-04-25T10:31:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b814c690 con 0x7f77b8071f70 2024-04-25T10:31:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b804ef30 con 0x7f77b8071f70 2024-04-25T10:31:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77a00b93e0 con 0x7f77b8071f70 2024-04-25T10:31:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77af400640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7794066f60 0x7f7794069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f77a011f510 con 0x7f77b8071f70 2024-04-25T10:31:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.142+0000 7f77ae000640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7794066f60 0x7f7794069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.146+0000 7f77ae000640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7794066f60 0x7f7794069420 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f77b80ae530 tx=0x7f77a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.146+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f77a00187e0 con 0x7f77b8071f70 2024-04-25T10:31:47.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77b814d810 con 0x7f77b8071f70 2024-04-25T10:31:47.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77af400640 1 -- 172.21.15.45:0/4065970119 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+13018 (secure 0 0 0) 0x7f77a00f0780 con 0x7f77b8071f70 2024-04-25T10:31:47.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:47.313 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":185,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:45.958714+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1455178901":"2024-04-26T10:31:42.758609+0000","172.21.15.45:6833/1455178901":"2024-04-26T10:31:42.758609+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7794066f60 msgr2=0x7f7794069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7794066f60 0x7f7794069420 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f77b80ae530 tx=0x7f77a8008040 comp rx=0 tx=0).stop 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 msgr2=0x7f77b80a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 0x7f77b80a9430 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b2c0 tx=0x7f77a001d430 comp rx=0 tx=0).stop 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 shutdown_connections 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7794066f60 0x7f7794069420 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77b81f7810 0x7f77b80a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77b80b2fc0 0x7f77b814c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 --2- 172.21.15.45:0/4065970119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77b8071f70 0x7f77b80a9430 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 >> 172.21.15.45:0/4065970119 conn(0x7f77b8066b70 msgr2=0x7f77b8154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.310+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 shutdown_connections 2024-04-25T10:31:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.314+0000 7f77bdc00640 1 -- 172.21.15.45:0/4065970119 wait complete. 2024-04-25T10:31:47.335 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:47.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:47.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 Processor -- start 2024-04-25T10:31:47.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 -- start start 2024-04-25T10:31:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4159540 0x7f76e40ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76e40aebd0 0x7f76e4071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e4071360 con 0x7f76e4072020 2024-04-25T10:31:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e40af030 con 0x7f76e40aebd0 2024-04-25T10:31:47.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76eb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4159920 con 0x7f76e4159540 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76dbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76e40aebd0 0x7f76e4071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76e9600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f76e4072020 0x7f76e415f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47452/0 (socket says 172.21.15.45:47452) 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76e9600640 1 -- 172.21.15.45:0/4184516248 learned_addr learned my addr 172.21.15.45:0/4184516248 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76e8c00640 1 --2- 172.21.15.45:0/4184516248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4159540 0x7f76e40ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76dbe00640 1 -- 172.21.15.45:0/4184516248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4159540 msgr2=0x7f76e40ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76dbe00640 1 --2- 172.21.15.45:0/4184516248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4159540 0x7f76e40ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76dbe00640 1 -- 172.21.15.45:0/4184516248 >> v1:172.21.15.45:6789/0 conn(0x7f76e4072020 legacy=0x7f76e415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76dbe00640 1 -- 172.21.15.45:0/4184516248 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e415c2f0 con 0x7f76e40aebd0 2024-04-25T10:31:47.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.478+0000 7f76e3e00640 1 -- 172.21.15.45:0/4184516248 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396273024 0 0) 0x7f76e4071360 con 0x7f76e4072020 2024-04-25T10:31:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76dbe00640 1 --2- 172.21.15.45:0/4184516248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e40aebd0 0x7f76e4071ae0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0097c0 tx=0x7f76cc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdd2fa4e554aad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76e3e00640 1 -- 172.21.15.45:0/4184516248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0b2070 con 0x7f76e40aebd0 2024-04-25T10:31:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76e3e00640 1 -- 172.21.15.45:0/4184516248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc0ad640 con 0x7f76e40aebd0 2024-04-25T10:31:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76e3e00640 1 -- 172.21.15.45:0/4184516248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0ada10 con 0x7f76e40aebd0 2024-04-25T10:31:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 -- 172.21.15.45:0/4184516248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e40aebd0 msgr2=0x7f76e4071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 --2- 172.21.15.45:0/4184516248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e40aebd0 0x7f76e4071ae0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0097c0 tx=0x7f76cc01c470 comp rx=0 tx=0).stop 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 -- 172.21.15.45:0/4184516248 shutdown_connections 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 --2- 172.21.15.45:0/4184516248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e40aebd0 0x7f76e4071ae0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 --2- 172.21.15.45:0/4184516248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4159540 0x7f76e40ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 -- 172.21.15.45:0/4184516248 >> 172.21.15.45:0/4184516248 conn(0x7f76e40669a0 msgr2=0x7f76e4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 -- 172.21.15.45:0/4184516248 shutdown_connections 2024-04-25T10:31:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.482+0000 7f76eb600640 1 -- 172.21.15.45:0/4184516248 wait complete. 2024-04-25T10:31:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 Processor -- start 2024-04-25T10:31:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- start start 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4072020 0x7f76e41495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76e40aebd0 0x7f76e41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 0x7f76e41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e406c0d0 con 0x7f76e4159540 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e415bfa0 con 0x7f76e40aebd0 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4072430 con 0x7f76e4072020 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 0x7f76e41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 0x7f76e41589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42010/0 (socket says 172.21.15.45:42010) 2024-04-25T10:31:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 -- 172.21.15.45:0/1780306418 learned_addr learned my addr 172.21.15.45:0/1780306418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e8c00640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4072020 0x7f76e41495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76dbe00640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76e40aebd0 0x7f76e41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 -- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4072020 msgr2=0x7f76e41495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4072020 0x7f76e41495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 -- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76e40aebd0 msgr2=0x7f76e41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76e40aebd0 0x7f76e41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 -- 172.21.15.45:0/1780306418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e41f9a00 con 0x7f76e4159540 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e9600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 0x7f76e41589e0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c7d0 tx=0x7f76dc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc018070 con 0x7f76e4159540 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc00a640 con 0x7f76e4159540 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00aa30 con 0x7f76e4159540 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e41f9ca0 con 0x7f76e4159540 2024-04-25T10:31:47.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e41fa260 con 0x7f76e4159540 2024-04-25T10:31:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.486+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e404ef30 con 0x7f76e4159540 2024-04-25T10:31:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.490+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76dc00abd0 con 0x7f76e4159540 2024-04-25T10:31:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.490+0000 7f76e1600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76c0066f60 0x7f76c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.490+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f76dc084410 con 0x7f76e4159540 2024-04-25T10:31:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.490+0000 7f76e8c00640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76c0066f60 0x7f76c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.490+0000 7f76e8c00640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76c0066f60 0x7f76c0069420 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b930 tx=0x7f76d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.494+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76dc055680 con 0x7f76e4159540 2024-04-25T10:31:47.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.658+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76e40cf470 con 0x7f76e4159540 2024-04-25T10:31:47.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.658+0000 7f76e1600640 1 -- 172.21.15.45:0/1780306418 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76dc054f80 con 0x7f76e4159540 2024-04-25T10:31:47.661 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:47.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.658+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76c0066f60 msgr2=0x7f76c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76c0066f60 0x7f76c0069420 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b930 tx=0x7f76d4008040 comp rx=0 tx=0).stop 2024-04-25T10:31:47.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 msgr2=0x7f76e41589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 0x7f76e41589e0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c7d0 tx=0x7f76dc00cc90 comp rx=0 tx=0).stop 2024-04-25T10:31:47.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 shutdown_connections 2024-04-25T10:31:47.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f76c0066f60 0x7f76c0069420 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76e4159540 0x7f76e41589e0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76e40aebd0 0x7f76e41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 --2- 172.21.15.45:0/1780306418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76e4072020 0x7f76e41495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 >> 172.21.15.45:0/1780306418 conn(0x7f76e40669a0 msgr2=0x7f76e40cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:47.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 shutdown_connections 2024-04-25T10:31:47.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.662+0000 7f76eb600640 1 -- 172.21.15.45:0/1780306418 wait complete. 2024-04-25T10:31:47.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:31:47.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 Processor -- start 2024-04-25T10:31:47.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 -- start start 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0ec1550b0 0x7fc0ec0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc0ec15b910 0x7fc0ec0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec071050 con 0x7fc0ec0cfe70 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec15bd80 con 0x7fc0ec1550b0 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0f1a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec15d040 con 0x7fc0ec15b910 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0ec1550b0 0x7fc0ec0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0ec1550b0 0x7fc0ec0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42024/0 (socket says 172.21.15.45:42024) 2024-04-25T10:31:47.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 -- 172.21.15.45:0/3574710965 learned_addr learned my addr 172.21.15.45:0/3574710965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:47.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eb400640 1 --2- 172.21.15.45:0/3574710965 >> v2:172.21.15.117:3300/0 conn(0x7fc0ec15b910 0x7fc0ec0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 -- 172.21.15.45:0/3574710965 >> v2:172.21.15.117:3300/0 conn(0x7fc0ec15b910 msgr2=0x7fc0ec0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:47.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 --2- 172.21.15.45:0/3574710965 >> v2:172.21.15.117:3300/0 conn(0x7fc0ec15b910 0x7fc0ec0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 -- 172.21.15.45:0/3574710965 >> v1:172.21.15.117:6789/0 conn(0x7fc0ec0cfe70 legacy=0x7fc0ec15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.822+0000 7fc0eaa00640 1 -- 172.21.15.45:0/3574710965 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0712f0 con 0x7fc0ec1550b0 2024-04-25T10:31:47.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0eaa00640 1 --2- 172.21.15.45:0/3574710965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec0cf930 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00bb10 tx=0x7fc0dc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984944728336c859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0ea000640 1 -- 172.21.15.45:0/3574710965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0b4070 con 0x7fc0ec1550b0 2024-04-25T10:31:47.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0ea000640 1 -- 172.21.15.45:0/3574710965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0dc0a7bb0 con 0x7fc0ec1550b0 2024-04-25T10:31:47.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0ea000640 1 -- 172.21.15.45:0/3574710965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0af440 con 0x7fc0ec1550b0 2024-04-25T10:31:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 -- 172.21.15.45:0/3574710965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 msgr2=0x7fc0ec0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/3574710965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec0cf930 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00bb10 tx=0x7fc0dc01ebf0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 -- 172.21.15.45:0/3574710965 shutdown_connections 2024-04-25T10:31:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/3574710965 >> v2:172.21.15.117:3300/0 conn(0x7fc0ec15b910 0x7fc0ec0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/3574710965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec0cf930 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 -- 172.21.15.45:0/3574710965 >> 172.21.15.45:0/3574710965 conn(0x7fc0ec066b70 msgr2=0x7fc0ec066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:47.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 -- 172.21.15.45:0/3574710965 shutdown_connections 2024-04-25T10:31:47.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.826+0000 7fc0f1a00640 1 -- 172.21.15.45:0/3574710965 wait complete. 2024-04-25T10:31:47.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 Processor -- start 2024-04-25T10:31:47.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- start start 2024-04-25T10:31:47.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ec0cfe70 0x7fc0ec13f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ec15b910 0x7fc0ec13c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec15c030 con 0x7fc0ec1550b0 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0720b0 con 0x7fc0ec0cfe70 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec15c1b0 con 0x7fc0ec15b910 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec152f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42034/0 (socket says 172.21.15.45:42034) 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 -- 172.21.15.45:0/2472364870 learned_addr learned my addr 172.21.15.45:0/2472364870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0eaa00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ec0cfe70 0x7fc0ec13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0eb400640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ec15b910 0x7fc0ec13c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 -- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ec15b910 msgr2=0x7fc0ec13c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ec15b910 0x7fc0ec13c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 -- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ec0cfe70 msgr2=0x7fc0ec13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ec0cfe70 0x7fc0ec13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 -- 172.21.15.45:0/2472364870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec13ce40 con 0x7fc0ec1550b0 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e2000640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec152f00 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c420 tx=0x7fc0d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40052d0 con 0x7fc0ec1550b0 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4005470 con 0x7fc0ec1550b0 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4007260 con 0x7fc0ec1550b0 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec153650 con 0x7fc0ec1550b0 2024-04-25T10:31:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec153c10 con 0x7fc0ec1550b0 2024-04-25T10:31:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.830+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec04ef30 con 0x7fc0ec1550b0 2024-04-25T10:31:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.834+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0d4007400 con 0x7fc0ec1550b0 2024-04-25T10:31:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.834+0000 7fc0e3400640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc0c8066f60 0x7fc0c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.834+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0d4084250 con 0x7fc0ec1550b0 2024-04-25T10:31:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.834+0000 7fc0eaa00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc0c8066f60 0x7fc0c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.834+0000 7fc0eaa00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc0c8066f60 0x7fc0c8069420 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0b1000 tx=0x7fc0dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:47.838+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0d4054e70 con 0x7fc0ec1550b0 2024-04-25T10:31:48.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.058+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7fc0ec0cbd40 con 0x7fc0ec1550b0 2024-04-25T10:31:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0e3400640 1 -- 172.21.15.45:0/2472364870 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v668) v1 ==== 136+0+0 (secure 0 0 0) 0x7fc0d4058f10 con 0x7fc0ec1550b0 2024-04-25T10:31:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc0c8066f60 msgr2=0x7fc0c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc0c8066f60 0x7fc0c8069420 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0b1000 tx=0x7fc0dc008040 comp rx=0 tx=0).stop 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 msgr2=0x7fc0ec152f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec152f00 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c420 tx=0x7fc0d400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 shutdown_connections 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc0c8066f60 0x7fc0c8069420 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ec15b910 0x7fc0ec13c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0ec1550b0 0x7fc0ec152f00 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 --2- 172.21.15.45:0/2472364870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ec0cfe70 0x7fc0ec13f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 >> 172.21.15.45:0/2472364870 conn(0x7fc0ec066b70 msgr2=0x7fc0ec0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 shutdown_connections 2024-04-25T10:31:48.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:48.974+0000 7fc0f1a00640 1 -- 172.21.15.45:0/2472364870 wait complete. 2024-04-25T10:31:49.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:31:49.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 Processor -- start 2024-04-25T10:31:49.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 -- start start 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f893815bb10 0x7f89380d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89380cff50 0x7f893815f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89380cd900 con 0x7f893815bef0 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380d0960 con 0x7f89380cff50 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380cc470 con 0x7f893815bb10 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89380cff50 0x7f893815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89380cff50 0x7f893815f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42048/0 (socket says 172.21.15.45:42048) 2024-04-25T10:31:49.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 -- 172.21.15.45:0/2563567135 learned_addr learned my addr 172.21.15.45:0/2563567135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 -- 172.21.15.45:0/2563567135 >> v2:172.21.15.117:3300/0 conn(0x7f893815bb10 msgr2=0x7f89380d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 --2- 172.21.15.45:0/2563567135 >> v2:172.21.15.117:3300/0 conn(0x7f893815bb10 0x7f89380d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 -- 172.21.15.45:0/2563567135 >> v1:172.21.15.117:6789/0 conn(0x7f893815bef0 legacy=0x7f89380cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.150+0000 7f893ce00640 1 -- 172.21.15.45:0/2563567135 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380d0680 con 0x7f89380cff50 2024-04-25T10:31:49.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ce00640 1 --2- 172.21.15.45:0/2563567135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89380cff50 0x7f893815f810 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f89380cf4a0 tx=0x7f893001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe9dfd97f2b594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f8936a00640 1 -- 172.21.15.45:0/2563567135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300a5750 con 0x7f89380cff50 2024-04-25T10:31:49.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f8936a00640 1 -- 172.21.15.45:0/2563567135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300a58f0 con 0x7f89380cff50 2024-04-25T10:31:49.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f8936a00640 1 -- 172.21.15.45:0/2563567135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300a5cc0 con 0x7f89380cff50 2024-04-25T10:31:49.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 -- 172.21.15.45:0/2563567135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89380cff50 msgr2=0x7f893815f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 --2- 172.21.15.45:0/2563567135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89380cff50 0x7f893815f810 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f89380cf4a0 tx=0x7f893001c980 comp rx=0 tx=0).stop 2024-04-25T10:31:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 -- 172.21.15.45:0/2563567135 shutdown_connections 2024-04-25T10:31:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 --2- 172.21.15.45:0/2563567135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89380cff50 0x7f893815f810 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 --2- 172.21.15.45:0/2563567135 >> v2:172.21.15.117:3300/0 conn(0x7f893815bb10 0x7f89380d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 -- 172.21.15.45:0/2563567135 >> 172.21.15.45:0/2563567135 conn(0x7f89380d19a0 msgr2=0x7f89380d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 -- 172.21.15.45:0/2563567135 shutdown_connections 2024-04-25T10:31:49.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 -- 172.21.15.45:0/2563567135 wait complete. 2024-04-25T10:31:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.154+0000 7f893ee00640 1 Processor -- start 2024-04-25T10:31:49.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- start start 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89380cff50 0x7f8938159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893815bb10 0x7f89380c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 0x7f89380c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8938168b50 con 0x7f893815bef0 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380d03a0 con 0x7f893815bb10 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8938168cd0 con 0x7f89380cff50 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 0x7f89380c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 0x7f89380c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42052/0 (socket says 172.21.15.45:42052) 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 -- 172.21.15.45:0/3219212419 learned_addr learned my addr 172.21.15.45:0/3219212419 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f8937e00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89380cff50 0x7f8938159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f8937400640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893815bb10 0x7f89380c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 -- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89380cff50 msgr2=0x7f8938159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89380cff50 0x7f8938159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 -- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893815bb10 msgr2=0x7f89380c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893815bb10 0x7f89380c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 -- 172.21.15.45:0/3219212419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89381588a0 con 0x7f893815bef0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ce00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 0x7f89380c8fb0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f893001a5b0 tx=0x7f8930019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300b8670 con 0x7f893815bef0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300be070 con 0x7f893815bef0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300b8a90 con 0x7f893815bef0 2024-04-25T10:31:49.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8938158b80 con 0x7f893815bef0 2024-04-25T10:31:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8938159140 con 0x7f893815bef0 2024-04-25T10:31:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.158+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893804ef30 con 0x7f893815bef0 2024-04-25T10:31:49.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.162+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89300b8c30 con 0x7f893815bef0 2024-04-25T10:31:49.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.162+0000 7f892fe00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8918066f60 0x7f8918069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.162+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8930120080 con 0x7f893815bef0 2024-04-25T10:31:49.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.162+0000 7f8937e00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8918066f60 0x7f8918069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.162+0000 7f8937e00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8918066f60 0x7f8918069420 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8928006fd0 tx=0x7f8928008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.166+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89300165a0 con 0x7f893815bef0 2024-04-25T10:31:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.330+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f89380b7920 con 0x7f893815bef0 2024-04-25T10:31:49.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f892fe00640 1 -- 172.21.15.45:0/3219212419 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f89300f12f0 con 0x7f893815bef0 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8918066f60 msgr2=0x7f8918069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8918066f60 0x7f8918069420 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8928006fd0 tx=0x7f8928008040 comp rx=0 tx=0).stop 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 msgr2=0x7f89380c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 0x7f89380c8fb0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f893001a5b0 tx=0x7f8930019720 comp rx=0 tx=0).stop 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 shutdown_connections 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8918066f60 0x7f8918069420 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f893815bef0 0x7f89380c8fb0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f893815bb10 0x7f89380c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 --2- 172.21.15.45:0/3219212419 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89380cff50 0x7f8938159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 >> 172.21.15.45:0/3219212419 conn(0x7f89380d19a0 msgr2=0x7f893815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 shutdown_connections 2024-04-25T10:31:49.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.334+0000 7f893ee00640 1 -- 172.21.15.45:0/3219212419 wait complete. 2024-04-25T10:31:49.357 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:31:49.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:49.422 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:31:49.469 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:31:49.470 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:49.470 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:49.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:49.479 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:49.479 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:49.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:49.522 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:31:49.523 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:49.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:31:49.531 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:31:49.532 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:31:49.574 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:31:49.574 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:31:49.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.710+0000 7fa17a400640 1 Processor -- start 2024-04-25T10:31:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.710+0000 7fa17a400640 1 -- start start 2024-04-25T10:31:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa17a400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1740d0040 0x7fa17415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa17a400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa174071050 con 0x7fa1741550b0 2024-04-25T10:31:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa17a400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17415d040 con 0x7fa17415b740 2024-04-25T10:31:49.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa17a400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1740712f0 con 0x7fa1740d0040 2024-04-25T10:31:49.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa173400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1741550b0 0x7fa1740cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47458/0 (socket says 172.21.15.45:47458) 2024-04-25T10:31:49.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa173400640 1 -- 172.21.15.45:0/2580965358 learned_addr learned my addr 172.21.15.45:0/2580965358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 --2- 172.21.15.45:0/2580965358 >> v2:172.21.15.117:3300/0 conn(0x7fa1740d0040 0x7fa17415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245277755 0 0) 0x7fa174071050 con 0x7fa1741550b0 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa158003530 con 0x7fa1741550b0 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3076081202 0 0) 0x7fa158003530 con 0x7fa1741550b0 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 >> v2:172.21.15.117:3300/0 conn(0x7fa1740d0040 msgr2=0x7fa17415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 --2- 172.21.15.45:0/2580965358 >> v2:172.21.15.117:3300/0 conn(0x7fa1740d0040 0x7fa17415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 >> v1:172.21.15.117:6789/0 conn(0x7fa17415b740 legacy=0x7fa1740b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740715d0 con 0x7fa1741550b0 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fa164001a00 con 0x7fa1741550b0 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 >> v1:172.21.15.45:6789/0 conn(0x7fa1741550b0 legacy=0x7fa1740cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1580039c0 0x7fa158003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa158004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1580051d0 0x7fa15800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740712f0 con 0x7fa158004810 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15800e280 con 0x7fa1580051d0 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15800e480 con 0x7fa1580039c0 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa158004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa173400640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1580039c0 0x7fa158003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa173e00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1580051d0 0x7fa15800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 -- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1580039c0 msgr2=0x7fa158003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1580039c0 0x7fa158003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 -- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1580051d0 msgr2=0x7fa15800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:49.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1580051d0 0x7fa15800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa16aa00640 1 -- 172.21.15.45:0/2580965358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158003530 con 0x7fa158004810 2024-04-25T10:31:49.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.714+0000 7fa173e00640 1 -- 172.21.15.45:0/2580965358 reap_dead start 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa16aa00640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa158004c90 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009ae0 tx=0x7fa15c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e5149ad7c39ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0b2070 con 0x7fa158004810 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c0a5550 con 0x7fa158004810 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa172a00640 1 -- 172.21.15.45:0/2580965358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0096b0 con 0x7fa158004810 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 -- 172.21.15.45:0/2580965358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 msgr2=0x7fa158004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa158004c90 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009ae0 tx=0x7fa15c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 -- 172.21.15.45:0/2580965358 shutdown_connections 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 --2- 172.21.15.45:0/2580965358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa158004c90 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 -- 172.21.15.45:0/2580965358 >> 172.21.15.45:0/2580965358 conn(0x7fa174066b70 msgr2=0x7fa174066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 -- 172.21.15.45:0/2580965358 shutdown_connections 2024-04-25T10:31:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.718+0000 7fa17a400640 1 -- 172.21.15.45:0/2580965358 wait complete. 2024-04-25T10:31:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 Processor -- start 2024-04-25T10:31:49.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- start start 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa1740a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1740b16b0 0x7fa174142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1740b1b30 0x7fa174143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740720b0 con 0x7fa158004810 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa174071b90 con 0x7fa1740b16b0 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17415bb90 con 0x7fa1740b1b30 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa1740a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa1740a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42078/0 (socket says 172.21.15.45:42078) 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 -- 172.21.15.45:0/3423412642 learned_addr learned my addr 172.21.15.45:0/3423412642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa16aa00640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1740b16b0 0x7fa174142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173e00640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1740b1b30 0x7fa174143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 -- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1740b1b30 msgr2=0x7fa174143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1740b1b30 0x7fa174143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 -- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1740b16b0 msgr2=0x7fa174142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1740b16b0 0x7fa174142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 -- 172.21.15.45:0/3423412642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174159bf0 con 0x7fa158004810 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173e00640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1740b1b30 0x7fa174143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa173400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa1740a94e0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fa164011840 tx=0x7fa164011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164010040 con 0x7fa158004810 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16400fba0 con 0x7fa158004810 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa174159ed0 con 0x7fa158004810 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164015410 con 0x7fa158004810 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17415a490 con 0x7fa158004810 2024-04-25T10:31:49.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.722+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17404ef30 con 0x7fa158004810 2024-04-25T10:31:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.726+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1640155b0 con 0x7fa158004810 2024-04-25T10:31:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.726+0000 7fa16be00640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa150066f90 0x7fa150069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.726+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa1640880e0 con 0x7fa158004810 2024-04-25T10:31:49.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.726+0000 7fa16aa00640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa150066f90 0x7fa150069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:49.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.726+0000 7fa16aa00640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa150066f90 0x7fa150069450 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0af000 tx=0x7fa15c0046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:49.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.730+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa164059350 con 0x7fa158004810 2024-04-25T10:31:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.926+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1740b2600 con 0x7fa158004810 2024-04-25T10:31:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.926+0000 7fa16be00640 1 -- 172.21.15.45:0/3423412642 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+3714 (secure 0 0 0) 0x7fa164058c50 con 0x7fa158004810 2024-04-25T10:31:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:49.930 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":133,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10153,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/2377632969","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2377632969},{"type":"v1","addr":"172.21.15.117:6836","nonce":2377632969}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10156,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/4225485169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":4225485169},{"type":"v1","addr":"172.21.15.117:6837","nonce":4225485169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":126},{"gid":10869,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2351485868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2351485868},{"type":"v1","addr":"172.21.15.45:6834","nonce":2351485868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3235208729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3235208729},{"type":"v1","addr":"172.21.15.45:6835","nonce":3235208729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133}],"filesystems":[]} 2024-04-25T10:31:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 133 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa150066f90 msgr2=0x7fa150069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa150066f90 0x7fa150069450 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0af000 tx=0x7fa15c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 msgr2=0x7fa1740a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa1740a94e0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fa164011840 tx=0x7fa164011d00 comp rx=0 tx=0).stop 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 shutdown_connections 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa150066f90 0x7fa150069450 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1740b1b30 0x7fa174143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1740b16b0 0x7fa174142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 --2- 172.21.15.45:0/3423412642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa158004810 0x7fa1740a94e0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 >> 172.21.15.45:0/3423412642 conn(0x7fa174066b70 msgr2=0x7fa1740b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 shutdown_connections 2024-04-25T10:31:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:49.930+0000 7fa17a400640 1 -- 172.21.15.45:0/3423412642 wait complete. 2024-04-25T10:31:49.951 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:31:49.951 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:31:49.951 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:31:49.951 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:49.952 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:31:49.952 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:31:49.952 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:31:49.952 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:49.952 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:31:49.952 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:31:49.953 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:31:49.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:49.953 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:31:49.953 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:31:49.953 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:31:49.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:31:49.965 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:31:49.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:31:49.966 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:31:49.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:31:49.969 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:31:49.969 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:31:49.970 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:31:49.970 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:31:50.009 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:31:50.009 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:50.010 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:31:50.011 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:50.016 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:31:50.016 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:50.018 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:31:50.018 DEBUG:teuthology.parallel:result is None 2024-04-25T10:31:50.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:31:50.097 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-04-25T10:31:50.097 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:31:50.110 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2024-04-25T10:31:50.110 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2024-04-25T10:31:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.162+0000 7fa34b200640 1 Processor -- start 2024-04-25T10:31:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.162+0000 7fa34b200640 1 -- start start 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa34b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa34415bdd0 0x7fa3440d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa34b200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa3440d0600 0x7fa3440b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa34b200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3440b1310 con 0x7fa3440cf6f0 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa34b200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3440ae5a0 con 0x7fa3440d0600 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa34b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa34415c1b0 con 0x7fa34415bdd0 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa33b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa3440cf6f0 0x7fa3440cfef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47468/0 (socket says 172.21.15.45:47468) 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa33b400640 1 -- 172.21.15.45:0/2881715060 learned_addr learned my addr 172.21.15.45:0/2881715060 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa343e00640 1 --2- 172.21.15.45:0/2881715060 >> v2:172.21.15.117:3300/0 conn(0x7fa34415bdd0 0x7fa3440d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa349200640 1 --2- 172.21.15.45:0/2881715060 >> v2:172.21.15.45:3300/0 conn(0x7fa3440d0600 0x7fa3440b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa343e00640 1 -- 172.21.15.45:0/2881715060 >> v2:172.21.15.45:3300/0 conn(0x7fa3440d0600 msgr2=0x7fa3440b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa343e00640 1 --2- 172.21.15.45:0/2881715060 >> v2:172.21.15.45:3300/0 conn(0x7fa3440d0600 0x7fa3440b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa343400640 1 -- 172.21.15.45:0/2881715060 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735092692 0 0) 0x7fa3440b1310 con 0x7fa3440cf6f0 2024-04-25T10:31:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa343e00640 1 -- 172.21.15.45:0/2881715060 >> v1:172.21.15.45:6789/0 conn(0x7fa3440cf6f0 legacy=0x7fa3440cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.166+0000 7fa343e00640 1 -- 172.21.15.45:0/2881715060 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440b15f0 con 0x7fa34415bdd0 2024-04-25T10:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa343e00640 1 --2- 172.21.15.45:0/2881715060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440d2430 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa344067740 tx=0x7fa33401c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f8a4a7b425e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa343400640 1 -- 172.21.15.45:0/2881715060 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340b2070 con 0x7fa34415bdd0 2024-04-25T10:31:50.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa343400640 1 -- 172.21.15.45:0/2881715060 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3340a5870 con 0x7fa34415bdd0 2024-04-25T10:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa343400640 1 -- 172.21.15.45:0/2881715060 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340a5c40 con 0x7fa34415bdd0 2024-04-25T10:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 -- 172.21.15.45:0/2881715060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 msgr2=0x7fa3440d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 --2- 172.21.15.45:0/2881715060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440d2430 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa344067740 tx=0x7fa33401c930 comp rx=0 tx=0).stop 2024-04-25T10:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 -- 172.21.15.45:0/2881715060 shutdown_connections 2024-04-25T10:31:50.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 --2- 172.21.15.45:0/2881715060 >> v2:172.21.15.45:3300/0 conn(0x7fa3440d0600 0x7fa3440b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 --2- 172.21.15.45:0/2881715060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440d2430 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 -- 172.21.15.45:0/2881715060 >> 172.21.15.45:0/2881715060 conn(0x7fa3440d1910 msgr2=0x7fa3440d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 -- 172.21.15.45:0/2881715060 shutdown_connections 2024-04-25T10:31:50.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 -- 172.21.15.45:0/2881715060 wait complete. 2024-04-25T10:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 Processor -- start 2024-04-25T10:31:50.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 -- start start 2024-04-25T10:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3440cf6f0 0x7fa3440b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.170+0000 7fa34b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 0x7fa3440c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440b2fa0 con 0x7fa3440d0600 2024-04-25T10:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440b2e20 con 0x7fa34415bdd0 2024-04-25T10:31:50.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3440b3120 con 0x7fa3440cf6f0 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 0x7fa3440c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 0x7fa3440c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42142/0 (socket says 172.21.15.45:42142) 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 -- 172.21.15.45:0/2354377066 learned_addr learned my addr 172.21.15.45:0/2354377066 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa343e00640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3440cf6f0 0x7fa3440b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa349200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 -- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3440cf6f0 msgr2=0x7fa3440b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3440cf6f0 0x7fa3440b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 -- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 msgr2=0x7fa3440ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 -- 172.21.15.45:0/2354377066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344155a20 con 0x7fa3440d0600 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa33b400640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 0x7fa3440c89e0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00c4b0 tx=0x7fa32c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c018070 con 0x7fa3440d0600 2024-04-25T10:31:50.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c007500 con 0x7fa3440d0600 2024-04-25T10:31:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa344155d00 con 0x7fa3440d0600 2024-04-25T10:31:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c004a30 con 0x7fa3440d0600 2024-04-25T10:31:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3441562c0 con 0x7fa3440d0600 2024-04-25T10:31:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34404ef30 con 0x7fa3440d0600 2024-04-25T10:31:50.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.174+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa32c004bd0 con 0x7fa3440d0600 2024-04-25T10:31:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.178+0000 7fa340c00640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa320066f60 0x7fa320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.178+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa32c083f70 con 0x7fa3440d0600 2024-04-25T10:31:50.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.178+0000 7fa343e00640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa320066f60 0x7fa320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.178+0000 7fa343e00640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa320066f60 0x7fa320069420 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa3340af000 tx=0x7fa3340a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:50.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.182+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa32c054ba0 con 0x7fa3440d0600 2024-04-25T10:31:50.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.354+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa344154610 con 0x7fa3440d0600 2024-04-25T10:31:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.990+0000 7fa340c00640 1 -- 172.21.15.45:0/2354377066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v187) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa32c058c40 con 0x7fa3440d0600 2024-04-25T10:31:50.994 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa320066f60 msgr2=0x7fa320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa320066f60 0x7fa320069420 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa3340af000 tx=0x7fa3340a7070 comp rx=0 tx=0).stop 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 msgr2=0x7fa3440c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 0x7fa3440c89e0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00c4b0 tx=0x7fa32c00c970 comp rx=0 tx=0).stop 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 shutdown_connections 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa320066f60 0x7fa320069420 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa34415bdd0 0x7fa3440ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3440d0600 0x7fa3440c89e0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 --2- 172.21.15.45:0/2354377066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3440cf6f0 0x7fa3440b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 >> 172.21.15.45:0/2354377066 conn(0x7fa3440d1910 msgr2=0x7fa34415b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 shutdown_connections 2024-04-25T10:31:50.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:50.994+0000 7fa34b200640 1 -- 172.21.15.45:0/2354377066 wait complete. 2024-04-25T10:31:51.023 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:31:51.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b89800640 1 Processor -- start 2024-04-25T10:31:51.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b89800640 1 -- start start 2024-04-25T10:31:51.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b89800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b840b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b89800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b84070ed0 con 0x7f5b8415a0e0 2024-04-25T10:31:51.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b89800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b840711b0 con 0x7f5b840b2fc0 2024-04-25T10:31:51.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b89800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84071490 con 0x7f5b84071f70 2024-04-25T10:31:51.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b82a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5b8415a0e0 0x7f5b8415b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47478/0 (socket says 172.21.15.45:47478) 2024-04-25T10:31:51.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b82a00640 1 -- 172.21.15.45:0/3165942330 learned_addr learned my addr 172.21.15.45:0/3165942330 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:51.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.166+0000 7f5b7a000640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b840b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543153239 0 0) 0x7f5b84070ed0 con 0x7f5b8415a0e0 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b68003530 con 0x7f5b8415a0e0 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493400495 0 0) 0x7f5b68003530 con 0x7f5b8415a0e0 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 msgr2=0x7f5b840b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b840b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 >> v1:172.21.15.117:6789/0 conn(0x7f5b840b2fc0 legacy=0x7f5b8406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b84071770 con 0x7f5b8415a0e0 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f5b74001a00 con 0x7f5b8415a0e0 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 >> v1:172.21.15.45:6789/0 conn(0x7f5b8415a0e0 legacy=0x7f5b8415b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b680039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68003f70 0x7f5b68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84071490 con 0x7f5b68004d50 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800dcc0 con 0x7f5b68003f70 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800dec0 con 0x7f5b84071f70 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82a00640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b680039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b83400640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b7a000640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68003f70 0x7f5b68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b83400640 1 -- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 msgr2=0x7f5b680039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b83400640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b680039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b83400640 1 -- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68003f70 msgr2=0x7f5b68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b83400640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b68003f70 0x7f5b68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b83400640 1 -- 172.21.15.45:0/3165942330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68003530 con 0x7f5b68004d50 2024-04-25T10:31:51.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.170+0000 7f5b82a00640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b84071f70 0x7f5b680039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:31:51.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b83400640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c001780 tx=0x7f5b7c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41b16684c4e6795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0013a0 con 0x7f5b68004d50 2024-04-25T10:31:51.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0a9070 con 0x7f5b68004d50 2024-04-25T10:31:51.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b82000640 1 -- 172.21.15.45:0/3165942330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c003600 con 0x7f5b68004d50 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- 172.21.15.45:0/3165942330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 msgr2=0x7f5b680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 --2- 172.21.15.45:0/3165942330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c001780 tx=0x7f5b7c01c420 comp rx=0 tx=0).stop 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b83400640 1 -- 172.21.15.45:0/3165942330 reap_dead start 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- 172.21.15.45:0/3165942330 shutdown_connections 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- 172.21.15.45:0/3165942330 >> 172.21.15.45:0/3165942330 conn(0x7f5b84066b70 msgr2=0x7f5b84066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- 172.21.15.45:0/3165942330 shutdown_connections 2024-04-25T10:31:51.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- 172.21.15.45:0/3165942330 wait complete. 2024-04-25T10:31:51.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 Processor -- start 2024-04-25T10:31:51.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- start start 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b8414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8415a640 0x7f5b8415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b84156190 0x7f5b84156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8415a4c0 con 0x7f5b68004d50 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840af030 con 0x7f5b84156190 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b89800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84070b50 con 0x7f5b8415a640 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b82a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b8414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b82a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b8414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42158/0 (socket says 172.21.15.45:42158) 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.174+0000 7f5b82a00640 1 -- 172.21.15.45:0/3815890060 learned_addr learned my addr 172.21.15.45:0/3815890060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7a000640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8415a640 0x7f5b8415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b83400640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b84156190 0x7f5b84156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b82a00640 1 -- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8415a640 msgr2=0x7f5b8415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b82a00640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8415a640 0x7f5b8415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b82a00640 1 -- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b84156190 msgr2=0x7f5b84156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b82a00640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b84156190 0x7f5b84156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b82a00640 1 -- 172.21.15.45:0/3815890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8414e460 con 0x7f5b68004d50 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b82a00640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b8414df20 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400afc0 tx=0x7f5b74007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74017020 con 0x7f5b68004d50 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b740019c0 con 0x7f5b68004d50 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74013800 con 0x7f5b68004d50 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8415b100 con 0x7f5b68004d50 2024-04-25T10:31:51.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b84154300 con 0x7f5b68004d50 2024-04-25T10:31:51.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8404ef30 con 0x7f5b68004d50 2024-04-25T10:31:51.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b7401a020 con 0x7f5b68004d50 2024-04-25T10:31:51.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7b400640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b60066f60 0x7f5b60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.178+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b740859c0 con 0x7f5b68004d50 2024-04-25T10:31:51.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.182+0000 7f5b7a000640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b60066f60 0x7f5b60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.182+0000 7f5b7a000640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b60066f60 0x7f5b60069420 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5b8415bac0 tx=0x7f5b6c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.182+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b74021020 con 0x7f5b68004d50 2024-04-25T10:31:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.406+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5b840703e0 con 0x7f5b68004d50 2024-04-25T10:31:51.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.406+0000 7f5b7b400640 1 -- 172.21.15.45:0/3815890060 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5b74056630 con 0x7f5b68004d50 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmu8v1NhAASdFiVGOq/flaZhKMziYTxg==", 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.411 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmoUP/BxAAJFuc3TYCyNmr5n2yERERRA==", 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.412 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmgAbdAhAA5bycmp+ueSw8MsNvQYm7wg==", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm4q9ADxAATs3SnGkaRLT7isEyNAHN8w==", 2024-04-25T10:31:51.413 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCbLSpmZoreMBAATUSVFvmEBEREDNm5mqPh3w==", 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCfLSpmk9xeABAApGfd1IpmY9i/Ek88JSz8WA==", 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.415 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCiLSpmOy4vCxAAa/ysfy+vg72AdIzcf2MSug==", 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQClLSpmBaq3FRAATqUDpHv7amL/C3NYbZMSuQ==", 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCrLSpmKfLIDRAA2QzG9IxzfStdMIwbQjHslg==", 2024-04-25T10:31:51.417 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCuLSpmQQbDGBAAAJI+HcKrCbhQwQolcvNovw==", 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCxLSpmsRE3IRAAFa17d03tFLN8xnUjmJ5Nxg==", 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-04-25T10:31:51.419 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQC0LSpmrQmuLRAAwJ5NX7h2OFBJ7MGgiGd4iQ==", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ==", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-04-25T10:31:51.420 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmO1nmHBAAuNVNKLUXGFAhHW6A1TBTwA==", 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmCEs+GRAASl8OEBs955jrZbI3C/N6Qg==", 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.422 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCaLSpmiTu6IhAAHkvz+I/E/n9Xkf+btzMDlw==", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm/hwLGxAAyqHUddGirr8KeUJSnvf2RA==", 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.423 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpm38esMhAAi9iw4/P4GEgjTosVB+YShA==", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQCZLSpmnZbFLhAA9NERCBaH9D69yxuoPz32AQ==", 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T10:31:51.425 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b60066f60 msgr2=0x7f5b60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b60066f60 0x7f5b60069420 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5b8415bac0 tx=0x7f5b6c009190 comp rx=0 tx=0).stop 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 msgr2=0x7f5b8414df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b8414df20 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400afc0 tx=0x7f5b74007430 comp rx=0 tx=0).stop 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 shutdown_connections 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5b60066f60 0x7f5b60069420 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b84156190 0x7f5b84156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b8415a640 0x7f5b8415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 --2- 172.21.15.45:0/3815890060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68004d50 0x7f5b8414df20 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 >> 172.21.15.45:0/3815890060 conn(0x7f5b84066b70 msgr2=0x7f5b8406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 shutdown_connections 2024-04-25T10:31:51.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.410+0000 7f5b89800640 1 -- 172.21.15.45:0/3815890060 wait complete. 2024-04-25T10:31:51.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:51.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 Processor -- start 2024-04-25T10:31:51.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 -- start start 2024-04-25T10:31:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a9c0cf930 0x7f6a9c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c0b2770 con 0x7f6a9c0cda10 2024-04-25T10:31:51.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c06c670 con 0x7f6a9c0cf930 2024-04-25T10:31:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6aa2000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c06c950 con 0x7f6a9c15bc40 2024-04-25T10:31:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6a93e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6a9c0cda10 0x7f6a9c0b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47492/0 (socket says 172.21.15.45:47492) 2024-04-25T10:31:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6a93e00640 1 -- 172.21.15.45:0/2847624420 learned_addr learned my addr 172.21.15.45:0/2847624420 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.570+0000 7f6a9be00640 1 --2- 172.21.15.45:0/2847624420 >> v2:172.21.15.45:3300/0 conn(0x7f6a9c0cf930 0x7f6a9c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9b400640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9be00640 1 -- 172.21.15.45:0/2847624420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 msgr2=0x7f6a9c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9be00640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9be00640 1 -- 172.21.15.45:0/2847624420 >> v1:172.21.15.45:6789/0 conn(0x7f6a9c0cda10 legacy=0x7f6a9c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9aa00640 1 -- 172.21.15.45:0/2847624420 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391067931 0 0) 0x7f6a9c0b2770 con 0x7f6a9c0cda10 2024-04-25T10:31:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9be00640 1 -- 172.21.15.45:0/2847624420 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c06cc30 con 0x7f6a9c0cf930 2024-04-25T10:31:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9b400640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:51.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9be00640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c0cfde0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009d90 tx=0x7f6a9401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c02c1f1220c19b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9aa00640 1 -- 172.21.15.45:0/2847624420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a940b2070 con 0x7f6a9c0cf930 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9aa00640 1 -- 172.21.15.45:0/2847624420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a940a57a0 con 0x7f6a9c0cf930 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6a9aa00640 1 -- 172.21.15.45:0/2847624420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a940a5b70 con 0x7f6a9c0cf930 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6aa2000640 1 -- 172.21.15.45:0/2847624420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 msgr2=0x7f6a9c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c0cfde0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009d90 tx=0x7f6a9401c8e0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6aa2000640 1 -- 172.21.15.45:0/2847624420 shutdown_connections 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c0cfde0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2847624420 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.574+0000 7f6aa2000640 1 -- 172.21.15.45:0/2847624420 >> 172.21.15.45:0/2847624420 conn(0x7f6a9c066b70 msgr2=0x7f6a9c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- 172.21.15.45:0/2847624420 shutdown_connections 2024-04-25T10:31:51.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- 172.21.15.45:0/2847624420 wait complete. 2024-04-25T10:31:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 Processor -- start 2024-04-25T10:31:51.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- start start 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a9c0cda10 0x7f6a9c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c14a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0b3850 con 0x7f6a9c0cf930 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0b36d0 con 0x7f6a9c0cda10 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0b39d0 con 0x7f6a9c15bc40 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c14a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c14a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42182/0 (socket says 172.21.15.45:42182) 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 -- 172.21.15.45:0/2261055641 learned_addr learned my addr 172.21.15.45:0/2261055641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a9b400640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a9c0cda10 0x7f6a9c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a9be00640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 -- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 msgr2=0x7f6a9c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 -- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a9c0cda10 msgr2=0x7f6a9c150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a9c0cda10 0x7f6a9c150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 -- 172.21.15.45:0/2261055641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c14ef60 con 0x7f6a9c0cf930 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93e00640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c14a2e0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400c480 tx=0x7f6a8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84018070 con 0x7f6a9c0cf930 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a84007500 con 0x7f6a9c0cf930 2024-04-25T10:31:51.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c14f240 con 0x7f6a9c0cf930 2024-04-25T10:31:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.578+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84004a30 con 0x7f6a9c0cf930 2024-04-25T10:31:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c14f800 con 0x7f6a9c0cf930 2024-04-25T10:31:51.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c04ef30 con 0x7f6a9c0cf930 2024-04-25T10:31:51.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a8401b070 con 0x7f6a9c0cf930 2024-04-25T10:31:51.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6a93400640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a78066f90 0x7f6a78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6a84084110 con 0x7f6a9c0cf930 2024-04-25T10:31:51.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6a9b400640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a78066f90 0x7f6a78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.582+0000 7f6a9b400640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a78066f90 0x7f6a78069450 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006fd0 tx=0x7f6a8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.586+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a840553d0 con 0x7f6a9c0cf930 2024-04-25T10:31:51.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.750+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a9c0af1c0 con 0x7f6a9c0cf930 2024-04-25T10:31:51.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.750+0000 7f6a93400640 1 -- 172.21.15.45:0/2261055641 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f6a84054cd0 con 0x7f6a9c0cf930 2024-04-25T10:31:51.753 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:51.753 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":189,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:51.003820+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a78066f90 msgr2=0x7f6a78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a78066f90 0x7f6a78069450 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006fd0 tx=0x7f6a8c008040 comp rx=0 tx=0).stop 2024-04-25T10:31:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 msgr2=0x7f6a9c14a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c14a2e0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400c480 tx=0x7f6a8400c940 comp rx=0 tx=0).stop 2024-04-25T10:31:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 shutdown_connections 2024-04-25T10:31:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6a78066f90 0x7f6a78069450 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a9c15bc40 0x7f6a9c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a9c0cf930 0x7f6a9c14a2e0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 --2- 172.21.15.45:0/2261055641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a9c0cda10 0x7f6a9c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 >> 172.21.15.45:0/2261055641 conn(0x7f6a9c066b70 msgr2=0x7f6a9c066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 shutdown_connections 2024-04-25T10:31:51.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.754+0000 7f6aa2000640 1 -- 172.21.15.45:0/2261055641 wait complete. 2024-04-25T10:31:51.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:31:51.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:31:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.914+0000 7f5873200640 1 Processor -- start 2024-04-25T10:31:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.914+0000 7f5873200640 1 -- start start 2024-04-25T10:31:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5873200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5873200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f586c070350 0x7f586c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5873200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586c15b080 con 0x7f586c15bac0 2024-04-25T10:31:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5873200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586c15adb0 con 0x7f586c070350 2024-04-25T10:31:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5873200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c15eb60 con 0x7f586c0aef80 2024-04-25T10:31:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5871200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f586c15bac0 0x7f586c0b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47506/0 (socket says 172.21.15.45:47506) 2024-04-25T10:31:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5871200640 1 -- 172.21.15.45:0/536074328 learned_addr learned my addr 172.21.15.45:0/536074328 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5863e00640 1 --2- 172.21.15.45:0/536074328 >> v2:172.21.15.117:3300/0 conn(0x7f586c070350 0x7f586c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f586be00640 1 --2- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f586b400640 1 -- 172.21.15.45:0/536074328 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698806066 0 0) 0x7f586c15b080 con 0x7f586c15bac0 2024-04-25T10:31:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f586b400640 1 -- 172.21.15.45:0/536074328 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5850003530 con 0x7f586c15bac0 2024-04-25T10:31:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5863e00640 1 -- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 msgr2=0x7f586c15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5863e00640 1 --2- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5863e00640 1 -- 172.21.15.45:0/536074328 >> v1:172.21.15.45:6789/0 conn(0x7f586c15bac0 legacy=0x7f586c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.918+0000 7f5863e00640 1 -- 172.21.15.45:0/536074328 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c15ee40 con 0x7f586c070350 2024-04-25T10:31:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5863e00640 1 --2- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c070350 0x7f586c15b580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f58540097c0 tx=0x7f585401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dbdf1ecebb4e7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f586b400640 1 -- 172.21.15.45:0/536074328 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58540ab680 con 0x7f586c070350 2024-04-25T10:31:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f586b400640 1 -- 172.21.15.45:0/536074328 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58540b0070 con 0x7f586c070350 2024-04-25T10:31:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f586b400640 1 -- 172.21.15.45:0/536074328 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58540aba40 con 0x7f586c070350 2024-04-25T10:31:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 -- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c070350 msgr2=0x7f586c15b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 --2- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c070350 0x7f586c15b580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f58540097c0 tx=0x7f585401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 -- 172.21.15.45:0/536074328 shutdown_connections 2024-04-25T10:31:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 --2- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c070350 0x7f586c15b580 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 --2- 172.21.15.45:0/536074328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 -- 172.21.15.45:0/536074328 >> 172.21.15.45:0/536074328 conn(0x7f586c0669a0 msgr2=0x7f586c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 -- 172.21.15.45:0/536074328 shutdown_connections 2024-04-25T10:31:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 -- 172.21.15.45:0/536074328 wait complete. 2024-04-25T10:31:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 Processor -- start 2024-04-25T10:31:51.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.922+0000 7f5873200640 1 -- start start 2024-04-25T10:31:51.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 0x7f586c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c1f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c15bac0 0x7f586c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c167820 con 0x7f586c070350 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0b3ae0 con 0x7f586c15bac0 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1679a0 con 0x7f586c0aef80 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 0x7f586c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 0x7f586c155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42194/0 (socket says 172.21.15.45:42194) 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 -- 172.21.15.45:0/1611582140 learned_addr learned my addr 172.21.15.45:0/1611582140 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 -- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 msgr2=0x7f586c1f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5863e00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c1f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5871200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c15bac0 0x7f586c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c1f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 -- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c15bac0 msgr2=0x7f586c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c15bac0 0x7f586c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 -- 172.21.15.45:0/1611582140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c155940 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5863e00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5871200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c15bac0 0x7f586c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f586be00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 0x7f586c155400 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e930 tx=0x7f585c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c017070 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c00ca00 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00cdb0 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c1f6b50 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c1f7110 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.926+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c04ef30 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.930+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f585c01a070 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.930+0000 7f5868c00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5848066f60 0x7f5848069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.930+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f585c084d20 con 0x7f586c070350 2024-04-25T10:31:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.930+0000 7f5863e00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5848066f60 0x7f5848069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.930+0000 7f5863e00640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5848066f60 0x7f5848069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f58540af000 tx=0x7f5854008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:51.934+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f585c056130 con 0x7f586c070350 2024-04-25T10:31:52.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.094+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f586c0b2fa0 con 0x7f586c070350 2024-04-25T10:31:52.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5868c00640 1 -- 172.21.15.45:0/1611582140 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f585c055a30 con 0x7f586c070350 2024-04-25T10:31:52.335 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:31:52.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5848066f60 msgr2=0x7f5848069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5848066f60 0x7f5848069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f58540af000 tx=0x7f5854008040 comp rx=0 tx=0).stop 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 msgr2=0x7f586c155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 0x7f586c155400 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e930 tx=0x7f585c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 shutdown_connections 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5848066f60 0x7f5848069420 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f586c15bac0 0x7f586c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f586c0aef80 0x7f586c1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 --2- 172.21.15.45:0/1611582140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f586c070350 0x7f586c155400 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 >> 172.21.15.45:0/1611582140 conn(0x7f586c0669a0 msgr2=0x7f586c0b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 shutdown_connections 2024-04-25T10:31:52.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.098+0000 7f5873200640 1 -- 172.21.15.45:0/1611582140 wait complete. 2024-04-25T10:31:52.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:31:52.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 Processor -- start 2024-04-25T10:31:52.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 -- start start 2024-04-25T10:31:52.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f4162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f40669a0 0x7f19f4066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40b0210 con 0x7f19f40b57e0 2024-04-25T10:31:52.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40aff30 con 0x7f19f40669a0 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19fb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4161840 con 0x7f19f415dab0 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f40669a0 0x7f19f4066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f40669a0 0x7f19f4066e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42210/0 (socket says 172.21.15.45:42210) 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 -- 172.21.15.45:0/642250926 learned_addr learned my addr 172.21.15.45:0/642250926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19ebe00640 1 --1- 172.21.15.45:0/642250926 >> v1:172.21.15.45:6789/0 conn(0x7f19f40b57e0 0x7f19f40af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47518/0 (socket says 172.21.15.45:47518) 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f8c00640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f4162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 -- 172.21.15.45:0/642250926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 msgr2=0x7f19f4162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f4162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 -- 172.21.15.45:0/642250926 >> v1:172.21.15.45:6789/0 conn(0x7f19f40b57e0 legacy=0x7f19f40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f9600640 1 -- 172.21.15.45:0/642250926 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4161b20 con 0x7f19f40669a0 2024-04-25T10:31:52.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.482+0000 7f19f8c00640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f4162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:52.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19f9600640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f4066e30 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009e50 tx=0x7f19ec01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898dfcc36b6573a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19f3e00640 1 -- 172.21.15.45:0/642250926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0a5700 con 0x7f19f40669a0 2024-04-25T10:31:52.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19f3e00640 1 -- 172.21.15.45:0/642250926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec0a58a0 con 0x7f19f40669a0 2024-04-25T10:31:52.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19f3e00640 1 -- 172.21.15.45:0/642250926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0a5c70 con 0x7f19f40669a0 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 -- 172.21.15.45:0/642250926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 msgr2=0x7f19f4066e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f4066e30 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009e50 tx=0x7f19ec01ca30 comp rx=0 tx=0).stop 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 -- 172.21.15.45:0/642250926 shutdown_connections 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f4066e30 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 --2- 172.21.15.45:0/642250926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f4162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 -- 172.21.15.45:0/642250926 >> 172.21.15.45:0/642250926 conn(0x7f19f406ac90 msgr2=0x7f19f406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 -- 172.21.15.45:0/642250926 shutdown_connections 2024-04-25T10:31:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.486+0000 7f19fb600640 1 -- 172.21.15.45:0/642250926 wait complete. 2024-04-25T10:31:52.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 Processor -- start 2024-04-25T10:31:52.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- start start 2024-04-25T10:31:52.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f40b57e0 0x7f19f4146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40d10e0 con 0x7f19f40669a0 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40d0f60 con 0x7f19f40b57e0 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40d1260 con 0x7f19f415dab0 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42218/0 (socket says 172.21.15.45:42218) 2024-04-25T10:31:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 -- 172.21.15.45:0/2393763787 learned_addr learned my addr 172.21.15.45:0/2393763787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f9600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19ebe00640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f40b57e0 0x7f19f4146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 -- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 msgr2=0x7f19f414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 -- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f40b57e0 msgr2=0x7f19f4146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f40b57e0 0x7f19f4146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 -- 172.21.15.45:0/2393763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f414b930 con 0x7f19f40669a0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f8c00640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f414fc90 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f19e400c820 tx=0x7f19e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4015070 con 0x7f19f40669a0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4007a30 con 0x7f19f40669a0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4007e40 con 0x7f19f40669a0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f414bc10 con 0x7f19f40669a0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f414c1d0 con 0x7f19f40669a0 2024-04-25T10:31:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.490+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f414c600 con 0x7f19f40669a0 2024-04-25T10:31:52.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.494+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19e4018070 con 0x7f19f40669a0 2024-04-25T10:31:52.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.494+0000 7f19f1600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19d0066f90 0x7f19d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.494+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19e4084c00 con 0x7f19f40669a0 2024-04-25T10:31:52.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.494+0000 7f19ebe00640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19d0066f90 0x7f19d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.494+0000 7f19ebe00640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19d0066f90 0x7f19d0069450 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f19f41fa0a0 tx=0x7f19dc007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.498+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19e408b080 con 0x7f19f40669a0 2024-04-25T10:31:52.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f19f404ef30 con 0x7f19f40669a0 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19f1600640 1 -- 172.21.15.45:0/2393763787 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v189) v1 ==== 84+0+65 (secure 0 0 0) 0x7f19e4055f60 con 0x7f19f40669a0 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:31:52.664 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19d0066f90 msgr2=0x7f19d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19d0066f90 0x7f19d0069450 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f19f41fa0a0 tx=0x7f19dc007cd0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 msgr2=0x7f19f414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f414fc90 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f19e400c820 tx=0x7f19e400cce0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 shutdown_connections 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f19d0066f90 0x7f19d0069450 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19f415dab0 0x7f19f414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19f40b57e0 0x7f19f4146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 --2- 172.21.15.45:0/2393763787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f40669a0 0x7f19f414fc90 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 >> 172.21.15.45:0/2393763787 conn(0x7f19f406ac90 msgr2=0x7f19f40af480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 shutdown_connections 2024-04-25T10:31:52.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.662+0000 7f19fb600640 1 -- 172.21.15.45:0/2393763787 wait complete. 2024-04-25T10:31:52.686 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:31:52.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:31:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 Processor -- start 2024-04-25T10:31:52.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 -- start start 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4d00cd6f0 0x7fc4d006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4d00d2270 0x7fc4d006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d006bea0 con 0x7fc4d006bac0 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d00cf8e0 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cfe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d00ccdb0 con 0x7fc4d00d2270 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4d00cd6f0 0x7fc4d006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4d00cd6f0 0x7fc4d006aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42220/0 (socket says 172.21.15.45:42220) 2024-04-25T10:31:52.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 -- 172.21.15.45:0/3292290042 learned_addr learned my addr 172.21.15.45:0/3292290042 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4c6000640 1 --2- 172.21.15.45:0/3292290042 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00d2270 0x7fc4d006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 -- 172.21.15.45:0/3292290042 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00d2270 msgr2=0x7fc4d006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 --2- 172.21.15.45:0/3292290042 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00d2270 0x7fc4d006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 -- 172.21.15.45:0/3292290042 >> v1:172.21.15.117:6789/0 conn(0x7fc4d006bac0 legacy=0x7fc4d015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4cea00640 1 -- 172.21.15.45:0/3292290042 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00cfb10 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.830+0000 7fc4c6000640 1 --2- 172.21.15.45:0/3292290042 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00d2270 0x7fc4d006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:52.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cea00640 1 --2- 172.21.15.45:0/3292290042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d006aea0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00718f0 tx=0x7fc4c001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6d2d60e9ca918f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4ce000640 1 -- 172.21.15.45:0/3292290042 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00a5680 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4ce000640 1 -- 172.21.15.45:0/3292290042 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c00a5820 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4ce000640 1 -- 172.21.15.45:0/3292290042 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00a5bf0 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 -- 172.21.15.45:0/3292290042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 msgr2=0x7fc4d006aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/3292290042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d006aea0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00718f0 tx=0x7fc4c001c8d0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 -- 172.21.15.45:0/3292290042 shutdown_connections 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/3292290042 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00d2270 0x7fc4d006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/3292290042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d006aea0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 -- 172.21.15.45:0/3292290042 >> 172.21.15.45:0/3292290042 conn(0x7fc4d00d1750 msgr2=0x7fc4d00d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 -- 172.21.15.45:0/3292290042 shutdown_connections 2024-04-25T10:31:52.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.834+0000 7fc4cfe00640 1 -- 172.21.15.45:0/3292290042 wait complete. 2024-04-25T10:31:52.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 Processor -- start 2024-04-25T10:31:52.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- start start 2024-04-25T10:31:52.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d006bac0 0x7fc4d00c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00d2270 0x7fc4d00b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00cdad0 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00d0690 con 0x7fc4d00d2270 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d00d0bb0 con 0x7fc4d006bac0 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d00b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42230/0 (socket says 172.21.15.45:42230) 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 -- 172.21.15.45:0/931361151 learned_addr learned my addr 172.21.15.45:0/931361151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cea00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d006bac0 0x7fc4d00c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cf400640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00d2270 0x7fc4d00b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 -- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d006bac0 msgr2=0x7fc4d00c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d006bac0 0x7fc4d00c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 -- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00d2270 msgr2=0x7fc4d00b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00d2270 0x7fc4d00b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 -- 172.21.15.45:0/931361151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00b12e0 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c6000640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d00b2620 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800c7c0 tx=0x7fc4b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8007a30 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8007bd0 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80113f0 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d00b1580 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d00b1b40 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.838+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d0075120 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.842+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4b8011590 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.842+0000 7fc4c7e00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4ac066f60 0x7fc4ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:52.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.842+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4b8084f20 con 0x7fc4d00cd6f0 2024-04-25T10:31:52.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.842+0000 7fc4cea00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4ac066f60 0x7fc4ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:52.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.842+0000 7fc4cea00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4ac066f60 0x7fc4ac069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae000 tx=0x7fc4c00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:52.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:52.846+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4b8015020 con 0x7fc4d00cd6f0 2024-04-25T10:31:53.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.006+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc4d0072f00 con 0x7fc4d00cd6f0 2024-04-25T10:31:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4c7e00640 1 -- 172.21.15.45:0/931361151 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v190) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4b8056280 con 0x7fc4d00cd6f0 2024-04-25T10:31:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4ac066f60 msgr2=0x7fc4ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4ac066f60 0x7fc4ac069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae000 tx=0x7fc4c00a7070 comp rx=0 tx=0).stop 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 msgr2=0x7fc4d00b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d00b2620 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800c7c0 tx=0x7fc4b800cc80 comp rx=0 tx=0).stop 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 shutdown_connections 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc4ac066f60 0x7fc4ac069420 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00d2270 0x7fc4d00b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4d00cd6f0 0x7fc4d00b2620 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 --2- 172.21.15.45:0/931361151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d006bac0 0x7fc4d00c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 >> 172.21.15.45:0/931361151 conn(0x7fc4d00d1750 msgr2=0x7fc4d00d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 shutdown_connections 2024-04-25T10:31:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:53.998+0000 7fc4cfe00640 1 -- 172.21.15.45:0/931361151 wait complete. 2024-04-25T10:31:54.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:31:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.178+0000 7f9cf3200640 1 Processor -- start 2024-04-25T10:31:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.178+0000 7f9cf3200640 1 -- start start 2024-04-25T10:31:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf3200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9cf40d1510 0x7f9cf40b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf3200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cf40d18f0 0x7f9cf406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf3200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40d0170 con 0x7f9cf40d18f0 2024-04-25T10:31:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf3200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40d03e0 con 0x7f9cf40d1510 2024-04-25T10:31:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40d06c0 con 0x7f9cf40cfbb0 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cf40d18f0 0x7f9cf406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9cf40d18f0 0x7f9cf406bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42232/0 (socket says 172.21.15.45:42232) 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 -- 172.21.15.45:0/1395714408 learned_addr learned my addr 172.21.15.45:0/1395714408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf1e00640 1 --2- 172.21.15.45:0/1395714408 >> v2:172.21.15.117:3300/0 conn(0x7f9cf40d1510 0x7f9cf40b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cebe00640 1 --2- 172.21.15.45:0/1395714408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 -- 172.21.15.45:0/1395714408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 msgr2=0x7f9cf40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 --2- 172.21.15.45:0/1395714408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 -- 172.21.15.45:0/1395714408 >> v2:172.21.15.117:3300/0 conn(0x7f9cf40d1510 msgr2=0x7f9cf40b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 --2- 172.21.15.45:0/1395714408 >> v2:172.21.15.117:3300/0 conn(0x7f9cf40d1510 0x7f9cf40b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.182+0000 7f9cf2800640 1 -- 172.21.15.45:0/1395714408 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf406b5f0 con 0x7f9cf40d18f0 2024-04-25T10:31:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf2800640 1 --2- 172.21.15.45:0/1395714408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d18f0 0x7f9cf406bb10 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4072fd0 tx=0x7f9cec01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c6268152fecea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf1400640 1 -- 172.21.15.45:0/1395714408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0a5860 con 0x7f9cf40d18f0 2024-04-25T10:31:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf1400640 1 -- 172.21.15.45:0/1395714408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0a5a00 con 0x7f9cf40d18f0 2024-04-25T10:31:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf1400640 1 -- 172.21.15.45:0/1395714408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0a5dd0 con 0x7f9cf40d18f0 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 -- 172.21.15.45:0/1395714408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d18f0 msgr2=0x7f9cf406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 --2- 172.21.15.45:0/1395714408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d18f0 0x7f9cf406bb10 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4072fd0 tx=0x7f9cec01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 -- 172.21.15.45:0/1395714408 shutdown_connections 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 --2- 172.21.15.45:0/1395714408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d18f0 0x7f9cf406bb10 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 --2- 172.21.15.45:0/1395714408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 --2- 172.21.15.45:0/1395714408 >> v2:172.21.15.117:3300/0 conn(0x7f9cf40d1510 0x7f9cf40b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 -- 172.21.15.45:0/1395714408 >> 172.21.15.45:0/1395714408 conn(0x7f9cf4072820 msgr2=0x7f9cf409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 -- 172.21.15.45:0/1395714408 shutdown_connections 2024-04-25T10:31:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 -- 172.21.15.45:0/1395714408 wait complete. 2024-04-25T10:31:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 Processor -- start 2024-04-25T10:31:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 -- start start 2024-04-25T10:31:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.186+0000 7f9cf3200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf4157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 0x7f9cf40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cf40d18f0 0x7f9cf40b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40b28c0 con 0x7f9cf40d1510 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40b2740 con 0x7f9cf40d18f0 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40d1d80 con 0x7f9cf40cfbb0 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 0x7f9cf40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 0x7f9cf40b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42238/0 (socket says 172.21.15.45:42238) 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf1e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf4157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 -- 172.21.15.45:0/940338893 learned_addr learned my addr 172.21.15.45:0/940338893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf2800640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cf40d18f0 0x7f9cf40b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 -- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 msgr2=0x7f9cf4157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf4157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 -- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cf40d18f0 msgr2=0x7f9cf40b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cf40d18f0 0x7f9cf40b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 -- 172.21.15.45:0/940338893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf4157590 con 0x7f9cf40d1510 2024-04-25T10:31:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cebe00640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 0x7f9cf40b5ae0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00d800 tx=0x7f9cdc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc018020 con 0x7f9cf40d1510 2024-04-25T10:31:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0096a0 con 0x7f9cf40d1510 2024-04-25T10:31:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf415dc40 con 0x7f9cf40d1510 2024-04-25T10:31:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc009a50 con 0x7f9cf40d1510 2024-04-25T10:31:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf415e200 con 0x7f9cf40d1510 2024-04-25T10:31:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf4075120 con 0x7f9cf40d1510 2024-04-25T10:31:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.190+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9cdc01b020 con 0x7f9cf40d1510 2024-04-25T10:31:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.194+0000 7f9cea000640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd0066f60 0x7f9cd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.194+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9cdc085770 con 0x7f9cf40d1510 2024-04-25T10:31:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.194+0000 7f9cf1e00640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd0066f60 0x7f9cd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.194+0000 7f9cf1e00640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd0066f60 0x7f9cd0069420 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007a60 tx=0x7f9ce4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.198+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cdc022020 con 0x7f9cf40d1510 2024-04-25T10:31:54.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:54.358+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9cf406ad90 con 0x7f9cf40d1510 2024-04-25T10:31:55.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.006+0000 7f9cea000640 1 -- 172.21.15.45:0/940338893 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v191) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9cdc0562f0 con 0x7f9cf40d1510 2024-04-25T10:31:55.009 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.006+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd0066f60 msgr2=0x7f9cd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.006+0000 7f9cf3200640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd0066f60 0x7f9cd0069420 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007a60 tx=0x7f9ce4008040 comp rx=0 tx=0).stop 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.006+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 msgr2=0x7f9cf40b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.006+0000 7f9cf3200640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 0x7f9cf40b5ae0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00d800 tx=0x7f9cdc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 shutdown_connections 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9cd0066f60 0x7f9cd0069420 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9cf40d18f0 0x7f9cf40b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cf40d1510 0x7f9cf40b5ae0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 --2- 172.21.15.45:0/940338893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9cf40cfbb0 0x7f9cf4157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 >> 172.21.15.45:0/940338893 conn(0x7f9cf4072820 msgr2=0x7f9cf409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 shutdown_connections 2024-04-25T10:31:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.010+0000 7f9cf3200640 1 -- 172.21.15.45:0/940338893 wait complete. 2024-04-25T10:31:55.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:31:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.194+0000 7fb1cf000640 1 Processor -- start 2024-04-25T10:31:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.194+0000 7fb1cf000640 1 -- start start 2024-04-25T10:31:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cf000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1c80721f0 0x7fb1c8166920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cf000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c8071360 con 0x7fb1c80aeda0 2024-04-25T10:31:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cf000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c8159920 con 0x7fb1c8159540 2024-04-25T10:31:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cf000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c815c2f0 con 0x7fb1c80721f0 2024-04-25T10:31:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1bfe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb1c80aeda0 0x7fb1c8071ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47130/0 (socket says 172.21.15.45:47130) 2024-04-25T10:31:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1bfe00640 1 -- 172.21.15.45:0/2509257786 learned_addr learned my addr 172.21.15.45:0/2509257786 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cd000640 1 --2- 172.21.15.45:0/2509257786 >> v2:172.21.15.45:3300/0 conn(0x7fb1c80721f0 0x7fb1c8166920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506328182 0 0) 0x7fb1c8071360 con 0x7fb1c80aeda0 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1ac003530 con 0x7fb1c80aeda0 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071443986 0 0) 0x7fb1c8159920 con 0x7fb1c8159540 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c8071360 con 0x7fb1c8159540 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cd000640 1 -- 172.21.15.45:0/2509257786 >> v1:172.21.15.117:6789/0 conn(0x7fb1c8159540 legacy=0x7fb1c80ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cd000640 1 -- 172.21.15.45:0/2509257786 >> v1:172.21.15.45:6789/0 conn(0x7fb1c80aeda0 legacy=0x7fb1c8071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.198+0000 7fb1cd000640 1 -- 172.21.15.45:0/2509257786 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c815c5d0 con 0x7fb1c80721f0 2024-04-25T10:31:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cd000640 1 --2- 172.21.15.45:0/2509257786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c8166920 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009d90 tx=0x7fb1c001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cf67269bc050da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00b2070 con 0x7fb1c80721f0 2024-04-25T10:31:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c00a5a00 con 0x7fb1c80721f0 2024-04-25T10:31:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1c7400640 1 -- 172.21.15.45:0/2509257786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00a5dd0 con 0x7fb1c80721f0 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- 172.21.15.45:0/2509257786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 msgr2=0x7fb1c8166920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2509257786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c8166920 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009d90 tx=0x7fb1c001c940 comp rx=0 tx=0).stop 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- 172.21.15.45:0/2509257786 shutdown_connections 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2509257786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c8166920 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- 172.21.15.45:0/2509257786 >> 172.21.15.45:0/2509257786 conn(0x7fb1c80669a0 msgr2=0x7fb1c8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- 172.21.15.45:0/2509257786 shutdown_connections 2024-04-25T10:31:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- 172.21.15.45:0/2509257786 wait complete. 2024-04-25T10:31:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 Processor -- start 2024-04-25T10:31:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- start start 2024-04-25T10:31:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c80aeda0 0x7fb1c814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c8159540 0x7fb1c81f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c8169680 con 0x7fb1c80721f0 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c806c0d0 con 0x7fb1c80aeda0 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1cf000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c8169800 con 0x7fb1c8159540 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c814ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35552/0 (socket says 172.21.15.45:35552) 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.202+0000 7fb1c7e00640 1 -- 172.21.15.45:0/2737018891 learned_addr learned my addr 172.21.15.45:0/2737018891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1bfe00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c80aeda0 0x7fb1c814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1cd000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c8159540 0x7fb1c81f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c7e00640 1 -- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c8159540 msgr2=0x7fb1c81f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c7e00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c8159540 0x7fb1c81f6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c7e00640 1 -- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c80aeda0 msgr2=0x7fb1c814e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c7e00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c80aeda0 0x7fb1c814e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c7e00640 1 -- 172.21.15.45:0/2737018891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c81532b0 con 0x7fb1c80721f0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1bfe00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c80aeda0 0x7fb1c814e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1cd000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c8159540 0x7fb1c81f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c7e00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c814ef90 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c430 tx=0x7fb1b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8017070 con 0x7fb1c80721f0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b80052f0 con 0x7fb1c80721f0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003840 con 0x7fb1c80721f0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c8153590 con 0x7fb1c80721f0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c8153b50 con 0x7fb1c80721f0 2024-04-25T10:31:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c804ef30 con 0x7fb1c80721f0 2024-04-25T10:31:55.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1b801a070 con 0x7fb1c80721f0 2024-04-25T10:31:55.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.206+0000 7fb1c4c00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1a4066f90 0x7fb1a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:55.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.210+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb1b8084350 con 0x7fb1c80721f0 2024-04-25T10:31:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.210+0000 7fb1bfe00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1a4066f90 0x7fb1a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.210+0000 7fb1bfe00640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1a4066f90 0x7fb1a4069450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80cc4b0 tx=0x7fb1b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.214+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1b80553b0 con 0x7fb1c80721f0 2024-04-25T10:31:55.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:55.410+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb1c80aeb50 con 0x7fb1c80721f0 2024-04-25T10:31:56.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.022+0000 7fb1c4c00640 1 -- 172.21.15.45:0/2737018891 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v136) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb1b8054cb0 con 0x7fb1c80721f0 2024-04-25T10:31:56.038 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 28 and data pool 29 2024-04-25T10:31:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1a4066f90 msgr2=0x7fb1a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1a4066f90 0x7fb1a4069450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80cc4b0 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2024-04-25T10:31:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 msgr2=0x7fb1c814ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c814ef90 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c430 tx=0x7fb1b800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 shutdown_connections 2024-04-25T10:31:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fb1a4066f90 0x7fb1a4069450 secure :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80cc4b0 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2024-04-25T10:31:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb1c8159540 0x7fb1c81f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c80aeda0 0x7fb1c814e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 --2- 172.21.15.45:0/2737018891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1c80721f0 0x7fb1c814ef90 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.042+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 >> 172.21.15.45:0/2737018891 conn(0x7fb1c80669a0 msgr2=0x7fb1c8066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.046+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 shutdown_connections 2024-04-25T10:31:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.046+0000 7fb1cf000640 1 -- 172.21.15.45:0/2737018891 wait complete. 2024-04-25T10:31:56.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:56.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a65800640 1 Processor -- start 2024-04-25T10:31:56.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a65800640 1 -- start start 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a65800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600b2a80 0x7f2a600b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a65800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a60070ed0 con 0x7f2a6015a0e0 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a65800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600711b0 con 0x7f2a60071f70 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a65800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600d0040 con 0x7f2a600b2a80 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a6015a0e0 0x7f2a6015b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47136/0 (socket says 172.21.15.45:47136) 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5ea00640 1 -- 172.21.15.45:0/3632294981 learned_addr learned my addr 172.21.15.45:0/3632294981 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:56.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5f400640 1 --2- 172.21.15.45:0/3632294981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600b2a80 0x7f2a600b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371262024 0 0) 0x7f2a60070ed0 con 0x7f2a6015a0e0 2024-04-25T10:31:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a40005740 con 0x7f2a6015a0e0 2024-04-25T10:31:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868052382 0 0) 0x7f2a600711b0 con 0x7f2a60071f70 2024-04-25T10:31:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a60070ed0 con 0x7f2a60071f70 2024-04-25T10:31:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5f400640 1 -- 172.21.15.45:0/3632294981 >> v1:172.21.15.117:6789/0 conn(0x7f2a60071f70 legacy=0x7f2a600cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5f400640 1 -- 172.21.15.45:0/3632294981 >> v1:172.21.15.45:6789/0 conn(0x7f2a6015a0e0 legacy=0x7f2a6015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.214+0000 7f2a5f400640 1 -- 172.21.15.45:0/3632294981 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60071490 con 0x7f2a600b2a80 2024-04-25T10:31:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a5f400640 1 --2- 172.21.15.45:0/3632294981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600b2a80 0x7f2a600b2f10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009dc0 tx=0x7f2a5801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b67ada7f9d58853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580b2070 con 0x7f2a600b2a80 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a580a5940 con 0x7f2a600b2a80 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a5e000640 1 -- 172.21.15.45:0/3632294981 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580a5cf0 con 0x7f2a600b2a80 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 -- 172.21.15.45:0/3632294981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600b2a80 msgr2=0x7f2a600b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 --2- 172.21.15.45:0/3632294981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600b2a80 0x7f2a600b2f10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009dc0 tx=0x7f2a5801c990 comp rx=0 tx=0).stop 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 -- 172.21.15.45:0/3632294981 shutdown_connections 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 --2- 172.21.15.45:0/3632294981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a600b2a80 0x7f2a600b2f10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 -- 172.21.15.45:0/3632294981 >> 172.21.15.45:0/3632294981 conn(0x7f2a60066b70 msgr2=0x7f2a60066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 -- 172.21.15.45:0/3632294981 shutdown_connections 2024-04-25T10:31:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.218+0000 7f2a65800640 1 -- 172.21.15.45:0/3632294981 wait complete. 2024-04-25T10:31:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 Processor -- start 2024-04-25T10:31:56.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 -- start start 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a60071f70 0x7f2a60154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600b2a80 0x7f2a60143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 0x7f2a600afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60070b50 con 0x7f2a6015a0e0 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015a4c0 con 0x7f2a600b2a80 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60072370 con 0x7f2a60071f70 2024-04-25T10:31:56.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 0x7f2a600afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 0x7f2a600afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35564/0 (socket says 172.21.15.45:35564) 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 -- 172.21.15.45:0/3506879625 learned_addr learned my addr 172.21.15.45:0/3506879625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a56000640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600b2a80 0x7f2a60143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5ea00640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a60071f70 0x7f2a60154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 -- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a60071f70 msgr2=0x7f2a60154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a60071f70 0x7f2a60154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 -- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600b2a80 msgr2=0x7f2a60143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600b2a80 0x7f2a60143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 -- 172.21.15.45:0/3506879625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a601f9d50 con 0x7f2a6015a0e0 2024-04-25T10:31:56.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5ea00640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a60071f70 0x7f2a60154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a5f400640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 0x7f2a600afac0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f2a580b0040 tx=0x7f2a58019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580b2050 con 0x7f2a6015a0e0 2024-04-25T10:31:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a580be070 con 0x7f2a6015a0e0 2024-04-25T10:31:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580ada70 con 0x7f2a6015a0e0 2024-04-25T10:31:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a601fa030 con 0x7f2a6015a0e0 2024-04-25T10:31:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.222+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a601fa5f0 con 0x7f2a6015a0e0 2024-04-25T10:31:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.226+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6004ef30 con 0x7f2a6015a0e0 2024-04-25T10:31:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.226+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a580a57c0 con 0x7f2a6015a0e0 2024-04-25T10:31:56.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.226+0000 7f2a57400640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066f90 0x7f2a3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.226+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a5811fc70 con 0x7f2a6015a0e0 2024-04-25T10:31:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.226+0000 7f2a5ea00640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066f90 0x7f2a3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.226+0000 7f2a5ea00640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066f90 0x7f2a3c069450 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000f70 tx=0x7f2a48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.230+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a580c7020 con 0x7f2a6015a0e0 2024-04-25T10:31:56.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.394+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a600703e0 con 0x7f2a6015a0e0 2024-04-25T10:31:56.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.394+0000 7f2a57400640 1 -- 172.21.15.45:0/3506879625 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f2a58016580 con 0x7f2a6015a0e0 2024-04-25T10:31:56.398 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:56.398 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":192,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:55.417607+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066f90 msgr2=0x7f2a3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066f90 0x7f2a3c069450 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000f70 tx=0x7f2a48009040 comp rx=0 tx=0).stop 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 msgr2=0x7f2a600afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 0x7f2a600afac0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f2a580b0040 tx=0x7f2a58019730 comp rx=0 tx=0).stop 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 shutdown_connections 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2a3c066f90 0x7f2a3c069450 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a6015a0e0 0x7f2a600afac0 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a600b2a80 0x7f2a60143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 --2- 172.21.15.45:0/3506879625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2a60071f70 0x7f2a60154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 >> 172.21.15.45:0/3506879625 conn(0x7f2a60066b70 msgr2=0x7f2a6014fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 shutdown_connections 2024-04-25T10:31:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.398+0000 7f2a65800640 1 -- 172.21.15.45:0/3506879625 wait complete. 2024-04-25T10:31:56.419 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:56.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 Processor -- start 2024-04-25T10:31:56.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 -- start start 2024-04-25T10:31:56.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe9f415b9e0 0x7fe9f40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 0x7fe9f40cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f40ccfb0 con 0x7fe9f415bdc0 2024-04-25T10:31:56.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f40cd290 con 0x7fe9f415b9e0 2024-04-25T10:31:56.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9fb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40cd570 con 0x7fe9f40d1c30 2024-04-25T10:31:56.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 0x7fe9f40cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f8c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe9f415b9e0 0x7fe9f40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 0x7fe9f40cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37192/0 (socket says 172.21.15.45:37192) 2024-04-25T10:31:56.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 -- 172.21.15.45:0/1246785197 learned_addr learned my addr 172.21.15.45:0/1246785197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:56.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 -- 172.21.15.45:0/1246785197 >> v2:172.21.15.117:3300/0 conn(0x7fe9f415b9e0 msgr2=0x7fe9f40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:56.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 --2- 172.21.15.45:0/1246785197 >> v2:172.21.15.117:3300/0 conn(0x7fe9f415b9e0 0x7fe9f40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 -- 172.21.15.45:0/1246785197 >> v1:172.21.15.117:6789/0 conn(0x7fe9f415bdc0 legacy=0x7fe9f40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.558+0000 7fe9f9600640 1 -- 172.21.15.45:0/1246785197 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40cd850 con 0x7fe9f40d1c30 2024-04-25T10:31:56.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9f9600640 1 --2- 172.21.15.45:0/1246785197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 0x7fe9f40cc950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009e50 tx=0x7fe9ec01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2771985287cb3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9f3e00640 1 -- 172.21.15.45:0/1246785197 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0a5710 con 0x7fe9f40d1c30 2024-04-25T10:31:56.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9f3e00640 1 -- 172.21.15.45:0/1246785197 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec0a58b0 con 0x7fe9f40d1c30 2024-04-25T10:31:56.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9f3e00640 1 -- 172.21.15.45:0/1246785197 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0a5c80 con 0x7fe9f40d1c30 2024-04-25T10:31:56.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 -- 172.21.15.45:0/1246785197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 msgr2=0x7fe9f40cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 --2- 172.21.15.45:0/1246785197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 0x7fe9f40cc950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009e50 tx=0x7fe9ec01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 -- 172.21.15.45:0/1246785197 shutdown_connections 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 --2- 172.21.15.45:0/1246785197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f40d1c30 0x7fe9f40cc950 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 --2- 172.21.15.45:0/1246785197 >> v2:172.21.15.117:3300/0 conn(0x7fe9f415b9e0 0x7fe9f40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 -- 172.21.15.45:0/1246785197 >> 172.21.15.45:0/1246785197 conn(0x7fe9f409a9d0 msgr2=0x7fe9f409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 -- 172.21.15.45:0/1246785197 shutdown_connections 2024-04-25T10:31:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.562+0000 7fe9fb600640 1 -- 172.21.15.45:0/1246785197 wait complete. 2024-04-25T10:31:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 Processor -- start 2024-04-25T10:31:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 -- start start 2024-04-25T10:31:56.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 0x7fe9f406c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9f415b9e0 0x7fe9f414a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f415bdc0 0x7fe9f406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c6630 con 0x7fe9f415b9e0 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c64b0 con 0x7fe9f40d1c30 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40c67b0 con 0x7fe9f415bdc0 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9f415b9e0 0x7fe9f414a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 0x7fe9f406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 0x7fe9f406c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52586/0 (socket says 172.21.15.45:52586) 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 -- 172.21.15.45:0/2047735278 learned_addr learned my addr 172.21.15.45:0/2047735278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f9600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f415bdc0 0x7fe9f406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 -- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f415bdc0 msgr2=0x7fe9f406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f415bdc0 0x7fe9f406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 -- 172.21.15.45:0/2047735278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9f415b9e0 msgr2=0x7fe9f414a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9f415b9e0 0x7fe9f414a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 -- 172.21.15.45:0/2047735278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f414a8c0 con 0x7fe9f40d1c30 2024-04-25T10:31:56.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f8c00640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 0x7fe9f406c830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400e9b0 tx=0x7fe9e400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4017070 con 0x7fe9f40d1c30 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e400c9a0 con 0x7fe9f40d1c30 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400cd70 con 0x7fe9f40d1c30 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f406b790 con 0x7fe9f40d1c30 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.566+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f406bd50 con 0x7fe9f40d1c30 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.570+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f404ef30 con 0x7fe9f40d1c30 2024-04-25T10:31:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.570+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9e401a070 con 0x7fe9f40d1c30 2024-04-25T10:31:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.570+0000 7fe9f1600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9d0066f60 0x7fe9d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.570+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9e4085860 con 0x7fe9f40d1c30 2024-04-25T10:31:56.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.570+0000 7fe9ebe00640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9d0066f60 0x7fe9d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.574+0000 7fe9ebe00640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9d0066f60 0x7fe9d0069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe9f41f9e20 tx=0x7fe9dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.574+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9e4056200 con 0x7fe9f40d1c30 2024-04-25T10:31:56.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.734+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9f40c6e70 con 0x7fe9f40d1c30 2024-04-25T10:31:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9f1600640 1 -- 172.21.15.45:0/2047735278 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+16684 (secure 0 0 0) 0x7fe9e405a2a0 con 0x7fe9f40d1c30 2024-04-25T10:31:56.740 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:56.740 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":192,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:55.417607+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:56.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9d0066f60 msgr2=0x7fe9d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9d0066f60 0x7fe9d0069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe9f41f9e20 tx=0x7fe9dc009040 comp rx=0 tx=0).stop 2024-04-25T10:31:56.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 msgr2=0x7fe9f406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 0x7fe9f406c830 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400e9b0 tx=0x7fe9e400ee70 comp rx=0 tx=0).stop 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 shutdown_connections 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe9d0066f60 0x7fe9d0069420 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9f415bdc0 0x7fe9f406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9f415b9e0 0x7fe9f414a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 --2- 172.21.15.45:0/2047735278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9f40d1c30 0x7fe9f406c830 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 >> 172.21.15.45:0/2047735278 conn(0x7fe9f409a9d0 msgr2=0x7fe9f406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 shutdown_connections 2024-04-25T10:31:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.738+0000 7fe9fb600640 1 -- 172.21.15.45:0/2047735278 wait complete. 2024-04-25T10:31:56.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:31:56.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f44400640 1 Processor -- start 2024-04-25T10:31:56.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f44400640 1 -- start start 2024-04-25T10:31:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f44400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1f3c0aeda0 0x7f1f3c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f44400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3c159920 con 0x7f1f3c072020 2024-04-25T10:31:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f44400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f3c15c2f0 con 0x7f1f3c159540 2024-04-25T10:31:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f44400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3c15c5d0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f42400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1f3c072020 0x7f1f3c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47146/0 (socket says 172.21.15.45:47146) 2024-04-25T10:31:56.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f42400640 1 -- 172.21.15.45:0/859411851 learned_addr learned my addr 172.21.15.45:0/859411851 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f41000640 1 --2- 172.21.15.45:0/859411851 >> v2:172.21.15.117:3300/0 conn(0x7f1f3c0aeda0 0x7f1f3c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207630699 0 0) 0x7f1f3c159920 con 0x7f1f3c072020 2024-04-25T10:31:56.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f24003530 con 0x7f1f3c072020 2024-04-25T10:31:56.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023669928 0 0) 0x7f1f3c15c2f0 con 0x7f1f3c159540 2024-04-25T10:31:56.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.906+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f3c159920 con 0x7f1f3c159540 2024-04-25T10:31:56.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f41000640 1 -- 172.21.15.45:0/859411851 >> v1:172.21.15.117:6789/0 conn(0x7f1f3c159540 legacy=0x7f1f3c0ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f41000640 1 -- 172.21.15.45:0/859411851 >> v1:172.21.15.45:6789/0 conn(0x7f1f3c072020 legacy=0x7f1f3c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f41000640 1 -- 172.21.15.45:0/859411851 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c15c8b0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f41000640 1 --2- 172.21.15.45:0/859411851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c071ae0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1f280097c0 tx=0x7f1f2801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c02f5ac1c94b4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280aa5f0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280af070 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f3be00640 1 -- 172.21.15.45:0/859411851 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280aa9b0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f44400640 1 -- 172.21.15.45:0/859411851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c0aeda0 msgr2=0x7f1f3c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f44400640 1 --2- 172.21.15.45:0/859411851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c071ae0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1f280097c0 tx=0x7f1f2801c470 comp rx=0 tx=0).stop 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f44400640 1 -- 172.21.15.45:0/859411851 shutdown_connections 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f44400640 1 --2- 172.21.15.45:0/859411851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c071ae0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.910+0000 7f1f44400640 1 -- 172.21.15.45:0/859411851 >> 172.21.15.45:0/859411851 conn(0x7f1f3c0669a0 msgr2=0x7f1f3c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 -- 172.21.15.45:0/859411851 shutdown_connections 2024-04-25T10:31:56.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 -- 172.21.15.45:0/859411851 wait complete. 2024-04-25T10:31:56.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 Processor -- start 2024-04-25T10:31:56.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 -- start start 2024-04-25T10:31:56.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f3c072020 0x7f1f3c13baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c159540 0x7f1f3c0af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c169940 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c072430 con 0x7f1f3c159540 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f44400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c169ac0 con 0x7f1f3c072020 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35594/0 (socket says 172.21.15.45:35594) 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 -- 172.21.15.45:0/1568897251 learned_addr learned my addr 172.21.15.45:0/1568897251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f42400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c159540 0x7f1f3c0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41a00640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f3c072020 0x7f1f3c13baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 -- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f3c072020 msgr2=0x7f1f3c13baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f3c072020 0x7f1f3c13baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 -- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c159540 msgr2=0x7f1f3c0af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c159540 0x7f1f3c0af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:56.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 -- 172.21.15.45:0/1568897251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c1602d0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f42400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c159540 0x7f1f3c0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41a00640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f3c072020 0x7f1f3c13baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f41000640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c151800 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f1f28019080 tx=0x7f1f2801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280be410 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280be5d0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.914+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280be9c0 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c13c340 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c13c900 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c04ef30 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f280bc020 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f39600640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1f1c066f90 0x7f1f1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:56.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f2811f240 con 0x7f1f3c0aeda0 2024-04-25T10:31:56.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.918+0000 7f1f41a00640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1f1c066f90 0x7f1f1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:56.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.922+0000 7f1f41a00640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1f1c066f90 0x7f1f1c069450 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c15bbc0 tx=0x7f1f30009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:56.922+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f28016090 con 0x7f1f3c0aeda0 2024-04-25T10:31:57.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:57.126+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1f3c15ff60 con 0x7f1f3c0aeda0 2024-04-25T10:31:58.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f39600640 1 -- 172.21.15.45:0/1568897251 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v139) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1f280efb60 con 0x7f1f3c0aeda0 2024-04-25T10:31:58.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1f1c066f90 msgr2=0x7f1f1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1f1c066f90 0x7f1f1c069450 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c15bbc0 tx=0x7f1f30009040 comp rx=0 tx=0).stop 2024-04-25T10:31:58.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 msgr2=0x7f1f3c151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c151800 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f1f28019080 tx=0x7f1f2801acb0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 shutdown_connections 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1f1c066f90 0x7f1f1c069450 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f3c159540 0x7f1f3c0af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f3c0aeda0 0x7f1f3c151800 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 --2- 172.21.15.45:0/1568897251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f3c072020 0x7f1f3c13baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 >> 172.21.15.45:0/1568897251 conn(0x7f1f3c0669a0 msgr2=0x7f1f3c06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 shutdown_connections 2024-04-25T10:31:58.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.042+0000 7f1f44400640 1 -- 172.21.15.45:0/1568897251 wait complete. 2024-04-25T10:31:58.070 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:31:58.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:31:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb6a00640 1 Processor -- start 2024-04-25T10:31:58.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb6a00640 1 -- start start 2024-04-25T10:31:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 0x7f8bb00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb6a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00cdab0 con 0x7f8bb006cf40 2024-04-25T10:31:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb6a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb00cd210 con 0x7f8bb00d0d00 2024-04-25T10:31:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00cd4f0 con 0x7f8bb00cc200 2024-04-25T10:31:58.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8bb006cf40 0x7f8bb015d830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47150/0 (socket says 172.21.15.45:47150) 2024-04-25T10:31:58.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 -- 172.21.15.45:0/2497321848 learned_addr learned my addr 172.21.15.45:0/2497321848 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baf400640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 0x7f8bb00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227067733 0 0) 0x7f8bb00cdab0 con 0x7f8bb006cf40 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b94003530 con 0x7f8bb006cf40 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912666325 0 0) 0x7f8bb00cd210 con 0x7f8bb00d0d00 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb00cdab0 con 0x7f8bb00d0d00 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176213147 0 0) 0x7f8b94003530 con 0x7f8bb006cf40 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 msgr2=0x7f8bb00ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 0x7f8bb00ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 >> v1:172.21.15.117:6789/0 conn(0x7f8bb00d0d00 legacy=0x7f8bb0162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00cd7d0 con 0x7f8bb006cf40 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f8ba8001a00 con 0x7f8bb006cf40 2024-04-25T10:31:58.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 >> v1:172.21.15.45:6789/0 conn(0x7f8bb006cf40 legacy=0x7f8bb015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 0x7f8b94003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004130 0x7f8b94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00cd4f0 con 0x7f8b94004d50 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9400bb90 con 0x7f8b94004130 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9400bd90 con 0x7f8bb00cc200 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bafe00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 0x7f8b94003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8baf400640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004130 0x7f8b94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 -- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 msgr2=0x7f8b94003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00cc200 0x7f8b94003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 -- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004130 msgr2=0x7f8b94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004130 0x7f8b94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.218+0000 7f8bb4a00640 1 -- 172.21.15.45:0/2497321848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b94003530 con 0x7f8b94004d50 2024-04-25T10:31:58.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb4a00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800a720 tx=0x7f8ba801d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83ac4f453c9dcef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800a370 con 0x7f8b94004d50 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba80ad650 con 0x7f8b94004d50 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8baea00640 1 -- 172.21.15.45:0/2497321848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80ada80 con 0x7f8b94004d50 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2497321848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b94004d50 msgr2=0x7f8b940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2497321848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800a720 tx=0x7f8ba801d240 comp rx=0 tx=0).stop 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb4a00640 1 -- 172.21.15.45:0/2497321848 reap_dead start 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2497321848 shutdown_connections 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2497321848 >> 172.21.15.45:0/2497321848 conn(0x7f8bb0072290 msgr2=0x7f8bb00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2497321848 shutdown_connections 2024-04-25T10:31:58.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.222+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2497321848 wait complete. 2024-04-25T10:31:58.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 Processor -- start 2024-04-25T10:31:58.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- start start 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004d50 0x7f8bb00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 0x7f8bb01f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00c9740 0x7f8bb00c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0162890 con 0x7f8bb0155010 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb006dc70 con 0x7f8b94004d50 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0162a10 con 0x7f8bb00c9740 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 0x7f8bb01f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 0x7f8bb01f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35612/0 (socket says 172.21.15.45:35612) 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 -- 172.21.15.45:0/2681468267 learned_addr learned my addr 172.21.15.45:0/2681468267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 -- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00c9740 msgr2=0x7f8bb00c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00c9740 0x7f8bb00c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 -- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004d50 msgr2=0x7f8bb00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004d50 0x7f8bb00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 -- 172.21.15.45:0/2681468267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0159210 con 0x7f8bb0155010 2024-04-25T10:31:58.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8baf400640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 0x7f8bb01f3b70 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c011a00 tx=0x7f8b9c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00fa70 con 0x7f8bb0155010 2024-04-25T10:31:58.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c00fc10 con 0x7f8bb0155010 2024-04-25T10:31:58.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c014440 con 0x7f8bb0155010 2024-04-25T10:31:58.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb01594f0 con 0x7f8bb0155010 2024-04-25T10:31:58.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb0159ab0 con 0x7f8bb0155010 2024-04-25T10:31:58.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.226+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb004ef30 con 0x7f8bb0155010 2024-04-25T10:31:58.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.230+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b9c012040 con 0x7f8bb0155010 2024-04-25T10:31:58.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.230+0000 7f8ba7e00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b90066f60 0x7f8b90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:58.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.230+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b9c0855f0 con 0x7f8bb0155010 2024-04-25T10:31:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.230+0000 7f8bafe00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b90066f60 0x7f8b90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:58.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.230+0000 7f8bafe00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b90066f60 0x7f8b90069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0005630 tx=0x7f8ba0007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:58.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.234+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b9c019030 con 0x7f8bb0155010 2024-04-25T10:31:58.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:58.438+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8bb0072ff0 con 0x7f8bb0155010 2024-04-25T10:31:59.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.042+0000 7f8ba7e00640 1 -- 172.21.15.45:0/2681468267 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v140) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8b9c01c360 con 0x7f8bb0155010 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b90066f60 msgr2=0x7f8b90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b90066f60 0x7f8b90069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0005630 tx=0x7f8ba0007a40 comp rx=0 tx=0).stop 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 msgr2=0x7f8bb01f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 0x7f8bb01f3b70 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c011a00 tx=0x7f8b9c011ec0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 shutdown_connections 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b90066f60 0x7f8b90069420 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb00c9740 0x7f8bb00c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8bb0155010 0x7f8bb01f3b70 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 --2- 172.21.15.45:0/2681468267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b94004d50 0x7f8bb00c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 >> 172.21.15.45:0/2681468267 conn(0x7f8bb0072290 msgr2=0x7f8bb00cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 shutdown_connections 2024-04-25T10:31:59.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.046+0000 7f8bb6a00640 1 -- 172.21.15.45:0/2681468267 wait complete. 2024-04-25T10:31:59.073 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:31:59.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:31:59.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 Processor -- start 2024-04-25T10:31:59.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 -- start start 2024-04-25T10:31:59.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c940d1820 0x7f3c940d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c9415e050 0x7f3c940b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c940b1870 con 0x7f3c940b13e0 2024-04-25T10:31:59.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c940b1af0 con 0x7f3c940d1820 2024-04-25T10:31:59.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c99e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940b1dd0 con 0x7f3c9415e050 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c8a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c9415e050 0x7f3c940b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c940d1820 0x7f3c940d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c940d1820 0x7f3c940d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52626/0 (socket says 172.21.15.45:52626) 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 -- 172.21.15.45:0/3776438029 learned_addr learned my addr 172.21.15.45:0/3776438029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 -- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c9415e050 msgr2=0x7f3c940b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 --2- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c9415e050 0x7f3c940b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 -- 172.21.15.45:0/3776438029 >> v1:172.21.15.117:6789/0 conn(0x7f3c940b13e0 legacy=0x7f3c940ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.218+0000 7f3c92a00640 1 -- 172.21.15.45:0/3776438029 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940b20b0 con 0x7f3c940d1820 2024-04-25T10:31:59.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c92a00640 1 --2- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940d1820 0x7f3c940d1c00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009870 tx=0x7f3c8401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c44834e5b2541a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c92000640 1 -- 172.21.15.45:0/3776438029 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c840a9620 con 0x7f3c940d1820 2024-04-25T10:31:59.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c92000640 1 -- 172.21.15.45:0/3776438029 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c840a97c0 con 0x7f3c940d1820 2024-04-25T10:31:59.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c92000640 1 -- 172.21.15.45:0/3776438029 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c840a9b90 con 0x7f3c940d1820 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 -- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940d1820 msgr2=0x7f3c940d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 --2- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940d1820 0x7f3c940d1c00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009870 tx=0x7f3c8401c560 comp rx=0 tx=0).stop 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 -- 172.21.15.45:0/3776438029 shutdown_connections 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 --2- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c9415e050 0x7f3c940b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 --2- 172.21.15.45:0/3776438029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940d1820 0x7f3c940d1c00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 -- 172.21.15.45:0/3776438029 >> 172.21.15.45:0/3776438029 conn(0x7f3c9409a9d0 msgr2=0x7f3c9409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 -- 172.21.15.45:0/3776438029 shutdown_connections 2024-04-25T10:31:59.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.222+0000 7f3c99e00640 1 -- 172.21.15.45:0/3776438029 wait complete. 2024-04-25T10:31:59.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 Processor -- start 2024-04-25T10:31:59.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 -- start start 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940b13e0 0x7f3c940cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c940d1820 0x7f3c94150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 0x7f3c9406e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9416ab20 con 0x7f3c9415e050 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9416a9a0 con 0x7f3c940b13e0 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9416aca0 con 0x7f3c940d1820 2024-04-25T10:31:59.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 0x7f3c9406e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c92a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940b13e0 0x7f3c940cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c8a000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c940d1820 0x7f3c94150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 0x7f3c9406e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35618/0 (socket says 172.21.15.45:35618) 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c92a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940b13e0 0x7f3c940cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52640/0 (socket says 172.21.15.45:52640) 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 -- 172.21.15.45:0/682988722 learned_addr learned my addr 172.21.15.45:0/682988722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 -- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c940d1820 msgr2=0x7f3c94150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c940d1820 0x7f3c94150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 -- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940b13e0 msgr2=0x7f3c940cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940b13e0 0x7f3c940cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 -- 172.21.15.45:0/682988722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940c8db0 con 0x7f3c9415e050 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c93400640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 0x7f3c9406e590 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00c770 tx=0x7f3c8c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c003b80 con 0x7f3c9415e050 2024-04-25T10:31:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c003d20 con 0x7f3c9415e050 2024-04-25T10:31:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00aa30 con 0x7f3c9415e050 2024-04-25T10:31:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c940c9090 con 0x7f3c9415e050 2024-04-25T10:31:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.226+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c940cdcb0 con 0x7f3c9415e050 2024-04-25T10:31:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.230+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9404ef30 con 0x7f3c9415e050 2024-04-25T10:31:59.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.230+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c8c00abd0 con 0x7f3c9415e050 2024-04-25T10:31:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.230+0000 7f3c8be00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c70066f90 0x7f3c70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.230+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c8c084e40 con 0x7f3c9415e050 2024-04-25T10:31:59.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.230+0000 7f3c92a00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c70066f90 0x7f3c70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.230+0000 7f3c92a00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c70066f90 0x7f3c70069450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3c840ae000 tx=0x7f3c840b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.234+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c940b1870 con 0x7f3c9415e050 2024-04-25T10:31:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.430+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c94070250 con 0x7f3c9415e050 2024-04-25T10:31:59.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.430+0000 7f3c8be00640 1 -- 172.21.15.45:0/682988722 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f3c8c055760 con 0x7f3c9415e050 2024-04-25T10:31:59.433 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:59.434 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":140,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:31:58.443777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11304":{"gid":11304,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3198476999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3198476999},{"type":"v1","addr":"172.21.15.45:6835","nonce":3198476999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:31:59.434 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 140 2024-04-25T10:31:59.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c70066f90 msgr2=0x7f3c70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c70066f90 0x7f3c70069450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3c840ae000 tx=0x7f3c840b3040 comp rx=0 tx=0).stop 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 msgr2=0x7f3c9406e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 0x7f3c9406e590 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00c770 tx=0x7f3c8c00cc30 comp rx=0 tx=0).stop 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 shutdown_connections 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3c70066f90 0x7f3c70069450 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c9415e050 0x7f3c9406e590 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c940d1820 0x7f3c94150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 --2- 172.21.15.45:0/682988722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c940b13e0 0x7f3c940cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 >> 172.21.15.45:0/682988722 conn(0x7f3c9409a9d0 msgr2=0x7f3c940d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 shutdown_connections 2024-04-25T10:31:59.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.434+0000 7f3c99e00640 1 -- 172.21.15.45:0/682988722 wait complete. 2024-04-25T10:31:59.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:31:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 Processor -- start 2024-04-25T10:31:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 -- start start 2024-04-25T10:31:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f1c0d1820 0x7f2f1c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 0x7f2f1c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0b1870 con 0x7f2f1c0b13e0 2024-04-25T10:31:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b1af0 con 0x7f2f1c0d1820 2024-04-25T10:31:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f22000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0b1dd0 con 0x7f2f1c15e050 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f1b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f1c0d1820 0x7f2f1c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 0x7f2f1c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 0x7f2f1c0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:37246/0 (socket says 172.21.15.45:37246) 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f1b400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f1c0d1820 0x7f2f1c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52644/0 (socket says 172.21.15.45:52644) 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 -- 172.21.15.45:0/7590470 learned_addr learned my addr 172.21.15.45:0/7590470 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f1be00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2f1c0b13e0 0x7f2f1c0ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:48384/0 (socket says 172.21.15.45:48384) 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 -- 172.21.15.45:0/7590470 >> v2:172.21.15.117:3300/0 conn(0x7f2f1c0d1820 msgr2=0x7f2f1c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 --2- 172.21.15.45:0/7590470 >> v2:172.21.15.117:3300/0 conn(0x7f2f1c0d1820 0x7f2f1c0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 -- 172.21.15.45:0/7590470 >> v1:172.21.15.117:6789/0 conn(0x7f2f1c0b13e0 legacy=0x7f2f1c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.594+0000 7f2f12a00640 1 -- 172.21.15.45:0/7590470 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0b20b0 con 0x7f2f1c15e050 2024-04-25T10:31:59.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f12a00640 1 --2- 172.21.15.45:0/7590470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 0x7f2f1c0b0ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009870 tx=0x7f2f0401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebe83c1c0807865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f1aa00640 1 -- 172.21.15.45:0/7590470 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040a9620 con 0x7f2f1c15e050 2024-04-25T10:31:59.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f1aa00640 1 -- 172.21.15.45:0/7590470 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f040a97c0 con 0x7f2f1c15e050 2024-04-25T10:31:59.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f1aa00640 1 -- 172.21.15.45:0/7590470 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040a9b90 con 0x7f2f1c15e050 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 -- 172.21.15.45:0/7590470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 msgr2=0x7f2f1c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 --2- 172.21.15.45:0/7590470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 0x7f2f1c0b0ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009870 tx=0x7f2f0401c560 comp rx=0 tx=0).stop 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 -- 172.21.15.45:0/7590470 shutdown_connections 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 --2- 172.21.15.45:0/7590470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c15e050 0x7f2f1c0b0ea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 --2- 172.21.15.45:0/7590470 >> v2:172.21.15.117:3300/0 conn(0x7f2f1c0d1820 0x7f2f1c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 -- 172.21.15.45:0/7590470 >> 172.21.15.45:0/7590470 conn(0x7f2f1c09a9d0 msgr2=0x7f2f1c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 -- 172.21.15.45:0/7590470 shutdown_connections 2024-04-25T10:31:59.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.598+0000 7f2f22000640 1 -- 172.21.15.45:0/7590470 wait complete. 2024-04-25T10:31:59.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 Processor -- start 2024-04-25T10:31:59.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 -- start start 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f1c0b13e0 0x7f2f1c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0d1820 0x7f2f1c15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 0x7f2f1c0b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c16ab20 con 0x7f2f1c15e050 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c16a9a0 con 0x7f2f1c0b13e0 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c16aca0 con 0x7f2f1c0d1820 2024-04-25T10:31:59.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 0x7f2f1c0b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 0x7f2f1c0b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35630/0 (socket says 172.21.15.45:35630) 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 -- 172.21.15.45:0/4238936261 learned_addr learned my addr 172.21.15.45:0/4238936261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1b400640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f1c0b13e0 0x7f2f1c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f12a00640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0d1820 0x7f2f1c15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 -- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0d1820 msgr2=0x7f2f1c15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0d1820 0x7f2f1c15bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 -- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f1c0b13e0 msgr2=0x7f2f1c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f1c0b13e0 0x7f2f1c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 -- 172.21.15.45:0/4238936261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c157ea0 con 0x7f2f1c15e050 2024-04-25T10:31:59.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f1be00640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 0x7f2f1c0b5dd0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c440 tx=0x7f2f1400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14018070 con 0x7f2f1c15e050 2024-04-25T10:31:59.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14007480 con 0x7f2f1c15e050 2024-04-25T10:31:59.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c1578e0 con 0x7f2f1c15e050 2024-04-25T10:31:59.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140047f0 con 0x7f2f1c15e050 2024-04-25T10:31:59.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.602+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c0b55d0 con 0x7f2f1c15e050 2024-04-25T10:31:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.606+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c04ef30 con 0x7f2f1c15e050 2024-04-25T10:31:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.606+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f14004990 con 0x7f2f1c15e050 2024-04-25T10:31:59.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.606+0000 7f2f13e00640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.606+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f14084230 con 0x7f2f1c15e050 2024-04-25T10:31:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.606+0000 7f2f1b400640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.606+0000 7f2f1b400640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0113e0 tx=0x7f2f0c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.610+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f1c0b1870 con 0x7f2f1c15e050 2024-04-25T10:31:59.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.774+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f1c151490 con 0x7f2f1c15e050 2024-04-25T10:31:59.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.774+0000 7f2f13e00640 1 -- 172.21.15.45:0/4238936261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f2f14054b60 con 0x7f2f1c15e050 2024-04-25T10:31:59.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:31:59.777 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":193,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:57.021531+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:31:59.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 msgr2=0x7f2ef8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0113e0 tx=0x7f2f0c00b040 comp rx=0 tx=0).stop 2024-04-25T10:31:59.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 msgr2=0x7f2f1c0b5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 0x7f2f1c0b5dd0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c440 tx=0x7f2f1400c900 comp rx=0 tx=0).stop 2024-04-25T10:31:59.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 shutdown_connections 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2ef8066f60 0x7f2ef8069420 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f1c15e050 0x7f2f1c0b5dd0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f1c0d1820 0x7f2f1c15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 --2- 172.21.15.45:0/4238936261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f1c0b13e0 0x7f2f1c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 >> 172.21.15.45:0/4238936261 conn(0x7f2f1c09a9d0 msgr2=0x7f2f1c0b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 shutdown_connections 2024-04-25T10:31:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.778+0000 7f2f22000640 1 -- 172.21.15.45:0/4238936261 wait complete. 2024-04-25T10:31:59.799 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:31:59.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:31:59.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 Processor -- start 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 -- start start 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f5415bc40 0x7f3f5406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f540b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540b2770 con 0x7f3f540cf930 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5406c670 con 0x7f3f5415bc40 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f5b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5406c950 con 0x7f3f540cda10 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f5415bc40 0x7f3f5406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3f5415bc40 0x7f3f5406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35646/0 (socket says 172.21.15.45:35646) 2024-04-25T10:31:59.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 -- 172.21.15.45:0/4130382150 learned_addr learned my addr 172.21.15.45:0/4130382150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f4b400640 1 --2- 172.21.15.45:0/4130382150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f540b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 -- 172.21.15.45:0/4130382150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 msgr2=0x7f3f540b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:31:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 --2- 172.21.15.45:0/4130382150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f540b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 -- 172.21.15.45:0/4130382150 >> v1:172.21.15.117:6789/0 conn(0x7f3f540cf930 legacy=0x7f3f5415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.938+0000 7f3f53e00640 1 -- 172.21.15.45:0/4130382150 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5406cc30 con 0x7f3f5415bc40 2024-04-25T10:31:59.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f53e00640 1 --2- 172.21.15.45:0/4130382150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5415bc40 0x7f3f5406c090 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f4401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc2e49b4b547643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f53400640 1 -- 172.21.15.45:0/4130382150 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440a7070 con 0x7f3f5415bc40 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f53400640 1 -- 172.21.15.45:0/4130382150 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440b0070 con 0x7f3f5415bc40 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f53400640 1 -- 172.21.15.45:0/4130382150 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440ab890 con 0x7f3f5415bc40 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 -- 172.21.15.45:0/4130382150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5415bc40 msgr2=0x7f3f5406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 --2- 172.21.15.45:0/4130382150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5415bc40 0x7f3f5406c090 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f4401c560 comp rx=0 tx=0).stop 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 -- 172.21.15.45:0/4130382150 shutdown_connections 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 --2- 172.21.15.45:0/4130382150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f540b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 --2- 172.21.15.45:0/4130382150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5415bc40 0x7f3f5406c090 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 -- 172.21.15.45:0/4130382150 >> 172.21.15.45:0/4130382150 conn(0x7f3f54066b70 msgr2=0x7f3f54066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 -- 172.21.15.45:0/4130382150 shutdown_connections 2024-04-25T10:31:59.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.942+0000 7f3f5b000640 1 -- 172.21.15.45:0/4130382150 wait complete. 2024-04-25T10:31:59.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 Processor -- start 2024-04-25T10:31:59.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 -- start start 2024-04-25T10:31:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f541fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 0x7f3f54153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5415bc40 0x7f3f54146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54168b50 con 0x7f3f540cf930 2024-04-25T10:31:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541689d0 con 0x7f3f5415bc40 2024-04-25T10:31:59.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54168cd0 con 0x7f3f540cda10 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 0x7f3f54153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 0x7f3f54153090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35654/0 (socket says 172.21.15.45:35654) 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 -- 172.21.15.45:0/2517066013 learned_addr learned my addr 172.21.15.45:0/2517066013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f53e00640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f541fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f59000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5415bc40 0x7f3f54146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 -- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 msgr2=0x7f3f541fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f541fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 -- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5415bc40 msgr2=0x7f3f54146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5415bc40 0x7f3f54146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 -- 172.21.15.45:0/2517066013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540af580 con 0x7f3f540cf930 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f59000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5415bc40 0x7f3f54146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f4b400640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 0x7f3f54153090 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00d5f0 tx=0x7f3f3c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0125f0 con 0x7f3f540cf930 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c019070 con 0x7f3f540cf930 2024-04-25T10:31:59.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0129b0 con 0x7f3f540cf930 2024-04-25T10:31:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540af860 con 0x7f3f540cf930 2024-04-25T10:31:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.946+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f540afe20 con 0x7f3f540cf930 2024-04-25T10:31:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.950+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5404ef30 con 0x7f3f540cf930 2024-04-25T10:31:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.950+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f3c012b50 con 0x7f3f540cf930 2024-04-25T10:31:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.950+0000 7f3f50c00640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:31:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.950+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f3c085440 con 0x7f3f540cf930 2024-04-25T10:31:59.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.950+0000 7f3f53e00640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:31:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.950+0000 7f3f53e00640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f30066f60 0x7f3f30069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3f440af000 tx=0x7f3f440a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:31:59.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:31:59.954+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f3c0563b0 con 0x7f3f540cf930 2024-04-25T10:32:00.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.114+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f5406d960 con 0x7f3f30066f60 2024-04-25T10:32:00.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f50c00640 1 -- 172.21.15.45:0/2517066013 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252490 (secure 0 0 0) 0x7f3f5406d960 con 0x7f3f30066f60 2024-04-25T10:32:00.126 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:00.127 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:32:00.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f30066f60 msgr2=0x7f3f30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f30066f60 0x7f3f30069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3f440af000 tx=0x7f3f440a7000 comp rx=0 tx=0).stop 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 msgr2=0x7f3f54153090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 0x7f3f54153090 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00d5f0 tx=0x7f3f3c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 shutdown_connections 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5415bc40 0x7f3f54146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f540cf930 0x7f3f54153090 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 --2- 172.21.15.45:0/2517066013 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f540cda10 0x7f3f541fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 >> 172.21.15.45:0/2517066013 conn(0x7f3f54066b70 msgr2=0x7f3f54158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 shutdown_connections 2024-04-25T10:32:00.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.122+0000 7f3f5b000640 1 -- 172.21.15.45:0/2517066013 wait complete. 2024-04-25T10:32:00.147 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":612,"stamp":"2024-04-25T10:31:59.072339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466644,"kb_used_data":20188,"kb_used_omap":12,"kb_used_meta":446323,"kb_avail":754508076,"statfs":{"total":773094113280,"available":772616269824,"internally_reserved":0,"allocated":20672512,"data_stored":9720784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457035312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.064658"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031301+0000","last_change":"2024-04-25T10:31:56.023445+0000","last_active":"2024-04-25T10:31:57.031301+0000","last_peered":"2024-04-25T10:31:57.031301+0000","last_clean":"2024-04-25T10:31:57.031301+0000","last_became_active":"2024-04-25T10:31:56.021503+0000","last_became_peered":"2024-04-25T10:31:56.021503+0000","last_unstale":"2024-04-25T10:31:57.031301+0000","last_undegraded":"2024-04-25T10:31:57.031301+0000","last_fullsized":"2024-04-25T10:31:57.031301+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:44.960404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034506+0000","last_change":"2024-04-25T10:31:56.045175+0000","last_active":"2024-04-25T10:31:57.034506+0000","last_peered":"2024-04-25T10:31:57.034506+0000","last_clean":"2024-04-25T10:31:57.034506+0000","last_became_active":"2024-04-25T10:31:56.044863+0000","last_became_peered":"2024-04-25T10:31:56.044863+0000","last_unstale":"2024-04-25T10:31:57.034506+0000","last_undegraded":"2024-04-25T10:31:57.034506+0000","last_fullsized":"2024-04-25T10:31:57.034506+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:58.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034408+0000","last_change":"2024-04-25T10:31:56.044289+0000","last_active":"2024-04-25T10:31:57.034408+0000","last_peered":"2024-04-25T10:31:57.034408+0000","last_clean":"2024-04-25T10:31:57.034408+0000","last_became_active":"2024-04-25T10:31:56.042526+0000","last_became_peered":"2024-04-25T10:31:56.042526+0000","last_unstale":"2024-04-25T10:31:57.034408+0000","last_undegraded":"2024-04-25T10:31:57.034408+0000","last_fullsized":"2024-04-25T10:31:57.034408+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:24.777713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402513+0000","last_change":"2024-04-25T10:31:56.036426+0000","last_active":"2024-04-25T10:31:57.402513+0000","last_peered":"2024-04-25T10:31:57.402513+0000","last_clean":"2024-04-25T10:31:57.402513+0000","last_became_active":"2024-04-25T10:31:56.036045+0000","last_became_peered":"2024-04-25T10:31:56.036045+0000","last_unstale":"2024-04-25T10:31:57.402513+0000","last_undegraded":"2024-04-25T10:31:57.402513+0000","last_fullsized":"2024-04-25T10:31:57.402513+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:33.363015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402550+0000","last_change":"2024-04-25T10:31:56.025083+0000","last_active":"2024-04-25T10:31:57.402550+0000","last_peered":"2024-04-25T10:31:57.402550+0000","last_clean":"2024-04-25T10:31:57.402550+0000","last_became_active":"2024-04-25T10:31:56.024811+0000","last_became_peered":"2024-04-25T10:31:56.024811+0000","last_unstale":"2024-04-25T10:31:57.402550+0000","last_undegraded":"2024-04-25T10:31:57.402550+0000","last_fullsized":"2024-04-25T10:31:57.402550+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:12.024481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.401991+0000","last_change":"2024-04-25T10:31:56.049902+0000","last_active":"2024-04-25T10:31:57.401991+0000","last_peered":"2024-04-25T10:31:57.401991+0000","last_clean":"2024-04-25T10:31:57.401991+0000","last_became_active":"2024-04-25T10:31:56.049278+0000","last_became_peered":"2024-04-25T10:31:56.049278+0000","last_unstale":"2024-04-25T10:31:57.401991+0000","last_undegraded":"2024-04-25T10:31:57.401991+0000","last_fullsized":"2024-04-25T10:31:57.401991+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:08.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034048+0000","last_change":"2024-04-25T10:31:56.034478+0000","last_active":"2024-04-25T10:31:57.034048+0000","last_peered":"2024-04-25T10:31:57.034048+0000","last_clean":"2024-04-25T10:31:57.034048+0000","last_became_active":"2024-04-25T10:31:56.034174+0000","last_became_peered":"2024-04-25T10:31:56.034174+0000","last_unstale":"2024-04-25T10:31:57.034048+0000","last_undegraded":"2024-04-25T10:31:57.034048+0000","last_fullsized":"2024-04-25T10:31:57.034048+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:12.836323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403597+0000","last_change":"2024-04-25T10:31:56.037618+0000","last_active":"2024-04-25T10:31:57.403597+0000","last_peered":"2024-04-25T10:31:57.403597+0000","last_clean":"2024-04-25T10:31:57.403597+0000","last_became_active":"2024-04-25T10:31:56.037024+0000","last_became_peered":"2024-04-25T10:31:56.037024+0000","last_unstale":"2024-04-25T10:31:57.403597+0000","last_undegraded":"2024-04-25T10:31:57.403597+0000","last_fullsized":"2024-04-25T10:31:57.403597+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:29.443748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399550+0000","last_change":"2024-04-25T10:31:56.044144+0000","last_active":"2024-04-25T10:31:57.399550+0000","last_peered":"2024-04-25T10:31:57.399550+0000","last_clean":"2024-04-25T10:31:57.399550+0000","last_became_active":"2024-04-25T10:31:56.043894+0000","last_became_peered":"2024-04-25T10:31:56.043894+0000","last_unstale":"2024-04-25T10:31:57.399550+0000","last_undegraded":"2024-04-25T10:31:57.399550+0000","last_fullsized":"2024-04-25T10:31:57.399550+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:58.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402674+0000","last_change":"2024-04-25T10:31:56.034797+0000","last_active":"2024-04-25T10:31:57.402674+0000","last_peered":"2024-04-25T10:31:57.402674+0000","last_clean":"2024-04-25T10:31:57.402674+0000","last_became_active":"2024-04-25T10:31:56.033015+0000","last_became_peered":"2024-04-25T10:31:56.033015+0000","last_unstale":"2024-04-25T10:31:57.402674+0000","last_undegraded":"2024-04-25T10:31:57.402674+0000","last_fullsized":"2024-04-25T10:31:57.402674+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:21.607757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034507+0000","last_change":"2024-04-25T10:31:56.040282+0000","last_active":"2024-04-25T10:31:57.034507+0000","last_peered":"2024-04-25T10:31:57.034507+0000","last_clean":"2024-04-25T10:31:57.034507+0000","last_became_active":"2024-04-25T10:31:56.037189+0000","last_became_peered":"2024-04-25T10:31:56.037189+0000","last_unstale":"2024-04-25T10:31:57.034507+0000","last_undegraded":"2024-04-25T10:31:57.034507+0000","last_fullsized":"2024-04-25T10:31:57.034507+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:31.440673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345250+0000","last_change":"2024-04-25T10:31:56.031240+0000","last_active":"2024-04-25T10:31:57.345250+0000","last_peered":"2024-04-25T10:31:57.345250+0000","last_clean":"2024-04-25T10:31:57.345250+0000","last_became_active":"2024-04-25T10:31:56.030869+0000","last_became_peered":"2024-04-25T10:31:56.030869+0000","last_unstale":"2024-04-25T10:31:57.345250+0000","last_undegraded":"2024-04-25T10:31:57.345250+0000","last_fullsized":"2024-04-25T10:31:57.345250+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:01.720443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030638+0000","last_change":"2024-04-25T10:31:56.023597+0000","last_active":"2024-04-25T10:31:57.030638+0000","last_peered":"2024-04-25T10:31:57.030638+0000","last_clean":"2024-04-25T10:31:57.030638+0000","last_became_active":"2024-04-25T10:31:56.022859+0000","last_became_peered":"2024-04-25T10:31:56.022859+0000","last_unstale":"2024-04-25T10:31:57.030638+0000","last_undegraded":"2024-04-25T10:31:57.030638+0000","last_fullsized":"2024-04-25T10:31:57.030638+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:03.724487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034141+0000","last_change":"2024-04-25T10:31:56.026046+0000","last_active":"2024-04-25T10:31:57.034141+0000","last_peered":"2024-04-25T10:31:57.034141+0000","last_clean":"2024-04-25T10:31:57.034141+0000","last_became_active":"2024-04-25T10:31:56.025760+0000","last_became_peered":"2024-04-25T10:31:56.025760+0000","last_unstale":"2024-04-25T10:31:57.034141+0000","last_undegraded":"2024-04-25T10:31:57.034141+0000","last_fullsized":"2024-04-25T10:31:57.034141+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:36.603692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402687+0000","last_change":"2024-04-25T10:31:56.034778+0000","last_active":"2024-04-25T10:31:57.402687+0000","last_peered":"2024-04-25T10:31:57.402687+0000","last_clean":"2024-04-25T10:31:57.402687+0000","last_became_active":"2024-04-25T10:31:56.034515+0000","last_became_peered":"2024-04-25T10:31:56.034515+0000","last_unstale":"2024-04-25T10:31:57.402687+0000","last_undegraded":"2024-04-25T10:31:57.402687+0000","last_fullsized":"2024-04-25T10:31:57.402687+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.394068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399054+0000","last_change":"2024-04-25T10:31:56.040209+0000","last_active":"2024-04-25T10:31:57.399054+0000","last_peered":"2024-04-25T10:31:57.399054+0000","last_clean":"2024-04-25T10:31:57.399054+0000","last_became_active":"2024-04-25T10:31:56.034440+0000","last_became_peered":"2024-04-25T10:31:56.034440+0000","last_unstale":"2024-04-25T10:31:57.399054+0000","last_undegraded":"2024-04-25T10:31:57.399054+0000","last_fullsized":"2024-04-25T10:31:57.399054+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:09.424061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031484+0000","last_change":"2024-04-25T10:31:56.030156+0000","last_active":"2024-04-25T10:31:57.031484+0000","last_peered":"2024-04-25T10:31:57.031484+0000","last_clean":"2024-04-25T10:31:57.031484+0000","last_became_active":"2024-04-25T10:31:56.029879+0000","last_became_peered":"2024-04-25T10:31:56.029879+0000","last_unstale":"2024-04-25T10:31:57.031484+0000","last_undegraded":"2024-04-25T10:31:57.031484+0000","last_fullsized":"2024-04-25T10:31:57.031484+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:01.093217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036497+0000","last_change":"2024-04-25T10:31:56.030778+0000","last_active":"2024-04-25T10:31:57.036497+0000","last_peered":"2024-04-25T10:31:57.036497+0000","last_clean":"2024-04-25T10:31:57.036497+0000","last_became_active":"2024-04-25T10:31:56.030499+0000","last_became_peered":"2024-04-25T10:31:56.030499+0000","last_unstale":"2024-04-25T10:31:57.036497+0000","last_undegraded":"2024-04-25T10:31:57.036497+0000","last_fullsized":"2024-04-25T10:31:57.036497+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:11.830077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403048+0000","last_change":"2024-04-25T10:31:56.036430+0000","last_active":"2024-04-25T10:31:57.403048+0000","last_peered":"2024-04-25T10:31:57.403048+0000","last_clean":"2024-04-25T10:31:57.403048+0000","last_became_active":"2024-04-25T10:31:56.036034+0000","last_became_peered":"2024-04-25T10:31:56.036034+0000","last_unstale":"2024-04-25T10:31:57.403048+0000","last_undegraded":"2024-04-25T10:31:57.403048+0000","last_fullsized":"2024-04-25T10:31:57.403048+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:41.664987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036311+0000","last_change":"2024-04-25T10:31:56.037514+0000","last_active":"2024-04-25T10:31:57.036311+0000","last_peered":"2024-04-25T10:31:57.036311+0000","last_clean":"2024-04-25T10:31:57.036311+0000","last_became_active":"2024-04-25T10:31:56.037130+0000","last_became_peered":"2024-04-25T10:31:56.037130+0000","last_unstale":"2024-04-25T10:31:57.036311+0000","last_undegraded":"2024-04-25T10:31:57.036311+0000","last_fullsized":"2024-04-25T10:31:57.036311+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:15.835205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036270+0000","last_change":"2024-04-25T10:31:56.028998+0000","last_active":"2024-04-25T10:31:57.036270+0000","last_peered":"2024-04-25T10:31:57.036270+0000","last_clean":"2024-04-25T10:31:57.036270+0000","last_became_active":"2024-04-25T10:31:56.028660+0000","last_became_peered":"2024-04-25T10:31:56.028660+0000","last_unstale":"2024-04-25T10:31:57.036270+0000","last_undegraded":"2024-04-25T10:31:57.036270+0000","last_fullsized":"2024-04-25T10:31:57.036270+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:51.698936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.401976+0000","last_change":"2024-04-25T10:31:56.049210+0000","last_active":"2024-04-25T10:31:57.401976+0000","last_peered":"2024-04-25T10:31:57.401976+0000","last_clean":"2024-04-25T10:31:57.401976+0000","last_became_active":"2024-04-25T10:31:56.048931+0000","last_became_peered":"2024-04-25T10:31:56.048931+0000","last_unstale":"2024-04-25T10:31:57.401976+0000","last_undegraded":"2024-04-25T10:31:57.401976+0000","last_fullsized":"2024-04-25T10:31:57.401976+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:01.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030130+0000","last_change":"2024-04-25T10:31:56.033948+0000","last_active":"2024-04-25T10:31:57.030130+0000","last_peered":"2024-04-25T10:31:57.030130+0000","last_clean":"2024-04-25T10:31:57.030130+0000","last_became_active":"2024-04-25T10:31:56.032652+0000","last_became_peered":"2024-04-25T10:31:56.032652+0000","last_unstale":"2024-04-25T10:31:57.030130+0000","last_undegraded":"2024-04-25T10:31:57.030130+0000","last_fullsized":"2024-04-25T10:31:57.030130+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:48.150813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402173+0000","last_change":"2024-04-25T10:31:56.032747+0000","last_active":"2024-04-25T10:31:57.402173+0000","last_peered":"2024-04-25T10:31:57.402173+0000","last_clean":"2024-04-25T10:31:57.402173+0000","last_became_active":"2024-04-25T10:31:56.032514+0000","last_became_peered":"2024-04-25T10:31:56.032514+0000","last_unstale":"2024-04-25T10:31:57.402173+0000","last_undegraded":"2024-04-25T10:31:57.402173+0000","last_fullsized":"2024-04-25T10:31:57.402173+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:15.668277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034593+0000","last_change":"2024-04-25T10:31:56.043336+0000","last_active":"2024-04-25T10:31:57.034593+0000","last_peered":"2024-04-25T10:31:57.034593+0000","last_clean":"2024-04-25T10:31:57.034593+0000","last_became_active":"2024-04-25T10:31:56.042781+0000","last_became_peered":"2024-04-25T10:31:56.042781+0000","last_unstale":"2024-04-25T10:31:57.034593+0000","last_undegraded":"2024-04-25T10:31:57.034593+0000","last_fullsized":"2024-04-25T10:31:57.034593+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:01.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403145+0000","last_change":"2024-04-25T10:31:56.039511+0000","last_active":"2024-04-25T10:31:57.403145+0000","last_peered":"2024-04-25T10:31:57.403145+0000","last_clean":"2024-04-25T10:31:57.403145+0000","last_became_active":"2024-04-25T10:31:56.039235+0000","last_became_peered":"2024-04-25T10:31:56.039235+0000","last_unstale":"2024-04-25T10:31:57.403145+0000","last_undegraded":"2024-04-25T10:31:57.403145+0000","last_fullsized":"2024-04-25T10:31:57.403145+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:47.429622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034419+0000","last_change":"2024-04-25T10:31:56.037447+0000","last_active":"2024-04-25T10:31:57.034419+0000","last_peered":"2024-04-25T10:31:57.034419+0000","last_clean":"2024-04-25T10:31:57.034419+0000","last_became_active":"2024-04-25T10:31:56.036915+0000","last_became_peered":"2024-04-25T10:31:56.036915+0000","last_unstale":"2024-04-25T10:31:57.034419+0000","last_undegraded":"2024-04-25T10:31:57.034419+0000","last_fullsized":"2024-04-25T10:31:57.034419+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:25.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403672+0000","last_change":"2024-04-25T10:31:56.037048+0000","last_active":"2024-04-25T10:31:57.403672+0000","last_peered":"2024-04-25T10:31:57.403672+0000","last_clean":"2024-04-25T10:31:57.403672+0000","last_became_active":"2024-04-25T10:31:56.036739+0000","last_became_peered":"2024-04-25T10:31:56.036739+0000","last_unstale":"2024-04-25T10:31:57.403672+0000","last_undegraded":"2024-04-25T10:31:57.403672+0000","last_fullsized":"2024-04-25T10:31:57.403672+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:03.724487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399446+0000","last_change":"2024-04-25T10:31:56.052285+0000","last_active":"2024-04-25T10:31:57.399446+0000","last_peered":"2024-04-25T10:31:57.399446+0000","last_clean":"2024-04-25T10:31:57.399446+0000","last_became_active":"2024-04-25T10:31:56.052005+0000","last_became_peered":"2024-04-25T10:31:56.052005+0000","last_unstale":"2024-04-25T10:31:57.399446+0000","last_undegraded":"2024-04-25T10:31:57.399446+0000","last_fullsized":"2024-04-25T10:31:57.399446+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.498103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345087+0000","last_change":"2024-04-25T10:31:56.045203+0000","last_active":"2024-04-25T10:31:57.345087+0000","last_peered":"2024-04-25T10:31:57.345087+0000","last_clean":"2024-04-25T10:31:57.345087+0000","last_became_active":"2024-04-25T10:31:56.044891+0000","last_became_peered":"2024-04-25T10:31:56.044891+0000","last_unstale":"2024-04-25T10:31:57.345087+0000","last_undegraded":"2024-04-25T10:31:57.345087+0000","last_fullsized":"2024-04-25T10:31:57.345087+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:19.694665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.035967+0000","last_change":"2024-04-25T10:31:56.033978+0000","last_active":"2024-04-25T10:31:57.035967+0000","last_peered":"2024-04-25T10:31:57.035967+0000","last_clean":"2024-04-25T10:31:57.035967+0000","last_became_active":"2024-04-25T10:31:56.033528+0000","last_became_peered":"2024-04-25T10:31:56.033528+0000","last_unstale":"2024-04-25T10:31:57.035967+0000","last_undegraded":"2024-04-25T10:31:57.035967+0000","last_fullsized":"2024-04-25T10:31:57.035967+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:07.112556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403637+0000","last_change":"2024-04-25T10:31:56.040924+0000","last_active":"2024-04-25T10:31:57.403637+0000","last_peered":"2024-04-25T10:31:57.403637+0000","last_clean":"2024-04-25T10:31:57.403637+0000","last_became_active":"2024-04-25T10:31:56.040568+0000","last_became_peered":"2024-04-25T10:31:56.040568+0000","last_unstale":"2024-04-25T10:31:57.403637+0000","last_undegraded":"2024-04-25T10:31:57.403637+0000","last_fullsized":"2024-04-25T10:31:57.403637+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.155270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034648+0000","last_change":"2024-04-25T10:31:56.045694+0000","last_active":"2024-04-25T10:31:57.034648+0000","last_peered":"2024-04-25T10:31:57.034648+0000","last_clean":"2024-04-25T10:31:57.034648+0000","last_became_active":"2024-04-25T10:31:56.045417+0000","last_became_peered":"2024-04-25T10:31:56.045417+0000","last_unstale":"2024-04-25T10:31:57.034648+0000","last_undegraded":"2024-04-25T10:31:57.034648+0000","last_fullsized":"2024-04-25T10:31:57.034648+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:08.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"192'4","reported_seq":18,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402663+0000","last_change":"2024-04-25T10:31:55.039969+0000","last_active":"2024-04-25T10:31:57.402663+0000","last_peered":"2024-04-25T10:31:57.402663+0000","last_clean":"2024-04-25T10:31:57.402663+0000","last_became_active":"2024-04-25T10:31:55.039769+0000","last_became_peered":"2024-04-25T10:31:55.039769+0000","last_unstale":"2024-04-25T10:31:57.402663+0000","last_undegraded":"2024-04-25T10:31:57.402663+0000","last_fullsized":"2024-04-25T10:31:57.402663+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:46.497160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402546+0000","last_change":"2024-04-25T10:31:56.045843+0000","last_active":"2024-04-25T10:31:57.402546+0000","last_peered":"2024-04-25T10:31:57.402546+0000","last_clean":"2024-04-25T10:31:57.402546+0000","last_became_active":"2024-04-25T10:31:56.045532+0000","last_became_peered":"2024-04-25T10:31:56.045532+0000","last_unstale":"2024-04-25T10:31:57.402546+0000","last_undegraded":"2024-04-25T10:31:57.402546+0000","last_fullsized":"2024-04-25T10:31:57.402546+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031234+0000","last_change":"2024-04-25T10:31:55.023720+0000","last_active":"2024-04-25T10:31:57.031234+0000","last_peered":"2024-04-25T10:31:57.031234+0000","last_clean":"2024-04-25T10:31:57.031234+0000","last_became_active":"2024-04-25T10:31:55.023387+0000","last_became_peered":"2024-04-25T10:31:55.023387+0000","last_unstale":"2024-04-25T10:31:57.031234+0000","last_undegraded":"2024-04-25T10:31:57.031234+0000","last_fullsized":"2024-04-25T10:31:57.031234+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:19.325725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399322+0000","last_change":"2024-04-25T10:31:56.042149+0000","last_active":"2024-04-25T10:31:57.399322+0000","last_peered":"2024-04-25T10:31:57.399322+0000","last_clean":"2024-04-25T10:31:57.399322+0000","last_became_active":"2024-04-25T10:31:56.041840+0000","last_became_peered":"2024-04-25T10:31:56.041840+0000","last_unstale":"2024-04-25T10:31:57.399322+0000","last_undegraded":"2024-04-25T10:31:57.399322+0000","last_fullsized":"2024-04-25T10:31:57.399322+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:38.911913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345384+0000","last_change":"2024-04-25T10:31:55.030296+0000","last_active":"2024-04-25T10:31:57.345384+0000","last_peered":"2024-04-25T10:31:57.345384+0000","last_clean":"2024-04-25T10:31:57.345384+0000","last_became_active":"2024-04-25T10:31:55.029941+0000","last_became_peered":"2024-04-25T10:31:55.029941+0000","last_unstale":"2024-04-25T10:31:57.345384+0000","last_undegraded":"2024-04-25T10:31:57.345384+0000","last_fullsized":"2024-04-25T10:31:57.345384+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:13.283029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034145+0000","last_change":"2024-04-25T10:31:56.040297+0000","last_active":"2024-04-25T10:31:57.034145+0000","last_peered":"2024-04-25T10:31:57.034145+0000","last_clean":"2024-04-25T10:31:57.034145+0000","last_became_active":"2024-04-25T10:31:56.033706+0000","last_became_peered":"2024-04-25T10:31:56.033706+0000","last_unstale":"2024-04-25T10:31:57.034145+0000","last_undegraded":"2024-04-25T10:31:57.034145+0000","last_fullsized":"2024-04-25T10:31:57.034145+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:16.395471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402661+0000","last_change":"2024-04-25T10:31:55.033280+0000","last_active":"2024-04-25T10:31:57.402661+0000","last_peered":"2024-04-25T10:31:57.402661+0000","last_clean":"2024-04-25T10:31:57.402661+0000","last_became_active":"2024-04-25T10:31:55.032990+0000","last_became_peered":"2024-04-25T10:31:55.032990+0000","last_unstale":"2024-04-25T10:31:57.402661+0000","last_undegraded":"2024-04-25T10:31:57.402661+0000","last_fullsized":"2024-04-25T10:31:57.402661+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:33.336788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402465+0000","last_change":"2024-04-25T10:31:56.045503+0000","last_active":"2024-04-25T10:31:57.402465+0000","last_peered":"2024-04-25T10:31:57.402465+0000","last_clean":"2024-04-25T10:31:57.402465+0000","last_became_active":"2024-04-25T10:31:56.044565+0000","last_became_peered":"2024-04-25T10:31:56.044565+0000","last_unstale":"2024-04-25T10:31:57.402465+0000","last_undegraded":"2024-04-25T10:31:57.402465+0000","last_fullsized":"2024-04-25T10:31:57.402465+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:40.920342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030355+0000","last_change":"2024-04-25T10:31:55.037216+0000","last_active":"2024-04-25T10:31:57.030355+0000","last_peered":"2024-04-25T10:31:57.030355+0000","last_clean":"2024-04-25T10:31:57.030355+0000","last_became_active":"2024-04-25T10:31:55.036643+0000","last_became_peered":"2024-04-25T10:31:55.036643+0000","last_unstale":"2024-04-25T10:31:57.030355+0000","last_undegraded":"2024-04-25T10:31:57.030355+0000","last_fullsized":"2024-04-25T10:31:57.030355+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:16.585281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402052+0000","last_change":"2024-04-25T10:31:56.045099+0000","last_active":"2024-04-25T10:31:57.402052+0000","last_peered":"2024-04-25T10:31:57.402052+0000","last_clean":"2024-04-25T10:31:57.402052+0000","last_became_active":"2024-04-25T10:31:56.044823+0000","last_became_peered":"2024-04-25T10:31:56.044823+0000","last_unstale":"2024-04-25T10:31:57.402052+0000","last_undegraded":"2024-04-25T10:31:57.402052+0000","last_fullsized":"2024-04-25T10:31:57.402052+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:26.446254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399076+0000","last_change":"2024-04-25T10:31:55.031406+0000","last_active":"2024-04-25T10:31:57.399076+0000","last_peered":"2024-04-25T10:31:57.399076+0000","last_clean":"2024-04-25T10:31:57.399076+0000","last_became_active":"2024-04-25T10:31:55.031130+0000","last_became_peered":"2024-04-25T10:31:55.031130+0000","last_unstale":"2024-04-25T10:31:57.399076+0000","last_undegraded":"2024-04-25T10:31:57.399076+0000","last_fullsized":"2024-04-25T10:31:57.399076+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:28.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030191+0000","last_change":"2024-04-25T10:31:56.039933+0000","last_active":"2024-04-25T10:31:57.030191+0000","last_peered":"2024-04-25T10:31:57.030191+0000","last_clean":"2024-04-25T10:31:57.030191+0000","last_became_active":"2024-04-25T10:31:56.038974+0000","last_became_peered":"2024-04-25T10:31:56.038974+0000","last_unstale":"2024-04-25T10:31:57.030191+0000","last_undegraded":"2024-04-25T10:31:57.030191+0000","last_fullsized":"2024-04-25T10:31:57.030191+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:31.153275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.035905+0000","last_change":"2024-04-25T10:31:55.032340+0000","last_active":"2024-04-25T10:31:57.035905+0000","last_peered":"2024-04-25T10:31:57.035905+0000","last_clean":"2024-04-25T10:31:57.035905+0000","last_became_active":"2024-04-25T10:31:55.032044+0000","last_became_peered":"2024-04-25T10:31:55.032044+0000","last_unstale":"2024-04-25T10:31:57.035905+0000","last_undegraded":"2024-04-25T10:31:57.035905+0000","last_fullsized":"2024-04-25T10:31:57.035905+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:00.183550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345313+0000","last_change":"2024-04-25T10:31:56.044302+0000","last_active":"2024-04-25T10:31:57.345313+0000","last_peered":"2024-04-25T10:31:57.345313+0000","last_clean":"2024-04-25T10:31:57.345313+0000","last_became_active":"2024-04-25T10:31:56.044051+0000","last_became_peered":"2024-04-25T10:31:56.044051+0000","last_unstale":"2024-04-25T10:31:57.345313+0000","last_undegraded":"2024-04-25T10:31:57.345313+0000","last_fullsized":"2024-04-25T10:31:57.345313+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:51.685428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031448+0000","last_change":"2024-04-25T10:31:55.035554+0000","last_active":"2024-04-25T10:31:57.031448+0000","last_peered":"2024-04-25T10:31:57.031448+0000","last_clean":"2024-04-25T10:31:57.031448+0000","last_became_active":"2024-04-25T10:31:55.034702+0000","last_became_peered":"2024-04-25T10:31:55.034702+0000","last_unstale":"2024-04-25T10:31:57.031448+0000","last_undegraded":"2024-04-25T10:31:57.031448+0000","last_fullsized":"2024-04-25T10:31:57.031448+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:10.580399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034702+0000","last_change":"2024-04-25T10:31:56.043084+0000","last_active":"2024-04-25T10:31:57.034702+0000","last_peered":"2024-04-25T10:31:57.034702+0000","last_clean":"2024-04-25T10:31:57.034702+0000","last_became_active":"2024-04-25T10:31:56.042460+0000","last_became_peered":"2024-04-25T10:31:56.042460+0000","last_unstale":"2024-04-25T10:31:57.034702+0000","last_undegraded":"2024-04-25T10:31:57.034702+0000","last_fullsized":"2024-04-25T10:31:57.034702+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:26.446254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402765+0000","last_change":"2024-04-25T10:31:55.031555+0000","last_active":"2024-04-25T10:31:57.402765+0000","last_peered":"2024-04-25T10:31:57.402765+0000","last_clean":"2024-04-25T10:31:57.402765+0000","last_became_active":"2024-04-25T10:31:55.030013+0000","last_became_peered":"2024-04-25T10:31:55.030013+0000","last_unstale":"2024-04-25T10:31:57.402765+0000","last_undegraded":"2024-04-25T10:31:57.402765+0000","last_fullsized":"2024-04-25T10:31:57.402765+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:52.569927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345380+0000","last_change":"2024-04-25T10:31:56.035415+0000","last_active":"2024-04-25T10:31:57.345380+0000","last_peered":"2024-04-25T10:31:57.345380+0000","last_clean":"2024-04-25T10:31:57.345380+0000","last_became_active":"2024-04-25T10:31:56.034925+0000","last_became_peered":"2024-04-25T10:31:56.034925+0000","last_unstale":"2024-04-25T10:31:57.345380+0000","last_undegraded":"2024-04-25T10:31:57.345380+0000","last_fullsized":"2024-04-25T10:31:57.345380+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:47.638947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"192'3","reported_seq":17,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403595+0000","last_change":"2024-04-25T10:31:55.033297+0000","last_active":"2024-04-25T10:31:57.403595+0000","last_peered":"2024-04-25T10:31:57.403595+0000","last_clean":"2024-04-25T10:31:57.403595+0000","last_became_active":"2024-04-25T10:31:55.032807+0000","last_became_peered":"2024-04-25T10:31:55.032807+0000","last_unstale":"2024-04-25T10:31:57.403595+0000","last_undegraded":"2024-04-25T10:31:57.403595+0000","last_fullsized":"2024-04-25T10:31:57.403595+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.828603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345271+0000","last_change":"2024-04-25T10:31:56.044048+0000","last_active":"2024-04-25T10:31:57.345271+0000","last_peered":"2024-04-25T10:31:57.345271+0000","last_clean":"2024-04-25T10:31:57.345271+0000","last_became_active":"2024-04-25T10:31:56.043694+0000","last_became_peered":"2024-04-25T10:31:56.043694+0000","last_unstale":"2024-04-25T10:31:57.345271+0000","last_undegraded":"2024-04-25T10:31:57.345271+0000","last_fullsized":"2024-04-25T10:31:57.345271+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:51.139081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"192'6","reported_seq":20,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036342+0000","last_change":"2024-04-25T10:31:55.025577+0000","last_active":"2024-04-25T10:31:57.036342+0000","last_peered":"2024-04-25T10:31:57.036342+0000","last_clean":"2024-04-25T10:31:57.036342+0000","last_became_active":"2024-04-25T10:31:55.025302+0000","last_became_peered":"2024-04-25T10:31:55.025302+0000","last_unstale":"2024-04-25T10:31:57.036342+0000","last_undegraded":"2024-04-25T10:31:57.036342+0000","last_fullsized":"2024-04-25T10:31:57.036342+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:30.433025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402142+0000","last_change":"2024-04-25T10:31:56.037644+0000","last_active":"2024-04-25T10:31:57.402142+0000","last_peered":"2024-04-25T10:31:57.402142+0000","last_clean":"2024-04-25T10:31:57.402142+0000","last_became_active":"2024-04-25T10:31:56.037368+0000","last_became_peered":"2024-04-25T10:31:56.037368+0000","last_unstale":"2024-04-25T10:31:57.402142+0000","last_undegraded":"2024-04-25T10:31:57.402142+0000","last_fullsized":"2024-04-25T10:31:57.402142+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:23.827566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403708+0000","last_change":"2024-04-25T10:31:55.037212+0000","last_active":"2024-04-25T10:31:57.403708+0000","last_peered":"2024-04-25T10:31:57.403708+0000","last_clean":"2024-04-25T10:31:57.403708+0000","last_became_active":"2024-04-25T10:31:55.036803+0000","last_became_peered":"2024-04-25T10:31:55.036803+0000","last_unstale":"2024-04-25T10:31:57.403708+0000","last_undegraded":"2024-04-25T10:31:57.403708+0000","last_fullsized":"2024-04-25T10:31:57.403708+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:26.167872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345225+0000","last_change":"2024-04-25T10:31:56.034558+0000","last_active":"2024-04-25T10:31:57.345225+0000","last_peered":"2024-04-25T10:31:57.345225+0000","last_clean":"2024-04-25T10:31:57.345225+0000","last_became_active":"2024-04-25T10:31:56.034244+0000","last_became_peered":"2024-04-25T10:31:56.034244+0000","last_unstale":"2024-04-25T10:31:57.345225+0000","last_undegraded":"2024-04-25T10:31:57.345225+0000","last_fullsized":"2024-04-25T10:31:57.345225+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:22.859699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402250+0000","last_change":"2024-04-25T10:31:55.037875+0000","last_active":"2024-04-25T10:31:57.402250+0000","last_peered":"2024-04-25T10:31:57.402250+0000","last_clean":"2024-04-25T10:31:57.402250+0000","last_became_active":"2024-04-25T10:31:55.037582+0000","last_became_peered":"2024-04-25T10:31:55.037582+0000","last_unstale":"2024-04-25T10:31:57.402250+0000","last_undegraded":"2024-04-25T10:31:57.402250+0000","last_fullsized":"2024-04-25T10:31:57.402250+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:13.004953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399085+0000","last_change":"2024-04-25T10:31:56.045978+0000","last_active":"2024-04-25T10:31:57.399085+0000","last_peered":"2024-04-25T10:31:57.399085+0000","last_clean":"2024-04-25T10:31:57.399085+0000","last_became_active":"2024-04-25T10:31:56.045549+0000","last_became_peered":"2024-04-25T10:31:56.045549+0000","last_unstale":"2024-04-25T10:31:57.399085+0000","last_undegraded":"2024-04-25T10:31:57.399085+0000","last_fullsized":"2024-04-25T10:31:57.399085+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:12.836323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402098+0000","last_change":"2024-04-25T10:31:55.034324+0000","last_active":"2024-04-25T10:31:57.402098+0000","last_peered":"2024-04-25T10:31:57.402098+0000","last_clean":"2024-04-25T10:31:57.402098+0000","last_became_active":"2024-04-25T10:31:55.033699+0000","last_became_peered":"2024-04-25T10:31:55.033699+0000","last_unstale":"2024-04-25T10:31:57.402098+0000","last_undegraded":"2024-04-25T10:31:57.402098+0000","last_fullsized":"2024-04-25T10:31:57.402098+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:21.826880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034213+0000","last_change":"2024-04-25T10:31:56.024832+0000","last_active":"2024-04-25T10:31:57.034213+0000","last_peered":"2024-04-25T10:31:57.034213+0000","last_clean":"2024-04-25T10:31:57.034213+0000","last_became_active":"2024-04-25T10:31:56.024562+0000","last_became_peered":"2024-04-25T10:31:56.024562+0000","last_unstale":"2024-04-25T10:31:57.034213+0000","last_undegraded":"2024-04-25T10:31:57.034213+0000","last_fullsized":"2024-04-25T10:31:57.034213+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:38.911913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"192'8","reported_seq":22,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.033962+0000","last_change":"2024-04-25T10:31:55.034513+0000","last_active":"2024-04-25T10:31:57.033962+0000","last_peered":"2024-04-25T10:31:57.033962+0000","last_clean":"2024-04-25T10:31:57.033962+0000","last_became_active":"2024-04-25T10:31:55.034161+0000","last_became_peered":"2024-04-25T10:31:55.034161+0000","last_unstale":"2024-04-25T10:31:57.033962+0000","last_undegraded":"2024-04-25T10:31:57.033962+0000","last_fullsized":"2024-04-25T10:31:57.033962+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.147622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402770+0000","last_change":"2024-04-25T10:31:56.031887+0000","last_active":"2024-04-25T10:31:57.402770+0000","last_peered":"2024-04-25T10:31:57.402770+0000","last_clean":"2024-04-25T10:31:57.402770+0000","last_became_active":"2024-04-25T10:31:56.031564+0000","last_became_peered":"2024-04-25T10:31:56.031564+0000","last_unstale":"2024-04-25T10:31:57.402770+0000","last_undegraded":"2024-04-25T10:31:57.402770+0000","last_fullsized":"2024-04-25T10:31:57.402770+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:50.816774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031530+0000","last_change":"2024-04-25T10:31:55.034357+0000","last_active":"2024-04-25T10:31:57.031530+0000","last_peered":"2024-04-25T10:31:57.031530+0000","last_clean":"2024-04-25T10:31:57.031530+0000","last_became_active":"2024-04-25T10:31:55.034084+0000","last_became_peered":"2024-04-25T10:31:55.034084+0000","last_unstale":"2024-04-25T10:31:57.031530+0000","last_undegraded":"2024-04-25T10:31:57.031530+0000","last_fullsized":"2024-04-25T10:31:57.031530+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:28.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036595+0000","last_change":"2024-04-25T10:31:56.025779+0000","last_active":"2024-04-25T10:31:57.036595+0000","last_peered":"2024-04-25T10:31:57.036595+0000","last_clean":"2024-04-25T10:31:57.036595+0000","last_became_active":"2024-04-25T10:31:56.025493+0000","last_became_peered":"2024-04-25T10:31:56.025493+0000","last_unstale":"2024-04-25T10:31:57.036595+0000","last_undegraded":"2024-04-25T10:31:57.036595+0000","last_fullsized":"2024-04-25T10:31:57.036595+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:21.926564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036580+0000","last_change":"2024-04-25T10:31:55.028670+0000","last_active":"2024-04-25T10:31:57.036580+0000","last_peered":"2024-04-25T10:31:57.036580+0000","last_clean":"2024-04-25T10:31:57.036580+0000","last_became_active":"2024-04-25T10:31:55.027986+0000","last_became_peered":"2024-04-25T10:31:55.027986+0000","last_unstale":"2024-04-25T10:31:57.036580+0000","last_undegraded":"2024-04-25T10:31:57.036580+0000","last_fullsized":"2024-04-25T10:31:57.036580+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:07.739012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034436+0000","last_change":"2024-04-25T10:31:56.036633+0000","last_active":"2024-04-25T10:31:57.034436+0000","last_peered":"2024-04-25T10:31:57.034436+0000","last_clean":"2024-04-25T10:31:57.034436+0000","last_became_active":"2024-04-25T10:31:56.036366+0000","last_became_peered":"2024-04-25T10:31:56.036366+0000","last_unstale":"2024-04-25T10:31:57.034436+0000","last_undegraded":"2024-04-25T10:31:57.034436+0000","last_fullsized":"2024-04-25T10:31:57.034436+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.498103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031071+0000","last_change":"2024-04-25T10:31:55.028330+0000","last_active":"2024-04-25T10:31:57.031071+0000","last_peered":"2024-04-25T10:31:57.031071+0000","last_clean":"2024-04-25T10:31:57.031071+0000","last_became_active":"2024-04-25T10:31:55.028050+0000","last_became_peered":"2024-04-25T10:31:55.028050+0000","last_unstale":"2024-04-25T10:31:57.031071+0000","last_undegraded":"2024-04-25T10:31:57.031071+0000","last_fullsized":"2024-04-25T10:31:57.031071+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:42.168210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399355+0000","last_change":"2024-04-25T10:31:56.042137+0000","last_active":"2024-04-25T10:31:57.399355+0000","last_peered":"2024-04-25T10:31:57.399355+0000","last_clean":"2024-04-25T10:31:57.399355+0000","last_became_active":"2024-04-25T10:31:56.041123+0000","last_became_peered":"2024-04-25T10:31:56.041123+0000","last_unstale":"2024-04-25T10:31:57.399355+0000","last_undegraded":"2024-04-25T10:31:57.399355+0000","last_fullsized":"2024-04-25T10:31:57.399355+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:25.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402500+0000","last_change":"2024-04-25T10:31:55.030692+0000","last_active":"2024-04-25T10:31:57.402500+0000","last_peered":"2024-04-25T10:31:57.402500+0000","last_clean":"2024-04-25T10:31:57.402500+0000","last_became_active":"2024-04-25T10:31:55.030401+0000","last_became_peered":"2024-04-25T10:31:55.030401+0000","last_unstale":"2024-04-25T10:31:57.402500+0000","last_undegraded":"2024-04-25T10:31:57.402500+0000","last_fullsized":"2024-04-25T10:31:57.402500+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:20.995626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036035+0000","last_change":"2024-04-25T10:31:56.034477+0000","last_active":"2024-04-25T10:31:57.036035+0000","last_peered":"2024-04-25T10:31:57.036035+0000","last_clean":"2024-04-25T10:31:57.036035+0000","last_became_active":"2024-04-25T10:31:56.033687+0000","last_became_peered":"2024-04-25T10:31:56.033687+0000","last_unstale":"2024-04-25T10:31:57.036035+0000","last_undegraded":"2024-04-25T10:31:57.036035+0000","last_fullsized":"2024-04-25T10:31:57.036035+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:21.871531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030761+0000","last_change":"2024-04-25T10:31:55.032688+0000","last_active":"2024-04-25T10:31:57.030761+0000","last_peered":"2024-04-25T10:31:57.030761+0000","last_clean":"2024-04-25T10:31:57.030761+0000","last_became_active":"2024-04-25T10:31:55.032452+0000","last_became_peered":"2024-04-25T10:31:55.032452+0000","last_unstale":"2024-04-25T10:31:57.030761+0000","last_undegraded":"2024-04-25T10:31:57.030761+0000","last_fullsized":"2024-04-25T10:31:57.030761+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:00.965601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399234+0000","last_change":"2024-04-25T10:31:56.052188+0000","last_active":"2024-04-25T10:31:57.399234+0000","last_peered":"2024-04-25T10:31:57.399234+0000","last_clean":"2024-04-25T10:31:57.399234+0000","last_became_active":"2024-04-25T10:31:56.051891+0000","last_became_peered":"2024-04-25T10:31:56.051891+0000","last_unstale":"2024-04-25T10:31:57.399234+0000","last_undegraded":"2024-04-25T10:31:57.399234+0000","last_fullsized":"2024-04-25T10:31:57.399234+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:16.395471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345252+0000","last_change":"2024-04-25T10:31:55.024672+0000","last_active":"2024-04-25T10:31:57.345252+0000","last_peered":"2024-04-25T10:31:57.345252+0000","last_clean":"2024-04-25T10:31:57.345252+0000","last_became_active":"2024-04-25T10:31:55.024385+0000","last_became_peered":"2024-04-25T10:31:55.024385+0000","last_unstale":"2024-04-25T10:31:57.345252+0000","last_undegraded":"2024-04-25T10:31:57.345252+0000","last_fullsized":"2024-04-25T10:31:57.345252+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:50.472352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402066+0000","last_change":"2024-04-25T10:31:56.036921+0000","last_active":"2024-04-25T10:31:57.402066+0000","last_peered":"2024-04-25T10:31:57.402066+0000","last_clean":"2024-04-25T10:31:57.402066+0000","last_became_active":"2024-04-25T10:31:56.036061+0000","last_became_peered":"2024-04-25T10:31:56.036061+0000","last_unstale":"2024-04-25T10:31:57.402066+0000","last_undegraded":"2024-04-25T10:31:57.402066+0000","last_fullsized":"2024-04-25T10:31:57.402066+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:32.434009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034336+0000","last_change":"2024-04-25T10:31:55.039698+0000","last_active":"2024-04-25T10:31:57.034336+0000","last_peered":"2024-04-25T10:31:57.034336+0000","last_clean":"2024-04-25T10:31:57.034336+0000","last_became_active":"2024-04-25T10:31:55.039360+0000","last_became_peered":"2024-04-25T10:31:55.039360+0000","last_unstale":"2024-04-25T10:31:57.034336+0000","last_undegraded":"2024-04-25T10:31:57.034336+0000","last_fullsized":"2024-04-25T10:31:57.034336+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:22.560691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403503+0000","last_change":"2024-04-25T10:31:56.040718+0000","last_active":"2024-04-25T10:31:57.403503+0000","last_peered":"2024-04-25T10:31:57.403503+0000","last_clean":"2024-04-25T10:31:57.403503+0000","last_became_active":"2024-04-25T10:31:56.040290+0000","last_became_peered":"2024-04-25T10:31:56.040290+0000","last_unstale":"2024-04-25T10:31:57.403503+0000","last_undegraded":"2024-04-25T10:31:57.403503+0000","last_fullsized":"2024-04-25T10:31:57.403503+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:20.449407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034063+0000","last_change":"2024-04-25T10:31:55.037702+0000","last_active":"2024-04-25T10:31:57.034063+0000","last_peered":"2024-04-25T10:31:57.034063+0000","last_clean":"2024-04-25T10:31:57.034063+0000","last_became_active":"2024-04-25T10:31:55.037430+0000","last_became_peered":"2024-04-25T10:31:55.037430+0000","last_unstale":"2024-04-25T10:31:57.034063+0000","last_undegraded":"2024-04-25T10:31:57.034063+0000","last_fullsized":"2024-04-25T10:31:57.034063+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.143165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036553+0000","last_change":"2024-04-25T10:31:56.028867+0000","last_active":"2024-04-25T10:31:57.036553+0000","last_peered":"2024-04-25T10:31:57.036553+0000","last_clean":"2024-04-25T10:31:57.036553+0000","last_became_active":"2024-04-25T10:31:56.028558+0000","last_became_peered":"2024-04-25T10:31:56.028558+0000","last_unstale":"2024-04-25T10:31:57.036553+0000","last_undegraded":"2024-04-25T10:31:57.036553+0000","last_fullsized":"2024-04-25T10:31:57.036553+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:09.225108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036618+0000","last_change":"2024-04-25T10:31:55.037572+0000","last_active":"2024-04-25T10:31:57.036618+0000","last_peered":"2024-04-25T10:31:57.036618+0000","last_clean":"2024-04-25T10:31:57.036618+0000","last_became_active":"2024-04-25T10:31:55.036683+0000","last_became_peered":"2024-04-25T10:31:55.036683+0000","last_unstale":"2024-04-25T10:31:57.036618+0000","last_undegraded":"2024-04-25T10:31:57.036618+0000","last_fullsized":"2024-04-25T10:31:57.036618+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:25.438605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031362+0000","last_change":"2024-04-25T10:31:56.040677+0000","last_active":"2024-04-25T10:31:57.031362+0000","last_peered":"2024-04-25T10:31:57.031362+0000","last_clean":"2024-04-25T10:31:57.031362+0000","last_became_active":"2024-04-25T10:31:56.039517+0000","last_became_peered":"2024-04-25T10:31:56.039517+0000","last_unstale":"2024-04-25T10:31:57.031362+0000","last_undegraded":"2024-04-25T10:31:57.031362+0000","last_fullsized":"2024-04-25T10:31:57.031362+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.528009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"193'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.033434+0000","last_change":"2024-04-25T10:31:55.026677+0000","last_active":"2024-04-25T10:31:57.033434+0000","last_peered":"2024-04-25T10:31:57.033434+0000","last_clean":"2024-04-25T10:31:57.033434+0000","last_became_active":"2024-04-25T10:31:55.026413+0000","last_became_peered":"2024-04-25T10:31:55.026413+0000","last_unstale":"2024-04-25T10:31:57.033434+0000","last_undegraded":"2024-04-25T10:31:57.033434+0000","last_fullsized":"2024-04-25T10:31:57.033434+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:02.982753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030633+0000","last_change":"2024-04-25T10:31:56.021872+0000","last_active":"2024-04-25T10:31:57.030633+0000","last_peered":"2024-04-25T10:31:57.030633+0000","last_clean":"2024-04-25T10:31:57.030633+0000","last_became_active":"2024-04-25T10:31:56.021320+0000","last_became_peered":"2024-04-25T10:31:56.021320+0000","last_unstale":"2024-04-25T10:31:57.030633+0000","last_undegraded":"2024-04-25T10:31:57.030633+0000","last_fullsized":"2024-04-25T10:31:57.030633+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.155270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399421+0000","last_change":"2024-04-25T10:31:55.035154+0000","last_active":"2024-04-25T10:31:57.399421+0000","last_peered":"2024-04-25T10:31:57.399421+0000","last_clean":"2024-04-25T10:31:57.399421+0000","last_became_active":"2024-04-25T10:31:55.034000+0000","last_became_peered":"2024-04-25T10:31:55.034000+0000","last_unstale":"2024-04-25T10:31:57.399421+0000","last_undegraded":"2024-04-25T10:31:57.399421+0000","last_fullsized":"2024-04-25T10:31:57.399421+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:19.441759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402367+0000","last_change":"2024-04-25T10:31:56.032695+0000","last_active":"2024-04-25T10:31:57.402367+0000","last_peered":"2024-04-25T10:31:57.402367+0000","last_clean":"2024-04-25T10:31:57.402367+0000","last_became_active":"2024-04-25T10:31:56.032425+0000","last_became_peered":"2024-04-25T10:31:56.032425+0000","last_unstale":"2024-04-25T10:31:57.402367+0000","last_undegraded":"2024-04-25T10:31:57.402367+0000","last_fullsized":"2024-04-25T10:31:57.402367+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.900836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034369+0000","last_change":"2024-04-25T10:31:55.035218+0000","last_active":"2024-04-25T10:31:57.034369+0000","last_peered":"2024-04-25T10:31:57.034369+0000","last_clean":"2024-04-25T10:31:57.034369+0000","last_became_active":"2024-04-25T10:31:55.034814+0000","last_became_peered":"2024-04-25T10:31:55.034814+0000","last_unstale":"2024-04-25T10:31:57.034369+0000","last_undegraded":"2024-04-25T10:31:57.034369+0000","last_fullsized":"2024-04-25T10:31:57.034369+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:43.952756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030322+0000","last_change":"2024-04-25T10:31:56.023799+0000","last_active":"2024-04-25T10:31:57.030322+0000","last_peered":"2024-04-25T10:31:57.030322+0000","last_clean":"2024-04-25T10:31:57.030322+0000","last_became_active":"2024-04-25T10:31:56.022199+0000","last_became_peered":"2024-04-25T10:31:56.022199+0000","last_unstale":"2024-04-25T10:31:57.030322+0000","last_undegraded":"2024-04-25T10:31:57.030322+0000","last_fullsized":"2024-04-25T10:31:57.030322+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:20.449407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034275+0000","last_change":"2024-04-25T10:31:55.036033+0000","last_active":"2024-04-25T10:31:57.034275+0000","last_peered":"2024-04-25T10:31:57.034275+0000","last_clean":"2024-04-25T10:31:57.034275+0000","last_became_active":"2024-04-25T10:31:55.035715+0000","last_became_peered":"2024-04-25T10:31:55.035715+0000","last_unstale":"2024-04-25T10:31:57.034275+0000","last_undegraded":"2024-04-25T10:31:57.034275+0000","last_fullsized":"2024-04-25T10:31:57.034275+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:02.716839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030626+0000","last_change":"2024-04-25T10:31:56.021795+0000","last_active":"2024-04-25T10:31:57.030626+0000","last_peered":"2024-04-25T10:31:57.030626+0000","last_clean":"2024-04-25T10:31:57.030626+0000","last_became_active":"2024-04-25T10:31:56.021464+0000","last_became_peered":"2024-04-25T10:31:56.021464+0000","last_unstale":"2024-04-25T10:31:57.030626+0000","last_undegraded":"2024-04-25T10:31:57.030626+0000","last_fullsized":"2024-04-25T10:31:57.030626+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:29.443748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403184+0000","last_change":"2024-04-25T10:31:55.036194+0000","last_active":"2024-04-25T10:31:57.403184+0000","last_peered":"2024-04-25T10:31:57.403184+0000","last_clean":"2024-04-25T10:31:57.403184+0000","last_became_active":"2024-04-25T10:31:55.035692+0000","last_became_peered":"2024-04-25T10:31:55.035692+0000","last_unstale":"2024-04-25T10:31:57.403184+0000","last_undegraded":"2024-04-25T10:31:57.403184+0000","last_fullsized":"2024-04-25T10:31:57.403184+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:08.416412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.033978+0000","last_change":"2024-04-25T10:31:56.035045+0000","last_active":"2024-04-25T10:31:57.033978+0000","last_peered":"2024-04-25T10:31:57.033978+0000","last_clean":"2024-04-25T10:31:57.033978+0000","last_became_active":"2024-04-25T10:31:56.034717+0000","last_became_peered":"2024-04-25T10:31:56.034717+0000","last_unstale":"2024-04-25T10:31:57.033978+0000","last_undegraded":"2024-04-25T10:31:57.033978+0000","last_fullsized":"2024-04-25T10:31:57.033978+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:09.424061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403509+0000","last_change":"2024-04-25T10:31:55.036736+0000","last_active":"2024-04-25T10:31:57.403509+0000","last_peered":"2024-04-25T10:31:57.403509+0000","last_clean":"2024-04-25T10:31:57.403509+0000","last_became_active":"2024-04-25T10:31:55.036431+0000","last_became_peered":"2024-04-25T10:31:55.036431+0000","last_unstale":"2024-04-25T10:31:57.403509+0000","last_undegraded":"2024-04-25T10:31:57.403509+0000","last_fullsized":"2024-04-25T10:31:57.403509+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:16.630558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399084+0000","last_change":"2024-04-25T10:31:56.025794+0000","last_active":"2024-04-25T10:31:57.399084+0000","last_peered":"2024-04-25T10:31:57.399084+0000","last_clean":"2024-04-25T10:31:57.399084+0000","last_became_active":"2024-04-25T10:31:56.024571+0000","last_became_peered":"2024-04-25T10:31:56.024571+0000","last_unstale":"2024-04-25T10:31:57.399084+0000","last_undegraded":"2024-04-25T10:31:57.399084+0000","last_fullsized":"2024-04-25T10:31:57.399084+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:36.603692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030134+0000","last_change":"2024-04-25T10:31:55.023947+0000","last_active":"2024-04-25T10:31:57.030134+0000","last_peered":"2024-04-25T10:31:57.030134+0000","last_clean":"2024-04-25T10:31:57.030134+0000","last_became_active":"2024-04-25T10:31:55.023673+0000","last_became_peered":"2024-04-25T10:31:55.023673+0000","last_unstale":"2024-04-25T10:31:57.030134+0000","last_undegraded":"2024-04-25T10:31:57.030134+0000","last_fullsized":"2024-04-25T10:31:57.030134+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.583722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399334+0000","last_change":"2024-04-25T10:31:56.035805+0000","last_active":"2024-04-25T10:31:57.399334+0000","last_peered":"2024-04-25T10:31:57.399334+0000","last_clean":"2024-04-25T10:31:57.399334+0000","last_became_active":"2024-04-25T10:31:56.035535+0000","last_became_peered":"2024-04-25T10:31:56.035535+0000","last_unstale":"2024-04-25T10:31:57.399334+0000","last_undegraded":"2024-04-25T10:31:57.399334+0000","last_fullsized":"2024-04-25T10:31:57.399334+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:24.777713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031056+0000","last_change":"2024-04-25T10:31:55.030797+0000","last_active":"2024-04-25T10:31:57.031056+0000","last_peered":"2024-04-25T10:31:57.031056+0000","last_clean":"2024-04-25T10:31:57.031056+0000","last_became_active":"2024-04-25T10:31:55.030452+0000","last_became_peered":"2024-04-25T10:31:55.030452+0000","last_unstale":"2024-04-25T10:31:57.031056+0000","last_undegraded":"2024-04-25T10:31:57.031056+0000","last_fullsized":"2024-04-25T10:31:57.031056+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:20.069780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":501,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030093+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:57.030093+0000","last_peered":"2024-04-25T10:31:57.030093+0000","last_clean":"2024-04-25T10:31:57.030093+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:57.030093+0000","last_undegraded":"2024-04-25T10:31:57.030093+0000","last_fullsized":"2024-04-25T10:31:57.030093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509613,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55772,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316068,"statfs":{"total":96636764160,"available":96579653632,"internally_reserved":0,"allocated":2437120,"data_stored":1068363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509614,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55772,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316068,"statfs":{"total":96636764160,"available":96579653632,"internally_reserved":0,"allocated":2441216,"data_stored":1067837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509613,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60516,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311324,"statfs":{"total":96636764160,"available":96574795776,"internally_reserved":0,"allocated":3039232,"data_stored":1659741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509613,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47440,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324400,"statfs":{"total":96636764160,"available":96588185600,"internally_reserved":0,"allocated":2428928,"data_stored":1067691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509613,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69592,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302248,"statfs":{"total":96636764160,"available":96565501952,"internally_reserved":0,"allocated":2433024,"data_stored":1067725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509614,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48032,"kb_used_data":2964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323808,"statfs":{"total":96636764160,"available":96587579392,"internally_reserved":0,"allocated":3035136,"data_stored":1658733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509614,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59928,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311912,"statfs":{"total":96636764160,"available":96575397888,"internally_reserved":0,"allocated":2428928,"data_stored":1067691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69592,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302248,"statfs":{"total":96636764160,"available":96565501952,"internally_reserved":0,"allocated":2428928,"data_stored":1063003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:32:00.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:32:00.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:00.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:00.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.326+0000 7fcda6200640 1 Processor -- start 2024-04-25T10:32:00.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.326+0000 7fcda6200640 1 -- start start 2024-04-25T10:32:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcda6200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015b9e0 0x7fcda00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcda6200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcda015bdc0 0x7fcda00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcda6200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda00ccfb0 con 0x7fcda00d1520 2024-04-25T10:32:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcda6200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00cd290 con 0x7fcda015bdc0 2024-04-25T10:32:00.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcda6200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00cd570 con 0x7fcda015b9e0 2024-04-25T10:32:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcda00d1520 0x7fcda00d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47166/0 (socket says 172.21.15.45:47166) 2024-04-25T10:32:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9fe00640 1 -- 172.21.15.45:0/2914234733 learned_addr learned my addr 172.21.15.45:0/2914234733 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9f400640 1 --2- 172.21.15.45:0/2914234733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015b9e0 0x7fcda00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd97e00640 1 --2- 172.21.15.45:0/2914234733 >> v2:172.21.15.117:3300/0 conn(0x7fcda015bdc0 0x7fcda00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9ea00640 1 -- 172.21.15.45:0/2914234733 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234886358 0 0) 0x7fcda00ccfb0 con 0x7fcda00d1520 2024-04-25T10:32:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9ea00640 1 -- 172.21.15.45:0/2914234733 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd84003530 con 0x7fcda00d1520 2024-04-25T10:32:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9f400640 1 -- 172.21.15.45:0/2914234733 >> v2:172.21.15.117:3300/0 conn(0x7fcda015bdc0 msgr2=0x7fcda00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9f400640 1 --2- 172.21.15.45:0/2914234733 >> v2:172.21.15.117:3300/0 conn(0x7fcda015bdc0 0x7fcda00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9f400640 1 -- 172.21.15.45:0/2914234733 >> v1:172.21.15.45:6789/0 conn(0x7fcda00d1520 legacy=0x7fcda00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.330+0000 7fcd9f400640 1 -- 172.21.15.45:0/2914234733 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00cd850 con 0x7fcda015b9e0 2024-04-25T10:32:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcd9f400640 1 --2- 172.21.15.45:0/2914234733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015b9e0 0x7fcda00cc410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd900097c0 tx=0x7fcd9001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=331730d522b1d446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcd9ea00640 1 -- 172.21.15.45:0/2914234733 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900b0070 con 0x7fcda015b9e0 2024-04-25T10:32:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcd9ea00640 1 -- 172.21.15.45:0/2914234733 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd900ab600 con 0x7fcda015b9e0 2024-04-25T10:32:00.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcd9ea00640 1 -- 172.21.15.45:0/2914234733 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900ab9b0 con 0x7fcda015b9e0 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 -- 172.21.15.45:0/2914234733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015b9e0 msgr2=0x7fcda00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 --2- 172.21.15.45:0/2914234733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015b9e0 0x7fcda00cc410 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd900097c0 tx=0x7fcd9001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 -- 172.21.15.45:0/2914234733 shutdown_connections 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 --2- 172.21.15.45:0/2914234733 >> v2:172.21.15.117:3300/0 conn(0x7fcda015bdc0 0x7fcda00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 --2- 172.21.15.45:0/2914234733 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015b9e0 0x7fcda00cc410 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 -- 172.21.15.45:0/2914234733 >> 172.21.15.45:0/2914234733 conn(0x7fcda009a9d0 msgr2=0x7fcda009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 -- 172.21.15.45:0/2914234733 shutdown_connections 2024-04-25T10:32:00.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 -- 172.21.15.45:0/2914234733 wait complete. 2024-04-25T10:32:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 Processor -- start 2024-04-25T10:32:00.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.334+0000 7fcda6200640 1 -- start start 2024-04-25T10:32:00.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 0x7fcda015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda015b9e0 0x7fcda00b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015bdc0 0x7fcda00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00c6630 con 0x7fcda00d1520 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00c64b0 con 0x7fcda015b9e0 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00c67b0 con 0x7fcda015bdc0 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 0x7fcda015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 0x7fcda015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35666/0 (socket says 172.21.15.45:35666) 2024-04-25T10:32:00.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 -- 172.21.15.45:0/243966308 learned_addr learned my addr 172.21.15.45:0/243966308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9fe00640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015bdc0 0x7fcda00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd97e00640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda015b9e0 0x7fcda00b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 -- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015bdc0 msgr2=0x7fcda00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015bdc0 0x7fcda00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 -- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda015b9e0 msgr2=0x7fcda00b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda015b9e0 0x7fcda00b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 -- 172.21.15.45:0/243966308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda015d4d0 con 0x7fcda00d1520 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd9f400640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 0x7fcda015a1a0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fcd90000c10 tx=0x7fcd90019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900b0040 con 0x7fcda00d1520 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd900c8b90 con 0x7fcda00d1520 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda00caa70 con 0x7fcda00d1520 2024-04-25T10:32:00.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90006e20 con 0x7fcda00d1520 2024-04-25T10:32:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda00cb900 con 0x7fcda00d1520 2024-04-25T10:32:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda004ef30 con 0x7fcda00d1520 2024-04-25T10:32:00.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.338+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd90003620 con 0x7fcda00d1520 2024-04-25T10:32:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.342+0000 7fcd97400640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd7c066f60 0x7fcd7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.342+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd9012e730 con 0x7fcda00d1520 2024-04-25T10:32:00.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.342+0000 7fcd97e00640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd7c066f60 0x7fcd7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.342+0000 7fcd97e00640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd7c066f60 0x7fcd7c069420 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcda006f860 tx=0x7fcd8800f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:00.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.346+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd900d0020 con 0x7fcda00d1520 2024-04-25T10:32:00.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.502+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcda00681f0 con 0x7fcd7c066f60 2024-04-25T10:32:00.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.510+0000 7fcd97400640 1 -- 172.21.15.45:0/243966308 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252490 (secure 0 0 0) 0x7fcd8805aa60 con 0x7fcd7c066f60 2024-04-25T10:32:00.515 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:00.517 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd7c066f60 msgr2=0x7fcd7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd7c066f60 0x7fcd7c069420 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcda006f860 tx=0x7fcd8800f040 comp rx=0 tx=0).stop 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 msgr2=0x7fcda015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 0x7fcda015a1a0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fcd90000c10 tx=0x7fcd90019240 comp rx=0 tx=0).stop 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 shutdown_connections 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcd7c066f60 0x7fcd7c069420 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcda015bdc0 0x7fcda00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda015b9e0 0x7fcda00b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 --2- 172.21.15.45:0/243966308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcda00d1520 0x7fcda015a1a0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 >> 172.21.15.45:0/243966308 conn(0x7fcda009a9d0 msgr2=0x7fcda00c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 shutdown_connections 2024-04-25T10:32:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.514+0000 7fcda6200640 1 -- 172.21.15.45:0/243966308 wait complete. 2024-04-25T10:32:00.537 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":612,"stamp":"2024-04-25T10:31:59.072339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466644,"kb_used_data":20188,"kb_used_omap":12,"kb_used_meta":446323,"kb_avail":754508076,"statfs":{"total":773094113280,"available":772616269824,"internally_reserved":0,"allocated":20672512,"data_stored":9720784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457035312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.064658"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031301+0000","last_change":"2024-04-25T10:31:56.023445+0000","last_active":"2024-04-25T10:31:57.031301+0000","last_peered":"2024-04-25T10:31:57.031301+0000","last_clean":"2024-04-25T10:31:57.031301+0000","last_became_active":"2024-04-25T10:31:56.021503+0000","last_became_peered":"2024-04-25T10:31:56.021503+0000","last_unstale":"2024-04-25T10:31:57.031301+0000","last_undegraded":"2024-04-25T10:31:57.031301+0000","last_fullsized":"2024-04-25T10:31:57.031301+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:44.960404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034506+0000","last_change":"2024-04-25T10:31:56.045175+0000","last_active":"2024-04-25T10:31:57.034506+0000","last_peered":"2024-04-25T10:31:57.034506+0000","last_clean":"2024-04-25T10:31:57.034506+0000","last_became_active":"2024-04-25T10:31:56.044863+0000","last_became_peered":"2024-04-25T10:31:56.044863+0000","last_unstale":"2024-04-25T10:31:57.034506+0000","last_undegraded":"2024-04-25T10:31:57.034506+0000","last_fullsized":"2024-04-25T10:31:57.034506+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:58.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034408+0000","last_change":"2024-04-25T10:31:56.044289+0000","last_active":"2024-04-25T10:31:57.034408+0000","last_peered":"2024-04-25T10:31:57.034408+0000","last_clean":"2024-04-25T10:31:57.034408+0000","last_became_active":"2024-04-25T10:31:56.042526+0000","last_became_peered":"2024-04-25T10:31:56.042526+0000","last_unstale":"2024-04-25T10:31:57.034408+0000","last_undegraded":"2024-04-25T10:31:57.034408+0000","last_fullsized":"2024-04-25T10:31:57.034408+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:24.777713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402513+0000","last_change":"2024-04-25T10:31:56.036426+0000","last_active":"2024-04-25T10:31:57.402513+0000","last_peered":"2024-04-25T10:31:57.402513+0000","last_clean":"2024-04-25T10:31:57.402513+0000","last_became_active":"2024-04-25T10:31:56.036045+0000","last_became_peered":"2024-04-25T10:31:56.036045+0000","last_unstale":"2024-04-25T10:31:57.402513+0000","last_undegraded":"2024-04-25T10:31:57.402513+0000","last_fullsized":"2024-04-25T10:31:57.402513+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:33.363015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402550+0000","last_change":"2024-04-25T10:31:56.025083+0000","last_active":"2024-04-25T10:31:57.402550+0000","last_peered":"2024-04-25T10:31:57.402550+0000","last_clean":"2024-04-25T10:31:57.402550+0000","last_became_active":"2024-04-25T10:31:56.024811+0000","last_became_peered":"2024-04-25T10:31:56.024811+0000","last_unstale":"2024-04-25T10:31:57.402550+0000","last_undegraded":"2024-04-25T10:31:57.402550+0000","last_fullsized":"2024-04-25T10:31:57.402550+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:12.024481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.401991+0000","last_change":"2024-04-25T10:31:56.049902+0000","last_active":"2024-04-25T10:31:57.401991+0000","last_peered":"2024-04-25T10:31:57.401991+0000","last_clean":"2024-04-25T10:31:57.401991+0000","last_became_active":"2024-04-25T10:31:56.049278+0000","last_became_peered":"2024-04-25T10:31:56.049278+0000","last_unstale":"2024-04-25T10:31:57.401991+0000","last_undegraded":"2024-04-25T10:31:57.401991+0000","last_fullsized":"2024-04-25T10:31:57.401991+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:08.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034048+0000","last_change":"2024-04-25T10:31:56.034478+0000","last_active":"2024-04-25T10:31:57.034048+0000","last_peered":"2024-04-25T10:31:57.034048+0000","last_clean":"2024-04-25T10:31:57.034048+0000","last_became_active":"2024-04-25T10:31:56.034174+0000","last_became_peered":"2024-04-25T10:31:56.034174+0000","last_unstale":"2024-04-25T10:31:57.034048+0000","last_undegraded":"2024-04-25T10:31:57.034048+0000","last_fullsized":"2024-04-25T10:31:57.034048+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:12.836323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403597+0000","last_change":"2024-04-25T10:31:56.037618+0000","last_active":"2024-04-25T10:31:57.403597+0000","last_peered":"2024-04-25T10:31:57.403597+0000","last_clean":"2024-04-25T10:31:57.403597+0000","last_became_active":"2024-04-25T10:31:56.037024+0000","last_became_peered":"2024-04-25T10:31:56.037024+0000","last_unstale":"2024-04-25T10:31:57.403597+0000","last_undegraded":"2024-04-25T10:31:57.403597+0000","last_fullsized":"2024-04-25T10:31:57.403597+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:29.443748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399550+0000","last_change":"2024-04-25T10:31:56.044144+0000","last_active":"2024-04-25T10:31:57.399550+0000","last_peered":"2024-04-25T10:31:57.399550+0000","last_clean":"2024-04-25T10:31:57.399550+0000","last_became_active":"2024-04-25T10:31:56.043894+0000","last_became_peered":"2024-04-25T10:31:56.043894+0000","last_unstale":"2024-04-25T10:31:57.399550+0000","last_undegraded":"2024-04-25T10:31:57.399550+0000","last_fullsized":"2024-04-25T10:31:57.399550+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:58.910927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402674+0000","last_change":"2024-04-25T10:31:56.034797+0000","last_active":"2024-04-25T10:31:57.402674+0000","last_peered":"2024-04-25T10:31:57.402674+0000","last_clean":"2024-04-25T10:31:57.402674+0000","last_became_active":"2024-04-25T10:31:56.033015+0000","last_became_peered":"2024-04-25T10:31:56.033015+0000","last_unstale":"2024-04-25T10:31:57.402674+0000","last_undegraded":"2024-04-25T10:31:57.402674+0000","last_fullsized":"2024-04-25T10:31:57.402674+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:21.607757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034507+0000","last_change":"2024-04-25T10:31:56.040282+0000","last_active":"2024-04-25T10:31:57.034507+0000","last_peered":"2024-04-25T10:31:57.034507+0000","last_clean":"2024-04-25T10:31:57.034507+0000","last_became_active":"2024-04-25T10:31:56.037189+0000","last_became_peered":"2024-04-25T10:31:56.037189+0000","last_unstale":"2024-04-25T10:31:57.034507+0000","last_undegraded":"2024-04-25T10:31:57.034507+0000","last_fullsized":"2024-04-25T10:31:57.034507+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:31.440673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345250+0000","last_change":"2024-04-25T10:31:56.031240+0000","last_active":"2024-04-25T10:31:57.345250+0000","last_peered":"2024-04-25T10:31:57.345250+0000","last_clean":"2024-04-25T10:31:57.345250+0000","last_became_active":"2024-04-25T10:31:56.030869+0000","last_became_peered":"2024-04-25T10:31:56.030869+0000","last_unstale":"2024-04-25T10:31:57.345250+0000","last_undegraded":"2024-04-25T10:31:57.345250+0000","last_fullsized":"2024-04-25T10:31:57.345250+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:01.720443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030638+0000","last_change":"2024-04-25T10:31:56.023597+0000","last_active":"2024-04-25T10:31:57.030638+0000","last_peered":"2024-04-25T10:31:57.030638+0000","last_clean":"2024-04-25T10:31:57.030638+0000","last_became_active":"2024-04-25T10:31:56.022859+0000","last_became_peered":"2024-04-25T10:31:56.022859+0000","last_unstale":"2024-04-25T10:31:57.030638+0000","last_undegraded":"2024-04-25T10:31:57.030638+0000","last_fullsized":"2024-04-25T10:31:57.030638+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:03.724487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034141+0000","last_change":"2024-04-25T10:31:56.026046+0000","last_active":"2024-04-25T10:31:57.034141+0000","last_peered":"2024-04-25T10:31:57.034141+0000","last_clean":"2024-04-25T10:31:57.034141+0000","last_became_active":"2024-04-25T10:31:56.025760+0000","last_became_peered":"2024-04-25T10:31:56.025760+0000","last_unstale":"2024-04-25T10:31:57.034141+0000","last_undegraded":"2024-04-25T10:31:57.034141+0000","last_fullsized":"2024-04-25T10:31:57.034141+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:36.603692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402687+0000","last_change":"2024-04-25T10:31:56.034778+0000","last_active":"2024-04-25T10:31:57.402687+0000","last_peered":"2024-04-25T10:31:57.402687+0000","last_clean":"2024-04-25T10:31:57.402687+0000","last_became_active":"2024-04-25T10:31:56.034515+0000","last_became_peered":"2024-04-25T10:31:56.034515+0000","last_unstale":"2024-04-25T10:31:57.402687+0000","last_undegraded":"2024-04-25T10:31:57.402687+0000","last_fullsized":"2024-04-25T10:31:57.402687+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:52.394068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399054+0000","last_change":"2024-04-25T10:31:56.040209+0000","last_active":"2024-04-25T10:31:57.399054+0000","last_peered":"2024-04-25T10:31:57.399054+0000","last_clean":"2024-04-25T10:31:57.399054+0000","last_became_active":"2024-04-25T10:31:56.034440+0000","last_became_peered":"2024-04-25T10:31:56.034440+0000","last_unstale":"2024-04-25T10:31:57.399054+0000","last_undegraded":"2024-04-25T10:31:57.399054+0000","last_fullsized":"2024-04-25T10:31:57.399054+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:09.424061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031484+0000","last_change":"2024-04-25T10:31:56.030156+0000","last_active":"2024-04-25T10:31:57.031484+0000","last_peered":"2024-04-25T10:31:57.031484+0000","last_clean":"2024-04-25T10:31:57.031484+0000","last_became_active":"2024-04-25T10:31:56.029879+0000","last_became_peered":"2024-04-25T10:31:56.029879+0000","last_unstale":"2024-04-25T10:31:57.031484+0000","last_undegraded":"2024-04-25T10:31:57.031484+0000","last_fullsized":"2024-04-25T10:31:57.031484+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:01.093217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036497+0000","last_change":"2024-04-25T10:31:56.030778+0000","last_active":"2024-04-25T10:31:57.036497+0000","last_peered":"2024-04-25T10:31:57.036497+0000","last_clean":"2024-04-25T10:31:57.036497+0000","last_became_active":"2024-04-25T10:31:56.030499+0000","last_became_peered":"2024-04-25T10:31:56.030499+0000","last_unstale":"2024-04-25T10:31:57.036497+0000","last_undegraded":"2024-04-25T10:31:57.036497+0000","last_fullsized":"2024-04-25T10:31:57.036497+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:11.830077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403048+0000","last_change":"2024-04-25T10:31:56.036430+0000","last_active":"2024-04-25T10:31:57.403048+0000","last_peered":"2024-04-25T10:31:57.403048+0000","last_clean":"2024-04-25T10:31:57.403048+0000","last_became_active":"2024-04-25T10:31:56.036034+0000","last_became_peered":"2024-04-25T10:31:56.036034+0000","last_unstale":"2024-04-25T10:31:57.403048+0000","last_undegraded":"2024-04-25T10:31:57.403048+0000","last_fullsized":"2024-04-25T10:31:57.403048+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:41.664987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036311+0000","last_change":"2024-04-25T10:31:56.037514+0000","last_active":"2024-04-25T10:31:57.036311+0000","last_peered":"2024-04-25T10:31:57.036311+0000","last_clean":"2024-04-25T10:31:57.036311+0000","last_became_active":"2024-04-25T10:31:56.037130+0000","last_became_peered":"2024-04-25T10:31:56.037130+0000","last_unstale":"2024-04-25T10:31:57.036311+0000","last_undegraded":"2024-04-25T10:31:57.036311+0000","last_fullsized":"2024-04-25T10:31:57.036311+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:15.835205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036270+0000","last_change":"2024-04-25T10:31:56.028998+0000","last_active":"2024-04-25T10:31:57.036270+0000","last_peered":"2024-04-25T10:31:57.036270+0000","last_clean":"2024-04-25T10:31:57.036270+0000","last_became_active":"2024-04-25T10:31:56.028660+0000","last_became_peered":"2024-04-25T10:31:56.028660+0000","last_unstale":"2024-04-25T10:31:57.036270+0000","last_undegraded":"2024-04-25T10:31:57.036270+0000","last_fullsized":"2024-04-25T10:31:57.036270+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:51.698936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.401976+0000","last_change":"2024-04-25T10:31:56.049210+0000","last_active":"2024-04-25T10:31:57.401976+0000","last_peered":"2024-04-25T10:31:57.401976+0000","last_clean":"2024-04-25T10:31:57.401976+0000","last_became_active":"2024-04-25T10:31:56.048931+0000","last_became_peered":"2024-04-25T10:31:56.048931+0000","last_unstale":"2024-04-25T10:31:57.401976+0000","last_undegraded":"2024-04-25T10:31:57.401976+0000","last_fullsized":"2024-04-25T10:31:57.401976+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:01.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030130+0000","last_change":"2024-04-25T10:31:56.033948+0000","last_active":"2024-04-25T10:31:57.030130+0000","last_peered":"2024-04-25T10:31:57.030130+0000","last_clean":"2024-04-25T10:31:57.030130+0000","last_became_active":"2024-04-25T10:31:56.032652+0000","last_became_peered":"2024-04-25T10:31:56.032652+0000","last_unstale":"2024-04-25T10:31:57.030130+0000","last_undegraded":"2024-04-25T10:31:57.030130+0000","last_fullsized":"2024-04-25T10:31:57.030130+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:48.150813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402173+0000","last_change":"2024-04-25T10:31:56.032747+0000","last_active":"2024-04-25T10:31:57.402173+0000","last_peered":"2024-04-25T10:31:57.402173+0000","last_clean":"2024-04-25T10:31:57.402173+0000","last_became_active":"2024-04-25T10:31:56.032514+0000","last_became_peered":"2024-04-25T10:31:56.032514+0000","last_unstale":"2024-04-25T10:31:57.402173+0000","last_undegraded":"2024-04-25T10:31:57.402173+0000","last_fullsized":"2024-04-25T10:31:57.402173+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:15.668277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034593+0000","last_change":"2024-04-25T10:31:56.043336+0000","last_active":"2024-04-25T10:31:57.034593+0000","last_peered":"2024-04-25T10:31:57.034593+0000","last_clean":"2024-04-25T10:31:57.034593+0000","last_became_active":"2024-04-25T10:31:56.042781+0000","last_became_peered":"2024-04-25T10:31:56.042781+0000","last_unstale":"2024-04-25T10:31:57.034593+0000","last_undegraded":"2024-04-25T10:31:57.034593+0000","last_fullsized":"2024-04-25T10:31:57.034593+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:01.191198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403145+0000","last_change":"2024-04-25T10:31:56.039511+0000","last_active":"2024-04-25T10:31:57.403145+0000","last_peered":"2024-04-25T10:31:57.403145+0000","last_clean":"2024-04-25T10:31:57.403145+0000","last_became_active":"2024-04-25T10:31:56.039235+0000","last_became_peered":"2024-04-25T10:31:56.039235+0000","last_unstale":"2024-04-25T10:31:57.403145+0000","last_undegraded":"2024-04-25T10:31:57.403145+0000","last_fullsized":"2024-04-25T10:31:57.403145+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:47.429622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034419+0000","last_change":"2024-04-25T10:31:56.037447+0000","last_active":"2024-04-25T10:31:57.034419+0000","last_peered":"2024-04-25T10:31:57.034419+0000","last_clean":"2024-04-25T10:31:57.034419+0000","last_became_active":"2024-04-25T10:31:56.036915+0000","last_became_peered":"2024-04-25T10:31:56.036915+0000","last_unstale":"2024-04-25T10:31:57.034419+0000","last_undegraded":"2024-04-25T10:31:57.034419+0000","last_fullsized":"2024-04-25T10:31:57.034419+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:25.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403672+0000","last_change":"2024-04-25T10:31:56.037048+0000","last_active":"2024-04-25T10:31:57.403672+0000","last_peered":"2024-04-25T10:31:57.403672+0000","last_clean":"2024-04-25T10:31:57.403672+0000","last_became_active":"2024-04-25T10:31:56.036739+0000","last_became_peered":"2024-04-25T10:31:56.036739+0000","last_unstale":"2024-04-25T10:31:57.403672+0000","last_undegraded":"2024-04-25T10:31:57.403672+0000","last_fullsized":"2024-04-25T10:31:57.403672+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:03.724487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399446+0000","last_change":"2024-04-25T10:31:56.052285+0000","last_active":"2024-04-25T10:31:57.399446+0000","last_peered":"2024-04-25T10:31:57.399446+0000","last_clean":"2024-04-25T10:31:57.399446+0000","last_became_active":"2024-04-25T10:31:56.052005+0000","last_became_peered":"2024-04-25T10:31:56.052005+0000","last_unstale":"2024-04-25T10:31:57.399446+0000","last_undegraded":"2024-04-25T10:31:57.399446+0000","last_fullsized":"2024-04-25T10:31:57.399446+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.498103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345087+0000","last_change":"2024-04-25T10:31:56.045203+0000","last_active":"2024-04-25T10:31:57.345087+0000","last_peered":"2024-04-25T10:31:57.345087+0000","last_clean":"2024-04-25T10:31:57.345087+0000","last_became_active":"2024-04-25T10:31:56.044891+0000","last_became_peered":"2024-04-25T10:31:56.044891+0000","last_unstale":"2024-04-25T10:31:57.345087+0000","last_undegraded":"2024-04-25T10:31:57.345087+0000","last_fullsized":"2024-04-25T10:31:57.345087+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:19.694665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.035967+0000","last_change":"2024-04-25T10:31:56.033978+0000","last_active":"2024-04-25T10:31:57.035967+0000","last_peered":"2024-04-25T10:31:57.035967+0000","last_clean":"2024-04-25T10:31:57.035967+0000","last_became_active":"2024-04-25T10:31:56.033528+0000","last_became_peered":"2024-04-25T10:31:56.033528+0000","last_unstale":"2024-04-25T10:31:57.035967+0000","last_undegraded":"2024-04-25T10:31:57.035967+0000","last_fullsized":"2024-04-25T10:31:57.035967+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:07.112556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403637+0000","last_change":"2024-04-25T10:31:56.040924+0000","last_active":"2024-04-25T10:31:57.403637+0000","last_peered":"2024-04-25T10:31:57.403637+0000","last_clean":"2024-04-25T10:31:57.403637+0000","last_became_active":"2024-04-25T10:31:56.040568+0000","last_became_peered":"2024-04-25T10:31:56.040568+0000","last_unstale":"2024-04-25T10:31:57.403637+0000","last_undegraded":"2024-04-25T10:31:57.403637+0000","last_fullsized":"2024-04-25T10:31:57.403637+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.155270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034648+0000","last_change":"2024-04-25T10:31:56.045694+0000","last_active":"2024-04-25T10:31:57.034648+0000","last_peered":"2024-04-25T10:31:57.034648+0000","last_clean":"2024-04-25T10:31:57.034648+0000","last_became_active":"2024-04-25T10:31:56.045417+0000","last_became_peered":"2024-04-25T10:31:56.045417+0000","last_unstale":"2024-04-25T10:31:57.034648+0000","last_undegraded":"2024-04-25T10:31:57.034648+0000","last_fullsized":"2024-04-25T10:31:57.034648+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:08.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1f","version":"192'4","reported_seq":18,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402663+0000","last_change":"2024-04-25T10:31:55.039969+0000","last_active":"2024-04-25T10:31:57.402663+0000","last_peered":"2024-04-25T10:31:57.402663+0000","last_clean":"2024-04-25T10:31:57.402663+0000","last_became_active":"2024-04-25T10:31:55.039769+0000","last_became_peered":"2024-04-25T10:31:55.039769+0000","last_unstale":"2024-04-25T10:31:57.402663+0000","last_undegraded":"2024-04-25T10:31:57.402663+0000","last_fullsized":"2024-04-25T10:31:57.402663+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:46.497160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402546+0000","last_change":"2024-04-25T10:31:56.045843+0000","last_active":"2024-04-25T10:31:57.402546+0000","last_peered":"2024-04-25T10:31:57.402546+0000","last_clean":"2024-04-25T10:31:57.402546+0000","last_became_active":"2024-04-25T10:31:56.045532+0000","last_became_peered":"2024-04-25T10:31:56.045532+0000","last_unstale":"2024-04-25T10:31:57.402546+0000","last_undegraded":"2024-04-25T10:31:57.402546+0000","last_fullsized":"2024-04-25T10:31:57.402546+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:54.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031234+0000","last_change":"2024-04-25T10:31:55.023720+0000","last_active":"2024-04-25T10:31:57.031234+0000","last_peered":"2024-04-25T10:31:57.031234+0000","last_clean":"2024-04-25T10:31:57.031234+0000","last_became_active":"2024-04-25T10:31:55.023387+0000","last_became_peered":"2024-04-25T10:31:55.023387+0000","last_unstale":"2024-04-25T10:31:57.031234+0000","last_undegraded":"2024-04-25T10:31:57.031234+0000","last_fullsized":"2024-04-25T10:31:57.031234+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:19.325725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399322+0000","last_change":"2024-04-25T10:31:56.042149+0000","last_active":"2024-04-25T10:31:57.399322+0000","last_peered":"2024-04-25T10:31:57.399322+0000","last_clean":"2024-04-25T10:31:57.399322+0000","last_became_active":"2024-04-25T10:31:56.041840+0000","last_became_peered":"2024-04-25T10:31:56.041840+0000","last_unstale":"2024-04-25T10:31:57.399322+0000","last_undegraded":"2024-04-25T10:31:57.399322+0000","last_fullsized":"2024-04-25T10:31:57.399322+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:38.911913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345384+0000","last_change":"2024-04-25T10:31:55.030296+0000","last_active":"2024-04-25T10:31:57.345384+0000","last_peered":"2024-04-25T10:31:57.345384+0000","last_clean":"2024-04-25T10:31:57.345384+0000","last_became_active":"2024-04-25T10:31:55.029941+0000","last_became_peered":"2024-04-25T10:31:55.029941+0000","last_unstale":"2024-04-25T10:31:57.345384+0000","last_undegraded":"2024-04-25T10:31:57.345384+0000","last_fullsized":"2024-04-25T10:31:57.345384+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:13.283029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034145+0000","last_change":"2024-04-25T10:31:56.040297+0000","last_active":"2024-04-25T10:31:57.034145+0000","last_peered":"2024-04-25T10:31:57.034145+0000","last_clean":"2024-04-25T10:31:57.034145+0000","last_became_active":"2024-04-25T10:31:56.033706+0000","last_became_peered":"2024-04-25T10:31:56.033706+0000","last_unstale":"2024-04-25T10:31:57.034145+0000","last_undegraded":"2024-04-25T10:31:57.034145+0000","last_fullsized":"2024-04-25T10:31:57.034145+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:16.395471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402661+0000","last_change":"2024-04-25T10:31:55.033280+0000","last_active":"2024-04-25T10:31:57.402661+0000","last_peered":"2024-04-25T10:31:57.402661+0000","last_clean":"2024-04-25T10:31:57.402661+0000","last_became_active":"2024-04-25T10:31:55.032990+0000","last_became_peered":"2024-04-25T10:31:55.032990+0000","last_unstale":"2024-04-25T10:31:57.402661+0000","last_undegraded":"2024-04-25T10:31:57.402661+0000","last_fullsized":"2024-04-25T10:31:57.402661+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:33.336788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402465+0000","last_change":"2024-04-25T10:31:56.045503+0000","last_active":"2024-04-25T10:31:57.402465+0000","last_peered":"2024-04-25T10:31:57.402465+0000","last_clean":"2024-04-25T10:31:57.402465+0000","last_became_active":"2024-04-25T10:31:56.044565+0000","last_became_peered":"2024-04-25T10:31:56.044565+0000","last_unstale":"2024-04-25T10:31:57.402465+0000","last_undegraded":"2024-04-25T10:31:57.402465+0000","last_fullsized":"2024-04-25T10:31:57.402465+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:40.920342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030355+0000","last_change":"2024-04-25T10:31:55.037216+0000","last_active":"2024-04-25T10:31:57.030355+0000","last_peered":"2024-04-25T10:31:57.030355+0000","last_clean":"2024-04-25T10:31:57.030355+0000","last_became_active":"2024-04-25T10:31:55.036643+0000","last_became_peered":"2024-04-25T10:31:55.036643+0000","last_unstale":"2024-04-25T10:31:57.030355+0000","last_undegraded":"2024-04-25T10:31:57.030355+0000","last_fullsized":"2024-04-25T10:31:57.030355+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:16.585281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402052+0000","last_change":"2024-04-25T10:31:56.045099+0000","last_active":"2024-04-25T10:31:57.402052+0000","last_peered":"2024-04-25T10:31:57.402052+0000","last_clean":"2024-04-25T10:31:57.402052+0000","last_became_active":"2024-04-25T10:31:56.044823+0000","last_became_peered":"2024-04-25T10:31:56.044823+0000","last_unstale":"2024-04-25T10:31:57.402052+0000","last_undegraded":"2024-04-25T10:31:57.402052+0000","last_fullsized":"2024-04-25T10:31:57.402052+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:26.446254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399076+0000","last_change":"2024-04-25T10:31:55.031406+0000","last_active":"2024-04-25T10:31:57.399076+0000","last_peered":"2024-04-25T10:31:57.399076+0000","last_clean":"2024-04-25T10:31:57.399076+0000","last_became_active":"2024-04-25T10:31:55.031130+0000","last_became_peered":"2024-04-25T10:31:55.031130+0000","last_unstale":"2024-04-25T10:31:57.399076+0000","last_undegraded":"2024-04-25T10:31:57.399076+0000","last_fullsized":"2024-04-25T10:31:57.399076+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:28.436100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030191+0000","last_change":"2024-04-25T10:31:56.039933+0000","last_active":"2024-04-25T10:31:57.030191+0000","last_peered":"2024-04-25T10:31:57.030191+0000","last_clean":"2024-04-25T10:31:57.030191+0000","last_became_active":"2024-04-25T10:31:56.038974+0000","last_became_peered":"2024-04-25T10:31:56.038974+0000","last_unstale":"2024-04-25T10:31:57.030191+0000","last_undegraded":"2024-04-25T10:31:57.030191+0000","last_fullsized":"2024-04-25T10:31:57.030191+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:31.153275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.035905+0000","last_change":"2024-04-25T10:31:55.032340+0000","last_active":"2024-04-25T10:31:57.035905+0000","last_peered":"2024-04-25T10:31:57.035905+0000","last_clean":"2024-04-25T10:31:57.035905+0000","last_became_active":"2024-04-25T10:31:55.032044+0000","last_became_peered":"2024-04-25T10:31:55.032044+0000","last_unstale":"2024-04-25T10:31:57.035905+0000","last_undegraded":"2024-04-25T10:31:57.035905+0000","last_fullsized":"2024-04-25T10:31:57.035905+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:00.183550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345313+0000","last_change":"2024-04-25T10:31:56.044302+0000","last_active":"2024-04-25T10:31:57.345313+0000","last_peered":"2024-04-25T10:31:57.345313+0000","last_clean":"2024-04-25T10:31:57.345313+0000","last_became_active":"2024-04-25T10:31:56.044051+0000","last_became_peered":"2024-04-25T10:31:56.044051+0000","last_unstale":"2024-04-25T10:31:57.345313+0000","last_undegraded":"2024-04-25T10:31:57.345313+0000","last_fullsized":"2024-04-25T10:31:57.345313+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:51.685428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031448+0000","last_change":"2024-04-25T10:31:55.035554+0000","last_active":"2024-04-25T10:31:57.031448+0000","last_peered":"2024-04-25T10:31:57.031448+0000","last_clean":"2024-04-25T10:31:57.031448+0000","last_became_active":"2024-04-25T10:31:55.034702+0000","last_became_peered":"2024-04-25T10:31:55.034702+0000","last_unstale":"2024-04-25T10:31:57.031448+0000","last_undegraded":"2024-04-25T10:31:57.031448+0000","last_fullsized":"2024-04-25T10:31:57.031448+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:10.580399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034702+0000","last_change":"2024-04-25T10:31:56.043084+0000","last_active":"2024-04-25T10:31:57.034702+0000","last_peered":"2024-04-25T10:31:57.034702+0000","last_clean":"2024-04-25T10:31:57.034702+0000","last_became_active":"2024-04-25T10:31:56.042460+0000","last_became_peered":"2024-04-25T10:31:56.042460+0000","last_unstale":"2024-04-25T10:31:57.034702+0000","last_undegraded":"2024-04-25T10:31:57.034702+0000","last_fullsized":"2024-04-25T10:31:57.034702+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:26.446254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402765+0000","last_change":"2024-04-25T10:31:55.031555+0000","last_active":"2024-04-25T10:31:57.402765+0000","last_peered":"2024-04-25T10:31:57.402765+0000","last_clean":"2024-04-25T10:31:57.402765+0000","last_became_active":"2024-04-25T10:31:55.030013+0000","last_became_peered":"2024-04-25T10:31:55.030013+0000","last_unstale":"2024-04-25T10:31:57.402765+0000","last_undegraded":"2024-04-25T10:31:57.402765+0000","last_fullsized":"2024-04-25T10:31:57.402765+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:52.569927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345380+0000","last_change":"2024-04-25T10:31:56.035415+0000","last_active":"2024-04-25T10:31:57.345380+0000","last_peered":"2024-04-25T10:31:57.345380+0000","last_clean":"2024-04-25T10:31:57.345380+0000","last_became_active":"2024-04-25T10:31:56.034925+0000","last_became_peered":"2024-04-25T10:31:56.034925+0000","last_unstale":"2024-04-25T10:31:57.345380+0000","last_undegraded":"2024-04-25T10:31:57.345380+0000","last_fullsized":"2024-04-25T10:31:57.345380+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:47.638947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"192'3","reported_seq":17,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403595+0000","last_change":"2024-04-25T10:31:55.033297+0000","last_active":"2024-04-25T10:31:57.403595+0000","last_peered":"2024-04-25T10:31:57.403595+0000","last_clean":"2024-04-25T10:31:57.403595+0000","last_became_active":"2024-04-25T10:31:55.032807+0000","last_became_peered":"2024-04-25T10:31:55.032807+0000","last_unstale":"2024-04-25T10:31:57.403595+0000","last_undegraded":"2024-04-25T10:31:57.403595+0000","last_fullsized":"2024-04-25T10:31:57.403595+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.828603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345271+0000","last_change":"2024-04-25T10:31:56.044048+0000","last_active":"2024-04-25T10:31:57.345271+0000","last_peered":"2024-04-25T10:31:57.345271+0000","last_clean":"2024-04-25T10:31:57.345271+0000","last_became_active":"2024-04-25T10:31:56.043694+0000","last_became_peered":"2024-04-25T10:31:56.043694+0000","last_unstale":"2024-04-25T10:31:57.345271+0000","last_undegraded":"2024-04-25T10:31:57.345271+0000","last_fullsized":"2024-04-25T10:31:57.345271+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:51.139081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"192'6","reported_seq":20,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036342+0000","last_change":"2024-04-25T10:31:55.025577+0000","last_active":"2024-04-25T10:31:57.036342+0000","last_peered":"2024-04-25T10:31:57.036342+0000","last_clean":"2024-04-25T10:31:57.036342+0000","last_became_active":"2024-04-25T10:31:55.025302+0000","last_became_peered":"2024-04-25T10:31:55.025302+0000","last_unstale":"2024-04-25T10:31:57.036342+0000","last_undegraded":"2024-04-25T10:31:57.036342+0000","last_fullsized":"2024-04-25T10:31:57.036342+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:30.433025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402142+0000","last_change":"2024-04-25T10:31:56.037644+0000","last_active":"2024-04-25T10:31:57.402142+0000","last_peered":"2024-04-25T10:31:57.402142+0000","last_clean":"2024-04-25T10:31:57.402142+0000","last_became_active":"2024-04-25T10:31:56.037368+0000","last_became_peered":"2024-04-25T10:31:56.037368+0000","last_unstale":"2024-04-25T10:31:57.402142+0000","last_undegraded":"2024-04-25T10:31:57.402142+0000","last_fullsized":"2024-04-25T10:31:57.402142+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:23.827566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403708+0000","last_change":"2024-04-25T10:31:55.037212+0000","last_active":"2024-04-25T10:31:57.403708+0000","last_peered":"2024-04-25T10:31:57.403708+0000","last_clean":"2024-04-25T10:31:57.403708+0000","last_became_active":"2024-04-25T10:31:55.036803+0000","last_became_peered":"2024-04-25T10:31:55.036803+0000","last_unstale":"2024-04-25T10:31:57.403708+0000","last_undegraded":"2024-04-25T10:31:57.403708+0000","last_fullsized":"2024-04-25T10:31:57.403708+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:26.167872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345225+0000","last_change":"2024-04-25T10:31:56.034558+0000","last_active":"2024-04-25T10:31:57.345225+0000","last_peered":"2024-04-25T10:31:57.345225+0000","last_clean":"2024-04-25T10:31:57.345225+0000","last_became_active":"2024-04-25T10:31:56.034244+0000","last_became_peered":"2024-04-25T10:31:56.034244+0000","last_unstale":"2024-04-25T10:31:57.345225+0000","last_undegraded":"2024-04-25T10:31:57.345225+0000","last_fullsized":"2024-04-25T10:31:57.345225+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:22.859699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402250+0000","last_change":"2024-04-25T10:31:55.037875+0000","last_active":"2024-04-25T10:31:57.402250+0000","last_peered":"2024-04-25T10:31:57.402250+0000","last_clean":"2024-04-25T10:31:57.402250+0000","last_became_active":"2024-04-25T10:31:55.037582+0000","last_became_peered":"2024-04-25T10:31:55.037582+0000","last_unstale":"2024-04-25T10:31:57.402250+0000","last_undegraded":"2024-04-25T10:31:57.402250+0000","last_fullsized":"2024-04-25T10:31:57.402250+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:13.004953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399085+0000","last_change":"2024-04-25T10:31:56.045978+0000","last_active":"2024-04-25T10:31:57.399085+0000","last_peered":"2024-04-25T10:31:57.399085+0000","last_clean":"2024-04-25T10:31:57.399085+0000","last_became_active":"2024-04-25T10:31:56.045549+0000","last_became_peered":"2024-04-25T10:31:56.045549+0000","last_unstale":"2024-04-25T10:31:57.399085+0000","last_undegraded":"2024-04-25T10:31:57.399085+0000","last_fullsized":"2024-04-25T10:31:57.399085+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:12.836323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402098+0000","last_change":"2024-04-25T10:31:55.034324+0000","last_active":"2024-04-25T10:31:57.402098+0000","last_peered":"2024-04-25T10:31:57.402098+0000","last_clean":"2024-04-25T10:31:57.402098+0000","last_became_active":"2024-04-25T10:31:55.033699+0000","last_became_peered":"2024-04-25T10:31:55.033699+0000","last_unstale":"2024-04-25T10:31:57.402098+0000","last_undegraded":"2024-04-25T10:31:57.402098+0000","last_fullsized":"2024-04-25T10:31:57.402098+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:21.826880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034213+0000","last_change":"2024-04-25T10:31:56.024832+0000","last_active":"2024-04-25T10:31:57.034213+0000","last_peered":"2024-04-25T10:31:57.034213+0000","last_clean":"2024-04-25T10:31:57.034213+0000","last_became_active":"2024-04-25T10:31:56.024562+0000","last_became_peered":"2024-04-25T10:31:56.024562+0000","last_unstale":"2024-04-25T10:31:57.034213+0000","last_undegraded":"2024-04-25T10:31:57.034213+0000","last_fullsized":"2024-04-25T10:31:57.034213+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:38.911913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.10","version":"192'8","reported_seq":22,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.033962+0000","last_change":"2024-04-25T10:31:55.034513+0000","last_active":"2024-04-25T10:31:57.033962+0000","last_peered":"2024-04-25T10:31:57.033962+0000","last_clean":"2024-04-25T10:31:57.033962+0000","last_became_active":"2024-04-25T10:31:55.034161+0000","last_became_peered":"2024-04-25T10:31:55.034161+0000","last_unstale":"2024-04-25T10:31:57.033962+0000","last_undegraded":"2024-04-25T10:31:57.033962+0000","last_fullsized":"2024-04-25T10:31:57.033962+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:17.147622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402770+0000","last_change":"2024-04-25T10:31:56.031887+0000","last_active":"2024-04-25T10:31:57.402770+0000","last_peered":"2024-04-25T10:31:57.402770+0000","last_clean":"2024-04-25T10:31:57.402770+0000","last_became_active":"2024-04-25T10:31:56.031564+0000","last_became_peered":"2024-04-25T10:31:56.031564+0000","last_unstale":"2024-04-25T10:31:57.402770+0000","last_undegraded":"2024-04-25T10:31:57.402770+0000","last_fullsized":"2024-04-25T10:31:57.402770+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:50.816774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031530+0000","last_change":"2024-04-25T10:31:55.034357+0000","last_active":"2024-04-25T10:31:57.031530+0000","last_peered":"2024-04-25T10:31:57.031530+0000","last_clean":"2024-04-25T10:31:57.031530+0000","last_became_active":"2024-04-25T10:31:55.034084+0000","last_became_peered":"2024-04-25T10:31:55.034084+0000","last_unstale":"2024-04-25T10:31:57.031530+0000","last_undegraded":"2024-04-25T10:31:57.031530+0000","last_fullsized":"2024-04-25T10:31:57.031530+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:28.127404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036595+0000","last_change":"2024-04-25T10:31:56.025779+0000","last_active":"2024-04-25T10:31:57.036595+0000","last_peered":"2024-04-25T10:31:57.036595+0000","last_clean":"2024-04-25T10:31:57.036595+0000","last_became_active":"2024-04-25T10:31:56.025493+0000","last_became_peered":"2024-04-25T10:31:56.025493+0000","last_unstale":"2024-04-25T10:31:57.036595+0000","last_undegraded":"2024-04-25T10:31:57.036595+0000","last_fullsized":"2024-04-25T10:31:57.036595+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:21.926564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.e","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036580+0000","last_change":"2024-04-25T10:31:55.028670+0000","last_active":"2024-04-25T10:31:57.036580+0000","last_peered":"2024-04-25T10:31:57.036580+0000","last_clean":"2024-04-25T10:31:57.036580+0000","last_became_active":"2024-04-25T10:31:55.027986+0000","last_became_peered":"2024-04-25T10:31:55.027986+0000","last_unstale":"2024-04-25T10:31:57.036580+0000","last_undegraded":"2024-04-25T10:31:57.036580+0000","last_fullsized":"2024-04-25T10:31:57.036580+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:07.739012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034436+0000","last_change":"2024-04-25T10:31:56.036633+0000","last_active":"2024-04-25T10:31:57.034436+0000","last_peered":"2024-04-25T10:31:57.034436+0000","last_clean":"2024-04-25T10:31:57.034436+0000","last_became_active":"2024-04-25T10:31:56.036366+0000","last_became_peered":"2024-04-25T10:31:56.036366+0000","last_unstale":"2024-04-25T10:31:57.034436+0000","last_undegraded":"2024-04-25T10:31:57.034436+0000","last_fullsized":"2024-04-25T10:31:57.034436+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:55.498103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"192'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031071+0000","last_change":"2024-04-25T10:31:55.028330+0000","last_active":"2024-04-25T10:31:57.031071+0000","last_peered":"2024-04-25T10:31:57.031071+0000","last_clean":"2024-04-25T10:31:57.031071+0000","last_became_active":"2024-04-25T10:31:55.028050+0000","last_became_peered":"2024-04-25T10:31:55.028050+0000","last_unstale":"2024-04-25T10:31:57.031071+0000","last_undegraded":"2024-04-25T10:31:57.031071+0000","last_fullsized":"2024-04-25T10:31:57.031071+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:42.168210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399355+0000","last_change":"2024-04-25T10:31:56.042137+0000","last_active":"2024-04-25T10:31:57.399355+0000","last_peered":"2024-04-25T10:31:57.399355+0000","last_clean":"2024-04-25T10:31:57.399355+0000","last_became_active":"2024-04-25T10:31:56.041123+0000","last_became_peered":"2024-04-25T10:31:56.041123+0000","last_unstale":"2024-04-25T10:31:57.399355+0000","last_undegraded":"2024-04-25T10:31:57.399355+0000","last_fullsized":"2024-04-25T10:31:57.399355+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:25.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402500+0000","last_change":"2024-04-25T10:31:55.030692+0000","last_active":"2024-04-25T10:31:57.402500+0000","last_peered":"2024-04-25T10:31:57.402500+0000","last_clean":"2024-04-25T10:31:57.402500+0000","last_became_active":"2024-04-25T10:31:55.030401+0000","last_became_peered":"2024-04-25T10:31:55.030401+0000","last_unstale":"2024-04-25T10:31:57.402500+0000","last_undegraded":"2024-04-25T10:31:57.402500+0000","last_fullsized":"2024-04-25T10:31:57.402500+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:20.995626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036035+0000","last_change":"2024-04-25T10:31:56.034477+0000","last_active":"2024-04-25T10:31:57.036035+0000","last_peered":"2024-04-25T10:31:57.036035+0000","last_clean":"2024-04-25T10:31:57.036035+0000","last_became_active":"2024-04-25T10:31:56.033687+0000","last_became_peered":"2024-04-25T10:31:56.033687+0000","last_unstale":"2024-04-25T10:31:57.036035+0000","last_undegraded":"2024-04-25T10:31:57.036035+0000","last_fullsized":"2024-04-25T10:31:57.036035+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:21.871531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.b","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030761+0000","last_change":"2024-04-25T10:31:55.032688+0000","last_active":"2024-04-25T10:31:57.030761+0000","last_peered":"2024-04-25T10:31:57.030761+0000","last_clean":"2024-04-25T10:31:57.030761+0000","last_became_active":"2024-04-25T10:31:55.032452+0000","last_became_peered":"2024-04-25T10:31:55.032452+0000","last_unstale":"2024-04-25T10:31:57.030761+0000","last_undegraded":"2024-04-25T10:31:57.030761+0000","last_fullsized":"2024-04-25T10:31:57.030761+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:00.965601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399234+0000","last_change":"2024-04-25T10:31:56.052188+0000","last_active":"2024-04-25T10:31:57.399234+0000","last_peered":"2024-04-25T10:31:57.399234+0000","last_clean":"2024-04-25T10:31:57.399234+0000","last_became_active":"2024-04-25T10:31:56.051891+0000","last_became_peered":"2024-04-25T10:31:56.051891+0000","last_unstale":"2024-04-25T10:31:57.399234+0000","last_undegraded":"2024-04-25T10:31:57.399234+0000","last_fullsized":"2024-04-25T10:31:57.399234+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:16.395471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.345252+0000","last_change":"2024-04-25T10:31:55.024672+0000","last_active":"2024-04-25T10:31:57.345252+0000","last_peered":"2024-04-25T10:31:57.345252+0000","last_clean":"2024-04-25T10:31:57.345252+0000","last_became_active":"2024-04-25T10:31:55.024385+0000","last_became_peered":"2024-04-25T10:31:55.024385+0000","last_unstale":"2024-04-25T10:31:57.345252+0000","last_undegraded":"2024-04-25T10:31:57.345252+0000","last_fullsized":"2024-04-25T10:31:57.345252+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:50.472352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402066+0000","last_change":"2024-04-25T10:31:56.036921+0000","last_active":"2024-04-25T10:31:57.402066+0000","last_peered":"2024-04-25T10:31:57.402066+0000","last_clean":"2024-04-25T10:31:57.402066+0000","last_became_active":"2024-04-25T10:31:56.036061+0000","last_became_peered":"2024-04-25T10:31:56.036061+0000","last_unstale":"2024-04-25T10:31:57.402066+0000","last_undegraded":"2024-04-25T10:31:57.402066+0000","last_fullsized":"2024-04-25T10:31:57.402066+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:32.434009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034336+0000","last_change":"2024-04-25T10:31:55.039698+0000","last_active":"2024-04-25T10:31:57.034336+0000","last_peered":"2024-04-25T10:31:57.034336+0000","last_clean":"2024-04-25T10:31:57.034336+0000","last_became_active":"2024-04-25T10:31:55.039360+0000","last_became_peered":"2024-04-25T10:31:55.039360+0000","last_unstale":"2024-04-25T10:31:57.034336+0000","last_undegraded":"2024-04-25T10:31:57.034336+0000","last_fullsized":"2024-04-25T10:31:57.034336+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:22.560691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403503+0000","last_change":"2024-04-25T10:31:56.040718+0000","last_active":"2024-04-25T10:31:57.403503+0000","last_peered":"2024-04-25T10:31:57.403503+0000","last_clean":"2024-04-25T10:31:57.403503+0000","last_became_active":"2024-04-25T10:31:56.040290+0000","last_became_peered":"2024-04-25T10:31:56.040290+0000","last_unstale":"2024-04-25T10:31:57.403503+0000","last_undegraded":"2024-04-25T10:31:57.403503+0000","last_fullsized":"2024-04-25T10:31:57.403503+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:20.449407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034063+0000","last_change":"2024-04-25T10:31:55.037702+0000","last_active":"2024-04-25T10:31:57.034063+0000","last_peered":"2024-04-25T10:31:57.034063+0000","last_clean":"2024-04-25T10:31:57.034063+0000","last_became_active":"2024-04-25T10:31:55.037430+0000","last_became_peered":"2024-04-25T10:31:55.037430+0000","last_unstale":"2024-04-25T10:31:57.034063+0000","last_undegraded":"2024-04-25T10:31:57.034063+0000","last_fullsized":"2024-04-25T10:31:57.034063+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:47.143165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036553+0000","last_change":"2024-04-25T10:31:56.028867+0000","last_active":"2024-04-25T10:31:57.036553+0000","last_peered":"2024-04-25T10:31:57.036553+0000","last_clean":"2024-04-25T10:31:57.036553+0000","last_became_active":"2024-04-25T10:31:56.028558+0000","last_became_peered":"2024-04-25T10:31:56.028558+0000","last_unstale":"2024-04-25T10:31:57.036553+0000","last_undegraded":"2024-04-25T10:31:57.036553+0000","last_fullsized":"2024-04-25T10:31:57.036553+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:09.225108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.036618+0000","last_change":"2024-04-25T10:31:55.037572+0000","last_active":"2024-04-25T10:31:57.036618+0000","last_peered":"2024-04-25T10:31:57.036618+0000","last_clean":"2024-04-25T10:31:57.036618+0000","last_became_active":"2024-04-25T10:31:55.036683+0000","last_became_peered":"2024-04-25T10:31:55.036683+0000","last_unstale":"2024-04-25T10:31:57.036618+0000","last_undegraded":"2024-04-25T10:31:57.036618+0000","last_fullsized":"2024-04-25T10:31:57.036618+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:25.438605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031362+0000","last_change":"2024-04-25T10:31:56.040677+0000","last_active":"2024-04-25T10:31:57.031362+0000","last_peered":"2024-04-25T10:31:57.031362+0000","last_clean":"2024-04-25T10:31:57.031362+0000","last_became_active":"2024-04-25T10:31:56.039517+0000","last_became_peered":"2024-04-25T10:31:56.039517+0000","last_unstale":"2024-04-25T10:31:57.031362+0000","last_undegraded":"2024-04-25T10:31:57.031362+0000","last_fullsized":"2024-04-25T10:31:57.031362+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:12.528009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.6","version":"193'1","reported_seq":15,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.033434+0000","last_change":"2024-04-25T10:31:55.026677+0000","last_active":"2024-04-25T10:31:57.033434+0000","last_peered":"2024-04-25T10:31:57.033434+0000","last_clean":"2024-04-25T10:31:57.033434+0000","last_became_active":"2024-04-25T10:31:55.026413+0000","last_became_peered":"2024-04-25T10:31:55.026413+0000","last_unstale":"2024-04-25T10:31:57.033434+0000","last_undegraded":"2024-04-25T10:31:57.033434+0000","last_fullsized":"2024-04-25T10:31:57.033434+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:02.982753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030633+0000","last_change":"2024-04-25T10:31:56.021872+0000","last_active":"2024-04-25T10:31:57.030633+0000","last_peered":"2024-04-25T10:31:57.030633+0000","last_clean":"2024-04-25T10:31:57.030633+0000","last_became_active":"2024-04-25T10:31:56.021320+0000","last_became_peered":"2024-04-25T10:31:56.021320+0000","last_unstale":"2024-04-25T10:31:57.030633+0000","last_undegraded":"2024-04-25T10:31:57.030633+0000","last_fullsized":"2024-04-25T10:31:57.030633+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:18.155270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399421+0000","last_change":"2024-04-25T10:31:55.035154+0000","last_active":"2024-04-25T10:31:57.399421+0000","last_peered":"2024-04-25T10:31:57.399421+0000","last_clean":"2024-04-25T10:31:57.399421+0000","last_became_active":"2024-04-25T10:31:55.034000+0000","last_became_peered":"2024-04-25T10:31:55.034000+0000","last_unstale":"2024-04-25T10:31:57.399421+0000","last_undegraded":"2024-04-25T10:31:57.399421+0000","last_fullsized":"2024-04-25T10:31:57.399421+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:19.441759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.402367+0000","last_change":"2024-04-25T10:31:56.032695+0000","last_active":"2024-04-25T10:31:57.402367+0000","last_peered":"2024-04-25T10:31:57.402367+0000","last_clean":"2024-04-25T10:31:57.402367+0000","last_became_active":"2024-04-25T10:31:56.032425+0000","last_became_peered":"2024-04-25T10:31:56.032425+0000","last_unstale":"2024-04-25T10:31:57.402367+0000","last_undegraded":"2024-04-25T10:31:57.402367+0000","last_fullsized":"2024-04-25T10:31:57.402367+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:23.900836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034369+0000","last_change":"2024-04-25T10:31:55.035218+0000","last_active":"2024-04-25T10:31:57.034369+0000","last_peered":"2024-04-25T10:31:57.034369+0000","last_clean":"2024-04-25T10:31:57.034369+0000","last_became_active":"2024-04-25T10:31:55.034814+0000","last_became_peered":"2024-04-25T10:31:55.034814+0000","last_unstale":"2024-04-25T10:31:57.034369+0000","last_undegraded":"2024-04-25T10:31:57.034369+0000","last_fullsized":"2024-04-25T10:31:57.034369+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:43.952756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030322+0000","last_change":"2024-04-25T10:31:56.023799+0000","last_active":"2024-04-25T10:31:57.030322+0000","last_peered":"2024-04-25T10:31:57.030322+0000","last_clean":"2024-04-25T10:31:57.030322+0000","last_became_active":"2024-04-25T10:31:56.022199+0000","last_became_peered":"2024-04-25T10:31:56.022199+0000","last_unstale":"2024-04-25T10:31:57.030322+0000","last_undegraded":"2024-04-25T10:31:57.030322+0000","last_fullsized":"2024-04-25T10:31:57.030322+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:20.449407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.034275+0000","last_change":"2024-04-25T10:31:55.036033+0000","last_active":"2024-04-25T10:31:57.034275+0000","last_peered":"2024-04-25T10:31:57.034275+0000","last_clean":"2024-04-25T10:31:57.034275+0000","last_became_active":"2024-04-25T10:31:55.035715+0000","last_became_peered":"2024-04-25T10:31:55.035715+0000","last_unstale":"2024-04-25T10:31:57.034275+0000","last_undegraded":"2024-04-25T10:31:57.034275+0000","last_fullsized":"2024-04-25T10:31:57.034275+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:02.716839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030626+0000","last_change":"2024-04-25T10:31:56.021795+0000","last_active":"2024-04-25T10:31:57.030626+0000","last_peered":"2024-04-25T10:31:57.030626+0000","last_clean":"2024-04-25T10:31:57.030626+0000","last_became_active":"2024-04-25T10:31:56.021464+0000","last_became_peered":"2024-04-25T10:31:56.021464+0000","last_unstale":"2024-04-25T10:31:57.030626+0000","last_undegraded":"2024-04-25T10:31:57.030626+0000","last_fullsized":"2024-04-25T10:31:57.030626+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:29.443748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403184+0000","last_change":"2024-04-25T10:31:55.036194+0000","last_active":"2024-04-25T10:31:57.403184+0000","last_peered":"2024-04-25T10:31:57.403184+0000","last_clean":"2024-04-25T10:31:57.403184+0000","last_became_active":"2024-04-25T10:31:55.035692+0000","last_became_peered":"2024-04-25T10:31:55.035692+0000","last_unstale":"2024-04-25T10:31:57.403184+0000","last_undegraded":"2024-04-25T10:31:57.403184+0000","last_fullsized":"2024-04-25T10:31:57.403184+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:08.416412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.033978+0000","last_change":"2024-04-25T10:31:56.035045+0000","last_active":"2024-04-25T10:31:57.033978+0000","last_peered":"2024-04-25T10:31:57.033978+0000","last_clean":"2024-04-25T10:31:57.033978+0000","last_became_active":"2024-04-25T10:31:56.034717+0000","last_became_peered":"2024-04-25T10:31:56.034717+0000","last_unstale":"2024-04-25T10:31:57.033978+0000","last_undegraded":"2024-04-25T10:31:57.033978+0000","last_fullsized":"2024-04-25T10:31:57.033978+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:09.424061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.403509+0000","last_change":"2024-04-25T10:31:55.036736+0000","last_active":"2024-04-25T10:31:57.403509+0000","last_peered":"2024-04-25T10:31:57.403509+0000","last_clean":"2024-04-25T10:31:57.403509+0000","last_became_active":"2024-04-25T10:31:55.036431+0000","last_became_peered":"2024-04-25T10:31:55.036431+0000","last_unstale":"2024-04-25T10:31:57.403509+0000","last_undegraded":"2024-04-25T10:31:57.403509+0000","last_fullsized":"2024-04-25T10:31:57.403509+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:16.630558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399084+0000","last_change":"2024-04-25T10:31:56.025794+0000","last_active":"2024-04-25T10:31:57.399084+0000","last_peered":"2024-04-25T10:31:57.399084+0000","last_clean":"2024-04-25T10:31:57.399084+0000","last_became_active":"2024-04-25T10:31:56.024571+0000","last_became_peered":"2024-04-25T10:31:56.024571+0000","last_unstale":"2024-04-25T10:31:57.399084+0000","last_undegraded":"2024-04-25T10:31:57.399084+0000","last_fullsized":"2024-04-25T10:31:57.399084+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:36.603692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":14,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030134+0000","last_change":"2024-04-25T10:31:55.023947+0000","last_active":"2024-04-25T10:31:57.030134+0000","last_peered":"2024-04-25T10:31:57.030134+0000","last_clean":"2024-04-25T10:31:57.030134+0000","last_became_active":"2024-04-25T10:31:55.023673+0000","last_became_peered":"2024-04-25T10:31:55.023673+0000","last_unstale":"2024-04-25T10:31:57.030134+0000","last_undegraded":"2024-04-25T10:31:57.030134+0000","last_fullsized":"2024-04-25T10:31:57.030134+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:40.583722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":12,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.399334+0000","last_change":"2024-04-25T10:31:56.035805+0000","last_active":"2024-04-25T10:31:57.399334+0000","last_peered":"2024-04-25T10:31:57.399334+0000","last_clean":"2024-04-25T10:31:57.399334+0000","last_became_active":"2024-04-25T10:31:56.035535+0000","last_became_peered":"2024-04-25T10:31:56.035535+0000","last_unstale":"2024-04-25T10:31:57.399334+0000","last_undegraded":"2024-04-25T10:31:57.399334+0000","last_fullsized":"2024-04-25T10:31:57.399334+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:55.001838+0000","last_clean_scrub_stamp":"2024-04-25T10:31:55.001838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:24.777713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"192'2","reported_seq":16,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.031056+0000","last_change":"2024-04-25T10:31:55.030797+0000","last_active":"2024-04-25T10:31:57.031056+0000","last_peered":"2024-04-25T10:31:57.031056+0000","last_clean":"2024-04-25T10:31:57.031056+0000","last_became_active":"2024-04-25T10:31:55.030452+0000","last_became_peered":"2024-04-25T10:31:55.030452+0000","last_unstale":"2024-04-25T10:31:57.031056+0000","last_undegraded":"2024-04-25T10:31:57.031056+0000","last_fullsized":"2024-04-25T10:31:57.031056+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:31:53.994190+0000","last_clean_scrub_stamp":"2024-04-25T10:31:53.994190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:20.069780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":501,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-04-25T10:31:57.030093+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:31:57.030093+0000","last_peered":"2024-04-25T10:31:57.030093+0000","last_clean":"2024-04-25T10:31:57.030093+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:31:57.030093+0000","last_undegraded":"2024-04-25T10:31:57.030093+0000","last_fullsized":"2024-04-25T10:31:57.030093+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509613,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55772,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316068,"statfs":{"total":96636764160,"available":96579653632,"internally_reserved":0,"allocated":2437120,"data_stored":1068363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509614,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55772,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316068,"statfs":{"total":96636764160,"available":96579653632,"internally_reserved":0,"allocated":2441216,"data_stored":1067837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509613,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60516,"kb_used_data":2968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311324,"statfs":{"total":96636764160,"available":96574795776,"internally_reserved":0,"allocated":3039232,"data_stored":1659741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509613,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47440,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324400,"statfs":{"total":96636764160,"available":96588185600,"internally_reserved":0,"allocated":2428928,"data_stored":1067691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509613,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69592,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302248,"statfs":{"total":96636764160,"available":96565501952,"internally_reserved":0,"allocated":2433024,"data_stored":1067725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509614,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48032,"kb_used_data":2964,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323808,"statfs":{"total":96636764160,"available":96587579392,"internally_reserved":0,"allocated":3035136,"data_stored":1658733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509614,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59928,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311912,"statfs":{"total":96636764160,"available":96575397888,"internally_reserved":0,"allocated":2428928,"data_stored":1067691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69592,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302248,"statfs":{"total":96636764160,"available":96565501952,"internally_reserved":0,"allocated":2428928,"data_stored":1063003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:32:00.541 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:32:00.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:32:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.726+0000 7f918de00640 1 Processor -- start 2024-04-25T10:32:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.726+0000 7f918de00640 1 -- start start 2024-04-25T10:32:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f918de00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f918815bbb0 0x7f91880cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f918de00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91880b4660 0x7f918806c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f918de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f91880cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f918de00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918806d050 con 0x7f91880b4660 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f918de00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918806ca90 con 0x7f918815bbb0 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f918de00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918806cd70 con 0x7f91880cf960 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91880b4660 0x7f918806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91880b4660 0x7f918806c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35680/0 (socket says 172.21.15.45:35680) 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 -- 172.21.15.45:0/1081889717 learned_addr learned my addr 172.21.15.45:0/1081889717 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f9187400640 1 --2- 172.21.15.45:0/1081889717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f91880cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f9186a00640 1 --2- 172.21.15.45:0/1081889717 >> v2:172.21.15.117:3300/0 conn(0x7f918815bbb0 0x7f91880cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 -- 172.21.15.45:0/1081889717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 msgr2=0x7f91880cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 --2- 172.21.15.45:0/1081889717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f91880cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 -- 172.21.15.45:0/1081889717 >> v2:172.21.15.117:3300/0 conn(0x7f918815bbb0 msgr2=0x7f91880cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 --2- 172.21.15.45:0/1081889717 >> v2:172.21.15.117:3300/0 conn(0x7f918815bbb0 0x7f91880cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 -- 172.21.15.45:0/1081889717 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880b2880 con 0x7f91880b4660 2024-04-25T10:32:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f917fe00640 1 --2- 172.21.15.45:0/1081889717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91880b4660 0x7f918806c120 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f918806e8b0 tx=0x7f917001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55db589b8fc08e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f9186000640 1 -- 172.21.15.45:0/1081889717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700b0070 con 0x7f91880b4660 2024-04-25T10:32:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f9186000640 1 -- 172.21.15.45:0/1081889717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91700ab600 con 0x7f91880b4660 2024-04-25T10:32:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.730+0000 7f9186000640 1 -- 172.21.15.45:0/1081889717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700ab9d0 con 0x7f91880b4660 2024-04-25T10:32:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- 172.21.15.45:0/1081889717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91880b4660 msgr2=0x7f918806c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- 172.21.15.45:0/1081889717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91880b4660 0x7f918806c120 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f918806e8b0 tx=0x7f917001c520 comp rx=0 tx=0).stop 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- 172.21.15.45:0/1081889717 shutdown_connections 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- 172.21.15.45:0/1081889717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f91880cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- 172.21.15.45:0/1081889717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91880b4660 0x7f918806c120 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- 172.21.15.45:0/1081889717 >> v2:172.21.15.117:3300/0 conn(0x7f918815bbb0 0x7f91880cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- 172.21.15.45:0/1081889717 >> 172.21.15.45:0/1081889717 conn(0x7f9188066b70 msgr2=0x7f9188066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- 172.21.15.45:0/1081889717 shutdown_connections 2024-04-25T10:32:00.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- 172.21.15.45:0/1081889717 wait complete. 2024-04-25T10:32:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 Processor -- start 2024-04-25T10:32:00.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- start start 2024-04-25T10:32:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880b4660 0x7f9188155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f918814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 0x7f918814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b3890 con 0x7f918815bbb0 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918806c870 con 0x7f91880b4660 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f918de00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b3a10 con 0x7f91880cf960 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 0x7f918814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 0x7f918814c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35684/0 (socket says 172.21.15.45:35684) 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 -- 172.21.15.45:0/3564063480 learned_addr learned my addr 172.21.15.45:0/3564063480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9186a00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880b4660 0x7f9188155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f917fe00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f918814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 -- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 msgr2=0x7f918814bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f918814bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 -- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880b4660 msgr2=0x7f9188155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880b4660 0x7f9188155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.734+0000 7f9187400640 1 -- 172.21.15.45:0/3564063480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918815cd90 con 0x7f918815bbb0 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f9187400640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 0x7f918814c4b0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f9180007e70 tx=0x7f918000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180014620 con 0x7f918815bbb0 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180019070 con 0x7f918815bbb0 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180014a20 con 0x7f918815bbb0 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918815d070 con 0x7f918815bbb0 2024-04-25T10:32:00.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918815d630 con 0x7f918815bbb0 2024-04-25T10:32:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918804ef30 con 0x7f918815bbb0 2024-04-25T10:32:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9180014bc0 con 0x7f918815bbb0 2024-04-25T10:32:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f917f400640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9164066f60 0x7f9164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:00.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91800857c0 con 0x7f918815bbb0 2024-04-25T10:32:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.738+0000 7f9186a00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9164066f60 0x7f9164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:00.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.742+0000 7f9186a00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9164066f60 0x7f9164069420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9178007940 tx=0x7f9178008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.742+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9180056180 con 0x7f918815bbb0 2024-04-25T10:32:00.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.962+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f91881562c0 con 0x7f918815bbb0 2024-04-25T10:32:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.966+0000 7f917f400640 1 -- 172.21.15.45:0/3564063480 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f918005a220 con 0x7f918815bbb0 2024-04-25T10:32:00.969 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T10:32:00.969 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCaLSpm9R0YExAApXxVtglEn73nrQk/ptYOIQ== 2024-04-25T10:32:00.969 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-04-25T10:32:00.969 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-04-25T10:32:00.969 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:32:00.970 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.966+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9164066f60 msgr2=0x7f9164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.966+0000 7f918de00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9164066f60 0x7f9164069420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9178007940 tx=0x7f9178008040 comp rx=0 tx=0).stop 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.966+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 msgr2=0x7f918814c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.966+0000 7f918de00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 0x7f918814c4b0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f9180007e70 tx=0x7f918000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 shutdown_connections 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9164066f60 0x7f9164069420 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f918815bbb0 0x7f918814c4b0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91880cf960 0x7f918814bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 --2- 172.21.15.45:0/3564063480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91880b4660 0x7f9188155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 >> 172.21.15.45:0/3564063480 conn(0x7f9188066b70 msgr2=0x7f91880ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 shutdown_connections 2024-04-25T10:32:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:00.970+0000 7f918de00640 1 -- 172.21.15.45:0/3564063480 wait complete. 2024-04-25T10:32:00.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:01.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 Processor -- start 2024-04-25T10:32:01.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 -- start start 2024-04-25T10:32:01.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe57c0ccb20 0x7fe57c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c0d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0ccfb0 con 0x7fe57c15b9e0 2024-04-25T10:32:01.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0cd290 con 0x7fe57c0ccb20 2024-04-25T10:32:01.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe580c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0cd570 con 0x7fe57c15bdc0 2024-04-25T10:32:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe57c0ccb20 0x7fe57c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe57a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c0d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe57c0ccb20 0x7fe57c0d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52722/0 (socket says 172.21.15.45:52722) 2024-04-25T10:32:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 -- 172.21.15.45:0/3445570514 learned_addr learned my addr 172.21.15.45:0/3445570514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 -- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 msgr2=0x7fe57c0d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 --2- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c0d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 -- 172.21.15.45:0/3445570514 >> v1:172.21.15.117:6789/0 conn(0x7fe57c15b9e0 legacy=0x7fe57c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe571400640 1 -- 172.21.15.45:0/3445570514 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0cd850 con 0x7fe57c0ccb20 2024-04-25T10:32:01.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.134+0000 7fe57a800640 1 --2- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c0d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe571400640 1 --2- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c0ccb20 0x7fe57c0d1520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe564009870 tx=0x7fe56401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea02ae1183dd07bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe579400640 1 -- 172.21.15.45:0/3445570514 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640b2070 con 0x7fe57c0ccb20 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe579400640 1 -- 172.21.15.45:0/3445570514 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5640a9070 con 0x7fe57c0ccb20 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe579400640 1 -- 172.21.15.45:0/3445570514 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640ad880 con 0x7fe57c0ccb20 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 -- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c0ccb20 msgr2=0x7fe57c0d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 --2- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c0ccb20 0x7fe57c0d1520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe564009870 tx=0x7fe56401c560 comp rx=0 tx=0).stop 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 -- 172.21.15.45:0/3445570514 shutdown_connections 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 --2- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c0d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 --2- 172.21.15.45:0/3445570514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c0ccb20 0x7fe57c0d1520 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 -- 172.21.15.45:0/3445570514 >> 172.21.15.45:0/3445570514 conn(0x7fe57c09a9d0 msgr2=0x7fe57c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 -- 172.21.15.45:0/3445570514 shutdown_connections 2024-04-25T10:32:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.138+0000 7fe580c00640 1 -- 172.21.15.45:0/3445570514 wait complete. 2024-04-25T10:32:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 Processor -- start 2024-04-25T10:32:01.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 -- start start 2024-04-25T10:32:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c15b9e0 0x7fe57c06f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 0x7fe57c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c65f0 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c6470 con 0x7fe57c15b9e0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c6770 con 0x7fe57c15bdc0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 0x7fe57c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe579e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c15b9e0 0x7fe57c06f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 0x7fe57c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35700/0 (socket says 172.21.15.45:35700) 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 -- 172.21.15.45:0/3704270912 learned_addr learned my addr 172.21.15.45:0/3704270912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe571400640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 -- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 msgr2=0x7fe57c06f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c06f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 -- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c15b9e0 msgr2=0x7fe57c06f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c15b9e0 0x7fe57c06f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 -- 172.21.15.45:0/3704270912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0b58d0 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe57a800640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 0x7fe57c0b5390 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe574009e60 tx=0x7fe57400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574018030 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57400cd90 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574003bc0 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c0b5ba0 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.142+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c0b6160 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.146+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c04ef30 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.146+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe57401b070 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.146+0000 7fe572a00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.146+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5740857e0 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.146+0000 7fe579e00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.146+0000 7fe579e00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007fd0 tx=0x7fe56c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.150+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe574022070 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.346+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe57c0cb110 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.346+0000 7fe572a00640 1 -- 172.21.15.45:0/3704270912 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4957 (secure 0 0 0) 0x7fe574010040 con 0x7fe57c0b4ee0 2024-04-25T10:32:01.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:01.350 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":140,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:31:58.443777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11304":{"gid":11304,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3198476999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3198476999},{"type":"v1","addr":"172.21.15.45:6835","nonce":3198476999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:32:01.351 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 140 2024-04-25T10:32:01.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 msgr2=0x7fe558069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007fd0 tx=0x7fe56c009040 comp rx=0 tx=0).stop 2024-04-25T10:32:01.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 msgr2=0x7fe57c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 0x7fe57c0b5390 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fe574009e60 tx=0x7fe57400c490 comp rx=0 tx=0).stop 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 shutdown_connections 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b4ee0 0x7fe57c0b5390 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c15bdc0 0x7fe57c06f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 --2- 172.21.15.45:0/3704270912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c15b9e0 0x7fe57c06f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 >> 172.21.15.45:0/3704270912 conn(0x7fe57c09a9d0 msgr2=0x7fe57c0cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 shutdown_connections 2024-04-25T10:32:01.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.350+0000 7fe580c00640 1 -- 172.21.15.45:0/3704270912 wait complete. 2024-04-25T10:32:01.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:31:55.417950+0000', 'modified': '2024-04-25T10:31:58.443777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11304': {'gid': 11304, 'name': 'c', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/3198476999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 3198476999}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 3198476999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:32:01.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:32:01.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:32:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.918+0000 7f947da00640 1 Processor -- start 2024-04-25T10:32:01.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.918+0000 7f947da00640 1 -- start start 2024-04-25T10:32:01.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f947da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f94780b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f947da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94780cd780 0x7f94780cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f947da00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947806be90 con 0x7f94780b1f50 2024-04-25T10:32:01.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f947da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780d0b50 con 0x7f94780cd780 2024-04-25T10:32:01.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f947da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947806b680 con 0x7f94780b23c0 2024-04-25T10:32:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f947d000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94780cd780 0x7f94780cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f946fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f94780b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f9477e00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f94780b1f50 0x7f947806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:48412/0 (socket says 172.21.15.45:48412) 2024-04-25T10:32:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f9477e00640 1 -- 172.21.15.45:0/1687660331 learned_addr learned my addr 172.21.15.45:0/1687660331 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:01.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f946fe00640 1 -- 172.21.15.45:0/1687660331 >> v2:172.21.15.117:3300/0 conn(0x7f94780cd780 msgr2=0x7f94780cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f946fe00640 1 --2- 172.21.15.45:0/1687660331 >> v2:172.21.15.117:3300/0 conn(0x7f94780cd780 0x7f94780cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f946fe00640 1 -- 172.21.15.45:0/1687660331 >> v1:172.21.15.117:6789/0 conn(0x7f94780b1f50 legacy=0x7f947806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f946fe00640 1 -- 172.21.15.45:0/1687660331 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947806b8d0 con 0x7f94780b23c0 2024-04-25T10:32:01.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.922+0000 7f9477400640 1 -- 172.21.15.45:0/1687660331 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911542169 0 0) 0x7f9468001120 con 0x7f94780b1f50 2024-04-25T10:32:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f946fe00640 1 --2- 172.21.15.45:0/1687660331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f94780b3dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f94600097c0 tx=0x7f946001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8666f99b8badadb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f9477400640 1 -- 172.21.15.45:0/1687660331 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600b0070 con 0x7f94780b23c0 2024-04-25T10:32:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f9477400640 1 -- 172.21.15.45:0/1687660331 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600ab600 con 0x7f94780b23c0 2024-04-25T10:32:01.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f9477400640 1 -- 172.21.15.45:0/1687660331 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600ab9d0 con 0x7f94780b23c0 2024-04-25T10:32:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 -- 172.21.15.45:0/1687660331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 msgr2=0x7f94780b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 --2- 172.21.15.45:0/1687660331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f94780b3dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f94600097c0 tx=0x7f946001c380 comp rx=0 tx=0).stop 2024-04-25T10:32:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 -- 172.21.15.45:0/1687660331 shutdown_connections 2024-04-25T10:32:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 --2- 172.21.15.45:0/1687660331 >> v2:172.21.15.117:3300/0 conn(0x7f94780cd780 0x7f94780cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 --2- 172.21.15.45:0/1687660331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f94780b3dd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 -- 172.21.15.45:0/1687660331 >> 172.21.15.45:0/1687660331 conn(0x7f94780b32b0 msgr2=0x7f94780b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 -- 172.21.15.45:0/1687660331 shutdown_connections 2024-04-25T10:32:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 -- 172.21.15.45:0/1687660331 wait complete. 2024-04-25T10:32:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 Processor -- start 2024-04-25T10:32:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.926+0000 7f947da00640 1 -- start start 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 0x7f947815ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f947815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94780cd780 0x7f947815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780d03c0 con 0x7f94780b1f50 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780d0240 con 0x7f94780cd780 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780d0540 con 0x7f94780b23c0 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 0x7f947815ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 0x7f947815ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35702/0 (socket says 172.21.15.45:35702) 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 -- 172.21.15.45:0/3494813794 learned_addr learned my addr 172.21.15.45:0/3494813794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947d000640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94780cd780 0x7f947815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f946fe00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f947815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 -- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 msgr2=0x7f947815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f947815a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 -- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94780cd780 msgr2=0x7f947815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94780cd780 0x7f947815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 -- 172.21.15.45:0/3494813794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947815d9e0 con 0x7f94780b1f50 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947d000640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94780cd780 0x7f947815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9477e00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 0x7f947815ed10 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f946800c4b0 tx=0x7f946800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9474c00640 1 -- 172.21.15.45:0/3494813794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468018070 con 0x7f94780b1f50 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9474c00640 1 -- 172.21.15.45:0/3494813794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468007500 con 0x7f94780b1f50 2024-04-25T10:32:01.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9474c00640 1 -- 172.21.15.45:0/3494813794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468004a30 con 0x7f94780b1f50 2024-04-25T10:32:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947815cae0 con 0x7f94780b1f50 2024-04-25T10:32:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947815d0a0 con 0x7f94780b1f50 2024-04-25T10:32:01.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.930+0000 7f9474c00640 1 -- 172.21.15.45:0/3494813794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9468004bd0 con 0x7f94780b1f50 2024-04-25T10:32:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9474c00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9454066f60 0x7f9454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9474c00640 1 -- 172.21.15.45:0/3494813794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9468084170 con 0x7f94780b1f50 2024-04-25T10:32:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 Processor -- start 2024-04-25T10:32:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f946fe00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9454066f60 0x7f9454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 -- start start 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f946fe00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9454066f60 0x7f9454069420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f94600ad000 tx=0x7f94600a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e90d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084d2e5f0 con 0x55b084e90920 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084c95da0 con 0x55b084e91240 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b084c1e150 con 0x55b084e46f30 2024-04-25T10:32:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e90d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e90d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35708/0 (socket says 172.21.15.45:35708) 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 -- 172.21.15.45:0/3223603246 learned_addr learned my addr 172.21.15.45:0/3223603246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f947d000640 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f946fe00640 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 -- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 msgr2=0x55b084e47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 -- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 msgr2=0x55b084e469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 -- 172.21.15.45:0/3223603246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b084c37ca0 con 0x55b084e90920 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f9477e00640 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e90d00 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f946805cc50 tx=0x7f9468054a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196124390b9dff29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f946cc00640 1 -- 172.21.15.45:0/3223603246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468052e20 con 0x55b084e90920 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f946cc00640 1 -- 172.21.15.45:0/3223603246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468001050 con 0x55b084e90920 2024-04-25T10:32:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.934+0000 7f946cc00640 1 -- 172.21.15.45:0/3223603246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468053f70 con 0x55b084e90920 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/3223603246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 msgr2=0x55b084e90d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e90d00 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f946805cc50 tx=0x7f9468054a20 comp rx=0 tx=0).stop 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/3223603246 shutdown_connections 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- 172.21.15.45:0/3223603246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e90d00 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/3223603246 >> 172.21.15.45:0/3223603246 conn(0x55b084e4d2a0 msgr2=0x55b084e452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/3223603246 shutdown_connections 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/3223603246 wait complete. 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- start start 2024-04-25T10:32:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 Processor -- start 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 0x55b084e7d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e90920 0x55b084e9aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e9cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084c50990 con 0x55b084e46f30 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084c91e40 con 0x55b084e91240 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b084c21900 con 0x55b084e90920 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 10 client.0 _ll_register_callbacks cb 0x55b084d491c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 0x55b084e7d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 0x55b084e7d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35716/0 (socket says 172.21.15.45:35716) 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 -- 172.21.15.45:0/72043144 learned_addr learned my addr 172.21.15.45:0/72043144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f946fe00640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e90920 0x55b084e9aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f947d000640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e9cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 -- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e90920 msgr2=0x55b084e9aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e90920 0x55b084e9aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 -- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 msgr2=0x55b084e9cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e9cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 -- 172.21.15.45:0/72043144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b084c4e630 con 0x55b084e46f30 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f947d000640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e9cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9477e00640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 0x55b084e7d560 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f9468050f80 tx=0x7f9468000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a4dfc3e02bbb93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9465600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9465600640 1 -- 172.21.15.45:0/72043144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468082390 con 0x55b084e46f30 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9465600640 1 -- 172.21.15.45:0/72043144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9468053f70 con 0x55b084e46f30 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9468001050 con 0x55b084e46f30 2024-04-25T10:32:01.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9465600640 1 -- 172.21.15.45:0/72043144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9468079050 con 0x55b084e46f30 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 10 client.11433 fetch_fsmap learned FSMap version 140 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x55b084c1e3a0 con 0x55b084e46f30 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.938+0000 7f9465600640 1 -- 172.21.15.45:0/72043144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468001050 con 0x55b084e46f30 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f9465600640 1 -- 172.21.15.45:0/72043144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 140) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9468082070 con 0x55b084e46f30 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 10 client.11433 fetch_fsmap finished waiting for FSMap version 140 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 10 client.11433 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 20 client.11433 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084e9fc80 0x55b084ea2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 4 client.11433 mds_command: new command op to 11304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 --> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b084c31a50 con 0x55b084e9fc80 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f946fe00640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084e9fc80 0x55b084ea2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f946fe00640 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084e9fc80 0x55b084ea2070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592807d3c1952155 server_cookie=9c68e08261361934 in_seq=0 out_seq=0 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f9465600640 10 client.11433 ms_handle_connect on v2:172.21.15.45:6833/3198476999 2024-04-25T10:32:01.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f9465600640 1 -- 172.21.15.45:0/72043144 <== mds.0 v2:172.21.15.45:6833/3198476999 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b084c31a50 con 0x55b084e9fc80 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f9465600640 10 client.11433 handle_command_reply: tid=0 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 client.11433 shutdown 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 msgr2=0x55b084e7d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 0x55b084e7d560 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f9468050f80 tx=0x7f9468000ee0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 shutdown_connections 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084e9fc80 0x55b084ea2070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084e9cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e90920 0x55b084e9aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 --2- 172.21.15.45:0/72043144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e46f30 0x55b084e7d560 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 >> 172.21.15.45:0/72043144 conn(0x55b084e4d2a0 msgr2=0x55b084e459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f9465600640 0 client.11433 ms_handle_reset on v2:172.21.15.45:6833/3198476999 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 shutdown_connections 2024-04-25T10:32:01.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 1 -- 172.21.15.45:0/72043144 wait complete. 2024-04-25T10:32:01.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.942+0000 7f948240e000 20 client.11433 trim_cache size 0 max 16384 2024-04-25T10:32:01.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 Processor -- start 2024-04-25T10:32:01.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 -- start start 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e46f30 0x55b084e9c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e91240 0x55b084e48320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947815c390 con 0x55b084e90920 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947815b790 con 0x55b084e46f30 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947815bd90 con 0x55b084e91240 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e9c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35724/0 (socket says 172.21.15.45:35724) 2024-04-25T10:32:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 -- 172.21.15.45:0/549834340 learned_addr learned my addr 172.21.15.45:0/549834340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f9477e00640 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e46f30 0x55b084e9c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 -- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e91240 msgr2=0x55b084e48320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e91240 0x55b084e48320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 -- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e46f30 msgr2=0x55b084e9c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e46f30 0x55b084e9c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 -- 172.21.15.45:0/549834340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b084e91070 con 0x55b084e90920 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f9477e00640 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e46f30 0x55b084e9c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f946fe00640 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e9c770 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f9460000c10 tx=0x7f9460009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f2ba27fb6e55ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f9467e00640 1 -- 172.21.15.45:0/549834340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600c9e60 con 0x55b084e90920 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f9467e00640 1 -- 172.21.15.45:0/549834340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600cf070 con 0x55b084e90920 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.958+0000 7f9467e00640 1 -- 172.21.15.45:0/549834340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600b0040 con 0x55b084e90920 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/549834340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 msgr2=0x55b084e9c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e9c770 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f9460000c10 tx=0x7f9460009190 comp rx=0 tx=0).stop 2024-04-25T10:32:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/549834340 shutdown_connections 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e91240 0x55b084e48320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e9c770 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- 172.21.15.45:0/549834340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e46f30 0x55b084e9c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/549834340 >> 172.21.15.45:0/549834340 conn(0x55b084e4d2a0 msgr2=0x55b084e4b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/549834340 shutdown_connections 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/549834340 wait complete. 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- start start 2024-04-25T10:32:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 Processor -- start 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e4a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084ed4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084c9a3d0 con 0x55b084e90920 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b084d51d50 con 0x55b084e91240 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b084c30980 con 0x55b084e46f30 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 10 client.0 _ll_register_callbacks cb 0x7f94680076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e4ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35734/0 (socket says 172.21.15.45:35734) 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 -- 172.21.15.45:0/206075788 learned_addr learned my addr 172.21.15.45:0/206075788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:01.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9477e00640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e4a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f947d000640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084ed4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 -- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 msgr2=0x55b084e4a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e4a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 -- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 msgr2=0x55b084ed4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084ed4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 -- 172.21.15.45:0/206075788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b084d17600 con 0x55b084e90920 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f947d000640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084ed4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f946fe00640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e4ab20 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f94600cc040 tx=0x7f946001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6810164051906a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9465600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9465600640 1 -- 172.21.15.45:0/206075788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600d8060 con 0x55b084e90920 2024-04-25T10:32:01.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94600cf070 con 0x55b084e90920 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9465600640 1 -- 172.21.15.45:0/206075788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94600c91e0 con 0x55b084e90920 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9465600640 1 -- 172.21.15.45:0/206075788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94600b0040 con 0x55b084e90920 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 10 client.11439 fetch_fsmap learned FSMap version 140 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x55b084c2a9b0 con 0x55b084e90920 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9465600640 1 -- 172.21.15.45:0/206075788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600cf070 con 0x55b084e90920 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f9465600640 1 -- 172.21.15.45:0/206075788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 140) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94600059a0 con 0x55b084e90920 2024-04-25T10:32:01.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 10 client.11439 fetch_fsmap finished waiting for FSMap version 140 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 10 client.11439 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 20 client.11439 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084ed4950 0x55b084ed54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 4 client.11439 mds_command: new command op to 11304 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.962+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 --> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b084ca32e0 con 0x55b084ed4950 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f9477e00640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084ed4950 0x55b084ed54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f9477e00640 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084ed4950 0x55b084ed54d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc739cf8e2f9a1b server_cookie=81b9731e6585f147 in_seq=0 out_seq=0 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f9465600640 10 client.11439 ms_handle_connect on v2:172.21.15.45:6833/3198476999 2024-04-25T10:32:01.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f9465600640 1 -- 172.21.15.45:0/206075788 <== mds.0 v2:172.21.15.45:6833/3198476999 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55b084ca32e0 con 0x55b084ed4950 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f9465600640 10 client.11439 handle_command_reply: tid=0 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 client.11439 shutdown 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 msgr2=0x55b084e4ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e4ab20 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f94600cc040 tx=0x7f946001abc0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 shutdown_connections 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:6833/3198476999,v1:172.21.15.45:6835/3198476999] conn(0x55b084ed4950 0x55b084ed54d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b084e91240 0x55b084ed4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b084e90920 0x55b084e4ab20 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 --2- 172.21.15.45:0/206075788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55b084e46f30 0x55b084e4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 >> 172.21.15.45:0/206075788 conn(0x55b084e4d2a0 msgr2=0x55b084e8fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f9465600640 0 client.11439 ms_handle_reset on v2:172.21.15.45:6833/3198476999 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 shutdown_connections 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 1 -- 172.21.15.45:0/206075788 wait complete. 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f948240e000 20 client.11439 trim_cache size 0 max 16384 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 11304, 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:32:01.975 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.9460320879999999, 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 140, 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 193, 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 193, 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.867061478 2024-04-25T10:32:01.976 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9454066f60 msgr2=0x7f9454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9454066f60 0x7f9454069420 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f94600ad000 tx=0x7f94600a6000 comp rx=0 tx=0).stop 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 msgr2=0x7f947815ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 0x7f947815ed10 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f946800c4b0 tx=0x7f946800c970 comp rx=0 tx=0).stop 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 shutdown_connections 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9454066f60 0x7f9454069420 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94780cd780 0x7f947815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94780b23c0 0x7f947815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 --2- 172.21.15.45:0/3494813794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94780b1f50 0x7f947815ed10 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 >> 172.21.15.45:0/3494813794 conn(0x7f94780b32b0 msgr2=0x7f94780cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:01.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 shutdown_connections 2024-04-25T10:32:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:01.966+0000 7f947da00640 1 -- 172.21.15.45:0/3494813794 wait complete. 2024-04-25T10:32:01.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:32:01.997 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:32:01.994+0000 7f282cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:32:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 Processor -- start 2024-04-25T10:32:02.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 -- start start 2024-04-25T10:32:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8e815bc40 0x7fe8e806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e80b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8e80cf930 0x7fe8e80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e80b2770 con 0x7fe8e815bc40 2024-04-25T10:32:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e806c670 con 0x7fe8e80cf930 2024-04-25T10:32:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8ecc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e806c950 con 0x7fe8e80cda10 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8e815bc40 0x7fe8e806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe8e815bc40 0x7fe8e806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35736/0 (socket says 172.21.15.45:35736) 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 -- 172.21.15.45:0/3862701959 learned_addr learned my addr 172.21.15.45:0/3862701959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e6800640 1 --2- 172.21.15.45:0/3862701959 >> v2:172.21.15.117:3300/0 conn(0x7fe8e80cf930 0x7fe8e80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8dd400640 1 --2- 172.21.15.45:0/3862701959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e80b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 -- 172.21.15.45:0/3862701959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 msgr2=0x7fe8e80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 --2- 172.21.15.45:0/3862701959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e80b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 -- 172.21.15.45:0/3862701959 >> v2:172.21.15.117:3300/0 conn(0x7fe8e80cf930 msgr2=0x7fe8e80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 --2- 172.21.15.45:0/3862701959 >> v2:172.21.15.117:3300/0 conn(0x7fe8e80cf930 0x7fe8e80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.130+0000 7fe8e5e00640 1 -- 172.21.15.45:0/3862701959 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e806cc30 con 0x7fe8e815bc40 2024-04-25T10:32:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8e5e00640 1 --2- 172.21.15.45:0/3862701959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e806c090 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80b3f80 tx=0x7fe8d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502119d799cc1795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8e5400640 1 -- 172.21.15.45:0/3862701959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80b0070 con 0x7fe8e815bc40 2024-04-25T10:32:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8e5400640 1 -- 172.21.15.45:0/3862701959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d80ab600 con 0x7fe8e815bc40 2024-04-25T10:32:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8e5400640 1 -- 172.21.15.45:0/3862701959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80ab9d0 con 0x7fe8e815bc40 2024-04-25T10:32:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 -- 172.21.15.45:0/3862701959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 msgr2=0x7fe8e806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/3862701959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e806c090 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80b3f80 tx=0x7fe8d801c520 comp rx=0 tx=0).stop 2024-04-25T10:32:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 -- 172.21.15.45:0/3862701959 shutdown_connections 2024-04-25T10:32:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/3862701959 >> v2:172.21.15.117:3300/0 conn(0x7fe8e80cf930 0x7fe8e80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/3862701959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e80b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/3862701959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e806c090 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 -- 172.21.15.45:0/3862701959 >> 172.21.15.45:0/3862701959 conn(0x7fe8e8066b70 msgr2=0x7fe8e8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 -- 172.21.15.45:0/3862701959 shutdown_connections 2024-04-25T10:32:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 -- 172.21.15.45:0/3862701959 wait complete. 2024-04-25T10:32:02.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.134+0000 7fe8ecc00640 1 Processor -- start 2024-04-25T10:32:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- start start 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e815d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e80cf930 0x7fe8e80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e80b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8160230 con 0x7fe8e815bc40 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81600b0 con 0x7fe8e80cf930 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81603b0 con 0x7fe8e80cda10 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e80b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e80b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35746/0 (socket says 172.21.15.45:35746) 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 -- 172.21.15.45:0/2638695728 learned_addr learned my addr 172.21.15.45:0/2638695728 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8dd400640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e80cf930 0x7fe8e80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e5e00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 -- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 msgr2=0x7fe8e815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 -- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e80cf930 msgr2=0x7fe8e80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e80cf930 0x7fe8e80ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 -- 172.21.15.45:0/2638695728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e815c2c0 con 0x7fe8e815bc40 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8e6800640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e80b0f20 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007e60 tx=0x7fe8e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0014420 con 0x7fe8e815bc40 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0019070 con 0x7fe8e815bc40 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0014820 con 0x7fe8e815bc40 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e815c5a0 con 0x7fe8e815bc40 2024-04-25T10:32:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e815cb60 con 0x7fe8e815bc40 2024-04-25T10:32:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.138+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e804ef30 con 0x7fe8e815bc40 2024-04-25T10:32:02.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.142+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8e00149c0 con 0x7fe8e815bc40 2024-04-25T10:32:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.142+0000 7fe8dea00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8c4066f60 0x7fe8c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.142+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8e0085810 con 0x7fe8e815bc40 2024-04-25T10:32:02.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.142+0000 7fe8e5e00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8c4066f60 0x7fe8c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.142+0000 7fe8e5e00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8c4066f60 0x7fe8c4069420 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80ad000 tx=0x7fe8d80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.146+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8e0056130 con 0x7fe8e815bc40 2024-04-25T10:32:02.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8e806d960 con 0x7fe8e815bc40 2024-04-25T10:32:02.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8dea00640 1 -- 172.21.15.45:0/2638695728 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v140) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8e005a1d0 con 0x7fe8e815bc40 2024-04-25T10:32:02.344 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8c4066f60 msgr2=0x7fe8c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8c4066f60 0x7fe8c4069420 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe8d80ad000 tx=0x7fe8d80a6000 comp rx=0 tx=0).stop 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 msgr2=0x7fe8e80b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e80b0f20 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007e60 tx=0x7fe8e000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 shutdown_connections 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe8c4066f60 0x7fe8c4069420 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8e815bc40 0x7fe8e80b0f20 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8e80cf930 0x7fe8e80ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 --2- 172.21.15.45:0/2638695728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8e80cda10 0x7fe8e815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:02.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 >> 172.21.15.45:0/2638695728 conn(0x7fe8e8066b70 msgr2=0x7fe8e8158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:02.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 shutdown_connections 2024-04-25T10:32:02.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:02.342+0000 7fe8ecc00640 1 -- 172.21.15.45:0/2638695728 wait complete. 2024-04-25T10:32:02.365 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:32:02.365 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:32:02.366 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:32:02.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.366 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:32:02.420 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:02.421 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T10:32:02.421 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:32:02.421 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:02.421 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T10:32:02.421 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:32:02.421 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41618sec preferred_lft 41618sec 2024-04-25T10:32:02.422 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-04-25T10:32:02.422 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.422 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:32:02.422 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:02.422 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T10:32:02.422 INFO:teuthology.orchestra.run.smithi045.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:32:02.423 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 02:42:a0:5c:86:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:02.423 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:32:02.423 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.423 INFO:teuthology.orchestra.run.smithi045.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:32:02.423 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:32:02.423 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:32:02.424 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:32:02.425 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:32:02.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.426 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:32:02.481 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T10:32:02.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.483 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T10:32:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T10:32:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:32:02.536 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T10:32:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:32:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T10:32:02.537 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:32:02.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.537 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec bash -c ' 2024-04-25T10:32:02.537 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T10:32:02.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:32:02.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:32:02.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:32:02.537 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T10:32:02.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.629 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:32:02.638 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:32:02.638 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:32:02.639 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:32:02.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T10:32:02.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:32:02.703 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:02.754 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:32:02.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:32:02.755 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:02.809 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T10:32:02.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:32:02.900 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:32:02.900 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:32:02.913 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T10:32:02.914 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T10:32:02.914 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.117:3300,172.21.15.117:3301" 2024-04-25T10:32:02.914 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@d7e12072-0435-4dda-862b-d1045153027e.cephfs=/ 2024-04-25T10:32:02.914 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.117:3300/172.21.15.117:3301" will pass to kernel 2024-04-25T10:32:02.918 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:32:02.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:02.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:02.940 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:32:02.998 INFO:teuthology.orchestra.run.smithi045.stdout:client11448 2024-04-25T10:32:02.999 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:32:02.999 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:32:03.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:03.051 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T10:32:03.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:03.110 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T10:32:03.160 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:32:03.161 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:32:03.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:03.212 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:32:03.353 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2024-04-25T10:32:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2024-04-25T10:32:03.354 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0907944 s, 69.3 MB/s 2024-04-25T10:32:03.355 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:32:03.355 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:32:03.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:03.364 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:32:03.479 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2024-04-25T10:32:03.479 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2024-04-25T10:32:03.479 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0641043 s, 98.1 MB/s 2024-04-25T10:32:03.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:32:03.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:32:03.525 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T10:32:03.525 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-04-25T10:32:03.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-04-25T10:32:03.527 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.528 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:32:03.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-04-25T10:32:03.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-04-25T10:32:03.531 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = bitmap 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-04-25T10:32:03.532 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-04-25T10:32:03.533 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:32:03.534 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-04-25T10:32:03.535 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-04-25T10:32:03.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds cache max size = 1000 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = False 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds log max segments = 1 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = False 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.537 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.538 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.539 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:32:03.540 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-04-25T10:32:03.541 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.542 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-04-25T10:32:03.543 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore allocator = bitmap 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 1/20 2024-04-25T10:32:03.544 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 1/20 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 4/10 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-04-25T10:32:03.545 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-04-25T10:32:03.546 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.547 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-04-25T10:32:03.548 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-04-25T10:32:03.549 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: mds cache max size = 1000 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = False 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: mds log max segments = 1 2024-04-25T10:32:03.550 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-04-25T10:32:03.551 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = False 2024-04-25T10:32:03.551 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:32:03.551 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.551 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:32:03.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:32:03.639 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-04-25T10:32:03.639 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-04-25T10:32:03.639 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:32:03.639 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-04-25T10:32:03.640 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:32:03.641 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on skipped message = False 2024-04-25T10:32:03.642 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-04-25T10:32:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.644 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: bdev async discard = True 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: bdev enable discard = True 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore allocator = bitmap 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 1/20 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 1/20 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-04-25T10:32:03.645 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 4/10 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.646 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.647 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-25T10:32:03.648 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = "" 2024-04-25T10:32:03.649 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: mds cache max size = 1000 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2024-04-25T10:32:03.650 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = False 2024-04-25T10:32:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: mds log max segments = 1 2024-04-25T10:32:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2024-04-25T10:32:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = False 2024-04-25T10:32:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2024-04-25T10:32:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.776 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T10:32:03.776 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-04-25T10:32:03.776 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:32:03.776 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-04-25T10:32:03.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.777 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:32:03.778 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-04-25T10:32:03.779 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-04-25T10:32:03.780 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = d7e12072-0435-4dda-862b-d1045153027e 2024-04-25T10:32:03.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.782 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.783 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.784 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: bdev async discard = True 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: bdev enable discard = True 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore allocator = bitmap 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-04-25T10:32:03.785 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-04-25T10:32:03.786 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:32:03.787 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.788 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-04-25T10:32:03.789 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:32:03.790 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:03.791 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-04-25T10:32:03.792 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout: mds cache max size = 1000 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-04-25T10:32:03.793 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = False 2024-04-25T10:32:03.794 INFO:teuthology.orchestra.run.smithi045.stdout: mds log max segments = 1 2024-04-25T10:32:03.794 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-04-25T10:32:03.794 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = False 2024-04-25T10:32:03.794 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-04-25T10:32:03.794 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-04-25T10:32:03.794 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-04-25T10:32:03.795 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:32:03.795 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:32:03.804 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:32:03.804 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:32:03.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:32:04.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce2200640 1 Processor -- start 2024-04-25T10:32:04.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce2200640 1 -- start start 2024-04-25T10:32:04.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce2200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efcdc0cfac0 0x7efcdc165770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce2200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc06d860 con 0x7efcdc071140 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce2200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc06db40 con 0x7efcdc0b5470 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce2200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcdc15b1a0 con 0x7efcdc0cfac0 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efcdbe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efcdc071140 0x7efcdc06cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47176/0 (socket says 172.21.15.45:47176) 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efcdbe00640 1 -- 172.21.15.45:0/1088110775 learned_addr learned my addr 172.21.15.45:0/1088110775 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce1800640 1 --2- 172.21.15.45:0/1088110775 >> v2:172.21.15.117:3300/0 conn(0x7efcdc0cfac0 0x7efcdc165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efcdb400640 1 -- 172.21.15.45:0/1088110775 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819693065 0 0) 0x7efcdc06d860 con 0x7efcdc071140 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efcdb400640 1 -- 172.21.15.45:0/1088110775 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc4003530 con 0x7efcdc071140 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce1800640 1 -- 172.21.15.45:0/1088110775 >> v1:172.21.15.117:6789/0 conn(0x7efcdc0b5470 legacy=0x7efcdc070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce1800640 1 -- 172.21.15.45:0/1088110775 >> v1:172.21.15.45:6789/0 conn(0x7efcdc071140 legacy=0x7efcdc06cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.046+0000 7efce1800640 1 -- 172.21.15.45:0/1088110775 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc06de20 con 0x7efcdc0cfac0 2024-04-25T10:32:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce1800640 1 --2- 172.21.15.45:0/1088110775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc165770 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009d90 tx=0x7efcd401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=745a9e41012ec162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efcdb400640 1 -- 172.21.15.45:0/1088110775 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd40b2070 con 0x7efcdc0cfac0 2024-04-25T10:32:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efcdb400640 1 -- 172.21.15.45:0/1088110775 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd40a5a00 con 0x7efcdc0cfac0 2024-04-25T10:32:04.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efcdb400640 1 -- 172.21.15.45:0/1088110775 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd40a5dd0 con 0x7efcdc0cfac0 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 -- 172.21.15.45:0/1088110775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 msgr2=0x7efcdc165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 --2- 172.21.15.45:0/1088110775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc165770 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009d90 tx=0x7efcd401c940 comp rx=0 tx=0).stop 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 -- 172.21.15.45:0/1088110775 shutdown_connections 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 --2- 172.21.15.45:0/1088110775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc165770 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 -- 172.21.15.45:0/1088110775 >> 172.21.15.45:0/1088110775 conn(0x7efcdc0b1c30 msgr2=0x7efcdc0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 -- 172.21.15.45:0/1088110775 shutdown_connections 2024-04-25T10:32:04.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.050+0000 7efce2200640 1 -- 172.21.15.45:0/1088110775 wait complete. 2024-04-25T10:32:04.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 Processor -- start 2024-04-25T10:32:04.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- start start 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efcdc071140 0x7efcdc1f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 0x7efcdc13db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc13c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0d0060 con 0x7efcdc0b5470 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0cfee0 con 0x7efcdc0cfac0 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0d01e0 con 0x7efcdc071140 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 0x7efcdc13db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 0x7efcdc13db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35750/0 (socket says 172.21.15.45:35750) 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 -- 172.21.15.45:0/989874732 learned_addr learned my addr 172.21.15.45:0/989874732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce1800640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc13c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce0e00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efcdc071140 0x7efcdc1f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 -- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efcdc071140 msgr2=0x7efcdc1f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efcdc071140 0x7efcdc1f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 -- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 msgr2=0x7efcdc13c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc13c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 -- 172.21.15.45:0/989874732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc1f7690 con 0x7efcdc0b5470 2024-04-25T10:32:04.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcdbe00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 0x7efcdc13db50 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7efcc800c500 tx=0x7efcc800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8017070 con 0x7efcdc0b5470 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc8005cf0 con 0x7efcdc0b5470 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc13cab0 con 0x7efcdc0b5470 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80072f0 con 0x7efcdc0b5470 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc13d070 con 0x7efcdc0b5470 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.054+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc075120 con 0x7efcdc0b5470 2024-04-25T10:32:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.058+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcc801a070 con 0x7efcdc0b5470 2024-04-25T10:32:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.058+0000 7efcd8c00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.058+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcc8084410 con 0x7efcdc0b5470 2024-04-25T10:32:04.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.058+0000 7efce0e00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.058+0000 7efce0e00640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efcbc066f60 0x7efcbc069420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efcdc159a20 tx=0x7efcd0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.062+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcdc1f7690 con 0x7efcdc0b5470 2024-04-25T10:32:04.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.226+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcdc06cdb0 con 0x7efcdc0b5470 2024-04-25T10:32:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.226+0000 7efcd8c00640 1 -- 172.21.15.45:0/989874732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16671 (secure 0 0 0) 0x7efcc8054d40 con 0x7efcdc0b5470 2024-04-25T10:32:04.229 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:04.229 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":193,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:31:57.021531+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.226+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efcbc066f60 msgr2=0x7efcbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.226+0000 7efce2200640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efcbc066f60 0x7efcbc069420 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efcdc159a20 tx=0x7efcd0009040 comp rx=0 tx=0).stop 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.226+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 msgr2=0x7efcdc13db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.226+0000 7efce2200640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 0x7efcdc13db50 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7efcc800c500 tx=0x7efcc800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 shutdown_connections 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcdc0cfac0 0x7efcdc13c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcdc0b5470 0x7efcdc13db50 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 --2- 172.21.15.45:0/989874732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efcdc071140 0x7efcdc1f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 >> 172.21.15.45:0/989874732 conn(0x7efcdc0b1c30 msgr2=0x7efcdc0b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 shutdown_connections 2024-04-25T10:32:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.230+0000 7efce2200640 1 -- 172.21.15.45:0/989874732 wait complete. 2024-04-25T10:32:04.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:31:53.058645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:31:54.409004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:32:04.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff848400640 1 Processor -- start 2024-04-25T10:32:04.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff848400640 1 -- start start 2024-04-25T10:32:04.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff848400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff840066fe0 0x7ff840161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff848400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400b40c0 con 0x7ff8400b5560 2024-04-25T10:32:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff848400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400b5b00 con 0x7ff840066c00 2024-04-25T10:32:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff848400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff840161d60 con 0x7ff840066fe0 2024-04-25T10:32:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff846400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff8400b5560 0x7ff8400ae850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47188/0 (socket says 172.21.15.45:47188) 2024-04-25T10:32:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff846400640 1 -- 172.21.15.45:0/1263803883 learned_addr learned my addr 172.21.15.45:0/1263803883 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:04.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff845000640 1 --2- 172.21.15.45:0/1263803883 >> v2:172.21.15.117:3300/0 conn(0x7ff840066fe0 0x7ff840161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff83fe00640 1 -- 172.21.15.45:0/1263803883 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210902011 0 0) 0x7ff8400b40c0 con 0x7ff8400b5560 2024-04-25T10:32:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff83fe00640 1 -- 172.21.15.45:0/1263803883 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff828003530 con 0x7ff8400b5560 2024-04-25T10:32:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff845000640 1 -- 172.21.15.45:0/1263803883 >> v1:172.21.15.117:6789/0 conn(0x7ff840066c00 legacy=0x7ff840072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff83fe00640 1 -- 172.21.15.45:0/1263803883 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423377928 0 0) 0x7ff8400b5b00 con 0x7ff840066c00 2024-04-25T10:32:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff845000640 1 -- 172.21.15.45:0/1263803883 >> v1:172.21.15.45:6789/0 conn(0x7ff8400b5560 legacy=0x7ff8400ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.394+0000 7ff845000640 1 -- 172.21.15.45:0/1263803883 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400b3a00 con 0x7ff840066fe0 2024-04-25T10:32:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff845000640 1 --2- 172.21.15.45:0/1263803883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066fe0 0x7ff840161820 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0097c0 tx=0x7ff82c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdb14c688f8f679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff83fe00640 1 -- 172.21.15.45:0/1263803883 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0b0070 con 0x7ff840066fe0 2024-04-25T10:32:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff83fe00640 1 -- 172.21.15.45:0/1263803883 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c0ab600 con 0x7ff840066fe0 2024-04-25T10:32:04.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff83fe00640 1 -- 172.21.15.45:0/1263803883 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0ab9d0 con 0x7ff840066fe0 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 -- 172.21.15.45:0/1263803883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066fe0 msgr2=0x7ff840161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 --2- 172.21.15.45:0/1263803883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066fe0 0x7ff840161820 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0097c0 tx=0x7ff82c01c380 comp rx=0 tx=0).stop 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 -- 172.21.15.45:0/1263803883 shutdown_connections 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 --2- 172.21.15.45:0/1263803883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066fe0 0x7ff840161820 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 -- 172.21.15.45:0/1263803883 >> 172.21.15.45:0/1263803883 conn(0x7ff8400b2560 msgr2=0x7ff8400b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 -- 172.21.15.45:0/1263803883 shutdown_connections 2024-04-25T10:32:04.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.398+0000 7ff848400640 1 -- 172.21.15.45:0/1263803883 wait complete. 2024-04-25T10:32:04.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 Processor -- start 2024-04-25T10:32:04.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 -- start start 2024-04-25T10:32:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066c00 0x7ff840139330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff840066fe0 0x7ff8401514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 0x7ff8400b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84015e540 con 0x7ff8400b5560 2024-04-25T10:32:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff840158620 con 0x7ff840066c00 2024-04-25T10:32:04.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84015e6c0 con 0x7ff840066fe0 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 0x7ff8400b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 0x7ff8400b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35752/0 (socket says 172.21.15.45:35752) 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 -- 172.21.15.45:0/4115248301 learned_addr learned my addr 172.21.15.45:0/4115248301 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff845a00640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066c00 0x7ff840139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff845000640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff840066fe0 0x7ff8401514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 -- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff840066fe0 msgr2=0x7ff8401514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff840066fe0 0x7ff8401514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 -- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066c00 msgr2=0x7ff840139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066c00 0x7ff840139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 -- 172.21.15.45:0/4115248301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84013a290 con 0x7ff8400b5560 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff845000640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff840066fe0 0x7ff8401514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff846400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 0x7ff8400b1550 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7ff83800cad0 tx=0x7ff83800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838017070 con 0x7ff8400b5560 2024-04-25T10:32:04.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83800ac30 con 0x7ff8400b5560 2024-04-25T10:32:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838012400 con 0x7ff8400b5560 2024-04-25T10:32:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff840138bd0 con 0x7ff8400b5560 2024-04-25T10:32:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.402+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff840139cd0 con 0x7ff8400b5560 2024-04-25T10:32:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.406+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84004ef30 con 0x7ff8400b5560 2024-04-25T10:32:04.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.406+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff83801a070 con 0x7ff8400b5560 2024-04-25T10:32:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.406+0000 7ff83d600640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff820066f60 0x7ff820069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.406+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff838084cd0 con 0x7ff8400b5560 2024-04-25T10:32:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.406+0000 7ff845a00640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff820066f60 0x7ff820069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.410+0000 7ff845a00640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff820066f60 0x7ff820069420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff834000f70 tx=0x7ff834009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.410+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff838055680 con 0x7ff8400b5560 2024-04-25T10:32:04.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.574+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff84016a9b0 con 0x7ff8400b5560 2024-04-25T10:32:04.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.574+0000 7ff83d600640 1 -- 172.21.15.45:0/4115248301 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff838059720 con 0x7ff8400b5560 2024-04-25T10:32:04.579 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff820066f60 msgr2=0x7ff820069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff820066f60 0x7ff820069420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7ff834000f70 tx=0x7ff834009040 comp rx=0 tx=0).stop 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 msgr2=0x7ff8400b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 0x7ff8400b1550 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7ff83800cad0 tx=0x7ff83800cf90 comp rx=0 tx=0).stop 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 shutdown_connections 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff820066f60 0x7ff820069420 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8400b5560 0x7ff8400b1550 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff840066fe0 0x7ff8401514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 --2- 172.21.15.45:0/4115248301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff840066c00 0x7ff840139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 >> 172.21.15.45:0/4115248301 conn(0x7ff8400b2560 msgr2=0x7ff840072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 shutdown_connections 2024-04-25T10:32:04.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.578+0000 7ff848400640 1 -- 172.21.15.45:0/4115248301 wait complete. 2024-04-25T10:32:04.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:32:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 Processor -- start 2024-04-25T10:32:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 -- start start 2024-04-25T10:32:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8800b25d0 0x7fd88006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88006aea0 con 0x7fd8800b2b00 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88006b460 con 0x7fd8800b25d0 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd886e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88006b180 con 0x7fd88015e000 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd877e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd8800b25d0 0x7fd88006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd884e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8800b2b00 0x7fd880161c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47200/0 (socket says 172.21.15.45:47200) 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd884e00640 1 -- 172.21.15.45:0/2159915835 learned_addr learned my addr 172.21.15.45:0/2159915835 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd87fe00640 1 --2- 172.21.15.45:0/2159915835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd877e00640 1 -- 172.21.15.45:0/2159915835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 msgr2=0x7fd88006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd877e00640 1 --2- 172.21.15.45:0/2159915835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88006b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd877e00640 1 -- 172.21.15.45:0/2159915835 >> v1:172.21.15.45:6789/0 conn(0x7fd8800b2b00 legacy=0x7fd880161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd877e00640 1 -- 172.21.15.45:0/2159915835 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800b1de0 con 0x7fd8800b25d0 2024-04-25T10:32:04.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.742+0000 7fd87f400640 1 -- 172.21.15.45:0/2159915835 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386952220 0 0) 0x7fd88006aea0 con 0x7fd8800b2b00 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd877e00640 1 --2- 172.21.15.45:0/2159915835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b25d0 0x7fd88006bbe0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fd8680097c0 tx=0x7fd86801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5322245a807a2d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd87f400640 1 -- 172.21.15.45:0/2159915835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680b1070 con 0x7fd8800b25d0 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd87f400640 1 -- 172.21.15.45:0/2159915835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8680ac640 con 0x7fd8800b25d0 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd87f400640 1 -- 172.21.15.45:0/2159915835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680aca10 con 0x7fd8800b25d0 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 -- 172.21.15.45:0/2159915835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b25d0 msgr2=0x7fd88006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 --2- 172.21.15.45:0/2159915835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b25d0 0x7fd88006bbe0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fd8680097c0 tx=0x7fd86801c3e0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 -- 172.21.15.45:0/2159915835 shutdown_connections 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 --2- 172.21.15.45:0/2159915835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b25d0 0x7fd88006bbe0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 --2- 172.21.15.45:0/2159915835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 -- 172.21.15.45:0/2159915835 >> 172.21.15.45:0/2159915835 conn(0x7fd8800b3430 msgr2=0x7fd8800b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 -- 172.21.15.45:0/2159915835 shutdown_connections 2024-04-25T10:32:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 -- 172.21.15.45:0/2159915835 wait complete. 2024-04-25T10:32:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 Processor -- start 2024-04-25T10:32:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.746+0000 7fd886e00640 1 -- start start 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8800b25d0 0x7fd880151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 0x7fd8800b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88016b0d0 con 0x7fd8800b2b00 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88016af50 con 0x7fd8800b25d0 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88016b250 con 0x7fd88015e000 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 0x7fd8800b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 0x7fd8800b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35758/0 (socket says 172.21.15.45:35758) 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 -- 172.21.15.45:0/231221034 learned_addr learned my addr 172.21.15.45:0/231221034 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd87fe00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8800b25d0 0x7fd880151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd884e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 -- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 msgr2=0x7fd88015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 -- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8800b25d0 msgr2=0x7fd880151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8800b25d0 0x7fd880151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 -- 172.21.15.45:0/231221034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880152680 con 0x7fd8800b2b00 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd884e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd877e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 0x7fd8800b56b0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fd86801a010 tx=0x7fd868019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680b9020 con 0x7fd8800b2b00 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8680b5410 con 0x7fd8800b2b00 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680b1040 con 0x7fd8800b2b00 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880152960 con 0x7fd8800b2b00 2024-04-25T10:32:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880152f20 con 0x7fd8800b2b00 2024-04-25T10:32:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.750+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88004ef30 con 0x7fd8800b2b00 2024-04-25T10:32:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.754+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8680bc020 con 0x7fd8800b2b00 2024-04-25T10:32:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.754+0000 7fd87cc00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd85c066f60 0x7fd85c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.754+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd86811ec70 con 0x7fd8800b2b00 2024-04-25T10:32:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.754+0000 7fd87fe00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd85c066f60 0x7fd85c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.754+0000 7fd87fe00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd85c066f60 0x7fd85c069420 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd870009b60 tx=0x7fd870009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.758+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8680c3030 con 0x7fd8800b2b00 2024-04-25T10:32:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8800b78a0 con 0x7fd8800b2b00 2024-04-25T10:32:04.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd87cc00640 1 -- 172.21.15.45:0/231221034 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd868016090 con 0x7fd8800b2b00 2024-04-25T10:32:04.925 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd85c066f60 msgr2=0x7fd85c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd85c066f60 0x7fd85c069420 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd870009b60 tx=0x7fd870009190 comp rx=0 tx=0).stop 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 msgr2=0x7fd8800b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 0x7fd8800b56b0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fd86801a010 tx=0x7fd868019180 comp rx=0 tx=0).stop 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 shutdown_connections 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd85c066f60 0x7fd85c069420 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015e000 0x7fd88015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8800b2b00 0x7fd8800b56b0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 --2- 172.21.15.45:0/231221034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8800b25d0 0x7fd880151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 >> 172.21.15.45:0/231221034 conn(0x7fd8800b3430 msgr2=0x7fd8800cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:04.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 shutdown_connections 2024-04-25T10:32:04.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:04.922+0000 7fd886e00640 1 -- 172.21.15.45:0/231221034 wait complete. 2024-04-25T10:32:04.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:32:05.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.086+0000 7f9ad0800640 1 Processor -- start 2024-04-25T10:32:05.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.086+0000 7f9ad0800640 1 -- start start 2024-04-25T10:32:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ad0800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 0x7f9ac815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ad0800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ac815f380 0x7f9ac80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ad0800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac8160650 con 0x7f9ac815fce0 2024-04-25T10:32:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ad0800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac8160930 con 0x7f9ac815f380 2024-04-25T10:32:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ad0800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80cd0e0 con 0x7f9ac81600c0 2024-04-25T10:32:05.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9ac815fce0 0x7f9ac80b2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48224/0 (socket says 172.21.15.45:48224) 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 -- 172.21.15.45:0/678960633 learned_addr learned my addr 172.21.15.45:0/678960633 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acd400640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 0x7f9ac815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ace800640 1 --2- 172.21.15.45:0/678960633 >> v2:172.21.15.117:3300/0 conn(0x7f9ac815f380 0x7f9ac80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990706769 0 0) 0x7f9ac8160650 con 0x7f9ac815fce0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab0003530 con 0x7f9ac815fce0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 69292164 0 0) 0x7f9ab0003530 con 0x7f9ac815fce0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 msgr2=0x7f9ac815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 0x7f9ac815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 >> v2:172.21.15.117:3300/0 conn(0x7f9ac815f380 msgr2=0x7f9ac80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 --2- 172.21.15.45:0/678960633 >> v2:172.21.15.117:3300/0 conn(0x7f9ac815f380 0x7f9ac80cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80cd3c0 con 0x7f9ac815fce0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f9abc001680 con 0x7f9ac815fce0 2024-04-25T10:32:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 >> v1:172.21.15.45:6789/0 conn(0x7f9ac815fce0 legacy=0x7f9ac80b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab00039c0 0x7f9ab0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 0x7f9ab0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab0004d50 0x7f9ab00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8160930 con 0x7f9ab00039c0 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80cd0e0 con 0x7f9ab0004d50 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab000b9d0 con 0x7f9ac81600c0 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab00039c0 0x7f9ab0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acd400640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 0x7f9ab0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9ace800640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab0004d50 0x7f9ab00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 -- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 msgr2=0x7f9ab0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac81600c0 0x7f9ab0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 -- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab0004d50 msgr2=0x7f9ab00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab0004d50 0x7f9ab00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.090+0000 7f9acde00640 1 -- 172.21.15.45:0/678960633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0003530 con 0x7f9ab00039c0 2024-04-25T10:32:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9acde00640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab00039c0 0x7f9ab0003de0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c2f0 tx=0x7f9abc01cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475b3aa8cf647dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc007d60 con 0x7f9ab00039c0 2024-04-25T10:32:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc0a9020 con 0x7f9ab00039c0 2024-04-25T10:32:05.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9acca00640 1 -- 172.21.15.45:0/678960633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc005ae0 con 0x7f9ab00039c0 2024-04-25T10:32:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 -- 172.21.15.45:0/678960633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab00039c0 msgr2=0x7f9ab0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 --2- 172.21.15.45:0/678960633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab00039c0 0x7f9ab0003de0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c2f0 tx=0x7f9abc01cd10 comp rx=0 tx=0).stop 2024-04-25T10:32:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ace800640 1 -- 172.21.15.45:0/678960633 reap_dead start 2024-04-25T10:32:05.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 -- 172.21.15.45:0/678960633 shutdown_connections 2024-04-25T10:32:05.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 -- 172.21.15.45:0/678960633 >> 172.21.15.45:0/678960633 conn(0x7f9ac80b3430 msgr2=0x7f9ac80b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 -- 172.21.15.45:0/678960633 shutdown_connections 2024-04-25T10:32:05.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 -- 172.21.15.45:0/678960633 wait complete. 2024-04-25T10:32:05.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.094+0000 7f9ad0800640 1 Processor -- start 2024-04-25T10:32:05.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- start start 2024-04-25T10:32:05.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab00039c0 0x7f9ac81f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac815e100 0x7f9ac80ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 0x7f9ac80b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b2cd0 con 0x7f9ac80c9b90 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b2b50 con 0x7f9ab00039c0 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80d0340 con 0x7f9ac815e100 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 0x7f9ac80b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 0x7f9ac80b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47536/0 (socket says 172.21.15.45:47536) 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 -- 172.21.15.45:0/2409394776 learned_addr learned my addr 172.21.15.45:0/2409394776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9acde00640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab00039c0 0x7f9ac81f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9acd400640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac815e100 0x7f9ac80ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 -- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac815e100 msgr2=0x7f9ac80ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac815e100 0x7f9ac80ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 -- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab00039c0 msgr2=0x7f9ac81f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab00039c0 0x7f9ac81f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 -- 172.21.15.45:0/2409394776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac81fa400 con 0x7f9ac80c9b90 2024-04-25T10:32:05.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ace800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 0x7f9ac80b5860 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8071a20 tx=0x7f9ac400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400fce0 con 0x7f9ac80c9b90 2024-04-25T10:32:05.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac401d070 con 0x7f9ac80c9b90 2024-04-25T10:32:05.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac81fa6e0 con 0x7f9ac80c9b90 2024-04-25T10:32:05.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4015670 con 0x7f9ac80c9b90 2024-04-25T10:32:05.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80aecc0 con 0x7f9ac80c9b90 2024-04-25T10:32:05.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.098+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac804ef30 con 0x7f9ac80c9b90 2024-04-25T10:32:05.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.102+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ac4015810 con 0x7f9ac80c9b90 2024-04-25T10:32:05.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.102+0000 7f9ac2000640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9aac066f90 0x7f9aac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.102+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ac40857d0 con 0x7f9ac80c9b90 2024-04-25T10:32:05.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.102+0000 7f9acde00640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9aac066f90 0x7f9aac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.102+0000 7f9acde00640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9aac066f90 0x7f9aac069450 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0aa000 tx=0x7f9abc001a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.106+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ac4010040 con 0x7f9ac80c9b90 2024-04-25T10:32:05.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.270+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ac80b78a0 con 0x7f9ac80c9b90 2024-04-25T10:32:05.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.270+0000 7f9ac2000640 1 -- 172.21.15.45:0/2409394776 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ac4056740 con 0x7f9ac80c9b90 2024-04-25T10:32:05.273 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9aac066f90 msgr2=0x7f9aac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9aac066f90 0x7f9aac069450 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0aa000 tx=0x7f9abc001a50 comp rx=0 tx=0).stop 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 msgr2=0x7f9ac80b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 0x7f9ac80b5860 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8071a20 tx=0x7f9ac400f1b0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 shutdown_connections 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9aac066f90 0x7f9aac069450 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ac80c9b90 0x7f9ac80b5860 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ac815e100 0x7f9ac80ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 --2- 172.21.15.45:0/2409394776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab00039c0 0x7f9ac81f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 >> 172.21.15.45:0/2409394776 conn(0x7f9ac80b3430 msgr2=0x7f9ac806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 shutdown_connections 2024-04-25T10:32:05.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.274+0000 7f9ad0800640 1 -- 172.21.15.45:0/2409394776 wait complete. 2024-04-25T10:32:05.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:32:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61e7000640 1 Processor -- start 2024-04-25T10:32:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61e7000640 1 -- start start 2024-04-25T10:32:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61e7000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61e015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61e7000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e0071050 con 0x7f61e01550b0 2024-04-25T10:32:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61e7000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e015d040 con 0x7f61e015b740 2024-04-25T10:32:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61e7000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00712f0 con 0x7f61e00d0040 2024-04-25T10:32:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61dfe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f61e01550b0 0x7f61e00cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48232/0 (socket says 172.21.15.45:48232) 2024-04-25T10:32:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61dfe00640 1 -- 172.21.15.45:0/1207458434 learned_addr learned my addr 172.21.15.45:0/1207458434 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61d7400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61e015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504839611 0 0) 0x7f61e0071050 con 0x7f61e01550b0 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c4005740 con 0x7f61e01550b0 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370671254 0 0) 0x7f61e015d040 con 0x7f61e015b740 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e0071050 con 0x7f61e015b740 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927226648 0 0) 0x7f61c4005740 con 0x7f61e01550b0 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 msgr2=0x7f61e015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61e015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 >> v1:172.21.15.117:6789/0 conn(0x7f61e015b740 legacy=0x7f61e00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e00715d0 con 0x7f61e01550b0 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f61d0001680 con 0x7f61e01550b0 2024-04-25T10:32:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61d7400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61e015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 >> v1:172.21.15.45:6789/0 conn(0x7f61e01550b0 legacy=0x7f61e00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61c4005d90 0x7f61c4006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61c4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61c40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00712f0 con 0x7f61c4006a20 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c400dc40 con 0x7f61c4005d90 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.434+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c400de40 con 0x7f61e00d0040 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61c4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61dfe00640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61c4005d90 0x7f61c4006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61e5000640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61c40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 -- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 msgr2=0x7f61c40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61c40073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 -- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61c4005d90 msgr2=0x7f61c4006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61c4005d90 0x7f61c4006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 -- 172.21.15.45:0/1207458434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4005740 con 0x7f61c4006a20 2024-04-25T10:32:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61dfe00640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61c4005d90 0x7f61c4006190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61e5000640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e00d0040 0x7f61c40073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61d7400640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61c4006ea0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009c80 tx=0x7f61c801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da7481c4c43a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80b2070 con 0x7f61c4006a20 2024-04-25T10:32:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c80a5870 con 0x7f61c4006a20 2024-04-25T10:32:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.438+0000 7f61df400640 1 -- 172.21.15.45:0/1207458434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c8009880 con 0x7f61c4006a20 2024-04-25T10:32:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- 172.21.15.45:0/1207458434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 msgr2=0x7f61c4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 --2- 172.21.15.45:0/1207458434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61c4006ea0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009c80 tx=0x7f61c801cbb0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e5000640 1 -- 172.21.15.45:0/1207458434 reap_dead start 2024-04-25T10:32:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- 172.21.15.45:0/1207458434 shutdown_connections 2024-04-25T10:32:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- 172.21.15.45:0/1207458434 >> 172.21.15.45:0/1207458434 conn(0x7f61e0066b70 msgr2=0x7f61e0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- 172.21.15.45:0/1207458434 shutdown_connections 2024-04-25T10:32:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- 172.21.15.45:0/1207458434 wait complete. 2024-04-25T10:32:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 Processor -- start 2024-04-25T10:32:05.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- start start 2024-04-25T10:32:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61e0151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61e00cea70 0x7f61e01420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e006d540 0x7f61e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00720b0 con 0x7f61c4006a20 2024-04-25T10:32:05.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0071b90 con 0x7f61e00cea70 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e7000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e015bbf0 con 0x7f61e006d540 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61e0151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61e0151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47564/0 (socket says 172.21.15.45:47564) 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 -- 172.21.15.45:0/3850146769 learned_addr learned my addr 172.21.15.45:0/3850146769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61e5000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e006d540 0x7f61e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61d7400640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61e00cea70 0x7f61e01420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 -- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e006d540 msgr2=0x7f61e006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e006d540 0x7f61e006d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 -- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61e00cea70 msgr2=0x7f61e01420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61e00cea70 0x7f61e01420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.442+0000 7f61dfe00640 1 -- 172.21.15.45:0/3850146769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e00cf010 con 0x7f61c4006a20 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dfe00640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61e0151f50 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e7e0 tx=0x7f61d000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000c870 con 0x7f61c4006a20 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0018070 con 0x7f61c4006a20 2024-04-25T10:32:05.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d000cc10 con 0x7f61c4006a20 2024-04-25T10:32:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e0157a10 con 0x7f61c4006a20 2024-04-25T10:32:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e0157fd0 con 0x7f61c4006a20 2024-04-25T10:32:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e01581d0 con 0x7f61c4006a20 2024-04-25T10:32:05.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61d001d460 con 0x7f61c4006a20 2024-04-25T10:32:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dcc00640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61bc066f60 0x7f61bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61d0085020 con 0x7f61c4006a20 2024-04-25T10:32:05.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.446+0000 7f61d7400640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61bc066f60 0x7f61bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.450+0000 7f61d7400640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61bc066f60 0x7f61bc069420 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f61c80af000 tx=0x7f61c80046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.450+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61e01581d0 con 0x7f61c4006a20 2024-04-25T10:32:05.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.622+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61e004ef30 con 0x7f61c4006a20 2024-04-25T10:32:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61dcc00640 1 -- 172.21.15.45:0/3850146769 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v193) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61d0055f00 con 0x7f61c4006a20 2024-04-25T10:32:05.627 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-04-25T10:32:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61bc066f60 msgr2=0x7f61bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61bc066f60 0x7f61bc069420 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f61c80af000 tx=0x7f61c80046d0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 msgr2=0x7f61e0151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61e0151f50 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e7e0 tx=0x7f61d000eca0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 shutdown_connections 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f61bc066f60 0x7f61bc069420 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61e006d540 0x7f61e006d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61e00cea70 0x7f61e01420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 --2- 172.21.15.45:0/3850146769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61c4006a20 0x7f61e0151f50 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 >> 172.21.15.45:0/3850146769 conn(0x7f61e0066b70 msgr2=0x7f61e015d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 shutdown_connections 2024-04-25T10:32:05.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.626+0000 7f61e7000640 1 -- 172.21.15.45:0/3850146769 wait complete. 2024-04-25T10:32:05.648 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:32:05.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:05.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:32:05.703 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:32:05.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:05.703 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:32:05.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.918+0000 7f0335e00640 1 Processor -- start 2024-04-25T10:32:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335e00640 1 -- start start 2024-04-25T10:32:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f03301451d0 0x7f03300cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300b2dd0 0x7f033006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033015f5e0 con 0x7f03300ce070 2024-04-25T10:32:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033015f8c0 con 0x7f03301451d0 2024-04-25T10:32:05.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033015fba0 con 0x7f03300b2dd0 2024-04-25T10:32:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f03301451d0 0x7f03300cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f03301451d0 0x7f03300cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34568/0 (socket says 172.21.15.45:34568) 2024-04-25T10:32:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f032fe00640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f03300ce070 0x7f03300b26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:50056/0 (socket says 172.21.15.45:50056) 2024-04-25T10:32:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0335400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300b2dd0 0x7f033006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 -- 172.21.15.45:0/810705213 learned_addr learned my addr 172.21.15.45:0/810705213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 -- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300b2dd0 msgr2=0x7f033006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 --2- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300b2dd0 0x7f033006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 -- 172.21.15.45:0/810705213 >> v1:172.21.15.117:6789/0 conn(0x7f03300ce070 legacy=0x7f03300b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.922+0000 7f0334a00640 1 -- 172.21.15.45:0/810705213 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033015fe80 con 0x7f03301451d0 2024-04-25T10:32:05.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0334a00640 1 --2- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03301451d0 0x7f03300cdb30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0324009870 tx=0x7f032401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b135e53ecde6442d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f032f400640 1 -- 172.21.15.45:0/810705213 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240b0070 con 0x7f03301451d0 2024-04-25T10:32:05.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f032f400640 1 -- 172.21.15.45:0/810705213 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0324004230 con 0x7f03301451d0 2024-04-25T10:32:05.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f032f400640 1 -- 172.21.15.45:0/810705213 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324003620 con 0x7f03301451d0 2024-04-25T10:32:05.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 -- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03301451d0 msgr2=0x7f03300cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 --2- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03301451d0 0x7f03300cdb30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0324009870 tx=0x7f032401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 -- 172.21.15.45:0/810705213 shutdown_connections 2024-04-25T10:32:05.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 --2- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300b2dd0 0x7f033006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 --2- 172.21.15.45:0/810705213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03301451d0 0x7f03300cdb30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 -- 172.21.15.45:0/810705213 >> 172.21.15.45:0/810705213 conn(0x7f0330074310 msgr2=0x7f033006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:05.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 -- 172.21.15.45:0/810705213 shutdown_connections 2024-04-25T10:32:05.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 -- 172.21.15.45:0/810705213 wait complete. 2024-04-25T10:32:05.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.926+0000 7f0335e00640 1 Processor -- start 2024-04-25T10:32:05.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- start start 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03300b2dd0 0x7f033015e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300ce070 0x7f03300ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03301451d0 0x7f033015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033006e170 con 0x7f03301451d0 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033006dff0 con 0x7f03300b2dd0 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033006e2f0 con 0x7f03300ce070 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0334a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03300b2dd0 0x7f033015e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03301451d0 0x7f033015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0334a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03300b2dd0 0x7f033015e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34576/0 (socket says 172.21.15.45:34576) 2024-04-25T10:32:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f032fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300ce070 0x7f03300ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0334a00640 1 -- 172.21.15.45:0/4023547140 learned_addr learned my addr 172.21.15.45:0/4023547140 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 -- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300ce070 msgr2=0x7f03300ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300ce070 0x7f03300ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 -- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03300b2dd0 msgr2=0x7f033015e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03300b2dd0 0x7f033015e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 -- 172.21.15.45:0/4023547140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300adca0 con 0x7f03301451d0 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335400640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03301451d0 0x7f033015b1f0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f032800ebd0 tx=0x7f032800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328018020 con 0x7f03301451d0 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032800cd20 con 0x7f03301451d0 2024-04-25T10:32:05.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03300a97a0 con 0x7f03301451d0 2024-04-25T10:32:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280146c0 con 0x7f03301451d0 2024-04-25T10:32:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03300a9d60 con 0x7f03301451d0 2024-04-25T10:32:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.930+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0330075120 con 0x7f03301451d0 2024-04-25T10:32:05.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.934+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f032801b020 con 0x7f03301451d0 2024-04-25T10:32:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.934+0000 7f032cc00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0310066f60 0x7f0310069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.934+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0328085c00 con 0x7f03301451d0 2024-04-25T10:32:05.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.934+0000 7f0334a00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0310066f60 0x7f0310069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:05.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.934+0000 7f0334a00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0310066f60 0x7f0310069420 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f032401c510 tx=0x7f03240a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:05.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:05.938+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0328022030 con 0x7f03301451d0 2024-04-25T10:32:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03300cd520 con 0x7f03301451d0 2024-04-25T10:32:06.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f032cc00640 1 -- 172.21.15.45:0/4023547140 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4957 (secure 0 0 0) 0x7f0328010040 con 0x7f03301451d0 2024-04-25T10:32:06.137 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:06.137 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":140,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:31:58.443777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11304":{"gid":11304,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/3198476999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3198476999},{"type":"v1","addr":"172.21.15.45:6835","nonce":3198476999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:32:06.137 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 140 2024-04-25T10:32:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0310066f60 msgr2=0x7f0310069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0310066f60 0x7f0310069420 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f032401c510 tx=0x7f03240a6000 comp rx=0 tx=0).stop 2024-04-25T10:32:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03301451d0 msgr2=0x7f033015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03301451d0 0x7f033015b1f0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f032800ebd0 tx=0x7f032800c490 comp rx=0 tx=0).stop 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 shutdown_connections 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f0310066f60 0x7f0310069420 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03301451d0 0x7f033015b1f0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03300ce070 0x7f03300ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 --2- 172.21.15.45:0/4023547140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03300b2dd0 0x7f033015e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 >> 172.21.15.45:0/4023547140 conn(0x7f0330074310 msgr2=0x7f03300b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 shutdown_connections 2024-04-25T10:32:06.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.134+0000 7f0335e00640 1 -- 172.21.15.45:0/4023547140 wait complete. 2024-04-25T10:32:06.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:32:06.329 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2024-04-25T10:32:06.329 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:32:06.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:32:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.466+0000 7fcf24600640 1 Processor -- start 2024-04-25T10:32:06.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.466+0000 7fcf24600640 1 -- start start 2024-04-25T10:32:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf24600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 0x7fcf1c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf24600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c15a770 con 0x7fcf1c15eac0 2024-04-25T10:32:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf24600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c15e000 con 0x7fcf1c15a390 2024-04-25T10:32:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf24600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0ce550 con 0x7fcf1c0cdd90 2024-04-25T10:32:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf22600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcf1c15eac0 0x7fcf1c0afb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48236/0 (socket says 172.21.15.45:48236) 2024-04-25T10:32:06.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf22600640 1 -- 172.21.15.45:0/915295494 learned_addr learned my addr 172.21.15.45:0/915295494 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21200640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 0x7fcf1c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368400519 0 0) 0x7fcf1c15a770 con 0x7fcf1c15eac0 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf04004640 con 0x7fcf1c15eac0 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993211109 0 0) 0x7fcf04004640 con 0x7fcf1c15eac0 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 msgr2=0x7fcf1c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 0x7fcf1c0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 >> v1:172.21.15.117:6789/0 conn(0x7fcf1c15a390 legacy=0x7fcf1c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0ce270 con 0x7fcf1c15eac0 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fcf14001a00 con 0x7fcf1c15eac0 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 >> v1:172.21.15.45:6789/0 conn(0x7fcf1c15eac0 legacy=0x7fcf1c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf04004b00 0x7fcf04004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040054b0 0x7fcf04005980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 0x7fcf040061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0ce550 con 0x7fcf04004b00 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0400cae0 con 0x7fcf040054b0 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0400cce0 con 0x7fcf1c0cdd90 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21c00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf04004b00 0x7fcf04004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21200640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040054b0 0x7fcf04005980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf22600640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 0x7fcf040061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21c00640 1 -- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 msgr2=0x7fcf040061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21c00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf1c0cdd90 0x7fcf040061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21c00640 1 -- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040054b0 msgr2=0x7fcf04005980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21c00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf040054b0 0x7fcf04005980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.470+0000 7fcf21c00640 1 -- 172.21.15.45:0/915295494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04004640 con 0x7fcf04004b00 2024-04-25T10:32:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf21c00640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf04004b00 0x7fcf04004ee0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fcf10001860 tx=0x7fcf1001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfaed7e1213d641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100b0070 con 0x7fcf04004b00 2024-04-25T10:32:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100ab600 con 0x7fcf04004b00 2024-04-25T10:32:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf1be00640 1 -- 172.21.15.45:0/915295494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10001460 con 0x7fcf04004b00 2024-04-25T10:32:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf24600640 1 -- 172.21.15.45:0/915295494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf04004b00 msgr2=0x7fcf04004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf24600640 1 --2- 172.21.15.45:0/915295494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf04004b00 0x7fcf04004ee0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fcf10001860 tx=0x7fcf1001c520 comp rx=0 tx=0).stop 2024-04-25T10:32:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf22600640 1 -- 172.21.15.45:0/915295494 reap_dead start 2024-04-25T10:32:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf24600640 1 -- 172.21.15.45:0/915295494 shutdown_connections 2024-04-25T10:32:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf24600640 1 -- 172.21.15.45:0/915295494 >> 172.21.15.45:0/915295494 conn(0x7fcf1c0b1d50 msgr2=0x7fcf1c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf24600640 1 -- 172.21.15.45:0/915295494 shutdown_connections 2024-04-25T10:32:06.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.474+0000 7fcf24600640 1 -- 172.21.15.45:0/915295494 wait complete. 2024-04-25T10:32:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 Processor -- start 2024-04-25T10:32:06.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- start start 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf04004b00 0x7fcf1c0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 0x7fcf1c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf1c1f6c60 0x7fcf1c157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0b0620 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0b04a0 con 0x7fcf1c1f6c60 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0b07a0 con 0x7fcf04004b00 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 0x7fcf1c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 0x7fcf1c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47588/0 (socket says 172.21.15.45:47588) 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 -- 172.21.15.45:0/2820617275 learned_addr learned my addr 172.21.15.45:0/2820617275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 -- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf04004b00 msgr2=0x7fcf1c0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf04004b00 0x7fcf1c0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 -- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf1c1f6c60 msgr2=0x7fcf1c157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf1c1f6c60 0x7fcf1c157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 -- 172.21.15.45:0/2820617275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0691f0 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf21200640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 0x7fcf1c157990 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fcf080119d0 tx=0x7fcf08011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800fad0 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0800fc70 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c0cac40 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08014440 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c0c6a30 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.478+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c04ef30 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.482+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf08012040 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.482+0000 7fcf19600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcefc066f60 0x7fcefc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.482+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf080856b0 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.482+0000 7fcf21c00640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcefc066f60 0x7fcefc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.482+0000 7fcf21c00640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcefc066f60 0x7fcefc069420 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcf10001860 tx=0x7fcf10001240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.486+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf1c0cac40 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.682+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcf1c068b20 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.686 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:32:06.682+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:32:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.686+0000 7fcf19600640 1 -- 172.21.15.45:0/2820617275 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v141) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcf08019030 con 0x7fcf1c1f67e0 2024-04-25T10:32:06.689 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:32:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcefc066f60 msgr2=0x7fcefc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcefc066f60 0x7fcefc069420 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcf10001860 tx=0x7fcf10001240 comp rx=0 tx=0).stop 2024-04-25T10:32:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 msgr2=0x7fcf1c157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 0x7fcf1c157990 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7fcf080119d0 tx=0x7fcf08011e90 comp rx=0 tx=0).stop 2024-04-25T10:32:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 shutdown_connections 2024-04-25T10:32:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fcefc066f60 0x7fcefc069420 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf1c1f6c60 0x7fcf1c157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf1c1f67e0 0x7fcf1c157990 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 --2- 172.21.15.45:0/2820617275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf04004b00 0x7fcf1c0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 >> 172.21.15.45:0/2820617275 conn(0x7fcf1c0b1d50 msgr2=0x7fcf1c06ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 shutdown_connections 2024-04-25T10:32:06.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.690+0000 7fcf24600640 1 -- 172.21.15.45:0/2820617275 wait complete. 2024-04-25T10:32:06.708 INFO:tasks.ceph.mds.c.smithi045.stderr: -5288> 2024-04-25T10:32:01.994+0000 7f282cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:32:06.722 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T10:32:06.762 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:32:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 Processor -- start 2024-04-25T10:32:06.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- start start 2024-04-25T10:32:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effc13680 0x556effcb51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effbe2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effcb84a0 0x556effcbc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556effcba1f0 con 0x556effc2c520 2024-04-25T10:32:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556effb899f0 con 0x556effc13680 2024-04-25T10:32:06.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556effcba370 con 0x556effcb84a0 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effbe2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effbe2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47608/0 (socket says 172.21.15.45:47608) 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 -- 172.21.15.45:0/4238927379 learned_addr learned my addr 172.21.15.45:0/4238927379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5828000640 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effcb84a0 0x556effcbc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5827600640 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effc13680 0x556effcb51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 -- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effcb84a0 msgr2=0x556effcbc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effcb84a0 0x556effcbc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 -- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effc13680 msgr2=0x556effcb51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effc13680 0x556effcb51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 -- 172.21.15.45:0/4238927379 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556effcb4280 con 0x556effc2c520 2024-04-25T10:32:06.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5827600640 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effc13680 0x556effcb51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5826c00640 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effbe2300 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f581400b2f0 tx=0x7f581400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f613def14cf40a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5825800640 1 -- 172.21.15.45:0/4238927379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400d510 con 0x556effc2c520 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5825800640 1 -- 172.21.15.45:0/4238927379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814008070 con 0x556effc2c520 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f5825800640 1 -- 172.21.15.45:0/4238927379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581400d8d0 con 0x556effc2c520 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- 172.21.15.45:0/4238927379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 msgr2=0x556effbe2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effbe2300 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f581400b2f0 tx=0x7f581400b7b0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- 172.21.15.45:0/4238927379 shutdown_connections 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effcb84a0 0x556effcbc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effbe2300 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 --2- 172.21.15.45:0/4238927379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effc13680 0x556effcb51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- 172.21.15.45:0/4238927379 >> 172.21.15.45:0/4238927379 conn(0x556effc0dfe0 msgr2=0x556effc13a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- 172.21.15.45:0/4238927379 shutdown_connections 2024-04-25T10:32:06.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.770+0000 7f582a43a080 1 -- 172.21.15.45:0/4238927379 wait complete. 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 Processor -- start 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 -- start start 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effc13680 0x556effcb3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effcab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effcb84a0 0x556effcb2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556effcb47c0 con 0x556effc2c520 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556effcb4520 con 0x556effcb84a0 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556effcb4a60 con 0x556effc13680 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effcab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effcab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47612/0 (socket says 172.21.15.45:47612) 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 -- 172.21.15.45:0/1988140679 learned_addr learned my addr 172.21.15.45:0/1988140679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5827600640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effc13680 0x556effcb3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5828000640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effcb84a0 0x556effcb2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 -- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effc13680 msgr2=0x556effcb3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556effc13680 0x556effcb3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 -- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effcb84a0 msgr2=0x556effcb2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556effcb84a0 0x556effcb2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 -- 172.21.15.45:0/1988140679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556effcb2bd0 con 0x556effc2c520 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f5826c00640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effcab350 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f5814000c10 tx=0x7f5814000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f581fe00640 1 -- 172.21.15.45:0/1988140679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581401c440 con 0x556effc2c520 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f581fe00640 1 -- 172.21.15.45:0/1988140679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814021020 con 0x556effc2c520 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f581fe00640 1 -- 172.21.15.45:0/1988140679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581401c640 con 0x556effc2c520 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556effcb2e70 con 0x556effc2c520 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.774+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556effcb33b0 con 0x556effc2c520 2024-04-25T10:32:06.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f581fe00640 1 -- 172.21.15.45:0/1988140679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5814017070 con 0x556effc2c520 2024-04-25T10:32:06.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f581fe00640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5808065970 0x7f5808067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f581fe00640 1 -- 172.21.15.45:0/1988140679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f58140857c0 con 0x556effc2c520 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f5827600640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5808065970 0x7f5808067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f582a43a080 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x556effcab890 0x556effcbe720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 --> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] -- osd_op(unknown.0.0:1 28.10 28:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e194) v9 -- 0x556effcb26d0 con 0x556effcab890 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f5828000640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x556effcab890 0x556effcbe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f5827600640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5808065970 0x7f5808067e30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5818005880 tx=0x7f581800c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.778+0000 7f5828000640 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x556effcab890 0x556effcbe720 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f5828000640 1 -- 172.21.15.45:0/1988140679 <== osd.0 v2:172.21.15.45:6808/3877436580 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=535b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+535 (crc 0 0 0) 0x7f58200b3040 con 0x556effcab890 2024-04-25T10:32:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:Writing to - 2024-04-25T10:32:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x556effcab890 msgr2=0x556effcbe720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x556effcab890 0x556effcbe720 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5808065970 msgr2=0x7f5808067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5808065970 0x7f5808067e30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5818005880 tx=0x7f581800c040 comp rx=0 tx=0).stop 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 msgr2=0x556effcab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 --2- 172.21.15.45:0/1988140679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556effc2c520 0x556effcab350 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f5814000c10 tx=0x7f5814000ee0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f5828000640 1 -- 172.21.15.45:0/1988140679 reap_dead start 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 shutdown_connections 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 >> 172.21.15.45:0/1988140679 conn(0x556effc0dfe0 msgr2=0x556effc0f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 shutdown_connections 2024-04-25T10:32:06.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.782+0000 7f582a43a080 1 -- 172.21.15.45:0/1988140679 wait complete. 2024-04-25T10:32:06.789 INFO:teuthology.orchestra.run.smithi045.stdout:i���#1*f�ҍ����@@`��������#1*f�ҍ#1*f,�h(8`#1*f�ҍ8`#1*f�ҍ��������#1*f,�h����`�������� 2024-04-25T10:32:06.789 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T10:32:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 Processor -- start 2024-04-25T10:32:06.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 -- start start 2024-04-25T10:32:06.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e390 0x55ccf3a92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e770 0x55ccf3b21cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3b22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccf3a828f0 con 0x55ccf3a81df0 2024-04-25T10:32:06.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccf39f79f0 con 0x55ccf3a7e390 2024-04-25T10:32:06.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.874+0000 7f2b43bf8080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccf3a82b90 con 0x55ccf3a7e770 2024-04-25T10:32:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3b22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3b22230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47620/0 (socket says 172.21.15.45:47620) 2024-04-25T10:32:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 -- 172.21.15.45:0/1845977771 learned_addr learned my addr 172.21.15.45:0/1845977771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b3be00640 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e770 0x55ccf3b21cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b40e00640 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e390 0x55ccf3a92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 -- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e770 msgr2=0x55ccf3b21cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e770 0x55ccf3b21cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 -- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e390 msgr2=0x55ccf3a92830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e390 0x55ccf3a92830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 -- 172.21.15.45:0/1845977771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccf3a55b00 con 0x55ccf3a81df0 2024-04-25T10:32:06.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b40e00640 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e390 0x55ccf3a92830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3b22230 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001680 tx=0x7f2b3c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7648d2cc74ed5630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b3aa00640 1 -- 172.21.15.45:0/1845977771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c017070 con 0x55ccf3a81df0 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b3aa00640 1 -- 172.21.15.45:0/1845977771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0026c0 con 0x55ccf3a81df0 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b3aa00640 1 -- 172.21.15.45:0/1845977771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c002ab0 con 0x55ccf3a81df0 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1845977771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 msgr2=0x55ccf3b22230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3b22230 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001680 tx=0x7f2b3c006f90 comp rx=0 tx=0).stop 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1845977771 shutdown_connections 2024-04-25T10:32:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3b22230 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e770 0x55ccf3b21cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1845977771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e390 0x55ccf3a92830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1845977771 >> 172.21.15.45:0/1845977771 conn(0x55ccf3a7dd40 msgr2=0x55ccf3a53790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1845977771 shutdown_connections 2024-04-25T10:32:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1845977771 wait complete. 2024-04-25T10:32:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 Processor -- start 2024-04-25T10:32:06.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- start start 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e390 0x55ccf3a747f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e770 0x55ccf3b108c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3a51580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccf3a9ab20 con 0x55ccf3a81df0 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccf3a55070 con 0x55ccf3a7e770 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b43bf8080 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccf3a9adc0 con 0x55ccf3a7e390 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3a51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3a51580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47634/0 (socket says 172.21.15.45:47634) 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.878+0000 7f2b41800640 1 -- 172.21.15.45:0/1810006064 learned_addr learned my addr 172.21.15.45:0/1810006064 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b3be00640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e770 0x55ccf3b108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b40e00640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e390 0x55ccf3a747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b41800640 1 -- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e390 msgr2=0x55ccf3a747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b41800640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ccf3a7e390 0x55ccf3a747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b41800640 1 -- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e770 msgr2=0x55ccf3b108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b41800640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccf3a7e770 0x55ccf3b108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b41800640 1 -- 172.21.15.45:0/1810006064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccf3b25c30 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b41800640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3a51580 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001390 tx=0x7f2b3c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b39600640 1 -- 172.21.15.45:0/1810006064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c022070 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b39600640 1 -- 172.21.15.45:0/1810006064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c006020 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b39600640 1 -- 172.21.15.45:0/1810006064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c017030 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ccf3a533d0 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccf3a524d0 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b39600640 1 -- 172.21.15.45:0/1810006064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b3c027450 con 0x55ccf3a81df0 2024-04-25T10:32:06.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b39600640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b20065970 0x7f2b20067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b39600640 1 -- 172.21.15.45:0/1810006064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2b3c085060 con 0x55ccf3a81df0 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.882+0000 7f2b40e00640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b20065970 0x7f2b20067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x55ccf3a52f30 0x55ccf3bc1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 --> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] -- osd_op(unknown.0.0:1 28.10 28:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=531b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194) v9 -- 0x55ccf3bc1c40 con 0x55ccf3a52f30 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b3be00640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x55ccf3a52f30 0x55ccf3bc1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b40e00640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b20065970 0x7f2b20067e30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ac000 tx=0x7f2b340a5000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b3be00640 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x55ccf3a52f30 0x55ccf3bc1700 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b3be00640 1 -- 172.21.15.45:0/1810006064 <== osd.0 v2:172.21.15.45:6808/3877436580 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v194'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2b2c009cd0 con 0x55ccf3a52f30 2024-04-25T10:32:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x55ccf3a52f30 msgr2=0x55ccf3bc1700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x55ccf3a52f30 0x55ccf3bc1700 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b20065970 msgr2=0x7f2b20067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2b20065970 0x7f2b20067e30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ac000 tx=0x7f2b340a5000 comp rx=0 tx=0).stop 2024-04-25T10:32:06.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 msgr2=0x55ccf3a51580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 --2- 172.21.15.45:0/1810006064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccf3a81df0 0x55ccf3a51580 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001390 tx=0x7f2b3c001260 comp rx=0 tx=0).stop 2024-04-25T10:32:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b41800640 1 -- 172.21.15.45:0/1810006064 reap_dead start 2024-04-25T10:32:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 shutdown_connections 2024-04-25T10:32:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 >> 172.21.15.45:0/1810006064 conn(0x55ccf3a7dd40 msgr2=0x55ccf3a80ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 shutdown_connections 2024-04-25T10:32:06.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:06.886+0000 7f2b43bf8080 1 -- 172.21.15.45:0/1810006064 wait complete. 2024-04-25T10:32:06.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T10:32:07.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.078+0000 7ff204000640 1 Processor -- start 2024-04-25T10:32:07.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff204000640 1 -- start start 2024-04-25T10:32:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff204000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1fc0d0d00 0x7ff1fc0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff204000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0cc200 0x7ff1fc162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff204000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1fc0cdab0 con 0x7ff1fc1626c0 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff204000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1fc0cd210 con 0x7ff1fc0d0d00 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff204000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0cd4f0 con 0x7ff1fc0cc200 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff201600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1fc0d0d00 0x7ff1fc0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff202000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1fc1626c0 0x7ff1fc06cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48240/0 (socket says 172.21.15.45:48240) 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff202000640 1 -- 172.21.15.45:0/1910034367 learned_addr learned my addr 172.21.15.45:0/1910034367 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff200c00640 1 --2- 172.21.15.45:0/1910034367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0cc200 0x7ff1fc162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff201600640 1 -- 172.21.15.45:0/1910034367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0cc200 msgr2=0x7ff1fc162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff201600640 1 --2- 172.21.15.45:0/1910034367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0cc200 0x7ff1fc162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff201600640 1 -- 172.21.15.45:0/1910034367 >> v1:172.21.15.45:6789/0 conn(0x7ff1fc1626c0 legacy=0x7ff1fc06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.082+0000 7ff201600640 1 -- 172.21.15.45:0/1910034367 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0cd7d0 con 0x7ff1fc0d0d00 2024-04-25T10:32:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff201600640 1 --2- 172.21.15.45:0/1910034367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0d0d00 0x7ff1fc0ccc30 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00097c0 tx=0x7ff1f001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb7d8b6cb0d0948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff1fbe00640 1 -- 172.21.15.45:0/1910034367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00b0070 con 0x7ff1fc0d0d00 2024-04-25T10:32:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff1fbe00640 1 -- 172.21.15.45:0/1910034367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f00ab600 con 0x7ff1fc0d0d00 2024-04-25T10:32:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff1fbe00640 1 -- 172.21.15.45:0/1910034367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00ab9d0 con 0x7ff1fc0d0d00 2024-04-25T10:32:07.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 -- 172.21.15.45:0/1910034367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0d0d00 msgr2=0x7ff1fc0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 --2- 172.21.15.45:0/1910034367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0d0d00 0x7ff1fc0ccc30 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00097c0 tx=0x7ff1f001c410 comp rx=0 tx=0).stop 2024-04-25T10:32:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 -- 172.21.15.45:0/1910034367 shutdown_connections 2024-04-25T10:32:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 --2- 172.21.15.45:0/1910034367 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0cc200 0x7ff1fc162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 --2- 172.21.15.45:0/1910034367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0d0d00 0x7ff1fc0ccc30 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 -- 172.21.15.45:0/1910034367 >> 172.21.15.45:0/1910034367 conn(0x7ff1fc072290 msgr2=0x7ff1fc0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 -- 172.21.15.45:0/1910034367 shutdown_connections 2024-04-25T10:32:07.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 -- 172.21.15.45:0/1910034367 wait complete. 2024-04-25T10:32:07.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.086+0000 7ff204000640 1 Processor -- start 2024-04-25T10:32:07.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- start start 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 0x7ff1fc0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0d0d00 0x7ff1fc1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1fc1626c0 0x7ff1fc154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc15e040 con 0x7ff1fc0cc200 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc06d860 con 0x7ff1fc1626c0 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc15e1c0 con 0x7ff1fc0d0d00 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 0x7ff1fc0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 0x7ff1fc0c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47648/0 (socket says 172.21.15.45:47648) 2024-04-25T10:32:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 -- 172.21.15.45:0/2032682415 learned_addr learned my addr 172.21.15.45:0/2032682415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff202000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1fc1626c0 0x7ff1fc154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff200c00640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0d0d00 0x7ff1fc1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 -- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0d0d00 msgr2=0x7ff1fc1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0d0d00 0x7ff1fc1549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 -- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1fc1626c0 msgr2=0x7ff1fc154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1fc1626c0 0x7ff1fc154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 -- 172.21.15.45:0/2032682415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0c5270 con 0x7ff1fc0cc200 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff200c00640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0d0d00 0x7ff1fc1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff202000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1fc1626c0 0x7ff1fc154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff201600640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 0x7ff1fc0c66e0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff1f001a040 tx=0x7ff1f00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00b8020 con 0x7ff1fc0cc200 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f00b4610 con 0x7ff1fc0cc200 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc0c4f90 con 0x7ff1fc0cc200 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00b0040 con 0x7ff1fc0cc200 2024-04-25T10:32:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc0b32c0 con 0x7ff1fc0cc200 2024-04-25T10:32:07.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.090+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc04ef30 con 0x7ff1fc0cc200 2024-04-25T10:32:07.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.094+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1f00bb020 con 0x7ff1fc0cc200 2024-04-25T10:32:07.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.094+0000 7ff1f9600640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1dc066f90 0x7ff1dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.094+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1f011ec70 con 0x7ff1fc0cc200 2024-04-25T10:32:07.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.094+0000 7ff200c00640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1dc066f90 0x7ff1dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.094+0000 7ff200c00640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1dc066f90 0x7ff1dc069450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc09e350 tx=0x7ff1e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.098+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1f00c2030 con 0x7ff1fc0cc200 2024-04-25T10:32:07.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.258+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7ff1fc072ff0 con 0x7ff1fc0cc200 2024-04-25T10:32:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.262+0000 7ff1f9600640 1 -- 172.21.15.45:0/2032682415 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v35)=0 v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7ff1f0016090 con 0x7ff1fc0cc200 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1dc066f90 msgr2=0x7ff1dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1dc066f90 0x7ff1dc069450 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc09e350 tx=0x7ff1e8008040 comp rx=0 tx=0).stop 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 msgr2=0x7ff1fc0c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 0x7ff1fc0c66e0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ff1f001a040 tx=0x7ff1f00191b0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 shutdown_connections 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff1dc066f90 0x7ff1dc069450 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1fc1626c0 0x7ff1fc154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1fc0d0d00 0x7ff1fc1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 --2- 172.21.15.45:0/2032682415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1fc0cc200 0x7ff1fc0c66e0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 >> 172.21.15.45:0/2032682415 conn(0x7ff1fc072290 msgr2=0x7ff1fc15f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 shutdown_connections 2024-04-25T10:32:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.266+0000 7ff204000640 1 -- 172.21.15.45:0/2032682415 wait complete. 2024-04-25T10:32:07.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:32:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.430+0000 7f966e000640 1 Processor -- start 2024-04-25T10:32:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.430+0000 7f966e000640 1 -- start start 2024-04-25T10:32:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f966e000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96680b23c0 0x7f96680b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f966e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680cd780 0x7f96680cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f966e000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966806be90 con 0x7f96680b1f50 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f966e000640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96680d0b50 con 0x7f96680b23c0 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f966e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966806b680 con 0x7f96680cd780 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96680b23c0 0x7f96680b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f96680b23c0 0x7f96680b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47656/0 (socket says 172.21.15.45:47656) 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 -- 172.21.15.45:0/132069535 learned_addr learned my addr 172.21.15.45:0/132069535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f966d600640 1 --2- 172.21.15.45:0/132069535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680cd780 0x7f96680cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 -- 172.21.15.45:0/132069535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680cd780 msgr2=0x7f96680cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 --2- 172.21.15.45:0/132069535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680cd780 0x7f96680cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 -- 172.21.15.45:0/132069535 >> v1:172.21.15.117:6789/0 conn(0x7f96680b1f50 legacy=0x7f966806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.434+0000 7f965fe00640 1 -- 172.21.15.45:0/132069535 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966806b8d0 con 0x7f96680b23c0 2024-04-25T10:32:07.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f965fe00640 1 --2- 172.21.15.45:0/132069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f96680b3dd0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f9650009870 tx=0x7f965001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70316e9639e274af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f9667e00640 1 -- 172.21.15.45:0/132069535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500b0070 con 0x7f96680b23c0 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f9667e00640 1 -- 172.21.15.45:0/132069535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96500ab600 con 0x7f96680b23c0 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f9667e00640 1 -- 172.21.15.45:0/132069535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500ab9d0 con 0x7f96680b23c0 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- 172.21.15.45:0/132069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 msgr2=0x7f96680b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 --2- 172.21.15.45:0/132069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f96680b3dd0 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f9650009870 tx=0x7f965001c440 comp rx=0 tx=0).stop 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- 172.21.15.45:0/132069535 shutdown_connections 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 --2- 172.21.15.45:0/132069535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680cd780 0x7f96680cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 --2- 172.21.15.45:0/132069535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f96680b3dd0 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- 172.21.15.45:0/132069535 >> 172.21.15.45:0/132069535 conn(0x7f96680b32b0 msgr2=0x7f96680b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- 172.21.15.45:0/132069535 shutdown_connections 2024-04-25T10:32:07.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- 172.21.15.45:0/132069535 wait complete. 2024-04-25T10:32:07.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 Processor -- start 2024-04-25T10:32:07.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- start start 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680b1f50 0x7f966809ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f9668159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96680cd780 0x7f9668159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680d03c0 con 0x7f96680b23c0 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96680d0240 con 0x7f96680cd780 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f966e000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96680d0540 con 0x7f96680b1f50 2024-04-25T10:32:07.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f965fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f9668159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f965fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f9668159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47670/0 (socket says 172.21.15.45:47670) 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.438+0000 7f965fe00640 1 -- 172.21.15.45:0/4272998535 learned_addr learned my addr 172.21.15.45:0/4272998535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f966d600640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96680cd780 0x7f9668159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f966cc00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680b1f50 0x7f966809ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f965fe00640 1 -- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680b1f50 msgr2=0x7f966809ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f965fe00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680b1f50 0x7f966809ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f965fe00640 1 -- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96680cd780 msgr2=0x7f9668159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f965fe00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96680cd780 0x7f9668159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f965fe00640 1 -- 172.21.15.45:0/4272998535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680c8960 con 0x7f96680b23c0 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f966cc00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680b1f50 0x7f966809ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:07.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f965fe00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f9668159990 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f965001a070 tx=0x7f96500191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500b0040 con 0x7f96680b23c0 2024-04-25T10:32:07.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96500b9070 con 0x7f96680b23c0 2024-04-25T10:32:07.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500b4660 con 0x7f96680b23c0 2024-04-25T10:32:07.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96680c8c40 con 0x7f96680b23c0 2024-04-25T10:32:07.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96680c9200 con 0x7f96680b23c0 2024-04-25T10:32:07.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9668075120 con 0x7f96680b23c0 2024-04-25T10:32:07.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96500b4800 con 0x7f96680b23c0 2024-04-25T10:32:07.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f9665600640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9644066f60 0x7f9644069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.442+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f965011ec00 con 0x7f96680b23c0 2024-04-25T10:32:07.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.446+0000 7f966cc00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9644066f60 0x7f9644069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.446+0000 7f966cc00640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9644066f60 0x7f9644069420 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f96680ccf40 tx=0x7f9658009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.446+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9650016150 con 0x7f96680b23c0 2024-04-25T10:32:07.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.646+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f96680ccc90 con 0x7f96680b23c0 2024-04-25T10:32:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f9665600640 1 -- 172.21.15.45:0/4272998535 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v142) v1 ==== 162+0+0 (secure 0 0 0) 0x7f96500efba0 con 0x7f96680b23c0 2024-04-25T10:32:07.691 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:32:07.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9644066f60 msgr2=0x7f9644069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9644066f60 0x7f9644069420 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f96680ccf40 tx=0x7f9658009040 comp rx=0 tx=0).stop 2024-04-25T10:32:07.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 msgr2=0x7f9668159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f9668159990 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f965001a070 tx=0x7f96500191e0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 shutdown_connections 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9644066f60 0x7f9644069420 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96680cd780 0x7f9668159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680b23c0 0x7f9668159990 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 --2- 172.21.15.45:0/4272998535 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96680b1f50 0x7f966809ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 >> 172.21.15.45:0/4272998535 conn(0x7f96680b32b0 msgr2=0x7f96680ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 shutdown_connections 2024-04-25T10:32:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.690+0000 7f966e000640 1 -- 172.21.15.45:0/4272998535 wait complete. 2024-04-25T10:32:07.719 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:07.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.858+0000 7f6d77200640 1 Processor -- start 2024-04-25T10:32:07.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.858+0000 7f6d77200640 1 -- start start 2024-04-25T10:32:07.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d77200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700cfd90 0x7f6d700ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d77200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d700cd110 0x7f6d700cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d77200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7015f3b0 con 0x7f6d700b0f70 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d77200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7015ef20 con 0x7f6d700cd110 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d77200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700ca810 con 0x7f6d700cfd90 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d700cd110 0x7f6d700cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d700cd110 0x7f6d700cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47672/0 (socket says 172.21.15.45:47672) 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 -- 172.21.15.45:0/2601744548 learned_addr learned my addr 172.21.15.45:0/2601744548 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d67400640 1 --2- 172.21.15.45:0/2601744548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700cfd90 0x7f6d700ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 -- 172.21.15.45:0/2601744548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700cfd90 msgr2=0x7f6d700ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 --2- 172.21.15.45:0/2601744548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700cfd90 0x7f6d700ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 -- 172.21.15.45:0/2601744548 >> v1:172.21.15.117:6789/0 conn(0x7f6d700b0f70 legacy=0x7f6d700b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 -- 172.21.15.45:0/2601744548 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7015e5f0 con 0x7f6d700cd110 2024-04-25T10:32:07.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.862+0000 7f6d75200640 1 --2- 172.21.15.45:0/2601744548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700cba20 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009e50 tx=0x7f6d6801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e2bfa87da1026b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d6f400640 1 -- 172.21.15.45:0/2601744548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680a5700 con 0x7f6d700cd110 2024-04-25T10:32:07.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d6f400640 1 -- 172.21.15.45:0/2601744548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d680a58a0 con 0x7f6d700cd110 2024-04-25T10:32:07.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d6f400640 1 -- 172.21.15.45:0/2601744548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680a5c70 con 0x7f6d700cd110 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- 172.21.15.45:0/2601744548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 msgr2=0x7f6d700cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 --2- 172.21.15.45:0/2601744548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700cba20 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009e50 tx=0x7f6d6801ca30 comp rx=0 tx=0).stop 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- 172.21.15.45:0/2601744548 shutdown_connections 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 --2- 172.21.15.45:0/2601744548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700cba20 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 --2- 172.21.15.45:0/2601744548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700cfd90 0x7f6d700ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- 172.21.15.45:0/2601744548 >> 172.21.15.45:0/2601744548 conn(0x7f6d700b1d50 msgr2=0x7f6d700b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- 172.21.15.45:0/2601744548 shutdown_connections 2024-04-25T10:32:07.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- 172.21.15.45:0/2601744548 wait complete. 2024-04-25T10:32:07.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 Processor -- start 2024-04-25T10:32:07.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- start start 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700b0f70 0x7f6d700c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d700cfd90 0x7f6d700c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70161b50 con 0x7f6d700cd110 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d701619d0 con 0x7f6d700cfd90 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d77200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70161cd0 con 0x7f6d700b0f70 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47684/0 (socket says 172.21.15.45:47684) 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d67400640 1 -- 172.21.15.45:0/4138370987 learned_addr learned my addr 172.21.15.45:0/4138370987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.866+0000 7f6d6fe00640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700b0f70 0x7f6d700c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d75200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d700cfd90 0x7f6d700c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d67400640 1 -- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700b0f70 msgr2=0x7f6d700c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d67400640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700b0f70 0x7f6d700c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d67400640 1 -- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d700cfd90 msgr2=0x7f6d700c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d67400640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d700cfd90 0x7f6d700c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d67400640 1 -- 172.21.15.45:0/4138370987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70099e50 con 0x7f6d700cd110 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d67400640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700c9b50 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800d5f0 tx=0x7f6d5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d580125f0 con 0x7f6d700cd110 2024-04-25T10:32:07.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58019070 con 0x7f6d700cd110 2024-04-25T10:32:07.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d580129b0 con 0x7f6d700cd110 2024-04-25T10:32:07.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d700c9390 con 0x7f6d700cd110 2024-04-25T10:32:07.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d70099b70 con 0x7f6d700cd110 2024-04-25T10:32:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7004ef30 con 0x7f6d700cd110 2024-04-25T10:32:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.870+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d58012b50 con 0x7f6d700cd110 2024-04-25T10:32:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.874+0000 7f6d6cc00640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d4c066fb0 0x7f6d4c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:07.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.874+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d580851a0 con 0x7f6d700cd110 2024-04-25T10:32:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.874+0000 7f6d6fe00640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d4c066fb0 0x7f6d4c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:07.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.874+0000 7f6d6fe00640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d4c066fb0 0x7f6d4c069470 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6d60001810 tx=0x7f6d60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:07.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:07.878+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d58055c70 con 0x7f6d700cd110 2024-04-25T10:32:08.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d700cda20 con 0x7f6d700cd110 2024-04-25T10:32:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d6cc00640 1 -- 172.21.15.45:0/4138370987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f6d58059d10 con 0x7f6d700cd110 2024-04-25T10:32:08.076 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:08.076 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:07.696208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:32:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 143 2024-04-25T10:32:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d4c066fb0 msgr2=0x7f6d4c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d4c066fb0 0x7f6d4c069470 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6d60001810 tx=0x7f6d60009040 comp rx=0 tx=0).stop 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 msgr2=0x7f6d700c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700c9b50 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800d5f0 tx=0x7f6d5800dab0 comp rx=0 tx=0).stop 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 shutdown_connections 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f6d4c066fb0 0x7f6d4c069470 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d700cfd90 0x7f6d700c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d700cd110 0x7f6d700c9b50 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 --2- 172.21.15.45:0/4138370987 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6d700b0f70 0x7f6d700c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 >> 172.21.15.45:0/4138370987 conn(0x7f6d700b1d50 msgr2=0x7f6d700d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 shutdown_connections 2024-04-25T10:32:08.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:08.074+0000 7f6d77200640 1 -- 172.21.15.45:0/4138370987 wait complete. 2024-04-25T10:32:08.500 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 143, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:31:55.417950+0000', 'modified': '2024-04-25T10:32:07.696208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 194, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11496': {'gid': 11496, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/1179754162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1179754162}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1179754162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:32:08.500 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11496:up:replay 2024-04-25T10:32:09.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 Processor -- start 2024-04-25T10:32:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 -- start start 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d8015bc40 0x7f4d8006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d800cda10 0x7f4d800b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cf930 0x7f4d800cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800b2770 con 0x7f4d8015bc40 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8006c670 con 0x7f4d800cda10 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d85800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8006c950 con 0x7f4d800cf930 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d7ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d8015bc40 0x7f4d8006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d7ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4d8015bc40 0x7f4d8006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47694/0 (socket says 172.21.15.45:47694) 2024-04-25T10:32:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.650+0000 7f4d7ea00640 1 -- 172.21.15.45:0/494920399 learned_addr learned my addr 172.21.15.45:0/494920399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d77e00640 1 --2- 172.21.15.45:0/494920399 >> v2:172.21.15.117:3300/0 conn(0x7f4d800cda10 0x7f4d800b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7f400640 1 --2- 172.21.15.45:0/494920399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cf930 0x7f4d800cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7ea00640 1 -- 172.21.15.45:0/494920399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cf930 msgr2=0x7f4d800cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7ea00640 1 --2- 172.21.15.45:0/494920399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cf930 0x7f4d800cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7ea00640 1 -- 172.21.15.45:0/494920399 >> v2:172.21.15.117:3300/0 conn(0x7f4d800cda10 msgr2=0x7f4d800b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7ea00640 1 --2- 172.21.15.45:0/494920399 >> v2:172.21.15.117:3300/0 conn(0x7f4d800cda10 0x7f4d800b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7ea00640 1 -- 172.21.15.45:0/494920399 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8006cc30 con 0x7f4d8015bc40 2024-04-25T10:32:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d77e00640 1 --2- 172.21.15.45:0/494920399 >> v2:172.21.15.117:3300/0 conn(0x7f4d800cda10 0x7f4d800b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:09.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7ea00640 1 --2- 172.21.15.45:0/494920399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d8006c090 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4d800726f0 tx=0x7f4d7001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543af21502ec1611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7e000640 1 -- 172.21.15.45:0/494920399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700b0070 con 0x7f4d8015bc40 2024-04-25T10:32:09.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7e000640 1 -- 172.21.15.45:0/494920399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d700ab600 con 0x7f4d8015bc40 2024-04-25T10:32:09.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.654+0000 7f4d7e000640 1 -- 172.21.15.45:0/494920399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700ab9d0 con 0x7f4d8015bc40 2024-04-25T10:32:09.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- 172.21.15.45:0/494920399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 msgr2=0x7f4d8006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- 172.21.15.45:0/494920399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d8006c090 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4d800726f0 tx=0x7f4d7001c520 comp rx=0 tx=0).stop 2024-04-25T10:32:09.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- 172.21.15.45:0/494920399 shutdown_connections 2024-04-25T10:32:09.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- 172.21.15.45:0/494920399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cf930 0x7f4d800cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- 172.21.15.45:0/494920399 >> v2:172.21.15.117:3300/0 conn(0x7f4d800cda10 0x7f4d800b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- 172.21.15.45:0/494920399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d8006c090 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- 172.21.15.45:0/494920399 >> 172.21.15.45:0/494920399 conn(0x7f4d80066b70 msgr2=0x7f4d80066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- 172.21.15.45:0/494920399 shutdown_connections 2024-04-25T10:32:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- 172.21.15.45:0/494920399 wait complete. 2024-04-25T10:32:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 Processor -- start 2024-04-25T10:32:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- start start 2024-04-25T10:32:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cda10 0x7f4d80154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d800cf930 0x7f4d801fa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d800afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80160270 con 0x7f4d8015bc40 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d801600f0 con 0x7f4d800cf930 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d85800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d801603f0 con 0x7f4d800cda10 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d800afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d800afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47696/0 (socket says 172.21.15.45:47696) 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 -- 172.21.15.45:0/506013425 learned_addr learned my addr 172.21.15.45:0/506013425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7ea00640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cda10 0x7f4d80154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d77e00640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d800cf930 0x7f4d801fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 -- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cda10 msgr2=0x7f4d80154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cda10 0x7f4d80154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 -- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d800cf930 msgr2=0x7f4d801fa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d800cf930 0x7f4d801fa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 -- 172.21.15.45:0/506013425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800b0520 con 0x7f4d8015bc40 2024-04-25T10:32:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.658+0000 7f4d7f400640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d800afdf0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007e90 tx=0x7f4d7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78014420 con 0x7f4d8015bc40 2024-04-25T10:32:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78019070 con 0x7f4d8015bc40 2024-04-25T10:32:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d800b0800 con 0x7f4d8015bc40 2024-04-25T10:32:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78014800 con 0x7f4d8015bc40 2024-04-25T10:32:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d800b0dc0 con 0x7f4d8015bc40 2024-04-25T10:32:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8004ef30 con 0x7f4d8015bc40 2024-04-25T10:32:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d780149a0 con 0x7f4d8015bc40 2024-04-25T10:32:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d76a00640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d5c066f60 0x7f4d5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4d780858b0 con 0x7f4d8015bc40 2024-04-25T10:32:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.662+0000 7f4d7ea00640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d5c066f60 0x7f4d5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.666+0000 7f4d7ea00640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d5c066f60 0x7f4d5c069420 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4d700ad000 tx=0x7f4d700a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:09.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.666+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d780560e0 con 0x7f4d8015bc40 2024-04-25T10:32:09.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.862+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d80157e80 con 0x7f4d8015bc40 2024-04-25T10:32:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d76a00640 1 -- 172.21.15.45:0/506013425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f4d7805a180 con 0x7f4d8015bc40 2024-04-25T10:32:09.867 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:09.867 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":145,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:08.702364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:32:09.867 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 145 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d5c066f60 msgr2=0x7f4d5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d5c066f60 0x7f4d5c069420 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4d700ad000 tx=0x7f4d700a6000 comp rx=0 tx=0).stop 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 msgr2=0x7f4d800afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d800afdf0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007e90 tx=0x7f4d7800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 shutdown_connections 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f4d5c066f60 0x7f4d5c069420 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8015bc40 0x7f4d800afdf0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d800cf930 0x7f4d801fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 --2- 172.21.15.45:0/506013425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d800cda10 0x7f4d80154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 >> 172.21.15.45:0/506013425 conn(0x7f4d80066b70 msgr2=0x7f4d80159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 shutdown_connections 2024-04-25T10:32:09.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:09.866+0000 7f4d85800640 1 -- 172.21.15.45:0/506013425 wait complete. 2024-04-25T10:32:10.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 145, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:31:55.417950+0000', 'modified': '2024-04-25T10:32:08.702364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 194, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11496': {'gid': 11496, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.45:6835/1179754162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1179754162}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1179754162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:32:10.290 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11496:up:rejoin 2024-04-25T10:32:11.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:32:11.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 Processor -- start 2024-04-25T10:32:11.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 -- start start 2024-04-25T10:32:11.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f535415fb10 0x7f53541625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:11.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53540cf4c0 0x7f53540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:11.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5354162c10 con 0x7f535406c180 2024-04-25T10:32:11.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540cfe70 con 0x7f535415fb10 2024-04-25T10:32:11.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5354162ef0 con 0x7f53540cf4c0 2024-04-25T10:32:11.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f535406c180 0x7f53540b2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48254/0 (socket says 172.21.15.45:48254) 2024-04-25T10:32:11.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359400640 1 -- 172.21.15.45:0/3837933458 learned_addr learned my addr 172.21.15.45:0/3837933458 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:11.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359e00640 1 --2- 172.21.15.45:0/3837933458 >> v2:172.21.15.117:3300/0 conn(0x7f535415fb10 0x7f53541625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:11.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535a800640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53540cf4c0 0x7f53540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5358a00640 1 -- 172.21.15.45:0/3837933458 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602864704 0 0) 0x7f5354162c10 con 0x7f535406c180 2024-04-25T10:32:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5358a00640 1 -- 172.21.15.45:0/3837933458 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533c003530 con 0x7f535406c180 2024-04-25T10:32:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359e00640 1 -- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53540cf4c0 msgr2=0x7f53540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359e00640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53540cf4c0 0x7f53540cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359e00640 1 -- 172.21.15.45:0/3837933458 >> v1:172.21.15.45:6789/0 conn(0x7f535406c180 legacy=0x7f53540b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f5359e00640 1 -- 172.21.15.45:0/3837933458 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53541631d0 con 0x7f535415fb10 2024-04-25T10:32:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.442+0000 7f535a800640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53540cf4c0 0x7f53540cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:11.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f5359e00640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535415fb10 0x7f53541625e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f53480097c0 tx=0x7f534801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d88a2413c8681863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:11.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f5358a00640 1 -- 172.21.15.45:0/3837933458 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53480b0070 con 0x7f535415fb10 2024-04-25T10:32:11.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f5358a00640 1 -- 172.21.15.45:0/3837933458 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53480ab600 con 0x7f535415fb10 2024-04-25T10:32:11.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f5358a00640 1 -- 172.21.15.45:0/3837933458 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53480ab9d0 con 0x7f535415fb10 2024-04-25T10:32:11.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 -- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535415fb10 msgr2=0x7f53541625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:11.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535415fb10 0x7f53541625e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f53480097c0 tx=0x7f534801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 -- 172.21.15.45:0/3837933458 shutdown_connections 2024-04-25T10:32:11.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f53540cf4c0 0x7f53540cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 --2- 172.21.15.45:0/3837933458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535415fb10 0x7f53541625e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 -- 172.21.15.45:0/3837933458 >> 172.21.15.45:0/3837933458 conn(0x7f53540743f0 msgr2=0x7f53540669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:11.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 -- 172.21.15.45:0/3837933458 shutdown_connections 2024-04-25T10:32:11.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.446+0000 7f535b200640 1 -- 172.21.15.45:0/3837933458 wait complete. 2024-04-25T10:32:11.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 Processor -- start 2024-04-25T10:32:11.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 -- start start 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535406c180 0x7f5354157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 0x7f535415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f535415fb10 0x7f53540cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535416f300 con 0x7f53540cf4c0 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535416f180 con 0x7f535406c180 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535416f480 con 0x7f535415fb10 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 0x7f535415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 0x7f535415bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47700/0 (socket says 172.21.15.45:47700) 2024-04-25T10:32:11.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 -- 172.21.15.45:0/3781951044 learned_addr learned my addr 172.21.15.45:0/3781951044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359e00640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535406c180 0x7f5354157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535a800640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f535415fb10 0x7f53540cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 -- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f535415fb10 msgr2=0x7f53540cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f535415fb10 0x7f53540cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 -- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535406c180 msgr2=0x7f5354157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535406c180 0x7f5354157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 -- 172.21.15.45:0/3781951044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5354156850 con 0x7f53540cf4c0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f5359400640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 0x7f535415bf20 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f534400c480 tx=0x7f534400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f53540cf4c0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344005cf0 con 0x7f53540cf4c0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5354156b30 con 0x7f53540cf4c0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344007290 con 0x7f53540cf4c0 2024-04-25T10:32:11.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.450+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535415c460 con 0x7f53540cf4c0 2024-04-25T10:32:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.454+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5354075120 con 0x7f53540cf4c0 2024-04-25T10:32:11.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.454+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f534401a070 con 0x7f53540cf4c0 2024-04-25T10:32:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.454+0000 7f534e000640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5338066f90 0x7f5338069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.454+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f53440843c0 con 0x7f53540cf4c0 2024-04-25T10:32:11.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.454+0000 7f5359e00640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5338066f90 0x7f5338069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:11.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.454+0000 7f5359e00640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5338066f90 0x7f5338069450 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f53480ad000 tx=0x7f53480a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:11.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.458+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5344054d10 con 0x7f53540cf4c0 2024-04-25T10:32:11.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.662+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53540cf090 con 0x7f53540cf4c0 2024-04-25T10:32:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.662+0000 7f534e000640 1 -- 172.21.15.45:0/3781951044 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f5344058db0 con 0x7f53540cf4c0 2024-04-25T10:32:11.665 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:32:11.665 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:09.710400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:32:11.665 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.662+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5338066f90 msgr2=0x7f5338069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.662+0000 7f535b200640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5338066f90 0x7f5338069450 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f53480ad000 tx=0x7f53480a6000 comp rx=0 tx=0).stop 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.662+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 msgr2=0x7f535415bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.662+0000 7f535b200640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 0x7f535415bf20 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f534400c480 tx=0x7f534400c940 comp rx=0 tx=0).stop 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 shutdown_connections 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f5338066f90 0x7f5338069450 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f535415fb10 0x7f53540cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f53540cf4c0 0x7f535415bf20 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 --2- 172.21.15.45:0/3781951044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535406c180 0x7f5354157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 >> 172.21.15.45:0/3781951044 conn(0x7f53540743f0 msgr2=0x7f53541588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 shutdown_connections 2024-04-25T10:32:11.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:11.666+0000 7f535b200640 1 -- 172.21.15.45:0/3781951044 wait complete. 2024-04-25T10:32:12.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 146, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:31:55.417950+0000', 'modified': '2024-04-25T10:32:09.710400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 194, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11496': {'gid': 11496, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/1179754162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1179754162}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1179754162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:32:12.091 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:32:12.091 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:32:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 Processor -- start 2024-04-25T10:32:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 -- start start 2024-04-25T10:32:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a2c15b910 0x7f1a2c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2c071050 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2c15bda0 con 0x7f1a2c15b910 2024-04-25T10:32:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.230+0000 7f1a34600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c15d040 con 0x7f1a2c1550b0 2024-04-25T10:32:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a2c15b910 0x7f1a2c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a2c15b910 0x7f1a2c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34744/0 (socket says 172.21.15.45:34744) 2024-04-25T10:32:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 -- 172.21.15.45:0/3687089035 learned_addr learned my addr 172.21.15.45:0/3687089035 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:50794/0 (socket says 172.21.15.45:50794) 2024-04-25T10:32:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 -- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 msgr2=0x7f1a2c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 --2- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 -- 172.21.15.45:0/3687089035 >> v1:172.21.15.117:6789/0 conn(0x7f1a2c0cfe70 legacy=0x7f1a2c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 -- 172.21.15.45:0/3687089035 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c0712f0 con 0x7f1a2c15b910 2024-04-25T10:32:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a2be00640 1 -- 172.21.15.45:0/3687089035 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574474686 0 0) 0x7f1a2c071050 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a32600640 1 --2- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c0b3260 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009e50 tx=0x7f1a2401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc18dd64d87e2073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a2be00640 1 -- 172.21.15.45:0/3687089035 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240a57f0 con 0x7f1a2c15b910 2024-04-25T10:32:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a2be00640 1 -- 172.21.15.45:0/3687089035 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a240a5990 con 0x7f1a2c15b910 2024-04-25T10:32:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.234+0000 7f1a2be00640 1 -- 172.21.15.45:0/3687089035 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240a5d60 con 0x7f1a2c15b910 2024-04-25T10:32:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 msgr2=0x7f1a2c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 --2- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c0b3260 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009e50 tx=0x7f1a2401ca30 comp rx=0 tx=0).stop 2024-04-25T10:32:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- 172.21.15.45:0/3687089035 shutdown_connections 2024-04-25T10:32:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 --2- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c0b3260 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 --2- 172.21.15.45:0/3687089035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- 172.21.15.45:0/3687089035 >> 172.21.15.45:0/3687089035 conn(0x7f1a2c066b70 msgr2=0x7f1a2c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- 172.21.15.45:0/3687089035 shutdown_connections 2024-04-25T10:32:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- 172.21.15.45:0/3687089035 wait complete. 2024-04-25T10:32:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 Processor -- start 2024-04-25T10:32:12.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- start start 2024-04-25T10:32:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 0x7f1a2c0af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c1463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c15c030 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c0720b0 con 0x7f1a2c15b910 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a34600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c15c1b0 con 0x7f1a2c1550b0 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 0x7f1a2c0af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 0x7f1a2c0af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47708/0 (socket says 172.21.15.45:47708) 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 -- 172.21.15.45:0/4265188526 learned_addr learned my addr 172.21.15.45:0/4265188526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a32600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31200640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c1463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 -- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 msgr2=0x7f1a2c1463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c1463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 -- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 msgr2=0x7f1a2c146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a31c00640 1 -- 172.21.15.45:0/4265188526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c158b30 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.238+0000 7f1a32600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a31c00640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 0x7f1a2c0af640 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000e930 tx=0x7f1a2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a29600640 1 -- 172.21.15.45:0/4265188526 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20017070 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a29600640 1 -- 172.21.15.45:0/4265188526 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2000ca00 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a29600640 1 -- 172.21.15.45:0/4265188526 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2000cdb0 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c158e10 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c1593d0 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a29600640 1 -- 172.21.15.45:0/4265188526 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a2001a070 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a29600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a0c066f60 0x7f1a0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a29600640 1 -- 172.21.15.45:0/4265188526 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1a200854a0 con 0x7f1a2c0cfe70 2024-04-25T10:32:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a37b82000 1 Processor -- start 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a31200640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a0c066f60 0x7f1a0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.242+0000 7f1a37b82000 1 -- start start 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31200640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a0c066f60 0x7f1a0c069420 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0affa0 tx=0x7f1a18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 0x5655325b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565532568f30 0x565532569310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655324505f0 con 0x5655325b2920 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655323b7da0 con 0x5655325b3240 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x565532340150 con 0x565532568f30 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 0x5655325b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 0x5655325b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47724/0 (socket says 172.21.15.45:47724) 2024-04-25T10:32:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 -- 172.21.15.45:0/4186139554 learned_addr learned my addr 172.21.15.45:0/4186139554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31200640 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a32600640 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565532568f30 0x565532569310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 -- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565532568f30 msgr2=0x565532569310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565532568f30 0x565532569310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 -- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 msgr2=0x5655325689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 -- 172.21.15.45:0/4186139554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565532359ca0 con 0x5655325b2920 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a31c00640 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 0x5655325b2d00 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009310 tx=0x7f1a200565c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eddb5a8800d762c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a1e000640 1 -- 172.21.15.45:0/4186139554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20021070 con 0x5655325b2920 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a1e000640 1 -- 172.21.15.45:0/4186139554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2005e510 con 0x5655325b2920 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a1e000640 1 -- 172.21.15.45:0/4186139554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20054100 con 0x5655325b2920 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- 172.21.15.45:0/4186139554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 msgr2=0x5655325b2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 0x5655325b2d00 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009310 tx=0x7f1a200565c0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- 172.21.15.45:0/4186139554 shutdown_connections 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x565532568f30 0x565532569310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 --2- 172.21.15.45:0/4186139554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5655325b2920 0x5655325b2d00 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- 172.21.15.45:0/4186139554 >> 172.21.15.45:0/4186139554 conn(0x56553256f2a0 msgr2=0x5655325672d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- 172.21.15.45:0/4186139554 shutdown_connections 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- 172.21.15.45:0/4186139554 wait complete. 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 -- start start 2024-04-25T10:32:12.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.246+0000 7f1a37b82000 1 Processor -- start 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x5655325b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x5655325ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x56553256caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x565532372990 con 0x565532568f30 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655323b3e40 con 0x5655325b3240 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x565532343900 con 0x5655325b2920 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 10 client.0 _ll_register_callbacks cb 0x56553246b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x5655325b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x5655325b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47740/0 (socket says 172.21.15.45:47740) 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 -- 172.21.15.45:0/2404028203 learned_addr learned my addr 172.21.15.45:0/2404028203 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31200640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x5655325ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a32600640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x56553256caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 -- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 msgr2=0x5655325ba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x5655325ba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 -- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 msgr2=0x56553256caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x56553256caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 -- 172.21.15.45:0/2404028203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565532370630 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x5655325b9c80 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c10 tx=0x7f1a200092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1ce122fb906ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 1 -- 172.21.15.45:0/2404028203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20052470 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 1 -- 172.21.15.45:0/2404028203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1a200850b0 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a2005e510 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 1 -- 172.21.15.45:0/2404028203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a20052610 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 10 client.11547 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5655323403a0 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 1 -- 172.21.15.45:0/2404028203 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2005e510 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 1 -- 172.21.15.45:0/2404028203 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a20055250 con 0x565532568f30 2024-04-25T10:32:12.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 10 client.11547 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 10 client.11547 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 20 client.11547 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56553256e160 0x5655325c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 4 client.11547 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565532353a50 con 0x56553256e160 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31200640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56553256e160 0x5655325c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 10 client.11547 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a31200640 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56553256e160 0x5655325c3ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50e8ddbc8d06319 server_cookie=5ddd55ef64f5cc9c in_seq=0 out_seq=0 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 1 -- 172.21.15.45:0/2404028203 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565532353a50 con 0x56553256e160 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.250+0000 7f1a12a00640 10 client.11547 handle_command_reply: tid=0 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 client.11547 shutdown 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 msgr2=0x5655325b9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x5655325b9c80 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f1a20000c10 tx=0x7f1a200092a0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 shutdown_connections 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56553256e160 0x5655325c3ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x56553256caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x5655325ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2404028203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x5655325b9c80 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a12a00640 0 client.11547 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 >> 172.21.15.45:0/2404028203 conn(0x56553256f2a0 msgr2=0x5655325679e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 shutdown_connections 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 1 -- 172.21.15.45:0/2404028203 wait complete. 2024-04-25T10:32:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.254+0000 7f1a37b82000 20 client.11547 trim_cache size 0 max 16384 2024-04-25T10:32:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 Processor -- start 2024-04-25T10:32:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- start start 2024-04-25T10:32:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x56553256c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x56553256c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c169980 con 0x565532568f30 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c15a200 con 0x5655325b3240 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c15a800 con 0x5655325b2920 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x56553256c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x56553256c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47748/0 (socket says 172.21.15.45:47748) 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 -- 172.21.15.45:0/2824557842 learned_addr learned my addr 172.21.15.45:0/2824557842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31200640 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x56553256c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a32600640 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 -- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 msgr2=0x56553256c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x56553256c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 -- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 msgr2=0x5655325bb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325bb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 -- 172.21.15.45:0/2824557842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565532439600 con 0x565532568f30 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x56553256c2c0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f1a200097c0 tx=0x7f1a20083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd077eb8ba23785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a1d600640 1 -- 172.21.15.45:0/2824557842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2005e510 con 0x565532568f30 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a1d600640 1 -- 172.21.15.45:0/2824557842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20075070 con 0x565532568f30 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a1d600640 1 -- 172.21.15.45:0/2824557842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20051b50 con 0x565532568f30 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- 172.21.15.45:0/2824557842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 msgr2=0x56553256c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x56553256c2c0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f1a200097c0 tx=0x7f1a20083000 comp rx=0 tx=0).stop 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- 172.21.15.45:0/2824557842 shutdown_connections 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b3240 0x5655325bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b2920 0x56553256c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2824557842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x56553256c2c0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- 172.21.15.45:0/2824557842 >> 172.21.15.45:0/2824557842 conn(0x56553256f2a0 msgr2=0x56553256bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- 172.21.15.45:0/2824557842 shutdown_connections 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- 172.21.15.45:0/2824557842 wait complete. 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 -- start start 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.270+0000 7f1a37b82000 1 Processor -- start 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x565532612c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b2920 0x5655325f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b3240 0x5655325f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655323bc3d0 con 0x565532568f30 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x565532473d50 con 0x5655325b2920 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x565532352980 con 0x5655325b3240 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 10 client.0 _ll_register_callbacks cb 0x7f1a200520a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x565532612c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x565532612c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47756/0 (socket says 172.21.15.45:47756) 2024-04-25T10:32:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 -- 172.21.15.45:0/2519497834 learned_addr learned my addr 172.21.15.45:0/2519497834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31200640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b2920 0x5655325f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a32600640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b3240 0x5655325f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 -- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b3240 msgr2=0x5655325f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b3240 0x5655325f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 -- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b2920 msgr2=0x5655325f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b2920 0x5655325f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 -- 172.21.15.45:0/2519497834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56553234c9b0 con 0x565532568f30 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31c00640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x565532612c60 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f1a20055d90 tx=0x7f1a200521d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83353440274a7ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 1 -- 172.21.15.45:0/2519497834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2005e510 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 1 -- 172.21.15.45:0/2519497834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1a2006dad0 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a20075070 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 1 -- 172.21.15.45:0/2519497834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a2006dcf0 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 10 client.11553 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x56553235f0b0 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 1 -- 172.21.15.45:0/2519497834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a20075070 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 1 -- 172.21.15.45:0/2519497834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a20066070 con 0x565532568f30 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 10 client.11553 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 10 client.11553 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 20 client.11553 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x5655325be620 0x5655325bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 4 client.11553 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5655323c52e0 con 0x5655325be620 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31200640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x5655325be620 0x5655325bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a31200640 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x5655325be620 0x5655325bea00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb222db9b54a84f9 server_cookie=ee10d210ec5f323f in_seq=0 out_seq=0 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 10 client.11553 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 1 -- 172.21.15.45:0/2519497834 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x5655323c52e0 con 0x5655325be620 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 10 client.11553 handle_command_reply: tid=0 2024-04-25T10:32:12.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 client.11553 shutdown 2024-04-25T10:32:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 msgr2=0x565532612c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x565532612c60 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f1a20055d90 tx=0x7f1a200521d0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 shutdown_connections 2024-04-25T10:32:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x5655325be620 0x5655325bea00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5655325b3240 0x5655325f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5655325b2920 0x5655325f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 --2- 172.21.15.45:0/2519497834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565532568f30 0x565532612c60 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a12a00640 0 client.11553 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 >> 172.21.15.45:0/2519497834 conn(0x56553256f2a0 msgr2=0x56553256c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 shutdown_connections 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 1 -- 172.21.15.45:0/2519497834 wait complete. 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.274+0000 7f1a37b82000 20 client.11553 trim_cache size 0 max 16384 2024-04-25T10:32:12.286 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 11496, 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:32:12.287 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.5777427289999997, 2024-04-25T10:32:12.288 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 146, 2024-04-25T10:32:12.288 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 194, 2024-04-25T10:32:12.288 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 194, 2024-04-25T10:32:12.288 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 5.5112645840000001 2024-04-25T10:32:12.288 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a0c066f60 msgr2=0x7f1a0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a0c066f60 0x7f1a0c069420 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c0affa0 tx=0x7f1a18009040 comp rx=0 tx=0).stop 2024-04-25T10:32:12.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 msgr2=0x7f1a2c0af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 0x7f1a2c0af640 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000e930 tx=0x7f1a2000edf0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 shutdown_connections 2024-04-25T10:32:12.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1a0c066f60 0x7f1a0c069420 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a2c15b910 0x7f1a2c146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a2c1550b0 0x7f1a2c1463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 --2- 172.21.15.45:0/4265188526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a2c0cfe70 0x7f1a2c0af640 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 >> 172.21.15.45:0/4265188526 conn(0x7f1a2c066b70 msgr2=0x7f1a2c15c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 shutdown_connections 2024-04-25T10:32:12.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.278+0000 7f1a34600640 1 -- 172.21.15.45:0/4265188526 wait complete. 2024-04-25T10:32:12.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:12.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:32:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.438+0000 7f00a5e00640 1 Processor -- start 2024-04-25T10:32:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.438+0000 7f00a5e00640 1 -- start start 2024-04-25T10:32:12.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f00a5e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f00a015bd10 0x7f00a00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f00a5e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00a00cf9e0 0x7f00a00cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f00a5e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a006c0e0 con 0x7f00a015c0f0 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f00a5e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a00b2ef0 con 0x7f00a00cf9e0 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f00a5e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a00d0350 con 0x7f00a015bd10 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00a00cf9e0 0x7f00a00cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00a00cf9e0 0x7f00a00cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47766/0 (socket says 172.21.15.45:47766) 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 -- 172.21.15.45:0/1488053315 learned_addr learned my addr 172.21.15.45:0/1488053315 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009ea00640 1 --2- 172.21.15.45:0/1488053315 >> v2:172.21.15.117:3300/0 conn(0x7f00a015bd10 0x7f00a00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 -- 172.21.15.45:0/1488053315 >> v2:172.21.15.117:3300/0 conn(0x7f00a015bd10 msgr2=0x7f00a00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 --2- 172.21.15.45:0/1488053315 >> v2:172.21.15.117:3300/0 conn(0x7f00a015bd10 0x7f00a00b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 -- 172.21.15.45:0/1488053315 >> v1:172.21.15.117:6789/0 conn(0x7f00a015c0f0 legacy=0x7f00a00b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 -- 172.21.15.45:0/1488053315 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a015c6a0 con 0x7f00a00cf9e0 2024-04-25T10:32:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.442+0000 7f009f400640 1 --2- 172.21.15.45:0/1488053315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a00cf9e0 0x7f00a00cfe10 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f00a0072810 tx=0x7f009801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c29c947e6155b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009e000640 1 -- 172.21.15.45:0/1488053315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980a5840 con 0x7f00a00cf9e0 2024-04-25T10:32:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009e000640 1 -- 172.21.15.45:0/1488053315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00980a59e0 con 0x7f00a00cf9e0 2024-04-25T10:32:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009e000640 1 -- 172.21.15.45:0/1488053315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980a5db0 con 0x7f00a00cf9e0 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- 172.21.15.45:0/1488053315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a00cf9e0 msgr2=0x7f00a00cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1488053315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a00cf9e0 0x7f00a00cfe10 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f00a0072810 tx=0x7f009801c980 comp rx=0 tx=0).stop 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- 172.21.15.45:0/1488053315 shutdown_connections 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1488053315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a00cf9e0 0x7f00a00cfe10 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1488053315 >> v2:172.21.15.117:3300/0 conn(0x7f00a015bd10 0x7f00a00b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- 172.21.15.45:0/1488053315 >> 172.21.15.45:0/1488053315 conn(0x7f00a0066b70 msgr2=0x7f00a0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- 172.21.15.45:0/1488053315 shutdown_connections 2024-04-25T10:32:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- 172.21.15.45:0/1488053315 wait complete. 2024-04-25T10:32:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 Processor -- start 2024-04-25T10:32:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- start start 2024-04-25T10:32:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a00cf9e0 0x7f00a00b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a015bd10 0x7f00a01f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 0x7f00a014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00b4cd0 con 0x7f00a015c0f0 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00b4b50 con 0x7f00a00cf9e0 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f00a5e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a006cdd0 con 0x7f00a015bd10 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 0x7f00a014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 0x7f00a014c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47770/0 (socket says 172.21.15.45:47770) 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009f400640 1 -- 172.21.15.45:0/1920131313 learned_addr learned my addr 172.21.15.45:0/1920131313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f009ea00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a00cf9e0 0x7f00a00b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.446+0000 7f0097e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a015bd10 0x7f00a01f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f009f400640 1 -- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a015bd10 msgr2=0x7f00a01f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f009f400640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a015bd10 0x7f00a01f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f009f400640 1 -- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a00cf9e0 msgr2=0x7f00a00b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f009f400640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a00cf9e0 0x7f00a00b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f009f400640 1 -- 172.21.15.45:0/1920131313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a014d6a0 con 0x7f00a015c0f0 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f0097e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a015bd10 0x7f00a01f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f009f400640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 0x7f00a014c6b0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f009801a5b0 tx=0x7f0098019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f0097400640 1 -- 172.21.15.45:0/1920131313 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980bc020 con 0x7f00a015c0f0 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f0097400640 1 -- 172.21.15.45:0/1920131313 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00980b8670 con 0x7f00a015c0f0 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a0154820 con 0x7f00a015c0f0 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f0097400640 1 -- 172.21.15.45:0/1920131313 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980b8a60 con 0x7f00a015c0f0 2024-04-25T10:32:12.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a0156530 con 0x7f00a015c0f0 2024-04-25T10:32:12.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.450+0000 7f0097400640 1 -- 172.21.15.45:0/1920131313 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00980bf020 con 0x7f00a015c0f0 2024-04-25T10:32:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f0097400640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f007c066f60 0x7f007c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f0097400640 1 -- 172.21.15.45:0/1920131313 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f00981200f0 con 0x7f00a015c0f0 2024-04-25T10:32:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f007c066f60 0x7f007c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 Processor -- start 2024-04-25T10:32:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 -- start start 2024-04-25T10:32:12.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f007c066f60 0x7f007c069420 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0090007a60 tx=0x7f0090008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada972f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada95cbf40 con 0x55ada972ec90 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada9533070 con 0x55ada96e5290 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada94bb1a0 con 0x55ada972f5b0 2024-04-25T10:32:12.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada972f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada972f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47772/0 (socket says 172.21.15.45:47772) 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 -- 172.21.15.45:0/2312263447 learned_addr learned my addr 172.21.15.45:0/2312263447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009f400640 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f0097e00640 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 -- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 msgr2=0x55ada96e4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96e4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 -- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 msgr2=0x55ada96e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 -- 172.21.15.45:0/2312263447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada94b9510 con 0x55ada972ec90 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f009ea00640 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada972f070 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f0090007fd0 tx=0x7f0090013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb47502720a42f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f008fe00640 1 -- 172.21.15.45:0/2312263447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009001c050 con 0x55ada972ec90 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f008fe00640 1 -- 172.21.15.45:0/2312263447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0090026070 con 0x55ada972ec90 2024-04-25T10:32:12.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f008fe00640 1 -- 172.21.15.45:0/2312263447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090021640 con 0x55ada972ec90 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 -- 172.21.15.45:0/2312263447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 msgr2=0x55ada972f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.454+0000 7f00a9282000 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada972f070 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f0090007fd0 tx=0x7f0090013000 comp rx=0 tx=0).stop 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- 172.21.15.45:0/2312263447 shutdown_connections 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96e4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 --2- 172.21.15.45:0/2312263447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada972f070 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- 172.21.15.45:0/2312263447 >> 172.21.15.45:0/2312263447 conn(0x55ada96e1240 msgr2=0x55ada96e3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- 172.21.15.45:0/2312263447 shutdown_connections 2024-04-25T10:32:12.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- 172.21.15.45:0/2312263447 wait complete. 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- start start 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 Processor -- start 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada96efe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada96f0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada94ed6c0 con 0x55ada96e5290 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada952f110 con 0x55ada972ec90 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada96e10c0 con 0x55ada972f5b0 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 10 client.0 _ll_register_callbacks cb 0x55ada96d0570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada96efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada96efe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47786/0 (socket says 172.21.15.45:47786) 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 -- 172.21.15.45:0/3443312154 learned_addr learned my addr 172.21.15.45:0/3443312154 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009f400640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f0097e00640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada96f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 -- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 msgr2=0x55ada96f08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96f08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 -- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 msgr2=0x55ada96f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada96f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 -- 172.21.15.45:0/3443312154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ada94bb3b0 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f009ea00640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada96efe30 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x55ada94f5810 tx=0x7f0090004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96024df30a9046a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f008cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f008cc00640 1 -- 172.21.15.45:0/3443312154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009002f070 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f008cc00640 1 -- 172.21.15.45:0/3443312154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0090021db0 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0090026070 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f008cc00640 1 -- 172.21.15.45:0/3443312154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0090016e50 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 10 client.11565 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55ada95b4e00 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.458+0000 7f008cc00640 1 -- 172.21.15.45:0/3443312154 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0090026070 con 0x55ada96e5290 2024-04-25T10:32:12.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f008cc00640 1 -- 172.21.15.45:0/3443312154 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f009003d080 con 0x55ada96e5290 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 10 client.11565 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 10 client.11565 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 20 client.11565 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada96f1180 0x55ada9740410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 4 client.11565 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ada94ced60 con 0x55ada96f1180 2024-04-25T10:32:12.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f0097e00640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada96f1180 0x55ada9740410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f0097e00640 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada96f1180 0x55ada9740410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b272466411de51 server_cookie=6b60ee2823e505f3 in_seq=0 out_seq=0 2024-04-25T10:32:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f008cc00640 10 client.11565 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f008cc00640 1 -- 172.21.15.45:0/3443312154 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ada94ced60 con 0x55ada96f1180 2024-04-25T10:32:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f008cc00640 10 client.11565 handle_command_reply: tid=0 2024-04-25T10:32:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 client.11565 shutdown 2024-04-25T10:32:12.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 msgr2=0x55ada96efe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada96efe30 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x55ada94f5810 tx=0x7f0090004740 comp rx=0 tx=0).stop 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 shutdown_connections 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada96f1180 0x55ada9740410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada96f08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada96f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 --2- 172.21.15.45:0/3443312154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada96efe30 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f008cc00640 0 client.11565 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 >> 172.21.15.45:0/3443312154 conn(0x55ada96e1240 msgr2=0x55ada96e3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 shutdown_connections 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 1 -- 172.21.15.45:0/3443312154 wait complete. 2024-04-25T10:32:12.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.462+0000 7f00a9282000 20 client.11565 trim_cache size 0 max 16384 2024-04-25T10:32:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 Processor -- start 2024-04-25T10:32:12.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- start start 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada96e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9740410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0202ca0 con 0x55ada972ec90 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0071050 con 0x55ada96e5290 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0070450 con 0x55ada972f5b0 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada96e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada96e4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47788/0 (socket says 172.21.15.45:47788) 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 -- 172.21.15.45:0/589329441 learned_addr learned my addr 172.21.15.45:0/589329441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f009ea00640 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f009f400640 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9740410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 -- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 msgr2=0x55ada9740410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9740410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 -- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 msgr2=0x55ada96e4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 -- 172.21.15.45:0/589329441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada958ac50 con 0x55ada972ec90 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f009ea00640 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f0097e00640 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada96e4b80 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f0088006f60 tx=0x7f0088008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2c3a923cc3ac70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f008f400640 1 -- 172.21.15.45:0/589329441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008800bcb0 con 0x55ada972ec90 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f008f400640 1 -- 172.21.15.45:0/589329441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008801f030 con 0x55ada972ec90 2024-04-25T10:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f008f400640 1 -- 172.21.15.45:0/589329441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088004220 con 0x55ada972ec90 2024-04-25T10:32:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- 172.21.15.45:0/589329441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 msgr2=0x55ada96e4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada96e4b80 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f0088006f60 tx=0x7f0088008040 comp rx=0 tx=0).stop 2024-04-25T10:32:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- 172.21.15.45:0/589329441 shutdown_connections 2024-04-25T10:32:12.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9740410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada972ec90 0x55ada96e4b80 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 --2- 172.21.15.45:0/589329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada96e5290 0x55ada96e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.478+0000 7f00a9282000 1 -- 172.21.15.45:0/589329441 >> 172.21.15.45:0/589329441 conn(0x55ada96e1240 msgr2=0x55ada96e9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- 172.21.15.45:0/589329441 shutdown_connections 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- 172.21.15.45:0/589329441 wait complete. 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- start start 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 Processor -- start 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada978f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada978f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9749e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada94d5bd0 con 0x55ada96e5290 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0070a50 con 0x55ada972ec90 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada952f110 con 0x55ada972f5b0 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 10 client.0 _ll_register_callbacks cb 0x7f0090007770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:12.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada978f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada978f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47798/0 (socket says 172.21.15.45:47798) 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 -- 172.21.15.45:0/3986989308 learned_addr learned my addr 172.21.15.45:0/3986989308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f0097e00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada978f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009f400640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9749e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 -- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 msgr2=0x55ada9749e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9749e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 -- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 msgr2=0x55ada978f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada978f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 -- 172.21.15.45:0/3986989308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ada94fb120 con 0x55ada96e5290 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f0097e00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada978f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f009ea00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada978f050 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f0090007840 tx=0x7f0090016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d05b238eddd7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f008cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f008cc00640 1 -- 172.21.15.45:0/3986989308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009001c070 con 0x55ada96e5290 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f008cc00640 1 -- 172.21.15.45:0/3986989308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0090034dc0 con 0x55ada96e5290 2024-04-25T10:32:12.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f008801f030 con 0x55ada96e5290 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f008cc00640 1 -- 172.21.15.45:0/3986989308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0090047050 con 0x55ada96e5290 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 10 client.11571 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55ada94da170 con 0x55ada96e5290 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f008cc00640 1 -- 172.21.15.45:0/3986989308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008801f030 con 0x55ada96e5290 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f008cc00640 1 -- 172.21.15.45:0/3986989308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f009003d020 con 0x55ada96e5290 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 10 client.11571 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 10 client.11571 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 20 client.11571 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada974a820 0x55ada974ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 4 client.11571 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55ada95405b0 con 0x55ada974a820 2024-04-25T10:32:12.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.482+0000 7f0097e00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada974a820 0x55ada974ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f0097e00640 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada974a820 0x55ada974ac00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6eafc9e5f4f74c server_cookie=a6b2b836a3efdb6f in_seq=0 out_seq=0 2024-04-25T10:32:12.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f008cc00640 10 client.11571 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f008cc00640 1 -- 172.21.15.45:0/3986989308 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ada95405b0 con 0x55ada974a820 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f008cc00640 10 client.11571 handle_command_reply: tid=0 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 client.11571 shutdown 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 msgr2=0x55ada978f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada978f050 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f0090007840 tx=0x7f0090016000 comp rx=0 tx=0).stop 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 shutdown_connections 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ada974a820 0x55ada974ac00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ada972f5b0 0x55ada9749e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ada972ec90 0x55ada978f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 --2- 172.21.15.45:0/3986989308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ada96e5290 0x55ada978f050 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 >> 172.21.15.45:0/3986989308 conn(0x55ada96e1240 msgr2=0x55ada96ea430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f008cc00640 0 client.11571 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 shutdown_connections 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 1 -- 172.21.15.45:0/3986989308 wait complete. 2024-04-25T10:32:12.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a9282000 20 client.11571 trim_cache size 0 max 16384 2024-04-25T10:32:12.492 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:12.492 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:32:12.492 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "0c309afa-42db-4c7c-9f1c-7b6d9f0b93e5", 2024-04-25T10:32:12.492 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:32:12.492 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f007c066f60 msgr2=0x7f007c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f007c066f60 0x7f007c069420 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0090007a60 tx=0x7f0090008040 comp rx=0 tx=0).stop 2024-04-25T10:32:12.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 msgr2=0x7f00a014c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 0x7f00a014c6b0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f009801a5b0 tx=0x7f0098019720 comp rx=0 tx=0).stop 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 shutdown_connections 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f007c066f60 0x7f007c069420 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a015c0f0 0x7f00a014c6b0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a015bd10 0x7f00a01f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 --2- 172.21.15.45:0/1920131313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a00cf9e0 0x7f00a00b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 >> 172.21.15.45:0/1920131313 conn(0x7f00a0066b70 msgr2=0x7f00a0066f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 shutdown_connections 2024-04-25T10:32:12.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.486+0000 7f00a5e00640 1 -- 172.21.15.45:0/1920131313 wait complete. 2024-04-25T10:32:12.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:12.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 Processor -- start 2024-04-25T10:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 -- start start 2024-04-25T10:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc066fe0 0x7f31bc161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f31bc0b5560 0x7f31bc0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc0b40c0 con 0x7f31bc066c00 2024-04-25T10:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc161d60 con 0x7f31bc0b5560 2024-04-25T10:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31c2000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0b5b00 con 0x7f31bc066fe0 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31bb400640 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f31bc066c00 0x7f31bc072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.45:50132/0 (socket says 172.21.15.45:50132) 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31bb400640 1 -- 172.21.15.45:0/1480564791 learned_addr learned my addr 172.21.15.45:0/1480564791 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31b2a00640 1 --2- 172.21.15.45:0/1480564791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc066fe0 0x7f31bc161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31bbe00640 1 --2- 172.21.15.45:0/1480564791 >> v2:172.21.15.117:3300/0 conn(0x7f31bc0b5560 0x7f31bc0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31baa00640 1 -- 172.21.15.45:0/1480564791 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350528532 0 0) 0x7f31bc0b40c0 con 0x7f31bc066c00 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31baa00640 1 -- 172.21.15.45:0/1480564791 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a0003530 con 0x7f31bc066c00 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31b2a00640 1 -- 172.21.15.45:0/1480564791 >> v2:172.21.15.117:3300/0 conn(0x7f31bc0b5560 msgr2=0x7f31bc0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31b2a00640 1 --2- 172.21.15.45:0/1480564791 >> v2:172.21.15.117:3300/0 conn(0x7f31bc0b5560 0x7f31bc0ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31b2a00640 1 -- 172.21.15.45:0/1480564791 >> v1:172.21.15.117:6789/0 conn(0x7f31bc066c00 legacy=0x7f31bc072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.650+0000 7f31b2a00640 1 -- 172.21.15.45:0/1480564791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc0b3a00 con 0x7f31bc066fe0 2024-04-25T10:32:12.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31b2a00640 1 --2- 172.21.15.45:0/1480564791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc066fe0 0x7f31bc161820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31a40097c0 tx=0x7f31a401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e169461e07535806 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31baa00640 1 -- 172.21.15.45:0/1480564791 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40b0070 con 0x7f31bc066fe0 2024-04-25T10:32:12.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31baa00640 1 -- 172.21.15.45:0/1480564791 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a40ab600 con 0x7f31bc066fe0 2024-04-25T10:32:12.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31baa00640 1 -- 172.21.15.45:0/1480564791 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40ab9b0 con 0x7f31bc066fe0 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 -- 172.21.15.45:0/1480564791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc066fe0 msgr2=0x7f31bc161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 --2- 172.21.15.45:0/1480564791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc066fe0 0x7f31bc161820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f31a40097c0 tx=0x7f31a401c380 comp rx=0 tx=0).stop 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 -- 172.21.15.45:0/1480564791 shutdown_connections 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 --2- 172.21.15.45:0/1480564791 >> v2:172.21.15.117:3300/0 conn(0x7f31bc0b5560 0x7f31bc0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 --2- 172.21.15.45:0/1480564791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc066fe0 0x7f31bc161820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 -- 172.21.15.45:0/1480564791 >> 172.21.15.45:0/1480564791 conn(0x7f31bc0b2560 msgr2=0x7f31bc0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 -- 172.21.15.45:0/1480564791 shutdown_connections 2024-04-25T10:32:12.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.654+0000 7f31c2000640 1 -- 172.21.15.45:0/1480564791 wait complete. 2024-04-25T10:32:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 Processor -- start 2024-04-25T10:32:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 -- start start 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc066c00 0x7f31bc0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 0x7f31bc160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc0b5560 0x7f31bc13adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc15e540 con 0x7f31bc066fe0 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1585e0 con 0x7f31bc066c00 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc15e6c0 con 0x7f31bc0b5560 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 0x7f31bc160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 0x7f31bc160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47806/0 (socket says 172.21.15.45:47806) 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 -- 172.21.15.45:0/1384997636 learned_addr learned my addr 172.21.15.45:0/1384997636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31bb400640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc066c00 0x7f31bc0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31bbe00640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc0b5560 0x7f31bc13adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 -- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc0b5560 msgr2=0x7f31bc13adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc0b5560 0x7f31bc13adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 -- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc066c00 msgr2=0x7f31bc0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc066c00 0x7f31bc0b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 -- 172.21.15.45:0/1384997636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc1f71a0 con 0x7f31bc066fe0 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31bb400640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc066c00 0x7f31bc0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b2a00640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 0x7f31bc160950 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f31a4019fb0 tx=0x7f31a4019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b3e00640 1 -- 172.21.15.45:0/1384997636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40b8020 con 0x7f31bc066fe0 2024-04-25T10:32:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b3e00640 1 -- 172.21.15.45:0/1384997636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a40b4410 con 0x7f31bc066fe0 2024-04-25T10:32:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31b3e00640 1 -- 172.21.15.45:0/1384997636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40b0040 con 0x7f31bc066fe0 2024-04-25T10:32:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc1f7480 con 0x7f31bc066fe0 2024-04-25T10:32:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.658+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1f7a40 con 0x7f31bc066fe0 2024-04-25T10:32:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31b3e00640 1 -- 172.21.15.45:0/1384997636 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f31a40bb020 con 0x7f31bc066fe0 2024-04-25T10:32:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31b3e00640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3198066f60 0x7f3198069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31b3e00640 1 -- 172.21.15.45:0/1384997636 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31a411eb20 con 0x7f31bc066fe0 2024-04-25T10:32:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3198066f60 0x7f3198069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 Processor -- start 2024-04-25T10:32:12.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 -- start start 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3198066f60 0x7f3198069420 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005890 tx=0x7f31ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f606bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f606c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5eb8ff0 con 0x55c1f60b59d0 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5e41150 con 0x55c1f60b62f0 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5eb5090 con 0x55c1f606c0a0 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47814/0 (socket says 172.21.15.45:47814) 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 -- 172.21.15.45:0/25661255 learned_addr learned my addr 172.21.15.45:0/25661255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31b2a00640 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f606bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bbe00640 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f606c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 -- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 msgr2=0x55c1f606c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f606c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 -- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 msgr2=0x55c1f606bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f606bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.662+0000 7f31bb400640 1 -- 172.21.15.45:0/25661255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f604f0d0 con 0x55c1f60b59d0 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60b5db0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005b80 tx=0x7f31ac010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4a42d78baf4ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31abe00640 1 -- 172.21.15.45:0/25661255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00db50 con 0x55c1f60b59d0 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31abe00640 1 -- 172.21.15.45:0/25661255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac007750 con 0x55c1f60b59d0 2024-04-25T10:32:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31abe00640 1 -- 172.21.15.45:0/25661255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac001430 con 0x55c1f60b59d0 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- 172.21.15.45:0/25661255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 msgr2=0x55c1f60b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60b5db0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005b80 tx=0x7f31ac010000 comp rx=0 tx=0).stop 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- 172.21.15.45:0/25661255 shutdown_connections 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f606c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f606bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- 172.21.15.45:0/25661255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60b5db0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- 172.21.15.45:0/25661255 >> 172.21.15.45:0/25661255 conn(0x55c1f6068020 msgr2=0x55c1f606a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- 172.21.15.45:0/25661255 shutdown_connections 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- 172.21.15.45:0/25661255 wait complete. 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- start start 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 Processor -- start 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 0x55c1f609a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b59d0 0x55c1f609ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f609b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5e7d250 con 0x55c1f606c0a0 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5e73690 con 0x55c1f60b62f0 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5e41a40 con 0x55c1f60b59d0 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31c54fa000 10 client.0 _ll_register_callbacks cb 0x55c1f6056e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 0x55c1f609a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 0x55c1f609a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47830/0 (socket says 172.21.15.45:47830) 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 -- 172.21.15.45:0/635978759 learned_addr learned my addr 172.21.15.45:0/635978759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31b2a00640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b59d0 0x55c1f609ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bbe00640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f609b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 -- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b59d0 msgr2=0x55c1f609ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b59d0 0x55c1f609ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 -- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 msgr2=0x55c1f609b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f609b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 -- 172.21.15.45:0/635978759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f5f661c0 con 0x55c1f606c0a0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31bb400640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 0x55c1f609a7f0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x55c1f5e753d0 tx=0x7f31ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd06a2644907d1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.666+0000 7f31a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 1 -- 172.21.15.45:0/635978759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac020720 con 0x55c1f606c0a0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 1 -- 172.21.15.45:0/635978759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31ac0208c0 con 0x55c1f606c0a0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31ac007750 con 0x55c1f606c0a0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 1 -- 172.21.15.45:0/635978759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ac034050 con 0x55c1f606c0a0 2024-04-25T10:32:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 10 client.11580 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55c1f5f62170 con 0x55c1f606c0a0 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 1 -- 172.21.15.45:0/635978759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac007750 con 0x55c1f606c0a0 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 1 -- 172.21.15.45:0/635978759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f31ac037020 con 0x55c1f606c0a0 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 10 client.11580 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 10 client.11580 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 20 client.11580 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c5c80 0x55c1f60c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 4 client.11580 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1f5e54c60 con 0x55c1f60c5c80 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31b2a00640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c5c80 0x55c1f60c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31b2a00640 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c5c80 0x55c1f60c8070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19abac2ec4bc2e92 server_cookie=c10c365b0d5b9dd0 in_seq=0 out_seq=0 2024-04-25T10:32:12.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 10 client.11580 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 1 -- 172.21.15.45:0/635978759 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1f5e54c60 con 0x55c1f60c5c80 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 10 client.11580 handle_command_reply: tid=0 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 client.11580 shutdown 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 msgr2=0x55c1f609a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 0x55c1f609a7f0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x55c1f5e753d0 tx=0x7f31ac012000 comp rx=0 tx=0).stop 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 shutdown_connections 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c5c80 0x55c1f60c8070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f609b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b59d0 0x55c1f609ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31a8c00640 0 client.11580 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 --2- 172.21.15.45:0/635978759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f606c0a0 0x55c1f609a7f0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 >> 172.21.15.45:0/635978759 conn(0x55c1f6068020 msgr2=0x55c1f606aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 shutdown_connections 2024-04-25T10:32:12.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 1 -- 172.21.15.45:0/635978759 wait complete. 2024-04-25T10:32:12.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.670+0000 7f31c54fa000 20 client.11580 trim_cache size 0 max 16384 2024-04-25T10:32:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 Processor -- start 2024-04-25T10:32:12.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 -- start start 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f606c0a0 0x55c1f61120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f6112630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b62f0 0x55c1f6112b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc203020 con 0x55c1f60b59d0 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc151fc0 con 0x55c1f606c0a0 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1513c0 con 0x55c1f60b62f0 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f6112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f6112630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47842/0 (socket says 172.21.15.45:47842) 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 -- 172.21.15.45:0/698717687 learned_addr learned my addr 172.21.15.45:0/698717687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31bbe00640 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b62f0 0x55c1f6112b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31bb400640 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f606c0a0 0x55c1f61120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 -- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b62f0 msgr2=0x55c1f6112b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b62f0 0x55c1f6112b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 -- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f606c0a0 msgr2=0x55c1f61120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f606c0a0 0x55c1f61120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 -- 172.21.15.45:0/698717687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1f5effde0 con 0x55c1f60b59d0 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31bb400640 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f606c0a0 0x55c1f61120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31b2a00640 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f6112630 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f31a4097d60 tx=0x7f31a40bf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d496faeeda63d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31ab400640 1 -- 172.21.15.45:0/698717687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40ef330 con 0x55c1f60b59d0 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31ab400640 1 -- 172.21.15.45:0/698717687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a40f3480 con 0x55c1f60b59d0 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.686+0000 7f31ab400640 1 -- 172.21.15.45:0/698717687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40f7a40 con 0x55c1f60b59d0 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/698717687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 msgr2=0x55c1f6112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f6112630 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f31a4097d60 tx=0x7f31a40bf040 comp rx=0 tx=0).stop 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/698717687 shutdown_connections 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f60b62f0 0x55c1f6112b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f6112630 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- 172.21.15.45:0/698717687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f606c0a0 0x55c1f61120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/698717687 >> 172.21.15.45:0/698717687 conn(0x55c1f6068020 msgr2=0x55c1f60c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/698717687 shutdown_connections 2024-04-25T10:32:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/698717687 wait complete. 2024-04-25T10:32:12.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- start start 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 Processor -- start 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f60c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f60c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5e53e00 con 0x55c1f60b59d0 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1519c0 con 0x55c1f60b62f0 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1f5eb8300 con 0x55c1f606c0a0 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 10 client.0 _ll_register_callbacks cb 0x55c1f6037040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47852/0 (socket says 172.21.15.45:47852) 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 -- 172.21.15.45:0/3216485417 learned_addr learned my addr 172.21.15.45:0/3216485417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31bb400640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f60c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31bbe00640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f60c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 -- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 msgr2=0x55c1f60c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f60c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 -- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 msgr2=0x55c1f60c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f60c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 -- 172.21.15.45:0/3216485417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1f5e5a8e0 con 0x55c1f60b59d0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31bb400640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f60c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31b2a00640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60c88c0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f31a401c8f0 tx=0x7f31ac016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c7d6bb8ca9c59c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31a8c00640 1 -- 172.21.15.45:0/3216485417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a411c040 con 0x55c1f60b59d0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31a8c00640 1 -- 172.21.15.45:0/3216485417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31a40f3120 con 0x55c1f60b59d0 2024-04-25T10:32:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31a40f3480 con 0x55c1f60b59d0 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31a8c00640 1 -- 172.21.15.45:0/3216485417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31a40f7a40 con 0x55c1f60b59d0 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 10 client.11586 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55c1f5f128f0 con 0x55c1f60b59d0 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31a8c00640 1 -- 172.21.15.45:0/3216485417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a40f3480 con 0x55c1f60b59d0 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31a8c00640 1 -- 172.21.15.45:0/3216485417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f31a4104040 con 0x55c1f60b59d0 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 10 client.11586 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 10 client.11586 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 20 client.11586 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c9760 0x55c1f611df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 4 client.11586 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.690+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1f5ec6530 con 0x55c1f60c9760 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31bb400640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c9760 0x55c1f611df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31bb400640 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c9760 0x55c1f611df60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602b312ee6a8add4 server_cookie=4860c44bf06cf2cf in_seq=0 out_seq=0 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31a8c00640 10 client.11586 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31a8c00640 1 -- 172.21.15.45:0/3216485417 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55c1f5ec6530 con 0x55c1f60c9760 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31a8c00640 10 client.11586 handle_command_reply: tid=0 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 client.11586 shutdown 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 msgr2=0x55c1f60c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60c88c0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f31a401c8f0 tx=0x7f31ac016070 comp rx=0 tx=0).stop 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 shutdown_connections 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55c1f60c9760 0x55c1f611df60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1f60b62f0 0x55c1f60c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c1f60b59d0 0x55c1f60c88c0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 --2- 172.21.15.45:0/3216485417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55c1f606c0a0 0x55c1f60c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 >> 172.21.15.45:0/3216485417 conn(0x55c1f6068020 msgr2=0x55c1f60b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31a8c00640 0 client.11586 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 shutdown_connections 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 1 -- 172.21.15.45:0/3216485417 wait complete. 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c54fa000 20 client.11586 trim_cache size 0 max 16384 2024-04-25T10:32:12.702 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "0c309afa-42db-4c7c-9f1c-7b6d9f0b93e5": { 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "0c309afa-42db-4c7c-9f1c-7b6d9f0b93e5", 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:32:12.703 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3198066f60 msgr2=0x7f3198069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3198066f60 0x7f3198069420 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005890 tx=0x7f31ac009040 comp rx=0 tx=0).stop 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 msgr2=0x7f31bc160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 0x7f31bc160950 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f31a4019fb0 tx=0x7f31a4019120 comp rx=0 tx=0).stop 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 shutdown_connections 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3198066f60 0x7f3198069420 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f31bc0b5560 0x7f31bc13adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31bc066fe0 0x7f31bc160950 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 --2- 172.21.15.45:0/1384997636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc066c00 0x7f31bc0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 >> 172.21.15.45:0/1384997636 conn(0x7f31bc0b2560 msgr2=0x7f31bc0b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 shutdown_connections 2024-04-25T10:32:12.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:12.694+0000 7f31c2000640 1 -- 172.21.15.45:0/1384997636 wait complete. 2024-04-25T10:32:12.705 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-04-25T10:32:12.698+0000 7f1b8e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:32:12.716 INFO:tasks.cephfs.filesystem:scrub status for tag:0c309afa-42db-4c7c-9f1c-7b6d9f0b93e5 - {'path': '/', 'tag': '0c309afa-42db-4c7c-9f1c-7b6d9f0b93e5', 'options': 'recursive,repair'} 2024-04-25T10:32:30.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:30.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:32:42.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:42.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:32:42.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 Processor -- start 2024-04-25T10:32:42.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 -- start start 2024-04-25T10:32:42.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0600af500 0x7fc0600cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc06015cf80 0x7fc06015d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0600cdce0 con 0x7fc0600b5340 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06015c0f0 con 0x7fc0600af500 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05fe00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06015c3d0 con 0x7fc06015cf80 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc056000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0600af500 0x7fc0600cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc056000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0600af500 0x7fc0600cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39876/0 (socket says 172.21.15.45:39876) 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.870+0000 7fc05ea00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0600b5340 0x7fc0600b5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41036/0 (socket says 172.21.15.45:41036) 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc056000640 1 -- 172.21.15.45:0/2396594709 learned_addr learned my addr 172.21.15.45:0/2396594709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:42.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc05f400640 1 --2- 172.21.15.45:0/2396594709 >> v2:172.21.15.117:3300/0 conn(0x7fc06015cf80 0x7fc06015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc056000640 1 -- 172.21.15.45:0/2396594709 >> v2:172.21.15.117:3300/0 conn(0x7fc06015cf80 msgr2=0x7fc06015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc056000640 1 --2- 172.21.15.45:0/2396594709 >> v2:172.21.15.117:3300/0 conn(0x7fc06015cf80 0x7fc06015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc056000640 1 -- 172.21.15.45:0/2396594709 >> v1:172.21.15.45:6789/0 conn(0x7fc0600b5340 legacy=0x7fc0600b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc05e000640 1 -- 172.21.15.45:0/2396594709 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162990214 0 0) 0x7fc0600cdce0 con 0x7fc0600b5340 2024-04-25T10:32:42.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc056000640 1 -- 172.21.15.45:0/2396594709 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06015c6b0 con 0x7fc0600af500 2024-04-25T10:32:42.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc056000640 1 --2- 172.21.15.45:0/2396594709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0600af500 0x7fc0600cd490 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fc048009870 tx=0x7fc04801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605f1fb808bbfe49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc05e000640 1 -- 172.21.15.45:0/2396594709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0480ab660 con 0x7fc0600af500 2024-04-25T10:32:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc05e000640 1 -- 172.21.15.45:0/2396594709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0480b0070 con 0x7fc0600af500 2024-04-25T10:32:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.874+0000 7fc05e000640 1 -- 172.21.15.45:0/2396594709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0480aba20 con 0x7fc0600af500 2024-04-25T10:32:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- 172.21.15.45:0/2396594709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0600af500 msgr2=0x7fc0600cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 --2- 172.21.15.45:0/2396594709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0600af500 0x7fc0600cd490 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fc048009870 tx=0x7fc04801c4c0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- 172.21.15.45:0/2396594709 shutdown_connections 2024-04-25T10:32:42.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 --2- 172.21.15.45:0/2396594709 >> v2:172.21.15.117:3300/0 conn(0x7fc06015cf80 0x7fc06015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 --2- 172.21.15.45:0/2396594709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0600af500 0x7fc0600cd490 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- 172.21.15.45:0/2396594709 >> 172.21.15.45:0/2396594709 conn(0x7fc06006cb60 msgr2=0x7fc06006cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- 172.21.15.45:0/2396594709 shutdown_connections 2024-04-25T10:32:42.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- 172.21.15.45:0/2396594709 wait complete. 2024-04-25T10:32:42.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 Processor -- start 2024-04-25T10:32:42.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- start start 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0600af500 0x7fc060153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0600b5340 0x7fc060144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 0x7fc06015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc060159790 con 0x7fc06015cf80 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc060159610 con 0x7fc0600af500 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05fe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06015dd60 con 0x7fc0600b5340 2024-04-25T10:32:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 0x7fc06015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0600af500 0x7fc060153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 0x7fc06015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39878/0 (socket says 172.21.15.45:39878) 2024-04-25T10:32:42.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc05f400640 1 -- 172.21.15.45:0/1618443012 learned_addr learned my addr 172.21.15.45:0/1618443012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:42.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.878+0000 7fc056000640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0600b5340 0x7fc060144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05f400640 1 -- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0600b5340 msgr2=0x7fc060144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05f400640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0600b5340 0x7fc060144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05f400640 1 -- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0600af500 msgr2=0x7fc060153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05f400640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0600af500 0x7fc060153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05f400640 1 -- 172.21.15.45:0/1618443012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060149cb0 con 0x7fc06015cf80 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05f400640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 0x7fc06015af60 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e7f0 tx=0x7fc05800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc057400640 1 -- 172.21.15.45:0/1618443012 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058017070 con 0x7fc06015cf80 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc057400640 1 -- 172.21.15.45:0/1618443012 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05800c730 con 0x7fc06015cf80 2024-04-25T10:32:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc057400640 1 -- 172.21.15.45:0/1618443012 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05800cb40 con 0x7fc06015cf80 2024-04-25T10:32:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc060149f90 con 0x7fc06015cf80 2024-04-25T10:32:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06014a550 con 0x7fc06015cf80 2024-04-25T10:32:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc057400640 1 -- 172.21.15.45:0/1618443012 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc05801a070 con 0x7fc06015cf80 2024-04-25T10:32:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc057400640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc03c066f60 0x7fc03c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.882+0000 7fc057400640 1 -- 172.21.15.45:0/1618443012 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc0580851a0 con 0x7fc06015cf80 2024-04-25T10:32:42.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc05ea00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc03c066f60 0x7fc03c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 Processor -- start 2024-04-25T10:32:42.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- start start 2024-04-25T10:32:42.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc05ea00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc03c066f60 0x7fc03c069420 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc0500012f0 tx=0x7fc050001770 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac7319db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac711cff0 con 0x55eac731a2f0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac70a5150 con 0x55eac73199d0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eac7119090 con 0x55eac72d00a0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39894/0 (socket says 172.21.15.45:39894) 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 -- 172.21.15.45:0/1722424148 learned_addr learned my addr 172.21.15.45:0/1722424148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc05ea00640 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac7319db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc05f400640 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 -- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 msgr2=0x55eac72d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 -- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 msgr2=0x55eac7319db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac7319db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 -- 172.21.15.45:0/1722424148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eac72b30d0 con 0x55eac731a2f0 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc056000640 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72cfb60 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fc048019130 tx=0x7fc048004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36370ccb5776903c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc04f400640 1 -- 172.21.15.45:0/1722424148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0480040c0 con 0x55eac731a2f0 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc04f400640 1 -- 172.21.15.45:0/1722424148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0480b0040 con 0x55eac731a2f0 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc04f400640 1 -- 172.21.15.45:0/1722424148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048004460 con 0x55eac731a2f0 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- 172.21.15.45:0/1722424148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 msgr2=0x55eac72cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72cfb60 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fc048019130 tx=0x7fc048004740 comp rx=0 tx=0).stop 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- 172.21.15.45:0/1722424148 shutdown_connections 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72cfb60 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 --2- 172.21.15.45:0/1722424148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac7319db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- 172.21.15.45:0/1722424148 >> 172.21.15.45:0/1722424148 conn(0x55eac72cc020 msgr2=0x55eac72ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- 172.21.15.45:0/1722424148 shutdown_connections 2024-04-25T10:32:42.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.886+0000 7fc068af4000 1 -- 172.21.15.45:0/1722424148 wait complete. 2024-04-25T10:32:42.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- start start 2024-04-25T10:32:42.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 Processor -- start 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac72fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac70e1250 con 0x55eac731a2f0 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac70d7690 con 0x55eac73199d0 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eac70a5a40 con 0x55eac72d00a0 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 10 client.0 _ll_register_callbacks cb 0x55eac72bae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39910/0 (socket says 172.21.15.45:39910) 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 -- 172.21.15.45:0/1658745716 learned_addr learned my addr 172.21.15.45:0/1658745716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05ea00640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc056000640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac72fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 -- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 msgr2=0x55eac72fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 -- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 msgr2=0x55eac72fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac72fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 -- 172.21.15.45:0/1658745716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eac71ca1c0 con 0x55eac731a2f0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc056000640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac72fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05f400640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72ff270 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e7c0 tx=0x7fc058059c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3954a2b4e1ba656d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc043e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc043e00640 1 -- 172.21.15.45:0/1658745716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058081060 con 0x55eac731a2f0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc043e00640 1 -- 172.21.15.45:0/1658745716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc058055140 con 0x55eac731a2f0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0480b0040 con 0x55eac731a2f0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc043e00640 1 -- 172.21.15.45:0/1658745716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc058072050 con 0x55eac731a2f0 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 10 client.11601 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55eac71c6170 con 0x55eac731a2f0 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc043e00640 1 -- 172.21.15.45:0/1658745716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0480b0040 con 0x55eac731a2f0 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc043e00640 1 -- 172.21.15.45:0/1658745716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc058051840 con 0x55eac731a2f0 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 10 client.11601 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 10 client.11601 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 20 client.11601 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac7329c80 0x55eac732c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 4 client.11601 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eac70b8c60 con 0x55eac7329c80 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.890+0000 7fc05ea00640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac7329c80 0x55eac732c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc043e00640 10 client.11601 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:42.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc05ea00640 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac7329c80 0x55eac732c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257f11f65cecbcde server_cookie=685d3d6ae163353a in_seq=0 out_seq=0 2024-04-25T10:32:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc043e00640 1 -- 172.21.15.45:0/1658745716 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eac70b8c60 con 0x55eac7329c80 2024-04-25T10:32:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc043e00640 10 client.11601 handle_command_reply: tid=0 2024-04-25T10:32:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 client.11601 shutdown 2024-04-25T10:32:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 msgr2=0x55eac72ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72ff270 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e7c0 tx=0x7fc058059c50 comp rx=0 tx=0).stop 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 shutdown_connections 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac7329c80 0x55eac732c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac72ff270 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac72fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 --2- 172.21.15.45:0/1658745716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac72fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc043e00640 0 client.11601 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 >> 172.21.15.45:0/1658745716 conn(0x55eac72cc020 msgr2=0x55eac72ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 shutdown_connections 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 1 -- 172.21.15.45:0/1658745716 wait complete. 2024-04-25T10:32:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.894+0000 7fc068af4000 20 client.11601 trim_cache size 0 max 16384 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 Processor -- start 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- start start 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 0x55eac73760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac73199d0 0x55eac7376630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac731a2f0 0x55eac7376b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc060202e50 con 0x55eac72d00a0 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06015a140 con 0x55eac731a2f0 2024-04-25T10:32:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0601494d0 con 0x55eac73199d0 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 0x55eac73760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 0x55eac73760f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39920/0 (socket says 172.21.15.45:39920) 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 -- 172.21.15.45:0/3530847847 learned_addr learned my addr 172.21.15.45:0/3530847847 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 -- 172.21.15.45:0/3530847847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac73199d0 msgr2=0x55eac7376630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac73199d0 0x55eac7376630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 -- 172.21.15.45:0/3530847847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac731a2f0 msgr2=0x55eac7376b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac731a2f0 0x55eac7376b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 -- 172.21.15.45:0/3530847847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eac7163de0 con 0x55eac72d00a0 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 0x55eac73760f0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fc050002240 tx=0x7fc050004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38922cf02740cc59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc04ea00640 1 -- 172.21.15.45:0/3530847847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050027e60 con 0x55eac72d00a0 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc04ea00640 1 -- 172.21.15.45:0/3530847847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05002d070 con 0x55eac72d00a0 2024-04-25T10:32:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc04ea00640 1 -- 172.21.15.45:0/3530847847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050015d30 con 0x55eac72d00a0 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- 172.21.15.45:0/3530847847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 msgr2=0x55eac73760f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 0x55eac73760f0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fc050002240 tx=0x7fc050004f90 comp rx=0 tx=0).stop 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- 172.21.15.45:0/3530847847 shutdown_connections 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac731a2f0 0x55eac7376b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac73199d0 0x55eac7376630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 --2- 172.21.15.45:0/3530847847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac72d00a0 0x55eac73760f0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- 172.21.15.45:0/3530847847 >> 172.21.15.45:0/3530847847 conn(0x55eac72cc020 msgr2=0x55eac7326fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- 172.21.15.45:0/3530847847 shutdown_connections 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- 172.21.15.45:0/3530847847 wait complete. 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.910+0000 7fc068af4000 1 -- start start 2024-04-25T10:32:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 Processor -- start 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac732c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac732c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac732ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac70b7e00 con 0x55eac731a2f0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc060159b40 con 0x55eac73199d0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eac711c300 con 0x55eac72d00a0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 10 client.0 _ll_register_callbacks cb 0x55eac729b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac732ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac732ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39922/0 (socket says 172.21.15.45:39922) 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 -- 172.21.15.45:0/3813262441 learned_addr learned my addr 172.21.15.45:0/3813262441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac732c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc056000640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac732c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 -- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 msgr2=0x55eac732c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac732c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 -- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 msgr2=0x55eac732c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac732c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 -- 172.21.15.45:0/3813262441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eac70be8e0 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac732c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05f400640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac732ce00 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fc05805dc20 tx=0x7fc058083530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336b49c9c3c66a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 1 -- 172.21.15.45:0/3813262441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05806d070 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 1 -- 172.21.15.45:0/3813262441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc05805aab0 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc05002d070 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 1 -- 172.21.15.45:0/3813262441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc058021040 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 10 client.11607 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55eac71768f0 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 1 -- 172.21.15.45:0/3813262441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05002d070 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 1 -- 172.21.15.45:0/3813262441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc05805acb0 con 0x55eac731a2f0 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 10 client.11607 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 10 client.11607 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 20 client.11607 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac737ff50 0x55eac7382340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 4 client.11607 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eac712a530 con 0x55eac737ff50 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac737ff50 0x55eac7382340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc05ea00640 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac737ff50 0x55eac7382340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457ba9c95da23cf8 server_cookie=a470d41b4529d6ab in_seq=0 out_seq=0 2024-04-25T10:32:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 10 client.11607 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 1 -- 172.21.15.45:0/3813262441 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55eac712a530 con 0x55eac737ff50 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc043e00640 10 client.11607 handle_command_reply: tid=0 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 client.11607 shutdown 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 msgr2=0x55eac732ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.914+0000 7fc068af4000 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac732ce00 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fc05805dc20 tx=0x7fc058083530 comp rx=0 tx=0).stop 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 shutdown_connections 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55eac737ff50 0x55eac7382340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eac731a2f0 0x55eac732ce00 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac73199d0 0x55eac732c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc043e00640 0 client.11607 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 --2- 172.21.15.45:0/3813262441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55eac72d00a0 0x55eac732c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 >> 172.21.15.45:0/3813262441 conn(0x55eac72cc020 msgr2=0x55eac73180d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 shutdown_connections 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 1 -- 172.21.15.45:0/3813262441 wait complete. 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc068af4000 20 client.11607 trim_cache size 0 max 16384 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:32:42.924 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc03c066f60 msgr2=0x7fc03c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc03c066f60 0x7fc03c069420 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc0500012f0 tx=0x7fc050001770 comp rx=0 tx=0).stop 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 msgr2=0x7fc06015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 0x7fc06015af60 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fc05800e7f0 tx=0x7fc05800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 shutdown_connections 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc03c066f60 0x7fc03c069420 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc06015cf80 0x7fc06015af60 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0600b5340 0x7fc060144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 --2- 172.21.15.45:0/1618443012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0600af500 0x7fc060153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 >> 172.21.15.45:0/1618443012 conn(0x7fc06006cb60 msgr2=0x7fc0600afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 shutdown_connections 2024-04-25T10:32:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:42.918+0000 7fc05fe00640 1 -- 172.21.15.45:0/1618443012 wait complete. 2024-04-25T10:32:42.939 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:32:42.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:42.939 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:32:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 Processor -- start 2024-04-25T10:32:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 -- start start 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f211c0b2910 0x7f211c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c06d1b0 con 0x7f211c15dd20 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c0cb390 con 0x7f211c0b2910 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c06dae0 con 0x7f211c162660 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.082+0000 7f2122200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f211c0b2910 0x7f211c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f211c0b2910 0x7f211c06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39928/0 (socket says 172.21.15.45:39928) 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 -- 172.21.15.45:0/3858273903 learned_addr learned my addr 172.21.15.45:0/3858273903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2120e00640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 -- 172.21.15.45:0/3858273903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 msgr2=0x7f211c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 -- 172.21.15.45:0/3858273903 >> v1:172.21.15.117:6789/0 conn(0x7f211c15dd20 legacy=0x7f211c0cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 -- 172.21.15.45:0/3858273903 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c06d490 con 0x7f211c0b2910 2024-04-25T10:32:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2120e00640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:32:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f2122200640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0b2910 0x7f211c06df70 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e80 tx=0x7f211401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2539d6475b5eef01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f211be00640 1 -- 172.21.15.45:0/3858273903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140a55d0 con 0x7f211c0b2910 2024-04-25T10:32:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f211be00640 1 -- 172.21.15.45:0/3858273903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140a5770 con 0x7f211c0b2910 2024-04-25T10:32:43.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.086+0000 7f211be00640 1 -- 172.21.15.45:0/3858273903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140a5b40 con 0x7f211c0b2910 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- 172.21.15.45:0/3858273903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0b2910 msgr2=0x7f211c06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0b2910 0x7f211c06df70 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e80 tx=0x7f211401c940 comp rx=0 tx=0).stop 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- 172.21.15.45:0/3858273903 shutdown_connections 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c0b2910 0x7f211c06df70 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 --2- 172.21.15.45:0/3858273903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- 172.21.15.45:0/3858273903 >> 172.21.15.45:0/3858273903 conn(0x7f211c0cd4b0 msgr2=0x7f211c0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- 172.21.15.45:0/3858273903 shutdown_connections 2024-04-25T10:32:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- 172.21.15.45:0/3858273903 wait complete. 2024-04-25T10:32:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 Processor -- start 2024-04-25T10:32:43.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- start start 2024-04-25T10:32:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f211c0b2910 0x7f211c154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 0x7f211c0c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c15e280 con 0x7f211c15dd20 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c15e100 con 0x7f211c0b2910 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c06ecc0 con 0x7f211c162660 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 0x7f211c0c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 0x7f211c0c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39944/0 (socket says 172.21.15.45:39944) 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 -- 172.21.15.45:0/2847965790 learned_addr learned my addr 172.21.15.45:0/2847965790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122200640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2121800640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f211c0b2910 0x7f211c154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 -- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 msgr2=0x7f211c0c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 -- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f211c0b2910 msgr2=0x7f211c154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f211c0b2910 0x7f211c154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2120e00640 1 -- 172.21.15.45:0/2847965790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c154580 con 0x7f211c15dd20 2024-04-25T10:32:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.090+0000 7f2122200640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2120e00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 0x7f211c0c69c0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f210800c820 tx=0x7f210800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2119600640 1 -- 172.21.15.45:0/2847965790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108015070 con 0x7f211c15dd20 2024-04-25T10:32:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2119600640 1 -- 172.21.15.45:0/2847965790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108004220 con 0x7f211c15dd20 2024-04-25T10:32:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2119600640 1 -- 172.21.15.45:0/2847965790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210800ce10 con 0x7f211c15dd20 2024-04-25T10:32:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c15b860 con 0x7f211c15dd20 2024-04-25T10:32:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c15be20 con 0x7f211c15dd20 2024-04-25T10:32:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2119600640 1 -- 172.21.15.45:0/2847965790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2108007830 con 0x7f211c15dd20 2024-04-25T10:32:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2119600640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20fc066f60 0x7f20fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.094+0000 7f2119600640 1 -- 172.21.15.45:0/2847965790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2108085200 con 0x7f211c15dd20 2024-04-25T10:32:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20fc066f60 0x7f20fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 Processor -- start 2024-04-25T10:32:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- start start 2024-04-25T10:32:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20fc066f60 0x7f20fc069420 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2110004e30 tx=0x7f2110009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x559495467070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x55949541cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x55949541d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x559495303f40 con 0x559495466c90 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55949526b070 con 0x55949541d290 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594951f31a0 con 0x5594954675b0 2024-04-25T10:32:43.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x559495467070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x559495467070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39956/0 (socket says 172.21.15.45:39956) 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 -- 172.21.15.45:0/1446364461 learned_addr learned my addr 172.21.15.45:0/1446364461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2122200640 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x55949541d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2120e00640 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x55949541cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 -- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 msgr2=0x55949541cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x55949541cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 -- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 msgr2=0x55949541d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x55949541d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 -- 172.21.15.45:0/1446364461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594951f1510 con 0x559495466c90 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f2121800640 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x559495467070 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f21100055d0 tx=0x7f2110011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f888d7bbe194df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f210e000640 1 -- 172.21.15.45:0/1446364461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110025070 con 0x559495466c90 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f210e000640 1 -- 172.21.15.45:0/1446364461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2110015880 con 0x559495466c90 2024-04-25T10:32:43.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f210e000640 1 -- 172.21.15.45:0/1446364461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110015c30 con 0x559495466c90 2024-04-25T10:32:43.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- 172.21.15.45:0/1446364461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 msgr2=0x559495467070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x559495467070 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f21100055d0 tx=0x7f2110011000 comp rx=0 tx=0).stop 2024-04-25T10:32:43.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- 172.21.15.45:0/1446364461 shutdown_connections 2024-04-25T10:32:43.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x55949541d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x55949541cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 --2- 172.21.15.45:0/1446364461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x559495467070 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- 172.21.15.45:0/1446364461 >> 172.21.15.45:0/1446364461 conn(0x559495419240 msgr2=0x55949541b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- 172.21.15.45:0/1446364461 shutdown_connections 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- 172.21.15.45:0/1446364461 wait complete. 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.098+0000 7f21276b4000 1 -- start start 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 Processor -- start 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x559495427e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559495466c90 0x559495428370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594952256c0 con 0x5594954675b0 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559495267110 con 0x55949541d290 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559495426200 con 0x559495466c90 2024-04-25T10:32:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 10 client.0 _ll_register_callbacks cb 0x559495408570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2122200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559495466c90 0x559495428370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2122200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39964/0 (socket says 172.21.15.45:39964) 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2122200640 1 -- 172.21.15.45:0/2263748267 learned_addr learned my addr 172.21.15.45:0/2263748267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2121800640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x559495427e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 -- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 msgr2=0x559495427e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x559495427e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 -- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 msgr2=0x5594954288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 -- 172.21.15.45:0/2263748267 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594951f33b0 con 0x559495466c90 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2121800640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x559495427e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2122200640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2120e00640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559495466c90 0x559495428370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f210800c7f0 tx=0x7f21080562e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f0df19bbcd2dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3301/0 2024-04-25T10:32:43.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 1 -- 172.21.15.45:0/2263748267 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210801e040 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 1 -- 172.21.15.45:0/2263748267 <== mon.2 v2:172.21.15.117:3301/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f210805df20 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2110015880 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 1 -- 172.21.15.45:0/2263748267 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210805e1b0 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 10 client.9263 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5594952ece00 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 1 -- 172.21.15.45:0/2263748267 <== mon.2 v2:172.21.15.117:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2110015880 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 1 -- 172.21.15.45:0/2263748267 <== mon.2 v2:172.21.15.117:3301/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21080830d0 con 0x559495466c90 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 10 client.9263 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 10 client.9263 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 20 client.9263 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495429180 0x559495478410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 4 client.9263 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559495206d60 con 0x559495429180 2024-04-25T10:32:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2121800640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495429180 0x559495478410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2121800640 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495429180 0x559495478410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72afe8b9aaf1f001 server_cookie=5678b05dc1247478 in_seq=0 out_seq=0 2024-04-25T10:32:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.102+0000 7f2102a00640 10 client.9263 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f2102a00640 1 -- 172.21.15.45:0/2263748267 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559495206d60 con 0x559495429180 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f2102a00640 10 client.9263 handle_command_reply: tid=0 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 client.9263 shutdown 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559495466c90 msgr2=0x559495428370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559495466c90 0x559495428370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f210800c7f0 tx=0x7f21080562e0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 shutdown_connections 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495429180 0x559495478410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x559495466c90 0x559495428370 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f2102a00640 0 client.9263 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 --2- 172.21.15.45:0/2263748267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x559495427e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 >> 172.21.15.45:0/2263748267 conn(0x559495419240 msgr2=0x55949541bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 shutdown_connections 2024-04-25T10:32:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 1 -- 172.21.15.45:0/2263748267 wait complete. 2024-04-25T10:32:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.106+0000 7f21276b4000 20 client.9263 trim_cache size 0 max 16384 2024-04-25T10:32:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 Processor -- start 2024-04-25T10:32:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- start start 2024-04-25T10:32:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55949541d290 0x55949547a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559495466c90 0x55949547a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c202cd0 con 0x5594954675b0 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0aee70 con 0x559495466c90 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1552d0 con 0x55949541d290 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39980/0 (socket says 172.21.15.45:39980) 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 -- 172.21.15.45:0/2311784607 learned_addr learned my addr 172.21.15.45:0/2311784607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2121800640 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55949541d290 0x55949547a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2120e00640 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559495466c90 0x55949547a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 -- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55949541d290 msgr2=0x55949547a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55949541d290 0x55949547a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 -- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559495466c90 msgr2=0x55949547a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559495466c90 0x55949547a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 -- 172.21.15.45:0/2311784607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594952c2c50 con 0x5594954675b0 2024-04-25T10:32:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f2122200640 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954c82d0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f21140093e0 tx=0x7f2114004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4373dad448790911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f210d600640 1 -- 172.21.15.45:0/2311784607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140c4070 con 0x5594954675b0 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f210d600640 1 -- 172.21.15.45:0/2311784607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140a5930 con 0x5594954675b0 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f210d600640 1 -- 172.21.15.45:0/2311784607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140a5ce0 con 0x5594954675b0 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- 172.21.15.45:0/2311784607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 msgr2=0x5594954c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954c82d0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f21140093e0 tx=0x7f2114004ac0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- 172.21.15.45:0/2311784607 shutdown_connections 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5594954675b0 0x5594954c82d0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559495466c90 0x55949547a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 --2- 172.21.15.45:0/2311784607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55949541d290 0x55949547a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- 172.21.15.45:0/2311784607 >> 172.21.15.45:0/2311784607 conn(0x559495419240 msgr2=0x559495421da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- 172.21.15.45:0/2311784607 shutdown_connections 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- 172.21.15.45:0/2311784607 wait complete. 2024-04-25T10:32:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 -- start start 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.122+0000 7f21276b4000 1 Processor -- start 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x5594954c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x5594954c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x559495481e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55949520dbd0 con 0x559495466c90 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0ae870 con 0x55949541d290 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x559495321f20 con 0x5594954675b0 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 10 client.0 _ll_register_callbacks cb 0x7f2110007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x5594954c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x5594954c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39982/0 (socket says 172.21.15.45:39982) 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 -- 172.21.15.45:0/1134979502 learned_addr learned my addr 172.21.15.45:0/1134979502 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2121800640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x5594954c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2122200640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x559495481e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 -- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 msgr2=0x559495481e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x559495481e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 -- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 msgr2=0x5594954c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x5594954c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 -- 172.21.15.45:0/1134979502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559495233120 con 0x559495466c90 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2120e00640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x5594954c73f0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f2108055a30 tx=0x7f2108083590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0afd4558cb9e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 1 -- 172.21.15.45:0/1134979502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108078060 con 0x559495466c90 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 1 -- 172.21.15.45:0/1134979502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2108007b20 con 0x559495466c90 2024-04-25T10:32:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21140a5930 con 0x559495466c90 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 1 -- 172.21.15.45:0/1134979502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210801e040 con 0x559495466c90 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 10 client.11625 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x559495212170 con 0x559495466c90 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 1 -- 172.21.15.45:0/1134979502 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21140a5930 con 0x559495466c90 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 1 -- 172.21.15.45:0/1134979502 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f210805ab10 con 0x559495466c90 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 10 client.11625 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 10 client.11625 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 20 client.11625 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495482820 0x559495482c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 4 client.11625 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x5594952785b0 con 0x559495482820 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2121800640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495482820 0x559495482c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2121800640 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495482820 0x559495482c00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbeee8e61454980 server_cookie=1d9b7a7a1e9e76ef in_seq=0 out_seq=0 2024-04-25T10:32:43.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 10 client.11625 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 1 -- 172.21.15.45:0/1134979502 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5594952785b0 con 0x559495482820 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f2102a00640 10 client.11625 handle_command_reply: tid=0 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.126+0000 7f21276b4000 1 client.11625 shutdown 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 msgr2=0x5594954c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x5594954c73f0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f2108055a30 tx=0x7f2108083590 comp rx=0 tx=0).stop 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 shutdown_connections 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x559495482820 0x559495482c00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x5594954675b0 0x559495481e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559495466c90 0x5594954c73f0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 --2- 172.21.15.45:0/1134979502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55949541d290 0x5594954c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 >> 172.21.15.45:0/1134979502 conn(0x559495419240 msgr2=0x5594954224b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2102a00640 0 client.11625 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 shutdown_connections 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 1 -- 172.21.15.45:0/1134979502 wait complete. 2024-04-25T10:32:43.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f21276b4000 20 client.11625 trim_cache size 0 max 16384 2024-04-25T10:32:43.135 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:43.135 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-25T10:32:43.135 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "5aeac735-59a1-4231-a473-d0a173734d3c", 2024-04-25T10:32:43.135 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-25T10:32:43.135 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20fc066f60 msgr2=0x7f20fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20fc066f60 0x7f20fc069420 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2110004e30 tx=0x7f2110009040 comp rx=0 tx=0).stop 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 msgr2=0x7f211c0c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 0x7f211c0c69c0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f210800c820 tx=0x7f210800cce0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 shutdown_connections 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f20fc066f60 0x7f20fc069420 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f211c162660 0x7f211c0c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f211c15dd20 0x7f211c0c69c0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 --2- 172.21.15.45:0/2847965790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f211c0b2910 0x7f211c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 >> 172.21.15.45:0/2847965790 conn(0x7f211c0cd4b0 msgr2=0x7f211c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 shutdown_connections 2024-04-25T10:32:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.130+0000 7f2122c00640 1 -- 172.21.15.45:0/2847965790 wait complete. 2024-04-25T10:32:43.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:32:43.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:32:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.290+0000 7fc1b6800640 1 Processor -- start 2024-04-25T10:32:43.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.290+0000 7fc1b6800640 1 -- start start 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1b6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1b0070350 0x7fc1b015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1b6800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1b00b31c0 0x7fc1b00b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1b6800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b015b080 con 0x7fc1b00aef80 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1b6800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b015adb0 con 0x7fc1b0070350 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1b6800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b015eb60 con 0x7fc1b00b31c0 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1a6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1b0070350 0x7fc1b015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1af400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc1b00aef80 0x7fc1b015b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41050/0 (socket says 172.21.15.45:41050) 2024-04-25T10:32:43.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1af400640 1 -- 172.21.15.45:0/2307851019 learned_addr learned my addr 172.21.15.45:0/2307851019 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1a6a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1b0070350 0x7fc1b015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39984/0 (socket says 172.21.15.45:39984) 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1afe00640 1 --2- 172.21.15.45:0/2307851019 >> v2:172.21.15.117:3300/0 conn(0x7fc1b00b31c0 0x7fc1b00b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2307851019 >> v2:172.21.15.117:3300/0 conn(0x7fc1b00b31c0 msgr2=0x7fc1b00b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/2307851019 >> v2:172.21.15.117:3300/0 conn(0x7fc1b00b31c0 0x7fc1b00b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2307851019 >> v1:172.21.15.45:6789/0 conn(0x7fc1b00aef80 legacy=0x7fc1b015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2307851019 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b015ee40 con 0x7fc1b0070350 2024-04-25T10:32:43.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.294+0000 7fc1aea00640 1 -- 172.21.15.45:0/2307851019 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357473181 0 0) 0x7fc1b015b080 con 0x7fc1b00aef80 2024-04-25T10:32:43.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/2307851019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b0070350 0x7fc1b015e480 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00d0750 tx=0x7fc19801eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51dac8169dc96d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1aea00640 1 -- 172.21.15.45:0/2307851019 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980b4070 con 0x7fc1b0070350 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1aea00640 1 -- 172.21.15.45:0/2307851019 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1980a7b30 con 0x7fc1b0070350 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1aea00640 1 -- 172.21.15.45:0/2307851019 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980af440 con 0x7fc1b0070350 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 -- 172.21.15.45:0/2307851019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b0070350 msgr2=0x7fc1b015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2307851019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b0070350 0x7fc1b015e480 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00d0750 tx=0x7fc19801eb50 comp rx=0 tx=0).stop 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 -- 172.21.15.45:0/2307851019 shutdown_connections 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2307851019 >> v2:172.21.15.117:3300/0 conn(0x7fc1b00b31c0 0x7fc1b00b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2307851019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b0070350 0x7fc1b015e480 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 -- 172.21.15.45:0/2307851019 >> 172.21.15.45:0/2307851019 conn(0x7fc1b00669a0 msgr2=0x7fc1b0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 -- 172.21.15.45:0/2307851019 shutdown_connections 2024-04-25T10:32:43.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 -- 172.21.15.45:0/2307851019 wait complete. 2024-04-25T10:32:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 Processor -- start 2024-04-25T10:32:43.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.298+0000 7fc1b6800640 1 -- start start 2024-04-25T10:32:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b0070350 0x7fc1b0146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 0x7fc1b01f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b00b31c0 0x7fc1b0157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006c250 con 0x7fc1b00aef80 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006c0d0 con 0x7fc1b0070350 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006c3d0 con 0x7fc1b00b31c0 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 0x7fc1b01f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 0x7fc1b01f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39992/0 (socket says 172.21.15.45:39992) 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2776155911 learned_addr learned my addr 172.21.15.45:0/2776155911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1af400640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b0070350 0x7fc1b0146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1afe00640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b00b31c0 0x7fc1b0157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b00b31c0 msgr2=0x7fc1b0157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b00b31c0 0x7fc1b0157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b0070350 msgr2=0x7fc1b0146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b0070350 0x7fc1b0146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 -- 172.21.15.45:0/2776155911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0155490 con 0x7fc1b00aef80 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1af400640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b0070350 0x7fc1b0146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:43.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1afe00640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b00b31c0 0x7fc1b0157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 0x7fc1b01f6e50 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b2040 tx=0x7fc19801b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a7e00640 1 -- 172.21.15.45:0/2776155911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980bf070 con 0x7fc1b00aef80 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a7e00640 1 -- 172.21.15.45:0/2776155911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1980afb70 con 0x7fc1b00aef80 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a7e00640 1 -- 172.21.15.45:0/2776155911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980b4040 con 0x7fc1b00aef80 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b0155770 con 0x7fc1b00aef80 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b01466c0 con 0x7fc1b00aef80 2024-04-25T10:32:43.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.302+0000 7fc1a7e00640 1 -- 172.21.15.45:0/2776155911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1980c2070 con 0x7fc1b00aef80 2024-04-25T10:32:43.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1a7e00640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc18c066f60 0x7fc18c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1a7e00640 1 -- 172.21.15.45:0/2776155911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc19811f560 con 0x7fc1b00aef80 2024-04-25T10:32:43.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc18c066f60 0x7fc18c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 Processor -- start 2024-04-25T10:32:43.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 -- start start 2024-04-25T10:32:43.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc18c066f60 0x7fc18c069420 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a0005b80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 0x55ecbd02ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbcfe40a0 0x55ecbcfe4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecbce30ff0 con 0x55ecbd02d9d0 2024-04-25T10:32:43.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecbcdb9150 con 0x55ecbd02e2f0 2024-04-25T10:32:43.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecbce2d090 con 0x55ecbcfe40a0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 0x55ecbd02ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 0x55ecbd02ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40006/0 (socket says 172.21.15.45:40006) 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 -- 172.21.15.45:0/2385288600 learned_addr learned my addr 172.21.15.45:0/2385288600 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1afe00640 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbcfe40a0 0x55ecbcfe4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 -- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbcfe40a0 msgr2=0x55ecbcfe4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbcfe40a0 0x55ecbcfe4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 -- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02e2f0 msgr2=0x55ecbcfe3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 -- 172.21.15.45:0/2385288600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecbcfc70d0 con 0x55ecbd02d9d0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc1af400640 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 0x55ecbd02ddb0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0001de0 tx=0x7fc1a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f305bb762ea1d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc19fe00640 1 -- 172.21.15.45:0/2385288600 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0015b40 con 0x55ecbd02d9d0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc19fe00640 1 -- 172.21.15.45:0/2385288600 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a0015ce0 con 0x55ecbd02d9d0 2024-04-25T10:32:43.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.306+0000 7fc19fe00640 1 -- 172.21.15.45:0/2385288600 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0021650 con 0x55ecbd02d9d0 2024-04-25T10:32:43.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/2385288600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 msgr2=0x55ecbd02ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 0x55ecbd02ddb0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0001de0 tx=0x7fc1a0012000 comp rx=0 tx=0).stop 2024-04-25T10:32:43.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/2385288600 shutdown_connections 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbcfe40a0 0x55ecbcfe4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/2385288600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02d9d0 0x55ecbd02ddb0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/2385288600 >> 172.21.15.45:0/2385288600 conn(0x55ecbcfe0020 msgr2=0x55ecbcfe2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/2385288600 shutdown_connections 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/2385288600 wait complete. 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- start start 2024-04-25T10:32:43.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 Processor -- start 2024-04-25T10:32:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 0x55ecbd0127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02d9d0 0x55ecbd012d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02e2f0 0x55ecbd013270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecbcdf5250 con 0x55ecbcfe40a0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecbcdeb690 con 0x55ecbd02d9d0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecbcdb9a40 con 0x55ecbd02e2f0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 10 client.0 _ll_register_callbacks cb 0x55ecbcfcee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 0x55ecbd0127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 0x55ecbd0127f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40010/0 (socket says 172.21.15.45:40010) 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 -- 172.21.15.45:0/4003604805 learned_addr learned my addr 172.21.15.45:0/4003604805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02d9d0 0x55ecbd012d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1afe00640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02e2f0 0x55ecbd013270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 -- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02e2f0 msgr2=0x55ecbd013270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02e2f0 0x55ecbd013270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 -- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02d9d0 msgr2=0x55ecbd012d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02d9d0 0x55ecbd012d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 -- 172.21.15.45:0/4003604805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecbcede1c0 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1af400640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 0x55ecbd0127f0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00012f0 tx=0x7fc1a0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8ae0195cd59697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc19cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc19cc00640 1 -- 172.21.15.45:0/4003604805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0015b40 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc19cc00640 1 -- 172.21.15.45:0/4003604805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc1a0021a70 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1a0015ce0 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc19cc00640 1 -- 172.21.15.45:0/4003604805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a0033050 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 10 client.11640 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55ecbceda170 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.310+0000 7fc19cc00640 1 -- 172.21.15.45:0/4003604805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a0015ce0 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc19cc00640 1 -- 172.21.15.45:0/4003604805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1a0036020 con 0x55ecbcfe40a0 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 10 client.11640 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 10 client.11640 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:43.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 20 client.11640 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd03dc80 0x55ecbd040070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 4 client.11640 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecbcdccc60 con 0x55ecbd03dc80 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd03dc80 0x55ecbd040070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd03dc80 0x55ecbd040070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d935037eeb6fdc server_cookie=d3618e5d06102732 in_seq=0 out_seq=0 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc19cc00640 10 client.11640 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc19cc00640 1 -- 172.21.15.45:0/4003604805 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecbcdccc60 con 0x55ecbd03dc80 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc19cc00640 10 client.11640 handle_command_reply: tid=0 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 client.11640 shutdown 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 msgr2=0x55ecbd0127f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 0x55ecbd0127f0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00012f0 tx=0x7fc1a0011000 comp rx=0 tx=0).stop 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 shutdown_connections 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd03dc80 0x55ecbd040070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02e2f0 0x55ecbd013270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbd02d9d0 0x55ecbd012d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/4003604805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbcfe40a0 0x55ecbd0127f0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 >> 172.21.15.45:0/4003604805 conn(0x55ecbcfe0020 msgr2=0x55ecbcfe2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc19cc00640 0 client.11640 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 shutdown_connections 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/4003604805 wait complete. 2024-04-25T10:32:43.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.314+0000 7fc1b9d4b000 20 client.11640 trim_cache size 0 max 16384 2024-04-25T10:32:43.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 Processor -- start 2024-04-25T10:32:43.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- start start 2024-04-25T10:32:43.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbcfe2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbcfe2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0202fd0 con 0x55ecbd02e2f0 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0154a60 con 0x55ecbcfe40a0 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0153e60 con 0x55ecbd02d9d0 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40018/0 (socket says 172.21.15.45:40018) 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 -- 172.21.15.45:0/110884785 learned_addr learned my addr 172.21.15.45:0/110884785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbcfe2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1af400640 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbcfe2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 -- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 msgr2=0x55ecbcfe2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbcfe2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 -- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 msgr2=0x55ecbcfe2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbcfe2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 -- 172.21.15.45:0/110884785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecbce77de0 con 0x55ecbd02e2f0 2024-04-25T10:32:43.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1afe00640 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe2e90 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009aa0 tx=0x7fc1a8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d2d714cb08877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc19f400640 1 -- 172.21.15.45:0/110884785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a801b070 con 0x55ecbd02e2f0 2024-04-25T10:32:43.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc19f400640 1 -- 172.21.15.45:0/110884785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8015c60 con 0x55ecbd02e2f0 2024-04-25T10:32:43.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc19f400640 1 -- 172.21.15.45:0/110884785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8020460 con 0x55ecbd02e2f0 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/110884785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 msgr2=0x55ecbcfe2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe2e90 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009aa0 tx=0x7fc1a8009190 comp rx=0 tx=0).stop 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/110884785 shutdown_connections 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbcfe2e90 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbcfe2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/110884785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbcfe2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/110884785 >> 172.21.15.45:0/110884785 conn(0x55ecbcfe0020 msgr2=0x55ecbd03afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/110884785 shutdown_connections 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.330+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/110884785 wait complete. 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- start start 2024-04-25T10:32:43.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 Processor -- start 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbd08d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbd08de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbd03ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecbcdcbe00 con 0x55ecbd02e2f0 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0154460 con 0x55ecbcfe40a0 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ecbce30300 con 0x55ecbd02d9d0 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 10 client.0 _ll_register_callbacks cb 0x55ecbcfaf040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbd08d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbd03ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbd03ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40020/0 (socket says 172.21.15.45:40020) 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 -- 172.21.15.45:0/3401579278 learned_addr learned my addr 172.21.15.45:0/3401579278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbd08d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35918/0 (socket says 172.21.15.45:35918) 2024-04-25T10:32:43.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1a6a00640 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbd08de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 -- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 msgr2=0x55ecbd08de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbd08de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 -- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 msgr2=0x55ecbd08d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbd08d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 -- 172.21.15.45:0/3401579278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecbcdd28e0 con 0x55ecbd02e2f0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1afe00640 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbd03ffc0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8008010 tx=0x7fc1a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a4f16cf1393e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc19cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc19cc00640 1 -- 172.21.15.45:0/3401579278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a802f060 con 0x55ecbd02e2f0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc19cc00640 1 -- 172.21.15.45:0/3401579278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(194..194 src has 1..194) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc1a8020660 con 0x55ecbd02e2f0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1a8015c60 con 0x55ecbd02e2f0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc19cc00640 1 -- 172.21.15.45:0/3401579278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a8035050 con 0x55ecbd02e2f0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 10 client.11646 fetch_fsmap learned FSMap version 146 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55ecbce8a8f0 con 0x55ecbd02e2f0 2024-04-25T10:32:43.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc19cc00640 1 -- 172.21.15.45:0/3401579278 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8015c60 con 0x55ecbd02e2f0 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc19cc00640 1 -- 172.21.15.45:0/3401579278 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1a8038020 con 0x55ecbd02e2f0 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 10 client.11646 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 10 client.11646 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 20 client.11646 populate_metadata read hostname 'smithi045' 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd040b10 0x55ecbd040f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 4 client.11646 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ecbce3e530 con 0x55ecbd040b10 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.334+0000 7fc1af400640 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd040b10 0x55ecbd040f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1af400640 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd040b10 0x55ecbd040f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43240bac5a65af2f server_cookie=a12206c27d4e2e32 in_seq=0 out_seq=0 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc19cc00640 10 client.11646 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc19cc00640 1 -- 172.21.15.45:0/3401579278 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55ecbce3e530 con 0x55ecbd040b10 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc19cc00640 10 client.11646 handle_command_reply: tid=0 2024-04-25T10:32:43.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 client.11646 shutdown 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 msgr2=0x55ecbd03ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbd03ffc0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8008010 tx=0x7fc1a8004740 comp rx=0 tx=0).stop 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 shutdown_connections 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x55ecbd040b10 0x55ecbd040f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecbd02e2f0 0x55ecbd03ffc0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ecbd02d9d0 0x55ecbd08de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 --2- 172.21.15.45:0/3401579278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecbcfe40a0 0x55ecbd08d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 >> 172.21.15.45:0/3401579278 conn(0x55ecbcfe0020 msgr2=0x55ecbd02c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc19cc00640 0 client.11646 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 shutdown_connections 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 1 -- 172.21.15.45:0/3401579278 wait complete. 2024-04-25T10:32:43.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b9d4b000 20 client.11646 trim_cache size 0 max 16384 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: "5aeac735-59a1-4231-a473-d0a173734d3c": { 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "5aeac735-59a1-4231-a473-d0a173734d3c", 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T10:32:43.345 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:32:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc18c066f60 msgr2=0x7fc18c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc18c066f60 0x7fc18c069420 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a0005b80 comp rx=0 tx=0).stop 2024-04-25T10:32:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 msgr2=0x7fc1b01f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:32:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 0x7fc1b01f6e50 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fc1980b2040 tx=0x7fc19801b8f0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 shutdown_connections 2024-04-25T10:32:43.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc18c066f60 0x7fc18c069420 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1b00b31c0 0x7fc1b0157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1b00aef80 0x7fc1b01f6e50 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 --2- 172.21.15.45:0/2776155911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1b0070350 0x7fc1b0146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:32:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 >> 172.21.15.45:0/2776155911 conn(0x7fc1b00669a0 msgr2=0x7fc1b00b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:32:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 shutdown_connections 2024-04-25T10:32:43.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:32:43.338+0000 7fc1b6800640 1 -- 172.21.15.45:0/2776155911 wait complete. 2024-04-25T10:32:43.360 INFO:tasks.cephfs.filesystem:scrub status for tag:5aeac735-59a1-4231-a473-d0a173734d3c - {'path': '/', 'tag': '5aeac735-59a1-4231-a473-d0a173734d3c', 'options': 'recursive,repair'} 2024-04-25T10:33:00.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:00.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:13.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:13.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:33:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 Processor -- start 2024-04-25T10:33:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 -- start start 2024-04-25T10:33:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 0x7fdb6815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 0x7fdb6815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6815b080 con 0x7fdb6815bac0 2024-04-25T10:33:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6815adb0 con 0x7fdb68070350 2024-04-25T10:33:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb6da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6815eb60 con 0x7fdb680aef80 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb5e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 0x7fdb6815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb66a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 0x7fdb6815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb66a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 0x7fdb6815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:50582/0 (socket says 172.21.15.45:50582) 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb5e000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 0x7fdb6815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36384/0 (socket says 172.21.15.45:36384) 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.513+0000 7fdb66a00640 1 -- 172.21.15.45:0/1737972083 learned_addr learned my addr 172.21.15.45:0/1737972083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66a00640 1 -- 172.21.15.45:0/1737972083 >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 msgr2=0x7fdb6815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66a00640 1 --2- 172.21.15.45:0/1737972083 >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 0x7fdb6815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66a00640 1 -- 172.21.15.45:0/1737972083 >> v1:172.21.15.117:6789/0 conn(0x7fdb6815bac0 legacy=0x7fdb680b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66a00640 1 -- 172.21.15.45:0/1737972083 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6815ee40 con 0x7fdb680aef80 2024-04-25T10:33:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb5e000640 1 --2- 172.21.15.45:0/1737972083 >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 0x7fdb6815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:13.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66a00640 1 --2- 172.21.15.45:0/1737972083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 0x7fdb6815e480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009870 tx=0x7fdb5801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abbda01d2dc1dcbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66000640 1 -- 172.21.15.45:0/1737972083 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580a9400 con 0x7fdb680aef80 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66000640 1 -- 172.21.15.45:0/1737972083 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb580a95a0 con 0x7fdb680aef80 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb66000640 1 -- 172.21.15.45:0/1737972083 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580a9970 con 0x7fdb680aef80 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb6da00640 1 -- 172.21.15.45:0/1737972083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 msgr2=0x7fdb6815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.517+0000 7fdb6da00640 1 --2- 172.21.15.45:0/1737972083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 0x7fdb6815e480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009870 tx=0x7fdb5801c560 comp rx=0 tx=0).stop 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- 172.21.15.45:0/1737972083 shutdown_connections 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 --2- 172.21.15.45:0/1737972083 >> v2:172.21.15.117:3300/0 conn(0x7fdb68070350 0x7fdb6815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 --2- 172.21.15.45:0/1737972083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb680aef80 0x7fdb6815e480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- 172.21.15.45:0/1737972083 >> 172.21.15.45:0/1737972083 conn(0x7fdb680669a0 msgr2=0x7fdb68066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- 172.21.15.45:0/1737972083 shutdown_connections 2024-04-25T10:33:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- 172.21.15.45:0/1737972083 wait complete. 2024-04-25T10:33:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 Processor -- start 2024-04-25T10:33:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- start start 2024-04-25T10:33:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb68070350 0x7fdb68151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 0x7fdb681fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb6815bac0 0x7fdb6815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6806c250 con 0x7fdb680aef80 2024-04-25T10:33:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6806c0d0 con 0x7fdb68070350 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb6da00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6806c3d0 con 0x7fdb6815bac0 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 0x7fdb681fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 0x7fdb681fa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51876/0 (socket says 172.21.15.45:51876) 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 -- 172.21.15.45:0/3781893858 learned_addr learned my addr 172.21.15.45:0/3781893858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb66a00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb68070350 0x7fdb68151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb67400640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb6815bac0 0x7fdb6815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 -- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb6815bac0 msgr2=0x7fdb6815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb6815bac0 0x7fdb6815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 -- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb68070350 msgr2=0x7fdb68151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb68070350 0x7fdb68151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 -- 172.21.15.45:0/3781893858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb680afcf0 con 0x7fdb680aef80 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb66a00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb68070350 0x7fdb68151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:13.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.521+0000 7fdb5e000640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 0x7fdb681fa5c0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000ebe0 tx=0x7fdb5000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb5f400640 1 -- 172.21.15.45:0/3781893858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50017070 con 0x7fdb680aef80 2024-04-25T10:33:13.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb5f400640 1 -- 172.21.15.45:0/3781893858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5000cc30 con 0x7fdb680aef80 2024-04-25T10:33:13.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb680affd0 con 0x7fdb680aef80 2024-04-25T10:33:13.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb5f400640 1 -- 172.21.15.45:0/3781893858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50013470 con 0x7fdb680aef80 2024-04-25T10:33:13.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb680cca10 con 0x7fdb680aef80 2024-04-25T10:33:13.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb5f400640 1 -- 172.21.15.45:0/3781893858 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb5001a070 con 0x7fdb680aef80 2024-04-25T10:33:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb5f400640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb44066f90 0x7fdb44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.525+0000 7fdb5f400640 1 -- 172.21.15.45:0/3781893858 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fdb50085430 con 0x7fdb680aef80 2024-04-25T10:33:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb66a00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb44066f90 0x7fdb44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 Processor -- start 2024-04-25T10:33:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- start start 2024-04-25T10:33:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb66a00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb44066f90 0x7fdb44069450 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdb580ae000 tx=0x7fdb580b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c031f4ff0 con 0x556c033f22f0 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0317d150 con 0x556c033f19d0 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c031f1090 con 0x556c033a80a0 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033a7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51878/0 (socket says 172.21.15.45:51878) 2024-04-25T10:33:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 -- 172.21.15.45:0/3464053946 learned_addr learned my addr 172.21.15.45:0/3464053946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb66a00640 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb67400640 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 -- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 msgr2=0x556c033a8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033a8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 -- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 msgr2=0x556c033f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 -- 172.21.15.45:0/3464053946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c0338b0d0 con 0x556c033f22f0 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb66a00640 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb5e000640 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033a7b60 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fdb500520a0 tx=0x7fdb5005c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5577b8f2f87f3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb57400640 1 -- 172.21.15.45:0/3464053946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50021050 con 0x556c033f22f0 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb57400640 1 -- 172.21.15.45:0/3464053946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50059d90 con 0x556c033f22f0 2024-04-25T10:33:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb57400640 1 -- 172.21.15.45:0/3464053946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5005e350 con 0x556c033f22f0 2024-04-25T10:33:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- 172.21.15.45:0/3464053946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 msgr2=0x556c033a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033a7b60 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7fdb500520a0 tx=0x7fdb5005c050 comp rx=0 tx=0).stop 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- 172.21.15.45:0/3464053946 shutdown_connections 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033a8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033a7b60 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 --2- 172.21.15.45:0/3464053946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- 172.21.15.45:0/3464053946 >> 172.21.15.45:0/3464053946 conn(0x556c033a4020 msgr2=0x556c033a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- 172.21.15.45:0/3464053946 shutdown_connections 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- 172.21.15.45:0/3464053946 wait complete. 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.529+0000 7fdb70f99000 1 -- start start 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 Processor -- start 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033d67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c031b9250 con 0x556c033f22f0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c031af690 con 0x556c033f19d0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c0317da40 con 0x556c033a80a0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 10 client.0 _ll_register_callbacks cb 0x556c03392e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033d7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51890/0 (socket says 172.21.15.45:51890) 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb67400640 1 -- 172.21.15.45:0/1425793354 learned_addr learned my addr 172.21.15.45:0/1425793354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb66a00640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033d67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 -- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 msgr2=0x556c033d67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033d67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 -- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 msgr2=0x556c033d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 -- 172.21.15.45:0/1425793354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c032a21c0 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb67400640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb5e000640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033d6d30 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000ca20 tx=0x7fdb50000bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cbb4deac7285ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 1 -- 172.21.15.45:0/1425793354 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500522b0 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 1 -- 172.21.15.45:0/1425793354 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fdb500833f0 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb50059d90 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 1 -- 172.21.15.45:0/1425793354 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb50021070 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 10 client.10828 fetch_fsmap learned FSMap version 146 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x556c0329e170 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 1 -- 172.21.15.45:0/1425793354 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50059d90 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 1 -- 172.21.15.45:0/1425793354 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb50082030 con 0x556c033f19d0 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 10 client.10828 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 10 client.10828 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 20 client.10828 populate_metadata read hostname 'smithi045' 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c03401c80 0x556c03404070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 4 client.10828 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c03190c60 con 0x556c03401c80 2024-04-25T10:33:13.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb66a00640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c03401c80 0x556c03404070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb66a00640 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c03401c80 0x556c03404070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e1e13ce78dffde server_cookie=740b59915a1f17d6 in_seq=0 out_seq=0 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.533+0000 7fdb4be00640 10 client.10828 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb4be00640 1 -- 172.21.15.45:0/1425793354 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c03190c60 con 0x556c03401c80 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb4be00640 10 client.10828 handle_command_reply: tid=0 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 client.10828 shutdown 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 msgr2=0x556c033d6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033d6d30 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000ca20 tx=0x7fdb50000bd0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 shutdown_connections 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c03401c80 0x556c03404070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c033d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c033d6d30 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 --2- 172.21.15.45:0/1425793354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c033d67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb4be00640 0 client.10828 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 >> 172.21.15.45:0/1425793354 conn(0x556c033a4020 msgr2=0x556c033a6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 shutdown_connections 2024-04-25T10:33:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 1 -- 172.21.15.45:0/1425793354 wait complete. 2024-04-25T10:33:13.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.537+0000 7fdb70f99000 20 client.10828 trim_cache size 0 max 16384 2024-04-25T10:33:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.549+0000 7fdb70f99000 1 Processor -- start 2024-04-25T10:33:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- start start 2024-04-25T10:33:13.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 0x556c0344e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c0344e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033f22f0 0x556c0344ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68203340 con 0x556c033a80a0 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68202520 con 0x556c033f19d0 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680cd6b0 con 0x556c033f22f0 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 0x556c0344e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 0x556c0344e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51896/0 (socket says 172.21.15.45:51896) 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 -- 172.21.15.45:0/570536079 learned_addr learned my addr 172.21.15.45:0/570536079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 -- 172.21.15.45:0/570536079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033f22f0 msgr2=0x556c0344ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033f22f0 0x556c0344ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 -- 172.21.15.45:0/570536079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 msgr2=0x556c0344e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c0344e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 -- 172.21.15.45:0/570536079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c0323bde0 con 0x556c033a80a0 2024-04-25T10:33:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb66a00640 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 0x556c0344e300 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fdb580a9800 tx=0x7fdb580a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ae5e27c09821fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb56a00640 1 -- 172.21.15.45:0/570536079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580bbad0 con 0x556c033a80a0 2024-04-25T10:33:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb56a00640 1 -- 172.21.15.45:0/570536079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb580bbc70 con 0x556c033a80a0 2024-04-25T10:33:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb56a00640 1 -- 172.21.15.45:0/570536079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580ca250 con 0x556c033a80a0 2024-04-25T10:33:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- 172.21.15.45:0/570536079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 msgr2=0x556c0344e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 0x556c0344e300 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fdb580a9800 tx=0x7fdb580a7000 comp rx=0 tx=0).stop 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- 172.21.15.45:0/570536079 shutdown_connections 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033f22f0 0x556c0344ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c0344e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 --2- 172.21.15.45:0/570536079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033a80a0 0x556c0344e300 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- 172.21.15.45:0/570536079 >> 172.21.15.45:0/570536079 conn(0x556c033a4020 msgr2=0x556c033fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- 172.21.15.45:0/570536079 shutdown_connections 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- 172.21.15.45:0/570536079 wait complete. 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 -- start start 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.553+0000 7fdb70f99000 1 Processor -- start 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c03404380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c034048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c03404e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0318fe00 con 0x556c033f22f0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb681559d0 con 0x556c033f19d0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c031f4300 con 0x556c033a80a0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 10 client.0 _ll_register_callbacks cb 0x556c03373040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c03404e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c03404e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51908/0 (socket says 172.21.15.45:51908) 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 -- 172.21.15.45:0/468605470 learned_addr learned my addr 172.21.15.45:0/468605470 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb5e000640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c034048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb66a00640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c03404380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 -- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 msgr2=0x556c03404380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c03404380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 -- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 msgr2=0x556c034048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c034048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 -- 172.21.15.45:0/468605470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c031968e0 con 0x556c033f22f0 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb67400640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c03404e00 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fdb600135a0 tx=0x7fdb60013a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18899b740253a65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 1 -- 172.21.15.45:0/468605470 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6001a070 con 0x556c033f22f0 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 1 -- 172.21.15.45:0/468605470 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fdb60015680 con 0x556c033f22f0 2024-04-25T10:33:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb580bbc70 con 0x556c033f22f0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 1 -- 172.21.15.45:0/468605470 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb60025050 con 0x556c033f22f0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 10 client.11661 fetch_fsmap learned FSMap version 146 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x556c0324e8f0 con 0x556c033f22f0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 1 -- 172.21.15.45:0/468605470 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb580bbc70 con 0x556c033f22f0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 1 -- 172.21.15.45:0/468605470 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb60028020 con 0x556c033f22f0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 10 client.11661 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 10 client.11661 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 20 client.11661 populate_metadata read hostname 'smithi045' 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c034059d0 0x556c03459f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 4 client.11661 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c03202530 con 0x556c034059d0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb66a00640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c034059d0 0x556c03459f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb66a00640 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c034059d0 0x556c03459f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef0e77ca017ec5bd server_cookie=1e496690d0b439a2 in_seq=0 out_seq=0 2024-04-25T10:33:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 10 client.11661 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 1 -- 172.21.15.45:0/468605470 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556c03202530 con 0x556c034059d0 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 10 client.11661 handle_command_reply: tid=0 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 client.11661 shutdown 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 msgr2=0x556c03404e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c03404e00 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fdb600135a0 tx=0x7fdb60013a60 comp rx=0 tx=0).stop 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 shutdown_connections 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x556c034059d0 0x556c03459f60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556c033f22f0 0x556c03404e00 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c033f19d0 0x556c034048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 --2- 172.21.15.45:0/468605470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x556c033a80a0 0x556c03404380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 >> 172.21.15.45:0/468605470 conn(0x556c033a4020 msgr2=0x556c033f00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb4be00640 0 client.11661 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 shutdown_connections 2024-04-25T10:33:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 1 -- 172.21.15.45:0/468605470 wait complete. 2024-04-25T10:33:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.557+0000 7fdb70f99000 20 client.11661 trim_cache size 0 max 16384 2024-04-25T10:33:13.567 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-25T10:33:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-25T10:33:13.568 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb44066f90 msgr2=0x7fdb44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb44066f90 0x7fdb44069450 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdb580ae000 tx=0x7fdb580b3040 comp rx=0 tx=0).stop 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 msgr2=0x7fdb681fa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 0x7fdb681fa5c0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000ebe0 tx=0x7fdb5000c520 comp rx=0 tx=0).stop 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 shutdown_connections 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb44066f90 0x7fdb44069450 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb6815bac0 0x7fdb6815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb680aef80 0x7fdb681fa5c0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 --2- 172.21.15.45:0/3781893858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb68070350 0x7fdb68151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 >> 172.21.15.45:0/3781893858 conn(0x7fdb680669a0 msgr2=0x7fdb6815d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 shutdown_connections 2024-04-25T10:33:13.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:13.561+0000 7fdb6da00640 1 -- 172.21.15.45:0/3781893858 wait complete. 2024-04-25T10:33:13.581 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:33:13.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.125+0000 7f401ea00640 1 Processor -- start 2024-04-25T10:33:14.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.125+0000 7f401ea00640 1 -- start start 2024-04-25T10:33:14.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f401ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 0x7f40181472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f401ea00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401806c600 con 0x7f4018146ed0 2024-04-25T10:33:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f401ea00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4018161bd0 con 0x7f4018071e50 2024-04-25T10:33:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f401ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4018161980 con 0x7f40180726b0 2024-04-25T10:33:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4018146ed0 0x7f4018071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:33498/0 (socket says 172.21.15.45:33498) 2024-04-25T10:33:14.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017e00640 1 -- 172.21.15.45:0/1044903672 learned_addr learned my addr 172.21.15.45:0/1044903672 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216533852 0 0) 0x7f401806c600 con 0x7f4018146ed0 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f401ca00640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 0x7f40181472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ffc003530 con 0x7f4018146ed0 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 821954567 0 0) 0x7f3ffc003530 con 0x7f4018146ed0 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 msgr2=0x7f40181472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 0x7f40181472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 >> v1:172.21.15.117:6789/0 conn(0x7f4018071e50 legacy=0x7f40180ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018161eb0 con 0x7f4018146ed0 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7f4008001680 con 0x7f4018146ed0 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 >> v1:172.21.15.45:6789/0 conn(0x7f4018146ed0 legacy=0x7f4018071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 0x7f3ffc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ffc003f30 0x7f3ffc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc004d50 0x7f3ffc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4018161980 con 0x7f3ffc003f30 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc00ba80 con 0x7f3ffc004d50 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc00bc80 con 0x7f40180726b0 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f400f400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ffc003f30 0x7f3ffc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f401ca00640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc004d50 0x7f3ffc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f4017e00640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 0x7f3ffc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f400f400640 1 -- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 msgr2=0x7f3ffc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f400f400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40180726b0 0x7f3ffc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f400f400640 1 -- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc004d50 msgr2=0x7f3ffc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f400f400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc004d50 0x7f3ffc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.129+0000 7f400f400640 1 -- 172.21.15.45:0/1044903672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc003530 con 0x7f3ffc003f30 2024-04-25T10:33:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f400f400640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ffc003f30 0x7f3ffc004810 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f40180b5eb0 tx=0x7f400001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4da2cc68bfddf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40000b5070 con 0x7f3ffc003f30 2024-04-25T10:33:14.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40000a8850 con 0x7f3ffc003f30 2024-04-25T10:33:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f4017400640 1 -- 172.21.15.45:0/1044903672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000001460 con 0x7f3ffc003f30 2024-04-25T10:33:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 -- 172.21.15.45:0/1044903672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ffc003f30 msgr2=0x7f3ffc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 --2- 172.21.15.45:0/1044903672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ffc003f30 0x7f3ffc004810 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f40180b5eb0 tx=0x7f400001ec30 comp rx=0 tx=0).stop 2024-04-25T10:33:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ca00640 1 -- 172.21.15.45:0/1044903672 reap_dead start 2024-04-25T10:33:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 -- 172.21.15.45:0/1044903672 shutdown_connections 2024-04-25T10:33:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 -- 172.21.15.45:0/1044903672 >> 172.21.15.45:0/1044903672 conn(0x7f40180669a0 msgr2=0x7f4018066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 -- 172.21.15.45:0/1044903672 shutdown_connections 2024-04-25T10:33:14.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 -- 172.21.15.45:0/1044903672 wait complete. 2024-04-25T10:33:14.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 Processor -- start 2024-04-25T10:33:14.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.133+0000 7f401ea00640 1 -- start start 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc003f30 0x7f4018141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40181f7af0 0x7f4018160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 0x7f40180c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4018147a70 con 0x7f40180c7680 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4018162470 con 0x7f3ffc003f30 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4018147bf0 con 0x7f40181f7af0 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 0x7f40180c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 0x7f40180c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51930/0 (socket says 172.21.15.45:51930) 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 -- 172.21.15.45:0/4174462162 learned_addr learned my addr 172.21.15.45:0/4174462162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 -- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40181f7af0 msgr2=0x7f4018160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40181f7af0 0x7f4018160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 -- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc003f30 msgr2=0x7f4018141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f4017e00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc003f30 0x7f4018141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc003f30 0x7f4018141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 -- 172.21.15.45:0/4174462162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018160d60 con 0x7f40180c7680 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f4017e00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc003f30 0x7f4018141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ca00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 0x7f40180c7b30 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f40181618c0 tx=0x7f401000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000fde0 con 0x7f40180c7680 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4010015430 con 0x7f40180c7680 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010015820 con 0x7f40180c7680 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401815f780 con 0x7f40180c7680 2024-04-25T10:33:14.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401815fd40 con 0x7f40180c7680 2024-04-25T10:33:14.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401804ef30 con 0x7f40180c7680 2024-04-25T10:33:14.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.137+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40100159c0 con 0x7f40180c7680 2024-04-25T10:33:14.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.141+0000 7f4014c00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ff4066f90 0x7f3ff4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.141+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f4010085c00 con 0x7f40180c7680 2024-04-25T10:33:14.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.141+0000 7f4017e00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ff4066f90 0x7f3ff4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.141+0000 7f4017e00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ff4066f90 0x7f3ff4069450 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4008000c10 tx=0x7f4008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.145+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4010024030 con 0x7f40180c7680 2024-04-25T10:33:14.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4018072490 con 0x7f40180c7680 2024-04-25T10:33:14.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f4014c00640 1 -- 172.21.15.45:0/4174462162 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f4010019030 con 0x7f40180c7680 2024-04-25T10:33:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:14.344 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:09.710400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:33:14.344 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ff4066f90 msgr2=0x7f3ff4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ff4066f90 0x7f3ff4069450 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4008000c10 tx=0x7f4008009040 comp rx=0 tx=0).stop 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 msgr2=0x7f40180c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 0x7f40180c7b30 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f40181618c0 tx=0x7f401000f580 comp rx=0 tx=0).stop 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 shutdown_connections 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3ff4066f90 0x7f3ff4069450 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40180c7680 0x7f40180c7b30 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f40181f7af0 0x7f4018160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 --2- 172.21.15.45:0/4174462162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ffc003f30 0x7f4018141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 >> 172.21.15.45:0/4174462162 conn(0x7f40180669a0 msgr2=0x7f4018070650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 shutdown_connections 2024-04-25T10:33:14.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.341+0000 7f401ea00640 1 -- 172.21.15.45:0/4174462162 wait complete. 2024-04-25T10:33:14.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 146, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:31:55.417950+0000', 'modified': '2024-04-25T10:32:09.710400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 194, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11496': {'gid': 11496, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/1179754162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1179754162}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1179754162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:33:14.366 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:33:14.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:33:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 Processor -- start 2024-04-25T10:33:14.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 -- start start 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fca301550b0 0x7fca300cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca3015b910 0x7fca300b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca30071050 con 0x7fca300cfe70 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3015bd80 con 0x7fca3015b910 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca37400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3015d040 con 0x7fca301550b0 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca3015b910 0x7fca300b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fca3015b910 0x7fca300b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51940/0 (socket says 172.21.15.45:51940) 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 -- 172.21.15.45:0/865800042 learned_addr learned my addr 172.21.15.45:0/865800042 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca34a00640 1 --2- 172.21.15.45:0/865800042 >> v2:172.21.15.117:3300/0 conn(0x7fca301550b0 0x7fca300cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 -- 172.21.15.45:0/865800042 >> v2:172.21.15.117:3300/0 conn(0x7fca301550b0 msgr2=0x7fca300cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 --2- 172.21.15.45:0/865800042 >> v2:172.21.15.117:3300/0 conn(0x7fca301550b0 0x7fca300cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 -- 172.21.15.45:0/865800042 >> v1:172.21.15.117:6789/0 conn(0x7fca300cfe70 legacy=0x7fca3015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.505+0000 7fca35400640 1 -- 172.21.15.45:0/865800042 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300712f0 con 0x7fca3015b910 2024-04-25T10:33:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca35400640 1 --2- 172.21.15.45:0/865800042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300b3260 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fca28009e50 tx=0x7fca2801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7ba07e74e2bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca2f400640 1 -- 172.21.15.45:0/865800042 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280a57a0 con 0x7fca3015b910 2024-04-25T10:33:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca2f400640 1 -- 172.21.15.45:0/865800042 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280a5940 con 0x7fca3015b910 2024-04-25T10:33:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca2f400640 1 -- 172.21.15.45:0/865800042 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280a5d10 con 0x7fca3015b910 2024-04-25T10:33:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 -- 172.21.15.45:0/865800042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 msgr2=0x7fca300b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 --2- 172.21.15.45:0/865800042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300b3260 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fca28009e50 tx=0x7fca2801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 -- 172.21.15.45:0/865800042 shutdown_connections 2024-04-25T10:33:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 --2- 172.21.15.45:0/865800042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300b3260 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 --2- 172.21.15.45:0/865800042 >> v2:172.21.15.117:3300/0 conn(0x7fca301550b0 0x7fca300cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 -- 172.21.15.45:0/865800042 >> 172.21.15.45:0/865800042 conn(0x7fca30066b70 msgr2=0x7fca30066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 -- 172.21.15.45:0/865800042 shutdown_connections 2024-04-25T10:33:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.509+0000 7fca37400640 1 -- 172.21.15.45:0/865800042 wait complete. 2024-04-25T10:33:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 Processor -- start 2024-04-25T10:33:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 -- start start 2024-04-25T10:33:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca300cfe70 0x7fca30149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca301550b0 0x7fca30151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015c030 con 0x7fca3015b910 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300720b0 con 0x7fca301550b0 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015c1b0 con 0x7fca300cfe70 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51954/0 (socket says 172.21.15.45:51954) 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 -- 172.21.15.45:0/864675799 learned_addr learned my addr 172.21.15.45:0/864675799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca34a00640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca300cfe70 0x7fca30149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca2fe00640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca301550b0 0x7fca30151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 -- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca300cfe70 msgr2=0x7fca30149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca300cfe70 0x7fca30149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 -- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca301550b0 msgr2=0x7fca30151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca301550b0 0x7fca30151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 -- 172.21.15.45:0/864675799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30159410 con 0x7fca3015b910 2024-04-25T10:33:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca35400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300ad840 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fca28019600 tx=0x7fca28019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca2cc00640 1 -- 172.21.15.45:0/864675799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280b8670 con 0x7fca3015b910 2024-04-25T10:33:14.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca2cc00640 1 -- 172.21.15.45:0/864675799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280be070 con 0x7fca3015b910 2024-04-25T10:33:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca301596f0 con 0x7fca3015b910 2024-04-25T10:33:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca2cc00640 1 -- 172.21.15.45:0/864675799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280b8a70 con 0x7fca3015b910 2024-04-25T10:33:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.513+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca30159cb0 con 0x7fca3015b910 2024-04-25T10:33:14.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca2cc00640 1 -- 172.21.15.45:0/864675799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca280b8c10 con 0x7fca3015b910 2024-04-25T10:33:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca2cc00640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca2cc00640 1 -- 172.21.15.45:0/864675799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fca281200d0 con 0x7fca3015b910 2024-04-25T10:33:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca34a00640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca3a7ef000 1 Processor -- start 2024-04-25T10:33:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca3a7ef000 1 -- start start 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.517+0000 7fca34a00640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fca24009b60 tx=0x7fca24009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f3b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f36c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047f2535f0 con 0x56047f3b6240 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047f1bada0 con 0x56047f3b5920 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56047f143150 con 0x56047f36bf30 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51958/0 (socket says 172.21.15.45:51958) 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 -- 172.21.15.45:0/644633875 learned_addr learned my addr 172.21.15.45:0/644633875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca35400640 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f36c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca34a00640 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f3b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 -- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 msgr2=0x56047f36c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f36c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 -- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 msgr2=0x56047f3b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f3b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 -- 172.21.15.45:0/644633875 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047f15cca0 con 0x56047f3b6240 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca34a00640 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f3b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca2fe00640 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36b9f0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fca1c001850 tx=0x7fca1c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116301cbb089facf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca21600640 1 -- 172.21.15.45:0/644633875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c01a070 con 0x56047f3b6240 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca21600640 1 -- 172.21.15.45:0/644633875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c005320 con 0x56047f3b6240 2024-04-25T10:33:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca21600640 1 -- 172.21.15.45:0/644633875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0069e0 con 0x56047f3b6240 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- 172.21.15.45:0/644633875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 msgr2=0x56047f36b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36b9f0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fca1c001850 tx=0x7fca1c000bd0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- 172.21.15.45:0/644633875 shutdown_connections 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f36c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36b9f0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/644633875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f3b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- 172.21.15.45:0/644633875 >> 172.21.15.45:0/644633875 conn(0x56047f3722a0 msgr2=0x56047f36a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- 172.21.15.45:0/644633875 shutdown_connections 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- 172.21.15.45:0/644633875 wait complete. 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- start start 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 Processor -- start 2024-04-25T10:33:14.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f36bf30 0x56047f3bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b5920 0x56047f3bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047f175990 con 0x56047f3b6240 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047f1b6e40 con 0x56047f36bf30 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.521+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56047f146900 con 0x56047f3b5920 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 10 client.0 _ll_register_callbacks cb 0x56047f26e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51972/0 (socket says 172.21.15.45:51972) 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 -- 172.21.15.45:0/560825142 learned_addr learned my addr 172.21.15.45:0/560825142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca2fe00640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b5920 0x56047f3bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca34a00640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f36bf30 0x56047f3bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 -- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b5920 msgr2=0x56047f3bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b5920 0x56047f3bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 -- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f36bf30 msgr2=0x56047f3bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f36bf30 0x56047f3bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 -- 172.21.15.45:0/560825142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56047f173630 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca35400640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36faa0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fca280f8b50 tx=0x7fca280f05e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d44e7cbd88a1d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 1 -- 172.21.15.45:0/560825142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280f0e80 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 1 -- 172.21.15.45:0/560825142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fca280f8bf0 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca1c005320 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 1 -- 172.21.15.45:0/560825142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca280f4ac0 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 10 client.11679 fetch_fsmap learned FSMap version 146 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x56047f1433a0 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 1 -- 172.21.15.45:0/560825142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c005320 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 1 -- 172.21.15.45:0/560825142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca280efe70 con 0x56047f3b6240 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 10 client.11679 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:33:14.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 10 client.11679 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 20 client.11679 populate_metadata read hostname 'smithi045' 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f371160 0x56047f3c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 4 client.11679 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56047f156a50 con 0x56047f371160 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca34a00640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f371160 0x56047f3c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca34a00640 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f371160 0x56047f3c6ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357ca537be94245a server_cookie=f1945a5dbf24031c in_seq=0 out_seq=0 2024-04-25T10:33:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 10 client.11679 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 1 -- 172.21.15.45:0/560825142 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56047f156a50 con 0x56047f371160 2024-04-25T10:33:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 10 client.11679 handle_command_reply: tid=0 2024-04-25T10:33:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 client.11679 shutdown 2024-04-25T10:33:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 msgr2=0x56047f36faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36faa0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fca280f8b50 tx=0x7fca280f05e0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 shutdown_connections 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f371160 0x56047f3c6ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b6240 0x56047f36faa0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b5920 0x56047f3bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/560825142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f36bf30 0x56047f3bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 >> 172.21.15.45:0/560825142 conn(0x56047f3722a0 msgr2=0x56047f36a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca16000640 0 client.11679 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 shutdown_connections 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 1 -- 172.21.15.45:0/560825142 wait complete. 2024-04-25T10:33:14.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.525+0000 7fca3a7ef000 20 client.11679 trim_cache size 0 max 16384 2024-04-25T10:33:14.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.541+0000 7fca3a7ef000 1 Processor -- start 2024-04-25T10:33:14.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.541+0000 7fca3a7ef000 1 -- start start 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 0x56047f410d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f4112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b6240 0x56047f4117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca30202410 con 0x56047f36bf30 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca30157120 con 0x56047f3b5920 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca30157720 con 0x56047f3b6240 2024-04-25T10:33:14.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 0x56047f410d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 0x56047f410d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51982/0 (socket says 172.21.15.45:51982) 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 -- 172.21.15.45:0/1202444908 learned_addr learned my addr 172.21.15.45:0/1202444908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca35400640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b6240 0x56047f4117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca2fe00640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f4112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 -- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b6240 msgr2=0x56047f4117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b6240 0x56047f4117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 -- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 msgr2=0x56047f4112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f4112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 -- 172.21.15.45:0/1202444908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047f23c600 con 0x56047f36bf30 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca35400640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b6240 0x56047f4117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca2fe00640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f4112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca34a00640 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 0x56047f410d70 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x56047f15b2f0 tx=0x7fca24014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951fbfeb5a656115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca20c00640 1 -- 172.21.15.45:0/1202444908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2401e040 con 0x56047f36bf30 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca20c00640 1 -- 172.21.15.45:0/1202444908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2401f040 con 0x56047f36bf30 2024-04-25T10:33:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca20c00640 1 -- 172.21.15.45:0/1202444908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca24003a40 con 0x56047f36bf30 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1202444908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 msgr2=0x56047f410d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 0x56047f410d70 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x56047f15b2f0 tx=0x7fca24014000 comp rx=0 tx=0).stop 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1202444908 shutdown_connections 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f3b6240 0x56047f4117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b5920 0x56047f4112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1202444908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f36bf30 0x56047f410d70 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1202444908 >> 172.21.15.45:0/1202444908 conn(0x56047f3722a0 msgr2=0x56047f36f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1202444908 shutdown_connections 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1202444908 wait complete. 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 -- start start 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.545+0000 7fca3a7ef000 1 Processor -- start 2024-04-25T10:33:14.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f3c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 0x56047f3c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b6240 0x56047f3fa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047f1bf3d0 con 0x56047f3b5920 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047f276d50 con 0x56047f3b6240 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x56047f155980 con 0x56047f36bf30 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 10 client.0 _ll_register_callbacks cb 0x7fca1c001270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 0x56047f3c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 0x56047f3c2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51992/0 (socket says 172.21.15.45:51992) 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 -- 172.21.15.45:0/1234940252 learned_addr learned my addr 172.21.15.45:0/1234940252 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca34a00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f3c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca35400640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b6240 0x56047f3fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 -- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 msgr2=0x56047f3c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f3c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 -- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b6240 msgr2=0x56047f3fa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b6240 0x56047f3fa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 -- 172.21.15.45:0/1234940252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56047f14f9b0 con 0x56047f3b5920 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca34a00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f3c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca2fe00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 0x56047f3c2ac0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009010 tx=0x7fca1c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b179f398b83315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 1 -- 172.21.15.45:0/1234940252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c025030 con 0x56047f3b5920 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 1 -- 172.21.15.45:0/1234940252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fca1c00f7b0 con 0x56047f3b5920 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca2401f040 con 0x56047f3b5920 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 1 -- 172.21.15.45:0/1234940252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca1c031050 con 0x56047f3b5920 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 10 client.11685 fetch_fsmap learned FSMap version 146 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x56047f1620b0 con 0x56047f3b5920 2024-04-25T10:33:14.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 1 -- 172.21.15.45:0/1234940252 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2401f040 con 0x56047f3b5920 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 1 -- 172.21.15.45:0/1234940252 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca1c034020 con 0x56047f3b5920 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 10 client.11685 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 10 client.11685 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 20 client.11685 populate_metadata read hostname 'smithi045' 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f3fa770 0x56047f3fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 4 client.11685 mds_command: new command op to 11496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 --> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56047f1c82e0 con 0x56047f3fa770 2024-04-25T10:33:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca34a00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f3fa770 0x56047f3fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca34a00640 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f3fa770 0x56047f3fab50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=165124e9f95b946 server_cookie=d725e46f14dc4243 in_seq=0 out_seq=0 2024-04-25T10:33:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 10 client.11685 ms_handle_connect on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 1 -- 172.21.15.45:0/1234940252 <== mds.0 v2:172.21.15.45:6833/1179754162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x56047f1c82e0 con 0x56047f3fa770 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 10 client.11685 handle_command_reply: tid=0 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 client.11685 shutdown 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 msgr2=0x56047f3c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 0x56047f3c2ac0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009010 tx=0x7fca1c007f90 comp rx=0 tx=0).stop 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 shutdown_connections 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:6833/1179754162,v1:172.21.15.45:6835/1179754162] conn(0x56047f3fa770 0x56047f3fab50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56047f3b6240 0x56047f3fa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56047f3b5920 0x56047f3c2ac0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca16000640 0 client.11685 ms_handle_reset on v2:172.21.15.45:6833/1179754162 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 --2- 172.21.15.45:0/1234940252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x56047f36bf30 0x56047f3c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 >> 172.21.15.45:0/1234940252 conn(0x56047f3722a0 msgr2=0x56047f3bd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 shutdown_connections 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 1 -- 172.21.15.45:0/1234940252 wait complete. 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.549+0000 7fca3a7ef000 20 client.11685 trim_cache size 0 max 16384 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "d7e12072-0435-4dda-862b-d1045153027e", 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 11496, 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T10:33:14.559 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 66.853986247999998, 2024-04-25T10:33:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 146, 2024-04-25T10:33:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 196, 2024-04-25T10:33:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 194, 2024-04-25T10:33:14.560 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 67.787508798999994 2024-04-25T10:33:14.560 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca10066f60 msgr2=0x7fca10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fca24009b60 tx=0x7fca24009190 comp rx=0 tx=0).stop 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 msgr2=0x7fca300ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300ad840 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fca28019600 tx=0x7fca28019790 comp rx=0 tx=0).stop 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 shutdown_connections 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca3015b910 0x7fca300ad840 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca301550b0 0x7fca30151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 --2- 172.21.15.45:0/864675799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca300cfe70 0x7fca30149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 >> 172.21.15.45:0/864675799 conn(0x7fca30066b70 msgr2=0x7fca3015d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 shutdown_connections 2024-04-25T10:33:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:14.553+0000 7fca37400640 1 -- 172.21.15.45:0/864675799 wait complete. 2024-04-25T10:33:14.574 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:33:14.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:33:14.685 DEBUG:teuthology.orchestra.run.smithi117:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:33:14.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:33:14.773 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:33:14.773 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:14.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:33:14.782 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:33:14.782 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:14.791 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:33:14.791 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:33:14.791 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:14.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:33:14.831 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:33:14.831 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:14.842 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:33:14.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:15.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 Processor -- start 2024-04-25T10:33:15.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 -- start start 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f67d00cda10 0x7f67d006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f67d00cf930 0x7f67d00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00b2770 con 0x7f67d015bc40 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d006c670 con 0x7f67d00cda10 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d6e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d006c950 con 0x7f67d00cf930 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67c7400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f67d00cda10 0x7f67d006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67c7400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f67d00cda10 0x7f67d006c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58204/0 (socket says 172.21.15.45:58204) 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67c7400640 1 -- 172.21.15.45:0/1293583135 learned_addr learned my addr 172.21.15.45:0/1293583135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.013+0000 7f67d4e00640 1 --2- 172.21.15.45:0/1293583135 >> v2:172.21.15.117:3300/0 conn(0x7f67d00cf930 0x7f67d00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67c7400640 1 -- 172.21.15.45:0/1293583135 >> v2:172.21.15.117:3300/0 conn(0x7f67d00cf930 msgr2=0x7f67d00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67c7400640 1 --2- 172.21.15.45:0/1293583135 >> v2:172.21.15.117:3300/0 conn(0x7f67d00cf930 0x7f67d00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67c7400640 1 -- 172.21.15.45:0/1293583135 >> v1:172.21.15.117:6789/0 conn(0x7f67d015bc40 legacy=0x7f67d00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67c7400640 1 -- 172.21.15.45:0/1293583135 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d006cc30 con 0x7f67d00cda10 2024-04-25T10:33:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67c7400640 1 --2- 172.21.15.45:0/1293583135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cda10 0x7f67d006c090 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f67d00ae4c0 tx=0x7f67b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3883a2e8b26b0f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67cf400640 1 -- 172.21.15.45:0/1293583135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80b0070 con 0x7f67d00cda10 2024-04-25T10:33:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67cf400640 1 -- 172.21.15.45:0/1293583135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80ab600 con 0x7f67d00cda10 2024-04-25T10:33:15.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67cf400640 1 -- 172.21.15.45:0/1293583135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80ab9d0 con 0x7f67d00cda10 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67d6e00640 1 -- 172.21.15.45:0/1293583135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cda10 msgr2=0x7f67d006c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.017+0000 7f67d6e00640 1 --2- 172.21.15.45:0/1293583135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cda10 0x7f67d006c090 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f67d00ae4c0 tx=0x7f67b801c490 comp rx=0 tx=0).stop 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- 172.21.15.45:0/1293583135 shutdown_connections 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 --2- 172.21.15.45:0/1293583135 >> v2:172.21.15.117:3300/0 conn(0x7f67d00cf930 0x7f67d00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 --2- 172.21.15.45:0/1293583135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cda10 0x7f67d006c090 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- 172.21.15.45:0/1293583135 >> 172.21.15.45:0/1293583135 conn(0x7f67d0066b70 msgr2=0x7f67d0066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- 172.21.15.45:0/1293583135 shutdown_connections 2024-04-25T10:33:15.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- 172.21.15.45:0/1293583135 wait complete. 2024-04-25T10:33:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 Processor -- start 2024-04-25T10:33:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- start start 2024-04-25T10:33:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67d00cda10 0x7f67d015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 0x7f67d015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d015bc40 0x7f67d0151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00b3850 con 0x7f67d00cf930 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00b36d0 con 0x7f67d015bc40 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d6e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00b39d0 con 0x7f67d00cda10 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 0x7f67d015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 0x7f67d015ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58210/0 (socket says 172.21.15.45:58210) 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 -- 172.21.15.45:0/372924139 learned_addr learned my addr 172.21.15.45:0/372924139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67cfe00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67d00cda10 0x7f67d015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67d4e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d015bc40 0x7f67d0151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 -- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67d00cda10 msgr2=0x7f67d015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67d00cda10 0x7f67d015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 -- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d015bc40 msgr2=0x7f67d0151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d015bc40 0x7f67d0151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67c7400640 1 -- 172.21.15.45:0/372924139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0151d90 con 0x7f67d00cf930 2024-04-25T10:33:15.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.021+0000 7f67cfe00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67d00cda10 0x7f67d015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:15.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67c7400640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 0x7f67d015ae50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f67b80190a0 tx=0x7f67b8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80b0040 con 0x7f67d00cf930 2024-04-25T10:33:15.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80b9070 con 0x7f67d00cf930 2024-04-25T10:33:15.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d015e880 con 0x7f67d00cf930 2024-04-25T10:33:15.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80b4880 con 0x7f67d00cf930 2024-04-25T10:33:15.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d015ee40 con 0x7f67d00cf930 2024-04-25T10:33:15.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d004ef30 con 0x7f67d00cf930 2024-04-25T10:33:15.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67b80b4a20 con 0x7f67d00cf930 2024-04-25T10:33:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67ccc00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67ac066f60 0x7f67ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.025+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f67b811ee90 con 0x7f67d00cf930 2024-04-25T10:33:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.029+0000 7f67cfe00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67ac066f60 0x7f67ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.029+0000 7f67cfe00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67ac066f60 0x7f67ac069420 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f67c0001670 tx=0x7f67c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.033+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67b8016110 con 0x7f67d00cf930 2024-04-25T10:33:15.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.229+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67d006d960 con 0x7f67d00cf930 2024-04-25T10:33:15.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.229+0000 7f67ccc00640 1 -- 172.21.15.45:0/372924139 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f67b80efe40 con 0x7f67d00cf930 2024-04-25T10:33:15.235 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:15.236 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:09.710400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:33:15.236 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2024-04-25T10:33:15.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67ac066f60 msgr2=0x7f67ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67ac066f60 0x7f67ac069420 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f67c0001670 tx=0x7f67c0009040 comp rx=0 tx=0).stop 2024-04-25T10:33:15.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 msgr2=0x7f67d015ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 0x7f67d015ae50 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f67b80190a0 tx=0x7f67b8019230 comp rx=0 tx=0).stop 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 shutdown_connections 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f67ac066f60 0x7f67ac069420 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d015bc40 0x7f67d0151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d00cf930 0x7f67d015ae50 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 --2- 172.21.15.45:0/372924139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f67d00cda10 0x7f67d015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 >> 172.21.15.45:0/372924139 conn(0x7f67d0066b70 msgr2=0x7f67d00b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:15.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 shutdown_connections 2024-04-25T10:33:15.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.233+0000 7f67d6e00640 1 -- 172.21.15.45:0/372924139 wait complete. 2024-04-25T10:33:15.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:33:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.397+0000 7f3f4e600640 1 Processor -- start 2024-04-25T10:33:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.397+0000 7f3f4e600640 1 -- start start 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f4e600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f481550b0 0x7f3f480cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f4e600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f480cfe70 0x7f3f4815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f4e600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f48071050 con 0x7f3f4815b740 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f4e600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4815d040 con 0x7f3f480cfe70 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f4e600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480712f0 con 0x7f3f481550b0 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f47e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f4815b740 0x7f3f4806d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52948/0 (socket says 172.21.15.45:52948) 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f47e00640 1 -- 172.21.15.45:0/3282473280 learned_addr learned my addr 172.21.15.45:0/3282473280 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f47400640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f481550b0 0x7f3f480cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f3ea00640 1 --2- 172.21.15.45:0/3282473280 >> v2:172.21.15.117:3300/0 conn(0x7f3f480cfe70 0x7f3f4815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f46a00640 1 -- 172.21.15.45:0/3282473280 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655789575 0 0) 0x7f3f48071050 con 0x7f3f4815b740 2024-04-25T10:33:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f46a00640 1 -- 172.21.15.45:0/3282473280 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c003530 con 0x7f3f4815b740 2024-04-25T10:33:15.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f3ea00640 1 -- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f481550b0 msgr2=0x7f3f480cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:15.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f3ea00640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f481550b0 0x7f3f480cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f3ea00640 1 -- 172.21.15.45:0/3282473280 >> v1:172.21.15.45:6789/0 conn(0x7f3f4815b740 legacy=0x7f3f4806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f3ea00640 1 -- 172.21.15.45:0/3282473280 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480715d0 con 0x7f3f480cfe70 2024-04-25T10:33:15.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.401+0000 7f3f47400640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f481550b0 0x7f3f480cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f3ea00640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f480cfe70 0x7f3f4815b200 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097c0 tx=0x7f3f3001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a714acf1607c093c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f46a00640 1 -- 172.21.15.45:0/3282473280 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300b0070 con 0x7f3f480cfe70 2024-04-25T10:33:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f46a00640 1 -- 172.21.15.45:0/3282473280 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f300ab600 con 0x7f3f480cfe70 2024-04-25T10:33:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f46a00640 1 -- 172.21.15.45:0/3282473280 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300ab9b0 con 0x7f3f480cfe70 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f480cfe70 msgr2=0x7f3f4815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f480cfe70 0x7f3f4815b200 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097c0 tx=0x7f3f3001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- 172.21.15.45:0/3282473280 shutdown_connections 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f480cfe70 0x7f3f4815b200 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 --2- 172.21.15.45:0/3282473280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f481550b0 0x7f3f480cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- 172.21.15.45:0/3282473280 >> 172.21.15.45:0/3282473280 conn(0x7f3f48066b70 msgr2=0x7f3f48066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:15.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- 172.21.15.45:0/3282473280 shutdown_connections 2024-04-25T10:33:15.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- 172.21.15.45:0/3282473280 wait complete. 2024-04-25T10:33:15.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 Processor -- start 2024-04-25T10:33:15.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- start start 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f480cfe70 0x7f3f48135680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f481550b0 0x7f3f48134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 0x7f3f48155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4815bcd0 con 0x7f3f4815b740 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4815bb50 con 0x7f3f481550b0 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.405+0000 7f3f4e600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4815c030 con 0x7f3f480cfe70 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 0x7f3f48155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 0x7f3f48155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58218/0 (socket says 172.21.15.45:58218) 2024-04-25T10:33:15.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 -- 172.21.15.45:0/137807369 learned_addr learned my addr 172.21.15.45:0/137807369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:15.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f3ea00640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f481550b0 0x7f3f48134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47400640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f480cfe70 0x7f3f48135680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 -- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f480cfe70 msgr2=0x7f3f48135680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f480cfe70 0x7f3f48135680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 -- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f481550b0 msgr2=0x7f3f48134ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f481550b0 0x7f3f48134ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 -- 172.21.15.45:0/137807369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48135bc0 con 0x7f3f4815b740 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f47e00640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 0x7f3f48155490 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ca20 tx=0x7f3f4000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40017070 con 0x7f3f4815b740 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4000abd0 con 0x7f3f4815b740 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40012400 con 0x7f3f4815b740 2024-04-25T10:33:15.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48156000 con 0x7f3f4815b740 2024-04-25T10:33:15.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f480ccd20 con 0x7f3f4815b740 2024-04-25T10:33:15.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4804ef30 con 0x7f3f4815b740 2024-04-25T10:33:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.409+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f4001a070 con 0x7f3f4815b740 2024-04-25T10:33:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.413+0000 7f3f3fe00640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f24066f60 0x7f3f24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.413+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f3f40084d90 con 0x7f3f4815b740 2024-04-25T10:33:15.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.413+0000 7f3f47400640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f24066f60 0x7f3f24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.413+0000 7f3f47400640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f24066f60 0x7f3f24069420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009580 tx=0x7f3f38012040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.417+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f40055650 con 0x7f3f4815b740 2024-04-25T10:33:15.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.581+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f48072350 con 0x7f3f4815b740 2024-04-25T10:33:15.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f3fe00640 1 -- 172.21.15.45:0/137807369 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17041 (secure 0 0 0) 0x7f3f400596f0 con 0x7f3f4815b740 2024-04-25T10:33:15.588 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:15.589 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":196,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:32:46.808432+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":1,"to":6}]},{"pgid":"29.22","mappings":[{"from":1,"to":4}]},{"pgid":"29.29","mappings":[{"from":1,"to":4}]},{"pgid":"29.2e","mappings":[{"from":1,"to":4}]},{"pgid":"29.3e","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/3198476999":"2024-04-26T10:32:06.688741+0000","172.21.15.45:6833/3198476999":"2024-04-26T10:32:06.688741+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f24066f60 msgr2=0x7f3f24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f24066f60 0x7f3f24069420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009580 tx=0x7f3f38012040 comp rx=0 tx=0).stop 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 msgr2=0x7f3f48155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 0x7f3f48155490 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ca20 tx=0x7f3f4000cee0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 shutdown_connections 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3f24066f60 0x7f3f24069420 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f4815b740 0x7f3f48155490 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f481550b0 0x7f3f48134ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 --2- 172.21.15.45:0/137807369 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f480cfe70 0x7f3f48135680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 >> 172.21.15.45:0/137807369 conn(0x7f3f48066b70 msgr2=0x7f3f480b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 shutdown_connections 2024-04-25T10:33:15.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.585+0000 7f3f4e600640 1 -- 172.21.15.45:0/137807369 wait complete. 2024-04-25T10:33:15.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:17:48.947609+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:31:53.058645+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:31:54.409004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:33:15.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:33:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 Processor -- start 2024-04-25T10:33:15.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 -- start start 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc15df20 0x7fdffc0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdffc0b1970 0x7fdffc0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdffc0b1e10 con 0x7fdffc0aeb50 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdffc0b20f0 con 0x7fdffc0b1970 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe003a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0b23d0 con 0x7fdffc15df20 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdffc0b1970 0x7fdffc0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdffc0b1970 0x7fdffc0ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58232/0 (socket says 172.21.15.45:58232) 2024-04-25T10:33:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 -- 172.21.15.45:0/1812917338 learned_addr learned my addr 172.21.15.45:0/1812917338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fe001000640 1 --2- 172.21.15.45:0/1812917338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc15df20 0x7fdffc0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 -- 172.21.15.45:0/1812917338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc15df20 msgr2=0x7fdffc0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 --2- 172.21.15.45:0/1812917338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc15df20 0x7fdffc0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 -- 172.21.15.45:0/1812917338 >> v1:172.21.15.117:6789/0 conn(0x7fdffc0aeb50 legacy=0x7fdffc0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.749+0000 7fdff3e00640 1 -- 172.21.15.45:0/1812917338 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc0b26b0 con 0x7fdffc0b1970 2024-04-25T10:33:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fdff3e00640 1 --2- 172.21.15.45:0/1812917338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc0ae610 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009870 tx=0x7fdfe401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e703a06fa2424f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fdffbe00640 1 -- 172.21.15.45:0/1812917338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40b5070 con 0x7fdffc0b1970 2024-04-25T10:33:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fdffbe00640 1 -- 172.21.15.45:0/1812917338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe40a8a50 con 0x7fdffc0b1970 2024-04-25T10:33:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fdffbe00640 1 -- 172.21.15.45:0/1812917338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40a8e20 con 0x7fdffc0b1970 2024-04-25T10:33:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 -- 172.21.15.45:0/1812917338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 msgr2=0x7fdffc0ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 --2- 172.21.15.45:0/1812917338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc0ae610 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009870 tx=0x7fdfe401ed50 comp rx=0 tx=0).stop 2024-04-25T10:33:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 -- 172.21.15.45:0/1812917338 shutdown_connections 2024-04-25T10:33:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 --2- 172.21.15.45:0/1812917338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc0ae610 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 --2- 172.21.15.45:0/1812917338 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc15df20 0x7fdffc0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 -- 172.21.15.45:0/1812917338 >> 172.21.15.45:0/1812917338 conn(0x7fdffc0d1960 msgr2=0x7fdffc0d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 -- 172.21.15.45:0/1812917338 shutdown_connections 2024-04-25T10:33:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.753+0000 7fe003a00640 1 -- 172.21.15.45:0/1812917338 wait complete. 2024-04-25T10:33:15.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 Processor -- start 2024-04-25T10:33:15.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 -- start start 2024-04-25T10:33:15.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc0aeb50 0x7fdffc15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc15df20 0x7fdffc160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc06beb0 con 0x7fdffc0b1970 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc15e300 con 0x7fdffc15df20 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc06c030 con 0x7fdffc0aeb50 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc14efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58236/0 (socket says 172.21.15.45:58236) 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 -- 172.21.15.45:0/3825186597 learned_addr learned my addr 172.21.15.45:0/3825186597 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe001000640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc0aeb50 0x7fdffc15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe001a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc15df20 0x7fdffc160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 -- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc0aeb50 msgr2=0x7fdffc15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc0aeb50 0x7fdffc15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 -- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc15df20 msgr2=0x7fdffc160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc15df20 0x7fdffc160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 -- 172.21.15.45:0/3825186597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc14f520 con 0x7fdffc0b1970 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe001a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc15df20 0x7fdffc160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff3e00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc14efe0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401d400 tx=0x7fdfe401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40c0070 con 0x7fdffc0b1970 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe40ae550 con 0x7fdffc0b1970 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40b5070 con 0x7fdffc0b1970 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc0b5580 con 0x7fdffc0b1970 2024-04-25T10:33:15.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.757+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc15fd20 con 0x7fdffc0b1970 2024-04-25T10:33:15.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.761+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdffc04ef30 con 0x7fdffc0b1970 2024-04-25T10:33:15.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.761+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfe40b9070 con 0x7fdffc0b1970 2024-04-25T10:33:15.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.761+0000 7fdff9600640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfd8066f60 0x7fdfd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:15.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.761+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fdfe4122830 con 0x7fdffc0b1970 2024-04-25T10:33:15.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.761+0000 7fe001000640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfd8066f60 0x7fdfd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:15.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.761+0000 7fe001000640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfd8066f60 0x7fdfd8069420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006aa0 tx=0x7fdfec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:15.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.765+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfe40188b0 con 0x7fdffc0b1970 2024-04-25T10:33:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdffc06eb30 con 0x7fdffc0b1970 2024-04-25T10:33:15.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fdff9600640 1 -- 172.21.15.45:0/3825186597 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v196) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdfe40f3060 con 0x7fdffc0b1970 2024-04-25T10:33:15.932 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T10:33:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfd8066f60 msgr2=0x7fdfd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfd8066f60 0x7fdfd8069420 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006aa0 tx=0x7fdfec008040 comp rx=0 tx=0).stop 2024-04-25T10:33:15.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 msgr2=0x7fdffc14efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc14efe0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401d400 tx=0x7fdfe401baf0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 shutdown_connections 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdfd8066f60 0x7fdfd8069420 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc15df20 0x7fdffc160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdffc0b1970 0x7fdffc14efe0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 --2- 172.21.15.45:0/3825186597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdffc0aeb50 0x7fdffc15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 >> 172.21.15.45:0/3825186597 conn(0x7fdffc0d1960 msgr2=0x7fdffc0d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 shutdown_connections 2024-04-25T10:33:15.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:15.929+0000 7fe003a00640 1 -- 172.21.15.45:0/3825186597 wait complete. 2024-04-25T10:33:15.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:33:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 Processor -- start 2024-04-25T10:33:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 -- start start 2024-04-25T10:33:16.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7febac0aef80 0x7febac15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febac0b31c0 0x7febac0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac15b080 con 0x7febac070350 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac15adb0 con 0x7febac0b31c0 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febb2800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac15eb60 con 0x7febac0aef80 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febabe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febac0b31c0 0x7febac0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febabe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7febac0b31c0 0x7febac0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58240/0 (socket says 172.21.15.45:58240) 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7febabe00640 1 -- 172.21.15.45:0/3200176025 learned_addr learned my addr 172.21.15.45:0/3200176025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.097+0000 7feba2a00640 1 --1- 172.21.15.45:0/3200176025 >> v1:172.21.15.45:6789/0 conn(0x7febac070350 0x7febac15b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52958/0 (socket says 172.21.15.45:52958) 2024-04-25T10:33:16.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febab400640 1 --2- 172.21.15.45:0/3200176025 >> v2:172.21.15.117:3300/0 conn(0x7febac0aef80 0x7febac15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febabe00640 1 -- 172.21.15.45:0/3200176025 >> v2:172.21.15.117:3300/0 conn(0x7febac0aef80 msgr2=0x7febac15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febabe00640 1 --2- 172.21.15.45:0/3200176025 >> v2:172.21.15.117:3300/0 conn(0x7febac0aef80 0x7febac15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febabe00640 1 -- 172.21.15.45:0/3200176025 >> v1:172.21.15.45:6789/0 conn(0x7febac070350 legacy=0x7febac15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febabe00640 1 -- 172.21.15.45:0/3200176025 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac15ee40 con 0x7febac0b31c0 2024-04-25T10:33:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febabe00640 1 --2- 172.21.15.45:0/3200176025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b31c0 0x7febac0b3610 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7febac06e220 tx=0x7feba401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52e5220f36d572a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febaaa00640 1 -- 172.21.15.45:0/3200176025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40ab430 con 0x7febac0b31c0 2024-04-25T10:33:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febaaa00640 1 -- 172.21.15.45:0/3200176025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40b0070 con 0x7febac0b31c0 2024-04-25T10:33:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.101+0000 7febaaa00640 1 -- 172.21.15.45:0/3200176025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40ab7f0 con 0x7febac0b31c0 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- 172.21.15.45:0/3200176025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b31c0 msgr2=0x7febac0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 --2- 172.21.15.45:0/3200176025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b31c0 0x7febac0b3610 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7febac06e220 tx=0x7feba401c520 comp rx=0 tx=0).stop 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- 172.21.15.45:0/3200176025 shutdown_connections 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 --2- 172.21.15.45:0/3200176025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0b31c0 0x7febac0b3610 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 --2- 172.21.15.45:0/3200176025 >> v2:172.21.15.117:3300/0 conn(0x7febac0aef80 0x7febac15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- 172.21.15.45:0/3200176025 >> 172.21.15.45:0/3200176025 conn(0x7febac0669a0 msgr2=0x7febac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- 172.21.15.45:0/3200176025 shutdown_connections 2024-04-25T10:33:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- 172.21.15.45:0/3200176025 wait complete. 2024-04-25T10:33:16.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 Processor -- start 2024-04-25T10:33:16.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- start start 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac070350 0x7febac153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 0x7febac0b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0b31c0 0x7febac0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1679a0 con 0x7febac0aef80 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac167820 con 0x7febac070350 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febb2800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac167b20 con 0x7febac0b31c0 2024-04-25T10:33:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 0x7febac0b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 0x7febac0b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58248/0 (socket says 172.21.15.45:58248) 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 -- 172.21.15.45:0/250967568 learned_addr learned my addr 172.21.15.45:0/250967568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 -- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0b31c0 msgr2=0x7febac0b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0b31c0 0x7febac0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febab400640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac070350 0x7febac153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 -- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac070350 msgr2=0x7febac153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac070350 0x7febac153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 -- 172.21.15.45:0/250967568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac155050 con 0x7febac0aef80 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7febab400640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac070350 0x7febac153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.105+0000 7feba2a00640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 0x7febac0b2130 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c790 tx=0x7feb9400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94017070 con 0x7febac0aef80 2024-04-25T10:33:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9400a640 con 0x7febac0aef80 2024-04-25T10:33:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac155330 con 0x7febac0aef80 2024-04-25T10:33:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9400a9f0 con 0x7febac0aef80 2024-04-25T10:33:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac154350 con 0x7febac0aef80 2024-04-25T10:33:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac04ef30 con 0x7febac0aef80 2024-04-25T10:33:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb9401a070 con 0x7febac0aef80 2024-04-25T10:33:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7feba3e00640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb88066f60 0x7feb88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7feb940846b0 con 0x7febac0aef80 2024-04-25T10:33:16.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.109+0000 7febab400640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb88066f60 0x7feb88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.113+0000 7febab400640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb88066f60 0x7feb88069420 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009ae0 tx=0x7feb9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.113+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb94054f70 con 0x7febac0aef80 2024-04-25T10:33:16.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7febac072590 con 0x7febac0aef80 2024-04-25T10:33:16.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7feba3e00640 1 -- 172.21.15.45:0/250967568 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v196) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb94059010 con 0x7febac0aef80 2024-04-25T10:33:16.281 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb88066f60 msgr2=0x7feb88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb88066f60 0x7feb88069420 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009ae0 tx=0x7feb9c008040 comp rx=0 tx=0).stop 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 msgr2=0x7febac0b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 0x7febac0b2130 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7feb9400c790 tx=0x7feb9400cc50 comp rx=0 tx=0).stop 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 shutdown_connections 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feb88066f60 0x7feb88069420 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febac0b31c0 0x7febac0b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febac0aef80 0x7febac0b2130 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 --2- 172.21.15.45:0/250967568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febac070350 0x7febac153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 >> 172.21.15.45:0/250967568 conn(0x7febac0669a0 msgr2=0x7febac0cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 shutdown_connections 2024-04-25T10:33:16.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.277+0000 7febb2800640 1 -- 172.21.15.45:0/250967568 wait complete. 2024-04-25T10:33:16.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:33:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc3a00640 1 Processor -- start 2024-04-25T10:33:16.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc3a00640 1 -- start start 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc3a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc3a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc0b40c0 con 0x7f9bbc0b5560 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc3a00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc0b5b00 con 0x7f9bbc066c00 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc3a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc161d60 con 0x7f9bbc066fe0 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc1a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9bbc0b5560 0x7f9bbc0ae850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52964/0 (socket says 172.21.15.45:52964) 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bc1a00640 1 -- 172.21.15.45:0/3260697207 learned_addr learned my addr 172.21.15.45:0/3260697207 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:16.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.441+0000 7f9bb3e00640 1 --2- 172.21.15.45:0/3260697207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764921164 0 0) 0x7f9bbc0b40c0 con 0x7f9bbc0b5560 2024-04-25T10:33:16.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba0003530 con 0x7f9bbc0b5560 2024-04-25T10:33:16.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173067239 0 0) 0x7f9bbc0b5b00 con 0x7f9bbc066c00 2024-04-25T10:33:16.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bbc0b40c0 con 0x7f9bbc066c00 2024-04-25T10:33:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bb3e00640 1 -- 172.21.15.45:0/3260697207 >> v1:172.21.15.117:6789/0 conn(0x7f9bbc066c00 legacy=0x7f9bbc072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bb3e00640 1 -- 172.21.15.45:0/3260697207 >> v1:172.21.15.45:6789/0 conn(0x7f9bbc0b5560 legacy=0x7f9bbc0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bb3e00640 1 -- 172.21.15.45:0/3260697207 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0b3a00 con 0x7f9bbc066fe0 2024-04-25T10:33:16.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bb3e00640 1 --2- 172.21.15.45:0/3260697207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc161820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40097f0 tx=0x7f9ba401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1146b1922d500a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40b0070 con 0x7f9bbc066fe0 2024-04-25T10:33:16.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba40ab600 con 0x7f9bbc066fe0 2024-04-25T10:33:16.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.445+0000 7f9bbbe00640 1 -- 172.21.15.45:0/3260697207 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40ab9b0 con 0x7f9bbc066fe0 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- 172.21.15.45:0/3260697207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 msgr2=0x7f9bbc161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/3260697207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc161820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40097f0 tx=0x7f9ba401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- 172.21.15.45:0/3260697207 shutdown_connections 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/3260697207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc161820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- 172.21.15.45:0/3260697207 >> 172.21.15.45:0/3260697207 conn(0x7f9bbc0b2560 msgr2=0x7f9bbc0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- 172.21.15.45:0/3260697207 shutdown_connections 2024-04-25T10:33:16.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- 172.21.15.45:0/3260697207 wait complete. 2024-04-25T10:33:16.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 Processor -- start 2024-04-25T10:33:16.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- start start 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 0x7f9bbc149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bbc0b5560 0x7f9bbc15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc15e540 con 0x7f9bbc066c00 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc158620 con 0x7f9bbc0b5560 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc3a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc15e6c0 con 0x7f9bbc066fe0 2024-04-25T10:33:16.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 0x7f9bbc149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 0x7f9bbc149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58262/0 (socket says 172.21.15.45:58262) 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 -- 172.21.15.45:0/733580677 learned_addr learned my addr 172.21.15.45:0/733580677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bb3e00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bbc0b5560 0x7f9bbc15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 -- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 msgr2=0x7f9bbc150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 -- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bbc0b5560 msgr2=0x7f9bbc15fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bbc0b5560 0x7f9bbc15fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 -- 172.21.15.45:0/733580677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc14a050 con 0x7f9bbc066c00 2024-04-25T10:33:16.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.449+0000 7f9bc1000640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 0x7f9bbc149b10 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00c4c0 tx=0x7f9bac00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0052d0 con 0x7f9bbc066c00 2024-04-25T10:33:16.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac005470 con 0x7f9bbc066c00 2024-04-25T10:33:16.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac012870 con 0x7f9bbc066c00 2024-04-25T10:33:16.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc1513a0 con 0x7f9bbc066c00 2024-04-25T10:33:16.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc14bfe0 con 0x7f9bbc066c00 2024-04-25T10:33:16.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc04ef30 con 0x7f9bbc066c00 2024-04-25T10:33:16.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bac012a10 con 0x7f9bbc066c00 2024-04-25T10:33:16.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb9600640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b98066f60 0x7f9b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f9bac084db0 con 0x7f9bbc066c00 2024-04-25T10:33:16.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.453+0000 7f9bb3e00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b98066f60 0x7f9b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.457+0000 7f9bb3e00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b98066f60 0x7f9b98069420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40ad000 tx=0x7f9ba40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.457+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bac089070 con 0x7f9bbc066c00 2024-04-25T10:33:16.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.617+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bbc0cfa00 con 0x7f9bbc066c00 2024-04-25T10:33:16.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.617+0000 7f9bb9600640 1 -- 172.21.15.45:0/733580677 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v196) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bac055cc0 con 0x7f9bbc066c00 2024-04-25T10:33:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b98066f60 msgr2=0x7f9b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b98066f60 0x7f9b98069420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40ad000 tx=0x7f9ba40a6000 comp rx=0 tx=0).stop 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 msgr2=0x7f9bbc149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 0x7f9bbc149b10 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00c4c0 tx=0x7f9bac00c980 comp rx=0 tx=0).stop 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 shutdown_connections 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9b98066f60 0x7f9b98069420 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bbc0b5560 0x7f9bbc15fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9bbc066fe0 0x7f9bbc150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 --2- 172.21.15.45:0/733580677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bbc066c00 0x7f9bbc149b10 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 >> 172.21.15.45:0/733580677 conn(0x7f9bbc0b2560 msgr2=0x7f9bbc0b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:16.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 shutdown_connections 2024-04-25T10:33:16.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.621+0000 7f9bc3a00640 1 -- 172.21.15.45:0/733580677 wait complete. 2024-04-25T10:33:16.643 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:16.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 Processor -- start 2024-04-25T10:33:16.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 -- start start 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef040d0040 0x7fef0415b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0415b740 0x7fef040b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef04071050 con 0x7fef041550b0 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0415d040 con 0x7fef040d0040 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef0a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef040712f0 con 0x7fef0415b740 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7feefbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef040d0040 0x7fef0415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7feefbe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef040d0040 0x7fef0415b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58276/0 (socket says 172.21.15.45:58276) 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7feefbe00640 1 -- 172.21.15.45:0/4057480188 learned_addr learned my addr 172.21.15.45:0/4057480188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.781+0000 7fef03e00640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0415b740 0x7fef040b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7feefbe00640 1 -- 172.21.15.45:0/4057480188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0415b740 msgr2=0x7fef040b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7feefbe00640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0415b740 0x7fef040b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7feefbe00640 1 -- 172.21.15.45:0/4057480188 >> v1:172.21.15.45:6789/0 conn(0x7fef041550b0 legacy=0x7fef040cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7feefbe00640 1 -- 172.21.15.45:0/4057480188 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040715d0 con 0x7fef040d0040 2024-04-25T10:33:16.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef03e00640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0415b740 0x7fef040b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:16.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7feefbe00640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef040d0040 0x7fef0415b200 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7feeec00bb10 tx=0x7feeec01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba6ed69a9c1b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef02a00640 1 -- 172.21.15.45:0/4057480188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0b4070 con 0x7fef040d0040 2024-04-25T10:33:16.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef02a00640 1 -- 172.21.15.45:0/4057480188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0a7cd0 con 0x7fef040d0040 2024-04-25T10:33:16.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef02a00640 1 -- 172.21.15.45:0/4057480188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0af660 con 0x7fef040d0040 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef0a800640 1 -- 172.21.15.45:0/4057480188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef040d0040 msgr2=0x7fef0415b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef0a800640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef040d0040 0x7fef0415b200 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7feeec00bb10 tx=0x7feeec01ec00 comp rx=0 tx=0).stop 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef0a800640 1 -- 172.21.15.45:0/4057480188 shutdown_connections 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef0a800640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0415b740 0x7fef040b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef0a800640 1 --2- 172.21.15.45:0/4057480188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef040d0040 0x7fef0415b200 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.785+0000 7fef0a800640 1 -- 172.21.15.45:0/4057480188 >> 172.21.15.45:0/4057480188 conn(0x7fef04066b70 msgr2=0x7fef04066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- 172.21.15.45:0/4057480188 shutdown_connections 2024-04-25T10:33:16.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- 172.21.15.45:0/4057480188 wait complete. 2024-04-25T10:33:16.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 Processor -- start 2024-04-25T10:33:16.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- start start 2024-04-25T10:33:16.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef040d0040 0x7fef04148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 0x7fef040cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0415b740 0x7fef040af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0415bcd0 con 0x7fef041550b0 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0415bb50 con 0x7fef0415b740 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0415c030 con 0x7fef040d0040 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 0x7fef040cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 0x7fef040cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58282/0 (socket says 172.21.15.45:58282) 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 -- 172.21.15.45:0/4050150610 learned_addr learned my addr 172.21.15.45:0/4050150610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef03400640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef040d0040 0x7fef04148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef03e00640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0415b740 0x7fef040af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 -- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef040d0040 msgr2=0x7fef04148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef040d0040 0x7fef04148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 -- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0415b740 msgr2=0x7fef040af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0415b740 0x7fef040af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 -- 172.21.15.45:0/4050150610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef040ccd70 con 0x7fef041550b0 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef03400640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef040d0040 0x7fef04148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefbe00640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 0x7fef040cc830 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7feeec00b2c0 tx=0x7feeec01b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0b4030 con 0x7fef041550b0 2024-04-25T10:33:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0b6070 con 0x7fef041550b0 2024-04-25T10:33:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef040cd040 con 0x7fef041550b0 2024-04-25T10:33:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.789+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0ad550 con 0x7fef041550b0 2024-04-25T10:33:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0414f2f0 con 0x7fef041550b0 2024-04-25T10:33:16.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0414f110 con 0x7fef041550b0 2024-04-25T10:33:16.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feeec0c1060 con 0x7fef041550b0 2024-04-25T10:33:16.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7feefb400640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feee0066f60 0x7feee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:16.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7feeec11f600 con 0x7fef041550b0 2024-04-25T10:33:16.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7fef03400640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feee0066f60 0x7feee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:16.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.793+0000 7fef03400640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feee0066f60 0x7feee0069420 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7feef4001710 tx=0x7feef4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:16.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.797+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef0414f110 con 0x7fef041550b0 2024-04-25T10:33:16.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.993+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef0404ef30 con 0x7fef041550b0 2024-04-25T10:33:16.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7feefb400640 1 -- 172.21.15.45:0/4050150610 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4959 (secure 0 0 0) 0x7feeec018820 con 0x7fef041550b0 2024-04-25T10:33:16.999 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:16.999 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:09.710400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:33:16.999 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feee0066f60 msgr2=0x7feee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feee0066f60 0x7feee0069420 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7feef4001710 tx=0x7feef4009040 comp rx=0 tx=0).stop 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 msgr2=0x7fef040cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 0x7fef040cc830 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7feeec00b2c0 tx=0x7feeec01b9a0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 shutdown_connections 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7feee0066f60 0x7feee0069420 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0415b740 0x7fef040af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef041550b0 0x7fef040cc830 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 --2- 172.21.15.45:0/4050150610 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef040d0040 0x7fef04148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 >> 172.21.15.45:0/4050150610 conn(0x7fef04066b70 msgr2=0x7fef040b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 shutdown_connections 2024-04-25T10:33:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:16.997+0000 7fef0a800640 1 -- 172.21.15.45:0/4050150610 wait complete. 2024-04-25T10:33:17.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:33:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.157+0000 7f7de1600640 1 Processor -- start 2024-04-25T10:33:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.157+0000 7f7de1600640 1 -- start start 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7de1600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ddc0d0040 0x7f7ddc15b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7de1600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc071050 con 0x7f7ddc15b740 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7de1600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc15d040 con 0x7f7ddc1550b0 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7de1600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0712f0 con 0x7f7ddc0d0040 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dd2000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7ddc0d0040 0x7f7ddc15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7ddb400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7ddc15b740 0x7f7ddc0b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52978/0 (socket says 172.21.15.45:52978) 2024-04-25T10:33:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7ddb400640 1 -- 172.21.15.45:0/3073974710 learned_addr learned my addr 172.21.15.45:0/3073974710 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dd2000640 1 -- 172.21.15.45:0/3073974710 >> v1:172.21.15.117:6789/0 conn(0x7f7ddc1550b0 legacy=0x7f7ddc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dd2000640 1 -- 172.21.15.45:0/3073974710 >> v1:172.21.15.45:6789/0 conn(0x7f7ddc15b740 legacy=0x7f7ddc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dd2000640 1 -- 172.21.15.45:0/3073974710 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0715d0 con 0x7f7ddc0d0040 2024-04-25T10:33:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dda000640 1 -- 172.21.15.45:0/3073974710 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529432158 0 0) 0x7f7ddc071050 con 0x7f7ddc15b740 2024-04-25T10:33:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dd2000640 1 --2- 172.21.15.45:0/3073974710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc0d0040 0x7f7ddc15b200 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40097c0 tx=0x7f7dc401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae0458b286bf10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dda000640 1 -- 172.21.15.45:0/3073974710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40b0070 con 0x7f7ddc0d0040 2024-04-25T10:33:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dda000640 1 -- 172.21.15.45:0/3073974710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc40ab600 con 0x7f7ddc0d0040 2024-04-25T10:33:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.161+0000 7f7dda000640 1 -- 172.21.15.45:0/3073974710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40ab9d0 con 0x7f7ddc0d0040 2024-04-25T10:33:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- 172.21.15.45:0/3073974710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc0d0040 msgr2=0x7f7ddc15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 --2- 172.21.15.45:0/3073974710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc0d0040 0x7f7ddc15b200 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40097c0 tx=0x7f7dc401c430 comp rx=0 tx=0).stop 2024-04-25T10:33:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- 172.21.15.45:0/3073974710 shutdown_connections 2024-04-25T10:33:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 --2- 172.21.15.45:0/3073974710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc0d0040 0x7f7ddc15b200 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- 172.21.15.45:0/3073974710 >> 172.21.15.45:0/3073974710 conn(0x7f7ddc066b70 msgr2=0x7f7ddc066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:17.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- 172.21.15.45:0/3073974710 shutdown_connections 2024-04-25T10:33:17.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- 172.21.15.45:0/3073974710 wait complete. 2024-04-25T10:33:17.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 Processor -- start 2024-04-25T10:33:17.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- start start 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ddc1550b0 0x7f7ddc14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 0x7f7ddc157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ddc14cda0 0x7f7ddc159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc15bcd0 con 0x7f7ddc15b740 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc15bb50 con 0x7f7ddc14cda0 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7de1600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1684c0 con 0x7f7ddc1550b0 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7dd2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 0x7f7ddc157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7dd2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 0x7f7ddc157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58310/0 (socket says 172.21.15.45:58310) 2024-04-25T10:33:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7dd2000640 1 -- 172.21.15.45:0/218334585 learned_addr learned my addr 172.21.15.45:0/218334585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7ddb400640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ddc14cda0 0x7f7ddc159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.165+0000 7f7ddaa00640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ddc1550b0 0x7f7ddc14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd2000640 1 -- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ddc1550b0 msgr2=0x7f7ddc14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd2000640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ddc1550b0 0x7f7ddc14e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd2000640 1 -- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ddc14cda0 msgr2=0x7f7ddc159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd2000640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ddc14cda0 0x7f7ddc159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd2000640 1 -- 172.21.15.45:0/218334585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc14f400 con 0x7f7ddc15b740 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7ddb400640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ddc14cda0 0x7f7ddc159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd2000640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 0x7f7ddc157f20 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40097c0 tx=0x7f7dc401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40b0050 con 0x7f7ddc15b740 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc40b1040 con 0x7f7ddc15b740 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4004240 con 0x7f7ddc15b740 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc14eb60 con 0x7f7ddc15b740 2024-04-25T10:33:17.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc14f120 con 0x7f7ddc15b740 2024-04-25T10:33:17.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc04ef30 con 0x7f7ddc15b740 2024-04-25T10:33:17.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dc40be060 con 0x7f7ddc15b740 2024-04-25T10:33:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd3400640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7db8066f60 0x7f7db8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f7dc411eb50 con 0x7f7ddc15b740 2024-04-25T10:33:17.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.169+0000 7f7ddaa00640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7db8066f60 0x7f7db8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.173+0000 7f7ddaa00640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7db8066f60 0x7f7db8069420 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc004e30 tx=0x7f7dcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.173+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dc4016050 con 0x7f7ddc15b740 2024-04-25T10:33:17.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.337+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ddc0aeb80 con 0x7f7ddc15b740 2024-04-25T10:33:17.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.337+0000 7f7dd3400640 1 -- 172.21.15.45:0/218334585 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17041 (secure 0 0 0) 0x7f7dc40bf3b0 con 0x7f7ddc15b740 2024-04-25T10:33:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:17.341 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":196,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:32:46.808432+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":1,"to":6}]},{"pgid":"29.22","mappings":[{"from":1,"to":4}]},{"pgid":"29.29","mappings":[{"from":1,"to":4}]},{"pgid":"29.2e","mappings":[{"from":1,"to":4}]},{"pgid":"29.3e","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/3198476999":"2024-04-26T10:32:06.688741+0000","172.21.15.45:6833/3198476999":"2024-04-26T10:32:06.688741+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7db8066f60 msgr2=0x7f7db8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7db8066f60 0x7f7db8069420 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc004e30 tx=0x7f7dcc009040 comp rx=0 tx=0).stop 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 msgr2=0x7f7ddc157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 0x7f7ddc157f20 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40097c0 tx=0x7f7dc401ac70 comp rx=0 tx=0).stop 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 shutdown_connections 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7db8066f60 0x7f7db8069420 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ddc14cda0 0x7f7ddc159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ddc15b740 0x7f7ddc157f20 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 --2- 172.21.15.45:0/218334585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ddc1550b0 0x7f7ddc14e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 >> 172.21.15.45:0/218334585 conn(0x7f7ddc066b70 msgr2=0x7f7ddc06d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 shutdown_connections 2024-04-25T10:33:17.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.341+0000 7f7de1600640 1 -- 172.21.15.45:0/218334585 wait complete. 2024-04-25T10:33:17.363 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:33:17.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:17.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9970c00640 1 Processor -- start 2024-04-25T10:33:17.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9970c00640 1 -- start start 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9970c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f996c1550b0 0x7f996c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9970c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c071050 con 0x7f996c15b910 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9970c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c15d040 con 0x7f996c0cfe70 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9970c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c15bd80 con 0x7f996c1550b0 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9969e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f996c1550b0 0x7f996c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f996a800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f996c15b910 0x7f996c0b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52990/0 (socket says 172.21.15.45:52990) 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9969e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f996c1550b0 0x7f996c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58314/0 (socket says 172.21.15.45:58314) 2024-04-25T10:33:17.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f996a800640 1 -- 172.21.15.45:0/1130831121 learned_addr learned my addr 172.21.15.45:0/1130831121 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9969e00640 1 -- 172.21.15.45:0/1130831121 >> v1:172.21.15.117:6789/0 conn(0x7f996c0cfe70 legacy=0x7f996c15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9969e00640 1 -- 172.21.15.45:0/1130831121 >> v1:172.21.15.45:6789/0 conn(0x7f996c15b910 legacy=0x7f996c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.501+0000 7f9969e00640 1 -- 172.21.15.45:0/1130831121 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0712f0 con 0x7f996c1550b0 2024-04-25T10:33:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9969e00640 1 --2- 172.21.15.45:0/1130831121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c1550b0 0x7f996c0cf930 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f995c0097f0 tx=0x7f995c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54957eba73116ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9969400640 1 -- 172.21.15.45:0/1130831121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0b0070 con 0x7f996c1550b0 2024-04-25T10:33:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9969400640 1 -- 172.21.15.45:0/1130831121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c0ab600 con 0x7f996c1550b0 2024-04-25T10:33:17.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9969400640 1 -- 172.21.15.45:0/1130831121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0ab9d0 con 0x7f996c1550b0 2024-04-25T10:33:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 -- 172.21.15.45:0/1130831121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c1550b0 msgr2=0x7f996c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 --2- 172.21.15.45:0/1130831121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c1550b0 0x7f996c0cf930 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f995c0097f0 tx=0x7f995c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 -- 172.21.15.45:0/1130831121 shutdown_connections 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 --2- 172.21.15.45:0/1130831121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c1550b0 0x7f996c0cf930 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 -- 172.21.15.45:0/1130831121 >> 172.21.15.45:0/1130831121 conn(0x7f996c066b70 msgr2=0x7f996c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 -- 172.21.15.45:0/1130831121 shutdown_connections 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.505+0000 7f9970c00640 1 -- 172.21.15.45:0/1130831121 wait complete. 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 Processor -- start 2024-04-25T10:33:17.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 -- start start 2024-04-25T10:33:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 0x7f996c156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996c1550b0 0x7f996c14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f996c15b910 0x7f996c148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1684c0 con 0x7f996c0cfe70 2024-04-25T10:33:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0720b0 con 0x7f996c1550b0 2024-04-25T10:33:17.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c168640 con 0x7f996c15b910 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 0x7f996c156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 0x7f996c156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58322/0 (socket says 172.21.15.45:58322) 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 -- 172.21.15.45:0/4290121873 learned_addr learned my addr 172.21.15.45:0/4290121873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9963e00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996c1550b0 0x7f996c14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f996a800640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f996c15b910 0x7f996c148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 -- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f996c15b910 msgr2=0x7f996c148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f996c15b910 0x7f996c148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 -- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996c1550b0 msgr2=0x7f996c14f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996c1550b0 0x7f996c14f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 -- 172.21.15.45:0/4290121873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c156b40 con 0x7f996c0cfe70 2024-04-25T10:33:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f996a800640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f996c15b910 0x7f996c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9969e00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 0x7f996c156460 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f995c0ae040 tx=0x7f995c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0b0050 con 0x7f996c0cfe70 2024-04-25T10:33:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c0bc070 con 0x7f996c0cfe70 2024-04-25T10:33:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0b5830 con 0x7f996c0cfe70 2024-04-25T10:33:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c156e20 con 0x7f996c0cfe70 2024-04-25T10:33:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.509+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c1576c0 con 0x7f996c0cfe70 2024-04-25T10:33:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.513+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c04ef30 con 0x7f996c0cfe70 2024-04-25T10:33:17.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.513+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f995c0b59d0 con 0x7f996c0cfe70 2024-04-25T10:33:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.513+0000 7f9962000640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9948066f60 0x7f9948069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.513+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f995c11f830 con 0x7f996c0cfe70 2024-04-25T10:33:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.513+0000 7f9963e00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9948066f60 0x7f9948069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.513+0000 7f9963e00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9948066f60 0x7f9948069420 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f996c150370 tx=0x7f9954009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.517+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f995c016090 con 0x7f996c0cfe70 2024-04-25T10:33:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.713+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f996c067160 con 0x7f996c0cfe70 2024-04-25T10:33:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.713+0000 7f9962000640 1 -- 172.21.15.45:0/4290121873 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f995c0f07d0 con 0x7f996c0cfe70 2024-04-25T10:33:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:17.718 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:09.710400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:33:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2024-04-25T10:33:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9948066f60 msgr2=0x7f9948069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9948066f60 0x7f9948069420 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f996c150370 tx=0x7f9954009040 comp rx=0 tx=0).stop 2024-04-25T10:33:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 msgr2=0x7f996c156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 0x7f996c156460 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f995c0ae040 tx=0x7f995c01ace0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 shutdown_connections 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f9948066f60 0x7f9948069420 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f996c15b910 0x7f996c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f996c1550b0 0x7f996c14f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 --2- 172.21.15.45:0/4290121873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f996c0cfe70 0x7f996c156460 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 >> 172.21.15.45:0/4290121873 conn(0x7f996c066b70 msgr2=0x7f996c06c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 shutdown_connections 2024-04-25T10:33:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.717+0000 7f9970c00640 1 -- 172.21.15.45:0/4290121873 wait complete. 2024-04-25T10:33:17.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 Processor -- start 2024-04-25T10:33:17.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 -- start start 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35d40ccb20 0x7f35d40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d415bdc0 0x7f35d40d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40ccfb0 con 0x7f35d415b9e0 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cd290 con 0x7f35d40ccb20 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35da600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40cd570 con 0x7f35d415bdc0 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35d40ccb20 0x7f35d40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35d40ccb20 0x7f35d40d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58338/0 (socket says 172.21.15.45:58338) 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 -- 172.21.15.45:0/3214293719 learned_addr learned my addr 172.21.15.45:0/3214293719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:17.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35d3e00640 1 --2- 172.21.15.45:0/3214293719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d415bdc0 0x7f35d40d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 -- 172.21.15.45:0/3214293719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d415bdc0 msgr2=0x7f35d40d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 --2- 172.21.15.45:0/3214293719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d415bdc0 0x7f35d40d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 -- 172.21.15.45:0/3214293719 >> v1:172.21.15.117:6789/0 conn(0x7f35d415b9e0 legacy=0x7f35d40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.877+0000 7f35caa00640 1 -- 172.21.15.45:0/3214293719 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40cd850 con 0x7f35d40ccb20 2024-04-25T10:33:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35caa00640 1 --2- 172.21.15.45:0/3214293719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d40ccb20 0x7f35d40d1520 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009870 tx=0x7f35bc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9ece08bd59aedb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35d2a00640 1 -- 172.21.15.45:0/3214293719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0b2070 con 0x7f35d40ccb20 2024-04-25T10:33:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35d2a00640 1 -- 172.21.15.45:0/3214293719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc0a9070 con 0x7f35d40ccb20 2024-04-25T10:33:17.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35d2a00640 1 -- 172.21.15.45:0/3214293719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0ad880 con 0x7f35d40ccb20 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 -- 172.21.15.45:0/3214293719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d40ccb20 msgr2=0x7f35d40d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 --2- 172.21.15.45:0/3214293719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d40ccb20 0x7f35d40d1520 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009870 tx=0x7f35bc01c560 comp rx=0 tx=0).stop 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 -- 172.21.15.45:0/3214293719 shutdown_connections 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 --2- 172.21.15.45:0/3214293719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d415bdc0 0x7f35d40d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 --2- 172.21.15.45:0/3214293719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d40ccb20 0x7f35d40d1520 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 -- 172.21.15.45:0/3214293719 >> 172.21.15.45:0/3214293719 conn(0x7f35d409a9d0 msgr2=0x7f35d409ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 -- 172.21.15.45:0/3214293719 shutdown_connections 2024-04-25T10:33:17.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.881+0000 7f35da600640 1 -- 172.21.15.45:0/3214293719 wait complete. 2024-04-25T10:33:17.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 Processor -- start 2024-04-25T10:33:17.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 -- start start 2024-04-25T10:33:17.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 0x7f35d414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d415bdc0 0x7f35d4159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d4153230 0x7f35d41536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c65f0 con 0x7f35d415b9e0 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c6470 con 0x7f35d415bdc0 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c6770 con 0x7f35d4153230 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 0x7f35d414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 0x7f35d414a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58350/0 (socket says 172.21.15.45:58350) 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 -- 172.21.15.45:0/3952200487 learned_addr learned my addr 172.21.15.45:0/3952200487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35caa00640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d415bdc0 0x7f35d4159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3e00640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d4153230 0x7f35d41536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 -- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d4153230 msgr2=0x7f35d41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d4153230 0x7f35d41536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 -- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d415bdc0 msgr2=0x7f35d4159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d415bdc0 0x7f35d4159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 -- 172.21.15.45:0/3952200487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4158d80 con 0x7f35d415b9e0 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3e00640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d4153230 0x7f35d41536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35d3400640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 0x7f35d414a720 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f35c4007fd0 tx=0x7f35c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4017070 con 0x7f35d415b9e0 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c40052d0 con 0x7f35d415b9e0 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d4159060 con 0x7f35d415b9e0 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.885+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4003840 con 0x7f35d415b9e0 2024-04-25T10:33:17.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.889+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d4159620 con 0x7f35d415b9e0 2024-04-25T10:33:17.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.889+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d404ef30 con 0x7f35d415b9e0 2024-04-25T10:33:17.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.889+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35c401a070 con 0x7f35d415b9e0 2024-04-25T10:33:17.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.889+0000 7f35cbe00640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35b0066f90 0x7f35b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:17.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.889+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f35c40844d0 con 0x7f35d415b9e0 2024-04-25T10:33:17.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.889+0000 7f35caa00640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35b0066f90 0x7f35b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:17.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.893+0000 7f35caa00640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35b0066f90 0x7f35b0069450 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0af000 tx=0x7f35bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:17.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:17.897+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35c4005480 con 0x7f35d415b9e0 2024-04-25T10:33:18.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.093+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35d40b3ed0 con 0x7f35d415b9e0 2024-04-25T10:33:18.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35cbe00640 1 -- 172.21.15.45:0/3952200487 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4959 (secure 0 0 0) 0x7f35c4054d00 con 0x7f35d415b9e0 2024-04-25T10:33:18.099 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:18.099 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:31:55.417950+0000","modified":"2024-04-25T10:32:09.710400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":194,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11496":{"gid":11496,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/1179754162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1179754162},{"type":"v1","addr":"172.21.15.45:6835","nonce":1179754162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:33:18.100 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 146 2024-04-25T10:33:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35b0066f90 msgr2=0x7f35b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35b0066f90 0x7f35b0069450 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0af000 tx=0x7f35bc008040 comp rx=0 tx=0).stop 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 msgr2=0x7f35d414a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 0x7f35d414a720 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f35c4007fd0 tx=0x7f35c400c890 comp rx=0 tx=0).stop 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 shutdown_connections 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f35b0066f90 0x7f35b0069450 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35d4153230 0x7f35d41536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35d415bdc0 0x7f35d4159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 --2- 172.21.15.45:0/3952200487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35d415b9e0 0x7f35d414a720 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 >> 172.21.15.45:0/3952200487 conn(0x7f35d409a9d0 msgr2=0x7f35d4154630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 shutdown_connections 2024-04-25T10:33:18.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.097+0000 7f35da600640 1 -- 172.21.15.45:0/3952200487 wait complete. 2024-04-25T10:33:18.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:33:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.257+0000 7fdf4f600640 1 Processor -- start 2024-04-25T10:33:18.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.257+0000 7fdf4f600640 1 -- start start 2024-04-25T10:33:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4f600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 0x7fdf4806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4f600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480b2770 con 0x7fdf4815bc40 2024-04-25T10:33:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4f600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4806c670 con 0x7fdf480cda10 2024-04-25T10:33:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4f600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806c950 con 0x7fdf480d0040 2024-04-25T10:33:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4cc00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdf4815bc40 0x7fdf480b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52996/0 (socket says 172.21.15.45:52996) 2024-04-25T10:33:18.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4cc00640 1 -- 172.21.15.45:0/1305307641 learned_addr learned my addr 172.21.15.45:0/1305307641 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 0x7fdf4806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982917325 0 0) 0x7fdf480b2770 con 0x7fdf4815bc40 2024-04-25T10:33:18.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf2c003530 con 0x7fdf4815bc40 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2973264141 0 0) 0x7fdf2c003530 con 0x7fdf4815bc40 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 msgr2=0x7fdf4806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 0x7fdf4806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 >> v1:172.21.15.117:6789/0 conn(0x7fdf480cda10 legacy=0x7fdf480cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4806cc30 con 0x7fdf4815bc40 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fdf38001680 con 0x7fdf4815bc40 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 >> v1:172.21.15.45:6789/0 conn(0x7fdf4815bc40 legacy=0x7fdf480b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 0x7fdf2c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c003f30 0x7fdf2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c004d50 0x7fdf2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806c950 con 0x7fdf2c004d50 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c00bb10 con 0x7fdf2c003f30 2024-04-25T10:33:18.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c00bd10 con 0x7fdf480d0040 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c004d50 0x7fdf2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4cc00640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 0x7fdf2c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf3fe00640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c003f30 0x7fdf2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 -- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 msgr2=0x7fdf2c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf480d0040 0x7fdf2c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 -- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c003f30 msgr2=0x7fdf2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c003f30 0x7fdf2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.261+0000 7fdf4d600640 1 -- 172.21.15.45:0/1305307641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c003530 con 0x7fdf2c004d50 2024-04-25T10:33:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4d600640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c004d50 0x7fdf2c0051d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fdf480b4510 tx=0x7fdf4001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cb6cc50577f09b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400b2070 con 0x7fdf2c004d50 2024-04-25T10:33:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf400a5c30 con 0x7fdf2c004d50 2024-04-25T10:33:18.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf47e00640 1 -- 172.21.15.45:0/1305307641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40009c10 con 0x7fdf2c004d50 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 -- 172.21.15.45:0/1305307641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c004d50 msgr2=0x7fdf2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 --2- 172.21.15.45:0/1305307641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf2c004d50 0x7fdf2c0051d0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fdf480b4510 tx=0x7fdf4001cc20 comp rx=0 tx=0).stop 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4d600640 1 -- 172.21.15.45:0/1305307641 reap_dead start 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 -- 172.21.15.45:0/1305307641 shutdown_connections 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 -- 172.21.15.45:0/1305307641 >> 172.21.15.45:0/1305307641 conn(0x7fdf48066b70 msgr2=0x7fdf48066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 -- 172.21.15.45:0/1305307641 shutdown_connections 2024-04-25T10:33:18.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 -- 172.21.15.45:0/1305307641 wait complete. 2024-04-25T10:33:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 Processor -- start 2024-04-25T10:33:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.265+0000 7fdf4f600640 1 -- start start 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c004d50 0x7fdf4814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 0x7fdf480b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf48071970 0x7fdf48071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806c210 con 0x7fdf480b0430 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806c090 con 0x7fdf2c004d50 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4806c390 con 0x7fdf48071970 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 0x7fdf480b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 0x7fdf480b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58370/0 (socket says 172.21.15.45:58370) 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 -- 172.21.15.45:0/4001657879 learned_addr learned my addr 172.21.15.45:0/4001657879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:18.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 -- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf48071970 msgr2=0x7fdf48071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf48071970 0x7fdf48071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 -- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c004d50 msgr2=0x7fdf4814a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4cc00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c004d50 0x7fdf4814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c004d50 0x7fdf4814a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 -- 172.21.15.45:0/4001657879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4815d310 con 0x7fdf480b0430 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4cc00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c004d50 0x7fdf4814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:18.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf3fe00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 0x7fdf480b08b0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fdf30011b70 tx=0x7fdf3000f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30019020 con 0x7fdf480b0430 2024-04-25T10:33:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3000fc30 con 0x7fdf480b0430 2024-04-25T10:33:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4815c270 con 0x7fdf480b0430 2024-04-25T10:33:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300050d0 con 0x7fdf480b0430 2024-04-25T10:33:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4815c830 con 0x7fdf480b0430 2024-04-25T10:33:18.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.269+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf480b0df0 con 0x7fdf480b0430 2024-04-25T10:33:18.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.273+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf3001c020 con 0x7fdf480b0430 2024-04-25T10:33:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.273+0000 7fdf45600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf24066f60 0x7fdf24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.273+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fdf30088880 con 0x7fdf480b0430 2024-04-25T10:33:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.273+0000 7fdf4cc00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf24066f60 0x7fdf24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.273+0000 7fdf4cc00640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf24066f60 0x7fdf24069420 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdf38007cd0 tx=0x7fdf38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.277+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf30023020 con 0x7fdf480b0430 2024-04-25T10:33:18.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf4804ef30 con 0x7fdf480b0430 2024-04-25T10:33:18.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf45600640 1 -- 172.21.15.45:0/4001657879 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17041 (secure 0 0 0) 0x7fdf30059790 con 0x7fdf480b0430 2024-04-25T10:33:18.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:18.445 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":196,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:32:46.808432+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:31:53.058645+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2024-04-25T10:31:54.409004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":1,"to":6}]},{"pgid":"29.22","mappings":[{"from":1,"to":4}]},{"pgid":"29.29","mappings":[{"from":1,"to":4}]},{"pgid":"29.2e","mappings":[{"from":1,"to":4}]},{"pgid":"29.3e","mappings":[{"from":1,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/3198476999":"2024-04-26T10:32:06.688741+0000","172.21.15.45:6833/3198476999":"2024-04-26T10:32:06.688741+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf24066f60 msgr2=0x7fdf24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf24066f60 0x7fdf24069420 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdf38007cd0 tx=0x7fdf38009040 comp rx=0 tx=0).stop 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 msgr2=0x7fdf480b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 0x7fdf480b08b0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fdf30011b70 tx=0x7fdf3000f5a0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 shutdown_connections 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdf24066f60 0x7fdf24069420 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf48071970 0x7fdf48071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf480b0430 0x7fdf480b08b0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 --2- 172.21.15.45:0/4001657879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf2c004d50 0x7fdf4814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 >> 172.21.15.45:0/4001657879 conn(0x7fdf48066b70 msgr2=0x7fdf48159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 shutdown_connections 2024-04-25T10:33:18.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.441+0000 7fdf4f600640 1 -- 172.21.15.45:0/4001657879 wait complete. 2024-04-25T10:33:18.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:33:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.605+0000 7fbc65000640 1 Processor -- start 2024-04-25T10:33:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.605+0000 7fbc65000640 1 -- start start 2024-04-25T10:33:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc65000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc6006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc65000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc600b5780 con 0x7fbc6006b2c0 2024-04-25T10:33:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc65000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc600b5a60 con 0x7fbc600b5010 2024-04-25T10:33:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc65000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b5d40 con 0x7fbc6015db40 2024-04-25T10:33:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbc6006b2c0 0x7fbc600b6820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53000/0 (socket says 172.21.15.45:53000) 2024-04-25T10:33:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5fe00640 1 -- 172.21.15.45:0/2499048247 learned_addr learned my addr 172.21.15.45:0/2499048247 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5f400640 1 --2- 172.21.15.45:0/2499048247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc6006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846795604 0 0) 0x7fbc600b5780 con 0x7fbc6006b2c0 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc44003530 con 0x7fbc6006b2c0 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946774856 0 0) 0x7fbc600b5a60 con 0x7fbc600b5010 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc600b5780 con 0x7fbc600b5010 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5f400640 1 -- 172.21.15.45:0/2499048247 >> v1:172.21.15.117:6789/0 conn(0x7fbc600b5010 legacy=0x7fbc6015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5f400640 1 -- 172.21.15.45:0/2499048247 >> v1:172.21.15.45:6789/0 conn(0x7fbc6006b2c0 legacy=0x7fbc600b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.609+0000 7fbc5f400640 1 -- 172.21.15.45:0/2499048247 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600b6020 con 0x7fbc6015db40 2024-04-25T10:33:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc5f400640 1 --2- 172.21.15.45:0/2499048247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc6006ac90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbc500097f0 tx=0x7fbc5001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b467c9eac256652c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500b5070 con 0x7fbc6015db40 2024-04-25T10:33:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc500a89d0 con 0x7fbc6015db40 2024-04-25T10:33:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc5ea00640 1 -- 172.21.15.45:0/2499048247 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500a8d80 con 0x7fbc6015db40 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 -- 172.21.15.45:0/2499048247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 msgr2=0x7fbc6006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 --2- 172.21.15.45:0/2499048247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc6006ac90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbc500097f0 tx=0x7fbc5001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 -- 172.21.15.45:0/2499048247 shutdown_connections 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 --2- 172.21.15.45:0/2499048247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc6006ac90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 -- 172.21.15.45:0/2499048247 >> 172.21.15.45:0/2499048247 conn(0x7fbc600d17b0 msgr2=0x7fbc600d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 -- 172.21.15.45:0/2499048247 shutdown_connections 2024-04-25T10:33:18.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 -- 172.21.15.45:0/2499048247 wait complete. 2024-04-25T10:33:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 Processor -- start 2024-04-25T10:33:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.613+0000 7fbc65000640 1 -- start start 2024-04-25T10:33:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 0x7fbc60156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc600b5010 0x7fbc600c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc600c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6006b750 con 0x7fbc6006b2c0 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b5450 con 0x7fbc600b5010 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6006b8d0 con 0x7fbc6015db40 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 0x7fbc60156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 0x7fbc60156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58374/0 (socket says 172.21.15.45:58374) 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 -- 172.21.15.45:0/2695463468 learned_addr learned my addr 172.21.15.45:0/2695463468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 -- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 msgr2=0x7fbc600c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc600c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 -- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc600b5010 msgr2=0x7fbc600c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc600b5010 0x7fbc600c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 -- 172.21.15.45:0/2695463468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600c95e0 con 0x7fbc6006b2c0 2024-04-25T10:33:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc5f400640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 0x7fbc60156c90 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c10 tx=0x7fbc5001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500c1070 con 0x7fbc6006b2c0 2024-04-25T10:33:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc500b0640 con 0x7fbc6006b2c0 2024-04-25T10:33:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600c98c0 con 0x7fbc6006b2c0 2024-04-25T10:33:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc500b5040 con 0x7fbc6006b2c0 2024-04-25T10:33:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc600c9e80 con 0x7fbc6006b2c0 2024-04-25T10:33:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc60075120 con 0x7fbc6006b2c0 2024-04-25T10:33:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.617+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc500c4070 con 0x7fbc6006b2c0 2024-04-25T10:33:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.621+0000 7fbc57400640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc3c066f60 0x7fbc3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.621+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fbc501228e0 con 0x7fbc6006b2c0 2024-04-25T10:33:18.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.621+0000 7fbc57e00640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc3c066f60 0x7fbc3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.621+0000 7fbc57e00640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc3c066f60 0x7fbc3c069420 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007fd0 tx=0x7fbc48007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:18.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.621+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc500cb070 con 0x7fbc6006b2c0 2024-04-25T10:33:18.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.821+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc6006ae60 con 0x7fbc6006b2c0 2024-04-25T10:33:18.824 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-04-25T10:33:18.821+0000 7ff8b7e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:33:18.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.821+0000 7fbc57400640 1 -- 172.21.15.45:0/2695463468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v147) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc50018830 con 0x7fbc6006b2c0 2024-04-25T10:33:18.826 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:33:18.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc3c066f60 msgr2=0x7fbc3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc3c066f60 0x7fbc3c069420 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007fd0 tx=0x7fbc48007900 comp rx=0 tx=0).stop 2024-04-25T10:33:18.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 msgr2=0x7fbc60156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 0x7fbc60156c90 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c10 tx=0x7fbc5001ba70 comp rx=0 tx=0).stop 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 shutdown_connections 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fbc3c066f60 0x7fbc3c069420 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc6015db40 0x7fbc600c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc600b5010 0x7fbc600c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 --2- 172.21.15.45:0/2695463468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc6006b2c0 0x7fbc60156c90 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 >> 172.21.15.45:0/2695463468 conn(0x7fbc600d17b0 msgr2=0x7fbc6006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:18.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 shutdown_connections 2024-04-25T10:33:18.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.825+0000 7fbc65000640 1 -- 172.21.15.45:0/2695463468 wait complete. 2024-04-25T10:33:18.838 INFO:tasks.ceph.mds.c.smithi045.stderr: -2603> 2024-04-25T10:32:12.698+0000 7f1b8e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:33:18.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:33:18.885 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-04-25T10:33:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 Processor -- start 2024-04-25T10:33:18.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 -- start start 2024-04-25T10:33:18.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc49c15bc10 0x7fc49c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc49c0cf980 0x7fc49c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:18.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c15fec0 con 0x7fc49c0d1830 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c1601a0 con 0x7fc49c0cf980 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc4a1a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0cc5b0 con 0x7fc49c15bc10 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc493e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc49c0cf980 0x7fc49c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc49b400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc49c0d1830 0x7fc49c0d1c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53016/0 (socket says 172.21.15.45:53016) 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc49b400640 1 -- 172.21.15.45:0/3396683401 learned_addr learned my addr 172.21.15.45:0/3396683401 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc49aa00640 1 --2- 172.21.15.45:0/3396683401 >> v2:172.21.15.117:3300/0 conn(0x7fc49c15bc10 0x7fc49c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc493e00640 1 -- 172.21.15.45:0/3396683401 >> v2:172.21.15.117:3300/0 conn(0x7fc49c15bc10 msgr2=0x7fc49c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc493e00640 1 --2- 172.21.15.45:0/3396683401 >> v2:172.21.15.117:3300/0 conn(0x7fc49c15bc10 0x7fc49c15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc493e00640 1 -- 172.21.15.45:0/3396683401 >> v1:172.21.15.45:6789/0 conn(0x7fc49c0d1830 legacy=0x7fc49c0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:18.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.993+0000 7fc493e00640 1 -- 172.21.15.45:0/3396683401 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0cc890 con 0x7fc49c0cf980 2024-04-25T10:33:19.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc493e00640 1 --2- 172.21.15.45:0/3396683401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0cf980 0x7fc49c0b1060 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fc49c067940 tx=0x7fc48401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba67fc79984435aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:19.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc49a000640 1 -- 172.21.15.45:0/3396683401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840b1070 con 0x7fc49c0cf980 2024-04-25T10:33:19.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc49a000640 1 -- 172.21.15.45:0/3396683401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4840ac5f0 con 0x7fc49c0cf980 2024-04-25T10:33:19.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc49a000640 1 -- 172.21.15.45:0/3396683401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840ac9c0 con 0x7fc49c0cf980 2024-04-25T10:33:19.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 -- 172.21.15.45:0/3396683401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0cf980 msgr2=0x7fc49c0b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:19.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/3396683401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0cf980 0x7fc49c0b1060 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fc49c067940 tx=0x7fc48401c480 comp rx=0 tx=0).stop 2024-04-25T10:33:19.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 -- 172.21.15.45:0/3396683401 shutdown_connections 2024-04-25T10:33:19.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/3396683401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0cf980 0x7fc49c0b1060 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/3396683401 >> v2:172.21.15.117:3300/0 conn(0x7fc49c15bc10 0x7fc49c15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 -- 172.21.15.45:0/3396683401 >> 172.21.15.45:0/3396683401 conn(0x7fc49c09a9d0 msgr2=0x7fc49c09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:19.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 -- 172.21.15.45:0/3396683401 shutdown_connections 2024-04-25T10:33:19.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:18.997+0000 7fc4a1a00640 1 -- 172.21.15.45:0/3396683401 wait complete. 2024-04-25T10:33:19.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 Processor -- start 2024-04-25T10:33:19.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- start start 2024-04-25T10:33:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc49c0cf980 0x7fc49c157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 0x7fc49c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc49c15bc10 0x7fc49c0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cd5c0 con 0x7fc49c0d1830 2024-04-25T10:33:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0b17b0 con 0x7fc49c15bc10 2024-04-25T10:33:19.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cd740 con 0x7fc49c0cf980 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 0x7fc49c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 0x7fc49c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58406/0 (socket says 172.21.15.45:58406) 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 -- 172.21.15.45:0/1207494989 learned_addr learned my addr 172.21.15.45:0/1207494989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc49b400640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc49c15bc10 0x7fc49c0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc49aa00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc49c0cf980 0x7fc49c157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 -- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc49c0cf980 msgr2=0x7fc49c157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc49c0cf980 0x7fc49c157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 -- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc49c15bc10 msgr2=0x7fc49c0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc49c15bc10 0x7fc49c0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 -- 172.21.15.45:0/1207494989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c15c100 con 0x7fc49c0d1830 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc49b400640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc49c15bc10 0x7fc49c0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc493e00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 0x7fc49c0b2270 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fc48401c960 tx=0x7fc484019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:19.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840b1040 con 0x7fc49c0d1830 2024-04-25T10:33:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4840ba070 con 0x7fc49c0d1830 2024-04-25T10:33:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c15c3e0 con 0x7fc49c0d1830 2024-04-25T10:33:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4840b5670 con 0x7fc49c0d1830 2024-04-25T10:33:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c15c9a0 con 0x7fc49c0d1830 2024-04-25T10:33:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.001+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c04ef30 con 0x7fc49c0d1830 2024-04-25T10:33:19.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.005+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4840b5810 con 0x7fc49c0d1830 2024-04-25T10:33:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.005+0000 7fc492a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc478066f60 0x7fc478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.005+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6026+0+0 (secure 0 0 0) 0x7fc48411eef0 con 0x7fc49c0d1830 2024-04-25T10:33:19.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.005+0000 7fc49aa00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc478066f60 0x7fc478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:19.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.005+0000 7fc49aa00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc478066f60 0x7fc478069420 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00b9d0 tx=0x7fc48c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.009+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4840a9070 con 0x7fc49c0d1830 2024-04-25T10:33:19.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.209+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc49c15d350 con 0x7fc49c0d1830 2024-04-25T10:33:19.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.825+0000 7fc492a00640 1 -- 172.21.15.45:0/1207494989 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v148) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc484016070 con 0x7fc49c0d1830 2024-04-25T10:33:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc478066f60 msgr2=0x7fc478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc478066f60 0x7fc478069420 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00b9d0 tx=0x7fc48c008040 comp rx=0 tx=0).stop 2024-04-25T10:33:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 msgr2=0x7fc49c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 0x7fc49c0b2270 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fc48401c960 tx=0x7fc484019220 comp rx=0 tx=0).stop 2024-04-25T10:33:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 shutdown_connections 2024-04-25T10:33:19.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc478066f60 0x7fc478069420 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc49c15bc10 0x7fc49c0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc49c0d1830 0x7fc49c0b2270 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 --2- 172.21.15.45:0/1207494989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc49c0cf980 0x7fc49c157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 >> 172.21.15.45:0/1207494989 conn(0x7fc49c09a9d0 msgr2=0x7fc49c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 shutdown_connections 2024-04-25T10:33:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:19.829+0000 7fc4a1a00640 1 -- 172.21.15.45:0/1207494989 wait complete. 2024-04-25T10:33:19.855 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:33:20.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 Processor -- start 2024-04-25T10:33:20.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 -- start start 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85280cf040 0x7f85280cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8528071ab0 0x7f85280cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8528071f80 con 0x7f852815bc00 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8528072260 con 0x7f85280cf040 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ee00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85280b3700 con 0x7f8528071ab0 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85280cf040 0x7f85280cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85280cf040 0x7f85280cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58412/0 (socket says 172.21.15.45:58412) 2024-04-25T10:33:20.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 -- 172.21.15.45:0/3181297637 learned_addr learned my addr 172.21.15.45:0/3181297637 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:20.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f852ce00640 1 --2- 172.21.15.45:0/3181297637 >> v2:172.21.15.117:3300/0 conn(0x7f8528071ab0 0x7f85280cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:20.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 -- 172.21.15.45:0/3181297637 >> v2:172.21.15.117:3300/0 conn(0x7f8528071ab0 msgr2=0x7f85280cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:20.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 --2- 172.21.15.45:0/3181297637 >> v2:172.21.15.117:3300/0 conn(0x7f8528071ab0 0x7f85280cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 -- 172.21.15.45:0/3181297637 >> v1:172.21.15.45:6789/0 conn(0x7f852815bc00 legacy=0x7f85280713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:20.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.049+0000 7f8527e00640 1 -- 172.21.15.45:0/3181297637 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852815b120 con 0x7f85280cf040 2024-04-25T10:33:20.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f8527e00640 1 --2- 172.21.15.45:0/3181297637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cf040 0x7f85280cf9e0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f85280b4e40 tx=0x7f851801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304e975a2e2b82dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:20.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f8527400640 1 -- 172.21.15.45:0/3181297637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180ab610 con 0x7f85280cf040 2024-04-25T10:33:20.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f8527400640 1 -- 172.21.15.45:0/3181297637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85180b0070 con 0x7f85280cf040 2024-04-25T10:33:20.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f8527400640 1 -- 172.21.15.45:0/3181297637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180ab9d0 con 0x7f85280cf040 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 -- 172.21.15.45:0/3181297637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cf040 msgr2=0x7f85280cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 --2- 172.21.15.45:0/3181297637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cf040 0x7f85280cf9e0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f85280b4e40 tx=0x7f851801c520 comp rx=0 tx=0).stop 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 -- 172.21.15.45:0/3181297637 shutdown_connections 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 --2- 172.21.15.45:0/3181297637 >> v2:172.21.15.117:3300/0 conn(0x7f8528071ab0 0x7f85280cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 --2- 172.21.15.45:0/3181297637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85280cf040 0x7f85280cf9e0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 -- 172.21.15.45:0/3181297637 >> 172.21.15.45:0/3181297637 conn(0x7f8528066b70 msgr2=0x7f8528066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 -- 172.21.15.45:0/3181297637 shutdown_connections 2024-04-25T10:33:20.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 -- 172.21.15.45:0/3181297637 wait complete. 2024-04-25T10:33:20.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.053+0000 7f852ee00640 1 Processor -- start 2024-04-25T10:33:20.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- start start 2024-04-25T10:33:20.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8528071ab0 0x7f8528140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:20.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85280cf040 0x7f85280af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:20.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 0x7f85280afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:20.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280aec90 con 0x7f852815bc00 2024-04-25T10:33:20.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85280cf630 con 0x7f85280cf040 2024-04-25T10:33:20.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85280aee10 con 0x7f8528071ab0 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 0x7f85280afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 0x7f85280afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58416/0 (socket says 172.21.15.45:58416) 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 -- 172.21.15.45:0/1962795132 learned_addr learned my addr 172.21.15.45:0/1962795132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f851f400640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85280cf040 0x7f85280af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f8527e00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8528071ab0 0x7f8528140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 -- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8528071ab0 msgr2=0x7f8528140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8528071ab0 0x7f8528140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 -- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85280cf040 msgr2=0x7f85280af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85280cf040 0x7f85280af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 -- 172.21.15.45:0/1962795132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85280a95d0 con 0x7f852815bc00 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f8527e00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8528071ab0 0x7f8528140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f851f400640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85280cf040 0x7f85280af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:20.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ce00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 0x7f85280afe90 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f8520009fd0 tx=0x7f852000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:20.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520014420 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8520019070 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520014800 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85280a98b0 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85280a9e70 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.057+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85280aa070 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.061+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85200149a0 con 0x7f852815bc00 2024-04-25T10:33:20.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.061+0000 7f8524c00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:20.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.061+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6026+0+0 (secure 0 0 0) 0x7f8520085940 con 0x7f852815bc00 2024-04-25T10:33:20.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.061+0000 7f8527e00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:20.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.061+0000 7f8527e00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f85180af000 tx=0x7f85180a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:20.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.065+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8520056800 con 0x7f852815bc00 2024-04-25T10:33:20.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.245+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f852804ef30 con 0x7f852815bc00 2024-04-25T10:33:20.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.905+0000 7f8524c00640 1 -- 172.21.15.45:0/1962795132 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v198) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8520056100 con 0x7f852815bc00 2024-04-25T10:33:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:33:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.905+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 msgr2=0x7f8504069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.905+0000 7f852ee00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f85180af000 tx=0x7f85180a7000 comp rx=0 tx=0).stop 2024-04-25T10:33:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.905+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 msgr2=0x7f85280afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.905+0000 7f852ee00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 0x7f85280afe90 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f8520009fd0 tx=0x7f852000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 shutdown_connections 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8504066f60 0x7f8504069420 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f852815bc00 0x7f85280afe90 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85280cf040 0x7f85280af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 --2- 172.21.15.45:0/1962795132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8528071ab0 0x7f8528140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 >> 172.21.15.45:0/1962795132 conn(0x7f8528066b70 msgr2=0x7f85280af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 shutdown_connections 2024-04-25T10:33:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:20.909+0000 7f852ee00640 1 -- 172.21.15.45:0/1962795132 wait complete. 2024-04-25T10:33:20.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:33:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 Processor -- start 2024-04-25T10:33:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 -- start start 2024-04-25T10:33:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f825c15bd10 0x7f825c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f825c15c0f0 0x7f825c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c0cf9e0 0x7f825c0cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0d0390 con 0x7f825c15bd10 2024-04-25T10:33:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c06c0e0 con 0x7f825c15c0f0 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8262a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c15c6a0 con 0x7f825c0cf9e0 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f825c15bd10 0x7f825c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f825c15bd10 0x7f825c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58430/0 (socket says 172.21.15.45:58430) 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 -- 172.21.15.45:0/207739353 learned_addr learned my addr 172.21.15.45:0/207739353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8260a00640 1 --2- 172.21.15.45:0/207739353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c0cf9e0 0x7f825c0cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8253400640 1 --2- 172.21.15.45:0/207739353 >> v2:172.21.15.117:3300/0 conn(0x7f825c15c0f0 0x7f825c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 -- 172.21.15.45:0/207739353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c0cf9e0 msgr2=0x7f825c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 --2- 172.21.15.45:0/207739353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c0cf9e0 0x7f825c0cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 -- 172.21.15.45:0/207739353 >> v2:172.21.15.117:3300/0 conn(0x7f825c15c0f0 msgr2=0x7f825c0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 --2- 172.21.15.45:0/207739353 >> v2:172.21.15.117:3300/0 conn(0x7f825c15c0f0 0x7f825c0b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f825be00640 1 -- 172.21.15.45:0/207739353 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c15c980 con 0x7f825c15bd10 2024-04-25T10:33:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.077+0000 7f8253400640 1 --2- 172.21.15.45:0/207739353 >> v2:172.21.15.117:3300/0 conn(0x7f825c15c0f0 0x7f825c0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f825be00640 1 --2- 172.21.15.45:0/207739353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c0b4610 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f825c067980 tx=0x7f824c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dd509ea63b0518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f825b400640 1 -- 172.21.15.45:0/207739353 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0aa610 con 0x7f825c15bd10 2024-04-25T10:33:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f825b400640 1 -- 172.21.15.45:0/207739353 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c0af070 con 0x7f825c15bd10 2024-04-25T10:33:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f825b400640 1 -- 172.21.15.45:0/207739353 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c0aa9d0 con 0x7f825c15bd10 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 -- 172.21.15.45:0/207739353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 msgr2=0x7f825c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 --2- 172.21.15.45:0/207739353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c0b4610 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f825c067980 tx=0x7f824c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 -- 172.21.15.45:0/207739353 shutdown_connections 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 --2- 172.21.15.45:0/207739353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c0cf9e0 0x7f825c0cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 --2- 172.21.15.45:0/207739353 >> v2:172.21.15.117:3300/0 conn(0x7f825c15c0f0 0x7f825c0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 --2- 172.21.15.45:0/207739353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c0b4610 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 -- 172.21.15.45:0/207739353 >> 172.21.15.45:0/207739353 conn(0x7f825c066b70 msgr2=0x7f825c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 -- 172.21.15.45:0/207739353 shutdown_connections 2024-04-25T10:33:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 -- 172.21.15.45:0/207739353 wait complete. 2024-04-25T10:33:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.081+0000 7f8262a00640 1 Processor -- start 2024-04-25T10:33:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- start start 2024-04-25T10:33:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f825c0cf9e0 0x7f825c14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c15c0f0 0x7f825c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0b2ea0 con 0x7f825c15bd10 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0b2d20 con 0x7f825c0cf9e0 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c06cdf0 con 0x7f825c15c0f0 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58440/0 (socket says 172.21.15.45:58440) 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 -- 172.21.15.45:0/429710453 learned_addr learned my addr 172.21.15.45:0/429710453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f825be00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f825c0cf9e0 0x7f825c14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8260a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c15c0f0 0x7f825c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 -- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c15c0f0 msgr2=0x7f825c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c15c0f0 0x7f825c071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 -- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f825c0cf9e0 msgr2=0x7f825c14ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f825c0cf9e0 0x7f825c14ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 -- 172.21.15.45:0/429710453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c14d3f0 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8253400640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c156d10 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f824400c7f0 tx=0x7f824400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244015070 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8244007820 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8244007c10 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c0b0c30 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c14cba0 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.085+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c04ef30 con 0x7f825c15bd10 2024-04-25T10:33:21.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.089+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8244018070 con 0x7f825c15bd10 2024-04-25T10:33:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.089+0000 7f8258c00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8238066f90 0x7f8238069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.089+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5588+0+0 (secure 0 0 0) 0x7f8244085030 con 0x7f825c15bd10 2024-04-25T10:33:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.089+0000 7f825be00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8238066f90 0x7f8238069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:21.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.089+0000 7f825be00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8238066f90 0x7f8238069450 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f824c0ae000 tx=0x7f824c0a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:21.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.093+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f825c14d3f0 con 0x7f825c15bd10 2024-04-25T10:33:21.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.257+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f825c06d9b0 con 0x7f825c15bd10 2024-04-25T10:33:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.909+0000 7f8258c00640 1 -- 172.21.15.45:0/429710453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v199) v1 ==== 169+0+0 (secure 0 0 0) 0x7f82440559c0 con 0x7f825c15bd10 2024-04-25T10:33:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T10:33:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8238066f90 msgr2=0x7f8238069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8238066f90 0x7f8238069450 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f824c0ae000 tx=0x7f824c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:33:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 msgr2=0x7f825c156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c156d10 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f824400c7f0 tx=0x7f824400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 shutdown_connections 2024-04-25T10:33:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8238066f90 0x7f8238069450 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f825c15c0f0 0x7f825c071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f825c15bd10 0x7f825c156d10 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 --2- 172.21.15.45:0/429710453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f825c0cf9e0 0x7f825c14ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 >> 172.21.15.45:0/429710453 conn(0x7f825c066b70 msgr2=0x7f825c068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 shutdown_connections 2024-04-25T10:33:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:21.913+0000 7f8262a00640 1 -- 172.21.15.45:0/429710453 wait complete. 2024-04-25T10:33:21.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T10:33:22.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.085+0000 7fe747800640 1 Processor -- start 2024-04-25T10:33:22.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.085+0000 7fe747800640 1 -- start start 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe747800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7400aef80 0x7fe74015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe747800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe740070350 0x7fe74015b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe747800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74015b080 con 0x7fe74015bac0 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe747800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74015adb0 con 0x7fe740070350 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe747800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74015eb60 con 0x7fe7400aef80 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe740070350 0x7fe74015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe740070350 0x7fe74015b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58454/0 (socket says 172.21.15.45:58454) 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe745800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe74015bac0 0x7fe7400b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53048/0 (socket says 172.21.15.45:53048) 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 -- 172.21.15.45:0/4030963539 learned_addr learned my addr 172.21.15.45:0/4030963539 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe744e00640 1 --2- 172.21.15.45:0/4030963539 >> v2:172.21.15.117:3300/0 conn(0x7fe7400aef80 0x7fe74015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 -- 172.21.15.45:0/4030963539 >> v2:172.21.15.117:3300/0 conn(0x7fe7400aef80 msgr2=0x7fe74015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:22.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 --2- 172.21.15.45:0/4030963539 >> v2:172.21.15.117:3300/0 conn(0x7fe7400aef80 0x7fe74015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 -- 172.21.15.45:0/4030963539 >> v1:172.21.15.45:6789/0 conn(0x7fe74015bac0 legacy=0x7fe7400b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.089+0000 7fe73fe00640 1 -- 172.21.15.45:0/4030963539 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74015ee40 con 0x7fe740070350 2024-04-25T10:33:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe73fe00640 1 --2- 172.21.15.45:0/4030963539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe740070350 0x7fe74015b580 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fe74006e220 tx=0x7fe72c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f1ce5a96814e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe73f400640 1 -- 172.21.15.45:0/4030963539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0aa660 con 0x7fe740070350 2024-04-25T10:33:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe73f400640 1 -- 172.21.15.45:0/4030963539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c0af070 con 0x7fe740070350 2024-04-25T10:33:22.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe73f400640 1 -- 172.21.15.45:0/4030963539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0aaa20 con 0x7fe740070350 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 -- 172.21.15.45:0/4030963539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe740070350 msgr2=0x7fe74015b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 --2- 172.21.15.45:0/4030963539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe740070350 0x7fe74015b580 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fe74006e220 tx=0x7fe72c01c520 comp rx=0 tx=0).stop 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 -- 172.21.15.45:0/4030963539 shutdown_connections 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 --2- 172.21.15.45:0/4030963539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe740070350 0x7fe74015b580 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 --2- 172.21.15.45:0/4030963539 >> v2:172.21.15.117:3300/0 conn(0x7fe7400aef80 0x7fe74015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 -- 172.21.15.45:0/4030963539 >> 172.21.15.45:0/4030963539 conn(0x7fe7400669a0 msgr2=0x7fe740066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 -- 172.21.15.45:0/4030963539 shutdown_connections 2024-04-25T10:33:22.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 -- 172.21.15.45:0/4030963539 wait complete. 2024-04-25T10:33:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 Processor -- start 2024-04-25T10:33:22.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.093+0000 7fe747800640 1 -- start start 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe740070350 0x7fe74014ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 0x7fe7400ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe74015bac0 0x7fe740150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe740165990 con 0x7fe7400aef80 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe740165810 con 0x7fe740070350 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe740165b10 con 0x7fe74015bac0 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 0x7fe7400ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 0x7fe7400ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58466/0 (socket says 172.21.15.45:58466) 2024-04-25T10:33:22.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 -- 172.21.15.45:0/137550368 learned_addr learned my addr 172.21.15.45:0/137550368 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 -- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe74015bac0 msgr2=0x7fe740150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe74015bac0 0x7fe740150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe744e00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe740070350 0x7fe74014ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 -- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe740070350 msgr2=0x7fe74014ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe740070350 0x7fe74014ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 -- 172.21.15.45:0/137550368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74014f630 con 0x7fe7400aef80 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73fe00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 0x7fe7400ce360 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fe72c019130 tx=0x7fe72c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0bd5f0 con 0x7fe7400aef80 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c0bd790 con 0x7fe7400aef80 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0bdb40 con 0x7fe7400aef80 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74006c270 con 0x7fe7400aef80 2024-04-25T10:33:22.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74006c830 con 0x7fe7400aef80 2024-04-25T10:33:22.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74004ef30 con 0x7fe7400aef80 2024-04-25T10:33:22.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe72c0b33f0 con 0x7fe7400aef80 2024-04-25T10:33:22.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.097+0000 7fe73cc00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe720066f60 0x7fe720069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.101+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe72c11f7a0 con 0x7fe7400aef80 2024-04-25T10:33:22.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.101+0000 7fe744e00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe720066f60 0x7fe720069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.101+0000 7fe744e00640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe720066f60 0x7fe720069420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe734009790 tx=0x7fe734008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.105+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe72c016110 con 0x7fe7400aef80 2024-04-25T10:33:22.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.265+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7fe7400ccdf0 con 0x7fe7400aef80 2024-04-25T10:33:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe73cc00640 1 -- 172.21.15.45:0/137550368 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe72c0f0480 con 0x7fe7400aef80 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe720066f60 msgr2=0x7fe720069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe720066f60 0x7fe720069420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe734009790 tx=0x7fe734008040 comp rx=0 tx=0).stop 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 msgr2=0x7fe7400ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 0x7fe7400ce360 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fe72c019130 tx=0x7fe72c01ad60 comp rx=0 tx=0).stop 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 shutdown_connections 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe720066f60 0x7fe720069420 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe74015bac0 0x7fe740150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7400aef80 0x7fe7400ce360 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 --2- 172.21.15.45:0/137550368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe740070350 0x7fe74014ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 >> 172.21.15.45:0/137550368 conn(0x7fe7400669a0 msgr2=0x7fe740066d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 shutdown_connections 2024-04-25T10:33:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.269+0000 7fe747800640 1 -- 172.21.15.45:0/137550368 wait complete. 2024-04-25T10:33:22.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:33:22.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 Processor -- start 2024-04-25T10:33:22.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 -- start start 2024-04-25T10:33:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2600b3480 0x7fd26015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2600cfb50 0x7fd260070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd260071bc0 0x7fd260072050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26015bb90 con 0x7fd2600cfb50 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2600ad840 con 0x7fd2600b3480 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd266c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26015e710 con 0x7fd260071bc0 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2600cfb50 0x7fd260070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2600cfb50 0x7fd260070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58476/0 (socket says 172.21.15.45:58476) 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 -- 172.21.15.45:0/3242511083 learned_addr learned my addr 172.21.15.45:0/3242511083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd264c00640 1 --2- 172.21.15.45:0/3242511083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd260071bc0 0x7fd260072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd25fe00640 1 --2- 172.21.15.45:0/3242511083 >> v2:172.21.15.117:3300/0 conn(0x7fd2600b3480 0x7fd26015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 -- 172.21.15.45:0/3242511083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd260071bc0 msgr2=0x7fd260072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 --2- 172.21.15.45:0/3242511083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd260071bc0 0x7fd260072050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 -- 172.21.15.45:0/3242511083 >> v2:172.21.15.117:3300/0 conn(0x7fd2600b3480 msgr2=0x7fd26015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 --2- 172.21.15.45:0/3242511083 >> v2:172.21.15.117:3300/0 conn(0x7fd2600b3480 0x7fd26015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd257400640 1 -- 172.21.15.45:0/3242511083 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26015e9f0 con 0x7fd2600cfb50 2024-04-25T10:33:22.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.433+0000 7fd25fe00640 1 --2- 172.21.15.45:0/3242511083 >> v2:172.21.15.117:3300/0 conn(0x7fd2600b3480 0x7fd26015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd257400640 1 --2- 172.21.15.45:0/3242511083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600cfb50 0x7fd260070350 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fd26006e5e0 tx=0x7fd24801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62a8b61ada2dc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd25f400640 1 -- 172.21.15.45:0/3242511083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480b0070 con 0x7fd2600cfb50 2024-04-25T10:33:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd25f400640 1 -- 172.21.15.45:0/3242511083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2480ab600 con 0x7fd2600cfb50 2024-04-25T10:33:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd25f400640 1 -- 172.21.15.45:0/3242511083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480ab9d0 con 0x7fd2600cfb50 2024-04-25T10:33:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 -- 172.21.15.45:0/3242511083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600cfb50 msgr2=0x7fd260070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 --2- 172.21.15.45:0/3242511083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600cfb50 0x7fd260070350 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fd26006e5e0 tx=0x7fd24801c520 comp rx=0 tx=0).stop 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 -- 172.21.15.45:0/3242511083 shutdown_connections 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 --2- 172.21.15.45:0/3242511083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd260071bc0 0x7fd260072050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 --2- 172.21.15.45:0/3242511083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600cfb50 0x7fd260070350 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 --2- 172.21.15.45:0/3242511083 >> v2:172.21.15.117:3300/0 conn(0x7fd2600b3480 0x7fd26015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 -- 172.21.15.45:0/3242511083 >> 172.21.15.45:0/3242511083 conn(0x7fd2600669a0 msgr2=0x7fd260066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 -- 172.21.15.45:0/3242511083 shutdown_connections 2024-04-25T10:33:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.437+0000 7fd266c00640 1 -- 172.21.15.45:0/3242511083 wait complete. 2024-04-25T10:33:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 Processor -- start 2024-04-25T10:33:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- start start 2024-04-25T10:33:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 0x7fd260159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2600b3480 0x7fd2600afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2600cfb50 0x7fd2601530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260070a10 con 0x7fd260071bc0 2024-04-25T10:33:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260070890 con 0x7fd2600b3480 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600ae530 con 0x7fd2600cfb50 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 0x7fd260159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 0x7fd260159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58492/0 (socket says 172.21.15.45:58492) 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 -- 172.21.15.45:0/1173253106 learned_addr learned my addr 172.21.15.45:0/1173253106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd264c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2600cfb50 0x7fd2601530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd257400640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2600b3480 0x7fd2600afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 -- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2600cfb50 msgr2=0x7fd2601530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2600cfb50 0x7fd2601530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 -- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2600b3480 msgr2=0x7fd2600afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2600b3480 0x7fd2600afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 -- 172.21.15.45:0/1173253106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600b0c20 con 0x7fd260071bc0 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25fe00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 0x7fd260159c30 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7fd250009fc0 tx=0x7fd25000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25001a070 con 0x7fd260071bc0 2024-04-25T10:33:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250003bc0 con 0x7fd260071bc0 2024-04-25T10:33:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2600b0660 con 0x7fd260071bc0 2024-04-25T10:33:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250004030 con 0x7fd260071bc0 2024-04-25T10:33:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2601528c0 con 0x7fd260071bc0 2024-04-25T10:33:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.441+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26004ef30 con 0x7fd260071bc0 2024-04-25T10:33:22.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.445+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd25001d070 con 0x7fd260071bc0 2024-04-25T10:33:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.445+0000 7fd25cc00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd23c066f60 0x7fd23c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.445+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd2500850b0 con 0x7fd260071bc0 2024-04-25T10:33:22.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.445+0000 7fd257400640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd23c066f60 0x7fd23c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.445+0000 7fd257400640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd23c066f60 0x7fd23c069420 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd2480ad000 tx=0x7fd2480a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.449+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd250010070 con 0x7fd260071bc0 2024-04-25T10:33:22.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.621+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd260158d80 con 0x7fd260071bc0 2024-04-25T10:33:22.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.625+0000 7fd25cc00640 1 -- 172.21.15.45:0/1173253106 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd250055c70 con 0x7fd260071bc0 2024-04-25T10:33:22.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd23c066f60 msgr2=0x7fd23c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd23c066f60 0x7fd23c069420 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd2480ad000 tx=0x7fd2480a6000 comp rx=0 tx=0).stop 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 msgr2=0x7fd260159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 0x7fd260159c30 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7fd250009fc0 tx=0x7fd25000d560 comp rx=0 tx=0).stop 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 shutdown_connections 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd23c066f60 0x7fd23c069420 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2600cfb50 0x7fd2601530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2600b3480 0x7fd2600afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 --2- 172.21.15.45:0/1173253106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd260071bc0 0x7fd260159c30 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 >> 172.21.15.45:0/1173253106 conn(0x7fd2600669a0 msgr2=0x7fd260158140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 shutdown_connections 2024-04-25T10:33:22.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.629+0000 7fd266c00640 1 -- 172.21.15.45:0/1173253106 wait complete. 2024-04-25T10:33:22.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:33:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc43cc00640 1 Processor -- start 2024-04-25T10:33:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc43cc00640 1 -- start start 2024-04-25T10:33:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc43cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 0x7fc4380b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc43cc00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43815f3b0 con 0x7fc4380cfd90 2024-04-25T10:33:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc43cc00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4380ca810 con 0x7fc4380cd2e0 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc43cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815ef20 con 0x7fc4380b0f70 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.789+0000 7fc42fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4380cfd90 0x7fc4380ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53050/0 (socket says 172.21.15.45:53050) 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 -- 172.21.15.45:0/3883875814 learned_addr learned my addr 172.21.15.45:0/3883875814 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634639307 0 0) 0x7fc43815f3b0 con 0x7fc4380cfd90 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435e00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 0x7fc4380b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc41c003530 con 0x7fc4380cfd90 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3171402392 0 0) 0x7fc41c003530 con 0x7fc4380cfd90 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 msgr2=0x7fc4380b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 0x7fc4380b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 >> v1:172.21.15.117:6789/0 conn(0x7fc4380cd2e0 legacy=0x7fc4380cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380b1890 con 0x7fc4380cfd90 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1928955508 0 0) 0x7fc420001680 con 0x7fc4380cfd90 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 >> v1:172.21.15.45:6789/0 conn(0x7fc4380cfd90 legacy=0x7fc4380ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c0039f0 0x7fc41c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc41c004810 0x7fc41c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 0x7fc41c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815ef20 con 0x7fc41c004810 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c00ba80 con 0x7fc41c0039f0 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c00bc80 con 0x7fc4380b0f70 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc41c004810 0x7fc41c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc435e00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c0039f0 0x7fc41c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc436800640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 0x7fc41c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 -- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 msgr2=0x7fc41c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380b0f70 0x7fc41c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 -- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c0039f0 msgr2=0x7fc41c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c0039f0 0x7fc41c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.793+0000 7fc42fe00640 1 -- 172.21.15.45:0/3883875814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c003530 con 0x7fc41c004810 2024-04-25T10:33:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc42fe00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc41c004810 0x7fc41c004c90 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fc438071060 tx=0x7fc42001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8dfa8c0841c810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420007d20 con 0x7fc41c004810 2024-04-25T10:33:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc420005ae0 con 0x7fc41c004810 2024-04-25T10:33:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc435400640 1 -- 172.21.15.45:0/3883875814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420007070 con 0x7fc41c004810 2024-04-25T10:33:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 -- 172.21.15.45:0/3883875814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc41c004810 msgr2=0x7fc41c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 --2- 172.21.15.45:0/3883875814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc41c004810 0x7fc41c004c90 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fc438071060 tx=0x7fc42001f370 comp rx=0 tx=0).stop 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc436800640 1 -- 172.21.15.45:0/3883875814 reap_dead start 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 -- 172.21.15.45:0/3883875814 shutdown_connections 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 -- 172.21.15.45:0/3883875814 >> 172.21.15.45:0/3883875814 conn(0x7fc4380b1d50 msgr2=0x7fc4380b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 -- 172.21.15.45:0/3883875814 shutdown_connections 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 -- 172.21.15.45:0/3883875814 wait complete. 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 Processor -- start 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 -- start start 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.797+0000 7fc43cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c004810 0x7fc438099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 0x7fc43806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380ad3f0 0x7fc4380ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815ebb0 con 0x7fc43806dfc0 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43815e8d0 con 0x7fc41c004810 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380b1350 con 0x7fc4380ad3f0 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 0x7fc43806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 0x7fc43806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58514/0 (socket says 172.21.15.45:58514) 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 -- 172.21.15.45:0/1103226110 learned_addr learned my addr 172.21.15.45:0/1103226110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc436800640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380ad3f0 0x7fc4380ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc435e00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c004810 0x7fc438099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 -- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380ad3f0 msgr2=0x7fc4380ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380ad3f0 0x7fc4380ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 -- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c004810 msgr2=0x7fc438099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c004810 0x7fc438099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 -- 172.21.15.45:0/1103226110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380c6550 con 0x7fc43806dfc0 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42fe00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 0x7fc43806e440 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fc42001f3b0 tx=0x7fc41c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4200047f0 con 0x7fc43806dfc0 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc420004990 con 0x7fc43806dfc0 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420004d40 con 0x7fc43806dfc0 2024-04-25T10:33:22.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4380c6830 con 0x7fc43806dfc0 2024-04-25T10:33:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380c6df0 con 0x7fc43806dfc0 2024-04-25T10:33:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43804ef30 con 0x7fc43806dfc0 2024-04-25T10:33:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.801+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc420007070 con 0x7fc43806dfc0 2024-04-25T10:33:22.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.805+0000 7fc42e000640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc414066f90 0x7fc414069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.805+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc420121c10 con 0x7fc43806dfc0 2024-04-25T10:33:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.805+0000 7fc435e00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc414066f90 0x7fc414069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.805+0000 7fc435e00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc414066f90 0x7fc414069450 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc4380cbff0 tx=0x7fc428009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:22.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:22.805+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4200c3030 con 0x7fc43806dfc0 2024-04-25T10:33:23.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.029+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7fc4380cda20 con 0x7fc43806dfc0 2024-04-25T10:33:23.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.917+0000 7fc42e000640 1 -- 172.21.15.45:0/1103226110 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v736) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc420019080 con 0x7fc43806dfc0 2024-04-25T10:33:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc414066f90 msgr2=0x7fc414069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc414066f90 0x7fc414069450 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc4380cbff0 tx=0x7fc428009190 comp rx=0 tx=0).stop 2024-04-25T10:33:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 msgr2=0x7fc43806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 0x7fc43806e440 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fc42001f3b0 tx=0x7fc41c005840 comp rx=0 tx=0).stop 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 shutdown_connections 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc414066f90 0x7fc414069450 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4380ad3f0 0x7fc4380ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc43806dfc0 0x7fc43806e440 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 --2- 172.21.15.45:0/1103226110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc41c004810 0x7fc438099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 >> 172.21.15.45:0/1103226110 conn(0x7fc4380b1d50 msgr2=0x7fc4380b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 shutdown_connections 2024-04-25T10:33:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:23.921+0000 7fc43cc00640 1 -- 172.21.15.45:0/1103226110 wait complete. 2024-04-25T10:33:23.952 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:33:23.952 INFO:tasks.cephfs_test_runner: 2024-04-25T10:33:23.952 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:33:23.952 INFO:tasks.cephfs_test_runner:Ran 13 tests in 892.911s 2024-04-25T10:33:23.953 INFO:tasks.cephfs_test_runner: 2024-04-25T10:33:23.953 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:33:23.953 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:33:23.966 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:33:23.975 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:33:23.976 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:33:23.976 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:23.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:33:23.987 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T10:33:23.987 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:24.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353400k,nr_inodes=4088350,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5117 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:33:24.044 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:33:24.045 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:24.055 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:33:24.055 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-25T10:33:24.055 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:33:24.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350300k,nr_inodes=4087575,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3013 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274988k,nr_inodes=818747,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274992k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:33:24.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.106 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:33:24.116 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:33:24.116 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:33:24.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:33:24.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:33:24.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T10:33:24.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.409 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T10:33:24.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T10:33:24.484 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-04-25T10:33:24.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:33:24.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.487 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-25T10:33:24.495 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:33:24.496 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:33:24.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.497 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:33:24.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.561 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:33:24.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.624 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-04-25T10:33:24.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.773 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-04-25T10:33:24.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:33:24.783 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-04-25T10:33:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-04-25T10:33:24.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:33:24.855 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:33:24.865 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:33:24.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:33:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.005+0000 7f92cfe00640 1 Processor -- start 2024-04-25T10:33:25.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.005+0000 7f92cfe00640 1 -- start start 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cfe00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92c80d0040 0x7f92c815b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cfe00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c8071050 con 0x7f92c81550b0 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cfe00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c815d040 con 0x7f92c80d0040 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c80712f0 con 0x7f92c815b740 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92c80d0040 0x7f92c815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f92c80d0040 0x7f92c815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56924/0 (socket says 172.21.15.45:56924) 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 -- 172.21.15.45:0/956238242 learned_addr learned my addr 172.21.15.45:0/956238242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cde00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 -- 172.21.15.45:0/956238242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 msgr2=0x7f92c80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 -- 172.21.15.45:0/956238242 >> v1:172.21.15.117:6789/0 conn(0x7f92c81550b0 legacy=0x7f92c80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cca00640 1 -- 172.21.15.45:0/956238242 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c80715d0 con 0x7f92c80d0040 2024-04-25T10:33:25.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.009+0000 7f92cde00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:25.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cca00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c815b200 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009870 tx=0x7f92b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a77e848e288d9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92c7e00640 1 -- 172.21.15.45:0/956238242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40a95f0 con 0x7f92c80d0040 2024-04-25T10:33:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92c7e00640 1 -- 172.21.15.45:0/956238242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40a9790 con 0x7f92c80d0040 2024-04-25T10:33:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92c7e00640 1 -- 172.21.15.45:0/956238242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40a9b60 con 0x7f92c80d0040 2024-04-25T10:33:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- 172.21.15.45:0/956238242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 msgr2=0x7f92c815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c815b200 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009870 tx=0x7f92b401c530 comp rx=0 tx=0).stop 2024-04-25T10:33:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- 172.21.15.45:0/956238242 shutdown_connections 2024-04-25T10:33:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 --2- 172.21.15.45:0/956238242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c815b200 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- 172.21.15.45:0/956238242 >> 172.21.15.45:0/956238242 conn(0x7f92c8066b70 msgr2=0x7f92c8066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- 172.21.15.45:0/956238242 shutdown_connections 2024-04-25T10:33:25.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- 172.21.15.45:0/956238242 wait complete. 2024-04-25T10:33:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 Processor -- start 2024-04-25T10:33:25.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- start start 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c80af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c81550b0 0x7f92c814ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815bcd0 con 0x7f92c80d0040 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815bb50 con 0x7f92c81550b0 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.013+0000 7f92cfe00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815c030 con 0x7f92c815b740 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c80af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c80af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56936/0 (socket says 172.21.15.45:56936) 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 -- 172.21.15.45:0/351318767 learned_addr learned my addr 172.21.15.45:0/351318767 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cde00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cca00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c81550b0 0x7f92c814ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 -- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 msgr2=0x7f92c80b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 -- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c81550b0 msgr2=0x7f92c814ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c81550b0 0x7f92c814ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 -- 172.21.15.45:0/351318767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c814b3c0 con 0x7f92c80d0040 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cd400640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c80af530 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00c420 tx=0x7f92bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc0052d0 con 0x7f92c80d0040 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc005470 con 0x7f92c80d0040 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc007260 con 0x7f92c80d0040 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c80b0580 con 0x7f92c80d0040 2024-04-25T10:33:25.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c80b0b30 con 0x7f92c80d0040 2024-04-25T10:33:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c804ef30 con 0x7f92c80d0040 2024-04-25T10:33:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.017+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92bc007400 con 0x7f92c80d0040 2024-04-25T10:33:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.021+0000 7f92c5600640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f90 0x7f92a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.021+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f92bc084250 con 0x7f92c80d0040 2024-04-25T10:33:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.021+0000 7f92cca00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f90 0x7f92a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.021+0000 7f92cca00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f90 0x7f92a8069450 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f92b40ae000 tx=0x7f92b40b3040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.021+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92bc0554f0 con 0x7f92c80d0040 2024-04-25T10:33:25.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.185+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92c80ae740 con 0x7f92a8066f90 2024-04-25T10:33:25.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.185+0000 7f92c5600640 1 -- 172.21.15.45:0/351318767 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13493 (secure 0 0 0) 0x7f92c80ae740 con 0x7f92a8066f90 2024-04-25T10:33:25.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:25.190 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:33:25.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f90 msgr2=0x7f92a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f90 0x7f92a8069450 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f92b40ae000 tx=0x7f92b40b3040 comp rx=0 tx=0).stop 2024-04-25T10:33:25.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 msgr2=0x7f92c80af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c80af530 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00c420 tx=0x7f92bc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 shutdown_connections 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f92a8066f90 0x7f92a8069450 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f92c815b740 0x7f92c80b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92c81550b0 0x7f92c814ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 --2- 172.21.15.45:0/351318767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92c80d0040 0x7f92c80af530 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 >> 172.21.15.45:0/351318767 conn(0x7f92c8066b70 msgr2=0x7f92c8159200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 shutdown_connections 2024-04-25T10:33:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.189+0000 7f92cfe00640 1 -- 172.21.15.45:0/351318767 wait complete. 2024-04-25T10:33:25.210 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":661,"stamp":"2024-04-25T10:33:25.095898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":475560,"kb_used_data":20776,"kb_used_omap":12,"kb_used_meta":454643,"kb_avail":754499160,"statfs":{"total":773094113280,"available":772607139840,"internally_reserved":0,"allocated":21274624,"data_stored":10016888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465554992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12622614,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.261846"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":513,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-25T10:33:22.524028+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:33:22.524028+0000","last_peered":"2024-04-25T10:33:22.524028+0000","last_clean":"2024-04-25T10:33:22.524028+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:33:22.524028+0000","last_undegraded":"2024-04-25T10:33:22.524028+0000","last_fullsized":"2024-04-25T10:33:22.524028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64744,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307096,"statfs":{"total":96636764160,"available":96570466304,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47524,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324316,"statfs":{"total":96636764160,"available":96588099584,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69672,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302168,"statfs":{"total":96636764160,"available":96565420032,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52264,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319576,"statfs":{"total":96636764160,"available":96583245824,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60004,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542333,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69664,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302176,"statfs":{"total":96636764160,"available":96565428224,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:33:25.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:33:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc19ce00640 1 Processor -- start 2024-04-25T10:33:25.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc19ce00640 1 -- start start 2024-04-25T10:33:25.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc19ce00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1980ae020 0x7fc1980ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc19ce00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980ad840 con 0x7fc198070c20 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc19ce00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19815e9f0 con 0x7fc1980cf930 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc19ce00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19815bb90 con 0x7fc1980ae020 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc196800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1980ae020 0x7fc1980ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc18fe00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc198070c20 0x7fc19815e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53894/0 (socket says 172.21.15.45:53894) 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc18fe00640 1 -- 172.21.15.45:0/3800573929 learned_addr learned my addr 172.21.15.45:0/3800573929 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc196800640 1 -- 172.21.15.45:0/3800573929 >> v1:172.21.15.117:6789/0 conn(0x7fc1980cf930 legacy=0x7fc1980cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc196800640 1 -- 172.21.15.45:0/3800573929 >> v1:172.21.15.45:6789/0 conn(0x7fc198070c20 legacy=0x7fc19815e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc196800640 1 -- 172.21.15.45:0/3800573929 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19815ecd0 con 0x7fc1980ae020 2024-04-25T10:33:25.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.349+0000 7fc195400640 1 -- 172.21.15.45:0/3800573929 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075427628 0 0) 0x7fc1980ad840 con 0x7fc198070c20 2024-04-25T10:33:25.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc196800640 1 --2- 172.21.15.45:0/3800573929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980ae020 0x7fc1980ae4b0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc19001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60773e2e674b2e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc195400640 1 -- 172.21.15.45:0/3800573929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900b2070 con 0x7fc1980ae020 2024-04-25T10:33:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc195400640 1 -- 172.21.15.45:0/3800573929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1900a5a00 con 0x7fc1980ae020 2024-04-25T10:33:25.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc195400640 1 -- 172.21.15.45:0/3800573929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900a5dd0 con 0x7fc1980ae020 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 -- 172.21.15.45:0/3800573929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980ae020 msgr2=0x7fc1980ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 --2- 172.21.15.45:0/3800573929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980ae020 0x7fc1980ae4b0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc19001c940 comp rx=0 tx=0).stop 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 -- 172.21.15.45:0/3800573929 shutdown_connections 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 --2- 172.21.15.45:0/3800573929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980ae020 0x7fc1980ae4b0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 -- 172.21.15.45:0/3800573929 >> 172.21.15.45:0/3800573929 conn(0x7fc198066b70 msgr2=0x7fc198066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 -- 172.21.15.45:0/3800573929 shutdown_connections 2024-04-25T10:33:25.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.353+0000 7fc19ce00640 1 -- 172.21.15.45:0/3800573929 wait complete. 2024-04-25T10:33:25.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 Processor -- start 2024-04-25T10:33:25.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 -- start start 2024-04-25T10:33:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc198070c20 0x7fc19815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980ae020 0x7fc1981470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 0x7fc19815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19816ba70 con 0x7fc1980cf930 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19816b8f0 con 0x7fc1980ae020 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19816bbf0 con 0x7fc198070c20 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 0x7fc19815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 0x7fc19815ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56956/0 (socket says 172.21.15.45:56956) 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 -- 172.21.15.45:0/4040001662 learned_addr learned my addr 172.21.15.45:0/4040001662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc195e00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc198070c20 0x7fc19815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc18fe00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980ae020 0x7fc1981470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 -- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc198070c20 msgr2=0x7fc19815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc198070c20 0x7fc19815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 -- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980ae020 msgr2=0x7fc1981470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980ae020 0x7fc1981470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 -- 172.21.15.45:0/4040001662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980cc4f0 con 0x7fc1980cf930 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc196800640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 0x7fc19815ce00 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc1900196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900b2050 con 0x7fc1980cf930 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1900be070 con 0x7fc1980cf930 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900ad890 con 0x7fc1980cf930 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980cc7d0 con 0x7fc1980cf930 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.357+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980ccd90 con 0x7fc1980cf930 2024-04-25T10:33:25.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.361+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19804ef30 con 0x7fc1980cf930 2024-04-25T10:33:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.361+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1900ada30 con 0x7fc1980cf930 2024-04-25T10:33:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.361+0000 7fc18e000640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc174066f60 0x7fc174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.361+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc19011fd00 con 0x7fc1980cf930 2024-04-25T10:33:25.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.361+0000 7fc195e00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc174066f60 0x7fc174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.361+0000 7fc195e00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc174066f60 0x7fc174069420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc1980cc0c0 tx=0x7fc188009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.365+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc190016560 con 0x7fc1980cf930 2024-04-25T10:33:25.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.521+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc198160890 con 0x7fc174066f60 2024-04-25T10:33:25.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc18e000640 1 -- 172.21.15.45:0/4040001662 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13493 (secure 0 0 0) 0x7fc198160890 con 0x7fc174066f60 2024-04-25T10:33:25.528 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:25.528 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:33:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc174066f60 msgr2=0x7fc174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc174066f60 0x7fc174069420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc1980cc0c0 tx=0x7fc188009040 comp rx=0 tx=0).stop 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 msgr2=0x7fc19815ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 0x7fc19815ce00 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc1900196e0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 shutdown_connections 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc174066f60 0x7fc174069420 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1980cf930 0x7fc19815ce00 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1980ae020 0x7fc1981470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 --2- 172.21.15.45:0/4040001662 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc198070c20 0x7fc19815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 >> 172.21.15.45:0/4040001662 conn(0x7fc198066b70 msgr2=0x7fc1980b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 shutdown_connections 2024-04-25T10:33:25.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.525+0000 7fc19ce00640 1 -- 172.21.15.45:0/4040001662 wait complete. 2024-04-25T10:33:25.548 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":661,"stamp":"2024-04-25T10:33:25.095898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":475560,"kb_used_data":20776,"kb_used_omap":12,"kb_used_meta":454643,"kb_avail":754499160,"statfs":{"total":773094113280,"available":772607139840,"internally_reserved":0,"allocated":21274624,"data_stored":10016888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465554992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12622614,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.261846"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":513,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-25T10:33:22.524028+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:33:22.524028+0000","last_peered":"2024-04-25T10:33:22.524028+0000","last_clean":"2024-04-25T10:33:22.524028+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:33:22.524028+0000","last_undegraded":"2024-04-25T10:33:22.524028+0000","last_fullsized":"2024-04-25T10:33:22.524028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64744,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307096,"statfs":{"total":96636764160,"available":96570466304,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47524,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324316,"statfs":{"total":96636764160,"available":96588099584,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69672,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302168,"statfs":{"total":96636764160,"available":96565420032,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52264,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319576,"statfs":{"total":96636764160,"available":96583245824,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60004,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542333,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69664,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302176,"statfs":{"total":96636764160,"available":96565428224,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:33:25.549 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:33:25.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:33:25.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.685+0000 7f50cb600640 1 Processor -- start 2024-04-25T10:33:25.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.685+0000 7f50cb600640 1 -- start start 2024-04-25T10:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50cb600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50c40d0350 0x7f50c40d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50cb600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c40ccaa0 con 0x7f50c415df60 2024-04-25T10:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50cb600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c40ccd80 con 0x7f50c406aea0 2024-04-25T10:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50cb600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c415e340 con 0x7f50c40d0350 2024-04-25T10:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50c3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50c40d0350 0x7f50c40d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50c3e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50c40d0350 0x7f50c40d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56966/0 (socket says 172.21.15.45:56966) 2024-04-25T10:33:25.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50c3e00640 1 -- 172.21.15.45:0/1135053121 learned_addr learned my addr 172.21.15.45:0/1135053121 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:25.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50c3e00640 1 -- 172.21.15.45:0/1135053121 >> v1:172.21.15.117:6789/0 conn(0x7f50c406aea0 legacy=0x7f50c406b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:25.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50c3e00640 1 -- 172.21.15.45:0/1135053121 >> v1:172.21.15.45:6789/0 conn(0x7f50c415df60 legacy=0x7f50c40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.689+0000 7f50c3e00640 1 -- 172.21.15.45:0/1135053121 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40cd060 con 0x7f50c40d0350 2024-04-25T10:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50c3e00640 1 --2- 172.21.15.45:0/1135053121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c40d07b0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009870 tx=0x7f50b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3e8f5d0b74b86b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50c3400640 1 -- 172.21.15.45:0/1135053121 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00b2070 con 0x7f50c40d0350 2024-04-25T10:33:25.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50c3400640 1 -- 172.21.15.45:0/1135053121 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c406c1d0 con 0x7f50c40d0350 2024-04-25T10:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50c3400640 1 -- 172.21.15.45:0/1135053121 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00ad8a0 con 0x7f50c40d0350 2024-04-25T10:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- 172.21.15.45:0/1135053121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 msgr2=0x7f50c40d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 --2- 172.21.15.45:0/1135053121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c40d07b0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009870 tx=0x7f50b001c490 comp rx=0 tx=0).stop 2024-04-25T10:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- 172.21.15.45:0/1135053121 shutdown_connections 2024-04-25T10:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 --2- 172.21.15.45:0/1135053121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c40d07b0 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- 172.21.15.45:0/1135053121 >> 172.21.15.45:0/1135053121 conn(0x7f50c40d1910 msgr2=0x7f50c40d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- 172.21.15.45:0/1135053121 shutdown_connections 2024-04-25T10:33:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- 172.21.15.45:0/1135053121 wait complete. 2024-04-25T10:33:25.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 Processor -- start 2024-04-25T10:33:25.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- start start 2024-04-25T10:33:25.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c406aea0 0x7f50c41498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c414bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415df60 0x7f50c4159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40d0ff0 con 0x7f50c40d0350 2024-04-25T10:33:25.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40d0e70 con 0x7f50c415df60 2024-04-25T10:33:25.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.693+0000 7f50cb600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c416ba30 con 0x7f50c406aea0 2024-04-25T10:33:25.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c414bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c414bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56968/0 (socket says 172.21.15.45:56968) 2024-04-25T10:33:25.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c8c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c406aea0 0x7f50c41498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 -- 172.21.15.45:0/1434628828 learned_addr learned my addr 172.21.15.45:0/1434628828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c9600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415df60 0x7f50c4159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 -- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c406aea0 msgr2=0x7f50c41498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c406aea0 0x7f50c41498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 -- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415df60 msgr2=0x7f50c4159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415df60 0x7f50c4159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 -- 172.21.15.45:0/1434628828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c414ac80 con 0x7f50c40d0350 2024-04-25T10:33:25.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c3e00640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c414bd20 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f50b001a0c0 tx=0x7f50b001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00b2040 con 0x7f50c40d0350 2024-04-25T10:33:25.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00bd070 con 0x7f50c40d0350 2024-04-25T10:33:25.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b00addc0 con 0x7f50c40d0350 2024-04-25T10:33:25.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c414af60 con 0x7f50c40d0350 2024-04-25T10:33:25.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c414b520 con 0x7f50c40d0350 2024-04-25T10:33:25.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c404ef30 con 0x7f50c40d0350 2024-04-25T10:33:25.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.697+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50b00c2460 con 0x7f50c40d0350 2024-04-25T10:33:25.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.701+0000 7f50c0c00640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50a4066f60 0x7f50a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:25.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.701+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f50b011ef90 con 0x7f50c40d0350 2024-04-25T10:33:25.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.701+0000 7f50c8c00640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50a4066f60 0x7f50a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:25.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.701+0000 7f50c8c00640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50a4066f60 0x7f50a4069420 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f50c40b7690 tx=0x7f50b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:25.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.705+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50b0016110 con 0x7f50c40d0350 2024-04-25T10:33:25.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.861+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50c406c1d0 con 0x7f50a4066f60 2024-04-25T10:33:25.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.861+0000 7f50c0c00640 1 -- 172.21.15.45:0/1434628828 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13493 (secure 0 0 0) 0x7f50c406c1d0 con 0x7f50a4066f60 2024-04-25T10:33:25.866 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:25.867 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:33:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50a4066f60 msgr2=0x7f50a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50a4066f60 0x7f50a4069420 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f50c40b7690 tx=0x7f50b8009040 comp rx=0 tx=0).stop 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 msgr2=0x7f50c414bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c414bd20 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f50b001a0c0 tx=0x7f50b001acd0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 shutdown_connections 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f50a4066f60 0x7f50a4069420 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415df60 0x7f50c4159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c40d0350 0x7f50c414bd20 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 --2- 172.21.15.45:0/1434628828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c406aea0 0x7f50c41498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 >> 172.21.15.45:0/1434628828 conn(0x7f50c40d1910 msgr2=0x7f50c4155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 shutdown_connections 2024-04-25T10:33:25.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:25.865+0000 7f50cb600640 1 -- 172.21.15.45:0/1434628828 wait complete. 2024-04-25T10:33:25.887 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":661,"stamp":"2024-04-25T10:33:25.095898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":475560,"kb_used_data":20776,"kb_used_omap":12,"kb_used_meta":454643,"kb_avail":754499160,"statfs":{"total":773094113280,"available":772607139840,"internally_reserved":0,"allocated":21274624,"data_stored":10016888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465554992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12622614,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.261846"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":513,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-25T10:33:22.524028+0000","last_change":"2024-04-25T10:17:50.626162+0000","last_active":"2024-04-25T10:33:22.524028+0000","last_peered":"2024-04-25T10:33:22.524028+0000","last_clean":"2024-04-25T10:33:22.524028+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:33:22.524028+0000","last_undegraded":"2024-04-25T10:33:22.524028+0000","last_fullsized":"2024-04-25T10:33:22.524028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:49.612839+0000","last_clean_scrub_stamp":"2024-04-25T10:17:49.612839+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:20.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509630,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509630,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64744,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307096,"statfs":{"total":96636764160,"available":96570466304,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509630,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47524,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324316,"statfs":{"total":96636764160,"available":96588099584,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69672,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302168,"statfs":{"total":96636764160,"available":96565420032,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509631,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52264,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319576,"statfs":{"total":96636764160,"available":96583245824,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60004,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542333,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69664,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302176,"statfs":{"total":96636764160,"available":96565428224,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:33:25.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:33:26.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.025+0000 7fe581400640 1 Processor -- start 2024-04-25T10:33:26.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe581400640 1 -- start start 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe581400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c1451d0 0x7fe57c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe581400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe57c0ce070 0x7fe57c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe581400640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c15f5e0 con 0x7fe57c0b2c00 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe581400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0b3060 con 0x7fe57c0ce070 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe581400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c15f8c0 con 0x7fe57c1451d0 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe573400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe57c0ce070 0x7fe57c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe580a00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe57c0b2c00 0x7fe57c0669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53914/0 (socket says 172.21.15.45:53914) 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe580a00640 1 -- 172.21.15.45:0/82544823 learned_addr learned my addr 172.21.15.45:0/82544823 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe57be00640 1 --2- 172.21.15.45:0/82544823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c1451d0 0x7fe57c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe573400640 1 -- 172.21.15.45:0/82544823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c1451d0 msgr2=0x7fe57c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe573400640 1 --2- 172.21.15.45:0/82544823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c1451d0 0x7fe57c0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe573400640 1 -- 172.21.15.45:0/82544823 >> v1:172.21.15.45:6789/0 conn(0x7fe57c0b2c00 legacy=0x7fe57c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe573400640 1 -- 172.21.15.45:0/82544823 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c15fba0 con 0x7fe57c0ce070 2024-04-25T10:33:26.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.029+0000 7fe57b400640 1 -- 172.21.15.45:0/82544823 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260477868 0 0) 0x7fe57c15f5e0 con 0x7fe57c0b2c00 2024-04-25T10:33:26.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe573400640 1 --2- 172.21.15.45:0/82544823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0ce070 0x7fe57c0b26c0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fe5640097c0 tx=0x7fe56401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafe0f2b6f376841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:26.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe57b400640 1 -- 172.21.15.45:0/82544823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640b0070 con 0x7fe57c0ce070 2024-04-25T10:33:26.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe57b400640 1 -- 172.21.15.45:0/82544823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5640ab600 con 0x7fe57c0ce070 2024-04-25T10:33:26.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe57b400640 1 -- 172.21.15.45:0/82544823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640ab9d0 con 0x7fe57c0ce070 2024-04-25T10:33:26.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 -- 172.21.15.45:0/82544823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0ce070 msgr2=0x7fe57c0b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:26.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 --2- 172.21.15.45:0/82544823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0ce070 0x7fe57c0b26c0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fe5640097c0 tx=0x7fe56401c410 comp rx=0 tx=0).stop 2024-04-25T10:33:26.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 -- 172.21.15.45:0/82544823 shutdown_connections 2024-04-25T10:33:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 --2- 172.21.15.45:0/82544823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0ce070 0x7fe57c0b26c0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 --2- 172.21.15.45:0/82544823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c1451d0 0x7fe57c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 -- 172.21.15.45:0/82544823 >> 172.21.15.45:0/82544823 conn(0x7fe57c074310 msgr2=0x7fe57c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 -- 172.21.15.45:0/82544823 shutdown_connections 2024-04-25T10:33:26.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 -- 172.21.15.45:0/82544823 wait complete. 2024-04-25T10:33:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 Processor -- start 2024-04-25T10:33:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.033+0000 7fe581400640 1 -- start start 2024-04-25T10:33:26.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 0x7fe57c15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c0ce070 0x7fe57c13f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c1451d0 0x7fe57c15e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c06ee60 con 0x7fe57c0b2c00 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c06ece0 con 0x7fe57c1451d0 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c06e2b0 con 0x7fe57c0ce070 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 0x7fe57c15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 0x7fe57c15b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56996/0 (socket says 172.21.15.45:56996) 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 -- 172.21.15.45:0/3540234824 learned_addr learned my addr 172.21.15.45:0/3540234824 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe573400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c0ce070 0x7fe57c13f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe580a00640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c1451d0 0x7fe57c15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 -- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c0ce070 msgr2=0x7fe57c13f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c0ce070 0x7fe57c13f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 -- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c1451d0 msgr2=0x7fe57c15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c1451d0 0x7fe57c15e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 -- 172.21.15.45:0/3540234824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c13fab0 con 0x7fe57c0b2c00 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe573400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c0ce070 0x7fe57c13f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe580a00640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c1451d0 0x7fe57c15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe57be00640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 0x7fe57c15b360 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00d5f0 tx=0x7fe56c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0125f0 con 0x7fe57c0b2c00 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c019070 con 0x7fe57c0b2c00 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c13fd90 con 0x7fe57c0b2c00 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0129b0 con 0x7fe57c0b2c00 2024-04-25T10:33:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c141970 con 0x7fe57c0b2c00 2024-04-25T10:33:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c075120 con 0x7fe57c0b2c00 2024-04-25T10:33:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.037+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe56c012b50 con 0x7fe57c0b2c00 2024-04-25T10:33:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.041+0000 7fe578c00640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.041+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe56c0851d0 con 0x7fe57c0b2c00 2024-04-25T10:33:26.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.041+0000 7fe573400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.041+0000 7fe573400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe5640ad000 tx=0x7fe564008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.045+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe56c056470 con 0x7fe57c0b2c00 2024-04-25T10:33:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.205+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe57c15bf80 con 0x7fe57c0b2c00 2024-04-25T10:33:26.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe578c00640 1 -- 172.21.15.45:0/3540234824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+13148 (secure 0 0 0) 0x7fe56c055d70 con 0x7fe57c0b2c00 2024-04-25T10:33:26.212 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:26.212 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":199,"fsid":"d7e12072-0435-4dda-862b-d1045153027e","created":"2024-04-25T10:17:35.819117+0000","modified":"2024-04-25T10:33:21.910684+0000","last_up_change":"2024-04-25T10:17:48.610236+0000","last_in_change":"2024-04-25T10:17:39.269192+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:17:48.947609+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1504397e-8735-47e0-a345-d9d96f7ea66c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6809","nonce":3877436580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6811","nonce":3877436580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6815","nonce":3877436580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3877436580},{"type":"v1","addr":"172.21.15.45:6813","nonce":3877436580}]},"public_addr":"172.21.15.45:6809/3877436580","cluster_addr":"172.21.15.45:6811/3877436580","heartbeat_back_addr":"172.21.15.45:6815/3877436580","heartbeat_front_addr":"172.21.15.45:6813/3877436580","state":["exists","up"]},{"osd":1,"uuid":"53fcceda-1557-4e7c-81ff-f2163ddc2f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6819","nonce":3290058656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6821","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6823","nonce":3290058656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6831","nonce":3290058656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3290058656},{"type":"v1","addr":"172.21.15.45:6828","nonce":3290058656}]},"public_addr":"172.21.15.45:6819/3290058656","cluster_addr":"172.21.15.45:6823/3290058656","heartbeat_back_addr":"172.21.15.45:6831/3290058656","heartbeat_front_addr":"172.21.15.45:6828/3290058656","state":["exists","up"]},{"osd":2,"uuid":"e6b66be6-3e3f-4880-9aec-e3caa98db3fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6817","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6818","nonce":962547941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6822","nonce":962547941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6827","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6829","nonce":962547941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":962547941},{"type":"v1","addr":"172.21.15.45:6825","nonce":962547941}]},"public_addr":"172.21.15.45:6818/962547941","cluster_addr":"172.21.15.45:6822/962547941","heartbeat_back_addr":"172.21.15.45:6829/962547941","heartbeat_front_addr":"172.21.15.45:6825/962547941","state":["exists","up"]},{"osd":3,"uuid":"a62f3c95-4998-4ff8-a6bb-0c3db9caf9b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6801","nonce":3296322295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6803","nonce":3296322295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6807","nonce":3296322295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3296322295},{"type":"v1","addr":"172.21.15.45:6805","nonce":3296322295}]},"public_addr":"172.21.15.45:6801/3296322295","cluster_addr":"172.21.15.45:6803/3296322295","heartbeat_back_addr":"172.21.15.45:6807/3296322295","heartbeat_front_addr":"172.21.15.45:6805/3296322295","state":["exists","up"]},{"osd":4,"uuid":"cdeadede-062f-4fd2-9199-ab0c21659438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6801","nonce":728189779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6805","nonce":728189779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6811","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6813","nonce":728189779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":728189779},{"type":"v1","addr":"172.21.15.117:6809","nonce":728189779}]},"public_addr":"172.21.15.117:6801/728189779","cluster_addr":"172.21.15.117:6805/728189779","heartbeat_back_addr":"172.21.15.117:6813/728189779","heartbeat_front_addr":"172.21.15.117:6809/728189779","state":["exists","up"]},{"osd":5,"uuid":"f0e03339-ad7a-4105-a0c1-f9370002931c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6804","nonce":2444936005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6808","nonce":2444936005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6815","nonce":2444936005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2444936005},{"type":"v1","addr":"172.21.15.117:6812","nonce":2444936005}]},"public_addr":"172.21.15.117:6804/2444936005","cluster_addr":"172.21.15.117:6808/2444936005","heartbeat_back_addr":"172.21.15.117:6815/2444936005","heartbeat_front_addr":"172.21.15.117:6812/2444936005","state":["exists","up"]},{"osd":6,"uuid":"adb80701-c6c3-4d37-9711-1b803f66a674","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6818","nonce":2565393264}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6819","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6821","nonce":2565393264}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6827","nonce":2565393264}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2565393264},{"type":"v1","addr":"172.21.15.117:6824","nonce":2565393264}]},"public_addr":"172.21.15.117:6818/2565393264","cluster_addr":"172.21.15.117:6821/2565393264","heartbeat_back_addr":"172.21.15.117:6827/2565393264","heartbeat_front_addr":"172.21.15.117:6824/2565393264","state":["exists","up"]},{"osd":7,"uuid":"445fb14c-32f4-4101-9d09-e7bcd2e969de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6820","nonce":2004387070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6823","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6825","nonce":2004387070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6831","nonce":2004387070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2004387070},{"type":"v1","addr":"172.21.15.117:6829","nonce":2004387070}]},"public_addr":"172.21.15.117:6820/2004387070","cluster_addr":"172.21.15.117:6825/2004387070","heartbeat_back_addr":"172.21.15.117:6831/2004387070","heartbeat_front_addr":"172.21.15.117:6829/2004387070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.124588+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.932967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.949871+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.968647+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.935283+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:44.981650+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:17:45.103539+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1179754162":"2024-04-26T10:33:18.826877+0000","172.21.15.45:6833/1179754162":"2024-04-26T10:33:18.826877+0000","172.21.15.45:6835/3198476999":"2024-04-26T10:32:06.688741+0000","172.21.15.45:6833/3198476999":"2024-04-26T10:32:06.688741+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:33:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 msgr2=0x7fe558069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe5640ad000 tx=0x7fe564008040 comp rx=0 tx=0).stop 2024-04-25T10:33:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 msgr2=0x7fe57c15b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 0x7fe57c15b360 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00d5f0 tx=0x7fe56c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 shutdown_connections 2024-04-25T10:33:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe558066f90 0x7fe558069450 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57c1451d0 0x7fe57c15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe57c0ce070 0x7fe57c13f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 --2- 172.21.15.45:0/3540234824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe57c0b2c00 0x7fe57c15b360 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 >> 172.21.15.45:0/3540234824 conn(0x7fe57c074310 msgr2=0x7fe57c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 shutdown_connections 2024-04-25T10:33:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:26.209+0000 7fe581400640 1 -- 172.21.15.45:0/3540234824 wait complete. 2024-04-25T10:33:27.233 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:33:27.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:27.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 Processor -- start 2024-04-25T10:33:27.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 -- start start 2024-04-25T10:33:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7800cf040 0x7fc7800cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc7800cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc780071f80 con 0x7fc7800713a0 2024-04-25T10:33:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc780072260 con 0x7fc7800cf040 2024-04-25T10:33:27.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc786600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800b3700 con 0x7fc78015bc00 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc77f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc7800cf040 0x7fc7800cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc7800cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc7800cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54456/0 (socket says 172.21.15.45:54456) 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 -- 172.21.15.45:0/1162617557 learned_addr learned my addr 172.21.15.45:0/1162617557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 -- 172.21.15.45:0/1162617557 >> v2:172.21.15.117:3300/0 conn(0x7fc7800cf040 msgr2=0x7fc7800cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 --2- 172.21.15.45:0/1162617557 >> v2:172.21.15.117:3300/0 conn(0x7fc7800cf040 0x7fc7800cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 -- 172.21.15.45:0/1162617557 >> v1:172.21.15.117:6789/0 conn(0x7fc7800713a0 legacy=0x7fc7800ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.389+0000 7fc776a00640 1 -- 172.21.15.45:0/1162617557 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78015b120 con 0x7fc78015bc00 2024-04-25T10:33:27.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc776a00640 1 --2- 172.21.15.45:0/1162617557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc7800cff20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc768009870 tx=0x7fc76801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bd0d706e87328b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc77ea00640 1 -- 172.21.15.45:0/1162617557 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680ab650 con 0x7fc78015bc00 2024-04-25T10:33:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc77ea00640 1 -- 172.21.15.45:0/1162617557 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7680b0070 con 0x7fc78015bc00 2024-04-25T10:33:27.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc77ea00640 1 -- 172.21.15.45:0/1162617557 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680aba10 con 0x7fc78015bc00 2024-04-25T10:33:27.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 -- 172.21.15.45:0/1162617557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 msgr2=0x7fc7800cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 --2- 172.21.15.45:0/1162617557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc7800cff20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc768009870 tx=0x7fc76801c560 comp rx=0 tx=0).stop 2024-04-25T10:33:27.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 -- 172.21.15.45:0/1162617557 shutdown_connections 2024-04-25T10:33:27.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 --2- 172.21.15.45:0/1162617557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc7800cff20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 --2- 172.21.15.45:0/1162617557 >> v2:172.21.15.117:3300/0 conn(0x7fc7800cf040 0x7fc7800cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 -- 172.21.15.45:0/1162617557 >> 172.21.15.45:0/1162617557 conn(0x7fc780066b70 msgr2=0x7fc780066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:27.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 -- 172.21.15.45:0/1162617557 shutdown_connections 2024-04-25T10:33:27.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.393+0000 7fc786600640 1 -- 172.21.15.45:0/1162617557 wait complete. 2024-04-25T10:33:27.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 Processor -- start 2024-04-25T10:33:27.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 -- start start 2024-04-25T10:33:27.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7800713a0 0x7fc78014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7800cf040 0x7fc7801546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc78014eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800cf5a0 con 0x7fc7800713a0 2024-04-25T10:33:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800cf420 con 0x7fc7800cf040 2024-04-25T10:33:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78015e500 con 0x7fc78015bc00 2024-04-25T10:33:27.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7800713a0 0x7fc78014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7800713a0 0x7fc78014e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57000/0 (socket says 172.21.15.45:57000) 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 -- 172.21.15.45:0/602164122 learned_addr learned my addr 172.21.15.45:0/602164122 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77fe00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc78014eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc776a00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7800cf040 0x7fc7801546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 -- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 msgr2=0x7fc78014eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc78015bc00 0x7fc78014eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 -- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7800cf040 msgr2=0x7fc7801546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7800cf040 0x7fc7801546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 -- 172.21.15.45:0/602164122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78006dfc0 con 0x7fc7800713a0 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc77f400640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7800713a0 0x7fc78014e390 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fc77000e9b0 tx=0x7fc77000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770017070 con 0x7fc7800713a0 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77000c9a0 con 0x7fc7800713a0 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000cd70 con 0x7fc7800713a0 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780156060 con 0x7fc7800713a0 2024-04-25T10:33:27.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.397+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78014e8d0 con 0x7fc7800713a0 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc78004ef30 con 0x7fc7800713a0 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc77001a070 con 0x7fc7800713a0 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc777e00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc770085150 con 0x7fc7800713a0 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc777e00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7fc75c06cec0 0x7fc75c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 --> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc75c06f990 con 0x7fc75c06cec0 2024-04-25T10:33:27.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc77008c050 con 0x7fc7800713a0 2024-04-25T10:33:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc776a00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc77fe00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7fc75c06cec0 0x7fc75c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc776a00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc75c066f60 0x7fc75c069420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc7680af000 tx=0x7fc7680a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.401+0000 7fc77fe00640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7fc75c06cec0 0x7fc75c06f2e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.405+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== osd.0 v2:172.21.15.45:6808/3877436580 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc75c06f990 con 0x7fc75c06cec0 2024-04-25T10:33:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 --> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc7800b2a10 con 0x7fc75c06cec0 2024-04-25T10:33:27.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc777e00640 1 -- 172.21.15.45:0/602164122 <== osd.0 v2:172.21.15.45:6808/3877436580 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc7800b2a10 con 0x7fc75c06cec0 2024-04-25T10:33:27.420 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:27.420 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7fc75c06cec0 msgr2=0x7fc75c06f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:6808/3877436580,v1:172.21.15.45:6809/3877436580] conn(0x7fc75c06cec0 0x7fc75c06f2e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc75c066f60 msgr2=0x7fc75c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fc75c066f60 0x7fc75c069420 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc7680af000 tx=0x7fc7680a7000 comp rx=0 tx=0).stop 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7800713a0 msgr2=0x7fc78014e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 --2- 172.21.15.45:0/602164122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7800713a0 0x7fc78014e390 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fc77000e9b0 tx=0x7fc77000ee70 comp rx=0 tx=0).stop 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc77fe00640 1 -- 172.21.15.45:0/602164122 reap_dead start 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 shutdown_connections 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 >> 172.21.15.45:0/602164122 conn(0x7fc780066b70 msgr2=0x7fc78015d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 shutdown_connections 2024-04-25T10:33:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.417+0000 7fc786600640 1 -- 172.21.15.45:0/602164122 wait complete. 2024-04-25T10:33:27.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:33:27.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.577+0000 7fd81b000640 1 Processor -- start 2024-04-25T10:33:27.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.577+0000 7fd81b000640 1 -- start start 2024-04-25T10:33:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd81b000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd8140cda10 0x7fd81406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd81b000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8140b2770 con 0x7fd8140cf930 2024-04-25T10:33:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd81b000640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81406c670 con 0x7fd81415bc40 2024-04-25T10:33:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd81b000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81406c950 con 0x7fd8140cda10 2024-04-25T10:33:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd819000640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8140cf930 0x7fd8140b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53916/0 (socket says 172.21.15.45:53916) 2024-04-25T10:33:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd819000640 1 -- 172.21.15.45:0/3760156098 learned_addr learned my addr 172.21.15.45:0/3760156098 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd80b400640 1 --2- 172.21.15.45:0/3760156098 >> v2:172.21.15.117:3300/0 conn(0x7fd8140cda10 0x7fd81406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254783109 0 0) 0x7fd8140b2770 con 0x7fd8140cf930 2024-04-25T10:33:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f8003530 con 0x7fd8140cf930 2024-04-25T10:33:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138408896 0 0) 0x7fd81406c670 con 0x7fd81415bc40 2024-04-25T10:33:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8140b2770 con 0x7fd81415bc40 2024-04-25T10:33:27.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3873026849 0 0) 0x7fd7f8003530 con 0x7fd8140cf930 2024-04-25T10:33:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd80b400640 1 -- 172.21.15.45:0/3760156098 >> v1:172.21.15.117:6789/0 conn(0x7fd81415bc40 legacy=0x7fd8140b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd80b400640 1 -- 172.21.15.45:0/3760156098 >> v1:172.21.15.45:6789/0 conn(0x7fd8140cf930 legacy=0x7fd8140b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.581+0000 7fd80b400640 1 -- 172.21.15.45:0/3760156098 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81406cc30 con 0x7fd8140cda10 2024-04-25T10:33:27.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd80b400640 1 --2- 172.21.15.45:0/3760156098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cda10 0x7fd81406c090 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0097c0 tx=0x7fd7fc01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72765e03a855bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0b0070 con 0x7fd8140cda10 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0ab400 con 0x7fd8140cda10 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd813400640 1 -- 172.21.15.45:0/3760156098 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c001a00 con 0x7fd8140cda10 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 -- 172.21.15.45:0/3760156098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cda10 msgr2=0x7fd81406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 --2- 172.21.15.45:0/3760156098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cda10 0x7fd81406c090 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0097c0 tx=0x7fd7fc01c330 comp rx=0 tx=0).stop 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 -- 172.21.15.45:0/3760156098 shutdown_connections 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 --2- 172.21.15.45:0/3760156098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cda10 0x7fd81406c090 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 -- 172.21.15.45:0/3760156098 >> 172.21.15.45:0/3760156098 conn(0x7fd814066b70 msgr2=0x7fd814066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 -- 172.21.15.45:0/3760156098 shutdown_connections 2024-04-25T10:33:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 -- 172.21.15.45:0/3760156098 wait complete. 2024-04-25T10:33:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 Processor -- start 2024-04-25T10:33:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.585+0000 7fd81b000640 1 -- start start 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8140cda10 0x7fd814152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cf930 0x7fd814153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 0x7fd8140af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd814169d70 con 0x7fd81415bc40 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd814169bf0 con 0x7fd8140cf930 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd814169ef0 con 0x7fd8140cda10 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 0x7fd8140af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 0x7fd8140af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57012/0 (socket says 172.21.15.45:57012) 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 -- 172.21.15.45:0/3577907417 learned_addr learned my addr 172.21.15.45:0/3577907417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd80b400640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cf930 0x7fd814153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd813e00640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8140cda10 0x7fd814152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 -- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8140cda10 msgr2=0x7fd814152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8140cda10 0x7fd814152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 -- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cf930 msgr2=0x7fd814153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cf930 0x7fd814153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 -- 172.21.15.45:0/3577907417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8141537d0 con 0x7fd81415bc40 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd80b400640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cf930 0x7fd814153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd813e00640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8140cda10 0x7fd814152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd819000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 0x7fd8140af780 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fd80c001fc0 tx=0x7fd80c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c017070 con 0x7fd81415bc40 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c012400 con 0x7fd81415bc40 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c012830 con 0x7fd81415bc40 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd814153ab0 con 0x7fd81415bc40 2024-04-25T10:33:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd814154070 con 0x7fd81415bc40 2024-04-25T10:33:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81404ef30 con 0x7fd81415bc40 2024-04-25T10:33:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.589+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd80c01a070 con 0x7fd81415bc40 2024-04-25T10:33:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.593+0000 7fd810c00640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd7f0066f90 0x7fd7f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.593+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd80c084db0 con 0x7fd81415bc40 2024-04-25T10:33:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.593+0000 7fd813e00640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd7f0066f90 0x7fd7f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.593+0000 7fd813e00640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd7f0066f90 0x7fd7f0069450 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd8040011f0 tx=0x7fd804009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.597+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd80c089070 con 0x7fd81415bc40 2024-04-25T10:33:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd814154240 con 0x7fd7f0066f90 2024-04-25T10:33:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd810c00640 1 -- 172.21.15.45:0/3577907417 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd804001d10 con 0x7fd7f0066f90 2024-04-25T10:33:27.760 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd7f0066f90 msgr2=0x7fd7f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd7f0066f90 0x7fd7f0069450 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd8040011f0 tx=0x7fd804009040 comp rx=0 tx=0).stop 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 msgr2=0x7fd8140af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 0x7fd8140af780 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fd80c001fc0 tx=0x7fd80c00a430 comp rx=0 tx=0).stop 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 shutdown_connections 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd7f0066f90 0x7fd7f0069450 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd81415bc40 0x7fd8140af780 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8140cf930 0x7fd814153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 --2- 172.21.15.45:0/3577907417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8140cda10 0x7fd814152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.757+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 >> 172.21.15.45:0/3577907417 conn(0x7fd814066b70 msgr2=0x7fd814159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.761+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 shutdown_connections 2024-04-25T10:33:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.761+0000 7fd81b000640 1 -- 172.21.15.45:0/3577907417 wait complete. 2024-04-25T10:33:27.782 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:33:27.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:27.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 Processor -- start 2024-04-25T10:33:27.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 -- start start 2024-04-25T10:33:27.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f712815bd10 0x7f71280b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71280cf9e0 0x7f71280cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712806c0e0 con 0x7f712815c0f0 2024-04-25T10:33:27.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71280b2ef0 con 0x7f71280cf9e0 2024-04-25T10:33:27.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712f400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71280d0350 con 0x7f712815bd10 2024-04-25T10:33:27.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71280cf9e0 0x7f71280cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f71280cf9e0 0x7f71280cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57028/0 (socket says 172.21.15.45:57028) 2024-04-25T10:33:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f712815bd10 0x7f71280b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 -- 172.21.15.45:0/1546371057 learned_addr learned my addr 172.21.15.45:0/1546371057 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 -- 172.21.15.45:0/1546371057 >> v2:172.21.15.117:3300/0 conn(0x7f712815bd10 msgr2=0x7f71280b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 --2- 172.21.15.45:0/1546371057 >> v2:172.21.15.117:3300/0 conn(0x7f712815bd10 0x7f71280b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 -- 172.21.15.45:0/1546371057 >> v1:172.21.15.117:6789/0 conn(0x7f712815c0f0 legacy=0x7f71280b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.925+0000 7f712d400640 1 -- 172.21.15.45:0/1546371057 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712815c6a0 con 0x7f71280cf9e0 2024-04-25T10:33:27.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712d400640 1 --2- 172.21.15.45:0/1546371057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71280cf9e0 0x7f71280cfe10 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f7128072810 tx=0x7f712001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9e28a8b489aea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f7127400640 1 -- 172.21.15.45:0/1546371057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200a5880 con 0x7f71280cf9e0 2024-04-25T10:33:27.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f7127400640 1 -- 172.21.15.45:0/1546371057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71200a5a20 con 0x7f71280cf9e0 2024-04-25T10:33:27.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f7127400640 1 -- 172.21.15.45:0/1546371057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200a5df0 con 0x7f71280cf9e0 2024-04-25T10:33:27.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 -- 172.21.15.45:0/1546371057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71280cf9e0 msgr2=0x7f71280cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 --2- 172.21.15.45:0/1546371057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71280cf9e0 0x7f71280cfe10 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f7128072810 tx=0x7f712001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 -- 172.21.15.45:0/1546371057 shutdown_connections 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 --2- 172.21.15.45:0/1546371057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71280cf9e0 0x7f71280cfe10 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 --2- 172.21.15.45:0/1546371057 >> v2:172.21.15.117:3300/0 conn(0x7f712815bd10 0x7f71280b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 -- 172.21.15.45:0/1546371057 >> 172.21.15.45:0/1546371057 conn(0x7f7128066b70 msgr2=0x7f7128066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 -- 172.21.15.45:0/1546371057 shutdown_connections 2024-04-25T10:33:27.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.929+0000 7f712f400640 1 -- 172.21.15.45:0/1546371057 wait complete. 2024-04-25T10:33:27.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 Processor -- start 2024-04-25T10:33:27.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- start start 2024-04-25T10:33:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71280cf9e0 0x7f712815e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f712815bd10 0x7f7128157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f712815c0f0 0x7f71281526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280b4cd0 con 0x7f712815c0f0 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71280b4b50 con 0x7f712815bd10 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712806cdd0 con 0x7f71280cf9e0 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f712815c0f0 0x7f71281526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f712815c0f0 0x7f71281526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57044/0 (socket says 172.21.15.45:57044) 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 -- 172.21.15.45:0/3153197138 learned_addr learned my addr 172.21.15.45:0/3153197138 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f7127e00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f712815bd10 0x7f7128157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712ca00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71280cf9e0 0x7f712815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 -- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71280cf9e0 msgr2=0x7f712815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71280cf9e0 0x7f712815e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 -- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f712815bd10 msgr2=0x7f7128157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f712815bd10 0x7f7128157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 -- 172.21.15.45:0/3153197138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128158340 con 0x7f712815c0f0 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f7127e00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f712815bd10 0x7f7128157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712d400640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f712815c0f0 0x7f71281526e0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f71200195d0 tx=0x7f7120019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200b8670 con 0x7f712815c0f0 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71200be070 con 0x7f712815c0f0 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200b8a50 con 0x7f712815c0f0 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7128158060 con 0x7f712815c0f0 2024-04-25T10:33:27.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7128154aa0 con 0x7f712815c0f0 2024-04-25T10:33:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.933+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f712804ef30 con 0x7f712815c0f0 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71200b8bf0 con 0x7f712815c0f0 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7108066f60 0x7f7108069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f712011fff0 con 0x7f712815c0f0 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7f710806cec0 0x7f710806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 --> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71200a5a20 con 0x7f710806cec0 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71201225d0 con 0x7f712815c0f0 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f712ca00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7108066f60 0x7f7108069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7127e00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7f710806cec0 0x7f710806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f712ca00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7108066f60 0x7f7108069420 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f711c009af0 tx=0x7f711c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7127e00640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7f710806cec0 0x7f710806f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.937+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== osd.1 v2:172.21.15.45:6816/3290058656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f71200a5a20 con 0x7f710806cec0 2024-04-25T10:33:27.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 --> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f712806d9b0 con 0x7f710806cec0 2024-04-25T10:33:27.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f7124c00640 1 -- 172.21.15.45:0/3153197138 <== osd.1 v2:172.21.15.45:6816/3290058656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f712806d9b0 con 0x7f710806cec0 2024-04-25T10:33:27.956 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:27.956 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:27.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7f710806cec0 msgr2=0x7f710806f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:6816/3290058656,v1:172.21.15.45:6819/3290058656] conn(0x7f710806cec0 0x7f710806f2e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:27.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7108066f60 msgr2=0x7f7108069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f7108066f60 0x7f7108069420 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f711c009af0 tx=0x7f711c008040 comp rx=0 tx=0).stop 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f712815c0f0 msgr2=0x7f71281526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 --2- 172.21.15.45:0/3153197138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f712815c0f0 0x7f71281526e0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f71200195d0 tx=0x7f7120019760 comp rx=0 tx=0).stop 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712d400640 1 -- 172.21.15.45:0/3153197138 reap_dead start 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 shutdown_connections 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 >> 172.21.15.45:0/3153197138 conn(0x7f7128066b70 msgr2=0x7f71280b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 shutdown_connections 2024-04-25T10:33:27.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:27.953+0000 7f712f400640 1 -- 172.21.15.45:0/3153197138 wait complete. 2024-04-25T10:33:27.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:33:28.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.113+0000 7f1990600640 1 Processor -- start 2024-04-25T10:33:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.113+0000 7f1990600640 1 -- start start 2024-04-25T10:33:28.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.113+0000 7f1990600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880d0350 0x7f19880d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.113+0000 7f1990600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f198806aea0 0x7f198806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.113+0000 7f1990600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19880ccaa0 con 0x7f198815df60 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f1990600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198815e340 con 0x7f198806aea0 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f1990600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880ccd80 con 0x7f19880d0350 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198dc00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f198815df60 0x7f19880cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53918/0 (socket says 172.21.15.45:53918) 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198dc00640 1 -- 172.21.15.45:0/1120522128 learned_addr learned my addr 172.21.15.45:0/1120522128 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198e600640 1 --2- 172.21.15.45:0/1120522128 >> v2:172.21.15.117:3300/0 conn(0x7f198806aea0 0x7f198806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198d200640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880d0350 0x7f19880d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f1987e00640 1 -- 172.21.15.45:0/1120522128 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201363858 0 0) 0x7f19880ccaa0 con 0x7f198815df60 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f1987e00640 1 -- 172.21.15.45:0/1120522128 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1970003530 con 0x7f198815df60 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198e600640 1 -- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880d0350 msgr2=0x7f19880d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198e600640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880d0350 0x7f19880d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198e600640 1 -- 172.21.15.45:0/1120522128 >> v1:172.21.15.45:6789/0 conn(0x7f198815df60 legacy=0x7f19880cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198e600640 1 -- 172.21.15.45:0/1120522128 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880cd060 con 0x7f198806aea0 2024-04-25T10:33:28.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.117+0000 7f198d200640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880d0350 0x7f19880d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:28.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f198e600640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f198806aea0 0x7f198806b310 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1980009d90 tx=0x7f198001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eead01c185f1f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1987e00640 1 -- 172.21.15.45:0/1120522128 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800b2070 con 0x7f198806aea0 2024-04-25T10:33:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1987e00640 1 -- 172.21.15.45:0/1120522128 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19800a5890 con 0x7f198806aea0 2024-04-25T10:33:28.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1987e00640 1 -- 172.21.15.45:0/1120522128 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800a5c60 con 0x7f198806aea0 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f198806aea0 msgr2=0x7f198806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f198806aea0 0x7f198806b310 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1980009d90 tx=0x7f198001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- 172.21.15.45:0/1120522128 shutdown_connections 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f198806aea0 0x7f198806b310 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 --2- 172.21.15.45:0/1120522128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19880d0350 0x7f19880d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- 172.21.15.45:0/1120522128 >> 172.21.15.45:0/1120522128 conn(0x7f19880d1910 msgr2=0x7f19880d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:28.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- 172.21.15.45:0/1120522128 shutdown_connections 2024-04-25T10:33:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- 172.21.15.45:0/1120522128 wait complete. 2024-04-25T10:33:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 Processor -- start 2024-04-25T10:33:28.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- start start 2024-04-25T10:33:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f198806aea0 0x7f19881605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880d0350 0x7f198815cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 0x7f198815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880d0ff0 con 0x7f198815df60 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19880d0e70 con 0x7f19880d0350 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f1990600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198816aa20 con 0x7f198806aea0 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f198e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 0x7f198815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f198e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 0x7f198815b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57060/0 (socket says 172.21.15.45:57060) 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f198e600640 1 -- 172.21.15.45:0/2764430579 learned_addr learned my addr 172.21.15.45:0/2764430579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f198dc00640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f198806aea0 0x7f19881605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.121+0000 7f198d200640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880d0350 0x7f198815cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f198e600640 1 -- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f198806aea0 msgr2=0x7f19881605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f198e600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f198806aea0 0x7f19881605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f198e600640 1 -- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880d0350 msgr2=0x7f198815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f198e600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880d0350 0x7f198815cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f198e600640 1 -- 172.21.15.45:0/2764430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880cc7a0 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f198e600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 0x7f198815b170 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f1980009d90 tx=0x7f1980019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800b2050 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19800be070 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988160b00 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800ad870 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1988161060 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198804ef30 con 0x7f198815df60 2024-04-25T10:33:28.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19800ada10 con 0x7f198815df60 2024-04-25T10:33:28.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1985600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1968066f60 0x7f1968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.125+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f198011fd30 con 0x7f198815df60 2024-04-25T10:33:28.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.129+0000 7f198dc00640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1968066f60 0x7f1968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.129+0000 7f198dc00640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1968066f60 0x7f1968069420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f197c007fd0 tx=0x7f197c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.133+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1988160b00 con 0x7f198815df60 2024-04-25T10:33:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.293+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f19880cbcd0 con 0x7f1968066f60 2024-04-25T10:33:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.293+0000 7f1985600640 1 -- 172.21.15.45:0/2764430579 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f19880cbcd0 con 0x7f1968066f60 2024-04-25T10:33:28.297 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1968066f60 msgr2=0x7f1968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1968066f60 0x7f1968069420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f197c007fd0 tx=0x7f197c009040 comp rx=0 tx=0).stop 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 msgr2=0x7f198815b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 0x7f198815b170 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f1980009d90 tx=0x7f1980019680 comp rx=0 tx=0).stop 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 shutdown_connections 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f1968066f60 0x7f1968069420 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f198815df60 0x7f198815b170 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19880d0350 0x7f198815cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 --2- 172.21.15.45:0/2764430579 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f198806aea0 0x7f19881605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 >> 172.21.15.45:0/2764430579 conn(0x7f19880d1910 msgr2=0x7f1988068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 shutdown_connections 2024-04-25T10:33:28.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.297+0000 7f1990600640 1 -- 172.21.15.45:0/2764430579 wait complete. 2024-04-25T10:33:28.319 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:33:28.319 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 Processor -- start 2024-04-25T10:33:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 -- start start 2024-04-25T10:33:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf400720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf40070ba0 0x7faf4015b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4015b7a0 con 0x7faf400b2af0 2024-04-25T10:33:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf400af150 con 0x7faf40070ba0 2024-04-25T10:33:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf44a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf40071040 con 0x7faf40157cb0 2024-04-25T10:33:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3e800640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faf400b2af0 0x7faf400b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53926/0 (socket says 172.21.15.45:53926) 2024-04-25T10:33:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3e800640 1 -- 172.21.15.45:0/1480767885 learned_addr learned my addr 172.21.15.45:0/1480767885 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3de00640 1 --2- 172.21.15.45:0/1480767885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf400720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf35400640 1 --2- 172.21.15.45:0/1480767885 >> v2:172.21.15.117:3300/0 conn(0x7faf40070ba0 0x7faf4015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3d400640 1 -- 172.21.15.45:0/1480767885 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142349767 0 0) 0x7faf38001120 con 0x7faf400b2af0 2024-04-25T10:33:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3d400640 1 -- 172.21.15.45:0/1480767885 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003530 con 0x7faf400b2af0 2024-04-25T10:33:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3de00640 1 -- 172.21.15.45:0/1480767885 >> v2:172.21.15.117:3300/0 conn(0x7faf40070ba0 msgr2=0x7faf4015b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3de00640 1 --2- 172.21.15.45:0/1480767885 >> v2:172.21.15.117:3300/0 conn(0x7faf40070ba0 0x7faf4015b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3de00640 1 -- 172.21.15.45:0/1480767885 >> v1:172.21.15.45:6789/0 conn(0x7faf400b2af0 legacy=0x7faf400b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf3de00640 1 -- 172.21.15.45:0/1480767885 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40071320 con 0x7faf40157cb0 2024-04-25T10:33:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.461+0000 7faf35400640 1 --2- 172.21.15.45:0/1480767885 >> v2:172.21.15.117:3300/0 conn(0x7faf40070ba0 0x7faf4015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf3de00640 1 --2- 172.21.15.45:0/1480767885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf400720e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf300097c0 tx=0x7faf3001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39869c4082deea73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf3d400640 1 -- 172.21.15.45:0/1480767885 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300b0070 con 0x7faf40157cb0 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf3d400640 1 -- 172.21.15.45:0/1480767885 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300ab600 con 0x7faf40157cb0 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf3d400640 1 -- 172.21.15.45:0/1480767885 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300ab9d0 con 0x7faf40157cb0 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 -- 172.21.15.45:0/1480767885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 msgr2=0x7faf400720e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 --2- 172.21.15.45:0/1480767885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf400720e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf300097c0 tx=0x7faf3001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 -- 172.21.15.45:0/1480767885 shutdown_connections 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 --2- 172.21.15.45:0/1480767885 >> v2:172.21.15.117:3300/0 conn(0x7faf40070ba0 0x7faf4015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 --2- 172.21.15.45:0/1480767885 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf400720e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 -- 172.21.15.45:0/1480767885 >> 172.21.15.45:0/1480767885 conn(0x7faf40066b70 msgr2=0x7faf40066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 -- 172.21.15.45:0/1480767885 shutdown_connections 2024-04-25T10:33:28.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.465+0000 7faf44a00640 1 -- 172.21.15.45:0/1480767885 wait complete. 2024-04-25T10:33:28.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 Processor -- start 2024-04-25T10:33:28.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 -- start start 2024-04-25T10:33:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf40070ba0 0x7faf4013a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400b2af0 0x7faf4013c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf4015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf40158210 con 0x7faf400b2af0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf40158090 con 0x7faf40070ba0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4006c0e0 con 0x7faf40157cb0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400b2af0 0x7faf4013c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400b2af0 0x7faf4013c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57076/0 (socket says 172.21.15.45:57076) 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 -- 172.21.15.45:0/2235605669 learned_addr learned my addr 172.21.15.45:0/2235605669 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf3e800640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf4015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf3de00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf40070ba0 0x7faf4013a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 -- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 msgr2=0x7faf4015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf40157cb0 0x7faf4015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 -- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf40070ba0 msgr2=0x7faf4013a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf40070ba0 0x7faf4013a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 -- 172.21.15.45:0/2235605669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4013d480 con 0x7faf400b2af0 2024-04-25T10:33:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf35400640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400b2af0 0x7faf4013c9f0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28018020 con 0x7faf400b2af0 2024-04-25T10:33:28.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28014680 con 0x7faf400b2af0 2024-04-25T10:33:28.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4013d760 con 0x7faf400b2af0 2024-04-25T10:33:28.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28014a50 con 0x7faf400b2af0 2024-04-25T10:33:28.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.469+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4013ae00 con 0x7faf400b2af0 2024-04-25T10:33:28.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf4013d9a0 con 0x7faf400b2af0 2024-04-25T10:33:28.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf2801b020 con 0x7faf400b2af0 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf36a00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f90 0x7faf1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7faf28086490 con 0x7faf400b2af0 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf36a00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7faf1c06cef0 0x7faf1c06f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 --> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf28009a50 con 0x7faf1c06cef0 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf28086840 con 0x7faf400b2af0 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf3de00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f90 0x7faf1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf3e800640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7faf1c06cef0 0x7faf1c06f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf3e800640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7faf1c06cef0 0x7faf1c06f310 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.473+0000 7faf3de00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f90 0x7faf1c069450 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7faf300ad000 tx=0x7faf300a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.477+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== osd.2 v2:172.21.15.45:6817/962547941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faf28009a50 con 0x7faf1c06cef0 2024-04-25T10:33:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 --> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faf4004ef30 con 0x7faf1c06cef0 2024-04-25T10:33:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf36a00640 1 -- 172.21.15.45:0/2235605669 <== osd.2 v2:172.21.15.45:6817/962547941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7faf4004ef30 con 0x7faf1c06cef0 2024-04-25T10:33:28.492 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:28.492 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:28.492 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:28.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7faf1c06cef0 msgr2=0x7faf1c06f310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:6817/962547941,v1:172.21.15.45:6818/962547941] conn(0x7faf1c06cef0 0x7faf1c06f310 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f90 msgr2=0x7faf1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf1c066f90 0x7faf1c069450 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7faf300ad000 tx=0x7faf300a6000 comp rx=0 tx=0).stop 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400b2af0 msgr2=0x7faf4013c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 --2- 172.21.15.45:0/2235605669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf400b2af0 0x7faf4013c9f0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).stop 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf3e800640 1 -- 172.21.15.45:0/2235605669 reap_dead start 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 shutdown_connections 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 >> 172.21.15.45:0/2235605669 conn(0x7faf40066b70 msgr2=0x7faf4006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 shutdown_connections 2024-04-25T10:33:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.489+0000 7faf44a00640 1 -- 172.21.15.45:0/2235605669 wait complete. 2024-04-25T10:33:28.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:33:28.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 Processor -- start 2024-04-25T10:33:28.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 -- start start 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa40c15f180 0x7fa40c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa40c0d0630 0x7fa40c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c0ae610 con 0x7fa40c0aedd0 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0d27b0 con 0x7fa40c15f180 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.649+0000 7fa414600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0cf570 con 0x7fa40c0d0630 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa40c15f180 0x7fa40c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa412600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa40c0aedd0 0x7fa40c0af200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53936/0 (socket says 172.21.15.45:53936) 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa412600640 1 -- 172.21.15.45:0/2684690170 learned_addr learned my addr 172.21.15.45:0/2684690170 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa40c15f180 0x7fa40c0ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57086/0 (socket says 172.21.15.45:57086) 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411200640 1 --2- 172.21.15.45:0/2684690170 >> v2:172.21.15.117:3300/0 conn(0x7fa40c0d0630 0x7fa40c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 -- 172.21.15.45:0/2684690170 >> v2:172.21.15.117:3300/0 conn(0x7fa40c0d0630 msgr2=0x7fa40c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 --2- 172.21.15.45:0/2684690170 >> v2:172.21.15.117:3300/0 conn(0x7fa40c0d0630 0x7fa40c0ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 -- 172.21.15.45:0/2684690170 >> v1:172.21.15.45:6789/0 conn(0x7fa40c0aedd0 legacy=0x7fa40c0af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 -- 172.21.15.45:0/2684690170 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0cf850 con 0x7fa40c15f180 2024-04-25T10:33:28.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa40be00640 1 -- 172.21.15.45:0/2684690170 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267829542 0 0) 0x7fa40c0ae610 con 0x7fa40c0aedd0 2024-04-25T10:33:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa411c00640 1 --2- 172.21.15.45:0/2684690170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c15f180 0x7fa40c0ce2f0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097c0 tx=0x7fa40001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a0afae2e5b8b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa40be00640 1 -- 172.21.15.45:0/2684690170 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000b1070 con 0x7fa40c15f180 2024-04-25T10:33:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa40be00640 1 -- 172.21.15.45:0/2684690170 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4000ac620 con 0x7fa40c15f180 2024-04-25T10:33:28.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.653+0000 7fa40be00640 1 -- 172.21.15.45:0/2684690170 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000ac9f0 con 0x7fa40c15f180 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- 172.21.15.45:0/2684690170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c15f180 msgr2=0x7fa40c0ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 --2- 172.21.15.45:0/2684690170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c15f180 0x7fa40c0ce2f0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097c0 tx=0x7fa40001c3a0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- 172.21.15.45:0/2684690170 shutdown_connections 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 --2- 172.21.15.45:0/2684690170 >> v2:172.21.15.117:3300/0 conn(0x7fa40c0d0630 0x7fa40c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 --2- 172.21.15.45:0/2684690170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c15f180 0x7fa40c0ce2f0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- 172.21.15.45:0/2684690170 >> 172.21.15.45:0/2684690170 conn(0x7fa40c0d1900 msgr2=0x7fa40c0d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- 172.21.15.45:0/2684690170 shutdown_connections 2024-04-25T10:33:28.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- 172.21.15.45:0/2684690170 wait complete. 2024-04-25T10:33:28.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 Processor -- start 2024-04-25T10:33:28.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- start start 2024-04-25T10:33:28.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 0x7fa40c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa40c0d0630 0x7fa40c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa40c15f180 0x7fa40c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0d2570 con 0x7fa40c0aedd0 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0d23f0 con 0x7fa40c0d0630 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa414600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1688a0 con 0x7fa40c15f180 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 0x7fa40c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 0x7fa40c0c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57088/0 (socket says 172.21.15.45:57088) 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 -- 172.21.15.45:0/827807466 learned_addr learned my addr 172.21.15.45:0/827807466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa412600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa40c15f180 0x7fa40c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411200640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa40c0d0630 0x7fa40c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 -- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa40c15f180 msgr2=0x7fa40c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa40c15f180 0x7fa40c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 -- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa40c0d0630 msgr2=0x7fa40c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa40c0d0630 0x7fa40c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 -- 172.21.15.45:0/827807466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c1572e0 con 0x7fa40c0aedd0 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.657+0000 7fa411c00640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 0x7fa40c0c89d0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097c0 tx=0x7fa400019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000b9020 con 0x7fa40c0aedd0 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4000b5410 con 0x7fa40c0aedd0 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000b1040 con 0x7fa40c0aedd0 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c1575c0 con 0x7fa40c0aedd0 2024-04-25T10:33:28.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c157b80 con 0x7fa40c0aedd0 2024-04-25T10:33:28.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c04ef30 con 0x7fa40c0aedd0 2024-04-25T10:33:28.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4000bc020 con 0x7fa40c0aedd0 2024-04-25T10:33:28.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa409600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa3ec066f60 0x7fa3ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:28.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa40011ea80 con 0x7fa40c0aedd0 2024-04-25T10:33:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.661+0000 7fa411200640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa3ec066f60 0x7fa3ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.665+0000 7fa411200640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa3ec066f60 0x7fa3ec069420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa40c15c550 tx=0x7fa3f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.665+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa400120e20 con 0x7fa40c0aedd0 2024-04-25T10:33:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa40c0c6b60 con 0x7fa3ec066f60 2024-04-25T10:33:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa409600640 1 -- 172.21.15.45:0/827807466 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa40c0c6b60 con 0x7fa3ec066f60 2024-04-25T10:33:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa3ec066f60 msgr2=0x7fa3ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa3ec066f60 0x7fa3ec069420 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa40c15c550 tx=0x7fa3f8008040 comp rx=0 tx=0).stop 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 msgr2=0x7fa40c0c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 0x7fa40c0c89d0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097c0 tx=0x7fa400019140 comp rx=0 tx=0).stop 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 shutdown_connections 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fa3ec066f60 0x7fa3ec069420 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa40c15f180 0x7fa40c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa40c0d0630 0x7fa40c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 --2- 172.21.15.45:0/827807466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa40c0aedd0 0x7fa40c0c89d0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 >> 172.21.15.45:0/827807466 conn(0x7fa40c0d1900 msgr2=0x7fa40c06c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 shutdown_connections 2024-04-25T10:33:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:28.829+0000 7fa414600640 1 -- 172.21.15.45:0/827807466 wait complete. 2024-04-25T10:33:28.856 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:33:28.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1737c00640 1 Processor -- start 2024-04-25T10:33:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1737c00640 1 -- start start 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1737c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f17300b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1737c00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173006c0e0 con 0x7f17300cf9e0 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1737c00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173015c6a0 con 0x7f173015bd10 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1737c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173015c980 con 0x7f173015c0f0 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1735c00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f17300cf9e0 0x7f173006c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53948/0 (socket says 172.21.15.45:53948) 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1735c00640 1 -- 172.21.15.45:0/3136672613 learned_addr learned my addr 172.21.15.45:0/3136672613 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1727e00640 1 --2- 172.21.15.45:0/3136672613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f17300b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639361851 0 0) 0x7f173006c0e0 con 0x7f17300cf9e0 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1714003530 con 0x7f17300cf9e0 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329948070 0 0) 0x7f173015c6a0 con 0x7f173015bd10 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f173006c0e0 con 0x7f173015bd10 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1727e00640 1 -- 172.21.15.45:0/3136672613 >> v1:172.21.15.117:6789/0 conn(0x7f173015bd10 legacy=0x7f17300b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1727e00640 1 -- 172.21.15.45:0/3136672613 >> v1:172.21.15.45:6789/0 conn(0x7f17300cf9e0 legacy=0x7f173006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.001+0000 7f1727e00640 1 -- 172.21.15.45:0/3136672613 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173015cc60 con 0x7f173015c0f0 2024-04-25T10:33:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1727e00640 1 --2- 172.21.15.45:0/3136672613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f17300b27e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f17180097f0 tx=0x7f171801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc4052ecf50aa182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180ab5f0 con 0x7f173015c0f0 2024-04-25T10:33:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17180b0070 con 0x7f173015c0f0 2024-04-25T10:33:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f172fe00640 1 -- 172.21.15.45:0/3136672613 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180ab9b0 con 0x7f173015c0f0 2024-04-25T10:33:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 -- 172.21.15.45:0/3136672613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 msgr2=0x7f17300b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 --2- 172.21.15.45:0/3136672613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f17300b27e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f17180097f0 tx=0x7f171801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 -- 172.21.15.45:0/3136672613 shutdown_connections 2024-04-25T10:33:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 --2- 172.21.15.45:0/3136672613 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f17300b27e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 -- 172.21.15.45:0/3136672613 >> 172.21.15.45:0/3136672613 conn(0x7f1730066b70 msgr2=0x7f1730066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 -- 172.21.15.45:0/3136672613 shutdown_connections 2024-04-25T10:33:29.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.005+0000 7f1737c00640 1 -- 172.21.15.45:0/3136672613 wait complete. 2024-04-25T10:33:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 Processor -- start 2024-04-25T10:33:29.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 -- start start 2024-04-25T10:33:29.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300cf9e0 0x7f1730158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173015bd10 0x7f1730154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f173014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173016a460 con 0x7f17300cf9e0 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173016a2e0 con 0x7f173015bd10 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173016a5e0 con 0x7f173015c0f0 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300cf9e0 0x7f1730158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300cf9e0 0x7f1730158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57100/0 (socket says 172.21.15.45:57100) 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 -- 172.21.15.45:0/3656974879 learned_addr learned my addr 172.21.15.45:0/3656974879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1727e00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173015bd10 0x7f1730154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735c00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f173014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 -- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 msgr2=0x7f173014c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f173015c0f0 0x7f173014c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 -- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173015bd10 msgr2=0x7f1730154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f173015bd10 0x7f1730154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 -- 172.21.15.45:0/3656974879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173015e060 con 0x7f17300cf9e0 2024-04-25T10:33:29.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1735200640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300cf9e0 0x7f1730158ba0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f172000c490 tx=0x7f172000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17200052d0 con 0x7f17300cf9e0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720005470 con 0x7f17300cf9e0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720012870 con 0x7f17300cf9e0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.009+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1730071f20 con 0x7f17300cf9e0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173014cc70 con 0x7f17300cf9e0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f173015e2a0 con 0x7f17300cf9e0 2024-04-25T10:33:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1720012a10 con 0x7f17300cf9e0 2024-04-25T10:33:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f172d600640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f170c066f60 0x7f170c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1720084940 con 0x7f17300cf9e0 2024-04-25T10:33:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f172d600640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f170c06cec0 0x7f170c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 --> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1720020e50 con 0x7f170c06cec0 2024-04-25T10:33:29.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17200892a0 con 0x7f17300cf9e0 2024-04-25T10:33:29.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f1727e00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f170c066f60 0x7f170c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f1735c00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f170c06cec0 0x7f170c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f1735c00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f170c06cec0 0x7f170c06f2e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.013+0000 7f1727e00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f170c066f60 0x7f170c069420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f17180af000 tx=0x7f17180a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.017+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== osd.3 v2:172.21.15.45:6800/3296322295 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1720020e50 con 0x7f170c06cec0 2024-04-25T10:33:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 --> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f173004ef30 con 0x7f170c06cec0 2024-04-25T10:33:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f172d600640 1 -- 172.21.15.45:0/3656974879 <== osd.3 v2:172.21.15.45:6800/3296322295 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f173004ef30 con 0x7f170c06cec0 2024-04-25T10:33:29.033 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:29.033 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:29.033 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f170c06cec0 msgr2=0x7f170c06f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:6800/3296322295,v1:172.21.15.45:6801/3296322295] conn(0x7f170c06cec0 0x7f170c06f2e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f170c066f60 msgr2=0x7f170c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f170c066f60 0x7f170c069420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f17180af000 tx=0x7f17180a7000 comp rx=0 tx=0).stop 2024-04-25T10:33:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300cf9e0 msgr2=0x7f1730158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 --2- 172.21.15.45:0/3656974879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17300cf9e0 0x7f1730158ba0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f172000c490 tx=0x7f172000c950 comp rx=0 tx=0).stop 2024-04-25T10:33:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1735c00640 1 -- 172.21.15.45:0/3656974879 reap_dead start 2024-04-25T10:33:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 shutdown_connections 2024-04-25T10:33:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 >> 172.21.15.45:0/3656974879 conn(0x7f1730066b70 msgr2=0x7f17300cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 shutdown_connections 2024-04-25T10:33:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.029+0000 7f1737c00640 1 -- 172.21.15.45:0/3656974879 wait complete. 2024-04-25T10:33:29.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:33:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 Processor -- start 2024-04-25T10:33:29.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 -- start start 2024-04-25T10:33:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff4e8066fe0 0x7ff4e815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4e80b5560 0x7ff4e8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e80b5970 con 0x7ff4e8066c00 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e80b5c20 con 0x7ff4e80b5560 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.189+0000 7ff4ed200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e80af6e0 con 0x7ff4e8066fe0 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4e80b5560 0x7ff4e8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e5e00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff4e8066c00 0x7ff4e806ed30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53952/0 (socket says 172.21.15.45:53952) 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4e80b5560 0x7ff4e8157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57112/0 (socket says 172.21.15.45:57112) 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e5e00640 1 -- 172.21.15.45:0/2315362263 learned_addr learned my addr 172.21.15.45:0/2315362263 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:29.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4dd400640 1 --2- 172.21.15.45:0/2315362263 >> v2:172.21.15.117:3300/0 conn(0x7ff4e8066fe0 0x7ff4e815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 -- 172.21.15.45:0/2315362263 >> v2:172.21.15.117:3300/0 conn(0x7ff4e8066fe0 msgr2=0x7ff4e815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:29.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 --2- 172.21.15.45:0/2315362263 >> v2:172.21.15.117:3300/0 conn(0x7ff4e8066fe0 0x7ff4e815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 -- 172.21.15.45:0/2315362263 >> v1:172.21.15.45:6789/0 conn(0x7ff4e8066c00 legacy=0x7ff4e806ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 -- 172.21.15.45:0/2315362263 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80af9c0 con 0x7ff4e80b5560 2024-04-25T10:33:29.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e6800640 1 --2- 172.21.15.45:0/2315362263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e80b5560 0x7ff4e8157b50 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7ff4e806e850 tx=0x7ff4e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5fef945f56ac6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e5400640 1 -- 172.21.15.45:0/2315362263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00ab650 con 0x7ff4e80b5560 2024-04-25T10:33:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e5400640 1 -- 172.21.15.45:0/2315362263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00b0070 con 0x7ff4e80b5560 2024-04-25T10:33:29.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.193+0000 7ff4e5400640 1 -- 172.21.15.45:0/2315362263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00aba10 con 0x7ff4e80b5560 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- 172.21.15.45:0/2315362263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e80b5560 msgr2=0x7ff4e8157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 --2- 172.21.15.45:0/2315362263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e80b5560 0x7ff4e8157b50 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7ff4e806e850 tx=0x7ff4e001c450 comp rx=0 tx=0).stop 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- 172.21.15.45:0/2315362263 shutdown_connections 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 --2- 172.21.15.45:0/2315362263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e80b5560 0x7ff4e8157b50 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 --2- 172.21.15.45:0/2315362263 >> v2:172.21.15.117:3300/0 conn(0x7ff4e8066fe0 0x7ff4e815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- 172.21.15.45:0/2315362263 >> 172.21.15.45:0/2315362263 conn(0x7ff4e80b24f0 msgr2=0x7ff4e80b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- 172.21.15.45:0/2315362263 shutdown_connections 2024-04-25T10:33:29.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- 172.21.15.45:0/2315362263 wait complete. 2024-04-25T10:33:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 Processor -- start 2024-04-25T10:33:29.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- start start 2024-04-25T10:33:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4e8066c00 0x7ff4e81443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 0x7ff4e81fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4e80b5560 0x7ff4e81faa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80af1e0 con 0x7ff4e8066fe0 2024-04-25T10:33:29.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80af060 con 0x7ff4e8066c00 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4ed200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e806ac90 con 0x7ff4e80b5560 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 0x7ff4e81fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 0x7ff4e81fa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57124/0 (socket says 172.21.15.45:57124) 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 -- 172.21.15.45:0/951547873 learned_addr learned my addr 172.21.15.45:0/951547873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4e5e00640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4e8066c00 0x7ff4e81443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4e6800640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4e80b5560 0x7ff4e81faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 -- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4e80b5560 msgr2=0x7ff4e81faa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4e80b5560 0x7ff4e81faa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 -- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4e8066c00 msgr2=0x7ff4e81443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4e8066c00 0x7ff4e81443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.197+0000 7ff4dd400640 1 -- 172.21.15.45:0/951547873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e815d5c0 con 0x7ff4e8066fe0 2024-04-25T10:33:29.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dd400640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 0x7ff4e81fa4c0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e960 tx=0x7ff4d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0017070 con 0x7ff4e8066fe0 2024-04-25T10:33:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d000ca00 con 0x7ff4e8066fe0 2024-04-25T10:33:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000cdb0 con 0x7ff4e8066fe0 2024-04-25T10:33:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e815d8a0 con 0x7ff4e8066fe0 2024-04-25T10:33:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e8159e90 con 0x7ff4e8066fe0 2024-04-25T10:33:29.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e804ef30 con 0x7ff4e8066fe0 2024-04-25T10:33:29.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4d001a070 con 0x7ff4e8066fe0 2024-04-25T10:33:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dea00640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4c4066f60 0x7ff4c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.201+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff4d0085190 con 0x7ff4e8066fe0 2024-04-25T10:33:29.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.205+0000 7ff4e5e00640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4c4066f60 0x7ff4c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.205+0000 7ff4e5e00640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4c4066f60 0x7ff4c4069420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80012f0 tx=0x7ff4d8001a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.205+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4d0055d50 con 0x7ff4e8066fe0 2024-04-25T10:33:29.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.365+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4e806e390 con 0x7ff4c4066f60 2024-04-25T10:33:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4dea00640 1 -- 172.21.15.45:0/951547873 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff4e806e390 con 0x7ff4c4066f60 2024-04-25T10:33:29.371 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:33:29.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4c4066f60 msgr2=0x7ff4c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4c4066f60 0x7ff4c4069420 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80012f0 tx=0x7ff4d8001a40 comp rx=0 tx=0).stop 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 msgr2=0x7ff4e81fa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 0x7ff4e81fa4c0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e960 tx=0x7ff4d000ee20 comp rx=0 tx=0).stop 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 shutdown_connections 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7ff4c4066f60 0x7ff4c4069420 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff4e80b5560 0x7ff4e81faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e8066fe0 0x7ff4e81fa4c0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 --2- 172.21.15.45:0/951547873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4e8066c00 0x7ff4e81443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 >> 172.21.15.45:0/951547873 conn(0x7ff4e80b24f0 msgr2=0x7ff4e8153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 shutdown_connections 2024-04-25T10:33:29.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.369+0000 7ff4ed200640 1 -- 172.21.15.45:0/951547873 wait complete. 2024-04-25T10:33:29.393 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:33:29.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:29.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 Processor -- start 2024-04-25T10:33:29.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 -- start start 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f243c15bd10 0x7f243c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f243c15c0f0 0x7f243c0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c06c0e0 con 0x7f243c0cf9e0 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c15c6a0 con 0x7f243c15c0f0 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2440e00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c15c980 con 0x7f243c15bd10 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2431400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f243c15c0f0 0x7f243c0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2431400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f243c15c0f0 0x7f243c0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57134/0 (socket says 172.21.15.45:57134) 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2431400640 1 -- 172.21.15.45:0/135879333 learned_addr learned my addr 172.21.15.45:0/135879333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f243a800640 1 --1- 172.21.15.45:0/135879333 >> v1:172.21.15.45:6789/0 conn(0x7f243c0cf9e0 0x7f243c06c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53954/0 (socket says 172.21.15.45:53954) 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.533+0000 7f2439e00640 1 --2- 172.21.15.45:0/135879333 >> v2:172.21.15.117:3300/0 conn(0x7f243c15bd10 0x7f243c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2431400640 1 -- 172.21.15.45:0/135879333 >> v2:172.21.15.117:3300/0 conn(0x7f243c15bd10 msgr2=0x7f243c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2431400640 1 --2- 172.21.15.45:0/135879333 >> v2:172.21.15.117:3300/0 conn(0x7f243c15bd10 0x7f243c0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2431400640 1 -- 172.21.15.45:0/135879333 >> v1:172.21.15.45:6789/0 conn(0x7f243c0cf9e0 legacy=0x7f243c06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2431400640 1 -- 172.21.15.45:0/135879333 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c15cc60 con 0x7f243c15c0f0 2024-04-25T10:33:29.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2439400640 1 -- 172.21.15.45:0/135879333 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374014893 0 0) 0x7f243c06c0e0 con 0x7f243c0cf9e0 2024-04-25T10:33:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2431400640 1 --2- 172.21.15.45:0/135879333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15c0f0 0x7f243c0b27e0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f243c072810 tx=0x7f242401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796be90451706d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2439400640 1 -- 172.21.15.45:0/135879333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240aa670 con 0x7f243c15c0f0 2024-04-25T10:33:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2439400640 1 -- 172.21.15.45:0/135879333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24240af070 con 0x7f243c15c0f0 2024-04-25T10:33:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2439400640 1 -- 172.21.15.45:0/135879333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240aaa30 con 0x7f243c15c0f0 2024-04-25T10:33:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 -- 172.21.15.45:0/135879333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15c0f0 msgr2=0x7f243c0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 --2- 172.21.15.45:0/135879333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15c0f0 0x7f243c0b27e0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f243c072810 tx=0x7f242401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 -- 172.21.15.45:0/135879333 shutdown_connections 2024-04-25T10:33:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 --2- 172.21.15.45:0/135879333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15c0f0 0x7f243c0b27e0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 --2- 172.21.15.45:0/135879333 >> v2:172.21.15.117:3300/0 conn(0x7f243c15bd10 0x7f243c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 -- 172.21.15.45:0/135879333 >> 172.21.15.45:0/135879333 conn(0x7f243c066b70 msgr2=0x7f243c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.537+0000 7f2440e00640 1 -- 172.21.15.45:0/135879333 shutdown_connections 2024-04-25T10:33:29.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- 172.21.15.45:0/135879333 wait complete. 2024-04-25T10:33:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 Processor -- start 2024-04-25T10:33:29.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- start start 2024-04-25T10:33:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f243c0cf9e0 0x7f243c139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15bd10 0x7f243c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f243c15c0f0 0x7f243c15da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0b2ea0 con 0x7f243c15bd10 2024-04-25T10:33:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0b2d20 con 0x7f243c15c0f0 2024-04-25T10:33:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c1692d0 con 0x7f243c0cf9e0 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15bd10 0x7f243c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15bd10 0x7f243c15d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57140/0 (socket says 172.21.15.45:57140) 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 -- 172.21.15.45:0/3082725428 learned_addr learned my addr 172.21.15.45:0/3082725428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f243a800640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f243c15c0f0 0x7f243c15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2439e00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f243c0cf9e0 0x7f243c139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 -- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f243c0cf9e0 msgr2=0x7f243c139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f243c0cf9e0 0x7f243c139890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 -- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f243c15c0f0 msgr2=0x7f243c15da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f243c15c0f0 0x7f243c15da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 -- 172.21.15.45:0/3082725428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c13c6c0 con 0x7f243c15bd10 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2431400640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15bd10 0x7f243c15d4f0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f2424019100 tx=0x7f242401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240bd5f0 con 0x7f243c15bd10 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24240bd790 con 0x7f243c15bd10 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24240bdb40 con 0x7f243c15bd10 2024-04-25T10:33:29.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c13c9a0 con 0x7f243c15bd10 2024-04-25T10:33:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.541+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c13cf60 con 0x7f243c15bd10 2024-04-25T10:33:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f243c13a130 con 0x7f243c15bd10 2024-04-25T10:33:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24240b33f0 con 0x7f243c15bd10 2024-04-25T10:33:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2432a00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2418066f60 0x7f2418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f242411f6a0 con 0x7f243c15bd10 2024-04-25T10:33:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2432a00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7f241806cec0 0x7f241806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 --> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f241806f990 con 0x7f241806cec0 2024-04-25T10:33:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2424126050 con 0x7f243c15bd10 2024-04-25T10:33:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2439e00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2418066f60 0x7f2418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f243a800640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7f241806cec0 0x7f241806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f2439e00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2418066f60 0x7f2418069420 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f242c00b930 tx=0x7f242c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.545+0000 7f243a800640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7f241806cec0 0x7f241806f2e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.549+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== osd.4 v2:172.21.15.117:6800/728189779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f241806f990 con 0x7f241806cec0 2024-04-25T10:33:29.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 --> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f243c04ef30 con 0x7f241806cec0 2024-04-25T10:33:29.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2432a00640 1 -- 172.21.15.45:0/3082725428 <== osd.4 v2:172.21.15.117:6800/728189779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f243c04ef30 con 0x7f241806cec0 2024-04-25T10:33:29.566 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:29.566 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:29.566 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:29.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7f241806cec0 msgr2=0x7f241806f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6800/728189779,v1:172.21.15.117:6801/728189779] conn(0x7f241806cec0 0x7f241806f2e0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2418066f60 msgr2=0x7f2418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f2418066f60 0x7f2418069420 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f242c00b930 tx=0x7f242c008040 comp rx=0 tx=0).stop 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15bd10 msgr2=0x7f243c15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 --2- 172.21.15.45:0/3082725428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c15bd10 0x7f243c15d4f0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f2424019100 tx=0x7f242401ad30 comp rx=0 tx=0).stop 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f243a800640 1 -- 172.21.15.45:0/3082725428 reap_dead start 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 shutdown_connections 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 >> 172.21.15.45:0/3082725428 conn(0x7f243c066b70 msgr2=0x7f243c15b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 shutdown_connections 2024-04-25T10:33:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.561+0000 7f2440e00640 1 -- 172.21.15.45:0/3082725428 wait complete. 2024-04-25T10:33:29.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:33:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.721+0000 7fdb82800640 1 Processor -- start 2024-04-25T10:33:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.721+0000 7fdb82800640 1 -- start start 2024-04-25T10:33:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb82800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb7c15bc40 0x7fdb7c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb82800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 0x7fdb7c0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb82800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c0b2770 con 0x7fdb7c0cf930 2024-04-25T10:33:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb82800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7c06c670 con 0x7fdb7c15bc40 2024-04-25T10:33:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb82800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c06c950 con 0x7fdb7c0cda10 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 0x7fdb7c0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 0x7fdb7c0b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54584/0 (socket says 172.21.15.45:54584) 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 -- 172.21.15.45:0/659222457 learned_addr learned my addr 172.21.15.45:0/659222457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb7b400640 1 --2- 172.21.15.45:0/659222457 >> v2:172.21.15.117:3300/0 conn(0x7fdb7c15bc40 0x7fdb7c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 -- 172.21.15.45:0/659222457 >> v2:172.21.15.117:3300/0 conn(0x7fdb7c15bc40 msgr2=0x7fdb7c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 --2- 172.21.15.45:0/659222457 >> v2:172.21.15.117:3300/0 conn(0x7fdb7c15bc40 0x7fdb7c06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 -- 172.21.15.45:0/659222457 >> v1:172.21.15.117:6789/0 conn(0x7fdb7c0cf930 legacy=0x7fdb7c15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.725+0000 7fdb73e00640 1 -- 172.21.15.45:0/659222457 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c06cc30 con 0x7fdb7c0cda10 2024-04-25T10:33:29.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb73e00640 1 --2- 172.21.15.45:0/659222457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 0x7fdb7c0b4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009870 tx=0x7fdb6401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f892a234e367d36e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb7aa00640 1 -- 172.21.15.45:0/659222457 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640a7070 con 0x7fdb7c0cda10 2024-04-25T10:33:29.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb7aa00640 1 -- 172.21.15.45:0/659222457 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb640b0070 con 0x7fdb7c0cda10 2024-04-25T10:33:29.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb7aa00640 1 -- 172.21.15.45:0/659222457 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640ab890 con 0x7fdb7c0cda10 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 -- 172.21.15.45:0/659222457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 msgr2=0x7fdb7c0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 --2- 172.21.15.45:0/659222457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 0x7fdb7c0b4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009870 tx=0x7fdb6401c560 comp rx=0 tx=0).stop 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 -- 172.21.15.45:0/659222457 shutdown_connections 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 --2- 172.21.15.45:0/659222457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c0cda10 0x7fdb7c0b4610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 --2- 172.21.15.45:0/659222457 >> v2:172.21.15.117:3300/0 conn(0x7fdb7c15bc40 0x7fdb7c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 -- 172.21.15.45:0/659222457 >> 172.21.15.45:0/659222457 conn(0x7fdb7c066b70 msgr2=0x7fdb7c066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 -- 172.21.15.45:0/659222457 shutdown_connections 2024-04-25T10:33:29.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 -- 172.21.15.45:0/659222457 wait complete. 2024-04-25T10:33:29.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 Processor -- start 2024-04-25T10:33:29.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.729+0000 7fdb82800640 1 -- start start 2024-04-25T10:33:29.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb7c0cda10 0x7fdb7c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 0x7fdb7c13a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c15bc40 0x7fdb7c0b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c168b50 con 0x7fdb7c0cf930 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c1689d0 con 0x7fdb7c0cda10 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c168cd0 con 0x7fdb7c15bc40 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 0x7fdb7c13a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 0x7fdb7c13a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57146/0 (socket says 172.21.15.45:57146) 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb7b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb7c0cda10 0x7fdb7c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 -- 172.21.15.45:0/3239427111 learned_addr learned my addr 172.21.15.45:0/3239427111 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb7be00640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c15bc40 0x7fdb7c0b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 -- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c15bc40 msgr2=0x7fdb7c0b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c15bc40 0x7fdb7c0b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 -- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb7c0cda10 msgr2=0x7fdb7c157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb7c0cda10 0x7fdb7c157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 -- 172.21.15.45:0/3239427111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c1581e0 con 0x7fdb7c0cf930 2024-04-25T10:33:29.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73e00640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 0x7fdb7c13a7f0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401ac10 tx=0x7fdb6401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640be5f0 con 0x7fdb7c0cf930 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb640be7b0 con 0x7fdb7c0cf930 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c1584c0 con 0x7fdb7c0cf930 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640beba0 con 0x7fdb7c0cf930 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c158a80 con 0x7fdb7c0cf930 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c04ef30 con 0x7fdb7c0cf930 2024-04-25T10:33:29.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.733+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb640b43f0 con 0x7fdb7c0cf930 2024-04-25T10:33:29.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.737+0000 7fdb73400640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb58066f60 0x7fdb58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:29.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.737+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fdb6411f230 con 0x7fdb7c0cf930 2024-04-25T10:33:29.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.737+0000 7fdb7b400640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb58066f60 0x7fdb58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:29.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.737+0000 7fdb7b400640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb58066f60 0x7fdb58069420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009b60 tx=0x7fdb6c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:29.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.741+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb640c4060 con 0x7fdb7c0cf930 2024-04-25T10:33:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.901+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb7c15f400 con 0x7fdb58066f60 2024-04-25T10:33:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.901+0000 7fdb73400640 1 -- 172.21.15.45:0/3239427111 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdb7c15f400 con 0x7fdb58066f60 2024-04-25T10:33:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:33:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb58066f60 msgr2=0x7fdb58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb58066f60 0x7fdb58069420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009b60 tx=0x7fdb6c009190 comp rx=0 tx=0).stop 2024-04-25T10:33:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 msgr2=0x7fdb7c13a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 0x7fdb7c13a7f0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401ac10 tx=0x7fdb6401ada0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 shutdown_connections 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fdb58066f60 0x7fdb58069420 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb7c15bc40 0x7fdb7c0b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb7c0cf930 0x7fdb7c13a7f0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 --2- 172.21.15.45:0/3239427111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb7c0cda10 0x7fdb7c157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 >> 172.21.15.45:0/3239427111 conn(0x7fdb7c066b70 msgr2=0x7fdb7c14ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 shutdown_connections 2024-04-25T10:33:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:29.905+0000 7fdb82800640 1 -- 172.21.15.45:0/3239427111 wait complete. 2024-04-25T10:33:29.927 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:33:29.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.069+0000 7f32f3800640 1 Processor -- start 2024-04-25T10:33:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.069+0000 7f32f3800640 1 -- start start 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f3800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f32ec162310 0x7f32ec15e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f3800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f32ec0b4f50 0x7f32ec0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f3800640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ec15e9b0 con 0x7f32ec1618a0 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f3800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec15f250 con 0x7f32ec162310 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f3800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec0afce0 con 0x7f32ec0b4f50 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f32ec162310 0x7f32ec15e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f32ec162310 0x7f32ec15e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57152/0 (socket says 172.21.15.45:57152) 2024-04-25T10:33:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 -- 172.21.15.45:0/3907406023 learned_addr learned my addr 172.21.15.45:0/3907406023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f1800640 1 --2- 172.21.15.45:0/3907406023 >> v2:172.21.15.117:3300/0 conn(0x7f32ec0b4f50 0x7f32ec0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 -- 172.21.15.45:0/3907406023 >> v2:172.21.15.117:3300/0 conn(0x7f32ec0b4f50 msgr2=0x7f32ec0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 --2- 172.21.15.45:0/3907406023 >> v2:172.21.15.117:3300/0 conn(0x7f32ec0b4f50 0x7f32ec0b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 -- 172.21.15.45:0/3907406023 >> v1:172.21.15.117:6789/0 conn(0x7f32ec1618a0 legacy=0x7f32ec0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:30.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 -- 172.21.15.45:0/3907406023 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec15ec90 con 0x7f32ec162310 2024-04-25T10:33:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32f0e00640 1 --2- 172.21.15.45:0/3907406023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec162310 0x7f32ec15e2b0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f32ec06f760 tx=0x7f32dc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e422981fa5dd5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.073+0000 7f32ebe00640 1 -- 172.21.15.45:0/3907406023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b0070 con 0x7f32ec162310 2024-04-25T10:33:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32ebe00640 1 -- 172.21.15.45:0/3907406023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0ab600 con 0x7f32ec162310 2024-04-25T10:33:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32ebe00640 1 -- 172.21.15.45:0/3907406023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0ab9d0 con 0x7f32ec162310 2024-04-25T10:33:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- 172.21.15.45:0/3907406023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec162310 msgr2=0x7f32ec15e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 --2- 172.21.15.45:0/3907406023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec162310 0x7f32ec15e2b0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f32ec06f760 tx=0x7f32dc01c3c0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- 172.21.15.45:0/3907406023 shutdown_connections 2024-04-25T10:33:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 --2- 172.21.15.45:0/3907406023 >> v2:172.21.15.117:3300/0 conn(0x7f32ec0b4f50 0x7f32ec0b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 --2- 172.21.15.45:0/3907406023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec162310 0x7f32ec15e2b0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- 172.21.15.45:0/3907406023 >> 172.21.15.45:0/3907406023 conn(0x7f32ec0b5610 msgr2=0x7f32ec0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- 172.21.15.45:0/3907406023 shutdown_connections 2024-04-25T10:33:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- 172.21.15.45:0/3907406023 wait complete. 2024-04-25T10:33:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 Processor -- start 2024-04-25T10:33:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- start start 2024-04-25T10:33:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec0b4f50 0x7f32ec15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32ec1618a0 0x7f32ec156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32ec162310 0x7f32ec156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0b1040 con 0x7f32ec0b4f50 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0b0ec0 con 0x7f32ec1618a0 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f3800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0b11c0 con 0x7f32ec162310 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec0b4f50 0x7f32ec15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec0b4f50 0x7f32ec15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57164/0 (socket says 172.21.15.45:57164) 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 -- 172.21.15.45:0/4007297384 learned_addr learned my addr 172.21.15.45:0/4007297384 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32e3e00640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32ec1618a0 0x7f32ec156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f1800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32ec162310 0x7f32ec156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 -- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32ec162310 msgr2=0x7f32ec156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32ec162310 0x7f32ec156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 -- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32ec1618a0 msgr2=0x7f32ec156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32ec1618a0 0x7f32ec156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f0e00640 1 -- 172.21.15.45:0/4007297384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec159e70 con 0x7f32ec0b4f50 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.077+0000 7f32f1800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32ec162310 0x7f32ec156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32f0e00640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec0b4f50 0x7f32ec15b010 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0ae040 tx=0x7f32dc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0bb070 con 0x7f32ec0b4f50 2024-04-25T10:33:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0b6430 con 0x7f32ec0b4f50 2024-04-25T10:33:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b0040 con 0x7f32ec0b4f50 2024-04-25T10:33:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec1595d0 con 0x7f32ec0b4f50 2024-04-25T10:33:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec159b90 con 0x7f32ec0b4f50 2024-04-25T10:33:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32ec159380 con 0x7f32ec0b4f50 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32dc0be070 con 0x7f32ec0b4f50 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f32c8066f60 0x7f32c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f32dc11e980 con 0x7f32ec0b4f50 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f32c806cec0 0x7f32c806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 --> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32dc0a9ec0 con 0x7f32c806cec0 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32dc11ee10 con 0x7f32ec0b4f50 2024-04-25T10:33:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32e3e00640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f32c8066f60 0x7f32c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.081+0000 7f32f1800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f32c806cec0 0x7f32c806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.085+0000 7f32e3e00640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f32c8066f60 0x7f32c8069420 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f32ec158920 tx=0x7f32d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.085+0000 7f32f1800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f32c806cec0 0x7f32c806f2e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.085+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== osd.5 v2:172.21.15.117:6802/2444936005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f32dc0a9ec0 con 0x7f32c806cec0 2024-04-25T10:33:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.097+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 --> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f32ec04ef30 con 0x7f32c806cec0 2024-04-25T10:33:30.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.097+0000 7f32e9600640 1 -- 172.21.15.45:0/4007297384 <== osd.5 v2:172.21.15.117:6802/2444936005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f32ec04ef30 con 0x7f32c806cec0 2024-04-25T10:33:30.102 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:30.102 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:30.102 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.097+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f32c806cec0 msgr2=0x7f32c806f2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.097+0000 7f32f3800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6802/2444936005,v1:172.21.15.117:6804/2444936005] conn(0x7f32c806cec0 0x7f32c806f2e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f32c8066f60 msgr2=0x7f32c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f32c8066f60 0x7f32c8069420 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f32ec158920 tx=0x7f32d4009040 comp rx=0 tx=0).stop 2024-04-25T10:33:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec0b4f50 msgr2=0x7f32ec15b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 --2- 172.21.15.45:0/4007297384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec0b4f50 0x7f32ec15b010 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0ae040 tx=0x7f32dc019160 comp rx=0 tx=0).stop 2024-04-25T10:33:30.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f1800640 1 -- 172.21.15.45:0/4007297384 reap_dead start 2024-04-25T10:33:30.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 shutdown_connections 2024-04-25T10:33:30.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 >> 172.21.15.45:0/4007297384 conn(0x7f32ec0b5610 msgr2=0x7f32ec0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 shutdown_connections 2024-04-25T10:33:30.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.101+0000 7f32f3800640 1 -- 172.21.15.45:0/4007297384 wait complete. 2024-04-25T10:33:30.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:33:30.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 Processor -- start 2024-04-25T10:33:30.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 -- start start 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69301550b0 0x7f69300cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f69300cfe70 0x7f693015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f693015b740 0x7f693006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6930071050 con 0x7f693015b740 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69300712f0 con 0x7f69300cfe70 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6935200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69300715d0 con 0x7f69301550b0 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f693015b740 0x7f693006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f693015b740 0x7f693006d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57176/0 (socket says 172.21.15.45:57176) 2024-04-25T10:33:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 -- 172.21.15.45:0/3415068413 learned_addr learned my addr 172.21.15.45:0/3415068413 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6927e00640 1 --2- 172.21.15.45:0/3415068413 >> v2:172.21.15.117:3300/0 conn(0x7f69300cfe70 0x7f693015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692de00640 1 --2- 172.21.15.45:0/3415068413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69301550b0 0x7f69300cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 -- 172.21.15.45:0/3415068413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69301550b0 msgr2=0x7f69300cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 --2- 172.21.15.45:0/3415068413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69301550b0 0x7f69300cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 -- 172.21.15.45:0/3415068413 >> v2:172.21.15.117:3300/0 conn(0x7f69300cfe70 msgr2=0x7f693015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 --2- 172.21.15.45:0/3415068413 >> v2:172.21.15.117:3300/0 conn(0x7f69300cfe70 0x7f693015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f692e800640 1 -- 172.21.15.45:0/3415068413 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300718b0 con 0x7f693015b740 2024-04-25T10:33:30.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.261+0000 7f6927e00640 1 --2- 172.21.15.45:0/3415068413 >> v2:172.21.15.117:3300/0 conn(0x7f69300cfe70 0x7f693015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f692e800640 1 --2- 172.21.15.45:0/3415068413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f693015b740 0x7f693006d1a0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f693006e290 tx=0x7f692801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576b28311a8c116e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f692d400640 1 -- 172.21.15.45:0/3415068413 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280ac020 con 0x7f693015b740 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f692d400640 1 -- 172.21.15.45:0/3415068413 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69280a6440 con 0x7f693015b740 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f692d400640 1 -- 172.21.15.45:0/3415068413 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280a6810 con 0x7f693015b740 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 -- 172.21.15.45:0/3415068413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f693015b740 msgr2=0x7f693006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 --2- 172.21.15.45:0/3415068413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f693015b740 0x7f693006d1a0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f693006e290 tx=0x7f692801c730 comp rx=0 tx=0).stop 2024-04-25T10:33:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 -- 172.21.15.45:0/3415068413 shutdown_connections 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 --2- 172.21.15.45:0/3415068413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f693015b740 0x7f693006d1a0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 --2- 172.21.15.45:0/3415068413 >> v2:172.21.15.117:3300/0 conn(0x7f69300cfe70 0x7f693015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 --2- 172.21.15.45:0/3415068413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69301550b0 0x7f69300cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 -- 172.21.15.45:0/3415068413 >> 172.21.15.45:0/3415068413 conn(0x7f6930066b70 msgr2=0x7f6930066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 -- 172.21.15.45:0/3415068413 shutdown_connections 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.265+0000 7f6935200640 1 -- 172.21.15.45:0/3415068413 wait complete. 2024-04-25T10:33:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 Processor -- start 2024-04-25T10:33:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- start start 2024-04-25T10:33:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69300cfe70 0x7f69300ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 0x7f6930157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693015b740 0x7f693013a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69300b33e0 con 0x7f69301550b0 2024-04-25T10:33:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69300b3260 con 0x7f69300cfe70 2024-04-25T10:33:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69300b3560 con 0x7f693015b740 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 0x7f6930157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 0x7f6930157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57180/0 (socket says 172.21.15.45:57180) 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 -- 172.21.15.45:0/550034642 learned_addr learned my addr 172.21.15.45:0/550034642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f692de00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69300cfe70 0x7f69300ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f692e800640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693015b740 0x7f693013a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 -- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693015b740 msgr2=0x7f693013a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693015b740 0x7f693013a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 -- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69300cfe70 msgr2=0x7f69300ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69300cfe70 0x7f69300ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 -- 172.21.15.45:0/550034642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693015c1b0 con 0x7f69301550b0 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f692de00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69300cfe70 0x7f69300ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f692e800640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693015b740 0x7f693013a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6927e00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 0x7f6930157530 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f691800e810 tx=0x7f691800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918017070 con 0x7f69301550b0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691800c870 con 0x7f69301550b0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6930157df0 con 0x7f69301550b0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800f040 con 0x7f69301550b0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6930157a70 con 0x7f69301550b0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.269+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693004ef30 con 0x7f69301550b0 2024-04-25T10:33:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.273+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f691801a070 con 0x7f69301550b0 2024-04-25T10:33:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.273+0000 7f6926000640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f690c066f90 0x7f690c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.273+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f6918084590 con 0x7f69301550b0 2024-04-25T10:33:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.273+0000 7f692de00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f690c066f90 0x7f690c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.273+0000 7f692de00640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f690c066f90 0x7f690c069450 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f69200079f0 tx=0x7f6920008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.277+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6918055860 con 0x7f69301550b0 2024-04-25T10:33:30.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.437+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6930158310 con 0x7f690c066f90 2024-04-25T10:33:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.437+0000 7f6926000640 1 -- 172.21.15.45:0/550034642 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6930158310 con 0x7f690c066f90 2024-04-25T10:33:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:33:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f690c066f90 msgr2=0x7f690c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f690c066f90 0x7f690c069450 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f69200079f0 tx=0x7f6920008040 comp rx=0 tx=0).stop 2024-04-25T10:33:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 msgr2=0x7f6930157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 0x7f6930157530 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f691800e810 tx=0x7f691800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 shutdown_connections 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f690c066f90 0x7f690c069450 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693015b740 0x7f693013a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69301550b0 0x7f6930157530 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 --2- 172.21.15.45:0/550034642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69300cfe70 0x7f69300ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 >> 172.21.15.45:0/550034642 conn(0x7f6930066b70 msgr2=0x7f6930158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 shutdown_connections 2024-04-25T10:33:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.441+0000 7f6935200640 1 -- 172.21.15.45:0/550034642 wait complete. 2024-04-25T10:33:30.462 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:33:30.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:30.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:30.518 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:30.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 Processor -- start 2024-04-25T10:33:30.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 -- start start 2024-04-25T10:33:30.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b7815bd10 0x7f8b780b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b780cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7806c0e0 con 0x7f8b7815c0f0 2024-04-25T10:33:30.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b780b2ef0 con 0x7f8b7815bd10 2024-04-25T10:33:30.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780d0390 con 0x7f8b780cf9e0 2024-04-25T10:33:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b780cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b75e00640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8b7815bd10 0x7f8b780b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b780cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.45:54620/0 (socket says 172.21.15.45:54620) 2024-04-25T10:33:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 -- 172.21.15.45:0/252361711 learned_addr learned my addr 172.21.15.45:0/252361711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 -- 172.21.15.45:0/252361711 >> v2:172.21.15.117:3300/0 conn(0x7f8b7815bd10 msgr2=0x7f8b780b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 --2- 172.21.15.45:0/252361711 >> v2:172.21.15.117:3300/0 conn(0x7f8b7815bd10 0x7f8b780b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 -- 172.21.15.45:0/252361711 >> v1:172.21.15.117:6789/0 conn(0x7f8b7815c0f0 legacy=0x7f8b780b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.605+0000 7f8b76800640 1 -- 172.21.15.45:0/252361711 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7815c6a0 con 0x7f8b780cf9e0 2024-04-25T10:33:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b76800640 1 --2- 172.21.15.45:0/252361711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b780cfe50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009e50 tx=0x7f8b7001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85fce699a51d89da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b74a00640 1 -- 172.21.15.45:0/252361711 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700a58d0 con 0x7f8b780cf9e0 2024-04-25T10:33:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b74a00640 1 -- 172.21.15.45:0/252361711 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b700a5a70 con 0x7f8b780cf9e0 2024-04-25T10:33:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b74a00640 1 -- 172.21.15.45:0/252361711 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700a5e40 con 0x7f8b780cf9e0 2024-04-25T10:33:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 -- 172.21.15.45:0/252361711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 msgr2=0x7f8b780cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 --2- 172.21.15.45:0/252361711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b780cfe50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009e50 tx=0x7f8b7001ca00 comp rx=0 tx=0).stop 2024-04-25T10:33:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 -- 172.21.15.45:0/252361711 shutdown_connections 2024-04-25T10:33:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 --2- 172.21.15.45:0/252361711 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b780cfe50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 --2- 172.21.15.45:0/252361711 >> v2:172.21.15.117:3300/0 conn(0x7f8b7815bd10 0x7f8b780b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 -- 172.21.15.45:0/252361711 >> 172.21.15.45:0/252361711 conn(0x7f8b78066b70 msgr2=0x7f8b78066f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 -- 172.21.15.45:0/252361711 shutdown_connections 2024-04-25T10:33:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.609+0000 7f8b7ca00640 1 -- 172.21.15.45:0/252361711 wait complete. 2024-04-25T10:33:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 Processor -- start 2024-04-25T10:33:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- start start 2024-04-25T10:33:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b7814af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7815bd10 0x7f8b78153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b7815c0f0 0x7f8b78153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780b4cd0 con 0x7f8b7815bd10 2024-04-25T10:33:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780b4b50 con 0x7f8b7815c0f0 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7806cdd0 con 0x7f8b780cf9e0 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7815bd10 0x7f8b78153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7815bd10 0x7f8b78153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57192/0 (socket says 172.21.15.45:57192) 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 -- 172.21.15.45:0/2420161833 learned_addr learned my addr 172.21.15.45:0/2420161833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75e00640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b7814af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b76800640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b7815c0f0 0x7f8b78153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 -- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 msgr2=0x7f8b7814af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8b780cf9e0 0x7f8b7814af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 -- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b7815c0f0 msgr2=0x7f8b78153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b7815c0f0 0x7f8b78153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 -- 172.21.15.45:0/2420161833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780af130 con 0x7f8b7815bd10 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b76800640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b7815c0f0 0x7f8b78153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b75400640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7815bd10 0x7f8b78153690 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c370 tx=0x7f8b6400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640052d0 con 0x7f8b7815bd10 2024-04-25T10:33:30.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b64005470 con 0x7f8b7815bd10 2024-04-25T10:33:30.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64007260 con 0x7f8b7815bd10 2024-04-25T10:33:30.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b780af410 con 0x7f8b7815bd10 2024-04-25T10:33:30.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b780af9d0 con 0x7f8b7815bd10 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.613+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b7804ef30 con 0x7f8b7815bd10 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b64007400 con 0x7f8b7815bd10 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b58066f90 0x7f8b58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8b64084120 con 0x7f8b7815bd10 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7f8b5806cef0 0x7f8b5806f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 --> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b5806f9c0 con 0x7f8b5806cef0 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b6408b050 con 0x7f8b7815bd10 2024-04-25T10:33:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b75e00640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b58066f90 0x7f8b58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b76800640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7f8b5806cef0 0x7f8b5806f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b75e00640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b58066f90 0x7f8b58069450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009b60 tx=0x7f8b68009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b76800640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7f8b5806cef0 0x7f8b5806f310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.617+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== osd.6 v2:172.21.15.117:6816/2565393264 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8b5806f9c0 con 0x7f8b5806cef0 2024-04-25T10:33:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 --> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8b7806d9b0 con 0x7f8b5806cef0 2024-04-25T10:33:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b6e000640 1 -- 172.21.15.45:0/2420161833 <== osd.6 v2:172.21.15.117:6816/2565393264 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8b7806d9b0 con 0x7f8b5806cef0 2024-04-25T10:33:30.636 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:30.636 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:30.636 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7f8b5806cef0 msgr2=0x7f8b5806f310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6816/2565393264,v1:172.21.15.117:6818/2565393264] conn(0x7f8b5806cef0 0x7f8b5806f310 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b58066f90 msgr2=0x7f8b58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8b58066f90 0x7f8b58069450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009b60 tx=0x7f8b68009190 comp rx=0 tx=0).stop 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7815bd10 msgr2=0x7f8b78153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 --2- 172.21.15.45:0/2420161833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b7815bd10 0x7f8b78153690 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c370 tx=0x7f8b6400c830 comp rx=0 tx=0).stop 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b76800640 1 -- 172.21.15.45:0/2420161833 reap_dead start 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 shutdown_connections 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 >> 172.21.15.45:0/2420161833 conn(0x7f8b78066b70 msgr2=0x7f8b7815e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 shutdown_connections 2024-04-25T10:33:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.633+0000 7f8b7ca00640 1 -- 172.21.15.45:0/2420161833 wait complete. 2024-04-25T10:33:30.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:33:30.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 Processor -- start 2024-04-25T10:33:30.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 -- start start 2024-04-25T10:33:30.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed600ce9a0 0x7fed600cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f570 0x7fed600ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600d15e0 con 0x7fed6015f190 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed600cfef0 con 0x7fed600ce9a0 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed68000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed600d0140 con 0x7fed6015f570 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.793+0000 7fed65600640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fed6015f190 0x7fed600ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53962/0 (socket says 172.21.15.45:53962) 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed65600640 1 -- 172.21.15.45:0/749979480 learned_addr learned my addr 172.21.15.45:0/749979480 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed64c00640 1 --2- 172.21.15.45:0/749979480 >> v2:172.21.15.117:3300/0 conn(0x7fed600ce9a0 0x7fed600cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed66000640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f570 0x7fed600ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed5fe00640 1 -- 172.21.15.45:0/749979480 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762609472 0 0) 0x7fed54001120 con 0x7fed6015f190 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed5fe00640 1 -- 172.21.15.45:0/749979480 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed48003530 con 0x7fed6015f190 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed64c00640 1 -- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f570 msgr2=0x7fed600ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed64c00640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f570 0x7fed600ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed64c00640 1 -- 172.21.15.45:0/749979480 >> v1:172.21.15.45:6789/0 conn(0x7fed6015f190 legacy=0x7fed600ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed64c00640 1 -- 172.21.15.45:0/749979480 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600d0420 con 0x7fed600ce9a0 2024-04-25T10:33:30.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed66000640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f570 0x7fed600ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:33:30.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed64c00640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed600ce9a0 0x7fed600cc5d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0097c0 tx=0x7fed4c01c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1397a8f04574ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed5fe00640 1 -- 172.21.15.45:0/749979480 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0b0070 con 0x7fed600ce9a0 2024-04-25T10:33:30.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed5fe00640 1 -- 172.21.15.45:0/749979480 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0ab600 con 0x7fed600ce9a0 2024-04-25T10:33:30.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.797+0000 7fed5fe00640 1 -- 172.21.15.45:0/749979480 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0ab9d0 con 0x7fed600ce9a0 2024-04-25T10:33:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed600ce9a0 msgr2=0x7fed600cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed600ce9a0 0x7fed600cc5d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0097c0 tx=0x7fed4c01c350 comp rx=0 tx=0).stop 2024-04-25T10:33:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- 172.21.15.45:0/749979480 shutdown_connections 2024-04-25T10:33:30.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f570 0x7fed600ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 --2- 172.21.15.45:0/749979480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed600ce9a0 0x7fed600cc5d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- 172.21.15.45:0/749979480 >> 172.21.15.45:0/749979480 conn(0x7fed600d1910 msgr2=0x7fed600d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- 172.21.15.45:0/749979480 shutdown_connections 2024-04-25T10:33:30.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- 172.21.15.45:0/749979480 wait complete. 2024-04-25T10:33:30.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 Processor -- start 2024-04-25T10:33:30.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- start start 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 0x7fed60154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f190 0x7fed60156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed6015f570 0x7fed6006f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed60168a50 con 0x7fed600ce9a0 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600d2640 con 0x7fed6015f570 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed68000640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed60168bd0 con 0x7fed6015f190 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 0x7fed60154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 0x7fed60154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57196/0 (socket says 172.21.15.45:57196) 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 -- 172.21.15.45:0/2820515621 learned_addr learned my addr 172.21.15.45:0/2820515621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed64c00640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f190 0x7fed60156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed66000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed6015f570 0x7fed6006f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 -- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f190 msgr2=0x7fed60156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f190 0x7fed60156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 -- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed6015f570 msgr2=0x7fed6006f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed6015f570 0x7fed6006f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.801+0000 7fed65600640 1 -- 172.21.15.45:0/2820515621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed601566d0 con 0x7fed600ce9a0 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed65600640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 0x7fed60154ce0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fed5400c6b0 tx=0x7fed5400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54005ac0 con 0x7fed600ce9a0 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54015070 con 0x7fed600ce9a0 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54007070 con 0x7fed600ce9a0 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6015a290 con 0x7fed600ce9a0 2024-04-25T10:33:30.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed601556b0 con 0x7fed600ce9a0 2024-04-25T10:33:30.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6004ef30 con 0x7fed600ce9a0 2024-04-25T10:33:30.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed54007210 con 0x7fed600ce9a0 2024-04-25T10:33:30.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed5d600640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed40066f60 0x7fed40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:30.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.805+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fed54088910 con 0x7fed600ce9a0 2024-04-25T10:33:30.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.809+0000 7fed64c00640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed40066f60 0x7fed40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:30.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.809+0000 7fed64c00640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed40066f60 0x7fed40069420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0ad000 tx=0x7fed4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:30.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.813+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed600d15e0 con 0x7fed600ce9a0 2024-04-25T10:33:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fed6015dd00 con 0x7fed40066f60 2024-04-25T10:33:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed5d600640 1 -- 172.21.15.45:0/2820515621 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fed6015dd00 con 0x7fed40066f60 2024-04-25T10:33:30.979 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed40066f60 msgr2=0x7fed40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed40066f60 0x7fed40069420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0ad000 tx=0x7fed4c008040 comp rx=0 tx=0).stop 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 msgr2=0x7fed60154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 0x7fed60154ce0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fed5400c6b0 tx=0x7fed5400cb70 comp rx=0 tx=0).stop 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 shutdown_connections 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fed40066f60 0x7fed40069420 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed6015f570 0x7fed6006f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed6015f190 0x7fed60156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 --2- 172.21.15.45:0/2820515621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed600ce9a0 0x7fed60154ce0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:30.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 >> 172.21.15.45:0/2820515621 conn(0x7fed600d1910 msgr2=0x7fed6015e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:30.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 shutdown_connections 2024-04-25T10:33:30.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:30.977+0000 7fed68000640 1 -- 172.21.15.45:0/2820515621 wait complete. 2024-04-25T10:33:31.001 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:33:31.002 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:33:31.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa9200640 1 Processor -- start 2024-04-25T10:33:31.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa9200640 1 -- start start 2024-04-25T10:33:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa9200640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fafa40b2ae0 0x7fafa4168d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa9200640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa406d1b0 con 0x7fafa4162660 2024-04-25T10:33:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa9200640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa406d490 con 0x7fafa415dd20 2024-04-25T10:33:31.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa9200640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa40cb390 con 0x7fafa40b2ae0 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7faf9be00640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fafa4162660 0x7fafa40b23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53968/0 (socket says 172.21.15.45:53968) 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7faf9be00640 1 -- 172.21.15.45:0/3064309826 learned_addr learned my addr 172.21.15.45:0/3064309826 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa3e00640 1 --2- 172.21.15.45:0/3064309826 >> v2:172.21.15.117:3300/0 conn(0x7fafa40b2ae0 0x7fafa4168d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809543388 0 0) 0x7fafa406d1b0 con 0x7fafa4162660 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf88003530 con 0x7fafa4162660 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757208045 0 0) 0x7fafa406d490 con 0x7fafa415dd20 2024-04-25T10:33:31.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa406d1b0 con 0x7fafa415dd20 2024-04-25T10:33:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa3e00640 1 -- 172.21.15.45:0/3064309826 >> v1:172.21.15.117:6789/0 conn(0x7fafa415dd20 legacy=0x7fafa40cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa3e00640 1 -- 172.21.15.45:0/3064309826 >> v1:172.21.15.45:6789/0 conn(0x7fafa4162660 legacy=0x7fafa40b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.145+0000 7fafa3e00640 1 -- 172.21.15.45:0/3064309826 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa406d770 con 0x7fafa40b2ae0 2024-04-25T10:33:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa3e00640 1 --2- 172.21.15.45:0/3064309826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 0x7fafa4168d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009d90 tx=0x7faf9c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875b76166ddffc64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0b2070 con 0x7fafa40b2ae0 2024-04-25T10:33:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c0a57a0 con 0x7fafa40b2ae0 2024-04-25T10:33:31.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa2a00640 1 -- 172.21.15.45:0/3064309826 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0a5b70 con 0x7fafa40b2ae0 2024-04-25T10:33:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 -- 172.21.15.45:0/3064309826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 msgr2=0x7fafa4168d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 --2- 172.21.15.45:0/3064309826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 0x7fafa4168d10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009d90 tx=0x7faf9c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 -- 172.21.15.45:0/3064309826 shutdown_connections 2024-04-25T10:33:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 --2- 172.21.15.45:0/3064309826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 0x7fafa4168d10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 -- 172.21.15.45:0/3064309826 >> 172.21.15.45:0/3064309826 conn(0x7fafa40cd4b0 msgr2=0x7fafa40cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 -- 172.21.15.45:0/3064309826 shutdown_connections 2024-04-25T10:33:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.149+0000 7fafa9200640 1 -- 172.21.15.45:0/3064309826 wait complete. 2024-04-25T10:33:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 Processor -- start 2024-04-25T10:33:31.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- start start 2024-04-25T10:33:31.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 0x7fafa40c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafa415dd20 0x7fafa4160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafa4162660 0x7fafa40c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa416ba70 con 0x7fafa415dd20 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa416bbf0 con 0x7fafa40b2ae0 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa416bd70 con 0x7fafa4162660 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafa415dd20 0x7fafa4160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafa415dd20 0x7fafa4160d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57204/0 (socket says 172.21.15.45:57204) 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 -- 172.21.15.45:0/3677663126 learned_addr learned my addr 172.21.15.45:0/3677663126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa3400640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 0x7fafa40c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa3e00640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafa4162660 0x7fafa40c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 -- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafa4162660 msgr2=0x7fafa40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fafa4162660 0x7fafa40c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 -- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 msgr2=0x7fafa40c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafa40b2ae0 0x7fafa40c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 -- 172.21.15.45:0/3677663126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa415ef50 con 0x7fafa415dd20 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9be00640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafa415dd20 0x7fafa4160d30 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005890 tx=0x7faf8c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c017070 con 0x7fafa415dd20 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c005cf0 con 0x7fafa415dd20 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0072f0 con 0x7fafa415dd20 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa415f230 con 0x7fafa415dd20 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa415f7f0 con 0x7fafa415dd20 2024-04-25T10:33:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.153+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fafa4075120 con 0x7fafa415dd20 2024-04-25T10:33:31.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf8c01a070 con 0x7fafa415dd20 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7faf9b400640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf80066f90 0x7faf80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7faf8c084160 con 0x7fafa415dd20 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7faf9b400640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7faf8006cef0 0x7faf8006f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 --> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf8c01be50 con 0x7faf8006cef0 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=199) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf8c08b050 con 0x7fafa415dd20 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7fafa3400640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf80066f90 0x7faf80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7fafa3e00640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7faf8006cef0 0x7faf8006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7fafa3400640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf80066f90 0x7faf80069450 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7faf940017c0 tx=0x7faf94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.157+0000 7fafa3e00640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7faf8006cef0 0x7faf8006f310 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.161+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== osd.7 v2:172.21.15.117:6817/2004387070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faf8c01be50 con 0x7faf8006cef0 2024-04-25T10:33:31.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 --> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fafa40cf5b0 con 0x7faf8006cef0 2024-04-25T10:33:31.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7faf9b400640 1 -- 172.21.15.45:0/3677663126 <== osd.7 v2:172.21.15.117:6817/2004387070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fafa40cf5b0 con 0x7faf8006cef0 2024-04-25T10:33:31.176 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T10:33:31.176 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:33:31.176 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T10:33:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7faf8006cef0 msgr2=0x7faf8006f310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6817/2004387070,v1:172.21.15.117:6820/2004387070] conn(0x7faf8006cef0 0x7faf8006f310 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf80066f90 msgr2=0x7faf80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7faf80066f90 0x7faf80069450 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7faf940017c0 tx=0x7faf94009040 comp rx=0 tx=0).stop 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafa415dd20 msgr2=0x7fafa4160d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 --2- 172.21.15.45:0/3677663126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafa415dd20 0x7fafa4160d30 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7faf8c005890 tx=0x7faf8c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa3e00640 1 -- 172.21.15.45:0/3677663126 reap_dead start 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 shutdown_connections 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 >> 172.21.15.45:0/3677663126 conn(0x7fafa40cd4b0 msgr2=0x7fafa40cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 shutdown_connections 2024-04-25T10:33:31.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.173+0000 7fafa9200640 1 -- 172.21.15.45:0/3677663126 wait complete. 2024-04-25T10:33:31.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:33:31.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.333+0000 7f8d12800640 1 Processor -- start 2024-04-25T10:33:31.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.333+0000 7f8d12800640 1 -- start start 2024-04-25T10:33:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d12800640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8d0c143d40 0x7f8d0c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d12800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d0c15ed90 0x7f8d0c0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c161690 0x7f8d0c161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d12800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c144120 con 0x7f8d0c15ed90 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d12800640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c15e050 con 0x7f8d0c143d40 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d12800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c162350 con 0x7f8d0c161690 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d0c15ed90 0x7f8d0c0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8d0c15ed90 0x7f8d0c0b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57212/0 (socket says 172.21.15.45:57212) 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 -- 172.21.15.45:0/392083408 learned_addr learned my addr 172.21.15.45:0/392083408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d11400640 1 --2- 172.21.15.45:0/392083408 >> v2:172.21.15.117:3300/0 conn(0x7f8d0c143d40 0x7f8d0c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d11e00640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c161690 0x7f8d0c161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 -- 172.21.15.45:0/392083408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c161690 msgr2=0x7f8d0c161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c161690 0x7f8d0c161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 -- 172.21.15.45:0/392083408 >> v2:172.21.15.117:3300/0 conn(0x7f8d0c143d40 msgr2=0x7f8d0c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 --2- 172.21.15.45:0/392083408 >> v2:172.21.15.117:3300/0 conn(0x7f8d0c143d40 0x7f8d0c0b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 -- 172.21.15.45:0/392083408 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c15e2b0 con 0x7f8d0c15ed90 2024-04-25T10:33:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d11e00640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c161690 0x7f8d0c161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.337+0000 7f8d10a00640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c15ed90 0x7f8d0c0b3400 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c06d900 tx=0x7f8cf801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e09717183ed2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d0be00640 1 -- 172.21.15.45:0/392083408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80b1070 con 0x7f8d0c15ed90 2024-04-25T10:33:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d0be00640 1 -- 172.21.15.45:0/392083408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf80ac630 con 0x7f8d0c15ed90 2024-04-25T10:33:31.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d0be00640 1 -- 172.21.15.45:0/392083408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80a8050 con 0x7f8d0c15ed90 2024-04-25T10:33:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- 172.21.15.45:0/392083408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c15ed90 msgr2=0x7f8d0c0b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c15ed90 0x7f8d0c0b3400 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c06d900 tx=0x7f8cf801c520 comp rx=0 tx=0).stop 2024-04-25T10:33:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- 172.21.15.45:0/392083408 shutdown_connections 2024-04-25T10:33:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c161690 0x7f8d0c161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- 172.21.15.45:0/392083408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c15ed90 0x7f8d0c0b3400 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- 172.21.15.45:0/392083408 >> v2:172.21.15.117:3300/0 conn(0x7f8d0c143d40 0x7f8d0c0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- 172.21.15.45:0/392083408 >> 172.21.15.45:0/392083408 conn(0x7f8d0c06cd20 msgr2=0x7f8d0c06d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- 172.21.15.45:0/392083408 shutdown_connections 2024-04-25T10:33:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- 172.21.15.45:0/392083408 wait complete. 2024-04-25T10:33:31.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 Processor -- start 2024-04-25T10:33:31.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- start start 2024-04-25T10:33:31.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d0c143d40 0x7f8d0c0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c15ed90 0x7f8d0c153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 0x7f8d0c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c06e930 con 0x7f8d0c161690 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c06e7b0 con 0x7f8d0c143d40 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d12800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c06eab0 con 0x7f8d0c15ed90 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 0x7f8d0c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 0x7f8d0c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57228/0 (socket says 172.21.15.45:57228) 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 -- 172.21.15.45:0/538357315 learned_addr learned my addr 172.21.15.45:0/538357315 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11400640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d0c143d40 0x7f8d0c0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d10a00640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c15ed90 0x7f8d0c153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 -- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c15ed90 msgr2=0x7f8d0c153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c15ed90 0x7f8d0c153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 -- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d0c143d40 msgr2=0x7f8d0c0adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.341+0000 7f8d11e00640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d0c143d40 0x7f8d0c0adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d11e00640 1 -- 172.21.15.45:0/538357315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c06f000 con 0x7f8d0c161690 2024-04-25T10:33:31.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d11e00640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 0x7f8d0c0ceeb0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f8d04007e70 tx=0x7f8d0400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401a070 con 0x7f8d0c161690 2024-04-25T10:33:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d04003bc0 con 0x7f8d0c161690 2024-04-25T10:33:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004030 con 0x7f8d0c161690 2024-04-25T10:33:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c155460 con 0x7f8d0c161690 2024-04-25T10:33:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c155a20 con 0x7f8d0c161690 2024-04-25T10:33:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c0745a0 con 0x7f8d0c161690 2024-04-25T10:33:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d0401d070 con 0x7f8d0c161690 2024-04-25T10:33:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d09600640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8cec066f60 0x7f8cec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8d04084ed0 con 0x7f8d0c161690 2024-04-25T10:33:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.345+0000 7f8d11400640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8cec066f60 0x7f8cec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.349+0000 7f8d11400640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8cec066f60 0x7f8cec069420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009b60 tx=0x7f8d00009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.349+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d04010070 con 0x7f8d0c161690 2024-04-25T10:33:31.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d0c06f4c0 con 0x7f8cec066f60 2024-04-25T10:33:31.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d09600640 1 -- 172.21.15.45:0/538357315 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8d0c06f4c0 con 0x7f8cec066f60 2024-04-25T10:33:31.516 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:33:31.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8cec066f60 msgr2=0x7f8cec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8cec066f60 0x7f8cec069420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009b60 tx=0x7f8d00009190 comp rx=0 tx=0).stop 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 msgr2=0x7f8d0c0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 0x7f8d0c0ceeb0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f8d04007e70 tx=0x7f8d0400d4a0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 shutdown_connections 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f8cec066f60 0x7f8cec069420 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d0c161690 0x7f8d0c0ceeb0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d0c15ed90 0x7f8d0c153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 --2- 172.21.15.45:0/538357315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d0c143d40 0x7f8d0c0adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 >> 172.21.15.45:0/538357315 conn(0x7f8d0c06cd20 msgr2=0x7f8d0c15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 shutdown_connections 2024-04-25T10:33:31.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.513+0000 7f8d12800640 1 -- 172.21.15.45:0/538357315 wait complete. 2024-04-25T10:33:31.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:33:31.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 Processor -- start 2024-04-25T10:33:31.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 -- start start 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe18815e050 0x7fe1880b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1880cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1880b1850 con 0x7fe1880d1820 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1880b1af0 con 0x7fe18815e050 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe18e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880b1dd0 con 0x7fe1880b13e0 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe187400640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe1880d1820 0x7fe1880d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53972/0 (socket says 172.21.15.45:53972) 2024-04-25T10:33:31.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe187400640 1 -- 172.21.15.45:0/632096752 learned_addr learned my addr 172.21.15.45:0/632096752 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe17ea00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe18815e050 0x7fe1880b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe187e00640 1 --2- 172.21.15.45:0/632096752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1880cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe186a00640 1 -- 172.21.15.45:0/632096752 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170131901 0 0) 0x7fe1880b1850 con 0x7fe1880d1820 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe17ea00640 1 -- 172.21.15.45:0/632096752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 msgr2=0x7fe1880cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe17ea00640 1 --2- 172.21.15.45:0/632096752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1880cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe17ea00640 1 -- 172.21.15.45:0/632096752 >> v1:172.21.15.45:6789/0 conn(0x7fe1880d1820 legacy=0x7fe1880d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.677+0000 7fe17ea00640 1 -- 172.21.15.45:0/632096752 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880b20b0 con 0x7fe18815e050 2024-04-25T10:33:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe17ea00640 1 --2- 172.21.15.45:0/632096752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18815e050 0x7fe1880b0ea0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fe1700097c0 tx=0x7fe17001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd328840cc8a5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe186a00640 1 -- 172.21.15.45:0/632096752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700ab610 con 0x7fe18815e050 2024-04-25T10:33:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe186a00640 1 -- 172.21.15.45:0/632096752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1700b0070 con 0x7fe18815e050 2024-04-25T10:33:31.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe186a00640 1 -- 172.21.15.45:0/632096752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700ab9d0 con 0x7fe18815e050 2024-04-25T10:33:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 -- 172.21.15.45:0/632096752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18815e050 msgr2=0x7fe1880b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 --2- 172.21.15.45:0/632096752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18815e050 0x7fe1880b0ea0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7fe1700097c0 tx=0x7fe17001c410 comp rx=0 tx=0).stop 2024-04-25T10:33:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 -- 172.21.15.45:0/632096752 shutdown_connections 2024-04-25T10:33:31.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 --2- 172.21.15.45:0/632096752 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1880cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 --2- 172.21.15.45:0/632096752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe18815e050 0x7fe1880b0ea0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 -- 172.21.15.45:0/632096752 >> 172.21.15.45:0/632096752 conn(0x7fe18809a9d0 msgr2=0x7fe18809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 -- 172.21.15.45:0/632096752 shutdown_connections 2024-04-25T10:33:31.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.681+0000 7fe18e800640 1 -- 172.21.15.45:0/632096752 wait complete. 2024-04-25T10:33:31.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 Processor -- start 2024-04-25T10:33:31.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- start start 2024-04-25T10:33:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1881510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 0x7fe188154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe18815e050 0x7fe188154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880d0dd0 con 0x7fe1880d1820 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880d0c50 con 0x7fe18815e050 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880d0f50 con 0x7fe1880b13e0 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 0x7fe188154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 0x7fe188154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57246/0 (socket says 172.21.15.45:57246) 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 -- 172.21.15.45:0/2352138826 learned_addr learned my addr 172.21.15.45:0/2352138826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe187400640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1881510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe187e00640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe18815e050 0x7fe188154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 -- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 msgr2=0x7fe1881510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1881510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 -- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe18815e050 msgr2=0x7fe188154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe18815e050 0x7fe188154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 -- 172.21.15.45:0/2352138826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880b7270 con 0x7fe1880d1820 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe187e00640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe18815e050 0x7fe188154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17ea00640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 0x7fe188154640 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fe17001a040 tx=0x7fe17001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700bf610 con 0x7fe1880d1820 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1700bf7d0 con 0x7fe1880d1820 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1700bfbc0 con 0x7fe1880d1820 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18815d350 con 0x7fe1880d1820 2024-04-25T10:33:31.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1881515e0 con 0x7fe1880d1820 2024-04-25T10:33:31.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.685+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18815d870 con 0x7fe1880d1820 2024-04-25T10:33:31.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.689+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1700bd020 con 0x7fe1880d1820 2024-04-25T10:33:31.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.689+0000 7fe17fe00640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:31.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.689+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fe17011f090 con 0x7fe1880d1820 2024-04-25T10:33:31.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.689+0000 7fe187400640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:31.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.689+0000 7fe187400640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe164066f60 0x7fe164069420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe1780012f0 tx=0x7fe178009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:31.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.693+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1700c6070 con 0x7fe1880d1820 2024-04-25T10:33:31.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 --> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe18804ef30 con 0x7fe164066f60 2024-04-25T10:33:31.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe17fe00640 1 -- 172.21.15.45:0/2352138826 <== mgr.4099 v2:172.21.15.117:6832/1944228396 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13406 (secure 0 0 0) 0x7fe18804ef30 con 0x7fe164066f60 2024-04-25T10:33:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T10:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe164066f60 msgr2=0x7fe164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe164066f60 0x7fe164069420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe1780012f0 tx=0x7fe178009040 comp rx=0 tx=0).stop 2024-04-25T10:33:31.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 msgr2=0x7fe188154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 0x7fe188154640 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fe17001a040 tx=0x7fe17001ac50 comp rx=0 tx=0).stop 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 shutdown_connections 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe18815e050 0x7fe188154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1880d1820 0x7fe188154640 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 --2- 172.21.15.45:0/2352138826 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1880b13e0 0x7fe1881510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 >> 172.21.15.45:0/2352138826 conn(0x7fe18809a9d0 msgr2=0x7fe1880af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 shutdown_connections 2024-04-25T10:33:31.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:31.853+0000 7fe18e800640 1 -- 172.21.15.45:0/2352138826 wait complete. 2024-04-25T10:33:31.877 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":664,"stamp":"2024-04-25T10:33:31.097128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":475560,"kb_used_data":20776,"kb_used_omap":12,"kb_used_meta":454643,"kb_avail":754499160,"statfs":{"total":773094113280,"available":772607139840,"internally_reserved":0,"allocated":21274624,"data_stored":10016888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465554992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12582912,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-8,"num_write_kb":-12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002171"},"pg_stats":[{"pgid":"1.0","version":"199'119","reported_seq":524,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-04-25T10:33:28.848564+0000","last_change":"2024-04-25T10:33:28.848596+0000","last_active":"2024-04-25T10:33:28.848564+0000","last_peered":"2024-04-25T10:33:28.848564+0000","last_clean":"2024-04-25T10:33:28.848564+0000","last_became_active":"2024-04-25T10:17:50.625858+0000","last_became_peered":"2024-04-25T10:17:50.625858+0000","last_unstale":"2024-04-25T10:33:28.848564+0000","last_undegraded":"2024-04-25T10:33:28.848564+0000","last_fullsized":"2024-04-25T10:33:28.848564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'119","last_scrub_stamp":"2024-04-25T10:33:28.848492+0000","last_deep_scrub":"199'119","last_deep_scrub_stamp":"2024-04-25T10:33:28.848492+0000","last_clean_scrub_stamp":"2024-04-25T10:33:28.848492+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:55.773218+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55844,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315996,"statfs":{"total":96636764160,"available":96579579904,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509631,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64744,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307096,"statfs":{"total":96636764160,"available":96570466304,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47524,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324316,"statfs":{"total":96636764160,"available":96588099584,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69672,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302168,"statfs":{"total":96636764160,"available":96565420032,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509632,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52264,"kb_used_data":3032,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319576,"statfs":{"total":96636764160,"available":96583245824,"internally_reserved":0,"allocated":3104768,"data_stored":1694887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60004,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311836,"statfs":{"total":96636764160,"available":96575320064,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542334,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69664,"kb_used_data":2452,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302176,"statfs":{"total":96636764160,"available":96565428224,"internally_reserved":0,"allocated":2510848,"data_stored":1104519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:33:31.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:33:32.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 Processor -- start 2024-04-25T10:33:32.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 -- start start 2024-04-25T10:33:32.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd558070350 0x7fd55815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5580b31c0 0x7fd5580b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55815b080 con 0x7fd5580aef80 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55815adb0 con 0x7fd5580b31c0 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55f600640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55815eb60 con 0x7fd558070350 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55d600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5580b31c0 0x7fd5580b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55d600640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5580b31c0 0x7fd5580b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57258/0 (socket says 172.21.15.45:57258) 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd55d600640 1 -- 172.21.15.45:0/918063094 learned_addr learned my addr 172.21.15.45:0/918063094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.017+0000 7fd54fe00640 1 --2- 172.21.15.45:0/918063094 >> v2:172.21.15.117:3300/0 conn(0x7fd558070350 0x7fd55815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55d600640 1 -- 172.21.15.45:0/918063094 >> v2:172.21.15.117:3300/0 conn(0x7fd558070350 msgr2=0x7fd55815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55d600640 1 --2- 172.21.15.45:0/918063094 >> v2:172.21.15.117:3300/0 conn(0x7fd558070350 0x7fd55815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55d600640 1 -- 172.21.15.45:0/918063094 >> v1:172.21.15.117:6789/0 conn(0x7fd5580aef80 legacy=0x7fd55815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55d600640 1 -- 172.21.15.45:0/918063094 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55815ee40 con 0x7fd5580b31c0 2024-04-25T10:33:32.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd54fe00640 1 --2- 172.21.15.45:0/918063094 >> v2:172.21.15.117:3300/0 conn(0x7fd558070350 0x7fd55815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:32.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55d600640 1 --2- 172.21.15.45:0/918063094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5580b31c0 0x7fd5580b3610 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fd55806e390 tx=0x7fd55001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602126f8c088182e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:32.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd557e00640 1 -- 172.21.15.45:0/918063094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5500a57d0 con 0x7fd5580b31c0 2024-04-25T10:33:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd557e00640 1 -- 172.21.15.45:0/918063094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5500a5970 con 0x7fd5580b31c0 2024-04-25T10:33:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd557e00640 1 -- 172.21.15.45:0/918063094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5500a5d40 con 0x7fd5580b31c0 2024-04-25T10:33:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 -- 172.21.15.45:0/918063094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5580b31c0 msgr2=0x7fd5580b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 --2- 172.21.15.45:0/918063094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5580b31c0 0x7fd5580b3610 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fd55806e390 tx=0x7fd55001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 -- 172.21.15.45:0/918063094 shutdown_connections 2024-04-25T10:33:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 --2- 172.21.15.45:0/918063094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5580b31c0 0x7fd5580b3610 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 --2- 172.21.15.45:0/918063094 >> v2:172.21.15.117:3300/0 conn(0x7fd558070350 0x7fd55815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 -- 172.21.15.45:0/918063094 >> 172.21.15.45:0/918063094 conn(0x7fd5580669a0 msgr2=0x7fd558066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 -- 172.21.15.45:0/918063094 shutdown_connections 2024-04-25T10:33:32.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.021+0000 7fd55f600640 1 -- 172.21.15.45:0/918063094 wait complete. 2024-04-25T10:33:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 Processor -- start 2024-04-25T10:33:32.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 -- start start 2024-04-25T10:33:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 0x7fd55814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5580aef80 0x7fd55814bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5580b31c0 0x7fd55814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581679a0 con 0x7fd558070350 2024-04-25T10:33:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558167820 con 0x7fd5580b31c0 2024-04-25T10:33:32.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd558167b20 con 0x7fd5580aef80 2024-04-25T10:33:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 0x7fd55814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 0x7fd55814b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57266/0 (socket says 172.21.15.45:57266) 2024-04-25T10:33:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 -- 172.21.15.45:0/3442758273 learned_addr learned my addr 172.21.15.45:0/3442758273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd54fe00640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5580aef80 0x7fd55814bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55d600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5580b31c0 0x7fd55814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 -- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5580aef80 msgr2=0x7fd55814bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5580aef80 0x7fd55814bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 -- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5580b31c0 msgr2=0x7fd55814c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5580b31c0 0x7fd55814c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 -- 172.21.15.45:0/3442758273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580ce7f0 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55cc00640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 0x7fd55814b5d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c420 tx=0x7fd54800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480052d0 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548005470 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548007260 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.025+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580b0f10 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.029+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5580b0430 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.029+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55804ef30 con 0x7fd558070350 2024-04-25T10:33:32.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.029+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd548007400 con 0x7fd558070350 2024-04-25T10:33:32.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.029+0000 7fd555600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd534066f60 0x7fd534069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.029+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd548084180 con 0x7fd558070350 2024-04-25T10:33:32.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.029+0000 7fd54fe00640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd534066f60 0x7fd534069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.033+0000 7fd54fe00640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd534066f60 0x7fd534069420 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd55814d820 tx=0x7fd540008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:32.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.033+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5580b0f10 con 0x7fd558070350 2024-04-25T10:33:32.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.197+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd5580725b0 con 0x7fd558070350 2024-04-25T10:33:32.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd548054e70 con 0x7fd558070350 2024-04-25T10:33:32.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd555600640 1 -- 172.21.15.45:0/3442758273 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd548058f10 con 0x7fd558070350 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd534066f60 msgr2=0x7fd534069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd534066f60 0x7fd534069420 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd55814d820 tx=0x7fd540008040 comp rx=0 tx=0).stop 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 msgr2=0x7fd55814b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 0x7fd55814b5d0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c420 tx=0x7fd54800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 shutdown_connections 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7fd534066f60 0x7fd534069420 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5580b31c0 0x7fd55814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5580aef80 0x7fd55814bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 --2- 172.21.15.45:0/3442758273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd558070350 0x7fd55814b5d0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 >> 172.21.15.45:0/3442758273 conn(0x7fd5580669a0 msgr2=0x7fd55806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:32.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 shutdown_connections 2024-04-25T10:33:32.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.201+0000 7fd55f600640 1 -- 172.21.15.45:0/3442758273 wait complete. 2024-04-25T10:33:32.225 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:33:32.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:33:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.361+0000 7f3d85400640 1 Processor -- start 2024-04-25T10:33:32.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.361+0000 7f3d85400640 1 -- start start 2024-04-25T10:33:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d85400640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d800cdd90 0x7f3d800ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d85400640 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d8015eac0 0x7f3d800cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d85400640 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8015a770 con 0x7f3d8015a390 2024-04-25T10:33:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d85400640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d800ce550 con 0x7f3d800cdd90 2024-04-25T10:33:32.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d85400640 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8015e000 con 0x7f3d8015eac0 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d800cdd90 0x7f3d800ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d800cdd90 0x7f3d800ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57278/0 (socket says 172.21.15.45:57278) 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 -- 172.21.15.45:0/3428441738 learned_addr learned my addr 172.21.15.45:0/3428441738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d7f200640 1 --2- 172.21.15.45:0/3428441738 >> v2:172.21.15.117:3300/0 conn(0x7f3d8015eac0 0x7f3d800cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 -- 172.21.15.45:0/3428441738 >> v2:172.21.15.117:3300/0 conn(0x7f3d8015eac0 msgr2=0x7f3d800cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 --2- 172.21.15.45:0/3428441738 >> v2:172.21.15.117:3300/0 conn(0x7f3d8015eac0 0x7f3d800cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 -- 172.21.15.45:0/3428441738 >> v1:172.21.15.117:6789/0 conn(0x7f3d8015a390 legacy=0x7f3d800cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d77e00640 1 -- 172.21.15.45:0/3428441738 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800ce270 con 0x7f3d800cdd90 2024-04-25T10:33:32.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.365+0000 7f3d7f200640 1 --2- 172.21.15.45:0/3428441738 >> v2:172.21.15.117:3300/0 conn(0x7f3d8015eac0 0x7f3d800cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:33:32.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d77e00640 1 --2- 172.21.15.45:0/3428441738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800cdd90 0x7f3d800ccb40 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f3d800afa80 tx=0x7f3d6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0857c0735112776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d7de00640 1 -- 172.21.15.45:0/3428441738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b1070 con 0x7f3d800cdd90 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d7de00640 1 -- 172.21.15.45:0/3428441738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3d680ac410 con 0x7f3d800cdd90 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d7de00640 1 -- 172.21.15.45:0/3428441738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680ac800 con 0x7f3d800cdd90 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 -- 172.21.15.45:0/3428441738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800cdd90 msgr2=0x7f3d800ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 --2- 172.21.15.45:0/3428441738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800cdd90 0x7f3d800ccb40 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f3d800afa80 tx=0x7f3d6801c520 comp rx=0 tx=0).stop 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 -- 172.21.15.45:0/3428441738 shutdown_connections 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 --2- 172.21.15.45:0/3428441738 >> v2:172.21.15.117:3300/0 conn(0x7f3d8015eac0 0x7f3d800cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 --2- 172.21.15.45:0/3428441738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d800cdd90 0x7f3d800ccb40 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 -- 172.21.15.45:0/3428441738 >> 172.21.15.45:0/3428441738 conn(0x7f3d800b1d50 msgr2=0x7f3d800b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 -- 172.21.15.45:0/3428441738 shutdown_connections 2024-04-25T10:33:32.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 -- 172.21.15.45:0/3428441738 wait complete. 2024-04-25T10:33:32.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 Processor -- start 2024-04-25T10:33:32.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 -- start start 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d800cdd90 0x7f3d800ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.369+0000 7f3d85400640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d8015a390 0x7f3d801fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 0x7f3d801fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800aff10 con 0x7f3d8015eac0 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800afd90 con 0x7f3d800cdd90 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800b0090 con 0x7f3d8015a390 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 0x7f3d801fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 0x7f3d801fa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57294/0 (socket says 172.21.15.45:57294) 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 -- 172.21.15.45:0/2246200057 learned_addr learned my addr 172.21.15.45:0/2246200057 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7e800640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d800cdd90 0x7f3d800ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d77e00640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d8015a390 0x7f3d801fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 -- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d8015a390 msgr2=0x7f3d801fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d8015a390 0x7f3d801fa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 -- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d800cdd90 msgr2=0x7f3d800ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d800cdd90 0x7f3d800ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 -- 172.21.15.45:0/2246200057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800cba10 con 0x7f3d8015eac0 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d7f200640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 0x7f3d801fa9c0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f3d78007e20 tx=0x7f3d7800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7801a070 con 0x7f3d8015eac0 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3d78003ba0 con 0x7f3d8015eac0 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800de80 con 0x7f3d8015eac0 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8015abd0 con 0x7f3d8015eac0 2024-04-25T10:33:32.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d800aedc0 con 0x7f3d8015eac0 2024-04-25T10:33:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8004ef30 con 0x7f3d8015eac0 2024-04-25T10:33:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.373+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d7801d070 con 0x7f3d8015eac0 2024-04-25T10:33:32.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.377+0000 7f3d76a00640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d5c066f60 0x7f3d5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:33:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.377+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3d78084ff0 con 0x7f3d8015eac0 2024-04-25T10:33:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.377+0000 7f3d7e800640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d5c066f60 0x7f3d5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:33:32.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.377+0000 7f3d7e800640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d5c066f60 0x7f3d5c069420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3d70001810 tx=0x7f3d70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:33:32.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.381+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d78010070 con 0x7f3d8015eac0 2024-04-25T10:33:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.573+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d8006c9f0 con 0x7f3d8015eac0 2024-04-25T10:33:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.573+0000 7f3d76a00640 1 -- 172.21.15.45:0/2246200057 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+3711 (secure 0 0 0) 0x7f3d78055bf0 con 0x7f3d8015eac0 2024-04-25T10:33:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T10:33:32.577 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":148,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6836/303953181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":303953181},{"type":"v1","addr":"172.21.15.117:6836","nonce":303953181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":10483,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3136185226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6835","nonce":3136185226},{"type":"v1","addr":"172.21.15.117:6837","nonce":3136185226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":11301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/3022665424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3022665424},{"type":"v1","addr":"172.21.15.45:6834","nonce":3022665424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":11769,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3774714909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":3774714909},{"type":"v1","addr":"172.21.15.45:6835","nonce":3774714909}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[]} 2024-04-25T10:33:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 148 2024-04-25T10:33:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d5c066f60 msgr2=0x7f3d5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d5c066f60 0x7f3d5c069420 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3d70001810 tx=0x7f3d70009040 comp rx=0 tx=0).stop 2024-04-25T10:33:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 msgr2=0x7f3d801fa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:33:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 0x7f3d801fa9c0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f3d78007e20 tx=0x7f3d7800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 shutdown_connections 2024-04-25T10:33:32.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:6832/1944228396,v1:172.21.15.117:6833/1944228396] conn(0x7f3d5c066f60 0x7f3d5c069420 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d8015eac0 0x7f3d801fa9c0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d8015a390 0x7f3d801fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 --2- 172.21.15.45:0/2246200057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d800cdd90 0x7f3d800ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:33:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 >> 172.21.15.45:0/2246200057 conn(0x7f3d800b1d50 msgr2=0x7f3d8006d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:33:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 shutdown_connections 2024-04-25T10:33:32.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T10:33:32.577+0000 7f3d85400640 1 -- 172.21.15.45:0/2246200057 wait complete. 2024-04-25T10:33:32.599 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:33:32.599 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:33:32.599 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:33:32.599 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:32.611 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:33:32.611 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:33:32.612 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:32.665 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:33:32.665 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:33:32.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:32.679 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:33:32.679 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:33:32.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:32.736 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:33:32.736 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:33:32.737 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:33:32.737 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:38.839 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:33:38.840 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:33:38.840 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:38.918 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:33:38.918 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:33:38.918 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:39.013 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:33:39.013 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:33:39.013 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:40.027 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-25T10:33:45.114 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:33:45.115 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:33:45.115 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:45.192 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:33:45.192 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:33:45.192 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:45.934 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~6s 2024-04-25T10:33:51.294 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:33:51.295 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:33:51.295 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.387 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:33:51.388 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:33:51.388 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.470 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:33:51.470 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:33:51.470 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:33:51.470 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.528 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:33:51.529 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:33:51.529 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.587 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:33:51.588 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:33:51.588 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:33:51.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T10:33:51.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-25T10:33:51.609 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:33:51.609 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:33:51.609 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.627 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:33:51.627 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:33:51.627 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:33:51.690 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:33:51.690 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:33:51.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T10:33:51.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:33:51.726 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:33:52.084 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:33:52.084 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:33:52.269 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:33:52.269 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:33:52.461 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:33:52.461 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:33:52.683 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:33:52.683 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:33:52.954 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:33:52.954 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:33:53.173 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:33:53.173 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:33:53.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:33:53.374 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:33:53.563 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:33:53.564 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/data/mon.a.tgz 2024-04-25T10:33:53.565 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-04-25T10:33:53.574 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.tExTbpUwMl 2024-04-25T10:33:53.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.tExTbpUwMl 2024-04-25T10:33:53.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.tExTbpUwMl 2024-04-25T10:33:53.819 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.tExTbpUwMl is 485KB 2024-04-25T10:33:53.888 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.tExTbpUwMl 2024-04-25T10:33:53.897 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/data/mon.b.tgz 2024-04-25T10:33:53.897 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-04-25T10:33:53.906 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.NSGx38nMEj 2024-04-25T10:33:53.906 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.NSGx38nMEj 2024-04-25T10:33:54.064 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.NSGx38nMEj 2024-04-25T10:33:54.138 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.NSGx38nMEj is 485KB 2024-04-25T10:33:54.205 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.NSGx38nMEj 2024-04-25T10:33:54.213 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/data/mon.c.tgz 2024-04-25T10:33:54.213 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-04-25T10:33:54.261 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.WASw3WrjSw 2024-04-25T10:33:54.261 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WASw3WrjSw 2024-04-25T10:33:54.412 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.WASw3WrjSw 2024-04-25T10:33:54.486 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.WASw3WrjSw is 485KB 2024-04-25T10:33:54.557 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.WASw3WrjSw 2024-04-25T10:33:54.565 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:33:54.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:33:54.568 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:33:54.639 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:33:54.673 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:33:54.674 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:33:54.675 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/remote/smithi045/crash 2024-04-25T10:33:54.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:33:54.707 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/remote/smithi117/crash 2024-04-25T10:33:54.708 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:33:54.731 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:33:54.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:54.753 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:54.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:54.846 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:33:54.864 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:33:54.865 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:33:54.889 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:33:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86023.log 2024-04-25T10:33:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79006.log 2024-04-25T10:33:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75681.log 2024-04-25T10:33:54.918 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86023.log.gz 2024-04-25T10:33:54.918 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32391.log 2024-04-25T10:33:54.918 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79006.log.gz 2024-04-25T10:33:54.919 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80428.log 2024-04-25T10:33:54.919 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75681.log.gz 2024-04-25T10:33:54.920 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52280.log 2024-04-25T10:33:54.920 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32391.log.gz 2024-04-25T10:33:54.921 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54153.log 2024-04-25T10:33:54.921 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80428.log.gz 2024-04-25T10:33:54.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84240.log 2024-04-25T10:33:54.922 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52280.log.gz 2024-04-25T10:33:54.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35018.log 2024-04-25T10:33:54.923 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54153.log.gz 2024-04-25T10:33:54.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35939.log 2024-04-25T10:33:54.923 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84240.log.gz 2024-04-25T10:33:54.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70807.log 2024-04-25T10:33:54.924 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35018.log.gz 2024-04-25T10:33:54.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32594.log 2024-04-25T10:33:54.925 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35939.log.gz 2024-04-25T10:33:54.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70937.log 2024-04-25T10:33:54.926 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70807.log.gz 2024-04-25T10:33:54.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2024-04-25T10:33:54.927 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32594.log.gz 2024-04-25T10:33:54.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85195.log 2024-04-25T10:33:54.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52176.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.70937.log.gz 2024-04-25T10:33:54.928 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72763.log 2024-04-25T10:33:54.928 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2024-04-25T10:33:54.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.85195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79917.log 0.0% 2024-04-25T10:33:54.929 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.0.85195.log.gz 2024-04-25T10:33:54.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32024.log 2024-04-25T10:33:54.930 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72763.log.gz 2024-04-25T10:33:54.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79220.log 2024-04-25T10:33:54.931 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79917.log.gz 2024-04-25T10:33:54.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85236.log 2024-04-25T10:33:54.932 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32024.log.gz 2024-04-25T10:33:54.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83634.log 2024-04-25T10:33:54.933 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79220.log.gz 2024-04-25T10:33:54.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85441.log 2024-04-25T10:33:54.934 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85236.log.gz 2024-04-25T10:33:54.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53277.log 2024-04-25T10:33:54.935 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.83634.log.gz 2024-04-25T10:33:54.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85523.log 2024-04-25T10:33:54.935 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85441.log.gz 2024-04-25T10:33:54.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67755.log 2024-04-25T10:33:54.936 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53277.log.gz 2024-04-25T10:33:54.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88558.log 2024-04-25T10:33:54.937 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85523.log.gz 2024-04-25T10:33:54.937 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:33:54.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87640.log 2024-04-25T10:33:54.938 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67755.log.gz 2024-04-25T10:33:54.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50991.log 2024-04-25T10:33:54.939 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88558.log.gz 2024-04-25T10:33:54.942 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:33:54.942 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27833.log 2024-04-25T10:33:54.942 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27706.log 2024-04-25T10:33:54.942 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27833.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:33:54.942 INFO:teuthology.orchestra.run.smithi117.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27833.log.gz 2024-04-25T10:33:54.942 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27778.log 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi117.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27706.log.gz 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33156.log 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87640.log.gz 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2%/var/log/ceph/ceph-client.admin.34810.log: -- replaced with /var/log/ceph/ceph-client.admin.50991.log.gz 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49506.log 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34242.log 2024-04-25T10:33:54.943 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-04-25T10:33:54.946 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27778.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27778.log.gz 2024-04-25T10:33:54.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33062.log 2024-04-25T10:33:54.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.3.33156.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:33:54.947 INFO:teuthology.orchestra.run.smithi117.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33156.log.gz 2024-04-25T10:33:54.947 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.1.33062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33062.log.gz 2024-04-25T10:33:54.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:33:54.947 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27857.log 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69833.log 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49506.log.gz 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52042.log 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34242.log.gz 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88124.log 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69833.log.gz 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70480.log 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52042.log.gz 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82600.log 2024-04-25T10:33:54.949 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88124.log.gz 2024-04-25T10:33:54.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89414.log 2024-04-25T10:33:54.950 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70480.log.gz 2024-04-25T10:33:54.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79664.log 2024-04-25T10:33:54.950 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82600.log.gz 2024-04-25T10:33:54.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51200.log 2024-04-25T10:33:54.950 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89414.log.gz 2024-04-25T10:33:54.954 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:33:54.954 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27881.log 2024-04-25T10:33:54.954 INFO:teuthology.orchestra.run.smithi117.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27857.log.gz 2024-04-25T10:33:54.954 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:33:54.954 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27881.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27881.log.gz 2024-04-25T10:33:54.955 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27754.log 2024-04-25T10:33:54.955 INFO:teuthology.orchestra.run.smithi117.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:33:54.955 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:33:54.955 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27754.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27754.log.gz 2024-04-25T10:33:54.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48273.log 2024-04-25T10:33:54.957 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79664.log.gz 2024-04-25T10:33:54.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70506.log 2024-04-25T10:33:54.957 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51200.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85867.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70506.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48273.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79716.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70506.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73447.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85867.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76167.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79716.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53875.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73447.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84783.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.76167.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33800.log 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53875.log.gz 2024-04-25T10:33:54.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89667.log 2024-04-25T10:33:54.959 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84783.log.gz 2024-04-25T10:33:54.959 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33800.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:33:54.959 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33800.log.gz 2024-04-25T10:33:54.960 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:33:54.960 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:33:54.961 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:33:54.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82548.log 2024-04-25T10:33:54.962 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89667.log.gz 2024-04-25T10:33:54.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85045.log 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82548.log.gz 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76624.log 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85045.log.gz 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77246.log 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82854.log 2024-04-25T10:33:54.963 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76624.log.gz 2024-04-25T10:33:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90111.log 2024-04-25T10:33:54.964 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77246.log.gz 2024-04-25T10:33:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89877.log 2024-04-25T10:33:54.964 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82854.log.gz 2024-04-25T10:33:54.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84811.log 2024-04-25T10:33:54.965 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90111.log.gz 2024-04-25T10:33:54.966 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82444.log 2024-04-25T10:33:54.966 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89877.log.gz 2024-04-25T10:33:54.966 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52506.log 2024-04-25T10:33:54.967 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27802.log 2024-04-25T10:33:54.968 INFO:teuthology.orchestra.run.smithi117.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:33:54.969 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84811.log.gz 2024-04-25T10:33:54.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89492.log 2024-04-25T10:33:54.969 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82444.log.gz 2024-04-25T10:33:54.969 INFO:teuthology.orchestra.run.smithi045.stderr: 94.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:33:54.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.52506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50099.log 2024-04-25T10:33:54.969 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52506.log.gz 2024-04-25T10:33:54.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48376.log 2024-04-25T10:33:54.971 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89492.log.gz 2024-04-25T10:33:54.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77218.log 2024-04-25T10:33:54.971 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50099.log.gz 2024-04-25T10:33:54.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51570.log 2024-04-25T10:33:54.972 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.27802.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:33:54.972 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27730.log 2024-04-25T10:33:54.972 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27730.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27802.log.gz 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77218.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48376.log.gz 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32392.log 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77218.log.gz 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85997.log 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51570.log.gz 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70271.log 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32392.log.gz 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53719.log 2024-04-25T10:33:54.974 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85997.log.gz 2024-04-25T10:33:54.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82099.log 2024-04-25T10:33:54.976 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70271.log.gz 2024-04-25T10:33:54.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35731.log 2024-04-25T10:33:54.976 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53719.log.gz 2024-04-25T10:33:54.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54071.log 2024-04-25T10:33:54.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35731.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.82099.log.gz 2024-04-25T10:33:54.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89955.log 2024-04-25T10:33:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35731.log.gz 2024-04-25T10:33:54.978 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85919.log 2024-04-25T10:33:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54071.log.gz 2024-04-25T10:33:54.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83684.log 2024-04-25T10:33:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89955.log.gz 2024-04-25T10:33:54.979 INFO:teuthology.orchestra.run.smithi117.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27730.log.gz 2024-04-25T10:33:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67861.log 2024-04-25T10:33:54.980 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85919.log.gz 2024-04-25T10:33:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75375.log 2024-04-25T10:33:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67861.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.83684.log.gz 2024-04-25T10:33:54.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86611.log 2024-04-25T10:33:54.981 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67861.log.gz 2024-04-25T10:33:54.982 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86160.log 2024-04-25T10:33:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.75375.log.gz 2024-04-25T10:33:54.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90033.log 2024-04-25T10:33:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86611.log.gz 2024-04-25T10:33:54.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76728.log 2024-04-25T10:33:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86160.log.gz 2024-04-25T10:33:54.984 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73142.log 2024-04-25T10:33:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90033.log.gz 2024-04-25T10:33:54.985 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70755.log 2024-04-25T10:33:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76728.log.gz 2024-04-25T10:33:54.986 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78980.log 2024-04-25T10:33:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73142.log.gz 2024-04-25T10:33:54.986 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54309.log 2024-04-25T10:33:54.987 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70755.log.gz 2024-04-25T10:33:54.987 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.78980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-04-25T10:33:54.987 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78980.log.gz 2024-04-25T10:33:54.988 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82919.log 2024-04-25T10:33:54.988 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54309.log.gz 2024-04-25T10:33:54.989 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-04-25T10:33:54.989 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-04-25T10:33:54.989 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36017.log 2024-04-25T10:33:54.990 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82919.log.gz 2024-04-25T10:33:54.990 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69328.log 2024-04-25T10:33:54.991 INFO:teuthology.orchestra.run.smithi045.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-04-25T10:33:54.991 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2024-04-25T10:33:54.992 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69328.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.36017.log.gz 2024-04-25T10:33:54.992 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85633.log 2024-04-25T10:33:54.992 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51492.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.69328.log.gz 2024-04-25T10:33:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87283.log 2024-04-25T10:33:54.993 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz 2024-04-25T10:33:54.993 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85633.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:33:54.993 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85633.log.gz 2024-04-25T10:33:54.994 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51832.log 2024-04-25T10:33:54.995 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87283.log.gz 2024-04-25T10:33:54.995 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84585.log 2024-04-25T10:33:54.996 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79560.log 2024-04-25T10:33:54.996 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51832.log.gz 2024-04-25T10:33:54.997 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51336.log 2024-04-25T10:33:54.997 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84585.log.gz 2024-04-25T10:33:54.998 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35913.log 2024-04-25T10:33:54.998 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79560.log.gz 2024-04-25T10:33:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32988.log 2024-04-25T10:33:54.999 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51336.log.gz 2024-04-25T10:33:54.999 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80734.log 2024-04-25T10:33:55.000 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35913.log.gz 2024-04-25T10:33:55.000 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34008.log 2024-04-25T10:33:55.001 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32988.log.gz 2024-04-25T10:33:55.001 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.80734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80734.log.gz 2024-04-25T10:33:55.001 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82050.log 2024-04-25T10:33:55.002 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67939.log 2024-04-25T10:33:55.002 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34008.log.gz 2024-04-25T10:33:55.003 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35783.log 2024-04-25T10:33:55.003 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.82050.log.gz 2024-04-25T10:33:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50610.log 2024-04-25T10:33:55.004 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67939.log.gz 2024-04-25T10:33:55.004 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48459.log 2024-04-25T10:33:55.005 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35783.log.gz 2024-04-25T10:33:55.005 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32130.log 2024-04-25T10:33:55.006 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50610.log.gz 2024-04-25T10:33:55.006 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89466.log 2024-04-25T10:33:55.007 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48459.log.gz 2024-04-25T10:33:55.007 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79476.log 2024-04-25T10:33:55.007 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32130.log.gz 2024-04-25T10:33:55.008 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72377.log 2024-04-25T10:33:55.008 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89466.log.gz 2024-04-25T10:33:55.009 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70558.log 2024-04-25T10:33:55.009 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79476.log.gz 2024-04-25T10:33:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49118.log 2024-04-25T10:33:55.010 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72377.log.gz 2024-04-25T10:33:55.010 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73905.log 2024-04-25T10:33:55.011 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70558.log.gz 2024-04-25T10:33:55.011 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52228.log 2024-04-25T10:33:55.012 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49118.log.gz 2024-04-25T10:33:55.012 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34516.log 2024-04-25T10:33:55.012 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73905.log.gz 2024-04-25T10:33:55.013 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54431.log 2024-04-25T10:33:55.013 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52228.log.gz 2024-04-25T10:33:55.014 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50021.log 2024-04-25T10:33:55.015 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34516.log.gz 2024-04-25T10:33:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.54431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54431.log.gz 2024-04-25T10:33:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53954.log 2024-04-25T10:33:55.015 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85893.log 2024-04-25T10:33:55.016 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50021.log.gz 2024-04-25T10:33:55.016 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80402.log 2024-04-25T10:33:55.016 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53954.log.gz 2024-04-25T10:33:55.017 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54008.log 2024-04-25T10:33:55.018 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85893.log.gz 2024-04-25T10:33:55.018 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69940.log 2024-04-25T10:33:55.018 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80402.log.gz 2024-04-25T10:33:55.019 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48485.log 2024-04-25T10:33:55.019 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54008.log.gz 2024-04-25T10:33:55.020 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80508.log 2024-04-25T10:33:55.020 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69940.log.gz 2024-04-25T10:33:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49294.log 2024-04-25T10:33:55.021 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48485.log.gz 2024-04-25T10:33:55.021 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49454.log 2024-04-25T10:33:55.022 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80508.log.gz 2024-04-25T10:33:55.022 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86795.log 2024-04-25T10:33:55.023 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49454.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49294.log.gz 2024-04-25T10:33:55.023 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49454.log.gz 2024-04-25T10:33:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53251.log 2024-04-25T10:33:55.024 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80206.log 2024-04-25T10:33:55.024 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86795.log.gz 2024-04-25T10:33:55.025 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81899.log 2024-04-25T10:33:55.025 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53251.log.gz 2024-04-25T10:33:55.026 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70193.log 2024-04-25T10:33:55.026 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80206.log.gz 2024-04-25T10:33:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.81899.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.81899.log.gz --verbose 2024-04-25T10:33:55.027 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-client.admin.66915.log 2024-04-25T10:33:55.027 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82985.log 2024-04-25T10:33:55.028 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70193.log.gz 2024-04-25T10:33:55.028 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76037.log 2024-04-25T10:33:55.029 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66915.log.gz 2024-04-25T10:33:55.029 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84266.log 2024-04-25T10:33:55.029 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82985.log.gz 2024-04-25T10:33:55.030 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84915.log 2024-04-25T10:33:55.030 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76037.log.gz 2024-04-25T10:33:55.031 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-04-25T10:33:55.031 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84266.log.gz 2024-04-25T10:33:55.032 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34784.log 2024-04-25T10:33:55.032 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84915.log.gz 2024-04-25T10:33:55.033 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34138.log 2024-04-25T10:33:55.033 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-04-25T10:33:55.033 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83366.log 2024-04-25T10:33:55.034 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-04-25T10:33:55.034 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75987.log 2024-04-25T10:33:55.034 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34138.log.gz 2024-04-25T10:33:55.035 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49007.log 2024-04-25T10:33:55.036 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83366.log.gz 2024-04-25T10:33:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-04-25T10:33:55.037 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.75987.log.gz 2024-04-25T10:33:55.037 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53641.log 2024-04-25T10:33:55.037 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49007.log.gz 2024-04-25T10:33:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31918.log 2024-04-25T10:33:55.038 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-04-25T10:33:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34675.log 2024-04-25T10:33:55.039 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53641.log.gz 2024-04-25T10:33:55.039 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69278.log 2024-04-25T10:33:55.039 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31918.log.gz 2024-04-25T10:33:55.040 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76115.log 2024-04-25T10:33:55.041 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34675.log.gz 2024-04-25T10:33:55.041 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83948.log 2024-04-25T10:33:55.042 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.69278.log.gz 2024-04-25T10:33:55.042 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86453.log 2024-04-25T10:33:55.042 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76115.log.gz 2024-04-25T10:33:55.043 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82887.log 2024-04-25T10:33:55.043 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83948.log.gz 2024-04-25T10:33:55.044 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77681.log 2024-04-25T10:33:55.044 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86453.log.gz 2024-04-25T10:33:55.044 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51167.log 2024-04-25T10:33:55.044 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82887.log.gz 2024-04-25T10:33:55.045 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75835.log 2024-04-25T10:33:55.046 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77681.log.gz 2024-04-25T10:33:55.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71033.log 2024-04-25T10:33:55.046 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51167.log.gz 2024-04-25T10:33:55.047 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35454.log 2024-04-25T10:33:55.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.71033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71033.log.gz 2024-04-25T10:33:55.048 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76520.log 91.3% 2024-04-25T10:33:55.048 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75835.log.gz 2024-04-25T10:33:55.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31837.log 2024-04-25T10:33:55.049 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35454.log.gz 2024-04-25T10:33:55.050 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76195.log 2024-04-25T10:33:55.050 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76520.log.gz 2024-04-25T10:33:55.050 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70885.log 2024-04-25T10:33:55.051 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31837.log.gz 2024-04-25T10:33:55.051 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35558.log 2024-04-25T10:33:55.052 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76195.log.gz 2024-04-25T10:33:55.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83340.log 2024-04-25T10:33:55.053 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70885.log.gz 2024-04-25T10:33:55.053 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72932.log 2024-04-25T10:33:55.053 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35558.log.gz 2024-04-25T10:33:55.054 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50454.log 2024-04-25T10:33:55.054 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.83340.log.gz 2024-04-25T10:33:55.055 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73395.log 2024-04-25T10:33:55.055 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72932.log.gz 2024-04-25T10:33:55.055 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89073.log 2024-04-25T10:33:55.056 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50454.log.gz 2024-04-25T10:33:55.056 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32792.log 2024-04-25T10:33:55.057 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73395.log.gz 2024-04-25T10:33:55.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88611.log 2024-04-25T10:33:55.058 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89073.log.gz 2024-04-25T10:33:55.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82340.log 2024-04-25T10:33:55.058 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32792.log.gz 2024-04-25T10:33:55.059 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87725.log 2024-04-25T10:33:55.059 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.88611.log.gz 2024-04-25T10:33:55.060 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70323.log 2024-04-25T10:33:55.060 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82340.log.gz 2024-04-25T10:33:55.060 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-04-25T10:33:55.061 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87725.log.gz 2024-04-25T10:33:55.061 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52722.log 2024-04-25T10:33:55.062 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70323.log.gz 2024-04-25T10:33:55.062 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32600.log 2024-04-25T10:33:55.062 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-04-25T10:33:55.063 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77655.log 2024-04-25T10:33:55.063 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52722.log.gz 2024-04-25T10:33:55.064 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48752.log 2024-04-25T10:33:55.064 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32600.log.gz 2024-04-25T10:33:55.065 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86075.log 2024-04-25T10:33:55.065 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77655.log.gz 2024-04-25T10:33:55.066 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87205.log 2024-04-25T10:33:55.066 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48752.log.gz 2024-04-25T10:33:55.066 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67403.log 2024-04-25T10:33:55.067 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86075.log.gz 2024-04-25T10:33:55.067 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80586.log 2024-04-25T10:33:55.067 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87205.log.gz 2024-04-25T10:33:55.068 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77333.log 2024-04-25T10:33:55.068 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67403.log.gz 2024-04-25T10:33:55.069 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72429.log 2024-04-25T10:33:55.069 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80586.log.gz 2024-04-25T10:33:55.070 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88870.log 2024-04-25T10:33:55.070 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77333.log.gz 2024-04-25T10:33:55.070 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52582.log 2024-04-25T10:33:55.071 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72429.log.gz 2024-04-25T10:33:55.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70070.log 2024-04-25T10:33:55.072 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52582.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.88870.log.gz 2024-04-25T10:33:55.072 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66967.log 2024-04-25T10:33:55.073 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52582.log.gz 2024-04-25T10:33:55.073 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82828.log 2024-04-25T10:33:55.073 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70070.log.gz 2024-04-25T10:33:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69228.log 2024-04-25T10:33:55.074 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66967.log.gz 2024-04-25T10:33:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76410.log 2024-04-25T10:33:55.075 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82828.log.gz 2024-04-25T10:33:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49402.log 2024-04-25T10:33:55.076 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69228.log.gz 2024-04-25T10:33:55.077 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52016.log 2024-04-25T10:33:55.077 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76410.log.gz 2024-04-25T10:33:55.077 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89310.log 2024-04-25T10:33:55.077 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49402.log.gz 2024-04-25T10:33:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89284.log 2024-04-25T10:33:55.078 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52016.log.gz 2024-04-25T10:33:55.079 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2024-04-25T10:33:55.080 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89310.log.gz 2024-04-25T10:33:55.080 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86533.log 2024-04-25T10:33:55.080 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89284.log.gz 2024-04-25T10:33:55.081 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84837.log 2024-04-25T10:33:55.081 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2024-04-25T10:33:55.082 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77164.log 2024-04-25T10:33:55.082 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86533.log.gz 2024-04-25T10:33:55.083 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79246.log 2024-04-25T10:33:55.083 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84837.log.gz 2024-04-25T10:33:55.083 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74035.log 2024-04-25T10:33:55.083 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77164.log.gz 2024-04-25T10:33:55.084 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70701.log 2024-04-25T10:33:55.084 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79246.log.gz 2024-04-25T10:33:55.085 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76546.log 2024-04-25T10:33:55.085 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74035.log.gz 2024-04-25T10:33:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54231.log 2024-04-25T10:33:55.086 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70701.log.gz 2024-04-25T10:33:55.086 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49150.log 2024-04-25T10:33:55.087 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76546.log.gz 2024-04-25T10:33:55.087 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34034.log 2024-04-25T10:33:55.088 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54231.log.gz 2024-04-25T10:33:55.088 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49150.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:33:55.088 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49150.log.gz 2024-04-25T10:33:55.089 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90059.log 2024-04-25T10:33:55.089 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34034.log.gz 2024-04-25T10:33:55.090 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49320.log 2024-04-25T10:33:55.091 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35148.log 2024-04-25T10:33:55.091 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90059.log.gz 2024-04-25T10:33:55.092 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75885.log 2024-04-25T10:33:55.092 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49320.log.gz 2024-04-25T10:33:55.093 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35148.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35148.log.gz 2024-04-25T10:33:55.093 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66993.log 2024-04-25T10:33:55.094 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32803.log 2024-04-25T10:33:55.094 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75885.log.gz 2024-04-25T10:33:55.095 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66993.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66993.log.gz 2024-04-25T10:33:55.095 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89981.log 2024-04-25T10:33:55.095 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90293.log 2024-04-25T10:33:55.096 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32803.log: /var/log/ceph/ceph-client.admin.89981.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32803.log.gz 2024-04-25T10:33:55.096 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.34597.log 2024-04-25T10:33:55.096 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89981.log.gz 2024-04-25T10:33:55.097 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89903.log 2024-04-25T10:33:55.097 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90293.log.gz 2024-04-25T10:33:55.098 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84214.log 2024-04-25T10:33:55.098 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34597.log.gz 2024-04-25T10:33:55.099 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51596.log 2024-04-25T10:33:55.099 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89903.log.gz 2024-04-25T10:33:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84214.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.84214.log.gz -- 2024-04-25T10:33:55.100 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.53459.log 2024-04-25T10:33:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85464.log 2024-04-25T10:33:55.101 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51596.log.gz 2024-04-25T10:33:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53459.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53459.log.gz -5 2024-04-25T10:33:55.102 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86937.log 2024-04-25T10:33:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49945.log 2024-04-25T10:33:55.103 INFO:teuthology.orchestra.run.smithi045.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.85464.log.gz 2024-04-25T10:33:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53355.log 2024-04-25T10:33:55.104 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.86937.log.gz 2024-04-25T10:33:55.105 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83392.log 2024-04-25T10:33:55.105 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49945.log.gz 2024-04-25T10:33:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34940.log 2024-04-25T10:33:55.106 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53355.log.gz 2024-04-25T10:33:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88676.log 2024-04-25T10:33:55.107 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83392.log.gz 2024-04-25T10:33:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79891.log 2024-04-25T10:33:55.108 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34940.log.gz 2024-04-25T10:33:55.109 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77439.log 2024-04-25T10:33:55.109 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88676.log.gz 2024-04-25T10:33:55.110 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88150.log 2024-04-25T10:33:55.110 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79891.log.gz 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35835.log 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77439.log.gz 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51362.log 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88150.log.gz 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35835.log.gz 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82001.log 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51362.log.gz 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88476.log 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-04-25T10:33:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79820.log 2024-04-25T10:33:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88476.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.82001.log.gz 2024-04-25T10:33:55.118 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88476.log.gz 2024-04-25T10:33:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52124.log 2024-04-25T10:33:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53589.log 2024-04-25T10:33:55.118 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79820.log.gz 2024-04-25T10:33:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67666.log 2024-04-25T10:33:55.119 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52124.log.gz 2024-04-25T10:33:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82522.log 2024-04-25T10:33:55.119 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53589.log.gz 2024-04-25T10:33:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52460.log 2024-04-25T10:33:55.120 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67666.log.gz 2024-04-25T10:33:55.121 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79084.log 2024-04-25T10:33:55.121 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82522.log.gz 2024-04-25T10:33:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-04-25T10:33:55.122 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52460.log.gz 2024-04-25T10:33:55.123 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73957.log 2024-04-25T10:33:55.123 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79084.log.gz 2024-04-25T10:33:55.123 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50534.log 2024-04-25T10:33:55.124 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-04-25T10:33:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88256.log 2024-04-25T10:33:55.124 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73957.log.gz 2024-04-25T10:33:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82230.log 2024-04-25T10:33:55.125 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50534.log.gz 2024-04-25T10:33:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51702.log 2024-04-25T10:33:55.126 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88256.log.gz 2024-04-25T10:33:55.127 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52096.log 2024-04-25T10:33:55.127 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82230.log.gz 2024-04-25T10:33:55.128 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72533.log 2024-04-25T10:33:55.128 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2024-04-25T10:33:55.129 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66753.log 2024-04-25T10:33:55.129 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52096.log.gz 2024-04-25T10:33:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72533.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:33:55.130 INFO:teuthology.orchestra.run.smithi045.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.72533.log.gz 2024-04-25T10:33:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88332.log 2024-04-25T10:33:55.131 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66753.log.gz 2024-04-25T10:33:55.132 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35044.log 2024-04-25T10:33:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79638.log 2024-04-25T10:33:55.133 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.88332.log.gz 2024-04-25T10:33:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85789.log 2024-04-25T10:33:55.134 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35044.log.gz 2024-04-25T10:33:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89201.log 2024-04-25T10:33:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85789.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79638.log.gz 2024-04-25T10:33:55.136 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85789.log.gz 2024-04-25T10:33:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75707.log 2024-04-25T10:33:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89641.log 2024-04-25T10:33:55.138 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89201.log: /var/log/ceph/ceph-client.admin.75707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89258.log 2024-04-25T10:33:55.138 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.89201.log.gz 2024-04-25T10:33:55.139 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75707.log.gz 2024-04-25T10:33:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-04-25T10:33:55.139 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89641.log.gz 2024-04-25T10:33:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67436.log 2024-04-25T10:33:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89258.log: /var/log/ceph/ceph-client.admin.35174.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89258.log.gz 2024-04-25T10:33:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89773.log 2024-04-25T10:33:55.141 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gz 2024-04-25T10:33:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83837.log 2024-04-25T10:33:55.142 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67436.log.gz 2024-04-25T10:33:55.143 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.89773.log: --verbose -- /var/log/ceph/ceph-client.admin.49895.log 2024-04-25T10:33:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83837.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89773.log.gz 2024-04-25T10:33:55.144 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83837.log.gz 2024-04-25T10:33:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87096.log 2024-04-25T10:33:55.145 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.49895.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.86390.log 2024-04-25T10:33:55.146 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87096.log: gzip -5 --verbose 91.2% -- /var/log/ceph/ceph-client.admin.51466.log -- replaced with /var/log/ceph/ceph-client.admin.49895.log.gz 2024-04-25T10:33:55.146 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-25T10:33:55.146 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87096.log.gz 2024-04-25T10:33:55.147 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70727.log 2024-04-25T10:33:55.147 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86390.log.gz 2024-04-25T10:33:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85549.log 2024-04-25T10:33:55.149 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51466.log.gz 2024-04-25T10:33:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69584.log 2024-04-25T10:33:55.149 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70727.log.gz 2024-04-25T10:33:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-04-25T10:33:55.151 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85549.log.gz 2024-04-25T10:33:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35991.log 2024-04-25T10:33:55.152 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69584.log.gz 2024-04-25T10:33:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32104.log 2024-04-25T10:33:55.153 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-04-25T10:33:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35991.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.35991.log.gz /var/log/ceph/ceph-client.admin.87465.log 2024-04-25T10:33:55.154 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-25T10:33:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36067.log 2024-04-25T10:33:55.155 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32104.log.gz 2024-04-25T10:33:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83444.log 2024-04-25T10:33:55.156 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87465.log.gz 2024-04-25T10:33:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72455.log 2024-04-25T10:33:55.157 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36067.log.gz 2024-04-25T10:33:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88740.log 2024-04-25T10:33:55.158 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83444.log.gz 2024-04-25T10:33:55.159 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50791.log 2024-04-25T10:33:55.159 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72455.log.gz 2024-04-25T10:33:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53141.log 2024-04-25T10:33:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88740.log: /var/log/ceph/ceph-client.admin.50791.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88740.log.gz 2024-04-25T10:33:55.161 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35965.log 2024-04-25T10:33:55.161 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50791.log.gz 2024-04-25T10:33:55.162 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32338.log 2024-04-25T10:33:55.162 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53141.log.gz 2024-04-25T10:33:55.163 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89388.log 2024-04-25T10:33:55.164 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35965.log.gz 2024-04-25T10:33:55.164 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87257.log 2024-04-25T10:33:55.165 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32338.log.gz 2024-04-25T10:33:55.165 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87666.log 2024-04-25T10:33:55.165 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89388.log.gz 2024-04-25T10:33:55.167 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50128.log 2024-04-25T10:33:55.167 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87257.log.gz 2024-04-25T10:33:55.168 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:33:55.168 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32591.log 2024-04-25T10:33:55.168 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87666.log: /var/log/ceph/ceph-client.admin.50128.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87666.log.gz 2024-04-25T10:33:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67694.log 2024-04-25T10:33:55.169 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50128.log.gz 2024-04-25T10:33:55.169 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33274.log 2024-04-25T10:33:55.169 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32591.log.gz 2024-04-25T10:33:55.170 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32286.log 2024-04-25T10:33:55.171 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33274.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67694.log.gz 2024-04-25T10:33:55.171 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33274.log.gz 2024-04-25T10:33:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54179.log 2024-04-25T10:33:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32379.log/var/log/ceph/ceph-client.admin.32286.log: 2024-04-25T10:33:55.173 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32286.log.gz 2024-04-25T10:33:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33196.log 2024-04-25T10:33:55.174 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54179.log.gz 2024-04-25T10:33:55.175 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70781.log 2024-04-25T10:33:55.175 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32379.log.gz 2024-04-25T10:33:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67298.log 2024-04-25T10:33:55.176 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33196.log: /var/log/ceph/ceph-client.admin.70781.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33196.log.gz 2024-04-25T10:33:55.177 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51041.log 2024-04-25T10:33:55.177 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70781.log.gz 2024-04-25T10:33:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67298.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67298.log.gz 2024-04-25T10:33:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67351.log 2024-04-25T10:33:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86336.log 2024-04-25T10:33:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67351.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51041.log.gz 2024-04-25T10:33:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72633.log 2024-04-25T10:33:55.180 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67351.log.gz 2024-04-25T10:33:55.181 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80232.log 2024-04-25T10:33:55.181 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86336.log.gz 2024-04-25T10:33:55.182 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.72633.log: /var/log/ceph/ceph-client.admin.35402.log 2024-04-25T10:33:55.182 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80232.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72633.log.gz 2024-04-25T10:33:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33378.log 2024-04-25T10:33:55.183 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80232.log.gz 2024-04-25T10:33:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73220.log 2024-04-25T10:33:55.184 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35402.log.gz 2024-04-25T10:33:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2024-04-25T10:33:55.186 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73220.log: gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.33378.log.gz -5 2024-04-25T10:33:55.186 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86663.log 2024-04-25T10:33:55.186 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73220.log.gz 2024-04-25T10:33:55.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53980.log 2024-04-25T10:33:55.187 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2024-04-25T10:33:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90137.log 2024-04-25T10:33:55.189 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53980.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86663.log.gz 2024-04-25T10:33:55.189 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73194.log 2024-04-25T10:33:55.189 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53980.log.gz 2024-04-25T10:33:55.190 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35122.log 2024-04-25T10:33:55.190 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90137.log.gz 2024-04-25T10:33:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.73194.log: -5 --verbose -- /var/log/ceph/ceph-client.0.53038.log 2024-04-25T10:33:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35122.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73194.log.gz 2024-04-25T10:33:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51898.log 2024-04-25T10:33:55.193 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35122.log.gz 2024-04-25T10:33:55.193 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.53038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53038.log.gz 2024-04-25T10:33:55.193 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86418.log 2024-04-25T10:33:55.194 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48927.log 2024-04-25T10:33:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51898.log: /var/log/ceph/ceph-client.admin.86418.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51898.log.gz 2024-04-25T10:33:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53485.log 2024-04-25T10:33:55.196 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86418.log.gz 2024-04-25T10:33:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89518.log 2024-04-25T10:33:55.197 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48927.log.gz 2024-04-25T10:33:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88044.log 2024-04-25T10:33:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89518.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53485.log.gz 2024-04-25T10:33:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2024-04-25T10:33:55.199 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89518.log.gz 2024-04-25T10:33:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74087.log 2024-04-25T10:33:55.200 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88044.log.gz 2024-04-25T10:33:55.200 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76773.log 2024-04-25T10:33:55.201 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49241.log: 88.0%/var/log/ceph/ceph-client.admin.74087.log: -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2024-04-25T10:33:55.202 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84703.log 2024-04-25T10:33:55.202 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74087.log.gz 2024-04-25T10:33:55.203 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72481.log 2024-04-25T10:33:55.203 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76773.log.gz 2024-04-25T10:33:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33746.log 2024-04-25T10:33:55.204 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84703.log: /var/log/ceph/ceph-client.admin.72481.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84703.log.gz 2024-04-25T10:33:55.205 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72874.log 2024-04-25T10:33:55.205 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72481.log.gz 2024-04-25T10:33:55.206 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33774.log 2024-04-25T10:33:55.206 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33746.log.gz 2024-04-25T10:33:55.207 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32078.log 2024-04-25T10:33:55.207 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72874.log.gz 2024-04-25T10:33:55.208 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33014.log 2024-04-25T10:33:55.208 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33774.log.gz 2024-04-25T10:33:55.209 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87044.log 2024-04-25T10:33:55.209 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32078.log.gz 2024-04-25T10:33:55.210 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84292.log 2024-04-25T10:33:55.211 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33014.log: /var/log/ceph/ceph-client.admin.87044.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33014.log.gz 2024-04-25T10:33:55.211 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84441.log 2024-04-25T10:33:55.211 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87044.log.gz 2024-04-25T10:33:55.212 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88230.log 2024-04-25T10:33:55.213 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84292.log.gz 2024-04-25T10:33:55.213 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75937.log 2024-04-25T10:33:55.213 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84441.log.gz 2024-04-25T10:33:55.214 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88230.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88230.log.gz 2024-04-25T10:33:55.214 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66542.log 2024-04-25T10:33:55.215 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53927.log 2024-04-25T10:33:55.216 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66542.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.75937.log.gzgzip 2024-04-25T10:33:55.216 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35323.log 2024-04-25T10:33:55.217 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66542.log.gz 2024-04-25T10:33:55.217 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51866.log 2024-04-25T10:33:55.218 INFO:teuthology.orchestra.run.smithi045.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53927.log.gz 2024-04-25T10:33:55.218 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86479.log 2024-04-25T10:33:55.219 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35323.log.gz 2024-04-25T10:33:55.219 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70987.log 2024-04-25T10:33:55.220 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51866.log.gz 2024-04-25T10:33:55.221 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33300.log 2024-04-25T10:33:55.221 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86479.log.gz 2024-04-25T10:33:55.222 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83206.log 2024-04-25T10:33:55.222 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70987.log: /var/log/ceph/ceph-client.admin.33300.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70987.log.gz 2024-04-25T10:33:55.223 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76273.log 2024-04-25T10:33:55.223 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33300.log.gz 2024-04-25T10:33:55.224 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83206.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83206.log.gzgzip 2024-04-25T10:33:55.224 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34701.log 2024-04-25T10:33:55.225 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70044.log 2024-04-25T10:33:55.225 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76273.log: 88.1%/var/log/ceph/ceph-client.admin.34701.log: -- replaced with /var/log/ceph/ceph-client.admin.76273.log.gz 2024-04-25T10:33:55.226 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73090.log 2024-04-25T10:33:55.226 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34701.log.gz 2024-04-25T10:33:55.227 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79690.log 2024-04-25T10:33:55.227 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70044.log.gz 2024-04-25T10:33:55.228 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66644.log 2024-04-25T10:33:55.229 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73090.log.gz 2024-04-25T10:33:55.229 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76598.log 2024-04-25T10:33:55.230 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79690.log.gz 2024-04-25T10:33:55.230 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50710.log 2024-04-25T10:33:55.231 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.66644.log.gz 2024-04-25T10:33:55.231 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79502.log 2024-04-25T10:33:55.231 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76598.log.gz 2024-04-25T10:33:55.232 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.50710.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-04-25T10:33:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79502.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50710.log.gz 2024-04-25T10:33:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72984.log 2024-04-25T10:33:55.234 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79502.log.gz 2024-04-25T10:33:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89125.log 2024-04-25T10:33:55.236 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-04-25T10:33:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69992.log 2024-04-25T10:33:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72984.log: /var/log/ceph/ceph-client.admin.89125.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72984.log.gz 2024-04-25T10:33:55.236 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70911.log 2024-04-25T10:33:55.237 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.89125.log.gz 2024-04-25T10:33:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87562.log 2024-04-25T10:33:55.238 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69992.log.gz 2024-04-25T10:33:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52202.log 2024-04-25T10:33:55.239 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70911.log.gz 2024-04-25T10:33:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87562.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87562.log.gz 2024-04-25T10:33:55.240 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73656.log 2024-04-25T10:33:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32790.log 2024-04-25T10:33:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52202.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52202.log.gz 2024-04-25T10:33:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54283.log 2024-04-25T10:33:55.242 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73656.log.gz 2024-04-25T10:33:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79398.log 2024-04-25T10:33:55.243 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32790.log.gz 2024-04-25T10:33:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54283.log: gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.54283.log.gz /var/log/ceph/ceph-client.admin.76063.log 2024-04-25T10:33:55.244 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-25T10:33:55.245 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.79398.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53381.log 2024-04-25T10:33:55.245 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79398.log.gz 2024-04-25T10:33:55.246 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69506.log 2024-04-25T10:33:55.246 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76063.log.gz 2024-04-25T10:33:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66941.log 2024-04-25T10:33:55.247 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53381.log.gz 2024-04-25T10:33:55.248 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67965.log 2024-04-25T10:33:55.248 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69506.log.gz 2024-04-25T10:33:55.249 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27519.log 2024-04-25T10:33:55.249 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66941.log.gz 2024-04-25T10:33:55.250 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-04-25T10:33:55.250 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67965.log.gz 2024-04-25T10:33:55.251 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.27519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72685.log 2024-04-25T10:33:55.252 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34862.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.27519.log.gz 2024-04-25T10:33:55.252 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73369.log 2024-04-25T10:33:55.252 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-04-25T10:33:55.253 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32387.log 2024-04-25T10:33:55.253 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72685.log.gz 2024-04-25T10:33:55.254 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73931.log 2024-04-25T10:33:55.254 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73369.log.gz 2024-04-25T10:33:55.255 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80180.log 2024-04-25T10:33:55.256 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32387.log.gz 2024-04-25T10:33:55.257 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67175.log 2024-04-25T10:33:55.257 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73931.log.gz 2024-04-25T10:33:55.257 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.80180.log: /var/log/ceph/ceph-client.admin.34568.log 2024-04-25T10:33:55.258 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80180.log.gz 2024-04-25T10:33:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84651.log 2024-04-25T10:33:55.259 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67175.log.gz 2024-04-25T10:33:55.259 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80638.log 2024-04-25T10:33:55.260 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34568.log.gz 2024-04-25T10:33:55.261 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83260.log 2024-04-25T10:33:55.261 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84651.log.gz 2024-04-25T10:33:55.262 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67377.log 2024-04-25T10:33:55.263 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83260.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.80638.log.gz 2024-04-25T10:33:55.263 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86871.log 2024-04-25T10:33:55.263 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83260.log.gz 2024-04-25T10:33:55.264 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78880.log 2024-04-25T10:33:55.264 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67377.log.gz 2024-04-25T10:33:55.265 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69378.log 2024-04-25T10:33:55.265 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86871.log.gz 2024-04-25T10:33:55.266 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.78880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67612.log 2024-04-25T10:33:55.267 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69378.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78880.log.gz 2024-04-25T10:33:55.267 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67638.log 2024-04-25T10:33:55.267 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69378.log.gz 2024-04-25T10:33:55.268 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50156.log 2024-04-25T10:33:55.268 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67612.log.gz 2024-04-25T10:33:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52254.log 2024-04-25T10:33:55.269 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67638.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67638.log.gz 2024-04-25T10:33:55.270 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85815.log 2024-04-25T10:33:55.270 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz 2024-04-25T10:33:55.271 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70640.log 2024-04-25T10:33:55.271 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52254.log.gz 2024-04-25T10:33:55.272 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73799.log 2024-04-25T10:33:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85815.log: /var/log/ceph/ceph-client.admin.70640.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85815.log.gz 2024-04-25T10:33:55.273 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34112.log 2024-04-25T10:33:55.273 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70640.log.gz 2024-04-25T10:33:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72796.log 2024-04-25T10:33:55.274 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73799.log.gz 2024-04-25T10:33:55.275 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89589.log 2024-04-25T10:33:55.275 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34112.log.gz 2024-04-25T10:33:55.276 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32376.log 2024-04-25T10:33:55.277 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72796.log.gz 2024-04-25T10:33:55.277 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70245.log 2024-04-25T10:33:55.277 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89589.log.gz 2024-04-25T10:33:55.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32376.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:33:55.279 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32376.log.gz 2024-04-25T10:33:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31865.log 2024-04-25T10:33:55.280 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70245.log.gz 2024-04-25T10:33:55.280 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76650.log 2024-04-25T10:33:55.282 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83607.log 2024-04-25T10:33:55.282 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31865.log.gz 2024-04-25T10:33:55.283 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69610.log 2024-04-25T10:33:55.283 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76650.log.gz 2024-04-25T10:33:55.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69721.log 2024-04-25T10:33:55.284 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83607.log.gz 2024-04-25T10:33:55.285 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86283.log 2024-04-25T10:33:55.285 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69610.log.gz 2024-04-25T10:33:55.286 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79943.log 2024-04-25T10:33:55.287 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69721.log.gz 2024-04-25T10:33:55.287 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69799.log 2024-04-25T10:33:55.288 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86283.log.gz 2024-04-25T10:33:55.288 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67809.log 2024-04-25T10:33:55.289 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79943.log.gz 2024-04-25T10:33:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52888.log 2024-04-25T10:33:55.290 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69799.log.gz 2024-04-25T10:33:55.290 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32791.log 2024-04-25T10:33:55.291 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67809.log.gz 2024-04-25T10:33:55.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79995.log 2024-04-25T10:33:55.292 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52888.log.gz 2024-04-25T10:33:55.293 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36129.log 2024-04-25T10:33:55.293 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32791.log.gz 2024-04-25T10:33:55.294 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48804.log 2024-04-25T10:33:55.295 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79995.log.gz 2024-04-25T10:33:55.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67559.log 2024-04-25T10:33:55.295 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36129.log.gz 2024-04-25T10:33:55.297 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70096.log 2024-04-25T10:33:55.297 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48804.log.gz 2024-04-25T10:33:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79110.log 2024-04-25T10:33:55.298 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67559.log.gz 2024-04-25T10:33:55.298 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83314.log 2024-04-25T10:33:55.299 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70096.log.gz 2024-04-25T10:33:55.300 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89362.log 2024-04-25T10:33:55.300 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79110.log.gz 2024-04-25T10:33:55.301 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53167.log 2024-04-25T10:33:55.301 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83314.log.gz 2024-04-25T10:33:55.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83037.log 2024-04-25T10:33:55.302 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89362.log.gz 2024-04-25T10:33:55.303 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89825.log 2024-04-25T10:33:55.303 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.53167.log.gz 2024-04-25T10:33:55.304 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77413.log 2024-04-25T10:33:55.304 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83037.log.gz 2024-04-25T10:33:55.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33694.log 2024-04-25T10:33:55.305 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89825.log.gz 2024-04-25T10:33:55.307 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87510.log 2024-04-25T10:33:55.307 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77413.log.gz 2024-04-25T10:33:55.308 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76825.log 2024-04-25T10:33:55.308 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33694.log.gz 2024-04-25T10:33:55.308 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83089.log 2024-04-25T10:33:55.309 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87510.log.gz 2024-04-25T10:33:55.310 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88176.log 2024-04-25T10:33:55.310 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76825.log.gz 2024-04-25T10:33:55.311 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87614.log 2024-04-25T10:33:55.311 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83089.log.gz 2024-04-25T10:33:55.312 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82697.log 2024-04-25T10:33:55.312 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88176.log.gz 2024-04-25T10:33:55.313 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82671.log 2024-04-25T10:33:55.313 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87614.log.gz 2024-04-25T10:33:55.314 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72507.log 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82697.log.gz 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49662.log 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82671.log.gz 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33904.log 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72507.log.gz 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67468.log 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33904.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.49662.log.gz 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88098.log 2024-04-25T10:33:55.320 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33904.log.gz 2024-04-25T10:33:55.321 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76955.log 2024-04-25T10:33:55.321 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67468.log.gz 2024-04-25T10:33:55.321 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70349.log 2024-04-25T10:33:55.321 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88098.log.gz 2024-04-25T10:33:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52842.log 2024-04-25T10:33:55.322 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76955.log.gz 2024-04-25T10:33:55.323 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89929.log 2024-04-25T10:33:55.324 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70349.log.gz 2024-04-25T10:33:55.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80482.log 2024-04-25T10:33:55.325 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52842.log.gz 2024-04-25T10:33:55.325 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50819.log 2024-04-25T10:33:55.326 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89929.log.gz 2024-04-25T10:33:55.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50047.log 2024-04-25T10:33:55.327 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.80482.log.gz 2024-04-25T10:33:55.328 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34328.log 2024-04-25T10:33:55.330 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50047.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50819.log.gz 2024-04-25T10:33:55.330 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73317.log 2024-04-25T10:33:55.330 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50047.log.gz/var/log/ceph/ceph-client.admin.34328.log: 2024-04-25T10:33:55.331 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69914.log 2024-04-25T10:33:55.331 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34328.log.gz 2024-04-25T10:33:55.331 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32182.log 2024-04-25T10:33:55.332 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73317.log.gz 2024-04-25T10:33:55.332 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77465.log 2024-04-25T10:33:55.333 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69914.log.gz 2024-04-25T10:33:55.334 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83063.log 2024-04-25T10:33:55.334 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32182.log.gz 2024-04-25T10:33:55.335 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87231.log 2024-04-25T10:33:55.335 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77465.log.gz 2024-04-25T10:33:55.336 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32156.log 2024-04-25T10:33:55.336 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83063.log.gz 2024-04-25T10:33:55.337 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67729.log 2024-04-25T10:33:55.337 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87231.log.gz 2024-04-25T10:33:55.338 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51518.log 2024-04-25T10:33:55.338 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32156.log.gz 2024-04-25T10:33:55.339 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73603.log 2024-04-25T10:33:55.339 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67729.log.gz 2024-04-25T10:33:55.340 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33982.log 2024-04-25T10:33:55.341 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51518.log.gz 2024-04-25T10:33:55.342 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87439.log 2024-04-25T10:33:55.342 INFO:teuthology.orchestra.run.smithi045.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.73603.log.gz 2024-04-25T10:33:55.342 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32208.log 2024-04-25T10:33:55.343 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33982.log.gz 2024-04-25T10:33:55.344 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79450.log 2024-04-25T10:33:55.344 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87439.log.gz 2024-04-25T10:33:55.345 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77639.log 2024-04-25T10:33:55.345 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32208.log.gz 2024-04-25T10:33:55.346 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34542.log 2024-04-25T10:33:55.346 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79450.log.gz 2024-04-25T10:33:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.77639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77639.log.gz 2024-04-25T10:33:55.347 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87536.log 2024-04-25T10:33:55.348 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85497.log 2024-04-25T10:33:55.349 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34542.log.gz 2024-04-25T10:33:55.349 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51252.log 2024-04-25T10:33:55.350 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87536.log.gz 2024-04-25T10:33:55.351 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85497.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:33:55.351 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85497.log.gz 2024-04-25T10:33:55.352 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85389.log 2024-04-25T10:33:55.352 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51252.log.gz 2024-04-25T10:33:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85841.log 2024-04-25T10:33:55.354 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70859.log 2024-04-25T10:33:55.355 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0%/var/log/ceph/ceph-client.admin.85841.log: -- replaced with /var/log/ceph/ceph-client.admin.85389.log.gz 2024-04-25T10:33:55.355 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82626.log 2024-04-25T10:33:55.355 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85841.log.gz 2024-04-25T10:33:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53511.log 2024-04-25T10:33:55.357 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70859.log.gz 2024-04-25T10:33:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82626.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82626.log.gz 2024-04-25T10:33:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73265.log 2024-04-25T10:33:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-04-25T10:33:55.360 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53511.log.gz 2024-04-25T10:33:55.361 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33590.log 2024-04-25T10:33:55.361 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73265.log.gz 2024-04-25T10:33:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52556.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-04-25T10:33:55.362 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84084.log 2024-04-25T10:33:55.365 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33590.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33590.log.gz 2024-04-25T10:33:55.365 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.84084.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80688.log 2024-04-25T10:33:55.365 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84084.log.gz 2024-04-25T10:33:55.366 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49869.log 2024-04-25T10:33:55.367 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86505.log 2024-04-25T10:33:55.367 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80688.log.gz 2024-04-25T10:33:55.368 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49869.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49869.log.gz 2024-04-25T10:33:55.369 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86505.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86505.log.gz 2024-04-25T10:33:55.370 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68041.log 2024-04-25T10:33:55.371 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80612.log 2024-04-25T10:33:55.372 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35376.log 2024-04-25T10:33:55.373 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68041.log.gz 2024-04-25T10:33:55.373 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80612.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80612.log.gz 2024-04-25T10:33:55.374 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35376.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35376.log.gz 2024-04-25T10:33:55.375 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53303.log 2024-04-25T10:33:55.376 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73577.log 2024-04-25T10:33:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80082.log 2024-04-25T10:33:55.378 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53303.log.gz 2024-04-25T10:33:55.378 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73577.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73577.log.gz 2024-04-25T10:33:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80082.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80082.log.gz 2024-04-25T10:33:55.380 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83494.log 2024-04-25T10:33:55.381 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82496.log 2024-04-25T10:33:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82148.log 2024-04-25T10:33:55.383 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83494.log.gz 2024-04-25T10:33:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82496.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82496.log.gz 2024-04-25T10:33:55.386 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82148.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.82148.log.gz 2024-04-25T10:33:55.386 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87822.log 2024-04-25T10:33:55.387 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2024-04-25T10:33:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84863.log 2024-04-25T10:33:55.389 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87822.log.gz 2024-04-25T10:33:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52410.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-04-25T10:33:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84863.log.gz 2024-04-25T10:33:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87179.log 2024-04-25T10:33:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50378.log 2024-04-25T10:33:55.394 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78830.log 2024-04-25T10:33:55.394 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87179.log.gz 2024-04-25T10:33:55.395 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50378.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50378.log.gz 2024-04-25T10:33:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.78830.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78830.log.gz 2024-04-25T10:33:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86689.log 2024-04-25T10:33:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50203.log 2024-04-25T10:33:55.400 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85284.log 2024-04-25T10:33:55.400 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.86689.log.gz 2024-04-25T10:33:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50203.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50203.log.gz 2024-04-25T10:33:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85284.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.85284.log.gz 2024-04-25T10:33:55.403 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50762.log 2024-04-25T10:33:55.404 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83180.log 2024-04-25T10:33:55.405 INFO:teuthology.orchestra.run.smithi117.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:33:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77020.log 2024-04-25T10:33:55.406 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50762.log.gz 2024-04-25T10:33:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83180.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83180.log.gz 2024-04-25T10:33:55.407 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77020.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77020.log.gz 2024-04-25T10:33:55.408 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48569.log 2024-04-25T10:33:55.409 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-04-25T10:33:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67220.log 2024-04-25T10:33:55.411 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48569.log.gz 2024-04-25T10:33:55.411 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51304.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-04-25T10:33:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67220.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67220.log.gz 2024-04-25T10:33:55.413 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72325.log 2024-04-25T10:33:55.414 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87070.log 2024-04-25T10:33:55.416 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-04-25T10:33:55.416 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72325.log.gz 2024-04-25T10:33:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87070.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87070.log.gz 2024-04-25T10:33:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-04-25T10:33:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69747.log 2024-04-25T10:33:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75525.log 2024-04-25T10:33:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67246.log 2024-04-25T10:33:55.421 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69747.log.gz 2024-04-25T10:33:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75525.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75525.log.gz 2024-04-25T10:33:55.423 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67246.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67246.log.gz 2024-04-25T10:33:55.423 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79313.log 2024-04-25T10:33:55.424 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69773.log 2024-04-25T10:33:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.33672.log 2024-04-25T10:33:55.426 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79313.log.gz 2024-04-25T10:33:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69773.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69773.log.gz 2024-04-25T10:33:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.2.33672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.33672.log.gz 2024-04-25T10:33:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52384.log 2024-04-25T10:33:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50428.log 2024-04-25T10:33:55.431 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79162.log 2024-04-25T10:33:55.431 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52384.log.gz 2024-04-25T10:33:55.432 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50428.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50428.log.gz 2024-04-25T10:33:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79162.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79162.log.gz 2024-04-25T10:33:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53115.log 2024-04-25T10:33:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-04-25T10:33:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83922.log 2024-04-25T10:33:55.437 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53115.log.gz 2024-04-25T10:33:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51780.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-04-25T10:33:55.439 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83922.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83922.log.gz 2024-04-25T10:33:55.439 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32581.log 2024-04-25T10:33:55.440 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77281.log 2024-04-25T10:33:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34268.log 2024-04-25T10:33:55.442 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32581.log.gz 2024-04-25T10:33:55.443 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77281.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77281.log.gz 2024-04-25T10:33:55.444 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34268.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34268.log.gz 2024-04-25T10:33:55.444 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79742.log 2024-04-25T10:33:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34490.log 2024-04-25T10:33:55.447 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72958.log 2024-04-25T10:33:55.447 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79742.log.gz 2024-04-25T10:33:55.448 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34490.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34490.log.gz 2024-04-25T10:33:55.449 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72958.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72958.log.gz 2024-04-25T10:33:55.449 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-04-25T10:33:55.451 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35861.log 2024-04-25T10:33:55.452 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33930.log 2024-04-25T10:33:55.452 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-04-25T10:33:55.453 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35861.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35861.log.gz 2024-04-25T10:33:55.454 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32374.log 2024-04-25T10:33:55.455 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33930.log.gz 2024-04-25T10:33:55.455 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35096.log 2024-04-25T10:33:55.457 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87757.log 2024-04-25T10:33:55.457 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32374.log.gz 2024-04-25T10:33:55.458 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35096.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35096.log.gz 2024-04-25T10:33:55.459 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87757.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87757.log.gz 2024-04-25T10:33:55.460 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66857.log 2024-04-25T10:33:55.461 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70532.log 2024-04-25T10:33:55.462 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-04-25T10:33:55.462 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66857.log.gz 2024-04-25T10:33:55.463 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70532.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70532.log.gz 2024-04-25T10:33:55.464 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48433.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-04-25T10:33:55.465 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34354.log 2024-04-25T10:33:55.466 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77517.log 2024-04-25T10:33:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84677.log 2024-04-25T10:33:55.468 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34354.log.gz 2024-04-25T10:33:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77517.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77517.log.gz 2024-04-25T10:33:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84677.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84677.log.gz 2024-04-25T10:33:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82574.log 2024-04-25T10:33:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53745.log 2024-04-25T10:33:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50941.log 2024-04-25T10:33:55.473 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82574.log.gz 2024-04-25T10:33:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53745.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53745.log.gz 2024-04-25T10:33:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51278.log 2024-04-25T10:33:55.476 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70584.log 2024-04-25T10:33:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51278.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50941.log.gz 2024-04-25T10:33:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90319.log 2024-04-25T10:33:55.478 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51278.log.gz 2024-04-25T10:33:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70584.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70584.log.gz 2024-04-25T10:33:55.479 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90319.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90319.log.gz 2024-04-25T10:33:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76494.log 2024-04-25T10:33:55.481 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75577.log 2024-04-25T10:33:55.482 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-04-25T10:33:55.483 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76494.log.gz 2024-04-25T10:33:55.483 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75577.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75577.log.gz 2024-04-25T10:33:55.485 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34407.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-04-25T10:33:55.485 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33852.log 2024-04-25T10:33:55.486 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67585.log 2024-04-25T10:33:55.488 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77593.log 2024-04-25T10:33:55.488 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33852.log.gz 2024-04-25T10:33:55.489 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67585.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67585.log.gz 2024-04-25T10:33:55.490 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77593.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77593.log.gz 2024-04-25T10:33:55.490 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52940.log 2024-04-25T10:33:55.492 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33326.log 2024-04-25T10:33:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34190.log 2024-04-25T10:33:55.494 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33326.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.52940.log.gz 2024-04-25T10:33:55.495 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33326.log.gz 2024-04-25T10:33:55.495 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34190.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34190.log.gz 2024-04-25T10:33:55.496 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27756.log 2024-04-25T10:33:55.497 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70382.log 2024-04-25T10:33:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.27756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49610.log 2024-04-25T10:33:55.499 INFO:teuthology.orchestra.run.smithi045.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27756.log.gz 2024-04-25T10:33:55.499 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70382.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70382.log.gz 2024-04-25T10:33:55.500 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49610.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49610.log.gz 2024-04-25T10:33:55.501 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73983.log 2024-04-25T10:33:55.502 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73853.log 2024-04-25T10:33:55.503 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73738.log 2024-04-25T10:33:55.504 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73983.log.gz 2024-04-25T10:33:55.505 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73853.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73853.log.gz 2024-04-25T10:33:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73738.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73738.log.gz 2024-04-25T10:33:55.506 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49712.log 2024-04-25T10:33:55.507 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33878.log 2024-04-25T10:33:55.509 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67097.log 2024-04-25T10:33:55.509 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49712.log.gz 2024-04-25T10:33:55.510 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33878.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33878.log.gz 2024-04-25T10:33:55.510 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76929.log 2024-04-25T10:33:55.511 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72175.log 2024-04-25T10:33:55.513 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67097.log.gz 2024-04-25T10:33:55.513 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66779.log 2024-04-25T10:33:55.513 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76929.log.gz 2024-04-25T10:33:55.514 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72175.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.72175.log.gz 2024-04-25T10:33:55.515 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66779.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66779.log.gz 2024-04-25T10:33:55.516 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73036.log 2024-04-25T10:33:55.517 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76221.log 2024-04-25T10:33:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70219.log 2024-04-25T10:33:55.524 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73036.log.gz 2024-04-25T10:33:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76221.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76221.log.gz 2024-04-25T10:33:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70219.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70219.log.gz 2024-04-25T10:33:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31739.log 2024-04-25T10:33:55.524 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84993.log 2024-04-25T10:33:55.525 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67149.log 2024-04-25T10:33:55.525 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31739.log.gz 2024-04-25T10:33:55.525 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84993.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84993.log.gz 2024-04-25T10:33:55.526 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67149.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67149.log.gz 2024-04-25T10:33:55.527 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34294.log 2024-04-25T10:33:55.528 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:33:55.529 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80560.log 2024-04-25T10:33:55.530 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34294.log.gz 2024-04-25T10:33:55.530 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48778.log 2024-04-25T10:33:55.532 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80560.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80560.log.gz 2024-04-25T10:33:55.532 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-04-25T10:33:55.533 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48778.log.gz 2024-04-25T10:33:55.534 INFO:teuthology.orchestra.run.smithi045.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:33:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34732.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-04-25T10:33:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84389.log 2024-04-25T10:33:55.537 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87955.log 2024-04-25T10:33:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85763.log 2024-04-25T10:33:55.538 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84389.log.gz 2024-04-25T10:33:55.539 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89747.log 2024-04-25T10:33:55.539 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.87955.log.gz 2024-04-25T10:33:55.540 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85763.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85763.log.gz 2024-04-25T10:33:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89747.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89747.log.gz 2024-04-25T10:33:55.542 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48326.log 2024-04-25T10:33:55.543 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79136.log 2024-04-25T10:33:55.544 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82366.log 2024-04-25T10:33:55.545 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79136.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48326.log.gz 2024-04-25T10:33:55.545 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79136.log.gz 2024-04-25T10:33:55.546 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49033.log 2024-04-25T10:33:55.547 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82366.log.gz 2024-04-25T10:33:55.547 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86127.log 2024-04-25T10:33:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70297.log 2024-04-25T10:33:55.549 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49033.log.gz 2024-04-25T10:33:55.550 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86127.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86127.log.gz 2024-04-25T10:33:55.551 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72275.log 2024-04-25T10:33:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85019.log 2024-04-25T10:33:55.552 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70297.log.gz 2024-04-25T10:33:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-04-25T10:33:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85019.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.72275.log.gz 2024-04-25T10:33:55.555 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85019.log.gz 2024-04-25T10:33:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48595.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-04-25T10:33:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52150.log 2024-04-25T10:33:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-04-25T10:33:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75629.log 2024-04-25T10:33:55.559 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52150.log.gz 2024-04-25T10:33:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53693.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-04-25T10:33:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75629.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75629.log.gz 2024-04-25T10:33:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86743.log 2024-04-25T10:33:55.562 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88920.log 2024-04-25T10:33:55.564 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74009.log 2024-04-25T10:33:55.564 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86743.log.gz 2024-04-25T10:33:55.565 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88920.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88920.log.gz 2024-04-25T10:33:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74009.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.74009.log.gz 2024-04-25T10:33:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79192.log 2024-04-25T10:33:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83974.log 2024-04-25T10:33:55.569 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32780.log 2024-04-25T10:33:55.569 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79192.log.gz 2024-04-25T10:33:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83974.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83974.log.gz 2024-04-25T10:33:55.570 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49085.log 2024-04-25T10:33:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32780.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32780.log.gz 2024-04-25T10:33:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90007.log 2024-04-25T10:33:55.573 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49085.log.gz 2024-04-25T10:33:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81925.log 2024-04-25T10:33:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48537.log 2024-04-25T10:33:55.575 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90007.log.gz 2024-04-25T10:33:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.81925.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81925.log.gz 2024-04-25T10:33:55.577 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48537.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48537.log.gz 2024-04-25T10:33:55.578 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-04-25T10:33:55.579 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67272.log 2024-04-25T10:33:55.580 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33248.log 2024-04-25T10:33:55.580 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-04-25T10:33:55.582 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67272.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67272.log.gz 2024-04-25T10:33:55.582 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33248.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33248.log.gz 2024-04-25T10:33:55.583 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83288.log 2024-04-25T10:33:55.584 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76572.log 2024-04-25T10:33:55.586 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83117.log 2024-04-25T10:33:55.586 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83288.log.gz 2024-04-25T10:33:55.587 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48699.log 2024-04-25T10:33:55.587 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76572.log.gz 2024-04-25T10:33:55.588 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83117.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83117.log.gz 2024-04-25T10:33:55.589 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31892.log 2024-04-25T10:33:55.589 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48699.log.gz 2024-04-25T10:33:55.590 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90085.log 2024-04-25T10:33:55.591 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67913.log 2024-04-25T10:33:55.591 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31892.log.gz 2024-04-25T10:33:55.592 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90085.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90085.log.gz 2024-04-25T10:33:55.593 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67913.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67913.log.gz 2024-04-25T10:33:55.594 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89799.log 2024-04-25T10:33:55.595 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67019.log 2024-04-25T10:33:55.596 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67533.log 2024-04-25T10:33:55.596 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89799.log.gz 2024-04-25T10:33:55.597 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67019.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67019.log.gz 2024-04-25T10:33:55.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73682.log 2024-04-25T10:33:55.599 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67533.log.gz 2024-04-25T10:33:55.599 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76358.log 2024-04-25T10:33:55.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83787.log 2024-04-25T10:33:55.601 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73682.log.gz 2024-04-25T10:33:55.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76358.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76358.log.gz 2024-04-25T10:33:55.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83787.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83787.log.gz 2024-04-25T10:33:55.604 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70414.log 2024-04-25T10:33:55.605 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53407.log 2024-04-25T10:33:55.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68087.log 2024-04-25T10:33:55.607 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70414.log.gz 2024-04-25T10:33:55.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53407.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53407.log.gz 2024-04-25T10:33:55.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.68087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68087.log.gz 2024-04-25T10:33:55.609 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49376.log 2024-04-25T10:33:55.610 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86987.log 2024-04-25T10:33:55.612 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69558.log 2024-04-25T10:33:55.612 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49376.log.gz 2024-04-25T10:33:55.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86987.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.86987.log.gz 2024-04-25T10:33:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69558.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69558.log.gz 2024-04-25T10:33:55.614 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79339.log 2024-04-25T10:33:55.616 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48856.log 2024-04-25T10:33:55.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82282.log 2024-04-25T10:33:55.617 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79339.log.gz 2024-04-25T10:33:55.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48856.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48856.log.gz 2024-04-25T10:33:55.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82282.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82282.log.gz 2024-04-25T10:33:55.620 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49059.log 2024-04-25T10:33:55.621 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31772.log 2024-04-25T10:33:55.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69428.log 2024-04-25T10:33:55.623 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49059.log.gz 2024-04-25T10:33:55.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31772.log.gz 2024-04-25T10:33:55.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69428.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69428.log.gz 2024-04-25T10:33:55.625 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86257.log 2024-04-25T10:33:55.626 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77111.log 2024-04-25T10:33:55.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34966.log 2024-04-25T10:33:55.628 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.86257.log.gz 2024-04-25T10:33:55.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77111.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77111.log.gz 2024-04-25T10:33:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34966.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34966.log.gz 2024-04-25T10:33:55.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85945.log 2024-04-25T10:33:55.631 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77085.log 2024-04-25T10:33:55.633 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32599.log 2024-04-25T10:33:55.634 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85945.log.gz 2024-04-25T10:33:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77085.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77085.log.gz 2024-04-25T10:33:55.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32599.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32599.log.gz 2024-04-25T10:33:55.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32386.log 2024-04-25T10:33:55.637 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70675.log 2024-04-25T10:33:55.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69695.log 2024-04-25T10:33:55.638 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32386.log.gz 2024-04-25T10:33:55.639 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70675.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70675.log.gz 2024-04-25T10:33:55.639 INFO:teuthology.orchestra.run.smithi117.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:33:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69695.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69695.log.gz 2024-04-25T10:33:55.641 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35651.log 2024-04-25T10:33:55.642 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35428.log 2024-04-25T10:33:55.643 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52634.log 2024-04-25T10:33:55.643 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35651.log.gz 2024-04-25T10:33:55.645 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35428.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35428.log.gz 2024-04-25T10:33:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52634.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52634.log.gz 2024-04-25T10:33:55.646 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67887.log 2024-04-25T10:33:55.647 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52664.log 2024-04-25T10:33:55.649 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85971.log 2024-04-25T10:33:55.649 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67887.log.gz 2024-04-25T10:33:55.650 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52664.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52664.log.gz 2024-04-25T10:33:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85971.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85971.log.gz 2024-04-25T10:33:55.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87927.log 2024-04-25T10:33:55.653 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51388.log 2024-04-25T10:33:55.654 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2024-04-25T10:33:55.654 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87927.log.gz 2024-04-25T10:33:55.655 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51388.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51388.log.gz 2024-04-25T10:33:55.657 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51754.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz 2024-04-25T10:33:55.657 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86715.log 2024-04-25T10:33:55.658 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88702.log 2024-04-25T10:33:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86715.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86715.log.gz 2024-04-25T10:33:55.660 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88702.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88702.log.gz 2024-04-25T10:33:55.661 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75759.log 2024-04-25T10:33:55.662 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51091.log 2024-04-25T10:33:55.663 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33222.log 2024-04-25T10:33:55.663 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75759.log.gz 2024-04-25T10:33:55.665 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51091.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51091.log.gz 2024-04-25T10:33:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33222.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33222.log.gz 2024-04-25T10:33:55.666 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73291.log 2024-04-25T10:33:55.667 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69636.log 2024-04-25T10:33:55.669 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72583.log 2024-04-25T10:33:55.669 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73291.log.gz 2024-04-25T10:33:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69636.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69636.log.gz 2024-04-25T10:33:55.670 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33826.log 2024-04-25T10:33:55.671 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86362.log 2024-04-25T10:33:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33826.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.72583.log.gz 2024-04-25T10:33:55.673 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72225.log 2024-04-25T10:33:55.674 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33826.log.gz 2024-04-25T10:33:55.674 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86362.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86362.log.gz 2024-04-25T10:33:55.676 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72225.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.72225.log.gz 2024-04-25T10:33:55.676 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33438.log 2024-04-25T10:33:55.677 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48223.log 2024-04-25T10:33:55.679 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84162.log 2024-04-25T10:33:55.679 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33438.log.gz 2024-04-25T10:33:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48223.log: /var/log/ceph/ceph-client.admin.84162.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48223.log.gz 2024-04-25T10:33:55.681 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84162.log.gz 2024-04-25T10:33:55.682 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88651.log 2024-04-25T10:33:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89440.log 2024-04-25T10:33:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82776.log 2024-04-25T10:33:55.685 INFO:teuthology.orchestra.run.smithi045.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.88651.log.gz 2024-04-25T10:33:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89440.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89440.log.gz 2024-04-25T10:33:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27637.log 2024-04-25T10:33:55.687 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82776.log.gz 2024-04-25T10:33:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69885.log 2024-04-25T10:33:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.27637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-04-25T10:33:55.689 INFO:teuthology.orchestra.run.smithi045.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27637.log.gz 2024-04-25T10:33:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69885.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69885.log.gz 2024-04-25T10:33:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35200.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-04-25T10:33:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72900.log 2024-04-25T10:33:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53901.log 2024-04-25T10:33:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84337.log 2024-04-25T10:33:55.695 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72900.log.gz 2024-04-25T10:33:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53901.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53901.log.gz 2024-04-25T10:33:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84337.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84337.log.gz 2024-04-25T10:33:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72659.log 2024-04-25T10:33:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75475.log 2024-04-25T10:33:55.700 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86637.log 2024-04-25T10:33:55.700 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72659.log.gz 2024-04-25T10:33:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75475.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.75475.log.gz 2024-04-25T10:33:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86637.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86637.log.gz 2024-04-25T10:33:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87848.log 2024-04-25T10:33:55.703 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69454.log 2024-04-25T10:33:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89175.log 2024-04-25T10:33:55.705 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87848.log.gz 2024-04-25T10:33:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69454.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69454.log.gz 2024-04-25T10:33:55.707 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90243.log 2024-04-25T10:33:55.707 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83145.log 2024-04-25T10:33:55.709 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90243.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89175.log.gz 2024-04-25T10:33:55.709 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88202.log 2024-04-25T10:33:55.709 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90243.log.gz 2024-04-25T10:33:55.710 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83145.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.83145.log.gz 2024-04-25T10:33:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88202.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88202.log.gz 2024-04-25T10:33:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86769.log 2024-04-25T10:33:55.712 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66805.log 2024-04-25T10:33:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83418.log 2024-04-25T10:33:55.715 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86769.log.gz 2024-04-25T10:33:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66805.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66805.log.gz 2024-04-25T10:33:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83418.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83418.log.gz 2024-04-25T10:33:55.717 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67835.log 2024-04-25T10:33:55.718 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-04-25T10:33:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48621.log 2024-04-25T10:33:55.720 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67835.log.gz 2024-04-25T10:33:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34649.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-04-25T10:33:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48621.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48621.log.gz 2024-04-25T10:33:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34836.log 2024-04-25T10:33:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88973.log 2024-04-25T10:33:55.726 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50891.log 2024-04-25T10:33:55.726 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34836.log.gz 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88973.log: /var/log/ceph/ceph-client.admin.50891.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.88973.log.gz 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50891.log.gz 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72822.log 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85334.log 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9%/var/log/ceph/ceph-client.admin.49584.log: -- replaced with /var/log/ceph/ceph-client.admin.72822.log.gz 2024-04-25T10:33:55.732 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-04-25T10:33:55.733 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85334.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.85334.log.gz 2024-04-25T10:33:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80313.log 2024-04-25T10:33:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34464.log 2024-04-25T10:33:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54335.log 2024-04-25T10:33:55.737 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80313.log.gz 2024-04-25T10:33:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34464.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34464.log.gz 2024-04-25T10:33:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54335.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.54335.log.gz 2024-04-25T10:33:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76702.log 2024-04-25T10:33:55.741 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85149.log 2024-04-25T10:33:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52723.log 2024-04-25T10:33:55.742 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76702.log.gz 2024-04-25T10:33:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85149.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.85149.log.gz 2024-04-25T10:33:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52723.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52723.log.gz 2024-04-25T10:33:55.745 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32234.log 2024-04-25T10:33:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83011.log 2024-04-25T10:33:55.747 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82802.log 2024-04-25T10:33:55.748 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32234.log: 87.9%/var/log/ceph/ceph-client.admin.83011.log: -- replaced with /var/log/ceph/ceph-client.admin.32234.log.gz 2024-04-25T10:33:55.748 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83011.log.gz 2024-04-25T10:33:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82802.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82802.log.gz 2024-04-25T10:33:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52068.log 2024-04-25T10:33:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53537.log 2024-04-25T10:33:55.753 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79969.log 2024-04-25T10:33:55.753 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52068.log.gz 2024-04-25T10:33:55.753 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53537.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53537.log.gz 2024-04-25T10:33:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79969.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79969.log.gz 2024-04-25T10:33:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67781.log 2024-04-25T10:33:55.756 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33170.log 2024-04-25T10:33:55.758 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76799.log 2024-04-25T10:33:55.758 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67781.log.gz 2024-04-25T10:33:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33170.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33170.log.gz 2024-04-25T10:33:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76799.log.gz 2024-04-25T10:33:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69532.log 2024-04-25T10:33:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88378.log 2024-04-25T10:33:55.763 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34992.log 2024-04-25T10:33:55.763 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69532.log.gz 2024-04-25T10:33:55.764 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.88378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.88378.log.gz 2024-04-25T10:33:55.765 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34992.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34992.log.gz 2024-04-25T10:33:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32260.log 2024-04-25T10:33:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79365.log 2024-04-25T10:33:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77543.log 2024-04-25T10:33:55.769 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32260.log.gz 2024-04-25T10:33:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79365.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79365.log.gz 2024-04-25T10:33:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77543.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.77543.log.gz 2024-04-25T10:33:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52358.log 2024-04-25T10:33:55.772 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83737.log 2024-04-25T10:33:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35757.log 2024-04-25T10:33:55.774 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52358.log.gz 2024-04-25T10:33:55.775 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83737.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83737.log.gz 2024-04-25T10:33:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35757.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35757.log.gz 2024-04-25T10:33:55.776 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77361.log 2024-04-25T10:33:55.778 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53615.log 2024-04-25T10:33:55.779 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77361.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:33:55.779 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77361.log.gz 2024-04-25T10:33:55.780 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53615.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53615.log.gz 2024-04-25T10:33:55.782 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90217.log 2024-04-25T10:33:55.783 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53667.log 2024-04-25T10:33:55.785 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88766.log 2024-04-25T10:33:55.785 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90217.log.gz 2024-04-25T10:33:55.786 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87901.log 2024-04-25T10:33:55.786 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53667.log.gz 2024-04-25T10:33:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35677.log 2024-04-25T10:33:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88766.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:33:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:33:55.788 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88766.log.gz 2024-04-25T10:33:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87901.log: /var/log/ceph/ceph-client.admin.35677.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87901.log.gz 2024-04-25T10:33:55.790 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35677.log.gz 2024-04-25T10:33:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73773.log 2024-04-25T10:33:55.792 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81951.log 2024-04-25T10:33:55.793 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73773.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73773.log.gz 2024-04-25T10:33:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.81951.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.81951.log.gz 2024-04-25T10:33:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76877.log 2024-04-25T10:33:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73825.log 2024-04-25T10:33:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76877.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76877.log.gz 2024-04-25T10:33:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73825.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73825.log.gz 2024-04-25T10:33:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66831.log 2024-04-25T10:33:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74139.log 2024-04-25T10:33:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66831.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66831.log.gz 2024-04-25T10:33:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74139.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.74139.log.gz 2024-04-25T10:33:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51964.log 2024-04-25T10:33:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82308.log 2024-04-25T10:33:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51964.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51964.log.gz 2024-04-25T10:33:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82308.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82308.log.gz 2024-04-25T10:33:55.808 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84136.log 2024-04-25T10:33:55.809 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84032.log 2024-04-25T10:33:55.811 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84136.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84136.log.gz 2024-04-25T10:33:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84032.log.gz 2024-04-25T10:33:55.812 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:33:55.814 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50660.log 2024-04-25T10:33:55.817 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.50660.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50660.log.gz 2024-04-25T10:33:55.818 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49558.log 2024-04-25T10:33:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49558.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49558.log.gz 2024-04-25T10:33:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88450.log 2024-04-25T10:33:55.827 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88450.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88450.log.gz 2024-04-25T10:33:55.827 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35586.log 2024-04-25T10:33:55.828 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35586.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35586.log.gz 2024-04-25T10:33:55.828 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88282.log 2024-04-25T10:33:55.830 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88282.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88282.log.gz 2024-04-25T10:33:55.830 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83232.log 2024-04-25T10:33:55.834 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83232.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83232.log.gz 2024-04-25T10:33:55.834 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72711.log 2024-04-25T10:33:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72711.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72711.log.gz 2024-04-25T10:33:55.837 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54205.log 2024-04-25T10:33:55.838 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32390.log 2024-04-25T10:33:55.840 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54205.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54205.log.gz 2024-04-25T10:33:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32390.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32390.log.gz 2024-04-25T10:33:55.843 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75603.log 2024-04-25T10:33:55.846 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75603.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75603.log.gz 2024-04-25T10:33:55.846 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87692.log 2024-04-25T10:33:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87692.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87692.log.gz 2024-04-25T10:33:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84493.log 2024-04-25T10:33:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51806.log 2024-04-25T10:33:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84493.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84493.log.gz 2024-04-25T10:33:55.855 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51806.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51806.log.gz 2024-04-25T10:33:55.855 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76442.log 2024-04-25T10:33:55.858 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76442.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76442.log.gz 2024-04-25T10:33:55.859 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70833.log 2024-04-25T10:33:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70833.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70833.log.gz 2024-04-25T10:33:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53193.log 2024-04-25T10:33:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32597.log 2024-04-25T10:33:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53193.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53193.log.gz 2024-04-25T10:33:55.867 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32597.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32597.log.gz 2024-04-25T10:33:55.868 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79612.log 2024-04-25T10:33:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79612.log.gz 2024-04-25T10:33:55.870 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89693.log 2024-04-25T10:33:55.873 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89693.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89693.log.gz 2024-04-25T10:33:55.874 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86821.log 2024-04-25T10:33:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85575.log 2024-04-25T10:33:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86821.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.86821.log.gz 2024-04-25T10:33:55.878 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53329.log 2024-04-25T10:33:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85575.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85575.log.gz 2024-04-25T10:33:55.881 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87309.log 2024-04-25T10:33:55.882 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53329.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53329.log.gz 2024-04-25T10:33:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87309.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87309.log.gz 2024-04-25T10:33:55.886 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84363.log 2024-04-25T10:33:55.888 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82392.log 2024-04-25T10:33:55.889 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84363.log.gz 2024-04-25T10:33:55.891 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53433.log 2024-04-25T10:33:55.892 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82392.log.gz 2024-04-25T10:33:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76332.log 2024-04-25T10:33:55.896 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53433.log.gz 2024-04-25T10:33:55.899 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.76332.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76332.log.gz 2024-04-25T10:33:55.899 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.87983.log 2024-04-25T10:33:55.902 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87983.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87983.log.gz 2024-04-25T10:33:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74189.log 2024-04-25T10:33:55.905 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74189.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74189.log.gz 2024-04-25T10:33:55.906 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79794.log 2024-04-25T10:33:55.908 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79794.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79794.log.gz 2024-04-25T10:33:55.909 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-04-25T10:33:55.912 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66592.log 2024-04-25T10:33:55.912 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48673.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-04-25T10:33:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66592.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66592.log.gz 2024-04-25T10:33:55.915 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34164.log 2024-04-25T10:33:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34164.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34164.log.gz 2024-04-25T10:33:55.918 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82256.log 2024-04-25T10:33:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82256.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82256.log.gz 2024-04-25T10:33:55.921 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76089.log 2024-04-25T10:33:55.923 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-04-25T10:33:55.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76089.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76089.log.gz 2024-04-25T10:33:55.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51990.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-04-25T10:33:55.928 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76676.log 2024-04-25T10:33:55.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76676.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76676.log.gz 2024-04-25T10:33:55.935 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-04-25T10:33:55.935 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74113.log 2024-04-25T10:33:55.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34888.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-04-25T10:33:55.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76903.log 2024-04-25T10:33:55.937 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74113.log.gz 2024-04-25T10:33:55.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50504.log 2024-04-25T10:33:55.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50504.log.gz 2024-04-25T10:33:55.944 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76903.log.gz 2024-04-25T10:33:55.944 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74061.log 2024-04-25T10:33:55.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74061.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74061.log.gz 2024-04-25T10:33:55.947 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72351.log 2024-04-25T10:33:55.950 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86559.log 2024-04-25T10:33:55.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72351.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72351.log.gz 2024-04-25T10:33:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86559.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86559.log.gz 2024-04-25T10:33:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73480.log 2024-04-25T10:33:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73480.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73480.log.gz 2024-04-25T10:33:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90269.log 2024-04-25T10:33:55.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90269.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90269.log.gz 2024-04-25T10:33:55.962 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79865.log 2024-04-25T10:33:55.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79865.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79865.log.gz 2024-04-25T10:33:55.963 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84519.log 2024-04-25T10:33:55.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84519.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84519.log.gz 2024-04-25T10:33:55.965 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73168.log 2024-04-25T10:33:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88502.log 2024-04-25T10:33:55.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73168.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73168.log.gz 2024-04-25T10:33:55.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88502.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88502.log.gz 2024-04-25T10:33:55.972 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69966.log 2024-04-25T10:33:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52608.log 2024-04-25T10:33:55.975 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69966.log.gz 2024-04-25T10:33:55.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84941.log 2024-04-25T10:33:55.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52608.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52608.log.gz 2024-04-25T10:33:55.980 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90191.log 2024-04-25T10:33:55.981 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84941.log.gz 2024-04-25T10:33:55.983 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66889.log 2024-04-25T10:33:55.987 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66889.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66889.log.gz 2024-04-25T10:33:55.987 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34914.log 2024-04-25T10:33:55.989 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90191.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90191.log.gz 2024-04-25T10:33:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34914.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34914.log.gz 2024-04-25T10:33:55.992 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73879.log 2024-04-25T10:33:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73879.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73879.log.gz 2024-04-25T10:33:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84006.log 2024-04-25T10:33:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84006.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84006.log.gz 2024-04-25T10:33:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86585.log 2024-04-25T10:33:56.002 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86585.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86585.log.gz 2024-04-25T10:33:56.002 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76384.log 2024-04-25T10:33:56.003 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85073.log 2024-04-25T10:33:56.005 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76384.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76384.log.gz 2024-04-25T10:33:56.008 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85073.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85073.log.gz 2024-04-25T10:33:56.008 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85601.log 2024-04-25T10:33:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85601.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85601.log.gz 2024-04-25T10:33:56.011 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75551.log 2024-04-25T10:33:56.012 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33118.log 2024-04-25T10:33:56.014 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75551.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75551.log.gz 2024-04-25T10:33:56.016 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51117.log 2024-04-25T10:33:56.017 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33118.log.gz 2024-04-25T10:33:56.020 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32808.log 2024-04-25T10:33:56.021 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51117.log.gz 2024-04-25T10:33:56.025 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32808.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32808.log.gz 2024-04-25T10:33:56.025 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73421.log 2024-04-25T10:33:56.028 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73421.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73421.log.gz 2024-04-25T10:33:56.028 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52724.log 2024-04-25T10:33:56.034 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52724.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52724.log.gz 2024-04-25T10:33:56.034 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48647.log 2024-04-25T10:33:56.034 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48647.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48647.log.gz 2024-04-25T10:33:56.034 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83896.log 2024-04-25T10:33:56.037 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83896.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83896.log.gz 2024-04-25T10:33:56.037 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log 2024-04-25T10:33:56.038 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82204.log 2024-04-25T10:33:56.040 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35480.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-04-25T10:33:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82204.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82204.log.gz 2024-04-25T10:33:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49348.log 2024-04-25T10:33:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49348.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49348.log.gz 2024-04-25T10:33:56.046 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52306.log 2024-04-25T10:33:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52306.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52306.log.gz 2024-04-25T10:33:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70612.log 2024-04-25T10:33:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33040.log 2024-04-25T10:33:56.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70612.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70612.log.gz 2024-04-25T10:33:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84967.log 2024-04-25T10:33:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33040.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33040.log.gz 2024-04-25T10:33:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84967.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84967.log.gz 2024-04-25T10:33:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2024-04-25T10:33:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33066.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2024-04-25T10:33:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34060.log 2024-04-25T10:33:56.064 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34060.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34060.log.gz 2024-04-25T10:33:56.065 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80341.log 2024-04-25T10:33:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80341.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80341.log.gz 2024-04-25T10:33:56.068 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76141.log 2024-04-25T10:33:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76141.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76141.log.gz 2024-04-25T10:33:56.071 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35070.log 2024-04-25T10:33:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32807.log 2024-04-25T10:33:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35070.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35070.log.gz 2024-04-25T10:33:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31944.log 2024-04-25T10:33:56.077 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32807.log.gz 2024-04-25T10:33:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34623.log 2024-04-25T10:33:56.080 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31944.log.gz 2024-04-25T10:33:56.083 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75655.log 2024-04-25T10:33:56.084 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34623.log.gz 2024-04-25T10:33:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73343.log 2024-04-25T10:33:56.087 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75655.log.gz 2024-04-25T10:33:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67071.log 2024-04-25T10:33:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73343.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73343.log.gz 2024-04-25T10:33:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67071.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67071.log.gz 2024-04-25T10:33:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73710.log 2024-04-25T10:33:56.097 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73710.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73710.log.gz 2024-04-25T10:33:56.097 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79534.log 2024-04-25T10:33:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79534.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79534.log.gz 2024-04-25T10:33:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32587.log 2024-04-25T10:33:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32587.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32587.log.gz 2024-04-25T10:33:56.103 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-04-25T10:33:56.106 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48830.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-04-25T10:33:56.106 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49532.log 2024-04-25T10:33:56.109 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86917.log 2024-04-25T10:33:56.109 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49532.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49532.log.gz 2024-04-25T10:33:56.112 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.86917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.86917.log.gz 2024-04-25T10:33:56.112 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35809.log 2024-04-25T10:33:56.114 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35809.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35809.log.gz 2024-04-25T10:33:56.115 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83557.log 2024-04-25T10:33:56.118 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83557.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.83557.log.gz 2024-04-25T10:33:56.118 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73064.log 2024-04-25T10:33:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73064.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73064.log.gz 2024-04-25T10:33:56.121 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54385.log 2024-04-25T10:33:56.124 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-04-25T10:33:56.124 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54385.log.gz 2024-04-25T10:33:56.127 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52992.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-04-25T10:33:56.127 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51624.log 2024-04-25T10:33:56.130 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51624.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51624.log.gz 2024-04-25T10:33:56.130 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31996.log 2024-04-25T10:33:56.133 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31996.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31996.log.gz 2024-04-25T10:33:56.133 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49480.log 2024-04-25T10:33:56.136 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49480.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49480.log.gz 2024-04-25T10:33:56.136 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49819.log 2024-04-25T10:33:56.143 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49819.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.49819.log.gz 2024-04-25T10:33:56.143 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87148.log 2024-04-25T10:33:56.143 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87148.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87148.log.gz 2024-04-25T10:33:56.143 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80258.log 2024-04-25T10:33:56.146 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80258.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80258.log.gz 2024-04-25T10:33:56.146 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53056.log 2024-04-25T10:33:56.150 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53056.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53056.log.gz 2024-04-25T10:33:56.150 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33092.log 2024-04-25T10:33:56.153 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33092.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33092.log.gz 2024-04-25T10:33:56.153 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51440.log 2024-04-25T10:33:56.156 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72737.log 2024-04-25T10:33:56.156 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51440.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51440.log.gz 2024-04-25T10:33:56.159 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72737.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72737.log.gz 2024-04-25T10:33:56.159 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87387.log 2024-04-25T10:33:56.162 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87387.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87387.log.gz 2024-04-25T10:33:56.162 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77137.log 2024-04-25T10:33:56.165 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77137.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77137.log.gz 2024-04-25T10:33:56.165 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77491.log 2024-04-25T10:33:56.170 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77491.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77491.log.gz 2024-04-25T10:33:56.170 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22663.log 2024-04-25T10:33:56.170 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.tmp-client.admin.22663.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22663.log.gz 2024-04-25T10:33:56.171 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85659.log 2024-04-25T10:33:56.173 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85659.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85659.log.gz 2024-04-25T10:33:56.174 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86101.log 2024-04-25T10:33:56.176 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84415.log 2024-04-25T10:33:56.177 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86101.log.gz 2024-04-25T10:33:56.179 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54448.log 2024-04-25T10:33:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84415.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84415.log.gz 2024-04-25T10:33:56.183 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.54448.log: --verbose -- /var/log/ceph/ceph-client.admin.88820.log 2024-04-25T10:33:56.184 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54448.log.gz 2024-04-25T10:33:56.187 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76306.log 2024-04-25T10:33:56.188 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.88820.log.gz 2024-04-25T10:33:56.190 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88528.log 2024-04-25T10:33:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88528.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88528.log.gz 2024-04-25T10:33:56.194 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76306.log.gz 2024-04-25T10:33:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84110.log 2024-04-25T10:33:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84110.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84110.log.gz 2024-04-25T10:33:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86049.log 2024-04-25T10:33:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86049.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86049.log.gz 2024-04-25T10:33:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72403.log 2024-04-25T10:33:56.207 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87588.log 2024-04-25T10:33:56.212 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72403.log.gz 2024-04-25T10:33:56.212 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87588.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87588.log.gz 2024-04-25T10:33:56.212 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32052.log 2024-04-25T10:33:56.215 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32052.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32052.log.gz 2024-04-25T10:33:56.215 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35887.log 2024-04-25T10:33:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78930.log 2024-04-25T10:33:56.218 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35887.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35887.log.gz 2024-04-25T10:33:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.78930.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78930.log.gz 2024-04-25T10:33:56.221 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73629.log 2024-04-25T10:33:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73629.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73629.log.gz 2024-04-25T10:33:56.224 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75785.log 2024-04-25T10:33:56.227 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48173.log 2024-04-25T10:33:56.228 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2024-04-25T10:33:56.229 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1%/var/log/ceph/ceph-client.admin.48173.log: -- replaced with /var/log/ceph/ceph-client.admin.75785.log.gz 2024-04-25T10:33:56.230 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48173.log.gz 2024-04-25T10:33:56.233 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49215.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2024-04-25T10:33:56.234 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88424.log 2024-04-25T10:33:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88424.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88424.log.gz 2024-04-25T10:33:56.237 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33352.log 2024-04-25T10:33:56.243 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33352.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33352.log.gz 2024-04-25T10:33:56.243 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33956.log 2024-04-25T10:33:56.243 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33956.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33956.log.gz 2024-04-25T10:33:56.243 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52687.log 2024-04-25T10:33:56.250 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52687.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52687.log.gz 2024-04-25T10:33:56.250 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66694.log 2024-04-25T10:33:56.250 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66694.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66694.log.gz 2024-04-25T10:33:56.250 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83540.log 2024-04-25T10:33:56.253 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.83540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.83540.log.gz 2024-04-25T10:33:56.253 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75911.log 2024-04-25T10:33:56.256 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87361.log 2024-04-25T10:33:56.256 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75911.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75911.log.gz 2024-04-25T10:33:56.259 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87361.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87361.log.gz 2024-04-25T10:33:56.259 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50228.log 2024-04-25T10:33:56.263 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50228.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50228.log.gz 2024-04-25T10:33:56.263 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49758.log 2024-04-25T10:33:56.265 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.49758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49758.log.gz 2024-04-25T10:33:56.265 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76988.log 2024-04-25T10:33:56.270 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70122.log 2024-04-25T10:33:56.270 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76988.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76988.log.gz 2024-04-25T10:33:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70122.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70122.log.gz 2024-04-25T10:33:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83870.log 2024-04-25T10:33:56.274 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.83870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85685.log 2024-04-25T10:33:56.275 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83870.log.gz 2024-04-25T10:33:56.276 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51414.log 2024-04-25T10:33:56.278 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85685.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85685.log.gz 2024-04-25T10:33:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.51414.log: --verbose -- /var/log/ceph/ceph-client.admin.48725.log 2024-04-25T10:33:56.281 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51414.log.gz 2024-04-25T10:33:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73116.log 2024-04-25T10:33:56.285 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48725.log.gz 2024-04-25T10:33:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.73116.log: --verbose -- /var/log/ceph/ceph-client.admin.35703.log 2024-04-25T10:33:56.288 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73116.log.gz 2024-04-25T10:33:56.290 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52530.log 2024-04-25T10:33:56.292 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35703.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35703.log.gz 2024-04-25T10:33:56.295 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77387.log 2024-04-25T10:33:56.295 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52530.log.gz 2024-04-25T10:33:56.297 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-04-25T10:33:56.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50328.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-04-25T10:33:56.302 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77387.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77387.log.gz 2024-04-25T10:33:56.303 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34433.log 2024-04-25T10:33:56.305 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34433.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34433.log.gz 2024-04-25T10:33:56.306 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80284.log 2024-04-25T10:33:56.308 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80284.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80284.log.gz 2024-04-25T10:33:56.309 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75733.log 2024-04-25T10:33:56.311 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75733.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75733.log.gz 2024-04-25T10:33:56.312 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77307.log 2024-04-25T10:33:56.314 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72848.log 2024-04-25T10:33:56.315 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77307.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77307.log.gz 2024-04-25T10:33:56.316 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53219.log 2024-04-25T10:33:56.318 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72848.log: 87.7%/var/log/ceph/ceph-client.admin.53219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89615.log 2024-04-25T10:33:56.319 INFO:teuthology.orchestra.run.smithi045.stderr:gzip 89.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.66490.log 2024-04-25T10:33:56.321 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89615.log: -- replaced with /var/log/ceph/ceph-client.admin.53219.log.gz 2024-04-25T10:33:56.321 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72848.log.gz 2024-04-25T10:33:56.321 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89615.log.gz 2024-04-25T10:33:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66490.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66490.log.gz 2024-04-25T10:33:56.324 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85737.log 2024-04-25T10:33:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85737.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85737.log.gz 2024-04-25T10:33:56.327 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87122.log 2024-04-25T10:33:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87122.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87122.log.gz 2024-04-25T10:33:56.330 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80123.log 2024-04-25T10:33:56.334 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85259.log 2024-04-25T10:33:56.334 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80123.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80123.log.gz 2024-04-25T10:33:56.335 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.85259.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.82470.log 2024-04-25T10:33:56.335 INFO:teuthology.orchestra.run.smithi045.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.85259.log.gz 2024-04-25T10:33:56.345 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82470.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82470.log.gz 2024-04-25T10:33:56.345 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54036.log 2024-04-25T10:33:56.345 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90163.log 2024-04-25T10:33:56.345 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54036.log.gz 2024-04-25T10:33:56.345 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76468.log 2024-04-25T10:33:56.346 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.90163.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90163.log.gz 2024-04-25T10:33:56.348 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.76468.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85711.log 2024-04-25T10:33:56.349 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76468.log.gz 2024-04-25T10:33:56.351 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.85711.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85711.log.gz 2024-04-25T10:33:56.352 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.76247.log 2024-04-25T10:33:56.355 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34086.log 2024-04-25T10:33:56.355 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76247.log.gz 2024-04-25T10:33:56.356 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50560.log 2024-04-25T10:33:56.362 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50560.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50560.log.gz 2024-04-25T10:33:56.362 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34086.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34086.log.gz 2024-04-25T10:33:56.362 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88588.log 2024-04-25T10:33:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88588.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88588.log.gz 2024-04-25T10:33:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81873.log 2024-04-25T10:33:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.81873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81873.log.gz 2024-04-25T10:33:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33144.log 2024-04-25T10:33:56.370 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80456.log 2024-04-25T10:33:56.372 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33144.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33144.log.gz 2024-04-25T10:33:56.372 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35614.log 2024-04-25T10:33:56.373 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80456.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80456.log.gz 2024-04-25T10:33:56.376 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35614.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35614.log.gz 2024-04-25T10:33:56.376 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87413.log 2024-04-25T10:33:56.379 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87413.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87413.log.gz 2024-04-25T10:33:56.379 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82724.log 2024-04-25T10:33:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82724.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82724.log.gz 2024-04-25T10:33:56.382 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89023.log 2024-04-25T10:33:56.385 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89023.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.89023.log.gz 2024-04-25T10:33:56.385 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48981.log 2024-04-25T10:33:56.389 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48981.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48981.log.gz 2024-04-25T10:33:56.389 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76851.log 2024-04-25T10:33:56.391 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.76851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69859.log 2024-04-25T10:33:56.391 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76851.log.gz 2024-04-25T10:33:56.392 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36113.log 2024-04-25T10:33:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69859.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69859.log.gz 2024-04-25T10:33:56.396 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79424.log 2024-04-25T10:33:56.396 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.36113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.36113.log.gz 2024-04-25T10:33:56.398 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49636.log 2024-04-25T10:33:56.399 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79424.log.gz 2024-04-25T10:33:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67991.log 2024-04-25T10:33:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49636.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49636.log.gz 2024-04-25T10:33:56.405 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67991.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67991.log.gz 2024-04-25T10:33:56.405 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2024-04-25T10:33:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50278.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2024-04-25T10:33:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79032.log 2024-04-25T10:33:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79032.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79032.log.gz 2024-04-25T10:33:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67123.log 2024-04-25T10:33:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67123.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67123.log.gz 2024-04-25T10:33:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33720.log 2024-04-25T10:33:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84553.log 2024-04-25T10:33:56.418 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33720.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33720.log.gz 2024-04-25T10:33:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84553.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84553.log.gz 2024-04-25T10:33:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-04-25T10:33:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48955.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-04-25T10:33:56.424 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85099.log 2024-04-25T10:33:56.427 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.85099.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.85099.log.gz 2024-04-25T10:33:56.428 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80376.log 2024-04-25T10:33:56.430 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48901.log 2024-04-25T10:33:56.430 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80376.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80376.log.gz 2024-04-25T10:33:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48901.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48901.log.gz 2024-04-25T10:33:56.433 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80049.log 2024-04-25T10:33:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80049.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80049.log.gz 2024-04-25T10:33:56.436 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33552.log 2024-04-25T10:33:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.33552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.33552.log.gz 2024-04-25T10:33:56.439 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69662.log 2024-04-25T10:33:56.440 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77190.log 2024-04-25T10:33:56.442 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69662.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69662.log.gz 2024-04-25T10:33:56.444 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.77190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75425.log 2024-04-25T10:33:56.445 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77190.log.gz 2024-04-25T10:33:56.447 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.75425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-04-25T10:33:56.449 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:33:56.449 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75425.log.gz 2024-04-25T10:33:56.451 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80021.log 2024-04-25T10:33:56.453 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49267.log: /var/log/ceph/ceph-client.admin.80021.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-04-25T10:33:56.453 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31970.log 2024-04-25T10:33:56.453 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80021.log.gz 2024-04-25T10:33:56.456 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31970.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31970.log.gz 2024-04-25T10:33:56.456 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70167.log 2024-04-25T10:33:56.457 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82750.log 2024-04-25T10:33:56.459 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70167.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70167.log.gz 2024-04-25T10:33:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82750.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82750.log.gz 2024-04-25T10:33:56.460 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79586.log 2024-04-25T10:33:56.462 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79058.log 2024-04-25T10:33:56.463 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79586.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79586.log.gz 2024-04-25T10:33:56.464 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79058.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79058.log.gz 2024-04-25T10:33:56.465 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84755.log 2024-04-25T10:33:56.466 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86192.log 2024-04-25T10:33:56.468 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84755.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84755.log.gz 2024-04-25T10:33:56.468 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86192.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86192.log.gz 2024-04-25T10:33:56.469 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84729.log 2024-04-25T10:33:56.470 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84058.log 2024-04-25T10:33:56.472 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84729.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84729.log.gz 2024-04-25T10:33:56.472 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84058.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84058.log.gz 2024-04-25T10:33:56.473 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54099.log 2024-04-25T10:33:56.474 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80534.log 2024-04-25T10:33:56.476 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54099.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54099.log.gz 2024-04-25T10:33:56.476 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.80534.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80534.log.gz 2024-04-25T10:33:56.477 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2024-04-25T10:33:56.478 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51728.log 2024-04-25T10:33:56.480 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51544.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz 2024-04-25T10:33:56.480 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51728.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2024-04-25T10:33:56.481 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79284.log 2024-04-25T10:33:56.482 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88794.log 2024-04-25T10:33:56.484 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79284.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79284.log.gz 2024-04-25T10:33:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88794.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88794.log.gz 2024-04-25T10:33:56.485 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79768.log 2024-04-25T10:33:56.486 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49971.log 2024-04-25T10:33:56.488 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.79768.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79768.log.gz 2024-04-25T10:33:56.489 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49971.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49971.log.gz 2024-04-25T10:33:56.490 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52721.log 2024-04-25T10:33:56.491 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88018.log 2024-04-25T10:33:56.493 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52721.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52721.log.gz 2024-04-25T10:33:56.493 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88018.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88018.log.gz 2024-04-25T10:33:56.494 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31683.log 2024-04-25T10:33:56.495 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89851.log 2024-04-25T10:33:56.497 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.31683.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31683.log.gz 2024-04-25T10:33:56.497 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89851.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89851.log.gz 2024-04-25T10:33:56.498 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67045.log 2024-04-25T10:33:56.499 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32312.log 2024-04-25T10:33:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67045.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67045.log.gz 2024-04-25T10:33:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32312.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32312.log.gz 2024-04-25T10:33:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70018.log 2024-04-25T10:33:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84188.log 2024-04-25T10:33:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70018.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70018.log.gz 2024-04-25T10:33:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84188.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84188.log.gz 2024-04-25T10:33:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54125.log 2024-04-25T10:33:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88070.log 2024-04-25T10:33:56.509 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54125.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54125.log.gz 2024-04-25T10:33:56.510 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.88070.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88070.log.gz 2024-04-25T10:33:56.510 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35232.log 2024-04-25T10:33:56.511 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53563.log 2024-04-25T10:33:56.513 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35232.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35232.log.gz 2024-04-25T10:33:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53563.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53563.log.gz 2024-04-25T10:33:56.514 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69480.log 2024-04-25T10:33:56.516 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87874.log 2024-04-25T10:33:56.517 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69480.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69480.log.gz 2024-04-25T10:33:56.518 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87874.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87874.log.gz 2024-04-25T10:33:56.519 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73010.log 2024-04-25T10:33:56.520 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89336.log 2024-04-25T10:33:56.521 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73010.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73010.log.gz 2024-04-25T10:33:56.522 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53778.log 2024-04-25T10:33:56.523 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89336.log.gz 2024-04-25T10:33:56.524 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73512.log 2024-04-25T10:33:56.525 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53778.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53778.log.gz 2024-04-25T10:33:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73512.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73512.log.gz 2024-04-25T10:33:56.526 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74235.log 2024-04-25T10:33:56.528 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32805.log 2024-04-25T10:33:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.74235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74235.log.gz 2024-04-25T10:33:56.530 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32805.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32805.log.gz 2024-04-25T10:33:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54257.log 2024-04-25T10:33:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82418.log 2024-04-25T10:33:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54257.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54257.log.gz 2024-04-25T10:33:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.82418.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.82418.log.gz 2024-04-25T10:33:56.535 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89544.log 2024-04-25T10:33:56.536 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87335.log 2024-04-25T10:33:56.538 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.89544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89544.log.gz 2024-04-25T10:33:56.538 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.87335.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87335.log.gz 2024-04-25T10:33:56.539 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34216.log 2024-04-25T10:33:56.540 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84467.log 2024-04-25T10:33:56.542 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.34216.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34216.log.gz 2024-04-25T10:33:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84467.log.gz 2024-04-25T10:33:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52914.log 2024-04-25T10:33:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86309.log 2024-04-25T10:33:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52914.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52914.log.gz 2024-04-25T10:33:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.86309.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86309.log.gz 2024-04-25T10:33:56.548 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84889.log 2024-04-25T10:33:56.549 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32575.log 2024-04-25T10:33:56.550 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.84889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84889.log.gz 2024-04-25T10:33:56.551 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.32575.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32575.log.gz 2024-04-25T10:33:56.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T10:33:56.960 INFO:teuthology.orchestra.run.smithi117.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:33:56.989 INFO:teuthology.orchestra.run.smithi117.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:33:57.370 INFO:teuthology.orchestra.run.smithi117.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:33:57.604 INFO:teuthology.orchestra.run.smithi045.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:33:57.748 INFO:teuthology.orchestra.run.smithi117.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:33:57.791 INFO:teuthology.orchestra.run.smithi045.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:33:57.796 INFO:teuthology.orchestra.run.smithi117.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:33:57.799 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-04-25T10:33:57.799 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m2.883s 2024-04-25T10:33:57.799 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m12.609s 2024-04-25T10:33:57.799 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.949s 2024-04-25T10:33:57.882 INFO:teuthology.orchestra.run.smithi045.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:33:58.023 INFO:teuthology.orchestra.run.smithi045.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:34:20.114 INFO:teuthology.orchestra.run.smithi045.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:34:21.448 INFO:teuthology.orchestra.run.smithi045.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:34:21.450 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-25T10:34:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m26.557s 2024-04-25T10:34:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m58.890s 2024-04-25T10:34:21.450 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m5.677s 2024-04-25T10:34:21.451 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:34:21.451 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/remote/smithi045/log 2024-04-25T10:34:21.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:34:28.210 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/remote/smithi117/log 2024-04-25T10:34:28.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:34:29.530 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:34:29.540 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:34:29.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:34:29.544 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:34:29.627 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:34:29.636 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:34:29.636 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:34:29.639 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:34:29.652 INFO:teuthology.orchestra.run.smithi045.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:34:29.652 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================== 2024-04-25T10:34:29.652 INFO:teuthology.orchestra.run.smithi045.stdout:+hv01.front.sepi 67.205.162.81 3 u 39 64 377 0.087 -3.726 0.423 2024-04-25T10:34:29.652 INFO:teuthology.orchestra.run.smithi045.stdout:+hv02.front.sepi 63.231.80.2 3 u 44 64 377 0.067 -2.398 0.588 2024-04-25T10:34:29.652 INFO:teuthology.orchestra.run.smithi045.stdout:*hv03.front.sepi 74.6.168.72 3 u 45 64 377 0.084 -1.910 0.356 2024-04-25T10:34:29.652 INFO:teuthology.orchestra.run.smithi045.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:34:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:34:29.654 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-04-25T10:34:29.654 INFO:teuthology.orchestra.run.smithi117.stdout:+hv01.front.sepi 67.205.162.81 3 u 43 64 377 0.076 -5.817 0.537 2024-04-25T10:34:29.654 INFO:teuthology.orchestra.run.smithi117.stdout:+hv02.front.sepi 63.231.80.2 3 u 43 64 377 0.102 -2.836 1.100 2024-04-25T10:34:29.654 INFO:teuthology.orchestra.run.smithi117.stdout:*hv03.front.sepi 74.6.168.72 3 u 48 64 377 0.105 -3.891 0.497 2024-04-25T10:34:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:34:29.655 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:34:29.664 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:34:29.664 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:34:29.677 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:34:29.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:34:29.705 INFO:teuthology.task.internal:Duration was 1702.284728 seconds 2024-04-25T10:34:29.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:34:29.719 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:34:29.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:34:29.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:34:29.829 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:34:29.829 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T10:34:29.829 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:34:29.850 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-04-25T10:34:29.850 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:34:29.870 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:34:29.870 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:34:29.893 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:34:29.956 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:34:29.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:34:29.985 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:34:30.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:34:30.088 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:34:30.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:34:30.109 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:34:30.129 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:34:30.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:34:30.157 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:34:30.172 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-04-25T10:34:30.191 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-04-25T10:34:30.209 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:34:30.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:34:30.246 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:34:30.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:34:30.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:34:30.267 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:34:30.268 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/remote/smithi045 2024-04-25T10:34:30.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:34:30.314 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673055/remote/smithi117 2024-04-25T10:34:30.314 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:34:30.340 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:34:30.340 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:34:30.353 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:34:30.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:34:30.397 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:34:30.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:34:30.411 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:34:30.411 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:34:30.414 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:34:30.420 INFO:teuthology.orchestra.run.smithi045.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:34 /home/ubuntu/cephtest 2024-04-25T10:34:30.438 INFO:teuthology.orchestra.run.smithi117.stdout: 655428 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:34 /home/ubuntu/cephtest 2024-04-25T10:34:30.439 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:34:30.449 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:34:30.481 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} duration: 1702.2847275733948 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:34:30.481 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:34:30.564 INFO:teuthology.run:pass